Commit graph

512 commits

Author SHA1 Message Date
Jesse Bate
7a33eb30ee fix: Linting errors 2026-06-11 00:10:36 +09:30
Jesse Bate
f1365f6fba perf: extend WebSocket batching to item_moved events
Bursts of completions (e.g. small playlist entries) fire one item_moved
frame per item. Route item_moved through the same ItemBatcher mechanism
as item_updated: one frame per 500ms tick, payload always a list.

ItemBatcher gains an optional key callable since item_moved payloads are
{"to", "preset", "item"} dicts rather than ItemDTOs; moves coalesce by
item _id with last write wins, so an item that moves twice within a tick
delivers only its final destination. The notifications service consumes
ITEM_MOVED from the event bus directly and is unaffected.

BREAKING CHANGE: the item_moved WebSocket event payload (data) is now
always an array of {to, preset, item} objects instead of a single one.
2026-06-10 23:47:25 +09:30
Jesse Bate
0cb3a95a41 perf: batch item_updated WebSocket events into a single per-tick payload
The per-download 500ms throttle still scales linearly with concurrency:
N active downloads produce up to 2N item_updated frames per second, each
with its own serialization pass and per-client send.

Add an ItemBatcher at the WebSocket boundary that coalesces dirty items
by _id (last write wins) and emits one item_updated frame per 500ms
tick whose payload is always a list of items, even for a single update.
Leading-edge emits keep isolated updates (e.g. renames via the API)
instant, and a trailing flush at most 500ms later delivers the rest.
Pending items are flushed on shutdown before clients disconnect.

Emitter call sites, the EventBus, and the StatusTracker throttle are
unchanged; the batcher only changes the wire format. The frontend now
types item_updated as StoreItem[] and iterates the array.

BREAKING CHANGE: the item_updated WebSocket event payload (data) is now
always an array of items instead of a single item object.
2026-06-10 23:38:27 +09:30
Jesse Bate
687c82cb32 feat: add YTP_MAX_HISTORY to cap history and prevent unbounded DB growth
Long-running instances accumulate history indefinitely. As the history
table grows, SQLite COUNT(*), paginated SELECT, and index scans degrade.

YTP_MAX_HISTORY (default: 0 = unlimited) caps the number of completed
download records kept. Enforcement is a hook in DataStore.put(): on
each insertion, the oldest excess entries are immediately evicted from
the in-memory OrderedDict and their deletes are enqueued to SQLite.
DataStore.load() also trims on startup in case the limit was lowered
between restarts. No background tasks, no DB reads needed for the check.
2026-06-10 20:29:18 +09:30
Jesse Bate
429f97a60c perf: exclude unused fields from ItemDTO list and WebSocket serialization
Every item_updated WebSocket event and every history/queue list response
was serializing the full ItemDTO including fields the UI never reads.

Add ItemDTO.serialize_for_list() which excludes: options,
template_chapter, temp_dir, total_bytes, total_bytes_estimate,
tmpfilename, archive_id. The global Encoder now uses serialize_for_list()
for all list/event contexts; the full serialize() remains available for
detail endpoints (GET /api/history/{id}).

Mark the excluded fields optional in the TypeScript StoreItem type.
2026-06-10 20:09:13 +09:30
Jesse Bate
e0dbc2d15d perf: throttle item_updated WebSocket events to 500ms intervals
yt-dlp fires progress hooks faster than the browser can usefully render
them. Previously every callback triggered a WebSocket emit → Vue
reactivity update → DOM re-render, causing unnecessary main-thread work
during active downloads.

Now status changes (e.g. downloading → finished) are always emitted
immediately, while progress-only updates are capped at one per 500ms
per download. A flush_pending() call at loop termination ensures the
final state is never dropped.
2026-06-10 19:32:31 +09:30
arabcoders
4881fabb15 feat: add YTP_DISABLE_EXEC to strip dangerous yt-dlp options 2026-06-08 09:29:31 +03:00
arabcoders
fa8ba1af34 feat: make log level runtime managable. 2026-05-28 21:26:23 +03:00
arabcoders
1267dbaf6b refactor: use single logger namespace 2026-05-28 19:07:11 +03:00
arabcoders
39ed3d0361 refactor: add more context to the actual log message. 2026-05-28 17:56:28 +03:00
arabcoders
cf90db3b4c refactor: update log mesages to take advantage of JSONL 2026-05-27 23:53:42 +03:00
arabcoders
434d3e981d feat: add system diagnostics endpoint. 2026-05-26 19:22:36 +03:00
arabcoders
64fdcac0f7 refactor: implement re-extraction logic and process-safe handling for non-picklable objects. 2026-05-25 20:06:34 +03:00
arabcoders
2e9684de5d refactor: standardize JSONL structure. 2026-05-24 21:19:25 +03:00
arabcoders
854f5f4b42 refactor: use jsonl for loggng for better log view experience 2026-05-18 09:44:29 +03:00
arabcoders
d18145f1c5 refactor: make clear message more clear 2026-05-18 07:44:37 +03:00
arabcoders
ee8218764f fix: suppress task related archive logs 2026-05-15 18:45:56 +03:00
arabcoders
734ef1c53c refactor: use item guid as thumbnail name for easier cleanup 2026-05-15 16:50:13 +03:00
arabcoders
d9dbbb18ea feat: thumbnail generation 2026-05-14 23:22:27 +03:00
arabcoders
9b3cc8b310 fix: respect sleep-requests arg in specific code paths. 2026-05-07 21:35:40 +03:00
arabcoders
285e661667 feat: add rename to history items. 2026-05-04 20:29:34 +03:00
arabcoders
11bd4204aa refactor: update terminal api 2026-04-29 22:25:39 +03:00
arabcoders
084f5082ad feat: add support for ignoring conditions in downloads and UI 2026-04-27 20:58:00 +03:00
arabcoders
30a0c30d94 refactor: fix slowness in deleting big number of items
Co-authored-by: Copilot <copilot@github.com>
2026-04-24 23:18:14 +03:00
arabcoders
3c1567930f feat: add system limits API endpoint and UI for download capacity overview 2026-04-22 22:02:40 +03:00
arabcoders
29777ff928 refactor: remove ui_update_title and standardize page title 2026-04-22 20:55:29 +03:00
arabcoders
03cabe6c70 refactor: wait on db flush before returning in API history clear response 2026-04-21 22:56:46 +03:00
arabcoders
00404fc6d4 refactor: made _static.py dynamic 2026-04-18 19:26:44 +03:00
arabcoders
dbff4c6798 feat(console): make terminal window more durable and support reloading and stopping commands. 2026-04-14 02:19:35 +03:00
arabcoders
c70d5340f1 feat: implement patch to make Windows Popen support stopping live stream 2026-04-11 23:31:48 +03:00
arabcoders
f9119f5a33 fix: better partial live download handling 2026-04-11 19:13:00 +03:00
arabcoders
1fa60c3909 fix: playlist keys pre-processing. Closes #585 2026-04-10 18:26:54 +03:00
arabcoders
c3132858e7 feat: add download_skipped flag to track intentionally skipped downloads 2026-04-10 18:07:05 +03:00
arabcoders
84077349c7 feat: improve status tracking in downloads 2026-02-14 20:05:08 +03:00
arabcoders
a0fef50248 fix: some times adding a link fails with UnboundLocalError 2026-02-14 18:49:52 +03:00
arabcoders
90e07ce71b feat: improve the file browser with pagination. 2026-02-13 21:46:01 +03:00
arabcoders
a3fc7f6abb fix: remove unnecessary terminator call and improve exception handling in status tracker 2026-02-13 18:16:18 +03:00
arabcoders
5844290d74 fix: sometimes downloads fails due to pickling issue 2026-02-08 03:45:56 +03:00
arabcoders
435882e16b chore: packages update and some formatting 2026-02-05 01:28:06 +03:00
arabcoders
abf4c5cffd fix: remove useless log statement. 2026-01-30 21:13:48 +03:00
arabcoders
f7c74c0a63 fix: upgrader use yt-dlp[default] as package name 2026-01-30 16:55:58 +03:00
arabcoders
dfcbd2d23d chore: update external packages 2026-01-30 16:31:20 +03:00
arabcoders
ed6cdf1f92 refactor: minor UX fixes. 2026-01-29 16:48:18 +03:00
arabcoders
dd0633c2aa refactor: improve error messaging during failure to load config 2026-01-28 17:09:08 +03:00
arabcoders
f88422e92b Refactor: move log related inside the logs module 2026-01-28 16:22:44 +03:00
arabcoders
b1a05b1d79 refactor: move archiver to be under yt_dlp submodule 2026-01-27 20:04:54 +03:00
arabcoders
2c5edc64f4 refactor: isolate yt_dlp related code into submodule 2026-01-27 19:36:13 +03:00
arabcoders
3383d5d5dc refactor: move streaming logic to a feature submodule 2026-01-27 17:42:12 +03:00
arabcoders
240a920b6c fix; proper fix for dispatching many websocket events 2026-01-27 11:46:00 +03:00
arabcoders
dae141a6c2 revert: changes to playlist processor 2026-01-27 10:41:17 +03:00