- 'Download X Wishlist' -> 'Process X Wishlist' everywhere (label + action_type):
matches the music side's 'process_wishlist', the already-process youtube action,
and reads right (movie/episode do search+pick+download, not just download).
action_types video_download_movie/episode_wishlist -> video_process_*.
- group the new automations as a two-stage pipeline in the seed list + builder
palette: Stage 1 SCANS (people/channels/playlists) fill the wishlist, Stage 2
PROCESSORS (movie/episode/youtube) drain it. icons/labels/drift test updated.
brand-new this session so no migration needed. 337 automation tests green.
the soulseek counterpart of the youtube drain — finally makes the people/airing
scans pay off. two automations (Auto-Download Movie / Episode Wishlist, hourly):
for each wished+released item, do a bounded blocking slskd search, pick the top
ACCEPTED release per the quality profile, and enqueue it exactly like a manual
grab (same add_video_download shape → the monitor finishes + organises it).
same standard as youtube: processes the WHOLE eligible wishlist (no total cap)
but searches a few at a time (max_concurrent, default 3, via a thread pool). a
busy guard skips the next hourly tick while a drain is still working. movies gate
on status='wanted' (skips monitored); episodes are all-wished. items already
downloading are skipped. quiet skip if the library folder isn't set.
reuses the real infra (build_query/slskd_search/_evaluate_hits/start_download/
download_monitor) — pure pick/select/record seam-tested; db queries added. 377
automation+video tests green.
playlists are followable but had no scan. new 'Scan Watchlist Playlists'
automation, sibling of the channel scan but a different rule: a playlist is a
curated finite set, so MIRROR it — wishlist every long-form video you don't have
plus any later additions (no forward-looking baseline, no last-N net).
playlist-as-show: videos wishlisted under the playlist's title, so the worker
files them as 'Playlist Name / Season YEAR / ... - date - title' (the ytdl-sub
tv_show_name-on-a-playlist convention). reuses all the channel plumbing — same
wishlist rows, same Download YouTube Wishlist drain, quality + org template.
seeded every 6h (Auto-Scan Watchlist Playlists); block + registration + icon +
drift test. dedup reuses wishlisted_video_ids_for_channel (parent_source_id=PL).
seam-tested; 944-test sweep green.
new 'Download YouTube Wishlist' automation: pushes a polite batch of wished
youtube videos into the shared video_downloads queue + spawns the yt-dlp worker
per video. skips in-flight ones (no double-grab); big backlogs drain over
several scheduled runs (batch_size, default 3). needs the youtube library folder
set.
- download_monitor: SKIP source='youtube' rows (owned by their worker thread, no
slskd transfer to match) — surgical, slskd path untouched.
- db.youtube_wishlist_to_download(): flat newest-first list of wished videos with
channel/title/date/thumb for organising.
block + registration + icon/label + drift test. all seam-tested.
two new video-side automation blocks that keep the wishlist fed:
- scan watchlist people: for each followed person, wishlist every un-owned
movie they acted in or directed (back catalog + upcoming). released ->
wanted, upcoming -> monitored (engine skips it til it's out, promotes on
release). grabs rich detail at add time (backdrop/cast/overview/etc +
provenance) into a new video_wishlist.detail_json col. fast re-runs skip
already-wishlisted + only promote.
- scan watchlist channels: for each followed youtube channel, wishlist new
long-form uploads (shorts excluded). forward-looking from follow time +
a last-N safety net (default 10). diffs against wishlisted/downloaded/
dismissed so it never dupes. pair with a 6h schedule trigger. scan-only;
fulfillment engine comes later.
both are pure handlers with injected seams + full seam tests. add_movie_to_
wishlist gains status + detail_json (promote-only upsert). no schema break,
music side untouched.
Unlike the cleanup twins, backup can't share the music handler — it's a different
DB file. Extract the music backup body into _backup_db_at(db_path, ...) (music
behaviour byte-identical, now a thin wrapper over DATABASE_PATH) and add
auto_backup_video_database pointing at VIDEO_DATABASE_PATH (video_library.db).
New video_backup_database action (scope='video' block + registry), owned_by='video'
system automation on the music cadence (every 3 days).
Tests: a REAL backup behaviour test — music backup lands next to music_library.db,
video backup next to video_library.db, no cross-contamination (this is the whole
reason it can't be shared); scope isolation; single video-owned seed; own handler.
Existing music maintenance tests (22) still green — refactor is non-regressing.
EXPECTED_ACTION_NAMES updated.
Same pattern: video_full_cleanup action (scope='video' block + registry), reuses
the shared auto_full_cleanup handler, owned_by='video' system automation on the
music cadence (every 12h). Music copy untouched. Seam tests + EXPECTED_ACTION_NAMES.
Same pattern as phase 2: video_clean_completed_downloads action (scope='video'
block + registry), reuses the shared auto_clean_completed_downloads handler, and
an owned_by='video' system automation on the music cadence (every 5 min). Music
copy untouched. Seam tests for scope isolation, single video-owned seed, and
shared-handler reuse; EXPECTED_ACTION_NAMES updated.
The music side's 'Clean Search History' automation now has a video counterpart so
it appears on the video Automations page too. Distinct action_type
video_clean_search_history (the system seeder keys on action_type, so reusing the
music key would collide), registered to the SAME shared handler so behaviour is
identical, scope='video' block (registry — users can build their own), and an
owned_by='video' system automation on the same 1h cadence. The music action/row
is untouched.
Seam tests: video-scoped only (not on music), music action still music-scoped,
exactly one video-owned system row at the 1h cadence, and it reuses the music
handler. Registration contract (EXPECTED_ACTION_NAMES) updated.
A deep scan is the equivalent of music's full refresh — it READS the server's
current state into video.db and prunes what's gone. It should NOT tell Plex to
rescan its disk. The deep-scan action types were wired to auto_video_scan_library
(nudge Plex + read); point them at the read-only auto_video_update_database in
'deep' mode instead. Update-db phase wording no longer says "new" for a full re-read;
deep-scan block descriptions clarify it's a read, not a disk-scan. Registration test
asserts the deep scans route to the read-only handler and never nudge the server.
Video twin of music's 'Auto-Deep Scan Library', split in two because Movies and TV
are separate libraries — scanning the TV library must not pull in new movies and
vice-versa.
- scanner: add a media_type param ('all'|'movie'|'show', friendly aliases) that
gates the movies vs shows passes (and their pruning), plus an in_progress busy
guard so the singleton scanner can't be stomped by an overlapping run.
- video_scan_library handler: thread media_type through, skip cleanly when the
scanner is busy, and name only the scanned library in the summary.
- two system automations (owned_by=video, weekly deep scan, staggered start delays):
'Auto-Deep Scan Movie Library' + 'Auto-Deep Scan TV Library'. Distinct action
types (video_deep_scan_movies / _tv) because the seeder keys on action_type; both
reuse the one handler, scoped via action_config.
- builder block gains a Library selector (Movies+TV / Movies / TV) so custom scans
can scope too; card label/icon maps cover the video action types.
Seam tests for scanner scope + busy guard, handler scope + skip, registration set.
Two Sonarr-parity features.
1) Per-episode live tracking. "Grab season" was headless (only a button label
changed); episode rows had a status span that was never populated. Now every
episode ROW shows its own live state — Searching → Downloading % → Downloaded
/ Failed — via epTrack() polling /downloads/status?id, matching the inline
movie tracker. Grab season lights all target rows at once; manual + per-source
auto grabs also light their row; reopening the modal resumes tracking in-flight
episodes (resumeEpisodeTracking via /downloads/active + search_ctx match).
Season batch grabs through the same payload as a manual grab (_pickAndGrab →
sendGrab(buildGrabPayload)).
2) Auto-wishlist airing episodes. New daily automation (video_add_airing_episodes):
reads the calendar for episodes airing TODAY for followed shows, skips owned
ones, adds the rest to the wishlist (idempotent). Handler uses injected seams
(calendar read + wishlist write) so it's unit-tested without a DB/server.
Registered + action block + seeded as a daily system automation (01:00),
owned_by=video.
Mirrors the music flow: a finished download batch → refresh the media server → (after
it indexes) pull the new media into the DB — so a downloaded movie/episode shows as
owned without waiting for the 6h scheduled scan.
Two event-based system automations (owned_by='video'):
- 'Auto-Scan Video After Downloads' (video_batch_complete → video_scan_server)
- 'Auto-Update Video Database After Scan' (video_library_scan_completed → video_update_database)
Pieces:
- core/video/download_events.py: a callback registry (core/video can't import the
engine — isolation). The monitor publishes batch-complete; web_server bridges it to
automation_engine.emit('video_batch_complete', …), like music's web_scan_manager.
- download_monitor: fires the batch-complete event once, when the last in-flight
download finishes (none queued/downloading/searching left).
- video_scan_server handler (stage 1): refresh server, wait a debounce for indexing,
then emit 'video_library_scan_completed' (mirrors music's time-based completion).
- video_update_database handler (stage 2): incremental read (newest-first, stop after
25 consecutive known — same as music).
- blocks: 2 video triggers + 2 video actions (scope='video'); registration; seeds.
kettui: seam tests for both handlers (refresh/wait/emit, incremental read, error paths),
the event registry (idempotent register, isolated failures), and the monitor (fires once
on last completion, never while work remains). 298 automation tests + isolation green.
The video side gets its OWN automations at music-side parity, kept separate so
nothing on the music side breaks. First twin: Scan Video Library — tells the media
server to rescan the user's SELECTED video sections (movies/TV, never music), then
reads the result into video.db so freshly-downloaded media shows as owned.
Architecture (scope tags + video twins on the shared engine):
- Handler core/automation/handlers/video_scan_library.py — pure function with
injected I/O (server_refresh / run_video_scan); production lazily binds
refresh_video_server_sections() + the video scanner. Owns its own progress.
Lives on the SHARED automation side so it may import core.video (isolation only
forbids core/video & api/video from importing music, not the reverse).
- blocks.py gains a 'scope' tag ('both' generic / 'video' video-only / absent=music)
+ blocks_for_scope(). The music /api/automations/blocks now filters out video
blocks; new isolated /api/video/automations/blocks serves the video palette.
- automation_engine seeds 'Scan Video Library' (owned_by='video', schedule 6h) so
it appears ONLY on the video Automations page; ensure_system_automations now
honours owned_by + action_config. Music page excludes owned_by='video' rows.
kettui: seam-level tests for every handler path (happy/no-server/scan-error/never-
raises/mode), scope filtering (music excludes video, video gets generics, music
parity preserved), seeding (owned_by + mode), registration drift guard. 39 new
tests; full automation suite (288) + isolation guards green.
Phase 2 of the redesign. The tool that judged quality by extension and auto-dumped
matches into the wishlist is gone; quality scanning is now the reviewed
quality_upgrade repair job.
Removed:
- Frontend: Tools-page Quality Scanner card, its JS handlers/poller/socket listener,
help tooltip + tour entry (webui index.html, core.js, helper.js, wishlist-tools.js).
- Backend: /api/quality-scanner/{start,status,stop} endpoints, the in-memory state +
executor + 1s socket broadcast, the QualityScannerDeps/run_quality_scanner shim.
- core/discovery/quality_scanner.py: the auto-acting worker + deps class (the shared
match/normalize helpers stay — the new job imports them).
Rewired:
- Automation 'start_quality_scan' action now triggers the quality_upgrade repair job
via repair_worker.run_job_now() (AutomationDeps gains run_repair_job_now, drops the
4 scanner fields). Action block's vestigial scope field removed (scope lives in the
job's settings now). NOTE: the 'quality_scan_completed' trigger no longer fires (the
repair job doesn't emit it).
- Updated all automation test _build_deps helpers + conftest tool-progress harness;
deleted the obsolete worker test. 528 affected tests pass; 6123 collect cleanly.
QUALITY_TIERS / _get_quality_tier_from_extension kept (used elsewhere).
Persist organize_by_playlist on mirrored playlists and run playlist-folder
downloads from the auto-sync pipeline instead of the global wishlist phase.
Register SoulSync library rows after playlist-folder post-processing, route
failed organize batches to the wishlist correctly, and skip sync-time
unmatched wishlist only when organize download handles retries.
Invalidate stale playlist track caches on refresh (Spotify and Deezer ARL),
re-mirror on refetch, and improve standalone playlist modals (re-analysis,
Open in Mirrored). Add filesystem missing-track detection and tests.
Co-authored-by: Cursor <cursoragent@cursor.com>
Follow-up to the personalized-playlists standardization PR. New
`personalized_pipeline` automation action syncs selected discover-
page playlists (Hidden Gems / Discovery Shuffle / Time Machine /
Genre / Daily Mix / Fresh Tape / The Archives / Seasonal Mix) to
the active media server + queues missing tracks for download.
Same pattern as the existing mirrored `playlist_pipeline` but two
phases instead of four — no REFRESH (no external source to re-pull)
and no DISCOVER (manager-backed snapshots are already metadata-
matched). Pipeline shape:
SNAPSHOT → SYNC → WISHLIST
Where SNAPSHOT either reads the persisted track list from
`PersonalizedPlaylistManager` (default) or refreshes it first when
`refresh_first=true` (cron use case: regenerate Hidden Gems nightly
and sync the fresh set).
Shared helper extraction:
PHASE 3 (SYNC loop) + PHASE 4 (WISHLIST tail) lifted out of mirrored
`playlist_pipeline` into `core/automation/handlers/_pipeline_shared.py`
as `run_sync_and_wishlist(deps, automation_id, playlists, sync_one_fn,
sync_id_for_fn, ...)`. Both pipelines call it. Mirrored injects
`auto_sync_playlist` as the per-playlist sync function; personalized
injects a thin wrapper that launches `_run_sync_task` directly with
a pre-built tracks_json. Same sync-state polling / progress emission
/ status counting / wishlist trigger logic — 0 duplication.
Files added:
- core/automation/handlers/_pipeline_shared.py
- core/automation/handlers/personalized_pipeline.py
- tests/automation/test_handlers_personalized_pipeline.py
Files changed:
- core/automation/handlers/playlist_pipeline.py: PHASE 3+4 replaced
with shared helper call (~100 lines deleted, 1 helper invocation
added; behavior identical).
- core/automation/deps.py: new `build_personalized_manager` field
(lazy builder so the pipeline gets a fresh PersonalizedPlaylistManager
per run).
- core/automation/handlers/__init__.py + registration.py: register
`personalized_pipeline` action with the shared `pipeline_running`
guard so it can't overlap mirrored.
- core/automation/blocks.py: new `personalized_pipeline` block
declaration with config_fields (kinds multi-select, refresh_first,
skip_wishlist).
- web_server.py: thread `_build_personalized_manager` into
AutomationDeps construction.
- All 5 automation test fixtures: `_build_deps` adds
`build_personalized_manager=lambda: None` stub.
- tests/automation/test_handler_registration.py:
EXPECTED_ACTION_NAMES + EXPECTED_GUARDED_ACTIONS gain
`personalized_pipeline`.
Trigger schema:
{
"_automation_id": "...",
"kinds": [
{"kind": "hidden_gems"},
{"kind": "time_machine", "variant": "1980s"},
{"kind": "seasonal_mix", "variant": "halloween"}
],
"refresh_first": false,
"skip_wishlist": false
}
Tests (14 new, 178 automation total):
- _track_to_sync_shape: basic shape, source ID fallback chain,
no-id returns empty string
- empty config / non-list kinds / empty kinds list all return
error + clear pipeline_running flag
- _build_payloads_for_kinds: skips invalid entries, skips kinds
with no tracks, refresh_first vs ensure dispatch, payload shape
+ sync_id format, manager exception swallowed continues
- _sync_personalized_playlist: launches background thread + returns
status='started'
- happy path: stubbed sync_states drives helper to completion, flag
cleaned up
Full suite: 3383 passed.
Note: the trigger UI block declares config_fields but the frontend
doesn't yet render the `personalized_playlist_select` multi-select
type — usable today via API; polished UI ships in a follow-up
frontend PR.
Per-handler boundary tests pin each handler's body in isolation.
Adding engine-boundary tests that pin the REGISTRATION layer:
- every expected action name registered, no drops, no extras
- guarded actions register a guard, unguarded ones don't
- every registered handler is callable
- every guard returns a bool
- all four progress callbacks registered in the right slots
- progress_init / progress_finish / record_history / on_library_scan_completed
are invocable through the engine's stored callable shape (not just
the bare extracted function)
- finish callback respects _manages_own_progress flag at the engine
boundary too
- library_scan_completed wiring registers a callback on the scan
manager and that callback fires engine.emit when invoked
- every handler returns a `{'status': ...}` dict on a minimal config
trigger -- proves no handler raises into the engine, even when its
guard / short-circuit / error path is the one taken
Uses a minimal _RecordingEngine that captures registrations + a
_RecordingScanMgr that captures completion callbacks. No real
AutomationEngine, no real Flask app, no real DB. The kettui standard
for refactor PRs: don't ship "behavior preserved" claim that's only
validated at the function boundary -- exercise the engine seam too.
EXPECTED_ACTION_NAMES + EXPECTED_GUARDED_ACTIONS frozen sets at the
top: any future drift (rename / drop / add a handler / change which
ones are guarded) fails this test immediately so refactor PRs can't
quietly mutate the registration shape.
13 new tests, 164 automation tests pass total.
