Compare commits

..

443 commits
main ... video

Author SHA1 Message Date
BoulderBadgeDad
8eadadc1ae youtube retention: auto-clean old channel episodes (per-channel keep window)
Some checks failed
Compile the app and run tests / sanity-check (push) Has been cancelled
new opt-in feature — default keeps everything, so nothing changes unless a channel sets it.

- history got smarter: youtube rows now carry channel_id + published_at (mined from
  search_ctx) + a pruned_at marker. new queries: youtube_channels_with_downloads,
  youtube_channel_episodes, mark_download_pruned.
- core/video/retention.py (pure): parse_retention + episodes_to_prune — age by UPLOAD
  date (published_at, filename fallback); 'count_N' keeps newest N, 'days_N' keeps last N
  days; undated episodes never pruned.
- handler auto_video_clean_youtube_episodes: for each channel with a policy, delete the
  out-of-window video + its -thumb/.nfo sidecars (only the exact recorded dest_path, never
  walks folders), then mark the history row pruned — KEPT so the scan never re-downloads it.
- 'Keep' dropdown in each channel's cog modal (Everything / Last 30 episodes / Last 3 / 6
  months); playlists excluded. seeded daily automation + register/block/label/icon/sort.
pure math + handler (i/o injected) + the prune-keeps-dedup DB contract all tested.
2026-06-26 17:24:59 -07:00
BoulderBadgeDad
151bb7b542 video: hourly 'Update Video Database' safety net (the second-trigger pattern)
dual triggers on one automation aren't a thing (one trigger per row, same as music), so
this adds a second automation sharing the handler — like Auto-Deep Scan TV/Movie already
do. 'Auto-Update Video Database (Hourly)' runs the same cheap incremental server-read on
an hourly schedule, so MANUAL library additions (which Plex auto-scans) appear within the
hour instead of waiting for the weekly deep scan; the existing after-scan one still gives
instant updates for SoulSync-initiated scans.

new action_type video_update_database_hourly (seeder keys on action_type) reusing
auto_video_update_database in incremental mode; registered, seeded (schedule 1h, 15m
initial delay), UI block, label/icon, sort order. tested.
2026-06-26 16:39:28 -07:00
BoulderBadgeDad
11c0c62a7e video enrichment: stop OMDb traceback spam from the detail/drawer path too
the earlier latch only covered _backfill_ratings. _fill_tmdb_ratings (hit by
tmdb_full_detail → detail pages, the download drawer's meta endpoint, the airing
automation's status lookups) still re-hit OMDb once per title and dumped a traceback
each time once over quota. now it shares the same _omdb_blocked latch: checks it before
calling, sets it + logs ONE quiet warning on OMDbAuthError. (the bulk OMDb worker already
cools down + auto-resumes, so it was never the spammer.) tested.
2026-06-26 16:28:37 -07:00
BoulderBadgeDad
75f23641b3 youtube downloads: show 'Importing' during the ffmpeg merge, not a stuck 100%
the '100% but still Downloading for a while' phase was yt-dlp merging audio+video
(ffmpeg) INSIDE the download call — status stayed 'downloading' the whole time. now a
postprocessor_hook flips the row to 'importing' the moment post-processing starts, so
the card shows 'Importing' through the merge AND the library move (which already set
it), then completes. matches the movies/TV treatment. hook wiring tested.
2026-06-26 15:37:08 -07:00
BoulderBadgeDad
f6360e90da youtube downloads: write episode sidecars (-thumb.jpg + .nfo), gated by toggles
best-in-class for Plex/Jellyfin, matching ytdl-sub: each youtube episode now gets a
'<name>-thumb.jpg' (episode art) + a '<name>.nfo' (<episodedetails>: title, plot from
the video description, aired date, season=year, episode=MMDD, channel, youtube uniqueid).

- yt-dlp now writes the thumbnail + an info.json next to the staged video; the import
  step renames the thumb, mines the json for the nfo, and cleans both up (so nothing
  litters the download folder when a toggle is off).
- gated by the SAME post-processing toggles as movie/TV (save_artwork / write_nfo),
  which already exist in Settings → Library; youtube just honours them now.
- flipped save_artwork + write_nfo defaults ON (cheap, local). download_subtitles stays
  opt-in (it hits OpenSubtitles — external + rate-limited).
build_episode_nfo + the gated sidecar I/O tested.
2026-06-26 13:57:52 -07:00
BoulderBadgeDad
02a319c350 gitignore: ignore database/*.histbak (the suffix slipped past the backup_* rule) 2026-06-26 13:41:50 -07:00
BoulderBadgeDad
ca1348b2c2 downloads: youtube 'open' goes to the in-app channel page, not youtube.com
the open button now opens the channel inside soulsync (channels-as-shows) via
video-open-detail {kind:'channel', source:'youtube', id}, matching how movie/show
cards open their in-app detail. enqueue_ctx now carries channel_id (new downloads)
and youtube_video_detail returns it too (so older downloads resolve the channel in
the drawer). drawer gets an 'Open channel' button alongside 'Open on YouTube';
falls back to the youtube video link only when the channel id is unknown.
2026-06-26 13:24:41 -07:00
BoulderBadgeDad
3f924812da downloads: youtube card open button → YouTube link (was opening a random show)
the open-show button rendered for youtube cards too; the click ran parseInt on the
youtube video id → ids starting with a digit became a small library id (e.g. 3 →
'3rd Rock from the Sun'), others became NaN → broken link. youtube cards now render a
real YouTube link for that button; movie/show/episode keep the open-detail button.
regression test pins the youtube branch.
2026-06-26 13:19:47 -07:00
BoulderBadgeDad
b916e6a2f3 downloads drawer: episode detail + youtube treatment + availability line
- EPISODE downloads now show the specific episode: still + 'S02E05 · air date' + that
  episode's own title + synopsis (was just the show synopsis). meta endpoint takes
  ?season=&episode= → engine.tmdb_season; show cast/logo stay as context.
- YOUTUBE drawer: big 16:9 thumbnail + channel · duration · views · upload date +
  description. new yt-meta route → db.youtube_video_detail (cached duration/views).
- AVAILABILITY line: the chosen source's free-slot/queue/speed snapshot, stashed in
  search_ctx at grab time (build_download_record) → 'Availability: ✓ free slot · queue 0
  · 2.1 MB/s'.
drawer also reworked into clean youtube / episode / movie-show branches; first paint
shows 'Loading…' instead of flashing 'no synopsis'. tested.
2026-06-26 13:06:44 -07:00
BoulderBadgeDad
c35ec43466 downloads drawer: rich TMDB detail + fix cast photos + stop OMDb quota burn
drawer upgrades (the TMDB call we already make returns all this — just render it):
- FIX cast photos — used c.profile_url; the field is c.photo. now shows headshots.
- title LOGO header (falls back to text) + meta line (year · rating · runtime · network).
- tagline, director/creator, ▶ trailer link, and a 'Watch on' provider-logo row.
- meta endpoint widened to return logo/cast-photo/rating/runtime/crew/trailer/providers.

OMDb spam fix (the 'Request limit reached!' tracebacks):
- the new bulk schedule-refresh called refresh_show_art per show, which did an OMDb
  ratings backfill each → blew the daily quota. refresh_show_art gains with_ratings;
  the automation passes False (it only needs episode schedules).
- _backfill_ratings now latches _omdb_blocked on OMDbAuthError → one quiet warning +
  stops calling OMDb for the rest of the process, instead of a traceback per show.
tested (cast/logo/trailer contract, no-ratings plumbing, the latch-off behavior).
2026-06-26 12:49:11 -07:00
BoulderBadgeDad
a30a70fafd automations page: friendly labels + icons for the video maintenance actions
the video-prefixed maintenance actions (clean_search_history, clean_completed_downloads,
full_cleanup, backup_database) + the new refresh_airing_schedules weren't in the action
label/icon maps, so they fell back to the gear + raw action_type ('⚙️ video_full_cleanup').
added all five to both maps (🗓️/🗑️//🧹/💾, mirroring the music side). regression test
asserts every seeded video automation resolves a label AND an icon, so this can't recur.
2026-06-26 11:51:29 -07:00
BoulderBadgeDad
9aee723d1a video: 'Refresh Airing TV Schedules' automation — keep the calendar current
the airing automation reads the LOCAL episodes table, which only refreshes on initial
enrichment / manual re-match / lazy on-view — so a newly-announced or rescheduled
episode could be missed indefinitely. (the deep TV scan is a SERVER scan, unrelated.)

new daily automation re-pulls TMDB episode schedules (air dates/stills) for still-airing
watchlist shows so the calendar is fresh when the airing run reads it:
- db.watchlist_continuing_shows(): effective-watchlist library shows that are still airing
  (skips tmdb-only follows — no episodes — and ended/canceled shows; keeps unknown status).
- handler mirrors the standard: injected fetch_shows + refresh_show seams, live per-show
  progress, _manages_own_progress; reuses engine.refresh_show_art (match + episode cascade).
- seeded daily at 23:00 (2h before the 01:00 airing run), registered, UI block, sorted in
  the page right before the airing automation.
seam-level tested + a regression that it's seeded before the airing run.
2026-06-26 11:47:15 -07:00
BoulderBadgeDad
9a8550661b downloads page: click-to-expand detail drawer (synopsis + cast + facts)
click a card → it expands inline into a detail drawer (open state survives the
in-place re-patches via _expanded):
- big backdrop + synopsis + genres + a cast strip (photos/names/characters),
  lazily fetched from TMDB by the grab's tmdb id (new /downloads/meta/<kind>/<id>
  endpoint → engine.tmdb_full_detail). youtube shows channel + description instead.
- a facts grid: status, quality target, release, format, source+queue, size,
  attempts, copyable dest path, full error.
- big actions: open in library / open on youtube / copy path / cancel / retry.
all type-themed (Cinema palette) and scoped to .vdpg-card. contract-tested + the
meta route is registered.
2026-06-26 11:30:36 -07:00
BoulderBadgeDad
458bf01599 downloads page: Cinema per-type theming + sidebar live count + status polish
- per-type colour (movie=azure / TV=violet / youtube=red) on a left accent bar, the
  status pill, the progress fill + quality chip — tell the three apart at a glance.
- bigger poster art + a type corner badge (reads even with a poster).
- status is now a coloured pill; queued/searching/importing get an indeterminate
  shimmer instead of a frozen bar; 'Importing' + 'Import failed' surfaced with their
  own context lines ('Moving into your library…').
- sidebar Downloads nav shows a live active-count badge, kept fresh off-page by a
  light poll (skips the fetch while the page's own poll is running).
all scoped to .vdpg-card so the music downloads page is untouched. contract-tested.
2026-06-26 11:22:56 -07:00
BoulderBadgeDad
cc8cc78b97 video downloads: consistent stage→import→library pipeline + 'Importing' status
both lanes now expose the post-processing phase instead of jumping downloading→completed:
- YouTube now STAGES in the shared download folder (a 'youtube' subdir) then transfers to
  the library — same as movies/TV — so no partial .part files land in the Plex folder.
  process_youtube_download gains stage_dir + move seams: download → 'importing' → move →
  completed (import_failed if the move dies; wish kept for retry). falls back to
  straight-to-library when no download folder is set.
- movies/TV: the organizer flips the row to 'importing' before the (slow) move + sidecars +
  subtitles, so that phase is visible.
- 'importing' is now an active status (get_active_video_downloads + youtube concurrency count
  + the downloads page status map), so the row stays on the page + holds its worker slot.
tested (staging + import-failure paths).
2026-06-26 11:11:29 -07:00
BoulderBadgeDad
12cec4c042 video search: availability-aware ranking (best-in-class, like music)
picks the most DOWNLOADABLE release, not just the biggest/fastest:
- peer_availability() mirrors the music side's scoring — free upload slot, tiered
  upload speed, graduated queue penalty.
- group_video_files now captures queueLength, picks the best PEER per release by
  (availability, then speed) instead of speed alone, and ranks hits by availability.
- _evaluate_hits sorts by (accepted, quality score, availability, size) so within a
  quality tier a free-slot/empty-queue source wins over one stuck behind a 1500-deep
  queue — fixes the downloads that sat at 0%.
queue/speed also surfaced on results for the UI. tested.
2026-06-26 10:54:56 -07:00
BoulderBadgeDad
e84d1bcde5 slskd search: throttle creation to avoid 429 storms
the real cause of the movie-search failures: slskd rate-limits search CREATION and
429s when exceeded — and the video path had NO throttle (the music side caps ~35/220s).
the thread-pool auto-grab fired searches in bursts (and each 429 returned instantly,
cascading into more), storming slskd into 429s; only a few slipped through.

add a shared throttle on start_search: min 2s between creations + a 35-per-220s window
cap (mirrors music) + a cooldown when a 429 IS hit (honors Retry-After) so it backs off
instead of hammering. pure reserve/cooldown logic tested.
2026-06-26 10:27:28 -07:00
BoulderBadgeDad
6a1fa78995 video wishlist processor: show the actual slskd error on 'search didn't run'
surface why start_search failed (slskd error string) in the automation log instead
of a generic 'not responding?'. seam now returns (candidates, error); handler
tolerates bare list/None too (test fakes). so we can see the real reason.
2026-06-26 10:15:23 -07:00
BoulderBadgeDad
1b3c18e4e6 slskd search: supply our own search id → fixes 'search didn't run'
root cause of the fast 'no results' / 'search didn't run': slskd WAS creating the
searches (they showed up + returned results), but start_search couldn't read the
search id back out of the POST response, returned no id, and the caller bailed
instantly — then raced to the next, which is why slskd showed different titles than
the log.

fix: generate the search id client-side and pass it to slskd (it honors a supplied
id), so we never depend on parsing the response. still prefer slskd's echoed id if
present (dict/list/bare-string), fall back to ours. also send Accept: application/json
so slskd answers in json.
2026-06-26 10:09:31 -07:00
BoulderBadgeDad
eb8498a792 video wishlist processor: surface 'search didn't run' vs genuine no-results
'No search results' was masking the case where slskd never accepted the search
(not configured / errored / rate-limited) — that returns instantly, which is why
some show up as no-results FAST instead of after the search window.

_search_for_retry now flags started=False (+ the slskd error) when there's no
search id; the processor logs "Search didn't run for 'X' — slskd not responding?"
(warning) and tallies it separately. so the next run tells us if the fast
no-results are really slskd refusing searches vs the source genuinely being empty.
2026-06-26 10:02:49 -07:00
BoulderBadgeDad
8fe73ac924 video: longer adaptive slskd wait + stop the automations page blink
two issues:
1. slskd search gave up at 22s (then stopped the search), but slskd gathers peers
   over its full ~60s window — so most movie searches were killed before results
   arrived. now waits up to 55s but returns early once results SETTLE (12+ hits, or
   no new hits for ~12s) so fast searches don't burn the whole window. matches the
   music side's longer wait.
2. the automations page re-rendered the WHOLE system section every 8s poll
   (replaceChild) → blink + wiped the socket's live progress. now it skips the
   rebuild unless something structural changed (added/removed/toggled/ran); live
   progress comes via socket, the countdown ticks locally.
2026-06-26 09:54:18 -07:00
BoulderBadgeDad
b55c245fe6 video downloads: re-download + clear history actions
after you delete a file, the scans dedup against download history so it won't
re-grab — this gives you the escape hatch:
- per-row 'Re-download' button in the History modal: forgets that grab (DELETE
  /downloads/history/<id>) so the next scan re-adds + re-downloads it.
- 'Clear' button: wipes the whole history (guarded confirm).
db.delete_download_history / clear_download_history(kind?) + the two endpoints.
tested (db methods + endpoints).
2026-06-26 09:44:56 -07:00
BoulderBadgeDad
f380e8ac27 youtube scans: dedup against downloaded videos so they don't re-download
a completed youtube download is removed from the wishlist (correct), but the
channel/playlist scans' downloaded_ids seam was stubbed to [] — so the next scan,
not seeing it wishlisted and not knowing it was downloaded, re-added it (still in
the channel's recent uploads / last-N net) → re-download loop for the last-N
videos every scan.

fix: db.downloaded_youtube_video_ids() pulls completed youtube grabs from the
permanent download history; both scans' _default_downloaded_ids now use it. so a
downloaded video stays gone. tested (db method + the scans already exclude
downloaded_ids in their pure logic).
2026-06-26 09:34:58 -07:00
BoulderBadgeDad
bacc528ba1 video search: stop slskd searches when done → fixes movie-search flooding
bug: start_search tells slskd to keep searching its full timeout (~60s), but
_search_for_retry only polled then walked away — never stopping the search. for
movies (popular → 12+ hits in <1s → worker early-breaks → fires the next search
immediately) this piled up dozens of 60s-long slskd searches at once. tv episodes
return fewer hits → workers block the full 22s → searches issue slowly → no pileup
(why tv looked fine and movies flooded).

fix: stop_search(id) (DELETE /api/v0/searches/{id}); _search_for_retry stops its
search in a finally, even on early break. concurrent slskd searches now ≈ the
worker pool (3). retry worker benefits too. tested.
2026-06-26 09:10:15 -07:00
BoulderBadgeDad
6531a19c7a video wishlist processor: log WHY nothing grabbed (no results vs quality reject)
'no acceptable release' was ambiguous — couldn't tell if slskd returned nothing
or returned hits the quality profile rejected. now each item logs 'No search
results for X' vs 'N result(s) for X, none accepted — <reason>' (reason from the
top hit), and the summary tallies '… · N had no results, M rejected on quality'.
makes the slskd-finds-nothing-for-video situation legible.
2026-06-26 08:44:40 -07:00
BoulderBadgeDad
fc045cd74e youtube download: don't clobber target_dir → fixes double-nested folders
bug: process_youtube_download wrote the ORGANISED dir (channel/Season YYYY) back
to the row's target_dir, but target_dir is supposed to be the youtube ROOT — and
plan_destination re-derives channel/season UNDER it. so any re-processing re-nested:
Channel/Season 2026/Channel/Season 2026/. that hit the 1-2 videos per channel that
got interrupted mid-download and re-queued by the orphan reaper.

fix: only record the organised filename for display; never write target_dir. the
root stays put so re-runs are idempotent. regression test runs it twice + asserts
no target_dir clobber + a stable (non-nested) dest dir.
2026-06-26 08:37:47 -07:00
BoulderBadgeDad
f0e1fda0fc video wishlist badge: poll so it reflects server-side removals (downloads)
the remaining badge bug: when a download finishes it removes its item from the
wishlist SERVER-SIDE, which fires no frontend 'changed' event — so the badge went
stale and only 'jumped' (e.g. down to the tv-only count) on the next navigation.

add a lightweight badge poll: refreshes the authoritative /wishlist/counts every
8s while downloads are active (uses the downloads page's _vdpgAnyActive), every
30s idle, paused when the tab is hidden. now the count tracks down live as wishlist
items get downloaded. frontend add/remove events still fire instantly as before.
2026-06-26 00:19:54 -07:00
BoulderBadgeDad
1722618c6d video wishlist badge: stop the second writer clobbering it with the TV-only count
the other half of the bug: the endpoint total was fixed, but setCounts() (movie/
episode load) AND setYtCounts() (youtube load) ALSO wrote the nav badge from their
OWN partial state. these load separately, and on the dashboard only one runs — so
whichever fired last overwrote the badge with a partial count (correct number,
then 'switches' to TV-only).

fix: the /wishlist/counts endpoint is the single source of truth for the grand
total; both setters now just call refreshBadge() to (re)sync from it instead of
computing a partial sum. regression test pins it.
2026-06-25 23:21:59 -07:00
BoulderBadgeDad
3fa7b48f4c video wishlist badge: count YouTube videos too (header + sidebar)
bug: the wishlist badge read /wishlist/counts 'total', which was movies+episodes
only — YouTube videos (kind='video') are counted by a separate method and were
left out. so a wishlist of only youtube videos showed NO number; it only lit up
once tv episodes were added.

fix: the /wishlist/counts endpoint now folds youtube_wishlist_counts().video into
the total (+ exposes video/channel counts). the badge already live-refreshes on
wishlist-changed events (which youtube add/remove fire), so it now updates for
youtube too. db methods unchanged (their byte-identical contract is intact).
regression tests: mixed + youtube-only.
2026-06-25 23:15:45 -07:00
BoulderBadgeDad
23c64e000a youtube playlists: same settings cog/modal as channels
playlist cards on the watchlist tab now get the same hover cog → the per-channel
settings modal (custom show-name + quality override). the modal is kind-aware so
it reads 'Playlist settings' + 'the playlist's real name'.

no backend change needed: playlist videos are wishlisted with channel_id =
playlist_id, so the existing get/set_channel_settings + enqueue override lookup
already key off the playlist id. UI-only: cog on playlistCard + data-kind passed
to openChannelSettings(id, title, kind). test covers both cogs.
2026-06-25 23:07:15 -07:00
BoulderBadgeDad
f734905ba1 youtube channels: per-channel settings modal UI (cog on the Channels tab)
hover a channel on the watchlist Channels tab -> a settings cog appears (mirror
of the unfollow x) -> opens a self-contained modal:
- Show name (folder): overrides the $channel folder/show-name token; blank uses
  the real channel name.
- 'Force a specific quality' toggle -> resolution/codec/container/60fps/HDR
  override; off = use the global youtube quality from Settings.

VideoYoutube.openChannelSettings(id, title) builds the modal, loads via GET, saves
via POST /youtube/channel/<id>/settings. .vyt-cset-* CSS. string-contract test.

(note: 10 pre-existing soundcloud music-orchestrator test failures are unrelated
to this work — they fail on a clean HEAD too.)
2026-06-25 23:01:35 -07:00
BoulderBadgeDad
7140b88948 youtube channels: per-channel overrides (custom show-name + quality) — core
storage + wiring + API for the channel settings modal (UI next):
- db.get/set_channel_settings(channel_id): per-channel {custom_name, quality}
  in the settings KV store (no schema change); blanks clear the override.
- enqueue applies them: custom_name overrides the $channel folder token; a
  quality override is stashed in the download row's search_ctx. the worker reads
  it back (quality_override_from_download) and uses it instead of the global
  youtube quality profile. global default still applies when no override.
- API GET/POST /youtube/channel/<id>/settings (GET also returns the global
  default quality for the modal's 'using default' hint).

pure helpers (enqueue_ctx, quality_override_from_download) + db + api seam-tested.
2026-06-25 22:53:45 -07:00
BoulderBadgeDad
5bb90053bf youtube fulfillment: orphan reaper for restart-stalled downloads
closes the real gap: a restart kills the yt-dlp worker threads but leaves their
rows at 'downloading', which the pump counts as busy -> the queue wedges and they
never finish.

track live worker dl_ids in _active_worker_ids (added at worker start, dropped in
finally). requeue_orphaned_youtube() puts any 'downloading' youtube row with no
live worker back to 'queued' so the pump re-runs it. after a restart the set is
empty, so all stuck rows recover; during normal operation active ones are
protected (no false positives, no timestamp guessing). the hourly drain calls it
before pumping + logs the count. seam-tested.
2026-06-25 22:48:54 -07:00
BoulderBadgeDad
846d217dca video automations: order the System list as a logical pipeline
the API returns system automations newest-created-first, so a re-seeded row
(e.g. the just-migrated movie/episode processors) jumps to the top and jumbles
the grouping. the video page now re-sorts its System list by an explicit order:
scans (fill) → processors (drain) → library scan/sync → maintenance. scoped to
the video page only (music ordering untouched); unknown/future actions fall to
the end keeping API order. string-contract test pins the scan<proc<maint order.
2026-06-25 22:38:26 -07:00
BoulderBadgeDad
10895a7c16 video automations: migrate the Download->Process rename in seeded DBs
bug: renaming the action_types only changed code — a db already seeded under the
old names kept the stale rows (the seeder only CREATES missing ones, never removes
or renames). so on restart you got BOTH 'Auto-Download Movie Wishlist' (orphaned,
dead action) AND 'Auto-Process Movie Wishlist'.

add _fix_wishlist_processor_rename (runs in ensure_system_automations like the
other _fix_* migrations): deletes the orphaned video_download_movie/episode_wishlist
system rows (clearing is_system first, since delete_automation guards system rows)
and renames the youtube row in place (its action_type was unchanged). idempotent.
tested.
2026-06-25 22:33:03 -07:00
BoulderBadgeDad
7d7d242e5d video automations: rename Download->Process + group into a clear pipeline
- 'Download X Wishlist' -> 'Process X Wishlist' everywhere (label + action_type):
  matches the music side's 'process_wishlist', the already-process youtube action,
  and reads right (movie/episode do search+pick+download, not just download).
  action_types video_download_movie/episode_wishlist -> video_process_*.
- group the new automations as a two-stage pipeline in the seed list + builder
  palette: Stage 1 SCANS (people/channels/playlists) fill the wishlist, Stage 2
  PROCESSORS (movie/episode/youtube) drain it. icons/labels/drift test updated.

brand-new this session so no migration needed. 337 automation tests green.
2026-06-25 22:26:31 -07:00
BoulderBadgeDad
b404420c76 video automations: auto-download movie + episode wishlist (soulseek)
the soulseek counterpart of the youtube drain — finally makes the people/airing
scans pay off. two automations (Auto-Download Movie / Episode Wishlist, hourly):
for each wished+released item, do a bounded blocking slskd search, pick the top
ACCEPTED release per the quality profile, and enqueue it exactly like a manual
grab (same add_video_download shape → the monitor finishes + organises it).

same standard as youtube: processes the WHOLE eligible wishlist (no total cap)
but searches a few at a time (max_concurrent, default 3, via a thread pool). a
busy guard skips the next hourly tick while a drain is still working. movies gate
on status='wanted' (skips monitored); episodes are all-wished. items already
downloading are skipped. quiet skip if the library folder isn't set.

reuses the real infra (build_query/slskd_search/_evaluate_hits/start_download/
download_monitor) — pure pick/select/record seam-tested; db queries added. 377
automation+video tests green.
2026-06-25 22:08:02 -07:00
BoulderBadgeDad
fb9459fb5c video automations: scan watchlist YouTube playlists (mirror-all)
playlists are followable but had no scan. new 'Scan Watchlist Playlists'
automation, sibling of the channel scan but a different rule: a playlist is a
curated finite set, so MIRROR it — wishlist every long-form video you don't have
plus any later additions (no forward-looking baseline, no last-N net).

playlist-as-show: videos wishlisted under the playlist's title, so the worker
files them as 'Playlist Name / Season YEAR / ... - date - title' (the ytdl-sub
tv_show_name-on-a-playlist convention). reuses all the channel plumbing — same
wishlist rows, same Download YouTube Wishlist drain, quality + org template.

seeded every 6h (Auto-Scan Watchlist Playlists); block + registration + icon +
drift test. dedup reuses wishlisted_video_ids_for_channel (parent_source_id=PL).
seam-tested; 944-test sweep green.
2026-06-25 21:18:53 -07:00
BoulderBadgeDad
b83f314617 video automations: seed the three new ones as system automations
they were only builder blocks (not in the Active list). seed them like the
airing job so they appear + run out of the box:
- Auto-Scan Watchlist People — daily 03:00
- Auto-Scan Watchlist Channels — every 6h
- Auto-Download YouTube Wishlist — every 1h

scans no-op cleanly if you follow nothing. softened the download handler: an
unset youtube folder is now a quiet skip (status completed), not a per-run error,
so non-youtube users don't see a recurring failure.
2026-06-25 20:56:37 -07:00
BoulderBadgeDad
7a90b008f0 youtube fulfillment: drain the whole wishlist, cap concurrency not total
ditch the per-run batch cap (a 200-video backlog would've taken weeks). now the
'Download YouTube Wishlist' automation queues the ENTIRE wishlist as 'queued'
rows, starts up to max_concurrent (default 3) right away, and each finished
download starts the next (one-out-one-in in the worker) so it all drains in a
controlled stream. the knob is 'max simultaneous downloads', not a total cap.

mirrors the music download worker's lesson (cap concurrency + space starts to
avoid yt-dlp 429s) but stays isolated on the video side:
- youtube_download: _pace() staggers fetch starts (3s); start_next_queued()
  claims+spawns the next; run_youtube_download chains on finish.
- db.count_active_youtube_downloads() + claim_next_youtube_queued() (atomic,
  race-safe). block field batch_size -> max_concurrent.

all seam-tested (pure select + pump); 680-test sweep green.
2026-06-25 20:10:45 -07:00
BoulderBadgeDad
3254e8a6c4 settings: youtube path template field on the Library tab
surface the youtube_template (built into organization.py) on Settings -> Library
Organization, next to the movie/episode templates: input + variable hints
($channel/$year/$date/$month/$day/$title/$videoid) + live preview, wired into
load/collect/save/reset. backend already round-trips it (load/save normalize).
2026-06-25 19:40:56 -07:00
BoulderBadgeDad
c47dccc7d7 youtube fulfillment: drain wishlist -> download queue
new 'Download YouTube Wishlist' automation: pushes a polite batch of wished
youtube videos into the shared video_downloads queue + spawns the yt-dlp worker
per video. skips in-flight ones (no double-grab); big backlogs drain over
several scheduled runs (batch_size, default 3). needs the youtube library folder
set.

- download_monitor: SKIP source='youtube' rows (owned by their worker thread, no
  slskd transfer to match) — surgical, slskd path untouched.
- db.youtube_wishlist_to_download(): flat newest-first list of wished videos with
  channel/title/date/thumb for organising.

block + registration + icon/label + drift test. all seam-tested.
2026-06-25 19:30:02 -07:00
BoulderBadgeDad
8e071719f9 youtube fulfillment: download worker (pure core)
new core/video/youtube_download.py: fetch a wished youtube video end to end.
plans the organised dest (channel/year/date template), builds yt-dlp opts from
the quality profile (format_selection), runs the download (injectable factory),
then on success marks the video_downloads row completed + archives to history +
removes it from the wishlist; on failure archives failed and KEEPS the wish to
retry. orchestration is pure (yt-dlp run + all db writes injected) + seam-tested;
run_youtube_download binds the real seams for the worker thread.

flows through the SAME video_downloads queue as movies/tv (model B) so the
downloads page + history work for youtube for free.
2026-06-25 19:24:58 -07:00
BoulderBadgeDad
00d05a04f5 video org: youtube channel template (Plex TV-by-date)
add a 'youtube' scope to render_path: channel=show, season=upload year,
episode named '$channel - $date - $title'. rides the existing $token
engine (sanitised, dangling-separator tidy). undated videos fall back
cleanly (no empty 'Season ' / no stray ' - '). default template editable
like the movie/episode ones; per-channel override comes with the settings
modal later.
2026-06-25 19:20:18 -07:00
BoulderBadgeDad
568e297d36 youtube quality: map profile -> yt-dlp format selection
pure format_selection(profile) -> {format, format_sort, merge_output_format}.
caps to the resolution ceiling (falls back to uncapped so above-cap-only videos
still grab), ranks codec/res/fps/sdr as soft prefs (never excludes a stream).
the one piece the youtube downloader needs regardless of how the engine is wired.
exact yt-dlp tokens tunable on live yt-dlp; tests pin the shape.
2026-06-25 19:11:35 -07:00
BoulderBadgeDad
31d284ab0d video automations: watchlist scans for people + channels
two new video-side automation blocks that keep the wishlist fed:

- scan watchlist people: for each followed person, wishlist every un-owned
  movie they acted in or directed (back catalog + upcoming). released ->
  wanted, upcoming -> monitored (engine skips it til it's out, promotes on
  release). grabs rich detail at add time (backdrop/cast/overview/etc +
  provenance) into a new video_wishlist.detail_json col. fast re-runs skip
  already-wishlisted + only promote.

- scan watchlist channels: for each followed youtube channel, wishlist new
  long-form uploads (shorts excluded). forward-looking from follow time +
  a last-N safety net (default 10). diffs against wishlisted/downloaded/
  dismissed so it never dupes. pair with a 6h schedule trigger. scan-only;
  fulfillment engine comes later.

both are pure handlers with injected seams + full seam tests. add_movie_to_
wishlist gains status + detail_json (promote-only upsert). no schema break,
music side untouched.
2026-06-25 19:07:06 -07:00
BoulderBadgeDad
877a86c9b0 Video Discover: split Top 10 today into Movies + TV Shows (Netflix-style)
The mixed /trending/all chart is movie-heavy — TV was nearly absent (only
1 of the top 10 was a show). Split it like Netflix: 'Top 10 today' now holds
two ranked rails, Movies and TV Shows, from the dedicated /trending/movie/day
and /trending/tv/day charts (full 10 of each).

- client.trending(window, kind): single-type charts force the kind into
  _disc_map (those endpoints omit media_type).
- engine.trending(window, kind): kind in the cache key.
- /discover/list: key=trending_movies_today / trending_tv_today, both treated
  as single un-paged charts; inherit the same hide-owned + ranked rendering.
- frontend: one 'Top 10 today' group, two ranked shelves titled Movies / TV
  Shows (group header carries the 'Top 10' framing).
2026-06-24 08:00:13 -07:00
BoulderBadgeDad
e7552d3c62 Video Discover: fix orphaned Top 10 numerals when hiding owned + lift 'New' up the page
Bug: with 'Hide owned' on, an owned title in the Top 10 (e.g. House of the
Dragon) had its card hidden by the global .vdsc-hide-owned rule while its big
rank numeral stayed — a blank gap with just a number. The ranked rail fetched
the true chart (ignoring hide_owned) precisely to keep the chart intact, which
collided with that CSS. Fix: ranked rail now honours hide_owned at fetch time
(owned dropped server-side, ranks stay contiguous 1-N), plus a CSS safety net
exempting ranked cards from the hide rule so a numeral can never be stranded.
When hide-owned is OFF the true chart still shows owned with the 'In Library'
ribbon.

Ordering: moved 'New & noteworthy' above the async collection/taste groups so
the two always-visible discovery rows (Top 10 + New) anchor the top of the
page, matching how streaming services surface new releases.
2026-06-24 07:33:58 -07:00
BoulderBadgeDad
24b79dc29e Video Discover: sticky 'jump to section' nav for the deep rail stack
The page is now a long stack of rail groups (For you / Top 10 / New /
Trending / Mood / Studios / Genres / More). A sticky chip bar lets you
jump straight to any section — Netflix/Disney-style category nav. Chips
stay in lockstep with the groups: async-only groups (foryou/collection/
taste) reveal their chip when filled; pruned groups hide theirs. Lives
inside the shelves host so it auto-hides in the Browse-all grid view.
2026-06-23 23:58:53 -07:00
BoulderBadgeDad
a275cbfb7a Video Discover: Top 10 ranked rail + New/Mood/Studios sections + NEW badge
Best-in-class discovery pass (Netflix/Disney-class):
- iconic ranked 'Top 10 today' rail (daily trending, big outlined rank
  numerals, true global chart via lang=any, single un-paged fetch)
- 'New & noteworthy' (date-windowed new releases + theatrical pipeline)
- 'By mood' (genre AND-combos w/ vote floors: feel-good, mind-bending…)
- 'From the studios' (Pixar/Ghibli/A24/Marvel/DreamWorks via company ids)
- 'Something different' (quick watches / epics / family) merged with gems
- 'NEW' flag on un-owned current-year cards
Preserves async-loader group ids (foryou/collection/taste). Empty rails
self-prune; wrong studio ids just drop their rail.
2026-06-23 23:55:30 -07:00
BoulderBadgeDad
bbf7abad1b video discover: wire Netflix-class TMDB filters (keywords/studio/network/cast/runtime/cert/release-window + daily trending)
Foundation for the best-in-class discover build. All additive — existing callers unaffected:
- clients.discover() gains keywords (mood/theme), companies (studio), networks (TV), cast/crew
  (people), min/max_runtime, certification (+country), vote_count_min override, and release_window
  ('last_30/90/365', computed date ranges) — each mapped to the right TMDB /discover param + gated
  to movie/tv where TMDB requires.
- engine.discover_filter() threads them through + into the cache key.
- engine.trending(window) adds the real-time 'day' chart (cache keyed by window) for a Top 10 row.
- /discover/list parses the new query params and a key=trending_today shortcut.

py_compile + ruff clean. Frontend rails (Top 10, mood, studio, new&upcoming, quick-watches) next.
2026-06-23 23:45:41 -07:00
BoulderBadgeDad
378ae39bef tests: add missing tests/video package marker
tests/video/__init__.py was created with the phase-1a gap-engine tests but never staged (those
commits added test files by explicit path). Every other test subfolder has a tracked __init__.py;
this keeps tests/video consistent and guards against import-file-mismatch under pytest's default
prepend import mode (the suite already has a real duplicate basename, test_selection.py).
2026-06-23 10:54:27 -07:00
BoulderBadgeDad
82cbc22577 video discover: organise rails into fixed, labelled sections (stable order)
The three async personalized loaders each did insertAdjacentHTML('afterbegin'), so 'Recommended
for you' / 'More like X' / gap rails landed in whatever order their fetch finished — the top of
the page reshuffled every load, and taste-based rails were scattered through generic ones.

Now Discover renders 6 authored groups in a fixed order, each with a header:
  For you · Finish your collection · More of what you like · Trending & popular ·
  Browse by genre · Hidden gems & more
Each async loader fills its own group's body (deterministic position) instead of racing to the
top; async-only groups (gaps) stay hidden until filled, and a group whose rails all drop out is
pruned. Extracted lazyShelfHtml/filledShelfHtml/shelfNav helpers; lazy-load, stagger, gen-guard,
see-all and scroll arrows all unchanged.
2026-06-23 10:39:41 -07:00
BoulderBadgeDad
09a1646a6a video discover: fill rails to ~20 via concurrent deep-paging (no more 4-card rows)
A heavy library with hide-owned on drained the old 8-page sequential fill before a rail filled,
leaving rows of 4-7 cards. Now the filtered-fill path pages up to 20 deep in concurrent waves of
5 (TTLCache + TMDB client are thread-safe), stopping as soon as it hits ~20 kept items or TMDB
runs out — so digging deep costs ~4 page-latencies instead of 20. Refactored the per-page filter
into consume(); trending + the no-filter path are unchanged in behaviour.
2026-06-23 10:20:52 -07:00
BoulderBadgeDad
4f24ac2733 video discover: stop duplicate 'Recommended for you' rows stacking on rail rebuild
Toggling service/language chips calls reloadRails(), which clears the shelves synchronously
but the personalized loaders (foryou/gaps/morelike) fetch async and insertAdjacentHTML afterbegin.
Rapid re-toggles let superseded in-flight fetches prepend anyway -> N 'Recommended for you' rows
piled up. Each rebuild now bumps state.railGen and every loader bails if its captured gen is
stale before prepending; chip-driven rebuilds are also debounced (350ms) so multi-select
coalesces into one rebuild. The 'On your streaming services' rail itself was building fine -
it was just buried under the duplicates.
2026-06-23 10:00:52 -07:00
BoulderBadgeDad
cd29239b2f video discover: move 'My services' out of the page-wide bar (it's a rail builder, not a filter)
'My services' only builds the optional 'On your streaming services' rail, but sitting under
the 'Across Discover' header it read as a page-wide filter (user confusion: 'i thought it
affected the whole page because that's what it says'). Moved it to its own self-describing row
('pick what you subscribe to — adds a rail to your feed') below the bar. 'Across Discover' now
holds only the genuinely page-wide controls: Hide owned + Languages. Markup/CSS only — the
data-vdsc-myprov hook is unchanged.
2026-06-23 09:53:36 -07:00
BoulderBadgeDad
1feb34621e video discover: cascade rail cards in instead of one block flash
Each rail showed a shimmer skeleton then revealed the whole row in a single fade — on a
big library with hide-owned on (which pages deep server-side) that reads as a long blank then
a pop. Cards now stagger in left-to-right via a per-card --i index (capped) + a 'backwards'
fill-mode keyframe (so the entrance animation releases and :hover transforms still work).
Applies to lazy rails (fillShelf) and the prepended personalized rows (staggerWithin).
2026-06-23 09:32:18 -07:00
BoulderBadgeDad
73395b9668 video discover: give the Browse panel its own language filter (self-contained search)
The Browse-all grid silently inherited the global rail language preference, so users
couldn't ad-hoc browse foreign cinema without changing their homepage prefs. Added a
language chipset to the Browse panel (auto-wired via the generic chip handler -> state.sel.lang)
and the grid now always sends lang= : a real code filters, 'any' opts OUT of the rail
preference entirely. Route treats lang=any as 'no language filter'. Added a 'Browse the full
catalog' eyebrow so the panel reads as a self-contained search, parallel to 'Across Discover'.

Hide-owned and the saved 'My services' pref stay single/global by design (they apply page-wide);
the Browse panel's provider chips remain its own grid filter.
2026-06-23 09:26:22 -07:00
BoulderBadgeDad
dffc5c2e5d video discover: lift page-wide controls into a distinct 'Across Discover' bar
Hide-owned + language + my-services were inside the Browse-all filter panel, making them read
as grid-only when they actually affect every rail. Moved them into a labelled 'Across Discover'
strip above the Browse panel (JS finds them by data-attribute, so no logic change). Browse panel
now holds only its grid filters (kind/sort/genre/source/decade).
2026-06-23 09:19:19 -07:00
BoulderBadgeDad
5906576a13 video discover phase 3: 'On your streaming services' rail
A saved streaming-services preference drives a personalized 'On your streaming services' rail:
- GET/POST /discover/providers-pref (TMDB provider ids); /discover/list OR-joins multiple
  providers (comma->pipe) into with_watch_providers.
- A 📺 multi-select in the toolbar (Netflix/Prime/Disney+/Max/Apple TV+/Hulu/Paramount+/Peacock);
  selecting services saves the preference and rebuilds the rails.
- The rail (high priority, after taste) appears once you've picked services, showing what's
  streaming on yours.
2026-06-23 00:33:35 -07:00
BoulderBadgeDad
6734b6ab22 video discover phase 4 (UI): 'Not interested' card button + ignore-list modal
- A ✕ 'Not interested' button on every un-owned Discover card (hover) — adds to the ignore list
  and fades the card out instantly.
- A '🚫 Ignore List' button top-right of the hero opens a vibey glassmorphic modal: a header
  explaining what it is, a search box to hide any movie/show directly (TMDB search), and a poster
  grid of everything hidden with one-click 'Un-hide'. Empty state guides the user.
- Card button + modal both POST /discover/ignore; ignored titles vanish from every rail (via
  _stamp_owned). Video-only, additive.
2026-06-23 00:29:46 -07:00
BoulderBadgeDad
3f344a8cd9 video discover phase 4 (backend): ignore list / 'Not interested'
Movie/show-level ignore list (episodes can't be individually ignored):
- video_ignored table (kind, tmdb_id, title/year/poster snapshot) + SCHEMA_VERSION 18->19.
- add_ignored / remove_ignored / list_ignored / ignored_keys DB methods.
- _stamp_owned (the choke point all discover results pass through) drops ignored titles, so
  every surface hides them uniformly + always-fresh (tiny indexed query). Person-gap rails
  (which bypass _stamp_owned) filtered in the gaps endpoint.
- GET/POST /discover/ignore (list / add / remove).
2026-06-23 00:25:10 -07:00
BoulderBadgeDad
d019e3aca1 video discover: hidden-gems rails (highly-rated, non-blockbuster)
Add 'Hidden Gems' (movies) + 'Critically Acclaimed Shows' rails — vote_average.desc with the
backend's vote_count floor filtering out single-vote noise, and the language preference applied.
Slot them above the decade/foreign rails.
2026-06-23 00:13:19 -07:00
BoulderBadgeDad
f1b6fd5e31 video discover: language preference UI (multi-select chips)
A 🌐 multi-select chip row in the Discover toolbar (EN/KO/JA/ES/FR/HI/DE/IT) to pick which
original languages appear in the general/curated rails. Loads the current preference from
/discover/languages, toggling a chip POSTs the new set and rebuilds the rails (never empty —
at least one stays on). Extracted reloadRails() (now shared by the hide-owned toggle + language
chips). Default EN, so the rails are English unless you opt more in.
2026-06-23 00:11:56 -07:00
BoulderBadgeDad
28fe3d2b0b video discover: preferred-languages filter (keep Bollywood etc. out of general feeds)
The general/curated rails (Popular/Trending/Top Rated + genre/decade) pull TMDB's GLOBAL lists,
flooding feeds with foreign-language titles (Bollywood). Add a multi-language preference:
- _disc_map now carries original_language (+ popularity) on each item.
- discover_languages setting (default 'en'); /discover/list post-filters general/curated rails
  to it (dropping known non-preferred-language titles) and pages deeper to keep rails full.
  Rails with an explicit lang (the dedicated foreign rails) bypass the filter.
- GET/POST /discover/languages to read/set the preference.
- Removed the hardcoded lang=en on general rails (the setting drives it now).
Default 'en' immediately fixes the Bollywood flood; UI to pick languages next.
2026-06-23 00:09:51 -07:00
BoulderBadgeDad
e7b1a239b4 video discover phase 2: blended 'Recommended for you' wall
A single personalized wall aggregating TMDB recommendations across many of your owned titles
(random_owned_titles seeds), ranked by consensus — a title recommended by more of your library
ranks higher (ties by rating then popularity), owned + seed titles excluded.
- core/video/discovery_recs.py: pure blend_recommendations (dedup/consensus/exclude), 7 tests.
- /api/video/discover/foryou aggregates ~12 seeds' recommendations.
- loadForYou() prepends the 'Recommended for you' rail on top of the stack; re-runs on the
  hide-owned toggle.
2026-06-23 00:05:26 -07:00
BoulderBadgeDad
d984895cba video discover: keep foreign titles out of general rails + fix hide-owned sparse rows
Two Discover UX issues:
- Foreign-language titles leaked into the general genre/decade rails. Added an
  original-language filter (with_original_language) through client.discover -> discover_filter
  -> /discover/list (?lang=); the genre/decade/'because you like' rails now pin lang=en, and a
  handful of dedicated foreign rails (Korean/Japanese/Spanish/French/Hindi) house non-English.
- 'Hide owned' + a huge library = nearly-empty rails (a 2-page batch was mostly owned, then
  CSS-hidden to almost nothing). /discover/list now takes hide_owned=1: it drops owned
  server-side and pages DEEPER (up to 8) until a rail has ~24 un-owned. fillShelf passes
  hide_owned when the toggle's on; toggling re-renders the rails (+ personalized rows) instead
  of just CSS-hiding cards.
2026-06-22 23:59:21 -07:00
BoulderBadgeDad
a68d9755a9 video discover: lazy collection-id backfill so 'complete your collections' lights up
Already-matched movies predate the tmdb_collection_id column, so the collection gap rails
were empty (only newly-enriched movies got the id). Add a self-healing backfill: each
/discover/gaps load fills the franchise id for up to 20 owned movies missing it
(eng.movie_collection reuses the matcher's belongs_to_collection read), recording 0 for
movies with no franchise so they're not re-checked. The 'no-franchise' 0 is excluded from
the rails. Backfill is wrapped/isolated so it can never break the gap response. Over a few
Discover visits the whole library fills in and 'Complete the <franchise>' rails populate.
2026-06-22 23:48:45 -07:00
BoulderBadgeDad
8236b95db0 fix: don't pass the 'custom' cookies sentinel to yt-dlp (unsupported browser error)
The #902 'paste cookies.txt' feature added a 'custom' sentinel value for
youtube.cookies_browser, but that feature wasn't merged to this branch — and ~7 call sites
(core/youtube_client.py x5, core/video/youtube.py, web_server.py) pass cookies_browser raw to
yt-dlp's cookiesfrombrowser, which rejects 'custom' ('ERROR: unsupported browser: custom') and
broke YouTube download/enrichment. Sanitize 'custom' -> '' (no browser) at every site:
youtube_client reads via a walrus filter, the other two guard the condition. 'custom' now
means 'no browser cookies' here (the cookiefile feature isn't on this branch). Latent on dev
too — only _youtube_cookie_opts was fixed there.
2026-06-22 23:41:37 -07:00
BoulderBadgeDad
78874313cc fix(video): move idx_movies_collection to _POST_INDEXES (broke DB init)
The new index on movies.tmdb_collection_id was in video_schema.sql, which executescript runs
BEFORE _ensure_columns adds the column on existing DBs — so 'CREATE INDEX ... ON
movies(tmdb_collection_id)' failed with 'no such column' and the whole video DB init aborted
(500s on every /api/video/* call). Moved the index to _POST_INDEXES (runs after the ALTERs),
matching the pattern the code comments already prescribe. The CREATE TABLE columns stay (fresh
DBs) + the ALTER migration stays (existing DBs); only the index moved.
2026-06-22 23:34:51 -07:00
BoulderBadgeDad
a8ce359cb7 video discover phase 1e: 'what am I missing' gap rails on the discover page
loadGaps() fetches /discover/gaps and prepends the gap rails ('Complete the <franchise>',
'More from <director/creator>') above the rail stack, mirroring loadMoreLike. Cards are the
standard un-owned TMDB cards — already actionable (VideoGet add-to-watchlist/get), so a
missing franchise entry or director film is one click from your queue. Best-effort/additive.
2026-06-22 23:30:05 -07:00
BoulderBadgeDad
ba6065f1b3 video discover phase 1b-d: gap-engine queries, engine collection fetch, /discover/gaps API
- video_database: owned_movie_tmdb_ids (diff set), owned_movie_collections (franchises you've
  started, most-invested first), top_owned_people (directors/creators you own the most).
- engine.collection(id): cached + owned-annotated franchise film list (person_detail already
  gives owned-annotated filmography).
- /api/video/discover/gaps: builds 'Complete the <franchise>' rails (collection_gaps) + 'More
  from <person>' rails (filmography_gaps, movies, vote-filtered) — the 'what am I missing' section.
All additive; gap diffs are the pure tested core.
2026-06-22 23:28:23 -07:00
BoulderBadgeDad
88553a95d2 video discover phase 1a: pure gap-engine core (collection + filmography diffs)
core/video/discovery_gaps.py — two pure diffs powering the 'what am I missing' rails:
collection_gaps (franchise entries you don't own, in collection order) and
filmography_gaps (a person's titles you don't own, deduped, kind/vote-filtered, ranked
by popularity). No I/O — the API wires owned-ids/collection-items/person-credits in.
9 tests.
2026-06-22 23:26:04 -07:00
BoulderBadgeDad
1f1a239486 video discover phase 0: persist TMDB collection (franchise) id per movie
Data layer for the 'complete your collections' gap engine. People/credits/genres are already
normalized + indexed, so the only missing signal was franchise membership:
- movies: + tmdb_collection_id (indexed) + tmdb_collection_name (schema + _COLUMN_MIGRATIONS,
  SCHEMA_VERSION 17->18, _ENRICH_META_COLS whitelist so enrichment_apply backfills them)
- enrichment match() reads belongs_to_collection (a standard movie-detail field, no extra call)
  and writes the id/name into the match metadata.
Additive + backfill-only (COALESCE), nothing existing rewired. (also noqa'd a pre-existing
OMDb S110 in the touched file to keep ruff clean.)
2026-06-22 23:24:42 -07:00
BoulderBadgeDad
d162966cee Video tools: add a Server Scan tool (trigger the media server to index new downloads)
This is the tool originally asked for — DISTINCT from the Library Scan (where
SoulSync reads the server into video.db). Server Scan tells Plex/Jellyfin to
rescan its OWN folders so newly-downloaded files get indexed, then a Library Scan
pulls them in. It's the manual twin of the post-download 'Scan Video Server'
automation, and targets Movies / TV / both like the Library Scan.

- POST /api/video/scan/server {media_type} -> refresh_video_server_sections (trigger)
- GET /api/video/scan/server/status?media_type -> {scanning:true|false|null} (live poll)
- new Server Scan card on the video Tools page + video-server-scan.js controller,
  mirroring the music live-status UX (phase + working bar); resumes if the page
  opens mid-scan. Server scans have no % (Plex doesn't report one) so the bar is a
  working indicator. Both backend functions already existed + are media-type aware.

Seam tests: trigger threads media_type (movie / default all), status reports the
scanning flag (True / null passthrough), and the blueprint exposes both routes.
2026-06-22 08:26:24 -07:00
BoulderBadgeDad
70ad5af378 Video tools: fix Library Scan card layout — wrap the 3 controls
Adding the Movies/TV target gave the scan card three controls (target + mode +
button), one more than music's two, overflowing the shared no-wrap flex row and
clipping the Scan Library button past the card edge. Scoped CSS on the video Tools
page lets the row wrap: two selects share the top row, the button takes its own
full-width row. Music's .tool-card-controls is untouched.
2026-06-22 08:20:47 -07:00
BoulderBadgeDad
3937cb4cb8 Video automations: Auto-Backup Database (video DB) — custom twin (phase 5)
Unlike the cleanup twins, backup can't share the music handler — it's a different
DB file. Extract the music backup body into _backup_db_at(db_path, ...) (music
behaviour byte-identical, now a thin wrapper over DATABASE_PATH) and add
auto_backup_video_database pointing at VIDEO_DATABASE_PATH (video_library.db).
New video_backup_database action (scope='video' block + registry), owned_by='video'
system automation on the music cadence (every 3 days).

Tests: a REAL backup behaviour test — music backup lands next to music_library.db,
video backup next to video_library.db, no cross-contamination (this is the whole
reason it can't be shared); scope isolation; single video-owned seed; own handler.
Existing music maintenance tests (22) still green — refactor is non-regressing.
EXPECTED_ACTION_NAMES updated.
2026-06-21 23:41:19 -07:00
BoulderBadgeDad
1b39e6aa3f Lint: silence S110 on two intentional best-effort swallows in video handlers (pre-existing)
ruff S110 flagged two try/except/pass in video handlers that predate this work.
Both are deliberate (a progress-log failure must not abort pruning; a probe's
uncertainty just keeps probing) — extend the existing BLE001 noqa to S110 with
the rationale. ruff check . is clean again.
2026-06-21 23:35:05 -07:00
BoulderBadgeDad
f111b5dda6 Video automations: Full Cleanup twin (phase 4)
Same pattern: video_full_cleanup action (scope='video' block + registry), reuses
the shared auto_full_cleanup handler, owned_by='video' system automation on the
music cadence (every 12h). Music copy untouched. Seam tests + EXPECTED_ACTION_NAMES.
2026-06-21 23:33:28 -07:00
BoulderBadgeDad
a6b2737a5c Video automations: Clean Completed Downloads twin (phase 3)
Same pattern as phase 2: video_clean_completed_downloads action (scope='video'
block + registry), reuses the shared auto_clean_completed_downloads handler, and
an owned_by='video' system automation on the music cadence (every 5 min). Music
copy untouched. Seam tests for scope isolation, single video-owned seed, and
shared-handler reuse; EXPECTED_ACTION_NAMES updated.
2026-06-21 23:32:28 -07:00
BoulderBadgeDad
3ef0990ffc Video automations: Clean Search History twin (phase 2)
The music side's 'Clean Search History' automation now has a video counterpart so
it appears on the video Automations page too. Distinct action_type
video_clean_search_history (the system seeder keys on action_type, so reusing the
music key would collide), registered to the SAME shared handler so behaviour is
identical, scope='video' block (registry — users can build their own), and an
owned_by='video' system automation on the same 1h cadence. The music action/row
is untouched.

Seam tests: video-scoped only (not on music), music action still music-scoped,
exactly one video-owned system row at the 1h cadence, and it reuses the music
handler. Registration contract (EXPECTED_ACTION_NAMES) updated.
2026-06-21 23:31:10 -07:00
BoulderBadgeDad
b26f664326 Video tools: scan a Movies OR TV library (not just both)
The video Library Scan tool only scanned 'all' — but movies and TV are
independent libraries (unlike music's single library). The scanner backend
already supported media_type='movie'|'show'|'all'; this just wires it up:
- /api/video/scan/request now reads media_type and threads it to request_scan
- the Tools card gains a target selector (All / Movies Only / TV Shows Only)
  alongside the existing mode dropdown, matching the music scan's UX
- the live status detail reflects the target (no confusing '0 shows' on a
  movies-only scan)

Seam test: the endpoint passes both mode and media_type through (default all/full,
explicit movie/deep, TV-only). Existing scanner media-type/scope tests unchanged.
2026-06-21 23:27:35 -07:00
BoulderBadgeDad
66bce2b83a Watchlist hygiene: auto-prune ended/canceled followed shows
You can eye-add a show to the watchlist before we know its status, so ended/canceled
shows leak in (auto-airing LIBRARY shows already exclude ended ones; explicit follows
don't). Fix it as cleanup-on-process, per Boulder: the daily 'Wishlist Today's Airings'
automation now runs a watchlist-tidy pass first — scans every explicit show follow,
resolves its status (local for owned, TMDB for tmdb-only follows), and removes any
that have ended/been canceled/completed. Only prunes on a DEFINITIVE terminal status;
unknown/lookup-error → left alone. Toggle prune_ended (default on); returns shows_pruned.

DB: followed_shows(). Pure prune_ended_show_follows() with injected seams; seam tests.
2026-06-21 15:05:20 -07:00
BoulderBadgeDad
7fac921afc Video show detail (TMDB source): show the Watchlist + Trailer actions
The TMDB-source show detail page rendered an empty action bar — renderActions
early-returned for source='tmdb' (a stale "previews have no actions" assumption that
predates the curated, tmdb_id-keyed watchlist), so the Watchlist button never showed
and the rest was skipped. Now an AIRING show gets the Watchlist button whether it's
owned or a TMDB preview (ended/cancelled stay terminal → no button); Trailer renders
from the payload; Get Missing stays library-only. Also fixed toggleWatchlist sending
a bogus library_id + 404 poster proxy for tmdb previews (data.id is the tmdb id there)
— it now omits library_id and uses the proxied TMDB poster, mirroring the card-hover add.
2026-06-21 14:57:32 -07:00
BoulderBadgeDad
0cf8654f47 Smart scan: poll the probe over a grace window (server auto-scan needs ~1-2 min)
The probe fired the instant a batch finished, but a fresh drop takes ~1-2 min to
appear even with the server's auto-scan ON — so it always missed and we crawled
anyway, defeating the optimization. Now probe_present_libraries POLLS each candidate
over a grace window (probe_grace_minutes, default 2), skipping a library's crawl as
soon as the server reports it has the item, and only crawling what's still missing
when grace expires. The probe target for a media type you DIDN'T just download is an
old item the server already has → confirmed instantly, no wait. grace=0 probes once.
2026-06-21 14:32:06 -07:00
BoulderBadgeDad
9e845e760e Smart post-download scan: skip the crawl when the server already has the grab (phase 3)
Scanning is expensive and most servers auto-ingest new files, so a full crawl after
every download is usually wasted. Stage 1 now probes per library: take the newest
completed grab of that type from download history and ask the server (cheap targeted
search) whether it already has it. If yes, the server auto-picked it up (and the
earlier ones) → skip that library's crawl + poll entirely. Only libraries the server
is missing get rescanned. Always emits so stage 2 still reads the new items in.

- sources: PlexVideoSource.has_item / JellyfinVideoSource.has_item (match movie by
  title+year, episode by show+SxE) + video_server_has_item() — conservative, any
  uncertainty → False so we scan.
- handler: per-scope skip decision fed by latest_completed + server_has_item seams;
  narrows the scan scope to only the missing libraries; toggle skip_if_present
  (default on). Returns scanned/skipped for visibility.

Seam tests: skip-both, scan-only-missing, no-history, toggle-off, probe-error→scan;
Plex has_item match tests.
2026-06-21 14:20:49 -07:00
BoulderBadgeDad
ed2fbf2ae4 Video download history: API + beautiful timeline modal (phase 2)
GET /api/video/downloads/history (paged, ?kind/search/outcome) + /history/<id>, both
returning the live tab counts. New self-contained modal (video-download-history.js,
.vdh-* styles) opened from a History button on the Downloads page: day-grouped
timeline of every grab with poster, title, S/E, quality/resolution/codec/size and an
outcome badge; rows expand in place to reveal the full detail (release, source/uploader,
codecs, dest path, grabbed/finished times, error). Tabs (All/Movies/TV), search,
load-more, and a live count badge on the button.
2026-06-21 14:15:37 -07:00
BoulderBadgeDad
6fe82b2a78 Video download history: permanent archive snapshotted at terminal status (phase 1)
video_downloads is a transient queue (hard-deleted on cleanup), so there was no record
of what SoulSync actually grabbed. Add a permanent video_download_history table +
capture: the monitor snapshots every terminal download (completed/import_failed/
cancelled/failed) into it, with rich metadata (title, year, S/E from search_ctx,
release, source, size, quality + parsed resolution/codec, dest path, poster, outcome,
timestamps). Idempotent per (download_id, outcome, dest_path).

DB methods: record_download_history, query_download_history (paged/kind/search),
download_history_detail, download_history_counts, latest_completed_download(media_type)
— the last is the probe target for the upcoming smart post-download scan. Schema v17.
2026-06-21 14:10:22 -07:00
BoulderBadgeDad
a16afd1f9e Post-download scan: wait until Plex's scan queue is actually idle, not a fixed 2min
A fixed debounce can't fit a big library — 8500 movies + 4500 shows scan sequentially
through Plex's queue and can take 10-20 min, so the old 120s wait read the DB before
Plex finished and fresh downloads showed up late. Now Stage 1 (video_scan_server)
fires the rescan then POLLS the server until its scan queue goes idle, then emits the
done event.

- sources: PlexVideoSource.is_scanning (section.refreshing + activity feed, scoped by
  media_type) and JellyfinVideoSource.is_scanning (scheduled-task state), plus
  video_server_scan_in_progress() returning True/False/None.
- handler: pure wait_for_server_scan(scan_status, sleep, …) — grace, then poll every
  interval until idle or a generous cap; falls back to the fixed wait only when the
  server can't report status (None). debounce_seconds is now that fallback; new
  max_wait_minutes caps the poll.

Seam tests for the poll logic (idle/poll/fallback/cap/lost-status), the handler wiring,
and Plex scan-status detection.
2026-06-21 13:44:52 -07:00
BoulderBadgeDad
5a53ffc8c2 Video deep scan: pure read/reconcile, not a Plex disk-scan trigger
A deep scan is the equivalent of music's full refresh — it READS the server's
current state into video.db and prunes what's gone. It should NOT tell Plex to
rescan its disk. The deep-scan action types were wired to auto_video_scan_library
(nudge Plex + read); point them at the read-only auto_video_update_database in
'deep' mode instead. Update-db phase wording no longer says "new" for a full re-read;
deep-scan block descriptions clarify it's a read, not a disk-scan. Registration test
asserts the deep scans route to the read-only handler and never nudge the server.
2026-06-21 13:27:55 -07:00
BoulderBadgeDad
14a32f6006 Video scan family: make every action movie/TV-aware + deep scans are real actions
The deep-scan action types weren't selectable builder actions, and Scan Video Server
/ Update Video Database had no movie-vs-TV dimension — inconsistent with the rest.

- video_deep_scan_tv / video_deep_scan_movies are now proper builder blocks
  (Deep Scan TV/Movie Library), not just system-automation action types.
- video_scan_server + video_update_database gain a media_type ('all'|'movie'|'show')
  config + selector, threaded through. The post-download chain carries the scope on
  the scan-done event, so a TV-only rescan updates only TV (stage 2 inherits it).
- refresh_video_server_sections / Plex+Jellyfin refresh_sections scope the server
  nudge to the chosen library; auto_video_scan_library now nudges only its library.
- shared normalize_media_type() in sources; update_database skips cleanly when the
  singleton scanner is busy. Defaults stay 'all' so existing chains are unchanged.

Seam tests for refresh scoping, scan-server scope+event, update-db scope/inherit/skip.
2026-06-21 13:04:39 -07:00
BoulderBadgeDad
cf47032660 Video deep scans: fixed weekly times (TV Mon 2am, Movies Tue 2am)
Switch the two deep-scan system automations from a rolling 7-day interval to
weekly_time at 02:00 server-local — TV Mondays, Movies Tuesdays. Different days
means they never overlap, and a fixed wall-clock time doesn't drift with restarts.
Drop initial_delay (the seeder arms timed system triggers). _fix_deep_scan_schedules
migrates the original interval rows to the weekly schedule (the seeder only creates
rows, never updates a drifted trigger); it skips once trigger_type is weekly_time so
a hand-tuned day/time sticks. Idempotent.
2026-06-21 12:39:53 -07:00
BoulderBadgeDad
b6320d1a30 Video: two system deep-scan automations (Movie + TV), independently scoped
Video twin of music's 'Auto-Deep Scan Library', split in two because Movies and TV
are separate libraries — scanning the TV library must not pull in new movies and
vice-versa.

- scanner: add a media_type param ('all'|'movie'|'show', friendly aliases) that
  gates the movies vs shows passes (and their pruning), plus an in_progress busy
  guard so the singleton scanner can't be stomped by an overlapping run.
- video_scan_library handler: thread media_type through, skip cleanly when the
  scanner is busy, and name only the scanned library in the summary.
- two system automations (owned_by=video, weekly deep scan, staggered start delays):
  'Auto-Deep Scan Movie Library' + 'Auto-Deep Scan TV Library'. Distinct action
  types (video_deep_scan_movies / _tv) because the seeder keys on action_type; both
  reuse the one handler, scoped via action_config.
- builder block gains a Library selector (Movies+TV / Movies / TV) so custom scans
  can scope too; card label/icon maps cover the video action types.

Seam tests for scanner scope + busy guard, handler scope + skip, registration set.
2026-06-21 12:14:46 -07:00
BoulderBadgeDad
36f944be25 Video automations: make the builder sidebar scrollable like the music side
The Automations page reuses the music builder, whose .automations-builder-view is
height:100% so its trigger/action sidebar + canvas scroll independently. On music it
fills #automations-page (a .page at height:100%); on video it sat in a .video-subpage
with no height, so height:100% collapsed to content height and the sidebar grew
instead of scrolling (looked unformatted). #video-page-host is itself a .page, so
give just the automations subpage a definite height to resolve the chain. Scoped to
automations so every other video page keeps its natural document-flow scroll.
2026-06-21 11:55:42 -07:00
BoulderBadgeDad
0514931140 Auto-wishlist airing: run at a fixed daily 1am, not a rolling 24h interval
The job shipped as a 24h 'schedule' because the system-automation seeder only armed
next_run for interval specs — a 'daily_time' spec sat idle and never fired. The
interval fired reliably but drifted with every restart (5min after startup, then
+24h) instead of a fixed wall-clock time, which is worse for 'today's airings' (you
want it queued overnight).

Fix, the robust way:
- Seeder now arms timed system triggers (daily/weekly/monthly) via next_run_at, not
  just interval ones. Event-based triggers still return None and are left alone.
- Spec -> daily_time {time:'01:00'} for fresh installs.
- _fix_airing_automation_schedule migrates the existing 24h-interval row to daily
  01:00 (the seeder only creates rows, never updates a drifted trigger). Idempotent.

_finish_run already reschedules daily_time to the next 1am, so it stays pinned.
2026-06-21 11:41:34 -07:00
BoulderBadgeDad
8053bf339c Video wishlist + watchlist: restyle toolbar controls to the video-side standard
The Movies/TV/YouTube (and Shows/People/Channels) tabs, search bar, sort select and
clear-all read as generic dark glass. Align them to the video side's polished
language: selected tab now lights up with an accent outline + ring glow (the same
focus treatment as the search field) instead of a filled accent block; search is a
focus-ring shell with an accent icon; sort drops the native OS arrow for a custom
chevron; every control shares one 42px height + 12px radius + accent-ring focus.
Same treatment applied to the watchlist page so the two match.
2026-06-21 11:05:28 -07:00
BoulderBadgeDad
c35f56bffa Video auto-wishlist: store the show poster too (last field the manual add had)
Field-by-field against the working manual 'add to wishlist', the automation now
matches it on every column EXCEPT the show poster: the get-modal stores
poster_url = '/api/video/poster/show/<library_id>', the automation stored None — so
the wishlist orb fell back to the show's initials and read as 'not matched'. Carry
the same proxy path. With library_id (last commit) + poster_url (this) + the
tmdb_season stills/overviews, an auto-added row is now identical to a manual one.
2026-06-21 10:27:01 -07:00
BoulderBadgeDad
94e06f2d50 Video auto-wishlist: store the show library_id (fixes 'show not matched')
The real difference from a manual add: the wishlist resolves a show's synopsis +
cast from /api/video/detail/show/<library_id>, and falls back to the TMDB endpoint
only when library_id is absent (which redirects/lacks cast for owned shows). A
manual add sends show.library_id; the automation sent none — so auto-added shows
read as 'not matched' with no synopsis/actors. The handler now carries the show's
library id (the calendar's show_id) through to the wishlist.
2026-06-21 10:11:13 -07:00
BoulderBadgeDad
d925c34ce5 Video auto-wishlist: schedule the automation so it actually fires
The system automation used trigger_type 'daily_time' with no initial_delay, but
the seeder only arms a next_run when a spec has initial_delay (and
_calc_delay_seconds doesn't parse a daily_time clock anyway) — so it registered
as 'event-based' and never auto-ran; it only fired when triggered by hand.
Switched to the proven scheduled pattern (24h 'schedule' + initial_delay, like
Auto-Scan Watchlist) so it runs once a day on its own.
2026-06-21 09:53:28 -07:00
BoulderBadgeDad
ae68750d9e Video auto-wishlist: pull episode metadata from TMDB, like a manual add
Root cause of the metadata loss: a MANUAL 'add to wishlist' gets its episode
data from the TMDB season fetch (engine.tmdb_season — absolute still, overview,
season poster), while the automation read the local DB episodes table, where
stills are frequently empty/Plex-relative. So auto-added episodes came in blank
even after carrying the DB values.

The handler now fetches the SAME TMDB season metadata (cached per season,
injected for tests) and prefers it, falling back to the calendar/DB values if
TMDB is unavailable. Auto-added episodes now match manual ones.
2026-06-21 09:41:12 -07:00
BoulderBadgeDad
cc2fb2ff79 Video auto-wishlist: carry episode synopsis + still into the wishlist
Auto-added airing episodes came in metadata-empty (no synopsis, no still) — the
handler only passed season/episode/title/air_date, dropping the overview the
calendar already returns and never fetching the still URL (calendar_upcoming
only returned a has_still flag, not the URL). Now calendar_upcoming also returns
e.still_url, and the handler carries overview + still_url through. The wishlist
renders the (Plex-relative) still via the same pimg() proxy as the show poster,
so it resolves. Idempotent upsert backfills the already-added empty rows on the
next run.
2026-06-21 09:20:27 -07:00
BoulderBadgeDad
fd66390648 Video automations: reliably delete the obsolete 'Scan Video Library' seed
_fix_video_scan_default set its 'done' flag even on runs where it deleted
nothing, so once the flag latched True the standalone 'Scan Video Library'
system automation survived forever (the row the post-download chain replaced).
Drop the flag entirely — get_system_automation_by_action already matches only
the is_system-seeded row, so the cleanup is safe to run every startup and
no-ops once the row is gone.
2026-06-21 08:59:32 -07:00
BoulderBadgeDad
5711963a6f Video watchlist: strip dirty titles when sorting (leading-space fix)
'Holy Marvels with Dennis Quaid' is stored with a leading space in the shows
table, so the watchlist sort key fell to ' holy marvels…' — and a leading space
sorts before 'a', jumping it to the top. .strip() the sort key so dirty titles
sort by their real first letter.
2026-06-21 08:52:49 -07:00
BoulderBadgeDad
49714a59ea Video watchlist: sort alphabetically by name by default
The default sort put manual follows first (newest date_added), then airing
shows A–Z — so recently-followed shows like 'Welcome to Widows Bay' and 'From'
jumped to the top. A manual follow is no more special than an auto-added airing
show; default now sorts everything by name A–Z. 'added' (newest first) stays as
an opt-in sort.
2026-06-21 08:47:59 -07:00
BoulderBadgeDad
01c101d24a Video: live per-episode download tracking + auto-wishlist today's airings
Two Sonarr-parity features.

1) Per-episode live tracking. "Grab season" was headless (only a button label
   changed); episode rows had a status span that was never populated. Now every
   episode ROW shows its own live state — Searching → Downloading % → Downloaded
   / Failed — via epTrack() polling /downloads/status?id, matching the inline
   movie tracker. Grab season lights all target rows at once; manual + per-source
   auto grabs also light their row; reopening the modal resumes tracking in-flight
   episodes (resumeEpisodeTracking via /downloads/active + search_ctx match).
   Season batch grabs through the same payload as a manual grab (_pickAndGrab →
   sendGrab(buildGrabPayload)).

2) Auto-wishlist airing episodes. New daily automation (video_add_airing_episodes):
   reads the calendar for episodes airing TODAY for followed shows, skips owned
   ones, adds the rest to the wishlist (idempotent). Handler uses injected seams
   (calendar read + wishlist write) so it's unit-tested without a DB/server.
   Registered + action block + seeded as a daily system automation (01:00),
   owned_by=video.
2026-06-21 02:27:08 -07:00
BoulderBadgeDad
a52dda6a7f Video quality: recognize plain WEB + accept resolution-only releases
Two over-rejections that filtered out legit releases as "unknown quality":
- the source parser only matched WEB-DL/WEBDL, not plain "WEB" (very common)
- a release with a known resolution but no recognized source had no tier

Now plain WEB parses as web-dl, and a resolution-only release assumes web so
it lands on a tier instead of being rejected (ffprobe verifies the real quality
after download). Truly quality-less packs still reject. Tests added.
2026-06-21 02:11:40 -07:00
BoulderBadgeDad
efa64db04d Video downloads: best-in-class post-process pipeline
End-to-end import for video grabs, mirroring the music side's rigor and the
Radarr/Sonarr standard, fully isolated in core/video + api/video.

- Importer: parse release -> ffprobe-verify (true resolution, reject corrupt/
  samples) -> templated rename into Movie (Year)/ + Show/Season NN/ -> copy or
  move, carry subtitles, upgrade-replace a worse copy.
- Library Organization settings: editable $token path templates + toggles
  (transfer mode, verify, replace, carry subs, save artwork, write NFO,
  download subtitles + langs). Stored in video.db; matches the music File
  Organization section's look.
- Sidecar writer: movie.nfo / tvshow.nfo + full artwork set (poster, fanart,
  clearlogo, season posters) from on-demand TMDB detail, and external .srt from
  OpenSubtitles. Owned re-grabs resolve their library tmdb_id; tmdb_full_detail
  bypasses the owned->library redirect so they enrich too.
- Import page: surfaces import_failed downloads, resolve by hand (library-first
  -> TMDB picker -> force-place) or dismiss; fires a library refresh on place.
- "Grab whole season": episode-level batch (reuses searchInto + _autoPick).
- Brutalist redesign of the download modal sources + result cards.

All new logic has seam-level tests (pure parsers/planners + injected I/O);
sidecars/subtitles are best-effort and never break an import.
2026-06-21 01:44:35 -07:00
BoulderBadgeDad
366b5a266d video download modal: cinematic redesign of sources + search results
Reworked the whole sources/results area for a more premium, Netflix-y feel.

Results — flat divider list → real CARDS: a bold cinematic quality badge (resolution
over source), the release name as the hero, the meta as a row of crisp PILLS (codec /
audio / HDR / repack / group) plus a clean availability stat, then size · verdict · Get
grouped on the right. Cards have depth, a hover-lift, an accent edge on accepted
releases, and the auto/grabbed pick lights up with an accent ring + glow.

Sources — toned the full per-source colour wash down to a sophisticated dark-glass row
with the brand colour as an ACCENT only (left-edge light that extends on hover + the
glassy icon tile), so it reads calm and designed instead of busy.

All functional hooks preserved (data-vdl-card/grab, .vdl-res-main tracker dock, status
states, auto-pick). node --check + CSS balance clean; tracking/auto wiring tests green.
2026-06-20 16:57:09 -07:00
BoulderBadgeDad
9bb6c4ebd0 video wishlist: add a 'Clear all' button (movies / TV / YouTube)
The wishlist page had no way to empty a tab — only per-item remove. Added a red-tinted
'Clear all' button in the toolbar that empties the ACTIVE tab in one click (after a
confirm), shown only when that tab has items.

- db.clear_wishlist(kind) maps the tab to its rows (movie→'movie', show→'episode',
  youtube→'video') and deletes them; returns the count.
- POST /api/video/wishlist/clear {kind: movie|show|youtube}.
- Toolbar button + clearAll() (confirm → clear → reload) + updateClearBtn() visibility.

Tests: per-tab clear leaves the others intact, unknown-kind/empty no-ops, the endpoint,
and the frontend wiring. node --check clean.
2026-06-20 16:42:57 -07:00
BoulderBadgeDad
21e1944784 video scan: only FULL resets enrichment; incremental/deep preserve it
A scan wrote server-provided fields straight over the row, so an incremental/deep
re-read wiped the TMDB-backfilled 'status' (Plex returns it blank) — clearing the
airing watchlist. Now matches the intended model: incremental (add recent) and deep
(coverage + prune) PRESERVE enrichment-owned fields the server left blank; only a FULL
scan clobbers them (an explicit reset / fresh start).

- _resilient_upsert gains preserve_enrichment (default True): on a conflict UPDATE,
  enrichment-owned columns (per _ENRICH_META_COLS: status/network/ratings/air dates/…)
  take the server value only when non-blank, else keep what's stored. A real server
  value still wins.
- upsert_movie/upsert_show_tree thread the flag; scanner passes preserve=(mode!='full').

Tests: preserve-on-blank, server-value-wins, full-resets, and the scanner picking the
right mode. 88 DB + 18 scanner tests + isolation green.
2026-06-20 15:06:44 -07:00
BoulderBadgeDad
bdcd929558 video automations: remove the standalone 'Scan Video Library' system automation
Superseded by the post-download chain (Auto-Scan Video After Downloads → Auto-Update
Video Database After Scan), which keeps the library fresh without a separate scheduled
scan. Two small changes: drop it from SYSTEM_AUTOMATIONS (no longer seeded), and a
one-time flag-guarded cleanup (_fix_video_scan_default, v3) that DELETES the existing
system row — so it actually stops running, not just hidden. The video_scan_library
action/handler/block stay, so a custom scan automation can still be built later.
Engine seed test updated. 66 automation tests pass.
2026-06-20 14:55:34 -07:00
BoulderBadgeDad
91eae710b4 video automations: post-download scan chain (parity with music's batch→scan→update)
Mirrors the music flow: a finished download batch → refresh the media server → (after
it indexes) pull the new media into the DB — so a downloaded movie/episode shows as
owned without waiting for the 6h scheduled scan.

Two event-based system automations (owned_by='video'):
  - 'Auto-Scan Video After Downloads' (video_batch_complete → video_scan_server)
  - 'Auto-Update Video Database After Scan' (video_library_scan_completed → video_update_database)

Pieces:
- core/video/download_events.py: a callback registry (core/video can't import the
  engine — isolation). The monitor publishes batch-complete; web_server bridges it to
  automation_engine.emit('video_batch_complete', …), like music's web_scan_manager.
- download_monitor: fires the batch-complete event once, when the last in-flight
  download finishes (none queued/downloading/searching left).
- video_scan_server handler (stage 1): refresh server, wait a debounce for indexing,
  then emit 'video_library_scan_completed' (mirrors music's time-based completion).
- video_update_database handler (stage 2): incremental read (newest-first, stop after
  25 consecutive known — same as music).
- blocks: 2 video triggers + 2 video actions (scope='video'); registration; seeds.

kettui: seam tests for both handlers (refresh/wait/emit, incremental read, error paths),
the event registry (idempotent register, isolated failures), and the monitor (fires once
on last completion, never while work remains). 298 automation tests + isolation green.
2026-06-20 14:47:00 -07:00
BoulderBadgeDad
297709baa4 video scan automation: incremental + schedule-only (no startup-proximate full scan)
The scheduled 'Scan Video Library' defaulted to a FULL scan (re-reads everything +
prunes) that fired ~5 min after every app start. A recurring scan should be light:
switched the seed to INCREMENTAL (newest-only, no prune) and pushed the first run a
full interval out so it runs on its 6h cadence rather than right after startup.
One-time migration (_fix_video_scan_default, flag-guarded) corrects existing rows that
still carry the old full+startup default, without overriding a deliberate user choice.
2026-06-20 14:34:14 -07:00
BoulderBadgeDad
bb1074cc60 video enrichment: add a GLOBAL 'Retry all failed' (all workers, all kinds)
Alongside the per-worker 'Retry all failed', the worker modal now has a topbar
'Retry all failed' that re-queues every failed/not_found item across ALL workers and
kinds in one click — one-shot recovery after an API outage left lots errored.

- db.retry_all_failed() derives the full service+kind set from the same _ENRICH /
  _BACKFILL maps the workers use (tmdb/tvdb + omdb + fanart/opensubtitles/trakt/
  tvmaze/anilist/wikidata + ryd/sponsorblock/dearrow), loops enrichment_retry, returns
  the total re-queued. POST /api/video/enrichment/retry-all-failed.
- Topbar button (amber, text) → calls it, toasts the count, refreshes the modal.

DB test (resets across matcher + backfill + youtube service, deterministic count) +
frontend wiring test. ruff + node --check clean.
2026-06-20 14:15:18 -07:00
BoulderBadgeDad
49222dd0b8 video calendar: watchlist-driven by default + an 'All library' toggle
The calendar pulled every airing show in the library regardless of whether you
follow it. Now it's scoped to the EFFECTIVE watchlist by default — explicit show
follows ∪ airing library shows (not muted), same logic as the Shows watchlist tab —
so it tracks what you actually care about, and you can mute a show off it. A
'Watchlist / All library' toggle on the calendar lets you flip to everything you
own (remembered in localStorage).

- calendar_upcoming(watchlist_only=) adds the watchlist filter; /calendar takes
  ?scope=watchlist|all (default watchlist).
- Calendar page gets a scope toggle (defaults watchlist, persists, refetches on
  change).

Tests: DB scope (followed-only / airing-default / mute drops out / all-library sees
all) + frontend wiring. node --check clean.
2026-06-20 14:07:40 -07:00
BoulderBadgeDad
7061001f66 video: add 'Add to Watchlist' button to person detail pages
Person follows are already supported (video_watchlist kind='person', add/remove/check
API, and the button shows on person CARDS) — but the person DETAIL page had no way to
follow or see if a person is followed. Added the standard watchlist button to the
person hero: renderWatchlist() builds it + lazily checks the followed state,
toggleWatch() adds/removes via the person-kind watchlist API, wired through the page's
delegated click handler. Same chrome as the movie/show pages.

4 wiring tests; node --check clean.
2026-06-20 13:50:34 -07:00
BoulderBadgeDad
b23a3e91d8 video scan: show a 'cleaning up' phase during the deep-scan prune
On a deep scan the progress bar hits 100% when the last item is read, but the prune
(delete orphaned rows + cascades) runs AFTER that and — on a big cleanup — takes a
few seconds, during which the scan still reads as running (can't start a new one,
workers still paused). Looked stuck at 100%. Now the scanner sets a 'cleaning up
removed movies/shows' phase around the prune so the UI shows it's finalizing, not
frozen. Test spies the phase at prune time. 18 scanner tests pass.
2026-06-20 13:44:35 -07:00
BoulderBadgeDad
7cd289e7b5 video scan: harden library scope + pause ALL enrichers during any scan
1) Scan only the MAPPED libraries — never fall back to 'all'. The scan path used
   _sections/_views with the selected name, but an empty name returned ALL sections
   of that type — so a missing/unreadable selection silently scanned every library
   (how the 4K movie + 'YouTube' TV libraries leaked in as movies/shows). New
   _scan_sections / _scan_views return [] when a kind isn't mapped; available_libraries
   still lists all (for the Settings dropdown). Now an unmapped kind scans NOTHING.

2) A library scan (full/incremental/deep) now pauses EVERY enricher, including the
   YouTube date enricher — a separate singleton outside engine.workers that kept
   running through scans. pause_for_scan/resume_after_scan pause+resume it too, only
   if it wasn't already manually paused (never override the user).

kettui: source-scope tests (mapped-only / unmapped-scans-nothing / listing still
shows all) + engine pause tests (pauses+resumes YT / preserves manual pause).
123 scanner/source/enrichment tests + isolation guard green.
2026-06-20 13:24:57 -07:00
BoulderBadgeDad
8349fa78dd video enrichment: fix DeArrow retry (was a silent no-op)
enrichment_retry handled ryd/sponsorblock + the keyed backfills, but DeArrow (also a
youtube_video_stats backfill, keyed on dearrow_status) fell through to nothing — so the
modal's Retry button did nothing for DeArrow. Fold dearrow into the youtube_video_stats
retry branch. Regression test re-queues failed dearrow rows, leaves matched ones.
2026-06-20 12:52:06 -07:00
BoulderBadgeDad
0eac0ea46e video Manage Workers modal: decouple coverage click from global priority; Retry-all covers all kinds
Two UX fixes from feedback:
- Clicking a coverage card on a worker was also setting the GLOBAL 'process first'
  priority (and silently re-queuing that kind's failed items) — so picking a worker's
  coverage reached across and re-prioritised every worker. Now a coverage card just
  switches the view; priority changes only via the top Movies/Shows/Auto tabs, and
  re-queuing is the explicit 'Retry all failed' button. (Removed the now-dead
  requeueFailed helper.)
- 'Retry all failed' now re-queues EVERY coverage kind the worker handles (movie+show,
  etc.), not just the tab you're viewing — matching the 'all' in the label.

3 wiring tests; node --check clean.
2026-06-20 12:50:18 -07:00
BoulderBadgeDad
d6776f9f57 video enrichment: detail backfill is TMDB-only (fix TVDB 404 spam + double-processing)
The details backfill queue is keyed on tmdb_id, but the gate (hasattr match) let the
TVDB worker run it too — feeding TMDB ids to TVDB's /series/{id}/extended (→ 404 on
every show) and double-processing each show (TMDB backfilled it, TVDB then 404'd but
still logged 'Backfilled'). Gate on self.service=='tmdb' so only the TMDB worker runs
it. Regression test: the TVDB worker no-ops (never calls its client, leaves the item
pending). 95 enrichment tests pass.
2026-06-20 12:38:03 -07:00
BoulderBadgeDad
57f254acaf video enrichment: background TMDB details backfill (fills 'status' on pre-matched items)
The media server pre-matches shows/movies (tmdb_id set), so the enrichment matcher
skips them and never fetches TMDB details — leaving details-only fields like `status`
(airing vs ended) blank on almost the whole library. That's why the watchlist's
airing-shows default only ever saw the handful of shows whose detail page had been
opened (the one path that force-fetches details). Library here: 3,371 matched shows,
only 18 with status.

Fix: a one-time-per-item details backfill that runs in the enrichment worker's idle
loop (after the episode-sync pass). New `details_synced` marker column on shows+movies;
detail_backfill_next/mark_details_synced/pending_count; worker._detail_backfill_one()
re-fetches an already-matched item's TMDB details and gap-fills (never clobbers server
data), then marks it done so it's attempted once. No re-scan needed — it heals the
existing library in place, and once status is populated the airing-watchlist reflects
real TV.

It's a background gap-fill on already-matched items (like episode coverage), so it
doesn't block the worker's 'Complete' status. kettui: DB seam tests + worker tests
(fills status / enrich-by-id / marks-done-when-absent). ruff + isolation guards green;
94 enrichment tests pass.
2026-06-20 12:33:45 -07:00
BoulderBadgeDad
9a3ca6ba4e video downloads: header 'Auto' picks ONE best across all sources; fix YT playlist leak
Auto: replaced 'Manual all' + 'Auto all' (which fired auto on every source = up to
one download PER source = duplicate copies) with a single header 'Auto' button. It
searches every source, waits for them all to settle, compares the accepted+grabbable
hits across ALL sources by quality-profile score (tie-break on availability), and
grabs exactly ONE winner — the chosen row gets the auto ring + live tracker. Per-source
Manual/Auto buttons are unchanged.

Bug fix: viewing a YouTube channel populated the playlist section in the SHARED
show-detail DOM; opening a real movie/show afterward still showed those playlists,
because ytResetPlaylists() used the kind-scoped q() (pointing at the wrong root on a
movie load) and wasn't called on normal loads. Now it targets the show subpage
directly and runs from resetExtras() (every detail load), so stale playlists are
always cleared.

node --check clean; 20 wiring/regression tests green; all video-only.
2026-06-20 10:48:55 -07:00
BoulderBadgeDad
f6f5561d0b video downloads: resume tracking on modal reopen + entirely new result-list design
Two fixes from feedback:

1) Reopening the download modal now KNOWS a download is already running. The view
   gets a persistent active-download banner at the top that looks the title up by
   media identity (/downloads/status?media_id=) on every open and polls while active
   — progress bar + release name + 'Track on Downloads ↗'. Suppressed while a result
   card is already tracking inline (fresh-grab case) so there's never a double
   indicator.

2) Result cards completely redesigned (third time's the charm): dropped the rounded
   cards + big resolution tile for a flat, release-list layout (Radarr/Prowlarr
   style) — hairline dividers, a small colour quality tag + source word on the left,
   the RELEASE NAME as the hero line, dense inline meta (codec · audio · HDR ·
   uploader · group) under it, then size · a compact ✓/✕ verdict flag · a compact
   accent 'Get' pill. The selected/auto/grabbed row tints + rings in place, and the
   live tracker still docks under the chosen row.

All video-only. node --check clean; 16 tracking/auto wiring tests + the status
endpoint test green.
2026-06-20 09:51:43 -07:00
BoulderBadgeDad
7e504e03c6 video downloads: live tracking on the grabbed result + movie detail, redesigned cards
After a grab (manual or Auto) the user now SEES what happened and can follow it:
- The chosen result card is spotlighted (Auto scrolls it into view) and grows a live
  tracker: a progress bar that follows the real download + a 'Track on Downloads ↗'
  button that closes the modal and jumps to the Downloads page. Polls the new
  GET /api/video/downloads/status?id= until the download reaches a terminal state.
- A movie's detail page shows a live download chip (progress bar + %) for any in-flight
  download of that title; clicking it jumps to Downloads. Looks up by media identity
  via /downloads/status?media_id=&media_source=, polls while active, clears on
  navigate-away. (video-detail.js)
- Result CARDS redesigned (the part you didn't like): a column card with a colour
  resolution badge, a green 'accepted' edge, cleaner hierarchy, and the grab button is
  now an accent 'Get' pill matching the source Auto button language.

Plumbing: new lightweight /downloads/status endpoint (by id, or by media for detail
pages); soulsync:video-navigate event in video-side.js to reach a top-level page from
anywhere; VideoGet.close exported so the tracker can dismiss the modal. All video-only.

Tests: status endpoint (by id + by media + null cases) in test_video_api.py;
tracking/detail/nav wiring in test_video_download_tracking.py. node --check clean;
isolation guards green.
2026-06-20 09:41:05 -07:00
BoulderBadgeDad
afe3d0d04d video download modal: redesign the Sources area + Manual/Auto buttons
The sources half felt weak next to the animated top half (quality chips + glowing
verdict), and the dark-text lightning Auto button looked off. Redesigned the whole
sources block:

- Buttons are now a cohesive pair: Manual = quiet ghost (outline), Auto = hero —
  brand-filled gradient, white text w/ shadow for legibility on bright brands, a
  soft continuous brand glow (vdlAutoGlow, --glow set per element) + a sheen sweep
  and a sparkle twinkle on hover. Swapped the harsh  for a clean monochrome ✦
  that inherits the button colour. 'Manual all'/'Auto all' header buttons speak the
  same language (ghost vs accent hero).
- Source rows: richer brand card — bigger glassy icon tile w/ inner highlight +
  halo, a stronger brand gradient, an inset top highlight, a glowing left brand
  edge (::before), and the status is now a brand-tinted pill (was bare dot+text)
  with state colours (scanning/done/none). Scan bar moved to ::after.
- Section labels get a small accent tick so both halves read 'designed'.
- Reduced-motion + mobile (full-width stacked buttons) handled.

Icon+label split into spans for finer control. node --check clean; 7 tests
updated/green.
2026-06-20 09:18:04 -07:00
BoulderBadgeDad
55658f15da video download modal: per-source 'Auto' button (search + grab the best release)
Each source in the movie/YouTube download view had one 'Search' button (manual —
you pick a release). Adds a second 'Auto' button beside it that runs the SAME
search and then auto-grabs the best release for your quality profile; renamed the
pair to 'Manual' / 'Auto' for clarity (+ a matching 'Auto all' beside 'Manual all').

How 'best' is chosen: the backend already returns hits ranked best-first
(accepted → score → availability — see test_downloads_search_endpoint_ranks_and_filters),
so Auto just waits for the search to settle, then takes the first accepted hit
that has an uploader and grabs it. The chosen release card gets a ring + the row
shows Auto-grabbing → Sent, so the pick is transparent.

- searchInto/_pollSearch gain an onDone callback (fires when results settle); the
  immediate (mock) path fires it too.
- doGrab refactored into shared buildGrabPayload + sendGrab so the manual button
  and _autoPick send an identical /grab request (incl. the auto-retry candidate pool).
- searchInto now drops stale _rows on start so an empty Auto search can't grab a
  prior search's hit.

Soulseek-grab-only for now (same as the manual button); non-soulseek sources say
'no release met your profile' until that grab path lands. TV show view (separate
onShowClick, still stub searches) untouched. 7 wiring tests; node --check clean.
2026-06-20 00:03:11 -07:00
BoulderBadgeDad
1fa3e1b599 video automations: working builder + New Automation button (own builder, not music's)
The video automations page had no way to CREATE automations and its card cog did
nothing — it called the music global showAutomationBuilder(), which swaps views
inside the (hidden-on-video) music page, so the builder 'opened' on the music tab
instead. Now the video side has its own builder.

- index.html: the video automations subpage gets its own list-view + builder-view
  (vauto- prefixed ids) and a '+ New Automation' button, swapping exactly like the
  music page. Save/Cancel/Back reuse the shared builder functions.
- stats-automations.js: the builder is now context-aware. A builder context holds
  the element ids + blocks endpoint + owned_by + reload callback. Music context is
  the default and byte-identical (all 17 id lookups go through _bEl() resolving the
  music ids). showVideoAutomationBuilder() sets a video context (vauto- ids,
  /api/video/automations/blocks, owned_by='video'); editAutomation() routes the
  card cog to the right builder by active side. Opening clears BOTH builders'
  canvases so cfg-* ids can't collide. Save tags owned_by from context and calls
  the context's reload. A generic config_fields renderer/reader (video-gated so
  music keeps its bespoke renderers) drives video block config like the mode select.
- video-automations.js: exposes window._reloadVideoAutomations so a save refreshes
  the video list.

11 wiring tests (test_video_automations_builder.py); node --check clean; music
builder path unchanged.
2026-06-19 23:02:52 -07:00
BoulderBadgeDad
5d488bceb4 tests: isolate the VIDEO database too (never open the real video_library.db)
conftest already redirected the MUSIC DB to a tmp path after test writes once
corrupted a user's music library (WSL/NTFS + WAL). The VIDEO side had the SAME
hazard but no guard: VideoDatabase()/get_video_db() with no path resolve from
VIDEO_DATABASE_PATH, defaulting to the real database/video_library.db, and its
enrichment threads WRITE — a default-path open during tests corrupted the real
video library (one table's btree pages; recovered via row-by-row salvage).

Fix: set VIDEO_DATABASE_PATH to the same throwaway tmp dir at conftest import,
before anything loads. Adds guard tests proving VideoDatabase()/get_video_db()
never resolve to database/video_library.db (mirrors the music guards).
2026-06-19 23:02:38 -07:00
BoulderBadgeDad
40149d09f7 video automations: 'Scan Video Library' — the first video twin (shared engine)
The video side gets its OWN automations at music-side parity, kept separate so
nothing on the music side breaks. First twin: Scan Video Library — tells the media
server to rescan the user's SELECTED video sections (movies/TV, never music), then
reads the result into video.db so freshly-downloaded media shows as owned.

Architecture (scope tags + video twins on the shared engine):
- Handler core/automation/handlers/video_scan_library.py — pure function with
  injected I/O (server_refresh / run_video_scan); production lazily binds
  refresh_video_server_sections() + the video scanner. Owns its own progress.
  Lives on the SHARED automation side so it may import core.video (isolation only
  forbids core/video & api/video from importing music, not the reverse).
- blocks.py gains a 'scope' tag ('both' generic / 'video' video-only / absent=music)
  + blocks_for_scope(). The music /api/automations/blocks now filters out video
  blocks; new isolated /api/video/automations/blocks serves the video palette.
- automation_engine seeds 'Scan Video Library' (owned_by='video', schedule 6h) so
  it appears ONLY on the video Automations page; ensure_system_automations now
  honours owned_by + action_config. Music page excludes owned_by='video' rows.

kettui: seam-level tests for every handler path (happy/no-server/scan-error/never-
raises/mode), scope filtering (music excludes video, video gets generics, music
parity preserved), seeding (owned_by + mode), registration drift guard. 39 new
tests; full automation suite (288) + isolation guards green.
2026-06-19 19:38:18 -07:00
BoulderBadgeDad
13b30a5997 video sources: add refresh_sections() — tell the server to rescan VIDEO sections
The foundation for the 'Scan Video Library' automation (the video twin of music's
Scan Library). PlexVideoSource.refresh_sections() triggers a Plex scan on the selected
movie/TV sections (section.update()); JellyfinVideoSource.refresh_sections() POSTs
/Items/{id}/Refresh per selected video view (the GET-only _make_request can't POST).
Module helper refresh_video_server_sections() gets the active source and refreshes —
scoped to the user's chosen video libraries (Settings), so it scans the CORRECT media,
not music. Video-only, additive; isolation guard + (my code) ruff clean.
2026-06-19 19:03:51 -07:00
BoulderBadgeDad
3698ed1ec0 video Automations: hide the music system automations (show only owned_by='video')
Per Boulder — the music system automations target music resources and don't belong on
the video side. The page now filters to owned_by='video' (a tag the upcoming video
automations will carry); none exist yet, so the System section is hidden and the empty
state reads 'Video automations coming soon — separate from the music ones'. The hub +
layout stay (video hub content + video automations come next). Balance clean.
2026-06-19 18:29:02 -07:00
BoulderBadgeDad
483cc61e08 video Automations hub: empty the music-specific tabs (keep Reference)
The Automation Hub's Pipelines / Singles / Quick Start / Tips panes are music content
(playlist pipelines, music recipes/guides). The video side will get its own content
there; for now those four panes are emptied to a 'Video … coming soon' placeholder
(scoped to the detached hub element, no id clash). Reference stays — it's generic
automation reference. Hub tabs + structure otherwise unchanged. Balance clean.
2026-06-19 18:18:31 -07:00
BoulderBadgeDad
71d48f704d video Automations: reuse the music page's own builders for an exact match
Stopped hand-rolling the cards/section and now call the music page's GLOBAL builders
(_buildAutomationSection / renderAutomationCard / _buildAutomationHub from
stats-automations.js) — so the System section, every automation card, AND the
Automation Hub (Pipelines/Singles/Quick Start/Tips/Reference tabs) render byte-for-byte
identical to the music page. The System section uses a unique id (no clash) and is the
only thing re-rendered on refresh; the hub is built once (static). Run/toggle go through
the reused music card handlers; we re-sync the section after. Balance clean.
2026-06-19 18:13:53 -07:00
BoulderBadgeDad
5d8db8c5a4 video Automations page: match the music page's exact layout
Was a bespoke .vauto- header; now mirrors the music automations page structure 1:1 —
.page-shell.automations-container > .dashboard-header (sweep + automation.png icon +
header-title/subtitle) > .automations-stats ('N Active · N System') > .automations-list
holding the protected '.automations-section.section-protected' System group (chevron +
label + count + collapse, persisted) with the same .automation-card rows. Reuses every
music class; driven by video JS via data-vauto-* hooks (no #id clash). Removed the dead
.vauto-* CSS. Balance clean.
2026-06-19 18:05:00 -07:00
BoulderBadgeDad
c78b90826a video: Automations page (shared system automations, video view)
Adds an Automations nav entry + page to the video side. The automation engine is
app-wide, so this surfaces the SAME system automations the music side runs — filtered
to is_system, EXCLUDING Refresh Beatport Cache and any user/playlist-pipeline ones.
Reads the shared /api/automations (no music imports), reuses the music .automation-*
card look, and supports run-now + enable toggle (system automations aren't editable).
Polls every 5s while the page is open. Frontend-only; balance clean.

Next: the library-refresh/scan automation wiring (Boulder has questions on how server
scanning works first).
2026-06-19 18:01:23 -07:00
BoulderBadgeDad
937f3094c6 video Downloads page: stop polling when off-page (incl. music side); 2s cadence
The active-downloads poll kept running in the background — the page-change event that
stops it only fires for video-side navigations, not when switching to the music side,
so /api/video/downloads/active was being hit forever. poll() now bails (and clears the
timer) whenever the Downloads page isn't on screen (data-side!=video or the subpage is
hidden). Also matched the music page's 2s active cadence (6s idle). Still HTTP polling,
same as the music downloads page (which polls /api/downloads/all every 2s — the app's
SocketIO is for other realtime, not the downloads list).
2026-06-19 17:28:45 -07:00
BoulderBadgeDad
3af3a1cd24 video downloads Phase C: auto-retry + alternate-query requery
When a grabbed release fails (transfer error / peer-cancel / never lands), the engine
now retries instead of giving up — the music-style depth:
- Grab stores the OTHER accepted results as a retry pool + the search context (schema
  v16: candidates / search_ctx / tried_queries / tried_files / attempts).
- core/video/retry.py (pure, tested): plan_retry() → try the next-best candidate; when
  the pool is dry, next_query() generates an ALTERNATE query (movie: drop the year; TV:
  numbering variants) to re-search; budget MAX_ATTEMPTS=6. merge_candidates dedupes
  against already-tried releases.
- Monitor: on failure, _fail_or_retry hops to the next candidate inline; if none, flips
  the row to a new 'searching' state and a background requery thread re-searches the
  alternate query, evaluates against the profile, and starts the best fresh hit — or
  marks failed once truly exhausted. 'searching' rows are owned by their thread.
- Page: 'Searching' status (Trying another release…) + a 'Nx' attempt badge.
16 tests (retry engine + candidate-retry transition); ruff clean on touched files.
2026-06-19 17:12:44 -07:00
BoulderBadgeDad
5661ccbfd2 video search: poll the FULL slskd timeout (~60s) + explain audio-only results
The poll capped at 32s but slskd results trickle in over ~50s (the music side waits
the whole search_timeout), so slow searches like 'Project Hail Mary' returned 'none'
before results landed. Now:
- /search/start returns poll_ms (slskd search_timeout + 8s); the UI polls that long
  (capped 80s), streaming results as they arrive, stopping early only once results
  clearly plateau (≥20s + stable) or hit 25.
- /search/poll returns total_files; when 0 video releases but slskd DID return files,
  the panel says 'returned N files, but none are video — likely audio/other for this
  title' instead of a blank 'none' (Soulseek is audio-heavy; many movie titles are
  audiobooks there). slskd_search.poll_search() returns {hits, total_files}.
Tests green, ruff + balance clean.
2026-06-19 16:55:53 -07:00
BoulderBadgeDad
48f6cc5e3c video Downloads cards: poster + movie details + Open-page button
Grabs now carry the movie's identity so the Downloads cards are rich, not anonymous:
- video_downloads gains media_id / media_source / year / poster_url (schema v15 +
  migration); grab stores them (passed from the get-modal → download view → grab).
- Cards show the POSTER in the art tile (emoji fallback), 'Title (Year)', a quality
  chip (1080p · BluRay · X265), and an ↗ Open button that jumps to the movie/show
  detail page (dispatches soulsync:video-open-detail). Cancel/retry unchanged.
16 tests green, ruff + balance clean.
2026-06-19 16:29:27 -07:00
BoulderBadgeDad
1c3f255a66 video Downloads page: actually match the music page (.adl-* layout, full width)
Scrapped the bespoke centered .vdpg- design and reused the music downloads page's
.adl-* classes for real parity: full-width .adl-layout, the segmented .adl-filter-pills,
the title with the accent download glyph, and the compact .adl-row (44px art tile +
.adl-row-info title/meta/error + .adl-row-status dot+label) — driven by the video JS
via data-vdpg-* hooks (no #id clashes with the music page). Per-row cancel reuses the
music hover-reveal .adl-row-cancel; retry mirrors it in accent. Kept the smooth
in-place patching (slim progress line for active rows). Removed the old .vdpg-* block;
balance clean.
2026-06-19 15:47:02 -07:00
BoulderBadgeDad
d25be2c3c6 video downloads Phase B: page redesign — filter tabs, cancel/retry, depth
Brings the Downloads page up toward the music page's depth:
- Filter tabs (music-style pills): All / Active / Completed / Failed, each with a live
  count; clicking filters the list. Cancelled rolls under Failed.
- Header actions: Cancel all (active) + Clear finished, shown only when relevant; a
  live 'N active · N done · N failed' subline.
- Per-row actions: ✕ Cancel on active rows (→ /downloads/cancel), ↻ Retry on
  failed/cancelled rows (→ /downloads/retry, re-grabs the release).
- Cancelled status styling (pill + dimmed row). Still the smooth in-place patching
  (no blink), adaptive polling, empty + filtered-empty states.
Balance clean. Phase C (auto-retry + alternate-query retry) next.
2026-06-19 15:34:50 -07:00
BoulderBadgeDad
0bb77bf782 video downloads Phase A: cancelled state, cancel/retry, monitor robustness
Fixes the 'cancelled but still shows running' stuck bug and adds real depth:
- classify_state now distinguishes 'cancelled' from 'failed'.
- Monitor is robust to slskd forgetting a transfer: if it's gone, it first tries to
  complete from the FILE on disk (survives the music 'Clean Completed Downloads'
  auto-clear), else counts misses and fails the row after ~8 polls instead of hanging
  on 'downloading' forever. Cancelled transfers → cancelled status.
- POST /downloads/cancel (slskd DELETE transfer + mark cancelled) and /downloads/retry
  (re-grab the same release). get_video_download(id) added; clear includes cancelled.
process_download stays pure (fs/slskd injected); 15 tests, ruff + guard clean.
Phase B (page: tabs/queue/history/cancel+retry buttons) next.
2026-06-19 15:31:51 -07:00
BoulderBadgeDad
42c67a6d4d video search: stream slskd results (start + poll) — fixes 'no results'
The old search did a 4.5s slskd search + 8s wait; slskd responses trickle in over
10-30s, so the window closed before results arrived (you'd see them in slskd but the
panel said none). Now it works like the music side:
- slskd_search.start_search() (uses the shared soulseek.search_timeout) + poll_responses().
- POST /downloads/search/start (mock = immediate; soulseek = returns a search id) +
  GET /downloads/search/poll. Shared _evaluate_hits ranks each poll's hits.
- UI streams: starts the search, polls every 1.3s, renders results live with a
  'searching…' badge, stops when results plateau (4 stable polls) or ~32s. Live
  re-renders suppress per-card entrance so it doesn't blink.
Tests green, ruff clean.
2026-06-19 15:23:47 -07:00
BoulderBadgeDad
cb56ee2bee video backfill: log the title (was 'None') — read item['title'] not ['name']
backfill_next returns rows keyed by title/kind/id/imdb_id (no 'name'), so the
'Enriched movie None via Trakt' spam was just the log line reading the wrong key.
Now logs the real title in the 'enriched' line, the fetch-failed line, and the
current-item status. Cosmetic only — the Trakt sweep itself was working correctly
(one-time backfill of the movie catalogue; each row marked once, never loops).
2026-06-19 15:12:02 -07:00
BoulderBadgeDad
7e0726d5eb video Downloads page: smooth in-place updates (no more blink) + polish
The blink was every poll re-running each card's entrance animation via innerHTML
churn. Now cards are created ONCE and PATCHED in place (data-st attribute swaps,
progress width glides over a 1.4s transition, meta only rewrites when its text
changes), so nothing re-animates on a tick — the music-downloads smoothness.

Also: adaptive polling (1.5s while active, 6s idle), border/pill colours TRANSITION
on status change, downloading cards get an accent tint + glow, a relative 'started Xs
ago', and the % moved inline. Balance clean.
2026-06-19 15:10:04 -07:00
BoulderBadgeDad
fff6448a14 video downloads: wire Grab + build the live Downloads page
Phase 3 — the UX:
- Grab button now actually grabs (Soulseek results only — they carry the slskd
  username/filename): POSTs /downloads/grab with the result + search context, shows
  ✓ on success, toasts 'Sent to Downloads', fires a refresh event. Endpoint returns
  size_bytes for the payload.
- New Downloads PAGE (video-downloads-page.js, .vdpg-*): every grab lands here.
  Polls /downloads/active while open and shows live status — type icon (🎬/📺/▶️),
  title + release, a pulsing 'Downloading' pill with a shimmering progress bar, then
  'Completed' with the file's library destination (→ /media/movies/…), or 'Failed'
  with the reason. Clear-finished button, empty state, staggered entrance, vibes.
  Reduced-motion honored. JS/CSS/HTML balance clean (verified w/ a real tokenizer).
2026-06-19 14:59:22 -07:00
BoulderBadgeDad
ceccb4ee65 video downloads: monitor thread + grab/active/clear endpoints
Phase 2 — the engine:
- core/video/download_monitor.py: a daemon thread polls slskd for active video
  downloads, updates progress, and on completion MOVES the file from the shared
  download folder into the per-type library folder + marks it completed. The
  per-download decision (process_download) is pure (fs + slskd injected) — 6 tests.
- POST /downloads/grab: validates (Soulseek-only v1), resolves the target library by
  kind, starts the slskd download, records the row, lazily starts the monitor.
- GET /downloads/active (list + ensure monitor running) · POST /downloads/clear
  (drop finished).
14 tests, isolation guard + ruff clean. Grab button + Downloads page next.
2026-06-19 14:53:18 -07:00
BoulderBadgeDad
9f687c061a video downloads pipeline: data + pure logic foundation
Phase 1 of the real grab→transfer pipeline:
- video.db video_downloads table (kind/title/release/source/username/filename/size/
  target_dir/dest_path/status/progress/error/timestamps) + CRUD (add/list/get_active/
  update/clear_finished). Schema v14.
- core/video/slskd_download.py: start_download (POST /transfers/downloads/{user}),
  list_downloads (GET → flattened), + pure classify_state (completed/failed/active),
  progress_pct, find_transfer.
- core/video/download_pipeline.py: pure find_completed_file (locate by basename in the
  download dir), dest_path_for, target_dir_for (kind → movies/tv/youtube library).

All filesystem/HTTP is injected or thin glue; 7 tests, isolation guard + ruff clean.
Monitor + grab endpoint + Downloads page next.
2026-06-19 14:50:27 -07:00
BoulderBadgeDad
84fac1f80a video downloads: share the INPUT folder with music; add libraries to compose
- Input/download folder is now the SAME shared config key the music side uses
  (soulseek.download_path via config_manager) — change it on either side and both
  follow, one physical download dir, simpler Docker mounts. Output libraries stay
  video-specific in video.db. ZERO music code touched (read/write the shared key only).
- docker-compose: documented the shared ./downloads input mount, and added the three
  video library OUTPUT mounts (./Movies→/media/movies, ./TV→/media/tv,
  ./YouTube→/media/youtube) matching the in-app placeholders.
- Test monkeypatches config_manager: asserts the input folder writes the shared
  soulseek.download_path (music sees it) and is NOT in video.db; libraries persist to
  video.db; legacy migration intact. ruff + guard + balance clean.
2026-06-19 14:40:22 -07:00
BoulderBadgeDad
89bedaf140 video downloads: split transfer folder into Movies / TV / YouTube libraries
One shared download (input) folder feeds three separate library (output) folders —
the engine routes a finished download to the one matching its type, so YouTube never
lands in your real TV library (own Plex/Jellyfin library + agent). Replaces the single
transfer_path with movies_path / tv_path / youtube_path (legacy transfer_path migrates
into Movies on first read). Settings UI: shared Download folder + 🎬 Movies / 📺 TV /
▶️ YouTube library inputs. Tests updated incl. the migration. ruff + balance clean.
2026-06-19 14:32:54 -07:00
BoulderBadgeDad
86fde7d62c video search UI: show slskd peers/username, live vs demo, errors
- Result meta now adapts to the source: Soulseek shows '👤 uploader · N peers · N
  slots' (real slskd availability); torrent/usenet keep '▲ seeders' (still mock).
- Results header shows a '● live' badge for real slskd searches vs an amber 'demo
  data' badge for the still-mocked sources — so it's never ambiguous again.
- slskd errors ('not configured', network) surface as a clear ⚠ line in the panel.
JS/CSS balance clean.
2026-06-19 14:17:45 -07:00
BoulderBadgeDad
305020f61e video search: REAL slskd search for the Soulseek source (was mocked)
The Soulseek ⌕ now actually queries your slskd instance instead of fabricating
results. core/video/slskd_search.py (isolated): build_query(scope,…) → POST
/api/v0/searches (shared soulseek.* URL+key) → poll /responses (bounded ~8s,
early-exit at 25) → keep video files → group_video_files() folds them into one hit
per release folder with a peer count + the fastest available source. Same
{title,size_bytes,…} shape, so parse→evaluate→rank/cards are unchanged.

Endpoint routes source=='soulseek' to slskd (torrent/usenet stay mocked), surfaces
'slskd not configured' / network errors, carries username/peers/slots/filename
through for the cards + a future real Grab, and caps to 40. Pure helpers tested (4);
isolation guard + ruff clean.
2026-06-19 14:16:44 -07:00
BoulderBadgeDad
bf17ef3fab video search: per-source result panels + redesigned, readable cards
- Each source now has its OWN results panel under its row (movie + episode), and
  the search passes the source through — so Soulseek / Torrent / Usenet each show
  their own distinct hits instead of one shared identical list.
- Card redesign for readability: a big colour-coded RESOLUTION tile (4K gold /
  1080p accent / 720p blue / SD) anchors each row; a plain-English quality summary
  leads (e.g. 'BluRay · X265 · DTS-HD' + an HDR/REPACK tag); the raw release name is
  demoted to a single muted truncated line; size · seeders · group sit below; the
  verdict pill (✓ Meets profile / ✕ reason) and ⤓ Grab stay on the right.

Dropped the old break-all mono title + badge-row layout. JS/CSS balance clean.
2026-06-19 13:58:43 -07:00
BoulderBadgeDad
3490092e3d video search: each source returns its own results (was identical)
mock_search now takes the source and gives Soulseek / Torrent / Usenet distinct
hits — different release-group pools, seeder magnitudes, and which qualities show
up (Soulseek drops the top remux, Usenet drops the cam, etc.), the way real
indexers differ. Endpoint forwards body.source. Fixes 'click Torrent shows the
same results as Soulseek'. +1 test.
2026-06-19 13:56:04 -07:00
BoulderBadgeDad
c5bffe51a5 video search UI: real result cards across movie/episode/season/show scopes
Wires the four scopes to the /downloads/search pipeline and renders beautiful,
profile-aware result cards (mock indexer for now):
  - Movie: per-source ⌕ / Search all → movie scope.
  - Episode: expand an episode → per-source ⌕ → episode scope (its own results panel).
  - Season: the season ⌕ → SEASON-PACK scope, results under the season header.
  - Whole show: top-bar 'Search whole show' → complete-SERIES-pack scope.
Each card shows the full release name (mono), colour-coded quality badges
(resolution/source/codec/HDR/audio/repack), size, seeders, and a verdict — green
'✓ Meets profile' or a dimmed '✕ <reason>' (e.g. not in your enabled tiers, wrong
season, over size cap). Accepted hits sort first and carry a ⤓ Grab button (stub).
Spinner while searching; staggered card entrance; reduced-motion honoured.
Replaced the faux-scan stubs. JS/CSS balance clean.
2026-06-19 13:44:50 -07:00
BoulderBadgeDad
44b074772b video search: evaluate/rank pipeline + mock indexer + /downloads/search
Phase 2 of live search (indexer mocked, pipeline real):
- quality_eval.evaluate_release(parsed, profile, scope, want_season/episode, size_gb)
  → {accepted, score, rejected, tier, quality_label}. Sonarr-style: rejects junk
  sources/codecs/3D, requires an ENABLED ladder tier, honours HDR-require + size caps,
  and VALIDATES scope (episode wants SxxExx, season wants the whole-season PACK, show
  wants a complete-series pack, with season/episode-number checks). Scores keepers by
  resolution/source/codec-pref/HDR/audio/repack for ranking.
- mock_search.py: deterministic stand-in indexer returning scope-shaped raw hits
  (the single swap-point for real slskd/Prowlarr).
- POST /downloads/search: mock → parse → evaluate → rank (accepted, score, seeders).

16 tests, isolation guard + ruff clean. UI wiring next.
2026-06-19 13:39:09 -07:00
BoulderBadgeDad
958a5cd0a4 fix: video sidebar highlight needed two clicks (music chrome wiped it)
setActivePageChrome cleared .active from EVERY .nav-button — including the video
sidebar's — but only re-highlighted music buttons ([data-page]). On the first video
nav it ran (via navigateToPage) and wiped the video selection right after the video
side set it; the second click hit navigateToPage's same-page early-return so it
didn't run, which is why the highlight only 'stuck' on the second click.

Scope the clear to .nav-button[data-page] (music nav only). The video side owns its
own .nav-button[data-video-page] highlight via setActiveNav. Zero music-side impact
(all music nav buttons carry data-page).
2026-06-19 13:36:00 -07:00
BoulderBadgeDad
9b1d76b4ff video search: scene/p2p release-title parser (Sonarr-style)
core/video/release_parse.py (pure, isolated): parse_release(title) → resolution,
source (remux/bluray/web-dl/webrip/hdtv/dvd/cam/screener/workprint), codec, HDR,
audio, group, repack/proper/3d, and crucially the SCOPE — single episode (SxxExx)
vs season pack (Sxx / Season N) vs complete-series pack (S01-S05 / COMPLETE). The
search layer uses this to validate that a hit matches what was searched. 9 tests,
isolation guard + ruff clean. First piece of the live search pipeline.
2026-06-19 13:28:37 -07:00
BoulderBadgeDad
8803c1bd0c video download: dedicated TV show view — season/episode picker
Shows now get their own download experience instead of the movie layout. Clicking
Download on a show WIDENS the modal (~680→940px, animated) and renders a tree:

  - Quality target chips + a show summary (N seasons · M episodes · X in library ·
    Y missing) + a master select-all + a primary 'Search N selected'.
  - Season cards (collapsible) with a tri-state season checkbox, 'N eps · M missing',
    and a per-season search button (auto-opens the season to reveal the scan).
  - Episode rows: checkbox, E# · title, In library/Missing/Upcoming badge. Default
    selection = everything you're MISSING (owned shown but unticked, upcoming locked).
  - Each episode expands inline to its branded per-source search strip (Soulseek/
    Torrent/Usenet), so you can grab one episode à la carte; or batch via the season/
    bulk buttons. Searches are the same faux-scan stub — no backend yet.

Library shows use shipped episodes; tmdb shows prefetch seasons via the existing
/tmdb/show/<id>/season/<n> endpoint. get-modal stashes the detail + tvId and routes
shows to VideoDownload.render({kind:'show',…}); movie/youtube paths unchanged.
Vibey: staggered entrance, branded scan bars, smooth collapse. Reduced-motion honored.
JS + CSS balance clean.
2026-06-19 12:06:23 -07:00
BoulderBadgeDad
ecd9c25e3c video download view: drop the footnote + make Sources section pop
- Removed the 'Automatic searching arrives…' footnote.
- Sources rows are no longer tame: each carries its own brand colour (Soulseek
  blue / Torrent amber / Usenet purple / YouTube red) driving a tinted gradient
  row, a left accent rail, a glowing brand icon TILE, and an in-brand hover lift
  with a coloured shadow. A live pulsing status DOT sits by the label, the Search
  button is brand-tinted, and the scan bar/'Searching…' state now glow in the
  source's own colour. Reduced-motion still honoured.

CSS + JS balance clean.
2026-06-19 11:42:43 -07:00
BoulderBadgeDad
ac7b47e4db video download view: take the vibe up a level (staggered + animated)
Pure motion polish on the in-place download view:

  - Staggered entrance — the view's blocks rise in sequence; target chips pop in
    with a spring + a one-shot sheen sweep; source rows slide in cascaded.
  - Quality chips glow on hover in the modal's vibe hue; source rows lift/shift
    with a playful emoji wiggle.
  - Owned verdict badge pops in (spring); 'pending' breathes while checking; the
    'Eligible for upgrade' badge gets a soft amber glow pulse.
  - 'Search all' button pulses to draw the eye + a sheen sweep on hover.
  - Stub search is now a satisfying faux-SCAN: the row shows 'Searching…' with
    animated dots + a moving scan bar, then resolves to 'coming soon' (staggered
    across rows on 'search all', so it ripples). Still no backend.
  - Honors prefers-reduced-motion (keeps layout, drops the motion).

CSS + JS balance clean.
2026-06-19 11:39:19 -07:00
BoulderBadgeDad
2a61ac6a13 video download: render the download view IN-PLACE in the get-modal (no 2nd modal)
Per feedback — one modal with a view transition beats close+open. Clicking
'Download' now swaps the get-modal's detail body for the download view (quality
target + owned verdict + per-source search) with a '← Back to details' button;
the selection-only sections (episodes/next/follow) and the Full page/Download/Add
buttons collapse, the all-related movie bits stay.

  - video-download-modal.js → video-download-view.js: now a reusable RENDERER,
    VideoDownload.render(container, opts), not its own overlay. A future YouTube
    trigger can render the same view into its own container (still universal).
  - get-modal: enterDownload/exitDownload toggle the in-place view; stashes the
    owned file on modalState to feed the verdict. Removed the separate .vdl
    overlay/hero/close CSS; added .vgm-dl/.vgm-back + [hidden] guards; chips now
    pick up the modal's --vgm-h vibe hue.

Pure front-end; balance + div + css brace checks clean.
2026-06-19 11:27:59 -07:00
BoulderBadgeDad
1dc727f0bb video: universal Download modal (movie/show/youtube) — visual scaffolding
Adds a 'Download' action to the get-modal that closes it and opens a new universal
Download modal (VideoDownload.open) shared by movies, shows and YouTube:

  - Quality TARGET shown as chips, read live from the profile you configured
    (cutoff / codec / HDR / rejects / size for p2p; resolution/codec/container/
    60fps/HDR for YouTube).
  - Owned copies get a REAL verdict: 'In your library · 720p · BluRay · X265' +
    a 'Meets your target' / 'Eligible for upgrade' badge with reasons, via
    /downloads/evaluate (the quality_eval seam).
  - Per-source rows (soulseek/torrent/usenet from the download config, or yt-dlp
    for YouTube) each with a Search button + a 'Search all'.

Per the ask, the searches are STUBS — clicking flips the row to 'coming soon'; no
backend yet (that's the engine phase). Beautiful .vdl-* styling mirrors the get-modal
vibe (per-title hue glow). New JS file wired into index.html. Pure front-end.
2026-06-19 11:06:11 -07:00
BoulderBadgeDad
e880be9910 video downloads: quality evaluation seam (owned-copy-vs-profile verdict)
The shared judge both the Download modal and the later-phase engine will use:
core/video/quality_eval.py (pure, isolated) — resolution_rank/resolution_label,
meets_cutoff (loose resolution target), and evaluate_owned(file, profile) →
{meets, resolution_label, reasons[]}. A copy is 'below target' when its resolution
is under the loose cutoff, or its codec is on the reject list.

Exposed as POST /api/video/downloads/evaluate (loads the stored profile, judges the
posted file). 9 tests green, isolation guard green, ruff clean.
2026-06-19 11:01:03 -07:00
BoulderBadgeDad
ddde6cba68 video downloads: separate YouTube quality profile (yt-dlp)
Boulder: 'youtube should have its own quality profile — there are not many options.'
Right — YouTube is grabbed with yt-dlp, not scene/p2p releases, so the Radarr-style
ladder/cutoff/rejects/HDR-audio tiers are meaningless. Added a small, separate profile:

  - core/video/youtube_quality.py (pure, isolated): max_resolution ceiling
    (best…360p), video_codec (any/av1/vp9/h264, soft), container (mp4/mkv/webm),
    prefer_60fps, allow_hdr. normalize/load/save → video.db youtube_quality_profile.
  - api/video/downloads.py: GET/POST /downloads/youtube-quality.
  - UI: a 'YouTube Quality' card on the Downloads tab (data-video-only) — resolution
    dropdown + codec/container segmented + 60fps/HDR checks, reusing the vq-* styles.
    Wired into onPageShown + the save-all chain.

The (later-phase) yt-dlp downloader maps these to a format/format_sort selection.
9 new tests green, isolation guard green, ruff clean, JS/HTML balance clean.
2026-06-19 10:45:38 -07:00
BoulderBadgeDad
9f38c34900 video quality profile: loose resolution cutoff + movie/episode size split
Addressing UX feedback (Boulder): the cutoff and size guard were confusing for a
library that holds BOTH movies and TV.

  - 'Upgrade until' is now a LOOSE resolution target (4K / 1080p / 720p / SD /
    'best — never stop'), not a specific source×resolution tier. 4K is always in
    the list regardless of which tiers are toggled on (the old dropdown only
    listed enabled tiers, so 4K vanished when off). Stored as cutoff_resolution.
  - Size guard split into Max movie size + Max episode size (runtime-aware, like
    Radarr's MB/min but human-readable) — a flat GB cap was meaningless across a
    2-hour movie and a 25-min episode. Dropped the confusing min slider.

Pure-logic + API + UI in lockstep; removed the now-dead per-row cutoff marker.
12 tests green, ruff clean, JS/HTML balance clean.
2026-06-19 10:40:53 -07:00
BoulderBadgeDad
cd13b1185e video quality profile UI: rich-curated ladder + cutoff + rejects + soft prefs
Rebuilds the Video Quality Profile card to match the new Radarr-class model:

  - Quality ladder: one ranked, toggleable source×resolution list (Remux·4K …
    SDTV) reusing the .hybrid-source-item styling (parity with Download Source).
    The cutoff tier shows an accent rail + 'cutoff' tag.
  - Upgrade until: a styled dropdown of the enabled tiers (the cutoff).
  - Never grab: red toggle chips for the hard rejects (cam/screener/workprint/3d/x264).
  - Preferences: segmented controls for codec (Any/HEVC/AV1), HDR (Don't care/
    Prefer/Require), audio (Any/Surround/Lossless/Atmos) + a Prefer REPACK check —
    all soft tie-breakers.
  - Size guard: min + max GB sliders (0 = no limit).

Dropped the old resolutions/source_priority/single-codec/fallback widgets. JS
delegation rewritten for the new data-vq-* hooks; same-verdict-as-HEAD balance,
div balance clean. Pure UI on the isolated video Downloads tab.
2026-06-19 08:43:47 -07:00
BoulderBadgeDad
23cefd3c15 video quality profile: rebuild as rich-curated Radarr-class model (logic + tests)
Replaces the simplified resolutions+4-sources+codec model with a real quality
ladder to actually stand in for Radarr/Sonarr:

  - tiers: source×resolution ladder (Remux-2160p … SDTV) as one ranked, toggleable
    list, best→worst, with a cutoff (stop upgrading once the library holds a tier
    at/above it — no endless re-grabbing).
  - rejects: hard blocks (cam/screener/workprint/3d, optional x264).
  - soft preferences (score/tie-break, never reject alone): prefer_codec (any/hevc/av1),
    prefer_hdr (off/prefer/require), prefer_audio (any/surround/lossless/atmos),
    prefer_repack.
  - size guard: min/max GB per item (0 = no limit), min pinned to a real cap.

Pure normalize/load/save (no DB/network), isolated from music. API endpoint is an
unchanged passthrough. 12 tests green, ruff clean. UI next.
2026-06-19 08:38:16 -07:00
BoulderBadgeDad
975f81f476 video downloads: quality profile resolution/source rows reuse the hybrid-source-item styling
Consistency — the Resolution order and Preferred-source order lists now use the exact
same .hybrid-source-item markup/CSS (card + arrows + priority number + pill toggle) as
the Download Source hybrid list, instead of the bespoke .vq-row styling. Resolution
keeps its enable toggle; source is order-only (no toggle), no brand icon (they're not
branded sources). Removed the now-dead .vq-row/.vq-arrow/.vq-toggle/.vq-rows CSS.

Pure UI; data-vq-* hooks preserved so the existing delegation/handlers are unchanged.
2026-06-19 08:13:24 -07:00
BoulderBadgeDad
16ba035330 video downloads: UX/visual polish on the Downloads tab (review feedback)
- Hidden leftover music sections: the broad hide rule now targets ANY
  [data-stg='downloads']:not([data-video-only]), not just .settings-group — so the
  music Quality-Profile tile (#quality-profile-tile) and the Retry-Logic collapsible
  (which are divs/section-headers, not .settings-group) no longer show as dead,
  un-openable shells on the video side.
- Reordered: Download + Transfer folders now come BEFORE the Download Source section.
- slskd connection section is now COLLAPSIBLE (collapsed by default) — it was a lot of
  fields; uses the same settings-section-header/body pattern as music's Retry Logic.
- Hybrid chain now reuses music's .hybrid-source-item markup/CSS for visual PARITY
  (icon + name + priority + pill toggle + arrows), minus the album/track badge.
- Quality profile copy clarified: 'Resolutions to accept' (order = preference) vs
  'Preferred source (tie-breaker)', clearer per-control hints.

Pure UI (markup/CSS/JS); backend + tests unchanged. JS verdict matches HEAD, HTML divs balanced.
2026-06-19 08:04:02 -07:00
BoulderBadgeDad
441f0e9ad1 video downloads: reword download_config docstring (a wrapped line started with 'from the music side' — tripped the core/video no-music-import guard, a test false positive) 2026-06-19 00:25:57 -07:00
BoulderBadgeDad
0a0859df7a video downloads (phase 4): shared slskd connection block on the video Downloads tab
The slskd CONNECTION settings (URL, API key, search timeout + buffer, min delay, min
peer speed, max peer queue, download timeout, auto-clear) are genuinely shared — one
slskd instance serves both sides — so the video Downloads tab surfaces them and they
read/write the app-wide config_manager soulseek.* keys, NOT video.db. Changing them
on the video side changes them for Music too (labeled 'shared with Music').

Deliberately EXCLUDES the music download/transfer paths and source mode/quality — those
stay video-specific (video.db, phases 1-3). The block shows only when soulseek is the
active/primary source (or in the hybrid chain). The 'Indexers & Downloaders' tab is
already fully shared (no video override — only data-stg='downloads' is hidden).

- /api/video/downloads/slskd GET/POST over config_manager (config.settings — shared app
  config, not music code; the isolation test still passes).
- video-settings.js loadSlskd/saveSlskd (minutes↔seconds for the timeout), gated by
  soulseekActive(), wired into onPageShown + the save chain.

68 video API tests green (incl. the no-music-imports guard + a shared-slskd test that
asserts it writes soulseek.* but never the video paths). Completes the Downloads-tab
settings batch (folders, quality profile, source/hybrid, shared slskd).
2026-06-19 00:20:44 -07:00
BoulderBadgeDad
4d45cc614f video downloads (phase 3): source mode + hybrid chain (soulseek/torrent/usenet only)
Video-specific Download Source section: a mode dropdown limited to Soulseek / Torrent /
Usenet / Hybrid (no streaming sources — those are music-only). In hybrid mode, a chain
builder lists the three sources with arrow-reorder + enable toggles (best-first), and
NO album-level/track-level badges (a music-only concept, per spec).

- core/video/download_config.py: pure normalize for download_mode + hybrid_order
  (validates to the 3 sources, dedupes, never empties); stored in video_settings.
- /api/video/downloads/config GET/POST now carries download_mode + hybrid_order
  alongside the folders.
- video-settings.js: dropdown + hybrid rows render/reorder/toggle, show the hybrid
  container only in hybrid mode, save on change. Reuses the .vq-row styling.

7 tests (6 pure + 1 API). Next: the shared slskd connection block (reads/writes the
music config_manager so both sides share one slskd) + confirming the shared Indexers tab.
2026-06-19 00:15:13 -07:00
BoulderBadgeDad
887ee01cb2 video downloads (phase 2): unified video quality profile
One profile across slskd/torrent/usenet — video quality is resolution + source +
codec (parsed from the release name/filename), not bitrate. Best-in-class settings UI
on the video Downloads tab:
- resolution tiers (4K/1080p/720p/480p) with per-tier toggle + arrow reordering
  (priority best-first), pill toggles
- source preference (BluRay > WEB-DL > WEBRip > HDTV), reorderable
- codec segmented control (Any / x265 / x264), Prefer-HDR toggle
- max-size-per-item slider (0 = no limit) + lower-quality fallback toggle

core/video/quality_profile.py: pure default/normalize/load/save (clamps size, rejects
bad codec, dedupes source list, recovers from corrupt JSON) — stored as JSON in
video.db's video_settings['quality_profile']. GET/POST /api/video/downloads/quality.
video-settings.js renders + saves it; isolated from music (imports only json/typing).

11 tests (9 pure + 2 API). Next: video source-mode + hybrid, then the shared slskd block.
2026-06-19 00:11:37 -07:00
BoulderBadgeDad
8392de9207 video downloads (phase 1): video-specific download + transfer folders
Foundation for the isolated video download settings. The Downloads tab is almost
entirely music-specific, so on the video side the music download sections are hidden
(video-side.css) and a data-video-only 'Video Download Folders' section takes their
place — an input (download) and output (transfer/library) folder, stored SEPARATELY
from the music soulseek.* paths in video.db's video_settings KV table.

- api/video/downloads.py: GET/POST /api/video/downloads/config (download_path,
  transfer_path), registered in the video blueprint. Imports nothing from music.
- video-settings.js: loadDownloads/saveDownloads wired into onPageShown + the
  video save-button chain.
- The shared 'Indexers & Downloaders' tab is left untouched (identical for both).

2 tests (round-trip + the isolation guard). Quality profile, video hybrid, and the
shared slskd block are the next phases.
2026-06-19 00:04:27 -07:00
BoulderBadgeDad
8c2f66bea9 video enrichment: keyless workers read 'Disabled' (not 'Not configured') when off
AniList defaults off (anime opt-in), so it showed 'Not configured' — implying a
missing API key, when it's keyless and just toggled off in Settings > Community Data
(No Key). Workers now report needs_key in get_stats (True for the key-gated
fanart/opensubtitles/trakt + all matchers; False for the keyless toggles). The
manager rail/pill + dashboard-header tooltip show 'Disabled' / 'Off — enable in
Settings' for a disabled keyless worker, and keep 'Not configured' only for ones
that genuinely need a key.

(The AniList on/off toggle already exists in the collapsed 'Community Data (No Key)'
settings frame — this just makes the status honest about what's needed.)
2026-06-18 23:29:22 -07:00
BoulderBadgeDad
f98ceecd68 video enrichment: show the 5 new workers in the dashboard-header orbs too
They were registered in the manager modal (WORKERS) + status poll (SERVICES) but the
dashboard HEADER renders its own per-worker .video-enrich-container buttons (with the
floating orb animation) keyed by data-video-enrich, and the orb engine has its own
WORKER_DEFS list — neither had the new services, so trakt/tvmaze/anilist/dearrow/
wikidata were absent from the header.

Added a header button + tooltip block for each (matching the fanart/opensubtitles
pattern, accent colors consistent with the manager orbs) and the matching WORKER_DEFS
entries in video-worker-orbs.js. Status was already wired via the SERVICES list, so
they now animate + report in the header identically to the existing workers.
2026-06-18 23:22:32 -07:00
BoulderBadgeDad
b3f704fbfd video enrichment: add Wikidata (official website link) — full parity, completes the batch
Fifth/final service. Keyless, movies + shows, on by default → toggle in the
'Community Data (No Key)' frame.
- WikidataWorker: two-step lookup — find the entity by IMDb id (haswbstatement P345)
  then read its official website (P856); stores wikidata_url. Registered in
  build_backfill_workers.
- DB: wikidata_url/status/attempted on movies + shows + _BACKFILL/_BACKFILL_COLS;
  both detail payloads return wikidata_url.
- config GET/POST wikidata_enabled; manager orb (green 🔗) + status poll; detail page
  'Official Site' link badge alongside IMDb/TMDB/TVDB.
- 3 new tests (incl. the two-step fetch) + fixed-set/config assertions.

34 backfill tests green, ruff clean. All five new services (Trakt, TVmaze, AniList,
DeArrow, Wikidata) now have the full worker/DB/connections/orb/manager/detail parity.
2026-06-18 23:13:22 -07:00
BoulderBadgeDad
c33810f39f video enrichment: add DeArrow (crowd-sourced YouTube titles) — full parity
Fourth service. Rides the YouTube-video enrich path (like RYD/SponsorBlock), keyless,
on by default → toggle in the YouTube-Extras frame.
- DeArrowWorker: fetches the branding API and stores the first non-original crowd
  title for a cached YouTube video. apply_youtube_dearrow + youtube_video_dearrow_title
  DB methods; dearrow_status added to the youtube_enrich next/breakdown whitelists.
- Schema: dearrow_title/status/attempted on youtube_video_stats (video_schema.sql +
  _COLUMN_MIGRATIONS for existing DBs).
- config GET/POST dearrow_enabled; manager orb (blue 🏹, video-kind) + status poll;
  YT video-detail panel shows a 'DeArrow' alt-title (payload via youtube.py + CSS).
- 4 new tests + fixed-set/config assertions.

30 backfill tests green. (My change is ruff-clean; the 12 pre-existing S110s in
api/video/youtube.py are unrelated tech debt on this branch, left untouched.)
2026-06-18 23:09:02 -07:00
BoulderBadgeDad
6f8a2a3f7a video enrichment: add AniList (anime score) — keyless GraphQL worker, opt-in
Third service. Keyless GraphQL (new _http_post_json helper) → enable toggle in the
'Community Data (No Key)' frame, OFF by default (anime-niche + title-search match).
- AniListWorker: TV-only, searches AniList by title and stores the anime averageScore
  (0-100), with a conservative normalized-title guard so a fuzzy anime search can't
  attach a score to a non-anime show. anilist_enabled toggle (default off).
- DB: anilist_score/status/attempted on shows + _BACKFILL/_BACKFILL_COLS (keyed on
  title); show_detail returns anilist_score.
- config GET/POST anilist_enabled (default 0); manager orb (blue 🎌) + status poll;
  detail page 'AniList 85%' chip (+ CSS).
- 4 new tests (incl. the title-mismatch rejection) + fixed-set/config assertions.

27 backfill tests green, ruff clean. (Note: the youtube_status_route test still
flaps on the sandbox WSL WAL disk-I/O — environmental, unrelated.)
2026-06-18 23:01:23 -07:00
BoulderBadgeDad
5a95619e22 video enrichment: add TVmaze (TV community rating) — keyless worker, full parity
Second service. Keyless (no API key) → an enable toggle in a new 'Community Data
(No Key)' connections frame, mirroring the YouTube-Extras pattern.
- TVmazeWorker: TV-only (no movie DB), looks a show up by imdb/thetvdb id and
  gap-fills the TVmaze community rating. On by default; tvmaze_enabled toggle.
- DB: tvmaze_rating/status/attempted on shows + _BACKFILL/_BACKFILL_COLS (show only);
  show_detail returns tvmaze_rating.
- config GET/POST tvmaze_enabled; manager orb (teal 📺, show-kind) + status poll;
  detail page TVmaze rating chip (+ CSS).
- 4 new tests + the 3 fixed-set/config-exact assertions updated.

Note: one UNRELATED test (youtube_status_route) flaps on a WSL WAL 'disk I/O error'
in this sandbox (pass/fail/fail across reruns of identical code) — environmental, not
this change; the TVmaze + config tests pass consistently.
2026-06-18 22:56:02 -07:00
BoulderBadgeDad
314e587094 video enrichment: add Trakt (community audience rating) — full worker parity
First of the new enrichment services, wired to the SAME standard as the rest:
- TraktWorker backfill (core/video/enrichment/backfill.py): looks a title up by its
  IMDb id (?extended=full) and gap-fills the community rating + vote count. Registered
  in build_backfill_workers.
- DB: trakt_rating/trakt_votes/trakt_status/trakt_attempted columns + _BACKFILL /
  _BACKFILL_COLS registration; detail payloads return trakt_rating/votes.
- Connections tab: Trakt service frame (Client ID field + Test button), wired into
  video-settings.js load/save/bindings and the /enrichment/config GET+POST.
- Worker-manager modal + orb: WORKERS registry entry (red ★) + status-poll SERVICES
  list, so it gets the same orb animation + per-service matched/pending/error card.
- Detail page: a 'Trakt 8.2' rating chip alongside IMDb/RT/Metacritic (+ CSS).

5 new tests + 2 fixed-set assertions updated; 249 video tests green, ruff clean.
2026-06-18 22:33:36 -07:00
BoulderBadgeDad
c55a4fa10e video detail: surface subtitle availability (OpenSubtitles) + fix B023
The OpenSubtitles backfill worker already collects which subtitle languages exist
for each title (movies.subtitle_langs / shows.subtitle_langs), but nothing showed
it. Now the detail payload returns it (parsed to a list) and the hero renders a
'Subtitles: English · Spanish · …' CC-tinted chip row under the genres — so you can
tell subs exist before grabbing the file. Hidden entirely when there's no data.

(The clearlogo hero was already implemented, so this targets the one genuinely
invisible enrichment field.)

Also bound the per-iteration loop var in the ratings-breakdown counter (pre-existing
ruff B023). 154 video tests green, ruff clean.
2026-06-18 21:58:06 -07:00
BoulderBadgeDad
f7f21f44c5 video dashboard: live system stats (memory + uptime) from the shared endpoint
The video dashboard's Memory Usage / System Uptime cards were stuck at their
markup defaults — flatten() only fed library + download figures from
/api/video/dashboard, so data-video-stat='memory'/'uptime' never updated.

Now it pulls those two from the SAME /api/system/stats the music dashboard uses
(one machine → identical figures), with an immediate fill on page-shown and a 10s
poll for live parity with music's push loop. Reached over HTTP (not music's
socket), so the video isolation contract holds; the poll cheaply no-ops whenever
the dashboard isn't the visible page.
2026-06-18 21:24:59 -07:00
BoulderBadgeDad
530d6f181f Video enrichment: add the 4 new workers to the dashboard header
- Dashboard header now shows fanart.tv / OpenSubtitles / YouTube Votes / SponsorBlock
  buttons alongside TMDB/TVDB/OMDb/YouTube (same chip + spinner + tooltip + click
  pause/resume + worker-orb animation).
- Socket status loop now iterates ALL engine workers (matchers + backfill) instead
  of a hardcoded 3, so new buttons get live 2s status with no extra wiring.
- video-enrichment.js SERVICES + video-worker-orbs.js WORKER_DEFS extended.
  header-actions already flex-wraps, so 8 buttons reflow cleanly.
2026-06-18 08:04:53 -07:00
BoulderBadgeDad
6f14f15dd6 Video enrichment: surface enriched YouTube data
- YouTube video cards show 👍/👎 like/dislike counts (from Return YouTube Dislike)
  next to views, threaded through ytEpisodeOf.
- New GET /youtube/video/<id>/segments exposes stored SponsorBlock segments so the
  player can offer skips. Update config-shape test for the new keys/toggles.
2026-06-17 23:35:59 -07:00
BoulderBadgeDad
21a1e5ddbb Video enrichment: wire new workers into Manage Workers modal + Settings
- Manage Workers modal: register fanart.tv / OpenSubtitles / YouTube Votes (RYD)
  / SponsorBlock in the worker rail (cards + animations + pause/resume come free
  via the shared .em-* design); add the 'video' entity kind.
- Settings: fanart.tv + OpenSubtitles API-key fields (with Test buttons) and a
  no-key 'YouTube Extras' toggle frame (RYD + SponsorBlock on/off).
- API /enrichment/config now reads/writes the two keys + the two toggles; a key
  change rebuilds the engine so the worker turns on immediately.
2026-06-17 23:31:33 -07:00
BoulderBadgeDad
278ba47519 Video enrichment: backfill-worker framework + 4 new workers (fanart.tv, OpenSubtitles, Return YouTube Dislike, SponsorBlock)
Adds a VideoBackfillWorker base that enriches already-identified items BY id
(vs the matcher workers), with the exact same lifecycle + get_stats() shape so
the engine registry, /api/video/enrichment routes, and Manage-Workers modal
drive them identically.

Workers:
- fanart.tv (free key): gap-fills logo/clearart/banner/backdrop/poster art
- OpenSubtitles (free key): records subtitle-language availability per title
- Return YouTube Dislike (no key): like/dislike estimates on cached videos
- SponsorBlock (no key): crowd segments per video

DB: youtube_video_stats + youtube_video_segments tables; fanart/subs columns;
backfill_next/mark/breakdown + youtube_enrich_* helpers; likes/dislikes merged
into get_channel_videos. Seam tests in tests/test_video_backfill.py.
2026-06-17 23:26:46 -07:00
BoulderBadgeDad
856f14824f Playlist/channel extraction: stop pinning a static user_agent for yt-dlp
Correcting the cookies theory — ytdl-sub is just CLI yt-dlp (no browser). The real
differentiators are a CURRENT yt-dlp + not fighting YouTube's client identity. We
were pinning a static user_agent in the yt-dlp opts, which yt-dlp recommends against
(it trips YouTube's heuristics and truncates large-playlist pagination). Dropped it
so yt-dlp manages its own (current) client — the lever that lets a fresh ytdl-sub
page further than us. (_UA is still used for the InnerTube requests.) Softened the
partial-playlist note to just 'Showing N of TOTAL videos.'
2026-06-17 22:21:12 -07:00
BoulderBadgeDad
67197ce956 Playlists: page the FULL list when YouTube cookies are set (like ytdl-sub)
Correcting my earlier 'YouTube caps at ~200' claim — that's the ANONYMOUS ceiling.
YouTube throttles unauthenticated playlist extraction to ~100 (yt-dlp) / ~200
(InnerTube); ytdl-sub gets everything because it runs with browser cookies. Our
code already wires cookiesfrombrowser via youtube.cookies_browser (same as the
music client) — it's just unset. The detail endpoint now resolves with a high
limit so a cookie-authenticated yt-dlp pages the WHOLE playlist, and uses whichever
source (yt-dlp vs InnerTube) returned more, with the true header count. The partial
note now tells you to add cookies in Settings to load the full list.
2026-06-17 22:07:50 -07:00
BoulderBadgeDad
8e56273c9d Playlist count: show the TRUE total + load ~200 (was capped at ~100 via yt-dlp)
A big playlist read '97' because yt-dlp flat caps playlist extraction at ~100 and
its playlist_count is often unset, so we showed the FETCHED count. Now the detail
endpoint overlays the InnerTube playlist browse (browseId VL<id>): curator-ordered
videos up to ~200 (vs yt-dlp's ~100) AND the real count from the header's
numVideosText (e.g. '512 videos'). The billboard shows the true total; when the
list is partial it says 'Showing the first N of TOTAL videos (YouTube limits
playlist browsing).' Live-validated (Veritasium uploads: total 512, 200 loaded).
103 tests green.

Honest cap: YouTube's browse only exposes ~200 of a playlist (page 2 returns no
continuation), so very large playlists show the right COUNT but list ~200.
2026-06-17 21:40:56 -07:00
BoulderBadgeDad
5506155abe YouTube videos: play in-app (stream via the embed overlay)
Clicking a video thumbnail now plays it inline — the thumbnails are play buttons
(data-vd-yt-play) that reuse the existing trailer overlay (youtube.com/embed/<id>
?autoplay=1). Applies to channel + playlist video cards and the playlist-section
mini-cards (which previously just linked out to YouTube). The rest of the row
still expands to details.
2026-06-17 21:29:04 -07:00
BoulderBadgeDad
4788b54c91 Playlist detail: fix 'Open on YouTube' + hero Watchlist button (were channel-only)
On a playlist page the hero actions were hardcoded for channels: 'Open on YouTube'
linked to /channel/<PL id> (opened as a channel, failed), and the Watchlist button
called the channel-follow handler. renderActions is now kind-aware — playlists link
to /playlist?list=<id> and the button follows/unfollows the PLAYLIST (new
toggleYtPlaylistFollowHero via a yt-pl-follow action).
2026-06-17 21:24:49 -07:00
BoulderBadgeDad
ba7562c5f2 Watchlist cards: show the channel/playlist video count, not '0 videos'
The channel card's 'N videos' was actually the count of WISHED videos (0 until you
wish some) — so every channel read '0 videos'. It now shows the REMEMBERED catalog
size (from youtube_channel_videos, which fills in as a channel is enriched/opened),
falling back to 'Channel' when nothing's cached yet. Wished count is kept as a
separate wished_count field. Playlists likewise show their video count (cached when
the playlist is followed-with-videos or opened) instead of a static 'Playlist'.
list_watchlist_channels/playlists return the remembered count; the playlist detail +
follow endpoints cache the list so the count is known. 138 tests green.
2026-06-17 21:13:33 -07:00
BoulderBadgeDad
abb9d0e89e + Wish: use the app-standard watchlist-button chrome (was bespoke)
The per-video wishlist toggle now reuses .library-artist-watchlist-btn (icon +
text, accent gradient) via a shared ytWishBtn() helper — compact in the dense
episode rows, icon-only in the tight playlist-section cards. 'Wishlist' ↔ 'In
Wishlist' with a green .watching state. Same behavior + data-vd-yt-wish hook;
just consistent chrome.
2026-06-17 21:02:08 -07:00
BoulderBadgeDad
113a5ded38 Fix: + Wish silently failed on playlist detail pages (no channel context -> 400)
The per-video + Wish derived its channel from data._channel, which only exists on
CHANNEL pages — on a playlist detail page it's data._playlist, so the add posted
no youtube_id and the endpoint 400'd (button reverted, nothing happened). Now it
falls back to the playlist's owner channel (channel_id/title, else the playlist
itself) so wishing a video works on playlist pages too.
2026-06-17 20:56:45 -07:00
BoulderBadgeDad
62b815a3fe Channel page: Add-to-watchlist on each playlist section
Channel playlists (the collapsible sections on a channel page) now each carry the
standard watchlist button — Add to Watchlist / In Watchlist — so you can follow a
channel's playlist without pasting its link. The /youtube/playlists/<id> endpoint
flags each playlist with its follow state to hydrate the button; the button click
is handled before the row's expand toggle (stopPropagation), and adds via
followPlaylist — the same plumbing the search chip + watchlist use. Followed ones
then appear in the watchlist Channels tab alongside channels.
2026-06-17 20:48:59 -07:00
BoulderBadgeDad
e141e3dcbd Playlist search result: standard 'Add to Watchlist' button (not 'Follow')
The pasted-playlist chip used the YouTube 'Follow' toggle; swapped it for the
app-standard watchlist button (library-artist-watchlist-btn — same look/wording
used across video + music): + Add to Watchlist ↔ ✓ In Watchlist. The search
toggle handler updates the icon/text spans + 'watching' class accordingly. No
global handler keys off that class, so reusing it is purely cosmetic.
2026-06-17 20:42:30 -07:00
BoulderBadgeDad
ef3d4cfdb4 YouTube playlists on the watchlist — frontend (search, detail, watchlist)
- search: paste a playlist link → it resolves to a 'YouTube playlist' chip
  (cover, owner, count) with Add-to-watchlist; clicking the chip opens it.
  (isPlaylistRef + playlistCard + followPlaylist/unfollowPlaylist helpers.)
- routing: /video-detail/youtube/playlist/<PL…> deep-links like a channel
  (string id); the open-detail event handles kind='playlist'.
- detail view: a FLAT list in the curator's order (no year-seasons, no season
  nav / view toggle, no catalog streaming — it's a partial set). Billboard shows
  'Playlist · N videos · owner'. Reuses the show-detail shell + the new
  duration/views cards.
- watchlist: followed playlists sit beside channels in the Channels tab (rounded
  square + a list badge), open on click, ✕ to unfollow.

Live-validated (Lex Fridman playlist resolves + renders in order). The 2 shell
test 'failures' are the pre-existing window.-assertion (no new globals added).
2026-06-17 20:33:31 -07:00
BoulderBadgeDad
28035f0d70 YouTube playlists on the watchlist — backend (resolve, follow, detail)
A playlist is now a first-class watchlist type alongside channels:
- core: parse_playlist_id (URL/bare id; rejects mixes RD… + personal lists) +
  resolve_playlist → {playlist_id, title, channel_title, video_count, thumbnail,
  videos} in the CURATOR's order (partial set, not by year).
- db: add/remove/list/watch_state for kind='playlist' video_watchlist rows
  (mirrors channels; same surrogate scheme, separate kind so they don't collide).
- api: /youtube/resolve now detects playlist links; /youtube/playlist/follow +
  /unfollow; the existing /youtube/playlist/<id> is upgraded to also return the
  playlist meta + following (serves the channel-page expansion AND the new detail
  view from one route); /youtube/channels also returns followed playlists.

Validated live (Lex Fridman playlist resolves title+owner+videos). 190 tests green.
Frontend (search detect + result card + detail view + watchlist display) next.
2026-06-17 20:21:44 -07:00
BoulderBadgeDad
eb8f803c36 Channel rail: sharp season posters (maxres thumbnail, was a cropped hqdefault)
The rail 'season poster' is a video thumbnail shown in a 2:3 portrait slot
(object-fit: cover), but InnerTube hands us hqdefault (480x360, often sqp-shrunk)
— so it got crop-zoomed and looked soft. Now the poster pulls maxresdefault
(1280x720; ~300KB vs ~23KB) via the image proxy, with a fallback chain
(maxres → original thumbnail → hide) so videos without a maxres thumb still
render. Only the big rail poster is upgraded; the small episode stills already
downscale crisply.
2026-06-17 19:17:29 -07:00
BoulderBadgeDad
cb5ae93f76 Channel page: restore the season view toggle (rail posters etc.) broken by search/sort
The search/sort change hardcoded pillsHTML() for the youtube branch, so the
view toggle did nothing and the rail view's season posters vanished. The youtube
branch now honours seasonView (rail / timeline / tabs / list) for the year nav,
with the new search+sort controls stacked above it; flat mode (search / most-
viewed / longest) still hides the per-year nav. Default view (rail) shows the
year posters again.
2026-06-17 19:11:30 -07:00
BoulderBadgeDad
9c3d1a5709 Channel page: search + sort within the channel (restores the lost browse controls)
The standalone channel page had sort/filter/search; the merge into show-detail
dropped them. Now that the whole catalog is client-side, they're instant:

- A controls row above the year pills: a debounced title search + a sort select
  (Newest / Oldest / Most viewed / Longest).
- Newest/Oldest keep the year-season grouping (reordered); search + Most-viewed +
  Longest collapse into one flat sorted 'results' list. Empty search → a clear
  'No videos match …' state.
- Refactored ytToShow's grouping into reusable helpers (ytGroupByYear / ytEpisodeOf)
  + ytRegroup(), which the streaming loop now uses too — so newly-streamed videos
  fold into whatever view/sort is active, and the master list stays intact.

Search focus is preserved across the regroup (same hack the music manager uses).
2026-06-17 18:45:40 -07:00
BoulderBadgeDad
c7b5a93290 Channel videos: duration badge + view count on every card (TV-parity)
TV episode rows show runtime; channel video cards showed nothing per video. The
InnerTube lockup already carries both — now we capture duration ('12:34') + an
approximate view count (parsed from '2.6M views') in innertube_parse_video_items,
remember them on youtube_channel_videos (new duration/view_count cols + migration),
and render a duration badge bottom-right of the thumb + 'N views' in the card meta.
The background stream backfills them onto the recent (yt-dlp) videos too, so the
whole catalog gets them. Live-validated (MrBeast: 32:08 / 50M, 30/30). 65 tests green.
2026-06-17 18:36:47 -07:00
BoulderBadgeDad
66f3f97460 Channel videos: POST + one page per request (kill the giant slow-request log spam)
Each /videos batch was 3 InnerTube pages + 0.4s sleeps (>1s → tripped the
slow-request WARNING) and carried the ~2KB continuation token in the URL, so
every batch dumped a giant warning line — a dozen per channel open. Now it's a
POST (token in the body, never the URL) fetching ONE page per call with no
server sleep, so each request is fast (<1s, no warning) and the frontend's 120ms
pacing keeps it polite. 17 youtube API tests green.
2026-06-17 18:19:32 -07:00
BoulderBadgeDad
267f11c848 Remember each channel: cache the video catalog + metadata (instant re-open)
Channel pages re-fetched everything every open (re-stream + ~3s yt-dlp metadata).
Now we remember what we learn about a channel and serve it cache-first:

- schema: youtube_channel_videos (list) + youtube_channel_meta (avatar/subs/tags/
  banner); dates stay in youtube_video_dates, merged on read. DB: cache_/get_
  channel_videos + cache_/get_channel_meta (upserts COALESCE so refreshes never
  drop fields).
- enricher: _enrich now REMEMBERS a channel — caches the full InnerTube catalog
  (list, via new innertube_channel_catalog) + metadata + dates, not just dates.
  Since it already sweeps followed channels, watchlisted channels get pre-warmed
  in the background → opening them is instant.
- channel endpoint: cache-first. A remembered channel renders from cache with NO
  network (returns from_cache:true); a miss resolves live (yt-dlp) and remembers
  it. The /videos batch endpoint caches every page it streams.
- frontend: cache hit renders the full catalog instantly, then re-streams to
  refresh QUIETLY (new uploads/date fixes); only the first (miss) load shows the
  'loading full history' banner.

Live-validated (MrBeast catalog: 90/3 pages, all titled+dated). 190 tests green.
2026-06-17 18:08:04 -07:00
BoulderBadgeDad
d91693ba53 Channel page: stream the FULL video catalog in batches (no more 90 cap)
The channel page was capped at the recent 90 (yt-dlp flat, hard min(90,limit)),
so prolific channels (Ninja Kidz etc.) showed only ~90 of hundreds. Now it loads
everything via YouTube's InnerTube API, paginated by continuation token — each
page fetched once (O(n), light on rate limits), unlike yt-dlp offset batches
which re-scan from the start every time.

- core: innertube_parse_video_items (keeps title+thumbnail, not just dates) +
  innertube_channel_videos_page(channel_id, continuation) → {videos, continuation}.
- api: GET /youtube/channel/<id>/videos?continuation=<tok> returns ~a batch of
  videos (approx dates, refined from the date cache) + the next token.
- frontend: ytLoadAllVideos streams batches after the fast initial 90 render,
  folding each into the year-seasons live — backfills dates on the videos already
  shown AND appends older ones, re-rendering per batch (episode grid only when the
  viewed season changed). Replaces the old date-only re-poll (this dates AND
  expands). Safety ceiling 2000; a 'Loading full history… N so far' indicator.

Validated live (MrBeast: 30/page, clean continuation, all dated). 101 tests green.
2026-06-17 17:42:21 -07:00
BoulderBadgeDad
5012f44cfa Video: deep-link every sidebar page (URL parity with the music side)
Only the detail page produced a real URL before; top-level video pages (search,
library, discover, calendar, watchlist, wishlist, …) pushed nothing and even
cleared the URL to '/'. Now each page deep-links to '/' + pageId (e.g.
/video-search), mirroring music's '/<page>' and the existing /video-detail/ scheme:

- navigate() pushes/replaces { videoPage } history state; parsePagePath/buildPagePath
  added alongside parseDetailPath/buildDetailPath.
- popstate restores a page URL (and hands the side back to music when Back crosses
  out of /video-*); boot restores a page deep link, re-asserting the URL against
  music's boot clobber (same tactic the detail boot already uses).
- applySide() is now chrome-only; switchSide()/boot drive navigation explicitly.
- Nav anchors carry the real href=/video-<page> (was '#') and the click handler
  lets ⌘/Ctrl/middle-click open a new tab — full parity with the music nav + cards.

Server already serves /video-* via the SPA catch-all; music side untouched
(isolated IIFE). Reload / Back / Forward / new-tab now work for every page.
2026-06-17 16:52:58 -07:00
BoulderBadgeDad
4c16448ab3 Calendar 'next up' hero: smoother panel motion + collapsed panels rest at 25%
- The hover snap was jarring because collapsed panels floored at min-width:10%
  (so panels jumped ~10%↔80%) on a fast-start easeOutCirc curve. Raised the floor
  to 25% (3 panels now settle 50/25/25 and hovering just glides the 50% across)
  and switched to a gentler ease (0.66s cubic-bezier(0.45,0,0.2,1)).
- Lengthened the dim-overlay + sub/actions fades (0.4→0.55s, 0.3→0.45s) so they
  move with the expansion instead of finishing early. Mobile stack unaffected
  (it overrides min-width:0).
2026-06-17 16:32:31 -07:00
BoulderBadgeDad
cf03c28831 Discover: taller hero slider (420->520 desktop, 340->420 tablet) 2026-06-17 16:26:17 -07:00
BoulderBadgeDad
a3adf5e63c Video Manage-Workers modal: realign to the music modal's exact design
The video modal reused music's shell but had drifted into a parallel mini-design:
it emitted its own markup (.em-ph-main / .em-pause-btn / .em-item / .em-pg /
.vem-logo / .vem-icon) styled separately in video-side.css, instead of the
music classes that already exist in style.css. So the panel header, list rows,
rail icons and pager all looked 'redesigned'.

Now the modal emits music's real markup and inherits its global .em-* styling:
- Panel header -> .em-hero (glow + .em-icon--lg + .em-pill + .em-ph-sub)
- Unmatched list -> .em-row (status stripe, art+glyph, 'tried Nd ago', ghost Retry)
- Rail icon -> .em-icon chip (with --i stagger); pager -> .em-pager + .em-btn
- Pause -> .em-btn/.em-btn--go; coverage cards gain the First/Done/N-left badges
- Open/close entrance+exit animation (.em-in / .em-closing), dropping .em-in
  after it plays so the 3s rail re-render doesn't replay the stagger
- Panel sets --accent-rgb so music's accent-keyed rules recolor per worker

Removed the now-dead bespoke CSS. Episodes stay a coverage card (per the backend:
episode art cascades from a show match; /priority only accepts movie/show), so
the top bar is correctly Movies/Shows/Auto. Music side untouched (shared classes
are global; video overrides were all #vem-overlay-scoped or now deleted).
2026-06-17 14:18:43 -07:00
BoulderBadgeDad
21ed97ff26 YouTube enrichment: legacy channels auto-upgrade to InnerTube (backwards compat)
Channels enriched before InnerTube existed (partial coverage, e.g. CaseyNeistat
36, Good Mythical MORE 25) were locked behind the 24h coverage gate and would
never reach the full ~240-date catalog on their own. Tag each enrichment row
with the source ('innertube'|'fallback'); legacy rows (method NULL after the
additive migration) bypass the gate ONCE so they re-enrich and upgrade, then
settle under the normal window. Non-destructive: existing follows, wished videos
(24), and cached dates (1176) are untouched — only coverage refreshes on next
open/sweep. Migration is additive (method TEXT); regression test added.
2026-06-17 13:51:10 -07:00
BoulderBadgeDad
91bd895e52 YouTube channels: follow = watchlist only (clean toast) + drop the misleading '90 videos'
Two reports:
- 'Following · 0 videos added' toast: following from the channel page sent a
  stub with no videos, so 0 were wished. Watchlisting a channel shouldn't bulk-
  wish its whole catalog anyway (that conflates watchlist/track with wishlist/
  acquire, unlike TV shows). follow() now sends only the channel fields and the
  toast reads 'Added to watchlist'. Videos are wished explicitly via + Wish.
- '90 videos' in every channel header: that was just our fetch cap (limit=90),
  not the real total — YouTube no longer exposes a trustworthy channel video
  count (videoCountText reports a shelf count: 142 for Veritasium, 71 for MrBeast).
  Dropped it; the accurate subscriber count already conveys the channel's scale.
2026-06-17 13:37:15 -07:00
BoulderBadgeDad
c00df179ba Channel page: fix re-poll never refreshing when recent videos collapse to one year
The date re-poll only re-rendered if the season COUNT increased — but for a
daily channel the 90 recent videos all land in the current year, so it went from
[2026(3), Earlier(87)] (2 seasons) to [2026(90)] (1 season): count dropped, so
the years (which WERE cached, per the logs) never showed and the spinner hung.
Now it re-renders whenever the 'Earlier videos' bucket shrinks too. Also extended
the poll window (20/45/80/120/160s) for channels queued behind others.
2026-06-17 13:22:23 -07:00
BoulderBadgeDad
b627f9f411 Channel page: 'fetching dates' indicator during the wait + watchlist toggle no longer reloads
- The InnerTube enrichment takes ~30-45s; while videos are still undated the page
  now shows a spinner 'Fetching upload dates from YouTube… your year-seasons will
  fill in shortly' (reuses the episode-syncing indicator), which clears once the
  years populate via the re-poll (or after the poll window).
- The Watchlist button on a channel was calling loadChannel() on follow — a full
  re-fetch + scroll-to-top that read as a page refresh. Now it just flips the
  button in place (data.following + renderActions), like unfollow already did.
2026-06-17 13:11:17 -07:00
BoulderBadgeDad
b5586d3f28 YouTube: enrich dates on ANY channel page open (not just followed)
Bug from the logs: opening MrBeast showed only '2026: 3, Earlier: 87' because
the InnerTube date enrichment only fired for FOLLOWED channels — MrBeast was
opened, not followed, so it never ran and the page had only RSS dates (and RSS
includes Shorts that don't match the Videos tab → ~3 real matches). Now opening
any channel page enqueues enrichment, so its full catalog gets dated in the
background and the years populate via the existing re-poll. (24h gate still
prevents re-sweeping; the enqueue still no-ops under tests.)
2026-06-17 13:05:25 -07:00
BoulderBadgeDad
59dafff96a YouTube dates: custom InnerTube parser as the primary bulk source (validated live)
A Beatport-style custom parser, but pointed at YouTube's own InnerTube browse API
— no key, no Java, no third-party proxies. Reads the channel 'Videos' tab's
lockupViewModel items (contentId + relative 'N ago' text → approximate date,
which is fine for year-seasons), paginating via continuation tokens.

Validated LIVE before wiring in: GMM/MrBeast/Veritasium each return 240 dated
videos in ~7s with correct year spreads (e.g. Veritasium 2017→2026), ~8 requests
per channel (light on rate limits). End-to-end enricher run cached 240 dates.

Enricher order is now: InnerTube (primary) → configured proxy (opt-in, only if
empty) → yt-dlp per-video (the fallback / basic method, for undated gaps + exact
dates). Pure parsing is fully unit-tested (relative-date conversion, lockup
filtering, continuation-token selection, pagination, guards). 182 video tests
green; additive — if InnerTube ever breaks, it returns {} and falls back, so
nothing breaks.
2026-06-17 12:52:20 -07:00
BoulderBadgeDad
a83df05ce2 YouTube dates: proxy is opt-in (public instances are dead); yt-dlp is the default
Live-tested the proxy against real channels — it does not work: Invidious public
APIs are 401/403 (auth-walled/disabled), and the one reachable Piped instance
returns EMPTY video lists for every channel (MrBeast/LTT/Veritasium all 0). Also
fixed a real bug: _harvest bailed on an empty first page instead of following the
nextpage token.

So the proxy is now OPT-IN via a youtube_proxy_instances setting (empty by
default → skipped entirely, no wasted dead calls). The default path is RSS +
the parallelized yt-dlp fallback, which actually works (your log: 9/36/25 dates
cached). Power users can point the setting at a live instance if they find one.
2026-06-17 12:30:21 -07:00
BoulderBadgeDad
964b9bfdd7 YouTube enricher: align INFO output fully with the other video workers
Verified the three engine workers (tmdb/tvdb/omdb) log per-item INFO under
video_enrichment.worker. Matched the youtube enricher to that exactly:
- INFO is now purely per-item ('Dated <ch> '<video>' -> <date>' / 'No date
  for …') + one terse per-channel summary ('Dated N/M videos for <ch>'),
  mirroring 'Matched … -> TMDB ID' / 'Synced full episode list…'.
- The 'YouTube dates:'-prefixed phase/diagnostic lines (enriching / proxy
  returned / failures) demoted to DEBUG.
2026-06-17 12:23:13 -07:00
BoulderBadgeDad
220f2ad4de YouTube enricher: consistent log output with the other video workers
- Logger renamed video.youtube_enrichment -> video_enrichment.youtube (same
  namespace as video_enrichment.worker).
- Per-item lines as it dates each video — 'Dated <channel> '<video>' -> <date>'
  / 'No date for …', mirroring the workers' 'Matched <kind> '<title>' -> TMDB
  ID: …'. (Phase lines kept for context: enriching / proxy returned / done.)
2026-06-17 12:19:55 -07:00
BoulderBadgeDad
722b80001e YouTube enricher: parallelize per-video date fallback (~3x faster)
Your log showed 'proxy returned 0' (public proxies dead) → the slow per-video
path. Date those recent uploads in a 3-worker thread pool instead of serially,
so a channel finishes in ~30s rather than 1-2 min; cached as each completes.
2026-06-17 12:18:29 -07:00
BoulderBadgeDad
3235b196b7 YouTube enricher: observable + fast-fail proxy + live channel re-poll
Addresses 'says running but I see nothing':
- Real logs: 'enriching <ch>…', 'proxy returned N', 'done — N proxy + N
  per-video (X/Y dated)' so you can see exactly what it's doing.
- Proxy timeout 8s→4s so dead instances (most public ones) fail fast instead of
  hanging ~40s before the yt-dlp fallback.
- Channel page now re-fetches a few times (25/60/110s) after load while videos
  are still undated, re-rendering only when NEW year-seasons appear — so dates
  the background enricher fills in pop in without a manual reload.
2026-06-17 12:06:54 -07:00
BoulderBadgeDad
3a24fc24b1 YouTube worker: push status over the socket like the others (stop polling)
The date enricher was the odd one out — the dashboard polled
/api/video/enrichment/youtube/status every 3s (flooding the access log) instead
of using the shared WebSocket the other three workers push on. Now the existing
_emit_video_enrichment_status_loop also emits 'enrichment:youtube' (same stats
shape), and video-enrichment.js binds it on the socket alongside tmdb/tvdb/omdb.
Removed the bespoke polling loop. One-time prime fetch on load stays (instant
initial state); everything after is socket-driven. Consistent with the others.
2026-06-17 11:59:39 -07:00
BoulderBadgeDad
9c3facfe7e Enricher: don't spawn the background daemon under pytest (test isolation)
The enrichment daemon was starting during API tests (it uses the default DB +
network), touching a stray real DB and causing sqlite disk-IO flakiness. enqueue()
now no-ops when PYTEST_CURRENT_TEST is set; _enrich() is still tested directly
with a tmp DB. 176 video tests green.
2026-06-17 11:44:21 -07:00
BoulderBadgeDad
217486b3c7 Fix test_channel_enrichment_tracking for coverage-aware gate (thin run retries in 15m; good run honours window) 2026-06-17 11:38:52 -07:00
BoulderBadgeDad
581735ac7d YouTube enricher: coverage-aware retry + show in Manage Workers modal
- Fixes 'stuck idle': a channel marked enriched with FEW dates (proxies were
  down) was locked out for 24h, so the improved yt-dlp fallback never re-ran.
  channel_dates_enriched_recently now retries in 15 min when the last run got
  <15 dates, and skips 24h only after a good run. So the catalog actually fills
  in once a source works.
- The YouTube Dates worker now appears in the Manage Workers modal: added to
  WORKERS (red ▶), youtube-aware rail subtitle, and a dedicated simple panel
  (status + what-it-does + channels-enriched / dates-cached / queued counts) —
  no per-kind match queue. Pause/resume works; polls every 3s like the rail.
2026-06-17 11:37:26 -07:00
BoulderBadgeDad
9524a40615 YouTube enricher: reliable full coverage without a proxy + sweep existing follows
Two gaps from real use:
- When all proxy instances are down (common), the fallback only dated WISHED
  videos, leaving most as 'Earlier videos'. Now on proxy failure it flat-resolves
  the channel's recent uploads and dates THOSE via yt-dlp (throttled, cached) —
  so years populate fully even with no working proxy.
- A channel followed before the enricher existed was never queued ('nothing to
  process'). /youtube/channels now sweeps: any followed channel not enriched
  recently gets enqueued — so viewing the Channels tab (or the wishlist badge
  refresh) picks up existing follows.

The detail page still reads cache+RSS (fast); the enricher fills the cache. 51
tests green.
2026-06-17 11:23:34 -07:00
BoulderBadgeDad
b6d8f07516 Dashboard: YouTube date enricher as a 4th worker orb + Follow→Watchlist consistency
- The standalone enricher now reports orb telemetry: stats()/pause()/resume();
  /enrichment/youtube/status (+pause/resume) special-cased in the route. Added
  the 4th header worker button (red ▶), WORKER_DEFS entry, and SERVICES wiring —
  it's not on the socket so video-enrichment.js polls it every 3s. It animates
  (idle → active orb + inbound pulses) while a followed channel's dates are being
  fetched, like the TMDB/TVDB/OMDb orbs.
- Channel detail now uses the SAME standard watchlist button as shows/movies
  (library-artist-watchlist-btn, ✓/+ icon, 'In Watchlist'/'Watchlist') instead
  of a bespoke 'Follow' — consistency. Still wired to the channel follow action.

176 backend tests green; JS balanced; music untouched.
2026-06-17 11:05:59 -07:00
BoulderBadgeDad
30dc587ebf Background date enricher for followed YouTube channels (no key)
Follow a channel → a background job fetches its FULL upload-date catalog so the
channel page's year-seasons populate fully, cached permanently (one-time per
channel, instant after).

- core/video/youtube.py: proxy_channel_dates() — no-key BULK dates via Piped/
  Invidious instances (tries several, paginates); parse_proxy_dates handles both
  shapes. The clean path the user wanted.
- core/video/youtube_enrichment.py: YoutubeDateEnricher daemon — enqueue(channel)
  → proxy bulk → cache; per-video yt-dlp only as a throttled fallback (cap 60,
  0.4s) for wished videos when every proxy is down. Skips channels enriched in
  the last 24h.
- db: youtube_channel_enrichment table + mark/check + wishlisted_video_ids_for_
  channel. Enqueued from /youtube/follow and on opening a followed channel.

Scoped to FOLLOWED channels only (per request). 174 tests green; enricher imports
nothing from music.
2026-06-17 10:52:24 -07:00
BoulderBadgeDad
ec5af17de7 Channel year-seasons: RSS dates + persistent date cache (real years that fill in)
Flat listing has no upload dates, so channels showed one 'All Videos' season.
Now real year-seasons, filled from cheap sources and cached so they grow:
- core: parse_rss_dates / channel_recent_dates — one public RSS GET dates the
  ~15 most-recent uploads (no yt-dlp, no bot risk).
- db: youtube_video_dates cache table + cache_video_dates / get_video_dates.
- /youtube/channel merges cached + RSS dates onto the flat videos (year-seasons)
  and caches what it learns; /youtube/video caches each fetched date too — so
  expanding/wishing videos progressively fills the catalog's years, instant on
  repeat. Dateless tail groups as 'Earlier videos'. Missing-only toggle hidden
  for channels. 84 db + 84 youtube/api tests green.

Full day-one coverage still needs the YouTube Data API's publishedAt (optional,
yt-dlp can't match it cheaply) — RSS + cache is the no-key best.
2026-06-17 10:30:54 -07:00
BoulderBadgeDad
ece4fbc792 Channel detail: fix episode-row layout + graceful seasons
- The YouTube episode rows reused .vd-ep's 5-column grid (index·thumb·info·
  badge·chev) but only have 4 children, so the thumb squished into the index
  slot and the Wish button stretched across the info column. Give .vd-ep--yt its
  own 4-column grid (thumb·info·wish·chev).
- Flat listing omits per-video dates, so all videos bucketed into a lone
  'Undated' season. Now a single dateless channel shows one 'All Videos' season
  (year-grouping still kicks in for any videos that DO have dates); the 'Missing
  only' toggle is hidden for channels (videos aren't owned/missing).
2026-06-17 10:07:51 -07:00
BoulderBadgeDad
3d67f51c29 Search avatars: proxy with a UA + graceful initials fallback
Screenshot showed channel results with empty circles, not the initials chip —
i.e. an avatar URL was present but the image was failing to load. Two causes
addressed:
- The image proxy fetched googleusercontent/yt3 with NO User-Agent, which the
  CDN 403s for some avatars → blank. Now sends a browser UA + Accept header.
- A failed avatar now swaps to the initials chip (onerror → outerHTML) instead
  of hiding the img and leaving an empty circle. Also proxy protocol-relative +
  deeper-subdomain YouTube image hosts. img-proxy tests green.
2026-06-17 09:57:42 -07:00
BoulderBadgeDad
0ff0d07c22 Search UX: no 'No results' flash before YouTube loads + avatar fallback
- The big one: an empty TMDB result no longer flashes 'No results' while the
  (slower) YouTube channel search is still in flight — a 'YouTube channels ·
  searching…' skeleton group (shimmer cards + spinner) shows instead, and the
  empty state only appears once BOTH sources resolve with nothing. Users no
  longer click away thinking nothing's happening.
- Channel avatars: fall back to the singular 'thumbnail' field when the flat
  results omit the thumbnails list, so fewer cards are photoless (the rest keep
  the clean initials placeholder). Reduced-motion respected.
2026-06-17 09:42:02 -07:00
BoulderBadgeDad
284acc8591 Merge YouTube channels into the TV-show detail page
Per request: a channel now opens the SAME detail page as a TV show instead of a
separate page. A channel renders through the show pipeline — currentKind='show'
(so the show container resolves) with d.kind='channel' + d.source='youtube'
driving content. Every change is an additive 'youtube' branch; the show/movie
path is byte-for-byte unchanged.

- Data transform (ytToShow): upload YEAR = season, video = episode, channel
  banner=backdrop, avatar=poster (proxied), tags=genres, subs/videos/views=meta.
- Source-aware seams: bbBackdrop/bbPoster/seasonArt, billboard meta + actions
  (Follow + Open-on-YouTube), episodeRow (ytEpisodeRow: Wish toggle), episode
  expand (loadEpisodeExtra → /youtube/video full description+stats).
- Channel-only: playlists as a section below episodes (collapsible, lazy-load
  their videos with wish toggles); per-video wish syncs across all cards; Follow.
- Routing: kind=channel → navigate('video-show-detail'); deep-link
  /video-detail/youtube/channel/<id> still parses (string id). All TMDB-only
  sections (cast/ratings/providers/etc.) auto-hide on empty channel data.

Retired the standalone video-channel.js + its subpage. JS/CSS balanced; music
untouched.
2026-06-17 09:17:41 -07:00
BoulderBadgeDad
9084f1b7bb Search: include YouTube channel results alongside TMDB
A text search now also surfaces matching YouTube channels (not just the paste-a-
link path). search_channels() extracts the channels-filtered YouTube results
page (flat); API GET /youtube/search hydrates a 'following' flag. The search
page fires it in parallel with the TMDB search and appends a 'YouTube channels'
group when it returns (best-effort; never blocks/breaks TMDB results). Cards open
the in-app channel page. 82 youtube+api tests green; balanced; music untouched.
2026-06-17 09:06:41 -07:00
BoulderBadgeDad
0456fa893f Channel page (UI): stats+tags, sort/filter/load-more, inline video detail, playlists
The channel detail page is now best-in-class, treated like a real show:
- Hero: stats ribbon (subs · videos · views) + the channel's topic tags as chips.
- Video grid: sort (newest/oldest/most-viewed), 'Wished only' filter, and a Load
  more that pages deeper than the first 60 uploads.
- Click any video → inline panel with its full description + likes/views (lazy
  /youtube/video fetch, cached); wish state syncs across every card with that id.
- Playlists rendered as collapsible 'seasons' that lazy-load their videos on
  expand (each with its own wish toggle).

.vc-* CSS; JS brace-balanced; music untouched.
2026-06-17 08:54:37 -07:00
BoulderBadgeDad
87750e2464 Channel page (backend): channel tags/views + playlists + playlist videos
- shape_channel now carries tags (≤12) + channel view_count for the stats ribbon.
- Refactored entry-shaping into _shape_entries (shared by uploads + playlists).
- channel_playlists(id) → playlists (as 'seasons'); playlist_videos(id) → its
  videos. API: GET /youtube/playlists/<channel_id>, GET /youtube/playlist/<id>
  (with per-video wished hydration). 79 youtube+api tests green.
2026-06-17 08:51:53 -07:00
BoulderBadgeDad
bbba2c8225 Fix: clicking a watchlist channel opened the dashboard, not its detail page
video-channel-detail was in DETAIL_PAGES but missing from VIDEO_PAGES, so
pageMeta() fell back to VIDEO_PAGES[0] (dashboard). Register it so navigate()
resolves the channel detail subpage.
2026-06-17 08:43:43 -07:00
BoulderBadgeDad
9348166bb5 YouTube: proxy channel/video images so avatars + thumbnails always load
The channel avatar (and video thumbnails) are yt3.googleusercontent.com /
i.ytimg.com URLs; hotlink/CORS policy could blank them (failed <img> hides on
the watchlist, falls to initials on the wishlist orb) — which is why the poster
'vanished' on both pages.

Extend the /api/video/img proxy allowlist to YouTube CDN hosts (ytimg.com /
ggpht.com / googleusercontent.com, https-only, still SSRF-safe) and route all
YouTube art through it: VideoYoutube.img() helper used by the watchlist channel
cards, search chip, the wishlist nebula orb/season/still art, and the channel
detail avatar/banner/thumbnails. 2 img-proxy tests green; JS balanced.
2026-06-17 08:40:33 -07:00
BoulderBadgeDad
b3fd6037bb YouTube: fix missing channel poster on the wishlist orb
Flat channel listing doesn't always surface the avatar, so it could be stored
null → the orb fell back to plain initials (looked like a missing poster). Two
fixes:
- Orb falls back to the channel's newest video thumbnail when the avatar is
  absent, so it's never blank.
- Opening the channel page (which resolves the real avatar) now backfills it
  onto every wished row via set_wishlist_channel_poster — so the actual channel
  avatar appears on the wishlist orb thereafter. 6 tests green.
2026-06-17 08:32:47 -07:00
BoulderBadgeDad
591138a8a1 Wishlist: clear the info bar when an orb collapses
The synopsis + cast live in the orb's always-present header (.vwsh-xhead), not
inside the collapsing seasons block, so closing an expanded show/channel left
the selected episode/video's synopsis + cast photos lingering until you
re-opened and re-closed. Collapsing an orb now wipes both side columns and drops
any episode selection, so the :empty columns hide cleanly.
2026-06-17 08:24:32 -07:00
BoulderBadgeDad
22adc3bc55 YouTube: pull FULL per-video metadata (lazy) for the wishlist info bar
Flat channel listing can't return descriptions, so selecting a video showed 'No
description'. Now we do a non-flat single-video extract on demand — the way the
TV nebula lazy-loads guest stars:
- core: video_detail(id) / shape_video() — description, views, likes, duration,
  tags, channel, webpage_url (full extract, yt-dlp injectable for tests).
- API: GET /youtube/video/<id> — returns it AND backfills the description onto
  the wishlist row (set_wishlist_video_overview), so re-opening is instant.
- Wishlist info bar (youtube): on select, lazy-fetch → real description +
  eyebrow (date · duration · views) + a 'Watch on YouTube' link; cached on the
  episode object. 'Loading details…' while in flight.

Mirrors the episode lazy-load + art-backfill patterns. 73 youtube+api tests
green; brace/CSS balanced; music untouched.
2026-06-17 08:22:45 -07:00
BoulderBadgeDad
5ab3ec90a8 YouTube next-level (UI): channel detail page + wishlist YouTube tab = TV nebula
Two big pieces:

1) Wishlist YouTube tab now renders through the EXACT TV nebula (channel = show
   orb, upload YEAR = season, video = episode) instead of a flat list. Made the
   nebula source-aware: a youtube orb/season/label opens the in-app channel page
   (not tmdb); season name shows the year; episode meta shows the upload date;
   removes route through the youtube source_id endpoints (video / year / whole
   channel); the info bar shows a selected video's description (no cast/no tmdb
   fetch). Identical look — music wl-* + TMDB path untouched.

2) New in-app YouTube channel detail page (video-channel.js, sibling of
   video-person.js) — opens via open-detail {kind:'channel'} from the watchlist
   card, the wishlist orb/season, and deep links (/video-detail/youtube/channel/
   <id>; router now accepts string ids + the new page). Banner hero, avatar,
   subs/handle/video stats, description, Follow toggle, and a video grid where
   each upload can be wished individually (duration + views + watch-on-YouTube).
   Watchlist channel cards now open this page instead of bouncing to YouTube.

video-side.css .vc-*; JS brace-balanced; backend tests green.
2026-06-17 01:20:13 -07:00
BoulderBadgeDad
8941da8b60 YouTube next-level (backend): year=season nebula shape + channel detail API
- Resolver: capture banner_url + separate avatar (by thumbnail id), keeping
  per-video duration/views for the detail hero.
- query_youtube_wishlist reshaped to the EXACT TV-nebula shape: channel = show,
  upload YEAR = season (newest first), video = episode (newest=ep1 within a
  year), carrying source='youtube' + youtube_id + per-video source_id. Surrogate
  int returned as tmdb_id so the nebula's int keying just works.
- New API: GET /youtube/channel/<id> (full channel detail — meta + deeper
  uploads + following + per-video wished flags) and POST /youtube/wishlist/add
  (per-video wish). DB: youtube_video_wish_state, remove_one_video_from_wishlist.
- Tests updated for the nebula shape + banner + detail endpoints. 82 DB + 69
  api/youtube tests green.
2026-06-17 01:10:53 -07:00
BoulderBadgeDad
f4f40c161b Fix: video DB fails to initialize on upgrade (no such column: source_id)
The two new partial indexes (idx_video_wishlist_video/_channel on source_id /
parent_source_id) were in video_schema.sql, which runs via executescript()
BEFORE the column ALTERs. On a fresh DB the CREATE TABLE includes the columns so
it's fine, but on an existing (pre-bridge) DB the table already exists without
them, so 'CREATE INDEX ... ON video_wishlist(source_id)' threw 'no such column'
and the whole init rolled back — the video side wouldn't load at all.

Move those two indexes out of the schema into VideoDatabase._ensure_indexes(),
run AFTER _ensure_columns(). Regression test simulates a pre-source DB and
asserts the in-place upgrade + the youtube path both work. 81 DB tests green.
2026-06-17 00:56:06 -07:00
BoulderBadgeDad
b440f2bcdc YouTube channels (4/4): UI — paste-to-follow, channel feeds, watchlist cards
Visual-first slice ties it together (window.VideoYoutube shared helper, all
.vyt-* CSS — music wl-* and the TMDB vwsh-* nebula untouched):
- Search: paste a channel link (or @handle) and instead of a title search you
  get a YouTube Follow chip — avatar, title/handle, a strip of recent stills,
  and a Follow button that follows + wishes recent uploads in one click.
- Wishlist: new YouTube tab. Channel = collapsible header (avatar, count,
  open-on-YouTube, Unfollow), videos = a flat newest-first thumbnail feed with
  per-video remove. Tab badge + sub-count wired.
- Watchlist: new Channels tab — followed channels as avatar cards with a wished-
  video count and an unfollow control; count badge kept fresh across follows.

JS brace-balanced, CSS balanced, 66 youtube+API tests green.
2026-06-17 00:47:24 -07:00
BoulderBadgeDad
d30149db14 YouTube channels (3/4): API endpoints
api/video/youtube.py wired into the blueprint:
- GET  /youtube/resolve   — preview a pasted channel (meta + recent uploads) +
                            a 'following' hydration flag; no writes.
- POST /youtube/follow    — {url} or pre-resolved {channel} → follow + wish its
                            recent uploads (capped 30). Returns channel + counts.
- POST /youtube/unfollow  — {youtube_id} → un-follow (keeps wished videos).
- GET  /youtube/channels  — followed channels for the watchlist page.
- GET  /youtube/wishlist  — wished videos grouped by channel.
- POST /youtube/wishlist/remove — scope channel|video.

yt-dlp call lives behind resolve (injectable/mockable). 38 API tests green.
2026-06-17 00:38:11 -07:00
BoulderBadgeDad
e952ea5f66 YouTube channels (2/4): schema bridge + DB methods
Bridges YouTube onto the existing watchlist/wishlist tables (the chosen
approach) via a generic source/source_id (+ parent_source_id on wishlist) and a
stable surrogate for the NOT NULL tmdb_id, so existing dedup/group-by machinery
is untouched. SCHEMA_VERSION 13, additive column migrations.

- Channel follow  = video_watchlist kind='channel' (source='youtube').
- Wished video    = video_wishlist  kind='video' grouped by parent channel.
- youtube_surrogate_id(), add/remove/list/hydrate channels, add_videos_to_wishlist,
  query_youtube_wishlist (channel=group, videos=newest-first feed),
  youtube_wishlist_counts, scoped removal.
- Kept wishlist_counts/watchlist_counts byte-identical (exact-equality tests);
  YouTube counts live on their own method. 80 DB tests green.
2026-06-17 00:34:28 -07:00
BoulderBadgeDad
68383a16b3 YouTube channels (1/4): channel resolver in core/video/youtube.py
First slice of 'follow a YouTube channel as a show'. Pure, testable resolver:
- parse_channel_url() normalizes any channel reference (@handle, /channel/UC..,
  /c/.., /user/.., bare handle, with/without scheme or tab suffix) to a
  canonical /videos uploads URL, and rejects non-channels (watch/playlist/
  shorts/home/non-youtube).
- shape_channel() maps yt-dlp's flat-extract dict to our shape (channel meta +
  recent uploads), picking best-res thumbs, parsing sparse publish dates
  (timestamp or upload_date → ISO, None when flat mode omits them), filtering
  null/id-less entries.
- resolve_channel() ties them together via an injectable YoutubeDL factory.

yt-dlp only (already a pinned dep), extract_flat for cheap listing. 28 seam
tests, no network. Schema bridge + API + UI are the next slices.
2026-06-17 00:25:34 -07:00
BoulderBadgeDad
b7cafb8b72 Calendar: lock multi-hero height so hover doesn't change the DOM height
The 'Next up' multi-panel used min-height, so when hovering re-wrapped a
narrowed panel's title onto more lines the whole hero grew taller and nudged the
page. Pin it to a fixed height and clamp panel titles to 2 lines; mobile stacked
layout falls back to height:auto.
2026-06-17 00:06:32 -07:00
BoulderBadgeDad
6d63016566 Calendar: keep hero side panels >=10% wide + make Compact the default view
- The 2-3 'Next up' panels now floor at min-width 10%, so when one is expanded
  the others stay visibly selectable instead of collapsing to slivers (reset to
  0 in the mobile stacked layout).
- Compact is now the default calendar view (still overridable + remembered via
  localStorage).
2026-06-17 00:03:49 -07:00
BoulderBadgeDad
987cee7cb3 Calendar: view switcher with a Compact (art-hidden) view
Adds a Cards/Compact toggle next to the week filter. Compact drops the 16:9
episode art (.vcal-art) — the big space eater — and tightens the cells so a lot
more episodes fit on screen at once; ownership (which the art's ✓ badge used to
show) becomes a green left accent stripe on owned cells. Just toggles a class on
the stable grid wrapper (no refetch), and the choice persists in localStorage.

Calendar-only; music side untouched.
2026-06-17 00:00:28 -07:00
BoulderBadgeDad
7c1644384d Calendar: time-aware 'Next up' (up to 3, diagonal panels) + auto-scroll to now
The hero used to lock onto the first episode of the week and never move. Now:
- 'Next up' is time-of-day aware on the current week: it shows the soonest
  episodes that haven't finished airing yet (90-min grace so a show that's on
  right now stays up; streaming/undated shows count as 'anytime today' so they
  don't expire at midnight), advancing as the day goes. Once the week has fully
  aired it falls back to the most recent.
- When 2-3 are upcoming, the hero splits into diagonal clip-path panels — the
  soonest leads, hovering any panel expands it full-width and collapses the
  others (secondary panels tease the title; sub + actions fade in on expand).
  Single upcoming = the original billboard. Stacks vertically on mobile.
- Opening the calendar (or hitting Today) now auto-scrolls to the band the
  wall-clock is in, so you land on 'now' instead of pre-dawn.

Calendar-only; music side untouched.
2026-06-16 23:57:14 -07:00
BoulderBadgeDad
f383c82beb Wishlist TV: season as a left poster panel, episodes flow to its right
The season header was a full-width bar holding a tiny 34px thumb + 'Season N'
with the episode grid stacked below — most of the row was dead space. Now each
season is a real poster panel on the LEFT (94px poster + name + count + 'View
show ->' on hover, the whole panel → show page; ✕ to remove) with the episode
grid filling the space to its RIGHT (denser 232px cards). Reads tighter and the
season poster is finally a meaningful element instead of a thumbnail.

Scoped under .vwsh-nebula; music wishlist untouched.
2026-06-16 23:46:03 -07:00
BoulderBadgeDad
ce84e715c3 Wishlist TV: flank the poster (synopsis | poster | cast) + graceful empty states
Fixes the expanded-show design from the screenshot:
- The header is now a 3-column row that flanks the poster: synopsis left, poster
  middle, cast right — using the wasted horizontal space instead of stacking
  everything vertically and pushing episodes down. Each side column hides (:empty)
  when there's nothing, so a collapsed bubble or a cast-less reality show just
  shows the centered poster (nebula grid unchanged). Stacks on mobile.
- Quiet episode-still placeholder (faint icon) instead of loud diagonal stripes
  for episodes TMDB has no still for.

All scoped under .vwsh-nebula; music wishlist untouched.
2026-06-16 23:40:27 -07:00
BoulderBadgeDad
817de567ed Wishlist TV: episode cast = guest stars + show regulars (no longer empty)
Most episodes have no TMDB guest stars, so 'episode cast' showed nothing. A
selected episode now lists its guest stars first, then the show's regular cast
(deduped), so actors always appear; the guests just lead.
2026-06-16 19:16:51 -07:00
BoulderBadgeDad
efcaea314c Wishlist TV: fix episode cast 404 (season number was undefined in URL)
query_wishlist episodes carry episode_number but not season_number (it lives on
the season), so the guest-cast fetch hit /episode/<tmdb>/undefined/<ep> -> 404 and
the S·E eyebrow read 'S undefined'. findEpisode now stamps season_number onto the
selected episode.
2026-06-16 19:11:40 -07:00
BoulderBadgeDad
7ef794fe74 Wishlist TV: episodes grouped under clickable season headers; show/season -> show page
Per feedback:
- Removed the 'View show' button. The info bar is now strictly synopsis (left) +
  cast (right).
- Episodes are shown grouped under a season header that links to the show page;
  clicking the show title or a season header navigates to the show detail. Clicking
  an episode SELECTS it (drives the info bar: episode synopsis + guest cast),
  click again to go back to show-level.
- Dropped the season-tile reveal step (episodes always visible when expanded).

Scoped under .vwsh-nebula; music wishlist untouched.
2026-06-16 19:07:46 -07:00
BoulderBadgeDad
11cc9aa789 Wishlist TV: episode-select drives a contextual info bar + explicit View-show button
Reworked per feedback:
- Clicking an episode SELECTS it (highlights), no longer navigates.
- A dedicated 'View show ->' button in the info bar is what opens the show.
- Info bar is contextual: nothing selected -> show synopsis + show cast; episode
  selected -> that episode's synopsis (from the wishlist row) + its guest cast
  (lazy /episode/<tmdb>/<s>/<e>). Click the episode again to go back to show-level.
- Changing/closing a season resets the bar to show-level.

Cast bubbles + View-show navigate; everything scoped under .vwsh-nebula (music
wishlist untouched).
2026-06-16 18:57:37 -07:00
BoulderBadgeDad
853dbcf40f Wishlist TV: show synopsis + clickable cast bubbles in the expanded panel
Your idea — use the empty space when a show is expanded. Expanding an orb now
lazily loads the show detail and lays out an info bar: the show SYNOPSIS on the
left, a row of CAST bubbles on the right (circular photos / initials fallback,
clickable -> opens the person page). This surfaces reliable show-level context
even for reality shows whose episodes have no TMDB overview (e.g. Love Island).

Reuses the existing detail endpoints (owned -> /detail/show, else /tmdb/show);
one fetch per show, cached on the group. All scoped under .vwsh-nebula — music
wishlist untouched.
2026-06-16 18:36:05 -07:00
BoulderBadgeDad
e8a0e1256b Wishlist: episode synopsis + clickable cards + FIFO/newest sort
More data in the roomier episode cards (asked for): episodes now carry a synopsis
(new video_wishlist.episode_overview, SCHEMA_VERSION 12 + migration; captured at
add-time, and the art-backfill fills it for old rows from the same tmdb_season
call). The card shows a 2-line synopsis under the meta line and is now clickable
-> opens the show detail (episodes have no page of their own).

Organize the wishlist two ways: added 'Oldest first' (FIFO) alongside 'Recently
added' (newest) — query_wishlist gains the 'oldest' sort for both movies + shows.

Tests: FIFO/newest ordering (with pinned add-times) + overview roundtrip/backfill.
105 passed. Music wishlist untouched.
2026-06-16 18:21:26 -07:00
BoulderBadgeDad
4d62df5c33 Wishlist TV: roomier episode cards + full-width episode area + polish
From the expanded-show screenshot: episodes were crammed into a 320px tile and
titles truncated. Reworked (all still scoped under .vwsh-nebula):

- Season tiles are now SELECTORS. Picking one drops its episodes full-width below
  the fan as 2-line cards: a bigger 16:9 still, the title wrapping to two lines,
  and a meta line (status dot · S/E · air date). Single-select; click again to
  close. Responsive grid, scrolls past ~360px.
- Softer count badge ('59 ep' muted pill instead of the loud number).
- Selected tile glows in the show hue; square bubbles bumped up so posters breathe.

Music wishlist untouched (verified: no bare .wl-* rules in video CSS).
2026-06-16 18:09:05 -07:00
BoulderBadgeDad
e5cd5cf08c Wishlist: real season posters (not the show poster) + extend backfill
Season tiles showed the show poster because the wishlist had no season art. Now
stored + used: new video_wishlist.season_poster_url (SCHEMA_VERSION 11 + migration);
the get-modal captures each season's poster at add-time (owned -> /poster/season
proxy, tmdb -> direct); query_wishlist exposes season.poster_url; tiles render the
real season poster (falling back to show poster, then placeholder).

Backfill generalized: /wishlist/backfill-stills -> /wishlist/backfill-art fills
BOTH stills and season posters from the same cached tmdb_season call (it already
returns the season poster). The page fires it once when either is missing.

Tests updated/added: art backfill targets + season-poster set + endpoint. 104 passed.
2026-06-16 17:42:12 -07:00
BoulderBadgeDad
2985514627 Wishlist: backfill episode stills for rows added before still-capture
The 'no episode images' was data, not a bug: existing episode rows predate
still-capture (81 rows, 0 stills). New /wishlist/backfill-stills fills them
cheaply — one cached tmdb_season call per (show, season), updating only rows
that lack a still. The show tab fires it once automatically when it sees missing
stills, then reloads so the images pop in.

DB: wishlist_still_backfill_targets + set_wishlist_still (won't clobber existing).
Tests: +2. Suites green.
2026-06-16 17:33:59 -07:00
BoulderBadgeDad
f7bd15d019 Wishlist TV: #4 progress bar + episode stills + square video bubbles
#4 acquisition progress: a thin done÷wanted bar across each bubble's bottom
(forward-prep — fills once the download engine lands).

Expanded-view richness: episodes now carry a still thumbnail. New video_wishlist
.still_url column (SCHEMA_VERSION 10 + migration); the get-modal captures the
still per episode (owned -> /poster/episode proxy, tmdb -> direct still_url) and
sends it through add; query_wishlist returns it; the episode row renders a 16:9
thumb (film-frame placeholder when absent) in a roomier expanded tile.

Subtle video identity: the bubbles are now rounded-SQUARES (the music orbs stay
circles). Every rule stays scoped under .vwsh-nebula — verified no bare .wl-*
rules in the video CSS, so the music wishlist is untouched.

Tests: +1 (still roundtrip). Backend 102 passed.
2026-06-16 17:27:16 -07:00
BoulderBadgeDad
599f36fdf3 Wishlist TV nebula: cinematic expand + season tags + rich tracks + sort
Next-level pass for the video nebula, ALL scoped under a video-only .vwsh-nebula
class so the music wishlist's global wl-* styling is untouched (verified: no bare
.wl-* rules in the video CSS).

1. Cinematic expand — an open orb bleeds the show's poster as a blurred, hue-
   tinted backdrop behind the season fan + glows the panel in the show's hue.
2. Season tags — each season tile stamps a bold 'S2' over its art so seasons
   read distinctly instead of identical posters.
3. Richer episode tracks — every episode line gets a colored status dot
   (wanted/searching/downloading/done/failed) + its air date.
4. Sort + count — a Recently added / Most wanted / A–Z sort (query_wishlist gains
   a sort param) and a live 'N shows · M episodes' subheader.

Tests: +1 (sort ordering). Backend 101 passed. Movies tab + music side untouched.
2026-06-16 17:16:59 -07:00
BoulderBadgeDad
d90ae493dd Wishlist TV: adopt the music 'Nebula' (show=orb, season=album, episode=track)
Ditched the filmstrip. The TV tab now renders the exact music wishlist nebula by
reusing its global wl-* classes: shows are glowing orbs sized by wanted-episode
count, click to expand into a season 'album fan' (tiles), click a season tile to
reveal its episodes; removes at episode/season + a show-remove × on the orb.
Everything's visible at once, no horizontal scrolling. Movies tab unchanged.

Only video-specific CSS is the show-remove button; behavior (orb/tile expand,
opens, removes) is wired via delegation in video-wishlist.js. Easy to tweak from
this shared base.
2026-06-16 17:05:51 -07:00
BoulderBadgeDad
1b456d548a Wishlist TV: filmstrip reels (replace the lame collapsible tree)
The music wishlist has its artist 'nebula'; the TV side now has its own rich,
TV-native metaphor. Each show is a reel: poster + one horizontal film strip per
season (dark band with sprocket-hole borders) made of episode 'cells'. A cell
shows E#, glows in the show's hue, and widens on hover to reveal the title;
remove ×s sit on each cell, each season label, and the show header. Poster/title
open the show detail. Status tints the cells (wanted/downloading/done/failed).
Movies tab unchanged.

query_wishlist(show) now also returns library_id so reels open the owned detail
when applicable. Backend suites green.
2026-06-16 16:54:19 -07:00
BoulderBadgeDad
bae8060c09 Calendar: 'Add missing to wishlist' catch-up button
A manual safety-net for the auto-promoter: queues episodes that have ALREADY
aired, are missing, and aren't yet on the wishlist. Upcoming episodes are left
alone (the calendar promotes them once they air), so it's a no-op on the current/
future weeks and useful when you page back to a past one.

- calendar_upcoming now returns the show's tmdb_id.
- /wishlist/check accepts {shows:[...]} -> by_show membership (db.wishlist_keys_
  for_shows), so the button only counts/adds what's genuinely not yet queued.
- Calendar: computes aired-missing (air_date < today, !has_file), checks wishlist
  membership, shows 'Add N missing to wishlist' when there's net-new; click groups
  by show -> /wishlist/add, toasts, fires soulsync:video-wishlist-changed, recomputes.

Tests: +2 (wishlist_keys_for_shows, /wishlist/check by_show). Backend: 100 passed.
2026-06-16 16:33:48 -07:00
BoulderBadgeDad
d3c875919f Wishlist page: header identical to the watchlist (kill the harsh glow)
My head glow was unblurred + mis-centered, so it read as a harsh purple band
with a gap above. Mirror the watchlist's exact head/glow/tabs/toolbar: blurred
accent radial clipped by the header, 36px title, pill tabs with count badges,
matching gutters + 900px breakpoint. Same chrome, different content.
2026-06-16 16:23:21 -07:00
BoulderBadgeDad
772703464b Wishlist page: match the watchlist container (full-width + 40px gutters)
It used max-width:1180 centered with no inner padding, so wide screens got big
empty side margins and the content looked squooshed. Mirror .vwlp-* instead:
full-width .vwsh-page, head/toolbar/body padded 40px (18px on small screens),
same head glow.
2026-06-16 16:18:17 -07:00
BoulderBadgeDad
bbd081c44b Video get-modal: wire 'Add to Wishlist' to real writes
No longer a stub. The modal now captures the title's tmdb id + per-episode meta
(title/air_date, including lazily-loaded tmdb seasons), and on submit:
- movie  -> POST /wishlist/add {movie} (owned -> 'queued for re-download')
- show   -> POST /wishlist/add {show, episodes:[selected S/E]}; if the 'Add to
  watchlist' tick is on, also POST /watchlist/add for the show.
Fires soulsync:video-wishlist-changed (+ watchlist-changed) so badges/pages
refresh, toasts the result, and closes. Every card surface (discover, search,
library, watchlist, detail) funnels through here, so they're all live now.
2026-06-16 16:14:19 -07:00
BoulderBadgeDad
261416fede Video wishlist: the page (Movies grid + grouped TV tree)
Tabbed Movies / TV page (mirrors the watchlist chrome). Movies render as a poster
grid with status pill + hover remove. TV groups into collapsible show -> season ->
episode rows with wanted/done roll-ups and a remove (x) at every level (episode /
season / whole show). Server-paged + searchable; updates the nav + hero badges
and listens for soulsync:video-wishlist-changed. Movie cards open detail.

Wires the pre-existing Wishlist nav button (added its badge) + subpage container +
.vwsh-* styles.
2026-06-16 16:11:45 -07:00
BoulderBadgeDad
6d3a59c8dc Video wishlist: API + dashboard count
- api/video/wishlist.py: GET /wishlist (paged movie|show tab, or counts-only),
  /wishlist/counts, POST /wishlist/add (movie OR show+episodes), /wishlist/remove
  (scope movie|show|season|episode), /wishlist/check (hydration). Registered in
  the blueprint.
- Dashboard 'wishlist' stat now reflects the real curated count (was a 0 stub).

Tests: +6 API (add movie/episodes, body validation, scoped removes, hydration,
routes registered). API suite 30 + DB suite 68 passing.
2026-06-16 16:07:28 -07:00
BoulderBadgeDad
110f23f555 Video wishlist: schema + DB layer (movies + episodes)
The curated 'get this' list. Atomic units are movies and episodes; adding a
whole show/season expands into episode rows (show/season are bulk ops). Upcoming
episodes stay out — the watchlist/calendar promote them once they air.

- video_wishlist table + two partial unique indexes (one per movie tmdb_id, one
  per (show tmdb_id, season, episode)) so the shapes don't collide and re-adds
  upsert. SCHEMA_VERSION 8 -> 9 (executescript creates it on existing DBs).
- DB: add_movie_to_wishlist, add_episodes_to_wishlist (bulk), remove_from_wishlist
  (movie/show/season/episode scope), query_wishlist (movies | shows grouped
  show->season->episode w/ wanted/done roll-ups, searched+paged), wishlist_counts,
  wishlist_state (hydration).

Tests: +7 (idempotent upserts, show-tree grouping, scoped removes, movie/episode
same-tmdb don't collide, hydration, search+paging). DB suite: 68 passed.
2026-06-16 16:03:25 -07:00
BoulderBadgeDad
246c650db4 Discover: fix grid pagination + session cache + responsive pass
Pagination (regression): the IntersectionObserver sentinel only fires on
intersection *changes*, so a short first page kept it on-screen and it never
re-fired — stuck at 20 — and I'd hidden the Load more button whenever IO exists,
leaving no fallback. Now:
- Load more button is always shown while there's more (the reliable control).
- Auto-load is self-correcting: track sentinel visibility and, after each load,
  pull again via rAF if it's still on-screen (rAF lets the observer update first
  so a cached category doesn't load every page at once).
- page increment moved inside loadGrid so the button + sentinel can't double-bump.

Caching: cachedFetch() memoizes /discover/list responses per URL for the session
(rails + grid pages) — revisits, paging and reopening a category are instant and
don't re-hit TMDB.

Responsive: small-screen pass for the Browse panel, hero (height/title/actions
full-width), grid header wrap, and the trailer close button (kept on-screen).
Also fixed the ambient layer's 130% width that could cause horizontal scroll.
2026-06-16 15:44:36 -07:00
BoulderBadgeDad
80d504f94d Discover: meaningful filter colors + drop the asymmetric edge-fade
Colors now carry meaning instead of cycling by position:
- segments (Movies/TV, sort) → the app accent (they're modes, not categories)
- genre chips → a thematic colour per genre (Horror red, Comedy gold, Sci-Fi
  cyan, Romance pink, …) via a name→colour map; unmapped fall back to neutral
- provider chips → each service's brand colour (Netflix red, Disney+ blue, Max
  purple, Hulu green, …)
- era chips → a single warm amber; 'All/Any …' reset chips → neutral grey

Edge-fade: removed the mask-image from .vdsc-rail and .vdsc-chips. On short
filter rows only the left fade landed (dimming the first chip) while the right
fell on empty space — reading as a one-sided fade over everything.
2026-06-16 15:26:12 -07:00
BoulderBadgeDad
b316d283f7 Discover: fix filter buttons rendering as bare text
The --c palette triples were space-separated (29 185 84), so rgba(var(--c), a)
expanded to the invalid rgba(29 185 84, a) and every fill/border/color was
dropped — leaving plain text. Match the codebase convention (--accent-rgb is
comma-separated) by making --c comma-separated, plus the rgba fallbacks.
2026-06-16 15:16:48 -07:00
BoulderBadgeDad
a1b9f8a827 Discover: restyle filter controls to the music album-detail button vibe
The fully-round accent pills + sliding-thumb segments didn't fit. Now every
filter control (kind/sort segments + genre/provider/era chips) shares the
album-detail action-button look: rounded-rect (9px), each tinted from a vibey
8-colour palette that cycles across the row (green/purple/blue/amber/pink/cyan/
coral/violet) — soft fill + border + coloured text, brightening on hover, the
selected one filling with its own colour. Dropped the sliding-thumb segmented
control + its now-dead JS (moveSeg/positionSegs).
2026-06-16 15:11:20 -07:00
BoulderBadgeDad
2b3ca12310 Discover: streaming-provider filter + infinite scroll + a11y polish
Provider filter (#4):
- client.discover() + engine.discover_filter() take a TMDB provider id and pass
  with_watch_providers + watch_region (engine._region) + flatrate. Browse gets a
  streaming-service chip row (Netflix/Prime/Disney+/Max/Apple TV+/Hulu/Paramount+/
  Peacock); the grid title reflects 'on <service>'.

Infinite scroll (#6):
- Grid paginates via an IntersectionObserver sentinel (600px lookahead) with a
  bottom spinner; the Load more button stays only as a no-IO fallback.

Polish (#7):
- Hero keyboard nav (←/→ when Discover is the visible view, ignoring inputs and
  while the trailer is open); focus-visible rings on chips/segments/cards/arrows.

Note: 'complete-the-franchise' rail (#5) needs a collection_id per movie, which
the schema doesn't store yet — deferred (would need an enrichment pass).

Tests: +1 (provider watch_region params); updated the discover_filter fake for
the new kwargs. Enrichment + API suites: 115 passed.
2026-06-16 14:55:13 -07:00
BoulderBadgeDad
a9ec025706 Discover: 'More like…' rails + in-app hero trailer
More-like rails (real personalization beyond genre):
- db.random_owned_titles() seeds from a few random owned titles (with tmdb_id);
  client.recommendations() + engine.recommendations() (cached + owned-annotated)
  fetch TMDB recs. New /discover/morelike interleaves movie/show seeds (max 3
  rails) and the page prepends them, pre-filled, above the rail stack with a
  gradient 'for you' title.

Hero trailer (cheap big visual win):
- client.video_trailer() (light /videos call, Trailer over Teaser) + engine
  .trailer() (day-cached) + /discover/trailer. Hero gets a 'Trailer' button that
  opens an in-app YouTube lightbox (autoplay, Esc/backdrop close, pauses the
  slideshow) — nothing leaves SoulSync.

Tests: +11 (recs parse/annotate/cache, trailer pick+fallback+cache, random owned
seeds owned-with-tmdb only, /discover/trailer + /discover/morelike endpoints).
Enrichment + API suites: 114 passed.
2026-06-16 14:44:44 -07:00
BoulderBadgeDad
5aa792d399 Discover: sliding segmented pills + chip polish + no-TMDB state
- Segmented controls (Kind/Sort) now have a highlight 'thumb' that springs
  between options (JS measures the active button → CSS var slide); repositions
  on click, page-show, and resize.
- Chips: brighter active gradient with a glow ring + subtle lift, smoother hover.
- No-TMDB empty state: genres are a static TMDB endpoint, so when they come back
  empty the page shows a 'Discover needs TMDB' card instead of a bare shell.
- Persist 'Hide owned' across sessions (localStorage); async image decoding on
  cards.
2026-06-16 14:21:59 -07:00
BoulderBadgeDad
8043db0ad4 Discover: redesigned Browse panel + more data per rail
Browse panel (was boring native dropdowns):
- Kind + Sort are now segmented pill controls; Genre + Era are horizontally
  scrollable, edge-faded chip rows with an accent-glow active state; primary
  'Browse all →' CTA. Genre chips rebuild when kind flips. Selection lives in
  state.sel (no <select> reads).

More data per rail (so 'Hide owned' doesn't gut a shelf):
- /discover/list gains a 'pages' param (1–3): fetches that many consecutive
  TMDB pages and concatenates them deduped in one response. Rails request
  pages=2 (~40 items); trending is a fixed list so extra pages are skipped.

Cleanup: removed dead .vdsc-filterbar/.vdsc-select CSS.

Tests: +3 (discover routes registered; multi-page concat+dedup; trending
fetched once despite pages). API suite: 22 passed.
2026-06-16 14:13:27 -07:00
BoulderBadgeDad
5c9b43a175 Discover v2: per-category pagination, personalization, hide-owned + perf
Performance:
- Batched ownership: new db.library_ids_for_tmdb() resolves a whole rail in one
  query per kind. _stamp_owned (now also used by search + trending) groups by
  kind, so a full Discover page drops from ~500 connections to a couple per rail.

Function/data:
- 'See all' on every rail opens it as a paged grid (Load more); the filter bar's
  Browse routes through the same generic category grid with a back button + title.
- Personalized 'Because you like <Genre>' rails seeded from your most-owned
  genres (new db.top_owned_genres + /discover/taste endpoint).
- 'Hide owned' toggle drops in-library titles from every rail/grid (CSS class,
  instant).

Visual vibes:
- Ambient page-top color bleed that follows the current hero slide's hue.
- Rail edge-fade mask, gentle fade-in on load, per-title hue glow on card hover.

Tests: +4 (batched id map, server scoping, one-query-per-kind stamp, top genres).
Full video enrichment + database suites: 145 passed.
2026-06-16 14:02:44 -07:00
BoulderBadgeDad
97a2023139 Video: prefetch first season in get-modal + seam tests for discover
- Get-modal: prefetch the first real (un-owned) season on open so its first
  expand is instant and its missing episodes pre-count in the footer (prefers
  Season 1 over Specials).
- Tests: 10 new seam tests for the Discover data layer — TMDBClient curated/
  discover/genres parsing (forced kind, decade date-range, tv first_air_date_year,
  backdrop+overview), and engine discover_curated/discover_filter/genre_list
  (owned annotation, caching, kind normalization, disabled-worker + error
  swallowing). Full video enrichment suite: 79 passed.
2026-06-16 13:44:44 -07:00
BoulderBadgeDad
a8ebc9b8f7 Video Discover page: trending hero + lazy genre/decade rails + filter grid
A browse-everything page for TMDB titles you don't own yet.

Backend:
- TMDBClient: discover() (genre/year/decade), curated() (popular / top_rated /
  now_playing / upcoming / on_the_air / airing_today), genres(); shared
  _disc_map() that carries backdrop + overview. trending() now routes through it.
- Engine: discover_curated / discover_filter / genre_list (cached + owned-
  annotated via library_id_for_tmdb) + _stamp_owned helper.
- api/video/discover.py: /discover/hero, /discover/genres, /discover/list
  (curated key | filtered kind+genre+year+decade+sort, paged).

Frontend (video-discover.js + .vdsc-* CSS + subpage markup):
- Cross-fading trending hero slideshow (per-title hue, dots, hover-pause,
  reduced-motion safe, click -> detail).
- Deep stack of Netflix-style rails (curated + every movie genre + decades),
  each lazy-loaded on scroll via IntersectionObserver, arrow scrollers,
  reusing the search card (owned 'In Library' / 'Preview' ribbon + get button).
- Filter bar (kind / genre / decade / sort) flips into a paged 'Load more' grid.
- Cards open detail through the shared soulsync:video-open-detail event.
2026-06-16 13:35:31 -07:00
BoulderBadgeDad
4b7a915d39 Video get-modal: lazy-load episodes for un-owned (tmdb) shows
A show you don't own ships its seasons with no episodes (loaded per-season
on expand, like the full detail page), so the modal mis-read '0 episodes'
as 'all owned' and seasons expanded to nothing.

- Render tmdb seasons as collapsible with an 'expand to load' placeholder;
  fetch /api/video/tmdb/show/<id>/season/<n> on first expand, render rows,
  pre-select the missing-aired episodes, enable the season select-all.
- Skip the 'you have every episode' hint when any season is lazy/un-owned;
  lazy seasons aren't tagged owned, so they stay visible under 'Missing only'.
- Next-episode line falls back to the payload's next_episode stub when no
  episodes are loaded (the only next-up source for un-owned shows).
2026-06-16 12:57:42 -07:00
BoulderBadgeDad
0071358125 Video get-modal: poster + ratings bar + owned-movie handling + next-up
- Owned movie shows 'In your library' (with best version quality) and the
  footer flips to 'Re-download' instead of '+ Add to Wishlist'.
- Poster thumbnail floats over the hero (hue halo + own rise animation),
  works for both library and TMDB sources.
- Ratings strip: branded IMDb / Rotten Tomatoes / Metacritic chips from the
  existing payload fields; renders only what's present.
- Airing shows show the soonest upcoming episode above the selector.
2026-06-16 12:46:07 -07:00
BoulderBadgeDad
d722224b26 Video get-modal: use the standard SoulSync buttons (discog-submit/cancel)
The footer's 'Add to Wishlist' + 'Full page' now use the same buttons as the
artist-detail / Download-Discography modal — .discog-submit-btn (primary, ⬇ +
label) and .discog-cancel-btn (secondary) — instead of the custom vgm buttons,
for consistency across the app. updateFooter sets the label span (keeps the
icon). Per-title hue glows stay on the hero/title/ambient.
2026-06-16 12:19:21 -07:00
BoulderBadgeDad
02e3b9d2e4 Video get-modal: premium vibe pass — per-title hue glows + motion
Each modal now glows in its own colour (stable hue hashed from the title, set as
--vgm-h):
- ambient hue halo around the modal + a hue-tinted hero scrim
- slow Ken Burns drift on the backdrop + a drifting light sweep across the hero
- staggered content rise on open; spring entrance for the modal
- title glow; hue-aware gradient CTA with lift + glow on hover
- episode rows grow an accent edge on hover; close button spins
- the 'Add to watchlist' row breathes a soft accent glow to invite the tick
All guarded by prefers-reduced-motion.
2026-06-16 12:15:18 -07:00
BoulderBadgeDad
13d44f3ba0 Remove download button and logic from video modal
Remove the 'Download' button and all related handling from the video-get modal. The diff deletes the data-vgm-download button markup, removes download branching in the click handler and the isDl flag, and stops updating download text/disabled state in the UI refresh. The wishlist path is now the single visual flow (toast always reports Wishlist + optional Watchlist); actual write and download-from-wishlist behavior will be implemented later.
2026-06-16 12:11:05 -07:00
BoulderBadgeDad
9a5d187c0d Video get-modal: fix owned-season dead-end under 'Missing only'
Fully-owned seasons used to render with a disabled checkbox and expand to
nothing (every episode hidden by the filter) — looked broken. Now:
- Fully-owned seasons are HIDDEN while 'Missing only' is on (the filter doing
  its job); turn it off to see + re-download them (owned episodes are
  selectable). Season meta reads 'owned · N eps' when shown.
- If you own EVERY episode, a hint appears: 'You have every episode. Turn off
  Missing only to re-download.' (instead of a blank list).
Partial seasons (with gaps) show + expand as before.
2026-06-16 11:55:59 -07:00
BoulderBadgeDad
419b939999 Video get-modal: owned episodes re-downloadable + 'Add to watchlist' offer
- Owned episodes are now SELECTABLE (checkbox, not pre-checked) so you can
  re-download a season/episode you already have; flip 'Missing only' off to see
  them. Every season (incl. fully-owned) gets a select-all. Select-all + the
  season checkbox state respect the 'Missing only' filter (no silently grabbing
  hidden owned episodes).
- Airing show you don't follow yet → an accent 'Add to watchlist' tickbox
  (default on) so grabbing episodes also starts you watching for new ones.
  Hidden for ended shows + shows you already follow. The Add-to-Wishlist stub
  toast notes '+ Watchlist' when it's ticked.
2026-06-16 11:42:57 -07:00
BoulderBadgeDad
e24e9bfe1e Video get-modal: TV episode selector (evolved Download Discography) + Download btn
Shows aren't wishlist items — episodes are. So the show modal now grows an
inline season/episode selector below the overview:
- Collapsible season cards (season-level select-all + 'N missing · M eps').
- Per-episode rows with three states: owned (locked ✓, 'In library'),
  upcoming (locked ◷, air date — these belong to the watchlist), and
  missing-aired (accent checkbox, PRE-SELECTED).
- 'Missing only' toggle (default on) hides owned to cut noise; live selected
  count; season checkboxes go indeterminate as you pick episodes.

Footer is now [count] · [Full page] · [Download N] · [Add N to Wishlist].
Movies keep a simple [Download] · [Add to Wishlist]. Add/Download are visual
stubs (toast the count) — the curated video_wishlist + download path come later.

Data is the existing show_detail seasons/episodes (owned + air_date).
2026-06-16 11:30:58 -07:00
BoulderBadgeDad
35957b426d Video: airing shows get BOTH the watchlist eye AND the get/download button
An airing show is ongoing (follow for new episodes) AND has a back-catalog you
may be missing (acquire) — so it needs both controls, not one or the other.
VideoGet.cardButton() now returns a control GROUP:
  person -> eye | movie -> get | airing show -> eye + get | ended show -> get
Library + watchlist-page cards route through it (the other surfaces already did).
New .vcard-ctrls wrapper lays the buttons out top-right; hover-reveal unchanged.

(Before, only ended shows showed the download button — which is why airing
shows looked like they had no way to acquire missing episodes.)
2026-06-16 11:07:53 -07:00
BoulderBadgeDad
638269d870 Video watchlist: rename the default sort 'Following' -> 'Default'
'Following' read like a filter (everything on the watchlist is followed); it's
just the default sort (explicit follows first, then airing-default shows A-Z).
2026-06-16 11:00:00 -07:00
BoulderBadgeDad
21163239da Video detail page: follow control uses the new curated watchlist
The detail actions row still toggled the old shows.monitored flag via
/api/video/monitor. Now it's consistent with the cards:
- The 'Watchlist' button appears only for AIRING shows (movies + ended shows are
  terminal — they keep acquisition, not a watch-follow).
- It reads/writes the new /api/video/watchlist add/remove, resolves the real
  watched state on load (airing library shows are on by default), confirms on
  remove (the standard dialog), toasts, and broadcasts the change so the nav
  badge + watchlist page stay in sync.
2026-06-16 10:44:59 -07:00
BoulderBadgeDad
240e386871 Video: wire the eye/get button onto every card surface
A shared VideoGet.cardButton({kind,tmdbId,libraryId,title,poster,status,source})
picks the right control (person->eye, movie->get, show->eye when airing/unknown
else get) so every surface stays consistent. Injected into person filmography
(known-for + credits), search results (titles + people), and detail-page cast +
similar/collection rails. Each renderer hydrates watched state after render;
card roots get position:relative so the button anchors; reveal on hover.
2026-06-16 10:41:07 -07:00
BoulderBadgeDad
964675443b Video watchlist: nav badge with the live count
Adds a .dl-nav-badge to the Watchlist nav entry, updated from
/api/video/watchlist/counts on boot, on every watchlist change, and whenever
the page loads. Hidden at 0.
2026-06-16 10:20:33 -07:00
BoulderBadgeDad
58ceca86b1 Video watchlist: richer show cards (status pill + ep count) + sort dropdown
- Backend: effective shows now carry status + owned/total episode counts (joined
  off the shows table); query_watchlist gains a sort (default | title | added).
- Cards: a status pill (Airing / Upcoming / Ended) top-left + '12/20 eps' meta
  under the title for shows.
- Toolbar: a sort select (Following / A-Z / Recently added) next to search.

82 video tests green.
2026-06-16 10:19:13 -07:00
BoulderBadgeDad
1b23b7da78 Video: contextual get-symbol button + detail/download modal (visual only)
The terminal-content counterpart to the watchlist eye. On library cards:
- airing show  -> watchlist eye (monitor for new episodes)
- movie / ended show -> a 'get' download symbol that opens a detail modal

- video-get-modal.js: VideoGet.btn() + VideoGet.isAiring() (the shared status
  test), and the modal — hero backdrop, eyebrow, title, meta (runtime/rating/
  tagline), genres, overview, pulled from the existing detail endpoint. Action
  buttons are VISUAL STUBS for now: 'Open full page' navigates; 'Add to
  Wishlist' just toasts 'coming soon' (real population is a later phase).
- query_library now selects s.status so cards can pick eye vs get.
- CSS for .vget-btn (hover-reveal, accent on hover) + the .vgm-* modal, styled
  to match the calendar episode modal.

82 video tests green (status is an additive column).
2026-06-16 10:11:16 -07:00
BoulderBadgeDad
4287385af6 Video watchlist page: server-paged + search bar (like the library)
The page rendered every follow + airing-default show at once (DOM + all posters)
— slow once the watchlist grows. Now it pages like the library:

- /api/video/watchlist?kind=&search=&page=&limit= returns {items, pagination,
  counts}; query_watchlist() filters by title + slices (effective list is
  bounded, so compute-then-slice, not heavier UNION SQL).
- Page reworked to a single grid: Shows/People tabs each load their own page;
  debounced search box; Prev/Next pager; tab badges show totals from counts.
- Only a page of cards (and lazy posters) render at a time.

4 tests added (DB paginate/search + endpoint). 82 video tests green.
2026-06-16 09:40:41 -07:00
BoulderBadgeDad
b344303f75 Watchlist eye: confirm before removing (the standard yes/no dialog)
The eye is small/easy to mis-click and the card vanishes on the watchlist page,
so removal now goes through the shared showConfirmDialog (core.js) — 'Remove
"<title>" from your watchlist?' — before un-following. Adding stays one-click
instant. Cancelling re-enables the button with no change.
2026-06-16 09:29:31 -07:00
BoulderBadgeDad
d49eb967a4 Fix: watchlist card click did a full page reload (~15s freeze)
The watchlist cards were bare <a href='/video-detail/...'> with no click
interceptor, so clicking one triggered a REAL browser navigation to the
client-routed URL — the server returns the app shell and the entire app
re-boots (every static asset re-fetched = the ~15s 'restart' freeze).

Mirror video-library.js: intercept plain left-clicks on the cards and dispatch
soulsync:video-open-detail {kind, id, source} for in-app SPA navigation instead.
Encodes the open target on each card (library shows by library id, people +
un-owned shows by tmdb id). Modified clicks (new tab) still use the real href;
the eye button's capture-phase handler already stops its own clicks.
2026-06-16 09:16:18 -07:00
BoulderBadgeDad
248e2c32f2 Video dashboard: curated watchlist count + clear the wishlist for now
The dashboard still read the old monitored-based views: watchlist from
v_watchlist (every monitored show) and wishlist from v_wishlist (every missing
movie/episode, since monitored defaults to 1). Repoint both:
- watchlist -> the curated watchlist_counts() total (follows + airing default).
- wishlist  -> 0 for now. The auto-everything v_wishlist isn't the intended
  curated wishlist; zero it (no live-DB mutation) until 'add to wishlist'
  population lands, then repoint at the real source.

Test updated to the new semantics (airing show counts; wishlist cleared).
2026-06-16 09:09:10 -07:00
BoulderBadgeDad
ffaa36105b Video watchlist: actively-airing library shows are watched by default
Owning a still-running show means you want its new episodes, so it's on the
watchlist without a click. Implemented as a computed default + explicit-override
so it stays correct:

- video_watchlist gains a 'state' column: 'follow' (explicit) | 'mute' (a
  tombstone — user un-followed an airing show that's on by default, so the
  default must not silently re-add it).
- Effective watchlist (list/state/counts) = explicit follows  ∪  library shows
  whose status isn't ended/canceled, minus mutes. Computed at READ time, so it
  always tracks the library + a show's status — no scanner hook, no re-seeding.
- remove() now writes a mute tombstone (idempotent) instead of deleting; add()
  sets state='follow' and clears any mute. Scoped to the active video server.

The existing library-card eye now paints 'watched' on airing shows by default;
clicking mutes, clicking again re-follows.

4 tests updated/added incl. the airing-default + mute + re-follow flow. 80 video
tests green.
2026-06-16 08:47:48 -07:00
BoulderBadgeDad
e5a4dda117 Video watchlist (shows + people): DB + endpoints + button + page (v1, no scan yet)
A curated follow-list for the video side, mirroring the music watchlist. v1 is
membership only — the monitoring/discovery engine is a later phase.

Backend:
- video_watchlist table (kind 'show'|'person', keyed on tmdb_id — the stable
  cross-context id both carry; library_id kept when owned). NOT the existing
  shows.monitored flag (that defaults to 1 / is library-only / has no people).
- VideoDatabase: add/remove/list/state/counts (upsert COALESCEs library_id +
  poster so a TMDB re-add can't wipe known data).
- /api/video/watchlist {GET, /add, /remove, /check, /counts}.
- query_library now selects s.tmdb_id so show cards can carry the key.

Frontend:
- video-watchlist-btn.js: shared eye button (the music ya-watchlist-btn mirror)
  — build/toggle/hydrate, one delegated capture-phase click handler, broadcasts
  soulsync:video-watchlist-changed so pages can react.
- Watchlist page (new subpage + video-watchlist.js): Shows / People tab switcher,
  poster grid to detail-page quality, reloads each visit, drops cards on unfollow.
- Wired the eye onto library TV-show cards (movies excluded — wishlist, not
  watch) + hydrate on render.

Tests: 6 new (DB upsert/COALESCE/state/counts + endpoint roundtrip/validation).
76 video tests green. Other card surfaces (cast, search, similar, filmography)
are the same VideoWatchlist.btn(...) one-liner — wired next.
2026-06-16 01:06:24 -07:00
BoulderBadgeDad
6d72b7ca11 Video Calendar: EPG-style guide with a shared time axis + premium polish
The 7 independent day-stacks had no shared time axis — you couldn't scan
'what's on in prime time' across the week. Restructured into a real guide:

- Rows = time-of-day bands (Morning / Afternoon / Prime Time / Late Night /
  Anytime) down a frozen left time-rail; 7 day columns; frozen header row +
  sticky corner. Untimed streaming drops land in Anytime; each card keeps its
  exact time. Empty bands (none all week) are hidden; empty cells show a dot.
- Live 'now' cue lights the today-column × current-time-band intersection.
- Prime Time rail glints gold.
- Keyboard nav (←/→ weeks, T = today), scoped to the visible page + no modal.
- Week-change crossfade; staggered card entrance runs only on load/week-change
  (not on filter re-renders); keyboard focus rings on cards.
- Subheader now shows owned/missing split.

Kept all existing interactions: tilt cards, breathing glows, skeleton shimmer,
parallax billboard, rich episode modal. Vanilla static files, no build step.
2026-06-16 00:35:35 -07:00
BoulderBadgeDad
57c5148207 Video Calendar: week navigation + owned/missing filter
Finish the TV calendar as a planning tool.

- Week nav: ‹ prev / Today / next › steps the 7-day window (endpoint takes
  ?start=; each window starts on today's weekday so "current day first" holds).
  Title reflects it (This Week / Next Week / In N weeks); Today highlight only
  shows when the real today is in view.
- Filter: All / In library / Missing, applied client-side (no refetch) — the
  hero + grid + count all respect it.
- Hero eyebrow is context-aware (NEXT UP this week, FEATURED on other weeks).
2026-06-15 19:32:57 -07:00
BoulderBadgeDad
5856eefd7c Video Calendar: featured billboard, tactile cards, faster art
Take the calendar visuals up a level + fix slow image loads.

- Featured "Next up" billboard: the soonest upcoming episode as a cinematic hero
  (backdrop, pulsing accent, show title, S·E, air time, "View details" → modal).
- Cards: cursor-following 3D tilt + hover lift/scale/glow bloom; ambient glow
  dialed way down so it's calm at rest and only blooms on interaction.
- Faster art: poster proxy takes ?w= and asks the source for a thumbnail (Plex
  transcoder w/ original fallback, Jellyfin maxWidth, TMDB size bucket) — calendar
  requests ~500px instead of full backdrops. Skeleton shimmer → fade-in on load.
2026-06-15 19:25:45 -07:00
BoulderBadgeDad
3a8c803a54 Video: scope library reads to the active server (Plex/Jellyfin don't commingle)
Storage was already per-server (movies/shows UNIQUE(server_source, server_id),
episodes via per-server show_id, prune_missing scoped) — but reads returned
every server's rows, so a Jellyfin scan would show up alongside Plex.

Mirror the music standard: scope reads to the active video server
(resolve_video_server). query_library, calendar_upcoming, dashboard_stats and
library_id_for_tmdb take a server_source; the dashboard/library/calendar
endpoints pass it. server_source=None keeps "all servers" (enrichment processes
every server; tests unchanged). No schema change, no data migration — existing
Plex data is untouched and simply hidden while Jellyfin is the active server.

Regression tests: same title on both servers stays two rows; scoped reads only
return the active server's data; deep-scan prune never touches the other server.
2026-06-15 18:45:28 -07:00
BoulderBadgeDad
8e00670491 Video Calendar: 7-day week grid with air times + episode modal
A new isolated Calendar page (/api/video/calendar) — every upcoming episode for
your owned shows across a real 7-day week (today first), as art cards sorted by
air time with a per-cell breathing colour glow.

- Air times: enrich shows with TVDB airsTime (new shows.airs_time column +
  migration); cells show + sort by time, streaming (untimed/00:00) = "Anytime".
  One-time background backfill re-queues already-matched shows for the time.
- Click an episode → styled modal (show backdrop hero, episode still/synopsis,
  air date+time, owned badge, genres, "about the show"), with an explicit
  "Open full show page" action instead of navigating on click.
- Isolated: reads only video_library.db, writes nothing to the music side.
2026-06-15 18:28:02 -07:00
BoulderBadgeDad
5b6ef295f8 Video detail: trailer billboard reveals with a left→right wipe + fade
Replace the flat opacity crossfade with a soft, feathered left→right wipe of the
trailer in (mask via @property, falls back to crossfade) — "a rollover and a
fade". The wipe now fires ONLY when YouTube reports PLAYING (handshake + state
events), not on a blind timer, with a 4.5s safety net. When the trailer ENDS the
original backdrop fades back in and the iframe is torn down.
2026-06-15 16:55:00 -07:00
BoulderBadgeDad
80dd2ff21c Video side: own server connection (Plex/Jellyfin), music-style picker, isolated
Give the video side its OWN server connection — pre-filled from music but stored
separately in video.db, fully isolated (video never writes music config/state).

- Effective config helpers (video_plex_config / video_jellyfin_config): video's
  own creds when set, else inherited read-only from music. resolve_video_server +
  _build_source + watch-link/poster/dashboard all use these (own db threaded in).
- Server Connection UI mirrors music's server picker (toggle = select + configure),
  scoped to Plex/Jellyfin, at the bottom of the Connections tab.
- Jellyfin: independent client built from video's creds; explicit USER picker like
  music (list users → that user's libraries); honors the pick, admin fallback.
- Honest connection diagnostics (reachable vs 401 vs no-users) instead of a vague
  "auth failed".
- Auto-save on change with toasts; the shared Save button is intercepted on the
  video side so it saves video settings (and can't fire a music save).
- Enrichment status now PUSHES over the socket like music (no browser polling /
  access-log flood); config save only rebuilds workers when an API key changed.
- Seam tests for effective-config inheritance/override + isolation guard.
2026-06-15 16:42:22 -07:00
BoulderBadgeDad
8f6f992670 video: self-contained video Source section; hide music's server settings on video
Per the desired model: video Settings has its OWN server settings, separate from
music. The 'Video Source' group now holds the Plex/Jellyfin pick AND the Movies/TV
library mapping (moved out of music's Plex panel so it's not coupled to the music
active server). The whole MUSIC 'Server Connections' group is hidden on the video
side. Video reuses the shared Plex/Jellyfin credentials: the picker shows both
servers, the configured ones selectable, the rest 'not connected — set up in Music
settings'. Music settings keep their own server settings, untouched.
2026-06-15 15:10:53 -07:00
BoulderBadgeDad
f93c211de3 fix: video side can configure server connections again, without changing music
The previous hard guard blocked saveSettings entirely on the video side — which
also blocked entering/saving Plex/Jellyfin connection creds (shared, legit). Now
the save runs, but on the video side it PRESERVES the persisted active_media_
server (window._persistedActiveServer) instead of reading the toggle — so video
can set up a connection without ever switching the music server. Auto-save stays
suppressed on the video side (no heavy music re-init from video field edits).
2026-06-15 14:59:36 -07:00
BoulderBadgeDad
5f1ad517c2 video: fully decouple video server from music (no fallback + hard save guard)
Two real coupling bugs, fixed:
- resolve_video_server still fell back to the music active server when no explicit
  video pick was set, so changing the music server changed video. Removed: video
  now uses ONLY an explicit video pick or the configured server(s) (Plex default
  when both). Changing the music server never changes video.
- The shared settings page could persist active_media_server from the video side.
  Guarded saveSettings itself (not just the debounced auto-save) so it NEVER runs
  while data-side=video — video saves only via /api/video/*.

Test: video does not follow the music active server. One-way isolation, both ways.
2026-06-15 14:52:37 -07:00
BoulderBadgeDad
a4d6e78bce fix: video side can no longer change the music server via the shared settings page
The settings page is shared; on the video side its auto-save (debouncedAutoSave-
Settings) still fired when editing VIDEO fields (TMDB key, watch region, autoplay),
and saveSettings reads the server toggle from the DOM + persists active_media_
server — so clicking the toggle to Jellyfin on the video side and then editing any
field would write active_media_server=jellyfin to the shared config (and trigger a
full music save). Guard: never auto-save music settings while data-side=video.
Video settings persist via /api/video/*; the toggle re-syncs to the real active
server on every Settings load. One-way isolation restored.
2026-06-15 14:42:36 -07:00
BoulderBadgeDad
b7ae32220b video: coverage-card click re-queues failed matches (match music manager)
The video worker manager's coverage cards only switched the view — they never
re-queued failed items, so not_found/error items sat in the retry cooldown and
the worker reported 'all matched' / idle while failures piled up. Clicking a
Movies/Shows coverage card now pins that group AND resets its failed matches
(not_found/error -> pending) so the worker sweeps them, mirroring the music
worker manager. (Episodes are a sync cascade, not a match queue, so unchanged.)
2026-06-15 14:27:25 -07:00
BoulderBadgeDad
df09111831 video: move Detail Pages prefs to the Library settings tab
Video Preferences group now carries data-stg='library' so it lives under the
Library settings tab on the video side (the infra already supported video-only
library settings). The Video Source panel moves to data-stg='connections' so it
stops showing on every tab.
2026-06-15 14:19:57 -07:00
BoulderBadgeDad
c9900d91b0 video: hide Navidrome + Standalone server toggles on the video side
They're music-only, so a CSS rule scoped to body[data-side=video] hides those two
toggle buttons in the shared Server Connections section. Plex/Jellyfin stay
(for connecting a video server). Music side untouched.
2026-06-15 14:12:04 -07:00
BoulderBadgeDad
b2dbd6dec5 video: Video Source settings panel + library gating; move Detail prefs to own group
Settings → Video Source shows which server video uses (✓ Plex/Jellyfin), a
Plex/Jellyfin picker when both are connected, or a clear 'connect Plex or
Jellyfin' message when neither (Navidrome/Standalone are music-only and not
offered). The Library shows a non-breaking 'no video server' banner + disables
Scan until one is connected. Detail Pages prefs moved into their own 'Video
Preferences' group. /api/video/server GET+POST drives it.
2026-06-15 13:55:42 -07:00
BoulderBadgeDad
f7d1b725d7 video: resolve the video server independently of the music active pointer
The video side now uses a configured Plex/Jellyfin on its own (resolve_video_
server), not config_manager.get_active_media_server(). So a music-only server
(Navidrome/SoulSync) never applies to video, and 'Navidrome for music + Plex for
video' works. Order: explicit video pick (video_server setting) → music-active if
video-capable → the single configured one → Plex if both → None. Seam tests cover
each case incl. the mixed setup.
2026-06-15 13:50:19 -07:00
BoulderBadgeDad
b2adc63a6a video: where-to-watch region (no longer hardcoded US)
A saved 'Where-to-watch region' picker in Settings → Detail Pages (19 common
regions, default US). The engine reads it for the providers in extras +
tmdb_detail (region in the cache key), and the detail page labels the section
'Where to Watch · <region>' so you know which market you're seeing.
2026-06-15 13:28:17 -07:00
BoulderBadgeDad
b8de46d2ad video: episode detail expand (guest stars + bigger still)
Click any episode (owned or missing) to expand it: a larger still, full overview,
and the episode's guest stars (clickable to the person page). Lazy-loaded per
episode from TMDB by the show's tmdb_id and cached. New client.episode_detail +
engine.episode_extra + /api/video/episode/<show_tmdb>/<season>/<episode>.
2026-06-15 13:20:15 -07:00
BoulderBadgeDad
3359e3c111 video: owned-media tech specs on movie detail (Plex-grade)
We already scanned codec/audio/source/size but only showed resolution. The movie
detail Details block now surfaces Quality / Video (HEVC, H.264…) / Audio / Source
(Blu-ray, WEB-DL…) / Size, and lists every version/edition you own when there are
multiple files. movie_detail now returns all media_files (not just the largest).
2026-06-15 13:13:26 -07:00
BoulderBadgeDad
621693ecc8 video: real TTL+LRU cache (thread-safe) + cache search — the kettui way
The inline dict cache was a latent bug: no lock (the engine singleton is hit by
concurrent Flask + worker threads) and a wholesale-clear cliff at 256 (nuked hot
entries). Extracted a thread-safe TTL+LRU TTLCache into an importable core/ module
with seam tests (expiry via injected clock, LRU-not-wholesale eviction, a
concurrency stress test). Engine now uses it; search is cached too (60s, ownership
re-stamped fresh). Deliberately NOT persisted to disk — durable data already lives
in video.db; that tier would be over-engineering for a self-hosted app.
2026-06-15 13:02:24 -07:00
BoulderBadgeDad
b567eb4808 video: cache person pages, lazy preview seasons + trending (close the gaps)
The owned-item extras and preview detail payloads were already cached (30-min
TTL); person_detail, tmdb_season, and trending were not — so person pages and
lazy preview seasons re-hit TMDB each view. Now cached too (trending 1h). Library
ownership is re-annotated fresh on each call so 'In Library' badges stay current
while the expensive TMDB fetch is reused. Tests for person + season caching.
2026-06-15 12:55:56 -07:00
BoulderBadgeDad
e5724c6faa video: sticky back button on detail/person pages
The detail pages are long now (cast, videos, photos, reviews…), so the absolute
back button scrolled out of reach. It's now position:sticky (pinned top-left as
you scroll), with a negative bottom margin so the billboard still sits full-bleed
under it.
2026-06-15 12:40:48 -07:00
BoulderBadgeDad
1423e50ccd video: fix Play/Trailer buttons vanishing on owned movies
maybeRefreshMovie re-fetched the library payload (no trailer/server — those come
from the extras call) and replaced data, so renderActions re-rendered without the
buttons right after they appeared. It now carries over the live extras fields
(server/trailer/next_episode), mirroring the show reloadDetail fix.
2026-06-15 12:28:32 -07:00
BoulderBadgeDad
3f439e4277 video: fix back-button loop when a 'More Like This' item is owned
Clicking a recommendation/cast/similar link to a title you OWN opens it via a
tmdb URL that redirects to the library detail — but the redirect PUSHED a new
history entry, so Back landed on the tmdb URL which redirected forward again =>
stuck loop + a self-referencing 'Back to <same movie>' label.

The redirect now REPLACES the redirecting entry (keeping its layer + origin)
instead of pushing, so Back unwinds cleanly to where you actually came from.
2026-06-15 12:18:05 -07:00
BoulderBadgeDad
2f3e4b128b video: autoplay billboard trailer (opt-in setting)
After a couple seconds on a detail page, a muted trailer plays behind the hero
(Netflix/Disney+ style) with mute/unmute + stop controls; the backdrop fades back
when stopped. Stops on navigate-away/modal-open (no orphaned audio).

Gated by a 'Autoplay trailers in the billboard' toggle in video Settings →
Detail Pages (default on). Backed by billboard_autoplay in video_settings, read
via a lightweight /api/video/prefs. Tests updated for the new config field.
2026-06-15 11:57:29 -07:00
BoulderBadgeDad
be54fccc63 video: featured TMDB review on movie/TV detail pages
extras() now returns a featured review (author, rating, snippet, date); the detail
page shows it in a card with a clamped body + Read more/less. In-app (no external
link).
2026-06-15 11:52:07 -07:00
BoulderBadgeDad
df8f6a2acd video: person page — photos gallery + lightbox + 'also known as'
person() now returns profile images (thumb+full) and also_known_as. The person
page shows an 'Also known as' line and a Photos rail that opens the shared
fullscreen lightbox (arrows/Esc/counter).
2026-06-15 11:49:43 -07:00
BoulderBadgeDad
8bbdd712f9 video: per-title accent on preview + person pages (same-origin image proxy)
Owned detail pages sample the poster for the per-title glow, but preview/person
pages fell back to the theme accent because their TMDB images are cross-origin
(canvas taint). Added /api/video/img — a same-origin proxy restricted to
image.tmdb.org (SSRF-safe) — so:
- preview (tmdb) detail samples its poster via the proxy → real accent;
- the person page samples the portrait → per-person accent on the ring/glow/role.
Tests: route registered + proxy rejects non-tmdb URLs.
2026-06-15 11:46:49 -07:00
BoulderBadgeDad
c5e30530c7 video: card hover shows an info badge, not a misleading play button
Search / trending / filmography poster cards open the DETAIL page on click, so
the center ▶ read as 'play' wrongly. Swapped it for an italic 'i' info badge
('view details').
2026-06-15 11:42:32 -07:00
BoulderBadgeDad
9c29414e6d video: detail pages — Photos gallery+lightbox, all Videos, Details/keywords, full cast
Frontend for the new data, on both movie + TV detail pages:
- Photos: a backdrops rail → fullscreen lightbox (‹ › nav, keyboard arrows, Esc,
  counter).
- Videos: a rail of every trailer/teaser/clip/featurette (YouTube thumbs) → opens
  in the existing player modal.
- Details: budget / box office / language / country + keyword tag chips.
- Cast & Crew gets a 'View all N' → full-cast modal (clickable to person; TV shows
  per-actor episode counts).
All cached server-side (instant re-open) and lazy-loaded images. Isolated; shell
tests cover the new sections + modals.
2026-06-15 11:23:31 -07:00
BoulderBadgeDad
6fcebe7c4b video: detail extras data layer — gallery, all videos, keywords, facts, full cast (+ caching)
One TMDB call (append_to_response) now also returns: image gallery (backdrops +
posters, thumb+full), all YouTube videos (ordered trailer→teaser→clip→…), keyword
tags, facts (budget/revenue/language/country), and the FULL cast (tv via
aggregate_credits with per-actor episode counts). Shared by item_extras (owned)
and full_detail/tmdb_detail (preview).

Caching: the engine memoizes the live TMDB extras + preview payloads (30-min TTL)
so re-opening a title is instant instead of re-hitting TMDB. Tests added.
2026-06-15 11:17:33 -07:00
BoulderBadgeDad
595ea2bfbd video: Where to Watch — provider logos are badges + one real link
TMDB only exposes a single aggregate 'where to watch' link (no per-provider deep
links), so N identical links read as broken. Streaming providers are now
non-clickable availability badges, followed by ONE accent 'Where to watch ↗'
link to the JustWatch/TMDB page. The Plex/Jellyfin tile keeps its real per-item
server deep link.
2026-06-15 11:00:15 -07:00
BoulderBadgeDad
e0b9245ba3 video: Play button reads 'Play on <logo>' (drop redundant text, better contrast)
The button showed the Plex logo AND the word 'Plex' ('[logo] Play on Plex'), all
white on a bright green bg — cluttered + hard to read. Now it's '▶ Play on
<server logo>' (the logo is the brand name) on a deeper green gradient so the
light Plex/Jellyfin wordmark reads clearly.
2026-06-15 10:54:49 -07:00
BoulderBadgeDad
b937343768 video: fix stuck ep-sync banner + vanishing Play/Trailer buttons on shows
Two bugs when a show's episode list backfills on view:
- The 'Fetching full episode list…' banner never hid: .vd-ep-syncing (and
  .vd-next-ep) set display:flex, which overrode the [hidden] attribute's
  display:none, so el.hidden=true did nothing. Added a guard so [hidden] always
  wins on the detail/search/person pages.
- Play & Trailer buttons vanished after the sync: reloadDetail replaced data with
  a fresh show_detail payload (no server/trailer — those come from extras), so
  renderActions re-rendered without them. reloadDetail now carries over the live
  extras fields (server/trailer/next_episode).
2026-06-15 10:32:04 -07:00
BoulderBadgeDad
81546cff69 video: detail-page feedback fixes (Play button style, crew links, ep-sync UX)
- Play button now matches the Trailer/Watchlist buttons exactly (same size/shape),
  just green — consistent hero buttons.
- Where to Watch: drop the duplicate streaming provider that matches your server
  (no more two 'Plex' entries). Providers still share TMDB's single JustWatch
  'where to watch' link (that's all TMDB gives).
- Director/Creator names (hero line + Cast & Crew section) are clickable → the
  in-app person page.
- Opening a show whose full episode list isn't cached yet now shows a 'Fetching
  the full episode list…' banner with a spinner, instead of a silent ~20s gap
  before missing episodes pop in.
2026-06-15 10:21:04 -07:00
BoulderBadgeDad
e94ea39187 video: skeleton loader for detail pages (shimmer billboard, not 'Loading…')
Replaces the plain loading text with a shimmering billboard placeholder (logo /
meta / overview / button bars) over the accent wash, so opening a title (esp. a
TMDB preview) feels instant and premium. Honors prefers-reduced-motion.
2026-06-15 09:59:39 -07:00
BoulderBadgeDad
8441ece6f0 video: person page — sort, department filter, age (best-in-class filmography)
- Sort dropdown: Newest / Oldest / Most popular.
- Department filter (Acting / Directing / Writing / …) for multi-hyphenates —
  only appears when a person has 2+ departments. Composes with the existing
  kind + ownership filters; every chip shows a CONTEXTUAL count (what you'd get
  if you clicked it, given the other active filters).
- Age in the hero meta ('47 years old', or lifespan + 'aged N' for the deceased).
- Backend: each person credit now carries its department (cast=Acting,
  crew=its TMDB department). Seam test added. 249 video-suite tests pass.
2026-06-15 09:57:39 -07:00
BoulderBadgeDad
6eed3d7775 video: best-in-class detail billboards (Play CTA, collections, recs, next-ep)
Movie + TV detail pages get the things a premium app surfaces:
- Primary 'Play on Plex/Jellyfin' button (white Netflix-style CTA with the server
  logo) in the billboard for owned items — deep-links straight to the item.
- 'Directed by' (movies) / 'Created by' (shows) line in the hero.
- Movies: a Collection/franchise row (the other films in the set), release-ordered.
- 'More Like This' now uses TMDB recommendations (better curated), similar as
  fallback.
- TV: a 'Next Episode' banner (S/E + name + air date) for continuing shows, and
  the selected season's overview under the season nav.
All in-app (cards drill into library/preview detail). Shell tests updated.
2026-06-15 09:50:07 -07:00
BoulderBadgeDad
f725235f44 video: detail extras now include collections, recommendations + next-episode
Backend groundwork for the best-in-class detail pages:
- Movies: belongs_to_collection → the franchise's films (2nd /collection call,
  release-ordered).
- Recommendations (better-curated than 'similar') alongside similar.
- TV: next_episode_to_air / last_episode_to_air stubs (season/ep/name/air date).
Shared by item_extras (owned) + full_detail/tmdb_detail (preview). Tests cover
recommendations+collection ordering and next-episode parsing.
2026-06-15 09:45:06 -07:00
BoulderBadgeDad
c5ff2567a8 video: OMDb daily-limit cools down + auto-resumes (vs hard pause)
'Request limit reached!' is the free-tier daily quota (1,000/day), not a bad key
— so the worker now idles ~30 min and auto-resumes instead of pausing for good.
A library bigger than the daily cap just spreads its ratings across days on its
own. A genuinely invalid/unactivated key still hard-pauses until fixed. Cooldown
reads as paused in the UI (+ a 'cooldown' flag and note). Item is never burned to
synced, so nothing is lost.
2026-06-15 09:34:47 -07:00
BoulderBadgeDad
38d48bae37 video: OMDb Test button shows the real reason (not just 'HTTP 401')
OMDb returns a JSON body even on 401, so surface its actual Error: 'Invalid API
key!' (nudges the user to click OMDb's activation email) vs 'Request limit
reached!' (free-tier daily quota). The worker's pause note uses the same message.
2026-06-15 09:29:09 -07:00
BoulderBadgeDad
d0c8aa9338 video: OMDb worker survives a bad/expired API key gracefully
The log flood you saw was the OMDb worker hitting a 401 (invalid key) on every
owned title: it logged a full traceback per item AND marked each one
ratings_synced=1 — which would've stopped them ever retrying once the key was
fixed. Root-cause fixes:

- OMDBClient.ratings raises a distinct OMDbAuthError on 401 / 'Invalid API key!'
  (vs a transient error vs a genuine no-data 200).
- Worker: on an auth error it PAUSES (transient, not persisted) with a reason
  note + one warning, instead of churning the whole library; the item is NOT
  marked synced. Transient errors no longer burn items either — they back off and
  pause after 3 in a row. Only a genuine 'no data' marks an item synced. Warnings
  are concise (no per-item tracebacks). get_stats exposes the pause 'note'.
- Fixing the key auto-recovers: saving a new/changed OMDb key re-queues every
  still-unrated title (resets the wrongly-burned ones), and the engine rebuild
  un-pauses the worker.

Seam tests: bad-key pause-without-burn, transient keep-item, ratings() raises on
401, key-change re-queues unrated. 227 video-suite tests pass.
2026-06-15 09:20:00 -07:00
BoulderBadgeDad
df889c0c6e video: Where-to-Watch server tile uses the real Plex/Jellyfin logo
The 'Play on your server' tile now shows the actual server logo (same Plex/
Jellyfin art as the header server toggle) on a dark tile with the green owned
glow, instead of a generic play glyph. Falls back to the play glyph if the logo
fails to load.
2026-06-15 09:11:19 -07:00
BoulderBadgeDad
0246842000 video: Where to Watch — play-on-your-server tile + clickable providers
The 'Where to Watch' section is now actionable:
- For an OWNED title it leads with a 'Play on Plex/Jellyfin' tile (green, play
  glyph) that deep-links straight to the item on your server — Plex via the
  app.plex.tv web app (machineIdentifier fetched once + cached), Jellyfin via its
  web detail page. Built in engine.item_extras from the row's server_source +
  server_id and the shared media-server config (same source poster.py uses).
- Streaming providers (TMDB/JustWatch) are now clickable → the where-to-watch
  page, with a hover lift.

Owned-only: preview (tmdb) items have no library row so they get no server tile.
Seam tests cover the Jellyfin + Plex link building and the unowned no-link case.
240 video-suite tests pass.
2026-06-15 08:49:35 -07:00
BoulderBadgeDad
7081dad94e video: person page — filter filmography + Known For by owned vs missing
Adds an ownership filter (All / In Library / Missing) next to the kind tabs on
the person page. Both the Known For rail and the full filmography respect it, so
you can see 'what of this person's work do I actually have' (or what's missing).

- Each credit already carries library_id (owned), so filtering is client-side.
- Contextual counts: the kind tabs count within the current ownership filter and
  vice-versa, so the numbers always match what you'll see.
- 'In Library' filter glows green (matches the owned ribbon); empty states for
  'you have everything' / 'nothing owned yet'.
2026-06-15 08:42:26 -07:00
BoulderBadgeDad
769dc62b87 video: cooler person hero + perf pass on search/person pages
Person hero glow-up:
- Cinematic ambient — blurred portrait + an accent colour mesh + vignette, masked
  to fade into the page (was a flat wash).
- Portrait gets a slowly-rotating accent gradient ring (masked donut, GPU
  transform) and a gentle float; an accent ring + glow frame.
- An accent role tagline ('ACTOR' / 'DIRECTOR' …) above a gradient-filled name,
  plus a credits-count chip. Honors prefers-reduced-motion.

Performance:
- Long filmography grids use content-visibility:auto + contain-intrinsic-size so
  the browser skips off-screen cards (cheap scroll), and skip replaying the
  entrance animation as cards recycle.
- Hero layers are static (painted once); only two tiny composited transforms
  animate. Posters/photos stay on small TMDB sizes + lazy-load; trending cached;
  search debounced + request-sequenced.

Pure visual/perf layer — same data + isolation. Shell/JS tests pass.
2026-06-15 08:37:32 -07:00
BoulderBadgeDad
d6d0dc537c video: smart multi-layer back button + next-level search/person
Smart back (mirrors music's artist-detail): the top-left back button now
remembers where you actually came from, many layers deep. It keeps an origin
stack ({page} or {detail title}) and stamps each history entry with its layer
depth, so:
- the label is dynamic — '← Back to Search' / '← Back to The Bear' / '← Back to
  <person>' — instead of a hardcoded 'Library'/'Back';
- backing out of the first layer returns to the page you started from (Search,
  Watchlist, wherever), not always the Library;
- browser Back and our button both unwind the chain one layer at a time, in sync.
Fixes: search → person → back → movie used to mislabel as 'Library' and dump you
in the library.

Next level:
- Search isn't a blank box when idle — a 'Trending this week' rail (TMDB
  trending, owned/preview annotated). Returns when you clear the query.
- Person page gets a 'Known For' hero rail (top titles by popularity) above a
  full filmography now sorted chronologically (newest first).

Backend: TMDBClient.trending + engine.trending (+library annotation), route
/api/video/trending. Isolated; 237 video-suite tests pass.
2026-06-15 08:30:17 -07:00
BoulderBadgeDad
87f414c8c7 video: cinematic Netflix-grade redesign of the Search + Person pages
Matches the show-detail page's vibe (--vd-accent-rgb glows, full-bleed, rise/
fade entrances) instead of the plain library shell.

Search:
- Cinematic hero — big title, ambient accent glow, a large glowing search bar
  that lights up on focus.
- Results are premium 2:3 poster cards: hover lift + accent glow, poster zoom,
  gradient overlay, a play affordance, owned/preview ribbon + rating chip.
  People render as circular portrait cards. Grouped rows with accent-pill counts.
- Polished empty/hint states with a floating icon.

Person:
- Full-bleed cinematic hero with an ambient blurred-portrait backdrop (per-person
  color), a glowing circular portrait, oversized name, meta as glass chips.
- Bio with Read more/less; filmography as the same premium poster cards with
  premium pill tabs (All / Movies / TV).

Pure visual layer — same data hooks, routing and isolation. Shell/JS isolation
tests still pass.
2026-06-15 08:12:53 -07:00
BoulderBadgeDad
288d44155d video: in-app Search + TMDB-backed (preview) detail + person pages
Search any movie / show / person (TMDB multi-search) entirely in-app. Results
that you already own link straight to the library detail; the rest open a
TMDB-backed 'preview' detail that reuses the exact same Netflix billboard UI
(direct image URLs, nothing owned/enriched). Everything resolves back into
SoulSync — no external links on un-owned titles.

- Search page (video-search.js): debounced /api/video/search, grouped
  movies/shows/people cards (reuses .library-artist-card) with owned/preview
  ribbons. People open the person page.
- Source-agnostic detail (video-detail.js): loads from /api/video/detail
  (library) or /api/video/tmdb (preview); art helpers pick proxy vs direct URLs;
  tmdb shows lazy-load episodes per season; owned-via-tmdb-url auto-redirects to
  the library detail.
- 'More Like This' now drills in-app (tmdb detail, redirects if owned); cast/crew
  link to a new in-app person page (bio + filmography, each credit owned/preview).
  Library credits now carry tmdb_id so owned-item cast is clickable too.
- Backend: TMDBClient.search/full_detail/person (+ shared _parse_extras);
  engine.search/tmdb_detail/tmdb_season/person_detail; db.library_id_for_tmdb;
  routes /search, /tmdb/<kind>/<id>, /tmdb/show/<id>/season/<n>, /person/<id>.

Isolated (one-way): video-only files, no music imports, music shell untouched.
Seam tests: search/full_detail parsing, tmdb_detail assemble+redirect, search +
person library annotation, library_id_for_tmdb, route registration, shell/JS
isolation. 234 video-suite tests pass.
2026-06-14 23:31:35 -07:00
BoulderBadgeDad
b50f7c12f4 video: promote OMDb to a full 3rd enrichment worker (parity with TMDB/TVDB)
OMDb now has the same setup as TMDB/TVDB: a yellow dashboard orb (★ glyph) that
spins/idles in the worker-orb animation, an entry in Manage Workers (Ratings
coverage cards, pause/resume, retry, search), and a BACKGROUND ratings pass.

- Worker 'ratings mode' (is_ratings): instead of a match queue it pulls
  ratings_next() (library items with an imdb_id and ratings_synced=0), fetches
  IMDb/RT/Metacritic, applies + marks synced. So the whole library gets ratings,
  not just titles you open (schema v7: ratings_synced).
- enrichment_breakdown/unmatched/retry get an 'omdb' branch (coverage =
  ratings-filled, not matched). build_clients includes omdb; the lazy on-view
  backfill uses the omdb worker's client.
- Dashboard orb + Manage Workers entry (★ glyph fallback where there's no logo),
  yellow accent.

Seam tests: omdb worker rates the queue (ratings mode), ratings breakdown.
2026-06-14 23:07:35 -07:00
BoulderBadgeDad
f06728b0a7 video detail: IMDb / Rotten Tomatoes / Metacritic ratings (OMDb)
Next-level: real critic/audience scores beyond the TMDB star. OMDb (free key,
keyed by the imdb_id we already capture) returns IMDb / RT / Metacritic.

- OMDBClient (ratings + test); built as a non-worker 'ratings_client' on the
  engine. _backfill_ratings runs in both lazy detail refreshes (overwrites, since
  ratings are dynamic). schema v6: imdb_rating / rt_rating / metacritic on
  movies + shows; show/movie payloads return them.
- Billboard renders branded rating badges (IMDb yellow, RT tomato/splat by
  fresh/rotten, Metacritic green/yellow/red by score). Lazy refresh also triggers
  when an imdb_id exists but ratings are missing.
- OMDb API-key frame in Settings (parity with TMDB/TVDB) + config GET/POST +
  /enrichment/omdb/test.

Seam tests: OMDb parse, engine ratings backfill, apply_ratings + payload, config
includes omdb.
2026-06-14 22:54:53 -07:00
BoulderBadgeDad
aff4ecccc6 video enrichment: background episode-sync pass (full lists for the whole library)
So library cards show real owned/total (e.g. 8/10) WITHOUT opening each show. When
the TMDB worker's match queue is clear, it pulls the full season/episode list for
one already-matched-but-unsynced show per loop (episode_sync_next), inserting
missing episodes + marking it synced. Over time every library show gets its full
list; the on-view lazy refresh still makes the one you open instant. TMDB-only;
counts toward the worker's pending so it shows busy (not 'Complete') while syncing,
and never loops on a single failing show.

Seam tests: episode_sync_next selection + pending count, worker syncs a pre-feature
matched show to the full list.
2026-06-14 22:39:55 -07:00
BoulderBadgeDad
e2a8d3339a video routing: re-assert the detail URL against late/async music redirects
The reload restore could still be clobbered if music's router redirects on a later
tick. Re-assert the /video-detail URL at 120/350/700ms after restore, but ONLY
while the detail subpage is still showing, so it survives an async redirect without
ever fighting genuine navigation away.
2026-06-14 22:27:42 -07:00
BoulderBadgeDad
53391372c3 video: fix detail reload (music router clobber) + reliably show missing episodes
Reload bug: music's router boots first, rewrites an unknown /video-detail/... URL
to /dashboard, and my init read the already-changed URL (no restore) AND dispatched
open-detail before video-detail.js was listening (empty page + stray back button).
Fix: capture the path at SCRIPT-EVAL time (before music boots) and DEFER the
restore to a macrotask so every DOMContentLoaded handler is registered and music's
initial routing has run — then re-assert the real URL. Reload/deep-link now restore
the exact item.

Missing-episodes bug: the full-episode-list cascade only ran via the lazy refresh,
which was gated on ART being missing — so a show that already had posters/logo
never pulled its episode list (stayed owned-only). Added shows.episodes_synced
(schema v5): the worker sets it after a full cascade; show_detail returns it; the
lazy refresh now triggers when NOT synced, so owned + missing episodes populate.
2026-06-14 22:26:37 -07:00
BoulderBadgeDad
0598f5fbda video: real-link routing for detail pages (parity with music artist-detail)
Mirrors the music side instead of a custom scheme: library cards are genuine
<a href='/video-detail/<source>/<kind>/<id>'> links, so reload / new-tab / Back /
Forward all work. Left-clicks are intercepted into SPA nav + history.pushState;
modifier-clicks fall through to the real URL.

- popstate restores the detail from the URL; the '← Library' back button is real
  history.back(); deep-link / reload to a /video-detail/... URL is restored on load
  (path captured before applySide can clear it). Server already serves the SPA for
  these paths (permissive catch-all) — no backend change.
- The path carries a SOURCE segment ('library' = a video.db id today; 'tmdb' /
  search results not yet in the library come later) — your library-vs-search split.
- Coexists with music's pathname router (only touches /video-detail/* and its own
  popstate); music's link/popstate handlers ignore these paths.

Tests: real-link cards, pushState/popstate routing, source segment.
2026-06-14 22:13:51 -07:00
BoulderBadgeDad
c565fec59d video: show the FULL episode list (owned + missing), Sonarr-style
Previously the episodes table held only what the server has (all 'Owned'), so the
detail page never showed what you're missing. Now the metadata provider defines
the full series structure and the server marks ownership:

- TMDB returns the full season list (poster optional) + full episode fields
  (title/air date/runtime/still/rating) per season.
- backfill_episodes UPSERTs: owned episodes keep has_file=1; episodes the server
  lacks are inserted as MISSING (has_file=0); fully-missing seasons get created.
  The cascade now iterates every TMDB season, not just the ones on the server.
- The scan prune only removes SERVER-originated rows (server_id set) that vanished,
  so enrichment-added missing episodes/seasons are never pruned on re-scan.

Season coverage (X / Y) is now meaningful, and the episode list shows Owned +
Missing together. Seam tests: missing-episode insert, fully-missing season,
prune preserves missing.
2026-06-14 22:02:58 -07:00
BoulderBadgeDad
6e526d7745 video detail: deeper TMDB extras — trailer, where-to-watch, more like this
Phase 4: dynamic extras fetched LIVE per view (providers change, so not cached)
via GET /detail/<kind>/<id>/extras → engine.item_extras → TMDB
(videos + watch/providers + similar in one call).
- Trailer: a '▶ Trailer' action that opens an in-app YouTube modal embed (Esc /
  click-away to close).
- Where to Watch: provider logos for the region (JustWatch via TMDB).
- More Like This: a poster row of similar titles linking out to TMDB.
Both movie + show pages; all keyless (same TMDB key).

Seam tests: extras parse (trailer priority, provider/similar shape), item_extras
gating on tmdb_id, route registered, markup hooks. (RT/Metacritic via OMDb needs
its own key — offered separately.)
2026-06-14 21:42:41 -07:00
BoulderBadgeDad
efa0632883 video detail: clearlogo hero (TMDB images — no new key)
Phase 3: the stylized transparent title logo replaces the text title in the
billboard (the big Plex/Netflix 'premium' jump). Sourced from TMDB images
(append_to_response=images, include_image_language=en,null) in the same detail
call — no Fanart key needed.

- schema v4: movies.logo_url / shows.logo_url (idempotent migration).
- TMDB client picks an English logo (then language-neutral, then any); enrichment
  backfills logo_url gap-only; show/movie payloads return 'logo'.
- Billboard shows the logo img (with graceful fallback to the text title on error
  / when absent; title kept visually-hidden for a11y). Lazy on-view refresh now
  also triggers when the logo is missing, so existing libraries fill it in.

Seam tests: English-logo pick, backfill + payload, schema.
2026-06-14 21:36:14 -07:00
BoulderBadgeDad
59c88fa0db video: Movie detail page (Netflix flat layout), movies now clickable
- Movie cards in the library now drill into a movie-detail page (both kinds use
  the same open-detail event / video-side navigation).
- New video-movie-detail subpage reuses the .vd-* hooks; video-detail.js is now
  kind-aware (root() targets the active page by kind, billboard/links/actions
  branch on movie vs show). Flat layout: billboard + a details strip (released /
  runtime / studio / status / critic score / quality) + the shared Cast & Crew row.
- Lazy on-view backfill for movies too: engine.refresh_movie_art re-fetches TMDB
  (cast/genres/backdrop/ratings) when missing, regardless of match status, via
  POST /detail/movie/<id>/refresh-art. movie_match_info added.

Seam tests: movie refresh backfills cast/genres, movie_match_info, route
registered, movie subpage markup, cards clickable for both kinds.
2026-06-14 21:31:51 -07:00
BoulderBadgeDad
b9b3b9eed3 video 'capture everything': cast & crew (people + credits) + cast row UI
Last capture piece, schema v3 (new people + credits tables; CREATE IF NOT EXISTS
migrates existing DBs on restart, no wipe):
- people deduped by tmdb_id; credits link to exactly one movie OR show (separate
  nullable FKs + CHECK, no polymorphic id) with department/job/character/order.
- TMDB client appends credits to the detail call (free) and parses cast (name,
  character, photo, billing order) + headline crew (directors/writers/creators).
- enrichment_apply backfills cast/crew gap-only (never clobbers); show/movie
  detail return cast + crew. Populates on view via the existing lazy refresh-art.
- Cast & Crew section on the detail page: grouped crew line + a horizontal
  cast row with circular TMDB headshots, names, characters (accent hover).

Seam tests: TMDB credit parse (+ job filtering, created_by), backfill + people
dedup across titles, gap-only no-clobber, payload shape.
2026-06-14 21:20:08 -07:00
BoulderBadgeDad
8002f93220 video: include season id in show_detail (fix /poster/season/undefined 404)
show_detail's season payload omitted the season id, so the frontend built
/api/video/poster/season/undefined and 404'd — season posters never showed even
once cached. Add the id; harden seasonArt to fall back to the show poster if id
is ever missing. Test pins that seasons carry an int id.
2026-06-14 21:02:35 -07:00
BoulderBadgeDad
986059626f video: lazy season-art backfill on detail view (fixes matched-show art gap)
Root cause: season posters / episode art backfill happen during a show's TMDB
*match*, but already-matched shows never re-run ('Retry all failed' only resets
not_found/error), so existing libraries never got the art.

Fix (Boulder's idea): fetch-on-view + cache. When a show detail opens and any
season lacks a poster, the page calls POST /detail/show/<id>/refresh-art →
engine.refresh_show_art re-fetches /tv/<id> via the TMDB client and backfills
season posters + episode art gap-only, regardless of match status. Cached, so
it's a one-time cost per show; runs once per view; re-renders when done.

Seam tests: refresh_show_art backfills a MATCHED show's seasons, needs TMDB
configured, show_match_info, route registered.
2026-06-14 20:56:57 -07:00
BoulderBadgeDad
fcd4af0efd video manage-workers modal: Process First Everywhere, search/filter, live glow
Brings the video modal to parity with music's:
- 'Process first everywhere' control (Movies/Shows/Auto) in the topbar — a global
  setting that pins which kind every worker processes first. enrichment_next takes
  a priority kind; the worker reads enrichment_priority each loop; GET/POST
  /api/video/enrichment/priority persists it. Reuses music's .em-global styling.
- Needs-matching bar now has a live count, status filter (All unmatched / Not
  found / Pending) and a debounced search (reusing .em-select / .em-search),
  matching the music modal. Episode view stays read-only.
- Live glow (scoped to #vem-overlay): pulsing running dot, accent glow on the
  selected worker row + active process-first/kind, and a pulsing 'now processing'
  chip in the worker accent. Music's shared .em-* styles untouched.

Seam tests: priority pins kind in enrichment_next + worker honors the setting,
priority endpoint GET/POST + validation, modal feature markup pinned.
2026-06-14 18:28:22 -07:00
BoulderBadgeDad
80f1051e8a video enrichment: cascade episode backfill from the TMDB show worker
Episodes ride along with their show instead of being a separate (tens-of-thousands)
queue: when the TMDB worker matches a show, it now backfills every season's
episodes — still / overview / rating — via /tv/<id>/season/<n> (one call per
season, gap-only so server data is never clobbered). Also backfills season
overviews.

The worker manager 'knows about it': the TMDB breakdown gains an Episodes
coverage entry (matched = has art, rest = pending), shown as its own card; the
Episodes view lists episodes still missing art. It's coverage-only, kept out of
the worker's idle/pending calc so it never blocks 'Complete'.

Seam tests: client season parse, worker cascade fills episodes, gap-only backfill
+ season overview, breakdown coverage (tmdb only), missing-art list, idle calc
ignores episode coverage.
2026-06-14 18:09:24 -07:00
BoulderBadgeDad
878e467f69 video enrichment: backfill season posters from TMDB (server usually lacks them)
The media server rarely has distinct per-season art, so season cards fell back to
a gradient. TMDB's show detail carries a poster_path per season — the show worker
now returns those, and enrichment_apply backfills seasons.poster_url for seasons
the server left without art (gap-only, never clobbers server art). The image
proxy streams a stored full URL (TMDB) directly vs. proxying a server path.

Seam tests: TMDB returns season posters, backfill fills only missing seasons.
2026-06-14 17:57:24 -07:00
BoulderBadgeDad
5e8143dd1d video scan: survive legacy UNIQUE on tmdb_id/tvdb_id (store the row, drop the dup id)
Existing DBs created movies.tmdb_id / shows.tvdb_id as inline UNIQUE (can't be
dropped via migration). The new model allows the same title in >1 library, so a
second movie/show with the same id raised IntegrityError and the scanner SKIPPED
it — dropping the title (observed: 'UNIQUE constraint failed: movies.tmdb_id',
movie 548522 skipped).

upsert_movie/upsert_show_tree now use a shared _resilient_upsert: on
IntegrityError, retry WITHOUT the id columns so the row is stored (just without
the colliding id) — same pattern enrichment_apply already used. Regression tests
for both movies and shows under a simulated legacy unique index.
2026-06-14 17:33:38 -07:00
BoulderBadgeDad
2306a5740c video enrichment: pull everything TMDB/TVDB offer + backfill gaps only
Enrichment now harvests the full detail payload (same call, no extra requests):
- TMDB: tagline, genres, rating (vote_average), runtime, status, first/last air
  date (shows), release date + runtime (movies) — on top of overview/backdrop/ids.
- TVDB: switches to /series/<id>/extended for overview + genres.

enrichment_apply now uses BACKFILL semantics: metadata columns are written via
COALESCE(NULLIF(col,''), ?) so enrichment only fills fields the media server
left empty — it never clobbers server-provided data. Genres backfill to the
normalised link tables only when the item has none yet. Whitelist expanded for
the new columns.

Seam tests: backfill-only (server overview/genres kept, gaps filled), genre
backfill when empty, TMDB full-metadata extraction.
2026-06-14 17:23:43 -07:00
BoulderBadgeDad
e1e0e29432 video 'capture everything' (phase 1): stills, genres, ratings, tagline
Captures the richer metadata the media server already exposes (schema v2;
idempotent migrations + CREATE IF NOT EXISTS, so an existing DB upgrades on
restart with no wipe):
- movies: tagline, rating (audience), rating_critic; shows: tagline, rating,
  first/last air date; episodes: still_url + rating.
- Genres as a normalised many-to-many (genres + movie_genres/show_genres link
  tables — no comma-blob), deduped, replace-on-upsert.
- Plex (.genres/.tagline/.audienceRating/.rating/.thumb) + Jellyfin (Genres/
  Taglines/CommunityRating/CriticRating/Premiere+EndDate/episode Primary) both
  extract them; episode stills served via /api/video/poster/episode/<id>.
- Detail payloads return genres/tagline/rating/air-dates + per-episode has_still;
  the billboard shows a tagline, ★ score, genre chips, and episode rows render
  REAL stills (no more orange placeholder once scanned).

Seam tests for genre dedup/replace, show+episode capture, episode still ref.
Cast/crew (people + credits) is the next phase.
2026-06-14 17:17:20 -07:00
BoulderBadgeDad
c450fa1f9a video detail: 4 season-nav views + view toggle, real Watchlist, Missing filter
Season selection is now switchable via a view toggle (persisted): poster RAIL
(scrollable season cards w/ coverage), TIMELINE band (segments sized by episode
count, filled by owned), TABS (pills), and the LIST dropdown. All drive the same
selection; episodes fade in on change.

- Watchlist button is now REAL: toggles shows.monitored via POST /api/video/monitor
  (set_monitored), reflects 'In Watchlist' state. show_detail returns monitored.
- 'Get Missing' + a 'Missing only' toolbar toggle filter the episode list to
  unowned episodes (actual downloading is the future acquisition subsystem).

Seam tests for the monitor endpoint + bad-input guards; shell hooks updated.
2026-06-14 17:05:51 -07:00
BoulderBadgeDad
20f1214b68 video detail: reuse the artist-hero button + badge styles
- Action buttons now reuse the exact artist-detail classes
  (.library-artist-watchlist-btn + .discog-download-btn/.discog-btn-compact with
  shimmer) instead of bespoke vd-btn styling — identical to the music hero. Wired
  by data-attr (music binds those by id, so no hijack).
- IMDb/TMDB/TVDB source links now render as .artist-hero-badge chips (logo + short
  text fallback, TVDB inverted), matching the artist hero's #artist-hero-badges
  treatment instead of the off-standard text tags.
2026-06-14 16:47:18 -07:00
BoulderBadgeDad
81f2127e71 video: redesign TV-detail as a Netflix billboard (break from the music layout)
Per feedback, this drops the Spotify/artist-page parallel entirely and goes
Netflix:
- Full-bleed billboard (edge-to-edge — breaks out of the host's 40px padding),
  big backdrop with Ken-Burns drift + layered scrims, oversized title, a Netflix
  meta row (owned% · year · rating · seasons · runtime · status), 3-line synopsis,
  and action buttons (Watchlist / Get Missing / external links).
- Per-show accent colour sampled from the poster (canvas) → drives the primary
  button glow, status, episode hover — the SoulSync 'vibe', per title.
- Custom season dropdown + rich episode rows (index, 16:9 thumb w/ hover play,
  title · runtime, 2-line synopsis, Owned/Missing) that fade/stagger in on season
  change.
- Backdrop falls back to a cover-cropped poster; episode stills + genres + cast
  arrive with the 'capture everything' phase. Watchlist/Get-Missing are visual
  pending their endpoints. Shell tests updated.
2026-06-14 16:38:24 -07:00
BoulderBadgeDad
519685fc32 video: rework TV-detail to match the artist-detail vibe + real season art
Addresses the 'feels basic' feedback:
- Hero is now a contained glass card with the backdrop blurred INSIDE it +
  gradient overlay (same treatment as the music artist hero) — no more bare
  gaps around the top/sides. Bigger poster, accent external-link chips
  (IMDb/TMDB/TVDB), refined badges + stat tiles.
- Seasons are a poster-art card grid (season = album) with coverage rings/bars
  and hover-lift, selecting one renders its episodes below (episode = track) —
  episode overviews now shown. Mirrors the artist album-grid -> tracklist.
- Scan now captures real per-season posters (Plex sh.seasons() thumbs / Jellyfin
  /Seasons Primary), served via get_art_ref('season') + /api/video/poster/season.
  Falls back to the show poster until a re-scan populates them.

Seam tests for the season art ref; shell markup tests still green.
2026-06-14 16:13:30 -07:00
BoulderBadgeDad
5a2113bd03 video: TV-show detail page (hero + season/episode tree), isolated
Drill-in from a show card: full-bleed backdrop + poster + title/badges/overview +
stat tiles, then seasons->episodes as collapsible accordions with owned/missing
state and per-season coverage bars (season = album, episode = track — inspired by
the music artist page, premium vibe).

- video-detail.js (isolated IIFE) renders from /api/video/detail/show/<id>;
  backdrop/poster via the proxy.
- Show cards dispatch soulsync:video-open-detail; video-side.js navigates to the
  (non-nav) detail subpage; back button reuses data-video-goto.
- Movies stay non-clickable until the movie-detail page lands next.
- .vd-* CSS scoped to the detail page; music untouched. Shell + isolation tests.
2026-06-14 15:59:29 -07:00
BoulderBadgeDad
50632f6392 video enrichment: log each match/not-found at INFO (visible in app.log)
The worker only logged exceptions, so a normal run looked dead — no parity with
the music workers' 'Matched ... -> ID' lines. Now logs each match (noting
'(by server id)' when it used the server's provider id) and each not-found at
INFO. Logger is soulsync.video_enrichment.worker, so it already propagates to
app.log; it just had nothing to say. caplog seam test pins it.
2026-06-14 15:54:53 -07:00
BoulderBadgeDad
9b607b3d1b video: detail-page data layer (show tree + movie) + backdrop proxy
Backend for the upcoming TV/movie detail pages, isolated to video.db:
- show_detail(id): show + seasons->episodes tree with owned/total roll-ups
  (season 0 -> 'Specials', missing-season-row episodes still grouped).
- movie_detail(id): movie + owned flag + best media-file (resolution/quality).
- get_art_ref generalizes the poster ref to poster|backdrop; new
  /api/video/backdrop/<kind>/<id> streams the hero art server-side (Jellyfin
  Backdrop vs Primary handled).
- /api/video/detail/{show,movie}/<id> endpoints.

Seam tests for the tree roll-ups, owned/file, art ref, and both endpoints.
2026-06-14 15:52:22 -07:00
BoulderBadgeDad
a483219746 video enrichment: distinguish transient 'error' from 'not_found' (match music)
A failed lookup CALL (network/429/5xx/timeout, or an expired TVDB token) was
recorded as 'not_found' — permanently logging a transient blip as 'no match'
and parking the item for retry_days. Now mirrors the music workers' proven
pattern:

- New 'error' status, distinct from 'not_found'; enrichment_next retries BOTH
  after retry_days, so errors recover and the queue still advances (no poison
  loop). breakdown/unmatched/retry-all and the modal account for it (shown with
  the outstanding/pending bucket).
- TMDB/TVDB clients raise on non-200 (429/5xx) so the worker records 'error',
  not a false not_found.
- TVDB re-authenticates once on a 401 (expired token) instead of failing every
  match for the rest of the run.

Seam tests: error!=not_found, error retried after window, 429 raises, TVDB
token refresh, UI accounts for errors.
2026-06-14 15:34:45 -07:00
BoulderBadgeDad
fc68a6e741 video enrichment: enrich by the server's provider id, not a title re-search
The deep scan stores tmdb_id/tvdb_id/imdb_id from Plex/Jellyfin, but the workers
only ever searched by title+year and ignored those ids — re-deriving matches the
server already had exact (wasteful, and a title search can mis-match).

enrichment_next now surfaces the row's known provider id; the worker forwards it
and the TMDB/TVDB clients fetch details BY ID (one call, no /search) when it's
present, falling back to title/year search only for items the server couldn't
identify. Still grabs the overview/backdrop the scan doesn't capture.
2026-06-14 15:03:26 -07:00
BoulderBadgeDad
50042607d7 video: amber paused accent + Manage Workers button identical to music
- Paused enrich buttons now get music's exact amber/yellow treatment (gradient,
  border, glow, hover) and an amber tooltip status — was just a flat opacity dim.
- Manage Workers button reuses music's .em-manage-btn* classes verbatim, so the
  logo sits in the same gradient icon-circle with glow and the pill matches
  pixel-for-pixel. Still wired by data-attribute (no inline handler), and music's
  orbs/handler can't touch it (scoped to #dashboard-page). Dropped the old
  bespoke .video-manage-workers-* CSS.
2026-06-14 14:24:32 -07:00
BoulderBadgeDad
8b01ada68a video: pause enrichment workers during library scans, resume after
Every scan (incremental / full / deep, both entry points) now steps the
enrichment workers aside to cut DB lock contention — same as music. Mirrors
music's contract exactly: pause ONLY workers that were running (a user's manual
pause is left alone), track which we paused, and resume just those in a finally
so success, cancel, and error all clean up. Auto-pause is transient
(persist=False) so it never leaks into the saved <service>_paused flag.

Hooks are injected by get_video_scanner; the scanner is inert without them
(tests build it directly, no engine spun up). Isolated to the video side.
2026-06-14 13:55:59 -07:00
BoulderBadgeDad
daecf9dfae video manage-workers modal: TVDB defaults to Shows, not an empty Movies view
TVDB enriches shows only, but selectWorker hardcoded state.kind='movie', so
picking TVDB queried tvdb+movie (always empty) and rendered a bogus Movies
view. Each worker now declares its kinds (tmdb: movie+show, tvdb: show) and the
panel defaults to the worker's first kind. Backend was already safe (returns
empty for unsupported service+kind); pinned that invariant with a seam test.
2026-06-14 13:50:49 -07:00
BoulderBadgeDad
a62e59dd3a video: isolated worker-orbs idle animation on the video dashboard
Port of webui/static/worker-orbs.js into video/video-worker-orbs.js — same
exact animation (physics/draw copied verbatim), but pointed at the video
dashboard header + the TMDB/TVDB enrich buttons + Manage Workers hub. Own
window.videoWorkerOrbs global, activated by the video side's page events;
music's orbs file is untouched and never learns about the video side.

video-enrichment.js feeds it real status as telemetry for the inbound pulses.
7s idle → floating orbs around the SoulSync logo, just like music.
2026-06-14 13:34:10 -07:00
BoulderBadgeDad
9d2468a50f video enrichment: persist worker pause state across restart
Pause/resume now write <service>_paused to video_settings, and the engine
restores each worker's saved pause when it's (re)built — mirrors music's
<service>_enrichment_paused boot flag. Isolated to video.db; music untouched.
2026-06-14 13:11:41 -07:00
BoulderBadgeDad
68a65a8e3d video enrichment modal: per-worker accent theming (matches music vibes)
The modal set no accent vars, so it fell back to the default purple. Now it
sets --row-accent on each rail row and --em-accent / --em-accent-rgb on the
panel from the selected worker — exactly how the music modal themes itself. The
modal's 23 rgba(var(--em-accent-rgb)) / var(--row-accent) rules now render in
TMDB light-blue / TVDB purple per selection.
2026-06-14 13:02:03 -07:00
BoulderBadgeDad
924d4ce8a6 video enrichment: fix accent rendering (was invalid rgba) + colors + TVDB invert
Root cause both buttons looked black: --ve-accent was space-separated
(1 180 228) but used in rgba(var(--ve-accent), a) -> invalid 'rgba(1 180 228, a)'
so the color silently failed. Switched --ve-accent to comma-separated (matching
music's --accent-rgb) and fixed all fallbacks -> the accent + glow now render.
- TMDB: vibey light blue (56,189,248); TVDB: purple (168,85,247).
- TVDB brand mark inverted everywhere (dashboard button, modal rail, settings
  frame) so the dark logo reads on the dark UI.
2026-06-14 12:59:22 -07:00
BoulderBadgeDad
798721d724 video enrichment: per-service accent on buttons + tooltips (the 'vibes')
The buttons were flat and the reused .tooltip-content rendered in music's
default purple. Now everything is driven by --ve-accent (on the container so the
tooltip inherits it):
- buttons mirror music's .musicbrainz-button — always-on accent glow, accent
  gradient/scale on hover, bigger glow when active; logo 24px @0.85 opacity with
  drop-shadow; spinner uses the accent.
- tooltip-content border/glow/arrow/header + the status value are tinted with
  the service accent (TMDB blue / TVDB green), overriding music's purple default
  scoped to the video tooltip only. Music untouched.
2026-06-14 12:47:21 -07:00
BoulderBadgeDad
0e58973d43 video enrichment: real TMDB/TVDB logos + music-style hover tooltips
- Logos (the URLs you gave) everywhere the services are listed: dashboard
  buttons, Manage-Workers modal rail, and the settings API-config frames —
  matching how music shows enrichment-service logos.
- Dashboard hover tooltips now reuse music's shared .tooltip-content/-header/
  -body/-status/-current/-progress classes + the same positioning, so they look
  identical to the music enrichment tooltips (Status / current item / Progress)
  instead of my bespoke style. Music CSS untouched (shared classes, reused).
2026-06-14 12:38:35 -07:00
BoulderBadgeDad
776afdd1fd video enrichment: survive legacy UNIQUE on tmdb_id/tvdb_id (no scan crash)
Existing DBs created before the schema dropped UNIQUE still have shows.tvdb_id /
movies.tmdb_id UNIQUE, so enrichment matching two items to the same id threw
'UNIQUE constraint failed' repeatedly. enrichment_apply now catches the
IntegrityError and retries without the id columns — keeps the existing
(authoritative) id and still records match_status + metadata. Non-destructive
(no table rebuild). Test simulates the legacy unique index.
2026-06-14 12:34:52 -07:00
BoulderBadgeDad
2901e4ec4d video settings: per-connection Test button (mirrors music's testConnection)
Each video connection item (TMDB/TVDB) now has a Test button that behaves like
music's: saves the key, hits POST /api/video/enrichment/<svc>/test, and toasts
the result via the shared showToast — isolated (own endpoint, own data-attr
handler, reuses the .test-button CSS).
- Client .test() pings TMDB /configuration and TVDB /login to verify the key.
- Endpoint returns {success,message,error}; unknown service -> 404.
94 tests green; music untouched.
2026-06-14 12:26:15 -07:00
BoulderBadgeDad
d35bb695ae video settings: stop music's verify loop choking on tvdb/tmdb ('Unknown service')
The video API-key frames had the real data-service attribute, so music's
settings.js verify loop (#settings-page .stg-service[data-service]) picked them
up and errored 'Unknown service: tvdb' — and it ran on the music side too
(shared DOM), so it WAS impacting music. Renamed to data-video-service: same
identical .api-service-frame look, but music's [data-service] selector can't
match them. Music untouched again.
2026-06-14 12:16:44 -07:00
BoulderBadgeDad
941ebd42d9 video enrichment 3a: isolated Manage-Workers modal
The dashboard 'Manage Workers' button now opens a video enrichment modal that
reuses music's global .em-* modal CSS (identical look) but is entirely its own,
isolated JS: own #vem-overlay, event-delegated (no inline handlers, no music
function calls), targets /api/video/enrichment, shows only TMDB/TVDB with
movie/show coverage.
- Rail of workers (status dot + coverage), panel with pause/resume, per-kind
  coverage cards (matched/not-found/pending segmented bars), and a paged
  unmatched browser with retry (item + retry-failed).
- Polls the selected worker every 3s. The few invented sub-classes are styled
  scoped to #vem-overlay so music is never affected. 87 tests green.
2026-06-14 12:04:05 -07:00
BoulderBadgeDad
0afd37351d test: relax video-enrichment isolation check (comment mentions music side)
The whole-file 'music' substring check tripped on a comment ('only polls on the
video side, so the engine isn't spun up on the music side'). Replaced with the
meaningful checks: no music API path (/api/enrichment/) and no music modal call
(openEnrichmentManager).
2026-06-14 11:53:09 -07:00
BoulderBadgeDad
aef80eb52d video enrichment 2: dashboard TMDB/TVDB buttons + Manage Workers (isolated)
Brings the worker buttons back onto the video dashboard header as real, live
controls — isolated (own CSS classes + own JS + /api/video/enrichment), music
untouched.
- TMDB/TVDB round buttons with per-service accent, a spinner that spins while
  the worker runs, and a hover tooltip (status / current item / progress).
- video-enrichment.js polls /api/video/enrichment/<svc>/status (only on the
  video side, so the video engine isn't spun up on the music side); click
  toggles pause/resume. Manage Workers button fires soulsync:video-open-workers
  for the modal (Phase 3).
86 tests green.
2026-06-14 11:52:31 -07:00
BoulderBadgeDad
65ff84aa6a video enrichment 1d: wire TMDB/TVDB API keys (workers turn on)
- GET/POST /api/video/enrichment/config saves the keys into video_settings;
  POST rebuilds the engine (stop old workers, rebuild clients with new keys) so
  they pick up the change live.
- video-settings.js loads the saved keys into the TMDB/TVDB fields on the
  Connections tab and saves them on change (workers enable once a key is set).
Backend is now end-to-end: key -> client.enabled -> worker matches the library
to TMDB/TVDB and fills ids + metadata. 91 tests green; real DB untouched.
2026-06-14 11:26:09 -07:00
BoulderBadgeDad
80679b02ba video enrichment 1c: isolated /api/video/enrichment blueprint
Mirrors the music enrichment API so the shared Manage-Workers modal can drive
video workers by pointing at /api/video/...:
- GET services; GET <service>/status (worker.get_stats); POST pause/resume;
  GET breakdown; GET unmatched (paged, kind/status/q); POST retry.
- Unknown service -> 404. Engine via the lazy singleton; DB queries via the
  isolated video DB. 6 API tests (services/status/breakdown/unmatched/pause/
  resume/retry/404) with an injected engine + fake clients.
2026-06-14 11:23:47 -07:00
BoulderBadgeDad
c17138bf7d video enrichment 1b: worker engine + TMDB/TVDB clients
Isolated enrichment subsystem mirroring the music worker pattern, on video.db.
- VideoEnrichmentWorker: daemon loop pulling enrichment_next -> client.match ->
  enrichment_apply; pause/resume/stop; get_stats in the same shape the music
  enrichment API returns (enabled/running/paused/idle/current_item/stats/
  progress/breakdown). Client injected -> loop fully unit-tested with a fake.
- TMDB/TVDB clients (thin adapters, keys from video_settings): .enabled +
  match(kind,title,year) -> {id, metadata}; validated live, worker logic tested.
- VideoEnrichmentEngine registry + lazy get_video_enrichment_engine() singleton.
8 tests (match/not_found/error-resilience/stats/disabled/engine/isolation).
2026-06-14 11:21:30 -07:00
BoulderBadgeDad
093e14bd5d video enrichment 1a: DB layer (match-status cols + migration + helpers)
Foundation for the video enrichment workers, mirroring music's per-source
columns/queries on video.db.
- Schema: tmdb_match_status/tmdb_last_attempted on movies; tmdb_+tvdb_ on shows.
  Idempotent ALTER-TABLE migration adds them to existing DBs on init.
- VideoDatabase helpers (service+kind -> columns map):
  enrichment_next (pending first, then not_found past retry window),
  enrichment_apply (sets id/status/last_attempted + whitelisted metadata,
  backfill-safe), enrichment_breakdown, enrichment_unmatched (paged), and
  enrichment_retry. Same shape as music's enrichment API so the shared modal can
  drive it. 30 DB tests green.
2026-06-14 11:18:46 -07:00
BoulderBadgeDad
13e03a624c video scan: capture the provider IDs the server already has (tmdb/imdb/tvdb)
The servers already matched everything to their agents — we were dropping the
IDs. Now we store them:
- Plex: parse item.guids (imdb://, tmdb://, tvdb://); Jellyfin: parse
  item.ProviderIds (added ProviderIds to the requested Fields).
- Stored on movies (tmdb_id, imdb_id), shows (tvdb_id, tmdb_id, imdb_id), and
  episodes (tvdb_id) via the upserts.
- Dropped the over-strict UNIQUE on movies.tmdb_id / shows.tvdb_id (same title
  can legitimately live in two libraries; we dedupe on server_id). Scanner now
  wraps each upsert in try/except so one bad item can't abort a scan.
Tests: guid/ProviderIds parsing + IDs persisted. 38 video-DB/scanner tests green.
2026-06-14 11:04:54 -07:00
BoulderBadgeDad
9eecd6d2c4 video Library: fix singular/plural kind bug (no posters + 'eps' on movies)
load() passed the plural API kind (movies/shows) to the card renderer + poster
URL, which expect the singular (movie/show). Result: movie cards fell into the
'shows' branch (bogus '0/0 eps') and poster URLs were /api/video/poster/movies/..
-> 404 -> no images on either tab. Now apiKind (plural) is used for the query
and cardKind (singular) for cards + poster proxy.
2026-06-14 10:20:27 -07:00
BoulderBadgeDad
be141d0073 video dashboard: Library card is a live scan tool (parity with music)
The Refresh/Deep Scan buttons already fired a scan, but the card gave no
feedback so it looked dead. Now it mirrors music's dashboard library card:
- a progress section (phase + bar + detail) appears during a scan, driven by
  the shared scan events (real percent);
- buttons disable while scanning;
- the card hydrates on load/return — if a scan is already running, video-scan.js
  re-emits progress and the card shows it;
- stats refresh when the scan finishes.
Reuses music's .library-status-progress classes. 84 tests green.
2026-06-14 09:28:14 -07:00
BoulderBadgeDad
68582af374 video Library: server-side paging + sort/filter + card badges (music parity)
Handles big libraries (your ~8500 movies) like music does instead of rendering
everything at once.
- DB: sort_title populated article-aware on upsert ('The Matrix' files under M);
  query_library(kind, search, letter, sort, status, page, limit) does all
  filtering/sorting/paging in SQL and returns music's pagination shape
  {page,total_pages,total_count,has_prev,has_next} + badge fields (resolution,
  owned/episode counts).
- GET /api/video/library now takes those params (per kind) instead of dumping
  everything.
- Library page: 75/page with ← Previous / Page X of Y / Next → (music's exact
  controls/classes), Sort (Title/Year/Recently Added) + Owned/Wanted filter,
  server-side search + A–Z. Cards gain a resolution chip (4K/1080p/…) and the
  owned/wanted meta. Still not clickable.
124 tests green.
2026-06-14 08:57:25 -07:00
BoulderBadgeDad
12789b6be8 video dashboard: title 'System Dashboard' to match music
Music's dashboard is 'System Dashboard' (neutral, no domain word), so the video
dashboard matches it instead of the inconsistent 'Dashboard'.
2026-06-14 08:44:31 -07:00
BoulderBadgeDad
fd8d305651 video side: drop redundant 'Video' prefix from page headers
On the video side the context is already video, so 'Video Dashboard' ->
'Dashboard' and the dashboard library card 'Video Library' -> 'Library'.
2026-06-14 08:42:02 -07:00
BoulderBadgeDad
9ae8c243c9 video Tools: rehydrate scan state on page refresh (parity with music)
The scan runs server-side, so on load video-scan.js now polls
/api/video/scan/status and, if a scan is in progress, restores the live UI
(Cancel button, moving progress bar, phase) and resumes polling — instead of
showing Idle after a refresh. Also re-checks when the Tools page is shown.
Because it re-emits the progress event, the Library/Dashboard scan affordances
rehydrate too.
2026-06-14 07:59:44 -07:00
BoulderBadgeDad
405e7097e3 video scan: align incremental + deep with music's logic
- Incremental now does smart early-stopping like music: skips already-known
  items and stops after 25 consecutive known (server lists recent first),
  instead of a blind fixed cap. Falls back to a full pass when the library is
  near-empty (<50), matching music's small-DB behavior.
- Deep scan gains music's 50% safety threshold: if removal would wipe >50% of a
  >100-row library, it skips (assumes a partial server response, not a real
  emptying) — prevents catastrophic deletion.
- Full Refresh already matched (re-read all, upsert, no removal).
Added DB helpers (server_ids, table_count). Tests: early-stop skips known,
small-lib fallback, 50% prune safety. 122 tests green.
2026-06-14 07:56:23 -07:00
BoulderBadgeDad
a5ff4a1dd8 video Tools: server-prefix the scan title (Plex/Jellyfin Library Scan)
Dashboard endpoint now returns the active media server; the Tools card title
becomes '<Server> Library Scan' (e.g. 'Plex Library Scan'), matching how music
prefixes 'Plex Database Updater'.
2026-06-14 07:47:40 -07:00
BoulderBadgeDad
b5d0b76fe6 video Tools: match music Database Updater (stats, cancel, real progress)
The scan tool now behaves like music's, not just looks like it:
- Card matches: help '?' button, 'Last Scan' line, and the Movies/Shows/
  Episodes/Size stats grid (populated from /api/video/dashboard on show + after
  a scan). Same .tool-card-stats markup.
- Real progress bar: scanner fetches item totals up front (Plex section.
  totalSize / Jellyfin TotalRecordCount) and reports a true percent as it
  processes; the bar actually moves (movies → shows) instead of sitting at 100%.
- Cancel: the Scan button toggles to 'Cancel' mid-scan and POSTs
  /api/video/scan/stop; the scanner checks a cancel flag between items and ends
  in a 'cancelled' state. Mirrors music's stop affordance.
Tests: percent reported, cancel stops midway + saves only processed items, stop
route registered, tool-card structure. 117 video/integrity tests green.
2026-06-14 07:35:57 -07:00
BoulderBadgeDad
da3f89314b video settings (Library tab): hide music-only content on video side
The Library tab is entirely music-specific today (file-org templates, music
library paths, post-processing, conversion, listening stats), so hide all of it
on the video side via one rule. Future video library settings (root folders,
video naming) just need data-video-only to remain visible. Music side untouched.
2026-06-14 00:55:35 -07:00
BoulderBadgeDad
6f653f190a video settings (Downloads): hide music-only path options
Marked data-music-only (hidden on the video side): Music Videos Dir, Playlists
Folder, Playlist Folder Style, M3U Entry Base Path — all music-specific. Music
side unchanged.
2026-06-14 00:54:21 -07:00
BoulderBadgeDad
9974845470 video scan: harden Jellyfin adapter (paging + per-item resilience)
- Paginate Jellyfin movie/series listings (StartIndex/Limit, 500/page) so large
  libraries aren't truncated on full/deep scans; incremental stays a single
  capped page.
- Per-item try/except in iter_movies/iter_shows (matches Plex) so one bad item
  doesn't abort the scan.
- Skip Jellyfin episodes with no IndexNumber (consistency with Plex; avoids
  ep-0 collisions). All three modes now solid for Plex + Jellyfin.
2026-06-14 00:53:07 -07:00
BoulderBadgeDad
fba47e9665 video Library page: music-library visuals + posters + search + A-Z
Rebuilt the Library page to reuse the music library's exact look — no
reinvention, just new data:
- Same classes: .library-container, .library-artist-card grid, .alphabet-
  selector, .library-search-input, loading/empty states. Movies/Shows tab pill
  is the only video-specific bit.
- Real posters via a server-side proxy: GET /api/video/poster/<kind>/<id>
  streams the Plex/Jellyfin artwork (token stays server-side); cards fall back
  to an emoji on miss. list_movies/list_shows now expose has_poster (no raw
  server paths leaked).
- Client-side search + A-Z letter filter (article-aware) over the loaded set;
  cards are divs (not clickable yet, per request). Scan button in the header
  reuses the shared scan controller and reloads on done.
110 tests green.
2026-06-14 00:51:47 -07:00
BoulderBadgeDad
9d3cf14cdd video settings: video API placeholders use the exact music service structure
Standardized the TMDB/TVDB placeholders to the same .api-service-frame
.stg-service accordion markup as every music API service (header +
toggleStgService accordion + body with API Key field + callback-info), plus the
same 'Expand All' header. No bespoke structure. Reuses the existing accordion
handlers (already defined, integrity test green).
2026-06-14 00:37:20 -07:00
BoulderBadgeDad
e982b19bc5 video settings: hide music API Configuration, add video API placeholders
On the video side the API Configuration section (Spotify/Tidal/Deezer/etc.) is
all music — hidden now (group marked data-music-only). In its place, a video API
Configuration group (data-video-only) with disabled TMDB + TVDB placeholders for
the metadata sources we'll likely use. Music side unchanged.
2026-06-14 00:34:42 -07:00
BoulderBadgeDad
7c8f06f427 video settings: hide the music-library picker on the video side
Video side was showing both the Music Library selector and the new Movies/TV
selectors. The music-library picker is irrelevant there (the Movies/TV mapping
replaces it), so hide it on the video side — music side is unchanged.
2026-06-14 00:31:02 -07:00
BoulderBadgeDad
41f6558c2d video settings: library mapping saves on change (match music), drop button
The Movies/TV selectors now save the moment you pick one — same as the music
'Music Library' selector right above them — instead of a separate 'Save
Libraries' button. Removed the button and the copied 'doesn't affect config
file' caption; a small inline status shows 'Saved'.
2026-06-14 00:29:53 -07:00
BoulderBadgeDad
576199bb2f video side: Movies/TV library mapping UI on the settings page
Right next to music's 'Music Library' selector, the video side now shows
'Movies Library' + 'TV Shows Library' dropdowns (data-video-only, hidden on the
music side). video-settings.js populates them from /api/video/libraries when
Settings opens on the video side and saves the choice back; the scanner then
reads only those libraries. Isolated IIFE, data-attr wired. 83 tests green.
2026-06-14 00:26:56 -07:00
BoulderBadgeDad
5b0b64bf3b video side: library mapping backend (pick Movies/TV library)
The scan no longer blindly grabs every movie/show section — it reads the
libraries you map, like music's 'pick your Music library'.
- GET /api/video/libraries: discover the active server's Movies/TV libraries
  (Plex sections by type / Jellyfin views by CollectionType) + current
  selection. POST: save {movies, tv} per server into video_settings.
- sources.py: _build_source(movies_lib, tv_lib) filters to the mapped library;
  get_active_video_source() (used by the scanner) loads the saved selection;
  list_video_libraries() lists them unfiltered for the UI. Falls back to all
  libraries when nothing is mapped yet.
- VideoDatabase.get/set_library_selection (per-server). 6 tests added; 33 green.
2026-06-14 00:24:01 -07:00
BoulderBadgeDad
0d86d84307 video side: Settings shows the real music settings page (identically)
Video Settings was a 'coming soon' placeholder. Now it reuses the actual
#settings-page, shown identically for now (no hiding of music-only bits yet) —
the foundation for adding the Movies/TV library mapping next.
- video-side.js: SHARED_PAGES maps video-settings -> the music 'settings' page;
  showPage sets body[data-video-page] and triggers the shared loadPageData
  loader (same init music navigation uses) instead of a video subpage.
- CSS reveals #settings-page (and hides the video host) when
  data-side=video + data-video-page=video-settings; id selector outranks the
  blanket music-page hide. 81 tests green.
2026-06-14 00:17:00 -07:00
BoulderBadgeDad
061079f0f6 video scan: don't crash on episodes with no episode number
Plex specials/unmatched episodes can have a null index -> getattr(...,0) still
returned None -> 'NOT NULL constraint failed: episodes.episode_number'.
- Plex adapter skips episodes with no index (logged), passes a real number.
- upsert_show_tree defensively skips any episode missing season/episode number,
  so no source can crash a scan. Test added.
2026-06-14 00:13:08 -07:00
BoulderBadgeDad
bc334df719 video scan: fix Plex read-timeout on large libraries
The scan inherited the shared client's 15s interactive timeout and fetched
episodes per-season (one request each), so a big library read-timed-out
mid-scan (port 32400).
- Dedicated Plex connection for scans with a 120s timeout (built from the
  shared config; doesn't touch the interactive client).
- Fetch a show's episodes in ONE show.episodes() call grouped by season,
  instead of seasons()+episodes() per season — far fewer round-trips.
- Per-item try/except in iter_movies/iter_shows so one slow/bad item is skipped
  and logged, never aborting the whole scan.
2026-06-13 23:58:04 -07:00
BoulderBadgeDad
7ab62674ed video side: reuse music styling for Tools + dashboard scan controls
The visuals were off because I'd invented CSS/markup instead of reusing the
shared design system. Fixed to match music exactly:
- Dashboard Library card now uses music's full markup — header icon, Refresh/
  Deep Scan buttons WITH their icons, and stat rows with icons (movies/shows/
  episodes/disk). Same .library-status-* classes, no custom CSS.
- Tools 'Library Scan' card now mirrors the music Database Updater: a mode
  dropdown (Incremental/Full Refresh/Deep Scan) + one Scan button inside
  .tool-card-controls + the standard progress bar. Styling comes for free from
  the generic music classes.
- Dropped bespoke .video-tool-btn/.video-scan-controls CSS and folded the
  separate video-tools.js into the shared video-scan.js (one fewer file). JS
  stays isolated only because it must hit /api/video + update video DOM.
110 tests green.
2026-06-13 23:50:41 -07:00
BoulderBadgeDad
71f126f9d2 video side: Tools page + scan controls on dashboard (3 modes)
- New Tools page (video nav + subpage, mirrors music tools styling): a Library
  Scan tool card with Incremental / Full Refresh / Deep Scan buttons + a live
  status line. Room for more maintenance jobs later.
- Dashboard Library card now has Refresh (full) + Deep Scan buttons, like the
  music dashboard.
- Shared video-scan.js controller: one place triggers + polls scans for all
  surfaces (wires any [data-video-scan-mode]/[data-video-scan]); emits
  soulsync:video-scan-progress/done. Library/Tools/Dashboard just listen — no
  duplicated fetch/poll. video-library.js refactored onto it; dashboard reloads
  stats on scan-done.
- All isolated IIFEs, data-attr wired (no inline onclick). video-tools added to
  the nav (13 pages). 110 tests green.
2026-06-13 23:34:28 -07:00
BoulderBadgeDad
04fb19c80c video scan: three modes (full refresh / incremental / deep)
Mirrors the music model (full_refresh vs smart incremental, plus deep_scan):
- incremental: only recently-added items from the server (Plex addedAt:desc /
  Jellyfin DateCreated, capped); upsert; no prune.
- full: every item; upsert all (refresh metadata + add new); no prune.
- deep: every item; upsert; prune what the server no longer has (empty-scan
  safety preserved).
scanner.request_scan/scan_sync take mode; /api/video/scan/request reads
{mode} from the body (default full); adapters take incremental=. Tests cover
deep-prunes / full-doesn't / empty-deep-safety / incremental-requests-recent.
2026-06-13 23:28:57 -07:00
BoulderBadgeDad
d7ab68c067 video side: Library page (lists movies/shows, scan trigger)
- GET /api/video/library -> {movies, shows} from video.db (VideoDatabase.
  list_movies/list_shows; shows carry episode_count + owned_count).
- Library page (video-library subpage, isolated video-library.js): tabbed
  Movies/Shows grid of poster cards, count, empty-state. A 'Scan Library'
  button POSTs /api/video/scan/request then polls /api/video/scan/status,
  showing live phase/counts, and refreshes the grid when done.
- Reuses the music dashboard-header chrome (icon title, sweep hidden) + the
  watchlist-button styling for the scan button; video-card grid styles added.
- All data-attr wired (no inline onclick); module is an isolated IIFE that
  listens for soulsync:video-page-shown. 105 tests green.

Now: video.db -> scanner -> /api/video -> live dashboard + Library page, all
isolated from music. Scanner adapters await live Plex/Jellyfin validation.
2026-06-13 23:17:48 -07:00
BoulderBadgeDad
6665ecaa12 video side: library scanner (server = source of truth) + scan API
Reads the active media server and mirrors it into video.db, adapting the music
scan pattern (ask the server, upsert, prune what's gone) — isolated from music.
- core/video/scanner.py: server-agnostic VideoLibraryScanner. Consumes a media
  source (duck-typed) yielding normalized dicts; upserts movies + show trees,
  prunes removed items, reports progress/state. Skips pruning when a scan
  returns nothing (transient-failure safety). Background thread + scan_sync.
- core/video/sources.py: Plex + Jellyfin adapters that REUSE the shared
  connected clients (MediaServerEngine) but own all video-section logic; produce
  normalized dicts. (Validated against a live server by design; scanner itself
  is fully unit-tested with a fake source.)
- api/video/scan.py: POST /api/video/scan/request, GET /api/video/scan/status.
- .gitignore: video_library.db + sidecars (mirrors music); tests inject a
  tmp DB so none is ever created in the repo.
Tests: scan populate/prune/empty-safety/no-source-error, isolation guard
(core/video imports nothing from music), scan routes registered. 101 green.
2026-06-13 23:13:50 -07:00
BoulderBadgeDad
462fa50423 video DB: server-sourced scan upserts (movies/shows/seasons/episodes)
Server (Plex/Jellyfin) is the source of truth, so every scanned row carries
(server_source, server_id) for upsert + stale-removal — mirroring how music
keys on server_source + ratingKey.

- schema: server_source/server_id columns on movies/shows/episodes (+ server_id
  on seasons); unique (server_source,server_id) on movies/shows (multiple NULLs
  allowed so wishlist rows never block).
- VideoDatabase.upsert_movie / upsert_show_tree: take normalized, server-
  agnostic dicts (a Plex/Jellyfin adapter produces them — DB never touches a
  media SDK), set has_file + media_files, and prune episodes/seasons the server
  no longer reports.
- prune_missing(): removes top-level movies/shows the scan didn't see (cascades
  clean children).
6 new tests (insert/update/file-replace, season/episode build+prune, top-level
prune); 18 video-DB tests green.
2026-06-13 23:02:03 -07:00
BoulderBadgeDad
401a9be0ec video side: live dashboard via isolated /api/video blueprint
First wire from video.db -> UI, kettui-style.
- api/video/ : isolated Flask blueprint (registered at /api/video with one
  additive line in web_server.py). Reads only video.db; imports nothing from
  the music API or DB.
- GET /api/video/dashboard -> VideoDatabase.dashboard_stats(): live library/
  download/watchlist/wishlist counts (real 0s on an empty DB).
- video-dashboard.js now fetches it and fills the stat cards + Watchlist/
  Wishlist header badges (formatted bytes/speed); falls back to zeros on error.
  uptime/memory stay at markup defaults for now (not video-domain).
- Tests: dashboard_stats counts (empty + populated), endpoint returns zeroed
  JSON via a Flask test client, blueprint exposes the route, and the video API
  imports nothing from music. 93 video/integrity tests green.
2026-06-13 22:40:48 -07:00
BoulderBadgeDad
402a1fec50 video side: video.db schema + isolated VideoDatabase
Separate SQLite file (database/video_library.db, env VIDEO_DATABASE_PATH),
fully disconnected from music — never imported by music, imports nothing from
music, no shared write lock.

Schema (database/video_schema.sql), designed to dodge the music DB's known
pain points:
- movies; shows->seasons->episodes; channels->channel_videos (YouTube as a
  first-class peer); media_files (the library); downloads (queue+history);
  activity feed; root_folders / quality_profiles / video_settings config.
- No polymorphic ids: media_files/downloads use separate nullable FKs + a CHECK
  that exactly one owner is set; real cascades.
- Explicit external-id columns (tmdb/tvdb/imdb/youtube), no source-id blob.
- Watchlist/Wishlist/Calendar are DERIVED VIEWS over monitored + file state
  (single source of truth, can't drift like music's wishlist table did).

VideoDatabase mirrors music's conventions (WAL, foreign_keys ON, 30s busy
timeout, Row factory, once-per-process init, user_version backstop) but is an
independent implementation. 13 seam tests: schema builds, CHECK constraints
reject bad rows, cascades fire, views return correct membership, KV roundtrips,
and a guard that the module imports nothing from music.
2026-06-13 22:30:55 -07:00
BoulderBadgeDad
dbe35fc023 video dashboard: drop the TMDB/TVDB/Trakt/OMDb placeholder row
Pointless until the real enrichment workers exist. Header keeps the icon
title, subtitle, Watchlist/Wishlist quick-nav and (hidden) sweep; the worker
button row will land later, matching music.
2026-06-13 20:06:26 -07:00
BoulderBadgeDad
a57951c038 video dashboard: header matches music (sweep hidden, video meta + quick-nav)
The video dashboard header now mirrors music's: icon + shimmer title,
subtitle, the Watchlist/Wishlist quick-nav (top-right), and the action-button
row. Differences, all isolated:
- Sweep band kept in markup but hidden on the video side (no animation for
  now; meta-source-driven equivalent may return later).
- Quick-nav reuses .watchlist-button/.wishlist-button styling but carries NO
  music IDs (no duplicate IDs, no music-JS binding) — navigates to the video
  Watchlist/Wishlist pages via data-video-goto.
- header-actions holds disabled TMDB/TVDB/Trakt/OMDb placeholder chips
  (.video-meta-button) standing in for music's enrichment buttons until the
  video meta sources are wired.
No inline onclick; 75 tests green.
2026-06-13 19:57:22 -07:00
BoulderBadgeDad
c84231dd4f video side: build the Dashboard page (mirrors music, isolated data)
Real first video page, reusing music's .dash-grid/.dash-card CSS for an
identical look — but every value is driven by isolated video JS, no music
code referenced.

Sections mirror the music dashboard, adapted:
- Service Status: Media Server / Download Client / Metadata Source
- System Stats: swaps 'Active Syncs' -> 'Disk Usage'; keeps download/speed/
  uptime/memory
- Library: Movies / Shows / Episodes / Disk Size
- Recent Syncs -> Recent Downloads (empty state for now)
- Quick Actions: Add Movie/Show, Watchlist, Downloads (navigate via
  data-video-goto)
- Recent Activity
- No enrichment section, no header sweep animation (per plan)

Mechanics:
- #video-page-host now holds .video-subpage sections; controller toggles one
  at a time and falls back to #video-placeholder-slot for unbuilt pages.
- video-side.js dispatches soulsync:video-page-shown; video-dashboard.js (new
  isolated IIFE) listens and applies a zeroed STUB until video.db exists.
  Single seam to swap for a real /api/video/dashboard fetch later.
- All wiring via data-attrs + addEventListener; no inline onclick (keeps the
  script-split integrity contract intact). 73 tests green.
2026-06-13 19:49:09 -07:00
BoulderBadgeDad
9330d66fcd video side: add Wishlist nav page
Completes the Watchlist+Wishlist pair (same as music). Watchlist monitors
shows/channels for new content; Wishlist is the wanted/missing queue
(movies, one-offs, failed grabs to retry). Placeholder for now.
2026-06-13 19:41:46 -07:00
BoulderBadgeDad
e3d3f453da video side: add Watchlist + Downloads nav pages
Following (Watchlist) and the download queue (Downloads) are core to a
movies/TV/YouTube manager — same names as music so they read intuitively.
Both wired via data-video-page (no inline onclick); placeholder for now.
2026-06-13 19:36:13 -07:00
BoulderBadgeDad
f5c0f3ca31 video side: subtitle -> 'Movies, TV & YouTube' (matches music length, describes the side) 2026-06-13 19:26:39 -07:00
BoulderBadgeDad
ac4bee75ef video side: 'Video Manager' subtitle + animated sliding-pill toggle
- Subtitle on the video side is now 'Video Manager' (was 'Video Sync & Manager').
  Music keeps 'Music Sync & Manager' — sync fits music, not video.
- Toggle is now a proper animated pill: a gradient thumb slides under the active
  side (CSS-driven off body[data-side], spring easing), each side has a small
  icon (music note / film), inset track. Still data-attr wired, no inline onclick.
2026-06-13 19:24:10 -07:00
BoulderBadgeDad
3202197740 video side: Music <-> Video sidebar toggle + video nav shell (isolated)
First slice of the video side, on the experimental branch. Purely additive and
fully isolated from music:
- A Music | Video toggle in the sidebar header; clicking flips body[data-side]
  (remembered in localStorage). The shared shell (logo, user, Support, Version)
  stays; only the nav set + subtitle swap.
- A second sidebar nav (.video-nav) with the video pages — Dashboard, Search,
  Discover, Library, Calendar, Import, Settings, Issues, Help & Docs — shown via
  CSS off body[data-side]. Service Status is hidden on the video side.
- A placeholder content host; real video pages land later.

Isolation contract held: index.html is +51/-0 (no music markup changed), music
JS/CSS untouched, nothing in music references the controller. The controller
(webui/static/video/video-side.js) is a self-contained IIFE wired purely via
addEventListener (no globals, no inline onclick) — so it can't affect music and
doesn't trip the script-split-integrity contract.

Tests: 6 video-shell structural/isolation tests + 64 script-integrity green.
2026-06-13 19:13:54 -07:00
448 changed files with 53757 additions and 30553 deletions

View file

@ -9,9 +9,9 @@ on:
workflow_dispatch: workflow_dispatch:
inputs: inputs:
version_tag: version_tag:
description: 'Version tag (e.g. 2.8.2)' description: 'Version tag (e.g. 2.7.2)'
required: true required: true
default: '2.8.2' default: '2.7.2'
jobs: jobs:
build-and-push: build-and-push:

15
.gitignore vendored
View file

@ -12,12 +12,15 @@ __pycache__/
# User-specific files (auto-created by the app if missing) # User-specific files (auto-created by the app if missing)
config/config.json config/config.json
config/youtube_cookies.txt database/music_library.db
# All app databases are live user data — never commit (music_library, video_library, …) database/music_library.db-shm
database/*.db database/music_library.db-wal
database/*.db-shm database/music_library.db.backup_*
database/*.db-wal database/video_library.db
database/*.db.backup_* database/video_library.db-shm
database/video_library.db-wal
database/video_library.db.backup_*
database/*.histbak
database/api_call_history.json database/api_call_history.json
storage/image_cache/ storage/image_cache/
logs/*.log logs/*.log

View file

@ -1,74 +0,0 @@
# discover page — best in class plan (#913 + full generator audit)
morning notes. did the work overnight. tl;dr at top, details below, all of it `break nothing` + tested.
## what i shipped tonight (done, tested, safe)
### 1. listening recommendations (#913) — went from BROKEN to best-in-class
the feature was silently producing **zero** recs on real data. dug in and found three stacked bugs in the generation:
- **wrong id key (the killer).** `similar_artists.source_artist_id` is a *source* id (spotify/itunes/deezer), but the scanner built its id→name map from `artists.id` (the internal row id). so every edge resolved to nothing → 0 recs. proved it on your live db: internal-id join = 0 rows, spotify-id join = 71,636 rows.
- **consensus could never fire.** it fed the ranker `get_top_similar_artists`, which does `GROUP BY similar_artist_name` + `MAX(source_artist_id)` — collapsing every similar artist down to a *single* seed. the whole point of the ranker is "artist X is similar to 3 of your seeds = strong signal," and that signal was being flattened away before it ever reached the ranker.
- **similarity strength thrown away.** each edge stores a 1-10 closeness rank; it was ignored (everything weighted equally).
the fix (all in the pure, tested core + thin scan wiring):
- build id→name from the **source-id columns**, query the **raw per-seed edges** (consensus preserved), and thread **similarity_rank** into the score so a seed's closest matches count for more.
- **recency-weighted seeds**: `weight = lifetime_plays + 1.5 × recent_30d_plays`. picks now track what you're into *now*, not just all-time totals.
result on your actual library (simulated through the real code path): **40 recommendations, 13 with multi-seed consensus, all 40 with cached art.** top picks: Arcangel (Bad Bunny + Ozuna + J Balvin), Melanie Martinez (Ariana + Billie), Maluma, De La Ghetto — all coherent, all explainable.
### 2. its own row on the discover page
new row **"Based On Your Listening"** — play-weighted, consensus-ranked artist cards with a **"Because you listen to X, Y"** line. sits right above the library-driven "Recommended For You" row. purely additive: new endpoint `/api/discover/listening-recommendations`, new loader, hides itself when empty.
**you need to run one watchlist scan** for the row to populate (the data regenerates during the scan — i did NOT touch your live db). before that scan the row just stays hidden; after it, it fills in.
> note: this is deliberately different from the existing "Recommended For You" row. that one is driven by your *whole library / watchlist*. this one is driven by your *actual listening intensity* — the ~30 artists you really play, not the thousands you happen to own.
### 3. Fresh Tape "only 5-10 tracks" — fixed
root cause: `get_discovery_recent_albums` orders `release_date DESC`, so announced-but-unreleased albums sort to the *top* and ate the 50-album budget. the scanner skipped them *after* the budget was already spent → only a handful of released albums left → 5-10 tracks. fixed by fetching a generous budget (300) **and** excluding next-year albums at the query, so released albums fill the budget. the precise same-year `is_future_release` skip stays as a second guard. downstream caps (6/artist, top 75, take 50) unchanged.
**tests:** 25 pure-core cases (consensus/similarity/recency) + 2 Fresh Tape regression tests, all green. full discovery suite (255) green. nothing else touched.
---
## best-in-class roadmap for listening recs (next phases — your call)
these are the levers to take it further. ordered by value-to-risk. none are required; tonight's work stands on its own.
| phase | what | value | risk | notes |
|---|---|---|---|---|
| **3** | **playable track row** ✅ DONE | high | low-med | shipped: "🎧 Your Listening Mix" row — a track playlist (play/queue/download/sync) right under the artist row. stored as full render-ready dicts (not pool-hydrated, so it can't shrink on pool rotation like Fresh Tape does). |
| **4** | **direct top-tracks fetch** ✅ DONE | high | med | shipped: scan fetches each recommended artist's top tracks (Spotify/Deezer), resolving the artist id by name-search when the similar-artist row lacks one — guarded by a strict name-match so it never pulls the wrong artist. bounded (top 20 recs), per-call guarded, fail-soft to the pool. iTunes has no top-tracks API → pool-only there. needs a live scan to populate. |
| **5** | **genre-affinity boost** | med | low | we already compute your genre breakdown. boost recs whose genres match your top genres → tighter taste alignment. pure scoring add. |
| **6** | **adventurousness dial** | med | low | the ranker already supports `min_seed_count` (consensus floor). expose it as a "Safe ↔ Adventurous" slider on the row. |
| **7** | **diversity pass** | low-med | low | avoid 40 recs all orbiting your single heaviest seed — cap picks-per-seed so the row spans your taste. |
the core is built to absorb all of these without re-plumbing — `similarity_from_rank`, `build_recency_weighted_seeds`, and the scoring formula are all pure + tested.
---
## full discover-page generator audit (every soulsync-built row, excluding last.fm + listenbrainz)
how each one is generated today, and whether it can be elevated. "clear win" = safe + additive. "product call" = needs your decision (changes the row's character).
### curated (built during the scan, then hydrated)
- **Fresh Tape / Release Radar** — new releases from watchlist+similar artists. **FIXED tonight** (see above). one more *clear win* available: hydration silently drops any curated id no longer in the discovery pool — could fall back to the stored `track_data_json` blob so the row can't shrink at read time.
- **The Archives / Discovery Weekly** — strong already. nice 3-tier popularity split + serendipity scoring (boost never-played artists, penalize overplayed). same hydration-drop caveat as Fresh Tape; same cheap fallback fix.
- **Seasonal Mix** — cleanest of the bunch. hydrates from a dedicated `seasonal_tracks` table (carries its own data), so it doesn't suffer the pool-drop problem. no bug.
### discovery-pool generators (live queries)
- **Popular Picks** — ranks by popularity DESC. solid. only nit: on iTunes (no popularity scale) it silently degrades to random — indistinguishable from Shuffle there. UI-label thing at most.
- **Hidden Gems***clear win*. currently `ORDER BY RANDOM()` over low-popularity tracks — so it's "random obscure," not "*best* obscure." a light ranking (popularity just under the threshold, or genre-affinity to you) would make it feel curated instead of arbitrary. (a deeper *product call*: add personalization like Archives has — bigger lift, changes its "pure underground" character.)
- **Genre Playlists** — good. pushes the genre match into SQL. `RANDOM()` ordering is fine for a browse; a popularity/affinity tiebreak (*clear win*) would make thin genres feel less arbitrary.
- **Discovery Shuffle** — random by design, correct. only possible add: exclude tracks already shown in other rows this refresh (needs a cross-section seen-set — medium plumbing).
- **Time Machine (by decade)***clear win, low risk*: decades are hardcoded, so a modern-only library shows 7 decade tabs, 5 empty. filter the tabs to decades that actually have pool data.
- **Daily Mix** — the weakest row. the "50% your library" half permanently returns nothing (library tracks have no source ids to play), so each Daily Mix is really just a relabeled Genre Playlist. real fix = backfill source ids into library rows (*schema-level, higher risk*) — worth a dedicated pass, not a quick tweak. also silently falls back to "top artists as pseudo-genres" when genre data is missing → "Daily Mix 1" becomes mislabeled artist-radio. gate/label that (*clear win*).
### cross-cutting
- **hydration fragility** (Fresh Tape + Archives): both depend on curated ids still living in the pool at read time; misses are dropped silently. Seasonal already solved this with a dedicated table. giving the two spotify-style rows the same data-blob fallback is the single most robust cross-cutting fix. low risk, clear win.
- **RANDOM-ordering pattern** (Hidden Gems, Shuffle, Genre, Decade): intentional for variety, but leaves quality signal on the table for the non-shuffle rows. adding a light ranking pass to Hidden Gems + Genre is the biggest "best-in-class" lever after tonight's work.
want me to take any of these? the Hidden Gems ranking + Time Machine empty-decade filter + the Fresh Tape/Archives hydration fallback are all safe, additive, same-shape-as-tonight wins i can knock out next.

BIN
Issue/1.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 563 KiB

View file

@ -1,13 +0,0 @@
**SoulSync 2.7.9** is out 🎉 a big one.
🎚️ **Best-quality downloads** — downloads now follow a ranked quality profile you drag to order (FLAC 24/192 → mp3). best-quality mode grabs the highest-quality copy across *every* source; priority mode gets an opt-in rank-based order toggle. quarantine is folded into the Downloads page + safer imports (AcoustID fail-closed, silence/truncation guards).
🎧 **Discover got smart** — "Based On Your Listening" ranks artists from who you *actually play*, and "Your Listening Mix" is a playable track playlist of their top tracks (works on any source). Fresh Tape fills properly now.
**Wing It Pool** — a new spot next to Discovery Pool to review + re-match the tracks Wing It guessed at (they used to be invisible).
🔁 **Auto-Sync redesign** — the scheduling board is now clean horizontal lanes instead of a side-scrolling column wall.
🐛 **Fixes** — multi-disc albums no longer show disc-2 as "missing" (#927), playlists no longer stuck on "Never Synced" (#925), and tracks can't import while quarantined (#928). thanks @ramonskie + @nick2000713 🙏
⚠️ **re-scan your library once** so the multi-disc fix can backfill disc numbers on existing tracks. enjoy! 🎶

View file

@ -1,13 +0,0 @@
**SoulSync 2.8.0** is out 🎉 a quality + reliability release.
🧹 **The Unverified queue, finally under control** — if you saw thousands of "unverified" rows piling up, this is for you. the AcoustID scan stops duplicating history rows, a one-time reconcile on startup clears the existing backlog from your library (no re-scan), and a new 🧹 *Clean orphaned* button sweeps dead rows whose file is gone. (#934 — thanks @nick2000713 for #938)
✂️ **Preview Clip Cleanup** — a new Tools job that finds the ~30s preview clips the HiFi source sometimes hands back instead of the full song, then deletes them and re-wishlists the real version. each finding has a ▶ Play button so you can confirm before approving.
💿 **Album Completeness handles split albums** — an album split across multiple library rows no longer shows every fragment as falsely "incomplete"; it groups the validated fragments into one correct finding. (#936 — thanks @ragnarlotus)
🐛 **Fixes** — pasted YouTube cookies no longer throw `unsupported browser: "custom"` on Docker (thanks HellRa1SeR); longer remasters aren't quarantined as "truncated" anymore (#937, thanks @diegocade1); "Add to Wishlist" from a discography went from ~1530s *per track* to instant; wishlist art renders for re-downloads; and **Clear Completed** is back on the Downloads page.
**Performance** — trimmed the dashboard GPU usage that was hammering Firefox/Zen (and Background Particles are OFF by default now), plus bounded the runaway memory growth that could lock the app up on big libraries. (#935 / #802)
enjoy! 🎶

View file

@ -1,15 +0,0 @@
**SoulSync 2.8.1** is out 🎉 a feature + reliability release.
🎧 **Export playlists to Spotify & Deezer** — the mirrored-playlist export now has **Sync to Spotify** and **Sync to Deezer** next to the ListenBrainz / JSPF options. it builds a playlist in your account from the IDs soulsync already has (the discovery cache first, then your library), so an already-discovered playlist exports **instantly with zero API calls**. re-exporting updates the same playlist instead of duplicating it, and an optional *"match missing tracks"* toggle confidently searches for the stragglers — a wrong-artist or karaoke version is left out, never guessed. the first Spotify export asks permission once. (#945)
🏷️ **Library Reorganize — Rename only** — a lighter action that just **renames your files** to your naming scheme: no re-tag, no quality/AcoustID re-check, no copy-to-staging. much faster on a NAS, and only touches files whose path actually changes. pick it from the new Action dropdown. (#875 — thanks @tsoulard / @Tacobell444)
💿 **Broader lossless handling** — lossy-copy now covers **all lossless formats**, not just FLAC (#941); and **DSD** (`.dsf`/`.dff`) is recognized as lossless instead of false-flagged "truncated" (#939).
🐛 **Download + search fixes** — an unbalanced bracket no longer false-fails as "file not found"; a file we couldn't quarantine is left for retry instead of deleted; "file not found" errors are actionable now; pasted Qobuz/Tidal links inject the exact track into manual search (#932); the Wing It pool "Fix Match" works again.
**Reduce visual effects, refined** — it no longer freezes functional motion (spinners, progress), only the expensive GPU stuff (blur, shadows, glow). worker orbs default OFF on Firefox and run at ~30fps under reduce-effects. plus a jellyfin scan watchdog fix for big libraries.
🔧 **Under the hood** — settings cleanup (#943, @nick2000713), spotify oauth hardening (#942) + npm security fixes (#944, HellRa1SeR).
enjoy! 🎶

View file

@ -1,9 +0,0 @@
**SoulSync 2.8.2** is out 🎉 a stability + performance release.
🎧 **Spotify, reliably** — the Docker boot hang is fixed: with Spotify as your primary source, an unreachable Spotify API could block startup so the container bound `:8008` but never served the UI. auth probes are now deferred during boot + capped with a timeout. the "re-auth didn't stick" bug is fixed too (the OAuth callback and the app were reading different token caches), and **Sync to Spotify** now works — it asks for playlist-write permission once, on-demand, leaving your normal login untouched. (#949 — thanks HellRa1SeR)
**The "slow after update" fix** — the post-update lag wasn't SoulSync, it was browser password managers (Bitwarden/1Password/etc.) rebuilding their autofill overlay on *every* DOM change. non-credential fields are now marked so they skip them — **~110× less main-thread blocking** in the reporter's benchmark. plus a new **Max Performance** mode (Settings → Appearance) that kills every effect for no-GPU / Docker setups. (#948 — thanks @nick2000713)
📥 **Large-library imports no longer time out** — dropping a whole library into staging used to make the import page scan every file synchronously and never load. the scan runs in the background now with a live "Scanning N of M…" progress, and fills in when done. (#947 — thanks @ramonskie)
enjoy! 🎶

73
api/video/__init__.py Normal file
View file

@ -0,0 +1,73 @@
"""SoulSync — VIDEO side API package (isolated).
A SEPARATE Flask blueprint from the music API (api_v1). It reads only
database/video_library.db via VideoDatabase and imports nothing from the music
API or music database layer. Registered in web_server.py with a single additive
line at url_prefix '/api/video', so music routing is untouched.
"""
from __future__ import annotations
import threading
from flask import Blueprint
from utils.logging_config import get_logger
logger = get_logger("video_api")
# Lazily-created, process-wide VideoDatabase handle. VideoDatabase itself guards
# schema init once-per-path, so this just avoids re-opening the wrapper.
_video_db = None
_video_db_lock = threading.Lock()
def get_video_db():
"""Return the shared VideoDatabase instance (created on first use)."""
global _video_db
if _video_db is None:
with _video_db_lock:
if _video_db is None:
from database.video_database import VideoDatabase
_video_db = VideoDatabase()
return _video_db
def create_video_blueprint() -> Blueprint:
"""Build the isolated /api/video blueprint with all video sub-routes."""
bp = Blueprint("video_api", __name__)
from .dashboard import register_routes as reg_dashboard
from .scan import register_routes as reg_scan
from .library import register_routes as reg_library
from .libraries import register_routes as reg_libraries
from .poster import register_routes as reg_poster
from .enrichment import register_routes as reg_enrichment
from .detail import register_routes as reg_detail
from .search import register_routes as reg_search
from .discover import register_routes as reg_discover
from .calendar import register_routes as reg_calendar
from .watchlist import register_routes as reg_watchlist
from .wishlist import register_routes as reg_wishlist
from .youtube import register_routes as reg_youtube
from .downloads import register_routes as reg_downloads
from .manual_import import register_routes as reg_manual_import
from .automations import register_routes as reg_automations
reg_dashboard(bp)
reg_scan(bp)
reg_library(bp)
reg_libraries(bp)
reg_poster(bp)
reg_enrichment(bp)
reg_detail(bp)
reg_search(bp)
reg_discover(bp)
reg_calendar(bp)
reg_watchlist(bp)
reg_wishlist(bp)
reg_youtube(bp)
reg_downloads(bp)
reg_manual_import(bp)
reg_automations(bp)
return bp

28
api/video/automations.py Normal file
View file

@ -0,0 +1,28 @@
"""Video automation builder support (isolated /api/video routes).
The automation ENGINE is app-wide (shared with the music side), but the
builder block palette is scoped: the video builder must only ever offer
video + generic blocks, never music-only ones. This endpoint returns that
scoped slice via ``core.automation.blocks.blocks_for_scope('video')``.
ISOLATION: imports only the shared automation block definitions (which
themselves import nothing music-specific) no music API / music DB.
Reading/running/toggling video automations goes through the shared
``/api/automations`` endpoints (owned_by='video' rows); only the scoped
block palette lives here.
"""
from __future__ import annotations
from flask import jsonify
from utils.logging_config import get_logger
logger = get_logger("video_api.automations")
def register_routes(bp):
@bp.route("/automations/blocks", methods=["GET"])
def video_automation_blocks():
from core.automation.blocks import blocks_for_scope
return jsonify(blocks_for_scope("video"))

79
api/video/calendar.py Normal file
View file

@ -0,0 +1,79 @@
"""Video Calendar — upcoming TV episodes for OWNED shows.
GET /api/video/calendar?days=N episodes airing from today through today+N-1,
grouped client-side into the agenda view. Isolated: reads only video_library.db
via VideoDatabase, writes nothing, never touches the music side.
"""
from __future__ import annotations
from datetime import date, datetime, timedelta
from flask import jsonify, request
from utils.logging_config import get_logger
logger = get_logger("video.calendar")
def register_routes(bp):
@bp.route("/calendar", methods=["GET"])
def video_calendar():
from . import get_video_db
try:
days = request.args.get("days", default=7, type=int) or 7
days = max(1, min(days, 31)) # one week (or a few) per view
today = date.today()
# Optional ?start=YYYY-MM-DD for week navigation; default = today.
start = today
start_s = request.args.get("start")
if start_s:
try:
start = datetime.strptime(start_s, "%Y-%m-%d").date()
except ValueError:
start = today
if abs((start - today).days) > 400: # sane bound around today
start = today
end = start + timedelta(days=days - 1)
db = get_video_db()
# One-time backfill: existing shows matched TVDB before air time was
# captured, so re-queue them once (background, only those missing it).
try:
if (db.get_setting("airs_time_backfill") or "") != "1":
n = db.requeue_shows_for_airtime()
db.set_setting("airs_time_backfill", "1")
if n:
logger.info("calendar: queued %d shows for TVDB air-time backfill", n)
except Exception:
logger.exception("airs_time backfill queue failed")
from core.video.sources import resolve_video_server
# scope: 'watchlist' (default — shows you follow/track) or 'all' (every
# airing show in the library). The toggle on the page sends ?scope=.
scope = (request.args.get("scope") or "watchlist").lower()
eps = db.calendar_upcoming(start.isoformat(), end.isoformat(),
server_source=resolve_video_server(),
watchlist_only=(scope != "all"))
# Per-date counts drive the day-strip dots without a second query.
counts: dict[str, int] = {}
owned = 0
for e in eps:
counts[e["air_date"]] = counts.get(e["air_date"], 0) + 1
if e.get("has_file"):
owned += 1
return jsonify({
"today": today.isoformat(), # real today (for the highlight)
"start": start.isoformat(), # window start (may be a future week)
"end": end.isoformat(),
"days": days,
"scope": "all" if scope == "all" else "watchlist",
"counts_by_date": counts,
"total": len(eps),
"owned": owned,
"episodes": eps,
})
except Exception:
logger.exception("video calendar failed")
return jsonify({"error": "calendar failed"}), 500

28
api/video/dashboard.py Normal file
View file

@ -0,0 +1,28 @@
"""Video dashboard endpoint — live counts from video.db.
GET /api/video/dashboard -> {library:{...}, downloads:{...}, watchlist, wishlist}
With an empty database every value is a real 0.
"""
from __future__ import annotations
from flask import jsonify
from utils.logging_config import get_logger
logger = get_logger("video_api.dashboard")
def register_routes(bp):
@bp.route("/dashboard", methods=["GET"])
def video_dashboard():
from . import get_video_db
from core.video.sources import resolve_video_server
try:
server = resolve_video_server() # the VIDEO server, not music's active
stats = get_video_db().dashboard_stats(server_source=server)
stats["server"] = server
return jsonify(stats)
except Exception:
logger.exception("Failed to build video dashboard stats")
return jsonify({"error": "Failed to load video dashboard stats"}), 500

135
api/video/detail.py Normal file
View file

@ -0,0 +1,135 @@
"""Video detail payloads (drill-in pages).
GET /api/video/detail/show/<id> show + seasonsepisodes tree (owned roll-ups)
GET /api/video/detail/movie/<id> movie + owned/file info
Reads only video.db; isolated from the music API.
"""
from __future__ import annotations
from flask import jsonify, request
from utils.logging_config import get_logger
logger = get_logger("video_api.detail")
def register_routes(bp):
@bp.route("/monitor", methods=["POST"])
def video_set_monitor():
from . import get_video_db
body = request.get_json(silent=True) or {}
kind, item_id = body.get("kind"), body.get("id")
if kind not in ("movie", "show") or not isinstance(item_id, int):
return jsonify({"error": "bad request"}), 400
ok = get_video_db().set_monitored(kind, item_id, bool(body.get("monitored")))
if not ok:
return jsonify({"error": "not found"}), 404
return jsonify({"success": True, "monitored": bool(body.get("monitored"))})
@bp.route("/detail/show/<int:show_id>", methods=["GET"])
def video_show_detail(show_id):
from . import get_video_db
data = get_video_db().show_detail(show_id)
if not data:
return jsonify({"error": "not found"}), 404
return jsonify(data)
@bp.route("/detail/movie/<int:movie_id>", methods=["GET"])
def video_movie_detail(movie_id):
from . import get_video_db
data = get_video_db().movie_detail(movie_id)
if not data:
return jsonify({"error": "not found"}), 404
return jsonify(data)
@bp.route("/detail/show/<int:show_id>/refresh-art", methods=["POST"])
def video_show_refresh_art(show_id):
"""Lazy on-view backfill: pull missing season posters / episode art from
TMDB and cache them. Best-effort never errors the page."""
try:
from core.video.enrichment.engine import get_video_enrichment_engine
res = get_video_enrichment_engine().refresh_show_art(show_id)
except Exception:
logger.exception("refresh-art failed for show %s", show_id)
res = {"ok": False, "reason": "error"}
return jsonify(res)
@bp.route("/detail/movie/<int:movie_id>/refresh-art", methods=["POST"])
def video_movie_refresh_art(movie_id):
"""Lazy on-view backfill for a movie (cast / genres / backdrop / ratings)."""
try:
from core.video.enrichment.engine import get_video_enrichment_engine
res = get_video_enrichment_engine().refresh_movie_art(movie_id)
except Exception:
logger.exception("refresh-art failed for movie %s", movie_id)
res = {"ok": False, "reason": "error"}
return jsonify(res)
@bp.route("/tmdb/<kind>/<int:tmdb_id>", methods=["GET"])
def video_tmdb_detail(kind, tmdb_id):
"""Full detail for a TMDB title not in the library (the search → detail
view). May return {redirect:{source,kind,id}} if it's actually owned."""
if kind not in ("movie", "show"):
return jsonify({"error": "bad kind"}), 400
try:
from core.video.enrichment.engine import get_video_enrichment_engine
d = get_video_enrichment_engine().tmdb_detail(kind, tmdb_id)
except Exception:
logger.exception("tmdb detail failed for %s %s", kind, tmdb_id)
d = None
if not d:
return jsonify({"error": "not found"}), 404
return jsonify(d)
@bp.route("/tmdb/show/<int:tv_id>/season/<int:season_number>", methods=["GET"])
def video_tmdb_season(tv_id, season_number):
"""Lazy per-season episodes for a TMDB (un-owned) show detail."""
try:
from core.video.enrichment.engine import get_video_enrichment_engine
d = get_video_enrichment_engine().tmdb_season(tv_id, season_number)
except Exception:
logger.exception("tmdb season failed for %s S%s", tv_id, season_number)
d = None
if not d:
return jsonify({"error": "not found"}), 404
return jsonify(d)
@bp.route("/episode/<int:tmdb_id>/<int:season>/<int:episode>", methods=["GET"])
def video_episode_extra(tmdb_id, season, episode):
"""Episode expand: guest stars + bigger still (by the SHOW's tmdb id)."""
try:
from core.video.enrichment.engine import get_video_enrichment_engine
d = get_video_enrichment_engine().episode_extra(tmdb_id, season, episode)
except Exception:
logger.exception("episode extra failed for %s S%sE%s", tmdb_id, season, episode)
d = None
if not d:
return jsonify({"error": "not found"}), 404
return jsonify(d)
@bp.route("/person/<int:tmdb_id>", methods=["GET"])
def video_person_detail(tmdb_id):
"""In-app person page: bio + filmography (each credit annotated owned/not)."""
try:
from core.video.enrichment.engine import get_video_enrichment_engine
d = get_video_enrichment_engine().person_detail(tmdb_id)
except Exception:
logger.exception("person detail failed for %s", tmdb_id)
d = None
if not d:
return jsonify({"error": "not found"}), 404
return jsonify(d)
@bp.route("/detail/<kind>/<int:item_id>/extras", methods=["GET"])
def video_detail_extras(kind, item_id):
"""Live TMDB extras (trailer / where-to-watch / similar) for the detail page."""
if kind not in ("movie", "show"):
return jsonify({}), 400
try:
from core.video.enrichment.engine import get_video_enrichment_engine
return jsonify(get_video_enrichment_engine().item_extras(kind, item_id))
except Exception:
logger.exception("extras failed for %s %s", kind, item_id)
return jsonify({})

386
api/video/discover.py Normal file
View file

@ -0,0 +1,386 @@
"""Video discover API — browse TMDB (movies/TV the user doesn't own yet).
GET /api/video/discover/hero trending titles w/ backdrops (slideshow)
GET /api/video/discover/genres {movie:[{id,name}], show:[{id,name}]}
GET /api/video/discover/list?... one shelf/grid of items, e.g.
?key=trending trending movies + shows
?key=<curated>&page= a canned list (popular_movies, top_shows)
?kind=movie|show&genre=&year=&decade=&sort=&page= a filtered browse
Items are annotated with ``library_id`` when already owned (so the card links to
the owned detail, not the TMDB preview). Reads only the enrichment engine +
video.db; isolated from the music API.
"""
from __future__ import annotations
import concurrent.futures
from flask import jsonify, request
from utils.logging_config import get_logger
logger = get_logger("video_api.discover")
def register_routes(bp):
@bp.route("/discover/hero", methods=["GET"])
def video_discover_hero():
"""A few trending titles that have a backdrop — drives the hero slideshow."""
from core.video.enrichment.engine import get_video_enrichment_engine
try:
items = [x for x in (get_video_enrichment_engine().trending() or [])
if x.get("backdrop")][:6]
except Exception:
logger.exception("discover hero failed")
items = []
return jsonify({"items": items})
@bp.route("/discover/taste", methods=["GET"])
def video_discover_taste():
"""The user's most-owned genres (movies + shows) → personalized rails."""
from . import get_video_db
try:
from core.video.sources import resolve_video_server
srv = resolve_video_server()
except Exception:
srv = None
db = get_video_db()
try:
return jsonify({"movie": db.top_owned_genres("movie", srv, 6),
"show": db.top_owned_genres("show", srv, 6)})
except Exception:
logger.exception("discover taste failed")
return jsonify({"movie": [], "show": []})
@bp.route("/discover/morelike", methods=["GET"])
def video_discover_morelike():
"""'More like <owned title>' rails — TMDB recommendations seeded from a few
random titles you already own (interleaved movie/show, max 3 rails)."""
from . import get_video_db
from core.video.enrichment.engine import get_video_enrichment_engine
try:
from core.video.sources import resolve_video_server
srv = resolve_video_server()
except Exception:
srv = None
db = get_video_db()
eng = get_video_enrichment_engine()
try:
seeds = db.random_owned_titles(2, srv)
movies = [s for s in seeds if s["kind"] == "movie"]
shows = [s for s in seeds if s["kind"] == "show"]
ordered = []
while (movies or shows) and len(ordered) < 3:
if movies:
ordered.append(movies.pop(0))
if shows and len(ordered) < 3:
ordered.append(shows.pop(0))
rails = []
for s in ordered:
items = [it for it in eng.recommendations(s["kind"], s["tmdb_id"])
if it.get("tmdb_id") != s["tmdb_id"]]
if len(items) >= 4:
rails.append({"title": "More like " + s["title"], "items": items[:30]})
return jsonify({"rails": rails})
except Exception:
logger.exception("discover morelike failed")
return jsonify({"rails": []})
@bp.route("/discover/foryou", methods=["GET"])
def video_discover_foryou():
"""A single 'Recommended for you' wall blended from many owned titles — a title
recommended by more of your library ranks higher (consensus)."""
from . import get_video_db
from core.video.enrichment.engine import get_video_enrichment_engine
from core.video.discovery_recs import blend_recommendations
try:
from core.video.sources import resolve_video_server
srv = resolve_video_server()
except Exception:
srv = None
db = get_video_db()
eng = get_video_enrichment_engine()
try:
seeds = db.random_owned_titles(6, srv) # up to 6 movies + 6 shows
seed_ids = [s["tmdb_id"] for s in seeds if s.get("tmdb_id")]
rec_lists = [eng.recommendations(s["kind"], s["tmdb_id"])
for s in seeds if s.get("tmdb_id")]
items = blend_recommendations(rec_lists, exclude_ids=seed_ids, limit=40)
return jsonify({"items": items})
except Exception:
logger.exception("discover foryou failed")
return jsonify({"items": []})
@bp.route("/discover/gaps", methods=["GET"])
def video_discover_gaps():
"""'What am I missing?' rails — franchises you've started but not finished, and
more from the directors/creators you own the most. Powered by the gap engine."""
from . import get_video_db
from core.video.enrichment.engine import get_video_enrichment_engine
from core.video.discovery_gaps import collection_gaps, filmography_gaps
try:
from core.video.sources import resolve_video_server
srv = resolve_video_server()
except Exception:
srv = None
db = get_video_db()
eng = get_video_enrichment_engine()
try:
# Lazy collection-id backfill: movies matched before the collection column
# exists have no franchise id. Fill a small batch each load (self-healing);
# isolated so a backfill hiccup never breaks the gap rails.
try:
for mv in db.movies_missing_collection(srv, limit=20):
coll = eng.movie_collection(mv["tmdb_id"])
if coll is not None:
db.set_movie_collection(mv["id"], coll.get("id"), coll.get("name"))
except Exception:
logger.exception("collection-id backfill batch failed")
owned = db.owned_movie_tmdb_ids(srv)
ignored = db.ignored_keys()
rails = []
# Complete your collections — top franchises you've started, missing entries.
for coll in db.owned_movie_collections(srv, limit=8):
missing = collection_gaps(owned, eng.collection(coll["collection_id"]))
if missing:
name = (coll.get("name") or "Collection").strip()
rails.append({"title": "Complete the " + name, "kind": "collection",
"items": missing[:30]})
# More from the people you own the most (directors / creators).
for person in db.top_owned_people(min_titles=2, limit=6, server_source=srv):
p = eng.person_detail(person["tmdb_id"])
if not p:
continue
missing = filmography_gaps(owned, p.get("credits") or [],
kinds=("movie",), min_vote_count=50, limit=30)
if ignored:
missing = [m for m in missing
if f"{m.get('kind')}:{m.get('tmdb_id')}" not in ignored]
if len(missing) >= 3:
rails.append({"title": "More from " + person["name"], "kind": "person",
"items": missing})
return jsonify({"rails": rails})
except Exception:
logger.exception("discover gaps failed")
return jsonify({"rails": []})
@bp.route("/discover/trailer", methods=["GET"])
def video_discover_trailer():
"""Best YouTube trailer {key,name} for a tmdb title (hero 'Trailer' button)."""
from core.video.enrichment.engine import get_video_enrichment_engine
kind = request.args.get("kind", "movie")
try:
tmdb_id = int(request.args.get("tmdb_id"))
except (TypeError, ValueError):
return jsonify({"trailer": None})
try:
tr = get_video_enrichment_engine().trailer(kind, tmdb_id)
except Exception:
logger.exception("discover trailer failed")
tr = None
return jsonify({"trailer": tr or None})
@bp.route("/discover/ignore", methods=["GET", "POST"])
def video_discover_ignore():
"""The Discover 'Not interested' list. GET -> {items}. POST
{action:'add'|'remove', kind, tmdb_id, title?, year?, poster?}."""
from . import get_video_db
db = get_video_db()
try:
if request.method == "GET":
return jsonify({"items": db.list_ignored()})
body = request.get_json(silent=True) or {}
kind, tmdb_id = body.get("kind"), body.get("tmdb_id")
if body.get("action") == "remove":
db.remove_ignored(kind, tmdb_id)
return jsonify({"success": True})
ok = db.add_ignored(kind, tmdb_id, body.get("title"), body.get("year"), body.get("poster"))
return jsonify({"success": ok})
except Exception:
logger.exception("discover ignore failed")
return jsonify({"success": False, "items": []})
@bp.route("/discover/languages", methods=["GET", "POST"])
def video_discover_languages():
"""Get/set the preferred original-languages for general rails (ISO-639-1 codes).
POST {languages: ['en','ko']} (or 'en,ko'); GET returns the current list."""
from . import get_video_db
db = get_video_db()
try:
if request.method == "POST":
body = request.get_json(silent=True) or {}
langs = body.get("languages")
if isinstance(langs, list):
val = ",".join(str(c).strip().lower() for c in langs if str(c).strip())
else:
val = ",".join(c.strip().lower() for c in str(langs or "").split(",") if c.strip())
db.set_setting("discover_languages", val or "en")
return jsonify({"success": True,
"languages": [c for c in (val or "en").split(",") if c]})
raw = db.get_setting("discover_languages", "en") or "en"
return jsonify({"languages": [c.strip() for c in raw.split(",") if c.strip()]})
except Exception:
logger.exception("discover languages get/set failed")
return jsonify({"languages": ["en"]})
@bp.route("/discover/providers-pref", methods=["GET", "POST"])
def video_discover_providers_pref():
"""The user's subscribed streaming services (TMDB provider ids) — drives the
'On your streaming services' rail. POST {providers:[8,9]} (or '8,9'); GET returns them."""
from . import get_video_db
db = get_video_db()
try:
if request.method == "POST":
body = request.get_json(silent=True) or {}
p = body.get("providers")
if isinstance(p, list):
val = ",".join(str(c).strip() for c in p if str(c).strip())
else:
val = ",".join(c.strip() for c in str(p or "").split(",") if c.strip())
db.set_setting("discover_providers", val)
return jsonify({"success": True, "providers": [c for c in val.split(",") if c]})
raw = db.get_setting("discover_providers", "") or ""
return jsonify({"providers": [c.strip() for c in raw.split(",") if c.strip()]})
except Exception:
logger.exception("discover providers-pref failed")
return jsonify({"providers": []})
@bp.route("/discover/genres", methods=["GET"])
def video_discover_genres():
"""Genre id→name maps for both kinds (powers the genre rails + filter)."""
from core.video.enrichment.engine import get_video_enrichment_engine
eng = get_video_enrichment_engine()
try:
return jsonify({"movie": eng.genre_list("movie"), "show": eng.genre_list("show")})
except Exception:
logger.exception("discover genres failed")
return jsonify({"movie": [], "show": []})
@bp.route("/discover/list", methods=["GET"])
def video_discover_list():
"""One shelf (rail) or one page of a filtered browse — see module docstring.
``pages`` (13, default 1) fetches that many consecutive TMDB pages and
concatenates them (deduped) in one response so a rail can show ~40 items
and still look full after 'Hide owned' drops the ones you have."""
from core.video.enrichment.engine import get_video_enrichment_engine
eng = get_video_enrichment_engine()
try:
page = max(1, int(request.args.get("page", 1) or 1))
except (TypeError, ValueError):
page = 1
try:
pages = min(3, max(1, int(request.args.get("pages", 1) or 1)))
except (TypeError, ValueError):
pages = 1
key = (request.args.get("key") or "").strip()
kind = request.args.get("kind", "movie")
genre = request.args.get("genre") or None
year = request.args.get("year") or None
decade = request.args.get("decade") or None
providers = request.args.get("providers") or None
if providers and "," in providers:
providers = providers.replace(",", "|") # TMDB with_watch_providers OR-join
sort = request.args.get("sort") or "popularity.desc"
# Netflix-class discover extensions (all optional). Comma -> pipe = TMDB OR-join.
keywords = (request.args.get("keywords") or "").replace(",", "|") or None
companies = (request.args.get("companies") or "").replace(",", "|") or None
networks = (request.args.get("networks") or "").replace(",", "|") or None
cast = request.args.get("cast") or None
crew = request.args.get("crew") or None
min_runtime = request.args.get("min_runtime") or None
max_runtime = request.args.get("max_runtime") or None
certification = request.args.get("certification") or None
release_window = request.args.get("release_window") or None
try:
vote_count_min = int(request.args["vote_count_min"]) if request.args.get("vote_count_min") else None
except (TypeError, ValueError):
vote_count_min = None
lang = (request.args.get("lang") or "").strip().lower() or None # explicit (foreign rail / browse)
# The Browse-all grid sends `lang=any` to opt OUT of the rail language preference
# entirely (ad-hoc search shows every language); a real code (en/ko/…) filters to it.
any_lang = lang in ("any", "all")
if any_lang:
lang = None
hide_owned = (request.args.get("hide_owned") or "") in ("1", "true", "yes")
# Preferred original-languages (multi) for GENERAL/curated rails — so the feeds
# aren't flooded with foreign titles (e.g. Bollywood in Popular/Trending). A rail
# with an explicit `lang` (a dedicated foreign rail) or `lang=any` (browse) bypasses
# this. Default 'en'.
prefer_langs = None
if not lang and not any_lang:
try:
from . import get_video_db
raw = get_video_db().get_setting("discover_languages", "en") or "en"
prefer_langs = {c.strip().lower() for c in raw.split(",") if c.strip()} or None
except Exception:
prefer_langs = {"en"}
def fetch(p):
if key == "trending":
return eng.trending()
if key == "trending_today":
return eng.trending(window="day") # the ranked 'Top 10 today' chart (mixed)
if key == "trending_movies_today":
return eng.trending(window="day", kind="movie") # split 'Top 10 Movies Today'
if key == "trending_tv_today":
return eng.trending(window="day", kind="show") # split 'Top 10 TV Shows Today'
if key:
return eng.discover_curated(key, page=p)
return eng.discover_filter(
kind, genre=genre, year=year, decade=decade, providers=providers,
sort_by=sort, page=p, language=lang, keywords=keywords, companies=companies,
networks=networks, cast=cast, crew=crew, min_runtime=min_runtime,
max_runtime=max_runtime, certification=certification,
vote_count_min=vote_count_min, release_window=release_window)
try:
items, seen = [], set()
def consume(batch):
"""Filter a page into `items` (dedup + hide-owned + language). Returns False
once TMDB hands back an empty page (it ran out stop paging)."""
if not batch:
return False
for it in batch:
dk = (it.get("kind"), it.get("tmdb_id"))
if dk in seen:
continue
seen.add(dk)
if hide_owned and it.get("library_id") is not None:
continue
if prefer_langs:
ol = (it.get("original_language") or "").lower()
if ol and ol not in prefer_langs:
continue # known foreign language not in your preference
items.append(it)
return True
# When filtering (hide-owned or language), page DEEPER and drop items server-side
# so a heavy library's rail still fills to ~target instead of showing 4 cards. We
# fetch the extra pages in concurrent WAVES (TTLCache + the TMDB client are
# thread-safe) so digging 20 pages deep costs ~4 page-latencies, not 20.
need_fill = hide_owned or bool(prefer_langs)
target = 20 if need_fill else 0
if key in ("trending", "trending_today", "trending_movies_today", "trending_tv_today"):
consume(fetch(page)) # a single fixed chart; never paged
elif not need_fill:
for offset in range(pages): # no filtering: respect requested page count
if not consume(fetch(page + offset)):
break
else:
MAX_PAGES, WAVE = 20, 5
offset, ran_out = 0, False
with concurrent.futures.ThreadPoolExecutor(max_workers=WAVE) as ex:
while offset < MAX_PAGES and len(items) < target and not ran_out:
batches = list(ex.map(fetch, [page + offset + i for i in range(WAVE)]))
for batch in batches:
if not consume(batch):
ran_out = True # an empty page = TMDB has no more
offset += WAVE
return jsonify({"items": items, "page": page})
except Exception:
logger.exception("discover list failed (key=%s)", key)
return jsonify({"items": [], "page": page})

514
api/video/downloads.py Normal file
View file

@ -0,0 +1,514 @@
"""Video-side download SETTINGS (isolated).
Persists the video download configuration in video.db's ``video_settings`` KV
table fully separate from the music ``soulseek.*`` paths so the two libraries
never share a folder or collide. The actual download fulfillment engine (wishlist
search grab) is a later roadmap phase; these endpoints just store/serve the
config the Settings Downloads tab edits.
Folders:
- INPUT (download) folder is SHARED with the music side it's the same
``config_manager`` key the music Download Settings use (``soulseek.download_path``),
so changing it on either side changes both (one physical download dir, simpler
Docker mounts). We only READ/WRITE that shared key; no music code is touched.
- OUTPUT (library) folders are video-specific and live in video.db, one per type:
``movies_path`` / ``tv_path`` / ``youtube_path``.
The engine routes a finished download to the library path matching its type. (Legacy
single video ``transfer_path`` is migrated into ``movies_path`` on first read.)
Connection settings that are genuinely SHARED with music (the slskd instance, the
torrent/usenet clients, Prowlarr indexers) are NOT stored here those live in the
music config_manager and are surfaced on the shared Indexers tab + shared slskd
block (a deliberate shared boundary, since they're one physical resource).
"""
from __future__ import annotations
from flask import jsonify, request
from utils.logging_config import get_logger
logger = get_logger("video_api.downloads")
# Video-specific OUTPUT library folders (video.db). The INPUT folder is the shared
# music key below — not in this list.
_PATH_KEYS = ("movies_path", "tv_path", "youtube_path")
# The shared input/download dir — the SAME config key the music side reads/writes.
_SHARED_DOWNLOAD_KEY = "soulseek.download_path"
# slskd CONNECTION settings genuinely SHARED with music — one slskd instance serves
# both sides, so these live in the app-wide config_manager (soulseek.*), NOT video.db.
# Deliberately excludes the music download/transfer PATHS and source mode/quality —
# those are video-specific (stored in video.db). Maps the video field name -> the
# shared config key + default. (config_manager is shared app config, not music code.)
_SLSKD_KEYS = {
"slskd_url": ("soulseek.slskd_url", "http://localhost:5030"),
"api_key": ("soulseek.api_key", ""),
"search_timeout": ("soulseek.search_timeout", 60),
"search_timeout_buffer": ("soulseek.search_timeout_buffer", 15),
"search_min_delay_seconds": ("soulseek.search_min_delay_seconds", 0),
"min_peer_upload_speed": ("soulseek.min_peer_upload_speed", 0),
"max_peer_queue": ("soulseek.max_peer_queue", 0),
"download_timeout": ("soulseek.download_timeout", 600), # seconds (UI shows minutes)
"auto_clear_searches": ("soulseek.auto_clear_searches", True),
}
def _evaluate_hits(raw, profile, scope, want_season, want_episode) -> list:
"""Parse → evaluate → rank a list of raw indexer hits against the quality profile.
Shared by the mock search and the live slskd start/poll endpoints."""
from core.video.quality_eval import evaluate_release
from core.video.release_parse import parse_release
results = []
for hit in raw:
parsed = parse_release(hit.get("title"))
size_gb = round((hit.get("size_bytes") or 0) / (1024 ** 3), 1)
verdict = evaluate_release(parsed, profile, scope=scope, want_season=want_season,
want_episode=want_episode, size_gb=size_gb)
# Availability = how downloadable the source is (slskd: free slot/queue/speed score
# from group_video_files; torrents/mock: seeders/peers). Ranks within a quality tier
# so we grab a free-slot/empty-queue release over one stuck behind a huge queue.
avail = hit.get("availability")
if avail is None:
avail = hit.get("seeders") if hit.get("seeders") is not None else (hit.get("peers") or 0)
results.append({
"title": hit.get("title"), "size_gb": size_gb, "size_bytes": hit.get("size_bytes") or 0,
"seeders": hit.get("seeders"), "peers": hit.get("peers"),
"username": hit.get("username"), "slots": hit.get("slots"),
"queue": hit.get("queue"), "speed": hit.get("speed"),
"filename": hit.get("filename"), "_avail": avail,
"quality_label": verdict["quality_label"], "accepted": verdict["accepted"],
"rejected": verdict["rejected"], "score": verdict["score"],
"resolution": parsed.get("resolution"), "source": parsed.get("source"),
"codec": parsed.get("codec"), "hdr": parsed.get("hdr"),
"audio": parsed.get("audio"), "group": parsed.get("group"),
"repack": parsed.get("repack") or parsed.get("proper"),
})
# accepted first, then quality-profile score, then availability, then bigger file.
results.sort(key=lambda r: (r["accepted"], r["score"], r["_avail"], r["size_bytes"]), reverse=True)
for r in results:
r.pop("_avail", None)
return results[:40]
def _search_ints(body):
def _int(v):
try:
return int(v)
except (TypeError, ValueError):
return None
return _int(body.get("season")), _int(body.get("episode")), _int(body.get("season_end"))
def register_routes(bp):
@bp.route("/downloads/config", methods=["GET"])
def video_downloads_config():
from . import get_video_db
from core.video.download_config import load as load_source
from config.settings import config_manager
db = get_video_db()
out = {k: db.get_setting(k) or "" for k in _PATH_KEYS}
if not out["movies_path"]: # migrate the legacy single transfer folder → Movies
out["movies_path"] = db.get_setting("transfer_path") or ""
out["download_path"] = config_manager.get(_SHARED_DOWNLOAD_KEY, "") or "" # shared w/ music
out.update(load_source(db)) # download_mode + hybrid_order
return jsonify(out)
@bp.route("/downloads/config", methods=["POST"])
def video_downloads_config_save():
from . import get_video_db
from core.video.download_config import save as save_source
db = get_video_db()
body = request.get_json(silent=True) or {}
for key in _PATH_KEYS:
if key in body:
db.set_setting(key, (str(body.get(key) or "")).strip())
if "download_path" in body: # SHARED with music — write the same config key
from config.settings import config_manager
config_manager.set(_SHARED_DOWNLOAD_KEY, (str(body.get("download_path") or "")).strip())
save_source(db, body) # download_mode + hybrid_order (validated)
return jsonify({"status": "saved"})
@bp.route("/downloads/history", methods=["GET"])
def video_downloads_history():
"""Paged permanent history of grabs (movies + episodes). ?kind=movie|show,
?search=, ?outcome=, ?page=, ?limit=. Always returns counts for the tabs."""
from . import get_video_db
try:
db = get_video_db()
kind = request.args.get("kind")
res = db.query_download_history(
kind=kind if kind in ("movie", "show") else None,
search=request.args.get("search", ""),
outcome=request.args.get("outcome") or None,
page=request.args.get("page", 1), limit=request.args.get("limit", 40))
return jsonify({"success": True, "counts": db.download_history_counts(), **res})
except Exception:
logger.exception("Failed to list video download history")
return jsonify({"success": False, "error": "Failed to load history"}), 500
@bp.route("/downloads/history/<int:history_id>", methods=["GET"])
def video_downloads_history_detail(history_id):
from . import get_video_db
d = get_video_db().download_history_detail(history_id)
if not d:
return jsonify({"success": False, "error": "not found"}), 404
return jsonify({"success": True, "item": d})
@bp.route("/downloads/history/<int:history_id>", methods=["DELETE"])
def video_downloads_history_delete(history_id):
"""Forget one grab — the 'Re-download' action. Removing its history row lets the
scans re-add + re-grab it (useful after you've deleted the file)."""
from . import get_video_db
return jsonify({"success": get_video_db().delete_download_history(history_id)})
@bp.route("/downloads/history/clear", methods=["POST"])
def video_downloads_history_clear():
"""Clear the permanent history (all, or one kind via {kind})."""
from . import get_video_db
body = request.get_json(silent=True) or {}
n = get_video_db().clear_download_history(kind=body.get("kind"))
return jsonify({"success": True, "removed": n})
@bp.route("/downloads/meta/<kind>/<int:tmdb_id>", methods=["GET"])
def video_download_meta(kind, tmdb_id):
"""Lazy TMDB detail for a download's expand drawer (logo, cast w/ photos, trailer,
where-to-watch, rating/runtime/genres), keyed by the grabbed title's TMDB id.
Best-effort the drawer still shows the download facts without it."""
if kind not in ("movie", "show"):
return jsonify({}), 400
try:
from core.video.enrichment.engine import get_video_enrichment_engine
d = get_video_enrichment_engine().tmdb_full_detail(kind, tmdb_id) or {}
if not d:
return jsonify({})
extras = d.get("_extras") or {}
tr = extras.get("trailer") or {}
director = next((c.get("name") for c in (d.get("crew") or [])
if (c.get("job") or "").lower() in ("director", "creator")), None)
# episode-specific detail (still + that episode's own title/overview/air date) when a
# specific episode is downloading — more relevant than the show synopsis.
episode = None
sn, en = request.args.get("season"), request.args.get("episode")
if kind == "show" and sn and en:
try:
season = get_video_enrichment_engine().tmdb_season(tmdb_id, int(sn)) or {}
ep = next((e for e in (season.get("episodes") or [])
if str(e.get("episode_number")) == str(int(en))), None)
if ep:
episode = {"season": int(sn), "episode": int(en), "title": ep.get("title"),
"overview": ep.get("overview"), "air_date": ep.get("air_date"),
"still_url": ep.get("still_url")}
except (ValueError, TypeError):
pass
return jsonify({
"title": d.get("title"), "overview": d.get("overview"), "tagline": d.get("tagline"),
"backdrop_url": d.get("backdrop_url"), "logo": d.get("logo"),
"genres": d.get("genres") or [], "rating": d.get("rating"),
"runtime_minutes": d.get("runtime_minutes"), "year": d.get("year"),
"network": d.get("network"), "studio": d.get("studio"),
"status": d.get("status"), "director": director, "episode": episode,
"cast": [{"name": c.get("name"), "character": c.get("character"), "photo": c.get("photo")}
for c in (d.get("cast") or [])[:10]],
"trailer_url": ("https://www.youtube.com/watch?v=" + tr["key"]) if tr.get("key") else None,
"providers": (extras.get("providers") or [])[:6],
"providers_link": extras.get("providers_link"),
})
except Exception:
logger.exception("download meta failed for %s %s", kind, tmdb_id)
return jsonify({})
@bp.route("/downloads/yt-meta/<video_id>", methods=["GET"])
def video_download_yt_meta(video_id):
"""Cached extra detail for a YouTube download's drawer (duration / views / thumbnail)."""
from . import get_video_db
try:
return jsonify(get_video_db().youtube_video_detail(video_id) or {})
except Exception:
logger.exception("yt meta failed for %s", video_id)
return jsonify({})
@bp.route("/downloads/quality", methods=["GET"])
def video_quality_profile():
from . import get_video_db
from core.video.quality_profile import load
return jsonify(load(get_video_db()))
@bp.route("/downloads/quality", methods=["POST"])
def video_quality_profile_save():
from . import get_video_db
from core.video.quality_profile import save
body = request.get_json(silent=True) or {}
return jsonify(save(get_video_db(), body))
@bp.route("/organization", methods=["GET"])
def video_organization():
"""The library-organisation settings: naming templates + post-process toggles."""
from . import get_video_db
from core.video.organization import load
return jsonify(load(get_video_db()))
@bp.route("/organization", methods=["POST"])
def video_organization_save():
from . import get_video_db
from core.video.organization import save
body = request.get_json(silent=True) or {}
return jsonify(save(get_video_db(), body))
@bp.route("/downloads/evaluate", methods=["POST"])
def video_quality_evaluate():
"""Judge a video file the user already owns against their quality profile —
powers the Download modal's 'In your library · (below your target)' line.
Body: {"file": {resolution, video_codec, }}."""
from . import get_video_db
from core.video.quality_eval import evaluate_owned
from core.video.quality_profile import load
body = request.get_json(silent=True) or {}
profile = load(get_video_db())
return jsonify(evaluate_owned(body.get("file"), profile))
@bp.route("/downloads/search", methods=["POST"])
def video_downloads_search():
"""Search a scope (movie / episode / season / series) and return candidates
ranked + filtered against the stored quality profile. The indexer is mocked
for now (core.video.mock_search) the parseevaluaterank pipeline is real,
so swapping in slskd/Prowlarr later needs no change here.
Body: {scope, title, year?, season?, episode?, season_end?}."""
from . import get_video_db
from core.video.mock_search import mock_search
from core.video.quality_profile import load as load_profile
body = request.get_json(silent=True) or {}
scope = str(body.get("scope") or "movie").lower()
title = body.get("title") or ""
source = str(body.get("source") or "").lower()
want_season, want_episode, season_end = _search_ints(body)
profile = load_profile(get_video_db())
live = False
if source == "soulseek":
from core.video.slskd_search import build_query, slskd_search
sres = slskd_search(build_query(scope, title, year=body.get("year"),
season=want_season, episode=want_episode))
if not sres.get("configured"):
return jsonify({"scope": scope, "results": [], "error": "slskd isn't configured — set its URL on Settings → Downloads."})
if sres.get("error"):
return jsonify({"scope": scope, "results": [], "error": "slskd: " + str(sres["error"])})
raw, live = sres["hits"], True
else:
raw = mock_search(scope, title, year=body.get("year"), season=want_season,
episode=want_episode, season_end=season_end, source=source)
return jsonify({"scope": scope, "live": live,
"results": _evaluate_hits(raw, profile, scope, want_season, want_episode)})
@bp.route("/downloads/search/start", methods=["POST"])
def video_downloads_search_start():
"""Begin a search. For mock sources the results come back immediately; for
Soulseek it returns a slskd search id to poll (results trickle in over ~30s,
like the music side) fixes 'no results' from waiting too briefly."""
from . import get_video_db
from core.video.mock_search import mock_search
from core.video.quality_profile import load as load_profile
body = request.get_json(silent=True) or {}
scope = str(body.get("scope") or "movie").lower()
title = body.get("title") or ""
source = str(body.get("source") or "").lower()
want_season, want_episode, season_end = _search_ints(body)
if source == "soulseek":
from core.video.slskd_search import build_query, search_timeout_ms, start_search
res = start_search(build_query(scope, title, year=body.get("year"),
season=want_season, episode=want_episode))
if not res.get("configured"):
return jsonify({"error": "slskd isn't configured — set its URL on Settings → Downloads."})
if res.get("error"):
return jsonify({"error": "slskd: " + str(res["error"])})
# how long the client should keep polling (slskd keeps searching this long).
return jsonify({"id": res["id"], "live": True, "complete": False,
"poll_ms": search_timeout_ms() + 8000})
# mock sources resolve in one shot
profile = load_profile(get_video_db())
raw = mock_search(scope, title, year=body.get("year"), season=want_season,
episode=want_episode, season_end=season_end, source=source)
return jsonify({"id": None, "live": False, "complete": True,
"results": _evaluate_hits(raw, profile, scope, want_season, want_episode)})
@bp.route("/downloads/search/poll", methods=["GET"])
def video_downloads_search_poll():
"""Current ranked results for an in-flight slskd search. Query: id, scope,
title, season?, episode?. The client polls until it stops growing or times out."""
from . import get_video_db
from core.video.quality_profile import load as load_profile
from core.video.slskd_search import poll_search
sid = request.args.get("id")
scope = str(request.args.get("scope") or "movie").lower()
want_season, want_episode, _ = _search_ints(request.args)
if not sid:
return jsonify({"results": [], "live": True, "total_files": 0})
profile = load_profile(get_video_db())
polled = poll_search(sid)
return jsonify({"live": True, "total_files": polled["total_files"],
"results": _evaluate_hits(polled["hits"], profile, scope, want_season, want_episode)})
@bp.route("/downloads/grab", methods=["POST"])
def video_downloads_grab():
"""Start a real download of a chosen release and track it. v1: Soulseek only.
Body: {kind, title, release_title, source, username, filename, size_bytes,
quality_label}."""
from . import get_video_db
from config.settings import config_manager
from core.video.download_monitor import ensure_started
from core.video.download_pipeline import target_dir_for
from core.video.slskd_download import start_download
body = request.get_json(silent=True) or {}
source = str(body.get("source") or "soulseek").lower()
if source != "soulseek":
return jsonify({"ok": False, "error": "Only Soulseek grabs are wired up so far."}), 400
username, filename = body.get("username"), body.get("filename")
if not username or not filename:
return jsonify({"ok": False, "error": "Missing the release's source info."}), 400
db = get_video_db()
paths = {k: db.get_setting(k) or "" for k in ("movies_path", "tv_path", "youtube_path")}
if not paths["movies_path"]:
paths["movies_path"] = db.get_setting("transfer_path") or ""
target = target_dir_for(body.get("kind"), paths)
if not target:
return jsonify({"ok": False, "error": "Set the library folder for this type on Settings → Downloads."}), 400
started = start_download(username, filename, body.get("size_bytes") or 0)
if not started.get("ok"):
return jsonify({"ok": False, "error": started.get("error") or "slskd refused the download."}), 502
import json as _json
from core.video.slskd_search import build_query
# The OTHER accepted results become the retry pool; the search context drives
# the alternate-query requery when the pool runs dry.
ctx = body.get("search_ctx") if isinstance(body.get("search_ctx"), dict) else {}
candidates = [c for c in (body.get("candidates") or []) if isinstance(c, dict) and c.get("filename") != filename]
first_query = build_query(ctx.get("scope") or body.get("kind") or "movie", ctx.get("title") or body.get("title"),
year=ctx.get("year"), season=ctx.get("season"), episode=ctx.get("episode"))
dl_id = db.add_video_download({
"kind": str(body.get("kind") or "movie"), "title": body.get("title"),
"release_title": body.get("release_title") or body.get("filename"),
"source": "soulseek", "username": username, "filename": filename,
"size_bytes": int(body.get("size_bytes") or 0), "quality_label": body.get("quality_label"),
"target_dir": target, "status": "downloading",
"media_id": (str(body.get("media_id")) if body.get("media_id") is not None else None),
"media_source": body.get("media_source"), "year": body.get("year"),
"poster_url": body.get("poster_url"),
"candidates": _json.dumps(candidates), "search_ctx": _json.dumps(ctx),
"tried_queries": _json.dumps([first_query] if first_query else []),
"tried_files": _json.dumps([filename]), "attempts": 0,
})
ensure_started(get_video_db)
return jsonify({"ok": True, "id": dl_id})
@bp.route("/downloads/active", methods=["GET"])
def video_downloads_active():
from . import get_video_db
from core.video.download_monitor import ensure_started
db = get_video_db()
ensure_started(get_video_db) # also (re)start the monitor when the page is open
return jsonify({"downloads": db.list_video_downloads()})
@bp.route("/downloads/status", methods=["GET"])
def video_downloads_status():
"""Lightweight live-tracking lookup — used by the Download modal's result
card (by ``id``) and a movie/show detail page (by ``media_id`` +
``media_source``, returning that title's most relevant download so the page
can show live progress). Returns ``{"download": {...}|null}``."""
from . import get_video_db
from core.video.download_monitor import ensure_started
db = get_video_db()
ensure_started(get_video_db)
dl_id = request.args.get("id")
if dl_id:
try:
return jsonify({"download": db.get_video_download(int(dl_id))})
except (TypeError, ValueError):
return jsonify({"download": None})
media_id = request.args.get("media_id")
if media_id:
media_source = request.args.get("media_source")
match = [r for r in db.list_video_downloads()
if str(r.get("media_id")) == str(media_id)
and (not media_source or r.get("media_source") == media_source)]
# list_video_downloads orders active-first then newest — so an active
# download (or else the most recent) for this title is simply match[0].
return jsonify({"download": match[0] if match else None})
return jsonify({"download": None})
@bp.route("/downloads/cancel", methods=["POST"])
def video_downloads_cancel():
from . import get_video_db
from core.video.slskd_download import cancel_download
body = request.get_json(silent=True) or {}
db = get_video_db()
dl = db.get_video_download(body.get("id"))
if not dl:
return jsonify({"ok": False, "error": "Download not found."}), 404
if dl["status"] in ("completed", "failed", "cancelled"):
return jsonify({"ok": True, "already": True})
cancel_download(dl.get("username"), dl.get("filename")) # best-effort; mark regardless
import time
db.update_video_download(dl["id"], status="cancelled", error="Cancelled",
completed_at=time.strftime("%Y-%m-%d %H:%M:%S"))
return jsonify({"ok": True})
@bp.route("/downloads/retry", methods=["POST"])
def video_downloads_retry():
"""Re-grab the SAME release (basic retry). Auto-retry + alternate-query retry
come in a later phase."""
from . import get_video_db
from core.video.download_monitor import ensure_started
from core.video.slskd_download import start_download
body = request.get_json(silent=True) or {}
db = get_video_db()
dl = db.get_video_download(body.get("id"))
if not dl:
return jsonify({"ok": False, "error": "Download not found."}), 404
if not dl.get("username") or not dl.get("filename"):
return jsonify({"ok": False, "error": "Nothing to retry from."}), 400
started = start_download(dl["username"], dl["filename"], dl.get("size_bytes") or 0)
if not started.get("ok"):
return jsonify({"ok": False, "error": started.get("error") or "slskd refused the download."}), 502
db.update_video_download(dl["id"], status="downloading", progress=0, error=None,
dest_path=None, completed_at=None)
ensure_started(get_video_db)
return jsonify({"ok": True})
@bp.route("/downloads/clear", methods=["POST"])
def video_downloads_clear():
from . import get_video_db
return jsonify({"cleared": get_video_db().clear_finished_video_downloads()})
@bp.route("/downloads/youtube-quality", methods=["GET"])
def video_youtube_quality():
# Separate, smaller profile — YouTube is yt-dlp, not scene/p2p releases.
from . import get_video_db
from core.video.youtube_quality import load
return jsonify(load(get_video_db()))
@bp.route("/downloads/youtube-quality", methods=["POST"])
def video_youtube_quality_save():
from . import get_video_db
from core.video.youtube_quality import save
body = request.get_json(silent=True) or {}
return jsonify(save(get_video_db(), body))
@bp.route("/downloads/slskd", methods=["GET"])
def video_slskd_config():
# SHARED with music — same slskd instance. Reads the app-wide config_manager.
from config.settings import config_manager
return jsonify({k: config_manager.get(cfg, default)
for k, (cfg, default) in _SLSKD_KEYS.items()})
@bp.route("/downloads/slskd", methods=["POST"])
def video_slskd_config_save():
from config.settings import config_manager
body = request.get_json(silent=True) or {}
for k, (cfg, _default) in _SLSKD_KEYS.items():
if k in body:
config_manager.set(cfg, body.get(k))
return jsonify({"status": "saved", "shared": True})

219
api/video/enrichment.py Normal file
View file

@ -0,0 +1,219 @@
"""Video enrichment API — mirrors the music enrichment endpoints so the shared
Manage-Workers modal can drive video workers by pointing at /api/video/...
GET /api/video/enrichment/services
GET /api/video/enrichment/<service>/status
POST /api/video/enrichment/<service>/pause | /resume
GET /api/video/enrichment/<service>/breakdown
GET /api/video/enrichment/<service>/unmatched?kind=&status=&q=&limit=&offset=
POST /api/video/enrichment/<service>/retry {kind, scope, item_id}
"""
from __future__ import annotations
from flask import jsonify, request
from utils.logging_config import get_logger
logger = get_logger("video_api.enrichment")
def _int(val, default):
try:
return int(val)
except (TypeError, ValueError):
return default
def register_routes(bp):
def engine():
from core.video.enrichment.engine import get_video_enrichment_engine
return get_video_enrichment_engine()
def _yt_enricher():
from core.video.youtube_enrichment import get_youtube_date_enricher
return get_youtube_date_enricher()
@bp.route("/enrichment/services", methods=["GET"])
def video_enrichment_services():
try:
return jsonify({"services": engine().services()})
except Exception:
logger.exception("video enrichment services failed")
return jsonify({"services": []})
@bp.route("/enrichment/config", methods=["GET"])
def video_enrichment_config():
from . import get_video_db
db = get_video_db()
return jsonify({
"tmdb_api_key": db.get_setting("tmdb_api_key") or "",
"tvdb_api_key": db.get_setting("tvdb_api_key") or "",
"omdb_api_key": db.get_setting("omdb_api_key") or "",
# Backfill-worker keys (free, optional) + no-key toggles.
"fanart_api_key": db.get_setting("fanart_api_key") or "",
"opensubtitles_api_key": db.get_setting("opensubtitles_api_key") or "",
"trakt_api_key": db.get_setting("trakt_api_key") or "",
"ryd_enabled": (db.get_setting("ryd_enabled") or "1") == "1",
"sponsorblock_enabled": (db.get_setting("sponsorblock_enabled") or "1") == "1",
"dearrow_enabled": (db.get_setting("dearrow_enabled") or "1") == "1",
"tvmaze_enabled": (db.get_setting("tvmaze_enabled") or "1") == "1",
"anilist_enabled": (db.get_setting("anilist_enabled") or "0") == "1",
"wikidata_enabled": (db.get_setting("wikidata_enabled") or "1") == "1",
"billboard_autoplay": (db.get_setting("billboard_autoplay") or "1") == "1",
"watch_region": (db.get_setting("watch_region") or "US").upper(),
})
@bp.route("/prefs", methods=["GET"])
def video_prefs():
# Lightweight UI prefs for the detail page (no API keys).
from . import get_video_db
db = get_video_db()
return jsonify({
"billboard_autoplay": (db.get_setting("billboard_autoplay") or "1") == "1",
"watch_region": (db.get_setting("watch_region") or "US").upper(),
})
@bp.route("/enrichment/config", methods=["POST"])
def video_enrichment_config_save():
from . import get_video_db
db = get_video_db()
body = request.get_json(silent=True) or {}
keys_changed = False
def put_key(field):
nonlocal keys_changed
if field in body:
val = body.get(field) or ""
if val != (db.get_setting(field) or ""):
keys_changed = True
db.set_setting(field, val)
put_key("tmdb_api_key")
put_key("tvdb_api_key")
put_key("fanart_api_key")
put_key("opensubtitles_api_key")
put_key("trakt_api_key")
# No-key worker on/off toggles (read live by the worker — no rebuild needed).
for flag in ("ryd_enabled", "sponsorblock_enabled", "dearrow_enabled",
"tvmaze_enabled", "anilist_enabled", "wikidata_enabled"):
if flag in body:
db.set_setting(flag, "1" if body.get(flag) else "0")
if "billboard_autoplay" in body:
db.set_setting("billboard_autoplay", "1" if body.get("billboard_autoplay") else "0")
if "watch_region" in body:
region = (body.get("watch_region") or "US").strip().upper()[:2] or "US"
db.set_setting("watch_region", region)
if "omdb_api_key" in body:
new_key = body.get("omdb_api_key") or ""
changed = new_key != (db.get_setting("omdb_api_key") or "")
db.set_setting("omdb_api_key", new_key)
keys_changed = keys_changed or changed
# A new/changed OMDb key → re-try every title that still has no rating
# (covers items wrongly marked 'synced' during a prior bad-key run).
if new_key and changed:
try:
for kind in ("movie", "show"):
db.enrichment_retry("omdb", kind, scope="failed")
except Exception:
logger.exception("video enrichment: omdb re-try reset failed")
# Only rebuild the workers when an API KEY actually changed — a prefs-only
# save (autoplay/region) must not churn the running enrichment engine
# (that restart was re-logging the OMDb limit warning on every save).
if keys_changed:
try:
from core.video.enrichment.engine import rebuild_video_enrichment_engine
rebuild_video_enrichment_engine()
except Exception:
logger.exception("video enrichment: engine rebuild after key change failed")
return jsonify({"status": "saved"})
@bp.route("/enrichment/<service>/status", methods=["GET"])
def video_enrichment_status(service):
if service == "youtube": # the standalone date enricher (not an engine worker)
return jsonify(_yt_enricher().stats())
w = engine().worker(service)
if not w:
return jsonify({"error": "unknown service"}), 404
return jsonify(w.get_stats())
@bp.route("/enrichment/<service>/pause", methods=["POST"])
def video_enrichment_pause(service):
if service == "youtube":
_yt_enricher().pause()
return jsonify({"status": "paused"})
w = engine().worker(service)
if not w:
return jsonify({"error": "unknown service"}), 404
w.pause()
return jsonify({"status": "paused"})
@bp.route("/enrichment/<service>/resume", methods=["POST"])
def video_enrichment_resume(service):
if service == "youtube":
_yt_enricher().resume()
return jsonify({"status": "running"})
w = engine().worker(service)
if not w:
return jsonify({"error": "unknown service"}), 404
w.resume()
return jsonify({"status": "running"})
@bp.route("/enrichment/priority", methods=["GET", "POST"])
def video_enrichment_priority():
from . import get_video_db
db = get_video_db()
if request.method == "POST":
body = request.get_json(silent=True) or {}
kind = body.get("priority") or ""
if kind not in ("", "movie", "show"):
return jsonify({"error": "bad priority"}), 400
db.set_setting("enrichment_priority", kind)
return jsonify({"success": True, "priority": kind})
return jsonify({"priority": db.get_setting("enrichment_priority") or ""})
@bp.route("/enrichment/<service>/test", methods=["POST"])
def video_enrichment_test(service):
w = engine().worker(service)
if not w:
return jsonify({"success": False, "error": "unknown service"}), 404
try:
ok, msg = w.client.test()
return jsonify({"success": bool(ok), "message": msg, "error": None if ok else msg})
except Exception:
logger.exception("video enrichment test failed for %s", service)
return jsonify({"success": False, "error": "Test failed"})
@bp.route("/enrichment/<service>/breakdown", methods=["GET"])
def video_enrichment_breakdown(service):
from . import get_video_db
return jsonify({"service": service, "breakdown": get_video_db().enrichment_breakdown(service)})
@bp.route("/enrichment/<service>/unmatched", methods=["GET"])
def video_enrichment_unmatched(service):
from . import get_video_db
kind = request.args.get("kind", "movie")
res = get_video_db().enrichment_unmatched(
service, kind,
status=request.args.get("status", "not_found"),
search=request.args.get("q") or None,
limit=_int(request.args.get("limit"), 50),
offset=_int(request.args.get("offset"), 0))
res.update({"service": service, "kind": kind})
return jsonify(res)
@bp.route("/enrichment/retry-all-failed", methods=["POST"])
def video_enrichment_retry_all_failed():
"""Global re-queue: reset every failed/not_found item across ALL workers and
kinds (one-click recovery after an outage). Returns the total re-queued."""
from . import get_video_db
return jsonify({"success": True, "reset": get_video_db().retry_all_failed()})
@bp.route("/enrichment/<service>/retry", methods=["POST"])
def video_enrichment_retry(service):
from . import get_video_db
body = request.get_json(silent=True) or {}
n = get_video_db().enrichment_retry(
service, body.get("kind", "movie"),
scope=body.get("scope", "failed"), item_id=body.get("item_id"))
return jsonify({"success": True, "reset": n})

182
api/video/libraries.py Normal file
View file

@ -0,0 +1,182 @@
"""Video library mapping endpoints.
GET /api/video/libraries -> discover the active server's Movies/TV libraries
+ the user's current selection.
POST /api/video/libraries -> save {movies, tv} (library titles) for the active
server. The scanner then reads only those.
"""
from __future__ import annotations
from flask import jsonify, request
from utils.logging_config import get_logger
logger = get_logger("video_api.libraries")
def register_routes(bp):
@bp.route("/libraries", methods=["GET"])
def video_libraries():
from . import get_video_db
try:
from core.video.sources import list_video_libraries, resolve_video_server
libs = list_video_libraries() or {"server": None, "movies": [], "tv": []}
server = libs.get("server") or resolve_video_server()
libs["selected"] = (get_video_db().get_library_selection(server)
if server else {"movies": None, "tv": None})
return jsonify(libs)
except Exception:
logger.exception("Failed to list video libraries")
return jsonify({"error": "Failed to list video libraries"}), 500
@bp.route("/server", methods=["GET"])
def video_server_status():
"""Which server the video side uses + which of Plex/Jellyfin are configured
(so the UI can show a picker, or a 'connect a server' message)."""
try:
from core.video.sources import (resolve_video_server,
video_plex_config, video_jellyfin_config)
plex = bool(video_plex_config().get("base_url"))
jelly = bool(video_jellyfin_config().get("base_url"))
return jsonify({"server": resolve_video_server(), "plex": plex, "jellyfin": jelly})
except Exception:
logger.exception("video server status failed")
return jsonify({"server": None, "plex": False, "jellyfin": False})
@bp.route("/server", methods=["POST"])
def video_server_set():
"""Set the explicit video-side server pick (only meaningful when both Plex
and Jellyfin are configured)."""
from . import get_video_db
body = request.get_json(silent=True) or {}
choice = body.get("server")
if choice not in ("plex", "jellyfin"):
return jsonify({"error": "bad server"}), 400
get_video_db().set_setting("video_server", choice)
return jsonify({"status": "saved", "server": choice})
@bp.route("/server-config", methods=["GET"])
def video_server_config_get():
"""The video side's OWN server connection — its stored creds when set, else
the values INHERITED (read-only) from music. 'inherited' flags tell the UI a
field is a placeholder it can override; tokens/keys are returned masked."""
try:
from core.video.sources import video_plex_config, video_jellyfin_config
p, j = video_plex_config(), video_jellyfin_config()
def mask(v):
v = v or ""
return ("" * 12) if v else ""
return jsonify({
"plex": {"base_url": p.get("base_url") or "", "token": mask(p.get("token")),
"has_token": bool(p.get("token")), "inherited": p.get("source") == "music"},
"jellyfin": {"base_url": j.get("base_url") or "", "api_key": mask(j.get("api_key")),
"has_key": bool(j.get("api_key")), "inherited": j.get("source") == "music"},
})
except Exception:
logger.exception("video server-config get failed")
return jsonify({"plex": {}, "jellyfin": {}})
@bp.route("/server-config", methods=["POST"])
def video_server_config_set():
"""Save the video side's OWN Plex/Jellyfin creds to video.db (NEVER the music
config). An empty/blank field clears that override video falls back to
inheriting music's value. A masked token (all •) is left untouched."""
from . import get_video_db
body = request.get_json(silent=True) or {}
db = get_video_db()
def is_mask(v):
return bool(v) and set(str(v)) == {""}
def put(key, val):
if is_mask(val):
return # unchanged masked secret — keep what's stored
db.set_setting(key, (val or "").strip())
plex = body.get("plex") or {}
jelly = body.get("jellyfin") or {}
if "base_url" in plex:
put("video_plex_url", plex.get("base_url"))
if "token" in plex:
put("video_plex_token", plex.get("token"))
if "base_url" in jelly:
put("video_jellyfin_url", jelly.get("base_url"))
if "api_key" in jelly:
put("video_jellyfin_key", jelly.get("api_key"))
return jsonify({"status": "saved"})
@bp.route("/server-config/test", methods=["POST"])
def video_server_config_test():
"""Test the video side's effective connection for one server, using its OWN
stored/inherited creds (so it verifies exactly what the video scan will use)."""
body = request.get_json(silent=True) or {}
which = body.get("server")
if which not in ("plex", "jellyfin"):
return jsonify({"success": False, "error": "bad server"}), 400
try:
if which == "plex":
from core.video.sources import video_plex_config, PLEX_SCAN_TIMEOUT
cfg = video_plex_config()
if not cfg.get("base_url") or not cfg.get("token"):
return jsonify({"success": False, "error": "Plex URL/token not set"})
from plexapi.server import PlexServer
srv = PlexServer(cfg["base_url"], cfg["token"], timeout=PLEX_SCAN_TIMEOUT)
return jsonify({"success": True, "message": "Connected to " + (srv.friendlyName or "Plex")})
from core.video.sources import video_jellyfin_config, video_jellyfin_test
ok, message = video_jellyfin_test(video_jellyfin_config())
if ok:
return jsonify({"success": True, "message": message})
return jsonify({"success": False, "error": message})
except Exception as e:
return jsonify({"success": False, "error": str(e) or "connection failed"})
@bp.route("/jellyfin/users", methods=["GET"])
def video_jellyfin_users():
"""List the Jellyfin server's users so the video side can pick one (its
libraries are scoped to that user) mirrors the music user picker. Uses
video's own effective Jellyfin creds."""
from . import get_video_db
try:
from core.video.sources import video_jellyfin_config
cfg = video_jellyfin_config()
base = (cfg.get("base_url") or "").rstrip("/")
key = cfg.get("api_key") or ""
if not base or not key:
return jsonify({"success": False, "users": []})
import requests
r = requests.get(base + "/Users", headers={"X-Emby-Token": key}, timeout=8)
if r.status_code != 200:
return jsonify({"success": False, "users": [], "error": "HTTP %d" % r.status_code})
users = r.json() or []
out = [{"id": u.get("Id"), "name": u.get("Name"),
"admin": bool((u.get("Policy") or {}).get("IsAdministrator"))}
for u in users if u.get("Id")]
selected = get_video_db().get_setting("video_jellyfin_user") or ""
return jsonify({"success": True, "users": out, "selected": selected})
except Exception as e:
return jsonify({"success": False, "users": [], "error": str(e)})
@bp.route("/jellyfin/user", methods=["POST"])
def video_jellyfin_user_set():
"""Persist the chosen Jellyfin user (its Id) for the video side."""
from . import get_video_db
body = request.get_json(silent=True) or {}
get_video_db().set_setting("video_jellyfin_user", (body.get("user") or "").strip())
return jsonify({"status": "saved"})
@bp.route("/libraries", methods=["POST"])
def save_video_libraries():
from . import get_video_db
try:
from core.video.sources import resolve_video_server
body = request.get_json(silent=True) or {}
server = resolve_video_server()
if not server:
return jsonify({"error": "no video server"}), 400
get_video_db().set_library_selection(server, body.get("movies"), body.get("tv"))
return jsonify({"status": "saved", "server": server})
except Exception:
logger.exception("Failed to save video library selection")
return jsonify({"error": "Failed to save selection"}), 500

34
api/video/library.py Normal file
View file

@ -0,0 +1,34 @@
"""Video library listing endpoint.
GET /api/video/library -> {"movies": [...], "shows": [...]}
Reads what the last scan mirrored from the media server into video.db.
"""
from __future__ import annotations
from flask import jsonify, request
from utils.logging_config import get_logger
logger = get_logger("video_api.library")
def register_routes(bp):
@bp.route("/library", methods=["GET"])
def video_library():
from . import get_video_db
from core.video.sources import resolve_video_server
try:
return jsonify(get_video_db().query_library(
request.args.get("kind", "movies"),
search=request.args.get("search") or None,
letter=request.args.get("letter") or None,
sort=request.args.get("sort", "title"),
status=request.args.get("status", "all"),
page=request.args.get("page", 1),
limit=request.args.get("limit", 75),
server_source=resolve_video_server(),
))
except Exception:
logger.exception("Failed to query video library")
return jsonify({"error": "Failed to load video library"}), 500

160
api/video/manual_import.py Normal file
View file

@ -0,0 +1,160 @@
"""Manual / failed-import resolution API (isolated).
When a finished download can't be auto-placed it's parked as ``import_failed`` with
the file left on disk (``dest_path`` points at it). The Import page surfaces these and
lets the user place them by hand:
GET /api/video/import/failed the queue of unplaced downloads
POST /api/video/import/<id>/place force-import to the user's chosen identity
POST /api/video/import/<id>/dismiss drop the row (optionally delete the file)
The identity picker on the page reuses the existing /api/video/search (TMDB, with a
``library_id`` annotation for owned titles) no new search endpoint needed here.
Reads only the video engine + video.db; nothing from the music side.
"""
from __future__ import annotations
import json
import os
from flask import jsonify, request
from utils.logging_config import get_logger
logger = get_logger("video_api.manual_import")
_KIND_FOR_SCOPE = {"movie": "movie", "episode": "show"}
def _ctx(row):
try:
c = json.loads(row.get("search_ctx") or "{}")
return c if isinstance(c, dict) else {}
except (ValueError, TypeError):
return {}
def _failed_view(row):
"""The render-ready shape for one unplaced download."""
c = _ctx(row)
return {
"id": row.get("id"),
"title": row.get("title"),
"kind": row.get("kind"),
"year": row.get("year"),
"reason": row.get("error"),
"file": row.get("dest_path"), # where the file is sitting, unplaced
"release_title": row.get("release_title"),
"poster_url": row.get("poster_url"),
"scope": c.get("scope"),
"season": c.get("season"),
"episode": c.get("episode"),
}
def register_routes(bp):
@bp.route("/import/failed", methods=["GET"])
def video_import_failed():
from . import get_video_db
rows = get_video_db().get_import_failed_video_downloads()
return jsonify({"success": True, "items": [_failed_view(r) for r in rows]})
@bp.route("/import/<int:dl_id>/place", methods=["POST"])
def video_import_place(dl_id):
"""Force-import an unplaced file to the user's chosen identity. Body:
{scope, title, year, season, episode, episode_title, media_id}."""
from . import get_video_db
from core.video import organization
from core.video.download_pipeline import target_dir_for
from core.video.importer import real_fs, run_import
from core.video.mediainfo import probe
db = get_video_db()
row = db.get_video_download(dl_id)
if not row or row.get("status") != "import_failed":
return jsonify({"success": False, "error": "Not an unplaced import."}), 404
src = row.get("dest_path")
if not src or not os.path.exists(src):
return jsonify({"success": False, "error": "The file is no longer on disk."}), 410
body = request.get_json(silent=True) or {}
scope = str(body.get("scope") or "").lower()
if scope not in ("movie", "episode"):
return jsonify({"success": False, "error": "Choose a movie or an episode."}), 400
paths = {k: db.get_setting(k) or "" for k in ("movies_path", "tv_path", "youtube_path")}
if not paths["movies_path"]:
paths["movies_path"] = db.get_setting("transfer_path") or ""
override = {
"scope": scope,
"title": body.get("title"),
"year": body.get("year"),
"season": body.get("season"),
"episode": body.get("episode"),
"episode_title": body.get("episode_title"),
"media_id": body.get("media_id"),
"target_dir": target_dir_for(_KIND_FOR_SCOPE[scope], paths),
}
settings = organization.load(db)
prober = probe if settings.get("verify_with_ffprobe", True) else None
patch = run_import(row, src, fs=real_fs(), prober=prober, settings=settings,
force=True, override=override)
try:
db.update_video_download(dl_id, **patch)
except Exception:
logger.exception("manual place: failed to persist import %s", dl_id)
return jsonify({"success": False, "error": "Couldn't save the result."}), 500
ok = patch.get("status") == "completed"
if ok:
# Write NFO + artwork sidecars for the chosen identity (best-effort), then
# refresh the server + DB the same way the auto-download path does
# (batch-complete → scan chain), so the manually-placed title shows up
# without waiting for a scheduled scan.
sidecar_dl = {"kind": _KIND_FOR_SCOPE[scope], "media_source": "tmdb",
"media_id": override.get("media_id"),
"poster_url": row.get("poster_url"),
"search_ctx": json.dumps({"scope": scope, "season": override.get("season"),
"episode": override.get("episode")})}
if settings.get("save_artwork") or settings.get("write_nfo"):
try:
from core.video.download_monitor import write_sidecars
from core.video.importer import real_fs
write_sidecars(db, sidecar_dl, patch["dest_path"], settings, real_fs())
except Exception:
logger.exception("manual place: sidecar write failed for %s", dl_id)
if settings.get("download_subtitles"):
try:
from core.video.download_monitor import write_subtitles_for
from core.video.importer import real_fs
write_subtitles_for(db, sidecar_dl, patch["dest_path"], settings, real_fs())
except Exception:
logger.exception("manual place: subtitle fetch failed for %s", dl_id)
try:
from core.video.download_events import notify_batch_complete
notify_batch_complete({"completed": 1, "manual": True})
except Exception:
logger.exception("manual place: batch-complete notify failed for %s", dl_id)
return jsonify({"success": ok, "status": patch.get("status"),
"dest_path": patch.get("dest_path"), "error": patch.get("error")})
@bp.route("/import/<int:dl_id>/dismiss", methods=["POST"])
def video_import_dismiss(dl_id):
"""Drop a failed-import row. Body {delete_file: bool} optionally removes the
unplaced file from disk too."""
from . import get_video_db
db = get_video_db()
row = db.get_video_download(dl_id)
if not row or row.get("status") != "import_failed":
return jsonify({"success": False, "error": "Not an unplaced import."}), 404
body = request.get_json(silent=True) or {}
if body.get("delete_file"):
src = row.get("dest_path")
if src and os.path.exists(src):
try:
os.remove(src)
except OSError:
logger.warning("dismiss: could not delete %s", src)
db.update_video_download(dl_id, status="cancelled",
error="Dismissed from manual import")
return jsonify({"success": True})

152
api/video/poster.py Normal file
View file

@ -0,0 +1,152 @@
"""Video poster proxy.
GET /api/video/poster/<kind>/<id> streams a movie/show poster from the media
server, server-side (so the Plex token / Jellyfin key never reaches the
browser). Falls back to 404 so the frontend shows its placeholder.
"""
from __future__ import annotations
from flask import Response, abort, request
from utils.logging_config import get_logger
logger = get_logger("video_api.poster")
def _req_width():
"""Optional ?w= thumbnail width (clamped) so the calendar/library don't pull
full-size art into tiny cells. None = original."""
try:
w = request.args.get("w", type=int)
except Exception:
w = None
if not w:
return None
return max(48, min(1600, w))
def _tmdb_resize(url, w, backdrop):
"""Rewrite a TMDB image URL's size segment (/t/p/<size>/...) to a bucket near
``w`` so episode stills load small. Best-effort unchanged if it doesn't match."""
import re
buckets = [300, 780, 1280] if backdrop else [185, 342, 500, 780]
pick = next((b for b in buckets if w <= b), buckets[-1])
return re.sub(r"/t/p/[^/]+/", "/t/p/w%d/" % pick, url, count=1)
def register_routes(bp):
def _stream_art(kind, item_id, art):
from . import get_video_db
ref = get_video_db().get_art_ref(kind, item_id, art)
if not ref or not ref.get("poster_url"):
abort(404)
try:
import requests
w = _req_width()
backdrop = art == "backdrop"
path = ref["poster_url"]
# Enrichment can store a full external URL (e.g. a TMDB season poster) —
# stream it directly; otherwise it's a server path needing the token.
if path.startswith("http://") or path.startswith("https://"):
if w and "image.tmdb.org" in path:
path = _tmdb_resize(path, w, backdrop)
upstream = requests.get(path, timeout=15, stream=True)
if upstream.status_code != 200:
abort(404)
resp = Response(upstream.iter_content(8192),
content_type=upstream.headers.get("Content-Type", "image/jpeg"))
resp.headers["Cache-Control"] = "public, max-age=86400"
return resp
# Art is served from the VIDEO side's effective connection (its own
# creds, or inherited from music) — that's where the item was scanned.
from core.video.sources import video_plex_config, video_jellyfin_config
source = ref.get("server_source")
fallback = None
if source == "plex":
cfg = video_plex_config()
base, token = cfg.get("base_url"), cfg.get("token")
if not base or not token:
abort(404)
base = base.rstrip("/")
params = {"X-Plex-Token": token}
if w:
# Plex photo transcoder → a right-sized JPEG instead of full art;
# fall back to the original if a server has transcoding disabled.
from urllib.parse import quote
h = int(w * (9 / 16 if backdrop else 3 / 2))
url = (base + "/photo/:/transcode?width=%d&height=%d&minSize=1&upscale=1&url=%s"
% (w, h, quote(ref["poster_url"], safe="")))
fallback = base + ref["poster_url"]
else:
url = base + ref["poster_url"]
elif source == "jellyfin":
cfg = video_jellyfin_config()
base, key = cfg.get("base_url"), cfg.get("api_key")
if not base:
abort(404)
image = "Backdrop" if backdrop else "Primary"
url = base.rstrip("/") + f"/Items/{ref['server_id']}/Images/{image}"
params = {"api_key": key} if key else {}
if w:
params["maxWidth"] = w
params["quality"] = 90
else:
abort(404)
upstream = requests.get(url, params=params, timeout=15, stream=True)
if upstream.status_code != 200 and fallback:
upstream = requests.get(fallback, params=params, timeout=15, stream=True)
if upstream.status_code != 200:
abort(404)
ctype = upstream.headers.get("Content-Type", "image/jpeg")
resp = Response(upstream.iter_content(8192), content_type=ctype)
resp.headers["Cache-Control"] = "public, max-age=86400"
return resp
except Exception:
logger.exception("video %s proxy failed for %s/%s", art, kind, item_id)
abort(404)
@bp.route("/poster/<kind>/<int:item_id>", methods=["GET"])
def video_poster(kind, item_id):
return _stream_art(kind, item_id, "poster")
@bp.route("/backdrop/<kind>/<int:item_id>", methods=["GET"])
def video_backdrop(kind, item_id):
return _stream_art(kind, item_id, "backdrop")
@bp.route("/img", methods=["GET"])
def video_img_proxy():
"""Same-origin image proxy (SSRF-safe allowlist). TMDB for accent-sampling,
and YouTube CDN (avatars/banners/thumbnails) so channel art loads reliably
regardless of hotlink/CORS policy."""
from flask import request
from urllib.parse import urlparse
url = request.args.get("u", "")
if not url.startswith("https://"):
abort(404)
host = (urlparse(url).hostname or "").lower()
# image.tmdb.org + any YouTube image host (yt3/yt4.ggpht, googleusercontent, *.ytimg)
ok = host == "image.tmdb.org" or any(
host == s or host.endswith("." + s) for s in ("ytimg.com", "ggpht.com", "googleusercontent.com"))
if not ok:
abort(404)
try:
import requests
# A browser UA — Google's image CDN (yt3/googleusercontent) 403s some
# avatars when fetched with no User-Agent, which blanked search results.
upstream = requests.get(url, timeout=15, stream=True, headers={
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36",
"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8",
})
if upstream.status_code != 200:
abort(404)
resp = Response(upstream.iter_content(8192),
content_type=upstream.headers.get("Content-Type", "image/jpeg"))
resp.headers["Cache-Control"] = "public, max-age=604800"
resp.headers["Access-Control-Allow-Origin"] = "*"
return resp
except Exception:
logger.exception("video image proxy failed for %s", url)
abort(404)

61
api/video/scan.py Normal file
View file

@ -0,0 +1,61 @@
"""Video library scan endpoints.
POST /api/video/scan/request -> start a background scan of the active server
GET /api/video/scan/status -> current scan progress/state
The scan READS the media server (source of truth) into video.db. Triggering the
server's own rescan (post-download) is wired separately into the download flow.
"""
from __future__ import annotations
from flask import jsonify, request
from utils.logging_config import get_logger
logger = get_logger("video_api.scan")
def register_routes(bp):
@bp.route("/scan/request", methods=["POST"])
def video_scan_request():
from . import get_video_db
from core.video.scanner import get_video_scanner
from core.video.sources import get_active_video_source
body = request.get_json(silent=True) or {}
mode = body.get("mode", "full")
# Which library to scan — movies and TV are independent libraries, so the
# UI can target one or both. The scanner normalizes/validates it.
media_type = body.get("media_type", "all")
scanner = get_video_scanner(get_video_db())
return jsonify(scanner.request_scan(get_active_video_source, mode, media_type))
@bp.route("/scan/status", methods=["GET"])
def video_scan_status():
from . import get_video_db
from core.video.scanner import get_video_scanner
return jsonify(get_video_scanner(get_video_db()).get_status())
@bp.route("/scan/stop", methods=["POST"])
def video_scan_stop():
from . import get_video_db
from core.video.scanner import get_video_scanner
return jsonify(get_video_scanner(get_video_db()).cancel())
# ── Server-side scan (distinct from the SoulSync-reads-server scan above) ──
# This tells the media server (Plex/Jellyfin) to rescan its OWN folders so
# newly-downloaded files get indexed — the manual twin of the post-download
# 'Scan Video Server' automation. media_type scopes it to Movies / TV / both.
@bp.route("/scan/server", methods=["POST"])
def video_scan_server():
from core.video.sources import refresh_video_server_sections
body = request.get_json(silent=True) or {}
media_type = body.get("media_type", "all")
return jsonify(refresh_video_server_sections(media_type))
@bp.route("/scan/server/status", methods=["GET"])
def video_scan_server_status():
# {scanning: true|false|null} — null when the adapter can't report state.
from core.video.sources import video_server_scan_in_progress
media_type = request.args.get("media_type", "all")
return jsonify({"scanning": video_server_scan_in_progress(media_type)})

43
api/video/search.py Normal file
View file

@ -0,0 +1,43 @@
"""Video search API (in-app, isolated).
GET /api/video/search?q=... TMDB multi-search (movies / shows / people),
movie/show results annotated with library_id
if already owned.
Everything resolves back into SoulSync results link to the library detail
(owned) or the TMDB-backed detail (not owned); people open the in-app person
page. Reads only the video engine + video.db.
"""
from __future__ import annotations
from flask import jsonify, request
from utils.logging_config import get_logger
logger = get_logger("video_api.search")
def register_routes(bp):
@bp.route("/search", methods=["GET"])
def video_search():
q = (request.args.get("q") or "").strip()
if not q:
return jsonify({"results": [], "query": ""})
try:
from core.video.enrichment.engine import get_video_enrichment_engine
results = get_video_enrichment_engine().search(q)
except Exception:
logger.exception("video search failed for %r", q)
results = []
return jsonify({"results": results, "query": q})
@bp.route("/trending", methods=["GET"])
def video_trending():
try:
from core.video.enrichment.engine import get_video_enrichment_engine
results = get_video_enrichment_engine().trending()
except Exception:
logger.exception("video trending failed")
results = []
return jsonify({"results": results})

122
api/video/watchlist.py Normal file
View file

@ -0,0 +1,122 @@
"""Video watchlist API — the user's curated follow-list of shows + people.
Mirrors the music watchlist's add/remove/list/check shape. v1 just manages
membership (so cards can toggle + the Watchlist page can render); the
monitoring/discovery engine that turns a follow into new downloads is a later
phase. Reads/writes only video_library.db via the shared VideoDatabase.
"""
from __future__ import annotations
from flask import jsonify, request
from utils.logging_config import get_logger
logger = get_logger("video_api.watchlist")
_KINDS = ("show", "person")
def _server():
"""Active video server_source (scopes the airing-show default). None on error."""
try:
from core.video.sources import resolve_video_server
return resolve_video_server()
except Exception:
return None
def register_routes(bp):
@bp.route("/watchlist", methods=["GET"])
def video_watchlist_list():
"""All watchlist entries grouped by kind (for the tabbed page).
Shows include actively-airing library shows by default, scoped to the
active video server so Plex/Jellyfin libraries don't commingle."""
from . import get_video_db
try:
db = get_video_db()
server = _server()
kind = request.args.get("kind")
counts = db.watchlist_counts(server_source=server)
if kind in _KINDS:
# Paged + searchable, like the library page.
res = db.query_watchlist(
kind, search=request.args.get("search", ""),
sort=request.args.get("sort", "default"),
page=request.args.get("page", 1), limit=request.args.get("limit", 60),
server_source=server)
return jsonify({"success": True, "kind": kind, "counts": counts, **res})
# No kind → grouped (counts + first-glance lists).
rows = db.list_watchlist(server_source=server)
shows = [r for r in rows if r.get("kind") == "show"]
people = [r for r in rows if r.get("kind") == "person"]
return jsonify({"success": True, "shows": shows, "people": people, "counts": counts})
except Exception:
logger.exception("Failed to list video watchlist")
return jsonify({"success": False, "error": "Failed to load watchlist"}), 500
@bp.route("/watchlist/counts", methods=["GET"])
def video_watchlist_counts():
from . import get_video_db
try:
return jsonify({"success": True, **get_video_db().watchlist_counts(server_source=_server())})
except Exception:
logger.exception("Failed to count video watchlist")
return jsonify({"success": False, "error": "Failed"}), 500
@bp.route("/watchlist/add", methods=["POST"])
def video_watchlist_add():
"""Add a show/person. Body: {kind, tmdb_id, title, poster_url?, library_id?}."""
from . import get_video_db
body = request.get_json(silent=True) or {}
kind = body.get("kind")
tmdb_id = body.get("tmdb_id")
title = (body.get("title") or "").strip()
if kind not in _KINDS or not tmdb_id or not title:
return jsonify({"success": False, "error": "kind, tmdb_id and title are required"}), 400
try:
ok = get_video_db().add_to_watchlist(
kind, int(tmdb_id), title,
poster_url=body.get("poster_url") or None,
library_id=body.get("library_id") or None)
if not ok:
return jsonify({"success": False, "error": "Could not add to watchlist"}), 400
return jsonify({"success": True, "watched": True})
except Exception:
logger.exception("Failed to add to video watchlist")
return jsonify({"success": False, "error": "Failed"}), 500
@bp.route("/watchlist/remove", methods=["POST"])
def video_watchlist_remove():
"""Remove a show/person. Body: {kind, tmdb_id}."""
from . import get_video_db
body = request.get_json(silent=True) or {}
kind = body.get("kind")
tmdb_id = body.get("tmdb_id")
if kind not in _KINDS or not tmdb_id:
return jsonify({"success": False, "error": "kind and tmdb_id are required"}), 400
try:
removed = get_video_db().remove_from_watchlist(kind, int(tmdb_id))
return jsonify({"success": True, "watched": False, "removed": removed})
except Exception:
logger.exception("Failed to remove from video watchlist")
return jsonify({"success": False, "error": "Failed"}), 500
@bp.route("/watchlist/check", methods=["POST"])
def video_watchlist_check():
"""Hydrate cards. Body: {kind, tmdb_ids: [...]} → {results: {id: true}}.
Only watched ids appear in results (absent = not watched)."""
from . import get_video_db
body = request.get_json(silent=True) or {}
kind = body.get("kind")
ids = body.get("tmdb_ids") or []
if kind not in _KINDS:
return jsonify({"success": False, "error": "kind is required"}), 400
try:
state = get_video_db().watchlist_state(kind, ids, server_source=_server())
# JSON object keys must be strings.
return jsonify({"success": True, "results": {str(k): True for k in state}})
except Exception:
logger.exception("Failed to check video watchlist")
return jsonify({"success": False, "error": "Failed"}), 500

185
api/video/wishlist.py Normal file
View file

@ -0,0 +1,185 @@
"""Video wishlist API — the curated 'get this' list (movies + episodes).
Atomic units are movies and episodes; adding a whole show or a season just hands
us the explicit episodes to expand into rows. v1 manages membership + the tabbed
Movies/TV page; the search/download engine that fulfils a wish is a later phase.
Reads/writes only video_library.db via the shared VideoDatabase.
"""
from __future__ import annotations
from flask import jsonify, request
from utils.logging_config import get_logger
logger = get_logger("video_api.wishlist")
_KINDS = ("movie", "show")
_SCOPES = ("movie", "show", "season", "episode")
def _server():
"""Active video server_source (stored on rows, informational). None on error."""
try:
from core.video.sources import resolve_video_server
return resolve_video_server()
except Exception:
return None
def register_routes(bp):
@bp.route("/wishlist", methods=["GET"])
def video_wishlist_list():
"""Paged slice for a tab (kind='movie'|'show'), or counts-only with no kind.
Shows are grouped showseasonepisode with wanted/done roll-ups."""
from . import get_video_db
try:
db = get_video_db()
counts = db.wishlist_counts()
kind = request.args.get("kind")
if kind in _KINDS:
res = db.query_wishlist(
kind, search=request.args.get("search", ""), sort=request.args.get("sort", "added"),
page=request.args.get("page", 1), limit=request.args.get("limit", 60))
return jsonify({"success": True, "kind": kind, "counts": counts, **res})
return jsonify({"success": True, "counts": counts})
except Exception:
logger.exception("Failed to list video wishlist")
return jsonify({"success": False, "error": "Failed to load wishlist"}), 500
@bp.route("/wishlist/counts", methods=["GET"])
def video_wishlist_counts():
from . import get_video_db
try:
db = get_video_db()
counts = db.wishlist_counts() # {movie, show, episode, total(movie+ep)}
yt = db.youtube_wishlist_counts() # {channel, video} (its own table-shape)
counts["video"] = yt.get("video", 0)
counts["channel"] = yt.get("channel", 0)
# The header/sidebar badge is the WHOLE wishlist — movies + episodes + YouTube
# videos — so fold the YouTube count into the total (it was movies+episodes only,
# which is why a YouTube-only wishlist showed no badge).
counts["total"] = counts.get("total", 0) + counts["video"]
return jsonify({"success": True, **counts})
except Exception:
logger.exception("Failed to count video wishlist")
return jsonify({"success": False, "error": "Failed"}), 500
@bp.route("/wishlist/add", methods=["POST"])
def video_wishlist_add():
"""Add a movie or a set of a show's episodes. Body is one of:
{"movie": {tmdb_id, title, year?, poster_url?, library_id?}}
{"show": {tmdb_id, title, poster_url?, library_id?},
"episodes": [{season_number, episode_number, title?, air_date?}, ]}"""
from . import get_video_db
body = request.get_json(silent=True) or {}
srv = _server()
db = get_video_db()
try:
movie = body.get("movie")
if movie and movie.get("tmdb_id") and (movie.get("title") or "").strip():
ok = db.add_movie_to_wishlist(
int(movie["tmdb_id"]), movie["title"].strip(), year=movie.get("year"),
poster_url=movie.get("poster_url") or None,
library_id=movie.get("library_id") or None, server_source=srv)
return jsonify({"success": ok, "added": 1 if ok else 0, "counts": db.wishlist_counts()})
show = body.get("show")
episodes = body.get("episodes") or []
if show and show.get("tmdb_id") and (show.get("title") or "").strip() and episodes:
n = db.add_episodes_to_wishlist(
int(show["tmdb_id"]), show["title"].strip(), episodes,
poster_url=show.get("poster_url") or None,
library_id=show.get("library_id") or None, server_source=srv)
return jsonify({"success": n > 0, "added": n, "counts": db.wishlist_counts()})
return jsonify({"success": False, "error": "movie or show+episodes required"}), 400
except Exception:
logger.exception("Failed to add to video wishlist")
return jsonify({"success": False, "error": "Failed"}), 500
@bp.route("/wishlist/remove", methods=["POST"])
def video_wishlist_remove():
"""Remove at any granularity. Body: {scope, tmdb_id, season_number?, episode_number?}
where scope movie|show|season|episode."""
from . import get_video_db
body = request.get_json(silent=True) or {}
scope, tmdb_id = body.get("scope"), body.get("tmdb_id")
if scope not in _SCOPES or not tmdb_id:
return jsonify({"success": False, "error": "scope and tmdb_id are required"}), 400
try:
db = get_video_db()
removed = db.remove_from_wishlist(
scope, tmdb_id=int(tmdb_id),
season_number=body.get("season_number"), episode_number=body.get("episode_number"))
return jsonify({"success": True, "removed": removed, "counts": db.wishlist_counts()})
except Exception:
logger.exception("Failed to remove from video wishlist")
return jsonify({"success": False, "error": "Failed"}), 500
@bp.route("/wishlist/clear", methods=["POST"])
def video_wishlist_clear():
"""Empty an entire wishlist tab. Body: {kind} where kind ∈ movie|show|youtube."""
from . import get_video_db
body = request.get_json(silent=True) or {}
kind = body.get("kind")
if kind not in ("movie", "show", "youtube"):
return jsonify({"success": False, "error": "kind must be movie|show|youtube"}), 400
try:
db = get_video_db()
removed = db.clear_wishlist(kind)
return jsonify({"success": True, "removed": removed,
"counts": db.wishlist_counts(), "youtube_counts": db.youtube_wishlist_counts()})
except Exception:
logger.exception("Failed to clear video wishlist")
return jsonify({"success": False, "error": "Failed"}), 500
@bp.route("/wishlist/backfill-art", methods=["POST"])
def video_wishlist_backfill_art():
"""Fill episode stills + season posters for rows that predate art-capture.
One tmdb_season call per (show, season); best-effort. Returns rows filled."""
from . import get_video_db
from core.video.enrichment.engine import get_video_enrichment_engine
db = get_video_db()
eng = get_video_enrichment_engine()
updated = 0
try:
for grp in db.wishlist_art_backfill_targets():
try:
se = eng.tmdb_season(grp["tmdb_id"], grp["season_number"]) or {}
except Exception:
continue
if se.get("poster_url"):
updated += db.set_wishlist_season_poster(grp["tmdb_id"], grp["season_number"], se["poster_url"])
for ep in (se.get("episodes") or []):
en = ep.get("episode_number")
if en is None:
continue
if ep.get("still_url") and db.set_wishlist_still(grp["tmdb_id"], grp["season_number"], en, ep["still_url"]):
updated += 1
if ep.get("overview"):
db.set_wishlist_episode_overview(grp["tmdb_id"], grp["season_number"], en, ep["overview"])
return jsonify({"success": True, "updated": updated})
except Exception:
logger.exception("wishlist art backfill failed")
return jsonify({"success": False, "updated": updated})
@bp.route("/wishlist/check", methods=["POST"])
def video_wishlist_check():
"""Hydrate cards/modal. Body: {movie_ids: [...], show_tmdb_id?} →
{movies: [ids already wished], episodes: ['S_E' already wished]}."""
from . import get_video_db
body = request.get_json(silent=True) or {}
try:
db = get_video_db()
st = db.wishlist_state(
movie_ids=body.get("movie_ids") or [], show_tmdb_id=body.get("show_tmdb_id"))
out = {"success": True, "movies": sorted(st["movies"]), "episodes": sorted(st["episodes"])}
shows = body.get("shows") # multi-show membership for the calendar button
if shows:
keys = db.wishlist_keys_for_shows(shows)
out["by_show"] = {str(tid): sorted(ks) for tid, ks in keys.items()}
return jsonify(out)
except Exception:
logger.exception("Failed to check video wishlist")
return jsonify({"success": False, "error": "Failed"}), 500

494
api/video/youtube.py Normal file
View file

@ -0,0 +1,494 @@
"""Video YouTube API — follow a channel as a "show", its uploads flow to the
wishlist (visual-first: no downloading yet).
GET /youtube/resolve?url=... preview a pasted channel (meta + recent
uploads) + whether it's already followed.
POST /youtube/follow {url} (or a pre-resolved channel) follow +
wish its videos. Returns the channel + counts.
POST /youtube/unfollow {youtube_id} un-follow (keeps wished videos).
GET /youtube/channels followed channels (for the watchlist page).
GET /youtube/wishlist wished videos grouped by channel (+ counts).
POST /youtube/wishlist/remove {scope: channel|video, source_id}.
yt-dlp only; reads/writes only video_library.db.
"""
from __future__ import annotations
from flask import jsonify, request
from utils.logging_config import get_logger
logger = get_logger("video_api.youtube")
# Cap how many recent uploads a follow pulls into the wishlist. Channels can have
# thousands of videos; flat listing is cheap but we don't want to wish them all.
_FOLLOW_LIMIT = 30
_RESOLVE_LIMIT = 24
def _server():
try:
from core.video.sources import resolve_video_server
return resolve_video_server()
except Exception:
return None
def register_routes(bp):
@bp.route("/youtube/video/<video_id>/segments", methods=["GET"])
def video_youtube_segments(video_id):
"""SponsorBlock crowd segments stored for a video (sponsor/intro/outro/…),
so the player can offer skips. [] until the SponsorBlock worker enriches it."""
from . import get_video_db
return jsonify({"video_id": video_id,
"segments": get_video_db().youtube_video_segments(video_id)})
@bp.route("/youtube/resolve", methods=["GET"])
def video_youtube_resolve():
"""Preview a pasted channel URL without committing. Returns the channel +
recent uploads, plus ``following`` so the button can hydrate."""
from . import get_video_db
from core.video import youtube as yt
url = (request.args.get("url") or "").strip()
if not url:
return jsonify({"success": False, "error": "url is required"}), 400
try:
limit = int(request.args.get("limit") or _RESOLVE_LIMIT)
except (TypeError, ValueError):
limit = _RESOLVE_LIMIT
try:
# A playlist link → resolve as a playlist (curator-ordered, partial set).
if yt.parse_playlist_id(url):
pl = yt.resolve_playlist(url, limit=max(1, min(50, limit)))
if not pl:
return jsonify({"success": False, "error": "Could not read that playlist"}), 404
following = bool(get_video_db().playlist_watch_state([pl["playlist_id"]]))
return jsonify({"success": True, "playlist": pl, "following": following})
channel = yt.resolve_channel(url, limit=max(1, min(50, limit)))
if not channel:
return jsonify({"success": False,
"error": "Not a YouTube channel or playlist link (paste a channel "
"URL like youtube.com/@handle, or a playlist link)"}), 404
following = bool(get_video_db().channel_watch_state([channel["youtube_id"]]))
return jsonify({"success": True, "channel": channel, "following": following})
except Exception:
logger.exception("youtube resolve failed for %r", url)
return jsonify({"success": False, "error": "Could not read that link"}), 500
@bp.route("/youtube/follow", methods=["POST"])
def video_youtube_follow():
"""Follow a channel + wish its recent uploads. Body: {url} (re-resolved) or
{channel: {...}} already resolved (avoids a second yt-dlp call)."""
from . import get_video_db
from core.video import youtube as yt
body = request.get_json(silent=True) or {}
db = get_video_db()
try:
channel = body.get("channel")
if not channel:
url = (body.get("url") or "").strip()
if not url:
return jsonify({"success": False, "error": "url or channel required"}), 400
channel = yt.resolve_channel(url, limit=_FOLLOW_LIMIT)
if not channel or not channel.get("youtube_id"):
return jsonify({"success": False, "error": "Could not resolve channel"}), 404
followed = db.add_channel_to_watchlist(channel)
added = db.add_videos_to_wishlist(channel, channel.get("videos") or [], server_source=_server())
if followed: # followed channels get their full upload-date catalog in the background
try:
from core.video.youtube_enrichment import get_youtube_date_enricher
get_youtube_date_enricher().enqueue(channel.get("youtube_id"), channel.get("title"))
except Exception:
pass
return jsonify({"success": followed, "following": followed, "added_videos": added,
"channel": {k: channel.get(k) for k in ("youtube_id", "title", "avatar_url")},
"counts": db.youtube_wishlist_counts()})
except Exception:
logger.exception("youtube follow failed")
return jsonify({"success": False, "error": "Failed to follow channel"}), 500
@bp.route("/youtube/unfollow", methods=["POST"])
def video_youtube_unfollow():
"""Un-follow a channel. Body: {youtube_id}. Wished videos are left in place."""
from . import get_video_db
body = request.get_json(silent=True) or {}
cid = (body.get("youtube_id") or "").strip()
if not cid:
return jsonify({"success": False, "error": "youtube_id is required"}), 400
try:
get_video_db().remove_channel_from_watchlist(cid)
return jsonify({"success": True, "following": False})
except Exception:
logger.exception("youtube unfollow failed")
return jsonify({"success": False, "error": "Failed"}), 500
@bp.route("/youtube/playlist/follow", methods=["POST"])
def video_youtube_playlist_follow():
"""Follow a YouTube playlist. Body: {playlist:{...}} (already resolved) or {url}."""
from . import get_video_db
from core.video import youtube as yt
body = request.get_json(silent=True) or {}
db = get_video_db()
try:
playlist = body.get("playlist")
if not playlist:
url = (body.get("url") or "").strip()
playlist = yt.resolve_playlist(url) if url else None
if not playlist or not playlist.get("playlist_id"):
return jsonify({"success": False, "error": "Could not resolve playlist"}), 404
ok = db.add_playlist_to_watchlist(playlist)
if ok and playlist.get("videos"): # remember the count straight away
try:
db.cache_channel_videos(playlist["playlist_id"], playlist["videos"])
except Exception:
pass
return jsonify({"success": ok, "following": ok,
"playlist": {k: playlist.get(k) for k in ("playlist_id", "title", "thumbnail_url")}})
except Exception:
logger.exception("youtube playlist follow failed")
return jsonify({"success": False, "error": "Failed to follow playlist"}), 500
@bp.route("/youtube/playlist/unfollow", methods=["POST"])
def video_youtube_playlist_unfollow():
from . import get_video_db
pid = ((request.get_json(silent=True) or {}).get("playlist_id") or "").strip()
if not pid:
return jsonify({"success": False, "error": "playlist_id is required"}), 400
try:
get_video_db().remove_playlist_from_watchlist(pid)
return jsonify({"success": True, "following": False})
except Exception:
logger.exception("youtube playlist unfollow failed")
return jsonify({"success": False, "error": "Failed"}), 500
@bp.route("/youtube/channels", methods=["GET"])
def video_youtube_channels():
"""Followed channels (newest first) for the watchlist page. Also sweeps:
any followed channel not date-enriched recently gets queued for the
background enricher (so existing follows get picked up, not just new ones)."""
from . import get_video_db
try:
db = get_video_db()
channels = db.list_watchlist_channels()
playlists = db.list_watchlist_playlists()
try:
from core.video.youtube_enrichment import get_youtube_date_enricher
enr = get_youtube_date_enricher()
for c in channels:
if not db.channel_dates_enriched_recently(c["youtube_id"]):
enr.enqueue(c["youtube_id"], c.get("title"))
except Exception:
pass
return jsonify({"success": True, "channels": channels, "playlists": playlists,
"counts": db.youtube_wishlist_counts()})
except Exception:
logger.exception("youtube channels list failed")
return jsonify({"success": False, "error": "Failed"}), 500
@bp.route("/youtube/channel/<channel_id>/settings", methods=["GET"])
def video_youtube_channel_settings(channel_id):
"""Per-channel overrides (custom show-name + quality), plus the global quality
default so the modal can show 'using default' until the user overrides."""
from . import get_video_db
from core.video.youtube_quality import default_profile, load as load_quality
db = get_video_db()
return jsonify({"success": True, "settings": db.get_channel_settings(channel_id) or {},
"default_quality": load_quality(db) or default_profile()})
@bp.route("/youtube/channel/<channel_id>/settings", methods=["POST"])
def video_youtube_channel_settings_save(channel_id):
"""Save per-channel overrides. A blank custom_name / absent quality clears that
override (falls back to the channel title / global quality). Body:
{custom_name?, quality?{...}}."""
from . import get_video_db
from core.video.youtube_quality import normalize as normalize_quality
db = get_video_db()
body = request.get_json(silent=True) or {}
out = {}
name = str(body.get("custom_name") or "").strip()
if name:
out["custom_name"] = name
if body.get("quality"): # falsy/absent → no override (use the global default)
out["quality"] = normalize_quality(body.get("quality"))
keep = str(body.get("retention") or "").strip()
if keep and keep != "all": # blank/'all' → no policy (keep everything)
out["retention"] = keep
db.set_channel_settings(channel_id, out)
return jsonify({"success": True, "settings": out})
@bp.route("/youtube/wishlist", methods=["GET"])
def video_youtube_wishlist():
"""Wished videos grouped by channel (channel = group, videos = feed)."""
from . import get_video_db
try:
db = get_video_db()
res = db.query_youtube_wishlist(
search=request.args.get("search", ""), sort=request.args.get("sort", "added"),
page=request.args.get("page", 1), limit=request.args.get("limit", 60))
return jsonify({"success": True, "counts": db.youtube_wishlist_counts(), **res})
except Exception:
logger.exception("youtube wishlist list failed")
return jsonify({"success": False, "error": "Failed"}), 500
@bp.route("/youtube/channel/<channel_id>", methods=["GET"])
def video_youtube_channel_detail(channel_id):
"""Full channel detail for the in-app channel page: meta + a deeper page of
uploads, the follow state, and per-video wished flags. Resolves live."""
from . import get_video_db
from core.video import youtube as yt
try:
limit = int(request.args.get("limit") or 60)
except (TypeError, ValueError):
limit = 60
try:
db = get_video_db()
cid = str(channel_id).strip()
following = bool(db.channel_watch_state([cid]))
# Opening a channel page → (re)remember it in the background (followed or
# not — you're looking at it). The enricher caches list + meta + dates.
try:
from core.video.youtube_enrichment import get_youtube_date_enricher
get_youtube_date_enricher().enqueue(cid)
except Exception:
pass
# CACHE-FIRST: a remembered channel renders instantly (no yt-dlp). The
# page's background re-stream + the enricher keep it fresh.
meta = db.get_channel_meta(cid)
cached_vids = db.get_channel_videos(cid)
from_cache = bool(meta and cached_vids)
if from_cache:
channel = {"youtube_id": cid, "title": meta.get("title"), "handle": meta.get("handle"),
"description": meta.get("description"), "avatar_url": meta.get("avatar_url"),
"banner_url": meta.get("banner_url"), "subscriber_count": meta.get("subscriber_count"),
"view_count": meta.get("view_count"), "tags": meta.get("tags") or [],
"videos": cached_vids}
else:
# MISS → fetch live (yt-dlp header + recent uploads) and remember it.
channel = yt.resolve_channel("https://www.youtube.com/channel/" + cid,
limit=max(1, min(90, limit)))
if not channel or not channel.get("youtube_id"):
return jsonify({"success": False, "error": "Channel not found"}), 404
cid = channel["youtube_id"]
db.cache_channel_meta(cid, channel)
db.cache_channel_videos(cid, channel.get("videos") or [])
if channel.get("avatar_url"):
try:
db.set_wishlist_channel_poster(cid, channel["avatar_url"])
except Exception:
pass
vids = channel.get("videos") or []
ids = [v.get("youtube_id") for v in vids]
wished = db.youtube_video_wish_state(ids)
# Dates → year-seasons. Cached list already carries them; only pull a
# fresh RSS (recent ~15) on a live MISS so the cache-hit stays instant.
dates = db.get_video_dates(ids)
if not from_cache:
try:
dates.update(yt.channel_recent_dates(cid) or {})
except Exception:
pass
for v in vids:
v["wished"] = v.get("youtube_id") in wished
if not v.get("published_at") and dates.get(v.get("youtube_id")):
v["published_at"] = dates[v["youtube_id"]]
try:
db.cache_video_dates([{"youtube_id": v["youtube_id"], "published_at": v.get("published_at")}
for v in vids if v.get("published_at")])
except Exception:
pass
return jsonify({"success": True, "kind": "channel", "source": "youtube",
"channel": channel, "following": following, "from_cache": from_cache})
except Exception:
logger.exception("youtube channel detail failed for %r", channel_id)
return jsonify({"success": False, "error": "Could not load channel"}), 500
@bp.route("/youtube/channel/<channel_id>/videos", methods=["POST"])
def video_youtube_channel_videos(channel_id):
"""ONE InnerTube page of a channel's videos — the channel page streams the
WHOLE catalog by re-POSTing with the continuation token from each response
(each page fetched once; no yt-dlp re-scan). POST (not GET) keeps the giant
continuation token out of the URL/access logs; the frontend paces the calls,
so each is fast and never trips the slow-request warning. Returns the videos
(dates refined from cache) + next ``continuation`` (null = no more)."""
from . import get_video_db
from core.video import youtube as yt
body = request.get_json(silent=True) or {}
cont = (body.get("continuation") or "").strip() or None
try:
db = get_video_db()
page = yt.innertube_channel_videos_page(channel_id, continuation=cont)
videos, token = page.get("videos") or [], page.get("continuation")
ids = [v.get("youtube_id") for v in videos if v.get("youtube_id")]
cached = db.get_video_dates(ids)
wished = db.youtube_video_wish_state(ids)
for v in videos:
vid = v.get("youtube_id")
if cached.get(vid): # a cached (possibly exact) date wins
v["published_at"] = cached[vid]
v["wished"] = vid in wished
try:
db.cache_channel_videos(channel_id, videos) # remember the list
db.cache_video_dates([{"youtube_id": v["youtube_id"], "published_at": v.get("published_at")}
for v in videos if v.get("youtube_id") and v.get("published_at")])
except Exception:
pass
return jsonify({"success": True, "videos": videos, "continuation": token})
except Exception:
logger.exception("youtube channel videos batch failed for %r", channel_id)
return jsonify({"success": False, "videos": [], "continuation": None}), 200
@bp.route("/youtube/video/<video_id>", methods=["GET"])
def video_youtube_video_detail(video_id):
"""Full metadata for one video (description, views, likes, duration, tags) —
fetched lazily when a video is selected. Persists the description onto its
wishlist row so re-opening is instant."""
from . import get_video_db
from core.video import youtube as yt
try:
v = yt.video_detail(video_id)
if not v or not v.get("youtube_id"):
return jsonify({"success": False, "error": "Video not found"}), 404
db = get_video_db()
# DeArrow crowd-sourced better title (if enriched) — surfaced as an
# alternative on the detail page.
try:
v["dearrow_title"] = db.youtube_video_dearrow_title(v["youtube_id"])
except Exception:
v["dearrow_title"] = None
if v.get("description"):
try:
db.set_wishlist_video_overview(v["youtube_id"], v["description"])
except Exception:
pass # persistence is best-effort; the detail still returns
if v.get("published_at"): # learned a real date → cache it for year-seasons
try:
db.cache_video_dates([{"youtube_id": v["youtube_id"], "published_at": v["published_at"]}])
except Exception:
pass
return jsonify({"success": True, "video": v})
except Exception:
logger.exception("youtube video detail failed for %r", video_id)
return jsonify({"success": False, "error": "Could not load video"}), 500
@bp.route("/youtube/search", methods=["GET"])
def video_youtube_search():
"""Channel search results for the search page (alongside TMDB), each with a
followed flag so the card can hydrate. Best-effort."""
from . import get_video_db
from core.video import youtube as yt
q = (request.args.get("q") or "").strip()
if not q:
return jsonify({"success": True, "channels": []})
try:
chans = yt.search_channels(q)
following = get_video_db().channel_watch_state([c["youtube_id"] for c in chans])
for c in chans:
c["following"] = c["youtube_id"] in following
return jsonify({"success": True, "channels": chans})
except Exception:
logger.exception("youtube search failed for %r", q)
return jsonify({"success": False, "channels": []})
@bp.route("/youtube/playlists/<channel_id>", methods=["GET"])
def video_youtube_playlists(channel_id):
"""The channel's playlists (collapsible sections on the channel page), each
flagged ``following`` so its Add-to-watchlist button hydrates."""
from . import get_video_db
from core.video import youtube as yt
try:
pls = yt.channel_playlists(channel_id)
followed = get_video_db().playlist_watch_state([p.get("playlist_id") for p in pls])
for p in pls:
p["following"] = p.get("playlist_id") in followed
return jsonify({"success": True, "playlists": pls})
except Exception:
logger.exception("youtube playlists failed for %r", channel_id)
return jsonify({"success": False, "error": "Failed"}), 500
@bp.route("/youtube/playlist/<playlist_id>", methods=["GET"])
def video_youtube_playlist(playlist_id):
"""A playlist's videos + metadata + follow state. Serves BOTH the channel-page
playlist expansion (reads ``videos``) and the standalone playlist detail view
(reads ``playlist`` + ``following``). Curator-ordered a partial set, NOT
grouped by year. Per-video wished + cached-date flags hydrate the toggles."""
from . import get_video_db
from core.video import youtube as yt
try:
limit = int(request.args.get("limit") or 200)
except (TypeError, ValueError):
limit = 200
try:
db = get_video_db()
# Pull the WHOLE playlist. With cookies configured (youtube.cookies_browser,
# like ytdl-sub) yt-dlp pages the full list; anonymous, YouTube throttles
# us to ~100, so the InnerTube catalog (~200 + the true header count) is the
# better source. Use whichever got more, with the true total.
pl = yt.resolve_playlist("https://www.youtube.com/playlist?list=" + playlist_id, limit=5000)
if not pl:
return jsonify({"success": False, "error": "Playlist not found"}), 404
try:
cat = yt.innertube_playlist_catalog(playlist_id)
if len(cat.get("videos") or []) > len(pl.get("videos") or []):
pl["videos"] = cat["videos"]
total = max(pl.get("video_count") or 0, cat.get("total") or 0)
if total:
pl["video_count"] = total
except Exception:
pass
vids = pl.get("videos") or []
ids = [v.get("youtube_id") for v in vids]
wished = db.youtube_video_wish_state(ids)
dates = db.get_video_dates(ids)
for v in vids:
v["wished"] = v.get("youtube_id") in wished
if not v.get("published_at") and dates.get(v.get("youtube_id")):
v["published_at"] = dates[v["youtube_id"]]
try:
db.cache_channel_videos(playlist_id, vids) # remember the count for the watchlist card
except Exception:
pass
return jsonify({"success": True, "videos": vids, "playlist": pl,
"following": bool(db.playlist_watch_state([pl["playlist_id"]])),
"kind": "playlist", "source": "youtube"})
except Exception:
logger.exception("youtube playlist failed for %r", playlist_id)
return jsonify({"success": False, "error": "Failed"}), 500
@bp.route("/youtube/wishlist/add", methods=["POST"])
def video_youtube_wishlist_add():
"""Wish specific videos (per-video add from the channel page). Body:
{channel: {youtube_id, title, avatar_url?}, videos: [{youtube_id, title, }]}."""
from . import get_video_db
body = request.get_json(silent=True) or {}
channel = body.get("channel") or {}
videos = body.get("videos") or []
if not channel.get("youtube_id") or not videos:
return jsonify({"success": False, "error": "channel and videos required"}), 400
try:
db = get_video_db()
n = db.add_videos_to_wishlist(channel, videos, server_source=_server())
return jsonify({"success": n > 0, "added": n, "counts": db.youtube_wishlist_counts()})
except Exception:
logger.exception("youtube wishlist add failed")
return jsonify({"success": False, "error": "Failed"}), 500
@bp.route("/youtube/wishlist/remove", methods=["POST"])
def video_youtube_wishlist_remove():
"""Remove wished videos. Body: {scope: 'channel'|'video', source_id}."""
from . import get_video_db
body = request.get_json(silent=True) or {}
scope = body.get("scope")
source_id = (body.get("source_id") or "").strip()
if scope not in ("channel", "video") or not source_id:
return jsonify({"success": False, "error": "scope and source_id are required"}), 400
try:
db = get_video_db()
removed = db.remove_youtube_from_wishlist(scope, source_id)
return jsonify({"success": True, "removed": removed, "counts": db.youtube_wishlist_counts()})
except Exception:
logger.exception("youtube wishlist remove failed")
return jsonify({"success": False, "error": "Failed"}), 500

View file

@ -44,8 +44,7 @@
}, },
"metadata_enhancement": { "metadata_enhancement": {
"enabled": true, "enabled": true,
"embed_album_art": true, "embed_album_art": true
"single_to_album": false
}, },
"file_organization": { "file_organization": {
"enabled": true, "enabled": true,

View file

@ -662,12 +662,7 @@ class ConfigManager:
# source whose art is smaller is skipped so the next source is # source whose art is smaller is skipped so the next source is
# tried — stops a low-res Cover Art Archive upload from winning. # tried — stops a low-res Cover Art Archive upload from winning.
# 0 disables the size gate. # 0 disables the size gate.
"min_art_size": 1000, "min_art_size": 1000
# When a track matches a SINGLE release, look up the parent ALBUM
# that contains it and tag it as that album, so it groups with its
# album-mates and gets the album cover (not the single's). Off by
# default — it's an extra per-import metadata lookup.
"single_to_album": False
}, },
"musicbrainz": { "musicbrainz": {
"embed_tags": True "embed_tags": True
@ -705,12 +700,6 @@ class ConfigManager:
}, },
"import": { "import": {
"staging_path": "./Staging", "staging_path": "./Staging",
# Master toggle for quality-filtering on import. On by default:
# downloaded files that don't meet the quality profile are
# quarantined instead of imported (same gate the download
# pipeline uses). Off → import everything regardless of quality;
# the library Quality Upgrade Scanner still flags them.
"quality_filter_enabled": True,
"replace_lower_quality": False, "replace_lower_quality": False,
# Use the top Staging folder as the artist (Artist/Album layouts, # Use the top Staging folder as the artist (Artist/Album layouts,
# mixtapes). On by default to preserve the long-standing import # mixtapes). On by default to preserve the long-standing import

View file

@ -32,8 +32,6 @@ from config.settings import config_manager
from core.amazon_client import AmazonClient, AmazonClientError from core.amazon_client import AmazonClient, AmazonClientError
from core.download_plugins.base import DownloadSourcePlugin from core.download_plugins.base import DownloadSourcePlugin
from core.download_plugins.types import AlbumResult, DownloadStatus, TrackResult from core.download_plugins.types import AlbumResult, DownloadStatus, TrackResult
from core.quality.model import AudioQuality
from core.quality.source_map import quality_tier_for_source
from utils.logging_config import get_logger from utils.logging_config import get_logger
logger = get_logger("amazon_download_client") logger = get_logger("amazon_download_client")
@ -78,12 +76,9 @@ class AmazonDownloadClient(DownloadSourcePlugin):
if download_path is None: if download_path is None:
download_path = config_manager.get("soulseek.download_path", "./downloads") download_path = config_manager.get("soulseek.download_path", "./downloads")
self.download_path = Path(download_path) self.download_path = Path(download_path)
try: self.download_path.mkdir(parents=True, exist_ok=True)
self.download_path.mkdir(parents=True, exist_ok=True)
except OSError as e:
logger.warning(f"Could not verify download path {self.download_path}: {e}")
self._quality = quality_tier_for_source("amazon", default="flac") self._quality = config_manager.get("amazon_download.quality", "flac")
self._allow_fallback = config_manager.get("amazon_download.allow_fallback", True) self._allow_fallback = config_manager.get("amazon_download.allow_fallback", True)
self._client = AmazonClient(preferred_codec=self._quality) self._client = AmazonClient(preferred_codec=self._quality)
@ -138,17 +133,11 @@ class AmazonDownloadClient(DownloadSourcePlugin):
album_map: Dict[str, AlbumResult] = {} album_map: Dict[str, AlbumResult] = {}
album_order: List[str] = [] album_order: List[str] = []
preferred = self._client.preferred_codec preferred = self._client.preferred_codec
# Search results only carry the codec (real sample_rate arrives at
# stream time). Claim the format honestly — FLAC for the lossless
# codec, lossy otherwise — so audio_quality derives a real format
# instead of the display label ("Lossless"), and the post-download
# probe pins the actual sample_rate/bit_depth.
amazon_q = AudioQuality(format='flac' if _codec_key(preferred) == 'flac' else 'aac')
for item in items: for item in items:
quality = _quality_label(preferred) quality = _quality_label(preferred)
if item.is_track: if item.is_track:
tr = TrackResult( track_results.append(TrackResult(
username="amazon", username="amazon",
filename=f"{item.asin}||{item.artist_name} - {item.title}", filename=f"{item.asin}||{item.artist_name} - {item.title}",
size=0, size=0,
@ -166,9 +155,7 @@ class AmazonDownloadClient(DownloadSourcePlugin):
"album_asin": item.album_asin, "album_asin": item.album_asin,
"isrc": item.isrc, "isrc": item.isrc,
}, },
) ))
tr.set_quality(amazon_q)
track_results.append(tr)
elif item.is_album: elif item.is_album:
album_asin = item.album_asin or item.asin album_asin = item.album_asin or item.asin
if album_asin not in album_map: if album_asin not in album_map:
@ -186,7 +173,6 @@ class AmazonDownloadClient(DownloadSourcePlugin):
title=item.title, title=item.title,
album=item.album_name, album=item.album_name,
) )
placeholder.set_quality(amazon_q)
album_map[album_asin] = AlbumResult( album_map[album_asin] = AlbumResult(
username="amazon", username="amazon",
album_path=album_asin, album_path=album_asin,

View file

@ -660,13 +660,8 @@ class AutoImportWorker:
auto_process = self._config_manager.get('auto_import.auto_process', True) auto_process = self._config_manager.get('auto_import.auto_process', True)
try: try:
# Phase 3: Identify. # Phase 3: Identify
# Re-identify (#889): if the user designated this exact file's release in identification = self._identify_folder(candidate)
# the Re-identify modal, a hint short-circuits the guessing — we match
# straight against the chosen album. No hint → byte-identical to before.
rematch_hint, identification = self._resolve_rematch_hint(candidate)
if identification is None:
identification = self._identify_folder(candidate)
if not identification: if not identification:
self._record_result(candidate, 'needs_identification', 0.0, self._record_result(candidate, 'needs_identification', 0.0,
error_message='Could not identify album from tags, folder name, or fingerprint') error_message='Could not identify album from tags, folder name, or fingerprint')
@ -695,10 +690,7 @@ class AutoImportWorker:
high_conf_matches = [m for m in match_result.get('matches', []) if m['confidence'] >= 0.8] high_conf_matches = [m for m in match_result.get('matches', []) if m['confidence'] >= 0.8]
has_strong_individual_matches = len(high_conf_matches) > 0 has_strong_individual_matches = len(high_conf_matches) > 0
# A re-identify is an explicit user choice — let it auto-process like a if (confidence >= threshold or has_strong_individual_matches) and auto_process:
# strong match (still gated on the global auto_process preference).
if (confidence >= threshold or has_strong_individual_matches
or rematch_hint is not None) and auto_process:
# Phase 5: Auto-process — insert an in-progress row # Phase 5: Auto-process — insert an in-progress row
# so the UI sees the import the moment it starts, # so the UI sees the import the moment it starts,
# then update it with the final status when done. # then update it with the final status when done.
@ -717,13 +709,6 @@ class AutoImportWorker:
confidence = max(confidence, effective_conf) confidence = max(confidence, effective_conf)
if success: if success:
self._bump_stat('auto_processed') self._bump_stat('auto_processed')
# Re-identify (#889): only NOW that the new home exists do we
# consume the hint and (if replace was chosen) delete the old
# row + file — so a failed import never loses the original. Pass
# the landing paths so we never delete a file the re-import landed
# at the SAME place (picking the release it's already in).
if rematch_hint is not None:
self._finalize_rematch_hint(rematch_hint, getattr(candidate, '_reid_final_paths', None))
else: else:
self._bump_stat('failed') self._bump_stat('failed')
@ -1018,75 +1003,6 @@ class AutoImportWorker:
except Exception: except Exception:
return False return False
# ── Re-identify hints (#889) ──
def _resolve_rematch_hint(self, candidate: 'FolderCandidate'):
"""If this staged file carries a user-designated re-identify hint, return
``(hint, identification)`` so matching skips the guessing tiers; otherwise
``(None, None)`` and the caller falls back to normal identification.
Fail-safe: ANY error (no table, DB hiccup) returns ``(None, None)`` so a
re-identify problem can never break ordinary auto-import. Only single-file
candidates are eligible a re-identify always stages exactly one track."""
try:
files = candidate.audio_files or []
if len(files) != 1:
return None, None
from core.imports.rematch_hints import (
build_identification_from_hint,
find_hint_for_file,
quick_file_signature,
)
file_path = files[0]
sig = quick_file_signature(file_path)
conn = self.database._get_connection()
try:
cursor = conn.cursor()
hint = find_hint_for_file(cursor, file_path, sig)
finally:
conn.close()
if hint is None:
return None, None
logger.info("[Auto-Import] Re-identify hint for %s%s '%s' (%s)",
candidate.name, hint.album_type or 'release',
hint.album_name or '?', hint.source)
return hint, build_identification_from_hint(hint)
except Exception as e:
logger.debug("[Auto-Import] rematch-hint lookup skipped: %s", e)
return None, None
def _finalize_rematch_hint(self, hint, new_paths=None) -> None:
"""Post-success: delete the replaced library row + file (if the user chose
replace) and consume the hint so it's single-use. ``new_paths`` are where the
re-import landed passed through so the same-home guard never deletes a file
the import wrote at the old location. Best-effort a cleanup failure is
logged, never raised, since the re-import already succeeded."""
try:
from core.imports.rematch_hints import consume_hint, delete_replaced_track
def _resolve_old(stored):
# The old row's path is a STORED path (Docker/media-server view) — map
# it to a file this process can actually unlink, same as everywhere else.
try:
from core.library.path_resolver import resolve_library_file_path
return resolve_library_file_path(stored, config_manager=getattr(self, '_config_manager', None))
except Exception:
return None
conn = self.database._get_connection()
try:
cursor = conn.cursor()
removed = delete_replaced_track(cursor, hint.replace_track_id,
resolve_fn=_resolve_old, new_paths=new_paths)
consume_hint(cursor, hint.id)
conn.commit()
finally:
conn.close()
if removed:
logger.info("[Auto-Import] Re-identify replaced old track — removed %s", removed)
except Exception as e:
logger.warning("[Auto-Import] rematch-hint finalize failed (import still OK): %s", e)
# ── Identification ── # ── Identification ──
def _identify_folder(self, candidate: FolderCandidate) -> Optional[Dict]: def _identify_folder(self, candidate: FolderCandidate) -> Optional[Dict]:
@ -1515,11 +1431,8 @@ class AutoImportWorker:
def _match_tracks(self, candidate: FolderCandidate, identification: Dict) -> Optional[Dict]: def _match_tracks(self, candidate: FolderCandidate, identification: Dict) -> Optional[Dict]:
"""Match staging files to the identified album's tracklist.""" """Match staging files to the identified album's tracklist."""
# Singles: no album tracklist to match against — the file IS the match. # Singles: no album tracklist to match against — the file IS the match
# force_album_match (set by a re-identify hint) overrides this: even a lone if candidate.is_single or identification.get('is_single'):
# staged file is matched INTO the chosen album, so it inherits the album's
# year / track number / art instead of the bare singles stub (#889).
if not identification.get('force_album_match') and (candidate.is_single or identification.get('is_single')):
conf = identification.get('identification_confidence', 0.7) conf = identification.get('identification_confidence', 0.7)
track_data = { track_data = {
'name': identification.get('track_name', identification.get('album_name', '')), 'name': identification.get('track_name', identification.get('album_name', '')),
@ -1687,7 +1600,6 @@ class AutoImportWorker:
processed = 0 processed = 0
errors = [] errors = []
reid_final_paths = [] # #889: where the pipeline landed each file (same-home guard)
all_matches = list(match_result.get('matches', [])) all_matches = list(match_result.get('matches', []))
# Album total duration — sum of every matched track's duration. # Album total duration — sum of every matched track's duration.
@ -1868,11 +1780,6 @@ class AutoImportWorker:
self._process_callback(context_key, context, file_path) self._process_callback(context_key, context, file_path)
processed += 1 processed += 1
# Capture where the pipeline actually landed the file (#889 same-home
# guard) — the pipeline writes it back into the mutable context.
_landed = context.get('_final_processed_path')
if _landed:
reid_final_paths.append(_landed)
logger.info(f"[Auto-Import] Processed: {track_number}. {track_name}") logger.info(f"[Auto-Import] Processed: {track_number}. {track_name}")
except Exception as e: except Exception as e:
@ -1896,13 +1803,6 @@ class AutoImportWorker:
except Exception as e: except Exception as e:
logger.debug("automation emit failed: %s", e) logger.debug("automation emit failed: %s", e)
# Stash landing paths on the candidate so _finalize_rematch_hint can avoid
# deleting a file the re-import landed at the SAME place (#889).
try:
candidate._reid_final_paths = reid_final_paths
except Exception as e:
logger.debug("could not stash reid final paths: %s", e)
return processed > 0 return processed > 0
# ── Database ── # ── Database ──

View file

@ -10,30 +10,37 @@ Three top-level lists:
- `ACTIONS` DO blocks: process_wishlist, scan_library, etc. - `ACTIONS` DO blocks: process_wishlist, scan_library, etc.
- `NOTIFICATIONS` THEN blocks: discord/pushbullet/telegram/webhook, - `NOTIFICATIONS` THEN blocks: discord/pushbullet/telegram/webhook,
plus fire_signal and run_script then-actions. plus fire_signal and run_script then-actions.
Each block carries an optional ``scope`` tag so the SAME definitions can
feed both the music and the (isolated) video automation builders:
- ``"both"`` generic; shown on both sides (schedule, notifications, ).
- ``"video"`` video-only; shown only on the video builder.
- absent treated as music-only (the default); never shown on video.
Use :func:`blocks_for_scope` to get the filtered lists for one side.
""" """
from __future__ import annotations from __future__ import annotations
TRIGGERS: list[dict] = [ TRIGGERS: list[dict] = [
{"type": "schedule", "label": "Schedule", "icon": "clock", "description": "Run on a timer interval", "available": True, {"type": "schedule", "label": "Schedule", "icon": "clock", "scope": "both", "description": "Run on a timer interval", "available": True,
"config_fields": [ "config_fields": [
{"key": "interval", "type": "number", "label": "Every", "default": 6, "min": 1}, {"key": "interval", "type": "number", "label": "Every", "default": 6, "min": 1},
{"key": "unit", "type": "select", "label": "Unit", {"key": "unit", "type": "select", "label": "Unit",
"options": [{"value": "minutes", "label": "Minutes"}, {"value": "hours", "label": "Hours"}, {"value": "days", "label": "Days"}], "options": [{"value": "minutes", "label": "Minutes"}, {"value": "hours", "label": "Hours"}, {"value": "days", "label": "Days"}],
"default": "hours"} "default": "hours"}
]}, ]},
{"type": "daily_time", "label": "Daily Time", "icon": "clock", "description": "Run every day at a specific time", "available": True, {"type": "daily_time", "label": "Daily Time", "icon": "clock", "scope": "both", "description": "Run every day at a specific time", "available": True,
"config_fields": [ "config_fields": [
{"key": "time", "type": "time", "label": "At", "default": "03:00"} {"key": "time", "type": "time", "label": "At", "default": "03:00"}
]}, ]},
{"type": "weekly_time", "label": "Weekly Schedule", "icon": "calendar", "description": "Run on specific days of the week at a set time", "available": True, {"type": "weekly_time", "label": "Weekly Schedule", "icon": "calendar", "scope": "both", "description": "Run on specific days of the week at a set time", "available": True,
"config_fields": [ "config_fields": [
{"key": "time", "type": "time", "label": "At", "default": "03:00"}, {"key": "time", "type": "time", "label": "At", "default": "03:00"},
{"key": "days", "type": "multi_select", "label": "Days", {"key": "days", "type": "multi_select", "label": "Days",
"options": [{"value": "mon", "label": "Mon"}, {"value": "tue", "label": "Tue"}, {"value": "wed", "label": "Wed"}, "options": [{"value": "mon", "label": "Mon"}, {"value": "tue", "label": "Tue"}, {"value": "wed", "label": "Wed"},
{"value": "thu", "label": "Thu"}, {"value": "fri", "label": "Fri"}, {"value": "sat", "label": "Sat"}, {"value": "sun", "label": "Sun"}]} {"value": "thu", "label": "Thu"}, {"value": "fri", "label": "Fri"}, {"value": "sat", "label": "Sat"}, {"value": "sun", "label": "Sun"}]}
]}, ]},
{"type": "app_started", "label": "App Started", "icon": "power", "description": "When SoulSync starts up", "available": True}, {"type": "app_started", "label": "App Started", "icon": "power", "scope": "both", "description": "When SoulSync starts up", "available": True},
{"type": "track_downloaded", "label": "Track Downloaded", "icon": "download", "description": "When a track finishes downloading", "available": True, {"type": "track_downloaded", "label": "Track Downloaded", "icon": "download", "description": "When a track finishes downloading", "available": True,
"has_conditions": True, "has_conditions": True,
"condition_fields": ["artist", "title", "album", "quality"], "condition_fields": ["artist", "title", "album", "quality"],
@ -106,16 +113,24 @@ TRIGGERS: list[dict] = [
"description": "When duplicate cleaner finishes", "available": True, "description": "When duplicate cleaner finishes", "available": True,
"variables": ["files_scanned", "duplicates_found", "space_freed"]}, "variables": ["files_scanned", "duplicates_found", "space_freed"]},
# Signal trigger # Signal trigger
{"type": "signal_received", "label": "Signal Received", "icon": "zap", {"type": "signal_received", "label": "Signal Received", "icon": "zap", "scope": "both",
"description": "When another automation fires a named signal", "available": True, "description": "When another automation fires a named signal", "available": True,
"config_fields": [ "config_fields": [
{"key": "signal_name", "type": "signal_input", "label": "Signal Name"} {"key": "signal_name", "type": "signal_input", "label": "Signal Name"}
], ],
"variables": ["signal_name"]}, "variables": ["signal_name"]},
# Webhook trigger # Webhook trigger
{"type": "webhook_received", "label": "Webhook Received", "icon": "globe", {"type": "webhook_received", "label": "Webhook Received", "icon": "globe", "scope": "both",
"description": "When an external API request is received (POST /api/v1/request)", "available": True, "description": "When an external API request is received (POST /api/v1/request)", "available": True,
"variables": ["query", "request_id", "source"]}, "variables": ["query", "request_id", "source"]},
# ── Video side (scope='video') — the post-download scan chain triggers ──
{"type": "video_batch_complete", "label": "Video Download Batch Done", "icon": "check-circle", "scope": "video",
"description": "When a batch of video downloads finishes", "available": True,
"variables": ["completed"]},
{"type": "video_library_scan_completed", "label": "Video Library Scan Done", "icon": "hard-drive", "scope": "video",
"description": "When the media server finishes rescanning your video sections", "available": True,
"variables": ["server"]},
] ]
@ -155,7 +170,7 @@ ACTIONS: list[dict] = [
{"key": "refresh_first", "type": "checkbox", "label": "Refresh playlists before sync (regenerate snapshots)", "default": False}, {"key": "refresh_first", "type": "checkbox", "label": "Refresh playlists before sync (regenerate snapshots)", "default": False},
{"key": "skip_wishlist", "type": "checkbox", "label": "Skip wishlist processing", "default": False}, {"key": "skip_wishlist", "type": "checkbox", "label": "Skip wishlist processing", "default": False},
]}, ]},
{"type": "notify_only", "label": "Notify Only", "icon": "bell", "description": "No action — just send notification", "available": True}, {"type": "notify_only", "label": "Notify Only", "icon": "bell", "scope": "both", "description": "No action — just send notification", "available": True},
# Phase 3 actions # Phase 3 actions
{"type": "start_database_update", "label": "Update Database", "icon": "database", {"type": "start_database_update", "label": "Update Database", "icon": "database",
"description": "Trigger library database refresh", "available": True, "description": "Trigger library database refresh", "available": True,
@ -184,7 +199,7 @@ ACTIONS: list[dict] = [
"description": "Clear quarantine, download queue, import folder, and search history in one sweep", "available": True}, "description": "Clear quarantine, download queue, import folder, and search history in one sweep", "available": True},
{"type": "deep_scan_library", "label": "Deep Scan Library", "icon": "search", {"type": "deep_scan_library", "label": "Deep Scan Library", "icon": "search",
"description": "Full library comparison without losing enrichment data", "available": True}, "description": "Full library comparison without losing enrichment data", "available": True},
{"type": "run_script", "label": "Run Script", "icon": "terminal", {"type": "run_script", "label": "Run Script", "icon": "terminal", "scope": "both",
"description": "Execute a script from the scripts folder", "available": True}, "description": "Execute a script from the scripts folder", "available": True},
{"type": "search_and_download", "label": "Search & Download", "icon": "download", {"type": "search_and_download", "label": "Search & Download", "icon": "download",
"description": "Search for a track and download the best match", "available": True, "description": "Search for a track and download the best match", "available": True,
@ -192,28 +207,154 @@ ACTIONS: list[dict] = [
{"key": "query", "type": "text", "label": "Search Query", {"key": "query", "type": "text", "label": "Search Query",
"placeholder": "Artist - Track (leave empty to use trigger's query)"} "placeholder": "Artist - Track (leave empty to use trigger's query)"}
]}, ]},
# ── Video side (isolated app, shared engine) ──────────────────────────
# Tagged scope='video' so they appear ONLY on the video automation
# builder, never the music one. Their handlers bridge into core.video.
{"type": "video_scan_library", "label": "Scan Video Library", "icon": "refresh", "scope": "video",
"description": "Tell the media server to rescan your selected movie/TV sections, then read what it found into SoulSync", "available": True,
"config_fields": [
{"key": "mode", "type": "select", "label": "Mode",
"options": [{"value": "full", "label": "Full (add + refresh)"},
{"value": "incremental", "label": "Incremental (recent only)"},
{"value": "deep", "label": "Deep (also remove missing)"}],
"default": "full"},
{"key": "media_type", "type": "select", "label": "Library",
"options": [{"value": "all", "label": "Movies + TV"},
{"value": "movie", "label": "Movies only"},
{"value": "show", "label": "TV only"}],
"default": "all"}
]},
# Post-download chain actions (two stages, like music's scan_library +
# start_database_update). Stage 1 nudges the server; stage 2 reads it in.
{"type": "video_scan_server", "label": "Scan Video Server", "icon": "refresh", "scope": "video",
"description": "Get the server to index new downloads (skips the scan if it already has them), waits until it finishes, then fires 'Video Library Scan Done'", "available": True,
"config_fields": [
{"key": "media_type", "type": "select", "label": "Library",
"options": [{"value": "all", "label": "Movies + TV"},
{"value": "movie", "label": "Movies only"},
{"value": "show", "label": "TV only"}],
"default": "all"},
{"key": "skip_if_present", "type": "checkbox", "label": "Skip the scan if the server already has the download", "default": True},
{"key": "probe_grace_minutes", "type": "number", "label": "Give the server's auto-scan this long to ingest first (min)", "default": 2, "min": 0},
{"key": "max_wait_minutes", "type": "number", "label": "Max wait for scan (min)", "default": 60, "min": 1},
{"key": "debounce_seconds", "type": "number", "label": "Fallback wait if status unknown (sec)", "default": 120, "min": 10}
]},
{"type": "video_update_database", "label": "Update Video Database", "icon": "database", "scope": "video",
"description": "Read newly-indexed media from the server into SoulSync (incremental)", "available": True,
"config_fields": [
{"key": "mode", "type": "select", "label": "Mode",
"options": [{"value": "incremental", "label": "Incremental (recent only)"},
{"value": "full", "label": "Full (add + refresh)"}],
"default": "incremental"},
{"key": "media_type", "type": "select", "label": "Library",
"options": [{"value": "all", "label": "Movies + TV"},
{"value": "movie", "label": "Movies only"},
{"value": "show", "label": "TV only"}],
"default": "all"}
]},
{"type": "video_update_database_hourly", "label": "Update Video Database (Hourly)", "icon": "database", "scope": "video",
"description": "The same incremental server-read on an hourly schedule, so manual library additions (which Plex auto-scans) appear within the hour instead of waiting for the weekly deep scan. Pair with a 1-hour Schedule trigger.", "available": True},
# Per-library deep-scan presets (the system 'Auto-Deep Scan TV/Movie Library' run
# these). Scope + deep mode are baked in by the registration wrapper, so no config
# fields — drag one in and it just deep-scans that library.
{"type": "video_deep_scan_tv", "label": "Deep Scan TV Library", "icon": "search", "scope": "video",
"description": "Full reconcile of the TV library: re-read every show from the server and drop ones it no longer has (a read, NOT a Plex disk-scan; never touches movies)", "available": True},
{"type": "video_deep_scan_movies", "label": "Deep Scan Movie Library", "icon": "search", "scope": "video",
"description": "Full reconcile of the Movie library: re-read every movie from the server and drop ones it no longer has (a read, NOT a Plex disk-scan; never touches TV)", "available": True},
{"type": "video_refresh_airing_schedules", "label": "Refresh Airing TV Schedules", "icon": "calendar", "scope": "video",
"description": "Re-pull the latest TMDB episode schedules (air dates, stills) for the still-airing shows on your watchlist, so the calendar the airing automation reads is current — newly-announced or rescheduled episodes get picked up instead of being missed. Skips ended/canceled shows. Pair with a daily Schedule a couple hours before the airing run.", "available": True},
{"type": "video_clean_youtube_episodes", "label": "Clean Old YouTube Episodes", "icon": "trash", "scope": "video",
"description": "Delete downloaded YouTube channel episodes that fall outside each channel's keep window (set per channel via its cog → Keep: e.g. last 30 episodes / last 36 months). Removes the video + its sidecars but keeps the history record so it's never re-downloaded. No-op for channels left on 'keep everything' (the default). Playlists are excluded. Pair with a daily Schedule.", "available": True},
{"type": "video_add_airing_episodes", "label": "Wishlist Today's Airings", "icon": "calendar", "scope": "video",
"description": "Sonarr-style: add every episode airing today (for shows you follow) to the wishlist, skipping ones you already own. Also tidies the watchlist by dropping shows that have ended/been canceled.", "available": True,
"config_fields": [
{"key": "prune_ended", "type": "checkbox", "label": "Also remove ended/canceled shows from the watchlist", "default": True}
]},
# ── Watchlist → Wishlist pipeline ────────────────────────────────────────
# Stage 1 — scans that FILL the wishlist from what you follow.
{"type": "video_scan_watchlist_people", "label": "Scan Watchlist People", "icon": "users", "scope": "video",
"description": "For everyone you follow on the watchlist, wishlist every movie they acted in or directed that you don't already own — the whole back catalog plus anything upcoming (kept as 'monitored' until it's released). First run backlogs everything; later runs are fast.", "available": True},
{"type": "video_scan_watchlist_channels", "label": "Scan Watchlist Channels", "icon": "youtube", "scope": "video",
"description": "For every YouTube channel you follow, wishlist its new long-form uploads (Shorts excluded). Forward-looking from when you followed, plus a safety net that always keeps the last N videos. Pair with a 6-hourly Schedule trigger — channels post at all hours.", "available": True,
"config_fields": [
{"key": "backfill_count", "type": "number", "label": "Always keep the last N videos from each channel", "default": 10, "min": 0}
]},
{"type": "video_scan_watchlist_playlists", "label": "Scan Watchlist Playlists", "icon": "list", "scope": "video",
"description": "For every YouTube playlist you follow, wishlist its videos you don't have yet (and anything later added to it) — mirrors the whole list. Each playlist becomes its own show in the library (playlist-as-show). Shorts excluded. Pair with a schedule.", "available": True},
# Stage 2 — processors that DRAIN the wishlist by downloading.
{"type": "video_process_movie_wishlist", "label": "Process Movie Wishlist", "icon": "download", "scope": "video",
"description": "Auto-grab your wished, released MOVIES from Soulseek: searches each, picks the best release per your quality profile, and downloads it (the monitor finishes + organises it). Skips unreleased ('monitored') ones and any already downloading. Needs the Movie library folder + slskd set. Pair with an hourly schedule.", "available": True,
"config_fields": [
{"key": "max_concurrent", "type": "number", "label": "Max simultaneous searches", "default": 3, "min": 1}
]},
{"type": "video_process_episode_wishlist", "label": "Process Episode Wishlist", "icon": "download", "scope": "video",
"description": "Auto-grab your wished EPISODES from Soulseek: searches each, picks the best release per your quality profile, and downloads it. Fed by the 'Wishlist Today's Airings' scan. Needs the TV library folder + slskd set. Pair with an hourly schedule.", "available": True,
"config_fields": [
{"key": "max_concurrent", "type": "number", "label": "Max simultaneous searches", "default": 3, "min": 1}
]},
{"type": "video_process_youtube_wishlist", "label": "Process YouTube Wishlist", "icon": "download", "scope": "video",
"description": "Download wished YouTube videos (yt-dlp), organised as a Plex 'TV by date' show (channel/year/date). Queues the WHOLE wishlist — the setting only limits how many download at the same time; each finished one starts the next, so it all drains. A completed download leaves the wishlist. Needs the YouTube library folder set on Settings → Downloads.", "available": True,
"config_fields": [
{"key": "max_concurrent", "type": "number", "label": "Max simultaneous downloads", "default": 3, "min": 1}
]},
# Video twins of the music maintenance actions. Distinct action_type (the
# system seeder keys on action_type, so a shared key would collide with the
# music row) but the SAME shared handler — the cleanup operates on the common
# download/search state, so behaviour stays identical. scope='video' keeps
# them on the video builder only; the music blocks above are untouched.
{"type": "video_clean_search_history", "label": "Clean Search History", "icon": "trash-2", "scope": "video",
"description": "Remove old searches from Soulseek", "available": True},
{"type": "video_clean_completed_downloads", "label": "Clean Completed Downloads", "icon": "check-square", "scope": "video",
"description": "Clear completed downloads and empty directories", "available": True},
{"type": "video_full_cleanup", "label": "Full Cleanup", "icon": "trash", "scope": "video",
"description": "Clear quarantine, download queue, import folder, and search history in one sweep", "available": True},
# Custom (NOT a shared handler): backs up video_library.db, not the music DB.
{"type": "video_backup_database", "label": "Backup Database", "icon": "save", "scope": "video",
"description": "Create a timestamped backup of the video library database", "available": True},
] ]
NOTIFICATIONS: list[dict] = [ NOTIFICATIONS: list[dict] = [
{"type": "discord_webhook", "label": "Discord Webhook", "icon": "message", "description": "Send a Discord notification", "available": True, {"type": "discord_webhook", "label": "Discord Webhook", "icon": "message", "scope": "both", "description": "Send a Discord notification", "available": True,
"variables": ["time", "name", "run_count", "status"]}, "variables": ["time", "name", "run_count", "status"]},
{"type": "pushbullet", "label": "Pushbullet", "icon": "push", "description": "Push notification to phone/desktop", "available": True, {"type": "pushbullet", "label": "Pushbullet", "icon": "push", "scope": "both", "description": "Push notification to phone/desktop", "available": True,
"variables": ["time", "name", "run_count", "status"]}, "variables": ["time", "name", "run_count", "status"]},
{"type": "telegram", "label": "Telegram", "icon": "message", "description": "Send a Telegram message", "available": True, {"type": "telegram", "label": "Telegram", "icon": "message", "scope": "both", "description": "Send a Telegram message", "available": True,
"variables": ["time", "name", "run_count", "status"]}, "variables": ["time", "name", "run_count", "status"]},
{"type": "webhook", "label": "Webhook (POST)", "icon": "globe", "description": "Send a POST request to any URL", "available": True, {"type": "webhook", "label": "Webhook (POST)", "icon": "globe", "scope": "both", "description": "Send a POST request to any URL", "available": True,
"variables": ["time", "name", "run_count", "status"]}, "variables": ["time", "name", "run_count", "status"]},
# Signal fire action # Signal fire action
{"type": "fire_signal", "label": "Fire Signal", "icon": "zap", {"type": "fire_signal", "label": "Fire Signal", "icon": "zap", "scope": "both",
"description": "Fire a signal that other automations can listen for", "available": True, "description": "Fire a signal that other automations can listen for", "available": True,
"config_fields": [ "config_fields": [
{"key": "signal_name", "type": "signal_input", "label": "Signal Name"} {"key": "signal_name", "type": "signal_input", "label": "Signal Name"}
]}, ]},
# Run script then-action # Run script then-action
{"type": "run_script", "label": "Run Script", "icon": "terminal", {"type": "run_script", "label": "Run Script", "icon": "terminal", "scope": "both",
"description": "Execute a script after the action completes", "available": True, "description": "Execute a script after the action completes", "available": True,
"config_fields": [ "config_fields": [
{"key": "script_name", "type": "script_select", "label": "Script"} {"key": "script_name", "type": "script_select", "label": "Script"}
]}, ]},
] ]
def _in_scope(block: dict, scope: str) -> bool:
"""A block belongs to ``scope`` if it's generic (``"both"``) or tagged for
that side. Untagged blocks default to ``"music"`` (the original behaviour),
so the video builder never picks up music-only blocks by accident."""
s = block.get("scope", "music")
return s == "both" or s == scope
def blocks_for_scope(scope: str = "music") -> dict:
"""Return the trigger/action/notification lists filtered to one side.
``scope="music"`` reproduces the pre-scope behaviour (everything except
video-only blocks); ``scope="video"`` returns generic + video-only blocks.
"""
return {
"triggers": [b for b in TRIGGERS if _in_scope(b, scope)],
"actions": [b for b in ACTIONS if _in_scope(b, scope)],
"notifications": [b for b in NOTIFICATIONS if _in_scope(b, scope)],
}

View file

@ -105,11 +105,10 @@ def auto_update_discovery_pool(config: Dict[str, Any], deps: AutomationDeps) ->
_MAX_BACKUPS = 5 _MAX_BACKUPS = 5
def auto_backup_database(config: Dict[str, Any], deps: AutomationDeps) -> Dict[str, Any]: def _backup_db_at(db_path: str, deps: AutomationDeps, automation_id) -> Dict[str, Any]:
"""Create a hot SQLite backup, then prune old backups so only the """Create a hot SQLite backup of ``db_path``, then prune old backups so only
newest ``_MAX_BACKUPS`` remain.""" the newest ``_MAX_BACKUPS`` remain. Shared by the music and video backup
automation_id = config.get('_automation_id') actions only the DB file differs (they can't share one backup)."""
db_path = os.environ.get('DATABASE_PATH', 'database/music_library.db')
if not os.path.exists(db_path): if not os.path.exists(db_path):
return {'status': 'error', 'reason': 'Database file not found'} return {'status': 'error', 'reason': 'Database file not found'}
@ -146,6 +145,20 @@ def auto_backup_database(config: Dict[str, Any], deps: AutomationDeps) -> Dict[s
return {'status': 'completed', 'backup_path': backup_path, 'size_mb': str(size_mb)} return {'status': 'completed', 'backup_path': backup_path, 'size_mb': str(size_mb)}
def auto_backup_database(config: Dict[str, Any], deps: AutomationDeps) -> Dict[str, Any]:
"""Hot SQLite backup of the MUSIC database (``DATABASE_PATH``)."""
db_path = os.environ.get('DATABASE_PATH', 'database/music_library.db')
return _backup_db_at(db_path, deps, config.get('_automation_id'))
def auto_backup_video_database(config: Dict[str, Any], deps: AutomationDeps) -> Dict[str, Any]:
"""Hot SQLite backup of the VIDEO database (``VIDEO_DATABASE_PATH`` /
video_library.db). Same logic as the music backup but a different DB file
the two can't share one backup, so this is a video-specific action."""
db_path = os.environ.get('VIDEO_DATABASE_PATH', 'database/video_library.db')
return _backup_db_at(db_path, deps, config.get('_automation_id'))
# ─── refresh_beatport_cache ────────────────────────────────────────── # ─── refresh_beatport_cache ──────────────────────────────────────────

View file

@ -26,6 +26,7 @@ from core.automation.handlers.maintenance import (
auto_cleanup_wishlist, auto_cleanup_wishlist,
auto_update_discovery_pool, auto_update_discovery_pool,
auto_backup_database, auto_backup_database,
auto_backup_video_database,
auto_refresh_beatport_cache, auto_refresh_beatport_cache,
) )
from core.automation.handlers.download_cleanup import ( from core.automation.handlers.download_cleanup import (
@ -35,6 +36,17 @@ from core.automation.handlers.download_cleanup import (
) )
from core.automation.handlers.run_script import auto_run_script from core.automation.handlers.run_script import auto_run_script
from core.automation.handlers.search_and_download import auto_search_and_download from core.automation.handlers.search_and_download import auto_search_and_download
from core.automation.handlers.video_auto_wishlist_airing import auto_video_add_airing_episodes
from core.automation.handlers.video_refresh_airing_schedules import auto_video_refresh_airing_schedules
from core.automation.handlers.video_clean_youtube import auto_video_clean_youtube_episodes
from core.automation.handlers.video_scan_watchlist_people import auto_video_scan_watchlist_people
from core.automation.handlers.video_scan_watchlist_channels import auto_video_scan_watchlist_channels
from core.automation.handlers.video_process_youtube_wishlist import auto_video_process_youtube_wishlist
from core.automation.handlers.video_scan_watchlist_playlists import auto_video_scan_watchlist_playlists
from core.automation.handlers.video_process_wishlist import auto_video_process_wishlist, is_running
from core.automation.handlers.video_scan_library import (
auto_video_scan_library, auto_video_scan_server, auto_video_update_database,
)
from core.automation.handlers.progress_callbacks import ( from core.automation.handlers.progress_callbacks import (
progress_init, progress_init,
progress_finish, progress_finish,
@ -150,6 +162,25 @@ def register_all(deps: AutomationDeps) -> None:
'clean_search_history', 'clean_search_history',
lambda config: auto_clean_search_history(config, deps), lambda config: auto_clean_search_history(config, deps),
) )
# Video twin — same handler, distinct action_type so the system seeder
# (keyed on action_type) creates a separate video-owned row.
engine.register_action_handler(
'video_clean_search_history',
lambda config: auto_clean_search_history(config, deps),
)
engine.register_action_handler(
'video_clean_completed_downloads',
lambda config: auto_clean_completed_downloads(config, deps),
)
engine.register_action_handler(
'video_full_cleanup',
lambda config: auto_full_cleanup(config, deps),
)
# Video DB backup — its OWN handler (video_library.db, not the music DB).
engine.register_action_handler(
'video_backup_database',
lambda config: auto_backup_video_database(config, deps),
)
engine.register_action_handler( engine.register_action_handler(
'clean_completed_downloads', 'clean_completed_downloads',
lambda config: auto_clean_completed_downloads(config, deps), lambda config: auto_clean_completed_downloads(config, deps),
@ -167,6 +198,96 @@ def register_all(deps: AutomationDeps) -> None:
lambda config: auto_search_and_download(config, deps), lambda config: auto_search_and_download(config, deps),
) )
# Video side (isolated app, shared engine). The video twins are tagged
# owned_by='video' on their automation rows so they never surface on the
# music automations page; the handlers bridge into core.video.
engine.register_action_handler(
'video_scan_library',
lambda config: auto_video_scan_library(config, deps),
)
# Per-library deep scans (the video twin of music's 'Auto-Deep Scan Library',
# split because Movies and TV are independent libraries). A deep scan READS the
# server's current state into video.db + prunes what's gone (a full reconcile) —
# it does NOT tell Plex to rescan its disk, so it runs through the read-only
# update-database handler in 'deep' mode, not the nudge+read scan-library one.
# Distinct action types so the seeder (which keys on action_type) sees two
# separate automations; both reuse the one handler, scoped via media_type.
engine.register_action_handler(
'video_deep_scan_movies',
lambda config: auto_video_update_database({**config, 'media_type': 'movie', 'mode': config.get('mode') or 'deep'}, deps),
)
engine.register_action_handler(
'video_deep_scan_tv',
lambda config: auto_video_update_database({**config, 'media_type': 'show', 'mode': config.get('mode') or 'deep'}, deps),
)
# Post-download chain: scan the server, then (on the scan-done event) update the DB.
engine.register_action_handler(
'video_scan_server',
lambda config: auto_video_scan_server(config, deps),
)
engine.register_action_handler(
'video_update_database',
lambda config: auto_video_update_database(config, deps),
)
# Same incremental update, but on an hourly SCHEDULE (not just after a SoulSync scan) —
# so manual library additions (which Plex auto-scans) show up within the hour instead of
# waiting for the weekly deep scan. A distinct action_type because the seeder keys on it.
engine.register_action_handler(
'video_update_database_hourly',
lambda config: auto_video_update_database({'mode': 'incremental', **config}, deps),
)
# Sonarr-style: wishlist every episode airing today (for followed shows).
engine.register_action_handler(
'video_add_airing_episodes',
lambda config: auto_video_add_airing_episodes(config, deps),
)
# Keep the calendar honest: re-pull TMDB episode schedules for still-airing watchlist
# shows (the airing automation above reads the LOCAL calendar, so it needs this fresh).
engine.register_action_handler(
'video_refresh_airing_schedules',
lambda config: auto_video_refresh_airing_schedules(config, deps),
)
# YouTube retention: delete channel episodes outside each channel's keep window (opt-in
# per channel; default keeps everything). The history row stays so it's not re-downloaded.
engine.register_action_handler(
'video_clean_youtube_episodes',
lambda config: auto_video_clean_youtube_episodes(config, deps),
)
# ── Watchlist → Wishlist pipeline ─────────────────────────────────────────
# Stage 1 — SCANS that fill the wishlist from what you follow.
# People: wishlist every un-owned movie followed actors/directors made (catalog + upcoming).
engine.register_action_handler(
'video_scan_watchlist_people',
lambda config: auto_video_scan_watchlist_people(config, deps),
)
# Channels: new long-form uploads from followed YouTube channels (forward + last-N net).
engine.register_action_handler(
'video_scan_watchlist_channels',
lambda config: auto_video_scan_watchlist_channels(config, deps),
)
# Playlists: mirror followed YouTube playlists (whole list + new additions; playlist-as-show).
engine.register_action_handler(
'video_scan_watchlist_playlists',
lambda config: auto_video_scan_watchlist_playlists(config, deps),
)
# Stage 2 — PROCESSORS that drain the wishlist by downloading. Movie/episode go through
# slskd (search → pick best → grab); the guard skips an hourly tick while a drain is still
# working. YouTube goes through yt-dlp (queue all, a few concurrent).
engine.register_action_handler(
'video_process_movie_wishlist',
lambda config: auto_video_process_wishlist(config, deps, media_type='movie'),
lambda: is_running('movie'),
)
engine.register_action_handler(
'video_process_episode_wishlist',
lambda config: auto_video_process_wishlist(config, deps, media_type='episode'),
lambda: is_running('episode'),
)
engine.register_action_handler(
'video_process_youtube_wishlist',
lambda config: auto_video_process_youtube_wishlist(config, deps),
)
# Progress + history callbacks: the engine invokes these around # Progress + history callbacks: the engine invokes these around
# each handler run. Lift the closures from # each handler run. Lift the closures from
# `web_server._register_automation_handlers` into thin lambdas # `web_server._register_automation_handlers` into thin lambdas

View file

@ -0,0 +1,216 @@
"""Automation handler: ``video_add_airing_episodes`` action.
Sonarr-style "monitor airings": add every episode airing TODAY for the TV shows you
follow on the video watchlist to the video WISHLIST, skipping ones you already own.
Runs on a daily schedule so the day's airings queue up to be grabbed automatically.
Like the other video handlers it lives on the SHARED automation side (so it may import
``core.video`` / ``api.video`` the isolation contract only forbids the reverse) and
owns its own progress reporting (``_manages_own_progress``). The calendar read + wishlist
write are injected seams, so the handler is a pure function: tests pass fakes and never
touch a DB or a media server; production lazily binds the real calls.
"""
from __future__ import annotations
from datetime import date
from typing import Any, Callable, Dict, List, Optional
from core.automation.deps import AutomationDeps
def _default_fetch_airing(today: str) -> List[Dict[str, Any]]:
"""Production wiring: the calendar's episodes airing on ``today`` for followed shows."""
from api.video import get_video_db
from core.video.sources import resolve_video_server
return get_video_db().calendar_upcoming(
today, today, server_source=resolve_video_server(), watchlist_only=True)
def _default_add_episodes(show_tmdb_id: Any, show_title: Any, episodes: List[Dict[str, Any]],
library_id: Any = None, poster_url: Any = None) -> int:
from api.video import get_video_db
from core.video.sources import resolve_video_server
return get_video_db().add_episodes_to_wishlist(
show_tmdb_id, show_title, episodes, poster_url=poster_url, library_id=library_id,
server_source=resolve_video_server())
def _show_poster_url(library_id: Any) -> Optional[str]:
"""The SAME poster a manual add stores for a library show — the show poster proxy
path the wishlist orb renders directly. Mirrors the get-modal's
pUrl = '/api/video/poster/show/<library_id>'. Without it the orb falls back to the
show's initials, reading as 'not matched'."""
return ('/api/video/poster/show/%s' % library_id) if library_id is not None else None
def _default_season_meta(tmdb_id: Any, season_number: Any):
"""The SAME TMDB season fetch the show modal uses for a manual add — so auto-added
episodes carry identical stills / overviews / season posters, not the patchy values
the local DB happens to hold."""
from core.video.enrichment.engine import get_video_enrichment_engine
return get_video_enrichment_engine().tmdb_season(tmdb_id, season_number)
# ── watchlist hygiene: drop shows that have ended/been canceled ───────────────
_TERMINAL = ('ended', 'canceled', 'cancelled', 'completed')
def _is_terminal_status(status) -> bool:
"""A show that won't air again — pointless to keep on the (watch-for-new) list."""
return str(status or '').strip().lower() in _TERMINAL
def _default_fetch_follows() -> List[Dict[str, Any]]:
from api.video import get_video_db
return get_video_db().followed_shows()
def _default_show_status(tmdb_id: Any) -> Optional[str]:
"""TMDB status for a follow with no local status (a tmdb-only follow). Cached by
the engine; returns None on any hiccup so we never prune on uncertainty."""
from core.video.enrichment.engine import get_video_enrichment_engine
d = get_video_enrichment_engine().tmdb_full_detail('show', tmdb_id) or {}
return d.get('status')
def _default_remove_show(tmdb_id: Any) -> None:
from api.video import get_video_db
get_video_db().remove_from_watchlist('show', tmdb_id)
def prune_ended_show_follows(deps, automation_id=None, *, fetch_follows=None,
show_status=None, remove_show=None) -> int:
"""Remove explicitly-followed shows that are no longer airing.
Auto-airing LIBRARY shows are already excluded by status, so this targets explicit
eye-button follows (which persist regardless). For follows with no local status (a
tmdb-only follow), look the status up on TMDB. Only prunes on a DEFINITIVE terminal
status unknown status is left alone. Pure: all I/O injected. Returns count removed."""
fetch_follows = fetch_follows or _default_fetch_follows
show_status = show_status or _default_show_status
remove_show = remove_show or _default_remove_show
removed = 0
for f in (fetch_follows() or []):
tid = f.get('tmdb_id')
if not tid:
continue
status = f.get('status')
if not status: # tmdb-only follow — ask TMDB
try:
status = show_status(tid)
except Exception: # noqa: BLE001 - never prune on a lookup failure
status = None
if status and _is_terminal_status(status):
try:
remove_show(tid)
removed += 1
deps.update_progress(
automation_id, log_line="Removed ended show '%s' from the watchlist"
% (f.get('title') or tid), log_type='info')
except Exception: # noqa: BLE001, S110 - a progress-log failure must not abort pruning
pass
return removed
def _season_lookup(season_meta, tmdb_id, season_number, cache):
"""(season_poster_url, {episode_number: tmdb_episode}) for a show+season, fetched
once and cached. A TMDB hiccup degrades to empty (DB values fill in)."""
key = (tmdb_id, season_number)
if key not in cache:
try:
sm = season_meta(tmdb_id, season_number) or {}
except Exception: # noqa: BLE001 - never let a metadata fetch break the run
sm = {}
emap = {e.get('episode_number'): e for e in (sm.get('episodes') or []) if isinstance(e, dict)}
cache[key] = (sm.get('poster_url'), emap)
return cache[key]
def auto_video_add_airing_episodes(
config: Dict[str, Any],
deps: AutomationDeps,
*,
fetch_airing: Optional[Callable[[str], List[Dict[str, Any]]]] = None,
add_episodes: Optional[Callable[[Any, Any, List[Dict[str, Any]]], int]] = None,
today_fn: Optional[Callable[[], str]] = None,
season_meta: Optional[Callable[[Any, Any], Any]] = None,
prune_follows: Optional[Callable[[], List[Dict[str, Any]]]] = None,
show_status: Optional[Callable[[Any], Any]] = None,
remove_show: Optional[Callable[[Any], None]] = None,
) -> Dict[str, Any]:
"""Add today's airing (unowned, followed-show) episodes to the video wishlist, and
first tidy the watchlist by dropping shows that have ended / been canceled.
Returns ``{'status': 'completed', 'episodes_added': int, 'shows': int,
'shows_pruned': int, ...}``."""
fetch_airing = fetch_airing or _default_fetch_airing
add_episodes = add_episodes or _default_add_episodes
season_meta = season_meta or _default_season_meta
today_fn = today_fn or (lambda: date.today().isoformat())
automation_id = config.get('_automation_id')
prune_ended = config.get('prune_ended', True)
try:
today = today_fn()
# Watchlist hygiene first: a followed show that has since ended/been canceled
# won't air again, so drop it (ended LIBRARY shows are already auto-excluded;
# this catches explicit eye-button follows).
pruned = 0
if prune_ended:
deps.update_progress(automation_id, phase='Tidying the watchlist…', progress=10,
log_line='Removing shows that have ended or been canceled', log_type='info')
pruned = prune_ended_show_follows(deps, automation_id, fetch_follows=prune_follows,
show_status=show_status, remove_show=remove_show)
deps.update_progress(automation_id, phase="Checking today's airings…", progress=25,
log_line='Reading the calendar for episodes airing today', log_type='info')
rows = fetch_airing(today) or []
# Group what to wish for by show: airing today, NOT already owned, with a
# real season/episode. add_episodes_to_wishlist is idempotent, so re-runs
# never duplicate.
by_show: Dict[tuple, Dict[str, Any]] = {}
season_cache: Dict[tuple, tuple] = {}
for r in rows:
if r.get('has_file'):
continue
tid = r.get('show_tmdb_id')
sn, en = r.get('season_number'), r.get('episode_number')
if not tid or sn is None or en is None:
continue
# Pull the SAME TMDB metadata a manual add gets (still + overview + season
# poster); fall back to the calendar/DB values if TMDB is unavailable.
poster, emap = _season_lookup(season_meta, tid, sn, season_cache)
tm = emap.get(en) or {}
# library_id (the show's library row id, given as show_id) is REQUIRED — the
# wishlist resolves a show's synopsis + cast from /detail/show/<library_id>;
# without it the show shows as un-matched with no synopsis/actors.
grp = by_show.setdefault((tid, r.get('show_title')),
{'library_id': r.get('show_id'), 'eps': []})
grp['eps'].append({
'season_number': sn,
'episode_number': en,
'title': r.get('title') or tm.get('title'),
'air_date': r.get('air_date') or tm.get('air_date'),
'overview': tm.get('overview') or r.get('overview'),
'still_url': tm.get('still_url') or r.get('still_url'),
'season_poster_url': poster,
})
added = 0
for (tid, title), grp in by_show.items():
added += int(add_episodes(tid, title, grp['eps'], grp['library_id'],
_show_poster_url(grp['library_id'])) or 0)
shows = len(by_show)
done = ('Added %d airing episode(s) across %d show(s) to the wishlist'
% (added, shows)) if added else 'No new airing episodes to wishlist today'
if pruned:
done += ' · pruned %d ended show(s)' % pruned
deps.update_progress(
automation_id, status='finished', progress=100, phase='Complete',
log_line=done, log_type='success')
return {'status': 'completed', 'episodes_added': added, 'shows': shows,
'shows_pruned': pruned, '_manages_own_progress': True}
except Exception as e: # noqa: BLE001
deps.update_progress(automation_id, status='error', phase='Error', log_line=str(e), log_type='error')
return {'status': 'error', 'error': str(e), '_manages_own_progress': True}

View file

@ -0,0 +1,130 @@
"""Automation handler: ``video_clean_youtube_episodes`` action.
YouTube retention: for each followed channel that has a per-channel retention policy (set in
the channel's cog modal — default 'keep everything'), delete the episodes that fall outside
the keep window (video file + its ``-thumb``/``.nfo`` sidecars). The history row is KEPT and
flagged pruned, so the scan's download-dedup still excludes it — a cleaned episode is never
re-downloaded. Playlists are mirror-the-whole-thing, so they're out of scope (channels only).
Runs daily. Pure channel list, per-channel policy, episode list, file deletion + the prune
mark are all injected seams; tests drive it with fakes and never touch a disk or DB.
"""
from __future__ import annotations
import logging
import os
from datetime import date
from typing import Any, Callable, Dict, List, Optional
from core.automation.deps import AutomationDeps
from core.video.retention import episodes_to_prune, parse_retention
logger = logging.getLogger(__name__)
def _default_fetch_channels() -> List[Any]:
from api.video import get_video_db
return get_video_db().youtube_channels_with_downloads()
def _default_channel_retention(channel_id: Any) -> Any:
from api.video import get_video_db
return (get_video_db().get_channel_settings(channel_id) or {}).get("retention")
def _default_fetch_episodes(channel_id: Any) -> List[Dict[str, Any]]:
from api.video import get_video_db
return get_video_db().youtube_channel_episodes(channel_id)
def _default_mark_pruned(history_id: Any, when: str) -> bool:
from api.video import get_video_db
return get_video_db().mark_download_pruned(history_id, when)
def _silent_remove(path: str) -> None:
try:
os.remove(path)
except OSError:
pass
def _default_delete_files(ep: Dict[str, Any]):
"""Delete the episode's video + its sidecars. Returns (ok, freed_bytes). Only ever touches
the exact recorded ``dest_path`` and its same-stem sidecars never walks a folder. A file
already gone counts as success (mark it pruned); a real delete error does NOT (so it retries)."""
path = ep.get("dest_path")
if not path:
return False, 0
size = 0
try:
if os.path.exists(path):
size = os.path.getsize(path)
os.remove(path)
except OSError:
logger.exception("retention: could not delete %s", path)
return False, 0
stem = os.path.splitext(path)[0]
for sc in (stem + ".nfo", stem + "-thumb.jpg", stem + "-thumb.jpeg",
stem + "-thumb.png", stem + "-thumb.webp"):
if os.path.exists(sc):
_silent_remove(sc)
return True, size
def _fmt_gb(b: int) -> str:
gb = (b or 0) / (1024 ** 3)
return ("%.1f GB" % gb) if gb >= 0.1 else "%d MB" % round((b or 0) / (1024 ** 2))
def auto_video_clean_youtube_episodes(
config: Dict[str, Any],
deps: AutomationDeps,
*,
fetch_channels: Optional[Callable[[], List[Any]]] = None,
channel_retention: Optional[Callable[[Any], Any]] = None,
fetch_episodes: Optional[Callable[[Any], List[Dict[str, Any]]]] = None,
delete_files: Optional[Callable[[Dict[str, Any]], Any]] = None,
mark_pruned: Optional[Callable[[Any, str], Any]] = None,
today_fn: Optional[Callable[[], str]] = None,
) -> Dict[str, Any]:
"""Delete out-of-window episodes for channels with a retention policy. Returns
``{'status': 'completed', 'deleted': int, 'channels': int, 'freed_bytes': int, ...}``."""
fetch_channels = fetch_channels or _default_fetch_channels
channel_retention = channel_retention or _default_channel_retention
fetch_episodes = fetch_episodes or _default_fetch_episodes
delete_files = delete_files or _default_delete_files
mark_pruned = mark_pruned or _default_mark_pruned
today_fn = today_fn or (lambda: date.today().isoformat())
automation_id = config.get("_automation_id")
try:
today = today_fn()
deps.update_progress(automation_id, phase="Checking retention…", progress=10,
log_line="Looking for channels with a retention policy", log_type="info")
deleted = freed = checked = 0
for cid in (fetch_channels() or []):
policy = channel_retention(cid)
if not parse_retention(policy): # 'keep everything' / unset → skip
continue
checked += 1
prune = episodes_to_prune(fetch_episodes(cid) or [], policy, today=today)
for ep in prune:
ok, size = delete_files(ep)
if not ok:
continue
mark_pruned(ep.get("id"), today) # keep the row → scan won't re-download it
deleted += 1
freed += size or 0
deps.update_progress(automation_id, log_type="info",
log_line="Removed '%s'" % (ep.get("title") or ep.get("media_id") or "episode"))
msg = ("Removed %d old episode(s) · freed %s" % (deleted, _fmt_gb(freed))) if deleted \
else "Nothing to clean — every channel within its retention window"
deps.update_progress(automation_id, status="finished", progress=100, phase="Complete",
log_line=msg, log_type="success")
return {"status": "completed", "deleted": deleted, "channels": checked,
"freed_bytes": freed, "_manages_own_progress": True}
except Exception as e: # noqa: BLE001
deps.update_progress(automation_id, status="error", phase="Error", log_line=str(e), log_type="error")
return {"status": "error", "error": str(e), "_manages_own_progress": True}

View file

@ -0,0 +1,293 @@
"""Automation handlers: ``video_process_movie_wishlist`` / ``video_process_episode_wishlist``.
The Soulseek counterpart of the YouTube wishlist drain the piece that finally makes the
people/airing scans pay off. For wished, RELEASED movies (and aired episodes) it searches
Soulseek, picks the best release per the quality profile, and enqueues the download; the
existing ``download_monitor`` finishes + organises + archives it, exactly like a manual grab.
Shape mirrors the YouTube drain (Boulder: "same standard"): it processes the WHOLE eligible
wishlist (no total cap), but the slow part each item needs a ~20s blocking Soulseek search
runs only a FEW at a time (``max_concurrent``). A ``guard`` keeps the next hourly tick from
overlapping a run that's still working, so it can't pile up.
Movies are gated on ``status='wanted'`` (released; skips 'monitored'/unreleased). Episodes
are all-wished (the airing scan only adds aired ones). Items already downloading are skipped
so re-runs never double-grab. The pick is the top ACCEPTED release the ranker already
encodes the quality profile's accept/reject/score, so no extra rules here.
Shared automation side (may import ``core.video`` / ``api.video``); owns its own progress.
The search + enqueue are injected seams, so selection/pick/record are pure + unit-tested.
"""
from __future__ import annotations
import json
import threading
from concurrent.futures import ThreadPoolExecutor
from typing import Any, Callable, Dict, Iterable, List, Optional
from core.automation.deps import AutomationDeps
# ── pure helpers ──────────────────────────────────────────────────────────────
def pick_best(candidates: List[Dict[str, Any]]) -> Optional[Dict[str, Any]]:
"""The best ACCEPTED release from a ranked candidate list. ``_evaluate_hits`` already
sorts best-first (accepted, score, availability), so the first accepted is the pick."""
for c in candidates or []:
if c.get("accepted"):
return c
return None
def item_key(item: Dict[str, Any], media_type: str) -> tuple:
"""Stable identity for de-duping a wished item against active downloads."""
if media_type == "movie":
return ("movie", str(item.get("tmdb_id")))
return ("episode", str(item.get("show_tmdb_id")),
int(item.get("season_number") or 0), int(item.get("episode_number") or 0))
def active_download_keys(active: Iterable[Dict[str, Any]]) -> set:
"""Identity keys for the movie/episode downloads already in flight, so we don't
re-grab them. Episodes read season/episode out of the row's ``search_ctx``."""
keys = set()
for d in active or []:
kind = str(d.get("kind") or "").lower()
if kind == "movie":
keys.add(("movie", str(d.get("media_id"))))
elif kind == "episode":
ctx = d.get("search_ctx")
if isinstance(ctx, str):
try:
ctx = json.loads(ctx)
except (ValueError, TypeError):
ctx = {}
ctx = ctx if isinstance(ctx, dict) else {}
keys.add(("episode", str(d.get("media_id")),
int(ctx.get("season") or 0), int(ctx.get("episode") or 0)))
return keys
def search_context(item: Dict[str, Any], media_type: str) -> Dict[str, Any]:
"""The ``search_ctx`` the download row carries (drives the monitor's requery)."""
if media_type == "movie":
return {"scope": "movie", "title": item.get("title"), "year": item.get("year")}
return {"scope": "episode", "title": item.get("show_title"),
"season": item.get("season_number"), "episode": item.get("episode_number"),
"year": (str(item.get("air_date") or "")[:4] or None)}
def build_download_record(item: Dict[str, Any], best: Dict[str, Any], candidates: List[Dict[str, Any]],
*, media_type: str, target_dir: str, query: Any) -> Dict[str, Any]:
"""The ``add_video_download`` row for a chosen release — identical shape to a manual
grab, so the monitor finishes it the same way (other accepted hits become the retry
pool)."""
ctx = search_context(item, media_type)
# stash the chosen source's peer stats so the drawer can show its availability snapshot
# (free slot / queue depth / speed at grab time). Retry ignores the extra key.
peer = {k: best.get(k) for k in ("slots", "queue", "speed", "availability") if best.get(k) is not None}
if peer:
ctx = {**ctx, "peer": peer}
rest = [c for c in (candidates or []) if c.get("filename") != best.get("filename")]
media_id = str(item.get("tmdb_id") if media_type == "movie" else item.get("show_tmdb_id"))
return {
"kind": media_type, "title": ctx["title"],
"release_title": best.get("title") or best.get("filename"),
"source": "soulseek", "username": best.get("username"), "filename": best.get("filename"),
"size_bytes": int(best.get("size_bytes") or 0), "quality_label": best.get("quality_label"),
"target_dir": target_dir, "status": "downloading",
"media_id": media_id, "media_source": "tmdb", "year": ctx.get("year"),
"poster_url": item.get("poster_url"),
"candidates": json.dumps(rest), "search_ctx": json.dumps(ctx),
"tried_queries": json.dumps([query] if query else []),
"tried_files": json.dumps([best.get("filename")]), "attempts": 0,
}
# ── production seams ──────────────────────────────────────────────────────────
def _default_fetch_items(media_type: str) -> List[Dict[str, Any]]:
from api.video import get_video_db
db = get_video_db()
return db.movie_wishlist_to_download() if media_type == "movie" else db.episode_wishlist_to_download()
def _default_active_keys(media_type: str) -> set:
from api.video import get_video_db
return active_download_keys(get_video_db().get_active_video_downloads())
def _default_target_dir(media_type: str) -> str:
from api.video import get_video_db
db = get_video_db()
if media_type == "movie":
return db.get_setting("movies_path") or db.get_setting("transfer_path") or ""
return db.get_setting("tv_path") or ""
def _default_search(item: Dict[str, Any], media_type: str):
"""A bounded blocking Soulseek search → ranked candidates (same path the retry worker +
manual search use). Returns [] for a real empty result, or **None** if the search never
ran (slskd not configured / errored / rate-limited) so the caller can say so."""
from api.video.downloads import _evaluate_hits
from core.video.download_monitor import _search_for_retry
from core.video.quality_profile import load as load_profile
from core.video.slskd_search import build_query
from api.video import get_video_db
ctx = search_context(item, media_type)
query = build_query(ctx["scope"], ctx["title"], year=ctx.get("year"),
season=ctx.get("season"), episode=ctx.get("episode"))
res = _search_for_retry(query) or {}
if res.get("started") is False:
return None, res.get("error") # slskd didn't accept the search — pass the reason
profile = load_profile(get_video_db())
return _evaluate_hits(res.get("hits") or [], profile, ctx["scope"],
ctx.get("season"), ctx.get("episode")), None
def _default_enqueue(item: Dict[str, Any], best: Dict[str, Any], candidates: List[Dict[str, Any]],
media_type: str, target_dir: str) -> bool:
"""Start the slskd transfer + write the download row (exactly like the manual flow),
then ensure the monitor is running. Returns True if slskd accepted it."""
from api.video import get_video_db
from core.video.download_monitor import ensure_started
from core.video.slskd_download import start_download
from core.video.slskd_search import build_query
started = start_download(best.get("username"), best.get("filename"), best.get("size_bytes") or 0)
if not started.get("ok"):
return False
ctx = search_context(item, media_type)
query = build_query(ctx["scope"], ctx["title"], year=ctx.get("year"),
season=ctx.get("season"), episode=ctx.get("episode"))
get_video_db().add_video_download(
build_download_record(item, best, candidates, media_type=media_type,
target_dir=target_dir, query=query))
ensure_started(get_video_db)
return True
# ── guard: keep an in-progress drain from overlapping the next tick ───────────
_running: Dict[str, bool] = {"movie": False, "episode": False}
def is_running(media_type: str) -> bool:
return bool(_running.get(media_type))
def auto_video_process_wishlist(
config: Dict[str, Any],
deps: AutomationDeps,
*,
media_type: str = "movie",
fetch_items: Optional[Callable[[str], List[Dict[str, Any]]]] = None,
active_keys: Optional[Callable[[str], Iterable]] = None,
target_dir: Optional[Callable[[str], str]] = None,
search: Optional[Callable[[Dict[str, Any], str], List[Dict[str, Any]]]] = None,
enqueue: Optional[Callable[..., bool]] = None,
) -> Dict[str, Any]:
"""Auto-grab the wished movies (or episodes): search Soulseek, pick the best release,
enqueue. Processes the whole eligible wishlist, a few searches at a time.
Returns ``{'status': 'completed', 'searched': int, 'grabbed': int, ...}``."""
fetch_items = fetch_items or _default_fetch_items
active_keys = active_keys or _default_active_keys
target_dir = target_dir or _default_target_dir
search = search or _default_search
enqueue = enqueue or _default_enqueue
automation_id = config.get('_automation_id')
concurrency = max(1, int(config.get('max_concurrent', 3) or 3))
label = 'movie' if media_type == 'movie' else 'episode'
_running[media_type] = True
try:
root = target_dir(media_type)
if not root:
where = 'Movie' if media_type == 'movie' else 'TV'
deps.update_progress(automation_id, status='finished', progress=100, phase='Complete',
log_line='%s library folder not set — skipping (Settings → Downloads)' % where,
log_type='info')
return {'status': 'completed', 'searched': 0, 'grabbed': 0,
'skipped': 'no_folder', '_manages_own_progress': True}
deps.update_progress(automation_id, phase='Checking the wishlist…', progress=5,
log_line='Looking for wished %ss to grab' % label, log_type='info')
items = fetch_items(media_type) or []
active = set(active_keys(media_type) or set())
todo = [it for it in items if item_key(it, media_type) not in active]
if not todo:
deps.update_progress(automation_id, status='finished', progress=100, phase='Complete',
log_line='Nothing new to grab (%d already in flight)' % len(active),
log_type='info')
return {'status': 'completed', 'searched': 0, 'grabbed': 0, '_manages_own_progress': True}
grabbed = [0]
searched = [0]
noresults = [0] # search came back empty (the source had nothing)
rejected = [0] # source had hits, but none passed the quality profile
notrun = [0] # the search never ran (slskd didn't accept it)
total = len(todo)
lock = threading.Lock()
def _one(it):
found = search(it, media_type)
# the seam returns (candidates, error); tolerate a bare list/None too (test fakes)
cands, err = found if isinstance(found, tuple) else (found, None)
didnt_run = cands is None # slskd not configured / errored / rate-limited
cands = cands or []
best = pick_best(cands)
ok = bool(best) and bool(enqueue(it, best, cands, media_type, root))
name = it.get('title') or it.get('show_title') or '?'
if media_type == 'episode':
name = "%s S%02dE%02d" % (name, int(it.get('season_number') or 0),
int(it.get('episode_number') or 0))
# tell apart: grabbed / search-didn't-run / source-empty / hits-but-all-rejected.
if ok:
msg, lt = "Grabbed '%s'" % name, 'success'
elif didnt_run:
msg = ("Search didn't run for '%s' — slskd error: %s" % (name, err)) if err \
else ("Search didn't run for '%s' — slskd not responding?" % name)
lt = 'warning'
elif not cands:
msg, lt = "No search results for '%s'" % name, 'info'
else:
why = (cands[0].get('rejected') or 'none met your quality profile')
msg, lt = "%d result(s) for '%s', none accepted — %s" % (len(cands), name, why), 'info'
with lock:
searched[0] += 1
if ok:
grabbed[0] += 1
elif didnt_run:
notrun[0] += 1
elif not cands:
noresults[0] += 1
else:
rejected[0] += 1
deps.update_progress(
automation_id, phase='Searching + grabbing…',
progress=10 + int(85 * searched[0] / max(total, 1)),
log_line=msg, log_type=lt)
with ThreadPoolExecutor(max_workers=concurrency) as ex:
list(ex.map(_one, todo))
# Headline with the WHY breakdown: it's the difference between "the source has
# nothing" (noresults) and "it has stuff but your quality profile rejects it" (rejected).
tail = []
if notrun[0]:
tail.append("%d search(es) didn't run (slskd?)" % notrun[0])
if noresults[0]:
tail.append('%d had no results' % noresults[0])
if rejected[0]:
tail.append('%d rejected on quality' % rejected[0])
breakdown = (' · ' + ', '.join(tail)) if tail else ''
done = ('Grabbed %d %s(s) of %d searched%s' % (grabbed[0], label, searched[0], breakdown)) if grabbed[0] \
else ('Searched %d %s(s), grabbed 0%s' % (searched[0], label, breakdown))
deps.update_progress(automation_id, status='finished', progress=100, phase='Complete',
log_line=done, log_type='success' if grabbed[0] else 'info')
return {'status': 'completed', 'searched': searched[0], 'grabbed': grabbed[0],
'noresults': noresults[0], 'rejected': rejected[0], 'notrun': notrun[0],
'_manages_own_progress': True}
except Exception as e: # noqa: BLE001
deps.update_progress(automation_id, status='error', phase='Error', log_line=str(e), log_type='error')
return {'status': 'error', 'error': str(e), '_manages_own_progress': True}
finally:
_running[media_type] = False

View file

@ -0,0 +1,200 @@
"""Automation handler: ``video_process_youtube_wishlist`` action.
The drain side of the YouTube fulfillment lane. The watchlist-channels scan keeps the
wishlist fed; THIS queues wished videos for download and keeps a few flowing at a time.
There is NO cap on how much of the wishlist gets processed it queues the WHOLE thing.
The only limit is how many download SIMULTANEOUSLY (``max_concurrent``, default 3): every
wished video becomes a ``queued`` row in the shared ``video_downloads`` table, the handler
starts up to the limit, and each finished download starts the next (one-out-one-in, in the
worker) so the entire queue drains in a controlled stream. This mirrors the music side's
download worker a concurrency cap plus a small inter-download delay (handled in the
worker) to avoid yt-dlp 429s but stays on the isolated video side.
The worker (``core.video.youtube_download``) downloads organises (channel/year/date)
archives to history removes the video from the wishlist, so a completed grab leaves the
wishlist and won't be re-queued; videos already queued or downloading are skipped so re-runs
never double-grab.
Shared automation side (may import ``core.video`` / ``api.video``); owns its own progress.
All I/O is injected as seams, so selection + the pump are pure unit-testable functions.
"""
from __future__ import annotations
from typing import Any, Callable, Dict, Iterable, List, Optional
from core.automation.deps import AutomationDeps
def videos_to_enqueue(wanted: List[Dict[str, Any]], already_ids: Iterable) -> List[Dict[str, Any]]:
"""Wished videos not already queued or downloading. NO cap — the whole backlog is
queued; concurrency is bounded at start time, not here. Pure."""
already = {str(x) for x in (already_ids or ()) if x}
out: List[Dict[str, Any]] = []
for v in wanted or []:
vid = v.get("video_id")
if vid and str(vid) not in already:
out.append(v)
return out
def slots_free(running: int, max_concurrent: int) -> int:
"""How many new downloads may start now given how many are already fetching. Pure."""
return max(0, int(max_concurrent) - max(0, int(running)))
def enqueue_ctx(video: Dict[str, Any], channel_settings: Dict[str, Any]) -> Dict[str, Any]:
"""The download row's ``search_ctx``, applying per-channel overrides: a custom show-name
(the ``$channel`` folder token) and/or a quality override. Pure."""
cs = channel_settings if isinstance(channel_settings, dict) else {}
ctx = {
"channel": cs.get("custom_name") or video.get("channel_title"),
"channel_id": video.get("channel_id"), # so the drawer can open the in-app channel page
"video_title": video.get("video_title"),
"published_at": video.get("published_at"),
}
if cs.get("quality"):
ctx["quality"] = cs["quality"]
return ctx
# ── production seams ──────────────────────────────────────────────────────────
def _default_youtube_root() -> str:
from api.video import get_video_db
return get_video_db().get_setting("youtube_path") or ""
def _default_fetch_wanted() -> List[Dict[str, Any]]:
from api.video import get_video_db
return get_video_db().youtube_wishlist_to_download()
def _default_active_ids() -> List[Any]:
from api.video import get_video_db
return [d.get("media_id") for d in get_video_db().get_active_video_downloads()
if d.get("source") == "youtube" and d.get("media_id")]
def _default_running_count() -> int:
from api.video import get_video_db
return get_video_db().count_active_youtube_downloads()
def _default_enqueue(video: Dict[str, Any], root: str) -> Any:
"""Create a QUEUED download row (no thread spawned here — the pump starts it). Returns
the row id."""
import json
from api.video import get_video_db
from core.video.sources import resolve_video_server
db = get_video_db()
ctx = enqueue_ctx(video, db.get_channel_settings(video.get("channel_id")))
ctx["server_source"] = resolve_video_server()
return db.add_video_download({
"kind": "youtube", "source": "youtube", "media_source": "youtube",
"title": video.get("video_title") or video.get("channel_title"),
"media_id": video.get("video_id"), "target_dir": root, "status": "queued",
"year": video.get("published_at"), "poster_url": video.get("thumbnail_url"),
"search_ctx": json.dumps(ctx),
})
def _default_start_next() -> Any:
"""Claim + start the next queued YouTube download (or None). The worker chains the rest."""
from api.video import get_video_db
from core.video.youtube_download import start_next_queued
return start_next_queued(get_video_db)
def _default_reap() -> int:
"""Recover downloads orphaned by a restart (stuck 'downloading', no live worker)."""
from api.video import get_video_db
from core.video.youtube_download import requeue_orphaned_youtube
return requeue_orphaned_youtube(get_video_db)
def auto_video_process_youtube_wishlist(
config: Dict[str, Any],
deps: AutomationDeps,
*,
youtube_root: Optional[Callable[[], str]] = None,
fetch_wanted: Optional[Callable[[], List[Dict[str, Any]]]] = None,
active_ids: Optional[Callable[[], Iterable]] = None,
running_count: Optional[Callable[[], int]] = None,
enqueue: Optional[Callable[[Dict[str, Any], str], Any]] = None,
start_next: Optional[Callable[[], Any]] = None,
reap: Optional[Callable[[], int]] = None,
) -> Dict[str, Any]:
"""Queue the whole YouTube wishlist for download and start up to ``max_concurrent`` now.
Returns ``{'status': 'completed', 'queued': int, 'started': int, 'running': int, ...}``."""
youtube_root = youtube_root or _default_youtube_root
fetch_wanted = fetch_wanted or _default_fetch_wanted
active_ids = active_ids or _default_active_ids
running_count = running_count or _default_running_count
enqueue = enqueue or _default_enqueue
start_next = start_next or _default_start_next
reap = reap or _default_reap
automation_id = config.get('_automation_id')
max_concurrent = max(1, int(config.get('max_concurrent', 3) or 3))
try:
root = youtube_root()
if not root:
# Always-on automation: a missing folder isn't a failure, it's "not set up for
# YouTube" — skip quietly so non-YouTube users don't see a recurring error.
deps.update_progress(automation_id, status='finished', progress=100, phase='Complete',
log_line='YouTube library folder not set — skipping (Settings → Downloads)',
log_type='info')
return {'status': 'completed', 'queued': 0, 'started': 0, 'running': 0,
'skipped': 'no_youtube_folder', '_manages_own_progress': True}
# Recover any downloads orphaned by a restart (stuck 'downloading', no worker) so
# they don't wedge the concurrency count — they go back to 'queued' and re-run.
recovered = int(reap() or 0)
if recovered:
deps.update_progress(automation_id, log_type='info',
log_line='Recovered %d stalled download(s) from a restart' % recovered)
deps.update_progress(automation_id, phase='Checking the YouTube wishlist…', progress=15,
log_line='Queueing new videos for download', log_type='info')
wanted = fetch_wanted() or []
already = list(active_ids() or [])
new = videos_to_enqueue(wanted, already)
queued = 0
for v in new:
try:
if enqueue(v, root) is not None:
queued += 1
except Exception: # noqa: BLE001 - one bad enqueue shouldn't stop the rest
deps.update_progress(automation_id, log_type='warning',
log_line="Couldn't queue '%s'" % (v.get('video_title') or v.get('video_id')))
# Fill the concurrency slots now; each finished download starts the next, so the
# whole queue drains on its own from here.
deps.update_progress(automation_id, phase='Starting downloads…', progress=70,
log_line='Queued %d new video(s)' % queued, log_type='info')
started = 0
for _ in range(slots_free(running_count() or 0, max_concurrent)):
if start_next() is None:
break
started += 1
running = (running_count() or 0)
if queued or started:
done = 'Queued %d new · %d downloading now (the rest drain automatically)' % (queued, running)
log_type = 'success'
elif running:
done = '%d already downloading; nothing new to queue' % running
log_type = 'info'
else:
done = 'No wished YouTube videos to download'
log_type = 'info'
deps.update_progress(automation_id, status='finished', progress=100, phase='Complete',
log_line=done, log_type=log_type)
return {'status': 'completed', 'queued': queued, 'started': started, 'running': running,
'_manages_own_progress': True}
except Exception as e: # noqa: BLE001
deps.update_progress(automation_id, status='error', phase='Error', log_line=str(e), log_type='error')
return {'status': 'error', 'error': str(e), '_manages_own_progress': True}

View file

@ -0,0 +1,90 @@
"""Automation handler: ``video_refresh_airing_schedules`` action.
Keeps the TV calendar honest. The "Wishlist Episodes Airing Today" automation reads the
LOCAL ``episodes`` table which is only refreshed from TMDB on initial enrichment, a manual
re-match, or lazily when you open a show's page. So a newly-announced or rescheduled episode
can sit unknown indefinitely, and the airing automation misses it.
This runs daily (a couple hours before the airing run) and re-pulls each still-airing
watchlist show's season/episode schedule from TMDB (air dates, stills, overviews) so the
calendar is current when the airing automation reads it.
Scope: the EFFECTIVE watchlist's continuing LIBRARY shows (follows airing library shows,
skipping ended/canceled they'll never air again, and skipping tmdb-only follows, which have
no episodes table to refresh). Like the other video handlers it owns its progress reporting
(``_manages_own_progress``); the show fetch + per-show refresh are injected seams, so the
handler is a pure function tests drive with fakes.
"""
from __future__ import annotations
from typing import Any, Callable, Dict, List, Optional
from core.automation.deps import AutomationDeps
def _default_fetch_shows() -> List[Dict[str, Any]]:
"""Production wiring: the still-airing watchlist shows that live in the library."""
from api.video import get_video_db
from core.video.sources import resolve_video_server
return get_video_db().watchlist_continuing_shows(resolve_video_server())
def _default_refresh_show(library_id: Any) -> Dict[str, Any]:
"""Re-pull a library show's TMDB season/episode schedule (the lazy on-view backfill,
invoked deliberately). Re-matches + cascades episodes, so air dates/stills refresh.
``with_ratings=False`` we only need schedules, and the per-show OMDb ratings call
would burn the daily quota across a whole watchlist."""
from core.video.enrichment.engine import get_video_enrichment_engine
return get_video_enrichment_engine().refresh_show_art(library_id, with_ratings=False) or {}
def auto_video_refresh_airing_schedules(
config: Dict[str, Any],
deps: AutomationDeps,
*,
fetch_shows: Optional[Callable[[], List[Dict[str, Any]]]] = None,
refresh_show: Optional[Callable[[Any], Dict[str, Any]]] = None,
) -> Dict[str, Any]:
"""Refresh the TMDB episode schedule for every still-airing watchlist library show, so
the airing automation's calendar read is current.
Returns ``{'status': 'completed', 'refreshed': int, 'failed': int, 'shows': int, ...}``."""
fetch_shows = fetch_shows or _default_fetch_shows
refresh_show = refresh_show or _default_refresh_show
automation_id = config.get('_automation_id')
try:
deps.update_progress(automation_id, phase='Finding shows to refresh…', progress=8,
log_line='Reading your watchlist for still-airing shows', log_type='info')
shows = fetch_shows() or []
total = len(shows)
if not total:
deps.update_progress(automation_id, status='finished', progress=100, phase='Complete',
log_line='No airing shows on your watchlist to refresh', log_type='success')
return {'status': 'completed', 'refreshed': 0, 'failed': 0, 'shows': 0,
'_manages_own_progress': True}
refreshed = failed = 0
for i, s in enumerate(shows):
title = s.get('title') or ('show %s' % s.get('library_id'))
deps.update_progress(
automation_id, phase='Refreshing TV schedules…', progress=10 + int(i / total * 85),
log_line="Pulling the latest episodes for '%s' (%d/%d)" % (title, i + 1, total),
log_type='info')
try:
ok = bool((refresh_show(s.get('library_id')) or {}).get('ok'))
except Exception: # noqa: BLE001 - one show failing must not stop the rest
ok = False
if ok:
refreshed += 1
else:
failed += 1
done = 'Refreshed %d show schedule(s)' % refreshed + (' · %d failed' % failed if failed else '')
deps.update_progress(automation_id, status='finished', progress=100, phase='Complete',
log_line=done, log_type='success')
return {'status': 'completed', 'refreshed': refreshed, 'failed': failed, 'shows': total,
'_manages_own_progress': True}
except Exception as e: # noqa: BLE001
deps.update_progress(automation_id, status='error', phase='Error', log_line=str(e), log_type='error')
return {'status': 'error', 'error': str(e), '_manages_own_progress': True}

View file

@ -0,0 +1,424 @@
"""Automation handler: ``video_scan_library`` action.
The VIDEO twin of music's ``scan_library``. It does two things, in order:
1. Tells the active media server (Plex/Jellyfin) to rescan ONLY the
user's selected VIDEO sections (the movie + TV libraries chosen in
Settings) never the music library.
2. Reads the server's current state into ``video.db`` so freshly-added
media shows up as owned on the video side.
Both run through injected seams (``server_refresh`` / ``run_video_scan``)
so the handler stays a pure function: production lazily binds the real
``core.video`` functions; tests pass fakes and never spin up Flask, a DB,
or a media-server client.
ISOLATION NOTE: this handler lives on the SHARED automation side, so it
is allowed to import ``core.video`` the isolation contract only forbids
``core/video`` and ``api/video`` from importing the MUSIC side, not the
other way round. Video core stays import-clean; the bridge lives here.
Like ``scan_library``, the handler owns its own progress reporting
(``_manages_own_progress: True``) so the engine doesn't stomp the live
phase string with a generic 'completed' label.
"""
from __future__ import annotations
import time
from typing import Any, Callable, Dict, Optional
from core.automation.deps import AutomationDeps
def _default_server_refresh(media_type: str = "all") -> Dict[str, Any]:
"""Production wiring: nudge the media server to rescan its video sections.
``media_type`` scopes it to the Movie or TV section; 'all' nudges both."""
from core.video.sources import refresh_video_server_sections
return refresh_video_server_sections(media_type)
def _default_run_video_scan(mode: str, media_type: str = "all") -> Dict[str, Any]:
"""Production wiring: read the server into video.db (blocking). ``media_type``
scopes it to one library ('movie' / 'show'); 'all' does both."""
from api.video import get_video_db
from core.video.scanner import get_video_scanner
from core.video.sources import get_active_video_source
return get_video_scanner(get_video_db()).scan_sync(get_active_video_source, mode, media_type)
def auto_video_scan_library(
config: Dict[str, Any],
deps: AutomationDeps,
*,
server_refresh: Optional[Callable[[], Dict[str, Any]]] = None,
run_video_scan: Optional[Callable[..., Dict[str, Any]]] = None,
) -> Dict[str, Any]:
"""Trigger a server-side video rescan, then mirror the result into video.db.
``config['media_type']`` scopes the scan to one library 'movie' or 'show'
(TV); 'all' (default) does both. Movies and TV are independent libraries, so
the Movie scan never touches TV and vice-versa.
Returns one of:
- ``{'status': 'completed', '_manages_own_progress': True, 'movies': .., 'shows': .., 'episodes': ..}``
- ``{'status': 'skipped', 'reason': '...'}`` (another scan was already running)
- ``{'status': 'error', 'error': '...', '_manages_own_progress': True}``
"""
server_refresh = server_refresh or _default_server_refresh
run_video_scan = run_video_scan or _default_run_video_scan
automation_id = config.get('_automation_id')
# 'full' is the safe default — upsert everything, never prune. 'deep' prunes
# what the server no longer has; only use it when the config asks explicitly.
mode = config.get('mode') or 'full'
media_type = config.get('media_type') or 'all'
lib_label = {'movie': 'Movie', 'show': 'TV'}.get(media_type, 'video')
try:
deps.update_progress(
automation_id,
phase=f'Asking media server to rescan {lib_label} sections...',
progress=10,
log_line=f'Triggering server-side {lib_label} scan',
log_type='info',
)
# Step 1 — best-effort server nudge, scoped to the same library we're about
# to read. A server that can't be triggered (none configured, or an adapter
# without refresh support) is surfaced as a warning, NOT a hard failure: the
# read below still mirrors whatever the server currently reports.
refresh = server_refresh(media_type) or {}
if not refresh.get('ok'):
deps.update_progress(
automation_id,
log_line='Server scan trigger unavailable: ' + str(refresh.get('error') or 'unknown'),
log_type='warning',
)
else:
sections = refresh.get('sections')
detail = str(sections) + ' video section(s)' if sections else 'selected video section(s)'
deps.update_progress(
automation_id,
progress=30,
log_line='Server rescanning ' + detail,
log_type='success',
)
# Step 2 — read the server into video.db (blocking; mirrors music's
# scan handler blocking until the scan resolves).
deps.update_progress(
automation_id,
phase=f'Reading {lib_label} library into SoulSync...',
progress=45,
)
result = run_video_scan(mode, media_type) or {}
state = result.get('state')
# Singleton scanner already busy with another scan (e.g. the Movie + TV
# deep scans firing close together) — skip cleanly, don't error.
if state == 'in_progress':
deps.update_progress(
automation_id, status='finished', phase='Skipped',
log_line='Another video scan is already running — skipping this run',
log_type='info',
)
return {'status': 'skipped', 'reason': 'a video scan is already running',
'_manages_own_progress': True}
if state == 'error':
err = result.get('error') or 'Video library scan failed'
deps.update_progress(
automation_id,
status='error',
phase='Error',
log_line=err,
log_type='error',
)
return {'status': 'error', 'error': err, '_manages_own_progress': True}
movies = int(result.get('movies', 0) or 0)
shows = int(result.get('shows', 0) or 0)
episodes = int(result.get('episodes', 0) or 0)
# Summary names only the scanned library so a TV scan doesn't read "0 movies".
if media_type == 'movie':
summary = f'Movie library scanned: {movies} movies'
elif media_type == 'show':
summary = f'TV library scanned: {shows} shows, {episodes} episodes'
else:
summary = f'Video library scanned: {movies} movies, {shows} shows, {episodes} episodes'
deps.update_progress(
automation_id,
status='finished',
progress=100,
phase='Complete',
log_line=summary,
log_type='success',
)
return {
'status': 'completed',
'_manages_own_progress': True,
'movies': movies,
'shows': shows,
'episodes': episodes,
}
except Exception as e: # noqa: BLE001 — automation handlers must never raise into the engine
deps.update_progress(
automation_id,
status='error',
phase='Error',
log_line=str(e),
log_type='error',
)
return {'status': 'error', 'error': str(e), '_manages_own_progress': True}
# ── post-download chain (video twin of music's scan_library → start_database_update) ──
# Split into two stages so the calendar/library reflect a download promptly, mirroring
# the music side: tell the server to rescan, WAIT until its scan queue is actually idle
# (a big library can take 10-20 min — a fixed wait would read too early), then fire
# 'video_library_scan_completed'; stage 2 listens for that and reads the fresh state.
def _default_scan_status(media_type: str = "all"):
"""Production wiring: True/False if the active server is/ isn't scanning, or None
when it can't report (so the caller falls back to a fixed wait)."""
from core.video.sources import video_server_scan_in_progress
return video_server_scan_in_progress(media_type)
def _default_latest_completed(media_type: str):
"""The newest completed grab of a type — the probe target for the smart skip."""
from api.video import get_video_db
return get_video_db().latest_completed_download(media_type)
def _default_server_has_item(media_type: str, item) -> bool:
"""Does the active server already have this specific grab indexed?"""
from core.video.sources import video_server_has_item
return video_server_has_item(media_type, item)
def wait_for_server_scan(scan_status, sleep, *, grace_seconds: int = 15,
interval_seconds: int = 10, cap_seconds: int = 3600,
fallback_seconds: int = 120) -> int:
"""Block until the media server's scan queue goes idle, then return ~seconds waited.
``scan_status()`` returns True (scanning), False (idle) or None (can't tell). After
a short grace so the just-triggered scan has time to register as running poll
every ``interval_seconds`` until idle or ``cap_seconds`` (a generous backstop for a
huge library). If the server can't report its state, fall back to a fixed
``fallback_seconds`` wait exactly the old behaviour. ``sleep`` is injected and
elapsed is counted from the sleeps, so tests never actually block."""
grace = max(0, grace_seconds)
if grace:
sleep(grace)
waited = grace
try:
status = scan_status()
except Exception: # noqa: BLE001 - status is best-effort
status = None
if status is None:
extra = max(0, fallback_seconds - waited)
if extra:
sleep(extra)
return waited + extra
while status is True and waited < cap_seconds:
sleep(interval_seconds)
waited += interval_seconds
try:
status = scan_status()
except Exception: # noqa: BLE001
status = None
if status is None: # lost the ability to tell — stop waiting, don't hang
break
return waited
def probe_present_libraries(candidates, has_item, sleep, *, grace_seconds: int = 120,
interval_seconds: int = 15):
"""Which libraries does the server ALREADY have the newest grab for?
``candidates`` = [(scope, item), ]. A freshly-dropped file takes ~1-2 min to show
up even with the server's auto-scan ON, so we POLL: check each candidate, and give
the server up to ``grace_seconds`` (re-checking every ``interval_seconds``) to
ingest before giving up on it. Returns the set of scopes confirmed present ( skip
their crawl); anything still missing when grace expires stays out ( gets crawled).
A probe that errors is treated as 'not present' (conservative). ``sleep`` injected."""
pending = list(candidates or [])
present = set()
waited = 0
while True:
still = []
for scope, item in pending:
try:
if has_item(scope, item):
present.add(scope)
continue
except Exception: # noqa: BLE001, S110 - uncertainty → keep probing, then scan
pass
still.append((scope, item))
pending = still
if not pending or waited >= grace_seconds:
return present
sleep(interval_seconds)
waited += interval_seconds
_LIB = {'movie': 'Movie', 'show': 'TV'}
def auto_video_scan_server(
config: Dict[str, Any],
deps: AutomationDeps,
*,
server_refresh: Optional[Callable[..., Dict[str, Any]]] = None,
sleep: Optional[Callable[[float], None]] = None,
emit: Optional[Callable[[str, Dict[str, Any]], None]] = None,
scan_status: Optional[Callable[..., Any]] = None,
latest_completed: Optional[Callable[[str], Any]] = None,
server_has_item: Optional[Callable[[str, Any], bool]] = None,
) -> Dict[str, Any]:
"""Stage 1: get the server to index our new downloads, then fire
'video_library_scan_completed' so the DB-update twin reads the fresh state.
SMART SKIP (``skip_if_present``, default on): scanning is expensive, and many
servers auto-ingest new files. So per library, we first probe whether the server
already has the NEWEST grab of that type (from download history) if it does, it
already picked everything up, and we skip that library's crawl. Only libraries the
server is missing get the (expensive) rescan + poll-until-idle. We always emit so
stage 2 still READS the new items into video.db. (Video twin of music's scan_library.)"""
server_refresh = server_refresh or _default_server_refresh
sleep = sleep or time.sleep
emit = emit or deps.engine.emit
scan_status = scan_status or _default_scan_status
latest_completed = latest_completed or _default_latest_completed
server_has_item = server_has_item or _default_server_has_item
automation_id = config.get('_automation_id')
media_type = config.get('media_type') or 'all'
skip_if_present = config.get('skip_if_present', True)
try:
fallback = int(config.get('debounce_seconds') or 120)
except (TypeError, ValueError):
fallback = 120
try:
cap = int(config.get('max_wait_minutes') or 60) * 60
except (TypeError, ValueError):
cap = 3600
try:
_pg = config.get('probe_grace_minutes') # 0 is valid (probe once, no wait)
grace = max(0, int(float(_pg if _pg is not None else 2) * 60))
except (TypeError, ValueError):
grace = 120
try:
# Which libraries actually need the expensive crawl? Skip any the server already
# has the newest grab for (it auto-ingested → nothing to find). Fresh drops take
# ~1-2 min to appear even with auto-scan ON, so POLL the probe over a grace
# window instead of checking once immediately (which would always miss).
scopes = ['movie', 'show'] if media_type == 'all' else \
([media_type] if media_type in ('movie', 'show') else ['movie', 'show'])
present = set()
if skip_if_present:
candidates = []
for sc in scopes:
try:
latest = latest_completed(sc)
except Exception: # noqa: BLE001
latest = None
if latest:
candidates.append((sc, latest))
if candidates:
deps.update_progress(automation_id, progress=15,
phase='Checking whether the server already has the new downloads…',
log_line='Probing the server (giving its auto-scan time to ingest)…',
log_type='info')
present = probe_present_libraries(candidates, server_has_item, sleep,
grace_seconds=grace)
for sc in present:
deps.update_progress(
automation_id, log_line=f'{_LIB[sc]} library: server already has the newest grab — skipping its scan',
log_type='info')
to_scan = [sc for sc in scopes if sc not in present]
waited, server_name = 0, ''
if to_scan:
scan_scope = 'all' if set(to_scan) == {'movie', 'show'} else to_scan[0]
lib_label = _LIB.get(scan_scope, 'video')
deps.update_progress(automation_id, phase=f'Asking media server to rescan {lib_label}', progress=20,
log_line=f'Triggering server-side {lib_label} scan', log_type='info')
refresh = server_refresh(scan_scope) or {}
if not refresh.get('ok'):
deps.update_progress(automation_id,
log_line='Server scan trigger unavailable: ' + str(refresh.get('error') or 'unknown'),
log_type='warning')
server_name = refresh.get('server') or ''
deps.update_progress(automation_id, phase='Waiting for the server to finish indexing…', progress=55)
waited = wait_for_server_scan(lambda: scan_status(scan_scope), sleep,
fallback_seconds=fallback, cap_seconds=cap)
else:
deps.update_progress(automation_id, progress=55,
log_line='Server already has all the new downloads — no scan needed',
log_type='success')
# Always emit with the ORIGINAL scope so stage 2 reads everything we grabbed.
emit('video_library_scan_completed', {'server': server_name, 'media_type': media_type})
done = (f'Server scan finished (~{waited}s) — updating the database' if to_scan
else 'Skipped server scan — updating the database')
deps.update_progress(automation_id, status='finished', progress=100, phase='Complete',
log_line=done, log_type='success')
return {'status': 'completed', '_manages_own_progress': True,
'scanned': to_scan, 'skipped': [s for s in scopes if s not in to_scan]}
except Exception as e: # noqa: BLE001
deps.update_progress(automation_id, status='error', phase='Error', log_line=str(e), log_type='error')
return {'status': 'error', 'error': str(e), '_manages_own_progress': True}
def auto_video_update_database(
config: Dict[str, Any],
deps: AutomationDeps,
*,
run_video_scan: Optional[Callable[..., Dict[str, Any]]] = None,
) -> Dict[str, Any]:
"""Stage 2: read the (now-rescanned) server into video.db — INCREMENTAL by default
(newest-first, stop after N consecutive known). Video twin of start_database_update.
``media_type`` scopes it to the Movie or TV library ('all' does both). When fired by
the post-download chain it inherits the scope of the scan that ran (carried on the
event), so a TV-only rescan updates only TV."""
run_video_scan = run_video_scan or _default_run_video_scan
automation_id = config.get('_automation_id')
mode = config.get('mode') or 'incremental'
media_type = (config.get('media_type')
or (config.get('_event_data') or {}).get('media_type')
or 'all')
lib_label = {'movie': 'Movie', 'show': 'TV'}.get(media_type, 'video')
# 'deep'/'full' re-read the whole library; 'incremental' only grabs new items.
phase = (f'Re-reading the {lib_label} library from the server…' if mode in ('deep', 'full')
else f'Reading new {lib_label} media into SoulSync…')
try:
deps.update_progress(automation_id, phase=phase, progress=40)
result = run_video_scan(mode, media_type) or {}
if result.get('state') == 'in_progress':
deps.update_progress(automation_id, status='finished', phase='Skipped',
log_line='Another video scan is already running — skipping this run',
log_type='info')
return {'status': 'skipped', 'reason': 'a video scan is already running',
'_manages_own_progress': True}
if result.get('state') == 'error':
err = result.get('error') or 'Video database update failed'
deps.update_progress(automation_id, status='error', phase='Error', log_line=err, log_type='error')
return {'status': 'error', 'error': err, '_manages_own_progress': True}
movies = int(result.get('movies', 0) or 0)
shows = int(result.get('shows', 0) or 0)
episodes = int(result.get('episodes', 0) or 0)
if media_type == 'movie':
summary = f'Video database updated: {movies} movies'
elif media_type == 'show':
summary = f'Video database updated: {shows} shows, {episodes} episodes'
else:
summary = f'Video database updated: {movies} movies, {shows} shows, {episodes} episodes'
deps.update_progress(automation_id, status='finished', progress=100, phase='Complete',
log_line=summary, log_type='success')
return {'status': 'completed', '_manages_own_progress': True,
'movies': movies, 'shows': shows, 'episodes': episodes}
except Exception as e: # noqa: BLE001
deps.update_progress(automation_id, status='error', phase='Error', log_line=str(e), log_type='error')
return {'status': 'error', 'error': str(e), '_manages_own_progress': True}

View file

@ -0,0 +1,235 @@
"""Automation handler: ``video_scan_watchlist_channels`` action.
The "what's new from the channels I follow" scan the piece that lets SoulSync replace
ytdl-sub-style YouTube auto-downloaders. For every YouTube channel on the video watchlist,
look at its recent uploads and wishlist the new long-form videos the user doesn't already
have, so the (future) fulfillment engine can grab them.
This runs on a SHORT schedule (channels publish at all hours pair it with a 6-hourly
Schedule trigger; 3h is fine too, the scan is cheap). It's forward-looking and dup-proof:
* **Baseline = follow time.** What the user had before following isn't our concern — only
uploads published on/after they followed the channel (the watchlist row's ``date_added``)
are "new" and get wishlisted, forever forward.
* **Last-N safety net.** A per-channel default (10) reaches a little BEFORE the baseline so
the user is always kept current on the most recent videos even right after following / if
a scan was missed. Global setting now; per-channel override (the hover settings modal,
like watchlist-artist settings) comes later.
* **Long-form only.** Shorts are excluded (the channel's Videos tab + a duration floor);
livestreams/premieres that haven't aired are skipped (future-dated).
* **Never duplicates.** Each candidate is diffed against what's already wishlisted, what's
already been downloaded (the permanent download-history ledger empty for YouTube until
the fulfillment engine lands, wired here so it's correct the day it does), and what the
user has dismissed.
Like the other video handlers it lives on the SHARED automation side (may import
``core.video`` / ``api.video`` the isolation contract only forbids the reverse) and owns
its own progress (``_manages_own_progress``). All I/O is injected as seams, so the selection
logic is a pure, unit-testable function; production lazily binds the real calls.
NOTE (follow-up): "remove from the YouTube wishlist" currently just deletes the row, so the
last-N net could re-add a video the user deliberately removed. The scan already respects a
``dismissed_ids`` seam wiring the wishlist-remove to record a dismissal (once the wishlist
is a real download queue) closes that loop. Tracked, out of scope for this scan-only phase.
"""
from __future__ import annotations
from datetime import date
from typing import Any, Callable, Dict, Iterable, List, Optional
from core.automation.deps import AutomationDeps
# ── pure selection ────────────────────────────────────────────────────────────
def is_short(video: Dict[str, Any], min_seconds: int) -> bool:
"""A YouTube Short — a known duration under the floor. Unknown duration is NOT
assumed short (the Videos-tab listing already excludes Shorts; this is a backstop)."""
d = video.get("duration_seconds")
return isinstance(d, (int, float)) and 0 < d < min_seconds
def long_form_uploads(uploads: Iterable, min_seconds: int) -> List[Dict[str, Any]]:
"""The channel's real videos, newest-first order preserved: drop Shorts + entries
with no id."""
return [v for v in (uploads or [])
if isinstance(v, dict) and v.get("youtube_id") and not is_short(v, min_seconds)]
def _day(value: Any) -> Optional[str]:
s = str(value or "").strip()
return s[:10] or None
def select_channel_video_gaps(
uploads: List[Dict[str, Any]],
*,
baseline_date: Optional[str],
backfill_count: int,
wishlisted_ids: Iterable = (),
dismissed_ids: Iterable = (),
downloaded_ids: Iterable = (),
today: str,
min_seconds: int = 60,
) -> List[Dict[str, Any]]:
"""The pure core: which of a channel's recent uploads to wishlist now.
``uploads`` is the channel's recent videos newest-first (rich shape: youtube_id, title,
published_at, duration_seconds, thumbnail_url, ). Keeps a long-form upload if it's not
already wishlisted / downloaded / dismissed AND either it's within the newest
``backfill_count`` (the safety net) or it was published on/after ``baseline_date`` (the
forward-looking part). Future-dated (unaired premiere/stream) is skipped. No I/O."""
longs = long_form_uploads(uploads, min_seconds)
n = max(0, int(backfill_count or 0))
net_ids = {v["youtube_id"] for v in longs[:n]}
excluded = set(wishlisted_ids or ()) | set(dismissed_ids or ()) | set(downloaded_ids or ())
out: List[Dict[str, Any]] = []
for v in longs:
vid = v["youtube_id"]
if vid in excluded:
continue
d = _day(v.get("published_at"))
if d and today and d > today:
continue # scheduled / unaired — not out yet
if vid in net_ids or (d and baseline_date and d >= baseline_date):
out.append(v)
return out
# ── production seams ──────────────────────────────────────────────────────────
def _default_fetch_channels() -> List[Dict[str, Any]]:
from api.video import get_video_db
return get_video_db().list_watchlist_channels()
def _default_fetch_uploads(channel_id: Any, limit: int) -> List[Dict[str, Any]]:
"""A channel's recent uploads (Videos tab → long-form, with durations) with upload
dates merged on from the cache + a cheap RSS pull the same composition the channel
detail endpoint uses. Caches any dates it learns so later scans get them free."""
from api.video import get_video_db
from core.video import youtube as yt
db = get_video_db()
cid = str(channel_id)
channel = yt.resolve_channel("https://www.youtube.com/channel/" + cid,
limit=max(1, min(90, int(limit)))) or {}
vids = channel.get("videos") or []
ids = [v.get("youtube_id") for v in vids if v.get("youtube_id")]
dates = db.get_video_dates(ids) or {}
try:
dates.update(yt.channel_recent_dates(cid) or {})
except Exception: # noqa: BLE001, S110 - RSS is best-effort; cached dates still apply
pass
for v in vids:
if not v.get("published_at") and dates.get(v.get("youtube_id")):
v["published_at"] = dates[v["youtube_id"]]
try:
db.cache_video_dates([{"youtube_id": v["youtube_id"], "published_at": v.get("published_at")}
for v in vids if v.get("published_at")])
except Exception: # noqa: BLE001, S110 - caching is opportunistic
pass
return vids
def _default_wishlisted_ids(channel_id: Any) -> List[Any]:
from api.video import get_video_db
return get_video_db().wishlisted_video_ids_for_channel(channel_id)
def _default_dismissed_ids(channel_id: Any) -> List[Any]:
# No YouTube "dismissed" store yet (video_ignored is movie/show-level). Returns empty;
# wiring wishlist-remove → dismiss is the tracked follow-up (see module docstring).
return []
def _default_downloaded_ids(channel_id: Any) -> List[Any]:
# Already-downloaded YouTube videos (global; a video id is unique). A completed download
# leaves the wishlist, so without this the scan would re-add + re-download it.
from api.video import get_video_db
return get_video_db().downloaded_youtube_video_ids()
def _default_add_videos(channel: Dict[str, Any], videos: List[Dict[str, Any]]) -> int:
from api.video import get_video_db
from core.video.sources import resolve_video_server
return get_video_db().add_videos_to_wishlist(channel, videos, server_source=resolve_video_server())
def auto_video_scan_watchlist_channels(
config: Dict[str, Any],
deps: AutomationDeps,
*,
fetch_channels: Optional[Callable[[], List[Dict[str, Any]]]] = None,
fetch_uploads: Optional[Callable[[Any, int], List[Dict[str, Any]]]] = None,
wishlisted_ids: Optional[Callable[[Any], Iterable]] = None,
dismissed_ids: Optional[Callable[[Any], Iterable]] = None,
downloaded_ids: Optional[Callable[[Any], Iterable]] = None,
add_videos: Optional[Callable[[Dict[str, Any], List[Dict[str, Any]]], int]] = None,
today_fn: Optional[Callable[[], str]] = None,
) -> Dict[str, Any]:
"""Scan every followed YouTube channel and wishlist its new long-form uploads.
Returns ``{'status': 'completed', 'channels': int, 'videos_added': int, ...}``."""
fetch_channels = fetch_channels or _default_fetch_channels
fetch_uploads = fetch_uploads or _default_fetch_uploads
wishlisted_ids = wishlisted_ids or _default_wishlisted_ids
dismissed_ids = dismissed_ids or _default_dismissed_ids
downloaded_ids = downloaded_ids or _default_downloaded_ids
add_videos = add_videos or _default_add_videos
today_fn = today_fn or (lambda: date.today().isoformat())
automation_id = config.get('_automation_id')
backfill = max(0, int(config.get('backfill_count', 10) or 0))
min_seconds = max(0, int(config.get('min_seconds', 60) or 0))
limit = max(backfill, 30)
try:
today = today_fn()
deps.update_progress(automation_id, phase='Reading your watchlist…', progress=5,
log_line='Loading the channels you follow', log_type='info')
channels = fetch_channels() or []
if not channels:
deps.update_progress(automation_id, status='finished', progress=100, phase='Complete',
log_line='No channels on the watchlist to scan', log_type='info')
return {'status': 'completed', 'channels': 0, 'videos_added': 0,
'_manages_own_progress': True}
added = 0
total = len(channels)
for i, ch in enumerate(channels):
cid = ch.get('youtube_id')
ctitle = ch.get('title') or cid
deps.update_progress(automation_id, phase='Scanning channels…',
progress=10 + int(80 * i / max(total, 1)),
log_line="Checking %s for new videos" % ctitle, log_type='info')
if not cid:
continue
try:
uploads = fetch_uploads(cid, limit) or []
except Exception: # noqa: BLE001 - one flaky channel shouldn't abort the scan
deps.update_progress(automation_id, log_line="Couldn't reach %s — skipping" % ctitle,
log_type='warning')
continue
baseline = _day(ch.get('date_added')) or today
gaps = select_channel_video_gaps(
uploads, baseline_date=baseline, backfill_count=backfill,
wishlisted_ids=wishlisted_ids(cid), dismissed_ids=dismissed_ids(cid),
downloaded_ids=downloaded_ids(cid), today=today, min_seconds=min_seconds)
if gaps:
n = int(add_videos({'youtube_id': cid, 'title': ctitle,
'avatar_url': ch.get('poster_url')}, gaps) or 0)
added += n
if n:
deps.update_progress(
automation_id, log_type='success',
log_line="Wishlisted %d new video(s) from %s" % (n, ctitle))
done = ('Wishlisted %d new video(s) across %d channel(s)' % (added, total)) if added \
else ('Channels are up to date — nothing new across %d channel(s)' % total)
deps.update_progress(automation_id, status='finished', progress=100, phase='Complete',
log_line=done, log_type='success')
return {'status': 'completed', 'channels': total, 'videos_added': added,
'_manages_own_progress': True}
except Exception as e: # noqa: BLE001
deps.update_progress(automation_id, status='error', phase='Error', log_line=str(e), log_type='error')
return {'status': 'error', 'error': str(e), '_manages_own_progress': True}

View file

@ -0,0 +1,296 @@
"""Automation handler: ``video_scan_watchlist_people`` action.
The video "watchlist" follows ongoing *things* shows, channels, and people. For shows
the ``video_add_airing_episodes`` automation already keeps the wishlist fed. This handler
does the equivalent for the PEOPLE you follow:
For every person on the watchlist, look up their filmography and wishlist every MOVIE the
user doesn't already own — the whole back catalog they acted in or directed, plus anything
upcoming. The first run is the meaty one (it backlogs everything); later runs are fast
because they skip movies already on the wishlist and only promote ones that have since been
released.
Design decisions (Boulder):
* MOVIES ONLY no TV episodes for a person (shows are handled by their own automation).
* Both ACTOR (Acting credits, minus "playing themselves") and DIRECTOR credits count.
* UNRELEASED movies are added as ``status='monitored'`` so the wishlist/download engine
leaves them alone until they're out; a later scan PROMOTES them to ``'wanted'``.
* Best-in-class UX: grab as much data as possible at add time (backdrop, overview, genres,
runtime, rating, top cast, director, release date, + provenance "because you follow X")
and stash it as a rich ``detail_json`` blob on the wishlist row, so the UI renders a full
card later without re-fetching.
Like the other video handlers this lives on the SHARED automation side (so it may import
``core.video`` / ``api.video`` the isolation contract only forbids the reverse) and owns
its own progress (``_manages_own_progress``). All I/O is injected as seams, so the logic is
a pure function in tests (no DB, no TMDB); production lazily binds the real calls.
"""
from __future__ import annotations
from datetime import date
from typing import Any, Callable, Dict, Iterable, List, Optional
from core.automation.deps import AutomationDeps
from core.video.discovery_gaps import filmography_gaps
# ── pure credit classification ────────────────────────────────────────────────
# An actor "playing themselves" in a documentary / talk show / award broadcast isn't a
# dramatic role — drop those so the wishlist stays films they actually acted in.
_SELF_MARKERS = ('self', 'himself', 'herself', 'themselves', 'archive footage',
'archival footage')
def is_self_credit(role) -> bool:
"""True for a 'plays themselves' / archive-footage credit (role text only)."""
r = str(role or '').strip().lower()
return bool(r) and any(m in r for m in _SELF_MARKERS)
def is_actor_movie_credit(c: Dict[str, Any]) -> bool:
return (c.get('kind') == 'movie'
and str(c.get('department') or '') == 'Acting'
and not is_self_credit(c.get('role')))
def is_director_movie_credit(c: Dict[str, Any]) -> bool:
return c.get('kind') == 'movie' and str(c.get('role') or '').strip().lower() == 'director'
def is_relevant_movie_credit(c: Any) -> bool:
"""A movie this person ACTED in (not as themselves) or DIRECTED."""
return isinstance(c, dict) and (is_actor_movie_credit(c) or is_director_movie_credit(c))
def is_released(date_str: Any, today: str) -> bool:
"""True if the release date is on/before ``today``. No date → NOT yet released, so we
monitor it (a later scan promotes once a real, past date arrives)."""
d = str(date_str or '').strip()
return bool(d) and d[:10] <= today
def _int_set(values: Iterable) -> set:
out = set()
for x in values or []:
try:
out.add(int(x))
except (TypeError, ValueError):
continue
return out
def select_person_movie_gaps(credits: List[Dict[str, Any]], owned_ids: Iterable,
ignored_ids: Iterable, *, today: str) -> List[Dict[str, Any]]:
"""The pure core: a followed person's un-owned actor/director MOVIE credits.
Keeps only relevant movie credits, drops owned + ignored + duplicates, ranks by
popularity (via ``filmography_gaps``), and tags each with the wishlist ``_status`` it
should get ``'wanted'`` if released, else ``'monitored'``. No I/O."""
relevant = [c for c in (credits or []) if is_relevant_movie_credit(c)]
ignored = _int_set(ignored_ids)
out: List[Dict[str, Any]] = []
for g in filmography_gaps(owned_ids, relevant, kinds=("movie",)):
if int(g['tmdb_id']) in ignored:
continue
g = dict(g)
g['_status'] = 'wanted' if is_released(g.get('date'), today) else 'monitored'
out.append(g)
return out
def build_detail_blob(detail: Optional[Dict[str, Any]], credit: Dict[str, Any],
person: Dict[str, Any]) -> Dict[str, Any]:
"""Trim TMDB full-detail down to a rich-but-lean card blob + provenance.
Drops the heavy ``_extras`` (similar/recommendations/keywords/reviews/providers) and
keeps what a wishlist card / detail view wants. Degrades to the credit's own fields when
``detail`` is missing or a library redirect."""
via = {
'person_tmdb_id': person.get('tmdb_id'),
'person_name': person.get('title') or person.get('name'),
'role': credit.get('role'),
'as': 'director' if is_director_movie_credit(credit) else 'actor',
}
if not detail or detail.get('redirect'):
return {
'title': credit.get('title'), 'year': credit.get('year'),
'release_date': credit.get('date'), 'poster_url': credit.get('poster'),
'added_via': via,
}
director = next((p.get('name') for p in (detail.get('crew') or [])
if str(p.get('job')) == 'Director'), None)
return {
'title': detail.get('title'),
'overview': detail.get('overview'),
'tagline': detail.get('tagline'),
'status': detail.get('status'),
'rating': detail.get('rating'),
'imdb_id': detail.get('imdb_id'),
'poster_url': detail.get('poster_url') or credit.get('poster'),
'backdrop_url': detail.get('backdrop_url'),
'logo': detail.get('logo'),
'genres': detail.get('genres') or [],
'runtime_minutes': detail.get('runtime_minutes'),
'studio': detail.get('studio'),
'year': detail.get('year') or credit.get('year'),
'release_date': detail.get('release_date') or credit.get('date'),
'cast': (detail.get('cast') or [])[:15],
'director': director,
'added_via': via,
}
# ── production seams ──────────────────────────────────────────────────────────
def _default_fetch_people() -> List[Dict[str, Any]]:
from api.video import get_video_db
return get_video_db().list_watchlist('person')
def _default_fetch_credits(tmdb_id: Any) -> List[Dict[str, Any]]:
from core.video.enrichment.engine import get_video_enrichment_engine
d = get_video_enrichment_engine().person_detail(tmdb_id) or {}
return d.get('credits') or []
def _default_fetch_detail(tmdb_id: Any) -> Optional[Dict[str, Any]]:
from core.video.enrichment.engine import get_video_enrichment_engine
return get_video_enrichment_engine().tmdb_detail('movie', tmdb_id)
def _default_owned_ids() -> Iterable:
from api.video import get_video_db
from core.video.sources import resolve_video_server
return get_video_db().owned_movie_tmdb_ids(resolve_video_server())
def _default_ignored_ids() -> List[Any]:
from api.video import get_video_db
return [r.get('tmdb_id') for r in (get_video_db().list_ignored() or [])
if r.get('kind') == 'movie']
def _default_wishlisted_status() -> Dict[int, str]:
from api.video import get_video_db
return get_video_db().wishlisted_movie_status()
def _default_add_movie(tmdb_id, title, *, year, poster_url, status, detail_json) -> bool:
from api.video import get_video_db
from core.video.sources import resolve_video_server
return get_video_db().add_movie_to_wishlist(
tmdb_id, title, year=year, poster_url=poster_url, status=status,
detail_json=detail_json, server_source=resolve_video_server())
def auto_video_scan_watchlist_people(
config: Dict[str, Any],
deps: AutomationDeps,
*,
fetch_people: Optional[Callable[[], List[Dict[str, Any]]]] = None,
fetch_credits: Optional[Callable[[Any], List[Dict[str, Any]]]] = None,
fetch_detail: Optional[Callable[[Any], Optional[Dict[str, Any]]]] = None,
owned_ids: Optional[Callable[[], Iterable]] = None,
ignored_ids: Optional[Callable[[], List[Any]]] = None,
wishlisted_status: Optional[Callable[[], Dict[int, str]]] = None,
add_movie: Optional[Callable[..., bool]] = None,
today_fn: Optional[Callable[[], str]] = None,
) -> Dict[str, Any]:
"""Scan every followed person's filmography and wishlist the movies the user is missing.
Returns ``{'status': 'completed', 'people': int, 'movies_added': int, 'upcoming': int,
'promoted': int, ...}`` ``movies_added`` = released wishlisted, ``upcoming`` = monitored
(unreleased) wishlisted, ``promoted`` = monitored rows flipped to wanted now they're out."""
fetch_people = fetch_people or _default_fetch_people
fetch_credits = fetch_credits or _default_fetch_credits
fetch_detail = fetch_detail or _default_fetch_detail
owned_ids = owned_ids or _default_owned_ids
ignored_ids = ignored_ids or _default_ignored_ids
wishlisted_status = wishlisted_status or _default_wishlisted_status
add_movie = add_movie or _default_add_movie
today_fn = today_fn or (lambda: date.today().isoformat())
automation_id = config.get('_automation_id')
try:
today = today_fn()
deps.update_progress(automation_id, phase='Reading your watchlist…', progress=5,
log_line='Loading the people you follow', log_type='info')
people = fetch_people() or []
if not people:
deps.update_progress(automation_id, status='finished', progress=100, phase='Complete',
log_line='No people on the watchlist to scan', log_type='info')
return {'status': 'completed', 'people': 0, 'movies_added': 0, 'upcoming': 0,
'promoted': 0, '_manages_own_progress': True}
owned = owned_ids() or set()
ignored = ignored_ids() or []
# Snapshot of what's already wishlisted {tmdb_id: status}; updated as we add so a
# movie credited to two followed people is handled once.
wished: Dict[int, str] = dict(wishlisted_status() or {})
added = upcoming = promoted = 0
total = len(people)
for i, person in enumerate(people):
pid = person.get('tmdb_id')
pname = person.get('title') or person.get('name') or pid
deps.update_progress(automation_id, phase='Scanning filmographies…',
progress=10 + int(80 * i / max(total, 1)),
log_line="Looking up %s's movies" % pname, log_type='info')
if not pid:
continue
try:
credits = fetch_credits(pid) or []
except Exception: # noqa: BLE001 - one bad lookup shouldn't abort the whole scan
deps.update_progress(automation_id, log_line="Couldn't fetch %s — skipping" % pname,
log_type='warning')
continue
for g in select_person_movie_gaps(credits, owned, ignored, today=today):
tid = int(g['tmdb_id'])
want = g['_status'] # 'wanted' | 'monitored'
existing = wished.get(tid)
if existing is not None:
# Already wishlisted — the only action left is promoting a monitored
# row that has since been released (don't re-fetch detail for it).
if existing == 'monitored' and want == 'wanted':
if add_movie(tid, g.get('title'), year=g.get('year'),
poster_url=g.get('poster'), status='wanted', detail_json=None):
wished[tid] = 'wanted'
promoted += 1
deps.update_progress(
automation_id, log_type='success',
log_line="'%s' is out now — promoted to wanted"
% (g.get('title') or tid))
continue
# New gap — grab the rich detail (cached), build the blob, add it.
try:
detail = fetch_detail(tid)
except Exception: # noqa: BLE001 - degrade to the cheap credit fields
detail = None
blob = build_detail_blob(detail, g, person)
if add_movie(tid, g.get('title') or blob.get('title'), year=g.get('year'),
poster_url=blob.get('poster_url') or g.get('poster'),
status=want, detail_json=blob):
wished[tid] = want
if want == 'wanted':
added += 1
else:
upcoming += 1
parts = []
if added:
parts.append('%d new movie(s)' % added)
if upcoming:
parts.append('%d upcoming' % upcoming)
if promoted:
parts.append('%d promoted' % promoted)
done = ('Wishlisted ' + ', '.join(parts) + ' across %d followed person(s)' % total) \
if parts else ('Watchlist is up to date — nothing new across %d person(s)' % total)
deps.update_progress(automation_id, status='finished', progress=100, phase='Complete',
log_line=done, log_type='success')
return {'status': 'completed', 'people': total, 'movies_added': added,
'upcoming': upcoming, 'promoted': promoted, '_manages_own_progress': True}
except Exception as e: # noqa: BLE001
deps.update_progress(automation_id, status='error', phase='Error', log_line=str(e), log_type='error')
return {'status': 'error', 'error': str(e), '_manages_own_progress': True}

View file

@ -0,0 +1,179 @@
"""Automation handler: ``video_scan_watchlist_playlists`` action.
Sibling of the watchlist-CHANNELS scan, with a deliberately different rule. A channel is a
creator posting new stuff over time, so that scan is forward-looking (new uploads + a
last-N net). A **playlist** is a curated, finite set someone assembled the reason you
follow it is "give me this whole list and keep it complete." So this scan MIRRORS the
playlist: it wishlists every long-form video in it you don't already have, plus anything
later added. No follow-date baseline, no last-N net.
Organisation: the playlist becomes its own "show" (playlist-as-show) its videos are
wishlisted under the playlist's title, so the download worker files them as
``Playlist Name / Season YEAR / Playlist Name - DATE - Title`` (matches the ytdl-sub
``tv_show_name``-on-a-playlist convention). All other plumbing is shared with channels: the
same wishlist rows, the same "Process YouTube Wishlist" drain, quality + org template.
Edge: a video in both a followed channel AND a followed playlist is wishlisted once (dedup
by video id); whichever scan touched it last sets its show-name. Accepted.
Pure selection with all I/O injected; shared automation side; owns its own progress.
"""
from __future__ import annotations
from datetime import date
from typing import Any, Callable, Dict, Iterable, List, Optional
from core.automation.deps import AutomationDeps
from core.automation.handlers.video_scan_watchlist_channels import _day, long_form_uploads
# how many playlist entries to read per scan (yt-dlp flat). Big enough for almost any real
# playlist; genuinely huge ones truncate (logged) rather than hammering YouTube each scan.
_PLAYLIST_FETCH_LIMIT = 1000
def select_playlist_video_gaps(
videos: List[Dict[str, Any]],
*,
wishlisted_ids: Iterable = (),
downloaded_ids: Iterable = (),
dismissed_ids: Iterable = (),
today: str,
min_seconds: int = 60,
) -> List[Dict[str, Any]]:
"""The pure core: every long-form video in a playlist not already wishlisted /
downloaded / dismissed (mirror the whole list). Future-dated (unaired) entries are
skipped. No baseline, no last-N a curated playlist is wanted in full. No I/O."""
excluded = set(wishlisted_ids or ()) | set(downloaded_ids or ()) | set(dismissed_ids or ())
out: List[Dict[str, Any]] = []
for v in long_form_uploads(videos, min_seconds):
vid = v["youtube_id"]
if vid in excluded:
continue
d = _day(v.get("published_at"))
if d and today and d > today:
continue # unaired premiere — not out yet
out.append(v)
return out
# ── production seams ──────────────────────────────────────────────────────────
def _default_fetch_playlists() -> List[Dict[str, Any]]:
from api.video import get_video_db
return get_video_db().list_watchlist_playlists()
def _default_fetch_videos(playlist_id: Any) -> List[Dict[str, Any]]:
"""A playlist's videos (flat, with durations) with upload dates merged from the cache.
Playlists have no per-channel RSS, so dates come from whatever's cached (filled over
time by the channel/InnerTube enrichers); undated entries organise cleanly."""
from api.video import get_video_db
from core.video import youtube as yt
db = get_video_db()
vids = yt.playlist_videos(str(playlist_id), limit=_PLAYLIST_FETCH_LIMIT) or []
ids = [v.get("youtube_id") for v in vids if v.get("youtube_id")]
dates = db.get_video_dates(ids) or {}
for v in vids:
if not v.get("published_at") and dates.get(v.get("youtube_id")):
v["published_at"] = dates[v["youtube_id"]]
return vids
def _default_wishlisted_ids(playlist_id: Any) -> List[Any]:
# parent_source_id holds the playlist id for playlist-sourced wishlist videos.
from api.video import get_video_db
return get_video_db().wishlisted_video_ids_for_channel(playlist_id)
def _default_downloaded_ids(playlist_id: Any) -> List[Any]:
# Already-downloaded YouTube videos (global). A completed download leaves the wishlist,
# so without this the scan would re-add + re-download it.
from api.video import get_video_db
return get_video_db().downloaded_youtube_video_ids()
def _default_dismissed_ids(playlist_id: Any) -> List[Any]:
return [] # no youtube "dismissed" store yet (see channels-scan note)
def _default_add_videos(playlist: Dict[str, Any], videos: List[Dict[str, Any]]) -> int:
"""Wishlist under the PLAYLIST as the show (title = playlist name → playlist-as-show)."""
from api.video import get_video_db
from core.video.sources import resolve_video_server
return get_video_db().add_videos_to_wishlist(playlist, videos, server_source=resolve_video_server())
def auto_video_scan_watchlist_playlists(
config: Dict[str, Any],
deps: AutomationDeps,
*,
fetch_playlists: Optional[Callable[[], List[Dict[str, Any]]]] = None,
fetch_videos: Optional[Callable[[Any], List[Dict[str, Any]]]] = None,
wishlisted_ids: Optional[Callable[[Any], Iterable]] = None,
downloaded_ids: Optional[Callable[[Any], Iterable]] = None,
dismissed_ids: Optional[Callable[[Any], Iterable]] = None,
add_videos: Optional[Callable[[Dict[str, Any], List[Dict[str, Any]]], int]] = None,
today_fn: Optional[Callable[[], str]] = None,
) -> Dict[str, Any]:
"""Mirror every followed YouTube playlist into the wishlist (whole list + new additions).
Returns ``{'status': 'completed', 'playlists': int, 'videos_added': int, ...}``."""
fetch_playlists = fetch_playlists or _default_fetch_playlists
fetch_videos = fetch_videos or _default_fetch_videos
wishlisted_ids = wishlisted_ids or _default_wishlisted_ids
downloaded_ids = downloaded_ids or _default_downloaded_ids
dismissed_ids = dismissed_ids or _default_dismissed_ids
add_videos = add_videos or _default_add_videos
today_fn = today_fn or (lambda: date.today().isoformat())
automation_id = config.get('_automation_id')
min_seconds = max(0, int(config.get('min_seconds', 60) or 0))
try:
today = today_fn()
deps.update_progress(automation_id, phase='Reading your watchlist…', progress=5,
log_line='Loading the playlists you follow', log_type='info')
playlists = fetch_playlists() or []
if not playlists:
deps.update_progress(automation_id, status='finished', progress=100, phase='Complete',
log_line='No playlists on the watchlist to scan', log_type='info')
return {'status': 'completed', 'playlists': 0, 'videos_added': 0,
'_manages_own_progress': True}
added = 0
total = len(playlists)
for i, pl in enumerate(playlists):
pid = pl.get('playlist_id')
ptitle = pl.get('title') or pid
deps.update_progress(automation_id, phase='Scanning playlists…',
progress=10 + int(80 * i / max(total, 1)),
log_line="Mirroring '%s'" % ptitle, log_type='info')
if not pid:
continue
try:
videos = fetch_videos(pid) or []
except Exception: # noqa: BLE001 - one flaky playlist shouldn't abort the scan
deps.update_progress(automation_id, log_line="Couldn't reach '%s' — skipping" % ptitle,
log_type='warning')
continue
gaps = select_playlist_video_gaps(
videos, wishlisted_ids=wishlisted_ids(pid), downloaded_ids=downloaded_ids(pid),
dismissed_ids=dismissed_ids(pid), today=today, min_seconds=min_seconds)
if gaps:
n = int(add_videos({'youtube_id': pid, 'title': ptitle,
'avatar_url': pl.get('poster_url')}, gaps) or 0)
added += n
if n:
deps.update_progress(
automation_id, log_type='success',
log_line="Wishlisted %d new video(s) from '%s'" % (n, ptitle))
done = ('Wishlisted %d new video(s) across %d playlist(s)' % (added, total)) if added \
else ('Playlists are up to date — nothing new across %d playlist(s)' % total)
deps.update_progress(automation_id, status='finished', progress=100, phase='Complete',
log_line=done, log_type='success')
return {'status': 'completed', 'playlists': total, 'videos_added': added,
'_manages_own_progress': True}
except Exception as e: # noqa: BLE001
deps.update_progress(automation_id, status='error', phase='Error', log_line=str(e), log_type='error')
return {'status': 'error', 'error': str(e), '_manages_own_progress': True}

View file

@ -145,6 +145,186 @@ SYSTEM_AUTOMATIONS = [
'action_type': 'full_cleanup', 'action_type': 'full_cleanup',
'initial_delay': 900, # 15 min after startup 'initial_delay': 900, # 15 min after startup
}, },
# ── Video side (isolated app, shared engine) ──────────────────────────
# owned_by='video' keeps these OFF the music automations page (it filters
# them out) and ON the video Automations page (it shows only these).
# Schedule-based for now; a video-download-complete event trigger can
# replace the schedule once that event is wired into the engine.
# (No standalone scheduled scan — the post-download chain below keeps the library
# fresh. The 'video_scan_library' action/block still exist for a custom automation.)
# Post-download chain (video twin of music's batch_complete → scan_library →
# library_scan_completed → start_database_update). Event-based, so a finished
# video download refreshes the server then pulls the new media into video.db.
{
'name': 'Auto-Scan Video After Downloads',
'trigger_type': 'video_batch_complete',
'trigger_config': {},
'action_type': 'video_scan_server',
'owned_by': 'video',
},
{
'name': 'Auto-Update Video Database After Scan',
'trigger_type': 'video_library_scan_completed',
'trigger_config': {},
'action_type': 'video_update_database',
'action_config': {'mode': 'incremental'},
'owned_by': 'video',
},
# Safety net: re-read the server hourly too, so MANUAL library additions (which Plex
# auto-scans) appear within the hour instead of waiting for the weekly deep scan. Same
# cheap incremental read as the after-scan one.
{
'name': 'Auto-Update Video Database (Hourly)',
'trigger_type': 'schedule',
'trigger_config': {'interval': 1, 'unit': 'hours'},
'action_type': 'video_update_database_hourly',
'action_config': {'mode': 'incremental'},
'initial_delay': 900,
'owned_by': 'video',
},
# Sonarr-style: once a day at 1am (server-local), wishlist every episode airing
# today for the shows you follow (skipping ones already owned) so the day's episodes
# are queued overnight. A fixed wall-clock 'daily_time' (not a rolling 24h interval
# that drifts with restarts) — the seeder now arms timed system triggers, and
# _fix_airing_automation_schedule migrates the old 24h-interval row.
# Runs before the airing automation so the calendar it reads is current — re-pulls
# TMDB episode schedules for still-airing watchlist shows (the airing read is LOCAL).
{
'name': 'Refresh Airing TV Schedules',
'trigger_type': 'daily_time',
'trigger_config': {'time': '23:00'},
'action_type': 'video_refresh_airing_schedules',
'owned_by': 'video',
},
{
'name': 'Auto-Wishlist Episodes Airing Today',
'trigger_type': 'daily_time',
'trigger_config': {'time': '01:00'},
'action_type': 'video_add_airing_episodes',
'owned_by': 'video',
},
# Video twins of the music maintenance jobs — same schedule + shared handler,
# distinct action_type + owned_by='video' so they seed as separate rows and
# show on the video Automations page (music's copies are untouched).
{
'name': 'Clean Search History',
'trigger_type': 'schedule',
'trigger_config': {'interval': 1, 'unit': 'hours'},
'action_type': 'video_clean_search_history',
'initial_delay': 600,
'owned_by': 'video',
},
{
'name': 'Clean Completed Downloads',
'trigger_type': 'schedule',
'trigger_config': {'interval': 5, 'unit': 'minutes'},
'action_type': 'video_clean_completed_downloads',
'initial_delay': 300,
'owned_by': 'video',
},
{
'name': 'Full Cleanup',
'trigger_type': 'schedule',
'trigger_config': {'interval': 12, 'unit': 'hours'},
'action_type': 'video_full_cleanup',
'initial_delay': 900,
'owned_by': 'video',
},
{
'name': 'Auto-Backup Database',
'trigger_type': 'schedule',
'trigger_config': {'interval': 3, 'unit': 'days'},
'action_type': 'video_backup_database',
'initial_delay': 600,
'owned_by': 'video',
},
# Video twin of music's 'Auto-Deep Scan Library', split into TWO because Movies
# and TV are independent libraries — a TV scan never pulls in new movies and
# vice-versa. Fixed weekly deep scan (re-read + prune removed) at 02:00 server-
# local: TV Mondays, Movies Tuesdays — different days so they never overlap. The
# seeder arms timed system triggers; _fix_deep_scan_schedules migrates the
# original rolling-7-day rows. (Busy guard in the scanner is still a safety net.)
{
'name': 'Auto-Deep Scan TV Library',
'trigger_type': 'weekly_time',
'trigger_config': {'time': '02:00', 'days': ['mon']},
'action_type': 'video_deep_scan_tv',
'action_config': {'mode': 'deep', 'media_type': 'show'},
'owned_by': 'video',
},
{
'name': 'Auto-Deep Scan Movie Library',
'trigger_type': 'weekly_time',
'trigger_config': {'time': '02:00', 'days': ['tue']},
'action_type': 'video_deep_scan_movies',
'action_config': {'mode': 'deep', 'media_type': 'movie'},
'owned_by': 'video',
},
# ── Watchlist → Wishlist pipeline ─────────────────────────────────────────
# Stage 1: SCANS that FILL the wishlist from what you follow. (The airing-episodes
# scan above is the show equivalent.) All no-op cleanly if you follow nobody.
{
'name': 'Auto-Scan Watchlist People', # followed actors/directors → wished movies
'trigger_type': 'daily_time', # daily; filmographies change slowly
'trigger_config': {'time': '03:00'}, # after airing/deep-scan jobs, no overlap
'action_type': 'video_scan_watchlist_people',
'owned_by': 'video',
},
{
'name': 'Auto-Scan Watchlist Channels', # followed YouTube channels → wished videos
'trigger_type': 'schedule',
'trigger_config': {'interval': 6, 'unit': 'hours'}, # YouTube posts at all hours
'action_type': 'video_scan_watchlist_channels',
'action_config': {'backfill_count': 10},
'initial_delay': 1200,
'owned_by': 'video',
},
{
'name': 'Auto-Scan Watchlist Playlists', # followed playlists (mirror-all) → wished videos
'trigger_type': 'schedule',
'trigger_config': {'interval': 6, 'unit': 'hours'},
'action_type': 'video_scan_watchlist_playlists',
'initial_delay': 1320,
'owned_by': 'video',
},
# Stage 2: PROCESSORS that DRAIN the wishlist by downloading. Hourly; each skips
# quietly until its library folder (+ slskd, for movie/episode) is configured.
{
'name': 'Auto-Process Movie Wishlist', # wished movies → slskd search/pick/grab
'trigger_type': 'schedule',
'trigger_config': {'interval': 1, 'unit': 'hours'},
'action_type': 'video_process_movie_wishlist',
'action_config': {'max_concurrent': 3},
'initial_delay': 1620,
'owned_by': 'video',
},
{
'name': 'Auto-Process Episode Wishlist', # wished episodes → slskd search/pick/grab
'trigger_type': 'schedule',
'trigger_config': {'interval': 1, 'unit': 'hours'},
'action_type': 'video_process_episode_wishlist',
'action_config': {'max_concurrent': 3},
'initial_delay': 1740,
'owned_by': 'video',
},
{
'name': 'Auto-Process YouTube Wishlist', # wished YouTube videos → yt-dlp download
'trigger_type': 'schedule',
'trigger_config': {'interval': 1, 'unit': 'hours'},
'action_type': 'video_process_youtube_wishlist',
'action_config': {'max_concurrent': 3},
'initial_delay': 1500,
'owned_by': 'video',
},
# YouTube retention: delete channel episodes outside each channel's keep window. No-op
# unless a channel opts in (cog modal → Keep); default keeps everything, so safe to seed.
{
'name': 'Auto-Clean Old YouTube Episodes',
'trigger_type': 'daily_time',
'trigger_config': {'time': '04:00'},
'action_type': 'video_clean_youtube_episodes',
'owned_by': 'video',
},
] ]
@ -237,8 +417,11 @@ class AutomationEngine:
trigger_type=spec['trigger_type'], trigger_type=spec['trigger_type'],
trigger_config=json.dumps(spec['trigger_config']), trigger_config=json.dumps(spec['trigger_config']),
action_type=spec['action_type'], action_type=spec['action_type'],
action_config='{}', action_config=json.dumps(spec.get('action_config', {})),
profile_id=1, profile_id=1,
# owned_by tags the side that owns this automation (e.g.
# 'video'), so the music page can exclude another side's rows.
owned_by=spec.get('owned_by'),
) )
if aid: if aid:
self.db.update_automation(aid, is_system=1) self.db.update_automation(aid, is_system=1)
@ -284,7 +467,117 @@ class AutomationEngine:
self.db.update_automation(existing['id'], next_run=nr) self.db.update_automation(existing['id'], next_run=nr)
logger.info(f"System automation '{spec['name']}' next_run set to {initial_delay}s from now") logger.info(f"System automation '{spec['name']}' next_run set to {initial_delay}s from now")
else: else:
logger.info(f"System automation '{spec['name']}' ready (event-based)") # No initial_delay. A timer-based timed trigger (daily/weekly/
# monthly at a fixed wall-clock time) still needs its next_run
# armed — compute it from the schedule. Genuinely event-based
# triggers (batch_complete, scan_done) have no next_run, so
# next_run_at returns None and we leave them alone. Don't clobber
# an existing future next_run (manual edit / restart-resume).
nr_dt = next_run_at(spec['trigger_type'], spec['trigger_config'],
now_utc=_utcnow(), default_tz=self._default_tz)
if nr_dt is not None and not existing.get('next_run'):
self.db.update_automation(existing['id'], next_run=_dt_to_db_str(nr_dt))
logger.info(f"System automation '{spec['name']}' next_run armed for {_dt_to_db_str(nr_dt)} (timed)")
else:
logger.info(f"System automation '{spec['name']}' ready (event-based)")
self._fix_video_scan_default()
self._fix_airing_automation_schedule()
self._fix_deep_scan_schedules()
self._fix_wishlist_processor_rename()
def _fix_video_scan_default(self):
"""Remove the obsolete standalone 'Scan Video Library' SYSTEM automation — it's
superseded by the post-download chain (Auto-Scan Video After Downloads
Auto-Update Video Database After Scan).
``get_system_automation_by_action`` matches ONLY a system-seeded row
(is_system=1), so a user's own scan automation is never touched. Idempotent —
safe to run on every startup; once the row is gone the lookup returns None and
it no-ops. (No flag guard: the old one could latch True without ever deleting,
which is exactly why the row survived earlier 'cleanups'.)"""
try:
auto = self.db.get_system_automation_by_action('video_scan_library')
if auto:
self.db.delete_automation(auto['id'])
logger.info("Removed superseded 'Scan Video Library' system automation (id=%s)",
auto.get('id'))
except Exception:
logger.exception("video scan cleanup failed")
def _fix_wishlist_processor_rename(self):
"""Migrate the wishlist processors' 'Download''Process' rename so a DB seeded
under the old names doesn't show stale duplicates.
The movie/episode ACTIONS were renamed (``video_download_*`` ``video_process_*``),
so the old seeded rows are now orphaned (dead action_type) while the new ones reseed
alongside them delete the orphans. ``delete_automation`` refuses system rows, so
clear ``is_system`` first. The YouTube action kept its type but its label changed, so
rename that row in place. Idempotent no-ops once the DB is clean."""
try:
for dead in ('video_download_movie_wishlist', 'video_download_episode_wishlist'):
auto = self.db.get_system_automation_by_action(dead)
if auto:
self.db.update_automation(auto['id'], is_system=0) # lift the delete guard
self.db.delete_automation(auto['id'])
logger.info("Removed orphaned '%s' system automation (renamed to process, id=%s)",
auto.get('name'), auto.get('id'))
yt = self.db.get_system_automation_by_action('video_process_youtube_wishlist')
if yt and yt.get('name') == 'Auto-Download YouTube Wishlist':
self.db.update_automation(yt['id'], name='Auto-Process YouTube Wishlist')
logger.info("Renamed YouTube wishlist automation → 'Auto-Process YouTube Wishlist' (id=%s)",
yt.get('id'))
except Exception:
logger.exception("wishlist processor rename migration failed")
def _fix_airing_automation_schedule(self):
"""Migrate 'Auto-Wishlist Episodes Airing Today' from the old rolling 24h
interval to a fixed daily 1am run.
It originally shipped as a 'schedule'/24h interval because the seeder only
armed interval specs a 'daily_time' spec sat idle and never fired. The 24h
interval fires reliably but at a time that drifts with every restart (5 min
after startup, then +24h). Now that the seeder arms timed triggers, rewrite
the live row to run at a fixed 1am (better for 'today's airings' — queues the
day overnight). Matches only the is_system row; idempotent (no-op once the row
is already daily_time)."""
try:
auto = self.db.get_system_automation_by_action('video_add_airing_episodes')
if not auto or auto.get('trigger_type') == 'daily_time':
return
cfg = {'time': '01:00'}
nr_dt = next_run_at('daily_time', cfg, now_utc=_utcnow(), default_tz=self._default_tz)
self.db.update_automation(
auto['id'], trigger_type='daily_time', trigger_config=json.dumps(cfg),
next_run=_dt_to_db_str(nr_dt) if nr_dt is not None else None)
logger.info("Migrated 'Auto-Wishlist Episodes Airing Today' to a fixed daily 01:00 (id=%s)",
auto.get('id'))
except Exception:
logger.exception("airing automation schedule migration failed")
def _fix_deep_scan_schedules(self):
"""Migrate the two video deep-scan system automations from the original
rolling 7-day interval to fixed weekly times TV Mondays 02:00, Movies
Tuesdays 02:00 (different days so they never overlap). The seeder only
creates rows, never updates a drifted trigger; this rewrites the live rows.
Only converts the original interval rows (skips once trigger_type is already
weekly_time, so a hand-tuned day/time sticks). Idempotent."""
targets = {
'video_deep_scan_tv': {'time': '02:00', 'days': ['mon']},
'video_deep_scan_movies': {'time': '02:00', 'days': ['tue']},
}
for action_type, cfg in targets.items():
try:
auto = self.db.get_system_automation_by_action(action_type)
if not auto or auto.get('trigger_type') == 'weekly_time':
continue
nr_dt = next_run_at('weekly_time', cfg, now_utc=_utcnow(), default_tz=self._default_tz)
self.db.update_automation(
auto['id'], trigger_type='weekly_time', trigger_config=json.dumps(cfg),
next_run=_dt_to_db_str(nr_dt) if nr_dt is not None else None)
logger.info("Set '%s' to weekly %s %s (id=%s)", auto.get('name'),
cfg['days'], cfg['time'], auto.get('id'))
except Exception:
logger.exception("deep-scan schedule migration failed for %s", action_type)
def get_system_automation_next_run_seconds(self, action_type): def get_system_automation_next_run_seconds(self, action_type):
"""Get seconds until next run for a system automation. Returns 0 if not found or disabled.""" """Get seconds until next run for a system automation. Returns 0 if not found or disabled."""

View file

@ -1,27 +0,0 @@
"""Boot-phase guard for non-blocking container startup.
While the gunicorn worker is importing ``web_server`` (module-level client and
worker initialization), external provider API probes must not block startup.
Network validation is deferred until ``mark_boot_complete()`` runs at the end
of that import pass.
"""
from __future__ import annotations
import threading
_boot_lock = threading.Lock()
_boot_active = True
def is_boot_phase() -> bool:
"""Return True while module import must avoid blocking provider API calls."""
with _boot_lock:
return _boot_active
def mark_boot_complete() -> None:
"""End the boot phase — provider clients may perform network probes again."""
global _boot_active
with _boot_lock:
_boot_active = False

View file

@ -117,48 +117,6 @@ def _is_full_track_payload(payload: Optional[Dict[str, Any]]) -> bool:
return 'track_position' in payload and 'contributors' in payload return 'track_position' in payload and 'contributors' in payload
def resolve_album_track_positions(session, base_url, album_ids, cache=None, sleep_s=0.2):
"""Build ``{str(track_id): track_position}`` for a set of Deezer album ids.
Deezer PLAYLIST and SEARCH track objects (and even the album object's embedded
``tracks.data``) omit ``track_position`` only ``/album/<id>/tracks`` and
``/track/<id>`` carry it. So numbering playlist tracks by their playlist index
silently poisons the real album track number, which then rides onto the
downloaded file's tag. This resolves the authoritative position per album
(cache-first, best-effort a failed album just isn't in the map)."""
import time as _time
positions: Dict[str, int] = {}
for aid in album_ids:
aid = str(aid)
at_list = None
if cache:
try:
ct = cache.get_entity('deezer', 'album_tracks', aid)
if ct and ct.get('data'):
at_list = ct['data']
except Exception: # noqa: BLE001 - cache is best-effort
at_list = None
if at_list is None:
try:
if sleep_s:
_time.sleep(sleep_s) # respect Deezer rate limits
r = session.get(f"{base_url}/album/{aid}/tracks", params={'limit': 500}, timeout=10)
if getattr(r, 'ok', False):
at_list = (r.json() or {}).get('data', [])
if cache and at_list is not None:
try:
cache.store_entity('deezer', 'album_tracks', aid, {'data': at_list})
except Exception as _cache_err: # noqa: BLE001
logger.debug("album_tracks cache store failed for %s: %s", aid, _cache_err)
except Exception: # noqa: BLE001 - never let metadata resolution break the fetch
at_list = None
for at in (at_list or []):
tp = at.get('track_position')
if at.get('id') and tp:
positions[str(at['id'])] = tp
return positions
# ==================== Dataclasses (match iTunesClient / SpotifyClient format) ==================== # ==================== Dataclasses (match iTunesClient / SpotifyClient format) ====================
@dataclass @dataclass
@ -1399,16 +1357,6 @@ class DeezerClient:
raw_tracks.extend(page_tracks) raw_tracks.extend(page_tracks)
# Real album track positions — playlist tracks don't carry track_position,
# so numbering by playlist index would poison the downloaded file's tag.
album_ids = {str(t.get('album', {}).get('id')) for t in raw_tracks if t.get('album', {}).get('id')}
try:
from core.metadata.cache import get_metadata_cache
_cache = get_metadata_cache()
except Exception:
_cache = None
track_positions = resolve_album_track_positions(self.session, self.BASE_URL, album_ids, _cache)
# Normalize tracks # Normalize tracks
tracks: List[Dict[str, Any]] = [] tracks: List[Dict[str, Any]] = []
for i, t in enumerate(raw_tracks, start=1): for i, t in enumerate(raw_tracks, start=1):
@ -1420,8 +1368,7 @@ class DeezerClient:
'artists': [artist_name], 'artists': [artist_name],
'album': t.get('album', {}).get('title', ''), 'album': t.get('album', {}).get('title', ''),
'duration_ms': t.get('duration', 0) * 1000, 'duration_ms': t.get('duration', 0) * 1000,
# REAL album position; the playlist index is a last resort only. 'track_number': i,
'track_number': track_positions.get(str(t.get('id'))) or i,
}) })
result = { result = {

View file

@ -21,7 +21,6 @@ from typing import Any, Dict, List, Optional, Tuple
import requests import requests
from core.download_plugins.types import AlbumResult, DownloadStatus, TrackResult from core.download_plugins.types import AlbumResult, DownloadStatus, TrackResult
from core.quality.source_map import quality_from_deezer, quality_tier_for_source
from utils.logging_config import get_logger from utils.logging_config import get_logger
logger = get_logger("deezer_download") logger = get_logger("deezer_download")
@ -93,10 +92,7 @@ class DeezerDownloadClient(DownloadSourcePlugin):
if download_path is None: if download_path is None:
download_path = config_manager.get('soulseek.download_path', './downloads') download_path = config_manager.get('soulseek.download_path', './downloads')
self.download_path = Path(download_path) self.download_path = Path(download_path)
try: self.download_path.mkdir(parents=True, exist_ok=True)
self.download_path.mkdir(parents=True, exist_ok=True)
except OSError as e:
logger.warning(f"Could not verify download path {self.download_path}: {e}")
# Engine reference is populated by set_engine() at registration # Engine reference is populated by set_engine() at registration
# time. None until orchestrator wires the registry. # time. None until orchestrator wires the registry.
@ -121,20 +117,14 @@ class DeezerDownloadClient(DownloadSourcePlugin):
self._license_token = None self._license_token = None
self._user_data = None self._user_data = None
self._authenticated = False self._authenticated = False
self._pending_arl: Optional[str] = None
# Quality preference # Quality preference
self._quality = quality_tier_for_source('deezer', default='flac') self._quality = config_manager.get('deezer_download.quality', 'flac')
# Try to authenticate on init if ARL is configured # Try to authenticate on init if ARL is configured
arl = config_manager.get('deezer_download.arl', '') arl = config_manager.get('deezer_download.arl', '')
if arl: if arl:
from core.boot_phase import is_boot_phase self._authenticate(arl)
if is_boot_phase():
self._pending_arl = arl
logger.debug("Deezer ARL present — authentication deferred until after boot")
else:
self._authenticate(arl)
logger.info(f"Deezer download client initialized (download path: {self.download_path})") logger.info(f"Deezer download client initialized (download path: {self.download_path})")
@ -233,66 +223,12 @@ class DeezerDownloadClient(DownloadSourcePlugin):
return self._authenticated return self._authenticated
def is_authenticated(self) -> bool: def is_authenticated(self) -> bool:
if self._pending_arl and not self._authenticated:
from core.boot_phase import is_boot_phase
if not is_boot_phase():
self._authenticate(self._pending_arl)
self._pending_arl = None
return self._authenticated return self._authenticated
async def check_connection(self) -> bool: async def check_connection(self) -> bool:
loop = asyncio.get_event_loop() loop = asyncio.get_event_loop()
return await loop.run_in_executor(None, self.is_available) return await loop.run_in_executor(None, self.is_available)
# ─── Playlist export (#945) ──────────────────────────────────
#
# UNOFFICIAL: rides the private gw-light gateway with the ARL session already used
# for downloads. Deezer shut their public developer API, so this is the only write
# path — and it's fragile by nature (breaks when Deezer changes internals).
def create_or_update_playlist(self, name, track_ids, *, existing_id=None,
public=False, description=""):
"""Create a Deezer playlist (or append to an existing one) from a mirrored
playlist's tracks. ``track_ids`` are stored ``deezer_id`` values per library track.
``existing_id`` set add to that playlist (idempotent re-export reuses the stored
target); unset create a new one. Returns
``{success, playlist_id, url, added, error}``."""
if not self._authenticated:
return {"success": False, "error": "Deezer is not connected (ARL)"}
song_ids = [str(t) for t in (track_ids or []) if t]
if not song_ids:
return {"success": False, "error": "No matching Deezer tracks to export"}
try:
songs = [[sid, i] for i, sid in enumerate(song_ids)]
if existing_id:
res = self._gw_call("playlist.addSongs",
{"playlist_id": int(existing_id), "songs": songs})
if res is None:
return {"success": False, "error": "Deezer rejected the playlist update"}
playlist_id = existing_id
else:
res = self._gw_call("playlist.create", {
"title": name, "description": description,
"is_public": bool(public), "songs": songs,
})
if res is None:
return {"success": False, "error": "Deezer rejected the playlist create"}
# gw 'playlist.create' returns the new playlist id (int) as `results`.
if isinstance(res, dict):
playlist_id = res.get("PLAYLIST_ID") or res.get("id")
else:
playlist_id = res
if not playlist_id:
return {"success": False, "error": "Deezer did not return a playlist id"}
return {
"success": True,
"playlist_id": str(playlist_id),
"url": f"https://www.deezer.com/playlist/{playlist_id}",
"added": len(song_ids),
}
except Exception as e:
return {"success": False, "error": str(e)}
def reconnect(self, arl: str = None) -> bool: def reconnect(self, arl: str = None) -> bool:
"""Re-authenticate with a new or existing ARL.""" """Re-authenticate with a new or existing ARL."""
if arl is None: if arl is None:
@ -481,12 +417,6 @@ class DeezerDownloadClient(DownloadSourcePlugin):
if aid: if aid:
album_ids.add(str(aid)) album_ids.add(str(aid))
album_release_dates = {} album_release_dates = {}
# Deezer PLAYLIST tracks do NOT carry `track_position` (only `/track/<id>`
# and `/album/<id>/tracks` do), so numbering them by their playlist index
# poisons the real album track number — which then rides into the wishlist
# and onto the downloaded file's tag (e.g. 'Apologize' tagged track 1 instead
# of 16). Resolve the REAL position from each album's track list (cache-first).
track_positions: Dict[str, int] = {} # str(track_id) -> album track_position
try: try:
from core.metadata.cache import get_metadata_cache from core.metadata.cache import get_metadata_cache
cache = get_metadata_cache() cache = get_metadata_cache()
@ -499,32 +429,24 @@ class DeezerDownloadClient(DownloadSourcePlugin):
cached = cache.get_entity('deezer', 'album', aid) cached = cache.get_entity('deezer', 'album', aid)
if cached and cached.get('release_date'): if cached and cached.get('release_date'):
album_release_dates[aid] = cached['release_date'] album_release_dates[aid] = cached['release_date']
continue
except Exception as e: except Exception as e:
logger.debug("cache get_entity album release_date: %s", e) logger.debug("cache get_entity album release_date: %s", e)
# Cache miss — fetch from API # Cache miss — fetch from API
if aid not in album_release_dates: try:
try: time.sleep(0.3) # Respect rate limits
time.sleep(0.3) # Respect rate limits a_resp = self._session.get(f'https://api.deezer.com/album/{aid}', timeout=10)
a_resp = self._session.get(f'https://api.deezer.com/album/{aid}', timeout=10) if a_resp.ok:
if a_resp.ok: a_data = a_resp.json()
a_data = a_resp.json() album_release_dates[aid] = a_data.get('release_date', '')
album_release_dates[aid] = a_data.get('release_date', '') # Store in metadata cache for future use
# Store in metadata cache for future use if cache:
if cache: try:
try: cache.store_entity('deezer', 'album', aid, a_data)
cache.store_entity('deezer', 'album', aid, a_data) except Exception as e:
except Exception as e: logger.debug("cache store_entity album release_date: %s", e)
logger.debug("cache store_entity album release_date: %s", e) except Exception as e:
except Exception as e: logger.debug("fetch deezer album release_date %s: %s", aid, e)
logger.debug("fetch deezer album release_date %s: %s", aid, e)
# Real album track positions (separate endpoint — playlist tracks AND the
# album object's embedded tracks both omit track_position). Cache-first.
try:
from core.deezer_client import resolve_album_track_positions
track_positions = resolve_album_track_positions(
self._session, 'https://api.deezer.com', album_ids, cache)
except Exception as e:
logger.debug("resolve deezer album track positions: %s", e)
tracks = [] tracks = []
for i, t in enumerate(raw_tracks, start=1): for i, t in enumerate(raw_tracks, start=1):
@ -545,9 +467,7 @@ class DeezerDownloadClient(DownloadSourcePlugin):
'id': album_id, 'id': album_id,
}, },
'duration_ms': t.get('duration', 0) * 1000, 'duration_ms': t.get('duration', 0) * 1000,
# REAL album position (resolved above); the playlist index is a last 'track_number': i,
# resort only when the album lookup failed, never the default.
'track_number': track_positions.get(str(t.get('id'))) or t.get('track_position') or i,
}) })
return { return {
@ -661,7 +581,7 @@ class DeezerDownloadClient(DownloadSourcePlugin):
bitrate = 128 bitrate = 128
quality = 'mp3' quality = 'mp3'
tr = TrackResult( results.append(TrackResult(
username='deezer_dl', username='deezer_dl',
filename=f"{track_id}||{artist} - {title}", filename=f"{track_id}||{artist} - {title}",
size=est_size, size=est_size,
@ -675,10 +595,7 @@ class DeezerDownloadClient(DownloadSourcePlugin):
title=title, title=title,
album=album, album=album,
track_number=item.get('track_position'), track_number=item.get('track_position'),
) ))
# Stamp CD-quality FLAC (16/44.1) so lossless ranks correctly.
tr.set_quality(quality_from_deezer(self._quality))
results.append(tr)
logger.info(f"Deezer search for '{query}' returned {len(results)} results") logger.info(f"Deezer search for '{query}' returned {len(results)} results")
return results, [] return results, []

View file

@ -1,340 +0,0 @@
"""Listening-driven recommendation core (#913).
PURE, side-effect-free ranking that turns "the artists you listen to most" plus
"who's similar to each" into:
1. a consensus-ranked list of artists you'd probably love but don't own, and
2. an aggregated candidate-track list for a generated playlist.
No DB / network / config here. The caller (the watchlist scanner) supplies the
seeds (top-played artists), the ``similar_artists`` rows per seed, and the
owned-artist set, then fetches top tracks for the winners. Keeping the decision
logic in one pure place makes it fully unit-testable without the live stack and
keeps the scan wiring thin and additive, so it can't disturb existing flows.
Scoring rationale (the "best in class" bit): a recommended artist's score is
``Σ over the seeds that recommend it of (seed_weight × similarity)``. That single
sum rewards all three signals at once **consensus** (an artist endorsed by many
of your seeds accumulates more terms), your **play weight** (heavier seeds push
harder), and **similarity strength** instead of a flat "appears in N lists".
``seed_count`` is exposed separately for display ("because you like A, B, C") and
as the adventurousness dial's lever (``min_seed_count``).
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Dict, List, Optional, Sequence, Set
def _norm(name: object) -> str:
return str(name or "").strip().lower()
def _positive_float(value: object, default: float = 1.0) -> float:
try:
f = float(value) # type: ignore[arg-type]
except (TypeError, ValueError):
return default
return f if f > 0 else default
def _get(row: object, attr: str):
"""Read a field from a dataclass row or a dict row."""
if isinstance(row, dict):
return row.get(attr)
return getattr(row, attr, None)
def choose_mix_fetch_source(active_source: object, active_can_fetch: bool) -> str:
"""Pick which source to fetch the "Listening Mix" top tracks from.
The mix is a list of (artist, title) pairs acquired via Soulseek, so the fetch source need
NOT match the user's active metadata source. Use the active source when it can fetch top
tracks itself (Spotify/Deezer); otherwise fall back to Deezer, whose public ``artist/{id}/top``
needs no auth and is available to every user so iTunes / Discogs / MusicBrainz users still
get a full mix without switching sources. Pure.
"""
if str(active_source or "").lower() in ("spotify", "deezer") and active_can_fetch:
return str(active_source).lower()
return "deezer"
def names_match(a: object, b: object) -> bool:
"""Strict artist-name equality after stripping case + non-alphanumerics.
Used to verify a name-search result before fetching that artist's top tracks, so the
"Listening Mix" can never pull the WRONG artist's songs (e.g. a same-name act). Exact
alphanumeric match: "Tyler, The Creator" == "Tyler The Creator", but "Drake" != "Drake Bell".
Pure.
"""
def _alnum(x: object) -> str:
return "".join(ch for ch in str(x or "").lower() if ch.isalnum())
na, nb = _alnum(a), _alnum(b)
return bool(na) and na == nb
def similarity_from_rank(rank: object, max_rank: int = 10) -> float:
"""Turn a stored ``similarity_rank`` (1 = most similar … 10 = least) into a 01 weight.
SoulSync stores each ``(seed similar)`` edge with a 110 rank (``1`` is the closest
match). The ranker multiplies this into the score so a seed's *closest* matches count
for more than its long-tail ones. Linear decay over the documented range: rank 1 1.0,
rank 5 0.6, rank 10 0.1, with a 0.1 floor so a far match still contributes. A
missing/garbage rank falls back to 1.0 (treat as "no rank info, full weight"). Pure.
"""
try:
r = int(rank)
except (TypeError, ValueError):
return 1.0
floor = round(1.0 / max_rank, 4)
if r <= 1:
return 1.0
if r >= max_rank:
return floor
return round((max_rank - r + 1) / max_rank, 4)
def build_recency_weighted_seeds(
top_artists: Sequence[dict],
recent_play_counts: Optional[Dict[str, float]] = None,
*,
recency_factor: float = 1.5,
) -> List[dict]:
"""Blend lifetime + recent play counts into seed weights — "what you're into NOW".
``weight = lifetime_plays + recency_factor × recent_plays``. An artist you've played a
lot *recently* outranks one you played a lot years ago, so the recommendations track
your current taste instead of your all-time history. ``recency_factor`` is the dial
(0 = pure lifetime). Returns ``[{'name', 'weight'}]`` for :func:`rank_recommended_artists`.
Pure the caller supplies both play-count maps from the listening history.
"""
recent = {_norm(k): _positive_float(v, 0.0) for k, v in (recent_play_counts or {}).items()}
out: List[dict] = []
for a in top_artists or ():
name = str(a.get("name") or "").strip()
if not name:
continue
lifetime = _positive_float(a.get("play_count", a.get("weight", 1.0)))
boost = recency_factor * recent.get(_norm(name), 0.0)
out.append({"name": name, "weight": lifetime + boost})
return out
def group_similars_by_seed(
seeds: Sequence[dict],
similar_rows: Sequence,
id_to_name: Dict[str, str],
*,
source_id_attr: str = "source_artist_id",
similar_name_attr: str = "similar_artist_name",
rank_attr: Optional[str] = None,
) -> Dict[str, List[dict]]:
"""Reshape flat ``similar_artists`` rows into ``{seed_name_lower: [{'name', 'score'?}]}``.
The stored rows key the similar artist by the SEED's source id (``source_artist_id``),
not its name, so :func:`rank_recommended_artists` can't consume them directly. This
resolves each row's source id to a name via ``id_to_name`` (``{source_artist_id:
artist_name}`` for the library, built by the caller) and keeps only rows that resolve
to one of the ``seeds``. Rows may be dataclass objects or dicts. Pure no I/O.
``id_to_name`` MUST be keyed by whatever id the edges actually store for SoulSync that
is the artist's SOURCE id (Spotify/iTunes/Deezer/MusicBrainz), NOT the internal row id.
When ``rank_attr`` is given, each row's rank is converted via :func:`similarity_from_rank`
and carried as ``score`` so closer matches weigh more; without it every similar comes out
score-less (the ranker then treats similarity as 1.0 original behavior).
"""
seed_names = {_norm(s.get("name")) for s in seeds}
seed_names.discard("")
id_to_norm = {str(k): _norm(v) for k, v in (id_to_name or {}).items()}
out: Dict[str, List[dict]] = {}
for row in similar_rows or ():
seed_name = id_to_norm.get(str(_get(row, source_id_attr) or ""), "")
if not seed_name or seed_name not in seed_names:
continue
sim_name = str(_get(row, similar_name_attr) or "").strip()
if not sim_name:
continue
entry = {"name": sim_name}
if rank_attr is not None:
entry["score"] = similarity_from_rank(_get(row, rank_attr))
out.setdefault(seed_name, []).append(entry)
return out
@dataclass
class RecommendedArtist:
"""One artist recommended from your listening, with the why."""
name: str # display name (first-seen casing)
score: float # Σ seed_weight × similarity
seed_count: int # distinct seeds endorsing it (consensus)
seeds: List[str] = field(default_factory=list) # display names of those seeds
def rank_recommended_artists(
seeds: Sequence[dict],
similars_by_seed: Dict[str, Sequence[dict]],
owned_artist_names: Optional[Set[str]] = None,
*,
limit: int = 30,
min_seed_count: int = 1,
) -> List[RecommendedArtist]:
"""Rank artists similar to your most-played by consensus + play weight + similarity.
Args:
seeds: ``[{'name': str, 'weight': float}]`` your top-played artists.
``weight`` (play count or any positive number) defaults to 1.0.
similars_by_seed: ``{seed_name_lower: [{'name': str, 'score': float}]}`` the
similar-artist rows for each seed. ``score`` is optional (defaults 1.0).
owned_artist_names: lowercased names already in the library excluded so the
result is artists you DON'T have. The seeds themselves are always excluded.
limit: max results.
min_seed_count: drop recommendations endorsed by fewer than N seeds the
adventurousness dial's "Safer" end raises this for higher-confidence picks.
Returns up to ``limit`` :class:`RecommendedArtist`, highest score first.
"""
owned = {_norm(a) for a in (owned_artist_names or set())}
seed_norms = {_norm(s.get("name")) for s in seeds}
seed_norms.discard("")
exclude = owned | seed_norms
acc: Dict[str, dict] = {}
for seed in seeds:
s_name = _norm(seed.get("name"))
if not s_name:
continue
s_display = str(seed.get("name") or "").strip()
weight = _positive_float(seed.get("weight", 1.0))
for sim in similars_by_seed.get(s_name, ()) or ():
a_norm = _norm(sim.get("name"))
if not a_norm or a_norm in exclude:
continue
sim_score = _positive_float(sim.get("score", 1.0))
row = acc.setdefault(
a_norm, {"name": str(sim.get("name") or "").strip(), "score": 0.0, "seeds": {}}
)
row["score"] += weight * sim_score
row["seeds"].setdefault(s_name, s_display) # one seed counts once
out: List[RecommendedArtist] = []
floor = max(1, int(min_seed_count))
for row in acc.values():
seed_count = len(row["seeds"])
if seed_count < floor:
continue
out.append(RecommendedArtist(
name=row["name"],
score=round(row["score"], 6),
seed_count=seed_count,
seeds=list(row["seeds"].values()),
))
out.sort(key=lambda r: (-r.score, -r.seed_count, r.name.lower()))
return out[:limit]
def aggregate_candidate_tracks(
recommended_artists: Sequence[RecommendedArtist],
top_tracks_by_artist: Dict[str, Sequence[dict]],
owned_track_keys: Optional[Set] = None,
*,
per_artist: int = 3,
limit: int = 50,
exclude_owned: bool = True,
) -> List[dict]:
"""Build the candidate track list for the generated playlist.
Takes the top ``per_artist`` tracks from each recommended artist **in artist-rank
order**, dedups by ``(artist, title)``, optionally drops owned tracks (the
"discovery" flavor) and caps at ``limit``. Each returned track dict is the source
track plus ``_seed_artist`` (which recommended artist it came from).
Args:
recommended_artists: ranked output of :func:`rank_recommended_artists`.
top_tracks_by_artist: ``{artist_name_lower: [track_dict, ...]}`` fetched by
the caller (Last.fm / source top tracks), NOT limited to a curated pool.
owned_track_keys: set of ``(artist_lower, title_lower)`` already in the library.
exclude_owned: drop tracks in ``owned_track_keys`` (discovery flavor). Set False
for a "replay" playlist of tracks you already own.
"""
owned = owned_track_keys or set()
seen: Set = set()
out: List[dict] = []
for art in recommended_artists:
tracks = top_tracks_by_artist.get(_norm(art.name), ()) or ()
taken = 0
for t in tracks:
if taken >= per_artist:
break
title = str(t.get("name") or t.get("title") or "").strip()
if not title:
continue
key = (_norm(art.name), _norm(title))
if key in seen:
continue
if exclude_owned and key in owned:
continue
seen.add(key)
out.append({**t, "_seed_artist": art.name})
taken += 1
if len(out) >= limit:
break
return out[:limit]
def to_mix_track(track: object, source: str) -> Optional[dict]:
"""Shape one source "top tracks" API dict into the flat dict the Discover compact
playlist row renders + syncs (the "Listening Mix" #913 playlist).
Spotify's ``artist_top_tracks`` and Deezer's ``get_artist_top_tracks`` both return the
same Spotify-shape object (``id, name, artists[], album{name,images[]}, duration_ms``).
This flattens that into the renderer's field names (``track_name/artist_name/album_name/
album_cover_url/duration_ms``), keeps the original under ``track_data_json`` for sync, and
sets the source-specific id field. Returns None for anything without a usable id/title so
the caller can filter. A ``name`` key is kept so :func:`aggregate_candidate_tracks` can
dedup by title. Pure no I/O.
"""
if not isinstance(track, dict):
return None
tid = track.get("id")
name = str(track.get("name") or "").strip()
if not tid or not name:
return None
artists = track.get("artists") or []
artist_name = ""
if artists and isinstance(artists[0], dict):
artist_name = str(artists[0].get("name") or "").strip()
album = track.get("album") if isinstance(track.get("album"), dict) else {}
album_name = str(album.get("name") or "").strip()
images = album.get("images") or []
cover = images[0].get("url") if images and isinstance(images[0], dict) else None
out = {
"track_id": str(tid),
"name": name, # for aggregate_candidate_tracks dedup
"track_name": name, # for the renderer
"artist_name": artist_name,
"album_name": album_name,
"album_cover_url": cover,
"duration_ms": track.get("duration_ms") or 0,
"track_data_json": track, # full payload for sync/download
"source": source,
}
id_field = {"spotify": "spotify_track_id", "deezer": "deezer_track_id",
"itunes": "itunes_track_id"}.get(source)
if id_field:
out[id_field] = str(tid)
return out
__all__ = [
"RecommendedArtist",
"choose_mix_fetch_source",
"names_match",
"similarity_from_rank",
"build_recency_weighted_seeds",
"to_mix_track",
"group_similars_by_seed",
"rank_recommended_artists",
"aggregate_candidate_tracks",
]

View file

@ -21,11 +21,7 @@ from core.metadata.registry import get_client_for_source
from core.metadata.types import Album from core.metadata.types import Album
from core.wishlist.payloads import ensure_wishlist_track_format from core.wishlist.payloads import ensure_wishlist_track_format
# Use the project logger namespace ("soulsync.*") so the scanner's progress and logger = logging.getLogger(__name__)
# diagnostics actually surface in the app log — plain getLogger(__name__) lands
# under "core.discovery.quality_scanner", which the app log view doesn't show.
from utils.logging_config import get_logger
logger = get_logger("discovery.quality_scanner")
# Per-source typed converter dispatch — same registry pattern as # Per-source typed converter dispatch — same registry pattern as

View file

@ -173,32 +173,6 @@ async def _database_only_find_track(spotify_track, candidate_pool=None):
logger.debug("sync match cache fast-path failed: %s", e) logger.debug("sync match cache fast-path failed: %s", e)
# --- End cache fast-path --- # --- End cache fast-path ---
# Durable manual library match (#787) — survives a library rescan (the
# sync_match_cache above does not), so a user's Find & Add pairing keeps
# sticking across auto-syncs instead of being re-matched from scratch (#895
# follow-up). Self-heals a stale library id via the stored file path.
if spotify_id:
try:
from core.artists.map import get_current_profile_id
m = db.find_manual_library_match_by_source_track_id(
get_current_profile_id(), str(spotify_id), active_server)
if m:
lib_id = m.get('library_track_id')
dt = db.get_track_by_id(lib_id) if lib_id is not None else None
if not dt and m.get('library_file_path'):
new_id = db.find_track_id_by_file_path(m['library_file_path'])
dt = db.get_track_by_id(new_id) if new_id else None
if dt:
class DatabaseTrackDurable:
def __init__(self, db_t):
self.ratingKey = db_t.id
self.title = db_t.title
self.id = db_t.id
logger.debug(f"Durable manual match hit: '{original_title}'{lib_id}")
return DatabaseTrackDurable(dt), 1.0
except Exception as e:
logger.debug("durable manual match fast-path failed: %s", e)
# Try each artist # Try each artist
for artist in spotify_track.artists: for artist in spotify_track.artists:
if isinstance(artist, str): if isinstance(artist, str):
@ -307,11 +281,35 @@ def run_sync_task(
# This avoids needing to re-fetch it from Spotify # This avoids needing to re-fetch it from Spotify
logger.info("Converting JSON tracks to SpotifyTrack objects...") logger.info("Converting JSON tracks to SpotifyTrack objects...")
# Store original track data with full album objects (for wishlist with cover art). # Store original track data with full album objects (for wishlist with cover art)
# Shared with the sync-detail "re-add to wishlist" action so both build the # Normalize formats for wishlist: album must be dict {'name': ...}, artists must be [{'name': ...}]
# IDENTICAL payload (album→dict + images, artists→dicts). Copy-safe. # Important: copy data — don't mutate tracks_json since SpotifyTrack expects List[str] artists
from core.sync.wishlist_readd import build_original_tracks_map original_tracks_map = {}
original_tracks_map = build_original_tracks_map(tracks_json) for t in tracks_json:
track_id = t.get('id', '')
if track_id:
normalized = dict(t)
# Normalize album to dict format, preserving images and metadata
raw_album = normalized.get('album', '')
if isinstance(raw_album, str):
normalized['album'] = {
'name': raw_album or normalized.get('name', 'Unknown Album'),
'images': [], 'album_type': 'single', 'total_tracks': 1, 'release_date': ''
}
elif not isinstance(raw_album, dict):
normalized['album'] = {
'name': str(raw_album) if raw_album else normalized.get('name', 'Unknown Album'),
'images': [], 'album_type': 'single', 'total_tracks': 1, 'release_date': ''
}
else:
# Dict — ensure required keys exist
raw_album.setdefault('name', 'Unknown Album')
raw_album.setdefault('images', [])
# Normalize artists to list of dicts
raw_artists = normalized.get('artists', [])
if raw_artists and isinstance(raw_artists[0], str):
normalized['artists'] = [{'name': a} for a in raw_artists]
original_tracks_map[track_id] = normalized
tracks = [] tracks = []
for i, t in enumerate(tracks_json): for i, t in enumerate(tracks_json):

View file

@ -32,26 +32,6 @@ from typing import Any, Callable
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
_UNKNOWN_ARTIST = 'Unknown Artist'
def resolve_display_artist(yt_artist: str, matched_artist: str) -> str:
"""The artist to show in the 'YT Artist' column (#909).
YouTube's flat playlist data carries no artist, so a track starts as
"Unknown Artist" and only gains a real name if per-video recovery succeeds.
When recovery comes up empty but the track still matched confidently, show
the matched artist instead of a misleading "Unknown Artist". Returns the
original ``yt_artist`` whenever it's already a real name (recovery worked) or
when there's no matched artist to fall back to — purely a display choice, the
match itself is unaffected.
"""
current = (yt_artist or '').strip()
if current and current != _UNKNOWN_ARTIST:
return current # recovery already gave a real name — keep it
fallback = (matched_artist or '').strip()
return fallback or _UNKNOWN_ARTIST # backfill from the match, else honest Unknown
@dataclass @dataclass
class YoutubeDiscoveryDeps: class YoutubeDiscoveryDeps:
@ -151,15 +131,14 @@ def run_youtube_discovery_worker(url_hash, deps: YoutubeDiscoveryDeps):
cached_match = cache_db.get_discovery_cache_match(cache_key[0], cache_key[1], discovery_source) cached_match = cache_db.get_discovery_cache_match(cache_key[0], cache_key[1], discovery_source)
if cached_match and deps.validate_discovery_cache_artist(cleaned_artist, cached_match): if cached_match and deps.validate_discovery_cache_artist(cleaned_artist, cached_match):
logger.debug(f"CACHE HIT [{i+1}/{len(tracks)}]: {cleaned_artist} - {cleaned_title}") logger.debug(f"CACHE HIT [{i+1}/{len(tracks)}]: {cleaned_artist} - {cleaned_title}")
_match_artist = deps.extract_artist_name(cached_match.get('artists', [''])[0]) if cached_match.get('artists') else ''
result = { result = {
'index': i, 'index': i,
'yt_track': cleaned_title, 'yt_track': cleaned_title,
'yt_artist': resolve_display_artist(cleaned_artist, _match_artist), 'yt_artist': cleaned_artist,
'status': 'Found', 'status': 'Found',
'status_class': 'found', 'status_class': 'found',
'spotify_track': cached_match.get('name', ''), 'spotify_track': cached_match.get('name', ''),
'spotify_artist': _match_artist, 'spotify_artist': deps.extract_artist_name(cached_match.get('artists', [''])[0]) if cached_match.get('artists') else '',
'spotify_album': cached_match.get('album', {}).get('name', '') if isinstance(cached_match.get('album'), dict) else cached_match.get('album', ''), 'spotify_album': cached_match.get('album', {}).get('name', '') if isinstance(cached_match.get('album'), dict) else cached_match.get('album', ''),
'duration': f"{int(track['duration_ms']) // 60000}:{(int(track['duration_ms']) % 60000) // 1000:02d}" if track['duration_ms'] else '0:00', 'duration': f"{int(track['duration_ms']) // 60000}:{(int(track['duration_ms']) % 60000) // 1000:02d}" if track['duration_ms'] else '0:00',
'discovery_source': discovery_source, 'discovery_source': discovery_source,
@ -286,17 +265,15 @@ def run_youtube_discovery_worker(url_hash, deps: YoutubeDiscoveryDeps):
best_confidence = confidence best_confidence = confidence
logger.info(f"Strategy 4 YouTube match (extended): {match.artists[0]} - {match.name} (confidence: {confidence:.3f})") logger.info(f"Strategy 4 YouTube match (extended): {match.artists[0]} - {match.name} (confidence: {confidence:.3f})")
# Create result entry. yt_artist falls back to the matched artist when # Create result entry
# YouTube/recovery left it "Unknown Artist" but we matched confidently (#909).
_match_artist = deps.extract_artist_name(matched_track.artists[0]) if matched_track else ''
result = { result = {
'index': i, 'index': i,
'yt_track': cleaned_title, 'yt_track': cleaned_title,
'yt_artist': resolve_display_artist(cleaned_artist, _match_artist), 'yt_artist': cleaned_artist,
'status': 'Found' if matched_track else 'Not Found', 'status': 'Found' if matched_track else 'Not Found',
'status_class': 'found' if matched_track else 'not-found', 'status_class': 'found' if matched_track else 'not-found',
'spotify_track': matched_track.name if matched_track else '', 'spotify_track': matched_track.name if matched_track else '',
'spotify_artist': _match_artist, 'spotify_artist': deps.extract_artist_name(matched_track.artists[0]) if matched_track else '',
'spotify_album': matched_track.album if matched_track else '', 'spotify_album': matched_track.album if matched_track else '',
'duration': f"{int(track['duration_ms']) // 60000}:{(int(track['duration_ms']) % 60000) // 1000:02d}" if track['duration_ms'] else '0:00', 'duration': f"{int(track['duration_ms']) // 60000}:{(int(track['duration_ms']) % 60000) // 1000:02d}" if track['duration_ms'] else '0:00',
'discovery_source': discovery_source, 'discovery_source': discovery_source,

View file

@ -25,7 +25,6 @@ big-bang switchover.
from __future__ import annotations from __future__ import annotations
import asyncio
import threading import threading
from typing import Any, Dict, Iterator, List, Optional, Tuple from typing import Any, Dict, Iterator, List, Optional, Tuple
@ -392,16 +391,6 @@ class DownloadEngine:
(tracks, albums) tuple, or ``([], [])`` when every source (tracks, albums) tuple, or ``([], [])`` when every source
in the chain is exhausted. in the chain is exhausted.
Priority mode is deliberately quality-AGNOSTIC at search time source
order is king and the first source that returns any tracks wins, exactly
matching pre-quality-system behaviour byte-for-byte (#896 review #3).
Quality-gating the priority path would deprioritise e.g. a soulseek
mp3 whose bitrate slskd omitted (``bitrate=None`` "unsatisfied"),
changing which source wins and adding latency for users who never opted
in. Cross-source quality pooling is the job of best_quality mode
(``search_all_sources``); final per-result ranking still happens in the
orchestrator's match/quality filter. RAW tracks are returned.
Replaces orchestrator's hand-rolled hybrid search loop. The Replaces orchestrator's hand-rolled hybrid search loop. The
chain is ordered (most-preferred first). chain is ordered (most-preferred first).
""" """
@ -417,10 +406,9 @@ class DownloadEngine:
try: try:
logger.info(f"Trying {source_name} (priority {i+1}): {query}") logger.info(f"Trying {source_name} (priority {i+1}): {query}")
tracks, albums = await plugin.search(query, timeout, progress_callback) tracks, albums = await plugin.search(query, timeout, progress_callback)
if not tracks: if tracks:
continue logger.info(f"{source_name} found {len(tracks)} tracks")
logger.info(f"{source_name} found {len(tracks)} tracks") return (tracks, albums)
return (tracks, albums)
except Exception as e: except Exception as e:
logger.warning(f"{source_name} search failed: {e}") logger.warning(f"{source_name} search failed: {e}")
@ -430,75 +418,6 @@ class DownloadEngine:
) )
return ([], []) return ([], [])
async def search_all_sources(self, query: str, source_chain,
timeout=None, progress_callback=None,
exclude_sources=None):
"""Best-quality mode: pool RAW tracks from EVERY configured source in
``source_chain`` instead of stopping at the first satisfying one.
Unlike :meth:`search_with_fallback`, no source short-circuits the
search the caller (orchestrator/worker) ranks the combined pool
bestworst by actual audio quality. ``exclude_sources`` drops sources
whose per-source retry budget is already spent (so their candidates
never re-enter the pool). Unconfigured / unregistered / raising sources
are skipped exactly like the fallback path. Returns
``(combined_tracks, combined_albums)``.
"""
excluded = {s.lower() for s in (exclude_sources or []) if s}
pooled_tracks = []
pooled_albums = []
# Per-source contribution for an honest pool log — e.g. a release-level
# source like usenet/torrent that returns nothing for a track-title
# query should read "usenet=0", not silently hide behind the chain name.
contributions = []
# Decide which sources to actually query, recording why the rest were
# skipped. Searches then run CONCURRENTLY so the pool waits only for the
# slowest source (e.g. usenet/Prowlarr, which can be slow) rather than
# the sum of every source's latency.
to_search = [] # (source_name, plugin)
for source_name in source_chain:
if source_name.lower() in excluded:
contributions.append(f"{source_name}=excluded")
continue
plugin = self._plugins.get(source_name)
if plugin is None:
logger.info(f"Skipping {source_name} (not available)")
contributions.append(f"{source_name}=unavailable")
continue
if hasattr(plugin, 'is_configured') and not plugin.is_configured():
logger.info(f"Skipping {source_name} (not configured)")
contributions.append(f"{source_name}=unconfigured")
continue
to_search.append((source_name, plugin))
async def _one(plugin):
return await plugin.search(query, timeout, progress_callback)
results = await asyncio.gather(
*[_one(plugin) for _, plugin in to_search],
return_exceptions=True,
)
for (source_name, _), result in zip(to_search, results, strict=True):
if isinstance(result, Exception):
logger.warning(f"{source_name} search failed: {result}")
contributions.append(f"{source_name}=error")
continue
tracks, albums = result
n = len(tracks) if tracks else 0
if tracks:
pooled_tracks.extend(tracks)
if albums:
pooled_albums.extend(albums)
contributions.append(f"{source_name}={n}")
logger.info(
"Best-quality pool: %d candidates [%s] for: %s",
len(pooled_tracks), ', '.join(contributions), query,
)
return (pooled_tracks, pooled_albums)
async def download_with_fallback(self, username: str, filename: str, async def download_with_fallback(self, username: str, filename: str,
file_size: int, source_chain) -> Optional[str]: file_size: int, source_chain) -> Optional[str]:
"""Try each source in ``source_chain`` until one accepts the """Try each source in ``source_chain`` until one accepts the

View file

@ -28,7 +28,6 @@ from config.settings import config_manager
from core.download_engine import DownloadEngine from core.download_engine import DownloadEngine
from core.download_plugins.registry import DownloadPluginRegistry, build_default_registry from core.download_plugins.registry import DownloadPluginRegistry, build_default_registry
from core.download_plugins.types import TrackResult, AlbumResult, DownloadStatus from core.download_plugins.types import TrackResult, AlbumResult, DownloadStatus
from core.quality.selection import load_search_mode
logger = get_logger("download_orchestrator") logger = get_logger("download_orchestrator")
@ -103,14 +102,12 @@ class DownloadOrchestrator:
deezer_dl = self.client('deezer_dl') deezer_dl = self.client('deezer_dl')
if deezer_arl and deezer_dl: if deezer_arl and deezer_dl:
deezer_dl.reconnect(deezer_arl) deezer_dl.reconnect(deezer_arl)
from core.quality.source_map import quality_tier_for_source deezer_dl._quality = config_manager.get('deezer_download.quality', 'flac')
deezer_dl._quality = quality_tier_for_source('deezer', default='flac')
# Reload Amazon quality preference (T2Tunes needs no reconnect — public proxy) # Reload Amazon quality preference (T2Tunes needs no reconnect — public proxy)
amazon = self.client('amazon') amazon = self.client('amazon')
if amazon: if amazon:
from core.quality.source_map import quality_tier_for_source quality = config_manager.get('amazon_download.quality', 'flac')
quality = quality_tier_for_source('amazon', default='flac')
amazon._quality = quality amazon._quality = quality
amazon._allow_fallback = config_manager.get('amazon_download.allow_fallback', True) amazon._allow_fallback = config_manager.get('amazon_download.allow_fallback', True)
if hasattr(amazon, '_client') and amazon._client: if hasattr(amazon, '_client') and amazon._client:
@ -143,10 +140,7 @@ class DownloadOrchestrator:
continue continue
if hasattr(client, 'download_path') and client.download_path != new_path: if hasattr(client, 'download_path') and client.download_path != new_path:
client.download_path = new_path client.download_path = new_path
try: client.download_path.mkdir(parents=True, exist_ok=True)
client.download_path.mkdir(parents=True, exist_ok=True)
except OSError as e:
logger.warning(f"Could not verify download path {new_path}: {e}")
# YouTube also caches path in yt-dlp opts # YouTube also caches path in yt-dlp opts
if hasattr(client, 'download_opts') and 'outtmpl' in client.download_opts: if hasattr(client, 'download_opts') and 'outtmpl' in client.download_opts:
client.download_opts['outtmpl'] = str(new_path / '%(title)s.%(ext)s') client.download_opts['outtmpl'] = str(new_path / '%(title)s.%(ext)s')
@ -348,11 +342,6 @@ class DownloadOrchestrator:
if not chain: if not chain:
logger.warning("Hybrid search exhausted: no eligible sources after exclusion filter") logger.warning("Hybrid search exhausted: no eligible sources after exclusion filter")
return [], [] return [], []
if load_search_mode() == 'best_quality':
logger.info(f"Best-quality search ({''.join(chain)}): {query}")
return await self.engine.search_all_sources(
query, chain, timeout, progress_callback,
)
logger.info(f"Hybrid search ({''.join(chain)}): {query}") logger.info(f"Hybrid search ({''.join(chain)}): {query}")
return await self.engine.search_with_fallback(query, chain, timeout, progress_callback) return await self.engine.search_with_fallback(query, chain, timeout, progress_callback)
@ -423,17 +412,9 @@ class DownloadOrchestrator:
if scored: if scored:
scored.sort(key=lambda x: x._match_confidence, reverse=True) scored.sort(key=lambda x: x._match_confidence, reverse=True)
# Match filter done (right track); now prefer the best quality filtered_results = scored
# among the confidence-passing survivors so streaming isn't
# quality-blind like Soulseek already isn't. Stable ranking
# keeps confidence order within an equal quality tier; the
# `or scored` fail-safe never leaves us with nothing to try.
from core.quality.selection import rank_for_profile
ranked, _ = rank_for_profile(scored)
filtered_results = ranked or scored
logger.info(f"Streaming validation: {len(scored)}/{len(tracks)} passed " logger.info(f"Streaming validation: {len(scored)}/{len(tracks)} passed "
f"(best: {scored[0]._match_confidence:.2f}, " f"(best: {scored[0]._match_confidence:.2f})")
f"quality pick: {filtered_results[0].audio_quality.label()})")
else: else:
logger.warning(f"No streaming results passed validation for: {query}") logger.warning(f"No streaming results passed validation for: {query}")
return None return None

View file

@ -13,11 +13,10 @@ import from a neutral package per Cin's contract-first standard.
from __future__ import annotations from __future__ import annotations
from dataclasses import dataclass, field from dataclasses import dataclass
from typing import Any, Dict, List, Optional from typing import Any, Dict, List, Optional
from core.imports.filename import parse_filename_metadata from core.imports.filename import parse_filename_metadata
from core.quality.model import AudioQuality
@dataclass @dataclass
@ -33,36 +32,6 @@ class SearchResult:
upload_speed: int upload_speed: int
queue_length: int queue_length: int
result_type: str = "track" # "track" or "album" result_type: str = "track" # "track" or "album"
# Rich quality metadata — populated by sources that provide it.
# None means "unknown", not "absent".
sample_rate: Optional[int] = None # Hz (e.g. 44100, 96000, 192000)
bit_depth: Optional[int] = None # bits per sample (16, 24)
@property
def audio_quality(self) -> AudioQuality:
"""Unified quality descriptor derived from this result's fields."""
return AudioQuality(
format=self.quality.lower() if self.quality else 'unknown',
bitrate=self.bitrate,
sample_rate=self.sample_rate,
bit_depth=self.bit_depth,
)
def set_quality(self, aq: AudioQuality) -> None:
"""Merge a mapped :class:`AudioQuality` onto this result's fields.
Used by streaming sources to stamp their claimed tier (Tidal/HiFi
tier strings, Qobuz API values, ) so ``audio_quality`` ranks
correctly. Mapper-provided fields win; a ``None`` from the mapper
leaves any already-reported value (e.g. a probed bitrate) intact.
"""
self.quality = aq.format
if aq.bitrate is not None:
self.bitrate = aq.bitrate
if aq.sample_rate is not None:
self.sample_rate = aq.sample_rate
if aq.bit_depth is not None:
self.bit_depth = aq.bit_depth
@property @property
def quality_score(self) -> float: def quality_score(self) -> float:
@ -158,19 +127,6 @@ class AlbumResult:
queue_length: int = 0 queue_length: int = 0
result_type: str = "album" result_type: str = "album"
@property
def audio_quality(self) -> AudioQuality:
"""Unified quality descriptor derived from dominant track quality."""
sample_rates = [t.sample_rate for t in self.tracks if t.sample_rate]
bit_depths = [t.bit_depth for t in self.tracks if t.bit_depth]
bitrates = [t.bitrate for t in self.tracks if t.bitrate]
return AudioQuality(
format=self.dominant_quality.lower() if self.dominant_quality else 'unknown',
bitrate=max(bitrates) if bitrates else None,
sample_rate=max(sample_rates) if sample_rates else None,
bit_depth=max(bit_depths) if bit_depths else None,
)
@property @property
def quality_score(self) -> float: def quality_score(self) -> float:
"""Calculate album quality score based on dominant quality and track count""" """Calculate album quality score based on dominant quality and track count"""

View file

@ -83,26 +83,9 @@ def clear_completed_local() -> int:
""" """
cleared = 0 cleared = 0
with tasks_lock: with tasks_lock:
# Protect tasks belonging to a still-active batch. A batch is "active"
# while any of its queued tasks is non-terminal (still searching /
# downloading / queued / post-processing). Pruning a batch's completed
# or failed tasks mid-run would yank them out of the Downloads page —
# and failed/cancelled rows aren't recoverable from library_history —
# so the user would never see them until the batch ended. Keep the whole
# active batch intact; it gets cleaned by a later run once it finishes.
protected_task_ids: set = set()
for batch in download_batches.values():
queue = batch.get('queue', []) if isinstance(batch, dict) else []
batch_active = any(
download_tasks.get(tid, {}).get('status') not in _TERMINAL_STATUSES
for tid in queue if tid in download_tasks
)
if batch_active:
protected_task_ids.update(queue)
task_ids_to_remove = [ task_ids_to_remove = [
tid for tid, task in download_tasks.items() tid for tid, task in download_tasks.items()
if task.get('status') in _TERMINAL_STATUSES and tid not in protected_task_ids if task.get('status') in _TERMINAL_STATUSES
] ]
for tid in task_ids_to_remove: for tid in task_ids_to_remove:
del download_tasks[tid] del download_tasks[tid]

View file

@ -47,55 +47,6 @@ from core.runtime_state import (
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
def _priority_sort_key(r):
"""Today's confidence-first key: never download a high-quality WRONG file."""
return (
getattr(r, 'confidence', 0) or 0,
getattr(r, 'quality_score', 0) or 0,
getattr(r, 'upload_speed', 0) or 0,
-(getattr(r, 'queue_length', 0) or 0),
getattr(r, 'free_upload_slots', 0) or 0,
getattr(r, 'size', 0) or 0,
)
def _quality_first_sort_key(r, targets):
"""Best-quality key: the user's profile quality rank dominates; all the
priority-mode signals (confidence, speed, ) become tiebreakers.
Every candidate reaching this point already passed match filtering, so it
is "correct enough" ordering by quality among correct candidates is safe.
Candidates with no usable quality info, or that match no target, sort last
(never dropped). Lower target index = better target, so it's negated to fit
the descending (reverse=True) sort.
"""
from core.quality.model import rank_candidate
aq = getattr(r, 'audio_quality', None)
if aq is None or not targets:
target_idx, tier = (len(targets) if targets else 0), 0.0
else:
try:
target_idx, tier = rank_candidate(aq, targets)
except Exception:
target_idx, tier = len(targets), 0.0
return (-target_idx, tier) + _priority_sort_key(r)
def order_candidates(candidates, *, quality_first=False, targets=None):
"""Return *candidates* ordered best-first for the download walk.
``quality_first=False`` (priority mode) confidence-first, byte-for-byte
today's behaviour. ``quality_first=True`` (best-quality mode) → the user's
profile quality rank dominates, confidence/peer signals break ties.
"""
if quality_first:
key = lambda r: _quality_first_sort_key(r, targets or [])
else:
key = _priority_sort_key
return sorted(candidates, key=key, reverse=True)
@dataclass @dataclass
class CandidatesDeps: class CandidatesDeps:
"""Bundle of cross-cutting deps the candidate-fallback logic needs.""" """Bundle of cross-cutting deps the candidate-fallback logic needs."""
@ -108,25 +59,25 @@ class CandidatesDeps:
on_download_completed: Callable on_download_completed: Callable
def attempt_download_with_candidates(task_id, candidates, track, batch_id=None, def attempt_download_with_candidates(task_id, candidates, track, batch_id=None, deps: CandidatesDeps = None):
deps: CandidatesDeps = None, *,
quality_first=False, quality_targets=None):
""" """
Attempts to download with fallback candidate logic (matches GUI's retry_parallel_download_with_fallback). Attempts to download with fallback candidate logic (matches GUI's retry_parallel_download_with_fallback).
Returns True if successful, False if all candidates fail. Returns True if successful, False if all candidates fail.
``quality_first`` (best-quality search mode) orders the walk by the user's
profile quality rank instead of confidence-first; ``quality_targets`` is the
profile target list used for that ranking. Defaults preserve priority-mode
behaviour exactly.
""" """
# Sort candidates. Priority mode: confidence-first, then peer quality — # Sort candidates by match confidence first, then peer quality. Upstream
# upstream Soulseek validation already considers peer speed/slots/queue when # Soulseek validation already considers peer speed/slots/queue when scores
# scores are close; preserve that signal instead of flattening ties back to # are close; preserve that signal here instead of flattening ties back to
# arbitrary slskd response order. Best-quality mode: profile quality rank # arbitrary slskd response order.
# dominates (all candidates here already passed match filtering). candidates.sort(
candidates = order_candidates( key=lambda r: (
candidates, quality_first=quality_first, targets=quality_targets, getattr(r, 'confidence', 0) or 0,
getattr(r, 'quality_score', 0) or 0,
getattr(r, 'upload_speed', 0) or 0,
-(getattr(r, 'queue_length', 0) or 0),
getattr(r, 'free_upload_slots', 0) or 0,
getattr(r, 'size', 0) or 0,
),
reverse=True,
) )
with tasks_lock: with tasks_lock:
@ -254,21 +205,6 @@ def attempt_download_with_candidates(task_id, candidates, track, batch_id=None,
'artists': _fallback_album_artists 'artists': _fallback_album_artists
} }
# #915: parity with Reorganize / manual Enrich. If the album context is lean
# (no release_date) and the user's PRIMARY metadata source isn't Spotify, hydrate
# it from that source — the same place a reorganize reads — so the download's
# $year folder, release_date and album_type match instead of dropping the year /
# defaulting to YYYY-01-01 and forcing a manual reorganize afterwards.
try:
from core.downloads.track_metadata_backfill import backfill_album_context_from_source
from core.metadata import registry as _meta_registry
from core.metadata.album_tracks import get_album_for_source as _get_album_for_source
backfill_album_context_from_source(
spotify_album_context, _meta_registry.get_primary_source(), _get_album_for_source,
)
except Exception as _bf_err: # noqa: BLE001 — never let backfill break a download
logger.debug("[Context] primary-source album backfill skipped: %s", _bf_err)
download_payload = candidate.__dict__ download_payload = candidate.__dict__
username = download_payload.get('username') username = download_payload.get('username')

View file

@ -77,16 +77,7 @@ def _normalize_for_finding(text: str) -> str:
return "" return ""
text = unidecode(text).lower() text = unidecode(text).lower()
text = re.sub(r'[._/]', ' ', text) text = re.sub(r'[._/]', ' ', text)
# Strip ONLY balanced bracket pairs (tags like "[FLAC]", "(Remastered 2016)"). text = re.sub(r'[\[\(].*?[\]\)]', '', text)
# The old combined pattern r'[\[\(].*?[\]\)]' allowed MISMATCHED delimiters, so a
# lone unbalanced '[' — slskd reports "[34 - You & Me (Flume Remix)" but saves the
# file as "34 - You & Me (Flume Remix)" — matched from that '[' all the way to the
# next ')', eating the entire title and collapsing the search target to "flac". The
# file then scored 0.40 against the real on-disk name and was reported "not found"
# despite sitting right there. Per-delimiter pairs can't over-consume; a stray
# unbalanced bracket simply survives to the alphanumeric strip below.
text = re.sub(r'\[[^\]]*\]', '', text)
text = re.sub(r'\([^)]*\)', '', text)
text = re.sub(r'[^a-z0-9\s-]', '', text) text = re.sub(r'[^a-z0-9\s-]', '', text)
return ' '.join(text.split()).strip() return ' '.join(text.split()).strip()

View file

@ -1,62 +0,0 @@
"""Match a file back to its download-history row when its path has drifted (#934).
``library_history.file_path`` is frozen at import time, but the file moves afterward
(media-server import, library reorganize) and ``tracks.file_path`` what the AcoustID
scanner reads no longer equals it. Matching on the exact path alone then fails twice:
the verification status never reaches the history row (verified tracks read "unverified"),
and a fresh ``acoustid_scan`` row gets inserted every run (thousands of duplicates).
This module picks the canonical history row by exact path first, then by FILENAME guarded
by a title check so a shared filename ("01 - Intro.flac") can never heal the wrong song.
Pure (no DB) so the matching rules are unit-testable; the caller does the SQL.
"""
from __future__ import annotations
import os
from typing import Iterable, Optional, Sequence, Tuple
def _norm_title(value) -> str:
"""Alphanumeric-only lowercase form, so "Song (Remaster)" vs "song remaster"
style drift between the download tag and the media-server tag still agrees."""
return ''.join(ch for ch in str(value or '').lower() if ch.isalnum())
def like_filename_filter(basename: str) -> str:
"""A ``LIKE ... ESCAPE '\\'`` pattern that coarsely matches rows whose path ends
in ``basename``. Escapes the LIKE metacharacters (``%`` ``_`` ``\\``) filenames
routinely contain underscores. Callers MUST still confirm with an exact basename
compare (``pick_history_row`` does), since ``'%name'`` also matches ``'xname'``."""
esc = basename.replace('\\', '\\\\').replace('%', '\\%').replace('_', '\\_')
return '%' + esc
def pick_history_row(candidates: Sequence[Tuple], *, current_paths: Iterable[str],
basename: str, title: str) -> Optional[int]:
"""Return the id of the history row to update for this file, or None.
``candidates``: ``(id, file_path, title, download_source)`` rows the DB pre-filtered
(exact path or filename LIKE). A row matches when its path equals the current path OR
its filename matches AND its title agrees the title guard prevents a shared filename
("01 - Intro.flac") from healing a different song's row. Among matches a REAL download
row is preferred over a synthetic ``acoustid_scan`` row, so the scanner heals the
genuine record and the caller can delete the synthetic duplicate. None when nothing
matches safely (caller then inserts a fresh row the "file SoulSync never downloaded"
intent)."""
paths = {p for p in current_paths if p}
want = _norm_title(title)
matches: list = [] # (id, is_exact, is_real)
for cid, cpath, ctitle, csource in candidates:
is_real = csource != 'acoustid_scan'
if cpath and cpath in paths:
matches.append((cid, True, is_real))
elif (basename and cpath and os.path.basename(cpath) == basename
and (not want or not _norm_title(ctitle) or _norm_title(ctitle) == want)):
matches.append((cid, False, is_real))
if not matches:
return None
# Prefer a REAL download row over a synthetic acoustid_scan row; within that, prefer an
# exact-path match over a filename match. Stable, so ties keep DB order (first/oldest id).
matches.sort(key=lambda m: (m[2], m[1]), reverse=True)
return matches[0][0]

View file

@ -26,7 +26,6 @@ Lifted verbatim from web_server.py. Dependencies injected via
from __future__ import annotations from __future__ import annotations
import logging import logging
import os
import shutil import shutil
import time import time
import traceback import traceback
@ -45,27 +44,6 @@ from core.runtime_state import (
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
# A task that has been in 'post_processing' longer than this is treated as stuck.
# Post-processing (AcoustID + quality + import) is serialized, so a large batch
# legitimately backs up — keep this generous so genuinely-slow imports aren't
# cut off mid-flight (the old 5-min cutoff falsely "completed" queued tasks).
_POST_PROCESSING_STUCK_TIMEOUT = 1800 # 30 minutes
def _resolve_stuck_post_processing_status(task: dict) -> str:
"""Decide the terminal status for a task stuck in post_processing.
Only call it 'completed' if the import actually produced a file on disk
(``final_file_path`` is set at the end of successful post-processing). Without
a real file, force-completing is a lie the task shows as a downloaded track
that isn't anywhere. Mark those 'failed' so they're retryable and honest.
"""
final_path = task.get('final_file_path')
if final_path and os.path.exists(final_path):
return 'completed'
return 'failed'
def _safe_batch_dirname(batch_id: str) -> str: def _safe_batch_dirname(batch_id: str) -> str:
safe = ''.join(ch if ch.isalnum() or ch in ('-', '_') else '_' for ch in str(batch_id or 'batch')) safe = ''.join(ch if ch.isalnum() or ch in ('-', '_') else '_' for ch in str(batch_id or 'batch'))
return safe or 'batch' return safe or 'batch'
@ -457,15 +435,9 @@ def on_download_completed(batch_id: str, task_id: str, success: bool, deps: Life
retrying_count += 1 retrying_count += 1
elif task_status == 'post_processing': elif task_status == 'post_processing':
task_age = current_time - task.get('status_change_time', current_time) task_age = current_time - task.get('status_change_time', current_time)
if task_age > _POST_PROCESSING_STUCK_TIMEOUT: if task_age > 300: # 5 minutes (post-processing should be fast)
new_status = _resolve_stuck_post_processing_status(task) logger.info(f"⏰ [Stuck Detection] Task {queue_task_id} stuck in post_processing for {task_age:.0f}s - forcing completion")
if new_status == 'completed': task['status'] = 'completed' # Assume it worked if file verification is taking too long
logger.info(f"⏰ [Stuck Detection] Task {queue_task_id} stuck in post_processing for {task_age:.0f}s but file exists — completing")
task['status'] = 'completed'
else:
logger.warning(f"⏰ [Stuck Detection] Task {queue_task_id} stuck in post_processing for {task_age:.0f}s with no output file — marking failed")
task['status'] = 'failed'
task['error_message'] = 'Post-processing timed out without producing a file'
finished_count += 1 finished_count += 1
else: else:
retrying_count += 1 retrying_count += 1
@ -695,15 +667,9 @@ def check_batch_completion_v2(batch_id: str, deps: LifecycleDeps) -> Optional[bo
retrying_count += 1 retrying_count += 1
elif task_status == 'post_processing': elif task_status == 'post_processing':
task_age = current_time - task.get('status_change_time', current_time) task_age = current_time - task.get('status_change_time', current_time)
if task_age > _POST_PROCESSING_STUCK_TIMEOUT: if task_age > 300: # 5 minutes (post-processing should be fast)
new_status = _resolve_stuck_post_processing_status(task) logger.info(f"⏰ [Stuck Detection V2] Task {task_id} stuck in post_processing for {task_age:.0f}s - forcing completion")
if new_status == 'completed': task['status'] = 'completed' # Assume it worked if file verification is taking too long
logger.info(f"⏰ [Stuck Detection V2] Task {task_id} stuck in post_processing for {task_age:.0f}s but file exists — completing")
task['status'] = 'completed'
else:
logger.warning(f"⏰ [Stuck Detection V2] Task {task_id} stuck in post_processing for {task_age:.0f}s with no output file — marking failed")
task['status'] = 'failed'
task['error_message'] = 'Post-processing timed out without producing a file'
finished_count += 1 finished_count += 1
else: else:
retrying_count += 1 retrying_count += 1

View file

@ -760,8 +760,7 @@ class WebUIDownloadMonitor:
# used_sources keys are formatted as "{username}_{filename}", so startswith is exact. # used_sources keys are formatted as "{username}_{filename}", so startswith is exact.
is_tidal = any(s.startswith('tidal_') for s in tried_sources) is_tidal = any(s.startswith('tidal_') for s in tried_sources)
if is_tidal: if is_tidal:
from core.quality.source_map import quality_tier_for_source tidal_quality = config_manager.get('tidal_download.quality', 'lossless')
tidal_quality = quality_tier_for_source('tidal', default='lossless')
allow_fb = config_manager.get('tidal_download.allow_fallback', True) allow_fb = config_manager.get('tidal_download.allow_fallback', True)
if tidal_quality == 'hires' and not allow_fb: if tidal_quality == 'hires' and not allow_fb:
task['error_message'] = ( task['error_message'] = (

View file

@ -1,55 +0,0 @@
"""Identify dead review-queue history rows whose file is gone (#934 follow-up).
The Unverified/Quarantine review queue is fed from ``library_history`` an
append-only log that is never pruned. When a file is deleted, replaced, or
re-downloaded elsewhere, its old ``unverified`` row lingers forever and can
never be healed (there's no file left to confirm). Those are *orphans*.
This decides which rows are orphans, given a ``resolve(row) -> path | None``
the caller wires to the real filesystem lookup. Pure (no DB, no filesystem) so
the rules including the safety gate are unit-testable.
Safety gate: a filesystem check mass-false-positives when the library mount is
down (every file looks missing). So if EVERY reviewed file is unreachable and
there are enough rows to judge, we flag it ``suspicious`` and the caller refuses
to delete better to clean nothing than to wipe a healthy log during an outage.
"""
from __future__ import annotations
from typing import Any, Callable, Sequence
def find_orphan_history_ids(
rows: Sequence[dict],
resolve: Callable[[dict], Any],
*,
min_for_safety: int = 5,
deletable: Callable[[dict], bool] | None = None,
) -> dict:
"""Return ``{'orphan_ids', 'checked', 'suspicious'}``.
A row is an orphan when it has a non-empty ``file_path`` but ``resolve`` can
find no file for it. ``suspicious`` is True when every checked row is
missing and there are at least ``min_for_safety`` of them the mount-down
signature; the caller should refuse to delete in that case.
``deletable`` (optional) protects rows from removal WITHOUT weakening the
safety gate: a protected row still counts toward ``checked`` and the
all-missing signal (so e.g. a few unverified orphans can't be swept during a
mount outage just because protected rows were filtered out first), but it
never appears in ``orphan_ids``. Default: every missing row is deletable.
"""
orphan_ids = []
checked = 0
missing = 0
for row in rows:
if not str((row.get('file_path') or '')).strip():
continue
checked += 1
if resolve(row) is None:
missing += 1
if deletable is None or deletable(row):
orphan_ids.append(row.get('id'))
suspicious = checked >= min_for_safety and missing == checked
return {'orphan_ids': orphan_ids, 'checked': checked, 'suspicious': suspicious}

View file

@ -171,21 +171,6 @@ def run_post_processing_worker(task_id: str, batch_id: str, deps: PostProcessDep
logger.info(f"[Post-Processing] Task {task_id} already completed by stream processor, skipping verification") logger.info(f"[Post-Processing] Task {task_id} already completed by stream processor, skipping verification")
return return
# RACE GUARD: the monitor sets status -> 'post_processing' immediately
# before submitting this worker. If the status is now anything else, the
# browser-poll post-processor already took ownership of this task — e.g.
# it quarantined the file and requeued the next-best candidate (status
# -> 'searching', source identity cleared). Bail WITHOUT marking failed
# or notifying batch completion: otherwise we clobber that in-flight
# retry with a bogus "missing file or source information" failure while a
# parallel attempt is importing the song.
if task['status'] != 'post_processing':
logger.info(
f"[Post-Processing] Task {task_id} no longer in 'post_processing' "
f"(now '{task['status']}') — another path took over, skipping"
)
return
# Extract file information for verification # Extract file information for verification
track_info = task.get('track_info', {}) track_info = task.get('track_info', {})
task_filename = task.get('filename') or track_info.get('filename') task_filename = task.get('filename') or track_info.get('filename')
@ -453,19 +438,7 @@ def run_post_processing_worker(task_id: str, batch_id: str, deps: PostProcessDep
logger.error(f"[Post-Processing] Task {task_id} was completed by stream processor - not marking as failed") logger.error(f"[Post-Processing] Task {task_id} was completed by stream processor - not marking as failed")
return return
download_tasks[task_id]['status'] = 'failed' download_tasks[task_id]['status'] = 'failed'
# slskd reported the transfer complete, but the finder never located download_tasks[task_id]['error_message'] = f'File not found on disk after {_file_search_max_retries} search attempts. Expected: {os.path.basename(task_filename)}'
# the file under the configured download folder. Name the folder we
# searched and the two real causes — "still being written" (timing)
# or "SoulSync's download path doesn't match slskd's" (the classic
# standalone config mismatch) — so the user can self-diagnose instead
# of getting an opaque "not found". (Discord: Shdjfgatdif.)
_searched_name = os.path.basename((task_filename or '').replace('\\', '/')) or task_filename
download_tasks[task_id]['error_message'] = (
f"slskd reported '{_searched_name}' downloaded, but it never appeared "
f"under the download folder ({download_dir}) after {_file_search_max_retries} "
f"checks. Either it's still being written, or SoulSync's download path "
f"doesn't match slskd's download directory — they must point at the same folder."
)
deps.on_download_completed(batch_id, task_id, False) deps.on_download_completed(batch_id, task_id, False)
return return

View file

@ -94,10 +94,6 @@ class StatusDeps:
run_async: Optional[Callable] = None run_async: Optional[Callable] = None
on_download_completed: Optional[Callable[[str, str, bool], None]] = None on_download_completed: Optional[Callable[[str, str, bool], None]] = None
get_persistent_download_history: Optional[Callable[[int], list[dict]]] = None get_persistent_download_history: Optional[Callable[[int], list[dict]]] = None
# Returns ALL library_history rows with verification_status in
# ('unverified', 'force_imported') — no recency limit, so historical
# entries are never buried by the general history tail cap.
get_unverified_download_history: Optional[Callable[[], list[dict]]] = None
# Streaming sources the engine fallback applies to. Soulseek goes through # Streaming sources the engine fallback applies to. Soulseek goes through
@ -800,13 +796,6 @@ def build_unified_downloads_response(limit: int, deps: StatusDeps) -> dict:
'progress': progress, 'progress': progress,
'error': task.get('error_message'), 'error': task.get('error_message'),
'verification_status': task.get('verification_status'), 'verification_status': task.get('verification_status'),
# library_history row id (set at import) so the Unverified review
# queue can act on a still-live completed task before it becomes
# a persistent-history row.
'history_id': task.get('history_id'),
# Real probed audio quality (mutagen-read from the actual file),
# surfaced so the Downloads page can show what was downloaded.
'quality': task.get('quality') or '',
'retry_info': task.get('retry_info'), 'retry_info': task.get('retry_info'),
'retry_trigger': task.get('retry_trigger'), 'retry_trigger': task.get('retry_trigger'),
'batch_id': batch_id, 'batch_id': batch_id,
@ -823,32 +812,6 @@ def build_unified_downloads_response(limit: int, deps: StatusDeps) -> dict:
'is_persistent_history': False, 'is_persistent_history': False,
}) })
# --- Unverified history (unconditional, no limit) ---
# Always load every library_history row that still needs human confirmation
# (verification_status IN ('unverified', 'force_imported')). This is NOT
# gated on len(items) < limit so that historical entries from past batches
# are visible even during a large active batch that would otherwise exhaust
# the limit before the history tail is read. Dedup against live tasks by
# identity so a track currently in post-processing isn't shown twice.
if deps.get_unverified_download_history is not None:
try:
unverified_entries = deps.get_unverified_download_history() or []
except Exception as exc:
logger.debug("[Downloads] unverified history lookup failed: %s", exc)
unverified_entries = []
for entry in unverified_entries:
item = _build_history_download_item(entry)
identity = _download_identity(item.get('title'), item.get('artist'), item.get('album'))
if identity in live_identities:
continue
items.append(item)
live_identities.add(identity)
# --- General recent-history tail (capped, recency-ordered) ---
# Fills in the completed/verified tail so the full Downloads list looks
# populated. Gated on len(items) < limit so a busy batch doesn't trigger
# an extra DB round-trip when we're already at capacity.
if deps.get_persistent_download_history is not None and len(items) < limit: if deps.get_persistent_download_history is not None and len(items) < limit:
history_limit = min(limit - len(items), _PERSISTENT_HISTORY_TAIL_LIMIT) history_limit = min(limit - len(items), _PERSISTENT_HISTORY_TAIL_LIMIT)
try: try:
@ -869,14 +832,7 @@ def build_unified_downloads_response(limit: int, deps: StatusDeps) -> dict:
live_identities.add(identity) live_identities.add(identity)
appended_history += 1 appended_history += 1
# Sort: active first (by priority), then by timestamp desc within each group. # Sort: active first (by priority), then by timestamp desc within each group
# NOTE: the array order is presentation-only — the Downloads page filters
# client-side per tab. What matters is that EVERY live task is present: an
# earlier `items[:limit]` truncation (active-first) starved completed/failed/
# unverified rows off the end during a busy batch, so those tabs stayed empty
# until the batch drained. `limit` now bounds only the persistent-history
# tail (handled above); live in-memory tasks are always returned in full
# (they're already bounded by the 5-min cleanup automation).
items.sort(key=lambda x: (x['priority'], -x['timestamp'])) items.sort(key=lambda x: (x['priority'], -x['timestamp']))
# Build batch summaries for the batch context panel # Build batch summaries for the batch context panel
@ -904,7 +860,7 @@ def build_unified_downloads_response(limit: int, deps: StatusDeps) -> dict:
return { return {
'success': True, 'success': True,
'downloads': items, 'downloads': items[:limit],
'total': len(items), 'total': len(items),
'batches': batch_summaries, 'batches': batch_summaries,
'timestamp': time.time(), 'timestamp': time.time(),

View file

@ -54,37 +54,6 @@ def _cand_user_file(candidate):
return getattr(candidate, 'username', None), getattr(candidate, 'filename', None) return getattr(candidate, 'username', None), getattr(candidate, 'filename', None)
def _candidate_ordering():
"""Return ``(quality_first, targets)`` for the active search mode + toggle.
The candidate walk is ordered by the user's profile quality rank
(bestworst) instead of confidence-first when EITHER:
- best-quality search mode is active (always quality-first), OR
- priority mode and the ``rank_candidates_by_quality`` toggle is on
(opt-in; default off keeps the byte-for-byte confidence-first walk).
Quality-first ordering also makes the version-mismatch force-import pick
the highest-quality candidate, because that fallback accepts the
first-tried (= best-ordered) quarantined entry.
Fails closed to confidence-first ordering on any error so a profile/DB
hiccup never blocks a download. See
docs/superpowers/specs/2026-06-14-best-quality-search-mode-design.md.
"""
try:
from core.quality.selection import (
load_search_mode,
load_profile_targets,
load_rank_candidates_by_quality,
)
if load_search_mode() == 'best_quality' or load_rank_candidates_by_quality():
targets, _ = load_profile_targets()
return True, targets
except Exception as exc:
logger.debug("[Modal Worker] quality ordering unavailable: %s", exc)
return False, None
def _try_cached_candidates(task_id, batch_id, track, deps): def _try_cached_candidates(task_id, batch_id, track, deps):
"""Quarantine-retry fast path: attempt the already-found candidates before """Quarantine-retry fast path: attempt the already-found candidates before
re-searching anything. re-searching anything.
@ -122,10 +91,7 @@ def _try_cached_candidates(task_id, batch_id, track, deps):
f"[Modal Worker] Quarantine retry: trying {len(remaining)} cached " f"[Modal Worker] Quarantine retry: trying {len(remaining)} cached "
f"candidate(s) before re-searching (task {task_id})" f"candidate(s) before re-searching (task {task_id})"
) )
_qf, _qt = _candidate_ordering() return deps.attempt_download_with_candidates(task_id, remaining, track, batch_id)
return deps.attempt_download_with_candidates(
task_id, remaining, track, batch_id, quality_first=_qf, quality_targets=_qt,
)
def _private_album_bundle_staging_miss_reason(batch_id: Optional[str], deps: Any) -> Optional[str]: def _private_album_bundle_staging_miss_reason(batch_id: Optional[str], deps: Any) -> Optional[str]:
@ -403,11 +369,6 @@ def download_track_worker(task_id: str, batch_id: Optional[str], deps: TaskWorke
logger.info(f"[Modal Worker] Generated {len(search_queries)} smart search queries for '{track.name}': {search_queries}") logger.info(f"[Modal Worker] Generated {len(search_queries)} smart search queries for '{track.name}': {search_queries}")
logger.info(f"[Modal Worker] About to start search loop for task {task_id} (track: '{track.name}')") logger.info(f"[Modal Worker] About to start search loop for task {task_id} (track: '{track.name}')")
# Best-quality search mode: the orchestrator already pooled candidates
# across every source for each query, so order the candidate walk by the
# user's profile quality rank (best→worst). Computed once per task.
_best_quality, _quality_targets = _candidate_ordering()
# 2. Sequential Query Search (matches GUI's start_search_worker_parallel logic) # 2. Sequential Query Search (matches GUI's start_search_worker_parallel logic)
search_diagnostics = [] # Track what happened per query for detailed error messages search_diagnostics = [] # Track what happened per query for detailed error messages
all_raw_results = [] # Collect raw results across queries for candidate review modal all_raw_results = [] # Collect raw results across queries for candidate review modal
@ -534,10 +495,7 @@ def download_track_worker(task_id: str, batch_id: Optional[str], deps: TaskWorke
download_tasks[task_id]['cached_candidates'] = candidates download_tasks[task_id]['cached_candidates'] = candidates
# Try to download with these candidates # Try to download with these candidates
success = deps.attempt_download_with_candidates( success = deps.attempt_download_with_candidates(task_id, candidates, track, batch_id)
task_id, candidates, track, batch_id,
quality_first=_best_quality, quality_targets=_quality_targets,
)
if success: if success:
# Download initiated successfully - let the download monitoring system handle completion # Download initiated successfully - let the download monitoring system handle completion
if batch_id: if batch_id:
@ -571,10 +529,7 @@ def download_track_worker(task_id: str, batch_id: Optional[str], deps: TaskWorke
# === HYBRID FALLBACK: If primary source failed, try remaining sources directly === # === HYBRID FALLBACK: If primary source failed, try remaining sources directly ===
# The orchestrator's hybrid search stops at the first source with results, even if # The orchestrator's hybrid search stops at the first source with results, even if
# those results all fail quality filtering. Try remaining sources individually. # those results all fail quality filtering. Try remaining sources individually.
# if getattr(deps.download_orchestrator, 'mode', '') == 'hybrid':
# Best-quality mode already searched EVERY source per query (the pool), so this
# block would only re-search the same sources — skip it there.
if not _best_quality and getattr(deps.download_orchestrator, 'mode', '') == 'hybrid':
try: try:
orch = deps.download_orchestrator orch = deps.download_orchestrator
hybrid_order = getattr(orch, 'hybrid_order', None) or [] hybrid_order = getattr(orch, 'hybrid_order', None) or []

View file

@ -13,48 +13,9 @@ Pure + import-safe: parsing only, no network.
from __future__ import annotations from __future__ import annotations
import re import re
from typing import Any, List, Optional, Tuple from typing import Any, Optional, Tuple
from urllib.parse import urlparse from urllib.parse import urlparse
def linked_track_id(track: Any) -> str:
"""The source track id stamped on a search result, read from
``_source_metadata['track_id']`` the field every ID-downloadable source
(Tidal, Qobuz) records. Empty string when absent. ``TrackResult`` has no
top-level ``id``, so callers must NOT use ``getattr(t, 'id')`` (that always
missed and left the pasted-link bubble a silent no-op #932)."""
meta = getattr(track, '_source_metadata', None)
if not isinstance(meta, dict):
return ''
return str(meta.get('track_id') or '')
def bubble_linked_track_first(tracks: List[Any], link_track_id: str) -> List[Any]:
"""Float the result whose source id matches a pasted link to the top so the
user sees the EXACT track they linked, not a fuzzy text-search lookalike
(#813/#932). Stable + a graceful no-op when no result carries the id."""
if not link_track_id or not tracks:
return tracks
target = str(link_track_id)
return sorted(tracks, key=lambda t: linked_track_id(t) != target)
def inject_linked_track_first(
tracks: List[Any], linked_result: Any, link_track_id: str
) -> List[Any]:
"""Put the EXACT linked track first.
When ``linked_result`` is the track fetched directly by id, prepend it and
drop any search duplicate of it so an obscure track a text search never
surfaced is still present and downloadable (#932). When it's None (the source
can't fetch one), fall back to bubbling a matching search result. Pure."""
if not link_track_id:
return tracks
target = str(link_track_id)
if linked_result is not None:
return [linked_result] + [t for t in tracks if linked_track_id(t) != target]
return bubble_linked_track_first(tracks, target)
# host substring → download source id. Only ID-downloadable streaming sources. # host substring → download source id. Only ID-downloadable streaming sources.
_HOSTS = ( _HOSTS = (
('tidal.com', 'tidal'), ('tidal.com', 'tidal'),

View file

@ -93,56 +93,6 @@ def _backfill_album_context(
album_context['image_url'] = first['url'] album_context['image_url'] = first['url']
# Placeholder album ids used when no real source album id is known — never queryable.
_SENTINEL_ALBUM_IDS = {'explicit_album', 'from_sync_modal', ''}
def backfill_album_context_from_source(
album_context: Dict[str, Any],
primary_source: Optional[str],
get_album_for_source_fn: Any,
) -> bool:
"""Hydrate a lean album context from the user's PRIMARY metadata source (#915).
Post-processing's only album backfill (:func:`hydrate_download_metadata`) goes through
``spotify_client.get_track_details`` Spotify-only. An iTunes/Deezer-primary user's
download therefore kept a lean context (no ``release_date``), so the path dropped the
``$year`` and the date defaulted to ``YYYY-01-01`` until they ran a Reorganize, which
reads the full album from the PRIMARY source. This closes that gap by doing the same:
fetch the full album from the primary source and backfill, so a download's pathing/tags
match what a later reorganize would produce.
``get_album_for_source_fn(source, album_id)`` is injected (the real one is
``core.metadata.album_tracks.get_album_for_source``) so this stays pure + testable.
No-op when: the context is already complete; the primary source is spotify (the existing
track-details path covers it); or no real source album id is present. Returns True when
it filled anything. Never raises a backfill failure must not break a download.
"""
if not isinstance(album_context, dict) or not _album_is_lean(album_context):
return False
if not primary_source or primary_source == 'spotify':
return False
album_id = album_context.get('id')
if not album_id or str(album_id) in _SENTINEL_ALBUM_IDS:
return False
try:
album = get_album_for_source_fn(primary_source, str(album_id))
except Exception as e: # noqa: BLE001 — defensive: never let backfill break a download
logger.warning("[Context] primary-source (%s) album backfill failed: %s", primary_source, e)
return False
if not isinstance(album, dict):
return False
before = album_context.get('release_date')
_backfill_album_context(album_context, {'album': album})
if album_context.get('release_date') and album_context.get('release_date') != before:
logger.info(
"[Context] Hydrated lean album context from primary source %s "
"(release_date=%r, total_tracks=%r)",
primary_source, album_context.get('release_date'), album_context.get('total_tracks'),
)
return True
def hydrate_download_metadata( def hydrate_download_metadata(
track: Any, track: Any,
track_info: Any, track_info: Any,
@ -222,5 +172,4 @@ def hydrate_download_metadata(
__all__ = [ __all__ = [
'ResolvedTrackMetadata', 'ResolvedTrackMetadata',
'hydrate_download_metadata', 'hydrate_download_metadata',
'backfill_album_context_from_source',
] ]

View file

@ -1,364 +0,0 @@
"""Wire the real cheapest-first sources for the export MBID waterfall (#903).
``mbid_resolver`` is the pure waterfall; this module supplies the real I/O behind each
source and assembles the ``resolve_fn`` the export job uses:
1. **cache** ``recording_mbid_cache`` (persistent (artist,title)->mbid).
2. **DB** a text-matched library track's ``tracks.musicbrainz_recording_id``.
3. **file** ``MUSICBRAINZ_RECORDING_ID`` tag of that track's file (when the DB row had
no recording id but the file was tagged on import).
4. **MusicBrainz** live ``match_recording(track, artist)`` (rate-limited tail).
Every source is wrapped so any failure (missing table, unreadable file, MB timeout) returns
None the waterfall just falls through, the export never breaks. ``build_resolve_fn`` also
writes a fresh non-cache hit back to the cache so the next export of the same song is free.
"""
from __future__ import annotations
import json
import threading
from typing import Any, Callable, Dict, List, Optional, Tuple
from utils.logging_config import get_logger
from core.exports.mbid_resolver import (
SRC_CACHE,
SRC_DB,
SRC_FILE,
SRC_MUSICBRAINZ,
normalize_key,
resolve_recording_mbid,
)
logger = get_logger("exports.export_sources")
def _db_match(artist: str, title: str) -> Tuple[Optional[str], Optional[str]]:
"""Text-match a library track by (artist, title); return (recording_mbid, file_path).
Either may be None. Fail-safe any DB error returns (None, None)."""
if not title:
return (None, None)
try:
from database.music_database import get_database
db = get_database()
conn = db._get_connection()
try:
cur = conn.cursor()
cur.execute(
"SELECT t.musicbrainz_recording_id, t.file_path "
"FROM tracks t JOIN artists a ON t.artist_id = a.id "
"WHERE LOWER(t.title) = LOWER(?) AND LOWER(a.name) = LOWER(?) "
"LIMIT 1",
(title, artist),
)
row = cur.fetchone()
if not row:
return (None, None)
mbid = row[0] if not hasattr(row, "keys") else row["musicbrainz_recording_id"]
fpath = row[1] if not hasattr(row, "keys") else row["file_path"]
return ((mbid or None), (fpath or None))
finally:
try:
conn.close()
except Exception: # noqa: S110
pass
except Exception as exc:
logger.debug(f"export db_match failed for '{artist} - {title}': {exc}")
return (None, None)
def db_recording_mbid(artist: str, title: str) -> Optional[str]:
"""Recording MBID stored on a matched library track (``musicbrainz_recording_id``)."""
return _db_match(artist, title)[0]
# Service → the tracks-table column carrying that service's track ID (set by enrichment).
# Trusted constants — never user input — so safe to interpolate into the SELECT below.
_SERVICE_ID_COLUMNS = {"spotify": "spotify_track_id", "deezer": "deezer_id"}
def db_service_track_id(artist: str, title: str, service: str) -> Optional[str]:
"""The service track ID (``spotify_track_id`` / ``deezer_id``) stored on a matched
library track what lets a mirrored playlist be exported BACK to Spotify/Deezer
without re-searching, since enrichment already pinned it (#945). Text-matches by
(artist, title), same as the MBID resolver. Fail-safe: any miss/error returns None."""
column = _SERVICE_ID_COLUMNS.get((service or "").lower())
if not column or not title:
return None
try:
from database.music_database import get_database
db = get_database()
conn = db._get_connection()
try:
cur = conn.cursor()
cur.execute(
f"SELECT t.{column} FROM tracks t JOIN artists a ON t.artist_id = a.id "
"WHERE LOWER(t.title) = LOWER(?) AND LOWER(a.name) = LOWER(?) LIMIT 1",
(title, artist),
)
row = cur.fetchone()
if not row:
return None
val = row[0] if not hasattr(row, "keys") else row[column]
return val or None
finally:
try:
conn.close()
except Exception: # noqa: S110
pass
except Exception as exc:
logger.debug(f"export service-id lookup failed for '{artist} - {title}' ({service}): {exc}")
return None
def build_service_resolve_fn(service: str) -> Callable[[str, str], Tuple[Optional[str], Optional[str]]]:
"""resolve_fn for service-playlist export: ``(artist, title) -> (service_track_id, 'library')``.
Plugs into ``resolve_playlist_tracks(..., id_key='service_track_id')`` exactly like the
MBID resolver plugs in for ListenBrainz."""
def resolve_fn(artist: str, title: str) -> Tuple[Optional[str], Optional[str]]:
tid = db_service_track_id(artist, title, service)
return (tid, "library" if tid else None)
return resolve_fn
def service_id_from_extra_data(track: Any, service: str) -> Optional[str]:
"""The target-service track ID the DISCOVERY step already resolved for this mirrored
track, read from its ``extra_data`` blob (#945 — Boulder: "all 50 are discovered to
Deezer already, it's not using any of that"). This is free (no API call) and reliable
(it's the same id used to mirror the track).
Only trusted when the track was discovered ON the export's target service — a
Deezer-discovered track carries a Deezer id under ``matched_data['id']``, and its
``provider`` is the service name. A ``wing_it_fallback`` provider (the low-confidence
guess path) deliberately does NOT match here, so those fall through to the library/
none path rather than risk a wrong track in the exported playlist."""
raw = track.get("extra_data") if isinstance(track, dict) else None
if not raw:
return None
try:
data = json.loads(raw) if isinstance(raw, str) else raw
except Exception:
return None
if not isinstance(data, dict) or not data.get("discovered"):
return None
if str(data.get("provider") or "").lower() != str(service or "").lower():
return None
matched = data.get("matched_data")
tid = matched.get("id") if isinstance(matched, dict) else None
return str(tid) if tid else None
def _track_field(track: Dict[str, Any], *names: str) -> str:
for n in names:
v = track.get(n)
if v:
return str(v)
return ""
def resolve_service_track_ids(
tracks: List[Dict[str, Any]],
service: str,
*,
db_fn: Optional[Callable[[str, str, str], Optional[str]]] = None,
search_id_fn: Optional[Callable[[str, str], Optional[str]]] = None,
on_progress: Optional[Callable[[int, int, Dict[str, Any]], None]] = None,
) -> Dict[str, Any]:
"""Resolve a mirrored playlist's tracks to target-service track IDs for export.
Waterfall per track: the discovery cache (``extra_data`` free + already confidently
matched) the library track's stored service id → (only when ``search_id_fn`` is
given, i.e. the opt-in backfill toggle) a confident live-search match. A track that
clears none of these is reported unmatched (caller skips it never a guessed/wrong
id). Returns ``{"resolved": [{artist, title, album, service_track_id}], "stats":
{...}}`` with ``from_cache`` / ``from_library`` / ``from_search`` / ``unmatched``
tallies for the status display.
"""
db_fn = db_fn or db_service_track_id
total = len(tracks or [])
resolved: List[Dict[str, Any]] = []
stats: Dict[str, Any] = {
"total": total, "resolved": 0, "unmatched": 0,
"from_cache": 0, "from_library": 0, "from_search": 0,
}
for i, t in enumerate(tracks or []):
if not isinstance(t, dict):
t = {}
artist = _track_field(t, "artist", "artist_name", "creator")
title = _track_field(t, "title", "track_name", "name")
album = _track_field(t, "album", "album_name", "release_name")
tid = service_id_from_extra_data(t, service)
if tid:
stats["from_cache"] += 1
else:
tid = db_fn(artist, title, service)
if tid:
stats["from_library"] += 1
elif search_id_fn is not None:
tid = search_id_fn(artist, title)
if tid:
stats["from_search"] += 1
resolved.append({"artist": artist, "title": title, "album": album,
"service_track_id": tid or None})
stats["resolved" if tid else "unmatched"] += 1
if on_progress is not None:
try:
on_progress(i + 1, total, stats)
except Exception: # noqa: S110 — a progress error must never fail the export
pass
return {"resolved": resolved, "stats": stats}
# Confidence floor for backfill, on the score_track scale (~1.5 = exact title + exact
# artist, thanks to the 1.5x artist boost). A cover/karaoke (x0.05) or a wrong-artist hit
# (no boost, caps ~1.0) can't clear this — so backfill never adds a guessed/wrong version.
BACKFILL_MIN_SCORE = 1.2
def search_service_track_id(
artist: str,
title: str,
*,
search_fn: Callable[[str], List[Any]],
min_score: float = BACKFILL_MIN_SCORE,
) -> Optional[str]:
"""Confident live-search match for export backfill (#945): search the target service
for (artist, title), rerank by relevance, and return the top match's id ONLY if it
clears the confidence floor. Below the floor None: the track is left out of the
export rather than risk a wrong/cover/karaoke version (the whole point of backfill is
coverage WITHOUT the wrong-track risk). ``search_fn(query) -> List[Track]`` is injected
so this is unit-testable without a live service."""
if not title:
return None
from core.metadata.relevance import build_combined_search_query, filter_and_rerank
query = build_combined_search_query(title, artist)
try:
candidates = list(search_fn(query) or [])
except Exception as exc:
logger.debug(f"export backfill search failed for '{artist} - {title}': {exc}")
return None
if not candidates:
return None
ranked = filter_and_rerank(
candidates, expected_title=title, expected_artist=artist, min_score=min_score,
)
if not ranked:
return None
tid = getattr(ranked[0], "id", None)
return str(tid) if tid else None
def file_recording_mbid(artist: str, title: str) -> Optional[str]:
"""Recording MBID read from the matched track's file tag (set on import post-processing)."""
_mbid, fpath = _db_match(artist, title)
if not fpath:
return None
try:
from mutagen import File as MutagenFile
audio = MutagenFile(fpath)
if audio is None or not getattr(audio, "tags", None):
return None
tags = audio.tags
# ID3 UFID (MusicBrainz), Vorbis/MP4 musicbrainz_trackid, etc.
for key in ("UFID:http://musicbrainz.org", "musicbrainz_trackid",
"MUSICBRAINZ_TRACKID", "----:com.apple.iTunes:MusicBrainz Track Id"):
try:
val = tags.get(key)
except Exception:
val = None
if not val:
continue
if hasattr(val, "data"): # ID3 UFID frame
val = val.data.decode("utf-8", "ignore")
if isinstance(val, (list, tuple)):
val = val[0] if val else ""
if isinstance(val, bytes):
val = val.decode("utf-8", "ignore")
val = str(val).strip()
if val:
return val
except Exception as exc:
logger.debug(f"export file_recording_mbid failed for {fpath}: {exc}")
return None
_mb_service = None
_mb_service_lock = threading.Lock()
def _get_mb_service():
"""Shared MusicBrainzService (client + cache + DB), created lazily so importing this
module never triggers a DB/network connection on paths that don't export."""
global _mb_service
if _mb_service is None:
with _mb_service_lock:
if _mb_service is None:
from core.musicbrainz_service import MusicBrainzService
from database.music_database import get_database
_mb_service = MusicBrainzService(get_database())
return _mb_service
def musicbrainz_recording_mbid(artist: str, title: str) -> Optional[str]:
"""Live MusicBrainz ``match_recording`` — the rate-limited tail."""
if not title:
return None
try:
svc = _get_mb_service()
if not svc:
return None
result = svc.match_recording(title, artist)
if result and result.get("mbid"):
return result["mbid"]
except Exception as exc:
logger.debug(f"export musicbrainz_recording_mbid failed for '{artist} - {title}': {exc}")
return None
def build_resolve_fn(
*,
db_fn: Callable[[str, str], Optional[str]] = db_recording_mbid,
file_fn: Callable[[str, str], Optional[str]] = file_recording_mbid,
mb_fn: Callable[[str, str], Optional[str]] = musicbrainz_recording_mbid,
cache_lookup: Optional[Callable[[str], Optional[str]]] = None,
cache_record: Optional[Callable[[str, str], bool]] = None,
) -> Callable[[str, str], Tuple[Optional[str], Optional[str]]]:
"""Assemble the export ``resolve_fn(artist, title) -> (mbid, source_label)``.
Runs cache -> DB -> file -> MusicBrainz, and writes a fresh (non-cache) hit back to the
persistent cache. All sources are injectable so the wiring is unit-testable; defaults
use the real cache module.
"""
if cache_lookup is None or cache_record is None:
from core.exports import recording_mbid_cache as _cache
cache_lookup = cache_lookup or _cache.lookup
cache_record = cache_record or _cache.record
def resolve_fn(artist: str, title: str) -> Tuple[Optional[str], Optional[str]]:
sources = [
(SRC_CACHE, lambda a, t: cache_lookup(normalize_key(a, t))),
(SRC_DB, db_fn),
(SRC_FILE, file_fn),
(SRC_MUSICBRAINZ, mb_fn),
]
mbid, label = resolve_recording_mbid(artist, title, sources)
if mbid and label and label != SRC_CACHE:
try:
cache_record(normalize_key(artist, title), mbid)
except Exception: # noqa: S110 — cache write is best-effort
pass
return (mbid, label)
return resolve_fn
__all__ = [
"build_resolve_fn",
"db_recording_mbid",
"file_recording_mbid",
"musicbrainz_recording_mbid",
]

View file

@ -1,89 +0,0 @@
"""Build a JSPF playlist (ListenBrainz-compatible) from resolved SoulSync tracks.
ListenBrainz's ``POST /1/playlist/create`` requires JSPF where **every track carries a
``identifier`` of ``https://musicbrainz.org/recording/<recording-mbid>``** text-only
entries (title/creator alone) are rejected. So a track can only be exported once we've
resolved its MusicBrainz *recording* MBID (see ``mbid_resolver``); tracks without one are
dropped here and surfaced to the user as "unmatched".
Pure + I/O-free: callers pass already-resolved track dicts, this returns the JSPF dict
(and a small coverage summary). The same JSPF is used for both the downloadable ``.jspf``
file and the direct create-playlist POST, so there's one source of truth for the shape.
"""
from __future__ import annotations
import re
from typing import Any, Dict, List, Tuple
MB_RECORDING_PREFIX = "https://musicbrainz.org/recording/"
# A MusicBrainz MBID is a canonical UUID. Validate to avoid emitting garbage identifiers
# that LB would reject (or, worse, that silently point nowhere).
_UUID_RE = re.compile(
r"^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$", re.IGNORECASE
)
def is_valid_recording_mbid(mbid: Any) -> bool:
"""True when ``mbid`` is a well-formed MusicBrainz UUID."""
return bool(mbid) and isinstance(mbid, str) and bool(_UUID_RE.match(mbid.strip()))
def _track_entry(track: Dict[str, Any]) -> Dict[str, Any] | None:
"""Build one JSPF track entry, or None if the track has no valid recording MBID."""
mbid = (track.get("recording_mbid") or "").strip() if isinstance(track.get("recording_mbid"), str) else ""
if not is_valid_recording_mbid(mbid):
return None
entry: Dict[str, Any] = {"identifier": f"{MB_RECORDING_PREFIX}{mbid}"}
# Optional, human-friendly fields — LB ignores them on create but they make the
# downloaded .jspf readable and round-trippable.
if track.get("title"):
entry["title"] = str(track["title"])
if track.get("artist"):
entry["creator"] = str(track["artist"])
if track.get("album"):
entry["album"] = str(track["album"])
return entry
def build_jspf(
title: str,
tracks: List[Dict[str, Any]],
*,
creator: str = "",
) -> Tuple[Dict[str, Any], Dict[str, int]]:
"""Build a ListenBrainz-compatible JSPF dict from resolved tracks.
``tracks`` is an ordered list of dicts with ``recording_mbid`` (required to be
included), plus optional ``title`` / ``artist`` / ``album``. Tracks without a valid
recording MBID are skipped (LB rejects them).
Returns ``(jspf, summary)`` where ``jspf`` is ``{"playlist": {...}}`` and ``summary``
is ``{"total", "included", "skipped"}`` for the coverage display.
"""
jspf_tracks: List[Dict[str, Any]] = []
for t in tracks or []:
if not isinstance(t, dict):
continue
entry = _track_entry(t)
if entry is not None:
jspf_tracks.append(entry)
playlist: Dict[str, Any] = {
"title": (title or "SoulSync Export").strip() or "SoulSync Export",
"track": jspf_tracks,
}
if creator:
playlist["creator"] = str(creator)
total = sum(1 for t in (tracks or []) if isinstance(t, dict))
summary = {
"total": total,
"included": len(jspf_tracks),
"skipped": total - len(jspf_tracks),
}
return {"playlist": playlist}, summary
__all__ = ["build_jspf", "is_valid_recording_mbid", "MB_RECORDING_PREFIX"]

View file

@ -1,88 +0,0 @@
"""Resolve a playlist track's MusicBrainz *recording* MBID, cheapest source first.
A ListenBrainz playlist export needs each track's recording MBID (``jspf_export``). A
SoulSync track can supply it from several places, in increasing cost:
1. **resolution cache** a prior (artist,title)->mbid result (persistent; reused across
playlists and runs, so the same song never costs twice).
2. **library DB** ``tracks.musicbrainz_recording_id`` (set by the MusicBrainz
enrichment worker).
3. **file tags** ``MUSICBRAINZ_RECORDING_ID`` written into the audio file on import
post-processing (catches tracks enriched at import but not via the worker).
4. **MusicBrainz lookup** a live ``match_recording(artist, title)`` (rate-limited
~1 req/s; the slow tail only hit when 13 miss).
This module is the **pure waterfall**: the caller passes ordered ``(label, fn)`` sources,
each ``fn(artist, title) -> mbid | None``, and ``resolve_recording_mbid`` returns the
first valid hit plus its label (for the live status / stats). The actual I/O (DB query,
mutagen read, MB request, cache read/write) lives in the export job that wires the real
sources so this stays trivially unit-testable and short-circuits correctly.
"""
from __future__ import annotations
import re
from typing import Any, Callable, List, Optional, Tuple
# Source labels (also used in the live-status breakdown).
SRC_CACHE = "cache"
SRC_DB = "db"
SRC_FILE = "file"
SRC_MUSICBRAINZ = "musicbrainz"
SRC_NONE = None
_UUID_RE = re.compile(
r"^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$", re.IGNORECASE
)
Source = Tuple[str, Callable[[str, str], Optional[str]]]
def _valid(mbid: Any) -> Optional[str]:
"""Return the trimmed MBID if it's a well-formed UUID, else None."""
if not isinstance(mbid, str):
return None
m = mbid.strip()
return m if _UUID_RE.match(m) else None
def normalize_key(artist: Any, title: Any) -> str:
"""Stable cache key for an (artist, title) pair — lower, punctuation-stripped,
whitespace-collapsed so trivial variations share a cache entry."""
def _n(v: Any) -> str:
s = re.sub(r"[^\w\s]", "", str(v or "").lower())
return re.sub(r"\s+", " ", s).strip()
return f"{_n(artist)}{_n(title)}"
def resolve_recording_mbid(
artist: str,
title: str,
sources: List[Source],
) -> Tuple[Optional[str], Optional[str]]:
"""Walk ``sources`` in order; return ``(mbid, label)`` of the first that yields a
valid recording MBID, or ``(None, None)`` when every source misses.
Each source is ``(label, fn)`` and ``fn(artist, title)`` returns an MBID or None. A
source that raises is treated as a miss (never aborts the waterfall) so one flaky
lookup (e.g. a MusicBrainz timeout) can't fail the whole export. Short-circuits: a
later/expensive source isn't called once an earlier one hits.
"""
for label, fn in sources or []:
try:
mbid = _valid(fn(artist, title))
except Exception:
mbid = None
if mbid:
return (mbid, label)
return (None, None)
__all__ = [
"resolve_recording_mbid",
"normalize_key",
"SRC_CACHE",
"SRC_DB",
"SRC_FILE",
"SRC_MUSICBRAINZ",
]

View file

@ -1,98 +0,0 @@
"""Orchestrate resolving a playlist's tracks to recording MBIDs for export (#903).
This is the testable heart of the export job: walk the playlist's tracks, resolve each to a
MusicBrainz recording MBID via an injected ``resolve_fn`` (which the job wires to the
cache -> DB -> file -> MusicBrainz waterfall), dedup repeated songs within the run so they
only cost one resolution, build the ordered "pseudo-playlist" of resolved tracks, and tally
live stats (resolved / unmatched / per-source / deduped) for the on-card status display.
Pure: all I/O (DB, file reads, MusicBrainz, cache) is behind ``resolve_fn`` and the optional
``on_progress`` callback, so the dedup + accounting logic is unit-testable without any
network or database. The returned ``resolved`` list feeds straight into ``jspf_export``.
"""
from __future__ import annotations
from typing import Any, Callable, Dict, List, Optional, Tuple
from core.exports.mbid_resolver import normalize_key
# resolve_fn(artist, title) -> (recording_mbid|None, source_label|None)
ResolveFn = Callable[[str, str], Tuple[Optional[str], Optional[str]]]
ProgressFn = Callable[[int, int, Dict[str, Any]], None]
def _field(track: Dict[str, Any], *names: str) -> str:
"""First non-empty value among ``names`` (handles both playlist + LB-cache shapes)."""
for n in names:
v = track.get(n)
if v:
return str(v)
return ""
def resolve_playlist_tracks(
tracks: List[Dict[str, Any]],
resolve_fn: ResolveFn,
*,
on_progress: Optional[ProgressFn] = None,
id_key: str = "recording_mbid",
) -> Dict[str, Any]:
"""Resolve every track to an ID and build the export pseudo-playlist.
``resolve_fn(artist, title) -> (id, source)`` returns whatever ID the target needs
a MusicBrainz recording MBID for ListenBrainz/JSPF (the default), or a Spotify/Deezer
track ID for service export. ``id_key`` names the field that ID lands under in each
resolved entry (defaults to ``recording_mbid`` so existing LB/JSPF callers are
untouched). The dedup + stats + ordering logic is identical regardless of ID type.
``tracks`` items may use ``artist``/``artist_name`` and ``title``/``track_name`` and
``album``/``album_name`` (both the mirrored-playlist and LB-cache shapes are accepted).
Returns ``{"resolved": [...], "stats": {...}}`` where each resolved entry is
``{artist, title, album, <id_key>}`` (the ID is None when unmatched), in original
order, and stats carries ``total, resolved, unmatched, deduped, by_source``.
"""
total = len(tracks or [])
memo: Dict[str, Tuple[Optional[str], Optional[str]]] = {}
resolved: List[Dict[str, Any]] = []
stats: Dict[str, Any] = {
"total": total, "resolved": 0, "unmatched": 0, "deduped": 0, "by_source": {},
}
for i, t in enumerate(tracks or []):
if not isinstance(t, dict):
t = {}
artist = _field(t, "artist", "artist_name", "creator")
title = _field(t, "title", "track_name", "name")
album = _field(t, "album", "album_name", "release_name")
key = normalize_key(artist, title)
if key in memo:
mbid, source = memo[key]
stats["deduped"] += 1
fresh = False
else:
mbid, source = resolve_fn(artist, title)
memo[key] = (mbid, source)
fresh = True
resolved.append({"artist": artist, "title": title, "album": album, id_key: mbid})
if mbid:
stats["resolved"] += 1
if fresh and source:
stats["by_source"][source] = stats["by_source"].get(source, 0) + 1
else:
stats["unmatched"] += 1
if on_progress is not None:
try:
on_progress(i + 1, total, stats)
except Exception: # noqa: S110 — a progress-display error must never fail the export
pass
return {"resolved": resolved, "stats": stats}
__all__ = ["resolve_playlist_tracks"]

View file

@ -1,126 +0,0 @@
"""Persistent (artist,title) -> MusicBrainz recording-MBID cache for playlist export.
The export waterfall (``core.exports.mbid_resolver``) ends in a live MusicBrainz lookup
that's rate-limited to ~1 req/s — the slow tail of exporting a big playlist. Remembering a
resolved recording MBID ONCE means the same song never costs a second lookup, across every
future export and every playlist it appears in.
Mirrors ``core.metadata.album_mbid_cache`` exactly: a tiny SQLite table, lazy DB accessor,
every function wrapped so any DB error degrades to a cache miss / no-op. If this module
breaks, exports still work they just re-resolve via the live waterfall like a cold cache.
Key is the normalized ``track_key`` from ``mbid_resolver.normalize_key(artist, title)``.
"""
from __future__ import annotations
import threading
from typing import Optional
from utils.logging_config import get_logger
logger = get_logger("exports.recording_mbid_cache")
_db_factory_lock = threading.Lock()
_db_factory = None
def _get_database():
"""Resolve the MusicDatabase singleton lazily; None on any failure (treated as miss)."""
global _db_factory
with _db_factory_lock:
if _db_factory is None:
try:
from database.music_database import get_database
_db_factory = get_database
except Exception as exc:
logger.warning(f"Recording-MBID cache: could not load database module: {exc}")
return None
try:
return _db_factory()
except Exception as exc:
logger.warning(f"Recording-MBID cache: database accessor failed: {exc}")
return None
def lookup(track_key: str) -> Optional[str]:
"""Read a cached recording MBID for ``track_key``; None on miss or any DB error."""
if not track_key:
return None
db = _get_database()
if db is None:
return None
conn = None
try:
conn = db._get_connection()
cursor = conn.cursor()
cursor.execute(
"SELECT recording_mbid FROM mb_recording_cache WHERE track_key = ? LIMIT 1",
(track_key,),
)
row = cursor.fetchone()
if row:
return (row[0] if not hasattr(row, "keys") else row["recording_mbid"]) or None
except Exception as exc:
logger.debug(f"Recording-MBID cache lookup failed: {exc}")
finally:
if conn is not None:
try:
conn.close()
except Exception: # noqa: S110 — finally cleanup
pass
return None
def record(track_key: str, recording_mbid: str) -> bool:
"""Persist ``track_key`` -> ``recording_mbid`` (idempotent). False on any failure."""
if not track_key or not recording_mbid:
return False
db = _get_database()
if db is None:
return False
conn = None
try:
conn = db._get_connection()
cursor = conn.cursor()
cursor.execute(
"INSERT OR REPLACE INTO mb_recording_cache "
"(track_key, recording_mbid, updated_at) VALUES (?, ?, CURRENT_TIMESTAMP)",
(track_key, recording_mbid),
)
conn.commit()
return True
except Exception as exc:
logger.debug(f"Recording-MBID cache record failed: {exc}")
return False
finally:
if conn is not None:
try:
conn.close()
except Exception: # noqa: S110 — finally cleanup
pass
def clear_all() -> bool:
"""Wipe the cache (tests / forced re-resolve)."""
db = _get_database()
if db is None:
return False
conn = None
try:
conn = db._get_connection()
cursor = conn.cursor()
cursor.execute("DELETE FROM mb_recording_cache")
conn.commit()
return True
except Exception as exc:
logger.warning(f"Recording-MBID cache clear failed: {exc}")
return False
finally:
if conn is not None:
try:
conn.close()
except Exception: # noqa: S110 — finally cleanup
pass
__all__ = ["lookup", "record", "clear_all"]

View file

@ -36,34 +36,9 @@ import requests as http_requests
from utils.logging_config import get_logger from utils.logging_config import get_logger
from config.settings import config_manager from config.settings import config_manager
from core.download_plugins.types import TrackResult, AlbumResult, DownloadStatus from core.download_plugins.types import TrackResult, AlbumResult, DownloadStatus
from core.quality.source_map import quality_from_tidal_tier, quality_tier_for_source
logger = get_logger("hifi_client") logger = get_logger("hifi_client")
# A media playlist whose total runtime is below this fraction of the track's
# real duration is a preview (some Monochrome instances only have 30s Tidal
# DOWNLOAD access — a 220s track comes back as ~30s of segments + ENDLIST).
_PREVIEW_DURATION_RATIO = 0.85
_EXTINF_RE = re.compile(r'#EXTINF:\s*([0-9.]+)')
def hls_total_seconds(playlist_text: str) -> float:
"""Sum the ``#EXTINF`` segment durations in an HLS media playlist."""
return sum(float(x) for x in _EXTINF_RE.findall(playlist_text or ''))
def is_preview_playlist(playlist_s: float, track_s: float,
ratio: float = _PREVIEW_DURATION_RATIO) -> bool:
"""True when the playlist runtime is far shorter than the track's real
duration (a preview). Returns False when either duration is unknown, so a
missing reference never false-positives the post-download audio guard is
the safety net.
"""
if not playlist_s or not track_s or track_s <= 0:
return False
return playlist_s < track_s * ratio
# HLS quality presets mapping to /trackManifests/ format parameters # HLS quality presets mapping to /trackManifests/ format parameters
HLS_QUALITY_MAP = { HLS_QUALITY_MAP = {
'hires': { 'hires': {
@ -165,81 +140,6 @@ def compute_new_default_pushes(all_defaults, offered, legacy_baseline, existing)
return to_add, new_offered return to_add, new_offered
_EXTINF_RE = re.compile(r'#EXTINF:\s*([0-9]+(?:\.[0-9]+)?)')
def sum_hls_segment_seconds(playlist_text: str) -> float:
"""Total audio seconds an HLS media playlist actually provides — the sum of its
``#EXTINF`` segment durations. This is the authoritative "how much audio is really
here" signal: a PREVIEW manifest serves only ~30s of segments even though the track
is full-length, so summing EXTINF catches it before we waste the download. Returns
0.0 when the playlist has no EXTINF lines (master playlists, legacy manifests) the
caller treats 0 as 'unknown', never as 'preview'."""
total = 0.0
for m in _EXTINF_RE.finditer(playlist_text or ''):
try:
total += float(m.group(1))
except (TypeError, ValueError):
continue
return total
def is_short_audio(actual_seconds: float, expected_seconds: float, threshold: float = 0.8) -> bool:
"""True when ``actual`` is meaningfully shorter than ``expected`` — i.e. a preview
clip or a truncated/corrupt download. Conservative: returns False whenever either
value is missing/zero (unknown never reject), and only trips below ``threshold``
of the expected length (previews are ~15% of full, so the margin is huge)."""
try:
a, e = float(actual_seconds or 0), float(expected_seconds or 0)
except (TypeError, ValueError):
return False
if a <= 0 or e <= 0:
return False
return a < e * threshold
def is_fake_lossless_bitrate(size_bytes, claimed_seconds, sample_rate, bits_per_sample,
channels, min_ratio: float = 0.30) -> bool:
"""True when a 'lossless' file's data is FAR too small for its claimed length — the
fingerprint of a ~30s preview whose STREAMINFO/container was faked to the full
duration (so every length header reads 'full' and only the bitrate gives it away).
Real FLAC is ~40-75% of raw PCM; a preview padded to full length implies single-digit
%. Conservative: 0 / bad inputs return False (never reject on unknowns)."""
try:
sz, secs = float(size_bytes or 0), float(claimed_seconds or 0)
sr, bits, ch = int(sample_rate or 0), int(bits_per_sample or 0), int(channels or 0)
except (TypeError, ValueError):
return False
if sz <= 0 or secs <= 0 or sr <= 0 or bits <= 0 or ch <= 0:
return False
return (sz * 8 / secs) < (sr * bits * ch) * min_ratio
def parse_ffmpeg_time(stderr_text) -> float:
"""The last ``time=HH:MM:SS.xx`` ffmpeg prints while decoding — the REAL decoded
length (immune to a faked container/STREAMINFO duration). 0.0 if not found."""
last = 0.0
for m in re.finditer(r'time=(\d+):(\d+):(\d+(?:\.\d+)?)', stderr_text or ''):
last = int(m.group(1)) * 3600 + int(m.group(2)) * 60 + float(m.group(3))
return last
def is_preview_download(real_seconds, reference_seconds, *, is_lossless, size_bytes,
sample_rate, bits_per_sample, channels):
"""Is a finished file a preview/truncated fake? Two independent signals, so it fires
even when the fakery declares full length at every layer:
1. DECODED length far below the reference (the ground truth, when a decoder ran);
2. for lossless, an impossibly-low implied bitrate (no decoder needed).
Returns ``(is_fake, reason)``."""
if real_seconds and is_short_audio(real_seconds, reference_seconds):
return True, "decoded %.0fs of %.0fs" % (real_seconds, reference_seconds)
if is_lossless and is_fake_lossless_bitrate(size_bytes, reference_seconds, sample_rate,
bits_per_sample, channels):
kbps = (float(size_bytes) * 8 / reference_seconds / 1000) if reference_seconds else 0
return True, "%.0fkbps lossless over %.0fs (far too low — a ~30s preview)" % (kbps, reference_seconds)
return False, ""
# Run the new-default push at most once per process. # Run the new-default push at most once per process.
_pushed_new_defaults = False _pushed_new_defaults = False
@ -257,10 +157,7 @@ class HiFiClient(DownloadSourcePlugin):
if download_path is None: if download_path is None:
download_path = config_manager.get('soulseek.download_path', './downloads') download_path = config_manager.get('soulseek.download_path', './downloads')
self.download_path = Path(download_path) self.download_path = Path(download_path)
try: self.download_path.mkdir(parents=True, exist_ok=True)
self.download_path.mkdir(parents=True, exist_ok=True)
except OSError as e:
logger.warning(f"Could not verify download path {self.download_path}: {e}")
self._instances = [] self._instances = []
self._instance_lock = threading.Lock() self._instance_lock = threading.Lock()
@ -725,8 +622,7 @@ class HiFiClient(DownloadSourcePlugin):
return init_uri, segment_uris return init_uri, segment_uris
def _get_hls_manifest(self, track_id: int, quality: str = 'lossless', def _get_hls_manifest(self, track_id: int, quality: str = 'lossless') -> Optional[Dict]:
expected_duration_s: float = 0) -> Optional[Dict]:
q_info = HLS_QUALITY_MAP.get(quality, HLS_QUALITY_MAP['lossless']) q_info = HLS_QUALITY_MAP.get(quality, HLS_QUALITY_MAP['lossless'])
formats = q_info['formats'] formats = q_info['formats']
@ -766,7 +662,6 @@ class HiFiClient(DownloadSourcePlugin):
logger.warning(f"Failed to parse HLS playlist for track {track_id}: {e}") logger.warning(f"Failed to parse HLS playlist for track {track_id}: {e}")
return None return None
media_text = playlist_text # the playlist that actually carries the EXTINF segments
if '#EXT-X-STREAM-INF' in playlist_text and segment_uris: if '#EXT-X-STREAM-INF' in playlist_text and segment_uris:
playlist_uri = segment_uris[0] playlist_uri = segment_uris[0]
try: try:
@ -774,26 +669,11 @@ class HiFiClient(DownloadSourcePlugin):
variant_resp = self.session.get(playlist_uri, allow_redirects=True, timeout=30) variant_resp = self.session.get(playlist_uri, allow_redirects=True, timeout=30)
variant_resp.raise_for_status() variant_resp.raise_for_status()
variant_text = variant_resp.text variant_text = variant_resp.text
media_text = variant_text
init_uri, segment_uris = self._parse_hls_playlist(variant_text, playlist_uri) init_uri, segment_uris = self._parse_hls_playlist(variant_text, playlist_uri)
except Exception as e: except Exception as e:
logger.warning(f"Failed to fetch variant playlist for track {track_id}: {e}") logger.warning(f"Failed to fetch variant playlist for track {track_id}: {e}")
return None return None
# Preview detection — some instances only have 30s Tidal DOWNLOAD
# access, returning a playlist far shorter than the real track. Decline
# it (and rotate the instance) so the orchestrator falls through to a
# real source instead of fetching a 30s file that gets quarantined.
playlist_s = hls_total_seconds(media_text)
if is_preview_playlist(playlist_s, expected_duration_s):
logger.warning(
f"HiFi manifest for track {track_id} ({quality}) is a "
f"{playlist_s:.0f}s preview of a {expected_duration_s:.0f}s track — "
f"declining this instance"
)
self._rotate_instance(self._current_instance)
return None
if init_uri: if init_uri:
logger.info(f"HiFi HLS manifest for track {track_id}: " logger.info(f"HiFi HLS manifest for track {track_id}: "
f"init segment + {len(segment_uris)} segments ({quality})") f"init segment + {len(segment_uris)} segments ({quality})")
@ -807,9 +687,6 @@ class HiFiClient(DownloadSourcePlugin):
'extension': q_info['extension'], 'extension': q_info['extension'],
'codec': q_info['codec'], 'codec': q_info['codec'],
'quality': quality, 'quality': quality,
# Real audio length the manifest provides (sum of EXTINF) — used to reject
# preview manifests before downloading. 0.0 = unknown (don't reject).
'manifest_duration': sum_hls_segment_seconds(media_text),
} }
def _get_legacy_track_manifest(self, track_id: int, quality: str = 'lossless') -> Optional[Dict]: def _get_legacy_track_manifest(self, track_id: int, quality: str = 'lossless') -> Optional[Dict]:
@ -834,57 +711,15 @@ class HiFiClient(DownloadSourcePlugin):
'quality': quality, 'quality': quality,
} }
@staticmethod
def _probe_audio_seconds(path) -> float:
"""Real decoded audio length of a finished file, via mutagen (already a dep).
0.0 on any failure the caller treats 0 as 'unknown' and never rejects on it."""
try:
from mutagen import File as _MutagenFile
mf = _MutagenFile(str(path))
info = getattr(mf, 'info', None) if mf is not None else None
if info is not None:
return float(getattr(info, 'length', 0) or 0)
except Exception as _probe_err: # noqa: BLE001
logger.debug("mutagen audio-length probe failed for %s: %s", path, _probe_err)
return 0.0
@staticmethod
def _find_ffmpeg():
ff = shutil.which('ffmpeg')
if ff:
return ff
cand = Path(__file__).parent.parent / 'tools' / ('ffmpeg.exe' if os.name == 'nt' else 'ffmpeg')
return str(cand) if cand.exists() else None
def _probe_real_seconds(self, path) -> float:
"""REAL decoded audio length via ffmpeg — decodes the actual frames, so it sees
through a faked STREAMINFO/container duration (a 30s preview claiming full
length decodes to 30s). 0.0 if ffmpeg is unavailable or on error."""
ff = self._find_ffmpeg()
if not ff:
return 0.0
try:
proc = subprocess.run(
[ff, '-hide_banner', '-nostdin', '-i', str(path), '-map', '0:a:0', '-f', 'null', '-'],
capture_output=True, text=True, timeout=180)
return parse_ffmpeg_time(proc.stderr)
except Exception:
return 0.0
@staticmethod
def _flac_props(path):
"""(sample_rate, bits_per_sample, channels) for the bitrate sanity check, or None."""
try:
from mutagen.flac import FLAC
si = FLAC(str(path)).info
return (si.sample_rate, si.bits_per_sample, si.channels)
except Exception:
return None
def _demux_flac(self, input_path: Path, output_path: Path) -> None: def _demux_flac(self, input_path: Path, output_path: Path) -> None:
ffmpeg = self._find_ffmpeg() ffmpeg = shutil.which('ffmpeg')
if not ffmpeg: if not ffmpeg:
raise RuntimeError('ffmpeg is required to demux FLAC from MP4. Install ffmpeg and retry.') tools_dir = Path(__file__).parent.parent / 'tools'
ffmpeg_candidate = tools_dir / ('ffmpeg.exe' if os.name == 'nt' else 'ffmpeg')
if ffmpeg_candidate.exists():
ffmpeg = str(ffmpeg_candidate)
else:
raise RuntimeError('ffmpeg is required to demux FLAC from MP4. Install ffmpeg and retry.')
try: try:
result = subprocess.run( result = subprocess.run(
@ -916,18 +751,13 @@ class HiFiClient(DownloadSourcePlugin):
loop = asyncio.get_event_loop() loop = asyncio.get_event_loop()
tracks = await loop.run_in_executor(None, lambda: self.search_raw(query)) tracks = await loop.run_in_executor(None, lambda: self.search_raw(query))
quality_key = quality_tier_for_source('hifi', default='lossless') quality_key = config_manager.get('hifi_download.quality', 'lossless')
q_info = HLS_QUALITY_MAP.get(quality_key, HLS_QUALITY_MAP['lossless']) q_info = HLS_QUALITY_MAP.get(quality_key, HLS_QUALITY_MAP['lossless'])
# HiFi is Tidal-backed; stamp the configured tier so the global
# ranker sees real sample_rate/bit_depth, not just 'flac'.
tier_quality = quality_from_tidal_tier(quality_key)
results = [] results = []
for t in tracks: for t in tracks:
try: try:
tr = self._to_track_result(t, q_info) tr = self._to_track_result(t, q_info)
tr.set_quality(tier_quality)
results.append(tr) results.append(tr)
except Exception as e: except Exception as e:
logger.debug(f"Skipping track result conversion: {e}") logger.debug(f"Skipping track result conversion: {e}")
@ -998,7 +828,7 @@ class HiFiClient(DownloadSourcePlugin):
) )
def _download_sync(self, download_id: str, track_id: int, display_name: str) -> Optional[str]: def _download_sync(self, download_id: str, track_id: int, display_name: str) -> Optional[str]:
quality_key = quality_tier_for_source('hifi', default='lossless') quality_key = config_manager.get('hifi_download.quality', 'lossless')
chain = ['hires', 'lossless', 'high', 'low'] chain = ['hires', 'lossless', 'high', 'low']
start = chain.index(quality_key) if quality_key in chain else 1 start = chain.index(quality_key) if quality_key in chain else 1
allow_fallback = config_manager.get('hifi_download.allow_fallback', True) allow_fallback = config_manager.get('hifi_download.allow_fallback', True)
@ -1006,26 +836,12 @@ class HiFiClient(DownloadSourcePlugin):
MIN_AUDIO_SIZE = 100 * 1024 MIN_AUDIO_SIZE = 100 * 1024
# Expected track length, drives every preview/truncation guard here:
# * _get_hls_manifest's pre-download is_preview_playlist check
# * the pre-download is_short_audio manifest check
# * the post-download is_preview_download faked-header decode check
# Best-effort: a 0 here just disables the duration checks, never rejects.
expected_s = 0.0
try:
info = self.get_track_info(track_id) or {}
expected_s = float(info.get('duration_s') or 0)
except Exception:
expected_s = 0.0
expected_duration_s = expected_s # alias for _get_hls_manifest's param name
for q_key in chain: for q_key in chain:
if self.shutdown_check and self.shutdown_check(): if self.shutdown_check and self.shutdown_check():
logger.info("Shutdown detected, aborting HiFi download") logger.info("Shutdown detected, aborting HiFi download")
return None return None
manifest_info = self._get_hls_manifest(track_id, quality=q_key, manifest_info = self._get_hls_manifest(track_id, quality=q_key)
expected_duration_s=expected_duration_s)
if ( if (
not manifest_info not manifest_info
or ( or (
@ -1036,18 +852,6 @@ class HiFiClient(DownloadSourcePlugin):
logger.warning(f"No HLS manifest at quality {q_key}, trying next") logger.warning(f"No HLS manifest at quality {q_key}, trying next")
continue continue
# Preview guard #1 (pre-download): a preview manifest serves only ~30s of
# segments for a full-length track. A preview means THIS SOURCE only has a
# preview of the track — lower quality tiers are the SAME preview — so abort
# HiFi entirely and let the orchestrator fall through to the next SOURCE
# (soulseek/youtube/…), rather than landing a lower-tier preview.
manifest_s = float(manifest_info.get('manifest_duration') or 0)
if is_short_audio(manifest_s, expected_s):
logger.warning(
"HiFi has only a PREVIEW of '%s' (manifest %.0fs of %.0fs at %s) — "
"failing HiFi so the next source is tried", display_name, manifest_s, expected_s, q_key)
return None
extension = manifest_info['extension'] extension = manifest_info['extension']
safe_name = re.sub(r'[<>:"/\\|?*]', '_', display_name) safe_name = re.sub(r'[<>:"/\\|?*]', '_', display_name)
out_filename = f"{safe_name}.{extension}" out_filename = f"{safe_name}.{extension}"
@ -1127,31 +931,6 @@ class HiFiClient(DownloadSourcePlugin):
out_path.unlink(missing_ok=True) out_path.unlink(missing_ok=True)
continue continue
# Preview guard #2 (post-download): the real catch. HiFi previews fake the
# FULL length in every header — manifest EXTINF, m4a moov, FLAC
# total_samples — so only the DECODED audio (or, for lossless, the
# bitrate) reveals the ~30s truth. Reference = the largest length any
# header claims (so the file's own faked claim becomes the bar its real
# audio must clear); is_preview_download decodes + bitrate-checks.
ref_s = max(expected_s, self._probe_audio_seconds(out_path))
real_s = self._probe_real_seconds(out_path)
props = self._flac_props(out_path) if is_flac else None
fake, why = is_preview_download(
real_s, ref_s, is_lossless=is_flac, size_bytes=final_size,
sample_rate=(props[0] if props else 0),
bits_per_sample=(props[1] if props else 0),
channels=(props[2] if props else 0))
if fake:
# A preview at this tier means the SOURCE only has a preview — every
# lower tier is the same 30s clip (and the lossy ones dodge the
# bitrate check). Abort HiFi so the orchestrator tries the next
# SOURCE, instead of cascading down into an accepted lower-tier preview.
logger.warning(
"HiFi has only a PREVIEW of '%s' (%s at %s) — failing HiFi so the "
"next source is tried", display_name, why, q_key)
out_path.unlink(missing_ok=True)
return None
logger.info(f"HiFi download complete ({q_key}): {out_path} " logger.info(f"HiFi download complete ({q_key}): {out_path} "
f"({final_size / (1024*1024):.1f} MB)") f"({final_size / (1024*1024):.1f} MB)")
return str(out_path) return str(out_path)

View file

@ -1,98 +0,0 @@
"""Canonical album grouping for the SoulSync standalone import.
SoulSync grouped imported tracks into albums by the album NAME string
(``_stable_soulsync_id("artist::album_name")``). That splits one release into
several album rows whenever the name string drifts between imports (case,
punctuation, ``(Deluxe Edition)`` suffixes, source-A-vs-B spelling), and every
downstream tool (Library Re-tag, Cover-Art Filler) then dresses each split row
in its own cover so songs that belong to one album end up with different art
(Sokhi).
This module is the pure, seam-testable heart of "group by canonical id, not
name": when an imported track carries a metadata-source RELEASE id, prefer
matching an existing album row by that id over the fragile name string, so the
SAME release always lands in ONE album row regardless of how its name was typed.
Scope (deliberate): this unifies differently-named imports of the SAME release.
It does NOT merge a track that genuinely matched a SINGLE release (a different
release id) into its parent album that needs single->album resolution upstream
and is a separate change. New imports only; existing rows are left untouched.
Pure SQL-over-a-cursor; no app singletons, so it tests against an in-memory DB.
"""
from __future__ import annotations
from typing import Any, Optional
from utils.logging_config import get_logger
logger = get_logger("imports.album_grouping")
# Album source-id columns this grouping may key on. An allowlist (not arbitrary
# interpolation) — the column name IS spliced into SQL, so it must be a known,
# trusted identifier. Mirrors get_library_source_id_columns()' 'album' values.
ALLOWED_ALBUM_SOURCE_COLS = frozenset({
"spotify_album_id",
"itunes_album_id",
"deezer_id",
"soul_id",
"discogs_id",
"musicbrainz_release_id",
})
def find_existing_soulsync_album_id(
cursor: Any,
*,
name_key_id: str,
artist_id: str,
album_name: str,
album_source_col: Optional[str] = None,
album_source_id: Optional[str] = None,
) -> Optional[str]:
"""Resolve the existing ``soulsync`` album row a track should join, or None
(caller inserts a new row keyed by ``name_key_id``).
Match precedence:
1. ``name_key_id`` the exact prior stable-name-hash id (unchanged
behaviour: a re-import with the identical name hits its own row).
2. ``album_source_col == album_source_id`` CANONICAL grouping: an
existing row already carrying THIS release's source id, so a
differently-named import of the same release unifies instead of
splitting. Only when the column is allow-listed and the id is non-empty.
3. ``(title, artist_id)`` the legacy name match (kept so nothing that
grouped before stops grouping now).
"""
cursor.execute(
"SELECT id FROM albums WHERE id = ? AND server_source = 'soulsync'",
(name_key_id,),
)
row = cursor.fetchone()
if row:
return row[0]
if album_source_col in ALLOWED_ALBUM_SOURCE_COLS and album_source_id:
try:
cursor.execute(
f"SELECT id FROM albums WHERE {album_source_col} = ? "
"AND server_source = 'soulsync' LIMIT 1",
(album_source_id,),
)
row = cursor.fetchone()
if row:
return row[0]
except Exception as exc:
# That source has no dedicated album column on this DB (e.g. Deezer
# doesn't split per-entity id columns) — fall through to the name
# match rather than break the import. Mirrors the guarded source-id
# UPDATE the caller already does on insert.
logger.debug("album source-id lookup skipped (%s): %s", album_source_col, exc)
cursor.execute(
"SELECT id FROM albums WHERE title COLLATE NOCASE = ? AND artist_id = ? "
"AND server_source = 'soulsync' LIMIT 1",
(album_name, artist_id),
)
row = cursor.fetchone()
return row[0] if row else None

View file

@ -156,11 +156,8 @@ def score_file_against_track(
score = 0.0 score = 0.0
# Title similarity (TITLE_WEIGHT). Falls back to filename stem when # Title similarity (TITLE_WEIGHT). Falls back to filename stem when
# the file has no title tag — strip a leading track-number prefix off that # the file has no title tag.
# stem (#890) so "01 - Sun It Rises" scores against "Sun It Rises".
title = file_tags.get('title') or os.path.splitext(os.path.basename(file_path))[0] title = file_tags.get('title') or os.path.splitext(os.path.basename(file_path))[0]
from core.imports.paths import strip_leading_track_number
title = strip_leading_track_number(title)
track_name = track.get('name', '') track_name = track.get('name', '')
score += similarity(title, track_name) * TITLE_WEIGHT score += similarity(title, track_name) * TITLE_WEIGHT

View file

@ -1,94 +0,0 @@
"""Resolve a track's position WITHIN its album's track list.
The bug this fixes: a track auto-downloaded from the playlist pipeline / wishlist /
watchlist is identified as belonging to an album, but the per-track position is
unknown Deezer's search/track and MusicBrainz's recording lookups don't carry a
track position (only their album endpoint does). ``detect_album_info_web`` then
leaves ``track_number = None``, the import pipeline falls through to the default-1
floor, and the file lands as ``01/1`` even though the album is known
(``core/imports/context.py``). Verified live: e.g. Deezer says "Obelisk" is track
9 of *The Grand Mirage*, but it was tagged 1/1.
This is the pure matcher: given the album's track list (fetched by the caller via
``core.metadata.album_tracks.get_album_tracks_for_source`` so this stays
source-agnostic and I/O-free) plus the track's own identifiers, return its real
``(track_number, disc_number)``. Match priority is by reliability:
1. **ISRC** an exact recording identity; trusted immediately.
2. **source track id** exact within this album.
3. **normalized title** last resort.
Returns ``(None, None)`` on no confident match, so the caller keeps its existing
behaviour (never worse than today).
"""
from __future__ import annotations
import re
from typing import Any, List, Optional, Tuple
def _norm_title(value: Any) -> str:
"""Lower, strip punctuation, collapse whitespace — for tolerant title match."""
s = re.sub(r"[^\w\s]", "", str(value or "").lower())
return re.sub(r"\s+", " ", s).strip()
def _pos_int(value: Any) -> Optional[int]:
try:
n = int(value)
except (TypeError, ValueError):
return None
return n if n >= 1 else None
def resolve_track_position_in_album(
album_tracks: List[dict],
*,
title: str = "",
track_id: str = "",
isrc: str = "",
) -> Tuple[Optional[int], Optional[int]]:
"""Return ``(track_number, disc_number)`` for this track within ``album_tracks``,
or ``(None, None)`` when no confident match is found.
``album_tracks`` is the list under ``get_album_tracks_for_source(...)['tracks']``
each entry has ``track_number`` / ``disc_number`` / ``id`` / ``name`` / ``isrc``.
Entries without a valid positive ``track_number`` are skipped. Pure: no I/O.
"""
if not album_tracks:
return (None, None)
want_isrc = str(isrc or "").strip().upper()
want_id = str(track_id or "").strip()
want_title = _norm_title(title)
by_id: Optional[Tuple[int, int]] = None
by_title: Optional[Tuple[int, int]] = None
for t in album_tracks:
if not isinstance(t, dict):
continue
tn = _pos_int(t.get("track_number"))
if tn is None:
continue
dn = _pos_int(t.get("disc_number")) or 1
# 1) ISRC — exact recording. Win immediately.
if want_isrc and str(t.get("isrc") or "").strip().upper() == want_isrc:
return (tn, dn)
# 2) source track id — exact within the album.
if by_id is None and want_id and str(t.get("id") or "").strip() == want_id:
by_id = (tn, dn)
# 3) normalized title — last resort.
if by_title is None and want_title and _norm_title(t.get("name")) == want_title:
by_title = (tn, dn)
if by_id is not None:
return by_id
if by_title is not None:
return by_title
return (None, None)
__all__ = ["resolve_track_position_in_album"]

View file

@ -8,10 +8,6 @@ from __future__ import annotations
from typing import Any, Dict, Optional from typing import Any, Dict, Optional
from utils.logging_config import get_logger
logger = get_logger("imports.context")
def _as_dict(value: Any) -> Dict[str, Any]: def _as_dict(value: Any) -> Dict[str, Any]:
return value if isinstance(value, dict) else {} return value if isinstance(value, dict) else {}
@ -183,12 +179,7 @@ def get_import_clean_title(
if not title: if not title:
track_info = get_import_track_info(context) track_info = get_import_track_info(context)
title = _first_value(track_info, "name", "title", default="") title = _first_value(track_info, "name", "title", default="")
title = str(title or default) return str(title or default)
# #890: strip a leading track-number prefix that leaked from a filename stem
# (e.g. "01 - Sun It Rises" → "Sun It Rises") so it matches the canonical title.
# Conservative — clean source titles ("7 Rings" etc.) pass through untouched.
from core.imports.paths import strip_leading_track_number
return strip_leading_track_number(title)
def get_import_clean_album( def get_import_clean_album(
@ -328,11 +319,7 @@ def build_import_album_info(
(album_info or {}).get("track_number") (album_info or {}).get("track_number")
or track_info.get("track_number") or track_info.get("track_number")
or original_search.get("track_number") or original_search.get("track_number")
# "Track 01" bug: default to 0 (the codebase's "unknown" sentinel, or 1
# same as total_tracks below), NOT 1. A fabricated 1 looks
# authoritative and blocks the pipeline's downstream recovery
# (embedded file tag / resolve chain); 0 lets it fall through.
or 0
) )
disc_number = ( disc_number = (
(album_info or {}).get("disc_number") (album_info or {}).get("disc_number")
@ -433,152 +420,20 @@ def detect_album_info_web(context, artist_context=None):
track_name.strip().lower(), track_name.strip().lower(),
artist_name.strip().lower(), artist_name.strip().lower(),
}: }:
_tn = track_info.get("track_number")
_dn = track_info.get("disc_number")
# The album is identified but discovery often doesn't carry the per-track
# POSITION — Deezer's search/track and MusicBrainz's recording lookups omit
# it (only their album endpoint has it). Without a position the pipeline
# falls through to the default-1 floor and files an album track as 01/1
# (e.g. Deezer says "Obelisk" is track 9 of The Grand Mirage). Resolve the
# REAL position from the album's own track list when we have its id.
# Fail-safe: leaves the numbers untouched on any miss, so behaviour is
# never worse than the old preserve-None-and-fall-through.
if _tn is None:
_tn, _dn = _resolve_album_position_from_source(context, artist_context, _dn)
return build_import_album_info( return build_import_album_info(
context, context,
album_info={ album_info={
"album_name": album_name, "album_name": album_name,
"track_number": _tn, # Preserve missing numbers as None so the import pipeline
"disc_number": _dn, # can fall through to ``extract_track_number_from_filename``
# at ``core/imports/pipeline.py:652`` instead of locking
# to track/disc 01 for every wishlist re-attempt.
"track_number": track_info.get("track_number"),
"disc_number": track_info.get("disc_number"),
"album_image_url": album_ctx.get("image_url", ""), "album_image_url": album_ctx.get("image_url", ""),
"confidence": 0.5, "confidence": 0.5,
}, },
force_album=True, force_album=True,
) )
# Last resort: the track matched a SINGLE with no usable album context — return None
# look up the parent ALBUM that actually contains it (gated, fail-safe).
return _resolve_single_to_parent_album(context, artist_context)
def _resolve_album_position_from_source(context, artist_context, current_disc):
"""Look up a track's real ``(track_number, disc_number)`` from its album's track
list, for the case where the album is known but discovery didn't carry a
position (Deezer/MusicBrainz search omit it).
Uses the SAME album id discovery already resolved (``get_import_source_ids``
``album_id``), so it re-homes the track onto its own album with no re-search and
no edition guessing. Matches by ISRC source track id title via the pure
``core.imports.album_position`` seam. Returns ``(None, current_disc)`` on any
miss/error so the caller falls back exactly as before never worse than today.
"""
try:
source = get_import_source(context)
ids = get_import_source_ids(context)
album_id = str(ids.get("album_id") or "")
if not source or not album_id:
return None, current_disc
from core.metadata.album_tracks import get_album_tracks_for_source
payload = get_album_tracks_for_source(source, album_id) or {}
tracks = payload.get("tracks") or []
if not tracks:
return None, current_disc
track_info = get_import_track_info(context)
original_search = get_import_original_search(context)
title = (track_info.get("name") or original_search.get("title") or "").strip()
isrc = str(track_info.get("isrc") or original_search.get("isrc") or "")
from core.imports.album_position import resolve_track_position_in_album
tn, dn = resolve_track_position_in_album(
tracks, title=title, track_id=str(ids.get("track_id") or ""), isrc=isrc)
if tn is not None:
logger.info("album-position: resolved '%s' to track %s/disc %s from album %s (%s)",
title, tn, dn, album_id, source)
return tn, (dn if dn is not None else current_disc)
return None, current_disc
except Exception as e:
logger.debug("album-position resolution failed: %s", e)
return None, current_disc
def _resolve_single_to_parent_album(context, artist_context):
"""A single-matched track -> a promoted album_info for its parent album, or
None. GATED by ``metadata_enhancement.single_to_album`` (default OFF it's a
per-import metadata lookup, so it's opt-in). Fail-safe: any miss/error returns
None so the track stays exactly as it was matched (never worse than today)."""
try:
from core.metadata.common import get_config_manager
if not get_config_manager().get("metadata_enhancement.single_to_album", False):
return None
except Exception:
return None
try:
source = get_import_source(context)
track_info = get_import_track_info(context)
original_search = get_import_original_search(context)
track_title = (track_info.get("name") or original_search.get("title") or "").strip()
artist_name = (extract_artist_name(artist_context)
or get_import_clean_artist(context, default="")).strip()
if not source or not track_title or not artist_name:
return None
artist_id = str(get_import_source_ids(context).get("artist_id") or "")
from core.metadata.album_tracks import (
get_artist_albums_for_source,
get_artist_album_tracks,
)
from core.imports.single_to_album import resolve_single_to_album
def _acc(o, *ks):
for k in ks:
v = o.get(k) if isinstance(o, dict) else getattr(o, k, None)
if v:
return v
return None
def fetch_candidates():
albums = get_artist_albums_for_source(
source, artist_id, artist_name=artist_name,
album_type="album", limit=20) or []
return [{"name": _acc(a, "name", "title"),
"album_type": _acc(a, "album_type") or "album",
"id": _acc(a, "id", "album_id")} for a in albums]
def fetch_tracks(alb):
payload = get_artist_album_tracks(
str(alb.get("id") or ""), artist_name=artist_name,
album_name=alb.get("name") or "") or {}
return [(_acc(t, "title", "name", "track_name") or "")
for t in (payload.get("tracks") or [])]
album = resolve_single_to_album(
track_title,
fetch_album_candidates=fetch_candidates,
fetch_album_tracks=fetch_tracks)
if not album or not album.get("name"):
return None
logger.info("single->album: re-homed '%s' onto parent album '%s'",
track_title, album["name"])
promoted = build_import_album_info(
context,
album_info={
"album_name": album["name"],
"track_number": track_info.get("track_number"),
"disc_number": track_info.get("disc_number"),
"album_image_url": "",
"confidence": 0.5,
},
force_album=True,
)
# build_import_album_info resolves album_name via get_import_clean_album,
# which prefers original_search.album (the SINGLE's name); override it
# with the resolved parent album so grouping + tags use the album.
promoted["album_name"] = album["name"]
return promoted
except Exception as e:
logger.debug("single->album resolution failed: %s", e)
return None

View file

@ -52,14 +52,6 @@ _DEFAULT_LENGTH_TOLERANCE_S = 3.0
_LENGTH_TOLERANCE_LONG_TRACK_S = 5.0 _LENGTH_TOLERANCE_LONG_TRACK_S = 5.0
_LONG_TRACK_THRESHOLD_S = 600.0 # 10 minutes _LONG_TRACK_THRESHOLD_S = 600.0 # 10 minutes
# A file that runs LONGER than the expected metadata is the opposite of a truncated
# download — it's almost always a different master/version (a remaster with a longer
# outro, an extended fade, an album cut vs the radio edit). The duration check exists to
# catch TRUNCATION (short files) and wildly-wrong matches, so on the auto default we allow
# more drift in the longer direction and keep the tight bound for short files. A wrong-song
# match still trips this — it's usually off by far more than 15s. (#937)
_LONGER_VERSION_TOLERANCE_S = 15.0
# Upper bound for the user-configurable override. Anything past 60s # Upper bound for the user-configurable override. Anything past 60s
# means the check is effectively off — cap defends against accidental # means the check is effectively off — cap defends against accidental
# nonsense like 9999 making logs misleading. Users who genuinely want # nonsense like 9999 making logs misleading. Users who genuinely want
@ -250,32 +242,18 @@ def check_audio_integrity(
if expected_length_s > _LONG_TRACK_THRESHOLD_S if expected_length_s > _LONG_TRACK_THRESHOLD_S
else _DEFAULT_LENGTH_TOLERANCE_S else _DEFAULT_LENGTH_TOLERANCE_S
) )
user_pinned_tolerance = False
else:
user_pinned_tolerance = True
checks["length_tolerance_s"] = length_tolerance_s checks["length_tolerance_s"] = length_tolerance_s
# Positive drift = the file runs LONGER than expected (not truncation). On the auto drift_s = abs(actual_length_s - expected_length_s)
# default, give the longer direction more room so legit longer masters/versions aren't
# quarantined (#937); a user-pinned tolerance is honoured symmetrically.
signed_drift_s = actual_length_s - expected_length_s
drift_s = abs(signed_drift_s)
checks["length_drift_s"] = drift_s checks["length_drift_s"] = drift_s
effective_tolerance_s = length_tolerance_s
if signed_drift_s > 0 and not user_pinned_tolerance:
effective_tolerance_s = max(length_tolerance_s, _LONGER_VERSION_TOLERANCE_S)
checks["effective_tolerance_s"] = effective_tolerance_s
if drift_s > effective_tolerance_s: if drift_s > length_tolerance_s:
runs_long = signed_drift_s > 0
return IntegrityResult( return IntegrityResult(
ok=False, ok=False,
reason=f"Duration mismatch: file is {actual_length_s:.1f}s, " reason=f"Duration mismatch: file is {actual_length_s:.1f}s, "
f"expected {expected_length_s:.1f}s " f"expected {expected_length_s:.1f}s "
f"(drift {drift_s:.1f}s > tolerance {effective_tolerance_s:.1f}s) — " f"(drift {drift_s:.1f}s > tolerance {length_tolerance_s:.1f}s) — "
+ ("runs longer than expected — likely a different version/master or wrong file" "likely truncated download or wrong file matched",
if runs_long
else "likely truncated download or wrong file matched"),
checks=checks, checks=checks,
) )

View file

@ -2,7 +2,6 @@
from __future__ import annotations from __future__ import annotations
import errno
import logging import logging
import os import os
import re import re
@ -103,41 +102,6 @@ def cleanup_slskd_dedup_siblings(source_path) -> List[str]:
return deleted return deleted
def _atomic_cross_device_move(src: Path, dst: Path) -> None:
"""Move ``src`` to ``dst`` across filesystems WITHOUT ever exposing a partial file at
the final path.
Copies into a hidden temp sibling of ``dst`` (same filesystem), fsyncs, then does an
atomic ``os.replace`` into place, then deletes ``src``. A media-server file watcher
(Jellyfin/Plex real-time monitoring) therefore only ever indexes the COMPLETE file
an incremental in-place copy was what Jellyfin could catch mid-write and cache with
null/incomplete metadata (tracks landing with no disc). Cleans up the temp on failure.
"""
src, dst = Path(src), Path(dst)
tmp = dst.parent / f".{dst.name}.ssync-tmp"
try:
with open(src, "rb") as f_src, open(tmp, "wb") as f_dst:
shutil.copyfileobj(f_src, f_dst)
f_dst.flush()
os.fsync(f_dst.fileno())
try:
shutil.copystat(str(src), str(tmp)) # preserve mtime/permissions (copy2-like)
except OSError:
pass
os.replace(str(tmp), str(dst)) # atomic within dst's filesystem
except Exception:
try:
if tmp.exists():
tmp.unlink()
except OSError:
pass
raise
try:
src.unlink()
except OSError:
logger.info(f"Could not delete source after cross-device move (may be owned by another process): {src}")
def safe_move_file(src, dst): def safe_move_file(src, dst):
"""Move a file safely across filesystems.""" """Move a file safely across filesystems."""
src = Path(src) src = Path(src)
@ -165,13 +129,7 @@ def safe_move_file(src, dst):
break break
try: try:
# Same-filesystem move: an atomic rename that also overwrites dst. A media-server shutil.move(str(src), str(dst))
# watcher (Jellyfin/Plex real-time monitoring) therefore never sees a partial file
# at the final name. Cross-filesystem raises EXDEV (some network mounts raise
# EPERM/EACCES) and we copy atomically below instead of letting the move write the
# destination incrementally — the partial-file-at-final-name is what caused tracks
# to land in Jellyfin with null/incomplete metadata (no disc).
os.replace(str(src), str(dst))
return return
except FileNotFoundError: except FileNotFoundError:
if dst.exists(): if dst.exists():
@ -179,6 +137,8 @@ def safe_move_file(src, dst):
return return
raise raise
except (OSError, PermissionError) as e: except (OSError, PermissionError) as e:
error_msg = str(e).lower()
if dst.exists() and dst.stat().st_size > 0: if dst.exists() and dst.stat().st_size > 0:
logger.warning(f"Move raised {type(e).__name__} but destination exists, treating as success: {e}") logger.warning(f"Move raised {type(e).__name__} but destination exists, treating as success: {e}")
try: try:
@ -187,21 +147,23 @@ def safe_move_file(src, dst):
logger.info(f"Could not delete source file (may be owned by another process): {src}") logger.info(f"Could not delete source file (may be owned by another process): {src}")
return return
error_msg = str(e).lower() if "cross-device" in error_msg or "operation not permitted" in error_msg or "permission denied" in error_msg:
cross_device = ( logger.warning(f"Cross-device move detected, using fallback copy method: {e}")
getattr(e, "errno", None) in (errno.EXDEV, errno.EPERM, errno.EACCES)
or "cross-device" in error_msg
or "operation not permitted" in error_msg
or "permission denied" in error_msg
)
if cross_device:
logger.warning(f"Cross-device move, using atomic copy+rename: {e}")
try: try:
_atomic_cross_device_move(src, dst) with open(src, "rb") as f_src:
logger.info(f"Successfully moved file atomically across filesystems: {src} -> {dst}") with open(dst, "wb") as f_dst:
shutil.copyfileobj(f_src, f_dst)
f_dst.flush()
os.fsync(f_dst.fileno())
try:
src.unlink()
except PermissionError:
logger.info(f"Could not delete source file (may be owned by another process): {src}")
logger.info(f"Successfully moved file using fallback method: {src} -> {dst}")
return return
except Exception as fallback_error: except Exception as fallback_error:
logger.error(f"Atomic cross-device move failed: {fallback_error}") logger.error(f"Fallback copy also failed: {fallback_error}")
raise raise
raise raise
@ -230,9 +192,6 @@ def get_audio_quality_string(file_path):
if ext == ".flac": if ext == ".flac":
from mutagen.flac import FLAC from mutagen.flac import FLAC
audio = FLAC(file_path) audio = FLAC(file_path)
sr = getattr(audio.info, "sample_rate", 0) or 0
if sr:
return f"FLAC {audio.info.bits_per_sample}bit/{sr / 1000:g}kHz"
return f"FLAC {audio.info.bits_per_sample}bit" return f"FLAC {audio.info.bits_per_sample}bit"
if ext == ".mp3": if ext == ".mp3":
@ -266,122 +225,6 @@ def get_audio_quality_string(file_path):
return "" return ""
def probe_audio_quality(file_path: str):
"""Read the actual file and return an AudioQuality with real measured values.
Uses mutagen to extract sample_rate, bit_depth, and bitrate from the
downloaded file these are ground-truth values, not estimates.
Returns None when the file cannot be read.
"""
from core.quality.model import AudioQuality
try:
ext = os.path.splitext(file_path)[1].lower().lstrip('.')
if ext == 'flac':
from mutagen.flac import FLAC
audio = FLAC(file_path)
return AudioQuality(
format='flac',
bitrate=audio.info.bitrate // 1000 if audio.info.bitrate else None,
sample_rate=audio.info.sample_rate,
bit_depth=audio.info.bits_per_sample,
)
if ext == 'mp3':
from mutagen.mp3 import MP3
audio = MP3(file_path)
return AudioQuality(
format='mp3',
bitrate=audio.info.bitrate // 1000,
sample_rate=audio.info.sample_rate,
)
if ext in ('m4a', 'aac', 'mp4'):
from mutagen.mp4 import MP4
audio = MP4(file_path)
# .m4a can carry AAC (lossy) OR ALAC (lossless) — only the real
# codec tells them apart, which is why extension-based classification
# defaults to 'aac' and we correct it here from the probed file.
codec = (getattr(audio.info, 'codec', '') or '').lower()
if 'alac' in codec:
return AudioQuality(
format='alac',
bitrate=audio.info.bitrate // 1000 if audio.info.bitrate else None,
sample_rate=audio.info.sample_rate,
bit_depth=getattr(audio.info, 'bits_per_sample', None) or None,
)
return AudioQuality(
format='aac',
bitrate=audio.info.bitrate // 1000,
sample_rate=audio.info.sample_rate,
)
if ext == 'ogg':
from mutagen.oggvorbis import OggVorbis
audio = OggVorbis(file_path)
return AudioQuality(
format='ogg',
bitrate=audio.info.bitrate // 1000,
sample_rate=audio.info.sample_rate,
)
if ext == 'opus':
from mutagen.oggopus import OggOpus
audio = OggOpus(file_path)
return AudioQuality(
format='opus',
bitrate=audio.info.bitrate // 1000,
sample_rate=audio.info.sample_rate,
)
if ext in ('wav', 'aiff', 'aif'):
# AIFF must use mutagen.aiff.AIFF — WAVE() can't parse it and would
# raise, making the file fail open and silently bypass the quality
# filter. Both are uncompressed PCM, so they share the 'wav' tier.
if ext == 'wav':
from mutagen.wave import WAVE
audio = WAVE(file_path)
else:
from mutagen.aiff import AIFF
audio = AIFF(file_path)
return AudioQuality(
format='wav',
bitrate=audio.info.bitrate // 1000 if audio.info.bitrate else None,
sample_rate=audio.info.sample_rate,
bit_depth=getattr(audio.info, 'bits_per_sample', None),
)
if ext == 'wma':
from mutagen.asf import ASF
audio = ASF(file_path)
return AudioQuality(
format='wma',
bitrate=audio.info.bitrate // 1000 if audio.info.bitrate else None,
sample_rate=getattr(audio.info, 'sample_rate', None),
)
if ext in ('dsf', 'dff'):
# DSD (DSD Stream File / DSDIFF) — 1-bit hi-res lossless (#939). mutagen
# reads .dsf (rate/bitrate/bit_depth); .dff has no mutagen reader, so it
# still classifies as the lossless 'dsf' tier just without measured detail.
sr = bd = br = None
if ext == 'dsf':
try:
from mutagen.dsf import DSF
info = DSF(file_path).info
sr = getattr(info, 'sample_rate', None)
bd = getattr(info, 'bits_per_sample', None)
br = info.bitrate // 1000 if getattr(info, 'bitrate', None) else None
except Exception: # noqa: S110 — unreadable DSF still classifies lossless, just without measured detail
pass
return AudioQuality(format='dsf', bitrate=br, sample_rate=sr, bit_depth=bd)
return None
except Exception as e:
logger.debug("probe_audio_quality failed for %s: %s", file_path, e)
return None
def get_quality_tier_from_extension(file_path): def get_quality_tier_from_extension(file_path):
"""Classify a file extension into a quality tier.""" """Classify a file extension into a quality tier."""
if not file_path: if not file_path:
@ -524,29 +367,14 @@ def downsample_hires_flac(final_path, context):
return None return None
def m4a_codec(path):
"""Codec of an .m4a/.mp4 container ('alac', 'aac', …) or None — lets the
lossless check tell ALAC (lossless) from AAC (lossy), since both are .m4a."""
try:
from mutagen.mp4 import MP4
return (getattr(MP4(path).info, 'codec', '') or '').lower() or None
except Exception:
return None
def create_lossy_copy(final_path): def create_lossy_copy(final_path):
"""Convert a lossless file (FLAC / ALAC / WAV / AIFF / DSD) to a lossy copy """Convert a FLAC file to a lossy copy using the configured codec."""
using the configured codec. Non-lossless inputs are skipped (#941).""" from mutagen.flac import FLAC
from core.quality.lossless import (
is_lossless_audio_path,
lossy_output_would_overwrite_source,
)
if not config_manager.get("lossy_copy.enabled", False): if not config_manager.get("lossy_copy.enabled", False):
return None return None
# Was FLAC-only; now any lossless source. .m4a is probed (ALAC vs AAC). if os.path.splitext(final_path)[1].lower() != ".flac":
if not is_lossless_audio_path(final_path, probe_codec=m4a_codec):
return None return None
codec = config_manager.get("lossy_copy.codec", "mp3").lower() codec = config_manager.get("lossy_copy.codec", "mp3").lower()
@ -575,16 +403,6 @@ def create_lossy_copy(final_path):
out_basename = out_basename.replace(original_quality, quality_label) out_basename = out_basename.replace(original_quality, quality_label)
out_path = os.path.join(os.path.dirname(out_path), out_basename) out_path = os.path.join(os.path.dirname(out_path), out_basename)
# Safety invariant: never write the lossy copy over its own source (an .m4a
# ALAC source + AAC target lands on the same .m4a path). ffmpeg runs with -y,
# so this guard MUST precede it — the later delete-original guard is too late.
if lossy_output_would_overwrite_source(final_path, out_path):
logger.info(
f"[Lossy Copy] Skipping — {codec.upper()} output would overwrite the "
f"source: {os.path.basename(final_path)}"
)
return None
ffmpeg_bin = shutil.which("ffmpeg") ffmpeg_bin = shutil.which("ffmpeg")
if not ffmpeg_bin: if not ffmpeg_bin:
local = os.path.join(os.path.dirname(__file__), "tools", "ffmpeg") local = os.path.join(os.path.dirname(__file__), "tools", "ffmpeg")

View file

@ -105,72 +105,30 @@ def move_to_quarantine(file_path: str, context: dict, reason: str, automation_en
def check_flac_bit_depth(file_path: str, context: dict) -> Optional[str]: def check_flac_bit_depth(file_path: str, context: dict) -> Optional[str]:
"""Legacy wrapper — delegates to check_quality_target. """Return a rejection message if a FLAC file violates the configured bit depth."""
if not context.get("_audio_quality", "").startswith("FLAC"):
Kept for callers that still pass trigger='bit_depth'; the new guard
covers bit_depth as part of the full quality target check.
"""
return check_quality_target(file_path, context)
def check_quality_target(file_path: str, context: dict) -> Optional[str]:
"""Return a rejection message when the downloaded file does not satisfy
the user's quality priority list.
Probes the actual file with mutagen (ground-truth sample_rate,
bit_depth, bitrate) and checks it against the profile's
``ranked_targets``. Falls back gracefully when fallback_enabled=True.
Works for all formats and all download sources no Soulseek-specific
logic here.
"""
from core.imports.file_ops import probe_audio_quality
from core.quality.selection import targets_from_profile, quality_meets_profile
# Master toggle (Settings → Import). When OFF, the quality check is skipped
# entirely and files import regardless of quality — the user opted out of
# quality-filtering on import. Default ON preserves existing behaviour. The
# library Quality Upgrade scanner still flags below-profile files either way.
if _get_config_manager().get("import.quality_filter_enabled", True) is False:
logger.debug(
"[QualityGuard] import.quality_filter_enabled=False — skipping quality "
"filter for %s", os.path.basename(file_path),
)
return None return None
aq = probe_audio_quality(file_path) quality_profile = MusicDatabase().get_quality_profile()
if aq is None: flac_config = quality_profile.get("qualities", {}).get("flac", {})
logger.debug("[QualityGuard] Could not probe %s — skipping check", os.path.basename(file_path)) flac_pref = flac_config.get("bit_depth", "any")
if flac_pref == "any":
return None return None
profile = MusicDatabase().get_quality_profile() actual_bits = context["_audio_quality"].replace("FLAC ", "").replace("bit", "")
targets, fallback_enabled = targets_from_profile(profile) if actual_bits == flac_pref:
if not targets:
return None return None
flac_fallback = flac_config.get("bit_depth_fallback", True)
downsample_enabled = _get_config_manager().get("lossy_copy.downsample_hires", False) downsample_enabled = _get_config_manager().get("lossy_copy.downsample_hires", False)
matched = quality_meets_profile(aq, targets)
track_info = context.get("track_info", {}) track_info = context.get("track_info", {})
track_name = track_info.get("name", os.path.basename(file_path)) track_name = track_info.get("name", os.path.basename(file_path))
actual_label = aq.label()
if matched: if flac_fallback or downsample_enabled:
logger.info("[QualityGuard] %s meets profile: %s", track_name, actual_label) if downsample_enabled:
logger.info("[FLAC Downsample] Accepted %s-bit FLAC (will be downsampled to %s-bit): %s", actual_bits, flac_pref, track_name)
else:
logger.warning("[FLAC Fallback] Accepted %s-bit FLAC (preferred %s-bit): %s", actual_bits, flac_pref, track_name)
return None return None
# No target matched return f"FLAC bit depth mismatch: file is {actual_bits}-bit, preference is {flac_pref}-bit"
best_label = targets[0].label if targets else "?"
if fallback_enabled or downsample_enabled:
logger.warning(
"[QualityGuard] %s did not match any target (got %s, wanted %s) — accepting via fallback",
track_name, actual_label, best_label,
)
return None
return (
f"Quality mismatch: file is {actual_label}, "
f"does not satisfy any configured target (best wanted: {best_label})"
)

View file

@ -158,33 +158,6 @@ def clean_track_title(track_title: str, artist_name: str) -> str:
return cleaned if cleaned else original return cleaned if cleaned else original
# A leading track-number prefix is EITHER a zero-padded number (01, 04, 099 — no
# real song title starts with one), OR a plain number followed by a real separator
# AND a space ("3 - ", "12. "). Deliberately NOT a bare "number space word", so it
# leaves "7 Rings", "99 Luftballons", "50 Ways to Leave Your Lover" and
# "1-800-273-8255" untouched.
_TRACK_NUM_PREFIX_RE = re.compile(r"^\s*(?:0\d{1,2}[\s._)\-]*|\d{1,3}\s*[._)\-]\s+)(?=\S)")
def strip_leading_track_number(title: str) -> str:
"""Conservatively remove a leading track-number prefix from a track title.
Fixes #890 — files named ``01 - Sun It Rises.flac`` whose stem leaks into the
title as ``01 - Sun It Rises``, which then never matches the canonical
``Sun It Rises`` (false "missing"). Only strips an unambiguous track-number
prefix; a coincidental leading number that's part of the title is preserved, and
it never reduces a title to empty or a bare number."""
s = (title or "").strip()
if not s:
return title or ""
stripped = _TRACK_NUM_PREFIX_RE.sub("", s, count=1).strip()
# Keep the original if stripping left nothing real — empty, a bare number, or
# only punctuation (e.g. "01 - " → "-"). A real title has a letter/digit.
if stripped.isdigit() or not re.search(r"[^\W_]", stripped):
return s
return stripped
def get_album_type_display(raw_type, track_count) -> str: def get_album_type_display(raw_type, track_count) -> str:

View file

@ -34,11 +34,9 @@ from core.imports.context import (
) )
from core.imports.file_integrity import check_audio_integrity, expected_duration_for_check, resolve_duration_tolerance from core.imports.file_integrity import check_audio_integrity, expected_duration_for_check, resolve_duration_tolerance
from core.imports.filename import extract_track_number_from_filename from core.imports.filename import extract_track_number_from_filename
from core.imports.guards import check_flac_bit_depth, check_quality_target, move_to_quarantine from core.imports.guards import check_flac_bit_depth, move_to_quarantine
from core.imports.silence import detect_broken_audio
from core.imports.quarantine import ( from core.imports.quarantine import (
approve_quarantine_entry, approve_quarantine_entry,
delete_quarantine_entry,
entry_id_from_quarantined_filename, entry_id_from_quarantined_filename,
list_quarantine_entries, list_quarantine_entries,
) )
@ -162,9 +160,7 @@ def import_rejection_reason(context: dict) -> str | None:
f"{context.get('_acoustid_failure_msg', 'fingerprint mismatch')}" f"{context.get('_acoustid_failure_msg', 'fingerprint mismatch')}"
) )
if context.get('_bitdepth_rejected'): if context.get('_bitdepth_rejected'):
return "rejected by quality filter" return "rejected by bit-depth filter"
if context.get('_silence_rejected'):
return "rejected by audio guard (incomplete or silent audio)"
if context.get('_race_guard_failed'): if context.get('_race_guard_failed'):
return "source file disappeared before import completed" return "source file disappeared before import completed"
return None return None
@ -325,14 +321,11 @@ def post_process_matched_download(context_key, context, file_path, runtime, meta
_mark_task_quarantined(context, quarantine_path) _mark_task_quarantined(context, quarantine_path)
logger.error(f"File quarantined due to integrity failure: {quarantine_path}") logger.error(f"File quarantined due to integrity failure: {quarantine_path}")
except Exception as quarantine_error: except Exception as quarantine_error:
# Quarantine MOVE failed (e.g. cross-device / permission on a NAS). logger.error(f"Quarantine failed ({quarantine_error}), deleting broken file: {file_path}")
# Do NOT delete — destroying a download we couldn't even quarantine is try:
# data loss and forces a re-download. Leave it in place so it can be os.remove(file_path)
# retried; the task is still marked failed below either way (#kettui). except Exception as del_error:
logger.error( logger.error(f"Could not delete broken file either: {del_error}")
f"Quarantine failed ({quarantine_error}) — leaving file in place "
f"for retry (not deleting): {file_path}"
)
with matched_context_lock: with matched_context_lock:
if context_key in matched_downloads_context: if context_key in matched_downloads_context:
@ -360,118 +353,6 @@ def post_process_matched_download(context_key, context, file_path, runtime, meta
f"drift={integrity.checks.get('length_drift_s', 'n/a')})" f"drift={integrity.checks.get('length_drift_s', 'n/a')})"
) )
# Audio-completeness guard — runs right where the length is verified,
# BEFORE the AcoustID/quality gates, so a truncated file (container
# claims the full length but only ~30s actually decodes, e.g. HiFi/
# Monochrome HLS assembly) or a mostly-silent file is caught regardless
# of its quality verdict and gets the right reason. Same quarantine +
# next-candidate retry pattern as the integrity check.
#
# Opt-in (default OFF): this is the one check that fully DECODES the file
# with ffmpeg, so it is the most CPU-expensive step in post-processing.
# Most preview/truncation cases are already caught at the source
# (HiFi/Qobuz have their own guards), so it stays off unless the user
# turns it on under Settings → Post-processing.
_audio_guard_enabled = config_manager.get('post_processing.audio_completeness_check', False)
_skip_audio = (not _audio_guard_enabled) or _should_skip_quarantine_check(context, 'silence')
audio_reason = None if _skip_audio else detect_broken_audio(file_path)
if audio_reason:
logger.error(f"[AudioGuard] Rejected {_basename}: {audio_reason}")
context['_silence_rejected'] = True
try:
quarantine_path = move_to_quarantine(
file_path, context, audio_reason, automation_engine,
trigger='silence',
)
_mark_task_quarantined(context, quarantine_path)
logger.warning("File quarantined — incomplete/silent audio: %s", quarantine_path)
except Exception as quarantine_error:
# Don't delete a file we couldn't quarantine — leave it for retry
# instead of forcing a re-download (data loss). See integrity block.
logger.error(
f"Quarantine failed ({quarantine_error}) — leaving file in place "
f"for retry (not deleting): {file_path}"
)
with matched_context_lock:
matched_downloads_context.pop(context_key, None)
task_id = context.get('task_id')
batch_id = context.get('batch_id')
# Try the next-best candidate before giving up — same pattern as
# the integrity / AcoustID / quality failures.
if _requeue_quarantined_task_for_retry(task_id, batch_id, 'silence'):
logger.info(
"Incomplete/silent audio for task %s — retrying next-best candidate: %s",
task_id, audio_reason,
)
return
if task_id:
with tasks_lock:
if task_id in download_tasks:
download_tasks[task_id]['status'] = 'failed'
download_tasks[task_id]['error_message'] = f"Audio guard: {audio_reason}"
if task_id and batch_id:
_notify_download_completed(batch_id, task_id, success=False)
return
# QUALITY GATE — runs BEFORE AcoustID so (1) a wrong-quality file is
# rejected without paying for an AcoustID fingerprint, and (2) the real
# audio quality is recorded on the context (→ quarantine sidecar) for
# EVERY trigger, so it's known when reviewing/approving any quarantined
# file. force_import still never fires on a quality mismatch.
context['_audio_quality'] = get_audio_quality_string(file_path)
if context['_audio_quality']:
logger.info(f"Audio quality detected: {context['_audio_quality']}")
_skip_quality = _should_skip_quarantine_check(context, 'bit_depth') or \
_should_skip_quarantine_check(context, 'quality')
rejection_reason = None if _skip_quality else check_quality_target(file_path, context)
if _skip_quality:
logger.info(f"[QualityGuard] Skipped (user approval) for {_basename}")
if rejection_reason:
try:
quarantine_path = move_to_quarantine(
file_path,
context,
rejection_reason,
automation_engine,
trigger='quality',
)
_mark_task_quarantined(context, quarantine_path)
logger.info(f"File quarantined due to quality mismatch: {quarantine_path}")
except Exception as quarantine_error:
# Don't delete a file we couldn't quarantine — leave it for retry
# instead of forcing a re-download (data loss). See integrity block.
logger.error(
f"Quarantine failed ({quarantine_error}) — leaving file in place "
f"for retry (not deleting): {file_path}"
)
context['_bitdepth_rejected'] = True
task_id = context.get('task_id')
batch_id = context.get('batch_id')
with matched_context_lock:
matched_downloads_context.pop(context_key, None)
# Try the next-best candidate before giving up — same pattern
# as AcoustID and integrity failures.
if _requeue_quarantined_task_for_retry(task_id, batch_id, 'quality'):
logger.info(
"Quality mismatch for task %s — retrying next-best candidate: %s",
task_id, rejection_reason,
)
return
if task_id:
with tasks_lock:
if task_id in download_tasks:
download_tasks[task_id]['status'] = 'failed'
download_tasks[task_id]['error_message'] = f"Quality filter: {rejection_reason}"
if task_id and batch_id:
_notify_download_completed(batch_id, task_id, success=False)
return
_skip_acoustid = _should_skip_quarantine_check(context, 'acoustid') _skip_acoustid = _should_skip_quarantine_check(context, 'acoustid')
if _skip_acoustid: if _skip_acoustid:
logger.info(f"[AcoustID] Skipped (user approval) for {_basename}") logger.info(f"[AcoustID] Skipped (user approval) for {_basename}")
@ -509,44 +390,24 @@ def post_process_matched_download(context_key, context, file_path, runtime, meta
logger.info(f"AcoustID verification result: {verification_result.value} - {verification_msg}") logger.info(f"AcoustID verification result: {verification_result.value} - {verification_msg}")
context['_acoustid_result'] = verification_result.value context['_acoustid_result'] = verification_result.value
# Fail-closed mode: when the user requires a hard AcoustID if verification_result == VerificationResult.FAIL:
# PASS, a SKIP (ran but couldn't confirm — no fingerprint
# match / cross-script metadata) is treated like a FAIL:
# quarantine + try the next candidate, instead of importing
# an unverified file. ERROR (rate-limit / infra) is NOT
# blocked — that would stall the whole pipeline during an
# outage; those still import with their existing flag.
require_verified = config_manager.get('acoustid.require_verified', False)
_skip_as_fail = (
require_verified
and verification_result == VerificationResult.SKIP
)
if _skip_as_fail:
verification_msg = (
f"AcoustID could not confirm the track and 'require verified' "
f"is on — rejecting unverified file ({verification_msg})"
)
logger.warning("[AcoustID] Require-verified: SKIP treated as FAIL — %s", verification_msg)
if verification_result == VerificationResult.FAIL or _skip_as_fail:
try: try:
quarantine_path = move_to_quarantine( quarantine_path = move_to_quarantine(
file_path, file_path,
context, context,
verification_msg, verification_msg,
automation_engine, automation_engine,
trigger='acoustid_unverified' if _skip_as_fail else 'acoustid', trigger='acoustid',
) )
_mark_task_quarantined(context, quarantine_path) _mark_task_quarantined(context, quarantine_path)
logger.error(f"File quarantined due to verification failure: {quarantine_path}") logger.error(f"File quarantined due to verification failure: {quarantine_path}")
except Exception as quarantine_error: except Exception as quarantine_error:
# Don't delete a file we couldn't quarantine — leave it for logger.error(f"Quarantine failed ({quarantine_error}), deleting wrong file: {file_path}")
# retry instead of forcing a re-download (data loss). The logger.error(f"Quarantine failed, deleting wrong file: {file_path}")
# task is still marked failed / requeued below. See integrity. try:
logger.error( os.remove(file_path)
f"Quarantine failed ({quarantine_error}) — leaving file " except Exception as del_error:
f"in place for retry (not deleting): {file_path}" logger.error(f"Could not delete wrong file either: {del_error}")
)
context['_acoustid_quarantined'] = True context['_acoustid_quarantined'] = True
context['_acoustid_failure_msg'] = verification_msg context['_acoustid_failure_msg'] = verification_msg
@ -716,6 +577,48 @@ def post_process_matched_download(context_key, context, file_path, runtime, meta
album_info.get('album_name', 'None'), album_info.get('album_name', 'None'),
) )
context['_audio_quality'] = get_audio_quality_string(file_path)
if context['_audio_quality']:
logger.info(f"Audio quality detected: {context['_audio_quality']}")
_skip_bit_depth = _should_skip_quarantine_check(context, 'bit_depth')
rejection_reason = None if _skip_bit_depth else check_flac_bit_depth(file_path, context)
if _skip_bit_depth:
logger.info(f"[BitDepth] Skipped (user approval) for {_basename}")
if rejection_reason:
try:
quarantine_path = move_to_quarantine(
file_path,
context,
rejection_reason,
automation_engine,
trigger='bit_depth',
)
_mark_task_quarantined(context, quarantine_path)
logger.info(f"File quarantined due to bit depth filter: {quarantine_path}")
except Exception as quarantine_error:
logger.error(f"Quarantine failed ({quarantine_error}), deleting file: {file_path}")
try:
os.remove(file_path)
except Exception as e:
logger.debug("delete quarantine fallback: %s", e)
context['_bitdepth_rejected'] = True
with matched_context_lock:
if context_key in matched_downloads_context:
del matched_downloads_context[context_key]
task_id = context.get('task_id')
batch_id = context.get('batch_id')
if task_id:
with tasks_lock:
if task_id in download_tasks:
download_tasks[task_id]['status'] = 'failed'
download_tasks[task_id]['error_message'] = f"Bit depth filter: {rejection_reason}"
if task_id and batch_id:
_notify_download_completed(batch_id, task_id, success=False)
return
file_ext = os.path.splitext(file_path)[1] file_ext = os.path.splitext(file_path)[1]
clean_track_name = get_import_clean_title( clean_track_name = get_import_clean_title(
context, context,
@ -726,17 +629,9 @@ def post_process_matched_download(context_key, context, file_path, runtime, meta
# See ``core/imports/track_number.py`` for the resolution # See ``core/imports/track_number.py`` for the resolution
# chain — pure function, unit-tested in isolation, single # chain — pure function, unit-tested in isolation, single
# place to fix the rule. # place to fix the rule.
from core.imports.track_number import resolve_track_number, read_embedded_track_number from core.imports.track_number import resolve_track_number
track_info_for_resolve = context.get('track_info') if isinstance(context, dict) else None track_info_for_resolve = context.get('track_info') if isinstance(context, dict) else None
# "Track 01" bug: a single Deezer track is matched via an endpoint track_number = resolve_track_number(album_info, track_info_for_resolve, file_path)
# that omits track_position, so the context never carried the real
# number. The downloaded file itself does (deemix/source wrote it),
# so read it as a source between metadata and the filename guess.
embedded_track_number = read_embedded_track_number(file_path)
track_number = resolve_track_number(
album_info, track_info_for_resolve, file_path,
embedded_track_number=embedded_track_number,
)
logger.debug( logger.debug(
"Final track_number processing: source=%s album_info=%s resolved=%s", "Final track_number processing: source=%s album_info=%s resolved=%s",
album_info.get('source', 'unknown'), album_info.get('source', 'unknown'),
@ -752,16 +647,6 @@ def post_process_matched_download(context_key, context, file_path, runtime, meta
album_info['clean_track_name'] = clean_track_name album_info['clean_track_name'] = clean_track_name
logger.info(f"[FIX] Updated album_info track_number to {track_number} for consistent metadata") logger.info(f"[FIX] Updated album_info track_number to {track_number} for consistent metadata")
# Sync the disc number the SAME way (and via the SAME resolver) the embedded
# tag will use — otherwise the "Disc N" folder is built from album_info's
# original disc while the tag takes the per-track disc, so a disc-2/3 track
# lands in the Disc 1 folder and every disc's tracks collapse together (Sokhi).
from core.imports.track_number import resolve_disc_for_track
_resolved_disc = resolve_disc_for_track(get_import_original_search(context), album_info)
if album_info.get('disc_number') != _resolved_disc:
logger.info(f"[FIX] Updated album_info disc_number to {_resolved_disc} for consistent metadata")
album_info['disc_number'] = _resolved_disc
final_path, _ = build_final_path_for_track(context, artist_context, album_info, file_ext) final_path, _ = build_final_path_for_track(context, artist_context, album_info, file_ext)
logger.info(f"Resolved path: '{final_path}'") logger.info(f"Resolved path: '{final_path}'")
context['_final_processed_path'] = final_path context['_final_processed_path'] = final_path
@ -1084,22 +969,6 @@ def post_process_matched_download_with_verification(context_key, context, file_p
if original_batch_id: if original_batch_id:
context['batch_id'] = original_batch_id context['batch_id'] = original_batch_id
# Quality / audio-guard quarantine. Unlike acoustid/integrity (whose
# retry+fail is driven by THIS wrapper below), the inner pipeline fully
# owns the quality and audio-guard outcome: it quarantines the file and
# then either re-queues the next-best candidate or marks the task failed
# and notifies. The wrapper must NOT continue to the "assume success"
# fall-through — that marked the quarantined file Completed, so the same
# track showed in BOTH Completed and Quarantine (e.g. an MP3-VBR rejected
# by a FLAC-only profile). It must also NOT mark it failed here, which
# would clobber a successful next-candidate retry. Just return.
if context.get('_bitdepth_rejected') or context.get('_silence_rejected'):
logger.info(
f"Task {task_id} quarantined by the quality/audio guard — inner "
f"pipeline already handled retry/fail; wrapper not marking completed"
)
return
if context.get('_race_guard_failed'): if context.get('_race_guard_failed'):
logger.info(f"Race guard: source file gone for task {task_id} — marking as failed") logger.info(f"Race guard: source file gone for task {task_id} — marking as failed")
with tasks_lock: with tasks_lock:
@ -1113,33 +982,6 @@ def post_process_matched_download_with_verification(context_key, context, file_p
return return
if context.get('_acoustid_quarantined'): if context.get('_acoustid_quarantined'):
# Race-condition guard: if the user approved an alternative quarantine
# entry for this track while this download was already in flight, the
# pipeline will have created a new quarantine entry for the just-finished
# file. Delete that entry and exit — the user's choice already won.
_approved_alt = False
if task_id:
with tasks_lock:
_ct = download_tasks.get(task_id)
if isinstance(_ct, dict) and _ct.get('_quarantine_approved_alternative'):
_approved_alt = True
if _approved_alt:
_eid = context.get('_quarantine_entry_id')
if _eid:
try:
from core.imports.guards import _get_config_manager
_dl_dir = _get_config_manager().get('soulseek.download_path', './downloads')
_qdir = os.path.join(docker_resolve_path(_dl_dir), 'ss_quarantine')
delete_quarantine_entry(_qdir, _eid)
logger.info(
f"[Quarantine] Discarded late entry {_eid} — task {task_id} "
f"was cancelled by prior quarantine approval"
)
except Exception as _dqe:
logger.debug(f"[Quarantine] Could not clean up late entry {_eid}: {_dqe}")
_notify_download_completed(batch_id, task_id, success=False)
return
failure_msg = context.get('_acoustid_failure_msg', 'AcoustID verification failed') failure_msg = context.get('_acoustid_failure_msg', 'AcoustID verification failed')
# Before failing outright, try the next-best candidate. The wrong # Before failing outright, try the next-best candidate. The wrong
# file was just quarantined; re-running the worker (with the bad # file was just quarantined; re-running the worker (with the bad
@ -1256,10 +1098,6 @@ def post_process_matched_download_with_verification(context_key, context, file_p
_mark_task_completed(task_id, context.get('track_info')) _mark_task_completed(task_id, context.get('track_info'))
if context.get('_verification_status'): if context.get('_verification_status'):
download_tasks[task_id]['verification_status'] = context['_verification_status'] download_tasks[task_id]['verification_status'] = context['_verification_status']
if context.get('_history_id'):
download_tasks[task_id]['history_id'] = context['_history_id']
if context.get('_audio_quality'):
download_tasks[task_id]['quality'] = context['_audio_quality']
with matched_context_lock: with matched_context_lock:
if context_key in matched_downloads_context: if context_key in matched_downloads_context:
del matched_downloads_context[context_key] del matched_downloads_context[context_key]
@ -1274,10 +1112,6 @@ def post_process_matched_download_with_verification(context_key, context, file_p
download_tasks[task_id]['metadata_enhanced'] = True download_tasks[task_id]['metadata_enhanced'] = True
if context.get('_verification_status'): if context.get('_verification_status'):
download_tasks[task_id]['verification_status'] = context['_verification_status'] download_tasks[task_id]['verification_status'] = context['_verification_status']
if context.get('_history_id'):
download_tasks[task_id]['history_id'] = context['_history_id']
if context.get('_audio_quality'):
download_tasks[task_id]['quality'] = context['_audio_quality']
redownload_ctx = download_tasks[task_id].get('_redownload_context') redownload_ctx = download_tasks[task_id].get('_redownload_context')
with matched_context_lock: with matched_context_lock:

View file

@ -153,76 +153,6 @@ def get_quarantined_source_keys(quarantine_dir: str) -> set:
return keys return keys
def quarantine_group_key(
expected_artist: Any, expected_track: Any, context: Any = None
) -> Optional[str]:
"""Grouping key for "the same intended download target".
#876: when several sources are downloaded for one wishlist/queue
track they each fail verification and land in quarantine as separate
entries. They are *alternatives for the same song*, so they should
group together and once the user accepts one, the rest are
redundant failed attempts at a song they now own.
The key identifies the *intended* target what SoulSync was trying to
fetch NOT the downloaded file's own tags. That matters: the file's
metadata is frequently *wrong* (that's why it failed acoustid /
integrity), whereas the target is fixed and identical across every
alternative for one song.
Uses ISRC when available (truly universal across sources and batches).
Falls back to normalized artist|track name, which is stable across
different batches and sources.
Source-specific IDs (Spotify track id, Qobuz id, uri) are intentionally
NOT used: the same song imported from different playlists or sources gets
different source IDs, so id-based keys break cross-batch sibling matching.
Returns ``None`` when nothing identifies the target (no usable id and
both name fields empty). Callers treat a ``None`` key as "its own
singleton group" — ungroupable entries must never collapse together.
"""
ti = {}
if isinstance(context, dict):
maybe_ti = context.get("track_info")
if isinstance(maybe_ti, dict):
ti = maybe_ti
isrc = str(ti.get("isrc") or "").strip().lower()
if isrc:
return f"isrc:{isrc}"
artist = " ".join(str(expected_artist or "").split()).lower()
track = " ".join(str(expected_track or "").split()).lower()
if not artist and not track:
return None
return f"nm:{artist}|{track}"
def find_quarantine_siblings(quarantine_dir: str, entry_id: str) -> List[str]:
"""Other entry ids that share ``entry_id``'s intended-target group key.
Returns the ids of every *other* quarantine entry whose
`expected_artist`/`expected_track` normalize to the same key as
``entry_id`` (see :func:`quarantine_group_key`). Excludes ``entry_id``
itself. Returns ``[]`` when the entry is missing, has an ungroupable
(``None``) key, or has no siblings. Never raises.
"""
if not entry_id:
return []
entries = list_quarantine_entries(quarantine_dir)
target_key = None
for e in entries:
if e.get("id") == entry_id:
target_key = e.get("group_key")
break
if target_key is None:
return []
return [
e["id"]
for e in entries
if e.get("id") != entry_id and e.get("group_key") == target_key
]
def list_quarantine_entries(quarantine_dir: str) -> List[Dict[str, Any]]: def list_quarantine_entries(quarantine_dir: str) -> List[Dict[str, Any]]:
"""Enumerate quarantined files paired with their sidecars. """Enumerate quarantined files paired with their sidecars.
@ -283,11 +213,6 @@ def list_quarantine_entries(quarantine_dir: str) -> List[Dict[str, Any]]:
"reason": sidecar.get("quarantine_reason", "Unknown reason"), "reason": sidecar.get("quarantine_reason", "Unknown reason"),
"expected_track": sidecar.get("expected_track", ""), "expected_track": sidecar.get("expected_track", ""),
"expected_artist": sidecar.get("expected_artist", ""), "expected_artist": sidecar.get("expected_artist", ""),
"group_key": quarantine_group_key(
sidecar.get("expected_artist", ""),
sidecar.get("expected_track", ""),
ctx,
),
"timestamp": sidecar.get("timestamp", ""), "timestamp": sidecar.get("timestamp", ""),
"size_bytes": size_bytes, "size_bytes": size_bytes,
"has_full_context": isinstance(sidecar.get("context"), dict), "has_full_context": isinstance(sidecar.get("context"), dict),
@ -295,10 +220,6 @@ def list_quarantine_entries(quarantine_dir: str) -> List[Dict[str, Any]]:
"source_username": source_username, "source_username": source_username,
"source_filename": source_filename, "source_filename": source_filename,
"thumb_url": _extract_context_thumb(ctx), "thumb_url": _extract_context_thumb(ctx),
# Real probed audio quality (recorded on the context before the
# quality/AcoustID gates) so the review UI shows what the file
# actually is when deciding to approve/delete.
"quality": ctx.get("_audio_quality", "") if isinstance(ctx, dict) else "",
} }
) )

View file

@ -1,92 +0,0 @@
"""#889 Phase 4/5: apply a re-identify — stage the library file + write the hint.
When the user confirms a release in the Re-identify modal, we:
1. COPY (never move) the track's library file into the auto-import staging folder,
so the original is untouched until the re-import succeeds,
2. fingerprint the staged copy (rename-proof binding), and
3. write a single-use hint carrying the chosen release's IDs (+ ``replace_track_id``
when 'replace original' is ticked).
The auto-import worker then picks the staged file up, finds the hint, and re-imports
it against the user-chosen release (Phase 2). The pieces here are split so the
naming + hint construction are pure/unit-tested and the actual copy is injectable.
"""
from __future__ import annotations
import os
import shutil
from typing import Any, Callable, Dict, Optional
from core.imports.paths import sanitize_filename
from core.imports.rematch_hints import RematchHint, quick_file_signature
def staged_destination(staging_dir: str, real_path: str, library_track_id: Any) -> str:
"""Where the staged copy lands: a single loose file in the staging ROOT (so the
worker treats it as a single-track candidate), named to keep the extension and
be unique + traceable to the track it re-identifies. The filename is cosmetic
matching is driven by the hint, not the name."""
base = os.path.basename(real_path)
stem, ext = os.path.splitext(base)
safe_stem = sanitize_filename(stem).strip() or "track"
name = f"{safe_stem} [reid-{library_track_id}]{ext}"
return os.path.join(staging_dir, name)
def stage_file_for_reidentify(
real_path: str,
staging_dir: str,
library_track_id: Any,
*,
copy_fn: Callable[[str, str], object] = shutil.copy2,
signature_fn: Callable[[str], Optional[str]] = quick_file_signature,
) -> Dict[str, Any]:
"""Copy the library file into staging and fingerprint the copy. Returns
``{staged_path, content_hash}``. Raises ``FileNotFoundError`` if the source is
gone (caller surfaces a clear error rather than writing a dangling hint)."""
if not real_path or not os.path.isfile(real_path):
raise FileNotFoundError(real_path or "(empty path)")
os.makedirs(staging_dir, exist_ok=True)
dest = staged_destination(staging_dir, real_path, library_track_id)
copy_fn(real_path, dest)
return {"staged_path": dest, "content_hash": signature_fn(dest)}
def build_reidentify_hint(
library_track_id: Any,
hint_fields: Dict[str, Any],
staged_path: str,
content_hash: Optional[str],
*,
replace: bool,
) -> RematchHint:
"""Pure: assemble the RematchHint from the resolved release fields + staging
info. ``replace_track_id`` is the library row to delete on success, but only
when 'replace original' was ticked. ``exempt_dedup`` is always True a
re-identify is explicit and must bypass dedup-skip."""
return RematchHint(
staged_path=staged_path,
content_hash=content_hash,
source=hint_fields.get("source") or "",
isrc=hint_fields.get("isrc"),
track_id=hint_fields.get("track_id"),
album_id=hint_fields.get("album_id"),
artist_id=hint_fields.get("artist_id"),
track_title=hint_fields.get("track_title"),
album_name=hint_fields.get("album_name"),
artist_name=hint_fields.get("artist_name"),
album_type=hint_fields.get("album_type"),
track_number=hint_fields.get("track_number"),
disc_number=hint_fields.get("disc_number"),
replace_track_id=(int(library_track_id) if replace and str(library_track_id).isdigit() else
(library_track_id if replace else None)),
exempt_dedup=True,
)
__all__ = [
"staged_destination",
"stage_file_for_reidentify",
"build_reidentify_hint",
]

View file

@ -1,334 +0,0 @@
"""Re-identify hints (#889) — a single-use, user-designated answer to "which
release does this already-imported track belong to".
Flow: the user clicks *Re-identify* on a library track, searches a source, and
picks the exact release (single / EP / album) it should live under. We write a
**hint** here and stage the file for auto-import. The import flow then reads the
hint at the very TOP of matching before any fuzzy tier builds the match from
these exact IDs, and consumes the row. So the original ambiguity that mis-filed
the track (which release?) is gone: the user already answered it.
Two safety properties live in the hint, not the import code:
- ``replace_track_id`` the library row to delete AFTER the re-import lands (so a
re-identify *replaces* rather than *duplicates*). Cleanup is deferred to success
so a failed import can never lose the file.
- ``exempt_dedup`` always set: a re-identify is an explicit user action and must
not be silently dropped by the quality dedup-skip (which would otherwise see the
incoming file as a duplicate of the very row we're replacing).
This module is pure DB mechanics over an injected ``cursor`` (sqlite3-style,
``?`` params) no connection management, no app state so the create / find /
consume seam is unit-tested against an in-memory DB with no live metadata client.
The binding is keyed on the staged path, with ``content_hash`` as a rename-proof
fallback in case the staging watcher normalizes the filename on ingest.
"""
from __future__ import annotations
import os
from dataclasses import dataclass
from typing import Any, Callable, Optional
# Columns in INSERT/SELECT order — single source of truth so the dataclass, the
# write, and the read can't drift apart.
_FIELDS = (
"staged_path",
"content_hash",
"source",
"isrc",
"track_id",
"album_id",
"artist_id",
"track_title",
"album_name",
"artist_name",
"album_type",
"track_number",
"disc_number",
"replace_track_id",
"exempt_dedup",
)
@dataclass
class RematchHint:
"""One user-designated re-identify answer. ``id``/``status`` are set by the DB."""
staged_path: str
source: str
content_hash: Optional[str] = None
isrc: Optional[str] = None
track_id: Optional[str] = None
album_id: Optional[str] = None
artist_id: Optional[str] = None
track_title: Optional[str] = None
album_name: Optional[str] = None
artist_name: Optional[str] = None
album_type: Optional[str] = None
track_number: Optional[int] = None
disc_number: Optional[int] = None
replace_track_id: Optional[int] = None
exempt_dedup: bool = True
id: Optional[int] = None
status: str = "pending"
def _values(self) -> tuple:
return (
self.staged_path,
self.content_hash,
self.source,
self.isrc,
self.track_id,
self.album_id,
self.artist_id,
self.track_title,
self.album_name,
self.artist_name,
self.album_type,
self.track_number,
self.disc_number,
self.replace_track_id,
1 if self.exempt_dedup else 0,
)
def _row_to_hint(row: Any) -> RematchHint:
"""Map a sqlite3.Row (or any mapping/sequence-by-name) to a RematchHint."""
def g(key, default=None):
try:
return row[key]
except (KeyError, IndexError, TypeError):
return default
return RematchHint(
id=g("id"),
staged_path=g("staged_path") or "",
content_hash=g("content_hash"),
source=g("source") or "",
isrc=g("isrc"),
track_id=g("track_id"),
album_id=g("album_id"),
artist_id=g("artist_id"),
track_title=g("track_title"),
album_name=g("album_name"),
artist_name=g("artist_name"),
album_type=g("album_type"),
track_number=g("track_number"),
disc_number=g("disc_number"),
replace_track_id=g("replace_track_id"),
exempt_dedup=bool(g("exempt_dedup", 1)),
status=g("status") or "pending",
)
def create_hint(cursor: Any, hint: RematchHint) -> int:
"""Insert a pending hint; return its new id. Caller owns commit."""
placeholders = ", ".join("?" for _ in _FIELDS)
cursor.execute(
f"INSERT INTO rematch_hints ({', '.join(_FIELDS)}) VALUES ({placeholders})",
hint._values(),
)
new_id = cursor.lastrowid
hint.id = new_id
return new_id
def find_hint_for_file(
cursor: Any,
staged_path: str,
content_hash: Optional[str] = None,
) -> Optional[RematchHint]:
"""Return the newest PENDING hint for a staged file, or ``None``.
Matched by exact ``staged_path`` first; if that misses and a ``content_hash``
is given, fall back to it (covers a staging watcher that renamed the file on
ingest). Only ``status='pending'`` rows are returned, so a consumed hint is
never reused."""
if staged_path:
cursor.execute(
"SELECT * FROM rematch_hints WHERE staged_path = ? AND status = 'pending' "
"ORDER BY id DESC LIMIT 1",
(staged_path,),
)
row = cursor.fetchone()
if row is not None:
return _row_to_hint(row)
# Try by basename too — the watcher may move the file into a different dir.
base = os.path.basename(staged_path)
if base and base != staged_path:
cursor.execute(
"SELECT * FROM rematch_hints WHERE staged_path LIKE ? AND status = 'pending' "
"ORDER BY id DESC LIMIT 1",
("%/" + base,),
)
row = cursor.fetchone()
if row is not None:
return _row_to_hint(row)
if content_hash:
cursor.execute(
"SELECT * FROM rematch_hints WHERE content_hash = ? AND status = 'pending' "
"ORDER BY id DESC LIMIT 1",
(content_hash,),
)
row = cursor.fetchone()
if row is not None:
return _row_to_hint(row)
return None
def consume_hint(cursor: Any, hint_id: int) -> None:
"""Mark a hint consumed (single-use). Caller owns commit."""
cursor.execute(
"UPDATE rematch_hints SET status = 'consumed', consumed_at = CURRENT_TIMESTAMP "
"WHERE id = ?",
(hint_id,),
)
def list_pending_hints(cursor: Any) -> list:
"""All pending hints (newest first) — for a 'pending re-identify' view and
orphan recovery when a staged file never imports."""
cursor.execute("SELECT * FROM rematch_hints WHERE status = 'pending' ORDER BY id DESC")
return [_row_to_hint(r) for r in cursor.fetchall()]
def build_identification_from_hint(hint: RematchHint) -> dict:
"""Turn a hint into the ``identification`` dict the auto-import matcher expects,
so a re-identify SKIPS the guessing tiers entirely and matches straight against
the user-chosen release. Mirrors the shape `_identify_folder` returns (album_id
/ source / track_number drive the album fetch + filetrack match)."""
return {
"album_id": hint.album_id or None,
"album_name": hint.album_name or hint.track_title or "",
"artist_name": hint.artist_name or "",
"artist_id": hint.artist_id or "",
"track_name": hint.track_title or "",
"track_id": hint.track_id or "",
"image_url": "",
"release_date": "",
"track_number": hint.track_number or 1,
"total_tracks": 1,
"source": hint.source,
"method": "rematch_hint",
"identification_confidence": 1.0,
# is_single reflects the CHOSEN release, but force_album_match makes the
# matcher FETCH that release (even for a lone staged file) instead of taking
# the singles fast-path — so the re-imported track gets the real album
# metadata: year, the correct in-album track number, and the album art.
"is_single": (str(hint.album_type or "").lower() == "single"),
"force_album_match": True,
"album_type": hint.album_type,
}
def _canonical(path: Optional[str]) -> str:
"""Canonical form of a path for same-file comparison (symlinks + case + sep)."""
if not path:
return ""
try:
return os.path.normcase(os.path.realpath(path))
except OSError:
return os.path.normcase(os.path.normpath(path))
def delete_replaced_track(
cursor: Any,
replace_track_id: Any,
*,
unlink=os.remove,
resolve_fn: Optional[Callable[[str], Optional[str]]] = None,
new_paths: Optional[list] = None,
) -> Optional[str]:
"""Remove the OLD library row a re-identify replaces, and its file.
Called only AFTER the re-import has landed the track at its new home, so the
original is never lost on failure. Safe by construction:
* **Same-home guard (CRITICAL):** if the re-import landed at the SAME file as the
old one (``new_paths`` the paths the import actually wrote), this is a no-op:
we DON'T delete the row or the file, because that file IS the re-imported track.
This is what stops "re-identify to the release it's already in" from deleting
the file (the import reuses the same row, so deleting it would orphan the file).
* the file is unlinked only if it still exists and **no other track row references
it** (guards against yanking a file a different row legitimately points to).
Returns the path it removed, or ``None`` if there was nothing to do. ``unlink`` is
injectable for tests. ``resolve_fn`` maps the STORED DB path to the file's actual
on-disk location (the stored path may be a Docker/media-server view this process
can't read literally — without it we'd delete the row but orphan the file)."""
if not replace_track_id:
return None
cursor.execute("SELECT file_path FROM tracks WHERE id = ?", (replace_track_id,))
row = cursor.fetchone()
if row is None:
return None
old_path = (row["file_path"] if not isinstance(row, (tuple, list)) else row[0]) or ""
if not old_path:
cursor.execute("DELETE FROM tracks WHERE id = ?", (replace_track_id,))
return None
# Resolve the old stored path to its real on-disk location up front.
real_path = old_path
if resolve_fn is not None:
try:
real_path = resolve_fn(old_path) or old_path
except Exception:
real_path = old_path
# Same-home guard: if the re-import wrote to this very file, do NOTHING — the row
# is the re-imported track's row and the file is its file. Deleting either would
# be data loss (the "picked the same release" bug).
if new_paths:
landed = {_canonical(p) for p in new_paths if p}
if _canonical(real_path) in landed or _canonical(old_path) in landed:
return None
cursor.execute("DELETE FROM tracks WHERE id = ?", (replace_track_id,))
# Only unlink if no surviving row still points at this file (rows store the
# stored path, so compare against the stored path, not the resolved one).
cursor.execute("SELECT 1 FROM tracks WHERE file_path = ? LIMIT 1", (old_path,))
if cursor.fetchone() is not None:
return None
try:
if os.path.exists(real_path): # real_path resolved above
unlink(real_path)
return real_path
except OSError:
pass
return None
def quick_file_signature(path: str, *, chunk: int = 65536) -> Optional[str]:
"""A cheap, rename-proof content fingerprint: size + first/last chunk, hashed.
Audio files are large, so a full hash is wasteful when we only need to re-bind
a hint to *this* file after a possible rename. Size + head + tail is plenty to
distinguish staged files in practice. Returns ``None`` if the file can't be
read (caller falls back to path-only binding)."""
import hashlib
try:
size = os.path.getsize(path)
h = hashlib.sha256()
h.update(str(size).encode())
with open(path, "rb") as f:
h.update(f.read(chunk))
if size > chunk:
f.seek(max(0, size - chunk))
h.update(f.read(chunk))
return h.hexdigest()
except OSError:
return None
__all__ = [
"RematchHint",
"create_hint",
"find_hint_for_file",
"consume_hint",
"list_pending_hints",
"build_identification_from_hint",
"delete_replaced_track",
"quick_file_signature",
]

View file

@ -1,247 +0,0 @@
"""#889 Phase 3: search a metadata source for the releases a track appears on.
The Re-identify modal lets the user search ANY configured source (tabs, defaulting
to the active one) and shows the SAME song across its different collections
single / EP / album so they can pick which release the track should be filed
under. Two steps, deliberately split:
* ``search_release_candidates(source, query)`` lightweight DISPLAY rows from the
normal typed ``search_tracks`` (title, artist, release name, type badge, year,
track count, art, ISRC, track_id). No album_id needed to draw the list.
* ``resolve_hint_fields(source, track_id)`` runs ONCE, on the row the user
picks: ``get_track_details`` yields the album_id / isrc / track#/disc the hint
needs. We don't pay that lookup for every search result, only the chosen one.
Pure normalization + injected client factory, so the search/normalize/resolve seam
is unit-tested with a fake client and no network.
"""
from __future__ import annotations
from typing import Any, Callable, Dict, List, Optional
def _get(obj: Any, key: str, default=None):
"""Read ``key`` from either an object (attr) or a mapping (item)."""
if obj is None:
return default
if isinstance(obj, dict):
return obj.get(key, default)
return getattr(obj, key, default)
def _year(release_date: Any) -> Optional[str]:
s = str(release_date or "").strip()
return s[:4] if len(s) >= 4 and s[:4].isdigit() else None
def infer_release_type(album_type: Any, total_tracks: Any) -> str:
"""Normalize a source's release type to one of album / ep / single / compilation.
Sources disagree: Spotify has no 'EP' EPs come back as ``album_type='single'``
with several tracks; MusicBrainz/Deezer label EPs properly. So when a 'single'
carries more than a handful of tracks, call it an EP for the badge. The actual
filing is unaffected that's driven by the real album_id, not this label."""
t = str(album_type or "").strip().lower()
try:
n = int(total_tracks) if total_tracks is not None else 0
except (TypeError, ValueError):
n = 0
if t in ("compilation", "comp"):
return "compilation"
if t == "ep":
return "ep"
if t == "album":
return "album" # an explicit album stays an album; only 'single' gets promoted to EP
if t == "single":
# 13 tracks → single; 4+ → almost always an EP in practice.
return "ep" if n >= 4 else "single"
# Unknown type: infer purely from track count.
if n >= 7:
return "album"
if n >= 4:
return "ep"
if n >= 1:
return "single"
return t or "album"
def normalize_search_result(result: Any, source: str) -> Optional[Dict[str, Any]]:
"""One typed search Track (or raw dict) → a display row, or ``None`` if it has
no usable id/title. ``album`` is just a name at search time; album_id is
resolved later for the picked row only."""
track_id = _get(result, "id") or _get(result, "track_id")
title = _get(result, "name") or _get(result, "title")
if not track_id or not title:
return None
artists = _get(result, "artists")
if isinstance(artists, list):
artist_name = ", ".join(str(_get(a, "name", a) if not isinstance(a, str) else a) for a in artists)
else:
artist_name = str(artists or _get(result, "artist") or "")
album = _get(result, "album")
album_name = album if isinstance(album, str) else (_get(album, "name") or "")
raw_type = _get(result, "album_type")
total = _get(result, "total_tracks")
ext = _get(result, "external_ids") or {}
isrc = _get(result, "isrc") or (ext.get("isrc") if isinstance(ext, dict) else None)
return {
"source": source,
"track_id": str(track_id),
"track_title": str(title),
"artist_name": artist_name,
"album_name": str(album_name or ""),
"album_type": infer_release_type(raw_type, total),
"raw_album_type": str(raw_type or ""),
"total_tracks": int(total) if isinstance(total, int) else None,
"year": _year(_get(result, "release_date")),
"image_url": _get(result, "image_url") or "",
"isrc": isrc or None,
}
def search_release_candidates(
source: str,
query: str,
*,
limit: int = 25,
client_factory: Optional[Callable[[str], Any]] = None,
) -> List[Dict[str, Any]]:
"""Search ``source`` for tracks matching ``query`` → normalized display rows.
Returns ``[]`` (never raises) when the source has no client or errors the UI
just shows an empty tab. Rows keep duplicate releases; the UI groups them."""
query = (query or "").strip()
if not query:
return []
factory = client_factory or _default_client_factory
try:
client = factory(source)
except Exception:
client = None
if client is None or not hasattr(client, "search_tracks"):
return []
try:
results = client.search_tracks(query, limit=limit)
except TypeError:
results = client.search_tracks(query) # clients with no limit kwarg
except Exception:
return []
rows: List[Dict[str, Any]] = []
for r in results or []:
row = normalize_search_result(r, source)
if row is not None:
rows.append(row)
return rows
def resolve_hint_fields(
source: str,
track_id: str,
*,
client_factory: Optional[Callable[[str], Any]] = None,
) -> Optional[Dict[str, Any]]:
"""Resolve the picked track to the fields a hint needs (album_id critically,
plus isrc / track# / disc# / album name+type). One lookup for one chosen row.
Returns ``None`` if it can't be resolved (caller surfaces an error)."""
factory = client_factory or _default_client_factory
try:
client = factory(source)
except Exception:
client = None
if client is None or not hasattr(client, "get_track_details"):
return None
try:
details = client.get_track_details(track_id)
except Exception:
return None
if not details:
return None
album = _get(details, "album") or {}
album_id = _get(album, "id") if not isinstance(album, str) else None
album_name = _get(album, "name") if not isinstance(album, str) else album
album_type = _get(album, "album_type") or _get(details, "album_type")
total = _get(album, "total_tracks") or _get(details, "total_tracks")
artists = _get(details, "artists") or []
artist_id = None
artist_name = ""
if isinstance(artists, list) and artists:
artist_id = _get(artists[0], "id")
artist_name = ", ".join(str(_get(a, "name", a) if not isinstance(a, str) else a) for a in artists)
ext = _get(details, "external_ids") or {}
isrc = _get(details, "isrc") or (ext.get("isrc") if isinstance(ext, dict) else None)
if not album_id:
return None # without an album_id the import can't fetch the tracklist
return {
"source": source,
"track_id": str(track_id),
"album_id": str(album_id),
"artist_id": str(artist_id) if artist_id else None,
"track_title": _get(details, "name") or _get(details, "title") or "",
"album_name": str(album_name or ""),
"artist_name": artist_name,
"album_type": infer_release_type(album_type, total),
"track_number": _get(details, "track_number"),
"disc_number": _get(details, "disc_number") or 1,
"isrc": isrc or None,
}
def _default_client_factory(source: str):
from core.metadata.registry import get_client_for_source
return get_client_for_source(source)
def available_sources() -> List[Dict[str, Any]]:
"""The source tabs for the modal: every metadata source with a live client,
the primary one flagged ``active`` so the UI selects it by default."""
from core.metadata.registry import (
METADATA_SOURCE_PRIORITY,
get_client_for_source,
get_primary_source,
)
try:
primary = get_primary_source()
except Exception:
primary = None
out: List[Dict[str, Any]] = []
seen = set()
for src in METADATA_SOURCE_PRIORITY:
if src in seen:
continue
seen.add(src)
try:
client = get_client_for_source(src)
except Exception:
client = None
if client is None or not hasattr(client, "search_tracks"):
continue
out.append({
"source": src,
"label": src.replace("_", " ").title(),
"active": src == primary,
})
# Guarantee the primary is selectable + first even if priority ordering missed it.
if primary and not any(s["active"] for s in out):
out.insert(0, {"source": primary, "label": primary.replace("_", " ").title(), "active": True})
return out
__all__ = [
"infer_release_type",
"normalize_search_result",
"search_release_candidates",
"resolve_hint_fields",
"available_sources",
]

View file

@ -3,12 +3,10 @@
from __future__ import annotations from __future__ import annotations
import os import os
import threading
import time
import uuid import uuid
from concurrent.futures import as_completed from concurrent.futures import as_completed
from dataclasses import dataclass from dataclasses import dataclass
from typing import Any, Callable, Dict, Optional from typing import Any, Callable, Dict
from core.imports.album import build_album_import_context, build_album_import_match_payload, resolve_album_artist_context from core.imports.album import build_album_import_context, build_album_import_match_payload, resolve_album_artist_context
from core.imports.context import get_import_context_artist, get_import_track_info, normalize_import_context from core.imports.context import get_import_context_artist, get_import_track_info, normalize_import_context
@ -18,7 +16,6 @@ from core.imports.staging import (
AUDIO_EXTENSIONS, AUDIO_EXTENSIONS,
get_import_suggestions_cache, get_import_suggestions_cache,
get_primary_source as _get_primary_source, get_primary_source as _get_primary_source,
get_primary_source_label as _get_primary_source_label,
get_staging_path as _get_staging_path, get_staging_path as _get_staging_path,
read_staging_file_metadata as _read_staging_file_metadata, read_staging_file_metadata as _read_staging_file_metadata,
refresh_import_suggestions_cache as _refresh_import_suggestions_cache, refresh_import_suggestions_cache as _refresh_import_suggestions_cache,
@ -51,7 +48,6 @@ class ImportRouteRuntime:
read_staging_file_metadata: Callable[[str, str], Dict[str, Any]] = _read_staging_file_metadata read_staging_file_metadata: Callable[[str, str], Dict[str, Any]] = _read_staging_file_metadata
read_tags: Callable[[str], Any] = _default_read_tags read_tags: Callable[[str], Any] = _default_read_tags
get_primary_source: Callable[[], str] = _get_primary_source get_primary_source: Callable[[], str] = _get_primary_source
get_primary_source_label: Callable[[], str] = _get_primary_source_label
search_import_albums: Callable[..., list] = _search_import_albums search_import_albums: Callable[..., list] = _search_import_albums
search_import_tracks: Callable[..., list] = _search_import_tracks search_import_tracks: Callable[..., list] = _search_import_tracks
build_album_import_match_payload: Callable[..., Dict[str, Any]] = build_album_import_match_payload build_album_import_match_payload: Callable[..., Dict[str, Any]] = build_album_import_match_payload
@ -73,219 +69,36 @@ class ImportRouteRuntime:
logger: Any = module_logger logger: Any = module_logger
# ── Shared staging scan ──────────────────────────────────────────────────────
# Opening the Import page fires staging files/groups/hints together; each used to
# os.walk the whole staging folder AND mutagen-read every file independently — 3×
# the directory walk + 3× the tag I/O on every page open (the import-page scan
# storm + memory spike, issue #935). They all need the same per-file tag data, so
# scan ONCE and let all three derive their views in-memory. A short TTL + a lock
# means the three near-simultaneous page-open requests (and any concurrent caller)
# share a single scan instead of each kicking off a full re-read.
_STAGING_SCAN_LOCK = threading.Lock()
_STAGING_SCAN_TTL = 6.0 # seconds — covers the page-open burst; re-scans after
_staging_scan_cache: Dict[str, Any] = {"path": None, "ts": 0.0, "records": None}
# Bumped by invalidate_staging_scan_cache() so a background scan that finishes after an
# import doesn't re-commit stale (pre-import) records (see the generation guard above).
_staging_scan_generation: Dict[str, int] = {"value": 0}
# Background-scan plumbing: a large staging folder (whole-library migration, #947) makes
# the synchronous scan exceed gunicorn's 120s request timeout. The runner moves the SAME
# scan off the request thread; the endpoints report progress instead of blocking.
_staging_scan_status: Dict[str, Any] = {
"status": "idle", "scanned": 0, "total": 0, "path": None, "error": None,
}
_staging_scan_status_lock = threading.Lock()
def _staging_cache_hit(staging_path: str) -> Optional[list]:
"""The cached records for ``staging_path`` if still fresh, else None (no scan triggered)."""
c = _staging_scan_cache
if (c["records"] is not None and c["path"] == staging_path
and (time.time() - c["ts"]) < _STAGING_SCAN_TTL):
return c["records"]
return None
def ensure_background_staging_scan(runtime: ImportRouteRuntime, staging_path: str) -> None:
"""Start a background scan for ``staging_path`` unless the cache is warm or a scan for
this path is already running. Idempotent safe to call on every request."""
if _staging_cache_hit(staging_path) is not None:
return
with _staging_scan_status_lock:
if (_staging_scan_status["status"] == "scanning"
and _staging_scan_status["path"] == staging_path):
return
_staging_scan_status.update({"status": "scanning", "scanned": 0, "total": 0,
"path": staging_path, "error": None})
def _run() -> None:
try:
_scan_staging_records(runtime, staging_path, progress=_staging_scan_status)
with _staging_scan_status_lock:
if _staging_scan_status["path"] == staging_path:
_staging_scan_status["status"] = "done"
except Exception as exc: # noqa: BLE001 — surface any scan error to the poller
with _staging_scan_status_lock:
_staging_scan_status.update({"status": "error", "error": str(exc)})
threading.Thread(target=_run, name="staging-scan", daemon=True).start()
def get_staging_records_or_status(runtime: ImportRouteRuntime, staging_path: str,
*, grace_seconds: float = 3.0) -> tuple[str, Any]:
"""Non-blocking staging access for the page endpoints. Returns ``("ready", records)``
when the cache is warm or the scan completes within ``grace_seconds`` (so small/normal
folders still answer in a single request), otherwise ``("scanning", status_dict)`` after
making sure a background scan is running."""
records = _staging_cache_hit(staging_path)
if records is not None:
return ("ready", records)
ensure_background_staging_scan(runtime, staging_path)
deadline = time.time() + max(0.0, grace_seconds)
while True:
records = _staging_cache_hit(staging_path)
if records is not None:
return ("ready", records)
with _staging_scan_status_lock:
status = dict(_staging_scan_status)
if status.get("status") == "error":
return ("error", status)
if time.time() >= deadline:
return ("scanning", status)
time.sleep(0.05)
def _records_or_scanning_payload(runtime: ImportRouteRuntime, staging_path: str):
"""Shared helper for the page endpoints: returns ``(records, None)`` when the scan is
ready, or ``(None, payload)`` when a background scan is still running the caller
returns that payload so the page polls + shows progress instead of blocking/timing out.
A scan error is re-raised so the endpoint's own try/except logs + returns it exactly as
when the scan ran inline (preserves the existing error contract)."""
state, val = get_staging_records_or_status(runtime, staging_path)
if state == "error":
raise RuntimeError(val.get("error") or "staging scan failed")
if state == "scanning":
return None, {"success": True, "scanning": True,
"progress": {"scanned": val.get("scanned", 0),
"total": val.get("total", 0)}}
return val, None
def staging_scan_status(runtime: ImportRouteRuntime) -> tuple[Dict[str, Any], int]:
"""Lightweight, instant scan-progress poll for the page (no grace-wait, no file I/O) —
``ready`` true once the cache is warm and the files/groups/hints calls will answer fast."""
try:
staging_path = runtime.get_staging_path()
except Exception as exc:
return {"success": False, "error": str(exc)}, 500
with _staging_scan_status_lock:
st = dict(_staging_scan_status)
return {
"success": True,
"ready": _staging_cache_hit(staging_path) is not None,
"status": st.get("status", "idle"),
"scanned": st.get("scanned", 0),
"total": st.get("total", 0),
"error": st.get("error"),
}, 200
def _scan_staging_records(runtime: ImportRouteRuntime, staging_path: str,
*, progress: Optional[Dict[str, Any]] = None) -> list[Dict[str, Any]]:
"""Walk staging + read each audio file's tags ONCE, returning per-file records
that staging files/groups/hints all derive from. Briefly cached + locked so the
page-open trio shares a single scan rather than each re-walking and re-reading.
``progress`` (optional, default None = unchanged behaviour) is a dict the scan
updates live with ``total`` (audio-file count, from a fast first pass) and ``scanned``
(tag-reads done so far) so a background runner can report progress. A generation guard
keeps a scan that finishes AFTER an import (which bumped ``_staging_scan_generation``)
from committing stale records to the cache."""
now = time.time()
cached = _staging_scan_cache
if (cached["records"] is not None and cached["path"] == staging_path
and (now - cached["ts"]) < _STAGING_SCAN_TTL):
return cached["records"]
with _STAGING_SCAN_LOCK:
# Double-check: another request may have filled the cache while we waited.
now = time.time()
if (cached["records"] is not None and cached["path"] == staging_path
and (now - cached["ts"]) < _STAGING_SCAN_TTL):
return cached["records"]
start_generation = _staging_scan_generation["value"]
# Pass 1 (fast): collect the audio-file list — no tag I/O — so we know the total.
audio_files: list[tuple[str, str, Optional[str]]] = []
if os.path.isdir(staging_path):
for root, _dirs, filenames in os.walk(staging_path):
rel_dir = os.path.relpath(root, staging_path)
top_folder = rel_dir.split(os.sep)[0] if rel_dir != "." else None
for fname in filenames:
if os.path.splitext(fname)[1].lower() in AUDIO_EXTENSIONS:
audio_files.append((root, fname, top_folder))
if progress is not None:
progress["total"] = len(audio_files)
progress["scanned"] = 0
# Pass 2 (slow): read each file's tags, updating progress as we go.
records: list[Dict[str, Any]] = []
for root, fname, top_folder in audio_files:
full_path = os.path.join(root, fname)
rel_path = os.path.relpath(full_path, staging_path)
meta = runtime.read_staging_file_metadata(full_path, rel_path)
records.append({
"filename": fname, "rel_path": rel_path, "full_path": full_path,
"extension": os.path.splitext(fname)[1].lower(),
"title": meta["title"], "album": meta["album"],
"artist": meta["artist"], "albumartist": meta["albumartist"],
"track_number": meta["track_number"], "disc_number": meta["disc_number"],
"top_folder": top_folder,
})
if progress is not None:
progress["scanned"] += 1
# Generation guard: if an import invalidated the cache mid-scan, these records are
# stale — return them to this caller but do NOT commit them as the shared cache.
if _staging_scan_generation["value"] == start_generation:
_staging_scan_cache.update({"path": staging_path, "ts": time.time(), "records": records})
return records
def invalidate_staging_scan_cache() -> None:
"""Drop the cached staging scan (call after an import moves/removes files so the
next files/groups/hints request reflects the new state immediately). Also bumps the
scan generation so an in-flight background scan won't re-commit pre-import records."""
_staging_scan_generation["value"] += 1
_staging_scan_cache.update({"path": None, "ts": 0.0, "records": None})
def staging_files(runtime: ImportRouteRuntime) -> tuple[Dict[str, Any], int]: def staging_files(runtime: ImportRouteRuntime) -> tuple[Dict[str, Any], int]:
"""Scan the staging folder and return audio files with tag metadata.""" """Scan the staging folder and return audio files with tag metadata."""
try: try:
staging_path = runtime.get_staging_path() staging_path = runtime.get_staging_path()
os.makedirs(staging_path, exist_ok=True) os.makedirs(staging_path, exist_ok=True)
records, scanning = _records_or_scanning_payload(runtime, staging_path) files = []
if scanning is not None: for root, _dirs, filenames in os.walk(staging_path):
return scanning, 200 for fname in filenames:
ext = os.path.splitext(fname)[1].lower()
if ext not in AUDIO_EXTENSIONS:
continue
full_path = os.path.join(root, fname)
rel_path = os.path.relpath(full_path, staging_path)
files = [ meta = runtime.read_staging_file_metadata(full_path, rel_path)
{
"filename": r["filename"], files.append(
"rel_path": r["rel_path"], {
"full_path": r["full_path"], "filename": fname,
"title": r["title"], "rel_path": rel_path,
"artist": r["albumartist"] or r["artist"] or "Unknown Artist", "full_path": full_path,
"album": r["album"], "title": meta["title"],
"track_number": r["track_number"], "artist": meta["albumartist"] or meta["artist"] or "Unknown Artist",
"disc_number": r["disc_number"], "album": meta["album"],
"extension": r["extension"], "track_number": meta["track_number"],
} "disc_number": meta["disc_number"],
for r in records "extension": ext,
] }
)
files.sort(key=lambda f: f["filename"].lower()) files.sort(key=lambda f: f["filename"].lower())
return {"success": True, "files": files, "staging_path": staging_path}, 200 return {"success": True, "files": files, "staging_path": staging_path}, 200
@ -301,28 +114,32 @@ def staging_groups(runtime: ImportRouteRuntime) -> tuple[Dict[str, Any], int]:
if not os.path.isdir(staging_path): if not os.path.isdir(staging_path):
return {"success": True, "groups": []}, 200 return {"success": True, "groups": []}, 200
records, scanning = _records_or_scanning_payload(runtime, staging_path)
if scanning is not None:
return scanning, 200
album_groups = {} album_groups = {}
for r in records: for root, _dirs, filenames in os.walk(staging_path):
album = r["album"] for fname in filenames:
artist = r["albumartist"] or r["artist"] ext = os.path.splitext(fname)[1].lower()
if not album or not artist: if ext not in AUDIO_EXTENSIONS:
continue continue
full_path = os.path.join(root, fname)
rel_path = os.path.relpath(full_path, staging_path)
key = (album.lower().strip(), artist.lower().strip()) meta = runtime.read_staging_file_metadata(full_path, rel_path)
if key not in album_groups: album = meta["album"]
album_groups[key] = {"album": album.strip(), "artist": artist.strip(), "files": []} artist = meta["albumartist"] or meta["artist"]
album_groups[key]["files"].append( if not album or not artist:
{ continue
"filename": r["filename"],
"full_path": r["full_path"], key = (album.lower().strip(), artist.lower().strip())
"title": r["title"], if key not in album_groups:
"track_number": r["track_number"], album_groups[key] = {"album": album.strip(), "artist": artist.strip(), "files": []}
} album_groups[key]["files"].append(
) {
"filename": fname,
"full_path": full_path,
"title": meta["title"],
"track_number": meta["track_number"],
}
)
groups = [] groups = []
for group in album_groups.values(): for group in album_groups.values():
@ -352,21 +169,30 @@ def staging_hints(runtime: ImportRouteRuntime) -> tuple[Dict[str, Any], int]:
if not os.path.isdir(staging_path): if not os.path.isdir(staging_path):
return {"success": True, "hints": []}, 200 return {"success": True, "hints": []}, 200
records, scanning = _records_or_scanning_payload(runtime, staging_path)
if scanning is not None:
return scanning, 200
tag_albums = {} tag_albums = {}
folder_hints = {} folder_hints = {}
for r in records: for root, _dirs, filenames in os.walk(staging_path):
if r["top_folder"]: audio_files = [f for f in filenames if os.path.splitext(f)[1].lower() in AUDIO_EXTENSIONS]
folder_hints[r["top_folder"]] = folder_hints.get(r["top_folder"], 0) + 1 if not audio_files:
continue
album = r["album"] rel_dir = os.path.relpath(root, staging_path)
artist = r["artist"] or r["albumartist"] if rel_dir != ".":
if album: top_folder = rel_dir.split(os.sep)[0]
key = (album.strip(), (artist or "").strip()) folder_hints[top_folder] = folder_hints.get(top_folder, 0) + len(audio_files)
tag_albums[key] = tag_albums.get(key, 0) + 1
for fname in audio_files:
full_path = os.path.join(root, fname)
try:
tags = runtime.read_tags(full_path)
if tags:
album = (tags.get("album") or [None])[0]
artist = (tags.get("artist") or (tags.get("albumartist") or [None]))[0]
if album:
key = (album.strip(), (artist or "").strip())
tag_albums[key] = tag_albums.get(key, 0) + 1
except Exception as exc:
runtime.logger.debug("tag read failed: %s", exc)
queries = [] queries = []
seen_queries_lower = set() seen_queries_lower = set()
@ -396,7 +222,7 @@ def staging_suggestions() -> tuple[Dict[str, Any], int]:
"success": True, "success": True,
"suggestions": cache["suggestions"], "suggestions": cache["suggestions"],
"ready": cache["built"], "ready": cache["built"],
"primary_source": _get_primary_source_label(), "primary_source": _get_primary_source(),
}, 200 }, 200
@ -413,10 +239,7 @@ def search_albums(runtime: ImportRouteRuntime, query: str, limit: int = 12) -> t
runtime.hydrabase_worker.enqueue(query, "albums") runtime.hydrabase_worker.enqueue(query, "albums")
albums = runtime.search_import_albums(query, limit=limit) albums = runtime.search_import_albums(query, limit=limit)
# The label names the user's CONFIGURED source (Spotify Free reads as return {"success": True, "albums": albums, "primary_source": primary_source}, 200
# 'spotify', not the deezer fallback the functional source downgrades to).
return {"success": True, "albums": albums,
"primary_source": runtime.get_primary_source_label()}, 200
except Exception as exc: except Exception as exc:
runtime.logger.error("Error searching albums for import: %s", exc) runtime.logger.error("Error searching albums for import: %s", exc)
return {"success": False, "error": str(exc)}, 500 return {"success": False, "error": str(exc)}, 500
@ -543,11 +366,6 @@ def album_process(runtime: ImportRouteRuntime, data: Dict[str, Any]) -> tuple[Di
) )
runtime.refresh_import_suggestions_cache() runtime.refresh_import_suggestions_cache()
# Files just left staging — drop the shared scan so the next files/groups/hints
# reflects reality immediately instead of waiting out the cache TTL.
if processed > 0:
invalidate_staging_scan_cache()
return {"success": True, "processed": processed, "total": len(matches), "errors": errors}, 200 return {"success": True, "processed": processed, "total": len(matches), "errors": errors}, 200
except Exception as exc: except Exception as exc:
runtime.logger.error("Error processing album import: %s", exc) runtime.logger.error("Error processing album import: %s", exc)
@ -567,8 +385,7 @@ def search_tracks(runtime: ImportRouteRuntime, query: str, limit: int = 10) -> t
runtime.hydrabase_worker.enqueue(query, "tracks") runtime.hydrabase_worker.enqueue(query, "tracks")
tracks = runtime.search_import_tracks(query, limit=limit) tracks = runtime.search_import_tracks(query, limit=limit)
return {"success": True, "tracks": tracks, return {"success": True, "tracks": tracks, "primary_source": primary_source}, 200
"primary_source": runtime.get_primary_source_label()}, 200
except Exception as exc: except Exception as exc:
runtime.logger.error("Error searching tracks for import: %s", exc) runtime.logger.error("Error searching tracks for import: %s", exc)
return {"success": False, "error": str(exc)}, 500 return {"success": False, "error": str(exc)}, 500
@ -683,10 +500,6 @@ def singles_process(runtime: ImportRouteRuntime, files: list[Dict[str, Any]]) ->
) )
runtime.refresh_import_suggestions_cache() runtime.refresh_import_suggestions_cache()
# Files just left staging — drop the shared scan so the list updates immediately.
if processed > 0:
invalidate_staging_scan_cache()
return {"success": True, "processed": processed, "total": len(files), "errors": errors}, 200 return {"success": True, "processed": processed, "total": len(files), "errors": errors}, 200
except Exception as exc: except Exception as exc:
runtime.logger.error("Error processing singles import: %s", exc) runtime.logger.error("Error processing singles import: %s", exc)

View file

@ -252,7 +252,7 @@ def record_library_history_download(context: Dict[str, Any]) -> None:
origin, origin_context = derive_download_origin(context) origin, origin_context = derive_download_origin(context)
db = get_database() db = get_database()
_history_id = db.add_library_history_entry( db.add_library_history_entry(
event_type="download", event_type="download",
title=title, title=title,
artist_name=artist_name, artist_name=artist_name,
@ -270,10 +270,6 @@ def record_library_history_download(context: Dict[str, Any]) -> None:
origin_context=origin_context, origin_context=origin_context,
verification_status=context.get("_verification_status"), verification_status=context.get("_verification_status"),
) )
# Stash the row id so the live download task can link to its
# library_history row (the Unverified review queue needs it).
if isinstance(_history_id, int) and _history_id > 0:
context["_history_id"] = _history_id
except Exception as e: except Exception as e:
logger.debug("library history record failed: %s", e) logger.debug("library history record failed: %s", e)
@ -569,22 +565,19 @@ def record_soulsync_library_entry(context: Dict[str, Any], artist_context: Dict[
# ── Album row: same insert-or-fill-empty-fields shape ── # ── Album row: same insert-or-fill-empty-fields shape ──
album_source_col = source_columns.get("album") album_source_col = source_columns.get("album")
# Group by CANONICAL release id when we have one (not just the name cursor.execute(
# string), so differently-named imports of the SAME release land in "SELECT id FROM albums WHERE id = ? AND server_source = 'soulsync'",
# one album row instead of splitting — which left the repair jobs (album_id,),
# dressing each split row in its own cover art (Sokhi). Precedence:
# name-hash id -> source release id -> (title, artist). Falls back to
# the legacy name match, so nothing that grouped before stops now.
from core.imports.album_grouping import find_existing_soulsync_album_id
existing_album_id = find_existing_soulsync_album_id(
cursor, name_key_id=album_id, artist_id=artist_id, album_name=album_name,
album_source_col=album_source_col, album_source_id=album_source_id,
) )
if existing_album_id is not None: row = cursor.fetchone()
album_id = existing_album_id if not row:
row = (album_id,) cursor.execute(
else: "SELECT id FROM albums WHERE title COLLATE NOCASE = ? AND artist_id = ? AND server_source = 'soulsync' LIMIT 1",
row = None (album_name, artist_id),
)
row = cursor.fetchone()
if row:
album_id = row[0]
if row: if row:
_fill_empty_columns( _fill_empty_columns(

View file

@ -1,289 +0,0 @@
"""Audio-completeness guard — detect files whose container duration looks
right but whose REAL audio is far shorter, or mostly silence.
Motivating bug: HiFi/Monochrome HLS assembly can yield a file whose container
claims the full track length (e.g. 3:08) while only ~30s of audio actually
decodes the rest is missing. The duration-agreement and quality guards both
pass (mutagen reads the container's 3:08 and the format/bitrate are fine), so
nothing catches it until you listen. ffmpeg's ``time=`` even reports 0 with no
error on such a file, so the robust signal is to DECODE the audio and compare
the real duration (sample count / sample rate, via ``astats``) against the
container duration. A separate ``silencedetect`` pass also flags genuine
silence-padding.
The parsers here are pure and unit-tested; the ffmpeg invocations are
integration glue that fails open (returns None) when ffmpeg or mutagen can't
run, so a tooling problem never blocks a legitimate import.
"""
from __future__ import annotations
import os
import re
import subprocess
from typing import Optional
from utils.logging_config import get_logger
logger = get_logger("imports.silence")
# Real decoded audio must cover at least this fraction of the container
# duration. A legit file decodes to ~100% (encoder padding aside); a truncated
# file decodes to a small fraction (the Blossom file: 30s of a 188s container
# = 16%). 0.85 leaves generous headroom against false positives.
DEFAULT_MIN_DURATION_RATIO = 0.85
_SAMPLES_RE = re.compile(r"Number of samples:\s*([0-9]+)")
# Defaults: treat audio below -50 dB lasting >= 2s as silence, and reject when
# more than half the track is silent. A normal song — even with quiet intros/
# outros — sits far below 0.5; a 30s-real + padded-silence file sits near 0.85.
DEFAULT_NOISE_DB = -50
DEFAULT_MIN_SILENCE_S = 2.0
DEFAULT_THRESHOLD = 0.5
_SILENCE_DURATION_RE = re.compile(r"silence_duration:\s*([0-9]+(?:\.[0-9]+)?)")
def silence_ratio_from_output(ffmpeg_stderr: str, total_duration_s: float) -> float:
"""Fraction of *total_duration_s* covered by detected silence.
Sums every ``silence_duration: X`` reported by ffmpeg ``silencedetect``
and divides by the track length. Capped at 1.0; returns 0.0 when the
duration is unknown/zero or no silence was reported.
"""
if not total_duration_s or total_duration_s <= 0:
return 0.0
total_silence = sum(float(m) for m in _SILENCE_DURATION_RE.findall(ffmpeg_stderr or ""))
if total_silence <= 0:
return 0.0
return min(total_silence / total_duration_s, 1.0)
def is_mostly_silent_reason(
ffmpeg_stderr: str,
total_duration_s: float,
*,
threshold: float = DEFAULT_THRESHOLD,
) -> Optional[str]:
"""Return a rejection reason when the silent fraction meets *threshold*."""
ratio = silence_ratio_from_output(ffmpeg_stderr, total_duration_s)
if ratio >= threshold:
pct = round(ratio * 100)
audible_s = round(total_duration_s * (1 - ratio))
return (
f"Audio is mostly silent: {pct}% silence (only ~{audible_s}s audible of "
f"{round(total_duration_s)}s) — likely a truncated/preview file padded "
f"to full length"
)
return None
def _ffmpeg_available() -> bool:
try:
subprocess.run(
["ffmpeg", "-version"],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
timeout=10, check=True,
)
return True
except (FileNotFoundError, subprocess.SubprocessError, OSError):
return False
def _probe_duration_s(file_path: str) -> Optional[float]:
try:
from mutagen import File as MutagenFile
audio = MutagenFile(file_path)
if audio and audio.info and getattr(audio.info, "length", None):
return float(audio.info.length)
except Exception as exc: # pragma: no cover - defensive
logger.debug("silence guard duration probe failed for %s: %s", file_path, exc)
return None
def detect_mostly_silent(
file_path: str,
*,
threshold: float = DEFAULT_THRESHOLD,
noise_db: int = DEFAULT_NOISE_DB,
min_silence_s: float = DEFAULT_MIN_SILENCE_S,
) -> Optional[str]:
"""Run ffmpeg ``silencedetect`` over *file_path* and return a rejection
reason when the file is mostly silence, else None.
Fails open: returns None when ffmpeg/mutagen are unavailable or error, so
a tooling problem never quarantines a legitimate file.
"""
if not _ffmpeg_available():
logger.debug("silence guard skipped — ffmpeg not available")
return None
total_duration_s = _probe_duration_s(file_path)
if not total_duration_s:
return None
try:
proc = subprocess.run(
[
"ffmpeg", "-hide_banner", "-nostats", "-i", file_path,
"-af", f"silencedetect=noise={noise_db}dB:d={min_silence_s}",
"-f", "null", "-",
],
stdout=subprocess.DEVNULL, stderr=subprocess.PIPE,
timeout=120,
)
except (subprocess.SubprocessError, OSError) as exc:
logger.debug("silence guard ffmpeg run failed for %s: %s", file_path, exc)
return None
stderr = proc.stderr.decode("utf-8", errors="replace") if proc.stderr else ""
return is_mostly_silent_reason(stderr, total_duration_s, threshold=threshold)
# ── Truncation: real decoded duration vs container duration ────────────────
def measured_duration_from_astats(astats_stderr: str, sample_rate: int) -> Optional[float]:
"""Real decoded audio duration in seconds from ffmpeg ``astats`` output.
``astats`` reports the per-channel ``Number of samples``; dividing by the
sample rate gives the true decoded length. Returns None when the sample
count or sample rate is unavailable.
"""
if not sample_rate or sample_rate <= 0:
return None
m = _SAMPLES_RE.search(astats_stderr or "")
if not m:
return None
return int(m.group(1)) / float(sample_rate)
def is_dsd_path(file_path: str) -> bool:
"""True for DSD audio (.dsf / .dff). The decoded-samples truncation check is
invalid for DSD: ffmpeg decodes DSD to PCM at a different rate than the DSD
container's 2.8 MHz, so samples ÷ container-sample-rate massively under-counts
and would falsely report the file as truncated (#939)."""
return os.path.splitext(str(file_path or ''))[1].lower() in ('.dsf', '.dff')
def incomplete_audio_reason(
measured_s: Optional[float],
container_s: Optional[float],
*,
min_ratio: float = DEFAULT_MIN_DURATION_RATIO,
) -> Optional[str]:
"""Return a rejection reason when the real decoded duration falls short of
the container duration (a truncated file whose metadata over-states length).
"""
if not measured_s or not container_s or container_s <= 0:
return None
if measured_s >= container_s * min_ratio:
return None
pct = round(measured_s / container_s * 100)
return (
f"Incomplete audio: only ~{round(measured_s)}s actually decodes of a "
f"{round(container_s)}s file ({pct}%) — truncated/broken download "
f"(container duration over-states the real audio)"
)
def _measured_audio_duration_s(file_path: str, sample_rate: int) -> Optional[float]:
try:
proc = subprocess.run(
["ffmpeg", "-hide_banner", "-nostats", "-i", file_path,
"-af", "astats=metadata=1", "-f", "null", "-"],
stdout=subprocess.DEVNULL, stderr=subprocess.PIPE, timeout=120,
)
except (subprocess.SubprocessError, OSError) as exc:
logger.debug("astats run failed for %s: %s", file_path, exc)
return None
stderr = proc.stderr.decode("utf-8", errors="replace") if proc.stderr else ""
return measured_duration_from_astats(stderr, sample_rate)
def detect_incomplete_audio(
file_path: str,
*,
min_ratio: float = DEFAULT_MIN_DURATION_RATIO,
) -> Optional[str]:
"""Decode the file and reject when the real audio is far shorter than the
container claims. Fails open when ffmpeg/mutagen are unavailable.
"""
if not _ffmpeg_available():
return None
try:
from mutagen import File as MutagenFile
audio = MutagenFile(file_path)
if not (audio and audio.info):
return None
container_s = float(getattr(audio.info, "length", 0) or 0)
sample_rate = int(getattr(audio.info, "sample_rate", 0) or 0)
except Exception as exc: # pragma: no cover - defensive
logger.debug("container probe failed for %s: %s", file_path, exc)
return None
measured_s = _measured_audio_duration_s(file_path, sample_rate)
return incomplete_audio_reason(measured_s, container_s, min_ratio=min_ratio)
def detect_broken_audio(
file_path: str,
*,
min_ratio: float = DEFAULT_MIN_DURATION_RATIO,
threshold: float = DEFAULT_THRESHOLD,
noise_db: int = DEFAULT_NOISE_DB,
min_silence_s: float = DEFAULT_MIN_SILENCE_S,
) -> Optional[str]:
"""Combined post-download audio guard: reject a file that is truncated
(real audio far shorter than the container) or mostly silence. Returns the
first failure reason, or None when the audio looks complete.
Runs a SINGLE ffmpeg decode pass with both the ``astats`` (truncation) and
``silencedetect`` (silence) filters chained one decode of the file feeds
both checks instead of two full decodes. Halves the CPU cost versus running
``detect_incomplete_audio`` and ``detect_mostly_silent`` back to back.
Fails open: returns None when ffmpeg/mutagen are unavailable or error, so a
tooling problem never quarantines a legitimate file.
"""
if not _ffmpeg_available():
logger.debug("audio guard skipped — ffmpeg not available")
return None
try:
from mutagen import File as MutagenFile
audio = MutagenFile(file_path)
if not (audio and audio.info):
return None
container_s = float(getattr(audio.info, "length", 0) or 0)
sample_rate = int(getattr(audio.info, "sample_rate", 0) or 0)
except Exception as exc: # pragma: no cover - defensive
logger.debug("container probe failed for %s: %s", file_path, exc)
return None
try:
proc = subprocess.run(
[
"ffmpeg", "-hide_banner", "-nostats", "-i", file_path,
"-af", f"astats=metadata=1,silencedetect=noise={noise_db}dB:d={min_silence_s}",
"-f", "null", "-",
],
stdout=subprocess.DEVNULL, stderr=subprocess.PIPE, timeout=120,
)
except (subprocess.SubprocessError, OSError) as exc:
logger.debug("audio guard ffmpeg run failed for %s: %s", file_path, exc)
return None
stderr = proc.stderr.decode("utf-8", errors="replace") if proc.stderr else ""
# Truncation check first (real audio far shorter than the container) — but
# NOT for DSD: the astats sample-count ÷ DSD-rate math is invalid there and
# would always false-positive (#939). Silence detection below still applies.
if not is_dsd_path(file_path):
measured_s = measured_duration_from_astats(stderr, sample_rate)
reason = incomplete_audio_reason(measured_s, container_s, min_ratio=min_ratio)
if reason:
return reason
# Then silence-padding (mostly-silent file).
return is_mostly_silent_reason(stderr, container_s, threshold=threshold)

View file

@ -1,124 +0,0 @@
"""Single -> parent-album resolution.
When a track is matched to a SINGLE release (album_type 'single', the single's
name usually equal to the track title), it carries the single's name + the
single's source album id. The canonical grouping in
[core/imports/album_grouping.py] then files it under a different album row than
its album-mates, and the album-grouped repair jobs dress that row in the
single's art — songs of one album end up with different covers (Sokhi).
This module re-homes such a track onto the ALBUM it actually belongs to, so it
carries the album's name/id and groups with the rest of the album.
Design: the SELECTION is a pure, conservative function (no I/O), and the lookup
loop takes INJECTED fetchers, so both are unit-testable without a live metadata
client. CONSERVATIVE by intent it only re-homes a track when a real
``album``-type release's tracklist *contains that exact track*. It never
promotes a genuine standalone single and never guesses, because a wrong
promotion would mis-home a real single onto an album (the inverse bug).
"""
from __future__ import annotations
import re
from typing import Any, Callable, Dict, List, Optional
_WS = re.compile(r"\s+")
# Trailing version qualifiers that differ between a single and its album cut but
# don't change track identity (kept conservative — only the obvious ones).
_QUALIFIER = re.compile(
r"\s*[\(\[]\s*(album version|single version|radio edit|remaster(ed)?( \d{4})?)\s*[\)\]]\s*$",
re.IGNORECASE,
)
def _norm(s: Any) -> str:
"""Lowercase, strip a trailing '(Album Version)'-style qualifier, collapse
whitespace so 'Song' matches 'Song (Album Version)'."""
t = str(s or "").strip().lower()
t = _QUALIFIER.sub("", t)
return _WS.sub(" ", t).strip()
def _get(obj: Any, *keys: str, default=None):
for k in keys:
if isinstance(obj, dict):
if obj.get(k) is not None:
return obj.get(k)
else:
v = getattr(obj, k, None)
if v is not None:
return v
return default
def select_parent_album(track_title: str, candidate_albums: List[Dict[str, Any]]) -> Optional[Dict[str, Any]]:
"""Pick the parent ALBUM for ``track_title`` from normalized candidates, or
None. Each candidate is ``{name, album_type, tracks: [title, ...], ...}``.
Conservative rules a candidate qualifies ONLY when:
* it is an ``album`` release (never single / ep / compilation), and
* its name is not just the track title (that IS the single), and
* its tracklist contains the track by exact normalized title.
Returns the FIRST qualifying candidate (caller passes them in priority
order, so the result is deterministic).
"""
tgt = _norm(track_title)
if not tgt:
return None
for alb in candidate_albums or []:
if str(_get(alb, "album_type", default="album")).lower() != "album":
continue
if _norm(_get(alb, "name", "title", default="")) == tgt:
continue
tracks = _get(alb, "tracks", default=[]) or []
if any(_norm(t) == tgt for t in tracks):
return alb
return None
def resolve_single_to_album(
track_title: str,
*,
fetch_album_candidates: Callable[[], List[Dict[str, Any]]],
fetch_album_tracks: Callable[[Dict[str, Any]], List[str]],
max_albums: int = 8,
) -> Optional[Dict[str, Any]]:
"""Find the parent album for a single-matched track. I/O is INJECTED so this
is testable without a live client:
* ``fetch_album_candidates()`` -> the artist's ALBUM-type releases (dicts
with name/album_type/id/source), in priority order.
* ``fetch_album_tracks(album)`` -> that album's track titles.
Probes at most ``max_albums`` albums, lazily (stops at the first that
contains the track). Fail-safe: any error / no confident match -> None
(the track stays as it was matched). Returns the normalized winning album
``{name, album_type, album_id, source, tracks}`` or None.
"""
if not _norm(track_title):
return None
try:
albums = fetch_album_candidates() or []
except Exception:
return None
probed = 0
for alb in albums:
if str(_get(alb, "album_type", default="album")).lower() != "album":
continue
if probed >= max_albums:
break
probed += 1
try:
tracks = fetch_album_tracks(alb) or []
except Exception:
continue
normalized = {
"name": _get(alb, "name", "title", default=""),
"album_type": "album",
"album_id": _get(alb, "id", "album_id"),
"source": _get(alb, "source"),
"tracks": list(tracks),
}
if select_parent_album(track_title, [normalized]):
return normalized
return None

View file

@ -56,12 +56,6 @@ def get_primary_source() -> str:
return _get_primary_source() return _get_primary_source()
def get_primary_source_label() -> str:
from core.metadata_service import get_primary_source_label as _get_primary_source_label
return _get_primary_source_label()
def get_source_priority(preferred_source: str): def get_source_priority(preferred_source: str):
from core.metadata_service import get_source_priority as _get_source_priority from core.metadata_service import get_source_priority as _get_source_priority

View file

@ -65,64 +65,17 @@ def _coerce_spotify_data(track_info: Any) -> dict:
return {} return {}
def read_embedded_track_number(file_path: str) -> Optional[int]:
"""Read the track position from a downloaded audio file's own tags.
Streaming sources (Deezer/deemix, Qobuz, Tidal) and most Soulseek
uploads write the correct album position into the file itself. That
tag is authoritative for the *source's* idea of the track's place on
its album more reliable than a filename guess so the resolver
consults it before falling back to the filename / default-1 floor.
Issue #874-adjacent / "Track 01" bug: a single Deezer track is matched
via Deezer's ``/search/track`` endpoint, which omits ``track_position``
(core/deezer_client.py), so the metadata context never carried the
real number but the downloaded file *does* (deemix wrote it). This
recovers it with no network call.
Returns a positive int, or None when the file has no usable
tracknumber tag / can't be read. Never raises. Handles the common
``"2/15"`` (number/total) form by taking the leading number.
"""
if not file_path:
return None
try:
from mutagen import File as MutagenFile
audio = MutagenFile(file_path, easy=True)
if audio is None:
return None
raw = audio.get('tracknumber')
if isinstance(raw, list):
raw = raw[0] if raw else None
if raw is None:
return None
# "2/15" -> "2"; bare "2" -> "2".
text = str(raw).split('/', 1)[0].strip()
return _coerce_positive(text)
except Exception:
return None
def resolve_track_number( def resolve_track_number(
album_info: Any, album_info: Any,
track_info: Any, track_info: Any,
file_path: str, file_path: str,
embedded_track_number: Any = None,
) -> Optional[int]: ) -> Optional[int]:
"""Walk the resolution chain and return the first valid positive """Walk the resolution chain and return the first valid positive
int found, or None when every source is missing / unusable. int found, or None when every source is missing / unusable.
Order: album_info -> track_info -> nested spotify_data -> filename -> Caller is responsible for the final default-1 floor leaving
``embedded_track_number`` (the source-written file tag, when the caller that out of this function so tests can pin "everything missing
supplies it). Caller is responsible for the final default-1 floor
leaving that out of this function so tests can pin "everything missing
returns None" separate from the floor behaviour. returns None" separate from the floor behaviour.
``embedded_track_number`` is passed in (not read here) so this stays a
pure function the file I/O lives in :func:`read_embedded_track_number`.
It is consulted **last**, only when every other source came up empty, so
it can never override a value the pre-fix resolver already produced it
only fills the gap that would otherwise hit the default-1 floor.
""" """
album_info = album_info if isinstance(album_info, dict) else {} album_info = album_info if isinstance(album_info, dict) else {}
track_info = track_info if isinstance(track_info, dict) else {} track_info = track_info if isinstance(track_info, dict) else {}
@ -143,69 +96,10 @@ def resolve_track_number(
# default-1 floor is the single source of that fallback — # default-1 floor is the single source of that fallback —
# otherwise this resolver would silently fill 1 and the # otherwise this resolver would silently fill 1 and the
# downstream floor logic would have no effect. # downstream floor logic would have no effect.
if file_path: if not file_path:
try: return None
from_filename = extract_explicit_track_number(file_path)
except Exception:
from_filename = None
ff = _coerce_positive(from_filename)
if ff is not None:
return ff
# Embedded source-written file tag is consulted LAST — only when every
# other source (metadata + the ripped-album "NN - Title" filename) came
# up empty. This is deliberate: it can ONLY fill the gap that would
# otherwise hit the caller's default-1 floor, so it never overrides a
# value the pre-fix resolver would have used. A correctly-named file
# with a stale/wrong embedded tag is therefore never regressed.
return _coerce_positive(embedded_track_number)
def normalize_disc_number(value) -> int:
"""Coerce a disc value to a positive int, defaulting to 1.
Every track in a multi-disc album MUST carry a disc number, or Jellyfin/Plex
leave the disc-less ones floating ungrouped above the disc sections (Sokhi's
"tracks 3/9/15 at the top"). Upstream sources can hand back 0, None, '', or a
non-numeric string for some tracks especially when a track resolved to a
different edition than its siblings and the tag-writer only wrote the disc
tag when it was truthy, so those tracks lost it entirely on the clear-then-
rewrite. Flooring to >=1 here means a track is never written disc-less.
"""
try: try:
n = int(value) # int, float, or clean int-string from_filename = extract_explicit_track_number(file_path)
except (TypeError, ValueError): except Exception:
try: from_filename = None
n = int(float(str(value).strip())) # tolerate "2.0" return _coerce_positive(from_filename)
except (TypeError, ValueError):
return 1
return n if n >= 1 else 1
def resolve_disc_for_track(original_search, album_info) -> int:
"""The disc number for a track — resolved IDENTICALLY for the 'Disc N' folder
(import pipeline) and the embedded tag (metadata.source), so the two can never
disagree.
Sokhi: the pipeline synced the resolved track_number into album_info (so the
folder matched the tag) but never did the same for disc the folder used
album_info's original disc (often 1) while the tag took the per-track disc
(e.g. 2/3 from a MusicBrainz multi-medium release). Result: a disc-2/3 track
landed in the Disc 1 folder, collapsing every disc's tracks into one folder.
Returns the first VALID positive disc the per-track search's, else the album
context's — else 1. A falsy/unknown (0/None/'') per-track disc falls through to
the album rather than flooring early. Single source of truth so both call sites
stay in lockstep."""
for src in ((original_search or {}), (album_info or {})):
raw = src.get("disc_number")
try:
n = int(raw)
except (TypeError, ValueError):
try:
n = int(float(str(raw).strip()))
except (TypeError, ValueError):
continue
if n >= 1:
return n
return 1

View file

@ -103,12 +103,9 @@ def select_version_mismatch_fallback(
# don't guess which the user wants. # don't guess which the user wants.
return None return None
# First tried = oldest = best (the retry walks candidates best-first; what # First tried = oldest = highest-confidence (the retry walks candidates
# "best" means follows the active ordering — confidence-first by default, or # best-first). The id is a "<date>_<time>_<name>" timestamp prefix, so the
# ranked-target quality when best_quality mode / the rank_candidates_by_quality # lexicographically smallest id is the earliest attempt.
# toggle is on, so this naturally accepts the highest-quality candidate then).
# The id is a "<date>_<time>_<name>" timestamp prefix, so the lexicographically
# smallest id is the earliest attempt.
return min((e for _, e in candidates), key=lambda e: e["id"]) return min((e for _, e in candidates), key=lambda e: e["id"])

View file

@ -739,27 +739,9 @@ class iTunesClient:
cache = get_metadata_cache() cache = get_metadata_cache()
cached = cache.get_entity('itunes', 'album', f"{album_id}_tracks") cached = cache.get_entity('itunes', 'album', f"{album_id}_tracks")
if cached and cached.get('items'): if cached and cached.get('items'):
# #918 follow-up: a tracks entry cached BEFORE the limit=200 fix is truncated return cached
# to 50 and survives in the persistent cache (30-day TTL), so every window that
# loads this album from cache still shows 50 — not just the one path that was
# re-fetched fresh. Self-heal: entries written by the fixed fetch carry
# `_complete`; a legacy entry without it is re-validated against the album's
# known trackCount and re-fetched if it's short. (trackCount comes from the
# collection metadata and is unaffected by the tracks-limit bug.)
if cached.get('_complete'):
return cached
album_meta = cache.get_entity('itunes', 'album', str(album_id))
expected = (album_meta or {}).get('trackCount')
if not (isinstance(expected, int) and expected > len(cached['items'])):
return cached
logger.info(
"iTunes album %s tracks cache looks truncated (%d cached < %d trackCount) — refetching",
album_id, len(cached['items']), expected,
)
# #918: the iTunes Lookup API returns only 50 related entities unless `limit` is results = self._lookup(id=album_id, entity='song')
# passed (max 200), so albums >50 tracks were truncated in the download window.
results = self._lookup(id=album_id, entity='song', limit=200)
if not results: if not results:
return None return None
@ -781,7 +763,7 @@ class iTunesClient:
try: try:
fb_results = self.session.get( fb_results = self.session.get(
self.LOOKUP_URL, self.LOOKUP_URL,
params={'id': album_id, 'entity': 'song', 'country': fallback, 'limit': 200}, # #918 params={'id': album_id, 'entity': 'song', 'country': fallback},
timeout=15 timeout=15
) )
if fb_results.status_code == 200: if fb_results.status_code == 200:
@ -867,11 +849,7 @@ class iTunesClient:
'items': tracks, 'items': tracks,
'total': len(tracks), 'total': len(tracks),
'limit': len(tracks), 'limit': len(tracks),
'next': None, 'next': None
# Marks this entry as fetched with the limit=200 query (#918) so the
# stale-cache self-heal above trusts it and never re-fetches in a loop —
# important for region-restricted albums where len(tracks) < trackCount.
'_complete': True,
} }
# Cache the album tracks listing # Cache the album tracks listing

View file

@ -5,7 +5,6 @@ from datetime import datetime
import json import json
from utils.logging_config import get_logger from utils.logging_config import get_logger
from config.settings import config_manager from config.settings import config_manager
from core.library.bulk_paginate import paginate_all_items
# Shared dataclasses live in the neutral media_server package — every # Shared dataclasses live in the neutral media_server package — every
# server client used to define a near-identical XTrackInfo / # server client used to define a near-identical XTrackInfo /
@ -106,7 +105,6 @@ class JellyfinTrack:
self.title = jellyfin_data.get('Name', 'Unknown Track') self.title = jellyfin_data.get('Name', 'Unknown Track')
self.duration = jellyfin_data.get('RunTimeTicks', 0) // 10000 # Convert from ticks to milliseconds self.duration = jellyfin_data.get('RunTimeTicks', 0) // 10000 # Convert from ticks to milliseconds
self.trackNumber = jellyfin_data.get('IndexNumber') self.trackNumber = jellyfin_data.get('IndexNumber')
self.discNumber = jellyfin_data.get('ParentIndexNumber') # multi-disc: disc number
self.year = jellyfin_data.get('ProductionYear') self.year = jellyfin_data.get('ProductionYear')
self.userRating = jellyfin_data.get('UserData', {}).get('Rating') self.userRating = jellyfin_data.get('UserData', {}).get('Rating')
self.addedAt = self._parse_date(jellyfin_data.get('DateCreated')) self.addedAt = self._parse_date(jellyfin_data.get('DateCreated'))
@ -513,8 +511,12 @@ class JellyfinClient(MediaServerClient):
try: try:
# SIMPLIFIED APPROACH: Fetch all tracks, then all albums separately (robust and fast) # SIMPLIFIED APPROACH: Fetch all tracks, then all albums separately (robust and fast)
logger.info("Fetching all tracks in bulk...") logger.info("Fetching all tracks in bulk...")
all_tracks = []
def _fetch_tracks_page(start_index, limit): start_index = 0
limit = 10000
consecutive_failures = 0
while True:
params = { params = {
'ParentId': self.music_library_id, 'ParentId': self.music_library_id,
'IncludeItemTypes': 'Audio', 'IncludeItemTypes': 'Audio',
@ -525,19 +527,41 @@ class JellyfinClient(MediaServerClient):
'StartIndex': start_index, 'StartIndex': start_index,
'Limit': limit 'Limit': limit
} }
response = self._make_request(f'/Users/{self.user_id}/Items', params) response = self._make_request(f'/Users/{self.user_id}/Items', params)
return response.get('Items', []) if response else None # None = failed page
if not response:
# Page in modest chunks so progress is reported every page — a single consecutive_failures += 1
# huge silent request used to trip the 300s no-progress watchdog on # Wait before retrying — the server may still be processing the timed-out request
# slow servers even though it was alive (see bulk_paginate docstring). time.sleep(5)
all_tracks = paginate_all_items( if limit > 1000:
_fetch_tracks_page, limit = limit // 2
report_progress=self._progress_callback, consecutive_failures = 0 # Reset — give the smaller batch a fair chance
label="tracks", logger.warning(f"Track fetch failed - reducing batch size to {limit}")
on_retry_wait=lambda: time.sleep(5), continue
) elif consecutive_failures >= 2:
logger.warning("Multiple track fetch failures at minimum batch size - stopping")
break
else:
logger.warning("Track fetch failed at minimum batch size - retrying once")
continue
consecutive_failures = 0
batch_tracks = response.get('Items', [])
if not batch_tracks:
break
all_tracks.extend(batch_tracks)
if len(batch_tracks) < limit:
break
start_index += limit
progress_msg = f"Fetched {len(all_tracks)} tracks so far..."
logger.info(f" {progress_msg} (batch size: {limit})")
if self._progress_callback:
self._progress_callback(progress_msg)
# Group tracks by album ID for instant lookup # Group tracks by album ID for instant lookup
self._track_cache = {} self._track_cache = {}
for track_data in all_tracks: for track_data in all_tracks:
@ -553,8 +577,12 @@ class JellyfinClient(MediaServerClient):
# STEP 2: Fetch all albums in bulk (same proven pattern) # STEP 2: Fetch all albums in bulk (same proven pattern)
logger.info("Fetching all albums in bulk...") logger.info("Fetching all albums in bulk...")
all_albums = []
def _fetch_albums_page(start_index, limit): start_index = 0
limit = 10000
consecutive_failures = 0
while True:
params = { params = {
'ParentId': self.music_library_id, 'ParentId': self.music_library_id,
'IncludeItemTypes': 'MusicAlbum', 'IncludeItemTypes': 'MusicAlbum',
@ -565,16 +593,41 @@ class JellyfinClient(MediaServerClient):
'StartIndex': start_index, 'StartIndex': start_index,
'Limit': limit 'Limit': limit
} }
response = self._make_request(f'/Users/{self.user_id}/Items', params) response = self._make_request(f'/Users/{self.user_id}/Items', params)
return response.get('Items', []) if response else None # None = failed page
if not response:
all_albums = paginate_all_items( consecutive_failures += 1
_fetch_albums_page, # Wait before retrying — the server may still be processing the timed-out request
report_progress=self._progress_callback, time.sleep(5)
label="albums", if limit > 1000:
on_retry_wait=lambda: time.sleep(5), limit = limit // 2
) consecutive_failures = 0 # Reset — give the smaller batch a fair chance
logger.warning(f"Album fetch failed - reducing batch size to {limit}")
continue
elif consecutive_failures >= 2:
logger.warning("Multiple album fetch failures at minimum batch size - stopping")
break
else:
logger.warning("Album fetch failed at minimum batch size - retrying once")
continue
consecutive_failures = 0
batch_albums = response.get('Items', [])
if not batch_albums:
break
all_albums.extend(batch_albums)
if len(batch_albums) < limit:
break
start_index += limit
progress_msg = f"Fetched {len(all_albums)} albums so far..."
logger.info(f" {progress_msg} (batch size: {limit})")
if self._progress_callback:
self._progress_callback(progress_msg)
# Group albums by artist ID for instant lookup # Group albums by artist ID for instant lookup
self._album_cache = {} self._album_cache = {}
for album_data in all_albums: for album_data in all_albums:
@ -1658,81 +1711,6 @@ class JellyfinClient(MediaServerClient):
logger.error(f"Error reconciling Jellyfin playlist '{playlist_name}': {e}") logger.error(f"Error reconciling Jellyfin playlist '{playlist_name}': {e}")
return False return False
def get_playlist_track_ids(self, playlist_id: str) -> List[str]:
"""The playlist's current track ids (Item Ids), in current order. [] on miss."""
if not self.ensure_connection():
return []
try:
resp = self._make_request(f'/Playlists/{playlist_id}/Items', {'UserId': self.user_id})
if not resp:
return []
return [str(i.get('Id')) for i in resp.get('Items', []) if i.get('Id')]
except Exception as e:
logger.error(f"Error getting Jellyfin playlist track ids '{playlist_id}': {e}")
return []
def reorder_playlist(self, playlist_id, playlist_name: str, ordered_ids) -> bool:
"""In-place reorder a playlist to an exact ordered track-id list ('Align
playlists'). Removes any current entry whose track NOT in ``ordered_ids``
('Mirror source' drops extras; 'Keep extras' keeps them in the list), then
moves each desired track to its target index via the Jellyfin Move endpoint.
Operates on the existing playlist (DELETE EntryIds + Items/{entryId}/Move/{i})
so its poster, name and Id survive no delete/recreate."""
if not self.ensure_connection():
return False
try:
import requests
ordered = [str(i) for i in (ordered_ids or []) if str(i)]
ordered_set = set(ordered)
# Entries carry both the track Id and the PlaylistItemId (entry id);
# move/remove operate on the entry id.
entries = [] # (track_id, entry_id) in current order
resp = self._make_request(f'/Playlists/{playlist_id}/Items', {'UserId': self.user_id})
if resp:
for item in resp.get('Items', []):
tid = str(item.get('Id') or '')
eid = str(item.get('PlaylistItemId') or '')
if tid:
entries.append((tid, eid))
if not entries:
logger.error(f"Jellyfin reorder: no entries for playlist {playlist_id}")
return False
by_tid = {tid: eid for tid, eid in entries}
hdr = {'X-Emby-Token': self.api_key}
# Drop entries not in the desired list (extras, for Mirror source).
extra_eids = [eid for tid, eid in entries if tid not in ordered_set and eid]
for i in range(0, len(extra_eids), 100):
batch = extra_eids[i:i + 100]
r = requests.delete(
f"{self.base_url}/Playlists/{playlist_id}/Items",
params={'EntryIds': ','.join(batch)}, headers=hdr, timeout=30,
)
if r.status_code not in (200, 204):
logger.error(f"Jellyfin reorder remove failed: HTTP {r.status_code}")
return False
# Move each desired track to its target index, ascending — each move
# lands the item exactly at index i without disturbing 0..i-1.
for idx, tid in enumerate(ordered):
eid = by_tid.get(tid)
if not eid:
continue
r = requests.post(
f"{self.base_url}/Playlists/{playlist_id}/Items/{eid}/Move/{idx}",
headers=hdr, timeout=30,
)
if r.status_code not in (200, 204):
logger.warning(f"Jellyfin reorder move failed for {tid}: HTTP {r.status_code}")
return False
logger.info(f"Aligned Jellyfin playlist '{playlist_name}' order ({len(ordered)} tracks)")
return True
except Exception as e:
logger.error(f"Error reordering Jellyfin playlist '{playlist_name}': {e}")
return False
def update_playlist(self, playlist_name: str, tracks) -> bool: def update_playlist(self, playlist_name: str, tracks) -> bool:
"""Update an existing playlist or create it if it doesn't exist""" """Update an existing playlist or create it if it doesn't exist"""
if not self.ensure_connection(): if not self.ensure_connection():

View file

@ -1,93 +0,0 @@
"""Paginate a bulk media-server fetch while feeding the no-progress watchdog.
The library scan fetches every track/album by paging a server API. The DB-update
watchdog (``core/database_update_health.py``) kills a job that reports no progress
for 300s. The old Jellyfin fetch used a single 10 000-item page, so a whole
library came back in ONE request that emitted NO progress while it was in flight
on a slow server that single request exceeded 300s and the watchdog declared the
job "stuck" even though it was alive, not hung (Discord: DXP4800 NAS, 7148 tracks,
"Fetching all tracks in bulk…").
``paginate_all_items`` pages at a size chosen so progress is emitted on a cadence
set by the PAGE SIZE, not the library size the watchdog is fed every page, so it
can never starve mid-fetch regardless of how big the library is. It is pure: all
I/O lives in the injected ``fetch_page``, so the pagination + progress + failure-
shrink logic is unit-testable without a server.
This does NOT change WHAT is fetched (same query, same fields, same items) only
how it's paged and that every page reports progress (the old loop skipped progress
on the final/only page, which is the entire bug for a sub-page-size library).
"""
from __future__ import annotations
from typing import Any, Callable, List, Optional
# Page size for bulk library fetches. Small enough that a single request stays
# well under the 300s no-progress watchdog even on a slow NAS, and that progress
# is reported every page. NOT a performance knob — a resilience/observability one.
DEFAULT_PAGE_SIZE = 1000
# Floor the failure-shrink can reach before giving up — a server that can't return
# even this many items in one request is genuinely struggling.
DEFAULT_MIN_PAGE_SIZE = 250
def paginate_all_items(
fetch_page: Callable[[int, int], Optional[List[Any]]],
*,
report_progress: Optional[Callable[[str], None]] = None,
label: str = "items",
page_size: int = DEFAULT_PAGE_SIZE,
min_page_size: int = DEFAULT_MIN_PAGE_SIZE,
on_retry_wait: Optional[Callable[[], None]] = None,
) -> List[Any]:
"""Page through ``fetch_page(start_index, limit)`` until the server is drained.
``fetch_page`` returns the page's items (a list, possibly empty = end), or
``None`` to signal a FAILED request (timeout/error) on failure the page size
is halved down to ``min_page_size`` and retried, then abandoned after two
consecutive failures at the floor.
Progress is reported after EVERY non-empty page (including the final/only one),
so a no-progress watchdog is fed on a cadence set by ``page_size`` never by
the total library size. Returns every item gathered.
"""
items: List[Any] = []
start_index = 0
limit = page_size
consecutive_failures = 0
while True:
batch = fetch_page(start_index, limit)
if batch is None: # failed request
consecutive_failures += 1
if on_retry_wait is not None:
on_retry_wait()
if limit > min_page_size:
limit = max(min_page_size, limit // 2)
consecutive_failures = 0 # give the smaller batch a fair chance
continue
if consecutive_failures >= 2:
break # struggling at the floor — stop with what we have
continue
consecutive_failures = 0
if not batch:
break # drained
items.extend(batch)
# Feed the watchdog on EVERY page — this is the line the old loop only ran
# when there was a *next* page, so a sub-page-size library reported nothing.
if report_progress is not None:
report_progress(f"Fetched {len(items)} {label} so far...")
if len(batch) < limit:
break # last (partial) page
start_index += limit
return items
__all__ = ["paginate_all_items", "DEFAULT_PAGE_SIZE", "DEFAULT_MIN_PAGE_SIZE"]

View file

@ -10,7 +10,6 @@ from __future__ import annotations
from dataclasses import dataclass from dataclasses import dataclass
import logging import logging
import os import os
import re
import shutil import shutil
import uuid import uuid
from typing import Any, Callable, Dict, Optional from typing import Any, Callable, Dict, Optional
@ -122,20 +121,6 @@ def import_existing_track_for_album_slot(album_id: str, payload: dict, deps: Mis
if album_data.get("server_source") and source_track.get("server_source") and album_data["server_source"] != source_track["server_source"]: if album_data.get("server_source") and source_track.get("server_source") and album_data["server_source"] != source_track["server_source"]:
raise MissingTrackImportError("Selected track belongs to a different library source", 400) raise MissingTrackImportError("Selected track belongs to a different library source", 400)
# #917: "I have this" rebuilds the destination path from album metadata. When the album row
# has no year, the rebuilt path drops the $year and the copied file lands in a NEW, yearless
# directory instead of the album's existing folder. Recover the year from a sibling track so
# the import reuses the same directory.
if not album_data.get("year"):
recovered_year = _existing_album_year_from_sibling(
database, album_id, deps.resolve_library_file_path_fn,
int(expected.get("disc_number") or 1), int(expected.get("track_number") or 1),
)
if recovered_year:
album_data["year"] = recovered_year
logger.info("[I Have This] recovered album year %s from existing folder for album %s",
recovered_year, album_id)
source_path = deps.resolve_library_file_path_fn(source_track.get("file_path")) source_path = deps.resolve_library_file_path_fn(source_track.get("file_path"))
if not source_path: if not source_path:
raise MissingTrackImportError(_file_not_found_message(source_track.get("file_path")), 404) raise MissingTrackImportError(_file_not_found_message(source_track.get("file_path")), 404)
@ -441,50 +426,6 @@ def _sync_imported_track(deps: MissingTrackImportDeps, track_id, expected_title:
logger.debug("Existing-track import server sync skipped/failed: %s", sync_err) logger.debug("Existing-track import server sync skipped/failed: %s", sync_err)
def _existing_album_year_from_sibling(
database,
album_id: str,
resolve_library_file_path_fn: Callable[[Optional[str]], Optional[str]],
target_disc: int,
target_track: int,
) -> Optional[str]:
"""Find the release year already baked into this album's on-disk folder (#917).
Read from a sibling track its own ``year`` column first, else a ``(YYYY)`` /
``[YYYY]`` in the album folder name so an "I have this" import reuses the album's
existing directory instead of rebuilding a yearless one. Returns the 4-digit year
string, or None when no signal exists.
"""
try:
with database._get_connection() as conn:
cursor = conn.cursor()
cursor.execute(
"""
SELECT file_path, year FROM tracks
WHERE album_id = ?
AND file_path IS NOT NULL AND file_path != ''
AND NOT (COALESCE(disc_number, 1) = ? AND track_number = ?)
ORDER BY COALESCE(disc_number, 1), track_number
LIMIT 12
""",
(album_id, target_disc, target_track),
)
rows = cursor.fetchall()
for row in rows:
year = row["year"]
if year is not None and str(year).strip()[:4].isdigit():
return str(year).strip()[:4]
resolved = resolve_library_file_path_fn(row["file_path"])
if resolved:
folder = os.path.basename(os.path.dirname(resolved))
match = re.search(r"[(\[](\d{4})[)\]]", folder) # "Album (2019)" / "Album [2019]"
if match:
return match.group(1)
except Exception as exc:
logger.debug("Could not recover album year from sibling for %s: %s", album_id, exc)
return None
def copy_album_identity_from_target_sibling( def copy_album_identity_from_target_sibling(
database, database,
album_id: str, album_id: str,

View file

@ -138,25 +138,10 @@ def _collect_base_dirs(
except Exception as e: except Exception as e:
logger.debug("music paths read failed: %s", e) logger.debug("music paths read failed: %s", e)
# Normalize to absolute forms so resolution does NOT depend on the calling
# thread's CWD. A relative config like "./Transfer" otherwise only resolves
# when os.path.isdir("./Transfer") happens to be true from the current CWD —
# which fails in background workers whose CWD isn't the app root, leaving
# base_dirs empty and every track "unresolved". For each candidate we try
# the raw form first (cheap, preserves an already-absolute path), then its
# os.path.abspath() form so "./Transfer" → "/app/Transfer".
expanded: list[str] = []
for c in candidates:
if not c:
continue
expanded.append(c)
if not os.path.isabs(c):
expanded.append(os.path.abspath(c))
# De-duplicate while preserving order, drop empties / non-existent dirs. # De-duplicate while preserving order, drop empties / non-existent dirs.
seen: set[str] = set() seen: set[str] = set()
out: list[str] = [] out: list[str] = []
for c in expanded: for c in candidates:
if not c or c in seen: if not c or c in seen:
continue continue
seen.add(c) seen.add(c)
@ -239,23 +224,10 @@ def resolve_library_file_path_with_diagnostic(
if not base_dirs: if not base_dirs:
return None, attempt return None, attempt
# Try progressively shorter path suffixes against each base dir. # Skip index 0 to avoid drive-letter / leading-slash artifacts
# # (e.g. "E:" or "" from a leading "/").
# Start at index 0 so a clean RELATIVE library path is tried in FULL first.
# SoulSync's own library scanner stores paths like
# "Asketa/Another Side/01 - Track.flac" (no leading slash) — index 0 is the
# artist folder and dropping it (the old range(1, ...)) meant the artist
# segment was never joined, so nothing under transfer/ ever resolved and
# every track looked unreadable to the quality scanner.
#
# For ABSOLUTE media-server paths ("/music/Artist/Album/track.flac") index 0
# is the empty leading segment and i=0 yields os.path.join(base, "", ...) ==
# base/Artist/... which simply won't exist and harmlessly falls through to
# i=1 ("music/...") etc. A Windows drive part ("E:") at i=0 likewise just
# fails on POSIX and falls through. So starting at 0 is safe for every form
# and only ADDS the relative-full-path match that was missing.
for base in base_dirs: for base in base_dirs:
for i in range(0, len(path_parts)): for i in range(1, len(path_parts)):
candidate = os.path.join(base, *path_parts[i:]) candidate = os.path.join(base, *path_parts[i:])
if os.path.exists(candidate): if os.path.exists(candidate):
return candidate, attempt return candidate, attempt

View file

@ -16,7 +16,6 @@ from core.runtime_state import (
download_tasks, download_tasks,
tasks_lock, tasks_lock,
) )
from core.metadata.album_tracks import get_album_for_source
from core.metadata.registry import ( from core.metadata.registry import (
get_deezer_client, get_deezer_client,
get_itunes_client, get_itunes_client,
@ -27,27 +26,6 @@ from database.music_database import get_database
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
def _album_data_from_source(full: dict, album_id: str, fallback_name: str) -> dict:
"""Build the redownload `album_data` from a primary-source get_album result (#915).
Mirrors the Spotify branch's album_data shape so iTunes/Deezer redownloads carry the
real release_date / album_type / total_tracks instead of a lean {'name': ...} that
drops the $year and forces a manual reorganize afterwards."""
images = full.get('images') or []
image_url = full.get('image_url') or ''
if not image_url and images and isinstance(images[0], dict):
image_url = images[0].get('url', '')
return {
'id': str(full.get('id') or album_id),
'name': full.get('name') or fallback_name,
'release_date': full.get('release_date', ''),
'album_type': full.get('album_type', 'album'),
'total_tracks': full.get('total_tracks', 0),
'images': images,
'image_url': image_url,
}
def _get_itunes_client(): def _get_itunes_client():
"""Mirror of web_server._get_itunes_client — delegates to registry.""" """Mirror of web_server._get_itunes_client — delegates to registry."""
return get_itunes_client() return get_itunes_client()
@ -163,26 +141,12 @@ def redownload_start(track_id):
'images': album_images, 'images': album_images,
'image_url': album_images[0]['url'] if album_images else '', 'image_url': album_images[0]['url'] if album_images else '',
} }
elif meta_source in ('itunes', 'deezer'): elif meta_source == 'itunes':
# #915: parity with the Spotify branch + Reorganize — pull the full album from track_number = full_track_details.get('trackNumber')
# the primary source so album_data carries release_date/album_type/total_tracks disc_number = full_track_details.get('discNumber', 1)
# (was lean {'name': ...}, which dropped the $year on iTunes/Deezer redownloads). elif meta_source == 'deezer':
if meta_source == 'itunes': track_number = full_track_details.get('track_position')
track_number = full_track_details.get('trackNumber') disc_number = full_track_details.get('disk_number', 1)
disc_number = full_track_details.get('discNumber', 1)
_alb_id = full_track_details.get('collectionId')
else:
track_number = full_track_details.get('track_position')
disc_number = full_track_details.get('disk_number', 1)
_alb_id = (full_track_details.get('album') or {}).get('id')
if _alb_id:
try:
_full_album = get_album_for_source(meta_source, str(_alb_id))
except Exception as _alb_err: # noqa: BLE001 — never let metadata break redownload
logger.debug("[Redownload] %s album fetch failed: %s", meta_source, _alb_err)
_full_album = None
if isinstance(_full_album, dict):
album_data = _album_data_from_source(_full_album, str(_alb_id), metadata.get('album', ''))
track_data = { track_data = {
'id': meta_id, 'id': meta_id,

View file

@ -1,53 +0,0 @@
"""What counts as a *residual* file — a leftover with no value once the audio it
accompanied is gone: OS junk, cover/scan images, and lyric/metadata sidecars.
Single source of truth shared by:
* the **Reorganize** cleanup, which strips these from a source dir after every
track has moved out (so the empty-dir pruner can take the folder), and
* the **Empty Folder Cleaner** job, which can optionally treat a folder holding
ONLY residual files as removable (#891).
Defining "disposable" in one place keeps the two features agreeing on what a "dead
folder" is. Pure predicates — no filesystem access — so they're unit-tested in
isolation. The whitelist is deliberately conservative: anything NOT recognized here
(a booklet ``.pdf``, a video, a ``.txt`` note) is treated as real content and kept.
"""
from __future__ import annotations
import os
# OS / tooling junk.
JUNK_FILES = {'.ds_store', 'thumbs.db', 'desktop.ini', '.directory', 'album.nfo~'}
# Cover art + booklet scans.
IMAGE_EXTS = {'.jpg', '.jpeg', '.png', '.gif', '.bmp', '.webp', '.tiff', '.tif'}
# Lyric / metadata / playlist sidecars that are worthless without their audio.
SIDECAR_EXTS = {'.lrc', '.nfo', '.cue', '.m3u', '.m3u8'}
def _ext(name: str) -> str:
return os.path.splitext(name or '')[1].lower()
def is_junk(name: str) -> bool:
return (name or '').lower() in JUNK_FILES
def is_image(name: str) -> bool:
return _ext(name) in IMAGE_EXTS
def is_sidecar(name: str) -> bool:
return _ext(name) in SIDECAR_EXTS
def is_disposable(name: str) -> bool:
"""True if this file is junk, a cover/scan image, or a lyric/metadata sidecar —
i.e. safe to delete from a folder that has no audio left."""
return is_junk(name) or is_image(name) or is_sidecar(name)
__all__ = [
'JUNK_FILES', 'IMAGE_EXTS', 'SIDECAR_EXTS',
'is_junk', 'is_image', 'is_sidecar', 'is_disposable',
]

View file

@ -1,104 +0,0 @@
"""Decision logic for the SoulSync standalone Deep Scan's untracked → Staging move.
The standalone deep scan (``_run_soulsync_deep_scan`` in web_server) walks the
Transfer folder, diffs it against the ``soulsync`` rows in the DB, and relocates
every file it can't find a DB record for into Staging for auto-import. That's fine
when Transfer is a scratch/landing area files arrive, get moved, imported, and
recorded, so a later scan only ever sees a few genuinely-new arrivals.
It is a DATA-LOSS trap when the DB is empty or out of sync with disk (a volume
swap, a DB reset, external tag edits) while Transfer holds the user's real library:
a path-only diff then flags the *entire* library as "untracked" and the scan
relocates all of it (issue #904). The same failure mode the orphan detector and the
media-server deep scan already guard against (``core.library.stale_guard``) this
path just never used the guard.
This module is the pure, testable decision: given the Transfer file set, the DB's
known paths, and the user's "Transfer is my permanent library" preference, decide
WHICH files are untracked and WHETHER it's safe to relocate them. The web layer does
only the I/O (walk/move/delete) based on the returned plan.
"""
from __future__ import annotations
from typing import Iterable, Set
from core.library.stale_guard import (
DEFAULT_MAX_ORPHAN_FRACTION,
DEFAULT_MIN_ORPHANS,
is_implausible_orphan_flood,
)
# Block reason codes (web layer turns these into a user-facing warning).
BLOCK_NONE = ""
BLOCK_TRANSFER_PERMANENT = "transfer_permanent"
BLOCK_DESYNC = "desync"
def _norm(path: str) -> str:
"""Normalize a path for cross-platform comparison (Windows vs Unix separators)."""
return str(path).replace("\\", "/")
def diff_untracked(transfer_files: Iterable[str], db_paths: Iterable[str]) -> Set[str]:
"""Files present in ``transfer_files`` but with no matching ``db_paths`` record.
Comparison is separator-normalized, so a DB path stored with one separator style
still matches the on-disk path. Pure no I/O. Returns the original (un-normalized)
transfer paths so the caller can act on the real filesystem entries.
"""
db_norm = {_norm(p) for p in db_paths if p}
return {f for f in transfer_files if _norm(f) not in db_norm}
def plan_standalone_deep_scan(
transfer_files: Iterable[str],
db_paths: Iterable[str],
*,
never_move: bool = False,
min_untracked: int = DEFAULT_MIN_ORPHANS,
max_fraction: float = DEFAULT_MAX_ORPHAN_FRACTION,
) -> dict:
"""Plan the untracked → Staging move for a standalone deep scan. Pure — no I/O.
Returns a dict:
* ``untracked`` (set[str]) Transfer files with no DB record.
* ``move_blocked`` (bool) True when the untracked files must NOT be relocated.
* ``block_reason`` (str) ``BLOCK_TRANSFER_PERMANENT`` / ``BLOCK_DESYNC`` / "".
The move is blocked when either:
* ``never_move`` is set (the user marked Transfer as their permanent library), or
* the untracked share is implausibly large (> ``min_untracked`` files AND
> ``max_fraction`` of the folder) the empty/desynced-DB signature, where a
path-only diff would relocate the whole library. Below that floor a normal
batch of new arrivals still moves as before.
``move_blocked`` is only ever True when there ARE untracked files; an empty scan
or a clean library returns ``move_blocked=False`` with no reason.
"""
transfer_set = set(transfer_files) # concrete (handles generators) + dedups
untracked = diff_untracked(transfer_set, db_paths)
total = len(transfer_set)
n_untracked = len(untracked)
if n_untracked == 0:
return {"untracked": untracked, "move_blocked": False, "block_reason": BLOCK_NONE}
if never_move:
return {"untracked": untracked, "move_blocked": True, "block_reason": BLOCK_TRANSFER_PERMANENT}
if is_implausible_orphan_flood(
n_untracked, total, min_orphans=min_untracked, max_fraction=max_fraction
):
return {"untracked": untracked, "move_blocked": True, "block_reason": BLOCK_DESYNC}
return {"untracked": untracked, "move_blocked": False, "block_reason": BLOCK_NONE}
__all__ = [
"diff_untracked",
"plan_standalone_deep_scan",
"BLOCK_NONE",
"BLOCK_TRANSFER_PERMANENT",
"BLOCK_DESYNC",
]

View file

@ -26,16 +26,14 @@ without a source ID are reported back to the caller and skipped
entirely. entirely.
""" """
import errno
import os import os
import re
import shutil import shutil
import threading import threading
import time import time
import uuid import uuid
from concurrent.futures import FIRST_COMPLETED, ThreadPoolExecutor, wait from concurrent.futures import FIRST_COMPLETED, ThreadPoolExecutor, wait
from dataclasses import dataclass, field from dataclasses import dataclass, field
from typing import Any, Callable, Dict, List, Optional, Set, Tuple from typing import Any, Callable, Dict, List, Optional, Set
# Per-album track concurrency. Matches the download workers' per-batch # Per-album track concurrency. Matches the download workers' per-batch
# concurrency (3) so reorganize feels comparable to a fresh download. # concurrency (3) so reorganize feels comparable to a fresh download.
@ -101,7 +99,6 @@ _ALBUM_ID_COLUMNS = {
'deezer': 'deezer_id', 'deezer': 'deezer_id',
'discogs': 'discogs_id', 'discogs': 'discogs_id',
'hydrabase': 'soul_id', 'hydrabase': 'soul_id',
'musicbrainz': 'musicbrainz_release_id',
} }
# Human-facing label for each source. # Human-facing label for each source.
@ -410,18 +407,6 @@ def _differentiators_in(norm_title: str) -> frozenset:
return frozenset(t for t in norm_title.split() if t in _VERSION_DIFFERENTIATORS) return frozenset(t for t in norm_title.split() if t in _VERSION_DIFFERENTIATORS)
# Featured-artist credit: "(feat. X)" / "[ft X]" / a trailing "feat. X". The
# parenthesised form is stripped wherever it appears; the bare form only when
# something follows it (so a song literally named "The Feat" is left alone, and
# "Defeat"/"Lift" never trip the word-boundary). Case-insensitive.
_FEAT_RE = re.compile(
r"""\s*[\(\[]\s*(?:feat|ft|featuring)\b\.?[^)\]]*[\)\]] # (feat. X) / [ft. X]
| \s+(?:feat|ft|featuring)\b\.?\s+\S.*$ # trailing feat. X ...
""",
re.IGNORECASE | re.VERBOSE,
)
def _normalize_title(value) -> str: def _normalize_title(value) -> str:
"""Lowercase + strip cosmetic punctuation and treat brackets / dashes """Lowercase + strip cosmetic punctuation and treat brackets / dashes
/ slashes as word separators so the same track named slightly / slashes as word separators so the same track named slightly
@ -433,17 +418,10 @@ def _normalize_title(value) -> str:
- ``Don't Stop Believin'`` ``Dont Stop Believin`` - ``Don't Stop Believin'`` ``Dont Stop Believin``
- ``Swimming Pools (Drank) - Extended Version`` - ``Swimming Pools (Drank) - Extended Version``
``Swimming Pools (Drank) (Extended Version)`` ``Swimming Pools (Drank) (Extended Version)``
- ``The Chase (feat. Big Artist)`` ``The Chase`` (#914)
""" """
if value is None: if value is None:
return '' return ''
out = str(value).strip() out = str(value).strip().lower()
# #914: drop featured-artist credits FIRST (while the parens are still here to
# bound the group). iTunes appends "(feat. X)" to track titles while a user's
# file is often just "The Chase" — the credit is metadata, not the song's
# identity, and leaving it in dropped the match ratio below the threshold so
# correctly-identified tracks reported as "not in the tracklist".
out = _FEAT_RE.sub('', out).lower()
# Strip characters that don't carry meaning across providers. # Strip characters that don't carry meaning across providers.
for ch in ('"', "'", '', '', '', '', '.', ',', '!', '?', for ch in ('"', "'", '', '', '', '', '.', ',', '!', '?',
'(', ')', '[', ']', '{', '}'): '(', ')', '[', ']', '{', '}'):
@ -1102,12 +1080,6 @@ def preview_album_reorganize(
'track_number': track.get('track_number', 0), 'track_number': track.get('track_number', 0),
'current_path': _trim_to_transfer(db_path, resolved, transfer_dir), 'current_path': _trim_to_transfer(db_path, resolved, transfer_dir),
'new_path': '', 'new_path': '',
# Absolute on-disk paths (additive). `current_path`/`new_path` above are
# display-trimmed; these carry the real paths so the rename-only executor
# acts on EXACTLY what the preview computed — no separate path logic that
# could drift from what the user saw (#875).
'current_path_abs': resolved or '',
'new_path_abs': '',
'file_exists': resolved is not None, 'file_exists': resolved is not None,
'unchanged': False, 'unchanged': False,
'collision': False, 'collision': False,
@ -1155,7 +1127,6 @@ def preview_album_reorganize(
new_full, _ok = build_final_path_fn( new_full, _ok = build_final_path_fn(
context, spotify_artist, album_info, file_ext, create_dirs=False context, spotify_artist, album_info, file_ext, create_dirs=False
) )
item['new_path_abs'] = new_full or ''
item['new_path'] = ( item['new_path'] = (
os.path.relpath(new_full, transfer_dir) os.path.relpath(new_full, transfer_dir)
if transfer_dir and new_full and new_full.startswith(transfer_dir) if transfer_dir and new_full and new_full.startswith(transfer_dir)
@ -1815,150 +1786,6 @@ def reorganize_album(
return summary return summary
def _rename_track_in_place(current_abs: str, new_abs: str) -> Tuple[bool, Optional[str]]:
"""Move ONE file from ``current_abs`` to ``new_abs`` in place — no copy, no re-tag,
no post-processing. Creates the destination folder, carries sibling-format files
(e.g. a lossy ``.opus`` alongside the ``.flac``) along with the renamed stem, and
falls back to a cross-device move when the rename crosses a filesystem boundary.
Refuses to overwrite a DIFFERENT existing file at the destination (returns an error
instead) never silent data loss. Returns ``(ok, error_message)``.
"""
try:
if current_abs and not os.path.exists(current_abs):
return False, 'source file no longer on disk'
same = os.path.normpath(current_abs) == os.path.normpath(new_abs)
if os.path.exists(new_abs) and not same:
return False, 'destination already exists'
os.makedirs(os.path.dirname(new_abs), exist_ok=True)
# Carry sibling-format audio to the same destination with the renamed stem —
# mirrors _finalize_track so lossy-copy pairs don't get orphaned.
for sibling_src in _find_sibling_audio_files(current_abs):
_move_sibling_to_destination(sibling_src, new_abs)
try:
os.rename(current_abs, new_abs)
except OSError as e:
if getattr(e, 'errno', None) == errno.EXDEV:
shutil.move(current_abs, new_abs) # crosses a filesystem boundary
else:
raise
return True, None
except Exception as e:
return False, str(e)
def reorganize_album_rename_only(
*,
album_id: str,
db,
transfer_dir: str,
resolve_file_path_fn: Callable[[Optional[str]], Optional[str]],
build_final_path_fn: Callable,
update_track_path_fn: Optional[Callable[[object, str], None]] = None,
cleanup_empty_dir_fn: Optional[Callable[[str], None]] = None,
on_progress: Optional[Callable[[dict], None]] = None,
primary_source: Optional[str] = None,
strict_source: bool = False,
metadata_source: str = 'api',
stop_check: Optional[Callable[[], bool]] = None,
preview_fn: Optional[Callable] = None,
) -> dict:
"""RENAME-ONLY reorganize (#875): move each track's file to the path the current
naming scheme dictates, and nothing else no copy-to-staging, no re-tag, no
quality/AcoustID checks.
It acts on EXACTLY what :func:`preview_album_reorganize` computed (injected via
``preview_fn`` for testability), so the apply can never disagree with what the user
saw, and ONLY files whose path actually changes are touched files marked
``unchanged`` are skipped, which is what keeps a rename from rewriting the whole
album (the #875 complaint). Tags and audio are left byte-for-byte alone.
Returns the same summary shape as :func:`reorganize_album`.
"""
preview_fn = preview_fn or preview_album_reorganize
summary = {
'status': 'completed', 'source': None, 'total': 0,
'moved': 0, 'skipped': 0, 'failed': 0, 'errors': [],
}
def _emit(**updates):
if on_progress is None:
return
try:
on_progress(updates)
except Exception as e:
logger.debug("[Reorganize/rename] progress emit failed: %s", e)
preview = preview_fn(
album_id=album_id, db=db, transfer_dir=transfer_dir,
resolve_file_path_fn=resolve_file_path_fn,
build_final_path_fn=build_final_path_fn,
primary_source=primary_source, strict_source=strict_source,
metadata_source=metadata_source,
)
summary['source'] = preview.get('source')
if not preview.get('success'):
summary['status'] = preview.get('status', 'error')
return summary
tracks = preview.get('tracks', [])
summary['total'] = len(tracks)
src_dirs_touched: Set[str] = set()
for t in tracks:
if stop_check and stop_check():
break
title = t.get('title', 'Unknown')
_emit(current_track=title)
# Skip anything that isn't a real, changing move. `unchanged` is the key one —
# it's why a rename no longer rewrites files whose name didn't change.
if (not t.get('matched') or t.get('unchanged')
or t.get('collision') or not t.get('new_path_abs')):
summary['skipped'] += 1
_emit(skipped=summary['skipped'])
continue
current_abs = t.get('current_path_abs')
new_abs = t.get('new_path_abs')
ok, err = _rename_track_in_place(current_abs, new_abs)
if not ok:
summary['failed'] += 1
summary['errors'].append({
'track_id': t.get('track_id'), 'title': title,
'error': err or 'rename failed',
})
_emit(failed=summary['failed'], errors=list(summary['errors']))
continue
# File is at its new home — update the DB directly (authoritative; no need to
# round-trip through a server scan to learn what we just did). On DB failure the
# file still moved; a library scan reconciles it, so we don't fail the track.
if update_track_path_fn:
try:
update_track_path_fn(t.get('track_id'), new_abs)
except Exception as db_err:
logger.warning(
"[Reorganize/rename] DB path update failed for %s: %s "
"(file moved to %s; a scan will reconcile)",
t.get('track_id'), db_err, new_abs,
)
if current_abs:
src_dirs_touched.add(os.path.dirname(current_abs))
summary['moved'] += 1
_emit(moved=summary['moved'],
processed=summary['moved'] + summary['skipped'] + summary['failed'])
if cleanup_empty_dir_fn:
for src_dir in src_dirs_touched:
try:
cleanup_empty_dir_fn(src_dir)
except Exception as e:
logger.debug("[Reorganize/rename] cleanup of %s failed: %s", src_dir, e)
return summary
def _find_artist_dir(dest_path: str, transfer_dir: str) -> Optional[str]: def _find_artist_dir(dest_path: str, transfer_dir: str) -> Optional[str]:
"""Walk up from ``dest_path`` until the parent equals ``transfer_dir``; """Walk up from ``dest_path`` until the parent equals ``transfer_dir``;
the directory at that point is the artist folder. Returns None if the directory at that point is the artist folder. Returns None if
@ -2020,8 +1847,14 @@ def _prune_empty_album_dirs(artist_dir: str) -> None:
# Sidecars that live alongside ONE audio file (same filename stem). # Sidecars that live alongside ONE audio file (same filename stem).
_TRACK_SIDECAR_EXTS = ('.lrc', '.nfo', '.txt', '.cue', '.json') _TRACK_SIDECAR_EXTS = ('.lrc', '.nfo', '.txt', '.cue', '.json')
# Album-level leftovers (cover images, .lrc, etc.) are classified by the shared # Sidecars that live at the ALBUM level (one per directory).
# `core.library.residual_files.is_disposable` predicate — see `_delete_album_sidecars`. _ALBUM_SIDECARS = (
'cover.jpg', 'cover.jpeg', 'cover.png',
'folder.jpg', 'folder.png',
'front.jpg', 'front.png',
'album.jpg', 'album.png',
'artwork.jpg', 'artwork.png',
)
# Audio extensions used to decide whether a source directory still has # Audio extensions used to decide whether a source directory still has
# tracks the user might care about (i.e. a per-track failure left audio # tracks the user might care about (i.e. a per-track failure left audio
@ -2121,30 +1954,16 @@ def _delete_track_sidecars(audio_path: str) -> None:
def _delete_album_sidecars(src_dir: str) -> None: def _delete_album_sidecars(src_dir: str) -> None:
"""Delete album-level *residual* files from ``src_dir`` — any cover/scan image, """Delete album-level sidecars (cover.jpg, folder.jpg, etc.) from
lyric/metadata sidecar (.lrc/.nfo/.cue/.m3u), or OS junk. Called during `src_dir`. Used during end-of-run cleanup when no audio files remain
end-of-run cleanup ONLY when no audio remains in the directory, so everything in the directory. Best-effort individual failures are debug-logged."""
here is leftover from the album that just moved (#891 — previously this only for name in _ALBUM_SIDECARS:
removed a fixed list of cover names, so ``back.jpg`` / ``disc.jpg`` / ``.webp`` sidecar = os.path.join(src_dir, name)
survived and kept the folder un-prunable). if os.path.isfile(sidecar):
Uses the shared ``is_disposable`` predicate so it agrees with the Empty Folder
Cleaner on what's a dead leftover; anything unrecognized (a booklet ``.pdf``, a
video) is deliberately LEFT. Best-effort individual failures are debug-logged."""
from core.library.residual_files import is_disposable
try:
entries = os.listdir(src_dir)
except OSError:
return
for name in entries:
if not is_disposable(name):
continue
full = os.path.join(src_dir, name)
if os.path.isfile(full):
try: try:
os.remove(full) os.remove(sidecar)
except OSError as e: except OSError as e:
logger.debug(f"[Reorganize] Couldn't remove residual file {full}: {e}") logger.debug(f"[Reorganize] Couldn't remove album sidecar {sidecar}: {e}")
def _has_remaining_audio(directory: str) -> bool: def _has_remaining_audio(directory: str) -> bool:

View file

@ -47,10 +47,7 @@ class LidarrDownloadClient(DownloadSourcePlugin):
if download_path is None: if download_path is None:
download_path = config_manager.get('soulseek.download_path', './downloads') download_path = config_manager.get('soulseek.download_path', './downloads')
self.download_path = Path(download_path) self.download_path = Path(download_path)
try: self.download_path.mkdir(parents=True, exist_ok=True)
self.download_path.mkdir(parents=True, exist_ok=True)
except OSError as e:
logger.warning(f"Could not verify download path {self.download_path}: {e}")
self.active_downloads: Dict[str, Dict[str, Any]] = {} self.active_downloads: Dict[str, Dict[str, Any]] = {}
self._download_lock = threading.Lock() self._download_lock = threading.Lock()

Some files were not shown because too many files have changed in this diff Show more