Commit graph

1476 commits

Author SHA1 Message Date
BoulderBadgeDad
aff4ecccc6 video enrichment: background episode-sync pass (full lists for the whole library)
So library cards show real owned/total (e.g. 8/10) WITHOUT opening each show. When
the TMDB worker's match queue is clear, it pulls the full season/episode list for
one already-matched-but-unsynced show per loop (episode_sync_next), inserting
missing episodes + marking it synced. Over time every library show gets its full
list; the on-view lazy refresh still makes the one you open instant. TMDB-only;
counts toward the worker's pending so it shows busy (not 'Complete') while syncing,
and never loops on a single failing show.

Seam tests: episode_sync_next selection + pending count, worker syncs a pre-feature
matched show to the full list.
2026-06-14 22:39:55 -07:00
BoulderBadgeDad
53391372c3 video: fix detail reload (music router clobber) + reliably show missing episodes
Reload bug: music's router boots first, rewrites an unknown /video-detail/... URL
to /dashboard, and my init read the already-changed URL (no restore) AND dispatched
open-detail before video-detail.js was listening (empty page + stray back button).
Fix: capture the path at SCRIPT-EVAL time (before music boots) and DEFER the
restore to a macrotask so every DOMContentLoaded handler is registered and music's
initial routing has run — then re-assert the real URL. Reload/deep-link now restore
the exact item.

Missing-episodes bug: the full-episode-list cascade only ran via the lazy refresh,
which was gated on ART being missing — so a show that already had posters/logo
never pulled its episode list (stayed owned-only). Added shows.episodes_synced
(schema v5): the worker sets it after a full cascade; show_detail returns it; the
lazy refresh now triggers when NOT synced, so owned + missing episodes populate.
2026-06-14 22:26:37 -07:00
BoulderBadgeDad
c565fec59d video: show the FULL episode list (owned + missing), Sonarr-style
Previously the episodes table held only what the server has (all 'Owned'), so the
detail page never showed what you're missing. Now the metadata provider defines
the full series structure and the server marks ownership:

- TMDB returns the full season list (poster optional) + full episode fields
  (title/air date/runtime/still/rating) per season.
- backfill_episodes UPSERTs: owned episodes keep has_file=1; episodes the server
  lacks are inserted as MISSING (has_file=0); fully-missing seasons get created.
  The cascade now iterates every TMDB season, not just the ones on the server.
- The scan prune only removes SERVER-originated rows (server_id set) that vanished,
  so enrichment-added missing episodes/seasons are never pruned on re-scan.

Season coverage (X / Y) is now meaningful, and the episode list shows Owned +
Missing together. Seam tests: missing-episode insert, fully-missing season,
prune preserves missing.
2026-06-14 22:02:58 -07:00
BoulderBadgeDad
6e526d7745 video detail: deeper TMDB extras — trailer, where-to-watch, more like this
Phase 4: dynamic extras fetched LIVE per view (providers change, so not cached)
via GET /detail/<kind>/<id>/extras → engine.item_extras → TMDB
(videos + watch/providers + similar in one call).
- Trailer: a '▶ Trailer' action that opens an in-app YouTube modal embed (Esc /
  click-away to close).
- Where to Watch: provider logos for the region (JustWatch via TMDB).
- More Like This: a poster row of similar titles linking out to TMDB.
Both movie + show pages; all keyless (same TMDB key).

Seam tests: extras parse (trailer priority, provider/similar shape), item_extras
gating on tmdb_id, route registered, markup hooks. (RT/Metacritic via OMDb needs
its own key — offered separately.)
2026-06-14 21:42:41 -07:00
BoulderBadgeDad
efa0632883 video detail: clearlogo hero (TMDB images — no new key)
Phase 3: the stylized transparent title logo replaces the text title in the
billboard (the big Plex/Netflix 'premium' jump). Sourced from TMDB images
(append_to_response=images, include_image_language=en,null) in the same detail
call — no Fanart key needed.

- schema v4: movies.logo_url / shows.logo_url (idempotent migration).
- TMDB client picks an English logo (then language-neutral, then any); enrichment
  backfills logo_url gap-only; show/movie payloads return 'logo'.
- Billboard shows the logo img (with graceful fallback to the text title on error
  / when absent; title kept visually-hidden for a11y). Lazy on-view refresh now
  also triggers when the logo is missing, so existing libraries fill it in.

Seam tests: English-logo pick, backfill + payload, schema.
2026-06-14 21:36:14 -07:00
BoulderBadgeDad
59c88fa0db video: Movie detail page (Netflix flat layout), movies now clickable
- Movie cards in the library now drill into a movie-detail page (both kinds use
  the same open-detail event / video-side navigation).
- New video-movie-detail subpage reuses the .vd-* hooks; video-detail.js is now
  kind-aware (root() targets the active page by kind, billboard/links/actions
  branch on movie vs show). Flat layout: billboard + a details strip (released /
  runtime / studio / status / critic score / quality) + the shared Cast & Crew row.
- Lazy on-view backfill for movies too: engine.refresh_movie_art re-fetches TMDB
  (cast/genres/backdrop/ratings) when missing, regardless of match status, via
  POST /detail/movie/<id>/refresh-art. movie_match_info added.

Seam tests: movie refresh backfills cast/genres, movie_match_info, route
registered, movie subpage markup, cards clickable for both kinds.
2026-06-14 21:31:51 -07:00
BoulderBadgeDad
b9b3b9eed3 video 'capture everything': cast & crew (people + credits) + cast row UI
Last capture piece, schema v3 (new people + credits tables; CREATE IF NOT EXISTS
migrates existing DBs on restart, no wipe):
- people deduped by tmdb_id; credits link to exactly one movie OR show (separate
  nullable FKs + CHECK, no polymorphic id) with department/job/character/order.
- TMDB client appends credits to the detail call (free) and parses cast (name,
  character, photo, billing order) + headline crew (directors/writers/creators).
- enrichment_apply backfills cast/crew gap-only (never clobbers); show/movie
  detail return cast + crew. Populates on view via the existing lazy refresh-art.
- Cast & Crew section on the detail page: grouped crew line + a horizontal
  cast row with circular TMDB headshots, names, characters (accent hover).

Seam tests: TMDB credit parse (+ job filtering, created_by), backfill + people
dedup across titles, gap-only no-clobber, payload shape.
2026-06-14 21:20:08 -07:00
BoulderBadgeDad
986059626f video: lazy season-art backfill on detail view (fixes matched-show art gap)
Root cause: season posters / episode art backfill happen during a show's TMDB
*match*, but already-matched shows never re-run ('Retry all failed' only resets
not_found/error), so existing libraries never got the art.

Fix (Boulder's idea): fetch-on-view + cache. When a show detail opens and any
season lacks a poster, the page calls POST /detail/show/<id>/refresh-art →
engine.refresh_show_art re-fetches /tv/<id> via the TMDB client and backfills
season posters + episode art gap-only, regardless of match status. Cached, so
it's a one-time cost per show; runs once per view; re-renders when done.

Seam tests: refresh_show_art backfills a MATCHED show's seasons, needs TMDB
configured, show_match_info, route registered.
2026-06-14 20:56:57 -07:00
BoulderBadgeDad
fcd4af0efd video manage-workers modal: Process First Everywhere, search/filter, live glow
Brings the video modal to parity with music's:
- 'Process first everywhere' control (Movies/Shows/Auto) in the topbar — a global
  setting that pins which kind every worker processes first. enrichment_next takes
  a priority kind; the worker reads enrichment_priority each loop; GET/POST
  /api/video/enrichment/priority persists it. Reuses music's .em-global styling.
- Needs-matching bar now has a live count, status filter (All unmatched / Not
  found / Pending) and a debounced search (reusing .em-select / .em-search),
  matching the music modal. Episode view stays read-only.
- Live glow (scoped to #vem-overlay): pulsing running dot, accent glow on the
  selected worker row + active process-first/kind, and a pulsing 'now processing'
  chip in the worker accent. Music's shared .em-* styles untouched.

Seam tests: priority pins kind in enrichment_next + worker honors the setting,
priority endpoint GET/POST + validation, modal feature markup pinned.
2026-06-14 18:28:22 -07:00
BoulderBadgeDad
80f1051e8a video enrichment: cascade episode backfill from the TMDB show worker
Episodes ride along with their show instead of being a separate (tens-of-thousands)
queue: when the TMDB worker matches a show, it now backfills every season's
episodes — still / overview / rating — via /tv/<id>/season/<n> (one call per
season, gap-only so server data is never clobbered). Also backfills season
overviews.

The worker manager 'knows about it': the TMDB breakdown gains an Episodes
coverage entry (matched = has art, rest = pending), shown as its own card; the
Episodes view lists episodes still missing art. It's coverage-only, kept out of
the worker's idle/pending calc so it never blocks 'Complete'.

Seam tests: client season parse, worker cascade fills episodes, gap-only backfill
+ season overview, breakdown coverage (tmdb only), missing-art list, idle calc
ignores episode coverage.
2026-06-14 18:09:24 -07:00
BoulderBadgeDad
878e467f69 video enrichment: backfill season posters from TMDB (server usually lacks them)
The media server rarely has distinct per-season art, so season cards fell back to
a gradient. TMDB's show detail carries a poster_path per season — the show worker
now returns those, and enrichment_apply backfills seasons.poster_url for seasons
the server left without art (gap-only, never clobbers server art). The image
proxy streams a stored full URL (TMDB) directly vs. proxying a server path.

Seam tests: TMDB returns season posters, backfill fills only missing seasons.
2026-06-14 17:57:24 -07:00
BoulderBadgeDad
2306a5740c video enrichment: pull everything TMDB/TVDB offer + backfill gaps only
Enrichment now harvests the full detail payload (same call, no extra requests):
- TMDB: tagline, genres, rating (vote_average), runtime, status, first/last air
  date (shows), release date + runtime (movies) — on top of overview/backdrop/ids.
- TVDB: switches to /series/<id>/extended for overview + genres.

enrichment_apply now uses BACKFILL semantics: metadata columns are written via
COALESCE(NULLIF(col,''), ?) so enrichment only fills fields the media server
left empty — it never clobbers server-provided data. Genres backfill to the
normalised link tables only when the item has none yet. Whitelist expanded for
the new columns.

Seam tests: backfill-only (server overview/genres kept, gaps filled), genre
backfill when empty, TMDB full-metadata extraction.
2026-06-14 17:23:43 -07:00
BoulderBadgeDad
e1e0e29432 video 'capture everything' (phase 1): stills, genres, ratings, tagline
Captures the richer metadata the media server already exposes (schema v2;
idempotent migrations + CREATE IF NOT EXISTS, so an existing DB upgrades on
restart with no wipe):
- movies: tagline, rating (audience), rating_critic; shows: tagline, rating,
  first/last air date; episodes: still_url + rating.
- Genres as a normalised many-to-many (genres + movie_genres/show_genres link
  tables — no comma-blob), deduped, replace-on-upsert.
- Plex (.genres/.tagline/.audienceRating/.rating/.thumb) + Jellyfin (Genres/
  Taglines/CommunityRating/CriticRating/Premiere+EndDate/episode Primary) both
  extract them; episode stills served via /api/video/poster/episode/<id>.
- Detail payloads return genres/tagline/rating/air-dates + per-episode has_still;
  the billboard shows a tagline, ★ score, genre chips, and episode rows render
  REAL stills (no more orange placeholder once scanned).

Seam tests for genre dedup/replace, show+episode capture, episode still ref.
Cast/crew (people + credits) is the next phase.
2026-06-14 17:17:20 -07:00
BoulderBadgeDad
519685fc32 video: rework TV-detail to match the artist-detail vibe + real season art
Addresses the 'feels basic' feedback:
- Hero is now a contained glass card with the backdrop blurred INSIDE it +
  gradient overlay (same treatment as the music artist hero) — no more bare
  gaps around the top/sides. Bigger poster, accent external-link chips
  (IMDb/TMDB/TVDB), refined badges + stat tiles.
- Seasons are a poster-art card grid (season = album) with coverage rings/bars
  and hover-lift, selecting one renders its episodes below (episode = track) —
  episode overviews now shown. Mirrors the artist album-grid -> tracklist.
- Scan now captures real per-season posters (Plex sh.seasons() thumbs / Jellyfin
  /Seasons Primary), served via get_art_ref('season') + /api/video/poster/season.
  Falls back to the show poster until a re-scan populates them.

Seam tests for the season art ref; shell markup tests still green.
2026-06-14 16:13:30 -07:00
BoulderBadgeDad
50632f6392 video enrichment: log each match/not-found at INFO (visible in app.log)
The worker only logged exceptions, so a normal run looked dead — no parity with
the music workers' 'Matched ... -> ID' lines. Now logs each match (noting
'(by server id)' when it used the server's provider id) and each not-found at
INFO. Logger is soulsync.video_enrichment.worker, so it already propagates to
app.log; it just had nothing to say. caplog seam test pins it.
2026-06-14 15:54:53 -07:00
BoulderBadgeDad
a483219746 video enrichment: distinguish transient 'error' from 'not_found' (match music)
A failed lookup CALL (network/429/5xx/timeout, or an expired TVDB token) was
recorded as 'not_found' — permanently logging a transient blip as 'no match'
and parking the item for retry_days. Now mirrors the music workers' proven
pattern:

- New 'error' status, distinct from 'not_found'; enrichment_next retries BOTH
  after retry_days, so errors recover and the queue still advances (no poison
  loop). breakdown/unmatched/retry-all and the modal account for it (shown with
  the outstanding/pending bucket).
- TMDB/TVDB clients raise on non-200 (429/5xx) so the worker records 'error',
  not a false not_found.
- TVDB re-authenticates once on a 401 (expired token) instead of failing every
  match for the rest of the run.

Seam tests: error!=not_found, error retried after window, 429 raises, TVDB
token refresh, UI accounts for errors.
2026-06-14 15:34:45 -07:00
BoulderBadgeDad
fc68a6e741 video enrichment: enrich by the server's provider id, not a title re-search
The deep scan stores tmdb_id/tvdb_id/imdb_id from Plex/Jellyfin, but the workers
only ever searched by title+year and ignored those ids — re-deriving matches the
server already had exact (wasteful, and a title search can mis-match).

enrichment_next now surfaces the row's known provider id; the worker forwards it
and the TMDB/TVDB clients fetch details BY ID (one call, no /search) when it's
present, falling back to title/year search only for items the server couldn't
identify. Still grabs the overview/backdrop the scan doesn't capture.
2026-06-14 15:03:26 -07:00
BoulderBadgeDad
8b01ada68a video: pause enrichment workers during library scans, resume after
Every scan (incremental / full / deep, both entry points) now steps the
enrichment workers aside to cut DB lock contention — same as music. Mirrors
music's contract exactly: pause ONLY workers that were running (a user's manual
pause is left alone), track which we paused, and resume just those in a finally
so success, cancel, and error all clean up. Auto-pause is transient
(persist=False) so it never leaks into the saved <service>_paused flag.

Hooks are injected by get_video_scanner; the scanner is inert without them
(tests build it directly, no engine spun up). Isolated to the video side.
2026-06-14 13:55:59 -07:00
BoulderBadgeDad
9d2468a50f video enrichment: persist worker pause state across restart
Pause/resume now write <service>_paused to video_settings, and the engine
restores each worker's saved pause when it's (re)built — mirrors music's
<service>_enrichment_paused boot flag. Isolated to video.db; music untouched.
2026-06-14 13:11:41 -07:00
BoulderBadgeDad
2901e4ec4d video settings: per-connection Test button (mirrors music's testConnection)
Each video connection item (TMDB/TVDB) now has a Test button that behaves like
music's: saves the key, hits POST /api/video/enrichment/<svc>/test, and toasts
the result via the shared showToast — isolated (own endpoint, own data-attr
handler, reuses the .test-button CSS).
- Client .test() pings TMDB /configuration and TVDB /login to verify the key.
- Endpoint returns {success,message,error}; unknown service -> 404.
94 tests green; music untouched.
2026-06-14 12:26:15 -07:00
BoulderBadgeDad
65ff84aa6a video enrichment 1d: wire TMDB/TVDB API keys (workers turn on)
- GET/POST /api/video/enrichment/config saves the keys into video_settings;
  POST rebuilds the engine (stop old workers, rebuild clients with new keys) so
  they pick up the change live.
- video-settings.js loads the saved keys into the TMDB/TVDB fields on the
  Connections tab and saves them on change (workers enable once a key is set).
Backend is now end-to-end: key -> client.enabled -> worker matches the library
to TMDB/TVDB and fills ids + metadata. 91 tests green; real DB untouched.
2026-06-14 11:26:09 -07:00
BoulderBadgeDad
c17138bf7d video enrichment 1b: worker engine + TMDB/TVDB clients
Isolated enrichment subsystem mirroring the music worker pattern, on video.db.
- VideoEnrichmentWorker: daemon loop pulling enrichment_next -> client.match ->
  enrichment_apply; pause/resume/stop; get_stats in the same shape the music
  enrichment API returns (enabled/running/paused/idle/current_item/stats/
  progress/breakdown). Client injected -> loop fully unit-tested with a fake.
- TMDB/TVDB clients (thin adapters, keys from video_settings): .enabled +
  match(kind,title,year) -> {id, metadata}; validated live, worker logic tested.
- VideoEnrichmentEngine registry + lazy get_video_enrichment_engine() singleton.
8 tests (match/not_found/error-resilience/stats/disabled/engine/isolation).
2026-06-14 11:21:30 -07:00
BoulderBadgeDad
13e03a624c video scan: capture the provider IDs the server already has (tmdb/imdb/tvdb)
The servers already matched everything to their agents — we were dropping the
IDs. Now we store them:
- Plex: parse item.guids (imdb://, tmdb://, tvdb://); Jellyfin: parse
  item.ProviderIds (added ProviderIds to the requested Fields).
- Stored on movies (tmdb_id, imdb_id), shows (tvdb_id, tmdb_id, imdb_id), and
  episodes (tvdb_id) via the upserts.
- Dropped the over-strict UNIQUE on movies.tmdb_id / shows.tvdb_id (same title
  can legitimately live in two libraries; we dedupe on server_id). Scanner now
  wraps each upsert in try/except so one bad item can't abort a scan.
Tests: guid/ProviderIds parsing + IDs persisted. 38 video-DB/scanner tests green.
2026-06-14 11:04:54 -07:00
BoulderBadgeDad
405e7097e3 video scan: align incremental + deep with music's logic
- Incremental now does smart early-stopping like music: skips already-known
  items and stops after 25 consecutive known (server lists recent first),
  instead of a blind fixed cap. Falls back to a full pass when the library is
  near-empty (<50), matching music's small-DB behavior.
- Deep scan gains music's 50% safety threshold: if removal would wipe >50% of a
  >100-row library, it skips (assumes a partial server response, not a real
  emptying) — prevents catastrophic deletion.
- Full Refresh already matched (re-read all, upsert, no removal).
Added DB helpers (server_ids, table_count). Tests: early-stop skips known,
small-lib fallback, 50% prune safety. 122 tests green.
2026-06-14 07:56:23 -07:00
BoulderBadgeDad
b5d0b76fe6 video Tools: match music Database Updater (stats, cancel, real progress)
The scan tool now behaves like music's, not just looks like it:
- Card matches: help '?' button, 'Last Scan' line, and the Movies/Shows/
  Episodes/Size stats grid (populated from /api/video/dashboard on show + after
  a scan). Same .tool-card-stats markup.
- Real progress bar: scanner fetches item totals up front (Plex section.
  totalSize / Jellyfin TotalRecordCount) and reports a true percent as it
  processes; the bar actually moves (movies → shows) instead of sitting at 100%.
- Cancel: the Scan button toggles to 'Cancel' mid-scan and POSTs
  /api/video/scan/stop; the scanner checks a cancel flag between items and ends
  in a 'cancelled' state. Mirrors music's stop affordance.
Tests: percent reported, cancel stops midway + saves only processed items, stop
route registered, tool-card structure. 117 video/integrity tests green.
2026-06-14 07:35:57 -07:00
BoulderBadgeDad
9974845470 video scan: harden Jellyfin adapter (paging + per-item resilience)
- Paginate Jellyfin movie/series listings (StartIndex/Limit, 500/page) so large
  libraries aren't truncated on full/deep scans; incremental stays a single
  capped page.
- Per-item try/except in iter_movies/iter_shows (matches Plex) so one bad item
  doesn't abort the scan.
- Skip Jellyfin episodes with no IndexNumber (consistency with Plex; avoids
  ep-0 collisions). All three modes now solid for Plex + Jellyfin.
2026-06-14 00:53:07 -07:00
BoulderBadgeDad
5b0b64bf3b video side: library mapping backend (pick Movies/TV library)
The scan no longer blindly grabs every movie/show section — it reads the
libraries you map, like music's 'pick your Music library'.
- GET /api/video/libraries: discover the active server's Movies/TV libraries
  (Plex sections by type / Jellyfin views by CollectionType) + current
  selection. POST: save {movies, tv} per server into video_settings.
- sources.py: _build_source(movies_lib, tv_lib) filters to the mapped library;
  get_active_video_source() (used by the scanner) loads the saved selection;
  list_video_libraries() lists them unfiltered for the UI. Falls back to all
  libraries when nothing is mapped yet.
- VideoDatabase.get/set_library_selection (per-server). 6 tests added; 33 green.
2026-06-14 00:24:01 -07:00
BoulderBadgeDad
061079f0f6 video scan: don't crash on episodes with no episode number
Plex specials/unmatched episodes can have a null index -> getattr(...,0) still
returned None -> 'NOT NULL constraint failed: episodes.episode_number'.
- Plex adapter skips episodes with no index (logged), passes a real number.
- upsert_show_tree defensively skips any episode missing season/episode number,
  so no source can crash a scan. Test added.
2026-06-14 00:13:08 -07:00
BoulderBadgeDad
bc334df719 video scan: fix Plex read-timeout on large libraries
The scan inherited the shared client's 15s interactive timeout and fetched
episodes per-season (one request each), so a big library read-timed-out
mid-scan (port 32400).
- Dedicated Plex connection for scans with a 120s timeout (built from the
  shared config; doesn't touch the interactive client).
- Fetch a show's episodes in ONE show.episodes() call grouped by season,
  instead of seasons()+episodes() per season — far fewer round-trips.
- Per-item try/except in iter_movies/iter_shows so one slow/bad item is skipped
  and logged, never aborting the whole scan.
2026-06-13 23:58:04 -07:00
BoulderBadgeDad
04fb19c80c video scan: three modes (full refresh / incremental / deep)
Mirrors the music model (full_refresh vs smart incremental, plus deep_scan):
- incremental: only recently-added items from the server (Plex addedAt:desc /
  Jellyfin DateCreated, capped); upsert; no prune.
- full: every item; upsert all (refresh metadata + add new); no prune.
- deep: every item; upsert; prune what the server no longer has (empty-scan
  safety preserved).
scanner.request_scan/scan_sync take mode; /api/video/scan/request reads
{mode} from the body (default full); adapters take incremental=. Tests cover
deep-prunes / full-doesn't / empty-deep-safety / incremental-requests-recent.
2026-06-13 23:28:57 -07:00
BoulderBadgeDad
6665ecaa12 video side: library scanner (server = source of truth) + scan API
Reads the active media server and mirrors it into video.db, adapting the music
scan pattern (ask the server, upsert, prune what's gone) — isolated from music.
- core/video/scanner.py: server-agnostic VideoLibraryScanner. Consumes a media
  source (duck-typed) yielding normalized dicts; upserts movies + show trees,
  prunes removed items, reports progress/state. Skips pruning when a scan
  returns nothing (transient-failure safety). Background thread + scan_sync.
- core/video/sources.py: Plex + Jellyfin adapters that REUSE the shared
  connected clients (MediaServerEngine) but own all video-section logic; produce
  normalized dicts. (Validated against a live server by design; scanner itself
  is fully unit-tested with a fake source.)
- api/video/scan.py: POST /api/video/scan/request, GET /api/video/scan/status.
- .gitignore: video_library.db + sidecars (mirrors music); tests inject a
  tmp DB so none is ever created in the repo.
Tests: scan populate/prune/empty-safety/no-source-error, isolation guard
(core/video imports nothing from music), scan routes registered. 101 green.
2026-06-13 23:13:50 -07:00
BoulderBadgeDad
177a4d8d05 #868: disambiguate same-name artists by owned-catalog overlap during enrichment
Enrichment matched artists by NAME ONLY (0.85 gate), so for a common name
('Rone' has ~5 artists) it stored whichever the source ranked first — often the
wrong one, which then drove a wrong/sparse library 'Standard' discography while
'Enhanced' (the real owned albums) showed the full set.

Fix — use the decisive signal the library already has (the albums you OWN):
- worker_utils: pick_artist_by_catalog() + catalog_overlap_score() +
  owned_album_titles()/release_titles(). When 2+ candidates clear the name gate,
  fetch each one's catalog and choose the one overlapping the owned albums; falls
  back to the current best-by-name pick when there's nothing to disambiguate or
  no overlap (so the common single-candidate path makes no extra API calls).
- Wired into Spotify (covers Spotify-Free, same client), iTunes, Deezer (now
  multi-candidate search_artists + get_artist_info store), and MusicBrainz
  (match_artist gains owned_titles; release-groups as the catalog).

Re-match path (#868):
- build_reset_query now also clears the stored source-ID column for artist/album
  item resets — previously a 're-match' only nulled match_status, so the worker's
  existing-id short-circuit re-confirmed the WRONG id and never re-resolved. Tracks
  excluded (ids live in tags, not a column).
- MusicBrainz also self-corrects its 90-day name->mbid cache: match_artist bypasses
  a cached mbid whose catalog has ZERO overlap with the owned albums, so a re-match
  isn't blocked by a stale wrong cache entry.

Tests: shared selector (9), per-worker disambiguation for all 4 sources + MB
backward-compat + MB cache-revalidation (8), reset-clears-id (2). 99 worker/
enrichment tests green.
2026-06-13 14:57:17 -07:00
BoulderBadgeDad
030d9bf9ff Quality Upgrade: best-in-class matching (direct track-ID tier, dedup-skip, duration guard)
Four refinements on top of the tiered matcher:

1. Direct source track-ID tier (new top tier): enrichment writes each source's own
   track ID into the file tags (spotify_track_id/deezer_track_id/itunes_track_id/...).
   If we have the active source's track ID, fetch that exact track by ID via
   get_track_details — zero search. Tiers are now: track-ID -> ISRC -> album->track
   -> artist+title. _read_file_ids reads ISRC + all per-source IDs in one tag read.

2. Skip already-proposed tracks: a re-run loads existing finding entity_ids for the
   job and skips those tracks before any API call (pending stays deduped, dismissed
   stays dismissed) — re-runs are cheap.

3. Wrong-version guard: the fuzzy tiers (album-search + track search) reject a
   candidate whose length differs from ours by >5s (live/edit/remix with same title).
   _load_tracks now selects t.duration; exact tiers (track-ID/ISRC/stored-album-ID)
   skip the guard.

4. Tighter album matching: same-title cuts in an album are disambiguated by closest
   duration when track_number doesn't decide it.

Findings record matched_via = track_id | isrc | album | search. 30 repair tests pass
(added track-ID tier, duration guard, dedup-skip, and unit coverage).
2026-06-13 13:34:48 -07:00
BoulderBadgeDad
777781db6a Quality Upgrade: tiered structured matching (ISRC -> album->track -> artist+title)
Replaces the blind fuzzy search with a smart hierarchy that uses the data we
already have, best identity first:

1. ISRC embedded in the file tags (enriched track) -> exact track.
2. Album -> track: use the album's stored source ID (albums.spotify_album_id /
   itunes_album_id / deezer_id / musicbrainz_release_id / audiodb_id) when the
   ALBUM is enriched (even if the track isn't); else find the album by searching
   'artist album', then locate our track in that album's tracklist by normalized
   title (track_number breaks ties). Pins the exact album context. (artist->album->track)
3. Plain artist+title search with similarity scoring. (artist->track) — loosest.

_load_tracks now returns dict rows (adds track_number + the album source-id
columns). Findings record matched_via = isrc | album | search. All clients
(spotify/deezer/itunes/discogs) expose search_albums + get_album_tracks with a
uniform {'items': [...]} shape, so the album tier is source-agnostic.

26 repair tests pass (added album-tier + _find_track_in_album coverage).
2026-06-13 13:00:16 -07:00
BoulderBadgeDad
3ea5b5181f Quality Upgrade: ISRC-first exact matching using the IDs enrichment already embedded
The job was doing a blind fuzzy search for every low-quality track, ignoring that
enrichment writes each track's ISRC + per-source IDs into the file tags. Now it
reads the file's embedded ISRC and resolves the EXACT track via each source's
'isrc:' search (universal cross-source key), guarded by an ISRC-equality check so
a source that ignores the syntax can't produce a false match — exact track, exact
album context, one call. Falls back to the name/artist fuzzy search only for
un-enriched tracks with no usable ISRC. Findings record matched_via=isrc|search.

4 new seam tests (guard accept/reject, ISRC-preferred-over-fuzzy, fuzzy fallback).
2026-06-13 12:43:46 -07:00
BoulderBadgeDad
b393866782 Remove old auto-acting Quality Scanner tool (replaced by Quality Upgrade Finder job)
Phase 2 of the redesign. The tool that judged quality by extension and auto-dumped
matches into the wishlist is gone; quality scanning is now the reviewed
quality_upgrade repair job.

Removed:
- Frontend: Tools-page Quality Scanner card, its JS handlers/poller/socket listener,
  help tooltip + tour entry (webui index.html, core.js, helper.js, wishlist-tools.js).
- Backend: /api/quality-scanner/{start,status,stop} endpoints, the in-memory state +
  executor + 1s socket broadcast, the QualityScannerDeps/run_quality_scanner shim.
- core/discovery/quality_scanner.py: the auto-acting worker + deps class (the shared
  match/normalize helpers stay — the new job imports them).

Rewired:
- Automation 'start_quality_scan' action now triggers the quality_upgrade repair job
  via repair_worker.run_job_now() (AutomationDeps gains run_repair_job_now, drops the
  4 scanner fields). Action block's vestigial scope field removed (scope lives in the
  job's settings now). NOTE: the 'quality_scan_completed' trigger no longer fires (the
  repair job doesn't emit it).
- Updated all automation test _build_deps helpers + conftest tool-progress harness;
  deleted the obsolete worker test. 528 affected tests pass; 6123 collect cleanly.

QUALITY_TIERS / _get_quality_tier_from_extension kept (used elsewhere).
2026-06-13 12:14:45 -07:00
BoulderBadgeDad
69dd4e1792 Quality Upgrade Finder: new findings-based repair job (replaces auto-acting Quality Scanner)
The old Quality Scanner tool judged quality by file EXTENSION only (a 128k and a
320k MP3 looked identical), ignored the bitrate-based quality profile, used min()
of enabled tiers so the default profile flagged the ENTIRE non-lossless library,
and auto-dumped every match into the wishlist with no review.

This new repair job does it properly:
- meets_preferred_quality(): pure, bitrate-AWARE decision honoring every enabled
  quality bucket (320 MP3 passes a FLAC+320+256 profile; 128 MP3 doesn't). Floor
  is the worst enabled bucket, not the best.
- scans watchlist artists or whole library, finds below-quality tracks, matches a
  better version at scan time (reusing the existing tested match helpers), emits a
  FINDING showing the match + confidence. Off by default; nothing auto-queued.
- _fix_quality_upgrade apply handler adds the matched track WITH album context to
  the wishlist — the user-approved version of what the old tool did silently.
- Transcode/fake-lossless detection intentionally left to the existing Fake
  Lossless Detector job.

12 seam tests incl. a regression pinning the default-profile flooding bug. The old
tool is still in place; removing it + rewiring its automation action is the next step.
2026-06-13 11:51:43 -07:00
BoulderBadgeDad
78f47f04d7 Merge branch 'dev' of https://github.com/Nezreka/SoulSync into dev 2026-06-13 11:16:06 -07:00
BoulderBadgeDad
846a9c75a0 #867: Tidal playlist discovery shows all tracks (was capped to ~21)
Two issues in the same path:
1. The shared discovery modal pre-renders one row per track from a
   separately-fetched frontend track list, then the poll dropped any backend
   result without a pre-rendered row (if (!row) return). When the frontend's
   track fetch came back rate-limited/partial (~21) while discovery's own fetch
   got all 59, the surplus results vanished. Now the modal CREATES a row for any
   result lacking one, so authoritative backend results drive the list (fixes
   all sources sharing the modal).
2. get_playlist hydrated a whole relationships page in one _get_tracks_batch
   call, but Tidal caps filter[id] at 20/request, silently truncating larger
   pages. Chunk to the cap like get_album_tracks already does.

Seam + regression tests (tests/test_tidal_playlist_batch_chunking.py).
2026-06-13 10:39:30 -07:00
BoulderBadgeDad
119c6e3196 Spotify (no-auth): report connected + 'Spotify (no-auth)' test result instead of a Deezer fallback
Status checks asked is_spotify_authenticated() (official OAuth only) instead of
is_spotify_metadata_available(), so a Spotify-Free primary read as disconnected.
get_primary_source_status had spotify_free awareness but it was dead code:
get_client_for_source('spotify') returns None unless officially authed, so the
free-availability probe never had a client. Fetch the client directly for that
check; add the missing free branch to the dashboard test message. Seam + regression tests.
2026-06-13 10:16:23 -07:00
BoulderBadgeDad
992fe7567d
Merge pull request #860 from nick2000713/fix/colon-title-normalization
fix: treat colon as separator in normalize_string so T:T matches T_T
2026-06-13 10:06:49 -07:00
BoulderBadgeDad
41f73f0c38 HiFi: auto-push genuinely-new default instances to existing installs once (so a newly-added working instance reaches everyone, not just Restore-Defaults clickers; removed defaults stay removed) 2026-06-13 09:31:15 -07:00
BoulderBadgeDad
fd7fd32dfa HiFi: add us-west.monochrome.tf to default instances (community-confirmed working, Sokhi) 2026-06-13 09:19:57 -07:00
BoulderBadgeDad
651b904e92 Watchlist: per-artist 'auto-download' toggle (follow-only) — off = discover/surface releases but skip the wishlist add; default on 2026-06-13 08:07:20 -07:00
BoulderBadgeDad
c62074d54a #865: resolve pasted SoundCloud links (incl. unlisted/private share URLs) via direct yt-dlp resolve; manual-search forces the SoundCloud source 2026-06-13 00:41:47 -07:00
BoulderBadgeDad
ba5d62946a Mirrored playlists: custom name alias (overrides display + sync name, survives upstream refresh) — card rename button like the source-ref editor 2026-06-13 00:23:56 -07:00
BoulderBadgeDad
0577dc92e5 #863: add diagnostic logging to YouTube artist-recovery so we can see per-track what it returns 2026-06-13 00:03:35 -07:00
BoulderBadgeDad
c72e83bc2f #863: move YouTube artist recovery out of the (synchronous) parse into the async discovery worker — parse is fast again, no 120s timeout risk 2026-06-12 23:46:29 -07:00
BoulderBadgeDad
4f24c2af6d #863: derive YouTube track artist from music fields / -Topic channel / 'Artist - Title' instead of the playlist owner 2026-06-12 20:14:55 -07:00
BoulderBadgeDad
cb2d920a9e #862: Library Reorganize falls back to tag-mode when an album has no source ID (media-server libs now actually reorganize) 2026-06-12 20:05:50 -07:00