Your log showed 'proxy returned 0' (public proxies dead) → the slow per-video
path. Date those recent uploads in a 3-worker thread pool instead of serially,
so a channel finishes in ~30s rather than 1-2 min; cached as each completes.
Addresses 'says running but I see nothing':
- Real logs: 'enriching <ch>…', 'proxy returned N', 'done — N proxy + N
per-video (X/Y dated)' so you can see exactly what it's doing.
- Proxy timeout 8s→4s so dead instances (most public ones) fail fast instead of
hanging ~40s before the yt-dlp fallback.
- Channel page now re-fetches a few times (25/60/110s) after load while videos
are still undated, re-rendering only when NEW year-seasons appear — so dates
the background enricher fills in pop in without a manual reload.
The enrichment daemon was starting during API tests (it uses the default DB +
network), touching a stray real DB and causing sqlite disk-IO flakiness. enqueue()
now no-ops when PYTEST_CURRENT_TEST is set; _enrich() is still tested directly
with a tmp DB. 176 video tests green.
Two gaps from real use:
- When all proxy instances are down (common), the fallback only dated WISHED
videos, leaving most as 'Earlier videos'. Now on proxy failure it flat-resolves
the channel's recent uploads and dates THOSE via yt-dlp (throttled, cached) —
so years populate fully even with no working proxy.
- A channel followed before the enricher existed was never queued ('nothing to
process'). /youtube/channels now sweeps: any followed channel not enriched
recently gets enqueued — so viewing the Channels tab (or the wishlist badge
refresh) picks up existing follows.
The detail page still reads cache+RSS (fast); the enricher fills the cache. 51
tests green.
- The standalone enricher now reports orb telemetry: stats()/pause()/resume();
/enrichment/youtube/status (+pause/resume) special-cased in the route. Added
the 4th header worker button (red ▶), WORKER_DEFS entry, and SERVICES wiring —
it's not on the socket so video-enrichment.js polls it every 3s. It animates
(idle → active orb + inbound pulses) while a followed channel's dates are being
fetched, like the TMDB/TVDB/OMDb orbs.
- Channel detail now uses the SAME standard watchlist button as shows/movies
(library-artist-watchlist-btn, ✓/＋ icon, 'In Watchlist'/'Watchlist') instead
of a bespoke 'Follow' — consistency. Still wired to the channel follow action.
176 backend tests green; JS balanced; music untouched.
Follow a channel → a background job fetches its FULL upload-date catalog so the
channel page's year-seasons populate fully, cached permanently (one-time per
channel, instant after).
- core/video/youtube.py: proxy_channel_dates() — no-key BULK dates via Piped/
Invidious instances (tries several, paginates); parse_proxy_dates handles both
shapes. The clean path the user wanted.
- core/video/youtube_enrichment.py: YoutubeDateEnricher daemon — enqueue(channel)
→ proxy bulk → cache; per-video yt-dlp only as a throttled fallback (cap 60,
0.4s) for wished videos when every proxy is down. Skips channels enriched in
the last 24h.
- db: youtube_channel_enrichment table + mark/check + wishlisted_video_ids_for_
channel. Enqueued from /youtube/follow and on opening a followed channel.
Scoped to FOLLOWED channels only (per request). 174 tests green; enricher imports
nothing from music.
Flat listing has no upload dates, so channels showed one 'All Videos' season.
Now real year-seasons, filled from cheap sources and cached so they grow:
- core: parse_rss_dates / channel_recent_dates — one public RSS GET dates the
~15 most-recent uploads (no yt-dlp, no bot risk).
- db: youtube_video_dates cache table + cache_video_dates / get_video_dates.
- /youtube/channel merges cached + RSS dates onto the flat videos (year-seasons)
and caches what it learns; /youtube/video caches each fetched date too — so
expanding/wishing videos progressively fills the catalog's years, instant on
repeat. Dateless tail groups as 'Earlier videos'. Missing-only toggle hidden
for channels. 84 db + 84 youtube/api tests green.
Full day-one coverage still needs the YouTube Data API's publishedAt (optional,
yt-dlp can't match it cheaply) — RSS + cache is the no-key best.
- The big one: an empty TMDB result no longer flashes 'No results' while the
(slower) YouTube channel search is still in flight — a 'YouTube channels ·
searching…' skeleton group (shimmer cards + spinner) shows instead, and the
empty state only appears once BOTH sources resolve with nothing. Users no
longer click away thinking nothing's happening.
- Channel avatars: fall back to the singular 'thumbnail' field when the flat
results omit the thumbnails list, so fewer cards are photoless (the rest keep
the clean initials placeholder). Reduced-motion respected.
A text search now also surfaces matching YouTube channels (not just the paste-a-
link path). search_channels() extracts the channels-filtered YouTube results
page (flat); API GET /youtube/search hydrates a 'following' flag. The search
page fires it in parallel with the TMDB search and appends a 'YouTube channels'
group when it returns (best-effort; never blocks/breaks TMDB results). Cards open
the in-app channel page. 82 youtube+api tests green; balanced; music untouched.
Flat channel listing can't return descriptions, so selecting a video showed 'No
description'. Now we do a non-flat single-video extract on demand — the way the
TV nebula lazy-loads guest stars:
- core: video_detail(id) / shape_video() — description, views, likes, duration,
tags, channel, webpage_url (full extract, yt-dlp injectable for tests).
- API: GET /youtube/video/<id> — returns it AND backfills the description onto
the wishlist row (set_wishlist_video_overview), so re-opening is instant.
- Wishlist info bar (youtube): on select, lazy-fetch → real description +
eyebrow (date · duration · views) + a 'Watch on YouTube' link; cached on the
episode object. 'Loading details…' while in flight.
Mirrors the episode lazy-load + art-backfill patterns. 73 youtube+api tests
green; brace/CSS balanced; music untouched.
- Resolver: capture banner_url + separate avatar (by thumbnail id), keeping
per-video duration/views for the detail hero.
- query_youtube_wishlist reshaped to the EXACT TV-nebula shape: channel = show,
upload YEAR = season (newest first), video = episode (newest=ep1 within a
year), carrying source='youtube' + youtube_id + per-video source_id. Surrogate
int returned as tmdb_id so the nebula's int keying just works.
- New API: GET /youtube/channel/<id> (full channel detail — meta + deeper
uploads + following + per-video wished flags) and POST /youtube/wishlist/add
(per-video wish). DB: youtube_video_wish_state, remove_one_video_from_wishlist.
- Tests updated for the nebula shape + banner + detail endpoints. 82 DB + 69
api/youtube tests green.
First slice of 'follow a YouTube channel as a show'. Pure, testable resolver:
- parse_channel_url() normalizes any channel reference (@handle, /channel/UC..,
/c/.., /user/.., bare handle, with/without scheme or tab suffix) to a
canonical /videos uploads URL, and rejects non-channels (watch/playlist/
shorts/home/non-youtube).
- shape_channel() maps yt-dlp's flat-extract dict to our shape (channel meta +
recent uploads), picking best-res thumbs, parsing sparse publish dates
(timestamp or upload_date → ISO, None when flat mode omits them), filtering
null/id-less entries.
- resolve_channel() ties them together via an injectable YoutubeDL factory.
yt-dlp only (already a pinned dep), extract_flat for cheap listing. 28 seam
tests, no network. Schema bridge + API + UI are the next slices.
Provider filter (#4):
- client.discover() + engine.discover_filter() take a TMDB provider id and pass
with_watch_providers + watch_region (engine._region) + flatrate. Browse gets a
streaming-service chip row (Netflix/Prime/Disney+/Max/Apple TV+/Hulu/Paramount+/
Peacock); the grid title reflects 'on <service>'.
Infinite scroll (#6):
- Grid paginates via an IntersectionObserver sentinel (600px lookahead) with a
bottom spinner; the Load more button stays only as a no-IO fallback.
Polish (#7):
- Hero keyboard nav (←/→ when Discover is the visible view, ignoring inputs and
while the trailer is open); focus-visible rings on chips/segments/cards/arrows.
Note: 'complete-the-franchise' rail (#5) needs a collection_id per movie, which
the schema doesn't store yet — deferred (would need an enrichment pass).
Tests: +1 (provider watch_region params); updated the discover_filter fake for
the new kwargs. Enrichment + API suites: 115 passed.
Performance:
- Batched ownership: new db.library_ids_for_tmdb() resolves a whole rail in one
query per kind. _stamp_owned (now also used by search + trending) groups by
kind, so a full Discover page drops from ~500 connections to a couple per rail.
Function/data:
- 'See all' on every rail opens it as a paged grid (Load more); the filter bar's
Browse routes through the same generic category grid with a back button + title.
- Personalized 'Because you like <Genre>' rails seeded from your most-owned
genres (new db.top_owned_genres + /discover/taste endpoint).
- 'Hide owned' toggle drops in-library titles from every rail/grid (CSS class,
instant).
Visual vibes:
- Ambient page-top color bleed that follows the current hero slide's hue.
- Rail edge-fade mask, gentle fade-in on load, per-title hue glow on card hover.
Tests: +4 (batched id map, server scoping, one-query-per-kind stamp, top genres).
Full video enrichment + database suites: 145 passed.
Storage was already per-server (movies/shows UNIQUE(server_source, server_id),
episodes via per-server show_id, prune_missing scoped) — but reads returned
every server's rows, so a Jellyfin scan would show up alongside Plex.
Mirror the music standard: scope reads to the active video server
(resolve_video_server). query_library, calendar_upcoming, dashboard_stats and
library_id_for_tmdb take a server_source; the dashboard/library/calendar
endpoints pass it. server_source=None keeps "all servers" (enrichment processes
every server; tests unchanged). No schema change, no data migration — existing
Plex data is untouched and simply hidden while Jellyfin is the active server.
Regression tests: same title on both servers stays two rows; scoped reads only
return the active server's data; deep-scan prune never touches the other server.
A new isolated Calendar page (/api/video/calendar) — every upcoming episode for
your owned shows across a real 7-day week (today first), as art cards sorted by
air time with a per-cell breathing colour glow.
- Air times: enrich shows with TVDB airsTime (new shows.airs_time column +
migration); cells show + sort by time, streaming (untimed/00:00) = "Anytime".
One-time background backfill re-queues already-matched shows for the time.
- Click an episode → styled modal (show backdrop hero, episode still/synopsis,
air date+time, owned badge, genres, "about the show"), with an explicit
"Open full show page" action instead of navigating on click.
- Isolated: reads only video_library.db, writes nothing to the music side.
Give the video side its OWN server connection — pre-filled from music but stored
separately in video.db, fully isolated (video never writes music config/state).
- Effective config helpers (video_plex_config / video_jellyfin_config): video's
own creds when set, else inherited read-only from music. resolve_video_server +
_build_source + watch-link/poster/dashboard all use these (own db threaded in).
- Server Connection UI mirrors music's server picker (toggle = select + configure),
scoped to Plex/Jellyfin, at the bottom of the Connections tab.
- Jellyfin: independent client built from video's creds; explicit USER picker like
music (list users → that user's libraries); honors the pick, admin fallback.
- Honest connection diagnostics (reachable vs 401 vs no-users) instead of a vague
"auth failed".
- Auto-save on change with toasts; the shared Save button is intercepted on the
video side so it saves video settings (and can't fire a music save).
- Enrichment status now PUSHES over the socket like music (no browser polling /
access-log flood); config save only rebuilds workers when an API key changed.
- Seam tests for effective-config inheritance/override + isolation guard.
Two real coupling bugs, fixed:
- resolve_video_server still fell back to the music active server when no explicit
video pick was set, so changing the music server changed video. Removed: video
now uses ONLY an explicit video pick or the configured server(s) (Plex default
when both). Changing the music server never changes video.
- The shared settings page could persist active_media_server from the video side.
Guarded saveSettings itself (not just the debounced auto-save) so it NEVER runs
while data-side=video — video saves only via /api/video/*.
Test: video does not follow the music active server. One-way isolation, both ways.
The video side now uses a configured Plex/Jellyfin on its own (resolve_video_
server), not config_manager.get_active_media_server(). So a music-only server
(Navidrome/SoulSync) never applies to video, and 'Navidrome for music + Plex for
video' works. Order: explicit video pick (video_server setting) → music-active if
video-capable → the single configured one → Plex if both → None. Seam tests cover
each case incl. the mixed setup.
A saved 'Where-to-watch region' picker in Settings → Detail Pages (19 common
regions, default US). The engine reads it for the providers in extras +
tmdb_detail (region in the cache key), and the detail page labels the section
'Where to Watch · <region>' so you know which market you're seeing.
Click any episode (owned or missing) to expand it: a larger still, full overview,
and the episode's guest stars (clickable to the person page). Lazy-loaded per
episode from TMDB by the show's tmdb_id and cached. New client.episode_detail +
engine.episode_extra + /api/video/episode/<show_tmdb>/<season>/<episode>.
The inline dict cache was a latent bug: no lock (the engine singleton is hit by
concurrent Flask + worker threads) and a wholesale-clear cliff at 256 (nuked hot
entries). Extracted a thread-safe TTL+LRU TTLCache into an importable core/ module
with seam tests (expiry via injected clock, LRU-not-wholesale eviction, a
concurrency stress test). Engine now uses it; search is cached too (60s, ownership
re-stamped fresh). Deliberately NOT persisted to disk — durable data already lives
in video.db; that tier would be over-engineering for a self-hosted app.
The owned-item extras and preview detail payloads were already cached (30-min
TTL); person_detail, tmdb_season, and trending were not — so person pages and
lazy preview seasons re-hit TMDB each view. Now cached too (trending 1h). Library
ownership is re-annotated fresh on each call so 'In Library' badges stay current
while the expensive TMDB fetch is reused. Tests for person + season caching.
extras() now returns a featured review (author, rating, snippet, date); the detail
page shows it in a card with a clamped body + Read more/less. In-app (no external
link).
person() now returns profile images (thumb+full) and also_known_as. The person
page shows an 'Also known as' line and a Photos rail that opens the shared
fullscreen lightbox (arrows/Esc/counter).
One TMDB call (append_to_response) now also returns: image gallery (backdrops +
posters, thumb+full), all YouTube videos (ordered trailer→teaser→clip→…), keyword
tags, facts (budget/revenue/language/country), and the FULL cast (tv via
aggregate_credits with per-actor episode counts). Shared by item_extras (owned)
and full_detail/tmdb_detail (preview).
Caching: the engine memoizes the live TMDB extras + preview payloads (30-min TTL)
so re-opening a title is instant instead of re-hitting TMDB. Tests added.
- Sort dropdown: Newest / Oldest / Most popular.
- Department filter (Acting / Directing / Writing / …) for multi-hyphenates —
only appears when a person has 2+ departments. Composes with the existing
kind + ownership filters; every chip shows a CONTEXTUAL count (what you'd get
if you clicked it, given the other active filters).
- Age in the hero meta ('47 years old', or lifespan + 'aged N' for the deceased).
- Backend: each person credit now carries its department (cast=Acting,
crew=its TMDB department). Seam test added. 249 video-suite tests pass.
'Request limit reached!' is the free-tier daily quota (1,000/day), not a bad key
— so the worker now idles ~30 min and auto-resumes instead of pausing for good.
A library bigger than the daily cap just spreads its ratings across days on its
own. A genuinely invalid/unactivated key still hard-pauses until fixed. Cooldown
reads as paused in the UI (+ a 'cooldown' flag and note). Item is never burned to
synced, so nothing is lost.
OMDb returns a JSON body even on 401, so surface its actual Error: 'Invalid API
key!' (nudges the user to click OMDb's activation email) vs 'Request limit
reached!' (free-tier daily quota). The worker's pause note uses the same message.
The log flood you saw was the OMDb worker hitting a 401 (invalid key) on every
owned title: it logged a full traceback per item AND marked each one
ratings_synced=1 — which would've stopped them ever retrying once the key was
fixed. Root-cause fixes:
- OMDBClient.ratings raises a distinct OMDbAuthError on 401 / 'Invalid API key!'
(vs a transient error vs a genuine no-data 200).
- Worker: on an auth error it PAUSES (transient, not persisted) with a reason
note + one warning, instead of churning the whole library; the item is NOT
marked synced. Transient errors no longer burn items either — they back off and
pause after 3 in a row. Only a genuine 'no data' marks an item synced. Warnings
are concise (no per-item tracebacks). get_stats exposes the pause 'note'.
- Fixing the key auto-recovers: saving a new/changed OMDb key re-queues every
still-unrated title (resets the wrongly-burned ones), and the engine rebuild
un-pauses the worker.
Seam tests: bad-key pause-without-burn, transient keep-item, ratings() raises on
401, key-change re-queues unrated. 227 video-suite tests pass.
The 'Where to Watch' section is now actionable:
- For an OWNED title it leads with a 'Play on Plex/Jellyfin' tile (green, play
glyph) that deep-links straight to the item on your server — Plex via the
app.plex.tv web app (machineIdentifier fetched once + cached), Jellyfin via its
web detail page. Built in engine.item_extras from the row's server_source +
server_id and the shared media-server config (same source poster.py uses).
- Streaming providers (TMDB/JustWatch) are now clickable → the where-to-watch
page, with a hover lift.
Owned-only: preview (tmdb) items have no library row so they get no server tile.
Seam tests cover the Jellyfin + Plex link building and the unowned no-link case.
240 video-suite tests pass.
Smart back (mirrors music's artist-detail): the top-left back button now
remembers where you actually came from, many layers deep. It keeps an origin
stack ({page} or {detail title}) and stamps each history entry with its layer
depth, so:
- the label is dynamic — '← Back to Search' / '← Back to The Bear' / '← Back to
<person>' — instead of a hardcoded 'Library'/'Back';
- backing out of the first layer returns to the page you started from (Search,
Watchlist, wherever), not always the Library;
- browser Back and our button both unwind the chain one layer at a time, in sync.
Fixes: search → person → back → movie used to mislabel as 'Library' and dump you
in the library.
Next level:
- Search isn't a blank box when idle — a 'Trending this week' rail (TMDB
trending, owned/preview annotated). Returns when you clear the query.
- Person page gets a 'Known For' hero rail (top titles by popularity) above a
full filmography now sorted chronologically (newest first).
Backend: TMDBClient.trending + engine.trending (+library annotation), route
/api/video/trending. Isolated; 237 video-suite tests pass.
Search any movie / show / person (TMDB multi-search) entirely in-app. Results
that you already own link straight to the library detail; the rest open a
TMDB-backed 'preview' detail that reuses the exact same Netflix billboard UI
(direct image URLs, nothing owned/enriched). Everything resolves back into
SoulSync — no external links on un-owned titles.
- Search page (video-search.js): debounced /api/video/search, grouped
movies/shows/people cards (reuses .library-artist-card) with owned/preview
ribbons. People open the person page.
- Source-agnostic detail (video-detail.js): loads from /api/video/detail
(library) or /api/video/tmdb (preview); art helpers pick proxy vs direct URLs;
tmdb shows lazy-load episodes per season; owned-via-tmdb-url auto-redirects to
the library detail.
- 'More Like This' now drills in-app (tmdb detail, redirects if owned); cast/crew
link to a new in-app person page (bio + filmography, each credit owned/preview).
Library credits now carry tmdb_id so owned-item cast is clickable too.
- Backend: TMDBClient.search/full_detail/person (+ shared _parse_extras);
engine.search/tmdb_detail/tmdb_season/person_detail; db.library_id_for_tmdb;
routes /search, /tmdb/<kind>/<id>, /tmdb/show/<id>/season/<n>, /person/<id>.
Isolated (one-way): video-only files, no music imports, music shell untouched.
Seam tests: search/full_detail parsing, tmdb_detail assemble+redirect, search +
person library annotation, library_id_for_tmdb, route registration, shell/JS
isolation. 234 video-suite tests pass.
OMDb now has the same setup as TMDB/TVDB: a yellow dashboard orb (★ glyph) that
spins/idles in the worker-orb animation, an entry in Manage Workers (Ratings
coverage cards, pause/resume, retry, search), and a BACKGROUND ratings pass.
- Worker 'ratings mode' (is_ratings): instead of a match queue it pulls
ratings_next() (library items with an imdb_id and ratings_synced=0), fetches
IMDb/RT/Metacritic, applies + marks synced. So the whole library gets ratings,
not just titles you open (schema v7: ratings_synced).
- enrichment_breakdown/unmatched/retry get an 'omdb' branch (coverage =
ratings-filled, not matched). build_clients includes omdb; the lazy on-view
backfill uses the omdb worker's client.
- Dashboard orb + Manage Workers entry (★ glyph fallback where there's no logo),
yellow accent.
Seam tests: omdb worker rates the queue (ratings mode), ratings breakdown.
Next-level: real critic/audience scores beyond the TMDB star. OMDb (free key,
keyed by the imdb_id we already capture) returns IMDb / RT / Metacritic.
- OMDBClient (ratings + test); built as a non-worker 'ratings_client' on the
engine. _backfill_ratings runs in both lazy detail refreshes (overwrites, since
ratings are dynamic). schema v6: imdb_rating / rt_rating / metacritic on
movies + shows; show/movie payloads return them.
- Billboard renders branded rating badges (IMDb yellow, RT tomato/splat by
fresh/rotten, Metacritic green/yellow/red by score). Lazy refresh also triggers
when an imdb_id exists but ratings are missing.
- OMDb API-key frame in Settings (parity with TMDB/TVDB) + config GET/POST +
/enrichment/omdb/test.
Seam tests: OMDb parse, engine ratings backfill, apply_ratings + payload, config
includes omdb.
So library cards show real owned/total (e.g. 8/10) WITHOUT opening each show. When
the TMDB worker's match queue is clear, it pulls the full season/episode list for
one already-matched-but-unsynced show per loop (episode_sync_next), inserting
missing episodes + marking it synced. Over time every library show gets its full
list; the on-view lazy refresh still makes the one you open instant. TMDB-only;
counts toward the worker's pending so it shows busy (not 'Complete') while syncing,
and never loops on a single failing show.
Seam tests: episode_sync_next selection + pending count, worker syncs a pre-feature
matched show to the full list.
Reload bug: music's router boots first, rewrites an unknown /video-detail/... URL
to /dashboard, and my init read the already-changed URL (no restore) AND dispatched
open-detail before video-detail.js was listening (empty page + stray back button).
Fix: capture the path at SCRIPT-EVAL time (before music boots) and DEFER the
restore to a macrotask so every DOMContentLoaded handler is registered and music's
initial routing has run — then re-assert the real URL. Reload/deep-link now restore
the exact item.
Missing-episodes bug: the full-episode-list cascade only ran via the lazy refresh,
which was gated on ART being missing — so a show that already had posters/logo
never pulled its episode list (stayed owned-only). Added shows.episodes_synced
(schema v5): the worker sets it after a full cascade; show_detail returns it; the
lazy refresh now triggers when NOT synced, so owned + missing episodes populate.
Previously the episodes table held only what the server has (all 'Owned'), so the
detail page never showed what you're missing. Now the metadata provider defines
the full series structure and the server marks ownership:
- TMDB returns the full season list (poster optional) + full episode fields
(title/air date/runtime/still/rating) per season.
- backfill_episodes UPSERTs: owned episodes keep has_file=1; episodes the server
lacks are inserted as MISSING (has_file=0); fully-missing seasons get created.
The cascade now iterates every TMDB season, not just the ones on the server.
- The scan prune only removes SERVER-originated rows (server_id set) that vanished,
so enrichment-added missing episodes/seasons are never pruned on re-scan.
Season coverage (X / Y) is now meaningful, and the episode list shows Owned +
Missing together. Seam tests: missing-episode insert, fully-missing season,
prune preserves missing.
Phase 4: dynamic extras fetched LIVE per view (providers change, so not cached)
via GET /detail/<kind>/<id>/extras → engine.item_extras → TMDB
(videos + watch/providers + similar in one call).
- Trailer: a '▶ Trailer' action that opens an in-app YouTube modal embed (Esc /
click-away to close).
- Where to Watch: provider logos for the region (JustWatch via TMDB).
- More Like This: a poster row of similar titles linking out to TMDB.
Both movie + show pages; all keyless (same TMDB key).
Seam tests: extras parse (trailer priority, provider/similar shape), item_extras
gating on tmdb_id, route registered, markup hooks. (RT/Metacritic via OMDb needs
its own key — offered separately.)
Phase 3: the stylized transparent title logo replaces the text title in the
billboard (the big Plex/Netflix 'premium' jump). Sourced from TMDB images
(append_to_response=images, include_image_language=en,null) in the same detail
call — no Fanart key needed.
- schema v4: movies.logo_url / shows.logo_url (idempotent migration).
- TMDB client picks an English logo (then language-neutral, then any); enrichment
backfills logo_url gap-only; show/movie payloads return 'logo'.
- Billboard shows the logo img (with graceful fallback to the text title on error
/ when absent; title kept visually-hidden for a11y). Lazy on-view refresh now
also triggers when the logo is missing, so existing libraries fill it in.
Seam tests: English-logo pick, backfill + payload, schema.
- Movie cards in the library now drill into a movie-detail page (both kinds use
the same open-detail event / video-side navigation).
- New video-movie-detail subpage reuses the .vd-* hooks; video-detail.js is now
kind-aware (root() targets the active page by kind, billboard/links/actions
branch on movie vs show). Flat layout: billboard + a details strip (released /
runtime / studio / status / critic score / quality) + the shared Cast & Crew row.
- Lazy on-view backfill for movies too: engine.refresh_movie_art re-fetches TMDB
(cast/genres/backdrop/ratings) when missing, regardless of match status, via
POST /detail/movie/<id>/refresh-art. movie_match_info added.
Seam tests: movie refresh backfills cast/genres, movie_match_info, route
registered, movie subpage markup, cards clickable for both kinds.
Last capture piece, schema v3 (new people + credits tables; CREATE IF NOT EXISTS
migrates existing DBs on restart, no wipe):
- people deduped by tmdb_id; credits link to exactly one movie OR show (separate
nullable FKs + CHECK, no polymorphic id) with department/job/character/order.
- TMDB client appends credits to the detail call (free) and parses cast (name,
character, photo, billing order) + headline crew (directors/writers/creators).
- enrichment_apply backfills cast/crew gap-only (never clobbers); show/movie
detail return cast + crew. Populates on view via the existing lazy refresh-art.
- Cast & Crew section on the detail page: grouped crew line + a horizontal
cast row with circular TMDB headshots, names, characters (accent hover).
Seam tests: TMDB credit parse (+ job filtering, created_by), backfill + people
dedup across titles, gap-only no-clobber, payload shape.
Root cause: season posters / episode art backfill happen during a show's TMDB
*match*, but already-matched shows never re-run ('Retry all failed' only resets
not_found/error), so existing libraries never got the art.
Fix (Boulder's idea): fetch-on-view + cache. When a show detail opens and any
season lacks a poster, the page calls POST /detail/show/<id>/refresh-art →
engine.refresh_show_art re-fetches /tv/<id> via the TMDB client and backfills
season posters + episode art gap-only, regardless of match status. Cached, so
it's a one-time cost per show; runs once per view; re-renders when done.
Seam tests: refresh_show_art backfills a MATCHED show's seasons, needs TMDB
configured, show_match_info, route registered.
Brings the video modal to parity with music's:
- 'Process first everywhere' control (Movies/Shows/Auto) in the topbar — a global
setting that pins which kind every worker processes first. enrichment_next takes
a priority kind; the worker reads enrichment_priority each loop; GET/POST
/api/video/enrichment/priority persists it. Reuses music's .em-global styling.
- Needs-matching bar now has a live count, status filter (All unmatched / Not
found / Pending) and a debounced search (reusing .em-select / .em-search),
matching the music modal. Episode view stays read-only.
- Live glow (scoped to #vem-overlay): pulsing running dot, accent glow on the
selected worker row + active process-first/kind, and a pulsing 'now processing'
chip in the worker accent. Music's shared .em-* styles untouched.
Seam tests: priority pins kind in enrichment_next + worker honors the setting,
priority endpoint GET/POST + validation, modal feature markup pinned.
Episodes ride along with their show instead of being a separate (tens-of-thousands)
queue: when the TMDB worker matches a show, it now backfills every season's
episodes — still / overview / rating — via /tv/<id>/season/<n> (one call per
season, gap-only so server data is never clobbered). Also backfills season
overviews.
The worker manager 'knows about it': the TMDB breakdown gains an Episodes
coverage entry (matched = has art, rest = pending), shown as its own card; the
Episodes view lists episodes still missing art. It's coverage-only, kept out of
the worker's idle/pending calc so it never blocks 'Complete'.
Seam tests: client season parse, worker cascade fills episodes, gap-only backfill
+ season overview, breakdown coverage (tmdb only), missing-art list, idle calc
ignores episode coverage.
The media server rarely has distinct per-season art, so season cards fell back to
a gradient. TMDB's show detail carries a poster_path per season — the show worker
now returns those, and enrichment_apply backfills seasons.poster_url for seasons
the server left without art (gap-only, never clobbers server art). The image
proxy streams a stored full URL (TMDB) directly vs. proxying a server path.
Seam tests: TMDB returns season posters, backfill fills only missing seasons.
Enrichment now harvests the full detail payload (same call, no extra requests):
- TMDB: tagline, genres, rating (vote_average), runtime, status, first/last air
date (shows), release date + runtime (movies) — on top of overview/backdrop/ids.
- TVDB: switches to /series/<id>/extended for overview + genres.
enrichment_apply now uses BACKFILL semantics: metadata columns are written via
COALESCE(NULLIF(col,''), ?) so enrichment only fills fields the media server
left empty — it never clobbers server-provided data. Genres backfill to the
normalised link tables only when the item has none yet. Whitelist expanded for
the new columns.
Seam tests: backfill-only (server overview/genres kept, gaps filled), genre
backfill when empty, TMDB full-metadata extraction.
Captures the richer metadata the media server already exposes (schema v2;
idempotent migrations + CREATE IF NOT EXISTS, so an existing DB upgrades on
restart with no wipe):
- movies: tagline, rating (audience), rating_critic; shows: tagline, rating,
first/last air date; episodes: still_url + rating.
- Genres as a normalised many-to-many (genres + movie_genres/show_genres link
tables — no comma-blob), deduped, replace-on-upsert.
- Plex (.genres/.tagline/.audienceRating/.rating/.thumb) + Jellyfin (Genres/
Taglines/CommunityRating/CriticRating/Premiere+EndDate/episode Primary) both
extract them; episode stills served via /api/video/poster/episode/<id>.
- Detail payloads return genres/tagline/rating/air-dates + per-episode has_still;
the billboard shows a tagline, ★ score, genre chips, and episode rows render
REAL stills (no more orange placeholder once scanned).
Seam tests for genre dedup/replace, show+episode capture, episode still ref.
Cast/crew (people + credits) is the next phase.
Addresses the 'feels basic' feedback:
- Hero is now a contained glass card with the backdrop blurred INSIDE it +
gradient overlay (same treatment as the music artist hero) — no more bare
gaps around the top/sides. Bigger poster, accent external-link chips
(IMDb/TMDB/TVDB), refined badges + stat tiles.
- Seasons are a poster-art card grid (season = album) with coverage rings/bars
and hover-lift, selecting one renders its episodes below (episode = track) —
episode overviews now shown. Mirrors the artist album-grid -> tracklist.
- Scan now captures real per-season posters (Plex sh.seasons() thumbs / Jellyfin
/Seasons Primary), served via get_art_ref('season') + /api/video/poster/season.
Falls back to the show poster until a re-scan populates them.
Seam tests for the season art ref; shell markup tests still green.
The worker only logged exceptions, so a normal run looked dead — no parity with
the music workers' 'Matched ... -> ID' lines. Now logs each match (noting
'(by server id)' when it used the server's provider id) and each not-found at
INFO. Logger is soulsync.video_enrichment.worker, so it already propagates to
app.log; it just had nothing to say. caplog seam test pins it.
A failed lookup CALL (network/429/5xx/timeout, or an expired TVDB token) was
recorded as 'not_found' — permanently logging a transient blip as 'no match'
and parking the item for retry_days. Now mirrors the music workers' proven
pattern:
- New 'error' status, distinct from 'not_found'; enrichment_next retries BOTH
after retry_days, so errors recover and the queue still advances (no poison
loop). breakdown/unmatched/retry-all and the modal account for it (shown with
the outstanding/pending bucket).
- TMDB/TVDB clients raise on non-200 (429/5xx) so the worker records 'error',
not a false not_found.
- TVDB re-authenticates once on a 401 (expired token) instead of failing every
match for the rest of the run.
Seam tests: error!=not_found, error retried after window, 429 raises, TVDB
token refresh, UI accounts for errors.
The deep scan stores tmdb_id/tvdb_id/imdb_id from Plex/Jellyfin, but the workers
only ever searched by title+year and ignored those ids — re-deriving matches the
server already had exact (wasteful, and a title search can mis-match).
enrichment_next now surfaces the row's known provider id; the worker forwards it
and the TMDB/TVDB clients fetch details BY ID (one call, no /search) when it's
present, falling back to title/year search only for items the server couldn't
identify. Still grabs the overview/backdrop the scan doesn't capture.
Every scan (incremental / full / deep, both entry points) now steps the
enrichment workers aside to cut DB lock contention — same as music. Mirrors
music's contract exactly: pause ONLY workers that were running (a user's manual
pause is left alone), track which we paused, and resume just those in a finally
so success, cancel, and error all clean up. Auto-pause is transient
(persist=False) so it never leaks into the saved <service>_paused flag.
Hooks are injected by get_video_scanner; the scanner is inert without them
(tests build it directly, no engine spun up). Isolated to the video side.
Pause/resume now write <service>_paused to video_settings, and the engine
restores each worker's saved pause when it's (re)built — mirrors music's
<service>_enrichment_paused boot flag. Isolated to video.db; music untouched.
Each video connection item (TMDB/TVDB) now has a Test button that behaves like
music's: saves the key, hits POST /api/video/enrichment/<svc>/test, and toasts
the result via the shared showToast — isolated (own endpoint, own data-attr
handler, reuses the .test-button CSS).
- Client .test() pings TMDB /configuration and TVDB /login to verify the key.
- Endpoint returns {success,message,error}; unknown service -> 404.
94 tests green; music untouched.
- GET/POST /api/video/enrichment/config saves the keys into video_settings;
POST rebuilds the engine (stop old workers, rebuild clients with new keys) so
they pick up the change live.
- video-settings.js loads the saved keys into the TMDB/TVDB fields on the
Connections tab and saves them on change (workers enable once a key is set).
Backend is now end-to-end: key -> client.enabled -> worker matches the library
to TMDB/TVDB and fills ids + metadata. 91 tests green; real DB untouched.
The servers already matched everything to their agents — we were dropping the
IDs. Now we store them:
- Plex: parse item.guids (imdb://, tmdb://, tvdb://); Jellyfin: parse
item.ProviderIds (added ProviderIds to the requested Fields).
- Stored on movies (tmdb_id, imdb_id), shows (tvdb_id, tmdb_id, imdb_id), and
episodes (tvdb_id) via the upserts.
- Dropped the over-strict UNIQUE on movies.tmdb_id / shows.tvdb_id (same title
can legitimately live in two libraries; we dedupe on server_id). Scanner now
wraps each upsert in try/except so one bad item can't abort a scan.
Tests: guid/ProviderIds parsing + IDs persisted. 38 video-DB/scanner tests green.
- Incremental now does smart early-stopping like music: skips already-known
items and stops after 25 consecutive known (server lists recent first),
instead of a blind fixed cap. Falls back to a full pass when the library is
near-empty (<50), matching music's small-DB behavior.
- Deep scan gains music's 50% safety threshold: if removal would wipe >50% of a
>100-row library, it skips (assumes a partial server response, not a real
emptying) — prevents catastrophic deletion.
- Full Refresh already matched (re-read all, upsert, no removal).
Added DB helpers (server_ids, table_count). Tests: early-stop skips known,
small-lib fallback, 50% prune safety. 122 tests green.
The scan tool now behaves like music's, not just looks like it:
- Card matches: help '?' button, 'Last Scan' line, and the Movies/Shows/
Episodes/Size stats grid (populated from /api/video/dashboard on show + after
a scan). Same .tool-card-stats markup.
- Real progress bar: scanner fetches item totals up front (Plex section.
totalSize / Jellyfin TotalRecordCount) and reports a true percent as it
processes; the bar actually moves (movies → shows) instead of sitting at 100%.
- Cancel: the Scan button toggles to 'Cancel' mid-scan and POSTs
/api/video/scan/stop; the scanner checks a cancel flag between items and ends
in a 'cancelled' state. Mirrors music's stop affordance.
Tests: percent reported, cancel stops midway + saves only processed items, stop
route registered, tool-card structure. 117 video/integrity tests green.
- Paginate Jellyfin movie/series listings (StartIndex/Limit, 500/page) so large
libraries aren't truncated on full/deep scans; incremental stays a single
capped page.
- Per-item try/except in iter_movies/iter_shows (matches Plex) so one bad item
doesn't abort the scan.
- Skip Jellyfin episodes with no IndexNumber (consistency with Plex; avoids
ep-0 collisions). All three modes now solid for Plex + Jellyfin.
The scan no longer blindly grabs every movie/show section — it reads the
libraries you map, like music's 'pick your Music library'.
- GET /api/video/libraries: discover the active server's Movies/TV libraries
(Plex sections by type / Jellyfin views by CollectionType) + current
selection. POST: save {movies, tv} per server into video_settings.
- sources.py: _build_source(movies_lib, tv_lib) filters to the mapped library;
get_active_video_source() (used by the scanner) loads the saved selection;
list_video_libraries() lists them unfiltered for the UI. Falls back to all
libraries when nothing is mapped yet.
- VideoDatabase.get/set_library_selection (per-server). 6 tests added; 33 green.
Plex specials/unmatched episodes can have a null index -> getattr(...,0) still
returned None -> 'NOT NULL constraint failed: episodes.episode_number'.
- Plex adapter skips episodes with no index (logged), passes a real number.
- upsert_show_tree defensively skips any episode missing season/episode number,
so no source can crash a scan. Test added.
The scan inherited the shared client's 15s interactive timeout and fetched
episodes per-season (one request each), so a big library read-timed-out
mid-scan (port 32400).
- Dedicated Plex connection for scans with a 120s timeout (built from the
shared config; doesn't touch the interactive client).
- Fetch a show's episodes in ONE show.episodes() call grouped by season,
instead of seasons()+episodes() per season — far fewer round-trips.
- Per-item try/except in iter_movies/iter_shows so one slow/bad item is skipped
and logged, never aborting the whole scan.
Mirrors the music model (full_refresh vs smart incremental, plus deep_scan):
- incremental: only recently-added items from the server (Plex addedAt:desc /
Jellyfin DateCreated, capped); upsert; no prune.
- full: every item; upsert all (refresh metadata + add new); no prune.
- deep: every item; upsert; prune what the server no longer has (empty-scan
safety preserved).
scanner.request_scan/scan_sync take mode; /api/video/scan/request reads
{mode} from the body (default full); adapters take incremental=. Tests cover
deep-prunes / full-doesn't / empty-deep-safety / incremental-requests-recent.
Reads the active media server and mirrors it into video.db, adapting the music
scan pattern (ask the server, upsert, prune what's gone) — isolated from music.
- core/video/scanner.py: server-agnostic VideoLibraryScanner. Consumes a media
source (duck-typed) yielding normalized dicts; upserts movies + show trees,
prunes removed items, reports progress/state. Skips pruning when a scan
returns nothing (transient-failure safety). Background thread + scan_sync.
- core/video/sources.py: Plex + Jellyfin adapters that REUSE the shared
connected clients (MediaServerEngine) but own all video-section logic; produce
normalized dicts. (Validated against a live server by design; scanner itself
is fully unit-tested with a fake source.)
- api/video/scan.py: POST /api/video/scan/request, GET /api/video/scan/status.
- .gitignore: video_library.db + sidecars (mirrors music); tests inject a
tmp DB so none is ever created in the repo.
Tests: scan populate/prune/empty-safety/no-source-error, isolation guard
(core/video imports nothing from music), scan routes registered. 101 green.
Enrichment matched artists by NAME ONLY (0.85 gate), so for a common name
('Rone' has ~5 artists) it stored whichever the source ranked first — often the
wrong one, which then drove a wrong/sparse library 'Standard' discography while
'Enhanced' (the real owned albums) showed the full set.
Fix — use the decisive signal the library already has (the albums you OWN):
- worker_utils: pick_artist_by_catalog() + catalog_overlap_score() +
owned_album_titles()/release_titles(). When 2+ candidates clear the name gate,
fetch each one's catalog and choose the one overlapping the owned albums; falls
back to the current best-by-name pick when there's nothing to disambiguate or
no overlap (so the common single-candidate path makes no extra API calls).
- Wired into Spotify (covers Spotify-Free, same client), iTunes, Deezer (now
multi-candidate search_artists + get_artist_info store), and MusicBrainz
(match_artist gains owned_titles; release-groups as the catalog).
Re-match path (#868):
- build_reset_query now also clears the stored source-ID column for artist/album
item resets — previously a 're-match' only nulled match_status, so the worker's
existing-id short-circuit re-confirmed the WRONG id and never re-resolved. Tracks
excluded (ids live in tags, not a column).
- MusicBrainz also self-corrects its 90-day name->mbid cache: match_artist bypasses
a cached mbid whose catalog has ZERO overlap with the owned albums, so a re-match
isn't blocked by a stale wrong cache entry.
Tests: shared selector (9), per-worker disambiguation for all 4 sources + MB
backward-compat + MB cache-revalidation (8), reset-clears-id (2). 99 worker/
enrichment tests green.
Four refinements on top of the tiered matcher:
1. Direct source track-ID tier (new top tier): enrichment writes each source's own
track ID into the file tags (spotify_track_id/deezer_track_id/itunes_track_id/...).
If we have the active source's track ID, fetch that exact track by ID via
get_track_details — zero search. Tiers are now: track-ID -> ISRC -> album->track
-> artist+title. _read_file_ids reads ISRC + all per-source IDs in one tag read.
2. Skip already-proposed tracks: a re-run loads existing finding entity_ids for the
job and skips those tracks before any API call (pending stays deduped, dismissed
stays dismissed) — re-runs are cheap.
3. Wrong-version guard: the fuzzy tiers (album-search + track search) reject a
candidate whose length differs from ours by >5s (live/edit/remix with same title).
_load_tracks now selects t.duration; exact tiers (track-ID/ISRC/stored-album-ID)
skip the guard.
4. Tighter album matching: same-title cuts in an album are disambiguated by closest
duration when track_number doesn't decide it.
Findings record matched_via = track_id | isrc | album | search. 30 repair tests pass
(added track-ID tier, duration guard, dedup-skip, and unit coverage).
Replaces the blind fuzzy search with a smart hierarchy that uses the data we
already have, best identity first:
1. ISRC embedded in the file tags (enriched track) -> exact track.
2. Album -> track: use the album's stored source ID (albums.spotify_album_id /
itunes_album_id / deezer_id / musicbrainz_release_id / audiodb_id) when the
ALBUM is enriched (even if the track isn't); else find the album by searching
'artist album', then locate our track in that album's tracklist by normalized
title (track_number breaks ties). Pins the exact album context. (artist->album->track)
3. Plain artist+title search with similarity scoring. (artist->track) — loosest.
_load_tracks now returns dict rows (adds track_number + the album source-id
columns). Findings record matched_via = isrc | album | search. All clients
(spotify/deezer/itunes/discogs) expose search_albums + get_album_tracks with a
uniform {'items': [...]} shape, so the album tier is source-agnostic.
26 repair tests pass (added album-tier + _find_track_in_album coverage).
The job was doing a blind fuzzy search for every low-quality track, ignoring that
enrichment writes each track's ISRC + per-source IDs into the file tags. Now it
reads the file's embedded ISRC and resolves the EXACT track via each source's
'isrc:' search (universal cross-source key), guarded by an ISRC-equality check so
a source that ignores the syntax can't produce a false match — exact track, exact
album context, one call. Falls back to the name/artist fuzzy search only for
un-enriched tracks with no usable ISRC. Findings record matched_via=isrc|search.
4 new seam tests (guard accept/reject, ISRC-preferred-over-fuzzy, fuzzy fallback).
Phase 2 of the redesign. The tool that judged quality by extension and auto-dumped
matches into the wishlist is gone; quality scanning is now the reviewed
quality_upgrade repair job.
Removed:
- Frontend: Tools-page Quality Scanner card, its JS handlers/poller/socket listener,
help tooltip + tour entry (webui index.html, core.js, helper.js, wishlist-tools.js).
- Backend: /api/quality-scanner/{start,status,stop} endpoints, the in-memory state +
executor + 1s socket broadcast, the QualityScannerDeps/run_quality_scanner shim.
- core/discovery/quality_scanner.py: the auto-acting worker + deps class (the shared
match/normalize helpers stay — the new job imports them).
Rewired:
- Automation 'start_quality_scan' action now triggers the quality_upgrade repair job
via repair_worker.run_job_now() (AutomationDeps gains run_repair_job_now, drops the
4 scanner fields). Action block's vestigial scope field removed (scope lives in the
job's settings now). NOTE: the 'quality_scan_completed' trigger no longer fires (the
repair job doesn't emit it).
- Updated all automation test _build_deps helpers + conftest tool-progress harness;
deleted the obsolete worker test. 528 affected tests pass; 6123 collect cleanly.
QUALITY_TIERS / _get_quality_tier_from_extension kept (used elsewhere).
The old Quality Scanner tool judged quality by file EXTENSION only (a 128k and a
320k MP3 looked identical), ignored the bitrate-based quality profile, used min()
of enabled tiers so the default profile flagged the ENTIRE non-lossless library,
and auto-dumped every match into the wishlist with no review.
This new repair job does it properly:
- meets_preferred_quality(): pure, bitrate-AWARE decision honoring every enabled
quality bucket (320 MP3 passes a FLAC+320+256 profile; 128 MP3 doesn't). Floor
is the worst enabled bucket, not the best.
- scans watchlist artists or whole library, finds below-quality tracks, matches a
better version at scan time (reusing the existing tested match helpers), emits a
FINDING showing the match + confidence. Off by default; nothing auto-queued.
- _fix_quality_upgrade apply handler adds the matched track WITH album context to
the wishlist — the user-approved version of what the old tool did silently.
- Transcode/fake-lossless detection intentionally left to the existing Fake
Lossless Detector job.
12 seam tests incl. a regression pinning the default-profile flooding bug. The old
tool is still in place; removing it + rewiring its automation action is the next step.
Two issues in the same path:
1. The shared discovery modal pre-renders one row per track from a
separately-fetched frontend track list, then the poll dropped any backend
result without a pre-rendered row (if (!row) return). When the frontend's
track fetch came back rate-limited/partial (~21) while discovery's own fetch
got all 59, the surplus results vanished. Now the modal CREATES a row for any
result lacking one, so authoritative backend results drive the list (fixes
all sources sharing the modal).
2. get_playlist hydrated a whole relationships page in one _get_tracks_batch
call, but Tidal caps filter[id] at 20/request, silently truncating larger
pages. Chunk to the cap like get_album_tracks already does.
Seam + regression tests (tests/test_tidal_playlist_batch_chunking.py).
Status checks asked is_spotify_authenticated() (official OAuth only) instead of
is_spotify_metadata_available(), so a Spotify-Free primary read as disconnected.
get_primary_source_status had spotify_free awareness but it was dead code:
get_client_for_source('spotify') returns None unless officially authed, so the
free-availability probe never had a client. Fetch the client directly for that
check; add the missing free branch to the dashboard test message. Seam + regression tests.
The reconcile read each completed task's final_file_path to find paths — but not
every import path sets it (the verification worker marks the task completed
without it), so tracks that imported via that path were silently dropped (user
saw 3 of 5 symlinks). Root cause: leaning on a fragile per-task field.
Now reconcile_batch_playlists identifies the organize playlists the batch touched
(its own + any reached via a completed track's source_info provenance) and
rebuilds each from CURRENT library ownership via _rebuild_one_from_db
(check_track_exists over membership). It just asks the library what's owned, so
it's robust to HOW a track imported (modal worker / slskd monitor / verification
worker) and still prunes tracks that left. Takes a db handle; all three callers
pass MusicDatabase().
Reconcile tests rewritten for the DB-rebuild form (organize batch, wishlist
provenance, non-organize skip, plain no-op). 973 downloads/imports/playlist
tests pass.
on_download_completed and check_batch_completion_v2 are duplicate completion
paths. Monitor-detected downloads (Deezer / slskd-monitor / verification-worker
imports) finish the batch via the V2 path, but the materialize reconcile was
only added to on_download_completed — so those batches never built playlist
folders (no '[Playlist Folder] Rebuilt' line at all). Add the same non-fatal
reconcile to the V2 path. Now all three completion points (both lifecycle paths
+ the master.py all-owned path) materialize. 550 tests pass.
Symmetric to the post-download reconcile (which handles ADDITIONS): when a
playlist's membership is re-synced (the mirror step — scheduled refresh or the
manual mirror endpoint), rebuild its folder from current membership WITH prune
IF it's organize-by-playlist. So a track that just LEFT the playlist has its
symlink cleaned up the instant membership changes, not only on the next download.
Factored a shared _rebuild_one_from_db (used by the manual 'Rebuild' button and
the mirror hook) + rebuild_mirrored_playlist_if_organized. Gated to organized
playlists, non-fatal at both mirror call sites.
Now the invariant 'folder = the playlist's current owned members' holds on every
change: additions caught at download, removals caught at mirror. 2 new tests
(removed track pruned; non-organized skipped). 985 + 277 tests pass.
Replaces the two organize-only triggers with a single reconcile_batch_playlists
called at both batch-completion points. It groups the batch's newly-resolved
tracks by their per-track playlist provenance:
- the batch's OWN organize playlist → full (re)build with prune, and
- a track that completed for a DIFFERENT playlist (e.g. a WISHLIST fulfilling a
track that belongs to an organize playlist) → ADDED to that folder, no prune.
So a late wishlist arrival now lands in its playlist folder immediately, instead
of only on the next sync/manual rebuild — the folder = the playlist's owned
members, kept true on every ownership change regardless of download path. Uses
the paths the batch already captured (no DB re-match, no waiting on the server
scan/sync). Non-fatal.
3 new reconcile tests (organize full-rebuild, wishlist add-without-prune, plain
batch no-op). 983 downloads/imports/playlist tests pass.
- Settings: 'Playlists Folder' path field (Unlock pattern, separate-root help
text), a Symlinks/Copies selector, and a 'Rebuild playlist folders now' button
(standard test-button style). Wired through PATH_INPUT_IDS / load / save, plus
'playlists' added to the settings save allowlist so it persists.
- POST /api/playlists/materialize/rebuild → rebuild_organized_playlists_from_db:
rebuilds every organize-by-playlist folder from CURRENT ownership, re-matching
each track with check_track_exists (name, not IDs) so it self-heals after a
reorganize / membership change. +1 test.
70 materialize tests + JS integrity pass; settings round-trip wiring verified.
- Routing (step 5): organize-by-playlist tracks no longer set the per-track
_playlist_folder_mode flag, so they import NORMALLY into Artist/Album — exactly
what a normal download does. _playlist_name provenance is kept (origin.py).
- Triggers (step 4): build the playlist folder from the batch's own payload at
both end-of-flow points — the all-owned path in master.py (no downloads, so the
lifecycle never runs) and the batch-complete hook in lifecycle.py (after
downloads). Both gated on playlist_folder_mode, both non-fatal.
Works for the all-owned case (the smack test that did nothing before) and for
mixed owned/downloaded, with no source-ID or mirrored-playlist dependency. The
materialized folder uses the default ./Playlists root + symlink mode until the
Settings UI is added.
Updated the master test to assert the new contract (provenance kept, routing
flag gone). 979 tests pass.
materialize_playlist_from_batch(batch, download_tasks, config) collects the real
on-disk path of every resolved track from the batch's OWN payload — owned via
analysis_results.matched_file_path, downloaded via tasks.final_file_path — runs
each through the playback path resolver (Docker-correct), de-dupes, and hands the
list to rebuild_playlist_folder. Gated on playlist_folder_mode.
No re-matching, no source IDs, no mirrored-playlist lookup — works for any
organize-by-playlist download including the all-owned case. 5 tests. Still
isolated; the triggers wire it in next.
- settings: playlists.materialize_path (separate root, mapped apart from the
music library so the media server never double-scans it) + materialize_mode
(symlink|copy).
- core/playlists/materialize.py: pure filesystem engine that (re)builds a
playlist folder of relative symlinks (or copies) into the real library —
idempotent, prunes stale entries, disambiguates filename collisions, never
escapes the root, and auto-falls-back to copy when the FS can't symlink.
No DB, no app state; ops injectable. 13 unit tests.
Isolated + additive — nothing live calls this yet (stitcher/trigger/routing
come next).
The download analysis already matches every track to a library row via
check_track_exists / manual match, then discarded the result. Keep it: each
analysis_results entry now carries matched_file_path + matched_track_id (the
owned file's real location, or None). Symmetrically, a completed download task
now records final_file_path (where the import landed).
Purely additive, no behavior change, no new matching, zero perf cost — just
stops throwing away what the pipeline already computed. This is the foundation
for playlist materialization: owned + downloaded tracks both report where their
real file is, so the folder can be built by name match, not source IDs.
A DB<->filesystem path mismatch (Docker volume change, remount, Music
Paths unset for the container) makes EVERY library file fail to resolve
to a DB track, so the orphan detector flags the whole library as
orphaned. The mass-orphan check only logged a warning and then created
the findings anyway — so a user batch-applying 'move to staging' or
'delete' would relocate or wipe their entire library.
Make it a hard skip (create zero findings) like the dead-file cleaner
and stale-removal paths already do (#828). Centralise the predicate as
is_implausible_orphan_flood() alongside is_implausible_stale_removal()
so the rule lives in one tested place. Small genuine orphan sets still
surface unchanged — only an implausibly large flood (>50% and >20) is
suppressed.
Tests: seam cases for the new predicate + scan-level regressions (mass
mismatch -> 0 findings; small genuine set -> still reported).
Extends the watchlist export to the full library. The exporter is now general
(core/exports/artist_export.py, renamed from watchlist_export) — adds tidal/qobuz
links and an extra_fields passthrough, so the library export also carries
lastfm/genius URLs + soul_id, and an optional "library counts" toggle adds owned
album/track counts per artist.
- GET /api/library/artists/export?format=&links=&contents= — pulls every artists
row, normalizes onto the canonical *_artist_id keys, optionally GROUP-BY counts
for album/track totals.
- The export modal is now openArtistExportModal(scope): "Export Library" button in
the library header + the existing "Export" on the watchlist bar (a thin wrapper).
Library mode shows the extra "library counts" toggle.
Tests (11): builder across formats + the new tidal/qobuz links + extra_fields
columns; watchlist + library endpoint wiring. 64 integrity green; ruff clean.
An "Export" button on the watchlist filter bar opens a modal (same aesthetic as the
artist DB-record inspector) to export your whole watchlist roster — each artist's
name + source IDs (spotify / musicbrainz / deezer / discogs / itunes / amazon),
with an optional "external links" toggle that adds the discography URLs built from
those IDs. Live preview, copy, and download in the chosen format.
- core/exports/watchlist_export.py: pure builder (json/csv/txt + links, present-IDs
only, deterministic columns) — the single source of truth, fully unit-tested.
- GET /api/watchlist/export?format=&links= shapes the roster + returns it (with
X-Export-Count / X-Export-Ext headers for the modal).
- Frontend reuses the DB-record helpers (_jsonSyntaxHighlight / _arecCopy).
Tests (8): builder across json/csv/txt, links on/off, present-ids-only, empty +
bad-format fallback, mime/ext, and endpoint wiring. ruff clean; 64 integrity green.
Scoped to the watchlist for v1; library-wide export + a "library contents"
(owned albums/tracks) option are natural follow-ups.
New Aria2 JSON-RPC adapter, alongside qBittorrent / Transmission / Deluge. Aria2's
RPC (default :6800/jsonrpc) maps cleanly onto the uniform adapter contract:
- the --rpc-secret token leads every call as "token:<secret>" (no username — the
secret uses the existing password field),
- addUri returns a GID (our torrent id); tellStatus → TorrentStatus with state
mapping (active→downloading, or seeding once the payload is complete; waiting→
queued; etc.),
- remove picks forceRemove vs removeDownloadResult by status, and (since aria2
doesn't delete files on remove) unlinks the file paths itself for delete_files,
- bare-host URLs get /jsonrpc appended.
Wired into adapter_for_type + the Settings dropdown (with a help note: port 6800,
secret in the Password field). All adapter methods go through the same interface,
so the stall/orphan handling and downloads pipeline work unchanged.
Tests (9): registry wiring, state mapping (incl. active→seeding), token-prefixed
params, /jsonrpc fixup, status parse (+ name fallback, no div-by-zero). 126 torrent
tests green; ruff clean.
A maintenance job to keep the music library tidy — finds empty folders left behind
by imports/relocations/deletions (empty artist/album dirs, or dirs holding only OS
junk like .DS_Store/Thumbs.db) and removes them.
Safety is the focus (deleting directories is destructive):
- only TRULY empty folders are flagged — a folder with a cover image or any audio
is never touched; only OS-junk files count as "no real content" (a setting),
- the library root + symlinked dirs are never removed,
- walks bottom-up so a parent left empty by its removable children cascades,
- the apply handler RE-CHECKS emptiness at delete time, so a folder that gained a
file between scan and apply is left alone.
dir_is_removable + remove_empty_folder are pure/injectable seams. Wired through the
job registry, repair_worker apply handler (_fix_empty_folder), fixable-types, and
the findings UI. Opt-in (default off), weekly interval.
Tests (10): removable decision (empty / real-file / surviving-subdir / junk-only /
strict mode) + apply re-check (removes empty + junk, refuses content/root/symlink).
Repair + integrity suites green; ruff clean.
A transient ping failure (network blip, Navidrome busy mid-scan) makes
_setup_client null out the configured creds, and _connection_attempted then
latches the client "disconnected" — so is_connected() returned False forever until
the user hit the manual Test button to re-read config. That's the reported
"disconnects every 5-10 min, reconnects instantly on Test."
Fix: ensure_connection no longer latches on a failed attempt — once a short
throttle (_RECONNECT_THROTTLE_S = 20s) elapses it re-attempts, and is_connected()
triggers that retry whenever it's currently disconnected. So a blip recovers on its
own within the next status check, no manual reconnect. The throttle prevents ping
storms when Navidrome is genuinely down.
Tests: transient failure self-heals after the throttle (and doesn't re-ping within
it); a connected client never re-pings; first connect attempts once. 115 navidrome/
media-server tests green.
Invariant: while security.require_login is on, every profile must have a login
password or it's locked out. Previously only the admin's own anti-lockout existed,
so members could be stranded (created without a password, or login flipped on while
passwordless members existed). Closed all the write-points:
core/security/login_provisioning.py (pure policy, single source of truth):
- members_without_password(profiles) — non-admin profiles that can't sign in
- create_needs_password(require_login) / removing_password_strands(require_login)
Wired into web_server:
- create_profile: while login is on, a new member must be given a password (400
otherwise) and it's set on creation.
- enable-login (settings save): refuses to turn login on while any member lacks a
password — lists them — same shape as the existing admin anti-lockout.
- set-password: refuses to CLEAR a password while login is on (would strand them).
UI: Create Profile form gains a login-password field (alongside the optional PIN);
the Manage Profiles per-member password button (prior commit) covers existing
members + changes.
Tests: pure policy seam + endpoint enforcement (create blocked w/o password when
on, allowed w/ password, no friction when off, clear blocked when on). 442
profile/settings/auth tests green; ruff clean.
Post-processing applies ReplayGain only to slskd/WebUI downloads — content added
via Lidarr, the REST API, or by hand never got it, and there was no way to (re)apply
RG to existing tracks or fix ones where analysis failed (raised in #437 + comments).
New ReplayGain Filler repair job (sibling of Lyrics/Cover Art fillers): scans for
tracks with no ReplayGain track-gain tag and creates a finding per track; the scan
only READS tags (cheap) and no-ops when ffmpeg is absent. Applying a finding runs
the same ffmpeg ebur128 analysis the import pipeline uses (gain = ref - LUFS) and
writes the RG tags in place — no moves, no re-matching. Opt-in (default off),
schedulable like the other maintenance jobs.
Wired: job registry, repair_worker apply handler (_fix_missing_replaygain) +
fixable-types, and the findings UI (label / fix-button / detail rows).
Tests: pure needs_replaygain decision (missing/blank/present/+0.00-is-tagged) +
the apply handler's analyze→compute→write seam with the pipeline gain formula,
ffmpeg-absent + missing-file guards, and registration. 93 repair tests green.
The dev-nightly build runs `ruff check .` before "Build and push to GHCR" in the
same job, so the three S110 (try/except/pass) errors introduced since the last
green build (ce6ce4d) failed the lint step and SKIPPED the image push entirely —
every dev-nightly since #704 went red, so the dev image was never rebuilt and none
of the recent fixes (incl. the #852 WebSocket login-bypass fix) ever shipped to
the image users pull.
All three are deliberate best-effort swallows; annotate them with the repo's
existing `# noqa: S110 — <reason>` convention rather than adding dead logging:
- relocate.py: tag write is best-effort (re-import re-derives tags)
- acoustid_scanner.py: verification-status tag is optional context
- web_server.py: audio-duration probe falls through to 0
ruff check . + compileall now clean; pytest already passed in CI at ce6ce4d.
PR #853 added artist album-list caching to Deezer, but unlike the Spotify path it
had no equivalent of the truncated-fetch guard: Deezer paginates, and a transient/
malformed response on page 2+ (artist with >100 albums) broke the loop and cached
the PARTIAL list as the full discography — serving an incomplete album list from
cache until TTL.
Fix: track whether pagination finished cleanly. A malformed/empty-of-data response
mid-walk now clears a `complete` flag and the artist→album-LIST is cached only when
complete. Individual album entities still cache regardless (each is complete; we
just have fewer). A clean end (short page / empty page / reached limit) still caches
as before.
Tests: a page-1-ok / page-2-errors walk no longer caches (second call refetches
instead of serving a permanently-incomplete list); a clean two-page walk still
caches (happy path intact). 181 deezer/metadata tests green.
noldevin: a magnet stuck "downloading metadata" ran 11h despite a 15-min stall
timeout, got cleared from SoulSync but left active in qbit, then re-grabbed as a
duplicate. Two bugs:
1. Stall never fired on metaDL. StallTracker reset its clock on any `downloaded`
byte increase, but a metaDL torrent's byte counter still ticks up from DHT/peer
protocol overhead while making no real progress — so the clock reset forever.
Fix: the byte counter only counts once metadata is in (size>0). During the
metadata phase (size==0) the only thing that counts as progress is *obtaining*
metadata, so a magnet that can't even do that within the timeout is correctly
flagged stalled. size=None preserves the old byte-only behavior (back-compat).
2. Orphaned in qbit. The monitor's stall exit removed the torrent, but the `error`
exit and the 6h deadline exit only marked the download failed — leaving the
torrent active in qbit, untracked here, so SoulSync re-grabbed the same dead
torrent (qbit logs the duplicate-add). Fix: both terminal exits now run
_cleanup_torrent (shared with the stall path), which removes+deletes (abandon)
or pauses per the stall action — nothing is left orphaned.
Tests (10 new): metaDL byte-noise no longer resets the clock (stalls at timeout);
obtaining metadata resets it; real byte-progress still tracked after metadata;
_cleanup_torrent removes+delete_files on abandon / pauses on pause / no-ops on
empty hash or no adapter / swallows a client error. 151 torrent tests green.
The #832 fix enforces the launch PIN / login via a Flask before_request hook, but
that hook does NOT run for the socketio handshake — empirically a normal endpoint
401s while /socket.io/ returns 200 with the gate on. So removing the client overlay
(Safari "Hide Distracting Items", devtools) + opening a socket streams live data
(downloads, logs, dashboard, notifications) completely unauthenticated.
Fix: the socketio connect handler now enforces the same check and returns False
(rejects the connection) when a gate is active and the session isn't verified.
Rejecting connect blocks every downstream WS event (subscribe/join), so all live
data is covered. core/security/ws_gate.is_ws_connection_blocked is the pure seam:
login mode (when on) > launch PIN > open, mirroring the HTTP gate exactly. Fails
OPEN on a config-read error, same as the HTTP gate.
Audited every other surface empirically with the gate on + unauthenticated: SSE
streams, catch-all pages, library/dashboard data, admin endpoints, search,
image-proxy, audio-stream (incl. a /etc/passwd traversal probe) all 401; /api/v1
key-gated. The WebSocket was the only hole.
Tests (10): pure gate logic (login>pin precedence, all on/off combos) + real
socketio.test_client integration — connect rejected when gate on + unauthenticated,
allowed when gate off or PIN verified.
Root cause (from the reporter's app.log): a ListenBrainz weekly playlist syncs
through the in-memory youtube_playlist_states discovery machine. When that live
state is lost — a Docker restart, or the discovery process ending while the user
waits for the media-server scan — the DB discover-download snapshot survives but
the live state is gone. Every recovery action (Cancel/Reset/Delete) then hit
`key not in states` and returned 404 "YouTube playlist not found" (hence the
confusing "Youtube" on a ListenBrainz playlist), leaving the playlist permanently
wedged with no way to dismiss or re-sync. Works for the maintainer because a
single session with no restart keeps the live state alive.
Fix — these are cleanup ops, so "the thing is already gone" is SUCCESS, not 404:
- cancel_sync core (shared by YouTube + ListenBrainz + Tidal/Deezer/Qobuz/...) →
missing key returns idempotent success.
- reset_youtube_playlist / delete_youtube_playlist → same.
The playlist becomes recoverable: Cancel/Reset clears the dead state and the user
re-syncs fresh.
Tests: cancel_sync core (missing key = idempotent 200 not 404; present key still
cancels + clears the worker + reverts phase); endpoint-level idempotency for
cancel/reset/delete; updated the old test that locked the 404 wedge. 834 sync/
discovery tests green.
The 'retag' fix corrects a mismatched file's tags/DB but leaves it in the WRONG
artist/album folder, so the library shows the right title while the file sits under
the previous track. AcoustID yields only title+artist (no reliable album), so an
in-place move has no safe target.
New 'relocate' action: retag the file, move it into Staging, drop the stale tracks
row, and clean up the emptied folder. The auto-import worker (which watches Staging)
re-identifies it with full metadata and files it correctly — reusing the import
pipeline instead of guessing a destination.
- core/repair_jobs/relocate.py: pure, injectable orchestration (retag -> move ->
drop row) + collision-safe staging_destination. Row is dropped only AFTER a
successful move, so a failed move never orphans the library entry.
- _fix_acoustid_mismatch gains the 'relocate' branch (thin wrapper: resolve path,
staging dir, drop-row closure, empty-parent cleanup).
- UI: "Relocate" button on the AcoustID-mismatch fix modal.
Tests (8): staging-dest collision suffixing; relocate happy path; tag-write failure
still relocates; FAILED move does NOT drop the row; no-tags skips write; a real
file move through safe_move_file; and a handler integration test (file moved to
staging + tracks row deleted end-to-end). Repair + integrity suites green.
resolve_history_audio_path drives a DESTRUCTIVE delete (os.remove), but lived
endpoint-bound in web_server with zero tests. Lifted to core/matching/history_paths
with injected effects (exists / resolve_library_path / lookup_titled_paths) so the
fallback chain — and the collision-safety that stops delete() from removing the
wrong same-title file — is a clean importable seam. web_server now wraps it (DB
lookup + os.path.exists + prefix resolver injected); behavior preserved.
9 tests lock it: recorded-path hit, prefix-resolve fallback, single tracks-table
candidate, and the safety rules — multiple same-title candidates with NO artist ->
None (refuse to guess), artist filter picks only the matching path, artist named
but unmatched -> None, no-title/empty-lookup -> None. Full suite green (5906).
The candidate matcher rejected valid downloads of titles with a '/' or ':' (e.g.
Sawano's "You See Big Girl / T:T") because the unified normalize() removed those
chars ("t:t" -> "tt") while keeping the '_' that source filenames substitute for
them ("T_T" -> "t_t") — the asymmetry tanked the similarity score. Now '/ \ : _'
all map to spaces before the strip, so "/ T:T" and "_ T_T" both normalize to "t t".
Verified on the real library: similarity for the Sawano pair 0.927 -> 1.000; only
348/40786 strings (0.85%, all containing those separators) change; worst-case
joined-variant match (e.g. "12:05" vs "1205") stays 0.889, well above the 0.70
title threshold — no match regressions. Fixes the matching half of #851.
Closes the forgot-login-password gap. A per-profile recovery question + answer lets
a locked-out user reset their own password.
- DB: additive recovery_question + recovery_answer_hash columns (idempotent
migration). set/get-question/verify/has methods; answer is hashed (pbkdf2) and
matched forgivingly (trim + lowercase + collapse whitespace). No recovery set →
never verifies.
- Endpoints (allowlisted in the login gate so they work pre-auth):
GET /api/auth/recovery-question?username= (generic 404 when absent),
POST /api/auth/recovery-reset {username, answer, new_password} — brute-force
limited; a correct answer sets the new password + authenticates the session.
POST /api/profiles/<id>/set-recovery (admin or self) to configure it.
Tests: set/get/verify, forgiving match, hashed-not-plaintext, no-recovery-never-
verifies, full reset flow (wrong answer rejected + password intact; correct answer
resets), unknown-user 404. 25 tests pass. Next: the Settings + login-screen UI.
The backend auth for opt-in username/password mode (security.require_login, default
off → zero change; the launch PIN + picker behave exactly as today).
- core/security/login_gate.py: pure gate (mirrors launch_lock) — when login mode is
on, an unauthenticated session reaches only the page shell, /api/auth/login,
/api/auth/logout, /api/profiles/current, /api/setup/status, and the key-authed
/api/v1 API. Deliberately does NOT expose the profile list pre-auth (you type your
name, not pick from a roster).
- _enforce_login before_request enforces it; _enforce_launch_pin no-ops when login
mode is on (login replaces the shared PIN, per design).
- POST /api/auth/login (username = profile name, case-insensitive; brute-force
limited per IP; generic error so names don't leak) + POST /api/auth/logout.
- Anti-lockout: the settings save refuses to turn ON login mode until the admin
account has a password.
Tests: gate blocks→login→access→logout→blocked; case-insensitive username; wrong
password / passwordless profile / unknown user all 401 generically; login list not
exposed pre-auth; can't enable login without an admin password. 12 tests pass. Next:
the login screen + set-password UI + the toggle (increment 3).
Lets SoulSync sit behind Authelia/Authentik/oauth2-proxy as the gatekeeper: when
security.auth_proxy_header names a header (e.g. Remote-User), a request carrying it
is treated as already-authenticated and passes the launch lock — the proxy did the
login (with 2FA).
- core/security/auth_proxy.py: trusted_proxy_user(get_header, header_name) — returns
the user iff the configured header is present + non-empty; empty header name (the
default) → always None → feature off.
- _enforce_launch_pin ORs it into pin_verified. OFF by default, so a direct install
is unaffected AND a client-spoofed header does nothing unless the operator opted in.
- Doc'd in Support/REVERSE-PROXY.md with the must-strip-client-headers warning.
This is the lightweight Tier 3 (auth-proxy integration), not a full per-user login —
the proxy owns identity; SoulSync trusts it.
Tests: helper off/on/blank/exception-safe; integration — trusted header passes the
gate, no header is locked, and (the safety pin) a spoofed header is IGNORED when the
feature is off. 6 tests pass.
Fold a conservative security-header set into the SAME opt-in proxy mode, so it's
zero-impact when off. When security.trust_reverse_proxy is on, an after_request
adds X-Content-Type-Options: nosniff, X-Frame-Options: SAMEORIGIN, and HSTS
(safe — only honoured over the proxy's HTTPS), via setdefault so it never clobbers
a header the proxy already set. No CSP (needs per-deploy tuning; better at the
proxy). When OFF (default), the after_request isn't registered → no headers added.
Tests: off adds none of the headers; on adds all three. Doc updated. 6 tests pass.
A publicly-exposed instance gated only by the launch PIN was brute-forceable. Added
a lenient in-memory failed-attempt limiter (core/security/rate_limit.py): 10 wrong
PINs from one IP within 5 min → 429 with Retry-After, failures age out on their own
(self-heal, no persistent lockout), and a CORRECT entry clears that IP instantly.
Wired into /api/profiles/verify-launch-pin. By design it can only ever trigger on a
flood of WRONG PINs — correct entry, a couple of typos, or a no-PIN install are
never affected, so normal use sees no change. Keyed per-IP so an attacker can't
lock out a legit user.
Tests: limiter is lenient under threshold, trips on a flood, success clears it,
failures self-heal, per-IP isolation; endpoint returns 429 after 10 wrong PINs with
Retry-After. 6 tests pass.
Tier 1 of "secure behind a reverse proxy". STRICTLY opt-in so direct/LAN installs
are byte-for-byte unchanged.
- core/security/reverse_proxy.py: apply_reverse_proxy_mode(app, config_get) — a
no-op unless security.trust_reverse_proxy=true. When OFF (default), the app is
untouched: no ProxyFix, X-Forwarded-* stays UNtrusted (a direct client can't
spoof its IP/scheme), session cookie keeps Flask defaults. When ON (operator is
behind nginx/Caddy/Traefik with TLS): trust one proxy hop's X-Forwarded-*, and
mark the session cookie Secure + SameSite=Lax. Any config error → safe no-op,
never breaks startup.
- Wired once at app init.
- Support/REVERSE-PROXY.md: nginx (with the Socket.IO Upgrade headers people
always miss) / Caddy / Traefik configs, the setting, and the "put auth in front
(Authelia/Authentik/oauth2-proxy)" recommendation + the off-for-plain-HTTP note.
Tests: off (and missing-key, and a config exception) is a strict no-op — not
ProxyFix-wrapped, cookie defaults intact; on wraps ProxyFix + secures the cookie;
and the real web_server app is NOT in proxy mode by default. 5 tests pass.
The enhanced-tab "Sync" button's stale-removal phase deleted any track whose file
wasn't on disk, with NO guard — so if the music storage was momentarily
unavailable (sleeping NAS, dropped mount, unmounted Docker volume, WSL hiccup),
os.path.exists returned False for EVERY file and one click wiped the whole artist
(tracks + their now-"empty" albums) from the DB. The deep-scan path already had a
50%-stale safety net (#828); this endpoint never got one.
- New core/library/stale_guard.py: is_implausible_stale_removal(missing, total) —
a tested rule (skip removal when missing > 50% of a >=5-track set), centralised
so every stale-removal site can share it.
- sync_artist_library: if the guard trips, SKIP removal (delete nothing), return
removal_skipped + warn; the frontend shows "storage may be offline — skipped"
instead of silently deleting. Empty-album cleanup now also only runs on the
non-skipped path and uses `album_id IS NOT NULL` (fixes the NOT IN-with-NULL
no-op). Frontend also refreshes the view on additions, not just removals.
- @admin_only on the endpoint — it deletes tracks + albums but was ungated, while
the sibling delete_album endpoint is gated.
Deep scan was already safe (different mechanism: server-diff + its own 50% guard).
Tests: guard unit rules; endpoint skips removal when all files missing (keeps the
tracks), removes only the genuinely-gone few otherwise, and 403s for non-admins.
7 new tests pass.
The #843 fallback saved the discovery-cache match using the client's
original_artist verbatim — but the client sends a joined "A, B, C" string, while
EVERY in-memory + sync path keys the cache by the first artist (artists[0]). So a
multi-artist fix (the reporter's exact "Cherrymoon Traxx, Hermol, SBM, BELS"
case) would have saved under a key the sync never looks up — the fix would
"succeed" with no error but silently never apply.
Reduce the client artist to the first (split on comma) in the no-state branch so
its cache key matches the in-memory/sync convention exactly. Single-artist tracks
are unaffected.
Test: no-state save now keys by the first artist, and a new test pins that the
no-state and in-memory paths produce an IDENTICAL cache key for the same
multi-artist track. 74 discovery tests pass.
The discovery FIX → Confirm flow 404'd with "Discovery state not found" whenever
the in-memory discovery state was gone — a server restart, or an imported
playlist that wasn't discovered in THIS process — even though the card is still
shown from persisted data (the reporter's log shows "Returning 0 stored ...
playlists for hydration", i.e. the in-memory states were empty).
The thing that actually makes a manual fix STICK is writing it to the discovery
cache (save_discovery_cache_match), keyed by the original track's name + artist —
which doesn't need the in-memory state at all. But the endpoint 404'd on the
missing state before reaching that write, so the fix was dead after a restart.
- update_discovery_match (core/discovery/endpoints.py) now only does the in-memory
result update when the state exists; the durable discovery-cache write always
runs, falling back to client-provided original_name/original_artist when there's
no in-memory state. With neither a state nor originals it still 404s (unchanged).
- The FIX confirm (wishlist-tools.js) now sends original_name/original_artist
(from the source track it already has) so the backend can key the cache.
Covers all sources that share the helper (tidal/deezer/qobuz/spotify-public).
Tests: no-state-but-originals saves the cache + returns success; no-state-no-
originals still 404s; existing with-state path unchanged. 73 discovery tests pass.
Regression in 2.6.9. The spotify_public source adapter (used by auto-sync /
refresh_mirrored) called scrape_spotify_embed() directly — the embed widget only
exposes ~100 tracks — instead of fetch_spotify_public(), the wrapper the rest of
the app uses, which pulls the full list via the paginated public API and only
falls back to the embed on failure. So initial discovery got the whole playlist
but every auto-sync re-fetch truncated it to 100.
Switched the adapter to fetch_spotify_public (same return shape — drop-in). Albums
still resolve via the embed (already whole); on any failure it falls back to the
embed exactly as before.
Test: the adapter returns all 150 tracks when the full fetch yields 150 (was
capped at 100). 29 adapter tests pass.
A wishlist track (or tracks in an album) that slskd accepted then REJECTED would
sit at "DOWNLOADING... 0%" indefinitely, spam an ERROR every status poll
("…Completed, Rejected - letting monitor handle retry"), and — for albums —
block the whole batch from ever completing.
Root defect: the status formatter's non-manual error branch keeps the task
'downloading' and trusts the retry monitor to resolve it, with NO backstop. When
the monitor can't make progress (a rejected transfer with no other source), the
task is stuck forever.
Backstop: measure how long the ERROR state has persisted (keyed off the task's
last status transition, so a slow-but-healthy transfer is never failed, and each
monitor-retry episode gets a fresh window). Once it exceeds the monitor's retry
window (60s, vs the monitor's ~15s) with no resolution, mark the task failed and
fire the worker-freeing completion callback so the batch can finish. Also log the
error ONCE per episode instead of every 2s poll. The healthy path is untouched —
a working retry transitions the task before the grace, so this never fires.
Manual picks still fail immediately (unchanged).
Tests: rejected-within-grace stays downloading; rejected-beyond-grace fails +
schedules completion; manual pick fails immediately. 45 status tests pass.
YouTube/Tidal/Qobuz results encode the name as ``id||title``. When the title
itself contains a '/' (e.g. the Sawano AoT track "YouSeeBIGGIRL/T:T"), two places
wrongly basename-split it on the slash and kept only the last segment ("T:T"):
- core/downloads/file_finder.py — the completed-download finder truncated its
search target to "T:T", so the real on-disk file (slash sanitised by the
writer) never matched → "not found after processing" → the download got
QUARANTINED. Now an encoded ``id||title`` keeps the whole title as the target
and contributes no remote-directory components; real Soulseek PATHS still get
basename + dir extraction unchanged.
- webui/static/downloads.js — the manual-search FILE column showed only "T:T".
Added a ``||``-aware short-label helper (mirrors the correct handling already
used elsewhere in the file); real file paths still show their basename.
Tests: the finder locates "YouSeeBIGGIRL∕T: T.mp3" from the encoded title
"…||YouSeeBIGGIRL/T:T" (the screenshot case), doesn't match an unrelated file,
and a genuine Soulseek path still resolves to its last segment. 21 finder tests
+ 64 script-split integrity tests pass.
- ⚠ Unverified filter rows gain actions: inline play (range-streamed from the
history file path, server-side only), YouTube compare, Approve -> new
human_verified status (tag + history + tracks; AcoustID scanner skips these
entirely), Delete (file + entry)
- API: /api/verification/<id>/stream|approve|delete (path only from DB row)
- backfill: history rows with acoustid_result='fail' that exist at all were
imported despite the failure = force_imported (covers pre-fix fallback
imports like the user's 'My Ordinary Life')
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The pipeline has three success exits (simple download, playlist folder mode,
main) but only the main one persisted the verification status — force-imported
playlist tracks got no tag, no history status, and never appeared in the
Unverified filter. Extracted _persist_verification_status() and call it at
every exit. One-time idempotent backfill derives status for existing history
rows from their recorded acoustid_result (pass->verified, skip->unverified).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- '⚠ Unverified' filter pill on the Downloads page lists completed downloads
whose verification status is unverified/force_imported (review queue)
- the quarantine-retry engine's attempt counter (already tracked internally)
is now surfaced: task.retry_info ('2/5') shows next to Searching/Downloading
in the modal and as 🔁 on the Downloads page rows, with the trigger in the
tooltip
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The persistent Completed list is built from library_history (not live tasks),
so the badge never showed after a session ended. Column added (additive),
written at import, passed through _build_history_download_item, rendered by
_adlVerifBadge next to the status label.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Also: evaluate() treats an empty expected artist as title-only comparison
(old scanner behaviour — a missing DB artist is no evidence of a wrong file),
and the thresholds are now defined once in the core and re-exported.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The library AcoustID scan now calls audio_verification.evaluate() (alias-aware
artist match + cross-script SKIP) instead of its own non-ASCII-stripping
_normalize and threshold logic, so it no longer false-flags correct anime-OST /
kanji tracks. Duration-collision guard kept as a scanner pre-check on the top
recording. evaluate() is now purely a title/artist/version/cross-script decision.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
verify_audio_file now calls audio_verification.evaluate() and re-exports
normalize/similarity/_alias_aware_artist_sim from the core, so import and the
library scan can no longer drift apart. Alias-rescue diagnostic moved to the core.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Background automations had no session, so get_current_profile_id() fell back to
admin (1) — wrong for a non-admin's scheduled job. Now the engine declares the
automation's owner around handler execution via a contextvar
(core/profile_context.py), and get_current_profile_id() consults it only when
there's NO web request. So:
- a real logged-in request always wins (foreground unchanged),
- admin + system automations are profile 1 → resolve to admin exactly as before
(the 8 admin-owned auto-sync pipelines behave identically),
- only non-admin-owned automations gain their correct identity, deep through the
whole call chain (incl. the per-profile client resolvers) — no threading
profile_id through dozens of signatures.
Reset in a finally so a pooled thread can't leak the override to the next job.
Tests: contextvar set/reset/nested; get_current_profile_id honours the override
only outside a request (a real session still wins); and end-to-end — the engine
runs a non-admin automation as profile 4, an admin one as 1, an explicit trigger
profile overrides the owner, and the context resets even when the handler raises.
27 + 4 tests pass.
Part 2 (next): point the sync handlers' source-playlist READ at
get_spotify_client_for_profile so a non-admin's auto-sync pulls THEIR playlist.
