SoulSync standalone matches library tracks without Plex fetchItem,
reports missing counts correctly, and skips server playlist writes.
Automation re-syncs when the mirror grows; after sync finishes, starts
organize download (organize-by-playlist) or wishlist processing.
UI: Spotify URL playlist-folder controls, organize toggle layout in the
discovery modal, reload organize preference when reopening Download Missing.
Co-authored-by: Cursor <cursoragent@cursor.com>
Closes the gap where similar artists only existed for WATCHLIST artists: a new
background worker populates them for the whole LIBRARY, slotting into the
existing enrichment-worker pattern (bubble + Manage Enrichment Workers modal,
status/pause/resume, matched/not_found/pending/errors).
Per source-matched library artist → get_musicmap_similar_artists(name, 25)
(the same matcher the artist-detail page uses: fetches MusicMap names, matches
each to the user's source chain — primary + active fallbacks — returns only
matched artists) → store via add_or_update_similar_artist keyed by the artist's
metadata source id, the SAME key the watchlist scanner + artist map use, so the
two cooperate (idempotent upsert + retry_days window).
- core/similar_artists_worker.py: pure seams (pick_source_artist_id,
map_payload_to_store_kwargs, process_artist) + the threaded worker; skips
artists not yet source-matched; classifies not_found vs transient error
(retry after 30d).
- DB migration: similar_artists_match_status / _last_attempted on artists
(mirrors every other source worker's tracking columns).
- Registered in EnrichmentService + instantiated in web_server, DEFAULT-PAUSED
(opt-in) like Amazon — MusicMap is scraped/outage-prone + this is library-wide.
- SERVICE_ENTITY_SUPPORT['similar_artists']=('artist',) so the modal breakdown
('artists with / without similars') + Retry work; manual-match (inapplicable
to a relationship) is gated out via relationship:true.
- 10 seam tests; existing 80 enrichment tests still pass.
Note: keys under profile 1 (single-profile setups); multi-profile is future work.
The on-canvas overlay text can't be copied (and can't be grabbed mid-freeze), so
when perf mode is on ('d'), the frontend now also POSTs the render timings to
/api/discover/artist-map/perf ~1.5x/sec, which logs them as [ARTMAP-PERF] in
app.log. Lets the bottleneck be diagnosed from the server side with no manual
copying.
Persist organize_by_playlist on mirrored playlists and run playlist-folder
downloads from the auto-sync pipeline instead of the global wishlist phase.
Register SoulSync library rows after playlist-folder post-processing, route
failed organize batches to the wishlist correctly, and skip sync-time
unmatched wishlist only when organize download handles retries.
Invalidate stale playlist track caches on refresh (Spotify and Deezer ARL),
re-mirror on refetch, and improve standalone playlist modals (re-analysis,
Open in Mirrored). Add filesystem missing-track detection and tests.
Co-authored-by: Cursor <cursoragent@cursor.com>
The no-auth 'add by link' path scrapes Spotify's embed widget, which only ever
contains ~100 tracks and can't paginate — so big public playlists got
truncated. This adds an in-house anonymous fetch that pulls the FULL list:
- core/spotify_public_api.py: reads the anonymous web-player accessToken Spotify
already embeds in its own open.spotify.com page HTML (no app credentials, and
no rotating TOTP secret for us to maintain), then paginates
/v1/playlists/{id}/tracks 100 at a time until the whole playlist is pulled.
Returns the embed scraper's exact shape. Pure helpers + injected http_get so
it's unit-testable without the network.
- core/spotify_public_scraper.fetch_spotify_public(): tries the full fetch for
playlists; on ANY failure (or for albums) falls back to scrape_spotify_embed.
Worst case == today's behaviour, so the link path can't regress.
- web_server: the link-tab endpoint and the authed flow's last-resort scrape
now both go through fetch_spotify_public.
Scoped entirely to the spotify_public_* (no-auth) path — the authenticated
playlist sync is untouched. 11 tests (token extraction, normalisation,
pagination past 100, and the embed-fallback orchestration).
Caveat: rides Spotify's undocumented page-embedded token — expected to break
when they change their page; it degrades to the embed fallback, never crashes.
Needs a live click-through to confirm the token path works end to end (can't
hit Spotify from the test env).
Per-worker processing-order override + UI polish.
Feature — pin an entity group to enrich first:
- Each worker normally runs artist -> album -> track. A user can pin one
group (artist/album/track) to run first from the modal; the worker keeps
that group first until it's exhausted, then resumes the normal chain.
- core/worker_utils.py: read_enrichment_priority() (reads
<service>_enrichment_priority each loop, live) + priority_pending_item()
(shared, whitelisted query returning the worker's expected item shape;
Spotify/iTunes get album_individual/track_individual via a type map).
- A guarded ~6-line hook at the top of all 11 workers' _get_next_item.
CRITICAL: when nothing is pinned (default) the hook returns immediately,
so default enrichment order is byte-identical to before. Discogs (no track)
and Genius (no album) only honor their supported entities.
- core/enrichment/api.py: GET/POST /api/enrichment/<id>/priority (+ config_get
hook); POST validates the entity against what the source enriches.
- 14 new tests (helper shapes, exhaustion, route get/set/clear/validate).
UI:
- Refined hero header: identity + inline status left, single Pause right,
'now enriching' quiet sub-line; overall coverage % moved into the stats
section ('82% matched · 1,203 of 1,460'). Hero gently pulses while running.
- New processing-order strip: artist→album→track steps showing the live phase
(pulsing 'now'), pinned group ('first' + 📌), and done/remaining; click a
step to pin it, click again for auto.
py_compile clean across all 11 workers; 52 enrichment tests green.
Dashboard 'enrichment bubbles' could pause/hover but offered no way to
*manage* a worker. This adds a full management modal opened from a new
header button, covering all 11 enrichment sources.
Backend (testable core helper + seam tests; no live-DB dependency):
- core/enrichment/unmatched.py: pure, whitelisted SQL builders for the
unmatched browser. service/entity validated against a support map (never
interpolated raw); search + pagination bound as params; tracks join albums
for artwork; limit capped at 200.
- database/music_database.py: get_enrichment_unmatched() +
get_enrichment_breakdown() (the breakdown splits matched/not_found/pending,
which the existing get_stats().progress lumps together).
- core/enrichment/api.py: GET /api/enrichment/<id>/{unmatched,breakdown} on
the existing blueprint + a db_getter hook.
- web_server.py: wire db_getter=get_database.
- tests/enrichment/test_unmatched.py: 19 tests across builders, DB methods,
and Flask routes.
Frontend (vanilla, matches app conventions):
- webui/static/enrichment-manager.js: worker rail with live status + coverage
micro-bars, accent-themed detail panel (hero header, segmented matched/
not_found/pending stat cards, current item, pause/resume), and a searchable
paginated unmatched browser with inline manual match (reusing
search-service + manual-match) and retry (clear-match re-queues).
- Polish: entrance/exit motion, scroll-lock, Escape, refresh control,
flicker-free polling (in-place updates), skeleton loaders, relative
timestamps, per-worker accent theming, real dashboard logos reused at
runtime (with the same invert/circle treatment), responsive rail.
- index.html: header button + script include. style.css: full styling.
Reuses existing pause/resume, status, and manual search+assign endpoints.
Backend tests green (19 new + 11 existing enrichment tests).
The sync editor renders server covers as <img src="/api/navidrome/cover/{id}">,
but no Flask route ever served that path — so every Navidrome cover 404'd, on
every album, art or not. The source (left) side then went blank too: a source
row with no native art (e.g. YouTube, which provides none at mirror time) falls
back to borrowing the matched server track's cover — i.e. that same dead route.
So both sides collapsed to nothing.
Fix:
- New NavidromeClient.build_cover_art_url(cover_id) — builds the absolute,
Subsonic-authenticated getCoverArt URL (base_url + token/salt), keeping
credentials server-side. Uses a FIXED cover-art salt so the URL is
deterministic for a given (server, password, cover_id): a rotating salt (as
in _generate_auth_params) would make every request a unique URL → image-cache
miss every time + a dead, never-reused cache row per fetch. Token auth doesn't
require a unique salt, and the password is never exposed (only its salted md5).
- New route /api/navidrome/cover/<cover_id> — resolves that URL and streams the
image through the shared image cache (same pattern as /api/image-proxy), with
a private max-age so the browser caches by the stable route URL.
Effect: server side works for any album that has art in Navidrome; matched
source rows with no native art now borrow the (now-working) server cover.
Unmatched YouTube rows stay blank — no image exists anywhere to show.
Tests: tests/test_navidrome_cover_url.py (8) — URL structure + salted-token auth
(never the raw password), determinism (same id -> same URL so the cache hits;
different id/password -> different URL), optional size, and the not-connected /
no-id / no-credentials guards.
Caveats: not executed against a live Navidrome (no server in CI) — the URL
builder is unit-tested; the route's cache→HTTP→bytes round-trip is read-verified
only. Scope is the sync editor's Navidrome route; Plex/Jellyfin server-cover
branches and any other modals using a different mechanism are untouched.
Three compounding bugs hit tracks whose source metadata is YouTube/streaming-
shaped — title "Artist - Song", artist "Official Artist"/"Artist - Topic"/
"ArtistVEVO" (reported: "Arctic Monkeys - Do I Wanna Know?" by "Official Arctic
Monkeys"). Server-agnostic — affects Plex/Jellyfin/Navidrome, not just the
reporter's Navidrome.
Bug A — the match fails. The confidence scorer and the editor's reconcile both
compared the raw "Artist - Song" title against the library's clean "Song"; the
length-ratio penalty + floor drove it to ~0.18 (NO-MATCH), so the track showed
unmatched while its server copy showed as an orphan "extra". New pure
core/text/source_title.py (clean_source_artist / strip_artist_prefix /
canonical_source_track) strips the channel/video decoration, applied at BOTH
matching seams: services/sync_service._find_track_in_media_server (tries raw
then canonical, keeps the best) and the editor reconcile. Conservative: a title
prefix is stripped only when it equals the artist, so "Self-Titled", "Jay-Z",
and "Marvin Gaye" (by another artist) are untouched, and the canonical form is
an additional best-of candidate so it can only help.
Bug B — manual matches never persisted. get_server_playlist_tracks built the
per-source entry WITHOUT source_track_id, so "Find & add" posted an empty id
and _persist_find_and_add_match returned early. The match reverted to "extra"
on reload and re-adding looped. The editor's 3-pass matcher is now lifted to a
pure, tested core.sync.playlist_reconcile.reconcile_playlist that includes
source_track_id (the frontend at pages-extra.js:1836 already reads + sends it).
Bug C — manual match duplicated + delete wiped all copies. "Find & add" always
inserted, so linking a source to an already-present server track appended a
duplicate (pos 72, 73...); remove filtered out EVERY entry with the target id.
New pure core.sync.playlist_edit (plan_playlist_add: link-don't-duplicate when
the target is already present; remove_one_occurrence: drop a single copy) wired
into the Plex/Jellyfin/Navidrome add + remove branches.
Tests (extreme): tests/test_source_title.py (35), tests/test_playlist_reconcile.py
(11 — incl. the reported case, parity for override/exact/fuzzy/extra, and
duplicate-server handling), tests/test_playlist_edit.py (12). 286 matching/sync
tests still pass.
Caveats: the sync_service change and the add/remove/editor endpoints are
read-verified, not executed against a live media server (none in CI). The pure
cores they call are exhaustively unit-tested; output-shape parity of the
reconcile lift is covered. Delete removes the first matching copy (duplicates
are identical, so harmless).
After an update, installs became unusable: the Amazon enrichment worker runs by
default, the default public T2Tunes proxy (t2tunes.site) was returning
503 'Amazon Music API is not initialized', and the worker treated every album
as an individual error -- logging an ERROR per item, churning network + DB
continuously across the whole library, and marking every row 'error' (a state
the retry tiers never re-attempt, so even after the proxy recovered nothing
re-enriched). The reporter couldn't reach the UI to turn it off.
Two-part fix:
1. Source-outage circuit breaker (core/amazon_outage.py, pure + tested):
- is_source_outage(exc) distinguishes a whole-source outage (HTTP 5xx,
'not initialized', connection failure, non-JSON error page) from a real
per-item miss (404, transient 400, etc.).
- On an outage the worker now leaves the item UNTOUCHED (so it's retried once
the proxy recovers instead of being permanently burned to 'error'), logs
ONCE per streak, and backs off with next_poll_delay_seconds() -- escalating
30s -> 60s -> ... capped at 30 min -- instead of grinding every 2s. It
auto-resumes the normal cadence the moment the source answers (success OR a
non-outage error both clear the streak).
- AmazonClientError now carries status_code so detection doesn't rely on
message parsing.
2. Opt-in by default (web_server.py): amazon_enrichment_paused now defaults to
True. Because enrichment depends on an external public proxy that can be
down, it stays paused unless the user explicitly enables it -- a proxy outage
can no longer take down installs that never opted in. (Behaviour change:
anyone on the old auto-on default is now paused; re-enable in Settings.)
Together: on update the worker is paused -> no flood -> UI accessible; opted-in
users are protected from future outages by the breaker.
Tests: tests/test_amazon_outage.py (12) pin the classifier across every error
surface (incl. the exact 503 'not initialized' case) and the back-off schedule
(monotonic, capped). 157 Amazon tests pass; lint clean.
Note: could not reproduce the exact 'UI fully unreachable' mechanism remotely
(WAL + 8 gthreads shouldn't hard-lock); the fix removes the flood/churn that is
the practical cause and defaults the feature off.
Lets users pick which providers' cover art to use and in what priority,
generalizing the single prefer_caa_art toggle into an ordered, mix-and-match
list (Sokhi's request). Fully opt-in: default album_art_order is [], so every
existing install is byte-for-byte unchanged until the user enables sources.
How it works:
- Per album, walk the user's ordered sources top-to-bottom; the first source
that actually has THIS album's cover wins. A miss falls through to the next;
if all miss, the download's own art is kept (today's default). The worst case
is always exactly the cover you'd get today -- never wrong art, never an
error into the download.
- Connection-gated: a source is only tried when the user is connected to it
(free sources CAA/Deezer/iTunes/AudioDB always; Spotify only when
authenticated). Tidal/Qobuz/HiFi deferred (cover-URL construction + no clean
core accessor -- not shipping unverified extraction).
- Album-match validated: a source's art is used only when the album it returns
matches the requested artist+album (significant-token subset, tolerant of
Deluxe/Remastered/articles/feat./multi-artist). A loose top search hit for a
different record is treated as a miss -> guarantees no wrong-album art.
- The list supersedes the legacy prefer_caa_art toggle: when album_art_order is
non-empty it is the sole authority (add 'caa' to the list to use Cover Art
Archive), and prefer_caa_art is neutralized for both the embedded-tag art and
cover.jpg paths. With an empty list, prefer_caa_art behaves exactly as before.
Implementation:
- core/metadata/art_sources.py: pure resolver -- effective_art_order (config +
legacy back-compat) and resolve_cover_art (ordered walk + fallback,
exception-safe per source). No network/config/DB; fully unit-testable.
- core/metadata/art_lookup.py: availability gating, per-source lookups against
existing clients (Deezer/iTunes/AudioDB/Spotify search + CAA via MBID),
album-match validation, per-album caching, and select_preferred_art_url --
the single gate the pipeline calls (no-op unless an explicit list is set).
- core/metadata/artwork.py: wired into embed_album_art_metadata and
download_cover_art, gated so no configured list == current behavior.
- web_server.py: GET /api/metadata/art-sources (connected sources only).
- config/settings.py: default album_art_order: [].
- webui (index.html + settings.js): reorderable list in Core Features reusing
the hybrid-source-list pattern + real service logos (with emoji fallback);
load/save wired through the existing metadata_enhancement settings flow.
loadArtSourceOrder populates the saved order synchronously (filtered to known
sources, not availability) so a save before the availability fetch resolves,
or a temporarily-disconnected source, can never wipe the saved order.
Tests: 40 unit/seam tests (resolver ordering/fallback/back-compat, availability,
per-source extraction, album-match validation incl. wrong-album/wrong-artist
rejection, caching, exception-safety, the off-by-default gate). Full metadata
suite still green (610 passed) -- the gated integration changes nothing when no
list is configured.
Note: the settings UI (DOM-heavy, not unit-testable in the JS harness) and the
live per-source art-fetch quality are validated by manual testing.
New GET /api/downloads/task/<id>/detail merges the live download task with its
library_history row (the data the Download History cards show) into one payload
the upcoming track-detail modal renders: status kind, title/artist/album,
source, quality, final location, AcoustID verdict, and expected-vs-downloaded.
Assembly + status classification live in core/downloads/track_detail.py as a
pure, importable build_track_detail()/classify_status_kind() (9 unit tests);
the endpoint is thin glue that looks up the matching history row by track.
Accepting a quarantined item re-imported the file correctly, but the download
modal kept showing 'Quarantined'. The re-import ran through the inner pipeline,
which doesn't mark task completion (that's the verification wrapper's job), and
the sidecar context had no task_id anyway (popped before quarantine).
The chooser's Accept now sends the originating task_id, and the endpoint
re-runs the import through the verification wrapper with that task_id (+ batch_id
looked up from the task), so the task is marked completed only after the file is
verified moved — the row flips to Completed on the next poll. Manager-tab
approvals (no task_id, no JSON body — handled via get_json(silent=True)) keep
the original inner-pipeline path.
Also clear has-candidates + the quarantine dataset on every status render so a
row that goes quarantined -> completed doesn't keep a stale chooser attached.
Clicking a quarantined track's status used to open the generic search modal,
identical to a plain failure — no way to review or recover the file. It now
opens a chooser:
- Listen: streams the file in-app via a new /api/quarantine/<id>/stream
endpoint (range-supported; the real audio Content-Type is recovered from the
sidecar since the on-disk file ends in .quarantined).
- Accept & Import: existing /approve (restore + re-import, gates bypassed).
- Search for a different result: the existing candidates modal (old behavior).
Non-quarantine failures (not_found / failed / cancelled) are unchanged — a
single click listener routes by dataset set at render time, so a task that
fails then later quarantines can't end up double-bound.
Also fixes the Accept failure on Windows: the Listen stream holds an open file
handle, so the subsequent restore move hit WinError 32 ('file in use') and the
endpoint mislabeled it 'thin sidecar'. Accept now releases the audio handle
before approving, and approve/recover moves retry briefly on transient OS locks
(_move_with_retry). Accept also auto-falls-back to Recover-to-Staging for
genuinely thin/orphaned sidecars.
Tests: stream-info resolution (sidecar + filename-fallback + missing), and
_move_with_retry success/give-up.
The save endpoint coerced library_track_id with int(), which rejected
every non-numeric id with "Invalid library track id". Library ids are
str(ratingKey) — numeric for Plex but GUIDs/hashes for Navidrome,
Jellyfin, and other Subsonic servers — and are stored in the TEXT
tracks.id column, so the coercion broke manual matching on every
non-Plex server.
Replace the int() coercion with a normalize_library_track_id() helper
that trims and rejects only empty input, passing the opaque string id
straight through. Plex numeric ids are unaffected (SQLite INTEGER
affinity still stores a clean numeric string as an int, so existing
matches are byte-identical) and no schema migration is needed (the
INTEGER column already stores non-numeric ids as text).
Tests: pure-helper cases (numeric/GUID/whitespace/empty) plus a real-DB
round-trip proving a GUID id saves, reads back unchanged, and enriches.
Post-incident hardening. A WAL-mode DB corrupted (most likely an interrupted
write during a hard restart), and the backup routine made it unrecoverable:
it (a) never checked integrity, so src.backup() faithfully copied the corrupt
pages into every rolling backup, and (b) pruned oldest-by-mtime, so each new
corrupt backup evicted the last good one. Result: all snapshots poisoned.
New core/db_integrity.py (pure, unit-tested):
- quick_check()/is_healthy(): fast read-only PRAGMA quick_check probe.
- safe_backup(): verifies the SOURCE is healthy BEFORE the Online-Backup copy
and the RESULT after; refuses + discards rather than save a corrupt copy.
- prune_backups(): rotation that NEVER deletes the most-recent verified-healthy
backup, even to honor max_keep — so a run of bad backups can't drop your last
good snapshot.
Wired into BOTH backup paths (the /api/database/backup endpoint and the
auto_backup_database automation handler) — they now refuse on integrity failure
(409 / error status, existing backups untouched) and prune safely.
Tests: tests/test_db_integrity.py (8) using REAL temp DBs incl. a physically
corrupted one — proves refuse-corrupt-source, discard-corrupt-result, and the
exact incident scenario (newest backups corrupt -> the older healthy one is
protected from pruning). Existing maintenance-handler backup test still green
(29 passed). compile + ruff clean.
NOTE: this prevents silent backup poisoning; it does NOT stop the underlying
corruption. Follow-ups still worth doing: WAL-checkpoint on clean shutdown +
a periodic live-DB integrity alert (so corruption is caught on day 1).
listening_history was populated ONLY from the media server; the web player
recorded nothing. Now a play heard ~10s logs to listening_history AND bumps
tracks.play_count/last_played — so the existing 'recently played' query reflects
actual SoulSync listening, and the Phase-2 smart-radio recency signal gets real
data.
- core/playback/play_log.build_play_event(): pure, DB-agnostic normalizer from
player payload -> listening_history event shape. Caller supplies the
timestamp (stays pure). Composite/streamed ids never become the int
db_track_id; bool ids rejected; missing title -> skip. 9 unit tests.
- MusicDatabase.record_web_player_play(): inserts the history row + increments
play_count/last_played for the library track in one call.
- /api/library/log-play: thin endpoint, server-side timestamp, best-effort
(logging failure never 500s / never affects playback).
- Frontend: npMaybeLogPlay on timeupdate fires once per track at the 10s
threshold (flag reset in setTrackInfo, set-before-fetch so it can't
double-fire), fully fire-and-forget.
Pure builder is unit-tested; the DB write can't run in-sandbox (real DB throws)
so it's a thin straightforward insert+update. JS + web_server parse clean.
Self-audit of the revamp surface found real bugs, now fixed:
- DOUBLE-ADVANCE race: crossfade starts ~6s before track end, but when the
track actually 'ended' fired, onAudioEnded ALSO advanced — two skips.
onAudioEnded now bails when npXfadeActive (crossfade owns the advance).
- STRAY CROSSFADE on manual skip/stop: skipping or stopping mid-fade left the
interval running, firing npFinishCrossfade on top of the manual change, and
left the second <audio> playing. Added npCancelCrossfade() (clears the timer,
tears down the 2nd audio, restores main volume) called at the top of
playQueueItem and in handleStop. The fade interval also self-checks
npXfadeActive each tick. npFinishCrossfade clears all flags cleanly so the
legitimate handoff isn't treated as an abort.
- stream_start: moved 'global stream_background_task' to function top (it was
declared inside an if-block — parsed, but brittle/bad form).
web_server parses; 76 streaming+radio tests pass; JS syntax clean; CSS balance
unchanged from HEAD.
Wires the StreamStateStore (Phase 3a) into the live routes so each browser/
device gets its OWN playback instead of every client sharing one global
stream_state. Fixes the long-standing limit where a second tab/device/listener
would hijack the one playback.
- _stream_session_id(): stable per-browser id stored in the Flask session
cookie (falls back to DEFAULT when no request context — e.g. the socket
broadcast thread — so single-user behavior is identical).
- _current_stream_state(): the StreamSession for the calling browser.
- Routes rewired to the caller's session + its own lock: /api/library/play,
/api/stream/start, /api/stream/status, /stream/audio, /api/stream/stop.
- Background tasks tracked per session (stream_tasks[sid]) instead of one
global Future, so stopping/replacing one listener's stream doesn't cancel
another's. Executor bumped 1 -> 4 workers so concurrent listeners don't
queue behind each other.
- Per-session staging: named sessions stage under Stream/<sid>/Stream so
listeners never clear each other's files; default session keeps flat Stream/.
- /stream/status now returns THIS listener's state, which is what the frontend
polls — so per-listener works without touching the socket broadcast (left
serving the default session, now vestigial for status).
Isolation invariant covered by tests/streaming/test_stream_state_store.py
(distinct sessions independent, default stable). The route-level cookie wiring
+ actual two-client no-collision behavior need live multi-client verification
(can't be tested without booting Flask + separate cookies) — EXPERIMENTAL,
needs Boulder to test with 2 browsers/devices. 33 streaming tests still pass;
web_server parses.
Crossfade was a no-op toggle. Real crossfade needs two tracks audible at once,
but /stream/audio only serves the ONE current track (single global
stream_state). So:
- web_server: extracted the range-serving body of /stream/audio into
_serve_audio_file_with_range, and added /stream/library-audio?path= which
serves an arbitrary LIBRARY file through it. Security: the path is resolved
via _resolve_library_file_path (same validator /api/library/play uses) so it
only serves files inside the configured transfer/download/media-library
dirs — not arbitrary disk.
- frontend: a second hidden <audio> (#audio-player-xfade) preloads the NEXT
library track when the current one is within 6s of ending (crossfade on,
not repeat-one), ramps the two volumes in opposite directions, then hands
off to playQueueItem so all normal now-playing state is set.
Honest limits (documented in code): library→library only (streamed tracks
hard-cut as before); there's a brief silent reload at hand-off because
playQueueItem re-points the single stream_state — the perceived crossfade has
already happened by then. EXPERIMENTAL — needs Boulder's live audio
verification; I can't test audio in-sandbox.
33 streaming tests still pass (stream_audio refactor is behavior-preserving).
Foundation for multi-listener playback. Today web_server.py keeps ONE global
stream_state dict + one lock (web_server.py:747), so the whole server shares a
single 'currently playing' — every tab/device is a remote for the same
playback and two listeners collide. That global is woven through ~22 sites and
isn't unit-testable where it lives.
Lifted into core/streaming/state.py WITHOUT changing behavior:
- StreamSession: one playback's state, dict-compatible (s['k'], s.get,
s.update, 'k' in s) so existing call sites work unchanged, each with its
OWN RLock so distinct sessions never block/clobber each other.
- StreamStateStore: registry of named sessions; lazy + race-safe create;
DEFAULT session reproduces today's exact single-global behavior. Also
drop()/active_ids()/session_ids() for the eventual per-listener wiring.
web_server.py now binds (DEFAULT) and
. Drop-in: every .update()/[k]/.get()/ site behaves identically. _set_stream_state routes a reassign
through session.replace() so the store's session stays the live object (it's
effectively dead — prepare.py only mutates in place — but safe now).
Honest scope: this is the PROVABLE half of Phase 3. The remaining half (3b:
derive a per-browser session id, per-session Stream/ staging, executor
concurrency, disconnect cleanup) is browser-coupled and can't be verified
without driving 2+ live clients — deferred to a live session. The store API is
already shaped for it.
Tests (tests/streaming/, 33 total):
- test_stream_state_store.py (19): session dict-compat, isolation, lazy
create, drop rules, active_ids, concurrent-create race safety.
- test_stream_state_callsite_compat.py (7): every real web_server access
pattern (library/play, stream/start, status, audio guard, stop, prepare
in-place mutation, set->replace) against the exact object web_server binds.
- test_prepare.py +1: real prepare worker drives an actual StreamSession.
76 streaming+radio tests green; ruff clean; web_server.py parses.
Two related bugs from the Slipknot album never finishing.
1) _poll_album_bundle_downloads hung when the peer stalled. The finish check
needs every transfer terminal (completed/failed); the #715 grace only covers
'slskd says Completed but file not on disk'. A transfer stuck InProgress /
Queued, or dropped by slskd, is none of those — so it blocked both the finish
AND the grace exit, and the poll spun to the full ~6h timeout.
Add a bundle-level stall guard: track a progress marker (#terminal transfers,
total bytes across pending). If NOTHING advances for _stall_grace (180s) —
no terminal transition AND no pending byte movement — the peer has stalled;
mark the stuck transfers failed so the existing finish/all-failed checks
resolve the bundle with whatever completed (missing tracks then fall back to
the per-track matcher). Conservative: only trips when EVERYTHING is frozen,
so a slow-but-progressing or still-queued transfer is unaffected.
2) Failed batches lingered in the UI forever ('No tracks loaded'). The
auto-cleanup gate removed only complete/error/cancelled phases — 'failed'
(e.g. an album-bundle hard failure) was missing, so it never aged out. Add
'failed' to the terminal set so it's removed after 5 minutes like the others.
Tests (tests/test_soulseek_album_poll_stall.py): stalled peer → gives up with the
completed subset (not the deadline); progressing bundle not falsely stalled;
all-stalled → empty; dropped transfers stall out; clean finish unaffected.
124 download/soulseek tests pass; ruff clean.
Stage 2: the manual 'Download Wishlist' flow now calls the same
_run_wishlist_cycle engine the auto timer uses, so a manual scan runs the exact
same code path as an auto scan. The old bespoke manual orchestration (build
payloads + SERIAL inline dispatch) is deleted — its grouping/dispatch was a
near-duplicate of auto's that had already drifted.
Behavior changes (all intended, discussed):
- Manual now dispatches album bundles in PARALLEL (album pool) like auto, instead
of serially on one thread. A single cycle='albums' engine call covers the whole
selection (albums bundled, singles/ungroupable -> per-track residual), so no
'both cycles' pass is needed.
- The manual placeholder batch_id is reused as the engine's first sub-batch
(first_batch_id), so the modal's existing poll target stays valid.
- WishlistManualDownloadRuntime gains album_bundle_executor (wired in web_server,
falls back to the shared pool when unset).
- 'Don't start manual while auto is running' is unchanged — the existing route
guard (is_wishlist_actually_processing -> 409) already covers it; no queue added.
NOT touched: process_wishlist_automatically's behavior (proven by test_automation
staying green in Stage 1) and the per-track download mechanics.
test_manual_download.py rewritten to characterize the new behavior (engine
dispatch via the executor, parallel, placeholder reuse, album-context). Full
wishlist suite green (131); wishlist + automation = 392 passed.
A 2.6.3 change (c3b88e69) split the wishlist albums cycle into one batch per
album. Each album batch runs an INLINE-BLOCKING soulseek/torrent/usenet
album-bundle download (album_bundle_dispatch.try_dispatch ->
download_album_to_staging) that holds its worker thread for the whole
search+download. All of these were submitted to the shared 3-worker
missing_download_executor -- the same pool used for per-track downloads AND the
manual 'Download Wishlist' analysis.
So a large Album-Completeness 'Fix all' (-> ~819 wishlist tracks -> ~82 per-album
batches) saturated all 3 workers with blocking album downloads; the manual
wishlist analysis could never get a thread ('Library Analysis' stuck on
Pending), the other ~79 batches sat in phase='queued' forever, and auto-cleanup
(which only evicts terminal-phase batches) never cleared them -> jam until
restart. Fixing batch STATUS would not help: the threads are blocked inside the
download, not waiting on a phase flip.
Fix: add a dedicated bounded album_bundle_executor (max_workers=3) and route the
AUTO per-album bundle batches to it, keeping the shared pool free for analysis /
per-track / the manual wishlist (which always starts now). Hung/slow album
downloads can only delay other album downloads, never the user-facing path.
Additive and decoupled; the submit site falls back to the shared pool when the
album pool isn't wired (older callers / tests) so behavior is unchanged there.
The manual path is untouched (it already runs album bundles serially on a single
thread, by design).
Tests (tests/wishlist/test_automation.py): album sub-batches route to the
dedicated pool while the residual per-track batch stays on the shared pool;
fallback-to-shared-pool when no album pool is wired. Existing auto-processing
tests still green (fallback preserves prior behavior). 707 passed across
wishlist + downloads suites.
The same provider ID is stored under inconsistent column names across tables
(deezer_id vs deezer_artist_id vs album_deezer_id vs similar_artist_deezer_id;
spotify/itunes keep an entity qualifier, others don't; musicbrainz uses three
nouns), so code checks 2-5 name variants everywhere.
Add core/source_ids.py as the single source of truth for (provider, entity) ->
column, with accessors that read an ID from a dict/sqlite3.Row robustly
(canonical column first, then known aliases). NO database columns are renamed —
these are the real names today; the registry just centralizes the knowledge.
Targeted adoption (behavior-identical, verified):
- core/artist_source_lookup.SOURCE_ID_FIELD now derives from the registry
instead of duplicating the mapping (values unchanged).
- web_server.py artist-detail builds artist_source_ids via source_id_map(...)
instead of a hand-rolled per-source .get() dict.
Broader call-site adoption deferred as clearly-scoped follow-up.
Tests: tests/test_source_ids_registry.py (canonical columns, alias fallback,
canonical-preferred, sqlite3.Row, source_id_map, SOURCE_ID_FIELD unchanged).
Existing artist_source_lookup + artist_full_detail suites still green.
Reported by @Yug1900: a Spotify playlist's overview shows the correct count
(e.g. 203) but the sync modal lists only 100, and only 100 would sync.
Root cause in /api/spotify/playlist/<id> (the Spotify-tab track fetch): owned
playlists fetch + paginate fine (sp.next loops over all pages). But when the
authenticated playlist_items call FAILS (intermittently, often a 403 on
followed / not-owned playlists) it fell straight back to the public embed
scraper, which is hard-capped at ~100 tracks and has no pagination — and the
result was returned as if complete. The overview total is correct because it
comes from the playlist *metadata* listing, which succeeds independently.
Fixes (additive — the owned/working path is unchanged):
- Retry the items fetch once before resorting to the embed scraper, so a
transient failure no longer silently truncates a large playlist.
- Guard against silent truncation: compare fetched count against the
playlist's known total; if short (or via the capped embed fallback), log a
warning and return `incomplete: true` + `expected_total` instead of
presenting a partial list as complete.
No behavior change when the fetch succeeds in full (incomplete=false, extra
fields ignored by the frontend). Lets a future UI tweak surface
"got 100 of 203 — retry" rather than silently dropping tracks.
Final cluster: the four structurally-identical snapshot endpoints
(discover_downloads, artist_bubbles, search_bubbles, beatport_bubbles) ->
core.discovery.endpoints.save_bubble_snapshot(...), wired via
_save_source_bubble_snapshot. All four validate a payload key, persist via
db.save_bubble_snapshot(kind, items, profile_id=...), and return a count +
timestamp; they differ only by:
- payload_key ('downloads' for discover, 'bubbles' for the rest) + its
no_data_error message.
- snapshot_kind, success_noun, and the info/except log subject + noun
("downloads"/"artists"/"albums/tracks"/"charts").
get_database / get_current_profile_id injected; get_json (request.json) invoked
inside the try, preserving the original 400/500 behavior incl. traceback dump.
Tests: +5 (missing key 400, None body 400, happy path with kind/profile/count/
timestamp, discover_downloads variant, exception -> 500). Full discovery suite:
210 passed.
web_server.py: -98 lines.
Ninth cluster: update_<source>_playlist_phase for the five sources sharing the
identical validation + full-message response (Tidal, Deezer, Qobuz,
Spotify-Public, YouTube) -> core.discovery.endpoints.update_playlist_phase(...),
wired via _update_source_playlist_phase + the _PHASE_LIST/_PHASE_LIST_YT
constants.
Per-source params:
- valid_phases — YouTube additionally allows 'parsed'.
- apply_extra_fields — Deezer/Qobuz/Spotify-Public also persist
download_process_id / converted_spotify_playlist_id from the body; Tidal and
YouTube do NOT, so they pass False (kept strictly 1:1 — the generic won't
apply those keys for them even if a caller sent them).
- not_found_message / error_label; get_json invoked inside the try.
NOT folded in: iTunes-Link — uses data.get('phase') (no "Phase not provided"
400) and returns a no-message payload.
Tests: +7 (404, missing-phase 400, invalid 400, happy path with extra-fields
suppressed, extra-fields applied when enabled, YouTube 'parsed' allowed,
exception -> 500). Full discovery suite: 205 passed.
web_server.py: -123 lines.
Eighth cluster, the heavyweights (~110 lines each). The fix-modal
update_<source>_discovery_match for the four sources with the identical
structure (Tidal, Deezer, Qobuz, Spotify-Public) ->
core.discovery.endpoints.update_discovery_match(...), wired via
_update_source_discovery_match. Applies the user-selected Spotify track to the
discovery result (status/artist/album/duration/spotify_data/match-count) and
writes the manual fix to the discovery cache.
Per-source pieces are params:
- source_log_label / error_label.
- original_track_key ('tidal_track' / 'deezer_track' / ...).
- original_artist_getter: Tidal handles string-or-object artists
(first_artist_str_or_obj); the rest assume strings (first_artist_plain).
- web_server helpers (join/extract artist, build_fix_modal_spotify_data,
cache-key, get_database, active-discovery-source) injected.
- get_json passed as a callable and invoked INSIDE the try, preserving the
original's "request.get_json() inside try" behavior (malformed body -> 500).
NOT folded in (genuinely divergent): iTunes-Link (saves spotify_data directly
via a different cache signature), YouTube (multi-key original_track fallback),
ListenBrainz (entirely different unmatch-capable structure, no cache write),
Beatport.
Tests: +9 (extractors; 400/404/400 guards; full happy path with result
mutation + duration formatting + match-count + cache-save args; no-increment
when already found; cache error swallowed; get_json raise -> 500). Full
discovery suite: 198 passed.
web_server.py: -400 lines.
Seventh cluster: start_<source>_sync for the five sources with the identical
flow (Tidal, Deezer, Qobuz, Spotify-Public, YouTube) ->
core.discovery.endpoints.start_sync(...), wired via _start_source_sync.
Validates phase, converts discovery results, seeds sync state, posts a
"... Sync Started" activity item, and submits to the sync executor. Per-source
pieces are params:
- sync_id_prefix (f"{prefix}_{key}"), not_found/not_ready messages, convert_fn.
- name/image accessors: Tidal reads an object (playlist_name_obj/
playlist_image_obj), the rest a dict (playlist_name_strict/playlist_image_dict).
- activity_label vs error_label DIFFER for Spotify-Public ("Spotify Link
Sync Started" activity, "Spotify Public" logs).
- submit_sync_task glue (_submit_sync_task) closes over sync_executor /
_run_sync_task / get_current_profile_id so the helper stays global-free.
NOT folded in: iTunes-Link (no final info log), ListenBrainz (submits the
task WITHOUT a playlist_image_url arg), Beatport (extra debug logging, chart).
Tests: +6 (404, not-ready 400, no-matches 400, full happy path with
state/sync-infra/submit/activity assertions, resync phases allowed,
exception -> 500). Full discovery suite: 189 passed.
web_server.py: -172 lines.
Sixth cluster: the bulk-hydration get_<source>_playlist_states endpoints for
the five sources that build the identical per-entry dict + {"states": [...]}
shape (Tidal, Deezer, Qobuz, Spotify-Public, iTunes-Link) ->
core.discovery.endpoints.get_playlist_states(states, *, error_label,
info_log_label=None), wired via _get_source_playlist_states.
iTunes-Link is the only one of the five without the "Returning N stored ..."
info log, so info_log_label is optional (iTunes passes None to suppress it).
NOT folded in: the YouTube/ListenBrainz get_all_*_playlists endpoints. They
return {"playlists": [...]} (different key) with a different field set
(url / created_at / playlist, no discovery_results) and filter out
mirrored_/profile-scoped entries — genuinely divergent, kept as-is.
Tests: +4 (list build + last_accessed bump + exact shape, empty, optional ids
default None, missing-required-field -> 500). Full discovery suite: 183 passed.
web_server.py: -116 lines.
Fifth cluster: reset_<source>_playlist for the four sources with byte-
identical bodies (Tidal, Deezer, Qobuz, Spotify-Public) ->
core.discovery.endpoints.reset_playlist(states, key, *, label,
not_found_message), wired via _reset_source_playlist. Resets phase/status to
'fresh', clears discovery/sync fields, cancels any discovery_future, and
preserves the original playlist payload.
Left with their own bodies (genuinely divergent):
- YouTube: status -> 'parsed' (not 'fresh'), no download_process_id, logs the
playlist name, "reset to fresh state".
- ListenBrainz: status -> 'cached', logs playlist title, returns
{"success": True, "phase": "fresh"} (different payload), _lb_state_key.
- iTunes-Link: state.update(...), no info log, "iTunes Link reset to fresh
phase".
Tests: +4 (404, full clear + playlist preserved + future cancelled, no-future
path, exception -> 500). Full discovery suite: 179 passed.
web_server.py: -100 lines.
Fourth cluster: get_<source>_discovery_status (all eight sources, Beatport
included) -> core.discovery.endpoints.get_discovery_status(states, key, *,
not_found_message, error_label), wired via _get_source_discovery_status.
Unlike sync-status, the discovery-status response shape is byte-identical
across every source (phase/status/progress/spotify_matches/spotify_total/
results/complete), so Beatport folds in here too. Only the 404 string
("... discovery not found" vs "... playlist not found" vs "Beatport chart
not found") and the except-log label vary. ListenBrainz key via _lb_state_key.
NOT touched this cluster: get_*_playlist_state (the sibling endpoints).
Those genuinely diverge per source — different id-key name (playlist_id /
url_hash / playlist_mbid), presence of url / created_at / download_process_id,
Tidal's playlist.__dict__ serialization, and YouTube's strict (non-.get)
field access. Folding them would need a flag pile that wouldn't be a clean
1:1, so they keep their own bodies.
Tests: +4 (404, full response + last_accessed bump, complete=False when not
'discovered', missing-field -> 500). Full discovery suite: 175 passed.
web_server.py: -155 lines.
Third cluster: the get_<source>_sync_status routes (Tidal, Deezer, Qobuz,
Spotify-Public, iTunes-Link, YouTube, ListenBrainz) -> core.discovery.
endpoints.get_sync_status(...), wired via _get_source_sync_status glue.
This cluster carries the real per-source quirks, all captured 1:1 as params:
- not_found_message (iTunes-Link uses "iTunes Link not found").
- error_label vs activity_subject — these DIFFER for Spotify-Public: the
activity feed says "Spotify Link playlist ..." while the except log says
"Error getting Spotify Public sync status".
- playlist-name accessor, three styles lifted verbatim as named helpers:
playlist_name_attr_or_unknown (Tidal: object .name), playlist_name_strict
(Deezer/Qobuz/Spotify-Public/iTunes: state['playlist']['name'], can raise),
playlist_name_safe (YouTube/ListenBrainz: .get default). The strict getter
preserves the original's behavior of raising -> 500 AFTER phase/sync_progress
were already mutated.
- ListenBrainz key via _lb_state_key (caller-resolved).
Beatport stays separate (different payload: status not sync_status, sync_id,
no lock, chart key).
Tests: +9 (3 name accessors incl. raise/fallback semantics; status 404s,
running-no-mutation, finished+activity, error+revert+activity, and strict-
getter-missing -> 500 after partial mutation). Full discovery suite: 171 passed.
web_server.py: -244 lines.
First cluster of the per-source playlist-discovery deduplication. The
convert_<source>_results_to_spotify_tracks functions (Tidal, Deezer, Qobuz,
Spotify-Public, YouTube, ListenBrainz) plus the already-generic
_convert_link_results_to_spotify_tracks were byte-identical apart from the
source label used in their log line.
Lift the shared body into core/discovery/endpoints.py as
convert_results_to_spotify_tracks(results, source_label); the 7 web_server
functions become 1-line delegations (names/signatures unchanged, so all
callers and behavior are identical — 1:1).
Beatport is intentionally NOT folded in: its converter coerces artist
objects to strings and emits a different track shape (source field, album
dict), so it keeps its own implementation.
Tests: tests/discovery/test_discovery_endpoints.py (12) pin both input
shapes (manual spotify_data / auto spotify_track+found), optional
track/disc numbers, falsy-0 omission, field defaults, skip-on-neither,
order preservation, if/elif precedence, empty input.
web_server.py: -209 lines. Full discovery suite: 151 passed.
Two things in this commit. Functional download / matched-download
behaviour is untouched — same JS handlers, same routes for the
download actions, same album-expand interaction.
VISUAL REDESIGN
- Glass search-bar card with accent radial wash + focus ring + pill
primary search button
- Source chip row above the search bar (see below)
- Always-visible compact filter pill row (Type / Format / Sort) —
pills carry both ``bs-filter-pill`` (new visual) and ``filter-btn``
(legacy class for ``resetFilters`` + ``applyFiltersAndSort`` in
wishlist-tools.js to keep working)
- Accent-tinted status pill matching the dashboard / auto-sync look
- Album result cards: glass card with accent left-edge stripe,
52px brand-tinted cover icon, chevron expand indicator, pill
action buttons (Download / Matched Album), accent glow on hover
- Track result cards: glass row with accent stripe, 44px icon,
pill action buttons (Stream / Download / Matched Download)
- Multi-disc separators inside expanded album track lists styled
with the accent treatment
- Responsive: action button columns stack vertically below 900px
New CSS lives in a self-contained ``webui/static/basic-search-v2.css``
sheet linked from index.html. Selectors are scoped to
``#basic-search-section`` for any class that already exists in
style.css (``.album-result-card``, ``.album-icon``, ``.track-*``,
etc.); the new ``bs-*`` prefixed classes for the search bar /
filters / source row / status are unscoped because they only exist
in the new markup. ``!important`` is used on the card-level rules
to defeat the original unscoped ``.album-result-card`` etc. rules
in style.css that would otherwise leak heavyweight padding /
box-shadow / 56px icon styles into the new design.
Also removed ``overflow: hidden`` from the original
``.album-result-card`` and ``.track-result-card`` rules in style.css
— those two classes only render in ``downloads.js`` basic search
results (verified via grep, two render sites only), so the
removal can't impact any other UI.
SOURCE PICKER (hybrid mode)
- New ``GET /api/search/sources`` endpoint returns the list of
active sources from the orchestrator's chain (or the single
active source in single-source mode).
- Frontend renders a chip row above the search bar. Click a chip
to target that source for the next search; the chip's brand
accent fills.
- In single-source mode the lone chip is rendered as a dashed-
border label so the user always knows what they're searching
but can't accidentally try to switch to sources that aren't
configured.
- ``/api/search`` accepts an optional ``source`` body param. When
set, ``core/search/basic.py:run_basic_search`` resolves the
client directly via ``orchestrator.client(source)`` and calls
its ``.search()`` instead of going through the hybrid chain.
- Backwards compatible: omitting ``source`` falls through to the
original ``orchestrator.search()`` call exactly as before.
Unknown source names also fall back to the default — typo
protection.
TESTS (5 new + 6 pre-existing = 11 total in test_search_basic.py)
- source param routes to specific client, NOT orchestrator chain
- no source param preserves original orchestrator-default behaviour
- unknown source name falls back to orchestrator default
- ``run_basic_soulseek_search`` backwards-compat alias preserved
- source-targeted path serialises albums + tracks correctly
101 search-suite tests pass.
- _SOULSYNC_BASE_VERSION → 2.6.4
- helper.js: '2.6.4' unreleased → 'May 28, 2026 — 2.6.4 release'
- .github/workflows/docker-publish.yml default version_tag → 2.6.4
- pr_description.md: rewrite for 2.6.4 with #721 as the headline patch,
2.6.3 fixes carried forward unchanged (2.6.3 was bumped on dev but
never reached main / docker, so 2.6.4 is the first release to ship
this batch)
- _SOULSYNC_BASE_VERSION → 2.6.3
- helper.js WHATS_NEW unreleased flag → 'May 27, 2026 — 2.6.3 release'
Note: .github/workflows/docker-publish.yml default version_tag was
also bumped to 2.6.3 locally, but .github is gitignored in this
repo — workflow updates need to land via the GitHub UI separately.
The workflow_dispatch input is overrideable at trigger time
regardless of the default, so this isn't blocking.
Two related leaks in ``storage/album_bundle_staging/<batch_id>/``:
1. **Soulseek bundle cleanup was excluded.** The per-batch cleanup
at the end of a bundle download gated on:
(album_bundle_source or '').lower() in ('torrent', 'usenet')
The comment justified it as "slskd keeps its own completed
folders" — but the Soulseek bundle path ALSO copies completed
files into the private staging dir (``soulseek_client.py:1599``,
``copy_audio_files_atomically(completed, Path(staging_dir))``)
for the per-track workers to claim. Those copies persisted
forever; long-running installs accumulated stale GB. Extended
the cleanup gate's allow-list to include ``soulseek`` so the
per-batch dir is removed on bundle completion — same code path
that already worked for torrent / usenet.
2. **No sweep for orphan dirs.** Any leftover ``<batch_id>``
subdir from a previous-session crash, an errored batch, or a
pre-fix Soulseek bundle stayed on disk forever. Added
``sweep_orphan_album_bundle_staging(staging_root, active_batch_ids)``
that runs ONCE at server startup, before any batch can register
a staging dir. Removes every ``<batch_id>``-shaped subdir
whose id isn't in the active set. Safe by construction:
- Only touches subdirs of the configured staging root.
- Name-shape check (``entry.name == _safe_batch_dirname(entry.name)``)
rejects hand-placed dirs like ``.git`` or stray docs.
- ``shutil.rmtree`` errors log + continue — sweep must not
crash app startup over a permission glitch.
- active_batch_ids normalised through ``_safe_batch_dirname``
so colon-bearing batch_ids match their on-disk form.
Wired into the web_server startup right after the stuck-flags
diagnostic so it fires before anything else touches batches.
Tests
- ``test_downloads_lifecycle.py`` gained one regression test
pinning that Soulseek bundles now have their staging dir
cleaned (sibling to the existing torrent test).
- ``test_album_bundle_staging_sweep.py`` (NEW, 11 tests)
covers: orphan removal with no actives, active dirs preserved,
special-char batch_id normalisation, no-op on missing /empty
/empty-string staging root, non-dir entries skipped, unsafe-
name dirs preserved (.git etc.), partial rmtree failure doesn't
abort the rest, listdir failure returns 0 cleanly, default
None active set, defensive against empty / None entries in
the active set.
488 downloads tests pass.
For users with an existing "clean up old files" automation pointed
at this dir: stop pointing it there if you want — the auto-cleanup
+ startup sweep cover it now. Or leave it as belt-and-suspenders
with a relaxed (1h+) mtime threshold so it can't race a mid-batch
download.
finished the release (#715)
Symptom (user @pavelcreates / @IamGroot60 on 2.6.2):
- Click Download on an album in the search modal
- slskd starts + completes every track of the release
- 22+ minutes after the last completed download, batch flips
to "failed" with no clear log line explaining why
- Per-track Soulseek downloads on the same machine were fine
Root cause: ``core/soulseek_client._resolve_downloaded_album_file``
probed three hard-coded candidate paths to locate each downloaded
file in the slskd download dir:
candidates = [
download_path / remote_filename,
download_path / basename,
download_path / *normalized_path_parts,
]
On the common slskd config ``directories.downloads.username = true``
slskd writes files at ``<download_dir>/<username>/<filename>`` —
none of the three candidates carry a username segment, so the
resolver returned None for every file even though the file was
physically present in a subdir one level deeper. ``_poll_album
_bundle_downloads`` saw 0 completed_paths, kept spinning, and
hit the master deadline (~30 min) before bailing the batch.
Why per-track worked: ``web_server._find_completed_file_robust``
already does a recursive walk-by-basename + path-confirm against
the remote directory components, so any layout slskd writes ends
up resolved. The bundle path didn't go through it.
Fix
- Lifted the robust finder into ``core/downloads/file_finder.py``
as a pure function ``find_completed_audio_file(download_dir,
api_filename, transfer_dir=None) -> (path, location)``. Zero
globals; recursive walk; handles slskd dedup suffix
``_<10+digit-timestamp>``, YouTube / Tidal ``id||title`` encoded
filenames, the AcoustID-quarantine subdir skip, basename
collisions disambiguated by remote-path components, and a
fuzzy-basename fallback above 0.85.
- ``_resolve_downloaded_album_file`` keeps the three-candidate
fast path (cheap probe for the slskd-flat default) but now
delegates to the new helper when none hit, instead of giving up.
- ``_poll_album_bundle_downloads`` tracks "slskd reports
Completed but local resolver returns None" per key. When every
remaining key has been in that state past a 45-second grace
window, the poll exits early with an explicit error pointing at
the likely ``soulseek.download_path`` mismatch instead of
silently spinning until the master deadline.
- ``web_server._find_completed_file_robust`` becomes a thin
delegate so both callers share one finder. Legacy inline impl
kept as ``_find_completed_file_robust_legacy`` for reference;
to be removed next release.
- Fixed misleading ``"(0 tracks, quality=)"`` log on the preflight-
reuse path — was reading attrs off a None ``picked`` object.
Tests (17 new in tests/downloads/test_file_finder.py)
- Flat slskd layout
- Username-prefixed (the #715 case)
- Full remote tree preserved
- Deeply nested username + tree
- File genuinely missing returns None
- Basename collision disambiguated by remote dirs
- Single basename match wins regardless of dirs
- slskd dedup suffix match
- Short ``_<digits>`` (year) not treated as dedup
- AcoustID quarantine subdir skipped
- YouTube / Tidal ``id||title`` encoded filenames
- transfer_dir fallback
- Both dirs miss → (None, None)
- Non-audio files ignored
- Empty api_filename
- Fuzzy match on punctuation variant
- Fuzzy rejects below threshold
475 downloads tests pass after the lift.
Five pre-existing lint errors on dev baseline (all introduced May 25-26
before this branch was cut) were blocking CI on this PR. Cleared as
courtesy fixes so the merge isn't gated on unrelated tech debt:
- web_server.py:22613 — F811 duplicate `urlparse` import inside
`_parse_itunes_link_url` (already imported at module top, line 20).
Removed from the inline `from urllib.parse import parse_qs, urlparse`;
kept `parse_qs` since that one is only used here.
- core/listenbrainz_manager.py:746 — S110 silenced with
`# noqa: S110 — best-effort lookup, delete proceeds either way`.
Matches the existing project convention used in web_server.py:1693,
core/watchlist/auto_scan.py:463, core/library_reorganize.py:548.
- core/playlists/sources/listenbrainz.py:236 — B905 `zip()` without
explicit `strict=`. Added `strict=False` — preserves existing
behaviour where `matched` can legitimately be shorter than
`match_indices` on partial discover failure.
- core/playlists/sources/listenbrainz.py:273 — S110 silenced with
`# noqa: S110 — caller falls back to last cached playlist on
refresh failure`.
- core/playlists/sources/soulsync_discovery.py:105 — S110 silenced
with `# noqa: S110 — manager persists last_generation_error on
failure; surface existing snapshot`. The existing multi-line
comment that already explained the swallow was rolled into the
noqa justification so the rule + reason live on one line.
Ruff `python -m ruff check .` now passes; 664 discovery + metadata
tests still pass.
Self-review pass on the prior three commits — kettui-style cleanup
that should have landed first time.
**Length-preference sort ordering (real bug):**
The `search_tracks_with_artist` stable sort that promoted length-known
recordings ran in `core/musicbrainz_search.py`, but the MB endpoint in
`web_server.py:search_musicbrainz_tracks` runs `rerank_tracks` after
it — which re-sorts by relevance score and dropped the length-pref
ordering down to tiebreaker-only. For canonical-same-song MB duplicates
that all score identically the tiebreaker survived, but the
order-of-operations was wrong.
Moved into `rerank_tracks` itself via a new `prefer_known_duration`
flag. Sort key sits between relevance score and the stable-order
tiebreaker so relevance still wins (length only decides ties, never
overrides a higher-relevance match). The MB endpoint opts in via
`prefer_known_duration=True`; Spotify / iTunes / Deezer callers stay
on the default-off path since their search results always include
length. Pinned with three new `TestRerankTracks` cases:
ties-promote-length, relevance-still-wins, default-off-unchanged.
**Route logic lifted to `core/discovery/manual_match.py`:**
Two pieces lived as inline route logic in `web_server.py` — the
`derive_manual_match_provider` fallback chain (payload.source →
active source → 'spotify') used by `update_youtube_discovery_match`,
and the `is_drifted_for_redo` predicate (cached provider differs from
active AND not manual_match) used by `prepare_mirrored_discovery`.
Per kettui's "extract logic from web_server.py, don't AST-parse it"
standard, both helpers now live in `core/discovery/manual_match.py`
with 12 dedicated unit tests covering fallback resolution order,
non-dict payload defenses, manual_match exemption from drift,
absent-provider legacy default, and edge cases.
Side benefits from the lift:
- `match_source` now derived once before the cache-save try block
instead of being duplicated in try + except (the except block existed
only because the original used `match_source` later — pre-computing
killed the duplication).
- `prepare_mirrored_discovery`'s `has_cached` check now reuses
`is_drifted_for_redo` with inverted polarity instead of restating
the field whitelist inline, so a future schema change only has to
land in one place.
- The mirrored-DB persist block now gates on `matched_data is not None`
to avoid a pre-existing latent NameError if the cache-save block
raised before matched_data construction.
**Enhanced toggle localStorage key now profile-scoped:**
`soulsync-library-view-mode` was global — two admin profiles would
share one preference. Wrapped in `_libraryViewModeKey()` which appends
`:${currentProfile.id}` when a profile is loaded, falls back to the
unsuffixed key otherwise (preserves pre-multi-profile saved values).
Tests:
- 12 new in `tests/discovery/test_manual_match.py` pinning both helpers.
- 3 new in `tests/metadata/test_relevance.py` pinning the
`prefer_known_duration` semantics.
- `test_search_tracks_with_artist_prefers_results_with_known_length`
renamed to `_does_not_resort_by_length` since the sort moved out of
this method. 664 tests pass across discovery + metadata suites.
User reported that manually mapping a mirrored-playlist track via the
Fix popup (either by search or by pasting an MBID) worked end-to-end
once — match saved, library track downloaded — but the next Playlist
Pipeline run flipped the track back to "Provider Changed" and forced
them to re-do the manual map every cycle.
Three independent issues were combining to cause this:
1. Hardcoded `provider: 'spotify'` on manual-fix save
`update_youtube_discovery_match` (the endpoint the Fix popup posts
to, also used by mirrored playlists since the frontend routes
`platform === 'mirrored'` through the YouTube endpoint) always
stamped the cached match as Spotify-provided. The Fix-popup cascade
actually queries the user's primary metadata source first and falls
back to Spotify / Deezer / iTunes / MusicBrainz — so a user on
MusicBrainz primary picking an MB result still had it saved as
`provider: 'spotify'`. The next prepare-discovery call (which
compares cached_provider to the active source) then immediately
classified the match as drifted and pending re-discovery. Fixed by
deriving `match_source` from `spotify_track.get('source')` (every
*_search_tracks endpoint stamps `source` on results) with a fallback
to `_get_active_discovery_source()` for the MBID-paste path (which
uses the lean flat shape that doesn't carry source). `matched_data['source']`
and the mirrored `extra_data['provider']` both now use the derived
value. `match_source` is also recomputed in the cache-save except
handler so the downstream mirrored-DB save still has it.
2. Discovery worker re-queueing manual matches as "incomplete"
`run_playlist_discovery_worker` in `core/discovery/playlist.py`
re-adds any track to `undiscovered_tracks` when its `matched_data`
lacks `track_number` or `album.id` / `album.release_date`. The
check was designed as a legacy-fix backfill for old discoveries
that lost those fields to a Track-dataclass stripping bug. But
manual fixes from the popup are *intentionally* lean — search-
result rows don't include `track_number` (none of the search
endpoints return it), and the MBID-lookup flat shape doesn't
carry `album.id` / `release_date` (the recording lookup returns
only `album.name`). So every manual match looked "incomplete" and
got re-discovered every pipeline run, overwriting the user's pick
with whatever the auto-search ranked first. Manual matches now
short-circuit ahead of the incomplete-data branch.
3. `prepare_mirrored_discovery` ignored the `manual_match` flag
Independent of the provider-stamping fix above, the prepare-
discovery endpoint that powers the mirrored-playlist UI did its
own `cached_provider != current_provider` check and didn't honour
manual_match either. Defence in depth — even if a future code
path stamps the wrong provider on a manual match, the flag now
anchors it as cached. `has_cached` also extended so manual
matches with off-provider stamps still count toward the cached
tally for phase classification.
Tests:
- new `test_manual_match_skipped_even_when_matched_data_incomplete`
in `tests/discovery/test_discovery_playlist.py` pins the worker
short-circuit using a realistic MB-shape matched_data (album dict
without id / release_date, no top-level track_number). 16 existing
tests still green; 848 across discovery / metadata / automation
suites pass.
ListenBrainz publishes "Weekly Jams for X" / "Weekly Exploration
for X" with a fresh MBID every week, and "Top Discoveries of YYYY
for X" / "Top Missed Recordings of YYYY for X" with a fresh MBID
every year. Auto-mirroring those per-period yielded one mirrored-
playlist row per week/year — useless for Auto-Sync schedules
because the underlying LB playlist never updates, only a brand new
playlist replaces it. The user accumulates 100+ dead Weekly Jams
rows per year if they discover regularly.
This commit collapses each family into a single ROLLING mirror
keyed by a synthetic ``source_playlist_id`` (e.g.
``lb_weekly_jams_Nezreka``). Each new period UPSERTs into the same
row, so the user gets one stable Auto-Sync schedule per series
that automatically picks up the latest period's tracks on every
refresh. Non-series LB playlists (user-created, collaborative,
Last.fm radios for a specific seed) continue to mirror under
their per-playlist MBID as before. Per-period LB playlists are
still visible + usable on the LB Sync tab — only the mirror layer
collapses.
- ``core/playlists/lb_series.py`` (new) — series-detect helper
with regex patterns + canonical-name + LIKE-pattern template
for each known LB family. Exposes
``detect_series(title)``, ``is_series_synthetic_id(id)``, and
``list_series_synthetic_ids()`` so both the JS auto-mirror hook
and the LB adapter can speak the same language.
- ``GET /api/listenbrainz/series-detect?title=...`` — thin HTTP
shim around ``detect_series`` so the auto-mirror JS doesn't
duplicate the regex.
- ``ListenBrainzPlaylistSource.get_playlist`` now recognizes
synthetic series ids — it queries the LB cache for the newest
cache row whose title matches the series' LIKE pattern and
resolves to that row's MBID before fetching tracks. The mirror's
meta keeps the synthetic id so refreshes always re-resolve to
the latest period.
- ``_mirrorListenBrainzAfterDiscovery`` (sync-services.js) calls
the new detect endpoint when discovery completes — if a match
comes back it swaps the per-period MBID for the synthetic id +
the canonical name. Existing Last.fm radio routing logic stays
intact (Last.fm radios aren't a series).
- ``ListenBrainzManager._cleanup_per_period_series_mirrors`` —
one-shot consolidation sweeper runs in ``_cleanup_old_playlists``
+ deletes any legacy per-period mirror rows so the consolidated
rolling mirror is the only one left. Idempotent — only matches
per-period titles ("Weekly Jams for ..., week of ...") and never
the canonical rolling-mirror titles ("ListenBrainz Weekly
Jams").
- 11 new tests pin the detector + synthetic-id helpers; 236 total
across adapter + automation + lb-series suites green.
Sibling to the ListenBrainz Sync tab from Phase 1c.1. Last.fm Radio
playlists already live in the same ``listenbrainz_playlists`` table
as LB ones (``playlist_type='lastfm_radio'``) and run through the
same MB-track discovery worker, so this tab is intentionally thin
— list + render + delegate. Card click hands straight off to the
LB Sync-tab click handler since the downstream modal + state
machine are identical.
- ``webui/index.html``: new ``<button data-tab="lastfm-sync">``
+ tab content container between the LB tab and the existing
Import / Mirrored tabs. Plus a ``<script>`` tag for the new
module.
- ``webui/static/sync-lastfm.js`` (new): ``loadLastfmSyncPlaylists``
hits the existing ``/api/discover/listenbrainz/lastfm-radio``
endpoint, ``renderLastfmSyncPlaylists`` mirrors the LB card
shape with a ``📻`` icon + a ``.lastfm-playlist-card`` brand
class, click handler forwards to
``handleListenBrainzSyncCardClick``.
- ``webui/static/sync-listenbrainz.js``: the shared 500ms refresh
loop now iterates LB + Last.fm cards in one pass and treats
either tab as "active" for liveness. No second loop needed.
- ``webui/static/sync-services.js``: new tab-activation branch in
``initializeSyncPage`` mirrors the LB pattern.
- ``webui/static/style.css``: ``.lastfm-icon`` SVG (Last.fm "as"
logo, red), and ``.lastfm-playlist-card`` joins the unified
card selector group with the Last.fm-red accent
(``rgba(213, 16, 7, ...)``).
- ``web_server.py``: the lastfm-radio endpoint now includes
``track_count`` in its JSPF payload (same fix as the LB
endpoints last commit).
- WHATS_NEW entry added under 2.6.3.
Mirrors created from Last.fm radios participate in the same auto-
trim Phase 1c.1's cascade-delete hook does — when the LB manager
rotates a stale ``lastfm_radio`` row out of its 5-most-recent
window, the matching ``source='lastfm'`` mirror row is removed
along with it. Library files stay on disk.
225 tests across adapter + automation suites still green; this
commit adds no Python paths to test.
Two follow-ups to the LB Sync tab work:
1. **Track counts all showed 0.** The
``/api/discover/listenbrainz/*`` endpoints assemble a JSPF-shaped
payload but drop the cached ``track_count`` field from the
underlying ``listenbrainz_playlists`` row — the JSON the frontend
sees only carries ``title`` / ``creator`` / ``annotation`` / an
empty ``track`` array. The Discover-page renderer worked around
it by hard-coding a fallback of 50; the Sync-page renderer had
no such fallback, so every card displayed "0 tracks". Backend
now includes ``track_count`` directly in each playlist payload
(it's already in the cached row) so any frontend can render an
accurate count without resorting to a default. JS still falls
back to ``annotation.track_count`` and then ``track.length`` for
older callers.
2. **LB playlists never landed in Mirrored Playlists.** The
existing ``/api/listenbrainz/sync/start/<mbid>`` endpoint runs
the converted Spotify tracks through ``_run_sync_task`` — i.e.
it pushes them to the user's media server (Plex / Jellyfin /
Navidrome / SoulSync) as a server-side playlist. It does NOT
call ``database.mirror_playlist``. So no ``mirrored_playlists``
row gets created and the playlist can't be picked up by the
Auto-Sync scheduler, can't show up under the Mirrored tab,
doesn't participate in pipeline automations — the whole point
of the Sync-tab unification.
Tidal works because Tidal mirrors on tab load with raw tracks
then enriches via discovery. LB tracks only have provider IDs
*after* discovery, so the equivalent moment for LB is "discovery
complete". Added ``_mirrorListenBrainzAfterDiscovery(mbid)``
that pulls the matched ``spotify_data`` out of
``discovery_results`` and posts to ``/api/mirror-playlist`` via
the existing ``mirrorPlaylist`` helper. Hooked into both the
WebSocket and HTTP-poll completion handlers of
``startListenBrainzDiscoveryPolling``. UPSERT-keyed on (source,
source_playlist_id, profile_id), so re-running discovery is a
safe no-op refresh.
Result: any LB playlist the user discovers (from either the
Discover page or the new Sync tab) now lands in
``mirrored_playlists`` with ``source='listenbrainz'`` + matched
tracks carrying canonical ``extra_data`` JSON, ready for the
Auto-Sync refresh + sync pipeline wired up in Phase 1a + 1b.
Adds ``discover_tracks(tracks) -> List[NormalizedTrack]`` to the
PlaylistSource interface. Sources whose tracks already carry
provider IDs (Spotify, Tidal, Qobuz, YouTube, Deezer, Spotify
public, iTunes link, SoulSync Discovery) inherit a no-op default;
ListenBrainz + Last.fm override to run the matching engine.
This closes the last gap before LB / Last.fm / SoulSync Discovery
can land as Sync-page mirror sources: the refresh handler now
calls ``source.discover_tracks(...)`` whenever a source returns
tracks with ``needs_discovery=True``, so mirrored LB rows arrive
already discovered + ready for the sync pipeline. Previously, LB
playlists ran through a separate state-machine worker tied to the
Discover-page UI, with results stored in ``discovery_cache``
instead of ``mirrored_playlist_tracks.extra_data``.
Changes:
- ``core/playlists/sources/base.py`` — PlaylistSource switches from
Protocol to ABC so a concrete default for ``discover_tracks``
can live on the base class. The four real-work methods stay
``@abstractmethod``; instantiating an adapter that forgets one
fails loudly at construction.
- ``core/discovery/matching.py`` (new) — pure ``match_mb_tracks``
helper that runs Strategy-1-only matching-engine queries against
Spotify (primary) or iTunes (fallback). No state machine, no
discovery-cache writes, no wing-it stub — that richer flow stays
in ``core/discovery/listenbrainz.py`` for the Discover-page UI.
- ``ListenBrainzPlaylistSource`` + ``LastFMPlaylistSource`` take
an optional ``discover_callable`` constructor arg. Last.fm reuses
the LB implementation since the track shape is identical.
- ``bootstrap.build_playlist_source_registry`` accepts a
``discover_callable`` kwarg and wires it into LB + Last.fm
adapters.
- ``web_server.py`` boot constructs the discovery callable from the
existing matching engine + ``_discovery_score_candidates`` +
Spotify / iTunes clients, passes through to the registry.
- ``refresh_mirrored.py`` adds a small ``_maybe_discover`` helper
that calls ``source.discover_tracks(...)`` between fetch and
``to_mirror_track_dict`` projection — only fires when at least
one track has ``needs_discovery=True``, so the normal Spotify /
Tidal / etc. refresh path stays a zero-cost pass-through.
Tests:
- 5 new adapter tests: default no-op pass-through, LB discovery
with mixed matches/misses, LB no-callable fallback, Last.fm
shares the LB implementation, mirror-dict spotify_hint emit.
- 1 new automation test: end-to-end LB refresh with a stub
discover_callable proves the matched_data lands in
``mirror_playlist_tracks.extra_data`` after the registry
refresh + discover hop.
225 tests across adapter + automation suites green.
Phase 1a of the Discover-to-Sync unification. The mirrored-playlist
refresh handler used to branch per-source through a ~190-line
if/elif chain (Spotify, Spotify public, Deezer, Tidal, YouTube).
Each branch hand-built its own ``extra_data`` JSON for the matched-
data block. With every new source we considered for Sync-page mirror
support (ListenBrainz, Last.fm radio, SoulSync Discovery, iTunes
link), that chain would have grown a new elif.
This commit lifts the per-source logic into the existing adapter
layer and collapses the dispatch to a registry lookup:
- ``core/playlists/sources/deezer.py`` — new adapter so the registry
covers every source the refresh handler previously branched on.
- ``core/playlists/sources/bootstrap.py`` — single helper that builds
a populated registry from injected getter callables. Both
``web_server.py`` boot and the automation test fixtures call it,
so the two construction paths can't drift.
- ``core/playlists/sources/base.py`` — ``to_mirror_track_dict``
projection helper centralises the NormalizedTrack → DB-row
conversion (including the discovered/matched_data and
spotify_hint extra_data shapes the downstream sync + wishlist
consumers already expect).
- Spotify adapter now populates ``extra['discovered']`` + an
``extra['matched_data']`` block when fetching via the authed API,
so Spotify mirrors keep landing pre-discovered (matches the
pre-refactor contract pinned by
``test_spotify_refresh_writes_to_db``).
- Spotify-public adapter populates ``extra['spotify_hint']`` so the
discovery worker can skip its search step and jump straight to
enrichment for the known track ID.
- All artist-name fields now project to first-artist-only across
every adapter — matches the pre-refactor mirror_playlist DB shape
(``t.artists[0]``).
``refresh_mirrored.py`` shrinks ~190 → ~80 lines and keeps:
- the file/beatport unrefreshable-source filter,
- URL extraction from ``description`` via ``require_refresh_url``
for spotify_public + youtube,
- the Spotify-public → authed-Spotify fallback when the user is
signed in (handler-level branch, not in any adapter),
- the Tidal-not-authenticated soft-skip log (skip, not error),
- existing-extra_data preservation across refreshes,
- the ``playlist_changed`` automation event emit on track-set delta.
Test scaffolding:
- ``_build_deps`` in ``tests/automation/test_handlers_playlist.py``
now builds a default registry from the passed clients via
``build_playlist_source_registry``, so existing refresh tests
exercise the same path without per-test changes. New tests cover
Tidal-not-authed soft-skip, Deezer refresh writes plain tracks,
YouTube refresh reads URL from description, and Spotify-public
uses authed Spotify when signed in.
- 4 new adapter tests for Deezer projection +
``to_mirror_track_dict`` (minimal track, Spotify matched_data,
Spotify-public spotify_hint).
- ``playlist_source_registry`` field on ``AutomationDeps`` defaults
to ``None`` so the other 5 automation test files (which don't
exercise refresh_mirrored) keep working unchanged.
220 tests across automation + adapter suites green.
New iTunes Link tab between Deezer Link and YouTube. Accepts album,
track, and playlist URLs from music.apple.com / iTunes. Pulls the
tracklist, runs it through the same discovery -> sync -> download
pipeline as the other link tabs.
Apple Music playlists go through amp-api with a Bearer JWT scraped
from the SPA. The legacy meta-tag and inline `"token":"..."` paths
are gone in the current music.apple.com SPA, so the extractor now
walks the page's `<script src>` list (prioritising index/chunk/main
bundles), fetches up to 8 JS bundles, regex-matches JWT-shaped
strings, and base64-decodes each payload to confirm it carries
Apple media-api claims (`root_https_origin`, or `iss + iat + exp`)
before trusting it. Filters out analytics / error-reporter JWTs that
also ship in the bundle.
Tokens are cached at module scope for 6h behind a threading.Lock so
the three-worker discovery executor doesn't thunder-herd Apple on
cold start, and amp-api calls go through a single helper that on
401 invalidates the cache, refetches the page, force-refreshes the
token, and retries the request once. The playlist fetcher memoises
the page HTML for the cache-miss path so we don't refetch it for
every paginated `/tracks` page.
spotify_public discovery worker accepts the new platform shape so
iTunes Link reuses the same matching code path as Deezer Link and
Spotify-public. UI bits live in the sync-services.js iTunes Link
tab, with platform plumbing through wishlist-tools.js for the
multi-source state map.
Extract repeated DB tag payload construction into a new _build_library_tag_db_data(track_data, album_genres) helper and replace in multiple endpoints. The helper builds the metadata dict (title, artist_name, track_artist, album_title, year, genres, track/disc numbers, bpm, track_count, thumb_url) and populates artists_list by splitting track_artist on ';'. Added tests (tests/test_library_tag_payload.py) to verify artists_list creation, genre propagation, thumb_url selection, and fallback behavior when track_artist is missing. This reduces duplication and ensures consistent tag payloads across tag-preview, batch preview, and tag-writing flows.
Update version references for the 2.6.2 release: change the workflow dispatch input description and default to 2.6.2 in .github/workflows/docker-publish.yml, and update the _SOULSYNC_BASE_VERSION constant in web_server.py to 2.6.2 so release metadata and build/version strings reflect the new patch release.
Three problems wrapped into one pass on the Playlist Auto-Sync surface:
1. Visual: the manager modal had its own vibe (radial gradient, pill
tabs, sky-blue chrome) that didn't line up with the rest of the
app. Reworked the modal shell, KPI summary, live pipeline monitor,
tab bar, schedule board sidebar, and column cards to use the
standard SoulSync patterns — gradient `#1a1a1a → #121212`,
accent-tinted 1px border, 20px radius, underline tabs, dense dark
card pattern that Automations + Library pages already use. Modal
now uses near-full screen so there's room for the schedule board
without horizontal scroll pain. Run history cards followed the
same path: slim horizontal row mirroring `.automation-card` plus
an expanded detail that mirrors the Automations run-history modal
(stats-grid + facts row + result pills + log section).
2. Hang: the previous SQL fix for the run-history "in library" count
added `COLLATE NOCASE` on the join columns of `tracks` and
`artists`. SQLite can't use `idx_artists_name` or `idx_tracks_title`
when the comparison collation doesn't match the column collation,
so the join did a full table scan per mirrored playlist track.
~18s per playlist × 30 playlists = `/api/mirrored-playlists` hung
indefinitely and the modal stayed at "Loading schedule…" forever.
Switched the join back to case-sensitive equality (~6ms per
playlist, 3000× faster). Spotify names canonicalize to the same
form as library imports so the recall loss is in the rounding
error of pure case-only mismatches.
3. Slowness: even after the hang fix, each modal open spent ~1.5s
gathering per-playlist status counts. The endpoint looped
`get_mirrored_playlist_status_counts(playlist_id)` per row, which
opened a fresh SQLite connection + PRAGMA setup each time. Added
`get_all_mirrored_playlist_status_counts(profile_id)` which
returns counts for every mirrored playlist owned by the active
profile in 4 batched `GROUP BY` queries over a single connection.
Modal load dropped to ~280ms.
Also fixed: `tracks.artist` reference in `get_mirrored_playlist_status_counts`
that never worked since the schema went relational — the query threw
"no such column", got swallowed by the try/except, and the in-library
count silently defaulted to 0 on every playlist. Rewired to join
through `artists`.
`get_mirrored_playlist_status_counts` (single-playlist) kept for
callers that still want it, but the modal endpoint uses the batched
version.
Persist per-playlist pipeline run snapshots from the shared playlist pipeline, expose a history API, and upgrade the Auto-Sync modal with live pipeline monitoring, Run now controls, and a runs-style history tab.
Expose playlist-native run and status endpoints that reuse the shared mirrored playlist pipeline engine while routing progress into playlist UI state.
Add a Run Pipeline action to mirrored playlist cards and modals with live status polling, and make the shared pipeline lock atomic for manual and scheduled callers.
Return normalized source_ref metadata from mirrored playlist APIs so the UI no longer has to infer editable refresh links from description fields. Accept Spotify embed URLs during source-ref repair and add coverage for source-ref health reporting.
Centralize mirrored playlist source reference normalization so edited links and IDs are stored consistently. Preserve URL-backed refresh refs, surface missing-source refresh failures, count background sync failures in pipeline summaries, and retry guarded automation skips after a short delay instead of losing a scheduled run. Add focused coverage for source refs, mirrored playlist source updates, refresh failures, and guarded retry behavior.
Keep album-bundle staging from replacing known per-track album numbers with the filename parser's default when staged files do not expose a real track number. Carry staging tag numbers through the cache, fall back to task metadata for private release staging, and cap hybrid album batches to one worker when Soulseek is first in the source order.
Include a capped recent tail of database-backed download history in the unified Downloads page so completed Deezer and other streaming downloads remain visible after runtime tasks are cleaned up or the container restarts. Use persistent download history for the dashboard finished count, keep live tasks authoritative for active rows, avoid showing the local clear-completed action for persisted history rows, and cover history hydration/deduping/capping in status tests.
MINOR bump: Qobuz playlist sync is the headline feature (#677), plus
the Import album search fallback-source surfacing fix (#681).
* web_server.py — _SOULSYNC_BASE_VERSION → 2.6.0
* webui/static/helper.js — split the 2.5.9 'Unreleased — dev cycle'
entries into a new 2.6.0 block with a real release-date marker;
bumped the _getLatestWhatsNewVersion fallback default; rolled the
'2.5.9 Release Stability Pass' modal section down to a generic
'Earlier in v2.5' aggregator now that 2.6.0 is the current release
* .github/workflows/docker-publish.yml — bumped manual version_tag
default to 2.6.0 so the next workflow_dispatch defaults right
Three follow-ups to the Qobuz playlist sync commit:
* webui/static/sync-services.js openYouTubeDiscoveryModal — the
syncing-phase "start polling on modal open" switch was missing the
isQobuz branch (the discovery-modal-close handler hit it but this
earlier hook didn't). Resuming a sync after a page refresh would have
fallen through to startYouTubeSyncPolling.
* webui/static/sync-services.js closeYouTubeDiscoveryModal — the
per-service phase reset block had Tidal, Deezer, Spotify Public,
Beatport branches but no Qobuz. After a Qobuz sync_complete or
download_complete, closing the modal wouldn't reset the card phase
back to 'discovered' or push the phase update to /api/qobuz/update_phase.
* web_server.py _emit_discovery_progress_loop — platform_states didn't
include 'qobuz', so WebSocket discovery progress broadcasts were
silently skipping Qobuz playlists. HTTP-poll fallback covers it but
this puts Qobuz on equal footing with the other services.
Qobuz joins Tidal and Deezer as a first-class playlist sync source.
New Qobuz tab on the Sync page lists user playlists + a virtual
Favorite Tracks entry, and clicks route through the same discovery →
sync → download pipeline the other services already use.
Backend:
* core/qobuz_client.py — new get_user_playlists, get_playlist,
get_user_favorite_tracks, get_user_favorite_tracks_count. Returns
normalized dicts (matches Deezer client shape, not Tidal's
dataclasses) so the discovery worker can iterate directly without
duck-typing. Virtual `qobuz-favorites` ID dispatches to favorites
fetcher inside get_playlist — same trick Tidal uses with
COLLECTION_PLAYLIST_ID. Both list endpoints paginate against
Qobuz's 500-cap limit.
* core/discovery/qobuz.py — new worker module. Mirrors
core/discovery/deezer.py: pause enrichment, iterate tracks,
hit discovery cache, fall back to _search_spotify_for_tidal_track,
build wing-it stub on miss, sync results to mirrored playlist.
* web_server.py — adds /api/qobuz/playlists, /playlist/<id>,
/discovery/start/<id>, /discovery/status/<id>, /discovery/update_match,
/playlists/states, /state/<id>, /reset/<id>, /delete/<id>,
/update_phase/<id>, /sync/start/<id>, /sync/status/<id>,
/sync/cancel/<id>. One-for-one with the Tidal + Deezer endpoint
sets. Qobuz discovery executor registered for clean shutdown.
Frontend:
* webui/static/sync-services.js — full handler set (loadQobuzPlaylists,
createQobuzCard, openQobuzDiscoveryModal, startQobuzDiscoveryPolling,
startQobuzPlaylistSync, startQobuzSyncPolling, cancelQobuzSync,
startQobuzDownloadMissing, rehydrateQobuzDownloadModal, etc.).
Reuses the shared YouTube discovery modal via fake `qobuz_<id>`
urlHash and is_qobuz_playlist flag. Shared switch statements in
getModalActionButtons / generateTableRowsFromState / Wing It helpers
in downloads.js gain new isQobuz branches alongside the existing
per-service ones.
* webui/index.html — new Qobuz tab button + content div, slotted
between Deezer and Deezer Link.
* webui/static/style.css — new .qobuz-icon for the tab icon.
* webui/static/core.js — qobuzPlaylists / qobuzPlaylistStates /
qobuzPlaylistsLoaded globals.
Followed the existing per-service pattern verbatim rather than
refactoring the duplicated transformers across Tidal / Deezer /
Spotify-public / YouTube / Mirrored — that refactor is its own follow-up
PR per the "don't break Tidal/Deezer" scope discipline. Adding the 6th
copy of a proven pattern is lower risk than collapsing 5 working
services behind a new abstraction.
Tests:
* tests/test_qobuz_playlists.py — 12 tests covering pagination,
normalization, favorites virtual-ID routing, artist-name fallback
chain (performer → album.artist → 'Unknown Artist'), and
unauthenticated short-circuits.
Three related improvements to the now-playing media player and the
"add to wishlist" / "download missing" modals.
1. Play buttons across track-list modals
Every track row in the download-missing modals (Spotify, Tidal,
YouTube, services, artist album, wishlist download-missing) and
the add-to-wishlist modal now carries a play button. Click runs
playTrackFromLibraryOrStream:
- If the track has a local file_path → playLibraryTrack
- Else POST /api/stats/resolve-track to find it in the library
by title + artist → playLibraryTrack
- Else fall back to _gsPlayTrack streaming
Backend ownership response gains track_id / title / file_path so
the wishlist modal's owned tracks can hand the right metadata
to the player without an extra round trip.
The add-to-wishlist modal previously showed the play button only
on owned tracks; now the button is unconditional so the streaming
fallback can take over for unowned ones (matches the standard
pattern from the rest of the app).
2. Clean media-player display titles
YouTube / Tidal / Qobuz / torrent / usenet plugins encode their
source-side identifier into the filename field as
<source_id>||<display> so download() can recover it later. The
media player's track-title renderer never knew about this
convention and showed strings like
"wvgFsXoGFnQ||Sometimes I Cry When I'm Alone" verbatim in the
now-playing UI. extractTrackTitle and setTrackInfo now strip the
<id>|| prefix defensively so any path into the player gets a
clean display.
Local library playback also fetches canonical metadata from
/api/stats/resolve-track when track.id is present so title /
artist / album / album art come straight from the SoulSync DB
instead of whatever the caller passed in. Falls back silently
to caller values on any error so playback never blocks on the
metadata fetch.
3. Lyrics panel + View Artist close
New collapsed lyrics panel between the playback controls and
queue panel. POST /api/lyrics/fetch (new backend endpoint)
prefers the local .lrc / .txt sidecar files SoulSync writes
during post-processing so downloaded tracks resolve lyrics with
zero network hits; falls back to LRClib exact-match (when album
+ duration are available) then to LRClib search.
Synced LRC results are parsed (handles multi-stamp lines for
repeated choruses), and the active line highlights + smooth-
scrolls into the middle of the viewport on every audio
timeupdate. Plain-text results render without highlighting.
Per-track cache prevents re-fetching when the user revisits the
same track. Lyrics fetch is fire-and-forget — failure shows
"No lyrics found" without ever blocking playback.
View Artist on the expanded player now calls
closeNowPlayingModal before navigating; the modal was previously
sitting open over the artist page, hiding it. Handler is bound
once and is a no-op when no artist_id is attached.
CSS additions are additive (new .modal-track-play-btn and
.np-lyrics-* rules); no existing styles touched. Backend endpoint
returns 200-with-success-false on any miss so callers can render
"no lyrics" without treating it as an error.
WHATS_NEW updated under 2.5.9 with two entries (lyrics + View
Artist close).
Reset profile PIN dialog controls each time it opens so stale profile-specific event listeners cannot submit an admin PIN against a previously selected profile.
Keep failed PIN attempts retryable and restrict launch-lock verification to the admin profile PIN only, so non-admin profile PINs cannot mark the admin lock as verified.
Update project version and release notes for 2.5.9. Changes: update .github/workflows/docker-publish.yml default/version_tag prompt to 2.5.9, bump _SOULSYNC_BASE_VERSION in web_server.py to 2.5.9, and replace the WHATS_NEW entry in webui/static/helper.js with detailed 2.5.9 release notes and a new VERSION_MODAL_SECTIONS entry. Also update the helper.js fallback for latest whats-new version to 2.5.9.
Probe public HiFi instances with the same trackManifests endpoint used by real downloads instead of the legacy /track endpoint. This prevents compatible instances from being falsely labeled search-only in Settings.
Centralize HiFi instance capability checks in HiFiClient and reuse manifest URI parsing with the download path.
Tests cover manifest-based capability detection, no legacy /track probe, and limited instances without a manifest URI.
Adds torrent/usenet as release-oriented download sources with album-bundle staging, live progress reporting, and post-processing that selects the requested audio file from completed releases instead of blindly importing the first file.
Keeps album-bundle behavior gated to single-source torrent/usenet album downloads, excludes release sources from hybrid album per-track searches, and allows hybrid non-album tracks to use release results safely.
Improves staged-release matching for featured/bonus track filenames while preserving version mismatches, records torrent/usenet provenance in library history, and updates service/status UI labels.
Covers the flow with focused lifecycle, status, staging, validation, task worker, post-processing, and import side-effect tests.
Route torrent and Usenet album bundles through private per-batch staging so Auto-Import cannot race public staging or duplicate imports.
Expose album-bundle progress in batch status and render it on the Downloads page while the external client is still downloading.
Tighten release handoff safety by rejecting archive path traversal, ignoring torrent candidates without a usable URL, and skipping Soulseek source reuse for torrent/Usenet batches.
Tests: .venv/bin/python -m pytest tests/downloads/test_downloads_status.py tests/test_album_bundle_dispatch.py tests/downloads/test_downloads_staging.py tests/test_torrent_usenet_plugins.py
Per code review: the album-bundle provenance override added in an
earlier commit reached into ``core.runtime_state.download_batches``
directly from inside the staging matcher. Sibling modules
shouldn't import each other's globals — the existing StagingDeps
pattern is the canonical way to inject everything else this helper
needs.
- core/downloads/staging.py: new optional ``get_batch_field``
callable on ``StagingDeps`` (defaults to None for backward compat
with any caller that doesn't know about it yet). The inline
``from core.runtime_state import download_batches`` is gone; the
helper now calls ``deps.get_batch_field(batch_id,
'album_bundle_source')`` and falls back to 'staging' when None
is returned. Accessor exceptions are swallowed with a debug log
so a deleted batch mid-process can't break the staging match.
- web_server.py: ``_build_staging_deps`` injects a small
``_staging_get_batch_field`` helper that wraps the tasks_lock +
download_batches dict access. Centralises the lock semantics in
one place — the staging module no longer needs to know about
the lock or the dict.
- tests/test_staging_album_provenance.py: 5 new tests covering the
full matrix — torrent override applied, usenet override applied,
no override falls back to 'staging', missing accessor (default
None) falls back to 'staging', accessor raising falls back to
'staging'. Each test seeds + cleans a synthetic task in
runtime_state so the test doesn't bleed state across the suite.
Third commit in the torrent + usenet rollout. SoulSync now also
speaks the two big usenet downloaders through a sibling adapter
contract that mirrors the torrent adapter set. All three layers are
now stood up — Prowlarr finds releases, the torrent adapter and the
usenet adapter each know how to ship work to the underlying client.
A later commit wires Prowlarr search results through the adapters
and through the archive-extract-match pipeline.
- core/usenet_clients/base.py: UsenetClientAdapter Protocol +
UsenetStatus dataclass. Uniform state set covers usenet-specific
phases (queued / downloading / extracting / verifying / repairing /
completed / failed / paused).
- core/usenet_clients/__init__.py: adapter_for_type factory +
get_active_adapter that reads usenet_client.type each call.
- core/usenet_clients/sabnzbd.py: REST adapter. ?apikey=... auth,
mode=addurl and mode=addfile (multipart) for add_nzb. Reads both
the active queue and the recent history so completed / failed
jobs surface in get_all. Parses SAB's HH:MM:SS ``timeleft`` into
seconds.
- core/usenet_clients/nzbget.py: JSON-RPC adapter. HTTP Basic auth,
``append`` method for add_nzb (auto-detects URL vs base64 NZB),
``editqueue`` with GroupPause/GroupResume/GroupDelete/GroupFinalDelete
for state changes. Reads NZBGet's 64-bit split size fields
(FileSizeHi + FileSizeLo) preferentially over the legacy
FileSizeMB aggregate.
- core/connection_test.py: 'usenet_client' branch picks the right
adapter, runs check_connection, surfaces per-client error
messages (different credentials needed).
- config/settings.py: usenet_client.{type, url, api_key, username,
password, category} defaults + both api_key and password marked
encrypted-at-rest.
- web_server.py: 'usenet_client' added to the /api/settings POST
allow-list.
- webui/index.html: new Usenet Client panel on the Indexers &
Downloaders tab. Type picker swaps the credential fields between
API-key (SABnzbd) and username+password (NZBGet).
- webui/static/settings.js: load/save wiring, updateUsenetClientUI
for the credential field swap, testUsenetClientConnection.
- webui/static/helper.js: WHATS_NEW + VERSION_MODAL_SECTIONS entry.
Second commit in the torrent + usenet rollout. SoulSync now speaks
three different BitTorrent client APIs through one uniform adapter
contract — picks the active client by config and dispatches the same
verbs to whichever backend the user uses. Each adapter handles its
own auth quirk (qBit cookie + CSRF Referer, Transmission session-id
renegotiation, Deluge JSON-RPC session) and maps native state
strings onto a shared 7-value set so the rest of the app stays
client-agnostic.
- core/torrent_clients/base.py: TorrentClientAdapter Protocol +
TorrentStatus dataclass. Eight verbs: is_configured, check_connection,
add_torrent (URL/magnet), add_torrent_file (raw bytes), get_status,
get_all, remove, pause, resume.
- core/torrent_clients/__init__.py: adapter_for_type factory +
get_active_adapter that reads torrent_client.type each call so
settings changes take effect without restart.
- core/torrent_clients/qbittorrent.py: WebUI v2 adapter. Cookie auth
via /api/v2/auth/login, transparent 403 re-login, Referer header
to satisfy qBit's CSRF guard. add_torrent returns the just-added
hash via /torrents/info sort=added_on (qBit's add endpoint doesn't
echo the hash).
- core/torrent_clients/transmission.py: RPC adapter. Auto-resolves
bare host URLs to /transmission/rpc, handles the 409 + new
X-Transmission-Session-Id renegotiation transparently, accepts
HTTP basic auth. add_torrent_file base64-encodes payload per spec.
- core/torrent_clients/deluge.py: Deluge 2.x JSON-RPC adapter.
Password-only auth, distinguishes magnet vs HTTP URL at the RPC
method layer, applies category via Label plugin (best-effort —
label plugin is optional).
- core/connection_test.py: 'torrent_client' branch picks the right
adapter, runs check_connection, surfaces a per-client error
message.
- config/settings.py: torrent_client.{type, url, username, password,
category, save_path} defaults + torrent_client.password in the
encrypted-at-rest secrets list.
- web_server.py: 'torrent_client' added to the /api/settings POST
allow-list so saved config persists.
- webui/index.html: new Torrent Client panel on the Indexers &
Downloaders tab — client-type dropdown, URL, username, password,
category, optional save path, Test Connection.
- webui/static/settings.js: load/save wiring + testTorrentClientConnection.
- webui/static/helper.js: WHATS_NEW + VERSION_MODAL_SECTIONS entry.
First commit toward torrent and usenet download sources. Prowlarr is
the indexer manager component of the *arr stack — it exposes Usenet
and torrent indexers behind a single Newznab-style API so SoulSync
doesn't have to integrate each indexer individually. This commit
wires up Prowlarr as a search-only source; the torrent and usenet
download client adapters land in the next commits and plug into
this search surface.
- core/prowlarr_client.py: sync-backed async client. is_configured,
check_connection, get_indexers, search by Newznab category. Music
category constants (3000 all / 3010 MP3 / 3040 lossless / etc.).
- core/connection_test.py: 'prowlarr' branch hits /api/v1/system/status
for the Test Connection button.
- web_server.py: GET /api/prowlarr/indexers returns the live indexer
list (id, name, protocol, enabled, privacy). Settings POST allow-list
now accepts 'prowlarr' so saved config persists.
- config/settings.py: prowlarr.{url, api_key, indexer_ids} defaults
plus prowlarr.api_key in the encrypted-at-rest secrets list.
- webui/index.html: new "Indexers & Downloaders" tab on Settings with
the Prowlarr panel (URL, API key, Test, Refresh Indexer List,
optional indexer-ID allowlist).
- webui/static/settings.js: load/save wiring, testProwlarrConnection,
loadProwlarrIndexers (HTML-escapes user-supplied indexer names).
- webui/static/helper.js: WHATS_NEW 2.6.0 unreleased block plus a
curated VERSION_MODAL_SECTIONS entry.
Prepare 2.5.8 release: update the workflow default version_tag and the app _SOULSYNC_BASE_VERSION to 2.5.8, add WHATS_NEW entries for 2.5.8 (fix blank artist pages for Python/git-pull installs, fix premature download completion before post-processing, add disk-backed artwork cache with SQLite, and add pre-download duration tolerancing for strict sources), and update the whats-new fallback to 2.5.8.
Add a disk-backed image cache with hashed browser URLs, SQLite metadata, size/type validation, stale fallback, and per-image fetch locking. Route normalized artwork through /api/image-cache while keeping /api/image-proxy as a compatibility shim, and align browser max-age with the image cache TTL. Add focused tests for cache behavior and image URL normalization.
Patch bump for the post-2.5.6 fix cycle. Nine entries shipped since the
2.5.6 release moved into a fresh 2.5.7 WHATS_NEW block — original 2.5.6
release notes left intact.
Touched:
- web_server.py: `_SOULSYNC_BASE_VERSION` 2.5.6 -> 2.5.7
- webui/static/helper.js: new `'2.5.7'` block with date marker + the
nine shipped fixes; fallback default in `_getLatestWhatsNewVersion`
bumped to '2.5.7'
- .github/workflows/docker-publish.yml: workflow_dispatch description
+ default tag both bumped to 2.5.7
What's in 2.5.7 (all post-2.5.6 cycle work):
- MB manual search recall fix (strict -> bare-query)
- MB album-detail 404 fix (invalid cover-art-archive include)
- Fix popup MBID paste field (#647)
- MB added to Fix popup auto-search cascade (#655)
- Docker /app/Stream pre-baked for rootless Docker (#656)
- slskd unreachable log spam suppression (#649)
- MB 'Other' release-groups now visible in discography (#650)
- Quarantined-source dedup on auto-wishlist cycles (#652)
- Unknown Artist Fixer ImportError fix (#646)
The cancel-trigger diagnostic logging commit (a685f9ca) is also in
2.5.7 but isn't user-facing so no WHATS_NEW entry.
Diagnostic-only change for issue Technodude reported: Tidal sync-playlist
downloads getting mass-cancelled mid-flight with no clear cause in the
logs. App.log shows ~91 second gaps between Tidal download start and
cancel — matches the monitor's 90s queue-timeout exactly — but none of
the monitor's WARNING log lines fire, so the trigger is ambiguous
between five `_should_retry_task` paths, three web_server cancel paths,
and the API endpoints.
Added a single `[CancelTrigger:<label>]` INFO log line immediately
before every `download_orchestrator.cancel_download(...)` call so the
next log dump pins down which path is firing.
Labels (grep-able, prefix tells the file, suffix tells the trigger):
monitor.not_in_live_transfers_90s
monitor.errored_state_retry
monitor.queued_state_timeout
monitor.stuck_at_0pct_timeout
monitor.unknown_state_no_progress_timeout
candidates.worker_cancelled_during_download
web.orphan_cleanup
web.cancel_download_task
web.atomic_cancel_v2
api.manual_cancel_single
api.public_cancel
The monitor's `deferred_ops` tuple grew from 3 elements to 4 (added
trigger label as last element). The dispatch loop unpacks both legacy
and new shapes so the change is backward-compatible for any in-flight
ops mid-deploy.
Zero behavior change. 367 download tests still green. WHATS_NEW left
untouched — diagnostic only, not user-facing.
After ship: ask Technodude to re-run the same sync playlist scenario,
attach the new app.log, grep `[CancelTrigger:` lines for the trigger
context, then write the actual fix.
The Fix Track Match modal's auto-search was hardcoded to query only
Spotify -> Deezer -> iTunes, ignoring MusicBrainz entirely — even for
users with MB set as their primary metadata source. MB-niche recordings
(canonical entries with diacritics, fringe / non-mainstream tracks that
the commercial catalogues don't carry) had no chance.
Wiring:
- New `MusicBrainzSearchClient.search_tracks_with_artist(track, artist,
limit)` for surfaces that already have title + artist split. Uses MB's
bare-query mode (strict=False) — diacritic-folded, alias/sortname
indexed — same recall rationale as the earlier MBID-paste endpoint.
- New route `GET /api/musicbrainz/search_tracks` mirrors the existing
/api/{spotify,itunes,deezer}/search_tracks endpoints exactly: accepts
`track`+`artist` (or legacy `query`) + `limit`, returns
`{tracks: [{id, name, artists, album, duration_ms, image_url, source}]}`.
Applies the same `core.metadata.relevance.rerank_tracks` pass Deezer /
iTunes use, which is critical because MB's free-text scoring weighs
title-text matches heavily and would otherwise rank cover / tribute
recordings above the canonical version.
- `_search_tracks_text` gains a `min_score` parameter. The cascade path
passes 20 (vs the enhanced-search-tab default of 80) so MB recordings
whose title doesn't literally contain the artist name still enter the
candidate pool — without that, "Army of Me" + "Bjork" only surfaces
the HIRS Collective cover (score 100) and drops Björk's canonical
recording (score 28). The rerank pass then surfaces Björk by artist
match. Verified against real MB API: pre-fix returned only the cover;
post-fix top 5 are all Björk.
- Fix popup `allSources` array (wishlist-tools.js) gets MB appended.
The existing `activeIdx` reorder logic moves MB to the front when
it's the active primary; otherwise MB sits last (1 req/sec rate
limit makes it the slowest source).
7 new unit tests on the adapter: bare-query mode is used, missing
artist falls back to None (drops AND-clause), empty inputs short-circuit,
low-score candidates are kept for rerank to handle, default strict +
default min_score behaviour preserved for the existing search-tab path,
client errors are swallowed so the cascade falls through to the next
source.
Discogs intentionally absent — Discogs has no track-level search API
(see core/discogs_client.py:575 — returns []). Adding a Flask endpoint
that always returns empty would be a permanent no-op.
Power-user escape hatch on the Discovery Fix Track Match modal — when
fuzzy auto-search ranks the wrong recording among many same-title
versions (10 remasters, live cuts, alt sessions), paste the MusicBrainz
recording URL or bare UUID into the new field and resolve straight to
that record.
Layout:
- Shape adapter `get_recording_flat(mbid)` lives in
`core/musicbrainz_search.py` next to existing `get_track_details`.
Returns the flat Fix-popup track shape (artists as `string[]`,
album as string, single `image_url`) — distinct from the
Spotify-shaped nested dict `get_track_details` returns.
- New route `GET /api/musicbrainz/recording/<mbid>` is a thin wrapper:
validates MBID format with an anchored UUID regex, calls the adapter,
returns 400 / 404 / 200 with no inline shape massaging.
- Frontend `parseMusicBrainzMbid()` lives in `shared-helpers.js` —
pure URL/UUID parser, reusable from other surfaces (failed-MB cache,
manual match) without duplication.
- Fix modal HTML gets one new input row + button; existing search row
and result render pipeline are untouched. New `lookupDiscoveryFixByMbid()`
fetches the endpoint and feeds the single result through the existing
`renderDiscoveryFixResults` -> confirm-dialog -> match pipeline, so MB-
paste matches go through the exact same selection flow as auto-search
results.
- Enter-key bound on the MBID input via a separate handler ref so its
lifecycle matches the search-input handlers without conflating the
two submit targets.
7 unit tests cover the adapter: happy path, empty/None MBID, MB returns
None, recording-without-release (empty album), multi-artist credits,
includes-list contract, and client-error swallow.
Out of scope: the Fix popup's fuzzy cascade is still hardcoded to
spotify/deezer/itunes regardless of which primary source the user has
configured. Adding MB to that cascade (when MB is the active primary)
is a separate concern.
Two bugs surfacing on the Fix popup and enhanced-search MB tab:
1. Strict Lucene phrase queries (`recording:"X" AND artist:"Y"`) killed
recall on user-facing manual search — diacritics ("Bjork" vs canonical
"Björk"), bracketed suffixes like "(Live)", and any AND-clause
mismatch returned zero results. Added `strict: bool = True` param to
`search_release` / `search_recording`; when False, sends a bare query
joining title + artist so MB hits alias/sortname indexes with
diacritic folding. `/api/musicbrainz/search` (Fix popup) and
`core/library/service_search.py` (service tabs) now pass strict=False.
Enrichment workers stay on strict mode — precision matters there
because they auto-accept the top hit above a confidence threshold.
2. Every MB album click was silently 404-ing — `_render_release_as_album`
passed `cover-art-archive` as an MB `inc` param, but it's not a valid
include for the /release resource (MB rejects with 400). The CAA flags
come back on every release response by default, so dropping the bad
include preserves the image-scope picker logic intact.
Add MusicBrainz watchlist artist ID storage, badges, linked-provider editing, and per-artist preferred source support.
Backfill watchlist MusicBrainz matches from already-enriched library artists so existing MusicBrainz worker matches appear in watchlist cards and settings.
Extend bulk watchlist add, liked artist matching, artist map source picking, and service status labels to recognize MusicBrainz, with regression tests for watchlist ID persistence and backfill.
Register MusicBrainz as a first-class metadata source alongside Deezer, iTunes, Spotify, Discogs, and Hydrabase. Expose the shared client through metadata services, add the settings option, and expand the MusicBrainz search adapter with source-compatible artist, album, track, and detail methods.
Carry MusicBrainz IDs through similar-artist discovery, recommended artists, artist map serialization, and personalized playlist selection. Update DB migrations and lookup filters so similar_artist_musicbrainz_id is preserved on older schemas and used for source requirements and library exclusion.
Normalize MusicBrainz album adapter output for import context and add regression coverage for registry mapping, typed album conversion, and similar-artist filtering. Verified by user with 120 focused tests passing.
Move the existing-file missing-track import workflow out of web_server.py and into core/library/missing_track_import.py.
Keep the Flask route focused on request wiring and response formatting while the service handles staging copy, post-processing, album identity tag inheritance, DB upsert, and media-server sync.
Show actionable missing album tracks in the enhanced library from canonical metadata, with a practical Manage flow for Add to Library or I Have This.
Implement I Have This as a non-destructive copy/import path: copy the chosen existing file, run normal post-processing with the missing track context, insert the real library row, and inherit album identity tags from target siblings so Navidrome does not split albums.
Improve the modal with selectable search results, visible import progress, disabled controls during import, and missing-track row styling.
- _SOULSYNC_BASE_VERSION in web_server.py
- WHATS_NEW key + date in helper.js (strips unreleased flag from Amazon entries)
- fallback version string in helper.js
- Artist cards, hero section, and enhanced view now show Amazon Music badges
when amazon_id is populated (AMAZON_LOGO_URL constant, orange #FF9900 brand)
- Enhanced view artist and album match status rows include amazon_match_status
chip with click-to-rematch via openManualMatchModal
- getServiceUrl: added amazon (album/track ASIN → music.amazon.com) and fixed
missing discogs entries; serviceLabels adds tidal/qobuz/amazon
- Enhanced view enhanced-artist-id-badges includes amazon_id entry
- DB SELECTs for library artists list and artist detail now return amazon_id;
both response dicts include the field
- watchlist_artists migration adds amazon_artist_id column
- Watchlist config GET: amazon_artist_id in SELECT/WHERE/response (index 18)
- Watchlist artists list response includes amazon_artist_id
- link-provider endpoint: amazon added to valid_providers and col_map
- _populateLinkedProviderSection: amazonId param + Amazon Music source row
- Watchlist card source badges render Amazon pill (watchlist-source-amazon CSS)
- _openSourceSearch labels map includes amazon
- service_search: amazon_worker injected via init(); _search_service amazon branch
uses search_artists/albums/tracks, same {id,name,image,extra} return shape
- _SERVICE_ID_COLUMNS: amazon → amazon_id for artist/album/track
- _init_service_search call passes amazon_worker_obj
- amazon_client._fetch_album_metas: 5-minute TTL cache per ASIN — cached hits
skip _rate_limit() and HTTP call entirely; fixes ~10s artist detail load
- registry.py: removed amazon from METADATA_SOURCE_PRIORITY and
METADATA_SOURCE_LABELS — T2Tunes has no discography API, cannot serve as a
primary metadata source; Amazon remains a download source + ASIN enricher
- Settings metadata source dropdown and help text updated accordingly
Two bugs in the library artist detail page when Amazon is the source:
1. No album art: get_artist_albums returned Album dataclasses with
image_url=None — it collected ASINs but never called _fetch_album_metas.
Now fetches metas for up to 10 albums (same cap as search_albums),
populating image_url, release_date, and total_tracks on each Album.
2. No singles: Album.from_search_hit hardcodes album_type="album" and
T2Tunes exposes no release type in search results. Added inference:
total_tracks==1 → album_type="single", which routes them to the
singles bucket in the discography categorizer.
Also passes album_name through _strip_edition and artist through
_primary_artist in get_artist_albums (parity with search_albums).
3. amazon_id missing from artist_source_ids in get_artist_detail:
the discography lookup never received the stored Amazon slug so
it always fell back to name search. Added 'amazon': artist_info.
get('amazon_id') to the dict alongside spotify/deezer/itunes/etc.
_get_enrichment_status had a hardcoded workers_info list. Amazon was
registered in the generic enrichment blueprint but never added here,
so the rate-monitor speedometer overlay and status API omitted it.
Adds ('amazon_enrichment', 'Amazon Music', lambda: amazon_worker)
to workers_info — same pattern as Deezer, Discogs, Tidal, Qobuz.
Adds full parity with Deezer/Qobuz/Tidal/Discogs in every dashboard
UI layer — orb button, live tooltip, WebSocket push, rate speedometer.
- webui/index.html: Amazon enrichment orb button after Discogs
- webui/static/amazon.svg: local icon (a + smile, same pattern as
hydrabase.png — avoids external URL dependency)
- webui/static/style.css: Amazon button/spinner/tooltip CSS with
FF9900 brand color; added to mobile tooltip suppress list
- webui/static/worker-orbs.js: Amazon orb in WORKER_DEFS [255,153,0]
- webui/static/api-monitor.js: Amazon in rate gauge services list,
label, and color map
- webui/static/enrichment.js: updateAmazonEnrichmentStatusFromData,
toggleAmazonEnrichment, DOMContentLoaded init + 2s poll
- webui/static/core.js: socket.on enrichment:amazon-enrichment listener
- web_server.py: amazon-enrichment added to _emit_enrichment_status_loop
workers dict so WebSocket pushes fire every 2s
Background worker matching library artists/albums/tracks to Amazon ASINs
via T2Tunes search. Follows same 6-tier priority queue as Deezer/iTunes/
Spotify/Qobuz/Tidal workers. Backfills artist thumbnails from album cover
stand-ins (T2Tunes exposes no direct artist images).
- core/amazon_worker.py: new AmazonWorker class with full parity
- database/music_database.py: expand _add_amazon_columns to cover
amazon_id/amazon_match_status/amazon_last_attempted on artists,
albums, and tracks (was artists-only)
- web_server.py: import, init, register in enrichment panel, add to
scan pause/resume dicts and rate monitor key map
- helper.js: WHATS_NEW 2.5.3 entry for enrichment worker
AcoustID verification was quarantining every Amazon track because T2Tunes
embeds [Explicit] and [feat. X] in stream tag titles/artists, but AcoustID
returns bare titles — triggering version-mismatch rejection on every track.
- get_track_details: apply _strip_edition to name/album, _primary_artist to
artist; wire s.track_number / s.disc_number instead of hardcoded None
- get_album_tracks: apply _strip_edition to name, _primary_artist to artist
Also fix TypeError crash in album download paths when disc_number is None
(present in dict but explicitly None, so .get('disc_number', 1) returns None):
- master.py run_full_missing_tracks_process: or 1 guard on both max() and disc_num
- candidates.py track_info extraction: or 1 guard on both disc_number reads
- web_server.py enhanced + standard album download max() calls: or 1 guard
- All search_raw calls switched from single-type to types="track,album" — T2Tunes only
returns results when both types are requested together
- _fetch_album_metas: parallel fetch (up to 5 workers) of album cover art via
album_metadata(asin) — T2Tunes search results carry no image URLs
- search_tracks: populates image_url, release_date, total_tracks from album meta
- search_artists: strips feat. credits via _primary_artist() so "Artist feat. X" and
"Artist ft. Y" collapse to one "Artist" entry; uses album cover as artist image
stand-in (same approach as iTunes — T2Tunes has no artist images)
- search_albums: name-based dedup (display_name + artist key) instead of ASIN-based;
populates image_url, release_date, total_tracks from album meta (cap 10 ASIN fetches)
- _strip_edition(): strips [Explicit]/(Explicit) from track/album names — explicit is
the default version; Clean/Edited/Censored labels kept as-is so they stay distinct
- get_album(): applies _strip_edition to name and _primary_artist to artist so
MusicBrainz preflight matching doesn't fail on "[Explicit]" album names
- get_album_tracks(): populates track_number and disc_number from T2TunesStreamInfo
instead of hardcoding None — fixes track ordering in multi-track album downloads
- get_artist() / get_artist_albums(): _unslugify() converts slug artist IDs back to
search names; _primary_artist() in comparison handles feat-annotated results
- SOURCE_ONLY_ARTIST_SOURCES: added "amazon" so artist detail page doesn't 404
- build_source_only_artist_detail: added amazon_client param + dispatch branch
- web_server.py: resolve amazon_client in _build_source_only_artist_detail wrapper;
add source_override=="amazon" branch in get_spotify_album_tracks endpoint
- 77 tests covering all above paths; all pass
- Add 'amazon' to VALID_SOURCES (and transitively VALID_STREAM_SOURCES)
in core/search/orchestrator.py so the backend accepts it as a
requested source without returning 400
- Add resolve_client('amazon') case — mirrors musicbrainz pattern,
gets the cached AmazonClient from the metadata registry
- Add 'amazon' to _alternate_sources() so it appears as a tab when
another source is primary (always available, no credentials)
- Add SERVICE_CONFIG_REGISTRY entry 'amazon': {'always': True} so
/api/settings/config-status reports it as configured
- Add SOURCE_LABELS['amazon'] and SOURCE_ORDER entry in
shared-helpers.js so both enhanced search and global search show
the Amazon Music tab
- Add 'amazon' to _ALWAYS_CONFIGURED_SOURCES so the picker never
dims the tab (no credentials required)
- Add .enh-tab-amazon.active CSS (Amazon orange #FF9900)
- 3530 tests pass
`validation.py` had amazon absent from `_streaming_sources`, causing
Amazon TrackResult objects (bitrate=None, size=0) to fall through to
the Soulseek P2P code path and get rejected by
`filter_results_by_quality_preference`. Every album track was marked
not found.
Fix: add 'amazon' to every streaming-source guard tuple/set that was
previously missing it:
- core/downloads/validation.py — primary bug fix (quality-filter bypass)
- core/downloads/status.py — _STREAMING_SOURCE_NAMES frozenset
- core/downloads/task_worker.py — hybrid fallback client map
- core/imports/side_effects.py — || filename→stream-id extraction
- web_server.py — is_streaming_source, transfer list display,
candidate source label, _try_source_reuse, _store_batch_source
- tests/test_download_plugin_conformance.py — registry count + parametrize
Also updates the 2.5.3 What's New entry to drop the stale
"not yet wired" disclaimer.
Follows the exact same standard as Tidal, Qobuz, HiFi, and Deezer.
registry.py — import + register AmazonDownloadClient as 'amazon'.
amazon_download_client.py — read amazon_download.quality / allow_fallback
from config on init; pass quality as preferred_codec to AmazonClient;
_download_sync codec waterfall respects allow_fallback flag.
download_orchestrator.py — reload_settings() updates preferred_codec +
allow_fallback on the live client after a settings save. 'amazon' added
to _streaming_sources so search_and_download_best routes it correctly.
api_call_tracker.py — 'amazon' registered in RATE_LIMITS (120/min),
SERVICE_LABELS, and SERVICE_ORDER so API call monitoring shows Amazon.
web_server.py — 'amazon_download' added to the settings service loop.
'amazon' added to serverless_sources (no slskd probe needed). Streaming
file-finder extended to handle amazon username + ||asin||title encoding
(extension-less fuzzy match, same as Tidal/Qobuz/HiFi). New endpoint:
GET /api/amazon/test-connection → checks T2Tunes proxy status.
webui/index.html — amazon-download-settings-container: quality dropdown
(flac/opus/eac3), allow-fallback checkbox, test-connection button.
webui/static/settings.js — 'Amazon Music' added to HYBRID_SOURCES,
_hybridSourceEnabled, allSources mode list, loadSettings(), saveSettings()
payload, updateDownloadSourceUI() show/hide + auto-test. New
testAmazonConnection() function.
Reproduced on the personalized playlist pipeline: selecting Fresh Tape
(or any kind) and running the automation surfaced
"Working outside of application context" in the UI.
Root cause: `get_current_profile_id` reads Flask's `g.profile_id` and
only catches `AttributeError`. Outside a request — automation engine,
sync threads, watchlist scanner — `g` raises `RuntimeError` instead,
so the except misses and the handler dies.
Mirrored playlist pipeline never hit this because it hardcodes
profile_id=1 in its sync call. The personalized pipeline calls
`deps.get_current_profile_id()` from a background thread, which is
what tripped the bug. Fresh Tape's generator also resolves the
profile via the same function — same path, same crash.
Fix: broaden the except to `(AttributeError, RuntimeError)` in all
three copies of the helper (`web_server.py`, `core/artists/map.py`,
`core/discovery/hero.py`). All three now safely degrade to profile_id=1
(admin profile) when called outside a request context — matches the
existing intent that single-admin installs Just Work.
No test changes — the existing pipeline tests stub the helper, so
they never exercised the bug. The fix is in the layer above the
stubs.
The action was registered + the block declared, but the automation
builder's per-action config renderer didn't have a case for
`personalized_pipeline` so users only saw the bare card with the
generic delay-minutes input — no way to select which playlists to
sync. This commit adds the multi-select picker.
Backend:
- `core/personalized/api.list_kinds(manager=...)` now optionally
takes a manager and includes the resolved variant list per kind
(calls each spec's variant_resolver(deps) when present). Singleton
kinds get an empty `variants` list. Variant-bearing kinds
(time_machine / genre_playlist / daily_mix / seasonal_mix) get
their full enumerated set.
- `web_server.py` `/api/personalized/kinds` route now passes a built
manager so the variants list lands in the response.
Frontend:
- `webui/static/stats-automations.js` `_renderBlockConfigFields`
gains a `personalized_pipeline` branch that renders a scrollable
multi-select picker:
- Singletons (Hidden Gems, Discovery Shuffle, Popular Picks,
Fresh Tape, The Archives) = one checkbox row per kind
- Variant kinds = a section header + one checkbox row per variant
(e.g. Time Machine: 1960s/1970s/.../2020s; Seasonal: halloween/
christmas/valentines/summer/spring/autumn)
- Pre-checks rows that match the existing `kinds` config on edit
- New `_autoLoadPersonalizedKinds(slotKey)` fetches `/api/personalized/kinds`
(cached after first load), renders the picker DOM, and pre-checks
saved selections via `data-kind` / `data-variant` attributes on
the checkboxes.
- `_renderBuilderCanvas` calls the loader for any `cfg-*-kinds-picker`
it finds in the freshly-rendered slots.
- The save-time `_collectActionConfig` walks the picker's checked
inputs (matched by `data-kind` attribute) and emits
`{kinds: [{kind, variant?}, ...], refresh_first, skip_wishlist}`
in the same shape the handler expects.
Tests:
- `tests/automation/test_automation_blocks.py::_FIELD_TYPES` adds
'personalized_playlist_select' so the block-shape regression test
accepts the new field type. (Test was failing because it whitelists
every field type used across all blocks.)
- 189 automation + personalized API tests pass; full suite intact.
Follow-up to the personalized-playlists standardization PR. New
`personalized_pipeline` automation action syncs selected discover-
page playlists (Hidden Gems / Discovery Shuffle / Time Machine /
Genre / Daily Mix / Fresh Tape / The Archives / Seasonal Mix) to
the active media server + queues missing tracks for download.
Same pattern as the existing mirrored `playlist_pipeline` but two
phases instead of four — no REFRESH (no external source to re-pull)
and no DISCOVER (manager-backed snapshots are already metadata-
matched). Pipeline shape:
SNAPSHOT → SYNC → WISHLIST
Where SNAPSHOT either reads the persisted track list from
`PersonalizedPlaylistManager` (default) or refreshes it first when
`refresh_first=true` (cron use case: regenerate Hidden Gems nightly
and sync the fresh set).
Shared helper extraction:
PHASE 3 (SYNC loop) + PHASE 4 (WISHLIST tail) lifted out of mirrored
`playlist_pipeline` into `core/automation/handlers/_pipeline_shared.py`
as `run_sync_and_wishlist(deps, automation_id, playlists, sync_one_fn,
sync_id_for_fn, ...)`. Both pipelines call it. Mirrored injects
`auto_sync_playlist` as the per-playlist sync function; personalized
injects a thin wrapper that launches `_run_sync_task` directly with
a pre-built tracks_json. Same sync-state polling / progress emission
/ status counting / wishlist trigger logic — 0 duplication.
Files added:
- core/automation/handlers/_pipeline_shared.py
- core/automation/handlers/personalized_pipeline.py
- tests/automation/test_handlers_personalized_pipeline.py
Files changed:
- core/automation/handlers/playlist_pipeline.py: PHASE 3+4 replaced
with shared helper call (~100 lines deleted, 1 helper invocation
added; behavior identical).
- core/automation/deps.py: new `build_personalized_manager` field
(lazy builder so the pipeline gets a fresh PersonalizedPlaylistManager
per run).
- core/automation/handlers/__init__.py + registration.py: register
`personalized_pipeline` action with the shared `pipeline_running`
guard so it can't overlap mirrored.
- core/automation/blocks.py: new `personalized_pipeline` block
declaration with config_fields (kinds multi-select, refresh_first,
skip_wishlist).
- web_server.py: thread `_build_personalized_manager` into
AutomationDeps construction.
- All 5 automation test fixtures: `_build_deps` adds
`build_personalized_manager=lambda: None` stub.
- tests/automation/test_handler_registration.py:
EXPECTED_ACTION_NAMES + EXPECTED_GUARDED_ACTIONS gain
`personalized_pipeline`.
Trigger schema:
{
"_automation_id": "...",
"kinds": [
{"kind": "hidden_gems"},
{"kind": "time_machine", "variant": "1980s"},
{"kind": "seasonal_mix", "variant": "halloween"}
],
"refresh_first": false,
"skip_wishlist": false
}
Tests (14 new, 178 automation total):
- _track_to_sync_shape: basic shape, source ID fallback chain,
no-id returns empty string
- empty config / non-list kinds / empty kinds list all return
error + clear pipeline_running flag
- _build_payloads_for_kinds: skips invalid entries, skips kinds
with no tracks, refresh_first vs ensure dispatch, payload shape
+ sync_id format, manager exception swallowed continues
- _sync_personalized_playlist: launches background thread + returns
status='started'
- happy path: stubbed sync_states drives helper to completion, flag
cleaned up
Full suite: 3383 passed.
Note: the trigger UI block declares config_fields but the frontend
doesn't yet render the `personalized_playlist_select` multi-select
type — usable today via API; polished UI ships in a follow-up
frontend PR.
Wraps the manager + generator dispatch behind one HTTP surface so
the UI can drop the patchwork `/api/discover/personalized/*` calls
in favor of a single REST shape. Legacy endpoints stay alive for
backward compat during the UI migration window.
New endpoints:
- GET /api/personalized/kinds — list every registered kind + metadata
- GET /api/personalized/playlists — list every persisted playlist for the active profile
- GET /api/personalized/playlist/<kind> — fetch singleton + tracks
- GET /api/personalized/playlist/<kind>/<variant> — fetch variant + tracks
- POST /api/personalized/playlist/<kind>/refresh — regenerate singleton
- POST /api/personalized/playlist/<kind>/<variant>/refresh — regenerate variant
- PUT /api/personalized/playlist/<kind>/config — patch singleton config
- PUT /api/personalized/playlist/<kind>/<variant>/config — patch variant config
Per-call manager construction wires the deps each generator needs:
- database (MusicDatabase singleton)
- service (PersonalizedPlaylistsService for legacy generator calls)
- seasonal_service (SeasonalDiscoveryService for seasonal_mix)
- get_current_profile_id (active profile accessor)
- get_active_discovery_source (source dispatcher)
API handlers themselves live as pure functions in
`core/personalized/api.py` so they're testable without Flask. The
Flask layer in `web_server.py` is a thin parse-body / call-handler /
jsonify wrapper.
11 new boundary tests (122 personalized total):
- list_kinds enumerates registry, exposes default config + tags
- list_playlists returns empty list when none exist, serializes
PlaylistRecord shape correctly
- get_playlist_with_tracks auto-creates on first access, returns
persisted tracks, raises ValueError on unknown kind
- refresh_playlist runs generator and returns track snapshot,
forwards config_overrides to the generator
- update_config patches stored config
3365 tests pass total. Manager construction triggers generator
registration via `from core.personalized import generators` import
side-effect.
Cleans up the four remaining inline callbacks at the bottom of
`web_server._register_automation_handlers` so the function is now
purely deps-construction + register_all + a logger.info line.
Lifted:
- `_progress_init`, `_progress_finish`, `_record_automation_history`,
and `_on_library_scan_completed` -> core/automation/handlers/progress_callbacks.py
Each is a top-level function that takes deps as a parameter; the
engine sees thin lambdas through `register_progress_callbacks` /
`register_library_scan_completed_emitter` (called from `register_all`).
Two new deps fields:
- `init_automation_progress` (delegates into the live progress tracker)
- `record_progress_history` (delegates into _auto_progress.record_history)
12 new boundary tests in tests/automation/test_progress_callbacks.py
pin every shape:
- progress_init forwards to init_automation_progress
- progress_finish skips when handler manages its own progress
(prevents double-emit of finished status)
- progress_finish: completed -> finished/Complete/success;
error -> error/Error/error; msg falls through error -> reason ->
status -> 'done'
- record_history threads the live db into the recorder
- on_library_scan_completed: no engine = noop, server type taken
from web_scan_manager._current_server_type, defaults to 'unknown'
- register_library_scan_completed_emitter: no scan manager = noop,
registered callback emits the right event when invoked
3256 tests pass, no regression.
Final state of `_register_automation_handlers`:
- Was: 1530 lines, 21 nested closures + 4 progress callbacks
- Now: ~50 lines, builds AutomationDeps and calls register_all
web_server.py: 34,220 -> 34,187 lines (-33 net, -1,406 across the
whole branch).
Final commit of the automation-handler refactor. With this commit
every closure that used to live in
`web_server._register_automation_handlers` is now a top-level
function in `core/automation/handlers/`.
Handlers extracted in this commit:
- start_database_update + deep_scan_library
-> core/automation/handlers/database_update.py
Both share the db_update_state monitoring pattern (poll until
status flips, stall detection emits warning at 10 min, 2-hour
outer timeout). Lifted into a shared `_run_with_progress` helper
inside the module so the per-handler bodies stay tiny.
- run_duplicate_cleaner -> core/automation/handlers/duplicate_cleaner.py
- start_quality_scan -> core/automation/handlers/quality_scanner.py
- clear_quarantine, cleanup_wishlist, update_discovery_pool,
backup_database, refresh_beatport_cache
-> core/automation/handlers/maintenance.py
Grouped because each body is short (~20-50 lines) and they share
no state — splitting into per-handler files would just add import
noise.
- clean_search_history, clean_completed_downloads, full_cleanup
-> core/automation/handlers/download_cleanup.py
Grouped because all three reach the download orchestrator,
tasks_lock, and download_batches/download_tasks accessors. The
full_cleanup multi-step orchestration shares phase-detection
logic with clean_completed_downloads.
- run_script -> core/automation/handlers/run_script.py
- search_and_download -> core/automation/handlers/search_and_download.py
`AutomationDeps` grew with the new dependency surface:
- get_db_update_state + db_update_lock + db_update_executor +
run_db_update_task + run_deep_scan_task
- get_duplicate_cleaner_state + duplicate_cleaner_lock +
duplicate_cleaner_executor + run_duplicate_cleaner
- get_quality_scanner_state + quality_scanner_lock +
quality_scanner_executor + run_quality_scanner
- download_orchestrator + run_async + tasks_lock +
get_download_batches + get_download_tasks +
sweep_empty_download_directories + get_staging_path
- docker_resolve_path + get_current_profile_id +
get_watchlist_scanner + get_app + get_beatport_data_cache
- set_db_update_automation_id (writes the legacy global so the live
DB-update progress callbacks still living in web_server.py keep
emitting against the active automation card)
`web_server._register_automation_handlers` is now ~50 lines: build
deps once, call register_all. The 667-line block of remaining
closure definitions and engine register calls is gone.
The final orphan was the `_db_update_automation_id` module global —
the DB-update progress callbacks at line ~14080 still read it
directly, so the extracted database_update handler propagates the
automation id through `deps.set_db_update_automation_id` (a closure
in web_server that writes the global). When the legacy callbacks
get extracted in a future PR the setter goes away.
Tests:
- tests/automation/test_handlers_maintenance.py adds 21 boundary
tests covering every newly-extracted handler shape: guard
short-circuits (already-running returns skipped), deps wiring
(set_db_update_automation_id called with the right id),
exception swallow contract, status returns, path-traversal
blocked in run_script, source-mode skip in clean_search_history,
active-batch skip in clean_completed_downloads, etc.
- 3244 tests pass (was 3223 — 21 new), no regression.
web_server.py: 35,593 -> 34,220 lines (-1,373 net across 3 commits).
Issue #1 from the extraction punch list is now COMPLETE.
Continues the lift from `web_server._register_automation_handlers`.
This commit extracts the four playlist-lifecycle closures:
- `refresh_mirrored` -> core/automation/handlers/refresh_mirrored.py
- `sync_playlist` -> core/automation/handlers/sync_playlist.py
- `discover_playlist` -> core/automation/handlers/discover_playlist.py
- `playlist_pipeline` -> core/automation/handlers/playlist_pipeline.py
The pipeline composes refresh + sync + discover, so all four ship
together. The pipeline imports the other three handler modules
directly (cross-handler call) instead of going through the engine,
preserving the "single trigger from the user's perspective" UX.
`AutomationDeps` grew to cover the new dependency surface:
- run_playlist_discovery_worker, run_sync_task, load_sync_status_file
(pre-existing background-task entry points)
- get_deezer_client, parse_youtube_playlist (per-source clients)
- get_sync_states (live mutable accessor for the sync UI's state dict)
`web_server._register_automation_handlers` now wires those plus the
existing infrastructure into a single `AutomationDeps` and calls
`register_all`. The 669-line block of closure definitions and engine
register calls (lines 959-1627 pre-edit) is gone -- the file shed
743 lines net on this commit.
`tests/automation/test_handlers_playlist.py` adds 17 new boundary
tests:
- discover_playlist: no_id error, specific_id starts worker, all=True
enumerates, no playlists in db
- refresh_mirrored: error path, source filter (file/beatport excluded),
Spotify happy path with auto-discovered marker, per-playlist
exception captured into errors counter
- sync_playlist: no_id, not_found, no_tracks, no-discovered-tracks
skip, discovered-track happy path, unchanged-since-last-sync skip
- playlist_pipeline: no_playlist clears running flag, no-refreshable
clears running flag, exception clears running flag
3223 tests pass. web_server.py: 35,593 -> 34,850 lines (743 removed).
Begins the lift of `web_server._register_automation_handlers` (1530
lines, 20 nested closures) into `core/automation/handlers/`. Each
extracted handler is a top-level function that accepts
`(config, deps)` instead of reaching for module-level globals --
makes them unit-testable in isolation.
Infrastructure:
- `core/automation/deps.py`: `AutomationDeps` (dependency-injection
bundle of clients + callables) and `AutomationState` (mutable flags
shared across handler invocations, with thread-safe accessors).
- `core/automation/handlers/__init__.py` + `registration.py`: one-stop
`register_all(deps)` that wires every extracted handler to the
engine.
First batch of handlers extracted:
- `process_wishlist` -> `core/automation/handlers/process_wishlist.py`
- `scan_watchlist` -> `core/automation/handlers/scan_watchlist.py`
- `scan_library` -> `core/automation/handlers/scan_library.py`
`web_server._register_automation_handlers` now builds the deps once
and calls `register_all(deps)` for the extracted batch. Remaining
17 closures still live below; subsequent commits in this branch
finish the lift.
14 boundary tests in `tests/automation/test_handlers_simple.py` pin
every shape: success path, exception swallow contract, fresh-vs-stale
state detection (scan_watchlist's id() trick), guard short-circuits,
state cleanup on exceptions, AutomationState concurrent-safe accessors.
All 101 automation tests pass; no regression.
The first token-leak fix scrubbed the artwork URL fixer's own log
calls. This catches three more sites that ALSO leaked tokens, plus
one upstream gap that let URL-encoded tokens slip through the
redactor.
Three sites in `web_server.py` (artist endpoint at line 8765-8773):
- "Artist image before fix: '...'" -- logged the raw image_url with
the auth token in plain form.
- "Artist image after fix: '...'" -- logged the URL-encoded form
after it had been wrapped in the image proxy
(`/api/image-proxy?url=<percent-encoded-token>`).
- "Final artist data being sent: {...}" -- dumped the entire
artist_info dict on every render, including the image_url field.
All three were dev-time debug noise. Removed entirely. The "No
artist image URL found" warning at line 8770 stays (no URL, just
the artist name).
One site in `core/discovery/sync.py:402`:
- "[PLAYLIST IMAGE] image_url=..." -- logged the playlist poster URL
during sync. Same auth-token leak risk for Plex / Jellyfin
playlists. Changed to log only `has_image=True/False`.
Upstream gap in `_redact_url_secrets`:
- The original regex only matched plain query params (`?key=value`).
When an auth-bearing URL gets wrapped inside another URL's query
string (our `/api/image-proxy?url=<encoded>` flow) the auth params
end up percent-encoded -- `%3FX-Plex-Token%3D...` -- and slipped
through.
- New second pattern catches the URL-encoded form. Both passes run
on every redact call; idempotent.
Verified manually:
/api/image-proxy?url=...%3FX-Plex-Token%3DABC...
-> /api/image-proxy?url=...%3FX-Plex-Token%3D***REDACTED***
6 artwork tests pass.
Adds an opt-in alternative metadata source for reorganize. The
existing API path (query Spotify / iTunes / Deezer / Discogs /
Hydrabase for the canonical tracklist) stays the default and is
unchanged. The new tag mode reads each file's embedded tags as the
source of truth instead -- useful for well-enriched libraries where
API drift can produce inconsistent renames, and avoids API calls
entirely.
- New pure helper `core/library/reorganize_tag_source.py` adapts the
output of `read_embedded_tags` (the same mutagen path the audit-
trail modal uses) to the `api_album` / `api_track` shapes that
`_build_post_process_context` already consumes. Handles ID3-style
"5/12" track + disc shapes, multi-value Artists tags, year
normalization across 5 date formats, releasetype canonical tokens,
multi-artist string splits across 9 separators.
- `plan_album_reorganize` accepts `metadata_source: 'api' | 'tags'`
(default 'api') and `resolve_file_path_fn`. Tag mode branches into
a new `_plan_from_tags` that reads each track's file and produces
per-item `api_album` + `api_track` instead of a shared one.
- `_run_post_process_for_track` accepts a per-item `api_album`
override so each file's own album metadata flows through post-
process (not a single shared dict).
- `total_discs` in tag mode honors the `totaldiscs` tag and the
trailing `/N` of an ID3 `discnumber = "1/2"`. Partial-album
reorganize still routes into the correct `Disc N/` subfolder when
the tag knows the total even if not all discs are present locally.
- Bare `discnumber = "1"` no longer poisons `total_discs` -- it
carries no total signal.
- `reorganize_album` surfaces a tag-mode-specific error when no
files are readable, instead of the API-mode "run enrichment first"
message which would mislead in tag mode.
- `QueueItem.metadata_source` field, `enqueue` / `enqueue_many`
pass-through, runner injects `item.metadata_source` into
`reorganize_album`.
- `web_server.py` endpoints accept `mode` body param. Falls back to
the `library.reorganize_metadata_source` config setting, then to
'api'. Strict allowlist (api / tags) -- anything else falls back.
- Frontend: per-album modal + reorganize-all modal both grow a new
"Metadata Mode" dropdown above the source picker. Tag mode hides
the source picker (irrelevant). Choice persisted in localStorage.
Both preview + execute fetches send `mode` in body.
Tests:
- 49 boundary tests on the pure helper pin every shape: ID3 "5/12",
multi-artist split, year normalization, releasetype validation,
total_discs precedence, defensive paths.
- 6 planner-level integration tests pin the wiring: tag-mode with
good tags, partial-disc with totaldiscs tag, file missing,
some-match-some-fail, defensive resolve_file_path_fn=None,
API-mode regression guard.
- All 3171 tests pass; 52 existing reorganize tests unchanged.
Discord report (netti93). The download flow runs `enhance_file_metadata`
(clears all tags) then `generate_lrc_file` (writes .lrc sidecar AND
embeds USLT). The retag flow only ran the first half — `enhance_file_metadata`
cleared USLT and there was no follow-up to restore it.
Two coordinated fixes (no new setting per kettui scope discipline —
user described it as "might even be an idea," consistency was the
load-bearing ask).
Fix 1 — retag calls generate_lrc_file after enhance
`core/library/retag.py:execute_retag` now invokes
`deps.generate_lrc_file` right after the `enhance_file_metadata`
call, mirroring the download pipeline. New `generate_lrc_file`
field on `RetagDeps`, defaults to None for backward compat with
any test caller that builds RetagDeps without it. Web_server's
`_build_retag_deps()` factory wires in the real
`core.metadata.lyrics.generate_lrc_file`.
Placement matters — runs BEFORE `safe_move_file` so the helper
sees the audio file at its current path with its existing sidecar
(which retag hasn't moved yet). After the embed, the audio file
gets moved with USLT now present; the sidecar move step that
follows is unaffected.
Fix 2 — create_lrc_file re-embeds from existing sidecar
`core/lyrics_client.py:create_lrc_file` used to early-return True
when an .lrc / .txt sidecar already existed (skipping the LRClib
fetch). For the retag case the sidecar is already there, so the
shortcut hit and USLT was never re-written. Now the helper reads
the existing sidecar and calls `_embed_lyrics` with its content
before returning. Empty / unreadable sidecars short-circuit
silently — defensive, no crash. Download flow unaffected because
no sidecar exists at fetch time.
7 boundary tests pin: existing .lrc triggers re-embed, existing
.txt triggers re-embed, empty sidecar skips embed, unreadable
sidecar swallows error, no sidecar falls through to LRClib (download
path regression guard), RetagDeps.generate_lrc_file field accepted,
field optional for backward compat.
Full suite: 3120 passed.
Closes#585. When a Spotify source track had a versioned suffix not
present in the local file ("Iron Man - 2012 - Remaster" vs "Iron Man"),
the auto-matcher missed the pair. User could click Find & Add to pick
the right local file — that worked, file got added to the Plex
playlist — but the source track stayed in Missing while the added
file appeared in Extra, because the matcher kept no record of the
user-confirmed pairing. On the next sync the source track re-tried
to download.
Fix: every Find & Add selection now writes a (spotify_track_id →
server_track_id) override into sync_match_cache at confidence=1.0.
The matching algorithm runs an override pass BEFORE the existing
exact and fuzzy passes, so any user-confirmed pair short-circuits
straight to "matched" without going through title normalization.
Covers every mismatch class — dash-suffix remasters, covers /
karaoke, alt masters, cross-language titles, typo'd local files.
- core/sync/match_overrides.py (new) — pure helpers
resolve_match_overrides + record_manual_match. 18 boundary tests
pin: cache hits, cache misses falling through to normal matching,
stale-cache (server track removed) handled gracefully, str/int
id coercion, partial cache hits, defensive against non-dict
inputs and DB exceptions.
- web_server.py — get_server_playlist_tracks runs the override
pre-pass before exact/fuzzy matching. server_playlist_add_track
accepts source_track_id + source_title + source_artist and
persists the override after every successful add (Plex / Jellyfin
/ Navidrome). source_track_id added to source_tracks payload so
the frontend has it.
- webui/static/pages-extra.js — _serverSelectTrack sends
source_track_id + source_title + source_artist when adding a
track from a mirrored playlist context.
- Sync match cache schema unchanged — already had UNIQUE
(spotify_track_id, server_source) which fits the override
semantics perfectly. Manual overrides distinguished from
auto-discovered matches by confidence=1.0.
Full suite: 3010 passed.
Closes#584. Quarantined files used to sit in ss_quarantine/ with a
thin sidecar — no UI, no recovery, no way to see what got dropped.
This adds the management surface the user needs without going to the
filesystem.
UI: new "Quarantine" button on the downloads page header opens a
modal with every quarantined file (filename, expected track/artist,
reason, when, size). Three actions per row:
- Approve (one-click): restores the file, re-runs the post-process
pipeline with ONLY the failing check skipped, lands in the library
with full tags + lyrics + scan
- Recover (legacy fallback): moves to Staging for thin-sidecar
entries that lack the embedded context Approve needs
- Delete: permanent removal of file + sidecar
Per-check bypass: context['_skip_quarantine_check'] = 'integrity' /
'acoustid' / 'bit_depth'. Skips ONLY the named check — other quality
gates stay live. No blanket bypass-all flag.
Sidecar expansion: move_to_quarantine now persists the full
json-serializable context via serialize_quarantine_context (drops
non-JSON-safe values, walks nested dicts/lists/sets, str-coerces
unknown objects) plus the trigger name. Existing thin sidecars are
detected and routed to Recover instead of Approve.
Pure helpers in core/imports/quarantine.py: list_quarantine_entries
/ delete_quarantine_entry / approve_quarantine_entry /
recover_to_staging / serialize_quarantine_context. 27 tests pin
every shape: orphan files / orphan sidecars / corrupt sidecars /
collision-safe filename restoration / full-context vs thin-sidecar
dispatch / json round-trip safety.
Four new endpoints in web_server.py — thin glue around the helpers:
GET /api/quarantine/list, DELETE /api/quarantine/<id>,
POST /api/quarantine/<id>/approve, POST /api/quarantine/<id>/recover.
Download modal status differentiates "🛡️ Quarantined" from
"❌ Failed" so recoverable files are visible at a glance — checked
against the error_message text, no schema change needed.
Pipeline changes are three minimal per-check conditionals at the
existing quarantine sites in core/imports/pipeline.py. Each
move_to_quarantine call now passes its trigger name so the sidecar
records which check fired.
Full suite: 2992 passed.
Move the remaining manual import endpoint logic out of web_server.py and into core.imports.routes behind ImportRouteRuntime. The Flask endpoints now stay as thin compatibility wrappers for album/track search, album match/process, single-file import processing, and batched singles processing.
Keep legacy test patch points intact by re-exporting build_album_import_match_payload from web_server and routing singles_process through an injected process_single_import_file callable. This preserves existing route-level monkeypatch behavior while keeping the extracted helper testable.
Add focused helper coverage for Hydrabase enqueueing, search limit clamping, album match payload forwarding, album import side effects, single-file worker outcomes, malformed manual matches, and singles aggregation/injected-worker behavior.
Verification: py_compile and git diff --check passed locally; bundled-Python smoke covered the extracted helpers. Claude reran the project tests and reported all tests passing.
Move import staging files/groups/hints/suggestions controller logic out of web_server.py and into core.imports.routes behind an ImportRouteRuntime dependency object. Keep the existing Flask routes as thin compatibility wrappers so the UI endpoint surface stays unchanged.
Add focused tests for staging file filtering, album grouping, hint generation, cached suggestions, empty missing staging paths, and error payloads from failed path/metadata reads.
Verification: py_compile passed for web_server.py, core/imports/routes.py, and tests/imports/test_import_routes.py. A bundled-Python smoke pass covered the extracted helper behavior; pytest was not available in this Windows shell because the bundled Python lacks pytest and the repo venv is WSL/Linux-only here.
Discord report: prolific artists (Bach, Beatles complete box,
deep dance/electronic catalogues) only showed ~50 entries in the
"Download Discography" modal.
`MetadataLookupOptions(limit=50, max_pages=0)` was hardcoded at
three call sites. Spotify's `max_pages=0` already paginates
through everything (per-page is clamped to 10 internally), so
Spotify-primary users were unaffected. But Deezer / iTunes /
Discogs / Hydrabase all honor the outer `limit` as a hard cap,
so non-Spotify users were silently clipped.
Bump `limit` to 200 at all three call sites — matches iTunes's
and Discogs's own internal caps and covers near-everyone's full
catalogue. Spotify behavior unchanged.
- web_server.py:9221 — discography endpoint (modal)
- web_server.py:8700 — artist-detail discography view
- core/artist_source_detail.py:129 — source-specific artist detail
- Replace the shell convenience script with a cross-platform Python launcher.
- Keep dev.sh as a Unix compatibility wrapper.
- Let the direct backend bind with host and port overrides.
- Update the root and webui README guidance for the new launcher.
- Preserve the backend startup behavior used by the old dev flow.
- Drop unused _resolve_webui_initial_* helpers from web_server.py.
- Remove template-side initial_nav_page and initial_client_page conditionals.
- Keep Vite asset injection and runtime page activation in the client.
Remove the Flask route-to-page helpers and stop passing initial active-page flags into the shell template.
The web UI now renders static page and nav markup, while the client-side shell remains responsible for establishing active page state after load. This keeps the hybrid Flask + Vite asset setup intact while reducing duplicated route/page ownership logic in the backend template layer.
Also added a previously missing /stream path to the spa exclusions
- File-based routing with tanstack router
- Persist top-level navigation state in url, even for most legacy pages
- Striving for an intuitive and simple folder structure where
route-related code is colocated, but the amount of files is still
kept to a minimum
- Replace native fetch with `ky`
- Familiar api, but more polished
Closes#572 (rhwc).
Navidrome has no API for setting an artist image — it reads
`artist.jpg` (or `folder.jpg`) from the artist folder during
library scans. SoulSync's `update_artist_poster` for Navidrome
was a no-op, so users only ever saw album-art-derived thumbnails
as the artist photo.
- new "Write Artist Image" button on artist detail page
- POST /api/artist/<id>/write-image-to-disk derives the artist
folder from any track's resolved file_path (reuses
_resolve_library_file_path so docker mount translation +
library.music_paths probes from #558 apply), fetches the photo
from the configured metadata source priority chain, downloads
with content-type validation, writes atomically via
`<filename>.tmp + os.replace`
- when active server is Navidrome, triggers a library scan
immediately so the file is picked up
- respects existing artist.jpg (frontend prompts before
overwriting) so user-supplied photos aren't clobbered
- works for plex / jellyfin too as a fallback layer — both
servers also read artist.jpg from disk
26 tests pin the pure helpers in core/library/artist_image.py:
folder derivation (trailing sep / empty / non-string), URL
picking (missing attr / whitespace / non-string), download
(non-image content-type / 404 / timeout / empty body), atomic
write (replace / temp-cleanup-on-failure / overwrite guard /
missing folder).
- new "Audit" button on each download row in the library history
modal opens a second modal visualizing the download lifecycle as
an interactive horizontal stepper (request → source → match →
verify → process → place) with click-to-expand detail cards
- hero header with album art + track title + meta line + status
pills (source / quality / acoustid result)
- three tabs: Lifecycle / Tags / Lyrics
- Tags tab reads the audio file live via mutagen at audit-open
time via new GET /api/library/history/<id>/file-tags endpoint;
file is the single source of truth so background enrichment
writes (audiodb / lastfm / genius / replaygain / lyrics fetch)
show up too. flat key/value rows stacked vertically (label-above-
value) so long MBIDs / URLs / joined genre lists wrap cleanly.
source IDs grouped per-service into 2-col sub-card grid.
- Lyrics tab renders the full transcript with dimmed timecodes.
- post-processing step infers observable changes from source-vs-
final state (format conversion, file rename via tag template,
folder template).
- "Download History" button also added to the Downloads page batch
panel header so it's reachable outside the dashboard.
- mobile responsive: tabs + stepper scroll horizontally, modal
goes full-screen, hero stacks below 480px.
19 helper tests pin the mutagen reader: id3 (TIT2/TPE1/TALB + TXXX
+ USLT + APIC), vorbis (FLAC dict + _id/_url passthrough), file
metadata (format / bitrate / duration), defensive paths (empty /
missing file / mutagen returns None / mutagen raises), stringify
edge cases (list / tuple / int / frame-with-text / whitespace).
- new track_already_owned helper wraps db.check_track_exists at
the same confidence threshold the discography backfill repair job
uses (0.7) — name+artist+album, format-agnostic so blasphemy-mode
libraries (flac → mp3 + delete original) match correctly
- endpoint runs the check after the artist + content-type filters and
before add_to_wishlist, so a second discography click on the same
artist no longer re-queues every track that already downloaded
- per-album response carries a new tracks_skipped_owned counter
alongside the existing artist/content/wishlist skip categories
Discord report (Skowl).
- drop tracks where the requested artist isn't named in track.artists
(keeps features, drops compilation / appears_on contamination)
- honor watchlist.global_include_live/remixes/acoustic/instrumentals
the same way the discography backfill repair job already does
- surface per-album skip counts in the ndjson stream (artist mismatch
+ content filter) so the ui can show what was filtered
Closes#559.
Reported: Clear History button on the Import page left zombie rows
behind. Every survivor showed "⧗ Processing" status from 2-9 days ago.
Trace: `_record_in_progress` inserts a `status='processing'` row up-front
so the UI can render the in-flight import while it runs; `_finalize_result`
updates it to `completed`/`failed` when the import finishes. When the
worker is killed mid-import (server restart, crash), the row never gets
finalized — stays at `processing` forever. The clear-history endpoint's
SQL `DELETE ... WHERE status IN (...)` listed every terminal status but
omitted `processing`, so zombies survived every click.
Fix: add `processing` to the delete list, but guard against nuking
genuinely-live imports by intersecting against the worker's
`_snapshot_active()` map — any folder hash currently registered in
`_active_imports` is excluded from the delete via an `AND folder_hash
NOT IN (...)` clause. `pending_review` deliberately left out so user
still has to approve/reject those explicitly.
One endpoint touched (`/api/auto-import/clear-completed` in
web_server.py). No worker changes — guard reuses the existing
`_snapshot_active()` method that the UI poller already calls.
5 new tests in `tests/imports/test_auto_import_clear_completed_endpoint.py`:
- Zombie `processing` rows swept, live `processing` row preserved
(folder_hash currently in `_active_imports` survives)
- Response count matches actual delete count
- Empty active-set branch (unparameterized DELETE) — pinned because
an empty SQL `IN ()` would be a syntax error
- Worker-unavailable returns 500 (pre-existing guard not regressed)
- `pending_review` rows always survive — never auto-swept
Full pytest sweep: 2758 passed (one pre-existing flaky timing test
on `test_import_singles_parallel.py` failed under full-suite CPU load,
passes in isolation in 2.95s — unrelated to this change).
Two-part fix to the Your Albums "Download Missing" flow on Discover.
Part A — UX redesign
The prior `downloadMissingYourAlbums()` ran a per-album loop that
fired direct-download tasks via `openDownloadMissingModalForYouTube`.
Reported as silently failing — "Queuing 2/2" toast with no actual
transfer activity. Even when downloads worked, bypassing the
wishlist meant no retry / dedup / rate-limit / source-fallback
handling.
Replaced with a selectable-grid modal mirroring the Download
Discography pattern from the library page. Click the download
button → opens a checkbox grid showing every missing album (cover,
title, artist, year, track count, source) → user picks what they
actually want → click "Add to Wishlist" → each album's tracks get
resolved + queued through the existing wishlist auto-download
processor. NDJSON progress stream renders ✓/✗ per album.
New JS helpers:
- `_openYourAlbumsBatchModal(missingAlbums)` — builds the modal
- `_renderYourAlbumsBatchCard(row, index)` — per-album card
- `_yourAlbumsBatchSelectAll(select)` — bulk toggle
- `_updateYourAlbumsBatchFooterCount()` — live count + button text
- `_closeYourAlbumsBatchModal()` — overlay teardown
- `_startYourAlbumsBatchAddToWishlist()` — submit handler, NDJSON
progress consumer
- `_yourAlbumsPickSource(album)` — picks the single best source-id
per row (priority: spotify → deezer → tidal → discogs)
Reuses the `.discog-*` CSS classes from the library Download
Discography modal — no new CSS. Reuses the existing
`/api/artist/<id>/download-discography` endpoint. The endpoint's URL
artist_id param is functionally unused (per-album payload carries
everything — verified by reading the endpoint body), so the modal
posts with placeholder `your-albums` and gets multi-artist
resolution for free without backend changes.
Part B — Tidal album resolution
Reported as the original bug: clicking download on Tidal-only albums
did nothing because `/api/discover/album/<source>/<album_id>` had no
`tidal` branch and `tidal_client` had no `get_album_tracks` method.
`core/tidal_client.py`: new `get_album_tracks(album_id, limit=None)`
method. Two-phase: cursor-walk
`/v2/albums/<id>/relationships/items?include=items` for track refs +
position metadata (`meta.trackNumber` + `meta.volumeNumber`),
batch-hydrate via existing `_get_tracks_batch` for artist/album
names. Returns `Track` objects with `track_number` and `disc_number`
attached. Sort by (disc, track) so multi-disc compilations render in
album order.
`web_server.py`: new `'tidal'` source branch in
`/api/discover/album/<source>/<album_id>`. Resolves album metadata
via `get_album`, tracks via `get_album_tracks`, cover art via inline
`?include=coverArt` lookup. Same response shape as Spotify/Deezer
branches.
`webui/static/discover.js`:
- `tidal_album_id` added to `trySources` for the single-album click
flow (`openYourAlbumDownload`)
- Same source picker drives the new batch modal
- Virtual-id generation includes `tidal_album_id` so Tidal-only
albums get stable identifiers across discover-album-* / your-
albums-* contexts
10 new tests in `tests/test_tidal_album_tracks.py` pin:
- Single-page walk + hydration
- Multi-page cursor chain
- Multi-disc sort order (disc 1 → 2 in track order each)
- `limit` short-circuit at page boundary
- No-token short-circuit (no API call)
- HTTP error returns empty
- 429 raises (propagates to `rate_limited` decorator for retry)
- Forward-compat type filter (skips non-track entries)
- Partial-batch hydration failure containment
- Empty-album short-circuit (no batch call)
Full pytest: 2693 passed.
Discord report (CJFC, 2026-04-26): syncing a Spotify playlist to the
server overwrote anything manually added to the server-side playlist.
The fix adds a per-sync mode picker next to the Sync button on the
playlist details modal — Replace (default, current delete-recreate
behavior) or Append only (preserves existing tracks, only adds new
ones). Useful when the source platform caps playlist size and the
user is manually building beyond it on the server.
Implementation:
* New `append_to_playlist(name, tracks)` method on Plex / Jellyfin /
Navidrome clients. Each uses the server's NATIVE append API:
- Plex: `existing_playlist.addItems(new_tracks)`
- Jellyfin: `POST /Playlists/<id>/Items?Ids=...&UserId=...`
- Navidrome: Subsonic `updatePlaylist?songIdToAdd=...`
Falls back to `create_playlist` when the playlist doesn't exist
yet (first sync). No delete-recreate, no backup playlist created
(preserves playlist creation date + metadata + non-soulsync-managed
tracks).
* Dedup-by-server-native-id (ratingKey for Plex, GUID for Jellyfin,
song-id for Navidrome) — never re-adds a track already on the
playlist. Server-native identity, not fuzzy title+artist match,
so it can't false-collide.
* `sync_service.sync_playlist` accepts `sync_mode='replace'|'append'`
kwarg. Single if/else branch dispatches to `append_to_playlist` or
`update_playlist`. Threaded through `core/discovery/sync.run_sync_task`
and the `/api/sync/start` HTTP handler. Validation on the API rejects
unknown mode strings (defaults to 'replace').
* Frontend: per-playlist `<select id="sync-mode-${id}">` rendered next
to the Sync button in both modal renderers (sync-spotify.js for
Spotify playlists, sync-services.js for Deezer ARL playlists).
`startPlaylistSync` reads the select at click time; missing select
(other callers like discover.js) defaults to 'replace' so backward
compat preserved without per-call-site updates.
* SoulSync standalone has no playlist methods at all and the modal
hides the Sync button entirely on it via `_isSoulsyncStandalone` —
dispatch never reaches that path, no defensive fallback needed.
15 new tests pin per-server append behavior:
- missing playlist → create_playlist delegation
- dedup filtering (existing IDs skipped, only new tracks added)
- empty new-track set short-circuits without API call
- failure paths return False without raising
- contract listing (KNOWN_PER_SERVER_METHODS includes
'append_to_playlist'; Plex / Jellyfin / Navidrome all implement)
Plus tests/discovery/test_discovery_sync.py fake `sync_playlist`
fixture got `sync_mode='replace'` default to match the new signature
(was breaking after the kwarg add; now passing).
WHATS_NEW entry under new '2.6.0' block (hidden by
`_getLatestWhatsNewVersion` until next release bump).
Closes CJFC discord request.
Adds the user's Tidal favorited tracks ("My Collection" in the Tidal
app) as a virtual playlist alongside their real playlists, mirroring
how Spotify's "Liked Songs" is treated.
Reporter (yug1900) located the working endpoint after the prior
`/v2/favorites?filter[type]=TRACKS` attempt returned empty data —
that endpoint is scoped to collections the third-party app created
itself, not personal favorites. Real endpoint:
GET /v2/userCollectionTracks/me/relationships/items
?countryCode=US&locale=en-US&include=items
Cursor-paginated (20 per page, follow `links.next` with
`page[cursor]=...` until exhausted). Response only carries
track-level attributes — artist + album NAMES come back as
relationship-link stubs, not embedded data.
Implementation:
* Two-phase fetch — `_iter_collection_track_ids` walks the cursor
chain to enumerate every track id (cheap, IDs only), then
`get_collection_tracks` batch-hydrates 20 IDs at a time through
the existing `_get_tracks_batch` helper which already knows how
to `include=artists,albums`. No duplication of the JSON:API
artist/album parse, no new dataclass shape.
* Virtual playlist `tidal-favorites` appended to the end of
`/api/tidal/playlists`. ID intentionally has no colon —
sync-services.js renderer interpolates IDs into CSS selectors
via template literals (`#tidal-card-${p.id} .foo`) and a `:`
would parse as a CSS pseudo-class operator.
* `tidal_client.get_playlist("tidal-favorites")` recognizes the
virtual id and dispatches to the collection path internally, so
every per-id consumer gets it for free: detail endpoint, mirror
auto-refresh automation, "build Spotify discovery from Tidal
playlist" flow.
OAuth scope expansion:
* Added `collection.read` to both OAuth flows (the
`core/tidal_client.py::authenticate` standalone path AND the
`web_server.py::auth_tidal` web flow — they were independent
scope strings that both needed updating).
* Added `prompt=consent` to both flows — without it Tidal silently
returns a token carrying only the ORIGINAL scope set even after
re-authentication, because Tidal treats the existing
authorization as still valid.
* New `disconnect()` method + `POST /api/tidal/disconnect`
endpoint + Disconnect button next to Authenticate in Settings →
Connections → Tidal — required for users whose existing token
predates the scope expansion (forces a clean grant).
Reconnect-needed UI hint:
* `_collection_needs_reconnect` flag set on 401/403 from the
collection endpoint, cleared on next successful walk, NOT set
on 5xx (transient server errors must not falsely tell the user
to reconnect).
* Listing endpoint reads the flag and surfaces a placeholder card
titled "Favorite Tracks (reconnect Tidal to enable)" with a
description pointing at Settings, so the user has something
visible to act on instead of a silently missing row.
Diagnostic logging — collection request URL + response status +
first 300 bytes of body now logged at info level so future "why
is my collection empty" reports can be diagnosed from app.log
without needing live reproduction.
22 new tests pin: cursor walk (full chain, max-ids cap mid-page +
at page boundary), auth gates (no token / 401 / 403 all bail
clean), reconnect-flag lifecycle (set on 401/403, cleared on next
successful walk, NOT set on 5xx), forward-compat type filter
(non-track entries skipped), count helper, batch hydration
delegation + chunking at the 20-per-batch cap, partial-batch
failure containment, virtual-id dispatch (real playlist ids still
flow through the normal path).
Closes#502.
Live-API verification revealed advanced-syntax queries hurt more
than they help on this endpoint. Switching the import-modal Deezer
search back to free-text + local rerank.
# What live testing showed
Hit Deezer's public API with both query forms for the issue #534
case (`Dirty White Boy` + `Foreigner`):
**Free-text (`q=Dirty White Boy Foreigner`):**
- Returns 21 results
- Real Foreigner Head Games studio cut at #1
- Live versions at #2-10
- Karaoke / cover variants at #11-15
**Advanced (`q=track:"Dirty White Boy" artist:"Foreigner"`):**
- Returns 12 results
- "(2008 Remaster)" at #1 — canonical Head Games cut MISSING from
top 8 entirely
- Live + alt-album versions follow
Advanced syntax DOES filter karaoke at the API level (none in the
12-result set vs. 5 at positions 11-15 in free-text), but it has
its own ranking bias that surfaces remasters / "Best Of" cuts
ahead of the canonical recording. Net regression for the user-
facing goal.
# Fix
1. Endpoint reverts to free-text query with local rerank applied.
2. Local rerank gains "remaster" / "remastered" / "reissue"
patterns under VARIANT_TAG_PATTERNS (soft 0.4× penalty — user
may want them but they shouldn't outrank the original).
3. Client kwarg support (`track=` / `artist=` / `album=`) preserved
for future opt-in callers (e.g. exact-match flows where API-
level filtering matters more than ranking).
# Verified end-to-end against live Deezer API
Re-ran the exact #534 case through the live API + new rerank.
Top 15 results post-rerank:
1. Dirty White Boy — Foreigner — Head Games ← REAL CUT AT TOP
2-10. Various Live versions
11-15. Karaoke / cover / tribute variants ← BURIED
Real Foreigner Head Games studio cut at #1, exactly the user's
ask.
# Tests
- `test_relevance.py` — variant tag patterns extended; existing
tests still pass (50 tests).
- `test_search_match_endpoints.py::test_joins_track_and_artist_into_free_text_query`
— replaces `test_passes_track_and_artist_as_kwargs`; verifies
endpoint sends free-text join, NOT field-scoped kwargs (the
prior test asserted the wrong direction now).
- Karaoke-burying assertion at the endpoint still pins the
user-visible behaviour.
- Client kwarg path tests untouched (still pin advanced-syntax
construction for future opt-in callers).
# Verification
- 75 relevance + endpoint + query tests pass
- 2445 full suite passes
- Ruff clean
- Live Deezer API shows real cut at #1 post-rerank
# Background
User reported (#534) that the import-modal "Search for Match" dialog
returned irrelevant results when Deezer was the metadata source.
Searching `Dirty White Boy` + `Foreigner` returned 5+ karaoke /
"originally performed by" / "in the style of" / "re-recorded" /
tribute-band results ranked above the actual Foreigner studio cut
from Head Games. User had to scroll past the junk every time, or
fall back to iTunes search which is much slower.
# Root cause — two layers
1. **Endpoint joined `track + artist` into free-text query.**
`/api/deezer/search_tracks` was passing `q=Dirty White Boy Foreigner`
to Deezer's `/search/track` API. Deezer fuzzy-matches that
string across title / lyrics / artist / album / contributors and
orders by global popularity — anything that appears across many
compilations outranks the canonical recording.
2. **No local rerank.** None of the search-modal endpoints applied
any post-filtering. Deezer's API order shipped straight to the
user.
# Fix — same architectural shape Cin would build
## Layer 1: field-scoped query at the client boundary
`core/deezer_client.py::search_tracks()` now accepts optional
`track`, `artist`, `album` kwargs. When provided, builds Deezer's
advanced search syntax: `q=track:"X" artist:"Y" album:"Z"`. Massive
relevance improvement because each term matches the right field
instead of fuzzy-matching everywhere.
Backward compat preserved: legacy free-text `query=` callers still
work unchanged. Field-scoped path takes precedence when both are
provided. Empty input fast-fails without an API call. Embedded
double-quotes stripped (Deezer's syntax has no escape mechanism).
## Layer 2: provider-neutral relevance reranker
New `core/metadata/relevance.py` module — pure-function rerank over
the canonical `Track` dataclass. Composable scoring:
- **Cover/karaoke patterns** (multiplier 0.05, effectively buries):
matches "karaoke", "originally performed by", "in the style of",
"made famous by", "tribute", "vocal version", "backing track",
"cover version", "re-recorded", "cover by", etc. across title,
album, AND artist fields. Catches the screenshot's exact junk:
artist credits like "Pop Music Workshop" / "The Karaoke Channel"
/ "Foreigner Tribute Band".
- **Variant tags** (multiplier 0.4): live / acoustic / demo /
instrumental / remix / radio edit / club mix etc. — softer
penalty since the user MAY want them. Skipped entirely when the
expected_title contains the same tag (so searching
"Track (Live)" still ranks Live versions first).
- **Exact artist boost** (multiplier 1.5): primary artist exactly
matches expected_artist after normalisation. Single strongest
signal for "this is the canonical recording".
- **Title + artist similarity** via SequenceMatcher (parentheticals
+ punctuation stripped before comparison).
- **Album-type weighting**: album=1.0 > single/ep=0.85 > compilation=0.7.
Compilations are more likely tribute / karaoke repackages.
Each component is a standalone function so tests pin them
individually without standing up the full pipeline.
## Wired at three search-modal endpoints
- `/api/deezer/search_tracks` — uses both layers (field-scoped
query + rerank).
- `/api/itunes/search_tracks` — uses rerank only (iTunes API has
no advanced-syntax search, but karaoke / cover variants still
leak through and need the local penalty).
- `/api/spotify/search_tracks` — already builds field-scoped
`track:X artist:Y` query; rerank added as the consistency safety
net so all three sources behave the same from the user's
perspective.
Other Deezer call sites (matching engine, watchlist scanner,
auto-import single-track ID) deliberately not touched in this PR
— they have their own elaborate scoring pipelines tuned to their
specific contexts and aren't surfacing the user-reported issue.
Per Cin: "don't refactor beyond what the task requires."
# Tests
71 new tests across 3 files:
- `tests/metadata/test_relevance.py` (50 tests) — every scoring
component pinned individually + the issue #534 screenshot
reproduced as a regression test (real Foreigner cut wins after
rerank, karaoke variants drop to bottom).
- `tests/metadata/test_deezer_search_query.py` (14 tests) —
advanced-syntax query construction, field-scoped wiring at the
client boundary, free-text path unchanged, kwargs win when
ambiguous, limit clamping, cache key consistency.
- `tests/imports/test_search_match_endpoints.py` (7 tests) —
end-to-end through Flask test client: Deezer endpoint passes
kwargs not joined query; karaoke buried at bottom for all three
sources; legacy query param still works without rerank.
# Verification
- 2441 full suite passes (+71 from baseline 2370)
- 0 failures (the prior watchdog flake fix held)
- Ruff clean across all changed files
- JS parses clean (`node -c webui/static/helper.js`)
# Architectural standards followed
- **Logic at the right boundary.** Query construction lives in the
client (every caller benefits from one change). Rerank lives in
a neutral module (`core/metadata/relevance.py`) over the
canonical `Track` dataclass — works for any source, not Deezer-
specific.
- **Explicit > implicit.** Every scoring rule has its own named
function. Pattern tables are module-level constants tests can
introspect.
- **Scope discipline.** Audited every Deezer search call site;
fixed the user-reported one + the consistent siblings. Did NOT
speculatively normalise every Deezer call across the codebase.
- **Backward compat.** Free-text `query=` callers untouched. Kwargs
added to existing client method signature with safe defaults.
- **Tests pin contract at correct boundary.** Pure-function rerank
tests don't mock anything; client-query tests stub at `_api_get`;
endpoint tests run through the real Flask app.
# Concurrency model
Pre-refactor concurrency was emergent + unbounded:
- The worker's `_run` thread called `_scan_cycle` every 60s,
processing candidates synchronously in a for-loop.
- The `/api/auto-import/scan-now` endpoint spawned a fresh
`threading.Thread(target=_scan_cycle)` per click — extra parallel
scan cycles on top of the timer.
- Multiple "Scan Now" clicks during in-flight processing → multiple
threads racing on `_processing_paths` / `_folder_snapshots` state,
no upper bound on concurrent scanners.
- `stop()` didn't wait for in-flight processing — could leave file
moves / tag writes / DB inserts mid-flight.
Refactor to the pattern Cin uses elsewhere (`missing_download_executor`,
`sync_executor`, `import_singles_executor` all use
`ThreadPoolExecutor(max_workers=3, thread_name_prefix=...)`):
- **One scan thread** — both timer + manual triggers go through
`trigger_scan()`, gated by a non-blocking `_scan_lock`. Duplicate
triggers no-op instead of stacking parallel scanners.
- **Bounded executor** — `ThreadPoolExecutor` (default 3 workers,
configurable via `auto_import.max_workers`) runs per-candidate
work. Each candidate runs to completion in its own pool thread;
up to N candidates run in parallel.
- `_scan_and_submit()` is fast — just enumeration + executor submit,
returns immediately, doesn't block on per-candidate work.
- `_process_one_candidate(candidate)` holds the per-candidate logic
identical to the old for-loop body, lifted into a method so the
pool can run multiple instances concurrently.
- `_submitted_hashes` set + lock dedupes candidates across the
timer + manual triggers so a candidate already queued / running
doesn't get re-submitted.
- `stop()` calls `executor.shutdown(wait=True)` — clean shutdown,
no orphaned file ops.
# Per-candidate UI state isolation
The executor refactor opened two concurrency holes that the old
sequential model masked. Both fixed in this commit:
1. **Scalar UI fields stomped across pool workers.** Pre-refactor
`_current_folder` / `_current_status` / `_current_track_*` were
safe under the sequential model — only one candidate processed
at a time, so the fields tracked the in-flight one. With three
pool workers writing the same fields, the polling UI saw garbage
like "Processing AlbumA, track 7/14: SongFromAlbumB".
Replaced with `_active_imports: Dict[hash, _ActiveImport]` keyed
on folder_hash, gated by `_active_lock`. Each pool worker owns
its own entry. Helpers `_register_active` / `_update_active` /
`_unregister_active` / `_snapshot_active` are the only API.
2. **Stats counters not thread-safe.** `self._stats[k] += 1` is
read-modify-write — under load, parallel pool workers drop
increments. New `_stats_lock` + `_bump_stat()` helper wraps every
mutation. `get_status()` reads under the same lock and returns
a copy.
# Endpoint change
`/api/auto-import/scan-now` no longer spawns its own scan thread —
calls `auto_import_worker.trigger_scan()` (which routes through the
shared lock + executor). Multiple clicks while a scan is in flight
no-op deterministically. Endpoint still wraps the call in a daemon
thread so the HTTP response returns immediately even if the staging
walk is slow.
# Backward compat
The scalar `_current_folder` / `_current_status` / `_current_track_*`
fields are preserved as **read-only properties** that resolve to the
FIRST active import. The existing `get_status()` payload still
includes those fields populated from the first entry — single-import
UIs (and the test fixture) keep working unchanged. New
`active_imports` array exposes the full multi-candidate state for
parallel-aware UIs.
# Behavior preserved
- Per-candidate identify / match / process logic byte-identical
- Live-progress state preserved (per candidate now)
- Stability gate / already-processed dedup preserved
- `_record_in_progress` / `_finalize_result` UI rows preserved
- Tag-based loose-file grouping unchanged
# Behavior changes
- Multiple albums process IN PARALLEL up to `max_workers`
- "Scan Now" while scan in progress no-ops (was: spawned another)
- `stop()` waits for in-flight pool work via `shutdown(wait=True)`
- Auto-import card now lists each in-flight album (one line per
active import) instead of a single shared progress line
# UI
`webui/static/stats-automations.js`:
- Progress widget reads `active_imports` array, renders one line
per in-flight album with per-candidate status / track index
- Falls back to the legacy summary line when payload doesn't
carry `active_imports` (older backend)
- Per-row "live processing" lookup now matches by `folder_hash`
through the array instead of by `folder_name` against scalars
# Tests added (`tests/imports/test_auto_import_executor.py`)
- Pool config: default max_workers=3, configurable via constructor
+ via `auto_import.max_workers` config, floors at 1
- Scan lock: 5 concurrent `trigger_scan()` calls run only 1 scan
while lock held; releases properly so subsequent triggers run
- Executor dispatch: 5 candidates → 5 process calls via the pool
- Bounded parallelism: max_workers=3 caps at 3 concurrent;
max_workers=2 caps at 2
- Cross-trigger dedup: candidate submitted in scan A doesn't get
re-submitted by scan B while still in-flight
- Graceful shutdown: `stop()` blocks until in-flight pool work
finishes
- Per-candidate state isolation: 2 parallel workers updating their
own candidate state don't interfere — each candidate's
track_index / track_name / folder_name reads back exactly as
written for that hash
- `get_status()` returns coherent `active_imports` array with
one entry per in-flight candidate; aggregate top-level
`current_status` is 'processing' when any entry is processing
- Unregister removes only that candidate, others stay visible
- Stats counter thread-safety: 1000 parallel bumps land at 1000
(the read-modify-write race regresses without the lock)
- `get_status()` stats snapshot is a copy, not a live reference
# Verification
- 17 new tests pass (executor + state isolation)
- 2347 full suite passes (1 pre-existing flaky test —
`test_watchdog_warns_about_stuck_workers` — passes in isolation,
unrelated)
- Ruff clean
radoslav-orlov reported every imported album landing in the soulsync
standalone library as "Unknown Artist" + the raw 10-digit album id
as the title + 0 tracks. Audit traced it to the click handler in the
import page dropping the source-of-the-album_id on its way to the
backend match endpoint.
Root cause:
`importPageSelectAlbum(albumId)` (the onclick on every suggestion /
search-result card) only passed the album_id string. The full search
response carried `source`, `name`, and `artist` per row — the
backend's `get_artist_album_tracks` needs source so it can route the
lookup to the metadata source the id actually came from. Without it,
the source chain tries each source's `get_album(id)` against an id
shaped for a different source — a Deezer numeric id against
Spotify's id format returns 404, against iTunes's collectionId range
returns 404, etc. — and falls through to the failure-fallback dict
in `get_artist_album_tracks`:
{
'success': False,
'album': {'name': album_name or album_id, 'total_tracks': 0,
'release_date': '', ...}, # no artist field at all
'tracks': [],
}
That broken album dict then flowed through `build_album_import_context`
→ post-processing pipeline → `record_soulsync_library_entry`, writing
"Unknown Artist" + album_id-as-title + 0 tracks rows into the
soulsync standalone library tables.
Why hybrid users hit it most: a Spotify-primary user searching for an
album → search returns the Spotify result PLUS Deezer fallbacks
(via `_search_albums_for_source`'s priority chain). Clicking a Deezer
fallback row then sent only the Deezer id to /album/match without
flagging that source — Spotify-first chain failed against the Deezer
id and the broken fallback got written.
Fix:
Frontend (`webui/static/stats-automations.js`):
- New `importPageState._albumLookup: { albumId: { id, name, artist,
source } }` populated by both card renderers (`_renderSuggestionCard`
+ the search-results render block) before they emit the onclick.
- `importPageSelectAlbum` reads source / name / artist from that
cache and includes them in the match POST body, so the backend
routes to the correct provider's `get_album` on the very first try.
- `_escAttr` applied to album_id in the onclick (defensive — ids
shouldn't contain quotes but `_escAttr` was already being used on
every other field interpolated into onclick attributes).
Backend (`web_server.py:import_album_match`):
- Defensive log warning when source is missing from the request body.
Catches any future regression where another caller (curl /
third-party / new UI flow) drops source again — it'll show up as
a visible warning in app.log instead of silently corrupting the
library.
Verification:
- Full pytest suite: 2264 passed, 1 skipped, 0 failed
- Ruff clean
- JS syntax clean
- Manual repro requires a real user flow (search albums on the
import page → click one → import) which isn't covered by the
existing unit tests; reviewer should verify against issue #524's
steps before merge.
Three follow-on fixes to the manual-search candidates modal once people
started actually using it:
1. NDJSON streaming. Manual search waited for every source to return
before showing anything. Now streams one event per source as each
completes — header line, source_results per source, done terminator.
Frontend appends rows incrementally via response.body.getReader().
2. Manual picks no longer auto-retry on failure. New _user_manual_pick
flag set on the task in /download-candidate. Both monitor retry
paths (not-in-live-transfers stuck + Errored state) bail on the
flag. Surfaces the failure to the user instead of silently picking
a different candidate via fresh search.
3. Non-Soulseek manual picks (youtube/tidal/qobuz/hifi/deezer/
soundcloud/lidarr) no longer stuck at "downloading 0%" forever. The
live_transfers IF branch now marks manual-pick tasks failed
directly when the engine reports Errored, instead of deferring to
the monitor (which bails on manual picks). Engine fallback in else
branch covers the rare race where the orchestrator's pre-populated
transfer lookup is missing the entry.
Plus a deadlock fix discovered along the way: the new failure path
synchronously called on_download_completed while holding tasks_lock,
which itself re-acquires the same Lock — non-reentrant
threading.Lock self-deadlocked the polling thread. While wedged, every
other endpoint that needed the lock (including /candidates → other
failed rows couldn't open modals) hung waiting. Moved completion
callbacks onto a daemon thread so the lock releases first.
Plus failed/not_found/cancelled rows are now ALWAYS clickable (not
just when the auto-search cached candidates) — the modal carries the
manual search bar, which is the user's recourse for empty results.
Plus manual download worker now runs on a dedicated thread instead of
competing with the batch's 3-worker missing_download_executor pool —
saturated batches no longer queue manual picks indefinitely.
All scoped to manual picks via the _user_manual_pick flag — auto
attempt flow byte-identical to before. Engine fallback gated on the
flag too so auto attempts in the else branch keep the original
do-nothing behavior (safety valve handles the stuck-forever case).
Also dropped _handle_failed_download from web_server.py — defined
but had no callers (dead code).
17 new unit tests pin the gate behavior:
- engine fallback: Errored/Cancelled/Succeeded/InProgress transitions,
manual-pick gate, terminal-state skip, soulseek skip, missing
download_id skip, engine returning None, orchestrator exception
- monitor: manual-pick skips not-in-live-transfers retry + Errored
retry
- IF-branch end-to-end: Errored marks failed, "Completed, Errored"
hits failure branch, auto attempts defer to monitor
Manual-search endpoint tests rewritten for NDJSON: 11 cases (validation,
single-source dispatch, parallel "all" dispatch, one-event-per-source
streaming shape, unconfigured-source skip + reject, header metadata,
per-source exception isolation).
Full suite 2259 passed, 1 skipped.
When an auto-download fails or returns "not found" with leftover
candidates, the user can already click the status cell to open a
modal showing those candidates and pick a different one. This adds
a manual search bar to that modal — type any query, hit search,
get a fresh round of results without having to bail out and start
over from the main search page.
Solves the case where the auto-query was bad (featured artist not
in title, parentheticals like "(Remastered 2019)" tripping the
matcher, slight artist-name variants, transliteration) but the
file genuinely exists on the source.
Frontend (downloads.js)
- Added a manual-search section above the existing auto-candidates
table inside the candidates modal.
- Source picker is smart per download mode:
- Single-source mode (soulseek-only / youtube-only / etc) shows
a "Searching X" label, no dropdown.
- Hybrid mode shows a dropdown with "All sources" default + every
configured source. Picking "All" runs parallel searches across
them and tags each result row with its source badge.
- Only configured sources show up; unconfigured are hidden.
- Validation: button disabled until query length >= 2, "Type at
least 2 characters" hint until threshold crosses.
- Loading state on search button while the request is in flight.
- Manual results render in a separate table above the existing
auto-candidates table, using the same row template (file /
quality / size / duration / user / ⬇ button) so the renderer
helper is shared.
- Click ⬇ reuses the existing `downloadCandidate(taskId, candidate,
trackName)` flow — same retry path, same AcoustID verification
when the file lands, no shortcut around the safety net.
- Re-running the search with a different query replaces the
previous manual results.
Backend (web_server.py)
- Extended `GET /api/downloads/task/<id>/candidates` response with:
- `download_mode` (e.g. 'hybrid', 'soulseek')
- `available_sources` (list of configured source IDs + labels)
- `source` field on each candidate (purely additive — frontend
auto-renderer ignores it on legacy code paths, manual-search
renderer uses it for the badge)
- Added `POST /api/downloads/task/<id>/manual-search`:
- Body: `{ query, source: 'all' | <source_id> }`
- Validates query length (>=2 trimmed) → 400
- Validates source against the configured-sources gate → 400
(rejects unconfigured sources even when explicitly named)
- For 'all': parallel `ThreadPoolExecutor` dispatch across every
configured download source, merged results
- For specific source: just that source
- Returns same shape as `/candidates` so the frontend renderer
is reused
- New module-level helpers: `_STREAMING_SOURCE_NAMES`,
`_infer_candidate_source`, `_serialize_candidate`,
`_list_available_download_sources`. The existing `/candidates`
endpoint also goes through `_serialize_candidate` so the source
badge is consistent across both flows.
Behavior preserved
- Existing modal layout / candidates table / ⬇ button are
byte-identical when the user doesn't use manual search.
- `downloadCandidate()` JS function untouched.
- `/candidates` and `/download-candidate` endpoints
backwards-compatible — only NEW fields added, nothing changed
or removed.
Tests
`tests/test_manual_search_endpoint.py` — 10 tests:
- `test_manual_search_validates_query_length`
- `test_manual_search_validates_source` (whitelist gate)
- `test_manual_search_handles_task_not_found` (404)
- `test_manual_search_dispatches_to_configured_source_only`
- `test_manual_search_all_dispatches_parallel`
- `test_manual_search_skips_unconfigured_sources`
- `test_manual_search_rejects_unconfigured_source_explicitly`
- `test_manual_search_returns_same_shape_as_candidates`
- `test_manual_search_single_source_mode_lists_source` (verifies
`available_sources` reflects the active mode)
- `test_manual_search_isolates_per_source_exceptions` (one source
throwing doesn't kill the merged result)
2242/2242 full suite green (was 2232 + 10 new). Ruff clean.
JS parses clean.
- `_SOULSYNC_BASE_VERSION` 2.4.2 → 2.4.3
- helper.js — flip 2.4.3 WHATS_NEW header to "May 8, 2026 — 2.4.3
release"; bump fallback default from 2.4.2 → 2.4.3
- docker-publish.yml — manual-trigger default tag 2.4.2 → 2.4.3
Drive-by — make sidebar version + version-modal subtitle dynamic.
The sidebar version button (`v2.4.1`) and version-modal subtitle
(`Version 2.4.1 — Latest Changes`) were hardcoded text in the HTML.
2.4.2 shipped without these getting bumped — silent drift, easy to
miss at every release.
Added a Flask context_processor that injects `soulsync_version` and
`soulsync_base_version` into every template, then templated the two
hardcoded values:
v{{ soulsync_base_version }}
Version {{ soulsync_base_version }} — Latest Changes
Now bumping `_SOULSYNC_BASE_VERSION` updates the UI everywhere it's
rendered. No more "I forgot to bump the sidebar" at release.
2232/2232 full suite green. Ruff clean. JS parses clean.
Owner decision: not worth shipping. The four library-driven personalized
sections were stubbed returning [] for ages because their schema
prereqs didn't exist; the prior commit re-enabled them by routing
through a new `_select_library_tracks` helper. Owner reviewed and chose
to delete the sections entirely instead.
Removed everywhere:
- `core/personalized_playlists.py` — `get_recently_added`,
`get_top_tracks`, `get_forgotten_favorites`, `get_familiar_favorites`
+ the `_select_library_tracks` helper (no other callers; verified
via grep).
- `web_server.py` — 4 route handlers
(`/api/discover/personalized/recently-added`, `top-tracks`,
`forgotten-favorites`, `familiar-favorites`).
- `webui/index.html` — 4 `<div class="discover-section">` blocks
(`#personalized-recently-added`, `#personalized-top-tracks`,
`#personalized-forgotten-favorites`,
`#personalized-familiar-favorites`).
- `webui/static/discover.js` — 4 load functions
(`loadPersonalizedRecentlyAdded`, `loadPersonalizedTopTracks`,
`loadPersonalizedForgottenFavorites`, `loadFamiliarFavorites`),
plus their entries in `loadDiscoverPage`'s Promise.all, plus
4 module-level state vars + 6 dead branches across
`openDownloadModalForDiscoverPlaylist` / `startDiscoverPlaylistSync`
and the sync-progress / rehydrate dispatchers.
- `webui/static/helper.js` — 4 tooltip / docs entries.
- `webui/static/sync-spotify.js` — 1 stale rehydrate dispatcher
branch (`discover_familiar_favorites`) caught during the global
grep pass.
- `tests/test_personalized_playlists_id_gate.py` — 3 library-method
tests + the test infrastructure that supported them
(`tracks` schema, `insert_library_track` helper). Documentation
header updated to reflect the deletion.
Net: -527 / +2 lines across 7 files.
What stays:
- Daily Mixes (also in personalized package, intentionally paused —
separate decision).
- Popular Picks + Hidden Gems + Discovery Shuffle (alive, not
affected by this deletion).
- All 14 tests in the personalized-playlists test file still pass.
- The PersonalizedPlaylistsService lift from the prior commit
(`_select_discovery_tracks` etc) — those are still in active use
by the surviving discovery_pool methods.
DISCOVER_TRACK_SELECTION_REVIEW.md at repo root contains historical
references to the four deleted endpoints. Treated as historical
context (same policy as WHATS_NEW), left alone.
2219/2219 full suite green (was 2222 - 3 deleted tests = 2219).
JS parses clean, ruff clean.
- `web_server.py` — `_SOULSYNC_BASE_VERSION` 2.4.1 → 2.4.2
- `webui/static/helper.js` — flip the 2.4.2 WHATS_NEW header from
"Unreleased — 2.4.2 dev cycle" to "May 7, 2026 — 2.4.2 release"
so the per-version block stops being filtered out by
`_getLatestWhatsNewVersion`. Also bumps the safety-net default
inside that helper from 2.4.1 → 2.4.2.
- `.github/workflows/docker-publish.yml` — manual-trigger default
tag bumped to match.
Drive-by fix: escaped a stray single quote in the `Internal: Download
Engine` 2.4.2 entry that broke `node --check` on the file
(`orchestrator.client('soulseek')` inside a single-quoted desc string
silently terminated the string mid-entry). Pre-existing, unrelated to
the bump but caught while validating JS parse for the release.
VERSION_MODAL_SECTIONS not rotated in this commit — separate
editorial pass.
Closes#513 (s66jones).
The artist detail page already showed a "Popular on Last.fm" sidebar —
list of an artist's top tracks by playcount, with a play button per row
but no download action. Issue #513 wanted a way to grab those tracks
the same way zotify let users grab "top X songs" without pulling the
full discography.
Pulls from the configured primary metadata source (Spotify
`artist_top_tracks`, Deezer `/artist/{id}/top`) when available, falls
back to the existing Last.fm display-only mode for sources that don't
expose popularity ranking (iTunes / Discogs / MusicBrainz). Source
label in the section title shifts to match.
Each row gets a hover-revealed download button that wishlists the
single track via the existing /api/add-album-to-wishlist endpoint
(preserves the track's real album metadata, so the wishlist worker
later places the file in its proper album folder).
A "Download All" footer button opens the standard download modal in
PLAYLIST context, not album context — the virtual playlist_id is
`top_tracks_<source>_<artistId>` which doesn't match any of the
album-prefix checks in `startMissingTracksProcess` (downloads.js).
That keeps `is_album_download=false`, so the master worker doesn't
inject a wrapper context as `_explicit_album_context`. Each track
downloads using its own real album metadata, files land in proper
per-album folders on disk (not a fake "Top Tracks" folder).
Backend additions:
- `SpotifyClient.get_artist_top_tracks(artist_id, country, limit)` —
wraps `spotipy.artist_top_tracks`, returns up to 10 tracks for the
market (Spotify's API cap). UI-side limit trim only.
- `DeezerClient.get_artist_top_tracks(artist_id, limit)` — wraps
`/artist/{id}/top?limit=N`, converts Deezer's raw shape to the same
Spotify-compatible dict layout (id, name, artists, album with
album_type / total_tracks / images, duration_ms, track_number,
disc_number) so downstream code doesn't branch on source.
- `GET /api/artist/<id>/top-tracks` — dispatches to whichever client
matches the primary source. Resolves per-source artist IDs from the
DB row first (matching what /discography already does) so a Spotify
ID in the URL still works when Deezer is primary, and vice versa.
Returns `{success, source, tracks, resolved_artist_id}` on hit;
`{success: False, reason: 'unsupported_source' | 'spotify_not_authenticated'
| 'deezer_unavailable' | 'no_tracks_found'}` on miss so the frontend
can decide whether to fall through to Last.fm.
Frontend:
- `_loadArtistTopTracks` tries the metadata source first, falls
through to the legacy `/api/artist/0/lastfm-top-tracks` call if the
source can't deliver. Section title and per-row UI shift based on
which source answered.
- New per-row `.hero-top-track-download` button (hover-revealed).
- New `.hero-top-tracks-download-all` footer button — only visible
when metadata-source mode rendered the list (Last.fm fallback hides
it since rows have no track IDs to download).
Tests: 10 new tests pin the client methods —
- Spotify: returns track list, honors UI limit cap, returns empty when
unauthed / artist_id missing / API throws.
- Deezer: shape conversion to Spotify-compatible dict, empty when no
data / artist_id missing, limit clamping at upper bound, default
fallback when limit=0, malformed entries skipped.
The Flask endpoint dispatcher itself isn't covered by the new test
file because importing web_server at test-collection time spins up
worker threads that race with caplog-using tests elsewhere in the
suite (specifically test_library_reorganize_orchestrator). Endpoint
verified manually; the underlying client methods (the load-bearing
logic) are covered.
2204/2204 full suite green (was 2194 + 10 new).
Catches the silent excepts the awk-based earlier sweeps missed:
- Bare `except:` followed by `pass` (also swallows KeyboardInterrupt
and SystemExit — actively wrong). Upgraded to `except Exception as
e: logger.debug("...: %s", e)`. ~14 sites across connection_detect,
soulseek_client, listenbrainz_manager, watchlist_scanner,
youtube_client, navidrome_client, jellyfin_client, web_server.
- `except Exception:` + pass that the awk pattern missed (e.g.
multi-line or unusual whitespace). ~31 sites across automation_engine,
database_update_worker, music_database, spotify_client, web_server,
others.
- 14 legitimate cleanup sites left silent with explicit `# noqa: S110`
+ comment explaining why (atexit handlers, finally-block conn.close
calls). Logging during shutdown can itself crash because file handles
get torn down before the handler fires.
Also enables `S110` rule in pyproject.toml so this pattern fails CI
going forward — drift fails at PR review instead of at runtime against
a wedged worker thread. Tests path keeps S110 ignored (test fixtures
legitimately use try-except-pass for cleanup).
Adds a WHATS_NEW entry to helper.js summarizing the full #369 sweep.
Verified: `python -m ruff check .` → All checks passed.
Verified: `python -m pytest tests/` → 2188 passed.
Closes#369
Replaces `except Exception: pass` blocks with `except Exception as e:
logger.debug(...)` so failures are inspectable in the log instead of
disappearing silently. Per JohnBaumb's request in #369.
- Pattern is consistent: `logger.debug("<context>: %s", e)` with lazy
formatter and 2-6 word context describing the operation.
- 2 atexit handlers (lines 2977, 2983) intentionally left silent — the
log file handles can be torn down before atexit fires, and a
separate `_atexit_silence_shutdown_logger_errors` already exists for
this exact reason.
- No behavior changes; control flow is unchanged. Test suite green
(2188 passed).
Refs #369
GitHub issue #505 (PopeBruhLXIX): users with multiple Plex music
libraries (e.g. one per Plex Home user, or two folder roots split
across separate library sections) only saw one library inside SoulSync
because the connection settings forced you to pick a single library
section. SoulSync's PlexClient stored exactly one ``self.music_library``
section reference and every read scanned only that one.
This change adds an opt-in "All Libraries (combined)" dropdown option
that flips the client into a server-wide read mode where every read
method (``get_all_artists`` / ``get_all_album_ids`` /
``search_tracks`` / ``get_library_stats`` / etc) dispatches through
``server.library.search(libtype=...)`` instead of querying a single
section. One Plex API call replaces N per-section iterations; Plex
handles the aggregation server-side.
Implementation:
- ``ALL_LIBRARIES_SENTINEL`` (``'__all_libraries__'``) — module-level
constant used as the saved DB preference value when the user picks
the synthetic "All Libraries" entry. Detection is one string compare
in ``_find_music_library`` / ``set_music_library_by_name``. Existing
preferences (real library names) are unaffected.
- ``self._all_libraries_mode`` (private flag) + ``is_all_libraries_mode()``
(public accessor for external callers). When True, ``music_library``
may stay None — ``is_fully_configured()`` recognizes the mode and
still returns True so dispatch sites don't bail.
- New private helpers ``_can_query``, ``_get_music_sections``,
``_all_artists``, ``_all_albums``, ``_all_tracks``, ``_search_general``,
``_search_artists_by_name``. Single dispatch point for the
section-vs-server branch — every read method funnels through them
so future drift fails at one place.
- New public helpers for downstream callers:
- ``get_recently_added_albums(maxresults, libtype)`` — used by
DatabaseUpdateWorker's deep-scan recent-content sweep
- ``get_recently_updated_albums(limit)`` — same
- ``get_music_library_locations()`` — returns folder roots, used
by web_server.py's file-path resolver
- ``trigger_library_scan`` and ``is_library_scanning`` fan out across
every music section in all-libraries mode.
- ``get_available_music_libraries`` prepends a synthetic
``{'title': 'All Libraries (combined)', 'value': sentinel}`` entry
ONLY when more than one music library exists. Single-library users
don't get the extra option. ``value`` field is the canonical
identifier the frontend submits to ``/api/plex/select-music-library``
(real libraries: title; synthetic: sentinel string). Backward-
compatible — entries without ``value`` fall back to ``title``.
Three crash points fixed in downstream consumers (would have failed
during a deep scan after the user picked all-libraries mode):
1. ``database_update_worker.py:411`` — bailed out with "No music
library found in Plex" because ``not self.media_client.music_library``
evaluated True in all-libraries mode (music_library is None there).
Now uses ``is_fully_configured()`` which recognizes the mode.
This was the root cause of the deep scan never starting.
2. ``database_update_worker.py:_get_recent_albums_plex`` — reached
``self.media_client.music_library.recentlyAdded()`` /
``.search()`` directly, AttributeError in all-libraries mode.
Now routes through the new helper methods.
3. ``web_server.py:10947`` (file-path resolver) — accessed
``music_library.locations``; gated on ``music_library`` truthy so
it didn't crash, but silently skipped all-libraries-mode locations.
Now uses ``get_music_library_locations()`` which unions across
sections.
Plus polish:
- ``/api/plex/clear-library`` also resets ``_all_libraries_mode``
so a fresh "select library" flow doesn't inherit stale mode state.
- ``/api/plex/music-libraries`` surfaces "All Libraries (combined)"
as ``current_library`` when in mode (settings UI displays correctly).
- Frontend ``loadPlexMusicLibraries`` uses ``library.value || library.title``
so the sentinel-keyed option submits the sentinel string, not the
human-readable label. Pre-select match handles both paths.
Honest tradeoffs (documented as known limitations):
- Same artist appearing in multiple Plex sections shows as separate
entries in SoulSync (no dedup). Plex returns distinct ratingKeys
for each. Cosmetic; revisit if it bites users.
- Write-back (genre / poster updates) targets one ratingKey at a time
— only updates that section's copy. Other sections' copies stay
unchanged.
- All-libraries mode includes any audiobook library that Plex
classifies as ``type='artist'``. Edge case, opt-in only.
Tests: 21 new tests in tests/media_server/test_plex_all_libraries.py
pin both single-library mode (regression guard) and all-libraries mode
for every refactored method. Existing test_plex_pinning.py fixture
updated to initialize the new flag. 63/63 media_server tests green,
2148/2148 full suite green.
Two fixes.
(1) Discography endpoint now does server-side per-source ID resolution.
When the user clicked Download Discography on a library artist, the
endpoint received whichever artist ID the frontend happened to pick
(spotify_artist_id || itunes_artist_id || deezer_id || library_db_id)
and dispatched it as-is to whichever source it queried. If the picked
ID didn't match the queried source's ID format, the lookup returned
wrong-artist results (numeric ID collisions) or fell back to a fuzzy
name search that picked a wrong artist.
Two reproducible cases:
- 50 Cent's library row had DB id 194687 — coincidentally a real
Deezer artist ID for "Young Hot Rod". When the frontend's
/enhanced fetch silently fell back to the DB id, the backend
sent 194687 to Deezer, and Deezer returned Young Hot Rod's
50 albums in 50 Cent's discography modal.
- Weird Al's library row had a stored Spotify ID. The frontend
sent that to Deezer, which rejected the alphanumeric ID and
fell back to fuzzy name search — which picked The Beatles
somehow, returning 45 Beatles albums.
The mechanism for per-source ID dispatch already exists in
``MetadataLookupOptions.artist_source_ids``, and the watchlist scanner
already uses it; the on-demand discography endpoint just wasn't wired
to it. Fix: when the URL artist_id matches a library row by ANY stored
ID (DB id, spotify_artist_id, itunes_artist_id, deezer_id, or
musicbrainz_id), pull every stored provider ID and pass them as
``artist_source_ids``. Each source gets its OWN stored ID regardless
of which one the URL carries. When the URL ID is a non-library
source-native ID and the row lookup misses entirely, behavior is
identical to before (single-ID dispatch fallback).
Logged the resolved per-source ID dict at INFO so future "wrong artist
showed up" diagnostics are immediately legible in app.log.
(2) Logger namespace fix in core/artists/quality.py and
core/metadata/multi_source_search.py.
Both modules used ``logging.getLogger(__name__)`` which resolves to
``core.artists.quality`` / ``core.metadata.multi_source_search`` —
neither under the ``soulsync`` namespace where the file handler is
wired. Result: every [Enhance], [MultiSourceSearch], and direct-lookup
INFO line was being written to a logger with no handlers and silently
dropped. App log showed the slow-request warning but no diagnostic
detail. Switched both to ``get_logger()`` from utils.logging_config so
the soulsync.* namespace picks them up. Same content, now actually
lands in app.log. Confirmed working in live test:
``[Enhance] Direct lookup matched: deezer ID 1476162252 → 'Desastre'``
No behavior change in any other caller. Empty ``artist_source_ids``
(no library row matched) reaches lookup as ``None`` → identical to
current single-ID dispatch path. Logger fix is pure routing — no
content change.
Followup on the previous Enhance refactor. Multi-source parallel text
search closed the worst case (users with no Spotify/Deezer getting
"unknown artist - unknown album - unknown track" wishlist entries),
but text search itself is still fragile against messy library tags:
"Title (Live)", featured artists in the artist field, etc. Download
Discography never had this problem because it resolves albums by stable
ID, not by name.
Enhance now does the same thing for tracks: for every metadata source
the user has configured, if the library track has the corresponding
stored ID (spotify_track_id / deezer_id / itunes_track_id / soul_id),
call client.get_track_details(stored_id) directly and convert to the
wishlist payload. First success wins. The user's configured primary
source is tried first so a Deezer-primary user gets Deezer payloads on
the wishlist entry (correct cover art / album shape) even when other
sources also have stored IDs for the same track.
Multi-source parallel text search stays as the fallback for tracks
with no stored IDs (e.g. manually imported, never enriched). Empty-
field rejection still gates the wishlist add.
Implementation:
- _STORED_ID_COLUMNS: source name → DB column mapping
(Discogs intentionally omitted — release-based, no per-track IDs)
- _enhanced_to_wishlist_payload: converts the get_track_details
intermediate "enhanced" shape (artists as [str]) to wishlist shape
(artists as [{'name': str}]). Spotify's raw_data is already in
wishlist shape, returned as-is when detected (preserves full
album.images that the enhanced top-level fields drop)
- _try_direct_lookup_all_sources: iterates sources preferred-first,
calls get_track_details on each that has both a stored ID and a
configured client, returns first complete-metadata payload
- spotify_client field removed from ArtistQualityDeps (no longer
used — Spotify direct lookup now flows through the generic
per-source loop using the entry from search_sources)
- _try_upgrade_to_rich_payload removed (was Spotify-only with broken
shape semantics for non-Spotify sources; search-fallback now uses
_build_payload_from_track consistently)
- get_primary_source() consulted to set the per-call preferred source
for direct-lookup priority
Also fixed a stale UI string: the Enhance modal toast read "Matching
tracks to Spotify and adding to wishlist..." regardless of which
sources were actually configured. Now reads "Matching tracks across
metadata sources...".
Tests:
- _build_deps mirrors web_server._resolve_search_sources: passing
spotify=spotify_obj auto-prepends ('spotify', spotify_obj) to
search_sources (Spotify is always added when configured in prod)
- 5 new tests pin the direct-lookup behavior:
- test_direct_lookup_via_deezer_id_skips_text_search
- test_direct_lookup_via_itunes_id_skips_text_search
- test_direct_lookup_prefers_user_primary_source
- test_direct_lookup_falls_through_to_text_search_when_no_stored_ids
- test_direct_lookup_failure_falls_through_to_text_search
- Reframed enhanced-format and search-fallback tests for the new
payload-build path (no album-image side call, search-fallback uses
_build_payload_from_track consistently)
- 22/22 quality tests green, 2133/2133 full suite green.
Track Redownload had been doing parallel multi-source metadata search
across every configured source the whole time; Enhance Quality was
running a single-source primary fallback that returned junk matches
with empty fields when the primary was iTunes (Discord report:
"unknown artist - unknown album - unknown track" wishlist entries
for users with neither Spotify nor Deezer connected).
Lift the redownload search into core/metadata/multi_source_search.py
and point both flows at it. Same scoring, same per-source query
optimization (Deezer's structured artist:/track: form), same
current-match flagging via stored source IDs.
ArtistQualityDeps now takes get_metadata_search_sources (returns
[(name, client), ...] for every configured source) instead of the
single-primary get_metadata_fallback_client + get_metadata_fallback_source.
Spotify direct-lookup stays as a fast-path optimization (only Spotify
exposes get_track_details(id) returning rich raw payload); when it
doesn't fire, the multi-source parallel search picks the cross-source
best match. Empty-field matches still rejected before wishlist add.
Tests: _build_deps helper updated to accept the new search_sources
contract while preserving fallback_client/fallback_source ergonomics.
Reframed tests for the new semantics — direct-lookup is no longer
gated on Spotify being the active primary; failure reason now lists
every searched source. Added a test pinning the no-sources-configured
prompt. 17/17 quality tests green, 2128/2128 full suite green.
Discord report: clicking Enhance Quality on an artist with neither
Spotify nor Deezer connected added tracks to the wishlist as
"unknown artist - unknown album - unknown track".
Root cause was structural. core/artists/quality.py had a hardcoded
Spotify-direct → Spotify-search → iTunes-fallback chain that ignored
the user's configured primary metadata source. When Spotify wasn't
connected, every track fell through to an iTunes-only fallback that
occasionally returned matches with empty fields (cleared the 0.7
confidence threshold but missing artist / album / title). Those
empty strings propagated through the wishlist payload normalizer's
truthy-check passthrough at core/wishlist/payloads.py:77-80 and the
UI rendered them as "Unknown" defaults.
Rewrote the flow source-agnostic:
- ArtistQualityDeps gains get_metadata_fallback_source. Flow resolves
the user's active primary source once up front.
- New _build_payload_from_track helper produces the Spotify-shaped
wishlist payload from any source's Track object — single place
that knows how to construct it (replaces the duplicate construction
in the Spotify-search and iTunes-fallback paths).
- New _search_match helper does generic confidence-scored search
against any client implementing search_tracks(query, limit). Same
0.7 threshold, same album-bonus weighting as before.
- New _has_complete_metadata validator rejects matches with empty
title / album / artists before they reach the wishlist.
- _spotify_direct_lookup kept as a Spotify-only optimization (only
Spotify exposes get_track_details(id) returning rich raw payload);
other sources fall through to search.
- Failure reason now names the active source: "No usable {source}
match — connect another metadata source for better coverage".
Result: Discogs users get a Discogs search. Hydrabase users get a
Hydrabase search. iTunes users get an iTunes search with empty-field
rejection. Spotify keeps its direct-lookup fast path.
6 new tests pin the architectural change:
- Primary-source dispatch routes to the configured client (Discogs,
not Spotify) when Spotify isn't primary
- Spotify direct-lookup is gated on Spotify being the active primary
(skipped when Discogs is configured even if track has spotify_track_id)
- Empty title / album / artists fields all reject the match
- Failure reason names the active source
Going line-by-line through the engine package + boot wiring. Five
small things worth fixing before Cin reads it:
(1) MediaServerEngine class docstring still claimed to be a "single
entry point for cross-server library operations" — but the prior
honesty pass cut all the cross-server dispatch wrappers because they
had no callers. Class is really lookup + small accessors now.
Docstring rewritten to match.
(2) configured_clients() had a dead `not hasattr(client, 'is_connected')`
branch. is_connected is in REQUIRED_METHODS so every client the
registry yields here implements it. Branch removed; comment notes
the reasoning.
(3) types.py imported `datetime` and `Dict` but used neither —
dead imports dropped.
(4) types.py docstring claimed "all four servers" defined an
XTrackInfo dataclass. Actually only Plex / Jellyfin / Navidrome
did; SoulSync uses richer per-track wrappers. Fixed.
(5) web_server.py boot:
- media_server_engine added to the chained `= None` declaration
so it's always defined before the try/except, defending against
the rare path where engine init AND fallback both raise.
- Outer engine init failure logger now uses exc_info=True for full
traceback (boot-time issues are rare but worth diagnosing).
- Nested fallback failure now logs explicitly instead of silently
leaving media_server_engine as None.
Tests: 2121 still pass.
Five tightening passes anticipating Cin / JohnBaumb's review nits:
(1) Engine no longer reaches into ``registry._instances`` private
attr. New public ``MediaServerRegistry.set_instance(name, client)``
method — engine constructor calls it for the ``clients=`` pre-built
case so internal storage stays encapsulated.
(2) Engine module docstring no longer overclaims. Originally said it
"Replaces the historic 33+ if/elif chains" — but only the four
uniform-shape ``is_connected`` chains were collapsed. The 19 chains
that do server-specific work (Plex raw API vs Jellyfin / Navidrome
client methods returning different shapes) stay explicit per the
"lift what's truly shared" standard. Docstring rewritten to say
exactly that.
(3) Per-method exception swallows upgraded from ``logger.debug`` to
``logger.warning``. Returning safe defaults stays the right behavior
for a read-side engine (Plex offline shouldn't crash the app), but
silent debug-level swallowing made debugging hard — JohnBaumb pushed
the download engine to surface real errors. Same treatment here:
default still safe, but the warning tells you Plex is down.
(4) ``_safe_init_media_client`` in web_server.py now logs the
exception type + traceback. Broad ``except Exception`` is still
intentional (any failure means that one server can't be used; the
others stay up) but the boot log now distinguishes config errors
(ConnectionError, AuthenticationError) from import / dependency
failures.
(5) Two new tests pin the encapsulation + fallback contracts:
- ``test_engine_with_empty_clients_dict_is_safe_to_use`` — empty
engine returns safe defaults on every method, doesn't raise.
- ``test_engine_uses_registry_set_instance_not_private_attr`` — spy
on registry.set_instance verifies engine uses the public method.
Same latent bug as add-track — replace-track and remove-track only
looked up the Plex playlist by name. Plex deletes + recreates
playlists on edit so the rating key the frontend cached can be
stale, name lookups can also fail (special chars, encoding). Both
now use the same id-first / name-fallback chain as the GET tracks
endpoint, with a diagnostic log when both lookups fail.
Pre-existing latent bug, not a refactor regression.
The /api/server/playlist/<id>/add-track endpoint only looked up the
target Plex playlist by name, but Plex deletes + recreates playlists
on edit so the rating key the frontend cached can be stale. The
companion GET /tracks endpoint already had id-first / name-fallback;
add-track now does the same.
Added a warning log on GET /tracks when BOTH lookups fail so the
"all source items show Find & Add" symptom (which happens when
server_tracks comes back empty) has a clear diagnostic in the log
instead of silently rendering an empty server column.
Not a refactor regression — the original code had the same name-only
lookup. The mass-replace of `plex_client` → `media_server_engine.client('plex')`
is byte-equivalent. Just surfacing the latent bug.
If MediaServerEngine init raised, ``media_server_engine`` was set
to None. Every downstream caller (now that the per-server globals
are gone) does ``media_server_engine.client('plex')`` style access
— which would AttributeError on the None.
Pre-refactor each per-server global had its own try/except so engine
failure didn't take down per-server dispatch sites. Preserve that
resilience by falling back to an empty MediaServerEngine — its
``client(name)`` returns None, downstream truthy checks treat that
as "not configured" exactly the same way the legacy globals did.
Per-server web_server.py globals (plex_client / jellyfin_client /
navidrome_client / soulsync_library_client) are gone. The engine now
owns the per-server client instances; web_server.py constructs them
inline into the engine init and routes everything through
media_server_engine.client('<name>').
Multi-client consumers refactored to take the engine instead of
separate per-server kwargs:
- services/sync_service.py: PlaylistSyncService.__init__ now takes
media_server_engine. Internal _get_active_media_client resolves the
active server's client through self._engine.client(name) instead of
the per-server self.X_client attributes.
- core/listening_stats_worker.py: ListeningStatsWorker takes
media_server_engine. The plex/jellyfin/navidrome dispatch in _poll
collapses to engine.client(active_server) (gated to those three
servers — SoulSync standalone has no listening data).
- core/web_scan_manager.py: WebScanManager takes media_server_engine
instead of the hand-keyed media_clients dict that drifted out of
sync with the engine.
- core/discovery/sync.py: SyncDeps holds media_server_engine instead
of plex_client / jellyfin_client. Playlist-image dispatch routes
through engine.client(name).
Web_server.py:
- Per-server globals removed from the chained `= None` init line
+ their try/except construction blocks. Replaced with a
_safe_init_media_client(factory, name) helper that captures
per-server init failures + passes the resulting clients straight
into the MediaServerEngine init dict.
- All construction sites (PlaylistSyncService, WebScanManager,
ListeningStatsWorker, SyncDeps, library_check) updated to receive
the engine instead of per-server clients.
Test fixtures (tests/discovery/test_discovery_sync.py) gain a
_FakeMediaServerEngine stub + the SyncDeps build helper passes
that instead of separate plex/jellyfin clients.
Pre-change web_server.py had ~70 direct attribute reaches against the
per-server globals (plex_client.X, jellyfin_client.X, navidrome_client.X,
soulsync_library_client.X) plus ~60 standalone refs (truthy checks,
media_client assignments, source-name tuples). The engine was wired
but only used in 4 places, so most of the codebase still hand-dispatched
— the exact "partially defeats the purpose of this refactor" critique
Cin landed on the download PR initially.
- All ~70 client.attribute reaches migrated to
media_server_engine.client('<name>').attribute. The chains in
web_server.py do server-specific work (Plex raw API, Jellyfin /
Navidrome client methods, all returning different shapes), so the
if/elif structure stays — but the per-server CLIENT REACH now goes
through the engine like Cin's POC pattern intended.
- All ~60 standalone refs migrated:
- if plex_client → if media_server_engine.client('plex')
- media_client = plex_client → media_client = media_server_engine.client('plex')
- ('plex', plex_client) tuples → ('plex', media_server_engine.client('plex'))
- Per-server globals (plex_client / jellyfin_client / navidrome_client /
soulsync_library_client) kept for now — external modules
(PlaylistSyncService, WebScanManager, ListeningStatsWorker, search
library check, discovery sync deps) still take them as kwargs.
Dropping them entirely needs a follow-up sweep across those modules.
Suite green (1961 pass).
Apply the Cin-1 / Cin-2 pattern from the download refactor PR to the
media server engine PR before review.
Cin-1 — explicit inheritance:
- PlexClient, JellyfinClient, NavidromeClient, SoulSyncClient now
explicitly inherit MediaServerClient instead of relying on
structural typing alone. Pre-change a reader of plex_client.py
had no way to know the class was supposed to satisfy the contract.
- Removed the engine + registry re-exports from
core/media_server/__init__.py to break the circular import that
the inheritance change introduced (importing the package now
triggered a chain that loaded clients before their base class
resolved). Submodules import directly: from
core.media_server.engine import MediaServerEngine, etc.
- Conformance test now also asserts isinstance() / issubclass()
against MediaServerClient — drift in any class fails at the test
boundary instead of at runtime.
Cin-2 — generic accessors + singleton:
- engine.configured_clients() — replaces the legacy per-server
`if X and X.is_connected(): clients[name] = X` chains in
web_server.py.
- engine.reload_config(name=None) — generic dispatch, so callers
pass the server name instead of reaching for plex_client.reload_config()
directly.
- get_media_server_engine() / set_media_server_engine() singleton
factory matching the get_metadata_engine() / get_download_orchestrator()
shape. web_server.py boots via set_media_server_engine(...) so
factory + global handle share state.
- 7 new tests pin the accessors + singleton behaviour.
Two findings from JohnBaumb on the engine refactor.
(1) Every download client returned None when self._engine was None,
just logging an error. The orchestrator's download_with_fallback
treated None as "source declined", so the user got no feedback —
download silently disappeared. Now each client raises a RuntimeError
on the engine-not-wired path. download_with_fallback already catches
plugin exceptions, logs a warning, and tries the next source — so
the visible behavior is "real error in logs + fallback to next
source" instead of "silent drop". Six clients touched (deezer, hifi,
qobuz, soundcloud, tidal, youtube). Pinning tests updated to expect
raise.
(2) Monitor's engine.get_all_downloads() walked every plugin
including soulseek, but the same monitor loop already pulled slskd
transfers via the transfers/downloads endpoint a few lines earlier —
soulseek's records were being fetched twice per tick. Same issue in
web_server.py's get_cached_transfer_data path. Added an exclude
parameter to engine.get_all_downloads(); both call sites now pass
('soulseek',). New test pins the exclude semantic.
Also fixed a stray 8-space over-indent on the for-loop body in
get_cached_transfer_data (cosmetic, JohnBaumb flagged the same
pattern in monitor.py earlier).
Two architectural cleanups on top of the download engine refactor.
(1) Shared dataclasses move to neutral plugin package.
TrackResult, AlbumResult, DownloadStatus, SearchResult lived in
core/soulseek_client.py for historical reasons — every other plugin
imported them from the soulseek module just to satisfy the contract,
coupling 8 clients to a sibling source for type imports only. Moved
them to the new core/download_plugins/types.py module and updated all
14 import sites across the deezer/hifi/lidarr/qobuz/soundcloud/tidal/
youtube clients, the engine, matching engine, redownload helper, and
tests. Clean break, no backward-compat re-export.
(2) web_server.py boots the orchestrator via the singleton factory.
After construction it now calls set_download_orchestrator(...) so
get_download_orchestrator() returns the same instance the global
handle points at instead of lazily building a separate orchestrator.
Matches the get_metadata_engine() pattern.
Hunted down the remaining sites where web_server.py still reached
into orchestrator per-source attributes. Most were silently broken
after Cin-5 dropped those attrs but were guarded by hasattr checks
that always returned False — empty download_clients dicts and
no-op reload paths.
- /api/library/track/<id>/redownload-search: replaced the 6 if/hasattr
per-source blocks (the exact pattern Cin called out in his review)
with a single download_orchestrator.configured_clients() call.
- Settings reload path: hasattr-guarded YouTube reload now resolves
via client('youtube') and tests for None.
- _try_source_reuse / _store_batch_source: slsk lookup gates on
hasattr(orch, 'client') instead of the dropped 'soulseek' attr.
- /api/soundcloud/status + Deezer ARL endpoints: same hasattr
swap.
The global handle in web_server.py was named soulseek_client for
historical reasons but the type has long been DownloadOrchestrator,
not SoulseekClient. Renamed the global plus every parameter/attribute
that carried the legacy name.
- web_server.py: global var renamed; all 99 references updated.
- api/, core/downloads/*, core/search/*, core/streaming/*,
services/sync_service.py: parameter names, dataclass fields, and
init() arg names renamed.
- Test fixtures (CandidatesDeps, MasterDeps, SearchDeps, etc.) and
the _build_deps helpers updated accordingly.
The core.soulseek_client module path and SoulseekClient class name
(the actual soulseek-only client) are unchanged — only the orchestrator
handle renamed. Module imports of TrackResult/AlbumResult/DownloadStatus
from core.soulseek_client preserved.
Removed the eight backward-compat attribute aliases on the orchestrator
(soulseek, youtube, tidal, qobuz, hifi, deezer_dl, lidarr, soundcloud).
External callers and the orchestrator's own internals now reach clients
through the generic alias-aware client(name) accessor.
- core/downloads/{master,monitor,validation}.py: migrated to client().
Monitor's per-source aggregation loop replaced with a single
engine.get_all_downloads() call.
- core/search/{orchestrator,stream}.py: migrated; stream.py drops the
hand-built mode-to-client dict.
- web_server.py: migrated /api/deezer/arl-* + tidal client lookup.
- core/download_orchestrator.py: internal self.soulseek /
self.deezer_dl reaches now route through self.client(); attr
assignments dropped from __init__; module docstring updated.
- Test fakes (_FakeSoulseek, _FakeSoulseekWithYT) expose client(name)
instead of stuffing per-source attributes.
- Conformance test re-pinned to the client() accessor contract.
Three correctness fixes from kettui's PR review plus the web_server
migration to generic accessors.
- Engine alias map: register_plugin accepts aliases tuple; get_plugin
+ cancel_download resolve through it. Fixes deezer_dl cancels
silently routing to soulseek.
- Orchestrator hybrid_order normalization: _resolve_source_chain
routes raw config names through registry.get_spec() so legacy
deezer_dl entries don't drop deezer from hybrid mode.
- Atomic update_record_unless_state on the engine: holds state_lock
across the check + write. Both _mark_terminal AND the success path
use it now so a Cancelled state set mid-impl can't be clobbered.
- web_server.py: 30 soulseek_client.<source> reaches migrated to
client("<source>"); shutdown-check setup migrated to generic
registry iteration; 4 hifi reload sites use reload_instances('hifi').
- 18 new tests pin every fix.
Two more sites in web_server.py replaced (tag-preview + batch
tag-preview server_type checks). Same pattern as C1: 3-way
if/elif → engine.is_connected().
Honest scope note: the recon agent counted 33 dispatch sites,
but most are deeply server-specific logic where each branch
does completely different work (playlist track replace, per-
server metadata sync, deep scan with server-specific helpers).
Lifting those would move per-server branches into engine helper
methods that route the same work — net zero LOC, more indirection.
Engine helps where the shape is TRULY uniform; the deep dispatches
stay explicit. Phase C ends here at 4 simple sites lifted.
Suite still green.
Two sites in web_server.py replaced:
- /status route's media-server connectivity check (4-way if/elif
for plex/jellyfin/navidrome/soulsync) → engine.is_connected()
- /api/playlists endpoint's server_connected check (3-way if/elif)
→ engine.is_connected()
Engine reads active_server config + dispatches to the right client
with internal connection caching preserved (the underlying clients
all cache is_connected() calls).
Engine constructor now accepts a pre-built clients={...} dict so
web_server.py wires the same instances as its existing per-client
globals — no double-init.
Suite still green. Per-server clients still accessible via
engine.client(name) for source-specific reaches.
`_create_finding` silently dedup-skipped re-discovered issues but
the caller incremented `findings_created` regardless. So a re-scan
that found the same issues as a prior scan reported 364 findings
in the badge while 0 NEW pending rows hit the db, leaving the
findings tab empty.
`_create_finding` now returns bool (True on insert, False on
dedup-skip / db error). All 16 repair jobs updated to only
increment `findings_created` on True. Added `findings_skipped_dedup`
counter surfaced in scan log: "Done: X scanned, 0 fixed, 0
findings (363 already existed), 0 errors".
Also fixed a missing `job_id` kwarg in album_tag_consistency that
was silently breaking finding creation for that scan.
Discord request: pull user's Discogs collection into the Your Albums
section on Discover, similar to how Spotify Liked Albums works.
Implementation extends the existing 3-source pipeline (Spotify /
Tidal / Deezer) to a 4-source pipeline with click-context dispatch —
Discogs-only albums open with rich Discogs release detail (vinyl/CD
format, year, label, country, tracklist). Mirrors the per-source
dispatch pattern from enhanced/global search.
Discogs client (`core/discogs_client.py`):
- New `get_authenticated_username()` resolves the username for the
configured personal token via Discogs's `/oauth/identity` endpoint.
Cached on the instance so subsequent collection page-fetches don't
re-hit it.
- New `get_user_collection(username=None, folder_id=0, per_page=100,
max_pages=50)` walks all pages of `/users/{username}/collection/
folders/{folder_id}/releases`. Returns normalized dicts ready for
upsert_liked_album. folder_id=0 = Discogs's "All" folder.
Pagination cap of max_pages*per_page = 5000 releases — bounds
runtime on heavy collections.
- New `get_release(release_id)` thin wrapper for `/releases/{id}` —
returns the raw API response so the album-detail endpoint can
render rich context.
- Both methods defensive: missing token → empty list, malformed
responses → skipped, falsy ids → None. Disambiguation suffix
stripping (`Madonna (3)` → `Madonna`) so Discogs artist names
match what Spotify/Tidal/Deezer use.
Schema (`database/music_database.py`):
- New `discogs_release_id TEXT` column on `liked_albums_pool`.
Migration uses the established `try SELECT, except ALTER TABLE`
pattern. Idempotent; safe on existing installs.
- Added the column to the canonical CREATE TABLE for fresh installs.
- `upsert_liked_album` extended with `'discogs': 'discogs_release_id'`
in BOTH the INSERT and UPDATE id-column maps so Discogs source_id
routes to the new column. INSERT statement column count + value
count updated together.
Backend (`web_server.py`):
- `/api/discover/your-albums/sources` — adds Discogs to the
`connected` list when `discogs.token` config is set.
- `_fetch_liked_albums` — new branch for Discogs. Lazy-imports
DiscogsClient, respects the `enabled_sources` config, walks the
collection, upserts each release. Same try/except shape as the
existing source branches.
- `/api/discover/album/<source>/<album_id>` — new `discogs` branch
fetches the release via DiscogsClient.get_release, normalizes the
Discogs tracklist format, parses Discogs's `MM:SS`/`HH:MM:SS`
duration strings to milliseconds, returns the same response shape
as the Spotify/Deezer/iTunes branches.
Frontend (`webui/static/discover.js`):
- `openYourAlbumsSourcesModal` — adds Discogs to `sourceInfo` with
the vinyl emoji icon. Existing toggle/save plumbing handles it.
- `openYourAlbumDownload` — restructured the per-source dispatch:
builds an ordered list of (source, id) tuples, tries each in turn,
breaks on the first successful response. Pure-Discogs albums go
straight to the Discogs detail endpoint → modal opens with Discogs
context. Multi-source albums prefer Spotify/Deezer first since
their tracklists carry proper streaming IDs ready for download.
Tests: `tests/test_discogs_collection_source.py` — 12 cases:
- get_user_collection: empty without token, normalizes response
shape, strips disambiguation suffix, handles missing year, skips
malformed releases, paginates correctly, caps at max_pages,
uses explicit username when provided.
- get_release: passes id through to /releases/{id}, returns None
for invalid ids without API call.
- liked_albums_pool: discogs_release_id round-trips through upsert
+ get; multi-source dedup carries both Spotify and Discogs IDs
on the same row.
Verified: full suite 1825 pass (12 new), ruff clean, smoke test
populating + reading the discogs_release_id column round-trips
correctly via the real DB.
WHATS_NEW entry under '2.4.2' dev cycle.
Discord request (Samuel [KC]): show how much disk space the library
takes on the Stats page. Implementation piggybacks on the existing
deep scan — Plex/Jellyfin/Navidrome all return file size in their
track API responses, so we read it during the deep scan and store
it on the tracks row. Aggregation is then a single SQL query — no
filesystem walk, no extra I/O during the scan, no separate stat
job. SoulSync standalone gets size from os.path.getsize at insert
time (different code path; the file is local when we write the row).
Schema (`database/music_database.py`):
- New `file_size INTEGER` column on `tracks`. Migration uses the
established `try SELECT, except ALTER TABLE ADD COLUMN` pattern.
Idempotent; safe on existing installs. NULL on legacy rows so
they don't contribute to totals until next deep scan refreshes.
- Added the column to the canonical CREATE TABLE so fresh installs
get it without going through the migration path.
Track-object plumbing:
- `core/jellyfin_client.py` — JellyfinTrack reads MediaSources[0].Size
alongside existing Bitrate read. None when 0 / missing.
- `core/navidrome_client.py` — NavidromeTrack reads `size` from
the Subsonic song object (int coercion + None on parse fail).
- `core/soulsync_client.py` — SoulSyncTrack does os.path.getsize
(only "server" where size has to come from disk).
- Plex needs no client-side change: track.media[0].parts[0].size
is read directly inside insert_or_update_media_track.
Persistence — TWO separate insert paths:
(a) `database/music_database.py:insert_or_update_media_track` —
Plex/Jellyfin/Navidrome flows. Reads file_size from Plex's
MediaPart OR `track_obj.file_size` wrapper attribute (defensive
Plex-attr-not-present check + > 0 type guard).
INSERT writes the new column.
UPDATE uses COALESCE(?, file_size) so a None from the server
on a re-sync (rare Jellyfin Size omission) doesn't blank an
existing value. Pinned via test.
(b) `core/imports/side_effects.py:record_soulsync_library_entry` —
SoulSync standalone flow. Completely separate code path: the
standalone deep scan moves files to staging for auto-import
rather than calling insert_or_update_media_track. After the
auto-import processes them, side_effects writes the tracks row
directly. Reads file_size via os.path.getsize(final_path) at
insert time (file is local) and includes it in the INSERT
column list. SoulSync only does INSERT-if-not-exists (no
UPDATE path), so no COALESCE concern.
Aggregator (`database/music_database.py:get_library_disk_usage`):
- SELECT COALESCE(SUM(file_size), 0), COUNT(file_size),
COUNT(*) - COUNT(file_size) for the totals.
- Per-format breakdown done in Python via os.path.splitext over
(file_path, file_size) rows — sidesteps SQLite's first-vs-last-dot
ambiguity for paths like /music/Kendrick/M.A.A.D City/01.flac.
- Defensive: skips empty paths, paths without extension, and
implausibly long extensions (>6 chars). Returns the full
empty-shape dict (NOT a partial / undefined) when the column
doesn't exist or queries fail, so the UI's `if (!data.has_data)`
branch handles fresh installs cleanly.
API + UI:
- `core/stats/queries.py` — thin pass-through get_library_disk_usage
matching the existing query-helper convention.
- `web_server.py` — new /api/stats/library-disk-usage endpoint
mirroring the /api/stats/db-storage pattern.
- `webui/index.html` — new card in System Statistics above the
Database Storage card.
- `webui/static/stats-automations.js` — _loadLibraryDiskUsage +
_renderLibraryDiskUsage. Empty state: "Run a Deep Scan to
populate (X tracks pending)". Partial: "X measured (+Y pending)".
Full: total + format bars proportional to the largest format.
- `webui/static/style.css` — .stats-disk-* styled to match the
Database Storage card.
Backward compatibility:
- Migration is additive; existing rows get NULL file_size; the
empty-shape return from the aggregator means the UI renders
cleanly without errors before any deep scan runs.
- Old installs upgrading will see "Run a Deep Scan to populate
(N tracks pending)". Running their next deep scan fills sizes —
the existing scan flow doesn't need any changes, just consumes
the new track-wrapper attribute.
Tests:
- `tests/test_library_disk_usage.py` — 13 cases covering schema
migration, NULL defaults on legacy inserts, fresh-install empty
shape, summing with mixed NULL/known sizes, per-format breakdown,
mixed-case extensions, paths with album-name dots, missing
extensions, empty file_path, implausibly long extensions,
JellyfinTrack.file_size persistence via insert_or_update_media_track,
COALESCE preservation on null re-sync.
- `tests/imports/test_import_side_effects.py` — extended the
existing record_soulsync_library_entry test to assert
track_row['file_size'] == os.path.getsize(final_path), pinning
the SoulSync-standalone path. Test fixture's tracks schema also
updated to include the file_size column.
Verified: full suite 1813 pass (13 new, 1 existing-test extension),
ruff clean, smoke test populating + reading the column round-trips
correctly.
WHATS_NEW entry under '2.4.2' dev cycle.
User: SoundCloud downloads finish correctly but the modal stays at
"Downloading... 0%" until "Processing..." flips on. Live percentage
never updates.
Root cause: my live-progress fix in 8de4a18 made the SoundCloud client
compute progress correctly via fragment_index/fragment_count — but the
percent never reached the modal because `get_cached_transfer_data` in
web_server.py iterates `[youtube, tidal, qobuz, hifi, deezer_dl,
lidarr]` to build the lookup that drives `task.progress`. SoundCloud
was missing from that loop, so `live_transfers_lookup` had no entry
for SoundCloud downloads, so `live_info` lookup at
`core/downloads/status.py:135` always missed, so `task_status['progress']`
defaulted to 0 the entire time.
Frontend was reading `task.progress` (rendered as
"Downloading... ${task.progress}%" in `webui/static/downloads.js:3142`),
which stayed at 0. The percentComplete field that the
`/api/downloads/status` endpoint includes for SoundCloud was correct;
this only affected the cached lookup used by the V2 task tracker.
Fix: include SoundCloud in the iteration. Used `getattr` fallback to
match the same pattern I used in `core/downloads/monitor.py` so older
soulseek_client snapshots without the attribute don't AttributeError.
Bonus: also wired the SoundCloud client's `set_shutdown_check` callback
in the startup block right after HiFi's. Previously the cooperative-
cancellation hook in `_progress_hook` would never fire on shutdown
because `self.shutdown_check` was None.
Verified: full suite 1732 passed, ruff clean. yt-dlp probe confirms
fragment_index / fragment_count are populated correctly during HLS
download (164 hook calls for a 19-fragment track), so the now-
exposed progress will increment smoothly from 0 to 99.9 and then
flip to Completed.
User report: switched download source to SoundCloud and noticed:
1. Download progress % stays at 0 until "suddenly done" — no live progress
2. Sidebar status indicator next to "SoundCloud" label is red
3. Dashboard service status card still shows "Soulseek" as the source name
Fix 1 — Live progress for HLS-segmented SoundCloud downloads
(`core/soundcloud_client.py`):
- yt-dlp's `total_bytes` / `total_bytes_estimate` for HLS describes the
CURRENT FRAGMENT, not the whole download. So the byte-based
percentage stayed near 0 the entire time — until 'finished' fired.
- Added `_update_download_progress_fragmented` which uses
`fragment_index` / `fragment_count` (which yt-dlp DOES populate
accurately for HLS) to compute a meaningful percentage. Total size
is extrapolated from per-fragment average for the bytes/remaining
display. Time-remaining estimate uses elapsed/index seconds-per-
fragment.
- The progress hook prefers fragment progress when both fragment_index
and fragment_count are present; falls back to byte-based for
non-fragmented (progressive MP3) downloads. Five new unit tests pin
the fragment-progress math, the 99.9% cap, and the defensive
zero-index / unknown-id paths.
Fix 2 — Sidebar status indicator stays green for SoundCloud mode
(`web_server.py`):
- The `/api/status` route's `serverless_sources` tuple decides whether
to even probe slskd. SoundCloud (and Lidarr) were missing — so when
the active source was SoundCloud, the route fell through to "test
slskd, mark not-relevant", which set `connected: False` and turned
the sidebar dot red even though SoundCloud was working.
- Added `'soundcloud'` and `'lidarr'` to the tuple. Both are
serverless from slskd's perspective, so the dot now stays green
whenever they're the active source.
Fix 3 — Dashboard service card title shows the active source
(`webui/static/shared-helpers.js`):
- The dashboard's "Download Source" card has its own
`sourceNames` map at line 3351 (separate from the sidebar map I
already updated at 3396). Missed it during the integration PR.
- Added `'lidarr'` and `'soundcloud'` so the card title now reads
"SoundCloud" / "Lidarr" instead of falling back to "Soulseek".
Bonus — Dashboard "Test Connection" button works for SoundCloud
(`core/connection_test.py`):
- The dashboard's Test Connection button on the download-source card
sends `service` based on the active source — so for SoundCloud it
was sending `service='soundcloud'`. `run_service_test` had no
branch for it, so it fell through to "Unknown service." and the
button always failed.
- Added a `soundcloud` branch that mirrors `/api/soundcloud/status`
behavior: confirms yt-dlp is installed, runs a real cheap probe,
returns a meaningful pass/fail. (HiFi has the same gap but no
user reported it; out of scope for this fix.)
Verified:
- 41 unit tests pass (5 new fragment-progress tests added)
- Full suite 1732 passed
- Ruff clean
Plug the previously-built SoundcloudClient (PR #478, the build-and-verify
phase) into every place a download source needs to appear. Follows the
same wiring contract as Tidal/Qobuz/HiFi/Deezer/Lidarr — orchestrator
routing, hybrid-mode picker, search dispatch, queue/cancel/clear,
provenance + library history, sidebar source label, settings UI all
work plug-and-play.
Backend wiring:
- `core/download_orchestrator.py` — import SoundcloudClient, _safe_init
it at startup, add to _client() lookup, get_source_status(),
check_connection's sources_to_check default, search source_names map,
search_and_download_best _streaming_sources tuple, download
source_map + source_names, and every iteration loop in
reload_settings download-path-update / get_all_downloads /
get_download_status / cancel_download (route + iterate) /
clear_all_completed_downloads / cancel_all_downloads.
- `core/downloads/monitor.py` — added SoundCloud to the per-client
loop that fetches active downloads outside the orchestrator (uses
getattr fallback for older soulseek_client snapshots).
- `core/downloads/task_worker.py` — added SoundCloud (and Lidarr,
which was missing too — bonus fix) to source_clients dict for hybrid
fallback dispatch.
- `core/downloads/validation.py` — added 'soundcloud' to
_streaming_sources so SoundCloud results go through the matching
engine validation path instead of the Soulseek quality-filter path.
- `core/imports/side_effects.py` — three call sites: source_map for
download_source label written to library_history, streaming-source
guard for the `||`-encoded stream_id parsing, and source_service
map for provenance recording. All three now include 'soundcloud'.
- `web_server.py` — five streaming-source detection tuples updated.
New `/api/soundcloud/status` endpoint returns
{available, configured, reachable} mirroring the Deezer/HiFi
status-endpoint pattern; reachability runs a real cheap yt-dlp
search so the settings Test Connection button gives a meaningful
pass/fail signal.
- `config/settings.py` — added empty `soundcloud_download` defaults
block so future tier-2 OAuth (SoundCloud Go+ session) doesn't have
to migrate existing configs.
Frontend:
- `webui/index.html` — new `<option value="soundcloud">` in the
download-source-mode dropdown, SoundCloud added to both hidden
legacy hybrid-source selects, new settings container with info
text + Test Connection button.
- `webui/static/settings.js` — HYBRID_SOURCES entry (with the
SoundCloud cloud SVG icon), _hybridSourceEnabled default,
updateDownloadSourceUI container display, allSources for legacy
hybrid picker, testSoundcloudConnection function (hits the new
status endpoint, color-codes the result), saveSettings
soundcloud_download empty block.
- `webui/static/shared-helpers.js` — sidebar source-name map
includes SoundCloud + Lidarr (Lidarr was also missing, bonus fix).
- `webui/static/helper.js` — WHATS_NEW entry under '2.4.2' dev cycle
describing the user-visible change in the chill terse voice.
Tests:
- `tests/test_download_orchestrator_soundcloud.py` — 14 integration
tests verifying the wiring: client constructed at startup, _client
lookup resolves 'soundcloud', get_source_status includes it,
download dispatcher routes username='soundcloud' to the SoundCloud
client (and unknown usernames still fall back to Soulseek), hybrid
search iterates SoundCloud when in order and skips it cleanly when
unconfigured, get_all_downloads / get_download_status / cancel /
clear walk SoundCloud, soundcloud-only mode dispatches only to
SoundCloud, _streaming_sources tuple in validation includes
'soundcloud'.
- `tests/downloads/test_download_orchestrator.py` — added
`soundcloud` to the test fixture's _build_orchestrator helper so
the new orchestrator attribute doesn't AttributeError in pre-
existing tests that bypass __init__.
Verified:
- Full suite green (1728 passed, 2 deselected for soundcloud_live)
- Ruff clean
- Live SoundCloud-only mode search returns 25 SoundCloud tracks for
"kendrick lamar luther" in <2s, returning properly-shaped
TrackResult objects with username='soundcloud' and dispatch-key
filename ready for the download path.
Out of scope (intentional deferrals):
- SoundCloud Go+ OAuth tier (256 kbps AAC) — anonymous-only for now.
Adding auth later is a settings-page extension, no orchestrator
changes needed.
- Album/playlist support — SoundCloud has playlists but they don't
map to the album model the rest of SoulSync expects. Singles only.
Reported case (Foxxify): Tidal returned error 1002 ("Invalid redirect
URI") on every authentication attempt for users accessing SoulSync
from a network IP. User had ``http://127.0.0.1:8889/tidal/callback``
registered in his Tidal Developer Portal — matching the SoulSync UI
default and docs.
Root cause: the /auth/tidal route at web_server.py:5594-5598 had a
"fallback: dynamically set based on request host" branch that fired
when ``tidal.redirect_uri`` config was empty AND the request didn't
come from localhost. That fallback overrode the TidalClient
constructor's safe default (``http://127.0.0.1:<port>/tidal/callback``)
with a uri built from request.host like
``http://192.168.x.x:8889/tidal/callback``. Tidal compares strings
exactly so this never matched the documented portal registration and
the user got 1002 before the consent screen even rendered.
The trap is the SoulSync settings UI displays the default URI as the
placeholder + "Current Redirect URI" display — but the placeholder
never gets saved to config unless the user explicitly clicks Save.
Most users who follow the docs (register the displayed default with
Tidal, then click Authenticate) hit the empty-config path and the
broken fallback.
Fix: drop the request-host fallback. Empty config falls back to the
constructor default that matches the documented portal registration.
The existing post-auth swap-step in the instructions page below
handles the Docker / remote-access case as designed:
1. SoulSync sends 127.0.0.1:8889 in the authorize URL → matches
portal → Tidal accepts.
2. User authorizes → Tidal redirects browser to 127.0.0.1:8889
(which fails locally — nothing on user's machine listens there).
3. Instructions tell user to swap 127.0.0.1 with the host they're
accessing SoulSync from.
4. Swapped URL hits the container's exposed callback port → auth
completes.
8 regression tests in tests/test_tidal_auth_redirect_uri.py:
- Configured redirect_uri sent verbatim (localhost / custom port /
explicit network IP)
- Empty config falls back to constructor default — NOT request.host
(the actual reported scenario, with explicit assertion message
warning if the bug returns)
- Empty config + localhost access uses the same default (sanity)
Full pytest 1635 passed; ruff clean.
Discord-reported (Foxxify): logging in to Qobuz via the Connect
button on Settings showed "Connected: <username> (Active)" but
underneath an error said "Qobuz not authenticated...", and the
dashboard indicator stayed yellow. Saving settings or reloading the
tab didn't help.
Root cause: SoulSync runs two QobuzClient instances side by side —
one through soulseek_client.qobuz for the /api/qobuz/auth/* endpoints,
and a second owned by the enrichment worker thread for thread safety.
The login flow only updated the auth-flow instance's in-memory state
(plus persisted to config). The dashboard's "configured" check at
web_server.py:3371 reads
``qobuz_enrichment_worker.client.user_auth_token`` — the WORKER's
instance — which still believed itself unauthenticated. The
connection-test step at core/connection_test.py:370 hits the same
worker instance for the same reason.
Fix: add ``QobuzClient.reload_credentials()`` — a public, network-free
method that re-reads the saved session from config and updates the
instance's in-memory state + session headers. Call it on the
enrichment worker's client immediately after a successful
``/api/qobuz/auth/login``, ``/api/qobuz/auth/token``, or
``/api/qobuz/auth/logout`` so the two instances stay in lockstep
without waiting for the next process restart.
Unlike the existing ``_restore_session()`` this skips the network
probe — the caller has just authenticated, so the token is known
good. A small ``_sync_qobuz_credentials_to_worker()`` helper in
web_server.py wraps the call so all three endpoints share one path.
10 new regression tests cover the populate / clear / partial-config
paths plus the actual two-instance-sync scenario from the bug report.
Full pytest 1555 passed (the one pre-existing flake in
test_tidal_auth_instructions.py is order-dependent and unrelated).
- keep existing /api/image-proxy URLs from being wrapped again
- reuse the shared metadata package instead of duplicating URL logic in web_server.py
- add regression coverage for proxy passthrough and internal URL normalization
- move Spotify status publishing onto auth, disconnect, and rate-limit transitions
- keep dashboard and debug consumers on the shared cached snapshot
- leave only the initial snapshot seed as a fallback probe
- move metadata-source and Spotify status caching out of web_server.py
- keep the public /status payload unchanged while shrinking server-side glue
- centralize invalidation and TTL handling in core/metadata/status.py
- cache Spotify auth and rate-limit status separately from the generic metadata source snapshot
- refresh Spotify status only on explicit auth/disconnect/test paths or after the TTL expires
- keep the legacy OAuth callback paths aligned with the same invalidation helper
- Keep the primary metadata provider snapshot generic and move Spotify auth/rate-limit details into a separate status object.
- Update the websocket fixture and dashboard/settings consumers to read the two buckets independently.
Followup to the enrichment-bubble registry consolidation. The
dashboard polling + click handlers all hit
/api/enrichment/<service>/{status,pause,resume} now, so the 30
hand-rolled per-service routes in web_server.py have zero callers
and can come out:
/api/musicbrainz/{status,pause,resume}
/api/audiodb/{status,pause,resume}
/api/discogs/{status,pause,resume}
/api/deezer/{status,pause,resume}
/api/spotify-enrichment/{status,pause,resume}
/api/itunes-enrichment/{status,pause,resume}
/api/lastfm-enrichment/{status,pause,resume}
/api/genius-enrichment/{status,pause,resume}
/api/tidal-enrichment/{status,pause,resume}
/api/qobuz-enrichment/{status,pause,resume}
Worker init blocks stay (they still construct the workers + persist
pause state). Section comment headers are preserved with a one-line
note pointing readers at the new generic blueprint.
Test fixtures in tests/conftest.py and
tests/metadata/test_enrichment_events.py also updated to use the
new URL paths so they reflect production reality. They were
synthetic stubs that never depended on the production routes —
purely cosmetic alignment.
Net: ~510 lines deleted from web_server.py. Full pytest 1541
passed; ruff clean.
The dashboard's enrichment-status bubbles (MusicBrainz, AudioDB,
Discogs, Deezer, Spotify, iTunes, Last.fm, Genius, Tidal, Qobuz) each
had its own copy-pasted /status, /pause, /resume route in web_server.py
— 30 routes that differed only in the worker reference and a couple
of per-service quirks (Spotify's rate-limit guard, Last.fm/Genius
yield-override behavior, Tidal/Qobuz extra status fields).
Replace them with a registry-driven blueprint:
- core/enrichment/services.py declares an EnrichmentService dataclass
with worker_getter, config_paused_key, pre_resume_check,
auto_pause_token, and extra_status_defaults — all variation captured
as data, no branching on service id.
- core/enrichment/api.py exposes a Flask blueprint with three routes
(/api/enrichment/<service>/{status,pause,resume}). Per-service
quirks are honored via the descriptor: Spotify's rate-limit ban
still returns 429 with `rate_limited: true`, Last.fm/Genius still
drop the auto-pause token and add the yield override, Tidal/Qobuz
still merge `authenticated: false` into the fallback payload.
- web_server.py registers all 10 services after their workers
initialize, wires the host-side hooks (config_manager.set,
_download_auto_paused.discard, _download_yield_override.add), and
registers the blueprint.
- webui/static/enrichment.js polling + click handlers now hit the
generic endpoints. The per-service `update<Service>StatusFromData`
functions are unchanged — they still process the same payload.
This is the cutover step. Old per-service routes are intentionally
left in place as a fallback during the soak period — they currently
have zero callers in the codebase and will be deleted in a follow-up
patch once production has run on the new pipeline for a few days.
27 new tests in tests/test_enrichment_services.py cover the registry
behavior + every quirk path through the generic blueprint (rate-limit
guard, auto-pause token cleanup, persisted-pause config keys, extra
default fields, worker-not-initialized fallback, exceptions). Full
suite 1541 passed; ruff clean.
Patch release wrapping up the 2.4.1 dev cycle. Highlights:
- Watchlist no longer re-downloads compilation/soundtrack tracks
(#458 dedup orphan cleanup + the album-match fix work in tandem
to stop the loop).
- Duplicate detector catches slskd dedup orphans via a second
filename-bucket pass.
- Beatport tab hidden temporarily — Cloudflare Turnstile blocks the
scraper and the official OAuth API is closed to public devs.
- Service worker for cover art + installable PWA manifest.
- Browser caching for static assets (1y) and discover pages (5min).
- Socket.IO same-origin default + admin-only /api/settings.
Files updated:
- web_server.py: _SOULSYNC_BASE_VERSION 2.4.0 -> 2.4.1
- webui/index.html: sidebar version button + modal subtitle
- webui/static/helper.js: WHATS_NEW dev-cycle marker -> release date,
fallback version in _getLatestWhatsNewVersion, 8 new
VERSION_MODAL_SECTIONS entries promoted from this cycle
- .github/workflows/docker-publish.yml: workflow_dispatch default
version_tag updated to 2.4.1
- Return a distinct post-auth warning page when Spotify OAuth completes but the client still does not report an authenticated session.
- Send the completion signal back to the opener so the settings UI can refresh and show the warning state immediately.
- Keep the standalone callback server and the main Flask callback path aligned on the same result-page helper.
- Make the Spotify auth completion popup notify the opener across callback origins.
- Refresh service status in the settings UI after auth completes so the button flips to Disconnect immediately.
- Keep the standalone callback instruction page and the main app flow working with the same completion signal.
- Send Spotify auth completion back to the opener so the settings page refreshes immediately
- Make the local auth flow go straight through to Spotify instead of showing the temporary instruction page
- Keep the remote/docker instruction page available for manual callback setups
- Sync Spotify status, connect/disconnect buttons, and metadata source selection after auth and disconnect
- Keep the disconnect behavior aligned with the active primary metadata source
- Hide the auth button when a Spotify session is active
- Treat disconnect as a session change, not a provider swap
- Share metadata source labels in the registry
- Tighten rate-limit copy around Spotify-specific behavior
Pytest tears down its log file handles before atexit runs. Every
"Shutting down ..." line a worker emits while stopping then crashes
Python's logger with "I/O operation on closed file" and floods CI
stderr with --- Logging error --- traceback blocks. The CI sanity
check workflow noticed once tests started importing web_server (the
Tidal-auth integration test PR + this parallel-imports PR are the
first two test files that boot the full module).
Adds a tiny atexit handler that flips ``logging.raiseExceptions =
False`` BEFORE the other shutdown handlers run. atexit's LIFO order
makes "registered last" mean "runs first", so this fires ahead of
cleanup_monitor / _atexit_shutdown / _atexit_save_history and any
log calls those make can't bubble the closed-stream traceback.
The shutdown messages themselves are best-effort debug
breadcrumbs, not data we need to preserve at process exit, so
silencing the internal handler errors costs nothing.
Discord-reported (fresh.dumbledore + maintainer ack): the
/api/import/singles/process route iterated staging files through a
plain Python for loop. Per-file work is dominated by metadata
search round-trips (Spotify/iTunes/Deezer/Discogs), so a multi-
track manual import on a typical home network was painfully slow.
Adds a dedicated import_singles_executor (3 workers) alongside the
existing executor pool, and refactors the route to submit every
file at once and aggregate results via as_completed. Worker count
balances throughput against any single provider's per-source rate
limits — the same shape used by missing_download_executor.
Extracts the per-file pipeline into _process_single_import_file
which returns a typed (status, payload) outcome:
- ("ok", final_title) on success
- ("error", message) for missing/malformed input or pipeline failure
The worker wraps its own exceptions so a single bad file can't
crash the batch; the route adds a belt-and-suspenders try/except
around future.result() for any worker-level surprises.
Pipeline thread-safety verified: post_process_matched_download
already serializes per-file via post_process_locks (one lock per
context_key — and each import gets a unique UUID context_key), DB
writes serialize through SQLite's WAL + busy_timeout, metadata
registry uses RLocks, no bare module-level mutable state.
Adds 9 regression tests:
- 4 worker-contract tests (missing file, malformed match, pipeline
exception wrapping, happy-path return shape)
- 2 executor-config tests (worker count, thread name prefix)
- 1 integration test that proves the route actually parallelizes
by checking wall-clock duration is well under sequential cost
- 1 mixed-outcome aggregation test
- 1 worker-crash recovery test
Doesn't address the related "stops on tab close" complaint —
that's a separate request-lifecycle issue that needs job_id +
polling, not just parallelism.
Discord-reported: clicking the Tidal "Authenticate" button on a
Docker setup landed users on a remote-access instructions page that
told them their callback URL would look like
http://127.0.0.1:8888/tidal/callback?code=... — Spotify's port,
hardcoded into the Tidal instructions. Users who followed those
instructions literally saved 8888 into their tidal.redirect_uri
setting; that mismatched their Tidal Developer App's registered
:8889 redirect URI and Tidal returned error 1002 (invalid redirect
URI) on every auth attempt.
Pull the port from the actual TidalClient.redirect_uri the OAuth
URL was just built with (urlparse), with the SOULSYNC_TIDAL_CALLBACK_PORT
env var as fallback when the URI can't be parsed. Both the Step 2
example and the Step 3 highlighted URL now reflect whatever Tidal
port the user is actually configured to use.
Adds 3 regression tests covering the reported scenario, custom
callback ports via SOULSYNC_TIDAL_CALLBACK_PORT, and a defensive
fallback when redirect_uri is unparseable. Tests hit the real
/auth/tidal route through Flask's test client and assert the
rendered HTML, so future hardcoded ports get caught immediately.
The /api/library/watchlist-all-unwatched endpoint required the
user's currently active metadata source's ID column on each library
artist. A Spotify-primary user with library artists only matched
against iTunes or Deezer saw them silently skipped — surfacing on
Discord as "Library and Watchlist not syncing correctly". The per-
artist Enhanced View sync sometimes "fixed" them because it triggered
metadata enrichment that occasionally populated the missing Spotify
ID, but couldn't help artists Spotify simply doesn't carry.
Extracts the picker as a standalone helper so it can be tested
directly:
core/watchlist/source_picker.py:pick_artist_id_for_watchlist
Picks the active source first when available, then falls back through
spotify -> itunes -> deezer -> discogs in registration order. Empty
strings count as missing. Numeric IDs are coerced to str so SQLite's
TEXT columns store them in the same form library code reads back.
Returns (None, None) only when the artist has zero source IDs — the
only legitimate skip reason now.
Adds 10 regression tests covering active-source priority for each
supported primary, fallback ordering through every secondary, the
zero-IDs base case, unrecognized active source (e.g. hydrabase still
falls through), empty-string handling, and numeric coercion.
The "Clean Search History" automation card kept showing a stale
'DownloadOrchestrator' object has no attribute 'base_url' error
even after the underlying handler bug was fixed in 77d20e9. Root
cause is in the engine, not that handler: AutomationEngine only
captured uncaught exceptions into last_error. Handlers that
report failure by RETURNING {'status': 'error', ...} were treated
as successful from the engine's perspective, so subsequent
gracefully-failing runs never updated the row to reflect the
current state.
Both the timer (run_automation) and event (_handle_event_trigger)
paths now extract the error string from a result whose status is
'error', falling through 'error' -> 'reason' -> 'message' -> a
placeholder so last_error is never None on actual failures
regardless of which key the handler chose. Existing behaviour for
raised exceptions and successful runs is preserved.
Also normalizes _auto_clean_search_history's return key from
'reason' to 'error' so older deployed engines that only check
the canonical key still see the failure.
Adds 7 regression tests covering every result shape the engine
might receive.
The bulk download_discography endpoint picked one metadata client
based on the configured primary source and called .get_album() on
every album with that single client. Albums whose IDs came from a
fallback/provider-specific source (e.g. Deezer-formatted IDs surfaced
through Hydrabase) failed with "Album not found" because the primary
client couldn't resolve them.
Bulk now uses the same source-aware resolver
(core.metadata.album_tracks.get_artist_album_tracks) the working
individual-album endpoint already uses, so the resolver's source-chain
walk finds each album under whichever provider actually has it. Also
adds explicit Discogs and Hydrabase support (the old if/elif chain
silently 500'd for those primaries).
Frontend (library.js + pages-extra.js) now sends a richer
`{ albums: [{id, name, artist_name, source}] }` payload so each album
can be resolved through its own source. The legacy `album_ids` payload
still works as a fallback path.
Closes#399.
Body byte-identical to the original. Spotify proxy via registry,
iTunes/Deezer client shims wrap registry helpers,
_resolve_library_file_path, _attempt_download_with_candidates, and
missing_download_executor are injected via init() right after
_init_wishlist_failed where all three deps are already defined.
web_server.py: 35239 → 35063 (-176 lines).
- search metadata providers in source-priority order for each generated query instead of caching one client for the whole scan
- keep the quality-scanner worker provider-neutral and preserve the no-provider error path
- update the quality-scanner tests and remove the obsolete web_server spotify_client injection
Body byte-identical to the original. Wishlist helpers come from
core.wishlist.* directly (aliased to the same names the body uses);
runtime state from core.runtime_state. automation_engine,
soulseek_client, and _sweep_empty_download_directories are injected
via init() right after _init_download_validation.
web_server.py: 35408 → 35239 (-169 lines).
Body byte-identical to the original. matching_engine and
soulseek_client are injected via init() right after _init_discover_hero
since both originals are constructed early in web_server.py boot
(L598/L610) and never rebound.
web_server.py: 35586 → 35408 (-178 lines).
Body byte-identical to the original. Spotify proxy via registry,
_get_active_discovery_source and get_current_profile_id redefined
as stateless shims, _get_metadata_fallback_client injected via init()
because it composes multiple registry helpers wired in web_server.py.
web_server.py: 35753 → 35586 (-167 lines).
Both function bodies (_discovery_score_candidates and
_search_spotify_for_tidal_track) are byte-identical to the originals.
The shared matching_engine instance is injected via init() right after
_init_connection_test; the spotify proxy + _get_metadata_fallback_source
shim follow the same pattern used elsewhere.
web_server.py: 36019 → 35753 (-266 lines).
Both function bodies byte-identical to the originals. The spotify
proxy resolves through core.metadata.registry; the tidal proxy is
backed by an injected getter so a Tidal re-auth that rebinds
web_server.tidal_client is visible. 13 state dicts and helpers are
injected via init() after _init_connection_test, when all deps
already exist.
web_server.py: 36260 → 36019 (-241 lines).
Body byte-identical to the original. Pure stdlib + requests, no
web_server-specific globals or runtime state — no init() needed.
web_server.py: 36500 → 36261 (-239 lines).
- Switch the download lifecycle over to the neutral wishlist track helper name
- Keep the old Spotify helper as a compatibility alias for older callers
- Store track_data as the primary failed-download wishlist payload key and add regression coverage
Body byte-identical to the original. Five deps (soulseek_client,
qobuz_enrichment_worker, hydrabase_client, docker_resolve_url,
docker_resolve_path) are injected via init() right after the
register_runtime_clients block — that is the earliest point at which
hydrabase_client is guaranteed to exist.
web_server.py: 36833 → 36500 (-333 lines).
Body byte-identical to the original. The shared state dict, lock,
docker_resolve_path helper, and automation engine are injected via
init() at the lift point, where all four originals are already defined.
web_server.py: 37015 → 36833 (-182 lines).
Lifts _search_service and its _detect_provider helper. Both bodies are
byte-identical to the originals. The nine enrichment worker handles
(spotify/itunes/mb/lastfm/genius/tidal/qobuz/discogs/audiodb) are
injected via init() right after qobuz is constructed, which is the
last worker to come up — and well before Flask starts accepting
requests, so the route handlers never see unbound workers.
web_server.py: 37245 → 37015 (-230 lines).
Lifts _match_liked_artists_to_all_sources and
_backfill_liked_artist_images. Both bodies are byte-identical to the
originals. Uses the same _SpotifyClientProxy + _get_*_client shim
pattern as core/artists/map.py so the bodies resolve their original
names without modification.
web_server.py: 37501 → 37245 (-256 lines).
Class body byte-identical to original. Module-level IS_SHUTTING_DOWN
flag is mirrored from web_server's own flag in _shutdown_runtime_components
so the monitor loop still sees shutdown signals at the right moment.
Eight web_server-side helpers (_make_context_key, _on_download_completed,
_run_post_processing_worker, _download_track_worker,
_start_next_batch_of_downloads, _orphaned_download_keys,
missing_download_executor, soulseek_client) are injected via init() after
register_runtime_clients, when all symbols are defined and well before
Flask starts accepting requests.
web_server.py: 38220 → 37501 (-719 lines).
Lifts get_artist_map_data, get_artist_map_genre_list,
get_artist_map_genres, and get_artist_map_explore (plus the
_artmap_cache_* helpers and _artist_map_cache dict) to a new module.
Bodies are byte-identical to the originals. web_server.py keeps
thin route shells that delegate to the lifted functions.
A _SpotifyClientProxy resolves the global spotify_client lazily via
core.metadata.registry.get_spotify_client() so a Spotify re-auth that
rebinds the cached client stays visible to the lifted bodies.
web_server.py: 39124 → 38220 (-904 lines).
Class body byte-identical to original. The shared metadata_update_state
dict is bound at import time via init() so the class body can mutate
it without web_server.py rebinding.
web_server.py: 39754 → 39122 (-632 lines).
The Spotify enrichment worker was auto-starting unconditionally at boot,
hammering /v1/search to match every track in the library against the
Spotify catalog regardless of which metadata source the user had
actually chosen as their primary. Users on Deezer, iTunes, Discogs,
or Hydrabase saw multi-hour 429 bans (typically 14400s) on Spotify
even though they never wanted Spotify-driven enrichment in the first
place — the worker generated dead API traffic the user neither asked
for nor benefited from.
Compounded by Spotify's February 2026 API tightening:
- /v1/search max limit cut from 50 to 10 per request, default from
20 to 5 — every track now needs more pagination, more requests.
- Sustained-rate detection more aggressive — repeated calls over
hours trigger automated long-form bans even when each individual
30-second window is well under the rolling limit.
Result: a user on Deezer would see their Spotify connection get banned
for 4 hours after about 30 tracks of enrichment activity, with no
recourse other than manually pausing the worker each session.
Two-part fix:
1. Boot gate (web_server.py): only auto-start the worker when
`get_primary_source() == 'spotify'`. Otherwise initialize in the
paused state with an explanatory log line. The settings UI manual
unpause control remains functional for users who explicitly want
background Spotify enrichment regardless of primary source.
Boot logic:
- User manually paused (existing config) → stays paused (preserved).
- Primary = 'spotify' → starts running (preserved).
- Primary != 'spotify' → starts paused with log line.
2. Daily budget reduction (core/spotify_worker.py): drop from 3000 to
500 items per calendar day. The 3000 cap was set when /v1/search
returned 50 results per call; now that it caps at 10, each track
needs roughly 5x the API load to find a confident match. 500/day
keeps the worker productive without crossing Spotify's hidden
sustained-rate detection threshold.
The runtime side of the boot gate — auto-pausing when the user
switches primary source mid-session — is out of scope. The settings
UI already exposes the manual toggle, and primary-source switches are
infrequent enough that requiring a manual unpause after the fact is
acceptable.
Full suite: 1355 passing. Ruff clean.
Final lift in the web_server.py extraction effort. Pulls two route
handlers + one background worker out of `web_server.py` into new
focused packages:
- `core/streaming/prepare.py` — 258-line stream-prep worker that
downloads a track to the local Stream/ folder for the browser audio
player.
- `core/playlists/explorer.py` — 305-line route handler for
`POST /api/playlist-explorer/build-tree` that streams an NDJSON
discography tree from a mirrored playlist.
What `prepare_stream_task` does:
1. Reset stream state to 'loading' with the new track info.
2. Clear any prior file from Stream/ (only one stream lives there).
3. Spin up a fresh asyncio event loop and `soulseek_client.download()`.
4. Poll progress every 1.5s. Queue timeout 15s; overall 60s.
5. On succeeded + bytes-match: find the file with retry, move into
Stream/, signal slskd completion, mark state 'ready' with file_path.
6. On error/timeout/cancel: state goes to 'error' or 'stopped'.
7. Finally: tear down the event loop cleanly.
What `playlist_explorer_build_tree` does:
1. Validate request, load playlist + tracks from DB.
2. Pick active metadata source (Spotify if authed, else fallback).
3. Group tracks by artist using discovered matched_data when the
provider matches the active source.
4. Stream NDJSON: meta line → one artist line per group → complete line.
5. Per artist: cache check → resolve discography → tag releases with
`in_playlist` flag based on title-similarity match → filter by mode
(`albums` = only matches; `discographies` = full disco).
6. Mark playlist as explored on completion.
Strict 1:1 byte parity:
Both functions exposed their dependencies through proxy patterns
established in earlier lifts (PR4–PR8). For prepare_stream_task,
`stream_state` is a deps property; for the explorer, Flask `request` /
`jsonify` / `Response` are injected via deps so the lifted body keeps
its native syntax. Both lifts verified ZERO diff against the original
after `deps.X` → global X normalization.
258 lines orig = 258 lines lifted (prepare_stream_task).
305 lines orig = 305 lines lifted (explorer).
Bonus cleanup: web_server.py's module-level `import shutil` and
`import glob` were now unused (only `_prepare_stream_task` used them
at module scope; every other reference is via inline `import shutil`
in respective function bodies). Removed both module-level imports —
ruff caught the F811 redefinitions and confirmed they're truly
redundant.
Dependencies for `PrepareStreamDeps` (11 fields):
config_manager, soulseek_client, stream_lock, project_root,
docker_resolve_path, find_streaming_download_in_all_downloads,
find_downloaded_file, extract_filename, cleanup_empty_directories,
plus 2 stream_state property delegates.
Dependencies for `PlaylistExplorerDeps` (9 fields):
Flask request/Response/jsonify, spotify_client, get_database,
get_active_discovery_source, get_metadata_fallback_client,
get_metadata_fallback_source, get_metadata_cache.
Tests: 6 new under tests/streaming/test_prepare.py (state init,
Stream/ folder creation + clearing, download-init failure, completed
+ moved + ready state, partial-bytes incomplete-warning path) plus 9
new under tests/playlists/test_explorer.py (5 validation early-exit
paths, streaming response shape with meta/complete lines, mark-
explored side effect, discovered-artist grouping using matched_data,
provider mismatch falling back to raw artist name).
Full suite: 1355 passing (was 1340). Ruff clean.
End of the web_server.py extraction effort. Started at ~45,000 lines
across PR4–PR8 + this commit; finished around 35,000 lines with the
heavy worker + route logic now living in domain-cohesive packages
under core/. The remaining bulk in web_server.py is route handlers,
service initialization, and the deferred 1530-line
`_register_automation_handlers` (startup-only, marginal lift value).
Pulls the 284-line artist quality enhancement helper out of
`web_server.py` into a new `core/artists/` package. Flask route handler
split: route + request parsing stay in web_server.py, the body lifts to
a pure function returning `(payload_dict, http_status_code)`.
What `enhance_artist_quality` does:
1. Validate request: track_ids must be non-empty, artist must exist.
2. Build a `track_lookup` from `database.get_artist_full_detail` so each
selected track resolves with its album context.
3. Per track:
- Read current quality tier from the file extension.
- Build `matched_track_data` for the wishlist entry, in priority
order:
- Spotify direct lookup via stored `spotify_track_id` (preferred).
Uses raw API data when available; otherwise rebuilds the payload
and pulls album images via a follow-up `get_album` call.
- Spotify search fallback using matching_engine queries with
artist+title similarity scoring (album-type bonus for albums,
smaller bonus for EPs). Stops at first >= 0.9 confidence match.
- iTunes/fallback source search with the same scoring shape.
- Add to wishlist via `wishlist_service.add_spotify_track_to_wishlist`
with `source_type='enhance'` and a `source_context` carrying the
original file path, format tier, bitrate, original_tier, and
artist_name.
- Tally `enhanced_count` / `failed_count` / per-track failure reasons.
4. Return `{success, enhanced_count, failed_count, failed_tracks}` 200.
Dependencies injected via `ArtistQualityDeps` (7 fields) — spotify_client,
matching_engine, get_database, get_wishlist_service,
get_current_profile_id, get_quality_tier_from_extension,
get_metadata_fallback_client.
Diff vs original after `deps.X` → global X normalization is **1 line of
cosmetic drift** — the success return now uses an explicit `(payload, 200)`
tuple to keep all returns shape-consistent for the wrapper. Flask treats
`jsonify(x)` and `(jsonify(x), 200)` identically. 284 lines orig = 285
lines lifted, body otherwise byte-identical.
Tests: 10 new under tests/artists/test_quality.py covering input
validation (empty track_ids, artist not found), Spotify direct lookup
via raw_data, Spotify direct lookup with enhanced format requiring
album image rebuild, Spotify search fallback, iTunes/fallback source
match path, track-not-found and no-file-path failure modes, complete
no-match failure, and source_context payload assertions (enhance flag,
file path, format tier, bitrate, source_type).
Full suite: 1340 passing (was 1330). Ruff clean.
Pulls the 258-line retag worker out of `web_server.py` into a new
`core/library/` package. Pure 1:1 lift — wrapper keeps the original
entry-point name so the retag-trigger endpoint continues to work
without changes.
What `execute_retag` does:
1. Fetch album + track metadata for the new `album_id` (Spotify or
iTunes — the Spotify client transparently falls back).
2. Load existing files in the retag group from the DB.
3. Match each existing track to a new Spotify track:
- Priority 1: same disc + track number.
- Priority 2: title similarity >= 0.6 (SequenceMatcher).
4. For each matched pair:
- Re-write metadata tags via `_enhance_file_metadata`.
- Compute the new path via `_build_final_path_for_track` and move
the audio file (plus .lrc / .txt sidecars) if the path changes.
- Drop an orphaned cover.jpg if it's left in an empty directory.
- Clean up empty parent directories left behind.
- Download the new cover art into the new album dir.
5. Update the retag group record with new artist / album / image /
total_tracks / release_date and the appropriate Spotify-or-iTunes
album ID (numeric → iTunes, alphanumeric → Spotify).
6. Mark the retag state 'finished' (or 'error' on exception).
Strict 1:1 byte parity:
The original mutated `retag_state` as a module global (the function
declared `global retag_state` even though it only mutates in place).
Here `retag_state` is exposed through the `RetagDeps` proxy as a Python
property so the lifted body keeps `name[key] = value` /
`name.update(...)` syntax. The property setter rebinds the
web_server.py reference if the function ever reassigns it (currently
it doesn't, but the setter is wired for parity with the watchlist lift).
Diff vs original after `deps.X` → global X normalization is **zero
differences** apart from the dropped `global retag_state` decl and the
inline `from database.music_database import get_database` (replaced by
deps.get_database()). 258 lines orig = 258 lines lifted, byte-identical
body otherwise.
Dependencies injected via `RetagDeps` (13 fields) — config_manager,
retag_lock, spotify_client, plus 8 callable helpers
(get_audio_quality_string, enhance_file_metadata,
build_final_path_for_track, safe_move_file, cleanup_empty_directories,
download_cover_art, docker_resolve_path, get_database) and 2 property
delegates (_get_retag_state / _set_retag_state).
Tests: 11 new under tests/library/test_retag.py covering setup error
paths (no album data, no album tracks, no existing tracks),
track-number priority match, title-similarity fallback, no-match skip,
missing file skip, file move when path changes, group record update
(spotify vs iTunes ID branching by alphanumeric vs numeric album_id),
multi-disc total_discs computation.
Full suite: 1330 passing (was 1319). Ruff clean.
Pulls the 390-line watchlist auto-scan orchestrator out of `web_server.py`
into a new `core/watchlist/` package. Watchlist (followed-artists scanner
that finds new releases) is a separate domain from kettui's wishlist
(failed-download retry queue), so this lift does not overlap with the
ongoing PR400-style extractions.
What `process_watchlist_scan_automatically` does:
1. Smart stuck-detection guard before acquiring the timer lock —
prevents deadlock when a previous scan flag is dangling past the
2-hour timeout.
2. Inside the timer lock: re-check + set the active scan flag with the
current timestamp.
3. Per-profile expansion (or single-profile when manually triggered):
- Watchlist count check + Spotify auth gate.
- Backfill missing artist images.
4. Initialize a fresh `watchlist_scan_state` dict (the deps property
setter rebinds the web_server.py module-level name so external
sentinel checks via id() comparison still detect the swap).
5. Pause enrichment workers, then call
`WatchlistScanner.scan_watchlist_artists` with a per-event progress
callback that translates scanner events into automation log lines.
6. Post-scan steps (skipped if the scan was cancelled mid-flight):
- Populate discovery pool from similar artists (per-profile).
- Refresh ListenBrainz playlists.
- Update current seasonal playlist (weekly cadence).
- Generate Last.fm radio playlists.
- Sync Spotify library cache.
- Activity feed entry + automation_engine.emit('watchlist_scan_completed').
7. On exception: mark state['status']='error', re-raise so the
automation wrapper records the failure.
8. Finally: resume enrichment workers, clear the scanner's rescan
cutoff, reset the auto-scanning flag.
Strict 1:1 byte parity:
The original mutated `watchlist_auto_scanning`,
`watchlist_auto_scanning_timestamp`, and `watchlist_scan_state` as
module globals (with a leading `global` decl). Here those names are
exposed through the `WatchlistAutoScanDeps` proxy as Python properties
so the lifted body keeps the same `name = value` / `name[key] = value`
shape. Property setters fan writes back to web_server.py via callback
pairs.
Diff vs original after `deps.X` → global X normalization is **zero
differences** apart from the dropped `global` declaration line — Python
doesn't need it once the names are property accesses on the deps object.
390 lines orig = 390 lines lifted, byte-identical body otherwise.
Dependencies injected via `WatchlistAutoScanDeps` (15 fields total) —
Flask app, spotify_client, automation_engine, watchlist_timer_lock, plus
5 callable helpers and 6 property delegate callbacks (paired
get/set for each of the three globals).
Tests: 11 new under tests/watchlist/test_auto_scan.py covering
stuck-detection guard, race-check inside lock, zero-watchlist short-
circuit, unauthenticated Spotify gate, successful scan with all post-
scan steps, automation event emission, activity feed logging,
cancellation mid-scan skipping post-steps, profile-scoped trigger,
flag reset in finally, rescan cutoff clear in finally.
Full suite: 1319 passing (was 1308). Ruff clean.
- let core.metadata.registry own per-profile Spotify client caching
- register the DB-backed profile credentials provider from web_server.py
- invalidate only the affected profile cache entry on save, delete, and auth
- make web_server.py read and refresh Spotify from core.metadata.registry
- add single-key metadata cache eviction for Spotify reauth
- export the new cache helper through the metadata package shims
Pulls the 201-line staging-folder shortcut out of `web_server.py` into
its own module under the existing `core/downloads/` package. Pure 1:1
lift — wrapper keeps the original entry-point name so the task worker's
existing call site continues to work without changes.
What `try_staging_match` does:
1. Pull the per-batch staging-file cache (one filesystem scan per batch).
2. For each staging entry, compute title + artist similarity using
SequenceMatcher and the matching engine's `normalize_string`. Require
title >= 0.80, then a combined score >= 0.75. The weighting flips
based on whether artist info is available on both sides:
- both have artist: 0.55*title + 0.45*artist
- either side missing artist: 0.80*title + 0.20*artist (lean on title)
3. Copy the matched file to the configured transfer dir (with a
"_staging" suffix when the destination filename already exists, to
avoid overwriting a legitimate prior download).
4. Mark the task as 'post_processing', username='staging',
staging_match=True.
5. Build a synthetic spotify_artist / spotify_album context (mirroring
the modal-worker logic so the file-organization template applies
cleanly) and store it under "staging_<task_id>". Two paths:
- Explicit context branch (track_info has _is_explicit_album_download)
→ real album/artist data copied through.
- Fallback branch → synthesized from track + track_info, with
`is_album_download` heuristically derived (album differs from title
and isn't "Unknown Album").
6. Hand off to `_post_process_matched_download_with_verification` which
does tagging, path building, AcoustID verification, and DB insertion.
Returns True if the staging shortcut won; False to fall through to the
normal Soulseek search path.
Dependencies injected via `StagingDeps` (5 fields) — config_manager,
matching_engine, get_staging_file_cache, docker_resolve_path,
post_process_matched_download_with_verification.
Diff vs original after `deps.X` → global X normalization is **zero
differences** — 201 lines orig = 201 lines lifted, byte-identical body
(including all whitespace, comments, log strings, and the inline
`from difflib import SequenceMatcher` / `import shutil` imports inside
the function body).
Tests: 9 new under tests/downloads/test_downloads_staging.py covering
no staging files / no track title / low-confidence match returning
False, exact match copying file + transitioning task state + invoking
post-processing, existing-file rename via `_staging` suffix, explicit
album context branch, fallback context synthesis (with both album-as-
album and album-equals-title cases), and copy failure (missing source
file) returning False.
Full suite: 1308 passing (was 1299). Ruff clean.
Missed worker from the PR5 discovery-workers series — Tidal sits in the
same domain as the deezer / spotify_public / listenbrainz / youtube /
beatport workers that were lifted in PR5b–PR5h, follows the same shape,
shares the same `_search_spotify_for_tidal_track` helper, and was simply
overlooked in the original inventory.
Pure 1:1 lift of the 212-line worker. Wrapper keeps the original
entry-point name so the existing call sites in web_server.py continue
to work without changes.
What `run_tidal_discovery_worker` does:
1. Pause enrichment workers (release shared resources).
2. For each Tidal track:
- Cancellation gate (state['cancelled']).
- Discovery cache lookup; cache hit short-circuits the search.
- SimpleNamespace-style track passed straight to
`_search_spotify_for_tidal_track` (the shared helper used by every
worker in this family).
- On Spotify match: build `match_data` preserving track_number /
disc_number from raw API data, image extracted from album images
or track object fallback, release_date filled from
track.release_date when album dict is missing it.
- On iTunes match: dict result populated as `match_data` with source
set to discovery_source, image extracted from album images.
- Save matched result to discovery cache.
- On miss: Wing It stub stored as 'wing-it' status (success ticked).
3. After all tracks: phase='discovered', activity feed entry, sync
discovery results back to mirrored playlist via
`_sync_discovery_results_to_mirrored` with 'tidal' tag.
4. On error: state['phase']='error' + status with error string.
5. Finally: resume enrichment workers.
Dependencies injected via `TidalDiscoveryDeps` (13 fields) —
tidal_discovery_states, spotify_client, plus 11 callable helpers
(pause/resume enrichment, get_active_discovery_source,
get_metadata_fallback_client, get_discovery_cache_key, get_database,
validate_discovery_cache_artist, search_spotify_for_tidal_track,
build_discovery_wing_it_stub, add_activity_item,
sync_discovery_results_to_mirrored). Same surface as the deezer worker.
Diff vs original after `deps.X` → global X normalization is **zero
differences** — 212 lines orig = 212 lines lifted, byte-identical body
(including all whitespace, comments, log strings).
Tests: 9 new under tests/discovery/test_discovery_tidal.py covering
cache hit short-circuit, Spotify tuple match (track/disc preservation),
iTunes dict match path, Wing It fallback, cancellation, completion
phase update, activity feed entry, mirrored sync invocation, per-track
error handling.
Full suite: 1299 passing (was 1290). Ruff clean.
First lift in the new PR6 batch. Pulls the 312-line candidate-fallback
download dispatcher out of `web_server.py` into a new module under the
existing `core/downloads/` package. Pure 1:1 lift — wrapper keeps the
original entry-point name so all callers (search/match pipeline) work
unchanged.
What `attempt_download_with_candidates` does:
1. Sort candidates by descending confidence.
2. For each candidate:
- Cancellation gates (3 points: top of loop, before download starts,
after download_id is assigned).
- Skip already-tried sources via the per-task `used_sources` set.
- Skip blacklisted sources (user-flagged bad matches).
- Race protection: bail when the task already has an active
download_id.
- `update_task_status('downloading')`, then `soulseek_client.download`.
3. On a successful download_id:
- Build `matched_downloads_context` entry keyed by
`make_context_key(username, filename)`.
- For tracks with clean Spotify metadata, pull track_number /
disc_number from (1) track_info → (2) track object → (3) Spotify
API call. When local album context is incomplete, the API response
backfills release_date / album_type / total_tracks / images / id.
- Set `is_album_download` based on explicit context flag or
heuristic (album differs from title, isn't "Unknown Album").
- Store task/batch IDs and track_info on the context for post-
processing + playlist-folder mode.
4. On a cancellation that wins the race after the download started:
- `cancel_download(...)` to stop the in-flight Soulseek transfer.
- `on_download_completed(batch_id, task_id, success=False)` to free
the worker slot.
5. On exception or download-start failure: reset task status to
'searching', continue to next candidate.
Dependencies injected via `CandidatesDeps` (7 fields) — soulseek_client,
spotify_client, run_async, get_database, update_task_status,
make_context_key, on_download_completed.
Diff vs original after `deps.X` → global X normalization is **zero
differences** — 312 lines orig = 312 lines lifted, byte-identical body
(including all whitespace, comments, log strings).
Tests: 14 new under tests/downloads/test_downloads_candidates.py
covering happy path (first candidate succeeds, confidence ordering),
used_sources dedup, blacklist skip, cancellation gates (cancelled
status, deleted task, active download_id, mid-flight cancel + cleanup
callback), failure paths (all candidates failed, exception during
download falls through to next), context payload (explicit album
context, track_number priority order, API backfill of incomplete album
metadata), and equal-confidence stable order.
Pre-existing behavior documented in tests:
`spotify_album_context['id']` initializes to a non-empty placeholder
'from_sync_modal' in the fallback path, so the API-backfill condition
`if not spotify_album_context.get('id')` never fires for the id field
specifically. Other album fields (release_date, album_type) backfill
fine because they default to empty.
Full suite: 1290 passing (was 1276). Ruff clean.
PR400 added imports for `check_and_remove_from_wishlist` and
`check_and_remove_track_from_wishlist_by_metadata` from
`core.wishlist.resolution` (aliased with leading underscores in
web_server.py) but left the original inline definitions of those
functions in place at L17139 and L17243. Python's later definition
wins, so the local defs were silently shadowing the imports — meaning
the new package versions were never actually called from web_server.py.
Ruff caught the redefinition (F811) and broke CI.
Deleted the inline definitions (176 lines). Imports at L143-144 now
serve all callers, and the package functions in
`core/wishlist/resolution.py` are actually exercised. Behavior is the
same: I diffed both versions before deleting and confirmed they're
functionally equivalent.
Tests: 1232 passing (no change). Ruff clean.
Final lift in the PR5 discovery-workers series. Pulls the 328-line
library quality scanner out of `web_server.py` into its own focused
module under `core/discovery/`. Pure 1:1 lift — wrapper keeps the
original entry-point name.
What the quality scanner does:
1. Reset scanner state (counters, results), load quality profile +
minimum acceptable tier from QUALITY_TIERS.
2. Load tracks from DB based on scope:
- 'watchlist' → tracks for watchlisted artists only.
- other → all library tracks.
3. For each track:
- Stop-request gate (state['status'] != 'running').
- Quality-tier check via _get_quality_tier_from_extension(file_path).
- Skip tracks meeting standards (tier_num <= min_acceptable_tier).
- For low-quality tracks: matching_engine search query gen, score
candidates against Spotify (artist + title similarity, album-type
bonus), pick best match >= 0.7 confidence.
- On match: add full Spotify track to wishlist via
`wishlist_service.add_spotify_track_to_wishlist` with
source_type='quality_scanner' and a source_context that captures
original file_path, format tier, bitrate, and match confidence.
4. After all tracks: status='finished', progress=100, activity feed
entry, emit `quality_scan_completed` event for automation engine.
5. On critical exception: status='error', error message captured.
Wishlist service interaction is via the public
`add_spotify_track_to_wishlist` API only — no overlap with kettui's
planned `core/wishlist/` package extraction (the import lives inside
the function, exactly as in the original, and will follow whatever
path that package takes).
Dependencies injected via `QualityScannerDeps` (8 fields) —
quality_scanner_state dict, quality_scanner_lock, QUALITY_TIERS
constant, spotify_client, matching_engine, automation_engine, plus 2
callable helpers (get_quality_tier_from_extension, add_activity_item).
Diff vs original after `deps.X` → global X normalization is **zero
differences** — 328 lines orig = 328 lines lifted, byte-identical body
(including all whitespace, comments, log strings, and the inline
`from core.wishlist_service import get_wishlist_service` /
`from database.music_database import MusicDatabase` imports at the
top of the function).
Tests: 11 new under tests/discovery/test_discovery_quality_scanner.py
covering state init/reset, no-watchlist-artists short-circuit,
unauthenticated Spotify error, high-quality skip, low-quality search
trigger, match → wishlist add (with full source_context payload),
no-match no-add, mid-loop stop request, completion phase + progress,
automation engine event emission, all-library scope load.
Full suite: 1152 passing (was 1141). Ruff clean.
End of the PR5 series — `web_server.py` lost ~328 lines on this commit
alone; total trim across PR5a–PR5h is ~2,400 lines of discovery worker
code moved into focused `core/discovery/*.py` modules. The remaining
discovery-adjacent worker `_process_watchlist_scan_automatically` was
deliberately deferred to avoid overlap with kettui's planned wishlist
extraction.
Seventh lift in the PR5 discovery-workers series. Pulls the 286-line
ListenBrainz discovery worker out of `web_server.py` into its own
focused module under `core/discovery/`. Pure 1:1 lift — wrapper keeps
the original entry-point name.
What the ListenBrainz discovery worker does:
1. Pause enrichment workers (release shared resources).
2. For each ListenBrainz track:
- Cancellation gate (state['phase'] != 'discovering').
- Discovery cache lookup; cache hit short-circuits the search.
- Strategy 1: matching_engine search queries with confidence scoring
against Spotify (preferred) or iTunes (fallback).
- Strategy 2: swapped artist/title query.
- Strategy 3: album-based query (uses album_name when available —
unique to LB, since YouTube tracks don't have album metadata).
- Strategy 4: extended search with limit=50.
- On match → save to discovery cache with image extracted from album
images or matched_track.image_url fallback.
- On miss → Wing It stub stored as 'wing-it' status.
3. After all tracks: phase='discovered', status='complete', activity feed
entry mentioning 'ListenBrainz Discovery Complete'.
4. On error: state['status']='error', phase='fresh'.
5. Finally: resume enrichment workers.
Dependencies injected via `ListenbrainzDiscoveryDeps` (16 fields) —
listenbrainz_playlist_states, spotify_client, matching_engine, plus 13
callable helpers (pause/resume enrichment, get_active_discovery_source,
get_metadata_fallback_client, get_discovery_cache_key, get_database,
validate_discovery_cache_artist, extract_artist_name,
spotify_rate_limited, discovery_score_candidates, get_metadata_cache,
build_discovery_wing_it_stub, add_activity_item).
Diff vs original after `deps.X` → global X normalization is **zero
differences** — 286 lines orig = 286 lines lifted, byte-identical body
(including all whitespace, comments, log strings).
Pre-existing bug preserved (not fixed): if `listenbrainz_playlist_states[
state_key]` raises KeyError on entry, the outer except handler tries to
mutate `state` which is unbound → secondary UnboundLocalError. Same bug
in the original (and the YouTube discovery worker). Documented here for
future cleanup but out of scope for the lift.
Tests: 11 new under tests/discovery/test_discovery_listenbrainz.py
covering cache hit short-circuit, Strategy 1 confidence match, Wing It
fallback, iTunes fallback (Spotify unauthenticated and rate-limited),
cancellation (phase change), completion phase update, activity feed
entry, per-track error handling, float duration_ms tolerance (regression
for the :02d format crash fixed earlier), enrichment workers resume on
finally.
Full suite: 1141 passing (was 1130). Ruff clean.
Sixth lift in the PR5 discovery-workers series. Pulls the 323-line
Beatport chart discovery worker out of `web_server.py` into its own
focused module under `core/discovery/`. Pure 1:1 lift — wrapper keeps
the original entry-point name.
What the Beatport discovery worker does:
1. Pause enrichment workers (release shared resources).
2. For each Beatport track:
- Cancellation gate (state['phase'] != 'discovering').
- Clean Beatport text (artist/title) of common annotations via
`clean_beatport_text` helper.
- Single-string artist normalization for "CID,Taylr Renee"-style
entries — split on comma, take the first.
- Discovery cache lookup; cache hit short-circuits the search and
normalizes cached artists from ['str'] → [{'name': 'str'}] to
match the frontend's expected list-of-objects shape.
- matching_engine search-query generation (with high min_confidence
of 0.9 to avoid bad matches).
- Strategy 1: scored candidates from initial Spotify/iTunes searches.
- Strategy 4: extended search with limit=50 if no high-confidence
match found.
- On Spotify match: format artists as [{'name': str}] objects, pull
full album object from raw cache when available, fallback to
reconstructed album dict otherwise.
- On iTunes match: format with image_url-derived album.images entry
(300x300 spec), source set to discovery_source.
- Save matched result to discovery cache when confidence >= 0.75
(note: lower than search threshold; discovery still benefits from
these less-confident matches as user-visible suggestions).
- On miss: Wing It stub stored as 'wing-it' status (success ticked).
3. After all tracks: phase='discovered', activity feed entry, sync
discovery results back to mirrored playlist via
`_sync_discovery_results_to_mirrored` with 'beatport' tag.
4. On error: state['phase']='fresh' + status='error'.
5. Finally: resume enrichment workers.
Dependencies injected via `BeatportDiscoveryDeps` (17 fields) —
beatport_chart_states, spotify_client, matching_engine, plus 14
callable helpers (pause/resume enrichment, get_active_discovery_source,
get_metadata_fallback_client, clean_beatport_text,
get_discovery_cache_key, get_database, validate_discovery_cache_artist,
spotify_rate_limited, discovery_score_candidates, get_metadata_cache,
build_discovery_wing_it_stub, add_activity_item,
sync_discovery_results_to_mirrored).
Diff vs original after `deps.X` → global X normalization is **zero
differences** — 323 lines orig = 323 lines lifted, byte-identical body
(including all whitespace, comments, log strings).
Tests: 12 new under tests/discovery/test_discovery_beatport.py covering
cache hit short-circuit (with cached-artist normalization), Spotify
match formatting (list and string artist inputs), iTunes match
(image_url to album.images), Wing It fallback, cancellation
(phase change), completion phase update, activity feed entry, mirrored
sync invocation, top-level error handler, per-track error handling,
comma-separated artist split.
Full suite: 1130 passing (was 1118). Ruff clean.
Fifth lift in the PR5 discovery-workers series. Pulls the 278-line
public-Spotify-link discovery worker out of `web_server.py` into its
own focused module under `core/discovery/`. Pure 1:1 lift — wrapper
keeps the original entry-point name.
What the Spotify Public discovery worker does:
1. Pause enrichment workers (release shared resources).
2. For each track:
- Cancellation gate (state['cancelled']).
- Normalize artists to plain string list (handles dict + str inputs).
- Discovery cache lookup; cache hit short-circuits the search and
populates display fields from the cached match.
- SimpleNamespace duck-type → `_search_spotify_for_tidal_track`
(shared search helper, returns tuple for Spotify or dict for iTunes).
- On Spotify match: build `match_data` preserving track_number /
disc_number from raw API data; image extracted from album images
or track object fallback; release_date filled from track.release_date
when album dict is missing it.
- On iTunes match: dict result → match_data with source set to
discovery_source; image extracted from album images.
- Save matched result to discovery cache.
- On miss: Wing It stub stored as 'wing-it' status.
3. After all tracks: phase='discovered', activity feed entry.
4. On error: state['phase']='error' + status with error string.
5. Finally: resume enrichment workers.
This worker is structurally close to the Deezer worker (see PR5d) but
intentionally diverges on:
- Track-data field names (`spotify_public_track` vs `deezer_track`).
- Artist normalization (Spotify Public can pass dicts or strings).
- No mirrored-playlist DB writeback (sync is handled separately).
Dependencies injected via `SpotifyPublicDiscoveryDeps` (12 fields) —
spotify_public_discovery_states, spotify_client, plus 10 callable
helpers (pause/resume enrichment, get_active_discovery_source,
get_metadata_fallback_client, get_discovery_cache_key, get_database,
validate_discovery_cache_artist, search_spotify_for_tidal_track,
build_discovery_wing_it_stub, add_activity_item).
Diff vs original after `deps.X` → global X normalization is **zero
differences** — 278 lines orig = 278 lines lifted, byte-identical body
(including all whitespace, comments, log strings).
Tests: 10 new under tests/discovery/test_discovery_spotify_public.py
covering cache hit short-circuit, dict-artist normalization, Spotify
tuple match (track/disc preservation), iTunes dict match path, Wing It
fallback, cancellation, completion phase update, activity feed entry,
top-level error handler, per-track error handling.
Full suite: 1118 passing (was 1108). Ruff clean.
Fourth lift in the PR5 discovery-workers series. Pulls the 270-line
Deezer discovery worker out of `web_server.py` into its own focused
module under `core/discovery/`. Pure 1:1 lift — wrapper keeps the
original entry-point name so the existing call sites continue to work
without changes.
What the Deezer discovery worker does:
1. Pause enrichment workers (release shared resources).
2. For each Deezer track:
- Cancellation gate (state['cancelled']).
- Discovery cache lookup; cache hit short-circuits the search and
populates display fields from the cached match (artist string,
album name).
- SimpleNamespace duck-type → `_search_spotify_for_tidal_track`
(shared search helper, returns tuple for Spotify or dict for iTunes).
- On Spotify match: build `match_data` preserving track_number /
disc_number from raw API data, image extracted from album images
or track object fallback, release_date filled from track.release_date
when album dict is missing it.
- On iTunes match: dict result populated as `match_data`, source set
to discovery_source, image extracted from album images.
- Save matched result to discovery cache.
- On miss: Wing It stub stored as 'wing-it' status.
3. After all tracks: phase='discovered', activity feed entry, sync
discovery results back to mirrored playlist via
`_sync_discovery_results_to_mirrored`.
4. On error: state['phase']='error' + status with error string.
5. Finally: resume enrichment workers.
Dependencies injected via `DeezerDiscoveryDeps` (13 fields) —
deezer_discovery_states dict, spotify_client, plus 11 callable helpers
(pause/resume enrichment, get_active_discovery_source,
get_metadata_fallback_client, get_discovery_cache_key, get_database,
validate_discovery_cache_artist, search_spotify_for_tidal_track,
build_discovery_wing_it_stub, add_activity_item,
sync_discovery_results_to_mirrored).
Diff vs original after `deps.X` → global X normalization is **zero
differences** — 270 lines orig = 270 lines lifted, byte-identical body
(including all whitespace, comments, log strings).
Tests: 10 new under tests/discovery/test_discovery_deezer.py covering
cache hit short-circuit, Spotify tuple match (track/disc number
preservation), iTunes dict match path, Wing It fallback, cancellation,
completion phase update, activity feed entry, mirrored sync invocation,
top-level error handler, per-track error handling.
Full suite: 1108 passing (was 1098). Ruff clean.
Third lift in the PR5 discovery-workers series. Pulls the 323-line
mirrored-playlist discovery worker out of `web_server.py` into its own
focused module under `core/discovery/`. Pure 1:1 lift — wrapper keeps
the original entry-point name so the existing call site
(`_run_playlist_discovery_worker(pls, automation_id=None)` from the
automation engine) continues to work without changes.
What the playlist discovery worker does:
1. Pause enrichment workers (release shared resources).
2. Pre-compute total track count across all playlists for the automation
progress card.
3. For each playlist:
- Fast pre-scan separates already-discovered tracks (skipped, unless
incomplete metadata or a Wing It stub) from undiscovered ones.
- For each undiscovered track:
- Cancellation gate via _playlist_discovery_cancelled set.
- Discovery cache lookup (with artist validation).
- matching_engine search-query generation, then Spotify (preferred)
or iTunes (fallback) search + scoring.
- Extended search fallback (limit=50) if no high-confidence match.
- On match → enrich album from metadata cache (id, images,
total_tracks, album_type, release_date, artists, plus track_number
and disc_number), build matched_data, write to track.extra_data,
save to discovery cache.
- On miss → Wing It stub stored as 'wing_it_fallback' provider.
4. After all playlists: emit `discovery_completed` event when at least
one new track was discovered, mark automation progress 'finished'.
5. On error → automation progress 'error', traceback printed.
6. Finally: resume enrichment workers.
Dependencies injected via `PlaylistDiscoveryDeps` (16 fields) —
spotify_client, matching_engine, automation_engine, the cancellation
set, plus 12 callable helpers (pause/resume enrichment,
get_active_discovery_source, get_metadata_fallback_client/source,
update_automation_progress, get_database, get_discovery_cache_key,
validate_discovery_cache_artist, discovery_score_candidates,
get_metadata_cache, build_discovery_wing_it_stub).
Diff vs original after `deps.X` → global X normalization is **zero
differences** — 323 lines orig = 323 lines lifted, byte-identical body
(including all whitespace, comments, log strings).
Tests: 15 new under tests/discovery/test_discovery_playlist.py covering
empty playlists, no-tracks playlist skip, complete-discovery skip,
incomplete-discovery re-run, Wing It always re-run, unmatched_by_user
respect, cache hit short-circuit, match above threshold (extra_data +
cache save), match below threshold falls to Wing It, iTunes fallback,
neither-provider error path, cancellation, discovery_completed event
emit, no-event on zero-discovered, multi-playlist grand_total
aggregation.
Full suite: 1098 passing (was 1083). Ruff clean.
yt_dlp sometimes returns float `duration_ms` for YouTube tracks. The
discovery workers format the duration with `f"{x // 60000}:{(x % 60000)
// 1000:02d}"` — and `:02d` requires an int. When the duration is a
float, the format string raises:
Unknown format code 'd' for object of type 'float'
Caught when running YouTube discovery on a real playlist (bbno$ tracks)
— every track failed with status='Error'.
Pre-existing bug, surfaced now because of yt_dlp returning float
durations on this playlist. Fixed at all 8 sites by casting through
`int()` before the `// 60000` and `% 60000` operations:
- core/discovery/youtube.py: 2 sites in run_youtube_discovery_worker
(cache hit + main result construction).
- web_server.py L29238/L29372: 2 sites in _run_listenbrainz_discovery_worker.
- web_server.py L40112/L40136/L40161/L40178: 4 sites in the YouTube
retry/pre-discovered results assembly path.
The `if duration_ms` / `if dur` guard already protects against None and 0,
so `int(...)` is only called on truthy numeric values.
Tests: 1 new regression test under tests/discovery/test_discovery_youtube.py
(`test_float_duration_does_not_crash_format`) — passes a float
duration_ms and asserts the worker completes without an error result.
Ruff clean.
Second lift in the PR5 discovery-workers series. Pulls the 332-line
YouTube discovery worker out of `web_server.py` into its own focused
module under `core/discovery/`. Pure 1:1 lift — wrappers keep the
original entry-point name so the two callers
(`youtube_discovery_executor.submit(_run_youtube_discovery_worker, ...)`)
continue to work without changes.
What the YouTube discovery worker does:
1. Pause enrichment workers (release shared resources).
2. For each YouTube playlist track:
- Cancellation check (phase != 'discovering' aborts).
- Discovery cache lookup; cache hit short-circuits the search.
- Strategy 1: matching_engine search queries with confidence scoring
against Spotify (preferred) or iTunes (fallback).
- Strategy 2: swapped artist/title query.
- Strategy 3: raw (untokenized) query.
- Strategy 4: extended search with limit=50.
- On match → save to discovery cache.
- On miss → build Wing It stub from raw source data.
3. After loop: phase='discovered', sort results by index, and for mirrored
playlists write extra_data back to the DB.
4. Activity feed entry with match summary.
5. On error → state['status']='error', phase='fresh'.
6. Finally: resume enrichment workers.
Dependencies injected via `YoutubeDiscoveryDeps` (16 fields) —
youtube_playlist_states, spotify_client, matching_engine, plus 13
callable helpers (pause/resume enrichment, get_active_discovery_source,
get_metadata_fallback_client, discovery cache key/validate, extract
artist name, spotify_rate_limited, discovery_score_candidates,
get_metadata_cache, build_discovery_wing_it_stub, get_database,
add_activity_item).
Diff vs original after `deps.X` → global X normalization is **zero
differences** — 332 lines orig = 332 lines lifted, byte-identical body
(including all whitespace).
Pre-existing bug preserved (not fixed): if `youtube_playlist_states[url_hash]`
raises KeyError on entry, the outer except handler tries to mutate
`state` which is unbound → secondary UnboundLocalError. Same bug in
the original. Documented here for future cleanup but out of scope
for the lift.
Tests: 14 new under tests/discovery/test_discovery_youtube.py covering
cache hit short-circuit, Strategy 1 confidence match, Wing It fallback,
iTunes fallback path (Spotify unauthenticated and rate-limited),
cancellation (phase changed), skip_discovery flag, completion phase
update, activity feed entry, mirrored playlist DB writeback, non-mirrored
no-writeback, enrichment workers pause/resume, error-during-loop resume,
results sorted by index after retry.
Full suite: 1082 passing (was 1068). Ruff clean.
- remove the redundant wishlist-service injection from the runtime wrappers
- keep the package owning its own singleton service access
- simplify the route runtime API and update the wishlist tests to match
- ignore unconfigured backends when clearing completed downloads
- keep the post-download cleanup route best-effort after a successful wishlist run
- add regression coverage for the orchestrator clear step
