start_playlist_sync validated the resolved mode with 'if sync_mode not in
(replace, append): sync_mode = replace' — a pre-existing clamp two lines below
the config read I added, which silently downgraded a configured 'reconcile' to
'replace'. So config=reconcile resolved correctly then got clobbered, and every
sync ran replace regardless of the setting (and incognito didn't help — it's
backend, not cache).
Replace the hand-rolled clamp with a pure normalize_sync_mode(requested,
configured) helper (VALID_SYNC_MODES includes reconcile) so the resolution is
testable and can't silently drop a mode again. Regression tests cover
reconcile-from-config, request-overrides-config, and unknown->replace.
Replace mode (default) deletes + recreates the server playlist every sync,
which wipes its custom image, description, and identity. Add an opt-in
'reconcile' sync mode that edits the existing playlist in place — adds the
tracks now in the source, removes the ones gone — without destroying the
object, so the user's custom art/description survive.
- Pure planner plan_playlist_reconcile(current, desired) -> {add, remove}.
- Per-client reconcile_playlist: Plex addItems/removeItems on the same object;
Navidrome Subsonic updatePlaylist delta (songIdToAdd / descending
songIndexToRemove); Jellyfin add + remove-by-PlaylistItemId on /Playlists/{id}/Items.
- sync_service: reconcile branch with a replace FALLBACK (if a server's in-place
edit is unavailable/fails, sync still succeeds destructively — logged loudly).
- Default stays 'replace' (no behavior change). New Settings > Playlist sync mode
picker (replace/reconcile/append) backed by playlist_sync.mode; per-request
sync_mode still overrides.
- Reconcile skips the post-sync source-image push so a custom poster isn't
re-clobbered (the bug).
Tests: planner (add/remove/dedupe/order/empty) + reconcile-or-replace dispatch
(success / false-fallback / exception-fallback / no-method). Per-server in-place
API calls need dev validation against real Plex/Jellyfin/Navidrome.
NOTE: opt-in only; default behavior unchanged.
Consistency follow-up: the Filler picked art via metadata source-priority +
its own prefer_source knob, ignoring metadata_enhancement.album_art_order —
the explicit cover-art source list that post-process embed and the Library
Re-tag job now use. So 'cover art sources' meant two different things.
Prefer select_preferred_art_url (the configured order) first; fall back to the
existing prefer_source / source-priority loop when no order is configured
(the default) — non-breaking, existing behavior unchanged for those users.
Help text updated. Test: configured order wins + skips the source loop.
User feedback (Sokhi): after changing cover-art sources, re-tag should
re-download fresh covers from THEM. The job took cover_url only from the
matched metadata source's album image, ignoring the user's configured
cover-art order. Now prefer select_preferred_art_url (the same
metadata_enhancement.album_art_order the post-process embed honors), falling
back to the source image when no order is configured (non-breaking).
'replace' cover mode already force-refreshes art on every matched album, and
the embed replaces existing art (no duplicate pictures) — so 'replace' + a
configured art order = fresh covers from those sources. Help text updated.
Tests: prefers configured source URL / falls back to source image when unset.
Find & Add on the playlist-sync page only wrote sync_match_cache, which is
DELETEd wholesale after every DB scan — so the source->library pairing (and
the user's manual matches) reverted to 'extra'/red-dot on the next shallow
scan. The three match stores (sync_match_cache, manual_library_track_matches,
discovery extra_data) were disconnected and all pointed at tracks.id, which a
rescan re-keys (esp. Jellyfin/Navidrome GUIDs).
Unify the match so it's one durable fact, recorded once, honored everywhere:
- Find & Add also writes a durable manual_library_track_matches row (one-way;
the manual-match tool has no playlist to act on, so no reverse). Carries the
library file path.
- New library_file_path column (idempotent migration) + find_track_id_by_file_path:
re-resolve a stale library_track_id after a rescan re-keys the track, and
self-heal the row.
- The sync compare display's override lookup now falls back to the durable
manual match (resolve_durable_match_server_id) when sync_match_cache misses —
so the pairing persists across a scan instead of reverting to a red dot.
Purely additive: only adds matches when the cache returns nothing.
Tests: durable resolver (valid / stale-reresolve+self-heal / no-match / not-in-
playlist / missing-methods), file_path persistence + find_track_id_by_file_path.
A bare host like '192.168.1.5:8080' or 'qbittorrent.lan:8080' (no scheme)
is what users naturally type, but requests then raises 'No connection
adapters were found for ...' — it can't pick an http/https adapter, and a
bare host:port even gets misparsed as scheme=host. This surfaced as the
generic 'qbittorrent probe failed' with a 'login error: No connection
adapters were found' in the logs.
Add normalize_client_url() in torrent_clients/base: default a missing scheme
to http:// (+ trim trailing slash), and route all three adapters'
_load_config through it. Transmission normalizes the base before appending
/transmission/rpc.
Tests: normalizer unit cases + per-adapter regression (bare host -> http://).
Note: usenet adapters (sabnzbd/nzbget) share the same pattern and need the
same treatment in a follow-up.
Follow-up hardening to #789. The selection was keyed purely by folder name,
so renaming a music folder in Navidrome silently reverted the scan to all
libraries. Now persist the folder id (stable across renames) as the primary
key alongside the name (kept for display + back-compat), and restore by id
first with a name fallback. Self-heals on reconnect: pre-id installs and
drifted/renamed names get the id + fresh name written back, so the settings
dropdown keeps highlighting the right folder.
Tests: restore-by-id-after-rename (+ name heal), name-fallback self-heals id,
no-drift writes nothing.
The saved music-folder selection was silently dropped on every reconnect.
_setup_client's restore step called the public get_music_folders(), which
starts with ensure_connection() — but we're already inside ensure_connection()
at that point (_is_connecting=True, _connection_attempted not yet set), so the
re-entrant call bailed and returned []. The restore matched nothing,
music_folder_id stayed None, and the per-call musicFolderId filters all
no-op'd → scans imported every library regardless of the user's choice.
Surfaces after any restart or settings save (reload_config resets the state).
Split get_music_folders() into the public method (does the connection check)
and a non-reentrant _fetch_music_folders() seam; the restore now calls the
seam directly (connection is already established + ping succeeded by then).
Regression + seam tests added.
- depth setting (light = core tags + matched source ids; full = same
multi-source enrichment cascade a fresh download gets, run additively
via embed_source_ids). Threaded through scan/finding/auto-apply and the
repair_worker fix handler.
- source now defaults to 'auto' (= your source priority / active source)
instead of blank.
- give native <option> popups a solid dark background (were white-on-white).
- tests for full-depth full_meta payload + enrich invocation + light no-op.
The job was the odd one out — auto_fix=False, no dry_run setting, so it never
showed the 'Dry Run' badge the other jobs do (the badge keys off
settings.dry_run === true). Aligned it to the standard pattern:
- auto_fix=True + dry_run setting defaulting True. Default behavior is unchanged
(findings only, nothing written) AND it now shows the Dry Run badge.
- Turning dry_run off makes the scan auto-apply in place (result.auto_fixed),
no finding — the opt-in 'just retag it' mode.
- Extracted a shared apply_track_plans() used by both the scan auto-apply and
the repair_worker fix handler (handler now resolves Docker paths then
delegates — one code path, no duplication).
Tests: dry_run=False auto-applies + writes + no finding; existing dry-run
finding/skip/apply tests still green. 410 passing.
The old per-download Retag Tool was limited (only native-pipeline downloads,
100-group cap, manual per-group) and did the wrong thing — it moved/reorganized
files instead of just tagging. It's superseded by the new Library Re-tag job
(whole-library, in-place) + the enhanced-library 'Write Tags' button.
Removed: the post-download record_retag_download ingestion hook (stops writing
retag_groups on every download), core/library/retag.py, the web_server state +
deps + /api/retag/* endpoints + the tool:retag WebSocket emit, the dashboard
card + both modals (index.html), the core.js socket handler, and the tools-page
wiring + help entry (wishlist-tools.js). Updated the import-pipeline test.
Verified: web_server parses, app + core imports OK, 392 tests pass, no live
references to removed symbols.
Left as inert (harmless) for a careful follow-up sweep: the retag_groups/
retag_tracks tables + their DB CRUD methods (no longer written/read), and the
now-orphaned retag JS helper functions (no entry point/wiring/socket calls them;
interspersed with wishlist functions, so not blind-deleted).
write_tags_to_file wrote the core fields + cover but never the source IDs
(Spotify/iTunes/MusicBrainz) the import post-process embeds. Added a focused
source.embed_known_source_ids() that writes ALREADY-KNOWN ids (from db_data)
via the canonical, Picard-compatible frame writer the import uses
(_write_embedded_metadata) — no API re-fetch, frames correct by construction.
write_tags_to_file now calls it whenever db_data carries id keys.
Fed from both paths: the enhanced-library 'Write Tags' button now carries the
track's spotify/itunes/musicbrainz ids, and the Library Re-tag job stamps the
matched album/track source ids onto each track. So both now write the full tag
set, not a subset.
New 'Library Re-tag' repair job (default-OFF, opt-in; weekly when enabled):
- Scans every source-matched album (spotify/itunes/deezer/musicbrainz album id),
pulls fresh metadata + tracklist from that source, reads each local track's
current tags, and uses the planner to compute per-field diffs.
- Dry-run by design: scan only CREATES findings — nothing touches a file. Each
finding is highly detailed: per-track old->new for every changed field, the
source used, the mode, a cover-art action, and any unmatched tracks, plus a
summary description. Settings: mode (overwrite | fill_missing), cover_art
(replace | fill_missing | skip), source override.
- Apply handler (_fix_library_retag in repair_worker): writes each track's
planned tags in place via tag_writer.write_tags_to_file (+ batch-embeds cover,
refreshes cover.jpg). Only adds/overwrites planned fields — no moves/renames/
re-match. Resolves Docker paths; read-only/unreachable files counted, never
crash. Media-server-only / unreachable tracks are skipped.
Registered in the job list + fix dispatch. The old per-download Retag Tool is
left untouched alongside this for now.
The testable core for the new library-wide re-tag job. Given a source album's
metadata + tracklist and the library tracks' current file tags, it:
- matches source tracks to library tracks (disc+track number, then title sim),
- computes the per-field diff (old -> new) for the dry-run finding,
- builds the minimal write_tags_to_file payload — only fields that actually
change under the chosen mode (overwrite vs fill-missing), so applying never
touches unrelated/unchanged tags.
No IO/network/DB — 10 unit tests cover matching, both modes, blank-source
fields, and the album-artist/track-count payload mapping.
Verification found a non-additive edge: embed_album_art_metadata uses FLAC
add_picture(), which APPENDS — so applying to an album where some tracks already
had art would have added a duplicate embedded picture. The apply now checks each
file and skips any that already carry art (shared _audio_has_art helper), so it
only ever ADDS art to files missing it. Test covers the skip (no re-embed).
Previously the filler only flagged albums whose DB thumb_url was empty and, on
apply, only updated that DB thumb_url — so albums whose files had no embedded
art and no cover.jpg (but whose DB row had a URL) were never found, and even
'applying' art never touched the files. That's the reported 'doesn't scan all
albums' gap.
New core.metadata.art_apply (reuses the post-processing standard so the user's
album_art_order is honored):
- album_has_art_on_disk(): cheap-first check — folder cover.jpg/folder.jpg
sidecar, then embedded art in a representative track (FLAC/ID3/MP4/Vorbis).
- apply_art_to_album_files(): embeds via embed_album_art_metadata + writes
cover.jpg via download_cover_art; only ADDS art (never rewrites the user's
tags); read-only/unwritable files are skipped + counted, never crash.
Scan now examines every titled album and flags it when art is missing in the DB
OR on disk. Apply embeds into the album's audio files + writes cover.jpg in
addition to the DB thumbnail (media-server-only albums fall back to DB-only).
Tests cover sidecar/embedded detection, the cheap-first short-circuit, and the
apply orchestration (embeds each file + cover.jpg; read-only failures counted).
The title/artist fallback search took results[0]'s artwork unconditionally, so
a loose full-text match returned the wrong album's cover (the 'new sources give
incorrect art' reports). Now it pulls a few results and only accepts one whose
title matches (subset, to allow Deluxe/Remaster) AND whose artist matches
exactly — the artist being the strong guard against wrong covers. Falls back to
an exact title match when a result carries no artist.
The album's own stored source-id path is unchanged (that id is authoritative).
Tests: wrong-artist rejected, skips wrong result for a matching one, + unit
coverage of the matcher (deluxe/feat/stopwords accepted, wrong artist/title
rejected).
qBittorrent 5.2.0 changed /api/v2/auth/login to return HTTP 204 (No Content)
on success instead of HTTP 200 with body 'Ok.'. The adapter required the body
to equal 'Ok.', so every login on 5.2.0+ failed with 'HTTP 204 body=' — the
connection probe and all torrent actions were broken.
Treat login as successful on the SID auth cookie and/or a success body: 'Ok.'
(<=5.1) or an empty HTTP 204 (>=5.2.0). Still reject bad creds, which
qBittorrent reports as HTTP 200 + 'Fails.' (not a 4xx).
Tests: 204-empty -> success, SID-cookie+empty-body -> success, 'Fails.' (even
with a stale cookie) -> failure.
- Stale-cache check (playlistTrackCacheIsStale) compared raw track_count to the
filtered/cached track list, so any playlist with local or unavailable tracks
always looked 'stale' and refetched + re-mirrored on every modal open. Now it
compares the upstream snapshot_id (stored at cache time in the shared fetch
choke point), and returns not-stale when no snapshot is available — explicit
invalidation on refresh still handles real changes.
- organize_download: guard executor.submit so a refused job cleans up the batch
instead of stranding it in 'analysis' (holding a limited analysis slot).
- Removed the dead, deprecated, unused mirrorSpotifyPlaylistTracks.
resolve_mirrored_playlist tried the mirrored-playlists primary key FIRST for
any all-digit ref. Deezer upstream ids are all-numeric, so a Deezer playlist id
was mistaken for the PK and the organize-by-playlist toggle resolved a wrong row
(or nothing) — the toggle silently wouldn't save / 'Open in Mirrored' missed.
Resolve by (source, source_playlist_id) first, fall back to PK only when the
source lookup misses. Thread the batch/wishlist source through the download-path
callers so numeric upstream ids resolve correctly there too. Spotify (base62
ids) is unaffected.
Seam tests: numeric Deezer id resolves by source (not PK), spotify alphanumeric
by source, PK fallback still works, profile-scoped, empty refs -> None.
The worker's WARNING observability proved the '38 errors' were almost all
MusicMap returning 404 (artist has no map page) — a genuine not-found, not a
fetch failure. But iter_musicmap_similar_artist_events flattened every
RequestException to status_code 502, and the worker maps 400/404 -> not_found
/ everything-else -> error, so these inflated the error count.
Surface the real HTTP status from the exception's response (404 stays 404),
falling back to 502 only when there's no response (timeout/connection drop,
which is correctly still an error eligible for retry).
Regression tests: 404 -> 404 (not_found), timeout -> 502 (error), 500 stays
error, plus an end-to-end worker check that a 404 result marks 'not_found'
and stores nothing.
Verified against live data: 1312/1313 stored similars carry a metadata source id,
but 1 slipped through name-only (a match on a source with no id column, e.g.
discogs). Enforce the standard: process_artist now SKIPS any similar whose match
doesn't map to a storable id column (spotify/itunes/deezer/musicbrainz) instead
of writing a useless name-only row. Regression test covers discogs-match + no-id
cases. Now 100% of newly-stored similars are actionable.
The kettui move: 38/79 fetches errored on the first live run, but they were
logged at DEBUG only — invisible in app.log, so the cause (rate-limit vs
no-providers vs bug) is unprovable. process_artist now returns a (status, count,
detail) triple carrying the error reason (status code + message / exception),
and the worker logs the first 15 errors per session at WARNING (rest DEBUG) +
keeps _last_error. No blind pacing tweak — let it run, read the real reason, then
fix the proven cause. Seam tests updated + assert the reason is captured.
- get_stats now reports PERSISTENT counts from the DB (matched/not_found/pending
+ a progress.artists breakdown) instead of in-memory session counters, so the
dashboard orb tooltip and the Manage modal agree (was showing 0 vs 14 after a
restart) and it survives restarts — same approach as the other workers.
- Orb tooltip reads progress.artists ('Artists: 14 / 15 (93%)') like the rest.
- Worker now defaults to ON (running) instead of opt-in-paused; still honors a
saved pause across restarts. It self-paces (~3s/artist) and backs off on
MusicMap outages, so the orb spins/active like the others when there's work.
10 seam tests pass.
SoulSync standalone matches library tracks without Plex fetchItem,
reports missing counts correctly, and skips server playlist writes.
Automation re-syncs when the mirror grows; after sync finishes, starts
organize download (organize-by-playlist) or wishlist processing.
UI: Spotify URL playlist-folder controls, organize toggle layout in the
discovery modal, reload organize preference when reopening Download Missing.
Co-authored-by: Cursor <cursoragent@cursor.com>
Closes the gap where similar artists only existed for WATCHLIST artists: a new
background worker populates them for the whole LIBRARY, slotting into the
existing enrichment-worker pattern (bubble + Manage Enrichment Workers modal,
status/pause/resume, matched/not_found/pending/errors).
Per source-matched library artist → get_musicmap_similar_artists(name, 25)
(the same matcher the artist-detail page uses: fetches MusicMap names, matches
each to the user's source chain — primary + active fallbacks — returns only
matched artists) → store via add_or_update_similar_artist keyed by the artist's
metadata source id, the SAME key the watchlist scanner + artist map use, so the
two cooperate (idempotent upsert + retry_days window).
- core/similar_artists_worker.py: pure seams (pick_source_artist_id,
map_payload_to_store_kwargs, process_artist) + the threaded worker; skips
artists not yet source-matched; classifies not_found vs transient error
(retry after 30d).
- DB migration: similar_artists_match_status / _last_attempted on artists
(mirrors every other source worker's tracking columns).
- Registered in EnrichmentService + instantiated in web_server, DEFAULT-PAUSED
(opt-in) like Amazon — MusicMap is scraped/outage-prone + this is library-wide.
- SERVICE_ENTITY_SUPPORT['similar_artists']=('artist',) so the modal breakdown
('artists with / without similars') + Retry work; manual-match (inapplicable
to a relationship) is gated out via relationship:true.
- 10 seam tests; existing 80 enrichment tests still pass.
Note: keys under profile 1 (single-profile setups); multi-profile is future work.
Persist organize_by_playlist on mirrored playlists and run playlist-folder
downloads from the auto-sync pipeline instead of the global wishlist phase.
Register SoulSync library rows after playlist-folder post-processing, route
failed organize batches to the wishlist correctly, and skip sync-time
unmatched wishlist only when organize download handles retries.
Invalidate stale playlist track caches on refresh (Spotify and Deezer ARL),
re-mirror on refetch, and improve standalone playlist modals (re-analysis,
Open in Mirrored). Add filesystem missing-track detection and tests.
Co-authored-by: Cursor <cursoragent@cursor.com>
- Verified end-to-end: fetch_public_playlist_full pulled all 236 tracks of the
test playlist via SpotipyFree (the library handles the client-auth that 429'd
the raw approach). Name + tracks correct.
- requirements.txt: declare spotipyFree>=1.1.2 as a normal pip dependency (like
spotDL, also MIT — aggregation, not vendored) + websockets (a transitive dep
SpotipyFree/spotapi needs that pip doesn't pull automatically). Code still
soft-imports + falls back to embed, so it's never a hard runtime requirement.
- meta fetch uses limit=1 (name/owner only) so we don't pull the whole list
twice. 9 tests green.
The in-house anonymous-token path is blocked by Spotify (429 without the web
player's rotating client-auth). Switch the full-fetch to SpotipyFree — the
maintained no-creds spotipy drop-in spotDL uses, which tracks that machinery.
- core/spotify_public_api.fetch_public_playlist_full now uses a SpotipyFree
client (playlist + playlist_items + next), normalising the spotipy-shaped
items to the embed scraper's shape. Injectable client_factory keeps it
unit-testable without the library or network. Dropped the dead in-house
token/pagination code.
- Licensing: SpotipyFree is GPL-3.0, so it is NOT bundled/required (SoulSync is
MIT). Optional, user-installed: the import is soft, and on ImportError (or any
failure) fetch_spotify_public falls back to the embed scraper (~100). So the
shipped project stays cleanly MIT and the link path never regresses.
- requirements.txt: documents it as a commented optional extra
(pip install SpotipyFree) with the GPL/MIT rationale.
- 9 tests: normalisation, pagination past 100, library-missing -> raises (->
fallback), and the embed-fallback orchestration.
Needs a live click-through with SpotipyFree installed to confirm the exact
class/method names match (SpotipyFree.Spotify / playlist / playlist_items).
The full-fetch's logs used a bare module logger that app.log doesn't capture,
so we couldn't see whether the API path succeeded or why it fell back. Route
them to 'soulsync.spotify_public' and log: token found?, embed parsed?, the
API HTTP status on a non-200, and pagination result. Lets us see the exact
failure (e.g. 401 vs 429) on the next link-tab test.
Live debugging the 'shows 100' report:
- The full playlist page no longer embeds an accessToken, and get_access_token
/ server-time now 403/404. The EMBED page (open.spotify.com/embed/playlist/{id})
still ships a usable anonymous token. Was fetching the wrong page -> no token
-> raised -> embed fallback (100). Now reads the embed page for the token.
- Confirmed live: token extraction + embed parse work; the token is accepted by
the Web API (429 rate-limit, not 401). Could not show >100 from here because
the test IP got rate-limited from probing; needs a clean-IP click-through.
While in there, made it more robust against the rate-limiting that's clearly in
play:
- Refactored scrape_spotify_embed -> reusable parse_embed_html.
- fetch_public_playlist_full now does ONE embed fetch for token + name + first
page (no separate metadata call = fewer requests = less 429 surface), then
paginates the API. If the API is unavailable/rate-limited, it keeps the embed
page's tracks (<=100) instead of raising — so the result is always >= today's
behaviour, never worse.
- 12 tests incl. the new API-fails-but-embed-tracks-survive path.
Caveat unchanged: rides Spotify's undocumented embed-page token; degrades to the
embed fallback, never crashes.
The no-auth 'add by link' path scrapes Spotify's embed widget, which only ever
contains ~100 tracks and can't paginate — so big public playlists got
truncated. This adds an in-house anonymous fetch that pulls the FULL list:
- core/spotify_public_api.py: reads the anonymous web-player accessToken Spotify
already embeds in its own open.spotify.com page HTML (no app credentials, and
no rotating TOTP secret for us to maintain), then paginates
/v1/playlists/{id}/tracks 100 at a time until the whole playlist is pulled.
Returns the embed scraper's exact shape. Pure helpers + injected http_get so
it's unit-testable without the network.
- core/spotify_public_scraper.fetch_spotify_public(): tries the full fetch for
playlists; on ANY failure (or for albums) falls back to scrape_spotify_embed.
Worst case == today's behaviour, so the link path can't regress.
- web_server: the link-tab endpoint and the authed flow's last-resort scrape
now both go through fetch_spotify_public.
Scoped entirely to the spotify_public_* (no-auth) path — the authenticated
playlist sync is untouched. 11 tests (token extraction, normalisation,
pagination past 100, and the embed-fallback orchestration).
Caveat: rides Spotify's undocumented page-embedded token — expected to break
when they change their page; it degrades to the embed fallback, never crashes.
Needs a live click-through to confirm the token path works end to end (can't
hit Spotify from the test env).
- #1 Unconfigured-source banner: when a source has enabled=false, show a
notice that browsing works but matches/retries won't run until it's set up.
- #2 Rate-limit detail: when rate_limited, surface 'resumes in ~Xm' (from the
status payload) instead of just a pill.
- #3 Richer rows: unmatched items now show parent context — an album's artist,
a track's album — via a parent expression in the query (+ test).
- #4 Bulk select: per-row checkboxes + a bulk bar to retry several at once
(capped concurrency), reusing the /retry item endpoint.
- #5 Remember last worker: selection persists in localStorage and is restored
on open; openEnrichmentManager(workerId) supports future deep-linking
(bubbles left on their pause-on-click behaviour).
- #6 Keyboard nav: ArrowUp/Down moves focus between rows; actions are native
buttons (Enter/Space) and Escape closes — list isn't poll-refreshed so focus
is stable.
53 enrichment tests green; JS syntax clean.
Per-worker processing-order override + UI polish.
Feature — pin an entity group to enrich first:
- Each worker normally runs artist -> album -> track. A user can pin one
group (artist/album/track) to run first from the modal; the worker keeps
that group first until it's exhausted, then resumes the normal chain.
- core/worker_utils.py: read_enrichment_priority() (reads
<service>_enrichment_priority each loop, live) + priority_pending_item()
(shared, whitelisted query returning the worker's expected item shape;
Spotify/iTunes get album_individual/track_individual via a type map).
- A guarded ~6-line hook at the top of all 11 workers' _get_next_item.
CRITICAL: when nothing is pinned (default) the hook returns immediately,
so default enrichment order is byte-identical to before. Discogs (no track)
and Genius (no album) only honor their supported entities.
- core/enrichment/api.py: GET/POST /api/enrichment/<id>/priority (+ config_get
hook); POST validates the entity against what the source enriches.
- 14 new tests (helper shapes, exhaustion, route get/set/clear/validate).
UI:
- Refined hero header: identity + inline status left, single Pause right,
'now enriching' quiet sub-line; overall coverage % moved into the stats
section ('82% matched · 1,203 of 1,460'). Hero gently pulses while running.
- New processing-order strip: artist→album→track steps showing the live phase
(pulsing 'now'), pinned group ('first' + 📌), and done/remaining; click a
step to pin it, click again for auto.
py_compile clean across all 11 workers; 52 enrichment tests green.
Fixes a correctness bug and adds bulk re-queuing.
- Bug: per-row 'Retry' used clear-match, which sets an item to not_found
with last_attempted=NULL. The worker only retries not_found items where
last_attempted < (now - 30d), and 'NULL < cutoff' is false in SQLite, so
those items were never re-queued. Fixed by resetting match_status to NULL
(pending), which every worker's queue picks up on the next pass.
- New POST /api/enrichment/<id>/retry with scope 'item' | 'failed'
(failed = re-queue every not_found item of an entity type), backed by a
pure whitelisted build_reset_query + MusicDatabase.reset_enrichment().
- UI: per-row Retry now hits /retry; a 'Retry all failed' bulk button appears
when the current entity has not-found items (confirm + count toast); a hint
line explains retry/match/auto-retry behaviour.
- 11 new tests (38 enrichment tests total, all green).
Dashboard 'enrichment bubbles' could pause/hover but offered no way to
*manage* a worker. This adds a full management modal opened from a new
header button, covering all 11 enrichment sources.
Backend (testable core helper + seam tests; no live-DB dependency):
- core/enrichment/unmatched.py: pure, whitelisted SQL builders for the
unmatched browser. service/entity validated against a support map (never
interpolated raw); search + pagination bound as params; tracks join albums
for artwork; limit capped at 200.
- database/music_database.py: get_enrichment_unmatched() +
get_enrichment_breakdown() (the breakdown splits matched/not_found/pending,
which the existing get_stats().progress lumps together).
- core/enrichment/api.py: GET /api/enrichment/<id>/{unmatched,breakdown} on
the existing blueprint + a db_getter hook.
- web_server.py: wire db_getter=get_database.
- tests/enrichment/test_unmatched.py: 19 tests across builders, DB methods,
and Flask routes.
Frontend (vanilla, matches app conventions):
- webui/static/enrichment-manager.js: worker rail with live status + coverage
micro-bars, accent-themed detail panel (hero header, segmented matched/
not_found/pending stat cards, current item, pause/resume), and a searchable
paginated unmatched browser with inline manual match (reusing
search-service + manual-match) and retry (clear-match re-queues).
- Polish: entrance/exit motion, scroll-lock, Escape, refresh control,
flicker-free polling (in-place updates), skeleton loaders, relative
timestamps, per-worker accent theming, real dashboard logos reused at
runtime (with the same invert/circle treatment), responsive rail.
- index.html: header button + script include. style.css: full styling.
Reuses existing pause/resume, status, and manual search+assign endpoints.
Backend tests green (19 new + 11 existing enrichment tests).
- B023: default_fetch_tracklist built a per-item lambda closing over the loop
variable `it`. Replaced with a module-level _item_get(item, key, default)
helper (takes the item as a param — no closure). Behavior unchanged; the
dict/object normalization test still passes.
- S110: the two best-effort guards in the canonical job (skip-already-pinned
read, estimate_scope active-server read) now carry `# noqa: S110 — <reason>`,
matching the repo's existing convention for intentional swallow-and-continue.
ruff check passes on all canonical files + tests; 30 affected tests green.
The canonical source_selection setting was rendering as a free-text box — easy
to typo an invalid mode. Added a generic choice mechanism so it's a dropdown:
- RepairJob.setting_options: {key: [allowed values]} (default {} — opt-in).
- CanonicalVersionResolveJob declares source_selection's three modes.
- repair_worker.get_all_job_info() includes setting_options in the job payload.
- enrichment.js renders a <select> (options prettified, current value selected)
for any key listed in setting_options; everything else renders by value type
as before. The save path already reads <select>.value as a string, so no
change needed there.
Generic — any future job can get dropdowns the same way. Jobs that don't
declare setting_options are untouched (empty dict -> existing input rendering).
Tests: source_selection exposes the 3 options and its default is one of them.
23 repair-job/worker + canonical tests pass (other jobs unaffected).
Per request, pack each finding with everything available WITHOUT extra API
calls (kettui: reuse what's already fetched, read the album row we already
loaded, degrade per-field, keep it tested):
- Pinned release's track titles — already fetched during scoring, so free
(capped at 60 to bound details_json).
- From the album row (free): year, DB track count, total duration, genres-free
context, and the album's currently-linked source IDs.
- file_track_titles (your library's titles) for a side-by-side with the release.
- Artist + album thumbs (artist via the guarded lookup) and names.
_describe_pin now renders: "Artist — Album (year)", the fit breakdown, "Currently
linked: … → pinning X", "Beat: <alternatives>", and the release tracklist — so
the card is judge-able at a glance, and the structured fields are in details for
a richer UI.
NOT included (would cost an extra per-album API fetch, left as opt-in): the
*release's* own year/type/cover/URL from get_album_for_source, vs the library's.
Tests: _describe_pin rich-render (year/linked/tracklist), resolver release-titles,
orchestration free-context fields. 94 canonical + reorganize regression pass.
Findings now carry artist_thumb_url alongside album_thumb_url (same key the
track-repair findings use, so the findings UI already renders it).
Fetched via a guarded _lookup_artist_thumb() — checks the artists table has a
thumb_url column first and swallows any error — rather than adding ar.thumb_url
to the shared load_album_and_tracks SELECT. The shared-loader approach was
tried first and REVERTED: it crashed reorganize on schemas whose artists table
has no thumb_url column (caught by 40 orchestrator tests). The lookup only runs
for albums that actually resolve, so it adds no cost to the no-source-id
short-circuit majority.
Tests: orchestration test asserts artist_name + album_thumb_url + artist_thumb_url
flow through. 47 canonical + 104 canonical/reorganize regression tests pass.
Live-run feedback: "Best-fit release: deezer (665666731), score 1.0" is too thin
to trust/accept. Each finding now explains WHY:
- score_release_detail() exposes the per-signal breakdown (count/duration/title)
instead of just the blended score.
- resolve_canonical_for_album returns an enriched result: the breakdown,
file_track_count vs release_track_count, and a `candidates` list of every
source it scored (so a finding can show what the winner beat).
- resolve_and_store adds album/artist/thumb context from the row it already
loaded (no extra query). Storage still only reads source/album_id/score.
- The job builds a real description via _describe_pin(), e.g.:
"Pin deezer release 665666731 (confidence 100%).
Fit to your library: 11 files vs 11 tracks on this release — track count
100%, durations 100%, titles 100%.
Beat: spotify 65% (17 tk)."
and a clearer title ("Pin deezer as canonical: <artist> — <album>").
Tests: resolver enrichment (breakdown + candidate comparison fields), and
_describe_pin (judge-able text incl. the beaten alternatives, and honest "n/a"
for a missing signal). 42 canonical tests pass.
Note: the description string carries the judge-able info regardless of UI; how
the findings tab renders the extra details keys (thumb image, candidates table)
is still UI-dependent and unverified.
Feedback from the live dry-run: the job was pinning whichever source best fit
the files regardless of which source it was, which was surprising — users
expect it to respect their active metadata source. Made it a per-job setting
instead of a baked-in policy.
source_selection (default 'active_preferred'):
- active_preferred — use the active/primary metadata source's release when the
album has an ID for it AND it clears the score floor; otherwise fall back to
the best-fit among the other sources. Respects the configured source but
self-heals when that link is clearly broken (below floor / no ID).
- active_only — only ever the active source; never considers others.
- best_fit — previous behavior: whichever source matches the files best.
resolve_canonical_for_album gains mode + primary_source; the orchestration
threads the primary source through; the job reads source_selection from its
settings. Note: active_preferred respects the active source as long as it clears
the floor, so it will NOT override a deluxe-vs-standard mismatch on the primary
(#767-Bug2) — that's what best_fit is for; the choice is now the user's.
Tests: per-mode coverage in test_canonical_resolver.py (active_preferred uses
primary when it fits, falls back when primary is below floor, keeps primary even
when another fits better; active_only pins primary / never falls back; best_fit
unchanged), orchestration default-mode test, and the setting default. 39
canonical tests pass.
The populate trigger that turns the (until now dormant) feature on. Until a user
enables and runs this job, no album has a canonical -> both read sides (Stages
3-4) fall back -> zero behavior change. So the whole feature ships safely off.
- core/repair_jobs/canonical_version_resolve.py — "Resolve Canonical Album
Versions". Iterates the active server's albums, skips ones already pinned, and
calls the tested resolve_and_store_canonical_for_album per album. Opt-in
(default_enabled=False) and dry-run-by-default: resolving compares an album's
candidate releases across sources (metadata-source API calls, once per album),
so it's deliberately user-triggered. Dry run reports a finding per album it
would pin; live mode stores. Registered in _JOB_MODULES.
- core/metadata/canonical_resolver.py — resolve_and_store gains store=True; the
job's dry run passes store=False to resolve-without-writing.
Tests: tests/test_canonical_version_job.py (6) — registered, opt-in + dry-run
defaults, live resolves+stores (auto_fixed), dry run creates findings without
persisting, already-pinned albums skipped. Registry loads all 19 jobs cleanly.
145 tests across the full feature + reorganize/track-repair/DB regression pass.
_resolve_album_tracklist gains a Fallback -1: if the album has a pinned
canonical (source, album_id), use it before the existing 6-level cascade — so
Track Number Repair resolves the SAME release the Reorganizer does (Stage 3) and
the two stop contradicting each other (#765, the Spotify-4 vs MusicBrainz-3
conflict).
Gated + additive: the entire existing cascade is untouched for albums without a
canonical, so this job's all-01-album rescue (which relies on the MusicBrainz/
AudioDB fallbacks for albums with no DB source ID) is fully preserved — that's
the regression we explicitly refused to take in a reactive fix.
New helper _lookup_canonical_from_db() mirrors _lookup_album_ids_from_db
(file-path -> track -> album), returns None when no DB / no match / columns
absent / unresolved.
Tests: tests/test_track_repair_canonical.py (4) — returns canonical when pinned,
None when unresolved / file untracked / no DB. Existing track_number_repair
tests still pass (no regression).
_resolve_source now prefers the album's pinned canonical (source, album_id) when
set, before the source-priority walk. So once an album's canonical is resolved,
reorganize agrees with Track Number Repair (Stage 4) and stops mislabelling a
standard album as deluxe (#767-Bug2).
Gated + side-effect-free: only changes behavior for albums that already carry a
canonical (none do until the populate step runs), an explicit user source pick
(strict_source) still wins over the canonical, and a failed canonical fetch
falls through to today's priority walk. So this stage is behavior-neutral until
canonical is populated.
Tests: tests/test_reorganize_canonical_source.py (4) — canonical preferred over
priority, fetch-failure falls back, strict_source ignores canonical, no-canonical
unchanged. 113 reorganize-orchestrator/tag-source/unknown-artist tests still pass
(no regression).
Completes Stage 2's populate path. Still dormant — no consumer calls it yet.
- resolve_and_store_canonical_for_album(db, album_id, ...): loads the album's
source IDs + its tracks' (duration_ms, title) from the DB via the SAME
loader the Reorganizer uses (load_album_and_tracks + _extract_source_ids), so
the canonical is chosen over exactly the source IDs the reorganizer sees;
scores off the DB track rows (the library's view of the files — no per-file
disk reads), resolves the best fit, and persists it. Returns the stored result
or None when unresolved.
- default_fetch_tracklist(): production fetcher wrapping
get_album_tracks_for_source, normalising to {title, track_number, duration_ms}
(duration best-effort; sec->ms; absent -> scorer leans on count+title).
Design note: chose LAZY resolution (Stages 3-4 consumers call this when they hit
an album with no canonical) over a standalone backfill repair job — no new
scheduling/UI surface, resolves only when a tool actually needs it, and stays
gated (NULL canonical = today's behavior).
Tests: tests/test_canonical_orchestration.py (5) — end-to-end on a real temp DB
(11 files pick the 11-track release over a 17-track deluxe and persist it),
no-source-ids -> None, missing-album -> None, and default_fetch_tracklist
normalization (dict items, seconds->ms) + failure -> None. All canonical +
DB-migration tests green.
Turns the Stage-1 scorer into an end-to-end resolver + persists the result.
Still DORMANT — no consumer reads it yet, so zero behavior change.
- core/metadata/canonical_resolver.py — resolve_canonical_for_album(): builds
candidate releases from the album's per-source IDs (in source-priority order),
fetches each tracklist via an INJECTED fetch_tracklist (so it's unit-testable
without live APIs), scores them with pick_canonical_release, and returns the
best-fit {source, album_id, score}. Skips sources with no id / failed fetch;
returns None when there are no files, no candidates, or nothing clears the
confidence floor.
- database/music_database.py — set_album_canonical() / get_album_canonical()
write/read the Stage-1 columns. get returns None when unresolved, which every
consumer will treat as "fall back to today's behavior".
Tests: tests/test_canonical_resolver.py (7) — best-fit beats priority, priority
breaks true ties, skips missing-id/failed-fetch sources, None on
no-candidates/no-files/below-floor, score rounding. tests/test_canonical_db.py
(4) — set/get round-trip incl. timestamp, unresolved -> None, overwrite,
missing-album -> False. 34 canonical + DB-migration tests pass.
Remaining for Stage 2 (the trigger): read on-disk file durations/titles for an
album, gather its source IDs, call the resolver, store — wired via a backfill
repair job + an enrichment hook. Then Stages 3-4 wire the Reorganizer and Track
Number Repair to READ the pinned canonical.
First stage of the canonical-album-version fix (#765 + #767-Bug2). Pins ONE
canonical (source, album_id) per album, chosen by best-fit to the user's actual
files, so the Reorganizer, Track Number Repair, and tagging stop re-resolving
independently and contradicting each other.
Ships DORMANT — nothing reads or writes the new data yet, so zero behavior
change. Later stages populate (Stage 2) and consume (Stages 3-4) it.
- core/metadata/canonical_version.py — pure scorer (the testable heart):
score_release_against_files() rates a candidate release by track-count fit +
duration alignment (greedy nearest within ±3s) + title overlap, dropping and
renormalizing missing signals so it never crashes on sparse metadata.
pick_canonical_release() takes candidates in source-priority order, picks the
best fit, breaks ties toward the earlier (higher-priority) candidate so the
choice is DETERMINISTIC — that determinism is what makes every tool agree
(#765), while count/duration fit picks the right EDITION (#767-Bug2). A
confidence floor (default 0.5) means a low-confidence guess is never pinned.
- database/music_database.py — additive, nullable columns on albums
(canonical_source / canonical_album_id / canonical_score /
canonical_resolved_at), guarded by the existing PRAGMA-table_info pattern.
NULL = unresolved = every consumer falls back to today's behavior.
Tests: tests/test_canonical_version.py (11) — edition discrimination (11 files
-> standard, 17 -> deluxe), deterministic priority tiebreak, duration
disambiguation on count ties, graceful degradation (no durations / counts only /
fuzzy titles), confidence floor, empty-input safety. tests/test_canonical_
columns_migration.py (4) — fresh DB has the columns, they're nullable w/ NULL
default, migration is idempotent, and it ALTERs them onto an old albums table.
60 DB/schema regression tests still pass.
