Pache711: a cover-art finding showed the (correct) found album art next to a
(wrong) artist image with one "Apply Art" button — no way to take one and
skip the other. Turned out "Apply Art" only ever applied ALBUM art anyway;
the artist image was display-only context, so the bundling was an illusion
the UI created.
Now the finding is genuinely multi-target:
- scan (missing_cover_art.py): also searches for an artist image (always, so
a WRONG existing one can be replaced — Boulder's call), name-matched
exactly. Stored as found_artist_url only when it differs from the current
artist thumb, so nothing is offered when there's nothing to change.
- apply (_fix_missing_cover_art): honors a target via _fix_action —
'album' (default, unchanged "Apply Art" behavior: DB thumb + embed +
cover.jpg), 'artist' (the artist's DB image), or 'both'. New _fix_artist_art
sets artists.thumb_url for the album's artist.
- UI: each found image gets its own apply button — "Use for album" /
"Use for artist". Applying either resolves the finding, so taking the
correct one and ignoring the wrong one IS "fix one, dismiss the other".
Current artist art shows as "(current)" context with no button.
Default stays album-only, so the plain Apply Art button and every existing
caller behave exactly as before. Tests: 5 on the apply targets (artist-only /
album-only / default / both / missing-url) against a real SQLite DB, plus the
existing cover-art suite updated for the new artist search. 107 repair/
cover-art/UI-integrity tests pass.
Ashh: the manual-match modal fuzzy-searches a service and shows the top 8.
When the right release isn't in those 8 (common title — their example was
"Idols", which returns 8 unrelated releases and not Yungblud's), there was no
way through. But the user usually already knows the exact MBID.
Now the modal's search box doubles as a direct-ID box. Paste a MusicBrainz
MBID (bare UUID or a musicbrainz.org URL) and SoulSync looks that exact
entity up and shows it as the single result to confirm + Match — no fighting
the search ranking.
- core/library/direct_id.py: pure detector, returns the canonical ID only
when the text unambiguously IS one (whole-query UUID, or a UUID inside a
musicbrainz.org URL). "Idols", "Yungblud Idols", a UUID buried in free
text → None, so normal search is never hijacked.
- _search_service: direct-ID fast path before the fuzzy search —
get_release (→ get_release_group fallback for albums) / get_artist /
get_recording. A pasted-but-unresolvable ID falls THROUGH to fuzzy search,
so a typo can't dead-end the modal.
- UI: MusicBrainz placeholder now says "…or paste a MusicBrainz ID/URL".
Detector is service-keyed so Spotify/iTunes/etc. direct IDs can be added
later; today only MusicBrainz has a confirmable direct lookup, matching the
reporter's ask + screenshot. 9 tests: detector truth table (bare/URL/plain/
buried/other-service) + dispatch (confirmed release, release-group fallback,
unresolvable→fuzzy, plain query skips direct lookup).
noldevin's first torrent was stuck "downloading metadata" — a dead magnet
with no peers. The poll loop would ride the full album deadline (6h default)
on it, holding the worker the whole time, with no built-in escape.
New stall handling, off the existing poll loop:
- core/download_plugins/torrent_stall.py — pure StallTracker (clock injected,
no I/O): forward byte progress resets a stall clock; once a torrent spends
the stall timeout in a working state (queued/downloading/stalled/error)
with zero progress, it's stalled. seeding/completed/paused never count.
Covers the metadata-stuck case (0 bytes, 0 progress) and a dead mid-download
swarm with one rule.
- _handle_stalled: 'abandon' (default) removes the torrent + its partial data
(a metadata stub is junk) and fails the download so the next source can try;
'pause' parks it in the client for the user. Adapter errors are swallowed —
the download still fails cleanly.
- two settings (download_source.torrent_stall_timeout_seconds = 600,
torrent_stall_action = 'abandon'); timeout 0 disables, restoring the old
ride-the-deadline behavior. Config-key driven, matching the existing
album_bundle_* tuning knobs (no UI form, same as those).
Tests: 18 on the tracker + settings (timeout trip, progress reset, idle-state
exemption, pause→resume clock restart, disable, parse tolerance) + 3 on the
plugin action path (abandon removes w/ delete_files, pause pauses, adapter
error survived). 158 torrent-family tests pass.
wolf39us: "It keeps unauthenticating... daily" — re-auth fixes it until the
next day. Mechanism: spotipy's token cache was a loose FILE at
config/.spotify_cache. /app/config is a declared VOLUME, but a compose file
that doesn't map it explicitly gets an ANONYMOUS volume — recreated empty on
every container pull. So a nightly Watchtower update kept all his settings
(config lives in the database now) while silently dropping the OAuth tokens.
His redirect-URI change won't help: callback URLs only matter during the
initial handshake, never for refresh.
New DatabaseTokenCache (spotipy CacheHandler) stores the token payload in
the same database-backed config store as every other setting — tokens now
survive exactly as long as the rest of the configuration does. The legacy
file is imported once on upgrade (no forced re-auth) and removed on logout;
a failed cache write logs and never raises (spotipy calls it mid-request).
Tests: roundtrip, JSON-string tolerance, one-time legacy import (store wins
after the file vanishes), garbage file ignored, logout clears both stores,
write failure never raises. 204 spotify tests pass.
Netti93's follow-up report (single artist at download time, correct only
after retag) reproduces as FIXED on current dev — verified live against
Deezer's API with his literal track ('VERLIEBT IN MICH', FAYAN feat.
Dalton) and his exact config, through the real tag writer onto a real MP3:
TPE1=FAYAN, TIT2 gains '(feat. Dalton)', TXXX:Artists=[FAYAN, Dalton].
His last test (2.5.6 / May-19 dev) predates the fixes that closed it
(d5de724f contributors upgrade hardening, 0769fcd5 collab-tag loss).
These tests pin the full direct-download shape so it can't quietly
regress: Deezer /search payload (one artist) + provider on the candidate
(not the context) -> contributors upgrade fires -> feat_in_title and
artist_separator both honored. Network-free (client mocked with the live
API's verified response shape).
37725457 fixed _match_to_itunes to use the real iTunes client and flagged
the cross-source corruption as a possibility. Boulder's live DB proves it
happened: 6 of his 9 watchlist "iTunes" ids EQUAL the artist's Deezer id
(Taylor Swift's "iTunes" id was her Deezer id 12246; the real one is
159260351) — written back when the misnamed MetadataService.itunes slot
held a DeezerClient. The June-4 batch (Green Day, SOAD, Vulfpeck, ...) got
NULL instead because the slot now holds the Spotify primary.
The fix alone can't heal those rows: the backfill only fills EMPTY ids, so
a wrong non-empty id is permanent. New migration clears itunes_artist_id
where it equals deezer_artist_id (the corruption signature — distinct id
spaces, so a legitimate equal pair is effectively impossible, and the worst
case is a NULL that re-matches correctly on the next scan). Idempotent by
construction; similar_artists checked clean (its backfill always used the
registry correctly).
Tests: corrupted row cleared / legit + no-deezer rows kept / idempotent —
via a real re-init with the per-process init memo cleared (an app restart).
Boulder noticed his recently added watchlist artists (June 4 batch) have no
iTunes match while Spotify/Deezer/MusicBrainz matched fine. The rotated log
has the receipt: "Cannot match to iTunes - MetadataService not available" ×8
→ "Backfilled 0/8 artists with itunes IDs", every scan.
_match_to_itunes was the only matcher with no fallback: it read the PRIVATE
_metadata_service attr, which is None whenever the scanner is constructed
from a spotify_client — the normal web_server wiring — and gave up, while a
lazy-loading metadata_service property sat right next to it and the deezer/
discogs/musicbrainz matchers all fall back to their registry clients. Bonus
landmine: even when set, metadata_service.itunes is the FALLBACK-client slot
and may actually be a DeezerClient (per _match_to_deezer's own comment), so
"iTunes" matching could have stored a Deezer artist ID as itunes_artist_id.
Now mirrors the other matchers: canonical registry get_itunes_client().
Self-healing — missing IDs are re-attempted every scan, so existing
watchlists backfill on the next run with no migration needed.
Tests: match works with _metadata_service=None (the exact production
condition), unconfident result returns None, missing client degrades
gracefully. 103 watchlist tests pass.
CI failed all 7 requeue tests that passed locally. Root cause is a real
shipping bug, not test flake: config/settings.py's default template set
retry_next_candidate_on_mismatch: False ("Default off — opt-in") while the
monitor reads it with inline default True and the PR documents it as ON.
Outcome split the userbase: a FRESH install (or CI's clean runner) gets the
template key = retry engine silently OFF; an existing config.json lacks the
key = inline True wins = engine ON. Same code, opposite behavior, decided by
install age.
- template aligned to True (the documented + approved default; existing
installs already behave this way via the inline default)
- the requeue tests now pin the toggle ON via the wiring helper instead of
reading the runner's ambient config — CI's fresh defaults vs a dev's
lived-in config.json must never decide whether they pass. _patch_config
composes (it wraps the pinned get and falls through).
64 retry-engine tests pass; fresh-default simulation confirms the toggle
resolves True.
Boulder's lock-in question caught it: the read-only pre-flight called
os.statvfs unconditionally, which doesn't exist on Windows, and the
AttributeError wasn't covered by the except OSError — the whole cover-art
apply would have crashed for every Windows install (docker images are
Linux, so the reporter was fine; the maintainer wasn't). getattr-guarded
now: no statvfs -> skip the pre-flight, per-file EROFS detection (errno is
cross-platform) still active. Test pins the no-statvfs path.
Tim (Discord): cover-art automation fails with '[Errno 30] Read-only file
system' on every file; he chmod 777'd and nothing changed — because EROFS is
the KERNEL refusing writes to a docker ':ro' volume mount, which no chmod
can fix. SoulSync's response was a wall of per-file warnings and a fix
result that still said success with a soft "(read-only?)" hint.
- apply_art_to_album_files now pre-flights the album folder with statvfs
(asks the kernel, writes nothing): a read-only mount short-circuits the
whole album instead of failing file by file. Belt: a per-file/cover EROFS
(overlay quirks where statvfs lies) still sets the flag.
- the repair worker's apply now FAILS the finding with the actual cure:
"remove ':ro' from the volume mapping and recreate the container — chmod
cannot change this". EACCES (a real permissions problem chmod CAN fix)
deliberately keeps the old soft path.
Tests: RO mount short-circuits before any file/cover write, save-time EROFS
still flagged, EACCES not conflated with EROFS. 29 art/repair tests pass.
Sokhi (continued from #806): volume-numbered series ('B小町 …キャラクター
ソングCD Vol.2' / 'Vol.2.5' / 'Vol.4' / 'Vol.4.5') got each other's art from
both normal downloads and the retag tool. Two distinct holes, one principle:
1. The art picker's _album_matches validates by significant-token SUBSET —
built to tolerate '(Deluxe)'/'- Remastered' suffixes. CJK strips out of
the normalizer entirely, so Vol.4 → {b,tv,cd,vol,4}, a clean subset of
Vol.4.5's {b,tv,cd,vol,4,5}: the wrong volume validated as "the same
album with a suffix". Affected every fuzzy art source (iTunes, Deezer,
AudioDB, Spotify) in downloads, retag, and the missing-art repair.
2. MusicBrainz match_release scores by string similarity — Vol.4 vs Vol.4.5
is 0.973, so the wrong volume could win the match outright, and its MBID
then feeds Cover Art Archive with NO downstream validation (CAA is
MBID-keyed, trusted by design). With Sokhi's MB metadata source this is
the likely path in his logs (his release-group 404s push re-matching).
The shared rule (core.text.title_match.numeric_tokens_differ): digit-bearing
tokens must be IDENTICAL between the two titles. A number on one side only —
volume, part, sequel, remaster year — is a different release, never a
suffix. '1989' vs '1989 (Deluxe)' still matches (digits shared); 'Album' vs
'Album 2' now rejects (sequels!). Art picker rejects outright (falls through
to next source / the download's own art — the designed cost of a false
reject); MB matcher halves the candidate's confidence, landing it below the
70 gate while the exact-volume result is untouched.
Tests: helper truth table, the exact reported pairs through _album_matches,
and match_release end-to-end (wrong volume alone → no match beats a wrong
MBID; exact volume beats near-identical wrong one despite lower MB score).
828 matching/metadata + 301 musicbrainz/retag/artwork tests pass.
Boulder: "Taylor Swift shows only 8 albums, nothing before 2022, no singles,
no EPs" — for every artist (actually: every WATCHLIST artist). Traced live:
get_artist_albums caches its result under an UNQUALIFIED key (no limit/page
info), and the watchlist's new-release probe (limit=5, max_pages=1 — the
April "reduce watchlist API calls ~90%" optimization) stored its truncated
single page in that same slot. The artist detail page reads the cache first,
so a watchlisted artist's page showed only the newest handful of releases —
newest-first, hence "nothing before 2022" — re-poisoned on every scan, with a
30-day TTL. When the source-priority fetch comes back tiny, the page's
fallback path quietly serves it, so the symptom looked like a discography
filter bug. Not related to the #808 matching change (that is a pure max(),
provably additive).
Three pieces:
- get_artist_albums tracks whether the fetch stopped while more pages
existed (truncated) and only caches COMPLETE discographies. Individual
albums keep their opportunistic caching — they're complete entities
regardless of pagination. A small real discography that fits one page
stays cacheable even under max_pages=1.
- MetadataCache.purge_artist_album_lists(): delete the already-poisoned
album-list entries (TTL would have kept them for weeks); lists rebuild
lazily on the next artist-page visit.
- one-time startup purge in web_server, config-guarded
(maintenance.album_cache_purge_v1), mirroring the startup-repair pattern.
Tests: truncated probe never stores the list (but still returns its page),
complete multi-page fetch caches, and a genuinely-small one-page discography
under max_pages=1 still caches. 1087 spotify/cache/watchlist/artist tests
pass.
carlosjfcasero: 'Champagne Supernova (OurVinyl Sessions)' is in the library
but the artist page shows it unowned and wishlist cleanup never removes it.
Measured with the real catalogs: Deezer/iTunes title the TRACK with the
qualifier while the library track is bare (the qualifier lives in the album
title) — and _calculate_track_confidence crushed that pair to ~0.17: the
"clean" titles keep parenthetical words, so the length-ratio penalty treats
'Champagne Supernova' vs 'Champagne Supernova (OurVinyl Sessions)' as
different songs. (Also confirmed: the OurVinyl release is absent from
Deezer's discography for the artist, so the standard page's 25-release list
not showing it is the source catalog, not a bug.)
Fix 1 — core.text.title_match.strip_redundant_context_qualifiers: a
parenthetical qualifier whose text appears (word-bounded) in the db track's
ALBUM title — or in the other title — restates release context and is
stripped for a comparison variant scored with its own length guard. Genuine
version markers keep their penalty: '(Live)' on a studio album appears in no
context and still blocks; '(Live)' on 'Live at Wembley' correctly matches —
owning the live album IS owning the live cut. Wired into
_calculate_track_confidence, so every check_track_exists consumer (wishlist
cleanup, discography dedup, repair jobs) benefits.
Fix 2 — the artist-page ownership endpoint's album gate: when album-aware
narrowing eliminates EVERY library candidate (the source's album naming just
doesn't resemble the library's — 'Jillette Johnson | OurVinyl Sessions' vs
'Champagne Supernova (OurVinyl Sessions)' ~0.5), fall back to artist-wide
title matching instead of declaring everything unowned off a failed
album-NAME comparison.
Tests: 8 — the exact reported pair end-to-end through check_track_exists,
word-boundary containment ('live' in 'alive' doesn't count), version-marker
safety both ways, and prefix songs still blocked. 1125 matching/wishlist/
library tests pass.
Review findings from PR #801, fixed as promised after merge:
- core/imports/version_mismatch_fallback.py and core/downloads/task_worker.py
used bare getLogger(__name__) — outside the soulsync.* namespace where
handlers attach, so the entire retry story (the [Modal Worker] search/retry
walk and, critically, the "accepting best quarantined candidate as last
resort" warning) never reached app.log. Same bug class as the prepare.py
fix; both moved to get_logger. A repo sweep shows 61 more modules with the
same pattern — noted as its own cleanup project.
- the full-suite run also caught a miss of MINE, not the PR's: the new
origin-history.js wasn't registered in the script-split integrity test, so
openDownloadOriginsModal failed onclick coverage. Registered — and the
onclick scan now iterates the NON_SPLIT_JS registry instead of its own
hardcoded copy, so the next standalone module can't silently skip coverage.
Merged dev verified: PR's 77 tests + 4233 full-suite tests pass (the only
exclusion is the eternal soundcloud /app file); integrity suite 64/64.
Watchlist scans add announced albums on purpose (so singles download the day
they drop), but the future-dated tracks leaked into two hot paths:
- Fresh Tape / Release Radar: future albums got NEGATIVE days_old, and the
recency score (100 - days*7) has no upper clamp — prereleases weren't just
slipping into the radar, they were mathematically FAVORED above every
released track. That's the "50% prerelease" report. The builder now skips
confidently-future albums (and clamps days_old to 0 as a belt).
- Wishlist processing: every auto cycle burned a full Soulseek search +
timeout per unreleased track (~60 tracks/cycle for the reporter). Both the
auto and manual flows now skip future-dated tracks with a counted log line.
They STAY in the wishlist and join the cycle automatically the day their
release date passes — no state, the date check is per-cycle. An explicit
manual track selection overrides the gate (the user asked for those).
The gate (core/metadata/release_dates.py, pure + tested) is conservative by
design: Spotify dates come as YYYY / YYYY-MM / YYYY-MM-DD, and a track only
gates when its date is CONFIDENTLY future at its stated precision. Release
day counts as released; garbage or missing dates never block anything
(including out-of-range months/days, which fall back a precision level).
Tests: 6 covering all precisions, the release-day boundary, garbage
tolerance, dict shapes, and ordered partitioning. 403 wishlist/watchlist
tests pass.
User ask: "a modal that lists the tracks downloaded via watchlist" — extended,
as discussed, to playlists too. One modal, two tabs, opened from the Watchlist
page (watchlist tab preselected) and the Sync page (playlists tab) — same
shared-modal-different-entry-points UX as the rest of the app.
The data: library_history recorded which SERVICE a file came from but never
what TRIGGERED it. New origin/origin_context columns (migration + index) are
written once at the import chokepoint via core/downloads/origin.py, a pure
tested deriver that reads, in priority: an explicit _dl_origin stamp (set at
batch-task creation for direct playlist batches, where the playlist context
otherwise only survived in folder mode), the wishlist provenance already
riding in track_info.source_info (watchlist_artist_name / playlist_name —
watchlist_scanner has stamped these for ages), and the folder-mode playlist
thread. Manual downloads stay unclassified by design. History starts from
now — provenance can't be conjured retroactively.
API: GET /api/download-origins?origin=watchlist|playlist (paged) and POST
/api/download-origins/delete — deletes the file on disk (resolved through the
shared container/host path resolver), the matching library track row, and the
history entries; a file that refuses deletion keeps its row and reports the
error instead of lying.
UI: webui/static/origin-history.js — tabbed modal in the revamp design
language (accent light-edge, pill tabs, entry rows reusing the
library-history-entry components), per-row delete + select-all bulk delete
with honest result toasts, empty/loading states, per-tab totals.
Tests: 8 — deriver priority/shapes (incl. the exact watchlist_scanner
source_info shape and JSON-string survival), origin filtering + counts,
row fetch/delete isolation between origins, delete-track-by-path.
The lock-in pass caught the cost hole: art is fetched PER TRACK, and the old
code never touched archive.org at all — so an archive.org outage was free,
while the new native-first chain would pay a 10s timeout on every track
(a 12-track album = +2 minutes, exactly the import-slowness class we spent
today killing). One failed original now puts originals on a 10-minute
cooldown: subsequent fetches go straight to the 1200px CDN midpoint (the
pre-#806 behavior, full speed) and recover automatically when the cooldown
expires. Locked by a test: track 1 pays the failure once, track 2 never
touches the original. (Also: the missing time import the first run caught.)
The CAA branch of _upgrade_art_url capped art at the /front-1200 thumbnail —
a deliberate flakiness trade-off, but the policy had rotted into inconsistency:
iTunes art already shipped at 3000x3000, and bare /front URLs (release-group
lookups — exactly what the Re-tag flow produces) bypassed the cap entirely,
which is how Sokhi observed retag delivering full-res while downloads got 1200.
CAA URLs now upgrade to the bare /front ORIGINAL (native res, frequently
3000px+). The flakiness concern that motivated the old cap is handled where it
belongs, in the fetch: _fetch_art_bytes now walks an attempt chain — original
-> /front-1200 midpoint -> the original sized thumbnail — so a flaky
archive.org degrades to the old 1200px behavior, never below it.
Tests updated to the new contract (+3 chain tests: native-first, flaky
degrades to 1200 not 250, full chain ends at the thumbnail). 623 metadata +
1267 art-path tests pass.
Caught live by the new lookup timing ("Genius track lookup took 242.4s"):
the 429 handler slept the backoff (30/60/120s) in the CALLING thread and then
re-raised anyway — the import pipeline waited 2x120s per track for lookups
that still failed. Worse, the pre-flight backoff wait also slept while
HOLDING the global Genius API lock, so every other Genius caller queued
serially behind the nap.
Now the backoff is a gate: a 429 opens a 30s->60s->120s window and re-raises
immediately; any call inside the window raises GeniusRateLimitedError on the
spot. The error subclasses requests.RequestException, so every existing
caller (the import's source lookups catch RequestException and skip; the
worker's per-item guards) already handles it as a one-line skip — lyrics and
Genius tags are garnish, nothing is allowed to WAIT for them.
Tests: backoff window fails fast (<0.5s vs the old full-window sleep), a 429
opens and escalates the gate without sleeping, the error is a
RequestException (the no-call-site-changes hinge), success decays the gate.
Measured during a live album download: ~4m15s per track in post-processing
(normal is ~20s), with the time vanishing silently inside embed_source_ids —
up to 5 MusicBrainz calls per track crawling against a degraded musicbrainz.org
while the MB enrichment worker kept eating the same ~1 req/s per-IP budget.
Only Spotify/Last.fm/Genius were in the yield set; MusicBrainz, Deezer, iTunes,
Discogs etc. kept grinding through downloads.
Policy (new core/enrichment/yield_policy, tested):
- downloads active -> ALL enrichment workers yield (post-processing touches
every metadata source). listening-stats (local-only) and repair
(user-scheduled) intentionally keep running.
- discovery active -> the API-contention five yield (spotify/itunes/deezer/
discogs/hydrabase) — discovery never paused anything before, despite the
pause helper literally defaulting to label='discovery'.
- user overrides and user-paused bookkeeping keep their existing semantics;
the dashboard yield_reason label now says WHICH foreground work caused it.
Observability (the 4-minute silence can never come back):
- every source lookup is timed; >2s logs a warning NAMING the source and
duration (core/metadata/source.py _call_source_lookup)
- the pipeline always logs "Metadata enhancement took X.Xs" per track
7 policy tests (incl. the motivating case: MB yields to downloads, keeps
running during discovery); 277 pipeline/enrichment tests pass.
A user reports ~0.7 MiB/s RSS growth; the one theory offered so far
(connection leak) was debunked, so instead of guessing: measure. New
core/diagnostics/memory_tracker wraps tracemalloc behind three GET endpoints
the user can drive from a browser:
/api/debug/memory/start begin tracing + baseline snapshot (idempotent)
/api/debug/memory/report top allocation sites by GROWTH since the baseline
(?top=N), with traced totals + process RSS so we
can see how much of the real growth tracing
accounts for; 15-frame tracebacks name the caller
/api/debug/memory/stop end tracing, free trace bookkeeping
Opt-in by design — tracemalloc shadows every allocation while active, so it
never runs by default. RSS via psutil with a /proc fallback.
Tests: report-without-tracking returns a hint (not an error); a real
start->hog->report->stop roundtrip attributes a genuine 5MB allocation to the
test file (fun fact encoded in the test: 'x'*1000 constant-folds into ONE
shared string and traces as ~40KB — the hog must allocate at runtime); the
stat formatter is duck-typed and unit-tested.
"Liked Songs" isn't a real Spotify playlist — no playlist URI exists for it;
the web UI invents the virtual id 'spotify:liked-songs' and Spotify serves the
collection via the saved-tracks endpoint. The playlist DETAIL endpoint special-
cases that id, but the mirrored refresh path resolves stored ids through
get_playlist_by_id, which fed the virtual id straight into sp.playlist() ->
"http status: 400 ... Unsupported URL / URI" on every sync cycle, silently.
get_playlist_by_id now special-cases the virtual id at the client seam (every
by-id resolver benefits, not just the mirror adapter): it builds the Playlist
from the existing get_saved_tracks() pagination, with the real owner name and
track count. New LIKED_SONGS_PLAYLIST_ID constant owns the magic string.
Safety: get_saved_tracks swallows fetch errors into [] — indistinguishable
from "no likes" — and the virtual playlist is only ever offered when likes
exist. An empty result therefore resolves as a FAILED refresh (None) instead
of a valid-looking empty playlist a mirror sync might propagate by clearing
the server-side copy.
Tests: virtual id resolves from saved tracks and never touches the playlist
endpoint, real ids still do (regression), the mirrored adapter seam returns a
full PlaylistDetail, and empty saved-tracks -> None. 473 passed across the
playlist/mirror/spotify families.
On path-mapped setups (Docker mounts etc.) the scan checked a bare
os.path.isfile() on the raw DB path — false for EVERY track — while the apply
handler resolves container/host mismatches. With a cover-art mode set, the
cover_action kept the album past the "anything to do?" gate, so every album
produced a finding with an empty tracks list whose apply could only ever fail
with "No tracks to re-tag in finding".
- the scan now resolves each track path with the same resolver the apply
handler uses (resolve_library_file_path) before reachability checks and
current-tag reads; plans carry the resolved path
- a finding can never be created with zero tracks — cover-action albums with
no usable tracks are skipped, with a debug log of why (unreachable/unmatched
counts) and the counts surfaced in the finding description
- unmatched-but-reachable tracks now get an art-only plan (empty db_data) so
album cover art covers ALL the album's files, not just source-matched ones —
apply_track_plans already treats empty db_data as a pure cover embed and
counts a failed cover download as skipped, never failed (now locked by tests)
- cover-only findings are titled "(cover art, N track(s))" instead of the
puzzling "(0 track(s))"
Tests: +5 (mapped paths resolve into plans, cover-with-nothing-reachable
creates no finding, unmatched -> art-only plan, art-only plan embeds cover,
failed cover download -> skipped). 87 passed across retag/repair/tag_writer.
Since the per-listener stream sessions refactor (Phase 3b), every browser gets
its own stream session — but the 1s 'tool:stream' socket broadcast still read
the legacy GLOBAL state (the DEFAULT session no real browser uses), so it told
every client "stopped" forever. The frontend skipped HTTP polling whenever the
WebSocket was up, so it only ever saw that wrong broadcast: the backend prep
downloaded the track, moved it into the session's stream folder and sat at
"ready" while the mini player showed nothing. Proxy users whose WebSockets
don't connect fell back to HTTP polling (session-correct) and streamed fine —
which is why this hid so well.
Fix: stream status is inherently per-listener, so stop pretending a global
broadcast can carry it —
- web_server.py: remove the 'tool:stream' emit from the tool-progress loop
(the broadcast thread has no request context; it can only ever see DEFAULT)
- media-player.js: the status poller always polls /api/stream/status (resolves
the caller's own session from the cookie); drop the dead broadcast handler
- core.js: unwire the 'tool:stream' socket listener
Observability fix that made this undebuggable: core/streaming/prepare.py used
getLogger(__name__) — outside the soulsync.* namespace where handlers attach —
so every prep log line (including failures) vanished from app.log. Moved to
get_logger("streaming.prepare") + a regression test locking the namespace.
34 streaming tests pass; ruff clean; web_server compiles; JS syntax-checked.
A single enriched against the deluxe gets every source ID pointing at the deluxe,
so the organizer filed it as e.g. track 2 of a 10-track album. Root cause: the
canonical resolver only ever scored the editions already linked — the correct
single was never even a candidate, and the misfit deluxe scored so low (0.1,
below the 0.5 floor) that nothing got pinned and the priority-walk grabbed the
deluxe anyway.
Fix, in three tested layers:
- resolve_canonical_for_album gains a fetch_alternates seam: when no linked
edition clears the floor, it scores the source's OTHER editions of the same
release and re-picks by best fit (dedup, injected, pure).
- default_fetch_alternates lists the artist's editions and keeps the same-release
ones (edition-blind name match: Deluxe / - Single / [Remastered] all collapse),
returning their tracklists. Favors recall; the scorer is the precision gate.
- _resolve_source does the misfit check inline: it fit-scores the walked edition
and only on a clear misfit searches for a better edition, then persists the pin
on apply (Track Number Repair + future runs agree). Cost-neutral and behavior-
identical for well-fitting albums (no extra API calls); strict_source and the
#758 manual lock are never overridden.
Tests: +4 resolver (expand/no-expand/dedupe/back-compat), +7 alternates (name
matcher + fetcher over fake APIs + cap), +3 organizer end-to-end (misfit->single
+pin, well-fit->no-expand, strict->no-expand). 300 passed across the reorganize
+ canonical family, lint clean.
User report: "Not authenticated with Spotify" daily + workers paused. The
log was misleading — is_spotify_authenticated() returns False for five
distinct reasons (no creds / rate-limit ban / post-ban cooldown / no
cached token / probe failure), and all five API call sites logged the
same bare "Not authenticated with Spotify". So a routine rate-limit ban
read as a logout, and the real cause (logged only at DEBUG) was invisible.
New pure describe_spotify_unavailable() maps the real state to a clear
message (priority matches is_spotify_authenticated): not-configured →
rate-limited ("ban ~Nm left (not a logout)") → post-ban cooldown →
no-token ("not connected — re-authenticate") → "auth check failed (token
refresh may have failed)". A side-effect-free client method
(_auth_unavailable_reason) gathers the live state (reads the cached token,
no API probe) and the 5 sites log it.
Now a daily ban is identifiable as a ban, and a genuine logout is
identifiable as one — so reports like this are diagnosable from the log
alone. 7 tests pin the priority/messaging. Full suite clean.
#798 follow-up. The worker's 500/day budget is a REAL-API ban shield, but
when it was hit the worker paused outright — even for a Spotify-Free user
with the uncapped free source available. So "I'm on Spotify Free" still
got capped overnight. The intuition is right: if it's ever using Spotify
Free, the budget shouldn't apply.
Fix: spent budget now becomes a third "use free" trigger (alongside
no-auth and rate-limited). When the real-API budget is exhausted and the
free source is available, the worker switches to free (uncapped) for the
rest of the day instead of pausing, then reverts to real-first on the
daily reset.
- should_use_free_fallback gains a budget_exhausted arg (free activates on
no-auth OR rate-limited OR spent-budget).
- the worker sets _budget_exhausted_use_free on ITS OWN client (a separate
instance from the search client — verified, so user searches still use
real auth), and clears it when the budget resets; _free_active() honors
the flag.
- get_stats() using_free reports the budget-bridge too, and the dashboard
bubble shows "Running (Spotify Free)" instead of "Daily Limit Reached"
(budgetStuck = exhausted AND not bridging).
A no-free user still pauses on the budget (nothing to bridge to). A pure
free-only worker never increments the budget at all. New gate test pins
the budget_exhausted trigger. Full suite clean.
Some tracks don't exist on the sources in the wanted cut — every copy is, say,
the instrumental. The retry engine correctly rejects each (version mismatch) and
gives up, leaving the track missing. New opt-in fallback: once a track's AcoustID
retries are fully exhausted, if every quarantined candidate for it failed the
SAME version mismatch (same matched version, e.g. all instrumental) and there are
>= N of them, accept the best (first-tried = oldest = highest-confidence) one.
Safety rules (core/imports/version_mismatch_fallback.py):
- Version mismatches only. Audio/artist mismatches (different recording) and
integrity/duration failures (truncated/wrong file) never participate.
- All qualifying entries must share the same matched version; a mix
(instrumental + live) is ambiguous → no acceptance.
- Re-import bypasses ONLY the AcoustID gate; integrity/duration/bit-depth still
run, so a truncated or genuinely wrong file is never let through here.
- Reuses the existing quarantine approve_quarantine_entry + re-verify dispatch.
Wired at the AcoustID give-up point in the verification wrapper. Two new
post_processing settings surfaced in the Retry Logic tile (default off):
accept_version_mismatch_fallback + version_mismatch_min_count.
Pure decision core + orchestration covered by tests (11). Acceptance logged at
WARNING with track + matched version.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The cross-script alias bridge (#442/#586) silently returned [] for some
artists ("Sawano Hiroyuki"). Root cause: the mb-only escape — built exactly
for the case where local string similarity is ~0 (romaji↔kanji) but MB's own
score is decisive — inspected scored[0], the COMBINED-score leader. When an
unrelated same-script decoy outranks the real artist on combined score (decoy:
sim 0.82 + mb_score 83 → combined 0.82, just under the 0.85 bar; real '澤野弘之':
sim 0 + mb_score 100 → combined 0.30, sorted last), the gate saw the decoy's
mb_score 83 (< 95), failed both paths, and cached an empty alias result.
Verification then scored the kanji artist 0% against the romaji expected name
and quarantined every correct file.
Evaluate the MB-SCORE leader independently of combined ranking for the mb-only
escape, and pull aliases from whichever entity actually passed (combined leader
for the combined path, MB-score leader for the mb-only path). The unambiguity
check now compares the top two raw MB scores. Same-script and single-result
paths are unchanged (regression-guarded by the existing #442/#586 tests).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The cached-first retry (8d98b755) abandoned a source after a single query:
the first run returns as soon as ONE query starts a download, so
cached_candidates held only that query's results. On a quarantine retry the
whole source was then excluded from re-search (via searched_sources), so the
later queries (e.g. "artist + album") never hit that source again — it jumped
to the next source after one query instead of exhausting all queries per
source.
Track searched QUERIES (searched_queries) instead of whole sources. A
quarantine retry now skips only the already-run queries (their candidates are
walked via cached-first) and still searches the not-yet-run queries against the
same source. Budget-exhausted sources (exhaustive mode) stay excluded, so the
source switch still fires when a source is genuinely spent.
Removes the now-dead searched_sources state (written but no longer read).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Each AcoustID/integrity quarantine retry re-ran the FULL search (all queries,
all sources) before picking the next-best candidate — so a track that failed
verification a dozen times re-queried Soulseek a dozen times (~3 min/cycle in
the field). The next-best pick was already sitting in cached_candidates.
Now the monitor flags the re-queue as a quarantine retry; the worker walks the
already-found candidates first (skipping used + budget-exhausted sources) and
hands them straight to the download path — no search. A source is searched
exactly once: once its candidates are cached, later quarantine retries exclude
it (searched_sources) so the hybrid chain falls through to a not-yet-searched
source instead of re-querying the spent one. Fresh downloads and the monitor's
dead-connection/stuck retries clear searched_sources and search fresh, so the
only re-search is for a genuinely new source or a dead peer.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
In exhaustive retry mode, a source that spent its whole per-source budget
(query_count × retries_per_query) gave up and failed the track outright —
never trying the other configured sources. For tracks where Soulseek has a
deep pool of wrong peers (e.g. an AcoustID title mismatch every copy shares),
the budget tripped long before HiFi/Tidal/… were ever reached.
Now, when a source's budget is spent, the monitor marks it exhausted on the
task and re-queues so the worker excludes it from the next hybrid search,
falling through to the next source in the chain. Each new source spends its
own fresh budget. The task only fails once no fallback source remains (or the
absolute total ceiling trips) — single-source mode still fails immediately,
since there's nothing to fall back to.
task_worker folds the exhausted-source set into both the orchestrator search
exclusion and the hybrid-fallback source list.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds an opt-in exhaustive mode to the quarantine-retry path. Default
behaviour is unchanged: a single global cap (MAX_QUARANTINE_RETRIES=5).
When post_processing.retry_exhaustive is on, each source gets its OWN
retry budget sized as query_count x retries_per_query. Soulseek peers
collapse to one 'soulseek' bucket; streaming plugins keep their name.
The worker now records query_count on the task; the budget scales with
the track's real query count. Loop protection is threefold: per-source
cap, used_sources exhaustion (the natural terminator), and an absolute
ceiling (MAX_TOTAL_QUARANTINE_RETRIES=100).
New settings (config + WebUI): retry_next_candidate_on_mismatch (master),
retry_exhaustive, retries_per_query (default 5).
Tests: 6 new cases covering per-source budgeting, source separation,
Soulseek-peer bucketing, query_count default, and the absolute ceiling.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When a downloaded file is quarantined because AcoustID verification or the
integrity/duration check fails, the task no longer dead-ends as failed — it
re-runs the worker on the next-best candidate, skipping the quarantined source.
Reuses the monitor's existing transfer-error retry machinery (used_sources +
cached_candidates + worker re-dispatch), just triggered from the post-process
verification wrapper's two quarantine branches instead of only on transfer
errors. Universal across sources (Soulseek, HiFi, Tidal, etc.) since all
batch/sync downloads funnel through post_process_matched_download_with_verification.
- monitor.requeue_quarantined_task_for_retry(): marks bad source used, resets
task to searching, resubmits worker. Guards: manual picks, cancelled tasks,
missing source id, and a MAX_QUARANTINE_RETRIES=5 loop cap.
- Opt-out via post_processing.retry_next_candidate_on_mismatch (default on).
- Manual quarantine approve is unaffected (_skip_quarantine_check='all' bypasses
the checks, so no quarantine flag, so no retry).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Users manually match an album to the regular edition, but enrichment/
repair keeps treating it as the deluxe (missing songs, renumbered tracks).
Root cause: an album has TWO identities — the enrichment match
(spotify_album_id, which manual-match sets and the worker already honors)
and a SEPARATE canonical version pin (canonical_album_id, added by #777).
The canonical pin is what track-number repair / reorganize / missing-track
detection actually read, and library_manual_match never wrote it — so it
was resolved independently and landed on the deluxe edition.
(So #777 did NOT solve #758: it added canonical pinning, but manual
matches didn't write the pin.)
Fix: a manual ALBUM match on a canonical-recognised source now also pins
AND locks the canonical version to the chosen release:
- new canonical_locked column (same migration pattern as the other
canonical cols).
- set_album_canonical(..., locked=False) gains an atomic WHERE-clause
guard: an auto write can't overwrite a locked pin; a manual write
(locked=True) always wins. get_album_canonical exposes `locked`.
- library_manual_match pins canonical for album matches via the pure
should_pin_manual_canonical(entity_type, source).
The auto resolve job already skips already-pinned albums, so the lock is
protected on two fronts; the new guard also covers any future
re-resolution. A new manual match still overrides.
18 tests: the pure gate (+ a sync-invariant test vs _ALBUM_ID_COLUMNS)
and the DB lock seam (auto can't clobber a manual lock; manual overrides;
auto-over-auto still works). Additive — locked defaults False, so the
auto path is unchanged unless a manual lock exists. Full suite clean.
#798 follow-up. The enrichment worker's own loop already bridges to the
no-creds Spotify Free source during a rate-limit ban (its guard checks
is_spotify_metadata_available()). But the resume button's pre-check
(_spotify_resume_pre_check) blocked resume on ANY rate-limit with no
awareness of Free — so a Free-opted-in user who got rate-limited was
locked out of restarting the worker, unable to fall through to the free
API.
Fix: the resume guard now mirrors the worker. Block only when
rate-limited AND nothing can serve (plain auth, no Free) — where resuming
would just sleep out the ban. When Free is available it serves during the
ban (is_spotify_authenticated() is False while banned, so
is_spotify_metadata_available() reports the free source), so resume is
allowed and the worker bridges via Free, then returns to real auth once
the ban lifts. Stays real-API-first; Free is only the bridge.
The rule is pinned in a pure helper should_block_rate_limited_resume()
next to the other gate functions, with 3 tests. Full suite clean (only
pre-existing soundcloud /app env failures remain).
A mis-grouped library track sits under a 'Various Artists' / '[Unknown
Album]' record while the file itself is correctly tagged. Write Tags
reads the DB and stamped that junk over the file — destroying the
correct tags. (Rematching never helped: a match only stores a source-ID
pointer, it never changes the local name/title the writer reads.)
Guard: never replace a real file value with a placeholder. Added at both
seams so preview and write agree:
- build_tag_diff marks such a field protected/no-change (the preview no
longer shows a wrong overwrite, and has_changes reflects reality),
- write_tags_to_file reads the file's current values and skips the
placeholder-over-real fields, preserving the file.
Field-agnostic and direction-safe: the guard fires ONLY when the DB
value is a placeholder AND the file holds a real one. A legitimate value
still writes — including a genuine 'Various Artists' album artist on a
real compilation, where the file has no conflicting real value, so the
guard doesn't fire. Every write_tags_to_file caller writes DB->file as a
correction, so blocking placeholder-over-real is correct for all of them
(the download post-process uses a different path, embed_source_ids).
23 new tests (placeholder detection, the guard fn, build_tag_diff on the
screenshot-#2 scenario, end-to-end FLAC write preserving real values and
still overwriting with real ones). 113 existing repair/retag/tag tests
pass.
The post-scan reconcile previously ran AFTER the worker's 'finished'
signal, which flips db_update_state status to 'finished'. Automations
wait for a scan by polling that status, so they stopped waiting before
the reconcile ran — and the dashboard/Tools card showed "Completed" then
flipped to "Reading file tags…". For incremental scans this was
invisible (sub-second); for a full refresh it was a real gap (a chained
automation would fire minutes before the IDs were filled).
Fix: the worker now routes completion through _emit_finished(), which
runs self.post_scan_hook (the reconcile) FIRST, then emits 'finished'.
The hook is injected by the web layer (it owns path resolution). So:
- status stays 'running' through the reconcile,
- the reconcile pushes its phase ("Reading file tags for N new tracks…")
and per-track progress through the SAME db_update_state callbacks the
scan already uses — so automations, the dashboard card, and the Tools
page all see it for free and wait for it,
- 'finished' is emitted exactly once, AFTER the reconcile — race-free, no
status blip a poll could catch,
- best-effort: a hook exception never blocks 'finished', so a scan can't
get stranded as perpetually 'running'.
Both scan entry points (_run_database_update_task, _run_deep_scan_task)
set the hook before run()/run_deep_scan(); the redundant post-run calls
are removed.
5 ordering tests pin the contract (hook-before-finished, finished still
fires without a hook, hook exception doesn't block finished, hook gets
the worker). Full suite clean (only pre-existing soundcloud /app env
failures remain).
Extends the manual "Import IDs from File Tags" backfill so newly-scanned
files get their embedded provider IDs pulled into the DB automatically —
no button press needed to keep up with new music.
How it works:
- insert_or_update_media_track now returns 'inserted' / 'updated' / False
(truthy-compatible; existing `if track_success` callers unaffected) so
the scan worker can tell a genuinely new row from an update.
- DatabaseUpdateWorker collects the ids it newly INSERTED this run
(self._new_track_ids) across all insert paths (Plex/Jellyfin/deep).
- After run()/run_deep_scan(), web_server calls _reconcile_after_scan(),
which gap-fills embedded IDs for just those new tracks. Runs as a
post-scan pass (the scan loop itself is untouched/fast — the media
server API never exposes these custom IDs, so the file must be read
once regardless; batching at the end keeps it out of the hot loop and
best-effort so it can never abort a scan). A progress phase ("Reading
file tags for N new tracks…") surfaces the full-refresh tail.
Shared engine:
- New reconcile_library() in core does the paging + lazy parent-map
loading (only loads albums/artists actually referenced — cheap when
scoped to a few new tracks) + per-page commits. BOTH the manual button
and the scan hook call it, so there's one tested orchestration, no
duplication. The backfill job was refactored onto it.
Same hardened safety: gap-fill only, atomically guarded against
overwrite, schema-introspected, idempotent. Scoped to new arrivals for
incremental/deep; full refresh re-inserts everything as new (recovering
the IDs a full-refresh wipe destroys).
+10 reconcile tests (reconcile_library scope/idempotency/progress/stop +
the engine). Full suite clean (only pre-existing soundcloud /app env
failures remain).
Files SoulSync (or MusicBrainz Picard) already tagged carry Spotify /
iTunes / MusicBrainz / Deezer / Tidal / AudioDB / Genius / Last.fm IDs in
their metadata. Enrichment workers gate their queues on
{provider}_match_status IS NULL, so reading those IDs back and gap-filling
the {provider}_id + match_status='matched' columns lets the workers skip
the API lookup entirely — big API savings on an already-tagged library.
New manual job in Tools -> Database & Scanning ("Import IDs from File
Tags"): scans every library file, reads embedded IDs, fills any that are
missing in the DB. Background job + progress card, mirroring the
write-tags-batch pattern.
core/library/embedded_id_reconcile.py (pure + tested):
- plan_reconcile(): gap-fill plan for a track + its album + artist. Only
empty id columns are planned; a disagreeing embedded id is a conflict,
never applied.
- apply_reconcile_plan(): one guarded UPDATE per id column —
WHERE id=? AND (col IS NULL OR col=''). The guard makes the fill atomic:
if an enrichment worker matched the same entity between our read and
this write, the UPDATE affects 0 rows instead of clobbering it. Columns
are introspected so a schema missing a provider's columns is skipped.
- reconcile_track_row(): per-track orchestration (id extraction, plan ->
apply, keeping the in-memory parent maps fresh for sibling tracks).
Job hardening: paged track scan (bounded memory), per-page commits (don't
starve concurrent workers), per-file try/finally (one bad file can't abort
the run), counters from real rowcount.
Scope: 19 column-fills across 8 providers. MB *recording* (track) id is
left out (UFID frame the reader doesn't surface; Vorbis key ambiguous) —
MB album+artist are covered. Amazon/ASIN deliberately excluded (ASIN is a
different namespace than the worker's amazon_id). All target columns
verified against the live schema.
Purely additive: new module, two new endpoints, one new Tools card —
no existing behavior changed. 20 unit tests (incl. the concurrency guard).
Full suite clean (only pre-existing soundcloud /app env failures remain).
AcoustID returns a recording's title/artist in their ORIGINAL script
(e.g. "久石譲" for Joe Hisaishi) while SoulSync's expected metadata is
romanized/English. A correct download then fails verification on two
walls: the title can never clear the 0.70 similarity bar cross-script,
and the only skip path that ignores the title required a near-perfect
0.95 fingerprint plus a resolved alias. Result: every non-English
artist trips it. Two complementary fixes, per the reporter's two ideas.
Graceful fix (automatic):
- New pure core/matching/script_compat.py detects when two strings are
in genuinely different writing systems (CJK/Hangul/Cyrillic/Greek/
Arabic/Hebrew/Thai vs Latin). Accented Latin (Beyoncé, Sigur Rós)
stays Latin — no false trigger.
- acoustid_verification.py: when the EXPECTED artist and the matched
artist span scripts AND the artist is confirmed via the existing
MusicBrainz alias bridge, SKIP instead of quarantine, without the
0.95 floor (the 0.80 trust floor already gates the fingerprint).
- Deliberately narrow: keyed on the ARTIST spanning scripts + being
confirmed. A same-script artist with only a cross-script title keeps
the stricter 0.95 floor, so the #607 wrong-file protection (Kendrick
R.O.T.C, low-fingerprint Japanese-title) is untouched.
Per-request toggle (manual escape hatch):
- New "Skip AcoustID verification" checkbox in the download-missing
modal beside "Force Download All".
- skip_acoustid threads request -> batch -> per-track track_info ->
download context (same path as _playlist_folder_mode), landing on
the existing _skip_quarantine_check='acoustid' bypass. No new
mechanism; only the AcoustID gate is bypassed (integrity/bit-depth
still run).
Tests:
- tests/matching/test_script_compat.py — script-boundary cases.
- test_acoustid_skip_logic.py — Joe Hisaishi SKIPs at 0.85; unconfirmed
cross-script artist still FAILs; same-script low-fingerprint still
FAILs.
- test_downloads_candidates.py — toggle injects the bypass; absent
toggle keeps verification.
Full suite: 5169 passed; only pre-existing soundcloud /app env failures
remain. Zero regressions.
Per the cleaner model: the free source only runs for users who explicitly picked
'Spotify Free' — not for every connected user. _free_wanted() is now just
_free_selected() (dropped the has-credentials auto-trigger). So:
- Plain 'Spotify' user, rate-limited -> waits out the ban as before (no surprise
background scraping, no ToS exposure for people who never chose free).
- 'Spotify Free' user, no auth -> free serves.
- 'Spotify Free' user who also connects an account -> official when healthy,
free bridges only during a rate-limit, then switches back.
Rewrote the metadata-source help text as a plain per-source list with a clear
note on how Spotify Free + a connected account interact. Gate tests updated to
pin the opt-in behavior (plain-Spotify ratelimit = no bridge; Spotify-Free
ratelimit = bridge).
Consistency fix: Spotify Free is now its own entry in the metadata-source
dropdown (alongside Spotify / iTunes / Deezer / MusicBrainz) instead of a
side-toggle. Stored as fallback_source='spotify' + spotify_free=true so all
downstream 'spotify' routing and the spotify_* columns are unchanged.
Refined gate model (no toggle):
- Connected user (has credentials) -> official; bridges to free AUTOMATICALLY
during a rate-limit ban (no opt-in needed).
- No-auth user -> must pick 'Spotify Free' in the dropdown; then free serves.
- Never opted into Spotify (no creds, didn't pick it) -> free never runs, so no
surprise scraping. _free_wanted() = has_credentials OR picked-spotify-free is
the guard.
- AUTHED + healthy -> official always; free never opens.
UI: dropdown gains 'Spotify Free (no credentials)' (selectable when the package
is installed — surfaced via status.free_installed, since selecting it is the
opt-in and can't depend on having selected it); load/save map the dropdown value
to the (fallback_source, spotify_free) pair; old checkbox removed.
Gate model pinned by 6 scenario tests (connected/healthy, connected/ratelimited
bridge, no-auth picked, no-auth not-opted-in, package-missing). 117 tests green.
Adds an opt-in no-creds Spotify metadata path: SpotifyClient serves SpotipyFree
data (real Spotify IDs) when metadata.spotify_free is enabled AND SpotipyFree is
installed AND there's no real Spotify auth. The data lands in the SAME spotify_*
columns, so the enrichment worker, search, and #775 lookups work UNCHANGED —
they just receive free-sourced data. The worker's only change is its availability
gate.
- core/spotify_free_metadata.py: SpotifyFreeMetadataClient + normalize_artist +
pure gate fns (should_use_free_fallback / should_offer_spotify_metadata /
spotify_free_installed).
- SpotifyClient: _free_enabled (opt-in, default OFF) / _free_available /
is_spotify_metadata_available / _free_active + in-client routing for
search_artists/tracks + get_album/artist/track_details/album_tracks/artist_albums.
- 3 scoped availability gates use is_spotify_metadata_available(): enrichment
worker loop, search resolve_client, watchlist. The ~40 discovery/user-library
sites stay auth-only (no sprawl, no user-data risk).
Authed users are untouched (gate closed when auth healthy). UI surfacing comes
next. Pure gates + normalizer tested; orchestrator resolve test added.
The #799 uniqueness-guard added a source_id_conflict(self.db, ...) call to the
MusicBrainz worker's artist path. Two TestWorkerAliasEnrichment fixtures build
the worker via __new__ and set only .database, not .db, so the new call raised
AttributeError and the artist was marked 'error'. Mirror the third fixture
(which already sets worker.db). Production always sets self.db in __init__ —
test-only gap exposed by the new code path.
Enumerates all 64 flag combinations and asserts should_rediscover matches the
verbatim pre-refactor inline logic for every case except the one intended fix
(manual_match beating a stale wing_it_fallback). Guarantees the auto Playlist
Pipeline behaves identically post-refactor — no regression.
A manually-fixed mirrored track silently reverted to 'Wing It' after re-running
discovery. Two compounding causes:
- extra_data is MERGED on save (update_mirrored_track_extra_data), and the
manual-fix DB write (web_server.py) didn't clear the prior wing_it_fallback
flag — so a track fixed after being a Wing It stub kept wing_it_fallback=True.
- the Playlist Pipeline pre-scan checked wing_it_fallback BEFORE manual_match
(if/elif), so the stale flag won: the track was re-discovered and, on a miss,
fell back to Wing It — discarding the user's pick.
Fix: extracted the pre-scan gate into core.discovery.manual_match.should_rediscover
(manual_match checked FIRST = authoritative, regardless of leftover flags), and
the manual-fix write now also clears wing_it_fallback/unmatched_by_user. Behavior
is identical for every other branch — only the manual-vs-wing-it ordering changes.
Tested at the seam incl. the exact regression (wing_it_fallback + manual_match
both set -> skip). 227 discovery tests green.
Ships the source-id cleanup to all users: a marker-gated one-time migration in
MusicDatabase init clears any source id (deezer/spotify/itunes/musicbrainz/
discogs/audiodb/qobuz/tidal) shared across differently-named artists — the
enrichment-corruption signature. Same-name cross-server duplicates are left
untouched (DISTINCT-name check). Cleared rows re-derive correct ids on the next
enrichment pass; the now name-guarded workers won't re-corrupt.
Runs once (CREATE TABLE _source_id_dedupe_v1 marker), idempotent, per-column
try/except so a missing column can't abort it. Test forces a re-run and asserts
corruption is cleared while a legit same-name dup survives.
