The payload has carried daily_budget {used, limit, exhausted} forever and the
dashboard rendered none of it. The avatar disc now wears a conic progress rim
that fills as the day's real-API budget is spent — green to 70%, amber to 95%,
red after — and flips purple once the worker has bridged to Spotify Free for
the rest of the day (using_free now included in the emit payload). Tooltip
carries the exact used/limit numbers.
A banned service used to just tint red. The payload carries the seconds
remaining (rl_remaining), so the bar now locks into a cooldown state: the live
VU dims, a red column drains away as the ban ticks down (largest remaining
seen is latched as the denominator — only remaining is sent), and an m:ss
timer counts to recovery. The moment the ban expires the track flashes green
('recovered') and the VU takes the stage back. 'Back in 4:00', not
'something's red'.
A thin accent marker sticks at each bar's recent maximum, holds ~1.2s, then
falls a few percent per update until it rests on the live fill — exactly how a
hardware VU meter's peak LED behaves. A traffic burst stays readable for a few
seconds after it's over instead of vanishing with the next 1s sample. Hidden
while it sits on the fill so idle bars don't carry a stray line.
The inactive->active transition (caught at the existing 30-frame state
refresh) now jolts the orb: a random-direction velocity kick with a briefly
lifted speed cap so it actually darts, a fast shallow size wobble (~1s decay),
and three sparks. A worker waking up reads as 'it sprang into action' instead
of just getting brighter.
Zero active workers + no pulses in flight for 75s eases the whole header into
a drowse (~4s): nucleus dims to embers, logo and orbs fade ~35-45%, spokes and
links fade ~70%, drift slows to a quarter speed (velocities keep integrating
so motion stays continuous). The first sign of work wakes it in ~0.3s with
three staggered rings blooming out of the nucleus. Idle and busy finally look
DIFFERENT — the contrast is what makes activity read as alive.
Each inbound pulse draws 3 fading ghost positions behind its head. The path is
parametric (eased t), so the tail is the same easing evaluated slightly in the
past — no position history, and it naturally stretches as the pulse accelerates
into the nucleus. Makes the energy flow legible at a glance.
A telemetry pulse used to just vanish on arrival. Now it makes contact: a small
expanding ring at the rim where it hits (pulse's own color, ~0.5s fade) plus
two debris sparks splashing back along the approach direction — the nucleus
reads as absorbing the work instead of deleting it. Capped pool (24), reset
alongside sparks/inflows on page switches and collapses.
Each worker orb gets a slow z-oscillation (-1 back .. +1 front); orbs draw in
painter's order with the hub pinned at z=0, so the cluster visibly swings
behind and in front of the nucleus instead of drifting on a flat plane. Depth
scales size ±18% and dims the back arc; the physics stay 2D. Effect eases out
during the hover-expand morph so orbs land on their buttons at natural size.
On page load every orb sat at the canvas origin: orbs are created at (0,0) and
the random scatter skipped orbs that weren't visible yet — which on load is all
of them, since the header hasn't been laid out when init() runs. The whole
cluster then drifted in from the top-left corner.
scatterOrbs() -> centerOrbs(): spawn the cluster dead-center of the canvas,
positioning every orb regardless of visibility (a few px of jitter on purpose —
the separation force ignores pairs closer than 0.1px, so a perfect stack would
never split). enterOrbState() also re-centers right after resizeCanvas(), so
activations get the true center even when init ran against a hidden 0x0 header
— and returning to the dashboard replays the center-bloom intro.
"Liked Songs" isn't a real Spotify playlist — no playlist URI exists for it;
the web UI invents the virtual id 'spotify:liked-songs' and Spotify serves the
collection via the saved-tracks endpoint. The playlist DETAIL endpoint special-
cases that id, but the mirrored refresh path resolves stored ids through
get_playlist_by_id, which fed the virtual id straight into sp.playlist() ->
"http status: 400 ... Unsupported URL / URI" on every sync cycle, silently.
get_playlist_by_id now special-cases the virtual id at the client seam (every
by-id resolver benefits, not just the mirror adapter): it builds the Playlist
from the existing get_saved_tracks() pagination, with the real owner name and
track count. New LIKED_SONGS_PLAYLIST_ID constant owns the magic string.
Safety: get_saved_tracks swallows fetch errors into [] — indistinguishable
from "no likes" — and the virtual playlist is only ever offered when likes
exist. An empty result therefore resolves as a FAILED refresh (None) instead
of a valid-looking empty playlist a mirror sync might propagate by clearing
the server-side copy.
Tests: virtual id resolves from saved tracks and never touches the playlist
endpoint, real ids still do (regression), the mirrored adapter seam returns a
full PlaylistDetail, and empty saved-tracks -> None. 473 passed across the
playlist/mirror/spotify families.
On path-mapped setups (Docker mounts etc.) the scan checked a bare
os.path.isfile() on the raw DB path — false for EVERY track — while the apply
handler resolves container/host mismatches. With a cover-art mode set, the
cover_action kept the album past the "anything to do?" gate, so every album
produced a finding with an empty tracks list whose apply could only ever fail
with "No tracks to re-tag in finding".
- the scan now resolves each track path with the same resolver the apply
handler uses (resolve_library_file_path) before reachability checks and
current-tag reads; plans carry the resolved path
- a finding can never be created with zero tracks — cover-action albums with
no usable tracks are skipped, with a debug log of why (unreachable/unmatched
counts) and the counts surfaced in the finding description
- unmatched-but-reachable tracks now get an art-only plan (empty db_data) so
album cover art covers ALL the album's files, not just source-matched ones —
apply_track_plans already treats empty db_data as a pure cover embed and
counts a failed cover download as skipped, never failed (now locked by tests)
- cover-only findings are titled "(cover art, N track(s))" instead of the
puzzling "(0 track(s))"
Tests: +5 (mapped paths resolve into plans, cover-with-nothing-reachable
creates no finding, unmatched -> art-only plan, art-only plan embeds cover,
failed cover download -> skipped). 87 passed across retag/repair/tag_writer.
Since the per-listener stream sessions refactor (Phase 3b), every browser gets
its own stream session — but the 1s 'tool:stream' socket broadcast still read
the legacy GLOBAL state (the DEFAULT session no real browser uses), so it told
every client "stopped" forever. The frontend skipped HTTP polling whenever the
WebSocket was up, so it only ever saw that wrong broadcast: the backend prep
downloaded the track, moved it into the session's stream folder and sat at
"ready" while the mini player showed nothing. Proxy users whose WebSockets
don't connect fell back to HTTP polling (session-correct) and streamed fine —
which is why this hid so well.
Fix: stream status is inherently per-listener, so stop pretending a global
broadcast can carry it —
- web_server.py: remove the 'tool:stream' emit from the tool-progress loop
(the broadcast thread has no request context; it can only ever see DEFAULT)
- media-player.js: the status poller always polls /api/stream/status (resolves
the caller's own session from the cookie); drop the dead broadcast handler
- core.js: unwire the 'tool:stream' socket listener
Observability fix that made this undebuggable: core/streaming/prepare.py used
getLogger(__name__) — outside the soulsync.* namespace where handlers attach —
so every prep log line (including failures) vanished from app.log. Moved to
get_logger("streaming.prepare") + a regression test locking the namespace.
34 streaming tests pass; ruff clean; web_server compiles; JS syntax-checked.
A single enriched against the deluxe gets every source ID pointing at the deluxe,
so the organizer filed it as e.g. track 2 of a 10-track album. Root cause: the
canonical resolver only ever scored the editions already linked — the correct
single was never even a candidate, and the misfit deluxe scored so low (0.1,
below the 0.5 floor) that nothing got pinned and the priority-walk grabbed the
deluxe anyway.
Fix, in three tested layers:
- resolve_canonical_for_album gains a fetch_alternates seam: when no linked
edition clears the floor, it scores the source's OTHER editions of the same
release and re-picks by best fit (dedup, injected, pure).
- default_fetch_alternates lists the artist's editions and keeps the same-release
ones (edition-blind name match: Deluxe / - Single / [Remastered] all collapse),
returning their tracklists. Favors recall; the scorer is the precision gate.
- _resolve_source does the misfit check inline: it fit-scores the walked edition
and only on a clear misfit searches for a better edition, then persists the pin
on apply (Track Number Repair + future runs agree). Cost-neutral and behavior-
identical for well-fitting albums (no extra API calls); strict_source and the
#758 manual lock are never overridden.
Tests: +4 resolver (expand/no-expand/dedupe/back-compat), +7 alternates (name
matcher + fetcher over fake APIs + cap), +3 organizer end-to-end (misfit->single
+pin, well-fit->no-expand, strict->no-expand). 300 passed across the reorganize
+ canonical family, lint clean.
User report: "Not authenticated with Spotify" daily + workers paused. The
log was misleading — is_spotify_authenticated() returns False for five
distinct reasons (no creds / rate-limit ban / post-ban cooldown / no
cached token / probe failure), and all five API call sites logged the
same bare "Not authenticated with Spotify". So a routine rate-limit ban
read as a logout, and the real cause (logged only at DEBUG) was invisible.
New pure describe_spotify_unavailable() maps the real state to a clear
message (priority matches is_spotify_authenticated): not-configured →
rate-limited ("ban ~Nm left (not a logout)") → post-ban cooldown →
no-token ("not connected — re-authenticate") → "auth check failed (token
refresh may have failed)". A side-effect-free client method
(_auth_unavailable_reason) gathers the live state (reads the cached token,
no API probe) and the 5 sites log it.
Now a daily ban is identifiable as a ban, and a genuine logout is
identifiable as one — so reports like this are diagnosable from the log
alone. 7 tests pin the priority/messaging. Full suite clean.
User request: the re-tag diff card shows old→new metadata per track but not
the file it applies to, so a wrong match is hard to spot before applying.
The finding already carries each track's file_path (details.tracks[].file_path
from the scan) — the renderer just wasn't showing it. Now each changed track
displays its filename beneath the title, so you can eyeball that the metadata
about to be written actually belongs to that file. Skipped when the label is
already the filename (track had no title). Pure frontend display change.
#798 follow-up. The worker's 500/day budget is a REAL-API ban shield, but
when it was hit the worker paused outright — even for a Spotify-Free user
with the uncapped free source available. So "I'm on Spotify Free" still
got capped overnight. The intuition is right: if it's ever using Spotify
Free, the budget shouldn't apply.
Fix: spent budget now becomes a third "use free" trigger (alongside
no-auth and rate-limited). When the real-API budget is exhausted and the
free source is available, the worker switches to free (uncapped) for the
rest of the day instead of pausing, then reverts to real-first on the
daily reset.
- should_use_free_fallback gains a budget_exhausted arg (free activates on
no-auth OR rate-limited OR spent-budget).
- the worker sets _budget_exhausted_use_free on ITS OWN client (a separate
instance from the search client — verified, so user searches still use
real auth), and clears it when the budget resets; _free_active() honors
the flag.
- get_stats() using_free reports the budget-bridge too, and the dashboard
bubble shows "Running (Spotify Free)" instead of "Daily Limit Reached"
(budgetStuck = exhausted AND not bridging).
A no-free user still pauses on the budget (nothing to bridge to). A pure
free-only worker never increments the budget at all. New gate test pins
the budget_exhausted trigger. Full suite clean.
All retry features introduced by this branch are now strictly opt-in.
With every setting at its default, behaviour is identical to before the
branch — tracks that fail AcoustID/integrity are quarantined and left
for manual review, same as before.
Users who want the new retry pipeline enable it in
Settings → Downloads → Retry Logic.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When a file is quarantined (AcoustID / integrity / bit-depth), the source
is moved away and _mark_task_quarantined sets _quarantine_entry_id on the
context. A second post_process_matched_download call with the same
file_path (caused by a monitor re-poll or concurrent dispatch before the
context is cleaned up) then hit the race guard — "source file gone, no
known destination" — and overwrote the in-flight retry with a failed status.
Fix: check _quarantine_entry_id before firing the race guard. If it is
set, the file is legitimately in quarantine and this is a stale duplicate;
return silently so the quarantine retry already in flight can proceed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The existing fallback (pipeline.py:1084) only ran inside
post_process_matched_download_with_verification — i.e. when a file *was*
downloaded and AcoustID retries were fully exhausted. If the retry *search*
itself found zero valid candidates (source returned nothing, or all failed
HiFi validation), the task was marked not_found and the fallback was never
reached, even though the quarantine already held N version-mismatch entries.
Fix: add try_version_mismatch_fallback to TaskWorkerDeps; in the "no valid
candidates" path of task_worker, invoke it before marking not_found when
is_quarantine_retry. Wired in _build_task_worker_deps via a new helper
(_try_version_mismatch_fallback_for_worker) that calls
try_accept_version_mismatch_fallback directly with the track's title and
artist and a reprocess lambda over _post_process_matched_download_with_verification.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Reinstate the Soulseek dependency (quality profile only affects Soulseek
downloads) that was dropped while fixing the empty-tile bug. Gate the whole
collapsible tile (#quality-profile-tile) as a unit instead of the inner group,
so it fully shows (Soulseek active + downloads tab) or fully hides — no empty
expandable shell. switchSettingsTab runs updateDownloadSourceUI after the
data-stg tab filter, so this gate is authoritative on tab switches.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Some tracks don't exist on the sources in the wanted cut — every copy is, say,
the instrumental. The retry engine correctly rejects each (version mismatch) and
gives up, leaving the track missing. New opt-in fallback: once a track's AcoustID
retries are fully exhausted, if every quarantined candidate for it failed the
SAME version mismatch (same matched version, e.g. all instrumental) and there are
>= N of them, accept the best (first-tried = oldest = highest-confidence) one.
Safety rules (core/imports/version_mismatch_fallback.py):
- Version mismatches only. Audio/artist mismatches (different recording) and
integrity/duration failures (truncated/wrong file) never participate.
- All qualifying entries must share the same matched version; a mix
(instrumental + live) is ambiguous → no acceptance.
- Re-import bypasses ONLY the AcoustID gate; integrity/duration/bit-depth still
run, so a truncated or genuinely wrong file is never let through here.
- Reuses the existing quarantine approve_quarantine_entry + re-verify dispatch.
Wired at the AcoustID give-up point in the verification wrapper. Two new
post_processing settings surfaced in the Retry Logic tile (default off):
accept_version_mismatch_fallback + version_mismatch_min_count.
Pure decision core + orchestration covered by tests (11). Acceptance logged at
WARNING with track + matched version.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Soulseek-only JS gate fought the settings tab filter for control of the
tile's display: gating the inner group left an empty expandable shell, gating
the wrapper hid the whole tile depending on activeSources/tab timing. Remove
the JS gate entirely and let the tab filter (data-stg="downloads" on
#quality-profile-tile) own visibility — identical to the working Retry Logic
tile. The tile now reliably shows on the Downloads tab.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Quality Profile tile expanded to an empty body: settings.js
updateSourceVisibility toggled only the inner #quality-profile-section
(Soulseek-only + downloads-tab gate), leaving the new collapsible tile's
header/body visible with hidden contents. Wrap the tile in
#quality-profile-tile and gate that wrapper as a unit instead, so the whole
tile shows (Soulseek active) or hides (otherwise) — no empty shell.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Settings reorg (Downloads page):
- Move the retry controls (retry next-best candidate, exhaustive retry,
retries-per-query) out of the Post-Processing tile into a new collapsible
"Retry Logic" tile on the Downloads tab (data-stg=downloads), collapsed by
default. Also decouples them from the post-processing master toggle, which
previously hid them when post-processing was disabled. Config keys are
unchanged (still post_processing.*); settings.js binds by element id so the
DOM move needs no JS change.
- Wrap the existing Quality Profile group in a matching collapsible tile,
collapsed by default.
Sidebar (reduce-effects):
- The perf PR (#793) gated .nav-button hover/active-hover behind
body:not(.reduce-effects), removing the highlight entirely in reduce-effects
mode. Restore the highlight there using only cheap properties (flat
background + border-color; the base already reserves a 1px transparent
border so there's no layout shift) while keeping the expensive gradient /
translateX transform / multi-layer box-shadow off.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The cross-script alias bridge (#442/#586) silently returned [] for some
artists ("Sawano Hiroyuki"). Root cause: the mb-only escape — built exactly
for the case where local string similarity is ~0 (romaji↔kanji) but MB's own
score is decisive — inspected scored[0], the COMBINED-score leader. When an
unrelated same-script decoy outranks the real artist on combined score (decoy:
sim 0.82 + mb_score 83 → combined 0.82, just under the 0.85 bar; real '澤野弘之':
sim 0 + mb_score 100 → combined 0.30, sorted last), the gate saw the decoy's
mb_score 83 (< 95), failed both paths, and cached an empty alias result.
Verification then scored the kanji artist 0% against the romaji expected name
and quarantined every correct file.
Evaluate the MB-SCORE leader independently of combined ranking for the mb-only
escape, and pull aliases from whichever entity actually passed (combined leader
for the combined path, MB-score leader for the mb-only path). The unambiguity
check now compares the top two raw MB scores. Same-script and single-result
paths are unchanged (regression-guarded by the existing #442/#586 tests).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The cached-first retry (8d98b755) abandoned a source after a single query:
the first run returns as soon as ONE query starts a download, so
cached_candidates held only that query's results. On a quarantine retry the
whole source was then excluded from re-search (via searched_sources), so the
later queries (e.g. "artist + album") never hit that source again — it jumped
to the next source after one query instead of exhausting all queries per
source.
Track searched QUERIES (searched_queries) instead of whole sources. A
quarantine retry now skips only the already-run queries (their candidates are
walked via cached-first) and still searches the not-yet-run queries against the
same source. Budget-exhausted sources (exhaustive mode) stay excluded, so the
source switch still fires when a source is genuinely spent.
Removes the now-dead searched_sources state (written but no longer read).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Each AcoustID/integrity quarantine retry re-ran the FULL search (all queries,
all sources) before picking the next-best candidate — so a track that failed
verification a dozen times re-queried Soulseek a dozen times (~3 min/cycle in
the field). The next-best pick was already sitting in cached_candidates.
Now the monitor flags the re-queue as a quarantine retry; the worker walks the
already-found candidates first (skipping used + budget-exhausted sources) and
hands them straight to the download path — no search. A source is searched
exactly once: once its candidates are cached, later quarantine retries exclude
it (searched_sources) so the hybrid chain falls through to a not-yet-searched
source instead of re-querying the spent one. Fresh downloads and the monitor's
dead-connection/stuck retries clear searched_sources and search fresh, so the
only re-search is for a genuinely new source or a dead peer.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
In exhaustive retry mode, a source that spent its whole per-source budget
(query_count × retries_per_query) gave up and failed the track outright —
never trying the other configured sources. For tracks where Soulseek has a
deep pool of wrong peers (e.g. an AcoustID title mismatch every copy shares),
the budget tripped long before HiFi/Tidal/… were ever reached.
Now, when a source's budget is spent, the monitor marks it exhausted on the
task and re-queues so the worker excludes it from the next hybrid search,
falling through to the next source in the chain. Each new source spends its
own fresh budget. The task only fails once no fallback source remains (or the
absolute total ceiling trips) — single-source mode still fails immediately,
since there's nothing to fall back to.
task_worker folds the exhausted-source set into both the orchestrator search
exclusion and the hybrid-fallback source list.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds an opt-in exhaustive mode to the quarantine-retry path. Default
behaviour is unchanged: a single global cap (MAX_QUARANTINE_RETRIES=5).
When post_processing.retry_exhaustive is on, each source gets its OWN
retry budget sized as query_count x retries_per_query. Soulseek peers
collapse to one 'soulseek' bucket; streaming plugins keep their name.
The worker now records query_count on the task; the budget scales with
the track's real query count. Loop protection is threefold: per-source
cap, used_sources exhaustion (the natural terminator), and an absolute
ceiling (MAX_TOTAL_QUARANTINE_RETRIES=100).
New settings (config + WebUI): retry_next_candidate_on_mismatch (master),
retry_exhaustive, retries_per_query (default 5).
Tests: 6 new cases covering per-source budgeting, source separation,
Soulseek-peer bucketing, query_count default, and the absolute ceiling.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When a downloaded file is quarantined because AcoustID verification or the
integrity/duration check fails, the task no longer dead-ends as failed — it
re-runs the worker on the next-best candidate, skipping the quarantined source.
Reuses the monitor's existing transfer-error retry machinery (used_sources +
cached_candidates + worker re-dispatch), just triggered from the post-process
verification wrapper's two quarantine branches instead of only on transfer
errors. Universal across sources (Soulseek, HiFi, Tidal, etc.) since all
batch/sync downloads funnel through post_process_matched_download_with_verification.
- monitor.requeue_quarantined_task_for_retry(): marks bad source used, resets
task to searching, resubmits worker. Guards: manual picks, cancelled tasks,
missing source id, and a MAX_QUARANTINE_RETRIES=5 loop cap.
- Opt-out via post_processing.retry_next_candidate_on_mismatch (default on).
- Manual quarantine approve is unaffected (_skip_quarantine_check='all' bypasses
the checks, so no quarantine flag, so no retry).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Users manually match an album to the regular edition, but enrichment/
repair keeps treating it as the deluxe (missing songs, renumbered tracks).
Root cause: an album has TWO identities — the enrichment match
(spotify_album_id, which manual-match sets and the worker already honors)
and a SEPARATE canonical version pin (canonical_album_id, added by #777).
The canonical pin is what track-number repair / reorganize / missing-track
detection actually read, and library_manual_match never wrote it — so it
was resolved independently and landed on the deluxe edition.
(So #777 did NOT solve #758: it added canonical pinning, but manual
matches didn't write the pin.)
Fix: a manual ALBUM match on a canonical-recognised source now also pins
AND locks the canonical version to the chosen release:
- new canonical_locked column (same migration pattern as the other
canonical cols).
- set_album_canonical(..., locked=False) gains an atomic WHERE-clause
guard: an auto write can't overwrite a locked pin; a manual write
(locked=True) always wins. get_album_canonical exposes `locked`.
- library_manual_match pins canonical for album matches via the pure
should_pin_manual_canonical(entity_type, source).
The auto resolve job already skips already-pinned albums, so the lock is
protected on two fronts; the new guard also covers any future
re-resolution. A new manual match still overrides.
18 tests: the pure gate (+ a sync-invariant test vs _ALBUM_ID_COLUMNS)
and the DB lock seam (auto can't clobber a manual lock; manual overrides;
auto-over-auto still works). Additive — locked defaults False, so the
auto path is unchanged unless a manual lock exists. Full suite clean.
#798 follow-up. The Spotify enrichment worker's daily budget and post-ban
cooldown both exist to protect the REAL authenticated API from bans. But
the worker bridges to the no-creds Spotify Free source during a ban (a
different, anonymous path), and those guards weren't free-aware:
- the budget guard slept the worker when the daily cap was hit, blocking
free work for no reason,
- free-served items still incremented the budget counter, draining the
real-API cap with calls that never touched the real API (so you'd
return from a ban with budget already spent), and
- the post-ban cooldown slept the worker even when serving via free.
Fix: compute free_serving = client._free_active() once per loop
(defensively wrapped → False on error → original behavior). When
free_serving:
- skip the daily-budget guard,
- skip the post-ban-cooldown guard,
- don't increment the daily budget.
So the budget is now strictly a cap on REAL Spotify API usage; free runs
unthrottled by it (the free client keeps its own inter-call pacing).
The decision input (_free_active) is already pinned by the gate-model
tests (free True exactly when rate-limited+Free / Free-primary, False when
authed+healthy). Full suite clean (only pre-existing soundcloud /app env
failures remain).
#798 follow-up. When the real Spotify API is banned but the worker keeps
matching via the no-creds Spotify Free source, every status surface still
read the literal rate_limited=True flag and showed "Rate Limited /
waiting Nm" — so the dashboard bubble looked paused/stuck even though the
worker (visible in Manage Workers) was actively matching.
- spotify_worker.get_stats() adds a `using_free` flag: rate_limited AND
is_spotify_metadata_available(). Computed ONLY when rate-limited, where
is_spotify_authenticated() returns False without an API probe, so the
2s status loop pays no quota cost.
- Dashboard bubble (enrichment.js): when using_free, the bubble is
'active', the tooltip says "Running (Spotify Free)" and "Now: X (via
Spotify Free)" instead of "Rate Limited / Waiting Nm". Clicking it
pauses (works) rather than hitting the resume-blocked toast.
- Manage Workers (enrichment-manager.js): status pill shows "Running
(Spotify Free)"; the warning banner is replaced with a calm "matching
via Spotify Free until the ban lifts" note.
The flag flows through both feeds (the /api/enrichment/spotify/status
poll and the WebSocket enrichment:* push) since both serialize
get_stats(). Genuinely-stuck (no-free) workers still show "Rate Limited".
#798 follow-up. The enrichment worker's own loop already bridges to the
no-creds Spotify Free source during a rate-limit ban (its guard checks
is_spotify_metadata_available()). But the resume button's pre-check
(_spotify_resume_pre_check) blocked resume on ANY rate-limit with no
awareness of Free — so a Free-opted-in user who got rate-limited was
locked out of restarting the worker, unable to fall through to the free
API.
Fix: the resume guard now mirrors the worker. Block only when
rate-limited AND nothing can serve (plain auth, no Free) — where resuming
would just sleep out the ban. When Free is available it serves during the
ban (is_spotify_authenticated() is False while banned, so
is_spotify_metadata_available() reports the free source), so resume is
allowed and the worker bridges via Free, then returns to real auth once
the ban lifts. Stays real-API-first; Free is only the bridge.
The rule is pinned in a pure helper should_block_rate_limited_resume()
next to the other gate functions, with 3 tests. Full suite clean (only
pre-existing soundcloud /app env failures remain).
A mis-grouped library track sits under a 'Various Artists' / '[Unknown
Album]' record while the file itself is correctly tagged. Write Tags
reads the DB and stamped that junk over the file — destroying the
correct tags. (Rematching never helped: a match only stores a source-ID
pointer, it never changes the local name/title the writer reads.)
Guard: never replace a real file value with a placeholder. Added at both
seams so preview and write agree:
- build_tag_diff marks such a field protected/no-change (the preview no
longer shows a wrong overwrite, and has_changes reflects reality),
- write_tags_to_file reads the file's current values and skips the
placeholder-over-real fields, preserving the file.
Field-agnostic and direction-safe: the guard fires ONLY when the DB
value is a placeholder AND the file holds a real one. A legitimate value
still writes — including a genuine 'Various Artists' album artist on a
real compilation, where the file has no conflicting real value, so the
guard doesn't fire. Every write_tags_to_file caller writes DB->file as a
correction, so blocking placeholder-over-real is correct for all of them
(the download post-process uses a different path, embed_source_ids).
23 new tests (placeholder detection, the guard fn, build_tag_diff on the
screenshot-#2 scenario, end-to-end FLAC write preserving real values and
still overwriting with real ones). 113 existing repair/retag/tag tests
pass.
The post-scan reconcile previously ran AFTER the worker's 'finished'
signal, which flips db_update_state status to 'finished'. Automations
wait for a scan by polling that status, so they stopped waiting before
the reconcile ran — and the dashboard/Tools card showed "Completed" then
flipped to "Reading file tags…". For incremental scans this was
invisible (sub-second); for a full refresh it was a real gap (a chained
automation would fire minutes before the IDs were filled).
Fix: the worker now routes completion through _emit_finished(), which
runs self.post_scan_hook (the reconcile) FIRST, then emits 'finished'.
The hook is injected by the web layer (it owns path resolution). So:
- status stays 'running' through the reconcile,
- the reconcile pushes its phase ("Reading file tags for N new tracks…")
and per-track progress through the SAME db_update_state callbacks the
scan already uses — so automations, the dashboard card, and the Tools
page all see it for free and wait for it,
- 'finished' is emitted exactly once, AFTER the reconcile — race-free, no
status blip a poll could catch,
- best-effort: a hook exception never blocks 'finished', so a scan can't
get stranded as perpetually 'running'.
Both scan entry points (_run_database_update_task, _run_deep_scan_task)
set the hook before run()/run_deep_scan(); the redundant post-run calls
are removed.
5 ordering tests pin the contract (hook-before-finished, finished still
fires without a hook, hook exception doesn't block finished, hook gets
the worker). Full suite clean (only pre-existing soundcloud /app env
failures remain).
Extends the manual "Import IDs from File Tags" backfill so newly-scanned
files get their embedded provider IDs pulled into the DB automatically —
no button press needed to keep up with new music.
How it works:
- insert_or_update_media_track now returns 'inserted' / 'updated' / False
(truthy-compatible; existing `if track_success` callers unaffected) so
the scan worker can tell a genuinely new row from an update.
- DatabaseUpdateWorker collects the ids it newly INSERTED this run
(self._new_track_ids) across all insert paths (Plex/Jellyfin/deep).
- After run()/run_deep_scan(), web_server calls _reconcile_after_scan(),
which gap-fills embedded IDs for just those new tracks. Runs as a
post-scan pass (the scan loop itself is untouched/fast — the media
server API never exposes these custom IDs, so the file must be read
once regardless; batching at the end keeps it out of the hot loop and
best-effort so it can never abort a scan). A progress phase ("Reading
file tags for N new tracks…") surfaces the full-refresh tail.
Shared engine:
- New reconcile_library() in core does the paging + lazy parent-map
loading (only loads albums/artists actually referenced — cheap when
scoped to a few new tracks) + per-page commits. BOTH the manual button
and the scan hook call it, so there's one tested orchestration, no
duplication. The backfill job was refactored onto it.
Same hardened safety: gap-fill only, atomically guarded against
overwrite, schema-introspected, idempotent. Scoped to new arrivals for
incremental/deep; full refresh re-inserts everything as new (recovering
the IDs a full-refresh wipe destroys).
+10 reconcile tests (reconcile_library scope/idempotency/progress/stop +
the engine). Full suite clean (only pre-existing soundcloud /app env
failures remain).
Files SoulSync (or MusicBrainz Picard) already tagged carry Spotify /
iTunes / MusicBrainz / Deezer / Tidal / AudioDB / Genius / Last.fm IDs in
their metadata. Enrichment workers gate their queues on
{provider}_match_status IS NULL, so reading those IDs back and gap-filling
the {provider}_id + match_status='matched' columns lets the workers skip
the API lookup entirely — big API savings on an already-tagged library.
New manual job in Tools -> Database & Scanning ("Import IDs from File
Tags"): scans every library file, reads embedded IDs, fills any that are
missing in the DB. Background job + progress card, mirroring the
write-tags-batch pattern.
core/library/embedded_id_reconcile.py (pure + tested):
- plan_reconcile(): gap-fill plan for a track + its album + artist. Only
empty id columns are planned; a disagreeing embedded id is a conflict,
never applied.
- apply_reconcile_plan(): one guarded UPDATE per id column —
WHERE id=? AND (col IS NULL OR col=''). The guard makes the fill atomic:
if an enrichment worker matched the same entity between our read and
this write, the UPDATE affects 0 rows instead of clobbering it. Columns
are introspected so a schema missing a provider's columns is skipped.
- reconcile_track_row(): per-track orchestration (id extraction, plan ->
apply, keeping the in-memory parent maps fresh for sibling tracks).
Job hardening: paged track scan (bounded memory), per-page commits (don't
starve concurrent workers), per-file try/finally (one bad file can't abort
the run), counters from real rowcount.
Scope: 19 column-fills across 8 providers. MB *recording* (track) id is
left out (UFID frame the reader doesn't surface; Vorbis key ambiguous) —
MB album+artist are covered. Amazon/ASIN deliberately excluded (ASIN is a
different namespace than the worker's amazon_id). All target columns
verified against the live schema.
Purely additive: new module, two new endpoints, one new Tools card —
no existing behavior changed. 20 unit tests (incl. the concurrency guard).
Full suite clean (only pre-existing soundcloud /app env failures remain).
AcoustID returns a recording's title/artist in their ORIGINAL script
(e.g. "久石譲" for Joe Hisaishi) while SoulSync's expected metadata is
romanized/English. A correct download then fails verification on two
walls: the title can never clear the 0.70 similarity bar cross-script,
and the only skip path that ignores the title required a near-perfect
0.95 fingerprint plus a resolved alias. Result: every non-English
artist trips it. Two complementary fixes, per the reporter's two ideas.
Graceful fix (automatic):
- New pure core/matching/script_compat.py detects when two strings are
in genuinely different writing systems (CJK/Hangul/Cyrillic/Greek/
Arabic/Hebrew/Thai vs Latin). Accented Latin (Beyoncé, Sigur Rós)
stays Latin — no false trigger.
- acoustid_verification.py: when the EXPECTED artist and the matched
artist span scripts AND the artist is confirmed via the existing
MusicBrainz alias bridge, SKIP instead of quarantine, without the
0.95 floor (the 0.80 trust floor already gates the fingerprint).
- Deliberately narrow: keyed on the ARTIST spanning scripts + being
confirmed. A same-script artist with only a cross-script title keeps
the stricter 0.95 floor, so the #607 wrong-file protection (Kendrick
R.O.T.C, low-fingerprint Japanese-title) is untouched.
Per-request toggle (manual escape hatch):
- New "Skip AcoustID verification" checkbox in the download-missing
modal beside "Force Download All".
- skip_acoustid threads request -> batch -> per-track track_info ->
download context (same path as _playlist_folder_mode), landing on
the existing _skip_quarantine_check='acoustid' bypass. No new
mechanism; only the AcoustID gate is bypassed (integrity/bit-depth
still run).
Tests:
- tests/matching/test_script_compat.py — script-boundary cases.
- test_acoustid_skip_logic.py — Joe Hisaishi SKIPs at 0.85; unconfirmed
cross-script artist still FAILs; same-script low-fingerprint still
FAILs.
- test_downloads_candidates.py — toggle injects the bypass; absent
toggle keeps verification.
Full suite: 5169 passed; only pre-existing soundcloud /app env failures
remain. Zero regressions.
Per the cleaner model: the free source only runs for users who explicitly picked
'Spotify Free' — not for every connected user. _free_wanted() is now just
_free_selected() (dropped the has-credentials auto-trigger). So:
- Plain 'Spotify' user, rate-limited -> waits out the ban as before (no surprise
background scraping, no ToS exposure for people who never chose free).
- 'Spotify Free' user, no auth -> free serves.
- 'Spotify Free' user who also connects an account -> official when healthy,
free bridges only during a rate-limit, then switches back.
Rewrote the metadata-source help text as a plain per-source list with a clear
note on how Spotify Free + a connected account interact. Gate tests updated to
pin the opt-in behavior (plain-Spotify ratelimit = no bridge; Spotify-Free
ratelimit = bridge).
The Settings dropdown reverted 'Spotify Free' because _isMetadataSourceSelectable
reads _lastStatusPayload.spotify.free_installed, but that payload comes from the
WebSocket status:update push (_build_status_payload) — which sent the raw spotify
dict. The availability flags were only added to the GET /status endpoint, so the
frontend never saw free_installed and bounced the selection.
Extract _spotify_status_with_availability() (metadata_available + free_installed)
and use it in BOTH _build_status_payload (WebSocket) and get_status (HTTP poll),
so they can't drift. Now 'Spotify Free' is selectable when SpotipyFree is
installed.
Consistency fix: Spotify Free is now its own entry in the metadata-source
dropdown (alongside Spotify / iTunes / Deezer / MusicBrainz) instead of a
side-toggle. Stored as fallback_source='spotify' + spotify_free=true so all
downstream 'spotify' routing and the spotify_* columns are unchanged.
Refined gate model (no toggle):
- Connected user (has credentials) -> official; bridges to free AUTOMATICALLY
during a rate-limit ban (no opt-in needed).
- No-auth user -> must pick 'Spotify Free' in the dropdown; then free serves.
- Never opted into Spotify (no creds, didn't pick it) -> free never runs, so no
surprise scraping. _free_wanted() = has_credentials OR picked-spotify-free is
the guard.
- AUTHED + healthy -> official always; free never opens.
UI: dropdown gains 'Spotify Free (no credentials)' (selectable when the package
is installed — surfaced via status.free_installed, since selecting it is the
opt-in and can't depend on having selected it); load/save map the dropdown value
to the (fallback_source, spotify_free) pair; old checkbox removed.
Gate model pinned by 6 scenario tests (connected/healthy, connected/ratelimited
bridge, no-auth picked, no-auth not-opted-in, package-missing). 117 tests green.
When Spotify Free is enabled, it now also bridges an official rate-limit ban for
authenticated users instead of stalling — search already did this (the gate
opens on no-auth OR rate-limit); this extends it to the enrichment worker.
- spotify_worker: the rate-limit guard now sleeps only when free CAN'T cover
(is_spotify_metadata_available() is False). Purely additive — with Spotify
Free off, that's False during a ban and the worker sleeps exactly as before.
Verified: toggle OFF + rate-limited -> sleeps (original); toggle ON -> bridges.
- Reframed the Settings toggle so connected users know it also covers rate-limits
("Use Spotify Free when Spotify is unavailable or rate-limited").
The official auth path is untouched; free never runs while authed Spotify works
normally.
Surfaces the opt-in Spotify Free source so it's usable end-to-end:
- Settings: 'Enable Spotify Free (no credentials)' toggle that saves
metadata.spotify_free (load + save wired). Clear best-effort/limitations note.
- config-status: adds spotify.metadata_available (configured OR free-available),
keeping the configured flag = has-credentials so the Connections indicator
stays honest. Search source picker shows Spotify when metadata_available.
- status payload: adds spotify.metadata_available; the Settings primary-source
selector now allows picking Spotify when authed OR free-available.
Verified gate composition: OFF by default (no surprise scraping); ON + no auth +
installed -> available & serving; AUTHED -> official always wins (free never
runs); missing package -> gracefully unavailable. JS + integrity + 111 tests green.
Adds an opt-in no-creds Spotify metadata path: SpotifyClient serves SpotipyFree
data (real Spotify IDs) when metadata.spotify_free is enabled AND SpotipyFree is
installed AND there's no real Spotify auth. The data lands in the SAME spotify_*
columns, so the enrichment worker, search, and #775 lookups work UNCHANGED —
they just receive free-sourced data. The worker's only change is its availability
gate.
- core/spotify_free_metadata.py: SpotifyFreeMetadataClient + normalize_artist +
pure gate fns (should_use_free_fallback / should_offer_spotify_metadata /
spotify_free_installed).
- SpotifyClient: _free_enabled (opt-in, default OFF) / _free_available /
is_spotify_metadata_available / _free_active + in-client routing for
search_artists/tracks + get_album/artist/track_details/album_tracks/artist_albums.
- 3 scoped availability gates use is_spotify_metadata_available(): enrichment
worker loop, search resolve_client, watchlist. The ~40 discovery/user-library
sites stay auth-only (no sprawl, no user-data risk).
Authed users are untouched (gate closed when auth healthy). UI surfacing comes
next. Pure gates + normalizer tested; orchestrator resolve test added.
The #799 uniqueness-guard added a source_id_conflict(self.db, ...) call to the
MusicBrainz worker's artist path. Two TestWorkerAliasEnrichment fixtures build
the worker via __new__ and set only .database, not .db, so the new call raised
AttributeError and the artist was marked 'error'. Mirror the third fixture
(which already sets worker.db). Production always sets self.db in __init__ —
test-only gap exposed by the new code path.
A leftover `.sync-tab-server { flex: 1.4 !important }` from the old equal-width
pills tab strip leaked past the brand-chip restyle (its !important beat the
chip's flex:0 0 auto), so the active Server Playlists pill spanned the whole row
instead of fitting its label. Dropped just that declaration — the tab now behaves
like every other chip; its bespoke gradient + the rest of the rule are untouched.
