Boulder: "all 50 tracks are discovered to Deezer already — it's not using any of that." Right —
the export only checked tracks.deezer_id (library) and ignored the IDs discovery already resolved
and stored in each mirrored track's extra_data. So tracks that were discovered+downloaded but not
separately enriched showed as "not on Deezer" and got dropped.
Adds a per-track waterfall for service export:
- service_id_from_extra_data(track, service): the id discovery already matched, read from
extra_data.matched_data.id — FREE (no API call) and reliable (it's the same id used to mirror
the track). Trusted only when discovered ON the target service (provider == service); a
wing_it_fallback (low-confidence guess) does NOT match here, so it falls through rather than
risk a wrong track in the export.
- resolve_service_track_ids(tracks, service): cache (extra_data) → library stored id → unmatched.
Reports from_cache / from_library / unmatched. _run_service_export now uses this instead of the
artist/title MBID-style resolver.
For Boulder's playlist this means all 50 resolve straight from the cache — full coverage, zero API
calls. (A live confident-search backfill for the genuinely-missing remainder is the optional next
step, gated + thresholded.)
9 new tests: extra_data id only when provider matches + wing_it excluded + bad-json/not-discovered
guards, the cache→library→unmatched waterfall with stat tallies, and _run_service_export resolving
straight from the cache end-to-end. 49 export tests green, ruff clean.
The broad except->None in db_service_track_id would mask a column/join typo as "no match"
for every track (so export would silently say "nothing to export"). Adds a test that builds
a real tracks/artists sqlite schema and runs the ACTUAL query — confirms it returns the right
spotify_track_id/deezer_id column, case-insensitively, and None on no-match. Closes the one
integration gap the mocked unit tests left open.
Ties the resolver + write clients into a working backend, reusing the ListenBrainz export's
resolve→push→store-target shape:
- _run_service_export(job, db, playlist_id, title, service, client, resolve_fn): resolves the
mirrored playlist's tracks to their stored service track IDs (id_key='service_track_id'),
guards "nothing matched", pushes via the injected write client, and stores the returned
playlist id as the export target so a re-export updates in place (idempotent, like LB #903).
Deps injected → unit-testable without a DB or live service.
- _run_playlist_export dispatches mode in {spotify, deezer} to it (builds the real client +
service resolver); the existing download/push (ListenBrainz/JSPF) flow is untouched.
- POST /api/playlists/<id>/export/service/<service> — distinct path so it can't collide with
the existing /export/listenbrainz route; validates the target, starts the background job,
returns {job_id} polled via the shared status endpoint.
5 orchestration tests (fake db/client/resolve_fn): success stores target + passes ids in order,
no-match → error with no push, client None → not-connected error, push failure surfaces the
client's error and stores nothing, re-export passes the existing target id. ruff clean.
Last piece: the modal options (Sync to Spotify / Deezer, gated on auth, unmatched count surfaced).
The Deezer write half of "export a mirrored playlist back to Deezer". Rides the private
gw-light gateway with the ARL session already authenticated for downloads (Deezer shut their
public developer API, so this is the only write path — unofficial and fragile by nature).
DeezerDownloadClient.create_or_update_playlist(name, track_ids, existing_id=None): playlist.create
with the songs (new), or playlist.addSongs to the stored target (re-export). track_ids are the
stored deezer_id per library track. Returns the same {success, playlist_id, url, added, error}
shape as the Spotify writer so the export job can treat both uniformly.
5 tests (mocked gateway): create sends playlist.create with positional songs, update sends
playlist.addSongs (no create), empty → error with no gw call, not-authed → error, gw rejection →
error. ruff clean. Additive — download paths untouched.
Both write clients done. Next: the export-job branch + endpoint (reusing get/set_playlist_export_
target for idempotency), then the modal options.
For exporting a mirrored playlist back to Spotify:
- The OAuth scope string was duplicated verbatim in 5 places (spotify_client, the per-profile
registry, and 3 web_server callbacks) — a drift hazard where the authorize URL and token
exchange could request different scopes and silently re-prompt/deny. Extracted ONE
SPOTIFY_OAUTH_SCOPE constant and pointed all 5 at it, and added playlist-modify-public/private
there. Existing users re-auth once to grant write; reads are unaffected.
- SpotifyClient.create_or_update_playlist(name, track_ids, existing_id=None): creates a playlist
owned by the authed user, or replaces an existing one's tracks in place (idempotent re-export).
Chunks at Spotify's 100-track cap. A pre-scope token gets a clear "reconnect Spotify" message
instead of a raw 403. Returns {success, playlist_id, url, added, error}.
6 tests: create-new adds tracks, update replaces (no create), >100 chunking, empty → error (no API
calls), not-authed → error, insufficient-scope → reconnect message. 268 spotify/oauth tests green,
ruff clean. Additive — read paths and existing tokens unchanged.
Next: Deezer write via the ARL gw-light gateway, then the export-job branch + endpoint + modal.
First piece of "export a mirrored playlist to Spotify/Deezer" (diegocade1). Reuses the exact
machinery the ListenBrainz/JSPF export already proves out, additively:
- resolve_playlist_tracks gains an `id_key` param (default "recording_mbid" → LB/JSPF callers
byte-for-byte unchanged). The dedup/stats/order logic is ID-agnostic; only the output field
name differs, so service export plugs in with id_key="service_track_id".
- export_sources gains db_service_track_id(artist, title, service) + build_service_resolve_fn —
text-matches a library track (same pattern as the MBID resolver) and returns its stored
spotify_track_id / deezer_id. Enrichment already pinned those IDs, so export is a lookup, not a
re-search — which is what makes the reverse direction reliable (no fuzzy guessing).
No schema change needed: get/set_playlist_export_target already key by service name, so Spotify/
Deezer targets store for free (idempotent re-export, like the LB #903 fix).
7 tests: LB default id_key unchanged, service id_key carries the id + unmatched handling, service→
column mapping, unknown-service/no-title guards, resolve_fn id+source. 38 export tests green,
ruff clean.
Remaining increments: Spotify write client (+ playlist-modify scope / re-auth), Deezer write via
the ARL gw-light gateway, the export-job branch + endpoint, the modal options.
The Wing It pool "Fix Match" search returned "no results" for everything (even obvious
tracks). Root cause: /api/spotify/search_tracks built a Spotify field-filtered query
(track:X artist:Y) and handed it to spotify_client.search_tracks, which falls back to the
user's configured source when official Spotify isn't serving the request. The fallback
(Deezer here) got the raw Spotify `track:…artist:…` syntax it can't parse and aborted the
connection (RemoteDisconnected) — so the user's perfectly working Deezer failed ONLY on
this path, on this query format. The iTunes and Deezer search endpoints already dropped
field syntax for exactly this reason; the Spotify one was the lone holdout.
fix:
- new pure helper relevance.build_combined_search_query(track, artist, legacy) — plain,
source-agnostic query; documents WHY field syntax is wrong here. the endpoint already
reranks by expected title/artist, so precision is recovered without the brittle syntax.
- the Spotify endpoint uses it (now consistent with iTunes/Deezer).
- frontend (searchPoolFix): surface the real error (auth / 500 / upstream abort) instead
of masking everything as "No results found" — which is what made this undiagnosable.
5 helper tests incl. the regression (output must contain no 'track:'/'artist:' syntax).
654 metadata/search tests green, 64 script-integrity green, ruff clean.
Threads the rename_only flag from the apply endpoint to the executor, additively (default
False everywhere → existing full-flow behaviour byte-for-byte unchanged):
- /api/library/album/<id>/reorganize-files reads `rename_only` from the body → enqueue.
- QueueItem gains rename_only (+ surfaced in to_dict for the status panel).
- reorganize_runner.build_runner takes build_final_path_fn and branches: a rename_only item
routes to reorganize_album_rename_only (no staging dir, no copy, no post-process); everything
else falls through to the full reorganize_album. Staging is only created for the full path now.
- web_server injects build_final_path_fn (= _build_final_path_for_track, the same builder the
preview uses) so apply matches the preview exactly.
Fixed a test landmine: _make_item returns a MagicMock, whose .rename_only is a truthy mock that
wrongly took the new branch — set it to False to match the real QueueItem default. +2 runner tests
(rename_only routes to the rename executor + creates no staging; missing path-builder → clean
setup_failed). 209 reorganize tests green, ruff clean.
Left: the modal (Full vs Rename-only) + optional post-rename server scan + the issue reply.
#875 (tsoulard/Tacobell444): the reorganize job runs the FULL download post-processing on every
track — copy to staging, re-tag, quality + AcoustID checks, then move. So it fails on the same
checks as downloads, is slow (a full copy per file on a NAS, not a rename), and re-touches EVERY
file even when only its name changes (Tacobell's "2 of 14 previewed but all 14 modified").
This adds the rename-only path users actually want for "just fix the filenames": move each file to
the path the current naming scheme dictates and nothing else — no copy, no re-tag, no checks. The
tags are already correct; only the on-disk filename/folder layout changes (their hardware DAP sorts
by filename).
Design (additive — the full flow is byte-for-byte untouched):
- preview_album_reorganize gains current_path_abs / new_path_abs (additive fields; existing
trimmed display paths unchanged) so the executor acts on EXACTLY what the preview computed —
apply can never disagree with what the user saw.
- reorganize_album_rename_only: consumes the preview (injected preview_fn for testability), and for
each track that's matched + actually changing + non-colliding, renames in place and updates the
SoulSync DB directly (authoritative — we just did the move, no need to round-trip a server scan).
unchanged tracks are SKIPPED — that's the fix for "every file got modified".
- _rename_track_in_place: os.rename with a cross-device (EXDEV) fallback to shutil.move, creates
the dest dir, carries sibling-format files (.flac+.opus) along, and refuses to overwrite a
different existing file (never silent data loss).
11 new tests incl. the headline regression (changed → moved + DB updated, unchanged → untouched),
collision/unmatched skip, overwrite-refusal, sibling carry, cross-device, stop, cleanup. 207
reorganize/library tests green, ruff clean. Endpoint flag + modal + post-rename server scan next.
The blurred 60fps worker-orb canvas is the main remaining Firefox lag source after the
#935 sweep (multiple Discord lag reports). So for a FIRST-TIME user with no saved
preference, default the orbs OFF on Firefox (smooth first impression where it's needed)
and ON everywhere else (full polish where the browser handles it). An explicit saved
choice ALWAYS wins — this only picks the default when the user hasn't chosen.
Done kettui-style with a SINGLE source of truth, not the dual browser-detection I first
floated (server UA + client _isFirefox would be the same fact in two places that can
drift — exactly the server/client class #943's green-flash fix just cleaned up):
- core/ui_appearance.py (new, pure + importable): is_firefox_user_agent +
resolve_worker_orbs_default(explicit, is_firefox) — explicit wins, unset → !firefox.
- web_server: the SERVER decides (UA via _request_is_firefox, request-context-safe) and
injects initial_worker_orbs_enabled; config default flipped None so "unset" is
distinguishable from an explicit False. The client just consumes the injected value
(init.js unchanged) — no client-side re-derivation of "is Firefox".
- settings.js: the orb checkbox default now reflects the server value when unset, so
saving Settings can't silently flip a first-time Firefox user's orbs back on.
No regression: Chrome users unchanged; users with an explicit setting unchanged (it
wins regardless of browser); /api/settings returns raw config so it can't clobber the
default for an unset value. Verified end-to-end through a real Flask request context
(Firefox→off, Chrome→on, explicit wins both ways, no crash outside a request). 8 pure
seam tests pin the contract; ruff clean.
"Reduce visual effects" was a sledgehammer: body.reduce-effects * forced
animation:none + transition-duration:0s on every element. That froze every CSS loading
spinner mid-rotation — including the dash-header worker-service spinners (musicbrainz /
spotify / deezer / … .active .<svc>-spinner) — which read as BROKEN rather than "off".
It also killed cheap hover feedback like the Quick Actions buttons.
The actual lag (esp. Firefox, see the #935 sweep) is backdrop-filter / box-shadow /
filter re-rasterizing every frame — NOT the animations themselves. Transform- and
opacity-only motion (the spinners) composites for ~free.
So: keep forcing the expensive properties to none (unchanged — that's the real fix),
but drop the blanket animation/transition kills. !important author declarations outrank
animation + transition declarations in the cascade, so any keyframe/transition that
tries to set blur/shadow/filter is still neutralized even while it runs — the spinner
spins, just without the glow. Net: functional spinners stay alive, Quick Actions hover
(transform + border-colour) returns, box-shadow transitions are no-ops (shadow forced
none), and the GPU-heavy rendering that caused the lag stays gone. The worker-orb CANVAS
is unaffected (JS-gated separately) and stays off under reduce-effects, as intended.
Static guard test pins the contract: the global rule must keep the expensive-property
kills and must NOT reintroduce blanket animation:none / transition:0s.
Discord (Shdjfgatdif, standalone): some downloads complete on disk but get marked failed with
"File not found on disk after 5 search attempts. Expected: <basename>" — which tells the user
nothing about where we looked or what to check.
This is deliberately a DIAGNOSTIC fix, not a behavior change. The finder + path handling are sound
(verified: docker_resolve_path no-ops in standalone, the finder walks the configured
soulseek.download_path and resolves a present file). When it still misses after slskd reported the
transfer Succeeded, the cause is environmental — either the file is still landing (timing) or, the
classic standalone gotcha, SoulSync's download_path doesn't point at slskd's actual download dir.
Neither is something our code can "fix"; the user fixes the config, or the file arrives.
So: name the folder we actually searched and spell out the two real causes, turning an opaque
failure into self-diagnosis ("oh, my download folder's wrong"). Retry/wait behavior is left
untouched on purpose — widening the window does nothing for a path mismatch and I can't justify it
for this user. Also normalizes the slskd backslash path so the reported filename is the leaf, not
the whole "@@@user\folder\file" string.
Updated the existing not-found test to pin the new actionable message (searched path + config
hint + filename). 588 downloads tests green, ruff clean.
Discord (Shdjfgatdif): "if a track isn't imported it should remain there, not be deleted, so we
can retry." He was seeing failed downloads disappear and having to re-download.
Normally a rejected file is QUARANTINED (moved to ss_quarantine, preserved + retryable), not
deleted. But all four quarantine blocks (integrity / silence / quality / acoustid) had the same
fallback: if move_to_quarantine itself raised, os.remove(file_path). On a NAS that move can fail
(cross-device / permissions), so the except fired and the user's download was DELETED — the worst
outcome, and exactly the re-download pain he reported.
fix: on quarantine failure, log and LEAVE the file in place — never delete. The task is still
marked failed and the batch still notified (that code runs after the try/except and never touched
the deleted file), so the only behaviour change is "preserved instead of destroyed". Reviewed
every os.remove in the pipeline: the remaining ones are success-path cleanups (replacing an
existing destination, or removing a redundant download when the track is already in the library at
equal/better quality) — left untouched.
regression test drives the REAL pipeline through integrity-rejection with quarantine forced to
raise, and asserts the source file is preserved while the task is still failed + notified.
1311 imports/downloads/quality tests green, ruff clean.
Discord (Shdjfgatdif): a downloaded .flac sat right there in the download folder but the import
flow reported "File not found on disk after 5 search attempts" and failed it.
root cause: slskd REPORTS the name as "[34 - You & Me (Flume Remix).flac" but SAVES it as
"34 - You & Me (Flume Remix).flac" (it strips the leading '['). The finder's fuzzy-match
normaliser used one combined bracket-strip — r'[\[\(].*?[\]\)]' — which allows MISMATCHED
delimiters, so the lone '[' matched all the way to the next ')', ate the whole title, and
collapsed the search target to just "flac". That scored 0.40 against the real filename (below the
0.85 floor) → "not found", despite the file being on disk. Confirmed by running the real code on
his exact filename.
fix: strip only BALANCED pairs (\[...\] and (...) separately). A stray unbalanced bracket now
survives to the alphanumeric strip instead of devouring the title. '[34 - You & Me (Flume Remix)'
→ matches at 1.00. Balanced tags like "[FLAC]" / "(Remastered 2016)" are still stripped (no
regression). Only used internally by the finder's fuzzy scorer — contained blast radius.
3 tests: his exact unbalanced-'[' filename, a stray-']' variant, and a balanced-tag no-regression
guard. 1311 imports/downloads/quality tests green, ruff clean.
Discord (DXP4800 NAS, 7148 tracks): library updates kept dying with "Update appears stuck — no
progress for 300s (last phase: Fetching all tracks in bulk...)". not actually hung.
root cause: the bulk track/album fetch used a single 10000-item page, so a whole library came
back in ONE request that emitted NO progress while in flight. the watchdog (database_update_health)
kills a job with no progress for 300s — so on a slow server that one silent request tripped it even
though it was alive, not stuck. raising the timeout cap only buys the silent request more rope; a
bigger library or slower disk just needs a higher number. the per-batch progress line also only ran
when there was a NEXT page, so a sub-page-size library reported nothing at all.
fix: extract a pure paginate_all_items seam (core/library/bulk_paginate.py) that pages in 1000s and
reports progress after EVERY page — so the watchdog is fed on a cadence set by page size, not library
size, and can't starve mid-fetch no matter how big the library. both Jellyfin bulk loops (tracks +
albums, same defect) now route through it. preserves the failure-shrink resilience (halve to a floor,
then give up). does NOT change what's fetched — same query, fields, items.
note: changes nothing about WHICH tracks come back; only how they're paged + that every page reports.
keep the raised cap on dev as a margin — this is the actual fix. Plex/Navidrome don't share the
pattern (checked). 9 seam tests incl. the watchdog-feed invariant (progress count scales with
N/page_size, never one call for the whole library) + the sub-page regression + failure-shrink.
467 jellyfin/library tests green, ruff clean.
#942's normalize_spotify_oauth_config trimmed whitespace/quotes (good — those can't be part of a
real credential) but ALSO rstrip("/")'d the redirect_uri. that's unsafe: Spotify matches the
redirect URI EXACTLY against the app's dashboard registration, and a trailing slash is a
legitimate part of a URI. stripping it would silently break anyone who registered '…/callback/'
(we'd send '…/callback' → INVALID_CLIENT: Invalid redirect URI) — trading one failure mode for a
sneakier one the user can't diagnose (SoulSync no longer sends what they typed).
drop the rstrip; keep the whitespace/quote trim. the value is now preserved verbatim apart from
unambiguous paste garbage. flipped the test that asserted the strip to assert the slash is kept
(and that whitespace/quotes around it are still trimmed), + a dedicated regression guard.
the #942 integration test mocks normalize, so it's unaffected. 262 spotify/oauth tests green.
credit: builds on HellRa1SeR's #942.
reopened by diegocade1: pasting a Qobuz track link still showed unrelated results. the earlier
fix (b1f061a) only BUBBLED the linked track to the top — but a pasted link is resolved to an
"artist title" text query and searched, and for an obscure track ("foreign lavennew" by colacola)
that text search returns broad lookalikes ("Foreign Bird", "Foreign Spies", …) and never the
actual track. nothing to bubble → user sees junk.
fix: since the link is already resolved via get_track(id), fetch that exact track AS a downloadable
result and inject it at the top (Qobuz downloads by id, so the result is fully usable). the text
search still runs for alternatives.
- QobuzClient.get_track_result(id): get_track + _qobuz_to_track_result; None on any failure.
- _qobuz_to_track_result gains require_streamable (default True for bulk search). the link fetch
passes False: track/get may OMIT the streamable flag, which would default-False and wrongly drop
the exact track the user explicitly asked for. (this closes the one shape assumption that
couldn't be verified against a live Qobuz API — the track is no longer gated on it.)
- track_link.inject_linked_track_first(tracks, linked_result, id): pure seam — prepend the fetched
result + drop any search duplicate; falls back to the bubble when no result was fetched.
- manual-search endpoint fetches linked_result defensively (getattr 'get_track_result') and calls
the seam. Tidal/HiFi (get_track returns a dict but the converter wants an object — shape
mismatch) have no get_track_result, so they keep the existing bubble path: NO regression.
14 tests: inject puts the fetched track first when search missed it / dedups a search copy / falls
back to bubble / str-safe id / noop; get_track_result convert/none/exception; and the REAL
converter builds a valid downloadable result from a track/get dict that OMITS streamable (search
path still rejects it). 85 track-link/qobuz tests green, ruff clean.
the frontend keeps its own copy of the lossless set (settings.js RT_LOSSLESS_FORMATS + the
index.html quality-profile dropdown) — runtime-fetching a yearly-changing list from the backend
isn't worth the coupling. but that duplication IS the exact root cause of #941 (a format added
to one place, not another). so instead of unifying, pin it: two tests parse the frontend lists
and assert they match the backend LOSSLESS_FORMATS. adding a new lossless format now fails CI
until it's added everywhere, instead of silently shipping a half-wired feature.
verified the guard catches drift (not a tautology): a simulated backend-only 'ape' addition
makes the equality fail. 18 lossless tests green, ruff clean.
radoslav-orlov: "create lossy copies of lossless tracks" only recognized FLAC, even though ALAC/
WAV/AIFF/DSD are now quality-profile formats. the FLAC knowledge was hardcoded in 3 separate
places (the import path, the Lossy Converter scan, and the fix executor) — exactly how a format
gets added in one spot but not another.
kettui-style fix — one canonical seam both sites route through, instead of 3 more string edits:
- new core/quality/lossless.py: is_lossless_format / is_lossless_audio_path (pure; injects a
codec probe for the ambiguous .m4a/.mp4 — ALAC vs AAC — so the decision stays testable with no
I/O), LOSSLESS_FORMATS (single source of truth, derived-consistent with model.tier_score), and
the lossy_output_would_overwrite_source safety invariant.
- create_lossy_copy + the Lossy Converter scan + repair_worker._fix_missing_lossy_copy all route
through it. SQL pre-filters by candidate extensions, then each file is confirmed (probing .m4a).
- SAFETY: a lossy copy must never be written over its own source — an .m4a ALAC source + AAC
target lands on the same .m4a path, and ffmpeg runs with -y. all three sites now bail on the
overwrite case BEFORE ffmpeg (the existing delete-original guard was too late — the source was
already clobbered). dropped a vestigial mutagen FLAC import; updated FLAC-only UI strings.
19 tests: full seam coverage (formats, the .m4a ALAC/AAC probe branch, candidate extensions, the
overwrite guard), a tier-model consistency test that fails if the lossless set drifts, and import-
site wiring tests — WAV now converts (was rejected), and the .m4a-ALAC+AAC overwrite case proves
ffmpeg NEVER runs. 286 quality/import/repair tests green, ruff clean.
diegocade1: DSD files (.dsf, ~500MB DSD64) were labeled "Low Quality" and nagged to upgrade.
two independent causes, both fixed (additive — no existing format/behaviour changed):
1) DSF was an unrecognized format -> bottom 'unknown' tier -> "Low Quality":
- source_map: map .dsf/.dff -> 'dsf' (also lights it up in AUDIO_EXTENSIONS, so Soulseek can
match a DSF if one exists)
- model.tier_score: 'dsf' base 102 (just above FLAC) — lands in the lossless range
- probe_audio_quality: add a DSD branch returning format='dsf' (mutagen.dsf for .dsf detail;
.dff classifies lossless without measured detail) instead of None
- settings UI: DSD in RT_LOSSLESS_FORMATS + a "DSD (DSF / DFF)" option in the profile dropdown
2) the actual cause of the screenshot's findings — the truncation guard falsely called DSF
"broken (only ~12% decodes)": ffmpeg decodes DSD to PCM at a different rate than the DSD
container's 2.8 MHz, so astats samples ÷ container-rate massively under-counts. now
detect_broken_audio skips the truncation check for DSD (silence detection still applies).
8 seam tests: dsf/dff -> 'dsf'; dsf tier in lossless range (with + without measured bitrate);
is_dsd_path; and a contrast pair proving the same 12%-decode numbers flag a .flac but skip a
.dsf. 230 quality/import/silence tests green, ruff + JS integrity clean.
ramonskie's thread finding: opening the Import page fires staging files + groups + hints
together, and each one independently os.walk'd the whole staging folder AND mutagen-read every
file's tags — 3x the directory walk and 3x the per-file tag I/O on every page open (the import
scan storm + memory spike on large staging folders).
they all need the same per-file tag data, so scan ONCE: _scan_staging_records walks staging and
reads each file's metadata a single time, returning per-file records that files/groups/hints all
derive from in-memory. a short TTL (6s) + a lock means the three near-simultaneous page-open
requests share one scan instead of each kicking off a full re-read; the lock also prevents
concurrent full scans. hints now derives from the same read_staging_file_metadata the other two
use (same underlying tags) instead of a separate read_tags pass. album/singles process drop the
cache on completion so the list updates immediately after files leave staging.
net: 1 walk + 1 tag-read-per-file on page open instead of 3. 2 tests (shared-scan: 3 endpoints =
1 read per file, not 3; hints updated to the shared reader) + autouse cache-clear fixture; 695
import/staging tests green.
reported (radoslav-orlov): the Stats page hangs ~20s on a 16GB/HDD box, GET /api/stats/cached
?range=7d -> 200 in ~20000ms, while it's instant on Boulder's SSD. the endpoint is documented
"instant" — it reads 3 small precomputed metadata blobs — but it then ran every top
artist/album/track image through normalize_image_url ON THE READ. that fixer calls
cache_url_for, which does a SQLite INSERT/UPDATE + commit under a global lock PER image. on an
HDD each commit fsyncs and contends with the background image fetcher -> ~20s for ~50 images.
(this is exactly the "image caching when we open a page" ramonskie flagged in the thread.)
fix: do the normalization in the background ListeningStatsWorker when it builds the cache
(_enrich_stats_items), so the cache stores browser-ready /api/image-cache/... URLs and the read
does ZERO per-image work. the read-path fixer stays as a cheap no-op (normalize_image_url early-
returns on already-proxied URLs), so an old raw-URL cache self-heals on the next rebuild with no
broken art in between. the registration writes now happen once per rebuild, off the hot path.
2 tests (existing enrich test relaxed to truthy since the value is normalized now; new
deterministic test stubs the fixer to prove every section's url is fixed at build time).
136 stats/listening/image tests green.
Docker report (HellRa1SeR, 2.7.8): pasting YouTube cookies threw
`ERROR: unsupported browser: "custom"`. the 'Paste cookies.txt' dropdown value is the sentinel
'custom', and youtube_client built `cookiesfrombrowser=('custom',)` from it — yt-dlp rejects
that since 'custom' isn't a browser. (server/Docker users have no local browser, so pasted
cookies.txt is the ONLY way to authenticate yt-dlp — e.g. for private 'Liked Music'.)
the correct precedence already existed in core.youtube_cookies.build_youtube_cookie_opts
('custom' -> cookiefile, browser name -> cookiesfrombrowser, falsy -> anonymous) and the web
layer used it — but youtube_client never got migrated and kept 5 inline cookiesfrombrowser
sites. route all of them through a shared _resolve_cookie_opts() that delegates to the tested
helper (reads youtube.cookies_browser + youtube.cookies_file, checks the file exists). also:
the no-cookies download retry now drops cookiefile too (not just cookiesfrombrowser), and
removed 3 now-dead config_manager imports.
3 regression tests (custom->cookiefile, browser unchanged, custom+missing-file->anonymous);
97 youtube tests green.
since 9a0e3b40 persisted completed downloads in the Downloads view, the Clear Completed button
was hidden for those rows and clear-completed only pruned live session tasks. after a restart
the page filled with persisted completed downloads with no way to clear them.
now Clear Completed clears BOTH:
- live session completed/failed tasks (clear_completed_local, unchanged), AND
- the persisted download-history tail: new clear_completed_download_history() deletes every
library_history event_type='download' row, so the list actually empties and stays empty.
this includes unverified rows (the verification review queue) by design: on a library where
verification never confirmed the imports, ALL completed downloads are 'unverified', so preserving
them made the button a no-op. it only removes HISTORY rows — the actual files and their tracks
entries are untouched, so nothing in the library is lost, only the 'needs verification' flags.
the action confirms first (showConfirmDialog, destructive) and the button now shows whenever any
completed/failed row is present.
3 seam tests (clears all incl unverified; leaves non-download history; empty=0); reconcile +
orphan + JS integrity suites green.
four fixes from the review (and a self-correction):
1) close the connection. reconcile_unverified_history_from_tracks opened a connection with no
finally/close. runs once per boot so GC reclaimed it, but now it's consistent + robust.
2) scope the tracks scan to the review queue. it built lookup dicts from EVERY verified/
human_verified track (~350k on a large library) on every boot while anything is unverified
(the normal state). now it loads the stuck rows first and skips verified tracks whose path
AND basename can't match any queued row, so dicts stay proportional to the queue, not the
library. behaviour identical (all 13 PR reconcile tests still pass).
3) close the title-less basename collision. a title-less history row fell back to filename-only
matching with no ambiguity check, so a generic name like "01 - Intro.flac" could heal a
DIFFERENT song to verified. now a title-less basename heal only fires when that basename is
unique among verified tracks; unique-basename rows still heal (recall preserved).
4) "Clean orphaned" protects force_imported rows (deliberate user decision, keep for human
approval) without weakening the mount-down safety gate. CRUCIAL self-correction: filtering
them out BEFORE the orphan check (my first cut) shrank the checked count below the threshold
and would have let a few unverified orphans be deleted during a mount outage. instead,
find_orphan_history_ids now takes a deletable predicate: protected rows still count toward
checked / all-missing (gate stays strong) but never enter the orphan_ids delete set.
3 new regression tests (title-less collision; deletable protects from delete; protected rows
still count toward the gate). 936 verification/acoustid/history/downloads tests green. builds
on nick2000713's #938.
clicking Download Discography → Add all to wishlist added ~1 track every 15-30s. trace: the
endpoint's per-track library-ownership check (track_already_owned → check_track_exists) ran the
LEGACY path — firing search_tracks for every title-variation × artist-variation, per track. on
a large library and an artist you own NOTHING of, STRATEGY-1 (indexed LIKE) always missed and
fell through to the fuzzy fallback (full-table scan), ~10-15 scans/track = the 15-30s. metadata
fetch was never the bottleneck (deezer returns each album in ~1s).
fix: pre-fetch the artist's owned tracks ONCE (get_candidate_albums_for_artist →
get_candidate_tracks_for_albums) and pass candidate_tracks to check_track_exists's batched
in-memory path — the same path the discography backfill job + completion-stream already use.
pass an EMPTY list (not None) when nothing is owned so the owns-nothing case still takes the
fast path → instant. per-track cost drops from ~20s to ~1ms.
safe: track_already_owned's only real caller is this endpoint; the new param is optional
(None = unchanged legacy behaviour for any other caller). normal ownership still detected (same
artist-variation breadth); the one divergence is a track owned ONLY via a compilation → a
harmless redundant wishlist add, which is the endpoint's explicitly-accepted failure mode and
already how the backfill job behaves. 4 new tests; 1134 discog/metadata/wishlist tests green.
#936 added fragmented-row grouping. two follow-ups found in review:
1) BUG: an excluded canonical sibling (a row pinned to the same canonical edition whose tracks
fail the strict fragment match) was emitted with canonical_items but no owned set, so
_build_missing_tracks flagged the ENTIRE tracklist as missing — including tracks the row
already owns (e.g. a 3-track fragment of a 12-track edition reported 12 missing, not 9).
now compute that sibling's own owned slots from its local tracks (shared
_owned_reference_for_tracks helper, same logic as the anchor) so it reports only what it's
actually missing and the count stays internally consistent.
2) PERF: _build_candidate_groups rescanned all albums for every canonical group — O(G*N), which
degrades badly once many editions are pinned (fine today at G=1, latent at scale). invert to
a single O(N) pass that assigns each row to its unique group; identical 'exactly one match'
semantics. also added a Stop/Pause check in _prepare_work_items, which now front-loads the
canonical lookups + matching.
3 new tests (excluded-sibling reports only its missing tracks; ambiguous candidate stays
independent; unambiguous candidate joins) — 17 completeness + 123 repair tests green.
root cause: the library stores album/artist art as media-server RELATIVE paths (Plex
/library/metadata/.., Jellyfin /Items/.., Navidrome /rest/..), which don't render in a browser
<img>. normal wishlist items carry Spotify CDN urls so they show fine, but LIBRARY-sourced
items — dead-file re-downloads and preview-clip re-fetches — carry the raw relative path, so
their album art came up blank. and the nebula only had artist photos for WATCHLISTED artists,
so non-watchlist orbs showed initials.
fix on READ in the wishlist tracks endpoint (so it also repairs items already in the wishlist,
no re-run needed), using the library data we already have:
- normalize each track's album.images url that needs it — relative/internal only, via the
canonical normalize_image_url; CDN urls are left untouched so already-rendering items can't
regress.
- build an artist-name -> normalized library-photo map and return it; the nebula seeds its
artist-image map from it (every wishlist artist), with curated watchlist photos overriding.
8 tests (predicate: relative/internal fixed, CDN untouched; album normalize in-place; artist
map build/skip-empty/idempotent/graceful). 237 wishlist+repair+JS tests green, ruff clean.
two field reports:
1) FIX-ALL skipped these findings — bulk_fix_findings() has a hardcoded fixable_types
allowlist that didn't include 'short_preview_track'. added it, so select-all/fix-all and
the per-page bulk-fix now cover this tool.
2) RE-WISHLISTED ITEM WAS ART-LESS — the payload pulled album art from the library thumb,
which is empty for un-enriched HiFi previews, so the wishlist orb showed initials (no album
OR artist image, since the orb falls back to album art). now the duration lookup also
captures the metadata source's CDN art from raw_data (spotify album.images / itunes
artworkUrl, upscaled) and stores it on the finding; the fix prefers that over the empty thumb.
3 new tests (art capture from spotify raw_data, itunes artwork upscale, fix uses finding art);
8 job tests + repair suite green.
The reconcile heals rows whose file is still in the library; it deliberately
leaves ORPHANS — history rows whose file is gone (deleted / replaced /
re-downloaded elsewhere). Those can never be healed (no file left to confirm)
and linger in the Unverified list forever. This adds an explicit, user-initiated
cleanup for them.
- core/downloads/orphan_history.py: pure, tested rule. A row is an orphan when
its file resolves nowhere; flags `suspicious` when EVERY reviewed file is
unreachable (the mount-down signature) so the caller refuses rather than
mass-delete a healthy log during an outage.
- POST /api/verification/clean-orphans (admin-only): runs it against
_resolve_history_audio_path (raw path -> prefix-swap resolver -> tracks-table
title fallback), refuses on the suspicious signature, and deletes only history
ROWS — never a file (the files are already gone).
- UI: "🧹 Clean orphaned" button in the Unverified bulk-actions row, with a
confirm dialog spelling out that it removes log rows only and refuses if the
library looks offline.
NEVER automatic / never at boot — a filesystem check during a mount outage would
otherwise wipe good history. 5 pure-rule tests + safety-gate coverage.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LWJk7EuM7YktQeNyqQwTZY
Second-pass audit of the startup reconcile found a real correctness bug in it:
the basename fallback had NO title guard, so a shared track-number filename
("01 - Intro.flac" in different albums) would heal the WRONG song — marking an
actually-unverified file 'verified' and silently dropping it from the review
queue. This is the exact collision the scan-time matcher (history_match.py)
guards against; the reconcile now mirrors it.
- basename match now requires the history row's title and the candidate track's
title to agree (alphanumeric-lowercase), only when BOTH are present (legacy
rows without a title still fall back to filename-only, like the matcher).
- exact-path matches stay unguarded (same path = same file, unambiguous).
- cheap early-out: skip the tracks scan entirely when no 'unverified' rows exist
(keeps the every-boot cost ~nil on healthy libraries).
3 new tests (collision must-not-heal, titles-agree heals, missing-title falls
back). 8 reconcile tests green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LWJk7EuM7YktQeNyqQwTZY
two field-reported issues: clicking the job sat at 'Starting…' with Scanned: 0 forever and
never showed the current track.
1) HANG — spotify get_track_details() defaults to allow_fallback=True, which scrapes the
configured metadata source when the official API isn't authed (HiFi users). that scrape is
slow and blocked the scan loop on the first track. now pass allow_fallback=False (official
only — fast, returns None cleanly) and fall through to iTunes/MusicBrainz.
2) NO LIVE UPDATE — progress was only pushed every 5 tracks via update_progress, never the
current item. now report_progress() every track (phase + 'artist — title' + scanned/total)
plus a start phase, so the UI moves and shows what it's checking.
also made the test track ids INTEGER to match production (tracks.id is INTEGER PRIMARY KEY),
exercising the real str(id) finding -> WHERE id=? round-trip. 5 tests green.
HiFi (and occasionally other) downloads sometimes deliver a ~30s preview clip instead of the
full song; it lands in the library looking real. new repair job scans short tracks (duration
<= 30s, configurable), looks up the EXPECTED length from the track's metadata source
(spotify/itunes/mb get_track_details), and flags any whose real length is much longer than the
file (default: >= 30s longer) as a preview clip.
approving the finding (repair_worker._fix_short_preview_track) deletes the preview file (path
resolved via _resolve_file_path like the other delete tools), drops the DB row so the track
goes missing, and re-adds it to the wishlist with the full payload (mirrors _fix_dead_file)
so the real version downloads. scan ONLY creates findings — nothing destructive without user
approval, like every other tool.
conservative: genuine short tracks (source agrees they're short) and tracks whose length can't
be verified are skipped, never flagged. registered the job + finding-type label/fix-button in
the UI. 5 tests (scan flag/skip/scope + fix delete+remove+wishlist); 89 repair tests green.
the duration-agreement check used abs() drift, so a file running LONGER than the metadata
(a remaster with a longer outro, an extended cut) was rejected the same as a truncated one —
e.g. A-Ha 'Take on Me' remaster at 228.5s vs 225.0s expected, quarantined for +3.5s.
but a longer file is the OPPOSITE of truncated. make the auto tolerance asymmetric: keep the
tight 3s/5s bound for SHORTER files (the truncation case the check exists for), allow up to
15s in the LONGER direction for version/master differences. a wrong song still trips it (off
by far more than 15s), and a user-pinned tolerance is honoured symmetrically. direction-aware
rejection message too. 4 new tests; 274 integrity/import tests green.
Complements the AcoustID-scan-time heal: re-links library_history rows still
showing 'unverified' to the verified/human_verified status their file already
carries in the tracks table — matching exact path AND basename, so a file that
moved (media-server import / reorganize) heals even though the stored history
path is frozen. Upgrade-only and non-destructive (no deletes, no bulk
migration).
Why this is needed on top of the scan-time fix:
- It clears the EXISTING backlog (e.g. 5551 rows) on the next restart with NO
re-fingerprinting and no AcoustID API calls — the file's status is already in
tracks from the prior scan; this just propagates it to the frozen history row.
- It covers human_verified files, which the AcoustID scan skips entirely
(file_verif_status == 'human_verified' returns early), so their stale history
rows would otherwise never heal.
Runs once on DB init (cheap, idempotent). 5 real-sqlite tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LWJk7EuM7YktQeNyqQwTZY
the AcoustID scanner matched library_history rows by EXACT file_path, but that path is
frozen at import time while the file moves afterward (media-server import / reorganize) —
so tracks.file_path (what the scan reads) no longer equals it. two failures resulted, both
introduced in 37ea6604: verified status never reached the history row (verified tracks kept
showing 'unverified'), and a fresh acoustid_scan row was INSERTed every run (5551 rows for
3675 songs).
- new pure, tested matcher (core/downloads/history_match.py): exact path → filename guarded
by title; prefers a real download row over a synthetic scan row.
- _persist_status now HEALS the matched row's path + status (so future scans match cleanly),
DELETES synthetic acoustid_scan duplicates by exact path (collision-free, never a real row),
and inserts only when the file genuinely has no row.
- a full AcoustID job now self-cleans existing duplicates — no destructive bulk migration.
8 matcher + 4 real-DB heal/dedup/insert tests; existing scanner tests updated to the new
seam (heal vs insert). 1076 acoustid/verification/download tests green.
save_watchlist_scan_run had a single caller — the manual scan endpoint. the automatic/
scheduled path (process_watchlist_scan_automatically) ran the full scan but never wrote a
history row, so nightly scans never showed up in the History modal — only manual ones.
- new shared helper persist_scan_run(database, state, ...) extracts the run from the
finished watchlist_scan_state and writes one history row
- the automatic path now stamps scan_run_id/scan_track_events and calls it
- the manual path is refactored onto the same helper so the two can't drift apart again
- history is global (no profile filter), so the all-profiles nightly scan records one
aggregate row (profile_id None → 1, never NULL)
tests: 4 new persist_scan_run seam tests (real DB) + 2 new auto-scan integration tests
proving the auto path actually records (completed + cancelled, exactly once). 420
watchlist/automation tests green.
a pasted track link IS resolved + searched, but the 'bubble the exact track to the top'
step read getattr(t,'id') — and TrackResult has no top-level id (the source id lives in
_source_metadata['track_id']). so the bubble was a silent no-op: the linked track sat buried
among fuzzy text-search lookalikes and the user saw unrelated tracks. qobuz made it worse —
_qobuz_to_track_result never stamped _source_metadata at all, so the track had no id to match.
- stamp _source_metadata={'source':'qobuz','track_id':...} on qobuz TrackResults (mirrors tidal)
- extract the bubble into pure, tested helpers (linked_track_id / bubble_linked_track_first)
that read _source_metadata['track_id'] — fixes it for tidal too, str/int-safe, stable no-op
19 track-link tests (+6 new) + 87 qobuz/download tests green.
_normalize_album_for_match stripped ANY trailing '- clause', so a real distinguishing
subtitle ('Clair Obscur: Expedition 33 - Nos vies en Lumière (Bonus Edition)') collapsed to
the same name as the OST → _albums_likely_match treated them as one album → the watchlist
marked unowned tracks of one edition as owned via the other and under-wishlisted.
- strip a trailing '- ...' clause ONLY when every token in it is an edition/format
qualifier (+ connectors / year-ordinal): '- Single', '- Acoustic Version', '- 2011
Remaster' still collapse, but real subtitles ('- Nos vies en Lumière', '- Volume 2',
'- Live in Berlin') are kept. Avoids the inverse regression (a same-album pair splitting
into a redownload loop), which a naive narrow strip would have caused.
- drop the loose substring shortcut + raise the fuzzy floor 0.6→0.85; genuine drift already
collapses to an EXACT match, so the looseness only ever produced false fuses.
blast radius: _albums_likely_match has exactly one caller (the allow-duplicates skip).
48 album-match tests pass (qualifier-suffix merges + edition-subtitle splits) + 219 watchlist.
the file faked spotipy + config.settings in sys.modules at import time with no teardown.
the fake config.settings had no ConfigManager, so depending on collection order it leaked
into tests/test_config_save_retry and intermittently failed the full suite. the real
modules import fine in the test env (spotipy is installed, config.settings has both
ConfigManager + config_manager), so the stubs were pure liability — removed them. album
tests still pass (10), the album+config combo that errored now passes (17), 573 repair/
config/canonical tests green.
#929 added 'musicbrainz' to library_reorganize._ALBUM_ID_COLUMNS but not to the
canonical layer, breaking the equality invariant test (canonical reads exactly what
reorganize reads). add musicbrainz to CANONICAL_ALBUM_SOURCES, and move it from the
'can't pin' param group to the 'pins' group in the manual-lock tests — now consistent,
and forward-compatible with pinning a deliberately-matched MB edition. inert at runtime
today (mb isn't in the manual source selector, so should_pin is never called with it).
540 canonical/reorganize tests green.
