resolve_history_audio_path drives a DESTRUCTIVE delete (os.remove), but lived
endpoint-bound in web_server with zero tests. Lifted to core/matching/history_paths
with injected effects (exists / resolve_library_path / lookup_titled_paths) so the
fallback chain — and the collision-safety that stops delete() from removing the
wrong same-title file — is a clean importable seam. web_server now wraps it (DB
lookup + os.path.exists + prefix resolver injected); behavior preserved.
9 tests lock it: recorded-path hit, prefix-resolve fallback, single tracks-table
candidate, and the safety rules — multiple same-title candidates with NO artist ->
None (refuse to guess), artist filter picks only the matching path, artist named
but unmatched -> None, no-title/empty-lookup -> None. Full suite green (5906).
The candidate matcher rejected valid downloads of titles with a '/' or ':' (e.g.
Sawano's "You See Big Girl / T:T") because the unified normalize() removed those
chars ("t:t" -> "tt") while keeping the '_' that source filenames substitute for
them ("T_T" -> "t_t") — the asymmetry tanked the similarity score. Now '/ \ : _'
all map to spaces before the strip, so "/ T:T" and "_ T_T" both normalize to "t t".
Verified on the real library: similarity for the Sawano pair 0.927 -> 1.000; only
348/40786 strings (0.85%, all containing those separators) change; worst-case
joined-variant match (e.g. "12:05" vs "1205") stays 0.889, well above the 0.70
title threshold — no match regressions. Fixes the matching half of #851.
The merged PR left the review-queue's mutating endpoints ungated. Both now require
admin, matching the Phase 3 destructive-endpoint convention:
- /api/verification/<id>/delete (os.remove + drops the history row) — @admin_only,
so a non-admin on a login/multi-profile instance can't delete library files.
- /api/verification/<id>/approve (flips verification_status + writes the tag) —
@admin_only; also wrapped its DB writes in `with db._get_connection()` for
rollback-on-error + codebase consistency (was a bare conn).
Read/playback endpoints (stream/play/compare/entry/config) stay open — the app's
LAN-read model. Tests: non-admin gets 403 on delete + approve; admin isn't blocked.
The Your Albums Discogs collection sync stored bare release_ids while
search/discography now store tagged ('r<id>') ones (#848). This didn't cause a live
bug — the pool dedups by normalized name, and discogs_release_id is only ever
re-fetched (which handles bare via release-first) — but it left the "type travels
with the ID" invariant half-applied. Now the collection sync tags its IDs too, so
every stored Discogs album ID is uniform and a future ID comparison can't be tripped
by mixed forms.
Collection items are always releases, so they're tagged 'r<id>'. Test locks the
stored value + that a tagged collection ID routes only to /releases (never /masters).
Closes the forgot-login-password gap. A per-profile recovery question + answer lets
a locked-out user reset their own password.
- DB: additive recovery_question + recovery_answer_hash columns (idempotent
migration). set/get-question/verify/has methods; answer is hashed (pbkdf2) and
matched forgivingly (trim + lowercase + collapse whitespace). No recovery set →
never verifies.
- Endpoints (allowlisted in the login gate so they work pre-auth):
GET /api/auth/recovery-question?username= (generic 404 when absent),
POST /api/auth/recovery-reset {username, answer, new_password} — brute-force
limited; a correct answer sets the new password + authenticates the session.
POST /api/profiles/<id>/set-recovery (admin or self) to configure it.
Tests: set/get/verify, forgiving match, hashed-not-plaintext, no-recovery-never-
verifies, full reset flow (wrong answer rejected + password intact; correct answer
resets), unknown-user 404. 25 tests pass. Next: the Settings + login-screen UI.
Pins the zero-impact guarantee: with require_login off (default), the launch PIN
still enforces exactly as before (the login deferral doesn't fire), and with both
off there's no gate at all (today's behavior).
The UI that makes opt-in login usable. Off by default → your LAN setup is unchanged
(none of this appears unless security.require_login is on).
- Login screen overlay (reuses the launch-PIN styling): username + password →
/api/auth/login → reload into the app. Shown when /api/profiles/current reports
login_required (checked before profile selection).
- POST /api/profiles/<id>/set-password (admin, or self) to set/clear a login
password, distinct from the PIN.
- Settings → Security: "Login password (admin account)" field + a "Require login"
toggle (with the anti-lockout note). Wired into the existing settings load/save.
- Sign-out button in the profile bar, revealed only in login mode (login_mode flag
on /api/profiles/current); soulsyncLogout() → /api/auth/logout → reload.
Tests: set-password sets/clears + verifies; /api/profiles/current signals
login_required. 20 login/password tests pass; 64 script-split integrity pass.
Remaining (small follow-up): a password field in the Manage Profiles edit form so
admins can set OTHER profiles' passwords from the UI (the endpoint already exists).
The backend auth for opt-in username/password mode (security.require_login, default
off → zero change; the launch PIN + picker behave exactly as today).
- core/security/login_gate.py: pure gate (mirrors launch_lock) — when login mode is
on, an unauthenticated session reaches only the page shell, /api/auth/login,
/api/auth/logout, /api/profiles/current, /api/setup/status, and the key-authed
/api/v1 API. Deliberately does NOT expose the profile list pre-auth (you type your
name, not pick from a roster).
- _enforce_login before_request enforces it; _enforce_launch_pin no-ops when login
mode is on (login replaces the shared PIN, per design).
- POST /api/auth/login (username = profile name, case-insensitive; brute-force
limited per IP; generic error so names don't leak) + POST /api/auth/logout.
- Anti-lockout: the settings save refuses to turn ON login mode until the admin
account has a password.
Tests: gate blocks→login→access→logout→blocked; case-insensitive username; wrong
password / passwordless profile / unknown user all 401 generically; login list not
exposed pre-auth; can't enable login without an admin password. 12 tests pass. Next:
the login screen + set-password UI + the toggle (increment 3).
Opt-in username/password login — profiles become real accounts. This is the data
layer: a per-profile login password, kept SEPARATE from the quick-switch PIN
(different security purpose; a 4-digit PIN must not become the password guarding a
public instance).
- Additive migration: profiles.password_hash column (idempotent, metadata-flagged).
- set_profile_password / verify_profile_password / profile_has_password /
get_profile_by_name (the login username = profile name, unique + case-insensitive).
- Security default: a profile with NO password is NOT loginable (verify returns
False) — unlike the PIN where "no PIN = always valid". You can't authenticate to
an account with no credential.
Tests: migration adds the column; set/verify; no-password-never-loginable; clearing;
name lookup; and password is fully independent of the PIN. 6 tests pass. Next:
the login endpoint + require_login gate (increment 2).
Lets SoulSync sit behind Authelia/Authentik/oauth2-proxy as the gatekeeper: when
security.auth_proxy_header names a header (e.g. Remote-User), a request carrying it
is treated as already-authenticated and passes the launch lock — the proxy did the
login (with 2FA).
- core/security/auth_proxy.py: trusted_proxy_user(get_header, header_name) — returns
the user iff the configured header is present + non-empty; empty header name (the
default) → always None → feature off.
- _enforce_launch_pin ORs it into pin_verified. OFF by default, so a direct install
is unaffected AND a client-spoofed header does nothing unless the operator opted in.
- Doc'd in Support/REVERSE-PROXY.md with the must-strip-client-headers warning.
This is the lightweight Tier 3 (auth-proxy integration), not a full per-user login —
the proxy owns identity; SoulSync trusts it.
Tests: helper off/on/blank/exception-safe; integration — trusted header passes the
gate, no header is locked, and (the safety pin) a spoofed header is IGNORED when the
feature is off. 6 tests pass.
Fold a conservative security-header set into the SAME opt-in proxy mode, so it's
zero-impact when off. When security.trust_reverse_proxy is on, an after_request
adds X-Content-Type-Options: nosniff, X-Frame-Options: SAMEORIGIN, and HSTS
(safe — only honoured over the proxy's HTTPS), via setdefault so it never clobbers
a header the proxy already set. No CSP (needs per-deploy tuning; better at the
proxy). When OFF (default), the after_request isn't registered → no headers added.
Tests: off adds none of the headers; on adds all three. Doc updated. 6 tests pass.
A publicly-exposed instance gated only by the launch PIN was brute-forceable. Added
a lenient in-memory failed-attempt limiter (core/security/rate_limit.py): 10 wrong
PINs from one IP within 5 min → 429 with Retry-After, failures age out on their own
(self-heal, no persistent lockout), and a CORRECT entry clears that IP instantly.
Wired into /api/profiles/verify-launch-pin. By design it can only ever trigger on a
flood of WRONG PINs — correct entry, a couple of typos, or a no-PIN install are
never affected, so normal use sees no change. Keyed per-IP so an attacker can't
lock out a legit user.
Tests: limiter is lenient under threshold, trips on a flood, success clears it,
failures self-heal, per-IP isolation; endpoint returns 429 after 10 wrong PINs with
Retry-After. 6 tests pass.
Tier 1 of "secure behind a reverse proxy". STRICTLY opt-in so direct/LAN installs
are byte-for-byte unchanged.
- core/security/reverse_proxy.py: apply_reverse_proxy_mode(app, config_get) — a
no-op unless security.trust_reverse_proxy=true. When OFF (default), the app is
untouched: no ProxyFix, X-Forwarded-* stays UNtrusted (a direct client can't
spoof its IP/scheme), session cookie keeps Flask defaults. When ON (operator is
behind nginx/Caddy/Traefik with TLS): trust one proxy hop's X-Forwarded-*, and
mark the session cookie Secure + SameSite=Lax. Any config error → safe no-op,
never breaks startup.
- Wired once at app init.
- Support/REVERSE-PROXY.md: nginx (with the Socket.IO Upgrade headers people
always miss) / Caddy / Traefik configs, the setting, and the "put auth in front
(Authelia/Authentik/oauth2-proxy)" recommendation + the off-for-plain-HTTP note.
Tests: off (and missing-key, and a config exception) is a strict no-op — not
ProxyFix-wrapped, cookie defaults intact; on wraps ProxyFix + secures the cookie;
and the real web_server app is NOT in proxy mode by default. 5 tests pass.
Per the original intent, "Sync" is now a single-artist deep scan: it uses the SAME
reconciliation source as the whole-library deep scan instead of a separate
disk-existence check.
- Phase 1 already calls the deep-scan worker's _process_artist_with_content; now it
passes seen_track_ids so the pull collects the server's current track IDs for the
artist (existing + new), exactly as the library deep scan does.
- Phase 2 stale = (artist's DB tracks for this server) − seen, then
delete_stale_tracks(server_source) — identical mechanism to deep scan, scoped to
one artist. The old os.path.exists disk check (which could mass-delete on an
unreachable mount) is gone.
- Removal only runs when the server pull SUCCEEDED — no trustworthy 'seen' set
(no server, unreachable, or a failed pull) → skip, never delete. The
is_implausible_stale_removal guard (>50% unseen) stays as the same safety net
deep scan has for a flaky response. @admin_only retained.
Tests rewritten for the server-diff model: removes only tracks the server no longer
has; guard skips when most are unseen; a failed pull skips removal entirely;
admin-only. 8 tests pass.
The enhanced-tab "Sync" button's stale-removal phase deleted any track whose file
wasn't on disk, with NO guard — so if the music storage was momentarily
unavailable (sleeping NAS, dropped mount, unmounted Docker volume, WSL hiccup),
os.path.exists returned False for EVERY file and one click wiped the whole artist
(tracks + their now-"empty" albums) from the DB. The deep-scan path already had a
50%-stale safety net (#828); this endpoint never got one.
- New core/library/stale_guard.py: is_implausible_stale_removal(missing, total) —
a tested rule (skip removal when missing > 50% of a >=5-track set), centralised
so every stale-removal site can share it.
- sync_artist_library: if the guard trips, SKIP removal (delete nothing), return
removal_skipped + warn; the frontend shows "storage may be offline — skipped"
instead of silently deleting. Empty-album cleanup now also only runs on the
non-skipped path and uses `album_id IS NOT NULL` (fixes the NOT IN-with-NULL
no-op). Frontend also refreshes the view on additions, not just removals.
- @admin_only on the endpoint — it deletes tracks + albums but was ungated, while
the sibling delete_album endpoint is gated.
Deep scan was already safe (different mechanism: server-diff + its own 50% guard).
Tests: guard unit rules; endpoint skips removal when all files missing (keeps the
tracks), removes only the genuinely-gone few otherwise, and 403s for non-admins.
7 new tests pass.
The #843 fallback saved the discovery-cache match using the client's
original_artist verbatim — but the client sends a joined "A, B, C" string, while
EVERY in-memory + sync path keys the cache by the first artist (artists[0]). So a
multi-artist fix (the reporter's exact "Cherrymoon Traxx, Hermol, SBM, BELS"
case) would have saved under a key the sync never looks up — the fix would
"succeed" with no error but silently never apply.
Reduce the client artist to the first (split on comma) in the no-state branch so
its cache key matches the in-memory/sync convention exactly. Single-artist tracks
are unaffected.
Test: no-state save now keys by the first artist, and a new test pins that the
no-state and in-memory paths produce an IDENTICAL cache key for the same
multi-artist track. 74 discovery tests pass.
The discovery FIX → Confirm flow 404'd with "Discovery state not found" whenever
the in-memory discovery state was gone — a server restart, or an imported
playlist that wasn't discovered in THIS process — even though the card is still
shown from persisted data (the reporter's log shows "Returning 0 stored ...
playlists for hydration", i.e. the in-memory states were empty).
The thing that actually makes a manual fix STICK is writing it to the discovery
cache (save_discovery_cache_match), keyed by the original track's name + artist —
which doesn't need the in-memory state at all. But the endpoint 404'd on the
missing state before reaching that write, so the fix was dead after a restart.
- update_discovery_match (core/discovery/endpoints.py) now only does the in-memory
result update when the state exists; the durable discovery-cache write always
runs, falling back to client-provided original_name/original_artist when there's
no in-memory state. With neither a state nor originals it still 404s (unchanged).
- The FIX confirm (wishlist-tools.js) now sends original_name/original_artist
(from the source track it already has) so the backend can key the cache.
Covers all sources that share the helper (tidal/deezer/qobuz/spotify-public).
Tests: no-state-but-originals saves the cache + returns success; no-state-no-
originals still 404s; existing with-state path unchanged. 73 discovery tests pass.
Regression in 2.6.9. The spotify_public source adapter (used by auto-sync /
refresh_mirrored) called scrape_spotify_embed() directly — the embed widget only
exposes ~100 tracks — instead of fetch_spotify_public(), the wrapper the rest of
the app uses, which pulls the full list via the paginated public API and only
falls back to the embed on failure. So initial discovery got the whole playlist
but every auto-sync re-fetch truncated it to 100.
Switched the adapter to fetch_spotify_public (same return shape — drop-in). Albums
still resolve via the embed (already whole); on any failure it falls back to the
embed exactly as before.
Test: the adapter returns all 150 tracks when the full fetch yields 150 (was
capped at 100). 29 adapter tests pass.
A wishlist track (or tracks in an album) that slskd accepted then REJECTED would
sit at "DOWNLOADING... 0%" indefinitely, spam an ERROR every status poll
("…Completed, Rejected - letting monitor handle retry"), and — for albums —
block the whole batch from ever completing.
Root defect: the status formatter's non-manual error branch keeps the task
'downloading' and trusts the retry monitor to resolve it, with NO backstop. When
the monitor can't make progress (a rejected transfer with no other source), the
task is stuck forever.
Backstop: measure how long the ERROR state has persisted (keyed off the task's
last status transition, so a slow-but-healthy transfer is never failed, and each
monitor-retry episode gets a fresh window). Once it exceeds the monitor's retry
window (60s, vs the monitor's ~15s) with no resolution, mark the task failed and
fire the worker-freeing completion callback so the batch can finish. Also log the
error ONCE per episode instead of every 2s poll. The healthy path is untouched —
a working retry transitions the task before the grace, so this never fires.
Manual picks still fail immediately (unchanged).
Tests: rejected-within-grace stays downloading; rejected-beyond-grace fails +
schedules completion; manual pick fails immediately. 45 status tests pass.
Reported via Find & Add (Billie Eilish "bad guy"): the track was in the library
and on Plex, but never showed in the modal's 20 results. Root cause (proven
against the real 307k-track DB): the search did `ORDER BY tracks.title`, which is
case-SENSITIVE in SQLite (BINARY collation sorts 'B' before 'b'). Billie's title
is lowercase "bad guy"; everyone else's is "Bad Guy", so all the capitalised ones
sorted first, filled the LIMIT, and her exact match landed at ~#25 — cut off.
- search_tracks now ranks by relevance: exact title match first (case-insensitive
via unidecode_lower), then prefix, then alphabetical — so an exact match can't
be sorted below the limit by a capital letter. Helps every caller.
- Added a rank-only `rank_artist` hint (never filters): Find & Add already knows
the source track's artist, so it now passes it and the exact title+artist match
floats to #1. Filtering was deliberately avoided — if the track is tagged under
a slightly different artist on the server, a filter would re-hide it.
Verified on the real DB: title-only "bad guy" now surfaces Billie at #4 (was
>#20); with the artist hint she's #1. Seam tests: lowercase exact title isn't
buried; rank hint floats the match without filtering; exact title beats a
superstring title. 10 tests pass.
Automations + auto-sync respect 'append' mode and preserve a server playlist's
description + cover image, but manually matching a missing track ("Find & add")
recreated the whole playlist and wiped them.
Root cause: the add-track endpoint's Jellyfin branch called
`update_playlist(<entire track list>)`, which deletes + recreates the playlist on
Jellyfin/Emby. Switched it to the purpose-built `append_to_playlist([the one
found track])` — the same in-place, dedupe-safe op the 'append' sync mode already
uses — so the playlist (and its description/image) is preserved and only the
missing track is added. append_to_playlist reads `.id` off the track, so the
endpoint now sets it (it previously only set ratingKey).
Plex (in-place addItems) and Navidrome (in-place Subsonic updatePlaylist) were
already non-destructive; Emby routes through the jellyfin branch, so this covers
it too.
Tests: the add-track endpoint appends in place and never calls update_playlist;
a link-to-existing-track touches nothing. 18 tests pass (incl. the existing
append-mode suite).
YouTube/Tidal/Qobuz results encode the name as ``id||title``. When the title
itself contains a '/' (e.g. the Sawano AoT track "YouSeeBIGGIRL/T:T"), two places
wrongly basename-split it on the slash and kept only the last segment ("T:T"):
- core/downloads/file_finder.py — the completed-download finder truncated its
search target to "T:T", so the real on-disk file (slash sanitised by the
writer) never matched → "not found after processing" → the download got
QUARANTINED. Now an encoded ``id||title`` keeps the whole title as the target
and contributes no remote-directory components; real Soulseek PATHS still get
basename + dir extraction unchanged.
- webui/static/downloads.js — the manual-search FILE column showed only "T:T".
Added a ``||``-aware short-label helper (mirrors the correct handling already
used elsewhere in the file); real file paths still show their basename.
Tests: the finder locates "YouSeeBIGGIRL∕T: T.mp3" from the encoded title
"…||YouSeeBIGGIRL/T:T" (the screenshot case), doesn't match an unrelated file,
and a genuine Soulseek path still resolves to its last segment. 21 finder tests
+ 64 script-split integrity tests pass.
- ⚠ Unverified filter rows gain actions: inline play (range-streamed from the
history file path, server-side only), YouTube compare, Approve -> new
human_verified status (tag + history + tracks; AcoustID scanner skips these
entirely), Delete (file + entry)
- API: /api/verification/<id>/stream|approve|delete (path only from DB row)
- backfill: history rows with acoustid_result='fail' that exist at all were
imported despite the failure = force_imported (covers pre-fix fallback
imports like the user's 'My Ordinary Life')
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Also: evaluate() treats an empty expected artist as title-only comparison
(old scanner behaviour — a missing DB artist is no evidence of a wrong file),
and the thresholds are now defined once in the core and re-exported.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The library AcoustID scan now calls audio_verification.evaluate() (alias-aware
artist match + cross-script SKIP) instead of its own non-ASCII-stripping
_normalize and threshold logic, so it no longer false-flags correct anime-OST /
kanji tracks. Duration-collision guard kept as a scanner pre-check on the top
recording. evaluate() is now purely a title/artist/version/cross-script decision.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
verify_audio_file now calls audio_verification.evaluate() and re-exports
normalize/similarity/_alias_aware_artist_sim from the core, so import and the
library scan can no longer drift apart. Alias-rescue diagnostic moved to the core.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
assert against a monkeypatched get_spotify_client sentinel instead of
web_server.spotify_client (which isn't a stable singleton across the suite) — same
fix already used for the per-profile builder tests.
The playlist source registry built the Spotify/Tidal adapters with a client
GETTER (resolved fresh on every read), but web_server passed `lambda: <global
client>`. Swapped those to get_spotify_client_for_profile /
get_tidal_client_for_profile.
Combined with part 1 (the engine running each automation as its owner), an
auto-sync pipeline now reads its source playlist through the OWNER's account:
- interactive sync → the user's session profile,
- background automation → the automation owner (via core.profile_context),
- admin / profile 1 → the global client, so the admin's existing auto-sync
pipelines pull exactly as before.
The adapters re-resolve per read, so a singleton registry is fine. Deezer/Qobuz
getters left global (their playlist login is tangled with downloads — deferred).
Tests: the Spotify/Tidal source adapters resolve the global client under admin
and re-resolve through the profile context per call (unconnected → safe global
fallback). 27 endpoint/profile tests pass.
Background automations had no session, so get_current_profile_id() fell back to
admin (1) — wrong for a non-admin's scheduled job. Now the engine declares the
automation's owner around handler execution via a contextvar
(core/profile_context.py), and get_current_profile_id() consults it only when
there's NO web request. So:
- a real logged-in request always wins (foreground unchanged),
- admin + system automations are profile 1 → resolve to admin exactly as before
(the 8 admin-owned auto-sync pipelines behave identically),
- only non-admin-owned automations gain their correct identity, deep through the
whole call chain (incl. the per-profile client resolvers) — no threading
profile_id through dozens of signatures.
Reset in a finally so a pooled thread can't leak the override to the next job.
Tests: contextvar set/reset/nested; get_current_profile_id honours the override
only outside a request (a real session still wins); and end-to-end — the engine
runs a non-admin automation as profile 4, an admin one as 1, an explicit trigger
profile overrides the owner, and the context resets even when the handler raises.
27 + 4 tests pass.
Part 2 (next): point the sync handlers' source-playlist READ at
get_spotify_client_for_profile so a non-admin's auto-sync pulls THEIR playlist.
My credential/admin-gating endpoint tests override DATABASE_PATH at import to a
temp dir, but with a custom prefix — which tripped the DB-isolation guard
(test_database_path_env_is_isolated requires 'soulsync-testdb-' in the path) once
those modules loaded. Still a temp DB (no real-DB risk), just the wrong prefix.
Switched to the soulsync-testdb- prefix so isolation is preserved and the guard
passes.
Third service (the easy one — ListenBrainz already had a working per-profile
token path). Consolidated all per-profile streaming accounts into the My Accounts
modal:
- My Accounts gains a ListenBrainz row with a token-paste connect (a new 'token'
service type alongside the OAuth-popup ones), reusing the existing
/api/profiles/me/listenbrainz save + the generic disconnect.
- Connections API reports listenbrainz status (connected + username).
- Personal Settings (the gear modal) dropped its Spotify/Tidal/ListenBrainz
sections — those duplicated My Accounts — and now shows only the per-profile
server-library selection (non-admin) or a pointer note (admin). The old
renderPersonalSettings{Spotify,Tidal,LB} functions are left defined but unused.
So every per-profile account connection (Spotify, Tidal, ListenBrainz) now lives
in one place. Tests: LB connect status + disconnect via the generic endpoint.
23 endpoint tests pass; 64 integrity tests pass.
Second service. Each profile connects its own Tidal; its playlist reads use that
account, everything else stays global. The gotcha vs Spotify: TidalClient loads
AND saves tokens to one global slot (tidal_tokens), so a naive per-profile client
would clobber the admin's tokens on refresh.
- get_tidal_client_for_profile builds a dedicated TidalClient seeded with the
profile's tokens, refreshed via the shared/global app creds, and OVERRIDES its
_save_tokens to persist to the PROFILE row — never the global slot. Admin
(profile 1) + unconnected profiles use the global client unchanged. Cached per
profile + evicted on (dis)connect.
- DB: set_profile_tidal_tokens / get_profile_tidal (encrypted); the OAuth callback
now uses them + evicts the cached client.
- Wired the Tidal playlist reads (list + tracks) to the per-profile client; the
module import line left intact.
- My Accounts: Tidal row (Connect via /auth/tidal?profile_id=, status, Disconnect).
Connections API extended; disconnect made generic (/<service>/disconnect).
Admin sees "managed in Settings" for every service.
Tests: per-profile token refresh writes to the profile and leaves the global
tidal_tokens untouched (the safety guarantee); connect status + disconnect;
admin/unconnected → global client. 22 endpoint tests pass.
Review/regression fix: the shared-app gate I added wrongly required a token
cache even for profiles that set their OWN app creds (legacy), breaking the
per-profile caching tests. Now a per-profile client is built when the profile
has its own app creds OR has connected (its own token cache) — otherwise it
falls back to the global/admin client. Made the fallback-safety tests
order-independent (monkeypatch get_spotify_client to a sentinel) since the real
global client isn't a stable singleton across the suite.
10 metadata-cache + resolution tests pass together.
First service of the per-profile playlist-auth feature. Each profile connects
its OWN Spotify account through the shared (admin's) app, getting its own token;
used for that profile's playlist reads. Admin + unconnected profiles + all
background workers keep using the global/admin client — fully non-regressive.
- Shared-app OAuth: get_spotify_client_for_profile + the /auth/spotify init &
callback now use the GLOBAL app creds (falling back from any legacy per-profile
app creds) with the profile's own token cache, and show_dialog=true forces the
account chooser so a user can't silently inherit the admin's Spotify session.
The builder gates on the profile's own token cache existing — no cache → global.
- My Accounts modal (new, all-profile-accessible via the profile bar): one-click
Connect/Disconnect Spotify + connection status (account name). GET
/api/profiles/me/connections + POST .../spotify/disconnect; admin's Spotify is
read-only here (managed in Settings).
- Wired the request-scoped reads to the per-profile client: the playlist LIST,
the playlist TRACKS view, liked-songs count, and user info — so a connected
user sees and opens THEIR OWN (incl. private) playlists, not the admin's.
Tests: builder falls back to the global client for admin/None/unconnected (the
non-regression guarantee); connections status reports unconnected; admin
disconnect rejected. 124 profile/spotify/gate/integrity tests pass.
Still on the global account (next step): sync/download jobs run in background
workers with no profile context — stamping the requesting profile onto the job
is the remaining wiring. Other services (Tidal/Deezer/Qobuz/Last.fm/ListenBrainz)
follow this same pattern.
The model shifted from "admin creates shared credential sets, users pick" to
"each profile self-auths its own playlist accounts". Removed the admin-facing
Connected Accounts manager: the Settings section, credential-sets.js, its CSS,
the script tag + integrity-registry entry, and the loadSettingsData hook.
The credential-sets backend (service_credentials tables + /api/credentials and
/api/profiles/me/services endpoints) is left in place but dormant — additive,
tested, harmless — rather than churn migrations that already ran on installs.
Per-profile self-auth reuses the existing per-profile columns + the
get_*_for_profile client pattern instead. The Service Status modal (admin-only)
is unaffected.
Regression from the #832 server-side launch gate (Beckid). On a PIN-required,
unverified session the gate 401'd /api/setup/status — which the frontend checks
BEFORE the PIN screen. The 401 left setup_complete undefined, so `!undefined`
relaunched the full setup wizard on every visit (cancel → PIN screen worked,
which was the tell).
Two-layer fix:
- Allowlist /api/setup/status in the launch gate (it's a harmless boolean, no
secrets) so the frontend gets the real answer even while locked.
- Make the frontend fail-safe: only launch the wizard on a definitive
setup_complete === false from an OK response — never on a 401/locked/ambiguous
one.
Test: locked session still 401s data endpoints but /api/setup/status returns
{setup_complete:true}; added a gate-allowlist assertion. 21 gate tests pass.
Correctness (the modal was lying): "Spotify (no auth)" is a COMPOSITE the
Settings page stores as fallback_source='spotify' + metadata.spotify_free=true,
not a literal 'spotify_free' value. The modal read the raw fallback_source and
showed plain "Spotify" as active even when Settings clearly said "(no auth)".
The endpoint now mirrors that mapping both ways — reports active='spotify_free'
when the flag is set, and switching to it writes fallback_source=spotify +
spotify_free=true (and clears the flag for any other source). Modal + Settings
now always agree.
Visual: the modal itself (not just the cards) is richer now —
- a hero header per tab: big brand-logo disc + "Active <kind> source" eyebrow +
the active name + a one-liner + an Active pill, all tinted by the brand color
with a soft radial glow (the Manage-Workers hero feel);
- the panel gained brand-tinted radial depth instead of flat black.
Test: spotify_free composite round-trips like Settings (stored split + reported
as spotify_free; flag clears on switch). 15 endpoint + 64 integrity tests pass.
Replaces the basic credential-pill quick-switch with a Manage-Workers-styled
modal (topbar + left rail + panel, entrance animation, brand-logo cards).
- Sidebar Service Status: whole panel opens the modal; clicking the Metadata /
Media Server / Download rows deep-links straight to that tab. Removed the
"switch ▸" hover text.
- Three tabs: Metadata (source logo cards, unavailable ones dimmed), Server
(Plex/Jellyfin/Navidrome/SoulSync logos), Download (Single⇄Hybrid segmented
toggle; Hybrid shows a draggable priority list). Logos reuse SOURCE_LABELS +
HYBRID_SOURCES; active card gets an accent ring + check.
- Admin writes the GLOBAL active source/server/download (reuses the same setters
+ client reloads as the Settings save, so changes take effect immediately).
Non-admins see it read-only (editable=false) — the per-profile override is the
next layer.
Backend: GET /api/profiles/me/active-sources (any profile; reports editable),
POST /api/profiles/active-sources (@admin_only; validates against the allowed
metadata/server/download lists, applies + reloads). New service-switch.js
(registered + in the integrity registry); old modal removed from
credential-sets.js (admin Connected Accounts manager stays).
Tests: 14 endpoint tests — read shape, admin sets metadata/hybrid+order
(reflected), bad-value 400s, non-admin read-only + 403 on write. 64 integrity
tests pass; real-app smoke confirms render + deep-links + the full set/reflect
cycle.
Frontend for the credential-set feature, matching the blocklist/house modal
style. Functional end to end against the existing endpoints; visuals are a
clean first pass to refine.
Admin manager (Settings → Connected Accounts, admin-only — empty for non-admins):
per service, the saved accounts render as pills with a delete ✕, and "+ Add
account" reveals an inline form built from each service's required fields.
Create POSTs /api/credentials; secrets are entered but never read back (the API
only returns id/label). Loads via loadCredentialSets() at the end of
loadSettingsData().
Quick-switch modal (sidebar Service Status is now clickable for ALL profiles):
shows, per service the admin set up, a "Default" pill + one pill per account,
highlighting the profile's current choice; clicking persists via
/api/profiles/me/services/select and re-renders. Empty-state message when the
admin hasn't configured any alternates.
webui/static/credential-sets.js (new, registered in index.html), house-style
CSS appended, sidebar made clickable, settings hook added. Registered the new
module in the script-split integrity test (onclick coverage). 64 integrity
tests pass; real-app smoke confirms index renders, the asset serves, and
admin-create → per-profile-list round-trips.
Note: selections are stored but not yet consumed by the live clients (the
resolver remains dormant) — wiring playlist-pull/enrichment to use a profile's
selected account is the next step.
Discogs masters (/masters/{id}) and releases (/releases/{id}) share one
numeric ID namespace, so release N and master N are different albums.
search_albums() (type=release) yields RELEASE ids, but get_album,
get_album_tracks and _fetch_and_cache_album all tried /masters first and
only fell back to /releases. Whenever a release id collided with a real
master id, the master lookup succeeded and returned a valid-but-wrong
album — the fallback never fired.
Tag the object type into the id string ('r12345' / 'm12345') at the single
parse point (Album.from_discogs_release) and route each fetch to the
matching endpoint. Legacy untagged ids are tried release-first (then master),
which self-heals pre-existing bad matches without a migration. The
enrichment worker now persists the tagged id too.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adversarial line-by-line review of the feature diff turned up:
- /api/database/update/stop was NOT @admin_only while its sibling start_update
was — a non-admin could abort a library scan. Gated.
- /api/metadata-cache/evict was NOT gated while its clear siblings were. Gated.
- validate_credential_payload now treats whitespace-only values as missing, so
a blank-but-spacey secret can't be saved to fail confusingly later.
Tests updated: both endpoints added to the admin-gating matrix; a whitespace-only
validation case added. 42 credential/gating tests pass.
Review also confirmed (no change needed): migration is idempotent + additive +
O(1); encryption round-trips with a non-dict guard; no SQL injection; stale
selections fall back to None safely; no secret ever returned to the browser;
the hybrid-drag index math is correct in both directions; the new resolver is
fully DORMANT (zero runtime callers) so existing client behaviour is untouched;
and @admin_only is a no-op for single-profile installs (default profile = admin).
Lets any profile pick which admin-created credential set is active for it,
without creating/seeing secrets:
- GET /api/profiles/me/services per-service options (id+label only) +
this profile's selected_id (stale-safe)
- POST /api/profiles/me/services/select {service, credential_id|null}
Not admin-gated by design — it only writes a per-profile pointer and exposes no
secrets. Validates the chosen set exists AND belongs to that service (can't
select a tidal set under spotify), and rejects unsupported services. null clears
back to the global/admin default.
Tests: a non-admin reads options + selects + clears (no secret in the response),
and selection rejects wrong-service / nonexistent / unsupported. 10 endpoint
tests total.
The audit found these were UI-hidden but API-open — any profile (or the
anonymous default-admin) could call them directly. Added @admin_only to the 15
that mutate SHARED/global state:
- DB: update, backup, backups DELETE, restore, vacuum
- library: track DELETE, album DELETE, tracks delete-batch, clear-match
- plex clear-library; metadata-cache clear + clear-musicbrainz
- internal API keys: list, generate, revoke
Deliberately NOT gated: profile-scoped own-data ops like /api/wishlist/clear
(clears the caller's OWN wishlist via profile_id) — gating that would wrongly
block a non-admin from managing their own data. Verified by test.
Zero change for single-profile installs (the default profile IS admin), so
existing users are unaffected; only genuine non-admin profiles get 403.
Tests: non-admin → 403 on all 15 (the 403 fires before the view body, so no
destructive op runs); admin not blocked on the read-only one; wishlist/clear
stays open to non-admins (over-gating guard). 17 tests.
Admin-only CRUD over the named credential sets from Phase 0:
- GET /api/credentials list all sets grouped by service (NO secrets)
- POST /api/credentials create {service,label,payload}, validated
- PUT /api/credentials/<id> update label and/or payload (partial)
- DELETE /api/credentials/<id> delete (clears any profile selections → fallback)
All four are @admin_only (non-admin → 403), payloads validated via
core.credentials.store, secrets never returned to the browser. Additive — no
existing endpoint or behaviour changes.
Tests: real web_server app + Flask test client (8) — create/list/update/delete
roundtrip, payload never leaks in list, missing-field/unsupported-service/blank-
label/duplicate(409)/404 validation, and the non-admin 403 gate on every write.
Verified the web_server import coexists with the rest of the suite (175 mixed
tests pass).
Groundwork for admin-created, per-profile-switchable credential sets ("pills")
across auth services (Spotify/Tidal/Deezer/Qobuz/Plex/Jellyfin/Navidrome).
Strictly additive and dormant — nothing reads it at runtime yet, so zero
behaviour change for existing installs.
- core/credentials/store.py: pure service registry + payload validation +
stale-safe active-set selection (pick_active_credential falls back to None
when a selected set was deleted, so a profile never breaks).
- migration service_credentials_v1: two new tables — service_credentials
(admin-created named sets; payload Fernet-encrypted at rest) and
profile_service_credentials (each profile's selected set per service).
- MusicDatabase CRUD: create/update/delete/list/get_service_credential
(list never returns the payload; get decrypts for the resolver), plus
set/get_profile_service_credential and resolve_profile_service_credential
(returns the profile's active payload or None → caller uses global default).
Tests: 12 — pure validation + stale-safe selection, and real-temp-DB storage
proving encryption round-trips, payload never lists, dup(service,label)
rejected, per-profile/per-service resolution, and delete clearing dangling
selections to a clean fallback. 95 migration/DB tests still pass.
The #831 watchlist work added the standalone webui/static/watchlist-history.js
(loaded in index.html) but didn't add it to NON_SPLIT_JS, so the onclick-
coverage test couldn't see its functions and failed on openWatchlistHistoryModal
referenced in the History button. Register it alongside its sibling
origin-history.js — same fix the registry comment was added for.
Found during the #832 audit: GET /api/settings returned dict(config_data) — and
config_data is DECRYPTED in memory — so every API key, OAuth secret, Plex/
Jellyfin token, and service password went to the browser in cleartext. Fernet
"encrypted at rest" protects a leaked DB file; it does nothing once the API
hands the plaintext to the client (devtools, HAR captures, an XSS, a screen
share, or a non-PIN'd LAN viewer).
Fix (centralized in ConfigManager):
- redacted_config() deep-copies config and replaces every _SENSITIVE_PATHS value
that's actually set with REDACTED_SENTINEL; unset secrets stay empty so the UI
still shows "not configured". Dict-valued secrets (tidal/qobuz OAuth sessions)
collapse to the sentinel too. GET /api/settings now serves this copy.
- set() ignores a write of REDACTED_SENTINEL to a sensitive path, so the masked
placeholder round-tripped by an unchanged settings form can never overwrite
the real secret. A real value still saves; an empty value still clears.
Frontend: secret inputs are type=password, so the sentinel renders as dots
(looks like a saved secret). _wireRedactedSecrets() clears the mask on focus so
editing types fresh rather than onto the sentinel, and re-masks on blur if left
untouched — so an unchanged secret round-trips the sentinel (kept), an edited
one saves the new value, and a deliberately emptied one clears.
Tests: every sensitive path masks; unset stays empty; dict secrets mask; live
config not mutated; sentinel round-trip keeps the real secret; real value
overwrites; empty clears; sentinel on a non-secret path writes normally.
9 new tests; 518 config-touching tests pass (1 pre-existing soundcloud mock
failure, unrelated — fails identically on a clean tree).
the-hang-man: tracks with an apostrophe (e.g. "I'm Upset") deleted the DB row
but left the file. The library DB stored the title with U+2019 (the curly form
Spotify/Apple metadata uses) while the file was written to disk with U+0027
(ASCII). _resolve_library_file_path compared the curly path byte-for-byte via
os.path.exists, missed every time, and reported "could not be deleted".
Fix: resolve confusable-tolerantly. New core/library/path_resolve.find_on_disk
descends the path component by component, taking an exact match when present and
otherwise folding a small set of typographic look-alikes (curly vs straight
quotes, en/em dash, ellipsis, nbsp) for the comparison ONLY — it never renames,
just finds the file that's actually there. Exact matches always win per
component, so paths that already resolved are byte-for-byte unaffected. This
also fixes existing mismatched files (no re-import) and every caller of
_resolve_library_file_path (sidecar cleanup, dead-file checks, streaming), not
just delete.
Case is deliberately NOT folded: a case-sensitive dataset (ext4/ZFS) can hold
names differing only by case, and folding could resolve the wrong file. The
reported failure is purely typographic.
Tests: real temp-file fixtures exercising the actual byte mismatch — curly-DB →
ascii-disk resolves, exact still works, confusable in a folder component, exact
wins when both encodings present, genuinely-different name does NOT collide,
missing file → None. 10 new tests; 949 resolver-adjacent tests pass.
Beckid: the admin launch PIN was a CLIENT-SIDE overlay only. `launch_pin_required`
just told the frontend to draw a fixed div over the app — removing it (Safari
"Hide Distracting Items", devtools, or any non-browser client like curl) gave
full unauthenticated access to every /api/* endpoint, because the server never
checked it. Anyone who reverse-proxies SoulSync publicly was wide open.
Fix: a before_request gate (_enforce_launch_pin) that rejects every request from
an unverified session while security.require_pin_on_launch is on. The decision
is a pure, unit-tested helper (core/security/launch_lock.request_is_locked) so
the allow/deny matrix can't silently regress. Allowed while locked: the page
shell + static assets, the unlock flow (current/list/select/verify/reset/logout),
and the public REST API /api/v1/ (its own @require_api_key governs it) — EXCEPT
/api/v1/api-keys-internal*, the "no auth required" key-management endpoints,
which stay locked so an attacker can't mint an API key and walk in the side door.
Everything else (data, settings, profile create/edit/delete/set-pin, socket.io)
is blocked.
A blocked top-level browser navigation (deep link / refresh on a sub-page like
/dashboard) is redirected to the root lock screen instead of dumping raw JSON —
detected via Sec-Fetch-Mode: navigate / Accept: text/html (is_html_navigation).
Programmatic fetch/XHR still get the JSON 401 so the frontend can react.
Also fixed the verified flag: get_current_profile POPPED launch_pin_verified
(one page load), but an enforced gate needs it to persist — now READ, so
verification lasts the session (until logout/expiry). No-ops entirely when
require_pin_on_launch is off (default).
Tests: full allow/deny matrix + navigation detection. 20 gate tests + 232
profile/security tests pass.
Boulder: the live display was a cramped ~600px box showing a fraction of the
data the scan already tracks, with no animation and no history.
Live scan deck (replaces the three-column box, full width):
- Header: pulsing live dot, "x / y artists" progress text, and two live
counter chips (found / added) that pop when they change.
- Animated progress bar (artist index / total) with a shimmer sweep.
- Stage: artist avatar with accent glow + name + readable phase line
("Checking album 2 of 5"), album art + album + current track.
- "Added to wishlist this run" feed: taller, bigger art, slide-in animation
that plays once per new track (feed re-renders only when it changes).
- All data was already in scan_state (current_artist_index, total_artists,
tracks_found/added_this_scan, current_phase) — just never displayed. The
legacy fullscreen-modal markup shares element ids and lacks the new ones,
so it keeps working untouched.
Scan History (persistent):
- New watchlist_scan_runs table — one row per run (status, timestamps,
artists/found/added counts) + the full track ledger JSON. Saved at scan
completion AND cancellation; idempotent on run_id; pruned to the last 100
runs. Wishlist rows erode as tracks download, so this is the durable record.
- GET /api/watchlist/scan/history (runs) + /history/<run_id>/tracks (ledger).
- New History button on the Watchlist page → modal in the origins/blocklist
house style: run cards (date, cancelled chip, artists/found/added stats)
expanding into the Added / Skipped track lists with art and badges.
Tests: save+fetch with ledger, idempotent re-save, prune keeps newest,
unknown-run empty, cancelled runs recorded. 398 watchlist/wishlist/history
tests pass; JS syntax-checked; all rendered strings escaped.
Tacobell444 (#707 follow-up): the scan summary said "New tracks: 19 • Added to
wishlist: 10" with no way to see which tracks those were — you had to scan your
wishlist and guess what was new.
Scan ledger: the scanner now records a per-run scan_track_events list (track,
artist, album, thumb, status added|skipped — skipped = found-new but declined
by add_to_wishlist: already queued or blocklisted; capped at 500). The status
endpoint already serializes scan_state, so the payload flows free. The
completed (and cancelled) scan summary on the Watchlist page gets a
"Show tracks" toggle expanding a styled list — Added section + Skipped section
with badges, reusing the live-feed row styling.
Download Origins grouping: the modal now groups entries by what triggered them
(watchlist artist / playlist name) with collapsible headers + counts instead of
a flat list with a per-row badge. Entries arrive newest-first so groups order
themselves by their newest download. Same row markup, checkboxes/delete intact.
Provenance: watchlist adds now stamp scan_run_id into wishlist source_info, so
per-run grouping is queryable later (future "what did run X add" views).
Tests: per-run ledger seam test (added + skipped statuses, album/artist fields,
FIFO unchanged). 316 watchlist/wishlist tests pass; JS syntax-checked.
Third round of the multi-artist report. The earlier fixes (Deezer contributors
upgrade, _artists_list, feat_in_title/artist_separator) were all in place and
correct — but gated on source == 'deezer', and on the real Search → Download
Now path NOTHING carried the source: core/search/sources.py serialized tracks
with no source field, search.js's enrichedTrack didn't add one, so
get_import_source() resolved '' and the whole Deezer-specific block silently
skipped. Files were tagged with only the primary artist until a Retag (which
rebuilds context with the source set — exactly why retagging always fixed it).
The earlier tests passed because they set context['source'] directly — the one
field the real flow never had (same mock-drift as the #823 append tests).
Reproduced with Netti93's exact track (deezer 3966840171) through the real
extract_source_metadata: before — source '', artists ['August Burns Red'];
after — source 'deezer', contributors fetched, artists ['August Burns Red',
'Polaris'], title 'Sonic Salvation (feat. Polaris)' per feat_in_title.
Fix, three layers:
- core/search/sources.py: serialized tracks/albums/artists carry "source"
(the canonical name the orchestrator already passes; '' when unnamed).
- core/imports/context.py get_import_source: also reads '_source' from the
nested dicts (track_info/original_search/album/artist) — additionally fixes
the discography/wishlist flows, which always passed '_source' that nothing
read.
- search.js: enrichedTrack + the album-download path carry source through to
the download task.
Tests: real-payload staging-shaped contexts (source in track_info, '_source'
shape, and the pre-fix sourceless shape staying safe — mocked Deezer client),
serializer source-field tests, resolver fallback tests; exact-shape serializer
tests updated for the new key. 1977 import/metadata/search tests pass (the
only 2 failures are the known soundcloud ones).
carlosjfcasero round 2: after 6fa956d6 append stopped recreating the playlist,
but every sync re-appended ALL matched tracks — every track N times. His log
shows it plainly: "added 22 new tracks to 'Disney' (skipped 0 already present)"
on a playlist that already had those 22.
Root cause: the dedupe read `{t.id for t in get_playlist_tracks(...)}` — but
JellyfinTrack only defines `ratingKey`, never `id`, so the existing-ids set was
ALWAYS empty and everything looked new. NavidromeTrack is also ratingKey-only,
so the Navidrome append had the identical bug. Plex (plexapi ratingKey) was
fine. The existing tests were green because they mocked existing tracks as
SimpleNamespace(id=...) — encoding the same wrong assumption as the code.
Fix:
- New pure planner plan_playlist_append(current, desired) in
core/sync/playlist_edit.py (next to the reconcile planner): order-preserving,
drops already-present ids, dedupes within desired, stringifies (Emby numeric
vs string safe).
- Jellyfin/Emby: existing ids fetched from the canonical
/Playlists/{id}/Items endpoint (same as reconcile — works for Jellyfin GUIDs
and Emby numeric ids), ratingKey fallback if that request fails.
- Navidrome: dedupe on ratingKey (the attribute that actually exists).
Tests: planner (skip-present incl. the reporter's unchanged-playlist case,
desired-order, dupes-within-desired, int/str ids, empties) + the append-mode
suite rewritten to pin the REAL shapes (raw Items dicts for Jellyfin,
ratingKey objects for Navidrome) + a new fallback-path test. 524
playlist/sync/jellyfin/navidrome tests pass.
carlosjfcasero round 2 (manual-add fix didn't help — different path). His log
pinned it: the mirrored sync auto-added 'Llamando a la tierra (Serenade From
the Stars)' by M-Clan every run even though his library has the song (stored
bare). Reproduced exactly: the subtitle restates no album context, so the #808
context strip keeps it, and the length-ratio penalty in
_calculate_track_confidence crushes the pair to 0.142 (needs 0.7). Sync →
"missing" → wishlist, forever; and the cleanup uses the SAME matcher, so it
deterministically never removed it. Self-reinforcing.
Fix at the matcher seam (benefits sync, cleanup, downloads, discography alike):
core/text/title_match.strip_subtitle_qualifiers(title, other) strips a
bracketed qualifier only when it (a) isn't restated in the other title, (b)
contains no version-marker token (EN + ES: live/remix/acoustic/version/
dueto/directo/vivo/...), and (c) introduces no new digit token ('(Pt. 2)',
'(2007)' stay different releases). Wired as a third comparison variant in
_calculate_track_confidence with its own length guard. Verified against his
log's other unmatched tracks: '(Live)' 0.15, '(Dueto 2007)' 0.179,
'(Versión 1988)' 0.167 all still correctly blocked — version qualifiers keep
their meaning; the M-Clan case goes 0.142 → 1.0 in both directions.
Also: sync's check_track_exists call now passes album= (cleanup already did),
enabling the album-aware fallback for multi-artist albums during sync.
Tests: tests/test_subtitle_qualifier_match.py — the reported case verbatim
(end-to-end through check_track_exists, both directions, batched candidate
path included), EN+ES version qualifiers still blocked, numeric guard,
'#769 Dani California' and '#808 OurVinyl' guards still hold. 1396
matcher/wishlist/sync tests pass.
Vicky-2418: Download Discography skipped some albums/singles as "No New Track"
even on a brand-new artist, while manual one-by-one worked. The log showed the
real reason wasn't ownership at all — it was "N skipped (artist mismatch)".
Root cause (confirmed against the iTunes API, not guessed): iTunes returns a
collab as ONE combined string. Narvent's "Miss You (Ambient Remix)" is credited
'TRVNSPORTER, Narvent & SKVLENT'. track_artist_matches did an exact full-string
compare ('trvnsporter, narvent & skvlent' == 'narvent' → False), so every
collaborator's discography entry was dropped — despite the #559 comment claiming
it "keeps features" (it didn't, because features arrive combined, not as a list).
Fix: match the requested artist as a COMPONENT of the credit — split on the
common separators (, & ; / feat ft featuring vs x), while still including the
full string so band names with internal separators ("Florence + the Machine")
match exactly. Component matching stays exact per name, so true contamination
(the #559 case — artist not credited at all) is still dropped, and substrings
("Drakeo the Ruler" ≠ "Drake") still don't match.
Also corrects an over-conservative #559 test that asserted "Drake & Future"
shouldn't match "Drake" — it should; Drake is a credited collaborator. The guard
drops uncredited artists, not legit collabs packed into one string.
Tests: the exact real case (Narvent in the combined credit) + each collaborator,
contamination still dropped, feat/ft/featuring/x forms, no substring false
positive. 36 filter tests + 747 discography/metadata tests pass.
Tacobell444: when tracks land in an album across multiple batches (a wishlist
run, the Album Completeness job, a missed track re-downloaded later), the folder
is rebuilt from API metadata each time — so when $albumtype or $year come back
blank/different on a later batch, the folder NAME changes and the album splits,
forcing a Reorganize.
Fix: build_final_path_for_track now checks whether the album already lives in a
single folder on disk and, if so, drops the new track there instead of a freshly
templated folder. Match (chosen): exact stored Spotify album id first, then a
STRICT >=0.85 name+artist match (vs the 0.7 used elsewhere) — a wrong match here
misplaces a file. New core/library/existing_album_folder.resolve_existing_album_folder
holds the logic; always-on with template fallback.
Safety rails: only returns a folder UNDER the transfer dir (never a read-only
library/NAS mount), only when the album lives in EXACTLY ONE folder (multiple =
disc subfolders, which DatabaseTrack can't disambiguate — those defer to the
template), and any failure falls through to the template path. Added
MusicDatabase.get_album_by_spotify_album_id for the id-first lookup.
Tests: single-folder reuse, no-match, below-threshold, multi-folder defer,
outside-transfer reject, id-first, missing transfer dir, no-files-on-disk.
8 tests; 1556 path/import/download tests pass (only the known soundcloud
failures remain).
macstainless: a Plex-on-macOS user running SoulSync in Docker had all 5,250
tracks flagged "dead" even though the files exist and play in Plex. Root cause:
the DB stores paths as Plex reported them (/Volumes/Core/Music/...), which don't
exist inside SoulSync's container. resolve_library_file_path() returns None for
"couldn't find it at any known base dir" — and for a mis-mounted library that's
EVERY track, not a deletion. The job treated None as "file deleted" and created
a finding per track.
Fix mirrors the existing transfer-folder abort: collect unresolvable tracks, and
if at least max_unresolved_fraction (default 0.5) of the library is unresolvable
once it's past min_tracks_for_guard (default 25), treat it as a path-mapping/
mount problem — abort with an actionable message (Docker mount / Settings →
Library → Music Paths) and create ZERO findings. A small fraction unresolvable
is still reported as genuine dead files, and tiny libraries (< min) report as
before. Both thresholds are configurable per the job's settings.
Tests: mass-unresolvable aborts with no findings; a lone dead file among real
ones is still reported; a small all-dead library still reports; thresholds
configurable. 54 repair tests pass.
Per Boulder's calls on the new enrichment toggle:
- Naming: "Spotify Free" was misleading (it's a hybrid — pick it, connect an
account, and sync still uses your official playlists). Relabel the user-facing
strings to "Spotify (no auth)" — the real distinction is needs-credentials vs
not. Internal value/key (spotify_free, _free_*) unchanged, so no migration.
- Default ON: metadata.spotify_free_enrichment now defaults True (worker + UI
load both treat unset as on). So bulk enrichment runs on the no-auth path by
default and the official account is reserved for interactive search/sync; turn
the toggle off to enrich through the connected account. The toggle overrides
auth for the worker (authed users still enrich via no-auth) — matching the
intended model.
- Worker runs on the toggle alone: is_spotify_metadata_available() now honors
_prefer_free (+ package installed), so the worker enriches via no-auth even
with no account connected and no 'no-auth' source selected. Only fires on a
client carrying the flag (the worker's own), so interactive/watchlist
availability is unchanged.
- UI: moved the toggle from "Metadata Source" to the Spotify section next to the
auth fields, always visible, on by default. Help notes the genre trade-off.
Tests: prefer_free makes metadata available without auth/source (and is inert
without the package); interactive availability unaffected. 218 Spotify tests pass.
Two related fixes at one root cause. Every catalog method gated the official API
on `use_spotify = is_spotify_authenticated()` and only fell to Free *after*
official failed. So when the client was authed but should defer to Free —
specifically when the worker's daily real-API budget was spent — it kept hitting
the official API anyway (just stopped *counting* it). The budget "bridge" never
actually diverted; it only stopped pausing.
Root-cause fix: the official gate is now `is_spotify_authenticated() and not
self._free_active()` across all 8 catalog methods (search_artists/albums/tracks,
get_artist/album/album_tracks/track_details/artist_albums). No-auth and
rate-limited are unchanged (auth is already False there); the change only affects
the cases where auth is True but we deliberately defer to Free. The user-account
methods and the metadata-availability helper are untouched.
New opt-in: metadata.spotify_free_enrichment. When set, the worker puts
`_prefer_free` on its OWN client and _free_active() honors it (needs only the
package installed — the flag is the opt-in — not the 'Spotify Free' source
choice). So a connected user can run bulk enrichment on the no-creds source to
spare their official quota, while interactive search/resolve stay official-first
(they use a different client that never sets the flag). Default off.
Tests: _free_active honors prefer_free (and is inert without the package);
search_albums defers to Free — official .sp raises if touched — both under
prefer_free AND under budget-exhaustion (the divert that previously never
happened). 215 Spotify tests pass.
SpotipyFree has no album-name search (search_albums returned []), and
SpotifyClient.search_albums had no Free branch at all — so when Spotify Free was
the active source (no-auth / rate-limit ban / spent budget), album matching
couldn't use Spotify and dropped straight to the iTunes/Deezer fallback. The
enrichment worker's per-album matching was effectively blind on Free.
Fix — work the gap using ONLY the free source:
- spotify_free_metadata: new search_albums_via_artist(artist, album) resolves the
best-matching artist, scans their discography (get_artist_albums_list — which
Free DOES support), and ranks by album-name similarity. Pure rank_albums_by_name
helper is unit-testable; the network method composes it.
- SpotifyClient.search_albums: add a Free branch mirroring search_artists/tracks
(official → Free-if-active → iTunes/Deezer), plus optional artist/album params
so the Free path has the names it needs. Bare-query callers skip it unchanged.
- spotify_worker: _process_album_individual passes artist+album so the bridge can
match albums on Free.
Tests: rank ordering + limit, via-artist picks the right artist and ranks,
empty on missing artist/album/no-match, and SpotifyClient.search_albums returns
Free albums (not the iTunes fallback) when free is active. 210 Spotify tests pass.
The manual album "Add to Wishlist" modal had NO ownership check at any layer —
the album view opened the modal without ownership info, the modal added every
track, and the backend (add_album_track_to_wishlist) added each unconditionally.
So adding an album you (partially) own dumped the owned tracks straight into the
wishlist (carlosjfcasero #825) — and the auto-cleanup doesn't reliably remove
them. The bulk discography path already dedups (full missing-track analysis);
this path didn't.
Backend (the reliable seam): add_album_track_to_wishlist now skips a track that
already exists in the library, gated on the same wishlist.allow_duplicate_tracks
toggle the watchlist scan + cleanup use — OFF → skip owned (returns
{success, skipped:true}), ON → add anyway. Default is ON, so default users are
unaffected; the quality re-download flow uses a different endpoint, so it's
untouched.
Frontend: handleAddToWishlist + addModalTracksToWishlist count skipped tracks
separately so the toast is honest ("Added 3 (5 already owned)" / "All N already
in your library") instead of falsely claiming owned tracks were added.
Tests: skips owned when duplicates off, adds missing when off, adds owned when
on (and doesn't even run the check then). 205 wishlist tests pass.
Sokhi: "downloads searching for way too many tracks at once" — a wishlist run
that fanned out into ~one batch per album. Verified the actual search/download
concurrency IS capped at 3 (single shared missing_download_executor), so it
wasn't really hammering slskd — but the display showed ~20 "searching" and the
batch list was a mess.
Root cause: run_full_missing_tracks_process was supposed to "block its album-pool
worker for the whole search+download" (that's what the dedicated album_bundle_
executor is for), but it RETURNED the instant it had STARTED the downloads. So
the album pool only throttled the fast analysis phase — every album batch blew
through analysis and immediately dumped its tracks into the shared download pool,
all pre-marked 'searching'. The intended serialization never happened.
Fix: add serialize= to run_full_missing_tracks_process. Album-bundle batches
(dispatched on album_bundle_executor) pass serialize=True and now hold their pool
slot via _wait_for_batch_drain() until every task in the batch reaches a terminal
state — so only ~N albums are in flight at once. The wait is passive (downloads
are driven by the monitor + completion callbacks on other threads, so no
deadlock) and bails on shutdown, a removed batch, or a safety cap. The residual /
playlist / manual paths run on the SHARED pool and pass serialize=False (blocking
there would steal a real download worker), so they're unchanged.
Tests: _wait_for_batch_drain returns immediately when all-terminal, waits until
tasks finish, bails on shutdown, respects the cap, handles a missing batch. 975
download/wishlist tests pass (only the pre-existing soundcloud /app failures).
carlosjfcasero: "append" sync mode still recreated the playlist (wiping image +
description) on both the sync-page auto-sync and the Playlist Pipeline. Root
cause: _run_sync_task defaulted sync_mode='replace', and every AUTOMATED caller
omits the mode — auto_sync_playlist (mirrored auto-sync + pipeline), the
iTunes-link sync, and Wing It. So those paths always replaced, ignoring the
user's chosen mode entirely. (Manual sync + the per-source discovery path already
passed a mode, which is why it only bit automated runs.)
Fix: when no mode is passed, _run_sync_task resolves the user's configured global
"Playlist sync mode" (normalize_sync_mode(None, playlist_sync.mode)) — the same
thing _submit_sync_task already does — instead of hardcoding 'replace'. The
global default is still 'replace', so users who never changed it are unaffected;
only those who set Append/Reconcile get the corrected behavior.
Tests: normalize_sync_mode(None,'append')→'append' (and 'replace' unchanged);
auto_sync_playlist must not force a mode (no sync_mode kwarg / no 7th positional)
so the resolution can happen. 896 sync/automation/discovery/playlist tests pass.
Part 1 stopped existing full dates being destroyed; this adds first-class support
for full release dates so they can be set + persisted instead of truncated to a
year at the DB layer.
- Schema: new nullable `release_date TEXT` on the albums table (idempotent
ALTER-ADD-COLUMN repair on startup + the live CREATE). NULL = year-only, every
reader falls back to albums.year, so it ships safe/dormant.
- Tag writer: write_tags_to_file + build_tag_diff prefer db_data['release_date']
(the full date) over the year int; _date_to_write writes the full date. When
there's no release_date it's exactly Part-1 behavior (year, preserving an
equally-specific existing file date).
- Retag read path: SELECT al.release_date in the tag-preview/write queries and
thread it into _build_library_tag_db_data.
- Manual edit: release_date added to ALBUM_EDITABLE_FIELDS + a "Release Date"
field (YYYY-MM-DD, validated client-side) in the album editor; the artist-album
query returns it so existing values show. User-set dates are authoritative.
- Enrichment: Spotify + iTunes workers store the source's full release_date
(YYYY-MM / YYYY-MM-DD) when present, only when empty — never clobbering a
manual value.
Tests: writer uses release_date over year + overrides an existing file date;
falls back to year when absent; diff compares the full date. Migration verified
idempotent + enrichment no-clobber. 1435 tag/retag/db/library tests pass.
Sokhi's log showed the actual cause: with the wishlist "allow duplicates" toggle
on, is_track_missing_from_library skips a track when _albums_likely_match() says
the wanted album and a library album are the same — and it was matching albums
that differ ONLY by a decimal volume number:
[AllowDup] Album match — skipping (wanted: '...Vol.5', library: '...Vol.5.5')
[AllowDup] Album match — skipping (wanted: '...Vol.5.5', library: '...Vol.4.5')
His character-song CDs share track titles across volumes, so tracks from albums
he DOESN'T have got skipped because a same-titled track sits in a similarly-named
volume he DOES have — the partially-filled discography never completed.
Root cause: _normalize_album_for_match strips the dot ("Vol.5.5" -> "vol 5 5"),
and _VOLUME_MARKER_RE captured only a single trailing digit, so Vol.5, Vol.5.5
and Vol.4.5 all reduced to marker "5" and the volume-disagreement guard never
fired. Fix: capture the full multi-part number ((\d+(?:\s+\d+)*)) so "5" / "5 5"
/ "4 5" are distinct and the guard correctly rejects the match.
(Not lookback — the log confirms lookback=all was already honored.)
Tests: decimal/multi-part volumes (incl. the real CJK names) now block the
match; identical decimal volumes + naming-drift cases still match. 111 watchlist
tests pass.
THE bug behind "embeds art but never writes cover.jpgs": _fix_missing_cover_art
passed `details['album_folder']` (= dirname of the raw DB path, e.g. Jellyfin's
/data/music/...) as the target folder. That path doesn't exist inside the
SoulSync container — only the resolved /app/... path does. So apply_art_to_album_
files' `os.path.isdir(target_dir)` was False and the ENTIRE cover.jpg block was
silently skipped: embedding still worked (it uses the resolved paths) and the DB
thumb updated, but the sidecar was never written. Exactly Sokhi's symptom + the
"Cover art already present — database thumbnail updated" toast (cover_written
stayed False).
Fix:
- _fix_missing_cover_art: derive the folder from the RESOLVED file
(os.path.dirname(resolved[0])), never the raw album_folder.
- apply_art_to_album_files: bulletproof it — if the passed folder doesn't exist,
fall back to the real directory of the files instead of silently skipping.
Tests: a non-existent folder still writes cover.jpg to the real file dir. 1348
cover/art/repair tests pass.
Root cause of Sokhi's endless 0-findings: the SCAN resolved each track's path
through the path-mapping layer with NO fallback, while the APPLY
(_fix_missing_cover_art) uses `_resolve_file_path(...) or p` — i.e. it falls
back to the raw DB path when mapping returns nothing. On his Docker setup the
mapping returns None, so the scan set has_local=False and skipped every album
(never looking at the folder), even though the apply WOULD have written the
cover.jpg from the raw path.
Fix: make the scan match the apply — if mapping returns nothing but the raw DB
path is a real file (container path == stored path), use it as-is. Now the scan
actually inspects the folder, sees the missing cover.jpg, and flags it; the
apply then writes it from the embedded art.
Tests: unresolved-path-but-real-file → flagged (sidecar_from_embedded); the
fallback does NOT fire for a non-existent path (media-server-only stays skipped).
Kept the [cover-diag] logging from the prior commit to confirm on Sokhi's run.
Sokhi: after the earlier flag fix, scans returned 0 matches — albums with
embedded art but no cover.jpg were treated as fully arted and skipped, so their
cover.jpg never got written.
Root cause: the scan used album_has_art_on_disk (True if EITHER embedded art OR
a cover.jpg exists), conflating the two. Now it checks them separately:
- a local album is flagged if it lacks embedded art, OR it lacks a cover.jpg
sidecar AND cover.jpg writing is enabled (metadata_enhancement.cover_art_
download — Boulder: "only scan for cover.jpgs when enabled").
- an album that has embedded art but no sidecar is fixable even when the API
finds no art: the apply writes cover.jpg from the EXISTING embedded art.
apply_art_to_album_files now writes the cover.jpg sidecar by extracting the
album's own embedded art (new extract_embedded_art) — consistent with the
files, no API call — and only falls back to download_cover_art when there's
nothing embedded to extract. _fix_missing_cover_art no longer bails on a
missing artwork_url when sidecar_from_embedded is set.
Tests: scan flags embedded-but-no-cover.jpg (incl. when API finds nothing),
still skips albums with both, still flags artless albums; apply writes cover.jpg
from embedded art (no download), falls back to download when none, skips when a
sidecar already exists; extract_embedded_art unit tests. 1344 cover/art/repair
tests pass.
When a track shows "Not found", the manual search now accepts a pasted Tidal or
Qobuz track link, not just a typed query (CubeComming: the fuzzy search misses
versions; he can find the track on Tidal but can't get it to appear).
How it works (robust, reuses the proven path): parse the link → (source,
track_id) → fetch the track via the source client's get_track → build a clean
"artist title (version)" query → run THAT source's normal search → bubble the
result whose id matches the link to the top. So the candidate is a normal,
already-downloadable streaming result — no hand-built download encoding — and
it downloads through the existing verified flow.
Degrades gracefully: if the source isn't connected or the link can't be
resolved, it falls back to a normal text search of the raw input — the user is
never worse off than typing it themselves. Scoped to Tidal + Qobuz (the
streaming sources that download by track id, with public track URLs); Soulseek
can't take a link (P2P, no ids), YouTube/SoundCloud are URL-native via a
different path (future).
- core/downloads/track_link.py: pure parse_download_track_link (tidal/qobuz
/track/<id>, slug/region suffixes, scheme-less) + query_from_track_payload
(per-source title/artist, Tidal version-append).
- manual-search endpoint: link detection → resolve → restrict to that source →
id-match bubble.
- placeholder hint mentions pasting a link; maxlength 200→300 for long URLs.
Tests: 14 (parser shapes + payload extraction incl. remix version-append +
qobuz performer/album-artist fallback). JS valid.
#775 already resolves pasted Spotify / Apple / MusicBrainz / Deezer links to an
exact artist/album/track on the Search page. Added Discogs to that set (the one
source in the request not already covered; Amazon left out per request).
- by_id resolver: discogs in SUPPORTED_SOURCES + _KNOWN_HOSTS; parse
/artist/<id>-slug, /release/<id>-slug, /master/<id>-slug (master→album), and
strip the slug to the leading numeric id. Discogs has no standalone track
URLs (tracks live inside a release), so artist + album resolve.
- Fetch dispatch: discogs albums use get_album(include_tracks=False) like
itunes/musicbrainz; artists use get_artist; both already return the common
normalized card shape. Updated the not-a-link hint to mention Discogs.
Frontend needs no change — it adopts whatever source the resolver returns and
Discogs is already a search source. Tests: parse release/master/artist
(slug-stripped, scheme-less) + resolve release→get_album(numeric id) and
artist→get_artist. 43 by_id tests pass.
CubeComming: manual imports of large hi-res Qobuz FLACs came out as empty
shells — no audio, no tags, no length/bitrate. Root cause: mutagen's in-place
save() rewrites the file, and it's NOT atomic; if the process is interrupted or
OOM-killed mid-write (he also reported the memory-growth issue #802), the file
is left truncated — audio and tags gone.
save_audio_file (the chokepoint both the enrichment tag-write AND wipe_source_
tags route through) now saves atomically: copy the original to a temp in the
same dir, write the modified tags into the copy, verify it's still valid audio
(duration > 0), then os.replace() it in. The original is untouched until that
atomic swap, so a crash can only orphan the temp — never destroy the user's
file. Falls back to the plain in-place save (byte-identical to before) when the
atomic path can't run, so nothing is ever left worse off. tag_writer's
write_tags_to_file routes through the same helper.
Verified the atomic path works with REAL mutagen on a real FLAC (audio length
preserved 1.0s→1.0s, tag written, temp cleaned). Tests: replace-on-success,
original-survives-save-failure, corrupt-temp-rejected, no-filename-plain-save,
+ a real-FLAC round-trip (skips without ffmpeg). 2443 import/metadata/tag tests
pass.
Sokhi: filler works but doesn't write cover.jpg. Cause: the sidecar write
(download_cover_art) respects the import-time "Download cover.jpg to album
folder" toggle, while embedding ignores it — so with that toggle off, art
embeds but no sidecar is written.
A job literally called Cover Art Filler should produce the complete art when
you explicitly run it. download_cover_art gains force=True (bypasses the toggle)
and the filler's apply path passes it. The import pipeline still calls without
force, so it keeps honoring the user's setting.
Tests: filler passes force=True; existing cover-write mocks updated for the new
kwarg. 1732 art/cover/repair/import tests pass.
Three related fixes to Tidal track matching and downloading:
1. version-field handling — Tidal stores remix/live/edit qualifiers in a
dedicated `track.version` attribute (e.g. name="Emerge",
version="Junkie XL Remix"), not in the track name. The qualifier
filter and the matcher only looked at name/album, so the exact
recording was discarded. Fold `version` into both the qualifier
haystack and the candidate title passed to MusicMatchingEngine.
2. divergent-version penalty — once versions are visible, OTHER cuts of
the same base become candidates ("(Shazam Remix)" vs "(southstar
Remix)"). Neither title is a prefix of the other, so the prefix-based
version check missed them and the raw ratio stayed high off the
shared base. Apply a heavy penalty when both titles carry different
version descriptors so the wrong cut can't outscore the threshold.
3. rate-limit backoff — the trackManifests endpoint is aggressively
rate-limited; a bare request failed 429 instantly, burned the quality
tier, re-queued the track and hammered again (a self-amplifying
storm). Honour Retry-After / exponential backoff with a bounded retry
count and shutdown-aware sleep.
Adds unit + end-to-end tests for all three.
Boulder (Plex): "flags every album, but everything has art." His albums show
art in the library (served from the embedded file art), but the DB thumb_url
cache column is empty — and the scan flagged on db_missing (empty thumb_url),
so every local album tripped it despite having perfectly good art in the files.
Now: a LOCAL album is flagged only when its files actually lack art
(disk_missing). An empty thumb_url is just a stale cache when the files have
art — not "missing cover art". db_missing still flags media-server-only albums
(no local files), where the DB thumb is the only art there is.
Tests: local+file-art+empty-thumb → NOT flagged (the bug); local+no-file-art →
still flagged; media-server-only+empty-thumb → still flagged.
The scan checked album_has_art_on_disk() on the RAW DB track path, while the
apply (_fix_missing_cover_art) resolves the path first. On any path-mapped
setup (docker mounts, a Plex/SoulSync path mismatch) the raw path isn't found,
disk art reads as "missing", and EVERY album gets flagged — then the apply
resolves the path, finds the art already there, and reports "already present".
Scan and apply disagreed purely because only the apply resolved paths.
Now the scan resolves the representative path the same way (resolve_library_
file_path, same transfer/download/config inputs the retag job uses) before
checking disk art. Unresolvable → treated as no-local-file (not claimed
disk-missing) so we never false-flag a file we simply can't reach.
Tests: disk check runs on the resolved path (thumb+art → not flagged);
unresolvable path → not flagged + art never checked on None.
JellyfinArtist sets self.thumb (→ artists.thumb_url on scan); JellyfinAlbum
never did. So for Jellyfin/Emby the library scan read getattr(album, 'thumb',
None) == None and albums.thumb_url stayed empty for the WHOLE library. Two
downstream effects: blank album art in the web UI, and the Cover Art Filler
flagging every album as "missing cover art" (db_missing = empty thumb_url) —
making the filler the only thing that ever populated the column, which is
backwards. It should come from the scan, like artist thumbs do.
Fix: JellyfinAlbum.thumb = /Items/<id>/Images/Primary (same shape as the artist
thumb, which already normalizes + displays via fix_artist_image_url). Now the
scan stores album thumbs, the UI shows art, and the filler only flags albums
that are genuinely missing art. Navidrome + Plex already set album thumbs.
Tests: album exposes the Primary-image thumb, None without an id, same shape as
the artist thumb.
The real reason Sokhi's cover-art fix "didn't work" and Boulder saw the same
message on a WRITABLE Windows box: _fix_missing_cover_art reported
"Updated database thumbnail, but could not write art to files (read-only?)"
whenever embedded==0 AND cover_written==False — but the overwhelmingly common
cause of that is every file ALREADY having embedded art (skipped, not failed).
The message blamed read-only on a perfectly writable library, which sent us
chasing a read-only ghost across three commits. Sokhi's /fix returns 200
(success), so he was never hitting the genuine EROFS path at all.
Now the message is derived from the art_result breakdown:
- embedded/cover written → "Applied cover art: …"
- failed>0 (non-EROFS) → "… check file/folder permissions"
- skipped>0 (already arted) → "Cover art already present on all N file(s)"
- otherwise → "no file artwork was applied"
Genuine read-only (EROFS) still hard-fails with the clear mount message.
Tests: already-arted→present (not read-only), failed→permissions, EROFS→hard
fail, embedded→success. 453 cover/repair/art tests pass.
Sokhi still hit the read-only error after the statvfs fix (6c3e285a). Root
cause was a gap that fix didn't cover: read_only_fs was only set from the
per-file EMBED write, but download_cover_art SWALLOWS its own cover.jpg EROFS
(logs "Error downloading cover.jpg" and returns). So when an album's tracks
already have embedded art, the embed loop is skipped, only the cover.jpg
sidecar write runs, its EROFS is swallowed, and the filler reported success
while spamming the log — exactly Sokhi's case.
Fix (no blast radius — download_cover_art still never re-raises, since its
import-pipeline callers aren't wrapped): on EROFS it now records
'_cover_read_only' on the passed context instead of just logging; apply_art_to_
album_files passes a capture dict and promotes that to read_only_fs. So a
cover-only read-only album now correctly surfaces the read-only message instead
of a silent success.
Tests: +1 — embed skipped (track already arted) + cover.jpg read-only →
read_only_fs True. 472 cover/art/repair tests pass.
#811 (reopen of #792, carlosjfcasero, Emby/Jellyfin): "append" mode clobbered
the playlist's custom image. The post-sync image push only excluded
'reconcile' — so append (which edits the playlist in place via
append_to_playlist) still re-pushed the source image over the user's poster
every sync. Now both in-place modes (reconcile + append) skip the image push;
only the destructive 'replace' (recreate-from-scratch) pushes it.
append_to_playlist + set_playlist_image were verified to NOT touch tracks or
description (image push only POSTs /Images/Primary), so this is the identity-
clobber fix for append.
Tests: append + reconcile preserve the image, replace still pushes it.
CubeComming #804: importing Coldplay "Yellow" (the 269s Parachutes album track,
correctly tagged) was quarantined — "Duration mismatch: file is 269.2s, expected
266.0s (drift 3.2s > tolerance 3.0s)". The expected 266s came from a re-resolved
*single* edition, not the file's actual album. The duration-agreement integrity
check exists to catch truncated/wrong slskd TRANSFERS — but a manual import is
the user's own already-tagged file being sorted, so checking it against a
re-resolved release just manufactures false quarantines.
Fix: both manual-import paths (singles + album) now mark the context
is_local_import; the integrity check skips the duration-agreement leg for local
imports via expected_duration_for_check() (new pure helper). The size +
mutagen-parse legs still run, so genuinely broken files are still caught — only
the release-vs-file duration comparison is skipped, and only for manual imports.
slskd downloads are completely unaffected.
This does NOT change the deeper matching (file still groups under Singles vs the
Parachutes album — the #767 canonical-version family); it stops the false
quarantine so the file imports.
Tests: 4 on the helper (local skips, download keeps, zero/None/garbage, string
coercion) + updated the routes context assertion. 557 import/integrity tests pass.
CubeComming #804: since 2.6.7, importing already-tagged files (Bruno Mars,
Coldplay) blanked EVERY tag and filed them under "Unknown Artist". Root cause:
both metadata-enhancement blocks in post_process_matched_download did
`except Exception: wipe_source_tags(file_path)` — a full audio.tags.clear() +
strip + clear_pictures. But enhancement throwing means NO new tags were
written, so wiping just destroys the originals. A transient enhancement error
on a well-tagged file = total metadata loss. (The reported "bitrate change" is
a red herring: mutagen padding on re-save, not a re-encode — ReplayGain only
reads via ffmpeg and tags via mutagen.)
Fix: gate the failure-path wipe on should_wipe_tags_on_enhancement_failure()
(new pure, tested policy) — only wipe UNMATCHED downloads (likely junk source
tags); NEVER wipe a clean/matched import, preserve its existing tags + log.
Unmatched-download behavior is unchanged, so the only thing that changes is the
broken case.
Tests: 3 pin the policy (clean→preserve, unmatched→strip, falsey→strip).
1211 import/pipeline/metadata tests pass.
The destructive job's findings-vs-auto toggle was 'auto_delete: False'. Renamed
to 'dry_run: True' to match the Re-tag job's convention and make the safe
default unmistakable: dry run ON (default) = findings only, deletes nothing;
dry run OFF = hands-off auto-delete. Behaviour-identical to the previous
default — just clearer + consistent. Help text + tests updated.
A Library Maintenance job that cleans up downloads tracked by Download Origins
once they pass a per-origin retention window — findings by default, opt-in
auto-delete.
A download is only ever proposed for deletion when ALL hold: older than its
origin's retention, NOT still in an actively-mirrored playlist / watched
artist, and played fewer than the keep-threshold (default 2 → "played more
than once is kept"). Only touches downloads recorded from the Download Origins
feature forward — never pre-existing or manual library.
- core/library/expired_cleanup.py: pure decision core (retention_cutoff,
is_expired, select_expired) — no DB/clock, fully tested. play_count is the
reliable listen signal (last_played is often unpopulated, so recency isn't
used).
- ExpiredDownloadCleanerJob: gathers facts (play_count via a new
get_origin_cleanup_candidates join; active-mirror via get_mirrored_playlists;
watch via get_watchlist_artists) and either creates 'expired_download'
findings or, with auto_delete on, deletes in-scan. Default OFF, both
retentions default 'off'. Settings auto-render in the Library Maintenance
panel (same as Cover Art / Lyrics / Re-tag).
- delete_origin_download(): shared delete (resolve path → remove file → drop
track row → drop history row); a file that won't delete keeps its row +
reports. Used by auto mode AND the _fix_expired_download apply handler.
- Frontend: type/action ('Delete')/result labels + finding detail render.
Tests: 9 on the pure brain (windows, off, per-origin, protected, play-count
threshold, bad age) + 7 on the job (no-op when off, findings, mirror/watch
protection, auto-delete, delete helper missing/real file). 185 repair/origin
tests pass.
Sokhi got a read-only error from the cover-art filler with NO ':ro' in his
compose. Root cause: my earlier Tim fix added a statvfs pre-flight that bailed
when f_flag & ST_RDONLY — but union/FUSE/network filesystems (mergerfs,
rclone, NFS), ubiquitous in self-hosted setups, misreport those mount flags.
A perfectly writable library could be flagged read-only and blocked. statvfs
is a guess; the only honest test is whether an actual write raises EROFS.
- Removed the statvfs pre-flight entirely. Read-only is now detected solely
from a real EROFS on the embed write, which also fast-fails the remaining
files (so no statvfs needed for the fast-fail Tim wanted either).
- Broadened the user message: a genuine read-only mount isn't always ':ro' —
could be a read-only host/NFS/SMB mount or a mergerfs read-only branch.
Tests: writable FS succeeds even when statvfs would claim read-only (the
regression), real-EROFS-on-write still flagged + bails the rest, EACCES still
not conflated with EROFS. Dropped the now-moot Windows-statvfs test (statvfs
is no longer referenced). 445 art/cover/repair tests pass.
Second lock-in catch: tracks.duration is stored in MILLISECONDS (schema), but
the scan passed it to LRClib as SECONDS. LRClib's exact-match-by-duration
strategy would never hit (215000s vs the real 215s), silently falling back to
the fuzzier title/artist search and storing the wrong duration in the finding.
Now divides by 1000 (guards against 0/garbage). Lyrics were still being found
via the fallback, so no track was missed — just less precise matching and a
wrong stored value. Test pins 215000ms → 215s.
Lock-in pass caught a real bug in 1051ef24: the retag lyrics path stuffed the
library title/artist into a plan's db_data to feed the lyrics query — but
db_data is exactly what write_tags_to_file writes ("only writes fields that
have DB values"). So an UNMATCHED track (one with no source match, meant to
get art/lyrics only) would have had its title/artist tags overwritten from
the library values — an unintended tag write on a track we never verified.
Fix: each plan now carries a separate READ-ONLY lyrics_meta
({title, artist, album}) sourced from the library track + album scope, kept
entirely out of db_data. apply_track_plans reads lyrics_meta for the query
(db_data fallback for older plans); unmatched plans keep db_data={} so no tags
are written. _fix_library_retag threads lyrics_meta through the manual-apply
path too.
Tests: +1 regression pinning that an unmatched lyrics plan calls
write_tags_to_file with EMPTY db_data (no title/artist leak) while still
fetching lyrics. 70 lyrics/retag/repair tests pass.
The lyrics sibling of the Cover Art Filler, plus retag integration — reusing
the existing LyricsClient (LRClib) the import pipeline already uses.
- lyrics_client: extracted the LRClib fetch (exact-match-with-duration →
search fallback) into a shared _fetch_remote_lyrics, used by both
create_lrc_file (unchanged behavior) and a new check-only has_remote_lyrics.
- MissingLyricsJob (core/repair_jobs/missing_lyrics.py): scans tracks with no
.lrc sidecar and — Option A — only flags ones LRClib actually has lyrics
for, so instrumentals/interludes are never surfaced or re-flagged. Registered
in the job list; default OFF; respects the lrclib_enabled toggle.
- _fix_missing_lyrics (repair_worker): applies a finding by fetching + writing
the .lrc and embedding lyrics via create_lrc_file.
- Re-tag tool: new 'lyrics' setting ('fetch'|'skip', default skip). When
'fetch', apply_track_plans now also fetches/refreshes the .lrc per track
(fetch-if-missing, re-embed-if-exists) — threaded through scan gates, finding
details, the auto-apply path, and the manual fix handler. Settings UI
auto-renders the dropdown from setting_options; no markup needed.
- Frontend: type/action/result labels for missing_lyrics + a finding detail
render case.
Tests: 12 — has_remote_lyrics truth table, sidecar detection, scan (only-
fixable / skip-existing / lrclib-disabled), the apply handler, and retag
lyrics_action on/off. 694 repair/lyrics/cover/retag tests pass.
Closes the last acquisition gap — user-initiated downloads. A blocklist isn't
a censor, so search + discography stay fully visible; instead the download
ACTION is gated, visibly and overridably:
- Download modal (start-missing-process): an up-front check — if the WHOLE
album or artist being downloaded is blocklisted, return 409 {blocked:true}
with the entity, before starting a batch. The modal shows "X is blocklisted
— download anyway?" and re-POSTs with ignore_blocklist:true on confirm
(threaded onto the batch so the Phase 2a per-track filter skips it).
Scattered single-track bans still fall through to the 2a filter quietly.
- Manual /api/download (search-result download): source-file-centric, so it
matches the blocked ARTIST by name; same 409 + confirm + override. search.js
now sends artist/title so the guard has something to match.
- Precedence confirmed: force-download overrides "already owned", NOT a ban
(the 2a filter runs on the force-expanded missing list).
Frontend: shared confirmBlockedDownload() helper; modal + search callers
handle the blocked response and retry with the override.
Tests: manual download blocked-by-name / unrelated-allowed / override-passes,
and the modal up-front 409 for a blocked album. 8 blocklist API tests pass.
Phase 1 guarded the wishlist; Phase 2a closes the other auto-acquisition path.
Playlist sync, album download, and discography backfill all flow through
run_full_missing_tracks_process, which queues missing tracks at one point —
right where the explicit-content filter already drops tracks. The blocklist
filter slots in beside it: each missing track is checked and a banned
artist/album/track is dropped before queueing (logged with a count), so a
blocked item can't slip in via these flows.
Same brain as Phase 1: the wishlist guard's matcher is generalized to
db.blocklist_reason_for_track(profile_id, track_data, source=None) — the new
`source` param lets the queue path supply the batch source, since an analysis
track dict may not carry a 'provider' field (artists still match by name
fallback regardless). One method, two callers (wishlist + queue), one cascade.
Manual single-track downloads (/api/download, candidate picker, redownload)
are deliberately NOT gated here — that's Phase 2b, pending a block-vs-warn-vs-
override policy decision.
Tests: source-fallback isolation (album id-only proves source drives the ID
match; artist name still matches sourceless), and a queue-filter simulation
mirroring master.py. 35 blocklist tests pass (the only failures in the
download family are the pre-existing soundcloud /app ones).
Completes Phase 1 on top of the backend (43c798a7):
- Cross-source backfill: core/blocklist/backfill.py is a pure injected-resolver
core (resolve only missing sources, never raises); core/blocklist/runtime.py
wires the real metadata clients with a confident name-match (exact
significant-token equality; album/track also require the parent artist when
both expose one — no wrong IDs hung on an entry). Resolution runs
synchronously at add time, so a ban is cross-source from the first scan;
the artist name-fallback in matching covers any gap.
- API: GET/POST/DELETE /api/blocklist (profile-scoped) + /api/blocklist/search
(thin wrapper over the manual-match service search on the active source, so
the modal needn't know the source). Add resolves the other sources before
storing.
- Modal (webui/static/blocklist.js): tabbed Artists/Albums/Tracks in the
revamp design language (accent light-edge, pill tabs, debounced search with
spinner + out-of-order guard, per-result Block, "currently blocked" list
with a match-status star and per-row remove). Opened by a new "Blocklist"
button on the watchlist page, next to Download Origins.
Tests: 5 backfill (fill-missing-only, None/exception handling, arg shape) + 4
API (search proxy, add→backfill→list→delete round trip, validation). Modal
registered in the script-split onclick-coverage test; JS syntax-checked.
A proper artist/album/track blacklist (distinct from download_blacklist, which
stays untouched). ID-keyed across metadata sources so a ban survives a source
switch; profile-scoped; cascade artist→album→track.
- core/blocklist/matching.py — pure decision core (no I/O): build an index from
rows, candidate_block_reason() walks track→album→artist. Same-source ID match
is primary; artist NAME is a fallback (covers the ID-backfill window);
albums/tracks are ID-only (common titles like "Greatest Hits" must not
false-positive across artists). Source-isolated so a numeric Deezer id can't
collide with a numeric iTunes id of a different entity.
- DB: new `blocklist` table (profile_id, entity_type, name, 4 source-id cols,
match_status) + CRUD, match-row fetch, backfill-pending query, id-backfill
update (COALESCE — fills NULLs only).
- Guard: _wishlist_blocklist_reason at the top of add_to_wishlist — every
auto-acquisition path funnels through it, so one check covers watchlist,
discography backfill, repair, manual add. Fails OPEN (a guard error never
blocks a legitimate add).
- Discovery unified IN: legacy discovery_artist_blacklist is migrated into the
blocklist on upgrade (replicated to every profile so no global ban silently
stops working; idempotent; legacy table kept for rollback). Discovery reads
(hero + personalized-playlist SQL) now union the blocklist, so a new-modal
ban filters discovery too.
Tests: 13 on the pure matcher (cascade, id-vs-name rules, source isolation,
precedence) + 10 on the DB/guard (CRUD, profile isolation, dedup, backfill,
end-to-end wishlist refusal + cascade + the discovery migration upgrade path).
50 blocklist/personalized tests pass.
