The Download Discography modal exposed only Albums/EPs/Singles, its EPs toggle did
nothing, and Live/Compilations/Featured were missing — so you couldn't fine-filter
a bulk download the way Artist Detail lets you browse.
Root cause: the modal's endpoint (/api/artist/<id>/discography) used the base
get_artist_discography, which lumps EPs into singles, and the modal only read
{albums, singles} — so the EPs bucket was always empty (dead toggle). It also had
no content-type (Live/Compilation/Featured) classification at all.
- Backend: the endpoint now uses get_artist_detail_discography — the SAME split
Artist Detail uses — and returns a separate `eps` list.
- Frontend: read `eps`; tag each card with data-is-live/compilation/featured via a
new shared _classifyReleaseContent() (also adopted by the Artist Detail cards so
the two can't drift); add Live/Compilations/Featured filter buttons; combined
category+content filtering. The download payload is built from VISIBLE checked
cards, so every toggle now actually changes what downloads.
- Regression test: get_artist_detail_discography splits an EP into the eps bucket.
Reported by @Lysticity: opening Settings reset the whole config to defaults. The
chain: GET /api/settings 500s (their env: ConfigManager missing redacted_config)
-> loadSettingsData() called response.json() WITHOUT checking response.ok, so the
error body {"error": ...} was treated as settings -> every field populated as
`settings.x?.y || ''` blanked to defaults -> autosave then wrote those defaults
over the real config.
Fix (settings.js): bail BEFORE touching any field when the response isn't ok / is
an error body, set window._settingsLoadFailed, and guard BOTH save paths
(debouncedAutoSaveSettings + saveSettings) on it. The flag clears on the next
successful load. So any load failure (500, lock, network) now leaves the saved
config untouched instead of wiping it.
The redacted_config method exists in all 2.7.x source + on dev (their 500 looks
like a stale/mismatched build), but the UI must not destroy config on ANY failed
load. Regression test pins redacted_config stays a callable method on the class
(its removal is exactly what 500s the endpoint).
The Favorites collection walker (_iter_collection_resource_ids) broke on ANY
non-200 — including a transient 429. So a rate-limit mid-pagination silently
truncated the collection: the log shows `status=429` then `Retrieved 98/100`,
and the mirror saved 98 of a 524-track favorites list. The auto-sync cycle only
"worked" because it dodged the 429. The regular-playlist paginator already
retries 429; the collection walker didn't.
Fix: retry the same cursor page with backoff (5/10/15/20s, 4 attempts) on 429,
mirroring the playlist paginator; 401/403 still bail (+ reconnect flag), other
non-200 still break. Regression tests: 429 mid-walk completes the full chain;
exhausted retries return partial without hanging; 429 doesn't set reconnect.
Adds a dedicated `get_library_history_unverified()` DB query that fetches
every library_history row with verification_status IN ('unverified',
'force_imported') with no recency cap. This is loaded unconditionally in
`build_unified_downloads_response` — not gated on `len(items) < limit` —
so historical unverified entries are never buried by a busy batch filling
the 200-row general limit, and entries from weeks/months ago aren't lost
in the 50-row recency-ordered history tail. Adds idx_lh_verification_status
for query performance and two regression tests.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Answers "does import respect quality?": yes — the pipeline already runs the
quality gate (check_quality_target) BEFORE AcoustID and quarantines files that
don't meet the profile (unless fallback/downsample is on). This adds an explicit
user switch over that behaviour.
- New config import.quality_filter_enabled (default True). When False,
check_quality_target returns None early so EVERY file imports regardless of
quality; the file is still probed and the library Quality Upgrade Scanner
still flags below-profile tracks. Default preserves current behaviour.
- Settings → Library: the Import Settings group is now a collapsible tile
(same pattern as Post-Processing) and gains the "Only import tracks that meet
your quality profile" toggle at the top, alongside replace-lower-quality and
folder-artist-override.
- settings.js populate/collect the new key; config schema default added.
- Tests: key-aware config stub (a blanket-False mock would wrongly disable the
filter) + a new test pinning toggle-OFF = accept below-target file.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ROOT CAUSE of the quality scanner's "18/18 could not be probed". The shared
resolver (core/library/path_resolver.py) suffix-walked starting at index 1,
which is correct for absolute media-server paths (/music/Artist/... — index 0
is the empty leading segment) but WRONG for SoulSync's own library, which
stores RELATIVE paths like "Asketa/Another Side/track.flac". Index 0 there is
the artist folder; dropping it meant the resolver joined base/Another Side/...
(no artist) and nothing ever matched — so every library track came back
unresolved and the probe opened a relative path that didn't exist from CWD.
Start the suffix walk at index 0 so the FULL relative path is tried first.
Safe for absolute paths (i=0 yields base//Artist/... which harmlessly misses
and falls through to i=1) and Windows drive parts (E: fails on POSIX, falls
through). Other tools (orphan/fake-lossless detectors) were unaffected because
they os.walk the transfer folder directly and never used this resolver.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1) _resolve_library_file_path now tries the FULL relative path (index 0)
against each base dir, not just suffixes. The library scanner stores clean
"Artist/Album/Track.flac" paths; skipping index 0 dropped the artist folder
so the file never resolved — every quality-scanner probe failed ("20/20
could not be probed"). Now they resolve under the transfer/library dir.
2) Quality gate moved BEFORE the AcoustID check in post_process_matched_download.
- A wrong-quality file is rejected without paying for an AcoustID fingerprint.
- context['_audio_quality'] is set before either gate quarantines, so the
real quality is recorded on the sidecar for EVERY quarantine trigger —
it's known when reviewing/approving any quarantined file.
- force_import still never fires on a quality mismatch (only AcoustID).
normalize_import_context mutates in place, so the moved block keeps its
context fields. New test pins the order + that AcoustID isn't run on a
quality reject.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Diagnostics revealed the real cause: the tracks table stores file_path
RELATIVE to the library root (e.g. "Asketa/Another Side/01-01 - Another
Side.flac"), so probing the raw path failed for the entire library — every
track came back unprobeable and was left unflagged ("20/20 could not be
probed").
The scanner now resolves each path via _resolve_library_file_path (checks
transfer/download/library dirs, same helper the rest of the app uses) before
probing, falling back to docker_resolve_path. Injected via deps for testability.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A task stuck in 'post_processing' past the cutoff was force-marked 'completed'
("assume it worked"). In a large batch, post-processing (AcoustID + quality +
import) is serialized and backs up, so tasks sit in post_processing while merely
QUEUED — then got falsely completed, showing as downloaded with no file on disk
(/Transfer empty).
Now: the cutoff is 30 min (was 5) so legit backlog isn't cut off, and when it
does fire the task is only completed if it actually produced a file
(final_file_path exists on disk) — otherwise marked failed (honest + retryable).
Applied at both stuck-detection sites (check_batch_completion + _v2).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The library quality scanner judged quality by FILE EXTENSION only
(get_quality_tier_from_extension) and read the legacy v2 `qualities` dict —
so every FLAC was "lossless tier 1" regardless of bit depth / sample rate. It
could never flag a 16-bit FLAC as upgradeable under a 24-bit profile, and it
ignored the v3 ranked_targets entirely. Completely inconsistent with the
download guard.
Now both share one core:
- selection.targets_from_profile(profile) — single profile→targets conversion
(v2→v3 migration), reused by load_profile_targets.
- selection.quality_meets_profile(aq, targets) — strict: meets iff the real
measured quality satisfies a ranked target (fallback ignored — it's a
download concession, not a definition of "good enough").
- guards.check_quality_target refactored to use both.
- quality_scanner probes real quality (probe_audio_quality) and checks against
the v3 targets via quality_meets_profile. Extension tier kept only as a
fallback label when a file can't be probed.
Result: the scan flags exactly what the download gate would reject — 16-bit
when you want 24-bit, wrong sample rate, MP3 when you want FLAC.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A file quarantined for QUALITY (e.g. MP3-VBR rejected by a FLAC-only profile)
showed up in BOTH the Completed and Quarantine tabs. Cause: the verification
wrapper (post_process_matched_download_with_verification) handled the
_acoustid_quarantined / _integrity_failure_msg / _race_guard_failed markers but
NOT the quality marker _bitdepth_rejected (nor _silence_rejected). A quality
quarantine leaves no _final_processed_path, so the wrapper hit the
"no final path — assuming success" branch and marked the task Completed.
Unlike acoustid/integrity (retry driven by the wrapper), the inner pipeline
already owns the quality/audio-guard outcome — it quarantines then re-queues the
next-best candidate or marks the task failed. So the wrapper now just returns
when it sees _bitdepth_rejected/_silence_rejected, without marking completed
(which clobbered both the quarantine state AND any successful retry).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The audiophile preset (fallback_enabled=False) still shipped a "FLAC 16-bit"
target in its ladder, so 16-bit FLAC matched and imported even though the name
implies hi-res-only. Split the ladder: audiophile now uses a strict 24-bit FLAC
list; balanced keeps 24-bit + 16-bit + MP3. Gives users a one-click strict
"24-bit only" profile that actually rejects 16-bit/lossy.
Not a matches_target bug — that correctly rejects 16-bit vs a 24-bit target;
the leak was the preset's target LIST including 16-bit (+ fallback accepting
off-list lossy like MP3-128).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root cause of "completed/failed/unverified don't show during a running
batch, only after it ends" (F5 didn't help): build_unified_downloads_response
sorted live tasks active-first (downloading/searching/queued = priority 0-3,
completed/failed = 4-7) then truncated the whole array at items[:limit] (300).
During a busy batch the active+queued tasks filled the limit and pushed every
terminal task off the end, so /api/downloads/all never returned them — the
Completed/Failed/Unverified tabs filter client-side and had nothing to show.
Fix: `limit` now bounds only the persistent-history tail. Live in-memory
tasks are always returned in full — they're already bounded by the 5-min
cleanup automation, and array order is presentation-only since the page
filters per tab client-side.
Verified with a repro (320 queued + 1 completed + 1 failed → terminal rows
were absent at limit=300; now present).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three fixes from on-device testing of best-quality mode:
1. clear_completed_local no longer prunes terminal tasks that belong to a
STILL-ACTIVE batch (one with non-terminal work remaining). The 5-min
"Clean Completed Downloads" automation was yanking completed/failed/
unverified rows out of download_tasks mid-run — and failed/cancelled
aren't in library_history — so they only reappeared after the batch
ended. Now the whole active batch stays intact until it finishes.
2. search_all_sources runs every source CONCURRENTLY (asyncio.gather)
instead of sequentially, so the pool waits only for the slowest source
(e.g. usenet/Prowlarr) in parallel rather than summing all latencies.
3. The pool log now reports per-source contribution counts
(e.g. "usenet=0, hifi=11, soulseek=1") instead of just echoing the
chain, so a release-level source that returns nothing for a track-title
query is visible rather than appearing to have been searched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds an opt-in search strategy toggle in the Quality Profile:
- priority (default): unchanged — first source in the hybrid chain that
meets a quality target wins.
- best_quality: pool candidates from EVERY source per query and download
them best→worst by actual audio quality; source order only breaks ties.
Implementation reuses existing plumbing so the retry system is untouched:
- engine.search_all_sources pools raw tracks across all configured,
non-exhausted sources (no first-source short-circuit).
- candidates.order_candidates: new quality_first sort path — profile
quality rank dominates, confidence/peer signals break ties. Priority
path is byte-for-byte unchanged (regression-locked by tests).
- task_worker passes quality_first + targets through; skips the redundant
hybrid-fallback block in best-quality mode (pool already covered it).
- Per-source retry budgets unchanged: a source that spends its budget is
added to exhausted_download_sources and thus dropped from the whole
pool. Independent of post_processing.retry_exhaustive.
- Query generator NOT touched.
Also clarifies the "Allow fallback" setting wording: it accepts OFF-LIST
quality as a last resort (not "walk down my list"), and notes that
lossy_copy.downsample_hires also bypasses the quality gate — the cause of
16-bit/MP3 files slipping through a 24-bit-only profile.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Some Monochrome instances only have 30-second Tidal DOWNLOAD access: the HLS
variant playlist for a 220s track comes back as ~30s of segments + ENDLIST
(verified live on us-west.monochrome.tf — lossless=30s, hires=403). The client
downloaded that 30s file, which then got quarantined by the new audio guard.
Detect it at manifest time: sum the playlist's EXTINF runtime and compare to
the track's real duration (get_track_info). When the playlist is < 85% of the
track, decline the manifest and rotate the instance, so the download falls
through to a real source (Soulseek/Qobuz/Tidal/Deezer) instead of fetching a
preview. Best-effort — unknown duration disables the check (the post-download
audio guard remains the safety net).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The actual HiFi/Monochrome bug isn't silence padding — it's a TRUNCATED file:
the container claims the full length (e.g. 3:08) but only ~30s of audio
decodes. silencedetect finds nothing (there's no silent audio, just missing
audio) and ffmpeg's time= even reports 0 with no error, so the duration and
quality guards all pass.
Detect it by decoding and comparing the real audio length (astats sample
count / sample rate) against the container duration: reject when the real
audio covers < 85% of the claimed length. detect_broken_audio() runs this
truncation check first, then the silence-ratio check. Wire it into the guard
that runs at the integrity/length verification point.
Verified on the real file: 'only ~30s actually decodes of a 188s file (16%)';
a normal 180s file is not flagged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
HiFi/Monochrome HLS assembly can produce a file with the correct container
duration but only ~30s of real audio + silence padding — the duration and
quality guards both pass, so nothing caught it until you listened. Add
core/imports/silence.py: ffmpeg silencedetect over the audio, reject when the
silent fraction exceeds 50%. Wire it into the post-download pipeline with the
same quarantine + next-candidate retry pattern as the quality guard
(trigger='silence'), and surface it via import_rejection_reason. Fails open
when ffmpeg/mutagen are unavailable so tooling problems never quarantine a
legit file.
Also mark 'quality filter' and 'silence guard' failures as recoverable
quarantine rows in the downloads UI (were shown as plain failures).
Verified end-to-end: a 30s-tone + 180s-silence FLAC is flagged '86% silence
(only ~30s audible of 210s)'; a 210s tone passes. 7 parser unit tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Remove the per-source download-quality dropdowns (Tidal/HiFi/Qobuz/Deezer/
Amazon) — with the global ranked-targets system they were redundant and
conflicting. Add quality_tier_for_source(): picks the LOWEST source tier
that satisfies the user's top target (respects the quality ceiling, saves
bandwidth) or the source's max as best effort. Every source's search +
download + retry path now derives its tier from the global profile instead
of config_manager.get('<source>_download.quality').
Settings keep the per-source allow_fallback toggles; the quality selects are
replaced with a note pointing at Quality Profile.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
check_flac_bit_depth now delegates to check_quality_target, which probes the
real file and treats bit depth as a MINIMUM (24-bit satisfies a 16-bit
target) — the old context-string parsing, per-quality bit_depth_fallback, and
'reject higher bit depth' semantics are gone. Rewrite the wrapper tests to
the probe-based model and update the rejection-reason assertion to the
unified 'quality filter' wording.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Quality guard: rejects with a 'file is X, wanted Y' reason (the string the
track-detail modal surfaces), accepts when a target is met or fallback is on,
skips when unprobeable.
force_import isolation: the 'quality' bypass must not skip the AcoustID check
and vice-versa; a quality reject persists trigger='quality' (not 'acoustid')
in the sidecar — so a quality mismatch never routes through the force_import
path (reserved for AcoustID version-mismatch).
Model: lossy matches a MINIMUM bitrate (>=, a range); lossless matches on bit
depth + sample rate, never exact bitrate, so a FLAC's varying bitrate (mono /
compression) can't falsely reject it. v2->v3 migration preserves order.
47 passing across the quality + guard suites.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add core/quality/selection.py: rank_with_targets() returns (ranked,
satisfied) where satisfied = a candidate meets a real target (strict).
load_profile_targets()/rank_for_profile() are the DB-backed wrappers.
search_with_fallback now skips a source that can deliver no target-meeting
quality and escalates to the next (source priority still wins among
satisfying sources; first source's results kept as fallback unless the
profile disables it). Returns RAW tracks — the satisfied check is a coarse
source gate; match-filtering + final ranking stay in the orchestrator so
the correct track is never pruned. Ranking is fail-open: a ranking error
never drops a source's real results.
Tested: rank_with_targets satisfied/fallback matrix + engine escalation,
stop-on-first, raw-not-pruned, fallback on/off. Amazon field test updated
for the corrected format token.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add core/quality/source_map.py centralising each source's tier->AudioQuality
mapping (Tidal/HiFi tiers, Qobuz real kHz/bit-depth, Deezer codes, Amazon
codec/tier). Add TrackResult.set_quality() to merge a mapped AudioQuality
onto a result. Wire HiFi, Qobuz, Deezer, Tidal, Amazon search results to
stamp real sample_rate/bit_depth so the global ranker no longer relies on
crude kbps heuristics for streaming sources. Fixes Qobuz/Amazon display
labels ('FLAC 24-bit/192kHz', 'Lossless') breaking format derivation.
Tested: 22 passing (mappers + set_quality merge semantics).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
wolf's report: Spotify shows 'Calma - Remix', Find & Add searches that literal
string, but the library stores the track as just 'Calma' (only the 3:58 duration
marks it the remix). The literal LIKE '%calma - remix%' misses, so it fell to the
OR-fuzzy fallback which floods on the common word 'remix' (20 unrelated '... remix'
hits). Dropping '- Remix' (searching 'Calma') finds it instantly.
Fix: search_tracks (and api_search_tracks) now retry on the BASE title — the part
before Spotify's ' - ' version separator — BEFORE the OR-fuzzy flood. So
'Calma - Remix' resolves to 'Calma' (or 'Calma (Remix)') and the noise fallback is
never reached when the base matches. New core.text.title_match.base_title_before_dash
(splits the first spaced ' - '; leaves bare hyphens like 'Up-Tight' alone).
Tests: pure helper (3) + real-DB integration reproducing the Calma case, the
parenthesized-remix variant, plain-title-unaffected, and no-flood (4). 64
search/match tests green.
The Deezer ARL field round-trips a redaction sentinel for a saved-but-untouched
secret (shown as dots). The save path already guards against the sentinel
overwriting the real token (ConfigManager.set), so the ARL was never actually
lost — but the connection TEST read the field value and sent the sentinel as the
token, so Deezer returned USER_ID=0 ('Invalid ARL token') after navigating away
and back. That false failure made it look like the ARL kept resetting.
Fix:
- ConfigManager.resolve_secret(key, posted): empty/sentinel posted value -> the
stored value; a real string -> a genuine new secret. Reusable for any secret
connection-test (single source of truth).
- /api/deezer-download/test now resolves the effective ARL via resolve_secret, so
an untouched field tests the stored token.
- testDeezerDownloadConnection() strips the sentinel before sending (untouched ->
empty -> backend uses the saved token).
Seam/regression tests for resolve_secret (sentinel/empty/none -> stored, real ->
passthrough, nothing stored -> empty). JS integrity 64 green.
Enrichment matched artists by NAME ONLY (0.85 gate), so for a common name
('Rone' has ~5 artists) it stored whichever the source ranked first — often the
wrong one, which then drove a wrong/sparse library 'Standard' discography while
'Enhanced' (the real owned albums) showed the full set.
Fix — use the decisive signal the library already has (the albums you OWN):
- worker_utils: pick_artist_by_catalog() + catalog_overlap_score() +
owned_album_titles()/release_titles(). When 2+ candidates clear the name gate,
fetch each one's catalog and choose the one overlapping the owned albums; falls
back to the current best-by-name pick when there's nothing to disambiguate or
no overlap (so the common single-candidate path makes no extra API calls).
- Wired into Spotify (covers Spotify-Free, same client), iTunes, Deezer (now
multi-candidate search_artists + get_artist_info store), and MusicBrainz
(match_artist gains owned_titles; release-groups as the catalog).
Re-match path (#868):
- build_reset_query now also clears the stored source-ID column for artist/album
item resets — previously a 're-match' only nulled match_status, so the worker's
existing-id short-circuit re-confirmed the WRONG id and never re-resolved. Tracks
excluded (ids live in tags, not a column).
- MusicBrainz also self-corrects its 90-day name->mbid cache: match_artist bypasses
a cached mbid whose catalog has ZERO overlap with the owned albums, so a re-match
isn't blocked by a stale wrong cache entry.
Tests: shared selector (9), per-worker disambiguation for all 4 sources + MB
backward-compat + MB cache-revalidation (8), reset-clears-id (2). 99 worker/
enrichment tests green.
Four refinements on top of the tiered matcher:
1. Direct source track-ID tier (new top tier): enrichment writes each source's own
track ID into the file tags (spotify_track_id/deezer_track_id/itunes_track_id/...).
If we have the active source's track ID, fetch that exact track by ID via
get_track_details — zero search. Tiers are now: track-ID -> ISRC -> album->track
-> artist+title. _read_file_ids reads ISRC + all per-source IDs in one tag read.
2. Skip already-proposed tracks: a re-run loads existing finding entity_ids for the
job and skips those tracks before any API call (pending stays deduped, dismissed
stays dismissed) — re-runs are cheap.
3. Wrong-version guard: the fuzzy tiers (album-search + track search) reject a
candidate whose length differs from ours by >5s (live/edit/remix with same title).
_load_tracks now selects t.duration; exact tiers (track-ID/ISRC/stored-album-ID)
skip the guard.
4. Tighter album matching: same-title cuts in an album are disambiguated by closest
duration when track_number doesn't decide it.
Findings record matched_via = track_id | isrc | album | search. 30 repair tests pass
(added track-ID tier, duration guard, dedup-skip, and unit coverage).
Replaces the blind fuzzy search with a smart hierarchy that uses the data we
already have, best identity first:
1. ISRC embedded in the file tags (enriched track) -> exact track.
2. Album -> track: use the album's stored source ID (albums.spotify_album_id /
itunes_album_id / deezer_id / musicbrainz_release_id / audiodb_id) when the
ALBUM is enriched (even if the track isn't); else find the album by searching
'artist album', then locate our track in that album's tracklist by normalized
title (track_number breaks ties). Pins the exact album context. (artist->album->track)
3. Plain artist+title search with similarity scoring. (artist->track) — loosest.
_load_tracks now returns dict rows (adds track_number + the album source-id
columns). Findings record matched_via = isrc | album | search. All clients
(spotify/deezer/itunes/discogs) expose search_albums + get_album_tracks with a
uniform {'items': [...]} shape, so the album tier is source-agnostic.
26 repair tests pass (added album-tier + _find_track_in_album coverage).
The job was doing a blind fuzzy search for every low-quality track, ignoring that
enrichment writes each track's ISRC + per-source IDs into the file tags. Now it
reads the file's embedded ISRC and resolves the EXACT track via each source's
'isrc:' search (universal cross-source key), guarded by an ISRC-equality check so
a source that ignores the syntax can't produce a false match — exact track, exact
album context, one call. Falls back to the name/artist fuzzy search only for
un-enriched tracks with no usable ISRC. Findings record matched_via=isrc|search.
4 new seam tests (guard accept/reject, ISRC-preferred-over-fuzzy, fuzzy fallback).
Phase 2 of the redesign. The tool that judged quality by extension and auto-dumped
matches into the wishlist is gone; quality scanning is now the reviewed
quality_upgrade repair job.
Removed:
- Frontend: Tools-page Quality Scanner card, its JS handlers/poller/socket listener,
help tooltip + tour entry (webui index.html, core.js, helper.js, wishlist-tools.js).
- Backend: /api/quality-scanner/{start,status,stop} endpoints, the in-memory state +
executor + 1s socket broadcast, the QualityScannerDeps/run_quality_scanner shim.
- core/discovery/quality_scanner.py: the auto-acting worker + deps class (the shared
match/normalize helpers stay — the new job imports them).
Rewired:
- Automation 'start_quality_scan' action now triggers the quality_upgrade repair job
via repair_worker.run_job_now() (AutomationDeps gains run_repair_job_now, drops the
4 scanner fields). Action block's vestigial scope field removed (scope lives in the
job's settings now). NOTE: the 'quality_scan_completed' trigger no longer fires (the
repair job doesn't emit it).
- Updated all automation test _build_deps helpers + conftest tool-progress harness;
deleted the obsolete worker test. 528 affected tests pass; 6123 collect cleanly.
QUALITY_TIERS / _get_quality_tier_from_extension kept (used elsewhere).
The old Quality Scanner tool judged quality by file EXTENSION only (a 128k and a
320k MP3 looked identical), ignored the bitrate-based quality profile, used min()
of enabled tiers so the default profile flagged the ENTIRE non-lossless library,
and auto-dumped every match into the wishlist with no review.
This new repair job does it properly:
- meets_preferred_quality(): pure, bitrate-AWARE decision honoring every enabled
quality bucket (320 MP3 passes a FLAC+320+256 profile; 128 MP3 doesn't). Floor
is the worst enabled bucket, not the best.
- scans watchlist artists or whole library, finds below-quality tracks, matches a
better version at scan time (reusing the existing tested match helpers), emits a
FINDING showing the match + confidence. Off by default; nothing auto-queued.
- _fix_quality_upgrade apply handler adds the matched track WITH album context to
the wishlist — the user-approved version of what the old tool did silently.
- Transcode/fake-lossless detection intentionally left to the existing Fake
Lossless Detector job.
12 seam tests incl. a regression pinning the default-profile flooding bug. The old
tool is still in place; removing it + rewiring its automation action is the next step.
Two issues in the same path:
1. The shared discovery modal pre-renders one row per track from a
separately-fetched frontend track list, then the poll dropped any backend
result without a pre-rendered row (if (!row) return). When the frontend's
track fetch came back rate-limited/partial (~21) while discovery's own fetch
got all 59, the surplus results vanished. Now the modal CREATES a row for any
result lacking one, so authoritative backend results drive the list (fixes
all sources sharing the modal).
2. get_playlist hydrated a whole relationships page in one _get_tracks_batch
call, but Tidal caps filter[id] at 20/request, silently truncating larger
pages. Chunk to the cap like get_album_tracks already does.
Seam + regression tests (tests/test_tidal_playlist_batch_chunking.py).
Status checks asked is_spotify_authenticated() (official OAuth only) instead of
is_spotify_metadata_available(), so a Spotify-Free primary read as disconnected.
get_primary_source_status had spotify_free awareness but it was dead code:
get_client_for_source('spotify') returns None unless officially authed, so the
free-availability probe never had a client. Fetch the client directly for that
check; add the missing free branch to the dashboard test message. Seam + regression tests.
The reconcile read each completed task's final_file_path to find paths — but not
every import path sets it (the verification worker marks the task completed
without it), so tracks that imported via that path were silently dropped (user
saw 3 of 5 symlinks). Root cause: leaning on a fragile per-task field.
Now reconcile_batch_playlists identifies the organize playlists the batch touched
(its own + any reached via a completed track's source_info provenance) and
rebuilds each from CURRENT library ownership via _rebuild_one_from_db
(check_track_exists over membership). It just asks the library what's owned, so
it's robust to HOW a track imported (modal worker / slskd monitor / verification
worker) and still prunes tracks that left. Takes a db handle; all three callers
pass MusicDatabase().
Reconcile tests rewritten for the DB-rebuild form (organize batch, wishlist
provenance, non-organize skip, plain no-op). 973 downloads/imports/playlist
tests pass.
Symmetric to the post-download reconcile (which handles ADDITIONS): when a
playlist's membership is re-synced (the mirror step — scheduled refresh or the
manual mirror endpoint), rebuild its folder from current membership WITH prune
IF it's organize-by-playlist. So a track that just LEFT the playlist has its
symlink cleaned up the instant membership changes, not only on the next download.
Factored a shared _rebuild_one_from_db (used by the manual 'Rebuild' button and
the mirror hook) + rebuild_mirrored_playlist_if_organized. Gated to organized
playlists, non-fatal at both mirror call sites.
Now the invariant 'folder = the playlist's current owned members' holds on every
change: additions caught at download, removals caught at mirror. 2 new tests
(removed track pruned; non-organized skipped). 985 + 277 tests pass.
Replaces the two organize-only triggers with a single reconcile_batch_playlists
called at both batch-completion points. It groups the batch's newly-resolved
tracks by their per-track playlist provenance:
- the batch's OWN organize playlist → full (re)build with prune, and
- a track that completed for a DIFFERENT playlist (e.g. a WISHLIST fulfilling a
track that belongs to an organize playlist) → ADDED to that folder, no prune.
So a late wishlist arrival now lands in its playlist folder immediately, instead
of only on the next sync/manual rebuild — the folder = the playlist's owned
members, kept true on every ownership change regardless of download path. Uses
the paths the batch already captured (no DB re-match, no waiting on the server
scan/sync). Non-fatal.
3 new reconcile tests (organize full-rebuild, wishlist add-without-prune, plain
batch no-op). 983 downloads/imports/playlist tests pass.
- Settings: 'Playlists Folder' path field (Unlock pattern, separate-root help
text), a Symlinks/Copies selector, and a 'Rebuild playlist folders now' button
(standard test-button style). Wired through PATH_INPUT_IDS / load / save, plus
'playlists' added to the settings save allowlist so it persists.
- POST /api/playlists/materialize/rebuild → rebuild_organized_playlists_from_db:
rebuilds every organize-by-playlist folder from CURRENT ownership, re-matching
each track with check_track_exists (name, not IDs) so it self-heals after a
reorganize / membership change. +1 test.
70 materialize tests + JS integrity pass; settings round-trip wiring verified.
- Routing (step 5): organize-by-playlist tracks no longer set the per-track
_playlist_folder_mode flag, so they import NORMALLY into Artist/Album — exactly
what a normal download does. _playlist_name provenance is kept (origin.py).
- Triggers (step 4): build the playlist folder from the batch's own payload at
both end-of-flow points — the all-owned path in master.py (no downloads, so the
lifecycle never runs) and the batch-complete hook in lifecycle.py (after
downloads). Both gated on playlist_folder_mode, both non-fatal.
Works for the all-owned case (the smack test that did nothing before) and for
mixed owned/downloaded, with no source-ID or mirrored-playlist dependency. The
materialized folder uses the default ./Playlists root + symlink mode until the
Settings UI is added.
Updated the master test to assert the new contract (provenance kept, routing
flag gone). 979 tests pass.
materialize_playlist_from_batch(batch, download_tasks, config) collects the real
on-disk path of every resolved track from the batch's OWN payload — owned via
analysis_results.matched_file_path, downloaded via tasks.final_file_path — runs
each through the playback path resolver (Docker-correct), de-dupes, and hands the
list to rebuild_playlist_folder. Gated on playlist_folder_mode.
No re-matching, no source IDs, no mirrored-playlist lookup — works for any
organize-by-playlist download including the all-owned case. 5 tests. Still
isolated; the triggers wire it in next.
- settings: playlists.materialize_path (separate root, mapped apart from the
music library so the media server never double-scans it) + materialize_mode
(symlink|copy).
- core/playlists/materialize.py: pure filesystem engine that (re)builds a
playlist folder of relative symlinks (or copies) into the real library —
idempotent, prunes stale entries, disambiguates filename collisions, never
escapes the root, and auto-falls-back to copy when the FS can't symlink.
No DB, no app state; ops injectable. 13 unit tests.
Isolated + additive — nothing live calls this yet (stitcher/trigger/routing
come next).
A DB<->filesystem path mismatch (Docker volume change, remount, Music
Paths unset for the container) makes EVERY library file fail to resolve
to a DB track, so the orphan detector flags the whole library as
orphaned. The mass-orphan check only logged a warning and then created
the findings anyway — so a user batch-applying 'move to staging' or
'delete' would relocate or wipe their entire library.
Make it a hard skip (create zero findings) like the dead-file cleaner
and stale-removal paths already do (#828). Centralise the predicate as
is_implausible_orphan_flood() alongside is_implausible_stale_removal()
so the rule lives in one tested place. Small genuine orphan sets still
surface unchanged — only an implausibly large flood (>50% and >20) is
suppressed.
Tests: seam cases for the new predicate + scan-level regressions (mass
mismatch -> 0 findings; small genuine set -> still reported).
Per feedback — instead of two export buttons (one on the watchlist filter bar, one
in the library header), there's now a single "Export" button. The modal gains a
Watchlist | Library scope toggle at the top; switching scope re-fetches and shows/
hides the "library counts" option (library-only). One place, both rosters.
Also relaxed the two export endpoint wiring tests — they asserted an empty DB,
which is false in a shared test run (the artists table may already hold rows); now
they assert a valid JSON array + headers/columns instead. The endpoints are
unchanged and verified against real data.
Extends the watchlist export to the full library. The exporter is now general
(core/exports/artist_export.py, renamed from watchlist_export) — adds tidal/qobuz
links and an extra_fields passthrough, so the library export also carries
lastfm/genius URLs + soul_id, and an optional "library counts" toggle adds owned
album/track counts per artist.
- GET /api/library/artists/export?format=&links=&contents= — pulls every artists
row, normalizes onto the canonical *_artist_id keys, optionally GROUP-BY counts
for album/track totals.
- The export modal is now openArtistExportModal(scope): "Export Library" button in
the library header + the existing "Export" on the watchlist bar (a thin wrapper).
Library mode shows the extra "library counts" toggle.
Tests (11): builder across formats + the new tidal/qobuz links + extra_fields
columns; watchlist + library endpoint wiring. 64 integrity green; ruff clean.
An "Export" button on the watchlist filter bar opens a modal (same aesthetic as the
artist DB-record inspector) to export your whole watchlist roster — each artist's
name + source IDs (spotify / musicbrainz / deezer / discogs / itunes / amazon),
with an optional "external links" toggle that adds the discography URLs built from
those IDs. Live preview, copy, and download in the chosen format.
- core/exports/watchlist_export.py: pure builder (json/csv/txt + links, present-IDs
only, deterministic columns) — the single source of truth, fully unit-tested.
- GET /api/watchlist/export?format=&links= shapes the roster + returns it (with
X-Export-Count / X-Export-Ext headers for the modal).
- Frontend reuses the DB-record helpers (_jsonSyntaxHighlight / _arecCopy).
Tests (8): builder across json/csv/txt, links on/off, present-ids-only, empty +
bad-format fallback, mime/ext, and endpoint wiring. ruff clean; 64 integrity green.
Scoped to the watchlist for v1; library-wide export + a "library contents"
(owned albums/tracks) option are natural follow-ups.
New Aria2 JSON-RPC adapter, alongside qBittorrent / Transmission / Deluge. Aria2's
RPC (default :6800/jsonrpc) maps cleanly onto the uniform adapter contract:
- the --rpc-secret token leads every call as "token:<secret>" (no username — the
secret uses the existing password field),
- addUri returns a GID (our torrent id); tellStatus → TorrentStatus with state
mapping (active→downloading, or seeding once the payload is complete; waiting→
queued; etc.),
- remove picks forceRemove vs removeDownloadResult by status, and (since aria2
doesn't delete files on remove) unlinks the file paths itself for delete_files,
- bare-host URLs get /jsonrpc appended.
Wired into adapter_for_type + the Settings dropdown (with a help note: port 6800,
secret in the Password field). All adapter methods go through the same interface,
so the stall/orphan handling and downloads pipeline work unchanged.
Tests (9): registry wiring, state mapping (incl. active→seeding), token-prefixed
params, /jsonrpc fixup, status parse (+ name fallback, no div-by-zero). 126 torrent
tests green; ruff clean.
A maintenance job to keep the music library tidy — finds empty folders left behind
by imports/relocations/deletions (empty artist/album dirs, or dirs holding only OS
junk like .DS_Store/Thumbs.db) and removes them.
Safety is the focus (deleting directories is destructive):
- only TRULY empty folders are flagged — a folder with a cover image or any audio
is never touched; only OS-junk files count as "no real content" (a setting),
- the library root + symlinked dirs are never removed,
- walks bottom-up so a parent left empty by its removable children cascades,
- the apply handler RE-CHECKS emptiness at delete time, so a folder that gained a
file between scan and apply is left alone.
dir_is_removable + remove_empty_folder are pure/injectable seams. Wired through the
job registry, repair_worker apply handler (_fix_empty_folder), fixable-types, and
the findings UI. Opt-in (default off), weekly interval.
Tests (10): removable decision (empty / real-file / surviving-subdir / junk-only /
strict mode) + apply re-check (removes empty + junk, refuses content/root/symlink).
Repair + integrity suites green; ruff clean.
A transient ping failure (network blip, Navidrome busy mid-scan) makes
_setup_client null out the configured creds, and _connection_attempted then
latches the client "disconnected" — so is_connected() returned False forever until
the user hit the manual Test button to re-read config. That's the reported
"disconnects every 5-10 min, reconnects instantly on Test."
Fix: ensure_connection no longer latches on a failed attempt — once a short
throttle (_RECONNECT_THROTTLE_S = 20s) elapses it re-attempts, and is_connected()
triggers that retry whenever it's currently disconnected. So a blip recovers on its
own within the next status check, no manual reconnect. The throttle prevents ping
storms when Navidrome is genuinely down.
Tests: transient failure self-heals after the throttle (and doesn't re-ping within
it); a connected client never re-pings; first connect attempts once. 115 navidrome/
media-server tests green.
Invariant: while security.require_login is on, every profile must have a login
password or it's locked out. Previously only the admin's own anti-lockout existed,
so members could be stranded (created without a password, or login flipped on while
passwordless members existed). Closed all the write-points:
core/security/login_provisioning.py (pure policy, single source of truth):
- members_without_password(profiles) — non-admin profiles that can't sign in
- create_needs_password(require_login) / removing_password_strands(require_login)
Wired into web_server:
- create_profile: while login is on, a new member must be given a password (400
otherwise) and it's set on creation.
- enable-login (settings save): refuses to turn login on while any member lacks a
password — lists them — same shape as the existing admin anti-lockout.
- set-password: refuses to CLEAR a password while login is on (would strand them).
UI: Create Profile form gains a login-password field (alongside the optional PIN);
the Manage Profiles per-member password button (prior commit) covers existing
members + changes.
Tests: pure policy seam + endpoint enforcement (create blocked w/o password when
on, allowed w/ password, no friction when off, clear blocked when on). 442
profile/settings/auth tests green; ruff clean.
Closes the gap where "Require login" was effectively admin-only: a member with no
password can't sign in and can't bootstrap one themselves (can't log in to reach
the setting). The set-password endpoint already allowed admin→anyone — this adds
the missing UI.
Each non-admin row in Manage Profiles gets a lock-icon button that opens an inline
form to set / change / remove that member's LOGIN password (separate from the
quick-switch PIN), with a confirm field + a hint explaining when it's used. Admin
rows don't get it (admin manages their own in Settings → Security, which keeps its
anti-lockout). textContent-only rendering, so a profile name can't inject markup.
Test: admin sets a member's password → the member can then authenticate
(verify_profile_password) and a wrong password fails; admin can clear it back to
no-login. 64 script-split integrity tests green.
Post-processing applies ReplayGain only to slskd/WebUI downloads — content added
via Lidarr, the REST API, or by hand never got it, and there was no way to (re)apply
RG to existing tracks or fix ones where analysis failed (raised in #437 + comments).
New ReplayGain Filler repair job (sibling of Lyrics/Cover Art fillers): scans for
tracks with no ReplayGain track-gain tag and creates a finding per track; the scan
only READS tags (cheap) and no-ops when ffmpeg is absent. Applying a finding runs
the same ffmpeg ebur128 analysis the import pipeline uses (gain = ref - LUFS) and
writes the RG tags in place — no moves, no re-matching. Opt-in (default off),
schedulable like the other maintenance jobs.
Wired: job registry, repair_worker apply handler (_fix_missing_replaygain) +
fixable-types, and the findings UI (label / fix-button / detail rows).
Tests: pure needs_replaygain decision (missing/blank/present/+0.00-is-tagged) +
the apply handler's analyze→compute→write seam with the pipeline gain formula,
ffmpeg-absent + missing-file guards, and registration. 93 repair tests green.
The integration test only exercised the launch-PIN path; the actual #852 report is
the "Sign in to SoulSync" username/password modal. Add login-mode cases: socketio
connect is rejected when require_login is on + unauthenticated, allowed once
login_authenticated. Confirms the WS gate covers both overlays.
A small glowing button at the bottom-right of the artist hero (library artists
only) opens a programmer-style modal showing the COMPLETE artists DB row — every
source id + match status, cached bios / tags / similar / urls, soul_id, timestamps,
the lot (62 columns) — plus owned album/track counts.
- Backend: GET /api/artist/<id>/record returns the full row with JSON-text columns
(genres, aliases, lastfm_tags/similar, discogs_urls, …) decoded into real
arrays/objects, + album/track counts. 404 for non-library artists.
- Frontend: editor-themed modal (Tokyo-night tokens) with a Fields tab (copyable,
filterable key/value rows) and a syntax-highlighted JSON tab. Copy-all-as-JSON,
per-value copy (HTTP/Docker clipboard fallback), and Save .json. Esc / click-out
to close. Helpers namespaced (_arecEsc) so they can't clobber the shared globals.
Tests: endpoint returns the full row with decoded JSON + counts; 404 for a missing
artist. 64 script-split integrity tests still green; ruff clean.
PR #853 added artist album-list caching to Deezer, but unlike the Spotify path it
had no equivalent of the truncated-fetch guard: Deezer paginates, and a transient/
malformed response on page 2+ (artist with >100 albums) broke the loop and cached
the PARTIAL list as the full discography — serving an incomplete album list from
cache until TTL.
Fix: track whether pagination finished cleanly. A malformed/empty-of-data response
mid-walk now clears a `complete` flag and the artist→album-LIST is cached only when
complete. Individual album entities still cache regardless (each is complete; we
just have fewer). A clean end (short page / empty page / reached limit) still caches
as before.
Tests: a page-1-ok / page-2-errors walk no longer caches (second call refetches
instead of serving a permanently-incomplete list); a clean two-page walk still
caches (happy path intact). 181 deezer/metadata tests green.
noldevin: a magnet stuck "downloading metadata" ran 11h despite a 15-min stall
timeout, got cleared from SoulSync but left active in qbit, then re-grabbed as a
duplicate. Two bugs:
1. Stall never fired on metaDL. StallTracker reset its clock on any `downloaded`
byte increase, but a metaDL torrent's byte counter still ticks up from DHT/peer
protocol overhead while making no real progress — so the clock reset forever.
Fix: the byte counter only counts once metadata is in (size>0). During the
metadata phase (size==0) the only thing that counts as progress is *obtaining*
metadata, so a magnet that can't even do that within the timeout is correctly
flagged stalled. size=None preserves the old byte-only behavior (back-compat).
2. Orphaned in qbit. The monitor's stall exit removed the torrent, but the `error`
exit and the 6h deadline exit only marked the download failed — leaving the
torrent active in qbit, untracked here, so SoulSync re-grabbed the same dead
torrent (qbit logs the duplicate-add). Fix: both terminal exits now run
_cleanup_torrent (shared with the stall path), which removes+deletes (abandon)
or pauses per the stall action — nothing is left orphaned.
Tests (10 new): metaDL byte-noise no longer resets the clock (stalls at timeout);
obtaining metadata resets it; real byte-progress still tracked after metadata;
_cleanup_torrent removes+delete_files on abandon / pauses on pause / no-ops on
empty hash or no adapter / swallows a client error. 151 torrent tests green.
The #832 fix enforces the launch PIN / login via a Flask before_request hook, but
that hook does NOT run for the socketio handshake — empirically a normal endpoint
401s while /socket.io/ returns 200 with the gate on. So removing the client overlay
(Safari "Hide Distracting Items", devtools) + opening a socket streams live data
(downloads, logs, dashboard, notifications) completely unauthenticated.
Fix: the socketio connect handler now enforces the same check and returns False
(rejects the connection) when a gate is active and the session isn't verified.
Rejecting connect blocks every downstream WS event (subscribe/join), so all live
data is covered. core/security/ws_gate.is_ws_connection_blocked is the pure seam:
login mode (when on) > launch PIN > open, mirroring the HTTP gate exactly. Fails
OPEN on a config-read error, same as the HTTP gate.
Audited every other surface empirically with the gate on + unauthenticated: SSE
streams, catch-all pages, library/dashboard data, admin endpoints, search,
image-proxy, audio-stream (incl. a /etc/passwd traversal probe) all 401; /api/v1
key-gated. The WebSocket was the only hole.
Tests (10): pure gate logic (login>pin precedence, all on/off combos) + real
socketio.test_client integration — connect rejected when gate on + unauthenticated,
allowed when gate off or PIN verified.
Root cause (from the reporter's app.log): a ListenBrainz weekly playlist syncs
through the in-memory youtube_playlist_states discovery machine. When that live
state is lost — a Docker restart, or the discovery process ending while the user
waits for the media-server scan — the DB discover-download snapshot survives but
the live state is gone. Every recovery action (Cancel/Reset/Delete) then hit
`key not in states` and returned 404 "YouTube playlist not found" (hence the
confusing "Youtube" on a ListenBrainz playlist), leaving the playlist permanently
wedged with no way to dismiss or re-sync. Works for the maintainer because a
single session with no restart keeps the live state alive.
Fix — these are cleanup ops, so "the thing is already gone" is SUCCESS, not 404:
- cancel_sync core (shared by YouTube + ListenBrainz + Tidal/Deezer/Qobuz/...) →
missing key returns idempotent success.
- reset_youtube_playlist / delete_youtube_playlist → same.
The playlist becomes recoverable: Cancel/Reset clears the dead state and the user
re-syncs fresh.
Tests: cancel_sync core (missing key = idempotent 200 not 404; present key still
cancels + clears the worker + reverts phase); endpoint-level idempotency for
cancel/reset/delete; updated the old test that locked the 404 wedge. 834 sync/
discovery tests green.
The 'retag' fix corrects a mismatched file's tags/DB but leaves it in the WRONG
artist/album folder, so the library shows the right title while the file sits under
the previous track. AcoustID yields only title+artist (no reliable album), so an
in-place move has no safe target.
New 'relocate' action: retag the file, move it into Staging, drop the stale tracks
row, and clean up the emptied folder. The auto-import worker (which watches Staging)
re-identifies it with full metadata and files it correctly — reusing the import
pipeline instead of guessing a destination.
- core/repair_jobs/relocate.py: pure, injectable orchestration (retag -> move ->
drop row) + collision-safe staging_destination. Row is dropped only AFTER a
successful move, so a failed move never orphans the library entry.
- _fix_acoustid_mismatch gains the 'relocate' branch (thin wrapper: resolve path,
staging dir, drop-row closure, empty-parent cleanup).
- UI: "Relocate" button on the AcoustID-mismatch fix modal.
Tests (8): staging-dest collision suffixing; relocate happy path; tag-write failure
still relocates; FAILED move does NOT drop the row; no-tags skips write; a real
file move through safe_move_file; and a handler integration test (file moved to
staging + tracks row deleted end-to-end). Repair + integrity suites green.
Locks the one-time backfill that derives verification_status for pre-column
library_history rows from acoustid_result: pass->verified, skip->unverified,
fail->force_imported; never overwrites an existing status (NULL-only); leaves
no-acoustid rows NULL; idempotent across repeated inits. Exercises the real
_initialize_database path (clears the per-process init guard to re-trigger it).
resolve_history_audio_path drives a DESTRUCTIVE delete (os.remove), but lived
endpoint-bound in web_server with zero tests. Lifted to core/matching/history_paths
with injected effects (exists / resolve_library_path / lookup_titled_paths) so the
fallback chain — and the collision-safety that stops delete() from removing the
wrong same-title file — is a clean importable seam. web_server now wraps it (DB
lookup + os.path.exists + prefix resolver injected); behavior preserved.
9 tests lock it: recorded-path hit, prefix-resolve fallback, single tracks-table
candidate, and the safety rules — multiple same-title candidates with NO artist ->
None (refuse to guess), artist filter picks only the matching path, artist named
but unmatched -> None, no-title/empty-lookup -> None. Full suite green (5906).
The candidate matcher rejected valid downloads of titles with a '/' or ':' (e.g.
Sawano's "You See Big Girl / T:T") because the unified normalize() removed those
chars ("t:t" -> "tt") while keeping the '_' that source filenames substitute for
them ("T_T" -> "t_t") — the asymmetry tanked the similarity score. Now '/ \ : _'
all map to spaces before the strip, so "/ T:T" and "_ T_T" both normalize to "t t".
Verified on the real library: similarity for the Sawano pair 0.927 -> 1.000; only
348/40786 strings (0.85%, all containing those separators) change; worst-case
joined-variant match (e.g. "12:05" vs "1205") stays 0.889, well above the 0.70
title threshold — no match regressions. Fixes the matching half of #851.
The merged PR left the review-queue's mutating endpoints ungated. Both now require
admin, matching the Phase 3 destructive-endpoint convention:
- /api/verification/<id>/delete (os.remove + drops the history row) — @admin_only,
so a non-admin on a login/multi-profile instance can't delete library files.
- /api/verification/<id>/approve (flips verification_status + writes the tag) —
@admin_only; also wrapped its DB writes in `with db._get_connection()` for
rollback-on-error + codebase consistency (was a bare conn).
Read/playback endpoints (stream/play/compare/entry/config) stay open — the app's
LAN-read model. Tests: non-admin gets 403 on delete + approve; admin isn't blocked.
The Your Albums Discogs collection sync stored bare release_ids while
search/discography now store tagged ('r<id>') ones (#848). This didn't cause a live
bug — the pool dedups by normalized name, and discogs_release_id is only ever
re-fetched (which handles bare via release-first) — but it left the "type travels
with the ID" invariant half-applied. Now the collection sync tags its IDs too, so
every stored Discogs album ID is uniform and a future ID comparison can't be tripped
by mixed forms.
Collection items are always releases, so they're tagged 'r<id>'. Test locks the
stored value + that a tagged collection ID routes only to /releases (never /masters).
Closes the forgot-login-password gap. A per-profile recovery question + answer lets
a locked-out user reset their own password.
- DB: additive recovery_question + recovery_answer_hash columns (idempotent
migration). set/get-question/verify/has methods; answer is hashed (pbkdf2) and
matched forgivingly (trim + lowercase + collapse whitespace). No recovery set →
never verifies.
- Endpoints (allowlisted in the login gate so they work pre-auth):
GET /api/auth/recovery-question?username= (generic 404 when absent),
POST /api/auth/recovery-reset {username, answer, new_password} — brute-force
limited; a correct answer sets the new password + authenticates the session.
POST /api/profiles/<id>/set-recovery (admin or self) to configure it.
Tests: set/get/verify, forgiving match, hashed-not-plaintext, no-recovery-never-
verifies, full reset flow (wrong answer rejected + password intact; correct answer
resets), unknown-user 404. 25 tests pass. Next: the Settings + login-screen UI.
Pins the zero-impact guarantee: with require_login off (default), the launch PIN
still enforces exactly as before (the login deferral doesn't fire), and with both
off there's no gate at all (today's behavior).
The UI that makes opt-in login usable. Off by default → your LAN setup is unchanged
(none of this appears unless security.require_login is on).
- Login screen overlay (reuses the launch-PIN styling): username + password →
/api/auth/login → reload into the app. Shown when /api/profiles/current reports
login_required (checked before profile selection).
- POST /api/profiles/<id>/set-password (admin, or self) to set/clear a login
password, distinct from the PIN.
- Settings → Security: "Login password (admin account)" field + a "Require login"
toggle (with the anti-lockout note). Wired into the existing settings load/save.
- Sign-out button in the profile bar, revealed only in login mode (login_mode flag
on /api/profiles/current); soulsyncLogout() → /api/auth/logout → reload.
Tests: set-password sets/clears + verifies; /api/profiles/current signals
login_required. 20 login/password tests pass; 64 script-split integrity pass.
Remaining (small follow-up): a password field in the Manage Profiles edit form so
admins can set OTHER profiles' passwords from the UI (the endpoint already exists).
The backend auth for opt-in username/password mode (security.require_login, default
off → zero change; the launch PIN + picker behave exactly as today).
- core/security/login_gate.py: pure gate (mirrors launch_lock) — when login mode is
on, an unauthenticated session reaches only the page shell, /api/auth/login,
/api/auth/logout, /api/profiles/current, /api/setup/status, and the key-authed
/api/v1 API. Deliberately does NOT expose the profile list pre-auth (you type your
name, not pick from a roster).
- _enforce_login before_request enforces it; _enforce_launch_pin no-ops when login
mode is on (login replaces the shared PIN, per design).
- POST /api/auth/login (username = profile name, case-insensitive; brute-force
limited per IP; generic error so names don't leak) + POST /api/auth/logout.
- Anti-lockout: the settings save refuses to turn ON login mode until the admin
account has a password.
Tests: gate blocks→login→access→logout→blocked; case-insensitive username; wrong
password / passwordless profile / unknown user all 401 generically; login list not
exposed pre-auth; can't enable login without an admin password. 12 tests pass. Next:
the login screen + set-password UI + the toggle (increment 3).
Opt-in username/password login — profiles become real accounts. This is the data
layer: a per-profile login password, kept SEPARATE from the quick-switch PIN
(different security purpose; a 4-digit PIN must not become the password guarding a
public instance).
- Additive migration: profiles.password_hash column (idempotent, metadata-flagged).
- set_profile_password / verify_profile_password / profile_has_password /
get_profile_by_name (the login username = profile name, unique + case-insensitive).
- Security default: a profile with NO password is NOT loginable (verify returns
False) — unlike the PIN where "no PIN = always valid". You can't authenticate to
an account with no credential.
Tests: migration adds the column; set/verify; no-password-never-loginable; clearing;
name lookup; and password is fully independent of the PIN. 6 tests pass. Next:
the login endpoint + require_login gate (increment 2).
Lets SoulSync sit behind Authelia/Authentik/oauth2-proxy as the gatekeeper: when
security.auth_proxy_header names a header (e.g. Remote-User), a request carrying it
is treated as already-authenticated and passes the launch lock — the proxy did the
login (with 2FA).
- core/security/auth_proxy.py: trusted_proxy_user(get_header, header_name) — returns
the user iff the configured header is present + non-empty; empty header name (the
default) → always None → feature off.
- _enforce_launch_pin ORs it into pin_verified. OFF by default, so a direct install
is unaffected AND a client-spoofed header does nothing unless the operator opted in.
- Doc'd in Support/REVERSE-PROXY.md with the must-strip-client-headers warning.
This is the lightweight Tier 3 (auth-proxy integration), not a full per-user login —
the proxy owns identity; SoulSync trusts it.
Tests: helper off/on/blank/exception-safe; integration — trusted header passes the
gate, no header is locked, and (the safety pin) a spoofed header is IGNORED when the
feature is off. 6 tests pass.
Fold a conservative security-header set into the SAME opt-in proxy mode, so it's
zero-impact when off. When security.trust_reverse_proxy is on, an after_request
adds X-Content-Type-Options: nosniff, X-Frame-Options: SAMEORIGIN, and HSTS
(safe — only honoured over the proxy's HTTPS), via setdefault so it never clobbers
a header the proxy already set. No CSP (needs per-deploy tuning; better at the
proxy). When OFF (default), the after_request isn't registered → no headers added.
Tests: off adds none of the headers; on adds all three. Doc updated. 6 tests pass.
A publicly-exposed instance gated only by the launch PIN was brute-forceable. Added
a lenient in-memory failed-attempt limiter (core/security/rate_limit.py): 10 wrong
PINs from one IP within 5 min → 429 with Retry-After, failures age out on their own
(self-heal, no persistent lockout), and a CORRECT entry clears that IP instantly.
Wired into /api/profiles/verify-launch-pin. By design it can only ever trigger on a
flood of WRONG PINs — correct entry, a couple of typos, or a no-PIN install are
never affected, so normal use sees no change. Keyed per-IP so an attacker can't
lock out a legit user.
Tests: limiter is lenient under threshold, trips on a flood, success clears it,
failures self-heal, per-IP isolation; endpoint returns 429 after 10 wrong PINs with
Retry-After. 6 tests pass.
Tier 1 of "secure behind a reverse proxy". STRICTLY opt-in so direct/LAN installs
are byte-for-byte unchanged.
- core/security/reverse_proxy.py: apply_reverse_proxy_mode(app, config_get) — a
no-op unless security.trust_reverse_proxy=true. When OFF (default), the app is
untouched: no ProxyFix, X-Forwarded-* stays UNtrusted (a direct client can't
spoof its IP/scheme), session cookie keeps Flask defaults. When ON (operator is
behind nginx/Caddy/Traefik with TLS): trust one proxy hop's X-Forwarded-*, and
mark the session cookie Secure + SameSite=Lax. Any config error → safe no-op,
never breaks startup.
- Wired once at app init.
- Support/REVERSE-PROXY.md: nginx (with the Socket.IO Upgrade headers people
always miss) / Caddy / Traefik configs, the setting, and the "put auth in front
(Authelia/Authentik/oauth2-proxy)" recommendation + the off-for-plain-HTTP note.
Tests: off (and missing-key, and a config exception) is a strict no-op — not
ProxyFix-wrapped, cookie defaults intact; on wraps ProxyFix + secures the cookie;
and the real web_server app is NOT in proxy mode by default. 5 tests pass.
Per the original intent, "Sync" is now a single-artist deep scan: it uses the SAME
reconciliation source as the whole-library deep scan instead of a separate
disk-existence check.
- Phase 1 already calls the deep-scan worker's _process_artist_with_content; now it
passes seen_track_ids so the pull collects the server's current track IDs for the
artist (existing + new), exactly as the library deep scan does.
- Phase 2 stale = (artist's DB tracks for this server) − seen, then
delete_stale_tracks(server_source) — identical mechanism to deep scan, scoped to
one artist. The old os.path.exists disk check (which could mass-delete on an
unreachable mount) is gone.
- Removal only runs when the server pull SUCCEEDED — no trustworthy 'seen' set
(no server, unreachable, or a failed pull) → skip, never delete. The
is_implausible_stale_removal guard (>50% unseen) stays as the same safety net
deep scan has for a flaky response. @admin_only retained.
Tests rewritten for the server-diff model: removes only tracks the server no longer
has; guard skips when most are unseen; a failed pull skips removal entirely;
admin-only. 8 tests pass.
The enhanced-tab "Sync" button's stale-removal phase deleted any track whose file
wasn't on disk, with NO guard — so if the music storage was momentarily
unavailable (sleeping NAS, dropped mount, unmounted Docker volume, WSL hiccup),
os.path.exists returned False for EVERY file and one click wiped the whole artist
(tracks + their now-"empty" albums) from the DB. The deep-scan path already had a
50%-stale safety net (#828); this endpoint never got one.
- New core/library/stale_guard.py: is_implausible_stale_removal(missing, total) —
a tested rule (skip removal when missing > 50% of a >=5-track set), centralised
so every stale-removal site can share it.
- sync_artist_library: if the guard trips, SKIP removal (delete nothing), return
removal_skipped + warn; the frontend shows "storage may be offline — skipped"
instead of silently deleting. Empty-album cleanup now also only runs on the
non-skipped path and uses `album_id IS NOT NULL` (fixes the NOT IN-with-NULL
no-op). Frontend also refreshes the view on additions, not just removals.
- @admin_only on the endpoint — it deletes tracks + albums but was ungated, while
the sibling delete_album endpoint is gated.
Deep scan was already safe (different mechanism: server-diff + its own 50% guard).
Tests: guard unit rules; endpoint skips removal when all files missing (keeps the
tracks), removes only the genuinely-gone few otherwise, and 403s for non-admins.
7 new tests pass.
The #843 fallback saved the discovery-cache match using the client's
original_artist verbatim — but the client sends a joined "A, B, C" string, while
EVERY in-memory + sync path keys the cache by the first artist (artists[0]). So a
multi-artist fix (the reporter's exact "Cherrymoon Traxx, Hermol, SBM, BELS"
case) would have saved under a key the sync never looks up — the fix would
"succeed" with no error but silently never apply.
Reduce the client artist to the first (split on comma) in the no-state branch so
its cache key matches the in-memory/sync convention exactly. Single-artist tracks
are unaffected.
Test: no-state save now keys by the first artist, and a new test pins that the
no-state and in-memory paths produce an IDENTICAL cache key for the same
multi-artist track. 74 discovery tests pass.
The discovery FIX → Confirm flow 404'd with "Discovery state not found" whenever
the in-memory discovery state was gone — a server restart, or an imported
playlist that wasn't discovered in THIS process — even though the card is still
shown from persisted data (the reporter's log shows "Returning 0 stored ...
playlists for hydration", i.e. the in-memory states were empty).
The thing that actually makes a manual fix STICK is writing it to the discovery
cache (save_discovery_cache_match), keyed by the original track's name + artist —
which doesn't need the in-memory state at all. But the endpoint 404'd on the
missing state before reaching that write, so the fix was dead after a restart.
- update_discovery_match (core/discovery/endpoints.py) now only does the in-memory
result update when the state exists; the durable discovery-cache write always
runs, falling back to client-provided original_name/original_artist when there's
no in-memory state. With neither a state nor originals it still 404s (unchanged).
- The FIX confirm (wishlist-tools.js) now sends original_name/original_artist
(from the source track it already has) so the backend can key the cache.
Covers all sources that share the helper (tidal/deezer/qobuz/spotify-public).
Tests: no-state-but-originals saves the cache + returns success; no-state-no-
originals still 404s; existing with-state path unchanged. 73 discovery tests pass.
Regression in 2.6.9. The spotify_public source adapter (used by auto-sync /
refresh_mirrored) called scrape_spotify_embed() directly — the embed widget only
exposes ~100 tracks — instead of fetch_spotify_public(), the wrapper the rest of
the app uses, which pulls the full list via the paginated public API and only
falls back to the embed on failure. So initial discovery got the whole playlist
but every auto-sync re-fetch truncated it to 100.
Switched the adapter to fetch_spotify_public (same return shape — drop-in). Albums
still resolve via the embed (already whole); on any failure it falls back to the
embed exactly as before.
Test: the adapter returns all 150 tracks when the full fetch yields 150 (was
capped at 100). 29 adapter tests pass.
A wishlist track (or tracks in an album) that slskd accepted then REJECTED would
sit at "DOWNLOADING... 0%" indefinitely, spam an ERROR every status poll
("…Completed, Rejected - letting monitor handle retry"), and — for albums —
block the whole batch from ever completing.
Root defect: the status formatter's non-manual error branch keeps the task
'downloading' and trusts the retry monitor to resolve it, with NO backstop. When
the monitor can't make progress (a rejected transfer with no other source), the
task is stuck forever.
Backstop: measure how long the ERROR state has persisted (keyed off the task's
last status transition, so a slow-but-healthy transfer is never failed, and each
monitor-retry episode gets a fresh window). Once it exceeds the monitor's retry
window (60s, vs the monitor's ~15s) with no resolution, mark the task failed and
fire the worker-freeing completion callback so the batch can finish. Also log the
error ONCE per episode instead of every 2s poll. The healthy path is untouched —
a working retry transitions the task before the grace, so this never fires.
Manual picks still fail immediately (unchanged).
Tests: rejected-within-grace stays downloading; rejected-beyond-grace fails +
schedules completion; manual pick fails immediately. 45 status tests pass.
Reported via Find & Add (Billie Eilish "bad guy"): the track was in the library
and on Plex, but never showed in the modal's 20 results. Root cause (proven
against the real 307k-track DB): the search did `ORDER BY tracks.title`, which is
case-SENSITIVE in SQLite (BINARY collation sorts 'B' before 'b'). Billie's title
is lowercase "bad guy"; everyone else's is "Bad Guy", so all the capitalised ones
sorted first, filled the LIMIT, and her exact match landed at ~#25 — cut off.
- search_tracks now ranks by relevance: exact title match first (case-insensitive
via unidecode_lower), then prefix, then alphabetical — so an exact match can't
be sorted below the limit by a capital letter. Helps every caller.
- Added a rank-only `rank_artist` hint (never filters): Find & Add already knows
the source track's artist, so it now passes it and the exact title+artist match
floats to #1. Filtering was deliberately avoided — if the track is tagged under
a slightly different artist on the server, a filter would re-hide it.
Verified on the real DB: title-only "bad guy" now surfaces Billie at #4 (was
>#20); with the artist hint she's #1. Seam tests: lowercase exact title isn't
buried; rank hint floats the match without filtering; exact title beats a
superstring title. 10 tests pass.
Automations + auto-sync respect 'append' mode and preserve a server playlist's
description + cover image, but manually matching a missing track ("Find & add")
recreated the whole playlist and wiped them.
Root cause: the add-track endpoint's Jellyfin branch called
`update_playlist(<entire track list>)`, which deletes + recreates the playlist on
Jellyfin/Emby. Switched it to the purpose-built `append_to_playlist([the one
found track])` — the same in-place, dedupe-safe op the 'append' sync mode already
uses — so the playlist (and its description/image) is preserved and only the
missing track is added. append_to_playlist reads `.id` off the track, so the
endpoint now sets it (it previously only set ratingKey).
Plex (in-place addItems) and Navidrome (in-place Subsonic updatePlaylist) were
already non-destructive; Emby routes through the jellyfin branch, so this covers
it too.
Tests: the add-track endpoint appends in place and never calls update_playlist;
a link-to-existing-track touches nothing. 18 tests pass (incl. the existing
append-mode suite).
YouTube/Tidal/Qobuz results encode the name as ``id||title``. When the title
itself contains a '/' (e.g. the Sawano AoT track "YouSeeBIGGIRL/T:T"), two places
wrongly basename-split it on the slash and kept only the last segment ("T:T"):
- core/downloads/file_finder.py — the completed-download finder truncated its
search target to "T:T", so the real on-disk file (slash sanitised by the
writer) never matched → "not found after processing" → the download got
QUARANTINED. Now an encoded ``id||title`` keeps the whole title as the target
and contributes no remote-directory components; real Soulseek PATHS still get
basename + dir extraction unchanged.
- webui/static/downloads.js — the manual-search FILE column showed only "T:T".
Added a ``||``-aware short-label helper (mirrors the correct handling already
used elsewhere in the file); real file paths still show their basename.
Tests: the finder locates "YouSeeBIGGIRL∕T: T.mp3" from the encoded title
"…||YouSeeBIGGIRL/T:T" (the screenshot case), doesn't match an unrelated file,
and a genuine Soulseek path still resolves to its last segment. 21 finder tests
+ 64 script-split integrity tests pass.
- ⚠ Unverified filter rows gain actions: inline play (range-streamed from the
history file path, server-side only), YouTube compare, Approve -> new
human_verified status (tag + history + tracks; AcoustID scanner skips these
entirely), Delete (file + entry)
- API: /api/verification/<id>/stream|approve|delete (path only from DB row)
- backfill: history rows with acoustid_result='fail' that exist at all were
imported despite the failure = force_imported (covers pre-fix fallback
imports like the user's 'My Ordinary Life')
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Also: evaluate() treats an empty expected artist as title-only comparison
(old scanner behaviour — a missing DB artist is no evidence of a wrong file),
and the thresholds are now defined once in the core and re-exported.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The library AcoustID scan now calls audio_verification.evaluate() (alias-aware
artist match + cross-script SKIP) instead of its own non-ASCII-stripping
_normalize and threshold logic, so it no longer false-flags correct anime-OST /
kanji tracks. Duration-collision guard kept as a scanner pre-check on the top
recording. evaluate() is now purely a title/artist/version/cross-script decision.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
verify_audio_file now calls audio_verification.evaluate() and re-exports
normalize/similarity/_alias_aware_artist_sim from the core, so import and the
library scan can no longer drift apart. Alias-rescue diagnostic moved to the core.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
assert against a monkeypatched get_spotify_client sentinel instead of
web_server.spotify_client (which isn't a stable singleton across the suite) — same
fix already used for the per-profile builder tests.
The playlist source registry built the Spotify/Tidal adapters with a client
GETTER (resolved fresh on every read), but web_server passed `lambda: <global
client>`. Swapped those to get_spotify_client_for_profile /
get_tidal_client_for_profile.
Combined with part 1 (the engine running each automation as its owner), an
auto-sync pipeline now reads its source playlist through the OWNER's account:
- interactive sync → the user's session profile,
- background automation → the automation owner (via core.profile_context),
- admin / profile 1 → the global client, so the admin's existing auto-sync
pipelines pull exactly as before.
The adapters re-resolve per read, so a singleton registry is fine. Deezer/Qobuz
getters left global (their playlist login is tangled with downloads — deferred).
Tests: the Spotify/Tidal source adapters resolve the global client under admin
and re-resolve through the profile context per call (unconnected → safe global
fallback). 27 endpoint/profile tests pass.
Background automations had no session, so get_current_profile_id() fell back to
admin (1) — wrong for a non-admin's scheduled job. Now the engine declares the
automation's owner around handler execution via a contextvar
(core/profile_context.py), and get_current_profile_id() consults it only when
there's NO web request. So:
- a real logged-in request always wins (foreground unchanged),
- admin + system automations are profile 1 → resolve to admin exactly as before
(the 8 admin-owned auto-sync pipelines behave identically),
- only non-admin-owned automations gain their correct identity, deep through the
whole call chain (incl. the per-profile client resolvers) — no threading
profile_id through dozens of signatures.
Reset in a finally so a pooled thread can't leak the override to the next job.
Tests: contextvar set/reset/nested; get_current_profile_id honours the override
only outside a request (a real session still wins); and end-to-end — the engine
runs a non-admin automation as profile 4, an admin one as 1, an explicit trigger
profile overrides the owner, and the context resets even when the handler raises.
27 + 4 tests pass.
Part 2 (next): point the sync handlers' source-playlist READ at
get_spotify_client_for_profile so a non-admin's auto-sync pulls THEIR playlist.
My credential/admin-gating endpoint tests override DATABASE_PATH at import to a
temp dir, but with a custom prefix — which tripped the DB-isolation guard
(test_database_path_env_is_isolated requires 'soulsync-testdb-' in the path) once
those modules loaded. Still a temp DB (no real-DB risk), just the wrong prefix.
Switched to the soulsync-testdb- prefix so isolation is preserved and the guard
passes.
Third service (the easy one — ListenBrainz already had a working per-profile
token path). Consolidated all per-profile streaming accounts into the My Accounts
modal:
- My Accounts gains a ListenBrainz row with a token-paste connect (a new 'token'
service type alongside the OAuth-popup ones), reusing the existing
/api/profiles/me/listenbrainz save + the generic disconnect.
- Connections API reports listenbrainz status (connected + username).
- Personal Settings (the gear modal) dropped its Spotify/Tidal/ListenBrainz
sections — those duplicated My Accounts — and now shows only the per-profile
server-library selection (non-admin) or a pointer note (admin). The old
renderPersonalSettings{Spotify,Tidal,LB} functions are left defined but unused.
So every per-profile account connection (Spotify, Tidal, ListenBrainz) now lives
in one place. Tests: LB connect status + disconnect via the generic endpoint.
23 endpoint tests pass; 64 integrity tests pass.
Second service. Each profile connects its own Tidal; its playlist reads use that
account, everything else stays global. The gotcha vs Spotify: TidalClient loads
AND saves tokens to one global slot (tidal_tokens), so a naive per-profile client
would clobber the admin's tokens on refresh.
- get_tidal_client_for_profile builds a dedicated TidalClient seeded with the
profile's tokens, refreshed via the shared/global app creds, and OVERRIDES its
_save_tokens to persist to the PROFILE row — never the global slot. Admin
(profile 1) + unconnected profiles use the global client unchanged. Cached per
profile + evicted on (dis)connect.
- DB: set_profile_tidal_tokens / get_profile_tidal (encrypted); the OAuth callback
now uses them + evicts the cached client.
- Wired the Tidal playlist reads (list + tracks) to the per-profile client; the
module import line left intact.
- My Accounts: Tidal row (Connect via /auth/tidal?profile_id=, status, Disconnect).
Connections API extended; disconnect made generic (/<service>/disconnect).
Admin sees "managed in Settings" for every service.
Tests: per-profile token refresh writes to the profile and leaves the global
tidal_tokens untouched (the safety guarantee); connect status + disconnect;
admin/unconnected → global client. 22 endpoint tests pass.
Review/regression fix: the shared-app gate I added wrongly required a token
cache even for profiles that set their OWN app creds (legacy), breaking the
per-profile caching tests. Now a per-profile client is built when the profile
has its own app creds OR has connected (its own token cache) — otherwise it
falls back to the global/admin client. Made the fallback-safety tests
order-independent (monkeypatch get_spotify_client to a sentinel) since the real
global client isn't a stable singleton across the suite.
10 metadata-cache + resolution tests pass together.
First service of the per-profile playlist-auth feature. Each profile connects
its OWN Spotify account through the shared (admin's) app, getting its own token;
used for that profile's playlist reads. Admin + unconnected profiles + all
background workers keep using the global/admin client — fully non-regressive.
- Shared-app OAuth: get_spotify_client_for_profile + the /auth/spotify init &
callback now use the GLOBAL app creds (falling back from any legacy per-profile
app creds) with the profile's own token cache, and show_dialog=true forces the
account chooser so a user can't silently inherit the admin's Spotify session.
The builder gates on the profile's own token cache existing — no cache → global.
- My Accounts modal (new, all-profile-accessible via the profile bar): one-click
Connect/Disconnect Spotify + connection status (account name). GET
/api/profiles/me/connections + POST .../spotify/disconnect; admin's Spotify is
read-only here (managed in Settings).
- Wired the request-scoped reads to the per-profile client: the playlist LIST,
the playlist TRACKS view, liked-songs count, and user info — so a connected
user sees and opens THEIR OWN (incl. private) playlists, not the admin's.
Tests: builder falls back to the global client for admin/None/unconnected (the
non-regression guarantee); connections status reports unconnected; admin
disconnect rejected. 124 profile/spotify/gate/integrity tests pass.
Still on the global account (next step): sync/download jobs run in background
workers with no profile context — stamping the requesting profile onto the job
is the remaining wiring. Other services (Tidal/Deezer/Qobuz/Last.fm/ListenBrainz)
follow this same pattern.
The model shifted from "admin creates shared credential sets, users pick" to
"each profile self-auths its own playlist accounts". Removed the admin-facing
Connected Accounts manager: the Settings section, credential-sets.js, its CSS,
the script tag + integrity-registry entry, and the loadSettingsData hook.
The credential-sets backend (service_credentials tables + /api/credentials and
/api/profiles/me/services endpoints) is left in place but dormant — additive,
tested, harmless — rather than churn migrations that already ran on installs.
Per-profile self-auth reuses the existing per-profile columns + the
get_*_for_profile client pattern instead. The Service Status modal (admin-only)
is unaffected.
Regression from the #832 server-side launch gate (Beckid). On a PIN-required,
unverified session the gate 401'd /api/setup/status — which the frontend checks
BEFORE the PIN screen. The 401 left setup_complete undefined, so `!undefined`
relaunched the full setup wizard on every visit (cancel → PIN screen worked,
which was the tell).
Two-layer fix:
- Allowlist /api/setup/status in the launch gate (it's a harmless boolean, no
secrets) so the frontend gets the real answer even while locked.
- Make the frontend fail-safe: only launch the wizard on a definitive
setup_complete === false from an OK response — never on a 401/locked/ambiguous
one.
Test: locked session still 401s data endpoints but /api/setup/status returns
{setup_complete:true}; added a gate-allowlist assertion. 21 gate tests pass.
Correctness (the modal was lying): "Spotify (no auth)" is a COMPOSITE the
Settings page stores as fallback_source='spotify' + metadata.spotify_free=true,
not a literal 'spotify_free' value. The modal read the raw fallback_source and
showed plain "Spotify" as active even when Settings clearly said "(no auth)".
The endpoint now mirrors that mapping both ways — reports active='spotify_free'
when the flag is set, and switching to it writes fallback_source=spotify +
spotify_free=true (and clears the flag for any other source). Modal + Settings
now always agree.
Visual: the modal itself (not just the cards) is richer now —
- a hero header per tab: big brand-logo disc + "Active <kind> source" eyebrow +
the active name + a one-liner + an Active pill, all tinted by the brand color
with a soft radial glow (the Manage-Workers hero feel);
- the panel gained brand-tinted radial depth instead of flat black.
Test: spotify_free composite round-trips like Settings (stored split + reported
as spotify_free; flag clears on switch). 15 endpoint + 64 integrity tests pass.
Replaces the basic credential-pill quick-switch with a Manage-Workers-styled
modal (topbar + left rail + panel, entrance animation, brand-logo cards).
- Sidebar Service Status: whole panel opens the modal; clicking the Metadata /
Media Server / Download rows deep-links straight to that tab. Removed the
"switch ▸" hover text.
- Three tabs: Metadata (source logo cards, unavailable ones dimmed), Server
(Plex/Jellyfin/Navidrome/SoulSync logos), Download (Single⇄Hybrid segmented
toggle; Hybrid shows a draggable priority list). Logos reuse SOURCE_LABELS +
HYBRID_SOURCES; active card gets an accent ring + check.
- Admin writes the GLOBAL active source/server/download (reuses the same setters
+ client reloads as the Settings save, so changes take effect immediately).
Non-admins see it read-only (editable=false) — the per-profile override is the
next layer.
Backend: GET /api/profiles/me/active-sources (any profile; reports editable),
POST /api/profiles/active-sources (@admin_only; validates against the allowed
metadata/server/download lists, applies + reloads). New service-switch.js
(registered + in the integrity registry); old modal removed from
credential-sets.js (admin Connected Accounts manager stays).
Tests: 14 endpoint tests — read shape, admin sets metadata/hybrid+order
(reflected), bad-value 400s, non-admin read-only + 403 on write. 64 integrity
tests pass; real-app smoke confirms render + deep-links + the full set/reflect
cycle.
Frontend for the credential-set feature, matching the blocklist/house modal
style. Functional end to end against the existing endpoints; visuals are a
clean first pass to refine.
Admin manager (Settings → Connected Accounts, admin-only — empty for non-admins):
per service, the saved accounts render as pills with a delete ✕, and "+ Add
account" reveals an inline form built from each service's required fields.
Create POSTs /api/credentials; secrets are entered but never read back (the API
only returns id/label). Loads via loadCredentialSets() at the end of
loadSettingsData().
Quick-switch modal (sidebar Service Status is now clickable for ALL profiles):
shows, per service the admin set up, a "Default" pill + one pill per account,
highlighting the profile's current choice; clicking persists via
/api/profiles/me/services/select and re-renders. Empty-state message when the
admin hasn't configured any alternates.
webui/static/credential-sets.js (new, registered in index.html), house-style
CSS appended, sidebar made clickable, settings hook added. Registered the new
module in the script-split integrity test (onclick coverage). 64 integrity
tests pass; real-app smoke confirms index renders, the asset serves, and
admin-create → per-profile-list round-trips.
Note: selections are stored but not yet consumed by the live clients (the
resolver remains dormant) — wiring playlist-pull/enrichment to use a profile's
selected account is the next step.
Discogs masters (/masters/{id}) and releases (/releases/{id}) share one
numeric ID namespace, so release N and master N are different albums.
search_albums() (type=release) yields RELEASE ids, but get_album,
get_album_tracks and _fetch_and_cache_album all tried /masters first and
only fell back to /releases. Whenever a release id collided with a real
master id, the master lookup succeeded and returned a valid-but-wrong
album — the fallback never fired.
Tag the object type into the id string ('r12345' / 'm12345') at the single
parse point (Album.from_discogs_release) and route each fetch to the
matching endpoint. Legacy untagged ids are tried release-first (then master),
which self-heals pre-existing bad matches without a migration. The
enrichment worker now persists the tagged id too.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adversarial line-by-line review of the feature diff turned up:
- /api/database/update/stop was NOT @admin_only while its sibling start_update
was — a non-admin could abort a library scan. Gated.
- /api/metadata-cache/evict was NOT gated while its clear siblings were. Gated.
- validate_credential_payload now treats whitespace-only values as missing, so
a blank-but-spacey secret can't be saved to fail confusingly later.
Tests updated: both endpoints added to the admin-gating matrix; a whitespace-only
validation case added. 42 credential/gating tests pass.
Review also confirmed (no change needed): migration is idempotent + additive +
O(1); encryption round-trips with a non-dict guard; no SQL injection; stale
selections fall back to None safely; no secret ever returned to the browser;
the hybrid-drag index math is correct in both directions; the new resolver is
fully DORMANT (zero runtime callers) so existing client behaviour is untouched;
and @admin_only is a no-op for single-profile installs (default profile = admin).
Lets any profile pick which admin-created credential set is active for it,
without creating/seeing secrets:
- GET /api/profiles/me/services per-service options (id+label only) +
this profile's selected_id (stale-safe)
- POST /api/profiles/me/services/select {service, credential_id|null}
Not admin-gated by design — it only writes a per-profile pointer and exposes no
secrets. Validates the chosen set exists AND belongs to that service (can't
select a tidal set under spotify), and rejects unsupported services. null clears
back to the global/admin default.
Tests: a non-admin reads options + selects + clears (no secret in the response),
and selection rejects wrong-service / nonexistent / unsupported. 10 endpoint
tests total.
The audit found these were UI-hidden but API-open — any profile (or the
anonymous default-admin) could call them directly. Added @admin_only to the 15
that mutate SHARED/global state:
- DB: update, backup, backups DELETE, restore, vacuum
- library: track DELETE, album DELETE, tracks delete-batch, clear-match
- plex clear-library; metadata-cache clear + clear-musicbrainz
- internal API keys: list, generate, revoke
Deliberately NOT gated: profile-scoped own-data ops like /api/wishlist/clear
(clears the caller's OWN wishlist via profile_id) — gating that would wrongly
block a non-admin from managing their own data. Verified by test.
Zero change for single-profile installs (the default profile IS admin), so
existing users are unaffected; only genuine non-admin profiles get 403.
Tests: non-admin → 403 on all 15 (the 403 fires before the view body, so no
destructive op runs); admin not blocked on the read-only one; wishlist/clear
stays open to non-admins (over-gating guard). 17 tests.
Admin-only CRUD over the named credential sets from Phase 0:
- GET /api/credentials list all sets grouped by service (NO secrets)
- POST /api/credentials create {service,label,payload}, validated
- PUT /api/credentials/<id> update label and/or payload (partial)
- DELETE /api/credentials/<id> delete (clears any profile selections → fallback)
All four are @admin_only (non-admin → 403), payloads validated via
core.credentials.store, secrets never returned to the browser. Additive — no
existing endpoint or behaviour changes.
Tests: real web_server app + Flask test client (8) — create/list/update/delete
roundtrip, payload never leaks in list, missing-field/unsupported-service/blank-
label/duplicate(409)/404 validation, and the non-admin 403 gate on every write.
Verified the web_server import coexists with the rest of the suite (175 mixed
tests pass).
Groundwork for admin-created, per-profile-switchable credential sets ("pills")
across auth services (Spotify/Tidal/Deezer/Qobuz/Plex/Jellyfin/Navidrome).
Strictly additive and dormant — nothing reads it at runtime yet, so zero
behaviour change for existing installs.
- core/credentials/store.py: pure service registry + payload validation +
stale-safe active-set selection (pick_active_credential falls back to None
when a selected set was deleted, so a profile never breaks).
- migration service_credentials_v1: two new tables — service_credentials
(admin-created named sets; payload Fernet-encrypted at rest) and
profile_service_credentials (each profile's selected set per service).
- MusicDatabase CRUD: create/update/delete/list/get_service_credential
(list never returns the payload; get decrypts for the resolver), plus
set/get_profile_service_credential and resolve_profile_service_credential
(returns the profile's active payload or None → caller uses global default).
Tests: 12 — pure validation + stale-safe selection, and real-temp-DB storage
proving encryption round-trips, payload never lists, dup(service,label)
rejected, per-profile/per-service resolution, and delete clearing dangling
selections to a clean fallback. 95 migration/DB tests still pass.
The #831 watchlist work added the standalone webui/static/watchlist-history.js
(loaded in index.html) but didn't add it to NON_SPLIT_JS, so the onclick-
coverage test couldn't see its functions and failed on openWatchlistHistoryModal
referenced in the History button. Register it alongside its sibling
origin-history.js — same fix the registry comment was added for.
Found during the #832 audit: GET /api/settings returned dict(config_data) — and
config_data is DECRYPTED in memory — so every API key, OAuth secret, Plex/
Jellyfin token, and service password went to the browser in cleartext. Fernet
"encrypted at rest" protects a leaked DB file; it does nothing once the API
hands the plaintext to the client (devtools, HAR captures, an XSS, a screen
share, or a non-PIN'd LAN viewer).
Fix (centralized in ConfigManager):
- redacted_config() deep-copies config and replaces every _SENSITIVE_PATHS value
that's actually set with REDACTED_SENTINEL; unset secrets stay empty so the UI
still shows "not configured". Dict-valued secrets (tidal/qobuz OAuth sessions)
collapse to the sentinel too. GET /api/settings now serves this copy.
- set() ignores a write of REDACTED_SENTINEL to a sensitive path, so the masked
placeholder round-tripped by an unchanged settings form can never overwrite
the real secret. A real value still saves; an empty value still clears.
Frontend: secret inputs are type=password, so the sentinel renders as dots
(looks like a saved secret). _wireRedactedSecrets() clears the mask on focus so
editing types fresh rather than onto the sentinel, and re-masks on blur if left
untouched — so an unchanged secret round-trips the sentinel (kept), an edited
one saves the new value, and a deliberately emptied one clears.
Tests: every sensitive path masks; unset stays empty; dict secrets mask; live
config not mutated; sentinel round-trip keeps the real secret; real value
overwrites; empty clears; sentinel on a non-secret path writes normally.
9 new tests; 518 config-touching tests pass (1 pre-existing soundcloud mock
failure, unrelated — fails identically on a clean tree).
the-hang-man: tracks with an apostrophe (e.g. "I'm Upset") deleted the DB row
but left the file. The library DB stored the title with U+2019 (the curly form
Spotify/Apple metadata uses) while the file was written to disk with U+0027
(ASCII). _resolve_library_file_path compared the curly path byte-for-byte via
os.path.exists, missed every time, and reported "could not be deleted".
Fix: resolve confusable-tolerantly. New core/library/path_resolve.find_on_disk
descends the path component by component, taking an exact match when present and
otherwise folding a small set of typographic look-alikes (curly vs straight
quotes, en/em dash, ellipsis, nbsp) for the comparison ONLY — it never renames,
just finds the file that's actually there. Exact matches always win per
component, so paths that already resolved are byte-for-byte unaffected. This
also fixes existing mismatched files (no re-import) and every caller of
_resolve_library_file_path (sidecar cleanup, dead-file checks, streaming), not
just delete.
Case is deliberately NOT folded: a case-sensitive dataset (ext4/ZFS) can hold
names differing only by case, and folding could resolve the wrong file. The
reported failure is purely typographic.
Tests: real temp-file fixtures exercising the actual byte mismatch — curly-DB →
ascii-disk resolves, exact still works, confusable in a folder component, exact
wins when both encodings present, genuinely-different name does NOT collide,
missing file → None. 10 new tests; 949 resolver-adjacent tests pass.
Beckid: the admin launch PIN was a CLIENT-SIDE overlay only. `launch_pin_required`
just told the frontend to draw a fixed div over the app — removing it (Safari
"Hide Distracting Items", devtools, or any non-browser client like curl) gave
full unauthenticated access to every /api/* endpoint, because the server never
checked it. Anyone who reverse-proxies SoulSync publicly was wide open.
Fix: a before_request gate (_enforce_launch_pin) that rejects every request from
an unverified session while security.require_pin_on_launch is on. The decision
is a pure, unit-tested helper (core/security/launch_lock.request_is_locked) so
the allow/deny matrix can't silently regress. Allowed while locked: the page
shell + static assets, the unlock flow (current/list/select/verify/reset/logout),
and the public REST API /api/v1/ (its own @require_api_key governs it) — EXCEPT
/api/v1/api-keys-internal*, the "no auth required" key-management endpoints,
which stay locked so an attacker can't mint an API key and walk in the side door.
Everything else (data, settings, profile create/edit/delete/set-pin, socket.io)
is blocked.
A blocked top-level browser navigation (deep link / refresh on a sub-page like
/dashboard) is redirected to the root lock screen instead of dumping raw JSON —
detected via Sec-Fetch-Mode: navigate / Accept: text/html (is_html_navigation).
Programmatic fetch/XHR still get the JSON 401 so the frontend can react.
Also fixed the verified flag: get_current_profile POPPED launch_pin_verified
(one page load), but an enforced gate needs it to persist — now READ, so
verification lasts the session (until logout/expiry). No-ops entirely when
require_pin_on_launch is off (default).
Tests: full allow/deny matrix + navigation detection. 20 gate tests + 232
profile/security tests pass.
Boulder: the live display was a cramped ~600px box showing a fraction of the
data the scan already tracks, with no animation and no history.
Live scan deck (replaces the three-column box, full width):
- Header: pulsing live dot, "x / y artists" progress text, and two live
counter chips (found / added) that pop when they change.
- Animated progress bar (artist index / total) with a shimmer sweep.
- Stage: artist avatar with accent glow + name + readable phase line
("Checking album 2 of 5"), album art + album + current track.
- "Added to wishlist this run" feed: taller, bigger art, slide-in animation
that plays once per new track (feed re-renders only when it changes).
- All data was already in scan_state (current_artist_index, total_artists,
tracks_found/added_this_scan, current_phase) — just never displayed. The
legacy fullscreen-modal markup shares element ids and lacks the new ones,
so it keeps working untouched.
Scan History (persistent):
- New watchlist_scan_runs table — one row per run (status, timestamps,
artists/found/added counts) + the full track ledger JSON. Saved at scan
completion AND cancellation; idempotent on run_id; pruned to the last 100
runs. Wishlist rows erode as tracks download, so this is the durable record.
- GET /api/watchlist/scan/history (runs) + /history/<run_id>/tracks (ledger).
- New History button on the Watchlist page → modal in the origins/blocklist
house style: run cards (date, cancelled chip, artists/found/added stats)
expanding into the Added / Skipped track lists with art and badges.
Tests: save+fetch with ledger, idempotent re-save, prune keeps newest,
unknown-run empty, cancelled runs recorded. 398 watchlist/wishlist/history
tests pass; JS syntax-checked; all rendered strings escaped.
Tacobell444 (#707 follow-up): the scan summary said "New tracks: 19 • Added to
wishlist: 10" with no way to see which tracks those were — you had to scan your
wishlist and guess what was new.
Scan ledger: the scanner now records a per-run scan_track_events list (track,
artist, album, thumb, status added|skipped — skipped = found-new but declined
by add_to_wishlist: already queued or blocklisted; capped at 500). The status
endpoint already serializes scan_state, so the payload flows free. The
completed (and cancelled) scan summary on the Watchlist page gets a
"Show tracks" toggle expanding a styled list — Added section + Skipped section
with badges, reusing the live-feed row styling.
Download Origins grouping: the modal now groups entries by what triggered them
(watchlist artist / playlist name) with collapsible headers + counts instead of
a flat list with a per-row badge. Entries arrive newest-first so groups order
themselves by their newest download. Same row markup, checkboxes/delete intact.
Provenance: watchlist adds now stamp scan_run_id into wishlist source_info, so
per-run grouping is queryable later (future "what did run X add" views).
Tests: per-run ledger seam test (added + skipped statuses, album/artist fields,
FIFO unchanged). 316 watchlist/wishlist tests pass; JS syntax-checked.
Third round of the multi-artist report. The earlier fixes (Deezer contributors
upgrade, _artists_list, feat_in_title/artist_separator) were all in place and
correct — but gated on source == 'deezer', and on the real Search → Download
Now path NOTHING carried the source: core/search/sources.py serialized tracks
with no source field, search.js's enrichedTrack didn't add one, so
get_import_source() resolved '' and the whole Deezer-specific block silently
skipped. Files were tagged with only the primary artist until a Retag (which
rebuilds context with the source set — exactly why retagging always fixed it).
The earlier tests passed because they set context['source'] directly — the one
field the real flow never had (same mock-drift as the #823 append tests).
Reproduced with Netti93's exact track (deezer 3966840171) through the real
extract_source_metadata: before — source '', artists ['August Burns Red'];
after — source 'deezer', contributors fetched, artists ['August Burns Red',
'Polaris'], title 'Sonic Salvation (feat. Polaris)' per feat_in_title.
Fix, three layers:
- core/search/sources.py: serialized tracks/albums/artists carry "source"
(the canonical name the orchestrator already passes; '' when unnamed).
- core/imports/context.py get_import_source: also reads '_source' from the
nested dicts (track_info/original_search/album/artist) — additionally fixes
the discography/wishlist flows, which always passed '_source' that nothing
read.
- search.js: enrichedTrack + the album-download path carry source through to
the download task.
Tests: real-payload staging-shaped contexts (source in track_info, '_source'
shape, and the pre-fix sourceless shape staying safe — mocked Deezer client),
serializer source-field tests, resolver fallback tests; exact-shape serializer
tests updated for the new key. 1977 import/metadata/search tests pass (the
only 2 failures are the known soundcloud ones).
carlosjfcasero round 2: after 6fa956d6 append stopped recreating the playlist,
but every sync re-appended ALL matched tracks — every track N times. His log
shows it plainly: "added 22 new tracks to 'Disney' (skipped 0 already present)"
on a playlist that already had those 22.
Root cause: the dedupe read `{t.id for t in get_playlist_tracks(...)}` — but
JellyfinTrack only defines `ratingKey`, never `id`, so the existing-ids set was
ALWAYS empty and everything looked new. NavidromeTrack is also ratingKey-only,
so the Navidrome append had the identical bug. Plex (plexapi ratingKey) was
fine. The existing tests were green because they mocked existing tracks as
SimpleNamespace(id=...) — encoding the same wrong assumption as the code.
Fix:
- New pure planner plan_playlist_append(current, desired) in
core/sync/playlist_edit.py (next to the reconcile planner): order-preserving,
drops already-present ids, dedupes within desired, stringifies (Emby numeric
vs string safe).
- Jellyfin/Emby: existing ids fetched from the canonical
/Playlists/{id}/Items endpoint (same as reconcile — works for Jellyfin GUIDs
and Emby numeric ids), ratingKey fallback if that request fails.
- Navidrome: dedupe on ratingKey (the attribute that actually exists).
Tests: planner (skip-present incl. the reporter's unchanged-playlist case,
desired-order, dupes-within-desired, int/str ids, empties) + the append-mode
suite rewritten to pin the REAL shapes (raw Items dicts for Jellyfin,
ratingKey objects for Navidrome) + a new fallback-path test. 524
playlist/sync/jellyfin/navidrome tests pass.
carlosjfcasero round 2 (manual-add fix didn't help — different path). His log
pinned it: the mirrored sync auto-added 'Llamando a la tierra (Serenade From
the Stars)' by M-Clan every run even though his library has the song (stored
bare). Reproduced exactly: the subtitle restates no album context, so the #808
context strip keeps it, and the length-ratio penalty in
_calculate_track_confidence crushes the pair to 0.142 (needs 0.7). Sync →
"missing" → wishlist, forever; and the cleanup uses the SAME matcher, so it
deterministically never removed it. Self-reinforcing.
Fix at the matcher seam (benefits sync, cleanup, downloads, discography alike):
core/text/title_match.strip_subtitle_qualifiers(title, other) strips a
bracketed qualifier only when it (a) isn't restated in the other title, (b)
contains no version-marker token (EN + ES: live/remix/acoustic/version/
dueto/directo/vivo/...), and (c) introduces no new digit token ('(Pt. 2)',
'(2007)' stay different releases). Wired as a third comparison variant in
_calculate_track_confidence with its own length guard. Verified against his
log's other unmatched tracks: '(Live)' 0.15, '(Dueto 2007)' 0.179,
'(Versión 1988)' 0.167 all still correctly blocked — version qualifiers keep
their meaning; the M-Clan case goes 0.142 → 1.0 in both directions.
Also: sync's check_track_exists call now passes album= (cleanup already did),
enabling the album-aware fallback for multi-artist albums during sync.
Tests: tests/test_subtitle_qualifier_match.py — the reported case verbatim
(end-to-end through check_track_exists, both directions, batched candidate
path included), EN+ES version qualifiers still blocked, numeric guard,
'#769 Dani California' and '#808 OurVinyl' guards still hold. 1396
matcher/wishlist/sync tests pass.
Vicky-2418: Download Discography skipped some albums/singles as "No New Track"
even on a brand-new artist, while manual one-by-one worked. The log showed the
real reason wasn't ownership at all — it was "N skipped (artist mismatch)".
Root cause (confirmed against the iTunes API, not guessed): iTunes returns a
collab as ONE combined string. Narvent's "Miss You (Ambient Remix)" is credited
'TRVNSPORTER, Narvent & SKVLENT'. track_artist_matches did an exact full-string
compare ('trvnsporter, narvent & skvlent' == 'narvent' → False), so every
collaborator's discography entry was dropped — despite the #559 comment claiming
it "keeps features" (it didn't, because features arrive combined, not as a list).
Fix: match the requested artist as a COMPONENT of the credit — split on the
common separators (, & ; / feat ft featuring vs x), while still including the
full string so band names with internal separators ("Florence + the Machine")
match exactly. Component matching stays exact per name, so true contamination
(the #559 case — artist not credited at all) is still dropped, and substrings
("Drakeo the Ruler" ≠ "Drake") still don't match.
Also corrects an over-conservative #559 test that asserted "Drake & Future"
shouldn't match "Drake" — it should; Drake is a credited collaborator. The guard
drops uncredited artists, not legit collabs packed into one string.
Tests: the exact real case (Narvent in the combined credit) + each collaborator,
contamination still dropped, feat/ft/featuring/x forms, no substring false
positive. 36 filter tests + 747 discography/metadata tests pass.
Tacobell444: when tracks land in an album across multiple batches (a wishlist
run, the Album Completeness job, a missed track re-downloaded later), the folder
is rebuilt from API metadata each time — so when $albumtype or $year come back
blank/different on a later batch, the folder NAME changes and the album splits,
forcing a Reorganize.
Fix: build_final_path_for_track now checks whether the album already lives in a
single folder on disk and, if so, drops the new track there instead of a freshly
templated folder. Match (chosen): exact stored Spotify album id first, then a
STRICT >=0.85 name+artist match (vs the 0.7 used elsewhere) — a wrong match here
misplaces a file. New core/library/existing_album_folder.resolve_existing_album_folder
holds the logic; always-on with template fallback.
Safety rails: only returns a folder UNDER the transfer dir (never a read-only
library/NAS mount), only when the album lives in EXACTLY ONE folder (multiple =
disc subfolders, which DatabaseTrack can't disambiguate — those defer to the
template), and any failure falls through to the template path. Added
MusicDatabase.get_album_by_spotify_album_id for the id-first lookup.
Tests: single-folder reuse, no-match, below-threshold, multi-folder defer,
outside-transfer reject, id-first, missing transfer dir, no-files-on-disk.
8 tests; 1556 path/import/download tests pass (only the known soundcloud
failures remain).
macstainless: a Plex-on-macOS user running SoulSync in Docker had all 5,250
tracks flagged "dead" even though the files exist and play in Plex. Root cause:
the DB stores paths as Plex reported them (/Volumes/Core/Music/...), which don't
exist inside SoulSync's container. resolve_library_file_path() returns None for
"couldn't find it at any known base dir" — and for a mis-mounted library that's
EVERY track, not a deletion. The job treated None as "file deleted" and created
a finding per track.
Fix mirrors the existing transfer-folder abort: collect unresolvable tracks, and
if at least max_unresolved_fraction (default 0.5) of the library is unresolvable
once it's past min_tracks_for_guard (default 25), treat it as a path-mapping/
mount problem — abort with an actionable message (Docker mount / Settings →
Library → Music Paths) and create ZERO findings. A small fraction unresolvable
is still reported as genuine dead files, and tiny libraries (< min) report as
before. Both thresholds are configurable per the job's settings.
Tests: mass-unresolvable aborts with no findings; a lone dead file among real
ones is still reported; a small all-dead library still reports; thresholds
configurable. 54 repair tests pass.
Per Boulder's calls on the new enrichment toggle:
- Naming: "Spotify Free" was misleading (it's a hybrid — pick it, connect an
account, and sync still uses your official playlists). Relabel the user-facing
strings to "Spotify (no auth)" — the real distinction is needs-credentials vs
not. Internal value/key (spotify_free, _free_*) unchanged, so no migration.
- Default ON: metadata.spotify_free_enrichment now defaults True (worker + UI
load both treat unset as on). So bulk enrichment runs on the no-auth path by
default and the official account is reserved for interactive search/sync; turn
the toggle off to enrich through the connected account. The toggle overrides
auth for the worker (authed users still enrich via no-auth) — matching the
intended model.
- Worker runs on the toggle alone: is_spotify_metadata_available() now honors
_prefer_free (+ package installed), so the worker enriches via no-auth even
with no account connected and no 'no-auth' source selected. Only fires on a
client carrying the flag (the worker's own), so interactive/watchlist
availability is unchanged.
- UI: moved the toggle from "Metadata Source" to the Spotify section next to the
auth fields, always visible, on by default. Help notes the genre trade-off.
Tests: prefer_free makes metadata available without auth/source (and is inert
without the package); interactive availability unaffected. 218 Spotify tests pass.
Two related fixes at one root cause. Every catalog method gated the official API
on `use_spotify = is_spotify_authenticated()` and only fell to Free *after*
official failed. So when the client was authed but should defer to Free —
specifically when the worker's daily real-API budget was spent — it kept hitting
the official API anyway (just stopped *counting* it). The budget "bridge" never
actually diverted; it only stopped pausing.
Root-cause fix: the official gate is now `is_spotify_authenticated() and not
self._free_active()` across all 8 catalog methods (search_artists/albums/tracks,
get_artist/album/album_tracks/track_details/artist_albums). No-auth and
rate-limited are unchanged (auth is already False there); the change only affects
the cases where auth is True but we deliberately defer to Free. The user-account
methods and the metadata-availability helper are untouched.
New opt-in: metadata.spotify_free_enrichment. When set, the worker puts
`_prefer_free` on its OWN client and _free_active() honors it (needs only the
package installed — the flag is the opt-in — not the 'Spotify Free' source
choice). So a connected user can run bulk enrichment on the no-creds source to
spare their official quota, while interactive search/resolve stay official-first
(they use a different client that never sets the flag). Default off.
Tests: _free_active honors prefer_free (and is inert without the package);
search_albums defers to Free — official .sp raises if touched — both under
prefer_free AND under budget-exhaustion (the divert that previously never
happened). 215 Spotify tests pass.
SpotipyFree has no album-name search (search_albums returned []), and
SpotifyClient.search_albums had no Free branch at all — so when Spotify Free was
the active source (no-auth / rate-limit ban / spent budget), album matching
couldn't use Spotify and dropped straight to the iTunes/Deezer fallback. The
enrichment worker's per-album matching was effectively blind on Free.
Fix — work the gap using ONLY the free source:
- spotify_free_metadata: new search_albums_via_artist(artist, album) resolves the
best-matching artist, scans their discography (get_artist_albums_list — which
Free DOES support), and ranks by album-name similarity. Pure rank_albums_by_name
helper is unit-testable; the network method composes it.
- SpotifyClient.search_albums: add a Free branch mirroring search_artists/tracks
(official → Free-if-active → iTunes/Deezer), plus optional artist/album params
so the Free path has the names it needs. Bare-query callers skip it unchanged.
- spotify_worker: _process_album_individual passes artist+album so the bridge can
match albums on Free.
Tests: rank ordering + limit, via-artist picks the right artist and ranks,
empty on missing artist/album/no-match, and SpotifyClient.search_albums returns
Free albums (not the iTunes fallback) when free is active. 210 Spotify tests pass.
The manual album "Add to Wishlist" modal had NO ownership check at any layer —
the album view opened the modal without ownership info, the modal added every
track, and the backend (add_album_track_to_wishlist) added each unconditionally.
So adding an album you (partially) own dumped the owned tracks straight into the
wishlist (carlosjfcasero #825) — and the auto-cleanup doesn't reliably remove
them. The bulk discography path already dedups (full missing-track analysis);
this path didn't.
Backend (the reliable seam): add_album_track_to_wishlist now skips a track that
already exists in the library, gated on the same wishlist.allow_duplicate_tracks
toggle the watchlist scan + cleanup use — OFF → skip owned (returns
{success, skipped:true}), ON → add anyway. Default is ON, so default users are
unaffected; the quality re-download flow uses a different endpoint, so it's
untouched.
Frontend: handleAddToWishlist + addModalTracksToWishlist count skipped tracks
separately so the toast is honest ("Added 3 (5 already owned)" / "All N already
in your library") instead of falsely claiming owned tracks were added.
Tests: skips owned when duplicates off, adds missing when off, adds owned when
on (and doesn't even run the check then). 205 wishlist tests pass.
Sokhi: "downloads searching for way too many tracks at once" — a wishlist run
that fanned out into ~one batch per album. Verified the actual search/download
concurrency IS capped at 3 (single shared missing_download_executor), so it
wasn't really hammering slskd — but the display showed ~20 "searching" and the
batch list was a mess.
Root cause: run_full_missing_tracks_process was supposed to "block its album-pool
worker for the whole search+download" (that's what the dedicated album_bundle_
executor is for), but it RETURNED the instant it had STARTED the downloads. So
the album pool only throttled the fast analysis phase — every album batch blew
through analysis and immediately dumped its tracks into the shared download pool,
all pre-marked 'searching'. The intended serialization never happened.
Fix: add serialize= to run_full_missing_tracks_process. Album-bundle batches
(dispatched on album_bundle_executor) pass serialize=True and now hold their pool
slot via _wait_for_batch_drain() until every task in the batch reaches a terminal
state — so only ~N albums are in flight at once. The wait is passive (downloads
are driven by the monitor + completion callbacks on other threads, so no
deadlock) and bails on shutdown, a removed batch, or a safety cap. The residual /
playlist / manual paths run on the SHARED pool and pass serialize=False (blocking
there would steal a real download worker), so they're unchanged.
Tests: _wait_for_batch_drain returns immediately when all-terminal, waits until
tasks finish, bails on shutdown, respects the cap, handles a missing batch. 975
download/wishlist tests pass (only the pre-existing soundcloud /app failures).
carlosjfcasero: "append" sync mode still recreated the playlist (wiping image +
description) on both the sync-page auto-sync and the Playlist Pipeline. Root
cause: _run_sync_task defaulted sync_mode='replace', and every AUTOMATED caller
omits the mode — auto_sync_playlist (mirrored auto-sync + pipeline), the
iTunes-link sync, and Wing It. So those paths always replaced, ignoring the
user's chosen mode entirely. (Manual sync + the per-source discovery path already
passed a mode, which is why it only bit automated runs.)
Fix: when no mode is passed, _run_sync_task resolves the user's configured global
"Playlist sync mode" (normalize_sync_mode(None, playlist_sync.mode)) — the same
thing _submit_sync_task already does — instead of hardcoding 'replace'. The
global default is still 'replace', so users who never changed it are unaffected;
only those who set Append/Reconcile get the corrected behavior.
Tests: normalize_sync_mode(None,'append')→'append' (and 'replace' unchanged);
auto_sync_playlist must not force a mode (no sync_mode kwarg / no 7th positional)
so the resolution can happen. 896 sync/automation/discovery/playlist tests pass.
Part 1 stopped existing full dates being destroyed; this adds first-class support
for full release dates so they can be set + persisted instead of truncated to a
year at the DB layer.
- Schema: new nullable `release_date TEXT` on the albums table (idempotent
ALTER-ADD-COLUMN repair on startup + the live CREATE). NULL = year-only, every
reader falls back to albums.year, so it ships safe/dormant.
- Tag writer: write_tags_to_file + build_tag_diff prefer db_data['release_date']
(the full date) over the year int; _date_to_write writes the full date. When
there's no release_date it's exactly Part-1 behavior (year, preserving an
equally-specific existing file date).
- Retag read path: SELECT al.release_date in the tag-preview/write queries and
thread it into _build_library_tag_db_data.
- Manual edit: release_date added to ALBUM_EDITABLE_FIELDS + a "Release Date"
field (YYYY-MM-DD, validated client-side) in the album editor; the artist-album
query returns it so existing values show. User-set dates are authoritative.
- Enrichment: Spotify + iTunes workers store the source's full release_date
(YYYY-MM / YYYY-MM-DD) when present, only when empty — never clobbering a
manual value.
Tests: writer uses release_date over year + overrides an existing file date;
falls back to year when absent; diff compares the full date. Migration verified
idempotent + enrichment no-clobber. 1435 tag/retag/db/library tests pass.
Sokhi's log showed the actual cause: with the wishlist "allow duplicates" toggle
on, is_track_missing_from_library skips a track when _albums_likely_match() says
the wanted album and a library album are the same — and it was matching albums
that differ ONLY by a decimal volume number:
[AllowDup] Album match — skipping (wanted: '...Vol.5', library: '...Vol.5.5')
[AllowDup] Album match — skipping (wanted: '...Vol.5.5', library: '...Vol.4.5')
His character-song CDs share track titles across volumes, so tracks from albums
he DOESN'T have got skipped because a same-titled track sits in a similarly-named
volume he DOES have — the partially-filled discography never completed.
Root cause: _normalize_album_for_match strips the dot ("Vol.5.5" -> "vol 5 5"),
and _VOLUME_MARKER_RE captured only a single trailing digit, so Vol.5, Vol.5.5
and Vol.4.5 all reduced to marker "5" and the volume-disagreement guard never
fired. Fix: capture the full multi-part number ((\d+(?:\s+\d+)*)) so "5" / "5 5"
/ "4 5" are distinct and the guard correctly rejects the match.
(Not lookback — the log confirms lookback=all was already honored.)
Tests: decimal/multi-part volumes (incl. the real CJK names) now block the
match; identical decimal volumes + naming-drift cases still match. 111 watchlist
tests pass.
THE bug behind "embeds art but never writes cover.jpgs": _fix_missing_cover_art
passed `details['album_folder']` (= dirname of the raw DB path, e.g. Jellyfin's
/data/music/...) as the target folder. That path doesn't exist inside the
SoulSync container — only the resolved /app/... path does. So apply_art_to_album_
files' `os.path.isdir(target_dir)` was False and the ENTIRE cover.jpg block was
silently skipped: embedding still worked (it uses the resolved paths) and the DB
thumb updated, but the sidecar was never written. Exactly Sokhi's symptom + the
"Cover art already present — database thumbnail updated" toast (cover_written
stayed False).
Fix:
- _fix_missing_cover_art: derive the folder from the RESOLVED file
(os.path.dirname(resolved[0])), never the raw album_folder.
- apply_art_to_album_files: bulletproof it — if the passed folder doesn't exist,
fall back to the real directory of the files instead of silently skipping.
Tests: a non-existent folder still writes cover.jpg to the real file dir. 1348
cover/art/repair tests pass.
Root cause of Sokhi's endless 0-findings: the SCAN resolved each track's path
through the path-mapping layer with NO fallback, while the APPLY
(_fix_missing_cover_art) uses `_resolve_file_path(...) or p` — i.e. it falls
back to the raw DB path when mapping returns nothing. On his Docker setup the
mapping returns None, so the scan set has_local=False and skipped every album
(never looking at the folder), even though the apply WOULD have written the
cover.jpg from the raw path.
Fix: make the scan match the apply — if mapping returns nothing but the raw DB
path is a real file (container path == stored path), use it as-is. Now the scan
actually inspects the folder, sees the missing cover.jpg, and flags it; the
apply then writes it from the embedded art.
Tests: unresolved-path-but-real-file → flagged (sidecar_from_embedded); the
fallback does NOT fire for a non-existent path (media-server-only stays skipped).
Kept the [cover-diag] logging from the prior commit to confirm on Sokhi's run.
Sokhi: after the earlier flag fix, scans returned 0 matches — albums with
embedded art but no cover.jpg were treated as fully arted and skipped, so their
cover.jpg never got written.
Root cause: the scan used album_has_art_on_disk (True if EITHER embedded art OR
a cover.jpg exists), conflating the two. Now it checks them separately:
- a local album is flagged if it lacks embedded art, OR it lacks a cover.jpg
sidecar AND cover.jpg writing is enabled (metadata_enhancement.cover_art_
download — Boulder: "only scan for cover.jpgs when enabled").
- an album that has embedded art but no sidecar is fixable even when the API
finds no art: the apply writes cover.jpg from the EXISTING embedded art.
apply_art_to_album_files now writes the cover.jpg sidecar by extracting the
album's own embedded art (new extract_embedded_art) — consistent with the
files, no API call — and only falls back to download_cover_art when there's
nothing embedded to extract. _fix_missing_cover_art no longer bails on a
missing artwork_url when sidecar_from_embedded is set.
Tests: scan flags embedded-but-no-cover.jpg (incl. when API finds nothing),
still skips albums with both, still flags artless albums; apply writes cover.jpg
from embedded art (no download), falls back to download when none, skips when a
sidecar already exists; extract_embedded_art unit tests. 1344 cover/art/repair
tests pass.
When a track shows "Not found", the manual search now accepts a pasted Tidal or
Qobuz track link, not just a typed query (CubeComming: the fuzzy search misses
versions; he can find the track on Tidal but can't get it to appear).
How it works (robust, reuses the proven path): parse the link → (source,
track_id) → fetch the track via the source client's get_track → build a clean
"artist title (version)" query → run THAT source's normal search → bubble the
result whose id matches the link to the top. So the candidate is a normal,
already-downloadable streaming result — no hand-built download encoding — and
it downloads through the existing verified flow.
Degrades gracefully: if the source isn't connected or the link can't be
resolved, it falls back to a normal text search of the raw input — the user is
never worse off than typing it themselves. Scoped to Tidal + Qobuz (the
streaming sources that download by track id, with public track URLs); Soulseek
can't take a link (P2P, no ids), YouTube/SoundCloud are URL-native via a
different path (future).
- core/downloads/track_link.py: pure parse_download_track_link (tidal/qobuz
/track/<id>, slug/region suffixes, scheme-less) + query_from_track_payload
(per-source title/artist, Tidal version-append).
- manual-search endpoint: link detection → resolve → restrict to that source →
id-match bubble.
- placeholder hint mentions pasting a link; maxlength 200→300 for long URLs.
Tests: 14 (parser shapes + payload extraction incl. remix version-append +
qobuz performer/album-artist fallback). JS valid.
#775 already resolves pasted Spotify / Apple / MusicBrainz / Deezer links to an
exact artist/album/track on the Search page. Added Discogs to that set (the one
source in the request not already covered; Amazon left out per request).
- by_id resolver: discogs in SUPPORTED_SOURCES + _KNOWN_HOSTS; parse
/artist/<id>-slug, /release/<id>-slug, /master/<id>-slug (master→album), and
strip the slug to the leading numeric id. Discogs has no standalone track
URLs (tracks live inside a release), so artist + album resolve.
- Fetch dispatch: discogs albums use get_album(include_tracks=False) like
itunes/musicbrainz; artists use get_artist; both already return the common
normalized card shape. Updated the not-a-link hint to mention Discogs.
Frontend needs no change — it adopts whatever source the resolver returns and
Discogs is already a search source. Tests: parse release/master/artist
(slug-stripped, scheme-less) + resolve release→get_album(numeric id) and
artist→get_artist. 43 by_id tests pass.
CubeComming: manual imports of large hi-res Qobuz FLACs came out as empty
shells — no audio, no tags, no length/bitrate. Root cause: mutagen's in-place
save() rewrites the file, and it's NOT atomic; if the process is interrupted or
OOM-killed mid-write (he also reported the memory-growth issue #802), the file
is left truncated — audio and tags gone.
save_audio_file (the chokepoint both the enrichment tag-write AND wipe_source_
tags route through) now saves atomically: copy the original to a temp in the
same dir, write the modified tags into the copy, verify it's still valid audio
(duration > 0), then os.replace() it in. The original is untouched until that
atomic swap, so a crash can only orphan the temp — never destroy the user's
file. Falls back to the plain in-place save (byte-identical to before) when the
atomic path can't run, so nothing is ever left worse off. tag_writer's
write_tags_to_file routes through the same helper.
Verified the atomic path works with REAL mutagen on a real FLAC (audio length
preserved 1.0s→1.0s, tag written, temp cleaned). Tests: replace-on-success,
original-survives-save-failure, corrupt-temp-rejected, no-filename-plain-save,
+ a real-FLAC round-trip (skips without ffmpeg). 2443 import/metadata/tag tests
pass.
Sokhi: filler works but doesn't write cover.jpg. Cause: the sidecar write
(download_cover_art) respects the import-time "Download cover.jpg to album
folder" toggle, while embedding ignores it — so with that toggle off, art
embeds but no sidecar is written.
A job literally called Cover Art Filler should produce the complete art when
you explicitly run it. download_cover_art gains force=True (bypasses the toggle)
and the filler's apply path passes it. The import pipeline still calls without
force, so it keeps honoring the user's setting.
Tests: filler passes force=True; existing cover-write mocks updated for the new
kwarg. 1732 art/cover/repair/import tests pass.
Three related fixes to Tidal track matching and downloading:
1. version-field handling — Tidal stores remix/live/edit qualifiers in a
dedicated `track.version` attribute (e.g. name="Emerge",
version="Junkie XL Remix"), not in the track name. The qualifier
filter and the matcher only looked at name/album, so the exact
recording was discarded. Fold `version` into both the qualifier
haystack and the candidate title passed to MusicMatchingEngine.
2. divergent-version penalty — once versions are visible, OTHER cuts of
the same base become candidates ("(Shazam Remix)" vs "(southstar
Remix)"). Neither title is a prefix of the other, so the prefix-based
version check missed them and the raw ratio stayed high off the
shared base. Apply a heavy penalty when both titles carry different
version descriptors so the wrong cut can't outscore the threshold.
3. rate-limit backoff — the trackManifests endpoint is aggressively
rate-limited; a bare request failed 429 instantly, burned the quality
tier, re-queued the track and hammered again (a self-amplifying
storm). Honour Retry-After / exponential backoff with a bounded retry
count and shutdown-aware sleep.
Adds unit + end-to-end tests for all three.
Boulder (Plex): "flags every album, but everything has art." His albums show
art in the library (served from the embedded file art), but the DB thumb_url
cache column is empty — and the scan flagged on db_missing (empty thumb_url),
so every local album tripped it despite having perfectly good art in the files.
Now: a LOCAL album is flagged only when its files actually lack art
(disk_missing). An empty thumb_url is just a stale cache when the files have
art — not "missing cover art". db_missing still flags media-server-only albums
(no local files), where the DB thumb is the only art there is.
Tests: local+file-art+empty-thumb → NOT flagged (the bug); local+no-file-art →
still flagged; media-server-only+empty-thumb → still flagged.
The scan checked album_has_art_on_disk() on the RAW DB track path, while the
apply (_fix_missing_cover_art) resolves the path first. On any path-mapped
setup (docker mounts, a Plex/SoulSync path mismatch) the raw path isn't found,
disk art reads as "missing", and EVERY album gets flagged — then the apply
resolves the path, finds the art already there, and reports "already present".
Scan and apply disagreed purely because only the apply resolved paths.
Now the scan resolves the representative path the same way (resolve_library_
file_path, same transfer/download/config inputs the retag job uses) before
checking disk art. Unresolvable → treated as no-local-file (not claimed
disk-missing) so we never false-flag a file we simply can't reach.
Tests: disk check runs on the resolved path (thumb+art → not flagged);
unresolvable path → not flagged + art never checked on None.
JellyfinArtist sets self.thumb (→ artists.thumb_url on scan); JellyfinAlbum
never did. So for Jellyfin/Emby the library scan read getattr(album, 'thumb',
None) == None and albums.thumb_url stayed empty for the WHOLE library. Two
downstream effects: blank album art in the web UI, and the Cover Art Filler
flagging every album as "missing cover art" (db_missing = empty thumb_url) —
making the filler the only thing that ever populated the column, which is
backwards. It should come from the scan, like artist thumbs do.
Fix: JellyfinAlbum.thumb = /Items/<id>/Images/Primary (same shape as the artist
thumb, which already normalizes + displays via fix_artist_image_url). Now the
scan stores album thumbs, the UI shows art, and the filler only flags albums
that are genuinely missing art. Navidrome + Plex already set album thumbs.
Tests: album exposes the Primary-image thumb, None without an id, same shape as
the artist thumb.
The real reason Sokhi's cover-art fix "didn't work" and Boulder saw the same
message on a WRITABLE Windows box: _fix_missing_cover_art reported
"Updated database thumbnail, but could not write art to files (read-only?)"
whenever embedded==0 AND cover_written==False — but the overwhelmingly common
cause of that is every file ALREADY having embedded art (skipped, not failed).
The message blamed read-only on a perfectly writable library, which sent us
chasing a read-only ghost across three commits. Sokhi's /fix returns 200
(success), so he was never hitting the genuine EROFS path at all.
Now the message is derived from the art_result breakdown:
- embedded/cover written → "Applied cover art: …"
- failed>0 (non-EROFS) → "… check file/folder permissions"
- skipped>0 (already arted) → "Cover art already present on all N file(s)"
- otherwise → "no file artwork was applied"
Genuine read-only (EROFS) still hard-fails with the clear mount message.
Tests: already-arted→present (not read-only), failed→permissions, EROFS→hard
fail, embedded→success. 453 cover/repair/art tests pass.
Sokhi still hit the read-only error after the statvfs fix (6c3e285a). Root
cause was a gap that fix didn't cover: read_only_fs was only set from the
per-file EMBED write, but download_cover_art SWALLOWS its own cover.jpg EROFS
(logs "Error downloading cover.jpg" and returns). So when an album's tracks
already have embedded art, the embed loop is skipped, only the cover.jpg
sidecar write runs, its EROFS is swallowed, and the filler reported success
while spamming the log — exactly Sokhi's case.
Fix (no blast radius — download_cover_art still never re-raises, since its
import-pipeline callers aren't wrapped): on EROFS it now records
'_cover_read_only' on the passed context instead of just logging; apply_art_to_
album_files passes a capture dict and promotes that to read_only_fs. So a
cover-only read-only album now correctly surfaces the read-only message instead
of a silent success.
Tests: +1 — embed skipped (track already arted) + cover.jpg read-only →
read_only_fs True. 472 cover/art/repair tests pass.
#811 (reopen of #792, carlosjfcasero, Emby/Jellyfin): "append" mode clobbered
the playlist's custom image. The post-sync image push only excluded
'reconcile' — so append (which edits the playlist in place via
append_to_playlist) still re-pushed the source image over the user's poster
every sync. Now both in-place modes (reconcile + append) skip the image push;
only the destructive 'replace' (recreate-from-scratch) pushes it.
append_to_playlist + set_playlist_image were verified to NOT touch tracks or
description (image push only POSTs /Images/Primary), so this is the identity-
clobber fix for append.
Tests: append + reconcile preserve the image, replace still pushes it.
CubeComming #804: importing Coldplay "Yellow" (the 269s Parachutes album track,
correctly tagged) was quarantined — "Duration mismatch: file is 269.2s, expected
266.0s (drift 3.2s > tolerance 3.0s)". The expected 266s came from a re-resolved
*single* edition, not the file's actual album. The duration-agreement integrity
check exists to catch truncated/wrong slskd TRANSFERS — but a manual import is
the user's own already-tagged file being sorted, so checking it against a
re-resolved release just manufactures false quarantines.
Fix: both manual-import paths (singles + album) now mark the context
is_local_import; the integrity check skips the duration-agreement leg for local
imports via expected_duration_for_check() (new pure helper). The size +
mutagen-parse legs still run, so genuinely broken files are still caught — only
the release-vs-file duration comparison is skipped, and only for manual imports.
slskd downloads are completely unaffected.
This does NOT change the deeper matching (file still groups under Singles vs the
Parachutes album — the #767 canonical-version family); it stops the false
quarantine so the file imports.
Tests: 4 on the helper (local skips, download keeps, zero/None/garbage, string
coercion) + updated the routes context assertion. 557 import/integrity tests pass.
CubeComming #804: since 2.6.7, importing already-tagged files (Bruno Mars,
Coldplay) blanked EVERY tag and filed them under "Unknown Artist". Root cause:
both metadata-enhancement blocks in post_process_matched_download did
`except Exception: wipe_source_tags(file_path)` — a full audio.tags.clear() +
strip + clear_pictures. But enhancement throwing means NO new tags were
written, so wiping just destroys the originals. A transient enhancement error
on a well-tagged file = total metadata loss. (The reported "bitrate change" is
a red herring: mutagen padding on re-save, not a re-encode — ReplayGain only
reads via ffmpeg and tags via mutagen.)
Fix: gate the failure-path wipe on should_wipe_tags_on_enhancement_failure()
(new pure, tested policy) — only wipe UNMATCHED downloads (likely junk source
tags); NEVER wipe a clean/matched import, preserve its existing tags + log.
Unmatched-download behavior is unchanged, so the only thing that changes is the
broken case.
Tests: 3 pin the policy (clean→preserve, unmatched→strip, falsey→strip).
1211 import/pipeline/metadata tests pass.
The destructive job's findings-vs-auto toggle was 'auto_delete: False'. Renamed
to 'dry_run: True' to match the Re-tag job's convention and make the safe
default unmistakable: dry run ON (default) = findings only, deletes nothing;
dry run OFF = hands-off auto-delete. Behaviour-identical to the previous
default — just clearer + consistent. Help text + tests updated.
A Library Maintenance job that cleans up downloads tracked by Download Origins
once they pass a per-origin retention window — findings by default, opt-in
auto-delete.
A download is only ever proposed for deletion when ALL hold: older than its
origin's retention, NOT still in an actively-mirrored playlist / watched
artist, and played fewer than the keep-threshold (default 2 → "played more
than once is kept"). Only touches downloads recorded from the Download Origins
feature forward — never pre-existing or manual library.
- core/library/expired_cleanup.py: pure decision core (retention_cutoff,
is_expired, select_expired) — no DB/clock, fully tested. play_count is the
reliable listen signal (last_played is often unpopulated, so recency isn't
used).
- ExpiredDownloadCleanerJob: gathers facts (play_count via a new
get_origin_cleanup_candidates join; active-mirror via get_mirrored_playlists;
watch via get_watchlist_artists) and either creates 'expired_download'
findings or, with auto_delete on, deletes in-scan. Default OFF, both
retentions default 'off'. Settings auto-render in the Library Maintenance
panel (same as Cover Art / Lyrics / Re-tag).
- delete_origin_download(): shared delete (resolve path → remove file → drop
track row → drop history row); a file that won't delete keeps its row +
reports. Used by auto mode AND the _fix_expired_download apply handler.
- Frontend: type/action ('Delete')/result labels + finding detail render.
Tests: 9 on the pure brain (windows, off, per-origin, protected, play-count
threshold, bad age) + 7 on the job (no-op when off, findings, mirror/watch
protection, auto-delete, delete helper missing/real file). 185 repair/origin
tests pass.
Sokhi got a read-only error from the cover-art filler with NO ':ro' in his
compose. Root cause: my earlier Tim fix added a statvfs pre-flight that bailed
when f_flag & ST_RDONLY — but union/FUSE/network filesystems (mergerfs,
rclone, NFS), ubiquitous in self-hosted setups, misreport those mount flags.
A perfectly writable library could be flagged read-only and blocked. statvfs
is a guess; the only honest test is whether an actual write raises EROFS.
- Removed the statvfs pre-flight entirely. Read-only is now detected solely
from a real EROFS on the embed write, which also fast-fails the remaining
files (so no statvfs needed for the fast-fail Tim wanted either).
- Broadened the user message: a genuine read-only mount isn't always ':ro' —
could be a read-only host/NFS/SMB mount or a mergerfs read-only branch.
Tests: writable FS succeeds even when statvfs would claim read-only (the
regression), real-EROFS-on-write still flagged + bails the rest, EACCES still
not conflated with EROFS. Dropped the now-moot Windows-statvfs test (statvfs
is no longer referenced). 445 art/cover/repair tests pass.
Second lock-in catch: tracks.duration is stored in MILLISECONDS (schema), but
the scan passed it to LRClib as SECONDS. LRClib's exact-match-by-duration
strategy would never hit (215000s vs the real 215s), silently falling back to
the fuzzier title/artist search and storing the wrong duration in the finding.
Now divides by 1000 (guards against 0/garbage). Lyrics were still being found
via the fallback, so no track was missed — just less precise matching and a
wrong stored value. Test pins 215000ms → 215s.
Lock-in pass caught a real bug in 1051ef24: the retag lyrics path stuffed the
library title/artist into a plan's db_data to feed the lyrics query — but
db_data is exactly what write_tags_to_file writes ("only writes fields that
have DB values"). So an UNMATCHED track (one with no source match, meant to
get art/lyrics only) would have had its title/artist tags overwritten from
the library values — an unintended tag write on a track we never verified.
Fix: each plan now carries a separate READ-ONLY lyrics_meta
({title, artist, album}) sourced from the library track + album scope, kept
entirely out of db_data. apply_track_plans reads lyrics_meta for the query
(db_data fallback for older plans); unmatched plans keep db_data={} so no tags
are written. _fix_library_retag threads lyrics_meta through the manual-apply
path too.
Tests: +1 regression pinning that an unmatched lyrics plan calls
write_tags_to_file with EMPTY db_data (no title/artist leak) while still
fetching lyrics. 70 lyrics/retag/repair tests pass.
The lyrics sibling of the Cover Art Filler, plus retag integration — reusing
the existing LyricsClient (LRClib) the import pipeline already uses.
- lyrics_client: extracted the LRClib fetch (exact-match-with-duration →
search fallback) into a shared _fetch_remote_lyrics, used by both
create_lrc_file (unchanged behavior) and a new check-only has_remote_lyrics.
- MissingLyricsJob (core/repair_jobs/missing_lyrics.py): scans tracks with no
.lrc sidecar and — Option A — only flags ones LRClib actually has lyrics
for, so instrumentals/interludes are never surfaced or re-flagged. Registered
in the job list; default OFF; respects the lrclib_enabled toggle.
- _fix_missing_lyrics (repair_worker): applies a finding by fetching + writing
the .lrc and embedding lyrics via create_lrc_file.
- Re-tag tool: new 'lyrics' setting ('fetch'|'skip', default skip). When
'fetch', apply_track_plans now also fetches/refreshes the .lrc per track
(fetch-if-missing, re-embed-if-exists) — threaded through scan gates, finding
details, the auto-apply path, and the manual fix handler. Settings UI
auto-renders the dropdown from setting_options; no markup needed.
- Frontend: type/action/result labels for missing_lyrics + a finding detail
render case.
Tests: 12 — has_remote_lyrics truth table, sidecar detection, scan (only-
fixable / skip-existing / lrclib-disabled), the apply handler, and retag
lyrics_action on/off. 694 repair/lyrics/cover/retag tests pass.
Closes the last acquisition gap — user-initiated downloads. A blocklist isn't
a censor, so search + discography stay fully visible; instead the download
ACTION is gated, visibly and overridably:
- Download modal (start-missing-process): an up-front check — if the WHOLE
album or artist being downloaded is blocklisted, return 409 {blocked:true}
with the entity, before starting a batch. The modal shows "X is blocklisted
— download anyway?" and re-POSTs with ignore_blocklist:true on confirm
(threaded onto the batch so the Phase 2a per-track filter skips it).
Scattered single-track bans still fall through to the 2a filter quietly.
- Manual /api/download (search-result download): source-file-centric, so it
matches the blocked ARTIST by name; same 409 + confirm + override. search.js
now sends artist/title so the guard has something to match.
- Precedence confirmed: force-download overrides "already owned", NOT a ban
(the 2a filter runs on the force-expanded missing list).
Frontend: shared confirmBlockedDownload() helper; modal + search callers
handle the blocked response and retry with the override.
Tests: manual download blocked-by-name / unrelated-allowed / override-passes,
and the modal up-front 409 for a blocked album. 8 blocklist API tests pass.
Phase 1 guarded the wishlist; Phase 2a closes the other auto-acquisition path.
Playlist sync, album download, and discography backfill all flow through
run_full_missing_tracks_process, which queues missing tracks at one point —
right where the explicit-content filter already drops tracks. The blocklist
filter slots in beside it: each missing track is checked and a banned
artist/album/track is dropped before queueing (logged with a count), so a
blocked item can't slip in via these flows.
Same brain as Phase 1: the wishlist guard's matcher is generalized to
db.blocklist_reason_for_track(profile_id, track_data, source=None) — the new
`source` param lets the queue path supply the batch source, since an analysis
track dict may not carry a 'provider' field (artists still match by name
fallback regardless). One method, two callers (wishlist + queue), one cascade.
Manual single-track downloads (/api/download, candidate picker, redownload)
are deliberately NOT gated here — that's Phase 2b, pending a block-vs-warn-vs-
override policy decision.
Tests: source-fallback isolation (album id-only proves source drives the ID
match; artist name still matches sourceless), and a queue-filter simulation
mirroring master.py. 35 blocklist tests pass (the only failures in the
download family are the pre-existing soundcloud /app ones).
Completes Phase 1 on top of the backend (43c798a7):
- Cross-source backfill: core/blocklist/backfill.py is a pure injected-resolver
core (resolve only missing sources, never raises); core/blocklist/runtime.py
wires the real metadata clients with a confident name-match (exact
significant-token equality; album/track also require the parent artist when
both expose one — no wrong IDs hung on an entry). Resolution runs
synchronously at add time, so a ban is cross-source from the first scan;
the artist name-fallback in matching covers any gap.
- API: GET/POST/DELETE /api/blocklist (profile-scoped) + /api/blocklist/search
(thin wrapper over the manual-match service search on the active source, so
the modal needn't know the source). Add resolves the other sources before
storing.
- Modal (webui/static/blocklist.js): tabbed Artists/Albums/Tracks in the
revamp design language (accent light-edge, pill tabs, debounced search with
spinner + out-of-order guard, per-result Block, "currently blocked" list
with a match-status star and per-row remove). Opened by a new "Blocklist"
button on the watchlist page, next to Download Origins.
Tests: 5 backfill (fill-missing-only, None/exception handling, arg shape) + 4
API (search proxy, add→backfill→list→delete round trip, validation). Modal
registered in the script-split onclick-coverage test; JS syntax-checked.
A proper artist/album/track blacklist (distinct from download_blacklist, which
stays untouched). ID-keyed across metadata sources so a ban survives a source
switch; profile-scoped; cascade artist→album→track.
- core/blocklist/matching.py — pure decision core (no I/O): build an index from
rows, candidate_block_reason() walks track→album→artist. Same-source ID match
is primary; artist NAME is a fallback (covers the ID-backfill window);
albums/tracks are ID-only (common titles like "Greatest Hits" must not
false-positive across artists). Source-isolated so a numeric Deezer id can't
collide with a numeric iTunes id of a different entity.
- DB: new `blocklist` table (profile_id, entity_type, name, 4 source-id cols,
match_status) + CRUD, match-row fetch, backfill-pending query, id-backfill
update (COALESCE — fills NULLs only).
- Guard: _wishlist_blocklist_reason at the top of add_to_wishlist — every
auto-acquisition path funnels through it, so one check covers watchlist,
discography backfill, repair, manual add. Fails OPEN (a guard error never
blocks a legitimate add).
- Discovery unified IN: legacy discovery_artist_blacklist is migrated into the
blocklist on upgrade (replicated to every profile so no global ban silently
stops working; idempotent; legacy table kept for rollback). Discovery reads
(hero + personalized-playlist SQL) now union the blocklist, so a new-modal
ban filters discovery too.
Tests: 13 on the pure matcher (cascade, id-vs-name rules, source isolation,
precedence) + 10 on the DB/guard (CRUD, profile isolation, dedup, backfill,
end-to-end wishlist refusal + cascade + the discovery migration upgrade path).
50 blocklist/personalized tests pass.
mlody95pl: Navidrome sync works, playback fails ("Failed to resume playback").
Root cause: SoulSync plays library tracks by reading the file off its OWN
disk (/api/library/play → resolve path → serve bytes). "Report Real Path"
gives the correct path STRING, but that's Navidrome's container path — the
files still have to be mounted into the SoulSync container to open them, and
the user's compose has /music commented out. So disk resolution 404s.
Navidrome is a streaming server, so requiring a disk mirror to play from it is
the real limitation. Now, when a library file isn't on SoulSync's disk and the
active server is Navidrome, playback streams through the server's own Subsonic
/rest/stream API — no mount needed:
- NavidromeClient.build_stream_url(song_id, max_bitrate) — token-authed
/rest/stream URL (mirrors build_cover_art_url; password never exposed).
- /api/library/play: on disk-miss, _build_library_stream_url (Navidrome-only;
uses the song id sent by the player, or a DB lookup by file_path) sets a
session stream_url instead of failing.
- /stream/audio: proxies that stream_url with Range passthrough so HTML5
seeking works, streaming upstream bytes through in 64KB chunks (no full-file
buffering).
- session state gains stream_url; the two library-play callers now send the
track's server id.
Disk playback is unchanged (file_path path still wins when the file resolves),
so Plex/Jellyfin and mounted-Navidrome setups behave exactly as before.
Tests: 7 on the URL builder (auth shape, no-transcode default, maxBitRate,
guards) + 4 on the play-fallback routing (navidrome-only, passed-id vs
DB-lookup, none). 200 navidrome/stream/media-server tests pass.
Pache711: a cover-art finding showed the (correct) found album art next to a
(wrong) artist image with one "Apply Art" button — no way to take one and
skip the other. Turned out "Apply Art" only ever applied ALBUM art anyway;
the artist image was display-only context, so the bundling was an illusion
the UI created.
Now the finding is genuinely multi-target:
- scan (missing_cover_art.py): also searches for an artist image (always, so
a WRONG existing one can be replaced — Boulder's call), name-matched
exactly. Stored as found_artist_url only when it differs from the current
artist thumb, so nothing is offered when there's nothing to change.
- apply (_fix_missing_cover_art): honors a target via _fix_action —
'album' (default, unchanged "Apply Art" behavior: DB thumb + embed +
cover.jpg), 'artist' (the artist's DB image), or 'both'. New _fix_artist_art
sets artists.thumb_url for the album's artist.
- UI: each found image gets its own apply button — "Use for album" /
"Use for artist". Applying either resolves the finding, so taking the
correct one and ignoring the wrong one IS "fix one, dismiss the other".
Current artist art shows as "(current)" context with no button.
Default stays album-only, so the plain Apply Art button and every existing
caller behave exactly as before. Tests: 5 on the apply targets (artist-only /
album-only / default / both / missing-url) against a real SQLite DB, plus the
existing cover-art suite updated for the new artist search. 107 repair/
cover-art/UI-integrity tests pass.
Ashh: the manual-match modal fuzzy-searches a service and shows the top 8.
When the right release isn't in those 8 (common title — their example was
"Idols", which returns 8 unrelated releases and not Yungblud's), there was no
way through. But the user usually already knows the exact MBID.
Now the modal's search box doubles as a direct-ID box. Paste a MusicBrainz
MBID (bare UUID or a musicbrainz.org URL) and SoulSync looks that exact
entity up and shows it as the single result to confirm + Match — no fighting
the search ranking.
- core/library/direct_id.py: pure detector, returns the canonical ID only
when the text unambiguously IS one (whole-query UUID, or a UUID inside a
musicbrainz.org URL). "Idols", "Yungblud Idols", a UUID buried in free
text → None, so normal search is never hijacked.
- _search_service: direct-ID fast path before the fuzzy search —
get_release (→ get_release_group fallback for albums) / get_artist /
get_recording. A pasted-but-unresolvable ID falls THROUGH to fuzzy search,
so a typo can't dead-end the modal.
- UI: MusicBrainz placeholder now says "…or paste a MusicBrainz ID/URL".
Detector is service-keyed so Spotify/iTunes/etc. direct IDs can be added
later; today only MusicBrainz has a confirmable direct lookup, matching the
reporter's ask + screenshot. 9 tests: detector truth table (bare/URL/plain/
buried/other-service) + dispatch (confirmed release, release-group fallback,
unresolvable→fuzzy, plain query skips direct lookup).
noldevin's first torrent was stuck "downloading metadata" — a dead magnet
with no peers. The poll loop would ride the full album deadline (6h default)
on it, holding the worker the whole time, with no built-in escape.
New stall handling, off the existing poll loop:
- core/download_plugins/torrent_stall.py — pure StallTracker (clock injected,
no I/O): forward byte progress resets a stall clock; once a torrent spends
the stall timeout in a working state (queued/downloading/stalled/error)
with zero progress, it's stalled. seeding/completed/paused never count.
Covers the metadata-stuck case (0 bytes, 0 progress) and a dead mid-download
swarm with one rule.
- _handle_stalled: 'abandon' (default) removes the torrent + its partial data
(a metadata stub is junk) and fails the download so the next source can try;
'pause' parks it in the client for the user. Adapter errors are swallowed —
the download still fails cleanly.
- two settings (download_source.torrent_stall_timeout_seconds = 600,
torrent_stall_action = 'abandon'); timeout 0 disables, restoring the old
ride-the-deadline behavior. Config-key driven, matching the existing
album_bundle_* tuning knobs (no UI form, same as those).
Tests: 18 on the tracker + settings (timeout trip, progress reset, idle-state
exemption, pause→resume clock restart, disable, parse tolerance) + 3 on the
plugin action path (abandon removes w/ delete_files, pause pauses, adapter
error survived). 158 torrent-family tests pass.
wolf39us: "It keeps unauthenticating... daily" — re-auth fixes it until the
next day. Mechanism: spotipy's token cache was a loose FILE at
config/.spotify_cache. /app/config is a declared VOLUME, but a compose file
that doesn't map it explicitly gets an ANONYMOUS volume — recreated empty on
every container pull. So a nightly Watchtower update kept all his settings
(config lives in the database now) while silently dropping the OAuth tokens.
His redirect-URI change won't help: callback URLs only matter during the
initial handshake, never for refresh.
New DatabaseTokenCache (spotipy CacheHandler) stores the token payload in
the same database-backed config store as every other setting — tokens now
survive exactly as long as the rest of the configuration does. The legacy
file is imported once on upgrade (no forced re-auth) and removed on logout;
a failed cache write logs and never raises (spotipy calls it mid-request).
Tests: roundtrip, JSON-string tolerance, one-time legacy import (store wins
after the file vanishes), garbage file ignored, logout clears both stores,
write failure never raises. 204 spotify tests pass.
Netti93's follow-up report (single artist at download time, correct only
after retag) reproduces as FIXED on current dev — verified live against
Deezer's API with his literal track ('VERLIEBT IN MICH', FAYAN feat.
Dalton) and his exact config, through the real tag writer onto a real MP3:
TPE1=FAYAN, TIT2 gains '(feat. Dalton)', TXXX:Artists=[FAYAN, Dalton].
His last test (2.5.6 / May-19 dev) predates the fixes that closed it
(d5de724f contributors upgrade hardening, 0769fcd5 collab-tag loss).
These tests pin the full direct-download shape so it can't quietly
regress: Deezer /search payload (one artist) + provider on the candidate
(not the context) -> contributors upgrade fires -> feat_in_title and
artist_separator both honored. Network-free (client mocked with the live
API's verified response shape).
37725457 fixed _match_to_itunes to use the real iTunes client and flagged
the cross-source corruption as a possibility. Boulder's live DB proves it
happened: 6 of his 9 watchlist "iTunes" ids EQUAL the artist's Deezer id
(Taylor Swift's "iTunes" id was her Deezer id 12246; the real one is
159260351) — written back when the misnamed MetadataService.itunes slot
held a DeezerClient. The June-4 batch (Green Day, SOAD, Vulfpeck, ...) got
NULL instead because the slot now holds the Spotify primary.
The fix alone can't heal those rows: the backfill only fills EMPTY ids, so
a wrong non-empty id is permanent. New migration clears itunes_artist_id
where it equals deezer_artist_id (the corruption signature — distinct id
spaces, so a legitimate equal pair is effectively impossible, and the worst
case is a NULL that re-matches correctly on the next scan). Idempotent by
construction; similar_artists checked clean (its backfill always used the
registry correctly).
Tests: corrupted row cleared / legit + no-deezer rows kept / idempotent —
via a real re-init with the per-process init memo cleared (an app restart).
Boulder noticed his recently added watchlist artists (June 4 batch) have no
iTunes match while Spotify/Deezer/MusicBrainz matched fine. The rotated log
has the receipt: "Cannot match to iTunes - MetadataService not available" ×8
→ "Backfilled 0/8 artists with itunes IDs", every scan.
_match_to_itunes was the only matcher with no fallback: it read the PRIVATE
_metadata_service attr, which is None whenever the scanner is constructed
from a spotify_client — the normal web_server wiring — and gave up, while a
lazy-loading metadata_service property sat right next to it and the deezer/
discogs/musicbrainz matchers all fall back to their registry clients. Bonus
landmine: even when set, metadata_service.itunes is the FALLBACK-client slot
and may actually be a DeezerClient (per _match_to_deezer's own comment), so
"iTunes" matching could have stored a Deezer artist ID as itunes_artist_id.
Now mirrors the other matchers: canonical registry get_itunes_client().
Self-healing — missing IDs are re-attempted every scan, so existing
watchlists backfill on the next run with no migration needed.
Tests: match works with _metadata_service=None (the exact production
condition), unconfident result returns None, missing client degrades
gracefully. 103 watchlist tests pass.
CI failed all 7 requeue tests that passed locally. Root cause is a real
shipping bug, not test flake: config/settings.py's default template set
retry_next_candidate_on_mismatch: False ("Default off — opt-in") while the
monitor reads it with inline default True and the PR documents it as ON.
Outcome split the userbase: a FRESH install (or CI's clean runner) gets the
template key = retry engine silently OFF; an existing config.json lacks the
key = inline True wins = engine ON. Same code, opposite behavior, decided by
install age.
- template aligned to True (the documented + approved default; existing
installs already behave this way via the inline default)
- the requeue tests now pin the toggle ON via the wiring helper instead of
reading the runner's ambient config — CI's fresh defaults vs a dev's
lived-in config.json must never decide whether they pass. _patch_config
composes (it wraps the pinned get and falls through).
64 retry-engine tests pass; fresh-default simulation confirms the toggle
resolves True.
Boulder's lock-in question caught it: the read-only pre-flight called
os.statvfs unconditionally, which doesn't exist on Windows, and the
AttributeError wasn't covered by the except OSError — the whole cover-art
apply would have crashed for every Windows install (docker images are
Linux, so the reporter was fine; the maintainer wasn't). getattr-guarded
now: no statvfs -> skip the pre-flight, per-file EROFS detection (errno is
cross-platform) still active. Test pins the no-statvfs path.
Tim (Discord): cover-art automation fails with '[Errno 30] Read-only file
system' on every file; he chmod 777'd and nothing changed — because EROFS is
the KERNEL refusing writes to a docker ':ro' volume mount, which no chmod
can fix. SoulSync's response was a wall of per-file warnings and a fix
result that still said success with a soft "(read-only?)" hint.
- apply_art_to_album_files now pre-flights the album folder with statvfs
(asks the kernel, writes nothing): a read-only mount short-circuits the
whole album instead of failing file by file. Belt: a per-file/cover EROFS
(overlay quirks where statvfs lies) still sets the flag.
- the repair worker's apply now FAILS the finding with the actual cure:
"remove ':ro' from the volume mapping and recreate the container — chmod
cannot change this". EACCES (a real permissions problem chmod CAN fix)
deliberately keeps the old soft path.
Tests: RO mount short-circuits before any file/cover write, save-time EROFS
still flagged, EACCES not conflated with EROFS. 29 art/repair tests pass.
Sokhi (continued from #806): volume-numbered series ('B小町 …キャラクター
ソングCD Vol.2' / 'Vol.2.5' / 'Vol.4' / 'Vol.4.5') got each other's art from
both normal downloads and the retag tool. Two distinct holes, one principle:
1. The art picker's _album_matches validates by significant-token SUBSET —
built to tolerate '(Deluxe)'/'- Remastered' suffixes. CJK strips out of
the normalizer entirely, so Vol.4 → {b,tv,cd,vol,4}, a clean subset of
Vol.4.5's {b,tv,cd,vol,4,5}: the wrong volume validated as "the same
album with a suffix". Affected every fuzzy art source (iTunes, Deezer,
AudioDB, Spotify) in downloads, retag, and the missing-art repair.
2. MusicBrainz match_release scores by string similarity — Vol.4 vs Vol.4.5
is 0.973, so the wrong volume could win the match outright, and its MBID
then feeds Cover Art Archive with NO downstream validation (CAA is
MBID-keyed, trusted by design). With Sokhi's MB metadata source this is
the likely path in his logs (his release-group 404s push re-matching).
The shared rule (core.text.title_match.numeric_tokens_differ): digit-bearing
tokens must be IDENTICAL between the two titles. A number on one side only —
volume, part, sequel, remaster year — is a different release, never a
suffix. '1989' vs '1989 (Deluxe)' still matches (digits shared); 'Album' vs
'Album 2' now rejects (sequels!). Art picker rejects outright (falls through
to next source / the download's own art — the designed cost of a false
reject); MB matcher halves the candidate's confidence, landing it below the
70 gate while the exact-volume result is untouched.
Tests: helper truth table, the exact reported pairs through _album_matches,
and match_release end-to-end (wrong volume alone → no match beats a wrong
MBID; exact volume beats near-identical wrong one despite lower MB score).
828 matching/metadata + 301 musicbrainz/retag/artwork tests pass.
Boulder: "Taylor Swift shows only 8 albums, nothing before 2022, no singles,
no EPs" — for every artist (actually: every WATCHLIST artist). Traced live:
get_artist_albums caches its result under an UNQUALIFIED key (no limit/page
info), and the watchlist's new-release probe (limit=5, max_pages=1 — the
April "reduce watchlist API calls ~90%" optimization) stored its truncated
single page in that same slot. The artist detail page reads the cache first,
so a watchlisted artist's page showed only the newest handful of releases —
newest-first, hence "nothing before 2022" — re-poisoned on every scan, with a
30-day TTL. When the source-priority fetch comes back tiny, the page's
fallback path quietly serves it, so the symptom looked like a discography
filter bug. Not related to the #808 matching change (that is a pure max(),
provably additive).
Three pieces:
- get_artist_albums tracks whether the fetch stopped while more pages
existed (truncated) and only caches COMPLETE discographies. Individual
albums keep their opportunistic caching — they're complete entities
regardless of pagination. A small real discography that fits one page
stays cacheable even under max_pages=1.
- MetadataCache.purge_artist_album_lists(): delete the already-poisoned
album-list entries (TTL would have kept them for weeks); lists rebuild
lazily on the next artist-page visit.
- one-time startup purge in web_server, config-guarded
(maintenance.album_cache_purge_v1), mirroring the startup-repair pattern.
Tests: truncated probe never stores the list (but still returns its page),
complete multi-page fetch caches, and a genuinely-small one-page discography
under max_pages=1 still caches. 1087 spotify/cache/watchlist/artist tests
pass.
carlosjfcasero: 'Champagne Supernova (OurVinyl Sessions)' is in the library
but the artist page shows it unowned and wishlist cleanup never removes it.
Measured with the real catalogs: Deezer/iTunes title the TRACK with the
qualifier while the library track is bare (the qualifier lives in the album
title) — and _calculate_track_confidence crushed that pair to ~0.17: the
"clean" titles keep parenthetical words, so the length-ratio penalty treats
'Champagne Supernova' vs 'Champagne Supernova (OurVinyl Sessions)' as
different songs. (Also confirmed: the OurVinyl release is absent from
Deezer's discography for the artist, so the standard page's 25-release list
not showing it is the source catalog, not a bug.)
Fix 1 — core.text.title_match.strip_redundant_context_qualifiers: a
parenthetical qualifier whose text appears (word-bounded) in the db track's
ALBUM title — or in the other title — restates release context and is
stripped for a comparison variant scored with its own length guard. Genuine
version markers keep their penalty: '(Live)' on a studio album appears in no
context and still blocks; '(Live)' on 'Live at Wembley' correctly matches —
owning the live album IS owning the live cut. Wired into
_calculate_track_confidence, so every check_track_exists consumer (wishlist
cleanup, discography dedup, repair jobs) benefits.
Fix 2 — the artist-page ownership endpoint's album gate: when album-aware
narrowing eliminates EVERY library candidate (the source's album naming just
doesn't resemble the library's — 'Jillette Johnson | OurVinyl Sessions' vs
'Champagne Supernova (OurVinyl Sessions)' ~0.5), fall back to artist-wide
title matching instead of declaring everything unowned off a failed
album-NAME comparison.
Tests: 8 — the exact reported pair end-to-end through check_track_exists,
word-boundary containment ('live' in 'alive' doesn't count), version-marker
safety both ways, and prefix songs still blocked. 1125 matching/wishlist/
library tests pass.
Review findings from PR #801, fixed as promised after merge:
- core/imports/version_mismatch_fallback.py and core/downloads/task_worker.py
used bare getLogger(__name__) — outside the soulsync.* namespace where
handlers attach, so the entire retry story (the [Modal Worker] search/retry
walk and, critically, the "accepting best quarantined candidate as last
resort" warning) never reached app.log. Same bug class as the prepare.py
fix; both moved to get_logger. A repo sweep shows 61 more modules with the
same pattern — noted as its own cleanup project.
- the full-suite run also caught a miss of MINE, not the PR's: the new
origin-history.js wasn't registered in the script-split integrity test, so
openDownloadOriginsModal failed onclick coverage. Registered — and the
onclick scan now iterates the NON_SPLIT_JS registry instead of its own
hardcoded copy, so the next standalone module can't silently skip coverage.
Merged dev verified: PR's 77 tests + 4233 full-suite tests pass (the only
exclusion is the eternal soundcloud /app file); integrity suite 64/64.
Watchlist scans add announced albums on purpose (so singles download the day
they drop), but the future-dated tracks leaked into two hot paths:
- Fresh Tape / Release Radar: future albums got NEGATIVE days_old, and the
recency score (100 - days*7) has no upper clamp — prereleases weren't just
slipping into the radar, they were mathematically FAVORED above every
released track. That's the "50% prerelease" report. The builder now skips
confidently-future albums (and clamps days_old to 0 as a belt).
- Wishlist processing: every auto cycle burned a full Soulseek search +
timeout per unreleased track (~60 tracks/cycle for the reporter). Both the
auto and manual flows now skip future-dated tracks with a counted log line.
They STAY in the wishlist and join the cycle automatically the day their
release date passes — no state, the date check is per-cycle. An explicit
manual track selection overrides the gate (the user asked for those).
The gate (core/metadata/release_dates.py, pure + tested) is conservative by
design: Spotify dates come as YYYY / YYYY-MM / YYYY-MM-DD, and a track only
gates when its date is CONFIDENTLY future at its stated precision. Release
day counts as released; garbage or missing dates never block anything
(including out-of-range months/days, which fall back a precision level).
Tests: 6 covering all precisions, the release-day boundary, garbage
tolerance, dict shapes, and ordered partitioning. 403 wishlist/watchlist
tests pass.
User ask: "a modal that lists the tracks downloaded via watchlist" — extended,
as discussed, to playlists too. One modal, two tabs, opened from the Watchlist
page (watchlist tab preselected) and the Sync page (playlists tab) — same
shared-modal-different-entry-points UX as the rest of the app.
The data: library_history recorded which SERVICE a file came from but never
what TRIGGERED it. New origin/origin_context columns (migration + index) are
written once at the import chokepoint via core/downloads/origin.py, a pure
tested deriver that reads, in priority: an explicit _dl_origin stamp (set at
batch-task creation for direct playlist batches, where the playlist context
otherwise only survived in folder mode), the wishlist provenance already
riding in track_info.source_info (watchlist_artist_name / playlist_name —
watchlist_scanner has stamped these for ages), and the folder-mode playlist
thread. Manual downloads stay unclassified by design. History starts from
now — provenance can't be conjured retroactively.
API: GET /api/download-origins?origin=watchlist|playlist (paged) and POST
/api/download-origins/delete — deletes the file on disk (resolved through the
shared container/host path resolver), the matching library track row, and the
history entries; a file that refuses deletion keeps its row and reports the
error instead of lying.
UI: webui/static/origin-history.js — tabbed modal in the revamp design
language (accent light-edge, pill tabs, entry rows reusing the
library-history-entry components), per-row delete + select-all bulk delete
with honest result toasts, empty/loading states, per-tab totals.
Tests: 8 — deriver priority/shapes (incl. the exact watchlist_scanner
source_info shape and JSON-string survival), origin filtering + counts,
row fetch/delete isolation between origins, delete-track-by-path.
The lock-in pass caught the cost hole: art is fetched PER TRACK, and the old
code never touched archive.org at all — so an archive.org outage was free,
while the new native-first chain would pay a 10s timeout on every track
(a 12-track album = +2 minutes, exactly the import-slowness class we spent
today killing). One failed original now puts originals on a 10-minute
cooldown: subsequent fetches go straight to the 1200px CDN midpoint (the
pre-#806 behavior, full speed) and recover automatically when the cooldown
expires. Locked by a test: track 1 pays the failure once, track 2 never
touches the original. (Also: the missing time import the first run caught.)
The CAA branch of _upgrade_art_url capped art at the /front-1200 thumbnail —
a deliberate flakiness trade-off, but the policy had rotted into inconsistency:
iTunes art already shipped at 3000x3000, and bare /front URLs (release-group
lookups — exactly what the Re-tag flow produces) bypassed the cap entirely,
which is how Sokhi observed retag delivering full-res while downloads got 1200.
CAA URLs now upgrade to the bare /front ORIGINAL (native res, frequently
3000px+). The flakiness concern that motivated the old cap is handled where it
belongs, in the fetch: _fetch_art_bytes now walks an attempt chain — original
-> /front-1200 midpoint -> the original sized thumbnail — so a flaky
archive.org degrades to the old 1200px behavior, never below it.
Tests updated to the new contract (+3 chain tests: native-first, flaky
degrades to 1200 not 250, full chain ends at the thumbnail). 623 metadata +
1267 art-path tests pass.
Caught live by the new lookup timing ("Genius track lookup took 242.4s"):
the 429 handler slept the backoff (30/60/120s) in the CALLING thread and then
re-raised anyway — the import pipeline waited 2x120s per track for lookups
that still failed. Worse, the pre-flight backoff wait also slept while
HOLDING the global Genius API lock, so every other Genius caller queued
serially behind the nap.
Now the backoff is a gate: a 429 opens a 30s->60s->120s window and re-raises
immediately; any call inside the window raises GeniusRateLimitedError on the
spot. The error subclasses requests.RequestException, so every existing
caller (the import's source lookups catch RequestException and skip; the
worker's per-item guards) already handles it as a one-line skip — lyrics and
Genius tags are garnish, nothing is allowed to WAIT for them.
Tests: backoff window fails fast (<0.5s vs the old full-window sleep), a 429
opens and escalates the gate without sleeping, the error is a
RequestException (the no-call-site-changes hinge), success decays the gate.
Measured during a live album download: ~4m15s per track in post-processing
(normal is ~20s), with the time vanishing silently inside embed_source_ids —
up to 5 MusicBrainz calls per track crawling against a degraded musicbrainz.org
while the MB enrichment worker kept eating the same ~1 req/s per-IP budget.
Only Spotify/Last.fm/Genius were in the yield set; MusicBrainz, Deezer, iTunes,
Discogs etc. kept grinding through downloads.
Policy (new core/enrichment/yield_policy, tested):
- downloads active -> ALL enrichment workers yield (post-processing touches
every metadata source). listening-stats (local-only) and repair
(user-scheduled) intentionally keep running.
- discovery active -> the API-contention five yield (spotify/itunes/deezer/
discogs/hydrabase) — discovery never paused anything before, despite the
pause helper literally defaulting to label='discovery'.
- user overrides and user-paused bookkeeping keep their existing semantics;
the dashboard yield_reason label now says WHICH foreground work caused it.
Observability (the 4-minute silence can never come back):
- every source lookup is timed; >2s logs a warning NAMING the source and
duration (core/metadata/source.py _call_source_lookup)
- the pipeline always logs "Metadata enhancement took X.Xs" per track
7 policy tests (incl. the motivating case: MB yields to downloads, keeps
running during discovery); 277 pipeline/enrichment tests pass.
A user reports ~0.7 MiB/s RSS growth; the one theory offered so far
(connection leak) was debunked, so instead of guessing: measure. New
core/diagnostics/memory_tracker wraps tracemalloc behind three GET endpoints
the user can drive from a browser:
/api/debug/memory/start begin tracing + baseline snapshot (idempotent)
/api/debug/memory/report top allocation sites by GROWTH since the baseline
(?top=N), with traced totals + process RSS so we
can see how much of the real growth tracing
accounts for; 15-frame tracebacks name the caller
/api/debug/memory/stop end tracing, free trace bookkeeping
Opt-in by design — tracemalloc shadows every allocation while active, so it
never runs by default. RSS via psutil with a /proc fallback.
Tests: report-without-tracking returns a hint (not an error); a real
start->hog->report->stop roundtrip attributes a genuine 5MB allocation to the
test file (fun fact encoded in the test: 'x'*1000 constant-folds into ONE
shared string and traces as ~40KB — the hog must allocate at runtime); the
stat formatter is duck-typed and unit-tested.
"Liked Songs" isn't a real Spotify playlist — no playlist URI exists for it;
the web UI invents the virtual id 'spotify:liked-songs' and Spotify serves the
collection via the saved-tracks endpoint. The playlist DETAIL endpoint special-
cases that id, but the mirrored refresh path resolves stored ids through
get_playlist_by_id, which fed the virtual id straight into sp.playlist() ->
"http status: 400 ... Unsupported URL / URI" on every sync cycle, silently.
get_playlist_by_id now special-cases the virtual id at the client seam (every
by-id resolver benefits, not just the mirror adapter): it builds the Playlist
from the existing get_saved_tracks() pagination, with the real owner name and
track count. New LIKED_SONGS_PLAYLIST_ID constant owns the magic string.
Safety: get_saved_tracks swallows fetch errors into [] — indistinguishable
from "no likes" — and the virtual playlist is only ever offered when likes
exist. An empty result therefore resolves as a FAILED refresh (None) instead
of a valid-looking empty playlist a mirror sync might propagate by clearing
the server-side copy.
Tests: virtual id resolves from saved tracks and never touches the playlist
endpoint, real ids still do (regression), the mirrored adapter seam returns a
full PlaylistDetail, and empty saved-tracks -> None. 473 passed across the
playlist/mirror/spotify families.
On path-mapped setups (Docker mounts etc.) the scan checked a bare
os.path.isfile() on the raw DB path — false for EVERY track — while the apply
handler resolves container/host mismatches. With a cover-art mode set, the
cover_action kept the album past the "anything to do?" gate, so every album
produced a finding with an empty tracks list whose apply could only ever fail
with "No tracks to re-tag in finding".
- the scan now resolves each track path with the same resolver the apply
handler uses (resolve_library_file_path) before reachability checks and
current-tag reads; plans carry the resolved path
- a finding can never be created with zero tracks — cover-action albums with
no usable tracks are skipped, with a debug log of why (unreachable/unmatched
counts) and the counts surfaced in the finding description
- unmatched-but-reachable tracks now get an art-only plan (empty db_data) so
album cover art covers ALL the album's files, not just source-matched ones —
apply_track_plans already treats empty db_data as a pure cover embed and
counts a failed cover download as skipped, never failed (now locked by tests)
- cover-only findings are titled "(cover art, N track(s))" instead of the
puzzling "(0 track(s))"
Tests: +5 (mapped paths resolve into plans, cover-with-nothing-reachable
creates no finding, unmatched -> art-only plan, art-only plan embeds cover,
failed cover download -> skipped). 87 passed across retag/repair/tag_writer.
Since the per-listener stream sessions refactor (Phase 3b), every browser gets
its own stream session — but the 1s 'tool:stream' socket broadcast still read
the legacy GLOBAL state (the DEFAULT session no real browser uses), so it told
every client "stopped" forever. The frontend skipped HTTP polling whenever the
WebSocket was up, so it only ever saw that wrong broadcast: the backend prep
downloaded the track, moved it into the session's stream folder and sat at
"ready" while the mini player showed nothing. Proxy users whose WebSockets
don't connect fell back to HTTP polling (session-correct) and streamed fine —
which is why this hid so well.
Fix: stream status is inherently per-listener, so stop pretending a global
broadcast can carry it —
- web_server.py: remove the 'tool:stream' emit from the tool-progress loop
(the broadcast thread has no request context; it can only ever see DEFAULT)
- media-player.js: the status poller always polls /api/stream/status (resolves
the caller's own session from the cookie); drop the dead broadcast handler
- core.js: unwire the 'tool:stream' socket listener
Observability fix that made this undebuggable: core/streaming/prepare.py used
getLogger(__name__) — outside the soulsync.* namespace where handlers attach —
so every prep log line (including failures) vanished from app.log. Moved to
get_logger("streaming.prepare") + a regression test locking the namespace.
34 streaming tests pass; ruff clean; web_server compiles; JS syntax-checked.
A single enriched against the deluxe gets every source ID pointing at the deluxe,
so the organizer filed it as e.g. track 2 of a 10-track album. Root cause: the
canonical resolver only ever scored the editions already linked — the correct
single was never even a candidate, and the misfit deluxe scored so low (0.1,
below the 0.5 floor) that nothing got pinned and the priority-walk grabbed the
deluxe anyway.
Fix, in three tested layers:
- resolve_canonical_for_album gains a fetch_alternates seam: when no linked
edition clears the floor, it scores the source's OTHER editions of the same
release and re-picks by best fit (dedup, injected, pure).
- default_fetch_alternates lists the artist's editions and keeps the same-release
ones (edition-blind name match: Deluxe / - Single / [Remastered] all collapse),
returning their tracklists. Favors recall; the scorer is the precision gate.
- _resolve_source does the misfit check inline: it fit-scores the walked edition
and only on a clear misfit searches for a better edition, then persists the pin
on apply (Track Number Repair + future runs agree). Cost-neutral and behavior-
identical for well-fitting albums (no extra API calls); strict_source and the
#758 manual lock are never overridden.
Tests: +4 resolver (expand/no-expand/dedupe/back-compat), +7 alternates (name
matcher + fetcher over fake APIs + cap), +3 organizer end-to-end (misfit->single
+pin, well-fit->no-expand, strict->no-expand). 300 passed across the reorganize
+ canonical family, lint clean.
User report: "Not authenticated with Spotify" daily + workers paused. The
log was misleading — is_spotify_authenticated() returns False for five
distinct reasons (no creds / rate-limit ban / post-ban cooldown / no
cached token / probe failure), and all five API call sites logged the
same bare "Not authenticated with Spotify". So a routine rate-limit ban
read as a logout, and the real cause (logged only at DEBUG) was invisible.
New pure describe_spotify_unavailable() maps the real state to a clear
message (priority matches is_spotify_authenticated): not-configured →
rate-limited ("ban ~Nm left (not a logout)") → post-ban cooldown →
no-token ("not connected — re-authenticate") → "auth check failed (token
refresh may have failed)". A side-effect-free client method
(_auth_unavailable_reason) gathers the live state (reads the cached token,
no API probe) and the 5 sites log it.
Now a daily ban is identifiable as a ban, and a genuine logout is
identifiable as one — so reports like this are diagnosable from the log
alone. 7 tests pin the priority/messaging. Full suite clean.
