Reported by sassmastawillis: the Album Completeness maintenance job
scans 3127 albums in 0.1 seconds and reports 0 findings — for every
user, regardless of whether their library is actually complete.
Restoring an older DB surfaced 7 correct findings, so the code logic
works; the DB state is what's making everything look complete.
Root cause: `albums.track_count` is only ever written by server-sync
paths — Plex's `leafCount`/`childCount` and SoulSync standalone's
`len(tracks)`. It's the OBSERVED count of tracks SoulSync has indexed,
which is always exactly what `COUNT(tracks)` returns for that album.
The completeness job treated it as the EXPECTED total and compared it
against the observed count. They're equal by construction, so
`actual >= expected` is always true: skip, 0.1s scan, 0 findings.
Fix: new `api_track_count INTEGER` column on `albums`, written only by
metadata-source code paths. Populated in two places so the scan is
fast and the fallback is robust.
1. Enrichment workers — shared helper `set_album_api_track_count`
in `core/worker_utils.py`. Called by each worker's existing
`_update_album` method alongside its other album-column UPDATEs:
- spotify_worker: `album_obj.total_tracks` from the Spotify Album
dataclass (already in hand, zero new API calls)
- itunes_worker: same, from the iTunes Album dataclass
- deezer_worker: `nb_tracks` from full_data, falling back to
search_data when the full lookup didn't run
- discogs_worker: count of tracklist rows where `type_=='track'`
(Discogs tracklists interleave heading and index rows that
shouldn't count as songs)
Helper skips the write on zero/None/negative/non-numeric inputs
so a source lacking track info can't clobber a good value a
different source already wrote. Caller owns the transaction —
helper just queues an UPDATE on the caller's cursor without
committing, so it batches cleanly with each worker's existing
multi-UPDATE pattern.
Hydrabase worker deliberately not touched — it's a P2P mirror
that doesn't write album metadata to the local DB. Hydrabase-
primary users hit the fallback path below.
2. Album Completeness repair job — new `al.api_track_count` column
in the SELECT, read first in the scan loop. On miss (album never
enriched, or enrichment workers haven't run yet on a fresh
install), falls through to the existing `_get_expected_total()`
API lookup and persists the result via the same shared helper
(wrapped in connection/commit management since the repair job
runs outside a worker's batched transaction).
Also removed `al.track_count` from the scan's SELECT — now unused
since the observed count was the whole source of this bug, and
leaving a dead SELECT would invite a future engineer to re-introduce
the same comparison.
Help text on the job card was reworded so it honestly describes
current behavior ("counts cached during normal enrichment are used
when available; otherwise the job queries a metadata source
directly") rather than the old "active provider first, then others
as fallback" phrasing, which doesn't match how the cache actually
fills — any enrichment worker that runs can populate it, and the
last writer wins. Document-only follow-up if this edge case ever
bites in practice: add a `api_track_count_source` column so the
scan can prefer the configured primary source's count over others
(e.g. deluxe vs. standard edition mismatches). Not worth the
complexity today.
For existing users, the first completeness scan after upgrade is
fast to the extent their library is already enriched: the workers
already ran and populated `api_track_count` on their normal schedule.
For brand-new installs, the scan's fallback path handles the cold
start — slower, but correct, and subsequent scans are fast.
Does NOT affect:
- Download / post-processing / wishlist / sync code paths — none
of them read `track_count` for completeness semantics.
- Plex / Jellyfin / Navidrome / standalone sync — still write
`track_count` exactly as before; `api_track_count` is a separate
column they never touch.
- Other repair jobs.
- Any UI path — same finding schema, just correct counts now.
Files:
- database/music_database.py — idempotent migration adding
`api_track_count INTEGER DEFAULT NULL` to the existing album-column
check block.
- core/worker_utils.py — new `set_album_api_track_count` helper with
the documented skip-on-bad-input contract.
- core/spotify_worker.py, itunes_worker.py, deezer_worker.py,
discogs_worker.py — one-liner call from each `_update_album`.
- core/repair_jobs/album_completeness.py — scan uses the cache;
fallback path persists API-lookup results via the shared helper;
help text updated to match actual behavior.
- tests/test_worker_utils_album_track_count.py — 9 tests covering
the helper's write/skip contract + no-commit invariant.
- tests/test_album_completeness_job.py — 2 tests for the repair
job's fallback-path wrapper.
- webui/static/helper.js — WHATS_NEW entry.
Credit: sassmastawillis spotted the bug; the "restored older DB
finds 7 albums" signal pinpointed DB state over code logic and
made the diagnosis tractable.
Typing "michael jackson" returned 7 identical-looking cards because
MusicBrainz has many different PEOPLE sharing a canonical name — the
King of Pop plus a NZ poet, a photographer, a mashup artist, a
didgeridoo player, and more, all scoring 80+ on exact-name match.
All 7 passed the score filter. All 7 rendered with the same
fallback image because iTunes/Deezer only know the famous one.
Fix dedupes by normalized (lowercase, whitespace-trimmed) name before
building Artist dataclasses. Keeps the highest-scoring entry per name,
so the King of Pop (score 100) wins over the others (all score 80-81).
Artists with genuinely different names stay separate — a search for
"the beatles" still surfaces tribute bands if they're above threshold.
Implementation note: fetch `max(limit*3, 10)` from MB instead of
`limit` directly, so the dedup pool is large enough to still return
`limit` distinct artists after collapsing duplicates. Previously the
raw fetch was capped at the caller's limit, which would have left
fewer-than-requested results after dedup for common names.
3 new tests (49 total):
- Dedupe collapses 5 same-named entries to 1 (keeps highest score).
- Dedup key is case-insensitive and whitespace-normalized.
- Dedup preserves distinct names ("The Beatles" vs "The Beatles Revival"
stay separate).
Live-verified: "michael jackson" now returns 1 card, "kendrick lamar"
returns 1 card.
Credit: kettui spotted duplicate Michael Jackson cards in the search UI.
Three bugs from kettui's follow-up review pass on the MusicBrainz
search PR, all fixed in one commit because they share UI context.
1. Missing artist images on MB artist results
MusicBrainz doesn't store artist images directly. My earlier commit
returned `image_url=None` on every artist result and trusted the
frontend's lazy-loader — but the lazy-loader's `/api/artist/<id>/image?
source=musicbrainz` endpoint had no handler for MusicBrainz, so it
silently returned None and the emoji placeholder stayed.
Fix plumbs the artist name through:
- `renderCompactSection` stashes `data-artist-name` on artist cards.
- `search.js` and `downloads.js` lazy-loaders pass `name=<artist>` as a
query param.
- `/api/artist/<id>/image` accepts an optional `name` param.
- `metadata_service.get_artist_image_url` has a new `musicbrainz`
branch: since MB has no artist art, it searches fallback sources
(iTunes/Deezer by configured priority) for the artist name and
returns the first image found.
Verified live — Metallica/Kendrick Lamar/Daft Punk all resolve to
Deezer artist images via the name lookup.
2. total_tracks off-by-one on tracks with a release
`_recording_to_track` initialized `total_tracks = 1` and then summed
media track-counts on top. For an 11-track album, it reported 12. An
adapter-level regression introduced when the recording-projection
helper was extracted during the main MB refactor.
Fix: initialize at 0, sum normally. Standalone recordings with no
release (can happen for uncredited remixes etc.) still report 1 via
an explicit fallback — so the existing "single track" case isn't
broken.
3. "Artist Album Title" queries buried specific albums in the
discography list
Bare-name queries like "The Beatles Abbey Road" used to resolve "The
Beatles" as the artist and then browse their full discography — Abbey
Road was buried alphabetically among 200+ releases instead of being
the top result.
Fix adds a title-hint extractor. When the query starts with the
resolved artist name followed by more words, the trailing portion is
treated as a title hint. Browse results are filtered to those whose
release-group title contains the hint. If the filter matches nothing,
falls back to text-search with the hint as the title (the "keep the
old split-by-whitespace fallback" path kettui called for). If text-
search also misses, shows the full discography rather than nothing.
10 new tests in tests/test_musicbrainz_search.py (46 total):
- Title-hint extractor: basic match, case-insensitive, whitespace
tolerance, bare-artist-no-hint, artist-not-prefix-no-hint, word-
boundary required (no false splits on "Metallicasomething").
- Browse filtering by title hint.
- Text-search fallback when the title hint matches nothing in browse.
- Bare-artist queries return the full discography unfiltered.
- total_tracks for single-release, multi-disc, and no-release cases.
Clicking a MusicBrainz album returned 404 because the browse-based
search path now stores release-GROUP MBIDs in Album.id, but `get_album`
still hit `/ws/2/release/<mbid>` directly. Release-group MBIDs don't
resolve as release MBIDs — MB 404s. User log:
GET /api/spotify/album/b88655ba...?source=musicbrainz → 404
Error fetching release b88655ba...: 404 Client Error
The fix requires a two-step resolution for the new browse path:
1. Look up the release-group with `inc=releases+artist-credits` to get
the list of releases inside (original + reissues + regional + promo
editions). MB release-groups routinely hold 5-20 releases.
2. Pick a representative release: prefer Official status over Promo,
prefer releases with a real tracklist over stubs, then earliest date.
3. Fetch that release's full tracklist via `get_release`.
Two extra seconds at the 1-rps rate limit, but it's on click, not on
search results rendering.
Structure:
- New `MusicBrainzClient.get_release_group(mbid, includes)` method.
- New `_pick_representative_release(releases)` helper encapsulates the
ranking logic.
- Tracklist projection extracted into `_render_release_as_album` so
both paths share the same shape construction.
- `get_album` tries release-group first; falls back to direct release
lookup when the MBID turns out to be a release from the text-search
fallback path.
- Canonical Album.id stays the release-group MBID so a re-fetch with
the same URL hits the same code path idempotently.
3 new tests (now 33 total):
- End-to-end release-group → release resolution with mocked client
- Fallback to direct release lookup when rg lookup misses
- Representative-release picker ranks correctly
Verified against live API with the exact MBID that 404'd for the user
(b88655ba... for DAMN. by Kendrick Lamar): now returns in 1.2s with
the full 14-track listing (BLOOD., DNA., YAH., ELEMENT., FEEL., ...).
Three related fixes to make album/track results look like a real
artist discography instead of a firehose of fan-compiled bootlegs.
1. Drop 'compilation' from the release-group browse primary-type filter.
MB's OR filter (`type=album|ep|single|compilation`) silently breaks
when 'compilation' is included — Metallica drops from 1076 matches
to 82 because `compilation` is a SECONDARY type on MB, not a primary
type. The invalid value corrupts the filter for all types, not just
itself. Now we request `type=album|ep|single` which returns the full
1076; actual compilations (primary=Album + secondary=[Compilation])
are filtered out by the studio-preference logic below.
2. Filter release-groups with non-studio secondary-types
(Live/Compilation/Soundtrack/Remix/Demo/Mixtape/Interview/Audiobook/
Audio drama). For Metallica, the first 100 browse results are 12
studio albums + 83 live bootlegs + 5 compilations — without this
filter the Albums section was dominated by 2019-2021 broadcast
recordings. Falls back to the unfiltered list if filtering leaves
the result set empty (covers live-only niche artists).
3. Sort chronologically ASC by first-release-date. Wikipedia-style
discography ordering — debut album on top, then chronological.
Previous DESC sort put the most recent release on top which, for
prolific artists, meant 2020s material before their classics.
Track side of the same fix:
- Re-orders each recording's `releases` array to put studio releases
first before `_recording_to_track` picks up the first release for
album context. Without this, MB's arbitrary release order often
buried the canonical studio album under random live bootlegs.
- Filters out recordings that only exist on live/compilation release-
groups (keeps the ones with at least one studio release). Falls
back to the full set if the artist has no studio recordings at all.
- Sorts recordings by earliest studio-release year ASC so classic
tracks surface first.
Smoke test against live MB API confirmed:
- Artists: [Metallica score=100]
- Albums: Kill 'Em All (1983) → Ride the Lightning → Master of Puppets
→ ...And Justice for All → Metallica (Black Album) → Load → Reload
→ St. Anger → Death Magnetic → Lulu (2011)
- Tracks: real Metallica recordings (Killing Time, Nothing Else
Matters, Creeping Death, etc.) — a few remastered demos still leak
in where MB metadata quality is thin, but the bulk is correct.
- Total latency: 3.5 seconds.
4 new tests covering the studio filter, live-only fallback, preferred
release ordering, and live-only recording exclusion.
Credit: kettui flagged the poor MB results during PR #371 review.
The previous commit's `browse_artist_recordings` call passed
`inc=releases+artist-credits` — but MusicBrainz's recording browse
endpoint rejects `inc=releases` with HTTP 400. The adapter's error
handler returned an empty list, so the Tracks section stayed empty
even though the fix was supposed to populate it.
Browse without release info is useless for our search UI (tracks
would render with no album), so swap to the fielded Lucene search
`arid:<mbid>` on the `/recording` endpoint. That's the canonical MB
pattern for "find recordings by this artist WITH release context":
- arid: search accepts the artist MBID and returns recordings with
`releases` (release-group, date, media) embedded in each result.
- One API call per lookup, same as browse would have been.
Renamed the method to `search_recordings_by_artist_mbid` so the name
matches its behaviour — it's a search, not a browse. Adapter updated
to call the new name; tests updated to match.
Verified against the live API: Metallica's MBID returns 5 recordings
in ~1.8 seconds (vs the previous 400 error).
Cover Art Archive URLs are deterministic from the MBID: a GET either
307-redirects to the image or returns 404. The previous adapter fired
`requests.head(timeout=3)` per search result to probe for the image
first. 10 results × 3s worst-case = up to 30s of blocking HEAD calls
before a search returned.
The probe was defensive overhead — the frontend already handles 404 via
`<img onerror>` fallback. Building the URL deterministically and letting
the browser load it lazily collapses the tail latency to the real MB API
calls (artist-search + browse = ~3s at the 1-rps rate limit).
Also prefer release-group scope over per-release scope when both are
available — release-group covers every edition of an album, so the hit
rate is noticeably higher than pinning to a specific regional release.
Removes now-unused `self._art_cache` and the `requests` import.
Bare name queries (typing 'metallica') now resolve to an artist MBID via
the fuzzy search added in the previous commit, then BROWSE that artist's
release-groups and recordings instead of text-searching release/recording
titles. That's the only way to fix the core garbage-results issue: MB
indexes release/recording titles, not artist names, so 'recording:metallica'
matches random tracks literally titled 'Metallica' (all scoring 100).
Structure:
- `_split_structured_query` — detects 'Artist - Title' / 'Artist – Title' /
'Artist — Title' shapes. When present, text-search is correct (user
gave an explicit title to match).
- `_resolve_top_artist` — memoized per-instance lookup for the top-scoring
artist MBID. Backend fires artists/albums/tracks searches in parallel
against one shared client instance, and albums+tracks both need the
same artist lookup. Cache + lock means one HTTP call instead of three.
- `_release_group_to_album` / `_recording_to_track` — shared projection
helpers between the browse and text paths so both paths return the
same dataclass shape.
Search flow per kind:
- `search_albums('metallica')` → resolve top artist → browse release-groups
with `type=album|ep|single|compilation` → sort by type priority then
release date desc → Album dataclasses for top N.
- `search_tracks('metallica')` → resolve top artist → browse recordings
with `inc=releases+artist-credits` → dedupe by normalized title (MB
has many live/compilation variants of the same song) → sort by release
date desc → Track dataclasses for top N.
- `search_albums('foo - bar')` → structured query → text-search path
(unchanged behavior, now score-filtered to 80+).
- `search_tracks('foo - bar')` → same.
- Both text-search paths also dedupe through `_search_albums_text` /
`_search_tracks_text` helpers, which apply the 80-score filter that
the artist-first path gets free from the resolver's threshold.
Also dedupes text-path tracks through the new `_recording_to_track`
helper, replacing ~60 lines of inline projection code. Net change is
more lines overall (browse + helpers) but the text paths shrank and
the garbage-results issue is fixed.
Credit: kettui flagged the missing Artists section + unusable track
results during PR #371 review.
`MusicBrainzSearchClient.search_artists` has been a `return []` stub
since the feature landed, with a comment claiming the MB tab 'doesn't
show artists.' That's why kettui saw a missing Artists section on the
search page — not a missing render, a hardcoded empty list.
Re-enable it properly:
- New `strict=False` parameter on `MusicBrainzClient.search_artist`
sends a bare Lucene query instead of `artist:"..."`. MusicBrainz
matches bare queries against alias+artist+sortname indexes together,
which is the right behavior for user-facing fuzzy search (finds
typos, aliases, sortname variants). `strict=True` remains the
default for enrichment/AcoustID callers that want exact matches.
- Adapter filters results to `score >= 80`. MB assigns a 0-100 Lucene
score on every hit; the true artist + close variants score 100,
tribute bands and lookalikes typically land in the 40-65 range.
The cutoff keeps "Metallica" (100) and drops "Black Metallica
Tribute Band" (60) without hand-curated lists.
- Results returned as the same `Artist` dataclass used elsewhere in
the search-tab adapter layer. `popularity` carries the MB score
(0-100) so the frontend can sort/highlight top matches if desired.
Add `browse_artist_release_groups(mbid)` and `browse_artist_recordings(mbid)`
to MusicBrainzClient. These hit `/ws/2/release-group?artist=<mbid>` and
`/ws/2/recording?artist=<mbid>` respectively — the correct MusicBrainz
pattern for "give me everything linked to this artist."
Why this matters: our current search adapter calls text-search
(`release?query=...` / `recording?query=...`) for albums and tracks,
which matches entity titles literally. Typing "metallica" hits unrelated
releases titled "Metallica" and recordings named "Metallica" by obscure
bands — every garbage match scores 100 because they're all exact title
matches on the wrong field.
Browse walks the artist→release-group and artist→recording links
directly. Once we know the artist's MBID (from `search_artist`), browse
returns their actual discography instead of title collisions.
No behavior change yet — search adapter still uses the old path. Follow-
up commit wires the new endpoints in.
Reference: https://musicbrainz.org/doc/MusicBrainz_API — "Browse queries
retrieve entities linked to a known entity" vs search.
MusicBrainz mandates a meaningful User-Agent with contact info, warning
that bare strings can trigger IP blocking under load. Our client was
sending `SoulSync/2.3` with no contact — and the search adapter passed
an app version hard-coded at "2.3" that's now stale (UI is at 2.40).
Fix: default contact to the project URL (`https://github.com/Nezreka/SoulSync`)
when no email is supplied, so every request lands as
`SoulSync/<version> ( https://github.com/Nezreka/SoulSync )`. Drop the
search-adapter version suffix to a generic "2" since the exact UI minor
version would add noise to every MB request without helping operators
track issues.
Reference: https://musicbrainz.org/doc/MusicBrainz_API — "it is
important that your application sets a proper User-Agent string."
JohnBaumb's review: "If we're going to refactor the web_server.py soon,
might as well start moving stuff away from web_server.py in our PRs.
_build_source_only_artist_detail, make it a module, it's perfect."
This continues the pattern the prior commit started with the source-ID
lookup helpers: move the pure data-building logic to a side-effect-free
core module, leave a thin wrapper in web_server.py that bridges the
Flask response and the module-global clients.
**core/artist_source_detail.py** — pure function that takes the artist id,
name, and source plus dependency-injected per-source clients (spotify,
deezer, itunes, discogs) and a Last.fm API key. Returns
(payload_dict, http_status) so it isn't coupled to Flask.
**web_server.py wrapper** — builds the client bag from the module globals
(checks Spotify auth, constructs the Discogs client from the configured
token, reads the Last.fm API key) and wraps the core return in jsonify.
147 lines of logic go away from web_server.py; the 24-line wrapper is
purely glue.
**tests/test_artist_source_detail.py** — 21 focused tests covering the
response envelope, the source-specific ID-field stamping for all six
supported sources, the dedup_variants=False contract (the behaviour
that originally motivated the split of MetadataLookupOptions), per-source
genre/follower extraction with safe handling of missing or throwing
clients, and the Last.fm enrichment branch including the no-key and
error-path cases. Runtime 0.26s.
Cin observed that database/api_call_history.json was occasionally
landing on disk truncated mid-write — `_load()` would log
`History file is not valid JSON, starting fresh` and 24h of metrics
would be lost.
Root cause: `save()` opened the file in 'w' mode (which truncates to
0 bytes immediately) and then streamed JSON via `json.dump`. Any
SIGINT/SIGTERM/crash between truncate and final write left the file
half-formed — exactly Cin's symptom of the JSON cutting off mid-array.
Switch to the standard atomic pattern: write to a sibling .tmp file,
flush + fsync, then `os.replace` (atomic on every platform we run on).
Failed writes also clean up the leftover .tmp file. The canonical
file is now either the previous good copy or the new good copy —
never a partial one.
Cin's review note: typing artist_name as plain `str` forced callers
that didn't have a name to pass `""` as a placeholder, which leaks the
parameter's emptiness contract into every call site and reads badly in
tests. Switching to `Optional[str] = None` lets callers omit it.
The function body's `if artist_name and active_server:` check already
handles None and "" identically, so no body changes were needed. Tests
that previously passed `artist_name=""` drop the argument; one new test
covers the omitted-arg path explicitly.
The web_server.py wrapper takes the same default for symmetry.
Cin pointed out that the prior version of test_artist_source_lookup.py
AST-parsed web_server.py to verify a constant and to string-match a
function's response keys. That was a workaround for the fact that
web_server.py can't be imported at test time (it boots Spotify,
Soulseek, Plex, etc.) — the right answer is to move the logic into a
side-effect-free module so it can be imported and tested directly.
This commit:
- adds core/artist_source_lookup.py containing the SOURCE_ID_FIELD
map, the SOURCE_ONLY_ARTIST_SOURCES set, and find_library_artist_for_source
- replaces the inline definitions in web_server.py with imports +
a thin wrapper that injects the active media server
- rewrites the tests to import from the core module directly:
* mapping correctness is now a plain equality assertion
* lookup behaviour is exercised against a real MusicDatabase
* the AST parse and the string-matching contract test class are
gone
- drops the _build_source_only_artist_detail contract test entirely
(the weakest of the four — it was just string-matching the function
body); when that function moves to core/ it can get a real
behavioural test alongside.
Test runtime drops from ~161s to ~5.8s. All 18 tests pass.
Source artists landing on /artist-detail were rendering an almost-blank
hero — image + name + a tiny Download button — because the backend
response only had {id, name, image_url, server_source: null, genres: []}.
The library.js renderers do their best with what they have, and that
wasn't much.
Backend changes (_build_source_only_artist_detail):
- Set the source-specific ID field (deezer_id / spotify_artist_id /
itunes_artist_id / discogs_id / soul_id / musicbrainz_id) on
artist_info so the corresponding service badge renders on the hero.
- Try the source's own get_artist_info / get_artist for genres +
followers (Spotify always; Deezer/iTunes/Discogs when available).
Spotify also fills image_url if metadata_service.get_artist_image_url
came up empty.
- Last.fm enrichment by artist name — bio + listeners + playcount +
lastfm_url. Mirrors what library artists get from the cached
enrichment workers but on demand for source artists.
- All enrichment lookups are wrapped in try/except so a 500 from any
one source doesn't break the whole response.
Frontend (library.js populateArtistDetailPage):
- Watchlist button now initialises for source artists too. Falls back
to artist.id + artist.name when there's no canonical Spotify
identity (which is the common case for non-library artists).
Discography dedup opt-out:
- Added dedup_variants flag to MetadataLookupOptions (default True so
library artists are unchanged). Source-only path now passes
dedup_variants=False so every "Deluxe Edition" / "Remastered" /
"Anniversary" variant the source returns is shown — matches the
inline /artists page behaviour the user was comparing against.
Result: source artists' hero now shows badges + bio + listeners +
playcount + watchlist button + genres in addition to image and name.
Discography lists every release the source returns, not the deduped
canonical view.
User reported searching "Maduk - Leave A Light On" on Tidal silently
downloaded Tom Walker's completely different song of the same name, then
embedded Maduk's metadata into Tom Walker's audio. Three layers of
defense all failed permissively. Two of them are fixed here; the third
(score formula weights) was left alone since these two together cover it.
Layer 1 fix — candidate artist gate (web_server.py:27782)
Old: `if _best_artist < 0.4 and confidence < 0.85: continue`
New: `if _best_artist < 0.5 and confidence < 0.85: continue`
SequenceMatcher returns exactly 0.400 for "maduk" vs "tom walker"
(5-char vs 10-char strings with coincidental char matches), which
slipped past the strict `< 0.4` check. The word-boundary containment
check earlier in the function already short-circuits legitimate
formatting variations to sim=1.0, so falling to SequenceMatcher means
strings are genuinely different. 0.5 closes the fencepost AND gives
a small safety buffer.
Layer 3 fix — AcoustID verification (acoustid_verification.py:316)
When title matches but artist doesn't AND expected artist isn't found
anywhere in AcoustID's returned recordings:
Old: always SKIP (let file through, assume cover/collab)
New: FAIL if artist_sim < 0.3 (clear mismatch)
SKIP if artist_sim >= 0.3 (ambiguous — cover/collab/formatting)
The 0.3 cutoff catches hard mismatches like Maduk/Tom Walker (sim ~0.2)
while preserving benefit-of-the-doubt for borderline artist formatting
differences. Legitimate covers and collabs where the expected artist
appears anywhere in AcoustID's recordings still PASS via the existing
secondary-match loop above.
Both fixes are defense-in-depth — either alone would have caught this
bug. Together they close the pre-download AND post-download gaps.
All 292 tests pass. Version bumped to 2.39 with changelog entries.
Tidal's search engine chokes on long queries with multiple qualifier
words (remix credits, edit labels, bonus-disc markers). User reported
case: "maduk transformations remixed fire away fred v remix" returns 0,
but shortening to "maduk transformations remixed fire away" works.
Behaviour change:
- On a 0-result search, retry with progressively-shortened variants
(capped at 5 total attempts, 100ms pause between).
- Variants (in priority order):
1. strip trailing "(...)" / "[...]"
2. strip all parentheticals/brackets
3-5. drop last 1 / 2 / 3 tokens
6. keep first half of tokens (rounded up)
- Dedupes so identical variants don't re-query.
Safety — qualifier-aware filter:
- Variant keywords (Live / Remix / Acoustic / Extended / Unplugged /
Instrumental / Karaoke / etc.) are extracted from the original query
using word-boundary match so "edit" doesn't match "edition" and
"mix" doesn't match "remixed".
- If the original query carries any qualifiers, fallback results MUST
contain those qualifiers in their track names — otherwise a shortened
query could silently downgrade "Song (Live)" to the studio "Song".
- Tracks that fail the filter are dropped. If no variant produces
qualifier-matching tracks, returns ([], []) — the same outcome as the
original code, so no regression.
Contract preservation:
- Never raises to caller (outer try/except catches orchestration errors).
- Returns ([], []) on any failure path, same as original.
- Original-query successes take the same code path as before — no
behavioural change for queries that already work.
- Defensive guards for None/empty/non-string query (early return).
Logging:
- Preserves original warning/error/info messages for back-compat log
scraping.
- Adds fallback-success INFO log ("Tidal fallback query succeeded: ...")
so successful retries are visible in production logs.
- Adds qualifier-filter INFO/DEBUG logs with kept/total counts.
- Per-attempt exception logs at DEBUG (not ERROR) to avoid noise when
retries succeed.
- Traceback preserved on final failure.
Tests (16 regression tests in tests/test_tidal_search_shortening.py):
- Skowl's reported query reaches his working variant within the cap.
- Paren/bracket stripping priority.
- Short queries produce no variants.
- All variants unique (dedup guard).
- Progressive token drops present for long queries.
- Qualifier extraction is word-bounded (no "edit" in "edition").
- Qualifier extraction is case-insensitive.
- Track name filter requires ALL qualifiers.
- Empty-qualifier list passes every track (original-query behaviour).
All 292 tests pass.
New smart template variable that emits "CD01" / "CD02" etc. in filenames
on multi-disc albums, and expands to empty string on single-disc albums
so mixed libraries don't end up with "CD01" on every single.
Template behaviour:
- total_discs > 1 -> "CD{disc:02d}" (zero-padded, CD prefix)
- total_discs <= 1 -> empty string
- Both $cdnum and ${cdnum} bracket form supported
- Empty value collapses cleanly via existing double-dash regex plus new
leading-dash cleanup pass
Wiring:
- _apply_path_template in web_server.py (download pipeline)
- _apply_path_template in core/repair_jobs/library_reorganize.py
(Reorganize repair job)
- total_discs added to every album-mode template context:
* download pipeline album branch (uses resolved total_discs even for
single-track downloads from search)
* per-album Reorganize preview + apply endpoints (pre-scan all track
tags once, take max disc_number)
* Library Reorganize repair job (already had album_total_discs map,
just added to context dict)
Leading-dash cleanup added to _get_file_path_from_template (web_server)
and _build_path_from_template (library_reorganize) so templates like
"$cdnum - $track - $title" don't leave "- 05 - Title" on single-disc
albums.
UI:
- Template hint in Settings -> File Organization documents $cdnum
- Template validation variable list includes $cdnum
- Reorganize modal variable reference shows $cdnum with example "CD01"
Verified:
- Multi-disc disc 1 -> "CD01 - 05 - Track"
- Multi-disc disc 2 -> "CD02 - 05 - Track"
- Single-disc -> "05 - Track" (no leading dash)
- Templates without $cdnum behave unchanged
- 276/276 tests pass
Two bugs reported in issue #320:
1. Auto-watchlist scan bypassed Global Override settings.
scan_watchlist_profile applied _apply_global_watchlist_overrides, but
the scheduled auto-scan called scan_watchlist_artists directly —
bypassing the override. Users who unchecked "Albums" or "Live" under
Watchlist → Global Override still saw full albums and live tracks
added during nightly scans (per-artist defaults, which include
everything, won).
Moved override application into scan_watchlist_artists itself so
every entry point respects it. scan_watchlist_profile now forwards
the apply_global_overrides flag through to avoid double-application.
2. is_live_version (watchlist + discography backfill) and
live_commentary_cleaner's content patterns used bare \blive\b, which
matched verb uses like "What We Live For" by American Authors,
"Live Forever" by Oasis, "Live and Let Die" by Wings.
Tightened the live patterns to require clear recording context:
(Live) / [Live Version] / - Live / Live at|from|in|on|version|
session|recording|performance|album|show|tour|concert|edit|cut|take
/ In Concert / On Stage / Unplugged / Concert.
Locked in 11 regression tests covering the reported false positives
(What We Live For, Live Forever, Living on a Prayer, Live and Let Die)
and the reported true positives (Dimension - Live at Big Day Out,
MTV Unplugged, etc.).
Version bumped to 2.37 with changelog entries.
Root-cause fix for "scanning 50 artists" then silence: when the master
repair worker was paused, force-run still kicked off _run_job but the
job's first wait_if_paused() blocked forever because is_paused was tied
to the master-enabled state. Force-run now bypasses master-pause —
scheduled runs still respect it.
Also fixes Fix All on discography findings doing nothing: the backend
bulk_fix_findings query had a fixable_types allowlist that excluded
missing_discography_track (and acoustid_mismatch). Added both.
Backfill job rebuild:
- auto_add_to_wishlist opt-in setting — creates findings AND pushes to
wishlist during the scan
- 3-option fix dialog (Add to Wishlist / Just Clear / Cancel) on single
Fix, Bulk Fix selection, and Fix All (page-level)
- Fix All "Just Clear" path uses the clear endpoint with job_id filter
instead of the generic "may delete files" bulk-fix warning
- Batched in-memory matching using get_candidate_albums_for_artist +
get_candidate_tracks_for_albums (same fast path the Library pages use)
- Rich album context per finding (id, name, album_type, release_date,
images, artists, total_tracks) — flows through the wishlist pipeline
so auto-processor classifies each track into the right cycle
(albums vs singles) and post-processing gets correct folder/tags/art
- Per-artist progress logs [N/50] Scanning ArtistName
- Default interval 24h (was 168h); all release types default on; settings
reordered with _section_* group headers (Core / Release Types /
Content Filters)
Repair settings UI:
- Generic _section_<name> key convention renders as an uppercase group
divider in the settings panel — any job can opt in
- .repair-setting-row gets a dashed bottom border so label↔toggle pairing
is visually clear
- _prettifyRepairSettingKey fixes acronym capitalization (EPs, not Eps)
Version bumped to 2.36 with changelog entries.
Two bugs kept this job from finding anything useful on a typical library.
1. Wrong Deezer column name. The artists table has a deezer_id column
(per music_database.py:1986), but the job looked for deezer_artist_id
in both _scan_artist (line 132) and _get_library_artists (line 345).
For Deezer-primary users, this meant the Deezer ID never made it into
the source_ids map, so get_artist_discography fell back to artist-
name-only search — slower and less accurate than an ID lookup.
2. Spotify-reported EPs were silently excluded. Spotify lumps EPs and
true singles under album_type='single'. The previous
_should_include_release short-circuited on album_type='single' and
returned the include_singles setting (default False), so 4-6 track
EPs on Spotify-primary libraries never survived the filter — even
though include_eps defaulted to True. Only 7+ track full albums
made it through. This is the main reason users felt the job did
nothing.
Fixes:
- Use the correct deezer_id column name in both reference sites.
- Restructure _should_include_release so only 'album', 'ep', and
'compilation' are trusted outright. Anything else (including
'single' and missing type) falls through to a track-count
disambiguation matching the download pipeline's _get_album_type_display:
1-3 tracks = true single, 4-6 = EP, 7+ = album. A Spotify-returned
'single' with 5 tracks now correctly counts as an EP.
Full suite stays at 263 passed. Ruff clean.
PR #340 added ruff to the build-and-test.yml CI gate, which surfaced
286 pre-existing lint errors. Left unfixed, every feature branch push
fails CI. This commit resolves all of them so CI goes green and
contributors can actually land work.
Auto-fixes (248 of 286): removed unused f-string prefixes (F541),
renamed unused loop control variables with underscore prefix (B007),
removed duplicate imports (F811).
Manually fixed 10 latent bugs ruff caught (all wrapped in try/except
today, silently failing):
- music_database.py: _add_discovery_tables() called undefined
conn.commit() — would have crashed the iTunes-support migration
for existing databases. Now uses cursor.connection.commit().
- web_server.py settings GET: referenced undefined download_orchestrator
when it should be soulseek_client. Feature (_source_status on the
settings payload) was silently missing for UI auto-disable logic.
- web_server.py _process_wishlist_automatically: active_server
undefined in track-ownership check. Auto-wishlist was falling
through to the error handler and re-downloading owned tracks.
- web_server.py start_wishlist_missing_downloads: same active_server
bug in the manual wishlist path.
- web_server.py _process_failed_tracks_to_wishlist_exact: emitted
wishlist_item_added automation event with undefined artist_name
and track. Automation event silently never fired correctly.
- web_server.py discovery metadata enrichment: referenced cache
without calling get_metadata_cache() first. Track enrichment from
cached API responses was silently skipped.
- web_server.py Beatport discovery worker: wing-it fallback branch
used undefined successful_discoveries variable. Wing-it counter
never incremented correctly. Now uses state['spotify_matches']
consistently with the rest of the function.
- web_server.py _run_full_missing_tracks_process: stale import json
mid-function shadowed the module-level import, making an earlier
json.dumps() call reference an unbound local (F823).
- web_server.py discovery loop: platform loop variable shadowed
the module-level platform import (F402).
- core/watchlist_scanner.py: 7 lambda captures of loop variables
(B023 classic Python closure-in-loop bug) now bind at creation.
No existing tests had to change. Full suite stays at 263 passed.
- Move /api/artist/<artist_id>/image resolution into core.metadata_service.
- Resolve artist artwork through source priority, with explicit source/plugin overrides preserved.
- Keep Spotify call tracking inside the client layer to avoid double counting.
- Update similar-artist lazy loading to pass source context and add service coverage.
- Relocate the streamed MusicMap similar-artist flow out of web_server.py and into core.metadata_service.
- Match similar artists through the configured source-priority chain instead of assuming Spotify first.
- Add iTunes artwork fallback so streamed artist payloads still carry image_url when search results are sparse.
- Cover the new service behavior with tests.
Artist detail pages ran check_album_exists_with_editions and check_track_exists
per discography item, each firing 5+ title variations times 3 artist variations
of fuzzy LIKE searches plus fallback broad-artist queries. For a 30-album artist
that was ~450 SQL round-trips just to answer "which of these do I own."
Hoist the artist's library albums and tracks into memory once per request via
two new helpers — get_candidate_albums_for_artist and get_candidate_tracks_for_albums —
and thread them through as optional candidate_albums / candidate_tracks params on
check_album_exists_with_editions, check_album_exists_with_completeness,
check_track_exists, check_album_completion, and check_single_completion.
Batched path scores the same _calculate_album_confidence / _calculate_track_confidence
against the in-memory list, preserving Smart Edition Matching and accuracy.
Title-only cross-artist fallback still fires for collaborative-album edge cases.
None on either param preserves legacy per-item SQL behavior for unaffected callers.
Applied to both /api/library/completion-stream (library artist detail page) and
iter_artist_discography_completion_events (Artists search page). Timing logs
added to confirm the pre-fetch cost and loop elapsed time.
On a Kendrick page load, per-album resolution drops from ~8 seconds to under
the 50ms streaming sleep floor. Observed ~100x SQL reduction on the happy path.
- collapse old multi-line debug bursts into single structured rows
- remove leftover DEBUG-style prefixes from message text
- keep the app log readable without losing useful trace detail
If the application was using a non-standard location for app.log, the other logs would still go to the default location. Now everything goes under the same, configured folder
When staging files are organized as Artist/Albums/AlbumFolder or
Artist/AlbumFolder, the auto-import now uses the parent folder name
as the artist instead of trusting embedded file tags.
Uses relative path from staging root to determine folder depth, so
albums directly in staging root don't accidentally pick up container
paths as artist names. Common category subfolder names (Albums,
Singles, EPs, Mixtapes, etc.) are recognized and skipped.
Fixes mixtapes and compilations where file tags have DJ names or
incorrect artists (e.g. files tagged as "Slim" in a 2Pac folder).
New repair job that scans each artist in the library, fetches their
full discography from metadata sources, and creates findings for any
tracks not already owned. Users review findings and click "Add to
Wishlist" to queue missing tracks for download.
Respects content filters (live/remix/acoustic/instrumental/compilation)
and release type filters (album/EP/single). Opt-in, disabled by default,
runs weekly, processes up to 50 artists per run with rate limiting.
Jobs with interval_hours set to 0 caused ZeroDivisionError in
_pick_next_job staleness calculation. Now skips jobs with invalid
(zero or negative) intervals.
The Subsonic getArtist endpoint doesn't support musicFolderId filtering,
so when an artist exists in multiple libraries, all their albums were
imported regardless of which music folder was selected in settings.
Now passes musicFolderId to getArtist (in case Navidrome supports it),
and as a fallback filters albums against a cached set of album IDs
built from getAlbumList2 (which reliably supports musicFolderId).
The set is built once per session and invalidated on folder change.
The high-confidence fingerprint skip (≥0.95) assumed title mismatches
were language/script differences and bypassed verification. But a high
fingerprint score just means AcoustID identified the audio confidently —
not that it matches the requested track. Now requires partial title
(≥0.55) or artist (≥0.60) similarity before skipping, so completely
wrong files (e.g. different song/artist from same remix producer) are
correctly rejected.
- Move album-track resolution into metadata_service
- Use the configured provider order instead of Spotify-first branching
- Switch the frontend to the unified /api/album/<id>/tracks endpoint
- Add tests for source-priority lookup, DB resolution, and formatting
iTunes API can return collection metadata without song tracks for
region-restricted albums. The _lookup fallback only checked if results
was empty, so a collection-only response was accepted and cached as
{'items': []}. All future lookups returned the cached empty result.
Three fixes:
- get_album_tracks now checks for actual song items and tries fallback
storefronts when only collection metadata is returned
- Skip cached results with empty items array (prevents stale cache hits)
- Backend returns descriptive 404 error, frontend surfaces it in toast
Fixed 5 critical gaps in the download orchestrator where lidarr was
missing from client loops: get_all_downloads, get_download_status,
cancel_download fallback, clear_all_completed_downloads, and
cancel_all_downloads. Without these, lidarr downloads were invisible
to the UI, couldn't be cancelled, and accumulated in memory.
Also: error messages now visible in download list (appended to
filename on error state), removed "(Development)" label from UI.
- pass provider-specific artist ids into the source-priority discography lookup
- stop relying on the local library artist id when querying external metadata
- add a regression test for source-specific artist id resolution
- Stop passing in spotify_id as the id in the UI, use the actual db id instead
- Fixes an issue where albums for another artist would end up being returned for the actual searched artist
- Remove the redundant artist_id filtering code
- Fixes an issue where not-currently-owned albums would be filtered out from the results, even if they were successfully fetched from the configured metadata provider
The retag fix for AcoustID mismatches was only updating the DB
record (title, artist_id) without writing corrected tags to the
actual audio file. Users would click Fix, the finding disappeared,
but the file on disk stayed unchanged. Now writes title and artist
tags to the file via Mutagen after the DB update.
Also fixed artist INSERT missing server_source when creating a new
artist during retag — now uses the active media server value.
The listening stats worker ran three N+1 query patterns on every
30-minute poll cycle:
1. _resolve_db_track_id was called once per history event (up to
500 events = 500 SELECTs).
2. _map_play_counts_to_db ran one SELECT per server track ID.
3. _enrich_stats_items ran one SELECT per top_artist, top_album,
and top_track (typically 60 extra queries per rebuild).
All three paths now use batched IN queries with 500-row chunks
(well under SQLite's default variable limit of 999). Case-insensitive
matching and LIMIT 1 semantics are preserved via setdefault() on the
Python-side result dict.
Track resolution uses SQLite row-value IN ((?,?), ...) on
(LOWER(title), LOWER(artist_name)), available in SQLite 3.15+
(bundled with Python 3.13).
Four enrichment workers (Last.fm, MusicBrainz, Tidal, Qobuz) had a
bug where every background loop re-processed the same rows because
the existing-ID short-circuit path never set match_status, and two
workers queried the wrong column when checking for an existing ID.
lastfm_worker._get_existing_id queried a non-existent lastfm_id
column; the real column is lastfm_url. The method now reads
lastfm_url for all three entity types.
musicbrainz_worker._get_existing_id queried musicbrainz_id for all
entity types, but albums use musicbrainz_release_id and tracks use
musicbrainz_recording_id. The method now uses a per-type column map.
All four workers (lastfm, musicbrainz, tidal, qobuz) now write
match_status='matched' when they short-circuit on an already-present
external ID, so these rows are no longer re-selected on the next
worker sweep.
A new migration (_backfill_match_status_for_existing_ids) runs once
on startup to retroactively set match_status='matched' for rows that
already have an external ID but NULL match_status. This covers legacy
data, manual matches, and rows populated from file tags outside the
worker.
MetadataCache.get_search_results previously looped over each cached
entity ID and issued one SELECT per ID, producing N extra queries per
cached search hit. It now resolves all entities in a single batched
IN query (chunked at 500 to stay under the SQLite variable limit),
then reconstructs the result list in the original result_ids order
using an in-memory dict lookup.
- broaden the artist-detail dedup helper to catch trailing parenthetical edition and remaster variants
- keep the legacy hyphenated suffix fallback for older metadata
- add regression coverage for language-specific Edition and remaster cases
- move artist-detail discography resolution onto the shared source-priority metadata service
- keep the variant dedup helper in the UI-facing adapter
- pass the chosen source through completion checks
- add coverage for the new adapter and dedup behavior
Move completion checks into metadata_service and make them follow the configured metadata source priority.
Drop the old test-mode path, remove the web_server wrapper indirection, and keep artist inference on explicit release metadata instead of guessing from a track search.
Add coverage for the source-priority completion behavior and the safer artist-name handling.
When no cached token exists, spotipy's auth probe starts an interactive
OAuth flow that binds 127.0.0.1:<redirect_port> inside the container.
This either steals Flask's port 8008 (crash loop) or binds loopback-only
on 8888 (unreachable from Docker host — 'connection reset by peer').
Now checks for a cached token before probing. If none exists, returns
False immediately so users authenticate via the SoulSync web UI instead.
No behavior change for already-authenticated users.
Fixes#269
New core/genre_filter.py with ~180 curated default genres. When strict
mode is enabled in Settings → Library Preferences → Genre Whitelist,
only whitelisted genres pass through during enrichment. Junk tags from
Last.fm (artist names, radio shows, playlist names) are silently dropped.
Applied at all 10 genre write points: Spotify, Last.fm, AudioDB, Deezer,
Discogs, iTunes, Qobuz enrichment workers + post-processing genre merge
+ initial download artist/album creation.
Strict mode is OFF by default — zero behavior change for existing users.
First enable auto-populates the whitelist with defaults. Users can add,
remove, search, and reset genres via the Settings UI.
The Duplicate Detector repair job had its own ignore_cross_album setting
that was independent of the global allow_duplicate_tracks setting. When
a user enabled 'Allow duplicate tracks across albums', the detector
still flagged same-titled tracks on different albums as duplicates.
Now respects the global setting — if duplicates are allowed, cross-album
matches are always skipped.
Users can now override which metadata provider (Spotify, Deezer, Apple Music,
Discogs) is used when scanning a specific watchlist artist for new releases.
The selector appears in the artist config modal and only shows sources the
artist has enrichment IDs for. Default behavior is unchanged — all artists
use the global metadata source unless explicitly overridden.
The redownload branch had `import json, uuid` locally inside the function,
which caused Python to treat `uuid` as a local variable for the entire
function scope. When the retag branch ran instead, `uuid` was unbound.
Both modules are already imported at the top of the file.
Move Hydrabase availability checks into metadata_service so source resolution owns the policy. Keep web_server delegating to the centralized helper and add tests for the enabled/disabled cases.
Move artist discography resolution into core metadata_service, introduce MetadataLookupOptions, and keep web_server focused on request handling. Add focused tests for the new service boundary and preserve current fallback behavior for now.
New MusicBrainz tab in Enhanced and Global search — finds tracks and
albums on MusicBrainz's community database with Cover Art Archive
images. Covers obscure tracks that Spotify/Deezer/iTunes miss.
- core/musicbrainz_search.py: search adapter with Track/Artist/Album
dataclasses, Cover Art Archive integration, smart query parsing
- Albums deduplicated (keeps best version with date and art)
- No artist results shown (MusicBrainz has no artist images)
- Album detail with full tracklist for download modal
- Smart word-boundary splitting for queries without separators
- Global search results container widened from 620px to 920px
- UI version bumped to 2.32
Files with embedded tags (artist+title from post-processing) were
failing import because the metadata search scored low (66%) and the
AcoustID result returned before the tag-preference code could run.
- Tag-based identification now returns 85% confidence when embedded
tags have an artist field, borrowing album art from weak metadata
- AcoustID search result only accepted at 80%+ confidence, otherwise
kept as fallback (doesn't short-circuit past tag preference)
- AcoustID None artist/title falls back to tag data via 'or' operator
- Stop retrying failed/unidentified items every scan cycle
Items with status needs_identification, failed, or rejected were not
in the skip list, causing them to be re-scanned and re-logged every
60 seconds indefinitely. Now skips all terminal statuses.
New 'soulsync' media server option manages the library directly from
the filesystem, bypassing Plex/Jellyfin/Navidrome entirely.
Two paths populate the library:
1. Downloads/imports write artist/album/track to DB immediately at
post-processing completion, with pre-populated enrichment IDs
(Spotify, Deezer, MusicBrainz) so workers skip re-discovery
2. soulsync_client.py scans Transfer folder for incremental/deep scan
via DatabaseUpdateWorker (same interface as server clients)
New files:
- core/soulsync_client.py: filesystem scanner implementing the same
interface as Plex/Jellyfin/Navidrome clients. Recursive folder scan,
Mutagen tag reading, artist/album/track grouping, hash-based stable
IDs, incremental scan by modification time.
Modified:
- web_server.py: _record_soulsync_library_entry() at post-processing
completion, client init, scan endpoint integration, status endpoint,
web_scan_manager media_clients dict, test-connection cache updates
- config/settings.py: accept 'soulsync' in set_active_media_server,
get_active_media_server_config, is_configured, validate_config
- core/web_scan_manager.py: add soulsync to server_client_map
Dedup: checks existing artist/album by name across ALL server sources
before inserting to avoid duplicates. Enrichment IDs only written when
the column is empty (won't overwrite existing data).
Race condition: scanner re-scanned folders while post-processing was
still moving files, causing partial matches and ghost failures. Now
tracks in-progress paths and skips them on subsequent scans.
Coverage penalty fix: individual tracks that match at 80%+ confidence
now auto-import even when overall album coverage is low (e.g. 2 of 18
tracks present). Previously low coverage killed the entire import.
Import page: stats bar, filter pills, Scan Now, Approve All, Clear
History (clears imported + failed), live scan progress.
- Track numbers defaulted to 1 instead of using metadata source values
- Release dates not captured, causing missing year in path templates
- Cover art missing for Deezer (direct image_url not checked)
- Track names in expanded view showed Unknown (wrong JSON field name)
- Read year/date from embedded file tags as fallback
- Add Deezer get_album_metadata/get_album_tracks fallbacks
- Handle Deezer tracks.data response format
Loose audio files in the staging root are now picked up alongside album
folders. Singles are identified via embedded tags, filename parsing
(Artist - Title.ext), or AcoustID fingerprinting, then matched against
the configured metadata source. Confidence-gated processing applies
the same way as album folders (90%+ auto, 70-90% review, <70% manual).
Soulseek results from "Various Artists", "VA", "Unknown Artist", and
"Unknown Album" folders are now rejected before scoring. These
compilation folders rarely contain properly tagged files for the target
artist.
Clearing the wishlist now also cancels any active wishlist download
batch and resets the auto-processing flag, so downloads don't keep
running after the source tracks are removed.
Priority 0 query (artist + album + title) was gated behind a download
mode check that excluded Soulseek, the source that benefits most from
it. Soulseek searches match against file paths where users organize as
Artist/Album/Track — without the album name, ambiguous artist names
could match wrong-artist results (e.g. "Bleakness" as an album folder
instead of an artist). Removed the mode gate so all sources get the
most specific query first.
Repair-worker album fills now generate explicit track IDs when copying rows, instead of relying on SQLite auto-assignment that no longer exists for TEXT primary keys. The unknown-artist fixer now does the same for new artists.
Also add a regression test for the album-fill copy branch and keep the AcoustID scanner resilient to legacy null-ID rows.
Full auto-import pipeline: background worker watches the staging folder,
identifies music using embedded tags → folder name parsing → AcoustID
fingerprinting, matches files to metadata source tracklists, and
processes high-confidence matches through the existing post-processing
pipeline automatically.
Worker: AutoImportWorker with start/stop/pause/resume, configurable
scan interval (default 60s), confidence threshold (default 90%), and
auto-process toggle. Processes one folder per cycle, alphabetical
order. Disc folder detection, stability checking, content hash dedup.
Confidence gate: 90%+ auto-processes silently, 70-90% queued as
pending review with approve/dismiss actions, <70% flagged for manual
identification. Track matching uses weighted algorithm (title 45%,
artist 15%, track number 30%, album tag 10%).
Database: auto_import_history table tracks every scan result with
folder hash, match data JSON, confidence, status, timestamps.
API: 7 endpoints — status, toggle, settings (GET/POST), results
(filtered/paginated), approve, reject.
UI: Auto tab on Import page with enable toggle, confidence slider,
scan interval selector. Live result cards with album art, confidence
bar (green/yellow/red), status badges, match stats. 5-second polling.
Switch similar-artist backfill to the shared provider-priority flow instead of assuming iTunes as the fallback.
Reuse the generic metadata search helpers, keep a compatibility alias for the old helper name, and update the scanner tests to cover the new path.
Add a regression test that verifies backfill walks each available fallback provider and persists the resolved IDs per source.
Shift similar-artist lookup to the shared metadata provider priority flow.
Use generic provider clients for search and metadata extraction instead of
branching on Spotify/iTunes-specific paths.
Add a regression test that verifies MusicMap matching queries the provider
priority list and preserves canonical metadata from the best match.
Make discovery pool population and curated playlists follow the configured metadata source order. Keep Spotify strict where fallback would corrupt source-specific IDs, and trim fan-out with smaller similar-artist samples and page caps. Leave the remaining incremental path for follow-up.
Reduce request volume in the discovery helpers while keeping the source-priority model intact.
- make cache_discovery_recent_albums source-priority aware
- cap Spotify artist-album pagination in the discovery and incremental paths
- reduce the similar-artist sample size for the cache-refresh helper
- keep Spotify strict where fallback would contaminate source-specific IDs
- add regression coverage for source order, strict Spotify lookups, and pagination caps
Watchlist scanner: empty discography (no new releases in lookback) was
treated as API failure, causing "Failed to get artist discography" for
artists like Kendrick Lamar who simply had no recent releases. Now
distinguishes None (API failure → try next source) from [] (success,
no new tracks). Spotify backfill now uses the authenticated client
instance instead of creating a fresh unauthenticated one.
Wishlist nebula: album remove now sends album_name (API updated to
accept album_name as fallback alongside album_id). Track remove
re-renders the nebula after deletion. Toned down processing pulse
animation.
Updated test to verify fallback triggers on API failure (None), not
on empty results.
Replaced track-count-only release selection with deterministic scoring
across 6 factors: track count match (40pts), release status (10pts),
country preference with US/worldwide bias (10pts), format preference
favoring Digital/CD over Vinyl/Cassette (10pts), barcode presence (3pts),
and date completeness (2pts). Same inputs always produce the same release.
Also fixed critical bug: _embed_source_ids was missing the context
parameter, silently skipping ALL source ID tag embedding since the
MusicBrainz consistency commit. Now passes context from the caller.
Make discovery pool population respect provider priority while keeping Spotify strict, and reduce unnecessary request volume in the hot discovery paths.
- keep discovery fan-out source-priority aware
- preserve cache use where freshness is not required
- cap Spotify artist-album pagination in discovery and cache refresh paths
- keep incremental release checks to a single page, since they only need the newest releases
- add regression coverage for provider order, strict Spotify handling, and pagination caps
Rewrote the AcoustID scanner job to scan all library tracks (via DB file
paths resolved to disk) instead of only the Transfer folder. Checkpoints
by track ID for robust resume across restarts. Defaults changed to
enabled, 24h interval, batch size 200.
Added _fix_acoustid_mismatch handler with three actions:
- retag: update DB title/artist to match actual audio content
- redownload: add expected track to wishlist and delete wrong file
- delete: remove wrong file and DB record
This catches cases like a file tagged as "Dinosaur Bones" that is
actually "Helicopters" — the scanner fingerprints the audio, detects
the mismatch, and the user can fix it from Library Maintenance findings.
Resolve Spotify artist matching through the exact Spotify client only, so watchlist ID backfill cannot drift to fallback-provider results. Remove the remaining preemptive provider availability check from the backfill loop.
Allow cached Spotify search results to return even when Spotify is rate-limited or temporarily unavailable, and remove redundant rate-limit gating after auth checks.
Drop the old active-provider artist lookup helpers from watchlist_scanner now that the web scan flow resolves sources through the shared metadata priority.
Keep the Spotify-specific feature toggles in place for discovery and sync paths that still use them.
Move the web watchlist scan core onto the shared metadata source priority so primary provider settings are respected during artist, album, and image resolution.
Add coverage for primary-source-first discography lookup and fallback to later providers when the primary source has no albums.
Bring placeholder tracklist skipping back into the shared watchlist scan path, and centralize the DB-only artist image backfill helper so both web scan entrypoints reuse the same logic.
Drop the legacy watchlist scan entrypoints that are no longer used by the web scan flow, and keep the live refresh path pointed at the shared scanner helper.
Move the shared watchlist scan loop into core/watchlist_scanner.py so web_server.py only handles triggers, locks, progress, and post-scan orchestration.
Manual and scheduled watchlist scans now share the same scanner-side core, while the web entrypoints keep profile selection and automation progress updates.
Two-layer detection: (1) check the Qobuz API response for sample=True
before downloading, and (2) validate actual file duration with mutagen
after download — if under 35 seconds, delete and return None. Qobuz
returns valid audio files for previews (~2-5MB FLAC) that pass the
existing 100KB size check, so duration is the reliable signal.
Artists with an existing spotify_artist_id but NULL spotify_match_status
were fetched by the priority queue every ~3 seconds. _process_artist
returned early (preserving the ID) without marking the status, so the
same artist was re-queued indefinitely — burning CPU and inflating API
call counters. Now marks the artist as 'matched' on the early-return
path.
Respect the configured metadata source order when looking up album years, and re-check provider availability during the scan so Spotify can drop out cleanly if it becomes rate-limited.
Cover art lookup now honors an explicit prefer_source first,
falls back to the runtime primary metadata source when unset,
and uses the shared source priority for the remaining fallbacks.
Use the shared metadata source priority when resolving album IDs,
album searches, and tracklists in track number repair.
Keeps Deezer and iTunes ahead of Spotify where configured, while
still allowing the job to fall back through other supported sources.
Unknown artist resolution now uses the shared metadata source priority and only filters to the sources that can actually participate in this job. Deezer and iTunes remain direct lookup sources, while Hydrabase can now join the title-search path when it is the configured priority source.
- Pass playlist image_url to _run_sync_task from all source-specific sync
start handlers (Deezer, Tidal, Spotify public, YouTube, automation mirror)
— previously only the /api/sync/start endpoint passed it
- Fix plex_client.set_playlist_image: use uploadPoster(url=) instead of
uploadPoster(data=) which is not a valid PlexAPI argument
- deezer_client: use picture_xl > picture_big > picture_medium fallback
for better cover art resolution
- tidal_client: extract image_url in get_playlist() from JSON:API
relationships (was only extracted in metadata-only listing)
- parse_youtube_playlist: capture playlist thumbnail from yt-dlp result
- Add visible logging for image upload attempts and outcomes
Spotify was being called for album/artist data fetching across multiple
background workers and the Artists page search even when the user had
Deezer or iTunes set as their primary metadata source. Being authenticated
for playlist sync was treated as permission to use Spotify for everything.
- watchlist_scanner: add _spotify_is_primary_source() that checks both
auth and primary source config; use it for all album/artist data fetching
(discovery pool, recent album caching, playlist curation, similar artist
ID matching, proactive ID backfill). _spotify_available_for_run() is kept
for sync_spotify_library_cache which must run regardless of primary source
- repair_jobs/metadata_gap_filler: gate Spotify ISRC lookup on primary
source being 'spotify'; MusicBrainz lookup unaffected
- repair_jobs/unknown_artist_fixer: replace hardcoded spotify_client with
source-aware client selection — primary source ID tried first, each ID
matched to its correct client (fixes latent bug passing Deezer IDs to
Spotify)
- web_server.py /api/match/search: Artists page search was hardcoded to
spotify_client.search_artists(); now uses _get_metadata_fallback_client()
so results come from the configured primary source
Adds a new Last.fm Radio section to the Discover page that lets users
search a track on Last.fm, generate a similar-tracks playlist, and run
it through the existing discovery/download/sync pipeline. Also generates
playlists automatically from top listening history during watchlist scans
(max once per week).
- core/lastfm_client.py: Add get_similar_tracks() using track.getsimilar
- core/listenbrainz_manager.py: Add save_lastfm_radio_playlist() with
deterministic MBID (MD5 seed), cleanup limit of 5 for lastfm_radio type
- web_server.py: Add /api/lastfm/configured, /api/lastfm/search/tracks,
/api/lastfm/radio/generate, /api/discover/listenbrainz/lastfm-radio;
fix playlist['name'] KeyError in discovery worker that was resetting
phase back to 'fresh' after completion
- core/watchlist_scanner.py: Add _generate_lastfm_radio_playlists() with
weekly throttle, called at end of scan_all_watchlist_artists()
- webui/index.html: Add #lastfm-radio-section above ListenBrainz section,
hidden unless Last.fm API key is configured
- webui/static/script.js: Search/generation/card-load functions; fix
discovery modal labels (Last.fm Radio vs ListenBrainz), description
update on completion, belt-and-suspenders completion handling inside
updateYouTubeDiscoveryModal; fix album/duration display for tracks
without metadata; music note SVG placeholder for missing art
- webui/static/style.css: Styles for search bar, dropdown, result rows
Centralize the ordered metadata source list and source-priority helper so album completeness and the repair worker follow the same Deezer/iTunes-first fallback order. This also removes the last duplicate priority logic from the touched repair paths.
Album completeness and any other repair job now uses the centralized source/client helpers instead of a worker-local Spotify client or override plumbing
- This keeps source selection aligned with the configured primary provider and removes the last Spotify-only special case from the job path.
This change ultimately is a step towards further centralizing the Spotify client access and the associated `is_spotify_authenticated` check.
- Currently these look-ups are done all over the place in different feature implementations directly, but moving forward, any feature that uses `get_primary_client` or `get_client_for_source` to access the Spotify client, won't have to duplicate any rate-limiting or auth checks as long as these getters are used
New core/replaygain.py module uses FFmpeg's ebur128 filter (already a
project dependency) to analyze integrated loudness and true peak, then
writes ReplayGain 2.0 tags (-18 LUFS reference) to MP3 (TXXX frames),
FLAC/OGG/Opus (Vorbis comments), and M4A/MP4 (freeform atoms).
Three analysis modes in the enhanced library view:
- Per-track RG button: synchronous single-track analysis (~1-3 s)
- Album "ReplayGain" button: background job writing both track gain
and album gain (mean LUFS across all album tracks) to every file
- Bulk bar "ReplayGain" button: batch track-gain for selected tracks
read_file_tags() in tag_writer.py extended with four new optional keys
(replaygain_track_gain/_peak, replaygain_album_gain/_peak) so existing
RG values surface in the tag-preview diff view. Purely additive — no
existing endpoints or DB schema changed.
Hardcoded ports 8888/8889 conflict when SoulSync runs behind Gluetun or
other containers that claim those ports. Introduce SOULSYNC_SPOTIFY_CALLBACK_PORT
and SOULSYNC_TIDAL_CALLBACK_PORT env vars (defaulting to 8888/8889) so
users can remap without rebuilding the image.
docker-compose.yml exposes the vars with comments explaining how to keep
the port mappings in sync with the redirect URI in Settings → Connections.
Track per-quality-tier failure reasons across all failure paths (stream
error, empty manifest, download exception, stub file, MP4 extraction
failure) and include them in the exhausted-tiers log message so failures
are diagnosable from logs.
When HiRes is configured with no fallback and all tiers are exhausted,
log an actionable hint directing the user to enable Quality Fallback.
Surface Tidal-specific error messages in the UI task on retry
exhaustion: distinguishes HiRes-unavailable (with actionable guidance)
from general Tidal auth/quality failures, rather than showing the
generic Soulseek error string.
Album completeness and downstream repair flow now follow the configured
primary provider first, with Discogs and Hydrabase support added alongside
existing Spotify, iTunes, and Deezer paths.
Keep spotify_track_id for compatibility while preserving source-aware track
IDs for provider-neutral handling.
Builds a new Your Albums section on the Discover page that aggregates
saved/liked albums from all connected services, mirroring the Your Artists
pattern. Deezer works via both OAuth and ARL.
- tidal_client: add get_favorite_albums() with V2/V1 API fallback
- deezer_client: add get_user_favorite_albums() via OAuth (user/me/albums)
- deezer_download_client: add get_user_favorite_albums() via ARL session
- music_database: add liked_albums_pool table (deduped by artist::album
normalized key), upsert_liked_album, get_liked_albums,
get_liked_albums_last_fetch, clear_liked_albums
- web_server: GET /api/discover/your-albums (ownership-checked, paginated),
GET /api/discover/your-albums/sources, POST /api/discover/your-albums/refresh,
_fetch_liked_albums background worker (Spotify + Tidal + Deezer OAuth/ARL)
- frontend: Your Albums section with source selector cog, album grid reusing
spotify-library-card styles, search/filter/sort/pagination, download missing
button, auto-refresh poll on first load
Also fix: Deezer greyed out in Your Artists sources when using ARL — connection
check now accepts ARL auth (deezer_dl.is_authenticated()) in addition to OAuth,
and _fetch_and_match_liked_artists falls back to ARL client for artist fetching.
Add get_user_favorite_artists(limit=200) to DeezerDownloadClient to fetch a user's favorite artists via the public API using an ARL-authenticated session (paginated, error-handled, returns deezer_id, name, image_url).
Update web_server to treat Deezer as connected if either OAuth or ARL is authenticated, and to fetch favorite artists from OAuth client when available or from soulseek_client.deezer_dl (ARL) otherwise. Fetched artists are upserted into the database and appropriate log/console messages and counters are updated.
get_playlist_tracks() had no Limit or StartIndex params, so Jellyfin
defaulted to ~100 items per response. This caused the Server Playlists
comparison to show most tracks as missing even though they were present.
Now paginates in batches of 1000 until a partial batch signals the last page.
Spotify album art: replace the 4-char size segment after '0000ab67616d'
with '82c1' to request the original uploaded master (up to 2000px+).
Applied via _upgrade_spotify_image_url() in Track, Artist, and Album
dataclass constructors and as a catch-all in _download_cover_art.
Scoped to the ab67616d album art prefix only — artist images use a
different prefix (ab676161) where the trick does not apply.
iTunes/Apple Music: replace '100x100bb' with '3000x3000bb' in all
artworkUrl100 replacements across Track, Artist, Album, and the
get_album images arrays. Also applied as a catch-all in _download_cover_art.
Deezer already uses cover_xl at its maximum — no changes needed there.
The fallback test (used when no audio files exist in the library) sends
a dummy fingerprint to the AcoustID API. The API correctly rejects the
dummy fingerprint but this is not error code 4 (invalid key), so the
test was returning False instead of True. Any non-code-4 error from the
fallback means the API key was accepted — only code 4 means a bad key.
Deezer and iTunes defaulted to 50 albums max, silently truncating large
discographies. Deezer now paginates (100 per page) up to 200. iTunes
raised to 200 (single call). All callers in web_server.py updated to
use the new defaults instead of hardcoding limit=50.
Also adds diagnostic logging for allow_duplicates album comparison
to help debug inconsistent singles behavior.
Singles like "idol" weren't added when the same song existed on a
different album because check_track_exists used album-aware matching
that found the track via album name. Now skips the album hint when
allow_duplicates is on so matching is title+artist only, then compares
album names ourselves with a strict 0.85 threshold. Only affects users
with allow_duplicates enabled.
The setting only affected wishlist dedup but the watchlist scanner's
library check still skipped tracks by title+artist regardless. Now
when allow_duplicates is enabled, the scanner compares album names
and only skips if the same album matches. Same song on a different
album is allowed through to the wishlist.
Deezer's API returns a contributors array with all credited artists on
a track, but only the primary artist field was used. Now extracts all
contributor names into the artists array for feature/collab tracks.
Only affects the per-track ARTIST tag (TPE1) — album artist, folder
paths, and matching are unchanged. Falls back to single primary artist
when contributors field is absent (search results) or has only one
entry.
Spotify lists unreleased albums with placeholder names like "Track 1",
"Track 2" before the real tracklist is revealed. The scanner was trying
to download these, searching Soulseek for "Track 1" by artist which
matches random files. Now skips any album where more than half the
tracks match the placeholder pattern. Covers both the watchlist scan
and discovery pool paths.
The old subset check treated "Paradise" as matching "Club Paradise"
because {'paradise'} is a subset of {'club', 'paradise'}. Both got
the same +0.10 bonus, so the wrong album could be selected.
Now uses SequenceMatcher for full-string similarity between the wanted
album name and each path segment. Exact matches (>= 0.85) get +0.10,
partial matches (>= 0.60) get +0.03, no match gets +0.00. No penalty
applied — purely adjusts bonus sizing so the correct album ranks higher.
Plex API can return Tag objects mixed with playlists — these lack the
playlistType attribute, causing AttributeError. Use getattr with safe
default instead of direct attribute access.
Add 3-tier Unknown Artist guard in post-processing: checks track_info
artists, original search result, then re-fetches from metadata API
before building folder paths or embedding tags. Prevents files from
landing in Unknown Artist folders when the download context has
incomplete artist data.
The junk entity filter checked track name but not artist_name, allowing
tracks like "Woman Like You by Unknown Artist" to be cached. Now rejects
any track or album where artist_name matches the junk names list
(unknown, unknown artist, empty, null, etc). Prevents stale incomplete
data from persisting across retries.
- Stop active DB update work before tearing down executor pools.
- Short-circuit scan completion callbacks during shutdown so in-flight timer ticks don’t queue follow-up work.
- Prevent the download monitor from draining deferred/completed tasks after shutdown starts.
- Make listening stats startup stop-aware so it exits cleanly if teardown begins during warmup.
- If a metadata update is already running, it can now observe should_stop and exit cleanly instead of continuing after SIGTERM.
- Add interruptible stop events to background workers so shutdown
wakes out of long sleeps instead of waiting on fixed delays.
- Stop scan managers, repair worker, executors, and cleanup helpers
deterministically so process exit does not leave background threads
alive.
- Add startup warnings for stale SQLite WAL/SHM sidecars so unclean
shutdowns are easier to spot before init/migration errors cascade.
- Prevent forced kills from leaving SQLite sidecars behind, which
made rollbacks to older branches fail with malformed database
errors.
7-step full-screen wizard: Welcome, Metadata Source, Download Source,
Paths & Media Server, Add Artists, First Download, Done. All settings
save to DB identically to the Settings page. Supports all 6 download
sources with inline config and test buttons. First download goes through
the full matched download pipeline with metadata context.
Fixes:
- Download clients (YouTube/HiFi/Tidal/Qobuz/Deezer) now reload
download_path when settings change instead of caching from init
- watchlist_artists table migrations now include deezer_artist_id and
discogs_artist_id in all 3 table rebuild locations (was being dropped)
- CREATE TABLE for watchlist_artists includes all provider ID columns
- Serverless download sources (YouTube/HiFi/Qobuz) show green status
instead of red disconnected on sidebar and dashboard
- Suppress repeated slskd 401 errors — logs once then silences until
connection recovers
Stripped 4,200+ emoji characters from print(), logger calls across
39 Python files. Logs are now clean text — easier to grep, more
professional, no encoding issues on terminals without Unicode support.
Seasonal config icons preserved for UI display.
Navidrome provides musicBrainzId on tracks — now captured during
database updates so the MusicBrainz enrichment worker can skip
tracks that already have an MBID.
Uses COALESCE on UPDATE to never overwrite existing enrichment data
with NULL (safe for Plex/Jellyfin which don't provide this field).
Inspired by PR #279 — fixed data loss bug in the original where
unconditional UPDATE would erase existing MBIDs.
Tag preview and writer now use track_artist (per-track artist) for the
Artist tag when available, falling back to artist_name (album artist)
when NULL. Album Artist tag always uses artist_name.
Fixes#277 — DJ mix albums no longer overwrite per-track artists
(Technique, Gouryella) with the album artist (Tiësto).
Lidarr integration:
- New core/lidarr_download_client.py with full interface parity
(search, download, status, cancel — same as Qobuz/Tidal/HiFi)
- Registered in download orchestrator with source routing
- Settings: URL + API key on Downloads tab with connection test
- Available as standalone source or in Hybrid mode priority order
- API key encrypted at rest
- All streaming source checks updated to include 'lidarr'
Lidarr downloads full albums via Usenet/torrent — SoulSync imports
only the tracks it needs and discards the rest.
Music video path validation:
- Empty/unconfigured path returns clear error instead of silent failure
- Write permission test before starting download
- Default changed from './MusicVideos' to empty (must be configured)
Clients are for the most part being initialized per-request, which leads to a lot of redundant client initialization, as well as noise on the logs, since each client initialization emits a row on the logs, eg. 'Deezer client initialized'
Click any video card in Music Videos tab to download. Flow:
1. Search primary metadata source for clean artist/title
2. Fall back to YouTube title parsing if no match
3. Download video via yt-dlp (best quality MP4)
4. Save to configured Music Videos folder as Artist/Title-video.mp4
UI shows circular progress ring on the thumbnail during download,
green checkmark on completion, red X on error (clickable to retry).
Cards are non-interactive while downloading.
Backend: /api/music-video/download and /api/music-video/status endpoints
YouTube client: download_music_video() method keeps video format
New "Music Videos" pill tab alongside Spotify/Deezer/iTunes/Discogs
in both enhanced search and global search. Searches YouTube via yt-dlp
and displays results in a video card grid with 16:9 thumbnails, play
overlay, duration badge, channel name, and view count.
- Backend: /api/enhanced-search/source/youtube_videos endpoint with
search_videos() method on YouTubeClient returning YouTubeSearchResult
- Frontend: Video grid layout with responsive cards, YouTube red tab
color, proper section hiding when switching between metadata and
video tabs
- Global search: Full parity with enhanced search video rendering
- No download functionality yet — display only
The finding dedup check only looked for 'pending' and 'resolved' status,
missing 'dismissed'. Dismissed findings were recreated as new entries on
every scan. Now includes 'dismissed' in the dedup check.
Orphan file detector improvements:
- Increased path suffix matching depth from 3 to 4 segments (covers
Genre/Artist/Album/track.flac paths)
- Added filename-based fallback when Mutagen can't read file tags —
parses title from "NN - Title [Quality].ext" pattern and matches
against parent/grandparent folder names as artist
Prevents downloading tracks from completely wrong artists by adding
minimum artist score gates:
- Soulseek: artist_score < 0.25 → reject (catches Belvedere vs Periphery)
- YouTube: artist_score < 0.15 → reject (catches lizzylou06 vs Muse)
Fixes artist substring matching to use word boundaries instead of plain
containment — "muse" no longer matches "museum", "art" no longer matches
"heart". This was causing false positives where wrong artists passed with
artist_score=1.0 due to accidental substring containment.
Improves similarity fallback by comparing against individual path segments
instead of the full filename, so misspelled artist names (Radiohedd vs
Radiohead) still match correctly.
Adjusts YouTube weights from Title 70%/Artist 10% to Title 60%/Artist 20%
to give artist more influence in YouTube matching.
Addresses user reports of unreleased albums being downloaded with garbage
content from wrong artists on Soulseek and YouTube.
All callers of _create_fallback_client() and _get_configured_fallback_source()
now use get_primary_client() and get_primary_source() directly. No more
legacy alias usage anywhere in the codebase.
Seasonal discovery had 3 use_spotify checks using is_authenticated()
(always True) instead of deriving from the configured source. Search API
(tracks, albums, artists) also defaulted to Spotify when authenticated.
All now check configured primary source first via get_primary_source().
All metadata source decisions now flow through get_primary_source() and
get_primary_client() in core/metadata_service.py. Previously 6 different
files reimplemented this logic with inconsistent defaults ('itunes' vs
'deezer') and auth checks, causing bugs when any one was missed.
Changes:
- metadata_service.py: Added canonical get_primary_source/get_primary_client
- web_server.py: _get_metadata_fallback_source() and _get_active_discovery_source()
are now thin wrappers delegating to metadata_service
- seasonal_discovery.py: _get_source() delegates to metadata_service
- personalized_playlists.py: _get_active_source() delegates to metadata_service
- spotify_client.py: Fixed _fallback_source default from 'itunes' to 'deezer'
- watchlist_scanner.py: _get_fallback_metadata_client() delegates to metadata_service
Future changes to source selection only need to update one file.
New repair job that scans the library for tracks filed under "Unknown
Artist" and corrects them. Resolves correct metadata by:
1. Reading embedded file tags (if file has correct artist)
2. Looking up by source track ID (Spotify/Deezer/iTunes)
3. Searching by title as last resort
Dry run mode (default) creates findings for review. Live mode re-tags
the audio file, moves it to the correct folder structure, and updates
the database. Includes fix handler for applying individual findings.
Seasonal discovery, personalized playlists, and playlist explorer all
defaulted to Spotify when authenticated, ignoring the user's configured
primary source. Now they read from config first.
Spotify's related_artists API (no Deezer/iTunes equivalent) is preserved
as a fallback for all users in personalized playlists. Artist discography
endpoint intentionally unchanged — ID-based lookups need the source that
owns the ID.
Deezer's /artist/{id}/albums endpoint returns albums without an artist
field, causing 98% of cached Deezer albums to have empty artist_name.
Now injects the known artist before caching.
Also fixes get_track_details cache validation — was trusting search
result cache (which has isrc but no track_position), returning
track_number=0. Now only trusts cache entries with track_position.
Discovery workers now respect the user's configured primary metadata
source instead of always using Spotify when authenticated. This
completes the intent of commit 3c211ea.
The core fix addresses data loss in the discovery→sync→wishlist→download
pipeline: the Track dataclass strips album metadata to a plain string,
losing album ID, track_number, release_date, and images. Discovery
workers now enrich results via get_track_details() to recover this data.
Deezer's get_track_details() cache validation was incorrectly trusting
search-result cache (which lacks track_position), returning track_number=0.
Also fixes wishlist download processing where albums without IDs couldn't
map to artists, and the fallback read 'artist' (singular) instead of
'artists' (plural), always producing "Unknown Artist".
Includes a one-time migration to purge stale discovery cache entries.
Duplicate finding detail now shows each version as clickable — user
can choose which to keep instead of relying on auto-selection. Added
track_number as tiebreaker in auto-pick (higher track number wins
over 01, catching leftover duplicates from the playlist sync track
number bug). Track number displayed in the detail view for clarity.
New automation action that executes user scripts from a dedicated
scripts/ directory. Available as both a DO action and THEN action.
Scripts are selected from a dropdown populated by /api/scripts.
Security: only scripts in the scripts dir can run, path traversal
blocked, no shell=True, stdout/stderr capped, configurable timeout
(max 300s). Scripts receive SOULSYNC_EVENT, SOULSYNC_AUTOMATION,
and SOULSYNC_SCRIPTS_DIR environment variables.
Includes Dockerfile + docker-compose.yml changes for the scripts
volume mount, and three example scripts (hello_world.sh,
system_info.py, notify_ntfy.sh).
The scanner was creating a finding for every file that couldn't be
identified by AcoustID, flooding the findings list with non-actionable
entries. Users saw the scanner "stuck scanning the same files over
and over" because the no-match findings were dismissed but recreated
on every run. Now only genuine mismatches (AcoustID identifies a
different track) create findings. Errors are counted and shown in
the job log with actual error messages for debugging.
After a successful playlist sync, if the source playlist has cover
art (Spotify, Tidal, Deezer, etc.), the image is downloaded and
uploaded as the playlist poster on the media server. Plex uses
uploadPoster(), Jellyfin/Emby uses POST /Items/{id}/Images/Primary.
Navidrome skipped (no playlist image API). Failure is silent — sync
result unchanged. Automation-triggered syncs and playlists without
images are unaffected.
Sync rehydration: after loading Deezer ARL playlists, checks each
for active syncs via /api/sync/status and re-attaches polling with
live card updates. Download rehydration: rehydrateModal now handles
deezer_arl_ playlist IDs, and openDownloadMissingModal routes cache
misses to the correct ARL endpoint. Fix All now prompts for dead
file action.
Album data caching: get_playlist_tracks now checks the metadata
cache before fetching album release dates from the Deezer API.
Cache hits are instant, misses are fetched and stored for future
use across all playlists. Import fixed from core.metadata_cache
instead of web_server to avoid circular dependency.
Dead file fix now prompts with two options: Re-download (existing
behavior — adds to wishlist + deletes DB entry) or Remove from DB
(just deletes the dead track record without re-downloading). Works
for both single and bulk fix. Solves the issue where dismissing
dead files didn't remove the underlying track record, causing them
to reappear on every scan.
Albums with zero local tracks were flagged as incomplete but the
auto-fill fix failed because there were no existing tracks to
determine the album folder or quality standard from. Now skipped
during scan — they'll be detected once tracks are actually added.
Qobuz added reCAPTCHA to their login endpoint, blocking automated
email/password auth for new users. Token login lets users paste
their X-User-Auth-Token from the browser DevTools after logging in
manually. Added to both Connections and Downloads tabs with
instructions. Existing email/password flow completely unchanged.
Backend validates token via user/get API and saves the session
identically to email/password login.
Albums announced but not yet released have no real audio available,
causing Soulseek to match random tracks with similar names. Both
discography methods (Spotify and generic client) now filter out
albums with release dates in the future. Skipped albums are not
marked as processed — they will be picked up on the first scan
after their release date passes.
The _is_valid_guid method only accepted 32-char hex GUIDs (Jellyfin
format) but Emby uses plain integer IDs like "12345". All matched
tracks were rejected as "invalid/empty IDs" causing playlist creation
to fail with zero tracks. Now accepts both numeric strings (Emby)
and hex GUIDs (Jellyfin).
New "Deezer" tab on sync page shows authenticated user's playlists,
identical to the Spotify tab pattern — same card layout, details
modal with track list, sync button, and download missing tracks
flow. Existing URL import renamed to "Deezer Link" tab (unchanged).
Backend: get_user_playlists() and get_playlist_tracks() on the
download client fetch via public API with ARL session cookies.
Album release dates batch-fetched for $year template variable.
Three new endpoints: arl-status, arl-playlists, arl-playlist/<id>.
Frontend: cards use Spotify-identical HTML structure with live sync
status, progress indicators, and View Progress/Results buttons.
Downloads reuse openDownloadMissingModal with zero modifications.
Track data cached on first open, instant on subsequent clicks.
Streaming matching: add artist gate rejecting candidates with artist
similarity below 0.4, raise threshold to 0.60, block fallback to
Soulseek filename matcher for Tidal/Qobuz/HiFi/Deezer. Fix single-
char artist containment bug where normalize_string strips non-ASCII
(e.g. "B小町" → "b") causing "b" to match any artist containing
that letter. Fixed in both score_track_match and the Soulseek scorer.
YouTube and Soulseek matching behavior unchanged.
Global search: add registerSearchDownload() calls to _gsClickAlbum
and _gsClickTrack so downloads create bubble snapshots on dashboard
and search page, matching the enhanced search standard.
Global search escaping: add _escAttr() helper to handle newlines in
album/artist names that broke inline onclick string literals.
Changed ignore_cross_album default from True to False. Re-downloads of
the same song create separate album entries, so the detector was skipping
them. Users who want to keep compilations/greatest-hits intact can toggle
it back on. Updated help text to explain when to use this setting.
API Call Tracker:
- Save/load 24h minute-bucketed history + events to database/api_call_history.json
- Persists across server restarts via atexit + signal handler hooks
- New record_event() for rate limit bans (called from _set_global_rate_limit)
- New get_debug_summary() for Copy Debug Info — 24h totals, peak cpm with
timestamp, per-endpoint breakdown, and last 20 rate limit events
- Fixed race condition: events iteration now inside lock during save
Spotify Rate Limit Mitigation:
- Enrichment worker: max_pages=5 on get_artist_albums (was unlimited — artist
with 217 albums caused 22 paginated API calls, now capped at 5)
- Enrichment worker: inter_item_sleep raised from 0.5s to 1.5s
Spotify Re-Auth Fix:
- Both OAuth callbacks (port 8008 + 8888) now clear rate limit ban AND
post-ban cooldown after successful re-auth — Spotify usable immediately
instead of stuck on Deezer fallback for 5 minutes
- Auth cache invalidated on both global client and enrichment worker client
New repair job that scans track and album titles for live performances,
commentary, interviews, skits, and spoken word content. Creates findings
for user review — no auto-fix.
Configurable per content type (live, commentary, interviews, spoken word),
with optional album title scanning and tracks/albums scope toggle.
Fix action removes track from DB + deletes file, cleans up empty albums
and directories. Follows existing repair job pattern exactly.
Tidal, Qobuz, HiFi, and Deezer results were blindly taking the first
API result with minimal validation. Now all streaming sources use
score_track_match() — same 60% title / 30% artist / 10% duration
weighting as Soulseek, plus version detection penalties.
- web_server.py get_valid_candidates(): replaced loose title-sim check
with matching engine scoring, version penalty for live/remix/acoustic
- download_orchestrator.py: optional expected_track param enables
scoring in search_and_download_best (backward compatible)
- sync_service.py: passes spotify_track for validation
- Fixed wrong class name (MusicMatchingEngine not MatchingEngine)
YOUR ARTISTS (major feature):
- Aggregates liked/followed artists from Spotify, Tidal, Last.fm, Deezer
- Matches to ALL metadata sources (Spotify, iTunes, Deezer, Discogs)
- DB-first matching: library → watchlist → cache → API search (capped)
- Image backfill from Spotify API for artists missing artwork
- Carousel on Discover page with 20 random matched artists
- View All modal with search, source filters, sort, pagination
- Artist info modal: hero image, matched source badges, genres, bio,
listeners/plays from Last.fm, watchlist toggle, view discography
- Auto-refresh with loading state on first load, polls until ready
- Deduplication by normalized name across all services
DEEZER OAUTH:
- Full OAuth flow: /auth/deezer + /deezer/callback
- Settings UI on Connections tab (App ID, Secret, Redirect URI)
- Token stored encrypted, auto-included in API calls
- get_user_favorite_artists() for liked artists pool
SERVICE CLIENTS:
- Spotify: added user-follow-read scope + get_followed_artists()
- Tidal: get_favorite_artists() with V2/V1 fallback
- Last.fm: get_authenticated_username() + get_user_top_artists()
FAILED MB LOOKUPS MANAGER:
- Manage button on Cache Health modal
- Browse/filter/search all failed MusicBrainz lookups
- Search MusicBrainz directly and manually match entries
- Optimized cache health queries (11 → 4 consolidated)
- Dashboard cache stats now poll every 15s
EXPLORER IMPROVEMENTS:
- Discover button on undiscovered playlist cards
- Status badges: explored/wishlisted/downloaded/ready
- Auto-refresh during discovery via polling
- Redesigned controls: prominent Explore button, icons
BUG FIXES:
- Fix album artist splitting on collab albums (collab mode fed
album-level artists instead of per-track)
- Fix cover.jpg not moving during library reorganize (post-pass sweep)
- Fix cover.jpg missing when album detection takes fallback path
- Fix wishlist auto-processing toast spam (was firing every 2s)
- Fix media player collapsing on short viewports
- Fix watchlist rate limiting (~90% fewer API calls)
- Configurable spotify.min_api_interval setting
- Better Retry-After header extraction
- Encrypt Last.fm and Discogs credentials at rest
- Add $discnum template variable (unpadded disc number)
New feature: Failed MusicBrainz Lookups management modal accessible
from Cache Health. Browse all failed lookups with type filter tabs,
search bar, pagination. Click any entry to search MusicBrainz and
manually match — saves MBID at 100% confidence. Clear individual
entries or bulk clear all.
Backend: 4 new endpoints — failed-mb-lookups list, mb-entry delete,
musicbrainz/search (artist/release/recording), mb-match save.
Performance: Cache health stats consolidated from 11 queries to 4
using CASE expressions. Added partial index on musicbrainz_cache for
failed lookups. Dashboard cache stats now poll every 15s instead of
single fire-and-forget fetch. Failed MB type counts cached on frontend,
only re-fetched after mutations.
Also includes: library reorganize now moves cover.jpg via post-pass
sidecar sweep, and changelog updates.
Addresses all three points from community rate-limiting report:
1. Watchlist scans fetched ALL albums then filtered — 262 albums = 27
API calls per artist. Now determines upfront if full discography is
needed: subsequent scans and time-bounded lookbacks use max_pages=1
(1 API call). Only "full discography" global setting fetches all.
2. MIN_API_INTERVAL (350ms) now configurable via spotify.min_api_interval
setting. Users who get rate-limited frequently can increase the delay.
Floor at 100ms to prevent abuse.
3. Retry-After header extraction improved: added diagnostic logging when
headers exist but lack Retry-After key, plus regex fallback to parse
the value from the error message string.
- Add discogs_artist_id column to watchlist_artists table (migration)
- Add discogs_artist_id to WatchlistArtist dataclass
- Add to get_watchlist_artists optional_columns and constructor
- Add update_watchlist_discogs_id DB method
- Backfill loop includes Discogs when token is configured
- Add _match_to_discogs for cross-provider artist matching
- Backfill maps updated: id_attr, match_fn, update_fn all include discogs
- Add _extract_discogs_fields to metadata cache — handles Discogs field
names (title vs name, images array, Artist - Title format)
- Worker uses _fetch_and_cache_artist/_fetch_and_cache_album helpers
that cache raw data while returning it for enrichment
- All search/lookup methods cache results for repeat queries
- Cache browser: Discogs stat pill, source filter, clear button, badge
- Fixes albums showing as 'Unknown' and artists missing images in cache
- get_album and get_album_tracks now try /masters/{id} first, fall back
to /releases/{id} — artist discography returns master IDs which are
in a different namespace than release IDs
- Fixes wrong album showing in download modal (master ID 3664443 for
GNX was hitting /releases/3664443 which is a different album)
- Add Discogs source override to all 6 artist/album/track endpoints
- Add discogs_id to _resolve_db_album_id lookup
- Remove upfront master detail fetching (was 15+ API calls, 40+ seconds)
- Discography loads from releases list only (~1 second, 2 API calls)
- Track counts populate on-demand via get_album_tracks when clicking album
- New get_album_tracks method: tries /masters first, falls back to /releases,
returns Spotify-compatible format with proper disc/track numbering
- Album type defaults to 'album' for masters without format metadata —
Discogs limitation, singles only detectable from individual release format
- Search results return format as list ['Vinyl', 'LP'] while artist
releases return comma-separated string — handle both
- Fixes "'list' object has no attribute 'lower'" error
- Master releases now fetch /masters/{id} to get actual tracklist,
genres, styles, and images — fixes 0/0 track count display
- Album type re-evaluated with real track count: 1-3 = single,
4-6 = EP, 7+ = album
- Cover art from master detail used when search results have none
- Tested: Kendrick Lamar shows correct track counts, proper types,
and images for all albums
- Fetch artist name first, then compare against each release's primary
artist — skip releases where the artist is listed after Feat./Ft./&
- "Beyoncé Feat. Kendrick Lamar" → skipped (Kendrick is featured)
- "Kendrick Lamar Feat. Rihanna" → kept (Kendrick is primary)
- Fixes artist pages showing unrelated albums from other artists
- Add _normalize_name helper and re import to DiscogsClient
- Prefer master releases over individual pressings to avoid duplicates
(multiple pressings of same album showing separately)
- Individual releases only included if no master exists for that title
- Skip non-main roles (appearances, features, remixes by others)
- Better album type detection from format string: catches LP, Album,
EP, Single, Compilation from comma-separated format field
- Fetch more results (3x limit) to compensate for filtering
- Fix source name mapping so sidebar/dashboard shows 'Discogs' instead
of falling through to 'iTunes'
- Fix album type detection: parse format string from artist releases
endpoint (e.g. "File, FLAC, Single, 320") to correctly identify
singles, EPs, albums, compilations — was defaulting everything to
'single' because track count was 0
- Remove fake track search that returned albums as tracks — Discogs
has no track-level search API, so tracks section is empty (honest)
- Track data available via album tracklists instead
- SpotifyClient: add _discogs lazy-load property, route _fallback to
DiscogsClient when configured (requires token, falls back to iTunes)
- web_server: _get_metadata_fallback_client returns DiscogsClient when
selected and token present
- Enhanced search: Discogs added as source tab with NDJSON streaming,
only available when token configured
- Alternate sources list includes Discogs when token is set
- Frontend: source labels, tab styling, fetch list all include Discogs
- Consistent with iTunes/Deezer pattern — same interfaces, same routing
- New core/discogs_worker.py — background worker enriching artists and
albums with Discogs metadata following AudioDBWorker pattern exactly
- Artist enrichment: discogs_id, bio, members, URLs, image backfill,
genre backfill, summary backfill from bio
- Album enrichment: discogs_id, genres, styles (400+ taxonomy), label,
catalog number, country, community rating, image backfill
- DB migration: discogs columns on artists (id, match_status, bio,
members, urls) and albums (id, match_status, genres, styles, label,
catno, country, rating, rating_count)
- Worker initialization with pause/resume persistence
- Status/pause/resume API endpoints
- Integrated into enrichment status system, rate monitor, auto-pause
during downloads/scans, WebSocket status emission
- New Discogs section on Settings → Connections with personal token input
- Discogs added as fallback metadata source option alongside iTunes/Deezer
- Token saved to discogs.token config key
- Discogs added to API rate monitor gauges (60/min with auth)
- Help text links to discogs.com/settings/developers for token generation
- Full parity with iTunes/Deezer clients — same Track/Artist/Album
dataclasses, same method signatures (search_artists, search_albums,
search_tracks, get_artist, get_album, get_artist_albums)
- 25 req/min unauthenticated, 60 req/min with free personal token
- Rate limited via same decorator pattern with API call tracking
- Unique data: 400+ genre/style taxonomy, label info, catalog numbers,
community ratings, artist bios
- Smart "Artist - Title" parsing for search results
- Release deduplication (Discogs has many pressings of same album)
- Track search via release tracklist extraction
- Tested: artist/album/track search, artist detail with bio, album
detail with full tracklist + genres + styles + label
- Was only backfilling the active provider — artists added via Deezer
never got Spotify/iTunes IDs, and vice versa
- Now backfills iTunes (always), Deezer (always), and Spotify (if
authenticated) at the start of every scan
- Added _match_to_deezer() and update_watchlist_deezer_id() for
Deezer cross-provider matching
- Generalized backfill with provider→attribute/function maps
- New 'webhook' then-action: sends HTTP POST with JSON payload to any
user-configured URL (Gotify, Home Assistant, Slack, n8n, etc.)
- Config: URL, optional custom headers (Key: Value per line with
variable substitution), optional custom message
- Payload includes all event variables as JSON fields
- 15s timeout, errors on 400+ status codes
- Follows exact same pattern as Discord/Pushbullet/Telegram handlers
- Frontend: config fields, config reader, icon, help docs
- Updated changelogs with webhook, M3U fix, orchestrator hardening
- Each of the 6 download clients initializes independently via
_safe_init() — one failing client no longer kills the orchestrator
- All methods guarded against None clients with appropriate fallbacks
- Init failures logged at startup and tracked in _init_failures list
- Copy Debug Info shows "Download Client Failures" section when any
client failed to initialize, or "ALL" if orchestrator itself is dead
- Add rate limiting to all 4 Spotify pagination loops (get_artist_albums,
get_user_playlists, get_playlist_tracks, get_album_tracks) — these
called sp.next() bypassing the rate_limited decorator entirely, causing
unthrottled API calls that triggered 429 bans
- Track pagination calls in API rate monitor (separate endpoint names)
- Increase DELAY_BETWEEN_ARTISTS from 2s to 4s in watchlist scanner
- Abort watchlist scan immediately if Spotify rate limit detected mid-scan
instead of continuing to hammer the API
