Commit graph

83 commits

Author SHA1 Message Date
Broque Thomas
735dd73865 fix(repair): rewire Unknown Artist Fixer deferred imports (#646)
The "Fix Unknown Artists" repair job crashed on every run with:

    ImportError: cannot import name '_build_path_from_template' from
    'core.repair_jobs.library_reorganize'

Commit ca5c9316 ("Rewrite Library Reorganize job to delegate to per-
album planner") moved the private path-builder + quality-string
helpers out of `core.repair_jobs.library_reorganize` and into the
import pipeline. `unknown_artist_fixer.py:163` still imported them
from the old module — its scan() defers the imports to avoid pulling
web_server's Flask boot into the test harness, so the broken target
only surfaces at runtime when the user actually runs the job. The
tool was completely unrunnable.

Re-wired the deferred imports:

    core.repair_jobs.library_reorganize._build_path_from_template
        -> core.imports.paths.get_file_path_from_template_raw
    core.repair_jobs.library_reorganize._get_audio_quality
        -> core.imports.file_ops.get_audio_quality_string

Both replacements have identical signatures + return shapes (verified
by inspecting library_reorganize's pre-refactor implementations vs
the import-pipeline equivalents):

    get_file_path_from_template_raw(template: str, context: dict)
        -> tuple[folder: str, filename_base: str]
    get_audio_quality_string(file_path: str) -> str

No call-site changes needed beyond the import target.

2 new regression tests in `tests/test_unknown_artist_fixer.py`:

    test_deferred_path_imports_resolve — runs the same import
    statements scan() runs, so the NEXT refactor that moves these
    helpers fails CI rather than reaching the user.

    test_deferred_path_helper_shape_matches_fixer_usage — pins the
    `(folder, filename_base)` 2-tuple contract the fixer's unpack
    relies on. Catches return-shape drift even when the import
    target stays valid.

Audited every consumer of `core.repair_jobs.library_reorganize` —
only one stale import (this file). The test suite covers the only
production caller.

5 fixer tests pass (3 existing + 2 new regression guards).
2026-05-19 22:09:15 -07:00
Broque Thomas
9cc09118bf AcoustID scanner: multi-candidate match + duration guard + multi-value retag
Closes #587. Three coordinated fixes per codex's diagnosis. AcoustID
verification gate left intact — these fixes target the upstream
scanner false-positive surface plus a separate retag-path gap.

Bug 1 — scanner used recordings[0] as authoritative

`core/repair_jobs/acoustid_scanner.py:_scan_file` only checked the
top fingerprint match's metadata. AcoustID often returns multiple
recordings per fingerprint (sample collisions, multi-MB-record
cases) and the wrong-credited recording can outrank the right-
credited one. Foxxify case 2 (Nana / Nana): top match credited the
wrong artist while a lower-ranked candidate matched the user's
expected metadata exactly.

Lifted the verifier's all-candidates check to a shared pure helper
`core/matching/acoustid_candidates.py:find_matching_recording`. Both
verifier and scanner can now ask "given these candidates, does ANY
of them match expected (title, artist)?" with the same contract.
Scanner suppresses the finding when any candidate matches.

Bug 2 — no duration check guards against fingerprint hash collisions

Foxxify case 3: 17-minute mashup edit fingerprinted to a 5-minute
late-70s Japanese hiphop track (different songs, fingerprint hash
collision on a sampled section). Scanner had no signal to detect
this and would have recommended retagging the 17-min file as the
5-min track.

`duration_mismatches_strongly` in the same helper module flags drifts
beyond max(60s, 35%). Scanner now skips findings when the candidate's
duration disagrees strongly with the file's expected duration. Loaded
duration via the existing tracks SQL (added `t.duration` to the
SELECT). Returns False when either side is unknown — no behavior
change for older rows without duration data.

Bug 3 — scanner retag bypassed multi-value ARTISTS tag setting

`core/repair_worker.py:_fix_wrong_song` called `write_tags_to_file`
with single-string artist updates. The writer only wrote TPE1
(single string) and never read the user's
`metadata_enhancement.tags.write_multi_artist` config. Multi-value
ARTISTS tags got stripped on every retag, contradicting the
post-download enrichment pipeline's behavior.

Per codex's pick (option B over routing through enhance_file_metadata),
extended `write_tags_to_file` with an optional `artists_list`
parameter. Each format-specific writer respects the config flag the
same way enrichment.py does:
- ID3: TPE1 stays as joined display string + TXXX:Artists multi-value
- Vorbis/Opus/FLAC: `artist` display string + `artists` multi-value key
- MP4: \xa9ART as list when on, single string when off

Scanner retag derives the per-artist list by splitting AcoustID's
credit through the existing `split_artist_credit` helper (same
separators the matching layer already uses).

Backward compatible: callers that don't pass `artists_list` get the
exact same single-string write as before. No regression for the
write_artist_image button or any other tag_writer caller.

15 tests on the candidate helper + duration guard.
13 tests on the tag_writer multi-value path (write/skip/single/
no-list cases for FLAC + the config-gate helper).
4 new scanner regression tests pinning lower-ranked candidate
suppression, no-suppression when no candidate matches, duration
mismatch skip, no-skip when duration matches.

Existing scanner tests updated for the new 11-column SQL select
(added duration column to fake schema + test row tuples).

Full suite: 3097 passed. Ruff clean.
2026-05-14 14:09:38 -07:00
Broque Thomas
7a23d60f28 AcoustID scanner: file-tag fallback for legacy compilation tracks
Follow-up to the prior compilation-album scanner fix. That patch
made the scanner read `tracks.track_artist` (per-track artist
column) via COALESCE so compilation tracks would compare against
the right value. But tracks downloaded BEFORE the `track_artist`
column existed have track_artist=NULL — COALESCE falls back to
album artist (the curator) and the wrong-comparison case returns.

Fix: explicit 3-tier resolution in `_scan_file`:

  1. DB `tracks.track_artist` if populated → trust it. Respects
     manual edits from the enhanced library view (user who curated
     the DB value but didn't re-tag the file gets their edit
     respected, not overridden by stale file tag).

  2. File's ARTIST tag via mutagen if present → use it. Tidal /
     Spotify / Deezer all write the per-track artist into the
     audio file at download time regardless of SoulSync's DB
     schema, so it's ground truth even when the DB column is
     stale or NULL. File is already open for fingerprinting so
     mutagen tag-read is essentially free.

  3. Album artist → final fallback for files without proper ARTIST
     tags AND no DB track_artist. Existing pre-fix behavior.

`_load_db_tracks` SELECT now surfaces `track_artist` (raw, may be
empty/NULL via NULLIF) and `album_artist` separately in addition
to the COALESCE'd `artist` field — so `_scan_file` can tell the
difference between 'DB has a curated value' and 'DB fell back to
album artist'. Without this distinction, the file-tag fallback
would create false positives when DB is curated but file is stale.

5 new tests (11 total in the file) pin:

  - File-tag-trumps-DB resolves the legacy NULL case (DB says
    'Andromedik' (album curator), file says 'Eclypse', AcoustID
    says 'Eclypse' → no flag)
  - Tag-missing falls back to album artist (preserves existing
    genuine-mismatch contract — file without tag + AcoustID
    mismatch still flags)
  - Mutagen exception swallowed (debug log, fall-through)
  - File-tag matches DB → no behavioral change
  - DB curated value trumps stale file tag (false-positive guard
    — user edited DB without re-tagging file shouldn't get flagged)

Two existing test fixtures (`_make_context` callers) updated to
the new 10-column row shape.

SQL behavior verified empirically against real SQLite: NULL and
empty-string both flow through NULLIF → None in Python →
file-tag-fallback path. Modern populated values trump file tag.
2026-05-11 11:39:19 -07:00
Broque Thomas
812db1fbbf AcoustID scanner: prefer track_artist for compilation albums
Discord report (Skowl): downloaded a compilation album ("High Tea
Music: Vol 1") where every track has a different artist (Eclypse,
Andromedik, T & Sugah, Gourski, etc.) and the AcoustID scanner
flagged every single track as Wrong Song. The file tags had the
correct per-track artist (e.g. "Eclypse" for "City Lights"), but
the scanner compared against the album-level artist ("Andromedik",
the curator). Raw similarity 12% → Wrong Song flag.

# Why the prior multi-value fix didn't help

Foxxify's case (just-merged PR): AcoustID returned multi-value
credit "Okayracer, aldrch & poptropicaslutz!" — primary IS in the
credit. Splitting found it.

Skowl's case: both sides single-value but DIFFERENT artists.
Splitter has nothing to find — Eclypse simply isn't in "Andromedik".
Different bug.

# Cause

Scanner SQL at `core/repair_jobs/acoustid_scanner.py:281` joined
the `artists` table via `tracks.artist_id` which points at the
ALBUM artist (the curator/label-name applied to every row in a
compilation). The `tracks.track_artist` column already holds the
correct per-track artist for compilations — populated by every
server-scan path (Plex `originalTitle`, Jellyfin `ArtistItems`,
Navidrome per-track `artist`) AND the auto-import / direct-download
post-process flow (`record_soulsync_library_entry` writes it when
different from album artist). Scanner just wasn't reading it.

# Fix

```sql
SELECT t.id, t.title,
       COALESCE(NULLIF(t.track_artist, ''), ar.name) AS artist,
       ...
```

Prefers per-track artist when populated, falls back to album artist
for legacy rows / single-artist albums where `track_artist` is NULL.
`NULLIF(t.track_artist, '')` handles the empty-string-instead-of-null
case some legacy rows might have.

# Composes with Foxxify's multi-value fix

For the rare compilation track where AcoustID ALSO returns a
multi-value credit (e.g. compilation track has multiple credited
performers), both paths work together — `track_artist` gives the
correct expected primary, then the helper splits the credit and
finds it.

# Tests added (2)

- `test_load_db_tracks_prefers_track_artist_for_compilation` —
  reporter's exact case: track with `track_artist='Eclypse'` AND
  `artist_id` pointing at album artist 'Andromedik' resolves to
  'Eclypse'. Second track with NULL `track_artist` falls back to
  album artist 'Andromedik' (single-artist + legacy compat).
- `test_load_db_tracks_falls_back_when_track_artist_empty_string`
  — empty string in `track_artist` (some legacy rows) → NULLIF
  returns NULL → COALESCE falls back to album artist.

Both use a real SQLite DB so the COALESCE/NULLIF logic + JOIN
runs against actual schema (SimpleNamespace fakes can't simulate
JOINs).

# Verification

- 6/6 scanner tests pass (2 new + 4 existing)
- 2586 full suite passes (+2 from prior commit)
- Ruff clean
2026-05-10 19:44:57 -07:00
Broque Thomas
df304eb016 AcoustID scanner: handle multi-value artist credits
Discord report (Foxxify): the AcoustID scanner repair job flagged
multi-artist tracks as Wrong Song because AcoustID returns the
FULL credit ("Okayracer, aldrch & poptropicaslutz!") while the
library DB carries only the primary artist ("Okayracer"). Raw
SequenceMatcher similarity scored ~43% — well below the 60%
threshold — so the scanner created a finding even though the
audio was correct. User couldn't fix without lowering the global
artist threshold to ~30% (which would let real mismatches through).

# Fix

Extended the shared `core/matching/artist_aliases.py::artist_names_match`
helper (originally lifted for #441) with credit-token splitting.
When the actual artist string contains common separators —

- punctuation: `,`  `&`  `;`  `/`  `+`
- keywords (whitespace-bounded): `feat.` `ft.` `featuring` `with`
  `vs.` `x`

— the helper splits into individual contributors and checks each
against the expected artist. Primary-in-credit cases now resolve
at 100% instead of 43%.

Two pattern groups because punctuation separators don't need
surrounding whitespace, but keyword separators MUST be
whitespace-bounded — otherwise we'd split artists with `x` /
`with` etc. in their names ("JAY-X" → "JAY-" / "" issue).

Composes with the existing alias path: cross-script multi-artist
credits ("Hiroyuki Sawano" expected, "澤野弘之, FeaturedJp"
actual) work via alias-token-against-credit-token compare.

# Wire-in

Scanner at `core/repair_jobs/acoustid_scanner.py:202` replaces
the raw `SequenceMatcher` call with `artist_names_match`. Pass
RAW artist strings (not pre-normalised by `_normalize`) so the
splitter can recognise separators — `_normalize` strips ALL
punctuation, which destroyed the very tokens the splitter needs.

The AcoustID post-download verifier (`core/acoustid_verification.py`)
already routes through `_alias_aware_artist_sim` which calls the
same helper — gets the multi-value benefit automatically without
a separate wire-in.

# New `split_artist_credit` exported helper

Pure-function helper for callers who want token-level access to
the credit list (debugging, UI, future per-token enrichment). Same
splitter logic, exposed as a top-level function.

# Tests added (14)

`tests/matching/test_artist_aliases.py` (+11):
- `TestSplitArtistCredit` — parametrised across 12 credit-string
  formats (comma, ampersand, semicolon, slash, plus, feat./ft./
  featuring, with, vs., x, single-token, empty), drops empty
  tokens, strips per-token whitespace
- `TestMultiValueCreditMatching` — reporter's exact case
  (Okayracer in 3-artist credit → 100%), primary in middle/end of
  credit, genuine-mismatch still fails, single-token actual falls
  through to direct compare, multi-value composes with aliases,
  threshold still respected

`tests/test_acoustid_scanner.py` (+3):
- Reporter's case end-to-end through `_scan_file` — fingerprint
  99% / title 100% / multi-artist credit → no finding created
- Genuine artist mismatch still creates finding (no false
  suppression of real mismatches)
- `JobResultStub` minimal scaffold for the integration tests

# Verification

- 14 new tests pass (49 helper + 5 scanner total in their files)
- 110 matching + scanner tests pass total
- 2584 full suite passes (+25 from baseline 2559)
- Ruff clean
- Reporter's exact case (Okayracer in `Okayracer, aldrch &
  poptropicaslutz!`) now scores 100% match → no Wrong Song flag
2026-05-10 19:17:59 -07:00
Broque Thomas
9602d1827c Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites
Catches the silent excepts the awk-based earlier sweeps missed:

- Bare `except:` followed by `pass` (also swallows KeyboardInterrupt
  and SystemExit — actively wrong). Upgraded to `except Exception as
  e: logger.debug("...: %s", e)`. ~14 sites across connection_detect,
  soulseek_client, listenbrainz_manager, watchlist_scanner,
  youtube_client, navidrome_client, jellyfin_client, web_server.
- `except Exception:` + pass that the awk pattern missed (e.g.
  multi-line or unusual whitespace). ~31 sites across automation_engine,
  database_update_worker, music_database, spotify_client, web_server,
  others.
- 14 legitimate cleanup sites left silent with explicit `# noqa: S110`
  + comment explaining why (atexit handlers, finally-block conn.close
  calls). Logging during shutdown can itself crash because file handles
  get torn down before the handler fires.

Also enables `S110` rule in pyproject.toml so this pattern fails CI
going forward — drift fails at PR review instead of at runtime against
a wedged worker thread. Tests path keeps S110 ignored (test fixtures
legitimately use try-except-pass for cleanup).

Adds a WHATS_NEW entry to helper.js summarizing the full #369 sweep.

Verified: `python -m ruff check .` → All checks passed.
Verified: `python -m pytest tests/` → 2188 passed.

Closes #369
2026-05-07 11:16:06 -07:00
Broque Thomas
e95452b465 Surface silent exceptions in workers + repair jobs — ~30 sites
Across all background workers (Spotify/Tidal/Deezer/Qobuz/iTunes/
Discogs/Genius/AudioDB/MusicBrainz/Last.fm/SoulID + the metadata-update
worker) and the repair-job scanners. All converted to
`logger.debug("...: %s", e)`.

Two `_e` renames in genius_worker and soulid_worker where outer scope
was already binding `e`. Two finally-block sites in repair_jobs/
library_reorganize.py left silent (conn.close on shutdown path).

Refs #369
2026-05-07 10:27:24 -07:00
Broque Thomas
ca5c93162c Rewrite Library Reorganize job to delegate to per-album planner
GitHub issue #500 (@bafoed). Library Reorganize repair job moved
album tracks to single-template paths because of a fragile
classification heuristic. Concrete symptom: a track at
``Surf Curse/Surf Curse - Nothing Yet (2017)/01 - Christine F.flac``
got proposed for a move to
``Surf Curse/Surf Curse - Christine F/Surf Curse - Christine F.flac``
(single template) instead of staying under the album folder.

Root cause: the job had its own tag-reading + transfer-folder-walk +
template-application implementation. The classification was
``is_album = (group_size > 1)`` where ``group_size`` was the count
of same-album tracks currently sitting in the transfer folder being
scanned. Two failure modes:

- only one track of an album was in the transfer folder (rest already
  moved to the library, or not yet downloaded), or
- album tags varied slightly across tracks (e.g. ``"Buds"`` vs
  ``"Buds (Bonus)"``)

Either case gave a 1-element group → routed through the SINGLE
template → wrong destination.

Rewrite — delegate to the per-album planner the artist-detail
"Reorganize" modal already uses:

- ``core.library_reorganize.preview_album_reorganize`` for path
  computation (DB-driven, knows the album has N tracks regardless of
  how many sit in transfer; album-vs-single is structurally correct)
- ``core.reorganize_queue.enqueue_many`` for apply mode; the queue
  worker dispatches via ``reorganize_album`` which handles file move
  + post-processing + DB update + sidecar through the same code path
  the per-album modal uses

Job's per-album loop:

- iterate albums for the active media server only (matches the artist-
  detail modal's scope; multi-server users won't have the job touch
  the inactive server's files at paths they can't see)
- preview each album, catch exceptions per-album so one bad row
  doesn't abort the scan
- branch on planner status:
  - ``no_album`` / ``no_tracks`` (race: album deleted mid-scan) →
    skip silently
  - ``no_source_id`` (album never enriched) → emit ONE album-level
    "needs enrichment first" finding (vs N per-track findings cluttering
    the UI)
  - ``planned`` → filter mismatched tracks (matched + new_path +
    not unchanged + file_exists), emit per-track findings (dry-run)
    or collect album for bulk enqueue (apply)
- bulk enqueue at end of loop using the queue's correct return-shape
  (``{'enqueued': N, 'already_queued': M, 'total': K}``)

What's gone (~500 LOC):
- ``_read_tag_metadata`` / ``_get_audio_quality`` / transfer-folder walk
- ``_load_album_years`` / ``_lookup_years_from_api`` (planner does this)
- ``_apply_path_template`` / ``_build_path_from_template``
- direct ``shutil.move`` + sidecar move logic (queue handles)
- the fragile ``is_album = group_size > 1`` heuristic — structurally gone
- ``move_sidecars`` setting (no longer applicable; queue's post-process
  re-downloads cover art at the destination)

What stays:
- dry-run vs apply toggle
- ``file_organization.enabled`` gate
- stop / pause respect
- progress reporting
- findings for the UI

Cleaner separation of concerns:
- this job: DB-known tracks at wrong paths (active server only)
- ``orphan_file_detector``: files on disk with no DB entry
- ``dead_file_cleaner``: DB entries pointing to nonexistent files

Tests: 12 tests in ``tests/test_library_reorganize.py`` pin the
delegation contract — every status branch, every track-filter case,
exception handling, apply-mode enqueue payload, active-server scope,
estimate-scope shape. Three obsolete ``_lookup_years_*`` tests removed
(year handling moved to planner).

Closes #500 (the misclassification half — orphan + dead-file are
downstream sync-gap symptoms, separate concern).
2026-05-06 21:18:20 -07:00
Broque Thomas
cf5461f2f1 Fix: maintenance findings badge inflated when scan dedup-skipped
`_create_finding` silently dedup-skipped re-discovered issues but
the caller incremented `findings_created` regardless. So a re-scan
that found the same issues as a prior scan reported 364 findings
in the badge while 0 NEW pending rows hit the db, leaving the
findings tab empty.

`_create_finding` now returns bool (True on insert, False on
dedup-skip / db error). All 16 repair jobs updated to only
increment `findings_created` on True. Added `findings_skipped_dedup`
counter surfaced in scan log: "Done: X scanned, 0 fixed, 0
findings (363 already existed), 0 errors".

Also fixed a missing `job_id` kwarg in album_tag_consistency that
was silently breaking finding creation for that scan.
2026-05-04 08:55:13 -07:00
Broque Thomas
4b15fe0b75 Fix album MBID inconsistency: detector + persistent release-MBID cache
Discord report (Samuel [KC]): tracks of the same album sometimes carry
different MUSICBRAINZ_ALBUMID tags, which causes Navidrome (and other
media servers grouping by album MBID) to split the album into multiple
entries. Two-part fix — one for existing libraries, one for the root
cause that lets new imports drift.

Part 1 — Detector + fix action (catches existing dissenters):

`core/repair_jobs/mbid_mismatch_detector.py`:
- New helpers: `_read_album_mbid_from_file` and
  `_write_album_mbid_to_file` use the Picard-standard tag conventions
  (`TXXX:MusicBrainz Album Id` for MP3, `MUSICBRAINZ_ALBUMID` for
  FLAC/OGG, `----:com.apple.iTunes:MusicBrainz Album Id` for MP4).
- New scan phase `_scan_album_mbid_consistency` runs after the
  existing track-MBID scan: groups tracks by DB `album_id`, reads
  each track's embedded album MBID, finds the consensus
  (most-common) MBID via `Counter`, flags dissenters. Tracks without
  an album MBID at all are skipped (they don't break Navidrome —
  only an explicit MBID disagreement does). Albums where MBIDs are
  perfectly tied (no clear consensus) are skipped too — surface as
  a manual decision instead of fixing toward a 1/N tie.
- New finding type `album_mbid_mismatch` carries `consensus_mbid`,
  `wrong_mbid`, `consensus_count`, `total_tracks_with_mbid`, and a
  human-readable reason string.

`core/repair_worker.py`:
- Added `'album_mbid_mismatch': self._fix_album_mbid_mismatch` to the
  fix dispatch dict and to the `fixable_types` tuple so auto-fix +
  bulk-fix paths pick it up.
- New `_fix_album_mbid_mismatch` method reads `consensus_mbid` from
  finding details, resolves the dissenter's file path via the shared
  library resolver, calls `_write_album_mbid_to_file` to rewrite the
  tag in place. Doesn't touch the album's other tracks (they're
  already in agreement).

Part 2 — Root cause fix (prevents new SoulSync imports from drifting):

The original in-memory `mb_release_cache` in `core/metadata/source.py`
maps `(normalized_album, artist) -> release_mbid` so per-track
enrichment of the same album hits the cache and writes the same
MUSICBRAINZ_ALBUMID to every track. That cache is bounded (4096
entries) and in-process — so cache eviction (when other albums are
processed in between) and server restart can BOTH cause
inconsistency. Per-track album-name variation (e.g. some tracks
tagged `"Album"`, others tagged `"Album (Deluxe)"`) and per-track
artist variation (features) make it worse.

`core/metadata/album_mbid_cache.py` (new module):
- DB-backed `lookup(normalized_album, artist) -> release_mbid` and
  `record(...)` functions. Same key shape as the in-memory cache.
- Strict additive design: every public function is wrapped in
  try/except and degrades to None / no-op on ANY database error.
  The existing in-memory cache + MusicBrainz lookup remains the
  authoritative fallback. If this module breaks, downloads continue
  exactly as they would today.

`database/music_database.py`:
- New `mb_album_release_cache` table with composite primary key
  `(normalized_album_key, artist_key)`. Reverse-lookup index on
  `release_mbid` for future debug tooling. Created via the existing
  `CREATE TABLE IF NOT EXISTS` migration pattern — idempotent, no
  schema version bump needed.

`core/metadata/source.py`:
- Surgical change inside the existing `embed_source_ids`
  in-memory-cache-miss branch: BEFORE calling MusicBrainz, consult
  the persistent cache. If a previous SoulSync run already resolved
  this album's release MBID, reuse it. After a successful MB lookup,
  store in BOTH caches. Both calls wrapped in defensive try/except
  so any failure falls through to existing logic.

Tests:
- `tests/metadata/test_album_mbid_cache.py` — 16 cache tests:
  round-trip, idempotent re-record, overwrite semantics, clear_all,
  album+artist independence (no Greatest Hits collisions),
  defensive None-on-empty-input, graceful degradation when the DB
  is unavailable / connection raises / commit fails, schema sanity
  (table + index exist after init).
- `tests/test_album_mbid_consistency.py` — 13 detector tests:
  tag read/write round-trip on real FLAC files, Picard-standard tag
  descriptors, defensive paths (unreadable file, empty input),
  detector behavior (agreement → no flags, lone dissenter → flag,
  ties → no flag, single-track albums → skipped, no-MBID tracks →
  skipped, unresolvable file paths → skipped).
- `tests/metadata/test_metadata_enrichment.py` — added autouse
  fixture monkeypatching the persistent cache to no-op for tests in
  this file. The existing tests pin per-call MB counts and
  in-memory cache state; without the fixture, persistent rows from
  earlier tests would bypass the MB call. Persistent layer has its
  own dedicated tests.

Verified: 1782 tests pass (29 new), ruff clean, smoke test confirms
end-to-end cache round-trip works.

WHATS_NEW entry under '2.4.2' dev cycle.
2026-05-03 17:16:39 -07:00
Broque Thomas
d8437c87c6 Fix Album Completeness Auto-Fill on Docker / shared-library setups (#476)
GitHub issue #476 (gabistek, Docker on Arch host): "Auto-Fill" / "Fix
Selected" on the Album Completeness findings page returned
"Could not determine album folder from existing tracks" for every album.
Reproduces on any setup where the media-server library lives outside the
SoulSync transfer/download folders — Docker is the headline case but
native installs that point Plex at a NAS via SMB hit it too.

Root cause: `core/repair_worker.py:_resolve_file_path` only probed the
transfer + download folders. Docker users have their Plex/Jellyfin
library bind-mounted at /music (or similar) — neither configured in
SoulSync. Every existing track got silently treated as missing, so
`album_folder` stayed None and the fix workflow bailed.

The same incomplete logic was duplicated four more times in the
repair_jobs/ modules, all with the same bug. Album Completeness was
just the most user-visible — the same setups were also producing false
"missing file" findings from Dead File Cleaner, silent skips in
MBID Mismatch Detector, etc.

The web server already had the correct logic at
`web_server.py:_resolve_library_file_path` (probes transfer + download
+ Plex-reported library locations + user-configured library.music_paths).
The repair workers had never been updated to match.

Fix:
- New `core/library/path_resolver.py` extracts the union logic into a
  single shared function `resolve_library_file_path()`. Probes (in
  order, deduped): explicit transfer/download kwargs, config-derived
  soulseek.transfer_path/download_path, Plex-reported library
  locations (when a plex_client is passed), user-configured
  library.music_paths. Each defensive: malformed config or a flaky
  Plex client degrades to the dirs that did succeed.
- `core/repair_worker.py:_resolve_file_path` becomes a delegating
  wrapper preserving the legacy signature, with a new `config_manager`
  kwarg. All 15 in-tree call sites updated to thread
  `self._config_manager` through.
- `core/repair_jobs/dead_file_cleaner.py`,
  `mbid_mismatch_detector.py`, and `lossy_converter.py` get the same
  treatment: duplicate function replaced with a thin wrapper, call
  sites pass `context.config_manager`.
- `core/repair_jobs/acoustid_scanner.py` and
  `unknown_artist_fixer.py` (which used to import from repair_worker)
  now call the shared resolver directly with `context.config_manager`.

Side benefit: every other repair job (Dead File Cleaner, MBID
Mismatch Detector, Lossy Converter, AcoustID Scanner, Unknown Artist
Fixer) also stops missing files in the media-server library mount.
Single fix unblocks five user-visible features.

Tests: `tests/library/test_path_resolver.py` — 20 cases covering all
four base-dir sources, suffix-walk algorithm, dedup, defensive paths
(None plex client, malformed config entries, raising config_manager.get,
broken plex attribute access), Docker path translation. Full suite
1677 passed locally.

WHATS_NEW entry under '2.4.2' dev cycle.
2026-05-03 10:11:06 -07:00
Broque Thomas
0e68109b68 Add filename-pass safety: require duration agreement or artist match
Self-review of the previous commit found a real false-positive risk in
the new filename-bucket pass: two unrelated songs that happen to share
a canonical filename (e.g. ``Yellow.mp3`` by Coldplay vs by some other
artist) would be grouped because all metadata gates were dropped.

The filename pass now layers a safety net under ``require_metadata_match=False``:

- If both rows carry a duration: must agree within 3 seconds. Same
  source download = identical duration; a 3+ second gap means
  different recordings.
- Else if both rows carry an artist: relaxed 0.6 similarity check —
  catches dedup orphans that share an artist tag while rejecting
  strangers-with-same-filename.
- Else (no duration AND at least one artist blank): skip — too little
  signal to safely group.

5 additional regression tests cover the false-positive prevention
paths plus the genuine dedup-orphan scenarios that must still be
caught after the safety net.
2026-05-01 13:15:57 -07:00
Broque Thomas
6e61890551 Stop watchlist re-downloading compilation tracks; catch slskd dedup orphans
Two related bugs reported on Discord by Mushy.

1. The watchlist re-downloaded the same OST track up to 7 times.

   ``is_track_missing_from_library`` compared Spotify's album name and
   the media-server scan's album name with a raw SequenceMatcher at a
   strict 0.85 threshold. Compilations and soundtracks routinely fail
   this — Spotify reports
   ``"Napoleon Dynamite (Music From The Motion Picture)"`` while the
   Plex / Navidrome / Jellyfin tag scan saves it as
   ``"Napoleon Dynamite OST"``. Raw similarity ≈ 0.49, so the scanner
   declared the track missing on every 30-minute scan and added it back
   to the wishlist. The wishlist then issued a fresh download. slskd
   appended ``_<19-digit-ns-timestamp>`` to each new copy because the
   target file already existed, and the user ended up with seven copies
   of one song in one folder.

   Fix: extract two pure helpers — ``_normalize_album_for_match``
   strips qualifier parentheticals (Music From X, OST, Deluxe Edition,
   Remastered, Anniversary, etc.) and trailing dash-clauses;
   ``_albums_likely_match`` checks equality after normalization,
   substring containment, and a relaxed 0.6 fuzzy ratio. A volume /
   part / disc / standalone-trailing-number guard rejects pairs like
   ``"Greatest Hits Vol. 1"`` vs ``"Greatest Hits Vol. 2"`` so the
   relaxed threshold doesn't introduce false positives on serialized
   releases. After this change the Napoleon Dynamite case collapses
   to ``"napoleon dynamite" == "napoleon dynamite"`` via the equality
   short-circuit and the redownload loop dies.

2. The duplicate detector found only one of the seven dupe files.

   The detector buckets tracks by the first 4 chars of their normalized
   tag title. Files written by slskd directly into a library folder
   often get inconsistent (or blank) tags from the media-server rescan,
   so the seven copies were bucketed apart by parsed title and never
   compared.

   Fix: refactor the per-bucket comparison into ``_scan_bucket``, then
   add a second pass — ``_build_filename_buckets`` re-buckets leftover
   tracks by canonical filename stem (slskd dedup tail stripped via
   ``_strip_slskd_dedup_suffix``, same regex the import-cleanup PR uses)
   plus extension. Filename agreement is itself strong evidence the
   files came from the same source download, so the second pass calls
   ``_scan_bucket`` with ``require_metadata_match=False`` to skip the
   title / artist / cross-album gates. The same-physical-file guard
   still runs so bind-mount duplicates aren't flagged.

72 new regression tests across two files cover the album-match
helpers (28 tests including the Napoleon Dynamite scenario, 7 volume
disagreements, 8 positive/negative pairs, 5 defensive cases) and the
new filename-bucket pass (16 tests across bucket construction, scan
integration, and existing title-pass behavior). Full pytest 1509
passed; ruff clean.

Reported by Mushy in Discord.
2026-05-01 12:57:50 -07:00
Broque Thomas
382e427117 Filter same-physical-file duplicates from duplicate detector
When users bind the same host music directory into both SoulSync
(e.g. /app/Transfer) and a media server like Plex (e.g.
/media/Music), both scans add a track row pointing at the same
physical file via different mount paths. The detector previously
flagged those as duplicate groups even though there's only one
file on disk.

New _is_same_physical_file helper filters pairs where:
- The trailing 3 path segments match (filename + album + artist
  folder), so they're the same release on disk.
- The leading mount roots actually differ.
- Durations agree within 1s when both rows carry duration data.

Adds 10 regression tests covering the reported scenario plus
edge cases (Windows separators, case differences, missing
durations, sibling-album false-positive guard).
2026-04-30 14:06:10 -07:00
Broque Thomas
a60546929e Fix Album Completeness job reporting zero findings for every album
Reported by sassmastawillis: the Album Completeness maintenance job
scans 3127 albums in 0.1 seconds and reports 0 findings — for every
user, regardless of whether their library is actually complete.
Restoring an older DB surfaced 7 correct findings, so the code logic
works; the DB state is what's making everything look complete.

Root cause: `albums.track_count` is only ever written by server-sync
paths — Plex's `leafCount`/`childCount` and SoulSync standalone's
`len(tracks)`. It's the OBSERVED count of tracks SoulSync has indexed,
which is always exactly what `COUNT(tracks)` returns for that album.
The completeness job treated it as the EXPECTED total and compared it
against the observed count. They're equal by construction, so
`actual >= expected` is always true: skip, 0.1s scan, 0 findings.

Fix: new `api_track_count INTEGER` column on `albums`, written only by
metadata-source code paths. Populated in two places so the scan is
fast and the fallback is robust.

1. Enrichment workers — shared helper `set_album_api_track_count`
   in `core/worker_utils.py`. Called by each worker's existing
   `_update_album` method alongside its other album-column UPDATEs:

   - spotify_worker: `album_obj.total_tracks` from the Spotify Album
     dataclass (already in hand, zero new API calls)
   - itunes_worker: same, from the iTunes Album dataclass
   - deezer_worker: `nb_tracks` from full_data, falling back to
     search_data when the full lookup didn't run
   - discogs_worker: count of tracklist rows where `type_=='track'`
     (Discogs tracklists interleave heading and index rows that
     shouldn't count as songs)

   Helper skips the write on zero/None/negative/non-numeric inputs
   so a source lacking track info can't clobber a good value a
   different source already wrote. Caller owns the transaction —
   helper just queues an UPDATE on the caller's cursor without
   committing, so it batches cleanly with each worker's existing
   multi-UPDATE pattern.

   Hydrabase worker deliberately not touched — it's a P2P mirror
   that doesn't write album metadata to the local DB. Hydrabase-
   primary users hit the fallback path below.

2. Album Completeness repair job — new `al.api_track_count` column
   in the SELECT, read first in the scan loop. On miss (album never
   enriched, or enrichment workers haven't run yet on a fresh
   install), falls through to the existing `_get_expected_total()`
   API lookup and persists the result via the same shared helper
   (wrapped in connection/commit management since the repair job
   runs outside a worker's batched transaction).

Also removed `al.track_count` from the scan's SELECT — now unused
since the observed count was the whole source of this bug, and
leaving a dead SELECT would invite a future engineer to re-introduce
the same comparison.

Help text on the job card was reworded so it honestly describes
current behavior ("counts cached during normal enrichment are used
when available; otherwise the job queries a metadata source
directly") rather than the old "active provider first, then others
as fallback" phrasing, which doesn't match how the cache actually
fills — any enrichment worker that runs can populate it, and the
last writer wins. Document-only follow-up if this edge case ever
bites in practice: add a `api_track_count_source` column so the
scan can prefer the configured primary source's count over others
(e.g. deluxe vs. standard edition mismatches). Not worth the
complexity today.

For existing users, the first completeness scan after upgrade is
fast to the extent their library is already enriched: the workers
already ran and populated `api_track_count` on their normal schedule.
For brand-new installs, the scan's fallback path handles the cold
start — slower, but correct, and subsequent scans are fast.

Does NOT affect:
- Download / post-processing / wishlist / sync code paths — none
  of them read `track_count` for completeness semantics.
- Plex / Jellyfin / Navidrome / standalone sync — still write
  `track_count` exactly as before; `api_track_count` is a separate
  column they never touch.
- Other repair jobs.
- Any UI path — same finding schema, just correct counts now.

Files:
- database/music_database.py — idempotent migration adding
  `api_track_count INTEGER DEFAULT NULL` to the existing album-column
  check block.
- core/worker_utils.py — new `set_album_api_track_count` helper with
  the documented skip-on-bad-input contract.
- core/spotify_worker.py, itunes_worker.py, deezer_worker.py,
  discogs_worker.py — one-liner call from each `_update_album`.
- core/repair_jobs/album_completeness.py — scan uses the cache;
  fallback path persists API-lookup results via the shared helper;
  help text updated to match actual behavior.
- tests/test_worker_utils_album_track_count.py — 9 tests covering
  the helper's write/skip contract + no-commit invariant.
- tests/test_album_completeness_job.py — 2 tests for the repair
  job's fallback-path wrapper.
- webui/static/helper.js — WHATS_NEW entry.

Credit: sassmastawillis spotted the bug; the "restored older DB
finds 7 albums" signal pinpointed DB state over code logic and
made the diagnosis tractable.
2026-04-24 12:39:41 -07:00
Broque Thomas
78fa83c8ac Add $cdnum template variable for multi-disc filenames
New smart template variable that emits "CD01" / "CD02" etc. in filenames
on multi-disc albums, and expands to empty string on single-disc albums
so mixed libraries don't end up with "CD01" on every single.

Template behaviour:
- total_discs > 1 -> "CD{disc:02d}" (zero-padded, CD prefix)
- total_discs <= 1 -> empty string
- Both $cdnum and ${cdnum} bracket form supported
- Empty value collapses cleanly via existing double-dash regex plus new
  leading-dash cleanup pass

Wiring:
- _apply_path_template in web_server.py (download pipeline)
- _apply_path_template in core/repair_jobs/library_reorganize.py
  (Reorganize repair job)
- total_discs added to every album-mode template context:
  * download pipeline album branch (uses resolved total_discs even for
    single-track downloads from search)
  * per-album Reorganize preview + apply endpoints (pre-scan all track
    tags once, take max disc_number)
  * Library Reorganize repair job (already had album_total_discs map,
    just added to context dict)

Leading-dash cleanup added to _get_file_path_from_template (web_server)
and _build_path_from_template (library_reorganize) so templates like
"$cdnum - $track - $title" don't leave "- 05 - Title" on single-disc
albums.

UI:
- Template hint in Settings -> File Organization documents $cdnum
- Template validation variable list includes $cdnum
- Reorganize modal variable reference shows $cdnum with example "CD01"

Verified:
- Multi-disc disc 1 -> "CD01 - 05 - Track"
- Multi-disc disc 2 -> "CD02 - 05 - Track"
- Single-disc      -> "05 - Track" (no leading dash)
- Templates without $cdnum behave unchanged
- 276/276 tests pass
2026-04-21 22:55:37 -07:00
Broque Thomas
e5d4d61c0e Fix watchlist content filters: live false positives + auto-scan bypass
Two bugs reported in issue #320:

1. Auto-watchlist scan bypassed Global Override settings.
   scan_watchlist_profile applied _apply_global_watchlist_overrides, but
   the scheduled auto-scan called scan_watchlist_artists directly —
   bypassing the override. Users who unchecked "Albums" or "Live" under
   Watchlist → Global Override still saw full albums and live tracks
   added during nightly scans (per-artist defaults, which include
   everything, won).

   Moved override application into scan_watchlist_artists itself so
   every entry point respects it. scan_watchlist_profile now forwards
   the apply_global_overrides flag through to avoid double-application.

2. is_live_version (watchlist + discography backfill) and
   live_commentary_cleaner's content patterns used bare \blive\b, which
   matched verb uses like "What We Live For" by American Authors,
   "Live Forever" by Oasis, "Live and Let Die" by Wings.

   Tightened the live patterns to require clear recording context:
   (Live) / [Live Version] / - Live / Live at|from|in|on|version|
   session|recording|performance|album|show|tour|concert|edit|cut|take
   / In Concert / On Stage / Unplugged / Concert.

   Locked in 11 regression tests covering the reported false positives
   (What We Live For, Live Forever, Living on a Prayer, Live and Let Die)
   and the reported true positives (Dimension - Live at Big Day Out,
   MTV Unplugged, etc.).

Version bumped to 2.37 with changelog entries.
2026-04-21 19:16:25 -07:00
Broque Thomas
457763cbab Rebuild Discography Backfill: auto-wishlist, Fix All, section UI
Root-cause fix for "scanning 50 artists" then silence: when the master
repair worker was paused, force-run still kicked off _run_job but the
job's first wait_if_paused() blocked forever because is_paused was tied
to the master-enabled state. Force-run now bypasses master-pause —
scheduled runs still respect it.

Also fixes Fix All on discography findings doing nothing: the backend
bulk_fix_findings query had a fixable_types allowlist that excluded
missing_discography_track (and acoustid_mismatch). Added both.

Backfill job rebuild:
- auto_add_to_wishlist opt-in setting — creates findings AND pushes to
  wishlist during the scan
- 3-option fix dialog (Add to Wishlist / Just Clear / Cancel) on single
  Fix, Bulk Fix selection, and Fix All (page-level)
- Fix All "Just Clear" path uses the clear endpoint with job_id filter
  instead of the generic "may delete files" bulk-fix warning
- Batched in-memory matching using get_candidate_albums_for_artist +
  get_candidate_tracks_for_albums (same fast path the Library pages use)
- Rich album context per finding (id, name, album_type, release_date,
  images, artists, total_tracks) — flows through the wishlist pipeline
  so auto-processor classifies each track into the right cycle
  (albums vs singles) and post-processing gets correct folder/tags/art
- Per-artist progress logs [N/50] Scanning ArtistName
- Default interval 24h (was 168h); all release types default on; settings
  reordered with _section_* group headers (Core / Release Types /
  Content Filters)

Repair settings UI:
- Generic _section_<name> key convention renders as an uppercase group
  divider in the settings panel — any job can opt in
- .repair-setting-row gets a dashed bottom border so label↔toggle pairing
  is visually clear
- _prettifyRepairSettingKey fixes acronym capitalization (EPs, not Eps)

Version bumped to 2.36 with changelog entries.
2026-04-21 18:44:43 -07:00
Broque Thomas
39a07e4bdf Fix Discography Backfill silently skipping most releases
Two bugs kept this job from finding anything useful on a typical library.

1. Wrong Deezer column name. The artists table has a deezer_id column
   (per music_database.py:1986), but the job looked for deezer_artist_id
   in both _scan_artist (line 132) and _get_library_artists (line 345).
   For Deezer-primary users, this meant the Deezer ID never made it into
   the source_ids map, so get_artist_discography fell back to artist-
   name-only search — slower and less accurate than an ID lookup.

2. Spotify-reported EPs were silently excluded. Spotify lumps EPs and
   true singles under album_type='single'. The previous
   _should_include_release short-circuited on album_type='single' and
   returned the include_singles setting (default False), so 4-6 track
   EPs on Spotify-primary libraries never survived the filter — even
   though include_eps defaulted to True. Only 7+ track full albums
   made it through. This is the main reason users felt the job did
   nothing.

Fixes:

- Use the correct deezer_id column name in both reference sites.

- Restructure _should_include_release so only 'album', 'ep', and
  'compilation' are trusted outright. Anything else (including
  'single' and missing type) falls through to a track-count
  disambiguation matching the download pipeline's _get_album_type_display:
  1-3 tracks = true single, 4-6 = EP, 7+ = album. A Spotify-returned
  'single' with 5 tracks now correctly counts as an EP.

Full suite stays at 263 passed. Ruff clean.
2026-04-21 17:26:38 -07:00
Broque Thomas
d9217237d2 Clean up 286 ruff lint errors to unblock CI and fix 10 latent bugs
PR #340 added ruff to the build-and-test.yml CI gate, which surfaced
286 pre-existing lint errors. Left unfixed, every feature branch push
fails CI. This commit resolves all of them so CI goes green and
contributors can actually land work.

Auto-fixes (248 of 286): removed unused f-string prefixes (F541),
renamed unused loop control variables with underscore prefix (B007),
removed duplicate imports (F811).

Manually fixed 10 latent bugs ruff caught (all wrapped in try/except
today, silently failing):

- music_database.py: _add_discovery_tables() called undefined
  conn.commit() — would have crashed the iTunes-support migration
  for existing databases. Now uses cursor.connection.commit().
- web_server.py settings GET: referenced undefined download_orchestrator
  when it should be soulseek_client. Feature (_source_status on the
  settings payload) was silently missing for UI auto-disable logic.
- web_server.py _process_wishlist_automatically: active_server
  undefined in track-ownership check. Auto-wishlist was falling
  through to the error handler and re-downloading owned tracks.
- web_server.py start_wishlist_missing_downloads: same active_server
  bug in the manual wishlist path.
- web_server.py _process_failed_tracks_to_wishlist_exact: emitted
  wishlist_item_added automation event with undefined artist_name
  and track. Automation event silently never fired correctly.
- web_server.py discovery metadata enrichment: referenced cache
  without calling get_metadata_cache() first. Track enrichment from
  cached API responses was silently skipped.
- web_server.py Beatport discovery worker: wing-it fallback branch
  used undefined successful_discoveries variable. Wing-it counter
  never incremented correctly. Now uses state['spotify_matches']
  consistently with the rest of the function.
- web_server.py _run_full_missing_tracks_process: stale import json
  mid-function shadowed the module-level import, making an earlier
  json.dumps() call reference an unbound local (F823).
- web_server.py discovery loop: platform loop variable shadowed
  the module-level platform import (F402).
- core/watchlist_scanner.py: 7 lambda captures of loop variables
  (B023 classic Python closure-in-loop bug) now bind at creation.

No existing tests had to change. Full suite stays at 263 passed.
2026-04-21 13:30:52 -07:00
Broque Thomas
ed97fecc31 Add Discography Backfill maintenance job
New repair job that scans each artist in the library, fetches their
full discography from metadata sources, and creates findings for any
tracks not already owned. Users review findings and click "Add to
Wishlist" to queue missing tracks for download.

Respects content filters (live/remix/acoustic/instrumental/compilation)
and release type filters (album/EP/single). Opt-in, disabled by default,
runs weekly, processes up to 50 artists per run with rate limiting.
2026-04-20 22:04:38 -07:00
Broque Thomas
c3f88a713a Fix unknown_artist_fixer crash on missing deezer_track_id column
The tracks table uses 'deezer_id' not 'deezer_track_id'. The query
and source field mapping both referenced the wrong column name.
2026-04-20 16:29:38 -07:00
Broque Thomas
0e8e3e86a0 Fix Duplicate Detector ignoring 'allow duplicate tracks across albums'
The Duplicate Detector repair job had its own ignore_cross_album setting
that was independent of the global allow_duplicate_tracks setting. When
a user enabled 'Allow duplicate tracks across albums', the detector
still flagged same-titled tracks on different albums as duplicates.
Now respects the global setting — if duplicates are allowed, cross-album
matches are always skipped.
2026-04-18 18:55:02 -07:00
Antti Kettunen
0e85931cc8 Fix track ID generation in repair flows
Repair-worker album fills now generate explicit track IDs when copying rows, instead of relying on SQLite auto-assignment that no longer exists for TEXT primary keys. The unknown-artist fixer now does the same for new artists.

Also add a regression test for the album-fill copy branch and keep the AcoustID scanner resilient to legacy null-ID rows.
2026-04-17 19:26:08 +03:00
Antti Kettunen
88e2527b96 Fix null-pointer error in acoustid_scanner
The root cause (null track ids) needs to be solved elsewhere, but this is a band-aid for now
2026-04-17 19:17:24 +03:00
Broque Thomas
223522ce99 Upgrade AcoustID scanner to scan full library with actionable fixes
Rewrote the AcoustID scanner job to scan all library tracks (via DB file
paths resolved to disk) instead of only the Transfer folder. Checkpoints
by track ID for robust resume across restarts. Defaults changed to
enabled, 24h interval, batch size 200.

Added _fix_acoustid_mismatch handler with three actions:
- retag: update DB title/artist to match actual audio content
- redownload: add expected track to wishlist and delete wrong file
- delete: remove wrong file and DB record

This catches cases like a file tagged as "Dinosaur Bones" that is
actually "Helicopters" — the scanner fingerprints the audio, detects
the mismatch, and the user can fix it from Library Maintenance findings.
2026-04-16 10:26:39 -07:00
Antti Kettunen
dab58766b7 Make library reorganize source-aware
Respect the configured metadata source order when looking up album years, and re-check provider availability during the scan so Spotify can drop out cleanly if it becomes rate-limited.
2026-04-15 21:35:58 +03:00
Antti Kettunen
03711c10df Make metadata gap filler source-aware
Only fetch source track details when ISRC enrichment is enabled, and show resolved source/track provenance in the UI.
2026-04-15 21:35:57 +03:00
Antti Kettunen
6df6ecb560 Respect source preference in cover art repair job
Cover art lookup now honors an explicit prefer_source first,
falls back to the runtime primary metadata source when unset,
and uses the shared source priority for the remaining fallbacks.
2026-04-15 21:28:12 +03:00
Antti Kettunen
e7faa9f02f Use metadata source priority in track number repair job
Use the shared metadata source priority when resolving album IDs,
album searches, and tracklists in track number repair.

Keeps Deezer and iTunes ahead of Spotify where configured, while
still allowing the job to fall back through other supported sources.
2026-04-15 21:28:12 +03:00
Antti Kettunen
6dca19ca1e Use metadata source priority in unknown artist fixer job
Unknown artist resolution now uses the shared metadata source priority and only filters to the sources that can actually participate in this job. Deezer and iTunes remain direct lookup sources, while Hydrabase can now join the title-search path when it is the configured priority source.
2026-04-15 21:28:10 +03:00
Broque Thomas
fe399636b2 Fix Spotify API calls leaking when Deezer/iTunes is primary source
Spotify was being called for album/artist data fetching across multiple
background workers and the Artists page search even when the user had
Deezer or iTunes set as their primary metadata source. Being authenticated
for playlist sync was treated as permission to use Spotify for everything.

- watchlist_scanner: add _spotify_is_primary_source() that checks both
  auth and primary source config; use it for all album/artist data fetching
  (discovery pool, recent album caching, playlist curation, similar artist
  ID matching, proactive ID backfill). _spotify_available_for_run() is kept
  for sync_spotify_library_cache which must run regardless of primary source
- repair_jobs/metadata_gap_filler: gate Spotify ISRC lookup on primary
  source being 'spotify'; MusicBrainz lookup unaffected
- repair_jobs/unknown_artist_fixer: replace hardcoded spotify_client with
  source-aware client selection — primary source ID tried first, each ID
  matched to its correct client (fixes latent bug passing Deezer IDs to
  Spotify)
- web_server.py /api/match/search: Artists page search was hardcoded to
  spotify_client.search_artists(); now uses _get_metadata_fallback_client()
  so results come from the configured primary source
2026-04-15 09:47:43 -07:00
BoulderBadgeDad
3618f3fa7f
Merge pull request #298 from kettui/fix/respect-metadata-provider-in-album-completeness-job
Honor primary metadata source in album_completeness job and associated repair flow
2026-04-15 07:15:15 -07:00
Antti Kettunen
106d202ccc Share metadata source priority
Centralize the ordered metadata source list and source-priority helper so album completeness and the repair worker follow the same Deezer/iTunes-first fallback order. This also removes the last duplicate priority logic from the touched repair paths.
2026-04-15 07:48:25 +03:00
Antti Kettunen
8a1ae00946 Utilize global Spotify client in repair jobs
Album completeness and any other repair job now uses the centralized source/client helpers instead of a worker-local Spotify client or override plumbing
  - This keeps source selection aligned with the configured primary provider and removes the last Spotify-only special case from the job path.

This change ultimately is a step towards further centralizing the Spotify client access and the associated `is_spotify_authenticated` check.
  - Currently these look-ups are done all over the place in different feature implementations directly, but moving forward, any feature that uses `get_primary_client` or `get_client_for_source` to access the Spotify client, won't have to duplicate any rate-limiting or auth checks as long as these getters are used
2026-04-15 07:18:34 +03:00
Broque Thomas
0edb8e93bb Fix MBID Mismatch Detector comparing MB title against DB title (filename) instead of embedded tag
The scan was reading `t.title` from the database for comparison, which is
populated from the filename rather than the file's embedded TITLE tag. Any
track whose filename differs from the clean MB title (e.g. includes artist,
album, bitrate) was incorrectly flagged as a mismatch.

Fix: extend `_read_mbid_from_file` → `_read_file_tags` to also read the
embedded TITLE tag (ID3 TIT2, Vorbis `title`, MP4 `©nam`). The comparison
now uses the embedded title, falling back to the DB title only when no TITLE
tag is present in the file.

Fixes #296
2026-04-14 12:19:41 -07:00
Antti Kettunen
1a459412a3 Honor primary metadata source in album_completeness job
Album completeness and downstream repair flow now follow the configured
primary provider first, with Discogs and Hydrabase support added alongside
existing Spotify, iTunes, and Deezer paths.

Keep spotify_track_id for compatibility while preserving source-aware track
IDs for provider-neutral handling.
2026-04-14 21:14:46 +03:00
Antti Kettunen
aec3047216 Improve graceful shutdown and rollback safety
- Add interruptible stop events to background workers so shutdown
  wakes out of long sleeps instead of waiting on fixed delays.
- Stop scan managers, repair worker, executors, and cleanup helpers
  deterministically so process exit does not leave background threads
  alive.
- Add startup warnings for stale SQLite WAL/SHM sidecars so unclean
  shutdowns are easier to spot before init/migration errors cascade.
- Prevent forced kills from leaving SQLite sidecars behind, which
  made rollbacks to older branches fail with malformed database
  errors.
2026-04-12 15:17:18 +03:00
Broque Thomas
805f72c5fd Add acappella to live/commentary cleaner patterns 2026-04-11 13:56:56 -07:00
Broque Thomas
5be6a46fb0 Fix dismissed findings reappearing and reduce false orphan detections
The finding dedup check only looked for 'pending' and 'resolved' status,
missing 'dismissed'. Dismissed findings were recreated as new entries on
every scan. Now includes 'dismissed' in the dedup check.

Orphan file detector improvements:
- Increased path suffix matching depth from 3 to 4 segments (covers
  Genre/Artist/Album/track.flac paths)
- Added filename-based fallback when Mutagen can't read file tags —
  parses title from "NN - Title [Quality].ext" pattern and matches
  against parent/grandparent folder names as artist
2026-04-10 17:42:39 -07:00
Broque Thomas
a6117d5174 Replace all legacy metadata_service imports with canonical functions
All callers of _create_fallback_client() and _get_configured_fallback_source()
now use get_primary_client() and get_primary_source() directly. No more
legacy alias usage anywhere in the codebase.
2026-04-10 12:52:36 -07:00
Broque Thomas
52a5d93018 Add Fix Unknown Artists maintenance job
New repair job that scans the library for tracks filed under "Unknown
Artist" and corrects them. Resolves correct metadata by:
1. Reading embedded file tags (if file has correct artist)
2. Looking up by source track ID (Spotify/Deezer/iTunes)
3. Searching by title as last resort

Dry run mode (default) creates findings for review. Live mode re-tags
the audio file, moves it to the correct folder structure, and updates
the database. Includes fix handler for applying individual findings.
2026-04-10 12:17:11 -07:00
Broque Thomas
7cfd1cae3f Fix AcoustID scanner creating thousands of no-match findings
The scanner was creating a finding for every file that couldn't be
identified by AcoustID, flooding the findings list with non-actionable
entries. Users saw the scanner "stuck scanning the same files over
and over" because the no-match findings were dismissed but recreated
on every run. Now only genuine mismatches (AcoustID identifies a
different track) create findings. Errors are counted and shown in
the job log with actual error messages for debugging.
2026-04-09 10:16:53 -07:00
Broque Thomas
ec87cb6d0e Skip zero-track albums in album completeness scanner
Albums with zero local tracks were flagged as incomplete but the
auto-fill fix failed because there were no existing tracks to
determine the album folder or quality standard from. Now skipped
during scan — they'll be detected once tracks are actually added.
2026-04-08 11:49:30 -07:00
Broque Thomas
1fb66b711d Fix Live/Commentary Cleaner missing _get_settings method 2026-04-06 07:45:07 -07:00
Broque Thomas
455945e6cb Fix duplicate detector missing cross-album duplicates (#252)
Changed ignore_cross_album default from True to False. Re-downloads of
the same song create separate album entries, so the detector was skipping
them. Users who want to keep compilations/greatest-hits intact can toggle
it back on. Updated help text to explain when to use this setting.
2026-04-05 18:48:03 -07:00
Broque Thomas
c3a3510c75 Add Live/Commentary Cleaner library maintenance job
New repair job that scans track and album titles for live performances,
commentary, interviews, skits, and spoken word content. Creates findings
for user review — no auto-fix.

Configurable per content type (live, commentary, interviews, spoken word),
with optional album title scanning and tracks/albums scope toggle.
Fix action removes track from DB + deletes file, cleans up empty albums
and directories. Follows existing repair job pattern exactly.
2026-04-05 00:27:55 -07:00
Broque Thomas
59587162cd Add metadata cache maintenance and health monitoring
Cache maintenance:
- Input validation rejects junk entities (Unknown Artist, empty names)
  from being cached, with exemptions for synthetic entries (_features,
  _tracks suffixes)
- CacheEvictorJob expanded to 4 phases: TTL eviction, junk cleanup,
  orphaned search cleanup, MusicBrainz failed lookup cleanup
- MusicBrainz null results now expire after 30 days (was 90) so failed
  lookups get retried sooner

Cache health UI:
- Polished modal accessible from Dashboard "Cache Health" button and
  repair dashboard health bar
- Shows health status banner (healthy/fair/poor), stat cards, source
  breakdown with colored progress bars, type pills, and metrics table
- Repair dashboard shows compact bar with health dot indicator
2026-03-30 07:40:20 -07:00
Broque Thomas
01b0b70515 Fix enhanced view reorganize not moving sidecars, add template debug logging
Enhanced album reorganize now moves LRC, cover.jpg, folder.jpg and other
sidecar files alongside audio files. Added debug log to library reorganize
repair job showing which template is loaded from config vs default.
2026-03-24 07:12:35 -07:00
Broque Thomas
f247841665 Fix Album Tag Consistency job: add missing _get_settings method 2026-03-24 06:38:05 -07:00