Commit graph

19 commits

Author SHA1 Message Date
BoulderBadgeDad
ee4d514d60 CI: silence S110 on three intentional best-effort swallows (unblocks dev build)
The dev-nightly build runs `ruff check .` before "Build and push to GHCR" in the
same job, so the three S110 (try/except/pass) errors introduced since the last
green build (ce6ce4d) failed the lint step and SKIPPED the image push entirely —
every dev-nightly since #704 went red, so the dev image was never rebuilt and none
of the recent fixes (incl. the #852 WebSocket login-bypass fix) ever shipped to
the image users pull.

All three are deliberate best-effort swallows; annotate them with the repo's
existing `# noqa: S110 — <reason>` convention rather than adding dead logging:
- relocate.py: tag write is best-effort (re-import re-derives tags)
- acoustid_scanner.py: verification-status tag is optional context
- web_server.py: audio-duration probe falls through to 0

ruff check . + compileall now clean; pytest already passed in CI at ce6ce4d.
2026-06-11 16:00:07 -07:00
dev
37ea6604c7 Fix import artist override and verification review 2026-06-11 01:28:31 +02:00
dev
97b40cbd43 feat(verification): review queue — listen/compare/approve/delete unverified downloads
- ⚠ Unverified filter rows gain actions: inline play (range-streamed from the
  history file path, server-side only), YouTube compare, Approve -> new
  human_verified status (tag + history + tracks; AcoustID scanner skips these
  entirely), Delete (file + entry)
- API: /api/verification/<id>/stream|approve|delete (path only from DB row)
- backfill: history rows with acoustid_result='fail' that exist at all were
  imported despite the failure = force_imported (covers pre-fix fallback
  imports like the user's 'My Ordinary Life')

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 01:28:31 +02:00
dev
9d1d09a571 feat(verification): persist status (db+tag), surface on Downloads, scan-aware force-imports
- import pipeline writes SOULSYNC_VERIFICATION tag + context status
  (verified / unverified / force_imported via version-mismatch fallback)
- downloads payload + UI badge (tooltip explains each state)
- AcoustID scan reads the tag: refreshes tracks.verification_status,
  reports force-imported mismatches as informational (clearly marked),
  optional skip via job setting skip_force_imported
- evaluate(): empty expected artist = title-only comparison (old scanner
  behaviour); thresholds single-sourced in the core

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 01:28:31 +02:00
dev
b981230d07 refactor(scanner): use shared verification core; stop false-flagging cross-script
The library AcoustID scan now calls audio_verification.evaluate() (alias-aware
artist match + cross-script SKIP) instead of its own non-ASCII-stripping
_normalize and threshold logic, so it no longer false-flags correct anime-OST /
kanji tracks. Duration-collision guard kept as a scanner pre-check on the top
recording. evaluate() is now purely a title/artist/version/cross-script decision.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 01:27:28 +02:00
Broque Thomas
9cc09118bf AcoustID scanner: multi-candidate match + duration guard + multi-value retag
Closes #587. Three coordinated fixes per codex's diagnosis. AcoustID
verification gate left intact — these fixes target the upstream
scanner false-positive surface plus a separate retag-path gap.

Bug 1 — scanner used recordings[0] as authoritative

`core/repair_jobs/acoustid_scanner.py:_scan_file` only checked the
top fingerprint match's metadata. AcoustID often returns multiple
recordings per fingerprint (sample collisions, multi-MB-record
cases) and the wrong-credited recording can outrank the right-
credited one. Foxxify case 2 (Nana / Nana): top match credited the
wrong artist while a lower-ranked candidate matched the user's
expected metadata exactly.

Lifted the verifier's all-candidates check to a shared pure helper
`core/matching/acoustid_candidates.py:find_matching_recording`. Both
verifier and scanner can now ask "given these candidates, does ANY
of them match expected (title, artist)?" with the same contract.
Scanner suppresses the finding when any candidate matches.

Bug 2 — no duration check guards against fingerprint hash collisions

Foxxify case 3: 17-minute mashup edit fingerprinted to a 5-minute
late-70s Japanese hiphop track (different songs, fingerprint hash
collision on a sampled section). Scanner had no signal to detect
this and would have recommended retagging the 17-min file as the
5-min track.

`duration_mismatches_strongly` in the same helper module flags drifts
beyond max(60s, 35%). Scanner now skips findings when the candidate's
duration disagrees strongly with the file's expected duration. Loaded
duration via the existing tracks SQL (added `t.duration` to the
SELECT). Returns False when either side is unknown — no behavior
change for older rows without duration data.

Bug 3 — scanner retag bypassed multi-value ARTISTS tag setting

`core/repair_worker.py:_fix_wrong_song` called `write_tags_to_file`
with single-string artist updates. The writer only wrote TPE1
(single string) and never read the user's
`metadata_enhancement.tags.write_multi_artist` config. Multi-value
ARTISTS tags got stripped on every retag, contradicting the
post-download enrichment pipeline's behavior.

Per codex's pick (option B over routing through enhance_file_metadata),
extended `write_tags_to_file` with an optional `artists_list`
parameter. Each format-specific writer respects the config flag the
same way enrichment.py does:
- ID3: TPE1 stays as joined display string + TXXX:Artists multi-value
- Vorbis/Opus/FLAC: `artist` display string + `artists` multi-value key
- MP4: \xa9ART as list when on, single string when off

Scanner retag derives the per-artist list by splitting AcoustID's
credit through the existing `split_artist_credit` helper (same
separators the matching layer already uses).

Backward compatible: callers that don't pass `artists_list` get the
exact same single-string write as before. No regression for the
write_artist_image button or any other tag_writer caller.

15 tests on the candidate helper + duration guard.
13 tests on the tag_writer multi-value path (write/skip/single/
no-list cases for FLAC + the config-gate helper).
4 new scanner regression tests pinning lower-ranked candidate
suppression, no-suppression when no candidate matches, duration
mismatch skip, no-skip when duration matches.

Existing scanner tests updated for the new 11-column SQL select
(added duration column to fake schema + test row tuples).

Full suite: 3097 passed. Ruff clean.
2026-05-14 14:09:38 -07:00
Broque Thomas
7a23d60f28 AcoustID scanner: file-tag fallback for legacy compilation tracks
Follow-up to the prior compilation-album scanner fix. That patch
made the scanner read `tracks.track_artist` (per-track artist
column) via COALESCE so compilation tracks would compare against
the right value. But tracks downloaded BEFORE the `track_artist`
column existed have track_artist=NULL — COALESCE falls back to
album artist (the curator) and the wrong-comparison case returns.

Fix: explicit 3-tier resolution in `_scan_file`:

  1. DB `tracks.track_artist` if populated → trust it. Respects
     manual edits from the enhanced library view (user who curated
     the DB value but didn't re-tag the file gets their edit
     respected, not overridden by stale file tag).

  2. File's ARTIST tag via mutagen if present → use it. Tidal /
     Spotify / Deezer all write the per-track artist into the
     audio file at download time regardless of SoulSync's DB
     schema, so it's ground truth even when the DB column is
     stale or NULL. File is already open for fingerprinting so
     mutagen tag-read is essentially free.

  3. Album artist → final fallback for files without proper ARTIST
     tags AND no DB track_artist. Existing pre-fix behavior.

`_load_db_tracks` SELECT now surfaces `track_artist` (raw, may be
empty/NULL via NULLIF) and `album_artist` separately in addition
to the COALESCE'd `artist` field — so `_scan_file` can tell the
difference between 'DB has a curated value' and 'DB fell back to
album artist'. Without this distinction, the file-tag fallback
would create false positives when DB is curated but file is stale.

5 new tests (11 total in the file) pin:

  - File-tag-trumps-DB resolves the legacy NULL case (DB says
    'Andromedik' (album curator), file says 'Eclypse', AcoustID
    says 'Eclypse' → no flag)
  - Tag-missing falls back to album artist (preserves existing
    genuine-mismatch contract — file without tag + AcoustID
    mismatch still flags)
  - Mutagen exception swallowed (debug log, fall-through)
  - File-tag matches DB → no behavioral change
  - DB curated value trumps stale file tag (false-positive guard
    — user edited DB without re-tagging file shouldn't get flagged)

Two existing test fixtures (`_make_context` callers) updated to
the new 10-column row shape.

SQL behavior verified empirically against real SQLite: NULL and
empty-string both flow through NULLIF → None in Python →
file-tag-fallback path. Modern populated values trump file tag.
2026-05-11 11:39:19 -07:00
Broque Thomas
812db1fbbf AcoustID scanner: prefer track_artist for compilation albums
Discord report (Skowl): downloaded a compilation album ("High Tea
Music: Vol 1") where every track has a different artist (Eclypse,
Andromedik, T & Sugah, Gourski, etc.) and the AcoustID scanner
flagged every single track as Wrong Song. The file tags had the
correct per-track artist (e.g. "Eclypse" for "City Lights"), but
the scanner compared against the album-level artist ("Andromedik",
the curator). Raw similarity 12% → Wrong Song flag.

# Why the prior multi-value fix didn't help

Foxxify's case (just-merged PR): AcoustID returned multi-value
credit "Okayracer, aldrch & poptropicaslutz!" — primary IS in the
credit. Splitting found it.

Skowl's case: both sides single-value but DIFFERENT artists.
Splitter has nothing to find — Eclypse simply isn't in "Andromedik".
Different bug.

# Cause

Scanner SQL at `core/repair_jobs/acoustid_scanner.py:281` joined
the `artists` table via `tracks.artist_id` which points at the
ALBUM artist (the curator/label-name applied to every row in a
compilation). The `tracks.track_artist` column already holds the
correct per-track artist for compilations — populated by every
server-scan path (Plex `originalTitle`, Jellyfin `ArtistItems`,
Navidrome per-track `artist`) AND the auto-import / direct-download
post-process flow (`record_soulsync_library_entry` writes it when
different from album artist). Scanner just wasn't reading it.

# Fix

```sql
SELECT t.id, t.title,
       COALESCE(NULLIF(t.track_artist, ''), ar.name) AS artist,
       ...
```

Prefers per-track artist when populated, falls back to album artist
for legacy rows / single-artist albums where `track_artist` is NULL.
`NULLIF(t.track_artist, '')` handles the empty-string-instead-of-null
case some legacy rows might have.

# Composes with Foxxify's multi-value fix

For the rare compilation track where AcoustID ALSO returns a
multi-value credit (e.g. compilation track has multiple credited
performers), both paths work together — `track_artist` gives the
correct expected primary, then the helper splits the credit and
finds it.

# Tests added (2)

- `test_load_db_tracks_prefers_track_artist_for_compilation` —
  reporter's exact case: track with `track_artist='Eclypse'` AND
  `artist_id` pointing at album artist 'Andromedik' resolves to
  'Eclypse'. Second track with NULL `track_artist` falls back to
  album artist 'Andromedik' (single-artist + legacy compat).
- `test_load_db_tracks_falls_back_when_track_artist_empty_string`
  — empty string in `track_artist` (some legacy rows) → NULLIF
  returns NULL → COALESCE falls back to album artist.

Both use a real SQLite DB so the COALESCE/NULLIF logic + JOIN
runs against actual schema (SimpleNamespace fakes can't simulate
JOINs).

# Verification

- 6/6 scanner tests pass (2 new + 4 existing)
- 2586 full suite passes (+2 from prior commit)
- Ruff clean
2026-05-10 19:44:57 -07:00
Broque Thomas
df304eb016 AcoustID scanner: handle multi-value artist credits
Discord report (Foxxify): the AcoustID scanner repair job flagged
multi-artist tracks as Wrong Song because AcoustID returns the
FULL credit ("Okayracer, aldrch & poptropicaslutz!") while the
library DB carries only the primary artist ("Okayracer"). Raw
SequenceMatcher similarity scored ~43% — well below the 60%
threshold — so the scanner created a finding even though the
audio was correct. User couldn't fix without lowering the global
artist threshold to ~30% (which would let real mismatches through).

# Fix

Extended the shared `core/matching/artist_aliases.py::artist_names_match`
helper (originally lifted for #441) with credit-token splitting.
When the actual artist string contains common separators —

- punctuation: `,`  `&`  `;`  `/`  `+`
- keywords (whitespace-bounded): `feat.` `ft.` `featuring` `with`
  `vs.` `x`

— the helper splits into individual contributors and checks each
against the expected artist. Primary-in-credit cases now resolve
at 100% instead of 43%.

Two pattern groups because punctuation separators don't need
surrounding whitespace, but keyword separators MUST be
whitespace-bounded — otherwise we'd split artists with `x` /
`with` etc. in their names ("JAY-X" → "JAY-" / "" issue).

Composes with the existing alias path: cross-script multi-artist
credits ("Hiroyuki Sawano" expected, "澤野弘之, FeaturedJp"
actual) work via alias-token-against-credit-token compare.

# Wire-in

Scanner at `core/repair_jobs/acoustid_scanner.py:202` replaces
the raw `SequenceMatcher` call with `artist_names_match`. Pass
RAW artist strings (not pre-normalised by `_normalize`) so the
splitter can recognise separators — `_normalize` strips ALL
punctuation, which destroyed the very tokens the splitter needs.

The AcoustID post-download verifier (`core/acoustid_verification.py`)
already routes through `_alias_aware_artist_sim` which calls the
same helper — gets the multi-value benefit automatically without
a separate wire-in.

# New `split_artist_credit` exported helper

Pure-function helper for callers who want token-level access to
the credit list (debugging, UI, future per-token enrichment). Same
splitter logic, exposed as a top-level function.

# Tests added (14)

`tests/matching/test_artist_aliases.py` (+11):
- `TestSplitArtistCredit` — parametrised across 12 credit-string
  formats (comma, ampersand, semicolon, slash, plus, feat./ft./
  featuring, with, vs., x, single-token, empty), drops empty
  tokens, strips per-token whitespace
- `TestMultiValueCreditMatching` — reporter's exact case
  (Okayracer in 3-artist credit → 100%), primary in middle/end of
  credit, genuine-mismatch still fails, single-token actual falls
  through to direct compare, multi-value composes with aliases,
  threshold still respected

`tests/test_acoustid_scanner.py` (+3):
- Reporter's case end-to-end through `_scan_file` — fingerprint
  99% / title 100% / multi-artist credit → no finding created
- Genuine artist mismatch still creates finding (no false
  suppression of real mismatches)
- `JobResultStub` minimal scaffold for the integration tests

# Verification

- 14 new tests pass (49 helper + 5 scanner total in their files)
- 110 matching + scanner tests pass total
- 2584 full suite passes (+25 from baseline 2559)
- Ruff clean
- Reporter's exact case (Okayracer in `Okayracer, aldrch &
  poptropicaslutz!`) now scores 100% match → no Wrong Song flag
2026-05-10 19:17:59 -07:00
Broque Thomas
cf5461f2f1 Fix: maintenance findings badge inflated when scan dedup-skipped
`_create_finding` silently dedup-skipped re-discovered issues but
the caller incremented `findings_created` regardless. So a re-scan
that found the same issues as a prior scan reported 364 findings
in the badge while 0 NEW pending rows hit the db, leaving the
findings tab empty.

`_create_finding` now returns bool (True on insert, False on
dedup-skip / db error). All 16 repair jobs updated to only
increment `findings_created` on True. Added `findings_skipped_dedup`
counter surfaced in scan log: "Done: X scanned, 0 fixed, 0
findings (363 already existed), 0 errors".

Also fixed a missing `job_id` kwarg in album_tag_consistency that
was silently breaking finding creation for that scan.
2026-05-04 08:55:13 -07:00
Broque Thomas
d8437c87c6 Fix Album Completeness Auto-Fill on Docker / shared-library setups (#476)
GitHub issue #476 (gabistek, Docker on Arch host): "Auto-Fill" / "Fix
Selected" on the Album Completeness findings page returned
"Could not determine album folder from existing tracks" for every album.
Reproduces on any setup where the media-server library lives outside the
SoulSync transfer/download folders — Docker is the headline case but
native installs that point Plex at a NAS via SMB hit it too.

Root cause: `core/repair_worker.py:_resolve_file_path` only probed the
transfer + download folders. Docker users have their Plex/Jellyfin
library bind-mounted at /music (or similar) — neither configured in
SoulSync. Every existing track got silently treated as missing, so
`album_folder` stayed None and the fix workflow bailed.

The same incomplete logic was duplicated four more times in the
repair_jobs/ modules, all with the same bug. Album Completeness was
just the most user-visible — the same setups were also producing false
"missing file" findings from Dead File Cleaner, silent skips in
MBID Mismatch Detector, etc.

The web server already had the correct logic at
`web_server.py:_resolve_library_file_path` (probes transfer + download
+ Plex-reported library locations + user-configured library.music_paths).
The repair workers had never been updated to match.

Fix:
- New `core/library/path_resolver.py` extracts the union logic into a
  single shared function `resolve_library_file_path()`. Probes (in
  order, deduped): explicit transfer/download kwargs, config-derived
  soulseek.transfer_path/download_path, Plex-reported library
  locations (when a plex_client is passed), user-configured
  library.music_paths. Each defensive: malformed config or a flaky
  Plex client degrades to the dirs that did succeed.
- `core/repair_worker.py:_resolve_file_path` becomes a delegating
  wrapper preserving the legacy signature, with a new `config_manager`
  kwarg. All 15 in-tree call sites updated to thread
  `self._config_manager` through.
- `core/repair_jobs/dead_file_cleaner.py`,
  `mbid_mismatch_detector.py`, and `lossy_converter.py` get the same
  treatment: duplicate function replaced with a thin wrapper, call
  sites pass `context.config_manager`.
- `core/repair_jobs/acoustid_scanner.py` and
  `unknown_artist_fixer.py` (which used to import from repair_worker)
  now call the shared resolver directly with `context.config_manager`.

Side benefit: every other repair job (Dead File Cleaner, MBID
Mismatch Detector, Lossy Converter, AcoustID Scanner, Unknown Artist
Fixer) also stops missing files in the media-server library mount.
Single fix unblocks five user-visible features.

Tests: `tests/library/test_path_resolver.py` — 20 cases covering all
four base-dir sources, suffix-walk algorithm, dedup, defensive paths
(None plex client, malformed config entries, raising config_manager.get,
broken plex attribute access), Docker path translation. Full suite
1677 passed locally.

WHATS_NEW entry under '2.4.2' dev cycle.
2026-05-03 10:11:06 -07:00
Antti Kettunen
88e2527b96 Fix null-pointer error in acoustid_scanner
The root cause (null track ids) needs to be solved elsewhere, but this is a band-aid for now
2026-04-17 19:17:24 +03:00
Broque Thomas
223522ce99 Upgrade AcoustID scanner to scan full library with actionable fixes
Rewrote the AcoustID scanner job to scan all library tracks (via DB file
paths resolved to disk) instead of only the Transfer folder. Checkpoints
by track ID for robust resume across restarts. Defaults changed to
enabled, 24h interval, batch size 200.

Added _fix_acoustid_mismatch handler with three actions:
- retag: update DB title/artist to match actual audio content
- redownload: add expected track to wishlist and delete wrong file
- delete: remove wrong file and DB record

This catches cases like a file tagged as "Dinosaur Bones" that is
actually "Helicopters" — the scanner fingerprints the audio, detects
the mismatch, and the user can fix it from Library Maintenance findings.
2026-04-16 10:26:39 -07:00
Antti Kettunen
aec3047216 Improve graceful shutdown and rollback safety
- Add interruptible stop events to background workers so shutdown
  wakes out of long sleeps instead of waiting on fixed delays.
- Stop scan managers, repair worker, executors, and cleanup helpers
  deterministically so process exit does not leave background threads
  alive.
- Add startup warnings for stale SQLite WAL/SHM sidecars so unclean
  shutdowns are easier to spot before init/migration errors cascade.
- Prevent forced kills from leaving SQLite sidecars behind, which
  made rollbacks to older branches fail with malformed database
  errors.
2026-04-12 15:17:18 +03:00
Broque Thomas
7cfd1cae3f Fix AcoustID scanner creating thousands of no-match findings
The scanner was creating a finding for every file that couldn't be
identified by AcoustID, flooding the findings list with non-actionable
entries. Users saw the scanner "stuck scanning the same files over
and over" because the no-match findings were dismissed but recreated
on every run. Now only genuine mismatches (AcoustID identifies a
different track) create findings. Errors are counted and shown in
the job log with actual error messages for debugging.
2026-04-09 10:16:53 -07:00
Broque Thomas
ab21855af3 Enrich repair findings with album art, artist images & live job progress
- All 9 repair jobs now emit report_progress() for real-time card updates
  (phase, log lines, per-item activity) via WebSocket repair:progress events
- Enrich finding details with album/artist thumb URLs across all repair jobs
  (dead_file, duplicate, metadata_gap, album_completeness, missing_cover_art,
  acoustid_scanner, track_number_repair, fake_lossless, orphan_file)
- Track number repair: return match_score from fuzzy matching, add suffix-based
  DB lookup for album/artist art (handles cross-environment path mismatches)
- Fix Plex/Jellyfin relative thumb URLs in findings endpoint via fix_artist_image_url
- Labeled media cards in finding detail panels (album title + artist name under images)
- Dashboard tooltip shows current job name + per-job progress instead of stale stats
2026-03-15 14:08:26 -07:00
Broque Thomas
4635ea895b include a '?' to include more details for each job. 2026-03-15 11:19:09 -07:00
Broque Thomas
363b3d0922 Fix MusicBrainz API call in metadata gap filler job 2026-03-14 09:46:02 -07:00
Broque Thomas
945f86c643 Library Repair Worker: multi-job background maintenance daemon with 10 jobs, findings system, and management modal 2026-03-14 09:09:18 -07:00