Commit graph

63 commits

Author SHA1 Message Date
Broque Thomas
ed97fecc31 Add Discography Backfill maintenance job
New repair job that scans each artist in the library, fetches their
full discography from metadata sources, and creates findings for any
tracks not already owned. Users review findings and click "Add to
Wishlist" to queue missing tracks for download.

Respects content filters (live/remix/acoustic/instrumental/compilation)
and release type filters (album/EP/single). Opt-in, disabled by default,
runs weekly, processes up to 50 artists per run with rate limiting.
2026-04-20 22:04:38 -07:00
Broque Thomas
c3f88a713a Fix unknown_artist_fixer crash on missing deezer_track_id column
The tracks table uses 'deezer_id' not 'deezer_track_id'. The query
and source field mapping both referenced the wrong column name.
2026-04-20 16:29:38 -07:00
Broque Thomas
0e8e3e86a0 Fix Duplicate Detector ignoring 'allow duplicate tracks across albums'
The Duplicate Detector repair job had its own ignore_cross_album setting
that was independent of the global allow_duplicate_tracks setting. When
a user enabled 'Allow duplicate tracks across albums', the detector
still flagged same-titled tracks on different albums as duplicates.
Now respects the global setting — if duplicates are allowed, cross-album
matches are always skipped.
2026-04-18 18:55:02 -07:00
Antti Kettunen
0e85931cc8 Fix track ID generation in repair flows
Repair-worker album fills now generate explicit track IDs when copying rows, instead of relying on SQLite auto-assignment that no longer exists for TEXT primary keys. The unknown-artist fixer now does the same for new artists.

Also add a regression test for the album-fill copy branch and keep the AcoustID scanner resilient to legacy null-ID rows.
2026-04-17 19:26:08 +03:00
Antti Kettunen
88e2527b96 Fix null-pointer error in acoustid_scanner
The root cause (null track ids) needs to be solved elsewhere, but this is a band-aid for now
2026-04-17 19:17:24 +03:00
Broque Thomas
223522ce99 Upgrade AcoustID scanner to scan full library with actionable fixes
Rewrote the AcoustID scanner job to scan all library tracks (via DB file
paths resolved to disk) instead of only the Transfer folder. Checkpoints
by track ID for robust resume across restarts. Defaults changed to
enabled, 24h interval, batch size 200.

Added _fix_acoustid_mismatch handler with three actions:
- retag: update DB title/artist to match actual audio content
- redownload: add expected track to wishlist and delete wrong file
- delete: remove wrong file and DB record

This catches cases like a file tagged as "Dinosaur Bones" that is
actually "Helicopters" — the scanner fingerprints the audio, detects
the mismatch, and the user can fix it from Library Maintenance findings.
2026-04-16 10:26:39 -07:00
Antti Kettunen
dab58766b7 Make library reorganize source-aware
Respect the configured metadata source order when looking up album years, and re-check provider availability during the scan so Spotify can drop out cleanly if it becomes rate-limited.
2026-04-15 21:35:58 +03:00
Antti Kettunen
03711c10df Make metadata gap filler source-aware
Only fetch source track details when ISRC enrichment is enabled, and show resolved source/track provenance in the UI.
2026-04-15 21:35:57 +03:00
Antti Kettunen
6df6ecb560 Respect source preference in cover art repair job
Cover art lookup now honors an explicit prefer_source first,
falls back to the runtime primary metadata source when unset,
and uses the shared source priority for the remaining fallbacks.
2026-04-15 21:28:12 +03:00
Antti Kettunen
e7faa9f02f Use metadata source priority in track number repair job
Use the shared metadata source priority when resolving album IDs,
album searches, and tracklists in track number repair.

Keeps Deezer and iTunes ahead of Spotify where configured, while
still allowing the job to fall back through other supported sources.
2026-04-15 21:28:12 +03:00
Antti Kettunen
6dca19ca1e Use metadata source priority in unknown artist fixer job
Unknown artist resolution now uses the shared metadata source priority and only filters to the sources that can actually participate in this job. Deezer and iTunes remain direct lookup sources, while Hydrabase can now join the title-search path when it is the configured priority source.
2026-04-15 21:28:10 +03:00
Broque Thomas
fe399636b2 Fix Spotify API calls leaking when Deezer/iTunes is primary source
Spotify was being called for album/artist data fetching across multiple
background workers and the Artists page search even when the user had
Deezer or iTunes set as their primary metadata source. Being authenticated
for playlist sync was treated as permission to use Spotify for everything.

- watchlist_scanner: add _spotify_is_primary_source() that checks both
  auth and primary source config; use it for all album/artist data fetching
  (discovery pool, recent album caching, playlist curation, similar artist
  ID matching, proactive ID backfill). _spotify_available_for_run() is kept
  for sync_spotify_library_cache which must run regardless of primary source
- repair_jobs/metadata_gap_filler: gate Spotify ISRC lookup on primary
  source being 'spotify'; MusicBrainz lookup unaffected
- repair_jobs/unknown_artist_fixer: replace hardcoded spotify_client with
  source-aware client selection — primary source ID tried first, each ID
  matched to its correct client (fixes latent bug passing Deezer IDs to
  Spotify)
- web_server.py /api/match/search: Artists page search was hardcoded to
  spotify_client.search_artists(); now uses _get_metadata_fallback_client()
  so results come from the configured primary source
2026-04-15 09:47:43 -07:00
BoulderBadgeDad
3618f3fa7f
Merge pull request #298 from kettui/fix/respect-metadata-provider-in-album-completeness-job
Honor primary metadata source in album_completeness job and associated repair flow
2026-04-15 07:15:15 -07:00
Antti Kettunen
106d202ccc Share metadata source priority
Centralize the ordered metadata source list and source-priority helper so album completeness and the repair worker follow the same Deezer/iTunes-first fallback order. This also removes the last duplicate priority logic from the touched repair paths.
2026-04-15 07:48:25 +03:00
Antti Kettunen
8a1ae00946 Utilize global Spotify client in repair jobs
Album completeness and any other repair job now uses the centralized source/client helpers instead of a worker-local Spotify client or override plumbing
  - This keeps source selection aligned with the configured primary provider and removes the last Spotify-only special case from the job path.

This change ultimately is a step towards further centralizing the Spotify client access and the associated `is_spotify_authenticated` check.
  - Currently these look-ups are done all over the place in different feature implementations directly, but moving forward, any feature that uses `get_primary_client` or `get_client_for_source` to access the Spotify client, won't have to duplicate any rate-limiting or auth checks as long as these getters are used
2026-04-15 07:18:34 +03:00
Broque Thomas
0edb8e93bb Fix MBID Mismatch Detector comparing MB title against DB title (filename) instead of embedded tag
The scan was reading `t.title` from the database for comparison, which is
populated from the filename rather than the file's embedded TITLE tag. Any
track whose filename differs from the clean MB title (e.g. includes artist,
album, bitrate) was incorrectly flagged as a mismatch.

Fix: extend `_read_mbid_from_file` → `_read_file_tags` to also read the
embedded TITLE tag (ID3 TIT2, Vorbis `title`, MP4 `©nam`). The comparison
now uses the embedded title, falling back to the DB title only when no TITLE
tag is present in the file.

Fixes #296
2026-04-14 12:19:41 -07:00
Antti Kettunen
1a459412a3 Honor primary metadata source in album_completeness job
Album completeness and downstream repair flow now follow the configured
primary provider first, with Discogs and Hydrabase support added alongside
existing Spotify, iTunes, and Deezer paths.

Keep spotify_track_id for compatibility while preserving source-aware track
IDs for provider-neutral handling.
2026-04-14 21:14:46 +03:00
Antti Kettunen
aec3047216 Improve graceful shutdown and rollback safety
- Add interruptible stop events to background workers so shutdown
  wakes out of long sleeps instead of waiting on fixed delays.
- Stop scan managers, repair worker, executors, and cleanup helpers
  deterministically so process exit does not leave background threads
  alive.
- Add startup warnings for stale SQLite WAL/SHM sidecars so unclean
  shutdowns are easier to spot before init/migration errors cascade.
- Prevent forced kills from leaving SQLite sidecars behind, which
  made rollbacks to older branches fail with malformed database
  errors.
2026-04-12 15:17:18 +03:00
Broque Thomas
805f72c5fd Add acappella to live/commentary cleaner patterns 2026-04-11 13:56:56 -07:00
Broque Thomas
5be6a46fb0 Fix dismissed findings reappearing and reduce false orphan detections
The finding dedup check only looked for 'pending' and 'resolved' status,
missing 'dismissed'. Dismissed findings were recreated as new entries on
every scan. Now includes 'dismissed' in the dedup check.

Orphan file detector improvements:
- Increased path suffix matching depth from 3 to 4 segments (covers
  Genre/Artist/Album/track.flac paths)
- Added filename-based fallback when Mutagen can't read file tags —
  parses title from "NN - Title [Quality].ext" pattern and matches
  against parent/grandparent folder names as artist
2026-04-10 17:42:39 -07:00
Broque Thomas
a6117d5174 Replace all legacy metadata_service imports with canonical functions
All callers of _create_fallback_client() and _get_configured_fallback_source()
now use get_primary_client() and get_primary_source() directly. No more
legacy alias usage anywhere in the codebase.
2026-04-10 12:52:36 -07:00
Broque Thomas
52a5d93018 Add Fix Unknown Artists maintenance job
New repair job that scans the library for tracks filed under "Unknown
Artist" and corrects them. Resolves correct metadata by:
1. Reading embedded file tags (if file has correct artist)
2. Looking up by source track ID (Spotify/Deezer/iTunes)
3. Searching by title as last resort

Dry run mode (default) creates findings for review. Live mode re-tags
the audio file, moves it to the correct folder structure, and updates
the database. Includes fix handler for applying individual findings.
2026-04-10 12:17:11 -07:00
Broque Thomas
7cfd1cae3f Fix AcoustID scanner creating thousands of no-match findings
The scanner was creating a finding for every file that couldn't be
identified by AcoustID, flooding the findings list with non-actionable
entries. Users saw the scanner "stuck scanning the same files over
and over" because the no-match findings were dismissed but recreated
on every run. Now only genuine mismatches (AcoustID identifies a
different track) create findings. Errors are counted and shown in
the job log with actual error messages for debugging.
2026-04-09 10:16:53 -07:00
Broque Thomas
ec87cb6d0e Skip zero-track albums in album completeness scanner
Albums with zero local tracks were flagged as incomplete but the
auto-fill fix failed because there were no existing tracks to
determine the album folder or quality standard from. Now skipped
during scan — they'll be detected once tracks are actually added.
2026-04-08 11:49:30 -07:00
Broque Thomas
1fb66b711d Fix Live/Commentary Cleaner missing _get_settings method 2026-04-06 07:45:07 -07:00
Broque Thomas
455945e6cb Fix duplicate detector missing cross-album duplicates (#252)
Changed ignore_cross_album default from True to False. Re-downloads of
the same song create separate album entries, so the detector was skipping
them. Users who want to keep compilations/greatest-hits intact can toggle
it back on. Updated help text to explain when to use this setting.
2026-04-05 18:48:03 -07:00
Broque Thomas
c3a3510c75 Add Live/Commentary Cleaner library maintenance job
New repair job that scans track and album titles for live performances,
commentary, interviews, skits, and spoken word content. Creates findings
for user review — no auto-fix.

Configurable per content type (live, commentary, interviews, spoken word),
with optional album title scanning and tracks/albums scope toggle.
Fix action removes track from DB + deletes file, cleans up empty albums
and directories. Follows existing repair job pattern exactly.
2026-04-05 00:27:55 -07:00
Broque Thomas
59587162cd Add metadata cache maintenance and health monitoring
Cache maintenance:
- Input validation rejects junk entities (Unknown Artist, empty names)
  from being cached, with exemptions for synthetic entries (_features,
  _tracks suffixes)
- CacheEvictorJob expanded to 4 phases: TTL eviction, junk cleanup,
  orphaned search cleanup, MusicBrainz failed lookup cleanup
- MusicBrainz null results now expire after 30 days (was 90) so failed
  lookups get retried sooner

Cache health UI:
- Polished modal accessible from Dashboard "Cache Health" button and
  repair dashboard health bar
- Shows health status banner (healthy/fair/poor), stat cards, source
  breakdown with colored progress bars, type pills, and metrics table
- Repair dashboard shows compact bar with health dot indicator
2026-03-30 07:40:20 -07:00
Broque Thomas
01b0b70515 Fix enhanced view reorganize not moving sidecars, add template debug logging
Enhanced album reorganize now moves LRC, cover.jpg, folder.jpg and other
sidecar files alongside audio files. Added debug log to library reorganize
repair job showing which template is loaded from config vs default.
2026-03-24 07:12:35 -07:00
Broque Thomas
f247841665 Fix Album Tag Consistency job: add missing _get_settings method 2026-03-24 06:38:05 -07:00
Broque Thomas
d75893bc30 Add Album Tag Consistency repair job: detect and fix inconsistent tags across album tracks
New maintenance job scans albums for tracks with mismatched album names, album
artist names, or MusicBrainz release IDs. These inconsistencies cause Navidrome
and other media servers to split one album into multiple entries. The fix
normalizes outlier tracks to the majority value by rewriting file tags.
2026-03-23 10:44:36 -07:00
Broque Thomas
429306c7f3 Fix enrichment retry loops, cover art finding dupes, and Spotify rate limit during art scan
- All 9 enrichment workers: stop auto-retrying 'error' status items (was infinite loop)
  Only 'not_found' items retry after configured days; errors require manual full refresh
- Cover art dedup: check both 'pending' AND 'resolved' findings to prevent recreation
- Cover art scanner: top-level Spotify rate limit check skips Spotify entirely when
  banned, falls back to iTunes/Deezer only, logs once instead of spamming 429s
2026-03-22 23:24:42 -07:00
Broque Thomas
bd27bbe1b2 Add independent Blasphemy Mode setting for lossy converter job
The lossy converter fix handler now reads delete_original from its
own job settings (repair.jobs.lossy_converter.settings.delete_original)
instead of the global lossy_copy.delete_original. Defaults to false.
Separate from the per-download Blasphemy Mode toggle in Settings.
2026-03-22 08:08:47 -07:00
Broque Thomas
adefe1c892 Register lossy_converter in repair jobs module list
Job was not appearing in Library Maintenance because the module
was not in the _JOB_MODULES import list.
2026-03-22 07:58:47 -07:00
Broque Thomas
bc5dd75c8e Add lossy converter repair job for retroactive FLAC conversion
New library maintenance job that scans for FLAC files missing a lossy
copy (MP3/Opus/AAC) and creates findings. Fix action converts via
ffmpeg using the configured codec/bitrate from Settings. Supports
Blasphemy Mode (delete original + update DB path). Finding details
store codec/bitrate from scan time for consistency. Disabled by
default, manual-run only, no auto-schedule.
2026-03-22 07:43:19 -07:00
Broque Thomas
4dba3757be Fix orphan detector false positives and add staging/delete choice
Orphan detector: add normalized tag matching that strips parentheticals
and brackets (feat. X, [FLAC 16bit], etc.) and tries first-artist-only
for comma-separated artists. Prevents false orphan flags for tracks
like "The Mountain (feat. Dennis Hopper...)" that exist in DB as
"The Mountain". All lookups remain O(1) set operations.

Orphan fix: replace auto-delete with user choice prompt. Single Fix
and Fix All both show modal asking "Move to Staging" or "Delete".
Move to Staging relocates file to import staging folder for proper
re-import with metadata matching. Fix action flows through API
endpoint → repair_worker.fix_finding → _fix_orphan_file handler.
Staging path uses docker_resolve_path for container compatibility.
2026-03-21 11:27:51 -07:00
Broque Thomas
c69040886e Collect release year from all metadata sources during post-processing
Post-processing now extracts release year from MusicBrainz, Deezer,
Tidal, Qobuz, and Spotify context (first source wins). Writes
ORIGINALDATE and DATE tags to file, and backfills the album year
in the DB if currently missing. Fixes Library Reorganize showing
blank years for Tidal-only downloads.

Also raises Library Reorganize API year lookup cap from 50 to 200.
2026-03-20 17:31:00 -07:00
Broque Thomas
81da57f306 Add min completion percentage filter to Album Completeness job
New setting: Min Completion % (default 0 = disabled). Skips albums
where the user has fewer than N% of tracks — filters out playlist
imports where a single track exists but the full album was never
intended to be downloaded. Catches real failed downloads (6/12)
while ignoring incidental singles (1/12).
2026-03-20 08:19:56 -07:00
Broque Thomas
dadc489b08 Add tag-based fallback to orphan detector to prevent false positives
When suffix path matching fails, reads file tags (title+artist) and
checks against DB tracks. Prevents false orphan detection from path
mismatches in Docker where DB paths differ from filesystem paths.
Only runs for files that fail suffix matching — zero overhead in
normal cases.
2026-03-19 22:01:27 -07:00
Broque Thomas
1b95c05041 Fix Track Number Repair findings returning 400 on fix
Finding was created with entity_id=None (file-based) but fix handler
required entity_id for DB update. Rewrote handler to work with file_path
as primary — writes corrected track number to file tags, renames file
if needed, updates DB path. Also stores total_tracks in finding details
for correct tag writing.
2026-03-19 19:21:24 -07:00
Broque Thomas
f0270ce7a5 Expand Album Completeness to support iTunes and Deezer sources
Was Spotify-only — users without Spotify got zero results. Now queries
albums with any source ID (spotify_album_id, itunes_album_id, deezer_id)
and uses the matching API client for track count and missing track lookup.
Falls back gracefully across sources with client-type detection.
2026-03-19 11:27:08 -07:00
Broque Thomas
e2345c659d Add mass orphan safety guard to prevent accidental library deletion
When >50% of files are flagged as orphans (likely a DB path mismatch),
findings are marked as warnings with mass_orphan flag. Fixing these
requires typing "witness me" to confirm — prevents nuking an entire
library from a false-positive orphan scan.
2026-03-19 11:04:11 -07:00
Broque Thomas
ade189fa38 Fix Library Reorganize producing (_) in paths when year is empty
_sanitize_context_values passed empty strings through _sanitize_filename
which converts '' to '_' (empty-name fallback). Template $album ($year)
became Album (_) instead of Album () which the cleanup regex couldn't
match. Now preserves empty strings so the existing () cleanup works.
2026-03-19 10:36:21 -07:00
Broque Thomas
4eb028a7ce Add fix handler for Library Reorganize findings and expand year sources
- Add _fix_path_mismatch handler so fixing Library Reorganize dry-run
  findings actually moves files (was returning 400 with no handler)
- Add path_mismatch to fixable_types and _execute_fix handler map
- Add recent_releases and wishlist_tracks as year sources in
  _load_album_years to cover more playlist-synced tracks missing years
- Add sys and json imports needed by new code
2026-03-19 09:24:09 -07:00
Broque Thomas
0de8841b14 Fix bulk Fix All ignoring Single/Album Dedup findings and expand version keywords
- Add single_album_redundant to fixable_types in bulk_fix_findings so
  Fix All actually includes these findings (Fix Selected worked, Fix All
  silently returned 0)
- Expand version keyword regex from 9 to 25 terms (remastered, deluxe,
  unplugged, etc.) to reduce false positives in Single/Album Dedup
- Add word boundary anchors to prevent substring matches (e.g. "live"
  inside "Alive", "edit" inside "Meditate")
- Cast similarity thresholds to float for config type safety
2026-03-19 08:53:14 -07:00
Broque Thomas
8754e160b0 Fix Album Completeness job scanning no albums due to wrong HAVING filter
The SQL HAVING clause filtered on local track count instead of expected
track count, excluding albums with fewer than 3 local tracks from the
scan entirely. Now fetches all Spotify-matched albums and filters by
expected track count in the loop.
2026-03-19 08:36:15 -07:00
Broque Thomas
0dd0d50837 Fix Single/Album Dedup flagging EP tracks as redundant singles
EPs are no longer classified as singles — removing EP tracks would
break the release and conflict with album completeness checks.
Only actual singles (album_type='single' or unknown type with <=2
tracks) are flagged as redundant when the same song exists on an album.
2026-03-18 17:48:01 -07:00
Broque Thomas
e1b203405f Add ignore cross-album duplicates setting to Duplicate Detector
Tracks on different albums (e.g., same song on a studio album and a
compilation) are no longer flagged as duplicates by default, keeping
albums complete. Setting is enabled by default and configurable via
the job settings UI.
2026-03-18 17:42:49 -07:00
Broque Thomas
ebc0713348 Add API year lookup fallback for Library Reorganize job
Playlist-synced tracks often have no year in file tags or the DB albums
table. Now checks discovery_pool release_date as a second DB source,
then batch-lookups remaining missing years from the user's active
metadata source (Spotify/iTunes/Deezer) capped at 50 API calls.
2026-03-18 14:49:35 -07:00
Broque Thomas
8abcf386d5 Fix duplicate detector ignoring similarity settings for remixes
Normalization was stripping parenthetical content before comparison,
so 'title' and 'title (xxx remix)' both became 'title' and always
matched at 1.0 regardless of the user's threshold setting.
2026-03-18 12:47:34 -07:00