soulsync/core/metadata/album_mbid_cache.py
Broque Thomas 4b15fe0b75 Fix album MBID inconsistency: detector + persistent release-MBID cache
Discord report (Samuel [KC]): tracks of the same album sometimes carry
different MUSICBRAINZ_ALBUMID tags, which causes Navidrome (and other
media servers grouping by album MBID) to split the album into multiple
entries. Two-part fix — one for existing libraries, one for the root
cause that lets new imports drift.

Part 1 — Detector + fix action (catches existing dissenters):

`core/repair_jobs/mbid_mismatch_detector.py`:
- New helpers: `_read_album_mbid_from_file` and
  `_write_album_mbid_to_file` use the Picard-standard tag conventions
  (`TXXX:MusicBrainz Album Id` for MP3, `MUSICBRAINZ_ALBUMID` for
  FLAC/OGG, `----:com.apple.iTunes:MusicBrainz Album Id` for MP4).
- New scan phase `_scan_album_mbid_consistency` runs after the
  existing track-MBID scan: groups tracks by DB `album_id`, reads
  each track's embedded album MBID, finds the consensus
  (most-common) MBID via `Counter`, flags dissenters. Tracks without
  an album MBID at all are skipped (they don't break Navidrome —
  only an explicit MBID disagreement does). Albums where MBIDs are
  perfectly tied (no clear consensus) are skipped too — surface as
  a manual decision instead of fixing toward a 1/N tie.
- New finding type `album_mbid_mismatch` carries `consensus_mbid`,
  `wrong_mbid`, `consensus_count`, `total_tracks_with_mbid`, and a
  human-readable reason string.

`core/repair_worker.py`:
- Added `'album_mbid_mismatch': self._fix_album_mbid_mismatch` to the
  fix dispatch dict and to the `fixable_types` tuple so auto-fix +
  bulk-fix paths pick it up.
- New `_fix_album_mbid_mismatch` method reads `consensus_mbid` from
  finding details, resolves the dissenter's file path via the shared
  library resolver, calls `_write_album_mbid_to_file` to rewrite the
  tag in place. Doesn't touch the album's other tracks (they're
  already in agreement).

Part 2 — Root cause fix (prevents new SoulSync imports from drifting):

The original in-memory `mb_release_cache` in `core/metadata/source.py`
maps `(normalized_album, artist) -> release_mbid` so per-track
enrichment of the same album hits the cache and writes the same
MUSICBRAINZ_ALBUMID to every track. That cache is bounded (4096
entries) and in-process — so cache eviction (when other albums are
processed in between) and server restart can BOTH cause
inconsistency. Per-track album-name variation (e.g. some tracks
tagged `"Album"`, others tagged `"Album (Deluxe)"`) and per-track
artist variation (features) make it worse.

`core/metadata/album_mbid_cache.py` (new module):
- DB-backed `lookup(normalized_album, artist) -> release_mbid` and
  `record(...)` functions. Same key shape as the in-memory cache.
- Strict additive design: every public function is wrapped in
  try/except and degrades to None / no-op on ANY database error.
  The existing in-memory cache + MusicBrainz lookup remains the
  authoritative fallback. If this module breaks, downloads continue
  exactly as they would today.

`database/music_database.py`:
- New `mb_album_release_cache` table with composite primary key
  `(normalized_album_key, artist_key)`. Reverse-lookup index on
  `release_mbid` for future debug tooling. Created via the existing
  `CREATE TABLE IF NOT EXISTS` migration pattern — idempotent, no
  schema version bump needed.

`core/metadata/source.py`:
- Surgical change inside the existing `embed_source_ids`
  in-memory-cache-miss branch: BEFORE calling MusicBrainz, consult
  the persistent cache. If a previous SoulSync run already resolved
  this album's release MBID, reuse it. After a successful MB lookup,
  store in BOTH caches. Both calls wrapped in defensive try/except
  so any failure falls through to existing logic.

Tests:
- `tests/metadata/test_album_mbid_cache.py` — 16 cache tests:
  round-trip, idempotent re-record, overwrite semantics, clear_all,
  album+artist independence (no Greatest Hits collisions),
  defensive None-on-empty-input, graceful degradation when the DB
  is unavailable / connection raises / commit fails, schema sanity
  (table + index exist after init).
- `tests/test_album_mbid_consistency.py` — 13 detector tests:
  tag read/write round-trip on real FLAC files, Picard-standard tag
  descriptors, defensive paths (unreadable file, empty input),
  detector behavior (agreement → no flags, lone dissenter → flag,
  ties → no flag, single-track albums → skipped, no-MBID tracks →
  skipped, unresolvable file paths → skipped).
- `tests/metadata/test_metadata_enrichment.py` — added autouse
  fixture monkeypatching the persistent cache to no-op for tests in
  this file. The existing tests pin per-call MB counts and
  in-memory cache state; without the fixture, persistent rows from
  earlier tests would bypass the MB call. Persistent layer has its
  own dedicated tests.

Verified: 1782 tests pass (29 new), ruff clean, smoke test confirms
end-to-end cache round-trip works.

WHATS_NEW entry under '2.4.2' dev cycle.
2026-05-03 17:16:39 -07:00

175 lines
5.9 KiB
Python

"""Persistent MusicBrainz release-MBID cache for albums.
The original in-memory `mb_release_cache` in `core/metadata/source.py`
maps `(normalized_album_name, artist_name) -> release_mbid` so per-track
enrichment of the same album hits the cache and writes the same
``MUSICBRAINZ_ALBUMID`` to every track's tags. That cache is a bounded
``OrderedDict`` (4096 entries) — bounded means it can evict entries
between tracks of the same album when other albums are processed in
between. Server restart drops it entirely. Either case can produce
inconsistent album MBIDs across tracks of the same album, which causes
Navidrome (and other media servers that group by album MBID) to split
the album into multiple entries.
This module is the persistent layer behind that cache. Same key shape,
backed by a tiny SQLite table so a successful lookup remembered ONCE
applies to every future track of the same album for the lifetime of
the install — not just the bounded in-memory window.
Strict additive design: every public function is wrapped in try/except
and degrades to a None / no-op return on any database error. The
existing in-memory cache + MusicBrainz lookup stays behind it as the
authoritative fallback. If this module breaks, downloads continue
exactly as they would today — just without the persistent benefit.
"""
from __future__ import annotations
import threading
from typing import Optional
from utils.logging_config import get_logger
logger = get_logger("metadata.album_mbid_cache")
# Lazy DB accessor — the cache module shouldn't trigger MusicDatabase
# import at module-load time (circular-import risk when source.py is
# imported during database initialization).
_db_factory_lock = threading.Lock()
_db_factory = None
def _get_database():
"""Resolve the MusicDatabase singleton lazily.
Returns None if anything goes wrong — callers MUST handle a None
return as "cache unavailable, fall through to MB lookup."
"""
global _db_factory
with _db_factory_lock:
if _db_factory is None:
try:
from database.music_database import get_database
_db_factory = get_database
except Exception as exc:
logger.warning(f"Persistent MBID cache: could not load database module: {exc}")
return None
try:
return _db_factory()
except Exception as exc:
logger.warning(f"Persistent MBID cache: database accessor failed: {exc}")
return None
def lookup(normalized_album_key: str, artist_key: str) -> Optional[str]:
"""Read a cached release MBID for the given (album, artist) pair.
Returns the stored MBID string if found, otherwise None. Never
raises — DB errors degrade silently to "cache miss" so the caller
falls through to MusicBrainz like it does today.
Args:
normalized_album_key: Output of ``normalize_album_cache_key`` —
already lowercased and stripped of edition parentheticals.
artist_key: Lowercased artist name (caller's responsibility to
pass a normalized key — keeps the schema uniform).
"""
if not normalized_album_key or not artist_key:
return None
db = _get_database()
if db is None:
return None
conn = None
try:
conn = db._get_connection()
cursor = conn.cursor()
cursor.execute(
"SELECT release_mbid FROM mb_album_release_cache "
"WHERE normalized_album_key = ? AND artist_key = ? LIMIT 1",
(normalized_album_key, artist_key),
)
row = cursor.fetchone()
if row:
mbid = row[0] if not hasattr(row, 'keys') else row['release_mbid']
return mbid or None
except Exception as exc:
logger.debug(f"Persistent MBID cache lookup failed: {exc}")
finally:
if conn is not None:
try:
conn.close()
except Exception:
pass
return None
def record(normalized_album_key: str, artist_key: str, release_mbid: str) -> bool:
"""Persist a (album, artist) -> release_mbid mapping.
Idempotent — uses INSERT OR REPLACE so re-recording the same key
just refreshes the timestamp. Returns True on success, False on
any failure. Failure is logged at debug level and never propagated
so a flaky DB write can't break the enrichment path.
"""
if not normalized_album_key or not artist_key or not release_mbid:
return False
db = _get_database()
if db is None:
return False
conn = None
try:
conn = db._get_connection()
cursor = conn.cursor()
cursor.execute(
"INSERT OR REPLACE INTO mb_album_release_cache "
"(normalized_album_key, artist_key, release_mbid, updated_at) "
"VALUES (?, ?, ?, CURRENT_TIMESTAMP)",
(normalized_album_key, artist_key, release_mbid),
)
conn.commit()
return True
except Exception as exc:
logger.debug(f"Persistent MBID cache record failed: {exc}")
return False
finally:
if conn is not None:
try:
conn.close()
except Exception:
pass
def clear_all() -> bool:
"""Wipe the persistent cache. Used by tests and by the maintenance
endpoint when a user wants to force a fresh MusicBrainz re-lookup
(e.g. after fixing widespread MBID inconsistencies)."""
db = _get_database()
if db is None:
return False
conn = None
try:
conn = db._get_connection()
cursor = conn.cursor()
cursor.execute("DELETE FROM mb_album_release_cache")
conn.commit()
return True
except Exception as exc:
logger.warning(f"Persistent MBID cache clear failed: {exc}")
return False
finally:
if conn is not None:
try:
conn.close()
except Exception:
pass
__all__ = ["lookup", "record", "clear_all"]