soulsync/core/library/track_identity.py
Broque Thomas ecb8939c80 Match library tracks by external IDs before fuzzy in watchlist scan
Reported case (CAL): a track already on disk got re-downloaded by the
watchlist scanner on every scan. Library DB had stale album metadata
for the file (track tagged on album "Left Alone") while the metadata
source reported it on a different album ("NPC" single). The
title+artist+album fuzzy block correctly said the album names didn't
match and declared the track missing — but the file's stable external
IDs (Spotify ID, ISRC, etc.) unambiguously identified it as the same
recording.

The earlier compilation-album fix (PR #461) handled qualifier drift
("OST" vs "Music From The Motion Picture"). This case is two
genuinely different album names referring to the same song.

Fix: provider-neutral external-ID short-circuit before the fuzzy
block in `is_track_missing_from_library`. Pulls every recognized ID
off the source track (Spotify / iTunes / Deezer / Tidal / Qobuz /
MusicBrainz / AudioDB / Hydrabase / ISRC), runs a single SELECT
against the indexed external-ID columns on the `tracks` table, and
treats any hit as "track exists in library — don't re-download".

If no IDs are available (older imports without enrichment, library
scans that didn't populate external IDs), falls through to the
existing fuzzy logic so the safety net stays intact.

New `core/library/track_identity.py` module with two helpers:
- `extract_external_ids(track)`: handles dict and object-style track
  shapes, direct-field aliases (spotify_id / spotify_track_id /
  SPOTIFY_TRACK_ID), and provider-disambiguated native `id` fields
  (when track has `provider='deezer'` and `id='X'`, treats X as a
  Deezer ID).
- `find_library_track_by_external_id(db, external_ids,
  server_source)`: builds an OR of indexed column matches with
  IS NOT NULL guards, optional server_source filter that also
  passes legacy NULL rows, single-row LIMIT.

ISRC bridges across providers — a library track imported via Deezer
can be matched against a Spotify scan when both sides carry the
same ISRC.

43 regression tests in `tests/test_library_track_identity.py`:
- 9 ID-extraction tests for direct fields (Spotify / iTunes / Deezer /
  ISRC / MBID / AudioDB / Hydrabase)
- 8 ID-extraction tests via the provider field (8 providers + source
  alias + missing-provider-ignored)
- 7 mixed/defensive tests (multiple IDs, object-style, empty strings,
  None track, numeric coercion)
- 8 lookup tests (per-provider + ISRC cross-bridge)
- 3 OR-semantics tests
- 4 server_source filter tests
- 2 ID-column-map sanity tests

Full pytest 1606 passed; ruff clean.
2026-05-02 16:06:59 -07:00

206 lines
7.4 KiB
Python

"""Match a metadata-source track against the library by stable external IDs.
Discord-reported (CAL): the watchlist scanner re-downloaded a track that
already existed on disk because the library DB had stale album metadata
(track tagged on album "Left Alone" while Spotify reported it as on the
"NPC" single). The matching logic relied on title + artist + album fuzzy
comparison; the album fuzzy correctly said the names didn't match, the
scanner declared the track missing, and the wishlist re-added + re-
downloaded it on every scan.
The track has a stable external identity though — every download embeds
Spotify / iTunes / Deezer / Tidal / Qobuz / MusicBrainz / AudioDB /
Hydrabase / ISRC IDs as both file tags AND DB columns. This module pulls
those IDs off either side and asks: do we already have a row in the
``tracks`` table whose external-ID column matches one of the source
track's IDs? If yes, the track is NOT missing, regardless of how the
album metadata drifted between sources.
Provider-neutral by design — no spotify-only paths.
"""
from __future__ import annotations
from typing import Any, Dict, List, Optional
from utils.logging_config import get_logger
logger = get_logger("library.track_identity")
# Maps the conceptual ID name (used in the source-track dict we extract
# below) to the column name on the library ``tracks`` table where that
# ID is persisted. Keep the column names in sync with the schema in
# ``database/music_database.py``.
EXTERNAL_ID_COLUMNS: Dict[str, str] = {
'spotify_id': 'spotify_track_id',
'itunes_id': 'itunes_track_id',
'deezer_id': 'deezer_id',
'tidal_id': 'tidal_id',
'qobuz_id': 'qobuz_id',
'mbid': 'musicbrainz_recording_id',
'audiodb_id': 'audiodb_id',
'soul_id': 'soul_id',
'isrc': 'isrc',
}
def _coerce(value: Any) -> Optional[str]:
"""Return value as a non-empty string, or None for empty / missing."""
if value is None:
return None
text = str(value).strip()
return text or None
def _get(track: Any, *names: str) -> Optional[str]:
"""Read the first non-empty attribute / dict key from ``names`` off
``track``. Accepts both dict-style and dataclass / object tracks."""
for name in names:
try:
value = track[name] if isinstance(track, dict) else getattr(track, name, None)
except (TypeError, KeyError):
value = None
coerced = _coerce(value)
if coerced is not None:
return coerced
return None
def extract_external_ids(track: Any) -> Dict[str, str]:
"""Pull every recognized external ID off a metadata-source track.
Handles the source-source naming drift: Spotify tracks expose ``id``
as the Spotify track ID; Deezer tracks expose ``id`` as the Deezer
track ID; iTunes tracks may use ``trackId`` or ``id``. The disamb-
iguating field is ``provider`` / ``source``. Tracks coming from a
SoulSync internal pipeline often carry every known ID set to its
source-specific value — we just collect whatever's there.
Returns a dict mapping conceptual ID name → ID value. Keys present
in ``EXTERNAL_ID_COLUMNS``. Empty dict when no IDs are available.
"""
if track is None:
return {}
ids: Dict[str, str] = {}
# Provider-neutral fields that carry their own name regardless of
# source. Most internal SoulSync tracks have these set; external
# source responses usually only have one of them populated.
direct_id_fields = {
'spotify_id': ('spotify_id', 'spotify_track_id', 'SPOTIFY_TRACK_ID'),
'itunes_id': ('itunes_id', 'itunes_track_id', 'trackId', 'ITUNES_TRACK_ID'),
'deezer_id': ('deezer_id', 'deezer_track_id', 'DEEZER_TRACK_ID'),
'tidal_id': ('tidal_id', 'tidal_track_id', 'TIDAL_TRACK_ID'),
'qobuz_id': ('qobuz_id', 'qobuz_track_id', 'QOBUZ_TRACK_ID'),
'mbid': ('musicbrainz_recording_id', 'mbid', 'MUSICBRAINZ_RECORDING_ID'),
'audiodb_id': ('audiodb_id', 'idTrack', 'AUDIODB_TRACK_ID'),
'soul_id': ('soul_id', 'SOUL_ID'),
'isrc': ('isrc', 'ISRC'),
}
for name, candidates in direct_id_fields.items():
value = _get(track, *candidates)
if value:
ids[name] = value
# Provider field tells us which native ``id`` belongs to. Without
# this, a Deezer track's ``id`` field would be silently ignored
# (we wouldn't know to map it to deezer_id).
provider = (_get(track, 'provider', 'source') or '').lower()
native_id = _get(track, 'id')
if native_id and provider:
provider_to_key = {
'spotify': 'spotify_id',
'itunes': 'itunes_id',
'deezer': 'deezer_id',
'tidal': 'tidal_id',
'qobuz': 'qobuz_id',
'musicbrainz': 'mbid',
'audiodb': 'audiodb_id',
'hydrabase': 'soul_id',
}
key = provider_to_key.get(provider)
if key and key not in ids:
ids[key] = native_id
return ids
def find_library_track_by_external_id(
db: Any,
*,
external_ids: Dict[str, str],
server_source: Optional[str] = None,
) -> Optional[Dict[str, Any]]:
"""Return a row from the ``tracks`` table whose any external ID
column matches one of the provided IDs, or None if no match.
Returns a sqlite3.Row-like dict so callers can read whatever fields
they want (id, title, file_path, etc.). When ``server_source`` is
set, restrict matches to tracks scanned from that media server —
avoids false positives when a user binds the same DB into multiple
profiles/servers.
Performance: every external_id column is indexed in the schema, so
each OR clause hits an index. Limit 1 because we only need to know
whether a match exists.
"""
if not external_ids:
return None
clauses: List[str] = []
params: List[Any] = []
for id_name, id_value in external_ids.items():
column = EXTERNAL_ID_COLUMNS.get(id_name)
if not column or not id_value:
continue
clauses.append(f"({column} = ? AND {column} IS NOT NULL AND {column} != '')")
params.append(id_value)
if not clauses:
return None
where_external = " OR ".join(clauses)
# Optional server_source filter
if server_source:
sql = (
f"SELECT * FROM tracks WHERE ({where_external}) "
f"AND (server_source = ? OR server_source IS NULL) LIMIT 1"
)
params.append(server_source)
else:
sql = f"SELECT * FROM tracks WHERE ({where_external}) LIMIT 1"
conn = None
try:
conn = db._get_connection()
cursor = conn.cursor()
cursor.execute(sql, params)
row = cursor.fetchone()
if row is None:
return None
# sqlite3.Row supports keys() — return as dict for caller stability.
try:
return dict(row)
except (TypeError, ValueError):
# Fallback for cursors that don't return Row objects.
cols = [c[0] for c in cursor.description]
return dict(zip(cols, row, strict=False))
except Exception as exc:
logger.debug(f"find_library_track_by_external_id query failed: {exc}")
return None
finally:
if conn is not None:
try:
conn.close()
except Exception:
pass
__all__ = [
'EXTERNAL_ID_COLUMNS',
'extract_external_ids',
'find_library_track_by_external_id',
]