Reported case (CAL): a track already on disk got re-downloaded by the
watchlist scanner on every scan. Library DB had stale album metadata
for the file (track tagged on album "Left Alone") while the metadata
source reported it on a different album ("NPC" single). The
title+artist+album fuzzy block correctly said the album names didn't
match and declared the track missing — but the file's stable external
IDs (Spotify ID, ISRC, etc.) unambiguously identified it as the same
recording.
The earlier compilation-album fix (PR #461) handled qualifier drift
("OST" vs "Music From The Motion Picture"). This case is two
genuinely different album names referring to the same song.
Fix: provider-neutral external-ID short-circuit before the fuzzy
block in `is_track_missing_from_library`. Pulls every recognized ID
off the source track (Spotify / iTunes / Deezer / Tidal / Qobuz /
MusicBrainz / AudioDB / Hydrabase / ISRC), runs a single SELECT
against the indexed external-ID columns on the `tracks` table, and
treats any hit as "track exists in library — don't re-download".
If no IDs are available (older imports without enrichment, library
scans that didn't populate external IDs), falls through to the
existing fuzzy logic so the safety net stays intact.
New `core/library/track_identity.py` module with two helpers:
- `extract_external_ids(track)`: handles dict and object-style track
shapes, direct-field aliases (spotify_id / spotify_track_id /
SPOTIFY_TRACK_ID), and provider-disambiguated native `id` fields
(when track has `provider='deezer'` and `id='X'`, treats X as a
Deezer ID).
- `find_library_track_by_external_id(db, external_ids,
server_source)`: builds an OR of indexed column matches with
IS NOT NULL guards, optional server_source filter that also
passes legacy NULL rows, single-row LIMIT.
ISRC bridges across providers — a library track imported via Deezer
can be matched against a Spotify scan when both sides carry the
same ISRC.
43 regression tests in `tests/test_library_track_identity.py`:
- 9 ID-extraction tests for direct fields (Spotify / iTunes / Deezer /
ISRC / MBID / AudioDB / Hydrabase)
- 8 ID-extraction tests via the provider field (8 providers + source
alias + missing-provider-ignored)
- 7 mixed/defensive tests (multiple IDs, object-style, empty strings,
None track, numeric coercion)
- 8 lookup tests (per-provider + ISRC cross-bridge)
- 3 OR-semantics tests
- 4 server_source filter tests
- 2 ID-column-map sanity tests
Full pytest 1606 passed; ruff clean.
206 lines
7.4 KiB
Python
206 lines
7.4 KiB
Python
"""Match a metadata-source track against the library by stable external IDs.
|
|
|
|
Discord-reported (CAL): the watchlist scanner re-downloaded a track that
|
|
already existed on disk because the library DB had stale album metadata
|
|
(track tagged on album "Left Alone" while Spotify reported it as on the
|
|
"NPC" single). The matching logic relied on title + artist + album fuzzy
|
|
comparison; the album fuzzy correctly said the names didn't match, the
|
|
scanner declared the track missing, and the wishlist re-added + re-
|
|
downloaded it on every scan.
|
|
|
|
The track has a stable external identity though — every download embeds
|
|
Spotify / iTunes / Deezer / Tidal / Qobuz / MusicBrainz / AudioDB /
|
|
Hydrabase / ISRC IDs as both file tags AND DB columns. This module pulls
|
|
those IDs off either side and asks: do we already have a row in the
|
|
``tracks`` table whose external-ID column matches one of the source
|
|
track's IDs? If yes, the track is NOT missing, regardless of how the
|
|
album metadata drifted between sources.
|
|
|
|
Provider-neutral by design — no spotify-only paths.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from typing import Any, Dict, List, Optional
|
|
|
|
from utils.logging_config import get_logger
|
|
|
|
logger = get_logger("library.track_identity")
|
|
|
|
|
|
# Maps the conceptual ID name (used in the source-track dict we extract
|
|
# below) to the column name on the library ``tracks`` table where that
|
|
# ID is persisted. Keep the column names in sync with the schema in
|
|
# ``database/music_database.py``.
|
|
EXTERNAL_ID_COLUMNS: Dict[str, str] = {
|
|
'spotify_id': 'spotify_track_id',
|
|
'itunes_id': 'itunes_track_id',
|
|
'deezer_id': 'deezer_id',
|
|
'tidal_id': 'tidal_id',
|
|
'qobuz_id': 'qobuz_id',
|
|
'mbid': 'musicbrainz_recording_id',
|
|
'audiodb_id': 'audiodb_id',
|
|
'soul_id': 'soul_id',
|
|
'isrc': 'isrc',
|
|
}
|
|
|
|
|
|
def _coerce(value: Any) -> Optional[str]:
|
|
"""Return value as a non-empty string, or None for empty / missing."""
|
|
if value is None:
|
|
return None
|
|
text = str(value).strip()
|
|
return text or None
|
|
|
|
|
|
def _get(track: Any, *names: str) -> Optional[str]:
|
|
"""Read the first non-empty attribute / dict key from ``names`` off
|
|
``track``. Accepts both dict-style and dataclass / object tracks."""
|
|
for name in names:
|
|
try:
|
|
value = track[name] if isinstance(track, dict) else getattr(track, name, None)
|
|
except (TypeError, KeyError):
|
|
value = None
|
|
coerced = _coerce(value)
|
|
if coerced is not None:
|
|
return coerced
|
|
return None
|
|
|
|
|
|
def extract_external_ids(track: Any) -> Dict[str, str]:
|
|
"""Pull every recognized external ID off a metadata-source track.
|
|
|
|
Handles the source-source naming drift: Spotify tracks expose ``id``
|
|
as the Spotify track ID; Deezer tracks expose ``id`` as the Deezer
|
|
track ID; iTunes tracks may use ``trackId`` or ``id``. The disamb-
|
|
iguating field is ``provider`` / ``source``. Tracks coming from a
|
|
SoulSync internal pipeline often carry every known ID set to its
|
|
source-specific value — we just collect whatever's there.
|
|
|
|
Returns a dict mapping conceptual ID name → ID value. Keys present
|
|
in ``EXTERNAL_ID_COLUMNS``. Empty dict when no IDs are available.
|
|
"""
|
|
if track is None:
|
|
return {}
|
|
|
|
ids: Dict[str, str] = {}
|
|
|
|
# Provider-neutral fields that carry their own name regardless of
|
|
# source. Most internal SoulSync tracks have these set; external
|
|
# source responses usually only have one of them populated.
|
|
direct_id_fields = {
|
|
'spotify_id': ('spotify_id', 'spotify_track_id', 'SPOTIFY_TRACK_ID'),
|
|
'itunes_id': ('itunes_id', 'itunes_track_id', 'trackId', 'ITUNES_TRACK_ID'),
|
|
'deezer_id': ('deezer_id', 'deezer_track_id', 'DEEZER_TRACK_ID'),
|
|
'tidal_id': ('tidal_id', 'tidal_track_id', 'TIDAL_TRACK_ID'),
|
|
'qobuz_id': ('qobuz_id', 'qobuz_track_id', 'QOBUZ_TRACK_ID'),
|
|
'mbid': ('musicbrainz_recording_id', 'mbid', 'MUSICBRAINZ_RECORDING_ID'),
|
|
'audiodb_id': ('audiodb_id', 'idTrack', 'AUDIODB_TRACK_ID'),
|
|
'soul_id': ('soul_id', 'SOUL_ID'),
|
|
'isrc': ('isrc', 'ISRC'),
|
|
}
|
|
for name, candidates in direct_id_fields.items():
|
|
value = _get(track, *candidates)
|
|
if value:
|
|
ids[name] = value
|
|
|
|
# Provider field tells us which native ``id`` belongs to. Without
|
|
# this, a Deezer track's ``id`` field would be silently ignored
|
|
# (we wouldn't know to map it to deezer_id).
|
|
provider = (_get(track, 'provider', 'source') or '').lower()
|
|
native_id = _get(track, 'id')
|
|
if native_id and provider:
|
|
provider_to_key = {
|
|
'spotify': 'spotify_id',
|
|
'itunes': 'itunes_id',
|
|
'deezer': 'deezer_id',
|
|
'tidal': 'tidal_id',
|
|
'qobuz': 'qobuz_id',
|
|
'musicbrainz': 'mbid',
|
|
'audiodb': 'audiodb_id',
|
|
'hydrabase': 'soul_id',
|
|
}
|
|
key = provider_to_key.get(provider)
|
|
if key and key not in ids:
|
|
ids[key] = native_id
|
|
|
|
return ids
|
|
|
|
|
|
def find_library_track_by_external_id(
|
|
db: Any,
|
|
*,
|
|
external_ids: Dict[str, str],
|
|
server_source: Optional[str] = None,
|
|
) -> Optional[Dict[str, Any]]:
|
|
"""Return a row from the ``tracks`` table whose any external ID
|
|
column matches one of the provided IDs, or None if no match.
|
|
|
|
Returns a sqlite3.Row-like dict so callers can read whatever fields
|
|
they want (id, title, file_path, etc.). When ``server_source`` is
|
|
set, restrict matches to tracks scanned from that media server —
|
|
avoids false positives when a user binds the same DB into multiple
|
|
profiles/servers.
|
|
|
|
Performance: every external_id column is indexed in the schema, so
|
|
each OR clause hits an index. Limit 1 because we only need to know
|
|
whether a match exists.
|
|
"""
|
|
if not external_ids:
|
|
return None
|
|
|
|
clauses: List[str] = []
|
|
params: List[Any] = []
|
|
for id_name, id_value in external_ids.items():
|
|
column = EXTERNAL_ID_COLUMNS.get(id_name)
|
|
if not column or not id_value:
|
|
continue
|
|
clauses.append(f"({column} = ? AND {column} IS NOT NULL AND {column} != '')")
|
|
params.append(id_value)
|
|
|
|
if not clauses:
|
|
return None
|
|
|
|
where_external = " OR ".join(clauses)
|
|
|
|
# Optional server_source filter
|
|
if server_source:
|
|
sql = (
|
|
f"SELECT * FROM tracks WHERE ({where_external}) "
|
|
f"AND (server_source = ? OR server_source IS NULL) LIMIT 1"
|
|
)
|
|
params.append(server_source)
|
|
else:
|
|
sql = f"SELECT * FROM tracks WHERE ({where_external}) LIMIT 1"
|
|
|
|
conn = None
|
|
try:
|
|
conn = db._get_connection()
|
|
cursor = conn.cursor()
|
|
cursor.execute(sql, params)
|
|
row = cursor.fetchone()
|
|
if row is None:
|
|
return None
|
|
# sqlite3.Row supports keys() — return as dict for caller stability.
|
|
try:
|
|
return dict(row)
|
|
except (TypeError, ValueError):
|
|
# Fallback for cursors that don't return Row objects.
|
|
cols = [c[0] for c in cursor.description]
|
|
return dict(zip(cols, row, strict=False))
|
|
except Exception as exc:
|
|
logger.debug(f"find_library_track_by_external_id query failed: {exc}")
|
|
return None
|
|
finally:
|
|
if conn is not None:
|
|
try:
|
|
conn.close()
|
|
except Exception:
|
|
pass
|
|
|
|
|
|
__all__ = [
|
|
'EXTERNAL_ID_COLUMNS',
|
|
'extract_external_ids',
|
|
'find_library_track_by_external_id',
|
|
]
|