Cin-pass on the #524 + multi-disc fixes. Pre-merge polish. Lifts: `core/imports/album_matching.py` `AutoImportWorker._match_tracks` was a 100+-line method buried in a 1400-line class. Testing it required monkey-patching `_read_file_tags` + mocking the metadata client just to exercise the matching algorithm. Per Cin's "lift logic out of monolithic classes" pattern (same shape as the album-info builders / discography / quality scanner lifts), moved the dedup + scoring into `core/imports/album_matching.py` as pure functions over already-fetched data. Helper exposes: - Constants for every match weight (TITLE_WEIGHT, ARTIST_WEIGHT, POSITION_WEIGHT, NEAR_POSITION_WEIGHT, CROSS_DISC_POSITION_WEIGHT, ALBUM_WEIGHT, MATCH_THRESHOLD). Magic numbers killed. - `dedupe_files_by_position(audio_files, file_tags, *, quality_rank)` — position-keyed quality dedup. - `score_file_against_track(file_path, file_tags, track, *, target_album, similarity)` — pure per-(file, track) scorer. - `match_files_to_tracks(audio_files, file_tags, tracks, *, target_album, similarity, quality_rank)` — full matching with greedy best-per-track + first-come-first-serve over deduped files. Worker shrinks from 100 lines of inline algorithm to 8 lines that fetch tags + delegate to the helper. Tests added (26 new across 3 files): `tests/imports/test_album_matching_helper.py` (19 tests): - Constants pin: weights sum to 1.0, threshold above position-only - `dedupe_files_by_position`: quality wins, cross-disc preserved, tag-less files passed through, first-wins on equal quality - `score_file_against_track`: perfect-agreement = 1.0, position needs both disc+track, near-position only same-disc, missing artist tags handled, disc field aliases (Spotify/Deezer/iTunes), filename fallback when title tag missing - `match_files_to_tracks`: happy path, file used at-most-once, below-threshold left unmatched - Edge case Cin would flag: tag-less file with strong filename title matches multi-disc album track via title alone (perfect-name scenario works); tag-less file with weak filename title against multi-disc API correctly stays unmatched (the behavior delta from the disc-aware fix — pinned so future readers see it's intentional) `tests/test_import_album_match_endpoint.py` (3 tests): - Backend warning fires when source missing from match POST - No warning fires on the legit path (catches noisy-warning regression) - Endpoint actually forwards source/name/artist to the payload builder (catches "logging the right warning but doing the wrong lookup" regression) `tests/test_import_page_album_lookup_pattern.py` (4 tests): - Source-text guard for the import-page #524 fix in stats-automations.js. Until the file is modularized enough for a behavioral JS test (under the existing tests/static/*.mjs pattern), regex-based assertions pin: the `_albumLookup` field exists, the click handler reads from it, both card renderers populate it before emitting onclick, and the cache stores `source` per entry. Caveat documented in the test module docstring. Verification: - All 26 new tests pass. - Existing multi-disc tests (test_auto_import_multi_disc_matching.py) still pass after the lift — proves the helper is behavior-equivalent to the inline implementation it replaced. - Full suite: 2293 passed, 1 flaky-timing failure (test_library_reorganize_orchestrator.py::test_watchdog_warns_about_stuck_workers — passes in isolation, fails only in full-suite runs, pre-existing, unrelated to this PR). - Ruff clean. Notes for the reviewer: - The frontend stats-automations.js JS test is structural-only. Behavioral JS testing for that file requires modularizing the ~7k-line monolith first — out of scope for this fix. - The cross-disc 5% consolation bonus is a small behavior change for users with weak/missing tag info on multi-disc albums. Pinned explicitly in `test_tagless_file_with_weak_title_unmatched_in_multidisc` so the trade-off is visible: correct multi-disc matching wins over optimistic position-only matching that produced wrong-disc files.
255 lines
8.9 KiB
Python
255 lines
8.9 KiB
Python
"""Album-track matching helpers — lifted out of
|
||
``AutoImportWorker._match_tracks`` so the matching logic is testable in
|
||
isolation without instantiating the worker, mocking the metadata
|
||
client, or monkey-patching ``_read_file_tags``.
|
||
|
||
The worker still owns:
|
||
- File-system traversal + tag reads
|
||
- Metadata client lookup + album_data fetch
|
||
- Album-vs-single routing
|
||
|
||
This module owns:
|
||
- Quality-aware deduplication keyed on the ``(disc_number, track_number)``
|
||
position tuple
|
||
- Weighted match scoring against the album's tracklist
|
||
- Returning the list of (track, file, confidence) matches + leftover
|
||
unmatched files
|
||
|
||
Both behaviors are pure functions over already-fetched data, so the
|
||
test surface is just dicts in / dicts out.
|
||
"""
|
||
|
||
from __future__ import annotations
|
||
|
||
import os
|
||
from typing import Any, Callable, Dict, List, Set, Tuple
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# Match-scoring weights
|
||
# ---------------------------------------------------------------------------
|
||
# Each weight is a fraction of the 0..1 confidence score the matcher
|
||
# accumulates per (file, track) pair. Sum of all maximum-bonus paths
|
||
# equals 1.0 in the happy case (perfect title + artist + position +
|
||
# album tag agreement).
|
||
#
|
||
# History note: the position bonus (30%) used to fire on track_number
|
||
# alone, which broke multi-disc albums where every disc has tracks 1..N.
|
||
# Disc-aware split (POSITION + CROSS_DISC) shipped 2026-05-09 after
|
||
# user reported Mr. Morale & The Big Steppers losing half its tracks
|
||
# during auto-import.
|
||
|
||
TITLE_WEIGHT = 0.45 # case-folded fuzzy title similarity
|
||
ARTIST_WEIGHT = 0.15 # albumartist (or artist) similarity
|
||
POSITION_WEIGHT = 0.30 # exact (disc_number, track_number) match
|
||
NEAR_POSITION_WEIGHT = 0.12 # off-by-one track number, same disc
|
||
CROSS_DISC_POSITION_WEIGHT = 0.05 # same track_number, different disc
|
||
ALBUM_WEIGHT = 0.10 # album tag similarity to target album
|
||
|
||
# A file scoring below this threshold against every track is treated
|
||
# as unmatched. Threshold sits below the per-component partial-match
|
||
# floor (~0.5 × 0.45 = 0.22) plus a small position consolation, so
|
||
# files with weak title agreement still need at least one strong signal.
|
||
MATCH_THRESHOLD = 0.4
|
||
|
||
|
||
SimilarityFn = Callable[[str, str], float]
|
||
QualityRankFn = Callable[[str], int]
|
||
|
||
|
||
def dedupe_files_by_position(
|
||
audio_files: List[str],
|
||
file_tags: Dict[str, Dict[str, Any]],
|
||
*,
|
||
quality_rank: QualityRankFn,
|
||
) -> List[str]:
|
||
"""Drop quality-duplicate files at the same ``(disc, track)``
|
||
position, keeping the higher-quality one.
|
||
|
||
The position key is ``(disc_number, track_number)`` — NOT
|
||
``track_number`` alone. Multi-disc albums where every disc has
|
||
tracks 1..N would otherwise collapse to one disc's worth of files
|
||
here, before the matcher even sees the rest.
|
||
|
||
Files with ``track_number == 0`` (no tag) all pass through —
|
||
can't dedupe positions we don't know.
|
||
"""
|
||
seen_positions: Dict[Tuple[int, int], str] = {}
|
||
deduped: List[str] = []
|
||
|
||
for f in audio_files:
|
||
tags = file_tags.get(f, {})
|
||
track_num = tags.get('track_number', 0) or 0
|
||
disc_num = tags.get('disc_number', 1) or 1
|
||
ext = os.path.splitext(f)[1].lower()
|
||
position_key = (disc_num, track_num)
|
||
|
||
if track_num > 0 and position_key in seen_positions:
|
||
prev_f = seen_positions[position_key]
|
||
prev_ext = os.path.splitext(prev_f)[1].lower()
|
||
if quality_rank(ext) > quality_rank(prev_ext):
|
||
deduped.remove(prev_f)
|
||
deduped.append(f)
|
||
seen_positions[position_key] = f
|
||
else:
|
||
deduped.append(f)
|
||
if track_num > 0:
|
||
seen_positions[position_key] = f
|
||
|
||
return deduped
|
||
|
||
|
||
def _extract_track_disc(track: Dict[str, Any]) -> int:
|
||
"""Pull disc number off an API track dict.
|
||
|
||
Different metadata sources spell the field differently:
|
||
Spotify ``disc_number``, Deezer ``disk_number``, iTunes
|
||
``discNumber``. Default to 1 when missing so single-disc albums
|
||
still match.
|
||
"""
|
||
return (
|
||
track.get('disc_number')
|
||
or track.get('disk_number')
|
||
or track.get('discNumber')
|
||
or 1
|
||
)
|
||
|
||
|
||
def _extract_track_artist(track: Dict[str, Any]) -> str:
|
||
artists = track.get('artists') or []
|
||
if not artists:
|
||
return ''
|
||
a = artists[0]
|
||
return a.get('name', str(a)) if isinstance(a, dict) else str(a)
|
||
|
||
|
||
def score_file_against_track(
|
||
file_path: str,
|
||
file_tags: Dict[str, Any],
|
||
track: Dict[str, Any],
|
||
*,
|
||
target_album: str,
|
||
similarity: SimilarityFn,
|
||
) -> float:
|
||
"""Compute the 0..1 confidence score for matching ``file_path``
|
||
(with its tags) to ``track`` (an API track dict).
|
||
|
||
Pure scoring — caller decides what to do with the score (compare
|
||
against ``MATCH_THRESHOLD``, pick best-per-track, etc).
|
||
"""
|
||
score = 0.0
|
||
|
||
# Title similarity (TITLE_WEIGHT). Falls back to filename stem when
|
||
# the file has no title tag.
|
||
title = file_tags.get('title') or os.path.splitext(os.path.basename(file_path))[0]
|
||
track_name = track.get('name', '')
|
||
score += similarity(title, track_name) * TITLE_WEIGHT
|
||
|
||
# Artist similarity (ARTIST_WEIGHT). Skipped if either side missing.
|
||
file_artist = file_tags.get('artist', '')
|
||
track_artist = _extract_track_artist(track)
|
||
if file_artist and track_artist:
|
||
score += similarity(file_artist, track_artist) * ARTIST_WEIGHT
|
||
|
||
# Position match (POSITION_WEIGHT / NEAR_POSITION_WEIGHT /
|
||
# CROSS_DISC_POSITION_WEIGHT). Gates on the (disc, track) tuple
|
||
# rather than track_number alone — see the module docstring's
|
||
# multi-disc history note.
|
||
file_track_num = file_tags.get('track_number', 0) or 0
|
||
track_num = track.get('track_number', 0) or 0
|
||
if file_track_num > 0 and track_num > 0:
|
||
file_disc = file_tags.get('disc_number', 1) or 1
|
||
track_disc = _extract_track_disc(track)
|
||
if file_track_num == track_num and file_disc == track_disc:
|
||
score += POSITION_WEIGHT
|
||
elif file_track_num == track_num and file_disc != track_disc:
|
||
# Same track number, different disc — small consolation so
|
||
# title/artist similarity has to carry the match. Common
|
||
# collision in deluxe / multi-disc releases where every
|
||
# disc has tracks numbered 1..N.
|
||
score += CROSS_DISC_POSITION_WEIGHT
|
||
elif abs(file_track_num - track_num) <= 1 and file_disc == track_disc:
|
||
score += NEAR_POSITION_WEIGHT
|
||
|
||
# Album tag bonus (ALBUM_WEIGHT). Helps disambiguate when the
|
||
# target_album name is a strong signal.
|
||
file_album = file_tags.get('album', '')
|
||
if file_album:
|
||
score += similarity(file_album, target_album) * ALBUM_WEIGHT
|
||
|
||
return score
|
||
|
||
|
||
def match_files_to_tracks(
|
||
audio_files: List[str],
|
||
file_tags: Dict[str, Dict[str, Any]],
|
||
tracks: List[Dict[str, Any]],
|
||
*,
|
||
target_album: str,
|
||
similarity: SimilarityFn,
|
||
quality_rank: QualityRankFn,
|
||
) -> Dict[str, Any]:
|
||
"""Match staging files to album tracks.
|
||
|
||
Returns a dict with:
|
||
- ``matches``: list of ``{'track': dict, 'file': str, 'confidence': float}``,
|
||
one per track that found a file scoring at or above
|
||
``MATCH_THRESHOLD``
|
||
- ``unmatched_files``: files left over after every track found its
|
||
best (or none)
|
||
|
||
Each file matches at most one track (best-scoring track that
|
||
accepted it wins). Each track matches at most one file (the highest-
|
||
scoring still-unused file).
|
||
|
||
Pure function — no side effects, no I/O, no metadata client. Easy
|
||
to unit-test by feeding tag dicts and track dicts directly.
|
||
"""
|
||
deduped = dedupe_files_by_position(audio_files, file_tags, quality_rank=quality_rank)
|
||
|
||
matches: List[Dict[str, Any]] = []
|
||
used_files: Set[str] = set()
|
||
|
||
for track in tracks:
|
||
best_file = None
|
||
best_score = 0.0
|
||
|
||
for f in deduped:
|
||
if f in used_files:
|
||
continue
|
||
tags = file_tags.get(f, {})
|
||
score = score_file_against_track(
|
||
f, tags, track,
|
||
target_album=target_album,
|
||
similarity=similarity,
|
||
)
|
||
if score > best_score and score >= MATCH_THRESHOLD:
|
||
best_score = score
|
||
best_file = f
|
||
|
||
if best_file:
|
||
used_files.add(best_file)
|
||
matches.append({
|
||
'track': track,
|
||
'file': best_file,
|
||
'confidence': round(best_score, 3),
|
||
})
|
||
|
||
return {
|
||
'matches': matches,
|
||
'unmatched_files': [f for f in deduped if f not in used_files],
|
||
}
|
||
|
||
|
||
__all__ = [
|
||
'TITLE_WEIGHT',
|
||
'ARTIST_WEIGHT',
|
||
'POSITION_WEIGHT',
|
||
'NEAR_POSITION_WEIGHT',
|
||
'CROSS_DISC_POSITION_WEIGHT',
|
||
'ALBUM_WEIGHT',
|
||
'MATCH_THRESHOLD',
|
||
'dedupe_files_by_position',
|
||
'score_file_against_track',
|
||
'match_files_to_tracks',
|
||
]
|