soulsync/core/imports/album_matching.py
Broque Thomas f9f74ac511 Lift auto-import matching to testable helper + pin contracts
Cin-pass on the #524 + multi-disc fixes. Pre-merge polish.

Lifts: `core/imports/album_matching.py`

`AutoImportWorker._match_tracks` was a 100+-line method buried in a
1400-line class. Testing it required monkey-patching `_read_file_tags`
+ mocking the metadata client just to exercise the matching algorithm.
Per Cin's "lift logic out of monolithic classes" pattern (same shape
as the album-info builders / discography / quality scanner lifts),
moved the dedup + scoring into `core/imports/album_matching.py` as
pure functions over already-fetched data.

Helper exposes:

- Constants for every match weight (TITLE_WEIGHT, ARTIST_WEIGHT,
  POSITION_WEIGHT, NEAR_POSITION_WEIGHT, CROSS_DISC_POSITION_WEIGHT,
  ALBUM_WEIGHT, MATCH_THRESHOLD). Magic numbers killed.
- `dedupe_files_by_position(audio_files, file_tags, *, quality_rank)` —
  position-keyed quality dedup.
- `score_file_against_track(file_path, file_tags, track, *,
  target_album, similarity)` — pure per-(file, track) scorer.
- `match_files_to_tracks(audio_files, file_tags, tracks, *,
  target_album, similarity, quality_rank)` — full matching with
  greedy best-per-track + first-come-first-serve over deduped files.

Worker shrinks from 100 lines of inline algorithm to 8 lines that
fetch tags + delegate to the helper.

Tests added (26 new across 3 files):

`tests/imports/test_album_matching_helper.py` (19 tests):
- Constants pin: weights sum to 1.0, threshold above position-only
- `dedupe_files_by_position`: quality wins, cross-disc preserved,
  tag-less files passed through, first-wins on equal quality
- `score_file_against_track`: perfect-agreement = 1.0, position
  needs both disc+track, near-position only same-disc, missing
  artist tags handled, disc field aliases (Spotify/Deezer/iTunes),
  filename fallback when title tag missing
- `match_files_to_tracks`: happy path, file used at-most-once,
  below-threshold left unmatched
- Edge case Cin would flag: tag-less file with strong filename title
  matches multi-disc album track via title alone (perfect-name
  scenario works); tag-less file with weak filename title against
  multi-disc API correctly stays unmatched (the behavior delta from
  the disc-aware fix — pinned so future readers see it's intentional)

`tests/test_import_album_match_endpoint.py` (3 tests):
- Backend warning fires when source missing from match POST
- No warning fires on the legit path (catches noisy-warning regression)
- Endpoint actually forwards source/name/artist to the payload
  builder (catches "logging the right warning but doing the wrong
  lookup" regression)

`tests/test_import_page_album_lookup_pattern.py` (4 tests):
- Source-text guard for the import-page #524 fix in stats-automations.js.
  Until the file is modularized enough for a behavioral JS test (under
  the existing tests/static/*.mjs pattern), regex-based assertions pin:
  the `_albumLookup` field exists, the click handler reads from it,
  both card renderers populate it before emitting onclick, and the
  cache stores `source` per entry. Caveat documented in the test
  module docstring.

Verification:
- All 26 new tests pass.
- Existing multi-disc tests (test_auto_import_multi_disc_matching.py)
  still pass after the lift — proves the helper is behavior-equivalent
  to the inline implementation it replaced.
- Full suite: 2293 passed, 1 flaky-timing failure
  (test_library_reorganize_orchestrator.py::test_watchdog_warns_about_stuck_workers
  — passes in isolation, fails only in full-suite runs, pre-existing,
  unrelated to this PR).
- Ruff clean.

Notes for the reviewer:

- The frontend stats-automations.js JS test is structural-only.
  Behavioral JS testing for that file requires modularizing the
  ~7k-line monolith first — out of scope for this fix.
- The cross-disc 5% consolation bonus is a small behavior change for
  users with weak/missing tag info on multi-disc albums. Pinned
  explicitly in `test_tagless_file_with_weak_title_unmatched_in_multidisc`
  so the trade-off is visible: correct multi-disc matching wins over
  optimistic position-only matching that produced wrong-disc files.
2026-05-09 09:13:23 -07:00

255 lines
8.9 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

"""Album-track matching helpers — lifted out of
``AutoImportWorker._match_tracks`` so the matching logic is testable in
isolation without instantiating the worker, mocking the metadata
client, or monkey-patching ``_read_file_tags``.
The worker still owns:
- File-system traversal + tag reads
- Metadata client lookup + album_data fetch
- Album-vs-single routing
This module owns:
- Quality-aware deduplication keyed on the ``(disc_number, track_number)``
position tuple
- Weighted match scoring against the album's tracklist
- Returning the list of (track, file, confidence) matches + leftover
unmatched files
Both behaviors are pure functions over already-fetched data, so the
test surface is just dicts in / dicts out.
"""
from __future__ import annotations
import os
from typing import Any, Callable, Dict, List, Set, Tuple
# ---------------------------------------------------------------------------
# Match-scoring weights
# ---------------------------------------------------------------------------
# Each weight is a fraction of the 0..1 confidence score the matcher
# accumulates per (file, track) pair. Sum of all maximum-bonus paths
# equals 1.0 in the happy case (perfect title + artist + position +
# album tag agreement).
#
# History note: the position bonus (30%) used to fire on track_number
# alone, which broke multi-disc albums where every disc has tracks 1..N.
# Disc-aware split (POSITION + CROSS_DISC) shipped 2026-05-09 after
# user reported Mr. Morale & The Big Steppers losing half its tracks
# during auto-import.
TITLE_WEIGHT = 0.45 # case-folded fuzzy title similarity
ARTIST_WEIGHT = 0.15 # albumartist (or artist) similarity
POSITION_WEIGHT = 0.30 # exact (disc_number, track_number) match
NEAR_POSITION_WEIGHT = 0.12 # off-by-one track number, same disc
CROSS_DISC_POSITION_WEIGHT = 0.05 # same track_number, different disc
ALBUM_WEIGHT = 0.10 # album tag similarity to target album
# A file scoring below this threshold against every track is treated
# as unmatched. Threshold sits below the per-component partial-match
# floor (~0.5 × 0.45 = 0.22) plus a small position consolation, so
# files with weak title agreement still need at least one strong signal.
MATCH_THRESHOLD = 0.4
SimilarityFn = Callable[[str, str], float]
QualityRankFn = Callable[[str], int]
def dedupe_files_by_position(
audio_files: List[str],
file_tags: Dict[str, Dict[str, Any]],
*,
quality_rank: QualityRankFn,
) -> List[str]:
"""Drop quality-duplicate files at the same ``(disc, track)``
position, keeping the higher-quality one.
The position key is ``(disc_number, track_number)`` — NOT
``track_number`` alone. Multi-disc albums where every disc has
tracks 1..N would otherwise collapse to one disc's worth of files
here, before the matcher even sees the rest.
Files with ``track_number == 0`` (no tag) all pass through —
can't dedupe positions we don't know.
"""
seen_positions: Dict[Tuple[int, int], str] = {}
deduped: List[str] = []
for f in audio_files:
tags = file_tags.get(f, {})
track_num = tags.get('track_number', 0) or 0
disc_num = tags.get('disc_number', 1) or 1
ext = os.path.splitext(f)[1].lower()
position_key = (disc_num, track_num)
if track_num > 0 and position_key in seen_positions:
prev_f = seen_positions[position_key]
prev_ext = os.path.splitext(prev_f)[1].lower()
if quality_rank(ext) > quality_rank(prev_ext):
deduped.remove(prev_f)
deduped.append(f)
seen_positions[position_key] = f
else:
deduped.append(f)
if track_num > 0:
seen_positions[position_key] = f
return deduped
def _extract_track_disc(track: Dict[str, Any]) -> int:
"""Pull disc number off an API track dict.
Different metadata sources spell the field differently:
Spotify ``disc_number``, Deezer ``disk_number``, iTunes
``discNumber``. Default to 1 when missing so single-disc albums
still match.
"""
return (
track.get('disc_number')
or track.get('disk_number')
or track.get('discNumber')
or 1
)
def _extract_track_artist(track: Dict[str, Any]) -> str:
artists = track.get('artists') or []
if not artists:
return ''
a = artists[0]
return a.get('name', str(a)) if isinstance(a, dict) else str(a)
def score_file_against_track(
file_path: str,
file_tags: Dict[str, Any],
track: Dict[str, Any],
*,
target_album: str,
similarity: SimilarityFn,
) -> float:
"""Compute the 0..1 confidence score for matching ``file_path``
(with its tags) to ``track`` (an API track dict).
Pure scoring — caller decides what to do with the score (compare
against ``MATCH_THRESHOLD``, pick best-per-track, etc).
"""
score = 0.0
# Title similarity (TITLE_WEIGHT). Falls back to filename stem when
# the file has no title tag.
title = file_tags.get('title') or os.path.splitext(os.path.basename(file_path))[0]
track_name = track.get('name', '')
score += similarity(title, track_name) * TITLE_WEIGHT
# Artist similarity (ARTIST_WEIGHT). Skipped if either side missing.
file_artist = file_tags.get('artist', '')
track_artist = _extract_track_artist(track)
if file_artist and track_artist:
score += similarity(file_artist, track_artist) * ARTIST_WEIGHT
# Position match (POSITION_WEIGHT / NEAR_POSITION_WEIGHT /
# CROSS_DISC_POSITION_WEIGHT). Gates on the (disc, track) tuple
# rather than track_number alone — see the module docstring's
# multi-disc history note.
file_track_num = file_tags.get('track_number', 0) or 0
track_num = track.get('track_number', 0) or 0
if file_track_num > 0 and track_num > 0:
file_disc = file_tags.get('disc_number', 1) or 1
track_disc = _extract_track_disc(track)
if file_track_num == track_num and file_disc == track_disc:
score += POSITION_WEIGHT
elif file_track_num == track_num and file_disc != track_disc:
# Same track number, different disc — small consolation so
# title/artist similarity has to carry the match. Common
# collision in deluxe / multi-disc releases where every
# disc has tracks numbered 1..N.
score += CROSS_DISC_POSITION_WEIGHT
elif abs(file_track_num - track_num) <= 1 and file_disc == track_disc:
score += NEAR_POSITION_WEIGHT
# Album tag bonus (ALBUM_WEIGHT). Helps disambiguate when the
# target_album name is a strong signal.
file_album = file_tags.get('album', '')
if file_album:
score += similarity(file_album, target_album) * ALBUM_WEIGHT
return score
def match_files_to_tracks(
audio_files: List[str],
file_tags: Dict[str, Dict[str, Any]],
tracks: List[Dict[str, Any]],
*,
target_album: str,
similarity: SimilarityFn,
quality_rank: QualityRankFn,
) -> Dict[str, Any]:
"""Match staging files to album tracks.
Returns a dict with:
- ``matches``: list of ``{'track': dict, 'file': str, 'confidence': float}``,
one per track that found a file scoring at or above
``MATCH_THRESHOLD``
- ``unmatched_files``: files left over after every track found its
best (or none)
Each file matches at most one track (best-scoring track that
accepted it wins). Each track matches at most one file (the highest-
scoring still-unused file).
Pure function — no side effects, no I/O, no metadata client. Easy
to unit-test by feeding tag dicts and track dicts directly.
"""
deduped = dedupe_files_by_position(audio_files, file_tags, quality_rank=quality_rank)
matches: List[Dict[str, Any]] = []
used_files: Set[str] = set()
for track in tracks:
best_file = None
best_score = 0.0
for f in deduped:
if f in used_files:
continue
tags = file_tags.get(f, {})
score = score_file_against_track(
f, tags, track,
target_album=target_album,
similarity=similarity,
)
if score > best_score and score >= MATCH_THRESHOLD:
best_score = score
best_file = f
if best_file:
used_files.add(best_file)
matches.append({
'track': track,
'file': best_file,
'confidence': round(best_score, 3),
})
return {
'matches': matches,
'unmatched_files': [f for f in deduped if f not in used_files],
}
__all__ = [
'TITLE_WEIGHT',
'ARTIST_WEIGHT',
'POSITION_WEIGHT',
'NEAR_POSITION_WEIGHT',
'CROSS_DISC_POSITION_WEIGHT',
'ALBUM_WEIGHT',
'MATCH_THRESHOLD',
'dedupe_files_by_position',
'score_file_against_track',
'match_files_to_tracks',
]