soulsync/tests/imports
Broque Thomas e11786ee40 Auto-import matching: fix Deezer source classification + bump tolerance
User report: all 6 staging candidates failing with "Could not match
tracks to album tracklist" despite identification correctly resolving
each album. 18 properly-tagged Chris Brown F.A.M.E. tracks, 21
properly-tagged Mr. Morale tracks, etc. — every match attempt
rejected by the duration sanity gate.

Root cause: I had Deezer in `_SECONDS_DURATION_SOURCES`, assuming
Deezer's `duration` field was raw seconds (which the API returns).
But `DeezerClient.get_album_tracks` already converts seconds → ms
INTERNALLY (`'duration_ms': item.get('duration', 0) * 1000`) before
the value reaches the matcher. My helper saw `source='deezer'` →
multiplied by 1000 again → 255000 ms became 255,000,000 ms (70 hours).
Every track-file pair failed the gate by a factor of 1000×.

Diagnostic chain that got me there:
1. Added `[Album Matching] No matches: X files, Y tracks, Z
   duration-rejected, W below threshold` summary log so future "0
   matches" reports surface the rejection reason.
2. Fixed the helper's logger from `logging.getLogger(__name__)` (which
   resolves outside the soulsync handler tree → invisible in app.log)
   to `get_logger("imports.album_matching")` (under the namespace the
   file handler watches).
3. Added per-rejection-type diagnostic showing actual file vs track
   duration values + raw track keys + source.

That third diagnostic surfaced `track 'United In Grief' resolved=255000000
(raw duration_ms=255000, raw duration=None, source='deezer')` —
making the bug obvious.

Fixes:

- Moved Deezer from `_SECONDS_DURATION_SOURCES` to
  `_MS_DURATION_SOURCES`. Comment documents WHY (the client converts
  before returning) so a future reader doesn't "fix" the
  classification back the wrong way.
- Bumped `DURATION_TOLERANCE_MS` from 3000 → 10000 (3s → 10s) to
  match Picard ~7s / Beets ~10-15s / Plex ~10s industry baselines.
  3s was a defensive copy of the post-download integrity check
  threshold but that's a different problem (catching truncated
  downloads, not identifying recordings across remasters/encodings).
- `_track_duration_ms` magnitude heuristic kept as fallback for
  unknown / missing source (mocked test data without `source` field).
- Added `Match aborted` warnings at the three earlier silent return
  points in `_match_tracks` (no client, no album_data, no tracks)
  so future "Could not match" reports show WHICH step bailed.
- Added per-run diagnostic in `match_files_to_tracks` that logs the
  first duration rejection's actual values — surfaces unit mismatches
  + drift problems without spamming N×M lines per run.

Test changes:

- `test_deezer_seconds_duration_converted_to_ms` renamed +
  rewritten as `test_deezer_already_normalised_to_ms_by_client`
  to pin the actual contract (matcher receives ms from the Deezer
  client, takes as-is).
- `test_track_duration_source_aware_dispatch` updated — Deezer test
  case now uses ms input + expects ms output.
- New `test_raw_deezer_seconds_falls_back_to_magnitude_heuristic`
  pins the rare edge case where raw Deezer items WITHOUT `source`
  reach the matcher (no client conversion path) — heuristic catches
  it.

Verification:
- 179 import tests pass after changes
- Live test: all 6 user staging candidates now matching at 95-100%
  confidence
- Multi-disc Mr. Morale lands with proper Disc 1 / Disc 2 / Disc 3
  folder structure
- Picard-tagged libraries hit MBID fast paths (verified earlier)
- Tracks process in parallel via the existing scan-now thread spawn
  (next commit refactors this to a proper bounded executor)
2026-05-09 15:53:17 -07:00
..
test_album_matching_exact_id.py Auto-import matching: fix Deezer source classification + bump tolerance 2026-05-09 15:53:17 -07:00
test_album_matching_helper.py Auto-import polish: real-file tag reader test, source-aware duration, pin consolation 2026-05-09 11:08:09 -07:00
test_auto_import_live_progress.py Auto-import: live per-track progress + in-progress history row 2026-05-02 22:34:09 -07:00
test_auto_import_multi_disc_matching.py Auto-import: respect disc_number in dedup + match scoring 2026-05-08 22:36:51 -07:00
test_auto_import_scanner_grouping.py Auto-import: dedup on folder_hash, not path — fixes silent-skip bug 2026-05-09 14:09:19 -07:00
test_auto_import_tag_reader_real_files.py Auto-import polish: real-file tag reader test, source-aware duration, pin consolation 2026-05-09 11:08:09 -07:00
test_build_single_import_context_typed_path.py Migrate album-info builders to typed Album path 2026-05-03 22:53:12 -07:00
test_dedup_orphan_cleanup.py Prune slskd dedup orphans after import 2026-05-01 09:35:08 -07:00
test_file_integrity.py Reject broken downloads before tagging via universal integrity check 2026-05-03 08:21:01 -07:00
test_import_album.py Rehome import helpers into core/imports 2026-04-27 19:54:44 +03:00
test_import_context.py Fix Spotify source ID fallback 2026-05-02 22:02:01 +03:00
test_import_file_ops.py Broaden import and metadata test coverage 2026-04-27 19:55:07 +03:00
test_import_filename.py Rehome import helpers into core/imports 2026-04-27 19:54:44 +03:00
test_import_guards.py Rehome import helpers into core/imports 2026-04-27 19:54:44 +03:00
test_import_paths.py fix: substitute \$cdnum in download paths and skip auto disc folder when template uses it 2026-04-28 21:32:24 -07:00
test_import_pipeline.py Reject broken downloads before tagging via universal integrity check 2026-05-03 08:21:01 -07:00
test_import_resolution_single_track_context.py Move metadata helpers into package modules 2026-04-29 11:28:42 +03:00
test_import_side_effects.py Add Library Disk Usage card to System Statistics 2026-05-03 20:17:06 -07:00
test_import_singles_route_through_album_path.py Route imported singles/EPs through album_path template 2026-04-30 21:33:09 -07:00
test_import_staging.py Rehome import helpers into core/imports 2026-04-27 19:54:44 +03:00
test_lossy_copy_delete_original.py Honor lossy_copy.delete_original after successful conversion 2026-05-02 14:26:46 -07:00