soulsync/tests
Broque Thomas 6e61890551 Stop watchlist re-downloading compilation tracks; catch slskd dedup orphans
Two related bugs reported on Discord by Mushy.

1. The watchlist re-downloaded the same OST track up to 7 times.

   ``is_track_missing_from_library`` compared Spotify's album name and
   the media-server scan's album name with a raw SequenceMatcher at a
   strict 0.85 threshold. Compilations and soundtracks routinely fail
   this — Spotify reports
   ``"Napoleon Dynamite (Music From The Motion Picture)"`` while the
   Plex / Navidrome / Jellyfin tag scan saves it as
   ``"Napoleon Dynamite OST"``. Raw similarity ≈ 0.49, so the scanner
   declared the track missing on every 30-minute scan and added it back
   to the wishlist. The wishlist then issued a fresh download. slskd
   appended ``_<19-digit-ns-timestamp>`` to each new copy because the
   target file already existed, and the user ended up with seven copies
   of one song in one folder.

   Fix: extract two pure helpers — ``_normalize_album_for_match``
   strips qualifier parentheticals (Music From X, OST, Deluxe Edition,
   Remastered, Anniversary, etc.) and trailing dash-clauses;
   ``_albums_likely_match`` checks equality after normalization,
   substring containment, and a relaxed 0.6 fuzzy ratio. A volume /
   part / disc / standalone-trailing-number guard rejects pairs like
   ``"Greatest Hits Vol. 1"`` vs ``"Greatest Hits Vol. 2"`` so the
   relaxed threshold doesn't introduce false positives on serialized
   releases. After this change the Napoleon Dynamite case collapses
   to ``"napoleon dynamite" == "napoleon dynamite"`` via the equality
   short-circuit and the redownload loop dies.

2. The duplicate detector found only one of the seven dupe files.

   The detector buckets tracks by the first 4 chars of their normalized
   tag title. Files written by slskd directly into a library folder
   often get inconsistent (or blank) tags from the media-server rescan,
   so the seven copies were bucketed apart by parsed title and never
   compared.

   Fix: refactor the per-bucket comparison into ``_scan_bucket``, then
   add a second pass — ``_build_filename_buckets`` re-buckets leftover
   tracks by canonical filename stem (slskd dedup tail stripped via
   ``_strip_slskd_dedup_suffix``, same regex the import-cleanup PR uses)
   plus extension. Filename agreement is itself strong evidence the
   files came from the same source download, so the second pass calls
   ``_scan_bucket`` with ``require_metadata_match=False`` to skip the
   title / artist / cross-album gates. The same-physical-file guard
   still runs so bind-mount duplicates aren't flagged.

72 new regression tests across two files cover the album-match
helpers (28 tests including the Napoleon Dynamite scenario, 7 volume
disagreements, 8 positive/negative pairs, 5 defensive cases) and the
new filename-bucket pass (16 tests across bucket construction, scan
integration, and existing title-pass behavior). Full pytest 1509
passed; ruff clean.

Reported by Mushy in Discord.
2026-05-01 12:57:50 -07:00
..
artists Lift enhance_artist_quality to core/artists/quality.py 2026-04-29 10:05:57 -07:00
automation Surface handler-returned errors in automation last_error 2026-04-30 15:45:28 -07:00
discovery Preserve artwork in quality scanner wishlist handoff 2026-04-30 21:42:16 +03:00
downloads Rename wishlist lifecycle helper 2026-04-30 08:06:02 +03:00
imports Prune slskd dedup orphans after import 2026-05-01 09:35:08 -07:00
library Lift _execute_retag to core/library/retag.py 2026-04-29 09:03:42 -07:00
metadata Simplify service status and labels 2026-05-01 12:06:58 +03:00
playlists Lift _prepare_stream_task + playlist_explorer_build_tree to core/ 2026-04-29 11:26:07 -07:00
search Search lift: pre-merge parity polish for cin's review 2026-04-27 15:32:48 -07:00
stats Lift /api/stats/* and /api/listening-stats/* into core/stats/ 2026-04-27 14:27:03 -07:00
streaming Mock streaming prep sleep in tests 2026-04-30 21:26:28 +03:00
watchlist Lift _process_watchlist_scan_automatically to core/watchlist/auto_scan.py 2026-04-29 08:17:41 -07:00
wishlist Coerce wishlist track counts before category checks 2026-04-30 21:42:43 +03:00
conftest.py Clarify Spotify auth actions 2026-05-01 10:36:50 +03:00
test_acoustid_scanner.py Fix null-pointer error in acoustid_scanner 2026-04-17 19:17:24 +03:00
test_album_completeness_job.py Fix DummyConfigManager position in album completeness job test 2026-04-27 19:55:06 +03:00
test_auth_usage_throttle.py test: auth last_used_at write throttle 2026-04-19 15:22:24 -07:00
test_config_save_retry.py Stop config retry tests from writing to the real DB 2026-04-30 14:07:00 -07:00
test_content_filter_regex.py Fix watchlist content filters: live false positives + auto-scan bypass 2026-04-21 19:16:25 -07:00
test_dashboard_events.py Realign test file names 2026-04-17 21:25:01 +03:00
test_discogs_track_count.py Discogs: count rows with empty type_ as real tracks too 2026-04-25 08:27:44 -07:00
test_downloads_pagination.py Lift shared runtime state into core 2026-04-27 19:54:44 +03:00
test_duplicate_detector_mount_paths.py Filter same-physical-file duplicates from duplicate detector 2026-04-30 14:06:10 -07:00
test_duplicate_detector_slskd_dedup.py Stop watchlist re-downloading compilation tracks; catch slskd dedup orphans 2026-05-01 12:57:50 -07:00
test_featured_artist_completion.py Match featured-artist tracks across discography completion 2026-04-30 19:31:11 -07:00
test_hifi_instance_methods.py validate hifi instance reorder against pre-existing instances 2026-04-28 20:19:01 -06:00
test_hls_parsing.py fix: pick OS-specific ffmpeg binary in hls demux fallback test 2026-04-28 21:13:54 -07:00
test_import_singles_parallel.py Restore soulsync logger state between parallel-imports tests 2026-05-01 08:45:17 -07:00
test_library_reorganize.py Fix CI test failures from incomplete dummy config and encoding 2026-04-19 13:04:55 -07:00
test_library_reorganize_orchestrator.py Reorganize: fix moved-count + status/total UX issues from PR #377 review 2026-04-25 09:07:44 -07:00
test_listening_stats_batch_queries.py test: listening stats worker batched query paths 2026-04-19 15:22:25 -07:00
test_missing_cover_art.py Fix CI test failures from incomplete dummy config and encoding 2026-04-19 13:04:55 -07:00
test_reorganize_db_methods.py Reorganize queue: race + dedupe fixes from kettui review 2026-04-26 08:40:24 -07:00
test_reorganize_queue.py Reorganize queue: race + dedupe fixes from kettui review 2026-04-26 08:40:24 -07:00
test_reorganize_runner.py Library reorganize: FIFO queue with live status panel 2026-04-25 18:01:32 -07:00
test_repair_worker_album_fill.py Fix CI test failures from incomplete dummy config and encoding 2026-04-19 13:04:55 -07:00
test_request_cleanup_timer.py test: api/request periodic cleanup timer 2026-04-19 15:22:25 -07:00
test_runtime_state.py Tighten metadata and import safety 2026-04-27 20:28:05 +03:00
test_script_split_integrity.py Unify artist detail: route source artists to standalone page, retire inline Artists page 2026-04-22 17:00:52 -07:00
test_socketio_cors.py Socket.IO CORS: handle self-review nits 2026-04-26 19:24:43 -07:00
test_spa_deep_linking.py Rename Search page id from 'downloads' to 'search', bump to 2.43 2026-04-22 13:22:49 -07:00
test_sync_discovery_scan_events.py Realign test file names 2026-04-17 21:25:01 +03:00
test_tidal_auth_instructions.py Show Tidal callback port (not Spotify's) in auth instructions 2026-04-30 20:54:06 -07:00
test_tidal_search_shortening.py Reject Tidal streams that silently downgrade from the requested quality 2026-04-24 13:12:30 -07:00
test_tool_progress_events.py Realign test file names 2026-04-17 21:25:01 +03:00
test_track_artist_matching.py Add fallback negative-case test for track-artist matching 2026-04-30 17:13:15 -07:00
test_track_number_repair.py Fix CI test failures from incomplete dummy config and encoding 2026-04-19 13:04:55 -07:00
test_track_search_single_query.py test: api_search_tracks single-query track search 2026-04-19 15:22:24 -07:00
test_unknown_artist_fixer.py Fix test using old deezer_track_id column name 2026-04-20 21:11:45 -07:00
test_watchlist_album_match.py Stop watchlist re-downloading compilation tracks; catch slskd dedup orphans 2026-05-01 12:57:50 -07:00
test_watchlist_bulk_add.py Bulk watchlist add: fall back through every source ID, not just active 2026-04-30 20:27:42 -07:00
test_watchlist_scanner_scan.py Move the import flows and related post-processing pipelines into separate modules 2026-04-27 19:54:43 +03:00
test_websocket_client_transport.py Repoint websocket transport test to core.js after split 2026-04-22 11:30:39 -07:00
test_websocket_infrastructure.py Clarify Spotify auth actions 2026-05-01 10:36:50 +03:00
test_wishlist_pagination.py test: wishlist SQL pagination and category filter 2026-04-19 15:22:24 -07:00
test_worker_existing_id_marks_matched.py test: enrichment worker re-processing fix and migration backfill 2026-04-19 15:22:24 -07:00
test_worker_utils_album_track_count.py Fix Album Completeness job reporting zero findings for every album 2026-04-24 12:39:41 -07:00