soulsync/tests/downloads
Broque Thomas 5771c5ba77 Album-bundle staging: clean Soulseek copies + sweep orphans at startup
Two related leaks in ``storage/album_bundle_staging/<batch_id>/``:

1. **Soulseek bundle cleanup was excluded.** The per-batch cleanup
   at the end of a bundle download gated on:
       (album_bundle_source or '').lower() in ('torrent', 'usenet')
   The comment justified it as "slskd keeps its own completed
   folders" — but the Soulseek bundle path ALSO copies completed
   files into the private staging dir (``soulseek_client.py:1599``,
   ``copy_audio_files_atomically(completed, Path(staging_dir))``)
   for the per-track workers to claim. Those copies persisted
   forever; long-running installs accumulated stale GB. Extended
   the cleanup gate's allow-list to include ``soulseek`` so the
   per-batch dir is removed on bundle completion — same code path
   that already worked for torrent / usenet.

2. **No sweep for orphan dirs.** Any leftover ``<batch_id>``
   subdir from a previous-session crash, an errored batch, or a
   pre-fix Soulseek bundle stayed on disk forever. Added
   ``sweep_orphan_album_bundle_staging(staging_root, active_batch_ids)``
   that runs ONCE at server startup, before any batch can register
   a staging dir. Removes every ``<batch_id>``-shaped subdir
   whose id isn't in the active set. Safe by construction:
     - Only touches subdirs of the configured staging root.
     - Name-shape check (``entry.name == _safe_batch_dirname(entry.name)``)
       rejects hand-placed dirs like ``.git`` or stray docs.
     - ``shutil.rmtree`` errors log + continue — sweep must not
       crash app startup over a permission glitch.
     - active_batch_ids normalised through ``_safe_batch_dirname``
       so colon-bearing batch_ids match their on-disk form.
   Wired into the web_server startup right after the stuck-flags
   diagnostic so it fires before anything else touches batches.

Tests
- ``test_downloads_lifecycle.py`` gained one regression test
  pinning that Soulseek bundles now have their staging dir
  cleaned (sibling to the existing torrent test).
- ``test_album_bundle_staging_sweep.py`` (NEW, 11 tests)
  covers: orphan removal with no actives, active dirs preserved,
  special-char batch_id normalisation, no-op on missing /empty
  /empty-string staging root, non-dir entries skipped, unsafe-
  name dirs preserved (.git etc.), partial rmtree failure doesn't
  abort the rest, listdir failure returns 0 cleanly, default
  None active set, defensive against empty / None entries in
  the active set.

488 downloads tests pass.

For users with an existing "clean up old files" automation pointed
at this dir: stop pointing it there if you want — the auto-cleanup
+ startup sweep cover it now. Or leave it as belt-and-suspenders
with a relaxed (1h+) mtime threshold so it can't race a mid-batch
download.
2026-05-27 22:18:42 -07:00
..
__init__.py PR4a: lift sync history recording to core/downloads/history.py 2026-04-27 20:37:16 -07:00
test_album_bundle_staging_sweep.py Album-bundle staging: clean Soulseek copies + sweep orphans at startup 2026-05-27 22:18:42 -07:00
test_background_download_worker.py Add 5 test additions JohnBaumb suggested 2026-05-05 13:04:18 -07:00
test_deezer_pinning.py Surface engine-not-wired errors + exclude soulseek from monitor aggregation 2026-05-05 12:20:51 -07:00
test_download_engine.py Add 5 test additions JohnBaumb suggested 2026-05-05 13:04:18 -07:00
test_download_orchestrator.py Fix Usenet settings reload without restart 2026-05-22 08:28:56 -07:00
test_downloads_cancel.py PR4b: lift cancel + clear download routes to core/downloads/cancel.py 2026-04-27 21:41:35 -07:00
test_downloads_candidates.py Trust user manual picks past AcoustID verification (#701) 2026-05-26 09:56:49 -07:00
test_downloads_cleanup.py PR4c: lift _automatic_wishlist_cleanup_after_db_update to core/downloads/cleanup.py 2026-04-27 22:23:41 -07:00
test_downloads_history.py PR4a: lift sync history recording to core/downloads/history.py 2026-04-27 20:37:16 -07:00
test_downloads_lifecycle.py Album-bundle staging: clean Soulseek copies + sweep orphans at startup 2026-05-27 22:18:42 -07:00
test_downloads_master.py Harden album bundle fallback flow 2026-05-24 16:15:36 -07:00
test_downloads_post_processing.py Add torrent and usenet release staging support 2026-05-21 14:22:21 -07:00
test_downloads_staging.py Fix album-bundle staging match + wishlist provenance (#700, #698) 2026-05-26 07:12:49 -07:00
test_downloads_status.py Wishlist: distinguish Queued from Analyzing for executor-pending batches 2026-05-27 14:52:02 -07:00
test_downloads_task_worker.py Add Soulseek album bundle downloads 2026-05-23 15:08:21 -07:00
test_downloads_validation.py Add torrent and usenet release staging support 2026-05-21 14:22:21 -07:00
test_file_finder.py Fix: Soulseek album-bundle downloads stuck on "failed" after slskd 2026-05-27 21:20:37 -07:00
test_hifi_pinning.py Support legacy HiFi track manifests 2026-05-21 18:11:14 -07:00
test_lidarr_pinning.py A8: Pin LidarrDownloadClient download lifecycle behavior 2026-05-04 12:10:49 -07:00
test_qobuz_pinning.py Surface engine-not-wired errors + exclude soulseek from monitor aggregation 2026-05-05 12:20:51 -07:00
test_rate_limit_policy.py E1: Add RateLimitPolicy declaration mechanism 2026-05-04 14:38:20 -07:00
test_soulseek_pinning.py Add Soulseek album bundle downloads 2026-05-23 15:08:21 -07:00
test_soundcloud_pinning.py Surface engine-not-wired errors + exclude soulseek from monitor aggregation 2026-05-05 12:20:51 -07:00
test_tidal_pinning.py Handle non-JSON Tidal auth responses 2026-05-20 14:04:45 -07:00
test_track_metadata_backfill.py Wishlist: harden Spotify backfill — poisoned tn=1 can't mask lean album 2026-05-27 16:47:26 -07:00
test_wishlist_aggregator.py Wishlist modal: surface most-advanced live phase, not least-complete 2026-05-26 22:35:53 -07:00
test_youtube_pinning.py Surface engine-not-wired errors + exclude soulseek from monitor aggregation 2026-05-05 12:20:51 -07:00