danvics/soulsync
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1
soulsync/core/wishlist
History
BoulderBadgeDad 0898014364 Fix #740: run wishlist album-bundle downloads on a dedicated pool 
A 2.6.3 change (c3b88e69) split the wishlist albums cycle into one batch per
album. Each album batch runs an INLINE-BLOCKING soulseek/torrent/usenet
album-bundle download (album_bundle_dispatch.try_dispatch ->
download_album_to_staging) that holds its worker thread for the whole
search+download. All of these were submitted to the shared 3-worker
missing_download_executor -- the same pool used for per-track downloads AND the
manual 'Download Wishlist' analysis.

So a large Album-Completeness 'Fix all' (-> ~819 wishlist tracks -> ~82 per-album
batches) saturated all 3 workers with blocking album downloads; the manual
wishlist analysis could never get a thread ('Library Analysis' stuck on
Pending), the other ~79 batches sat in phase='queued' forever, and auto-cleanup
(which only evicts terminal-phase batches) never cleared them -> jam until
restart. Fixing batch STATUS would not help: the threads are blocked inside the
download, not waiting on a phase flip.

Fix: add a dedicated bounded album_bundle_executor (max_workers=3) and route the
AUTO per-album bundle batches to it, keeping the shared pool free for analysis /
per-track / the manual wishlist (which always starts now). Hung/slow album
downloads can only delay other album downloads, never the user-facing path.
Additive and decoupled; the submit site falls back to the shared pool when the
album pool isn't wired (older callers / tests) so behavior is unchanged there.
The manual path is untouched (it already runs album bundles serially on a single
thread, by design).

Tests (tests/wishlist/test_automation.py): album sub-batches route to the
dedicated pool while the residual per-track batch stays on the shared pool;
fallback-to-shared-pool when no album pool is wired. Existing auto-processing
tests still green (fallback preserves prior behavior). 707 passed across
wishlist + downloads suites.
2026-05-29 14:31:10 -07:00
..
__init__.py Extract wishlist logic into dedicated package 2026-04-28 21:17:24 +03:00
album_grouping.py Wishlist: only engage album-bundle when multiple tracks from same album (PR 1/4) 2026-05-27 13:42:04 -07:00
classification.py Coerce wishlist track counts before category checks 2026-04-30 21:42:43 +03:00
payloads.py Wishlist: fix three regressions causing all imports to land as track 01 with no year 2026-05-27 15:39:22 -07:00
presence.py Add module logger + surface silent exceptions in 7 logger-less files — 12 sites 2026-05-07 10:27:04 -07:00
processing.py Fix #740: run wishlist album-bundle downloads on a dedicated pool 2026-05-29 14:31:10 -07:00
reporting.py Extract wishlist logic into dedicated package 2026-04-28 21:17:24 +03:00
resolution.py Make wishlist respect configured providers 2026-04-30 07:41:22 +03:00
routes.py Wishlist: harden Spotify backfill — poisoned tn=1 can't mask lean album 2026-05-27 16:47:26 -07:00
selection.py Make wishlist respect configured providers 2026-04-30 07:41:22 +03:00
service.py Accept wishlist track_data aliases 2026-04-30 07:59:43 +03:00
state.py Extract wishlist logic into dedicated package 2026-04-28 21:17:24 +03:00