soulsync/core
Broque Thomas f13d339584 Usenet album poll: tolerate SAB queue→history handoff, emit terminal failure (#706)
User reported usenet album downloads getting stuck on "downloading
release" while SABnzbd reported the job as complete. Container restart
did not help; reproducible on every usenet album download.

Three independent issues all causing the same symptom — the download
modal freezes mid-flow with no error surfaced to the user:

1. SAB queue → history transition window
   SAB removes a slot from its queue BEFORE adding it to the history,
   and on a busy server (par2 verify, unrar, multi-file move) that
   window can span several poll iterations. The poll treated a single
   None status as terminal failure ("disappeared from client") and
   gave up. Now the poll tolerates up to ~10s of consecutive misses
   (5 polls at the default 2s interval) before declaring the job gone.

2. SAB queue states like `Pp` were unmapped
   `_SAB_QUEUE_STATE_MAP` didn't cover SAB's `Pp` (post-processing
   summary), `Unpacking`, `Trying`, `Deleted`, or the `Prop_paused`
   / `Prop_failed` variants. Unmapped states fell through to the
   default-'error' fallback, and the poll loop only treated explicit
   'failed' / 'completed' as terminal — 'error' was neither, so the
   loop spun until the 6-hour timeout. Map now covers every Status
   value from SAB's `sabnzbd/api.py`, and the poll treats the default-
   'error' fallback as a transient miss (warn-logged, retry within
   the same tolerance window) so a brand-new unmapped state can't
   infinite-loop the way `Pp` did here.

3. No terminal failure emit
   The poll only logged on failure / timeout / disappeared — never
   called the progress callback with 'failed', so the download modal
   stayed at the last 'downloading' emit forever. Plumb a 'failed'
   emit through every failure exit path so the UI flips out of the
   downloading state when the poll gives up.

Plus:

4. SAB direct nzo_ids lookup instead of paging all-history
   `_get_status_sync` was fetching the latest 50 history entries on
   every poll and iterating to find the target nzo_id. On busy
   servers (many recent downloads), the target job could roll past
   the 50-entry window and look like a "disappeared" job. Replaced
   with a targeted `mode=queue&nzo_ids=<id>` → `mode=history&nzo_ids=<id>`
   chain. Falls back to the bulk path for SAB versions that pre-date
   the nzo_ids filter — the transient-miss tolerance covers any
   short-lived gap there too.

Implementation:

Lifted the album-bundle poll loop out of `usenet.py` and `torrent.py`
into `core/download_plugins/album_bundle.py:poll_album_download` —
near-duplicate implementations are now a single function with deps
injected so it's testable in isolation (kettui's extract-don't-AST-parse
standard; can't unit-test a `time.sleep` loop inside a plugin method).
The lifted helper takes:
- `get_status` callable bound to job_id, so the same loop works for
  usenet UsenetStatus and torrent TorrentStatus shapes
- `complete_states` set so torrent's `{'seeding', 'completed'}` and
  usenet's `{'completed'}` both Just Work
- `failed_states` set so torrent's `{'error'}` is terminal while
  usenet's default-'error' fallback is transient
- `transient_miss_threshold` (default 5 ≈ 10s at 2s poll)
- `sleep` / `monotonic` injectables for deterministic tests

Per-track flows in both plugins gained the same transient-miss
tolerance inline — they don't use the emit pattern (update an
`active_downloads[id]` row dict via lock instead), so reusing the
helper would have required threading a no-op emit through. Inline
fix is small enough.

Tests:
- 11 new tests in `tests/test_album_bundle.py:poll_album_download`
  cover the happy path, transient-miss tolerance with recovery,
  hard-failure threshold, explicit-failed surface, timeout-emit,
  default-'error' transient treatment, shutdown clean exit,
  torrent's `seeding`-counts-as-complete, save_path captured across
  iterations, and adapter-exception treated as transient miss.
- 521 download-suite tests pass (33 in test_album_bundle, others
  pin existing torrent + usenet contracts).
- Ruff clean.

Closes #706.
2026-05-27 09:42:51 -07:00
..
artists Complete MusicBrainz watchlist source parity 2026-05-18 19:19:25 -07:00
automation Fold provider-matching into PlaylistSource contract (Phase 1b) 2026-05-26 13:07:01 -07:00
discovery Branch cleanup: lift manual-match helpers, fix length-pref ordering, profile-scope view toggle 2026-05-27 07:43:21 -07:00
download_engine Add module logger + surface silent exceptions in 7 logger-less files — 12 sites 2026-05-07 10:27:04 -07:00
download_plugins Usenet album poll: tolerate SAB queue→history handoff, emit terminal failure (#706) 2026-05-27 09:42:51 -07:00
downloads Wishlist modal: surface most-advanced live phase, not least-complete 2026-05-26 22:35:53 -07:00
enrichment Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
imports Fix album-bundle downloads landing every track as track 1 2026-05-26 21:04:27 -07:00
library fix(musicbrainz): user-facing search recall + album-detail 404 2026-05-19 15:38:24 -07:00
matching AcoustID + quarantine modal: three bug fixes (closes #607, closes #608) 2026-05-15 08:55:06 -07:00
media_server Server playlist sync: append mode (preserve user-added tracks) 2026-05-10 22:52:11 -07:00
metadata MB rerank: prefer_known_duration is now a score boost, not a tiebreaker 2026-05-27 08:13:28 -07:00
personalized Personalized pipeline: auto-refresh stale snapshots after watchlist scan 2026-05-15 20:53:03 -07:00
playlists Resolve pre-existing ruff lint errors blocking CI 2026-05-27 08:33:36 -07:00
repair_jobs Fix Picard albumartist orphan false positives 2026-05-22 08:43:57 -07:00
search feat(import): show MusicBrainz variants 2026-05-24 21:29:22 +03:00
stats Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
streaming Cin-6: Rename soulseek_client global → download_orchestrator 2026-05-04 23:23:32 -07:00
sync Persist Find & Add selections as permanent server-playlist match overrides 2026-05-14 09:39:24 -07:00
text Address Cin review: extract helpers, indexed pool fetch, tidy nits 2026-05-24 23:33:55 -07:00
torrent_clients fix(torrent): use before/after diff to recover qBit info-hash 2026-05-20 18:11:30 -07:00
usenet_clients Usenet album poll: tolerate SAB queue→history handoff, emit terminal failure (#706) 2026-05-27 09:42:51 -07:00
watchlist Complete MusicBrainz watchlist source parity 2026-05-18 19:19:25 -07:00
webui Extract WebUI asset helpers 2026-05-13 22:26:25 +03:00
wishlist Wishlist: reify run id + gate cycle toggle on last-sibling completion 2026-05-26 21:50:42 -07:00
workers Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
acoustid_client.py Clean up 286 ruff lint errors to unblock CI and fix 10 latent bugs 2026-04-21 13:30:52 -07:00
acoustid_verification.py AcoustID + quarantine modal: three bug fixes (closes #607, closes #608) 2026-05-15 08:55:06 -07:00
album_consistency.py Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
amazon_client.py fix(amazon): search albums/artists and track numbers for t2tunes 2026-05-19 14:04:26 -07:00
amazon_download_client.py Amazon download client: write final size==transferred before returning file path 2026-05-16 11:09:04 -07:00
amazon_worker.py Harden Amazon worker schema migration 2026-05-18 15:47:04 -07:00
api_call_tracker.py Wire Amazon Music as a first-class download source 2026-05-16 09:40:50 -07:00
archive_pipeline.py fix(downloads): harden album bundle staging 2026-05-20 21:39:06 -07:00
artist_source_detail.py Fix MusicBrainz artist detail showing MBID as name 2026-05-19 12:34:35 -07:00
artist_source_lookup.py Add amazon_id column to artists table for full source parity 2026-05-16 17:06:54 -07:00
audiodb_client.py Add API Rate Monitor dashboard with real-time speedometer gauges 2026-04-02 10:46:43 -07:00
audiodb_worker.py AudioDB worker: stop infinite loop on direct-ID lookup failure (#553) 2026-05-11 14:18:55 -07:00
auto_import_worker.py Auto-import: fall through to other metadata sources when primary returns no match 2026-05-12 12:32:18 -07:00
automation_engine.py Merge pull request #594 from Skowll/telegram_thread_id 2026-05-26 11:47:08 -07:00
connection_detect.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
connection_test.py Add torrent and usenet release staging support 2026-05-21 14:22:21 -07:00
database_update_worker.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
debug_info.py Fix Copy Debug Info music_source + surface missing services 2026-05-12 16:47:55 -07:00
deezer_client.py Fix Deezer contributors tagging silently dropping for cache-polluted tracks 2026-05-14 11:10:51 -07:00
deezer_download_client.py Surface silent exceptions in metadata clients — 37 sites 2026-05-07 09:33:38 -07:00
deezer_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
discogs_client.py Discogs: strip artist disambiguation suffixes at every name surface (#634) 2026-05-27 09:05:47 -07:00
discogs_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
download_orchestrator.py Fix Usenet settings reload without restart 2026-05-22 08:28:56 -07:00
genius_client.py Add API Rate Monitor dashboard with real-time speedometer gauges 2026-04-02 10:46:43 -07:00
genius_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
genre_filter.py Expand default genre whitelist from 223 to 272 genres 2026-04-18 20:32:49 -07:00
hifi_client.py Log exception when inferring HiFi manifest ext 2026-05-21 18:24:56 -07:00
hydrabase_client.py feat: add explicit badges to discography modal and artist-detail cards 2026-05-12 23:35:08 -04:00
hydrabase_worker.py Improve graceful shutdown and rollback safety 2026-04-12 15:17:18 +03:00
image_cache.py fix(lint): silence ruff B007 + S110 2026-05-20 16:18:55 -07:00
itunes_client.py feat: add explicit badges to discography modal and artist-detail cards 2026-05-12 23:35:08 -04:00
itunes_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
jellyfin_client.py Server playlist sync: append mode (preserve user-added tracks) 2026-05-10 22:52:11 -07:00
lastfm_client.py Add Last.fm Track Radio to Discover page 2026-04-14 23:41:36 -07:00
lastfm_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
library_reorganize.py Reorganize: optional embedded-tag mode (closes #592) 2026-05-15 07:56:18 -07:00
lidarr_download_client.py Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
listenbrainz_client.py Remove emojis from all Python log and print statements 2026-04-11 21:11:02 -07:00
listenbrainz_manager.py Resolve pre-existing ruff lint errors blocking CI 2026-05-27 08:33:36 -07:00
listening_stats_worker.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
lyrics_client.py Retag now re-embeds LYRICS tag instead of leaving it empty 2026-05-14 15:52:05 -07:00
matching_engine.py Merge remote-tracking branch 'origin/dev' into refactor/media-server-engine 2026-05-05 20:36:31 -07:00
metadata_service.py Add MusicBrainz as a metadata source 2026-05-18 18:47:13 -07:00
musicbrainz_client.py Show MusicBrainz release variants in import 2026-05-24 09:33:19 -07:00
musicbrainz_search.py Branch cleanup: lift manual-match helpers, fix length-pref ordering, profile-scope view toggle 2026-05-27 07:43:21 -07:00
musicbrainz_service.py Cross-script artist aliases: include canonical name + non-strict fallback 2026-05-14 13:07:15 -07:00
musicbrainz_worker.py Alias resolution polish: lazy-fire on direct-match failure + worker backfill 2026-05-10 17:02:02 -07:00
navidrome_client.py Preserve Navidrome album cover art 2026-05-25 19:51:38 -07:00
personalized_playlists.py Add MusicBrainz as a metadata source 2026-05-18 18:47:13 -07:00
plex_client.py Server playlist sync: append mode (preserve user-added tracks) 2026-05-10 22:52:11 -07:00
prowlarr_client.py feat(settings): add Prowlarr integration as indexer aggregator 2026-05-20 14:41:54 -07:00
qobuz_client.py Fetch all Qobuz favorite tracks for discovery 2026-05-24 01:17:38 -07:00
qobuz_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
reorganize_queue.py Reorganize: optional embedded-tag mode (closes #592) 2026-05-15 07:56:18 -07:00
reorganize_runner.py Reorganize: optional embedded-tag mode (closes #592) 2026-05-15 07:56:18 -07:00
repair_worker.py fix(album-completeness): block cross-artist auto-fill 2026-05-21 16:49:12 -07:00
replaygain.py Add module logger + surface silent exceptions in 7 logger-less files — 12 sites 2026-05-07 10:27:04 -07:00
runtime_state.py Add module logger + surface silent exceptions in 7 logger-less files — 12 sites 2026-05-07 10:27:04 -07:00
seasonal_discovery.py Fix source artist detail navigation from discover modals 2026-05-18 13:50:10 -07:00
socketio_cors.py Socket.IO CORS: handle self-review nits 2026-04-26 19:24:43 -07:00
soulid_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
soulseek_client.py Add Soulseek album bundle downloads 2026-05-23 15:08:21 -07:00
soulsync_client.py Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
soundcloud_client.py Surface engine-not-wired errors + exclude soulseek from monitor aggregation 2026-05-05 12:20:51 -07:00
spotify_client.py Add download buttons + bulk action to artist top-tracks sidebar 2026-05-07 15:44:47 -07:00
spotify_public_scraper.py Add Spotify Link tab for public playlist/album scraping without API credentials 2026-03-15 21:25:05 -07:00
spotify_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
tag_writer.py AcoustID scanner: multi-candidate match + duration guard + multi-value retag 2026-05-14 14:09:38 -07:00
tidal_client.py Your Albums: selectable wishlist modal + Tidal album resolution 2026-05-11 12:36:16 -07:00
tidal_download_client.py Handle non-JSON Tidal auth responses 2026-05-20 14:04:45 -07:00
tidal_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
watchlist_scanner.py Fix missing album art for non-Spotify sources + animate Downloads nav icon 2026-05-18 20:24:13 -07:00
web_scan_manager.py MS Cin-5: Drop per-server globals — engine owns the clients 2026-05-05 18:05:45 -07:00
wishlist_service.py Extract wishlist logic into dedicated package 2026-04-28 21:17:24 +03:00
worker_utils.py Repair stale media schema during refresh 2026-05-21 17:41:54 -07:00
youtube_client.py Escape dash-leading YouTube search queries 2026-05-24 08:54:47 -07:00