soulsync/core
Broque Thomas e20994e1c7 Manual picks: stream results, don't auto-retry, fix stuck-at-0%
Three follow-on fixes to the manual-search candidates modal once people
started actually using it:

1. NDJSON streaming. Manual search waited for every source to return
   before showing anything. Now streams one event per source as each
   completes — header line, source_results per source, done terminator.
   Frontend appends rows incrementally via response.body.getReader().

2. Manual picks no longer auto-retry on failure. New _user_manual_pick
   flag set on the task in /download-candidate. Both monitor retry
   paths (not-in-live-transfers stuck + Errored state) bail on the
   flag. Surfaces the failure to the user instead of silently picking
   a different candidate via fresh search.

3. Non-Soulseek manual picks (youtube/tidal/qobuz/hifi/deezer/
   soundcloud/lidarr) no longer stuck at "downloading 0%" forever. The
   live_transfers IF branch now marks manual-pick tasks failed
   directly when the engine reports Errored, instead of deferring to
   the monitor (which bails on manual picks). Engine fallback in else
   branch covers the rare race where the orchestrator's pre-populated
   transfer lookup is missing the entry.

Plus a deadlock fix discovered along the way: the new failure path
synchronously called on_download_completed while holding tasks_lock,
which itself re-acquires the same Lock — non-reentrant
threading.Lock self-deadlocked the polling thread. While wedged, every
other endpoint that needed the lock (including /candidates → other
failed rows couldn't open modals) hung waiting. Moved completion
callbacks onto a daemon thread so the lock releases first.

Plus failed/not_found/cancelled rows are now ALWAYS clickable (not
just when the auto-search cached candidates) — the modal carries the
manual search bar, which is the user's recourse for empty results.

Plus manual download worker now runs on a dedicated thread instead of
competing with the batch's 3-worker missing_download_executor pool —
saturated batches no longer queue manual picks indefinitely.

All scoped to manual picks via the _user_manual_pick flag — auto
attempt flow byte-identical to before. Engine fallback gated on the
flag too so auto attempts in the else branch keep the original
do-nothing behavior (safety valve handles the stuck-forever case).

Also dropped _handle_failed_download from web_server.py — defined
but had no callers (dead code).

17 new unit tests pin the gate behavior:
- engine fallback: Errored/Cancelled/Succeeded/InProgress transitions,
  manual-pick gate, terminal-state skip, soulseek skip, missing
  download_id skip, engine returning None, orchestrator exception
- monitor: manual-pick skips not-in-live-transfers retry + Errored
  retry
- IF-branch end-to-end: Errored marks failed, "Completed, Errored"
  hits failure branch, auto attempts defer to monitor

Manual-search endpoint tests rewritten for NDJSON: 11 cases (validation,
single-source dispatch, parallel "all" dispatch, one-event-per-source
streaming shape, unconfigured-source skip + reject, header metadata,
per-source exception isolation).

Full suite 2259 passed, 1 skipped.
2026-05-08 15:12:58 -07:00
..
artists Surface silent exceptions in core/artists — 23 sites 2026-05-07 09:49:29 -07:00
automation Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
discovery Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
download_engine Add module logger + surface silent exceptions in 7 logger-less files — 12 sites 2026-05-07 10:27:04 -07:00
download_plugins Address Copilot doc-drift review 2026-05-05 15:46:48 -07:00
downloads Manual picks: stream results, don't auto-retry, fix stuck-at-0% 2026-05-08 15:12:58 -07:00
enrichment Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
imports Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
library Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
media_server Final review-pass nits — class docstring, dead branch, dead imports, boot resilience 2026-05-05 22:59:33 -07:00
metadata Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
playlists Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
repair_jobs Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
search Cin-6: Rename soulseek_client global → download_orchestrator 2026-05-04 23:23:32 -07:00
stats Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
streaming Cin-6: Rename soulseek_client global → download_orchestrator 2026-05-04 23:23:32 -07:00
watchlist Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
wishlist Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
workers Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
acoustid_client.py Clean up 286 ruff lint errors to unblock CI and fix 10 latent bugs 2026-04-21 13:30:52 -07:00
acoustid_verification.py Reject AcoustID matches whose version disagrees with the expected track 2026-05-07 13:25:30 -07:00
album_consistency.py Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
api_call_tracker.py Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
artist_source_detail.py Move metadata helpers into package modules 2026-04-29 11:28:42 +03:00
artist_source_lookup.py Make artist_name Optional in find_library_artist_for_source 2026-04-22 22:15:03 -07:00
audiodb_client.py Add API Rate Monitor dashboard with real-time speedometer gauges 2026-04-02 10:46:43 -07:00
audiodb_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
auto_import_worker.py Surface silent exceptions in import pipeline — 11 sites 2026-05-07 09:50:15 -07:00
automation_engine.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
connection_detect.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
connection_test.py Cin-6: Rename soulseek_client global → download_orchestrator 2026-05-04 23:23:32 -07:00
database_update_worker.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
debug_info.py Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
deezer_client.py Add download buttons + bulk action to artist top-tracks sidebar 2026-05-07 15:44:47 -07:00
deezer_download_client.py Surface silent exceptions in metadata clients — 37 sites 2026-05-07 09:33:38 -07:00
deezer_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
discogs_client.py Surface silent exceptions in metadata clients — 37 sites 2026-05-07 09:33:38 -07:00
discogs_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
download_orchestrator.py Lift shared download dataclasses + boot via singleton factory 2026-05-05 09:08:39 -07:00
genius_client.py Add API Rate Monitor dashboard with real-time speedometer gauges 2026-04-02 10:46:43 -07:00
genius_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
genre_filter.py Expand default genre whitelist from 223 to 272 genres 2026-04-18 20:32:49 -07:00
hifi_client.py Surface engine-not-wired errors + exclude soulseek from monitor aggregation 2026-05-05 12:20:51 -07:00
hydrabase_client.py Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
hydrabase_worker.py Improve graceful shutdown and rollback safety 2026-04-12 15:17:18 +03:00
itunes_client.py Surface silent exceptions in metadata clients — 37 sites 2026-05-07 09:33:38 -07:00
itunes_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
jellyfin_client.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
lastfm_client.py Add Last.fm Track Radio to Discover page 2026-04-14 23:41:36 -07:00
lastfm_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
library_reorganize.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
lidarr_download_client.py Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
listenbrainz_client.py Remove emojis from all Python log and print statements 2026-04-11 21:11:02 -07:00
listenbrainz_manager.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
listening_stats_worker.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
lyrics_client.py Remove emojis from all Python log and print statements 2026-04-11 21:11:02 -07:00
matching_engine.py Merge remote-tracking branch 'origin/dev' into refactor/media-server-engine 2026-05-05 20:36:31 -07:00
metadata_service.py Move profile Spotify cache into registry 2026-04-29 12:36:37 +03:00
musicbrainz_client.py MusicBrainz: Resolve release-group MBIDs to a release on album click 2026-04-24 08:48:02 -07:00
musicbrainz_search.py MusicBrainz: Dedupe same-named homonyms in artist search results 2026-04-24 10:27:45 -07:00
musicbrainz_service.py Add metadata cache maintenance and health monitoring 2026-03-30 07:40:20 -07:00
musicbrainz_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
navidrome_client.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
personalized_playlists.py Discover: sharpen track selection (diversity, source-aware popularity, library dedup, SQL genre) 2026-05-08 08:49:22 -07:00
plex_client.py Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
qobuz_client.py Address Copilot doc-drift review 2026-05-05 15:46:48 -07:00
qobuz_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
reorganize_queue.py Reorganize queue: race + dedupe fixes from kettui review 2026-04-26 08:40:24 -07:00
reorganize_runner.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
repair_worker.py Surface silent exceptions in repair_worker — 16 sites 2026-05-07 09:17:51 -07:00
replaygain.py Add module logger + surface silent exceptions in 7 logger-less files — 12 sites 2026-05-07 10:27:04 -07:00
runtime_state.py Add module logger + surface silent exceptions in 7 logger-less files — 12 sites 2026-05-07 10:27:04 -07:00
seasonal_discovery.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
socketio_cors.py Socket.IO CORS: handle self-review nits 2026-04-26 19:24:43 -07:00
soulid_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
soulseek_client.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
soulsync_client.py Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
soundcloud_client.py Surface engine-not-wired errors + exclude soulseek from monitor aggregation 2026-05-05 12:20:51 -07:00
spotify_client.py Add download buttons + bulk action to artist top-tracks sidebar 2026-05-07 15:44:47 -07:00
spotify_public_scraper.py Add Spotify Link tab for public playlist/album scraping without API credentials 2026-03-15 21:25:05 -07:00
spotify_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
tag_writer.py Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
tidal_client.py Surface silent exceptions in metadata clients — 37 sites 2026-05-07 09:33:38 -07:00
tidal_download_client.py Surface engine-not-wired errors + exclude soulseek from monitor aggregation 2026-05-05 12:20:51 -07:00
tidal_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
watchlist_scanner.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
web_scan_manager.py MS Cin-5: Drop per-server globals — engine owns the clients 2026-05-05 18:05:45 -07:00
wishlist_service.py Extract wishlist logic into dedicated package 2026-04-28 21:17:24 +03:00
worker_utils.py Fix Album Completeness job reporting zero findings for every album 2026-04-24 12:39:41 -07:00
youtube_client.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00