Commit graph

9 commits

Author SHA1 Message Date
dev
70732ad80e Downloads: lazy multi-query quarantine retry — exhaust all queries per source
The cached-first retry (8d98b755) abandoned a source after a single query:
the first run returns as soon as ONE query starts a download, so
cached_candidates held only that query's results. On a quarantine retry the
whole source was then excluded from re-search (via searched_sources), so the
later queries (e.g. "artist + album") never hit that source again — it jumped
to the next source after one query instead of exhausting all queries per
source.

Track searched QUERIES (searched_queries) instead of whole sources. A
quarantine retry now skips only the already-run queries (their candidates are
walked via cached-first) and still searches the not-yet-run queries against the
same source. Budget-exhausted sources (exhaustive mode) stay excluded, so the
source switch still fires when a source is genuinely spent.

Removes the now-dead searched_sources state (written but no longer read).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-06 16:44:14 +02:00
dev
07ca7eacfa Downloads: cached-first quarantine retry — stop re-searching the same source
Each AcoustID/integrity quarantine retry re-ran the FULL search (all queries,
all sources) before picking the next-best candidate — so a track that failed
verification a dozen times re-queried Soulseek a dozen times (~3 min/cycle in
the field). The next-best pick was already sitting in cached_candidates.

Now the monitor flags the re-queue as a quarantine retry; the worker walks the
already-found candidates first (skipping used + budget-exhausted sources) and
hands them straight to the download path — no search. A source is searched
exactly once: once its candidates are cached, later quarantine retries exclude
it (searched_sources) so the hybrid chain falls through to a not-yet-searched
source instead of re-querying the spent one. Fresh downloads and the monitor's
dead-connection/stuck retries clear searched_sources and search fresh, so the
only re-search is for a genuinely new source or a dead peer.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-06 16:44:14 +02:00
BoulderBadgeDad
2824c25ec6 Album bundle: let Soulseek staging-misses fall through to per-track/cross-source fallback (#743)
A Soulseek album bundle stages whichever single folder scored best. If that
folder doesn't contain every track the album needs, the missing tracks were
marked not_found with no fallback — even in hybrid mode where later sources
(Deezer, YouTube, etc.) could fill them. The staging-miss short-circuit fired
for Soulseek because 'soulseek' was lumped into the torrent/usenet source set
when album bundles were added, and album_bundle_partial only reflects whether
the files found IN the folder downloaded, not whether the folder had every
needed track.

Drop 'soulseek' from the short-circuit (keep torrent/usenet). A track not
claimed from the staged Soulseek folder now falls through to the normal
per-track Soulseek search and, in hybrid mode, onward down the configured
chain. Unlike torrent/usenet — where per-track search re-adds the same
release — Soulseek per-track search is a genuine per-file network search, so
this is correct and cheap. Realizes the original author's stated intent
('keep partial bundles from blocking per-track fallback') robustly, since the
partial flag couldn't detect a folder that was simply missing tracks.

Only affects tracks NOT claimed from staging — fully-staged albums claim every
track via try_staging_match and never reach this gate, so working albums are
unchanged. Likely also mitigates #755 (all-album-import failures now fall
through to per-track instead of dying).

Tests: rewrote the two Soulseek staged-miss tests to assert fall-through
(single + hybrid-first); kept the torrent guard; added a usenet guard test.
2026-05-31 11:13:08 -07:00
Broque Thomas
6c226613bf Add Soulseek album bundle downloads
Route primary Soulseek album downloads through the album-bundle staging flow, reusing preflight-selected folders when available. In hybrid mode, only the first configured source can claim whole-album bundle behavior so later Soulseek fallback keeps the existing per-track/source-reuse path.

Allow Soulseek album bundles to stage completed tracks when some same-source transfers fail or time out, and keep partial bundles from blocking per-track fallback. Add coverage for dispatcher gating, master flow ordering, task-worker staged-miss behavior, and Soulseek bundle polling.
2026-05-23 15:08:21 -07:00
Broque Thomas
dee200b267 Add torrent usenet PR notes and test updates
Adds the PR description for the torrent/usenet release-staging feature and updates drifted tests for the new plugin registry entries and search exclude_sources signature.

Verified with the focused pytest command covering cancellation and default registry source registration.
2026-05-21 15:28:48 -07:00
Broque Thomas
6c9b43225a Add torrent and usenet release staging support
Adds torrent/usenet as release-oriented download sources with album-bundle staging, live progress reporting, and post-processing that selects the requested audio file from completed releases instead of blindly importing the first file.

Keeps album-bundle behavior gated to single-source torrent/usenet album downloads, excludes release sources from hybrid album per-track searches, and allows hybrid non-album tracks to use release results safely.

Improves staged-release matching for featured/bonus track filenames while preserving version mismatches, records torrent/usenet provenance in library history, and updates service/status UI labels.

Covers the flow with focused lifecycle, status, staging, validation, task worker, post-processing, and import side-effect tests.
2026-05-21 14:22:21 -07:00
Broque Thomas
2aff3dc210 Filter SoundCloud previews at every entry point + fix hybrid fallback regression
The earlier validation-only filter only ran in the auto-search
scoring path. SoundCloud preview snippets still leaked through:

- The candidate-review modal cached raw search results (pre-validation),
  so previews were visible and clickable for manual retry — and the
  manual-pick download path bypassed validation entirely, downloading
  the preview anyway.
- The not-found raw-results cache stored unfiltered top-20s.

Lift the preview filter into a reusable filter_soundcloud_previews()
helper and apply it at every entry point: validation scoring (still),
modal-cache fallback when validation drops everything, and the
not-found raw-results path. Previews now never reach the cache, the
matcher, or the manual-pick UI. Drops candidates < 35s or below half
the expected duration, gated on expected > 60s so genuine short
tracks still pass. 7 new unit tests pin the helper.

Also fixed a silent regression in core/downloads/task_worker.py's
hybrid-fallback path. Cin-5 dropped the per-source attrs from the
orchestrator (orch.soulseek, orch.youtube, etc.), but the fallback
loop still resolved sources via getattr(orch, '<src>', None) — every
lookup silently returned None, so remaining_sources came back empty
and the fallback never ran. Now uses orch.client(name) like the rest
of the codebase. Updated the test fake to expose client() too — the
old test was passing because the loop was effectively dead.
2026-05-05 10:26:25 -07:00
Broque Thomas
61ba3a15de Cin-6: Rename soulseek_client global → download_orchestrator
The global handle in web_server.py was named soulseek_client for
historical reasons but the type has long been DownloadOrchestrator,
not SoulseekClient. Renamed the global plus every parameter/attribute
that carried the legacy name.

- web_server.py: global var renamed; all 99 references updated.
- api/, core/downloads/*, core/search/*, core/streaming/*,
  services/sync_service.py: parameter names, dataclass fields, and
  init() arg names renamed.
- Test fixtures (CandidatesDeps, MasterDeps, SearchDeps, etc.) and
  the _build_deps helpers updated accordingly.

The core.soulseek_client module path and SoulseekClient class name
(the actual soulseek-only client) are unchanged — only the orchestrator
handle renamed. Module imports of TrackResult/AlbumResult/DownloadStatus
from core.soulseek_client preserved.
2026-05-04 23:23:32 -07:00
Broque Thomas
f0955420c3 PR4f: lift _download_track_worker to core/downloads/task_worker.py
Sixth sub-PR in the download orchestrator series. Strict 1:1 lift —
zero behavior change. ~333 lines moved out of web_server.py.

What moved:
- _download_track_worker → download_track_worker

Dependencies bundled in `TaskWorkerDeps` (10 callbacks):
- soulseek_client (with .mode + .hybrid_order + subclient attrs for hybrid
  fallback: .soulseek/.youtube/.tidal/.qobuz/.hifi/.deezer_dl)
- matching_engine (.generate_download_queries)
- run_async
- try_source_reuse, store_batch_source, try_staging_match,
  get_valid_candidates, attempt_download_with_candidates,
  recover_worker_slot (web_server.py helpers — large, will lift in
  follow-up PRs)
- on_download_completed (deferred to PR4g batch lifecycle)

Direct imports for already-lifted: download_tasks, tasks_lock from
core.runtime_state. SpotifyTrack from core.spotify_client.

Behavior parity:
- Same control flow: missing-task short-circuit → cancellation
  checkpoint with V2/legacy split → SpotifyTrack reconstruction with
  artist/album normalization → source-reuse shortcut → staging-match
  shortcut → searching-state init → query generation (matching
  engine + legacy fallbacks: track+first-artist-word with The-prefix
  handling, track-only, paren/bracket-cleaned) → case-insensitive
  dedup → sequential query loop with cancellation checks before/
  during/after each search → hybrid fallback across remaining sources
  using first 2 queries → not_found marking with diagnostics → 2-tier
  exception recovery (failed marking + emergency worker_slot recovery)
- Same logger messages text-for-text (so log filters keep working)
- Same locking pattern (tasks_lock around every download_tasks read/
  write, with the 2.0s timeout fallback in the exception path)
- Same `cached_candidates` storage for retry fallback + raw-results
  storage for candidate review modal (top 20 per query without valid
  candidates)
- Same V2 detection via `playlist_id` field — V2 tasks don't trigger
  on_download_completed for cancellation (V2 atomic cancel handles
  the worker slot itself)

Tests: 19 new under tests/downloads/test_downloads_task_worker.py
covering early-return guards (missing/cancelled-V2/cancelled-legacy/
cancelled-no-batch), source reuse + staging shortcuts, search loop
happy path, no-results not_found, raw-results-stored-when-no-valid-
candidates, attempt-download-failure-falls-through, cancellation mid-
flight returns without completion, hybrid fallback (with + without
hybrid mode), critical exception with + without recovery callback,
query generation edge cases (The-prefix, paren cleanup, dedup).

Full suite: 1001 passing (was 982). Ruff clean.
2026-04-28 07:50:01 -07:00