Adds an opt-in `rank_candidates_by_quality` profile flag. When on, the
priority-mode download walk orders candidates by the ranked-target quality
(confidence/speed only break ties) instead of confidence-first. Default off
keeps the byte-for-byte old behaviour, so existing installs are unaffected.
Best-quality search mode is always quality-first regardless of the flag; the
toggle only affects priority mode. Search-time source selection is unchanged —
nothing is skipped, so a track can never go missing, only the order in which
copies are tried changes.
The version-mismatch force-import follows automatically: it accepts the
first-tried (= best-ordered) quarantined candidate, which is the highest-quality
one once the walk is quality-first. No change to its selection logic needed.
- core/quality/selection.py: load_rank_candidates_by_quality() (fail-closed).
- core/downloads/task_worker.py: _best_quality_ordering -> _candidate_ordering;
quality-first when best_quality mode OR the toggle is on.
- database/music_database.py: default profile carries the flag (False).
- web_server.py: flag is preserved globally across preset apply/reset, like
search_mode.
- core/imports/version_mismatch_fallback.py: comment clarified (no behaviour
change).
Tests (TDD): load_rank_candidates_by_quality default/enabled/disabled/error;
_candidate_ordering across all mode+toggle combinations + fail-closed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds an opt-in search strategy toggle in the Quality Profile:
- priority (default): unchanged — first source in the hybrid chain that
meets a quality target wins.
- best_quality: pool candidates from EVERY source per query and download
them best→worst by actual audio quality; source order only breaks ties.
Implementation reuses existing plumbing so the retry system is untouched:
- engine.search_all_sources pools raw tracks across all configured,
non-exhausted sources (no first-source short-circuit).
- candidates.order_candidates: new quality_first sort path — profile
quality rank dominates, confidence/peer signals break ties. Priority
path is byte-for-byte unchanged (regression-locked by tests).
- task_worker passes quality_first + targets through; skips the redundant
hybrid-fallback block in best-quality mode (pool already covered it).
- Per-source retry budgets unchanged: a source that spends its budget is
added to exhausted_download_sources and thus dropped from the whole
pool. Independent of post_processing.retry_exhaustive.
- Query generator NOT touched.
Also clarifies the "Allow fallback" setting wording: it accepts OFF-LIST
quality as a last resort (not "walk down my list"), and notes that
lossy_copy.downsample_hires also bypasses the quality gate — the cause of
16-bit/MP3 files slipping through a 24-bit-only profile.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Review findings from PR #801, fixed as promised after merge:
- core/imports/version_mismatch_fallback.py and core/downloads/task_worker.py
used bare getLogger(__name__) — outside the soulsync.* namespace where
handlers attach, so the entire retry story (the [Modal Worker] search/retry
walk and, critically, the "accepting best quarantined candidate as last
resort" warning) never reached app.log. Same bug class as the prepare.py
fix; both moved to get_logger. A repo sweep shows 61 more modules with the
same pattern — noted as its own cleanup project.
- the full-suite run also caught a miss of MINE, not the PR's: the new
origin-history.js wasn't registered in the script-split integrity test, so
openDownloadOriginsModal failed onclick coverage. Registered — and the
onclick scan now iterates the NON_SPLIT_JS registry instead of its own
hardcoded copy, so the next standalone module can't silently skip coverage.
Merged dev verified: PR's 77 tests + 4233 full-suite tests pass (the only
exclusion is the eternal soundcloud /app file); integrity suite 64/64.
The existing fallback (pipeline.py:1084) only ran inside
post_process_matched_download_with_verification — i.e. when a file *was*
downloaded and AcoustID retries were fully exhausted. If the retry *search*
itself found zero valid candidates (source returned nothing, or all failed
HiFi validation), the task was marked not_found and the fallback was never
reached, even though the quarantine already held N version-mismatch entries.
Fix: add try_version_mismatch_fallback to TaskWorkerDeps; in the "no valid
candidates" path of task_worker, invoke it before marking not_found when
is_quarantine_retry. Wired in _build_task_worker_deps via a new helper
(_try_version_mismatch_fallback_for_worker) that calls
try_accept_version_mismatch_fallback directly with the track's title and
artist and a reprocess lambda over _post_process_matched_download_with_verification.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The cached-first retry (8d98b755) abandoned a source after a single query:
the first run returns as soon as ONE query starts a download, so
cached_candidates held only that query's results. On a quarantine retry the
whole source was then excluded from re-search (via searched_sources), so the
later queries (e.g. "artist + album") never hit that source again — it jumped
to the next source after one query instead of exhausting all queries per
source.
Track searched QUERIES (searched_queries) instead of whole sources. A
quarantine retry now skips only the already-run queries (their candidates are
walked via cached-first) and still searches the not-yet-run queries against the
same source. Budget-exhausted sources (exhaustive mode) stay excluded, so the
source switch still fires when a source is genuinely spent.
Removes the now-dead searched_sources state (written but no longer read).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Each AcoustID/integrity quarantine retry re-ran the FULL search (all queries,
all sources) before picking the next-best candidate — so a track that failed
verification a dozen times re-queried Soulseek a dozen times (~3 min/cycle in
the field). The next-best pick was already sitting in cached_candidates.
Now the monitor flags the re-queue as a quarantine retry; the worker walks the
already-found candidates first (skipping used + budget-exhausted sources) and
hands them straight to the download path — no search. A source is searched
exactly once: once its candidates are cached, later quarantine retries exclude
it (searched_sources) so the hybrid chain falls through to a not-yet-searched
source instead of re-querying the spent one. Fresh downloads and the monitor's
dead-connection/stuck retries clear searched_sources and search fresh, so the
only re-search is for a genuinely new source or a dead peer.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
In exhaustive retry mode, a source that spent its whole per-source budget
(query_count × retries_per_query) gave up and failed the track outright —
never trying the other configured sources. For tracks where Soulseek has a
deep pool of wrong peers (e.g. an AcoustID title mismatch every copy shares),
the budget tripped long before HiFi/Tidal/… were ever reached.
Now, when a source's budget is spent, the monitor marks it exhausted on the
task and re-queues so the worker excludes it from the next hybrid search,
falling through to the next source in the chain. Each new source spends its
own fresh budget. The task only fails once no fallback source remains (or the
absolute total ceiling trips) — single-source mode still fails immediately,
since there's nothing to fall back to.
task_worker folds the exhausted-source set into both the orchestrator search
exclusion and the hybrid-fallback source list.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds an opt-in exhaustive mode to the quarantine-retry path. Default
behaviour is unchanged: a single global cap (MAX_QUARANTINE_RETRIES=5).
When post_processing.retry_exhaustive is on, each source gets its OWN
retry budget sized as query_count x retries_per_query. Soulseek peers
collapse to one 'soulseek' bucket; streaming plugins keep their name.
The worker now records query_count on the task; the budget scales with
the track's real query count. Loop protection is threefold: per-source
cap, used_sources exhaustion (the natural terminator), and an absolute
ceiling (MAX_TOTAL_QUARANTINE_RETRIES=100).
New settings (config + WebUI): retry_next_candidate_on_mismatch (master),
retry_exhaustive, retries_per_query (default 5).
Tests: 6 new cases covering per-source budgeting, source separation,
Soulseek-peer bucketing, query_count default, and the absolute ceiling.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A Soulseek album bundle stages whichever single folder scored best. If that
folder doesn't contain every track the album needs, the missing tracks were
marked not_found with no fallback — even in hybrid mode where later sources
(Deezer, YouTube, etc.) could fill them. The staging-miss short-circuit fired
for Soulseek because 'soulseek' was lumped into the torrent/usenet source set
when album bundles were added, and album_bundle_partial only reflects whether
the files found IN the folder downloaded, not whether the folder had every
needed track.
Drop 'soulseek' from the short-circuit (keep torrent/usenet). A track not
claimed from the staged Soulseek folder now falls through to the normal
per-track Soulseek search and, in hybrid mode, onward down the configured
chain. Unlike torrent/usenet — where per-track search re-adds the same
release — Soulseek per-track search is a genuine per-file network search, so
this is correct and cheap. Realizes the original author's stated intent
('keep partial bundles from blocking per-track fallback') robustly, since the
partial flag couldn't detect a folder that was simply missing tracks.
Only affects tracks NOT claimed from staging — fully-staged albums claim every
track via try_staging_match and never reach this gate, so working albums are
unchanged. Likely also mitigates #755 (all-album-import failures now fall
through to per-track instead of dying).
Tests: rewrote the two Soulseek staged-miss tests to assert fall-through
(single + hybrid-first); kept the torrent guard; added a usenet guard test.
Route primary Soulseek album downloads through the album-bundle staging flow, reusing preflight-selected folders when available. In hybrid mode, only the first configured source can claim whole-album bundle behavior so later Soulseek fallback keeps the existing per-track/source-reuse path.
Allow Soulseek album bundles to stage completed tracks when some same-source transfers fail or time out, and keep partial bundles from blocking per-track fallback. Add coverage for dispatcher gating, master flow ordering, task-worker staged-miss behavior, and Soulseek bundle polling.
Adds torrent/usenet as release-oriented download sources with album-bundle staging, live progress reporting, and post-processing that selects the requested audio file from completed releases instead of blindly importing the first file.
Keeps album-bundle behavior gated to single-source torrent/usenet album downloads, excludes release sources from hybrid album per-track searches, and allows hybrid non-album tracks to use release results safely.
Improves staged-release matching for featured/bonus track filenames while preserving version mismatches, records torrent/usenet provenance in library history, and updates service/status UI labels.
Covers the flow with focused lifecycle, status, staging, validation, task worker, post-processing, and import side-effect tests.
When a user picks Hybrid mode AND downloads an album, the per-track
search loop fires once per track. Torrent / usenet are release-level
sources — Prowlarr returns album torrents, none of which score
meaningfully against an individual track title. Without filtering,
every track triggered a redundant Prowlarr search, qBit rejected
duplicate hashes after the first, and the run only worked at all
because Auto-Import swept Staging behind the scenes. Confusing
logs, wasted searches, brittle timing.
Fix: thread an optional ``exclude_sources`` parameter through
``DownloadOrchestrator.search``. When the per-track worker detects
that the active batch is an album AND mode is hybrid, it passes
``['torrent', 'usenet']`` so the hybrid chain skips them and falls
through to per-track-compatible sources (Soulseek / streaming).
Gate is narrow on purpose:
- Hybrid + album → skip torrent / usenet (THIS fix)
- Single-source torrent / usenet + album → album-bundle flow on
the master worker (already shipped)
- Hybrid + single-track batch (basic search / wishlist / playlist
of singles) → torrent / usenet still tried, validation.py's
album-name fallback gives them a shot
Excluded list logged at INFO when applied so the behavior is
visible in logs ("Hybrid search: excluding ['torrent', 'usenet']
for this query"). Default ``exclude_sources=None`` keeps every
non-task-worker caller (basic search, stream search, search-and-
download-best, automation handlers) on the original code path.
`validation.py` had amazon absent from `_streaming_sources`, causing
Amazon TrackResult objects (bitrate=None, size=0) to fall through to
the Soulseek P2P code path and get rejected by
`filter_results_by_quality_preference`. Every album track was marked
not found.
Fix: add 'amazon' to every streaming-source guard tuple/set that was
previously missing it:
- core/downloads/validation.py — primary bug fix (quality-filter bypass)
- core/downloads/status.py — _STREAMING_SOURCE_NAMES frozenset
- core/downloads/task_worker.py — hybrid fallback client map
- core/imports/side_effects.py — || filename→stream-id extraction
- web_server.py — is_streaming_source, transfer list display,
candidate source label, _try_source_reuse, _store_batch_source
- tests/test_download_plugin_conformance.py — registry count + parametrize
Also updates the 2.5.3 What's New entry to drop the stale
"not yet wired" disclaimer.
The earlier validation-only filter only ran in the auto-search
scoring path. SoundCloud preview snippets still leaked through:
- The candidate-review modal cached raw search results (pre-validation),
so previews were visible and clickable for manual retry — and the
manual-pick download path bypassed validation entirely, downloading
the preview anyway.
- The not-found raw-results cache stored unfiltered top-20s.
Lift the preview filter into a reusable filter_soundcloud_previews()
helper and apply it at every entry point: validation scoring (still),
modal-cache fallback when validation drops everything, and the
not-found raw-results path. Previews now never reach the cache, the
matcher, or the manual-pick UI. Drops candidates < 35s or below half
the expected duration, gated on expected > 60s so genuine short
tracks still pass. 7 new unit tests pin the helper.
Also fixed a silent regression in core/downloads/task_worker.py's
hybrid-fallback path. Cin-5 dropped the per-source attrs from the
orchestrator (orch.soulseek, orch.youtube, etc.), but the fallback
loop still resolved sources via getattr(orch, '<src>', None) — every
lookup silently returned None, so remaining_sources came back empty
and the fallback never ran. Now uses orch.client(name) like the rest
of the codebase. Updated the test fake to expose client() too — the
old test was passing because the loop was effectively dead.
The global handle in web_server.py was named soulseek_client for
historical reasons but the type has long been DownloadOrchestrator,
not SoulseekClient. Renamed the global plus every parameter/attribute
that carried the legacy name.
- web_server.py: global var renamed; all 99 references updated.
- api/, core/downloads/*, core/search/*, core/streaming/*,
services/sync_service.py: parameter names, dataclass fields, and
init() arg names renamed.
- Test fixtures (CandidatesDeps, MasterDeps, SearchDeps, etc.) and
the _build_deps helpers updated accordingly.
The core.soulseek_client module path and SoulseekClient class name
(the actual soulseek-only client) are unchanged — only the orchestrator
handle renamed. Module imports of TrackResult/AlbumResult/DownloadStatus
from core.soulseek_client preserved.
Plug the previously-built SoundcloudClient (PR #478, the build-and-verify
phase) into every place a download source needs to appear. Follows the
same wiring contract as Tidal/Qobuz/HiFi/Deezer/Lidarr — orchestrator
routing, hybrid-mode picker, search dispatch, queue/cancel/clear,
provenance + library history, sidebar source label, settings UI all
work plug-and-play.
Backend wiring:
- `core/download_orchestrator.py` — import SoundcloudClient, _safe_init
it at startup, add to _client() lookup, get_source_status(),
check_connection's sources_to_check default, search source_names map,
search_and_download_best _streaming_sources tuple, download
source_map + source_names, and every iteration loop in
reload_settings download-path-update / get_all_downloads /
get_download_status / cancel_download (route + iterate) /
clear_all_completed_downloads / cancel_all_downloads.
- `core/downloads/monitor.py` — added SoundCloud to the per-client
loop that fetches active downloads outside the orchestrator (uses
getattr fallback for older soulseek_client snapshots).
- `core/downloads/task_worker.py` — added SoundCloud (and Lidarr,
which was missing too — bonus fix) to source_clients dict for hybrid
fallback dispatch.
- `core/downloads/validation.py` — added 'soundcloud' to
_streaming_sources so SoundCloud results go through the matching
engine validation path instead of the Soulseek quality-filter path.
- `core/imports/side_effects.py` — three call sites: source_map for
download_source label written to library_history, streaming-source
guard for the `||`-encoded stream_id parsing, and source_service
map for provenance recording. All three now include 'soundcloud'.
- `web_server.py` — five streaming-source detection tuples updated.
New `/api/soundcloud/status` endpoint returns
{available, configured, reachable} mirroring the Deezer/HiFi
status-endpoint pattern; reachability runs a real cheap yt-dlp
search so the settings Test Connection button gives a meaningful
pass/fail signal.
- `config/settings.py` — added empty `soundcloud_download` defaults
block so future tier-2 OAuth (SoundCloud Go+ session) doesn't have
to migrate existing configs.
Frontend:
- `webui/index.html` — new `<option value="soundcloud">` in the
download-source-mode dropdown, SoundCloud added to both hidden
legacy hybrid-source selects, new settings container with info
text + Test Connection button.
- `webui/static/settings.js` — HYBRID_SOURCES entry (with the
SoundCloud cloud SVG icon), _hybridSourceEnabled default,
updateDownloadSourceUI container display, allSources for legacy
hybrid picker, testSoundcloudConnection function (hits the new
status endpoint, color-codes the result), saveSettings
soundcloud_download empty block.
- `webui/static/shared-helpers.js` — sidebar source-name map
includes SoundCloud + Lidarr (Lidarr was also missing, bonus fix).
- `webui/static/helper.js` — WHATS_NEW entry under '2.4.2' dev cycle
describing the user-visible change in the chill terse voice.
Tests:
- `tests/test_download_orchestrator_soundcloud.py` — 14 integration
tests verifying the wiring: client constructed at startup, _client
lookup resolves 'soundcloud', get_source_status includes it,
download dispatcher routes username='soundcloud' to the SoundCloud
client (and unknown usernames still fall back to Soulseek), hybrid
search iterates SoundCloud when in order and skips it cleanly when
unconfigured, get_all_downloads / get_download_status / cancel /
clear walk SoundCloud, soundcloud-only mode dispatches only to
SoundCloud, _streaming_sources tuple in validation includes
'soundcloud'.
- `tests/downloads/test_download_orchestrator.py` — added
`soundcloud` to the test fixture's _build_orchestrator helper so
the new orchestrator attribute doesn't AttributeError in pre-
existing tests that bypass __init__.
Verified:
- Full suite green (1728 passed, 2 deselected for soundcloud_live)
- Ruff clean
- Live SoundCloud-only mode search returns 25 SoundCloud tracks for
"kendrick lamar luther" in <2s, returning properly-shaped
TrackResult objects with username='soundcloud' and dispatch-key
filename ready for the download path.
Out of scope (intentional deferrals):
- SoundCloud Go+ OAuth tier (256 kbps AAC) — anonymous-only for now.
Adding auth later is a settings-page extension, no orchestrator
changes needed.
- Album/playlist support — SoundCloud has playlists but they don't
map to the album model the rest of SoulSync expects. Singles only.
