Adds torrent/usenet as release-oriented download sources with album-bundle staging, live progress reporting, and post-processing that selects the requested audio file from completed releases instead of blindly importing the first file.
Keeps album-bundle behavior gated to single-source torrent/usenet album downloads, excludes release sources from hybrid album per-track searches, and allows hybrid non-album tracks to use release results safely.
Improves staged-release matching for featured/bonus track filenames while preserving version mismatches, records torrent/usenet provenance in library history, and updates service/status UI labels.
Covers the flow with focused lifecycle, status, staging, validation, task worker, post-processing, and import side-effect tests.
Wraps up the code-review refactor pass.
- config/settings.py: ``download_source`` defaults gain
``album_bundle_poll_interval_seconds`` (default 2s) and
``album_bundle_timeout_seconds`` (default 6h, was a hard-coded
``6 * 60 * 60`` magic constant in torrent.py). The plugin reads
these via ``album_bundle.get_poll_interval`` /
``get_poll_timeout`` with safe fallback to the defaults when the
config value is missing / non-numeric. ``mode`` doc-comment
extended to list ``torrent`` and ``usenet``.
- core/downloads/validation.py: comment block above the album-name
fallback rewritten to document when the fallback actually runs
now — single-track hybrid downloads only, because the album-
bundle gate handles single-source mode and the hybrid chain
filter strips torrent / usenet from album batches. Code path
unchanged; just clarifies the contract for the next reader.
- webui/static/helper.js: WHATS_NEW entry summarising the refactor
pass (helper extraction, dispatch lift, staging deps injection,
atomic copy, configurable timeout, test additions).
The /loop of: extract → inject → test was sweep enough to drop the
gate code's coupling to 2-3 modules and put 49 unit tests behind
the new boundaries. Code-review feedback addressed:
1. album_bundle.py extracted ✓
2. Dispatch lifted out of master.py ✓
3. staging.py decoupled from runtime_state ✓
4. Validation fallback scope documented ✓
5. Poll timeout config-driven ✓
6. ``amazon`` provenance owned in a prior commit ✓
7. End-to-end-shaped tests added (test_album_bundle_dispatch.py)
8. Auto-Import race closed via atomic copy ✓
Live-test bug: Spotify-flow downloads with Torrent Only as the
active source produced 'download_failed' for every track. Searches
hit Prowlarr fine but no candidate ever got picked. Root cause was
in core/downloads/validation.py's get_valid_candidates:
- The streaming-source allowlist for the structured-metadata path
didn't include 'torrent' / 'usenet', so torrent results fell into
the Soulseek matching branch.
- Soulseek matching parses ``candidate.filename`` as a slskd-style
``Artist/Album/Track.flac`` path. Torrent / usenet filenames are
encoded as ``<download_url>||<display_name>`` so the orchestrator
can recover the URL — splitting that string on slashes produced
garbage path segments that never matched the expected artist,
every candidate failed the artist-folder gate, returned [], track
status flipped to 'not_found'.
Fixes:
- _streaming_sources now includes 'torrent' and 'usenet'. They take
the structured-metadata scoring path that reads r.title / r.artist
directly (the projection layer pre-fills both correctly).
- Artist gate skipped for torrent/usenet, same as YouTube. Album-
level releases legitimately don't expose per-track artist — the
projection falls back to the indexer name as the 'artist' field,
which would otherwise fail the gate against every Spotify artist.
- New album-name fallback scoring: for torrent/usenet only, the
candidate title is ALSO scored against the wanted track's
spotify_track.album field, and the max of (track-title score,
album-title score) wins. This makes a candidate titled
"GNX (2024) [FLAC]" match every track on the GNX album rather
than scoring near zero against a specific track title like
"Luther (with SZA)". match_type 'album_release' for visibility.
All 9 existing validation tests still pass.
Add duration tolerance logic and pre-download rejection for structured sources (tidal, qobuz, hifi, deezer_dl, amazon) when candidate duration deviates beyond allowed tolerance. Introduces helper functions _duration_tolerance_seconds and _duration_mismatch_exceeds_integrity_tolerance and uses resolve_duration_tolerance from core.imports.file_integrity. Log and skip candidates that would fail post-processing integrity checks to avoid wasted downloads. Update tests to include matching engine stub and new cases covering rejection and acceptance based on duration tolerance; also adjust imports and test fixtures.
`validation.py` had amazon absent from `_streaming_sources`, causing
Amazon TrackResult objects (bitrate=None, size=0) to fall through to
the Soulseek P2P code path and get rejected by
`filter_results_by_quality_preference`. Every album track was marked
not found.
Fix: add 'amazon' to every streaming-source guard tuple/set that was
previously missing it:
- core/downloads/validation.py — primary bug fix (quality-filter bypass)
- core/downloads/status.py — _STREAMING_SOURCE_NAMES frozenset
- core/downloads/task_worker.py — hybrid fallback client map
- core/imports/side_effects.py — || filename→stream-id extraction
- web_server.py — is_streaming_source, transfer list display,
candidate source label, _try_source_reuse, _store_batch_source
- tests/test_download_plugin_conformance.py — registry count + parametrize
Also updates the 2.5.3 What's New entry to drop the stale
"not yet wired" disclaimer.
A "type beat" is an instrumental track produced in another artist's
style, uploaded to SoundCloud and tagged with that artist's name to
game search ranking. They show up as candidates for major-label
tracks (e.g. "Eminem - Greatest (Kamikaze) Type Beat - Sit Down" for
"Greatest" by Eminem) and have nothing to do with the real song.
Add 'type beat' to the version-keyword list so the scorer applies the
0.4x penalty + flags the result as wrong_version. Currently the
matcher rejects them via low text-similarity scores anyway, but the
explicit keyword makes the rejection deterministic and gives a clear
diagnostic in the logs / modal.
The earlier validation-only filter only ran in the auto-search
scoring path. SoundCloud preview snippets still leaked through:
- The candidate-review modal cached raw search results (pre-validation),
so previews were visible and clickable for manual retry — and the
manual-pick download path bypassed validation entirely, downloading
the preview anyway.
- The not-found raw-results cache stored unfiltered top-20s.
Lift the preview filter into a reusable filter_soundcloud_previews()
helper and apply it at every entry point: validation scoring (still),
modal-cache fallback when validation drops everything, and the
not-found raw-results path. Previews now never reach the cache, the
matcher, or the manual-pick UI. Drops candidates < 35s or below half
the expected duration, gated on expected > 60s so genuine short
tracks still pass. 7 new unit tests pin the helper.
Also fixed a silent regression in core/downloads/task_worker.py's
hybrid-fallback path. Cin-5 dropped the per-source attrs from the
orchestrator (orch.soulseek, orch.youtube, etc.), but the fallback
loop still resolved sources via getattr(orch, '<src>', None) — every
lookup silently returned None, so remaining_sources came back empty
and the fallback never ran. Now uses orch.client(name) like the rest
of the codebase. Updated the test fake to expose client() too — the
old test was passing because the loop was effectively dead.
SoundCloud serves a ~30s preview clip for tracks gated behind Go+
or login (extremely common for major-label uploads — what's actually
on SoundCloud is bootlegs, fan reuploads, type beats, and these
previews). yt-dlp accepts the preview as the download payload, the
post-download integrity check catches the duration mismatch and
quarantines the file, but the user only sees "all candidates failed"
with no obvious explanation.
Filter at validation time when we know expected_duration: drop
SoundCloud candidates whose duration is below half the expected
length OR within ~5s of the 30s preview boundary, gated on
expected being non-trivially long (>60s) so genuinely short tracks
still pass through.
The global handle in web_server.py was named soulseek_client for
historical reasons but the type has long been DownloadOrchestrator,
not SoulseekClient. Renamed the global plus every parameter/attribute
that carried the legacy name.
- web_server.py: global var renamed; all 99 references updated.
- api/, core/downloads/*, core/search/*, core/streaming/*,
services/sync_service.py: parameter names, dataclass fields, and
init() arg names renamed.
- Test fixtures (CandidatesDeps, MasterDeps, SearchDeps, etc.) and
the _build_deps helpers updated accordingly.
The core.soulseek_client module path and SoulseekClient class name
(the actual soulseek-only client) are unchanged — only the orchestrator
handle renamed. Module imports of TrackResult/AlbumResult/DownloadStatus
from core.soulseek_client preserved.
Removed the eight backward-compat attribute aliases on the orchestrator
(soulseek, youtube, tidal, qobuz, hifi, deezer_dl, lidarr, soundcloud).
External callers and the orchestrator's own internals now reach clients
through the generic alias-aware client(name) accessor.
- core/downloads/{master,monitor,validation}.py: migrated to client().
Monitor's per-source aggregation loop replaced with a single
engine.get_all_downloads() call.
- core/search/{orchestrator,stream}.py: migrated; stream.py drops the
hand-built mode-to-client dict.
- web_server.py: migrated /api/deezer/arl-* + tidal client lookup.
- core/download_orchestrator.py: internal self.soulseek /
self.deezer_dl reaches now route through self.client(); attr
assignments dropped from __init__; module docstring updated.
- Test fakes (_FakeSoulseek, _FakeSoulseekWithYT) expose client(name)
instead of stuffing per-source attributes.
- Conformance test re-pinned to the client() accessor contract.
Plug the previously-built SoundcloudClient (PR #478, the build-and-verify
phase) into every place a download source needs to appear. Follows the
same wiring contract as Tidal/Qobuz/HiFi/Deezer/Lidarr — orchestrator
routing, hybrid-mode picker, search dispatch, queue/cancel/clear,
provenance + library history, sidebar source label, settings UI all
work plug-and-play.
Backend wiring:
- `core/download_orchestrator.py` — import SoundcloudClient, _safe_init
it at startup, add to _client() lookup, get_source_status(),
check_connection's sources_to_check default, search source_names map,
search_and_download_best _streaming_sources tuple, download
source_map + source_names, and every iteration loop in
reload_settings download-path-update / get_all_downloads /
get_download_status / cancel_download (route + iterate) /
clear_all_completed_downloads / cancel_all_downloads.
- `core/downloads/monitor.py` — added SoundCloud to the per-client
loop that fetches active downloads outside the orchestrator (uses
getattr fallback for older soulseek_client snapshots).
- `core/downloads/task_worker.py` — added SoundCloud (and Lidarr,
which was missing too — bonus fix) to source_clients dict for hybrid
fallback dispatch.
- `core/downloads/validation.py` — added 'soundcloud' to
_streaming_sources so SoundCloud results go through the matching
engine validation path instead of the Soulseek quality-filter path.
- `core/imports/side_effects.py` — three call sites: source_map for
download_source label written to library_history, streaming-source
guard for the `||`-encoded stream_id parsing, and source_service
map for provenance recording. All three now include 'soundcloud'.
- `web_server.py` — five streaming-source detection tuples updated.
New `/api/soundcloud/status` endpoint returns
{available, configured, reachable} mirroring the Deezer/HiFi
status-endpoint pattern; reachability runs a real cheap yt-dlp
search so the settings Test Connection button gives a meaningful
pass/fail signal.
- `config/settings.py` — added empty `soundcloud_download` defaults
block so future tier-2 OAuth (SoundCloud Go+ session) doesn't have
to migrate existing configs.
Frontend:
- `webui/index.html` — new `<option value="soundcloud">` in the
download-source-mode dropdown, SoundCloud added to both hidden
legacy hybrid-source selects, new settings container with info
text + Test Connection button.
- `webui/static/settings.js` — HYBRID_SOURCES entry (with the
SoundCloud cloud SVG icon), _hybridSourceEnabled default,
updateDownloadSourceUI container display, allSources for legacy
hybrid picker, testSoundcloudConnection function (hits the new
status endpoint, color-codes the result), saveSettings
soundcloud_download empty block.
- `webui/static/shared-helpers.js` — sidebar source-name map
includes SoundCloud + Lidarr (Lidarr was also missing, bonus fix).
- `webui/static/helper.js` — WHATS_NEW entry under '2.4.2' dev cycle
describing the user-visible change in the chill terse voice.
Tests:
- `tests/test_download_orchestrator_soundcloud.py` — 14 integration
tests verifying the wiring: client constructed at startup, _client
lookup resolves 'soundcloud', get_source_status includes it,
download dispatcher routes username='soundcloud' to the SoundCloud
client (and unknown usernames still fall back to Soulseek), hybrid
search iterates SoundCloud when in order and skips it cleanly when
unconfigured, get_all_downloads / get_download_status / cancel /
clear walk SoundCloud, soundcloud-only mode dispatches only to
SoundCloud, _streaming_sources tuple in validation includes
'soundcloud'.
- `tests/downloads/test_download_orchestrator.py` — added
`soundcloud` to the test fixture's _build_orchestrator helper so
the new orchestrator attribute doesn't AttributeError in pre-
existing tests that bypass __init__.
Verified:
- Full suite green (1728 passed, 2 deselected for soundcloud_live)
- Ruff clean
- Live SoundCloud-only mode search returns 25 SoundCloud tracks for
"kendrick lamar luther" in <2s, returning properly-shaped
TrackResult objects with username='soundcloud' and dispatch-key
filename ready for the download path.
Out of scope (intentional deferrals):
- SoundCloud Go+ OAuth tier (256 kbps AAC) — anonymous-only for now.
Adding auth later is a settings-page extension, no orchestrator
changes needed.
- Album/playlist support — SoundCloud has playlists but they don't
map to the album model the rest of SoulSync expects. Singles only.
Body byte-identical to the original. matching_engine and
soulseek_client are injected via init() right after _init_discover_hero
since both originals are constructed early in web_server.py boot
(L598/L610) and never rebound.
web_server.py: 35586 → 35408 (-178 lines).
