Commit graph

100 commits

Author SHA1 Message Date
BoulderBadgeDad
b07359cdb5 downloads: make the "file not found" failure actionable instead of opaque
Discord (Shdjfgatdif, standalone): some downloads complete on disk but get marked failed with
"File not found on disk after 5 search attempts. Expected: <basename>" — which tells the user
nothing about where we looked or what to check.

This is deliberately a DIAGNOSTIC fix, not a behavior change. The finder + path handling are sound
(verified: docker_resolve_path no-ops in standalone, the finder walks the configured
soulseek.download_path and resolves a present file). When it still misses after slskd reported the
transfer Succeeded, the cause is environmental — either the file is still landing (timing) or, the
classic standalone gotcha, SoulSync's download_path doesn't point at slskd's actual download dir.
Neither is something our code can "fix"; the user fixes the config, or the file arrives.

So: name the folder we actually searched and spell out the two real causes, turning an opaque
failure into self-diagnosis ("oh, my download folder's wrong"). Retry/wait behavior is left
untouched on purpose — widening the window does nothing for a path mismatch and I can't justify it
for this user. Also normalizes the slskd backslash path so the reported filename is the leaf, not
the whole "@@@user\folder\file" string.

Updated the existing not-found test to pin the new actionable message (searched path + config
hint + filename). 588 downloads tests green, ruff clean.
2026-06-28 15:32:47 -07:00
BoulderBadgeDad
551df0c3ca downloads: fix file-finder collapsing on an unbalanced bracket (false "not found")
Discord (Shdjfgatdif): a downloaded .flac sat right there in the download folder but the import
flow reported "File not found on disk after 5 search attempts" and failed it.

root cause: slskd REPORTS the name as "[34 - You & Me (Flume Remix).flac" but SAVES it as
"34 - You & Me (Flume Remix).flac" (it strips the leading '['). The finder's fuzzy-match
normaliser used one combined bracket-strip — r'[\[\(].*?[\]\)]' — which allows MISMATCHED
delimiters, so the lone '[' matched all the way to the next ')', ate the whole title, and
collapsed the search target to just "flac". That scored 0.40 against the real filename (below the
0.85 floor) → "not found", despite the file being on disk. Confirmed by running the real code on
his exact filename.

fix: strip only BALANCED pairs (\[...\] and (...) separately). A stray unbalanced bracket now
survives to the alphanumeric strip instead of devouring the title. '[34 - You & Me (Flume Remix)'
→ matches at 1.00. Balanced tags like "[FLAC]" / "(Remastered 2016)" are still stripped (no
regression). Only used internally by the finder's fuzzy scorer — contained blast radius.

3 tests: his exact unbalanced-'[' filename, a stray-']' variant, and a balanced-tag no-regression
guard. 1311 imports/downloads/quality tests green, ruff clean.
2026-06-28 15:10:30 -07:00
BoulderBadgeDad
b72febbf1c manual search: INJECT the exact pasted-link track, don't rely on text search surfacing it (#932)
reopened by diegocade1: pasting a Qobuz track link still showed unrelated results. the earlier
fix (b1f061a) only BUBBLED the linked track to the top — but a pasted link is resolved to an
"artist title" text query and searched, and for an obscure track ("foreign lavennew" by colacola)
that text search returns broad lookalikes ("Foreign Bird", "Foreign Spies", …) and never the
actual track. nothing to bubble → user sees junk.

fix: since the link is already resolved via get_track(id), fetch that exact track AS a downloadable
result and inject it at the top (Qobuz downloads by id, so the result is fully usable). the text
search still runs for alternatives.

- QobuzClient.get_track_result(id): get_track + _qobuz_to_track_result; None on any failure.
- _qobuz_to_track_result gains require_streamable (default True for bulk search). the link fetch
  passes False: track/get may OMIT the streamable flag, which would default-False and wrongly drop
  the exact track the user explicitly asked for. (this closes the one shape assumption that
  couldn't be verified against a live Qobuz API — the track is no longer gated on it.)
- track_link.inject_linked_track_first(tracks, linked_result, id): pure seam — prepend the fetched
  result + drop any search duplicate; falls back to the bubble when no result was fetched.
- manual-search endpoint fetches linked_result defensively (getattr 'get_track_result') and calls
  the seam. Tidal/HiFi (get_track returns a dict but the converter wants an object — shape
  mismatch) have no get_track_result, so they keep the existing bubble path: NO regression.

14 tests: inject puts the fetched track first when search missed it / dedups a search copy / falls
back to bubble / str-safe id / noop; get_track_result convert/none/exception; and the REAL
converter builds a valid downloadable result from a track/get dict that OMITS streamable (search
path still rejects it). 85 track-link/qobuz tests green, ruff clean.
2026-06-28 12:42:20 -07:00
BoulderBadgeDad
b1f061a2a8 manual search: float the pasted Qobuz/Tidal track to the top (#932)
a pasted track link IS resolved + searched, but the 'bubble the exact track to the top'
step read getattr(t,'id') — and TrackResult has no top-level id (the source id lives in
_source_metadata['track_id']). so the bubble was a silent no-op: the linked track sat buried
among fuzzy text-search lookalikes and the user saw unrelated tracks. qobuz made it worse —
_qobuz_to_track_result never stamped _source_metadata at all, so the track had no id to match.

- stamp _source_metadata={'source':'qobuz','track_id':...} on qobuz TrackResults (mirrors tidal)
- extract the bubble into pure, tested helpers (linked_track_id / bubble_linked_track_first)
  that read _source_metadata['track_id'] — fixes it for tidal too, str/int-safe, stable no-op
19 track-link tests (+6 new) + 87 qobuz/download tests green.
2026-06-26 21:45:41 -07:00
dev
ab05508acc feat(quality): rank-based candidate ordering toggle for priority mode
Adds an opt-in `rank_candidates_by_quality` profile flag. When on, the
priority-mode download walk orders candidates by the ranked-target quality
(confidence/speed only break ties) instead of confidence-first. Default off
keeps the byte-for-byte old behaviour, so existing installs are unaffected.

Best-quality search mode is always quality-first regardless of the flag; the
toggle only affects priority mode. Search-time source selection is unchanged —
nothing is skipped, so a track can never go missing, only the order in which
copies are tried changes.

The version-mismatch force-import follows automatically: it accepts the
first-tried (= best-ordered) quarantined candidate, which is the highest-quality
one once the walk is quality-first. No change to its selection logic needed.

- core/quality/selection.py: load_rank_candidates_by_quality() (fail-closed).
- core/downloads/task_worker.py: _best_quality_ordering -> _candidate_ordering;
  quality-first when best_quality mode OR the toggle is on.
- database/music_database.py: default profile carries the flag (False).
- web_server.py: flag is preserved globally across preset apply/reset, like
  search_mode.
- core/imports/version_mismatch_fallback.py: comment clarified (no behaviour
  change).

Tests (TDD): load_rank_candidates_by_quality default/enabled/disabled/error;
_candidate_ordering across all mode+toggle combinations + fail-closed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 22:00:57 +02:00
dev
05b36704c3 fix(post-processing): prevent double-claim race that fails imported tracks
Two subsystems post-process the same completed transfer: the browser-poll
status endpoint (web_server) and the background download monitor. Both watch
the same slskd/streaming transfers and each launches the verification
pipeline. When one path quarantines + requeues the next-best candidate
(clearing username/filename, status -> 'searching'), the monitor's
already-submitted run_post_processing_worker then runs, finds no source info,
and falsely marks the task 'failed' ("missing file or source information") —
clobbering the in-flight retry while a parallel attempt imports the song.

Fix: a single atomic claim (downloading/queued -> post_processing under
tasks_lock) so exactly one path processes each download.

- runtime_state: new claim_for_post_processing() helper
- post_processing: race guard — worker bails (no fail/notify) if the task is
  no longer 'post_processing' when it runs
- web_server: both poll paths (Soulseek + streaming) claim before launching;
  claim is released on thread-launch failure

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 21:03:33 +02:00
BoulderBadgeDad
ed0a2079cf
Merge pull request #896 from nick2000713/feature/best-quality-search-mode
Global quality system: real-audio verification, best-quality search & quality profiles (please try...not ready to merge)
2026-06-24 20:27:38 -07:00
dev
774d0f6c65 feat(quality): make every audio format controllable via ranked targets
The ranked-target list is now the single source of truth for which formats
download, in the user's exact priority order, for ALL sources — no hardcoded
format hierarchy decides anything. A candidate passes only if it matches a
ranked target; if nothing matches, the existing Use-Fallback toggle decides.

- source_map: new shared format_from_extension() + AUDIO_EXTENSIONS — one
  source of truth for extension→format used by every extension-based source, so
  adding a format lights it up everywhere. Soulseek now classifies through it
  (opus/wav/aiff were previously dropped as 'unknown').
- file_ops.probe_audio_quality (generic import-time guard, all sources): add
  WMA; detect ALAC from the real codec (an .m4a is AAC or ALAC).
- soulseek: drop the AAC-specific opt-in gate — AAC now follows the same
  universal rule as every format.
- model.tier_score: documented as ONLY a same-format tiebreak + fallback order,
  never cross-format priority (the list owns that); add opus/alac bases.
- UI: ranked-target editor offers all formats (FLAC/ALAC/WAV·AIFF lossless with
  bit-depth+sample-rate; MP3/AAC/OGG/Opus/WMA lossy with min-bitrate).
- tests: AAC retargeted to the universal model; new coverage for
  format_from_extension and matches_target across all formats.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 23:01:20 +02:00
BoulderBadgeDad
2b17ed8451 #915: post-processing hydrates lean album context from the PRIMARY source (parity with Reorganize)
Root cause: the only album-context backfill in the download path (hydrate_download_metadata) goes
through spotify_client.get_track_details — Spotify-only. An iTunes/Deezer-primary user's download
kept a lean context (no release_date), so the path dropped $year and the date defaulted to
YYYY-01-01 — until they ran a Reorganize, which reads the full album from the PRIMARY source. That
asymmetry IS the bug.

Fix: when the context is lean and the primary source isn't Spotify, hydrate it from that source via
get_album_for_source — the exact path Reorganize/Enrich use. Verified the primary source returns the
real data (live iTunes get_album for the reporter's album: release_date 2024-04-17, not 2024-01-01).

backfill_album_context_from_source is a pure, injected-fn seam: 6 tests (hydrate, no-op when
complete / spotify-primary / sentinel-id, stays-lean on None, swallows source errors). 552 downloads
tests green.
2026-06-23 18:46:35 -07:00
dev
b761229a00 merge: pull upstream/main (2.7.4) into feature/best-quality-search-mode
- Keep our v3 ranked-targets quality system (filter_and_rank, QualityTarget)
  in soulseek_client.py, settings.js, database presets, and index.html
- Take upstream removal of standalone quality-scanner code:
  QualityScannerDeps + run_quality_scanner moved to repair job
  (core/repair_jobs/quality_upgrade_scanner.py)
- Take upstream AAC-tier addition in database/music_database.py default profile
- Take upstream removal of /api/quality-scanner/* routes from web_server.py
- Remove test_discovery_quality_scanner.py (deleted upstream)
- 47 upstream commits absorbed (2.7.3 + 2.7.4 including re-identify flow,
  dead-folder cleanup, track-number prefix strip, and more)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-19 16:54:51 +02:00
BoulderBadgeDad
400b35d655 #886: AAC as an opt-in Soulseek quality tier (purely additive, off by default)
radoslav-orlov: add AAC as a download quality option. AAC is more efficient than
MP3, so it's useful for Soulseek/torrents (streaming sources pick their own
codec; Amazon — the AAC-heavy one — is down).

Additive by construction: every quality tier already defaults enabled=false and
the waterfall is built only from enabled tiers, so AAC ships OFF and the bucketer
routes a not-enabled AAC file to the 'other' bucket EXACTLY as today (where it was
silently dropped). Only a user who turns AAC on makes it a first-class tier,
ranked above MP3 / below FLAC (priority 1.5, min-kbps gate so junk AAC can't beat
a good MP3).

- music_database: aac tier (disabled) in the default profile + all 3 presets.
- soulseek_client: map .m4a -> 'aac' in both result parsers (was 'unknown' ->
  dropped); add the 'aac' bucket + a gated branch + a fallback size limit.
- settings UI: an 'AAC' tier toggle (unchecked) between FLAC and MP3; save
  defaults its priority to 1.5 so upgraded profiles rank it right on first save.

7 seam tests pinning the additive guarantee (aac absent/disabled -> dropped as
before; FLAC/MP3 selection unchanged; aac on -> selectable, below FLAC, above
MP3); 81 quality/soulseek tests pass, ruff clean. quality_upgrade left untouched
(its AAC handling is unchanged).
2026-06-18 13:45:52 -07:00
dev
b928f4df43 fix(downloads): always surface all unverified history on Downloads page
Adds a dedicated `get_library_history_unverified()` DB query that fetches
every library_history row with verification_status IN ('unverified',
'force_imported') with no recency cap. This is loaded unconditionally in
`build_unified_downloads_response` — not gated on `len(items) < limit` —
so historical unverified entries are never buried by a busy batch filling
the 200-row general limit, and entries from weeks/months ago aren't lost
in the 50-row recency-ordered history tail. Adds idx_lh_verification_status
for query performance and two regression tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-15 19:46:26 +02:00
dev
9af31c3706 fix(downloads): stop phantom-completing stuck post_processing tasks
A task stuck in 'post_processing' past the cutoff was force-marked 'completed'
("assume it worked"). In a large batch, post-processing (AcoustID + quality +
import) is serialized and backs up, so tasks sit in post_processing while merely
QUEUED — then got falsely completed, showing as downloaded with no file on disk
(/Transfer empty).

Now: the cutoff is 30 min (was 5) so legit backlog isn't cut off, and when it
does fire the task is only completed if it actually produced a file
(final_file_path exists on disk) — otherwise marked failed (honest + retryable).
Applied at both stuck-detection sites (check_batch_completion + _v2).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 23:25:33 +02:00
dev
f55a4fcf72 fix(downloads): stop active tasks from starving terminal rows out of the list
Root cause of "completed/failed/unverified don't show during a running
batch, only after it ends" (F5 didn't help): build_unified_downloads_response
sorted live tasks active-first (downloading/searching/queued = priority 0-3,
completed/failed = 4-7) then truncated the whole array at items[:limit] (300).
During a busy batch the active+queued tasks filled the limit and pushed every
terminal task off the end, so /api/downloads/all never returned them — the
Completed/Failed/Unverified tabs filter client-side and had nothing to show.

Fix: `limit` now bounds only the persistent-history tail. Live in-memory
tasks are always returned in full — they're already bounded by the 5-min
cleanup automation, and array order is presentation-only since the page
filters per tab client-side.

Verified with a repro (320 queued + 1 completed + 1 failed → terminal rows
were absent at limit=300; now present).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 21:36:27 +02:00
dev
e594ac9799 fix(downloads): keep terminal tasks visible during active batch + concurrent pool
Three fixes from on-device testing of best-quality mode:

1. clear_completed_local no longer prunes terminal tasks that belong to a
   STILL-ACTIVE batch (one with non-terminal work remaining). The 5-min
   "Clean Completed Downloads" automation was yanking completed/failed/
   unverified rows out of download_tasks mid-run — and failed/cancelled
   aren't in library_history — so they only reappeared after the batch
   ended. Now the whole active batch stays intact until it finishes.

2. search_all_sources runs every source CONCURRENTLY (asyncio.gather)
   instead of sequentially, so the pool waits only for the slowest source
   (e.g. usenet/Prowlarr) in parallel rather than summing all latencies.

3. The pool log now reports per-source contribution counts
   (e.g. "usenet=0, hifi=11, soulseek=1") instead of just echoing the
   chain, so a release-level source that returns nothing for a track-title
   query is visible rather than appearing to have been searched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 20:08:56 +02:00
dev
d484046523 feat(downloads): best-quality search mode + clearer fallback UI
Adds an opt-in search strategy toggle in the Quality Profile:

- priority (default): unchanged — first source in the hybrid chain that
  meets a quality target wins.
- best_quality: pool candidates from EVERY source per query and download
  them best→worst by actual audio quality; source order only breaks ties.

Implementation reuses existing plumbing so the retry system is untouched:
- engine.search_all_sources pools raw tracks across all configured,
  non-exhausted sources (no first-source short-circuit).
- candidates.order_candidates: new quality_first sort path — profile
  quality rank dominates, confidence/peer signals break ties. Priority
  path is byte-for-byte unchanged (regression-locked by tests).
- task_worker passes quality_first + targets through; skips the redundant
  hybrid-fallback block in best-quality mode (pool already covered it).
- Per-source retry budgets unchanged: a source that spends its budget is
  added to exhausted_download_sources and thus dropped from the whole
  pool. Independent of post_processing.retry_exhaustive.
- Query generator NOT touched.

Also clarifies the "Allow fallback" setting wording: it accepts OFF-LIST
quality as a last resort (not "walk down my list"), and notes that
lossy_copy.downsample_hires also bypasses the quality gate — the cause of
16-bit/MP3 files slipping through a 24-bit-only profile.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 18:46:01 +02:00
BoulderBadgeDad
aa5d747327 Playlists: wire materialize triggers + retire per-track routing flag
- Routing (step 5): organize-by-playlist tracks no longer set the per-track
  _playlist_folder_mode flag, so they import NORMALLY into Artist/Album — exactly
  what a normal download does. _playlist_name provenance is kept (origin.py).
- Triggers (step 4): build the playlist folder from the batch's own payload at
  both end-of-flow points — the all-owned path in master.py (no downloads, so the
  lifecycle never runs) and the batch-complete hook in lifecycle.py (after
  downloads). Both gated on playlist_folder_mode, both non-fatal.

Works for the all-owned case (the smack test that did nothing before) and for
mixed owned/downloaded, with no source-ID or mirrored-playlist dependency. The
materialized folder uses the default ./Playlists root + symlink mode until the
Settings UI is added.

Updated the master test to assert the new contract (provenance kept, routing
flag gone). 979 tests pass.
2026-06-12 13:37:59 -07:00
BoulderBadgeDad
56bdcee434 Fix: rejected slskd download hung the task at 'downloading' forever (#836)
A wishlist track (or tracks in an album) that slskd accepted then REJECTED would
sit at "DOWNLOADING... 0%" indefinitely, spam an ERROR every status poll
("…Completed, Rejected - letting monitor handle retry"), and — for albums —
block the whole batch from ever completing.

Root defect: the status formatter's non-manual error branch keeps the task
'downloading' and trusts the retry monitor to resolve it, with NO backstop. When
the monitor can't make progress (a rejected transfer with no other source), the
task is stuck forever.

Backstop: measure how long the ERROR state has persisted (keyed off the task's
last status transition, so a slow-but-healthy transfer is never failed, and each
monitor-retry episode gets a fresh window). Once it exceeds the monitor's retry
window (60s, vs the monitor's ~15s) with no resolution, mark the task failed and
fire the worker-freeing completion callback so the batch can finish. Also log the
error ONCE per episode instead of every 2s poll. The healthy path is untouched —
a working retry transitions the task before the grace, so this never fires.
Manual picks still fail immediately (unchanged).

Tests: rejected-within-grace stays downloading; rejected-beyond-grace fails +
schedules completion; manual pick fails immediately. 45 status tests pass.
2026-06-10 18:16:22 -07:00
BoulderBadgeDad
b36392c62b Fix: a '/' in a song title was treated as a path separator (#835)
YouTube/Tidal/Qobuz results encode the name as ``id||title``. When the title
itself contains a '/' (e.g. the Sawano AoT track "YouSeeBIGGIRL/T:T"), two places
wrongly basename-split it on the slash and kept only the last segment ("T:T"):

- core/downloads/file_finder.py — the completed-download finder truncated its
  search target to "T:T", so the real on-disk file (slash sanitised by the
  writer) never matched → "not found after processing" → the download got
  QUARANTINED. Now an encoded ``id||title`` keeps the whole title as the target
  and contributes no remote-directory components; real Soulseek PATHS still get
  basename + dir extraction unchanged.
- webui/static/downloads.js — the manual-search FILE column showed only "T:T".
  Added a ``||``-aware short-label helper (mirrors the correct handling already
  used elsewhere in the file); real file paths still show their basename.

Tests: the finder locates "YouSeeBIGGIRL∕T: T.mp3" from the encoded title
"…||YouSeeBIGGIRL/T:T" (the screenshot case), doesn't match an unrelated file,
and a genuine Soulseek path still resolves to its last segment. 21 finder tests
+ 64 script-split integrity tests pass.
2026-06-10 16:29:09 -07:00
BoulderBadgeDad
8d133ecd60 Wishlist: serialize album-bundle downloads so they stop flooding the search pool (Sokhi #740)
Sokhi: "downloads searching for way too many tracks at once" — a wishlist run
that fanned out into ~one batch per album. Verified the actual search/download
concurrency IS capped at 3 (single shared missing_download_executor), so it
wasn't really hammering slskd — but the display showed ~20 "searching" and the
batch list was a mess.

Root cause: run_full_missing_tracks_process was supposed to "block its album-pool
worker for the whole search+download" (that's what the dedicated album_bundle_
executor is for), but it RETURNED the instant it had STARTED the downloads. So
the album pool only throttled the fast analysis phase — every album batch blew
through analysis and immediately dumped its tracks into the shared download pool,
all pre-marked 'searching'. The intended serialization never happened.

Fix: add serialize= to run_full_missing_tracks_process. Album-bundle batches
(dispatched on album_bundle_executor) pass serialize=True and now hold their pool
slot via _wait_for_batch_drain() until every task in the batch reaches a terminal
state — so only ~N albums are in flight at once. The wait is passive (downloads
are driven by the monitor + completion callbacks on other threads, so no
deadlock) and bails on shutdown, a removed batch, or a safety cap. The residual /
playlist / manual paths run on the SHARED pool and pass serialize=False (blocking
there would steal a real download worker), so they're unchanged.

Tests: _wait_for_batch_drain returns immediately when all-terminal, waits until
tasks finish, bails on shutdown, respects the cap, handles a missing batch. 975
download/wishlist tests pass (only the pre-existing soundcloud /app failures).
2026-06-09 10:06:53 -07:00
BoulderBadgeDad
cea0e9d63c Manual download search: paste a Tidal/Qobuz track link to grab the exact version (#813)
When a track shows "Not found", the manual search now accepts a pasted Tidal or
Qobuz track link, not just a typed query (CubeComming: the fuzzy search misses
versions; he can find the track on Tidal but can't get it to appear).

How it works (robust, reuses the proven path): parse the link → (source,
track_id) → fetch the track via the source client's get_track → build a clean
"artist title (version)" query → run THAT source's normal search → bubble the
result whose id matches the link to the top. So the candidate is a normal,
already-downloadable streaming result — no hand-built download encoding — and
it downloads through the existing verified flow.

Degrades gracefully: if the source isn't connected or the link can't be
resolved, it falls back to a normal text search of the raw input — the user is
never worse off than typing it themselves. Scoped to Tidal + Qobuz (the
streaming sources that download by track id, with public track URLs); Soulseek
can't take a link (P2P, no ids), YouTube/SoundCloud are URL-native via a
different path (future).

- core/downloads/track_link.py: pure parse_download_track_link (tidal/qobuz
  /track/<id>, slug/region suffixes, scheme-less) + query_from_track_payload
  (per-source title/artist, Tidal version-append).
- manual-search endpoint: link detection → resolve → restrict to that source →
  id-match bubble.
- placeholder hint mentions pasting a link; maxlength 200→300 for long URLs.

Tests: 14 (parser shapes + payload extraction incl. remix version-append +
qobuz performer/album-artist fallback). JS valid.
2026-06-08 15:03:54 -07:00
dev
70732ad80e Downloads: lazy multi-query quarantine retry — exhaust all queries per source
The cached-first retry (8d98b755) abandoned a source after a single query:
the first run returns as soon as ONE query starts a download, so
cached_candidates held only that query's results. On a quarantine retry the
whole source was then excluded from re-search (via searched_sources), so the
later queries (e.g. "artist + album") never hit that source again — it jumped
to the next source after one query instead of exhausting all queries per
source.

Track searched QUERIES (searched_queries) instead of whole sources. A
quarantine retry now skips only the already-run queries (their candidates are
walked via cached-first) and still searches the not-yet-run queries against the
same source. Budget-exhausted sources (exhaustive mode) stay excluded, so the
source switch still fires when a source is genuinely spent.

Removes the now-dead searched_sources state (written but no longer read).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-06 16:44:14 +02:00
dev
07ca7eacfa Downloads: cached-first quarantine retry — stop re-searching the same source
Each AcoustID/integrity quarantine retry re-ran the FULL search (all queries,
all sources) before picking the next-best candidate — so a track that failed
verification a dozen times re-queried Soulseek a dozen times (~3 min/cycle in
the field). The next-best pick was already sitting in cached_candidates.

Now the monitor flags the re-queue as a quarantine retry; the worker walks the
already-found candidates first (skipping used + budget-exhausted sources) and
hands them straight to the download path — no search. A source is searched
exactly once: once its candidates are cached, later quarantine retries exclude
it (searched_sources) so the hybrid chain falls through to a not-yet-searched
source instead of re-querying the spent one. Fresh downloads and the monitor's
dead-connection/stuck retries clear searched_sources and search fresh, so the
only re-search is for a genuinely new source or a dead peer.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-06 16:44:14 +02:00
BoulderBadgeDad
2604704a27 #797: stop AcoustID quarantining correct non-English-artist downloads
AcoustID returns a recording's title/artist in their ORIGINAL script
(e.g. "久石譲" for Joe Hisaishi) while SoulSync's expected metadata is
romanized/English. A correct download then fails verification on two
walls: the title can never clear the 0.70 similarity bar cross-script,
and the only skip path that ignores the title required a near-perfect
0.95 fingerprint plus a resolved alias. Result: every non-English
artist trips it. Two complementary fixes, per the reporter's two ideas.

Graceful fix (automatic):
- New pure core/matching/script_compat.py detects when two strings are
  in genuinely different writing systems (CJK/Hangul/Cyrillic/Greek/
  Arabic/Hebrew/Thai vs Latin). Accented Latin (Beyoncé, Sigur Rós)
  stays Latin — no false trigger.
- acoustid_verification.py: when the EXPECTED artist and the matched
  artist span scripts AND the artist is confirmed via the existing
  MusicBrainz alias bridge, SKIP instead of quarantine, without the
  0.95 floor (the 0.80 trust floor already gates the fingerprint).
- Deliberately narrow: keyed on the ARTIST spanning scripts + being
  confirmed. A same-script artist with only a cross-script title keeps
  the stricter 0.95 floor, so the #607 wrong-file protection (Kendrick
  R.O.T.C, low-fingerprint Japanese-title) is untouched.

Per-request toggle (manual escape hatch):
- New "Skip AcoustID verification" checkbox in the download-missing
  modal beside "Force Download All".
- skip_acoustid threads request -> batch -> per-track track_info ->
  download context (same path as _playlist_folder_mode), landing on
  the existing _skip_quarantine_check='acoustid' bypass. No new
  mechanism; only the AcoustID gate is bypassed (integrity/bit-depth
  still run).

Tests:
- tests/matching/test_script_compat.py — script-boundary cases.
- test_acoustid_skip_logic.py — Joe Hisaishi SKIPs at 0.85; unconfirmed
  cross-script artist still FAILs; same-script low-fingerprint still
  FAILs.
- test_downloads_candidates.py — toggle injects the bypass; absent
  toggle keeps verification.

Full suite: 5169 passed; only pre-existing soundcloud /app env failures
remain. Zero regressions.
2026-06-05 16:02:01 -07:00
Francesco Durighetto
9ff2e7084a Fix organize-by-playlist downloads: library entries, wishlist, and stale Spotify cache
Persist organize_by_playlist on mirrored playlists and run playlist-folder
downloads from the auto-sync pipeline instead of the global wishlist phase.
Register SoulSync library rows after playlist-folder post-processing, route
failed organize batches to the wishlist correctly, and skip sync-time
unmatched wishlist only when organize download handles retries.

Invalidate stale playlist track caches on refresh (Spotify and Deezer ARL),
re-mirror on refetch, and improve standalone playlist modals (re-analysis,
Open in Mirrored). Add filesystem missing-track detection and tests.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-03 10:26:32 +02:00
BoulderBadgeDad
e4bbcfda1b Downloads: add per-track detail endpoint for the track-detail modal
New GET /api/downloads/task/<id>/detail merges the live download task with its
library_history row (the data the Download History cards show) into one payload
the upcoming track-detail modal renders: status kind, title/artist/album,
source, quality, final location, AcoustID verdict, and expected-vs-downloaded.

Assembly + status classification live in core/downloads/track_detail.py as a
pure, importable build_track_detail()/classify_status_kind() (9 unit tests);
the endpoint is thin glue that looks up the matching history row by track.
2026-05-31 20:18:35 -07:00
BoulderBadgeDad
2824c25ec6 Album bundle: let Soulseek staging-misses fall through to per-track/cross-source fallback (#743)
A Soulseek album bundle stages whichever single folder scored best. If that
folder doesn't contain every track the album needs, the missing tracks were
marked not_found with no fallback — even in hybrid mode where later sources
(Deezer, YouTube, etc.) could fill them. The staging-miss short-circuit fired
for Soulseek because 'soulseek' was lumped into the torrent/usenet source set
when album bundles were added, and album_bundle_partial only reflects whether
the files found IN the folder downloaded, not whether the folder had every
needed track.

Drop 'soulseek' from the short-circuit (keep torrent/usenet). A track not
claimed from the staged Soulseek folder now falls through to the normal
per-track Soulseek search and, in hybrid mode, onward down the configured
chain. Unlike torrent/usenet — where per-track search re-adds the same
release — Soulseek per-track search is a genuine per-file network search, so
this is correct and cheap. Realizes the original author's stated intent
('keep partial bundles from blocking per-track fallback') robustly, since the
partial flag couldn't detect a folder that was simply missing tracks.

Only affects tracks NOT claimed from staging — fully-staged albums claim every
track via try_staging_match and never reach this gate, so working albums are
unchanged. Likely also mitigates #755 (all-album-import failures now fall
through to per-track instead of dying).

Tests: rewrote the two Soulseek staged-miss tests to assert fall-through
(single + hybrid-first); kept the torrent guard; added a usenet guard test.
2026-05-31 11:13:08 -07:00
Broque Thomas
5771c5ba77 Album-bundle staging: clean Soulseek copies + sweep orphans at startup
Two related leaks in ``storage/album_bundle_staging/<batch_id>/``:

1. **Soulseek bundle cleanup was excluded.** The per-batch cleanup
   at the end of a bundle download gated on:
       (album_bundle_source or '').lower() in ('torrent', 'usenet')
   The comment justified it as "slskd keeps its own completed
   folders" — but the Soulseek bundle path ALSO copies completed
   files into the private staging dir (``soulseek_client.py:1599``,
   ``copy_audio_files_atomically(completed, Path(staging_dir))``)
   for the per-track workers to claim. Those copies persisted
   forever; long-running installs accumulated stale GB. Extended
   the cleanup gate's allow-list to include ``soulseek`` so the
   per-batch dir is removed on bundle completion — same code path
   that already worked for torrent / usenet.

2. **No sweep for orphan dirs.** Any leftover ``<batch_id>``
   subdir from a previous-session crash, an errored batch, or a
   pre-fix Soulseek bundle stayed on disk forever. Added
   ``sweep_orphan_album_bundle_staging(staging_root, active_batch_ids)``
   that runs ONCE at server startup, before any batch can register
   a staging dir. Removes every ``<batch_id>``-shaped subdir
   whose id isn't in the active set. Safe by construction:
     - Only touches subdirs of the configured staging root.
     - Name-shape check (``entry.name == _safe_batch_dirname(entry.name)``)
       rejects hand-placed dirs like ``.git`` or stray docs.
     - ``shutil.rmtree`` errors log + continue — sweep must not
       crash app startup over a permission glitch.
     - active_batch_ids normalised through ``_safe_batch_dirname``
       so colon-bearing batch_ids match their on-disk form.
   Wired into the web_server startup right after the stuck-flags
   diagnostic so it fires before anything else touches batches.

Tests
- ``test_downloads_lifecycle.py`` gained one regression test
  pinning that Soulseek bundles now have their staging dir
  cleaned (sibling to the existing torrent test).
- ``test_album_bundle_staging_sweep.py`` (NEW, 11 tests)
  covers: orphan removal with no actives, active dirs preserved,
  special-char batch_id normalisation, no-op on missing /empty
  /empty-string staging root, non-dir entries skipped, unsafe-
  name dirs preserved (.git etc.), partial rmtree failure doesn't
  abort the rest, listdir failure returns 0 cleanly, default
  None active set, defensive against empty / None entries in
  the active set.

488 downloads tests pass.

For users with an existing "clean up old files" automation pointed
at this dir: stop pointing it there if you want — the auto-cleanup
+ startup sweep cover it now. Or leave it as belt-and-suspenders
with a relaxed (1h+) mtime threshold so it can't race a mid-batch
download.
2026-05-27 22:18:42 -07:00
Broque Thomas
f976a6da53 Fix: Soulseek album-bundle downloads stuck on "failed" after slskd
finished the release (#715)

Symptom (user @pavelcreates / @IamGroot60 on 2.6.2):
- Click Download on an album in the search modal
- slskd starts + completes every track of the release
- 22+ minutes after the last completed download, batch flips
  to "failed" with no clear log line explaining why
- Per-track Soulseek downloads on the same machine were fine

Root cause: ``core/soulseek_client._resolve_downloaded_album_file``
probed three hard-coded candidate paths to locate each downloaded
file in the slskd download dir:

  candidates = [
      download_path / remote_filename,
      download_path / basename,
      download_path / *normalized_path_parts,
  ]

On the common slskd config ``directories.downloads.username = true``
slskd writes files at ``<download_dir>/<username>/<filename>`` —
none of the three candidates carry a username segment, so the
resolver returned None for every file even though the file was
physically present in a subdir one level deeper. ``_poll_album
_bundle_downloads`` saw 0 completed_paths, kept spinning, and
hit the master deadline (~30 min) before bailing the batch.

Why per-track worked: ``web_server._find_completed_file_robust``
already does a recursive walk-by-basename + path-confirm against
the remote directory components, so any layout slskd writes ends
up resolved. The bundle path didn't go through it.

Fix
- Lifted the robust finder into ``core/downloads/file_finder.py``
  as a pure function ``find_completed_audio_file(download_dir,
  api_filename, transfer_dir=None) -> (path, location)``. Zero
  globals; recursive walk; handles slskd dedup suffix
  ``_<10+digit-timestamp>``, YouTube / Tidal ``id||title`` encoded
  filenames, the AcoustID-quarantine subdir skip, basename
  collisions disambiguated by remote-path components, and a
  fuzzy-basename fallback above 0.85.
- ``_resolve_downloaded_album_file`` keeps the three-candidate
  fast path (cheap probe for the slskd-flat default) but now
  delegates to the new helper when none hit, instead of giving up.
- ``_poll_album_bundle_downloads`` tracks "slskd reports
  Completed but local resolver returns None" per key. When every
  remaining key has been in that state past a 45-second grace
  window, the poll exits early with an explicit error pointing at
  the likely ``soulseek.download_path`` mismatch instead of
  silently spinning until the master deadline.
- ``web_server._find_completed_file_robust`` becomes a thin
  delegate so both callers share one finder. Legacy inline impl
  kept as ``_find_completed_file_robust_legacy`` for reference;
  to be removed next release.
- Fixed misleading ``"(0 tracks, quality=)"`` log on the preflight-
  reuse path — was reading attrs off a None ``picked`` object.

Tests (17 new in tests/downloads/test_file_finder.py)
- Flat slskd layout
- Username-prefixed (the #715 case)
- Full remote tree preserved
- Deeply nested username + tree
- File genuinely missing returns None
- Basename collision disambiguated by remote dirs
- Single basename match wins regardless of dirs
- slskd dedup suffix match
- Short ``_<digits>`` (year) not treated as dedup
- AcoustID quarantine subdir skipped
- YouTube / Tidal ``id||title`` encoded filenames
- transfer_dir fallback
- Both dirs miss → (None, None)
- Non-audio files ignored
- Empty api_filename
- Fuzzy match on punctuation variant
- Fuzzy rejects below threshold

475 downloads tests pass after the lift.
2026-05-27 21:20:37 -07:00
Broque Thomas
45ecf2730d Wishlist: harden Spotify backfill — poisoned tn=1 can't mask lean album
Residual per-track wishlist downloads (single tracks from different
albums, below the album-bundle threshold) were producing folders
without a year subfolder whenever the wishlist row carried a stale
``track_number=1`` from an older payload default.

Why: ``core/downloads/candidates.py`` had a single API-fetch branch
that served two concerns — resolving the track position AND
hydrating the lean ``spotify_album_context`` (release_date /
total_tracks / cover image) — gated entirely on track_number being
unresolved. When the wishlist row's ``track_number`` happened to
be 1 (a poisoned default rather than a real value), the gate
short-circuited and the album hydration the same call would have
done was skipped. Deezer-sourced discovery matches don't ship
release_date in their search-result album shape, so without the
backfill the folder lost its year.

The two concerns split:
  - track_number resolution keeps its track_info → track object →
    API precedence chain. track_info defaults still win.
  - album hydration runs whenever release_date or total_tracks are
    missing, independent of where (or whether) track_number was
    resolved.

The single API round-trip still serves both — the cost contract
is preserved. The side-effect coupling is gone.

Lifted into ``core/downloads/track_metadata_backfill.py``
(``hydrate_download_metadata``) so the precedence chain is pinned
in isolation. 24 unit tests cover the precedence chain, the
poisoned-tn=1 regression case, defensive non-dict/None inputs,
the cost guard (API called at most once per invocation), and
disc_number resolution.

Also lands the upstream piece: ``core/wishlist/routes.py:_build_track_data``
no longer defaults ``track_number=1`` / ``disc_number=1`` /
``total_tracks=1`` / ``release_date=''`` when the library-modal add
payload omits them. Missing values now flow through as ``None`` so
the downstream pipeline can detect-and-recover instead of locking
to a fake position.
2026-05-27 16:47:26 -07:00
Broque Thomas
6841128dc2 Wishlist: distinguish Queued from Analyzing for executor-pending batches
PR 4 of 4 in the wishlist-album-bundle issue series. UI fix only —
zero behavior change.

User's 26-track wishlist run rendered all 26 sub-batches as
"Analyzing..." simultaneously. Pre-fix the rows were created with
``phase='analysis'`` BEFORE being submitted to ``missing_download_executor``
(max_workers=3 by default), so 23 batches sat in the executor queue
visually identical to the 3 actually running. Misled users into
thinking SoulSync was processing 26 in parallel; really only 3 ever
ran at once with the rest waiting their turn.

Fix:
- Wishlist auto-flow submission sites now create batch rows with
  ``phase='queued'``.
- The master worker (``core/downloads/master.py:328``) already flipped
  phase to ``'analysis'`` as its first action on entry — that
  transition becomes the real signal that the executor picked the
  batch up.
- ``core/downloads/status.py`` surfaces ``analysis_progress`` for
  the ``queued`` phase too so the UI has the track count to render
  "Queued — N tracks" instead of an empty card.
- Frontend (``webui/static/pages-extra.js``, ``downloads.js``) renders
  "Queued " for ``phase='queued'`` distinct from the spinner-laden
  "Analyzing..." for ``phase='analysis'``.

Scope choices:
- Only the auto-wishlist submission sites flipped this PR
  (``core/wishlist/processing.py:860`` album sub-batches +
  ``core/wishlist/processing.py:907`` residual). The manual-wishlist
  sites at ``:451`` and ``:627`` use the same executor + worker, but
  those create a caller-allocated batch_id that the frontend polls
  immediately — wanted to verify the manual-poll path handles
  ``queued`` cleanly before flipping those. Trivial follow-up.
- Other submission sites in album_bundle_dispatch / web_server.py /
  task_worker.py left untouched — they don't go through the
  executor-queue pattern that causes this UI confusion.

Tests:
- Updated ``test_process_wishlist_automatically_creates_batch_for_matching_tracks``
  to assert ``phase='queued'`` on creation (was ``'analysis'``); explanatory
  comment names the executor-pool reason.
- New ``test_queued_phase_surfaces_analysis_progress_for_ui_count`` in
  ``tests/downloads/test_downloads_status.py`` pinning the new
  ``queued ⊂ analysis_progress`` rendering contract.
- 884 tests pass across wishlist + downloads + imports suites.
- Ruff clean on changed Python files; JS syntax OK on changed
  webui files.

PR 3 (sibling-completion gate) was investigated and dropped — the
"1/26 finalized" symptom turns out to be downstream of the
staging-match bug (PR 2's instrumentation will catch it on the
user's next reproduction run), not an independent sibling-gate bug.
The gate logic itself is correct.
2026-05-27 14:52:02 -07:00
Broque Thomas
4555ff7eb9 Wishlist modal: surface most-advanced live phase, not least-complete
The sibling-merge aggregator from 7f751202 used "least-complete
phase wins", which made the modal appear frozen during parallel
album bundle downloads. The task table is phase-gated to
downloading/complete/error in downloads.js — so whenever any
sibling was still in album_downloading, the merged phase stayed
there and tasks for the sibling that had advanced past its bundle
never rendered. User reported: both albums downloading on slskd,
modal blank until one completes fully.

Flip the rule: surface the most-advanced live phase so the modal
renders task progress as soon as any sibling reaches it. The
all-siblings-in-album_downloading case still surfaces
album_downloading (bundle progress UI is correct there); error
stays sticky.

Updated WHATS_NEW under 2.6.3 to describe the corrected behavior.
Two new tests pin the regression:
- downloading + album_downloading → downloading
- album_downloading + album_downloading → album_downloading
2026-05-26 22:35:53 -07:00
Broque Thomas
7f751202d2 Wishlist modal: merge sibling sub-batches into one status response
Phase 1c.2.1 splits each wishlist run across multiple
``download_batches`` rows (per-album bundle dispatch). The
download-missing modal opens against the original batch_id
allocated by ``start_manual_wishlist_download_batch`` /
``process_wishlist_automatically``. Pre-fix that batch_id was
just one sibling among N, so the modal went stale as soon as the
primary sub-batch finished — subsequent albums downloaded fine
but no live status reached the UI.

Fix: backend merges every sibling sub-batch's tasks +
analysis_results into the response keyed under the originally-
requested batch_id. Modal sees one unified view of the whole run
without knowing about the split. Frontend untouched.

Architecture (Kettui standards):

- ``core/downloads/wishlist_aggregator.py`` — pure
  ``merge_wishlist_run_status(primary, siblings)`` helper.
  No IO, no runtime state, no globals. Lifted out of
  ``status.py`` so the merge contract can be pinned via unit
  tests without standing up the live ``download_batches`` /
  ``download_tasks`` state.
- ``core/downloads/status.py``'s ``build_batched_status`` now
  pre-indexes ``download_batches`` by ``wishlist_run_id`` inside
  the existing ``tasks_lock`` snapshot, then runs the merge
  helper whenever a requested batch has a sibling.

Merge rules pinned by 12 tests:

- ``track_index`` re-indexed globally 0..N-1 across the merged
  ``analysis_results`` so the modal's ``data-track-index`` DOM
  keys don't collide between siblings. Tasks' ``track_index``
  follows the same remap so the analysis-results ↔ tasks
  cross-reference stays intact.
- ``task_id`` is uuid per task — no collision concern.
- Phase: error is sticky; otherwise the LEAST-complete
  pre-terminal phase wins (analysis < album_downloading <
  downloading). All-complete returns ``complete``; mixed
  complete + active returns ``downloading`` so the modal stays
  alive until every sibling lands.
- ``album_bundle``: picks whichever sibling currently has an
  active bundle download (state in
  ``{searching, downloading, downloading_release, staging}``).
  Falls back to the first non-empty bundle so a completed run
  still shows a progress bar.
- ``analysis_progress`` summed across siblings.
- ``active_count`` summed; ``max_concurrent`` keeps primary's
  value as the representative.
- ``playlist_id`` + ``playlist_name`` preserved from the primary
  (the row the modal originally opened against).

Legacy single-batch wishlist runs (no ``wishlist_run_id`` on the
batch) skip the merge entirely — passthrough. Back-compat by
absence.

1108 tests across downloads + wishlist + automation + imports +
playlist-sources + lb-series suites green. 12 new aggregator
tests pin the merge contract.

Closes the open UX gap from the Phase 1c.2.1 ship — modal now
tracks every sibling sub-batch's progress for the full duration
of the wishlist run.
2026-05-26 22:17:04 -07:00
Broque Thomas
b5755d6307 Trust user manual picks past AcoustID verification (#701)
When a task failed AcoustID verification and got quarantined, opening
the candidates modal and manually picking a different file would just
re-quarantine it. The manual-pick path through
`_attempt_download_with_candidates` ran full post-processing with no
quarantine bypass — so if the alternate file disagreed with AcoustID's
stored metadata too (common for live versions, remasters, regional
title differences, fingerprint coverage gaps) the file landed right
back in quarantine. User got stuck in the loop.

The Approve button on quarantined rows already handles the "I want
this exact file" case via `_skip_quarantine_check='all'`. The
candidates modal handles the "I want a different file" case — same
user intent, opposite direction, but the bypass plumbing didn't carry
through.

`/api/downloads/task/<id>/download-candidate` already sets
`task['_user_manual_pick'] = True`. `attempt_download_with_candidates`
now reads that flag under tasks_lock alongside `used_sources` and,
when set, injects `_skip_quarantine_check='acoustid'` plus
`_user_manual_pick=True` into the stored `matched_downloads_context`
entry. The acoustid-only scope is deliberate: integrity + bit-depth
gates still run because those check the new file's actual condition
(corruption, sample rate) rather than its identity — only the
metadata-mismatch gate is the user-override case.

Auto-search picks (the normal task-worker path) leave the flag unset
and continue to run full AcoustID verification, preserving the
existing safety net for non-user-initiated downloads.

Tests:
- positive: manual-pick task → stored context has
  `_skip_quarantine_check='acoustid'` and `_user_manual_pick=True`
- negative: auto-search task → stored context has neither key,
  AcoustID still runs as before

Full suite 3976 pass.
2026-05-26 09:56:49 -07:00
Broque Thomas
85ba93f16f Fix album-bundle staging match + wishlist provenance (#700, #698)
Root cause (#700): the Soulseek album-bundle path downloads whole
releases into a private staging dir, then per-track workers claim
those files via the staging-match shortcut. When slskd files arrived
without ID3 tags (common for FLAC rips), the staging cache fell back
to the filename stem as the title — and stems shaped like
"Artist - Album - 03 - Title" could not clear the 0.80 title-
similarity threshold against the clean Spotify track name. Every
track in the album went not_found, the batch ended "failed" in the
Downloads UI with an empty queue, and the bundle-downloaded files
just sat unused in staging.

Fix: in _staging_title_variants, add a trailing-title variant by
extracting the segments after a bare track-number block (e.g. "03")
between " - " delimiters. Conservative — only fires when a clear
digit segment is present, so real song titles with dashes like
"Hold Me - Live" are left intact. Generated as an additional variant
alongside the existing raw/compacted/feat-stripped/bonus-stripped
forms, so behavior on already-matching files is unchanged.

Downstream (#698): the album-bundle staging miss pushed every failed
track to the wishlist labelled as a playlist track, and a couple of
fallback paths in ensure_wishlist_track_format and the slskd-result
reconstruction hardcoded album_type='single' / total_tracks=1 on the
stored album dict. On wishlist requeue the path builder saw
album_type='single' and routed the download through single_path,
dumping the file in the Singles tree even though it belonged to an
album. (Running Reorganize would fix it because the DB album linkage
was still correct, but the file landed in the wrong place first.)

Fixes:
- new resolve_wishlist_source_type_for_batch() returns 'album' for
  is_album_download batches; wishlist_failed.py now calls it instead
  of hardcoding 'playlist'
- build_wishlist_source_context() threads album_context /
  artist_context / is_album_download from the batch into the wishlist
  row so future requeue logic has authoritative routing data
- the non-dict-album fallback in ensure_wishlist_track_format and
  the slskd-result reconstruction default album_type='album' (and
  total_tracks=0 = unknown) instead of lying with 'single'/1; the
  existing setdefault chain handles dict-shaped album data unchanged

Tests:
- 2 staging-match tests pin the new tail-extraction behavior against
  a realistic untagged slskd stem, plus a negative test that confirms
  a dash-in-title without a digit segment still does NOT extract a
  variant
- 2 payload tests pin the album_type='album' default for both
  fallback paths
- 4 processing tests pin resolve_wishlist_source_type_for_batch()
  and the album-context threading in build_wishlist_source_context()

3974 pass; no behavioural change on already-working flows.
2026-05-26 07:12:49 -07:00
Broque Thomas
a7ca7ddfad Harden album bundle fallback flow
Delay torrent and usenet album-bundle dispatch until missing-track analysis confirms there is work to do, matching the Soulseek album flow and avoiding release downloads for already-owned albums.

Clear private album-bundle staging state when a release-level source intentionally falls back to per-track mode so workers can use the normal staging/search path instead of an empty private bundle directory.

Verified by user: focused downloads master tests passed, 2 passed.
2026-05-24 16:15:36 -07:00
Broque Thomas
0bea332aed Preserve album bundle track numbers
Keep album-bundle staging from replacing known per-track album numbers with the filename parser's default when staged files do not expose a real track number. Carry staging tag numbers through the cache, fall back to task metadata for private release staging, and cap hybrid album batches to one worker when Soulseek is first in the source order.
2026-05-24 13:59:44 -07:00
Broque Thomas
9a0e3b4011 Persist completed downloads in downloads view
Include a capped recent tail of database-backed download history in the unified Downloads page so completed Deezer and other streaming downloads remain visible after runtime tasks are cleaned up or the container restarts. Use persistent download history for the dashboard finished count, keep live tasks authoritative for active rows, avoid showing the local clear-completed action for persisted history rows, and cover history hydration/deduping/capping in status tests.
2026-05-24 10:02:00 -07:00
Broque Thomas
6c226613bf Add Soulseek album bundle downloads
Route primary Soulseek album downloads through the album-bundle staging flow, reusing preflight-selected folders when available. In hybrid mode, only the first configured source can claim whole-album bundle behavior so later Soulseek fallback keeps the existing per-track/source-reuse path.

Allow Soulseek album bundles to stage completed tracks when some same-source transfers fail or time out, and keep partial bundles from blocking per-track fallback. Add coverage for dispatcher gating, master flow ordering, task-worker staged-miss behavior, and Soulseek bundle polling.
2026-05-23 15:08:21 -07:00
Broque Thomas
a41eccbe3c Fix Usenet settings reload without restart
Refresh registry-backed download plugins when settings are saved so cached Prowlarr clients pick up new indexer credentials immediately. This preserves active download state by reloading existing plugin instances instead of rebuilding the registry.

Add regression coverage for orchestrator reload fanout and the Usenet plugin's cached ProwlarrClient refresh path.
2026-05-22 08:28:56 -07:00
Broque Thomas
763888e671 Support legacy HiFi track manifests
Add fallback support for public hifi-api instances that expose playback through /track/ instead of /trackManifests/. The capability checker now accepts either manifest shape, and downloads can use direct URLs decoded from the legacy base64 manifest.

Tests cover legacy instance capability detection and download-manifest fallback while preserving the newer trackManifests path.
2026-05-21 18:11:14 -07:00
Broque Thomas
fae13226e5 Check HiFi download capability via manifests
Probe public HiFi instances with the same trackManifests endpoint used by real downloads instead of the legacy /track endpoint. This prevents compatible instances from being falsely labeled search-only in Settings.

Centralize HiFi instance capability checks in HiFiClient and reuse manifest URI parsing with the download path.

Tests cover manifest-based capability detection, no legacy /track probe, and limited instances without a manifest URI.
2026-05-21 18:04:10 -07:00
Broque Thomas
dee200b267 Add torrent usenet PR notes and test updates
Adds the PR description for the torrent/usenet release-staging feature and updates drifted tests for the new plugin registry entries and search exclude_sources signature.

Verified with the focused pytest command covering cancellation and default registry source registration.
2026-05-21 15:28:48 -07:00
Broque Thomas
6c9b43225a Add torrent and usenet release staging support
Adds torrent/usenet as release-oriented download sources with album-bundle staging, live progress reporting, and post-processing that selects the requested audio file from completed releases instead of blindly importing the first file.

Keeps album-bundle behavior gated to single-source torrent/usenet album downloads, excludes release sources from hybrid album per-track searches, and allows hybrid non-album tracks to use release results safely.

Improves staged-release matching for featured/bonus track filenames while preserving version mismatches, records torrent/usenet provenance in library history, and updates service/status UI labels.

Covers the flow with focused lifecycle, status, staging, validation, task worker, post-processing, and import side-effect tests.
2026-05-21 14:22:21 -07:00
Broque Thomas
8b0de9eb76 fix(downloads): harden album bundle staging
Route torrent and Usenet album bundles through private per-batch staging so Auto-Import cannot race public staging or duplicate imports.

Expose album-bundle progress in batch status and render it on the Downloads page while the external client is still downloading.

Tighten release handoff safety by rejecting archive path traversal, ignoring torrent candidates without a usable URL, and skipping Soulseek source reuse for torrent/Usenet batches.

Tests: .venv/bin/python -m pytest tests/downloads/test_downloads_status.py tests/test_album_bundle_dispatch.py tests/downloads/test_downloads_staging.py tests/test_torrent_usenet_plugins.py
2026-05-20 21:39:06 -07:00
Broque Thomas
3375b6c4bd Handle non-JSON Tidal auth responses
Detect JSON decode-like exceptions from Tidal's token endpoint and return a safer, more actionable error message. Adds a _looks_like_json_decode_error helper and special-cases that error in check_device_auth to log the non-JSON response and advise disabling VPN/proxy/network filtering and restarting SoulSync. A test was added to ensure the user-facing message does not leak the raw exception text while still returning an error status. Other errors continue to fall back to the existing behavior.
2026-05-20 14:04:45 -07:00
Broque Thomas
2fc08e199e Enforce duration tolerance for strict sources
Add duration tolerance logic and pre-download rejection for structured sources (tidal, qobuz, hifi, deezer_dl, amazon) when candidate duration deviates beyond allowed tolerance. Introduces helper functions _duration_tolerance_seconds and _duration_mismatch_exceeds_integrity_tolerance and uses resolve_duration_tolerance from core.imports.file_integrity. Log and skip candidates that would fail post-processing integrity checks to avoid wasted downloads. Update tests to include matching engine stub and new cases covering rejection and acceptance based on duration tolerance; also adjust imports and test fixtures.
2026-05-20 11:24:57 -07:00
Broque Thomas
79ad4d885d fix(quarantine): drop already-quarantined sources from candidate picker (#652)
When a file failed AcoustID verification and got quarantined, the next
auto-wishlist cycle would search for the same track, the deterministic
quality picker would re-select the same (uploader, filename) source,
re-download it, and re-quarantine it. Users woke up to hundreds of
duplicate .quarantined entries from a single bad upload — same source
URL repeatedly, byte-for-byte identical files.

Root cause: `SoulseekClient.filter_results_by_quality_preference` ranks
candidates by quality + bitrate density only. Quarantine history wasn't
consulted, so a high-bitrate FLAC upload with a wrong-track AcoustID
fingerprint kept winning the picker against every other candidate.

Fix shape:

- New helper `core/imports/quarantine.py::get_quarantined_source_keys`
  reads every quarantine sidecar's `context.original_search_result`
  and returns the set of `(username, filename)` tuples for O(1)
  membership checks. Sidecars missing the context field (legacy thin
  sidecars written pre-Feb 2026, or orphaned files) and corrupt JSON
  are skipped silently — defensive against transient FS / encoding
  issues.

- `SoulseekClient._drop_quarantined_sources` runs the membership
  filter against incoming TrackResults, drops matches, logs a single
  INFO line with the skip count. Called first inside
  `filter_results_by_quality_preference` so all four callers
  (search-and-download, master worker, validation, orchestrator)
  benefit transparently.

- Approving or deleting a quarantine entry removes its sidecar, so
  the dedup key disappears from the set on the next search — gives
  the user a way to opt back in to a previously-quarantined source
  without restarting the app.

7 helper tests cover: missing dir, empty dir, well-formed sidecars
collected as tuples, legacy sidecars skipped, empty source fields
skipped (so empty-string keys can't accidentally drop unrelated
results), corrupt JSON tolerated, duplicate quarantines collapse.

5 integration tests pin: clean candidates pass, known-bad candidates
drop, missing quarantine dir returns input unchanged, filesystem
errors swallowed (defensive), full `filter_results_by_quality_preference`
runs the dedup BEFORE the quality picker — so a high-quality
quarantined source can't win on bitrate.

692 existing download + import tests still green. Cosmetic surface
of the fix is invisible — same UX as today when no quarantine entries
exist; loop only kicks in once a sidecar has been written.

Out of scope: bulk-select / multi-delete UI for the quarantine tab —
S-Bryce mentioned this as a separate pain point in the issue, but
it's its own UX work, not a one-commit drive-by.
2026-05-19 21:19:50 -07:00
Broque Thomas
54e4ba843f fix(soulseek): suppress connection-error log spam when slskd unreachable (#649)
When slskd_url is configured but the host is unreachable (slskd not
running, wrong port, host.docker.internal not resolving), the frontend's
/api/downloads/status polling fanned out to every download plugin
including Soulseek. soulseek_client._make_request hit a DNS / connect
failure on each poll and logged it at ERROR. Result: one
"Cannot connect to host host.docker.internal:5030" log line every
~2-3 seconds for the entire duration of any download — visible spam
even when the user wasn't using Soulseek at all.

Caught aiohttp.ClientConnectorError explicitly in both _make_request
and _make_direct_request. First failure emits one WARNING with
actionable context (start slskd, or clear soulseek.slskd_url if you
don't use Soulseek). Subsequent failures demote to DEBUG. The
_last_unreachable_logged flag resets on any successful (200/201/204)
response so a later outage warns again — suppression is per-outage,
not per-process-lifetime. Same shape as the existing _last_401_logged
suppression for auth failures.

The architectural gap (status polling fans out to soulseek even when
the user has soulseek disabled in their active download sources) is
intentionally left for a follow-up. The plugin-iteration code lives
in core/download_engine/engine.py and core/download_orchestrator.py;
threading a "skip-when-not-active" gate through every caller is a
bigger refactor than this user-facing log cleanup warrants. The
WARNING-once message tells the user what to do in the meantime.

5 new pinning tests cover the suppression contract: connection error
returns None (not raises), first failure WARNs + sets flag, repeats
stay quiet, successful response resets the flag, _make_direct_request
follows the same pattern, and non-connection exceptions still log at
ERROR so real bugs aren't hidden behind the new suppression.
2026-05-19 19:44:17 -07:00
Broque Thomas
aaf312cd34 Honor manual library matches across source labels
Manual matches can be created from sync history as mirrored while wishlist and download flows later see the same track as wishlist or a provider source. Add a shared track-level lookup that falls back from exact source/id to source_track_id and title/artist, then use it for wishlist adds, cleanup, and download analysis so mapped tracks are not re-added or redownloaded.

Add coverage for mirrored-source matches being honored by wishlist cleanup and download batches, including the internal wishlist force-download path.
2026-05-18 16:32:38 -07:00