Diagnostics revealed the real cause: the tracks table stores file_path
RELATIVE to the library root (e.g. "Asketa/Another Side/01-01 - Another
Side.flac"), so probing the raw path failed for the entire library — every
track came back unprobeable and was left unflagged ("20/20 could not be
probed").
The scanner now resolves each path via _resolve_library_file_path (checks
transfer/download/library dirs, same helper the rest of the app uses) before
probing, falling back to docker_resolve_path. Injected via deps for testability.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A task stuck in 'post_processing' past the cutoff was force-marked 'completed'
("assume it worked"). In a large batch, post-processing (AcoustID + quality +
import) is serialized and backs up, so tasks sit in post_processing while merely
QUEUED — then got falsely completed, showing as downloaded with no file on disk
(/Transfer empty).
Now: the cutoff is 30 min (was 5) so legit backlog isn't cut off, and when it
does fire the task is only completed if it actually produced a file
(final_file_path exists on disk) — otherwise marked failed (honest + retryable).
Applied at both stuck-detection sites (check_batch_completion + _v2).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The worker used logging.getLogger(__name__) → "core.discovery.quality_scanner",
which the app log view (soulsync.*) doesn't surface — so the scan looked like it
did nothing ("API Starting scan" straight to "quality_scan_completed" with no
worker output). Switched to get_logger("discovery.quality_scanner") so "Found N
tracks", "Profile targets", and the unprobeable-file diagnostics show up.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The unified scanner must READ each file to judge real bit depth/sample rate
(extension alone can't tell 16-bit from 24-bit FLAC). If the stored library
path doesn't resolve to a readable file in this container, every probe returns
None and — since an unprobeable file can't be judged — the whole library passes
silently ("scans nothing").
Now: resolve the path via docker_resolve_path before probing, and count +
log unprobeable files (first 5 paths at WARNING, plus an end-of-scan summary
"N/M tracks could not be probed"). This makes a systematic path/mount mismatch
visible instead of an empty result.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The library quality scanner judged quality by FILE EXTENSION only
(get_quality_tier_from_extension) and read the legacy v2 `qualities` dict —
so every FLAC was "lossless tier 1" regardless of bit depth / sample rate. It
could never flag a 16-bit FLAC as upgradeable under a 24-bit profile, and it
ignored the v3 ranked_targets entirely. Completely inconsistent with the
download guard.
Now both share one core:
- selection.targets_from_profile(profile) — single profile→targets conversion
(v2→v3 migration), reused by load_profile_targets.
- selection.quality_meets_profile(aq, targets) — strict: meets iff the real
measured quality satisfies a ranked target (fallback ignored — it's a
download concession, not a definition of "good enough").
- guards.check_quality_target refactored to use both.
- quality_scanner probes real quality (probe_audio_quality) and checks against
the v3 targets via quality_meets_profile. Extension tier kept only as a
fallback label when a file can't be probed.
Result: the scan flags exactly what the download gate would reject — 16-bit
when you want 24-bit, wrong sample rate, MP3 when you want FLAC.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A file quarantined for QUALITY (e.g. MP3-VBR rejected by a FLAC-only profile)
showed up in BOTH the Completed and Quarantine tabs. Cause: the verification
wrapper (post_process_matched_download_with_verification) handled the
_acoustid_quarantined / _integrity_failure_msg / _race_guard_failed markers but
NOT the quality marker _bitdepth_rejected (nor _silence_rejected). A quality
quarantine leaves no _final_processed_path, so the wrapper hit the
"no final path — assuming success" branch and marked the task Completed.
Unlike acoustid/integrity (retry driven by the wrapper), the inner pipeline
already owns the quality/audio-guard outcome — it quarantines then re-queues the
next-best candidate or marks the task failed. So the wrapper now just returns
when it sees _bitdepth_rejected/_silence_rejected, without marking completed
(which clobbered both the quarantine state AND any successful retry).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root cause of "completed/failed/unverified don't show during a running
batch, only after it ends" (F5 didn't help): build_unified_downloads_response
sorted live tasks active-first (downloading/searching/queued = priority 0-3,
completed/failed = 4-7) then truncated the whole array at items[:limit] (300).
During a busy batch the active+queued tasks filled the limit and pushed every
terminal task off the end, so /api/downloads/all never returned them — the
Completed/Failed/Unverified tabs filter client-side and had nothing to show.
Fix: `limit` now bounds only the persistent-history tail. Live in-memory
tasks are always returned in full — they're already bounded by the 5-min
cleanup automation, and array order is presentation-only since the page
filters per tab client-side.
Verified with a repro (320 queued + 1 completed + 1 failed → terminal rows
were absent at limit=300; now present).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three fixes from on-device testing of best-quality mode:
1. clear_completed_local no longer prunes terminal tasks that belong to a
STILL-ACTIVE batch (one with non-terminal work remaining). The 5-min
"Clean Completed Downloads" automation was yanking completed/failed/
unverified rows out of download_tasks mid-run — and failed/cancelled
aren't in library_history — so they only reappeared after the batch
ended. Now the whole active batch stays intact until it finishes.
2. search_all_sources runs every source CONCURRENTLY (asyncio.gather)
instead of sequentially, so the pool waits only for the slowest source
(e.g. usenet/Prowlarr) in parallel rather than summing all latencies.
3. The pool log now reports per-source contribution counts
(e.g. "usenet=0, hifi=11, soulseek=1") instead of just echoing the
chain, so a release-level source that returns nothing for a track-title
query is visible rather than appearing to have been searched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds an opt-in search strategy toggle in the Quality Profile:
- priority (default): unchanged — first source in the hybrid chain that
meets a quality target wins.
- best_quality: pool candidates from EVERY source per query and download
them best→worst by actual audio quality; source order only breaks ties.
Implementation reuses existing plumbing so the retry system is untouched:
- engine.search_all_sources pools raw tracks across all configured,
non-exhausted sources (no first-source short-circuit).
- candidates.order_candidates: new quality_first sort path — profile
quality rank dominates, confidence/peer signals break ties. Priority
path is byte-for-byte unchanged (regression-locked by tests).
- task_worker passes quality_first + targets through; skips the redundant
hybrid-fallback block in best-quality mode (pool already covered it).
- Per-source retry budgets unchanged: a source that spends its budget is
added to exhausted_download_sources and thus dropped from the whole
pool. Independent of post_processing.retry_exhaustive.
- Query generator NOT touched.
Also clarifies the "Allow fallback" setting wording: it accepts OFF-LIST
quality as a last resort (not "walk down my list"), and notes that
lossy_copy.downsample_hires also bypasses the quality gate — the cause of
16-bit/MP3 files slipping through a 24-bit-only profile.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
get_audio_quality_string now appends the FLAC sample rate so the Downloads
quality chip and library history read e.g. 'FLAC 24bit/96kHz' instead of just
'FLAC 24bit' — surfaces hi-res frequency (44.1/48/96/192kHz) at a glance.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Unverified review actions (play/audit/approve/delete) only rendered for
persistent-history rows, so a freshly-completed unverified download — still a
live task without a 'history-<id>' task_id — showed no buttons until it aged
into history (Quarantine always worked because it uses the quarantine entry
id). Thread the library_history row id from import through to the live task
(add_library_history_entry now returns lastrowid -> context._history_id ->
task.history_id -> /api/downloads/all), and resolve verifHistoryId from it.
Also surface the real probed audio quality (mutagen-read from the file, e.g.
'FLAC 24bit') on completed rows as a chip, so you can see what was actually
downloaded.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Some Monochrome instances only have 30-second Tidal DOWNLOAD access: the HLS
variant playlist for a 220s track comes back as ~30s of segments + ENDLIST
(verified live on us-west.monochrome.tf — lossless=30s, hires=403). The client
downloaded that 30s file, which then got quarantined by the new audio guard.
Detect it at manifest time: sum the playlist's EXTINF runtime and compare to
the track's real duration (get_track_info). When the playlist is < 85% of the
track, decline the manifest and rotate the instance, so the download falls
through to a real source (Soulseek/Qobuz/Tidal/Deezer) instead of fetching a
preview. Best-effort — unknown duration disables the check (the post-download
audio guard remains the safety net).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The actual HiFi/Monochrome bug isn't silence padding — it's a TRUNCATED file:
the container claims the full length (e.g. 3:08) but only ~30s of audio
decodes. silencedetect finds nothing (there's no silent audio, just missing
audio) and ffmpeg's time= even reports 0 with no error, so the duration and
quality guards all pass.
Detect it by decoding and comparing the real audio length (astats sample
count / sample rate) against the container duration: reject when the real
audio covers < 85% of the claimed length. detect_broken_audio() runs this
truncation check first, then the silence-ratio check. Wire it into the guard
that runs at the integrity/length verification point.
Verified on the real file: 'only ~30s actually decodes of a 188s file (16%)';
a normal 180s file is not flagged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The silence guard sat after the quality guard, so a strict quality profile
quarantined every file before silence detection ever ran. Move it to right
after check_audio_integrity (where the length is verified) and before the
AcoustID/quality gates, so a mostly-silent file is caught regardless of its
quality verdict and reported with the correct reason. Same quarantine +
next-candidate retry pattern (trigger='silence').
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
HiFi/Monochrome HLS assembly can produce a file with the correct container
duration but only ~30s of real audio + silence padding — the duration and
quality guards both pass, so nothing caught it until you listened. Add
core/imports/silence.py: ffmpeg silencedetect over the audio, reject when the
silent fraction exceeds 50%. Wire it into the post-download pipeline with the
same quarantine + next-candidate retry pattern as the quality guard
(trigger='silence'), and surface it via import_rejection_reason. Fails open
when ffmpeg/mutagen are unavailable so tooling problems never quarantine a
legit file.
Also mark 'quality filter' and 'silence guard' failures as recoverable
quarantine rows in the downloads UI (were shown as plain failures).
Verified end-to-end: a 30s-tone + 180s-silence FLAC is flagged '86% silence
(only ~30s audible of 210s)'; a 210s tone passes. 7 parser unit tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Remove the per-source download-quality dropdowns (Tidal/HiFi/Qobuz/Deezer/
Amazon) — with the global ranked-targets system they were redundant and
conflicting. Add quality_tier_for_source(): picks the LOWEST source tier
that satisfies the user's top target (respects the quality ceiling, saves
bandwidth) or the source's max as best effort. Every source's search +
download + retry path now derives its tier from the global profile instead
of config_manager.get('<source>_download.quality').
Settings keep the per-source allow_fallback toggles; the quality selects are
replaced with a note pointing at Quality Profile.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
search_and_download_best applied confidence scoring to streaming results
but never quality-ranked them — only the Soulseek path did. Apply
rank_for_profile to the confidence-passing survivors so the best version
wins (match first, then quality). Stable ranking keeps confidence order
within an equal tier; an "or scored" fail-safe keeps a candidate to try.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add core/quality/selection.py: rank_with_targets() returns (ranked,
satisfied) where satisfied = a candidate meets a real target (strict).
load_profile_targets()/rank_for_profile() are the DB-backed wrappers.
search_with_fallback now skips a source that can deliver no target-meeting
quality and escalates to the next (source priority still wins among
satisfying sources; first source's results kept as fallback unless the
profile disables it). Returns RAW tracks — the satisfied check is a coarse
source gate; match-filtering + final ranking stay in the orchestrator so
the correct track is never pruned. Ranking is fail-open: a ranking error
never drops a source's real results.
Tested: rank_with_targets satisfied/fallback matrix + engine escalation,
stop-on-first, raw-not-pruned, fallback on/off. Amazon field test updated
for the corrected format token.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add core/quality/source_map.py centralising each source's tier->AudioQuality
mapping (Tidal/HiFi tiers, Qobuz real kHz/bit-depth, Deezer codes, Amazon
codec/tier). Add TrackResult.set_quality() to merge a mapped AudioQuality
onto a result. Wire HiFi, Qobuz, Deezer, Tidal, Amazon search results to
stamp real sample_rate/bit_depth so the global ranker no longer relies on
crude kbps heuristics for streaming sources. Fixes Qobuz/Amazon display
labels ('FLAC 24-bit/192kHz', 'Lossless') breaking format derivation.
Tested: 22 passing (mappers + set_quality merge semantics).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replaces the Soulseek-only bit-depth heuristic with a source-agnostic
quality system that works across all download sources.
## core/quality/model.py (new)
- AudioQuality dataclass: format, bitrate, sample_rate, bit_depth
- QualityTarget: one ranked entry in the user's priority list
- filter_and_rank(): source-neutral candidate ranking
- rank_candidate(): scores any AudioQuality against ranked_targets
- v2_qualities_to_ranked_targets(): migration helper
## core/download_plugins/types.py
- SearchResult gains sample_rate + bit_depth fields
- audio_quality property returns unified AudioQuality
- AlbumResult gets audio_quality aggregated from tracks
## core/soulseek_client.py
- Parses slskd attributes array (type 4=sample_rate, type 5=bit_depth)
- Real values instead of kbps heuristic
- filter_results_by_quality_preference() replaced by filter_and_rank()
## database/music_database.py
- Quality profile v3 with ranked_targets list
- Auto-migration v2 → v3 on load
- Presets (audiophile/balanced/space_saver) updated to v3
## core/imports/file_ops.py
- probe_audio_quality(): reads actual downloaded file via mutagen
returns AudioQuality with ground-truth values
## core/imports/guards.py
- check_quality_target(): replaces check_flac_bit_depth
checks all formats/sources against ranked_targets
- check_flac_bit_depth() kept as backwards-compat wrapper
## core/imports/pipeline.py
- Uses check_quality_target() instead of check_flac_bit_depth()
- Quality mismatch triggers _requeue_quarantined_task_for_retry('quality')
so next-best candidate is tried before failing (same as AcoustID)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two issues in the same path:
1. The shared discovery modal pre-renders one row per track from a
separately-fetched frontend track list, then the poll dropped any backend
result without a pre-rendered row (if (!row) return). When the frontend's
track fetch came back rate-limited/partial (~21) while discovery's own fetch
got all 59, the surplus results vanished. Now the modal CREATES a row for any
result lacking one, so authoritative backend results drive the list (fixes
all sources sharing the modal).
2. get_playlist hydrated a whole relationships page in one _get_tracks_batch
call, but Tidal caps filter[id] at 20/request, silently truncating larger
pages. Chunk to the cap like get_album_tracks already does.
Seam + regression tests (tests/test_tidal_playlist_batch_chunking.py).
Status checks asked is_spotify_authenticated() (official OAuth only) instead of
is_spotify_metadata_available(), so a Spotify-Free primary read as disconnected.
get_primary_source_status had spotify_free awareness but it was dead code:
get_client_for_source('spotify') returns None unless officially authed, so the
free-availability probe never had a client. Fetch the client directly for that
check; add the missing free branch to the dashboard test message. Seam + regression tests.
The reconcile read each completed task's final_file_path to find paths — but not
every import path sets it (the verification worker marks the task completed
without it), so tracks that imported via that path were silently dropped (user
saw 3 of 5 symlinks). Root cause: leaning on a fragile per-task field.
Now reconcile_batch_playlists identifies the organize playlists the batch touched
(its own + any reached via a completed track's source_info provenance) and
rebuilds each from CURRENT library ownership via _rebuild_one_from_db
(check_track_exists over membership). It just asks the library what's owned, so
it's robust to HOW a track imported (modal worker / slskd monitor / verification
worker) and still prunes tracks that left. Takes a db handle; all three callers
pass MusicDatabase().
Reconcile tests rewritten for the DB-rebuild form (organize batch, wishlist
provenance, non-organize skip, plain no-op). 973 downloads/imports/playlist
tests pass.
on_download_completed and check_batch_completion_v2 are duplicate completion
paths. Monitor-detected downloads (Deezer / slskd-monitor / verification-worker
imports) finish the batch via the V2 path, but the materialize reconcile was
only added to on_download_completed — so those batches never built playlist
folders (no '[Playlist Folder] Rebuilt' line at all). Add the same non-fatal
reconcile to the V2 path. Now all three completion points (both lifecycle paths
+ the master.py all-owned path) materialize. 550 tests pass.
Symmetric to the post-download reconcile (which handles ADDITIONS): when a
playlist's membership is re-synced (the mirror step — scheduled refresh or the
manual mirror endpoint), rebuild its folder from current membership WITH prune
IF it's organize-by-playlist. So a track that just LEFT the playlist has its
symlink cleaned up the instant membership changes, not only on the next download.
Factored a shared _rebuild_one_from_db (used by the manual 'Rebuild' button and
the mirror hook) + rebuild_mirrored_playlist_if_organized. Gated to organized
playlists, non-fatal at both mirror call sites.
Now the invariant 'folder = the playlist's current owned members' holds on every
change: additions caught at download, removals caught at mirror. 2 new tests
(removed track pruned; non-organized skipped). 985 + 277 tests pass.
Replaces the two organize-only triggers with a single reconcile_batch_playlists
called at both batch-completion points. It groups the batch's newly-resolved
tracks by their per-track playlist provenance:
- the batch's OWN organize playlist → full (re)build with prune, and
- a track that completed for a DIFFERENT playlist (e.g. a WISHLIST fulfilling a
track that belongs to an organize playlist) → ADDED to that folder, no prune.
So a late wishlist arrival now lands in its playlist folder immediately, instead
of only on the next sync/manual rebuild — the folder = the playlist's owned
members, kept true on every ownership change regardless of download path. Uses
the paths the batch already captured (no DB re-match, no waiting on the server
scan/sync). Non-fatal.
3 new reconcile tests (organize full-rebuild, wishlist add-without-prune, plain
batch no-op). 983 downloads/imports/playlist tests pass.
- Settings: 'Playlists Folder' path field (Unlock pattern, separate-root help
text), a Symlinks/Copies selector, and a 'Rebuild playlist folders now' button
(standard test-button style). Wired through PATH_INPUT_IDS / load / save, plus
'playlists' added to the settings save allowlist so it persists.
- POST /api/playlists/materialize/rebuild → rebuild_organized_playlists_from_db:
rebuilds every organize-by-playlist folder from CURRENT ownership, re-matching
each track with check_track_exists (name, not IDs) so it self-heals after a
reorganize / membership change. +1 test.
70 materialize tests + JS integrity pass; settings round-trip wiring verified.
- Routing (step 5): organize-by-playlist tracks no longer set the per-track
_playlist_folder_mode flag, so they import NORMALLY into Artist/Album — exactly
what a normal download does. _playlist_name provenance is kept (origin.py).
- Triggers (step 4): build the playlist folder from the batch's own payload at
both end-of-flow points — the all-owned path in master.py (no downloads, so the
lifecycle never runs) and the batch-complete hook in lifecycle.py (after
downloads). Both gated on playlist_folder_mode, both non-fatal.
Works for the all-owned case (the smack test that did nothing before) and for
mixed owned/downloaded, with no source-ID or mirrored-playlist dependency. The
materialized folder uses the default ./Playlists root + symlink mode until the
Settings UI is added.
Updated the master test to assert the new contract (provenance kept, routing
flag gone). 979 tests pass.
materialize_playlist_from_batch(batch, download_tasks, config) collects the real
on-disk path of every resolved track from the batch's OWN payload — owned via
analysis_results.matched_file_path, downloaded via tasks.final_file_path — runs
each through the playback path resolver (Docker-correct), de-dupes, and hands the
list to rebuild_playlist_folder. Gated on playlist_folder_mode.
No re-matching, no source IDs, no mirrored-playlist lookup — works for any
organize-by-playlist download including the all-owned case. 5 tests. Still
isolated; the triggers wire it in next.
- settings: playlists.materialize_path (separate root, mapped apart from the
music library so the media server never double-scans it) + materialize_mode
(symlink|copy).
- core/playlists/materialize.py: pure filesystem engine that (re)builds a
playlist folder of relative symlinks (or copies) into the real library —
idempotent, prunes stale entries, disambiguates filename collisions, never
escapes the root, and auto-falls-back to copy when the FS can't symlink.
No DB, no app state; ops injectable. 13 unit tests.
Isolated + additive — nothing live calls this yet (stitcher/trigger/routing
come next).
The download analysis already matches every track to a library row via
check_track_exists / manual match, then discarded the result. Keep it: each
analysis_results entry now carries matched_file_path + matched_track_id (the
owned file's real location, or None). Symmetrically, a completed download task
now records final_file_path (where the import landed).
Purely additive, no behavior change, no new matching, zero perf cost — just
stops throwing away what the pipeline already computed. This is the foundation
for playlist materialization: owned + downloaded tracks both report where their
real file is, so the folder can be built by name match, not source IDs.
A DB<->filesystem path mismatch (Docker volume change, remount, Music
Paths unset for the container) makes EVERY library file fail to resolve
to a DB track, so the orphan detector flags the whole library as
orphaned. The mass-orphan check only logged a warning and then created
the findings anyway — so a user batch-applying 'move to staging' or
'delete' would relocate or wipe their entire library.
Make it a hard skip (create zero findings) like the dead-file cleaner
and stale-removal paths already do (#828). Centralise the predicate as
is_implausible_orphan_flood() alongside is_implausible_stale_removal()
so the rule lives in one tested place. Small genuine orphan sets still
surface unchanged — only an implausibly large flood (>50% and >20) is
suppressed.
Tests: seam cases for the new predicate + scan-level regressions (mass
mismatch -> 0 findings; small genuine set -> still reported).
Extends the watchlist export to the full library. The exporter is now general
(core/exports/artist_export.py, renamed from watchlist_export) — adds tidal/qobuz
links and an extra_fields passthrough, so the library export also carries
lastfm/genius URLs + soul_id, and an optional "library counts" toggle adds owned
album/track counts per artist.
- GET /api/library/artists/export?format=&links=&contents= — pulls every artists
row, normalizes onto the canonical *_artist_id keys, optionally GROUP-BY counts
for album/track totals.
- The export modal is now openArtistExportModal(scope): "Export Library" button in
the library header + the existing "Export" on the watchlist bar (a thin wrapper).
Library mode shows the extra "library counts" toggle.
Tests (11): builder across formats + the new tidal/qobuz links + extra_fields
columns; watchlist + library endpoint wiring. 64 integrity green; ruff clean.
