388 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Broque Thomas
|
9cf1fe492b |
Personalized playlists (5/5): WHATS_NEW entry
User-facing summary of the standardization work — all 8 personalized discover-page playlists unified behind one storage layer, manager, and REST surface. Prerequisite for the playlist pipeline integration landing in the next PR. |
||
|
Broque Thomas
|
19a18ba992 |
Dashboard activity feed: stop showing 'NaNmo ago'
Recent activity items on the dashboard all rendered 'NaNmo ago' because the formatter parsed `activity.time` (a human label like 'Now' / 'Just now') with `new Date(...)` -> Invalid Date -> NaN arithmetic -> 'NaNmo ago'. Backend (`core/runtime_state.add_activity_item`) has always emitted `activity.timestamp` (Unix epoch seconds) alongside the label. Frontend now uses the epoch for relative-time formatting via a new local `_activityTimeAgo` helper: - typeof timestamp === 'number' -> diff against Date.now() in ms - < 60s -> 'Just now' - < 60m -> 'Nm ago' - < 24h -> 'Nh ago' - < 30d -> 'Nd ago' - otherwise 'Nmo ago' - falls back to the literal `activity.time` label only when no timestamp is present (legacy items / future shapes) Both call sites in api-monitor.js (initial render + timestamp-only refresh path) updated to the new helper. |
||
|
Broque Thomas
|
d9529fc801 |
Token leak round 2: artist endpoint + playlist sync + URL-encoded redaction
The first token-leak fix scrubbed the artwork URL fixer's own log
calls. This catches three more sites that ALSO leaked tokens, plus
one upstream gap that let URL-encoded tokens slip through the
redactor.
Three sites in `web_server.py` (artist endpoint at line 8765-8773):
- "Artist image before fix: '...'" -- logged the raw image_url with
the auth token in plain form.
- "Artist image after fix: '...'" -- logged the URL-encoded form
after it had been wrapped in the image proxy
(`/api/image-proxy?url=<percent-encoded-token>`).
- "Final artist data being sent: {...}" -- dumped the entire
artist_info dict on every render, including the image_url field.
All three were dev-time debug noise. Removed entirely. The "No
artist image URL found" warning at line 8770 stays (no URL, just
the artist name).
One site in `core/discovery/sync.py:402`:
- "[PLAYLIST IMAGE] image_url=..." -- logged the playlist poster URL
during sync. Same auth-token leak risk for Plex / Jellyfin
playlists. Changed to log only `has_image=True/False`.
Upstream gap in `_redact_url_secrets`:
- The original regex only matched plain query params (`?key=value`).
When an auth-bearing URL gets wrapped inside another URL's query
string (our `/api/image-proxy?url=<encoded>` flow) the auth params
end up percent-encoded -- `%3FX-Plex-Token%3D...` -- and slipped
through.
- New second pattern catches the URL-encoded form. Both passes run
on every redact call; idempotent.
Verified manually:
/api/image-proxy?url=...%3FX-Plex-Token%3DABC...
-> /api/image-proxy?url=...%3FX-Plex-Token%3D***REDACTED***
6 artwork tests pass.
|
||
|
Broque Thomas
|
2fe1926074 |
Stop leaking Plex / Jellyfin / Navidrome tokens into app.log
The artwork URL normalizer was logging the full constructed media- server URL on every cover-art lookup at INFO level, including the auth query params (X-Plex-Token / X-Emby-Token / Subsonic t+s+p). Those lines pile up in app.log on disk -- anyone with read access to the log file gains full read access to the user's media server. Also dropped the noisy per-call "Plex/Jellyfin/Navidrome config - base_url: ..., token: ..." INFO lines that fired on every thumbnail. Even the truncated `token[:10]` form is enough partial-known-plaintext to be uncomfortable to leak. - New `_redact_url_secrets` helper masks the values of X-Plex-Token, X-Emby-Token, api_key, apikey, Subsonic t / s / p, generic token / password query params. Regex anchored on `?` or `&` boundary so short keys like `t` don't false-match inside `format=Jpg`. - "Fixed URL: ..." log calls moved from INFO to DEBUG so they don't persist by default, and the URL passed in is run through the redactor first. - Per-call "Plex config - ..." / "Jellyfin config - ..." / "Navidrome config - ..." INFO lines removed entirely. Config inspection has dedicated UI; per-thumbnail spam belongs to no one. - Error-path logging (line 149) also routed through the redactor in case the failing URL had auth params attached. Users with existing app.log files containing the leaked tokens should rotate / wipe the log. Plex tokens can be regenerated by signing out of all devices in Plex settings; Jellyfin api_keys can be revoked from the dashboard; Navidrome users should rotate the account password. |
||
|
Broque Thomas
|
b42cafa150 |
AcoustID + quarantine modal: three bug fixes (closes #607, closes #608)
Issue #607 (AfonsoG6) -- two AcoustID problems: 1. Live recordings false-quarantining as "Version mismatch: expected '... (Live at Venue)' (live) but file is '...' (original)" because MusicBrainz often stores the recording entity with a bare title -- the venue / live annotation lives on the release entity, not the recording. The audio fingerprint correctly identifies the live recording, but the title-text comparison flagged it as wrong. New pure helper `core/matching/version_mismatch.py:is_acceptable_version_mismatch` accepts the mismatch only when: - One-sided AND involves 'live': exactly one side is 'live' and the other is bare 'original'. Two-sided mismatches stay strict. - Fingerprint score >= 0.85 (stricter than the existing 0.80 minimum -- escape valve only fires when AcoustID is more confident than its own threshold). - Bare title similarity >= 0.70. - Artist similarity >= 0.60. Other version markers (instrumental, remix, acoustic, demo, etc) stay strict -- those have distinct fingerprints AND MB always annotates them in the recording title. The existing test_acoustid_version_mismatch.py suite passes unchanged. 2. Audio-mismatch failure message reported "identified as '' by '' (artist=100%)" when AcoustID returned multiple recordings -- prior code mixed `recordings[0]`'s strings (which can be empty) with `best_rec`'s scores. Now uses `matched_title` / `matched_artist` consistently in both the high-confidence-skip path and the final fail message. Issue #608 (AfonsoG6) -- quarantine modal: 3. Approve / Delete buttons silently no-op'd when the filename contained an apostrophe -- the unescaped quote broke the inline JS in the onclick handler. Now wraps the id via `escapeHtml(JSON.stringify(id))`, which round-trips quotes / backslashes / unicode / newlines safely through the HTML attribute to JS string boundary. 4. Bonus UX: quarantine entry expanded view now shows source uploader (username) and original soulseek filename when the sidecar carries that context -- helps trace which uploader the bad file came from. Backend exposes `source_username` + `source_filename` fields from `sidecar.context.original_search_result`. Degrades to '' on legacy thin sidecars. Tests: - 23 new boundary tests in tests/matching/test_version_mismatch.py pin every shape: equal versions trivial, one-sided live both directions, threshold floors (each just below default -> reject), two-sided strict, non-live one-sided strict (covers exact test_instrumental_returned_for_vocal_request_fails scenario), custom-threshold overrides. - 4 existing test_acoustid_version_mismatch.py tests pass unchanged. - 507 AcoustID / matching / imports tests pass. |
||
|
Broque Thomas
|
b05ba5d498 |
Reorganize: optional embedded-tag mode (closes #592)
Adds an opt-in alternative metadata source for reorganize. The existing API path (query Spotify / iTunes / Deezer / Discogs / Hydrabase for the canonical tracklist) stays the default and is unchanged. The new tag mode reads each file's embedded tags as the source of truth instead -- useful for well-enriched libraries where API drift can produce inconsistent renames, and avoids API calls entirely. - New pure helper `core/library/reorganize_tag_source.py` adapts the output of `read_embedded_tags` (the same mutagen path the audit- trail modal uses) to the `api_album` / `api_track` shapes that `_build_post_process_context` already consumes. Handles ID3-style "5/12" track + disc shapes, multi-value Artists tags, year normalization across 5 date formats, releasetype canonical tokens, multi-artist string splits across 9 separators. - `plan_album_reorganize` accepts `metadata_source: 'api' | 'tags'` (default 'api') and `resolve_file_path_fn`. Tag mode branches into a new `_plan_from_tags` that reads each track's file and produces per-item `api_album` + `api_track` instead of a shared one. - `_run_post_process_for_track` accepts a per-item `api_album` override so each file's own album metadata flows through post- process (not a single shared dict). - `total_discs` in tag mode honors the `totaldiscs` tag and the trailing `/N` of an ID3 `discnumber = "1/2"`. Partial-album reorganize still routes into the correct `Disc N/` subfolder when the tag knows the total even if not all discs are present locally. - Bare `discnumber = "1"` no longer poisons `total_discs` -- it carries no total signal. - `reorganize_album` surfaces a tag-mode-specific error when no files are readable, instead of the API-mode "run enrichment first" message which would mislead in tag mode. - `QueueItem.metadata_source` field, `enqueue` / `enqueue_many` pass-through, runner injects `item.metadata_source` into `reorganize_album`. - `web_server.py` endpoints accept `mode` body param. Falls back to the `library.reorganize_metadata_source` config setting, then to 'api'. Strict allowlist (api / tags) -- anything else falls back. - Frontend: per-album modal + reorganize-all modal both grow a new "Metadata Mode" dropdown above the source picker. Tag mode hides the source picker (irrelevant). Choice persisted in localStorage. Both preview + execute fetches send `mode` in body. Tests: - 49 boundary tests on the pure helper pin every shape: ID3 "5/12", multi-artist split, year normalization, releasetype validation, total_discs precedence, defensive paths. - 6 planner-level integration tests pin the wiring: tag-mode with good tags, partial-disc with totaldiscs tag, file missing, some-match-some-fail, defensive resolve_file_path_fn=None, API-mode regression guard. - All 3171 tests pass; 52 existing reorganize tests unchanged. |
||
|
Broque Thomas
|
2ccada088d |
Dashboard: cursor-following accent blob + darker cards
Two-layer accent glow that follows the cursor across the bento grid: - Soft halo (1280px, blur 48) lerps toward target with a delay; bright inner core (540px, blur 18, screen-blended) lerps faster. - Both layers gently pulse on different rhythms so the blob feels alive even when stationary. - Target = cursor position when hovering any .dash-card; otherwise the grid center (idle resting position). On leaving cards/gap, blob waits 1.5s before drifting back to center -- a small dwell that lets it feel intentional rather than skittish. - Card backgrounds darkened to near-black with stronger borders for contrast against the accent glow. Performance: - requestAnimationFrame loop runs only while the blob is moving and idles when settled at the target. - Two-pass per frame: read all getBoundingClientRect() first, then write CSS vars in a second pass -- one layout flush per frame instead of one per card. - IntersectionObserver snaps to grid center the first time the dashboard becomes visible (handles the case where home page is hidden at attach time). Honors the existing reduce-effects setting: - CSS hides both blob layers via body.reduce-effects. - JS MutationObserver on body class kills the rAF loop when toggled on; re-snaps to center and restarts when toggled off. - prefers-reduced-motion media query disables the pulse animations. |
||
|
Broque Thomas
|
acce083675 |
Dashboard bento grid redesign + responsive breakpoints
Replaces the old stacked dashboard with a bento grid: services, stats, library, syncs, tools, activity, enrichment each live in their own card. - 3-col on desktop (>=1500px), 2-col on laptop, 2-col tighter on tablet, 1-col stack on mobile (<700px). Sub-grids inside each card adapt at every breakpoint (service tiles 3-2-1, stat cards 3-2, gauge tiles 10-5-4-3-2). - Cards use the user's accent color for glow + hover border + CTA icons (was hardcoded per-card hues). - Mount fade-up with per-card stagger; subtle bloom drift; reduced-motion honored. - Enrichment row collapses the per-service gauge tile (hides the 3-stat row, scales the gauge SVG to fill the tile width) so all 10 services fit on one row at desktop. - Recent syncs stacks vertically inside its bento card instead of overflowing horizontally. - Every existing id, button, and JS hook preserved -- no behavior change, pure visual + responsive overhaul. |
||
|
Broque Thomas
|
544cdb49fd | Bump version to 2.5.3 | ||
|
Broque Thomas
|
2f284efa57 |
Retag now re-embeds LYRICS tag instead of leaving it empty
Discord report (netti93). The download flow runs `enhance_file_metadata` (clears all tags) then `generate_lrc_file` (writes .lrc sidecar AND embeds USLT). The retag flow only ran the first half — `enhance_file_metadata` cleared USLT and there was no follow-up to restore it. Two coordinated fixes (no new setting per kettui scope discipline — user described it as "might even be an idea," consistency was the load-bearing ask). Fix 1 — retag calls generate_lrc_file after enhance `core/library/retag.py:execute_retag` now invokes `deps.generate_lrc_file` right after the `enhance_file_metadata` call, mirroring the download pipeline. New `generate_lrc_file` field on `RetagDeps`, defaults to None for backward compat with any test caller that builds RetagDeps without it. Web_server's `_build_retag_deps()` factory wires in the real `core.metadata.lyrics.generate_lrc_file`. Placement matters — runs BEFORE `safe_move_file` so the helper sees the audio file at its current path with its existing sidecar (which retag hasn't moved yet). After the embed, the audio file gets moved with USLT now present; the sidecar move step that follows is unaffected. Fix 2 — create_lrc_file re-embeds from existing sidecar `core/lyrics_client.py:create_lrc_file` used to early-return True when an .lrc / .txt sidecar already existed (skipping the LRClib fetch). For the retag case the sidecar is already there, so the shortcut hit and USLT was never re-written. Now the helper reads the existing sidecar and calls `_embed_lyrics` with its content before returning. Empty / unreadable sidecars short-circuit silently — defensive, no crash. Download flow unaffected because no sidecar exists at fetch time. 7 boundary tests pin: existing .lrc triggers re-embed, existing .txt triggers re-embed, empty sidecar skips embed, unreadable sidecar swallows error, no sidecar falls through to LRClib (download path regression guard), RetagDeps.generate_lrc_file field accepted, field optional for backward compat. Full suite: 3120 passed. |
||
|
Broque Thomas
|
30f017d1f0 |
Stop writing TRCK as "6/0" when album total_tracks is unknown
Discord report (netti93): downloaded album tracks were tagged with
TRCK = "6/0" instead of "6/13" when source data was incomplete. The
retag tool wrote correct "6/13" because core/tag_writer.py already
handled the case.
Trace: core/metadata/enrichment.py:105 formatted unconditionally as
f"{track_number}/{total_tracks}" and many album-dict construction
sites pass total_tracks: 0 (per types.py, 0 means "unknown" — not a
real count). That 0 propagated straight to disk.
Fix at the consumer boundary so every album-dict constructor stays
unchanged. Lifted to pure helper
core/metadata/track_number_format.py:format_track_number_tag that
drops the /N suffix when total is 0 / None / negative — emits just
"6" instead. Matches retag's behavior + ID3 spec convention (TRCK
can be "N" or "N/M"). MP4 trkn tuple gets the same treatment via
format_track_number_tuple returning (6, 0) per spec's "unknown
total" marker.
Wired into all three format-write sites in enrichment.py: ID3 (TRCK),
Vorbis (tracknumber), MP4 (trkn). When source data has correct
total_tracks (album downloads via the metadata-source pipeline,
retag flow), behavior unchanged — still writes "6/13".
16 boundary tests pin every shape: known total / zero total / none
total / none track / zero track / negative inputs / string coercion
/ unparseable strings / floats truncate.
Full suite: 3113 passed.
|
||
|
Broque Thomas
|
9cc09118bf |
AcoustID scanner: multi-candidate match + duration guard + multi-value retag
Closes #587. Three coordinated fixes per codex's diagnosis. AcoustID verification gate left intact — these fixes target the upstream scanner false-positive surface plus a separate retag-path gap. Bug 1 — scanner used recordings[0] as authoritative `core/repair_jobs/acoustid_scanner.py:_scan_file` only checked the top fingerprint match's metadata. AcoustID often returns multiple recordings per fingerprint (sample collisions, multi-MB-record cases) and the wrong-credited recording can outrank the right- credited one. Foxxify case 2 (Nana / Nana): top match credited the wrong artist while a lower-ranked candidate matched the user's expected metadata exactly. Lifted the verifier's all-candidates check to a shared pure helper `core/matching/acoustid_candidates.py:find_matching_recording`. Both verifier and scanner can now ask "given these candidates, does ANY of them match expected (title, artist)?" with the same contract. Scanner suppresses the finding when any candidate matches. Bug 2 — no duration check guards against fingerprint hash collisions Foxxify case 3: 17-minute mashup edit fingerprinted to a 5-minute late-70s Japanese hiphop track (different songs, fingerprint hash collision on a sampled section). Scanner had no signal to detect this and would have recommended retagging the 17-min file as the 5-min track. `duration_mismatches_strongly` in the same helper module flags drifts beyond max(60s, 35%). Scanner now skips findings when the candidate's duration disagrees strongly with the file's expected duration. Loaded duration via the existing tracks SQL (added `t.duration` to the SELECT). Returns False when either side is unknown — no behavior change for older rows without duration data. Bug 3 — scanner retag bypassed multi-value ARTISTS tag setting `core/repair_worker.py:_fix_wrong_song` called `write_tags_to_file` with single-string artist updates. The writer only wrote TPE1 (single string) and never read the user's `metadata_enhancement.tags.write_multi_artist` config. Multi-value ARTISTS tags got stripped on every retag, contradicting the post-download enrichment pipeline's behavior. Per codex's pick (option B over routing through enhance_file_metadata), extended `write_tags_to_file` with an optional `artists_list` parameter. Each format-specific writer respects the config flag the same way enrichment.py does: - ID3: TPE1 stays as joined display string + TXXX:Artists multi-value - Vorbis/Opus/FLAC: `artist` display string + `artists` multi-value key - MP4: \xa9ART as list when on, single string when off Scanner retag derives the per-artist list by splitting AcoustID's credit through the existing `split_artist_credit` helper (same separators the matching layer already uses). Backward compatible: callers that don't pass `artists_list` get the exact same single-string write as before. No regression for the write_artist_image button or any other tag_writer caller. 15 tests on the candidate helper + duration guard. 13 tests on the tag_writer multi-value path (write/skip/single/ no-list cases for FLAC + the config-gate helper). 4 new scanner regression tests pinning lower-ranked candidate suppression, no-suppression when no candidate matches, duration mismatch skip, no-skip when duration matches. Existing scanner tests updated for the new 11-column SQL select (added duration column to fake schema + test row tuples). Full suite: 3097 passed. Ruff clean. |
||
|
Broque Thomas
|
0aa18b0180 |
Cross-script artist aliases: include canonical name + non-strict fallback
Closes #586. Follow-up to #442 — Cyrillic / kanji canonical names weren't bridging cross-script comparisons. Reporter case: "Dmitry Yablonsky" tracks quarantined as audio mismatch with file identified as "Русская филармония, Дмитрий Яблонский" (4% artist sim) even though the Cyrillic spelling is just the Russian transliteration. Codex diagnosed three layered bugs in the alias resolution chain. This fixes all three. Bug 1 — fetch_artist_aliases ignores canonical name + sort-name `core/musicbrainz_service.py:fetch_artist_aliases` only read `data['aliases']`. For artists where MB's canonical `name` IS the cross-script form (and the Latin spelling lives only in aliases — or vice versa), the missing direction never made it into the returned list. Fix: include both `data['name']` and `data['sort-name']` alongside the explicit alias entries (deduped, also pulls each alias entry's sort-name when present). Bug 2 — lookup_artist_aliases ran search in strict mode only Strict mode queries `artist:"..."` only and skips MB's alias and sortname indexes. Cross-script searches found nothing under strict because the user's Latin input never matches a Cyrillic canonical name in the artist index. Fix: lifted the search-and-score logic to a private helper `_search_and_score_artists(name, strict=)` and fall back to non-strict when strict returns empty OR all results fail the trust gate. Non-strict (bare query) hits all indexes. Bug 3 — trust gate weighted local similarity 70% Combined score = local_sim * 0.7 + mb_score/100 * 0.3. Cross-script pairs have local sim ~0 → combined ~0.30 → below the 0.85 threshold → cached as empty even when MB's own confidence was 100. Fix: added an MB-only escape — when MB score is >= 95 AND the result is unambiguous (top result's MB score leads the runner-up by >= 5), accept regardless of local similarity. The existing combined-score path stays intact for same-script matches (#442 Hiroyuki Sawano case still passes via that path). 12 new tests pin every layer: - fetch_artist_aliases canonical-name inclusion + dedup against alias entries + missing-canonical handling + exception path - strict-then-non-strict fallback (empty-strict + low-strict-score) - trust gate MB-only escape + low-confidence rejection + ambiguity rejection (two artists same MB score) + same-script regression - end-to-end reporter scenario with the real `artist_names_match` helper proving the bridge works for "Русская филармония, Дмитрий Яблонский" vs expected "Dmitry Yablonsky" Existing alias tests in `test_artist_alias_service.py` updated to reflect: canonical name now appears in `fetch_artist_aliases` output, lookup makes 2 search calls (strict + non-strict fallback) on first cache miss instead of 1. Full suite: 3065 passed. |
||
|
Broque Thomas
|
e7ecaca3fd |
Fix MTV Unplugged & live-album false-quarantine pipeline
Closes #589. Tracks from MTV Unplugged / Live At / unplugged albums consistently failed AcoustID verification with "Version mismatch: expected (live) but file is (original)". Two upstream bugs fed into the false positive — the AcoustID gate itself was correctly catching the wrong file Tidal had selected. Codex diagnosed all three layers, this fixes the two upstream causes and leaves the verifier alone. Bug 1 — album-scoped library check false-misses owned albums `core/downloads/master.py:184` scored "Shy Away (MTV Unplugged Live)" (source title from playlist) vs "Shy Away" (local DB stored title) with raw string similarity. Massive length asymmetry → ~0.3 → below the 0.7 threshold → marked missing. Combined with the `allow_duplicates and batch_is_album` short-circuit that disables the global fallback for album downloads, the user's already-owned album re-triggered every track for download. Explains the screenshot showing "0 found / 7 missing" on an album the user manually placed. New pure helper `core/matching/album_context_title.py:strip_redundant_album_suffix` strips trailing parenthetical / bracket / dash suffixes whose tokens are fully subsumed by the album context — at least one version marker (live / unplugged / acoustic / session / concert / tour) overlapping with the album, and every other token is either a known marker, a year, a tolerated noise word, or a word from the album title. Album-context-implied "live" added when the album mentions unplugged / concert / tour / session. Wired into the album-confirmed scope ONLY (not global matching). Compares both raw and normalized source titles per album track and takes the max similarity, so the helper returning the input unchanged (when album doesn't imply version context) preserves the pre-fix behavior. Bug 2 — Tidal qualifier filter only ran on fallback searches `core/tidal_download_client.py:345` set `is_fallback = attempt_idx > 0` and only filtered when `is_fallback and required_qualifiers`. Primary search returned all results unfiltered, so a query for "Shy Away (MTV Unplugged Live)" could accept the studio cut if Tidal happened to rank it first. Now the qualifier filter applies to BOTH primary and fallback search attempts — log message updated to indicate which path triggered. Bug 3 — qualifier check ignored album.name The legacy `_track_name_contains_qualifiers` only inspected the track name. For concert / unplugged releases the live signal typically lives in the album title, not the track title. New `_track_matches_qualifiers` accepts a track object and inspects both `track.name` AND `track.album.name`. Legacy helper preserved to keep its existing test contract. AcoustID version-mismatch gate at core/acoustid_verification.py left intact — it correctly catches genuinely-wrong files that slip through upstream filters. The In My Feelings (Instrumental) test that pins this behavior continues to pass. 19 tests on the album-context helper covering MTV Unplugged variants, dash/parens/brackets suffix shapes, year tolerance, plural-form markers, the implied-live set, anti-regression cases (instrumental/remix on a studio album must NOT be stripped), empty/none defensive paths. 13 tests on the Tidal qualifier helper covering legacy track-name-only behavior preserved, qualifier in track name alone, qualifier in album name alone (the MTV Unplugged scenario), multi-qualifier requirements, no-qualifiers always passes, defensive against missing track.album, word-boundary avoiding substring false-matches, _extract_qualifiers picking up live + unplugged from the user's exact reporter query. Full suite: 3053 passed. |
||
|
Broque Thomas
|
c9d4b02a02 |
Fix Deezer contributors tagging silently dropping for cache-polluted tracks
Closes #588. Contributing-artist tagging worked for some tracks but silently dropped them for others — most reproducibly when the album had been fetched before the per-track post-process ran. Trace: get_track_details cache check used `track_position in cached` as the "full payload" sentinel. Both `/track/<id>` AND `/album/<id>/tracks` set track_position. Only `/track/<id>` sets the `contributors` array. When album-tracks data hit the cache first, get_track_details returned the partial record → _build_enhanced_track found no contributors → metadata-source contributors-upgrade silently fell back to single-artist. Reporter's case (Andrea Botez - Sacrifice): the album fetch logged "Retrieved 4 tracks for album 673558211" before the post-process, which cached all 4 tracks as partial records. The contributors- upgrade then hit the partial cache and the upgrade log line never fired because len(upgraded) was never > 1. Lifted cache-validity to a pure helper `_is_full_track_payload` that requires BOTH `track_position` AND `contributors` key presence. Empty list `[]` is valid — single-artist tracks fetched via `/track/<id>` carry it explicitly. Partial cache hits fall through to a fresh `/track/<id>` fetch, which writes the full payload back to cache. 11 boundary tests pin every shape: full payload, single-artist with empty contributors list, partial album-tracks shape, search-result shape, none/non-dict, and the cache-hit/cache-miss/api-failure paths on get_track_details (including the exact reporter-scenario regression). Full suite: 3021 passed. |
||
|
Broque Thomas
|
083355ec8c |
Persist Find & Add selections as permanent server-playlist match overrides
Closes #585. When a Spotify source track had a versioned suffix not present in the local file ("Iron Man - 2012 - Remaster" vs "Iron Man"), the auto-matcher missed the pair. User could click Find & Add to pick the right local file — that worked, file got added to the Plex playlist — but the source track stayed in Missing while the added file appeared in Extra, because the matcher kept no record of the user-confirmed pairing. On the next sync the source track re-tried to download. Fix: every Find & Add selection now writes a (spotify_track_id → server_track_id) override into sync_match_cache at confidence=1.0. The matching algorithm runs an override pass BEFORE the existing exact and fuzzy passes, so any user-confirmed pair short-circuits straight to "matched" without going through title normalization. Covers every mismatch class — dash-suffix remasters, covers / karaoke, alt masters, cross-language titles, typo'd local files. - core/sync/match_overrides.py (new) — pure helpers resolve_match_overrides + record_manual_match. 18 boundary tests pin: cache hits, cache misses falling through to normal matching, stale-cache (server track removed) handled gracefully, str/int id coercion, partial cache hits, defensive against non-dict inputs and DB exceptions. - web_server.py — get_server_playlist_tracks runs the override pre-pass before exact/fuzzy matching. server_playlist_add_track accepts source_track_id + source_title + source_artist and persists the override after every successful add (Plex / Jellyfin / Navidrome). source_track_id added to source_tracks payload so the frontend has it. - webui/static/pages-extra.js — _serverSelectTrack sends source_track_id + source_title + source_artist when adding a track from a mirrored playlist context. - Sync match cache schema unchanged — already had UNIQUE (spotify_track_id, server_source) which fits the override semantics perfectly. Manual overrides distinguished from auto-discovered matches by confidence=1.0. Full suite: 3010 passed. |
||
|
Broque Thomas
|
d0d65946c8 |
Polish quarantine UI — fold into Library History modal as third tab
Standalone Quarantine button + modal felt out of place — duplicated
the chrome of the existing Library History modal but with worse
styling and behavior. Folded the quarantine list into the existing
modal as a third tab next to Downloads + Server Imports.
UI changes:
- Removed the standalone Quarantine button on the Downloads page
header and the standalone modal HTML
- Added third tab to library-history-tabs with a count badge
- loadLibraryHistory dispatches to loadQuarantineList when the
quarantine tab is active
- Quarantine entries render as library-history-entry cards using
the exact same class chrome as Downloads + Imports (thumb
placeholder, title + meta, badge, relative time via
formatHistoryTime, expandable details panel)
- Per-row actions styled as lh-audit-btn to match the existing
Audit button look
- Approve / Recover / Delete now use the themed showConfirmDialog
+ showToast — no more native browser alert / confirm
Backend endpoints + pure helpers + tests unchanged from
|
||
|
Broque Thomas
|
f4cff78f13 |
Quarantine management — list, approve, delete, recover
Closes #584. Quarantined files used to sit in ss_quarantine/ with a thin sidecar — no UI, no recovery, no way to see what got dropped. This adds the management surface the user needs without going to the filesystem. UI: new "Quarantine" button on the downloads page header opens a modal with every quarantined file (filename, expected track/artist, reason, when, size). Three actions per row: - Approve (one-click): restores the file, re-runs the post-process pipeline with ONLY the failing check skipped, lands in the library with full tags + lyrics + scan - Recover (legacy fallback): moves to Staging for thin-sidecar entries that lack the embedded context Approve needs - Delete: permanent removal of file + sidecar Per-check bypass: context['_skip_quarantine_check'] = 'integrity' / 'acoustid' / 'bit_depth'. Skips ONLY the named check — other quality gates stay live. No blanket bypass-all flag. Sidecar expansion: move_to_quarantine now persists the full json-serializable context via serialize_quarantine_context (drops non-JSON-safe values, walks nested dicts/lists/sets, str-coerces unknown objects) plus the trigger name. Existing thin sidecars are detected and routed to Recover instead of Approve. Pure helpers in core/imports/quarantine.py: list_quarantine_entries / delete_quarantine_entry / approve_quarantine_entry / recover_to_staging / serialize_quarantine_context. 27 tests pin every shape: orphan files / orphan sidecars / corrupt sidecars / collision-safe filename restoration / full-context vs thin-sidecar dispatch / json round-trip safety. Four new endpoints in web_server.py — thin glue around the helpers: GET /api/quarantine/list, DELETE /api/quarantine/<id>, POST /api/quarantine/<id>/approve, POST /api/quarantine/<id>/recover. Download modal status differentiates "🛡️ Quarantined" from "❌ Failed" so recoverable files are visible at a glance — checked against the error_message text, no schema change needed. Pipeline changes are three minimal per-check conditionals at the existing quarantine sites in core/imports/pipeline.py. Each move_to_quarantine call now passes its trigger name so the sidecar records which check fired. Full suite: 2992 passed. |
||
|
Broque Thomas
|
177bd85355 |
Configurable duration tolerance for downloaded-file integrity check
Previously hardcoded at 3s (5s for tracks >10min) — files drifting past that got quarantined with no user override. Live recordings, alternate masterings, and some legitimate uploads routinely drift further. New setting `post_processing.duration_tolerance_seconds`. Default 0 means "use auto-scaled defaults" (unchanged behavior for users who don't touch it). Positive value overrides the per-track defaults. Capped at 60s — past that the check is effectively off. Logic lifted to pure helper `resolve_duration_tolerance` in file_integrity.py. Coerces every plausible input (None / empty / zero / negative / unparseable / above-cap / numeric string / float) to either a float override or None for auto. 12 tests pin every shape. Wired into `core/imports/pipeline.py` at the integrity-check call site — runs for ALL matched downloads (Soulseek / Tidal / Qobuz / HiFi / YouTube / Deezer-direct) since they all share that pipeline. Settings UI input under Settings → Metadata → Post-Processing. |
||
|
Broque Thomas
|
0769fcd5cc |
Fix Soulseek downloads losing collab artist tags
Soulseek matched-download contexts populate `original_search_result` with `artist` (singular string) and no `artists` list — the full multi-artist array lives on `track_info` (the matched Spotify track object). `extract_source_metadata` only read `original_search.artists`, so the Soulseek path always fell through to the single-artist branch and TPE1 ended up with the primary artist only. Deezer-direct downloads were unaffected because their context populates `original_search.artists` as a proper list. Lifted artist resolution into a pure helper `core/metadata/artist_resolution.py:resolve_track_artists` that walks `original_search.artists` → `track_info.artists` → `artist_dict.name` fallback chain. Normalizes mixed list-item shapes (Spotify-style dicts, bare strings, anything else stringified) and drops empty entries. 13 new tests pin the resolution order, fallback chain, mixed-shape normalization, whitespace stripping, and empty/none handling. The existing `_artists_list` no-fall-through test in `test_multi_artist_tag_settings.py` was updated to reflect the new contract (always populated; multi-value write still gated on `len > 1`) plus a new regression test for the Soulseek shape. Composes with the existing Deezer per-track upgrade (still fires when single-artist + track_id available) and feat_in_title / artist_separator settings (still drive the joined ARTIST string downstream). |
||
|
Broque Thomas
|
831ddc97d8 |
Polish Download Missing modal tracklist
Pure CSS tune-up scoped to .download-missing-modal-content. Column layout, table semantics, and every JS hook (#match-*, #download-*, .track-*, .download-tracks-tbody-*) untouched. Adds: - Hairline row dividers + cleaner cell padding - Hover gets accent gradient sweep + 3px inset edge bar - Monospace track numbers (glow accent on row hover) + monospace tabular duration - Status cells in both columns get uppercase micro-caps with a leading colored dot + soft glow halo (green/amber/blue/orange/red), pulses while checking + downloading - Artist column centered - Soft scrollbar |
||
|
Broque Thomas
|
e407504e03 |
Fix search source picker defaulting to Spotify regardless of config
Enhanced search + global search popover always opened with the
Spotify icon active even when the user's primary metadata source
was Deezer / iTunes / Discogs / etc.
Trace: shared-helpers.js createSearchController reads
/status.metadata_source to pick the initial active icon, then
gates with SOURCE_LABELS[src]. Backend returns metadata_source
as a dict ({source, connected, response_time, ...}) — used
elsewhere for connection-state display — so SOURCE_LABELS[<dict>]
was always undefined, the guard never fired, and activeSource
silently stayed at the hardcoded 'spotify' default.
Fix reads .source off the dict (with fallback to plain-string for
forward compat). Other consumers already used ?.source — this was
the only stale call site.
|
||
|
BoulderBadgeDad
|
dddf761d0b
|
Merge pull request #388 from kettui/feature/vite-webapp
Lay the groundwork for webui React transition |
||
|
Broque Thomas
|
fc366184b2 |
Raise discography limit from 50 to 200
Discord report: prolific artists (Bach, Beatles complete box, deep dance/electronic catalogues) only showed ~50 entries in the "Download Discography" modal. `MetadataLookupOptions(limit=50, max_pages=0)` was hardcoded at three call sites. Spotify's `max_pages=0` already paginates through everything (per-page is clamped to 10 internally), so Spotify-primary users were unaffected. But Deezer / iTunes / Discogs / Hydrabase all honor the outer `limit` as a hard cap, so non-Spotify users were silently clipped. Bump `limit` to 200 at all three call sites — matches iTunes's and Discogs's own internal caps and covers near-everyone's full catalogue. Spotify behavior unchanged. - web_server.py:9221 — discography endpoint (modal) - web_server.py:8700 — artist-detail discography view - core/artist_source_detail.py:129 — source-specific artist detail |
||
|
Antti Kettunen
|
48aec3f6f3
|
Remove legacy issues shell code
- Delete the static issues page renderer and detail modal helpers - Keep the React issues route as the only implementation - Drop the dead mobile CSS and troubleshooter hook that only targeted the removed shell |
||
|
Antti Kettunen
|
43db30608d
|
Add initial webui page migration analysis | ||
|
Antti Kettunen
|
d98dcd8606
|
Initial Vite app scaffolding & issues page impl
- File-based routing with tanstack router
- Persist top-level navigation state in url, even for most legacy pages
- Striving for an intuitive and simple folder structure where
route-related code is colocated, but the amount of files is still
kept to a minimum
- Replace native fetch with `ky`
- Familiar api, but more polished
|
||
|
Broque Thomas
|
89246a7304 |
Write artist.jpg to artist folder so Navidrome shows real photos
Closes #572 (rhwc). Navidrome has no API for setting an artist image — it reads `artist.jpg` (or `folder.jpg`) from the artist folder during library scans. SoulSync's `update_artist_poster` for Navidrome was a no-op, so users only ever saw album-art-derived thumbnails as the artist photo. - new "Write Artist Image" button on artist detail page - POST /api/artist/<id>/write-image-to-disk derives the artist folder from any track's resolved file_path (reuses _resolve_library_file_path so docker mount translation + library.music_paths probes from #558 apply), fetches the photo from the configured metadata source priority chain, downloads with content-type validation, writes atomically via `<filename>.tmp + os.replace` - when active server is Navidrome, triggers a library scan immediately so the file is picked up - respects existing artist.jpg (frontend prompts before overwriting) so user-supplied photos aren't clobbered - works for plex / jellyfin too as a fallback layer — both servers also read artist.jpg from disk 26 tests pin the pure helpers in core/library/artist_image.py: folder derivation (trailing sep / empty / non-string), URL picking (missing attr / whitespace / non-string), download (non-image content-type / 404 / timeout / empty body), atomic write (replace / temp-cleanup-on-failure / overwrite guard / missing folder). |
||
|
Broque Thomas
|
641c72d7f1 | Bump version to 2.5.2 | ||
|
Broque Thomas
|
6ce185491d |
Add per-download Audit Trail modal to Library History
- new "Audit" button on each download row in the library history modal opens a second modal visualizing the download lifecycle as an interactive horizontal stepper (request → source → match → verify → process → place) with click-to-expand detail cards - hero header with album art + track title + meta line + status pills (source / quality / acoustid result) - three tabs: Lifecycle / Tags / Lyrics - Tags tab reads the audio file live via mutagen at audit-open time via new GET /api/library/history/<id>/file-tags endpoint; file is the single source of truth so background enrichment writes (audiodb / lastfm / genius / replaygain / lyrics fetch) show up too. flat key/value rows stacked vertically (label-above- value) so long MBIDs / URLs / joined genre lists wrap cleanly. source IDs grouped per-service into 2-col sub-card grid. - Lyrics tab renders the full transcript with dimmed timecodes. - post-processing step infers observable changes from source-vs- final state (format conversion, file rename via tag template, folder template). - "Download History" button also added to the Downloads page batch panel header so it's reachable outside the dashboard. - mobile responsive: tabs + stepper scroll horizontally, modal goes full-screen, hero stacks below 480px. 19 helper tests pin the mutagen reader: id3 (TIT2/TPE1/TALB + TXXX + USLT + APIC), vorbis (FLAC dict + _id/_url passthrough), file metadata (format / bitrate / duration), defensive paths (empty / missing file / mutagen returns None / mutagen raises), stringify edge cases (list / tuple / int / frame-with-text / whitespace). |
||
|
Broque Thomas
|
46206b3240 | Pin type='track' / type='artist' collision case for album-type normalizer | ||
|
Broque Thomas
|
5eae24b8bb |
Fix $albumtype defaulting to album for non-Spotify sources
- legacy duck-typed builder only checked the `album_type` key; deezer uses `record_type`, tidal uses `type` (uppercase), some flattened musicbrainz shapes use `primary-type` — all defaulted to album, so EPs and singles ended up filed under Album/ in user templates that reference $albumtype - widen lookup to album_type / record_type / type / primary-type and route through new pure `_normalize_album_type` helper that case-folds + validates against the canonical token set (album / single / ep / compilation), unknown → album - typed-converter path (spotify / deezer / itunes / discogs / mb / hydrabase / qobuz) unchanged — those were already correct Discord report (CAL). |
||
|
Broque Thomas
|
1715e4d52f | Bump version to 2.5.1 | ||
|
Broque Thomas
|
b9feed1a67 |
Add min delay between slskd searches (Bell Canada anti-abuse fix)
- new soulseek.search_min_delay_seconds knob forces a gap between consecutive searches; smooths the burst pattern that trips ISP anti-abuse (Reddit report: Bell Canada cuts the WAN after rapid peer-connection spikes) even when the existing 35/220 sliding-window cap isn't hit - throttle math lifted to a pure compute_search_wait_seconds helper so the gate logic is testable independent of asyncio.sleep + the singleton client - new field on settings → connections → soulseek; default 0 = disabled so existing users see no change 15 helper-boundary tests pin defaults / no-throttle, sliding-window cap (legacy), min-delay (the new burst-smoother), max-of-both gates, and defensive paths. |
||
|
Broque Thomas
|
6233860d66 |
Fix Copy Debug Info music_source + surface missing services
- music_source / spotify_connected / spotify_rate_limited were reading a non-existent 'spotify' key on _status_cache and silently falling through to the missing-value default (always 'unknown' / False). Routed through the canonical accessors get_primary_source + get_spotify_status now. - added hydrabase_connected, youtube_available, hifi_instance_count, and always_available_metadata_sources so the debug dump reflects the full service surface - removed a local re-import of get_spotify_status that was making python 3.12 treat the name as function-scoped, breaking the new lambda above it (NameError on free variable) — module-level import already exists 11 endpoint-level tests pin music_source / spotify_* / hydrabase_* / youtube_available / always_available_metadata_sources / hifi_instance_count and the defensive fall-through paths when each lookup raises. |
||
|
Broque Thomas
|
4892baf8d4 |
Skip already-owned tracks during download discography
- new track_already_owned helper wraps db.check_track_exists at the same confidence threshold the discography backfill repair job uses (0.7) — name+artist+album, format-agnostic so blasphemy-mode libraries (flac → mp3 + delete original) match correctly - endpoint runs the check after the artist + content-type filters and before add_to_wishlist, so a second discography click on the same artist no longer re-queues every track that already downloaded - per-album response carries a new tracks_skipped_owned counter alongside the existing artist/content/wishlist skip categories Discord report (Skowl). |
||
|
Broque Thomas
|
d4ad5bf57f |
Filter cross-artist + content-type tracks during download discography
- drop tracks where the requested artist isn't named in track.artists (keeps features, drops compilation / appears_on contamination) - honor watchlist.global_include_live/remixes/acoustic/instrumentals the same way the discography backfill repair job already does - surface per-album skip counts in the ndjson stream (artist mismatch + content filter) so the ui can show what was filtered Closes #559. |
||
|
Broque Thomas
|
56ae10693b |
Album Completeness: surface diagnostic when resolver can't find album folder
GitHub issue #558: clicking Auto-Fill / Fix Selected on the Album Completeness findings page returned a flat "Could not determine album folder from existing tracks" error with no diagnostic. Reporter is on Navidrome on Docker — the path resolver in `core/library/path_resolver.py` couldn't find any of the album's tracks on disk because Navidrome's Subsonic API doesn't expose filesystem library paths the way Plex's API does (probed in #476). Default settings → `library.music_paths` empty → no base directories to probe → silent None. User had no signal about what to configure. Not a regression of #476 — that fix targeted Plex auto-discovery and worked correctly for it. Navidrome was never covered because the protocol gives the resolver nothing to probe. Fix scoped to the diagnostic surface, not auto-magic discovery: - Added `resolve_library_file_path_with_diagnostic` returning `(resolved, ResolveAttempt)`. ResolveAttempt records what the resolver tried — `raw_path_existed`, `base_dirs_tried`, `had_config_manager`, `had_plex_client`. Pure data, no rendering opinions. - Legacy `resolve_library_file_path` becomes a thin wrapper that drops the attempt; every existing call site is unchanged. - `RepairWorker._fix_incomplete_album` now uses the diagnostic helper and renders a multi-part error via `_build_unresolvable_album_folder_error`: names the active media server, shows one sample DB-recorded path, lists every base directory the resolver actually probed, and points the user at Settings → Library → Music Paths as the actionable fix. - Distinguishes empty-base-dirs vs tried-and-failed cases so the user knows whether to add a mount or fix the existing one. - No auto-probing of common Docker conventions (`/music`, `/media`, etc). Speculative — could resolve to wrong dirs on the suffix-walk if a conventional path happens to contain a partial collision. User stays in control. 12 new tests: - 7 in `tests/library/test_path_resolver.py`: tuple-shape contract, raw-path-existed short-circuit, base-dirs listed even on walk failure, had-flags reflect caller inputs, no-base-dirs returns None with empty attempt, legacy `resolve_library_file_path` delegates correctly across happy / suffix-walk / failure paths. - 8 in `tests/test_repair_worker_unresolvable_folder_error.py`: active server name in error, sample DB path verbatim, base dirs listed, empty-base-dirs phrased differently, Settings hint always present, defensive against None attempt / missing sample / missing config_manager. Full pytest sweep: 2774 passed. |
||
|
Broque Thomas
|
698ecc99f0 |
Import history: Clear History button now sweeps stuck 'processing' rows
Reported: Clear History button on the Import page left zombie rows behind. Every survivor showed "⧗ Processing" status from 2-9 days ago. Trace: `_record_in_progress` inserts a `status='processing'` row up-front so the UI can render the in-flight import while it runs; `_finalize_result` updates it to `completed`/`failed` when the import finishes. When the worker is killed mid-import (server restart, crash), the row never gets finalized — stays at `processing` forever. The clear-history endpoint's SQL `DELETE ... WHERE status IN (...)` listed every terminal status but omitted `processing`, so zombies survived every click. Fix: add `processing` to the delete list, but guard against nuking genuinely-live imports by intersecting against the worker's `_snapshot_active()` map — any folder hash currently registered in `_active_imports` is excluded from the delete via an `AND folder_hash NOT IN (...)` clause. `pending_review` deliberately left out so user still has to approve/reject those explicitly. One endpoint touched (`/api/auto-import/clear-completed` in web_server.py). No worker changes — guard reuses the existing `_snapshot_active()` method that the UI poller already calls. 5 new tests in `tests/imports/test_auto_import_clear_completed_endpoint.py`: - Zombie `processing` rows swept, live `processing` row preserved (folder_hash currently in `_active_imports` survives) - Response count matches actual delete count - Empty active-set branch (unparameterized DELETE) — pinned because an empty SQL `IN ()` would be a syntax error - Worker-unavailable returns 500 (pre-existing guard not regressed) - `pending_review` rows always survive — never auto-swept Full pytest sweep: 2758 passed (one pre-existing flaky timing test on `test_import_singles_parallel.py` failed under full-suite CPU load, passes in isolation in 2.95s — unrelated to this change). |
||
|
Broque Thomas
|
3af2d34cee |
Auto-import: fall through to other metadata sources when primary returns no match
Discord report: 16 Bandcamp indie albums sat in staging because auto-import couldn't identify them, but the manual search bar at the bottom of the Import Music tab found the same albums fine. Trace: `_search_metadata_source` only queried `get_primary_source()` — single source, no fallback. Meanwhile `search_import_albums` (manual search bar) already iterated `get_source_priority(get_primary_source())` and broke on the first source with results. Asymmetric behavior, same album: manual worked, auto-import didn't. Fix: lift `_search_metadata_source` to use the same source-chain pattern. Try primary first; if it returns nothing OR scores below the 0.4 threshold, fall through to the next source in priority order. First source producing a strong-enough match wins. Result dict carries the `source` that actually matched (not the primary name) so downstream `_match_tracks` calls the right client. Defensive per-source try/except so a rate-limited or auth-failed source doesn't abort the chain. Unconfigured sources (client=None) silently skipped. Cin-shape lift: scoring math extracted to pure `_score_album_search_result` helper so the weight tweaks (album 50% / artist 20% / track-count 30%) are pinned at the function boundary, independent of the orchestrator (per-source iteration, exception containment, threshold check). Weight constants exposed at module level (`_ALBUM_NAME_WEIGHT`, `_ARTIST_NAME_WEIGHT`, `_TRACK_COUNT_WEIGHT`) — greppable, bumpable in one place. Pre-extraction these were magic numbers inline. 27 new tests: - 9 integration tests in `test_auto_import_multi_source_fallback.py`: primary-success path unchanged (no fallback fires, only primary client called), primary-empty falls through, primary-weak-score falls through, first fallback success stops the chain (no wasted API calls on remaining sources), all-sources-fail returns None, per-source exception contained, unconfigured-source skipped, result `source` field reflects winning source, `identification_confidence` from winning source. - 18 helper tests in `test_album_search_scoring.py`: weights sum to 1.0, album weight dominant (invariant pin), perfect-match returns 1.0, per-component contribution (album / artist / track-count), Bandcamp vs streaming track-count mismatch (7-files vs 4-tracks case still scores ~0.87 above threshold), zero-track-count and zero-file guards, huge-mismatch non-negative guard, list-of-strings artist shape, missing `.name` / `.artists` / `None` total_tracks edge cases. Backwards compatible: single-source users see no change — chain just has one entry. Existing test `test_search_metadata_source_extracts_artist_id_from_dict_artist` needed one extra patch line for `get_source_priority`. Full pytest sweep: 2754 passed. |
||
|
Broque Thomas
|
d5de724f9b |
Multi-artist Deezer upgrade + double-append guard hardening
Two follow-ups to the multi-artist tag settings PR:
1. Deezer contributors upgrade — closes the "known limitation"
flagged in the prior commit. Deezer's `/search` endpoint only
returns the primary artist for each track; the full contributors
array (feat., remix collaborators, producers credited as artists)
lives on `/track/<id>` and gets parsed by `_build_enhanced_track`.
Without the upgrade Deezer-sourced tracks never got multi-artist
tags even with the right settings on.
Fix in `core/metadata/source.py`: when source==deezer AND the
search response had a single artist AND a track_id is available,
fetch full track details via `get_deezer_client().get_track_details`
and replace `all_artists` with the upgraded list.
- One extra API call per affected Deezer track
- Skipped when search already returned multiple (no-op fast path)
- Skipped for non-Deezer sources (Spotify/Tidal/iTunes search
responses already include all artists)
- Skipped when no track_id is available
- Defensive try/except: on /track/<id> failure (network error,
deezer client unavailable), fall through to the search-result
list — never lose the data we already had
2. Double-append guard hardened with a word-boundary regex.
Prior commit checked for `"feat." not in title.lower() and "(ft."
not in title.lower()` — too narrow. Source platforms produce
wildly different feat-marker conventions: "(feat. X)", "(Feat X)",
"(FEAT X)", "(Featuring X)", "[feat. X]", "ft. X" (no parens),
"FT. X", etc. Any of these as the SOURCE title would cause a
double-append: `"Track (Feat X) (feat. Y)"`.
Replaced with `re.search(r'\b(?:feat|feat\.|featuring|ft|ft\.)\b',
title, IGNORECASE)`. Word-boundary regex catches every common
variant. Substring matches like "Aftermath" containing `ft`
correctly fall through to the append path (pinned by a regression
test).
16 new tests (29 total in the file):
- 9 parametrized variants of the double-append guard
- 1 substring guard ("Aftermath")
- 6 Deezer upgrade scenarios (fires when expected, doesn't fire
for non-Deezer / multi-artist search / no track_id, defensive
fall-through on failure, no false-positive when /track/<id>
confirms single artist)
Full pytest 2727 passed.
|
||
|
Broque Thomas
|
c11a5b7eab |
Multi-artist tag settings: implement artist_separator + feat_in_title + populate _artists_list
Three settings on Settings → Metadata → Tags were partially or
completely unimplemented. Reporter (Netti93) traced each one.
(1) `write_multi_artist` only "worked" because of a never-populated
`_artists_list` field. `core/metadata/source.py` built
`metadata["artist"]` as a hardcoded ", "-joined string but never
assigned `metadata["_artists_list"]`. `core/metadata/enrichment.py`
line 107 reads that field and gates the multi-value tag write
on `len(_artists_list) > 1` — always saw an empty list, silently
no-op'd the write.
(2) `artist_separator` (default ", ") was referenced in the UI +
settings.js save path but ZERO Python code read the value. Every
multi-artist track ended up with hardcoded ", " regardless of
what the user picked.
(3) `feat_in_title` (when true: pull featured artists into the title
as " (feat. X, Y)" and leave only primary in the ARTIST tag —
Picard convention) had no implementation at all.
Fix in source.py:
* Populate `_artists_list` from the search response's artists array
* Read `feat_in_title` and `artist_separator` configs
* When `feat_in_title=True` and >1 artist: ARTIST = primary only,
append "(feat. X, Y)" to title with double-append guard
* Else: ARTIST = artists joined with `artist_separator`
* Single-artist case unaffected by either setting
Double-append guard uses a word-boundary regex catching all common
"feat" variants source platforms produce — `feat`, `feat.`,
`featuring`, `ft`, `ft.` — case-insensitive. Substring matches
(e.g. "Aftermath" containing "ft") correctly fall through to the
append path.
Fix in enrichment.py ID3 branch:
* TPE1 stays as the display string (with separator or primary-only
per the user's settings)
* Multi-value list goes to a separate `TXXX:Artists` frame (Picard
convention) when `write_multi_artist` is on
* Pre-fix the ID3 path wrote TPE1 twice — single-string then list
— and the second `add` overwrote the first, clobbering both the
configured separator AND the feat_in_title semantics. Vorbis path
was already correct (separate "artist" + "artists" keys).
Known limitation (flagged in WHATS_NEW): Deezer's `/search` endpoint
only returns the primary artist. The full contributors array lives
on `/track/<id>`. Enrichment uses search-result data so Deezer-
sourced tracks may still get only the primary artist until a follow-
up commit wires the per-track contributors fetch into the enrichment
flow. Spotify, Tidal, and iTunes search responses include all
artists so they work now.
23 new tests in `tests/metadata/test_multi_artist_tag_settings.py`:
* `_artists_list` populated for multi/single/no-artist cases
* `artist_separator` drives ARTIST string (default ", " + custom
";" + custom "; " + " & ")
* Single-artist case unaffected by either setting
* `feat_in_title=True` pulls featured to title, leaves primary in
ARTIST
* `feat_in_title` no-op for single artist
* Double-append guard recognizes 9 source-title variants ("(feat.
X)", "(Feat. X)", "(FEAT X)", "(feat X)", "(Featuring X)",
"[feat. X]", "ft. X", "(ft X)", "FT. X")
* Substring guard test pins "Aftermath" doesn't false-positive
* Combined-settings precedence: feat_in_title wins ARTIST string
but `_artists_list` carries everyone for multi-value tag
Full pytest 2711 passed.
|
||
|
Broque Thomas
|
fc573a5f19 |
AudioDB worker: stop infinite loop on direct-ID lookup failure (#553)
Track enrichment was stuck in a constant retry loop. Logs showed
nothing but `Read timed out. (read timeout=10)` from
`lookup_track_by_id` repeating against the same track ID. AudioDB
itself was being hammered nonstop with no progress.
Cause: when an entity already has `audiodb_id` populated (from a
manual match or earlier scan) but `audiodb_match_status` is still
NULL — an inconsistent state some import paths can leave behind —
the worker tries a direct ID lookup. If that lookup fails (returns
None on timeout, which AudioDB's `track.php` endpoint hits
frequently because it's slow), the prior code logged "preserving
manual match" and returned WITHOUT marking status. Row stayed NULL
→ queue's NULL-status filter picked it up next tick → tried direct
lookup → timed out → returned → infinite loop.
The "preserve manual match" intent was correct: don't fall through
to the name-search path because that could overwrite a manually-set
`audiodb_id` with a wrong guess. Bug was the missing `_mark_status`
call before the early return.
Fix:
* `_process_item` direct-lookup-failure branch now calls
`_mark_status(item_type, item_id, 'error')` before returning. The
existing `audiodb_id` is preserved (column not touched). Queue's
NULL-status filter no longer re-picks the row.
* `_get_next_item` retry-cutoff queue priorities (4/5/6) extended
from `audiodb_match_status = 'not_found'` to
`audiodb_match_status IN ('not_found', 'error')`. Same `retry_days`
window. Transient AudioDB outages still recover automatically;
permanently-broken IDs eventually get re-attempted once a month
rather than staying errored forever.
5 new tests in `tests/test_audiodb_worker_stuck_track.py` use a real
SQLite DB (not mocks) so the SQL queries are actually exercised:
- lookup-returns-None marks status='error' (no infinite loop)
- lookup-raises-exception marks status='error' (defensive)
- lookup-success preserves the existing match-success path
- error-status row past retry-cutoff gets picked up again
- error-status row within cutoff stays skipped (loop prevention
works)
Only triggers for entities in the inconsistent `audiodb_id` set +
`match_status` NULL state. Happy path and already-matched /
already-not-found rows unchanged. Full pytest 2698 passed.
Closes #553.
|
||
|
Broque Thomas
|
decb62dcc9 |
Docker: pre-bake /app/Staging + writability audit (fix restart loop)
Discord report: container refused to start after pulling latest.
Logs showed `mkdir: cannot create directory '/app/Staging':
Permission denied`. `set -e` in entrypoint.sh then aborted the script
and the container restart-looped.
Root cause traced to commit
|
||
|
Broque Thomas
|
4fb9f38798 |
Your Albums: selectable wishlist modal + Tidal album resolution
Two-part fix to the Your Albums "Download Missing" flow on Discover. Part A — UX redesign The prior `downloadMissingYourAlbums()` ran a per-album loop that fired direct-download tasks via `openDownloadMissingModalForYouTube`. Reported as silently failing — "Queuing 2/2" toast with no actual transfer activity. Even when downloads worked, bypassing the wishlist meant no retry / dedup / rate-limit / source-fallback handling. Replaced with a selectable-grid modal mirroring the Download Discography pattern from the library page. Click the download button → opens a checkbox grid showing every missing album (cover, title, artist, year, track count, source) → user picks what they actually want → click "Add to Wishlist" → each album's tracks get resolved + queued through the existing wishlist auto-download processor. NDJSON progress stream renders ✓/✗ per album. New JS helpers: - `_openYourAlbumsBatchModal(missingAlbums)` — builds the modal - `_renderYourAlbumsBatchCard(row, index)` — per-album card - `_yourAlbumsBatchSelectAll(select)` — bulk toggle - `_updateYourAlbumsBatchFooterCount()` — live count + button text - `_closeYourAlbumsBatchModal()` — overlay teardown - `_startYourAlbumsBatchAddToWishlist()` — submit handler, NDJSON progress consumer - `_yourAlbumsPickSource(album)` — picks the single best source-id per row (priority: spotify → deezer → tidal → discogs) Reuses the `.discog-*` CSS classes from the library Download Discography modal — no new CSS. Reuses the existing `/api/artist/<id>/download-discography` endpoint. The endpoint's URL artist_id param is functionally unused (per-album payload carries everything — verified by reading the endpoint body), so the modal posts with placeholder `your-albums` and gets multi-artist resolution for free without backend changes. Part B — Tidal album resolution Reported as the original bug: clicking download on Tidal-only albums did nothing because `/api/discover/album/<source>/<album_id>` had no `tidal` branch and `tidal_client` had no `get_album_tracks` method. `core/tidal_client.py`: new `get_album_tracks(album_id, limit=None)` method. Two-phase: cursor-walk `/v2/albums/<id>/relationships/items?include=items` for track refs + position metadata (`meta.trackNumber` + `meta.volumeNumber`), batch-hydrate via existing `_get_tracks_batch` for artist/album names. Returns `Track` objects with `track_number` and `disc_number` attached. Sort by (disc, track) so multi-disc compilations render in album order. `web_server.py`: new `'tidal'` source branch in `/api/discover/album/<source>/<album_id>`. Resolves album metadata via `get_album`, tracks via `get_album_tracks`, cover art via inline `?include=coverArt` lookup. Same response shape as Spotify/Deezer branches. `webui/static/discover.js`: - `tidal_album_id` added to `trySources` for the single-album click flow (`openYourAlbumDownload`) - Same source picker drives the new batch modal - Virtual-id generation includes `tidal_album_id` so Tidal-only albums get stable identifiers across discover-album-* / your- albums-* contexts 10 new tests in `tests/test_tidal_album_tracks.py` pin: - Single-page walk + hydration - Multi-page cursor chain - Multi-disc sort order (disc 1 → 2 in track order each) - `limit` short-circuit at page boundary - No-token short-circuit (no API call) - HTTP error returns empty - 429 raises (propagates to `rate_limited` decorator for retry) - Forward-compat type filter (skips non-track entries) - Partial-batch hydration failure containment - Empty-album short-circuit (no batch call) Full pytest: 2693 passed. |
||
|
Broque Thomas
|
7a23d60f28 |
AcoustID scanner: file-tag fallback for legacy compilation tracks
Follow-up to the prior compilation-album scanner fix. That patch
made the scanner read `tracks.track_artist` (per-track artist
column) via COALESCE so compilation tracks would compare against
the right value. But tracks downloaded BEFORE the `track_artist`
column existed have track_artist=NULL — COALESCE falls back to
album artist (the curator) and the wrong-comparison case returns.
Fix: explicit 3-tier resolution in `_scan_file`:
1. DB `tracks.track_artist` if populated → trust it. Respects
manual edits from the enhanced library view (user who curated
the DB value but didn't re-tag the file gets their edit
respected, not overridden by stale file tag).
2. File's ARTIST tag via mutagen if present → use it. Tidal /
Spotify / Deezer all write the per-track artist into the
audio file at download time regardless of SoulSync's DB
schema, so it's ground truth even when the DB column is
stale or NULL. File is already open for fingerprinting so
mutagen tag-read is essentially free.
3. Album artist → final fallback for files without proper ARTIST
tags AND no DB track_artist. Existing pre-fix behavior.
`_load_db_tracks` SELECT now surfaces `track_artist` (raw, may be
empty/NULL via NULLIF) and `album_artist` separately in addition
to the COALESCE'd `artist` field — so `_scan_file` can tell the
difference between 'DB has a curated value' and 'DB fell back to
album artist'. Without this distinction, the file-tag fallback
would create false positives when DB is curated but file is stale.
5 new tests (11 total in the file) pin:
- File-tag-trumps-DB resolves the legacy NULL case (DB says
'Andromedik' (album curator), file says 'Eclypse', AcoustID
says 'Eclypse' → no flag)
- Tag-missing falls back to album artist (preserves existing
genuine-mismatch contract — file without tag + AcoustID
mismatch still flags)
- Mutagen exception swallowed (debug log, fall-through)
- File-tag matches DB → no behavioral change
- DB curated value trumps stale file tag (false-positive guard
— user edited DB without re-tagging file shouldn't get flagged)
Two existing test fixtures (`_make_context` callers) updated to
the new 10-column row shape.
SQL behavior verified empirically against real SQLite: NULL and
empty-string both flow through NULLIF → None in Python →
file-tag-fallback path. Modern populated values trump file tag.
|
||
|
Broque Thomas
|
f4c433c151 |
Tidal: rewire favorite albums + artists to V2 user-collection endpoints
Discord: Discover → Your Albums (and Your Artists) was returning nothing for Tidal users regardless of how many albums/artists they'd favorited. Audit found `get_favorite_albums` and `get_favorite_artists` called the deprecated `/v2/favorites?filter[type]=ALBUMS|ARTISTS` endpoint — that endpoint returns 404 for personal favorites because it's scoped to collections the third-party app created itself. The V1 fallback (`/v1/users/<id>/favorites/...`) is also dead because modern OAuth tokens carry `collection.read` instead of the legacy `r_usr` scope V1 demands (returns 403). Same root cause as the favorited-tracks fix from #502. Fix: rewire to the working V2 user-collection endpoints — `/v2/userCollectionAlbums/me/relationships/items` and `/v2/userCollectionArtists/me/relationships/items` — using the same cursor-paginated pattern shipped for tracks. Architecture: * ID enumeration lifted into a generic `_iter_collection_resource_ids(path, expected_type, max_ids)` helper so tracks / albums / artists all share one walker. Three thin wrappers preserve the per-resource public surface (`_iter_collection_track_ids`, `_iter_collection_album_ids`, `_iter_collection_artist_ids`). Net deduped ~80 lines that would otherwise be three near-identical copies. * Batch hydration via `/v2/{albums|artists}?filter[id]=...&include=...` with extended JSON:API include semantics. One request returns up to 20 albums + their artists + cover artworks all in `included[]` (or 20 artists + their profile artworks). Three static helpers parse the response: - `_build_included_maps(included)` → indexes the array by type so per-resource lookup is O(1) per relationship ref - `_first_artist_name(rels, artists_map)` → resolves primary artist from relationships block; '' on missing/unknown - `_first_artwork_url(rel, artworks_map)` → picks `files[0]` (Tidal returns artwork files largest-first, so this gets the highest-resolution variant — typically 1280×1280) * Public methods (`get_favorite_albums`, `get_favorite_artists`) preserve the prior return shape — list of dicts matching what `database.upsert_liked_album` / `upsert_liked_artist` consume — so the discover aggregator path in `web_server.py` stays byte-identical. No caller changes needed. * Deleted ~240 lines of dead code: the V2-favorites paths AND the V1 fallback paths from the old method bodies. Both are dead against modern OAuth tokens. 24 new tests in `tests/test_tidal_favorite_albums_artists.py` pin: * Cursor-walker dispatch (album/artist iters pass correct path + expected_type to the generic walker) * Included-map building (groups by type, skips items missing id) * Artist + artwork relationship resolution (full + missing rels + unknown id + no files cases) * Batch hydration parse for albums (full attributes, missing relationships fall through to defaults, type-filter excludes non-album entries, `filter[id]` param is comma-joined) * Batch hydration parse for artists (same shape coverage) * End-to-end orchestrator behavior (walk → batch → return, empty-input short-circuits without API call, BATCH_SIZE chunking on 41 IDs → 20/20/1, exception-from-iter returns []) Endpoint paths empirically verified against live Tidal API: `userCollectionArtists/me/relationships/items` returned 200 + 5 real artist refs for the test account. `userCollectionAlbums/...` returned 200 + empty (account has 0 album favorites currently) but the response shape is correct. The deprecated `/v2/favorites?filter[type]=ALBUMS` returned 404. The V1 `/v1/users/<id>/favorites/albums` returned 403 with explicit "Token is missing required scope. Required scopes: r_usr" message. WHATS_NEW entry under existing '2.5.1' block. Full pytest: 2678 passed. |
||
|
Broque Thomas
|
e1d3c59bdc | WHATS_NEW: move append-mode entry to 2.5.1 block | ||
|
Broque Thomas
|
6fe85f2f37 |
Server playlist sync: append mode (preserve user-added tracks)
Discord report (CJFC, 2026-04-26): syncing a Spotify playlist to the
server overwrote anything manually added to the server-side playlist.
The fix adds a per-sync mode picker next to the Sync button on the
playlist details modal — Replace (default, current delete-recreate
behavior) or Append only (preserves existing tracks, only adds new
ones). Useful when the source platform caps playlist size and the
user is manually building beyond it on the server.
Implementation:
* New `append_to_playlist(name, tracks)` method on Plex / Jellyfin /
Navidrome clients. Each uses the server's NATIVE append API:
- Plex: `existing_playlist.addItems(new_tracks)`
- Jellyfin: `POST /Playlists/<id>/Items?Ids=...&UserId=...`
- Navidrome: Subsonic `updatePlaylist?songIdToAdd=...`
Falls back to `create_playlist` when the playlist doesn't exist
yet (first sync). No delete-recreate, no backup playlist created
(preserves playlist creation date + metadata + non-soulsync-managed
tracks).
* Dedup-by-server-native-id (ratingKey for Plex, GUID for Jellyfin,
song-id for Navidrome) — never re-adds a track already on the
playlist. Server-native identity, not fuzzy title+artist match,
so it can't false-collide.
* `sync_service.sync_playlist` accepts `sync_mode='replace'|'append'`
kwarg. Single if/else branch dispatches to `append_to_playlist` or
`update_playlist`. Threaded through `core/discovery/sync.run_sync_task`
and the `/api/sync/start` HTTP handler. Validation on the API rejects
unknown mode strings (defaults to 'replace').
* Frontend: per-playlist `<select id="sync-mode-${id}">` rendered next
to the Sync button in both modal renderers (sync-spotify.js for
Spotify playlists, sync-services.js for Deezer ARL playlists).
`startPlaylistSync` reads the select at click time; missing select
(other callers like discover.js) defaults to 'replace' so backward
compat preserved without per-call-site updates.
* SoulSync standalone has no playlist methods at all and the modal
hides the Sync button entirely on it via `_isSoulsyncStandalone` —
dispatch never reaches that path, no defensive fallback needed.
15 new tests pin per-server append behavior:
- missing playlist → create_playlist delegation
- dedup filtering (existing IDs skipped, only new tracks added)
- empty new-track set short-circuits without API call
- failure paths return False without raising
- contract listing (KNOWN_PER_SERVER_METHODS includes
'append_to_playlist'; Plex / Jellyfin / Navidrome all implement)
Plus tests/discovery/test_discovery_sync.py fake `sync_playlist`
fixture got `sync_mode='replace'` default to match the new signature
(was breaking after the kwarg add; now passing).
WHATS_NEW entry under new '2.6.0' block (hidden by
`_getLatestWhatsNewVersion` until next release bump).
Closes CJFC discord request.
|
||
|
Broque Thomas
|
1d6e213b16 | version bump | ||
|
Broque Thomas
|
f28f9808db |
Tidal: surface Favorite Tracks as virtual playlist (issue #502)
Adds the user's Tidal favorited tracks ("My Collection" in the Tidal
app) as a virtual playlist alongside their real playlists, mirroring
how Spotify's "Liked Songs" is treated.
Reporter (yug1900) located the working endpoint after the prior
`/v2/favorites?filter[type]=TRACKS` attempt returned empty data —
that endpoint is scoped to collections the third-party app created
itself, not personal favorites. Real endpoint:
GET /v2/userCollectionTracks/me/relationships/items
?countryCode=US&locale=en-US&include=items
Cursor-paginated (20 per page, follow `links.next` with
`page[cursor]=...` until exhausted). Response only carries
track-level attributes — artist + album NAMES come back as
relationship-link stubs, not embedded data.
Implementation:
* Two-phase fetch — `_iter_collection_track_ids` walks the cursor
chain to enumerate every track id (cheap, IDs only), then
`get_collection_tracks` batch-hydrates 20 IDs at a time through
the existing `_get_tracks_batch` helper which already knows how
to `include=artists,albums`. No duplication of the JSON:API
artist/album parse, no new dataclass shape.
* Virtual playlist `tidal-favorites` appended to the end of
`/api/tidal/playlists`. ID intentionally has no colon —
sync-services.js renderer interpolates IDs into CSS selectors
via template literals (`#tidal-card-${p.id} .foo`) and a `:`
would parse as a CSS pseudo-class operator.
* `tidal_client.get_playlist("tidal-favorites")` recognizes the
virtual id and dispatches to the collection path internally, so
every per-id consumer gets it for free: detail endpoint, mirror
auto-refresh automation, "build Spotify discovery from Tidal
playlist" flow.
OAuth scope expansion:
* Added `collection.read` to both OAuth flows (the
`core/tidal_client.py::authenticate` standalone path AND the
`web_server.py::auth_tidal` web flow — they were independent
scope strings that both needed updating).
* Added `prompt=consent` to both flows — without it Tidal silently
returns a token carrying only the ORIGINAL scope set even after
re-authentication, because Tidal treats the existing
authorization as still valid.
* New `disconnect()` method + `POST /api/tidal/disconnect`
endpoint + Disconnect button next to Authenticate in Settings →
Connections → Tidal — required for users whose existing token
predates the scope expansion (forces a clean grant).
Reconnect-needed UI hint:
* `_collection_needs_reconnect` flag set on 401/403 from the
collection endpoint, cleared on next successful walk, NOT set
on 5xx (transient server errors must not falsely tell the user
to reconnect).
* Listing endpoint reads the flag and surfaces a placeholder card
titled "Favorite Tracks (reconnect Tidal to enable)" with a
description pointing at Settings, so the user has something
visible to act on instead of a silently missing row.
Diagnostic logging — collection request URL + response status +
first 300 bytes of body now logged at info level so future "why
is my collection empty" reports can be diagnosed from app.log
without needing live reproduction.
22 new tests pin: cursor walk (full chain, max-ids cap mid-page +
at page boundary), auth gates (no token / 401 / 403 all bail
clean), reconnect-flag lifecycle (set on 401/403, cleared on next
successful walk, NOT set on 5xx), forward-compat type filter
(non-track entries skipped), count helper, batch hydration
delegation + chunking at the 20-per-batch cap, partial-batch
failure containment, virtual-id dispatch (real playlist ids still
flow through the normal path).
Closes #502.
|
||
|
Broque Thomas
|
b5b6673216 |
Reorganize: hint at Unknown Artist Fixer for placeholder-metadata rows
Phase B of foxxify discord report. Pre-#524 manual-import bug left
some albums in the library with `artist=Unknown Artist` and `album.title
= <numeric album_id>`. Reorganize couldn't place them (no usable
metadata source ID) and emitted a generic "run enrichment first" hint
that doesn't apply — enrichment can't fix these rows. The right tool
is the existing `Fix Unknown Artists` repair job (reads file tags,
re-resolves metadata, re-tags + moves files).
Discoverability gap, not a logic gap. Reorganize now detects the bad-
metadata shape (Unknown Artist OR album.title that's a 6+ digit
numeric id) and emits a clear "run the Fix Unknown Artists repair
job" hint at both reason-emit sites (planner + executor). No
duplication of fixer logic.
WHATS_NEW entry covers both Phase A (orphan-format sibling handling,
already committed in
|
||
|
Broque Thomas
|
812db1fbbf |
AcoustID scanner: prefer track_artist for compilation albums
Discord report (Skowl): downloaded a compilation album ("High Tea
Music: Vol 1") where every track has a different artist (Eclypse,
Andromedik, T & Sugah, Gourski, etc.) and the AcoustID scanner
flagged every single track as Wrong Song. The file tags had the
correct per-track artist (e.g. "Eclypse" for "City Lights"), but
the scanner compared against the album-level artist ("Andromedik",
the curator). Raw similarity 12% → Wrong Song flag.
# Why the prior multi-value fix didn't help
Foxxify's case (just-merged PR): AcoustID returned multi-value
credit "Okayracer, aldrch & poptropicaslutz!" — primary IS in the
credit. Splitting found it.
Skowl's case: both sides single-value but DIFFERENT artists.
Splitter has nothing to find — Eclypse simply isn't in "Andromedik".
Different bug.
# Cause
Scanner SQL at `core/repair_jobs/acoustid_scanner.py:281` joined
the `artists` table via `tracks.artist_id` which points at the
ALBUM artist (the curator/label-name applied to every row in a
compilation). The `tracks.track_artist` column already holds the
correct per-track artist for compilations — populated by every
server-scan path (Plex `originalTitle`, Jellyfin `ArtistItems`,
Navidrome per-track `artist`) AND the auto-import / direct-download
post-process flow (`record_soulsync_library_entry` writes it when
different from album artist). Scanner just wasn't reading it.
# Fix
```sql
SELECT t.id, t.title,
COALESCE(NULLIF(t.track_artist, ''), ar.name) AS artist,
...
```
Prefers per-track artist when populated, falls back to album artist
for legacy rows / single-artist albums where `track_artist` is NULL.
`NULLIF(t.track_artist, '')` handles the empty-string-instead-of-null
case some legacy rows might have.
# Composes with Foxxify's multi-value fix
For the rare compilation track where AcoustID ALSO returns a
multi-value credit (e.g. compilation track has multiple credited
performers), both paths work together — `track_artist` gives the
correct expected primary, then the helper splits the credit and
finds it.
# Tests added (2)
- `test_load_db_tracks_prefers_track_artist_for_compilation` —
reporter's exact case: track with `track_artist='Eclypse'` AND
`artist_id` pointing at album artist 'Andromedik' resolves to
'Eclypse'. Second track with NULL `track_artist` falls back to
album artist 'Andromedik' (single-artist + legacy compat).
- `test_load_db_tracks_falls_back_when_track_artist_empty_string`
— empty string in `track_artist` (some legacy rows) → NULLIF
returns NULL → COALESCE falls back to album artist.
Both use a real SQLite DB so the COALESCE/NULLIF logic + JOIN
runs against actual schema (SimpleNamespace fakes can't simulate
JOINs).
# Verification
- 6/6 scanner tests pass (2 new + 4 existing)
- 2586 full suite passes (+2 from prior commit)
- Ruff clean
|
||
|
Broque Thomas
|
df304eb016 |
AcoustID scanner: handle multi-value artist credits
Discord report (Foxxify): the AcoustID scanner repair job flagged
multi-artist tracks as Wrong Song because AcoustID returns the
FULL credit ("Okayracer, aldrch & poptropicaslutz!") while the
library DB carries only the primary artist ("Okayracer"). Raw
SequenceMatcher similarity scored ~43% — well below the 60%
threshold — so the scanner created a finding even though the
audio was correct. User couldn't fix without lowering the global
artist threshold to ~30% (which would let real mismatches through).
# Fix
Extended the shared `core/matching/artist_aliases.py::artist_names_match`
helper (originally lifted for #441) with credit-token splitting.
When the actual artist string contains common separators —
- punctuation: `,` `&` `;` `/` `+`
- keywords (whitespace-bounded): `feat.` `ft.` `featuring` `with`
`vs.` `x`
— the helper splits into individual contributors and checks each
against the expected artist. Primary-in-credit cases now resolve
at 100% instead of 43%.
Two pattern groups because punctuation separators don't need
surrounding whitespace, but keyword separators MUST be
whitespace-bounded — otherwise we'd split artists with `x` /
`with` etc. in their names ("JAY-X" → "JAY-" / "" issue).
Composes with the existing alias path: cross-script multi-artist
credits ("Hiroyuki Sawano" expected, "澤野弘之, FeaturedJp"
actual) work via alias-token-against-credit-token compare.
# Wire-in
Scanner at `core/repair_jobs/acoustid_scanner.py:202` replaces
the raw `SequenceMatcher` call with `artist_names_match`. Pass
RAW artist strings (not pre-normalised by `_normalize`) so the
splitter can recognise separators — `_normalize` strips ALL
punctuation, which destroyed the very tokens the splitter needs.
The AcoustID post-download verifier (`core/acoustid_verification.py`)
already routes through `_alias_aware_artist_sim` which calls the
same helper — gets the multi-value benefit automatically without
a separate wire-in.
# New `split_artist_credit` exported helper
Pure-function helper for callers who want token-level access to
the credit list (debugging, UI, future per-token enrichment). Same
splitter logic, exposed as a top-level function.
# Tests added (14)
`tests/matching/test_artist_aliases.py` (+11):
- `TestSplitArtistCredit` — parametrised across 12 credit-string
formats (comma, ampersand, semicolon, slash, plus, feat./ft./
featuring, with, vs., x, single-token, empty), drops empty
tokens, strips per-token whitespace
- `TestMultiValueCreditMatching` — reporter's exact case
(Okayracer in 3-artist credit → 100%), primary in middle/end of
credit, genuine-mismatch still fails, single-token actual falls
through to direct compare, multi-value composes with aliases,
threshold still respected
`tests/test_acoustid_scanner.py` (+3):
- Reporter's case end-to-end through `_scan_file` — fingerprint
99% / title 100% / multi-artist credit → no finding created
- Genuine artist mismatch still creates finding (no false
suppression of real mismatches)
- `JobResultStub` minimal scaffold for the integration tests
# Verification
- 14 new tests pass (49 helper + 5 scanner total in their files)
- 110 matching + scanner tests pass total
- 2584 full suite passes (+25 from baseline 2559)
- Ruff clean
- Reporter's exact case (Okayracer in `Okayracer, aldrch &
poptropicaslutz!`) now scores 100% match → no Wrong Song flag
|
||
|
Broque Thomas
|
80cf16339c |
Deezer cover art: upgrade CDN URL to 1900×1900 (was embedding 1000×1000)
Discord report (Tim): downloaded cover art via Deezer metadata
source came out visibly blurry in Navidrome / on phones — large
displays exposed the limited resolution.
# Cause
Deezer's API returns `cover_xl` URLs at 1000×1000. The underlying
CDN actually serves up to 1900×1900 by rewriting the size segment
in the URL path (same trick the iTunes mzstatic + Spotify scdn
upgrades already use). SoulSync wasn't doing the rewrite — every
Deezer-sourced cover got embedded at 1000×1000 regardless of how
much higher resolution the CDN had available.
# Verified empirically
```
$ for size in 1000 1400 1800 1900 2000; do curl -I "...{size}x{size}-..."; done
1000: 200 OK 106 KB
1400: 200 OK 198 KB
1800: 200 OK 331 KB
1900: 200 OK 371 KB
2000: 403 Forbidden
```
1900 is the safe ceiling. Above that the CDN returns 403. CDN
serves source-native bytes when source < target (smaller-source
albums get same bytes whether we ask for 1000 or 1900), so asking
for 1900 universally is safe.
# Fix
New `_upgrade_deezer_cover_url(url, target_size=1900)` helper in
`core/deezer_client.py`. Pure function, mirrors the
`_upgrade_spotify_image_url` pattern that already lives in
`core/spotify_client.py`. Defensive on every input shape:
- Empty / None → returned as-is
- Non-Deezer URL (no `dzcdn`) → returned as-is
- No size segment in URL → returned as-is
- Already at/above target → returned as-is (idempotent, never
downgrades)
Applied at both cover-download sites:
- `core/metadata/artwork.py::download_cover_art` — auto post-process
flow. Mirrors the existing iTunes mzstatic upgrade right above it.
- `core/tag_writer.py::download_cover_art` — enhanced library view's
"Write Tags to File" feature.
# Scope discipline
- Helper applied at the DOWNLOAD boundary, not the source extraction
point in `deezer_client.py`. Means cached entries in the metadata
cache + DB row `image_url` columns keep the original 1000×1000 URL
Deezer's API returned. Future CDN behavior changes only affect the
download path, not stored data.
- Pre-existing `prefer_caa_art` toggle (Settings → Library →
Post-Processing) untouched — orthogonal workaround for users who
want even higher quality (MusicBrainz Cover Art Archive, often
3000×3000+).
- iTunes / Spotify upgrade paths untouched — they already worked.
# Tests added (16)
`tests/metadata/test_deezer_cover_url_upgrade.py`:
- Standard upgrade: default target 1900 on cover URL, alternate
dzcdn host (`e-cdns-images.dzcdn.net` vs `cdn-images.dzcdn.net`),
artist picture URLs (same path pattern), 500×500 source upgrades
too
- Custom target size: smaller target = no-op (never downgrade),
larger target works
- Idempotent: already at/above target returned unchanged
- Defensive on non-Deezer URLs: parametrised across 5 hosts
(Spotify scdn, iTunes mzstatic, MB CAA, Last.fm, random) — all
returned untouched
- Defensive on malformed Deezer URL (no size segment) → returned
as-is
- Empty / None handling
# Verification
- 16/16 helper tests pass
- 560/560 metadata + imports tests pass (no regression)
- 2559 full suite passes
- Ruff clean
|
||
|
Broque Thomas
|
80e9398e16 | WHATS_NEW: cross-script artist names no longer quarantine files (#442) | ||
|
Broque Thomas
|
c02d51d60d |
Plex: trigger_library_scan + is_library_scanning use auto-detected section — fixes #535
# Bug
Plex servers with the music library named anything other than "Music"
(Música, Musique, Musik, Musica, 音乐, موسيقى, etc.) hit this error
after every import cycle:
soulsync.plex_client - ERROR - Failed to trigger library scan
for 'Music': Invalid library section: Music
soulsync.web_scan_manager - ERROR - Failed to initiate PLEX
library scan via web
Side effect: `wishlist.processing` kept reporting "Missing from
media server after sync" for tracks that DID import correctly, so
they got perpetually re-added to the wishlist.
# Root cause
`_find_music_library` correctly auto-detects the music section by
`section.type == 'artist'` and stores it on `self.music_library` —
works for any locale because the type is language-neutral. Read
methods (`get_artists`, etc.) route through `_get_music_sections`
which returns `[self.music_library]`, so they never had the bug.
But `trigger_library_scan` and `is_library_scanning` ignored
`self.music_library` and called
`self.server.library.section(library_name)` directly with the
hardcoded `"Music"` default. `server.library.section('Music')`
raises `NotFound` on any server whose section isn't literally
named "Music".
# Fix
Both methods now prefer `self.music_library` first, fall back to
literal `library_name` lookup only when auto-detection hasn't
populated the cached reference (test fixtures, edge cases).
`is_library_scanning`'s activity-feed match also corrected to
filter by the resolved section's actual title — the prior code
matched `library_name.lower() in activity_title.lower()` which
defaults to "music" and would never match activities for
non-English sections.
`trigger_library_scan`'s success log line now surfaces the actual
section title (`Música`) instead of the unused `library_name`
default ("Music") — confusing when debugging on non-English servers.
# Tests added (13)
`tests/media_server/test_plex_non_english_section_name.py`:
- `test_uses_auto_detected_section_regardless_of_locale` — parametrised
across 6 locale variants (Música, Musique, Musik, Musica, 音乐, موسيقى).
Each verifies trigger_library_scan calls the auto-detected
section's `update()`, NOT a literal-name fallback. Stub raises
AssertionError on `server.library.section()` so a regression that
re-introduces the fallback fails loudly.
- `test_falls_back_to_literal_lookup_when_no_auto_detection` —
backward compat: music_library=None → literal lookup as before.
- `test_explicit_library_name_arg_used_only_when_no_auto_detection` —
auto-detected wins over explicit kwarg when both available.
- `test_logs_correct_section_label_on_success` — log line surfaces
resolved section title.
- 4 symmetric tests for is_library_scanning covering refreshing-attr
check, activity-feed title match, no-match for unrelated sections,
fallback path.
# Verification
- 13 new tests pass
- 84/84 media_server tests pass (no regression in the existing
Plex / Jellyfin / Navidrome suite)
- 2458 full suite passes (+13 from baseline)
- Ruff clean
|
||
|
Broque Thomas
|
402d851cac |
Deezer search: drop advanced-syntax at endpoint, free-text + rerank wins
Live-API verification revealed advanced-syntax queries hurt more than they help on this endpoint. Switching the import-modal Deezer search back to free-text + local rerank. # What live testing showed Hit Deezer's public API with both query forms for the issue #534 case (`Dirty White Boy` + `Foreigner`): **Free-text (`q=Dirty White Boy Foreigner`):** - Returns 21 results - Real Foreigner Head Games studio cut at #1 - Live versions at #2-10 - Karaoke / cover variants at #11-15 **Advanced (`q=track:"Dirty White Boy" artist:"Foreigner"`):** - Returns 12 results - "(2008 Remaster)" at #1 — canonical Head Games cut MISSING from top 8 entirely - Live + alt-album versions follow Advanced syntax DOES filter karaoke at the API level (none in the 12-result set vs. 5 at positions 11-15 in free-text), but it has its own ranking bias that surfaces remasters / "Best Of" cuts ahead of the canonical recording. Net regression for the user- facing goal. # Fix 1. Endpoint reverts to free-text query with local rerank applied. 2. Local rerank gains "remaster" / "remastered" / "reissue" patterns under VARIANT_TAG_PATTERNS (soft 0.4× penalty — user may want them but they shouldn't outrank the original). 3. Client kwarg support (`track=` / `artist=` / `album=`) preserved for future opt-in callers (e.g. exact-match flows where API- level filtering matters more than ranking). # Verified end-to-end against live Deezer API Re-ran the exact #534 case through the live API + new rerank. Top 15 results post-rerank: 1. Dirty White Boy — Foreigner — Head Games ← REAL CUT AT TOP 2-10. Various Live versions 11-15. Karaoke / cover / tribute variants ← BURIED Real Foreigner Head Games studio cut at #1, exactly the user's ask. # Tests - `test_relevance.py` — variant tag patterns extended; existing tests still pass (50 tests). - `test_search_match_endpoints.py::test_joins_track_and_artist_into_free_text_query` — replaces `test_passes_track_and_artist_as_kwargs`; verifies endpoint sends free-text join, NOT field-scoped kwargs (the prior test asserted the wrong direction now). - Karaoke-burying assertion at the endpoint still pins the user-visible behaviour. - Client kwarg path tests untouched (still pin advanced-syntax construction for future opt-in callers). # Verification - 75 relevance + endpoint + query tests pass - 2445 full suite passes - Ruff clean - Live Deezer API shows real cut at #1 post-rerank |
||
|
Broque Thomas
|
59992d42a8 |
Deezer search: free-text fallback when advanced query returns 0
Defensive followup to the relevance fix. Deezer's advanced search
syntax (`artist:"X"`) is documented as substring match, but in
practice it's brittle on artist name variants ("Foreigner [US]",
"The Foreigner") and on tracks indexed under non-canonical title
spellings. When the advanced query returns nothing, we'd previously
land at "No matches" — a regression vs. pre-fix behaviour where
free-text would have returned a less-relevant but non-empty set.
Fix: when the advanced query returns 0 results AND the caller used
field-scoped kwargs, fall back to a free-text join of the same
kwargs and re-query. Caller-side rerank still tightens whatever the
fallback returns, so the worst-case post-fix behaviour is the
pre-fix behaviour — never strictly worse.
Pulled the cache + parse + store dance into a private helper
(`_search_tracks_with_query`) so the orchestration can call it
twice (advanced → fallback) without code duplication. Single API
call when the advanced query has results — no wasted requests.
Diagnostic logger.debug fires when the fallback triggers so we can
see in production whether it's happening (and to which queries).
# Tests added (4)
- `test_falls_back_to_free_text_when_advanced_empty` — advanced
query returns 0, free-text returns hits; client returns the
free-text hits + both API calls fire.
- `test_no_fallback_when_advanced_query_has_results` — single hit
on advanced query → no second API call.
- `test_no_fallback_when_legacy_free_text_call` — legacy callers
already exhausted the only path; empty result is final.
- `test_no_fallback_when_query_unchanged` — empty kwargs path
doesn't trigger the fallback branch (used_advanced=False).
# Existing tests updated
The 4 prior `TestSearchTracksQueryWiring` + `TestSearchTracksCacheKey`
tests were stubbing `_api_get` to return empty `{'data': []}` and
asserting `assert_called_once`. With the new fallback, those stubs
trigger a second API call and the assertions break — even though
the FIRST call construction is what the tests cared about. Updated
the stubs to return one fake hit so the fallback doesn't fire, and
switched to `call_args_list[0]` for first-call inspection.
# Verification
- 18/18 deezer query tests pass (14 prior + 4 new)
- 2445 full suite passes (+4 from prior commit)
- Ruff clean
|
||
|
Broque Thomas
|
1cc37081a6 |
Fix Deezer search relevance — issue #534
# Background User reported (#534) that the import-modal "Search for Match" dialog returned irrelevant results when Deezer was the metadata source. Searching `Dirty White Boy` + `Foreigner` returned 5+ karaoke / "originally performed by" / "in the style of" / "re-recorded" / tribute-band results ranked above the actual Foreigner studio cut from Head Games. User had to scroll past the junk every time, or fall back to iTunes search which is much slower. # Root cause — two layers 1. **Endpoint joined `track + artist` into free-text query.** `/api/deezer/search_tracks` was passing `q=Dirty White Boy Foreigner` to Deezer's `/search/track` API. Deezer fuzzy-matches that string across title / lyrics / artist / album / contributors and orders by global popularity — anything that appears across many compilations outranks the canonical recording. 2. **No local rerank.** None of the search-modal endpoints applied any post-filtering. Deezer's API order shipped straight to the user. # Fix — same architectural shape Cin would build ## Layer 1: field-scoped query at the client boundary `core/deezer_client.py::search_tracks()` now accepts optional `track`, `artist`, `album` kwargs. When provided, builds Deezer's advanced search syntax: `q=track:"X" artist:"Y" album:"Z"`. Massive relevance improvement because each term matches the right field instead of fuzzy-matching everywhere. Backward compat preserved: legacy free-text `query=` callers still work unchanged. Field-scoped path takes precedence when both are provided. Empty input fast-fails without an API call. Embedded double-quotes stripped (Deezer's syntax has no escape mechanism). ## Layer 2: provider-neutral relevance reranker New `core/metadata/relevance.py` module — pure-function rerank over the canonical `Track` dataclass. Composable scoring: - **Cover/karaoke patterns** (multiplier 0.05, effectively buries): matches "karaoke", "originally performed by", "in the style of", "made famous by", "tribute", "vocal version", "backing track", "cover version", "re-recorded", "cover by", etc. across title, album, AND artist fields. Catches the screenshot's exact junk: artist credits like "Pop Music Workshop" / "The Karaoke Channel" / "Foreigner Tribute Band". - **Variant tags** (multiplier 0.4): live / acoustic / demo / instrumental / remix / radio edit / club mix etc. — softer penalty since the user MAY want them. Skipped entirely when the expected_title contains the same tag (so searching "Track (Live)" still ranks Live versions first). - **Exact artist boost** (multiplier 1.5): primary artist exactly matches expected_artist after normalisation. Single strongest signal for "this is the canonical recording". - **Title + artist similarity** via SequenceMatcher (parentheticals + punctuation stripped before comparison). - **Album-type weighting**: album=1.0 > single/ep=0.85 > compilation=0.7. Compilations are more likely tribute / karaoke repackages. Each component is a standalone function so tests pin them individually without standing up the full pipeline. ## Wired at three search-modal endpoints - `/api/deezer/search_tracks` — uses both layers (field-scoped query + rerank). - `/api/itunes/search_tracks` — uses rerank only (iTunes API has no advanced-syntax search, but karaoke / cover variants still leak through and need the local penalty). - `/api/spotify/search_tracks` — already builds field-scoped `track:X artist:Y` query; rerank added as the consistency safety net so all three sources behave the same from the user's perspective. Other Deezer call sites (matching engine, watchlist scanner, auto-import single-track ID) deliberately not touched in this PR — they have their own elaborate scoring pipelines tuned to their specific contexts and aren't surfacing the user-reported issue. Per Cin: "don't refactor beyond what the task requires." # Tests 71 new tests across 3 files: - `tests/metadata/test_relevance.py` (50 tests) — every scoring component pinned individually + the issue #534 screenshot reproduced as a regression test (real Foreigner cut wins after rerank, karaoke variants drop to bottom). - `tests/metadata/test_deezer_search_query.py` (14 tests) — advanced-syntax query construction, field-scoped wiring at the client boundary, free-text path unchanged, kwargs win when ambiguous, limit clamping, cache key consistency. - `tests/imports/test_search_match_endpoints.py` (7 tests) — end-to-end through Flask test client: Deezer endpoint passes kwargs not joined query; karaoke buried at bottom for all three sources; legacy query param still works without rerank. # Verification - 2441 full suite passes (+71 from baseline 2370) - 0 failures (the prior watchdog flake fix held) - Ruff clean across all changed files - JS parses clean (`node -c webui/static/helper.js`) # Architectural standards followed - **Logic at the right boundary.** Query construction lives in the client (every caller benefits from one change). Rerank lives in a neutral module (`core/metadata/relevance.py`) over the canonical `Track` dataclass — works for any source, not Deezer- specific. - **Explicit > implicit.** Every scoring rule has its own named function. Pattern tables are module-level constants tests can introspect. - **Scope discipline.** Audited every Deezer search call site; fixed the user-reported one + the consistent siblings. Did NOT speculatively normalise every Deezer call across the codebase. - **Backward compat.** Free-text `query=` callers untouched. Kwargs added to existing client method signature with safe defaults. - **Tests pin contract at correct boundary.** Pure-function rerank tests don't mock anything; client-query tests stub at `_api_get`; endpoint tests run through the real Flask app. |
||
|
Broque Thomas
|
abab663eb7 |
Auto-import: album duration = album total + conservative re-import UPDATE path
Two pre-existing parity gaps in `record_soulsync_library_entry` that the prior parity commits left untouched. Both close real holes between auto-import writes and what the soulsync_client deep scan would have produced. # Gap 1: Album duration was the first-imported track's duration `record_soulsync_library_entry` is called once per track. The album INSERT only fires for the FIRST track of a new album (subsequent tracks find the album row already exists). The INSERT was passing `duration_ms` — `track_info["duration_ms"]` — as the album's `duration` column. That's the duration of one track, not the album total. Compare to `SoulSyncAlbum.duration` in soulsync_client which is `sum(t.duration for t in self._tracks)`. Fix: - Worker computes `album_total_duration_ms = sum(...)` across every matched track and threads it onto context as `album.duration_ms`. - side_effects reads that value (or falls back to the per-track duration for legacy non-auto-import callers) and writes it as the album row's `duration`. # Gap 2: Re-imports of the same artist/album were insert-only When the SELECT-by-id or SELECT-by-name found an existing soulsync artist or album row, the function skipped completely — no UPDATE path. Meant: artist genres / thumb / source-id reflected ONLY whatever the FIRST imported album supplied, never refreshing as more albums by that artist landed. Ten more imports later, the artist row still held whatever the first random import wrote. Conservative fix: when an existing row matches, run an UPDATE that fills only the columns whose current value is NULL or empty. Never overwrites populated values — protects manual edits + enrichment-worker writes the same way the scanner UPDATE path preserves enrichment columns. Implementation note: the empty-check happens in Python, NOT SQL. Initial pass tried `COALESCE(NULLIF(col, ''), NULLIF(col, 0), ?)` but SQLite's `NULLIF(text_col, 0)` returns the original text value instead of NULL — different types, no coercion. So the SQL-only conditional was unreliable on text columns. New helper does `SELECT cols FROM table WHERE id`, compares each column in Python, and emits UPDATE clauses only for the ones that need filling. Allowlist defense: f-string column names go through `_SOULSYNC_FILLABLE_COLUMNS` validation before interpolation. Misuse adding new columns without an allowlist update fails closed (logger.debug + skip). # Tests added (4) - `test_album_duration_uses_album_total_not_single_track` — album with single-track context carrying explicit `album.duration_ms = 2_500_000` writes 2_500_000 to the album row, not the per-track 200_000 fallback. - `test_re_import_fills_empty_artist_fields` — first import lands artist with empty thumb + empty genres; second import for same artist with thumb + genres present updates the existing row. - `test_re_import_does_not_clobber_populated_artist_fields` — first import writes rich genres + thumb; second import with worse / different metadata leaves the existing row untouched. - `test_re_import_fills_empty_source_id_when_missing` — first import had no source artist ID; second import does — fills the empty `spotify_artist_id` column on the existing row. # Verification - 10/10 side-effects tests pass (including 4 new + 4 from prior parity commit + 2 history/provenance) - 217 imports tests pass (no regression) - 2369 full suite passes (+4 from prior, +22 PR-total from baseline 2347) - 1 pre-existing flake (`test_watchdog_warns_about_stuck_workers`, passes in isolation, unrelated) - Ruff clean |
||
|
Broque Thomas
|
f628009ab4 |
Auto-import: aggregate GENRE tags onto artists row + harden ISRC/MBID types
Cin pre-review followup. Two small parity gaps the prior commits left
open:
# 1. Genre tags land on the standalone artists row
`soulsync_client._scan_transfer` aggregates the GENRE tag across every
track in an album and surfaces it on `SoulSyncAlbum.genres` (which the
DatabaseUpdateWorker writes to the artists+albums row). Auto-import
was hardcoding `'spotify_artist': {'genres': []}` so the imported
artists row landed with empty genres — felt hollow compared to a
Plex/Jellyfin scan, which both pull genres from their respective APIs.
Fix:
- `_read_file_tags` now reads the GENRE tag (mutagen easy mode handles
MP3/FLAC/M4A consistently; some files carry multiple genres so it's
always returned as a list).
- `_process_matches` aggregates genres from each matched file's tags
into a deduped insertion-order list. Dedup is case-insensitive but
preserves original casing — so "Hip-Hop, Rap, Trap" reads naturally
in the JSON column instead of "hip-hop, rap, trap".
- Worker context's `spotify_artist['genres']` carries the aggregated
list, which `record_soulsync_library_entry` already filters via
`core.genre_filter.filter_genres` and writes to the artists row.
# 2. Defensive str() cast for ISRC + MBID
`_build_album_track_entry` already coerces ISRC + MBID to string today
(via `str(isrc) if isrc else ''`). But if a future metadata-source
client returns int / None for either ID, the worker would propagate
the wrong type and side_effects.py's `.strip()` would AttributeError.
Cheap insurance: explicit `str()` cast in the worker before assignment
to track_info. Future-proofs against client drift.
# Tests added (3, in test_auto_import_context_shape.py):
- `test_context_aggregates_genres_from_track_tags` — multi-file
album with overlapping genre lists produces deduped, insertion-
ordered, original-case-preserved result. Stubs `_read_file_tags`
with monkeypatch so we don't need real audio.
- `test_context_genres_empty_when_no_tags` — files without GENRE
tag → empty list. Standalone library write handles gracefully
(genres column stays empty / NULL).
- `test_context_isrc_mbid_coerced_to_string` — hostile types
(int 12345678, None, int 999) coerced to safe strings before
reaching track_info.
# Verification
- 14/14 context-shape tests pass (11 prior + 3 new)
- 213 imports tests pass (no regression)
- 2365 full suite passes (+3 from prior, +18 PR-total)
- 1 pre-existing flake (`test_watchdog_warns_about_stuck_workers`,
passes in isolation)
- Ruff clean
|
||
|
Broque Thomas
|
ec7da89434 |
Auto-import: surface artist source-id from metadata search response
Cin pre-review followup to the standalone library parity commit. The
prior commit fixed `spotify_artist['id']` from the wrong copy-paste
value (`identification['album_id']`) to read from
`identification['artist_id']`, but the identification dict produced
by `_search_metadata_source` and `_search_single_track` never set
`artist_id` — both extracted artist NAME from the search response
and discarded the source ID sitting right next to it. Net effect of
the prior commit: artists row source-id stayed NULL, just for a more
honest reason than before.
Now properly extracted:
- `_search_metadata_source` reads `best_result.artists[0]['id']`
alongside the artist name and returns it on the identification dict
as `artist_id`.
- `_search_single_track` does the same for single-track identification.
- `_identify_single`'s tag-based-confidence path forwards
`result.get('artist_id')` so the artist source-id propagates even
when high-confidence local tags override the search result's name.
Result: identification dict now carries `artist_id` whenever the
metadata source returned an artist with an ID. The worker context
already plumbs it onto `spotify_artist['id']` and
`spotify_album['artists'][0]['id']`, so the standalone library write
finally populates `<source>_artist_id` on the artists row.
Tests added (3, in `test_auto_import_context_shape.py`):
- `test_context_artist_id_uses_identification_artist_id` — when the
identification dict carries `artist_id`, context propagates it
onto `spotify_artist['id']` AND
`spotify_album['artists'][0]['id']`. Pins that the prior copy-
paste bug (artist['id'] = album_id) doesn't return.
- `test_context_artist_id_is_empty_when_identification_missing_it` —
fallback case (filename-only identification): context gets empty
string, NOT album_id. Honest failure mode.
- `test_search_metadata_source_extracts_artist_id_from_dict_artist`
— black-box test of `_search_metadata_source`: feed it a
spotify-shaped result with `artists[0]['id']` and verify
identification dict carries it forward.
Verification:
- 11/11 context-shape tests pass (8 prior + 3 new)
- 210 imports tests pass (no regression)
- 2362 full suite passes (+3 from prior commit, +15 PR-total)
- 1 pre-existing flake (`test_watchdog_warns_about_stuck_workers`,
passes in isolation)
- Ruff clean
|
||
|
Broque Thomas
|
8493be207e |
Auto-import: SoulSync standalone library writes server-quality rows
# Background
SoulSync standalone is meant to be a full replacement for Plex /
Jellyfin / Navidrome — files imported via auto-import (or any other
import path) should land in the database with the same field richness
a media-server scan would write. They weren't.
# Gaps fixed
The auto-import worker built a context dict for each track and handed
it to `_post_process_matched_download` (the same callback the regular
download flow uses). That dict was missing three things downstream
needed:
1. **No `source` field anywhere.** `record_soulsync_library_entry`
reads `get_import_source(context)` to pick the source-aware ID
columns (`spotify_track_id` / `deezer_id` / `itunes_track_id` /
etc.) on the artists / albums / tracks rows. With no source, the
resolver returned an empty string → `get_library_source_id_columns("")`
returned an empty dict → the `UPDATE tracks SET <source>_id = ?`
blocks were silently skipped. Result: every auto-imported track
landed with NULL on every source-id column. Watchlist scans
(which match by stable source IDs to detect "this track is already
in library") couldn't recognise these rows and would re-download
them on the next pass.
2. **No `_download_username='auto_import'`.** Both
`record_library_history_download` and `record_download_provenance`
default to "Soulseek" when no `username` is in the context. Every
staging-folder import was being labelled as a Soulseek download
in library history + provenance — false signal in the UI.
3. **No per-recording IDs (`isrc`, `musicbrainz_recording_id`) on
track_info.** The Navidrome scanner already writes
`musicbrainz_recording_id` directly to the tracks row when present.
Picard-tagged libraries always carry MBID; metadata sources
(Spotify via MusicBrainz enrichment, Deezer, etc.) carry ISRC.
Auto-import had access to both via the metadata-source response
but didn't propagate them — so the soulsync row went in with
NULL on both columns.
# Changes
**`core/auto_import_worker.py` — `_process_matches`:**
- Top-level `'source': source` (from `identification['source']`)
- `'_download_username': 'auto_import'`
- `track_info['isrc']`, `track_info['musicbrainz_recording_id']` —
pulled from the per-track payload returned by the metadata source
- `track_info['album_id']` — back-reference so source-aware ID
resolution works on sources whose API nests album under
`track.album.id` rather than `track.album_id`
- `spotify_artist['id']` now correctly carries the artist's source ID
(was `identification['album_id']`, a copy-paste bug from the
original implementation that made artist-id resolution fall back
to fuzzy matching)
- `spotify_album['artists'][0]['id']` carries artist source ID for
the same resolution path
**`core/imports/side_effects.py`:**
- `record_library_history_download` source_map: add
`"auto_import": "Auto-Import"` — tags imported tracks correctly
- `record_download_provenance` source_service: add
`"auto_import": "auto_import"` — provenance shows real source
- `record_soulsync_library_entry` track INSERT: now includes
`musicbrainz_recording_id` + `isrc` columns (matches
`insert_or_update_media_track`'s shape for Navidrome /
Plex / Jellyfin scans). Both default to NULL when not present.
# Behavior preserved
- Files still land in the same library template path (no path-build
change)
- Other media-server flows (Plex / Jellyfin / Navidrome users)
unaffected — `record_soulsync_library_entry` still gates on
`get_active_media_server() == "soulsync"`. Auto-import on those
servers continues to drop the file in the library folder + emits
`batch_complete` for the scan-trigger automation, same as before.
- Direct downloads (search → Download button) unaffected — they
already passed `source` + `username` correctly.
# Tests added
`tests/imports/test_auto_import_context_shape.py` (8 tests, new file):
- Worker context carries `source` for every metadata source
(parametrised across spotify / deezer / itunes / discogs)
- `_download_username='auto_import'` set unconditionally
- ISRC + MBID propagate from track payload to track_info when present
- ISRC + MBID default to empty string when absent (downstream
normalises to NULL at write time)
- track_info includes album-id back-reference
`tests/imports/test_import_side_effects.py` (4 new tests + 2 schema
column adds):
- `record_soulsync_library_entry` writes mbid + isrc columns when
present in track_info
- Deezer source maps to deezer_id column (regression case for
source-aware column resolver)
- `record_library_history_download` labels `_download_username=
'auto_import'` as "Auto-Import" not "Soulseek"
- `record_download_provenance` registers source_service as
"auto_import" not "soulseek"
# Verification
- 8/8 new context-shape tests pass
- 6/6 side-effects tests pass (4 new + 2 existing)
- 207 imports tests pass
- 2359 full suite passes (+12 from baseline 2347, no regressions)
- 1 pre-existing flake (`test_watchdog_warns_about_stuck_workers`,
passes in isolation, unrelated to this change)
- Ruff clean
|
||
|
Broque Thomas
|
eb68873ec9 |
WHATS_NEW: keep dev-cycle entries under 2.4.3 (no premature 2.4.4 block)
Per the semver workflow the version string only bumps at release time, so the running dev work on the 2.4.3 line should stay listed under 2.4.3 (not pre-create a 2.4.4 block). Merged the prior '2.4.4' key's six dev entries into the top of '2.4.3', above the existing "May 8, 2026 — 2.4.3 release" date marker, with a "Unreleased — 2.4.3 patch work" date marker so the visual split between unreleased + released entries is preserved. `_getLatestWhatsNewVersion` resolves to the current build version (2.4.3 in `_SOULSYNC_BASE_VERSION`); with the 2.4.4 key gone, the helper modal now surfaces the dev work alongside the released entries when the user opens "What's New", instead of being silently hidden until a future build bump. The release-time bump remains the canonical step that splits "unreleased" entries off into their own version block — done as the last commit on dev before merging dev → main. No code changes — pure WHATS_NEW reorganisation. |
||
|
Broque Thomas
|
8a6ee7a2c7 |
Auto-import: bounded ThreadPoolExecutor + per-candidate UI state isolation
# Concurrency model Pre-refactor concurrency was emergent + unbounded: - The worker's `_run` thread called `_scan_cycle` every 60s, processing candidates synchronously in a for-loop. - The `/api/auto-import/scan-now` endpoint spawned a fresh `threading.Thread(target=_scan_cycle)` per click — extra parallel scan cycles on top of the timer. - Multiple "Scan Now" clicks during in-flight processing → multiple threads racing on `_processing_paths` / `_folder_snapshots` state, no upper bound on concurrent scanners. - `stop()` didn't wait for in-flight processing — could leave file moves / tag writes / DB inserts mid-flight. Refactor to the pattern Cin uses elsewhere (`missing_download_executor`, `sync_executor`, `import_singles_executor` all use `ThreadPoolExecutor(max_workers=3, thread_name_prefix=...)`): - **One scan thread** — both timer + manual triggers go through `trigger_scan()`, gated by a non-blocking `_scan_lock`. Duplicate triggers no-op instead of stacking parallel scanners. - **Bounded executor** — `ThreadPoolExecutor` (default 3 workers, configurable via `auto_import.max_workers`) runs per-candidate work. Each candidate runs to completion in its own pool thread; up to N candidates run in parallel. - `_scan_and_submit()` is fast — just enumeration + executor submit, returns immediately, doesn't block on per-candidate work. - `_process_one_candidate(candidate)` holds the per-candidate logic identical to the old for-loop body, lifted into a method so the pool can run multiple instances concurrently. - `_submitted_hashes` set + lock dedupes candidates across the timer + manual triggers so a candidate already queued / running doesn't get re-submitted. - `stop()` calls `executor.shutdown(wait=True)` — clean shutdown, no orphaned file ops. # Per-candidate UI state isolation The executor refactor opened two concurrency holes that the old sequential model masked. Both fixed in this commit: 1. **Scalar UI fields stomped across pool workers.** Pre-refactor `_current_folder` / `_current_status` / `_current_track_*` were safe under the sequential model — only one candidate processed at a time, so the fields tracked the in-flight one. With three pool workers writing the same fields, the polling UI saw garbage like "Processing AlbumA, track 7/14: SongFromAlbumB". Replaced with `_active_imports: Dict[hash, _ActiveImport]` keyed on folder_hash, gated by `_active_lock`. Each pool worker owns its own entry. Helpers `_register_active` / `_update_active` / `_unregister_active` / `_snapshot_active` are the only API. 2. **Stats counters not thread-safe.** `self._stats[k] += 1` is read-modify-write — under load, parallel pool workers drop increments. New `_stats_lock` + `_bump_stat()` helper wraps every mutation. `get_status()` reads under the same lock and returns a copy. # Endpoint change `/api/auto-import/scan-now` no longer spawns its own scan thread — calls `auto_import_worker.trigger_scan()` (which routes through the shared lock + executor). Multiple clicks while a scan is in flight no-op deterministically. Endpoint still wraps the call in a daemon thread so the HTTP response returns immediately even if the staging walk is slow. # Backward compat The scalar `_current_folder` / `_current_status` / `_current_track_*` fields are preserved as **read-only properties** that resolve to the FIRST active import. The existing `get_status()` payload still includes those fields populated from the first entry — single-import UIs (and the test fixture) keep working unchanged. New `active_imports` array exposes the full multi-candidate state for parallel-aware UIs. # Behavior preserved - Per-candidate identify / match / process logic byte-identical - Live-progress state preserved (per candidate now) - Stability gate / already-processed dedup preserved - `_record_in_progress` / `_finalize_result` UI rows preserved - Tag-based loose-file grouping unchanged # Behavior changes - Multiple albums process IN PARALLEL up to `max_workers` - "Scan Now" while scan in progress no-ops (was: spawned another) - `stop()` waits for in-flight pool work via `shutdown(wait=True)` - Auto-import card now lists each in-flight album (one line per active import) instead of a single shared progress line # UI `webui/static/stats-automations.js`: - Progress widget reads `active_imports` array, renders one line per in-flight album with per-candidate status / track index - Falls back to the legacy summary line when payload doesn't carry `active_imports` (older backend) - Per-row "live processing" lookup now matches by `folder_hash` through the array instead of by `folder_name` against scalars # Tests added (`tests/imports/test_auto_import_executor.py`) - Pool config: default max_workers=3, configurable via constructor + via `auto_import.max_workers` config, floors at 1 - Scan lock: 5 concurrent `trigger_scan()` calls run only 1 scan while lock held; releases properly so subsequent triggers run - Executor dispatch: 5 candidates → 5 process calls via the pool - Bounded parallelism: max_workers=3 caps at 3 concurrent; max_workers=2 caps at 2 - Cross-trigger dedup: candidate submitted in scan A doesn't get re-submitted by scan B while still in-flight - Graceful shutdown: `stop()` blocks until in-flight pool work finishes - Per-candidate state isolation: 2 parallel workers updating their own candidate state don't interfere — each candidate's track_index / track_name / folder_name reads back exactly as written for that hash - `get_status()` returns coherent `active_imports` array with one entry per in-flight candidate; aggregate top-level `current_status` is 'processing' when any entry is processing - Unregister removes only that candidate, others stay visible - Stats counter thread-safety: 1000 parallel bumps land at 1000 (the read-modify-write race regresses without the lock) - `get_status()` stats snapshot is a copy, not a live reference # Verification - 17 new tests pass (executor + state isolation) - 2347 full suite passes (1 pre-existing flaky test — `test_watchdog_warns_about_stuck_workers` — passes in isolation, unrelated) - Ruff clean |
||
|
Broque Thomas
|
3246490800 |
Auto-import: MBID/ISRC fast paths + duration sanity gate
Brings the auto-import matcher to picard / beets / roon parity by reaching for the existing AcoustID-grade infrastructure (typed Album foundation, integrity check thresholds) and layering id-based exact matches on top of the fuzzy scorer. Picard-tagged libraries now land every track with full confidence on the first pass. Three layered phases in `core/imports/album_matching.match_files_to_tracks`: 1. **MBID exact match** — file has `musicbrainz_trackid` tag, source returns the same id → instant pair, full confidence, no fuzzy scoring. Picard's primary identifier; per-recording. 2. **ISRC exact match** — file has `isrc` tag, source returns the same id → same fast-path, slightly lower priority than mbid (isrc can be shared across remasters). Both ids normalised before compare (uppercase + strip dashes/spaces for isrc, lowercase for mbid). 3. **Duration sanity gate** — files in the fuzzy phase whose audio length differs from the candidate track's duration by more than `DURATION_TOLERANCE_MS` (3s, matching the post-download integrity check) are rejected before scoring runs. Defends against the cross-disc / cross-release / wrong-edit problem the integrity check used to catch only AFTER the file had already been moved + tagged + db-inserted. Tag reader (`_read_file_tags`) extended: - Reads `isrc` (uppercased, strip / / spaces normalisation deferred to matcher) - Reads `musicbrainz_trackid` as `mbid` (lowercased) - Reads `audio.info.length` and converts to `duration_ms` to match the metadata-source convention Metadata-source layer (`_build_album_track_entry`) extended: - Propagates `isrc` from top-level OR `external_ids.isrc` (spotify shape — would otherwise be stripped before reaching the matcher) - Propagates `musicbrainz_id` from top-level OR `external_ids.mbid` / `external_ids.musicbrainz` - Without this layer, fast paths would silently never fire in production even though unit tests pass — pinned by `test_album_track_entry_propagates_isrc_and_mbid_from_source` 18 new tests in `tests/imports/test_album_matching_exact_id.py`: - Direct: `find_exact_id_matches` with mbid, isrc, isrc normalisation, mbid > isrc priority, spotify-shape `external_ids.isrc`, no-id empty result, file-used-at-most-once - Direct: `duration_sanity_ok` within / outside tolerance, missing durations defer - End-to-end via `match_files_to_tracks`: mbid match short-circuits fuzzy scoring, id-matched files excluded from fuzzy phase, duration gate rejects wrong-disc collisions in fuzzy phase, normal matches pass through the gate, missing durations fall through, deezer seconds-vs-ms conversion, full picard-tagged 10-track album via mbid only - Production-shape: `_build_album_track_entry` propagates isrc + mbid from spotify-shape (`external_ids.isrc`) AND itunes-shape (top- level `isrc`) Verification: - 35 album-matching tests pass total (17 helper + 18 fast-path) - 23 multi-disc tests still pass after the extension (additive) - Full suite: 2311 passed (+18 new), 1 pre-existing flaky timing test failure (`test_watchdog_warns_about_stuck_workers` — passes in isolation, fails only in full-suite runs, unrelated to this PR) - Ruff clean For users: - Picard / Beets / Mp3Tag-tagged libraries (anyone who's organised their music) get instant perfect-confidence matches every time. - Soulseek-tagged downloads (which usually carry isrc when sourced via metadata-aware soulseekers) get the fast path too. - Naively-named files with no useful tags fall through to the improved fuzzy + duration-gated path — same correctness as before for the common case, much harder for the matcher to confidently pair the wrong file. - One step closer to standalone-DB feature parity with plex / jellyfin / navidrome scanners. Acoustid fingerprint fallback (for files with NO useful tags AND no MBID/ISRC) is the next followup PR. |
||
|
Broque Thomas
|
c03edc3cb4 |
Auto-import: respect disc_number in dedup + match scoring
Caught while live-testing the #524 fix with kendrick lamar mr morale & the big steppers (3 discs). User dropped discs 1+2 loose in staging root + disc 3 in its own folder, every file perfectly tagged with disc_number/track_number/title — only 9 tracks ended up in the library, the rest got integrity-rejected and quarantined. Two related bugs in `AutoImportWorker._match_tracks`: 1. **Quality dedup keyed on track_number alone.** The dedup loop kept `seen_track_nums[track_number] = file` and dropped any later file with the same number, treating it as a quality duplicate. On a multi-disc release where every disc has tracks 1..N, that collapses the album to one disc's worth of files BEFORE the matcher runs. User's 18 loose disc-1+disc-2 files reduced to 9 before any title/disc info was even consulted. 2. **Match scoring ignored disc_number.** The 30% track-number bonus fired whenever `ft[track_number] == track_num` regardless of disc. File with tag (disc=2, track=6, "Auntie Diaries", 281s) got the full bonus matching API track (disc=1, track=6, "Rich Interlude", 103s) — wrong file → wrong destination → integrity check correctly rejected and quarantined the file. Same for tracks 7, 8, 9. Fix: - Dedup keys on `(disc_number, track_number)` tuples — multi-disc files with parallel numbering all survive. - Match scoring's 30% bonus only when BOTH disc AND track agree. Cross-disc same-track-number collisions get a small 5% consolation bonus so title similarity has to carry the match (covers cases where tag disc info is missing or wrong). - API track disc_number read from `disc_number` (Spotify) / `disk_number` (Deezer) / `discNumber` (iTunes) defaulting to 1. 4 new pinning tests in `tests/imports/test_auto_import_multi_disc_matching.py`: - 18-file 2-disc regression case (dedup preserves all) - (disc=2, track=6) file matches API (disc=2, track=6) track, not the disc-1 same-numbered track - Single-disc albums still match normally (no regression) - Quality dedup within a single (disc, track) position still picks higher-quality format (.flac over .mp3) Verification: - 2268 full pytest suite passes (+4 new), 1 skipped, 0 failed - Ruff clean Same branch as the #524 fix because both surfaced from the same import session — easier reviewer context if they ship together. |
||
|
Broque Thomas
|
f58f202d32 |
Fix manual album import losing source — issue #524
radoslav-orlov reported every imported album landing in the soulsync
standalone library as "Unknown Artist" + the raw 10-digit album id
as the title + 0 tracks. Audit traced it to the click handler in the
import page dropping the source-of-the-album_id on its way to the
backend match endpoint.
Root cause:
`importPageSelectAlbum(albumId)` (the onclick on every suggestion /
search-result card) only passed the album_id string. The full search
response carried `source`, `name`, and `artist` per row — the
backend's `get_artist_album_tracks` needs source so it can route the
lookup to the metadata source the id actually came from. Without it,
the source chain tries each source's `get_album(id)` against an id
shaped for a different source — a Deezer numeric id against
Spotify's id format returns 404, against iTunes's collectionId range
returns 404, etc. — and falls through to the failure-fallback dict
in `get_artist_album_tracks`:
{
'success': False,
'album': {'name': album_name or album_id, 'total_tracks': 0,
'release_date': '', ...}, # no artist field at all
'tracks': [],
}
That broken album dict then flowed through `build_album_import_context`
→ post-processing pipeline → `record_soulsync_library_entry`, writing
"Unknown Artist" + album_id-as-title + 0 tracks rows into the
soulsync standalone library tables.
Why hybrid users hit it most: a Spotify-primary user searching for an
album → search returns the Spotify result PLUS Deezer fallbacks
(via `_search_albums_for_source`'s priority chain). Clicking a Deezer
fallback row then sent only the Deezer id to /album/match without
flagging that source — Spotify-first chain failed against the Deezer
id and the broken fallback got written.
Fix:
Frontend (`webui/static/stats-automations.js`):
- New `importPageState._albumLookup: { albumId: { id, name, artist,
source } }` populated by both card renderers (`_renderSuggestionCard`
+ the search-results render block) before they emit the onclick.
- `importPageSelectAlbum` reads source / name / artist from that
cache and includes them in the match POST body, so the backend
routes to the correct provider's `get_album` on the very first try.
- `_escAttr` applied to album_id in the onclick (defensive — ids
shouldn't contain quotes but `_escAttr` was already being used on
every other field interpolated into onclick attributes).
Backend (`web_server.py:import_album_match`):
- Defensive log warning when source is missing from the request body.
Catches any future regression where another caller (curl /
third-party / new UI flow) drops source again — it'll show up as
a visible warning in app.log instead of silently corrupting the
library.
Verification:
- Full pytest suite: 2264 passed, 1 skipped, 0 failed
- Ruff clean
- JS syntax clean
- Manual repro requires a real user flow (search albums on the
import page → click one → import) which isn't covered by the
existing unit tests; reviewer should verify against issue #524's
steps before merge.
|
||
|
Broque Thomas
|
e20994e1c7 |
Manual picks: stream results, don't auto-retry, fix stuck-at-0%
Three follow-on fixes to the manual-search candidates modal once people started actually using it: 1. NDJSON streaming. Manual search waited for every source to return before showing anything. Now streams one event per source as each completes — header line, source_results per source, done terminator. Frontend appends rows incrementally via response.body.getReader(). 2. Manual picks no longer auto-retry on failure. New _user_manual_pick flag set on the task in /download-candidate. Both monitor retry paths (not-in-live-transfers stuck + Errored state) bail on the flag. Surfaces the failure to the user instead of silently picking a different candidate via fresh search. 3. Non-Soulseek manual picks (youtube/tidal/qobuz/hifi/deezer/ soundcloud/lidarr) no longer stuck at "downloading 0%" forever. The live_transfers IF branch now marks manual-pick tasks failed directly when the engine reports Errored, instead of deferring to the monitor (which bails on manual picks). Engine fallback in else branch covers the rare race where the orchestrator's pre-populated transfer lookup is missing the entry. Plus a deadlock fix discovered along the way: the new failure path synchronously called on_download_completed while holding tasks_lock, which itself re-acquires the same Lock — non-reentrant threading.Lock self-deadlocked the polling thread. While wedged, every other endpoint that needed the lock (including /candidates → other failed rows couldn't open modals) hung waiting. Moved completion callbacks onto a daemon thread so the lock releases first. Plus failed/not_found/cancelled rows are now ALWAYS clickable (not just when the auto-search cached candidates) — the modal carries the manual search bar, which is the user's recourse for empty results. Plus manual download worker now runs on a dedicated thread instead of competing with the batch's 3-worker missing_download_executor pool — saturated batches no longer queue manual picks indefinitely. All scoped to manual picks via the _user_manual_pick flag — auto attempt flow byte-identical to before. Engine fallback gated on the flag too so auto attempts in the else branch keep the original do-nothing behavior (safety valve handles the stuck-forever case). Also dropped _handle_failed_download from web_server.py — defined but had no callers (dead code). 17 new unit tests pin the gate behavior: - engine fallback: Errored/Cancelled/Succeeded/InProgress transitions, manual-pick gate, terminal-state skip, soulseek skip, missing download_id skip, engine returning None, orchestrator exception - monitor: manual-pick skips not-in-live-transfers retry + Errored retry - IF-branch end-to-end: Errored marks failed, "Completed, Errored" hits failure branch, auto attempts defer to monitor Manual-search endpoint tests rewritten for NDJSON: 11 cases (validation, single-source dispatch, parallel "all" dispatch, one-event-per-source streaming shape, unconfigured-source skip + reject, header metadata, per-source exception isolation). Full suite 2259 passed, 1 skipped. |
||
|
Broque Thomas
|
996575fab3 |
Add manual search to the failed-track candidates modal
When an auto-download fails or returns "not found" with leftover
candidates, the user can already click the status cell to open a
modal showing those candidates and pick a different one. This adds
a manual search bar to that modal — type any query, hit search,
get a fresh round of results without having to bail out and start
over from the main search page.
Solves the case where the auto-query was bad (featured artist not
in title, parentheticals like "(Remastered 2019)" tripping the
matcher, slight artist-name variants, transliteration) but the
file genuinely exists on the source.
Frontend (downloads.js)
- Added a manual-search section above the existing auto-candidates
table inside the candidates modal.
- Source picker is smart per download mode:
- Single-source mode (soulseek-only / youtube-only / etc) shows
a "Searching X" label, no dropdown.
- Hybrid mode shows a dropdown with "All sources" default + every
configured source. Picking "All" runs parallel searches across
them and tags each result row with its source badge.
- Only configured sources show up; unconfigured are hidden.
- Validation: button disabled until query length >= 2, "Type at
least 2 characters" hint until threshold crosses.
- Loading state on search button while the request is in flight.
- Manual results render in a separate table above the existing
auto-candidates table, using the same row template (file /
quality / size / duration / user / ⬇ button) so the renderer
helper is shared.
- Click ⬇ reuses the existing `downloadCandidate(taskId, candidate,
trackName)` flow — same retry path, same AcoustID verification
when the file lands, no shortcut around the safety net.
- Re-running the search with a different query replaces the
previous manual results.
Backend (web_server.py)
- Extended `GET /api/downloads/task/<id>/candidates` response with:
- `download_mode` (e.g. 'hybrid', 'soulseek')
- `available_sources` (list of configured source IDs + labels)
- `source` field on each candidate (purely additive — frontend
auto-renderer ignores it on legacy code paths, manual-search
renderer uses it for the badge)
- Added `POST /api/downloads/task/<id>/manual-search`:
- Body: `{ query, source: 'all' | <source_id> }`
- Validates query length (>=2 trimmed) → 400
- Validates source against the configured-sources gate → 400
(rejects unconfigured sources even when explicitly named)
- For 'all': parallel `ThreadPoolExecutor` dispatch across every
configured download source, merged results
- For specific source: just that source
- Returns same shape as `/candidates` so the frontend renderer
is reused
- New module-level helpers: `_STREAMING_SOURCE_NAMES`,
`_infer_candidate_source`, `_serialize_candidate`,
`_list_available_download_sources`. The existing `/candidates`
endpoint also goes through `_serialize_candidate` so the source
badge is consistent across both flows.
Behavior preserved
- Existing modal layout / candidates table / ⬇ button are
byte-identical when the user doesn't use manual search.
- `downloadCandidate()` JS function untouched.
- `/candidates` and `/download-candidate` endpoints
backwards-compatible — only NEW fields added, nothing changed
or removed.
Tests
`tests/test_manual_search_endpoint.py` — 10 tests:
- `test_manual_search_validates_query_length`
- `test_manual_search_validates_source` (whitelist gate)
- `test_manual_search_handles_task_not_found` (404)
- `test_manual_search_dispatches_to_configured_source_only`
- `test_manual_search_all_dispatches_parallel`
- `test_manual_search_skips_unconfigured_sources`
- `test_manual_search_rejects_unconfigured_source_explicitly`
- `test_manual_search_returns_same_shape_as_candidates`
- `test_manual_search_single_source_mode_lists_source` (verifies
`available_sources` reflects the active mode)
- `test_manual_search_isolates_per_source_exceptions` (one source
throwing doesn't kill the merged result)
2242/2242 full suite green (was 2232 + 10 new). Ruff clean.
JS parses clean.
|
||
|
Broque Thomas
|
d556ec0fa7 |
Bump version to 2.4.3 + make sidebar version dynamic
- `_SOULSYNC_BASE_VERSION` 2.4.2 → 2.4.3
- helper.js — flip 2.4.3 WHATS_NEW header to "May 8, 2026 — 2.4.3
release"; bump fallback default from 2.4.2 → 2.4.3
- docker-publish.yml — manual-trigger default tag 2.4.2 → 2.4.3
Drive-by — make sidebar version + version-modal subtitle dynamic.
The sidebar version button (`v2.4.1`) and version-modal subtitle
(`Version 2.4.1 — Latest Changes`) were hardcoded text in the HTML.
2.4.2 shipped without these getting bumped — silent drift, easy to
miss at every release.
Added a Flask context_processor that injects `soulsync_version` and
`soulsync_base_version` into every template, then templated the two
hardcoded values:
v{{ soulsync_base_version }}
Version {{ soulsync_base_version }} — Latest Changes
Now bumping `_SOULSYNC_BASE_VERSION` updates the UI everywhere it's
rendered. No more "I forgot to bump the sidebar" at release.
2232/2232 full suite green. Ruff clean. JS parses clean.
|
||
|
Broque Thomas
|
d75ae48981 |
Discover: sharpen track selection (diversity, source-aware popularity, library dedup, SQL genre)
Four selection-quality fixes on the SoulSync-made discover playlists.
None change public method signatures; all are tightenings on what's
already there.
(1) Diversity for Hidden Gems + Discovery Shuffle
Both used to be `RANDOM() LIMIT N` with no diversity. Could return
50 tracks from one artist or 20 from one album if the discovery
pool happened to be skewed. Both now over-fetch 3x and run the
existing `_apply_diversity_filter`:
- Hidden Gems: max 2 per album, 3 per artist
- Discovery Shuffle: max 2 per album, 2 per artist (tighter — shuffle
should feel maximally varied)
(2) Source-aware popularity thresholds
`popularity >= 60` for "Popular Picks" and `popularity < 40` for
"Hidden Gems" was Spotify-shaped (0-100 scale). Deezer writes its
`rank` value into that column (often six-digit integers); iTunes
writes nothing meaningful. For Deezer-primary users:
- Popular Picks pulled essentially everything (rank >= 60 = all)
- Hidden Gems pulled essentially nothing (rank < 40 = none)
New `_get_popularity_thresholds(source)` helper returns per-source
values:
- Spotify: (60, 40) — the existing 0-100 scale
- Deezer: (500_000, 100_000) — ballpark from real rank values
- iTunes / unknown: (None, None) — skip the popularity filter
entirely, fall back to random + diversity
`get_popular_picks` and `get_hidden_gems` now consult the helper.
When threshold is None they skip the popularity SQL filter. Diversity
+ ID gate still apply.
(3) Push genre keyword filter into SQL
`get_genre_playlist` used to fetch `limit=1_000_000` rows into Python
then run a substring keyword filter on `artist_genres`. Bad on big
discovery pools.
Now the keyword OR chain is generated as SQL placeholders:
AND (artist_genres LIKE ? OR artist_genres LIKE ? OR ...)
Each placeholder gets `f'%{keyword.lower()}%'` via `extra_params`.
`fetch_limit` drops back to `limit * 10`. `_genre_matches` Python
helper deleted (only intra-file caller; verified via grep).
Parent-genre expansion via `GENRE_MAPPING` preserved — keywords list
feeds the LIKE chain unchanged.
(4) Filter out tracks already in library
Discovery pool can include tracks the user already owns. Hidden Gems
/ Shuffle / Popular Picks shouldn't surface those.
`_select_discovery_tracks` gained `exclude_owned: bool = True`
parameter. When True, adds a correlated NOT EXISTS subquery against
the `tracks` table covering all 3 source IDs:
AND NOT EXISTS (
SELECT 1 FROM tracks t WHERE
(t.spotify_track_id IS NOT NULL AND t.spotify_track_id = discovery_pool.spotify_track_id)
OR (t.itunes_track_id IS NOT NULL AND t.itunes_track_id = discovery_pool.itunes_track_id)
OR (t.deezer_id IS NOT NULL AND t.deezer_id = discovery_pool.deezer_track_id)
)
Note column-name asymmetry: tracks.deezer_id vs
discovery_pool.deezer_track_id. Inline comment marks the trap. All
5 public discovery methods automatically benefit (default True).
Seasonal Playlist doesn't go through the helper so it's unaffected
(curated content, dedup is wrong intent there).
Tests
12 new tests in `tests/test_personalized_playlists_id_gate.py` (27
total in the file):
- Hidden Gems + Discovery Shuffle apply diversity (cap proven by
inserting 10 same-artist + same-album rows and asserting return
count ≤ per-album cap)
- Popularity thresholds: Spotify (60, 40), Deezer larger scale,
iTunes None / None
- Popular Picks skips threshold filter when None
- Genre playlist pushes filter to SQL (parent + child genre expansion)
- Owned-track exclusion: filtered when match, kept when no match,
opt-out flag works
- Deezer column-name asymmetry pinned (regression footgun)
Test fixture re-added the minimal `tracks` table (4 columns: id,
spotify_track_id, itunes_track_id, deezer_id) — only what the new
NOT EXISTS subquery needs to join. Plus `insert_library_track`
helper.
Verification
- 27/27 in this test file pass (15 prior + 12 new)
- 2232/2232 full suite green
- ruff clean
LOC delta:
- core/personalized_playlists.py: 1030 → 1101 (+71)
- tests/test_personalized_playlists_id_gate.py: 352 → 616 (+264)
|
||
|
Broque Thomas
|
959562f6b0 |
Delete Recently Added / Top Tracks / Forgotten Favorites / Familiar Favorites
Owner decision: not worth shipping. The four library-driven personalized sections were stubbed returning [] for ages because their schema prereqs didn't exist; the prior commit re-enabled them by routing through a new `_select_library_tracks` helper. Owner reviewed and chose to delete the sections entirely instead. Removed everywhere: - `core/personalized_playlists.py` — `get_recently_added`, `get_top_tracks`, `get_forgotten_favorites`, `get_familiar_favorites` + the `_select_library_tracks` helper (no other callers; verified via grep). - `web_server.py` — 4 route handlers (`/api/discover/personalized/recently-added`, `top-tracks`, `forgotten-favorites`, `familiar-favorites`). - `webui/index.html` — 4 `<div class="discover-section">` blocks (`#personalized-recently-added`, `#personalized-top-tracks`, `#personalized-forgotten-favorites`, `#personalized-familiar-favorites`). - `webui/static/discover.js` — 4 load functions (`loadPersonalizedRecentlyAdded`, `loadPersonalizedTopTracks`, `loadPersonalizedForgottenFavorites`, `loadFamiliarFavorites`), plus their entries in `loadDiscoverPage`'s Promise.all, plus 4 module-level state vars + 6 dead branches across `openDownloadModalForDiscoverPlaylist` / `startDiscoverPlaylistSync` and the sync-progress / rehydrate dispatchers. - `webui/static/helper.js` — 4 tooltip / docs entries. - `webui/static/sync-spotify.js` — 1 stale rehydrate dispatcher branch (`discover_familiar_favorites`) caught during the global grep pass. - `tests/test_personalized_playlists_id_gate.py` — 3 library-method tests + the test infrastructure that supported them (`tracks` schema, `insert_library_track` helper). Documentation header updated to reflect the deletion. Net: -527 / +2 lines across 7 files. What stays: - Daily Mixes (also in personalized package, intentionally paused — separate decision). - Popular Picks + Hidden Gems + Discovery Shuffle (alive, not affected by this deletion). - All 14 tests in the personalized-playlists test file still pass. - The PersonalizedPlaylistsService lift from the prior commit (`_select_discovery_tracks` etc) — those are still in active use by the surviving discovery_pool methods. DISCOVER_TRACK_SELECTION_REVIEW.md at repo root contains historical references to the four deleted endpoints. Treated as historical context (same policy as WHATS_NEW), left alone. 2219/2219 full suite green (was 2222 - 3 deleted tests = 2219). JS parses clean, ruff clean. |
||
|
Broque Thomas
|
44dd7f980f |
Discover: unify Decade + Genre tabbed browsers
Both tabbed-browser sections — Time Machine ("Decade") and Browse by
Genre — re-implemented the same lifecycle by hand: fetch tabs list,
render the tab strip, attach click handlers, fetch content per tab,
render track list with sync + download action buttons + sync-status
block, handle empty/error/loading states. ~314 lines of identical
boilerplate split across two browsers.
Lifted into one shared `createTabbedBrowserSection(config)` helper.
Each browser is now a thin wrapper:
```js
const ctrl = createTabbedBrowserSection({
id: 'decade-browser',
tabsContainerEl: '#decade-tabs',
contentContainerEl: '#decade-content',
fetchTabs: async () => { ... },
renderTabButton: (tab, isActive) => `<button>...</button>`,
fetchTabContent: async (tab) => { ... },
renderTabContent: (tracks, tab) => `...`,
onTabContentRendered: (tab, contentEl) => { ... },
emptyMessage / errorMessage,
});
```
Migrated:
- `loadDecadeBrowserTabs` 85 → 3 lines
- `loadDecadeTracks` 67 → 3 lines
- `loadGenreBrowserTabs` 92 → 3 lines
- `loadGenreTracks` 70 → 3 lines
Helper: ~125 lines + ~100 lines of per-browser config blocks +
~25 lines of shared `_renderTabbedTrackList` (the two browsers had
byte-identical track-row markup so it lifted cleanly).
Public function names preserved — the four migrated functions stay
on the same signature so existing callers (`loadDiscoverPage`,
refresh buttons, inline handlers) don't change.
Side effects preserved — `decadeTracksCache[year]`, `activeDecade`,
`genreTracksCache[name]`, `activeGenre`, `availableGenres` still
mutated at the same lifecycle moments. The decade-specific
`startDecadeSync(decade)` and genre-specific `startGenreSync(name)`
sync-button handlers stay where they are; they're click handlers
attached to rendered content, not part of the tab lifecycle.
What didn't fit (intentionally left alone):
- `_renderCompactTrackRow` (the existing shared track-row helper) is
NOT used by the tabbed browsers — they had their own template
with a `track_data_json` fallback chain `_renderCompactTrackRow`
doesn't do. Unifying these two would change behavior for
non-tabbed sections, so the tabbed-browser variant lives as
`_renderTabbedTrackList`. Future cleanup could merge them by
giving `_renderCompactTrackRow` an opt-in fallback flag.
- `switchDecadeTab` / `switchGenreTab` still know about cache shape
so they can skip refetch on already-loaded tabs. Keeping that
in the per-browser switch is fine — it's a click handler, not
lifecycle.
Net: 8546 → 8578 LOC on `discover.js` (+32). Helper boilerplate
offsets the line count, but the win is single-source-of-truth, not
raw line reduction.
`node --check` clean. 2222/2222 full suite green.
|
||
|
Broque Thomas
|
c557d9196e |
Discover controller — Cin pre-review polish
Three changes tightening the controller before opening the PR. DROP MAGIC `extractItems` DEFAULTS Controller used to auto-pull `data.items` / `data.albums` / `data.artists` / `data.tracks` / `data.results` when no extractor was supplied. Removed the fallback chain — every section now MUST provide an explicit `extractItems(data) => array`. Validated at register-time so misuse fails immediately, not silently on first load against an endpoint that happened to return two arrays. Cin standard: explicit > implicit. Magic key-grabbing could pick the wrong one in edge cases (e.g. an endpoint returning both `data.albums` and `data.results` would have grabbed albums when the section actually wanted results). All 10 existing controller call sites already passed explicit extractors, so no migration churn — this is purely tightening the contract for future sections. REPLACE `renderItems` NULL-RETURN CONVENTION WITH `manualDom: true` Your Albums and similar sections that delegate to existing renderers that target a CHILD element of `contentEl` used to signal "leave the container alone" by returning null/undefined from `renderItems`. That convention is easy to confuse with an accidental missing-return error. Replaced with an explicit `manualDom: true` config flag. Renderer is still called for its side-effects, controller just skips the innerHTML swap. Clearer intent at the call site. Updated `loadYourAlbums` to use the new flag. PIN THE CONTROLLER CONTRACT WITH JS TESTS Added `tests/static/test_discover_section_controller.mjs` — 32 tests covering the controller's lifecycle contract: - Config validation (every required field, mutual exclusivity of fetchUrl/data, type checks on contentEl) - Happy-path fetch → parse → render - Empty state (default empty render, hideWhenEmpty + sectionEl, success=false treated as empty, custom isSuccess override) - Stale state (fires when isStale returns true, wins over empty, custom renderStale override) - Error state (HTTP non-ok, fetch throws, showErrorToast fires window.showToast, default off doesn't fire) - No-fetch `data:` mode (value + function form, doesn't call fetch) - manualDom mode (skips innerHTML swap, still calls renderer) - Callable `fetchUrl` (resolved at load time, refresh re-resolves) - Load coalescing (concurrent loads share one fetch) - Refresh bypasses coalescing (re-fires fetch every call) - Hook error containment (throwing renderer/onSuccess hooks don't crash the controller) Runs via Node's stable built-in `--test` runner — no package.json, no jest/vitest dependency, no compile step. Just `node --test`. Pytest wrapper at `tests/test_discover_section_controller_js.py` shells out to node and asserts clean exit, so the JS tests fail the regular pytest sweep if the controller contract drifts. Skipped gracefully when node isn't available or is < 22. Closes the "controller is a contract, pin it at the test boundary" gap that Cin would have flagged on review. VERIFICATION - 2205/2205 full pytest suite green (was 2204 + 1 new wrapper) - 32/32 `node --test` pass on the controller test file directly - ruff clean - node --check clean on all touched JS files |
||
|
Broque Thomas
|
dc2323cde6 |
Discover cleanup: controller extensions, toast errors, migrate skipped sections
Follow-up to the controller migration commits. Closes out the extension list the per-section migrations surfaced as needed. CONTROLLER EXTENSIONS - Callable `fetchUrl: () => string` — resolves the seasonal-playlist recreate-on-key-change hack from the prior commit. - No-fetch `data:` mode — value or `() => value`. Lets render-only sections like Seasonal Albums use the controller without inventing a fake endpoint. Mutually exclusive with `fetchUrl`; validated up front so misuse fails at register-time. - `beforeLoad(ctx)` hook — runs before the spinner shows. Lets dynamically-inserted sections like Because You Listen To ensure their `contentEl` exists before the visibility check. - `onSuccess(data, ctx)` hook — runs after the success gate but before isEmpty / isStale. Cleaner home for sibling header / subtitle / button updates than folding them into renderItems. - `isStale(items, data)` + `onStale(ctx)` + `renderStale(items, data)` + `staleMessage` — third render state for "data is empty BUT upstream is still discovering". Stale wins over empty when both apply. Default stale UI is the same spinner block used elsewhere. - `showErrorToast: true` config — opens a global `showToast(...)` in addition to the in-section error block. Default off; sections that have no recovery action shouldn't shout at the user. - `renderItems` returning null/undefined now leaves contentEl untouched. Lets a renderer do its own DOM manipulation (e.g. delegating to an existing grid-render fn that targets a child element) without fighting the controller's innerHTML swap. MIGRATED THE 2 SKIPPED SECTIONS - `loadYourAlbums` — uses `isStale`/`onStale`/`renderStale` for the stale-fetch state, `onSuccess` for the subtitle/filters/download side-effects, `hideWhenEmpty` + `sectionEl` for the truly-empty case, `renderItems` returning null since it delegates to the existing `_renderYourAlbumsGrid` + `_renderYourAlbumsPagination`. - `loadSeasonalAlbums` — uses no-fetch `data:` mode because the parent `loadSeasonalContent` already fetched the season payload. `beforeLoad` updates the sibling title/subtitle text. ERROR TOASTS ON ALL MIGRATED SECTIONS Every migrated section now has `showErrorToast: true`. Section load failures surface a global toast instead of silently spinning forever or swallowing into console.debug. Same pattern JohnBaumb #369 asked for at the Python layer, applied at the UI layer. SHARED SYNC-STATUS BLOCK Lifted the duplicated decade-tab + genre-tab sync-status HTML (✓ completed | ⏳ pending | ✗ failed | percentage) into a single `_renderSyncStatusBlock(idPrefix)` helper. Two call sites now share one implementation. ListenBrainz playlists keep their own block because the semantics differ — matching progress (total / matched / failed) vs download progress. DEAD-SECTION AUDIT — NONE DEAD Audited the 13 supposedly-dead hidden sections from DISCOVER_REVIEW.md. All 13 are alive: gated on user data (discovery pool, library content, metadata cache) and self-surface when their data exists via `style.display = 'block'` on the success path. The review's grep missed the toggle. No deletions made. DAILY MIXES ORPHAN CALL Removed the orphaned `loadPersonalizedDailyMixes()` call from `blockDiscoveryArtist` — Daily Mixes is intentionally paused (its load call in `loadDiscoverPage` is commented out) so refreshing it from the post-block hook was a no-op. 2204/2204 full suite green. JS parses clean (`node --check`). |
||
|
Broque Thomas
|
4ee78bb973 |
Migrate 7 more discover sections to the shared controller
Follow-up to the foundation commit. Drops the hand-rolled try/catch + spinner injection + empty-state HTML + error-swallow in seven sections by routing them through `createDiscoverSectionController`. Each section keeps its existing public function name + signature so callers, refresh buttons, and dashboard wiring don't notice the swap. Migrated: - `loadDiscoverReleaseRadar` (Fresh Tape) - `loadDiscoverWeekly` (The Archives) - `loadDecadeBrowser` (Time Machine intro carousel) - `loadGenreBrowser` (Browse by Genre intro carousel) - `loadSeasonalPlaylist` (Seasonal Mix) - `loadYourArtists` - `loadBecauseYouListenTo` Skipped (don't fit the controller's single-fetch / single-render-target shape): - `loadYourAlbums` — paginated grid + filters, updates four separate UI elements (subtitle, filter chips, download button, grid). - `loadSeasonalAlbums` — receives pre-fetched data from `loadSeasonalContent`; no fetch URL to satisfy. Hidden / dead sections (~13 of them — `loadPersonalized*`, `loadDiscoveryShuffle`, `loadFamiliarFavorites`, `loadCache*`) untouched in this pass. Separate audit commit will surface or kill them. Two side-effects worth noting: - `loadDecadeBrowser` and `loadGenreBrowser` migrated for completeness, but neither appears wired into `loadDiscoverPage` or any inline handler. May be dead code — flagged for the audit pass. - `loadSeasonalPlaylist` needs a per-load fetch URL (varies by `currentSeasonKey`); worked around by recreating the controller when the key changes. Cleaner option: extend the controller to accept a `fetchUrl: () => string` callable form. Tracked in the follow-up extension list below. Controller extension candidates surfaced for follow-up: - Callable `fetchUrl` (resolves the seasonal playlist recreate-on-key-change hack) - Explicit `isStale` / `onStale` hook (so Your Artists doesn't fold stale handling into renderItems) - `beforeLoad` / `ensureContentEl` hook (so Because You Listen To can let the controller own the dynamic container creation) - No-fetch `data:` mode (so render-only sections like Seasonal Albums can use the controller too) - `onSuccess(data)` hook (cleaner home for header / subtitle side-effects vs folding them into renderItems) Net: -76 lines in `discover.js` even after adding the per-section render helpers. 2204/2204 full suite green. JS parses clean. |
||
|
Broque Thomas
|
07a71f0432 |
Discover section controller foundation + migrate Recent Releases
Every section on the discover page (Recent Releases, Your Artists,
Your Albums, Seasonal Albums, Seasonal Mix, Fresh Tape, The Archives,
Build Playlist, Time Machine, Browse by Genre, ListenBrainz Playlists,
Because You Listen To, plus ~13 hidden sections) currently
re-implements the same lifecycle by hand:
1. show a loading spinner in the carousel container
2. fetch the section's endpoint
3. parse the response, decide if the data is empty
4. either render the items, show an empty-state, or show an error
5. wire post-render handlers (download buttons, hover behavior, etc)
6. maybe expose refresh()
~30 sections worth of duplicated boilerplate, all subtly drifting.
Different empty-state messages. Different error handling (some
`console.debug`, some silently swallowed, some leave the spinner
spinning forever). Different sync-status icons (✓/⏳/✗ vs ♪/✓/✗).
No consistent error toast.
Lifted the lifecycle into a shared `createDiscoverSectionController`
in `webui/static/discover-section-controller.js`. Renderers stay
per-section because section data shapes legitimately differ — album
cards vs artist circles vs playlist tiles vs track rows. The
controller is the wrapper, not a forced visual abstraction.
Foundation contract:
createDiscoverSectionController({
id: 'recent-releases', // for diagnostic logging
contentEl: '#carousel', // selector or Element
fetchUrl: '/api/discover/...',
extractItems: (data) => [...], // pull list from response
renderItems: (items, data, ctx) => '<html>',
onRendered: (ctx) => { ... }, // optional post-render hook
loadingMessage / emptyMessage / errorMessage: copy
sectionEl + hideWhenEmpty: optional whole-section visibility
isSuccess / isEmpty: optional gate overrides
})
Returns `{ load, refresh, destroy, getState }`. Validates config up
front so misuse fails at register-time, not silently on load. Coalesces
concurrent loads (same in-flight promise returned) so a double-click
or repeated trigger doesn't double-fetch. `refresh()` bypasses the
coalesce so the refresh button always re-fires. Errors are logged
(console.debug by default, console.error when verboseErrors=true).
Renderer hook errors are caught + logged so a buggy render callback
can't tear down the controller — keeps the page resilient.
Migrated `Recent Releases` as the proof — simplest album-card shape,
no source-gating, no refresh button. Verified the contract covers it
end-to-end. The legacy `loadDiscoverRecentReleases` entry-point stays
public so existing callers don't change; internally it lazy-builds
the controller and triggers `load()`.
NOT in this commit:
- Other section migrations (one section per follow-up commit, keeps
reviews small + lets us sequence the work)
- Registry-driven section list (so the dead-section audit becomes
registry deletions instead of section-by-section removal)
- Global error toast wrapper
- Per-section "requires X primary source" gate
- Sync-status icon renderer unification
Once every section is on the controller, the discover-page cleanup
work (kill the 13 dead sections, standardize sync-status icons, add
error toasts) becomes single-line registry-level edits instead of
30 separate section-by-section rewrites.
2204/2204 full suite green. JS parses clean (`node --check`). Manual
smoke deferred until follow-up commits — Recent Releases unchanged
on the wire (same endpoint, same payload shape, same render output).
|
||
|
Broque Thomas
|
6aafcaae93 |
Bump version to 2.4.2
- `web_server.py` — `_SOULSYNC_BASE_VERSION` 2.4.1 → 2.4.2
- `webui/static/helper.js` — flip the 2.4.2 WHATS_NEW header from
"Unreleased — 2.4.2 dev cycle" to "May 7, 2026 — 2.4.2 release"
so the per-version block stops being filtered out by
`_getLatestWhatsNewVersion`. Also bumps the safety-net default
inside that helper from 2.4.1 → 2.4.2.
- `.github/workflows/docker-publish.yml` — manual-trigger default
tag bumped to match.
Drive-by fix: escaped a stray single quote in the `Internal: Download
Engine` 2.4.2 entry that broke `node --check` on the file
(`orchestrator.client('soulseek')` inside a single-quoted desc string
silently terminated the string mid-entry). Pre-existing, unrelated to
the bump but caught while validating JS parse for the release.
VERSION_MODAL_SECTIONS not rotated in this commit — separate
editorial pass.
|
||
|
Broque Thomas
|
1a2da016e4 |
Add download buttons + bulk action to artist top-tracks sidebar
Closes #513 (s66jones). The artist detail page already showed a "Popular on Last.fm" sidebar — list of an artist's top tracks by playcount, with a play button per row but no download action. Issue #513 wanted a way to grab those tracks the same way zotify let users grab "top X songs" without pulling the full discography. Pulls from the configured primary metadata source (Spotify `artist_top_tracks`, Deezer `/artist/{id}/top`) when available, falls back to the existing Last.fm display-only mode for sources that don't expose popularity ranking (iTunes / Discogs / MusicBrainz). Source label in the section title shifts to match. Each row gets a hover-revealed download button that wishlists the single track via the existing /api/add-album-to-wishlist endpoint (preserves the track's real album metadata, so the wishlist worker later places the file in its proper album folder). A "Download All" footer button opens the standard download modal in PLAYLIST context, not album context — the virtual playlist_id is `top_tracks_<source>_<artistId>` which doesn't match any of the album-prefix checks in `startMissingTracksProcess` (downloads.js). That keeps `is_album_download=false`, so the master worker doesn't inject a wrapper context as `_explicit_album_context`. Each track downloads using its own real album metadata, files land in proper per-album folders on disk (not a fake "Top Tracks" folder). Backend additions: - `SpotifyClient.get_artist_top_tracks(artist_id, country, limit)` — wraps `spotipy.artist_top_tracks`, returns up to 10 tracks for the market (Spotify's API cap). UI-side limit trim only. - `DeezerClient.get_artist_top_tracks(artist_id, limit)` — wraps `/artist/{id}/top?limit=N`, converts Deezer's raw shape to the same Spotify-compatible dict layout (id, name, artists, album with album_type / total_tracks / images, duration_ms, track_number, disc_number) so downstream code doesn't branch on source. - `GET /api/artist/<id>/top-tracks` — dispatches to whichever client matches the primary source. Resolves per-source artist IDs from the DB row first (matching what /discography already does) so a Spotify ID in the URL still works when Deezer is primary, and vice versa. Returns `{success, source, tracks, resolved_artist_id}` on hit; `{success: False, reason: 'unsupported_source' | 'spotify_not_authenticated' | 'deezer_unavailable' | 'no_tracks_found'}` on miss so the frontend can decide whether to fall through to Last.fm. Frontend: - `_loadArtistTopTracks` tries the metadata source first, falls through to the legacy `/api/artist/0/lastfm-top-tracks` call if the source can't deliver. Section title and per-row UI shift based on which source answered. - New per-row `.hero-top-track-download` button (hover-revealed). - New `.hero-top-tracks-download-all` footer button — only visible when metadata-source mode rendered the list (Last.fm fallback hides it since rows have no track IDs to download). Tests: 10 new tests pin the client methods — - Spotify: returns track list, honors UI limit cap, returns empty when unauthed / artist_id missing / API throws. - Deezer: shape conversion to Spotify-compatible dict, empty when no data / artist_id missing, limit clamping at upper bound, default fallback when limit=0, malformed entries skipped. The Flask endpoint dispatcher itself isn't covered by the new test file because importing web_server at test-collection time spins up worker threads that race with caplog-using tests elsewhere in the suite (specifically test_library_reorganize_orchestrator). Endpoint verified manually; the underlying client methods (the load-bearing logic) are covered. 2204/2204 full suite green (was 2194 + 10 new). |
||
|
Broque Thomas
|
01c528fd5f |
Reject AcoustID matches whose version disagrees with the expected track
Discord report (corruption [BWC]): downloads coming through as the
instrumental cut when a vocal track was requested. The verification
step's `_normalize` function strips parentheticals and version-suffix
tags ("(Instrumental)", "- Live", etc) so legitimate name variations
don't false-fail the title-similarity check. That also means "In My
Feelings" and "In My Feelings (Instrumental)" both normalize to "in
my feelings", title similarity is 1.0, and the wrong cut passes
verification.
Detect the version label on each side BEFORE normalization runs. If
the expected and matched recordings disagree on version (one is
original, the other is instrumental / live / acoustic / remix /
etc), return FAIL — the fingerprint identified a real song, just
not the version the caller asked for.
Reuses `MusicMatchingEngine.detect_version_type` so the same regex
patterns the pre-download Soulseek matcher applies also drive
post-download verification. No duplicated tables.
Also gates the secondary fallback scan, so a wrong-version variant
sitting in the same fingerprint cluster can't win the loop after
the best match has already been version-rejected.
6 tests pin the behavior:
- instrumental returned for vocal request → FAIL
- vocal returned for instrumental request → FAIL
- live vs acoustic → FAIL
- matching versions on both sides → PASS
- original-to-original happy path → PASS (regression guard)
- secondary scan skips wrong-version recordings → not PASS
2194/2194 full suite green (was 2188 + 6 new).
|
||
|
Broque Thomas
|
caa1c198e5 |
Fix non-admin profiles defaulting to Spotify on search picker
Closes #515 (jaruca). Search-picker controller in shared-helpers.js resolved the user's configured primary metadata source by fetching `/api/settings`. That endpoint is `@admin_only` (it returns full config including credentials), so non-admin profiles got a 403 and the controller silently fell back to the hardcoded `'spotify'` default — admin's chosen source (deezer / itunes / discogs / etc) was ignored on every non-admin profile, forcing manual reselection each session. Switched to `/status`, which is public and already exposes the resolved `metadata_source` for the dashboard. Same value the picker needs — different endpoint that doesn't gate non-admins. Admins see no behavior change. Non-admins now see admin's configured primary source as the default active icon. Refs #515 |
||
|
Broque Thomas
|
9602d1827c |
Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites
Catches the silent excepts the awk-based earlier sweeps missed:
- Bare `except:` followed by `pass` (also swallows KeyboardInterrupt
and SystemExit — actively wrong). Upgraded to `except Exception as
e: logger.debug("...: %s", e)`. ~14 sites across connection_detect,
soulseek_client, listenbrainz_manager, watchlist_scanner,
youtube_client, navidrome_client, jellyfin_client, web_server.
- `except Exception:` + pass that the awk pattern missed (e.g.
multi-line or unusual whitespace). ~31 sites across automation_engine,
database_update_worker, music_database, spotify_client, web_server,
others.
- 14 legitimate cleanup sites left silent with explicit `# noqa: S110`
+ comment explaining why (atexit handlers, finally-block conn.close
calls). Logging during shutdown can itself crash because file handles
get torn down before the handler fires.
Also enables `S110` rule in pyproject.toml so this pattern fails CI
going forward — drift fails at PR review instead of at runtime against
a wedged worker thread. Tests path keeps S110 ignored (test fixtures
legitimately use try-except-pass for cleanup).
Adds a WHATS_NEW entry to helper.js summarizing the full #369 sweep.
Verified: `python -m ruff check .` → All checks passed.
Verified: `python -m pytest tests/` → 2188 passed.
Closes #369
|
||
|
Broque Thomas
|
4c11375930 |
Repair job card badge — show pending count, not last-scan count
Discord report: Duplicate Detector card said "372 findings" and Cover
Art Filler said "60 findings", but clicking the Findings tab's Pending
filter showed 0. User read it as "findings aren't being created" —
looked like a detector bug.
Actual cause: the badge sourced ``last_run.findings_created``
(historical "found in last scan") without considering current state.
After the user (or bulk-fix automation) resolved or dismissed those
findings, they no longer appeared on the Pending tab — but the badge
kept showing the last-scan number in red urgent styling.
Backend was correct end-to-end: detectors create pending rows,
bulk-fix moves them to resolved, Findings tab filters by status.
Only the badge display lied about current state.
Fix:
- ``RepairWorker._get_pending_count_by_job()`` — single SQL aggregation
returning ``{job_id: pending_count}`` for every job with pending
findings. O(1) lookup per job instead of N round trips.
- ``get_all_job_info()`` calls it once per request and adds
``pending_findings_count`` to each job's API response.
- ``enrichment.js`` job card now branches on the count:
- ``> 0`` → red ``"X pending"`` badge (urgent, action needed)
- ``= 0`` AND last scan found something → muted grey ``"X found in
last scan"`` (historical context, no action needed)
- New CSS class ``.repair-flow-badge.findings-historical`` for the
muted slate color so the two states are visually distinct.
User-visible result with the screenshotted state (372 dup / 60 cover-
art findings, all resolved):
- Before: red "372 findings" / "60 findings" — implied 432 things to
do, but Findings tab showed 0 pending
- After: grey "372 found in last scan" / "60 found in last scan" —
the badge text tells the user the count is historical, no surprise
when Pending is empty
Tests: 3 new tests in ``tests/test_create_finding_dedup_counter.py``
pin the per-job pending count helper:
- returns ``{job_id: count}`` based on status='pending' rows only;
resolved + dismissed rows excluded
- empty dict when no pending findings exist
- gracefully returns ``{}`` on DB error (badge falls back to
historical count via the existing JS ``or 0`` safety)
2188/2188 full suite green. Pure UI/state-display fix — no detector
logic, no backend behavior change.
|
||
|
Broque Thomas
|
5c69b853b4 |
Bound slskd HTTP timeout — fixes worker thread deadlock
GitHub issue #499 (@bafoed). Big initial sync of Spotify playlists worked for 2-3 hours then downloads silently stopped: - 3 active tasks stuck in "Searching" state, replaced every ~10 min by different ones - slskd UI showed no actual searches happening - Debug log: orphaned-task count grew over time, no jobs executed - Container restart was the only fix (bought another 2-3 hours) - Not a rate limit (rates showed 0/min) Root cause: ``core/soulseek_client.py`` constructed ``aiohttp.ClientSession()`` with no timeout at four sites. When slskd hung on a request (overloaded, transient network blip, internal stall), the HTTP call blocked indefinitely — and the worker thread blocked with it. The download executor only has ``ThreadPoolExecutor(max_workers=3)``, so once 3 worker threads were wedged on hung calls, no further downloads could start. Batch-level "stuck detection" (10-minute timer in ``check_batch_completion_v2``) was correctly marking tasks ``not_found`` and trying to start replacements, but the executor pool was exhausted — replacements queued forever inside the executor with no thread to run them. Symptom: tasks rotating every ~10 min at the batch level while the underlying executor stayed wedged. Fix: bounded ``aiohttp.ClientTimeout`` (total 120s, connect 15s, sock_read 60s) on every slskd ``ClientSession`` construction. Module- level constant ``_SLSKD_DEFAULT_TIMEOUT`` so the four sites stay in lockstep — future sites get the same protection by reusing the constant. Why these timeouts are safe: - Every slskd API call is metadata-level (search submission, status polls, download enqueue, transfer state queries). None stream files — slskd handles file transfer via its own peer-to-peer infrastructure entirely outside our HTTP requests. - Legitimate metadata calls finish in seconds. 120s ceiling is ~50× the normal latency. Timeout handling: - ``asyncio.TimeoutError`` caught explicitly BEFORE the generic ``except Exception`` — surfaces "slskd timed out" specifically in logs (debuggable instead of buried as "Error making API request"). - Returns None to the caller (same code path as a 5xx response or any other failure). No new error path; callers already handle None as "request failed". - Worker thread unblocks immediately → executor pool stays healthy → downloads keep flowing. Sites updated: - ``_make_request`` (general /api/v0/ helper, line 152) — used for every slskd API operation - ``_make_direct_request`` (non-/api/v0/ helper, line 235) - ``_explore_api_endpoints`` Swagger fetch (line 1566) — diagnostic - ``_explore_api_endpoints`` per-endpoint probe (line 1617) — diagnostic Tests: 3 new tests in ``tests/downloads/test_soulseek_pinning.py`` pin: - ``_SLSKD_DEFAULT_TIMEOUT`` is bounded (total set, ≤300s ceiling, connect ≤60s) — guards against future regressions that drop or unbound the timeout - ``_make_request`` returns None on ``asyncio.TimeoutError`` rather than raising — pins the caller contract - ``_make_direct_request`` returns None on ``asyncio.TimeoutError`` 2185/2185 full suite green. Closes #499. |
||
|
Broque Thomas
|
ca5c93162c |
Rewrite Library Reorganize job to delegate to per-album planner
GitHub issue #500 (@bafoed). Library Reorganize repair job moved album tracks to single-template paths because of a fragile classification heuristic. Concrete symptom: a track at ``Surf Curse/Surf Curse - Nothing Yet (2017)/01 - Christine F.flac`` got proposed for a move to ``Surf Curse/Surf Curse - Christine F/Surf Curse - Christine F.flac`` (single template) instead of staying under the album folder. Root cause: the job had its own tag-reading + transfer-folder-walk + template-application implementation. The classification was ``is_album = (group_size > 1)`` where ``group_size`` was the count of same-album tracks currently sitting in the transfer folder being scanned. Two failure modes: - only one track of an album was in the transfer folder (rest already moved to the library, or not yet downloaded), or - album tags varied slightly across tracks (e.g. ``"Buds"`` vs ``"Buds (Bonus)"``) Either case gave a 1-element group → routed through the SINGLE template → wrong destination. Rewrite — delegate to the per-album planner the artist-detail "Reorganize" modal already uses: - ``core.library_reorganize.preview_album_reorganize`` for path computation (DB-driven, knows the album has N tracks regardless of how many sit in transfer; album-vs-single is structurally correct) - ``core.reorganize_queue.enqueue_many`` for apply mode; the queue worker dispatches via ``reorganize_album`` which handles file move + post-processing + DB update + sidecar through the same code path the per-album modal uses Job's per-album loop: - iterate albums for the active media server only (matches the artist- detail modal's scope; multi-server users won't have the job touch the inactive server's files at paths they can't see) - preview each album, catch exceptions per-album so one bad row doesn't abort the scan - branch on planner status: - ``no_album`` / ``no_tracks`` (race: album deleted mid-scan) → skip silently - ``no_source_id`` (album never enriched) → emit ONE album-level "needs enrichment first" finding (vs N per-track findings cluttering the UI) - ``planned`` → filter mismatched tracks (matched + new_path + not unchanged + file_exists), emit per-track findings (dry-run) or collect album for bulk enqueue (apply) - bulk enqueue at end of loop using the queue's correct return-shape (``{'enqueued': N, 'already_queued': M, 'total': K}``) What's gone (~500 LOC): - ``_read_tag_metadata`` / ``_get_audio_quality`` / transfer-folder walk - ``_load_album_years`` / ``_lookup_years_from_api`` (planner does this) - ``_apply_path_template`` / ``_build_path_from_template`` - direct ``shutil.move`` + sidecar move logic (queue handles) - the fragile ``is_album = group_size > 1`` heuristic — structurally gone - ``move_sidecars`` setting (no longer applicable; queue's post-process re-downloads cover art at the destination) What stays: - dry-run vs apply toggle - ``file_organization.enabled`` gate - stop / pause respect - progress reporting - findings for the UI Cleaner separation of concerns: - this job: DB-known tracks at wrong paths (active server only) - ``orphan_file_detector``: files on disk with no DB entry - ``dead_file_cleaner``: DB entries pointing to nonexistent files Tests: 12 tests in ``tests/test_library_reorganize.py`` pin the delegation contract — every status branch, every track-filter case, exception handling, apply-mode enqueue payload, active-server scope, estimate-scope shape. Three obsolete ``_lookup_years_*`` tests removed (year handling moved to planner). Closes #500 (the misclassification half — orphan + dead-file are downstream sync-gap symptoms, separate concern). |
||
|
Broque Thomas
|
cceffbd8ec |
Honor manually-matched source IDs in per-source enrichment workers
GitHub issue #501 (@Tacobell444). After manually matching an album to a specific source ID via the match-chip UI, clicking "Enrich" on that album would fuzzy-search by name and overwrite the manual match with whatever the search returned — or revert the match status to ``not_found`` if name search missed. Reorganize then read the now- wrong ID and moved files to the wrong destination. Root cause was in the per-source enrichment workers' ``_process_*_individual`` methods. Several workers (Spotify, iTunes) ran search-by-name unconditionally with no check for an existing stored ID. Others (Deezer, Tidal, Qobuz) skipped on existing-ID but without refreshing metadata — preserved the ID but didn't actually honor the user's intent of "use this match to pull fresh data". Cin-shape lift: same fix needed in 5 workers, so extracted the shared behavior into ``core/enrichment/manual_match_honoring.py``: honor_stored_match( db, entity_table, entity_id, id_column, client_fetch_fn, on_match_fn, log_prefix, ) -> bool Per-worker variability (DB column name, client fetch method, response shape) plugs in via callbacks. Workers call the helper at the top of ``_process_album_individual`` / ``_process_track_individual``; if it returns True, the manual match was honored and the search-by-name fallback is skipped. If False (no stored ID, fetch failed, or empty response), the worker's existing search-by-name flow runs as before. Workers wired: - spotify_worker — album + track (was overwriting; now honors) - itunes_worker — album + track (was overwriting; now honors) - deezer_worker — album + track (was skip-on-id; now refreshes) - tidal_worker — album + track (was skip-on-id; now refreshes) - qobuz_worker — album + track (was skip-on-id; now refreshes) Workers left alone (already correct): - discogs_worker — already had inline stored-ID fast path that refreshes metadata. Same behavior, just inline; refactoring to use the shared helper would be churn for zero behavior change. - audiodb_worker — same — inline fast path with full metadata refresh. - musicbrainz_worker — preserves existing MBID and marks status, which is the correct behavior for MB (the MBID itself is the match payload — no separate metadata fetch). - lastfm_worker / genius_worker — name-based services with no source-specific IDs to honor. Inherent re-search per call. Reorganize fixed indirectly — it always honored stored IDs correctly via ``library_reorganize._extract_source_ids``. The "Reorganize broken" symptom was downstream of broken Enrich corrupting the stored ID. Tests: - ``tests/enrichment/test_manual_match_honoring.py`` — 11 tests pinning the shared helper contract: stored-ID fast path, no-ID fallthrough, empty-string treated as no ID, missing row, fetch exception caught and falls through, fetch returns None falls through, callback exceptions propagate, configurable table + column, defensive table-name whitelist. - Per-worker wiring NOT tested individually — the workers depend on live DB / client objects that are heavy to mock. The shared helper's contract is pinned; per-worker call sites are short enough to verify by code review. 2173/2173 full suite green. Closes #501. |
||
|
Broque Thomas
|
fd5ccf4cb8 |
Fix "no such table: hifi_instances" via defensive lazy-create
GitHub issue #503 (@hadshaw21). Adding a HiFi instance via downloader settings popped up ``no such table: hifi_instances`` even though "Test Connection" and "Check All Instances" both worked. Root cause: ``MusicDatabase._initialize_database`` runs every ``CREATE TABLE`` + every migration step inside one sqlite transaction. Python's sqlite3 module doesn't autocommit DDL by default, so if any later migration step throws on a user's specific DB shape (e.g. an old volume from a prior SoulSync version with quirky schema state), the WHOLE batch rolls back — including the ``hifi_instances`` CREATE that ran earlier in the function. The user's next boot retries init, hits the same migration failure, rolls back again. The ``hifi_instances`` table never lands no matter how many restarts. Fix: defensive lazy-create. New ``_ensure_hifi_instances_table(cursor)`` helper runs ``CREATE TABLE IF NOT EXISTS`` on demand, called immediately before every CRUD operation that touches ``hifi_instances``: - ``get_hifi_instances`` / ``get_all_hifi_instances`` (read) - ``add_hifi_instance`` / ``remove_hifi_instance`` (CRUD) - ``toggle_hifi_instance`` / ``reorder_hifi_instances`` (CRUD) - ``seed_hifi_instances`` (defaults seed) Idempotent — costs one no-op CREATE check when the table is already present, fully recovers from a broken init state. Read methods now return empty instead of raising when init failed; write methods work end-to-end. Doesn't paper over the underlying init issue (still worth tracking which migration step breaks for which user DB shapes — separate concern) but makes HiFi instance management self-healing in the meantime. Tests: - 7 obsolete tests that pinned ``raises sqlite3.OperationalError`` removed — that contract is no longer correct - 7 new tests pin the lazy-create behavior: every CRUD method works against a DB that's missing the ``hifi_instances`` table, verifying the table gets created and the operation completes 2162/2162 full suite green. Pure additive — no behavior change for users with a healthy DB; affected users get back to working hifi instance management. Closes #503. |
||
|
Broque Thomas
|
9f2813fce4 |
Add cross-section dedup to all-libraries listing layer
Followup to the all-libraries-mode commit. Without dedup, a Plex Home family where two users both have "Drake" in their music libraries would see "Drake" twice in SoulSync's library list — Plex returns distinct ratingKeys for each section's copy of the same artist. Dedup design — applied selectively, NOT everywhere: - ``_dedupe_artists(artists)``: groups by lowercased title, picks the canonical entry by ``leafCount`` (more tracks wins). Active ONLY in all-libraries mode; single-library mode is a no-op fast path with zero behavior change. - ``_dedupe_albums(albums)``: same but keys on (lowercased parentTitle, lowercased title) so two artists with identically-titled albums (e.g. self-titled releases) stay separate. Applied to: - ``get_all_artists()`` — public listing for the library view - ``get_library_stats()`` — count matches what user sees in the list Deliberately NOT applied to: - ``get_all_artist_ids()`` / ``get_all_album_ids()`` — these feed removal detection (compare returned ratingKey set against DB-linked ratingKeys to decide what's been removed). Deduping here would falsely flag non-canonical ratingKeys as "removed" and prune SoulSync's DB tracks that are linked to them. Pinned by two CRITICAL tests. - ``_all_tracks()`` — track count stays raw because the same track in two sections IS two distinct files / Plex entries, not a logical duplicate. - ``_search_general()`` and ``search_tracks`` Stage 1/2 — search results stay raw so cross-section matches aren't lost. Stage 1 may miss cross-section tracks for the same artist but Stage 2's server-wide track search catches them. Logging: when raw vs deduped artist counts differ, ``get_all_artists`` logs both so users can see "Found 4697 artists across all music sections (4521 unique after cross-section dedup)" — surfaces the overlap clearly. Tests: 8 new tests in test_plex_all_libraries.py pin: - canonical pick by leafCount (artists + albums) - case-insensitive name match - single-library no-op path (zero behavior change for those users) - album dedup keys on (artist, title) so different-artist same-title albums stay separate - ``get_all_artists`` listing applies dedup - ``get_all_artist_ids`` does NOT dedup (CRITICAL — removal detection) - ``get_all_album_ids`` does NOT dedup (CRITICAL — removal detection) - ``get_library_stats`` uses deduped counts for artists/albums but raw count for tracks Existing pre-stat test updated to use distinct mock instances — ``[MagicMock()] * 5`` creates five references to one mock which now correctly collapses under dedup. 71/71 media_server tests green, 2162/2162 full suite green. Honest known limitation acknowledged in WHATS_NEW + version modal: write-back (genre / poster / metadata updates) targets one ratingKey at a time — only updates the canonical section's copy of an artist if it exists in multiple. Other section's copy stays unchanged. Document and revisit if it matters. |
||
|
Broque Thomas
|
620c41f1ac |
Add "All Libraries (combined)" mode to PlexClient
GitHub issue #505 (PopeBruhLXIX): users with multiple Plex music libraries (e.g. one per Plex Home user, or two folder roots split across separate library sections) only saw one library inside SoulSync because the connection settings forced you to pick a single library section. SoulSync's PlexClient stored exactly one ``self.music_library`` section reference and every read scanned only that one. This change adds an opt-in "All Libraries (combined)" dropdown option that flips the client into a server-wide read mode where every read method (``get_all_artists`` / ``get_all_album_ids`` / ``search_tracks`` / ``get_library_stats`` / etc) dispatches through ``server.library.search(libtype=...)`` instead of querying a single section. One Plex API call replaces N per-section iterations; Plex handles the aggregation server-side. Implementation: - ``ALL_LIBRARIES_SENTINEL`` (``'__all_libraries__'``) — module-level constant used as the saved DB preference value when the user picks the synthetic "All Libraries" entry. Detection is one string compare in ``_find_music_library`` / ``set_music_library_by_name``. Existing preferences (real library names) are unaffected. - ``self._all_libraries_mode`` (private flag) + ``is_all_libraries_mode()`` (public accessor for external callers). When True, ``music_library`` may stay None — ``is_fully_configured()`` recognizes the mode and still returns True so dispatch sites don't bail. - New private helpers ``_can_query``, ``_get_music_sections``, ``_all_artists``, ``_all_albums``, ``_all_tracks``, ``_search_general``, ``_search_artists_by_name``. Single dispatch point for the section-vs-server branch — every read method funnels through them so future drift fails at one place. - New public helpers for downstream callers: - ``get_recently_added_albums(maxresults, libtype)`` — used by DatabaseUpdateWorker's deep-scan recent-content sweep - ``get_recently_updated_albums(limit)`` — same - ``get_music_library_locations()`` — returns folder roots, used by web_server.py's file-path resolver - ``trigger_library_scan`` and ``is_library_scanning`` fan out across every music section in all-libraries mode. - ``get_available_music_libraries`` prepends a synthetic ``{'title': 'All Libraries (combined)', 'value': sentinel}`` entry ONLY when more than one music library exists. Single-library users don't get the extra option. ``value`` field is the canonical identifier the frontend submits to ``/api/plex/select-music-library`` (real libraries: title; synthetic: sentinel string). Backward- compatible — entries without ``value`` fall back to ``title``. Three crash points fixed in downstream consumers (would have failed during a deep scan after the user picked all-libraries mode): 1. ``database_update_worker.py:411`` — bailed out with "No music library found in Plex" because ``not self.media_client.music_library`` evaluated True in all-libraries mode (music_library is None there). Now uses ``is_fully_configured()`` which recognizes the mode. This was the root cause of the deep scan never starting. 2. ``database_update_worker.py:_get_recent_albums_plex`` — reached ``self.media_client.music_library.recentlyAdded()`` / ``.search()`` directly, AttributeError in all-libraries mode. Now routes through the new helper methods. 3. ``web_server.py:10947`` (file-path resolver) — accessed ``music_library.locations``; gated on ``music_library`` truthy so it didn't crash, but silently skipped all-libraries-mode locations. Now uses ``get_music_library_locations()`` which unions across sections. Plus polish: - ``/api/plex/clear-library`` also resets ``_all_libraries_mode`` so a fresh "select library" flow doesn't inherit stale mode state. - ``/api/plex/music-libraries`` surfaces "All Libraries (combined)" as ``current_library`` when in mode (settings UI displays correctly). - Frontend ``loadPlexMusicLibraries`` uses ``library.value || library.title`` so the sentinel-keyed option submits the sentinel string, not the human-readable label. Pre-select match handles both paths. Honest tradeoffs (documented as known limitations): - Same artist appearing in multiple Plex sections shows as separate entries in SoulSync (no dedup). Plex returns distinct ratingKeys for each. Cosmetic; revisit if it bites users. - Write-back (genre / poster updates) targets one ratingKey at a time — only updates that section's copy. Other sections' copies stay unchanged. - All-libraries mode includes any audiobook library that Plex classifies as ``type='artist'``. Edge case, opt-in only. Tests: 21 new tests in tests/media_server/test_plex_all_libraries.py pin both single-library mode (regression guard) and all-libraries mode for every refactored method. Existing test_plex_pinning.py fixture updated to initialize the new flag. 63/63 media_server tests green, 2148/2148 full suite green. |
||
|
Broque Thomas
|
822759740d |
Fix Download Discography pulling wrong artist + log routing
Two fixes. (1) Discography endpoint now does server-side per-source ID resolution. When the user clicked Download Discography on a library artist, the endpoint received whichever artist ID the frontend happened to pick (spotify_artist_id || itunes_artist_id || deezer_id || library_db_id) and dispatched it as-is to whichever source it queried. If the picked ID didn't match the queried source's ID format, the lookup returned wrong-artist results (numeric ID collisions) or fell back to a fuzzy name search that picked a wrong artist. Two reproducible cases: - 50 Cent's library row had DB id 194687 — coincidentally a real Deezer artist ID for "Young Hot Rod". When the frontend's /enhanced fetch silently fell back to the DB id, the backend sent 194687 to Deezer, and Deezer returned Young Hot Rod's 50 albums in 50 Cent's discography modal. - Weird Al's library row had a stored Spotify ID. The frontend sent that to Deezer, which rejected the alphanumeric ID and fell back to fuzzy name search — which picked The Beatles somehow, returning 45 Beatles albums. The mechanism for per-source ID dispatch already exists in ``MetadataLookupOptions.artist_source_ids``, and the watchlist scanner already uses it; the on-demand discography endpoint just wasn't wired to it. Fix: when the URL artist_id matches a library row by ANY stored ID (DB id, spotify_artist_id, itunes_artist_id, deezer_id, or musicbrainz_id), pull every stored provider ID and pass them as ``artist_source_ids``. Each source gets its OWN stored ID regardless of which one the URL carries. When the URL ID is a non-library source-native ID and the row lookup misses entirely, behavior is identical to before (single-ID dispatch fallback). Logged the resolved per-source ID dict at INFO so future "wrong artist showed up" diagnostics are immediately legible in app.log. (2) Logger namespace fix in core/artists/quality.py and core/metadata/multi_source_search.py. Both modules used ``logging.getLogger(__name__)`` which resolves to ``core.artists.quality`` / ``core.metadata.multi_source_search`` — neither under the ``soulsync`` namespace where the file handler is wired. Result: every [Enhance], [MultiSourceSearch], and direct-lookup INFO line was being written to a logger with no handlers and silently dropped. App log showed the slow-request warning but no diagnostic detail. Switched both to ``get_logger()`` from utils.logging_config so the soulsync.* namespace picks them up. Same content, now actually lands in app.log. Confirmed working in live test: ``[Enhance] Direct lookup matched: deezer ID 1476162252 → 'Desastre'`` No behavior change in any other caller. Empty ``artist_source_ids`` (no library row matched) reaches lookup as ``None`` → identical to current single-ID dispatch path. Logger fix is pure routing — no content change. |
||
|
Broque Thomas
|
3befe9349c |
Direct ID lookup in Enhance Quality, like Download Discography
Followup on the previous Enhance refactor. Multi-source parallel text
search closed the worst case (users with no Spotify/Deezer getting
"unknown artist - unknown album - unknown track" wishlist entries),
but text search itself is still fragile against messy library tags:
"Title (Live)", featured artists in the artist field, etc. Download
Discography never had this problem because it resolves albums by stable
ID, not by name.
Enhance now does the same thing for tracks: for every metadata source
the user has configured, if the library track has the corresponding
stored ID (spotify_track_id / deezer_id / itunes_track_id / soul_id),
call client.get_track_details(stored_id) directly and convert to the
wishlist payload. First success wins. The user's configured primary
source is tried first so a Deezer-primary user gets Deezer payloads on
the wishlist entry (correct cover art / album shape) even when other
sources also have stored IDs for the same track.
Multi-source parallel text search stays as the fallback for tracks
with no stored IDs (e.g. manually imported, never enriched). Empty-
field rejection still gates the wishlist add.
Implementation:
- _STORED_ID_COLUMNS: source name → DB column mapping
(Discogs intentionally omitted — release-based, no per-track IDs)
- _enhanced_to_wishlist_payload: converts the get_track_details
intermediate "enhanced" shape (artists as [str]) to wishlist shape
(artists as [{'name': str}]). Spotify's raw_data is already in
wishlist shape, returned as-is when detected (preserves full
album.images that the enhanced top-level fields drop)
- _try_direct_lookup_all_sources: iterates sources preferred-first,
calls get_track_details on each that has both a stored ID and a
configured client, returns first complete-metadata payload
- spotify_client field removed from ArtistQualityDeps (no longer
used — Spotify direct lookup now flows through the generic
per-source loop using the entry from search_sources)
- _try_upgrade_to_rich_payload removed (was Spotify-only with broken
shape semantics for non-Spotify sources; search-fallback now uses
_build_payload_from_track consistently)
- get_primary_source() consulted to set the per-call preferred source
for direct-lookup priority
Also fixed a stale UI string: the Enhance modal toast read "Matching
tracks to Spotify and adding to wishlist..." regardless of which
sources were actually configured. Now reads "Matching tracks across
metadata sources...".
Tests:
- _build_deps mirrors web_server._resolve_search_sources: passing
spotify=spotify_obj auto-prepends ('spotify', spotify_obj) to
search_sources (Spotify is always added when configured in prod)
- 5 new tests pin the direct-lookup behavior:
- test_direct_lookup_via_deezer_id_skips_text_search
- test_direct_lookup_via_itunes_id_skips_text_search
- test_direct_lookup_prefers_user_primary_source
- test_direct_lookup_falls_through_to_text_search_when_no_stored_ids
- test_direct_lookup_failure_falls_through_to_text_search
- Reframed enhanced-format and search-fallback tests for the new
payload-build path (no album-image side call, search-fallback uses
_build_payload_from_track consistently)
- 22/22 quality tests green, 2133/2133 full suite green.
|
||
|
Broque Thomas
|
7316646b01 |
Extract multi-source search; Enhance Quality matches Redownload coverage
Track Redownload had been doing parallel multi-source metadata search across every configured source the whole time; Enhance Quality was running a single-source primary fallback that returned junk matches with empty fields when the primary was iTunes (Discord report: "unknown artist - unknown album - unknown track" wishlist entries for users with neither Spotify nor Deezer connected). Lift the redownload search into core/metadata/multi_source_search.py and point both flows at it. Same scoring, same per-source query optimization (Deezer's structured artist:/track: form), same current-match flagging via stored source IDs. ArtistQualityDeps now takes get_metadata_search_sources (returns [(name, client), ...] for every configured source) instead of the single-primary get_metadata_fallback_client + get_metadata_fallback_source. Spotify direct-lookup stays as a fast-path optimization (only Spotify exposes get_track_details(id) returning rich raw payload); when it doesn't fire, the multi-source parallel search picks the cross-source best match. Empty-field matches still rejected before wishlist add. Tests: _build_deps helper updated to accept the new search_sources contract while preserving fallback_client/fallback_source ergonomics. Reframed tests for the new semantics — direct-lookup is no longer gated on Spotify being the active primary; failure reason now lists every searched source. Added a test pinning the no-sources-configured prompt. 17/17 quality tests green, 2128/2128 full suite green. |
||
|
Broque Thomas
|
4a27f3c245 |
Source-agnostic Enhance Quality flow + reject empty matches
Discord report: clicking Enhance Quality on an artist with neither
Spotify nor Deezer connected added tracks to the wishlist as
"unknown artist - unknown album - unknown track".
Root cause was structural. core/artists/quality.py had a hardcoded
Spotify-direct → Spotify-search → iTunes-fallback chain that ignored
the user's configured primary metadata source. When Spotify wasn't
connected, every track fell through to an iTunes-only fallback that
occasionally returned matches with empty fields (cleared the 0.7
confidence threshold but missing artist / album / title). Those
empty strings propagated through the wishlist payload normalizer's
truthy-check passthrough at core/wishlist/payloads.py:77-80 and the
UI rendered them as "Unknown" defaults.
Rewrote the flow source-agnostic:
- ArtistQualityDeps gains get_metadata_fallback_source. Flow resolves
the user's active primary source once up front.
- New _build_payload_from_track helper produces the Spotify-shaped
wishlist payload from any source's Track object — single place
that knows how to construct it (replaces the duplicate construction
in the Spotify-search and iTunes-fallback paths).
- New _search_match helper does generic confidence-scored search
against any client implementing search_tracks(query, limit). Same
0.7 threshold, same album-bonus weighting as before.
- New _has_complete_metadata validator rejects matches with empty
title / album / artists before they reach the wishlist.
- _spotify_direct_lookup kept as a Spotify-only optimization (only
Spotify exposes get_track_details(id) returning rich raw payload);
other sources fall through to search.
- Failure reason now names the active source: "No usable {source}
match — connect another metadata source for better coverage".
Result: Discogs users get a Discogs search. Hydrabase users get a
Hydrabase search. iTunes users get an iTunes search with empty-field
rejection. Spotify keeps its direct-lookup fast path.
6 new tests pin the architectural change:
- Primary-source dispatch routes to the configured client (Discogs,
not Spotify) when Spotify isn't primary
- Spotify direct-lookup is gated on Spotify being the active primary
(skipped when Discogs is configured even if track has spotify_track_id)
- Empty title / album / artists fields all reject the match
- Failure reason names the active source
|
||
|
Broque Thomas
|
b0dc139b72 |
Sync WHATS_NEW with current engine surface
The "Media Server Engine Foundation" entry was written when the engine still had safe-default routing wrappers for optional methods. Those were dropped in the honesty pass. Entry now matches reality: - Lists the actual engine surface (6 methods: client / active_client / active_server / is_connected / configured_clients / reload_config) instead of claiming "uniform safe defaults for optional methods" - References KNOWN_PER_SERVER_METHODS as the data-only listing (replaces the old OPTIONAL_METHODS naming) - Cites real test counts (42 total) instead of the stale 35 - Drops the "33+ dispatch sites" overclaim (was already partial); the actual framing is "uniform-shape chains lifted, ~18 server-specific chains stay explicit per the lift-what's-truly-shared standard" |
||
|
Broque Thomas
|
f230c93890 |
Merge remote-tracking branch 'origin/dev' into refactor/media-server-engine
# Conflicts: # core/matching_engine.py # services/sync_service.py # web_server.py # webui/static/helper.js |
||
|
Broque Thomas
|
edb6d1bc33 |
Drop dead per-server class imports + update WHATS_NEW
- services/sync_service.py: dropped unused PlexClient / JellyfinClient / NavidromeClient class imports. After the engine refactor the service only needs TrackInfo for type annotations; the class imports were dead. - WHATS_NEW: extended the media server engine review-pass entry to cover the followup commits (Cin-5 per-server global removal + Gap 1 shared types lift) so the changelog matches the actual branch state. |
||
|
Broque Thomas
|
d3f8a06d7a | WHATS_NEW entry for media server engine review pass | ||
|
Broque Thomas
|
2c0a0da9ea |
Address Copilot doc-drift review
Four stale doc/comment references caught by Copilot's pass:
- core/download_plugins/base.py: TYPE_CHECKING comment said the
shared dataclasses lived in core.soulseek_client. They were moved
to core.download_plugins.types in this PR. Comment updated.
- core/qobuz_client.py: reload_credentials docstring still referenced
soulseek_client.client('qobuz') after the global rename to
download_orchestrator. Updated to download_orchestrator.client(...).
- webui/static/helper.js: the older WHATS_NEW entries for the plugin
contract + engine refactor still claimed backward-compat
self.<source> attributes were preserved. Followup commits in the
same PR removed them. Each entry now flags the followup explicitly
and points at the "Drop Backward-Compat Per-Source Attrs" entry
above it so the changelog is internally consistent.
- docs/download-engine-refactor-plan.md: Compatibility commitments
section listed orchestrator.<source> attribute preservation as a
guarantee. Cin's review pass removed those attrs (and renamed the
global handle from soulseek_client to download_orchestrator) — both
are breaking changes for in-tree callers (which were migrated) and
in-flight branches (which will need to update). Section rewritten
to document the actual outcome.
|
||
|
Broque Thomas
|
2aff3dc210 |
Filter SoundCloud previews at every entry point + fix hybrid fallback regression
The earlier validation-only filter only ran in the auto-search scoring path. SoundCloud preview snippets still leaked through: - The candidate-review modal cached raw search results (pre-validation), so previews were visible and clickable for manual retry — and the manual-pick download path bypassed validation entirely, downloading the preview anyway. - The not-found raw-results cache stored unfiltered top-20s. Lift the preview filter into a reusable filter_soundcloud_previews() helper and apply it at every entry point: validation scoring (still), modal-cache fallback when validation drops everything, and the not-found raw-results path. Previews now never reach the cache, the matcher, or the manual-pick UI. Drops candidates < 35s or below half the expected duration, gated on expected > 60s so genuine short tracks still pass. 7 new unit tests pin the helper. Also fixed a silent regression in core/downloads/task_worker.py's hybrid-fallback path. Cin-5 dropped the per-source attrs from the orchestrator (orch.soulseek, orch.youtube, etc.), but the fallback loop still resolved sources via getattr(orch, '<src>', None) — every lookup silently returned None, so remaining_sources came back empty and the fallback never ran. Now uses orch.client(name) like the rest of the codebase. Updated the test fake to expose client() too — the old test was passing because the loop was effectively dead. |
||
|
Broque Thomas
|
563204ceae |
Drop SoundCloud preview snippets before scoring
SoundCloud serves a ~30s preview clip for tracks gated behind Go+ or login (extremely common for major-label uploads — what's actually on SoundCloud is bootlegs, fan reuploads, type beats, and these previews). yt-dlp accepts the preview as the download payload, the post-download integrity check catches the duration mismatch and quarantines the file, but the user only sees "all candidates failed" with no obvious explanation. Filter at validation time when we know expected_duration: drop SoundCloud candidates whose duration is below half the expected length OR within ~5s of the 30s preview boundary, gated on expected being non-trivially long (>60s) so genuinely short tracks still pass through. |
||
|
Broque Thomas
|
d17365296a |
Lift shared download dataclasses + boot via singleton factory
Two architectural cleanups on top of the download engine refactor. (1) Shared dataclasses move to neutral plugin package. TrackResult, AlbumResult, DownloadStatus, SearchResult lived in core/soulseek_client.py for historical reasons — every other plugin imported them from the soulseek module just to satisfy the contract, coupling 8 clients to a sibling source for type imports only. Moved them to the new core/download_plugins/types.py module and updated all 14 import sites across the deezer/hifi/lidarr/qobuz/soundcloud/tidal/ youtube clients, the engine, matching engine, redownload helper, and tests. Clean break, no backward-compat re-export. (2) web_server.py boots the orchestrator via the singleton factory. After construction it now calls set_download_orchestrator(...) so get_download_orchestrator() returns the same instance the global handle points at instead of lazily building a separate orchestrator. Matches the get_metadata_engine() pattern. |
||
|
Broque Thomas
|
61ba3a15de |
Cin-6: Rename soulseek_client global → download_orchestrator
The global handle in web_server.py was named soulseek_client for historical reasons but the type has long been DownloadOrchestrator, not SoulseekClient. Renamed the global plus every parameter/attribute that carried the legacy name. - web_server.py: global var renamed; all 99 references updated. - api/, core/downloads/*, core/search/*, core/streaming/*, services/sync_service.py: parameter names, dataclass fields, and init() arg names renamed. - Test fixtures (CandidatesDeps, MasterDeps, SearchDeps, etc.) and the _build_deps helpers updated accordingly. The core.soulseek_client module path and SoulseekClient class name (the actual soulseek-only client) are unchanged — only the orchestrator handle renamed. Module imports of TrackResult/AlbumResult/DownloadStatus from core.soulseek_client preserved. |
||
|
Broque Thomas
|
7519c3d50c |
Cin-5: Drop per-source attrs from orchestrator
Removed the eight backward-compat attribute aliases on the orchestrator
(soulseek, youtube, tidal, qobuz, hifi, deezer_dl, lidarr, soundcloud).
External callers and the orchestrator's own internals now reach clients
through the generic alias-aware client(name) accessor.
- core/downloads/{master,monitor,validation}.py: migrated to client().
Monitor's per-source aggregation loop replaced with a single
engine.get_all_downloads() call.
- core/search/{orchestrator,stream}.py: migrated; stream.py drops the
hand-built mode-to-client dict.
- web_server.py: migrated /api/deezer/arl-* + tidal client lookup.
- core/download_orchestrator.py: internal self.soulseek /
self.deezer_dl reaches now route through self.client(); attr
assignments dropped from __init__; module docstring updated.
- Test fakes (_FakeSoulseek, _FakeSoulseekWithYT) expose client(name)
instead of stuffing per-source attributes.
- Conformance test re-pinned to the client() accessor contract.
|
||
|
Broque Thomas
|
d0eac87601 |
Cin review: alias resolution, atomic terminal write, generic accessors
Three correctness fixes from kettui's PR review plus the web_server
migration to generic accessors.
- Engine alias map: register_plugin accepts aliases tuple; get_plugin
+ cancel_download resolve through it. Fixes deezer_dl cancels
silently routing to soulseek.
- Orchestrator hybrid_order normalization: _resolve_source_chain
routes raw config names through registry.get_spec() so legacy
deezer_dl entries don't drop deezer from hybrid mode.
- Atomic update_record_unless_state on the engine: holds state_lock
across the check + write. Both _mark_terminal AND the success path
use it now so a Cancelled state set mid-impl can't be clobbered.
- web_server.py: 30 soulseek_client.<source> reaches migrated to
client("<source>"); shutdown-check setup migrated to generic
registry iteration; 4 hifi reload sites use reload_instances('hifi').
- 18 new tests pin every fix.
|
||
|
Broque Thomas
|
650327ba18 |
Phase E: Add WHATS_NEW entry for media server engine refactor
Internal-track entry covering the media server engine + contract + the honest-scope note explaining why we lifted the 4 truly-uniform is_connected dispatches and left the deep server-specific dispatches explicit (each does fundamentally different work per server, so lifting would just move per-server branches into engine helper methods). |
||
|
Broque Thomas
|
95835b05ee |
H: Add WHATS_NEW entry for download engine refactor
Internal-track entry covering the engine package, background download worker, state lift, rate-limit policy declarations, and hybrid fallback chain. Mentions the ~700 LOC reduction + 85 new tests + zero behavior change. |
||
|
Broque Thomas
|
f9b763587d |
Add plugin conformance tests + WHATS_NEW entry
19 parametrized tests pin every registered plugin class's structural conformance to DownloadSourcePlugin: every required method present + async-ness matches the protocol. Drift in any source fails at the test boundary instead of at runtime against a live download. Class-level checks (not instance-level) — instantiating real clients in fixtures pollutes module state via tidalapi etc. imports and breaks downstream tests. |
||
|
Broque Thomas
|
749a772ff5 |
Findings tab: auto-switch to all-status when 0 pending exist
Companion to the badge count fix. When the findings tab opens with the default "pending" filter and returns 0 rows but other statuses (resolved/dismissed/auto-fixed) do have rows, the filter auto-switches to "All Status" and a small notice explains the switch. Stops the empty "all clear" state from masking carry-over findings from prior scans. |
||
|
Broque Thomas
|
cf5461f2f1 |
Fix: maintenance findings badge inflated when scan dedup-skipped
`_create_finding` silently dedup-skipped re-discovered issues but the caller incremented `findings_created` regardless. So a re-scan that found the same issues as a prior scan reported 364 findings in the badge while 0 NEW pending rows hit the db, leaving the findings tab empty. `_create_finding` now returns bool (True on insert, False on dedup-skip / db error). All 16 repair jobs updated to only increment `findings_created` on True. Added `findings_skipped_dedup` counter surfaced in scan log: "Done: X scanned, 0 fixed, 0 findings (363 already existed), 0 errors". Also fixed a missing `job_id` kwarg in album_tag_consistency that was silently breaking finding creation for that scan. |
||
|
Broque Thomas
|
77c54ab7a7 |
Migrate discography + quality scanner to typed Album path
Three more album-shape consumers now route through Album.from_<source>_dict() when caller passes a known source: - _build_discography_release_dict (artist discography cards) - _build_artist_detail_release_card (artist detail release cards) - _normalize_track_album (quality scanner result normalization) Legacy duck-typing stays as fallback for unknown source, non-dict input, or converter errors. Pure additive — existing callers without source kwarg unchanged. |
||
|
Broque Thomas
|
967c7f7c0a |
Migrate album-info builders to typed Album path
Steps 2+3 of typed metadata migration. Two album-info builders now route through Album.from_<source>_dict() when caller passes a known source: - _build_album_info (album-tracks lookups) - _build_single_import_context_payload (single-track import context) Legacy duck-typing stays as fallback for unknown source, non-dict input, or converter errors. Pure additive — existing callers without source kwarg unchanged. |
||
|
Broque Thomas
|
529486a2d1 |
Foundation: typed Album/Track/Artist + per-provider converters
New core/metadata/types.py with canonical dataclasses + classmethod converters for spotify/itunes/deezer/discogs/musicbrainz/hydrabase. Each converter is the single place that knows that provider's wire shape — addresses the duck-typing pattern Cin flagged. Pure additive: no consumer code changed. Follow-up PRs migrate consumers one at a time. Migration plan at docs/metadata-types-migration.md. Tests: 32 cases pin per-provider semantics + cross-provider invariants. Also stabilized a flaky discogs test that depended on local config state. |
||
|
Broque Thomas
|
4b23bee4a9 |
Add Discogs collection as a Your Albums source
Discord request: pull user's Discogs collection into the Your Albums
section on Discover, similar to how Spotify Liked Albums works.
Implementation extends the existing 3-source pipeline (Spotify /
Tidal / Deezer) to a 4-source pipeline with click-context dispatch —
Discogs-only albums open with rich Discogs release detail (vinyl/CD
format, year, label, country, tracklist). Mirrors the per-source
dispatch pattern from enhanced/global search.
Discogs client (`core/discogs_client.py`):
- New `get_authenticated_username()` resolves the username for the
configured personal token via Discogs's `/oauth/identity` endpoint.
Cached on the instance so subsequent collection page-fetches don't
re-hit it.
- New `get_user_collection(username=None, folder_id=0, per_page=100,
max_pages=50)` walks all pages of `/users/{username}/collection/
folders/{folder_id}/releases`. Returns normalized dicts ready for
upsert_liked_album. folder_id=0 = Discogs's "All" folder.
Pagination cap of max_pages*per_page = 5000 releases — bounds
runtime on heavy collections.
- New `get_release(release_id)` thin wrapper for `/releases/{id}` —
returns the raw API response so the album-detail endpoint can
render rich context.
- Both methods defensive: missing token → empty list, malformed
responses → skipped, falsy ids → None. Disambiguation suffix
stripping (`Madonna (3)` → `Madonna`) so Discogs artist names
match what Spotify/Tidal/Deezer use.
Schema (`database/music_database.py`):
- New `discogs_release_id TEXT` column on `liked_albums_pool`.
Migration uses the established `try SELECT, except ALTER TABLE`
pattern. Idempotent; safe on existing installs.
- Added the column to the canonical CREATE TABLE for fresh installs.
- `upsert_liked_album` extended with `'discogs': 'discogs_release_id'`
in BOTH the INSERT and UPDATE id-column maps so Discogs source_id
routes to the new column. INSERT statement column count + value
count updated together.
Backend (`web_server.py`):
- `/api/discover/your-albums/sources` — adds Discogs to the
`connected` list when `discogs.token` config is set.
- `_fetch_liked_albums` — new branch for Discogs. Lazy-imports
DiscogsClient, respects the `enabled_sources` config, walks the
collection, upserts each release. Same try/except shape as the
existing source branches.
- `/api/discover/album/<source>/<album_id>` — new `discogs` branch
fetches the release via DiscogsClient.get_release, normalizes the
Discogs tracklist format, parses Discogs's `MM:SS`/`HH:MM:SS`
duration strings to milliseconds, returns the same response shape
as the Spotify/Deezer/iTunes branches.
Frontend (`webui/static/discover.js`):
- `openYourAlbumsSourcesModal` — adds Discogs to `sourceInfo` with
the vinyl emoji icon. Existing toggle/save plumbing handles it.
- `openYourAlbumDownload` — restructured the per-source dispatch:
builds an ordered list of (source, id) tuples, tries each in turn,
breaks on the first successful response. Pure-Discogs albums go
straight to the Discogs detail endpoint → modal opens with Discogs
context. Multi-source albums prefer Spotify/Deezer first since
their tracklists carry proper streaming IDs ready for download.
Tests: `tests/test_discogs_collection_source.py` — 12 cases:
- get_user_collection: empty without token, normalizes response
shape, strips disambiguation suffix, handles missing year, skips
malformed releases, paginates correctly, caps at max_pages,
uses explicit username when provided.
- get_release: passes id through to /releases/{id}, returns None
for invalid ids without API call.
- liked_albums_pool: discogs_release_id round-trips through upsert
+ get; multi-source dedup carries both Spotify and Discogs IDs
on the same row.
Verified: full suite 1825 pass (12 new), ruff clean, smoke test
populating + reading the discogs_release_id column round-trips
correctly via the real DB.
WHATS_NEW entry under '2.4.2' dev cycle.
|
||
|
Broque Thomas
|
e84d187e76 |
Drop redundant standalone "Your Spotify Library" section on Discover
Discover page used to show two near-identical sections: - "Your Albums" — cross-source aggregator across Spotify / Deezer / etc with a gear button to configure sources, search, status filter, sort options, and a download-missing action. - "Your Spotify Library" — Spotify-only with the same grid UI, same refresh / download-missing buttons, same filter / sort controls. The Spotify-only section was a strict subset of what Your Albums already covers (Spotify is one of the configurable sources). User flagged the redundancy when scoping the upcoming Discogs integration and asked for the duplicate to be removed. Removal scope: - `webui/index.html` — drop the `#spotify-library-section` block (42 lines). - `webui/static/discover.js` — drop the dead JS (~335 lines): state vars `spotifyLibraryAlbums` / `spotifyLibraryPage` / etc, all the loaders / renderers / pagination / click handlers, and the `loadSpotifyLibrarySection()` call in `loadDiscoverPage`'s Promise.all. - `webui/static/helper.js` — drop the helper annotation entry at `#spotify-library-section` and the matching guided-tour entry. Backend untouched. The Spotify saved-albums cache (`spotify_library_albums` table + watchlist_scanner upsert/cleanup + `/api/discover/spotify-library` endpoint + the DAO methods) is shared infrastructure that Your Albums reads from when Spotify is one of its configured sources. Removing the UI section just removes the duplicate surface — Spotify saved albums still appear in Your Albums via the existing source dispatch. CSS class names (`.spotify-library-grid`, `.spotify-library-search`, `.spotify-library-pagination`) intentionally remain on the surviving Your Albums elements — they share the same visual styling and renaming would be churn for no benefit. Verified: full suite 1813 pass (no new tests — pure UI/dead-code removal). Backend endpoint behavior unchanged. WHATS_NEW entry under '2.4.2' dev cycle. |
||
|
Broque Thomas
|
2ab460f5c4 |
Add Library Disk Usage card to System Statistics
Discord request (Samuel [KC]): show how much disk space the library
takes on the Stats page. Implementation piggybacks on the existing
deep scan — Plex/Jellyfin/Navidrome all return file size in their
track API responses, so we read it during the deep scan and store
it on the tracks row. Aggregation is then a single SQL query — no
filesystem walk, no extra I/O during the scan, no separate stat
job. SoulSync standalone gets size from os.path.getsize at insert
time (different code path; the file is local when we write the row).
Schema (`database/music_database.py`):
- New `file_size INTEGER` column on `tracks`. Migration uses the
established `try SELECT, except ALTER TABLE ADD COLUMN` pattern.
Idempotent; safe on existing installs. NULL on legacy rows so
they don't contribute to totals until next deep scan refreshes.
- Added the column to the canonical CREATE TABLE so fresh installs
get it without going through the migration path.
Track-object plumbing:
- `core/jellyfin_client.py` — JellyfinTrack reads MediaSources[0].Size
alongside existing Bitrate read. None when 0 / missing.
- `core/navidrome_client.py` — NavidromeTrack reads `size` from
the Subsonic song object (int coercion + None on parse fail).
- `core/soulsync_client.py` — SoulSyncTrack does os.path.getsize
(only "server" where size has to come from disk).
- Plex needs no client-side change: track.media[0].parts[0].size
is read directly inside insert_or_update_media_track.
Persistence — TWO separate insert paths:
(a) `database/music_database.py:insert_or_update_media_track` —
Plex/Jellyfin/Navidrome flows. Reads file_size from Plex's
MediaPart OR `track_obj.file_size` wrapper attribute (defensive
Plex-attr-not-present check + > 0 type guard).
INSERT writes the new column.
UPDATE uses COALESCE(?, file_size) so a None from the server
on a re-sync (rare Jellyfin Size omission) doesn't blank an
existing value. Pinned via test.
(b) `core/imports/side_effects.py:record_soulsync_library_entry` —
SoulSync standalone flow. Completely separate code path: the
standalone deep scan moves files to staging for auto-import
rather than calling insert_or_update_media_track. After the
auto-import processes them, side_effects writes the tracks row
directly. Reads file_size via os.path.getsize(final_path) at
insert time (file is local) and includes it in the INSERT
column list. SoulSync only does INSERT-if-not-exists (no
UPDATE path), so no COALESCE concern.
Aggregator (`database/music_database.py:get_library_disk_usage`):
- SELECT COALESCE(SUM(file_size), 0), COUNT(file_size),
COUNT(*) - COUNT(file_size) for the totals.
- Per-format breakdown done in Python via os.path.splitext over
(file_path, file_size) rows — sidesteps SQLite's first-vs-last-dot
ambiguity for paths like /music/Kendrick/M.A.A.D City/01.flac.
- Defensive: skips empty paths, paths without extension, and
implausibly long extensions (>6 chars). Returns the full
empty-shape dict (NOT a partial / undefined) when the column
doesn't exist or queries fail, so the UI's `if (!data.has_data)`
branch handles fresh installs cleanly.
API + UI:
- `core/stats/queries.py` — thin pass-through get_library_disk_usage
matching the existing query-helper convention.
- `web_server.py` — new /api/stats/library-disk-usage endpoint
mirroring the /api/stats/db-storage pattern.
- `webui/index.html` — new card in System Statistics above the
Database Storage card.
- `webui/static/stats-automations.js` — _loadLibraryDiskUsage +
_renderLibraryDiskUsage. Empty state: "Run a Deep Scan to
populate (X tracks pending)". Partial: "X measured (+Y pending)".
Full: total + format bars proportional to the largest format.
- `webui/static/style.css` — .stats-disk-* styled to match the
Database Storage card.
Backward compatibility:
- Migration is additive; existing rows get NULL file_size; the
empty-shape return from the aggregator means the UI renders
cleanly without errors before any deep scan runs.
- Old installs upgrading will see "Run a Deep Scan to populate
(N tracks pending)". Running their next deep scan fills sizes —
the existing scan flow doesn't need any changes, just consumes
the new track-wrapper attribute.
Tests:
- `tests/test_library_disk_usage.py` — 13 cases covering schema
migration, NULL defaults on legacy inserts, fresh-install empty
shape, summing with mixed NULL/known sizes, per-format breakdown,
mixed-case extensions, paths with album-name dots, missing
extensions, empty file_path, implausibly long extensions,
JellyfinTrack.file_size persistence via insert_or_update_media_track,
COALESCE preservation on null re-sync.
- `tests/imports/test_import_side_effects.py` — extended the
existing record_soulsync_library_entry test to assert
track_row['file_size'] == os.path.getsize(final_path), pinning
the SoulSync-standalone path. Test fixture's tracks schema also
updated to include the file_size column.
Verified: full suite 1813 pass (13 new, 1 existing-test extension),
ruff clean, smoke test populating + reading the column round-trips
correctly.
WHATS_NEW entry under '2.4.2' dev cycle.
|
||
|
Broque Thomas
|
776d195f71 |
Fix: ReplayGain wrote same +52 dB gain to every track
User report: every downloaded track in an album came out with
``replaygain_track_gain: +52.00 dB`` regardless of actual loudness.
Root cause: the parser at ``core/replaygain.py:79`` used
``re.search('I:\s+...')`` which returns the FIRST match. ffmpeg's
ebur128 filter emits ``I:`` per measurement window (running partial
integrated loudness) AND in a final Summary block. The first
per-window reading is at t=0.5s — almost always ~-70 LUFS because
nearly every track starts with silence / encoder padding. So:
gain = RG2_reference - lufs = -18 - (-70) = +52.00 dB
…on EVERY track. Same regex pattern, same first per-window match,
same +52 dB written to every file's REPLAYGAIN_TRACK_GAIN tag.
Verified by running ffmpeg ebur128 against a real generated FLAC
and inspecting the stderr output — first per-window line at t=0.5s
shows ``I: -70.0 LUFS`` (silent intro), and the Summary block at
the end shows the real integrated value (e.g. ``I: -27.8 LUFS``
for the test sine wave). Old code captured the -70.0 reading.
Fix: anchor LUFS parsing to the ``Summary:`` block via
``stderr.rfind('Summary:')``. The Summary block is always emitted
last and contains the authoritative final integrated loudness.
Peak parsing already worked correctly (per-window output uses
``TPK:``/``FTPK:`` labels; only the Summary uses ``Peak:``), but
applied the same Summary anchor for consistency.
Defensive fallback: if no Summary block is present (truncated
output / unusual ffmpeg version), use the LAST per-window reading
instead of the first. Still better than the buggy first-window
behavior.
Smoke verified end-to-end: a freshly-generated FLAC of a -24 dBFS
sine wave now reports LUFS=-27.80, gain=+9.80 dB (correct, was
+52.00 before fix).
Tests: ``tests/test_replaygain_summary_parse.py`` — 7 cases pinning
the parser behavior with realistic ffmpeg ebur128 stderr samples:
- Summary value parsed correctly even when first per-window is -70
- Resulting gain is realistic (NOT +52)
- Two tracks with same first per-window but different summaries get
different LUFS (regression assertion for "all tracks same gain")
- Per-window reading higher than Summary doesn't leak through
- Fallback to last per-window when Summary absent
- Clean RuntimeError raised when no LUFS values anywhere
- Peak still correctly anchored to Summary
Verified: full suite 1800 pass (7 new), ruff clean.
WHATS_NEW entry under '2.4.2' dev cycle.
|
||
|
Broque Thomas
|
04a14f7e96 |
Fix: tasks showed Completed when file was quarantined
User caught downloading Kendrick Mr. Morale: three tracks (Rich Interlude, Savior Interlude, Savior) showed ✅ Completed in the modal but were missing on disk. Log forensics revealed two layered bugs. Bug 1 — Verification wrapper assumed success on quarantined files (`core/imports/pipeline.py`): The outer `post_process_matched_download_with_verification` had a fallback at the "no `_final_processed_path` in context" branch that marked the task completed and notified `success=True`. The inner post-processor sets `_final_processed_path` only when the file actually reaches its destination. Integrity-rejected files (`_integrity_failure_msg` set) and race-guard-failed files (`_race_guard_failed` set) get quarantined or skipped without ever setting `_final_processed_path`, so they fell straight into the "assume success" branch. Confirmed in user's log: No _final_processed_path in context for task d5b88b84-... — cannot verify, assuming success That line fired for the same task right after the integrity check quarantined the source file. Result: ✅ Completed in UI, file in quarantine, never delivered. Fix: explicit checks for `_integrity_failure_msg` and `_race_guard_failed` markers BEFORE the assume-success fallback. Either marker set → task status='failed' with descriptive error_message + `_notify_download_completed(success=False)`. The pre-existing assume-success behavior preserved when no failure markers are set (some legitimate flows complete without setting `_final_processed_path`). Bug 2 — AcoustID skip-logic too lenient (`core/acoustid_verification.py`): The "language/script" exemption was: if best_score >= 0.95 and (title_sim >= 0.55 or artist_sim >= ARTIST_MATCH_THRESHOLD): The OR-clause fired for English-vs-English titles by the same artist that share NO actual content. Confirmed in user's log: requested "Rich (Interlude)" by Kendrick Lamar, AcoustID identified the audio as "R.O.T.C. (interlude)" by Kendrick Lamar (a totally different song from his 2010 mixtape) — same artist scored ≥ARTIST threshold, shared word "interlude" pushed title_sim above 0.55, skip fired. Verification returned SKIP instead of FAIL, the wrong file was accepted as the answer for three different track requests. Fix: skip now requires positive evidence the mismatch is a real language/script case: (a) Non-ASCII chars present in either title AND artist matches strongly → real transliteration case (kanji ↔ romaji etc) (b) BOTH title_sim >= 0.80 AND artist_sim >= ARTIST threshold → minor punctuation/casing differences English-vs-English with very different titles by the same artist no longer skipped — verification correctly returns FAIL, the wrong file gets quarantined, the new wrapper logic above marks the task failed. Tests: - `tests/test_integrity_failure_marks_task_failed.py` — 4 cases pinning the wrapper-level state machine: integrity marker → failed, race-guard marker → failed, no markers → still assumes success (legacy path preserved), integrity-failure-takes-priority over missing-final-path fallback. - `tests/test_acoustid_skip_logic.py` — 7 cases pinning the skip exemption: user's R.O.T.C-vs-Rich case → FAIL (regression test), Savior-vs-R.O.T.C → FAIL (same bug surface), Japanese kanji → romaji → SKIP (real language case still works), MAAD vs M.A.A.D → PASS or SKIP (punctuation tolerance), low fingerprint score → never skipped, high score but artist mismatch → no longer skipped, Crown vs Crown of Thorns → no longer skipped. Verified: full suite 1793 pass (11 new), ruff clean. WHATS_NEW entry under '2.4.2' dev cycle. |
||
|
Broque Thomas
|
4b15fe0b75 |
Fix album MBID inconsistency: detector + persistent release-MBID cache
Discord report (Samuel [KC]): tracks of the same album sometimes carry different MUSICBRAINZ_ALBUMID tags, which causes Navidrome (and other media servers grouping by album MBID) to split the album into multiple entries. Two-part fix — one for existing libraries, one for the root cause that lets new imports drift. Part 1 — Detector + fix action (catches existing dissenters): `core/repair_jobs/mbid_mismatch_detector.py`: - New helpers: `_read_album_mbid_from_file` and `_write_album_mbid_to_file` use the Picard-standard tag conventions (`TXXX:MusicBrainz Album Id` for MP3, `MUSICBRAINZ_ALBUMID` for FLAC/OGG, `----:com.apple.iTunes:MusicBrainz Album Id` for MP4). - New scan phase `_scan_album_mbid_consistency` runs after the existing track-MBID scan: groups tracks by DB `album_id`, reads each track's embedded album MBID, finds the consensus (most-common) MBID via `Counter`, flags dissenters. Tracks without an album MBID at all are skipped (they don't break Navidrome — only an explicit MBID disagreement does). Albums where MBIDs are perfectly tied (no clear consensus) are skipped too — surface as a manual decision instead of fixing toward a 1/N tie. - New finding type `album_mbid_mismatch` carries `consensus_mbid`, `wrong_mbid`, `consensus_count`, `total_tracks_with_mbid`, and a human-readable reason string. `core/repair_worker.py`: - Added `'album_mbid_mismatch': self._fix_album_mbid_mismatch` to the fix dispatch dict and to the `fixable_types` tuple so auto-fix + bulk-fix paths pick it up. - New `_fix_album_mbid_mismatch` method reads `consensus_mbid` from finding details, resolves the dissenter's file path via the shared library resolver, calls `_write_album_mbid_to_file` to rewrite the tag in place. Doesn't touch the album's other tracks (they're already in agreement). Part 2 — Root cause fix (prevents new SoulSync imports from drifting): The original in-memory `mb_release_cache` in `core/metadata/source.py` maps `(normalized_album, artist) -> release_mbid` so per-track enrichment of the same album hits the cache and writes the same MUSICBRAINZ_ALBUMID to every track. That cache is bounded (4096 entries) and in-process — so cache eviction (when other albums are processed in between) and server restart can BOTH cause inconsistency. Per-track album-name variation (e.g. some tracks tagged `"Album"`, others tagged `"Album (Deluxe)"`) and per-track artist variation (features) make it worse. `core/metadata/album_mbid_cache.py` (new module): - DB-backed `lookup(normalized_album, artist) -> release_mbid` and `record(...)` functions. Same key shape as the in-memory cache. - Strict additive design: every public function is wrapped in try/except and degrades to None / no-op on ANY database error. The existing in-memory cache + MusicBrainz lookup remains the authoritative fallback. If this module breaks, downloads continue exactly as they would today. `database/music_database.py`: - New `mb_album_release_cache` table with composite primary key `(normalized_album_key, artist_key)`. Reverse-lookup index on `release_mbid` for future debug tooling. Created via the existing `CREATE TABLE IF NOT EXISTS` migration pattern — idempotent, no schema version bump needed. `core/metadata/source.py`: - Surgical change inside the existing `embed_source_ids` in-memory-cache-miss branch: BEFORE calling MusicBrainz, consult the persistent cache. If a previous SoulSync run already resolved this album's release MBID, reuse it. After a successful MB lookup, store in BOTH caches. Both calls wrapped in defensive try/except so any failure falls through to existing logic. Tests: - `tests/metadata/test_album_mbid_cache.py` — 16 cache tests: round-trip, idempotent re-record, overwrite semantics, clear_all, album+artist independence (no Greatest Hits collisions), defensive None-on-empty-input, graceful degradation when the DB is unavailable / connection raises / commit fails, schema sanity (table + index exist after init). - `tests/test_album_mbid_consistency.py` — 13 detector tests: tag read/write round-trip on real FLAC files, Picard-standard tag descriptors, defensive paths (unreadable file, empty input), detector behavior (agreement → no flags, lone dissenter → flag, ties → no flag, single-track albums → skipped, no-MBID tracks → skipped, unresolvable file paths → skipped). - `tests/metadata/test_metadata_enrichment.py` — added autouse fixture monkeypatching the persistent cache to no-op for tests in this file. The existing tests pin per-call MB counts and in-memory cache state; without the fixture, persistent rows from earlier tests would bypass the MB call. Persistent layer has its own dedicated tests. Verified: 1782 tests pass (29 new), ruff clean, smoke test confirms end-to-end cache round-trip works. WHATS_NEW entry under '2.4.2' dev cycle. |
||
|
Broque Thomas
|
e577f3cf1f |
Fix three Lidarr bugs that prevented it from being a real download source
Investigation surfaced that Lidarr was wired into the orchestrator but
the actual download flow had blockers:
1. **Wrong file misfiled.** Lidarr grabs whole albums; SoulSync's
matched-context post-processing wants the SPECIFIC track the user
requested. Old code copied every track in the album and reported
`imported_files[0]` as `file_path` — almost always pointing to
track 1, not the user's actual track. Post-processing then tagged
track 1 with the requested track's metadata. Misfiling on every
real download.
Fix: parse the wanted track title out of the dispatch display name
(which `_search_sync` already builds as
`f"{artist} - {album} - {track_title}"`), look it up against
Lidarr's `track` API, resolve the matching `trackFileId` to a path,
and copy ONLY that file. Punctuation-tolerant fuzzy match handles
the common "m.A.A.d city" vs "maad city" case. Album-level
dispatches (no track in the display) preserve the old first-file
fallback so existing album-grab UX is unchanged.
2. **Hardcoded `metadataProfileId=1`.** Required by Lidarr's
artist-add API. On installs where the user deleted/recreated
metadata profiles, that id no longer exists and the call fails
with HTTP 400 — which silently breaks every download flow that
needs to add an artist. Real-world Lidarr installs do this all
the time.
Fix: `_get_metadata_profile_id()` calls Lidarr's `metadataprofile`
API and returns the first available id. Falls back to 1 only when
the API call fails entirely (preserves previous behavior so this
change can't make things worse).
3. **Polling never broke the outer loop on completion.** The inner
`for item in queue['records']` had `break` statements at status
transitions, but those only escaped the queue iteration — the
outer `for poll in range(max_polls)` kept spinning until the
600-poll timeout even after the album was clearly imported.
`for/else` semantics didn't apply because completion was detected
inside the inner loop, not by it running to exhaustion.
Fix: replaced with an explicit `download_complete` flag set when
`album/{id}` reports `trackFileCount > 0` (the authoritative
completion signal — works even when the queue record disappeared
between polls). Outer loop breaks immediately once the flag flips.
Helper functions added: `_extract_wanted_track_title` (staticmethod,
splits the display name; >=3 parts → track dispatch, 2 parts → album
dispatch), `_normalize_for_match` (lowercase + strip punctuation +
collapse whitespace for fuzzy compare), `_title_similarity` (cheap
score: equal=1.0, substring=0.85, token-overlap-ratio otherwise),
`_pick_track_file_for_wanted` (orchestrates the API calls).
Settings tooltip updated to be honest about Lidarr's natural shape:
album-grabber, no-op for playlist sync, hybrid mode falls through to
other sources for track searches. Sets correct expectations.
Tests: `tests/test_lidarr_download_client.py` — 21 isolated tests
covering pure helpers (title extraction, normalization, similarity)
and the file-picker integration paths (matching path, punctuation
tolerance, below-threshold fallback, missing trackFileId, missing
file on disk, API failures, malformed responses). No live Lidarr
needed — `_api_get` mocked at the client boundary.
Isolation: ONLY touches `core/lidarr_download_client.py`, the Lidarr
settings tooltip in `webui/index.html`, the Lidarr WHATS_NEW entry
in `webui/static/helper.js`, and the new test file. No changes to
the orchestrator, other download clients, the import pipeline,
side_effects, web_server.py, settings.js, or any shared validation /
monitor / task_worker code. Other download sources are not affected
in any way.
Verified: 1753 tests pass (21 new), ruff clean.
|
||
|
Broque Thomas
|
75fe04907f |
Wire SoundCloud as a first-class download source
Plug the previously-built SoundcloudClient (PR #478, the build-and-verify phase) into every place a download source needs to appear. Follows the same wiring contract as Tidal/Qobuz/HiFi/Deezer/Lidarr — orchestrator routing, hybrid-mode picker, search dispatch, queue/cancel/clear, provenance + library history, sidebar source label, settings UI all work plug-and-play. Backend wiring: - `core/download_orchestrator.py` — import SoundcloudClient, _safe_init it at startup, add to _client() lookup, get_source_status(), check_connection's sources_to_check default, search source_names map, search_and_download_best _streaming_sources tuple, download source_map + source_names, and every iteration loop in reload_settings download-path-update / get_all_downloads / get_download_status / cancel_download (route + iterate) / clear_all_completed_downloads / cancel_all_downloads. - `core/downloads/monitor.py` — added SoundCloud to the per-client loop that fetches active downloads outside the orchestrator (uses getattr fallback for older soulseek_client snapshots). - `core/downloads/task_worker.py` — added SoundCloud (and Lidarr, which was missing too — bonus fix) to source_clients dict for hybrid fallback dispatch. - `core/downloads/validation.py` — added 'soundcloud' to _streaming_sources so SoundCloud results go through the matching engine validation path instead of the Soulseek quality-filter path. - `core/imports/side_effects.py` — three call sites: source_map for download_source label written to library_history, streaming-source guard for the `||`-encoded stream_id parsing, and source_service map for provenance recording. All three now include 'soundcloud'. - `web_server.py` — five streaming-source detection tuples updated. New `/api/soundcloud/status` endpoint returns {available, configured, reachable} mirroring the Deezer/HiFi status-endpoint pattern; reachability runs a real cheap yt-dlp search so the settings Test Connection button gives a meaningful pass/fail signal. - `config/settings.py` — added empty `soundcloud_download` defaults block so future tier-2 OAuth (SoundCloud Go+ session) doesn't have to migrate existing configs. Frontend: - `webui/index.html` — new `<option value="soundcloud">` in the download-source-mode dropdown, SoundCloud added to both hidden legacy hybrid-source selects, new settings container with info text + Test Connection button. - `webui/static/settings.js` — HYBRID_SOURCES entry (with the SoundCloud cloud SVG icon), _hybridSourceEnabled default, updateDownloadSourceUI container display, allSources for legacy hybrid picker, testSoundcloudConnection function (hits the new status endpoint, color-codes the result), saveSettings soundcloud_download empty block. - `webui/static/shared-helpers.js` — sidebar source-name map includes SoundCloud + Lidarr (Lidarr was also missing, bonus fix). - `webui/static/helper.js` — WHATS_NEW entry under '2.4.2' dev cycle describing the user-visible change in the chill terse voice. Tests: - `tests/test_download_orchestrator_soundcloud.py` — 14 integration tests verifying the wiring: client constructed at startup, _client lookup resolves 'soundcloud', get_source_status includes it, download dispatcher routes username='soundcloud' to the SoundCloud client (and unknown usernames still fall back to Soulseek), hybrid search iterates SoundCloud when in order and skips it cleanly when unconfigured, get_all_downloads / get_download_status / cancel / clear walk SoundCloud, soundcloud-only mode dispatches only to SoundCloud, _streaming_sources tuple in validation includes 'soundcloud'. - `tests/downloads/test_download_orchestrator.py` — added `soundcloud` to the test fixture's _build_orchestrator helper so the new orchestrator attribute doesn't AttributeError in pre- existing tests that bypass __init__. Verified: - Full suite green (1728 passed, 2 deselected for soundcloud_live) - Ruff clean - Live SoundCloud-only mode search returns 25 SoundCloud tracks for "kendrick lamar luther" in <2s, returning properly-shaped TrackResult objects with username='soundcloud' and dispatch-key filename ready for the download path. Out of scope (intentional deferrals): - SoundCloud Go+ OAuth tier (256 kbps AAC) — anonymous-only for now. Adding auth later is a settings-page extension, no orchestrator changes needed. - Album/playlist support — SoundCloud has playlists but they don't map to the album model the rest of SoulSync expects. Singles only. |
||
|
Broque Thomas
|
d8437c87c6 |
Fix Album Completeness Auto-Fill on Docker / shared-library setups (#476)
GitHub issue #476 (gabistek, Docker on Arch host): "Auto-Fill" / "Fix Selected" on the Album Completeness findings page returned "Could not determine album folder from existing tracks" for every album. Reproduces on any setup where the media-server library lives outside the SoulSync transfer/download folders — Docker is the headline case but native installs that point Plex at a NAS via SMB hit it too. Root cause: `core/repair_worker.py:_resolve_file_path` only probed the transfer + download folders. Docker users have their Plex/Jellyfin library bind-mounted at /music (or similar) — neither configured in SoulSync. Every existing track got silently treated as missing, so `album_folder` stayed None and the fix workflow bailed. The same incomplete logic was duplicated four more times in the repair_jobs/ modules, all with the same bug. Album Completeness was just the most user-visible — the same setups were also producing false "missing file" findings from Dead File Cleaner, silent skips in MBID Mismatch Detector, etc. The web server already had the correct logic at `web_server.py:_resolve_library_file_path` (probes transfer + download + Plex-reported library locations + user-configured library.music_paths). The repair workers had never been updated to match. Fix: - New `core/library/path_resolver.py` extracts the union logic into a single shared function `resolve_library_file_path()`. Probes (in order, deduped): explicit transfer/download kwargs, config-derived soulseek.transfer_path/download_path, Plex-reported library locations (when a plex_client is passed), user-configured library.music_paths. Each defensive: malformed config or a flaky Plex client degrades to the dirs that did succeed. - `core/repair_worker.py:_resolve_file_path` becomes a delegating wrapper preserving the legacy signature, with a new `config_manager` kwarg. All 15 in-tree call sites updated to thread `self._config_manager` through. - `core/repair_jobs/dead_file_cleaner.py`, `mbid_mismatch_detector.py`, and `lossy_converter.py` get the same treatment: duplicate function replaced with a thin wrapper, call sites pass `context.config_manager`. - `core/repair_jobs/acoustid_scanner.py` and `unknown_artist_fixer.py` (which used to import from repair_worker) now call the shared resolver directly with `context.config_manager`. Side benefit: every other repair job (Dead File Cleaner, MBID Mismatch Detector, Lossy Converter, AcoustID Scanner, Unknown Artist Fixer) also stops missing files in the media-server library mount. Single fix unblocks five user-visible features. Tests: `tests/library/test_path_resolver.py` — 20 cases covering all four base-dir sources, suffix-walk algorithm, dedup, defensive paths (None plex client, malformed config entries, raising config_manager.get, broken plex attribute access), Docker path translation. Full suite 1677 passed locally. WHATS_NEW entry under '2.4.2' dev cycle. |
||
|
Broque Thomas
|
42f3026eef |
Reject broken downloads before tagging via universal integrity check
Discord report (fresh.dumbledore [VRN]): slskd sometimes ships broken files
(truncated transfers, corrupt FLAC, wrong file substituted on filename match).
They flowed through post-processing and only surfaced later — Plex/Jellyfin
scan failures, dead-air playback, duplicate detector tripping over the wrong
length. By that point the file was already tagged, copied, mirrored to the
media server, and recorded in provenance.
New module `core/imports/file_integrity.py`:
- `check_audio_integrity(path, expected_duration_ms=None) -> IntegrityResult`
- Three tiered checks, cheapest to most expensive:
1. File size sanity (catches 0-byte stubs and stub transfers)
2. Mutagen parse (catches header damage, wrong-format-with-right-extension)
3. Duration agreement vs. metadata source's expected length, ±3s tolerance
(5s for tracks over 10 minutes — long tracks naturally drift more)
- Returns IntegrityResult with `ok`, human-readable `reason`, and per-check
`checks` dict for debugging
- Never raises; pathological inputs return ok=False with explanation
Pipeline integration in `core/imports/pipeline.py:post_process_matched_download`:
- Hooks between the existing file-stability wait and AcoustID verification
- On failure: quarantine via existing `move_to_quarantine` helper, mark task
failed with descriptive error, clear matched-context, fire
`on_download_completed(success=False)` so the slot is released for retry
- Mirrors the existing AcoustID-failure path so retry behavior stays consistent
- Wrapped in try/except so an unexpected failure inside the check itself
cannot block downloads — logs and continues
This is intentionally tier 1: universal across formats, no external deps.
A future tier could verify FLAC STREAMINFO MD5 by decoding audio (needs
flac binary or libflac wrapper) — skipped for now since tier 1 catches the
dominant Discord-reported cases (truncated, 0-byte, wrong file).
Tests:
- `tests/imports/test_file_integrity.py` — 14 cases covering all three check
tiers, edge cases (zero/negative expected duration, long-track wider
tolerance, caller tolerance override), and the mutagen-unavailable
degradation path
- `tests/imports/test_import_pipeline.py` — two existing tests use 5-byte
fixture files that the new check would reject; they monkeypatch the
integrity check since they're testing plumbing (notification +
metadata_runtime forwarding), not integrity behavior
WHATS_NEW entry under '2.4.2' dev cycle.
|
||
|
Broque Thomas
|
cdd408b6f3 |
Auto-import: live card updates + multi-disc + featured-artist tag fixes
The 'Live Per-Track Progress' work shipped a backend in-progress row + top-of-tab
progress text but the history cards themselves stayed visually stale during
processing — lowercase "processing" badge, neutral styling, no per-track hint.
Smoke-testing also surfaced two latent identification bugs that prevented
multi-disc rips with features (Kendrick GKMC Deluxe) from importing at all.
Card-level live progress (`webui/static/stats-automations.js`):
- Cache `/api/auto-import/status` response in `_autoImportLastStatus`; poller
awaits status before re-rendering results so the card has the live data.
- Add 'processing' entries to statusLabels / statusIcons / statusClass.
- When card folder_name matches `current_folder`, swap the meta line to
`track N/M: <track name>` and tag the matching row in the expanded list
as `auto-import-track-row-active`; prior rows tag as `-row-done`.
Card styling (`webui/static/style.css`):
- `.auto-import-processing` blue left border, `.auto-import-badge-processing`
pulse animation, active/done track-row classes.
Multi-disc enumeration (`core/auto_import_worker.py:_scan_directory`):
- Old code skipped disc folders during recursion AND only attached them to a
parent that had its own loose audio. A folder containing only `Disc 1/`,
`Disc 2/` was invisible. Now: when a directory has only disc subdirs and no
loose audio, treat that directory itself as the album candidate. Disc folders
still skipped when standing alone.
- Add `FolderCandidate.is_staging_root` flag (set when the staging dir itself
becomes the candidate via this path) so identification can refuse to use the
meaningless folder name.
Tag identification (`core/auto_import_worker.py:_identify_from_tags`):
- Per-track `artist` tag fragmented consensus on albums with features
("Kendrick Lamar" / "Kendrick Lamar, Drake" / "Kendrick Lamar, Dr. Dre"
produced 3 separate `(album, artist)` keys for one album). Now group by
album first, then pick the most-common artist within that album group.
- `_read_file_tags` now prefers `albumartist` over `artist` for album-level
identity; falls back to `artist` for files without albumartist.
- Add INFO-level log when tag identification rejects, showing top albums and
their counts so the user can diagnose multi-disc / tagging issues.
Folder-name false-match guard (`core/auto_import_worker.py:_identify_folder`):
- When `is_staging_root` is set, skip the folder-name strategy entirely. Logs
the skip and falls through to AcoustID. Without this, dropping disc folders
directly into staging caused the scanner to search the metadata source for
the literal name "Staging", which false-matched against random albums (e.g.
"Stamina, Dinos" — a French rap album — at 13% confidence).
What's New entries added under 2.4.2 dev cycle.
|
||
|
Broque Thomas
|
783c543c3e |
Auto-import: live per-track progress + in-progress history row
User reported (Mushy / generally) that dropping an album into the staging folder left the auto-import history blank for the entire processing window — sometimes 5+ minutes for a full album. Pre- existing UX gap, not caused by the recent context-builder refactor. Two root causes: 1. ``_record_result`` only fired AFTER ``_process_matches`` returned. For a 14-track album with ~30s/track post-processing, that meant ~7 minutes of zero rows in auto_import_history → nothing for ``/api/auto-import/results`` to return → empty UI. 2. ``_current_status`` only ever transitioned between 'idle' and 'scanning' — never 'processing'. ``get_status()`` had no per- track index/name fields, so the UI had no way to render "Processing track 3/14: Mine" even if it wanted to. Fix: - New ``_record_in_progress`` inserts a status='processing' row up-front (before the per-track loop starts) so the UI sees the import the moment it begins. Returns the row id. - New ``_finalize_result`` updates that same row with the final outcome (completed/failed) when processing finishes. One row per album, not per track — keeps the history list clean. - Both share ``_serialize_match_data`` (extracted from the original ``_record_result``) so the in-progress row carries the same match payload shape the existing review UI already understands. - ``_process_matches`` updates ``_current_track_index``, ``_current_track_total``, and ``_current_track_name`` BEFORE each per-track callback fires, so a polling UI sees consistent "processing N/M: <name>" snapshots. - ``_scan_cycle`` flips ``_current_status`` to 'processing' before the per-album loop, resets it + the per-track fields after. Defensive ``finally`` clears progress even if the inner code path raised. - ``get_status()`` exposes the new fields so the UI's existing /api/auto-import/status polling picks them up. - Frontend (stats-automations.js): renders the new ``current_status='processing'`` state with track index/total/name in the existing progress bar element. New 'processing' status class for styling parity with 'scanning'. 8 regression tests in tests/imports/test_auto_import_live_progress.py: - get_status surfaces the new fields with sane defaults - track_index advances 1, 2, 3 during a 3-track loop - track_total set BEFORE the first callback fires (no '1/0' flicker) - _record_in_progress writes status='processing' with no processed_at - _finalize_result updates the same row to completed + processed_at, no second insert - _finalize_result with failed status leaves processed_at NULL - _finalize_result with row_id=None is a safe no-op - Per-track fields cleared by _scan_cycle's finally block Full pytest 1643 passed; ruff clean. |
||
|
Broque Thomas
|
29089b35b3 |
Honor configured Tidal redirect_uri, drop request-host fallback
Reported case (Foxxify): Tidal returned error 1002 ("Invalid redirect
URI") on every authentication attempt for users accessing SoulSync
from a network IP. User had ``http://127.0.0.1:8889/tidal/callback``
registered in his Tidal Developer Portal — matching the SoulSync UI
default and docs.
Root cause: the /auth/tidal route at web_server.py:5594-5598 had a
"fallback: dynamically set based on request host" branch that fired
when ``tidal.redirect_uri`` config was empty AND the request didn't
come from localhost. That fallback overrode the TidalClient
constructor's safe default (``http://127.0.0.1:<port>/tidal/callback``)
with a uri built from request.host like
``http://192.168.x.x:8889/tidal/callback``. Tidal compares strings
exactly so this never matched the documented portal registration and
the user got 1002 before the consent screen even rendered.
The trap is the SoulSync settings UI displays the default URI as the
placeholder + "Current Redirect URI" display — but the placeholder
never gets saved to config unless the user explicitly clicks Save.
Most users who follow the docs (register the displayed default with
Tidal, then click Authenticate) hit the empty-config path and the
broken fallback.
Fix: drop the request-host fallback. Empty config falls back to the
constructor default that matches the documented portal registration.
The existing post-auth swap-step in the instructions page below
handles the Docker / remote-access case as designed:
1. SoulSync sends 127.0.0.1:8889 in the authorize URL → matches
portal → Tidal accepts.
2. User authorizes → Tidal redirects browser to 127.0.0.1:8889
(which fails locally — nothing on user's machine listens there).
3. Instructions tell user to swap 127.0.0.1 with the host they're
accessing SoulSync from.
4. Swapped URL hits the container's exposed callback port → auth
completes.
8 regression tests in tests/test_tidal_auth_redirect_uri.py:
- Configured redirect_uri sent verbatim (localhost / custom port /
explicit network IP)
- Empty config falls back to constructor default — NOT request.host
(the actual reported scenario, with explicit assertion message
warning if the bug returns)
- Empty config + localhost access uses the same default (sanity)
Full pytest 1635 passed; ruff clean.
|
||
|
Broque Thomas
|
34ba26f5c8 |
Persist source IDs at download time + backfill onto tracks on sync
Followup to fix/watchlist-external-id-match. The companion PR closed the demand side — the watchlist scanner asks for tracks by external IDs before falling back to fuzzy. But for users on Plex / Jellyfin / Navidrome the supply side was still broken: tracks.spotify_track_id (and the other ID columns) only got populated by the asynchronous enrichment workers, sometimes hours after the file was actually written. During that window the ID match fell through to fuzzy and the bug returned. We were already collecting every ID during post-processing — they live in the `pp` dict in core/metadata/source.py:embed_source_ids and get embedded into file tags. We just dropped the in-memory copy afterwards. This PR persists them and uses them: - Schema migration adds spotify_track_id / itunes_track_id / deezer_track_id / tidal_track_id / qobuz_track_id / musicbrainz_recording_id / audiodb_id / soul_id / isrc columns + indexes to the existing track_downloads table (already keyed by file_path). - core/metadata/source.py:embed_source_ids exposes pp["id_tags"] and the resolved ISRC back to the import context as _embedded_id_tags / _isrc. - core/imports/side_effects.py:record_download_provenance reads those context fields and passes them to db.record_track_download, which now accepts the new ID kwargs and persists them. - New db.get_provenance_by_file_path with exact + basename-suffix fallback (handles container mount-root differences between download-time path and media-server-reported path). - New db.backfill_track_external_ids_from_provenance copies IDs from track_downloads onto a tracks row idempotently — COALESCE on every column preserves any value the enrichment worker already wrote (enrichment is more authoritative for late binding). - database/music_database.py:insert_or_update_media_track (the single insertion point used by every Plex / Jellyfin / Navidrome sync) calls the backfill immediately after each INSERT/UPDATE. - New core/library/track_identity.py:find_provenance_by_external_id used as a second-tier fallback in watchlist_scanner.is_track_missing _from_library — catches the window between download and media-server sync. Caller checks os.path.exists on the provenance file_path before treating it as "already in library" so a deleted file doesn't prevent re-download. Effect: freshly downloaded files become ID-recognizable to the watchlist on the very next scan, no enrichment-wait window. 19 regression tests in tests/test_provenance_id_persistence.py: - Schema migration adds expected columns + indexes - record_track_download persists every ID kwarg - record_track_download backward-compat (old kwargs still work) - get_provenance_by_file_path: exact match, basename fallback for mount-root differences, multi-record latest-wins, defensive None - backfill: copies all IDs, preserves existing via COALESCE, no-op when no provenance exists - find_provenance_by_external_id: per-ID lookup, ISRC cross-bridge, OR semantics, latest-wins on multiple matches Out of scope: backfilling provenance for files downloaded BEFORE this PR (their track_downloads rows don't carry the new IDs). Those continue to wait for enrichment. Acceptable — only affects historical files; new downloads benefit immediately. Full pytest 1625 passed; ruff clean. |
||
|
Broque Thomas
|
ecb8939c80 |
Match library tracks by external IDs before fuzzy in watchlist scan
Reported case (CAL): a track already on disk got re-downloaded by the
watchlist scanner on every scan. Library DB had stale album metadata
for the file (track tagged on album "Left Alone") while the metadata
source reported it on a different album ("NPC" single). The
title+artist+album fuzzy block correctly said the album names didn't
match and declared the track missing — but the file's stable external
IDs (Spotify ID, ISRC, etc.) unambiguously identified it as the same
recording.
The earlier compilation-album fix (PR #461) handled qualifier drift
("OST" vs "Music From The Motion Picture"). This case is two
genuinely different album names referring to the same song.
Fix: provider-neutral external-ID short-circuit before the fuzzy
block in `is_track_missing_from_library`. Pulls every recognized ID
off the source track (Spotify / iTunes / Deezer / Tidal / Qobuz /
MusicBrainz / AudioDB / Hydrabase / ISRC), runs a single SELECT
against the indexed external-ID columns on the `tracks` table, and
treats any hit as "track exists in library — don't re-download".
If no IDs are available (older imports without enrichment, library
scans that didn't populate external IDs), falls through to the
existing fuzzy logic so the safety net stays intact.
New `core/library/track_identity.py` module with two helpers:
- `extract_external_ids(track)`: handles dict and object-style track
shapes, direct-field aliases (spotify_id / spotify_track_id /
SPOTIFY_TRACK_ID), and provider-disambiguated native `id` fields
(when track has `provider='deezer'` and `id='X'`, treats X as a
Deezer ID).
- `find_library_track_by_external_id(db, external_ids,
server_source)`: builds an OR of indexed column matches with
IS NOT NULL guards, optional server_source filter that also
passes legacy NULL rows, single-row LIMIT.
ISRC bridges across providers — a library track imported via Deezer
can be matched against a Spotify scan when both sides carry the
same ISRC.
43 regression tests in `tests/test_library_track_identity.py`:
- 9 ID-extraction tests for direct fields (Spotify / iTunes / Deezer /
ISRC / MBID / AudioDB / Hydrabase)
- 8 ID-extraction tests via the provider field (8 providers + source
alias + missing-provider-ignored)
- 7 mixed/defensive tests (multiple IDs, object-style, empty strings,
None track, numeric coercion)
- 8 lookup tests (per-provider + ISRC cross-bridge)
- 3 OR-semantics tests
- 4 server_source filter tests
- 2 ID-column-map sanity tests
Full pytest 1606 passed; ruff clean.
|
||
|
Broque Thomas
|
486116c34f |
Honor lossy_copy.delete_original after successful conversion
Reported case (CAL): with lossy_copy.enabled=True, lossy_copy.delete_original=True, and codec=mp3, every download left both the original FLAC AND the converted MP3 in the target folder. Users opting into a lossy-only library ended up dual-format on every import. Root cause: ``core/imports/file_ops.py:create_lossy_copy`` reads ``lossy_copy.codec`` and ``lossy_copy.bitrate`` from config but never reads ``lossy_copy.delete_original``. The setting is only consulted by the pre-move source-vanished check at ``core/imports/pipeline.py:651`` (so the pipeline knows to look for a lossy variant when the FLAC has already moved on), but no code path actually deletes the source after conversion. Fix: after ffmpeg returns success and the QUALITY tag is written, check ``lossy_copy.delete_original`` and ``os.remove`` the original when enabled. Belt-and-suspenders: - Same-path guard (``os.path.normpath(out_path) != os.path.normpath(final_path)``) prevents accidentally wiping the just-converted file if a future codec choice somehow resolves out_path to the source path. - ``FileNotFoundError`` is treated as success (concurrent worker / dedup cleanup got there first). - Other ``OSError`` (permission denied, locked file) is logged but doesn't propagate — the conversion already succeeded, the user just has to clean up the original manually. Failure paths skip the delete: - ffmpeg returns non-zero → returns None, original stays - lossy_copy.enabled=False → early return before conversion runs - delete_original=False (default) → original stays 7 regression tests cover honored-when-enabled, kept-when-disabled, default-keep, ffmpeg-failure-path, lossy-disabled-path, racing-delete, and locked-file paths. Full pytest 1563 passed; ruff clean. Note: this PR does NOT address the second bug CAL mentioned (track re-downloaded despite already existing on disk). That symptom is caused by stale album metadata on the user's existing files — the library DB has the track tagged on a different album than the metadata source reports — combined with wishlist.allow_duplicate_tracks defaulting to True. Same class of issue partially addressed in PR fix/watchlist-redownload-and-duplicate-detection but compilation- album drift is the only currently-handled case. Tracking separately. |
||
|
Broque Thomas
|
99dbe265de |
Sync Qobuz auth to enrichment worker after login
Discord-reported (Foxxify): logging in to Qobuz via the Connect button on Settings showed "Connected: <username> (Active)" but underneath an error said "Qobuz not authenticated...", and the dashboard indicator stayed yellow. Saving settings or reloading the tab didn't help. Root cause: SoulSync runs two QobuzClient instances side by side — one through soulseek_client.qobuz for the /api/qobuz/auth/* endpoints, and a second owned by the enrichment worker thread for thread safety. The login flow only updated the auth-flow instance's in-memory state (plus persisted to config). The dashboard's "configured" check at web_server.py:3371 reads ``qobuz_enrichment_worker.client.user_auth_token`` — the WORKER's instance — which still believed itself unauthenticated. The connection-test step at core/connection_test.py:370 hits the same worker instance for the same reason. Fix: add ``QobuzClient.reload_credentials()`` — a public, network-free method that re-reads the saved session from config and updates the instance's in-memory state + session headers. Call it on the enrichment worker's client immediately after a successful ``/api/qobuz/auth/login``, ``/api/qobuz/auth/token``, or ``/api/qobuz/auth/logout`` so the two instances stay in lockstep without waiting for the next process restart. Unlike the existing ``_restore_session()`` this skips the network probe — the caller has just authenticated, so the token is known good. A small ``_sync_qobuz_credentials_to_worker()`` helper in web_server.py wraps the call so all three endpoints share one path. 10 new regression tests cover the populate / clear / partial-config paths plus the actual two-instance-sync scenario from the bug report. Full pytest 1555 passed (the one pre-existing flake in test_tidal_auth_instructions.py is order-dependent and unrelated). |
||
|
Antti Kettunen
|
a2176af00e
|
Rename metadata source status selectors
- Switch the dashboard/sidebar service-status card from spotify-branded ids to metadata-source ids - Update the shared status helpers to target the renamed metadata-source card - Keep the actual Spotify auth and settings UI unchanged |
||
|
Antti Kettunen
|
36267618a3
|
Rename status cache to metadata_source
Expose the primary metadata provider status under a generic cache key and update the websocket fixture plus frontend readers to match. |
||
|
Broque Thomas
|
7e32618f86 |
Drop old per-service enrichment routes after registry cutover
Followup to the enrichment-bubble registry consolidation. The
dashboard polling + click handlers all hit
/api/enrichment/<service>/{status,pause,resume} now, so the 30
hand-rolled per-service routes in web_server.py have zero callers
and can come out:
/api/musicbrainz/{status,pause,resume}
/api/audiodb/{status,pause,resume}
/api/discogs/{status,pause,resume}
/api/deezer/{status,pause,resume}
/api/spotify-enrichment/{status,pause,resume}
/api/itunes-enrichment/{status,pause,resume}
/api/lastfm-enrichment/{status,pause,resume}
/api/genius-enrichment/{status,pause,resume}
/api/tidal-enrichment/{status,pause,resume}
/api/qobuz-enrichment/{status,pause,resume}
Worker init blocks stay (they still construct the workers + persist
pause state). Section comment headers are preserved with a one-line
note pointing readers at the new generic blueprint.
Test fixtures in tests/conftest.py and
tests/metadata/test_enrichment_events.py also updated to use the
new URL paths so they reflect production reality. They were
synthetic stubs that never depended on the production routes —
purely cosmetic alignment.
Net: ~510 lines deleted from web_server.py. Full pytest 1541
passed; ruff clean.
|
||
|
Broque Thomas
|
98c04cf332 |
Consolidate enrichment bubble routes behind a service registry
The dashboard's enrichment-status bubbles (MusicBrainz, AudioDB,
Discogs, Deezer, Spotify, iTunes, Last.fm, Genius, Tidal, Qobuz) each
had its own copy-pasted /status, /pause, /resume route in web_server.py
— 30 routes that differed only in the worker reference and a couple
of per-service quirks (Spotify's rate-limit guard, Last.fm/Genius
yield-override behavior, Tidal/Qobuz extra status fields).
Replace them with a registry-driven blueprint:
- core/enrichment/services.py declares an EnrichmentService dataclass
with worker_getter, config_paused_key, pre_resume_check,
auto_pause_token, and extra_status_defaults — all variation captured
as data, no branching on service id.
- core/enrichment/api.py exposes a Flask blueprint with three routes
(/api/enrichment/<service>/{status,pause,resume}). Per-service
quirks are honored via the descriptor: Spotify's rate-limit ban
still returns 429 with `rate_limited: true`, Last.fm/Genius still
drop the auto-pause token and add the yield override, Tidal/Qobuz
still merge `authenticated: false` into the fallback payload.
- web_server.py registers all 10 services after their workers
initialize, wires the host-side hooks (config_manager.set,
_download_auto_paused.discard, _download_yield_override.add), and
registers the blueprint.
- webui/static/enrichment.js polling + click handlers now hit the
generic endpoints. The per-service `update<Service>StatusFromData`
functions are unchanged — they still process the same payload.
This is the cutover step. Old per-service routes are intentionally
left in place as a fallback during the soak period — they currently
have zero callers in the codebase and will be deleted in a follow-up
patch once production has run on the new pipeline for a few days.
27 new tests in tests/test_enrichment_services.py cover the registry
behavior + every quirk path through the generic blueprint (rate-limit
guard, auto-pause token cleanup, persisted-pause config keys, extra
default fields, worker-not-initialized fallback, exceptions). Full
suite 1541 passed; ruff clean.
|
||
|
Broque Thomas
|
0c4fad033d |
Show artist breadcrumb on sidebar Library button when on artist-detail page
Artist-detail is a "pseudo-page" reachable from Library, the unified
Search page, and the global search popover. It has no [data-page]
match in the sidebar, so navigateToPage's bulk-active-removal left
every nav button unhighlighted while the user was viewing an artist —
the sidebar offered no visual anchor for where they were.
Now:
- navigateToPage('artist-detail') falls back to highlighting the
Library button when no [data-page] match exists, anchoring the
sidebar to the canonical home for artist detail views.
- A new _updateSidebarLibraryBreadcrumb() helper rewrites the Library
button label between plain "Library" and a "Library / Artist Name"
breadcrumb based on currentPage + artistDetailPageState. Long names
(>14 chars) truncate with an ellipsis; the full name shows on hover
via the title attribute.
- Called from navigateToPage (entering / leaving the page) and from
loadArtistDetailData (covers same-page artist switches in the
similar-artist chain where currentPage stays 'artist-detail').
CSS adds .nav-text-root / .nav-text-sep / .nav-text-context selectors
so the "Library" anchor word stays visually dominant while the
separator and artist name dim to a secondary tier — readable but not
competing for attention.
Pure visual change. No backend touched. No new tests (DOM-only).
|
||
|
Broque Thomas
|
84810b4de4 |
Bump version to 2.4.1
Patch release wrapping up the 2.4.1 dev cycle. Highlights: - Watchlist no longer re-downloads compilation/soundtrack tracks (#458 dedup orphan cleanup + the album-match fix work in tandem to stop the loop). - Duplicate detector catches slskd dedup orphans via a second filename-bucket pass. - Beatport tab hidden temporarily — Cloudflare Turnstile blocks the scraper and the official OAuth API is closed to public devs. - Service worker for cover art + installable PWA manifest. - Browser caching for static assets (1y) and discover pages (5min). - Socket.IO same-origin default + admin-only /api/settings. Files updated: - web_server.py: _SOULSYNC_BASE_VERSION 2.4.0 -> 2.4.1 - webui/index.html: sidebar version button + modal subtitle - webui/static/helper.js: WHATS_NEW dev-cycle marker -> release date, fallback version in _getLatestWhatsNewVersion, 8 new VERSION_MODAL_SECTIONS entries promoted from this cycle - .github/workflows/docker-publish.yml: workflow_dispatch default version_tag updated to 2.4.1 |
||
|
Broque Thomas
|
6e61890551 |
Stop watchlist re-downloading compilation tracks; catch slskd dedup orphans
Two related bugs reported on Discord by Mushy. 1. The watchlist re-downloaded the same OST track up to 7 times. ``is_track_missing_from_library`` compared Spotify's album name and the media-server scan's album name with a raw SequenceMatcher at a strict 0.85 threshold. Compilations and soundtracks routinely fail this — Spotify reports ``"Napoleon Dynamite (Music From The Motion Picture)"`` while the Plex / Navidrome / Jellyfin tag scan saves it as ``"Napoleon Dynamite OST"``. Raw similarity ≈ 0.49, so the scanner declared the track missing on every 30-minute scan and added it back to the wishlist. The wishlist then issued a fresh download. slskd appended ``_<19-digit-ns-timestamp>`` to each new copy because the target file already existed, and the user ended up with seven copies of one song in one folder. Fix: extract two pure helpers — ``_normalize_album_for_match`` strips qualifier parentheticals (Music From X, OST, Deluxe Edition, Remastered, Anniversary, etc.) and trailing dash-clauses; ``_albums_likely_match`` checks equality after normalization, substring containment, and a relaxed 0.6 fuzzy ratio. A volume / part / disc / standalone-trailing-number guard rejects pairs like ``"Greatest Hits Vol. 1"`` vs ``"Greatest Hits Vol. 2"`` so the relaxed threshold doesn't introduce false positives on serialized releases. After this change the Napoleon Dynamite case collapses to ``"napoleon dynamite" == "napoleon dynamite"`` via the equality short-circuit and the redownload loop dies. 2. The duplicate detector found only one of the seven dupe files. The detector buckets tracks by the first 4 chars of their normalized tag title. Files written by slskd directly into a library folder often get inconsistent (or blank) tags from the media-server rescan, so the seven copies were bucketed apart by parsed title and never compared. Fix: refactor the per-bucket comparison into ``_scan_bucket``, then add a second pass — ``_build_filename_buckets`` re-buckets leftover tracks by canonical filename stem (slskd dedup tail stripped via ``_strip_slskd_dedup_suffix``, same regex the import-cleanup PR uses) plus extension. Filename agreement is itself strong evidence the files came from the same source download, so the second pass calls ``_scan_bucket`` with ``require_metadata_match=False`` to skip the title / artist / cross-album gates. The same-physical-file guard still runs so bind-mount duplicates aren't flagged. 72 new regression tests across two files cover the album-match helpers (28 tests including the Napoleon Dynamite scenario, 7 volume disagreements, 8 positive/negative pairs, 5 defensive cases) and the new filename-bucket pass (16 tests across bucket construction, scan integration, and existing title-pass behavior). Full pytest 1509 passed; ruff clean. Reported by Mushy in Discord. |
||
|
Broque Thomas
|
ab884292d1 |
Hide Beatport tab temporarily
Some checks failed
Compile the app and run tests / sanity-check (push) Has been cancelled
Beatport added Cloudflare Turnstile to every public page on beatport.com. The unified scraper now receives bot-challenge HTML instead of real content, so all /api/beatport/* endpoints return 500 with "Could not fetch Beatport homepage". The official Beatport v4 API is locked behind OAuth application registration that isn't open to the public — confirmed via the docs at api.beatport.com/v4/docs and community projects (beets-beatport4). The public docs SPA client_id only accepts browser-based flows (post-message redirect URI), which can't be driven server-side. Hide the Beatport tab on the Sync page so users stop hitting the broken endpoints. Backend routes and beatport_unified_scraper.py stay in code — revival is a one-attribute HTML change once Cloudflare relaxes or a workaround is found. Reported via the homepage 500 spam in user logs. |
||
|
Broque Thomas
|
46d8e15674 |
Prune slskd dedup orphans after import
slskd appends "_<19-digit unix-nanosecond timestamp>" to a downloaded filename when the destination already contains a same-named file (concurrent downloads of the same track, partial-file retries after a connection drop, cancelled-then-redownloaded files, the same track surfacing in multiple synced playlists). The file-finder code already recognized the suffix when matching a download to its source — but after the canonical file moved into the library, the leftover "_<timestamp>" siblings sat orphaned in the downloads folder forever. Reported on Discord by Shdjfgatdif. cleanup_slskd_dedup_siblings() runs at the end of each successful import (3 safe_move_file sites in pipeline.py) and prunes any remaining siblings that strip down to the canonical stem with the same extension. Conservative match (>= 18 trailing digits) keeps legitimate filenames like "Track 5" and "Album 1995" untouched. Per- file unlink failures are swallowed so a single locked file doesn't block the rest. 17 regression tests cover the suffix-strip primitive, orphan removal, no-op cases, mismatched extensions, subdirectories, and partial-failure recovery. |
||
|
Antti Kettunen
|
55603be14c
|
Clarify Spotify auth flow and sync UI
- Send Spotify auth completion back to the opener so the settings page refreshes immediately - Make the local auth flow go straight through to Spotify instead of showing the temporary instruction page - Keep the remote/docker instruction page available for manual callback setups - Sync Spotify status, connect/disconnect buttons, and metadata source selection after auth and disconnect - Keep the disconnect behavior aligned with the active primary metadata source |
||
|
Broque Thomas
|
6cdcf778f3 |
Lift /api/automations/* into core/automation/
Routes moved to thin parse-args/jsonify handlers; logic now lives in three focused modules under core/automation/. 436 lines deleted from web_server.py; 53 added back as wrappers. Module split: - core/automation/api.py — CRUD + run + history helpers. Each function takes (database, automation_engine, ...) explicitly and returns (response_body, http_status). Includes signal cycle detection preflight checks for create + update. - core/automation/progress.py — owns the in-memory progress state dict + lock (mirroring the original web_server.py globals as module-level shared state so all callers see one view), init/update/history helpers, and the WebSocket emit loop. - core/automation/signals.py — collect_known_signals for the builder autocomplete. Out of scope (deferred): - _register_automation_handlers — the 23+ action handler closures stay in web_server.py because each one is tightly coupled to feature- specific implementations (wishlist, watchlist, library scan, etc.). - Worker functions (_process_wishlist_automatically, etc.) — belong with their feature lifts. - _run_sync_task / _run_playlist_discovery_worker — sync + discovery PRs. Behavior preserved 1:1: - Same route response shapes + status codes - Same JSON field hydration (trigger_config, action_config, notify_config, last_result, then_actions) - Same backward-compat: empty then_actions + notify_type set → synthesize then_actions from notify_type/notify_config - Same signal cycle detection behavior on create + update - Same system-automation protection on delete + duplicate - Same reschedule/cancel logic on toggle + bulk-toggle + update - Same progress state shape (status, progress, phase, current_item, log capped at 50, started_at/finished_at, action_type) - Same emit-on-finish socketio push from update_progress - Same emit loop semantics (1s tick, snapshot active states, reap finished after window) Pre-existing bugs preserved (will fix in follow-up PRs): - emit_progress_loop uses naive datetime.now() against tz-aware started_at/finished_at, so the timeout-zombie check raises TypeError → caught → never fires, and the cleanup-after-window check raises → caught → state is reaped on FIRST tick regardless of the window. Tests document this behavior so the next PR can flip them to the corrected expectation. Tests: 72 new under tests/automation/ (signals 10, progress 24, api 38). Full suite: 861 passing (was 789). Ruff clean. |
||
|
Broque Thomas
|
fd7b56e58c |
Lift /api/search and /api/enhanced-search/* into core/search/
Routes moved to thin parse-args/jsonify handlers; logic now lives in six focused modules under core/search/. 720 lines deleted from web_server.py; 109 added back as wrappers; ~700 lines of new core code plus ~700 lines of tests. Module split: - core/search/cache.py — TTL+LRU cache for enhanced-search responses, keyed by (query, active_server, fallback_source, hydrabase_active, source_tag) so config changes don't poison stale entries. - core/search/sources.py — per-kind metadata search (artists/albums/ tracks) and the multi-kind ThreadPoolExecutor that fans them out. - core/search/library_check.py — library + wishlist presence check with Plex thumb URL resolution; profile-aware wishlist with legacy fallback for older DBs missing the profile_id column. - core/search/stream.py — single-track preview search; effective stream mode resolution, query-variant generation, retry walk, matching engine integration. - core/search/basic.py — flat Soulseek file search, quality-sorted. - core/search/orchestrator.py — main enhanced-search dispatch (short-query fast path, single-source bypass, hydrabase-primary fan out, alternate source list builder), NDJSON streaming generator for /source/<src>, and the SearchDeps dataclass that bundles the cross-cutting deps. Routes pass clients (spotify, hydrabase, hydrabase_worker, soulseek) and helpers (config_manager, fix_artist_image_url, _is_hydrabase_active, _get_metadata_fallback_*, _run_background_ comparison, run_async, dev_mode_enabled_provider) into core/search via a SearchDeps bundle built per-request. fix_artist_image_url stays in web_server.py because it touches 31 other call sites. Behavior preserved 1:1: - Same response shapes (db_artists, spotify_artists, spotify_albums, spotify_tracks, primary_source, metadata_source, alternate_sources, source_available) - Same NDJSON line ordering (artists/albums/tracks as they finish, plus done marker) - Same per-kind exception swallowing - Same hydrabase-worker mirror on dev mode - Same cache key shape (5-tuple) and TTL/LRU semantics - Same stream-track effective-mode resolution including the Soulseek-coerce-to-YouTube edge case - Same library-check Plex thumb URL rewriting and wishlist fallback for older DBs Tests: 94 new (cache TTL/LRU/key, sources happy/partial/all-fail, library presence with library + wishlist + thumbs, stream effective mode + query gen + retry, orchestrator client resolution + short query + single source + fan-out alternates + hydrabase primary + NDJSON drain). Full suite: 788 passing (was 694). Ruff clean. |
||
|
Broque Thomas
|
f51b75da7e |
Lift /api/stats/* and /api/listening-stats/* into core/stats/
Stats route logic moves into core/stats/queries.py as pure-ish functions that take dependencies (database, image-url fixer, listening worker) as arguments. The 13 route handlers in web_server.py shrink to thin parse-args / jsonify wrappers. What moved to core/stats/queries.py: - stats_cached: 3-key metadata cache lookup + image url fix-up - stats_overview / timeline / genres / library_health / db_storage - stats_top_artists / top_albums / top_tracks: top-N + DB enrichment - stats_recent: listening_history readback - stats_resolve_track: title+artist -> file_path lookup for playback - listening_stats_sync: spawns daemon thread that runs worker._poll - listening_stats_status: stats payload, with None-worker fallback shape No behavior change. Same response shapes, same error handling, same silent-except on per-row enrichment failure. fix_artist_image_url stays in web_server.py and is passed through as a callback so we don't have to lift its config_manager / media-server dependencies in this PR. Adds tests/stats/test_stats_queries.py — 27 tests covering happy paths, edge cases, image-url plumbing, worker glue. Ruff clean. 694 tests pass (was 667 + 27 new). |
||
|
Broque Thomas
|
f11b91a5c6 |
Service worker for cover art + PWA manifest
Addresses #365 (reported by JohnBaumb), parts 3 & 5. Client-side IDB / sessionStorage data cache (part 4) deferred to its own PR. Cover art on Library and Discover used to re-fetch from the source CDN on every page visit. Now a service worker caches images locally in CacheStorage with cache-first strategy — second visit serves art instantly with zero network round-trips. PWA manifest added so the app is installable to home screen / desktop. Service worker (`webui/static/sw.js`): - Cache-first for images: 10 known CDN hosts (Spotify, Last.fm, Apple, Deezer, Discogs, MusicBrainz CAA, YouTube thumbnails) plus the local `/api/image-proxy` endpoint plus same-origin .png/.jpg/ .webp/.gif/.svg paths. Cross-origin file-extension matches are refused so we don't accidentally cache trackers. - Stale-while-revalidate for `/static/*`: serve cached instantly, refresh in background. Combined with the existing `?v=static_v` cache-bust, deploys still ship live (different query → different cache entry, old ages out). - HTML / API / everything else: no caching, pass through. - Cache-versioned (CACHE_VERSION = 'v1'); activate handler wipes any cache whose name doesn't match the current version. - skipWaiting + clients.claim so deploys propagate to open tabs without requiring a full close-and-reopen. PWA manifest (`webui/static/manifest.json`): - Standalone display mode, theme color #1db954 (matches --accent-rgb). - Two icons (192, 512) with both `any` and `maskable` purpose, generated from favicon.png with aspect-preserving transparent padding so the existing logo lands inside the safe zone for OS-applied masks. Wiring: - `web_server.py` adds a `/sw.js` route that serves the file from root scope (a service worker only controls URLs at or below its served path; `/static/sw.js` would scope to `/static/*` only). `Cache-Control: no-cache` on the SW response so deploys propagate on next page load instead of being pinned by the 1yr static cache the rest of /static/ uses. - `webui/index.html` adds the manifest link, theme-color meta, and an apple-touch-icon for iOS. - `webui/static/init.js` registers the SW on `window.load`. Feature-detected — no-op on browsers without serviceWorker support or on non-secure origins (SW requires https or localhost). One bug caught + fixed during line-by-line self-review: `_staleWhileRevalidate` could return null to `respondWith()` when both the cache miss AND the network fetch failed (the `.catch(() => null)` collapsed the rejection to null, which then short-circuited through the falsy chain). Now explicitly awaits the network promise and falls back to `Response.error()` when it resolves to null — matches the `_cacheFirst` pattern. Browser-verified: sw.js registers, status "activated and is running" in DevTools. 603 tests pass. |
||
|
Broque Thomas
|
b0e7dae7c6 |
Cache static assets 1y + cache discover GETs 5min
Addresses #365 (reported by JohnBaumb), parts 1 & 2 of the proposal. Service worker, client-side IDB/sessionStorage, and PWA manifest deferred to follow-up PRs. 1. Static asset cache (CSS/JS/icons/fonts). `SEND_FILE_MAX_AGE_DEFAULT` flipped from 0 to 31536000 (1 year) in production. Safe because every static URL is bust-tagged with `?v=static_v` (computed once per process start), so each server restart effectively invalidates every cached asset for every user. Within a single deploy, repeat page loads hit zero round-trips on static files — was a 304 round-trip per asset before. Dev override (`SOULSYNC_WEB_DEV_NO_CACHE=1`) keeps it at 0 so iterating on JS/CSS doesn't need a server restart between edits. Collateral fixes from the bump: - Music streaming endpoint (L16140): `response.headers.add('Cache-Control', 'no-cache')` → bracket-assign. Under the old max-age=0, send_file set `no-cache` and `.add()` duplicated harmlessly. Under the new max-age=31536000, `.add()` would APPEND a second Cache-Control value → two conflicting headers, browser-undefined behavior. Bracket-assign replaces. - Backup download endpoint (L25181): explicit `Cache-Control: no-store` on the response so DB backups don't inherit the new long max-age — sensitive content, must never cache. 2. Discover GET browser cache (5 min). New `@app.after_request` hook scoped to `/api/discover/` and `/api/discovery/` paths, GET method, 2xx responses only. Sets `Cache-Control: public, max-age=300`. Skipped when the endpoint already set its own Cache-Control. Toggling between Discover sections within 5 min serves from browser cache, no backend hit. Try/except wraps the hook body and logs a warning if anything throws — never let a header-tagging bug turn a successful response into a 500. (Logging instead of `pass` since silent except-pass is exactly the anti-pattern issue #369 is about.) Audited every other Cache-Control set site in web_server.py — only the two `send_file` callers needed adjustment. Range-branch streaming uses `Response()` directly, unaffected by the config change. 603 tests pass. |
||
|
Broque Thomas
|
01b7d50311 |
Gate /api/settings endpoints behind admin profile
Closes #370 (reported by JohnBaumb). The /api/settings endpoint and three siblings (/log-level, /config-status, /verify) had no auth check — any logged-in profile could read or modify service tokens, OAuth secrets, and API keys. Cin's "minimum" suggestion from the issue: gate to admin profile. Added an `admin_only` decorator near `get_current_profile_id` that returns 403 when the current profile isn't admin (id=1). Applied to all four endpoints. Auth model note (documented in the decorator docstring): SoulSync's existing model is "trust local network" — single-admin / no-multi- profile installs default `get_current_profile_id()` to 1, so the gate is a no-op for solo users. The decorator is meaningful in multi-profile setups where non-admin sessions exist. Tightening to real per-request auth is out of scope. Did NOT consolidate with api/settings.py (Cin's "better" suggestion): that endpoint uses API-key auth (for external tools), the web_server.py copy uses session/profile auth (for the web UI). Different consumers, different auth models — merging would break one or the other. 603 tests pass. |
||
|
Broque Thomas
|
efd2960629 |
Merge remote-tracking branch 'origin/dev' into fix/socketio-cors-wildcard
# Conflicts: # webui/static/helper.js |
||
|
Broque Thomas
|
22fda5dd94 |
Trim yt-dlp pin comment, drop misleading WHATS_NEW page link
Self-review nits on PR #384: - requirements.txt: 5-line comment for one pin → 1 line. Rationale lives in commit body and #367; no need to repeat in-tree. - helper.js: dropped `page: 'settings'` from the yt-dlp WHATS_NEW entry. Settings page has no yt-dlp UI; the link would have navigated users somewhere irrelevant. 553 tests pass. |
||
|
Broque Thomas
|
77a781caba |
Pin yt-dlp in requirements.txt, drop pip install from entrypoint
Closes #367 (reported by JohnBaumb). The Docker entrypoint ran `pip install -U yt-dlp --quiet --no-cache-dir` on every container start. Three problems with that: - Non-deterministic startup: each restart could pick up a different yt-dlp version, making "works on my machine" debugging harder. - Network dependency at boot: PyPI being slow/unreachable gated the app coming up. - In-place upgrades inside running containers can race with active yt-dlp invocations and aren't a great pattern. Picked Option A from the issue: pin to an exact version in requirements.txt (`yt-dlp==2026.3.17`) and remove the entrypoint install entirely. yt-dlp comes baked into the image now via the existing `pip install -r requirements.txt` in the Dockerfile. Tradeoff: YouTube fixes ship via SoulSync releases now instead of "next container restart". The pin is documented inline with how to bump it. Net change: -3 entrypoint lines, requirements.txt pin tightened, WHATS_NEW '2.4.1' block opened (entries hidden until version bumps). 553 tests pass. |