Commit graph

68 commits

Author SHA1 Message Date
BoulderBadgeDad
f0a6d5e696 discovery: add apply_adventurousness — pure popularity-penalty re-rank (aurral-style)
Both Discover rec rows already exclude what you own/watch, so novelty is baked in; the missing lever
is a popularity penalty. apply_adventurousness(items, level) re-ranks dicts (score + optional 0-100
popularity) so globally-popular candidates sink as the dial rises. Pure + reusable across both rec
rows. level<=0 returns the input order unchanged (a copy) — fully additive, no regression; 1.0 applies
the full penalty (a popularity-100 pick loses 70% of its score). Missing popularity is never penalised.

5 seam tests (no-op+copy, demotion, proportional penalty, missing-pop, clamping). 37 pass. Wiring
(scan stores popularity -> routes re-rank live -> Settings slider) is the next increment.
2026-06-29 16:03:16 -07:00
BoulderBadgeDad
9cb5c4d40d Listening Mix: source-independent track fetch (Deezer public fallback)
The mix is (artist, title) pairs acquired via Soulseek, so the recommendation fetch needn't match the user's active source. When the active source can't fetch top tracks (iTunes/Discogs/MusicBrainz — or Spotify when unauthed), fall back to Deezer's public artist/{id}/top (no auth, available to everyone). All five active sources now build a full mix without switching; the name-search + names_match guard still prevents wrong-artist results. New pure helper choose_mix_fetch_source + tests.
2026-06-25 10:37:16 -07:00
BoulderBadgeDad
9c91ba29bf Discover: listening-driven recommendations + mix (#913), Fresh Tape fix
#913 was silently producing 0 recs: similar_artists.source_artist_id is a SOURCE id (Spotify/etc.), but the scan keyed id->name by internal artists.id (resolved nothing), and the consensus ranker was fed the name-collapsed get_top_similar_artists (consensus could never fire). Fixed + elevated:

- id->name keyed by source-id columns; raw per-seed edges (real consensus); similarity_rank threaded into the score; recency-weighted seeds (recent plays boost lifetime favs)
- new 'Based On Your Listening' artist row (/api/discover/listening-recommendations) with 'because you listen to X' explanations
- new 'Your Listening Mix' track row: each rec's top tracks via a guarded, name-resolved Spotify/Deezer fetch (falls back to the discovery pool), stored as full render dicts so the row can't shrink on pool rotation
- pure tested core: similarity_from_rank, build_recency_weighted_seeds, to_mix_track, names_match (+ rank-aware grouping)

Fresh Tape (5-10 tracks): future-dated albums sorted to the top of get_discovery_recent_albums and ate the 50-album budget before the is_future_release skip ran. Add exclude_future_years + fetch a generous budget; downstream caps unchanged. Regression tested.

Also drop the per-track block 'X' from the compact playlist rows (wrong spot). Plan/audit in DISCOVER_BEST_IN_CLASS_PLAN.md.
2026-06-25 10:15:20 -07:00
BoulderBadgeDad
ed0a2079cf
Merge pull request #896 from nick2000713/feature/best-quality-search-mode
Global quality system: real-audio verification, best-quality search & quality profiles (please try...not ready to merge)
2026-06-24 20:27:38 -07:00
BoulderBadgeDad
d0966ec262 Sync wishlist re-add: build the IDENTICAL payload the auto-add uses
Root cause (the real one): the auto-add passes original_tracks_map[id] — tracks_json
run through a specific normalization (album->dict with images/album_type/total_tracks/
release_date, artists->dicts). My re-add hand-rolled a different shape, so the stored
spotify_data didn't match and the wishlist's nebula (which reads spotify_data.album.
images[0].url) had no cover, plus album/single classification could differ.

Fix: extract that normalization into one shared build_original_tracks_map() and use it
in BOTH the live sync (core.discovery.sync) and the re-add. The re-add now resolves the
track by source_track_id through the same map — byte-identical payload. Verified on a
real sync row: re-add payload == live-sync payload, album.images present. (The shared
normalizer is also copy-safe, fixing a latent tracks_json mutation in the old inline
version.)

Fallback (track absent from tracks_json) rebuilds through the same normalizer with the
cover seeded from the row's image_url. 10 tests incl. a direct parity assertion.
2026-06-24 16:41:40 -07:00
BoulderBadgeDad
c21031b9bc #913: add group_similars_by_seed assembly helper (pure, tested)
The stored similar_artists rows key the similar artist by the SEED's source/db id, not its name,
so rank_recommended_artists can't consume them directly. group_similars_by_seed resolves each
row's source id to a seed name via a caller-supplied id_to_name map and reshapes to the
{seed_name: [{'name': similar}]} the ranker wants — the fragile id->name join, now pure + tested
(dataclass + dict rows, unknown-id drop, non-seed drop, group->rank end-to-end). 15 tests total.
2026-06-23 22:53:57 -07:00
BoulderBadgeDad
9ad5188610 #913: listening-driven recommendation core (pure, tested)
New, fully-additive module — the heart of the 'expand Because You Listen To into a real
listening-driven block' plan. Two pure functions, no DB/network/config:

- rank_recommended_artists(seeds, similars_by_seed, owned): consensus-ranked artists you'd love
  but don't own. Score = Σ over endorsing seeds of (play_weight × similarity) — rewards consensus,
  play weight and similarity strength in one sum. Excludes owned + seeds; min_seed_count is the
  adventurousness dial's lever; exposes seed_count + which seeds ('because you like A, B, C').
- aggregate_candidate_tracks(recs, top_tracks_by_artist, owned): per-artist-capped, deduped,
  rank-ordered candidate list for the generated playlist; exclude_owned toggles discovery vs replay.

11 tests (consensus vs single, play-weight, similarity, owned/seed exclusion, min_seed_count,
case-insensitive dedup, per-artist cap, owned exclusion, total limit, empty-artist skip). Nothing
existing touched — wiring into the watchlist scan + playlist sync comes next.
2026-06-23 22:47:43 -07:00
BoulderBadgeDad
9f5bc0de89 #909: backfill the YT-artist column from a confident match instead of 'Unknown Artist'
YouTube's flat playlist extraction returns ONLY the title (verified: no artist/channel/uploader
field at all), so a track starts as 'Unknown Artist' and only gains a name if per-video recovery
succeeds. When recovery comes up empty (no cookies / age-gated / bot-checked) but the track still
matched confidently, the worker threw the match's artist away and left the column 'Unknown Artist'
— the #909 symptom.

Now the displayed yt_artist falls back to the matched artist when it's still Unknown. Display-only:
the match itself, track['artists'], cache, and download flow are untouched, so a real recovered
name always wins and an unmatched/error row honestly stays Unknown. Extracted resolve_display_artist
as a pure, tested seam; applied in the cache-hit and fresh-match result paths (the error path has
no match to draw from).
2026-06-23 11:54:55 -07:00
nick2000713
63374b32f1 Merge remote-tracking branch 'nezreka/dev' into feature/best-quality-search-mode
# Conflicts:
#	core/hifi_client.py
2026-06-23 11:33:50 +02:00
BoulderBadgeDad
cf2b4d7d38 Playlist sync: honor the DURABLE manual match, not just the volatile cache
Find & Add was being forgotten on the next auto-sync. It persists two ways — a fast
sync_match_cache override AND a durable manual_library_match (#787) that survives a
rescan — but BOTH sync matchers (services.sync_service._find_track_in_media_server and
the DB-only fallback) only consulted the volatile cache. A library rescan wipes that
cache, so the next 'replace' auto-sync re-matched the track from scratch and the user
had to Find & Add it again.

Both matchers now fall back to the durable manual match when the cache misses
(self-healing a stale library id via the stored file path), exactly like the compare
view already does via resolve_durable_match_server_id. So a Find & Add pairing sticks
across rescans + auto-syncs. Seam tests: cache-wiped→durable hit, stale-id self-heal,
no-match→fuzzy fall-through.
2026-06-21 18:15:20 -07:00
dev
b761229a00 merge: pull upstream/main (2.7.4) into feature/best-quality-search-mode
- Keep our v3 ranked-targets quality system (filter_and_rank, QualityTarget)
  in soulseek_client.py, settings.js, database presets, and index.html
- Take upstream removal of standalone quality-scanner code:
  QualityScannerDeps + run_quality_scanner moved to repair job
  (core/repair_jobs/quality_upgrade_scanner.py)
- Take upstream AAC-tier addition in database/music_database.py default profile
- Take upstream removal of /api/quality-scanner/* routes from web_server.py
- Remove test_discovery_quality_scanner.py (deleted upstream)
- 47 upstream commits absorbed (2.7.3 + 2.7.4 including re-identify flow,
  dead-folder cleanup, track-number prefix strip, and more)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-19 16:54:51 +02:00
dev
ff061324ba fix(quality-scanner): resolve relative library paths before probing
Diagnostics revealed the real cause: the tracks table stores file_path
RELATIVE to the library root (e.g. "Asketa/Another Side/01-01 - Another
Side.flac"), so probing the raw path failed for the entire library — every
track came back unprobeable and was left unflagged ("20/20 could not be
probed").

The scanner now resolves each path via _resolve_library_file_path (checks
transfer/download/library dirs, same helper the rest of the app uses) before
probing, falling back to docker_resolve_path. Injected via deps for testability.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 23:30:46 +02:00
dev
446465a833 fix(quality-scanner): log under soulsync namespace so progress is visible
The worker used logging.getLogger(__name__) → "core.discovery.quality_scanner",
which the app log view (soulsync.*) doesn't surface — so the scan looked like it
did nothing ("API Starting scan" straight to "quality_scan_completed" with no
worker output). Switched to get_logger("discovery.quality_scanner") so "Found N
tracks", "Profile targets", and the unprobeable-file diagnostics show up.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 23:21:04 +02:00
dev
501868d9c9 fix(quality-scanner): resolve library paths + surface unprobeable files
The unified scanner must READ each file to judge real bit depth/sample rate
(extension alone can't tell 16-bit from 24-bit FLAC). If the stored library
path doesn't resolve to a readable file in this container, every probe returns
None and — since an unprobeable file can't be judged — the whole library passes
silently ("scans nothing").

Now: resolve the path via docker_resolve_path before probing, and count +
log unprobeable files (first 5 paths at WARNING, plus an end-of-scan summary
"N/M tracks could not be probed"). This makes a systematic path/mount mismatch
visible instead of an empty result.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 23:12:04 +02:00
dev
d2abec4a92 feat(quality): unify quality scanner onto the real ranked-target core (strict)
The library quality scanner judged quality by FILE EXTENSION only
(get_quality_tier_from_extension) and read the legacy v2 `qualities` dict —
so every FLAC was "lossless tier 1" regardless of bit depth / sample rate. It
could never flag a 16-bit FLAC as upgradeable under a 24-bit profile, and it
ignored the v3 ranked_targets entirely. Completely inconsistent with the
download guard.

Now both share one core:
- selection.targets_from_profile(profile) — single profile→targets conversion
  (v2→v3 migration), reused by load_profile_targets.
- selection.quality_meets_profile(aq, targets) — strict: meets iff the real
  measured quality satisfies a ranked target (fallback ignored — it's a
  download concession, not a definition of "good enough").
- guards.check_quality_target refactored to use both.
- quality_scanner probes real quality (probe_audio_quality) and checks against
  the v3 targets via quality_meets_profile. Extension tier kept only as a
  fallback label when a file can't be probed.

Result: the scan flags exactly what the download gate would reject — 16-bit
when you want 24-bit, wrong sample rate, MP3 when you want FLAC.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 22:50:01 +02:00
BoulderBadgeDad
b393866782 Remove old auto-acting Quality Scanner tool (replaced by Quality Upgrade Finder job)
Phase 2 of the redesign. The tool that judged quality by extension and auto-dumped
matches into the wishlist is gone; quality scanning is now the reviewed
quality_upgrade repair job.

Removed:
- Frontend: Tools-page Quality Scanner card, its JS handlers/poller/socket listener,
  help tooltip + tour entry (webui index.html, core.js, helper.js, wishlist-tools.js).
- Backend: /api/quality-scanner/{start,status,stop} endpoints, the in-memory state +
  executor + 1s socket broadcast, the QualityScannerDeps/run_quality_scanner shim.
- core/discovery/quality_scanner.py: the auto-acting worker + deps class (the shared
  match/normalize helpers stay — the new job imports them).

Rewired:
- Automation 'start_quality_scan' action now triggers the quality_upgrade repair job
  via repair_worker.run_job_now() (AutomationDeps gains run_repair_job_now, drops the
  4 scanner fields). Action block's vestigial scope field removed (scope lives in the
  job's settings now). NOTE: the 'quality_scan_completed' trigger no longer fires (the
  repair job doesn't emit it).
- Updated all automation test _build_deps helpers + conftest tool-progress harness;
  deleted the obsolete worker test. 528 affected tests pass; 6123 collect cleanly.

QUALITY_TIERS / _get_quality_tier_from_extension kept (used elsewhere).
2026-06-13 12:14:45 -07:00
BoulderBadgeDad
0577dc92e5 #863: add diagnostic logging to YouTube artist-recovery so we can see per-track what it returns 2026-06-13 00:03:35 -07:00
BoulderBadgeDad
c72e83bc2f #863: move YouTube artist recovery out of the (synchronous) parse into the async discovery worker — parse is fast again, no 120s timeout risk 2026-06-12 23:46:29 -07:00
BoulderBadgeDad
87e5e1fa23 #702: make mirrored-playlist cancel/reset/delete idempotent (un-wedge LB weekly sync)
Root cause (from the reporter's app.log): a ListenBrainz weekly playlist syncs
through the in-memory youtube_playlist_states discovery machine. When that live
state is lost — a Docker restart, or the discovery process ending while the user
waits for the media-server scan — the DB discover-download snapshot survives but
the live state is gone. Every recovery action (Cancel/Reset/Delete) then hit
`key not in states` and returned 404 "YouTube playlist not found" (hence the
confusing "Youtube" on a ListenBrainz playlist), leaving the playlist permanently
wedged with no way to dismiss or re-sync. Works for the maintainer because a
single session with no restart keeps the live state alive.

Fix — these are cleanup ops, so "the thing is already gone" is SUCCESS, not 404:
- cancel_sync core (shared by YouTube + ListenBrainz + Tidal/Deezer/Qobuz/...) →
  missing key returns idempotent success.
- reset_youtube_playlist / delete_youtube_playlist → same.
The playlist becomes recoverable: Cancel/Reset clears the dead state and the user
re-syncs fresh.

Tests: cancel_sync core (missing key = idempotent 200 not 404; present key still
cancels + clears the worker + reverts phase); endpoint-level idempotency for
cancel/reset/delete; updated the old test that locked the 404 wedge. 834 sync/
discovery tests green.
2026-06-11 12:55:55 -07:00
BoulderBadgeDad
5bc27e6268 #843 follow-up: key the no-state cache save by the FIRST artist
The #843 fallback saved the discovery-cache match using the client's
original_artist verbatim — but the client sends a joined "A, B, C" string, while
EVERY in-memory + sync path keys the cache by the first artist (artists[0]). So a
multi-artist fix (the reporter's exact "Cherrymoon Traxx, Hermol, SBM, BELS"
case) would have saved under a key the sync never looks up — the fix would
"succeed" with no error but silently never apply.

Reduce the client artist to the first (split on comma) in the no-state branch so
its cache key matches the in-memory/sync convention exactly. Single-artist tracks
are unaffected.

Test: no-state save now keys by the first artist, and a new test pins that the
no-state and in-memory paths produce an IDENTICAL cache key for the same
multi-artist track. 74 discovery tests pass.
2026-06-10 18:57:43 -07:00
BoulderBadgeDad
0afa3c9705 Fix: "Discovery state not found" when fixing a match after restart/import (#843)
The discovery FIX → Confirm flow 404'd with "Discovery state not found" whenever
the in-memory discovery state was gone — a server restart, or an imported
playlist that wasn't discovered in THIS process — even though the card is still
shown from persisted data (the reporter's log shows "Returning 0 stored ...
playlists for hydration", i.e. the in-memory states were empty).

The thing that actually makes a manual fix STICK is writing it to the discovery
cache (save_discovery_cache_match), keyed by the original track's name + artist —
which doesn't need the in-memory state at all. But the endpoint 404'd on the
missing state before reaching that write, so the fix was dead after a restart.

- update_discovery_match (core/discovery/endpoints.py) now only does the in-memory
  result update when the state exists; the durable discovery-cache write always
  runs, falling back to client-provided original_name/original_artist when there's
  no in-memory state. With neither a state nor originals it still 404s (unchanged).
- The FIX confirm (wishlist-tools.js) now sends original_name/original_artist
  (from the source track it already has) so the backend can key the cache.

Covers all sources that share the helper (tidal/deezer/qobuz/spotify-public).
Tests: no-state-but-originals saves the cache + returns success; no-state-no-
originals still 404s; existing with-state path unchanged. 73 discovery tests pass.
2026-06-10 18:49:32 -07:00
BoulderBadgeDad
40e3dac881 Sync: append mode preserves the playlist image like reconcile (#811)
#811 (reopen of #792, carlosjfcasero, Emby/Jellyfin): "append" mode clobbered
the playlist's custom image. The post-sync image push only excluded
'reconcile' — so append (which edits the playlist in place via
append_to_playlist) still re-pushed the source image over the user's poster
every sync. Now both in-place modes (reconcile + append) skip the image push;
only the destructive 'replace' (recreate-from-scratch) pushes it.

append_to_playlist + set_playlist_image were verified to NOT touch tracks or
description (image push only POSTs /Images/Primary), so this is the identity-
clobber fix for append.

Tests: append + reconcile preserve the image, replace still pushes it.
2026-06-07 23:25:42 -07:00
BoulderBadgeDad
c20150e370 #799: stop mirrored discovery from reverting a manual fix to Wing It
A manually-fixed mirrored track silently reverted to 'Wing It' after re-running
discovery. Two compounding causes:

- extra_data is MERGED on save (update_mirrored_track_extra_data), and the
  manual-fix DB write (web_server.py) didn't clear the prior wing_it_fallback
  flag — so a track fixed after being a Wing It stub kept wing_it_fallback=True.
- the Playlist Pipeline pre-scan checked wing_it_fallback BEFORE manual_match
  (if/elif), so the stale flag won: the track was re-discovered and, on a miss,
  fell back to Wing It — discarding the user's pick.

Fix: extracted the pre-scan gate into core.discovery.manual_match.should_rediscover
(manual_match checked FIRST = authoritative, regardless of leftover flags), and
the manual-fix write now also clears wing_it_fallback/unmatched_by_user. Behavior
is identical for every other branch — only the manual-vs-wing-it ordering changes.

Tested at the seam incl. the exact regression (wing_it_fallback + manual_match
both set -> skip). 227 discovery tests green.
2026-06-05 11:07:32 -07:00
BoulderBadgeDad
ef9af0cab5 Fix S110: log instead of pass in canonical search-query add 2026-06-04 21:17:37 -07:00
BoulderBadgeDad
bc2432d9f6 Fix #785 (cont.): also search canonical title in discovery worker
The scoring best-of only helps if the right candidates were returned. File/CSV
titles ('Artist - Title') made the search query carry the artist prefix; add
canonical-title search queries so the correct tracks are actually found, then
the scorer best-of matches them. Additive (extra queries only when the title
canonicalizes differently).
2026-06-04 21:16:41 -07:00
BoulderBadgeDad
6cb753e7a1 Fix #785: file/CSV playlists fail to match (raw 'Artist - Title' titles)
#768 added canonical_source_track to the live-sync matcher and the playlist
editor reconcile, but NOT to the two paths that actually run for file/CSV
mirrored playlists: the discovery worker (core/discovery/playlist.py) and the
DB-only matcher (core/discovery/sync.py). YouTube playlists are cleaned at
ingest, so they matched; file playlists fed the raw 'Arctic Monkeys - Do I
Wanna Know?' title into search+scoring and never matched the library's clean
'Do I Wanna Know?' → reported missing / shown as 'extra'.

Add a conservative canonical best-of to both: score with the raw title AND the
canonicalized one, keep the better. canonical_source_track only strips an
'<artist> - ' prefix when it equals the artist, so it can only add a candidate.

Tests: _canonical_best_score seam (file-style match / clean title scored once /
keeps original when better).
2026-06-04 21:13:23 -07:00
BoulderBadgeDad
939c660498 Fix #792: 'reconcile' playlist sync mode (edit in place, keep image/description)
Replace mode (default) deletes + recreates the server playlist every sync,
which wipes its custom image, description, and identity. Add an opt-in
'reconcile' sync mode that edits the existing playlist in place — adds the
tracks now in the source, removes the ones gone — without destroying the
object, so the user's custom art/description survive.

- Pure planner plan_playlist_reconcile(current, desired) -> {add, remove}.
- Per-client reconcile_playlist: Plex addItems/removeItems on the same object;
  Navidrome Subsonic updatePlaylist delta (songIdToAdd / descending
  songIndexToRemove); Jellyfin add + remove-by-PlaylistItemId on /Playlists/{id}/Items.
- sync_service: reconcile branch with a replace FALLBACK (if a server's in-place
  edit is unavailable/fails, sync still succeeds destructively — logged loudly).
- Default stays 'replace' (no behavior change). New Settings > Playlist sync mode
  picker (replace/reconcile/append) backed by playlist_sync.mode; per-request
  sync_mode still overrides.
- Reconcile skips the post-sync source-image push so a custom poster isn't
  re-clobbered (the bug).

Tests: planner (add/remove/dedupe/order/empty) + reconcile-or-replace dispatch
(success / false-fallback / exception-fallback / no-method). Per-server in-place
API calls need dev validation against real Plex/Jellyfin/Navidrome.

NOTE: opt-in only; default behavior unchanged.
2026-06-04 15:15:49 -07:00
kekkokk
0b1fdba2a1 Fix standalone mirrored playlist sync and post-sync downloads.
SoulSync standalone matches library tracks without Plex fetchItem,
reports missing counts correctly, and skips server playlist writes.
Automation re-syncs when the mirror grows; after sync finishes, starts
organize download (organize-by-playlist) or wishlist processing.

UI: Spotify URL playlist-folder controls, organize toggle layout in the
discovery modal, reload organize preference when reopening Download Missing.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-04 00:24:00 +02:00
Francesco Durighetto
9ff2e7084a Fix organize-by-playlist downloads: library entries, wishlist, and stale Spotify cache
Persist organize_by_playlist on mirrored playlists and run playlist-folder
downloads from the auto-sync pipeline instead of the global wishlist phase.
Register SoulSync library rows after playlist-folder post-processing, route
failed organize batches to the wishlist correctly, and skip sync-time
unmatched wishlist only when organize download handles retries.

Invalidate stale playlist track caches on refresh (Spotify and Deezer ARL),
re-mirror on refetch, and improve standalone playlist modals (re-analysis,
Open in Mirrored). Add filesystem missing-track detection and tests.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-03 10:26:32 +02:00
BoulderBadgeDad
482d5fbc79 Fix: Spotify sync crash 'unexpected keyword argument candidate_pool'
When no media server is connected, discovery/sync patches sync_service's
matcher with a database-only implementation. sync_service calls it as
_find_track_in_media_server(track, candidate_pool=...) (a per-artist candidate
cache), but the database-only override took only (spotify_track) — so every
sync raised 'database_only_find_track() got an unexpected keyword argument
candidate_pool' and aborted.

Lift the override from a nested closure to a module-level
_database_only_find_track(spotify_track, candidate_pool=None) so it (a) accepts
the kwarg for interface parity with the real matcher and (b) is importable and
unit-testable. The DB-only path queries the library directly via
check_track_exists, so it accepts but doesn't need the candidate cache. Also
dropped the dead original_find_track local.

Tests: signature includes candidate_pool; called with candidate_pool={} returns
(None, 0.0) on no match; returns the match when the DB has it.
2026-05-31 23:00:24 -07:00
BoulderBadgeDad
d5f6a14ba1 Discovery lift (10/N): save_*_bubble_snapshot -> shared helper
Final cluster: the four structurally-identical snapshot endpoints
(discover_downloads, artist_bubbles, search_bubbles, beatport_bubbles) ->
core.discovery.endpoints.save_bubble_snapshot(...), wired via
_save_source_bubble_snapshot. All four validate a payload key, persist via
db.save_bubble_snapshot(kind, items, profile_id=...), and return a count +
timestamp; they differ only by:
- payload_key ('downloads' for discover, 'bubbles' for the rest) + its
  no_data_error message.
- snapshot_kind, success_noun, and the info/except log subject + noun
  ("downloads"/"artists"/"albums/tracks"/"charts").

get_database / get_current_profile_id injected; get_json (request.json) invoked
inside the try, preserving the original 400/500 behavior incl. traceback dump.

Tests: +5 (missing key 400, None body 400, happy path with kind/profile/count/
timestamp, discover_downloads variant, exception -> 500). Full discovery suite:
210 passed.

web_server.py: -98 lines.
2026-05-28 18:17:36 -07:00
BoulderBadgeDad
4caf36deb1 Discovery lift (9/N): update_*_playlist_phase -> shared helper
Ninth cluster: update_<source>_playlist_phase for the five sources sharing the
identical validation + full-message response (Tidal, Deezer, Qobuz,
Spotify-Public, YouTube) -> core.discovery.endpoints.update_playlist_phase(...),
wired via _update_source_playlist_phase + the _PHASE_LIST/_PHASE_LIST_YT
constants.

Per-source params:
- valid_phases — YouTube additionally allows 'parsed'.
- apply_extra_fields — Deezer/Qobuz/Spotify-Public also persist
  download_process_id / converted_spotify_playlist_id from the body; Tidal and
  YouTube do NOT, so they pass False (kept strictly 1:1 — the generic won't
  apply those keys for them even if a caller sent them).
- not_found_message / error_label; get_json invoked inside the try.

NOT folded in: iTunes-Link — uses data.get('phase') (no "Phase not provided"
400) and returns a no-message payload.

Tests: +7 (404, missing-phase 400, invalid 400, happy path with extra-fields
suppressed, extra-fields applied when enabled, YouTube 'parsed' allowed,
exception -> 500). Full discovery suite: 205 passed.

web_server.py: -123 lines.
2026-05-28 18:07:16 -07:00
BoulderBadgeDad
50ebfbd82f Discovery lift (8/N): update_*_discovery_match -> shared helper
Eighth cluster, the heavyweights (~110 lines each). The fix-modal
update_<source>_discovery_match for the four sources with the identical
structure (Tidal, Deezer, Qobuz, Spotify-Public) ->
core.discovery.endpoints.update_discovery_match(...), wired via
_update_source_discovery_match. Applies the user-selected Spotify track to the
discovery result (status/artist/album/duration/spotify_data/match-count) and
writes the manual fix to the discovery cache.

Per-source pieces are params:
- source_log_label / error_label.
- original_track_key ('tidal_track' / 'deezer_track' / ...).
- original_artist_getter: Tidal handles string-or-object artists
  (first_artist_str_or_obj); the rest assume strings (first_artist_plain).
- web_server helpers (join/extract artist, build_fix_modal_spotify_data,
  cache-key, get_database, active-discovery-source) injected.
- get_json passed as a callable and invoked INSIDE the try, preserving the
  original's "request.get_json() inside try" behavior (malformed body -> 500).

NOT folded in (genuinely divergent): iTunes-Link (saves spotify_data directly
via a different cache signature), YouTube (multi-key original_track fallback),
ListenBrainz (entirely different unmatch-capable structure, no cache write),
Beatport.

Tests: +9 (extractors; 400/404/400 guards; full happy path with result
mutation + duration formatting + match-count + cache-save args; no-increment
when already found; cache error swallowed; get_json raise -> 500). Full
discovery suite: 198 passed.

web_server.py: -400 lines.
2026-05-28 17:58:02 -07:00
BoulderBadgeDad
17c9e9b7b9 Discovery lift (7/N): start_*_sync -> shared helper
Seventh cluster: start_<source>_sync for the five sources with the identical
flow (Tidal, Deezer, Qobuz, Spotify-Public, YouTube) ->
core.discovery.endpoints.start_sync(...), wired via _start_source_sync.

Validates phase, converts discovery results, seeds sync state, posts a
"... Sync Started" activity item, and submits to the sync executor. Per-source
pieces are params:
- sync_id_prefix (f"{prefix}_{key}"), not_found/not_ready messages, convert_fn.
- name/image accessors: Tidal reads an object (playlist_name_obj/
  playlist_image_obj), the rest a dict (playlist_name_strict/playlist_image_dict).
- activity_label vs error_label DIFFER for Spotify-Public ("Spotify Link
  Sync Started" activity, "Spotify Public" logs).
- submit_sync_task glue (_submit_sync_task) closes over sync_executor /
  _run_sync_task / get_current_profile_id so the helper stays global-free.

NOT folded in: iTunes-Link (no final info log), ListenBrainz (submits the
task WITHOUT a playlist_image_url arg), Beatport (extra debug logging, chart).

Tests: +6 (404, not-ready 400, no-matches 400, full happy path with
state/sync-infra/submit/activity assertions, resync phases allowed,
exception -> 500). Full discovery suite: 189 passed.

web_server.py: -172 lines.
2026-05-28 17:45:09 -07:00
BoulderBadgeDad
7b6615b65a Discovery lift (6/N): get_*_playlist_states -> shared helper
Sixth cluster: the bulk-hydration get_<source>_playlist_states endpoints for
the five sources that build the identical per-entry dict + {"states": [...]}
shape (Tidal, Deezer, Qobuz, Spotify-Public, iTunes-Link) ->
core.discovery.endpoints.get_playlist_states(states, *, error_label,
info_log_label=None), wired via _get_source_playlist_states.

iTunes-Link is the only one of the five without the "Returning N stored ..."
info log, so info_log_label is optional (iTunes passes None to suppress it).

NOT folded in: the YouTube/ListenBrainz get_all_*_playlists endpoints. They
return {"playlists": [...]} (different key) with a different field set
(url / created_at / playlist, no discovery_results) and filter out
mirrored_/profile-scoped entries — genuinely divergent, kept as-is.

Tests: +4 (list build + last_accessed bump + exact shape, empty, optional ids
default None, missing-required-field -> 500). Full discovery suite: 183 passed.

web_server.py: -116 lines.
2026-05-28 17:31:33 -07:00
BoulderBadgeDad
44b032b6c0 Discovery lift (5/N): reset_*_playlist -> shared helper
Fifth cluster: reset_<source>_playlist for the four sources with byte-
identical bodies (Tidal, Deezer, Qobuz, Spotify-Public) ->
core.discovery.endpoints.reset_playlist(states, key, *, label,
not_found_message), wired via _reset_source_playlist. Resets phase/status to
'fresh', clears discovery/sync fields, cancels any discovery_future, and
preserves the original playlist payload.

Left with their own bodies (genuinely divergent):
- YouTube: status -> 'parsed' (not 'fresh'), no download_process_id, logs the
  playlist name, "reset to fresh state".
- ListenBrainz: status -> 'cached', logs playlist title, returns
  {"success": True, "phase": "fresh"} (different payload), _lb_state_key.
- iTunes-Link: state.update(...), no info log, "iTunes Link reset to fresh
  phase".

Tests: +4 (404, full clear + playlist preserved + future cancelled, no-future
path, exception -> 500). Full discovery suite: 179 passed.

web_server.py: -100 lines.
2026-05-28 17:15:13 -07:00
BoulderBadgeDad
8a9ed677ab Discovery lift (4/N): get_*_discovery_status -> shared helper
Fourth cluster: get_<source>_discovery_status (all eight sources, Beatport
included) -> core.discovery.endpoints.get_discovery_status(states, key, *,
not_found_message, error_label), wired via _get_source_discovery_status.

Unlike sync-status, the discovery-status response shape is byte-identical
across every source (phase/status/progress/spotify_matches/spotify_total/
results/complete), so Beatport folds in here too. Only the 404 string
("... discovery not found" vs "... playlist not found" vs "Beatport chart
not found") and the except-log label vary. ListenBrainz key via _lb_state_key.

NOT touched this cluster: get_*_playlist_state (the sibling endpoints).
Those genuinely diverge per source — different id-key name (playlist_id /
url_hash / playlist_mbid), presence of url / created_at / download_process_id,
Tidal's playlist.__dict__ serialization, and YouTube's strict (non-.get)
field access. Folding them would need a flag pile that wouldn't be a clean
1:1, so they keep their own bodies.

Tests: +4 (404, full response + last_accessed bump, complete=False when not
'discovered', missing-field -> 500). Full discovery suite: 175 passed.

web_server.py: -155 lines.
2026-05-28 17:00:20 -07:00
BoulderBadgeDad
aad1d2b8f0 Discovery lift (3/N): get_*_sync_status -> shared helper
Third cluster: the get_<source>_sync_status routes (Tidal, Deezer, Qobuz,
Spotify-Public, iTunes-Link, YouTube, ListenBrainz) -> core.discovery.
endpoints.get_sync_status(...), wired via _get_source_sync_status glue.

This cluster carries the real per-source quirks, all captured 1:1 as params:
- not_found_message (iTunes-Link uses "iTunes Link not found").
- error_label vs activity_subject — these DIFFER for Spotify-Public: the
  activity feed says "Spotify Link playlist ..." while the except log says
  "Error getting Spotify Public sync status".
- playlist-name accessor, three styles lifted verbatim as named helpers:
  playlist_name_attr_or_unknown (Tidal: object .name), playlist_name_strict
  (Deezer/Qobuz/Spotify-Public/iTunes: state['playlist']['name'], can raise),
  playlist_name_safe (YouTube/ListenBrainz: .get default). The strict getter
  preserves the original's behavior of raising -> 500 AFTER phase/sync_progress
  were already mutated.
- ListenBrainz key via _lb_state_key (caller-resolved).

Beatport stays separate (different payload: status not sync_status, sync_id,
no lock, chart key).

Tests: +9 (3 name accessors incl. raise/fallback semantics; status 404s,
running-no-mutation, finished+activity, error+revert+activity, and strict-
getter-missing -> 500 after partial mutation). Full discovery suite: 171 passed.

web_server.py: -244 lines.
2026-05-28 16:51:16 -07:00
BoulderBadgeDad
2d76a7c061 Discovery lift (2/N): cancel_*_sync + delete_*_playlist -> shared helpers
Second cluster. Two more sets of byte-identical per-source bodies:

cancel_<source>_sync (Tidal, Deezer, Qobuz, Spotify-Public, iTunes-Link,
YouTube, ListenBrainz) -> core.discovery.endpoints.cancel_sync(states, key,
*, label, not_found_message, sync_lock, sync_states, active_sync_workers).
Returns (payload, status_code); a thin web_server glue (_cancel_source_sync)
wires the sync-infra globals + jsonify. Caller passes the resolved key
(ListenBrainz transforms via _lb_state_key) and the exact 404 string
(iTunes-Link uses "iTunes Link not found").

delete_<source>_playlist (Tidal, Deezer, Qobuz, Spotify-Public) ->
delete_playlist_state(states, key, *, label, not_found_message), wired via
_delete_source_playlist.

Intentionally left with their own bodies (genuinely divergent, not 1:1):
- Beatport cancel (cancels a stored sync_future, no message, warning log).
- iTunes-Link / YouTube / ListenBrainz / Beatport deletes (different
  success messages, info-log wording, playlist-name extraction, /remove
  route, chart key).

Tests: +11 in tests/discovery/test_discovery_endpoints.py covering cancel
(404, active-worker cancel + state revert, worker-absent, no-sync-in-progress,
label in message, exception->500) and delete (404, future cancel + removal,
no/falsy future, exception->500 leaves state). Full discovery suite: 162 passed.

web_server.py: -216 lines.
2026-05-28 16:12:51 -07:00
BoulderBadgeDad
628395eda5 Discovery lift (1/N): convert_*_results_to_spotify_tracks -> shared helper
First cluster of the per-source playlist-discovery deduplication. The
convert_<source>_results_to_spotify_tracks functions (Tidal, Deezer, Qobuz,
Spotify-Public, YouTube, ListenBrainz) plus the already-generic
_convert_link_results_to_spotify_tracks were byte-identical apart from the
source label used in their log line.

Lift the shared body into core/discovery/endpoints.py as
convert_results_to_spotify_tracks(results, source_label); the 7 web_server
functions become 1-line delegations (names/signatures unchanged, so all
callers and behavior are identical — 1:1).

Beatport is intentionally NOT folded in: its converter coerces artist
objects to strings and emits a different track shape (source field, album
dict), so it keeps its own implementation.

Tests: tests/discovery/test_discovery_endpoints.py (12) pin both input
shapes (manual spotify_data / auto spotify_track+found), optional
track/disc numbers, falsy-0 omission, field defaults, skip-on-neither,
order preservation, if/elif precedence, empty input.

web_server.py: -209 lines. Full discovery suite: 151 passed.
2026-05-28 15:57:23 -07:00
Broque Thomas
997732ee63 Wishlist: fix three regressions causing all imports to land as track 01 with no year
Real-world regression triggered by the album-bundle work earlier in
2.6.3. Tracks with full Spotify metadata were importing as
``01 - <title>`` under ``Artist - Album/`` (no year), even when the
source filename carried the correct track number and Spotify's
release_date was available.

Investigation via DB inspection of stored wishlist rows:

```
"Never Gonna Give You Up" → track_number=None,  release_date=""
"idfc"                    → track_number=1,    release_date=""
"No Sleep Till Brooklyn"  → track_number=1,    release_date=""
```

Source-of-truth Spotify metadata had release_date AND real track
positions, but the wishlist row was poisoned. Three regressions
compounded the loss:

**Fix A — ``track_object_to_dict`` (``core/wishlist/payloads.py:295``)
preserved only album.name during Track→dict conversion.**

Pre-fix:
```python
album_name = "Unknown Album"
if hasattr(track_object, "album") and track_object.album:
    if hasattr(track_object.album, "name"):
        album_name = track_object.album.name
    else:
        album_name = str(track_object.album)

result = {
    ...
    "album": {"name": album_name},   # ← release_date / images / etc. all dropped
    ...
}
```

When a wishlist payload arrived as a Track dataclass instead of a
raw spotify_data dict, the Track→dict conversion stripped
release_date, images, album_type, total_tracks, id, and album-level
artists. Every wishlist row added through this path landed in the
DB with ``album={'name': X}`` only.

Post-fix: three branches handle the three album shapes
- ``album_attr`` is a dict → ``dict(album_attr)`` preserves every key
- ``album_attr`` is a sub-object → pull all common Album-dataclass
  attrs (id, release_date, album_type, total_tracks, images, ...)
- ``album_attr`` is a bare string → build a dict from the track
  object's adjacent attrs (release_date, album_id, album_type, ...)
  and surface ``image_url`` as ``album.images``

**Fix B — ``core/discovery/playlist.py:309`` only added
``track_number`` / ``disc_number`` keys when truthy.**

Pre-fix:
```python
matched_data = { 'id': ..., 'name': ..., ... }   # no track_number / disc_number
if track_number:
    matched_data['track_number'] = track_number
if disc_number:
    matched_data['disc_number'] = disc_number
```

Deezer-sourced matches always hit this branch with ``track_number=None``
because the cache enrichment at line 304 reads ``_raw.get('track_number')``
literally, but Deezer's raw shape uses ``track_position``. So the key
was omitted from ``matched_data``, downstream consumers couldn't
distinguish "missing key" from "value is 1", and the chain silently
filled 1.

Post-fix: keys are ALWAYS present (None when unknown). Also adds a
``best_match.track_number`` fallback so the Track-dataclass-mapped
value (which DOES include ``track_position``→``track_number``
mapping) gets used when the cache lookup misses.

**Fix C — Pipeline only consulted ``album_info.track_number`` before
falling to the filename (``core/imports/pipeline.py:645``).**

VA-collection source files like ``417 Fountains of Wayne - Stacys
Mom.flac`` have a leading playlist-position number that isn't the
album track number. The previous chain (album_info → filename →
floor-1) couldn't recover the real position because the filename
extractor either returned 417 (wrong) or None (caught by the floor).
But the wishlist payload's ``track_info.spotify_data.track_number``
HAD the right answer all along — Spotify says Stacy's Mom is track
3 on Welcome Interstate Managers.

Post-fix: resolution chain extracted into ``core/imports/track_number.py:resolve_track_number``
as a pure function:
1. ``album_info.track_number`` (album-bundle dispatch authoritative)
2. ``track_info.track_number`` (per-track flow payload)
3. ``track_info.spotify_data.track_number`` (nested fallback)
4. ``extract_explicit_track_number(file_path)`` (filename, returns
   0 when no numeric prefix — vs the default helper that returns 1)
5. Caller (pipeline) applies the final >=1 floor

Each step coerces to a positive int or falls through to the next.
Pure function = unit-testable in isolation = single place to fix
the rule.

**Test coverage (37 new tests):**

- ``tests/wishlist/test_payloads.py`` (+4) — Track→dict conversion
  preserves full album dict (dict / object / string album shapes) +
  None-track-number stays None.
- ``tests/discovery/test_discovery_playlist.py`` (+2) — matched_data
  always includes track_number/disc_number keys (None when unknown)
  + falls back to best_match attrs when cache misses.
- ``tests/imports/test_track_number_resolver.py`` (+16) — every
  resolution-chain branch pinned: album_info-wins, track_info
  fallback, spotify_data nested, JSON-string parsing, garbage-string
  fall-through, zero / negative / non-numeric / string-numeric
  coercion, filename fallback, explicit extractor vs default
  extractor semantics, defensive None inputs, VA-collection
  filename behaviour, all-sources-missing → None.

1571 wider-suite tests pass (wishlist + imports + discovery +
downloads + metadata). Ruff clean.

**Migration note:** existing wishlist rows that were saved under
the OLD ``track_object_to_dict`` (with stripped album metadata) still
have ``release_date=''`` in the DB blob. Those won't self-heal — the
next attempt loads from the poisoned blob. Users can remove + re-add
those tracks to refresh, or wait for the next sync run that
re-discovers them with full metadata. No automatic migration shipped
in this PR (scope creep — the forward path is fixed, backfill is a
separate concern).
2026-05-27 15:39:22 -07:00
Broque Thomas
8dbbf13c61 Branch cleanup: lift manual-match helpers, fix length-pref ordering, profile-scope view toggle
Self-review pass on the prior three commits — kettui-style cleanup
that should have landed first time.

**Length-preference sort ordering (real bug):**
The `search_tracks_with_artist` stable sort that promoted length-known
recordings ran in `core/musicbrainz_search.py`, but the MB endpoint in
`web_server.py:search_musicbrainz_tracks` runs `rerank_tracks` after
it — which re-sorts by relevance score and dropped the length-pref
ordering down to tiebreaker-only. For canonical-same-song MB duplicates
that all score identically the tiebreaker survived, but the
order-of-operations was wrong.

Moved into `rerank_tracks` itself via a new `prefer_known_duration`
flag. Sort key sits between relevance score and the stable-order
tiebreaker so relevance still wins (length only decides ties, never
overrides a higher-relevance match). The MB endpoint opts in via
`prefer_known_duration=True`; Spotify / iTunes / Deezer callers stay
on the default-off path since their search results always include
length. Pinned with three new `TestRerankTracks` cases:
ties-promote-length, relevance-still-wins, default-off-unchanged.

**Route logic lifted to `core/discovery/manual_match.py`:**
Two pieces lived as inline route logic in `web_server.py` — the
`derive_manual_match_provider` fallback chain (payload.source →
active source → 'spotify') used by `update_youtube_discovery_match`,
and the `is_drifted_for_redo` predicate (cached provider differs from
active AND not manual_match) used by `prepare_mirrored_discovery`.
Per kettui's "extract logic from web_server.py, don't AST-parse it"
standard, both helpers now live in `core/discovery/manual_match.py`
with 12 dedicated unit tests covering fallback resolution order,
non-dict payload defenses, manual_match exemption from drift,
absent-provider legacy default, and edge cases.

Side benefits from the lift:
- `match_source` now derived once before the cache-save try block
  instead of being duplicated in try + except (the except block existed
  only because the original used `match_source` later — pre-computing
  killed the duplication).
- `prepare_mirrored_discovery`'s `has_cached` check now reuses
  `is_drifted_for_redo` with inverted polarity instead of restating
  the field whitelist inline, so a future schema change only has to
  land in one place.
- The mirrored-DB persist block now gates on `matched_data is not None`
  to avoid a pre-existing latent NameError if the cache-save block
  raised before matched_data construction.

**Enhanced toggle localStorage key now profile-scoped:**
`soulsync-library-view-mode` was global — two admin profiles would
share one preference. Wrapped in `_libraryViewModeKey()` which appends
`:${currentProfile.id}` when a profile is loaded, falls back to the
unsuffixed key otherwise (preserves pre-multi-profile saved values).

Tests:
- 12 new in `tests/discovery/test_manual_match.py` pinning both helpers.
- 3 new in `tests/metadata/test_relevance.py` pinning the
  `prefer_known_duration` semantics.
- `test_search_tracks_with_artist_prefers_results_with_known_length`
  renamed to `_does_not_resort_by_length` since the sort moved out of
  this method. 664 tests pass across discovery + metadata suites.
2026-05-27 07:43:21 -07:00
Broque Thomas
39f582a690 Mirrored playlist: stop Playlist Pipeline from reverting manual Fix-popup matches
User reported that manually mapping a mirrored-playlist track via the
Fix popup (either by search or by pasting an MBID) worked end-to-end
once — match saved, library track downloaded — but the next Playlist
Pipeline run flipped the track back to "Provider Changed" and forced
them to re-do the manual map every cycle.

Three independent issues were combining to cause this:

1. Hardcoded `provider: 'spotify'` on manual-fix save
   `update_youtube_discovery_match` (the endpoint the Fix popup posts
   to, also used by mirrored playlists since the frontend routes
   `platform === 'mirrored'` through the YouTube endpoint) always
   stamped the cached match as Spotify-provided. The Fix-popup cascade
   actually queries the user's primary metadata source first and falls
   back to Spotify / Deezer / iTunes / MusicBrainz — so a user on
   MusicBrainz primary picking an MB result still had it saved as
   `provider: 'spotify'`. The next prepare-discovery call (which
   compares cached_provider to the active source) then immediately
   classified the match as drifted and pending re-discovery. Fixed by
   deriving `match_source` from `spotify_track.get('source')` (every
   *_search_tracks endpoint stamps `source` on results) with a fallback
   to `_get_active_discovery_source()` for the MBID-paste path (which
   uses the lean flat shape that doesn't carry source). `matched_data['source']`
   and the mirrored `extra_data['provider']` both now use the derived
   value. `match_source` is also recomputed in the cache-save except
   handler so the downstream mirrored-DB save still has it.

2. Discovery worker re-queueing manual matches as "incomplete"
   `run_playlist_discovery_worker` in `core/discovery/playlist.py`
   re-adds any track to `undiscovered_tracks` when its `matched_data`
   lacks `track_number` or `album.id` / `album.release_date`. The
   check was designed as a legacy-fix backfill for old discoveries
   that lost those fields to a Track-dataclass stripping bug. But
   manual fixes from the popup are *intentionally* lean — search-
   result rows don't include `track_number` (none of the search
   endpoints return it), and the MBID-lookup flat shape doesn't
   carry `album.id` / `release_date` (the recording lookup returns
   only `album.name`). So every manual match looked "incomplete" and
   got re-discovered every pipeline run, overwriting the user's pick
   with whatever the auto-search ranked first. Manual matches now
   short-circuit ahead of the incomplete-data branch.

3. `prepare_mirrored_discovery` ignored the `manual_match` flag
   Independent of the provider-stamping fix above, the prepare-
   discovery endpoint that powers the mirrored-playlist UI did its
   own `cached_provider != current_provider` check and didn't honour
   manual_match either. Defence in depth — even if a future code
   path stamps the wrong provider on a manual match, the flag now
   anchors it as cached. `has_cached` also extended so manual
   matches with off-provider stamps still count toward the cached
   tally for phase classification.

Tests:
- new `test_manual_match_skipped_even_when_matched_data_incomplete`
  in `tests/discovery/test_discovery_playlist.py` pins the worker
  short-circuit using a realistic MB-shape matched_data (album dict
  without id / release_date, no top-level track_number). 16 existing
  tests still green; 848 across discovery / metadata / automation
  suites pass.
2026-05-27 06:59:58 -07:00
Broque Thomas
246503066b Fold provider-matching into PlaylistSource contract (Phase 1b)
Adds ``discover_tracks(tracks) -> List[NormalizedTrack]`` to the
PlaylistSource interface. Sources whose tracks already carry
provider IDs (Spotify, Tidal, Qobuz, YouTube, Deezer, Spotify
public, iTunes link, SoulSync Discovery) inherit a no-op default;
ListenBrainz + Last.fm override to run the matching engine.

This closes the last gap before LB / Last.fm / SoulSync Discovery
can land as Sync-page mirror sources: the refresh handler now
calls ``source.discover_tracks(...)`` whenever a source returns
tracks with ``needs_discovery=True``, so mirrored LB rows arrive
already discovered + ready for the sync pipeline. Previously, LB
playlists ran through a separate state-machine worker tied to the
Discover-page UI, with results stored in ``discovery_cache``
instead of ``mirrored_playlist_tracks.extra_data``.

Changes:

- ``core/playlists/sources/base.py`` — PlaylistSource switches from
  Protocol to ABC so a concrete default for ``discover_tracks``
  can live on the base class. The four real-work methods stay
  ``@abstractmethod``; instantiating an adapter that forgets one
  fails loudly at construction.
- ``core/discovery/matching.py`` (new) — pure ``match_mb_tracks``
  helper that runs Strategy-1-only matching-engine queries against
  Spotify (primary) or iTunes (fallback). No state machine, no
  discovery-cache writes, no wing-it stub — that richer flow stays
  in ``core/discovery/listenbrainz.py`` for the Discover-page UI.
- ``ListenBrainzPlaylistSource`` + ``LastFMPlaylistSource`` take
  an optional ``discover_callable`` constructor arg. Last.fm reuses
  the LB implementation since the track shape is identical.
- ``bootstrap.build_playlist_source_registry`` accepts a
  ``discover_callable`` kwarg and wires it into LB + Last.fm
  adapters.
- ``web_server.py`` boot constructs the discovery callable from the
  existing matching engine + ``_discovery_score_candidates`` +
  Spotify / iTunes clients, passes through to the registry.
- ``refresh_mirrored.py`` adds a small ``_maybe_discover`` helper
  that calls ``source.discover_tracks(...)`` between fetch and
  ``to_mirror_track_dict`` projection — only fires when at least
  one track has ``needs_discovery=True``, so the normal Spotify /
  Tidal / etc. refresh path stays a zero-cost pass-through.

Tests:

- 5 new adapter tests: default no-op pass-through, LB discovery
  with mixed matches/misses, LB no-callable fallback, Last.fm
  shares the LB implementation, mirror-dict spotify_hint emit.
- 1 new automation test: end-to-end LB refresh with a stub
  discover_callable proves the matched_data lands in
  ``mirror_playlist_tracks.extra_data`` after the registry
  refresh + discover hop.

225 tests across adapter + automation suites green.
2026-05-26 13:07:01 -07:00
Broque Thomas
718eb0cb10 Add iTunes / Apple Music link import tab on Sync page
New iTunes Link tab between Deezer Link and YouTube. Accepts album,
track, and playlist URLs from music.apple.com / iTunes. Pulls the
tracklist, runs it through the same discovery -> sync -> download
pipeline as the other link tabs.

Apple Music playlists go through amp-api with a Bearer JWT scraped
from the SPA. The legacy meta-tag and inline `"token":"..."` paths
are gone in the current music.apple.com SPA, so the extractor now
walks the page's `<script src>` list (prioritising index/chunk/main
bundles), fetches up to 8 JS bundles, regex-matches JWT-shaped
strings, and base64-decodes each payload to confirm it carries
Apple media-api claims (`root_https_origin`, or `iss + iat + exp`)
before trusting it. Filters out analytics / error-reporter JWTs that
also ship in the bundle.

Tokens are cached at module scope for 6h behind a threading.Lock so
the three-worker discovery executor doesn't thunder-herd Apple on
cold start, and amp-api calls go through a single helper that on
401 invalidates the cache, refetches the page, force-refreshes the
token, and retries the request once. The playlist fetcher memoises
the page HTML for the cache-miss path so we don't refetch it for
every paginated `/tracks` page.

spotify_public discovery worker accepts the new platform shape so
iTunes Link reuses the same matching code path as Deezer Link and
Spotify-public. UI bits live in the sync-services.js iTunes Link
tab, with platform plumbing through wishlist-tools.js for the
multi-source state map.
2026-05-25 22:32:18 -07:00
Broque Thomas
a34eae1445 Add Qobuz playlist sync to Sync page (#677)
Qobuz joins Tidal and Deezer as a first-class playlist sync source.
New Qobuz tab on the Sync page lists user playlists + a virtual
Favorite Tracks entry, and clicks route through the same discovery →
sync → download pipeline the other services already use.

Backend:
* core/qobuz_client.py — new get_user_playlists, get_playlist,
  get_user_favorite_tracks, get_user_favorite_tracks_count. Returns
  normalized dicts (matches Deezer client shape, not Tidal's
  dataclasses) so the discovery worker can iterate directly without
  duck-typing. Virtual `qobuz-favorites` ID dispatches to favorites
  fetcher inside get_playlist — same trick Tidal uses with
  COLLECTION_PLAYLIST_ID. Both list endpoints paginate against
  Qobuz's 500-cap limit.
* core/discovery/qobuz.py — new worker module. Mirrors
  core/discovery/deezer.py: pause enrichment, iterate tracks,
  hit discovery cache, fall back to _search_spotify_for_tidal_track,
  build wing-it stub on miss, sync results to mirrored playlist.
* web_server.py — adds /api/qobuz/playlists, /playlist/<id>,
  /discovery/start/<id>, /discovery/status/<id>, /discovery/update_match,
  /playlists/states, /state/<id>, /reset/<id>, /delete/<id>,
  /update_phase/<id>, /sync/start/<id>, /sync/status/<id>,
  /sync/cancel/<id>. One-for-one with the Tidal + Deezer endpoint
  sets. Qobuz discovery executor registered for clean shutdown.

Frontend:
* webui/static/sync-services.js — full handler set (loadQobuzPlaylists,
  createQobuzCard, openQobuzDiscoveryModal, startQobuzDiscoveryPolling,
  startQobuzPlaylistSync, startQobuzSyncPolling, cancelQobuzSync,
  startQobuzDownloadMissing, rehydrateQobuzDownloadModal, etc.).
  Reuses the shared YouTube discovery modal via fake `qobuz_<id>`
  urlHash and is_qobuz_playlist flag. Shared switch statements in
  getModalActionButtons / generateTableRowsFromState / Wing It helpers
  in downloads.js gain new isQobuz branches alongside the existing
  per-service ones.
* webui/index.html — new Qobuz tab button + content div, slotted
  between Deezer and Deezer Link.
* webui/static/style.css — new .qobuz-icon for the tab icon.
* webui/static/core.js — qobuzPlaylists / qobuzPlaylistStates /
  qobuzPlaylistsLoaded globals.

Followed the existing per-service pattern verbatim rather than
refactoring the duplicated transformers across Tidal / Deezer /
Spotify-public / YouTube / Mirrored — that refactor is its own follow-up
PR per the "don't break Tidal/Deezer" scope discipline. Adding the 6th
copy of a proven pattern is lower risk than collapsing 5 working
services behind a new abstraction.

Tests:
* tests/test_qobuz_playlists.py — 12 tests covering pagination,
  normalization, favorites virtual-ID routing, artist-name fallback
  chain (performer → album.artist → 'Unknown Artist'), and
  unauthenticated short-circuits.
2026-05-23 23:27:36 -07:00
Broque Thomas
5bc5fbb662 Add MusicBrainz as a metadata source
Register MusicBrainz as a first-class metadata source alongside Deezer, iTunes, Spotify, Discogs, and Hydrabase. Expose the shared client through metadata services, add the settings option, and expand the MusicBrainz search adapter with source-compatible artist, album, track, and detail methods.

Carry MusicBrainz IDs through similar-artist discovery, recommended artists, artist map serialization, and personalized playlist selection. Update DB migrations and lookup filters so similar_artist_musicbrainz_id is preserved on older schemas and used for source requirements and library exclusion.

Normalize MusicBrainz album adapter output for import context and add regression coverage for registry mapping, typed album conversion, and similar-artist filtering. Verified by user with 120 focused tests passing.
2026-05-18 18:47:13 -07:00
Broque Thomas
08725094db get_current_profile_id: catch RuntimeError so background callers don't crash
Reproduced on the personalized playlist pipeline: selecting Fresh Tape
(or any kind) and running the automation surfaced
"Working outside of application context" in the UI.

Root cause: `get_current_profile_id` reads Flask's `g.profile_id` and
only catches `AttributeError`. Outside a request — automation engine,
sync threads, watchlist scanner — `g` raises `RuntimeError` instead,
so the except misses and the handler dies.

Mirrored playlist pipeline never hit this because it hardcodes
profile_id=1 in its sync call. The personalized pipeline calls
`deps.get_current_profile_id()` from a background thread, which is
what tripped the bug. Fresh Tape's generator also resolves the
profile via the same function — same path, same crash.

Fix: broaden the except to `(AttributeError, RuntimeError)` in all
three copies of the helper (`web_server.py`, `core/artists/map.py`,
`core/discovery/hero.py`). All three now safely degrade to profile_id=1
(admin profile) when called outside a request context — matches the
existing intent that single-admin installs Just Work.

No test changes — the existing pipeline tests stub the helper, so
they never exercised the bug. The fix is in the layer above the
stubs.
2026-05-15 21:06:17 -07:00
Broque Thomas
d9529fc801 Token leak round 2: artist endpoint + playlist sync + URL-encoded redaction
The first token-leak fix scrubbed the artwork URL fixer's own log
calls. This catches three more sites that ALSO leaked tokens, plus
one upstream gap that let URL-encoded tokens slip through the
redactor.

Three sites in `web_server.py` (artist endpoint at line 8765-8773):

- "Artist image before fix: '...'" -- logged the raw image_url with
  the auth token in plain form.
- "Artist image after fix: '...'" -- logged the URL-encoded form
  after it had been wrapped in the image proxy
  (`/api/image-proxy?url=<percent-encoded-token>`).
- "Final artist data being sent: {...}" -- dumped the entire
  artist_info dict on every render, including the image_url field.

All three were dev-time debug noise. Removed entirely. The "No
artist image URL found" warning at line 8770 stays (no URL, just
the artist name).

One site in `core/discovery/sync.py:402`:

- "[PLAYLIST IMAGE] image_url=..." -- logged the playlist poster URL
  during sync. Same auth-token leak risk for Plex / Jellyfin
  playlists. Changed to log only `has_image=True/False`.

Upstream gap in `_redact_url_secrets`:

- The original regex only matched plain query params (`?key=value`).
  When an auth-bearing URL gets wrapped inside another URL's query
  string (our `/api/image-proxy?url=<encoded>` flow) the auth params
  end up percent-encoded -- `%3FX-Plex-Token%3D...` -- and slipped
  through.
- New second pattern catches the URL-encoded form. Both passes run
  on every redact call; idempotent.

Verified manually:
  /api/image-proxy?url=...%3FX-Plex-Token%3DABC...
  -> /api/image-proxy?url=...%3FX-Plex-Token%3D***REDACTED***

6 artwork tests pass.
2026-05-15 09:33:15 -07:00
Broque Thomas
6fe85f2f37 Server playlist sync: append mode (preserve user-added tracks)
Discord report (CJFC, 2026-04-26): syncing a Spotify playlist to the
server overwrote anything manually added to the server-side playlist.
The fix adds a per-sync mode picker next to the Sync button on the
playlist details modal — Replace (default, current delete-recreate
behavior) or Append only (preserves existing tracks, only adds new
ones). Useful when the source platform caps playlist size and the
user is manually building beyond it on the server.

Implementation:

* New `append_to_playlist(name, tracks)` method on Plex / Jellyfin /
  Navidrome clients. Each uses the server's NATIVE append API:
    - Plex: `existing_playlist.addItems(new_tracks)`
    - Jellyfin: `POST /Playlists/<id>/Items?Ids=...&UserId=...`
    - Navidrome: Subsonic `updatePlaylist?songIdToAdd=...`
  Falls back to `create_playlist` when the playlist doesn't exist
  yet (first sync). No delete-recreate, no backup playlist created
  (preserves playlist creation date + metadata + non-soulsync-managed
  tracks).
* Dedup-by-server-native-id (ratingKey for Plex, GUID for Jellyfin,
  song-id for Navidrome) — never re-adds a track already on the
  playlist. Server-native identity, not fuzzy title+artist match,
  so it can't false-collide.
* `sync_service.sync_playlist` accepts `sync_mode='replace'|'append'`
  kwarg. Single if/else branch dispatches to `append_to_playlist` or
  `update_playlist`. Threaded through `core/discovery/sync.run_sync_task`
  and the `/api/sync/start` HTTP handler. Validation on the API rejects
  unknown mode strings (defaults to 'replace').
* Frontend: per-playlist `<select id="sync-mode-${id}">` rendered next
  to the Sync button in both modal renderers (sync-spotify.js for
  Spotify playlists, sync-services.js for Deezer ARL playlists).
  `startPlaylistSync` reads the select at click time; missing select
  (other callers like discover.js) defaults to 'replace' so backward
  compat preserved without per-call-site updates.
* SoulSync standalone has no playlist methods at all and the modal
  hides the Sync button entirely on it via `_isSoulsyncStandalone` —
  dispatch never reaches that path, no defensive fallback needed.

15 new tests pin per-server append behavior:
  - missing playlist → create_playlist delegation
  - dedup filtering (existing IDs skipped, only new tracks added)
  - empty new-track set short-circuits without API call
  - failure paths return False without raising
  - contract listing (KNOWN_PER_SERVER_METHODS includes
    'append_to_playlist'; Plex / Jellyfin / Navidrome all implement)

Plus tests/discovery/test_discovery_sync.py fake `sync_playlist`
fixture got `sync_mode='replace'` default to match the new signature
(was breaking after the kwarg add; now passing).

WHATS_NEW entry under new '2.6.0' block (hidden by
`_getLatestWhatsNewVersion` until next release bump).

Closes CJFC discord request.
2026-05-10 22:52:11 -07:00