Commit graph

1514 commits

Author SHA1 Message Date
BoulderBadgeDad
4743dfd644 playlist export: opt-in confident-search backfill for the unmatched tail (#945, increment 7)
Adds the third resolver stage for tracks the discovery cache + library can't resolve — a live
search of the target service, gated behind a "Match missing tracks" toggle so the API cost is opt-in.

The whole point is coverage WITHOUT the wrong-track risk, so it's a CONFIDENT match, not "search
and grab":
- search_service_track_id(artist, title, search_fn): searches the service, reranks via the existing
  relevance scorer (filter_and_rerank), and returns the top hit's id ONLY if it clears
  BACKFILL_MIN_SCORE (1.2 on the score_track scale). A wrong-artist hit (no 1.5x exact-artist boost,
  caps ~1.0) or a karaoke/cover (x0.05) can't clear the floor → None, and the track is left out
  rather than added wrong. search_fn injected → unit-testable without a live service.
- resolve_service_track_ids gains an optional search_id_fn: cache → library → search. Tallies
  from_search separately.
- _run_service_export builds the search fn from the service's metadata search client only when
  job['backfill'] is set; the endpoint reads `backfill` from the body; the modal adds the toggle and
  the status line shows "(N matched live)".

Store-back of confident matches deferred: a mirrored-only track may have no library row to write to,
so persisting needs the track→library mapping — a follow-up, not correctness.

9 new tests incl. the safety ones: wrong-artist rejected, karaoke/cover rejected, real-over-cover
picked, fail-safe on search error, and the cache→library→search waterfall + toggle wiring (on/off).
28 export/orchestration tests green, 64 script-integrity green, ruff clean.
2026-06-28 23:02:08 -07:00
BoulderBadgeDad
37c8b06a27 playlist export: resolve IDs from the discovery cache first (#945, increment 6)
Boulder: "all 50 tracks are discovered to Deezer already — it's not using any of that." Right —
the export only checked tracks.deezer_id (library) and ignored the IDs discovery already resolved
and stored in each mirrored track's extra_data. So tracks that were discovered+downloaded but not
separately enriched showed as "not on Deezer" and got dropped.

Adds a per-track waterfall for service export:
- service_id_from_extra_data(track, service): the id discovery already matched, read from
  extra_data.matched_data.id — FREE (no API call) and reliable (it's the same id used to mirror
  the track). Trusted only when discovered ON the target service (provider == service); a
  wing_it_fallback (low-confidence guess) does NOT match here, so it falls through rather than
  risk a wrong track in the export.
- resolve_service_track_ids(tracks, service): cache (extra_data) → library stored id → unmatched.
  Reports from_cache / from_library / unmatched. _run_service_export now uses this instead of the
  artist/title MBID-style resolver.

For Boulder's playlist this means all 50 resolve straight from the cache — full coverage, zero API
calls. (A live confident-search backfill for the genuinely-missing remainder is the optional next
step, gated + thresholded.)

9 new tests: extra_data id only when provider matches + wing_it excluded + bad-json/not-discovered
guards, the cache→library→unmatched waterfall with stat tallies, and _run_service_export resolving
straight from the cache end-to-end. 49 export tests green, ruff clean.
2026-06-28 21:39:39 -07:00
BoulderBadgeDad
465a77f01d playlist export: Spotify/Deezer export job + endpoint (#945, increment 4)
Ties the resolver + write clients into a working backend, reusing the ListenBrainz export's
resolve→push→store-target shape:

- _run_service_export(job, db, playlist_id, title, service, client, resolve_fn): resolves the
  mirrored playlist's tracks to their stored service track IDs (id_key='service_track_id'),
  guards "nothing matched", pushes via the injected write client, and stores the returned
  playlist id as the export target so a re-export updates in place (idempotent, like LB #903).
  Deps injected → unit-testable without a DB or live service.
- _run_playlist_export dispatches mode in {spotify, deezer} to it (builds the real client +
  service resolver); the existing download/push (ListenBrainz/JSPF) flow is untouched.
- POST /api/playlists/<id>/export/service/<service> — distinct path so it can't collide with
  the existing /export/listenbrainz route; validates the target, starts the background job,
  returns {job_id} polled via the shared status endpoint.

5 orchestration tests (fake db/client/resolve_fn): success stores target + passes ids in order,
no-match → error with no push, client None → not-connected error, push failure surfaces the
client's error and stores nothing, re-export passes the existing target id. ruff clean.

Last piece: the modal options (Sync to Spotify / Deezer, gated on auth, unmatched count surfaced).
2026-06-28 21:20:06 -07:00
BoulderBadgeDad
37c2b9b569 spotify: playlist-write client + single-source OAuth scope (#945, increment 2)
For exporting a mirrored playlist back to Spotify:

- The OAuth scope string was duplicated verbatim in 5 places (spotify_client, the per-profile
  registry, and 3 web_server callbacks) — a drift hazard where the authorize URL and token
  exchange could request different scopes and silently re-prompt/deny. Extracted ONE
  SPOTIFY_OAUTH_SCOPE constant and pointed all 5 at it, and added playlist-modify-public/private
  there. Existing users re-auth once to grant write; reads are unaffected.
- SpotifyClient.create_or_update_playlist(name, track_ids, existing_id=None): creates a playlist
  owned by the authed user, or replaces an existing one's tracks in place (idempotent re-export).
  Chunks at Spotify's 100-track cap. A pre-scope token gets a clear "reconnect Spotify" message
  instead of a raw 403. Returns {success, playlist_id, url, added, error}.

6 tests: create-new adds tracks, update replaces (no create), >100 chunking, empty → error (no API
calls), not-authed → error, insufficient-scope → reconnect message. 268 spotify/oauth tests green,
ruff clean. Additive — read paths and existing tokens unchanged.

Next: Deezer write via the ARL gw-light gateway, then the export-job branch + endpoint + modal.
2026-06-28 21:12:53 -07:00
BoulderBadgeDad
c593a17ac2 spotify search endpoint: plain query, not field-scoped — fixes pool-fix "no results"
The Wing It pool "Fix Match" search returned "no results" for everything (even obvious
tracks). Root cause: /api/spotify/search_tracks built a Spotify field-filtered query
(track:X artist:Y) and handed it to spotify_client.search_tracks, which falls back to the
user's configured source when official Spotify isn't serving the request. The fallback
(Deezer here) got the raw Spotify `track:…artist:…` syntax it can't parse and aborted the
connection (RemoteDisconnected) — so the user's perfectly working Deezer failed ONLY on
this path, on this query format. The iTunes and Deezer search endpoints already dropped
field syntax for exactly this reason; the Spotify one was the lone holdout.

fix:
- new pure helper relevance.build_combined_search_query(track, artist, legacy) — plain,
  source-agnostic query; documents WHY field syntax is wrong here. the endpoint already
  reranks by expected title/artist, so precision is recovered without the brittle syntax.
- the Spotify endpoint uses it (now consistent with iTunes/Deezer).
- frontend (searchPoolFix): surface the real error (auth / 500 / upstream abort) instead
  of masking everything as "No results found" — which is what made this undiagnosable.

5 helper tests incl. the regression (output must contain no 'track:'/'artist:' syntax).
654 metadata/search tests green, 64 script-integrity green, ruff clean.
2026-06-28 19:48:04 -07:00
BoulderBadgeDad
5df873655f library reorganize: wire rename-only mode through the queue (#875)
Threads the rename_only flag from the apply endpoint to the executor, additively (default
False everywhere → existing full-flow behaviour byte-for-byte unchanged):

- /api/library/album/<id>/reorganize-files reads `rename_only` from the body → enqueue.
- QueueItem gains rename_only (+ surfaced in to_dict for the status panel).
- reorganize_runner.build_runner takes build_final_path_fn and branches: a rename_only item
  routes to reorganize_album_rename_only (no staging dir, no copy, no post-process); everything
  else falls through to the full reorganize_album. Staging is only created for the full path now.
- web_server injects build_final_path_fn (= _build_final_path_for_track, the same builder the
  preview uses) so apply matches the preview exactly.

Fixed a test landmine: _make_item returns a MagicMock, whose .rename_only is a truthy mock that
wrongly took the new branch — set it to False to match the real QueueItem default. +2 runner tests
(rename_only routes to the rename executor + creates no staging; missing path-builder → clean
setup_failed). 209 reorganize tests green, ruff clean.

Left: the modal (Full vs Rename-only) + optional post-rename server scan + the issue reply.
2026-06-28 19:08:42 -07:00
BoulderBadgeDad
a62d2d4310 ui appearance: default worker orbs OFF on Firefox for first-time users
The blurred 60fps worker-orb canvas is the main remaining Firefox lag source after the
#935 sweep (multiple Discord lag reports). So for a FIRST-TIME user with no saved
preference, default the orbs OFF on Firefox (smooth first impression where it's needed)
and ON everywhere else (full polish where the browser handles it). An explicit saved
choice ALWAYS wins — this only picks the default when the user hasn't chosen.

Done kettui-style with a SINGLE source of truth, not the dual browser-detection I first
floated (server UA + client _isFirefox would be the same fact in two places that can
drift — exactly the server/client class #943's green-flash fix just cleaned up):

- core/ui_appearance.py (new, pure + importable): is_firefox_user_agent +
  resolve_worker_orbs_default(explicit, is_firefox) — explicit wins, unset → !firefox.
- web_server: the SERVER decides (UA via _request_is_firefox, request-context-safe) and
  injects initial_worker_orbs_enabled; config default flipped None so "unset" is
  distinguishable from an explicit False. The client just consumes the injected value
  (init.js unchanged) — no client-side re-derivation of "is Firefox".
- settings.js: the orb checkbox default now reflects the server value when unset, so
  saving Settings can't silently flip a first-time Firefox user's orbs back on.

No regression: Chrome users unchanged; users with an explicit setting unchanged (it
wins regardless of browser); /api/settings returns raw config so it can't clobber the
default for an unset value. Verified end-to-end through a real Flask request context
(Firefox→off, Chrome→on, explicit wins both ways, no crash outside a request). 8 pure
seam tests pin the contract; ruff clean.
2026-06-28 16:50:22 -07:00
BoulderBadgeDad
6a388c43fc
Merge pull request #943 from nick2000713/ui/settings-page-cleanup
UI/settings page cleanup
2026-06-28 16:05:38 -07:00
dev
1382cb6117 Bootstrap saved appearance effects early 2026-06-29 00:01:56 +02:00
dev
b43e44219a Merge remote-tracking branch 'upstream/main' into ui/settings-page-cleanup
# Conflicts:
#	webui/static/style.css
2026-06-28 23:29:17 +02:00
BoulderBadgeDad
a1b0e787bc
Merge pull request #942 from HellRa1SeR/fix-spotify-oauth
Fix spotify oauth: credential normalization
2026-06-28 13:03:16 -07:00
dev
d4e2dccd73 Render saved appearance before CSS paints 2026-06-28 21:55:47 +02:00
BoulderBadgeDad
b72febbf1c manual search: INJECT the exact pasted-link track, don't rely on text search surfacing it (#932)
reopened by diegocade1: pasting a Qobuz track link still showed unrelated results. the earlier
fix (b1f061a) only BUBBLED the linked track to the top — but a pasted link is resolved to an
"artist title" text query and searched, and for an obscure track ("foreign lavennew" by colacola)
that text search returns broad lookalikes ("Foreign Bird", "Foreign Spies", …) and never the
actual track. nothing to bubble → user sees junk.

fix: since the link is already resolved via get_track(id), fetch that exact track AS a downloadable
result and inject it at the top (Qobuz downloads by id, so the result is fully usable). the text
search still runs for alternatives.

- QobuzClient.get_track_result(id): get_track + _qobuz_to_track_result; None on any failure.
- _qobuz_to_track_result gains require_streamable (default True for bulk search). the link fetch
  passes False: track/get may OMIT the streamable flag, which would default-False and wrongly drop
  the exact track the user explicitly asked for. (this closes the one shape assumption that
  couldn't be verified against a live Qobuz API — the track is no longer gated on it.)
- track_link.inject_linked_track_first(tracks, linked_result, id): pure seam — prepend the fetched
  result + drop any search duplicate; falls back to the bubble when no result was fetched.
- manual-search endpoint fetches linked_result defensively (getattr 'get_track_result') and calls
  the seam. Tidal/HiFi (get_track returns a dict but the converter wants an object — shape
  mismatch) have no get_track_result, so they keep the existing bubble path: NO regression.

14 tests: inject puts the fetched track first when search missed it / dedups a search copy / falls
back to bubble / str-safe id / noop; get_track_result convert/none/exception; and the REAL
converter builds a valid downloadable result from a track/get dict that OMITS streamable (search
path still rejects it). 85 track-link/qobuz tests green, ruff clean.
2026-06-28 12:42:20 -07:00
Siddharth Pradhan
579617f861 normalize spotify credentials during Oauth 2026-06-28 13:36:02 -04:00
BoulderBadgeDad
d72f6396f0 lint: justify 4 best-effort try/except/pass (S110) so CI ruff passes
the CI ruff gate flagged 4 S110s in code added this cycle: the /api/debug/memory/objects
endpoint (len() on an exotic object; optional psutil rss) and the GC sweeper (malloc_trim
resolution + the trim call — absent on musl/non-Linux). all are genuinely best-effort, so add
'# noqa: S110' with a one-line reason on each. ruff check . is clean.
2026-06-28 00:03:19 -07:00
BoulderBadgeDad
2b8f6f8611 Release 2.8.0: version bump + docker-publish tag + What's New / version modal + PR description
- web_server: _SOULSYNC_BASE_VERSION 2.7.9 -> 2.8.0
- docker-publish.yml: default version_tag -> 2.8.0
- pr_description.md: rewritten for 2.8.0 (preview-clip cleanup, unverified-queue self-heal #934,
  album-completeness split albums #936, clear-completed, youtube cookies, #937, discography
  speed, wishlist art, dashboard perf #935, bounded memory #802)
- helper.js: WHATS_NEW carries the 2.8.0 block + folded "Earlier versions" summary;
  VERSION_MODAL_SECTIONS leads with 2.8.0 highlights, rolls 2.7.9 into an aggregator
2026-06-27 23:58:22 -07:00
BoulderBadgeDad
d30273985f downloads: restore Clear Completed for persisted history (clears the whole completed list)
since 9a0e3b40 persisted completed downloads in the Downloads view, the Clear Completed button
was hidden for those rows and clear-completed only pruned live session tasks. after a restart
the page filled with persisted completed downloads with no way to clear them.

now Clear Completed clears BOTH:
  - live session completed/failed tasks (clear_completed_local, unchanged), AND
  - the persisted download-history tail: new clear_completed_download_history() deletes every
    library_history event_type='download' row, so the list actually empties and stays empty.

this includes unverified rows (the verification review queue) by design: on a library where
verification never confirmed the imports, ALL completed downloads are 'unverified', so preserving
them made the button a no-op. it only removes HISTORY rows — the actual files and their tracks
entries are untouched, so nothing in the library is lost, only the 'needs verification' flags.
the action confirms first (showConfirmDialog, destructive) and the button now shows whenever any
completed/failed row is present.

3 seam tests (clears all incl unverified; leaves non-download history; empty=0); reconcile +
orphan + JS integrity suites green.
2026-06-27 23:37:06 -07:00
BoulderBadgeDad
4487a9d2dc unverified-history reconcile + orphan-clean: hardening follow-up to #938
four fixes from the review (and a self-correction):

1) close the connection. reconcile_unverified_history_from_tracks opened a connection with no
   finally/close. runs once per boot so GC reclaimed it, but now it's consistent + robust.

2) scope the tracks scan to the review queue. it built lookup dicts from EVERY verified/
   human_verified track (~350k on a large library) on every boot while anything is unverified
   (the normal state). now it loads the stuck rows first and skips verified tracks whose path
   AND basename can't match any queued row, so dicts stay proportional to the queue, not the
   library. behaviour identical (all 13 PR reconcile tests still pass).

3) close the title-less basename collision. a title-less history row fell back to filename-only
   matching with no ambiguity check, so a generic name like "01 - Intro.flac" could heal a
   DIFFERENT song to verified. now a title-less basename heal only fires when that basename is
   unique among verified tracks; unique-basename rows still heal (recall preserved).

4) "Clean orphaned" protects force_imported rows (deliberate user decision, keep for human
   approval) without weakening the mount-down safety gate. CRUCIAL self-correction: filtering
   them out BEFORE the orphan check (my first cut) shrank the checked count below the threshold
   and would have let a few unverified orphans be deleted during a mount outage. instead,
   find_orphan_history_ids now takes a deletable predicate: protected rows still count toward
   checked / all-missing (gate stays strong) but never enter the orphan_ids delete set.

3 new regression tests (title-less collision; deletable protects from delete; protected rows
still count toward the gate). 936 verification/acoustid/history/downloads tests green. builds
on nick2000713's #938.
2026-06-27 22:35:16 -07:00
BoulderBadgeDad
d84fba14ba
Merge pull request #938 from nick2000713/fix/unverified-acoustid-934
Fix/unverified acoustid 934
2026-06-27 22:14:01 -07:00
BoulderBadgeDad
50c17ec9f7 discography 'add to wishlist': batch the per-track ownership check (fixes ~15-30s/track)
clicking Download Discography → Add all to wishlist added ~1 track every 15-30s. trace: the
endpoint's per-track library-ownership check (track_already_owned → check_track_exists) ran the
LEGACY path — firing search_tracks for every title-variation × artist-variation, per track. on
a large library and an artist you own NOTHING of, STRATEGY-1 (indexed LIKE) always missed and
fell through to the fuzzy fallback (full-table scan), ~10-15 scans/track = the 15-30s. metadata
fetch was never the bottleneck (deezer returns each album in ~1s).

fix: pre-fetch the artist's owned tracks ONCE (get_candidate_albums_for_artist →
get_candidate_tracks_for_albums) and pass candidate_tracks to check_track_exists's batched
in-memory path — the same path the discography backfill job + completion-stream already use.
pass an EMPTY list (not None) when nothing is owned so the owns-nothing case still takes the
fast path → instant. per-track cost drops from ~20s to ~1ms.

safe: track_already_owned's only real caller is this endpoint; the new param is optional
(None = unchanged legacy behaviour for any other caller). normal ownership still detected (same
artist-variation breadth); the one divergence is a track owned ONLY via a compilation → a
harmless redundant wishlist add, which is the endpoint's explicitly-accepted failure mode and
already how the backfill job behaves. 4 new tests; 1134 discog/metadata/wishlist tests green.
2026-06-27 19:51:28 -07:00
dev
0be1952222 downloads(#934): opt-in "Clean orphaned" action for dead review-queue rows
The reconcile heals rows whose file is still in the library; it deliberately
leaves ORPHANS — history rows whose file is gone (deleted / replaced /
re-downloaded elsewhere). Those can never be healed (no file left to confirm)
and linger in the Unverified list forever. This adds an explicit, user-initiated
cleanup for them.

- core/downloads/orphan_history.py: pure, tested rule. A row is an orphan when
  its file resolves nowhere; flags `suspicious` when EVERY reviewed file is
  unreachable (the mount-down signature) so the caller refuses rather than
  mass-delete a healthy log during an outage.
- POST /api/verification/clean-orphans (admin-only): runs it against
  _resolve_history_audio_path (raw path -> prefix-swap resolver -> tracks-table
  title fallback), refuses on the suspicious signature, and deletes only history
  ROWS — never a file (the files are already gone).
- UI: "🧹 Clean orphaned" button in the Unverified bulk-actions row, with a
  confirm dialog spelling out that it removes log rows only and refuses if the
  library looks offline.

NEVER automatic / never at boot — a filesystem check during a mount outage would
otherwise wipe good history. 5 pure-rule tests + safety-gate coverage.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LWJk7EuM7YktQeNyqQwTZY
2026-06-28 02:04:08 +02:00
BoulderBadgeDad
b85e9b40d9 perf: malloc_trim after GC so RSS actually drops (caps the peak) (#802)
the growth-triggered collects were firing but RSS still climbed to 2.2GB before snapping to
1.2GB — because gc.collect() freed the python objects but glibc hoarded the memory rather than
returning it to the OS, so RSS stayed at the high-water mark. add malloc_trim(0) after each
collect to hand freed arenas back to the OS, so incremental collects genuinely lower RSS and
the sawtooth caps near floor+200MB instead of overshooting. best-effort (skipped on musl/non-
linux). also tightened the growth trigger 250->200MB.
2026-06-27 14:34:59 -07:00
BoulderBadgeDad
b4583f23be perf: growth-triggered GC instead of fixed timer — caps the peak (#802)
the 60s timer overshot: browsing piled plexapi cyclic garbage faster than once-a-minute caught
it, so RSS hit ~2.2GB before a sweep (then dropped to 1.2GB). switch to polling RSS cheaply
(every 8s) and collecting as soon as it grows +250MB since the last sweep — so it fires DURING
a heavy browse and caps the peak near floor+250MB instead of running to 2GB+. keeps a 120s
backstop for slow idle accumulation.
2026-06-27 14:24:05 -07:00
BoulderBadgeDad
f383a62c4f perf: periodic full GC to bound RSS (plexapi cyclic Element trees) (#802)
measured the 'resource hungry' / lockup issue: browsing every page grows RSS ~300MB -> 1.8GB
and it stays. it's not a leak — it's deferred cyclic collection. plexapi parses Plex responses
into XML Element trees whose nodes reference each other in cycles; Python's generational GC
leaves them in gen2 and sweeps it rarely, so ~227k Element objects pile up. forcing gc.collect()
reclaimed ~700MB instantly (1.8GB -> 1.1GB live), confirming.

add a daemon that runs a full gc.collect() every 60s so the cyclic garbage is reclaimed on a
cadence instead of climbing into lock-up. full collect is ~tens of ms; once a minute is
negligible. this is the root of the reporter's 2GB + ramonskie's spike too.
2026-06-27 14:17:58 -07:00
BoulderBadgeDad
de9e78cf95 debug: add lightweight one-shot /api/debug/memory/objects (gc, no tracemalloc)
tracemalloc's continuous tracing locks up a loaded app, so add a one-shot gc-based memory
breakdown: top object types by total size AND by count, plus the biggest individual containers
(>1MB). a runaway 'count' points at an unbounded cache; a big bytes/str total points at blob
retention. lets us pinpoint the RSS growth (300MB -> 1.8GB after browsing) without tracing.
2026-06-27 14:02:46 -07:00
BoulderBadgeDad
5a54ffe14a dashboard: show SoulSync's own RAM next to system memory %
the Memory Usage stat showed only global system memory (psutil.virtual_memory().percent).
add the process's own resident set size (RSS) — the real 'how much RAM SoulSync uses' number —
formatted MB under 1GB, GB above. headline stays the system %, subtitle now reads 'SoulSync ·
612 MB' instead of the generic 'Current usage'. graceful fallback if psutil errors / older
backend. useful context after the recent RAM-footprint discussions.
2026-06-27 13:42:14 -07:00
BoulderBadgeDad
eddaea2f93 watchlist history: record automatic scans too (#933)
save_watchlist_scan_run had a single caller — the manual scan endpoint. the automatic/
scheduled path (process_watchlist_scan_automatically) ran the full scan but never wrote a
history row, so nightly scans never showed up in the History modal — only manual ones.

- new shared helper persist_scan_run(database, state, ...) extracts the run from the
  finished watchlist_scan_state and writes one history row
- the automatic path now stamps scan_run_id/scan_track_events and calls it
- the manual path is refactored onto the same helper so the two can't drift apart again
- history is global (no profile filter), so the all-profiles nightly scan records one
  aggregate row (profile_id None → 1, never NULL)

tests: 4 new persist_scan_run seam tests (real DB) + 2 new auto-scan integration tests
proving the auto path actually records (completed + cancelled, exactly once). 420
watchlist/automation tests green.
2026-06-27 00:42:58 -07:00
BoulderBadgeDad
b1f061a2a8 manual search: float the pasted Qobuz/Tidal track to the top (#932)
a pasted track link IS resolved + searched, but the 'bubble the exact track to the top'
step read getattr(t,'id') — and TrackResult has no top-level id (the source id lives in
_source_metadata['track_id']). so the bubble was a silent no-op: the linked track sat buried
among fuzzy text-search lookalikes and the user saw unrelated tracks. qobuz made it worse —
_qobuz_to_track_result never stamped _source_metadata at all, so the track had no id to match.

- stamp _source_metadata={'source':'qobuz','track_id':...} on qobuz TrackResults (mirrors tidal)
- extract the bubble into pure, tested helpers (linked_track_id / bubble_linked_track_first)
  that read _source_metadata['track_id'] — fixes it for tidal too, str/int-safe, stable no-op
19 track-link tests (+6 new) + 87 qobuz/download tests green.
2026-06-26 21:45:41 -07:00
BoulderBadgeDad
30ff0bde49 Release 2.7.9: version bump + What's New / version modal + PR description + docker-publish default tag
- _SOULSYNC_BASE_VERSION -> 2.7.9; docker-publish default version_tag -> 2.7.9
- pr_description.md rewritten for 2.7.9 (best-quality downloads + quality profile #896, Discover listening recs + Listening Mix #913, Wing It Pool, Auto-Sync lane redesign, multi-disc #927, sync labels #925, post-processing race #928)
- WHATS_NEW: 2.7.8 block -> 2.7.9 (+ brief earlier-versions); VERSION_MODAL_SECTIONS: 2.7.9 highlights promoted, 2.7.8 rolled into an Earlier aggregator
- RELEASE_2.7.9_discord.md: mini Discord post
2026-06-25 16:11:51 -07:00
BoulderBadgeDad
3c33e31985
Merge pull request #928 from nick2000713/fix/post-processing-race-and-followups
Fix import-vs-quarantine race + opt-in rank-based download order + quality-settings UI cleanup
2026-06-25 14:52:53 -07:00
BoulderBadgeDad
dc813d67c1
Merge pull request #926 from ramonskie/fix/issue-925-playlist-sync-label
Fix playlist sync status labels
2026-06-25 14:39:42 -07:00
BoulderBadgeDad
9847d6f0a9 Wing It Pool: two-card landing (review + resolved), matching the Discovery Pool
Opens to the same category-card landing the Discovery Pool uses, with two cards: 'guesses to review' (unverified wing-it) and 'resolved manually' (ones you've Fixed) — click to drill in, Back to return. Previously it jumped straight to a single list.

To populate the resolved list, the /fix endpoint now stamps was_wing_it on the rewritten extra_data (the wing_it_fallback flag is otherwise lost on fix), and get_wing_it_pool gained a resolved flag + the stats return both counts. Fixing/re-matching from either card refreshes in place. Seam test updated for both states.
2026-06-25 14:15:42 -07:00
BoulderBadgeDad
602b035bad Wing It Pool: review + re-match tracks Wing It auto-matched
Wing It auto-matches tracks to the server library on a best-effort guess; those tracks are flagged wing_it_fallback in extra_data and count as 'discovered', so the Discovery Pool hides them — there was no way to see or audit the guesses. New 'Wing It Pool' button (next to Discovery Pool on the Mirrored Playlists tab) opens a modal listing them with a per-playlist filter + search; 'Fix Match' reuses the Discovery Pool's fix flow (/api/discovery-pool/fix), and a manual match drops the track from the pool on refresh.

No new table or provider hooks needed — the wing-it flag is already persisted, so this is a pure query (get_wing_it_pool / get_wing_it_pool_stats, cloning the failed-pool LIKE pattern) + a /api/wing-it-pool endpoint + a cloned modal. Found 81 wing-it tracks on a real library. Seam-tested (include unverified / exclude manual-matched / scope by playlist+profile).
2026-06-25 13:57:50 -07:00
dev
ab05508acc feat(quality): rank-based candidate ordering toggle for priority mode
Adds an opt-in `rank_candidates_by_quality` profile flag. When on, the
priority-mode download walk orders candidates by the ranked-target quality
(confidence/speed only break ties) instead of confidence-first. Default off
keeps the byte-for-byte old behaviour, so existing installs are unaffected.

Best-quality search mode is always quality-first regardless of the flag; the
toggle only affects priority mode. Search-time source selection is unchanged —
nothing is skipped, so a track can never go missing, only the order in which
copies are tried changes.

The version-mismatch force-import follows automatically: it accepts the
first-tried (= best-ordered) quarantined candidate, which is the highest-quality
one once the walk is quality-first. No change to its selection logic needed.

- core/quality/selection.py: load_rank_candidates_by_quality() (fail-closed).
- core/downloads/task_worker.py: _best_quality_ordering -> _candidate_ordering;
  quality-first when best_quality mode OR the toggle is on.
- database/music_database.py: default profile carries the flag (False).
- web_server.py: flag is preserved globally across preset apply/reset, like
  search_mode.
- core/imports/version_mismatch_fallback.py: comment clarified (no behaviour
  change).

Tests (TDD): load_rank_candidates_by_quality default/enabled/disabled/error;
_candidate_ordering across all mode+toggle combinations + fail-closed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 22:00:57 +02:00
dev
05b36704c3 fix(post-processing): prevent double-claim race that fails imported tracks
Two subsystems post-process the same completed transfer: the browser-poll
status endpoint (web_server) and the background download monitor. Both watch
the same slskd/streaming transfers and each launches the verification
pipeline. When one path quarantines + requeues the next-best candidate
(clearing username/filename, status -> 'searching'), the monitor's
already-submitted run_post_processing_worker then runs, finds no source info,
and falsely marks the task 'failed' ("missing file or source information") —
clobbering the in-flight retry while a parallel attempt imports the song.

Fix: a single atomic claim (downloading/queued -> post_processing under
tasks_lock) so exactly one path processes each download.

- runtime_state: new claim_for_post_processing() helper
- post_processing: race guard — worker bails (no fail/notify) if the task is
  no longer 'post_processing' when it runs
- web_server: both poll paths (Soulseek + streaming) claim before launching;
  claim is released on thread-launch failure

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 21:03:33 +02:00
BoulderBadgeDad
9c91ba29bf Discover: listening-driven recommendations + mix (#913), Fresh Tape fix
#913 was silently producing 0 recs: similar_artists.source_artist_id is a SOURCE id (Spotify/etc.), but the scan keyed id->name by internal artists.id (resolved nothing), and the consensus ranker was fed the name-collapsed get_top_similar_artists (consensus could never fire). Fixed + elevated:

- id->name keyed by source-id columns; raw per-seed edges (real consensus); similarity_rank threaded into the score; recency-weighted seeds (recent plays boost lifetime favs)
- new 'Based On Your Listening' artist row (/api/discover/listening-recommendations) with 'because you listen to X' explanations
- new 'Your Listening Mix' track row: each rec's top tracks via a guarded, name-resolved Spotify/Deezer fetch (falls back to the discovery pool), stored as full render dicts so the row can't shrink on pool rotation
- pure tested core: similarity_from_rank, build_recency_weighted_seeds, to_mix_track, names_match (+ rank-aware grouping)

Fresh Tape (5-10 tracks): future-dated albums sorted to the top of get_discovery_recent_albums and ate the 50-album budget before the is_future_release skip ran. Add exclude_future_years + fetch a generous budget; downstream caps unchanged. Regression tested.

Also drop the per-track block 'X' from the compact playlist rows (wrong spot). Plan/audit in DISCOVER_BEST_IN_CLASS_PLAN.md.
2026-06-25 10:15:20 -07:00
ramonskie
ad657f02a8 Fix playlist sync status labels 2026-06-25 13:43:24 +02:00
BoulderBadgeDad
ed0a2079cf
Merge pull request #896 from nick2000713/feature/best-quality-search-mode
Global quality system: real-audio verification, best-quality search & quality profiles (please try...not ready to merge)
2026-06-24 20:27:38 -07:00
BoulderBadgeDad
f010fbc487 Release 2.7.8: version bump + What's New / version modal + PR description + docker-publish default tag
- _SOULSYNC_BASE_VERSION -> 2.7.8; docker-publish default version_tag -> 2.7.8
- pr_description.md rewritten for 2.7.8 (align playlists + re-add-to-wishlist-from-sync
  features, the #922 Spotify-Free label fix, and the #918 iTunes-cache self-heal follow-up)
- WHATS_NEW: replaced the 2.7.7 block with 2.7.8 (current release + brief 'earlier versions')
- VERSION_MODAL_SECTIONS: promoted the 2.7.8 highlights, rolled 2.7.7 into an 'Earlier' aggregator
2026-06-24 19:02:44 -07:00
BoulderBadgeDad
e148f859e7 Sync detail modal: click '→ Wishlist' to re-add a track with the original context
In the dashboard Recent Syncs detail modal, the '→ Wishlist' status on unmatched
tracks is now a button. Clicking it re-adds that exact track to the wishlist with
the SAME context the sync used (source_type='playlist' + the playlist's name/id +
failure_reason), so it's indistinguishable from the original auto-add.

- reconstruct_sync_track_data() (pure, tested): prefers the full cached track from
  tracks_json (by source_track_id, then index) so album art/full data carry over;
  falls back to the track_result fields; refuses non-'wishlist' rows and rows with
  no id (can't re-wishlist a matched/unidentifiable track).
- POST /api/sync/history/<id>/track/<i>/wishlist resolves the entry server-side and
  calls the wishlist service; idempotent (reports added vs already-on-wishlist).
- button shows a busy state then '✓ Re-added' / '✓ On wishlist'.

7 pure tests (full-track preference, id-vs-index match, fallback rebuild, non-
wishlist + out-of-range refusal). JS/PY/ruff clean.
2026-06-24 16:09:55 -07:00
BoulderBadgeDad
49d3c77808 Align playlists: add Jellyfin support (in-place reorder via Move endpoint)
Completes align across all three servers. Jellyfin reorders in place: DELETE the
extra entries (Mirror) then POST /Playlists/{id}/Items/{entryId}/Move/{index} for
each desired track in ascending order — so the playlist's poster/name/Id survive
(no delete-recreate), same as Plex/Navidrome. Mirrors the existing reconcile path's
entry-id handling (PlaylistItemId via /Playlists/{id}/Items).

- jellyfin reorder_playlist() + get_playlist_track_ids(); reuses the shared, tested
  plan_align_rewrite planner (no new pure logic).
- /align endpoint + frontend gate now cover navidrome|plex|jellyfin.

UNTESTED LIVE: no Jellyfin instance to verify against (same status as the Navidrome
path). Plex is the only one confirmed working end-to-end so far.
2026-06-24 13:20:32 -07:00
BoulderBadgeDad
bac5da9177 Align playlists: add Plex support + cover art + modal redesign
The align buttons were gated to Navidrome, so Plex users (the actual tester) never
saw them. Plex reorders in place via plexapi moveItem/removeItems — preserves the
playlist's poster/summary/ratingKey (no delete-recreate), same spirit as Navidrome's
overwrite.

- plex_client.reorder_playlist(): moves each desired track into sequence, removes
  any current item not in the ordered list (Mirror drops extras; Keep includes them).
  get_playlist_track_ids() feeds the shared tested plan_align_rewrite.
- /align endpoint dispatches navidrome + plex; reuses the pure planner for both.
- frontend gate opened to navidrome|plex.
- modal redesigned: cover art per row, gradient header, pop/fade animation, hover
  rows, real polish (was a plain numbered list).

plexapi moveItem/removeItems signatures verified against the installed version.
2026-06-24 13:12:09 -07:00
BoulderBadgeDad
606d1f951d Align playlists: reorder a server playlist to the source order (Navidrome)
Adds the 'Align playlists' action to the out-of-order modal — a dedicated,
order-only write path that does NOT touch the normal sync. Subsonic has no
per-track move, so it overwrites the song list in source order via createPlaylist
+ playlistId (same primitive replace-mode uses; identity/id preserved).

- plan_align_rewrite() (pure, tested): matched server ids in source order; every
  one must already be in the playlist (never injects a track); extras either
  dropped ('Mirror source') or parked at the end ('Keep extras'); returns None on
  stale data so a vanished track can't be written.
- navidrome rewrite_playlist_order() primitive (raw ordered ids).
- /api/server/playlist/<id>/align: validates ids are in the live playlist, then
  rewrites. Navidrome-only for now (Plex/Jellyfin reorder = follow-up).
- modal gets two explained options; missing tracks are NOT added (normal sync's
  job) and that's stated. Metadata-free by design — it only reshuffles existing
  server ids, so there's no sync-parity surface.

Open: confirm createPlaylist+playlistId preserves the playlist comment/image on a
live Navidrome (same risk as replace mode); add a re-apply step if it doesn't.
2026-06-24 12:58:01 -07:00
BoulderBadgeDad
ecd2500c39 Server playlist editor: surface 'accurate but out of order' + read-only server-order view
The editor renders the server column in SOURCE order (reconcile_playlist pairs
each server track to its source row), so a reordered-but-same-membership playlist
read as '5 matched / in sync' when Navidrome's real order actually differed — the
reorder never reaching the server was invisible.

- compute_order_status() (pure, tested): matched tracks' server positions must be
  strictly ascending in source order; uses RELATIVE order so missing/extra tracks
  never false-flag. reconcile entries now carry server_index (additive).
- endpoint returns order_status + server_order (the server's actual sequence).
- editor shows an amber 'out of order' badge on the server column when membership
  matches but sequence differs, opening a read-only modal of the real server order.
  One-way: source order stays the source of truth; no server-side editing.

Tests reproduce the reported 'Real Love Baby moved to #2' case + guard against
false-flagging on missing/extra. The actual 'sync order' WRITE is a separate
follow-up (membership/extra semantics + live identity-preservation test pending).
2026-06-24 12:15:47 -07:00
dev
83f21b248d fix(quarantine): manager-tab approve immediately marks task completed
When approving from the quarantine manager (no task_id in request), the
re-import ran through the simple pipeline path with no task to update, so
the downloads list stayed on 'failed' until the batch drained.

Scan download_tasks for the task whose quarantine_entry_id matches the
approved entry. If found, re-run through the verification wrapper with that
task_id so the task is marked completed immediately in the live downloads list.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 13:05:25 +02:00
dev
8f66432592 fix(quarantine): discard late quarantine entries when alternative approved
Race condition: cancelling the retry task didn't stop an already-in-flight
download from completing and creating a new quarantine entry via the pipeline.

- Approve endpoint now sets _quarantine_approved_alternative=True on the
  cancelled task alongside status='cancelled'
- Pipeline completion handler checks this flag when _acoustid_quarantined
  is set: immediately deletes the freshly-created quarantine entry and
  returns, so no stale entry accumulates from the race

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 13:04:06 +02:00
dev
842bc0ab34 fix(quarantine): auto-delete siblings + cancel retry on approve (#920)
Multiple quarantine entries accumulate per track (one per retry attempt)
and approving one left the others behind and kept the retry running.

- Approve now sends remove_siblings=true — backend sibling deletion was
  already implemented but the flag was never passed from the UI
- Toast message now reports how many duplicate candidates were removed
- After approve, the backend cancels any in-flight quarantine-retry task
  for the same track (matched by title) so the engine stops fetching new
  candidates once the user has already accepted one
- Approve All also sends remove_siblings=true

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 12:37:33 +02:00
BoulderBadgeDad
d647fc8ad1 Release 2.7.7: version bump + What's New / version modal + PR description + docker-publish default tag
- _SOULSYNC_BASE_VERSION -> 2.7.7; docker-publish default version_tag -> 2.7.7
- pr_description.md rewritten for 2.7.7 (the #915 primary-source parity headline, #913 listening-recs
  foundation, jellyfin atomic-write, and the #905/#908/#909/#910/#911/#912/#914/#916/#917/#918 batch)
- WHATS_NEW: replaced the 2.7.6 block with 2.7.7 (current release + a brief 'earlier versions' summary)
- VERSION_MODAL_SECTIONS: promoted the 2.7.7 highlights, rolled 2.7.6 into the 'Earlier' aggregator
2026-06-23 23:24:39 -07:00
dev
e8cc7ca2c8 fix(acoustid): distinguish unverified-quarantine from mismatch + gate unverified tab on require_verified
- pipeline: use trigger='acoustid_unverified' (not 'acoustid') when
  require_verified=ON rejects an unconfirmed track — quarantine badge now
  shows "ACOUSTID UNVERIFIED" instead of "ACOUSTID MISMATCH"
- web_server: /api/verification/config now also returns require_verified
- pages-extra: collapse the Unverified sub-view to quarantine-only when
  require_verified=true (same path as acoustid_enabled=false); new trigger
  entry in _VERIF_QUAR_TRIGGERS for acoustid_unverified

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 00:33:53 +02:00
BoulderBadgeDad
fc2c38ad97 #908: get YouTube playlists past the ~100-track cap (yt-dlp #16943 workaround)
YouTube Music 'Liked Music' (and any large playlist) only returned ~104 tracks. Diagnosed it
to a YouTube/yt-dlp regression (upstream #16943): the webpage-based playlist path stops at the
first ~100-item continuation page. Not a SoulSync cap (no limit in the parse path) and not the
user's cookies/IP — reproduced on a fully-served public playlist.

Fix: pass extractor_args youtubetab:skip=webpage so yt-dlp pages via the InnerTube API instead.
Verified live on the reporter's setup: 100 -> 200 entries on a large playlist (the workaround is
itself partial upstream, but a major improvement until yt-dlp PR #16948 lands). Single touch point
— parse_youtube_playlist is the only place that lists a YouTube playlist.
2026-06-23 13:35:04 -07:00