Commit graph

2592 commits

Author SHA1 Message Date
Broque Thomas
8299dc211e Lift _run_duplicate_cleaner to core/library/duplicate_cleaner.py
Body byte-identical to the original. The shared state dict, lock,
docker_resolve_path helper, and automation engine are injected via
init() at the lift point, where all four originals are already defined.

web_server.py: 37015 → 36833 (-182 lines).
2026-04-29 20:10:22 -07:00
BoulderBadgeDad
8a46b7ef68
Merge pull request #430 from Nezreka/refactor/lift-search-service
Lift _search_service to core/library/service_search.py
2026-04-29 18:09:20 -07:00
Broque Thomas
dae7f21265 Lift _search_service to core/library/service_search.py
Lifts _search_service and its _detect_provider helper. Both bodies are
byte-identical to the originals. The nine enrichment worker handles
(spotify/itunes/mb/lastfm/genius/tidal/qobuz/discogs/audiodb) are
injected via init() right after qobuz is constructed, which is the
last worker to come up — and well before Flask starts accepting
requests, so the route handlers never see unbound workers.

web_server.py: 37245 → 37015 (-230 lines).
2026-04-29 18:06:23 -07:00
BoulderBadgeDad
423242a18a
Merge pull request #429 from Nezreka/refactor/lift-liked-artist-match
Lift liked-artist matching to core/artists/liked_match.py
2026-04-29 17:38:36 -07:00
Broque Thomas
0e237f14d4 Lift liked-artist matching to core/artists/liked_match.py
Lifts _match_liked_artists_to_all_sources and
_backfill_liked_artist_images. Both bodies are byte-identical to the
originals. Uses the same _SpotifyClientProxy + _get_*_client shim
pattern as core/artists/map.py so the bodies resolve their original
names without modification.

web_server.py: 37501 → 37245 (-256 lines).
2026-04-29 17:34:28 -07:00
BoulderBadgeDad
3714a46434
Merge pull request #428 from Nezreka/refactor/lift-download-monitor
Lift WebUIDownloadMonitor to core/downloads/monitor.py
2026-04-29 16:50:39 -07:00
Broque Thomas
9e8787c002 Lift WebUIDownloadMonitor to core/downloads/monitor.py
Class body byte-identical to original. Module-level IS_SHUTTING_DOWN
flag is mirrored from web_server's own flag in _shutdown_runtime_components
so the monitor loop still sees shutdown signals at the right moment.

Eight web_server-side helpers (_make_context_key, _on_download_completed,
_run_post_processing_worker, _download_track_worker,
_start_next_batch_of_downloads, _orphaned_download_keys,
missing_download_executor, soulseek_client) are injected via init() after
register_runtime_clients, when all symbols are defined and well before
Flask starts accepting requests.

web_server.py: 38220 → 37501 (-719 lines).
2026-04-29 16:28:52 -07:00
BoulderBadgeDad
e25daa6914
Merge pull request #427 from Nezreka/refactor/lift-artist-map
Lift artist map endpoints to core/artists/map.py
2026-04-29 15:51:27 -07:00
Broque Thomas
5875372ae0 Lift artist map endpoints to core/artists/map.py
Lifts get_artist_map_data, get_artist_map_genre_list,
get_artist_map_genres, and get_artist_map_explore (plus the
_artmap_cache_* helpers and _artist_map_cache dict) to a new module.
Bodies are byte-identical to the originals. web_server.py keeps
thin route shells that delegate to the lifted functions.

A _SpotifyClientProxy resolves the global spotify_client lazily via
core.metadata.registry.get_spotify_client() so a Spotify re-auth that
rebinds the cached client stays visible to the lifted bodies.

web_server.py: 39124 → 38220 (-904 lines).
2026-04-29 15:48:37 -07:00
BoulderBadgeDad
c0dc96e029
Merge pull request #426 from Nezreka/refactor/lift-web-metadata-update-worker
Lift WebMetadataUpdateWorker to core/workers/metadata_update.py
2026-04-29 14:28:14 -07:00
Broque Thomas
7ca786539e Lift WebMetadataUpdateWorker to core/workers/metadata_update.py
Class body byte-identical to original. The shared metadata_update_state
dict is bound at import time via init() so the class body can mutate
it without web_server.py rebinding.

web_server.py: 39754 → 39122 (-632 lines).
2026-04-29 14:20:41 -07:00
BoulderBadgeDad
f4e6b49a35
Merge pull request #425 from Nezreka/fix/spotify-worker-respect-primary-source
fix: pause Spotify worker on non-Spotify primary + cut daily budget t…
2026-04-29 13:18:26 -07:00
Broque Thomas
1d5f1e2047 fix: pause Spotify worker on non-Spotify primary + cut daily budget to 500
The Spotify enrichment worker was auto-starting unconditionally at boot,
hammering /v1/search to match every track in the library against the
Spotify catalog regardless of which metadata source the user had
actually chosen as their primary. Users on Deezer, iTunes, Discogs,
or Hydrabase saw multi-hour 429 bans (typically 14400s) on Spotify
even though they never wanted Spotify-driven enrichment in the first
place — the worker generated dead API traffic the user neither asked
for nor benefited from.

Compounded by Spotify's February 2026 API tightening:
- /v1/search max limit cut from 50 to 10 per request, default from
  20 to 5 — every track now needs more pagination, more requests.
- Sustained-rate detection more aggressive — repeated calls over
  hours trigger automated long-form bans even when each individual
  30-second window is well under the rolling limit.

Result: a user on Deezer would see their Spotify connection get banned
for 4 hours after about 30 tracks of enrichment activity, with no
recourse other than manually pausing the worker each session.

Two-part fix:

1. Boot gate (web_server.py): only auto-start the worker when
   `get_primary_source() == 'spotify'`. Otherwise initialize in the
   paused state with an explanatory log line. The settings UI manual
   unpause control remains functional for users who explicitly want
   background Spotify enrichment regardless of primary source.

   Boot logic:
   - User manually paused (existing config) → stays paused (preserved).
   - Primary = 'spotify' → starts running (preserved).
   - Primary != 'spotify' → starts paused with log line.

2. Daily budget reduction (core/spotify_worker.py): drop from 3000 to
   500 items per calendar day. The 3000 cap was set when /v1/search
   returned 50 results per call; now that it caps at 10, each track
   needs roughly 5x the API load to find a confident match. 500/day
   keeps the worker productive without crossing Spotify's hidden
   sustained-rate detection threshold.

The runtime side of the boot gate — auto-pausing when the user
switches primary source mid-session — is out of scope. The settings
UI already exposes the manual toggle, and primary-source switches are
infrequent enough that requiring a manual unpause after the fact is
acceptable.

Full suite: 1355 passing. Ruff clean.
2026-04-29 12:46:08 -07:00
BoulderBadgeDad
58a4c1905b
Merge pull request #419 from kettui/refactor/metadata-service-split-and-metadata-client-management-optimizations
Split metadata service logic into separate modules, move client management out of web_server
2026-04-29 12:36:03 -07:00
BoulderBadgeDad
53d4c0a09c
Merge pull request #424 from Nezreka/refactor/lift-final-streaming-playlists
Lift _prepare_stream_task + playlist_explorer_build_tree to core/
2026-04-29 11:34:01 -07:00
Broque Thomas
5c8b8b271a Lift _prepare_stream_task + playlist_explorer_build_tree to core/
Final lift in the web_server.py extraction effort. Pulls two route
handlers + one background worker out of `web_server.py` into new
focused packages:

- `core/streaming/prepare.py` — 258-line stream-prep worker that
  downloads a track to the local Stream/ folder for the browser audio
  player.
- `core/playlists/explorer.py` — 305-line route handler for
  `POST /api/playlist-explorer/build-tree` that streams an NDJSON
  discography tree from a mirrored playlist.

What `prepare_stream_task` does:

1. Reset stream state to 'loading' with the new track info.
2. Clear any prior file from Stream/ (only one stream lives there).
3. Spin up a fresh asyncio event loop and `soulseek_client.download()`.
4. Poll progress every 1.5s. Queue timeout 15s; overall 60s.
5. On succeeded + bytes-match: find the file with retry, move into
   Stream/, signal slskd completion, mark state 'ready' with file_path.
6. On error/timeout/cancel: state goes to 'error' or 'stopped'.
7. Finally: tear down the event loop cleanly.

What `playlist_explorer_build_tree` does:

1. Validate request, load playlist + tracks from DB.
2. Pick active metadata source (Spotify if authed, else fallback).
3. Group tracks by artist using discovered matched_data when the
   provider matches the active source.
4. Stream NDJSON: meta line → one artist line per group → complete line.
5. Per artist: cache check → resolve discography → tag releases with
   `in_playlist` flag based on title-similarity match → filter by mode
   (`albums` = only matches; `discographies` = full disco).
6. Mark playlist as explored on completion.

Strict 1:1 byte parity:
Both functions exposed their dependencies through proxy patterns
established in earlier lifts (PR4–PR8). For prepare_stream_task,
`stream_state` is a deps property; for the explorer, Flask `request` /
`jsonify` / `Response` are injected via deps so the lifted body keeps
its native syntax. Both lifts verified ZERO diff against the original
after `deps.X` → global X normalization.

258 lines orig = 258 lines lifted (prepare_stream_task).
305 lines orig = 305 lines lifted (explorer).

Bonus cleanup: web_server.py's module-level `import shutil` and
`import glob` were now unused (only `_prepare_stream_task` used them
at module scope; every other reference is via inline `import shutil`
in respective function bodies). Removed both module-level imports —
ruff caught the F811 redefinitions and confirmed they're truly
redundant.

Dependencies for `PrepareStreamDeps` (11 fields):
config_manager, soulseek_client, stream_lock, project_root,
docker_resolve_path, find_streaming_download_in_all_downloads,
find_downloaded_file, extract_filename, cleanup_empty_directories,
plus 2 stream_state property delegates.

Dependencies for `PlaylistExplorerDeps` (9 fields):
Flask request/Response/jsonify, spotify_client, get_database,
get_active_discovery_source, get_metadata_fallback_client,
get_metadata_fallback_source, get_metadata_cache.

Tests: 6 new under tests/streaming/test_prepare.py (state init,
Stream/ folder creation + clearing, download-init failure, completed
+ moved + ready state, partial-bytes incomplete-warning path) plus 9
new under tests/playlists/test_explorer.py (5 validation early-exit
paths, streaming response shape with meta/complete lines, mark-
explored side effect, discovered-artist grouping using matched_data,
provider mismatch falling back to raw artist name).

Full suite: 1355 passing (was 1340). Ruff clean.

End of the web_server.py extraction effort. Started at ~45,000 lines
across PR4–PR8 + this commit; finished around 35,000 lines with the
heavy worker + route logic now living in domain-cohesive packages
under core/. The remaining bulk in web_server.py is route handlers,
service initialization, and the deferred 1530-line
`_register_automation_handlers` (startup-only, marginal lift value).
2026-04-29 11:26:07 -07:00
BoulderBadgeDad
c89f7cc22e
Merge pull request #423 from Nezreka/refactor/lift-artists-quality
Lift enhance_artist_quality to core/artists/quality.py
2026-04-29 10:34:07 -07:00
Broque Thomas
91978656a5 Lift enhance_artist_quality to core/artists/quality.py
Pulls the 284-line artist quality enhancement helper out of
`web_server.py` into a new `core/artists/` package. Flask route handler
split: route + request parsing stay in web_server.py, the body lifts to
a pure function returning `(payload_dict, http_status_code)`.

What `enhance_artist_quality` does:

1. Validate request: track_ids must be non-empty, artist must exist.
2. Build a `track_lookup` from `database.get_artist_full_detail` so each
   selected track resolves with its album context.
3. Per track:
   - Read current quality tier from the file extension.
   - Build `matched_track_data` for the wishlist entry, in priority
     order:
     - Spotify direct lookup via stored `spotify_track_id` (preferred).
       Uses raw API data when available; otherwise rebuilds the payload
       and pulls album images via a follow-up `get_album` call.
     - Spotify search fallback using matching_engine queries with
       artist+title similarity scoring (album-type bonus for albums,
       smaller bonus for EPs). Stops at first >= 0.9 confidence match.
     - iTunes/fallback source search with the same scoring shape.
   - Add to wishlist via `wishlist_service.add_spotify_track_to_wishlist`
     with `source_type='enhance'` and a `source_context` carrying the
     original file path, format tier, bitrate, original_tier, and
     artist_name.
   - Tally `enhanced_count` / `failed_count` / per-track failure reasons.
4. Return `{success, enhanced_count, failed_count, failed_tracks}` 200.

Dependencies injected via `ArtistQualityDeps` (7 fields) — spotify_client,
matching_engine, get_database, get_wishlist_service,
get_current_profile_id, get_quality_tier_from_extension,
get_metadata_fallback_client.

Diff vs original after `deps.X` → global X normalization is **1 line of
cosmetic drift** — the success return now uses an explicit `(payload, 200)`
tuple to keep all returns shape-consistent for the wrapper. Flask treats
`jsonify(x)` and `(jsonify(x), 200)` identically. 284 lines orig = 285
lines lifted, body otherwise byte-identical.

Tests: 10 new under tests/artists/test_quality.py covering input
validation (empty track_ids, artist not found), Spotify direct lookup
via raw_data, Spotify direct lookup with enhanced format requiring
album image rebuild, Spotify search fallback, iTunes/fallback source
match path, track-not-found and no-file-path failure modes, complete
no-match failure, and source_context payload assertions (enhance flag,
file path, format tier, bitrate, source_type).

Full suite: 1340 passing (was 1330). Ruff clean.
2026-04-29 10:05:57 -07:00
BoulderBadgeDad
501ef1ba63
Merge pull request #422 from Nezreka/refactor/lift-library-retag
Lift _execute_retag to core/library/retag.py
2026-04-29 09:39:51 -07:00
Broque Thomas
3a6597561a Lift _execute_retag to core/library/retag.py
Pulls the 258-line retag worker out of `web_server.py` into a new
`core/library/` package. Pure 1:1 lift — wrapper keeps the original
entry-point name so the retag-trigger endpoint continues to work
without changes.

What `execute_retag` does:

1. Fetch album + track metadata for the new `album_id` (Spotify or
   iTunes — the Spotify client transparently falls back).
2. Load existing files in the retag group from the DB.
3. Match each existing track to a new Spotify track:
   - Priority 1: same disc + track number.
   - Priority 2: title similarity >= 0.6 (SequenceMatcher).
4. For each matched pair:
   - Re-write metadata tags via `_enhance_file_metadata`.
   - Compute the new path via `_build_final_path_for_track` and move
     the audio file (plus .lrc / .txt sidecars) if the path changes.
   - Drop an orphaned cover.jpg if it's left in an empty directory.
   - Clean up empty parent directories left behind.
   - Download the new cover art into the new album dir.
5. Update the retag group record with new artist / album / image /
   total_tracks / release_date and the appropriate Spotify-or-iTunes
   album ID (numeric → iTunes, alphanumeric → Spotify).
6. Mark the retag state 'finished' (or 'error' on exception).

Strict 1:1 byte parity:
The original mutated `retag_state` as a module global (the function
declared `global retag_state` even though it only mutates in place).
Here `retag_state` is exposed through the `RetagDeps` proxy as a Python
property so the lifted body keeps `name[key] = value` /
`name.update(...)` syntax. The property setter rebinds the
web_server.py reference if the function ever reassigns it (currently
it doesn't, but the setter is wired for parity with the watchlist lift).

Diff vs original after `deps.X` → global X normalization is **zero
differences** apart from the dropped `global retag_state` decl and the
inline `from database.music_database import get_database` (replaced by
deps.get_database()). 258 lines orig = 258 lines lifted, byte-identical
body otherwise.

Dependencies injected via `RetagDeps` (13 fields) — config_manager,
retag_lock, spotify_client, plus 8 callable helpers
(get_audio_quality_string, enhance_file_metadata,
build_final_path_for_track, safe_move_file, cleanup_empty_directories,
download_cover_art, docker_resolve_path, get_database) and 2 property
delegates (_get_retag_state / _set_retag_state).

Tests: 11 new under tests/library/test_retag.py covering setup error
paths (no album data, no album tracks, no existing tracks),
track-number priority match, title-similarity fallback, no-match skip,
missing file skip, file move when path changes, group record update
(spotify vs iTunes ID branching by alphanumeric vs numeric album_id),
multi-disc total_discs computation.

Full suite: 1330 passing (was 1319). Ruff clean.
2026-04-29 09:03:42 -07:00
BoulderBadgeDad
12b9ca48df
Merge pull request #421 from Nezreka/refactor/lift-watchlist-scan
Lift _process_watchlist_scan_automatically to core/watchlist/auto_sca…
2026-04-29 08:34:08 -07:00
Broque Thomas
2b2003ba4c Lift _process_watchlist_scan_automatically to core/watchlist/auto_scan.py
Pulls the 390-line watchlist auto-scan orchestrator out of `web_server.py`
into a new `core/watchlist/` package. Watchlist (followed-artists scanner
that finds new releases) is a separate domain from kettui's wishlist
(failed-download retry queue), so this lift does not overlap with the
ongoing PR400-style extractions.

What `process_watchlist_scan_automatically` does:

1. Smart stuck-detection guard before acquiring the timer lock —
   prevents deadlock when a previous scan flag is dangling past the
   2-hour timeout.
2. Inside the timer lock: re-check + set the active scan flag with the
   current timestamp.
3. Per-profile expansion (or single-profile when manually triggered):
   - Watchlist count check + Spotify auth gate.
   - Backfill missing artist images.
4. Initialize a fresh `watchlist_scan_state` dict (the deps property
   setter rebinds the web_server.py module-level name so external
   sentinel checks via id() comparison still detect the swap).
5. Pause enrichment workers, then call
   `WatchlistScanner.scan_watchlist_artists` with a per-event progress
   callback that translates scanner events into automation log lines.
6. Post-scan steps (skipped if the scan was cancelled mid-flight):
   - Populate discovery pool from similar artists (per-profile).
   - Refresh ListenBrainz playlists.
   - Update current seasonal playlist (weekly cadence).
   - Generate Last.fm radio playlists.
   - Sync Spotify library cache.
   - Activity feed entry + automation_engine.emit('watchlist_scan_completed').
7. On exception: mark state['status']='error', re-raise so the
   automation wrapper records the failure.
8. Finally: resume enrichment workers, clear the scanner's rescan
   cutoff, reset the auto-scanning flag.

Strict 1:1 byte parity:
The original mutated `watchlist_auto_scanning`,
`watchlist_auto_scanning_timestamp`, and `watchlist_scan_state` as
module globals (with a leading `global` decl). Here those names are
exposed through the `WatchlistAutoScanDeps` proxy as Python properties
so the lifted body keeps the same `name = value` / `name[key] = value`
shape. Property setters fan writes back to web_server.py via callback
pairs.

Diff vs original after `deps.X` → global X normalization is **zero
differences** apart from the dropped `global` declaration line — Python
doesn't need it once the names are property accesses on the deps object.
390 lines orig = 390 lines lifted, byte-identical body otherwise.

Dependencies injected via `WatchlistAutoScanDeps` (15 fields total) —
Flask app, spotify_client, automation_engine, watchlist_timer_lock, plus
5 callable helpers and 6 property delegate callbacks (paired
get/set for each of the three globals).

Tests: 11 new under tests/watchlist/test_auto_scan.py covering
stuck-detection guard, race-check inside lock, zero-watchlist short-
circuit, unauthenticated Spotify gate, successful scan with all post-
scan steps, automation event emission, activity feed logging,
cancellation mid-scan skipping post-steps, profile-scoped trigger,
flag reset in finally, rescan cutoff clear in finally.

Full suite: 1319 passing (was 1308). Ruff clean.
2026-04-29 08:17:41 -07:00
Antti Kettunen
e6c2bee427
Move profile Spotify cache into registry
- let core.metadata.registry own per-profile Spotify client caching
- register the DB-backed profile credentials provider from web_server.py
- invalidate only the affected profile cache entry on save, delete, and auth
2026-04-29 12:36:37 +03:00
Antti Kettunen
11be8834eb
Use metadata registry for web_server clients
- make web_server.py read and refresh Spotify from core.metadata.registry
- add single-key metadata cache eviction for Spotify reauth
- export the new cache helper through the metadata package shims
2026-04-29 12:27:59 +03:00
Antti Kettunen
50e1ae3a3f
Move metadata helpers into package modules
- split metadata lookup logic into core/metadata/*
- keep core/metadata_service.py as the legacy barrel
- update tests and artist-detail code to patch concrete modules
2026-04-29 11:28:42 +03:00
BoulderBadgeDad
962dbf4558
Merge pull request #418 from Nezreka/refactor/lift-downloads-staging
Lift _try_staging_match to core/downloads/staging.py
2026-04-28 23:28:28 -07:00
Broque Thomas
a2e068eaba Lift _try_staging_match to core/downloads/staging.py
Pulls the 201-line staging-folder shortcut out of `web_server.py` into
its own module under the existing `core/downloads/` package. Pure 1:1
lift — wrapper keeps the original entry-point name so the task worker's
existing call site continues to work without changes.

What `try_staging_match` does:

1. Pull the per-batch staging-file cache (one filesystem scan per batch).
2. For each staging entry, compute title + artist similarity using
   SequenceMatcher and the matching engine's `normalize_string`. Require
   title >= 0.80, then a combined score >= 0.75. The weighting flips
   based on whether artist info is available on both sides:
   - both have artist: 0.55*title + 0.45*artist
   - either side missing artist: 0.80*title + 0.20*artist (lean on title)
3. Copy the matched file to the configured transfer dir (with a
   "_staging" suffix when the destination filename already exists, to
   avoid overwriting a legitimate prior download).
4. Mark the task as 'post_processing', username='staging',
   staging_match=True.
5. Build a synthetic spotify_artist / spotify_album context (mirroring
   the modal-worker logic so the file-organization template applies
   cleanly) and store it under "staging_<task_id>". Two paths:
   - Explicit context branch (track_info has _is_explicit_album_download)
     → real album/artist data copied through.
   - Fallback branch → synthesized from track + track_info, with
     `is_album_download` heuristically derived (album differs from title
     and isn't "Unknown Album").
6. Hand off to `_post_process_matched_download_with_verification` which
   does tagging, path building, AcoustID verification, and DB insertion.

Returns True if the staging shortcut won; False to fall through to the
normal Soulseek search path.

Dependencies injected via `StagingDeps` (5 fields) — config_manager,
matching_engine, get_staging_file_cache, docker_resolve_path,
post_process_matched_download_with_verification.

Diff vs original after `deps.X` → global X normalization is **zero
differences** — 201 lines orig = 201 lines lifted, byte-identical body
(including all whitespace, comments, log strings, and the inline
`from difflib import SequenceMatcher` / `import shutil` imports inside
the function body).

Tests: 9 new under tests/downloads/test_downloads_staging.py covering
no staging files / no track title / low-confidence match returning
False, exact match copying file + transitioning task state + invoking
post-processing, existing-file rename via `_staging` suffix, explicit
album context branch, fallback context synthesis (with both album-as-
album and album-equals-title cases), and copy failure (missing source
file) returning False.

Full suite: 1308 passing (was 1299). Ruff clean.
2026-04-28 23:26:15 -07:00
BoulderBadgeDad
7d9ee39090
Merge pull request #417 from Nezreka/refactor/lift-discovery-tidal
Lift _run_tidal_discovery_worker to core/discovery/tidal.py
2026-04-28 23:07:45 -07:00
Broque Thomas
793593de51 Lift _run_tidal_discovery_worker to core/discovery/tidal.py
Missed worker from the PR5 discovery-workers series — Tidal sits in the
same domain as the deezer / spotify_public / listenbrainz / youtube /
beatport workers that were lifted in PR5b–PR5h, follows the same shape,
shares the same `_search_spotify_for_tidal_track` helper, and was simply
overlooked in the original inventory.

Pure 1:1 lift of the 212-line worker. Wrapper keeps the original
entry-point name so the existing call sites in web_server.py continue
to work without changes.

What `run_tidal_discovery_worker` does:

1. Pause enrichment workers (release shared resources).
2. For each Tidal track:
   - Cancellation gate (state['cancelled']).
   - Discovery cache lookup; cache hit short-circuits the search.
   - SimpleNamespace-style track passed straight to
     `_search_spotify_for_tidal_track` (the shared helper used by every
     worker in this family).
   - On Spotify match: build `match_data` preserving track_number /
     disc_number from raw API data, image extracted from album images
     or track object fallback, release_date filled from
     track.release_date when album dict is missing it.
   - On iTunes match: dict result populated as `match_data` with source
     set to discovery_source, image extracted from album images.
   - Save matched result to discovery cache.
   - On miss: Wing It stub stored as 'wing-it' status (success ticked).
3. After all tracks: phase='discovered', activity feed entry, sync
   discovery results back to mirrored playlist via
   `_sync_discovery_results_to_mirrored` with 'tidal' tag.
4. On error: state['phase']='error' + status with error string.
5. Finally: resume enrichment workers.

Dependencies injected via `TidalDiscoveryDeps` (13 fields) —
tidal_discovery_states, spotify_client, plus 11 callable helpers
(pause/resume enrichment, get_active_discovery_source,
get_metadata_fallback_client, get_discovery_cache_key, get_database,
validate_discovery_cache_artist, search_spotify_for_tidal_track,
build_discovery_wing_it_stub, add_activity_item,
sync_discovery_results_to_mirrored). Same surface as the deezer worker.

Diff vs original after `deps.X` → global X normalization is **zero
differences** — 212 lines orig = 212 lines lifted, byte-identical body
(including all whitespace, comments, log strings).

Tests: 9 new under tests/discovery/test_discovery_tidal.py covering
cache hit short-circuit, Spotify tuple match (track/disc preservation),
iTunes dict match path, Wing It fallback, cancellation, completion
phase update, activity feed entry, mirrored sync invocation, per-track
error handling.

Full suite: 1299 passing (was 1290). Ruff clean.
2026-04-28 23:02:24 -07:00
BoulderBadgeDad
fe936c4c7c
Merge pull request #416 from Nezreka/refactor/lift-downloads-candidates-context
PR6: lift _attempt_download_with_candidates to core/downloads/candida…
2026-04-28 22:28:35 -07:00
Antti Kettunen
a759f778b6
Move metadata API into package
- add package-owned metadata API, cache, registry, and lookup modules
- keep legacy metadata_service and metadata_cache paths as explicit shims
- update metadata call sites and tests to use package-owned helpers
2026-04-29 08:10:18 +03:00
Broque Thomas
1c43ca2eef PR6: lift _attempt_download_with_candidates to core/downloads/candidates.py
First lift in the new PR6 batch. Pulls the 312-line candidate-fallback
download dispatcher out of `web_server.py` into a new module under the
existing `core/downloads/` package. Pure 1:1 lift — wrapper keeps the
original entry-point name so all callers (search/match pipeline) work
unchanged.

What `attempt_download_with_candidates` does:

1. Sort candidates by descending confidence.
2. For each candidate:
   - Cancellation gates (3 points: top of loop, before download starts,
     after download_id is assigned).
   - Skip already-tried sources via the per-task `used_sources` set.
   - Skip blacklisted sources (user-flagged bad matches).
   - Race protection: bail when the task already has an active
     download_id.
   - `update_task_status('downloading')`, then `soulseek_client.download`.
3. On a successful download_id:
   - Build `matched_downloads_context` entry keyed by
     `make_context_key(username, filename)`.
   - For tracks with clean Spotify metadata, pull track_number /
     disc_number from (1) track_info → (2) track object → (3) Spotify
     API call. When local album context is incomplete, the API response
     backfills release_date / album_type / total_tracks / images / id.
   - Set `is_album_download` based on explicit context flag or
     heuristic (album differs from title, isn't "Unknown Album").
   - Store task/batch IDs and track_info on the context for post-
     processing + playlist-folder mode.
4. On a cancellation that wins the race after the download started:
   - `cancel_download(...)` to stop the in-flight Soulseek transfer.
   - `on_download_completed(batch_id, task_id, success=False)` to free
     the worker slot.
5. On exception or download-start failure: reset task status to
   'searching', continue to next candidate.

Dependencies injected via `CandidatesDeps` (7 fields) — soulseek_client,
spotify_client, run_async, get_database, update_task_status,
make_context_key, on_download_completed.

Diff vs original after `deps.X` → global X normalization is **zero
differences** — 312 lines orig = 312 lines lifted, byte-identical body
(including all whitespace, comments, log strings).

Tests: 14 new under tests/downloads/test_downloads_candidates.py
covering happy path (first candidate succeeds, confidence ordering),
used_sources dedup, blacklist skip, cancellation gates (cancelled
status, deleted task, active download_id, mid-flight cancel + cleanup
callback), failure paths (all candidates failed, exception during
download falls through to next), context payload (explicit album
context, track_number priority order, API backfill of incomplete album
metadata), and equal-confidence stable order.

Pre-existing behavior documented in tests:
`spotify_album_context['id']` initializes to a non-empty placeholder
'from_sync_modal' in the fallback path, so the API-backfill condition
`if not spotify_album_context.get('id')` never fires for the id field
specifically. Other album fields (release_date, album_type) backfill
fine because they default to empty.

Full suite: 1290 passing (was 1276). Ruff clean.
2026-04-28 22:06:33 -07:00
BoulderBadgeDad
a0d2d28c83
Merge pull request #415 from Nezreka/fix/cdnum-template-substitution
fix: substitute \$cdnum in download paths and skip auto disc folder w…
2026-04-28 21:42:42 -07:00
Broque Thomas
d97d105b97 fix: substitute \$cdnum in download paths and skip auto disc folder when template uses it
User report: multi-disc albums on the latest dev had literal "\$cdnum"
in their filenames instead of the expected "CDxx" label, plus a
redundant "Disc N" folder on top of the in-filename label.

Two bugs in core/imports/paths.py:

1. _replace_template_variables (the substitution helper used by every
   download path builder) had no handling for \$cdnum or \${cdnum}. The
   matching helper in web_server.py and core/repair_jobs/library_reorganize.py
   did the substitution; this one didn't, so production downloads passed
   the placeholder through unchanged. Added a cdnum_value computation
   (CD%02d when total_discs > 1, empty otherwise) plus the corresponding
   bracket_map entry and \$cdnum replace before \$track (matches the
   ordering in the other path builders).

2. The album-path branch of build_final_path_for_track auto-injected a
   "Disc N" folder whenever total_discs > 1, suppressed only when the
   template contained \$disc. Templates using \$cdnum (or \${disc} /
   \${discnum} / \${cdnum}) got both a "CDxx" label in the filename and
   the auto folder. Widened the user_controls_disc check to cover all
   the disc-bearing placeholders.

Bonus cleanup along the way:

- Folder-part stripping now drops a leading \$cdnum token (mirrors the
  existing \$disc / \$discnum / \$quality strip — defensive against an
  empty cdnum landing alone in a folder segment).
- Filename cleanup now strips a leading "  - " left behind when \$cdnum
  expands to empty on a single-disc album (mirrors the same regex in
  library_reorganize.py).
- album_template config access switched from the dotted-path key to the
  nested-dict access pattern used by the rest of the function — handles
  both production config_manager and the flat _Config used in tests.

Tests: 4 new under tests/imports/test_import_paths.py
- multi-disc cdnum substitution produces "CD02"
- single-disc cdnum collapses to empty
- folder-part containing only \$cdnum is dropped
- build_final_path_for_track with \$cdnum template produces no auto
  "Disc N" folder

Full suite: 1276 passing (was 1272). Ruff clean.
2026-04-28 21:32:24 -07:00
BoulderBadgeDad
278f30e336
Merge pull request #414 from Nezreka/fix/hls-test-windows-extension
fix: pick OS-specific ffmpeg binary in hls demux fallback test
2026-04-28 21:17:19 -07:00
Broque Thomas
4feedff8f5 fix: pick OS-specific ffmpeg binary in hls demux fallback test
`test_demux_flac_uses_tools_dir_fallback` hard-coded `tools_dir / "ffmpeg"`
in its `fake_exists` stub, but `_demux_flac` looks for `ffmpeg.exe` on
Windows (os.name == 'nt'). Result: the fake_exists stub never matched,
the code fell through to the "ffmpeg is required" RuntimeError instead
of the expected "ffmpeg failed" subprocess error, and the test failed
on Windows. Linux CI passed because os.name == 'posix' uses bare
"ffmpeg".

Pick the binary name based on `os.name` to match what `_demux_flac`
actually probes for. Asserts on the matching candidate path.

Tests: 20 passing on Windows (was 19/20). Ruff clean.
2026-04-28 21:13:54 -07:00
BoulderBadgeDad
e504099439
Merge pull request #393 from elmerohueso/hifi-fixes
fix HIFI downloads to get LOSSLESS and HI_RES again
2026-04-28 21:06:41 -07:00
BoulderBadgeDad
109ce21df4
Merge pull request #413 from Nezreka/fix/async-wishlist-cleanup
Fix/async wishlist cleanup
2026-04-28 20:41:54 -07:00
Broque Thomas
8bb0459345 fix: drop dead inline copies of wishlist removal helpers shadowing imports
PR400 added imports for `check_and_remove_from_wishlist` and
`check_and_remove_track_from_wishlist_by_metadata` from
`core.wishlist.resolution` (aliased with leading underscores in
web_server.py) but left the original inline definitions of those
functions in place at L17139 and L17243. Python's later definition
wins, so the local defs were silently shadowing the imports — meaning
the new package versions were never actually called from web_server.py.
Ruff caught the redefinition (F811) and broke CI.

Deleted the inline definitions (176 lines). Imports at L143-144 now
serve all callers, and the package functions in
`core/wishlist/resolution.py` are actually exercised. Behavior is the
same: I diffed both versions before deleting and confirmed they're
functionally equivalent.

Tests: 1232 passing (no change). Ruff clean.
2026-04-28 20:40:27 -07:00
Broque Thomas
99a763dace fix: drop redundant library-cleanup pass from wishlist download flows
Both the auto and manual wishlist download paths called
`remove_tracks_already_in_library` before submitting the batch — a
serial DB lookup per track per artist (~1s/track on a 24-track
wishlist). The batches set `force_download_all=True` which is
explicitly documented as "skip the expensive library check" — the
pre-flight cleanup was contradicting that flag.

Removed the cleanup call from both flows. Kept `remove_wishlist_duplicates`
(fast SQL DELETE) and the standalone `/api/wishlist/cleanup` endpoint
that exposes the library scan as explicit user-triggered maintenance.

Safety check on the trade-off:
- post-processing at `core/imports/pipeline.py:576-624` already handles
  re-downloads defensively: existing file with metadata → skip overwrite
  + delete source duplicate, no library corruption.
- Master worker's analysis loop normally removes wishlist entries for
  found tracks via `_check_and_remove_track_from_wishlist_by_metadata`,
  so stale wishlist entries should be rare in practice.
- Worst case for the rare orphan: one redundant download attempt that
  the post-processing layer no-ops on. Bandwidth waste, not data damage.

Tests updated:
- `..._does_not_run_library_cleanup` (renamed from `_skips_enhance_tracks_during_cleanup`)
  asserts no DB track-existence checks happen and no wishlist removals
  fire — both `enhance` and "owned" tracks reach the master worker.
- `..._marks_batch_complete_when_wishlist_genuinely_empty` (renamed from
  `..._after_cleanup`) covers the path where the wishlist starts empty.

Full suite: 1232 passing. Ruff clean.
2026-04-28 20:30:12 -07:00
Broque Thomas
6a25dcd49e fix: move manual wishlist cleanup into background worker
The manual wishlist download endpoint blocked the request thread on a
slow library-cleanup pass before submitting the batch — for a 24-track
wishlist that's ~50 per-track DB lookups serialised in the request
handler, taking 30+ seconds before the frontend got a response. The
modal sat at "Pending..." with no progress visible the whole time.

Split start_manual_wishlist_download_batch into:

1. SYNC path (request handler):
   - Generate batch_id, create download_batches entry with phase=analysis
     and analysis_total=0 placeholder.
   - Submit a single bg job (`_prepare_and_run_manual_wishlist_batch`) to
     the missing-download executor.
   - Return 200 with batch_id immediately. Frontend can start polling
     /api/active-processes status right away.

2. BG path (executor thread):
   - db.remove_wishlist_duplicates (slow-ish, single SQL)
   - remove_tracks_already_in_library (the slow one — per-track DB checks)
   - wishlist_service.get_wishlist_tracks_for_download
   - sanitize + dedupe + filter (track_ids / category)
   - Update batch.analysis_total with the real filtered count
   - add_activity_item("Wishlist Download Started", ...)
   - run_full_missing_tracks_process (master worker)

Edge case: if cleanup empties the wishlist, the bg job marks the batch
phase='complete' with error='No tracks in wishlist' (instead of the old
synchronous 400 response). Frontend status poll picks this up and the
modal can close cleanly.

Tests: existing 2 manual-download tests updated to drive the bg job
explicitly via a new `_run_submitted_bg_job` helper. Added 2 new tests:
- `..._returns_immediately_with_placeholder` — proves the sync path
  doesn't trigger any cleanup or master-worker calls; analysis_total=0.
- `..._marks_batch_complete_when_wishlist_empty_after_cleanup` —
  cleanup empties the list, master worker never invoked, batch ends
  with phase='complete'.

Full suite: 1232 passing (was 1230). Ruff clean.
2026-04-28 20:10:43 -07:00
BoulderBadgeDad
8019e13a2e
Merge pull request #400 from kettui/refactor/extract-wishlist-code
Extract wishlist core logic from web_server.py
2026-04-28 20:02:39 -07:00
elmerohueso
1c07acc349 per-segment retry for tidal and hifi 2026-04-28 20:19:01 -06:00
elmerohueso
95b1a8507b replaced onclick handlers with event listeners to resolve possible xss vector from single quotes 2026-04-28 20:19:01 -06:00
elmerohueso
7f94597706 validate hifi instance reorder against pre-existing instances 2026-04-28 20:19:01 -06:00
elmerohueso
ef3790d146 change hifi instance DELETE to use query string 2026-04-28 20:19:01 -06:00
elmerohueso
e4a94b286b hls tests 2026-04-28 20:19:00 -06:00
elmerohueso
198b637372 hifi db method tests 2026-04-28 20:19:00 -06:00
elmerohueso
eedd040318 update hifi db methods to return, rather than quash, sqlite errors 2026-04-28 20:19:00 -06:00
elmerohueso
788b7011d0 fix hifi instance reorder and enable/disable 2026-04-28 20:19:00 -06:00