Commit graph

260 commits

Author SHA1 Message Date
Broque Thomas
698b1fa4f3 clear current downloads 2026-02-27 11:18:13 -08:00
Broque Thomas
2dc72a39b7 Add music library selection for Navidrome
SoulSync was importing from all Navidrome libraries regardless of user access restrictions. Added a "Music Library"   dropdown in Navidrome settings that lets users scope imports to a specific music folder. Uses the Subsonic musicFolderId       parameter on artist, album, and search API calls. Selecting "All Libraries" reverts to the previous behavior.
2026-02-27 09:37:58 -08:00
Broque Thomas
890d47928d Fix AcoustID false rejections for tracks with featured artists in square brackets
AcoustID sometimes returns featuring info in square brackets like [W/ Barnes Blvd.] instead of parentheses. The       normalizer only stripped parenthetical featuring tags, so these tracks failed verification and got quarantined despite being   correct. Now strips [W/ ...], [with ...], [feat. ...], and [ft. ...] bracket patterns too.
2026-02-27 08:57:12 -08:00
Broque Thomas
c4bf6ff0f3 jellyfin server connection fix 2026-02-27 08:47:50 -08:00
Broque Thomas
3c0d1fc187 Add album artists and watchlist artist fallback
Store album artists in the Spotify track data structure (handles both dict and object album forms) so downstream processing has access to album-level artist metadata. Also update missing-track processing to fall back to source_info['watchlist_artist_name'] when artist_name is absent, ensuring artist context is preserved for legacy/watchlist-sourced entries.
2026-02-26 08:58:44 -08:00
Broque Thomas
477615593e Increase Jellyfin API timeout default to 120
Raise the default Jellyfin API timeout for bulk operations from 30s to 120s to better handle slow servers and large database syncs. Updated core/jellyfin_client.py to set bulk_timeout to 120, webui/index.html to show 120s in the settings input, and webui/static/script.js to use 120 as the fallback when loading settings. Aligns UI and client behavior to reduce timeout errors during heavy syncs.
2026-02-26 08:28:08 -08:00
Broque Thomas
70156bdb98 Configurable Jellyfin API timeout and retries
Add a configurable API timeout for Jellyfin bulk requests and improve fetch retry behavior. UI: adds an "API Timeout (seconds)" field (15–300s, default 30) in webui/index.html and persists it via webui/static/script.js (load/save). Client: jellyfin_client.py now reads api_timeout from config_manager, uses it as the bulk timeout and computes a sensible non-bulk timeout (max(5, bulk_timeout//6)). Fetch loops for tracks and albums were hardened: reducing batch size now resets consecutive failure counters, log messages were clarified, and the stopping/retry thresholds were adjusted to avoid premature aborts at minimum batch sizes.
2026-02-26 08:21:20 -08:00
Broque Thomas
e77dbf3f1e Refactor matching + use improved discovery scoring
Introduce a generic score_track_match(...) in core/matching_engine.py and make calculate_match_confidence(...) delegate to it. The new scorer is source-agnostic, consolidates artist/title/duration logic (core-title fast path, cleaned similarity, weighted 60/30/10 scoring) and improves artist matching.

In web_server.py add cache-validation (_validate_discovery_cache_artist) and a reusable _discovery_score_candidates(...) helper that calls the new scorer. Propagate per-match confidence through discovery flows (Tidal, YouTube, ListenBrainz, Beatport), increase Spotify/iTunes search limits, add an extended high-limit search strategy, tighten per-source thresholds, and save match confidence to the discovery cache. Overall this centralizes and standardizes matching logic and improves accuracy/validation for cached discovery results.
2026-02-25 22:43:33 -08:00
Broque Thomas
a0828c7aad Improve album grouping and normalize suffixes
Expand track title normalization and refine album grouping logic.

- core/acoustid_verification.py: Broadened the parenthetical-suffix regex to strip year-based remasters and additional variants (e.g. "2025 Remaster", "single edit", "album edit") while still removing common extras like (Live), (Deluxe), (Radio Edit), and featuring tags.
- web_server.py: Restrict the smart album grouping to only run for singles/auto-detected albums; explicit album downloads now preserve the original Spotify album name to avoid mangling names (e.g. reworked/remastered vs deluxe). Added explicit logging for both smart grouping and skipped grouping paths. The verification post-processing worker now checks an is_album_download flag in context and skips re-grouping when true, with fallback logging on errors.

These changes prevent unintended renaming of explicit album downloads and improve normalization of common title suffixes.
2026-02-25 15:39:01 -08:00
Broque Thomas
4bff57cb70 Handle edit versions, improve cleanup & thresholds
- matching_engine.py: Add 'single edit' and 'album edit' tokens and clarify radio edit comment so edit/cut variants are recognized as different cuts rather than being silently normalized away.
- database/music_database.py: Fix SQL param ordering by appending server_source to params; add a pre-step to strip "(with ...)" / "[with ...]" only when used inside brackets (so titles like "Stay With Me" are preserved); stop removing edit/version tokens in the generic cleanup and document that radio/single/album edits are treated as distinct by the similarity scorer to avoid incorrect matches.
- web_server.py: Increase DB match confidence threshold from 0.70 to 0.80 and update the runtime check accordingly.

These changes prevent edit/cut variants from being conflated with original recordings, improve title normalization for "with" featuring syntax in brackets, and fix a params ordering bug and a too-low match threshold.
2026-02-25 12:09:53 -08:00
Broque Thomas
63624a4f6e Use nonces and drain interleaved stats in Hydrabase
Use UUID nonces to correlate requests/responses and robustly handle interleaved stats/heartbeat messages from the Hydrabase server. Adds _extract_stats and _extract_results helpers, records last_peer_count and timestamp, and loops on recv() with timeouts to drain non-result messages until the matching response (or a results message without a nonce) arrives. Mirrors the same nonce/send-and-drain logic in HydrabaseWorker, adds necessary imports (time, uuid), and improves logging and timeout handling to avoid returning stale or misattributed data.
2026-02-24 19:16:14 -08:00
Broque Thomas
f90d6567f7 Hydrabase: improve parsing add discography/tracks
Improve Hydrabase response handling and add discography/album track helpers. core/hydrabase_client.py: extract peer counts from stats.connectedPeers, handle new "response" key and stats-only or unexpected response shapes (return empty instead of wrapping), and add search_discography() and get_album_tracks() to map Hydrabase results into Album/Track objects. web_server.py: avoid redundant Hydrabase round-trips by passing precomputed hydrabase_counts into the background comparison worker; prefer Hydrabase for artist discography and album track import when active (with Spotify fallback); and route album-context searches to Hydrabase when configured. These changes reduce duplicate network calls and improve robustness against varied Hydrabase payloads.
2026-02-24 18:42:48 -08:00
Broque Thomas
1283041836 Integrate Hydrabase P2P metadata client
Add Hydrabase support as an optional/dev metadata source and comparison tool.

- Add core/hydrabase_client.py: synchronous Hydrabase WebSocket client that normalizes results to Track/Artist/Album types and exposes raw access.
- Update config/settings.py: add hydrabase settings (url, api_key, auto_connect) and getter.
- Update web_server.py: integrate HydrabaseClient, initialize client alongside the existing HydrabaseWorker, add auto-reconnect using saved config, persist credentials on connect/disconnect, add endpoints for status and stored comparisons, background comparison runner (Hydrabase vs Spotify vs iTunes), and adapt multiple search endpoints to optionally use Hydrabase as the primary metadata source with fallbacks.
- Update web UI (webui/index.html, webui/static/script.js, webui/static/style.css): add network stats and source comparison UI, pre-fill saved credentials, show peer count, load/display comparisons, update disconnect behavior to disable dev mode, and add Hydrabase badge styling.

Behavioral notes: when dev mode + Hydrabase are active, searches can be served from Hydrabase and comparisons to Spotify/iTunes are run in background; when Hydrabase fails the code falls back to Spotify/iTunes. Saved Hydrabase credentials are persisted for auto-reconnect; disconnect disables dev mode and auto_connect.

Files touched: config/settings.py, core/hydrabase_client.py, web_server.py, webui/index.html, webui/static/script.js, webui/static/style.css.
2026-02-24 08:11:48 -08:00
Broque Thomas
1c34967fd3 Raise Spotify API interval; pause enrichments
Increase Spotify client rate limit and reduce API contention during watchlist scans. Changes:

- core/spotify_client.py: Bumped MIN_API_INTERVAL from 0.2s to 0.35s (~171 calls/min) to stay safely under Spotify's ~180/min limit.
- web_server.py: In start_watchlist_scan and automatic scan flow, pause spotify_enrichment_worker and itunes_enrichment_worker before scanning (tracking with _enrichment_was_running/_itunes_enrichment_was_running) and resume them in finally blocks; added console prints for pause/resume. This prevents enrichment workers from contending for API quota during long scans.
- webui/static/script.js: Improved enrichment status tooltip logic to prioritize explicit currentType and then fall back to completion-based inference with explicit branches for artists, albums, and tracks for clearer progress text.

These changes aim to avoid API rate violations and make scan progress display more predictable.
2026-02-23 21:22:27 -08:00
Broque Thomas
81617b06aa Reset watchlist scan timestamps on clear/period change
When clearing the wishlist or changing the discovery lookback period, reset watchlist_artists.last_scan_timestamp to NULL so subsequent scans can re-discover older releases that were previously filtered by an earlier scan timestamp. clear_wishlist now deletes wishlist_tracks, updates last_scan_timestamp for all watchlist artists, and logs the number of tracks cleared and artists reset. set_discovery_lookback_period also resets last_scan_timestamp and reports how many artists were reset. Minor whitespace cleanups in watchlist_scanner and web_server included.
2026-02-23 16:47:04 -08:00
Broque Thomas
7eee2be38c Add release_date to Track and UI
Add a release_date field to the Track dataclass for both iTunes and Spotify clients (iTunes: parsed from releaseDate, Spotify: from album.release_date). Propagate release_date into enhanced search results in web_server and into the client-side script so album objects include release_date when available. Also broaden playlistId matching in the missing-tracks process to include 'enhanced_search_track_'. Removed SQLite SHM/WAL files from the repo (cleanup of DB temporary files). These changes enable showing and using track release dates across the app.
2026-02-23 16:25:23 -08:00
Broque Thomas
fabec1e455 Handle duplicate artists and ratingKey migrations
Add duplicate-artist detection/merge and handle ratingKey (ID) migrations for artists and albums. Introduces MusicDatabase.merge_duplicate_artists that picks a canonical artist (most enrichment data), merges enrichment fields, migrates albums/tracks, and removes duplicates; DatabaseUpdateWorker now runs this merge during updates (even when no new content) and after orphan cleanup. insert_or_update_artist/album now detect same-name/title + server_source collisions (ratingKey changes), inserting a new record while preserving enrichment and migrating references, with safe deletion of old rows. Also deduplicate artist listing queries so results show a single canonical row per name+server_source while aggregating album/track counts across duplicates. Logging improved to report merge/migration outcomes.
2026-02-23 13:15:06 -08:00
Broque Thomas
666e65ed3b Add repair scanning, tracklist fallbacks, UI polling
Introduce deferred album repair scanning and robust tracklist resolution plus UI/presentation tweaks.

- core/repair_worker.py: Add lazy MusicBrainz and AudioDB client accessors, per-batch folder queues, and register_folder/process_batch to defer folder scans until a batch completes. Implement cascading tracklist resolution (_resolve_album_tracklist) using Spotify/iTunes IDs, Spotify track→album lookup, album search, MusicBrainz release lookup, and AudioDB→MusicBrainz fallback. Add helpers to read Spotify track IDs, MusicBrainz album IDs, album/artist tags from files, MB/AudioDB fetchers, placeholder-ID filtering, and rename associated .lrc files when renaming audio files. Cache/locking and background-threaded scanning included. Improves resilience to missing/placeholder IDs and avoids circular imports.

- web_server.py: Register album folders for repair in post-processing and trigger repair_worker.process_batch when batches complete (multiple completion paths) so scans run after downloads finish.

- webui/static/script.js: Reduce unnecessary background work by skipping many fetches when the tab is hidden, and refresh dashboard-specific data on visibility change to ensure UI updates after OAuth or tab switch.

- webui/static/style.css: Replace glassmorphic backdrop-filters with opaque dark gradients for sidebar and main content to improve GPU rendering and visual consistency.

Overall: Adds reliable post-download album repair scanning using multiple metadata sources and reduces unnecessary client polling and heavy CSS effects for better performance and robustness.
2026-02-23 10:18:31 -08:00
Broque Thomas
24bfc2462d Add Spotify & iTunes workers; update repair worker
Add full-featured SpotifyWorker and iTunesWorker background workers to enrich artists, albums, and tracks with external metadata using batch cascading searches, fuzzy name matching, ID validation, and DB backfills. Update RepairWorker to re-read the transfer path from the database each scan, resolve host paths when running in Docker, and trigger immediate rescans when the transfer path changes; remove the static config_manager dependency. Also include supporting changes to the database layer and web UI/server (stats, controls, and styles) to integrate the new workers and reflect updated worker status.
2026-02-22 22:29:10 -08:00
Broque Thomas
2aa529f8e4 Use new Spotify /items endpoint with fallback
Add _get_playlist_items_page to call the new playlists/{id}/items endpoint (Feb 2026 API migration) and fall back to spotipy.playlist_items (old /tracks) on 403/404. Update _get_playlist_tracks and web_server.get_playlist_tracks to use the new helper to avoid 403 errors for Development Mode apps while preserving compatibility with Extended Quota Mode.
2026-02-22 19:15:11 -08:00
Broque Thomas
ce474749d5 Add library repair worker and UI
Introduce a RepairWorker to scan the transfer folder and automatically detect/repair broken album track numbers (e.g. the "all tracks = 01" bug). The worker uses mutagen to read/write tags, fuzzy-matches titles against an album tracklist (Spotify/iTunes via a SpotifyClient), updates filenames and the tracks DB file_path when renamed, and caches album tracklists. It also adds DB schema support (repair_status, repair_last_checked, and an index).

Integrates the worker into the web server: initializes and starts the worker, and exposes /api/repair/status, /api/repair/pause and /api/repair/resume endpoints. Adds UI elements (button, tooltip), client-side JS to poll and control the worker, CSS for visuals/animations, and a new image asset (whisoul.png).
2026-02-21 15:00:23 -08:00
Broque Thomas
11b6c6db97 Add API response docs and track metadata
Add a new docs/api-response-shapes.md describing expected Spotify/iTunes dataclass and raw-dict response shapes and client behavior. Also update core/wishlist_service.py to include 'track_number' (default 1) and 'disc_number' (default 1) in each formatted track dict so consumers receive track ordering metadata.
2026-02-21 10:11:09 -08:00
Broque Thomas
49a6c58ea8 Add Hydrabase P2P mirror worker
Introduce a Hydrabase P2P mirror worker and integrate it into the web UI and server flows. Adds core/hydrabase_worker.py: a background thread with a capped queue (1000), enqueue API, rate limiting, basic stats (sent/dropped/errors), and logic to send JSON requests over a provided WebSocket (responses received and discarded). Integrates the worker into web_server.py (import, startup init, status/pause/resume endpoints, and enqueues queries from multiple search endpoints when dev mode is enabled). Adds UI elements, JavaScript polling/toggle logic, and CSS styling for a Hydrabase status button in webui (index.html, static/script.js, static/style.css) to display and control worker state.
2026-02-21 02:15:43 -08:00
Broque Thomas
7726e86a78 Fix false positive Soulseek connection test by checking network state 2026-02-20 11:32:52 -08:00
Broque Thomas
3f0854e070 Fix AcoustID verification: MusicBrainz metadata fallback and quarantine reliability 2026-02-19 14:29:59 -08:00
Broque Thomas
8cf0950d3b Quality filter: use bitrate density instead of file size, cache Library/Discover pages, extend Beatport cache to 24h 2026-02-19 07:40:49 -08:00
Broque Thomas
e7e939bdd5 Retry errored items and prevent incomplete Deezer matches 2026-02-18 20:04:58 -08:00
Broque Thomas
a7cc558fb3 update ui when metadata worker finishes. 2026-02-18 18:57:00 -08:00
Broque Thomas
7b6e94772e fixed an issue wher ecollaborating artists would have the album listed as their own on library and artist page. 2026-02-18 17:11:53 -08:00
Broque Thomas
2ab52a340b Add Deezer enrichment for artists, albums, and track
Add Deezer as a third metadata enrichment source. Enriches tracks with BPM and explicit flags, albums with
   record labels, explicit flags, and type classification (album/single/EP), and backfills artwork and genres across
  all entities. Includes background worker with priority queue, rate-limited API client, database migration, server
  endpoints, and UI button with purple-themed status tooltip.
2026-02-18 16:41:24 -08:00
Broque Thomas
eee05115e4 Add AudioDB enrichment for artists, albums, and tracks
Integrated TheAudioDB as a metadata enrichment source with a background worker that scans artists, albums,   and tracks in priority order. Stores style, mood, and AudioDB IDs with automatic backfill of artwork and genres.
  Includes artist ID cross-verification from album/track results to correct mismatches caused by same-name artists.
2026-02-18 14:31:30 -08:00
Broque Thomas
1a4395cc95 Add AudioDB enrichment for artists, albums, and tracks
Integrated TheAudioDB as a metadata enrichment source. A background worker scans the library in priority order (artists → albums → tracks), matching entities via fuzzy name comparison and storing style, mood, and AudioDB IDs. Includes rate limiting, 30-day retry for not-found items, and a UI tooltip showing phase-based scan progress.
2026-02-18 14:08:01 -08:00
Broque Thomas
d54f433277 Update spotify_client.py 2026-02-18 12:23:23 -08:00
Broque Thomas
5e61a15f7f Add Spotify disconnect button and cache auth checks
Add a UI button to disconnect Spotify and fall back to iTunes/Apple Music without restarting.    Cache is_spotify_authenticated() with a 60s TTL to reduce redundant API calls (~46 call sites were each
  triggering a live sp.current_user() call). Fix status endpoint calling the auth check twice per poll,
  and ensure both OAuth callback handlers (port 8008 Flask route and port 8888 dedicated server)
  invalidate the status cache so the UI updates immediately after authentication.
2026-02-18 11:40:07 -08:00
Broque Thomas
308f0f9711 Add retry logic and adaptive rate limiting to watchlist scan
Spotify's @rate_limited decorator now retries on 429/5xx with exponential backoff (up to 5 retries) instead of sleeping once and raising. Watchlist scan delays scale dynamically based on lookback setting and artist count to prevent sustained API pressure. A circuit breaker pauses the scan after consecutive rate-limit failures.
2026-02-17 16:45:09 -08:00
Broque Thomas
c6a7c99ad7 additional logging for navidrome troubleshooting connection status 2026-02-17 08:32:51 -08:00
Broque Thomas
40324f4b22 album name in folder path is no weighted more effectively when trying to choose an album with a specific album source. 2026-02-16 17:59:18 -08:00
Broque Thomas
d1259c4b62 Add multi-disc album support with automatic disc subfolder organization
Multi-disc albums (e.g., deluxe editions, double albums) now automatically organize
  tracks into Disc 1/, Disc 2/ subfolders within the album folder. Detection uses the
  disc_number field from Spotify's API — when an album has total_discs > 1, subfolders
  are created. Single-disc albums are completely unaffected.

  - Plumb disc_number through all download paths (enhanced, non-enhanced, download
    missing modal, wishlist)
  - Compute total_discs from album tracklist and store on album context
  - Modify path builder to insert Disc N/ subfolder for multi-disc albums
  - Preserve disc_number when tracks fail and get re-added to wishlist
  - Preserve disc_number when adding tracks to wishlist from library page
  - Add visual disc separators in Soulseek search result track lists
2026-02-16 13:10:03 -08:00
Broque Thomas
1071337659 fix normalization issue on comparison with acoustID 2026-02-16 10:21:29 -08:00
Broque Thomas
f9b0316d0f fix str object has no attribute get error. 2026-02-16 10:13:56 -08:00
Broque Thomas
ff0cfd53c5 fix 404 error due to spotify api changes 2026-02-16 09:25:28 -08:00
Broque Thomas
ff3f547612 update Spotify API search limits for February 2026 changes 2026-02-15 21:01:14 -08:00
Broque Thomas
5f12634f61 fix seasonal discovery showing "Various Artists" and using wrong download mode 2026-02-15 19:29:20 -08:00
Broque Thomas
79b0591b34 fix issue where itunes/spotify seasonal discovery pools were not isolated. 2026-02-15 15:56:43 -08:00
Broque Thomas
0d66f884e6 fix - jellyfin servers can not select a user when determing which library to scan. 2026-02-15 14:09:44 -08:00
Broque Thomas
51515bc8a1 update spotify api in response to their bullshit 2026-02-15 11:46:19 -08:00
Broque Thomas
883b43b015 fix issue where wishlist would not delete entire album.
Also fixed issue where a warning was displayed.
2026-02-15 10:48:58 -08:00
Broque Thomas
a5c4783da6 clean up orphaned downloads before they gunk up the download folder. 2026-02-15 10:03:28 -08:00
Broque Thomas
d907558abf Fixed issue where itunes only watchlist artists would update the 'last scanned' property after watchlist scan 2026-02-13 16:10:19 -08:00
Broque Thomas
f2fb498693 Fix Navidrome incremental update to use getAlbumList2 API for recent albums
Summary: Navidrome incremental updates always found 0 new tracks because _get_recent_albums_navidrome() fetched all artists, sampled only the first 200, collected their albums, and sorted by created date — missing artists beyond the first 200 entirely. Replaced this with a single getAlbumList2?type=newest Subsonic API call that directly returns albums sorted by library addition date, matching how Jellyfin and Plex already use their native "recently added" endpoints.
2026-02-12 13:58:20 -08:00