Commit graph

248 commits

Author SHA1 Message Date
Broque Thomas
1283041836 Integrate Hydrabase P2P metadata client
Add Hydrabase support as an optional/dev metadata source and comparison tool.

- Add core/hydrabase_client.py: synchronous Hydrabase WebSocket client that normalizes results to Track/Artist/Album types and exposes raw access.
- Update config/settings.py: add hydrabase settings (url, api_key, auto_connect) and getter.
- Update web_server.py: integrate HydrabaseClient, initialize client alongside the existing HydrabaseWorker, add auto-reconnect using saved config, persist credentials on connect/disconnect, add endpoints for status and stored comparisons, background comparison runner (Hydrabase vs Spotify vs iTunes), and adapt multiple search endpoints to optionally use Hydrabase as the primary metadata source with fallbacks.
- Update web UI (webui/index.html, webui/static/script.js, webui/static/style.css): add network stats and source comparison UI, pre-fill saved credentials, show peer count, load/display comparisons, update disconnect behavior to disable dev mode, and add Hydrabase badge styling.

Behavioral notes: when dev mode + Hydrabase are active, searches can be served from Hydrabase and comparisons to Spotify/iTunes are run in background; when Hydrabase fails the code falls back to Spotify/iTunes. Saved Hydrabase credentials are persisted for auto-reconnect; disconnect disables dev mode and auto_connect.

Files touched: config/settings.py, core/hydrabase_client.py, web_server.py, webui/index.html, webui/static/script.js, webui/static/style.css.
2026-02-24 08:11:48 -08:00
Broque Thomas
1c34967fd3 Raise Spotify API interval; pause enrichments
Increase Spotify client rate limit and reduce API contention during watchlist scans. Changes:

- core/spotify_client.py: Bumped MIN_API_INTERVAL from 0.2s to 0.35s (~171 calls/min) to stay safely under Spotify's ~180/min limit.
- web_server.py: In start_watchlist_scan and automatic scan flow, pause spotify_enrichment_worker and itunes_enrichment_worker before scanning (tracking with _enrichment_was_running/_itunes_enrichment_was_running) and resume them in finally blocks; added console prints for pause/resume. This prevents enrichment workers from contending for API quota during long scans.
- webui/static/script.js: Improved enrichment status tooltip logic to prioritize explicit currentType and then fall back to completion-based inference with explicit branches for artists, albums, and tracks for clearer progress text.

These changes aim to avoid API rate violations and make scan progress display more predictable.
2026-02-23 21:22:27 -08:00
Broque Thomas
81617b06aa Reset watchlist scan timestamps on clear/period change
When clearing the wishlist or changing the discovery lookback period, reset watchlist_artists.last_scan_timestamp to NULL so subsequent scans can re-discover older releases that were previously filtered by an earlier scan timestamp. clear_wishlist now deletes wishlist_tracks, updates last_scan_timestamp for all watchlist artists, and logs the number of tracks cleared and artists reset. set_discovery_lookback_period also resets last_scan_timestamp and reports how many artists were reset. Minor whitespace cleanups in watchlist_scanner and web_server included.
2026-02-23 16:47:04 -08:00
Broque Thomas
7eee2be38c Add release_date to Track and UI
Add a release_date field to the Track dataclass for both iTunes and Spotify clients (iTunes: parsed from releaseDate, Spotify: from album.release_date). Propagate release_date into enhanced search results in web_server and into the client-side script so album objects include release_date when available. Also broaden playlistId matching in the missing-tracks process to include 'enhanced_search_track_'. Removed SQLite SHM/WAL files from the repo (cleanup of DB temporary files). These changes enable showing and using track release dates across the app.
2026-02-23 16:25:23 -08:00
Broque Thomas
fabec1e455 Handle duplicate artists and ratingKey migrations
Add duplicate-artist detection/merge and handle ratingKey (ID) migrations for artists and albums. Introduces MusicDatabase.merge_duplicate_artists that picks a canonical artist (most enrichment data), merges enrichment fields, migrates albums/tracks, and removes duplicates; DatabaseUpdateWorker now runs this merge during updates (even when no new content) and after orphan cleanup. insert_or_update_artist/album now detect same-name/title + server_source collisions (ratingKey changes), inserting a new record while preserving enrichment and migrating references, with safe deletion of old rows. Also deduplicate artist listing queries so results show a single canonical row per name+server_source while aggregating album/track counts across duplicates. Logging improved to report merge/migration outcomes.
2026-02-23 13:15:06 -08:00
Broque Thomas
666e65ed3b Add repair scanning, tracklist fallbacks, UI polling
Introduce deferred album repair scanning and robust tracklist resolution plus UI/presentation tweaks.

- core/repair_worker.py: Add lazy MusicBrainz and AudioDB client accessors, per-batch folder queues, and register_folder/process_batch to defer folder scans until a batch completes. Implement cascading tracklist resolution (_resolve_album_tracklist) using Spotify/iTunes IDs, Spotify track→album lookup, album search, MusicBrainz release lookup, and AudioDB→MusicBrainz fallback. Add helpers to read Spotify track IDs, MusicBrainz album IDs, album/artist tags from files, MB/AudioDB fetchers, placeholder-ID filtering, and rename associated .lrc files when renaming audio files. Cache/locking and background-threaded scanning included. Improves resilience to missing/placeholder IDs and avoids circular imports.

- web_server.py: Register album folders for repair in post-processing and trigger repair_worker.process_batch when batches complete (multiple completion paths) so scans run after downloads finish.

- webui/static/script.js: Reduce unnecessary background work by skipping many fetches when the tab is hidden, and refresh dashboard-specific data on visibility change to ensure UI updates after OAuth or tab switch.

- webui/static/style.css: Replace glassmorphic backdrop-filters with opaque dark gradients for sidebar and main content to improve GPU rendering and visual consistency.

Overall: Adds reliable post-download album repair scanning using multiple metadata sources and reduces unnecessary client polling and heavy CSS effects for better performance and robustness.
2026-02-23 10:18:31 -08:00
Broque Thomas
24bfc2462d Add Spotify & iTunes workers; update repair worker
Add full-featured SpotifyWorker and iTunesWorker background workers to enrich artists, albums, and tracks with external metadata using batch cascading searches, fuzzy name matching, ID validation, and DB backfills. Update RepairWorker to re-read the transfer path from the database each scan, resolve host paths when running in Docker, and trigger immediate rescans when the transfer path changes; remove the static config_manager dependency. Also include supporting changes to the database layer and web UI/server (stats, controls, and styles) to integrate the new workers and reflect updated worker status.
2026-02-22 22:29:10 -08:00
Broque Thomas
2aa529f8e4 Use new Spotify /items endpoint with fallback
Add _get_playlist_items_page to call the new playlists/{id}/items endpoint (Feb 2026 API migration) and fall back to spotipy.playlist_items (old /tracks) on 403/404. Update _get_playlist_tracks and web_server.get_playlist_tracks to use the new helper to avoid 403 errors for Development Mode apps while preserving compatibility with Extended Quota Mode.
2026-02-22 19:15:11 -08:00
Broque Thomas
ce474749d5 Add library repair worker and UI
Introduce a RepairWorker to scan the transfer folder and automatically detect/repair broken album track numbers (e.g. the "all tracks = 01" bug). The worker uses mutagen to read/write tags, fuzzy-matches titles against an album tracklist (Spotify/iTunes via a SpotifyClient), updates filenames and the tracks DB file_path when renamed, and caches album tracklists. It also adds DB schema support (repair_status, repair_last_checked, and an index).

Integrates the worker into the web server: initializes and starts the worker, and exposes /api/repair/status, /api/repair/pause and /api/repair/resume endpoints. Adds UI elements (button, tooltip), client-side JS to poll and control the worker, CSS for visuals/animations, and a new image asset (whisoul.png).
2026-02-21 15:00:23 -08:00
Broque Thomas
11b6c6db97 Add API response docs and track metadata
Add a new docs/api-response-shapes.md describing expected Spotify/iTunes dataclass and raw-dict response shapes and client behavior. Also update core/wishlist_service.py to include 'track_number' (default 1) and 'disc_number' (default 1) in each formatted track dict so consumers receive track ordering metadata.
2026-02-21 10:11:09 -08:00
Broque Thomas
49a6c58ea8 Add Hydrabase P2P mirror worker
Introduce a Hydrabase P2P mirror worker and integrate it into the web UI and server flows. Adds core/hydrabase_worker.py: a background thread with a capped queue (1000), enqueue API, rate limiting, basic stats (sent/dropped/errors), and logic to send JSON requests over a provided WebSocket (responses received and discarded). Integrates the worker into web_server.py (import, startup init, status/pause/resume endpoints, and enqueues queries from multiple search endpoints when dev mode is enabled). Adds UI elements, JavaScript polling/toggle logic, and CSS styling for a Hydrabase status button in webui (index.html, static/script.js, static/style.css) to display and control worker state.
2026-02-21 02:15:43 -08:00
Broque Thomas
7726e86a78 Fix false positive Soulseek connection test by checking network state 2026-02-20 11:32:52 -08:00
Broque Thomas
3f0854e070 Fix AcoustID verification: MusicBrainz metadata fallback and quarantine reliability 2026-02-19 14:29:59 -08:00
Broque Thomas
8cf0950d3b Quality filter: use bitrate density instead of file size, cache Library/Discover pages, extend Beatport cache to 24h 2026-02-19 07:40:49 -08:00
Broque Thomas
e7e939bdd5 Retry errored items and prevent incomplete Deezer matches 2026-02-18 20:04:58 -08:00
Broque Thomas
a7cc558fb3 update ui when metadata worker finishes. 2026-02-18 18:57:00 -08:00
Broque Thomas
7b6e94772e fixed an issue wher ecollaborating artists would have the album listed as their own on library and artist page. 2026-02-18 17:11:53 -08:00
Broque Thomas
2ab52a340b Add Deezer enrichment for artists, albums, and track
Add Deezer as a third metadata enrichment source. Enriches tracks with BPM and explicit flags, albums with
   record labels, explicit flags, and type classification (album/single/EP), and backfills artwork and genres across
  all entities. Includes background worker with priority queue, rate-limited API client, database migration, server
  endpoints, and UI button with purple-themed status tooltip.
2026-02-18 16:41:24 -08:00
Broque Thomas
eee05115e4 Add AudioDB enrichment for artists, albums, and tracks
Integrated TheAudioDB as a metadata enrichment source with a background worker that scans artists, albums,   and tracks in priority order. Stores style, mood, and AudioDB IDs with automatic backfill of artwork and genres.
  Includes artist ID cross-verification from album/track results to correct mismatches caused by same-name artists.
2026-02-18 14:31:30 -08:00
Broque Thomas
1a4395cc95 Add AudioDB enrichment for artists, albums, and tracks
Integrated TheAudioDB as a metadata enrichment source. A background worker scans the library in priority order (artists → albums → tracks), matching entities via fuzzy name comparison and storing style, mood, and AudioDB IDs. Includes rate limiting, 30-day retry for not-found items, and a UI tooltip showing phase-based scan progress.
2026-02-18 14:08:01 -08:00
Broque Thomas
d54f433277 Update spotify_client.py 2026-02-18 12:23:23 -08:00
Broque Thomas
5e61a15f7f Add Spotify disconnect button and cache auth checks
Add a UI button to disconnect Spotify and fall back to iTunes/Apple Music without restarting.    Cache is_spotify_authenticated() with a 60s TTL to reduce redundant API calls (~46 call sites were each
  triggering a live sp.current_user() call). Fix status endpoint calling the auth check twice per poll,
  and ensure both OAuth callback handlers (port 8008 Flask route and port 8888 dedicated server)
  invalidate the status cache so the UI updates immediately after authentication.
2026-02-18 11:40:07 -08:00
Broque Thomas
308f0f9711 Add retry logic and adaptive rate limiting to watchlist scan
Spotify's @rate_limited decorator now retries on 429/5xx with exponential backoff (up to 5 retries) instead of sleeping once and raising. Watchlist scan delays scale dynamically based on lookback setting and artist count to prevent sustained API pressure. A circuit breaker pauses the scan after consecutive rate-limit failures.
2026-02-17 16:45:09 -08:00
Broque Thomas
c6a7c99ad7 additional logging for navidrome troubleshooting connection status 2026-02-17 08:32:51 -08:00
Broque Thomas
40324f4b22 album name in folder path is no weighted more effectively when trying to choose an album with a specific album source. 2026-02-16 17:59:18 -08:00
Broque Thomas
d1259c4b62 Add multi-disc album support with automatic disc subfolder organization
Multi-disc albums (e.g., deluxe editions, double albums) now automatically organize
  tracks into Disc 1/, Disc 2/ subfolders within the album folder. Detection uses the
  disc_number field from Spotify's API — when an album has total_discs > 1, subfolders
  are created. Single-disc albums are completely unaffected.

  - Plumb disc_number through all download paths (enhanced, non-enhanced, download
    missing modal, wishlist)
  - Compute total_discs from album tracklist and store on album context
  - Modify path builder to insert Disc N/ subfolder for multi-disc albums
  - Preserve disc_number when tracks fail and get re-added to wishlist
  - Preserve disc_number when adding tracks to wishlist from library page
  - Add visual disc separators in Soulseek search result track lists
2026-02-16 13:10:03 -08:00
Broque Thomas
1071337659 fix normalization issue on comparison with acoustID 2026-02-16 10:21:29 -08:00
Broque Thomas
f9b0316d0f fix str object has no attribute get error. 2026-02-16 10:13:56 -08:00
Broque Thomas
ff0cfd53c5 fix 404 error due to spotify api changes 2026-02-16 09:25:28 -08:00
Broque Thomas
ff3f547612 update Spotify API search limits for February 2026 changes 2026-02-15 21:01:14 -08:00
Broque Thomas
5f12634f61 fix seasonal discovery showing "Various Artists" and using wrong download mode 2026-02-15 19:29:20 -08:00
Broque Thomas
79b0591b34 fix issue where itunes/spotify seasonal discovery pools were not isolated. 2026-02-15 15:56:43 -08:00
Broque Thomas
0d66f884e6 fix - jellyfin servers can not select a user when determing which library to scan. 2026-02-15 14:09:44 -08:00
Broque Thomas
51515bc8a1 update spotify api in response to their bullshit 2026-02-15 11:46:19 -08:00
Broque Thomas
883b43b015 fix issue where wishlist would not delete entire album.
Also fixed issue where a warning was displayed.
2026-02-15 10:48:58 -08:00
Broque Thomas
a5c4783da6 clean up orphaned downloads before they gunk up the download folder. 2026-02-15 10:03:28 -08:00
Broque Thomas
d907558abf Fixed issue where itunes only watchlist artists would update the 'last scanned' property after watchlist scan 2026-02-13 16:10:19 -08:00
Broque Thomas
f2fb498693 Fix Navidrome incremental update to use getAlbumList2 API for recent albums
Summary: Navidrome incremental updates always found 0 new tracks because _get_recent_albums_navidrome() fetched all artists, sampled only the first 200, collected their albums, and sorted by created date — missing artists beyond the first 200 entirely. Replaced this with a single getAlbumList2?type=newest Subsonic API call that directly returns albums sorted by library addition date, matching how Jellyfin and Plex already use their native "recently added" endpoints.
2026-02-12 13:58:20 -08:00
Broque Thomas
965f93ab31 Fix: Validate iTunes explicit albums have tracks before deduplication
Fixed issue where broken iTunes explicit album IDs were preferred over working clean versions during deduplication. Some iTunes explicit albums (e.g., "Mr. Morale & The Big Steppers" ID 1623854804) report track counts in metadata but return 0 tracks when queried.
Added validation in itunes_client.py get_artist_albums() to verify explicit albums actually have tracks before keeping them. If an explicit version has 0 tracks, it's skipped and the clean version is used instead.
This fixes:
- "No tracks found" error when clicking affected albums
- Incorrect track count mismatches (19/18) caused by broken API data
The validation only runs for explicit albums during deduplication, minimal performance impact.
2026-02-09 21:01:40 -08:00
Broque Thomas
178956afd6 proactive fix for upcoming spotify api changes 2026-02-07 14:54:34 -08:00
Broque Thomas
d08a2e91a2 feat: embed MusicBrainz, Spotify/iTunes IDs, ISRC, and merged genres into audio file tags
Enrich downloaded audio files with external identifiers and improved genre metadata in a single post-processing write. During metadata enhancement, the app now looks up the MusicBrainz recording and artist MBIDs, retrieves the ISRC and MusicBrainz genres from a follow-up detail lookup, merges them with Spotify's artist-level genres (deduplicated, capped at 5), and embeds everything alongside the Spotify/iTunes track, artist, and album IDs. All MusicBrainz API calls are serialized through the existing global rate limiter, making concurrent download workers safe without needing to pause the background worker. Includes a database migration adding Spotify/iTunes ID columns to the library tables.
2026-02-05 21:26:19 -08:00
Broque Thomas
d9efcbdf99 feat: AcoustID audio verification, MusicBrainz enrichment UI, v1.5
Add optional post-download audio fingerprint verification using AcoustID.
  Downloads are verified against expected track/artist using fuzzy string
  matching on AcoustID results. Mismatched files are quarantined and
  automatically added to the wishlist for retry.

  - AcoustID verification with title/artist fuzzy matching (not MBID comparison)
  - Quarantine system with JSON metadata sidecars for failed verifications
  - fpcalc binary auto-download for Windows, macOS (universal), and Linux
  - MusicBrainz enrichment worker with live status UI and track badges
  - Settings page AcoustID section with real-fingerprint connection test
  - Source reuse for album downloads to keep tracks from same Soulseek user
  - Enhanced search queries for better track matching
  - Bug fixes: wishlist tracking, album splitting, regex & handling, log rotation
2026-02-05 16:33:07 -08:00
Broque Thomas
8cdf548561 fix - cleared status too early before it could be read for live status updated. Musicbrainz 2026-02-03 21:26:15 -08:00
Broque Thomas
2a4cab3f96 fix polling for musicbrainz worker 2026-02-03 19:47:31 -08:00
Broque Thomas
cee5590718 feat(ui): add MusicBrainz enrichment status UI with real-time monitoring
MusicBrainz library enrichment with real-time
status monitoring and manual control.
Features:
- Status icon button in dashboard header with glassmorphic design
- Animated loading spinner during active enrichment
- Hover tooltip showing:
  - Worker status (Running/Paused/Idle)
  - Currently processing item
  - Artist matching progress with percentage
- Click-to-toggle pause/resume functionality
- Auto-polling every 2 seconds for live updates
Backend Changes:
- Added GET /api/musicbrainz/status endpoint
- Added POST /api/musicbrainz/pause endpoint
- Added POST /api/musicbrainz/resume endpoint
- Worker tracks current_item for UI display
- get_stats() returns enhanced status data
Frontend Changes:
- New MusicBrainz button component with tooltip
- Premium CSS styling with animations
- JavaScript polling and state management
- Positioned tooltip below button with centered arrow
Files Modified:
- web_server.py: API endpoints and worker initialization
- core/musicbrainz_worker.py: current_item tracking
- webui/index.html: Button and tooltip structure
- webui/static/style.css: Complete styling (240 lines)
- webui/static/script.js: Polling and interaction logic (115 lines)
2026-02-03 15:20:04 -08:00
Broque Thomas
35e65b2106 When downloading albums/EPs from Soulseek, after the first track downloads successfully, browse the source's folder and reuse it for subsequent tracks. This keeps entire albums coming from one source instead of scattering across many users.
- Single-worker mode for album batches (sequential downloads enable clean source reuse)
  - Browse API integration to list files in a source's directory
  - Failed source tracking per-batch to avoid retrying broken sources
  - Graduated quality scoring for upload speed, queue length, and free slots
  - Track number fallback fix (uses Spotify track number instead of hardcoded 1)
  - Duplicate completion guard to prevent double-decrement of active worker count
  - Dedicated logging for source reuse and post-processing diagnostics
2026-02-01 00:42:39 -08:00
Broque Thomas
da44278f12 add fourth search query for enhanced matching with both a cleaned and artist removed search. 2026-01-31 20:53:56 -08:00
Broque Thomas
7d8bb2b88a Revert "Add album-level Soulseek search and fix unbounded log growth"
This reverts commit 258fd7a8ae.
2026-01-31 18:42:37 -08:00
Broque Thomas
bbac51412d Revert "album level search for wishlist albums/eps"
This reverts commit 28ca3f5ce5.
2026-01-31 18:42:17 -08:00
Broque Thomas
28ca3f5ce5 album level search for wishlist albums/eps 2026-01-31 11:21:35 -08:00