Commit graph

1976 commits

Author SHA1 Message Date
Antti Kettunen
7f5384a89e Add some more shutdown checks for websocket handlers 2026-04-12 18:30:11 +03:00
Antti Kettunen
29d964e8b0 Tighten up teardown logic
- Stop active DB update work before tearing down executor pools.
- Short-circuit scan completion callbacks during shutdown so in-flight timer ticks don’t queue follow-up work.
- Prevent the download monitor from draining deferred/completed tasks after shutdown starts.
- Make listening stats startup stop-aware so it exits cleanly if teardown begins during warmup.
- If a metadata update is already running, it can now observe should_stop and exit cleanly instead of continuing after SIGTERM.
2026-04-12 18:13:27 +03:00
Antti Kettunen
aec3047216 Improve graceful shutdown and rollback safety
- Add interruptible stop events to background workers so shutdown
  wakes out of long sleeps instead of waiting on fixed delays.
- Stop scan managers, repair worker, executors, and cleanup helpers
  deterministically so process exit does not leave background threads
  alive.
- Add startup warnings for stale SQLite WAL/SHM sidecars so unclean
  shutdowns are easier to spot before init/migration errors cascade.
- Prevent forced kills from leaving SQLite sidecars behind, which
  made rollbacks to older branches fail with malformed database
  errors.
2026-04-12 15:17:18 +03:00
Broque Thomas
fff76a4be0 Add optional slskd service to docker-compose.yml
Commented-out slskd block that users can uncomment to run both services
together. Shares the ./downloads volume so SoulSync reads slskd's
downloads directly. Includes API key env var and setup instructions.
2026-04-12 00:07:46 -07:00
Broque Thomas
d607054f16 Add centralized Downloads page to What's New modal 2026-04-12 00:00:28 -07:00
Broque Thomas
7798f56885 Add centralized Downloads page with live status across all sources
New sidebar page showing every download task across the app in a unified
live-updating list. Tracks from Sync, Discover, Artists, Search, and
Wishlist all appear in one place.

Features:
- Filter pills: All / Active / Queued / Completed / Failed
- Section headers grouping by status category
- Track position (3 of 19) for album/playlist batches
- Album art, artist/album metadata, batch context, error messages
- Status dots with accent glow for active, green for complete, red fail
- Clear Completed button removes terminal items from tracker
- Nav badge shows active download count from any page via WebSocket

Fixes artist [object Object] display — handles all format variations
(list of dicts, list of strings, dict, string) for artist and album
fields in the API response.
2026-04-11 23:58:22 -07:00
Broque Thomas
ce8c3b9cbb Fix Hybrid status check to prioritize serverless sources
Serverless check (YouTube/HiFi/Qobuz) now runs before the slskd
connection test. Prevents 4-minute timeout hangs when Hybrid mode
includes both Soulseek and serverless sources but slskd isn't running.
Also reports green if any serverless source is in the hybrid order,
not just the first.
2026-04-11 23:30:34 -07:00
Broque Thomas
4560256bfe Add informative help text and tips to every setup wizard step
Each step now explains what it does and how it connects to the rest of
SoulSync. Metadata step explains catalog vs download source. Download
step explains the search-match-download pipeline and Hybrid mode. Paths
step explains the two-folder system. Watchlist step explains Discover
page, scanner schedule, and per-artist filters. First Download step
explains the full tagging and organization pipeline. Done page adds a
2x3 tips grid covering Sync, Wishlist, Automations, Notifications,
Interactive Help, and Settings.
2026-04-11 23:26:01 -07:00
Broque Thomas
eff22f55d7 Wire up automatic first-run detection for setup wizard
Wizard now shows automatically on fresh installs. Detection uses a
server-side flag (setup.completed) plus download_source.mode as a
fallback for existing users who configured settings before the wizard
existed. Config.json template defaults no longer fool the check.

Script load order fixed — setup-wizard.js loads before script.js so
openSetupWizard exists when DOMContentLoaded fires. Both finish and
skip paths set the server flag and localStorage, then continue app
initialization via callback.
2026-04-11 23:03:51 -07:00
Broque Thomas
08de91685d Add first-run setup wizard and fix download path reloading
7-step full-screen wizard: Welcome, Metadata Source, Download Source,
Paths & Media Server, Add Artists, First Download, Done. All settings
save to DB identically to the Settings page. Supports all 6 download
sources with inline config and test buttons. First download goes through
the full matched download pipeline with metadata context.

Fixes:
- Download clients (YouTube/HiFi/Tidal/Qobuz/Deezer) now reload
  download_path when settings change instead of caching from init
- watchlist_artists table migrations now include deezer_artist_id and
  discogs_artist_id in all 3 table rebuild locations (was being dropped)
- CREATE TABLE for watchlist_artists includes all provider ID columns
- Serverless download sources (YouTube/HiFi/Qobuz) show green status
  instead of red disconnected on sidebar and dashboard
- Suppress repeated slskd 401 errors — logs once then silences until
  connection recovers
2026-04-11 22:45:16 -07:00
Broque Thomas
71e4df65e3 Remove emojis from all Python log and print statements
Stripped 4,200+ emoji characters from print(), logger calls across
39 Python files. Logs are now clean text — easier to grep, more
professional, no encoding issues on terminals without Unicode support.

Seasonal config icons preserved for UI display.
2026-04-11 21:11:02 -07:00
Broque Thomas
faba4d5847 Add missing pages to profile page access and home page options
Profile creation was missing Listening Stats, Playlist Explorer, and
Issues from the page access checkboxes. Home page dropdown was missing
Stats, Playlist Explorer, and Help & Docs. Both admin and self-edit
pageLabels dicts updated to match.
2026-04-11 19:41:03 -07:00
Broque Thomas
a3ab5adcba Backfill MusicBrainz recording ID from Navidrome during database scan
Navidrome provides musicBrainzId on tracks — now captured during
database updates so the MusicBrainz enrichment worker can skip
tracks that already have an MBID.

Uses COALESCE on UPDATE to never overwrite existing enrichment data
with NULL (safe for Plex/Jellyfin which don't provide this field).

Inspired by PR #279 — fixed data loss bug in the original where
unconditional UPDATE would erase existing MBIDs.
2026-04-11 18:47:54 -07:00
Broque Thomas
7fb4cf6b0e Use per-track artist in tag writer for compilations and DJ mixes
Tag preview and writer now use track_artist (per-track artist) for the
Artist tag when available, falling back to artist_name (album artist)
when NULL. Album Artist tag always uses artist_name.

Fixes #277 — DJ mix albums no longer overwrite per-track artists
(Technique, Gouryella) with the album artist (Tiësto).
2026-04-11 18:31:11 -07:00
Broque Thomas
f5ac7c9261 Include track_artist in track search queries for collab/feature matching 2026-04-11 18:13:07 -07:00
Broque Thomas
d39db04ac0 Standardize Discogs worker hover tooltip to match other enrichment buttons 2026-04-11 17:40:41 -07:00
Broque Thomas
1e078192f0 Add track_artist column for per-track artist on compilations/DJ mixes
Phase 1: data collection only — no behavior changes.

Adds nullable track_artist column to tracks table. During database
updates (incremental, full refresh, deep scan), extracts per-track
artist from the media server when it differs from the album artist:
- Plex: originalTitle field
- Jellyfin/Emby: ArtistItems[0] vs AlbumArtists[0]
- Navidrome: artist attribute vs album artist name

NULL for normal albums (track artist = album artist). Populated only
when the media server reports a different per-track artist.

UPDATE uses COALESCE to never overwrite existing data with NULL.
2026-04-11 17:12:28 -07:00
Broque Thomas
805f72c5fd Add acappella to live/commentary cleaner patterns 2026-04-11 13:56:56 -07:00
Broque Thomas
c461f0071d Skip empty search results when navigating back from artist detail page 2026-04-11 13:54:23 -07:00
Broque Thomas
b3d3c017ed Use multi-stage Docker build to reduce image size
Builder stage compiles Python dependencies with gcc/build tools.
Runtime stage only includes curl, gosu, ffmpeg, and libchromaprint-tools.
Build tools are not shipped in the final image, reducing size and
attack surface.

Inspired by kettui's PR #273.
2026-04-11 13:39:16 -07:00
BoulderBadgeDad
8977120ba8
Merge pull request #274 from kettui/feat/metadata-client-caching
Add caching for metadata clients
2026-04-11 13:07:54 -07:00
BoulderBadgeDad
6d0ffae5fb
Merge pull request #275 from kettui/fix/spotify-ratelimited-search
Add missing rate-limit handling for Spotify search requests
2026-04-11 11:26:45 -07:00
BoulderBadgeDad
2ecc434c7a
Merge pull request #276 from kettui/feat/misc-cleanup
Reduce redundant logging during watchlist scan, initialize db once at startup
2026-04-11 11:10:38 -07:00
Broque Thomas
0d55356eaf Update What's New with music videos, Lidarr source, and bug fixes
Added sections for Music Videos search/download, Lidarr download source
(development), and recent bug fixes (dismissed findings, orphan detector,
duplicate audio streams, logs directory, stale discovery re-processing).
2026-04-11 09:50:33 -07:00
Broque Thomas
26ebb68961 Mark Lidarr download source as Development in UI 2026-04-11 09:38:09 -07:00
Broque Thomas
121f221a4f Prevent duplicate audio streams from rapid play button clicks
Added _streamLock flag to startStream() that prevents concurrent stream
requests when the user clicks play multiple times before the first
request completes. Lock is released in finally block so it always
clears on success, error, or exception.

Previously, rapid clicks would fire multiple stream requests to the
backend, each creating a separate audio playback — the only way to
stop them was a browser force refresh.
2026-04-11 09:01:30 -07:00
Broque Thomas
6fb32228e0 Create logs directory before source_reuse.log handler to prevent startup crash 2026-04-11 08:39:20 -07:00
Broque Thomas
6b2352411b Add Lidarr to all remaining source lists in orchestrator and web_server 2026-04-11 08:05:54 -07:00
Broque Thomas
9926c2f700 Add Lidarr to hybrid download source priority list 2026-04-11 08:01:09 -07:00
Broque Thomas
fc38ec4787 Add Lidarr as 7th download source and validate music video path
Lidarr integration:
- New core/lidarr_download_client.py with full interface parity
  (search, download, status, cancel — same as Qobuz/Tidal/HiFi)
- Registered in download orchestrator with source routing
- Settings: URL + API key on Downloads tab with connection test
- Available as standalone source or in Hybrid mode priority order
- API key encrypted at rest
- All streaming source checks updated to include 'lidarr'

Lidarr downloads full albums via Usenet/torrent — SoulSync imports
only the tracks it needs and discards the rest.

Music video path validation:
- Empty/unconfigured path returns clear error instead of silent failure
- Write permission test before starting download
- Default changed from './MusicVideos' to empty (must be configured)
2026-04-11 07:59:12 -07:00
Antti Kettunen
337ec7309b Initialize the database only once per process
Since MusicDatabase is initialized per-thread (which I don't dare to change), we end up needlessly calling _initialize_database for each client that gets created, thus making a ton of redundant db initialization / migration calls over and over again throughout the process' lifetime
2026-04-11 14:35:58 +03:00
Antti Kettunen
4946ff0d03 Remove redundant repetition of lookback period change during watchlist scan
Unnecessary noise on the logs
2026-04-11 14:01:52 +03:00
Antti Kettunen
45f608bf12 Utilize cached spotify client in get_spotify_artist_discography 2026-04-11 13:55:14 +03:00
Antti Kettunen
fd6335a66e Add / improve metadata client caching
Clients are for the most part being initialized per-request, which leads to a lot of redundant client initialization, as well as noise on the logs, since each client initialization emits a row on the logs, eg. 'Deezer client initialized'
2026-04-11 13:55:10 +03:00
Antti Kettunen
1b979193eb Skip Spotify requests for the rest of the watchlist scan if rate-limited
State is stored per-scan
2026-04-11 13:41:05 +03:00
Antti Kettunen
36dbb3357e Add rate-limit handling for Spotify searches 2026-04-11 13:14:50 +03:00
Antti Kettunen
6b6c866d03 Add separate requirements file for dev dependencies 2026-04-11 12:58:20 +03:00
Broque Thomas
1dcdccb282 Fix music video download in global search and improve progress visibility
- Moved _downloadMusicVideo to top-level scope so global search can use
  it (was inside enhanced search conditional that only runs on downloads page)
- Global search video cards use base64 data attributes to avoid JSON
  escaping issues in onclick handlers
- Darkened thumbnail overlay during download for better progress visibility
- Larger progress ring (52px) with accent-colored glow shadow
2026-04-10 23:23:35 -07:00
Broque Thomas
c02eb0eb0a Fix global search music video click handler — remove double HTML escaping 2026-04-10 23:15:22 -07:00
Broque Thomas
54b7a0f0e8 Add music video download with progress and metadata matching
Click any video card in Music Videos tab to download. Flow:
1. Search primary metadata source for clean artist/title
2. Fall back to YouTube title parsing if no match
3. Download video via yt-dlp (best quality MP4)
4. Save to configured Music Videos folder as Artist/Title-video.mp4

UI shows circular progress ring on the thumbnail during download,
green checkmark on completion, red X on error (clickable to retry).
Cards are non-interactive while downloading.

Backend: /api/music-video/download and /api/music-video/status endpoints
YouTube client: download_music_video() method keeps video format
2026-04-10 23:14:20 -07:00
Broque Thomas
b44bb34b44 Add Music Videos search tab to enhanced and global search
New "Music Videos" pill tab alongside Spotify/Deezer/iTunes/Discogs
in both enhanced search and global search. Searches YouTube via yt-dlp
and displays results in a video card grid with 16:9 thumbnails, play
overlay, duration badge, channel name, and view count.

- Backend: /api/enhanced-search/source/youtube_videos endpoint with
  search_videos() method on YouTubeClient returning YouTubeSearchResult
- Frontend: Video grid layout with responsive cards, YouTube red tab
  color, proper section hiding when switching between metadata and
  video tabs
- Global search: Full parity with enhanced search video rendering
- No download functionality yet — display only
2026-04-10 23:07:14 -07:00
Broque Thomas
1f0ef08b48 Add Music Videos directory setting for Plex music video support
New configurable path for storing music videos separately from audio
files, following Plex's global music video folder convention.

- Settings: library.music_videos_path (default: ./MusicVideos)
- UI: Music Videos Dir field on Settings Downloads tab with lock/unlock
- Docker: /app/MusicVideos volume mount in Dockerfile and docker-compose
- Added 'library' to settings save whitelist (was missing — music_paths
  also wasn't persisting through main settings save)
- No download functionality yet — path infrastructure only
2026-04-10 22:01:27 -07:00
Broque Thomas
5be6a46fb0 Fix dismissed findings reappearing and reduce false orphan detections
The finding dedup check only looked for 'pending' and 'resolved' status,
missing 'dismissed'. Dismissed findings were recreated as new entries on
every scan. Now includes 'dismissed' in the dedup check.

Orphan file detector improvements:
- Increased path suffix matching depth from 3 to 4 segments (covers
  Genre/Artist/Album/track.flac paths)
- Added filename-based fallback when Mutagen can't read file tags —
  parses title from "NN - Title [Quality].ext" pattern and matches
  against parent/grandparent folder names as artist
2026-04-10 17:42:39 -07:00
Broque Thomas
acb4479313 Re-discover tracks with incomplete metadata in playlist pipeline
The discovery worker skipped already-discovered tracks even when their
matched_data was incomplete (missing track_number, release_date, album
ID). These stale discoveries from before the enrichment fix would
persist forever, causing the automation pipeline to keep producing
tracks with no year, no track numbers, and no cover art.

Now treats discovered tracks as undiscovered if they're missing
track_number AND have no release_date or album ID, so the enriched
discovery pipeline fills in the gaps on the next run.
2026-04-10 17:11:42 -07:00
Broque Thomas
94e0671eb4 Update What's New with metadata pipeline and matching engine fixes
Adds two new sections to the version modal covering the Unknown Artist
fix, centralized metadata source selection, Deezer cache fix, sync
completion feedback, Fix Unknown Artists maintenance job, and the
matching engine artist gate improvements.
2026-04-10 14:21:37 -07:00
Broque Thomas
603d66ba5d Add artist gate and fix substring matching in matching engine
Prevents downloading tracks from completely wrong artists by adding
minimum artist score gates:
- Soulseek: artist_score < 0.25 → reject (catches Belvedere vs Periphery)
- YouTube: artist_score < 0.15 → reject (catches lizzylou06 vs Muse)

Fixes artist substring matching to use word boundaries instead of plain
containment — "muse" no longer matches "museum", "art" no longer matches
"heart". This was causing false positives where wrong artists passed with
artist_score=1.0 due to accidental substring containment.

Improves similarity fallback by comparing against individual path segments
instead of the full filename, so misspelled artist names (Radiohedd vs
Radiohead) still match correctly.

Adjusts YouTube weights from Title 70%/Artist 10% to Title 60%/Artist 20%
to give artist more influence in YouTube matching.

Addresses user reports of unreleased albums being downloaded with garbage
content from wrong artists on Soulseek and YouTube.
2026-04-10 13:45:07 -07:00
Broque Thomas
a6117d5174 Replace all legacy metadata_service imports with canonical functions
All callers of _create_fallback_client() and _get_configured_fallback_source()
now use get_primary_client() and get_primary_source() directly. No more
legacy alias usage anywhere in the codebase.
2026-04-10 12:52:36 -07:00
Broque Thomas
10a2766557 Fix remaining Spotify-first source selection in seasonal discovery and search API
Seasonal discovery had 3 use_spotify checks using is_authenticated()
(always True) instead of deriving from the configured source. Search API
(tracks, albums, artists) also defaulted to Spotify when authenticated.

All now check configured primary source first via get_primary_source().
2026-04-10 12:47:16 -07:00
Broque Thomas
498c22e7c3 Centralize metadata source selection in core/metadata_service.py
All metadata source decisions now flow through get_primary_source() and
get_primary_client() in core/metadata_service.py. Previously 6 different
files reimplemented this logic with inconsistent defaults ('itunes' vs
'deezer') and auth checks, causing bugs when any one was missed.

Changes:
- metadata_service.py: Added canonical get_primary_source/get_primary_client
- web_server.py: _get_metadata_fallback_source() and _get_active_discovery_source()
  are now thin wrappers delegating to metadata_service
- seasonal_discovery.py: _get_source() delegates to metadata_service
- personalized_playlists.py: _get_active_source() delegates to metadata_service
- spotify_client.py: Fixed _fallback_source default from 'itunes' to 'deezer'
- watchlist_scanner.py: _get_fallback_metadata_client() delegates to metadata_service

Future changes to source selection only need to update one file.
2026-04-10 12:34:25 -07:00
Broque Thomas
52a5d93018 Add Fix Unknown Artists maintenance job
New repair job that scans the library for tracks filed under "Unknown
Artist" and corrects them. Resolves correct metadata by:
1. Reading embedded file tags (if file has correct artist)
2. Looking up by source track ID (Spotify/Deezer/iTunes)
3. Searching by title as last resort

Dry run mode (default) creates findings for review. Live mode re-tags
the audio file, moves it to the correct folder structure, and updates
the database. Includes fix handler for applying individual findings.
2026-04-10 12:17:11 -07:00