New automation action that executes user scripts from a dedicated
scripts/ directory. Available as both a DO action and THEN action.
Scripts are selected from a dropdown populated by /api/scripts.
Security: only scripts in the scripts dir can run, path traversal
blocked, no shell=True, stdout/stderr capped, configurable timeout
(max 300s). Scripts receive SOULSYNC_EVENT, SOULSYNC_AUTOMATION,
and SOULSYNC_SCRIPTS_DIR environment variables.
Includes Dockerfile + docker-compose.yml changes for the scripts
volume mount, and three example scripts (hello_world.sh,
system_info.py, notify_ntfy.sh).
After a successful playlist sync, if the source playlist has cover
art (Spotify, Tidal, Deezer, etc.), the image is downloaded and
uploaded as the playlist poster on the media server. Plex uses
uploadPoster(), Jellyfin/Emby uses POST /Items/{id}/Images/Primary.
Navidrome skipped (no playlist image API). Failure is silent — sync
result unchanged. Automation-triggered syncs and playlists without
images are unaffected.
What's New: Added Deezer user playlists, Qobuz token auth, streaming
source artist gate, download history provenance, and comprehensive
fixes section covering artist name casing, future album skip, track
number fix, Emby sync, discovery fix fallback, and all new settings.
Help docs: Updated Qobuz auth to mention token option, added Music
Library Paths, Replace Lower Quality, and HiFi Instance Health to
Other Settings section.
New setting in Settings > Library lets users add folder paths where
their music files live. The file resolver checks these paths when
looking for library files, solving Docker path mismatches and multi-
folder libraries. Required for tag writing, streaming, and orphan
detection when the media server reports paths that differ from what
SoulSync can see. Docker users mount their music folder(s) with
read-write access and add the container-side path. Default is empty
— existing users see no change.
New toggle in Settings > Library: "Replace lower quality files on
import". When enabled, if a track already exists in the library at
a lower quality tier (e.g. MP3) and a higher quality version (e.g.
FLAC) is imported from staging, the existing file is replaced.
Comparison uses the existing QUALITY_TIERS system (lossless > opus/
ogg > m4a > mp3). When disabled (default), existing behavior is
unchanged — existing tracks are always kept. Also applies to regular
downloads that land on an existing file.
Discovery fix modal now tries the user's active metadata source
first, then falls back through all available sources (Spotify,
Deezer, iTunes) in sequence. Previously hardcoded Spotify with no
fallback, leaving users without Spotify stuck on "Searching...".
New /api/deezer/search_tracks endpoint exposes the existing
DeezerClient.search_tracks() method for the fix modal. Same
request/response format as Spotify and iTunes endpoints.
Qobuz added reCAPTCHA to their login endpoint, blocking automated
email/password auth for new users. Token login lets users paste
their X-User-Auth-Token from the browser DevTools after logging in
manually. Added to both Connections and Downloads tabs with
instructions. Existing email/password flow completely unchanged.
Backend validates token via user/get API and saves the session
identically to email/password login.
New "Check All Instances" button in Settings > Downloads > HiFi
shows each configured instance with color-coded status: green for
fully working (can download), orange for search-only (downloads
fail), red for offline/SSL error/timeout. Helps users understand
why HiFi downloads aren't working when community instances go down.
When downloading individual tracks from a playlist sync, the album
detection fallback hardcoded track_number to 1 instead of using the
value already available in the download context from the playlist
metadata. Now checks original_search.track_number and track_info
.track_number before falling back to 1. Full album downloads and
Spotify API lookup paths are completely unchanged.
New "Deezer" tab on sync page shows authenticated user's playlists,
identical to the Spotify tab pattern — same card layout, details
modal with track list, sync button, and download missing tracks
flow. Existing URL import renamed to "Deezer Link" tab (unchanged).
Backend: get_user_playlists() and get_playlist_tracks() on the
download client fetch via public API with ARL session cookies.
Album release dates batch-fetched for $year template variable.
Three new endpoints: arl-status, arl-playlists, arl-playlist/<id>.
Frontend: cards use Spotify-identical HTML structure with live sync
status, progress indicators, and View Progress/Results buttons.
Downloads reuse openDownloadMissingModal with zero modifications.
Track data cached on first open, instant on subsequent clicks.
Library page: new dropdown filter to show artists matched or unmatched
to any metadata source (Spotify, MusicBrainz, Deezer, Discogs, etc).
Select "No Discogs" to find artists needing manual Discogs matching.
Filter applied as WHERE clause on the source ID columns.
Discogs enrichment: added to valid_services whitelist, _enrichment_locks,
and _run_single_enrichment handler. The Enrich button was returning an
error when Discogs was selected from the dropdown.
Discogs was missing from valid_services, _enrichment_locks, and
_run_single_enrichment handler. The Enrich button on the enhanced
library page returned "service must be one of" error when Discogs
was selected. Added handler using _process_item pattern matching
other batch-style workers, with guard against track-level enrichment.
Settings > Advanced now shows database size, free pages, and auto-
vacuum mode. Two actions: Compact Database (full VACUUM to reclaim
dead space) and Enable Incremental Vacuum (one-time setup for
automatic page reclamation). Both have confirmation dialogs warning
about lock time on large databases. Info refreshes on Advanced tab
switch and after each operation.
Streaming matching: add artist gate rejecting candidates with artist
similarity below 0.4, raise threshold to 0.60, block fallback to
Soulseek filename matcher for Tidal/Qobuz/HiFi/Deezer. Fix single-
char artist containment bug where normalize_string strips non-ASCII
(e.g. "B小町" → "b") causing "b" to match any artist containing
that letter. Fixed in both score_track_match and the Soulseek scorer.
YouTube and Soulseek matching behavior unchanged.
Global search: add registerSearchDownload() calls to _gsClickAlbum
and _gsClickTrack so downloads create bubble snapshots on dashboard
and search page, matching the enhanced search standard.
Global search escaping: add _escAttr() helper to handle newlines in
album/artist names that broke inline onclick string literals.
Entries are now compact cards that expand on click to reveal source
details. Shows expected vs downloaded title/artist with red mismatch
highlighting. Source artist column added to DB. Streaming track IDs
extracted from the id||name filename pattern. File and ID always on
their own line to avoid edge-case misplacement.
Track original source filename, track ID, and AcoustID verification
result for every download. Helps debug wrong-file downloads from
streaming sources like Tidal. Each column migrated independently
for crash safety. Frontend shows source detail line and color-coded
AcoustID badge per entry. Button renamed to "Download History".
FLAC → Opus/AAC conversion only tried to read embedded art from the
source FLAC. If art was only in cover.jpg (not embedded), the lossy
copy got no art. Now falls back to cover.jpg in the same directory.
Previously skipped for Tidal/Qobuz/HiFi/Deezer as "trusted API sources"
but streaming APIs can return wrong versions (live, remix, cover).
AcoustID now runs for every download source when enabled.
Added user-follow-read scope to all 5 SpotifyOAuth instances in
web_server.py (was only in spotify_client.py). Fixed OAuth callbacks
using is_authenticated() instead of is_spotify_authenticated() — the
former returns True with iTunes/Deezer fallback, masking auth failures.
Credit: kettui (PR #253) identified both issues.
When using templates like "albums/$albumartist/$album/$track - $title",
post-processing created Transfer/ArtistName/ before computing the
template path, leaving an empty folder. _build_final_path_for_track
handles all directory creation based on the template — the premature
makedirs was redundant and incorrect.
- search_artists() now filters by active media server — no more duplicate
results from Plex/Jellyfin/Navidrome showing the same artist 3 times
- Per-artist Sync button re-fetches artist name from media server, catches
renames (e.g., Plex changing "Kendrick Lamar" to "eastside k-boy")
- Global search track click opens download modal directly instead of
navigating to enhanced search page (matches enhanced search behavior)
New download_source column on library_history table records which source
(Soulseek, Tidal, Qobuz, HiFi, YouTube, Deezer) each track was downloaded
from. Extracted from context username during post-processing.
Frontend shows source badge alongside quality badge on each download entry.
Source breakdown bar below tabs shows per-source totals with color-coded
chips (e.g., "Soulseek: 847 | Tidal: 203"). Includes DB migration for
existing installs. Existing entries show quality only (source is NULL).
Replaced single "Change" button with per-source rows showing match
status for each provider (Spotify, Apple Music, Deezer, Discogs).
Each row has Fix/Match button that searches that specific source API,
plus clear button to remove individual matches.
- Per-source search uses _search_service (same as enrichment modal)
- Backend: added Discogs to valid providers, empty ID clears match
- Fixed provider validation to accept 'discogs' alongside others
- Clear sets DB column to NULL instead of rejecting empty string
Manual scan path was only backfilling the active provider (e.g., only
Spotify IDs if Spotify was active). Now matches the auto-scan behavior:
backfills iTunes, Deezer, Spotify (if auth'd), and Discogs (if token)
for all watchlist artists before scanning begins.
Removed cross-guards that blocked wishlist downloads while watchlist scans
ran (and vice versa). Downloads use bandwidth, scans use API calls —
different resources. The per-call rate limiter handles any API contention.
- Automation engine: each handler now has self-only guard (no cross-block)
- _process_wishlist_automatically: removed watchlist scanning check
- Pipeline Phase 4: removed watchlist scanning check
- Manual watchlist scan endpoint: removed wishlist processing check
Users with large watchlists (6+ hour scans) will now see downloads
starting immediately instead of waiting for the scan to finish.
Initial load fetches 200 tracks instead of all. "Load More" button at
bottom shows (200 of 5,000) and loads next batch on click. Backend now
returns total count alongside limited results for both album and singles
categories. Rendering logic unchanged — just operates on smaller sets.
Removed as_dict=True from all three OAuth callback sites. Spotipy will
return token string directly in future versions — our code only checks
truthiness so both dict and string work.
API Call Tracker:
- Save/load 24h minute-bucketed history + events to database/api_call_history.json
- Persists across server restarts via atexit + signal handler hooks
- New record_event() for rate limit bans (called from _set_global_rate_limit)
- New get_debug_summary() for Copy Debug Info — 24h totals, peak cpm with
timestamp, per-endpoint breakdown, and last 20 rate limit events
- Fixed race condition: events iteration now inside lock during save
Spotify Rate Limit Mitigation:
- Enrichment worker: max_pages=5 on get_artist_albums (was unlimited — artist
with 217 albums caused 22 paginated API calls, now capped at 5)
- Enrichment worker: inter_item_sleep raised from 0.5s to 1.5s
Spotify Re-Auth Fix:
- Both OAuth callbacks (port 8008 + 8888) now clear rate limit ban AND
post-ban cooldown after successful re-auth — Spotify usable immediately
instead of stuck on Deezer fallback for 5 minutes
- Auth cache invalidated on both global client and enrichment worker client
Playlist Pipeline — single automation that runs the full playlist lifecycle:
refresh → discover → sync → download missing. Replaces 4-automation signal
chains. Phase-aware progress display (Phase 1/4, 2/4, etc.), guard function
prevents concurrent runs, fire-and-forget wishlist at end. Re-sync loop
catches newly downloaded tracks on next scheduled run.
- New action type 'playlist_pipeline' with handler, blocks endpoint config,
builder UI (playlist select, process all, skip wishlist checkboxes),
help modal, result display map, and Hub template
- Removed 3 redundant Hub templates (Release Radar, Discovery Weekly,
Playlist Auto-Sync) — all replaced by the pipeline
- Fixed sync completion polling (status is 'finished' not 'complete')
- Fixed refresh handler progress hijack (null out _automation_id)
- Fixed matched_tracks field access from sync_states result
Also in this commit:
- Wishlist badges on enhanced search and global search tracks (amber)
- Discogs added to manual enrichment modal search + artist/album dropdowns
- Profile PIN forgot recovery on profile selection dialog
Watchlist map nodes used w.get('key', '') which returns None when the
DB column is NULL (key exists with None value). None serialized to JSON
null, which is falsy in JS, causing 'No source ID' throw for every
artist click.
- Changed to `w.get('key') or ''` for all ID fields (coerces None→'')
- Added discogs_id to watchlist nodes (was missing entirely)
- Removed hard throw when no source ID — falls back to name-based lookup
- Added console.error logging for future diagnosis
Tidal, Qobuz, HiFi, and Deezer results were blindly taking the first
API result with minimal validation. Now all streaming sources use
score_track_match() — same 60% title / 30% artist / 10% duration
weighting as Soulseek, plus version detection penalties.
- web_server.py get_valid_candidates(): replaced loose title-sim check
with matching engine scoring, version penalty for live/remix/acoustic
- download_orchestrator.py: optional expected_track param enables
scoring in search_and_download_best (backward compatible)
- sync_service.py: passes spotify_track for validation
- Fixed wrong class name (MusicMatchingEngine not MatchingEngine)
cover.jpg was always written from Spotify/iTunes URL (640x640) when the
first track in an album reached _download_cover_art before MusicBrainz
lookup completed. Later tracks with MBID skipped because file existed.
Fix: when cover.jpg exists but is small (<200KB) and we now have a CAA
MBID, attempt to upgrade it with the high-res CAA version. If CAA fetch
fails, keep existing cover — no pointless overwrites.
Also adds Forgot PIN recovery to the profile selection PIN dialog,
reusing the same credential verification flow as the launch lock screen.
Backend reset endpoint now accepts profile_id parameter.
Runs at scan start after ID backfill — zero API calls, all DB lookups:
1. Metadata cache artist image (any source)
2. Deezer direct URL from stored deezer_artist_id
3. Deezer ID from metadata cache by name (even if not on watchlist row)
4. Album art fallback (iTunes artists have no artist images)
Also fixes update path for artists with no external IDs — falls back
to direct row ID update instead of silently matching zero rows.
- Image proxy endpoint (/api/image-proxy) for canvas CORS — allowlisted CDNs,
browser-like UA for Deezer, 24h cache headers. Direct CORS first, proxy fallback.
- Server-side 5-min cache on all artist map endpoints with auto-invalidation
on watchlist add/remove, scan complete, and new MusicMap discoveries.
- Explorer fetches similar artists from MusicMap on-the-fly when none stored,
saves to DB for instant future visits. Validates artist names against
Spotify/iTunes API before loading map — rejects gibberish with 404.
- Genre map per-genre cap removed (was 300 backend, 400 frontend).
- Center node in Explorer uses type 'center' not 'watchlist' — no longer
misidentified as a watchlist artist.
- Error overlay auto-dismisses after 2.5s and returns to Discover page.
- Helper What's New restructured with dated sections (April 4/3/2/1, March),
trimmed from ~80 to ~38 entries, date headers styled as purple dividers.
- Version modal updated with Artist Map section and recent fixes.
get_top_similar_artists now accepts require_source parameter to filter
by source ID in SQL. Previously fetched 200 artists then post-filtered,
but cycling logic (last_featured ASC) rotated artists without IDs to
the front, causing all 200 to be filtered out.
Both /api/discover/hero and /api/discover/similar-artists now pass
require_source=active_source so only artists with valid IDs are returned.
- Metadata cache backfill: batch-lookup all node names across all sources
to fill missing IDs, images, and genres
- Source-aware navigation: View Discography passes correct source to
artist page so non-active-source artists load correctly
- Constellation hover effect: 800ms delay, fade in/out animation, dim
overlay with glowing connection lines to related artists
- Click ripple animation on node selection
- Related artists list in info modal with clickable navigation
- Rich tooltip with artist photo, name, genres on hover
- Removed node dragging (caused visual desync with offscreen buffer)
- Performance: cached constellation lookups, lighter cache query
(no raw_json), canvas.width for proper DPR overlay coverage
Visual canvas map on Discover page showing watchlist artists as large
anchor bubbles surrounded by their similar artists, sized by relevance.
Layout: golden angle spiral for watchlist nodes with push-apart guarantee,
spiral packing with spatial grid collision detection for similar artists.
Offscreen buffer rendering for smooth pan/zoom (single drawImage blit).
Features:
- 320px watchlist bubbles, similar sized 25-55% by rank/occurrence
- Search bar with instant filter + animated zoom-to-node
- Tooltip with artist photo, name, genre tags on hover
- Touch support: single finger pan, pinch zoom, tap to click
- Zoom +/- buttons and fit-to-screen with smooth 250ms animation
- Click opens artist info modal (same as Your Artists)
- Loading overlay with image count progress
- Async image loading via createImageBitmap (non-blocking)
YOUR ARTISTS (major feature):
- Aggregates liked/followed artists from Spotify, Tidal, Last.fm, Deezer
- Matches to ALL metadata sources (Spotify, iTunes, Deezer, Discogs)
- DB-first matching: library → watchlist → cache → API search (capped)
- Image backfill from Spotify API for artists missing artwork
- Carousel on Discover page with 20 random matched artists
- View All modal with search, source filters, sort, pagination
- Artist info modal: hero image, matched source badges, genres, bio,
listeners/plays from Last.fm, watchlist toggle, view discography
- Auto-refresh with loading state on first load, polls until ready
- Deduplication by normalized name across all services
DEEZER OAUTH:
- Full OAuth flow: /auth/deezer + /deezer/callback
- Settings UI on Connections tab (App ID, Secret, Redirect URI)
- Token stored encrypted, auto-included in API calls
- get_user_favorite_artists() for liked artists pool
SERVICE CLIENTS:
- Spotify: added user-follow-read scope + get_followed_artists()
- Tidal: get_favorite_artists() with V2/V1 fallback
- Last.fm: get_authenticated_username() + get_user_top_artists()
FAILED MB LOOKUPS MANAGER:
- Manage button on Cache Health modal
- Browse/filter/search all failed MusicBrainz lookups
- Search MusicBrainz directly and manually match entries
- Optimized cache health queries (11 → 4 consolidated)
- Dashboard cache stats now poll every 15s
EXPLORER IMPROVEMENTS:
- Discover button on undiscovered playlist cards
- Status badges: explored/wishlisted/downloaded/ready
- Auto-refresh during discovery via polling
- Redesigned controls: prominent Explore button, icons
BUG FIXES:
- Fix album artist splitting on collab albums (collab mode fed
album-level artists instead of per-track)
- Fix cover.jpg not moving during library reorganize (post-pass sweep)
- Fix cover.jpg missing when album detection takes fallback path
- Fix wishlist auto-processing toast spam (was firing every 2s)
- Fix media player collapsing on short viewports
- Fix watchlist rate limiting (~90% fewer API calls)
- Configurable spotify.min_api_interval setting
- Better Retry-After header extraction
- Encrypt Last.fm and Discogs credentials at rest
- Add $discnum template variable (unpadded disc number)
$disc gives zero-padded "01", $discnum gives raw "1". Useful for
Plex-style naming like "102 - Track.flac" (disc 1 track 02).
Added to both _get_file_path_from_template and _get_file_path_from_template_raw
at all 6 replacement sites. $discnum always replaced before $disc to
prevent partial matching. No existing behavior changed.
1. Discover button on undiscovered playlist cards — triggers discovery
directly from Explorer instead of redirecting to Sync page. Button
changes to "Open" to reopen modal after closing.
2. Status badges on playlist cards: checkmark (in library), heart
(wishlisted), star (fully discovered), percentage (needs discovery).
Meta line shows "N in library · M wishlisted" counts.
3. Auto-refresh: polls every 5s during active discovery to update cards.
WebSocket listener for discovery:progress events. Cards refresh when
discovery completes.
4. Explored tracking: playlists get green checkmark badge after tree is
built (session-only, resets on reload).
Backend: new get_mirrored_playlist_status_counts with fail-safe design —
core discovery counts use simple reliable queries, library/wishlist
counts are best-effort extras that won't break discovery detection.
Card layout redesigned: badges inline with playlist name, discover
button below meta text, no more absolute positioning overlaps.
New feature: Failed MusicBrainz Lookups management modal accessible
from Cache Health. Browse all failed lookups with type filter tabs,
search bar, pagination. Click any entry to search MusicBrainz and
manually match — saves MBID at 100% confidence. Clear individual
entries or bulk clear all.
Backend: 4 new endpoints — failed-mb-lookups list, mb-entry delete,
musicbrainz/search (artist/release/recording), mb-match save.
Performance: Cache health stats consolidated from 11 queries to 4
using CASE expressions. Added partial index on musicbrainz_cache for
failed lookups. Dashboard cache stats now poll every 15s instead of
single fire-and-forget fetch. Failed MB type counts cached on frontend,
only re-fetched after mutations.
Also includes: library reorganize now moves cover.jpg via post-pass
sidecar sweep, and changelog updates.
When _detect_album_info_web couldn't find a good Spotify match, the
fallback dict lacked album_image_url — so _download_cover_art silently
skipped the download even though the URL was available in the context's
spotify_album object.
Now _download_cover_art accepts optional context parameter and falls
back to spotify_album.image_url or spotify_album.images[0].url when
album_info lacks the URL. All 4 call sites updated to pass context.
Feed collab mode album-level artists instead of per-track artists so
$albumartist and the album_artist tag are consistent across all tracks
in an album. Fixes media servers (Navidrome/Jellyfin/Plex) showing one
album split under multiple artist names (e.g. KPOP Demon Hunters).
- _build_final_path_for_track: resolve $albumartist from explicit batch
context or spotify_album.artists, pass album-level _artists_list to
collab mode instead of per-track artists
- _extract_spotify_metadata: same album-level artists for album_artist
tag collab resolution
- Wishlist path: pre-compute per-album artist map so all tracks from
the same album get the same artist context
- Download worker: propagate album artists array in spotify_album_context
- Test connection now shows "Discogs connection successful!" when
Discogs is the active fallback (was showing "iTunes")
- Spotify disconnect message shows correct fallback name
- Watchlist artist list: discogs_artist_id in API response
- Watchlist source badges: Discogs badge on artist cards
- Watchlist config modal: discogs_artist_id in SQL query, WHERE clause,
response, and linked provider section with badge
- CSS for watchlist-source-discogs and watchlist-provider-badge.discogs
