Commit graph

1920 commits

Author SHA1 Message Date
Broque Thomas
d9e2e129b0 Allow selecting which duplicate to keep in duplicate detector
Duplicate finding detail now shows each version as clickable — user
can choose which to keep instead of relying on auto-selection. Added
track_number as tiebreaker in auto-pick (higher track number wins
over 01, catching leftover duplicates from the playlist sync track
number bug). Track number displayed in the detail view for clarity.
2026-04-09 18:46:49 -07:00
Broque Thomas
959bca2b8d Add Run Script action to automation engine
New automation action that executes user scripts from a dedicated
scripts/ directory. Available as both a DO action and THEN action.
Scripts are selected from a dropdown populated by /api/scripts.

Security: only scripts in the scripts dir can run, path traversal
blocked, no shell=True, stdout/stderr capped, configurable timeout
(max 300s). Scripts receive SOULSYNC_EVENT, SOULSYNC_AUTOMATION,
and SOULSYNC_SCRIPTS_DIR environment variables.

Includes Dockerfile + docker-compose.yml changes for the scripts
volume mount, and three example scripts (hello_world.sh,
system_info.py, notify_ntfy.sh).
2026-04-09 18:20:29 -07:00
Broque Thomas
5ed819a062 Show SoulSync logo as fallback for missing sidebar album art
Sidebar media player now shows the SoulSync logo instead of a broken
image icon when no album art is available or when no track is playing.
Default src, onerror fallback, and clear-player paths all use
/static/trans2.png.
2026-04-09 10:51:48 -07:00
Broque Thomas
7cfd1cae3f Fix AcoustID scanner creating thousands of no-match findings
The scanner was creating a finding for every file that couldn't be
identified by AcoustID, flooding the findings list with non-actionable
entries. Users saw the scanner "stuck scanning the same files over
and over" because the no-match findings were dismissed but recreated
on every run. Now only genuine mismatches (AcoustID identifies a
different track) create findings. Errors are counted and shown in
the job log with actual error messages for debugging.
2026-04-09 10:16:53 -07:00
Broque Thomas
963a003ca0 Set playlist poster image on Plex/Jellyfin/Emby after sync
After a successful playlist sync, if the source playlist has cover
art (Spotify, Tidal, Deezer, etc.), the image is downloaded and
uploaded as the playlist poster on the media server. Plex uses
uploadPoster(), Jellyfin/Emby uses POST /Items/{id}/Images/Primary.
Navidrome skipped (no playlist image API). Failure is silent — sync
result unchanged. Automation-triggered syncs and playlists without
images are unaffected.
2026-04-09 10:08:18 -07:00
Broque Thomas
1e69d813e6 Update What's New and Help docs with recent changes
Added dead file fix options, Deezer ARL rehydration, and album
data caching to the Fixes & Improvements section and Help docs.
2026-04-09 09:44:15 -07:00
Broque Thomas
4178c1eb56 Fix Deezer ARL sync/download rehydration and add album data caching
Sync rehydration: after loading Deezer ARL playlists, checks each
for active syncs via /api/sync/status and re-attaches polling with
live card updates. Download rehydration: rehydrateModal now handles
deezer_arl_ playlist IDs, and openDownloadMissingModal routes cache
misses to the correct ARL endpoint. Fix All now prompts for dead
file action.

Album data caching: get_playlist_tracks now checks the metadata
cache before fetching album release dates from the Deezer API.
Cache hits are instant, misses are fetched and stored for future
use across all playlists. Import fixed from core.metadata_cache
instead of web_server to avoid circular dependency.
2026-04-09 09:38:13 -07:00
Broque Thomas
37d325ee10 Add "Remove from DB" option for dead file findings
Dead file fix now prompts with two options: Re-download (existing
behavior — adds to wishlist + deletes DB entry) or Remove from DB
(just deletes the dead track record without re-downloading). Works
for both single and bulk fix. Solves the issue where dismissing
dead files didn't remove the underlying track record, causing them
to reappear on every scan.
2026-04-08 16:19:48 -07:00
Broque Thomas
163498e142 Update What's New modal and Help docs for all recent changes
What's New: Added Deezer user playlists, Qobuz token auth, streaming
source artist gate, download history provenance, and comprehensive
fixes section covering artist name casing, future album skip, track
number fix, Emby sync, discovery fix fallback, and all new settings.

Help docs: Updated Qobuz auth to mention token option, added Music
Library Paths, Replace Lower Quality, and HiFi Instance Health to
Other Settings section.
2026-04-08 13:30:21 -07:00
Broque Thomas
f603f92868 Add configurable music library paths for file resolution
New setting in Settings > Library lets users add folder paths where
their music files live. The file resolver checks these paths when
looking for library files, solving Docker path mismatches and multi-
folder libraries. Required for tag writing, streaming, and orphan
detection when the media server reports paths that differ from what
SoulSync can see. Docker users mount their music folder(s) with
read-write access and add the container-side path. Default is empty
— existing users see no change.
2026-04-08 13:24:15 -07:00
Broque Thomas
195484441e Fix all artist names stored as lowercase in database
A @staticmethod _normalize_artist_name (for liked artists dedup)
shadowed the instance method of the same name (for artist import).
The static version lowercased everything, so every artist name was
stored lowercase during database scans. Renamed the static method
to _normalize_artist_name_for_dedup. Existing lowercase names will
be corrected automatically on the next database scan.
2026-04-08 12:49:38 -07:00
Broque Thomas
ec87cb6d0e Skip zero-track albums in album completeness scanner
Albums with zero local tracks were flagged as incomplete but the
auto-fill fix failed because there were no existing tracks to
determine the album folder or quality standard from. Now skipped
during scan — they'll be detected once tracks are actually added.
2026-04-08 11:49:30 -07:00
Broque Thomas
c6bebd5e09 Add opt-in setting to replace lower quality files on import
New toggle in Settings > Library: "Replace lower quality files on
import". When enabled, if a track already exists in the library at
a lower quality tier (e.g. MP3) and a higher quality version (e.g.
FLAC) is imported from staging, the existing file is replaced.
Comparison uses the existing QUALITY_TIERS system (lossless > opus/
ogg > m4a > mp3). When disabled (default), existing behavior is
unchanged — existing tracks are always kept. Also applies to regular
downloads that land on an existing file.
2026-04-08 11:32:09 -07:00
Broque Thomas
e1c5ae0eb5 Fix discovery fix search fallback and add Deezer search endpoint
Discovery fix modal now tries the user's active metadata source
first, then falls back through all available sources (Spotify,
Deezer, iTunes) in sequence. Previously hardcoded Spotify with no
fallback, leaving users without Spotify stuck on "Searching...".

New /api/deezer/search_tracks endpoint exposes the existing
DeezerClient.search_tracks() method for the fix modal. Same
request/response format as Spotify and iTunes endpoints.
2026-04-08 11:04:22 -07:00
Broque Thomas
3d96752087 Add Qobuz auth token login as CAPTCHA bypass alternative
Qobuz added reCAPTCHA to their login endpoint, blocking automated
email/password auth for new users. Token login lets users paste
their X-User-Auth-Token from the browser DevTools after logging in
manually. Added to both Connections and Downloads tabs with
instructions. Existing email/password flow completely unchanged.
Backend validates token via user/get API and saves the session
identically to email/password login.
2026-04-08 10:37:38 -07:00
Broque Thomas
e1f7b8f5cc Add HiFi API instance health check to Settings
New "Check All Instances" button in Settings > Downloads > HiFi
shows each configured instance with color-coded status: green for
fully working (can download), orange for search-only (downloads
fail), red for offline/SSL error/timeout. Helps users understand
why HiFi downloads aren't working when community instances go down.
2026-04-08 09:58:55 -07:00
Broque Thomas
6180135f94 Add Deezer ARL token to Connections tab with bidirectional sync
ARL field now appears on both Connections tab (under Deezer OAuth)
and Downloads tab. Both fields are populated from the same config
key and synced bidirectionally via input listeners. Editing either
field instantly updates the other.
2026-04-08 08:02:43 -07:00
Broque Thomas
ba7eb1c5e7 Remove debug test activity feed message from startup 2026-04-08 07:48:18 -07:00
Broque Thomas
bd45e89515 Fix playlist sync tracks always tagged as track number 01
When downloading individual tracks from a playlist sync, the album
detection fallback hardcoded track_number to 1 instead of using the
value already available in the download context from the playlist
metadata. Now checks original_search.track_number and track_info
.track_number before falling back to 1. Full album downloads and
Spotify API lookup paths are completely unchanged.
2026-04-08 07:45:11 -07:00
Broque Thomas
0ad385be9f Update Help & Docs with missing features and new additions
New sections: Download History (source provenance, AcoustID badges),
Global Search, Deezer ARL Playlists, Deezer Link (split), Database
Maintenance (VACUUM/incremental), Artist Map (3 modes), Listening
Stats, Smart Delete (3 options), Track Redownload (3-step wizard),
Library Issues, Playlist Explorer.

Updated sections: Overview (Discogs, Emby), Connecting Services
table (Discogs row, Emby), Deezer Playlists (rewritten for ARL),
Service Credentials (Discogs token), Library Standard View (source
filter dropdown), Service Matching (10 services, Enrich dropdown),
Repair & Maintenance (expanded to all 16 jobs with table, Witness
Me safety dialog).
2026-04-07 20:13:14 -07:00
Broque Thomas
06e32d84c3 Skip future/unreleased albums in watchlist scanner
Albums announced but not yet released have no real audio available,
causing Soulseek to match random tracks with similar names. Both
discography methods (Spotify and generic client) now filter out
albums with release dates in the future. Skipped albums are not
marked as processed — they will be picked up on the first scan
after their release date passes.
2026-04-07 13:31:15 -07:00
Broque Thomas
08a7408d8b Fix playlist sync failing on Emby due to integer ID validation
The _is_valid_guid method only accepted 32-char hex GUIDs (Jellyfin
format) but Emby uses plain integer IDs like "12345". All matched
tracks were rejected as "invalid/empty IDs" causing playlist creation
to fail with zero tracks. Now accepts both numeric strings (Emby)
and hex GUIDs (Jellyfin).
2026-04-07 13:12:51 -07:00
Broque Thomas
31518a3ef3 Add Deezer user playlists tab via ARL authentication
New "Deezer" tab on sync page shows authenticated user's playlists,
identical to the Spotify tab pattern — same card layout, details
modal with track list, sync button, and download missing tracks
flow. Existing URL import renamed to "Deezer Link" tab (unchanged).

Backend: get_user_playlists() and get_playlist_tracks() on the
download client fetch via public API with ARL session cookies.
Album release dates batch-fetched for $year template variable.
Three new endpoints: arl-status, arl-playlists, arl-playlist/<id>.

Frontend: cards use Spotify-identical HTML structure with live sync
status, progress indicators, and View Progress/Results buttons.
Downloads reuse openDownloadMissingModal with zero modifications.
Track data cached on first open, instant on subsequent clicks.
2026-04-07 12:47:15 -07:00
Broque Thomas
e65b6bab67 Add metadata source filter to library and fix Discogs enrichment
Library page: new dropdown filter to show artists matched or unmatched
to any metadata source (Spotify, MusicBrainz, Deezer, Discogs, etc).
Select "No Discogs" to find artists needing manual Discogs matching.
Filter applied as WHERE clause on the source ID columns.

Discogs enrichment: added to valid_services whitelist, _enrichment_locks,
and _run_single_enrichment handler. The Enrich button was returning an
error when Discogs was selected from the dropdown.
2026-04-07 09:29:48 -07:00
Broque Thomas
d597123a40 Add Discogs to enrichment service whitelist
Discogs was missing from valid_services, _enrichment_locks, and
_run_single_enrichment handler. The Enrich button on the enhanced
library page returned "service must be one of" error when Discogs
was selected. Added handler using _process_item pattern matching
other batch-style workers, with guard against track-level enrichment.
2026-04-07 08:45:01 -07:00
Broque Thomas
23b80a0077 Add database maintenance UI with VACUUM and incremental vacuum
Settings > Advanced now shows database size, free pages, and auto-
vacuum mode. Two actions: Compact Database (full VACUUM to reclaim
dead space) and Enable Incremental Vacuum (one-time setup for
automatic page reclamation). Both have confirmation dialogs warning
about lock time on large databases. Info refreshes on Advanced tab
switch and after each operation.
2026-04-06 22:57:52 -07:00
Broque Thomas
88890f816f Fix download history timestamps always showing 'Just now'
SQLite CURRENT_TIMESTAMP stores UTC but the format lacks a timezone
marker. JavaScript parsed it as local time, causing future-dated
timestamps and negative diffs that always fell into 'Just now'.
Normalize by appending 'Z' to mark as UTC before parsing.
2026-04-06 22:24:38 -07:00
Broque Thomas
06defcfa3d Fix streaming source matching and global search download bubbles
Streaming matching: add artist gate rejecting candidates with artist
similarity below 0.4, raise threshold to 0.60, block fallback to
Soulseek filename matcher for Tidal/Qobuz/HiFi/Deezer. Fix single-
char artist containment bug where normalize_string strips non-ASCII
(e.g. "B小町" → "b") causing "b" to match any artist containing
that letter. Fixed in both score_track_match and the Soulseek scorer.
YouTube and Soulseek matching behavior unchanged.

Global search: add registerSearchDownload() calls to _gsClickAlbum
and _gsClickTrack so downloads create bubble snapshots on dashboard
and search page, matching the enhanced search standard.

Global search escaping: add _escAttr() helper to handle newlines in
album/artist names that broke inline onclick string literals.
2026-04-06 22:15:21 -07:00
Broque Thomas
410bddd102 Redesign download history with collapsible entries and full provenance
Entries are now compact cards that expand on click to reveal source
details. Shows expected vs downloaded title/artist with red mismatch
highlighting. Source artist column added to DB. Streaming track IDs
extracted from the id||name filename pattern. File and ID always on
their own line to avoid edge-case misplacement.
2026-04-06 19:33:02 -07:00
Broque Thomas
f8fbcb507c Add source provenance and AcoustID result to download history
Track original source filename, track ID, and AcoustID verification
result for every download. Helps debug wrong-file downloads from
streaming sources like Tidal. Each column migrated independently
for crash safety. Frontend shows source detail line and color-coded
AcoustID badge per entry. Button renamed to "Download History".
2026-04-06 16:18:58 -07:00
Broque Thomas
afd5125262 Add collapsible accordion UI to Settings Connections tab
Visual overhaul of the API Configuration section: each service frame
is now a collapsible accordion with brand-colored dots, chevron
indicators, and smooth expand/collapse animations. Includes an
Expand All / Collapse All toggle. No functional changes — all element
IDs, save/load logic, and tab switching preserved.
2026-04-06 12:39:05 -07:00
Broque Thomas
e73c1e69c2 Reorder Settings Connections tab for better UX
Metadata Source selector moved to top (sets context for everything below).
Services grouped logically: metadata sources (Spotify, iTunes, Deezer,
Discogs) → streaming services (Tidal, Qobuz) → enrichment (Last.fm,
Genius) → verification (AcoustID) → scrobbling (ListenBrainz) → Hydrabase.
2026-04-06 11:56:01 -07:00
Broque Thomas
893bff79cd Update README with session changes
- AcoustID now runs for all sources, not just Soulseek/YouTube
- Metadata source is user-selectable, Spotify no longer auto-overrides
- Streaming match validation applies same scoring as Soulseek
- Playlist Pipeline automation added
- Live/Commentary Cleaner maintenance job added
- Discogs enrichment worker (10 workers total)
2026-04-06 11:36:54 -07:00
Broque Thomas
3c211eaac8 Make Spotify a selectable metadata source instead of auto-override
Spotify was automatically the primary metadata source whenever
authenticated, ignoring the user's fallback dropdown selection. Now:

- Dropdown renamed "Fallback Source" → "Primary Source" with Spotify
  added as an option alongside iTunes, Deezer, Discogs
- User's selection drives search results, discovery page, status display
- Enhanced search uses selected primary source first, others as tabs
- Discovery page filters by selected source's artist IDs
- Spotify auth still works for playlists, followed artists, enrichment
- Default is 'deezer' — users who want Spotify select it explicitly
2026-04-06 11:27:49 -07:00
Broque Thomas
db5bdb9c59 Fix Opus/AAC missing cover art when source FLAC has no embedded pictures
FLAC → Opus/AAC conversion only tried to read embedded art from the
source FLAC. If art was only in cover.jpg (not embedded), the lossy
copy got no art. Now falls back to cover.jpg in the same directory.
2026-04-06 10:14:08 -07:00
Broque Thomas
03564ed026 Enable AcoustID verification for all download sources
Previously skipped for Tidal/Qobuz/HiFi/Deezer as "trusted API sources"
but streaming APIs can return wrong versions (live, remix, cover).
AcoustID now runs for every download source when enabled.
2026-04-06 09:53:54 -07:00
Broque Thomas
f3f0234628 Stop tracking user-specific files (config, database, logs)
Co-Authored-By: Antti <12547765+kettui@users.noreply.github.com>
2026-04-06 09:33:14 -07:00
Broque Thomas
91f662646b Fix Spotify OAuth scope mismatch + auth validation in callbacks (#253)
Added user-follow-read scope to all 5 SpotifyOAuth instances in
web_server.py (was only in spotify_client.py). Fixed OAuth callbacks
using is_authenticated() instead of is_spotify_authenticated() — the
former returns True with iTunes/Deezer fallback, masking auth failures.

Credit: kettui (PR #253) identified both issues.
2026-04-06 08:20:26 -07:00
Broque Thomas
1fb66b711d Fix Live/Commentary Cleaner missing _get_settings method 2026-04-06 07:45:07 -07:00
Broque Thomas
d16aaab0aa Fix empty artist folder created at base level with custom path templates
When using templates like "albums/$albumartist/$album/$track - $title",
post-processing created Transfer/ArtistName/ before computing the
template path, leaving an empty folder. _build_final_path_for_track
handles all directory creation based on the template — the premature
makedirs was redundant and incorrect.
2026-04-05 22:05:29 -07:00
Broque Thomas
51a433a558 Fix duplicate artists in search, per-artist name refresh, global search track click
- search_artists() now filters by active media server — no more duplicate
  results from Plex/Jellyfin/Navidrome showing the same artist 3 times
- Per-artist Sync button re-fetches artist name from media server, catches
  renames (e.g., Plex changing "Kendrick Lamar" to "eastside k-boy")
- Global search track click opens download modal directly instead of
  navigating to enhanced search page (matches enhanced search behavior)
2026-04-05 20:41:43 -07:00
Broque Thomas
20c1828cde Fix orphan file bulk action showing deletion warning before staging option
Reversed the flow: user now sees "Move to Staging" vs "Delete" choice
first, instead of the scary "permanently delete N files" dialog. Staging
gets a friendly confirmation. Delete ≤50 gets standard confirm. Delete
>50 still requires witness-me safety gate. Updated prompt text to
clarify staging is safe and reversible. (#252)
2026-04-05 19:12:14 -07:00
Broque Thomas
455945e6cb Fix duplicate detector missing cross-album duplicates (#252)
Changed ignore_cross_album default from True to False. Re-downloads of
the same song create separate album entries, so the detector was skipping
them. Users who want to keep compilations/greatest-hits intact can toggle
it back on. Updated help text to explain when to use this setting.
2026-04-05 18:48:03 -07:00
Broque Thomas
a2b9e32d04 Add download source tracking to library history modal
New download_source column on library_history table records which source
(Soulseek, Tidal, Qobuz, HiFi, YouTube, Deezer) each track was downloaded
from. Extracted from context username during post-processing.

Frontend shows source badge alongside quality badge on each download entry.
Source breakdown bar below tabs shows per-source totals with color-coded
chips (e.g., "Soulseek: 847 | Tidal: 203"). Includes DB migration for
existing installs. Existing entries show quality only (source is NULL).
2026-04-05 18:03:46 -07:00
Broque Thomas
7a24431e46 Redesign watchlist linked artist section with per-source match controls
Replaced single "Change" button with per-source rows showing match
status for each provider (Spotify, Apple Music, Deezer, Discogs).
Each row has Fix/Match button that searches that specific source API,
plus clear button to remove individual matches.

- Per-source search uses _search_service (same as enrichment modal)
- Backend: added Discogs to valid providers, empty ID clears match
- Fixed provider validation to accept 'discogs' alongside others
- Clear sets DB column to NULL instead of rejecting empty string
2026-04-05 14:20:35 -07:00
Broque Thomas
d4ce345ae2 Backfill all metadata source IDs during manual watchlist scan
Manual scan path was only backfilling the active provider (e.g., only
Spotify IDs if Spotify was active). Now matches the auto-scan behavior:
backfills iTunes, Deezer, Spotify (if auth'd), and Discogs (if token)
for all watchlist artists before scanning begins.
2026-04-05 14:09:14 -07:00
Broque Thomas
8344cf2c7a Decouple wishlist processing and watchlist scanning — allow concurrent execution
Removed cross-guards that blocked wishlist downloads while watchlist scans
ran (and vice versa). Downloads use bandwidth, scans use API calls —
different resources. The per-call rate limiter handles any API contention.

- Automation engine: each handler now has self-only guard (no cross-block)
- _process_wishlist_automatically: removed watchlist scanning check
- Pipeline Phase 4: removed watchlist scanning check
- Manual watchlist scan endpoint: removed wishlist processing check

Users with large watchlists (6+ hour scans) will now see downloads
starting immediately instead of waiting for the scan to finish.
2026-04-05 13:25:44 -07:00
Broque Thomas
8ba1585646 Paginate wishlist modal to prevent browser crash on large wishlists (5K+)
Initial load fetches 200 tracks instead of all. "Load More" button at
bottom shows (200 of 5,000) and loads next batch on click. Backend now
returns total count alongside limited results for both album and singles
categories. Rendering logic unchanged — just operates on smaller sets.
2026-04-05 13:15:24 -07:00
Broque Thomas
5cecb46788 Fix spotipy deprecation warning for get_access_token(as_dict=True)
Removed as_dict=True from all three OAuth callback sites. Spotipy will
return token string directly in future versions — our code only checks
truthiness so both dict and string work.
2026-04-05 12:43:26 -07:00
Broque Thomas
e674a79c88 Persist API call history, record rate limit events, fix Spotify re-auth issues
API Call Tracker:
- Save/load 24h minute-bucketed history + events to database/api_call_history.json
- Persists across server restarts via atexit + signal handler hooks
- New record_event() for rate limit bans (called from _set_global_rate_limit)
- New get_debug_summary() for Copy Debug Info — 24h totals, peak cpm with
  timestamp, per-endpoint breakdown, and last 20 rate limit events
- Fixed race condition: events iteration now inside lock during save

Spotify Rate Limit Mitigation:
- Enrichment worker: max_pages=5 on get_artist_albums (was unlimited — artist
  with 217 albums caused 22 paginated API calls, now capped at 5)
- Enrichment worker: inter_item_sleep raised from 0.5s to 1.5s

Spotify Re-Auth Fix:
- Both OAuth callbacks (port 8008 + 8888) now clear rate limit ban AND
  post-ban cooldown after successful re-auth — Spotify usable immediately
  instead of stuck on Deezer fallback for 5 minutes
- Auth cache invalidated on both global client and enrichment worker client
2026-04-05 12:36:58 -07:00