GitHub issue #505 (PopeBruhLXIX): users with multiple Plex music
libraries (e.g. one per Plex Home user, or two folder roots split
across separate library sections) only saw one library inside SoulSync
because the connection settings forced you to pick a single library
section. SoulSync's PlexClient stored exactly one ``self.music_library``
section reference and every read scanned only that one.
This change adds an opt-in "All Libraries (combined)" dropdown option
that flips the client into a server-wide read mode where every read
method (``get_all_artists`` / ``get_all_album_ids`` /
``search_tracks`` / ``get_library_stats`` / etc) dispatches through
``server.library.search(libtype=...)`` instead of querying a single
section. One Plex API call replaces N per-section iterations; Plex
handles the aggregation server-side.
Implementation:
- ``ALL_LIBRARIES_SENTINEL`` (``'__all_libraries__'``) — module-level
constant used as the saved DB preference value when the user picks
the synthetic "All Libraries" entry. Detection is one string compare
in ``_find_music_library`` / ``set_music_library_by_name``. Existing
preferences (real library names) are unaffected.
- ``self._all_libraries_mode`` (private flag) + ``is_all_libraries_mode()``
(public accessor for external callers). When True, ``music_library``
may stay None — ``is_fully_configured()`` recognizes the mode and
still returns True so dispatch sites don't bail.
- New private helpers ``_can_query``, ``_get_music_sections``,
``_all_artists``, ``_all_albums``, ``_all_tracks``, ``_search_general``,
``_search_artists_by_name``. Single dispatch point for the
section-vs-server branch — every read method funnels through them
so future drift fails at one place.
- New public helpers for downstream callers:
- ``get_recently_added_albums(maxresults, libtype)`` — used by
DatabaseUpdateWorker's deep-scan recent-content sweep
- ``get_recently_updated_albums(limit)`` — same
- ``get_music_library_locations()`` — returns folder roots, used
by web_server.py's file-path resolver
- ``trigger_library_scan`` and ``is_library_scanning`` fan out across
every music section in all-libraries mode.
- ``get_available_music_libraries`` prepends a synthetic
``{'title': 'All Libraries (combined)', 'value': sentinel}`` entry
ONLY when more than one music library exists. Single-library users
don't get the extra option. ``value`` field is the canonical
identifier the frontend submits to ``/api/plex/select-music-library``
(real libraries: title; synthetic: sentinel string). Backward-
compatible — entries without ``value`` fall back to ``title``.
Three crash points fixed in downstream consumers (would have failed
during a deep scan after the user picked all-libraries mode):
1. ``database_update_worker.py:411`` — bailed out with "No music
library found in Plex" because ``not self.media_client.music_library``
evaluated True in all-libraries mode (music_library is None there).
Now uses ``is_fully_configured()`` which recognizes the mode.
This was the root cause of the deep scan never starting.
2. ``database_update_worker.py:_get_recent_albums_plex`` — reached
``self.media_client.music_library.recentlyAdded()`` /
``.search()`` directly, AttributeError in all-libraries mode.
Now routes through the new helper methods.
3. ``web_server.py:10947`` (file-path resolver) — accessed
``music_library.locations``; gated on ``music_library`` truthy so
it didn't crash, but silently skipped all-libraries-mode locations.
Now uses ``get_music_library_locations()`` which unions across
sections.
Plus polish:
- ``/api/plex/clear-library`` also resets ``_all_libraries_mode``
so a fresh "select library" flow doesn't inherit stale mode state.
- ``/api/plex/music-libraries`` surfaces "All Libraries (combined)"
as ``current_library`` when in mode (settings UI displays correctly).
- Frontend ``loadPlexMusicLibraries`` uses ``library.value || library.title``
so the sentinel-keyed option submits the sentinel string, not the
human-readable label. Pre-select match handles both paths.
Honest tradeoffs (documented as known limitations):
- Same artist appearing in multiple Plex sections shows as separate
entries in SoulSync (no dedup). Plex returns distinct ratingKeys
for each. Cosmetic; revisit if it bites users.
- Write-back (genre / poster updates) targets one ratingKey at a time
— only updates that section's copy. Other sections' copies stay
unchanged.
- All-libraries mode includes any audiobook library that Plex
classifies as ``type='artist'``. Edge case, opt-in only.
Tests: 21 new tests in tests/media_server/test_plex_all_libraries.py
pin both single-library mode (regression guard) and all-libraries mode
for every refactored method. Existing test_plex_pinning.py fixture
updated to initialize the new flag. 63/63 media_server tests green,
2148/2148 full suite green.
Two fixes.
(1) Discography endpoint now does server-side per-source ID resolution.
When the user clicked Download Discography on a library artist, the
endpoint received whichever artist ID the frontend happened to pick
(spotify_artist_id || itunes_artist_id || deezer_id || library_db_id)
and dispatched it as-is to whichever source it queried. If the picked
ID didn't match the queried source's ID format, the lookup returned
wrong-artist results (numeric ID collisions) or fell back to a fuzzy
name search that picked a wrong artist.
Two reproducible cases:
- 50 Cent's library row had DB id 194687 — coincidentally a real
Deezer artist ID for "Young Hot Rod". When the frontend's
/enhanced fetch silently fell back to the DB id, the backend
sent 194687 to Deezer, and Deezer returned Young Hot Rod's
50 albums in 50 Cent's discography modal.
- Weird Al's library row had a stored Spotify ID. The frontend
sent that to Deezer, which rejected the alphanumeric ID and
fell back to fuzzy name search — which picked The Beatles
somehow, returning 45 Beatles albums.
The mechanism for per-source ID dispatch already exists in
``MetadataLookupOptions.artist_source_ids``, and the watchlist scanner
already uses it; the on-demand discography endpoint just wasn't wired
to it. Fix: when the URL artist_id matches a library row by ANY stored
ID (DB id, spotify_artist_id, itunes_artist_id, deezer_id, or
musicbrainz_id), pull every stored provider ID and pass them as
``artist_source_ids``. Each source gets its OWN stored ID regardless
of which one the URL carries. When the URL ID is a non-library
source-native ID and the row lookup misses entirely, behavior is
identical to before (single-ID dispatch fallback).
Logged the resolved per-source ID dict at INFO so future "wrong artist
showed up" diagnostics are immediately legible in app.log.
(2) Logger namespace fix in core/artists/quality.py and
core/metadata/multi_source_search.py.
Both modules used ``logging.getLogger(__name__)`` which resolves to
``core.artists.quality`` / ``core.metadata.multi_source_search`` —
neither under the ``soulsync`` namespace where the file handler is
wired. Result: every [Enhance], [MultiSourceSearch], and direct-lookup
INFO line was being written to a logger with no handlers and silently
dropped. App log showed the slow-request warning but no diagnostic
detail. Switched both to ``get_logger()`` from utils.logging_config so
the soulsync.* namespace picks them up. Same content, now actually
lands in app.log. Confirmed working in live test:
``[Enhance] Direct lookup matched: deezer ID 1476162252 → 'Desastre'``
No behavior change in any other caller. Empty ``artist_source_ids``
(no library row matched) reaches lookup as ``None`` → identical to
current single-ID dispatch path. Logger fix is pure routing — no
content change.
Followup on the previous Enhance refactor. Multi-source parallel text
search closed the worst case (users with no Spotify/Deezer getting
"unknown artist - unknown album - unknown track" wishlist entries),
but text search itself is still fragile against messy library tags:
"Title (Live)", featured artists in the artist field, etc. Download
Discography never had this problem because it resolves albums by stable
ID, not by name.
Enhance now does the same thing for tracks: for every metadata source
the user has configured, if the library track has the corresponding
stored ID (spotify_track_id / deezer_id / itunes_track_id / soul_id),
call client.get_track_details(stored_id) directly and convert to the
wishlist payload. First success wins. The user's configured primary
source is tried first so a Deezer-primary user gets Deezer payloads on
the wishlist entry (correct cover art / album shape) even when other
sources also have stored IDs for the same track.
Multi-source parallel text search stays as the fallback for tracks
with no stored IDs (e.g. manually imported, never enriched). Empty-
field rejection still gates the wishlist add.
Implementation:
- _STORED_ID_COLUMNS: source name → DB column mapping
(Discogs intentionally omitted — release-based, no per-track IDs)
- _enhanced_to_wishlist_payload: converts the get_track_details
intermediate "enhanced" shape (artists as [str]) to wishlist shape
(artists as [{'name': str}]). Spotify's raw_data is already in
wishlist shape, returned as-is when detected (preserves full
album.images that the enhanced top-level fields drop)
- _try_direct_lookup_all_sources: iterates sources preferred-first,
calls get_track_details on each that has both a stored ID and a
configured client, returns first complete-metadata payload
- spotify_client field removed from ArtistQualityDeps (no longer
used — Spotify direct lookup now flows through the generic
per-source loop using the entry from search_sources)
- _try_upgrade_to_rich_payload removed (was Spotify-only with broken
shape semantics for non-Spotify sources; search-fallback now uses
_build_payload_from_track consistently)
- get_primary_source() consulted to set the per-call preferred source
for direct-lookup priority
Also fixed a stale UI string: the Enhance modal toast read "Matching
tracks to Spotify and adding to wishlist..." regardless of which
sources were actually configured. Now reads "Matching tracks across
metadata sources...".
Tests:
- _build_deps mirrors web_server._resolve_search_sources: passing
spotify=spotify_obj auto-prepends ('spotify', spotify_obj) to
search_sources (Spotify is always added when configured in prod)
- 5 new tests pin the direct-lookup behavior:
- test_direct_lookup_via_deezer_id_skips_text_search
- test_direct_lookup_via_itunes_id_skips_text_search
- test_direct_lookup_prefers_user_primary_source
- test_direct_lookup_falls_through_to_text_search_when_no_stored_ids
- test_direct_lookup_failure_falls_through_to_text_search
- Reframed enhanced-format and search-fallback tests for the new
payload-build path (no album-image side call, search-fallback uses
_build_payload_from_track consistently)
- 22/22 quality tests green, 2133/2133 full suite green.
Track Redownload had been doing parallel multi-source metadata search
across every configured source the whole time; Enhance Quality was
running a single-source primary fallback that returned junk matches
with empty fields when the primary was iTunes (Discord report:
"unknown artist - unknown album - unknown track" wishlist entries
for users with neither Spotify nor Deezer connected).
Lift the redownload search into core/metadata/multi_source_search.py
and point both flows at it. Same scoring, same per-source query
optimization (Deezer's structured artist:/track: form), same
current-match flagging via stored source IDs.
ArtistQualityDeps now takes get_metadata_search_sources (returns
[(name, client), ...] for every configured source) instead of the
single-primary get_metadata_fallback_client + get_metadata_fallback_source.
Spotify direct-lookup stays as a fast-path optimization (only Spotify
exposes get_track_details(id) returning rich raw payload); when it
doesn't fire, the multi-source parallel search picks the cross-source
best match. Empty-field matches still rejected before wishlist add.
Tests: _build_deps helper updated to accept the new search_sources
contract while preserving fallback_client/fallback_source ergonomics.
Reframed tests for the new semantics — direct-lookup is no longer
gated on Spotify being the active primary; failure reason now lists
every searched source. Added a test pinning the no-sources-configured
prompt. 17/17 quality tests green, 2128/2128 full suite green.
Discord report: clicking Enhance Quality on an artist with neither
Spotify nor Deezer connected added tracks to the wishlist as
"unknown artist - unknown album - unknown track".
Root cause was structural. core/artists/quality.py had a hardcoded
Spotify-direct → Spotify-search → iTunes-fallback chain that ignored
the user's configured primary metadata source. When Spotify wasn't
connected, every track fell through to an iTunes-only fallback that
occasionally returned matches with empty fields (cleared the 0.7
confidence threshold but missing artist / album / title). Those
empty strings propagated through the wishlist payload normalizer's
truthy-check passthrough at core/wishlist/payloads.py:77-80 and the
UI rendered them as "Unknown" defaults.
Rewrote the flow source-agnostic:
- ArtistQualityDeps gains get_metadata_fallback_source. Flow resolves
the user's active primary source once up front.
- New _build_payload_from_track helper produces the Spotify-shaped
wishlist payload from any source's Track object — single place
that knows how to construct it (replaces the duplicate construction
in the Spotify-search and iTunes-fallback paths).
- New _search_match helper does generic confidence-scored search
against any client implementing search_tracks(query, limit). Same
0.7 threshold, same album-bonus weighting as before.
- New _has_complete_metadata validator rejects matches with empty
title / album / artists before they reach the wishlist.
- _spotify_direct_lookup kept as a Spotify-only optimization (only
Spotify exposes get_track_details(id) returning rich raw payload);
other sources fall through to search.
- Failure reason now names the active source: "No usable {source}
match — connect another metadata source for better coverage".
Result: Discogs users get a Discogs search. Hydrabase users get a
Hydrabase search. iTunes users get an iTunes search with empty-field
rejection. Spotify keeps its direct-lookup fast path.
6 new tests pin the architectural change:
- Primary-source dispatch routes to the configured client (Discogs,
not Spotify) when Spotify isn't primary
- Spotify direct-lookup is gated on Spotify being the active primary
(skipped when Discogs is configured even if track has spotify_track_id)
- Empty title / album / artists fields all reject the match
- Failure reason names the active source
Going line-by-line through the engine package + boot wiring. Five
small things worth fixing before Cin reads it:
(1) MediaServerEngine class docstring still claimed to be a "single
entry point for cross-server library operations" — but the prior
honesty pass cut all the cross-server dispatch wrappers because they
had no callers. Class is really lookup + small accessors now.
Docstring rewritten to match.
(2) configured_clients() had a dead `not hasattr(client, 'is_connected')`
branch. is_connected is in REQUIRED_METHODS so every client the
registry yields here implements it. Branch removed; comment notes
the reasoning.
(3) types.py imported `datetime` and `Dict` but used neither —
dead imports dropped.
(4) types.py docstring claimed "all four servers" defined an
XTrackInfo dataclass. Actually only Plex / Jellyfin / Navidrome
did; SoulSync uses richer per-track wrappers. Fixed.
(5) web_server.py boot:
- media_server_engine added to the chained `= None` declaration
so it's always defined before the try/except, defending against
the rare path where engine init AND fallback both raise.
- Outer engine init failure logger now uses exc_info=True for full
traceback (boot-time issues are rare but worth diagnosing).
- Nested fallback failure now logs explicitly instead of silently
leaving media_server_engine as None.
Tests: 2121 still pass.
Five tightening passes anticipating Cin / JohnBaumb's review nits:
(1) Engine no longer reaches into ``registry._instances`` private
attr. New public ``MediaServerRegistry.set_instance(name, client)``
method — engine constructor calls it for the ``clients=`` pre-built
case so internal storage stays encapsulated.
(2) Engine module docstring no longer overclaims. Originally said it
"Replaces the historic 33+ if/elif chains" — but only the four
uniform-shape ``is_connected`` chains were collapsed. The 19 chains
that do server-specific work (Plex raw API vs Jellyfin / Navidrome
client methods returning different shapes) stay explicit per the
"lift what's truly shared" standard. Docstring rewritten to say
exactly that.
(3) Per-method exception swallows upgraded from ``logger.debug`` to
``logger.warning``. Returning safe defaults stays the right behavior
for a read-side engine (Plex offline shouldn't crash the app), but
silent debug-level swallowing made debugging hard — JohnBaumb pushed
the download engine to surface real errors. Same treatment here:
default still safe, but the warning tells you Plex is down.
(4) ``_safe_init_media_client`` in web_server.py now logs the
exception type + traceback. Broad ``except Exception`` is still
intentional (any failure means that one server can't be used; the
others stay up) but the boot log now distinguishes config errors
(ConnectionError, AuthenticationError) from import / dependency
failures.
(5) Two new tests pin the encapsulation + fallback contracts:
- ``test_engine_with_empty_clients_dict_is_safe_to_use`` — empty
engine returns safe defaults on every method, doesn't raise.
- ``test_engine_uses_registry_set_instance_not_private_attr`` — spy
on registry.set_instance verifies engine uses the public method.
Same latent bug as add-track — replace-track and remove-track only
looked up the Plex playlist by name. Plex deletes + recreates
playlists on edit so the rating key the frontend cached can be
stale, name lookups can also fail (special chars, encoding). Both
now use the same id-first / name-fallback chain as the GET tracks
endpoint, with a diagnostic log when both lookups fail.
Pre-existing latent bug, not a refactor regression.
The /api/server/playlist/<id>/add-track endpoint only looked up the
target Plex playlist by name, but Plex deletes + recreates playlists
on edit so the rating key the frontend cached can be stale. The
companion GET /tracks endpoint already had id-first / name-fallback;
add-track now does the same.
Added a warning log on GET /tracks when BOTH lookups fail so the
"all source items show Find & Add" symptom (which happens when
server_tracks comes back empty) has a clear diagnostic in the log
instead of silently rendering an empty server column.
Not a refactor regression — the original code had the same name-only
lookup. The mass-replace of `plex_client` → `media_server_engine.client('plex')`
is byte-equivalent. Just surfacing the latent bug.
If MediaServerEngine init raised, ``media_server_engine`` was set
to None. Every downstream caller (now that the per-server globals
are gone) does ``media_server_engine.client('plex')`` style access
— which would AttributeError on the None.
Pre-refactor each per-server global had its own try/except so engine
failure didn't take down per-server dispatch sites. Preserve that
resilience by falling back to an empty MediaServerEngine — its
``client(name)`` returns None, downstream truthy checks treat that
as "not configured" exactly the same way the legacy globals did.
Per-server web_server.py globals (plex_client / jellyfin_client /
navidrome_client / soulsync_library_client) are gone. The engine now
owns the per-server client instances; web_server.py constructs them
inline into the engine init and routes everything through
media_server_engine.client('<name>').
Multi-client consumers refactored to take the engine instead of
separate per-server kwargs:
- services/sync_service.py: PlaylistSyncService.__init__ now takes
media_server_engine. Internal _get_active_media_client resolves the
active server's client through self._engine.client(name) instead of
the per-server self.X_client attributes.
- core/listening_stats_worker.py: ListeningStatsWorker takes
media_server_engine. The plex/jellyfin/navidrome dispatch in _poll
collapses to engine.client(active_server) (gated to those three
servers — SoulSync standalone has no listening data).
- core/web_scan_manager.py: WebScanManager takes media_server_engine
instead of the hand-keyed media_clients dict that drifted out of
sync with the engine.
- core/discovery/sync.py: SyncDeps holds media_server_engine instead
of plex_client / jellyfin_client. Playlist-image dispatch routes
through engine.client(name).
Web_server.py:
- Per-server globals removed from the chained `= None` init line
+ their try/except construction blocks. Replaced with a
_safe_init_media_client(factory, name) helper that captures
per-server init failures + passes the resulting clients straight
into the MediaServerEngine init dict.
- All construction sites (PlaylistSyncService, WebScanManager,
ListeningStatsWorker, SyncDeps, library_check) updated to receive
the engine instead of per-server clients.
Test fixtures (tests/discovery/test_discovery_sync.py) gain a
_FakeMediaServerEngine stub + the SyncDeps build helper passes
that instead of separate plex/jellyfin clients.
Pre-change web_server.py had ~70 direct attribute reaches against the
per-server globals (plex_client.X, jellyfin_client.X, navidrome_client.X,
soulsync_library_client.X) plus ~60 standalone refs (truthy checks,
media_client assignments, source-name tuples). The engine was wired
but only used in 4 places, so most of the codebase still hand-dispatched
— the exact "partially defeats the purpose of this refactor" critique
Cin landed on the download PR initially.
- All ~70 client.attribute reaches migrated to
media_server_engine.client('<name>').attribute. The chains in
web_server.py do server-specific work (Plex raw API, Jellyfin /
Navidrome client methods, all returning different shapes), so the
if/elif structure stays — but the per-server CLIENT REACH now goes
through the engine like Cin's POC pattern intended.
- All ~60 standalone refs migrated:
- if plex_client → if media_server_engine.client('plex')
- media_client = plex_client → media_client = media_server_engine.client('plex')
- ('plex', plex_client) tuples → ('plex', media_server_engine.client('plex'))
- Per-server globals (plex_client / jellyfin_client / navidrome_client /
soulsync_library_client) kept for now — external modules
(PlaylistSyncService, WebScanManager, ListeningStatsWorker, search
library check, discovery sync deps) still take them as kwargs.
Dropping them entirely needs a follow-up sweep across those modules.
Suite green (1961 pass).
Apply the Cin-1 / Cin-2 pattern from the download refactor PR to the
media server engine PR before review.
Cin-1 — explicit inheritance:
- PlexClient, JellyfinClient, NavidromeClient, SoulSyncClient now
explicitly inherit MediaServerClient instead of relying on
structural typing alone. Pre-change a reader of plex_client.py
had no way to know the class was supposed to satisfy the contract.
- Removed the engine + registry re-exports from
core/media_server/__init__.py to break the circular import that
the inheritance change introduced (importing the package now
triggered a chain that loaded clients before their base class
resolved). Submodules import directly: from
core.media_server.engine import MediaServerEngine, etc.
- Conformance test now also asserts isinstance() / issubclass()
against MediaServerClient — drift in any class fails at the test
boundary instead of at runtime.
Cin-2 — generic accessors + singleton:
- engine.configured_clients() — replaces the legacy per-server
`if X and X.is_connected(): clients[name] = X` chains in
web_server.py.
- engine.reload_config(name=None) — generic dispatch, so callers
pass the server name instead of reaching for plex_client.reload_config()
directly.
- get_media_server_engine() / set_media_server_engine() singleton
factory matching the get_metadata_engine() / get_download_orchestrator()
shape. web_server.py boots via set_media_server_engine(...) so
factory + global handle share state.
- 7 new tests pin the accessors + singleton behaviour.
Two findings from JohnBaumb on the engine refactor.
(1) Every download client returned None when self._engine was None,
just logging an error. The orchestrator's download_with_fallback
treated None as "source declined", so the user got no feedback —
download silently disappeared. Now each client raises a RuntimeError
on the engine-not-wired path. download_with_fallback already catches
plugin exceptions, logs a warning, and tries the next source — so
the visible behavior is "real error in logs + fallback to next
source" instead of "silent drop". Six clients touched (deezer, hifi,
qobuz, soundcloud, tidal, youtube). Pinning tests updated to expect
raise.
(2) Monitor's engine.get_all_downloads() walked every plugin
including soulseek, but the same monitor loop already pulled slskd
transfers via the transfers/downloads endpoint a few lines earlier —
soulseek's records were being fetched twice per tick. Same issue in
web_server.py's get_cached_transfer_data path. Added an exclude
parameter to engine.get_all_downloads(); both call sites now pass
('soulseek',). New test pins the exclude semantic.
Also fixed a stray 8-space over-indent on the for-loop body in
get_cached_transfer_data (cosmetic, JohnBaumb flagged the same
pattern in monitor.py earlier).
Two architectural cleanups on top of the download engine refactor.
(1) Shared dataclasses move to neutral plugin package.
TrackResult, AlbumResult, DownloadStatus, SearchResult lived in
core/soulseek_client.py for historical reasons — every other plugin
imported them from the soulseek module just to satisfy the contract,
coupling 8 clients to a sibling source for type imports only. Moved
them to the new core/download_plugins/types.py module and updated all
14 import sites across the deezer/hifi/lidarr/qobuz/soundcloud/tidal/
youtube clients, the engine, matching engine, redownload helper, and
tests. Clean break, no backward-compat re-export.
(2) web_server.py boots the orchestrator via the singleton factory.
After construction it now calls set_download_orchestrator(...) so
get_download_orchestrator() returns the same instance the global
handle points at instead of lazily building a separate orchestrator.
Matches the get_metadata_engine() pattern.
Hunted down the remaining sites where web_server.py still reached
into orchestrator per-source attributes. Most were silently broken
after Cin-5 dropped those attrs but were guarded by hasattr checks
that always returned False — empty download_clients dicts and
no-op reload paths.
- /api/library/track/<id>/redownload-search: replaced the 6 if/hasattr
per-source blocks (the exact pattern Cin called out in his review)
with a single download_orchestrator.configured_clients() call.
- Settings reload path: hasattr-guarded YouTube reload now resolves
via client('youtube') and tests for None.
- _try_source_reuse / _store_batch_source: slsk lookup gates on
hasattr(orch, 'client') instead of the dropped 'soulseek' attr.
- /api/soundcloud/status + Deezer ARL endpoints: same hasattr
swap.
The global handle in web_server.py was named soulseek_client for
historical reasons but the type has long been DownloadOrchestrator,
not SoulseekClient. Renamed the global plus every parameter/attribute
that carried the legacy name.
- web_server.py: global var renamed; all 99 references updated.
- api/, core/downloads/*, core/search/*, core/streaming/*,
services/sync_service.py: parameter names, dataclass fields, and
init() arg names renamed.
- Test fixtures (CandidatesDeps, MasterDeps, SearchDeps, etc.) and
the _build_deps helpers updated accordingly.
The core.soulseek_client module path and SoulseekClient class name
(the actual soulseek-only client) are unchanged — only the orchestrator
handle renamed. Module imports of TrackResult/AlbumResult/DownloadStatus
from core.soulseek_client preserved.
Removed the eight backward-compat attribute aliases on the orchestrator
(soulseek, youtube, tidal, qobuz, hifi, deezer_dl, lidarr, soundcloud).
External callers and the orchestrator's own internals now reach clients
through the generic alias-aware client(name) accessor.
- core/downloads/{master,monitor,validation}.py: migrated to client().
Monitor's per-source aggregation loop replaced with a single
engine.get_all_downloads() call.
- core/search/{orchestrator,stream}.py: migrated; stream.py drops the
hand-built mode-to-client dict.
- web_server.py: migrated /api/deezer/arl-* + tidal client lookup.
- core/download_orchestrator.py: internal self.soulseek /
self.deezer_dl reaches now route through self.client(); attr
assignments dropped from __init__; module docstring updated.
- Test fakes (_FakeSoulseek, _FakeSoulseekWithYT) expose client(name)
instead of stuffing per-source attributes.
- Conformance test re-pinned to the client() accessor contract.
Three correctness fixes from kettui's PR review plus the web_server
migration to generic accessors.
- Engine alias map: register_plugin accepts aliases tuple; get_plugin
+ cancel_download resolve through it. Fixes deezer_dl cancels
silently routing to soulseek.
- Orchestrator hybrid_order normalization: _resolve_source_chain
routes raw config names through registry.get_spec() so legacy
deezer_dl entries don't drop deezer from hybrid mode.
- Atomic update_record_unless_state on the engine: holds state_lock
across the check + write. Both _mark_terminal AND the success path
use it now so a Cancelled state set mid-impl can't be clobbered.
- web_server.py: 30 soulseek_client.<source> reaches migrated to
client("<source>"); shutdown-check setup migrated to generic
registry iteration; 4 hifi reload sites use reload_instances('hifi').
- 18 new tests pin every fix.
Two more sites in web_server.py replaced (tag-preview + batch
tag-preview server_type checks). Same pattern as C1: 3-way
if/elif → engine.is_connected().
Honest scope note: the recon agent counted 33 dispatch sites,
but most are deeply server-specific logic where each branch
does completely different work (playlist track replace, per-
server metadata sync, deep scan with server-specific helpers).
Lifting those would move per-server branches into engine helper
methods that route the same work — net zero LOC, more indirection.
Engine helps where the shape is TRULY uniform; the deep dispatches
stay explicit. Phase C ends here at 4 simple sites lifted.
Suite still green.
Two sites in web_server.py replaced:
- /status route's media-server connectivity check (4-way if/elif
for plex/jellyfin/navidrome/soulsync) → engine.is_connected()
- /api/playlists endpoint's server_connected check (3-way if/elif)
→ engine.is_connected()
Engine reads active_server config + dispatches to the right client
with internal connection caching preserved (the underlying clients
all cache is_connected() calls).
Engine constructor now accepts a pre-built clients={...} dict so
web_server.py wires the same instances as its existing per-client
globals — no double-init.
Suite still green. Per-server clients still accessible via
engine.client(name) for source-specific reaches.
`_create_finding` silently dedup-skipped re-discovered issues but
the caller incremented `findings_created` regardless. So a re-scan
that found the same issues as a prior scan reported 364 findings
in the badge while 0 NEW pending rows hit the db, leaving the
findings tab empty.
`_create_finding` now returns bool (True on insert, False on
dedup-skip / db error). All 16 repair jobs updated to only
increment `findings_created` on True. Added `findings_skipped_dedup`
counter surfaced in scan log: "Done: X scanned, 0 fixed, 0
findings (363 already existed), 0 errors".
Also fixed a missing `job_id` kwarg in album_tag_consistency that
was silently breaking finding creation for that scan.
Discord request: pull user's Discogs collection into the Your Albums
section on Discover, similar to how Spotify Liked Albums works.
Implementation extends the existing 3-source pipeline (Spotify /
Tidal / Deezer) to a 4-source pipeline with click-context dispatch —
Discogs-only albums open with rich Discogs release detail (vinyl/CD
format, year, label, country, tracklist). Mirrors the per-source
dispatch pattern from enhanced/global search.
Discogs client (`core/discogs_client.py`):
- New `get_authenticated_username()` resolves the username for the
configured personal token via Discogs's `/oauth/identity` endpoint.
Cached on the instance so subsequent collection page-fetches don't
re-hit it.
- New `get_user_collection(username=None, folder_id=0, per_page=100,
max_pages=50)` walks all pages of `/users/{username}/collection/
folders/{folder_id}/releases`. Returns normalized dicts ready for
upsert_liked_album. folder_id=0 = Discogs's "All" folder.
Pagination cap of max_pages*per_page = 5000 releases — bounds
runtime on heavy collections.
- New `get_release(release_id)` thin wrapper for `/releases/{id}` —
returns the raw API response so the album-detail endpoint can
render rich context.
- Both methods defensive: missing token → empty list, malformed
responses → skipped, falsy ids → None. Disambiguation suffix
stripping (`Madonna (3)` → `Madonna`) so Discogs artist names
match what Spotify/Tidal/Deezer use.
Schema (`database/music_database.py`):
- New `discogs_release_id TEXT` column on `liked_albums_pool`.
Migration uses the established `try SELECT, except ALTER TABLE`
pattern. Idempotent; safe on existing installs.
- Added the column to the canonical CREATE TABLE for fresh installs.
- `upsert_liked_album` extended with `'discogs': 'discogs_release_id'`
in BOTH the INSERT and UPDATE id-column maps so Discogs source_id
routes to the new column. INSERT statement column count + value
count updated together.
Backend (`web_server.py`):
- `/api/discover/your-albums/sources` — adds Discogs to the
`connected` list when `discogs.token` config is set.
- `_fetch_liked_albums` — new branch for Discogs. Lazy-imports
DiscogsClient, respects the `enabled_sources` config, walks the
collection, upserts each release. Same try/except shape as the
existing source branches.
- `/api/discover/album/<source>/<album_id>` — new `discogs` branch
fetches the release via DiscogsClient.get_release, normalizes the
Discogs tracklist format, parses Discogs's `MM:SS`/`HH:MM:SS`
duration strings to milliseconds, returns the same response shape
as the Spotify/Deezer/iTunes branches.
Frontend (`webui/static/discover.js`):
- `openYourAlbumsSourcesModal` — adds Discogs to `sourceInfo` with
the vinyl emoji icon. Existing toggle/save plumbing handles it.
- `openYourAlbumDownload` — restructured the per-source dispatch:
builds an ordered list of (source, id) tuples, tries each in turn,
breaks on the first successful response. Pure-Discogs albums go
straight to the Discogs detail endpoint → modal opens with Discogs
context. Multi-source albums prefer Spotify/Deezer first since
their tracklists carry proper streaming IDs ready for download.
Tests: `tests/test_discogs_collection_source.py` — 12 cases:
- get_user_collection: empty without token, normalizes response
shape, strips disambiguation suffix, handles missing year, skips
malformed releases, paginates correctly, caps at max_pages,
uses explicit username when provided.
- get_release: passes id through to /releases/{id}, returns None
for invalid ids without API call.
- liked_albums_pool: discogs_release_id round-trips through upsert
+ get; multi-source dedup carries both Spotify and Discogs IDs
on the same row.
Verified: full suite 1825 pass (12 new), ruff clean, smoke test
populating + reading the discogs_release_id column round-trips
correctly via the real DB.
WHATS_NEW entry under '2.4.2' dev cycle.
Discord request (Samuel [KC]): show how much disk space the library
takes on the Stats page. Implementation piggybacks on the existing
deep scan — Plex/Jellyfin/Navidrome all return file size in their
track API responses, so we read it during the deep scan and store
it on the tracks row. Aggregation is then a single SQL query — no
filesystem walk, no extra I/O during the scan, no separate stat
job. SoulSync standalone gets size from os.path.getsize at insert
time (different code path; the file is local when we write the row).
Schema (`database/music_database.py`):
- New `file_size INTEGER` column on `tracks`. Migration uses the
established `try SELECT, except ALTER TABLE ADD COLUMN` pattern.
Idempotent; safe on existing installs. NULL on legacy rows so
they don't contribute to totals until next deep scan refreshes.
- Added the column to the canonical CREATE TABLE so fresh installs
get it without going through the migration path.
Track-object plumbing:
- `core/jellyfin_client.py` — JellyfinTrack reads MediaSources[0].Size
alongside existing Bitrate read. None when 0 / missing.
- `core/navidrome_client.py` — NavidromeTrack reads `size` from
the Subsonic song object (int coercion + None on parse fail).
- `core/soulsync_client.py` — SoulSyncTrack does os.path.getsize
(only "server" where size has to come from disk).
- Plex needs no client-side change: track.media[0].parts[0].size
is read directly inside insert_or_update_media_track.
Persistence — TWO separate insert paths:
(a) `database/music_database.py:insert_or_update_media_track` —
Plex/Jellyfin/Navidrome flows. Reads file_size from Plex's
MediaPart OR `track_obj.file_size` wrapper attribute (defensive
Plex-attr-not-present check + > 0 type guard).
INSERT writes the new column.
UPDATE uses COALESCE(?, file_size) so a None from the server
on a re-sync (rare Jellyfin Size omission) doesn't blank an
existing value. Pinned via test.
(b) `core/imports/side_effects.py:record_soulsync_library_entry` —
SoulSync standalone flow. Completely separate code path: the
standalone deep scan moves files to staging for auto-import
rather than calling insert_or_update_media_track. After the
auto-import processes them, side_effects writes the tracks row
directly. Reads file_size via os.path.getsize(final_path) at
insert time (file is local) and includes it in the INSERT
column list. SoulSync only does INSERT-if-not-exists (no
UPDATE path), so no COALESCE concern.
Aggregator (`database/music_database.py:get_library_disk_usage`):
- SELECT COALESCE(SUM(file_size), 0), COUNT(file_size),
COUNT(*) - COUNT(file_size) for the totals.
- Per-format breakdown done in Python via os.path.splitext over
(file_path, file_size) rows — sidesteps SQLite's first-vs-last-dot
ambiguity for paths like /music/Kendrick/M.A.A.D City/01.flac.
- Defensive: skips empty paths, paths without extension, and
implausibly long extensions (>6 chars). Returns the full
empty-shape dict (NOT a partial / undefined) when the column
doesn't exist or queries fail, so the UI's `if (!data.has_data)`
branch handles fresh installs cleanly.
API + UI:
- `core/stats/queries.py` — thin pass-through get_library_disk_usage
matching the existing query-helper convention.
- `web_server.py` — new /api/stats/library-disk-usage endpoint
mirroring the /api/stats/db-storage pattern.
- `webui/index.html` — new card in System Statistics above the
Database Storage card.
- `webui/static/stats-automations.js` — _loadLibraryDiskUsage +
_renderLibraryDiskUsage. Empty state: "Run a Deep Scan to
populate (X tracks pending)". Partial: "X measured (+Y pending)".
Full: total + format bars proportional to the largest format.
- `webui/static/style.css` — .stats-disk-* styled to match the
Database Storage card.
Backward compatibility:
- Migration is additive; existing rows get NULL file_size; the
empty-shape return from the aggregator means the UI renders
cleanly without errors before any deep scan runs.
- Old installs upgrading will see "Run a Deep Scan to populate
(N tracks pending)". Running their next deep scan fills sizes —
the existing scan flow doesn't need any changes, just consumes
the new track-wrapper attribute.
Tests:
- `tests/test_library_disk_usage.py` — 13 cases covering schema
migration, NULL defaults on legacy inserts, fresh-install empty
shape, summing with mixed NULL/known sizes, per-format breakdown,
mixed-case extensions, paths with album-name dots, missing
extensions, empty file_path, implausibly long extensions,
JellyfinTrack.file_size persistence via insert_or_update_media_track,
COALESCE preservation on null re-sync.
- `tests/imports/test_import_side_effects.py` — extended the
existing record_soulsync_library_entry test to assert
track_row['file_size'] == os.path.getsize(final_path), pinning
the SoulSync-standalone path. Test fixture's tracks schema also
updated to include the file_size column.
Verified: full suite 1813 pass (13 new, 1 existing-test extension),
ruff clean, smoke test populating + reading the column round-trips
correctly.
WHATS_NEW entry under '2.4.2' dev cycle.
User: SoundCloud downloads finish correctly but the modal stays at
"Downloading... 0%" until "Processing..." flips on. Live percentage
never updates.
Root cause: my live-progress fix in 8de4a18 made the SoundCloud client
compute progress correctly via fragment_index/fragment_count — but the
percent never reached the modal because `get_cached_transfer_data` in
web_server.py iterates `[youtube, tidal, qobuz, hifi, deezer_dl,
lidarr]` to build the lookup that drives `task.progress`. SoundCloud
was missing from that loop, so `live_transfers_lookup` had no entry
for SoundCloud downloads, so `live_info` lookup at
`core/downloads/status.py:135` always missed, so `task_status['progress']`
defaulted to 0 the entire time.
Frontend was reading `task.progress` (rendered as
"Downloading... ${task.progress}%" in `webui/static/downloads.js:3142`),
which stayed at 0. The percentComplete field that the
`/api/downloads/status` endpoint includes for SoundCloud was correct;
this only affected the cached lookup used by the V2 task tracker.
Fix: include SoundCloud in the iteration. Used `getattr` fallback to
match the same pattern I used in `core/downloads/monitor.py` so older
soulseek_client snapshots without the attribute don't AttributeError.
Bonus: also wired the SoundCloud client's `set_shutdown_check` callback
in the startup block right after HiFi's. Previously the cooperative-
cancellation hook in `_progress_hook` would never fire on shutdown
because `self.shutdown_check` was None.
Verified: full suite 1732 passed, ruff clean. yt-dlp probe confirms
fragment_index / fragment_count are populated correctly during HLS
download (164 hook calls for a 19-fragment track), so the now-
exposed progress will increment smoothly from 0 to 99.9 and then
flip to Completed.
User report: switched download source to SoundCloud and noticed:
1. Download progress % stays at 0 until "suddenly done" — no live progress
2. Sidebar status indicator next to "SoundCloud" label is red
3. Dashboard service status card still shows "Soulseek" as the source name
Fix 1 — Live progress for HLS-segmented SoundCloud downloads
(`core/soundcloud_client.py`):
- yt-dlp's `total_bytes` / `total_bytes_estimate` for HLS describes the
CURRENT FRAGMENT, not the whole download. So the byte-based
percentage stayed near 0 the entire time — until 'finished' fired.
- Added `_update_download_progress_fragmented` which uses
`fragment_index` / `fragment_count` (which yt-dlp DOES populate
accurately for HLS) to compute a meaningful percentage. Total size
is extrapolated from per-fragment average for the bytes/remaining
display. Time-remaining estimate uses elapsed/index seconds-per-
fragment.
- The progress hook prefers fragment progress when both fragment_index
and fragment_count are present; falls back to byte-based for
non-fragmented (progressive MP3) downloads. Five new unit tests pin
the fragment-progress math, the 99.9% cap, and the defensive
zero-index / unknown-id paths.
Fix 2 — Sidebar status indicator stays green for SoundCloud mode
(`web_server.py`):
- The `/api/status` route's `serverless_sources` tuple decides whether
to even probe slskd. SoundCloud (and Lidarr) were missing — so when
the active source was SoundCloud, the route fell through to "test
slskd, mark not-relevant", which set `connected: False` and turned
the sidebar dot red even though SoundCloud was working.
- Added `'soundcloud'` and `'lidarr'` to the tuple. Both are
serverless from slskd's perspective, so the dot now stays green
whenever they're the active source.
Fix 3 — Dashboard service card title shows the active source
(`webui/static/shared-helpers.js`):
- The dashboard's "Download Source" card has its own
`sourceNames` map at line 3351 (separate from the sidebar map I
already updated at 3396). Missed it during the integration PR.
- Added `'lidarr'` and `'soundcloud'` so the card title now reads
"SoundCloud" / "Lidarr" instead of falling back to "Soulseek".
Bonus — Dashboard "Test Connection" button works for SoundCloud
(`core/connection_test.py`):
- The dashboard's Test Connection button on the download-source card
sends `service` based on the active source — so for SoundCloud it
was sending `service='soundcloud'`. `run_service_test` had no
branch for it, so it fell through to "Unknown service." and the
button always failed.
- Added a `soundcloud` branch that mirrors `/api/soundcloud/status`
behavior: confirms yt-dlp is installed, runs a real cheap probe,
returns a meaningful pass/fail. (HiFi has the same gap but no
user reported it; out of scope for this fix.)
Verified:
- 41 unit tests pass (5 new fragment-progress tests added)
- Full suite 1732 passed
- Ruff clean
Plug the previously-built SoundcloudClient (PR #478, the build-and-verify
phase) into every place a download source needs to appear. Follows the
same wiring contract as Tidal/Qobuz/HiFi/Deezer/Lidarr — orchestrator
routing, hybrid-mode picker, search dispatch, queue/cancel/clear,
provenance + library history, sidebar source label, settings UI all
work plug-and-play.
Backend wiring:
- `core/download_orchestrator.py` — import SoundcloudClient, _safe_init
it at startup, add to _client() lookup, get_source_status(),
check_connection's sources_to_check default, search source_names map,
search_and_download_best _streaming_sources tuple, download
source_map + source_names, and every iteration loop in
reload_settings download-path-update / get_all_downloads /
get_download_status / cancel_download (route + iterate) /
clear_all_completed_downloads / cancel_all_downloads.
- `core/downloads/monitor.py` — added SoundCloud to the per-client
loop that fetches active downloads outside the orchestrator (uses
getattr fallback for older soulseek_client snapshots).
- `core/downloads/task_worker.py` — added SoundCloud (and Lidarr,
which was missing too — bonus fix) to source_clients dict for hybrid
fallback dispatch.
- `core/downloads/validation.py` — added 'soundcloud' to
_streaming_sources so SoundCloud results go through the matching
engine validation path instead of the Soulseek quality-filter path.
- `core/imports/side_effects.py` — three call sites: source_map for
download_source label written to library_history, streaming-source
guard for the `||`-encoded stream_id parsing, and source_service
map for provenance recording. All three now include 'soundcloud'.
- `web_server.py` — five streaming-source detection tuples updated.
New `/api/soundcloud/status` endpoint returns
{available, configured, reachable} mirroring the Deezer/HiFi
status-endpoint pattern; reachability runs a real cheap yt-dlp
search so the settings Test Connection button gives a meaningful
pass/fail signal.
- `config/settings.py` — added empty `soundcloud_download` defaults
block so future tier-2 OAuth (SoundCloud Go+ session) doesn't have
to migrate existing configs.
Frontend:
- `webui/index.html` — new `<option value="soundcloud">` in the
download-source-mode dropdown, SoundCloud added to both hidden
legacy hybrid-source selects, new settings container with info
text + Test Connection button.
- `webui/static/settings.js` — HYBRID_SOURCES entry (with the
SoundCloud cloud SVG icon), _hybridSourceEnabled default,
updateDownloadSourceUI container display, allSources for legacy
hybrid picker, testSoundcloudConnection function (hits the new
status endpoint, color-codes the result), saveSettings
soundcloud_download empty block.
- `webui/static/shared-helpers.js` — sidebar source-name map
includes SoundCloud + Lidarr (Lidarr was also missing, bonus fix).
- `webui/static/helper.js` — WHATS_NEW entry under '2.4.2' dev cycle
describing the user-visible change in the chill terse voice.
Tests:
- `tests/test_download_orchestrator_soundcloud.py` — 14 integration
tests verifying the wiring: client constructed at startup, _client
lookup resolves 'soundcloud', get_source_status includes it,
download dispatcher routes username='soundcloud' to the SoundCloud
client (and unknown usernames still fall back to Soulseek), hybrid
search iterates SoundCloud when in order and skips it cleanly when
unconfigured, get_all_downloads / get_download_status / cancel /
clear walk SoundCloud, soundcloud-only mode dispatches only to
SoundCloud, _streaming_sources tuple in validation includes
'soundcloud'.
- `tests/downloads/test_download_orchestrator.py` — added
`soundcloud` to the test fixture's _build_orchestrator helper so
the new orchestrator attribute doesn't AttributeError in pre-
existing tests that bypass __init__.
Verified:
- Full suite green (1728 passed, 2 deselected for soundcloud_live)
- Ruff clean
- Live SoundCloud-only mode search returns 25 SoundCloud tracks for
"kendrick lamar luther" in <2s, returning properly-shaped
TrackResult objects with username='soundcloud' and dispatch-key
filename ready for the download path.
Out of scope (intentional deferrals):
- SoundCloud Go+ OAuth tier (256 kbps AAC) — anonymous-only for now.
Adding auth later is a settings-page extension, no orchestrator
changes needed.
- Album/playlist support — SoundCloud has playlists but they don't
map to the album model the rest of SoulSync expects. Singles only.
Reported case (Foxxify): Tidal returned error 1002 ("Invalid redirect
URI") on every authentication attempt for users accessing SoulSync
from a network IP. User had ``http://127.0.0.1:8889/tidal/callback``
registered in his Tidal Developer Portal — matching the SoulSync UI
default and docs.
Root cause: the /auth/tidal route at web_server.py:5594-5598 had a
"fallback: dynamically set based on request host" branch that fired
when ``tidal.redirect_uri`` config was empty AND the request didn't
come from localhost. That fallback overrode the TidalClient
constructor's safe default (``http://127.0.0.1:<port>/tidal/callback``)
with a uri built from request.host like
``http://192.168.x.x:8889/tidal/callback``. Tidal compares strings
exactly so this never matched the documented portal registration and
the user got 1002 before the consent screen even rendered.
The trap is the SoulSync settings UI displays the default URI as the
placeholder + "Current Redirect URI" display — but the placeholder
never gets saved to config unless the user explicitly clicks Save.
Most users who follow the docs (register the displayed default with
Tidal, then click Authenticate) hit the empty-config path and the
broken fallback.
Fix: drop the request-host fallback. Empty config falls back to the
constructor default that matches the documented portal registration.
The existing post-auth swap-step in the instructions page below
handles the Docker / remote-access case as designed:
1. SoulSync sends 127.0.0.1:8889 in the authorize URL → matches
portal → Tidal accepts.
2. User authorizes → Tidal redirects browser to 127.0.0.1:8889
(which fails locally — nothing on user's machine listens there).
3. Instructions tell user to swap 127.0.0.1 with the host they're
accessing SoulSync from.
4. Swapped URL hits the container's exposed callback port → auth
completes.
8 regression tests in tests/test_tidal_auth_redirect_uri.py:
- Configured redirect_uri sent verbatim (localhost / custom port /
explicit network IP)
- Empty config falls back to constructor default — NOT request.host
(the actual reported scenario, with explicit assertion message
warning if the bug returns)
- Empty config + localhost access uses the same default (sanity)
Full pytest 1635 passed; ruff clean.
Discord-reported (Foxxify): logging in to Qobuz via the Connect
button on Settings showed "Connected: <username> (Active)" but
underneath an error said "Qobuz not authenticated...", and the
dashboard indicator stayed yellow. Saving settings or reloading the
tab didn't help.
Root cause: SoulSync runs two QobuzClient instances side by side —
one through soulseek_client.qobuz for the /api/qobuz/auth/* endpoints,
and a second owned by the enrichment worker thread for thread safety.
The login flow only updated the auth-flow instance's in-memory state
(plus persisted to config). The dashboard's "configured" check at
web_server.py:3371 reads
``qobuz_enrichment_worker.client.user_auth_token`` — the WORKER's
instance — which still believed itself unauthenticated. The
connection-test step at core/connection_test.py:370 hits the same
worker instance for the same reason.
Fix: add ``QobuzClient.reload_credentials()`` — a public, network-free
method that re-reads the saved session from config and updates the
instance's in-memory state + session headers. Call it on the
enrichment worker's client immediately after a successful
``/api/qobuz/auth/login``, ``/api/qobuz/auth/token``, or
``/api/qobuz/auth/logout`` so the two instances stay in lockstep
without waiting for the next process restart.
Unlike the existing ``_restore_session()`` this skips the network
probe — the caller has just authenticated, so the token is known
good. A small ``_sync_qobuz_credentials_to_worker()`` helper in
web_server.py wraps the call so all three endpoints share one path.
10 new regression tests cover the populate / clear / partial-config
paths plus the actual two-instance-sync scenario from the bug report.
Full pytest 1555 passed (the one pre-existing flake in
test_tidal_auth_instructions.py is order-dependent and unrelated).
- keep existing /api/image-proxy URLs from being wrapped again
- reuse the shared metadata package instead of duplicating URL logic in web_server.py
- add regression coverage for proxy passthrough and internal URL normalization
- move Spotify status publishing onto auth, disconnect, and rate-limit transitions
- keep dashboard and debug consumers on the shared cached snapshot
- leave only the initial snapshot seed as a fallback probe
- move metadata-source and Spotify status caching out of web_server.py
- keep the public /status payload unchanged while shrinking server-side glue
- centralize invalidation and TTL handling in core/metadata/status.py
- cache Spotify auth and rate-limit status separately from the generic metadata source snapshot
- refresh Spotify status only on explicit auth/disconnect/test paths or after the TTL expires
- keep the legacy OAuth callback paths aligned with the same invalidation helper
- Keep the primary metadata provider snapshot generic and move Spotify auth/rate-limit details into a separate status object.
- Update the websocket fixture and dashboard/settings consumers to read the two buckets independently.
Followup to the enrichment-bubble registry consolidation. The
dashboard polling + click handlers all hit
/api/enrichment/<service>/{status,pause,resume} now, so the 30
hand-rolled per-service routes in web_server.py have zero callers
and can come out:
/api/musicbrainz/{status,pause,resume}
/api/audiodb/{status,pause,resume}
/api/discogs/{status,pause,resume}
/api/deezer/{status,pause,resume}
/api/spotify-enrichment/{status,pause,resume}
/api/itunes-enrichment/{status,pause,resume}
/api/lastfm-enrichment/{status,pause,resume}
/api/genius-enrichment/{status,pause,resume}
/api/tidal-enrichment/{status,pause,resume}
/api/qobuz-enrichment/{status,pause,resume}
Worker init blocks stay (they still construct the workers + persist
pause state). Section comment headers are preserved with a one-line
note pointing readers at the new generic blueprint.
Test fixtures in tests/conftest.py and
tests/metadata/test_enrichment_events.py also updated to use the
new URL paths so they reflect production reality. They were
synthetic stubs that never depended on the production routes —
purely cosmetic alignment.
Net: ~510 lines deleted from web_server.py. Full pytest 1541
passed; ruff clean.
The dashboard's enrichment-status bubbles (MusicBrainz, AudioDB,
Discogs, Deezer, Spotify, iTunes, Last.fm, Genius, Tidal, Qobuz) each
had its own copy-pasted /status, /pause, /resume route in web_server.py
— 30 routes that differed only in the worker reference and a couple
of per-service quirks (Spotify's rate-limit guard, Last.fm/Genius
yield-override behavior, Tidal/Qobuz extra status fields).
Replace them with a registry-driven blueprint:
- core/enrichment/services.py declares an EnrichmentService dataclass
with worker_getter, config_paused_key, pre_resume_check,
auto_pause_token, and extra_status_defaults — all variation captured
as data, no branching on service id.
- core/enrichment/api.py exposes a Flask blueprint with three routes
(/api/enrichment/<service>/{status,pause,resume}). Per-service
quirks are honored via the descriptor: Spotify's rate-limit ban
still returns 429 with `rate_limited: true`, Last.fm/Genius still
drop the auto-pause token and add the yield override, Tidal/Qobuz
still merge `authenticated: false` into the fallback payload.
- web_server.py registers all 10 services after their workers
initialize, wires the host-side hooks (config_manager.set,
_download_auto_paused.discard, _download_yield_override.add), and
registers the blueprint.
- webui/static/enrichment.js polling + click handlers now hit the
generic endpoints. The per-service `update<Service>StatusFromData`
functions are unchanged — they still process the same payload.
This is the cutover step. Old per-service routes are intentionally
left in place as a fallback during the soak period — they currently
have zero callers in the codebase and will be deleted in a follow-up
patch once production has run on the new pipeline for a few days.
27 new tests in tests/test_enrichment_services.py cover the registry
behavior + every quirk path through the generic blueprint (rate-limit
guard, auto-pause token cleanup, persisted-pause config keys, extra
default fields, worker-not-initialized fallback, exceptions). Full
suite 1541 passed; ruff clean.
Patch release wrapping up the 2.4.1 dev cycle. Highlights:
- Watchlist no longer re-downloads compilation/soundtrack tracks
(#458 dedup orphan cleanup + the album-match fix work in tandem
to stop the loop).
- Duplicate detector catches slskd dedup orphans via a second
filename-bucket pass.
- Beatport tab hidden temporarily — Cloudflare Turnstile blocks the
scraper and the official OAuth API is closed to public devs.
- Service worker for cover art + installable PWA manifest.
- Browser caching for static assets (1y) and discover pages (5min).
- Socket.IO same-origin default + admin-only /api/settings.
Files updated:
- web_server.py: _SOULSYNC_BASE_VERSION 2.4.0 -> 2.4.1
- webui/index.html: sidebar version button + modal subtitle
- webui/static/helper.js: WHATS_NEW dev-cycle marker -> release date,
fallback version in _getLatestWhatsNewVersion, 8 new
VERSION_MODAL_SECTIONS entries promoted from this cycle
- .github/workflows/docker-publish.yml: workflow_dispatch default
version_tag updated to 2.4.1
- Return a distinct post-auth warning page when Spotify OAuth completes but the client still does not report an authenticated session.
- Send the completion signal back to the opener so the settings UI can refresh and show the warning state immediately.
- Keep the standalone callback server and the main Flask callback path aligned on the same result-page helper.
- Make the Spotify auth completion popup notify the opener across callback origins.
- Refresh service status in the settings UI after auth completes so the button flips to Disconnect immediately.
- Keep the standalone callback instruction page and the main app flow working with the same completion signal.
- Send Spotify auth completion back to the opener so the settings page refreshes immediately
- Make the local auth flow go straight through to Spotify instead of showing the temporary instruction page
- Keep the remote/docker instruction page available for manual callback setups
- Sync Spotify status, connect/disconnect buttons, and metadata source selection after auth and disconnect
- Keep the disconnect behavior aligned with the active primary metadata source
- Hide the auth button when a Spotify session is active
- Treat disconnect as a session change, not a provider swap
- Share metadata source labels in the registry
- Tighten rate-limit copy around Spotify-specific behavior
Pytest tears down its log file handles before atexit runs. Every
"Shutting down ..." line a worker emits while stopping then crashes
Python's logger with "I/O operation on closed file" and floods CI
stderr with --- Logging error --- traceback blocks. The CI sanity
check workflow noticed once tests started importing web_server (the
Tidal-auth integration test PR + this parallel-imports PR are the
first two test files that boot the full module).
Adds a tiny atexit handler that flips ``logging.raiseExceptions =
False`` BEFORE the other shutdown handlers run. atexit's LIFO order
makes "registered last" mean "runs first", so this fires ahead of
cleanup_monitor / _atexit_shutdown / _atexit_save_history and any
log calls those make can't bubble the closed-stream traceback.
The shutdown messages themselves are best-effort debug
breadcrumbs, not data we need to preserve at process exit, so
silencing the internal handler errors costs nothing.
Discord-reported (fresh.dumbledore + maintainer ack): the
/api/import/singles/process route iterated staging files through a
plain Python for loop. Per-file work is dominated by metadata
search round-trips (Spotify/iTunes/Deezer/Discogs), so a multi-
track manual import on a typical home network was painfully slow.
Adds a dedicated import_singles_executor (3 workers) alongside the
existing executor pool, and refactors the route to submit every
file at once and aggregate results via as_completed. Worker count
balances throughput against any single provider's per-source rate
limits — the same shape used by missing_download_executor.
Extracts the per-file pipeline into _process_single_import_file
which returns a typed (status, payload) outcome:
- ("ok", final_title) on success
- ("error", message) for missing/malformed input or pipeline failure
The worker wraps its own exceptions so a single bad file can't
crash the batch; the route adds a belt-and-suspenders try/except
around future.result() for any worker-level surprises.
Pipeline thread-safety verified: post_process_matched_download
already serializes per-file via post_process_locks (one lock per
context_key — and each import gets a unique UUID context_key), DB
writes serialize through SQLite's WAL + busy_timeout, metadata
registry uses RLocks, no bare module-level mutable state.
Adds 9 regression tests:
- 4 worker-contract tests (missing file, malformed match, pipeline
exception wrapping, happy-path return shape)
- 2 executor-config tests (worker count, thread name prefix)
- 1 integration test that proves the route actually parallelizes
by checking wall-clock duration is well under sequential cost
- 1 mixed-outcome aggregation test
- 1 worker-crash recovery test
Doesn't address the related "stops on tab close" complaint —
that's a separate request-lifecycle issue that needs job_id +
polling, not just parallelism.
Discord-reported: clicking the Tidal "Authenticate" button on a
Docker setup landed users on a remote-access instructions page that
told them their callback URL would look like
http://127.0.0.1:8888/tidal/callback?code=... — Spotify's port,
hardcoded into the Tidal instructions. Users who followed those
instructions literally saved 8888 into their tidal.redirect_uri
setting; that mismatched their Tidal Developer App's registered
:8889 redirect URI and Tidal returned error 1002 (invalid redirect
URI) on every auth attempt.
Pull the port from the actual TidalClient.redirect_uri the OAuth
URL was just built with (urlparse), with the SOULSYNC_TIDAL_CALLBACK_PORT
env var as fallback when the URI can't be parsed. Both the Step 2
example and the Step 3 highlighted URL now reflect whatever Tidal
port the user is actually configured to use.
Adds 3 regression tests covering the reported scenario, custom
callback ports via SOULSYNC_TIDAL_CALLBACK_PORT, and a defensive
fallback when redirect_uri is unparseable. Tests hit the real
/auth/tidal route through Flask's test client and assert the
rendered HTML, so future hardcoded ports get caught immediately.
The /api/library/watchlist-all-unwatched endpoint required the
user's currently active metadata source's ID column on each library
artist. A Spotify-primary user with library artists only matched
against iTunes or Deezer saw them silently skipped — surfacing on
Discord as "Library and Watchlist not syncing correctly". The per-
artist Enhanced View sync sometimes "fixed" them because it triggered
metadata enrichment that occasionally populated the missing Spotify
ID, but couldn't help artists Spotify simply doesn't carry.
Extracts the picker as a standalone helper so it can be tested
directly:
core/watchlist/source_picker.py:pick_artist_id_for_watchlist
Picks the active source first when available, then falls back through
spotify -> itunes -> deezer -> discogs in registration order. Empty
strings count as missing. Numeric IDs are coerced to str so SQLite's
TEXT columns store them in the same form library code reads back.
Returns (None, None) only when the artist has zero source IDs — the
only legitimate skip reason now.
Adds 10 regression tests covering active-source priority for each
supported primary, fallback ordering through every secondary, the
zero-IDs base case, unrecognized active source (e.g. hydrabase still
falls through), empty-string handling, and numeric coercion.
The "Clean Search History" automation card kept showing a stale
'DownloadOrchestrator' object has no attribute 'base_url' error
even after the underlying handler bug was fixed in 77d20e9. Root
cause is in the engine, not that handler: AutomationEngine only
captured uncaught exceptions into last_error. Handlers that
report failure by RETURNING {'status': 'error', ...} were treated
as successful from the engine's perspective, so subsequent
gracefully-failing runs never updated the row to reflect the
current state.
Both the timer (run_automation) and event (_handle_event_trigger)
paths now extract the error string from a result whose status is
'error', falling through 'error' -> 'reason' -> 'message' -> a
placeholder so last_error is never None on actual failures
regardless of which key the handler chose. Existing behaviour for
raised exceptions and successful runs is preserved.
Also normalizes _auto_clean_search_history's return key from
'reason' to 'error' so older deployed engines that only check
the canonical key still see the failure.
Adds 7 regression tests covering every result shape the engine
might receive.
The bulk download_discography endpoint picked one metadata client
based on the configured primary source and called .get_album() on
every album with that single client. Albums whose IDs came from a
fallback/provider-specific source (e.g. Deezer-formatted IDs surfaced
through Hydrabase) failed with "Album not found" because the primary
client couldn't resolve them.
Bulk now uses the same source-aware resolver
(core.metadata.album_tracks.get_artist_album_tracks) the working
individual-album endpoint already uses, so the resolver's source-chain
walk finds each album under whichever provider actually has it. Also
adds explicit Discogs and Hydrabase support (the old if/elif chain
silently 500'd for those primaries).
Frontend (library.js + pages-extra.js) now sends a richer
`{ albums: [{id, name, artist_name, source}] }` payload so each album
can be resolved through its own source. The legacy `album_ids` payload
still works as a fallback path.
Closes#399.
Body byte-identical to the original. Spotify proxy via registry,
iTunes/Deezer client shims wrap registry helpers,
_resolve_library_file_path, _attempt_download_with_candidates, and
missing_download_executor are injected via init() right after
_init_wishlist_failed where all three deps are already defined.
web_server.py: 35239 → 35063 (-176 lines).
- search metadata providers in source-priority order for each generated query instead of caching one client for the whole scan
- keep the quality-scanner worker provider-neutral and preserve the no-provider error path
- update the quality-scanner tests and remove the obsolete web_server spotify_client injection
Body byte-identical to the original. Wishlist helpers come from
core.wishlist.* directly (aliased to the same names the body uses);
runtime state from core.runtime_state. automation_engine,
soulseek_client, and _sweep_empty_download_directories are injected
via init() right after _init_download_validation.
web_server.py: 35408 → 35239 (-169 lines).
Body byte-identical to the original. matching_engine and
soulseek_client are injected via init() right after _init_discover_hero
since both originals are constructed early in web_server.py boot
(L598/L610) and never rebound.
web_server.py: 35586 → 35408 (-178 lines).
Body byte-identical to the original. Spotify proxy via registry,
_get_active_discovery_source and get_current_profile_id redefined
as stateless shims, _get_metadata_fallback_client injected via init()
because it composes multiple registry helpers wired in web_server.py.
web_server.py: 35753 → 35586 (-167 lines).
Both function bodies (_discovery_score_candidates and
_search_spotify_for_tidal_track) are byte-identical to the originals.
The shared matching_engine instance is injected via init() right after
_init_connection_test; the spotify proxy + _get_metadata_fallback_source
shim follow the same pattern used elsewhere.
web_server.py: 36019 → 35753 (-266 lines).
Both function bodies byte-identical to the originals. The spotify
proxy resolves through core.metadata.registry; the tidal proxy is
backed by an injected getter so a Tidal re-auth that rebinds
web_server.tidal_client is visible. 13 state dicts and helpers are
injected via init() after _init_connection_test, when all deps
already exist.
web_server.py: 36260 → 36019 (-241 lines).
Body byte-identical to the original. Pure stdlib + requests, no
web_server-specific globals or runtime state — no init() needed.
web_server.py: 36500 → 36261 (-239 lines).
- Switch the download lifecycle over to the neutral wishlist track helper name
- Keep the old Spotify helper as a compatibility alias for older callers
- Store track_data as the primary failed-download wishlist payload key and add regression coverage
Body byte-identical to the original. Five deps (soulseek_client,
qobuz_enrichment_worker, hydrabase_client, docker_resolve_url,
docker_resolve_path) are injected via init() right after the
register_runtime_clients block — that is the earliest point at which
hydrabase_client is guaranteed to exist.
web_server.py: 36833 → 36500 (-333 lines).
Body byte-identical to the original. The shared state dict, lock,
docker_resolve_path helper, and automation engine are injected via
init() at the lift point, where all four originals are already defined.
web_server.py: 37015 → 36833 (-182 lines).
Lifts _search_service and its _detect_provider helper. Both bodies are
byte-identical to the originals. The nine enrichment worker handles
(spotify/itunes/mb/lastfm/genius/tidal/qobuz/discogs/audiodb) are
injected via init() right after qobuz is constructed, which is the
last worker to come up — and well before Flask starts accepting
requests, so the route handlers never see unbound workers.
web_server.py: 37245 → 37015 (-230 lines).
Lifts _match_liked_artists_to_all_sources and
_backfill_liked_artist_images. Both bodies are byte-identical to the
originals. Uses the same _SpotifyClientProxy + _get_*_client shim
pattern as core/artists/map.py so the bodies resolve their original
names without modification.
web_server.py: 37501 → 37245 (-256 lines).
Class body byte-identical to original. Module-level IS_SHUTTING_DOWN
flag is mirrored from web_server's own flag in _shutdown_runtime_components
so the monitor loop still sees shutdown signals at the right moment.
Eight web_server-side helpers (_make_context_key, _on_download_completed,
_run_post_processing_worker, _download_track_worker,
_start_next_batch_of_downloads, _orphaned_download_keys,
missing_download_executor, soulseek_client) are injected via init() after
register_runtime_clients, when all symbols are defined and well before
Flask starts accepting requests.
web_server.py: 38220 → 37501 (-719 lines).
Lifts get_artist_map_data, get_artist_map_genre_list,
get_artist_map_genres, and get_artist_map_explore (plus the
_artmap_cache_* helpers and _artist_map_cache dict) to a new module.
Bodies are byte-identical to the originals. web_server.py keeps
thin route shells that delegate to the lifted functions.
A _SpotifyClientProxy resolves the global spotify_client lazily via
core.metadata.registry.get_spotify_client() so a Spotify re-auth that
rebinds the cached client stays visible to the lifted bodies.
web_server.py: 39124 → 38220 (-904 lines).
Class body byte-identical to original. The shared metadata_update_state
dict is bound at import time via init() so the class body can mutate
it without web_server.py rebinding.
web_server.py: 39754 → 39122 (-632 lines).
The Spotify enrichment worker was auto-starting unconditionally at boot,
hammering /v1/search to match every track in the library against the
Spotify catalog regardless of which metadata source the user had
actually chosen as their primary. Users on Deezer, iTunes, Discogs,
or Hydrabase saw multi-hour 429 bans (typically 14400s) on Spotify
even though they never wanted Spotify-driven enrichment in the first
place — the worker generated dead API traffic the user neither asked
for nor benefited from.
Compounded by Spotify's February 2026 API tightening:
- /v1/search max limit cut from 50 to 10 per request, default from
20 to 5 — every track now needs more pagination, more requests.
- Sustained-rate detection more aggressive — repeated calls over
hours trigger automated long-form bans even when each individual
30-second window is well under the rolling limit.
Result: a user on Deezer would see their Spotify connection get banned
for 4 hours after about 30 tracks of enrichment activity, with no
recourse other than manually pausing the worker each session.
Two-part fix:
1. Boot gate (web_server.py): only auto-start the worker when
`get_primary_source() == 'spotify'`. Otherwise initialize in the
paused state with an explanatory log line. The settings UI manual
unpause control remains functional for users who explicitly want
background Spotify enrichment regardless of primary source.
Boot logic:
- User manually paused (existing config) → stays paused (preserved).
- Primary = 'spotify' → starts running (preserved).
- Primary != 'spotify' → starts paused with log line.
2. Daily budget reduction (core/spotify_worker.py): drop from 3000 to
500 items per calendar day. The 3000 cap was set when /v1/search
returned 50 results per call; now that it caps at 10, each track
needs roughly 5x the API load to find a confident match. 500/day
keeps the worker productive without crossing Spotify's hidden
sustained-rate detection threshold.
The runtime side of the boot gate — auto-pausing when the user
switches primary source mid-session — is out of scope. The settings
UI already exposes the manual toggle, and primary-source switches are
infrequent enough that requiring a manual unpause after the fact is
acceptable.
Full suite: 1355 passing. Ruff clean.
Final lift in the web_server.py extraction effort. Pulls two route
handlers + one background worker out of `web_server.py` into new
focused packages:
- `core/streaming/prepare.py` — 258-line stream-prep worker that
downloads a track to the local Stream/ folder for the browser audio
player.
- `core/playlists/explorer.py` — 305-line route handler for
`POST /api/playlist-explorer/build-tree` that streams an NDJSON
discography tree from a mirrored playlist.
What `prepare_stream_task` does:
1. Reset stream state to 'loading' with the new track info.
2. Clear any prior file from Stream/ (only one stream lives there).
3. Spin up a fresh asyncio event loop and `soulseek_client.download()`.
4. Poll progress every 1.5s. Queue timeout 15s; overall 60s.
5. On succeeded + bytes-match: find the file with retry, move into
Stream/, signal slskd completion, mark state 'ready' with file_path.
6. On error/timeout/cancel: state goes to 'error' or 'stopped'.
7. Finally: tear down the event loop cleanly.
What `playlist_explorer_build_tree` does:
1. Validate request, load playlist + tracks from DB.
2. Pick active metadata source (Spotify if authed, else fallback).
3. Group tracks by artist using discovered matched_data when the
provider matches the active source.
4. Stream NDJSON: meta line → one artist line per group → complete line.
5. Per artist: cache check → resolve discography → tag releases with
`in_playlist` flag based on title-similarity match → filter by mode
(`albums` = only matches; `discographies` = full disco).
6. Mark playlist as explored on completion.
Strict 1:1 byte parity:
Both functions exposed their dependencies through proxy patterns
established in earlier lifts (PR4–PR8). For prepare_stream_task,
`stream_state` is a deps property; for the explorer, Flask `request` /
`jsonify` / `Response` are injected via deps so the lifted body keeps
its native syntax. Both lifts verified ZERO diff against the original
after `deps.X` → global X normalization.
258 lines orig = 258 lines lifted (prepare_stream_task).
305 lines orig = 305 lines lifted (explorer).
Bonus cleanup: web_server.py's module-level `import shutil` and
`import glob` were now unused (only `_prepare_stream_task` used them
at module scope; every other reference is via inline `import shutil`
in respective function bodies). Removed both module-level imports —
ruff caught the F811 redefinitions and confirmed they're truly
redundant.
Dependencies for `PrepareStreamDeps` (11 fields):
config_manager, soulseek_client, stream_lock, project_root,
docker_resolve_path, find_streaming_download_in_all_downloads,
find_downloaded_file, extract_filename, cleanup_empty_directories,
plus 2 stream_state property delegates.
Dependencies for `PlaylistExplorerDeps` (9 fields):
Flask request/Response/jsonify, spotify_client, get_database,
get_active_discovery_source, get_metadata_fallback_client,
get_metadata_fallback_source, get_metadata_cache.
Tests: 6 new under tests/streaming/test_prepare.py (state init,
Stream/ folder creation + clearing, download-init failure, completed
+ moved + ready state, partial-bytes incomplete-warning path) plus 9
new under tests/playlists/test_explorer.py (5 validation early-exit
paths, streaming response shape with meta/complete lines, mark-
explored side effect, discovered-artist grouping using matched_data,
provider mismatch falling back to raw artist name).
Full suite: 1355 passing (was 1340). Ruff clean.
End of the web_server.py extraction effort. Started at ~45,000 lines
across PR4–PR8 + this commit; finished around 35,000 lines with the
heavy worker + route logic now living in domain-cohesive packages
under core/. The remaining bulk in web_server.py is route handlers,
service initialization, and the deferred 1530-line
`_register_automation_handlers` (startup-only, marginal lift value).
Pulls the 284-line artist quality enhancement helper out of
`web_server.py` into a new `core/artists/` package. Flask route handler
split: route + request parsing stay in web_server.py, the body lifts to
a pure function returning `(payload_dict, http_status_code)`.
What `enhance_artist_quality` does:
1. Validate request: track_ids must be non-empty, artist must exist.
2. Build a `track_lookup` from `database.get_artist_full_detail` so each
selected track resolves with its album context.
3. Per track:
- Read current quality tier from the file extension.
- Build `matched_track_data` for the wishlist entry, in priority
order:
- Spotify direct lookup via stored `spotify_track_id` (preferred).
Uses raw API data when available; otherwise rebuilds the payload
and pulls album images via a follow-up `get_album` call.
- Spotify search fallback using matching_engine queries with
artist+title similarity scoring (album-type bonus for albums,
smaller bonus for EPs). Stops at first >= 0.9 confidence match.
- iTunes/fallback source search with the same scoring shape.
- Add to wishlist via `wishlist_service.add_spotify_track_to_wishlist`
with `source_type='enhance'` and a `source_context` carrying the
original file path, format tier, bitrate, original_tier, and
artist_name.
- Tally `enhanced_count` / `failed_count` / per-track failure reasons.
4. Return `{success, enhanced_count, failed_count, failed_tracks}` 200.
Dependencies injected via `ArtistQualityDeps` (7 fields) — spotify_client,
matching_engine, get_database, get_wishlist_service,
get_current_profile_id, get_quality_tier_from_extension,
get_metadata_fallback_client.
Diff vs original after `deps.X` → global X normalization is **1 line of
cosmetic drift** — the success return now uses an explicit `(payload, 200)`
tuple to keep all returns shape-consistent for the wrapper. Flask treats
`jsonify(x)` and `(jsonify(x), 200)` identically. 284 lines orig = 285
lines lifted, body otherwise byte-identical.
Tests: 10 new under tests/artists/test_quality.py covering input
validation (empty track_ids, artist not found), Spotify direct lookup
via raw_data, Spotify direct lookup with enhanced format requiring
album image rebuild, Spotify search fallback, iTunes/fallback source
match path, track-not-found and no-file-path failure modes, complete
no-match failure, and source_context payload assertions (enhance flag,
file path, format tier, bitrate, source_type).
Full suite: 1340 passing (was 1330). Ruff clean.
Pulls the 258-line retag worker out of `web_server.py` into a new
`core/library/` package. Pure 1:1 lift — wrapper keeps the original
entry-point name so the retag-trigger endpoint continues to work
without changes.
What `execute_retag` does:
1. Fetch album + track metadata for the new `album_id` (Spotify or
iTunes — the Spotify client transparently falls back).
2. Load existing files in the retag group from the DB.
3. Match each existing track to a new Spotify track:
- Priority 1: same disc + track number.
- Priority 2: title similarity >= 0.6 (SequenceMatcher).
4. For each matched pair:
- Re-write metadata tags via `_enhance_file_metadata`.
- Compute the new path via `_build_final_path_for_track` and move
the audio file (plus .lrc / .txt sidecars) if the path changes.
- Drop an orphaned cover.jpg if it's left in an empty directory.
- Clean up empty parent directories left behind.
- Download the new cover art into the new album dir.
5. Update the retag group record with new artist / album / image /
total_tracks / release_date and the appropriate Spotify-or-iTunes
album ID (numeric → iTunes, alphanumeric → Spotify).
6. Mark the retag state 'finished' (or 'error' on exception).
Strict 1:1 byte parity:
The original mutated `retag_state` as a module global (the function
declared `global retag_state` even though it only mutates in place).
Here `retag_state` is exposed through the `RetagDeps` proxy as a Python
property so the lifted body keeps `name[key] = value` /
`name.update(...)` syntax. The property setter rebinds the
web_server.py reference if the function ever reassigns it (currently
it doesn't, but the setter is wired for parity with the watchlist lift).
Diff vs original after `deps.X` → global X normalization is **zero
differences** apart from the dropped `global retag_state` decl and the
inline `from database.music_database import get_database` (replaced by
deps.get_database()). 258 lines orig = 258 lines lifted, byte-identical
body otherwise.
Dependencies injected via `RetagDeps` (13 fields) — config_manager,
retag_lock, spotify_client, plus 8 callable helpers
(get_audio_quality_string, enhance_file_metadata,
build_final_path_for_track, safe_move_file, cleanup_empty_directories,
download_cover_art, docker_resolve_path, get_database) and 2 property
delegates (_get_retag_state / _set_retag_state).
Tests: 11 new under tests/library/test_retag.py covering setup error
paths (no album data, no album tracks, no existing tracks),
track-number priority match, title-similarity fallback, no-match skip,
missing file skip, file move when path changes, group record update
(spotify vs iTunes ID branching by alphanumeric vs numeric album_id),
multi-disc total_discs computation.
Full suite: 1330 passing (was 1319). Ruff clean.
Pulls the 390-line watchlist auto-scan orchestrator out of `web_server.py`
into a new `core/watchlist/` package. Watchlist (followed-artists scanner
that finds new releases) is a separate domain from kettui's wishlist
(failed-download retry queue), so this lift does not overlap with the
ongoing PR400-style extractions.
What `process_watchlist_scan_automatically` does:
1. Smart stuck-detection guard before acquiring the timer lock —
prevents deadlock when a previous scan flag is dangling past the
2-hour timeout.
2. Inside the timer lock: re-check + set the active scan flag with the
current timestamp.
3. Per-profile expansion (or single-profile when manually triggered):
- Watchlist count check + Spotify auth gate.
- Backfill missing artist images.
4. Initialize a fresh `watchlist_scan_state` dict (the deps property
setter rebinds the web_server.py module-level name so external
sentinel checks via id() comparison still detect the swap).
5. Pause enrichment workers, then call
`WatchlistScanner.scan_watchlist_artists` with a per-event progress
callback that translates scanner events into automation log lines.
6. Post-scan steps (skipped if the scan was cancelled mid-flight):
- Populate discovery pool from similar artists (per-profile).
- Refresh ListenBrainz playlists.
- Update current seasonal playlist (weekly cadence).
- Generate Last.fm radio playlists.
- Sync Spotify library cache.
- Activity feed entry + automation_engine.emit('watchlist_scan_completed').
7. On exception: mark state['status']='error', re-raise so the
automation wrapper records the failure.
8. Finally: resume enrichment workers, clear the scanner's rescan
cutoff, reset the auto-scanning flag.
Strict 1:1 byte parity:
The original mutated `watchlist_auto_scanning`,
`watchlist_auto_scanning_timestamp`, and `watchlist_scan_state` as
module globals (with a leading `global` decl). Here those names are
exposed through the `WatchlistAutoScanDeps` proxy as Python properties
so the lifted body keeps the same `name = value` / `name[key] = value`
shape. Property setters fan writes back to web_server.py via callback
pairs.
Diff vs original after `deps.X` → global X normalization is **zero
differences** apart from the dropped `global` declaration line — Python
doesn't need it once the names are property accesses on the deps object.
390 lines orig = 390 lines lifted, byte-identical body otherwise.
Dependencies injected via `WatchlistAutoScanDeps` (15 fields total) —
Flask app, spotify_client, automation_engine, watchlist_timer_lock, plus
5 callable helpers and 6 property delegate callbacks (paired
get/set for each of the three globals).
Tests: 11 new under tests/watchlist/test_auto_scan.py covering
stuck-detection guard, race-check inside lock, zero-watchlist short-
circuit, unauthenticated Spotify gate, successful scan with all post-
scan steps, automation event emission, activity feed logging,
cancellation mid-scan skipping post-steps, profile-scoped trigger,
flag reset in finally, rescan cutoff clear in finally.
Full suite: 1319 passing (was 1308). Ruff clean.
- let core.metadata.registry own per-profile Spotify client caching
- register the DB-backed profile credentials provider from web_server.py
- invalidate only the affected profile cache entry on save, delete, and auth
- make web_server.py read and refresh Spotify from core.metadata.registry
- add single-key metadata cache eviction for Spotify reauth
- export the new cache helper through the metadata package shims
Pulls the 201-line staging-folder shortcut out of `web_server.py` into
its own module under the existing `core/downloads/` package. Pure 1:1
lift — wrapper keeps the original entry-point name so the task worker's
existing call site continues to work without changes.
What `try_staging_match` does:
1. Pull the per-batch staging-file cache (one filesystem scan per batch).
2. For each staging entry, compute title + artist similarity using
SequenceMatcher and the matching engine's `normalize_string`. Require
title >= 0.80, then a combined score >= 0.75. The weighting flips
based on whether artist info is available on both sides:
- both have artist: 0.55*title + 0.45*artist
- either side missing artist: 0.80*title + 0.20*artist (lean on title)
3. Copy the matched file to the configured transfer dir (with a
"_staging" suffix when the destination filename already exists, to
avoid overwriting a legitimate prior download).
4. Mark the task as 'post_processing', username='staging',
staging_match=True.
5. Build a synthetic spotify_artist / spotify_album context (mirroring
the modal-worker logic so the file-organization template applies
cleanly) and store it under "staging_<task_id>". Two paths:
- Explicit context branch (track_info has _is_explicit_album_download)
→ real album/artist data copied through.
- Fallback branch → synthesized from track + track_info, with
`is_album_download` heuristically derived (album differs from title
and isn't "Unknown Album").
6. Hand off to `_post_process_matched_download_with_verification` which
does tagging, path building, AcoustID verification, and DB insertion.
Returns True if the staging shortcut won; False to fall through to the
normal Soulseek search path.
Dependencies injected via `StagingDeps` (5 fields) — config_manager,
matching_engine, get_staging_file_cache, docker_resolve_path,
post_process_matched_download_with_verification.
Diff vs original after `deps.X` → global X normalization is **zero
differences** — 201 lines orig = 201 lines lifted, byte-identical body
(including all whitespace, comments, log strings, and the inline
`from difflib import SequenceMatcher` / `import shutil` imports inside
the function body).
Tests: 9 new under tests/downloads/test_downloads_staging.py covering
no staging files / no track title / low-confidence match returning
False, exact match copying file + transitioning task state + invoking
post-processing, existing-file rename via `_staging` suffix, explicit
album context branch, fallback context synthesis (with both album-as-
album and album-equals-title cases), and copy failure (missing source
file) returning False.
Full suite: 1308 passing (was 1299). Ruff clean.
Missed worker from the PR5 discovery-workers series — Tidal sits in the
same domain as the deezer / spotify_public / listenbrainz / youtube /
beatport workers that were lifted in PR5b–PR5h, follows the same shape,
shares the same `_search_spotify_for_tidal_track` helper, and was simply
overlooked in the original inventory.
Pure 1:1 lift of the 212-line worker. Wrapper keeps the original
entry-point name so the existing call sites in web_server.py continue
to work without changes.
What `run_tidal_discovery_worker` does:
1. Pause enrichment workers (release shared resources).
2. For each Tidal track:
- Cancellation gate (state['cancelled']).
- Discovery cache lookup; cache hit short-circuits the search.
- SimpleNamespace-style track passed straight to
`_search_spotify_for_tidal_track` (the shared helper used by every
worker in this family).
- On Spotify match: build `match_data` preserving track_number /
disc_number from raw API data, image extracted from album images
or track object fallback, release_date filled from
track.release_date when album dict is missing it.
- On iTunes match: dict result populated as `match_data` with source
set to discovery_source, image extracted from album images.
- Save matched result to discovery cache.
- On miss: Wing It stub stored as 'wing-it' status (success ticked).
3. After all tracks: phase='discovered', activity feed entry, sync
discovery results back to mirrored playlist via
`_sync_discovery_results_to_mirrored` with 'tidal' tag.
4. On error: state['phase']='error' + status with error string.
5. Finally: resume enrichment workers.
Dependencies injected via `TidalDiscoveryDeps` (13 fields) —
tidal_discovery_states, spotify_client, plus 11 callable helpers
(pause/resume enrichment, get_active_discovery_source,
get_metadata_fallback_client, get_discovery_cache_key, get_database,
validate_discovery_cache_artist, search_spotify_for_tidal_track,
build_discovery_wing_it_stub, add_activity_item,
sync_discovery_results_to_mirrored). Same surface as the deezer worker.
Diff vs original after `deps.X` → global X normalization is **zero
differences** — 212 lines orig = 212 lines lifted, byte-identical body
(including all whitespace, comments, log strings).
Tests: 9 new under tests/discovery/test_discovery_tidal.py covering
cache hit short-circuit, Spotify tuple match (track/disc preservation),
iTunes dict match path, Wing It fallback, cancellation, completion
phase update, activity feed entry, mirrored sync invocation, per-track
error handling.
Full suite: 1299 passing (was 1290). Ruff clean.
First lift in the new PR6 batch. Pulls the 312-line candidate-fallback
download dispatcher out of `web_server.py` into a new module under the
existing `core/downloads/` package. Pure 1:1 lift — wrapper keeps the
original entry-point name so all callers (search/match pipeline) work
unchanged.
What `attempt_download_with_candidates` does:
1. Sort candidates by descending confidence.
2. For each candidate:
- Cancellation gates (3 points: top of loop, before download starts,
after download_id is assigned).
- Skip already-tried sources via the per-task `used_sources` set.
- Skip blacklisted sources (user-flagged bad matches).
- Race protection: bail when the task already has an active
download_id.
- `update_task_status('downloading')`, then `soulseek_client.download`.
3. On a successful download_id:
- Build `matched_downloads_context` entry keyed by
`make_context_key(username, filename)`.
- For tracks with clean Spotify metadata, pull track_number /
disc_number from (1) track_info → (2) track object → (3) Spotify
API call. When local album context is incomplete, the API response
backfills release_date / album_type / total_tracks / images / id.
- Set `is_album_download` based on explicit context flag or
heuristic (album differs from title, isn't "Unknown Album").
- Store task/batch IDs and track_info on the context for post-
processing + playlist-folder mode.
4. On a cancellation that wins the race after the download started:
- `cancel_download(...)` to stop the in-flight Soulseek transfer.
- `on_download_completed(batch_id, task_id, success=False)` to free
the worker slot.
5. On exception or download-start failure: reset task status to
'searching', continue to next candidate.
Dependencies injected via `CandidatesDeps` (7 fields) — soulseek_client,
spotify_client, run_async, get_database, update_task_status,
make_context_key, on_download_completed.
Diff vs original after `deps.X` → global X normalization is **zero
differences** — 312 lines orig = 312 lines lifted, byte-identical body
(including all whitespace, comments, log strings).
Tests: 14 new under tests/downloads/test_downloads_candidates.py
covering happy path (first candidate succeeds, confidence ordering),
used_sources dedup, blacklist skip, cancellation gates (cancelled
status, deleted task, active download_id, mid-flight cancel + cleanup
callback), failure paths (all candidates failed, exception during
download falls through to next), context payload (explicit album
context, track_number priority order, API backfill of incomplete album
metadata), and equal-confidence stable order.
Pre-existing behavior documented in tests:
`spotify_album_context['id']` initializes to a non-empty placeholder
'from_sync_modal' in the fallback path, so the API-backfill condition
`if not spotify_album_context.get('id')` never fires for the id field
specifically. Other album fields (release_date, album_type) backfill
fine because they default to empty.
Full suite: 1290 passing (was 1276). Ruff clean.
PR400 added imports for `check_and_remove_from_wishlist` and
`check_and_remove_track_from_wishlist_by_metadata` from
`core.wishlist.resolution` (aliased with leading underscores in
web_server.py) but left the original inline definitions of those
functions in place at L17139 and L17243. Python's later definition
wins, so the local defs were silently shadowing the imports — meaning
the new package versions were never actually called from web_server.py.
Ruff caught the redefinition (F811) and broke CI.
Deleted the inline definitions (176 lines). Imports at L143-144 now
serve all callers, and the package functions in
`core/wishlist/resolution.py` are actually exercised. Behavior is the
same: I diffed both versions before deleting and confirmed they're
functionally equivalent.
Tests: 1232 passing (no change). Ruff clean.
Final lift in the PR5 discovery-workers series. Pulls the 328-line
library quality scanner out of `web_server.py` into its own focused
module under `core/discovery/`. Pure 1:1 lift — wrapper keeps the
original entry-point name.
What the quality scanner does:
1. Reset scanner state (counters, results), load quality profile +
minimum acceptable tier from QUALITY_TIERS.
2. Load tracks from DB based on scope:
- 'watchlist' → tracks for watchlisted artists only.
- other → all library tracks.
3. For each track:
- Stop-request gate (state['status'] != 'running').
- Quality-tier check via _get_quality_tier_from_extension(file_path).
- Skip tracks meeting standards (tier_num <= min_acceptable_tier).
- For low-quality tracks: matching_engine search query gen, score
candidates against Spotify (artist + title similarity, album-type
bonus), pick best match >= 0.7 confidence.
- On match: add full Spotify track to wishlist via
`wishlist_service.add_spotify_track_to_wishlist` with
source_type='quality_scanner' and a source_context that captures
original file_path, format tier, bitrate, and match confidence.
4. After all tracks: status='finished', progress=100, activity feed
entry, emit `quality_scan_completed` event for automation engine.
5. On critical exception: status='error', error message captured.
Wishlist service interaction is via the public
`add_spotify_track_to_wishlist` API only — no overlap with kettui's
planned `core/wishlist/` package extraction (the import lives inside
the function, exactly as in the original, and will follow whatever
path that package takes).
Dependencies injected via `QualityScannerDeps` (8 fields) —
quality_scanner_state dict, quality_scanner_lock, QUALITY_TIERS
constant, spotify_client, matching_engine, automation_engine, plus 2
callable helpers (get_quality_tier_from_extension, add_activity_item).
Diff vs original after `deps.X` → global X normalization is **zero
differences** — 328 lines orig = 328 lines lifted, byte-identical body
(including all whitespace, comments, log strings, and the inline
`from core.wishlist_service import get_wishlist_service` /
`from database.music_database import MusicDatabase` imports at the
top of the function).
Tests: 11 new under tests/discovery/test_discovery_quality_scanner.py
covering state init/reset, no-watchlist-artists short-circuit,
unauthenticated Spotify error, high-quality skip, low-quality search
trigger, match → wishlist add (with full source_context payload),
no-match no-add, mid-loop stop request, completion phase + progress,
automation engine event emission, all-library scope load.
Full suite: 1152 passing (was 1141). Ruff clean.
End of the PR5 series — `web_server.py` lost ~328 lines on this commit
alone; total trim across PR5a–PR5h is ~2,400 lines of discovery worker
code moved into focused `core/discovery/*.py` modules. The remaining
discovery-adjacent worker `_process_watchlist_scan_automatically` was
deliberately deferred to avoid overlap with kettui's planned wishlist
extraction.
Seventh lift in the PR5 discovery-workers series. Pulls the 286-line
ListenBrainz discovery worker out of `web_server.py` into its own
focused module under `core/discovery/`. Pure 1:1 lift — wrapper keeps
the original entry-point name.
What the ListenBrainz discovery worker does:
1. Pause enrichment workers (release shared resources).
2. For each ListenBrainz track:
- Cancellation gate (state['phase'] != 'discovering').
- Discovery cache lookup; cache hit short-circuits the search.
- Strategy 1: matching_engine search queries with confidence scoring
against Spotify (preferred) or iTunes (fallback).
- Strategy 2: swapped artist/title query.
- Strategy 3: album-based query (uses album_name when available —
unique to LB, since YouTube tracks don't have album metadata).
- Strategy 4: extended search with limit=50.
- On match → save to discovery cache with image extracted from album
images or matched_track.image_url fallback.
- On miss → Wing It stub stored as 'wing-it' status.
3. After all tracks: phase='discovered', status='complete', activity feed
entry mentioning 'ListenBrainz Discovery Complete'.
4. On error: state['status']='error', phase='fresh'.
5. Finally: resume enrichment workers.
Dependencies injected via `ListenbrainzDiscoveryDeps` (16 fields) —
listenbrainz_playlist_states, spotify_client, matching_engine, plus 13
callable helpers (pause/resume enrichment, get_active_discovery_source,
get_metadata_fallback_client, get_discovery_cache_key, get_database,
validate_discovery_cache_artist, extract_artist_name,
spotify_rate_limited, discovery_score_candidates, get_metadata_cache,
build_discovery_wing_it_stub, add_activity_item).
Diff vs original after `deps.X` → global X normalization is **zero
differences** — 286 lines orig = 286 lines lifted, byte-identical body
(including all whitespace, comments, log strings).
Pre-existing bug preserved (not fixed): if `listenbrainz_playlist_states[
state_key]` raises KeyError on entry, the outer except handler tries to
mutate `state` which is unbound → secondary UnboundLocalError. Same bug
in the original (and the YouTube discovery worker). Documented here for
future cleanup but out of scope for the lift.
Tests: 11 new under tests/discovery/test_discovery_listenbrainz.py
covering cache hit short-circuit, Strategy 1 confidence match, Wing It
fallback, iTunes fallback (Spotify unauthenticated and rate-limited),
cancellation (phase change), completion phase update, activity feed
entry, per-track error handling, float duration_ms tolerance (regression
for the :02d format crash fixed earlier), enrichment workers resume on
finally.
Full suite: 1141 passing (was 1130). Ruff clean.
Sixth lift in the PR5 discovery-workers series. Pulls the 323-line
Beatport chart discovery worker out of `web_server.py` into its own
focused module under `core/discovery/`. Pure 1:1 lift — wrapper keeps
the original entry-point name.
What the Beatport discovery worker does:
1. Pause enrichment workers (release shared resources).
2. For each Beatport track:
- Cancellation gate (state['phase'] != 'discovering').
- Clean Beatport text (artist/title) of common annotations via
`clean_beatport_text` helper.
- Single-string artist normalization for "CID,Taylr Renee"-style
entries — split on comma, take the first.
- Discovery cache lookup; cache hit short-circuits the search and
normalizes cached artists from ['str'] → [{'name': 'str'}] to
match the frontend's expected list-of-objects shape.
- matching_engine search-query generation (with high min_confidence
of 0.9 to avoid bad matches).
- Strategy 1: scored candidates from initial Spotify/iTunes searches.
- Strategy 4: extended search with limit=50 if no high-confidence
match found.
- On Spotify match: format artists as [{'name': str}] objects, pull
full album object from raw cache when available, fallback to
reconstructed album dict otherwise.
- On iTunes match: format with image_url-derived album.images entry
(300x300 spec), source set to discovery_source.
- Save matched result to discovery cache when confidence >= 0.75
(note: lower than search threshold; discovery still benefits from
these less-confident matches as user-visible suggestions).
- On miss: Wing It stub stored as 'wing-it' status (success ticked).
3. After all tracks: phase='discovered', activity feed entry, sync
discovery results back to mirrored playlist via
`_sync_discovery_results_to_mirrored` with 'beatport' tag.
4. On error: state['phase']='fresh' + status='error'.
5. Finally: resume enrichment workers.
Dependencies injected via `BeatportDiscoveryDeps` (17 fields) —
beatport_chart_states, spotify_client, matching_engine, plus 14
callable helpers (pause/resume enrichment, get_active_discovery_source,
get_metadata_fallback_client, clean_beatport_text,
get_discovery_cache_key, get_database, validate_discovery_cache_artist,
spotify_rate_limited, discovery_score_candidates, get_metadata_cache,
build_discovery_wing_it_stub, add_activity_item,
sync_discovery_results_to_mirrored).
Diff vs original after `deps.X` → global X normalization is **zero
differences** — 323 lines orig = 323 lines lifted, byte-identical body
(including all whitespace, comments, log strings).
Tests: 12 new under tests/discovery/test_discovery_beatport.py covering
cache hit short-circuit (with cached-artist normalization), Spotify
match formatting (list and string artist inputs), iTunes match
(image_url to album.images), Wing It fallback, cancellation
(phase change), completion phase update, activity feed entry, mirrored
sync invocation, top-level error handler, per-track error handling,
comma-separated artist split.
Full suite: 1130 passing (was 1118). Ruff clean.
Fifth lift in the PR5 discovery-workers series. Pulls the 278-line
public-Spotify-link discovery worker out of `web_server.py` into its
own focused module under `core/discovery/`. Pure 1:1 lift — wrapper
keeps the original entry-point name.
What the Spotify Public discovery worker does:
1. Pause enrichment workers (release shared resources).
2. For each track:
- Cancellation gate (state['cancelled']).
- Normalize artists to plain string list (handles dict + str inputs).
- Discovery cache lookup; cache hit short-circuits the search and
populates display fields from the cached match.
- SimpleNamespace duck-type → `_search_spotify_for_tidal_track`
(shared search helper, returns tuple for Spotify or dict for iTunes).
- On Spotify match: build `match_data` preserving track_number /
disc_number from raw API data; image extracted from album images
or track object fallback; release_date filled from track.release_date
when album dict is missing it.
- On iTunes match: dict result → match_data with source set to
discovery_source; image extracted from album images.
- Save matched result to discovery cache.
- On miss: Wing It stub stored as 'wing-it' status.
3. After all tracks: phase='discovered', activity feed entry.
4. On error: state['phase']='error' + status with error string.
5. Finally: resume enrichment workers.
This worker is structurally close to the Deezer worker (see PR5d) but
intentionally diverges on:
- Track-data field names (`spotify_public_track` vs `deezer_track`).
- Artist normalization (Spotify Public can pass dicts or strings).
- No mirrored-playlist DB writeback (sync is handled separately).
Dependencies injected via `SpotifyPublicDiscoveryDeps` (12 fields) —
spotify_public_discovery_states, spotify_client, plus 10 callable
helpers (pause/resume enrichment, get_active_discovery_source,
get_metadata_fallback_client, get_discovery_cache_key, get_database,
validate_discovery_cache_artist, search_spotify_for_tidal_track,
build_discovery_wing_it_stub, add_activity_item).
Diff vs original after `deps.X` → global X normalization is **zero
differences** — 278 lines orig = 278 lines lifted, byte-identical body
(including all whitespace, comments, log strings).
Tests: 10 new under tests/discovery/test_discovery_spotify_public.py
covering cache hit short-circuit, dict-artist normalization, Spotify
tuple match (track/disc preservation), iTunes dict match path, Wing It
fallback, cancellation, completion phase update, activity feed entry,
top-level error handler, per-track error handling.
Full suite: 1118 passing (was 1108). Ruff clean.
Fourth lift in the PR5 discovery-workers series. Pulls the 270-line
Deezer discovery worker out of `web_server.py` into its own focused
module under `core/discovery/`. Pure 1:1 lift — wrapper keeps the
original entry-point name so the existing call sites continue to work
without changes.
What the Deezer discovery worker does:
1. Pause enrichment workers (release shared resources).
2. For each Deezer track:
- Cancellation gate (state['cancelled']).
- Discovery cache lookup; cache hit short-circuits the search and
populates display fields from the cached match (artist string,
album name).
- SimpleNamespace duck-type → `_search_spotify_for_tidal_track`
(shared search helper, returns tuple for Spotify or dict for iTunes).
- On Spotify match: build `match_data` preserving track_number /
disc_number from raw API data, image extracted from album images
or track object fallback, release_date filled from track.release_date
when album dict is missing it.
- On iTunes match: dict result populated as `match_data`, source set
to discovery_source, image extracted from album images.
- Save matched result to discovery cache.
- On miss: Wing It stub stored as 'wing-it' status.
3. After all tracks: phase='discovered', activity feed entry, sync
discovery results back to mirrored playlist via
`_sync_discovery_results_to_mirrored`.
4. On error: state['phase']='error' + status with error string.
5. Finally: resume enrichment workers.
Dependencies injected via `DeezerDiscoveryDeps` (13 fields) —
deezer_discovery_states dict, spotify_client, plus 11 callable helpers
(pause/resume enrichment, get_active_discovery_source,
get_metadata_fallback_client, get_discovery_cache_key, get_database,
validate_discovery_cache_artist, search_spotify_for_tidal_track,
build_discovery_wing_it_stub, add_activity_item,
sync_discovery_results_to_mirrored).
Diff vs original after `deps.X` → global X normalization is **zero
differences** — 270 lines orig = 270 lines lifted, byte-identical body
(including all whitespace, comments, log strings).
Tests: 10 new under tests/discovery/test_discovery_deezer.py covering
cache hit short-circuit, Spotify tuple match (track/disc number
preservation), iTunes dict match path, Wing It fallback, cancellation,
completion phase update, activity feed entry, mirrored sync invocation,
top-level error handler, per-track error handling.
Full suite: 1108 passing (was 1098). Ruff clean.
Third lift in the PR5 discovery-workers series. Pulls the 323-line
mirrored-playlist discovery worker out of `web_server.py` into its own
focused module under `core/discovery/`. Pure 1:1 lift — wrapper keeps
the original entry-point name so the existing call site
(`_run_playlist_discovery_worker(pls, automation_id=None)` from the
automation engine) continues to work without changes.
What the playlist discovery worker does:
1. Pause enrichment workers (release shared resources).
2. Pre-compute total track count across all playlists for the automation
progress card.
3. For each playlist:
- Fast pre-scan separates already-discovered tracks (skipped, unless
incomplete metadata or a Wing It stub) from undiscovered ones.
- For each undiscovered track:
- Cancellation gate via _playlist_discovery_cancelled set.
- Discovery cache lookup (with artist validation).
- matching_engine search-query generation, then Spotify (preferred)
or iTunes (fallback) search + scoring.
- Extended search fallback (limit=50) if no high-confidence match.
- On match → enrich album from metadata cache (id, images,
total_tracks, album_type, release_date, artists, plus track_number
and disc_number), build matched_data, write to track.extra_data,
save to discovery cache.
- On miss → Wing It stub stored as 'wing_it_fallback' provider.
4. After all playlists: emit `discovery_completed` event when at least
one new track was discovered, mark automation progress 'finished'.
5. On error → automation progress 'error', traceback printed.
6. Finally: resume enrichment workers.
Dependencies injected via `PlaylistDiscoveryDeps` (16 fields) —
spotify_client, matching_engine, automation_engine, the cancellation
set, plus 12 callable helpers (pause/resume enrichment,
get_active_discovery_source, get_metadata_fallback_client/source,
update_automation_progress, get_database, get_discovery_cache_key,
validate_discovery_cache_artist, discovery_score_candidates,
get_metadata_cache, build_discovery_wing_it_stub).
Diff vs original after `deps.X` → global X normalization is **zero
differences** — 323 lines orig = 323 lines lifted, byte-identical body
(including all whitespace, comments, log strings).
Tests: 15 new under tests/discovery/test_discovery_playlist.py covering
empty playlists, no-tracks playlist skip, complete-discovery skip,
incomplete-discovery re-run, Wing It always re-run, unmatched_by_user
respect, cache hit short-circuit, match above threshold (extra_data +
cache save), match below threshold falls to Wing It, iTunes fallback,
neither-provider error path, cancellation, discovery_completed event
emit, no-event on zero-discovered, multi-playlist grand_total
aggregation.
Full suite: 1098 passing (was 1083). Ruff clean.
yt_dlp sometimes returns float `duration_ms` for YouTube tracks. The
discovery workers format the duration with `f"{x // 60000}:{(x % 60000)
// 1000:02d}"` — and `:02d` requires an int. When the duration is a
float, the format string raises:
Unknown format code 'd' for object of type 'float'
Caught when running YouTube discovery on a real playlist (bbno$ tracks)
— every track failed with status='Error'.
Pre-existing bug, surfaced now because of yt_dlp returning float
durations on this playlist. Fixed at all 8 sites by casting through
`int()` before the `// 60000` and `% 60000` operations:
- core/discovery/youtube.py: 2 sites in run_youtube_discovery_worker
(cache hit + main result construction).
- web_server.py L29238/L29372: 2 sites in _run_listenbrainz_discovery_worker.
- web_server.py L40112/L40136/L40161/L40178: 4 sites in the YouTube
retry/pre-discovered results assembly path.
The `if duration_ms` / `if dur` guard already protects against None and 0,
so `int(...)` is only called on truthy numeric values.
Tests: 1 new regression test under tests/discovery/test_discovery_youtube.py
(`test_float_duration_does_not_crash_format`) — passes a float
duration_ms and asserts the worker completes without an error result.
Ruff clean.
Second lift in the PR5 discovery-workers series. Pulls the 332-line
YouTube discovery worker out of `web_server.py` into its own focused
module under `core/discovery/`. Pure 1:1 lift — wrappers keep the
original entry-point name so the two callers
(`youtube_discovery_executor.submit(_run_youtube_discovery_worker, ...)`)
continue to work without changes.
What the YouTube discovery worker does:
1. Pause enrichment workers (release shared resources).
2. For each YouTube playlist track:
- Cancellation check (phase != 'discovering' aborts).
- Discovery cache lookup; cache hit short-circuits the search.
- Strategy 1: matching_engine search queries with confidence scoring
against Spotify (preferred) or iTunes (fallback).
- Strategy 2: swapped artist/title query.
- Strategy 3: raw (untokenized) query.
- Strategy 4: extended search with limit=50.
- On match → save to discovery cache.
- On miss → build Wing It stub from raw source data.
3. After loop: phase='discovered', sort results by index, and for mirrored
playlists write extra_data back to the DB.
4. Activity feed entry with match summary.
5. On error → state['status']='error', phase='fresh'.
6. Finally: resume enrichment workers.
Dependencies injected via `YoutubeDiscoveryDeps` (16 fields) —
youtube_playlist_states, spotify_client, matching_engine, plus 13
callable helpers (pause/resume enrichment, get_active_discovery_source,
get_metadata_fallback_client, discovery cache key/validate, extract
artist name, spotify_rate_limited, discovery_score_candidates,
get_metadata_cache, build_discovery_wing_it_stub, get_database,
add_activity_item).
Diff vs original after `deps.X` → global X normalization is **zero
differences** — 332 lines orig = 332 lines lifted, byte-identical body
(including all whitespace).
Pre-existing bug preserved (not fixed): if `youtube_playlist_states[url_hash]`
raises KeyError on entry, the outer except handler tries to mutate
`state` which is unbound → secondary UnboundLocalError. Same bug in
the original. Documented here for future cleanup but out of scope
for the lift.
Tests: 14 new under tests/discovery/test_discovery_youtube.py covering
cache hit short-circuit, Strategy 1 confidence match, Wing It fallback,
iTunes fallback path (Spotify unauthenticated and rate-limited),
cancellation (phase changed), skip_discovery flag, completion phase
update, activity feed entry, mirrored playlist DB writeback, non-mirrored
no-writeback, enrichment workers pause/resume, error-during-loop resume,
results sorted by index after retry.
Full suite: 1082 passing (was 1068). Ruff clean.
- remove the redundant wishlist-service injection from the runtime wrappers
- keep the package owning its own singleton service access
- simplify the route runtime API and update the wishlist tests to match
- ignore unconfigured backends when clearing completed downloads
- keep the post-download cleanup route best-effort after a successful wishlist run
- add regression coverage for the orchestrator clear step
First lift in the new PR5 discovery-workers series. Pulls the 448-line
playlist sync background worker out of `web_server.py` into its own
focused module under `core/discovery/`. Pure 1:1 lift — wrappers keep
the original entry-point name so the four callers
(`sync_executor.submit(_run_sync_task, ...)`) continue to work without
changes.
What the sync worker does:
1. Convert frontend JSON tracks → SpotifyTrack/SpotifyPlaylist objects.
2. Normalize artist/album shapes for downstream wishlist parity.
3. Wire a progress_callback that updates `sync_states` + automation card.
4. Patch sync_service for database-only fallback when no media server is
connected.
5. `run_async(sync_service.sync_playlist(...))` and capture the result.
6. Update sync_states to 'finished', push playlist poster image to
Plex / Jellyfin / Emby, record sync history (with re-sync vs new-sync
branching), emit `playlist_synced` event for automation engine, and
persist sync status with a tracks_hash for smart-skip on the next
scheduled sync.
7. On exception → mark error in sync_states + automation; finally clear
progress callback + drop `_original_tracks_map` from sync_service.
Dependencies injected via `SyncDeps` (11 fields) — config_manager,
sync_service, plex_client, jellyfin_client, automation_engine, run_async,
record_sync_history_start, update_automation_progress,
update_and_save_sync_status, sync_states dict, sync_lock. The only
structural drift from a pure paste is the top-of-function variable
binding: original used `global sync_states, sync_service`, lifted version
rebinds them as locals from deps (`sync_states = deps.sync_states` etc.)
since the names aren't module-level in the new file. Same behaviour
otherwise — diff against the original after `deps.X` → global X
normalization is **zero differences**.
Tests: 18 new under tests/discovery/test_discovery_sync.py covering
sync history recording (new + resync), setup error path (with and
without automation_id), missing sync_service handling, sync_playlist
exception handling, successful sync state transition, unmatched-tracks
summary, playlist image upload (plex + jellyfin + zero-synced gate),
automation engine emit, automation progress finished call, sync history
DB persistence (completion + match_details), tracks_hash persistence,
and finally-block cleanup (callback clear + map drop).
Full suite: 1068 passing (was 1050). Ruff clean.
Kicks off the PR5 series — 9 discovery workers totaling ~2,400 lines
across `_run_sync_task`, `_run_*_discovery_worker` family,
`_run_quality_scanner`, and `_process_watchlist_scan_automatically`.
Wishlist-related extractions deliberately skipped to avoid overlap with
kettui's planned `core/wishlist/` package.
- add module-level loggers for the wishlist package instead of threading the web server logger through runtime objects
- default wishlist helper runtimes and cleanup helpers to their package logger while still allowing test overrides
- keep web_server.py as a thin caller that no longer injects its logger into wishlist flows
- extract the remaining wishlist endpoint behavior from web_server.py into core/wishlist/routes.py
- keep web_server.py as a thin Flask adapter around the new route helpers
- add tests that cover wishlist counts, stats, track listing, clear/remove flows, cycle updates, and album-track adds
- add core/wishlist as the home for wishlist payload, resolution, state, processing, reporting, and selection helpers
- move wishlist-specific tests into tests/wishlist alongside the new package layout
- keep web_server.py and the import/search callers as thin adapters for now
Final extraction in the download orchestrator series. Lifts the 586-line
master worker that drives the entire missing-tracks pipeline from
`web_server.py` into `core/downloads/master.py`. Pure 1:1 lift — wrappers
keep the original entry-point name so the three callers
(`missing_download_executor.submit(_run_full_missing_tracks_process, ...)`)
continue to work without changes.
What the master worker does:
1. PHASE 1 ANALYSIS — per-track DB ownership check with album fast path
(lookup album by name+artist, match tracks within it) plus a
MusicBrainz release-cache preflight so per-track post-processing all
uses the same release MBID (prevents Navidrome album splits).
2. Wishlist removal for tracks already in the library.
3. Explicit-content filter.
4. PHASE 2 transition — if nothing missing, mark batch complete, update
per-source playlist phases, kick auto-wishlist completion handler.
5. Soulseek album pre-flight — search for a complete album folder before
falling back to track-by-track search, cache the source for reuse.
6. Wishlist album grouping — derive per-album disc counts and resolve ONE
artist context per album so collab albums don't fold-split.
7. Task creation with explicit album/artist context injection +
playlist-folder-mode flag propagation.
8. Hand off to download_monitor + start_next_batch_of_downloads.
9. Error handler — phase=error, reset YouTube playlist phase to
'discovered', reset auto-wishlist globals on auto-initiated batches.
Dependencies injected via `MasterDeps` (21 fields) — wide surface
covering config, MB caches/locks, soulseek client, source-page state
dicts, multiple callbacks (wishlist removal, explicit filter, executor
+ auto-completion fn, monitor, start_next_batch). The only behaviour
difference from a pure paste is `import traceback` hoisted to module
scope (was inline in the except block) — same behaviour. Trailing
whitespace on two blank lines also got normalized away by the editor;
neither has any runtime effect.
`reset_wishlist_auto_processing` callback wraps the
`global wishlist_auto_processing, wishlist_auto_processing_timestamp`
write + `wishlist_timer_lock` since `global` can't reach back into
web_server.py from a separate module.
Tests: 21 new under tests/downloads/test_downloads_master.py covering
analysis-phase state, force_download_all, found-track wishlist removal,
explicit filter, no-missing complete + per-source state updates, auto
wishlist completion submit, album fast path (direct + fallthrough),
MB preflight (caches both keys, no-mb-worker no-op), task creation
(queue + tasks dict, explicit context for albums, wishlist album
grouping consistency, playlist folder mode), monitor + next-batch
handoff, multi-disc total_discs computation, error handler (phase set,
youtube reset, auto wishlist reset), and batch-removed-mid-flight
defensive path.
Full suite: 1050 passing (was 1029). Ruff clean.
End of the PR4 series — `web_server.py` lost ~590 lines on this commit
alone; total trim across PR4a–PR4h is ~2900 lines of orchestrator code
moved into focused `core/downloads/*.py` modules.
Seventh sub-PR in the download orchestrator series. Strict 1:1 lift —
zero behavior change. ~570 lines moved out of web_server.py.
What moved (lifted as 3 tightly-coupled functions in one module):
- _start_next_batch_of_downloads → start_next_batch_of_downloads
- _on_download_completed → on_download_completed
- _check_batch_completion_v2 → check_batch_completion_v2
Dependencies bundled in `LifecycleDeps` (15+ refs):
- config_manager, automation_engine, download_monitor, repair_worker,
mb_worker (live globals)
- is_shutting_down (lambda over IS_SHUTTING_DOWN flag)
- get_batch_lock (web_server helper for batch_locks dict)
- submit_download_track_worker (lambda wrapping
missing_download_executor.submit + _download_track_worker)
- submit_failed_to_wishlist + submit_failed_to_wishlist_with_auto_completion
(async, used by on_download_completed) AND process_failed_to_wishlist
+ process_failed_to_wishlist_with_auto_completion (sync, used by
check_batch_completion_v2 — direct call matches original v2 behavior;
the non-v2 path always submitted to executor)
- ensure_spotify_track_format, get_track_artist_name,
check_and_remove_from_wishlist, regenerate_batch_m3u (web_server
helpers — large, will lift in follow-up PRs)
- youtube_playlist_states, tidal_discovery_states,
deezer_discovery_states, spotify_public_discovery_states
(per-source playlist state dicts — phase transitions on batch
completion)
Direct imports for already-lifted helpers:
- core.runtime_state.{download_tasks, download_batches, tasks_lock,
add_activity_item}
- core.downloads.history.record_sync_history_completion (PR4a)
- core.album_consistency.run_album_consistency
- core.metadata.common.get_file_lock
Behavior parity verified line-by-line:
- start_next_batch: same batch lock acquisition, same shutdown gate,
same V2-cancelled-task skip, same searching-status-set-before-submit,
same submit-fails-no-ghost-worker semantics
- on_download_completed: same duplicate-call detection (skip decrement
but still check completion), same failed-track tracking with
spotify_track formatting + activity items + automation event emission,
same wishlist removal on success, same active_count decrement, same
stuck-detection (searching > 10min → not_found, post_processing >
5min → completed), same M3U regeneration + repair worker hand-off
+ album consistency pass + wishlist failed-tracks submission
- check_batch_completion_v2: same finished-count tally, same stuck
detection, same already-complete short-circuit returning True, same
per-source playlist phase updates, same album consistency pass,
same DIRECT (sync) wishlist processing call (NOT submit-to-executor
— matches original v2 which called process_* functions directly)
CRITICAL drift caught + fixed during review:
- Initial lift had v2 routing wishlist calls through submit_* deps
(async). Original v2 called process_* directly (sync). Added separate
process_* deps to LifecycleDeps and routed v2 to them. Tests updated.
Two minor defensive additions documented:
- `is_auto_batch = False` initialized before conditional in v2 (Python
scope rules made this unnecessary in original, but explicit is safer)
- Variable rename inside the queue-completion-check loop in
on_download_completed: `task_id` → `queue_task_id` to avoid shadowing
the outer parameter. Log output preserves the same task ID.
Tests: 28 new under tests/downloads/test_downloads_lifecycle.py
covering start-next (early-returns, shutdown gate, max_concurrent,
cancelled-task skip, searching-status-set, submit-failure-no-ghost,
orphan task), on-complete (decrement, duplicate skip, failed/cancelled
tracking, automation emit, wishlist removal, batch completion + emit
+ source phase update, stuck detection, auto vs manual routing),
check-v2 (missing batch, not-complete, complete-marking, already-
complete, auto routing, exception handling).
Full suite: 1029 passing (was 1001). Ruff clean.
Fifth sub-PR in the download orchestrator series. Strict 1:1 lift —
zero behavior change.
What moved:
- _build_batch_status_data → build_batch_status_data
- get_batch_download_status route body → build_single_batch_status
- get_batched_download_statuses route body → build_batched_status
- get_all_downloads_unified route body → build_unified_downloads_response
- Status priority dict → module-level _STATUS_PRIORITY constant
Dependencies bundled in `StatusDeps` dataclass:
- config_manager, docker_resolve_path, find_completed_file,
make_context_key, submit_post_processing (lambda wrapping
missing_download_executor.submit + _run_post_processing_worker),
get_cached_transfer_data
Direct imports from core.runtime_state for download_tasks /
download_batches / tasks_lock (already lifted by kettui).
Behavior parity:
- Same response payload shape across all 3 endpoints
- Same safety-valve mutation: stuck downloading task with file recovered
→ status='post_processing' + submit worker; stuck searching → not_found;
stuck downloading no file → failed
- Same live transfer state mapping (Cancelled/Canceled, Failed/Errored/
Rejected/TimedOut, Completed/Succeeded with byte-mismatch verification,
InProgress, default queued)
- Same intermediate post_processing status promotion + single-shot worker
submission (only when status != 'post_processing')
- Same 'Errored' handling: keeps current status to let monitor retry
- Same 17-key item dict in unified response with same field order
- Same artist/album/artwork normalization (handles string, dict, list,
list-of-dicts, list-of-strings variants)
- Same sort: (priority asc, -timestamp desc)
- Same batch summary aggregation
- Same items[:limit] slicing
- Same logger messages text-for-text
- Same lock scope (single tasks_lock per call) — no new contention
Pre-existing bug preserved (will fix in follow-up PR):
- batched_status `debug_info` block iterates `response["batches"]` and
guards with `if "error" not in batch_status`. Every successful
payload includes `"error": batch.get('error')` (key always present,
value usually None) so the guard is always False and debug_info
never populates in production. Test documents the buggy behavior so
the next PR can flip the check to `batch_status.get('error') is None`.
Tests: 32 new under tests/downloads/test_downloads_status.py covering
phase routing (analysis vs downloading vs unknown), task formatting +
sort + V2 fields, every live transfer state mapping (Cancelled,
Succeeded with full + partial bytes, InProgress, Errored, terminal-
not-overridden), safety valve (stuck searching → not_found, stuck
downloading recovered → post_processing, stuck downloading no file →
failed), all 3 route helpers (single, batched, unified), unified
artist/album/artwork normalization, batch summary aggregation, limit
slicing, plus debug_info bug documentation.
Full suite: 982 passing (was 950). Ruff clean.
Fourth sub-PR in the download orchestrator series. Strict 1:1 lift —
zero behavior change. ~407 lines moved out of web_server.py.
What moved:
- _run_post_processing_worker → run_post_processing_worker
The lifted function is intentionally kept as one ~400-line block to
preserve byte-for-byte parity with the original. Refactoring it into
smaller helpers (context lookup, file search loop, transfer-folder
handler, downloads-folder handler) gets its own follow-up PR.
Dependencies: 9 callbacks bundled in `PostProcessDeps` dataclass.
- config_manager, soulseek_client, run_async (live refs)
- docker_resolve_path, extract_filename, make_context_key
(small utilities still in web_server.py — will lift in a future PR
alongside other shared utilities)
- find_completed_file (file search helper, still in web_server.py)
- enhance_file_metadata, wipe_source_tags (web_server wrappers around
core.metadata.enrichment)
- post_process_with_verification (web_server wrapper around
core.imports.pipeline)
- mark_task_completed (wraps runtime_state.mark_task_completed +
session counter)
- on_download_completed (deferred to PR4g batch lifecycle)
Direct imports for already-lifted helpers (no injection needed):
- core.imports.album_naming.resolve_album_group
- core.imports.context.{get_import_clean_title, get_import_clean_album,
get_import_original_search, get_import_context_artist,
get_import_context_album, normalize_import_context}
- core.imports.filename.extract_track_number_from_filename
- core.metadata.enrichment (re-exported as metadata_enrichment)
- core.runtime_state.{download_tasks, tasks_lock,
matched_downloads_context, matched_context_lock}
Behavior parity:
- Same control flow: missing-task short-circuit → cancelled/completed
short-circuit → missing-filename failure → docker path resolution →
context lookup with fuzzy fallback → expected filename generation →
YouTube special-case path resolution → 5-attempt search loop with
Strategy 1 (original filename in download+transfer) and Strategy 2
(expected final filename in transfer) → file-not-found failure →
transfer-folder handler with metadata enhancement → downloads-folder
handler with full post-process verification
- Same retry count (5), sleep duration (5s), per-attempt logging
- Same album_info dict construction with is_album=True for explicit
album downloads
- Same album grouping skip when context.is_album_download is True
- Same wipe_source_tags fallback when enhancement context missing
- Same matched_downloads_context cleanup on success
- Same exception swallowing at processing-error and critical-error
layers, both setting status='failed' + error_message + calling
on_download_completed(b, t, success=False)
- Every logger message text preserved verbatim (so log filters keep
working)
Tests: 16 new under tests/downloads/test_downloads_post_processing.py
covering missing task, cancelled, already-completed, stream_processed,
missing filename + username, file-not-found-after-retries with sleep
mocked, stream-processor-completes-mid-search, transfer-folder with
metadata enhanced + with no context (wipes tags), downloads-folder
with + without context, processing exception, critical outer
exception, YouTube special path, fuzzy context matching.
Full suite: 950 passing (was 934). Ruff clean.
Third sub-PR in the download orchestrator series. Strict 1:1 lift —
zero behavior change.
What moved:
- _automatic_wishlist_cleanup_after_db_update → cleanup_wishlist_after_db_update
The lifted fn takes config_manager as an arg (so core/downloads/cleanup.py
doesn't need to import web_server). Other deps (wishlist_service,
MusicDatabase, get_database) stay as in-function imports — matches the
original deferred-import pattern.
The single caller in web_server.py (missing_download_executor.submit at
L18028) keeps using the same wrapper name with no signature change.
Behavior parity:
- Same per-profile iteration via get_all_profiles()
- Same essential-field skip (no name / no artists / no spotify_track_id)
- Same artist normalization (string / dict / fallback to str())
- Same 0.7 confidence threshold for db match
- Same break-on-first-artist-match semantics
- Same album extraction (dict.name vs string passthrough)
- Same active_server pulled via config_manager.get_active_media_server()
- Same per-track exception swallowing inside the loops
- Same top-level exception swallow with traceback.print_exc()
- Same logger messages (exact text match for "[Auto Cleanup]" prefix)
Tests: 13 new under tests/downloads/test_downloads_cleanup.py covering
empty wishlist short-circuit, found-in-db removal, missed track stays,
low-confidence skip, missing-fields skip, dict + string artist formats,
break-on-first-match, multi-profile walk, album dict/string handling,
db check failure continuing to next artist, top-level exception swallow,
active server propagation.
Full suite: 934 passing (was 921). Ruff clean.
Second sub-PR in the download orchestrator series. Strict 1:1 lift —
zero behavior change.
What moved:
- cancel_download (single slskd cancel) → cancel_single_download
- cancel_all_downloads (cancel + clear + sweep) → cancel_all_active
- clear_finished_downloads (slskd clear + sweep) → clear_finished_active
- clear_completed_downloads (local task tracker prune) → clear_completed_local
Slskd-touching helpers take (soulseek_client, run_async, sweep_callback)
explicitly so the route layer wires the live client + the existing
_sweep_empty_download_directories helper. The local-state helper imports
download_tasks/download_batches/batch_locks/tasks_lock straight from
core.runtime_state since those are module-level shared globals.
Prep change: `batch_locks` dict moved from web_server.py global into
core/runtime_state.py alongside the other download globals. web_server.py
re-imports from runtime_state so the ~3 existing call sites in
web_server.py keep resolving without modification. Identity preserved
(same dict across all importers).
Out of scope (deferred to PR4g batch lifecycle):
- cancel_download_task (calls _on_download_completed)
- cancel_task_v2 + _atomic_cancel_task + _find_task_by_playlist_track
(manipulate batch active_count directly, deeply coupled to lifecycle)
Behavior parity:
- Same response shapes + status codes on each route
- Same call order (cancel_all → clear_all_completed → sweep)
- Same conditional sweep on clear_finished (skipped on failure)
- Same sweep ALWAYS runs after cancel_all even if clear_all returns False
(matches original — clear failure was non-fatal in cancel_all path)
- Same TERMINAL_STATUSES set: completed/failed/not_found/cancelled/skipped/
already_owned (lifted to module-level constant)
- Same empty-batch pruning + same batch_locks cleanup
- Same lock acquisition pattern (single tasks_lock)
Tests: 14 new under tests/downloads/test_downloads_cancel.py covering
single cancel, cancel-all happy + failure paths, clear-finished + sweep
gate, local task pruning across all 7 active/terminal states, batch
queue trimming, batch_locks cleanup.
Full suite: 921 passing (was 907). Ruff clean.
First sub-PR in the download orchestrator series. Strict 1:1 lift —
zero behavior change.
What moved:
- _record_sync_history_start → record_sync_history_start
- _record_sync_history_completion → record_sync_history_completion
- _detect_sync_source → detect_sync_source
- Source prefix map → module-level _SOURCE_PREFIX_MAP constant
What stayed:
- web_server.py keeps three thin wrappers (_detect_sync_source,
_record_sync_history_start, _record_sync_history_completion) that
delegate into core/downloads/history.py. ~60 callers of these names
in web_server.py keep resolving without touching every site.
Each lifted function takes `database` as an arg (was
`db = MusicDatabase()` inline). The wrappers construct
`MusicDatabase()` per call to mirror the exact original behavior —
each invocation got a fresh DB connection.
Behavior parity:
- Same SQL UPDATE statement (preserves the in-place update path when
a sync_history entry already exists for the playlist_id)
- Same JSON serialization with ensure_ascii=False
- Same thumb URL extraction order (album_context.images → image_url
→ first track album.images)
- Same per-track result shape (index, name, artist, album, image_url,
duration_ms, source_track_id, status, confidence, matched_track,
download_status)
- Same status mapping (found/not_found, completed/failed)
- Same best-effort exception swallowing (sync history failure must
never break the actual download)
- Reads `download_tasks` from core.runtime_state (already lifted by
kettui in PR378)
Tests: 34 new under tests/downloads/test_downloads_history.py
covering source detection (16 prefixes), start happy paths + thumb
extraction + duplicate-update + DB error swallowing, completion stats
+ per-track results JSON shape + edge cases.
Full suite: 907 passing (was 873). Ruff clean.
The endpoint was returning a 200-line literal dict inline. Moved the
three lists (TRIGGERS, ACTIONS, NOTIFICATIONS) to module-level constants
in core/automation/blocks.py. Route shrinks to 7 lines. Data is now
importable for tests + future docs.
Added 8 shape tests so a typo in the dict (missing 'type', wrong
field type, missing options on a select, etc.) gets caught by CI
instead of breaking the builder UI silently.
The `known_signals` field stays computed at request time via
_collect_known_signals(database) since it's dynamic.
No behavior change. Same response shape. 869 tests passing (was 861).
Ruff clean.
Routes moved to thin parse-args/jsonify handlers; logic now lives in
three focused modules under core/automation/. 436 lines deleted from
web_server.py; 53 added back as wrappers.
Module split:
- core/automation/api.py — CRUD + run + history helpers. Each function
takes (database, automation_engine, ...) explicitly and returns
(response_body, http_status). Includes signal cycle detection
preflight checks for create + update.
- core/automation/progress.py — owns the in-memory progress state dict
+ lock (mirroring the original web_server.py globals as module-level
shared state so all callers see one view), init/update/history
helpers, and the WebSocket emit loop.
- core/automation/signals.py — collect_known_signals for the builder
autocomplete.
Out of scope (deferred):
- _register_automation_handlers — the 23+ action handler closures stay
in web_server.py because each one is tightly coupled to feature-
specific implementations (wishlist, watchlist, library scan, etc.).
- Worker functions (_process_wishlist_automatically, etc.) — belong
with their feature lifts.
- _run_sync_task / _run_playlist_discovery_worker — sync + discovery
PRs.
Behavior preserved 1:1:
- Same route response shapes + status codes
- Same JSON field hydration (trigger_config, action_config,
notify_config, last_result, then_actions)
- Same backward-compat: empty then_actions + notify_type set →
synthesize then_actions from notify_type/notify_config
- Same signal cycle detection behavior on create + update
- Same system-automation protection on delete + duplicate
- Same reschedule/cancel logic on toggle + bulk-toggle + update
- Same progress state shape (status, progress, phase, current_item,
log capped at 50, started_at/finished_at, action_type)
- Same emit-on-finish socketio push from update_progress
- Same emit loop semantics (1s tick, snapshot active states, reap
finished after window)
Pre-existing bugs preserved (will fix in follow-up PRs):
- emit_progress_loop uses naive datetime.now() against tz-aware
started_at/finished_at, so the timeout-zombie check raises
TypeError → caught → never fires, and the cleanup-after-window
check raises → caught → state is reaped on FIRST tick regardless
of the window. Tests document this behavior so the next PR can
flip them to the corrected expectation.
Tests: 72 new under tests/automation/ (signals 10, progress 24,
api 38). Full suite: 861 passing (was 789). Ruff clean.
Three drifts caught in line-by-line review against the pre-lift
web_server.py. All addressed for strict 1:1 behavior parity.
1. /api/enhanced-search/source/<src> now returns plain JSON
`{"artists":[],"albums":[],"tracks":[],"available":false}` (or
`{"videos":[],"available":false}` for youtube_videos) when the
source's client isn't available, matching the original endpoint
contract. Previously streamed an NDJSON `{"type":"done"}` line
instead.
Restructured by splitting the orchestrator into resolve+stream
helpers:
- `resolve_client(source_name, deps)` — already existed, used
for /api/enhanced-search single-source mode
- `resolve_youtube_videos_client(deps)` — new, returns the
soulseek_client.youtube subclient or None
- `stream_metadata_source(source_name, query, client)` — pure
NDJSON generator, caller resolves client first
- `stream_youtube_videos(query, youtube_client, run_async)` —
same shape for the yt-dlp path
The route now decides plain-JSON-vs-stream based on resolution
result, mirroring the original control flow exactly.
2. core/search/library_check.py — reverted the defensive `(x or '')`
and `getattr(plex_client, 'server', None) is not None` patterns
to original byte-for-byte (`x.get('name', '')`,
`plex_client.server`, no try/except around `get_plex_config`).
Lift PR shouldn't change crash semantics; if the original raises
on malformed input, mine should too. Pre-existing edge cases get
their own follow-up PR.
3. core/search/stream.py — same revert: `soulseek_client.youtube`
instead of `getattr(..., 'youtube', None)` etc.
Also removed the module-level `EMPTY_SOURCE` from sources.py and
moved its (per-call) duplicate into _fan_out_response as a local —
the original used a per-request local dict and the identity-check
behavior depends on that. Module-level was a footgun for future
mutations.
789 tests still pass (95 search), ruff clean.
Routes moved to thin parse-args/jsonify handlers; logic now lives in
six focused modules under core/search/. 720 lines deleted from
web_server.py; 109 added back as wrappers; ~700 lines of new core code
plus ~700 lines of tests.
Module split:
- core/search/cache.py — TTL+LRU cache for enhanced-search responses,
keyed by (query, active_server, fallback_source, hydrabase_active,
source_tag) so config changes don't poison stale entries.
- core/search/sources.py — per-kind metadata search (artists/albums/
tracks) and the multi-kind ThreadPoolExecutor that fans them out.
- core/search/library_check.py — library + wishlist presence check
with Plex thumb URL resolution; profile-aware wishlist with legacy
fallback for older DBs missing the profile_id column.
- core/search/stream.py — single-track preview search; effective stream
mode resolution, query-variant generation, retry walk, matching
engine integration.
- core/search/basic.py — flat Soulseek file search, quality-sorted.
- core/search/orchestrator.py — main enhanced-search dispatch
(short-query fast path, single-source bypass, hydrabase-primary fan
out, alternate source list builder), NDJSON streaming generator
for /source/<src>, and the SearchDeps dataclass that bundles the
cross-cutting deps.
Routes pass clients (spotify, hydrabase, hydrabase_worker, soulseek)
and helpers (config_manager, fix_artist_image_url,
_is_hydrabase_active, _get_metadata_fallback_*, _run_background_
comparison, run_async, dev_mode_enabled_provider) into core/search via
a SearchDeps bundle built per-request. fix_artist_image_url stays in
web_server.py because it touches 31 other call sites.
Behavior preserved 1:1:
- Same response shapes (db_artists, spotify_artists, spotify_albums,
spotify_tracks, primary_source, metadata_source, alternate_sources,
source_available)
- Same NDJSON line ordering (artists/albums/tracks as they finish, plus
done marker)
- Same per-kind exception swallowing
- Same hydrabase-worker mirror on dev mode
- Same cache key shape (5-tuple) and TTL/LRU semantics
- Same stream-track effective-mode resolution including the
Soulseek-coerce-to-YouTube edge case
- Same library-check Plex thumb URL rewriting and wishlist fallback
for older DBs
Tests: 94 new (cache TTL/LRU/key, sources happy/partial/all-fail,
library presence with library + wishlist + thumbs, stream effective
mode + query gen + retry, orchestrator client resolution + short
query + single source + fan-out alternates + hydrabase primary +
NDJSON drain). Full suite: 788 passing (was 694).
Ruff clean.
Stats route logic moves into core/stats/queries.py as pure-ish functions
that take dependencies (database, image-url fixer, listening worker) as
arguments. The 13 route handlers in web_server.py shrink to thin
parse-args / jsonify wrappers.
What moved to core/stats/queries.py:
- stats_cached: 3-key metadata cache lookup + image url fix-up
- stats_overview / timeline / genres / library_health / db_storage
- stats_top_artists / top_albums / top_tracks: top-N + DB enrichment
- stats_recent: listening_history readback
- stats_resolve_track: title+artist -> file_path lookup for playback
- listening_stats_sync: spawns daemon thread that runs worker._poll
- listening_stats_status: stats payload, with None-worker fallback shape
No behavior change. Same response shapes, same error handling, same
silent-except on per-row enrichment failure. fix_artist_image_url
stays in web_server.py and is passed through as a callback so we
don't have to lift its config_manager / media-server dependencies in
this PR.
Adds tests/stats/test_stats_queries.py — 27 tests covering happy
paths, edge cases, image-url plumbing, worker glue.
Ruff clean. 694 tests pass (was 667 + 27 new).
Lifted-then-not-deleted leftovers from the PR378 merge:
- web_server.py `_resolve_album_group` and `_build_final_path_for_track`
were already imported at module top from `core/imports/`. Removed the
shadowing local copies.
- Mutagen reimports (FLAC/MP4/OggVorbis) at L17736-17738 shadowed the
top-of-file imports. Picture/MP4Cover/MP4FreeForm were unused. Dropped
the whole block.
- core/imports/context.py: `getattr(artist, "name")` -> `artist.name`
(B009).
Ruff clean, 667 tests pass.
- keep single-track import lookup in imports/resolution.py
- normalize simple-download search_result data before wishlist matching
- run wishlist cleanup for simple-download post-processing
- keep source-only artist detail on resolved names and MB short-circuit
- pass the selected manual match through singles import
- keep the import context source-aware so artist and album stay correct
- avoid treating non-Spotify IDs as wishlist Spotify IDs
- make wishlist logging and local variable names source-neutral
- Move the import pipeline runtime factory into core.imports.pipeline
- Move the metadata runtime factory into core.metadata.enrichment
- Keep the web server wiring thin and drop the shared glue module
- Add contract tests that keep the two runtime bundles separate
- Relocate the shared metadata helper module from core/metadata_common.py into core/metadata/common.py.
- Update the new metadata package, the import pipeline, and the web entrypoint to use the package-scoped helper.
- Keep the shared config, mutagen, file-lock, and tag-writing helpers centralized without touching unrelated files.
- Pass the live runtime bundle into the shared metadata facade so worker-backed source enrichment can actually run.
- Forward runtime from the import pipeline and web-server wrapper into embed_source_ids.
- Add a regression test that verifies the runtime object reaches the source-ID embedding path.
- Keep existing metadata_cache and metadata_service at the top level for now
- Move the new branch-local metadata helpers under core/metadata
- Share MusicBrainz release cache state from core.metadata.source and update import sites
- Move app-wide task and activity registries out of core/imports
- Share one runtime-state module across the web server, API, and import pipeline
- Keep import-specific helpers focused on context and post-processing
- Move import flow modules into a dedicated package
- Update app and test imports to the new namespace
- Group the import-focused tests under tests/imports
- remove runtime from metadata helper APIs where it only carried config, logger, mutagen, and database access
- keep runtime only for the source-ID enrichment path that still needs live worker handles
- add the new metadata helper modules and update the tests to match the slimmer interfaces
- Extract the import pipeline, album import, staging, path, file ops, guards, runtime state, side effects, and metadata enrichment out of .
- Canonicalize the refactored import path around and remove legacy , , , and request shapes from the import endpoints.
- Make album and track metadata lookups follow the configured provider priority instead of hard-coding Spotify, while still falling back when needed.
- Update the import routes and frontend payloads to use the new core helpers.
- Add coverage for the extracted helpers and the refactored import flows.
PS. apologies to anyone who might check this commit out - the intention was to start small, but things kinda snowballed out of control at some point since the logic just kept going on and on, and everything kinda had to be changed all at once for it all to make any sense
Addresses #365 (reported by JohnBaumb), parts 3 & 5. Client-side
IDB / sessionStorage data cache (part 4) deferred to its own PR.
Cover art on Library and Discover used to re-fetch from the source
CDN on every page visit. Now a service worker caches images locally
in CacheStorage with cache-first strategy — second visit serves art
instantly with zero network round-trips. PWA manifest added so the
app is installable to home screen / desktop.
Service worker (`webui/static/sw.js`):
- Cache-first for images: 10 known CDN hosts (Spotify, Last.fm,
Apple, Deezer, Discogs, MusicBrainz CAA, YouTube thumbnails) plus
the local `/api/image-proxy` endpoint plus same-origin .png/.jpg/
.webp/.gif/.svg paths. Cross-origin file-extension matches are
refused so we don't accidentally cache trackers.
- Stale-while-revalidate for `/static/*`: serve cached instantly,
refresh in background. Combined with the existing `?v=static_v`
cache-bust, deploys still ship live (different query → different
cache entry, old ages out).
- HTML / API / everything else: no caching, pass through.
- Cache-versioned (CACHE_VERSION = 'v1'); activate handler wipes any
cache whose name doesn't match the current version.
- skipWaiting + clients.claim so deploys propagate to open tabs
without requiring a full close-and-reopen.
PWA manifest (`webui/static/manifest.json`):
- Standalone display mode, theme color #1db954 (matches --accent-rgb).
- Two icons (192, 512) with both `any` and `maskable` purpose,
generated from favicon.png with aspect-preserving transparent
padding so the existing logo lands inside the safe zone for
OS-applied masks.
Wiring:
- `web_server.py` adds a `/sw.js` route that serves the file from
root scope (a service worker only controls URLs at or below its
served path; `/static/sw.js` would scope to `/static/*` only).
`Cache-Control: no-cache` on the SW response so deploys propagate
on next page load instead of being pinned by the 1yr static cache
the rest of /static/ uses.
- `webui/index.html` adds the manifest link, theme-color meta, and
an apple-touch-icon for iOS.
- `webui/static/init.js` registers the SW on `window.load`.
Feature-detected — no-op on browsers without serviceWorker support
or on non-secure origins (SW requires https or localhost).
One bug caught + fixed during line-by-line self-review:
`_staleWhileRevalidate` could return null to `respondWith()` when
both the cache miss AND the network fetch failed (the `.catch(() =>
null)` collapsed the rejection to null, which then short-circuited
through the falsy chain). Now explicitly awaits the network promise
and falls back to `Response.error()` when it resolves to null —
matches the `_cacheFirst` pattern.
Browser-verified: sw.js registers, status "activated and is running"
in DevTools. 603 tests pass.
Self-review nit on b0e7dae. Discover data is user-specific (hero
artists from your watchlist, similar artists from your taste,
recently-played derivations, etc.) — `Cache-Control: public` would let
intermediate proxies (corporate caching proxy, Cloudflare with cache
rules, Nginx with proxy_cache) store one user's response and serve
it to another. Privacy leak.
Switched to `private, max-age=300`. Browser-only cache, proxies skip.
Static assets stay `public` (shared content — everyone gets the same
library.js). Streaming and backup endpoints already correct
(`no-cache` and `no-store` respectively).
603 tests pass.
Addresses #365 (reported by JohnBaumb), parts 1 & 2 of the proposal.
Service worker, client-side IDB/sessionStorage, and PWA manifest
deferred to follow-up PRs.
1. Static asset cache (CSS/JS/icons/fonts).
`SEND_FILE_MAX_AGE_DEFAULT` flipped from 0 to 31536000 (1 year) in
production. Safe because every static URL is bust-tagged with
`?v=static_v` (computed once per process start), so each server
restart effectively invalidates every cached asset for every user.
Within a single deploy, repeat page loads hit zero round-trips on
static files — was a 304 round-trip per asset before.
Dev override (`SOULSYNC_WEB_DEV_NO_CACHE=1`) keeps it at 0 so
iterating on JS/CSS doesn't need a server restart between edits.
Collateral fixes from the bump:
- Music streaming endpoint (L16140): `response.headers.add('Cache-Control',
'no-cache')` → bracket-assign. Under the old max-age=0, send_file
set `no-cache` and `.add()` duplicated harmlessly. Under the new
max-age=31536000, `.add()` would APPEND a second Cache-Control
value → two conflicting headers, browser-undefined behavior.
Bracket-assign replaces.
- Backup download endpoint (L25181): explicit `Cache-Control:
no-store` on the response so DB backups don't inherit the new
long max-age — sensitive content, must never cache.
2. Discover GET browser cache (5 min).
New `@app.after_request` hook scoped to `/api/discover/` and
`/api/discovery/` paths, GET method, 2xx responses only. Sets
`Cache-Control: public, max-age=300`. Skipped when the endpoint
already set its own Cache-Control. Toggling between Discover
sections within 5 min serves from browser cache, no backend hit.
Try/except wraps the hook body and logs a warning if anything
throws — never let a header-tagging bug turn a successful response
into a 500. (Logging instead of `pass` since silent except-pass is
exactly the anti-pattern issue #369 is about.)
Audited every other Cache-Control set site in web_server.py — only
the two `send_file` callers needed adjustment. Range-branch streaming
uses `Response()` directly, unaffected by the config change.
603 tests pass.
Closes#370 (reported by JohnBaumb).
The /api/settings endpoint and three siblings (/log-level,
/config-status, /verify) had no auth check — any logged-in profile
could read or modify service tokens, OAuth secrets, and API keys.
Cin's "minimum" suggestion from the issue: gate to admin profile.
Added an `admin_only` decorator near `get_current_profile_id` that
returns 403 when the current profile isn't admin (id=1). Applied
to all four endpoints.
Auth model note (documented in the decorator docstring): SoulSync's
existing model is "trust local network" — single-admin / no-multi-
profile installs default `get_current_profile_id()` to 1, so the
gate is a no-op for solo users. The decorator is meaningful in
multi-profile setups where non-admin sessions exist. Tightening to
real per-request auth is out of scope.
Did NOT consolidate with api/settings.py (Cin's "better" suggestion):
that endpoint uses API-key auth (for external tools), the web_server.py
copy uses session/profile auth (for the web UI). Different consumers,
different auth models — merging would break one or the other.
603 tests pass.
Six items from a Cin-style line-by-line pass on PR #383:
- resolve_cors_origins: list of non-string entries (`[None, 123]`) now
drops them instead of coercing to junk strings like `'None'`/`'123'`.
- will_reject: backwards-compat shim removed. Production callers always
pass `request.scheme` (Flask-guaranteed); the shim only existed for
tests/non-Flask callers and made the production code path branchier
than necessary. Tests now pass scheme explicitly.
- maybe_log: redundant `if not origin` early-return dropped. will_reject
handles missing origin (engineio's own behavior — server.py:207).
- RejectionLogger.__init__: `int(dedup_cap)` wrapped in try/except so
bad-type input falls back to DEFAULT_DEDUP_CAP instead of raising.
- web_server.py: docstring on the before_request hook explains why the
hook fires on every request (Flask doesn't scope before_request to a
path prefix; the early-return string compare is the cheapest option).
- settings.js: cors-origins URL regex tightened from `[^\s/]+` to
`[^\s/?#]+` so query/fragment chars don't pass validation. Engineio
would silently fail to match those anyway; better to flag at save.
Test changes:
- parametrize gained an explicit `scheme` column (12 cases updated).
- New explicit case: scheme-mismatch rejects (engineio compares full
`{scheme}://{host}` strings).
- `test_will_reject_falls_back_to_host_only_when_no_scheme_info`
deleted — the shim it tested is gone.
- `test_will_reject_honors_x_forwarded_host` now passes scheme info.
Net: -9 production lines, -3 test lines. Production code path is
straight-line. 603 tests pass.
Self-review pass on the security fix uncovered five issues, all fixed
here:
1. will_reject scheme handling. Engineio compares full {scheme}://{host}
strings, not just hostnames. A TLS-terminating proxy can leave the
backend seeing http while the browser's Origin is https — engineio
rejects, but the original predictor said "allow" → no helpful log
line. Added request_scheme + forwarded_proto params, build full
candidate strings to match engineio.
2. EITHER-forwarded-header rule. Engineio adds the forwarded candidate
when EITHER X-Forwarded-Proto OR X-Forwarded-Host is present (it
falls back to HTTP_HOST for the missing one). The original predictor
only added it when forwarded_host was set — false negative for
misconfigs sending only X-Forwarded-Proto. Now mirrors engineio.
3. will_reject incorrectly rejected missing-Origin requests. Engineio
(server.py:207: `if origin: validate`) skips CORS validation when
no Origin header is sent — non-browser clients (curl etc.) are
intentionally permitted. The original code rejected them. Test was
asserting the wrong behavior. Both fixed.
4. RejectionLogger had unbounded dedup set growth. A hostile actor
opening connections from many distinct fake origins would fill
memory unboundedly. Capped at 100 unique origins (configurable);
when cap hit, one overflow notice is emitted and further rejections
are silently dropped until restart.
5. Lock pattern: the overflow log path called logger.warning() while
holding the dedup lock, inconsistent with the normal path. Fixed
to pick the message under the lock and log after release. Critical
section is now minimal and uniform.
Plus polish:
- Stale module docstring fixed (said "empty list" instead of "None").
- settings.js validates each cors_origins line against a URL regex on
save; toasts a one-shot warning if entries are malformed (resolver
silently filters them, but user gets feedback now).
- web_server.py wiring passes request.scheme + X-Forwarded-Proto so
the predictor has full proxy info.
Tests:
- 51 unit tests in tests/test_socketio_cors.py (was 45). New cases:
* scheme comparison (5 cases including TLS-terminating proxies)
* forwarded_proto-alone misconfig
* missing-origin matches engineio (was asserting wrong behavior)
* dedup cap with overflow + reset
* default cap is reasonable (uses public DEFAULT_DEDUP_CAP constant)
Engineio behavior independently verified by reading engineio/server.py
and engineio/base_server.py source. Predictor mirrors both files.
604 tests pass.
Closes#366 (reported by JohnBaumb).
Socket.IO was initialized with `cors_allowed_origins='*'`, accepting
WebSocket connections from any origin. A malicious site could open a
WS to a user's local SoulSync instance and exfiltrate live progress /
toast / activity events.
This commit:
- Defaults to engineio's same-origin behavior (`cors_allowed_origins=None`),
which automatically honors X-Forwarded-Host so reverse proxies that
send that header (Caddy / Traefik by default, properly-configured
Nginx) work transparently.
- Adds a `security.cors_origins` config setting + Settings → Security
textarea where users behind unusual proxies / Electron wrappers /
cross-origin integrations can whitelist their origin. Accepts comma
or newline separated values; `*` on its own line opts back into the
legacy wildcard with a startup-warning log.
- Logs a clear warning the first time engineio rejects each unique
origin, naming the rejected Origin and request Host and pointing
users to the settings field. Without this, engineio silently 403s
the upgrade and the user just sees a half-broken UI with no clue
why. Threadsafe dedup so a hostile origin can't spam logs.
Logic lives in `core/socketio_cors.py` (resolver, rejection
predictor, dedup logger class, startup-status emitter) — pure
functions, no Flask dependency. `web_server.py` adds 23 lines of
wiring and imports.
Important catch during review: my first pass used `cors_allowed_origins=[]`
as the "secure default." Reading engineio's source revealed `[]` actually
means "DISABLE CORS HANDLING" (engineio/server.py:202: `if cors_allowed_origins != []:`)
— identical security to `'*'`. Fixed to use `None` (engineio's actual
same-origin sentinel) and pinned with a regression test that asserts
the resolver never returns `[]` for any input shape.
Tests:
- tests/test_socketio_cors.py — 45 unit tests covering 19 resolver shape
cases (None, empty, whitespace, comma, newline, garbage types, lists),
the `[]`-must-never-be-returned security regression, 12 rejection
prediction cases, X-Forwarded-Host handling, dedup logger behavior,
threadsafe race (8 threads × 50 hammers → exactly 1 warning), and
startup-status emitter outputs.
Frontend:
- Settings → Security gains an "Allowed WebSocket Origins" textarea
with help text explaining same-origin default + when to add a domain
+ the `*` opt-out.
- helper.js — new '2.4.1' WHATS_NEW block (hidden until version bump)
with a chill-voice entry describing the change.
Conftest.py left at `'*'` — test environment, no security concern.
598 tests pass.
The version modal pulled its content from /api/version-info — a 295-line
hand-curated Python dict in web_server.py. The "What's New" panel pulled
its content from WHATS_NEW in helper.js. Same release notes, two files,
two languages, hand-edited at every release — drift was inevitable
(and happened: the kettui-fix entries I added recently differed in
detail between the two surfaces).
This commit makes helper.js the single editing surface:
- Adds VERSION_MODAL_SECTIONS const in helper.js right beside WHATS_NEW,
with a comment block documenting the relationship: WHATS_NEW is the
per-version detailed log used by the helper popover; VERSION_MODAL_SECTIONS
is the curated highlight reel shown by the sidebar version button. Both
edited at release time, both in the same file.
- Rewires showVersionInfo() in downloads.js to read from those consts
directly. No backend round-trip; the changelog content ships in the
same JS bundle the browser already loaded.
- Deletes the /api/version-info route and its 295-line version_data dict.
- Updates the line-39 comment to drop the now-stale "version-info endpoint"
reference.
Note: this is collocation, not true unification. WHATS_NEW and
VERSION_MODAL_SECTIONS are still two distinct structures with overlapping
content, linked by a comment convention rather than a shared schema. A
deeper refactor (e.g. a `featured` flag on WHATS_NEW entries that the
modal aggregates) was rejected as out-of-scope — the curated section
titles ("Earlier in v2.3", "Recent Fixes") aren't 1:1 mappable to
WHATS_NEW entries. Saving for a follow-up if the drift problem persists.
Risk audit:
- Load order: helper.js loads at line 7967, downloads.js at line 7873.
Both classic scripts execute synchronously before any clickable
interaction, so showVersionInfo (only invoked on the version-button
onclick) always sees both consts defined.
- populateVersionModal() unchanged — receives the same {title, subtitle,
sections: [{title, description, features, usage_note?}]} shape.
- Stale-cache window during deploy: old downloads.js hitting a 404 on
the deleted endpoint falls through to the existing catch + toast path
("Failed to load version information"). Cache-buster ?v=static_v
resolves on next page load.
553 tests pass. helper.js + downloads.js parse cleanly. No residual
references to /api/version-info anywhere in the repo.
_OLD_V22_NOTES (655 lines) and _OLD_V2_NOTES (556 lines) were
triple-quoted Python strings holding old release-notes JSON. No code
references them — `grep _OLD_V22_NOTES|_OLD_V2_NOTES` returns only
the definitions themselves. They were leftover from earlier
version-info refactors and have been sitting in the file unread.
Pure deletion. No behavior change.
- _SOULSYNC_BASE_VERSION → 2.4.0 (was 2.39).
- Migrate WHATS_NEW key '2.40' → '2.4.0', strip unreleased flags off
the 27 entries shipping in this release, set release date.
- Replace parseFloat() version compare with proper int-tuple semver
comparator — parseFloat('2.4.0') and parseFloat('2.4.1') both return
2.4, which would have made future patch bumps invisible to the
What's New surfacing logic.
Five issues kettui flagged on PR #377:
- Worker race (reorganize_queue.py): _next_queued() picked an item and
released the lock, then re-acquired to flip status='running'. A
cancel() landing in that window marked the item cancelled but the
worker still ran it. Replaced with _claim_next_or_wait() that picks
AND flips under one lock acquisition.
- Wakeup race (reorganize_queue.py): _wakeup.clear() after the empty
check could lose an enqueue's _wakeup.set(), parking a freshly-queued
album for up to 60 seconds. Replaced Lock + Event with a single
threading.Condition; cond.wait() releases and re-acquires atomically
on notify.
- Bulk dedupe (reorganize_queue.py:enqueue_many): looped single-item
enqueue, so a duplicate album_id later in the same batch could slip
through if the worker finished the first copy before the loop
reached the second. Now holds the lock for the whole batch and tracks
a per-batch seen set, so intra-batch duplicates dedupe against each
other and not just pre-existing items.
- Preview button stuck disabled (library.js:loadReorganizePreview):
early returns and thrown errors skipped the re-enable line. Moved
state into a canApply flag committed in finally, so any exit path
lands the button correctly.
- DB helpers swallowing failures (music_database.py): get_album_display_meta
and get_artist_albums_for_reorganize used to catch every Exception
and return None / [], so a real DB outage masqueraded as "album not
found" / "no albums". Now lets exceptions bubble; the route layer
already wraps them as 500.
Tests:
- test_cancel_and_run_are_mutually_exclusive — hammers enqueue+cancel
pairs and asserts the invariant that no successfully-cancelled item
ever ran (catches regressions to the atomic pick).
- test_enqueue_many_dedupes_batch_internal_duplicates — pins the
intra-batch dedupe.
- test_get_album_display_meta_propagates_db_errors and
test_get_artist_albums_for_reorganize_propagates_db_errors — pin
the bubble-up behavior.
Changelog updated in helper.js and version modal.
Replaces the single-slot "one reorganize at a time, return 409 on collision"
model with a per-user FIFO queue. Buttons stay clickable, "Reorganize All"
is one backend call instead of an N-call JS loop, and a status panel mounted
at the top of the artist actions bar shows live progress (active item,
queued count, recent completions) with per-item cancel buttons.
Backend
- core/reorganize_queue.py: singleton queue + worker thread, dedupe-on-
enqueue, cancel rules (queued cancellable, running not), enqueue_many
for bulk operations, progress fan-out via update_active_progress
- core/reorganize_runner.py: factory builds the worker's runner closure
with injected dependencies. Reads config per-call so changing the
download path in Settings takes effect on the next reorganize without
a server restart
- database/music_database.py: get_album_display_meta and
get_artist_albums_for_reorganize — moves the SQL out of route handlers
- web_server.py: thin enqueue/snapshot/cancel/clear endpoints, runner
registration at module load. Old _reorganize_state globals + status
endpoint deleted. Static-asset cache buster (?v=<server-start>)
added so JS/CSS updates ship live without users clearing cache
Frontend
- webui/static/library.js: status panel mount, polling (1.5s when
active, 8s when idle), expand/collapse, per-item cancel, debounced
enhanced-view reload (one reload per artist batch instead of N).
Per-album reorganize button paints with queued/running indicator
and short-circuits to a toast when the album is already in queue
- webui/static/style.css: panel + button styling matching the existing
glass-UI accents
- webui/static/helper.js + version modal: WHATS_NEW entry
Tests (22 new)
- tests/test_reorganize_queue.py (19 tests): FIFO order, dedupe,
per-item source, cancel rules, continue-on-failure, snapshot
shape, progress propagation, bulk enqueue
- tests/test_reorganize_runner.py (4 tests): per-call config reads,
setup-failure summary, dependency injection, progress fan-out
- tests/test_reorganize_db_methods.py (7 tests): SQL JOIN behavior,
ordering, fallback for blank strings, artist isolation
Full suite 549 passed in 27s.
Four changes addressing kettui's PR #377 review comments:
1. **`_finalize_track` no longer over-counts on DB failure (🔴 bug).**
The function previously bailed on DB-update failure but
`_process_one_track` still incremented `summary['moved']`
unconditionally — overstating how many tracks the UI knows are
at their new locations. Fixed by:
- `_finalize_track` now returns ``bool`` (True only when DB row
was updated AND original was dealt with)
- Caller checks the return; on False, records as a failed track
with a clear message ("Track landed at new location but DB
update failed — file is at both old and new paths until library
scan re-indexes")
- Existing `test_db_update_failure_leaves_original_in_place` now
also asserts `moved == 0`, `failed == 1`, and that the error
message names the cause
2. **`executeReorganize` toast no longer says "undefined tracks" (🐛
bug).** `/reorganize` doesn't return `result.total` anymore (the
track count is determined server-side after planning), so the
"Reorganizing undefined tracks..." string was meaningless. Now uses
`result.message` from the backend instead.
3. **`_pollReorganizeStatus` distinguishes completed from skipped
(🟡 risk).** Backend now propagates the orchestrator's status
(`completed` / `no_source_id` / `no_album` / `no_tracks` /
`setup_failed` / `error`) into `_reorganize_state['result_status']`
so the frontend can warn appropriately. Two new helpers:
- `_classifyReorganizeOutcome(state)` — returns 'success' only
when `result_status === 'completed'` AND `failed === 0`;
'warning' otherwise
- `_formatReorganizeResultMessage(state)` — returns a message
specific to the outcome ("Reorganize skipped — album has no
metadata source ID. Run enrichment first." for `no_source_id`,
etc.)
Zero-failure non-completed runs now show as warnings instead of
green checkmarks.
4. **Bulk mode no longer counts skipped albums as succeeded (🟡
risk).** `_executeReorganizeAll`'s loop was treating any HTTP
200 response as success, ignoring the orchestrator's actual
outcome for that album. Fixed by:
- `_waitForReorganizeComplete()` now resolves with the final
state object (was: void)
- Loop checks `finalState.result_status === 'completed'` AND
`finalState.failed === 0` before counting `succeeded++`;
otherwise increments `skipped` (with a per-album warning
toast) or `failed` accordingly
- Final summary toast now reads
"Reorganized N of M albums, K skipped, J failed" and only
shows green when nothing was skipped or failed
All four addressed in a single commit because they form one
coherent UX-correctness fix — the bug bug (#1) and the count-
overstatement bug (#4) both made the user see "everything succeeded"
when reality was different. Together they make the UI honestly
reflect what actually happened.
Files:
- core/library_reorganize.py — `_finalize_track` returns bool,
`_process_one_track` reads it
- web_server.py — `_reorganize_state['result_status']` populated
from orchestrator's summary on success and on exception
- webui/static/library.js — `_classifyReorganizeOutcome` /
`_formatReorganizeResultMessage` helpers, single-album +
bulk-mode flows both consume them
- tests/test_library_reorganize_orchestrator.py — strengthened
the existing DB-failure test to assert moved/failed counts
Credit: kettui — four PR #377 review comments named all of these
precisely with line numbers and severity.
Reported on Discord by winecountrygames. The library "Reorganize" tool
had several layered bugs that all traced to the same root cause: the
endpoint reinvented every wheel post-processing already turns — its own
template engine, its own disc-number resolution from file tags, its own
sidecar sweep, its own collision detection — and each had drifted from
the canonical path used by fresh downloads. Reported symptoms:
- 3-disc Aerosmith deluxe collapsed to a flat single-disc layout
- Half the tracks on other albums silently skipped, no error / no count
- Re-runs left empty leftover album folders cluttering the artist dir
Architecture: stop reinventing wheels. Route reorganize through exactly
the same pipeline downloads use. Per-album:
1. Fetch the canonical tracklist from a metadata source (Spotify /
iTunes / Deezer / Discogs / Hydrabase) using the album's stored
source IDs. New `core/library_reorganize.py::plan_album_reorganize`
does this — primary-source-first, fall through priority chain
unless the user picked a specific source in the modal (strict mode).
2. For each local track, find the matching API entry via a scored
candidate matcher. Score components: exact-title (100),
substring-with-length-ratio (40-90), track-number agreement (20).
Hard reject when the two titles have different version
differentiators (Remix vs no-remix means different recordings,
not annotation drift). Below threshold = unmatched, surfaced as
"not in source's tracklist, left in place" rather than silently
mis-routing.
3. Copy the file to a per-album staging directory, build the same
context dict the import flow builds (`spotify_album` /
`track_info` / etc. with `is_album_download=True` so the path
builder enters ALBUM mode, not SINGLE mode), call
`_post_process_matched_download(...)` — same function fresh
downloads use. Post-process handles tagging, multi-disc subfolder
decisions, sidecar regeneration, AcoustID verification.
4. Read `context['_final_processed_path']` to learn where it landed.
Update `tracks.file_path` in the DB BEFORE removing the original
(DB-update failure leaves the file at both locations, recoverable
via library scan; the reverse would orphan the row). Delete
per-track sidecars (post-process recreates them at the new
destination).
3 concurrent workers per album via ThreadPoolExecutor, matching the
download path's per-batch worker count. State mutations all guarded by
a single lock; staging filenames carry a UUID prefix so concurrent
copies of identically-named source files don't overwrite each other.
Source picker in the modal lets the user choose which source to read
the tracklist from. Two endpoints feed it:
- `/api/library/album/<id>/reorganize/sources` — sources for THIS
album that are both authed AND have a stored ID. For the per-
album modal.
- `/api/library/reorganize/sources` — all authed sources globally.
For the bulk "Reorganize All" modal where per-album ID coverage
varies.
When the user picks a specific source, the orchestrator runs in
`strict_source=True` mode (no fallback chain) — picking Spotify means
"use Spotify or fail", not "use Spotify and silently fall back."
Preview endpoint shares the same planning logic as apply via
`preview_album_reorganize` — the destination path comes from the same
`_build_final_path_for_track` post-process uses, so what you see in
the preview is exactly what you get on apply.
Empty destination folders (from earlier failed runs OR from the
current run when post-process creates a dir then fails AcoustID)
get cleaned up after each successful run: walk up to the artist
folder from any successful destination, prune empty album-sibling
folders one level deep. Bounded scope = won't touch unrelated user
dirs.
Web_server.py shrinks by ~450 net lines. The endpoint handler is now
a thin wrapper that builds injected callables (path resolver, post-
process function, DB updater, empty-dir cleaner), spawns a thread
that calls `reorganize_album()`, and returns. All actual logic lives
in `core/library_reorganize.py` where it's unit-testable without
spinning up Flask.
Frontend cleanup: the per-call template input in both reorganize
modals (per-album and bulk) was redundant — the backend always uses
the configured global download template. Removed the input and the
variables-grid reference UI it was for.
39 new unit tests pin every contract:
- source resolution (no_source_id when album has none, fallthrough
chain when primary returns nothing, strict mode bypasses fallback)
- matcher scoring (exact / substring / multi-disc disambiguation /
smart-quote tolerance / dash-vs-parens / bonus-track substring /
Remix-vs-original differentiator rejection / "Real" doesn't false-
match "Real Real Real" / track-number-only no longer fires)
- file safety (DB-update failure leaves original in place, post-
process failure leaves original in place, post-process exception
caught and original preserved, success removes original AND
updates DB in the right order)
- sidecar handling (per-track .lrc/.nfo deleted on success, kept on
failure; album-level cover.jpg/folder.jpg cleaned only when
directory has no remaining audio)
- staging cleanup (recreated between tracks because post-process
nukes it, dir cleaned up on success AND on failure)
- destination-dir prune (empty siblings removed, real album with
files preserved, no recursive sweep)
- source picker (only authed-with-stored-ID sources for per-album,
all authed sources for bulk; strict mode doesn't fall back)
- concurrency (3 workers in flight, state stays consistent under
races, stop_check cuts off pending tasks)
- preview parity (preview produces same destination as apply for
multi-disc; ALBUM mode not SINGLE mode; unmatched/no-path tracks
surfaced with reasons)
Limitations (deliberate punts, NOT in this PR):
- Renamed local titles on multi-disc albums where track_number
also disagrees: matcher returns nothing (track is "not in
source"). Fixable by using duration_ms as a tertiary signal.
- Per-track in-modal source switching with per-album track-count
hints (would need a second API call before opening the modal).
- UI status panel on the artist page during a run — currently
just toasts. Documented as a follow-up PR.
Files:
- core/library_reorganize.py — new module: plan_album_reorganize,
preview_album_reorganize, reorganize_album, available_sources_for_album,
authed_sources, _score_candidate, helpers for staging/post-
processing/finalizing, sidecar + dest-dir cleanup
- core/metadata_service.py — no changes; reused get_album_for_source,
get_album_tracks_for_source, get_source_priority,
get_client_for_source
- web_server.py — three endpoints (preview / apply / sources GETs)
are thin wrappers; -450 net lines
- tests/test_library_reorganize_orchestrator.py — 39 tests covering
every contract above
- webui/static/library.js — source picker UI in both modals; dead
template input + variables-grid removed
- webui/static/style.css — dropdown option styling fix (white-on-
white was unreadable)
Reported on Discord by winecountrygames — his bug report named the
trigger button (Enhanced view → Reorganize All) and both symptoms
(multi-disc collapse, half-album skip), which let the diagnosis go
straight to the architectural problem.
Three bugs from kettui's follow-up review pass on the MusicBrainz
search PR, all fixed in one commit because they share UI context.
1. Missing artist images on MB artist results
MusicBrainz doesn't store artist images directly. My earlier commit
returned `image_url=None` on every artist result and trusted the
frontend's lazy-loader — but the lazy-loader's `/api/artist/<id>/image?
source=musicbrainz` endpoint had no handler for MusicBrainz, so it
silently returned None and the emoji placeholder stayed.
Fix plumbs the artist name through:
- `renderCompactSection` stashes `data-artist-name` on artist cards.
- `search.js` and `downloads.js` lazy-loaders pass `name=<artist>` as a
query param.
- `/api/artist/<id>/image` accepts an optional `name` param.
- `metadata_service.get_artist_image_url` has a new `musicbrainz`
branch: since MB has no artist art, it searches fallback sources
(iTunes/Deezer by configured priority) for the artist name and
returns the first image found.
Verified live — Metallica/Kendrick Lamar/Daft Punk all resolve to
Deezer artist images via the name lookup.
2. total_tracks off-by-one on tracks with a release
`_recording_to_track` initialized `total_tracks = 1` and then summed
media track-counts on top. For an 11-track album, it reported 12. An
adapter-level regression introduced when the recording-projection
helper was extracted during the main MB refactor.
Fix: initialize at 0, sum normally. Standalone recordings with no
release (can happen for uncredited remixes etc.) still report 1 via
an explicit fallback — so the existing "single track" case isn't
broken.
3. "Artist Album Title" queries buried specific albums in the
discography list
Bare-name queries like "The Beatles Abbey Road" used to resolve "The
Beatles" as the artist and then browse their full discography — Abbey
Road was buried alphabetically among 200+ releases instead of being
the top result.
Fix adds a title-hint extractor. When the query starts with the
resolved artist name followed by more words, the trailing portion is
treated as a title hint. Browse results are filtered to those whose
release-group title contains the hint. If the filter matches nothing,
falls back to text-search with the hint as the title (the "keep the
old split-by-whitespace fallback" path kettui called for). If text-
search also misses, shows the full discography rather than nothing.
10 new tests in tests/test_musicbrainz_search.py (46 total):
- Title-hint extractor: basic match, case-insensitive, whitespace
tolerance, bare-artist-no-hint, artist-not-prefix-no-hint, word-
boundary required (no false splits on "Metallicasomething").
- Browse filtering by title hint.
- Text-search fallback when the title hint matches nothing in browse.
- Bare-artist queries return the full discography unfiltered.
- total_tracks for single-release, multi-disc, and no-release cases.
Cin flagged that Soulseek was always rendered as configured in the
source picker, even on dev instances with no slskd set up — letting
users click it and fire searches that could never succeed.
Three coordinated changes:
1. web_server.py SERVICE_CONFIG_REGISTRY: add Soulseek entry requiring
`slskd_url`. /api/settings/config-status now reports its real state
alongside every other service.
2. shared-helpers.js _ALWAYS_CONFIGURED_SOURCES: drop 'soulseek'. The
set is now just MusicBrainz + YouTube Music Videos (sources that
genuinely don't need user creds). Soulseek goes through the normal
config-status code path.
3. shared-helpers.js openSettingsForSource: special-case Soulseek to
route to Settings → Downloads tab (where slskd URL field lives,
gated behind the download-source-mode dropdown) and scroll to the
#soulseek-url input. Every other source still routes to Connections
and scrolls to its .stg-service card. Without this, Soulseek's
"click to configure" landed on a Connections card that doesn't
exist (Soulseek's URL/key fields are scoped to the download-source
selection on the Downloads tab).
The hourly `clean_search_history` automation was crashing with
`'DownloadOrchestrator' object has no attribute 'base_url'`. The guard
was written before the orchestrator refactor — `soulseek_client` is now
a DownloadOrchestrator that wraps individual download clients, with the
real Soulseek client sitting at `.soulseek`.
Two other call sites in web_server.py (lines 2634, 3092) already used
the correct `soulseek_client.soulseek.base_url` pattern with a getattr
guard. This call site was missed during the refactor.
Fix: reach through the orchestrator the same way the other sites do.
When a completed download's track_info has neither an `id` field nor a
`wishlist_id`, Methods 1-3 of _check_and_remove_from_wishlist() all skip
without defining `wishlist_tracks`. Method 4 (fuzzy match) then hits
`if not wishlist_tracks:` and raises UnboundLocalError, which the call
sites catch + log but silently skip the wishlist removal for that track.
Path became more common after the batch-queue-system refactor started
routing non-Spotify-id completions (e.g. discover sync tracks downloaded
under a non-Spotify primary source) through the same completion handler.
Fix: initialize `wishlist_tracks = []` at the top of the try block so
Method 3's reassignment still works and Method 4's `if not wishlist_tracks`
guard always has a defined value to test.
Credit to RENOxDECEPTION (JohnBaumb) for pinpointing the variable-scope
issue during PR #357 testing.
JohnBaumb's review: "If we're going to refactor the web_server.py soon,
might as well start moving stuff away from web_server.py in our PRs.
_build_source_only_artist_detail, make it a module, it's perfect."
This continues the pattern the prior commit started with the source-ID
lookup helpers: move the pure data-building logic to a side-effect-free
core module, leave a thin wrapper in web_server.py that bridges the
Flask response and the module-global clients.
**core/artist_source_detail.py** — pure function that takes the artist id,
name, and source plus dependency-injected per-source clients (spotify,
deezer, itunes, discogs) and a Last.fm API key. Returns
(payload_dict, http_status) so it isn't coupled to Flask.
**web_server.py wrapper** — builds the client bag from the module globals
(checks Spotify auth, constructs the Discogs client from the configured
token, reads the Last.fm API key) and wraps the core return in jsonify.
147 lines of logic go away from web_server.py; the 24-line wrapper is
purely glue.
**tests/test_artist_source_detail.py** — 21 focused tests covering the
response envelope, the source-specific ID-field stamping for all six
supported sources, the dedup_variants=False contract (the behaviour
that originally motivated the split of MetadataLookupOptions), per-source
genre/follower extraction with safe handling of missing or throwing
clients, and the Last.fm enrichment branch including the no-key and
error-path cases. Runtime 0.26s.
Cin's review note: typing artist_name as plain `str` forced callers
that didn't have a name to pass `""` as a placeholder, which leaks the
parameter's emptiness contract into every call site and reads badly in
tests. Switching to `Optional[str] = None` lets callers omit it.
The function body's `if artist_name and active_server:` check already
handles None and "" identically, so no body changes were needed. Tests
that previously passed `artist_name=""` drop the argument; one new test
covers the omitted-arg path explicitly.
The web_server.py wrapper takes the same default for symmetry.
Cin pointed out that the prior version of test_artist_source_lookup.py
AST-parsed web_server.py to verify a constant and to string-match a
function's response keys. That was a workaround for the fact that
web_server.py can't be imported at test time (it boots Spotify,
Soulseek, Plex, etc.) — the right answer is to move the logic into a
side-effect-free module so it can be imported and tested directly.
This commit:
- adds core/artist_source_lookup.py containing the SOURCE_ID_FIELD
map, the SOURCE_ONLY_ARTIST_SOURCES set, and find_library_artist_for_source
- replaces the inline definitions in web_server.py with imports +
a thin wrapper that injects the active media server
- rewrites the tests to import from the core module directly:
* mapping correctness is now a plain equality assertion
* lookup behaviour is exercised against a real MusicDatabase
* the AST parse and the string-matching contract test class are
gone
- drops the _build_source_only_artist_detail contract test entirely
(the weakest of the four — it was just string-matching the function
body); when that function moves to core/ it can get a real
behavioural test alongside.
Test runtime drops from ~161s to ~5.8s. All 18 tests pass.
The library-enrichment query inside /api/watchlist/artist/<id>/config
queries the `artists` table but used the column names from the
watchlist_artists table:
WHERE spotify_artist_id = ? OR itunes_artist_id = ?
OR deezer_artist_id = ? OR discogs_artist_id = ?
The `artists` table actually uses `deezer_id` and `discogs_id` for
those two columns (only `watchlist_artists` uses the `_artist_id`
suffix). The mismatch threw `no such column: deezer_artist_id` on
every config GET, which was caught by the surrounding try/except and
logged — releases came back empty and Spotify/genres etc. fell back
to defaults.
Visible side effects: the request that LOOKED slow ('1420.2ms') and
the recurring ERROR line in app.log every time a watchlist artist
overlay opened.
Watchlist-config GET now returns proper banner_url / summary / style
/ mood / label / genres for Deezer- and Discogs-source artists too.
The other watchlist queries in this endpoint (42302 / 42315 / 42379)
correctly target watchlist_artists and stay as-is.
Three issues from screenshots:
1. .collection-overview was hidden for source artists (CSS rule too
aggressive). It actually renders fine — just shows 0/N "missing"
for each release type, which is useful info. Removed from the hide
rules.
2. #artist-hero-sidebar (Top Tracks "Popular on Last.fm") was also
hidden. The renderer (_loadArtistTopTracks in library.js) already
fetches by artist name via /api/artist/0/lastfm-top-tracks, so it
works for source artists too. Removed from the hide rules.
3. Clicking a source-artist result for someone you ALREADY have in
the library was loading the bare source view instead of the
library view (bug). Backend now does a "library upgrade" lookup
in get_artist_detail: when the direct ID lookup misses but a
source param is provided, search the artists table by the source-
specific ID column (deezer_id / spotify_artist_id / etc.). If a
match exists, use that library PK and the rest of the library
path runs normally — owned releases, enrichment, completion
bars, all the goodies you'd see if you'd clicked from Library.
Falls back to a name match within the active server, then to the
source-only response if nothing matches.
The remaining library-only items (artist-enrichment-coverage, Radio
button, Enhance Quality button) stay hidden for source artists since
they all require owned tracks.
LastFMClient() with no args has no api_key -> get_artist_info silently
returns None -> source artists never see bio/listeners/playcount even
though my previous commit was supposedly fetching them.
Now reads config_manager.get('lastfm.api_key') and only attempts the
enrichment when the key is configured. Users without Last.fm credentials
in Settings simply get image+name+badges+genres on source artists
(everything except bio + stats), which is fine.
Source artists landing on /artist-detail were rendering an almost-blank
hero — image + name + a tiny Download button — because the backend
response only had {id, name, image_url, server_source: null, genres: []}.
The library.js renderers do their best with what they have, and that
wasn't much.
Backend changes (_build_source_only_artist_detail):
- Set the source-specific ID field (deezer_id / spotify_artist_id /
itunes_artist_id / discogs_id / soul_id / musicbrainz_id) on
artist_info so the corresponding service badge renders on the hero.
- Try the source's own get_artist_info / get_artist for genres +
followers (Spotify always; Deezer/iTunes/Discogs when available).
Spotify also fills image_url if metadata_service.get_artist_image_url
came up empty.
- Last.fm enrichment by artist name — bio + listeners + playcount +
lastfm_url. Mirrors what library artists get from the cached
enrichment workers but on demand for source artists.
- All enrichment lookups are wrapped in try/except so a 500 from any
one source doesn't break the whole response.
Frontend (library.js populateArtistDetailPage):
- Watchlist button now initialises for source artists too. Falls back
to artist.id + artist.name when there's no canonical Spotify
identity (which is the common case for non-library artists).
Discography dedup opt-out:
- Added dedup_variants flag to MetadataLookupOptions (default True so
library artists are unchanged). Source-only path now passes
dedup_variants=False so every "Deluxe Edition" / "Remastered" /
"Anniversary" variant the source returns is shown — matches the
inline /artists page behaviour the user was comparing against.
Result: source artists' hero now shows badges + bio + listeners +
playcount + watchlist button + genres in addition to image and name.
Discography lists every release the source returns, not the deduped
canonical view.
Part D + E of the deferred cleanup + the final version bump that
publishes the whole Search/Artists unification project.
Deletions:
- webui/static/artists.js (1903 lines) — removed entirely. The 2
remaining externally-referenced helpers (lazyLoadArtistImages +
showCompletionError) moved into shared-helpers.js first.
- webui/index.html — 140-line #artists-page HTML block and the
<script src="artists.js"> tag both removed.
init.js wiring:
- 'case artists:' removed from loadPageData switch (no page to init).
- navigateToPage top-level alias extended: 'artists' → 'search'
(same pattern as the existing 'downloads' → 'search' alias).
Legacy /artists bookmarks land on the unified Search page, the
natural place to find an artist now.
- _getPageFromPath now maps artist-detail → library as its parent
(was artists). Matches the existing library-nav-highlight at
init.js:2161.
Version bump:
- _SOULSYNC_BASE_VERSION 2.39 → 2.40.
- WHATS_NEW entries lose the 'unreleased' scaffolding and gain a
new top entry summarizing the unified artist-detail page + the
final artists.js retirement.
- version-info modal gets a 'Search & Artists Unification' section
at the top.
- The _getLatestWhatsNewVersion filter added during the unreleased-
tracking phase is rolled back — entries now display as soon as
they land in WHATS_NEW, matching the pre-unification behaviour.
Test suite:
- tests/test_script_split_integrity.py SPLIT_MODULES updated:
'artists.js' dropped, 'shared-helpers.js' added. escapeHtml's
cross-file dupe list entry updated to reference shared-helpers.
- 354/354 tests pass.
User-visible result after this commit:
- Sidebar: Search, Downloads, Discover, Library, Wishlist, etc. —
no more Artists entry.
- Click any artist anywhere: lands on the same /artist-detail page.
- Search page has a source dropdown; Soulseek is just another option.
- Legacy /downloads and /artists URLs alias to /search.
- Version button shows v2.3 (Docker major); "What's New" panel
opens to the unification summary.
Closes the project Cin requested in Discord. Future work: source-aware
/api/artist-detail could be extended to fall back through the whole
source priority chain when a specific source is given but returns no
discography. Not needed for the current flows.
Part A of the deferred unification cleanup. The standalone artist-
detail endpoint used to 404 whenever `artist_id` wasn't a local library
primary key, which is exactly what source artists (Deezer/Spotify/
iTunes/etc.) have. That forced the Phase 4a revert: source artists had
to use the inline Artists page because this endpoint couldn't handle
them.
New behaviour:
- Library PK path — unchanged. Existing callers see the same response.
- `/api/artist-detail/<id>?source=<src>&name=<name>` with source in
(spotify, itunes, deezer, discogs, hydrabase, musicbrainz) — when
the library DB lookup misses, synthesize a response by:
• fetching artist image via metadata_service.get_artist_image_url
with source_override (the helper already backing /api/artist/
<id>/image)
• fetching discography via metadata_service.get_artist_detail_
discography with MetadataLookupOptions(source_override=source,
artist_source_ids={source: artist_id})
• returning { success, artist: {id, name, image_url, server_source:
null, genres: []}, discography, enrichment_coverage: {} }
- Library PK missing AND no source — preserves the 404 (caller didn't
give enough info to fall back).
Frontend plumbing: library.js loadArtistDetailData now appends
?source=<src>&name=<name> to the fetch URL when
artistDetailPageState.currentArtistSource is set. The field is already
seeded by navigateToArtistDetail's third arg (added during the earlier
unification work), so no new state plumbing is needed.
populateArtistDetailPage gracefully handles the missing-library-data
case per earlier exploration — owned_releases empty is fine,
enrichment_coverage optional, spotify_artist_id optional.
Part B will re-route the source-artist callsites (Search / Discover /
Watchlist / etc.) back through navigateToArtistDetail so they actually
exercise this new fallback path.
Reverts the 2.40→2.49 version spam from this session — every phase
commit was bumping the display version when the whole Search/Artists
unification project should really be a single release.
Changes:
- _SOULSYNC_BASE_VERSION back to 2.39
- All session-level version-info sections consolidated — the endpoint
response is back to the pre-session 2.39 shape
- helper.js WHATS_NEW entries for 2.40–2.49 collapsed into a single
'2.40' block with one bullet per phase, marked unreleased
- _getLatestWhatsNewVersion / _showOlderNotes filter out entries
whose version is higher than the current build, so the 2.40 block
won't fire the 'new' badge or appear in the What's New panel until
we actually flip the build version
- Picks up the artist-detail back-button fix from the previous turn
(falls back to browser history when the user reached the inline
detail from outside the Artists page)
When the unification project is done, a single commit that bumps
_SOULSYNC_BASE_VERSION to 2.40 will publish the whole folded entry.
Phase 4a (9361c29) mistakenly routed every artist click to
navigateToArtistDetail, which fetches /api/artist-detail/<id>. That
endpoint only knows how to look up local DB primary keys. For source
artists (Spotify/Deezer/iTunes/etc.) the id is a metadata-source id,
not a library PK — so clicks 404'd out.
Library artists (db_artists section in search results, library page
clicks, stats links, media player) continue to go to the standalone
/artist-detail page as before. Source artists now route back to the
Artists page's inline view via selectArtistForDetail, which calls
/api/artist/<id>/discography with a source param — the endpoint that
actually handles non-library IDs.
Reverted 7 migration points:
- search.js: Enhanced Search source-artists onClick
- downloads.js: global widget _gsClickArtist non-library branch
- downloads.js: _navigateToArtistFromModal fallback
- discover.js: viewRecommendedArtistDiscography
- discover.js: viewDiscoverHeroDiscography
- discover.js: 'Your Artists' card name-click inline HTML
- discover.js: 'Your Artists' info-modal 'View All' button
- discover.js: artist-map context menu
- discover.js: genre-deep-dive artist click
- api-monitor.js: watchlist artist discography view
Phase 4a's goal of "one artist page for everything" is deferred —
it needs backend work on /api/artist-detail to accept a source param
and fall back to metadata-source lookup when the local DB lookup
fails. Keeping the signature extension on navigateToArtistDetail
(source parameter) in place for when that lands.
Phase 4c of the Search/Artists unification — docs-only cleanup.
The click-for-help system and the 'Your First Download' guided tour
referenced elements that no longer exist (the Basic/Enhanced toggle,
the embedded download-manager toggle, the active/finished queue
panels). Updated annotations + tour steps to match the current UI.
- New annotation for .search-source-picker-container (the dropdown)
- Removed 6 annotations for deleted elements
- 'first-download' tour now walks users through the source picker
and uses page: 'search' (PAGE_TOUR_MAP accepts both 'search' and
the legacy 'downloads' id so older bookmarks still match)
- Retired the 'artists-browse' standalone tour — no sidebar entry
- Dropped the dead #finished-queue detection in the setup milestone
check (the dashboard stat card is the single source of truth)
Phase 4b of the Search/Artists unification. Cin flagged that 'Artists'
in the sidebar read like a library section but was actually a
dedicated artist-search page, duplicating what unified Search already
does. Removed the sidebar entry so users funnel through Search.
- Sidebar Artists button gone
- 'Browse Artists' on empty Watchlist now opens Search
- 'View artist from Wishlist' opens Search pre-filled with the name
- Profile Home Page + Page Access drop the Artists option
artists.js stays on disk: it defines ~30 shared helpers used across
the app (escapeHtml, openDownloadMissingModalForArtistAlbum, service
status, download bubbles, image helpers) that library/discover/etc.
depend on. Wholesale deletion would orphan too much. The inline
Artists page and its selectArtistForDetail flow are still there —
just unreachable from the sidebar — so /artists deep links keep
working for bookmarks.
Phase 4a of the Search/Artists unification. The app had two artist-
detail implementations: the standalone page Library navigates to via
navigateToArtistDetail (its own route, deep-link support, highlights
Library in the sidebar), and an inline state inside the Artists page
reached via selectArtistForDetail. They rendered similar content but
were separate code paths and kept drifting apart (PR #356 just had
to fix source propagation in both).
Every external caller of selectArtistForDetail (9 sites across
api-monitor.js, discover.js, downloads.js, search.js) now calls
navigateToArtistDetail(id, name, source) directly. Removed ~63 lines
of the navigate-then-setTimeout-then-select dance. Source context
(Spotify/iTunes/Deezer/etc.) carries cleanly through via the new
third argument.
Artists sidebar entry, its inline search, and selectArtistForDetail
all still work — they just have no external callers. Phase 4b will
retire the sidebar entry and artists.js.
Phase 3c of the Search/Artists unification. The Search page carried
a second copy of the Download Manager (active + finished queues,
clear/cancel-all buttons) that was hidden by default and duplicated
the dedicated Downloads page. That duplicate is now gone.
Removed:
- Side-panel HTML block and the toggle button that showed/hid it
- ~290 lines of polling + render infra in downloads.js: loadDownloads-
Data, startDownloadPolling/stopDownloadPolling, updateDownload-
Queues, renderQueue, updateTabCounts/updateDownloadStats,
initializeDownloadTabs/switchDownloadTab, cancelDownloadItem,
clearFinishedDownloads, cancelAllDownloads, and the
activeDownloads/finishedDownloads globals
- initializeDownloadManagerToggle and its call from init.js
- Stopped hitting /api/downloads/status every second on the Search
page (the dedicated Downloads page already polls its own view)
CSS grid for the Search page collapsed from '1fr 370px' to '1fr' now
that the right panel is gone. Unused .controls-panel__* / .download-
manager__* / .downloads-side-panel CSS rules kept in place — harmless,
can be pruned later.
Phase 3b of the Search/Artists unification. The Search page's
internal id was 'downloads', which clashed with the actual Downloads
page (id 'active-downloads') and confused anyone reading the code.
Renamed to 'search' across HTML, navigation, DOM selectors, and the
deep-link route list.
Backwards compat: navigateToPage('downloads') aliases to 'search'
at the top of the function; /downloads URL still serves index.html
and the client router resolves the page correctly; profile ACL
checks accept both 'search' and 'downloads' so existing profiles
with 'downloads' in allowed_pages keep working without migration.
Sidebar label unchanged. Zero visual change — pure internal tidy.
Phase 3 of the Search/Artists unification. The Search page's two-mode
toggle is replaced by a single 'Search from' dropdown: All sources
(Auto), Spotify, Apple Music, Deezer, Discogs, Hydrabase, MusicBrainz,
or Soulseek (raw files). Auto keeps today's fan-out behavior for
backwards compatibility; picking a specific source hits only that
provider. 'Soulseek' routes to the raw-file basic section, so one
picker covers both old modes. Loading text and the enhanced fetch
now respect the selected source. Zero API changes — uses the source
param added in 2.40 and the shared fetch helper from 2.41.
Phase 2 of the Search/Artists unification: the Search page dropdown
and the global spotlight widget both POST to /api/enhanced-search
with identical boilerplate. Extracted into enhancedSearchFetch() in
search.js (loaded before downloads.js). Both callers migrated. Zero
UX change — purely sets up Phase 3 to wire a source picker in one
place instead of two.
Phase 1 of the Search/Artists unification project: the endpoint now
accepts an optional `source` (spotify, itunes, deezer, discogs,
hydrabase, musicbrainz) so callers can target a single metadata source
instead of always fanning out. Omitted or `auto` preserves current
multi-source behavior — no existing callers break. Cache keys include
the source tag so per-source and fan-out results don't collide.
User reported searching "Maduk - Leave A Light On" on Tidal silently
downloaded Tom Walker's completely different song of the same name, then
embedded Maduk's metadata into Tom Walker's audio. Three layers of
defense all failed permissively. Two of them are fixed here; the third
(score formula weights) was left alone since these two together cover it.
Layer 1 fix — candidate artist gate (web_server.py:27782)
Old: `if _best_artist < 0.4 and confidence < 0.85: continue`
New: `if _best_artist < 0.5 and confidence < 0.85: continue`
SequenceMatcher returns exactly 0.400 for "maduk" vs "tom walker"
(5-char vs 10-char strings with coincidental char matches), which
slipped past the strict `< 0.4` check. The word-boundary containment
check earlier in the function already short-circuits legitimate
formatting variations to sim=1.0, so falling to SequenceMatcher means
strings are genuinely different. 0.5 closes the fencepost AND gives
a small safety buffer.
Layer 3 fix — AcoustID verification (acoustid_verification.py:316)
When title matches but artist doesn't AND expected artist isn't found
anywhere in AcoustID's returned recordings:
Old: always SKIP (let file through, assume cover/collab)
New: FAIL if artist_sim < 0.3 (clear mismatch)
SKIP if artist_sim >= 0.3 (ambiguous — cover/collab/formatting)
The 0.3 cutoff catches hard mismatches like Maduk/Tom Walker (sim ~0.2)
while preserving benefit-of-the-doubt for borderline artist formatting
differences. Legitimate covers and collabs where the expected artist
appears anywhere in AcoustID's recordings still PASS via the existing
secondary-match loop above.
Both fixes are defense-in-depth — either alone would have caught this
bug. Together they close the pre-download AND post-download gaps.
All 292 tests pass. Version bumped to 2.39 with changelog entries.
New smart template variable that emits "CD01" / "CD02" etc. in filenames
on multi-disc albums, and expands to empty string on single-disc albums
so mixed libraries don't end up with "CD01" on every single.
Template behaviour:
- total_discs > 1 -> "CD{disc:02d}" (zero-padded, CD prefix)
- total_discs <= 1 -> empty string
- Both $cdnum and ${cdnum} bracket form supported
- Empty value collapses cleanly via existing double-dash regex plus new
leading-dash cleanup pass
Wiring:
- _apply_path_template in web_server.py (download pipeline)
- _apply_path_template in core/repair_jobs/library_reorganize.py
(Reorganize repair job)
- total_discs added to every album-mode template context:
* download pipeline album branch (uses resolved total_discs even for
single-track downloads from search)
* per-album Reorganize preview + apply endpoints (pre-scan all track
tags once, take max disc_number)
* Library Reorganize repair job (already had album_total_discs map,
just added to context dict)
Leading-dash cleanup added to _get_file_path_from_template (web_server)
and _build_path_from_template (library_reorganize) so templates like
"$cdnum - $track - $title" don't leave "- 05 - Title" on single-disc
albums.
UI:
- Template hint in Settings -> File Organization documents $cdnum
- Template validation variable list includes $cdnum
- Reorganize modal variable reference shows $cdnum with example "CD01"
Verified:
- Multi-disc disc 1 -> "CD01 - 05 - Track"
- Multi-disc disc 2 -> "CD02 - 05 - Track"
- Single-disc -> "05 - Track" (no leading dash)
- Templates without $cdnum behave unchanged
- 276/276 tests pass
Three closely-related changes bundled together. The UI work exposed the
backend bug when I tried to cancel a Deezer download and saw it marked
cancelled in the DB but continuing in the background.
Backend — cancel_task_v2 orchestrator dispatch fix:
The slskd-specific cancel block was written back when soulseek_client
was a raw SoulseekClient. It was later swapped to DownloadOrchestrator
(which doesn't expose .base_url / ._make_request), so the first
diagnostic log line crashed with AttributeError. The outer try/except
swallowed it, leaving streaming downloads (YouTube / Tidal / Qobuz /
HiFi / Deezer / Lidarr) running in the background after the user
clicked cancel.
Replaced the ~80-line block with a single
soulseek_client.cancel_download(download_id, username, remove=True)
call — the orchestrator's dispatch picks the right client by username,
same path /api/downloads/cancel already uses successfully.
Per-row cancel button (fancy):
Circular X button on .adl-row for rows in active or queued state.
Hidden by default (opacity 0, translateX + scale), fades in + settles
on .adl-row:hover with a cubic-bezier overshoot. Own :hover gives a
1.12x scale pop and brighter red glow. Touch devices (@media
(hover: none)) keep it visible.
Backend: surfaced playlist_id in /api/downloads/all items so the
frontend can hit cancel_task_v2 without a second lookup. Frontend:
adlCancelRow(btnEl, playlistId, trackIndex) with double-click guard
via data-cancelling + adl-row-cancel-pending class.
Cancel All header button:
Red-themed button next to "Clear Completed". Only visible when any
task is in downloading / searching / post_processing / queued state —
auto-hides the moment the last one finishes. Confirm dialog shows
"Cancel N tasks across M batches?". Iterates _adlBatches, calls
/api/playlists/<batch_id>/cancel_batch sequentially (same endpoint
each modal's "Cancel All" and the per-batch-card cancel use). Disables
during the loop, mixed/success/error toast based on result.
All 276 tests pass.
Download-status meta enrichment only checked spotify_album.images[0].url
for the card artwork. That's the Spotify-API shape, but the context
builder for wishlist and manually-fixed tracks populates spotify_album
with image_url (singular string) and no images array. Result: those
tracks downloaded and post-processed fine (different path) but the
downloads page showed a placeholder note icon.
Enrichment now falls through three spots before giving up:
1. spotify_album.images[0].url (Spotify-originated)
2. spotify_album.image_url (wishlist / fixed discovery)
3. track_info.image_url (some discovery flows)
Pure read-side fix — no changes to the context builder, so existing
behaviour for Spotify-primary users is unchanged.
Companion fix to the provider-hardcode bug (6ceedc8). The cache
matched_data built by the 5 update_match / fix endpoints was dropping
image_url and album.images when album came back as a bare string —
common for Deezer and iTunes search results. Cache hits on re-discovery
then produced downloads with no artwork.
Each save site now carries image info through:
- album_obj gets image_url + images:[{url}] populated from spotify_track.image_url
- matched_data adds top-level image_url for pipeline consumers that check there
- Works for both dict-shaped album (Spotify) and string-shaped album (Deezer/iTunes)
Mirrors the handling already present in _build_fix_modal_spotify_data for
the in-memory result['spotify_data'] — explains why the UI showed art fine
during the fix modal but the cached entry lost it after restart.
save_discovery_cache_match uses INSERT OR REPLACE, so existing bad cache
entries refresh when the user re-fixes the track. No manual cache clearing
needed.
Added to 2.38 changelog (same round of discovery-fix work).
Five update_match endpoints hardcoded the provider as 'spotify' when
saving manual fixes to the discovery cache, but the re-discovery worker
queries the cache with _get_active_discovery_source() — the user's
actual primary. If the primary was Deezer/iTunes/Discogs/Hydrabase, the
provider column never matched, so every manual fix looked like it
vanished on restart.
Replaced 'spotify' with _get_active_discovery_source() at all 5 sites:
- Tidal update_match (web_server.py:34569)
- Deezer update_match (web_server.py:36235)
- Spotify Public update_match (web_server.py:37084)
- YouTube update_match (web_server.py:38037)
- Discovery Pool fix (web_server.py:49787)
Now symmetric with how the auto-discovery workers already save. Spotify-
primary users see no change (the hardcoded value matched their source).
Version bumped to 2.38 with changelog + version-info entries.
Two bugs reported in issue #320:
1. Auto-watchlist scan bypassed Global Override settings.
scan_watchlist_profile applied _apply_global_watchlist_overrides, but
the scheduled auto-scan called scan_watchlist_artists directly —
bypassing the override. Users who unchecked "Albums" or "Live" under
Watchlist → Global Override still saw full albums and live tracks
added during nightly scans (per-artist defaults, which include
everything, won).
Moved override application into scan_watchlist_artists itself so
every entry point respects it. scan_watchlist_profile now forwards
the apply_global_overrides flag through to avoid double-application.
2. is_live_version (watchlist + discography backfill) and
live_commentary_cleaner's content patterns used bare \blive\b, which
matched verb uses like "What We Live For" by American Authors,
"Live Forever" by Oasis, "Live and Let Die" by Wings.
Tightened the live patterns to require clear recording context:
(Live) / [Live Version] / - Live / Live at|from|in|on|version|
session|recording|performance|album|show|tour|concert|edit|cut|take
/ In Concert / On Stage / Unplugged / Concert.
Locked in 11 regression tests covering the reported false positives
(What We Live For, Live Forever, Living on a Prayer, Live and Let Die)
and the reported true positives (Dimension - Live at Big Day Out,
MTV Unplugged, etc.).
Version bumped to 2.37 with changelog entries.
Root-cause fix for "scanning 50 artists" then silence: when the master
repair worker was paused, force-run still kicked off _run_job but the
job's first wait_if_paused() blocked forever because is_paused was tied
to the master-enabled state. Force-run now bypasses master-pause —
scheduled runs still respect it.
Also fixes Fix All on discography findings doing nothing: the backend
bulk_fix_findings query had a fixable_types allowlist that excluded
missing_discography_track (and acoustid_mismatch). Added both.
Backfill job rebuild:
- auto_add_to_wishlist opt-in setting — creates findings AND pushes to
wishlist during the scan
- 3-option fix dialog (Add to Wishlist / Just Clear / Cancel) on single
Fix, Bulk Fix selection, and Fix All (page-level)
- Fix All "Just Clear" path uses the clear endpoint with job_id filter
instead of the generic "may delete files" bulk-fix warning
- Batched in-memory matching using get_candidate_albums_for_artist +
get_candidate_tracks_for_albums (same fast path the Library pages use)
- Rich album context per finding (id, name, album_type, release_date,
images, artists, total_tracks) — flows through the wishlist pipeline
so auto-processor classifies each track into the right cycle
(albums vs singles) and post-processing gets correct folder/tags/art
- Per-artist progress logs [N/50] Scanning ArtistName
- Default interval 24h (was 168h); all release types default on; settings
reordered with _section_* group headers (Core / Release Types /
Content Filters)
Repair settings UI:
- Generic _section_<name> key convention renders as an uppercase group
divider in the settings panel — any job can opt in
- .repair-setting-row gets a dashed bottom border so label↔toggle pairing
is visually clear
- _prettifyRepairSettingKey fixes acronym capitalization (EPs, not Eps)
Version bumped to 2.36 with changelog entries.
User reported: after clicking Fix on a Not-Found discovery track and
picking a replacement in the fix modal, the resulting download card had
no cover image, and the track seemed to still behave like a Wing-It
stub. Both suspicions were correct. Three compounding bugs:
1. /api/spotify/search_tracks returned only id/name/artists/album/
duration_ms — no image_url — unlike the sibling /api/itunes/ and
/api/deezer/ endpoints which include image_url. The fix modal had
no image data to work with when users searched via Spotify.
2. Frontend selectDiscoveryFixTrack discarded any image info it did get
and posted the same minimal shape to the backend.
3. All 7 backend discovery/update_match endpoints built
result['spotify_data'] with 'album' as a bare string (track.album
which is just the album name). The download pipeline expects
spotify_album to be a dict with image_url or images[].url — a string
yields blank cover art. Normal discovery workers already build album
as a rich dict; the fix-modal path was the anomaly.
4. Bonus: result['wing_it_fallback'] was never cleared on manual match.
Tracks fixed after the auto Wing-It fallback kept the flag set, so
downstream code checking it treated them as wing-it even though the
user picked real metadata.
Changes:
- New helper _build_fix_modal_spotify_data(spotify_track) in web_server.py
near _build_discovery_wing_it_stub. Handles both string and dict album
inputs, normalises to a dict with image_url and images populated when
the payload carries one. Matches the shape produced by normal discovery
so downstream code is happy on both paths.
- /api/spotify/search_tracks now returns image_url (parity with iTunes
and Deezer endpoints).
- All 7 discovery/update_match endpoints (youtube, tidal, deezer,
spotify-public, listenbrainz, beatport — 6 via the identical pattern
plus the listenbrainz variant with its None branch) now:
* use the helper to build spotify_data (album as dict + top-level
image_url)
* explicitly set result['wing_it_fallback'] = False
- selectDiscoveryFixTrack forwards track.image_url in the POST body and
mirrors the helper's output in the local state update so the UI
reflects the same shape immediately.
Audit: every downstream reader of spotify_data['album'] is already
dict-or-string tolerant (isinstance checks at lines 2073, 2158, 25844,
28938, 29011, etc.) so promoting album from string to dict is safe.
Normal discovery already sets it as a dict, so we're moving the fix-
modal path to match the existing majority case.
Full suite stays at 263 passed. Ruff clean.
User reported pausing the Spotify/Last.fm/Genius enrichment worker via the
dashboard bubble would silently turn back on "by itself". Real cause was
a race in the download-yield auto-pause/resume loop (_emit_enrichment_worker_stats_loop):
1. Download starts. Loop sees worker running, auto-pauses it, adds its
name to _download_auto_paused.
2. User clicks the enrichment bubble to pause — already paused visually,
but they want it to STAY off. Pause endpoint sets config_manager
'_enrichment_paused' to True and calls worker.pause() — but does not
remove the name from _download_auto_paused.
3. Download finishes. Loop sees 'not downloading and name in
_download_auto_paused' and blindly flips w.paused = False,
overriding the user's explicit pause. Config still says paused,
but the worker is actually running.
Two defensive fixes:
- Auto-resume block now checks the user's persisted config intent before
flipping the worker on. If {name}_enrichment_paused is True in config,
the name is dropped from _download_auto_paused without touching
w.paused — user's pause stays honored.
- Pause endpoints for spotify-enrichment, lastfm-enrichment, and
genius-enrichment now also discard from _download_auto_paused so a
stale marker can't trigger this race again.
Both together mean the auto-pause loop can no longer override a manual
pause regardless of ordering.
Full suite stays at 263 passed. Ruff clean.
Adds green/yellow header gradient on each service card showing whether the
user has filled in credentials, plus an expand-triggered verification layer
that surfaces working-or-not status inline.
Backend (web_server.py):
- SERVICE_CONFIG_REGISTRY mapping each of the 11 services in Connections to
its config requirements. Supports required-keys, always-green, any-of,
and custom-check semantics (Tidal uses token-file check, Qobuz accepts
either email/password OR cached auth token).
- _is_service_configured(service) — cheap config presence check, no APIs hit.
- GET /api/settings/config-status — returns {service: {configured}} for all
services in one call. Drives the page-load gradient.
- POST /api/settings/verify — takes {services: [...]}, runs
run_service_test per service, caches results 5 min in-memory, parallelizes
with ThreadPoolExecutor(max_workers=3) to avoid self-rate-limiting. Query
param ?force=true busts cache.
- Added verify branches for iTunes, Deezer, Discogs, Qobuz, Hydrabase in
run_service_test (previously missing — these services couldn't be tested).
HTML (webui/index.html):
- data-service="..." on all 11 .stg-service containers so JS can map card
to backend service name.
CSS (webui/static/style.css):
- .status-configured gradient (subtle green, left-to-transparent fade)
- .status-missing gradient (yellow, same shape)
- Spinner badge in header for .status-checking state
- "Testing connection…" status line style inside panel body
- Red warning bar style for verify failures at top of expanded panel
- Brand dot now glows always (was only glowing when expanded); hover and
expand states intensify the glow progressively.
JS (webui/static/script.js):
- applyServiceStatusGradients() fetches config-status and applies
green/yellow class per card. Called on Connections tab activate + after
any settings save.
- _stgVerifyServices(services, {force}) — batch verify POST, tracks
in-flight state, renders spinners/status lines/warnings per service.
- toggleStgService() fires single-service verify when a card is expanded
(not on collapse). Skipped if a verify is already in flight for that
service.
- toggleAllServiceAccordions() fires one batched verify for all 11 services
when "Expand All" is clicked; skipped on "Collapse All".
- _stgRefreshAfterSave() — after settings save, refreshes gradient (cheap)
and re-verifies only the cards the user currently has expanded (so
freshly-edited credentials show their new verify result immediately,
without re-pinging every service).
Failure UI: top-of-panel red warning bar with the error message (e.g.
"Discogs token rejected (HTTP 401)", "Hydrabase not connected…"). Removed
automatically on next successful verify.
No existing tests changed. Full suite stays at 263 passed. Ruff clean.
PR #340 added ruff to the build-and-test.yml CI gate, which surfaced
286 pre-existing lint errors. Left unfixed, every feature branch push
fails CI. This commit resolves all of them so CI goes green and
contributors can actually land work.
Auto-fixes (248 of 286): removed unused f-string prefixes (F541),
renamed unused loop control variables with underscore prefix (B007),
removed duplicate imports (F811).
Manually fixed 10 latent bugs ruff caught (all wrapped in try/except
today, silently failing):
- music_database.py: _add_discovery_tables() called undefined
conn.commit() — would have crashed the iTunes-support migration
for existing databases. Now uses cursor.connection.commit().
- web_server.py settings GET: referenced undefined download_orchestrator
when it should be soulseek_client. Feature (_source_status on the
settings payload) was silently missing for UI auto-disable logic.
- web_server.py _process_wishlist_automatically: active_server
undefined in track-ownership check. Auto-wishlist was falling
through to the error handler and re-downloading owned tracks.
- web_server.py start_wishlist_missing_downloads: same active_server
bug in the manual wishlist path.
- web_server.py _process_failed_tracks_to_wishlist_exact: emitted
wishlist_item_added automation event with undefined artist_name
and track. Automation event silently never fired correctly.
- web_server.py discovery metadata enrichment: referenced cache
without calling get_metadata_cache() first. Track enrichment from
cached API responses was silently skipped.
- web_server.py Beatport discovery worker: wing-it fallback branch
used undefined successful_discoveries variable. Wing-it counter
never incremented correctly. Now uses state['spotify_matches']
consistently with the rest of the function.
- web_server.py _run_full_missing_tracks_process: stale import json
mid-function shadowed the module-level import, making an earlier
json.dumps() call reference an unbound local (F823).
- web_server.py discovery loop: platform loop variable shadowed
the module-level platform import (F402).
- core/watchlist_scanner.py: 7 lambda captures of loop variables
(B023 classic Python closure-in-loop bug) now bind at creation.
No existing tests had to change. Full suite stays at 263 passed.
- Move /api/artist/<artist_id>/image resolution into core.metadata_service.
- Resolve artist artwork through source priority, with explicit source/plugin overrides preserved.
- Keep Spotify call tracking inside the client layer to avoid double counting.
- Update similar-artist lazy loading to pass source context and add service coverage.
- Relocate the streamed MusicMap similar-artist flow out of web_server.py and into core.metadata_service.
- Match similar artists through the configured source-priority chain instead of assuming Spotify first.
- Add iTunes artwork fallback so streamed artist payloads still carry image_url when search results are sparse.
- Cover the new service behavior with tests.
Artist detail pages ran check_album_exists_with_editions and check_track_exists
per discography item, each firing 5+ title variations times 3 artist variations
of fuzzy LIKE searches plus fallback broad-artist queries. For a 30-album artist
that was ~450 SQL round-trips just to answer "which of these do I own."
Hoist the artist's library albums and tracks into memory once per request via
two new helpers — get_candidate_albums_for_artist and get_candidate_tracks_for_albums —
and thread them through as optional candidate_albums / candidate_tracks params on
check_album_exists_with_editions, check_album_exists_with_completeness,
check_track_exists, check_album_completion, and check_single_completion.
Batched path scores the same _calculate_album_confidence / _calculate_track_confidence
against the in-memory list, preserving Smart Edition Matching and accuracy.
Title-only cross-artist fallback still fires for collaborative-album edge cases.
None on either param preserves legacy per-item SQL behavior for unaffected callers.
Applied to both /api/library/completion-stream (library artist detail page) and
iter_artist_discography_completion_events (Artists search page). Timing logs
added to confirm the pre-fetch cost and loop elapsed time.
On a Kendrick page load, per-album resolution drops from ~8 seconds to under
the 50ms streaming sleep floor. Observed ~100x SQL reduction on the happy path.
The reorganize endpoints built a template context without albumtype,
so ${albumtype} silently fell through to the engine's hardcoded "Album"
default — EPs, singles, and compilations all landed in Albums/.
Wires albumtype through from the albums table's record_type column
(populated by Spotify/Deezer/iTunes enrichment workers) with track-count
fallback when record_type is missing. New helper mirrors the download
pipeline's classification logic so reorganize produces the same folders
as initial placement. Also handles Deezer's raw 'compile' value which
the Deezer worker writes directly without mapping.
- collapse old multi-line debug bursts into single structured rows
- remove leftover DEBUG-style prefixes from message text
- keep the app log readable without losing useful trace detail
If the application was using a non-standard location for app.log, the other logs would still go to the default location. Now everything goes under the same, configured folder
print calls only end up in stdout, so there will be no trace of them
once docker loses access to its own logs. Using the logger makes sure
that logs end up in the filesystem as well
New "Reorganize All" button in enhanced library artist header processes
all albums sequentially using the configured path template.
Version bumped to 2.35. Updated What's New modal with major features
(Discography Backfill, Multi-Artist Tagging, Enriched Downloads,
Template Delimiters, Reorganize All). Updated helper.js changelog
with all April 20 fixes and features.
Three new settings in Paths & Organization:
- Artist Tag Separator: choose comma, semicolon, or slash between artists
- Write multi-value ARTISTS tag: each artist as separate tag value for
Navidrome/Jellyfin multi-artist linking (FLAC ARTISTS key, ID3 TPE1
multi-value, MP4 multi-entry)
- Move featured artists to title: keep only primary artist in ARTIST
tag, append others as (feat. ...) in track title
All opt-in with defaults matching current behavior. Raw artist list
stored on metadata dict for tag writers to access without re-parsing.
The downloads page previously showed only title and source per download.
Now shows album artwork thumbnail, artist name, album name, source badge,
and quality badge (after post-processing). All metadata comes from the
existing matched_downloads_context — no extra API calls needed.
Falls back gracefully to title-only display when context metadata is
not available (e.g. orphaned Soulseek transfers with no task mapping).
Users can now append literal text to template variables using curly
braces: ${albumtype}s produces "Albums", "Singles", "EPs". Without
braces, $albumtypes was rejected as an unknown variable by validation.
Both syntaxes work: $albumtype (plain) and ${albumtype} (delimited).
Bracket vars are resolved first to prevent partial matching conflicts.
Validation updated for album, single, and playlist templates.
soulseek_client is the DownloadOrchestrator, not the SoulseekClient.
The base_url check needs to go through soulseek_client.soulseek.base_url
to reach the actual slskd client instance.
Playlist folder mode passed album_info=None to _enhance_file_metadata,
which crashed on the first .get() call. The try/except caught it and
called _wipe_source_tags, stripping ALL metadata from the file.
Now normalizes None album_info to empty dict at the top of the function,
and adds a cover art fallback that pulls the image URL from the
spotify_album context when album_info doesn't have one.
The download monitor and transfer cache were calling slskd every second
during active downloads regardless of whether Soulseek was configured.
Users not using slskd got ERROR log spam from failed connection attempts
to host.docker.internal:5030.
Now checks if soulseek is in the active download mode/hybrid order
before making any slskd API calls. Also calls non-Soulseek download
clients directly instead of going through the orchestrator (which
redundantly hit slskd just to discard the results).
Two fixes:
- Single-track downloads from search didn't know total_discs, so multi-disc
albums like HIStory never got Disc N/ subfolders. Now resolves total_discs
from the album's track listing (cached) or from disc_number > 1.
- Allow duplicate tracks setting was ignored during album download analysis.
Global per-track search found the track in any album and marked it "found",
skipping the download. Now when allow_duplicates is enabled for album
downloads, only checks ownership within the target album.
- Move album-track resolution into metadata_service
- Use the configured provider order instead of Spotify-first branching
- Switch the frontend to the unified /api/album/<id>/tracks endpoint
- Add tests for source-priority lookup, DB resolution, and formatting
iTunes API can return collection metadata without song tracks for
region-restricted albums. The _lookup fallback only checked if results
was empty, so a collection-only response was accepted and cached as
{'items': []}. All future lookups returned the cached empty result.
Three fixes:
- get_album_tracks now checks for actual song items and tries fallback
storefronts when only collection metadata is returned
- Skip cached results with empty items array (prevents stale cache hits)
- Backend returns descriptive 404 error, frontend surfaces it in toast
Track ownership: check-tracks endpoint now filters by album context
when provided, preventing false "Found" when a track exists in a
different album by the same artist (e.g. Thriller on HIStory).
Wing-it wishlist: manual "Add to Wishlist" button now skips wing-it
fallback tracks (wing_it_ ID prefix), matching the behavior of
failed download and failed sync paths.
Debug info: watchlist/wishlist/automation counts were always 0
because get_db() doesn't exist — fixed to get_database().
The /api/artist/{id}/album/{id}/tracks endpoint was hardcoded to use
spotify_client and returned 401 if Spotify wasn't authenticated,
even when the user's primary source was Deezer or iTunes. Now uses
the configured primary metadata source via _get_metadata_fallback_client
with Spotify as fallback. Also gives a clearer error message when
no metadata source is available at all.
Some metadata APIs return fewer or no results for all-lowercase
queries. Title-case the query when it's all lowercase before
sending to the API ("foreigner" → "Foreigner"). Mixed-case input
is left as-is. Confidence scoring still uses the original query.
Fixed 5 critical gaps in the download orchestrator where lidarr was
missing from client loops: get_all_downloads, get_download_status,
cancel_download fallback, clear_all_completed_downloads, and
cancel_all_downloads. Without these, lidarr downloads were invisible
to the UI, couldn't be cancelled, and accumulated in memory.
Also: error messages now visible in download list (appended to
filename on error state), removed "(Development)" label from UI.
- pass provider-specific artist ids into the source-priority discography lookup
- stop relying on the local library artist id when querying external metadata
- add a regression test for source-specific artist id resolution
- Stop passing in spotify_id as the id in the UI, use the actual db id instead
- Fixes an issue where albums for another artist would end up being returned for the actual searched artist
- Remove the redundant artist_id filtering code
- Fixes an issue where not-currently-owned albums would be filtered out from the results, even if they were successfully fetched from the configured metadata provider
M3U files were generated when the download batch completed but
before post-processing finished tagging and moving files. Paths
pointed to download locations instead of final library paths,
making every track show as missing. Now regenerates the M3U from
the backend batch completion handler after all post-processing
is guaranteed done, resolving real file paths from the library DB.
Skips overwrite if zero tracks resolve to avoid replacing a
partially-good M3U with an all-missing one.
The retag fix for AcoustID mismatches was only updating the DB
record (title, artist_id) without writing corrected tags to the
actual audio file. Users would click Fix, the finding disappeared,
but the file on disk stayed unchanged. Now writes title and artist
tags to the file via Mutagen after the DB update.
Also fixed artist INSERT missing server_source when creating a new
artist during retag — now uses the active media server value.
Auto-wishlist albums cycle was passing is_album=True to
_get_batch_max_concurrent which returns 1 for soulseek mode.
This restriction is for folder-based album grabs from a single
peer, not individual track downloads. Wishlist always does
single-track downloads regardless of cycle, so it should use
the user's configured concurrency setting.
_adlFetch() fetches /api/downloads/all?limit=300 then _adlUpdateBadge()
was counting active statuses from that truncated array, overwriting
the real server-side count maintained by WebSocket. Removed the
badge update from _adlFetch — the WebSocket status push already
keeps it accurate.
Fix modal results: sort standard album versions above live, remix,
cover, soundtrack, remaster, and deluxe variants so users see the
original studio track first instead of obscure versions.
Plex Find & Add: tracks were always appended to the end of the
playlist because addItems ignores position. Now moves the track
to the correct slot after adding via moveItem.
Discovery Fix modal search results now sort standard album versions
above live recordings, remixes, covers, soundtracks, remasters,
deluxe editions, and other variants. Fixes cases where searching
"Mother Danzig" returned a live version first, or "Even Flow Pearl
Jam" returned a soundtrack instead of the original from Ten.
Unmatch: found tracks in playlist discovery now have a red X button
to remove bad matches. Clears match data, sets back to Not Found,
persists in DB for mirrored playlists, and respects user choice on
re-discovery runs (won't re-match automatically).
Video naming: new path template in Settings with $artist, $title,
$artistletter, $year variables. Default unchanged ($artist/$title-video)
so existing Plex setups aren't affected.
slskd logs: Clean Search History automation skips when Soulseek is
not the active download source, eliminating connection error spam.
Video naming: new path template in Settings → Paths & Organization
with $artist, $artistletter, $title, $year variables. Default
unchanged ($artist/$title-video → Artist/Title-video.mp4) so
existing Plex setups aren't affected. Users can remove the -video
suffix or reorganize however they like.
slskd logs: the Clean Search History automation now skips when
Soulseek is not the active download source, eliminating noisy
connection error logs for users who don't use Soulseek.
When playlist discovery fails to match a track on any metadata API,
instead of marking it "Not Found" and excluding it from downloads,
automatically build stub metadata from the raw source title/artist
and include it in the download queue. Soulseek searches with the
raw data, post-processing enhances whatever it can find.
All 7 discovery workers updated: YouTube, ListenBrainz, Tidal,
Deezer, Spotify Public, Beatport, and automated mirrored playlists.
Amber "Wing It" badge distinguishes stubs from real API matches.
Fix button still available so users can manually find a proper match.
Wing It stubs persist in DB for mirrored playlists and are
re-attempted on future discovery runs. Failed wing-it downloads
skip wishlist per-track (checked by wing_it_ ID prefix) so real
matched failures in the same batch still go to wishlist normally.
soul_id.startsWith() threw TypeError for non-string values, crashing
the entire card rendering pipeline. Letter-specific filters worked
because the problematic artist wasn't in those filtered results.
Added String() wrapper on all 3 soul_id.startsWith calls and a
try-catch around individual card rendering so one bad card can't
take down the whole page.
- Flask catch-all route serves index.html for client-side paths, excluding api/static/auth/callback/status prefixes.- navigateToPage pushes history state so URL reflects current page.- popstate listener handles browser back/forward without reloading.- Initial load reads window.location to restore the page after refresh or direct link.- artist-detail and playlist-explorer fall back to parent pages since they need runtime context.
- move artist-detail discography resolution onto the shared source-priority metadata service
- keep the variant dedup helper in the UI-facing adapter
- pass the chosen source through completion checks
- add coverage for the new adapter and dedup behavior
The ID resolver tried int() conversion first, which fails for text-based
IDs from Navidrome/Jellyfin. Now tries direct string match first (works
for both text and integer IDs), then integer fallback, then source
columns. Also added discogs_id to source column search. Fixes#323
Track and album delete with file removal now also cleans up associated
lyrics sidecar files (.lrc synced, .txt plain) that share the same
base filename as the audio file. Fixes#322
Move completion checks into metadata_service and make them follow the configured metadata source priority.
Drop the old test-mode path, remove the web_server wrapper indirection, and keep artist inference on explicit release metadata instead of guessing from a track search.
Add coverage for the source-priority completion behavior and the safer artist-name handling.
_resolve_db_album_id was missing deezer_album_id from stored ID checks
and hardcoded Spotify for the name-based search fallback. When Spotify
was rate limited (common for new Navidrome users), no fallback was tried
and the album returned 404.
Now checks all stored IDs (spotify, deezer, itunes, discogs) in priority
order matching the active metadata source, and falls back through all
available sources for name-based search instead of only Spotify.
- Fix level filter showing nothing: now uses heuristic classification
for print() output (error/traceback/failed→ERROR, warn→WARNING, etc.)
in addition to exact logger format matching
- Speed up WebSocket updates from 2s to 0.5s polling
- Add search box with 300ms debounce — filters both initial load and live
- Use DocumentFragment for batch DOM appends (performance)
- Increase line cap from 1000 to 2000
- Backend search parameter support in /api/logs/tail
Terminal-style real-time log viewer with:
- Log file selector (app, post-processing, acoustid, source reuse)
- Color-coded log levels (DEBUG gray, INFO blue, WARNING yellow, ERROR red)
- Level filter buttons (All/Debug/Info/Warn/Error)
- Auto-scroll with toggle, copy and clear buttons
- Live updates via WebSocket (2s polling, pushes new lines)
- Initial load fetches last 200 lines via REST API
- 1000-line display cap with oldest lines trimmed
Also fixes Advanced tab settings (Discovery Pool, Security, etc.) being
hidden inside collapsed Library Preferences section body — misplaced
closing div caused them to be invisible.
New toggle in Settings → Library → Post-Processing: "Apply ReplayGain
tags after download". When enabled, analyzes loudness via ffmpeg's
ebur128 filter and writes track-level ReplayGain gain/peak tags.
Runs after metadata tagging but before lossy copy so both files get
the tags. Off by default — adds a few seconds per track.
Applied to both album and playlist/single download paths.
New core/genre_filter.py with ~180 curated default genres. When strict
mode is enabled in Settings → Library Preferences → Genre Whitelist,
only whitelisted genres pass through during enrichment. Junk tags from
Last.fm (artist names, radio shows, playlist names) are silently dropped.
Applied at all 10 genre write points: Spotify, Last.fm, AudioDB, Deezer,
Discogs, iTunes, Qobuz enrichment workers + post-processing genre merge
+ initial download artist/album creation.
Strict mode is OFF by default — zero behavior change for existing users.
First enable auto-populates the whitelist with defaults. Users can add,
remove, search, and reset genres via the Settings UI.
Per-artist log lines now show the full details string from the worker
(e.g. "5 albums, 0 new tracks (150 existing updated)") instead of
just "5 albums, 0 tracks". Finished message shows "library up to date"
when no new content is found instead of "0 successful, 0 failed".
Full Refresh now clears all soulsync library records and rebuilds from
file tags in the output folder. Reads tags via Mutagen, groups by
artist/album, creates DB records with stable IDs. Files stay in place.
Previously Full Refresh did nothing for standalone — just returned.
Dashboard scan polling checked for 'completed' but backend sets 'finished'.
Added 'finished' to the completion check so polling stops, button resets,
stats refresh, and toast fires correctly. Also fixed deep scan reporting
stale record removals as 'failed' instead of 'successful'.
Playlist and single track downloads pass None as album_info to
_enhance_file_metadata. The downstream _extract_spotify_metadata
called .get() on it without a null guard, crashing with AttributeError.
All user-facing labels, docs, help text, tooltips, error messages, and debug
info output updated. Backend config keys, variable names, actual path values,
and Docker volume mounts are completely unchanged — zero functional impact.
Users can now override which metadata provider (Spotify, Deezer, Apple Music,
Discogs) is used when scanning a specific watchlist artist for new releases.
The selector appears in the artist config modal and only shows sources the
artist has enrichment IDs for. Default behavior is unchanged — all artists
use the global metadata source unless explicitly overridden.
The gunicorn PR blocked direct Python execution with SystemExit.
Replaced with _DIRECT_RUN flag at top and startup block at bottom
so both paths work:
- python web_server.py (Werkzeug dev server, Windows compatible)
- gunicorn -c gunicorn.conf.py wsgi:application (production)
Switch the web UI from Werkzeug's built-in server to Gunicorn for a more stable production deployment path.
Keep a separate dev config so local runs still reload quickly, while the production path uses a dedicated WSGI entrypoint and cleaner startup behavior.
The main motivation is to reduce the websocket teardown noise and make the server behavior more predictable under the app's mostly background-driven workload.
Added -vn flag to all codec ffmpeg commands (MP3, Opus, AAC) to strip
video/image streams during conversion. Embedded cover art in FLAC
files caused ffmpeg to fail when the output muxer couldn't handle
the image stream, producing 0KB output files. Cover art is
re-embedded afterwards by Mutagen.
The dedup key (normalized_title, year) caused different albums from
the same year to collide when title normalization stripped too much.
The "prefer more tracks" logic then kept compilations over studio
albums.
Two fixes:
- Title similarity check: if normalized titles are <85% similar,
they're different albums, not variants — keep both
- Compilation deprioritization: studio albums win over compilations
and "best of" collections when they do collide
Move Hydrabase availability checks into metadata_service so source resolution owns the policy. Keep web_server delegating to the centralized helper and add tests for the enabled/disabled cases.
Move artist discography resolution into core metadata_service, introduce MetadataLookupOptions, and keep web_server focused on request handling. Add focused tests for the new service boundary and preserve current fallback behavior for now.
Delay alternate-source fan-out until the primary enhanced-search response arrives, and stagger those follow-up requests so they do not all compete at once. Also parallelize artist, album, and track lookups inside each metadata source request to shorten the time the UI thread spends waiting on remote APIs. This keeps the single-worker web UI more responsive under the app's chatty search flow.
New MusicBrainz tab in Enhanced and Global search — finds tracks and
albums on MusicBrainz's community database with Cover Art Archive
images. Covers obscure tracks that Spotify/Deezer/iTunes miss.
- core/musicbrainz_search.py: search adapter with Track/Artist/Album
dataclasses, Cover Art Archive integration, smart query parsing
- Albums deduplicated (keeps best version with date and art)
- No artist results shown (MusicBrainz has no artist images)
- Album detail with full tracklist for download modal
- Smart word-boundary splitting for queries without separators
- Global search results container widened from 620px to 920px
- UI version bumped to 2.32
SoulSync Standalone Library is now the first section in both the
version modal and What's New popup. Auto-Import section updated with
all improvements (recursive scan, singles, tag preference, AcoustID).
New Downloads & Soulseek section groups download-related improvements.
Recent Fixes cleaned up — feature items moved to proper sections.
Deep scan for standalone mode:
- Scans Transfer folder for all audio files
- Compares against soulsync DB records by file_path
- Moves untracked files to Staging for auto-import processing
- Removes stale DB records where files no longer exist
- Cleans orphaned albums and artists with no tracks
Incremental scan skips for standalone — library updates at download
time, no periodic scanning needed. Both changes are purely additive
and only activate when server_type is 'soulsync'.
All sync-related buttons hidden when active server is SoulSync
Standalone. Covers static buttons (querySelectorAll on status update)
and dynamic modal buttons (_isSoulsyncStandalone flag).
UI version bumped to 2.31 (Docker stays at 2.3).
Single track ownership check was calling check_track_exists without
server_source, matching against all servers instead of the active one.
Album and EP checks already passed server_source correctly — this was
the only missing spot. Affects all server types.
Previously reused existing plex/jellyfin artist IDs, causing soulsync
tracks to be invisible on the library page (filtered by server_source).
Now always creates soulsync-specific artist and album records with
server_source='soulsync', avoiding PK collisions with hash suffixes.
Fourth server option on the Connections tab with SoulSync logo and
'Standalone' label. Config panel shows Transfer folder path and
Verify Folder button. Test connection counts audio files in the
Transfer folder. Settings save/load properly detects soulsync toggle.
New 'soulsync' media server option manages the library directly from
the filesystem, bypassing Plex/Jellyfin/Navidrome entirely.
Two paths populate the library:
1. Downloads/imports write artist/album/track to DB immediately at
post-processing completion, with pre-populated enrichment IDs
(Spotify, Deezer, MusicBrainz) so workers skip re-discovery
2. soulsync_client.py scans Transfer folder for incremental/deep scan
via DatabaseUpdateWorker (same interface as server clients)
New files:
- core/soulsync_client.py: filesystem scanner implementing the same
interface as Plex/Jellyfin/Navidrome clients. Recursive folder scan,
Mutagen tag reading, artist/album/track grouping, hash-based stable
IDs, incremental scan by modification time.
Modified:
- web_server.py: _record_soulsync_library_entry() at post-processing
completion, client init, scan endpoint integration, status endpoint,
web_scan_manager media_clients dict, test-connection cache updates
- config/settings.py: accept 'soulsync' in set_active_media_server,
get_active_media_server_config, is_configured, validate_config
- core/web_scan_manager.py: add soulsync to server_client_map
Dedup: checks existing artist/album by name across ALL server sources
before inserting to avoid duplicates. Enrichment IDs only written when
the column is empty (won't overwrite existing data).
Race condition: scanner re-scanned folders while post-processing was
still moving files, causing partial matches and ghost failures. Now
tracks in-progress paths and skips them on subsequent scans.
Coverage penalty fix: individual tracks that match at 80%+ confidence
now auto-import even when overall album coverage is low (e.g. 2 of 18
tracks present). Previously low coverage killed the entire import.
Import page: stats bar, filter pills, Scan Now, Approve All, Clear
History (clears imported + failed), live scan progress.
Album delete now shows a smart delete dialog with two options:
- Remove from Library (DB only, files untouched)
- Delete Files Too (removes DB records AND deletes audio files from
disk, cleans up empty album folder)
Backend /api/library/album/<id> DELETE now accepts ?delete_files=true
parameter, resolves each track's file path, and removes files before
deleting DB records. Reports files_deleted and files_failed counts.
The ffmpeg failure log was truncated to 200 chars which only showed the
version banner, hiding the actual error message. Now strips the banner
preamble and shows up to 500 chars of the real error with exit code.
Also cleans up 0KB output files left behind when ffmpeg fails.
Soulseek results from "Various Artists", "VA", "Unknown Artist", and
"Unknown Album" folders are now rejected before scoring. These
compilation folders rarely contain properly tagged files for the target
artist.
Clearing the wishlist now also cancels any active wishlist download
batch and resets the auto-processing flag, so downloads don't keep
running after the source tracks are removed.
Split Downloads page into main list (left) and batch panel (right).
Each active batch gets a color-coded card with artwork thumbnail,
progress bar, per-track status with download percentages, and
expandable track list. Download rows get matching color indicators.
- Click batch name to open its download/wishlist modal
- Filter icon narrows main list to one batch with clear banner
- Collapsible panel toggle for full-width list view
- Completed batches fade out after 15 seconds
- 7-day batch history with source type color dots
- Artwork fallback shows colored initial when no art available
- Per-track progress: download %, spinner for searching, proc label
- source_page column on sync_history for UI origin tracking
- /api/downloads/all includes batch summaries and per-track progress
- /api/downloads/batch-history endpoint for history queries
- Responsive layout, overflow-x hidden to prevent scroll flicker
Priority 0 query (artist + album + title) was gated behind a download
mode check that excluded Soulseek, the source that benefits most from
it. Soulseek searches match against file paths where users organize as
Artist/Album/Track — without the album name, ambiguous artist names
could match wrong-artist results (e.g. "Bleakness" as an album folder
instead of an artist). Removed the mode gate so all sources get the
most specific query first.
Dashboard stats (every 10s) and download status endpoint were
unconditionally calling slskd transfers/downloads API, causing
connection timeout spam for users with a slskd URL configured but
using YouTube/Tidal/etc as their download source. Now checks both
download source mode and status cache before making the API call.
Adds April 17 entries for Auto-Import, Wishlist Nebula, automation group
management, bidirectional artist sync, provider-agnostic discovery, live
sidebar badges, and critical source ID embedding fix. Version modal
reorganized to lead with current features and summarize earlier v2.2 work.
Full auto-import pipeline: background worker watches the staging folder,
identifies music using embedded tags → folder name parsing → AcoustID
fingerprinting, matches files to metadata source tracklists, and
processes high-confidence matches through the existing post-processing
pipeline automatically.
Worker: AutoImportWorker with start/stop/pause/resume, configurable
scan interval (default 60s), confidence threshold (default 90%), and
auto-process toggle. Processes one folder per cycle, alphabetical
order. Disc folder detection, stability checking, content hash dedup.
Confidence gate: 90%+ auto-processes silently, 70-90% queued as
pending review with approve/dismiss actions, <70% flagged for manual
identification. Track matching uses weighted algorithm (title 45%,
artist 15%, track number 30%, album tag 10%).
Database: auto_import_history table tracks every scan result with
folder hash, match data JSON, confidence, status, timestamps.
API: 7 endpoints — status, toggle, settings (GET/POST), results
(filtered/paginated), approve, reject.
UI: Auto tab on Import page with enable toggle, confidence slider,
scan interval selector. Live result cards with album art, confidence
bar (green/yellow/red), status badges, match stats. 5-second polling.
Artist Sync button on enhanced library page now does true bidirectional
sync: Phase 1 pulls new albums/tracks from the media server using the
DatabaseUpdateWorker in deep scan mode (preserves enrichment), Phase 2
removes stale DB entries for files no longer on disk. Works for Plex,
Jellyfin, and Navidrome. Toast shows +albums, +tracks, -stale counts.
Repair jobs tab redesigned: 2-column grid layout with glass gradient
cards, accent top line on hover, hover lift effect, job description
text below name, running state with pulsing accent bar. Responsive
to single column under 900px.
Fixed deezer_artist_id → deezer_id column name on artists table lookup.
The artists table uses 'deezer_id' but the enhanced library artist
lookup was querying 'deezer_artist_id' (the watchlist_artists column
name). Fixed to use the correct column name.
Watchlist scanner: empty discography (no new releases in lookback) was
treated as API failure, causing "Failed to get artist discography" for
artists like Kendrick Lamar who simply had no recent releases. Now
distinguishes None (API failure → try next source) from [] (success,
no new tracks). Spotify backfill now uses the authenticated client
instance instead of creating a fresh unauthenticated one.
Wishlist nebula: album remove now sends album_name (API updated to
accept album_name as fallback alongside album_id). Track remove
re-renders the nebula after deletion. Toned down processing pulse
animation.
Updated test to verify fallback triggers on API failure (None), not
on empty results.
Full automation page upgrade with group management and drag-and-drop:
Backend: batch_update_group() and bulk_set_enabled() DB methods, new
PUT /api/automations/group and POST /api/automations/bulk-toggle endpoints.
Group headers: rename (inline edit), delete (choice dialog — keep
automations or delete all), bulk toggle (enable/disable all in group).
Actions appear on hover, styled as small icon buttons.
Drag and drop: non-system cards are draggable between group sections.
Drop zones show dashed accent border feedback. Collapsed sections
auto-expand on 500ms drag-hover. System/Hub sections dimmed during drag.
dragenter counter pattern handles child element bubbling.
Delete group dialog: glass card modal with three options — keep
automations (move to My Automations), delete everything, or cancel.
The MusicBrainz consistency change referenced 'context' inside
_embed_source_ids(), but that variable was never passed to the function.
Every download since that commit silently skipped ALL source ID tags
(Spotify, MusicBrainz, Deezer, AudioDB, Tidal, Qobuz, Last.fm, Genius)
with the error 'name context is not defined' caught as non-fatal.
Fix: pass context from _enhance_file_metadata to _embed_source_ids,
with None default for backward compatibility.
Adaptive card on the Dashboard showing library state with four modes:
- No server: gold accent, directs to Settings
- Disconnected: gold warning with troubleshooting guidance
- Empty library: blue accent with prominent Scan Now button
- Healthy: green accent with stats grid (artists/albums/tracks/DB size),
Refresh button (incremental) and Deep Scan button (full re-check)
Stats displayed as mini cards with individual icons. Animated glow orb,
gradient accent top line, shimmer progress bar during scans. Deep scan
added to /api/database/update endpoint (deep_scan flag) — re-checks
every track, adds new ones, removes stale, preserves enrichment data.
Confirmation dialog explains what deep scan does before starting.
Soulseek source files often carry the uploader's MusicBrainz IDs from
different releases. When post-processing skipped tag clearing (missing
spotify_album context in wishlist batches, or enhancement exceptions),
these conflicting IDs persisted and caused Navidrome to split one album
into multiple entries.
Added _wipe_source_tags() — a lightweight emergency tag wipe that clears
all tags without writing new ones. Called in every failure/skip path:
stream processor exception, playlist mode exception, verification worker
missing context, and verification worker exception. Idempotent and
wrapped in try/except so it never interferes with the existing flow.
When the DB stored a path the resolver couldn't map to a local file
(common with Navidrome virtual paths or Docker path mismatches), file
deletion was silently skipped — the DB record was removed but the file
stayed on disk with no indication to the user.
Now logs the resolution failure with the stored path, returns a
file_error in the API response, and the frontend shows a warning toast
explaining the file wasn't deleted plus a second toast with the specific
reason (e.g. Navidrome 'Report Real Path' instructions).
Keep the weekly Last.fm radio generation step in the web watchlist scan post-processing chain so the higher-level scan behavior stays intact after moving the scan loop into the shared scanner core.
Bring placeholder tracklist skipping back into the shared watchlist scan path, and centralize the DB-only artist image backfill helper so both web scan entrypoints reuse the same logic.
Drop the legacy watchlist scan entrypoints that are no longer used by the web scan flow, and keep the live refresh path pointed at the shared scanner helper.
Move the shared watchlist scan loop into core/watchlist_scanner.py so web_server.py only handles triggers, locks, progress, and post-scan orchestration.
Manual and scheduled watchlist scans now share the same scanner-side core, while the web entrypoints keep profile selection and automation progress updates.
Recording MBIDs are now pulled from the matched release tracklist instead
of independent match_recording() searches, guaranteeing the recording ID
is consistent with the selected release. Batch-level artist name is used
for release cache keys so all tracks hit the same preflight-cached entry
even when Soulseek metadata spells the artist differently. A post-batch
consistency pass (run_album_consistency) rewrites album-level tags on all
files after the batch completes — the safety net that prevents Navidrome
album splits even when per-track lookups drift.
New POST /api/v1/request endpoint accepts a search query from external
sources (Discord bots, Home Assistant, curl) and triggers the
search-match-download pipeline asynchronously. Returns a request_id
for status polling via GET /api/v1/request/<id>. Optional notify_url
for callback on completion.
Also adds webhook_received trigger type and search_and_download action
type to the automation engine, so users can build custom flows like
"when webhook received → search & download → notify Discord".
Includes info panel in Settings showing endpoint URL and curl example.
Two bugs: (1) 'wishlist' was missing from the settings save whitelist,
so the toggle silently reset to ON on every page reload. (2) The
wishlist cleanup function unconditionally removed tracks sharing the
same name+artist regardless of album, ignoring the allow_duplicates
setting. Now when allow_duplicates is on, the dedup key includes the
album name so same song from different albums can coexist.
Prints all SOULSYNC_* environment variables before OAuth servers start,
helping diagnose reported Unraid issue where callback port env var is
present in the container but not seen by the Python process.
Auth instruction pages and log messages now use the actual configured
callback port instead of hardcoding 8888. Added startup logging that
prints whether SOULSYNC_SPOTIFY/TIDAL_CALLBACK_PORT env vars were
detected, helping diagnose Unraid/Docker env var issues. Also fixes
uses_main_port detection for custom callback ports and moves the
wishlist button handler to global init so it works on all pages.
Explored status was stored only in frontend memory; on reload the badge
disappeared because the API never returned it. Added explored_at column
to mirrored_playlists (auto-migrated), written when build-tree completes,
and read back via SELECT * so the badge survives page refreshes.
- Pass playlist image_url to _run_sync_task from all source-specific sync
start handlers (Deezer, Tidal, Spotify public, YouTube, automation mirror)
— previously only the /api/sync/start endpoint passed it
- Fix plex_client.set_playlist_image: use uploadPoster(url=) instead of
uploadPoster(data=) which is not a valid PlexAPI argument
- deezer_client: use picture_xl > picture_big > picture_medium fallback
for better cover art resolution
- tidal_client: extract image_url in get_playlist() from JSON:API
relationships (was only extracted in metadata-only listing)
- parse_youtube_playlist: capture playlist thumbnail from yt-dlp result
- Add visible logging for image upload attempts and outcomes
Spotify was being called for album/artist data fetching across multiple
background workers and the Artists page search even when the user had
Deezer or iTunes set as their primary metadata source. Being authenticated
for playlist sync was treated as permission to use Spotify for everything.
- watchlist_scanner: add _spotify_is_primary_source() that checks both
auth and primary source config; use it for all album/artist data fetching
(discovery pool, recent album caching, playlist curation, similar artist
ID matching, proactive ID backfill). _spotify_available_for_run() is kept
for sync_spotify_library_cache which must run regardless of primary source
- repair_jobs/metadata_gap_filler: gate Spotify ISRC lookup on primary
source being 'spotify'; MusicBrainz lookup unaffected
- repair_jobs/unknown_artist_fixer: replace hardcoded spotify_client with
source-aware client selection — primary source ID tried first, each ID
matched to its correct client (fixes latent bug passing Deezer IDs to
Spotify)
- web_server.py /api/match/search: Artists page search was hardcoded to
spotify_client.search_artists(); now uses _get_metadata_fallback_client()
so results come from the configured primary source
Adds a new Last.fm Radio section to the Discover page that lets users
search a track on Last.fm, generate a similar-tracks playlist, and run
it through the existing discovery/download/sync pipeline. Also generates
playlists automatically from top listening history during watchlist scans
(max once per week).
- core/lastfm_client.py: Add get_similar_tracks() using track.getsimilar
- core/listenbrainz_manager.py: Add save_lastfm_radio_playlist() with
deterministic MBID (MD5 seed), cleanup limit of 5 for lastfm_radio type
- web_server.py: Add /api/lastfm/configured, /api/lastfm/search/tracks,
/api/lastfm/radio/generate, /api/discover/listenbrainz/lastfm-radio;
fix playlist['name'] KeyError in discovery worker that was resetting
phase back to 'fresh' after completion
- core/watchlist_scanner.py: Add _generate_lastfm_radio_playlists() with
weekly throttle, called at end of scan_all_watchlist_artists()
- webui/index.html: Add #lastfm-radio-section above ListenBrainz section,
hidden unless Last.fm API key is configured
- webui/static/script.js: Search/generation/card-load functions; fix
discovery modal labels (Last.fm Radio vs ListenBrainz), description
update on completion, belt-and-suspenders completion handling inside
updateYouTubeDiscoveryModal; fix album/duration display for tracks
without metadata; music note SVG placeholder for missing art
- webui/static/style.css: Styles for search bar, dropdown, result rows
New core/replaygain.py module uses FFmpeg's ebur128 filter (already a
project dependency) to analyze integrated loudness and true peak, then
writes ReplayGain 2.0 tags (-18 LUFS reference) to MP3 (TXXX frames),
FLAC/OGG/Opus (Vorbis comments), and M4A/MP4 (freeform atoms).
Three analysis modes in the enhanced library view:
- Per-track RG button: synchronous single-track analysis (~1-3 s)
- Album "ReplayGain" button: background job writing both track gain
and album gain (mean LUFS across all album tracks) to every file
- Bulk bar "ReplayGain" button: batch track-gain for selected tracks
read_file_tags() in tag_writer.py extended with four new optional keys
(replaygain_track_gain/_peak, replaygain_album_gain/_peak) so existing
RG values surface in the tag-preview diff view. Purely additive — no
existing endpoints or DB schema changed.
Singles could not be saved as a flat file (e.g. "$artist - $title")
because the frontend blocked any template without a "/" and the
backend path builder treated an empty folder_path as falsy, falling
through to the hardcoded nested-folder structure.
Frontend: removed the must-include-slash validation for single
templates only (album templates still require it).
Backend: changed condition from `if folder_path and filename_base`
to `if filename_base` so an empty folder_path is handled correctly
as a flat drop into the transfer root.
Hardcoded ports 8888/8889 conflict when SoulSync runs behind Gluetun or
other containers that claim those ports. Introduce SOULSYNC_SPOTIFY_CALLBACK_PORT
and SOULSYNC_TIDAL_CALLBACK_PORT env vars (defaulting to 8888/8889) so
users can remap without rebuilding the image.
docker-compose.yml exposes the vars with comments explaining how to keep
the port mappings in sync with the redirect URI in Settings → Connections.
Track per-quality-tier failure reasons across all failure paths (stream
error, empty manifest, download exception, stub file, MP4 extraction
failure) and include them in the exhausted-tiers log message so failures
are diagnosable from logs.
When HiRes is configured with no fallback and all tiers are exhausted,
log an actionable hint directing the user to enable Quality Fallback.
Surface Tidal-specific error messages in the UI task on retry
exhaustion: distinguishes HiRes-unavailable (with actionable guidance)
from general Tidal auth/quality failures, rather than showing the
generic Soulseek error string.
track_downloads stores local Windows paths but tracks table stores
server-side paths (Plex/Jellyfin). Both the track_id lookup (NULL
due to failed auto-link at insert time) and exact file_path fallback
were failing.
Added filename-suffix LIKE matching as a final fallback in
get_track_source_info, plus a back-link so the track_id gets written
back for fast future lookups. Also improved the auto-link in
record_track_download to use the same suffix matching when exact path
fails.
M3U entries now resolve actual file paths from the DB instead of
synthesising a fake 'Artist - Title.mp3' string that no media server
could use. Adds optional M3U Entry Base Path setting (Downloads tab)
so servers requiring absolute paths (e.g. /mnt/music) can be supported.
- New POST /api/generate-playlist-m3u endpoint: per-artist batch DB
lookups with fuzzy title matching, prefixes entry_base_path when set
- autoSavePlaylistM3U and exportPlaylistAsM3U now call the new endpoint
- M3U Entry Base Path input added below Music Videos Dir in settings,
follows path-input-group pattern with Unlock button and autosave
Server Playlists was filtered to only show playlists matching mirrored_playlists entries,
but Discover syncs are stored in sync_history (not mirrored_playlists), so they were
excluded. Adds GET /api/sync/history/names returning distinct synced playlist names,
and includes those in the filter alongside mirrored playlists.
_try_staging_match() built a minimal context missing spotify_artist,
spotify_album, is_album_download, and has_clean_spotify_data. Post-
processing returned early at the missing-spotify_artist guard and the
copied file was left at the transfer root with its original filename.
Now mirrors the sync modal worker's context-building: uses
_explicit_album_context/_explicit_artist_context when available
(artist-page album downloads), falls back to track.album/track.artists
for playlists and sync modal. track_number and disc_number are also
forwarded so multi-disc albums land in the correct Disc N/ subfolder.
import_album_process never computed total_discs, so spotify_album.total_discs
always defaulted to 1 in _build_final_path_for_track. Now pre-computes
total_discs from the matched tracks before the per-track loop.
Builds a new Your Albums section on the Discover page that aggregates
saved/liked albums from all connected services, mirroring the Your Artists
pattern. Deezer works via both OAuth and ARL.
- tidal_client: add get_favorite_albums() with V2/V1 API fallback
- deezer_client: add get_user_favorite_albums() via OAuth (user/me/albums)
- deezer_download_client: add get_user_favorite_albums() via ARL session
- music_database: add liked_albums_pool table (deduped by artist::album
normalized key), upsert_liked_album, get_liked_albums,
get_liked_albums_last_fetch, clear_liked_albums
- web_server: GET /api/discover/your-albums (ownership-checked, paginated),
GET /api/discover/your-albums/sources, POST /api/discover/your-albums/refresh,
_fetch_liked_albums background worker (Spotify + Tidal + Deezer OAuth/ARL)
- frontend: Your Albums section with source selector cog, album grid reusing
spotify-library-card styles, search/filter/sort/pagination, download missing
button, auto-refresh poll on first load
Also fix: Deezer greyed out in Your Artists sources when using ARL — connection
check now accepts ARL auth (deezer_dl.is_authenticated()) in addition to OAuth,
and _fetch_and_match_liked_artists falls back to ARL client for artist fetching.
Add get_user_favorite_artists(limit=200) to DeezerDownloadClient to fetch a user's favorite artists via the public API using an ARL-authenticated session (paginated, error-handled, returns deezer_id, name, image_url).
Update web_server to treat Deezer as connected if either OAuth or ARL is authenticated, and to fetch favorite artists from OAuth client when available or from soulseek_client.deezer_dl (ARL) otherwise. Fetched artists are upserted into the database and appropriate log/console messages and counters are updated.
Three issues fixed:
1. Plex add-track used delete+recreate (Playlist.create) which was
unreliable — switched to addItems() which atomically appends the
track without touching existing playlist items.
2. After a successful add, the UI only did an optimistic local update.
On reopen the automatic matcher ran fresh and couldn't connect the
manually selected track to the source slot, making it look unfixed.
Now both add and replace re-fetch the compare view from the server
so the matcher sees the actual updated Plex state.
3. Matching algorithm was too strict for common title variants. Added
_norm_title() which strips feat./ft., remaster/remastered, and
edition qualifiers before comparison — so "Boy 1904" matches
"Boy 1904 (2019 Remaster)" and "Float Away" matches "Float Away
(feat. Flamingosis & Eric Benny Bloom)". Display titles unchanged.
Spotify album art: replace the 4-char size segment after '0000ab67616d'
with '82c1' to request the original uploaded master (up to 2000px+).
Applied via _upgrade_spotify_image_url() in Track, Artist, and Album
dataclass constructors and as a catch-all in _download_cover_art.
Scoped to the ab67616d album art prefix only — artist images use a
different prefix (ab676161) where the trick does not apply.
iTunes/Apple Music: replace '100x100bb' with '3000x3000bb' in all
artworkUrl100 replacements across Track, Artist, Album, and the
get_album images arrays. Also applied as a catch-all in _download_cover_art.
Deezer already uses cover_xl at its maximum — no changes needed there.
The status cache's source field was being set by _get_metadata_fallback_source(),
which internally calls is_spotify_authenticated() a second time via module import.
Any inconsistency between that path and the direct auth check already performed
would return 'deezer', poisoning the cache — and since the WebSocket takes over
from the HTTP poll, it never corrected itself.
Now reads config directly and reuses the single auth probe result.
Gear button next to View All opens a sources modal letting users pick
which connected services (Spotify, Tidal, Last.fm, Deezer) contribute
artists to the Your Artists carousel. Setting saved via standard
/api/settings endpoint under discover.your_artists_sources.
- GET /api/discover/your-artists/sources returns enabled config + which
services are currently connected
- _fetch_and_match_liked_artists skips sources not in the enabled list
- Disconnected services shown dimmed and non-interactive in modal
- Saving with nothing selected blocked with error toast
- Remove z-index from .sidebar-header (fixes artist map overlap)
- Add padding-bottom to #automations-list-view (search bar overlap fix)
The fallback test (used when no audio files exist in the library) sends
a dummy fingerprint to the AcoustID API. The API correctly rejects the
dummy fingerprint but this is not error code 4 (invalid key), so the
test was returning False instead of True. Any non-code-4 error from the
fallback means the API key was accepted — only code 4 means a bad key.
Instrument web_server.py to record Spotify API usage by calling core.api_call_tracker.api_call_tracker.record_call before various Spotify SDK calls. Added local imports and record_call invocations around artist, playlist, playlist_tracks_page, current_user_saved_tracks, artists_batch, track and related endpoints across functions such as get_artist_image, get_playlist_tracks, watchlist_artist_config, enrich_similar_artists, get_your_artist_info, get_artist_map_explore, import_album_process and import_singles_process. This enables per-endpoint monitoring and helps with rate-limit tracking while keeping the import local to avoid top-level dependency issues.
Deezer and iTunes defaulted to 50 albums max, silently truncating large
discographies. Deezer now paginates (100 per page) up to 200. iTunes
raised to 200 (single call). All callers in web_server.py updated to
use the new defaults instead of hardcoding limit=50.
Also adds diagnostic logging for allow_duplicates album comparison
to help debug inconsistent singles behavior.
Version bump to 2.3 with rewritten What's New modal covering all
changes since v2.2. Docker publish workflow default updated.
Sidebar improvements:
- Header stays pinned at top while nav and player scroll beneath it
- Media player collapses to compact single-line when no track is
playing, expands to full size when playback starts
Fixes:
- Server playlists endpoint Plex Tag object crash (getattr fix)
- Server playlists tab auto-refreshes after download completion
- Fixed dead code syntax error in archived version notes
M3U generator was calling .join() on an array of artist objects instead
of extracting .name first, producing "[object Object] - Track Name".
Now handles all artist formats: array of objects, array of strings,
single string, single object.
Also fix "name 'database' is not defined" error when updating album
year in post-processing — was using bare 'database' instead of
get_database() helper.
Plex API can return Tag objects mixed with playlists — these lack the
playlistType attribute, causing AttributeError. Use getattr with safe
default instead of direct attribute access.
Add 3-tier Unknown Artist guard in post-processing: checks track_info
artists, original search result, then re-fetches from metadata API
before building folder paths or embedding tags. Prevents files from
landing in Unknown Artist folders when the download context has
incomplete artist data.
- Stop active DB update work before tearing down executor pools.
- Short-circuit scan completion callbacks during shutdown so in-flight timer ticks don’t queue follow-up work.
- Prevent the download monitor from draining deferred/completed tasks after shutdown starts.
- Make listening stats startup stop-aware so it exits cleanly if teardown begins during warmup.
- If a metadata update is already running, it can now observe should_stop and exit cleanly instead of continuing after SIGTERM.
- Add interruptible stop events to background workers so shutdown
wakes out of long sleeps instead of waiting on fixed delays.
- Stop scan managers, repair worker, executors, and cleanup helpers
deterministically so process exit does not leave background threads
alive.
- Add startup warnings for stale SQLite WAL/SHM sidecars so unclean
shutdowns are easier to spot before init/migration errors cascade.
- Prevent forced kills from leaving SQLite sidecars behind, which
made rollbacks to older branches fail with malformed database
errors.
There's a lot side-effects happening at import-time (eg. client initialization etc.), some of which include meaningful logging as well (eg. migration-related logging in MusicDatabase client).
If we delay the logging initialization to the __main__ block, we'll lose out on these early logs
New sidebar page showing every download task across the app in a unified
live-updating list. Tracks from Sync, Discover, Artists, Search, and
Wishlist all appear in one place.
Features:
- Filter pills: All / Active / Queued / Completed / Failed
- Section headers grouping by status category
- Track position (3 of 19) for album/playlist batches
- Album art, artist/album metadata, batch context, error messages
- Status dots with accent glow for active, green for complete, red fail
- Clear Completed button removes terminal items from tracker
- Nav badge shows active download count from any page via WebSocket
Fixes artist [object Object] display — handles all format variations
(list of dicts, list of strings, dict, string) for artist and album
fields in the API response.
Serverless check (YouTube/HiFi/Qobuz) now runs before the slskd
connection test. Prevents 4-minute timeout hangs when Hybrid mode
includes both Soulseek and serverless sources but slskd isn't running.
Also reports green if any serverless source is in the hybrid order,
not just the first.
Wizard now shows automatically on fresh installs. Detection uses a
server-side flag (setup.completed) plus download_source.mode as a
fallback for existing users who configured settings before the wizard
existed. Config.json template defaults no longer fool the check.
Script load order fixed — setup-wizard.js loads before script.js so
openSetupWizard exists when DOMContentLoaded fires. Both finish and
skip paths set the server flag and localStorage, then continue app
initialization via callback.
7-step full-screen wizard: Welcome, Metadata Source, Download Source,
Paths & Media Server, Add Artists, First Download, Done. All settings
save to DB identically to the Settings page. Supports all 6 download
sources with inline config and test buttons. First download goes through
the full matched download pipeline with metadata context.
Fixes:
- Download clients (YouTube/HiFi/Tidal/Qobuz/Deezer) now reload
download_path when settings change instead of caching from init
- watchlist_artists table migrations now include deezer_artist_id and
discogs_artist_id in all 3 table rebuild locations (was being dropped)
- CREATE TABLE for watchlist_artists includes all provider ID columns
- Serverless download sources (YouTube/HiFi/Qobuz) show green status
instead of red disconnected on sidebar and dashboard
- Suppress repeated slskd 401 errors — logs once then silences until
connection recovers
Stripped 4,200+ emoji characters from print(), logger calls across
39 Python files. Logs are now clean text — easier to grep, more
professional, no encoding issues on terminals without Unicode support.
Seasonal config icons preserved for UI display.
Tag preview and writer now use track_artist (per-track artist) for the
Artist tag when available, falling back to artist_name (album artist)
when NULL. Album Artist tag always uses artist_name.
Fixes#277 — DJ mix albums no longer overwrite per-track artists
(Technique, Gouryella) with the album artist (Tiësto).
Lidarr integration:
- New core/lidarr_download_client.py with full interface parity
(search, download, status, cancel — same as Qobuz/Tidal/HiFi)
- Registered in download orchestrator with source routing
- Settings: URL + API key on Downloads tab with connection test
- Available as standalone source or in Hybrid mode priority order
- API key encrypted at rest
- All streaming source checks updated to include 'lidarr'
Lidarr downloads full albums via Usenet/torrent — SoulSync imports
only the tracks it needs and discards the rest.
Music video path validation:
- Empty/unconfigured path returns clear error instead of silent failure
- Write permission test before starting download
- Default changed from './MusicVideos' to empty (must be configured)
Clients are for the most part being initialized per-request, which leads to a lot of redundant client initialization, as well as noise on the logs, since each client initialization emits a row on the logs, eg. 'Deezer client initialized'
Click any video card in Music Videos tab to download. Flow:
1. Search primary metadata source for clean artist/title
2. Fall back to YouTube title parsing if no match
3. Download video via yt-dlp (best quality MP4)
4. Save to configured Music Videos folder as Artist/Title-video.mp4
UI shows circular progress ring on the thumbnail during download,
green checkmark on completion, red X on error (clickable to retry).
Cards are non-interactive while downloading.
Backend: /api/music-video/download and /api/music-video/status endpoints
YouTube client: download_music_video() method keeps video format
New "Music Videos" pill tab alongside Spotify/Deezer/iTunes/Discogs
in both enhanced search and global search. Searches YouTube via yt-dlp
and displays results in a video card grid with 16:9 thumbnails, play
overlay, duration badge, channel name, and view count.
- Backend: /api/enhanced-search/source/youtube_videos endpoint with
search_videos() method on YouTubeClient returning YouTubeSearchResult
- Frontend: Video grid layout with responsive cards, YouTube red tab
color, proper section hiding when switching between metadata and
video tabs
- Global search: Full parity with enhanced search video rendering
- No download functionality yet — display only
New configurable path for storing music videos separately from audio
files, following Plex's global music video folder convention.
- Settings: library.music_videos_path (default: ./MusicVideos)
- UI: Music Videos Dir field on Settings Downloads tab with lock/unlock
- Docker: /app/MusicVideos volume mount in Dockerfile and docker-compose
- Added 'library' to settings save whitelist (was missing — music_paths
also wasn't persisting through main settings save)
- No download functionality yet — path infrastructure only
The discovery worker skipped already-discovered tracks even when their
matched_data was incomplete (missing track_number, release_date, album
ID). These stale discoveries from before the enrichment fix would
persist forever, causing the automation pipeline to keep producing
tracks with no year, no track numbers, and no cover art.
Now treats discovered tracks as undiscovered if they're missing
track_number AND have no release_date or album ID, so the enriched
discovery pipeline fills in the gaps on the next run.
Adds two new sections to the version modal covering the Unknown Artist
fix, centralized metadata source selection, Deezer cache fix, sync
completion feedback, Fix Unknown Artists maintenance job, and the
matching engine artist gate improvements.
All metadata source decisions now flow through get_primary_source() and
get_primary_client() in core/metadata_service.py. Previously 6 different
files reimplemented this logic with inconsistent defaults ('itunes' vs
'deezer') and auth checks, causing bugs when any one was missed.
Changes:
- metadata_service.py: Added canonical get_primary_source/get_primary_client
- web_server.py: _get_metadata_fallback_source() and _get_active_discovery_source()
are now thin wrappers delegating to metadata_service
- seasonal_discovery.py: _get_source() delegates to metadata_service
- personalized_playlists.py: _get_active_source() delegates to metadata_service
- spotify_client.py: Fixed _fallback_source default from 'itunes' to 'deezer'
- watchlist_scanner.py: _get_fallback_metadata_client() delegates to metadata_service
Future changes to source selection only need to update one file.
Seasonal discovery, personalized playlists, and playlist explorer all
defaulted to Spotify when authenticated, ignoring the user's configured
primary source. Now they read from config first.
Spotify's related_artists API (no Deezer/iTunes equivalent) is preserved
as a fallback for all users in personalized playlists. Artist discography
endpoint intentionally unchanged — ID-based lookups need the source that
owns the ID.
When a playlist sync has unmatched tracks sent to wishlist, the
completion toast now shows the specific track names instead of just
a count. Uses warning style so it stands out. The unmatched track
list is included in the sync state result so it's available for
both live status polling and notification history.
Addresses #272 — silent sync failures where users couldn't tell
which tracks out of 150+ failed to match their Plex library.
Discovery workers now respect the user's configured primary metadata
source instead of always using Spotify when authenticated. This
completes the intent of commit 3c211ea.
The core fix addresses data loss in the discovery→sync→wishlist→download
pipeline: the Track dataclass strips album metadata to a plain string,
losing album ID, track_number, release_date, and images. Discovery
workers now enrich results via get_track_details() to recover this data.
Deezer's get_track_details() cache validation was incorrectly trusting
search-result cache (which lacks track_position), returning track_number=0.
Also fixes wishlist download processing where albums without IDs couldn't
map to artists, and the fallback read 'artist' (singular) instead of
'artists' (plural), always producing "Unknown Artist".
Includes a one-time migration to purge stale discovery cache entries.
New automation action that executes user scripts from a dedicated
scripts/ directory. Available as both a DO action and THEN action.
Scripts are selected from a dropdown populated by /api/scripts.
Security: only scripts in the scripts dir can run, path traversal
blocked, no shell=True, stdout/stderr capped, configurable timeout
(max 300s). Scripts receive SOULSYNC_EVENT, SOULSYNC_AUTOMATION,
and SOULSYNC_SCRIPTS_DIR environment variables.
Includes Dockerfile + docker-compose.yml changes for the scripts
volume mount, and three example scripts (hello_world.sh,
system_info.py, notify_ntfy.sh).
After a successful playlist sync, if the source playlist has cover
art (Spotify, Tidal, Deezer, etc.), the image is downloaded and
uploaded as the playlist poster on the media server. Plex uses
uploadPoster(), Jellyfin/Emby uses POST /Items/{id}/Images/Primary.
Navidrome skipped (no playlist image API). Failure is silent — sync
result unchanged. Automation-triggered syncs and playlists without
images are unaffected.
What's New: Added Deezer user playlists, Qobuz token auth, streaming
source artist gate, download history provenance, and comprehensive
fixes section covering artist name casing, future album skip, track
number fix, Emby sync, discovery fix fallback, and all new settings.
Help docs: Updated Qobuz auth to mention token option, added Music
Library Paths, Replace Lower Quality, and HiFi Instance Health to
Other Settings section.
New setting in Settings > Library lets users add folder paths where
their music files live. The file resolver checks these paths when
looking for library files, solving Docker path mismatches and multi-
folder libraries. Required for tag writing, streaming, and orphan
detection when the media server reports paths that differ from what
SoulSync can see. Docker users mount their music folder(s) with
read-write access and add the container-side path. Default is empty
— existing users see no change.
New toggle in Settings > Library: "Replace lower quality files on
import". When enabled, if a track already exists in the library at
a lower quality tier (e.g. MP3) and a higher quality version (e.g.
FLAC) is imported from staging, the existing file is replaced.
Comparison uses the existing QUALITY_TIERS system (lossless > opus/
ogg > m4a > mp3). When disabled (default), existing behavior is
unchanged — existing tracks are always kept. Also applies to regular
downloads that land on an existing file.
Discovery fix modal now tries the user's active metadata source
first, then falls back through all available sources (Spotify,
Deezer, iTunes) in sequence. Previously hardcoded Spotify with no
fallback, leaving users without Spotify stuck on "Searching...".
New /api/deezer/search_tracks endpoint exposes the existing
DeezerClient.search_tracks() method for the fix modal. Same
request/response format as Spotify and iTunes endpoints.
Qobuz added reCAPTCHA to their login endpoint, blocking automated
email/password auth for new users. Token login lets users paste
their X-User-Auth-Token from the browser DevTools after logging in
manually. Added to both Connections and Downloads tabs with
instructions. Existing email/password flow completely unchanged.
Backend validates token via user/get API and saves the session
identically to email/password login.
New "Check All Instances" button in Settings > Downloads > HiFi
shows each configured instance with color-coded status: green for
fully working (can download), orange for search-only (downloads
fail), red for offline/SSL error/timeout. Helps users understand
why HiFi downloads aren't working when community instances go down.
When downloading individual tracks from a playlist sync, the album
detection fallback hardcoded track_number to 1 instead of using the
value already available in the download context from the playlist
metadata. Now checks original_search.track_number and track_info
.track_number before falling back to 1. Full album downloads and
Spotify API lookup paths are completely unchanged.
New "Deezer" tab on sync page shows authenticated user's playlists,
identical to the Spotify tab pattern — same card layout, details
modal with track list, sync button, and download missing tracks
flow. Existing URL import renamed to "Deezer Link" tab (unchanged).
Backend: get_user_playlists() and get_playlist_tracks() on the
download client fetch via public API with ARL session cookies.
Album release dates batch-fetched for $year template variable.
Three new endpoints: arl-status, arl-playlists, arl-playlist/<id>.
Frontend: cards use Spotify-identical HTML structure with live sync
status, progress indicators, and View Progress/Results buttons.
Downloads reuse openDownloadMissingModal with zero modifications.
Track data cached on first open, instant on subsequent clicks.
Library page: new dropdown filter to show artists matched or unmatched
to any metadata source (Spotify, MusicBrainz, Deezer, Discogs, etc).
Select "No Discogs" to find artists needing manual Discogs matching.
Filter applied as WHERE clause on the source ID columns.
Discogs enrichment: added to valid_services whitelist, _enrichment_locks,
and _run_single_enrichment handler. The Enrich button was returning an
error when Discogs was selected from the dropdown.
Discogs was missing from valid_services, _enrichment_locks, and
_run_single_enrichment handler. The Enrich button on the enhanced
library page returned "service must be one of" error when Discogs
was selected. Added handler using _process_item pattern matching
other batch-style workers, with guard against track-level enrichment.
Settings > Advanced now shows database size, free pages, and auto-
vacuum mode. Two actions: Compact Database (full VACUUM to reclaim
dead space) and Enable Incremental Vacuum (one-time setup for
automatic page reclamation). Both have confirmation dialogs warning
about lock time on large databases. Info refreshes on Advanced tab
switch and after each operation.
Streaming matching: add artist gate rejecting candidates with artist
similarity below 0.4, raise threshold to 0.60, block fallback to
Soulseek filename matcher for Tidal/Qobuz/HiFi/Deezer. Fix single-
char artist containment bug where normalize_string strips non-ASCII
(e.g. "B小町" → "b") causing "b" to match any artist containing
that letter. Fixed in both score_track_match and the Soulseek scorer.
YouTube and Soulseek matching behavior unchanged.
Global search: add registerSearchDownload() calls to _gsClickAlbum
and _gsClickTrack so downloads create bubble snapshots on dashboard
and search page, matching the enhanced search standard.
Global search escaping: add _escAttr() helper to handle newlines in
album/artist names that broke inline onclick string literals.
Entries are now compact cards that expand on click to reveal source
details. Shows expected vs downloaded title/artist with red mismatch
highlighting. Source artist column added to DB. Streaming track IDs
extracted from the id||name filename pattern. File and ID always on
their own line to avoid edge-case misplacement.
Track original source filename, track ID, and AcoustID verification
result for every download. Helps debug wrong-file downloads from
streaming sources like Tidal. Each column migrated independently
for crash safety. Frontend shows source detail line and color-coded
AcoustID badge per entry. Button renamed to "Download History".
FLAC → Opus/AAC conversion only tried to read embedded art from the
source FLAC. If art was only in cover.jpg (not embedded), the lossy
copy got no art. Now falls back to cover.jpg in the same directory.
Previously skipped for Tidal/Qobuz/HiFi/Deezer as "trusted API sources"
but streaming APIs can return wrong versions (live, remix, cover).
AcoustID now runs for every download source when enabled.
Added user-follow-read scope to all 5 SpotifyOAuth instances in
web_server.py (was only in spotify_client.py). Fixed OAuth callbacks
using is_authenticated() instead of is_spotify_authenticated() — the
former returns True with iTunes/Deezer fallback, masking auth failures.
Credit: kettui (PR #253) identified both issues.
When using templates like "albums/$albumartist/$album/$track - $title",
post-processing created Transfer/ArtistName/ before computing the
template path, leaving an empty folder. _build_final_path_for_track
handles all directory creation based on the template — the premature
makedirs was redundant and incorrect.
- search_artists() now filters by active media server — no more duplicate
results from Plex/Jellyfin/Navidrome showing the same artist 3 times
- Per-artist Sync button re-fetches artist name from media server, catches
renames (e.g., Plex changing "Kendrick Lamar" to "eastside k-boy")
- Global search track click opens download modal directly instead of
navigating to enhanced search page (matches enhanced search behavior)
New download_source column on library_history table records which source
(Soulseek, Tidal, Qobuz, HiFi, YouTube, Deezer) each track was downloaded
from. Extracted from context username during post-processing.
Frontend shows source badge alongside quality badge on each download entry.
Source breakdown bar below tabs shows per-source totals with color-coded
chips (e.g., "Soulseek: 847 | Tidal: 203"). Includes DB migration for
existing installs. Existing entries show quality only (source is NULL).
Replaced single "Change" button with per-source rows showing match
status for each provider (Spotify, Apple Music, Deezer, Discogs).
Each row has Fix/Match button that searches that specific source API,
plus clear button to remove individual matches.
- Per-source search uses _search_service (same as enrichment modal)
- Backend: added Discogs to valid providers, empty ID clears match
- Fixed provider validation to accept 'discogs' alongside others
- Clear sets DB column to NULL instead of rejecting empty string
Manual scan path was only backfilling the active provider (e.g., only
Spotify IDs if Spotify was active). Now matches the auto-scan behavior:
backfills iTunes, Deezer, Spotify (if auth'd), and Discogs (if token)
for all watchlist artists before scanning begins.
Removed cross-guards that blocked wishlist downloads while watchlist scans
ran (and vice versa). Downloads use bandwidth, scans use API calls —
different resources. The per-call rate limiter handles any API contention.
- Automation engine: each handler now has self-only guard (no cross-block)
- _process_wishlist_automatically: removed watchlist scanning check
- Pipeline Phase 4: removed watchlist scanning check
- Manual watchlist scan endpoint: removed wishlist processing check
Users with large watchlists (6+ hour scans) will now see downloads
starting immediately instead of waiting for the scan to finish.
Initial load fetches 200 tracks instead of all. "Load More" button at
bottom shows (200 of 5,000) and loads next batch on click. Backend now
returns total count alongside limited results for both album and singles
categories. Rendering logic unchanged — just operates on smaller sets.
Removed as_dict=True from all three OAuth callback sites. Spotipy will
return token string directly in future versions — our code only checks
truthiness so both dict and string work.
API Call Tracker:
- Save/load 24h minute-bucketed history + events to database/api_call_history.json
- Persists across server restarts via atexit + signal handler hooks
- New record_event() for rate limit bans (called from _set_global_rate_limit)
- New get_debug_summary() for Copy Debug Info — 24h totals, peak cpm with
timestamp, per-endpoint breakdown, and last 20 rate limit events
- Fixed race condition: events iteration now inside lock during save
Spotify Rate Limit Mitigation:
- Enrichment worker: max_pages=5 on get_artist_albums (was unlimited — artist
with 217 albums caused 22 paginated API calls, now capped at 5)
- Enrichment worker: inter_item_sleep raised from 0.5s to 1.5s
Spotify Re-Auth Fix:
- Both OAuth callbacks (port 8008 + 8888) now clear rate limit ban AND
post-ban cooldown after successful re-auth — Spotify usable immediately
instead of stuck on Deezer fallback for 5 minutes
- Auth cache invalidated on both global client and enrichment worker client
Playlist Pipeline — single automation that runs the full playlist lifecycle:
refresh → discover → sync → download missing. Replaces 4-automation signal
chains. Phase-aware progress display (Phase 1/4, 2/4, etc.), guard function
prevents concurrent runs, fire-and-forget wishlist at end. Re-sync loop
catches newly downloaded tracks on next scheduled run.
- New action type 'playlist_pipeline' with handler, blocks endpoint config,
builder UI (playlist select, process all, skip wishlist checkboxes),
help modal, result display map, and Hub template
- Removed 3 redundant Hub templates (Release Radar, Discovery Weekly,
Playlist Auto-Sync) — all replaced by the pipeline
- Fixed sync completion polling (status is 'finished' not 'complete')
- Fixed refresh handler progress hijack (null out _automation_id)
- Fixed matched_tracks field access from sync_states result
Also in this commit:
- Wishlist badges on enhanced search and global search tracks (amber)
- Discogs added to manual enrichment modal search + artist/album dropdowns
- Profile PIN forgot recovery on profile selection dialog
Watchlist map nodes used w.get('key', '') which returns None when the
DB column is NULL (key exists with None value). None serialized to JSON
null, which is falsy in JS, causing 'No source ID' throw for every
artist click.
- Changed to `w.get('key') or ''` for all ID fields (coerces None→'')
- Added discogs_id to watchlist nodes (was missing entirely)
- Removed hard throw when no source ID — falls back to name-based lookup
- Added console.error logging for future diagnosis
Tidal, Qobuz, HiFi, and Deezer results were blindly taking the first
API result with minimal validation. Now all streaming sources use
score_track_match() — same 60% title / 30% artist / 10% duration
weighting as Soulseek, plus version detection penalties.
- web_server.py get_valid_candidates(): replaced loose title-sim check
with matching engine scoring, version penalty for live/remix/acoustic
- download_orchestrator.py: optional expected_track param enables
scoring in search_and_download_best (backward compatible)
- sync_service.py: passes spotify_track for validation
- Fixed wrong class name (MusicMatchingEngine not MatchingEngine)
cover.jpg was always written from Spotify/iTunes URL (640x640) when the
first track in an album reached _download_cover_art before MusicBrainz
lookup completed. Later tracks with MBID skipped because file existed.
Fix: when cover.jpg exists but is small (<200KB) and we now have a CAA
MBID, attempt to upgrade it with the high-res CAA version. If CAA fetch
fails, keep existing cover — no pointless overwrites.
Also adds Forgot PIN recovery to the profile selection PIN dialog,
reusing the same credential verification flow as the launch lock screen.
Backend reset endpoint now accepts profile_id parameter.
Runs at scan start after ID backfill — zero API calls, all DB lookups:
1. Metadata cache artist image (any source)
2. Deezer direct URL from stored deezer_artist_id
3. Deezer ID from metadata cache by name (even if not on watchlist row)
4. Album art fallback (iTunes artists have no artist images)
Also fixes update path for artists with no external IDs — falls back
to direct row ID update instead of silently matching zero rows.
- Image proxy endpoint (/api/image-proxy) for canvas CORS — allowlisted CDNs,
browser-like UA for Deezer, 24h cache headers. Direct CORS first, proxy fallback.
- Server-side 5-min cache on all artist map endpoints with auto-invalidation
on watchlist add/remove, scan complete, and new MusicMap discoveries.
- Explorer fetches similar artists from MusicMap on-the-fly when none stored,
saves to DB for instant future visits. Validates artist names against
Spotify/iTunes API before loading map — rejects gibberish with 404.
- Genre map per-genre cap removed (was 300 backend, 400 frontend).
- Center node in Explorer uses type 'center' not 'watchlist' — no longer
misidentified as a watchlist artist.
- Error overlay auto-dismisses after 2.5s and returns to Discover page.
- Helper What's New restructured with dated sections (April 4/3/2/1, March),
trimmed from ~80 to ~38 entries, date headers styled as purple dividers.
- Version modal updated with Artist Map section and recent fixes.
get_top_similar_artists now accepts require_source parameter to filter
by source ID in SQL. Previously fetched 200 artists then post-filtered,
but cycling logic (last_featured ASC) rotated artists without IDs to
the front, causing all 200 to be filtered out.
Both /api/discover/hero and /api/discover/similar-artists now pass
require_source=active_source so only artists with valid IDs are returned.
- Metadata cache backfill: batch-lookup all node names across all sources
to fill missing IDs, images, and genres
- Source-aware navigation: View Discography passes correct source to
artist page so non-active-source artists load correctly
- Constellation hover effect: 800ms delay, fade in/out animation, dim
overlay with glowing connection lines to related artists
- Click ripple animation on node selection
- Related artists list in info modal with clickable navigation
- Rich tooltip with artist photo, name, genres on hover
- Removed node dragging (caused visual desync with offscreen buffer)
- Performance: cached constellation lookups, lighter cache query
(no raw_json), canvas.width for proper DPR overlay coverage
