198 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Broque Thomas
|
620c41f1ac |
Add "All Libraries (combined)" mode to PlexClient
GitHub issue #505 (PopeBruhLXIX): users with multiple Plex music libraries (e.g. one per Plex Home user, or two folder roots split across separate library sections) only saw one library inside SoulSync because the connection settings forced you to pick a single library section. SoulSync's PlexClient stored exactly one ``self.music_library`` section reference and every read scanned only that one. This change adds an opt-in "All Libraries (combined)" dropdown option that flips the client into a server-wide read mode where every read method (``get_all_artists`` / ``get_all_album_ids`` / ``search_tracks`` / ``get_library_stats`` / etc) dispatches through ``server.library.search(libtype=...)`` instead of querying a single section. One Plex API call replaces N per-section iterations; Plex handles the aggregation server-side. Implementation: - ``ALL_LIBRARIES_SENTINEL`` (``'__all_libraries__'``) — module-level constant used as the saved DB preference value when the user picks the synthetic "All Libraries" entry. Detection is one string compare in ``_find_music_library`` / ``set_music_library_by_name``. Existing preferences (real library names) are unaffected. - ``self._all_libraries_mode`` (private flag) + ``is_all_libraries_mode()`` (public accessor for external callers). When True, ``music_library`` may stay None — ``is_fully_configured()`` recognizes the mode and still returns True so dispatch sites don't bail. - New private helpers ``_can_query``, ``_get_music_sections``, ``_all_artists``, ``_all_albums``, ``_all_tracks``, ``_search_general``, ``_search_artists_by_name``. Single dispatch point for the section-vs-server branch — every read method funnels through them so future drift fails at one place. - New public helpers for downstream callers: - ``get_recently_added_albums(maxresults, libtype)`` — used by DatabaseUpdateWorker's deep-scan recent-content sweep - ``get_recently_updated_albums(limit)`` — same - ``get_music_library_locations()`` — returns folder roots, used by web_server.py's file-path resolver - ``trigger_library_scan`` and ``is_library_scanning`` fan out across every music section in all-libraries mode. - ``get_available_music_libraries`` prepends a synthetic ``{'title': 'All Libraries (combined)', 'value': sentinel}`` entry ONLY when more than one music library exists. Single-library users don't get the extra option. ``value`` field is the canonical identifier the frontend submits to ``/api/plex/select-music-library`` (real libraries: title; synthetic: sentinel string). Backward- compatible — entries without ``value`` fall back to ``title``. Three crash points fixed in downstream consumers (would have failed during a deep scan after the user picked all-libraries mode): 1. ``database_update_worker.py:411`` — bailed out with "No music library found in Plex" because ``not self.media_client.music_library`` evaluated True in all-libraries mode (music_library is None there). Now uses ``is_fully_configured()`` which recognizes the mode. This was the root cause of the deep scan never starting. 2. ``database_update_worker.py:_get_recent_albums_plex`` — reached ``self.media_client.music_library.recentlyAdded()`` / ``.search()`` directly, AttributeError in all-libraries mode. Now routes through the new helper methods. 3. ``web_server.py:10947`` (file-path resolver) — accessed ``music_library.locations``; gated on ``music_library`` truthy so it didn't crash, but silently skipped all-libraries-mode locations. Now uses ``get_music_library_locations()`` which unions across sections. Plus polish: - ``/api/plex/clear-library`` also resets ``_all_libraries_mode`` so a fresh "select library" flow doesn't inherit stale mode state. - ``/api/plex/music-libraries`` surfaces "All Libraries (combined)" as ``current_library`` when in mode (settings UI displays correctly). - Frontend ``loadPlexMusicLibraries`` uses ``library.value || library.title`` so the sentinel-keyed option submits the sentinel string, not the human-readable label. Pre-select match handles both paths. Honest tradeoffs (documented as known limitations): - Same artist appearing in multiple Plex sections shows as separate entries in SoulSync (no dedup). Plex returns distinct ratingKeys for each. Cosmetic; revisit if it bites users. - Write-back (genre / poster updates) targets one ratingKey at a time — only updates that section's copy. Other sections' copies stay unchanged. - All-libraries mode includes any audiobook library that Plex classifies as ``type='artist'``. Edge case, opt-in only. Tests: 21 new tests in tests/media_server/test_plex_all_libraries.py pin both single-library mode (regression guard) and all-libraries mode for every refactored method. Existing test_plex_pinning.py fixture updated to initialize the new flag. 63/63 media_server tests green, 2148/2148 full suite green. |
||
|
Broque Thomas
|
822759740d |
Fix Download Discography pulling wrong artist + log routing
Two fixes. (1) Discography endpoint now does server-side per-source ID resolution. When the user clicked Download Discography on a library artist, the endpoint received whichever artist ID the frontend happened to pick (spotify_artist_id || itunes_artist_id || deezer_id || library_db_id) and dispatched it as-is to whichever source it queried. If the picked ID didn't match the queried source's ID format, the lookup returned wrong-artist results (numeric ID collisions) or fell back to a fuzzy name search that picked a wrong artist. Two reproducible cases: - 50 Cent's library row had DB id 194687 — coincidentally a real Deezer artist ID for "Young Hot Rod". When the frontend's /enhanced fetch silently fell back to the DB id, the backend sent 194687 to Deezer, and Deezer returned Young Hot Rod's 50 albums in 50 Cent's discography modal. - Weird Al's library row had a stored Spotify ID. The frontend sent that to Deezer, which rejected the alphanumeric ID and fell back to fuzzy name search — which picked The Beatles somehow, returning 45 Beatles albums. The mechanism for per-source ID dispatch already exists in ``MetadataLookupOptions.artist_source_ids``, and the watchlist scanner already uses it; the on-demand discography endpoint just wasn't wired to it. Fix: when the URL artist_id matches a library row by ANY stored ID (DB id, spotify_artist_id, itunes_artist_id, deezer_id, or musicbrainz_id), pull every stored provider ID and pass them as ``artist_source_ids``. Each source gets its OWN stored ID regardless of which one the URL carries. When the URL ID is a non-library source-native ID and the row lookup misses entirely, behavior is identical to before (single-ID dispatch fallback). Logged the resolved per-source ID dict at INFO so future "wrong artist showed up" diagnostics are immediately legible in app.log. (2) Logger namespace fix in core/artists/quality.py and core/metadata/multi_source_search.py. Both modules used ``logging.getLogger(__name__)`` which resolves to ``core.artists.quality`` / ``core.metadata.multi_source_search`` — neither under the ``soulsync`` namespace where the file handler is wired. Result: every [Enhance], [MultiSourceSearch], and direct-lookup INFO line was being written to a logger with no handlers and silently dropped. App log showed the slow-request warning but no diagnostic detail. Switched both to ``get_logger()`` from utils.logging_config so the soulsync.* namespace picks them up. Same content, now actually lands in app.log. Confirmed working in live test: ``[Enhance] Direct lookup matched: deezer ID 1476162252 → 'Desastre'`` No behavior change in any other caller. Empty ``artist_source_ids`` (no library row matched) reaches lookup as ``None`` → identical to current single-ID dispatch path. Logger fix is pure routing — no content change. |
||
|
Broque Thomas
|
3befe9349c |
Direct ID lookup in Enhance Quality, like Download Discography
Followup on the previous Enhance refactor. Multi-source parallel text
search closed the worst case (users with no Spotify/Deezer getting
"unknown artist - unknown album - unknown track" wishlist entries),
but text search itself is still fragile against messy library tags:
"Title (Live)", featured artists in the artist field, etc. Download
Discography never had this problem because it resolves albums by stable
ID, not by name.
Enhance now does the same thing for tracks: for every metadata source
the user has configured, if the library track has the corresponding
stored ID (spotify_track_id / deezer_id / itunes_track_id / soul_id),
call client.get_track_details(stored_id) directly and convert to the
wishlist payload. First success wins. The user's configured primary
source is tried first so a Deezer-primary user gets Deezer payloads on
the wishlist entry (correct cover art / album shape) even when other
sources also have stored IDs for the same track.
Multi-source parallel text search stays as the fallback for tracks
with no stored IDs (e.g. manually imported, never enriched). Empty-
field rejection still gates the wishlist add.
Implementation:
- _STORED_ID_COLUMNS: source name → DB column mapping
(Discogs intentionally omitted — release-based, no per-track IDs)
- _enhanced_to_wishlist_payload: converts the get_track_details
intermediate "enhanced" shape (artists as [str]) to wishlist shape
(artists as [{'name': str}]). Spotify's raw_data is already in
wishlist shape, returned as-is when detected (preserves full
album.images that the enhanced top-level fields drop)
- _try_direct_lookup_all_sources: iterates sources preferred-first,
calls get_track_details on each that has both a stored ID and a
configured client, returns first complete-metadata payload
- spotify_client field removed from ArtistQualityDeps (no longer
used — Spotify direct lookup now flows through the generic
per-source loop using the entry from search_sources)
- _try_upgrade_to_rich_payload removed (was Spotify-only with broken
shape semantics for non-Spotify sources; search-fallback now uses
_build_payload_from_track consistently)
- get_primary_source() consulted to set the per-call preferred source
for direct-lookup priority
Also fixed a stale UI string: the Enhance modal toast read "Matching
tracks to Spotify and adding to wishlist..." regardless of which
sources were actually configured. Now reads "Matching tracks across
metadata sources...".
Tests:
- _build_deps mirrors web_server._resolve_search_sources: passing
spotify=spotify_obj auto-prepends ('spotify', spotify_obj) to
search_sources (Spotify is always added when configured in prod)
- 5 new tests pin the direct-lookup behavior:
- test_direct_lookup_via_deezer_id_skips_text_search
- test_direct_lookup_via_itunes_id_skips_text_search
- test_direct_lookup_prefers_user_primary_source
- test_direct_lookup_falls_through_to_text_search_when_no_stored_ids
- test_direct_lookup_failure_falls_through_to_text_search
- Reframed enhanced-format and search-fallback tests for the new
payload-build path (no album-image side call, search-fallback uses
_build_payload_from_track consistently)
- 22/22 quality tests green, 2133/2133 full suite green.
|
||
|
Broque Thomas
|
7316646b01 |
Extract multi-source search; Enhance Quality matches Redownload coverage
Track Redownload had been doing parallel multi-source metadata search across every configured source the whole time; Enhance Quality was running a single-source primary fallback that returned junk matches with empty fields when the primary was iTunes (Discord report: "unknown artist - unknown album - unknown track" wishlist entries for users with neither Spotify nor Deezer connected). Lift the redownload search into core/metadata/multi_source_search.py and point both flows at it. Same scoring, same per-source query optimization (Deezer's structured artist:/track: form), same current-match flagging via stored source IDs. ArtistQualityDeps now takes get_metadata_search_sources (returns [(name, client), ...] for every configured source) instead of the single-primary get_metadata_fallback_client + get_metadata_fallback_source. Spotify direct-lookup stays as a fast-path optimization (only Spotify exposes get_track_details(id) returning rich raw payload); when it doesn't fire, the multi-source parallel search picks the cross-source best match. Empty-field matches still rejected before wishlist add. Tests: _build_deps helper updated to accept the new search_sources contract while preserving fallback_client/fallback_source ergonomics. Reframed tests for the new semantics — direct-lookup is no longer gated on Spotify being the active primary; failure reason now lists every searched source. Added a test pinning the no-sources-configured prompt. 17/17 quality tests green, 2128/2128 full suite green. |
||
|
Broque Thomas
|
4a27f3c245 |
Source-agnostic Enhance Quality flow + reject empty matches
Discord report: clicking Enhance Quality on an artist with neither
Spotify nor Deezer connected added tracks to the wishlist as
"unknown artist - unknown album - unknown track".
Root cause was structural. core/artists/quality.py had a hardcoded
Spotify-direct → Spotify-search → iTunes-fallback chain that ignored
the user's configured primary metadata source. When Spotify wasn't
connected, every track fell through to an iTunes-only fallback that
occasionally returned matches with empty fields (cleared the 0.7
confidence threshold but missing artist / album / title). Those
empty strings propagated through the wishlist payload normalizer's
truthy-check passthrough at core/wishlist/payloads.py:77-80 and the
UI rendered them as "Unknown" defaults.
Rewrote the flow source-agnostic:
- ArtistQualityDeps gains get_metadata_fallback_source. Flow resolves
the user's active primary source once up front.
- New _build_payload_from_track helper produces the Spotify-shaped
wishlist payload from any source's Track object — single place
that knows how to construct it (replaces the duplicate construction
in the Spotify-search and iTunes-fallback paths).
- New _search_match helper does generic confidence-scored search
against any client implementing search_tracks(query, limit). Same
0.7 threshold, same album-bonus weighting as before.
- New _has_complete_metadata validator rejects matches with empty
title / album / artists before they reach the wishlist.
- _spotify_direct_lookup kept as a Spotify-only optimization (only
Spotify exposes get_track_details(id) returning rich raw payload);
other sources fall through to search.
- Failure reason now names the active source: "No usable {source}
match — connect another metadata source for better coverage".
Result: Discogs users get a Discogs search. Hydrabase users get a
Hydrabase search. iTunes users get an iTunes search with empty-field
rejection. Spotify keeps its direct-lookup fast path.
6 new tests pin the architectural change:
- Primary-source dispatch routes to the configured client (Discogs,
not Spotify) when Spotify isn't primary
- Spotify direct-lookup is gated on Spotify being the active primary
(skipped when Discogs is configured even if track has spotify_track_id)
- Empty title / album / artists fields all reject the match
- Failure reason names the active source
|
||
|
Broque Thomas
|
b0dc139b72 |
Sync WHATS_NEW with current engine surface
The "Media Server Engine Foundation" entry was written when the engine still had safe-default routing wrappers for optional methods. Those were dropped in the honesty pass. Entry now matches reality: - Lists the actual engine surface (6 methods: client / active_client / active_server / is_connected / configured_clients / reload_config) instead of claiming "uniform safe defaults for optional methods" - References KNOWN_PER_SERVER_METHODS as the data-only listing (replaces the old OPTIONAL_METHODS naming) - Cites real test counts (42 total) instead of the stale 35 - Drops the "33+ dispatch sites" overclaim (was already partial); the actual framing is "uniform-shape chains lifted, ~18 server-specific chains stay explicit per the lift-what's-truly-shared standard" |
||
|
Broque Thomas
|
f230c93890 |
Merge remote-tracking branch 'origin/dev' into refactor/media-server-engine
# Conflicts: # core/matching_engine.py # services/sync_service.py # web_server.py # webui/static/helper.js |
||
|
Broque Thomas
|
edb6d1bc33 |
Drop dead per-server class imports + update WHATS_NEW
- services/sync_service.py: dropped unused PlexClient / JellyfinClient / NavidromeClient class imports. After the engine refactor the service only needs TrackInfo for type annotations; the class imports were dead. - WHATS_NEW: extended the media server engine review-pass entry to cover the followup commits (Cin-5 per-server global removal + Gap 1 shared types lift) so the changelog matches the actual branch state. |
||
|
Broque Thomas
|
d3f8a06d7a | WHATS_NEW entry for media server engine review pass | ||
|
Broque Thomas
|
2c0a0da9ea |
Address Copilot doc-drift review
Four stale doc/comment references caught by Copilot's pass:
- core/download_plugins/base.py: TYPE_CHECKING comment said the
shared dataclasses lived in core.soulseek_client. They were moved
to core.download_plugins.types in this PR. Comment updated.
- core/qobuz_client.py: reload_credentials docstring still referenced
soulseek_client.client('qobuz') after the global rename to
download_orchestrator. Updated to download_orchestrator.client(...).
- webui/static/helper.js: the older WHATS_NEW entries for the plugin
contract + engine refactor still claimed backward-compat
self.<source> attributes were preserved. Followup commits in the
same PR removed them. Each entry now flags the followup explicitly
and points at the "Drop Backward-Compat Per-Source Attrs" entry
above it so the changelog is internally consistent.
- docs/download-engine-refactor-plan.md: Compatibility commitments
section listed orchestrator.<source> attribute preservation as a
guarantee. Cin's review pass removed those attrs (and renamed the
global handle from soulseek_client to download_orchestrator) — both
are breaking changes for in-tree callers (which were migrated) and
in-flight branches (which will need to update). Section rewritten
to document the actual outcome.
|
||
|
Broque Thomas
|
2aff3dc210 |
Filter SoundCloud previews at every entry point + fix hybrid fallback regression
The earlier validation-only filter only ran in the auto-search scoring path. SoundCloud preview snippets still leaked through: - The candidate-review modal cached raw search results (pre-validation), so previews were visible and clickable for manual retry — and the manual-pick download path bypassed validation entirely, downloading the preview anyway. - The not-found raw-results cache stored unfiltered top-20s. Lift the preview filter into a reusable filter_soundcloud_previews() helper and apply it at every entry point: validation scoring (still), modal-cache fallback when validation drops everything, and the not-found raw-results path. Previews now never reach the cache, the matcher, or the manual-pick UI. Drops candidates < 35s or below half the expected duration, gated on expected > 60s so genuine short tracks still pass. 7 new unit tests pin the helper. Also fixed a silent regression in core/downloads/task_worker.py's hybrid-fallback path. Cin-5 dropped the per-source attrs from the orchestrator (orch.soulseek, orch.youtube, etc.), but the fallback loop still resolved sources via getattr(orch, '<src>', None) — every lookup silently returned None, so remaining_sources came back empty and the fallback never ran. Now uses orch.client(name) like the rest of the codebase. Updated the test fake to expose client() too — the old test was passing because the loop was effectively dead. |
||
|
Broque Thomas
|
563204ceae |
Drop SoundCloud preview snippets before scoring
SoundCloud serves a ~30s preview clip for tracks gated behind Go+ or login (extremely common for major-label uploads — what's actually on SoundCloud is bootlegs, fan reuploads, type beats, and these previews). yt-dlp accepts the preview as the download payload, the post-download integrity check catches the duration mismatch and quarantines the file, but the user only sees "all candidates failed" with no obvious explanation. Filter at validation time when we know expected_duration: drop SoundCloud candidates whose duration is below half the expected length OR within ~5s of the 30s preview boundary, gated on expected being non-trivially long (>60s) so genuinely short tracks still pass through. |
||
|
Broque Thomas
|
d17365296a |
Lift shared download dataclasses + boot via singleton factory
Two architectural cleanups on top of the download engine refactor. (1) Shared dataclasses move to neutral plugin package. TrackResult, AlbumResult, DownloadStatus, SearchResult lived in core/soulseek_client.py for historical reasons — every other plugin imported them from the soulseek module just to satisfy the contract, coupling 8 clients to a sibling source for type imports only. Moved them to the new core/download_plugins/types.py module and updated all 14 import sites across the deezer/hifi/lidarr/qobuz/soundcloud/tidal/ youtube clients, the engine, matching engine, redownload helper, and tests. Clean break, no backward-compat re-export. (2) web_server.py boots the orchestrator via the singleton factory. After construction it now calls set_download_orchestrator(...) so get_download_orchestrator() returns the same instance the global handle points at instead of lazily building a separate orchestrator. Matches the get_metadata_engine() pattern. |
||
|
Broque Thomas
|
61ba3a15de |
Cin-6: Rename soulseek_client global → download_orchestrator
The global handle in web_server.py was named soulseek_client for historical reasons but the type has long been DownloadOrchestrator, not SoulseekClient. Renamed the global plus every parameter/attribute that carried the legacy name. - web_server.py: global var renamed; all 99 references updated. - api/, core/downloads/*, core/search/*, core/streaming/*, services/sync_service.py: parameter names, dataclass fields, and init() arg names renamed. - Test fixtures (CandidatesDeps, MasterDeps, SearchDeps, etc.) and the _build_deps helpers updated accordingly. The core.soulseek_client module path and SoulseekClient class name (the actual soulseek-only client) are unchanged — only the orchestrator handle renamed. Module imports of TrackResult/AlbumResult/DownloadStatus from core.soulseek_client preserved. |
||
|
Broque Thomas
|
7519c3d50c |
Cin-5: Drop per-source attrs from orchestrator
Removed the eight backward-compat attribute aliases on the orchestrator
(soulseek, youtube, tidal, qobuz, hifi, deezer_dl, lidarr, soundcloud).
External callers and the orchestrator's own internals now reach clients
through the generic alias-aware client(name) accessor.
- core/downloads/{master,monitor,validation}.py: migrated to client().
Monitor's per-source aggregation loop replaced with a single
engine.get_all_downloads() call.
- core/search/{orchestrator,stream}.py: migrated; stream.py drops the
hand-built mode-to-client dict.
- web_server.py: migrated /api/deezer/arl-* + tidal client lookup.
- core/download_orchestrator.py: internal self.soulseek /
self.deezer_dl reaches now route through self.client(); attr
assignments dropped from __init__; module docstring updated.
- Test fakes (_FakeSoulseek, _FakeSoulseekWithYT) expose client(name)
instead of stuffing per-source attributes.
- Conformance test re-pinned to the client() accessor contract.
|
||
|
Broque Thomas
|
d0eac87601 |
Cin review: alias resolution, atomic terminal write, generic accessors
Three correctness fixes from kettui's PR review plus the web_server
migration to generic accessors.
- Engine alias map: register_plugin accepts aliases tuple; get_plugin
+ cancel_download resolve through it. Fixes deezer_dl cancels
silently routing to soulseek.
- Orchestrator hybrid_order normalization: _resolve_source_chain
routes raw config names through registry.get_spec() so legacy
deezer_dl entries don't drop deezer from hybrid mode.
- Atomic update_record_unless_state on the engine: holds state_lock
across the check + write. Both _mark_terminal AND the success path
use it now so a Cancelled state set mid-impl can't be clobbered.
- web_server.py: 30 soulseek_client.<source> reaches migrated to
client("<source>"); shutdown-check setup migrated to generic
registry iteration; 4 hifi reload sites use reload_instances('hifi').
- 18 new tests pin every fix.
|
||
|
Broque Thomas
|
650327ba18 |
Phase E: Add WHATS_NEW entry for media server engine refactor
Internal-track entry covering the media server engine + contract + the honest-scope note explaining why we lifted the 4 truly-uniform is_connected dispatches and left the deep server-specific dispatches explicit (each does fundamentally different work per server, so lifting would just move per-server branches into engine helper methods). |
||
|
Broque Thomas
|
95835b05ee |
H: Add WHATS_NEW entry for download engine refactor
Internal-track entry covering the engine package, background download worker, state lift, rate-limit policy declarations, and hybrid fallback chain. Mentions the ~700 LOC reduction + 85 new tests + zero behavior change. |
||
|
Broque Thomas
|
f9b763587d |
Add plugin conformance tests + WHATS_NEW entry
19 parametrized tests pin every registered plugin class's structural conformance to DownloadSourcePlugin: every required method present + async-ness matches the protocol. Drift in any source fails at the test boundary instead of at runtime against a live download. Class-level checks (not instance-level) — instantiating real clients in fixtures pollutes module state via tidalapi etc. imports and breaks downstream tests. |
||
|
Broque Thomas
|
749a772ff5 |
Findings tab: auto-switch to all-status when 0 pending exist
Companion to the badge count fix. When the findings tab opens with the default "pending" filter and returns 0 rows but other statuses (resolved/dismissed/auto-fixed) do have rows, the filter auto-switches to "All Status" and a small notice explains the switch. Stops the empty "all clear" state from masking carry-over findings from prior scans. |
||
|
Broque Thomas
|
cf5461f2f1 |
Fix: maintenance findings badge inflated when scan dedup-skipped
`_create_finding` silently dedup-skipped re-discovered issues but the caller incremented `findings_created` regardless. So a re-scan that found the same issues as a prior scan reported 364 findings in the badge while 0 NEW pending rows hit the db, leaving the findings tab empty. `_create_finding` now returns bool (True on insert, False on dedup-skip / db error). All 16 repair jobs updated to only increment `findings_created` on True. Added `findings_skipped_dedup` counter surfaced in scan log: "Done: X scanned, 0 fixed, 0 findings (363 already existed), 0 errors". Also fixed a missing `job_id` kwarg in album_tag_consistency that was silently breaking finding creation for that scan. |
||
|
Broque Thomas
|
77c54ab7a7 |
Migrate discography + quality scanner to typed Album path
Three more album-shape consumers now route through Album.from_<source>_dict() when caller passes a known source: - _build_discography_release_dict (artist discography cards) - _build_artist_detail_release_card (artist detail release cards) - _normalize_track_album (quality scanner result normalization) Legacy duck-typing stays as fallback for unknown source, non-dict input, or converter errors. Pure additive — existing callers without source kwarg unchanged. |
||
|
Broque Thomas
|
967c7f7c0a |
Migrate album-info builders to typed Album path
Steps 2+3 of typed metadata migration. Two album-info builders now route through Album.from_<source>_dict() when caller passes a known source: - _build_album_info (album-tracks lookups) - _build_single_import_context_payload (single-track import context) Legacy duck-typing stays as fallback for unknown source, non-dict input, or converter errors. Pure additive — existing callers without source kwarg unchanged. |
||
|
Broque Thomas
|
529486a2d1 |
Foundation: typed Album/Track/Artist + per-provider converters
New core/metadata/types.py with canonical dataclasses + classmethod converters for spotify/itunes/deezer/discogs/musicbrainz/hydrabase. Each converter is the single place that knows that provider's wire shape — addresses the duck-typing pattern Cin flagged. Pure additive: no consumer code changed. Follow-up PRs migrate consumers one at a time. Migration plan at docs/metadata-types-migration.md. Tests: 32 cases pin per-provider semantics + cross-provider invariants. Also stabilized a flaky discogs test that depended on local config state. |
||
|
Broque Thomas
|
4b23bee4a9 |
Add Discogs collection as a Your Albums source
Discord request: pull user's Discogs collection into the Your Albums
section on Discover, similar to how Spotify Liked Albums works.
Implementation extends the existing 3-source pipeline (Spotify /
Tidal / Deezer) to a 4-source pipeline with click-context dispatch —
Discogs-only albums open with rich Discogs release detail (vinyl/CD
format, year, label, country, tracklist). Mirrors the per-source
dispatch pattern from enhanced/global search.
Discogs client (`core/discogs_client.py`):
- New `get_authenticated_username()` resolves the username for the
configured personal token via Discogs's `/oauth/identity` endpoint.
Cached on the instance so subsequent collection page-fetches don't
re-hit it.
- New `get_user_collection(username=None, folder_id=0, per_page=100,
max_pages=50)` walks all pages of `/users/{username}/collection/
folders/{folder_id}/releases`. Returns normalized dicts ready for
upsert_liked_album. folder_id=0 = Discogs's "All" folder.
Pagination cap of max_pages*per_page = 5000 releases — bounds
runtime on heavy collections.
- New `get_release(release_id)` thin wrapper for `/releases/{id}` —
returns the raw API response so the album-detail endpoint can
render rich context.
- Both methods defensive: missing token → empty list, malformed
responses → skipped, falsy ids → None. Disambiguation suffix
stripping (`Madonna (3)` → `Madonna`) so Discogs artist names
match what Spotify/Tidal/Deezer use.
Schema (`database/music_database.py`):
- New `discogs_release_id TEXT` column on `liked_albums_pool`.
Migration uses the established `try SELECT, except ALTER TABLE`
pattern. Idempotent; safe on existing installs.
- Added the column to the canonical CREATE TABLE for fresh installs.
- `upsert_liked_album` extended with `'discogs': 'discogs_release_id'`
in BOTH the INSERT and UPDATE id-column maps so Discogs source_id
routes to the new column. INSERT statement column count + value
count updated together.
Backend (`web_server.py`):
- `/api/discover/your-albums/sources` — adds Discogs to the
`connected` list when `discogs.token` config is set.
- `_fetch_liked_albums` — new branch for Discogs. Lazy-imports
DiscogsClient, respects the `enabled_sources` config, walks the
collection, upserts each release. Same try/except shape as the
existing source branches.
- `/api/discover/album/<source>/<album_id>` — new `discogs` branch
fetches the release via DiscogsClient.get_release, normalizes the
Discogs tracklist format, parses Discogs's `MM:SS`/`HH:MM:SS`
duration strings to milliseconds, returns the same response shape
as the Spotify/Deezer/iTunes branches.
Frontend (`webui/static/discover.js`):
- `openYourAlbumsSourcesModal` — adds Discogs to `sourceInfo` with
the vinyl emoji icon. Existing toggle/save plumbing handles it.
- `openYourAlbumDownload` — restructured the per-source dispatch:
builds an ordered list of (source, id) tuples, tries each in turn,
breaks on the first successful response. Pure-Discogs albums go
straight to the Discogs detail endpoint → modal opens with Discogs
context. Multi-source albums prefer Spotify/Deezer first since
their tracklists carry proper streaming IDs ready for download.
Tests: `tests/test_discogs_collection_source.py` — 12 cases:
- get_user_collection: empty without token, normalizes response
shape, strips disambiguation suffix, handles missing year, skips
malformed releases, paginates correctly, caps at max_pages,
uses explicit username when provided.
- get_release: passes id through to /releases/{id}, returns None
for invalid ids without API call.
- liked_albums_pool: discogs_release_id round-trips through upsert
+ get; multi-source dedup carries both Spotify and Discogs IDs
on the same row.
Verified: full suite 1825 pass (12 new), ruff clean, smoke test
populating + reading the discogs_release_id column round-trips
correctly via the real DB.
WHATS_NEW entry under '2.4.2' dev cycle.
|
||
|
Broque Thomas
|
e84d187e76 |
Drop redundant standalone "Your Spotify Library" section on Discover
Discover page used to show two near-identical sections: - "Your Albums" — cross-source aggregator across Spotify / Deezer / etc with a gear button to configure sources, search, status filter, sort options, and a download-missing action. - "Your Spotify Library" — Spotify-only with the same grid UI, same refresh / download-missing buttons, same filter / sort controls. The Spotify-only section was a strict subset of what Your Albums already covers (Spotify is one of the configurable sources). User flagged the redundancy when scoping the upcoming Discogs integration and asked for the duplicate to be removed. Removal scope: - `webui/index.html` — drop the `#spotify-library-section` block (42 lines). - `webui/static/discover.js` — drop the dead JS (~335 lines): state vars `spotifyLibraryAlbums` / `spotifyLibraryPage` / etc, all the loaders / renderers / pagination / click handlers, and the `loadSpotifyLibrarySection()` call in `loadDiscoverPage`'s Promise.all. - `webui/static/helper.js` — drop the helper annotation entry at `#spotify-library-section` and the matching guided-tour entry. Backend untouched. The Spotify saved-albums cache (`spotify_library_albums` table + watchlist_scanner upsert/cleanup + `/api/discover/spotify-library` endpoint + the DAO methods) is shared infrastructure that Your Albums reads from when Spotify is one of its configured sources. Removing the UI section just removes the duplicate surface — Spotify saved albums still appear in Your Albums via the existing source dispatch. CSS class names (`.spotify-library-grid`, `.spotify-library-search`, `.spotify-library-pagination`) intentionally remain on the surviving Your Albums elements — they share the same visual styling and renaming would be churn for no benefit. Verified: full suite 1813 pass (no new tests — pure UI/dead-code removal). Backend endpoint behavior unchanged. WHATS_NEW entry under '2.4.2' dev cycle. |
||
|
Broque Thomas
|
2ab460f5c4 |
Add Library Disk Usage card to System Statistics
Discord request (Samuel [KC]): show how much disk space the library
takes on the Stats page. Implementation piggybacks on the existing
deep scan — Plex/Jellyfin/Navidrome all return file size in their
track API responses, so we read it during the deep scan and store
it on the tracks row. Aggregation is then a single SQL query — no
filesystem walk, no extra I/O during the scan, no separate stat
job. SoulSync standalone gets size from os.path.getsize at insert
time (different code path; the file is local when we write the row).
Schema (`database/music_database.py`):
- New `file_size INTEGER` column on `tracks`. Migration uses the
established `try SELECT, except ALTER TABLE ADD COLUMN` pattern.
Idempotent; safe on existing installs. NULL on legacy rows so
they don't contribute to totals until next deep scan refreshes.
- Added the column to the canonical CREATE TABLE so fresh installs
get it without going through the migration path.
Track-object plumbing:
- `core/jellyfin_client.py` — JellyfinTrack reads MediaSources[0].Size
alongside existing Bitrate read. None when 0 / missing.
- `core/navidrome_client.py` — NavidromeTrack reads `size` from
the Subsonic song object (int coercion + None on parse fail).
- `core/soulsync_client.py` — SoulSyncTrack does os.path.getsize
(only "server" where size has to come from disk).
- Plex needs no client-side change: track.media[0].parts[0].size
is read directly inside insert_or_update_media_track.
Persistence — TWO separate insert paths:
(a) `database/music_database.py:insert_or_update_media_track` —
Plex/Jellyfin/Navidrome flows. Reads file_size from Plex's
MediaPart OR `track_obj.file_size` wrapper attribute (defensive
Plex-attr-not-present check + > 0 type guard).
INSERT writes the new column.
UPDATE uses COALESCE(?, file_size) so a None from the server
on a re-sync (rare Jellyfin Size omission) doesn't blank an
existing value. Pinned via test.
(b) `core/imports/side_effects.py:record_soulsync_library_entry` —
SoulSync standalone flow. Completely separate code path: the
standalone deep scan moves files to staging for auto-import
rather than calling insert_or_update_media_track. After the
auto-import processes them, side_effects writes the tracks row
directly. Reads file_size via os.path.getsize(final_path) at
insert time (file is local) and includes it in the INSERT
column list. SoulSync only does INSERT-if-not-exists (no
UPDATE path), so no COALESCE concern.
Aggregator (`database/music_database.py:get_library_disk_usage`):
- SELECT COALESCE(SUM(file_size), 0), COUNT(file_size),
COUNT(*) - COUNT(file_size) for the totals.
- Per-format breakdown done in Python via os.path.splitext over
(file_path, file_size) rows — sidesteps SQLite's first-vs-last-dot
ambiguity for paths like /music/Kendrick/M.A.A.D City/01.flac.
- Defensive: skips empty paths, paths without extension, and
implausibly long extensions (>6 chars). Returns the full
empty-shape dict (NOT a partial / undefined) when the column
doesn't exist or queries fail, so the UI's `if (!data.has_data)`
branch handles fresh installs cleanly.
API + UI:
- `core/stats/queries.py` — thin pass-through get_library_disk_usage
matching the existing query-helper convention.
- `web_server.py` — new /api/stats/library-disk-usage endpoint
mirroring the /api/stats/db-storage pattern.
- `webui/index.html` — new card in System Statistics above the
Database Storage card.
- `webui/static/stats-automations.js` — _loadLibraryDiskUsage +
_renderLibraryDiskUsage. Empty state: "Run a Deep Scan to
populate (X tracks pending)". Partial: "X measured (+Y pending)".
Full: total + format bars proportional to the largest format.
- `webui/static/style.css` — .stats-disk-* styled to match the
Database Storage card.
Backward compatibility:
- Migration is additive; existing rows get NULL file_size; the
empty-shape return from the aggregator means the UI renders
cleanly without errors before any deep scan runs.
- Old installs upgrading will see "Run a Deep Scan to populate
(N tracks pending)". Running their next deep scan fills sizes —
the existing scan flow doesn't need any changes, just consumes
the new track-wrapper attribute.
Tests:
- `tests/test_library_disk_usage.py` — 13 cases covering schema
migration, NULL defaults on legacy inserts, fresh-install empty
shape, summing with mixed NULL/known sizes, per-format breakdown,
mixed-case extensions, paths with album-name dots, missing
extensions, empty file_path, implausibly long extensions,
JellyfinTrack.file_size persistence via insert_or_update_media_track,
COALESCE preservation on null re-sync.
- `tests/imports/test_import_side_effects.py` — extended the
existing record_soulsync_library_entry test to assert
track_row['file_size'] == os.path.getsize(final_path), pinning
the SoulSync-standalone path. Test fixture's tracks schema also
updated to include the file_size column.
Verified: full suite 1813 pass (13 new, 1 existing-test extension),
ruff clean, smoke test populating + reading the column round-trips
correctly.
WHATS_NEW entry under '2.4.2' dev cycle.
|
||
|
Broque Thomas
|
776d195f71 |
Fix: ReplayGain wrote same +52 dB gain to every track
User report: every downloaded track in an album came out with
``replaygain_track_gain: +52.00 dB`` regardless of actual loudness.
Root cause: the parser at ``core/replaygain.py:79`` used
``re.search('I:\s+...')`` which returns the FIRST match. ffmpeg's
ebur128 filter emits ``I:`` per measurement window (running partial
integrated loudness) AND in a final Summary block. The first
per-window reading is at t=0.5s — almost always ~-70 LUFS because
nearly every track starts with silence / encoder padding. So:
gain = RG2_reference - lufs = -18 - (-70) = +52.00 dB
…on EVERY track. Same regex pattern, same first per-window match,
same +52 dB written to every file's REPLAYGAIN_TRACK_GAIN tag.
Verified by running ffmpeg ebur128 against a real generated FLAC
and inspecting the stderr output — first per-window line at t=0.5s
shows ``I: -70.0 LUFS`` (silent intro), and the Summary block at
the end shows the real integrated value (e.g. ``I: -27.8 LUFS``
for the test sine wave). Old code captured the -70.0 reading.
Fix: anchor LUFS parsing to the ``Summary:`` block via
``stderr.rfind('Summary:')``. The Summary block is always emitted
last and contains the authoritative final integrated loudness.
Peak parsing already worked correctly (per-window output uses
``TPK:``/``FTPK:`` labels; only the Summary uses ``Peak:``), but
applied the same Summary anchor for consistency.
Defensive fallback: if no Summary block is present (truncated
output / unusual ffmpeg version), use the LAST per-window reading
instead of the first. Still better than the buggy first-window
behavior.
Smoke verified end-to-end: a freshly-generated FLAC of a -24 dBFS
sine wave now reports LUFS=-27.80, gain=+9.80 dB (correct, was
+52.00 before fix).
Tests: ``tests/test_replaygain_summary_parse.py`` — 7 cases pinning
the parser behavior with realistic ffmpeg ebur128 stderr samples:
- Summary value parsed correctly even when first per-window is -70
- Resulting gain is realistic (NOT +52)
- Two tracks with same first per-window but different summaries get
different LUFS (regression assertion for "all tracks same gain")
- Per-window reading higher than Summary doesn't leak through
- Fallback to last per-window when Summary absent
- Clean RuntimeError raised when no LUFS values anywhere
- Peak still correctly anchored to Summary
Verified: full suite 1800 pass (7 new), ruff clean.
WHATS_NEW entry under '2.4.2' dev cycle.
|
||
|
Broque Thomas
|
04a14f7e96 |
Fix: tasks showed Completed when file was quarantined
User caught downloading Kendrick Mr. Morale: three tracks (Rich Interlude, Savior Interlude, Savior) showed ✅ Completed in the modal but were missing on disk. Log forensics revealed two layered bugs. Bug 1 — Verification wrapper assumed success on quarantined files (`core/imports/pipeline.py`): The outer `post_process_matched_download_with_verification` had a fallback at the "no `_final_processed_path` in context" branch that marked the task completed and notified `success=True`. The inner post-processor sets `_final_processed_path` only when the file actually reaches its destination. Integrity-rejected files (`_integrity_failure_msg` set) and race-guard-failed files (`_race_guard_failed` set) get quarantined or skipped without ever setting `_final_processed_path`, so they fell straight into the "assume success" branch. Confirmed in user's log: No _final_processed_path in context for task d5b88b84-... — cannot verify, assuming success That line fired for the same task right after the integrity check quarantined the source file. Result: ✅ Completed in UI, file in quarantine, never delivered. Fix: explicit checks for `_integrity_failure_msg` and `_race_guard_failed` markers BEFORE the assume-success fallback. Either marker set → task status='failed' with descriptive error_message + `_notify_download_completed(success=False)`. The pre-existing assume-success behavior preserved when no failure markers are set (some legitimate flows complete without setting `_final_processed_path`). Bug 2 — AcoustID skip-logic too lenient (`core/acoustid_verification.py`): The "language/script" exemption was: if best_score >= 0.95 and (title_sim >= 0.55 or artist_sim >= ARTIST_MATCH_THRESHOLD): The OR-clause fired for English-vs-English titles by the same artist that share NO actual content. Confirmed in user's log: requested "Rich (Interlude)" by Kendrick Lamar, AcoustID identified the audio as "R.O.T.C. (interlude)" by Kendrick Lamar (a totally different song from his 2010 mixtape) — same artist scored ≥ARTIST threshold, shared word "interlude" pushed title_sim above 0.55, skip fired. Verification returned SKIP instead of FAIL, the wrong file was accepted as the answer for three different track requests. Fix: skip now requires positive evidence the mismatch is a real language/script case: (a) Non-ASCII chars present in either title AND artist matches strongly → real transliteration case (kanji ↔ romaji etc) (b) BOTH title_sim >= 0.80 AND artist_sim >= ARTIST threshold → minor punctuation/casing differences English-vs-English with very different titles by the same artist no longer skipped — verification correctly returns FAIL, the wrong file gets quarantined, the new wrapper logic above marks the task failed. Tests: - `tests/test_integrity_failure_marks_task_failed.py` — 4 cases pinning the wrapper-level state machine: integrity marker → failed, race-guard marker → failed, no markers → still assumes success (legacy path preserved), integrity-failure-takes-priority over missing-final-path fallback. - `tests/test_acoustid_skip_logic.py` — 7 cases pinning the skip exemption: user's R.O.T.C-vs-Rich case → FAIL (regression test), Savior-vs-R.O.T.C → FAIL (same bug surface), Japanese kanji → romaji → SKIP (real language case still works), MAAD vs M.A.A.D → PASS or SKIP (punctuation tolerance), low fingerprint score → never skipped, high score but artist mismatch → no longer skipped, Crown vs Crown of Thorns → no longer skipped. Verified: full suite 1793 pass (11 new), ruff clean. WHATS_NEW entry under '2.4.2' dev cycle. |
||
|
Broque Thomas
|
4b15fe0b75 |
Fix album MBID inconsistency: detector + persistent release-MBID cache
Discord report (Samuel [KC]): tracks of the same album sometimes carry different MUSICBRAINZ_ALBUMID tags, which causes Navidrome (and other media servers grouping by album MBID) to split the album into multiple entries. Two-part fix — one for existing libraries, one for the root cause that lets new imports drift. Part 1 — Detector + fix action (catches existing dissenters): `core/repair_jobs/mbid_mismatch_detector.py`: - New helpers: `_read_album_mbid_from_file` and `_write_album_mbid_to_file` use the Picard-standard tag conventions (`TXXX:MusicBrainz Album Id` for MP3, `MUSICBRAINZ_ALBUMID` for FLAC/OGG, `----:com.apple.iTunes:MusicBrainz Album Id` for MP4). - New scan phase `_scan_album_mbid_consistency` runs after the existing track-MBID scan: groups tracks by DB `album_id`, reads each track's embedded album MBID, finds the consensus (most-common) MBID via `Counter`, flags dissenters. Tracks without an album MBID at all are skipped (they don't break Navidrome — only an explicit MBID disagreement does). Albums where MBIDs are perfectly tied (no clear consensus) are skipped too — surface as a manual decision instead of fixing toward a 1/N tie. - New finding type `album_mbid_mismatch` carries `consensus_mbid`, `wrong_mbid`, `consensus_count`, `total_tracks_with_mbid`, and a human-readable reason string. `core/repair_worker.py`: - Added `'album_mbid_mismatch': self._fix_album_mbid_mismatch` to the fix dispatch dict and to the `fixable_types` tuple so auto-fix + bulk-fix paths pick it up. - New `_fix_album_mbid_mismatch` method reads `consensus_mbid` from finding details, resolves the dissenter's file path via the shared library resolver, calls `_write_album_mbid_to_file` to rewrite the tag in place. Doesn't touch the album's other tracks (they're already in agreement). Part 2 — Root cause fix (prevents new SoulSync imports from drifting): The original in-memory `mb_release_cache` in `core/metadata/source.py` maps `(normalized_album, artist) -> release_mbid` so per-track enrichment of the same album hits the cache and writes the same MUSICBRAINZ_ALBUMID to every track. That cache is bounded (4096 entries) and in-process — so cache eviction (when other albums are processed in between) and server restart can BOTH cause inconsistency. Per-track album-name variation (e.g. some tracks tagged `"Album"`, others tagged `"Album (Deluxe)"`) and per-track artist variation (features) make it worse. `core/metadata/album_mbid_cache.py` (new module): - DB-backed `lookup(normalized_album, artist) -> release_mbid` and `record(...)` functions. Same key shape as the in-memory cache. - Strict additive design: every public function is wrapped in try/except and degrades to None / no-op on ANY database error. The existing in-memory cache + MusicBrainz lookup remains the authoritative fallback. If this module breaks, downloads continue exactly as they would today. `database/music_database.py`: - New `mb_album_release_cache` table with composite primary key `(normalized_album_key, artist_key)`. Reverse-lookup index on `release_mbid` for future debug tooling. Created via the existing `CREATE TABLE IF NOT EXISTS` migration pattern — idempotent, no schema version bump needed. `core/metadata/source.py`: - Surgical change inside the existing `embed_source_ids` in-memory-cache-miss branch: BEFORE calling MusicBrainz, consult the persistent cache. If a previous SoulSync run already resolved this album's release MBID, reuse it. After a successful MB lookup, store in BOTH caches. Both calls wrapped in defensive try/except so any failure falls through to existing logic. Tests: - `tests/metadata/test_album_mbid_cache.py` — 16 cache tests: round-trip, idempotent re-record, overwrite semantics, clear_all, album+artist independence (no Greatest Hits collisions), defensive None-on-empty-input, graceful degradation when the DB is unavailable / connection raises / commit fails, schema sanity (table + index exist after init). - `tests/test_album_mbid_consistency.py` — 13 detector tests: tag read/write round-trip on real FLAC files, Picard-standard tag descriptors, defensive paths (unreadable file, empty input), detector behavior (agreement → no flags, lone dissenter → flag, ties → no flag, single-track albums → skipped, no-MBID tracks → skipped, unresolvable file paths → skipped). - `tests/metadata/test_metadata_enrichment.py` — added autouse fixture monkeypatching the persistent cache to no-op for tests in this file. The existing tests pin per-call MB counts and in-memory cache state; without the fixture, persistent rows from earlier tests would bypass the MB call. Persistent layer has its own dedicated tests. Verified: 1782 tests pass (29 new), ruff clean, smoke test confirms end-to-end cache round-trip works. WHATS_NEW entry under '2.4.2' dev cycle. |
||
|
Broque Thomas
|
e577f3cf1f |
Fix three Lidarr bugs that prevented it from being a real download source
Investigation surfaced that Lidarr was wired into the orchestrator but
the actual download flow had blockers:
1. **Wrong file misfiled.** Lidarr grabs whole albums; SoulSync's
matched-context post-processing wants the SPECIFIC track the user
requested. Old code copied every track in the album and reported
`imported_files[0]` as `file_path` — almost always pointing to
track 1, not the user's actual track. Post-processing then tagged
track 1 with the requested track's metadata. Misfiling on every
real download.
Fix: parse the wanted track title out of the dispatch display name
(which `_search_sync` already builds as
`f"{artist} - {album} - {track_title}"`), look it up against
Lidarr's `track` API, resolve the matching `trackFileId` to a path,
and copy ONLY that file. Punctuation-tolerant fuzzy match handles
the common "m.A.A.d city" vs "maad city" case. Album-level
dispatches (no track in the display) preserve the old first-file
fallback so existing album-grab UX is unchanged.
2. **Hardcoded `metadataProfileId=1`.** Required by Lidarr's
artist-add API. On installs where the user deleted/recreated
metadata profiles, that id no longer exists and the call fails
with HTTP 400 — which silently breaks every download flow that
needs to add an artist. Real-world Lidarr installs do this all
the time.
Fix: `_get_metadata_profile_id()` calls Lidarr's `metadataprofile`
API and returns the first available id. Falls back to 1 only when
the API call fails entirely (preserves previous behavior so this
change can't make things worse).
3. **Polling never broke the outer loop on completion.** The inner
`for item in queue['records']` had `break` statements at status
transitions, but those only escaped the queue iteration — the
outer `for poll in range(max_polls)` kept spinning until the
600-poll timeout even after the album was clearly imported.
`for/else` semantics didn't apply because completion was detected
inside the inner loop, not by it running to exhaustion.
Fix: replaced with an explicit `download_complete` flag set when
`album/{id}` reports `trackFileCount > 0` (the authoritative
completion signal — works even when the queue record disappeared
between polls). Outer loop breaks immediately once the flag flips.
Helper functions added: `_extract_wanted_track_title` (staticmethod,
splits the display name; >=3 parts → track dispatch, 2 parts → album
dispatch), `_normalize_for_match` (lowercase + strip punctuation +
collapse whitespace for fuzzy compare), `_title_similarity` (cheap
score: equal=1.0, substring=0.85, token-overlap-ratio otherwise),
`_pick_track_file_for_wanted` (orchestrates the API calls).
Settings tooltip updated to be honest about Lidarr's natural shape:
album-grabber, no-op for playlist sync, hybrid mode falls through to
other sources for track searches. Sets correct expectations.
Tests: `tests/test_lidarr_download_client.py` — 21 isolated tests
covering pure helpers (title extraction, normalization, similarity)
and the file-picker integration paths (matching path, punctuation
tolerance, below-threshold fallback, missing trackFileId, missing
file on disk, API failures, malformed responses). No live Lidarr
needed — `_api_get` mocked at the client boundary.
Isolation: ONLY touches `core/lidarr_download_client.py`, the Lidarr
settings tooltip in `webui/index.html`, the Lidarr WHATS_NEW entry
in `webui/static/helper.js`, and the new test file. No changes to
the orchestrator, other download clients, the import pipeline,
side_effects, web_server.py, settings.js, or any shared validation /
monitor / task_worker code. Other download sources are not affected
in any way.
Verified: 1753 tests pass (21 new), ruff clean.
|
||
|
Broque Thomas
|
75fe04907f |
Wire SoundCloud as a first-class download source
Plug the previously-built SoundcloudClient (PR #478, the build-and-verify phase) into every place a download source needs to appear. Follows the same wiring contract as Tidal/Qobuz/HiFi/Deezer/Lidarr — orchestrator routing, hybrid-mode picker, search dispatch, queue/cancel/clear, provenance + library history, sidebar source label, settings UI all work plug-and-play. Backend wiring: - `core/download_orchestrator.py` — import SoundcloudClient, _safe_init it at startup, add to _client() lookup, get_source_status(), check_connection's sources_to_check default, search source_names map, search_and_download_best _streaming_sources tuple, download source_map + source_names, and every iteration loop in reload_settings download-path-update / get_all_downloads / get_download_status / cancel_download (route + iterate) / clear_all_completed_downloads / cancel_all_downloads. - `core/downloads/monitor.py` — added SoundCloud to the per-client loop that fetches active downloads outside the orchestrator (uses getattr fallback for older soulseek_client snapshots). - `core/downloads/task_worker.py` — added SoundCloud (and Lidarr, which was missing too — bonus fix) to source_clients dict for hybrid fallback dispatch. - `core/downloads/validation.py` — added 'soundcloud' to _streaming_sources so SoundCloud results go through the matching engine validation path instead of the Soulseek quality-filter path. - `core/imports/side_effects.py` — three call sites: source_map for download_source label written to library_history, streaming-source guard for the `||`-encoded stream_id parsing, and source_service map for provenance recording. All three now include 'soundcloud'. - `web_server.py` — five streaming-source detection tuples updated. New `/api/soundcloud/status` endpoint returns {available, configured, reachable} mirroring the Deezer/HiFi status-endpoint pattern; reachability runs a real cheap yt-dlp search so the settings Test Connection button gives a meaningful pass/fail signal. - `config/settings.py` — added empty `soundcloud_download` defaults block so future tier-2 OAuth (SoundCloud Go+ session) doesn't have to migrate existing configs. Frontend: - `webui/index.html` — new `<option value="soundcloud">` in the download-source-mode dropdown, SoundCloud added to both hidden legacy hybrid-source selects, new settings container with info text + Test Connection button. - `webui/static/settings.js` — HYBRID_SOURCES entry (with the SoundCloud cloud SVG icon), _hybridSourceEnabled default, updateDownloadSourceUI container display, allSources for legacy hybrid picker, testSoundcloudConnection function (hits the new status endpoint, color-codes the result), saveSettings soundcloud_download empty block. - `webui/static/shared-helpers.js` — sidebar source-name map includes SoundCloud + Lidarr (Lidarr was also missing, bonus fix). - `webui/static/helper.js` — WHATS_NEW entry under '2.4.2' dev cycle describing the user-visible change in the chill terse voice. Tests: - `tests/test_download_orchestrator_soundcloud.py` — 14 integration tests verifying the wiring: client constructed at startup, _client lookup resolves 'soundcloud', get_source_status includes it, download dispatcher routes username='soundcloud' to the SoundCloud client (and unknown usernames still fall back to Soulseek), hybrid search iterates SoundCloud when in order and skips it cleanly when unconfigured, get_all_downloads / get_download_status / cancel / clear walk SoundCloud, soundcloud-only mode dispatches only to SoundCloud, _streaming_sources tuple in validation includes 'soundcloud'. - `tests/downloads/test_download_orchestrator.py` — added `soundcloud` to the test fixture's _build_orchestrator helper so the new orchestrator attribute doesn't AttributeError in pre- existing tests that bypass __init__. Verified: - Full suite green (1728 passed, 2 deselected for soundcloud_live) - Ruff clean - Live SoundCloud-only mode search returns 25 SoundCloud tracks for "kendrick lamar luther" in <2s, returning properly-shaped TrackResult objects with username='soundcloud' and dispatch-key filename ready for the download path. Out of scope (intentional deferrals): - SoundCloud Go+ OAuth tier (256 kbps AAC) — anonymous-only for now. Adding auth later is a settings-page extension, no orchestrator changes needed. - Album/playlist support — SoundCloud has playlists but they don't map to the album model the rest of SoulSync expects. Singles only. |
||
|
Broque Thomas
|
d8437c87c6 |
Fix Album Completeness Auto-Fill on Docker / shared-library setups (#476)
GitHub issue #476 (gabistek, Docker on Arch host): "Auto-Fill" / "Fix Selected" on the Album Completeness findings page returned "Could not determine album folder from existing tracks" for every album. Reproduces on any setup where the media-server library lives outside the SoulSync transfer/download folders — Docker is the headline case but native installs that point Plex at a NAS via SMB hit it too. Root cause: `core/repair_worker.py:_resolve_file_path` only probed the transfer + download folders. Docker users have their Plex/Jellyfin library bind-mounted at /music (or similar) — neither configured in SoulSync. Every existing track got silently treated as missing, so `album_folder` stayed None and the fix workflow bailed. The same incomplete logic was duplicated four more times in the repair_jobs/ modules, all with the same bug. Album Completeness was just the most user-visible — the same setups were also producing false "missing file" findings from Dead File Cleaner, silent skips in MBID Mismatch Detector, etc. The web server already had the correct logic at `web_server.py:_resolve_library_file_path` (probes transfer + download + Plex-reported library locations + user-configured library.music_paths). The repair workers had never been updated to match. Fix: - New `core/library/path_resolver.py` extracts the union logic into a single shared function `resolve_library_file_path()`. Probes (in order, deduped): explicit transfer/download kwargs, config-derived soulseek.transfer_path/download_path, Plex-reported library locations (when a plex_client is passed), user-configured library.music_paths. Each defensive: malformed config or a flaky Plex client degrades to the dirs that did succeed. - `core/repair_worker.py:_resolve_file_path` becomes a delegating wrapper preserving the legacy signature, with a new `config_manager` kwarg. All 15 in-tree call sites updated to thread `self._config_manager` through. - `core/repair_jobs/dead_file_cleaner.py`, `mbid_mismatch_detector.py`, and `lossy_converter.py` get the same treatment: duplicate function replaced with a thin wrapper, call sites pass `context.config_manager`. - `core/repair_jobs/acoustid_scanner.py` and `unknown_artist_fixer.py` (which used to import from repair_worker) now call the shared resolver directly with `context.config_manager`. Side benefit: every other repair job (Dead File Cleaner, MBID Mismatch Detector, Lossy Converter, AcoustID Scanner, Unknown Artist Fixer) also stops missing files in the media-server library mount. Single fix unblocks five user-visible features. Tests: `tests/library/test_path_resolver.py` — 20 cases covering all four base-dir sources, suffix-walk algorithm, dedup, defensive paths (None plex client, malformed config entries, raising config_manager.get, broken plex attribute access), Docker path translation. Full suite 1677 passed locally. WHATS_NEW entry under '2.4.2' dev cycle. |
||
|
Broque Thomas
|
42f3026eef |
Reject broken downloads before tagging via universal integrity check
Discord report (fresh.dumbledore [VRN]): slskd sometimes ships broken files
(truncated transfers, corrupt FLAC, wrong file substituted on filename match).
They flowed through post-processing and only surfaced later — Plex/Jellyfin
scan failures, dead-air playback, duplicate detector tripping over the wrong
length. By that point the file was already tagged, copied, mirrored to the
media server, and recorded in provenance.
New module `core/imports/file_integrity.py`:
- `check_audio_integrity(path, expected_duration_ms=None) -> IntegrityResult`
- Three tiered checks, cheapest to most expensive:
1. File size sanity (catches 0-byte stubs and stub transfers)
2. Mutagen parse (catches header damage, wrong-format-with-right-extension)
3. Duration agreement vs. metadata source's expected length, ±3s tolerance
(5s for tracks over 10 minutes — long tracks naturally drift more)
- Returns IntegrityResult with `ok`, human-readable `reason`, and per-check
`checks` dict for debugging
- Never raises; pathological inputs return ok=False with explanation
Pipeline integration in `core/imports/pipeline.py:post_process_matched_download`:
- Hooks between the existing file-stability wait and AcoustID verification
- On failure: quarantine via existing `move_to_quarantine` helper, mark task
failed with descriptive error, clear matched-context, fire
`on_download_completed(success=False)` so the slot is released for retry
- Mirrors the existing AcoustID-failure path so retry behavior stays consistent
- Wrapped in try/except so an unexpected failure inside the check itself
cannot block downloads — logs and continues
This is intentionally tier 1: universal across formats, no external deps.
A future tier could verify FLAC STREAMINFO MD5 by decoding audio (needs
flac binary or libflac wrapper) — skipped for now since tier 1 catches the
dominant Discord-reported cases (truncated, 0-byte, wrong file).
Tests:
- `tests/imports/test_file_integrity.py` — 14 cases covering all three check
tiers, edge cases (zero/negative expected duration, long-track wider
tolerance, caller tolerance override), and the mutagen-unavailable
degradation path
- `tests/imports/test_import_pipeline.py` — two existing tests use 5-byte
fixture files that the new check would reject; they monkeypatch the
integrity check since they're testing plumbing (notification +
metadata_runtime forwarding), not integrity behavior
WHATS_NEW entry under '2.4.2' dev cycle.
|
||
|
Broque Thomas
|
cdd408b6f3 |
Auto-import: live card updates + multi-disc + featured-artist tag fixes
The 'Live Per-Track Progress' work shipped a backend in-progress row + top-of-tab
progress text but the history cards themselves stayed visually stale during
processing — lowercase "processing" badge, neutral styling, no per-track hint.
Smoke-testing also surfaced two latent identification bugs that prevented
multi-disc rips with features (Kendrick GKMC Deluxe) from importing at all.
Card-level live progress (`webui/static/stats-automations.js`):
- Cache `/api/auto-import/status` response in `_autoImportLastStatus`; poller
awaits status before re-rendering results so the card has the live data.
- Add 'processing' entries to statusLabels / statusIcons / statusClass.
- When card folder_name matches `current_folder`, swap the meta line to
`track N/M: <track name>` and tag the matching row in the expanded list
as `auto-import-track-row-active`; prior rows tag as `-row-done`.
Card styling (`webui/static/style.css`):
- `.auto-import-processing` blue left border, `.auto-import-badge-processing`
pulse animation, active/done track-row classes.
Multi-disc enumeration (`core/auto_import_worker.py:_scan_directory`):
- Old code skipped disc folders during recursion AND only attached them to a
parent that had its own loose audio. A folder containing only `Disc 1/`,
`Disc 2/` was invisible. Now: when a directory has only disc subdirs and no
loose audio, treat that directory itself as the album candidate. Disc folders
still skipped when standing alone.
- Add `FolderCandidate.is_staging_root` flag (set when the staging dir itself
becomes the candidate via this path) so identification can refuse to use the
meaningless folder name.
Tag identification (`core/auto_import_worker.py:_identify_from_tags`):
- Per-track `artist` tag fragmented consensus on albums with features
("Kendrick Lamar" / "Kendrick Lamar, Drake" / "Kendrick Lamar, Dr. Dre"
produced 3 separate `(album, artist)` keys for one album). Now group by
album first, then pick the most-common artist within that album group.
- `_read_file_tags` now prefers `albumartist` over `artist` for album-level
identity; falls back to `artist` for files without albumartist.
- Add INFO-level log when tag identification rejects, showing top albums and
their counts so the user can diagnose multi-disc / tagging issues.
Folder-name false-match guard (`core/auto_import_worker.py:_identify_folder`):
- When `is_staging_root` is set, skip the folder-name strategy entirely. Logs
the skip and falls through to AcoustID. Without this, dropping disc folders
directly into staging caused the scanner to search the metadata source for
the literal name "Staging", which false-matched against random albums (e.g.
"Stamina, Dinos" — a French rap album — at 13% confidence).
What's New entries added under 2.4.2 dev cycle.
|
||
|
Broque Thomas
|
783c543c3e |
Auto-import: live per-track progress + in-progress history row
User reported (Mushy / generally) that dropping an album into the staging folder left the auto-import history blank for the entire processing window — sometimes 5+ minutes for a full album. Pre- existing UX gap, not caused by the recent context-builder refactor. Two root causes: 1. ``_record_result`` only fired AFTER ``_process_matches`` returned. For a 14-track album with ~30s/track post-processing, that meant ~7 minutes of zero rows in auto_import_history → nothing for ``/api/auto-import/results`` to return → empty UI. 2. ``_current_status`` only ever transitioned between 'idle' and 'scanning' — never 'processing'. ``get_status()`` had no per- track index/name fields, so the UI had no way to render "Processing track 3/14: Mine" even if it wanted to. Fix: - New ``_record_in_progress`` inserts a status='processing' row up-front (before the per-track loop starts) so the UI sees the import the moment it begins. Returns the row id. - New ``_finalize_result`` updates that same row with the final outcome (completed/failed) when processing finishes. One row per album, not per track — keeps the history list clean. - Both share ``_serialize_match_data`` (extracted from the original ``_record_result``) so the in-progress row carries the same match payload shape the existing review UI already understands. - ``_process_matches`` updates ``_current_track_index``, ``_current_track_total``, and ``_current_track_name`` BEFORE each per-track callback fires, so a polling UI sees consistent "processing N/M: <name>" snapshots. - ``_scan_cycle`` flips ``_current_status`` to 'processing' before the per-album loop, resets it + the per-track fields after. Defensive ``finally`` clears progress even if the inner code path raised. - ``get_status()`` exposes the new fields so the UI's existing /api/auto-import/status polling picks them up. - Frontend (stats-automations.js): renders the new ``current_status='processing'`` state with track index/total/name in the existing progress bar element. New 'processing' status class for styling parity with 'scanning'. 8 regression tests in tests/imports/test_auto_import_live_progress.py: - get_status surfaces the new fields with sane defaults - track_index advances 1, 2, 3 during a 3-track loop - track_total set BEFORE the first callback fires (no '1/0' flicker) - _record_in_progress writes status='processing' with no processed_at - _finalize_result updates the same row to completed + processed_at, no second insert - _finalize_result with failed status leaves processed_at NULL - _finalize_result with row_id=None is a safe no-op - Per-track fields cleared by _scan_cycle's finally block Full pytest 1643 passed; ruff clean. |
||
|
Broque Thomas
|
29089b35b3 |
Honor configured Tidal redirect_uri, drop request-host fallback
Reported case (Foxxify): Tidal returned error 1002 ("Invalid redirect
URI") on every authentication attempt for users accessing SoulSync
from a network IP. User had ``http://127.0.0.1:8889/tidal/callback``
registered in his Tidal Developer Portal — matching the SoulSync UI
default and docs.
Root cause: the /auth/tidal route at web_server.py:5594-5598 had a
"fallback: dynamically set based on request host" branch that fired
when ``tidal.redirect_uri`` config was empty AND the request didn't
come from localhost. That fallback overrode the TidalClient
constructor's safe default (``http://127.0.0.1:<port>/tidal/callback``)
with a uri built from request.host like
``http://192.168.x.x:8889/tidal/callback``. Tidal compares strings
exactly so this never matched the documented portal registration and
the user got 1002 before the consent screen even rendered.
The trap is the SoulSync settings UI displays the default URI as the
placeholder + "Current Redirect URI" display — but the placeholder
never gets saved to config unless the user explicitly clicks Save.
Most users who follow the docs (register the displayed default with
Tidal, then click Authenticate) hit the empty-config path and the
broken fallback.
Fix: drop the request-host fallback. Empty config falls back to the
constructor default that matches the documented portal registration.
The existing post-auth swap-step in the instructions page below
handles the Docker / remote-access case as designed:
1. SoulSync sends 127.0.0.1:8889 in the authorize URL → matches
portal → Tidal accepts.
2. User authorizes → Tidal redirects browser to 127.0.0.1:8889
(which fails locally — nothing on user's machine listens there).
3. Instructions tell user to swap 127.0.0.1 with the host they're
accessing SoulSync from.
4. Swapped URL hits the container's exposed callback port → auth
completes.
8 regression tests in tests/test_tidal_auth_redirect_uri.py:
- Configured redirect_uri sent verbatim (localhost / custom port /
explicit network IP)
- Empty config falls back to constructor default — NOT request.host
(the actual reported scenario, with explicit assertion message
warning if the bug returns)
- Empty config + localhost access uses the same default (sanity)
Full pytest 1635 passed; ruff clean.
|
||
|
Broque Thomas
|
34ba26f5c8 |
Persist source IDs at download time + backfill onto tracks on sync
Followup to fix/watchlist-external-id-match. The companion PR closed the demand side — the watchlist scanner asks for tracks by external IDs before falling back to fuzzy. But for users on Plex / Jellyfin / Navidrome the supply side was still broken: tracks.spotify_track_id (and the other ID columns) only got populated by the asynchronous enrichment workers, sometimes hours after the file was actually written. During that window the ID match fell through to fuzzy and the bug returned. We were already collecting every ID during post-processing — they live in the `pp` dict in core/metadata/source.py:embed_source_ids and get embedded into file tags. We just dropped the in-memory copy afterwards. This PR persists them and uses them: - Schema migration adds spotify_track_id / itunes_track_id / deezer_track_id / tidal_track_id / qobuz_track_id / musicbrainz_recording_id / audiodb_id / soul_id / isrc columns + indexes to the existing track_downloads table (already keyed by file_path). - core/metadata/source.py:embed_source_ids exposes pp["id_tags"] and the resolved ISRC back to the import context as _embedded_id_tags / _isrc. - core/imports/side_effects.py:record_download_provenance reads those context fields and passes them to db.record_track_download, which now accepts the new ID kwargs and persists them. - New db.get_provenance_by_file_path with exact + basename-suffix fallback (handles container mount-root differences between download-time path and media-server-reported path). - New db.backfill_track_external_ids_from_provenance copies IDs from track_downloads onto a tracks row idempotently — COALESCE on every column preserves any value the enrichment worker already wrote (enrichment is more authoritative for late binding). - database/music_database.py:insert_or_update_media_track (the single insertion point used by every Plex / Jellyfin / Navidrome sync) calls the backfill immediately after each INSERT/UPDATE. - New core/library/track_identity.py:find_provenance_by_external_id used as a second-tier fallback in watchlist_scanner.is_track_missing _from_library — catches the window between download and media-server sync. Caller checks os.path.exists on the provenance file_path before treating it as "already in library" so a deleted file doesn't prevent re-download. Effect: freshly downloaded files become ID-recognizable to the watchlist on the very next scan, no enrichment-wait window. 19 regression tests in tests/test_provenance_id_persistence.py: - Schema migration adds expected columns + indexes - record_track_download persists every ID kwarg - record_track_download backward-compat (old kwargs still work) - get_provenance_by_file_path: exact match, basename fallback for mount-root differences, multi-record latest-wins, defensive None - backfill: copies all IDs, preserves existing via COALESCE, no-op when no provenance exists - find_provenance_by_external_id: per-ID lookup, ISRC cross-bridge, OR semantics, latest-wins on multiple matches Out of scope: backfilling provenance for files downloaded BEFORE this PR (their track_downloads rows don't carry the new IDs). Those continue to wait for enrichment. Acceptable — only affects historical files; new downloads benefit immediately. Full pytest 1625 passed; ruff clean. |
||
|
Broque Thomas
|
ecb8939c80 |
Match library tracks by external IDs before fuzzy in watchlist scan
Reported case (CAL): a track already on disk got re-downloaded by the
watchlist scanner on every scan. Library DB had stale album metadata
for the file (track tagged on album "Left Alone") while the metadata
source reported it on a different album ("NPC" single). The
title+artist+album fuzzy block correctly said the album names didn't
match and declared the track missing — but the file's stable external
IDs (Spotify ID, ISRC, etc.) unambiguously identified it as the same
recording.
The earlier compilation-album fix (PR #461) handled qualifier drift
("OST" vs "Music From The Motion Picture"). This case is two
genuinely different album names referring to the same song.
Fix: provider-neutral external-ID short-circuit before the fuzzy
block in `is_track_missing_from_library`. Pulls every recognized ID
off the source track (Spotify / iTunes / Deezer / Tidal / Qobuz /
MusicBrainz / AudioDB / Hydrabase / ISRC), runs a single SELECT
against the indexed external-ID columns on the `tracks` table, and
treats any hit as "track exists in library — don't re-download".
If no IDs are available (older imports without enrichment, library
scans that didn't populate external IDs), falls through to the
existing fuzzy logic so the safety net stays intact.
New `core/library/track_identity.py` module with two helpers:
- `extract_external_ids(track)`: handles dict and object-style track
shapes, direct-field aliases (spotify_id / spotify_track_id /
SPOTIFY_TRACK_ID), and provider-disambiguated native `id` fields
(when track has `provider='deezer'` and `id='X'`, treats X as a
Deezer ID).
- `find_library_track_by_external_id(db, external_ids,
server_source)`: builds an OR of indexed column matches with
IS NOT NULL guards, optional server_source filter that also
passes legacy NULL rows, single-row LIMIT.
ISRC bridges across providers — a library track imported via Deezer
can be matched against a Spotify scan when both sides carry the
same ISRC.
43 regression tests in `tests/test_library_track_identity.py`:
- 9 ID-extraction tests for direct fields (Spotify / iTunes / Deezer /
ISRC / MBID / AudioDB / Hydrabase)
- 8 ID-extraction tests via the provider field (8 providers + source
alias + missing-provider-ignored)
- 7 mixed/defensive tests (multiple IDs, object-style, empty strings,
None track, numeric coercion)
- 8 lookup tests (per-provider + ISRC cross-bridge)
- 3 OR-semantics tests
- 4 server_source filter tests
- 2 ID-column-map sanity tests
Full pytest 1606 passed; ruff clean.
|
||
|
Broque Thomas
|
486116c34f |
Honor lossy_copy.delete_original after successful conversion
Reported case (CAL): with lossy_copy.enabled=True, lossy_copy.delete_original=True, and codec=mp3, every download left both the original FLAC AND the converted MP3 in the target folder. Users opting into a lossy-only library ended up dual-format on every import. Root cause: ``core/imports/file_ops.py:create_lossy_copy`` reads ``lossy_copy.codec`` and ``lossy_copy.bitrate`` from config but never reads ``lossy_copy.delete_original``. The setting is only consulted by the pre-move source-vanished check at ``core/imports/pipeline.py:651`` (so the pipeline knows to look for a lossy variant when the FLAC has already moved on), but no code path actually deletes the source after conversion. Fix: after ffmpeg returns success and the QUALITY tag is written, check ``lossy_copy.delete_original`` and ``os.remove`` the original when enabled. Belt-and-suspenders: - Same-path guard (``os.path.normpath(out_path) != os.path.normpath(final_path)``) prevents accidentally wiping the just-converted file if a future codec choice somehow resolves out_path to the source path. - ``FileNotFoundError`` is treated as success (concurrent worker / dedup cleanup got there first). - Other ``OSError`` (permission denied, locked file) is logged but doesn't propagate — the conversion already succeeded, the user just has to clean up the original manually. Failure paths skip the delete: - ffmpeg returns non-zero → returns None, original stays - lossy_copy.enabled=False → early return before conversion runs - delete_original=False (default) → original stays 7 regression tests cover honored-when-enabled, kept-when-disabled, default-keep, ffmpeg-failure-path, lossy-disabled-path, racing-delete, and locked-file paths. Full pytest 1563 passed; ruff clean. Note: this PR does NOT address the second bug CAL mentioned (track re-downloaded despite already existing on disk). That symptom is caused by stale album metadata on the user's existing files — the library DB has the track tagged on a different album than the metadata source reports — combined with wishlist.allow_duplicate_tracks defaulting to True. Same class of issue partially addressed in PR fix/watchlist-redownload-and-duplicate-detection but compilation- album drift is the only currently-handled case. Tracking separately. |
||
|
Broque Thomas
|
99dbe265de |
Sync Qobuz auth to enrichment worker after login
Discord-reported (Foxxify): logging in to Qobuz via the Connect button on Settings showed "Connected: <username> (Active)" but underneath an error said "Qobuz not authenticated...", and the dashboard indicator stayed yellow. Saving settings or reloading the tab didn't help. Root cause: SoulSync runs two QobuzClient instances side by side — one through soulseek_client.qobuz for the /api/qobuz/auth/* endpoints, and a second owned by the enrichment worker thread for thread safety. The login flow only updated the auth-flow instance's in-memory state (plus persisted to config). The dashboard's "configured" check at web_server.py:3371 reads ``qobuz_enrichment_worker.client.user_auth_token`` — the WORKER's instance — which still believed itself unauthenticated. The connection-test step at core/connection_test.py:370 hits the same worker instance for the same reason. Fix: add ``QobuzClient.reload_credentials()`` — a public, network-free method that re-reads the saved session from config and updates the instance's in-memory state + session headers. Call it on the enrichment worker's client immediately after a successful ``/api/qobuz/auth/login``, ``/api/qobuz/auth/token``, or ``/api/qobuz/auth/logout`` so the two instances stay in lockstep without waiting for the next process restart. Unlike the existing ``_restore_session()`` this skips the network probe — the caller has just authenticated, so the token is known good. A small ``_sync_qobuz_credentials_to_worker()`` helper in web_server.py wraps the call so all three endpoints share one path. 10 new regression tests cover the populate / clear / partial-config paths plus the actual two-instance-sync scenario from the bug report. Full pytest 1555 passed (the one pre-existing flake in test_tidal_auth_instructions.py is order-dependent and unrelated). |
||
|
Antti Kettunen
|
a2176af00e
|
Rename metadata source status selectors
- Switch the dashboard/sidebar service-status card from spotify-branded ids to metadata-source ids - Update the shared status helpers to target the renamed metadata-source card - Keep the actual Spotify auth and settings UI unchanged |
||
|
Antti Kettunen
|
36267618a3
|
Rename status cache to metadata_source
Expose the primary metadata provider status under a generic cache key and update the websocket fixture plus frontend readers to match. |
||
|
Broque Thomas
|
7e32618f86 |
Drop old per-service enrichment routes after registry cutover
Followup to the enrichment-bubble registry consolidation. The
dashboard polling + click handlers all hit
/api/enrichment/<service>/{status,pause,resume} now, so the 30
hand-rolled per-service routes in web_server.py have zero callers
and can come out:
/api/musicbrainz/{status,pause,resume}
/api/audiodb/{status,pause,resume}
/api/discogs/{status,pause,resume}
/api/deezer/{status,pause,resume}
/api/spotify-enrichment/{status,pause,resume}
/api/itunes-enrichment/{status,pause,resume}
/api/lastfm-enrichment/{status,pause,resume}
/api/genius-enrichment/{status,pause,resume}
/api/tidal-enrichment/{status,pause,resume}
/api/qobuz-enrichment/{status,pause,resume}
Worker init blocks stay (they still construct the workers + persist
pause state). Section comment headers are preserved with a one-line
note pointing readers at the new generic blueprint.
Test fixtures in tests/conftest.py and
tests/metadata/test_enrichment_events.py also updated to use the
new URL paths so they reflect production reality. They were
synthetic stubs that never depended on the production routes —
purely cosmetic alignment.
Net: ~510 lines deleted from web_server.py. Full pytest 1541
passed; ruff clean.
|
||
|
Broque Thomas
|
98c04cf332 |
Consolidate enrichment bubble routes behind a service registry
The dashboard's enrichment-status bubbles (MusicBrainz, AudioDB,
Discogs, Deezer, Spotify, iTunes, Last.fm, Genius, Tidal, Qobuz) each
had its own copy-pasted /status, /pause, /resume route in web_server.py
— 30 routes that differed only in the worker reference and a couple
of per-service quirks (Spotify's rate-limit guard, Last.fm/Genius
yield-override behavior, Tidal/Qobuz extra status fields).
Replace them with a registry-driven blueprint:
- core/enrichment/services.py declares an EnrichmentService dataclass
with worker_getter, config_paused_key, pre_resume_check,
auto_pause_token, and extra_status_defaults — all variation captured
as data, no branching on service id.
- core/enrichment/api.py exposes a Flask blueprint with three routes
(/api/enrichment/<service>/{status,pause,resume}). Per-service
quirks are honored via the descriptor: Spotify's rate-limit ban
still returns 429 with `rate_limited: true`, Last.fm/Genius still
drop the auto-pause token and add the yield override, Tidal/Qobuz
still merge `authenticated: false` into the fallback payload.
- web_server.py registers all 10 services after their workers
initialize, wires the host-side hooks (config_manager.set,
_download_auto_paused.discard, _download_yield_override.add), and
registers the blueprint.
- webui/static/enrichment.js polling + click handlers now hit the
generic endpoints. The per-service `update<Service>StatusFromData`
functions are unchanged — they still process the same payload.
This is the cutover step. Old per-service routes are intentionally
left in place as a fallback during the soak period — they currently
have zero callers in the codebase and will be deleted in a follow-up
patch once production has run on the new pipeline for a few days.
27 new tests in tests/test_enrichment_services.py cover the registry
behavior + every quirk path through the generic blueprint (rate-limit
guard, auto-pause token cleanup, persisted-pause config keys, extra
default fields, worker-not-initialized fallback, exceptions). Full
suite 1541 passed; ruff clean.
|
||
|
Broque Thomas
|
0c4fad033d |
Show artist breadcrumb on sidebar Library button when on artist-detail page
Artist-detail is a "pseudo-page" reachable from Library, the unified
Search page, and the global search popover. It has no [data-page]
match in the sidebar, so navigateToPage's bulk-active-removal left
every nav button unhighlighted while the user was viewing an artist —
the sidebar offered no visual anchor for where they were.
Now:
- navigateToPage('artist-detail') falls back to highlighting the
Library button when no [data-page] match exists, anchoring the
sidebar to the canonical home for artist detail views.
- A new _updateSidebarLibraryBreadcrumb() helper rewrites the Library
button label between plain "Library" and a "Library / Artist Name"
breadcrumb based on currentPage + artistDetailPageState. Long names
(>14 chars) truncate with an ellipsis; the full name shows on hover
via the title attribute.
- Called from navigateToPage (entering / leaving the page) and from
loadArtistDetailData (covers same-page artist switches in the
similar-artist chain where currentPage stays 'artist-detail').
CSS adds .nav-text-root / .nav-text-sep / .nav-text-context selectors
so the "Library" anchor word stays visually dominant while the
separator and artist name dim to a secondary tier — readable but not
competing for attention.
Pure visual change. No backend touched. No new tests (DOM-only).
|
||
|
Broque Thomas
|
84810b4de4 |
Bump version to 2.4.1
Patch release wrapping up the 2.4.1 dev cycle. Highlights: - Watchlist no longer re-downloads compilation/soundtrack tracks (#458 dedup orphan cleanup + the album-match fix work in tandem to stop the loop). - Duplicate detector catches slskd dedup orphans via a second filename-bucket pass. - Beatport tab hidden temporarily — Cloudflare Turnstile blocks the scraper and the official OAuth API is closed to public devs. - Service worker for cover art + installable PWA manifest. - Browser caching for static assets (1y) and discover pages (5min). - Socket.IO same-origin default + admin-only /api/settings. Files updated: - web_server.py: _SOULSYNC_BASE_VERSION 2.4.0 -> 2.4.1 - webui/index.html: sidebar version button + modal subtitle - webui/static/helper.js: WHATS_NEW dev-cycle marker -> release date, fallback version in _getLatestWhatsNewVersion, 8 new VERSION_MODAL_SECTIONS entries promoted from this cycle - .github/workflows/docker-publish.yml: workflow_dispatch default version_tag updated to 2.4.1 |
||
|
Broque Thomas
|
6e61890551 |
Stop watchlist re-downloading compilation tracks; catch slskd dedup orphans
Two related bugs reported on Discord by Mushy. 1. The watchlist re-downloaded the same OST track up to 7 times. ``is_track_missing_from_library`` compared Spotify's album name and the media-server scan's album name with a raw SequenceMatcher at a strict 0.85 threshold. Compilations and soundtracks routinely fail this — Spotify reports ``"Napoleon Dynamite (Music From The Motion Picture)"`` while the Plex / Navidrome / Jellyfin tag scan saves it as ``"Napoleon Dynamite OST"``. Raw similarity ≈ 0.49, so the scanner declared the track missing on every 30-minute scan and added it back to the wishlist. The wishlist then issued a fresh download. slskd appended ``_<19-digit-ns-timestamp>`` to each new copy because the target file already existed, and the user ended up with seven copies of one song in one folder. Fix: extract two pure helpers — ``_normalize_album_for_match`` strips qualifier parentheticals (Music From X, OST, Deluxe Edition, Remastered, Anniversary, etc.) and trailing dash-clauses; ``_albums_likely_match`` checks equality after normalization, substring containment, and a relaxed 0.6 fuzzy ratio. A volume / part / disc / standalone-trailing-number guard rejects pairs like ``"Greatest Hits Vol. 1"`` vs ``"Greatest Hits Vol. 2"`` so the relaxed threshold doesn't introduce false positives on serialized releases. After this change the Napoleon Dynamite case collapses to ``"napoleon dynamite" == "napoleon dynamite"`` via the equality short-circuit and the redownload loop dies. 2. The duplicate detector found only one of the seven dupe files. The detector buckets tracks by the first 4 chars of their normalized tag title. Files written by slskd directly into a library folder often get inconsistent (or blank) tags from the media-server rescan, so the seven copies were bucketed apart by parsed title and never compared. Fix: refactor the per-bucket comparison into ``_scan_bucket``, then add a second pass — ``_build_filename_buckets`` re-buckets leftover tracks by canonical filename stem (slskd dedup tail stripped via ``_strip_slskd_dedup_suffix``, same regex the import-cleanup PR uses) plus extension. Filename agreement is itself strong evidence the files came from the same source download, so the second pass calls ``_scan_bucket`` with ``require_metadata_match=False`` to skip the title / artist / cross-album gates. The same-physical-file guard still runs so bind-mount duplicates aren't flagged. 72 new regression tests across two files cover the album-match helpers (28 tests including the Napoleon Dynamite scenario, 7 volume disagreements, 8 positive/negative pairs, 5 defensive cases) and the new filename-bucket pass (16 tests across bucket construction, scan integration, and existing title-pass behavior). Full pytest 1509 passed; ruff clean. Reported by Mushy in Discord. |
||
|
Broque Thomas
|
ab884292d1 |
Hide Beatport tab temporarily
Some checks failed
Compile the app and run tests / sanity-check (push) Has been cancelled
Beatport added Cloudflare Turnstile to every public page on beatport.com. The unified scraper now receives bot-challenge HTML instead of real content, so all /api/beatport/* endpoints return 500 with "Could not fetch Beatport homepage". The official Beatport v4 API is locked behind OAuth application registration that isn't open to the public — confirmed via the docs at api.beatport.com/v4/docs and community projects (beets-beatport4). The public docs SPA client_id only accepts browser-based flows (post-message redirect URI), which can't be driven server-side. Hide the Beatport tab on the Sync page so users stop hitting the broken endpoints. Backend routes and beatport_unified_scraper.py stay in code — revival is a one-attribute HTML change once Cloudflare relaxes or a workaround is found. Reported via the homepage 500 spam in user logs. |
||
|
Broque Thomas
|
46d8e15674 |
Prune slskd dedup orphans after import
slskd appends "_<19-digit unix-nanosecond timestamp>" to a downloaded filename when the destination already contains a same-named file (concurrent downloads of the same track, partial-file retries after a connection drop, cancelled-then-redownloaded files, the same track surfacing in multiple synced playlists). The file-finder code already recognized the suffix when matching a download to its source — but after the canonical file moved into the library, the leftover "_<timestamp>" siblings sat orphaned in the downloads folder forever. Reported on Discord by Shdjfgatdif. cleanup_slskd_dedup_siblings() runs at the end of each successful import (3 safe_move_file sites in pipeline.py) and prunes any remaining siblings that strip down to the canonical stem with the same extension. Conservative match (>= 18 trailing digits) keeps legitimate filenames like "Track 5" and "Album 1995" untouched. Per- file unlink failures are swallowed so a single locked file doesn't block the rest. 17 regression tests cover the suffix-strip primitive, orphan removal, no-op cases, mismatched extensions, subdirectories, and partial-failure recovery. |