Commit graph

2763 commits

Author SHA1 Message Date
Broque Thomas
397a1c73df ID-first fallback for replace-track + remove-track too
Same latent bug as add-track — replace-track and remove-track only
looked up the Plex playlist by name. Plex deletes + recreates
playlists on edit so the rating key the frontend cached can be
stale, name lookups can also fail (special chars, encoding). Both
now use the same id-first / name-fallback chain as the GET tracks
endpoint, with a diagnostic log when both lookups fail.

Pre-existing latent bug, not a refactor regression.
2026-05-05 20:06:08 -07:00
Broque Thomas
218af65606 ID-first fallback for server-playlist add-track + diagnostic logging
The /api/server/playlist/<id>/add-track endpoint only looked up the
target Plex playlist by name, but Plex deletes + recreates playlists
on edit so the rating key the frontend cached can be stale. The
companion GET /tracks endpoint already had id-first / name-fallback;
add-track now does the same.

Added a warning log on GET /tracks when BOTH lookups fail so the
"all source items show Find & Add" symptom (which happens when
server_tracks comes back empty) has a clear diagnostic in the log
instead of silently rendering an empty server column.

Not a refactor regression — the original code had the same name-only
lookup. The mass-replace of `plex_client` → `media_server_engine.client('plex')`
is byte-equivalent. Just surfacing the latent bug.
2026-05-05 19:54:31 -07:00
Broque Thomas
03d1c36637 Fallback to empty MediaServerEngine if init fails
If MediaServerEngine init raised, ``media_server_engine`` was set
to None. Every downstream caller (now that the per-server globals
are gone) does ``media_server_engine.client('plex')`` style access
— which would AttributeError on the None.

Pre-refactor each per-server global had its own try/except so engine
failure didn't take down per-server dispatch sites. Preserve that
resilience by falling back to an empty MediaServerEngine — its
``client(name)`` returns None, downstream truthy checks treat that
as "not configured" exactly the same way the legacy globals did.
2026-05-05 18:42:21 -07:00
Broque Thomas
edb6d1bc33 Drop dead per-server class imports + update WHATS_NEW
- services/sync_service.py: dropped unused PlexClient / JellyfinClient
  / NavidromeClient class imports. After the engine refactor the
  service only needs TrackInfo for type annotations; the class
  imports were dead.
- WHATS_NEW: extended the media server engine review-pass entry to
  cover the followup commits (Cin-5 per-server global removal +
  Gap 1 shared types lift) so the changelog matches the actual
  branch state.
2026-05-05 18:36:36 -07:00
Broque Thomas
2ebaf2e6e3 MS Gap 1: Lift shared TrackInfo + PlaylistInfo to neutral types module
Plex / Jellyfin / Navidrome each defined a near-identical
XTrackInfo (id / title / artist / album / duration / track_number /
year / rating) and XPlaylistInfo (id / title / description /
duration / leaf_count / tracks). Three classes that grew up by
copy-paste — not a real contract difference.

Lifted both to core/media_server/types.py as canonical TrackInfo +
PlaylistInfo. Per-server constructors live as classmethods on the
unified class (TrackInfo.from_plex_track, PlaylistInfo.from_plex_playlist)
matching the metadata Album.from_X_dict pattern Cin's POC uses.
Heavy plexapi imports stay lazy under TYPE_CHECKING.

- core/plex_client.py / jellyfin_client.py / navidrome_client.py:
  per-server XTrackInfo / XPlaylistInfo dataclass definitions
  removed; each module now imports TrackInfo + PlaylistInfo from
  the neutral package and uses the shared name internally.
- core/matching_engine.py: was annotating callers with PlexTrackInfo
  even though sync_service hands it Jellyfin / Navidrome instances
  at runtime when those servers are active. Annotation is now the
  unified TrackInfo, so signatures match the actual contract.
- services/sync_service.py: same import + annotation update.
2026-05-05 18:25:28 -07:00
Broque Thomas
a6bb5f5b43 MS Cin-5: Drop per-server globals — engine owns the clients
Per-server web_server.py globals (plex_client / jellyfin_client /
navidrome_client / soulsync_library_client) are gone. The engine now
owns the per-server client instances; web_server.py constructs them
inline into the engine init and routes everything through
media_server_engine.client('<name>').

Multi-client consumers refactored to take the engine instead of
separate per-server kwargs:

- services/sync_service.py: PlaylistSyncService.__init__ now takes
  media_server_engine. Internal _get_active_media_client resolves the
  active server's client through self._engine.client(name) instead of
  the per-server self.X_client attributes.
- core/listening_stats_worker.py: ListeningStatsWorker takes
  media_server_engine. The plex/jellyfin/navidrome dispatch in _poll
  collapses to engine.client(active_server) (gated to those three
  servers — SoulSync standalone has no listening data).
- core/web_scan_manager.py: WebScanManager takes media_server_engine
  instead of the hand-keyed media_clients dict that drifted out of
  sync with the engine.
- core/discovery/sync.py: SyncDeps holds media_server_engine instead
  of plex_client / jellyfin_client. Playlist-image dispatch routes
  through engine.client(name).

Web_server.py:
- Per-server globals removed from the chained `= None` init line
  + their try/except construction blocks. Replaced with a
  _safe_init_media_client(factory, name) helper that captures
  per-server init failures + passes the resulting clients straight
  into the MediaServerEngine init dict.
- All construction sites (PlaylistSyncService, WebScanManager,
  ListeningStatsWorker, SyncDeps, library_check) updated to receive
  the engine instead of per-server clients.

Test fixtures (tests/discovery/test_discovery_sync.py) gain a
_FakeMediaServerEngine stub + the SyncDeps build helper passes
that instead of separate plex/jellyfin clients.
2026-05-05 18:05:45 -07:00
Broque Thomas
d3f8a06d7a WHATS_NEW entry for media server engine review pass 2026-05-05 17:43:05 -07:00
Broque Thomas
1bc5017592 MS Cin-3 + Cin-4: Route web_server through engine instead of per-client globals
Pre-change web_server.py had ~70 direct attribute reaches against the
per-server globals (plex_client.X, jellyfin_client.X, navidrome_client.X,
soulsync_library_client.X) plus ~60 standalone refs (truthy checks,
media_client assignments, source-name tuples). The engine was wired
but only used in 4 places, so most of the codebase still hand-dispatched
— the exact "partially defeats the purpose of this refactor" critique
Cin landed on the download PR initially.

- All ~70 client.attribute reaches migrated to
  media_server_engine.client('<name>').attribute. The chains in
  web_server.py do server-specific work (Plex raw API, Jellyfin /
  Navidrome client methods, all returning different shapes), so the
  if/elif structure stays — but the per-server CLIENT REACH now goes
  through the engine like Cin's POC pattern intended.
- All ~60 standalone refs migrated:
  - if plex_client → if media_server_engine.client('plex')
  - media_client = plex_client → media_client = media_server_engine.client('plex')
  - ('plex', plex_client) tuples → ('plex', media_server_engine.client('plex'))
- Per-server globals (plex_client / jellyfin_client / navidrome_client /
  soulsync_library_client) kept for now — external modules
  (PlaylistSyncService, WebScanManager, ListeningStatsWorker, search
  library check, discovery sync deps) still take them as kwargs.
  Dropping them entirely needs a follow-up sweep across those modules.

Suite green (1961 pass).
2026-05-05 17:40:27 -07:00
Broque Thomas
49f7679eef MS Cin-1 + Cin-2: Explicit contract inheritance + generic accessors
Apply the Cin-1 / Cin-2 pattern from the download refactor PR to the
media server engine PR before review.

Cin-1 — explicit inheritance:
- PlexClient, JellyfinClient, NavidromeClient, SoulSyncClient now
  explicitly inherit MediaServerClient instead of relying on
  structural typing alone. Pre-change a reader of plex_client.py
  had no way to know the class was supposed to satisfy the contract.
- Removed the engine + registry re-exports from
  core/media_server/__init__.py to break the circular import that
  the inheritance change introduced (importing the package now
  triggered a chain that loaded clients before their base class
  resolved). Submodules import directly: from
  core.media_server.engine import MediaServerEngine, etc.
- Conformance test now also asserts isinstance() / issubclass()
  against MediaServerClient — drift in any class fails at the test
  boundary instead of at runtime.

Cin-2 — generic accessors + singleton:
- engine.configured_clients() — replaces the legacy per-server
  `if X and X.is_connected(): clients[name] = X` chains in
  web_server.py.
- engine.reload_config(name=None) — generic dispatch, so callers
  pass the server name instead of reaching for plex_client.reload_config()
  directly.
- get_media_server_engine() / set_media_server_engine() singleton
  factory matching the get_metadata_engine() / get_download_orchestrator()
  shape. web_server.py boots via set_media_server_engine(...) so
  factory + global handle share state.
- 7 new tests pin the accessors + singleton behaviour.
2026-05-05 16:59:05 -07:00
Broque Thomas
650327ba18 Phase E: Add WHATS_NEW entry for media server engine refactor
Internal-track entry covering the media server engine + contract +
the honest-scope note explaining why we lifted the 4 truly-uniform
is_connected dispatches and left the deep server-specific dispatches
explicit (each does fundamentally different work per server, so
lifting would just move per-server branches into engine helper
methods).
2026-05-04 21:45:29 -07:00
Broque Thomas
b769f02f04 C2: Migrate remaining uniform is_connected dispatches
Two more sites in web_server.py replaced (tag-preview + batch
tag-preview server_type checks). Same pattern as C1: 3-way
if/elif → engine.is_connected().

Honest scope note: the recon agent counted 33 dispatch sites,
but most are deeply server-specific logic where each branch
does completely different work (playlist track replace, per-
server metadata sync, deep scan with server-specific helpers).
Lifting those would move per-server branches into engine helper
methods that route the same work — net zero LOC, more indirection.
Engine helps where the shape is TRULY uniform; the deep dispatches
stay explicit. Phase C ends here at 4 simple sites lifted.

Suite still green.
2026-05-04 21:28:09 -07:00
Broque Thomas
971d683ebd C1: Migrate status-check dispatch sites to engine
Two sites in web_server.py replaced:
- /status route's media-server connectivity check (4-way if/elif
  for plex/jellyfin/navidrome/soulsync) → engine.is_connected()
- /api/playlists endpoint's server_connected check (3-way if/elif)
  → engine.is_connected()

Engine reads active_server config + dispatches to the right client
with internal connection caching preserved (the underlying clients
all cache is_connected() calls).

Engine constructor now accepts a pre-built clients={...} dict so
web_server.py wires the same instances as its existing per-client
globals — no double-init.

Suite still green. Per-server clients still accessible via
engine.client(name) for source-specific reaches.
2026-05-04 21:25:20 -07:00
Broque Thomas
6b54ca6598 Phase B: Add MediaServerEngine skeleton
`MediaServerEngine` reads the active server from config + dispatches
to the corresponding registered client. Per-server reaches still
work through `engine.client(name)`.

Required-method dispatch (is_connected, ensure_connection,
get_all_artists, get_all_album_ids) returns safe defaults when
the active client failed to initialize OR when the method raises.

Optional-method dispatch (search_tracks, trigger_library_scan,
is_library_scanning, get_library_stats, get_recently_added_albums)
checks hasattr first — SoulSync standalone has no
trigger_library_scan or get_library_stats, engine no-ops with
appropriate defaults instead of forcing every client to declare
stub methods.

10 new engine tests pin: active-server resolution, required
dispatch routing, exception safety, missing-optional-method
fallback shape. Suite still green (1951 passed).

Engine isn't on any production code path yet — Phase C migrates
the 33 web_server.py dispatch sites to call engine.method()
instead of hand-branching by active_server name.
2026-05-04 21:14:17 -07:00
Broque Thomas
50fe4bec97 A4: Pin SoulSyncClient observable behavior
5 tests pin the SoulSync standalone client surface — the
structurally-different one (no auth, no API, no library scan).
is_connected just checks os.path.isdir(transfer_path).
ensure_connection reloads config first so the user changing
the transfer_path takes effect without a process restart.
get_all_album_ids returns a set of MD5-hashed string ids
matching cross-server uniform set semantics.
2026-05-04 21:03:37 -07:00
Broque Thomas
edcdaaa993 A3: Pin NavidromeClient observable behavior
4 tests pin the Navidrome client surface. Auth shape: base_url +
username + password (no token model — salt generated per request).
get_all_album_ids paginates getAlbumList2 and returns a set of
string ids matching cross-server uniform set semantics.
2026-05-04 21:01:11 -07:00
Broque Thomas
5da2cfec46 A2: Pin JellyfinClient observable behavior
5 tests pin Jellyfin client surface. is_connected requires ALL
four of base_url + api_key + user_id + music_library_id (stricter
than Plex's is_connected). get_all_album_ids returns a set of
string GUID ids matching the cross-server uniform set semantics.
2026-05-04 20:53:55 -07:00
Broque Thomas
c1da56b2c2 A1: Pin PlexClient observable behavior
6 tests pin the Plex client surface the engine will dispatch
through after Phase B/C migrations:
- is_connected returns False on no-server, True on server-present
- is_fully_configured requires BOTH server AND music_library
- get_all_artists empty list on not-connected, iterates
  music_library.searchArtists() when connected
- get_all_album_ids returns a set of STRING ratingKey values
  (coerced from Plex ints so semantics match Jellyfin GUIDs +
  Navidrome string ids)

Phase A pinning catches behavior drift during web_server.py
dispatch-site migrations (Phase C) and engine adapter wiring
(Phase B).
2026-05-04 20:53:00 -07:00
Broque Thomas
f702196dca Phase 0: Add MediaServerClient contract + registry
`core/media_server/` package with the Protocol contract that
every media server client (Plex, Jellyfin, Navidrome, SoulSync
standalone) satisfies, plus the registry that holds them.

Required methods conservatively limited to the four every server
truly implements today: is_connected, ensure_connection,
get_all_artists, get_all_album_ids. Other generic methods
(search_tracks, trigger_library_scan, get_recently_added_albums,
etc.) are listed as OPTIONAL — present on most servers but not
all (SoulSync has no library-scan API since it walks the filesystem
directly; Jellyfin uses a different search shape). Phase B's
engine adapters route around the gaps with per-server fallback
instead of forcing every client to declare a no-op stub.

Same registry shape as the download plugin registry — single
source of truth for which servers exist + name resolution. Adding
a 5th server (Subsonic, Emby, etc.) becomes one register call
plus the new client class.

5 conformance tests pin every server class implements every
required method. Plan doc at docs/media-server-engine-refactor-plan.md.

Pure additive — no consumer routes through the contract or
registry yet. Suite still green (1921 passed).
2026-05-04 20:49:25 -07:00
BoulderBadgeDad
0eaac77627
Merge pull request #493 from Nezreka/fix/findings-tab-empty
Fix/findings tab empty
2026-05-04 09:16:39 -07:00
Broque Thomas
05bfb724a8 Update mbid consistency test mock to match new create_finding bool contract 2026-05-04 09:11:01 -07:00
Broque Thomas
749a772ff5 Findings tab: auto-switch to all-status when 0 pending exist
Companion to the badge count fix. When the findings tab opens with
the default "pending" filter and returns 0 rows but other statuses
(resolved/dismissed/auto-fixed) do have rows, the filter
auto-switches to "All Status" and a small notice explains the
switch. Stops the empty "all clear" state from masking carry-over
findings from prior scans.
2026-05-04 09:04:06 -07:00
Broque Thomas
cf5461f2f1 Fix: maintenance findings badge inflated when scan dedup-skipped
`_create_finding` silently dedup-skipped re-discovered issues but
the caller incremented `findings_created` regardless. So a re-scan
that found the same issues as a prior scan reported 364 findings
in the badge while 0 NEW pending rows hit the db, leaving the
findings tab empty.

`_create_finding` now returns bool (True on insert, False on
dedup-skip / db error). All 16 repair jobs updated to only
increment `findings_created` on True. Added `findings_skipped_dedup`
counter surfaced in scan log: "Done: X scanned, 0 fixed, 0
findings (363 already existed), 0 errors".

Also fixed a missing `job_id` kwarg in album_tag_consistency that
was silently breaking finding creation for that scan.
2026-05-04 08:55:13 -07:00
BoulderBadgeDad
ea741df286
Merge pull request #492 from Nezreka/refactor/typed-album-consumers
Migrate discography + quality scanner to typed Album path
2026-05-04 08:19:28 -07:00
Broque Thomas
77c54ab7a7 Migrate discography + quality scanner to typed Album path
Three more album-shape consumers now route through
Album.from_<source>_dict() when caller passes a known source:
- _build_discography_release_dict (artist discography cards)
- _build_artist_detail_release_card (artist detail release cards)
- _normalize_track_album (quality scanner result normalization)

Legacy duck-typing stays as fallback for unknown source,
non-dict input, or converter errors. Pure additive — existing
callers without source kwarg unchanged.
2026-05-04 08:12:40 -07:00
BoulderBadgeDad
5c947fc3e0
Merge pull request #491 from Nezreka/refactor/migrate-build-album-info-to-typed
Migrate album-info builders to typed Album path
2026-05-03 22:56:43 -07:00
Broque Thomas
967c7f7c0a Migrate album-info builders to typed Album path
Steps 2+3 of typed metadata migration. Two album-info builders now
route through Album.from_<source>_dict() when caller passes a
known source:
- _build_album_info (album-tracks lookups)
- _build_single_import_context_payload (single-track import context)

Legacy duck-typing stays as fallback for unknown source, non-dict
input, or converter errors. Pure additive — existing callers
without source kwarg unchanged.
2026-05-03 22:53:12 -07:00
BoulderBadgeDad
e12969fbbe
Merge pull request #490 from Nezreka/refactor/typed-metadata-types-foundation
Refactor/typed metadata types foundation
2026-05-03 22:33:54 -07:00
Broque Thomas
eab1297afc Add Qobuz + Tidal album converters
Audit caught two missing providers from the foundation pr. Both
return album-shaped data via their clients (search + download
flows). Tidal uses tidalapi objects rather than dicts so the
converter is from_tidal_object, not _dict.

Enrichment-only providers (lastfm/genius/acoustid/listenbrainz/
audiodb) intentionally have no album converter — they enrich
existing rows, never return album shapes.

Tests: +8 cases. 40 total now.
2026-05-03 22:30:19 -07:00
Broque Thomas
529486a2d1 Foundation: typed Album/Track/Artist + per-provider converters
New core/metadata/types.py with canonical dataclasses + classmethod
converters for spotify/itunes/deezer/discogs/musicbrainz/hydrabase.
Each converter is the single place that knows that provider's wire
shape — addresses the duck-typing pattern Cin flagged.

Pure additive: no consumer code changed. Follow-up PRs migrate
consumers one at a time. Migration plan at
docs/metadata-types-migration.md.

Tests: 32 cases pin per-provider semantics + cross-provider
invariants. Also stabilized a flaky discogs test that depended on
local config state.
2026-05-03 22:21:32 -07:00
BoulderBadgeDad
e0b15a9e69
Merge pull request #489 from Nezreka/feat/discogs-collection-source
Feat/discogs collection source
2026-05-03 21:48:32 -07:00
Broque Thomas
09cea9f013 Show toast hint when toggling a disconnected source on Your Albums
The Your Albums sources modal silently bailed on toggle clicks for
disconnected sources — toggle did nothing, no feedback, users had no
way to know why. Surfaced when users tried to enable Discogs without
having set a Discogs token first; same UX gap existed for the other
sources too but went unreported because most users have Spotify
connected by default.

Added per-source hint messages so the toast tells users exactly
where to set up credentials. Bonus: subtitle update after save now
includes 'discogs' in the source-name map (was undefined before,
fell through to lowercase 'discogs' in the rendered text).

Affects only the Your Albums sources modal — toggle behavior
unchanged for connected sources.
2026-05-03 21:44:56 -07:00
Broque Thomas
4b23bee4a9 Add Discogs collection as a Your Albums source
Discord request: pull user's Discogs collection into the Your Albums
section on Discover, similar to how Spotify Liked Albums works.
Implementation extends the existing 3-source pipeline (Spotify /
Tidal / Deezer) to a 4-source pipeline with click-context dispatch —
Discogs-only albums open with rich Discogs release detail (vinyl/CD
format, year, label, country, tracklist). Mirrors the per-source
dispatch pattern from enhanced/global search.

Discogs client (`core/discogs_client.py`):
- New `get_authenticated_username()` resolves the username for the
  configured personal token via Discogs's `/oauth/identity` endpoint.
  Cached on the instance so subsequent collection page-fetches don't
  re-hit it.
- New `get_user_collection(username=None, folder_id=0, per_page=100,
  max_pages=50)` walks all pages of `/users/{username}/collection/
  folders/{folder_id}/releases`. Returns normalized dicts ready for
  upsert_liked_album. folder_id=0 = Discogs's "All" folder.
  Pagination cap of max_pages*per_page = 5000 releases — bounds
  runtime on heavy collections.
- New `get_release(release_id)` thin wrapper for `/releases/{id}` —
  returns the raw API response so the album-detail endpoint can
  render rich context.
- Both methods defensive: missing token → empty list, malformed
  responses → skipped, falsy ids → None. Disambiguation suffix
  stripping (`Madonna (3)` → `Madonna`) so Discogs artist names
  match what Spotify/Tidal/Deezer use.

Schema (`database/music_database.py`):
- New `discogs_release_id TEXT` column on `liked_albums_pool`.
  Migration uses the established `try SELECT, except ALTER TABLE`
  pattern. Idempotent; safe on existing installs.
- Added the column to the canonical CREATE TABLE for fresh installs.
- `upsert_liked_album` extended with `'discogs': 'discogs_release_id'`
  in BOTH the INSERT and UPDATE id-column maps so Discogs source_id
  routes to the new column. INSERT statement column count + value
  count updated together.

Backend (`web_server.py`):
- `/api/discover/your-albums/sources` — adds Discogs to the
  `connected` list when `discogs.token` config is set.
- `_fetch_liked_albums` — new branch for Discogs. Lazy-imports
  DiscogsClient, respects the `enabled_sources` config, walks the
  collection, upserts each release. Same try/except shape as the
  existing source branches.
- `/api/discover/album/<source>/<album_id>` — new `discogs` branch
  fetches the release via DiscogsClient.get_release, normalizes the
  Discogs tracklist format, parses Discogs's `MM:SS`/`HH:MM:SS`
  duration strings to milliseconds, returns the same response shape
  as the Spotify/Deezer/iTunes branches.

Frontend (`webui/static/discover.js`):
- `openYourAlbumsSourcesModal` — adds Discogs to `sourceInfo` with
  the vinyl emoji icon. Existing toggle/save plumbing handles it.
- `openYourAlbumDownload` — restructured the per-source dispatch:
  builds an ordered list of (source, id) tuples, tries each in turn,
  breaks on the first successful response. Pure-Discogs albums go
  straight to the Discogs detail endpoint → modal opens with Discogs
  context. Multi-source albums prefer Spotify/Deezer first since
  their tracklists carry proper streaming IDs ready for download.

Tests: `tests/test_discogs_collection_source.py` — 12 cases:
- get_user_collection: empty without token, normalizes response
  shape, strips disambiguation suffix, handles missing year, skips
  malformed releases, paginates correctly, caps at max_pages,
  uses explicit username when provided.
- get_release: passes id through to /releases/{id}, returns None
  for invalid ids without API call.
- liked_albums_pool: discogs_release_id round-trips through upsert
  + get; multi-source dedup carries both Spotify and Discogs IDs
  on the same row.

Verified: full suite 1825 pass (12 new), ruff clean, smoke test
populating + reading the discogs_release_id column round-trips
correctly via the real DB.

WHATS_NEW entry under '2.4.2' dev cycle.
2026-05-03 21:27:46 -07:00
BoulderBadgeDad
8b41670717
Merge pull request #488 from Nezreka/chore/drop-redundant-spotify-library-section
Drop redundant standalone "Your Spotify Library" section on Discover
2026-05-03 20:55:23 -07:00
Broque Thomas
e84d187e76 Drop redundant standalone "Your Spotify Library" section on Discover
Discover page used to show two near-identical sections:
- "Your Albums" — cross-source aggregator across Spotify / Deezer /
  etc with a gear button to configure sources, search, status filter,
  sort options, and a download-missing action.
- "Your Spotify Library" — Spotify-only with the same grid UI, same
  refresh / download-missing buttons, same filter / sort controls.

The Spotify-only section was a strict subset of what Your Albums
already covers (Spotify is one of the configurable sources). User
flagged the redundancy when scoping the upcoming Discogs integration
and asked for the duplicate to be removed.

Removal scope:
- `webui/index.html` — drop the `#spotify-library-section` block (42
  lines).
- `webui/static/discover.js` — drop the dead JS (~335 lines): state
  vars `spotifyLibraryAlbums` / `spotifyLibraryPage` / etc, all the
  loaders / renderers / pagination / click handlers, and the
  `loadSpotifyLibrarySection()` call in `loadDiscoverPage`'s
  Promise.all.
- `webui/static/helper.js` — drop the helper annotation entry at
  `#spotify-library-section` and the matching guided-tour entry.

Backend untouched. The Spotify saved-albums cache
(`spotify_library_albums` table + watchlist_scanner upsert/cleanup
+ `/api/discover/spotify-library` endpoint + the DAO methods) is
shared infrastructure that Your Albums reads from when Spotify is
one of its configured sources. Removing the UI section just removes
the duplicate surface — Spotify saved albums still appear in Your
Albums via the existing source dispatch.

CSS class names (`.spotify-library-grid`, `.spotify-library-search`,
`.spotify-library-pagination`) intentionally remain on the surviving
Your Albums elements — they share the same visual styling and
renaming would be churn for no benefit.

Verified: full suite 1813 pass (no new tests — pure UI/dead-code
removal). Backend endpoint behavior unchanged. WHATS_NEW entry
under '2.4.2' dev cycle.
2026-05-03 20:52:44 -07:00
BoulderBadgeDad
3dc27034e5
Merge pull request #487 from Nezreka/feat/library-disk-usage-stat
Add Library Disk Usage card to System Statistics
2026-05-03 20:21:09 -07:00
Broque Thomas
2ab460f5c4 Add Library Disk Usage card to System Statistics
Discord request (Samuel [KC]): show how much disk space the library
takes on the Stats page. Implementation piggybacks on the existing
deep scan — Plex/Jellyfin/Navidrome all return file size in their
track API responses, so we read it during the deep scan and store
it on the tracks row. Aggregation is then a single SQL query — no
filesystem walk, no extra I/O during the scan, no separate stat
job. SoulSync standalone gets size from os.path.getsize at insert
time (different code path; the file is local when we write the row).

Schema (`database/music_database.py`):
- New `file_size INTEGER` column on `tracks`. Migration uses the
  established `try SELECT, except ALTER TABLE ADD COLUMN` pattern.
  Idempotent; safe on existing installs. NULL on legacy rows so
  they don't contribute to totals until next deep scan refreshes.
- Added the column to the canonical CREATE TABLE so fresh installs
  get it without going through the migration path.

Track-object plumbing:
- `core/jellyfin_client.py` — JellyfinTrack reads MediaSources[0].Size
  alongside existing Bitrate read. None when 0 / missing.
- `core/navidrome_client.py` — NavidromeTrack reads `size` from
  the Subsonic song object (int coercion + None on parse fail).
- `core/soulsync_client.py` — SoulSyncTrack does os.path.getsize
  (only "server" where size has to come from disk).
- Plex needs no client-side change: track.media[0].parts[0].size
  is read directly inside insert_or_update_media_track.

Persistence — TWO separate insert paths:

(a) `database/music_database.py:insert_or_update_media_track` —
    Plex/Jellyfin/Navidrome flows. Reads file_size from Plex's
    MediaPart OR `track_obj.file_size` wrapper attribute (defensive
    Plex-attr-not-present check + > 0 type guard).
    INSERT writes the new column.
    UPDATE uses COALESCE(?, file_size) so a None from the server
    on a re-sync (rare Jellyfin Size omission) doesn't blank an
    existing value. Pinned via test.

(b) `core/imports/side_effects.py:record_soulsync_library_entry` —
    SoulSync standalone flow. Completely separate code path: the
    standalone deep scan moves files to staging for auto-import
    rather than calling insert_or_update_media_track. After the
    auto-import processes them, side_effects writes the tracks row
    directly. Reads file_size via os.path.getsize(final_path) at
    insert time (file is local) and includes it in the INSERT
    column list. SoulSync only does INSERT-if-not-exists (no
    UPDATE path), so no COALESCE concern.

Aggregator (`database/music_database.py:get_library_disk_usage`):
- SELECT COALESCE(SUM(file_size), 0), COUNT(file_size),
  COUNT(*) - COUNT(file_size) for the totals.
- Per-format breakdown done in Python via os.path.splitext over
  (file_path, file_size) rows — sidesteps SQLite's first-vs-last-dot
  ambiguity for paths like /music/Kendrick/M.A.A.D City/01.flac.
- Defensive: skips empty paths, paths without extension, and
  implausibly long extensions (>6 chars). Returns the full
  empty-shape dict (NOT a partial / undefined) when the column
  doesn't exist or queries fail, so the UI's `if (!data.has_data)`
  branch handles fresh installs cleanly.

API + UI:
- `core/stats/queries.py` — thin pass-through get_library_disk_usage
  matching the existing query-helper convention.
- `web_server.py` — new /api/stats/library-disk-usage endpoint
  mirroring the /api/stats/db-storage pattern.
- `webui/index.html` — new card in System Statistics above the
  Database Storage card.
- `webui/static/stats-automations.js` — _loadLibraryDiskUsage +
  _renderLibraryDiskUsage. Empty state: "Run a Deep Scan to
  populate (X tracks pending)". Partial: "X measured (+Y pending)".
  Full: total + format bars proportional to the largest format.
- `webui/static/style.css` — .stats-disk-* styled to match the
  Database Storage card.

Backward compatibility:
- Migration is additive; existing rows get NULL file_size; the
  empty-shape return from the aggregator means the UI renders
  cleanly without errors before any deep scan runs.
- Old installs upgrading will see "Run a Deep Scan to populate
  (N tracks pending)". Running their next deep scan fills sizes —
  the existing scan flow doesn't need any changes, just consumes
  the new track-wrapper attribute.

Tests:
- `tests/test_library_disk_usage.py` — 13 cases covering schema
  migration, NULL defaults on legacy inserts, fresh-install empty
  shape, summing with mixed NULL/known sizes, per-format breakdown,
  mixed-case extensions, paths with album-name dots, missing
  extensions, empty file_path, implausibly long extensions,
  JellyfinTrack.file_size persistence via insert_or_update_media_track,
  COALESCE preservation on null re-sync.
- `tests/imports/test_import_side_effects.py` — extended the
  existing record_soulsync_library_entry test to assert
  track_row['file_size'] == os.path.getsize(final_path), pinning
  the SoulSync-standalone path. Test fixture's tracks schema also
  updated to include the file_size column.

Verified: full suite 1813 pass (13 new, 1 existing-test extension),
ruff clean, smoke test populating + reading the column round-trips
correctly.

WHATS_NEW entry under '2.4.2' dev cycle.
2026-05-03 20:17:06 -07:00
BoulderBadgeDad
c66d307c0c
Merge pull request #486 from Nezreka/fix/replaygain-parses-summary-not-first-window
Fix: ReplayGain wrote same +52 dB gain to every track
2026-05-03 19:19:23 -07:00
Broque Thomas
776d195f71 Fix: ReplayGain wrote same +52 dB gain to every track
User report: every downloaded track in an album came out with
``replaygain_track_gain: +52.00 dB`` regardless of actual loudness.

Root cause: the parser at ``core/replaygain.py:79`` used
``re.search('I:\s+...')`` which returns the FIRST match. ffmpeg's
ebur128 filter emits ``I:`` per measurement window (running partial
integrated loudness) AND in a final Summary block. The first
per-window reading is at t=0.5s — almost always ~-70 LUFS because
nearly every track starts with silence / encoder padding. So:

    gain = RG2_reference - lufs = -18 - (-70) = +52.00 dB

…on EVERY track. Same regex pattern, same first per-window match,
same +52 dB written to every file's REPLAYGAIN_TRACK_GAIN tag.

Verified by running ffmpeg ebur128 against a real generated FLAC
and inspecting the stderr output — first per-window line at t=0.5s
shows ``I: -70.0 LUFS`` (silent intro), and the Summary block at
the end shows the real integrated value (e.g. ``I: -27.8 LUFS``
for the test sine wave). Old code captured the -70.0 reading.

Fix: anchor LUFS parsing to the ``Summary:`` block via
``stderr.rfind('Summary:')``. The Summary block is always emitted
last and contains the authoritative final integrated loudness.
Peak parsing already worked correctly (per-window output uses
``TPK:``/``FTPK:`` labels; only the Summary uses ``Peak:``), but
applied the same Summary anchor for consistency.

Defensive fallback: if no Summary block is present (truncated
output / unusual ffmpeg version), use the LAST per-window reading
instead of the first. Still better than the buggy first-window
behavior.

Smoke verified end-to-end: a freshly-generated FLAC of a -24 dBFS
sine wave now reports LUFS=-27.80, gain=+9.80 dB (correct, was
+52.00 before fix).

Tests: ``tests/test_replaygain_summary_parse.py`` — 7 cases pinning
the parser behavior with realistic ffmpeg ebur128 stderr samples:
- Summary value parsed correctly even when first per-window is -70
- Resulting gain is realistic (NOT +52)
- Two tracks with same first per-window but different summaries get
  different LUFS (regression assertion for "all tracks same gain")
- Per-window reading higher than Summary doesn't leak through
- Fallback to last per-window when Summary absent
- Clean RuntimeError raised when no LUFS values anywhere
- Peak still correctly anchored to Summary

Verified: full suite 1800 pass (7 new), ruff clean.

WHATS_NEW entry under '2.4.2' dev cycle.
2026-05-03 19:08:35 -07:00
BoulderBadgeDad
fbf4bad47a
Merge pull request #485 from Nezreka/fix/integrity-rejection-marks-task-failed
Fix: tasks showed Completed when file was quarantined
2026-05-03 18:50:33 -07:00
Broque Thomas
04a14f7e96 Fix: tasks showed Completed when file was quarantined
User caught downloading Kendrick Mr. Morale: three tracks (Rich
Interlude, Savior Interlude, Savior) showed  Completed in the modal
but were missing on disk. Log forensics revealed two layered bugs.

Bug 1 — Verification wrapper assumed success on quarantined files
(`core/imports/pipeline.py`):

The outer `post_process_matched_download_with_verification` had a
fallback at the "no `_final_processed_path` in context" branch that
marked the task completed and notified `success=True`. The inner
post-processor sets `_final_processed_path` only when the file
actually reaches its destination. Integrity-rejected files
(`_integrity_failure_msg` set) and race-guard-failed files
(`_race_guard_failed` set) get quarantined or skipped without ever
setting `_final_processed_path`, so they fell straight into the
"assume success" branch.

Confirmed in user's log:
  No _final_processed_path in context for task d5b88b84-... —
  cannot verify, assuming success

That line fired for the same task right after the integrity check
quarantined the source file. Result:  Completed in UI, file in
quarantine, never delivered.

Fix: explicit checks for `_integrity_failure_msg` and
`_race_guard_failed` markers BEFORE the assume-success fallback.
Either marker set → task status='failed' with descriptive
error_message + `_notify_download_completed(success=False)`. The
pre-existing assume-success behavior preserved when no failure
markers are set (some legitimate flows complete without setting
`_final_processed_path`).

Bug 2 — AcoustID skip-logic too lenient
(`core/acoustid_verification.py`):

The "language/script" exemption was:
  if best_score >= 0.95 and (title_sim >= 0.55 or
                              artist_sim >= ARTIST_MATCH_THRESHOLD):

The OR-clause fired for English-vs-English titles by the same artist
that share NO actual content. Confirmed in user's log: requested
"Rich (Interlude)" by Kendrick Lamar, AcoustID identified the audio
as "R.O.T.C. (interlude)" by Kendrick Lamar (a totally different
song from his 2010 mixtape) — same artist scored ≥ARTIST threshold,
shared word "interlude" pushed title_sim above 0.55, skip fired.
Verification returned SKIP instead of FAIL, the wrong file was
accepted as the answer for three different track requests.

Fix: skip now requires positive evidence the mismatch is a real
language/script case:
  (a) Non-ASCII chars present in either title AND artist matches strongly
      → real transliteration case (kanji ↔ romaji etc)
  (b) BOTH title_sim >= 0.80 AND artist_sim >= ARTIST threshold
      → minor punctuation/casing differences

English-vs-English with very different titles by the same artist no
longer skipped — verification correctly returns FAIL, the wrong file
gets quarantined, the new wrapper logic above marks the task failed.

Tests:
- `tests/test_integrity_failure_marks_task_failed.py` — 4 cases
  pinning the wrapper-level state machine: integrity marker → failed,
  race-guard marker → failed, no markers → still assumes success
  (legacy path preserved), integrity-failure-takes-priority over
  missing-final-path fallback.
- `tests/test_acoustid_skip_logic.py` — 7 cases pinning the skip
  exemption: user's R.O.T.C-vs-Rich case → FAIL (regression test),
  Savior-vs-R.O.T.C → FAIL (same bug surface), Japanese kanji →
  romaji → SKIP (real language case still works), MAAD vs M.A.A.D →
  PASS or SKIP (punctuation tolerance), low fingerprint score →
  never skipped, high score but artist mismatch → no longer skipped,
  Crown vs Crown of Thorns → no longer skipped.

Verified: full suite 1793 pass (11 new), ruff clean.

WHATS_NEW entry under '2.4.2' dev cycle.
2026-05-03 18:28:32 -07:00
BoulderBadgeDad
85ee0e8021
Merge pull request #484 from Nezreka/fix/album-mbid-consistency
Fix album MBID inconsistency: detector + persistent release-MBID cache
2026-05-03 17:55:26 -07:00
Broque Thomas
4b15fe0b75 Fix album MBID inconsistency: detector + persistent release-MBID cache
Discord report (Samuel [KC]): tracks of the same album sometimes carry
different MUSICBRAINZ_ALBUMID tags, which causes Navidrome (and other
media servers grouping by album MBID) to split the album into multiple
entries. Two-part fix — one for existing libraries, one for the root
cause that lets new imports drift.

Part 1 — Detector + fix action (catches existing dissenters):

`core/repair_jobs/mbid_mismatch_detector.py`:
- New helpers: `_read_album_mbid_from_file` and
  `_write_album_mbid_to_file` use the Picard-standard tag conventions
  (`TXXX:MusicBrainz Album Id` for MP3, `MUSICBRAINZ_ALBUMID` for
  FLAC/OGG, `----:com.apple.iTunes:MusicBrainz Album Id` for MP4).
- New scan phase `_scan_album_mbid_consistency` runs after the
  existing track-MBID scan: groups tracks by DB `album_id`, reads
  each track's embedded album MBID, finds the consensus
  (most-common) MBID via `Counter`, flags dissenters. Tracks without
  an album MBID at all are skipped (they don't break Navidrome —
  only an explicit MBID disagreement does). Albums where MBIDs are
  perfectly tied (no clear consensus) are skipped too — surface as
  a manual decision instead of fixing toward a 1/N tie.
- New finding type `album_mbid_mismatch` carries `consensus_mbid`,
  `wrong_mbid`, `consensus_count`, `total_tracks_with_mbid`, and a
  human-readable reason string.

`core/repair_worker.py`:
- Added `'album_mbid_mismatch': self._fix_album_mbid_mismatch` to the
  fix dispatch dict and to the `fixable_types` tuple so auto-fix +
  bulk-fix paths pick it up.
- New `_fix_album_mbid_mismatch` method reads `consensus_mbid` from
  finding details, resolves the dissenter's file path via the shared
  library resolver, calls `_write_album_mbid_to_file` to rewrite the
  tag in place. Doesn't touch the album's other tracks (they're
  already in agreement).

Part 2 — Root cause fix (prevents new SoulSync imports from drifting):

The original in-memory `mb_release_cache` in `core/metadata/source.py`
maps `(normalized_album, artist) -> release_mbid` so per-track
enrichment of the same album hits the cache and writes the same
MUSICBRAINZ_ALBUMID to every track. That cache is bounded (4096
entries) and in-process — so cache eviction (when other albums are
processed in between) and server restart can BOTH cause
inconsistency. Per-track album-name variation (e.g. some tracks
tagged `"Album"`, others tagged `"Album (Deluxe)"`) and per-track
artist variation (features) make it worse.

`core/metadata/album_mbid_cache.py` (new module):
- DB-backed `lookup(normalized_album, artist) -> release_mbid` and
  `record(...)` functions. Same key shape as the in-memory cache.
- Strict additive design: every public function is wrapped in
  try/except and degrades to None / no-op on ANY database error.
  The existing in-memory cache + MusicBrainz lookup remains the
  authoritative fallback. If this module breaks, downloads continue
  exactly as they would today.

`database/music_database.py`:
- New `mb_album_release_cache` table with composite primary key
  `(normalized_album_key, artist_key)`. Reverse-lookup index on
  `release_mbid` for future debug tooling. Created via the existing
  `CREATE TABLE IF NOT EXISTS` migration pattern — idempotent, no
  schema version bump needed.

`core/metadata/source.py`:
- Surgical change inside the existing `embed_source_ids`
  in-memory-cache-miss branch: BEFORE calling MusicBrainz, consult
  the persistent cache. If a previous SoulSync run already resolved
  this album's release MBID, reuse it. After a successful MB lookup,
  store in BOTH caches. Both calls wrapped in defensive try/except
  so any failure falls through to existing logic.

Tests:
- `tests/metadata/test_album_mbid_cache.py` — 16 cache tests:
  round-trip, idempotent re-record, overwrite semantics, clear_all,
  album+artist independence (no Greatest Hits collisions),
  defensive None-on-empty-input, graceful degradation when the DB
  is unavailable / connection raises / commit fails, schema sanity
  (table + index exist after init).
- `tests/test_album_mbid_consistency.py` — 13 detector tests:
  tag read/write round-trip on real FLAC files, Picard-standard tag
  descriptors, defensive paths (unreadable file, empty input),
  detector behavior (agreement → no flags, lone dissenter → flag,
  ties → no flag, single-track albums → skipped, no-MBID tracks →
  skipped, unresolvable file paths → skipped).
- `tests/metadata/test_metadata_enrichment.py` — added autouse
  fixture monkeypatching the persistent cache to no-op for tests in
  this file. The existing tests pin per-call MB counts and
  in-memory cache state; without the fixture, persistent rows from
  earlier tests would bypass the MB call. Persistent layer has its
  own dedicated tests.

Verified: 1782 tests pass (29 new), ruff clean, smoke test confirms
end-to-end cache round-trip works.

WHATS_NEW entry under '2.4.2' dev cycle.
2026-05-03 17:16:39 -07:00
BoulderBadgeDad
e3ff9f7b26
Merge pull request #483 from Nezreka/fix/lidarr-album-downloader-bugs
Fix three Lidarr bugs that prevented it from being a real download so…
2026-05-03 15:53:28 -07:00
Broque Thomas
e577f3cf1f Fix three Lidarr bugs that prevented it from being a real download source
Investigation surfaced that Lidarr was wired into the orchestrator but
the actual download flow had blockers:

1. **Wrong file misfiled.** Lidarr grabs whole albums; SoulSync's
   matched-context post-processing wants the SPECIFIC track the user
   requested. Old code copied every track in the album and reported
   `imported_files[0]` as `file_path` — almost always pointing to
   track 1, not the user's actual track. Post-processing then tagged
   track 1 with the requested track's metadata. Misfiling on every
   real download.

   Fix: parse the wanted track title out of the dispatch display name
   (which `_search_sync` already builds as
   `f"{artist} - {album} - {track_title}"`), look it up against
   Lidarr's `track` API, resolve the matching `trackFileId` to a path,
   and copy ONLY that file. Punctuation-tolerant fuzzy match handles
   the common "m.A.A.d city" vs "maad city" case. Album-level
   dispatches (no track in the display) preserve the old first-file
   fallback so existing album-grab UX is unchanged.

2. **Hardcoded `metadataProfileId=1`.** Required by Lidarr's
   artist-add API. On installs where the user deleted/recreated
   metadata profiles, that id no longer exists and the call fails
   with HTTP 400 — which silently breaks every download flow that
   needs to add an artist. Real-world Lidarr installs do this all
   the time.

   Fix: `_get_metadata_profile_id()` calls Lidarr's `metadataprofile`
   API and returns the first available id. Falls back to 1 only when
   the API call fails entirely (preserves previous behavior so this
   change can't make things worse).

3. **Polling never broke the outer loop on completion.** The inner
   `for item in queue['records']` had `break` statements at status
   transitions, but those only escaped the queue iteration — the
   outer `for poll in range(max_polls)` kept spinning until the
   600-poll timeout even after the album was clearly imported.
   `for/else` semantics didn't apply because completion was detected
   inside the inner loop, not by it running to exhaustion.

   Fix: replaced with an explicit `download_complete` flag set when
   `album/{id}` reports `trackFileCount > 0` (the authoritative
   completion signal — works even when the queue record disappeared
   between polls). Outer loop breaks immediately once the flag flips.

Helper functions added: `_extract_wanted_track_title` (staticmethod,
splits the display name; >=3 parts → track dispatch, 2 parts → album
dispatch), `_normalize_for_match` (lowercase + strip punctuation +
collapse whitespace for fuzzy compare), `_title_similarity` (cheap
score: equal=1.0, substring=0.85, token-overlap-ratio otherwise),
`_pick_track_file_for_wanted` (orchestrates the API calls).

Settings tooltip updated to be honest about Lidarr's natural shape:
album-grabber, no-op for playlist sync, hybrid mode falls through to
other sources for track searches. Sets correct expectations.

Tests: `tests/test_lidarr_download_client.py` — 21 isolated tests
covering pure helpers (title extraction, normalization, similarity)
and the file-picker integration paths (matching path, punctuation
tolerance, below-threshold fallback, missing trackFileId, missing
file on disk, API failures, malformed responses). No live Lidarr
needed — `_api_get` mocked at the client boundary.

Isolation: ONLY touches `core/lidarr_download_client.py`, the Lidarr
settings tooltip in `webui/index.html`, the Lidarr WHATS_NEW entry
in `webui/static/helper.js`, and the new test file. No changes to
the orchestrator, other download clients, the import pipeline,
side_effects, web_server.py, settings.js, or any shared validation /
monitor / task_worker code. Other download sources are not affected
in any way.

Verified: 1753 tests pass (21 new), ruff clean.
2026-05-03 15:49:38 -07:00
BoulderBadgeDad
c8618ba0d4
Merge pull request #482 from Nezreka/feat/soundcloud-integration
Feat/soundcloud integration
2026-05-03 13:26:39 -07:00
Broque Thomas
c9dbf421dc SoundCloud progress UI fix: include SoundCloud in cached transfer lookup
User: SoundCloud downloads finish correctly but the modal stays at
"Downloading... 0%" until "Processing..." flips on. Live percentage
never updates.

Root cause: my live-progress fix in 8de4a18 made the SoundCloud client
compute progress correctly via fragment_index/fragment_count — but the
percent never reached the modal because `get_cached_transfer_data` in
web_server.py iterates `[youtube, tidal, qobuz, hifi, deezer_dl,
lidarr]` to build the lookup that drives `task.progress`. SoundCloud
was missing from that loop, so `live_transfers_lookup` had no entry
for SoundCloud downloads, so `live_info` lookup at
`core/downloads/status.py:135` always missed, so `task_status['progress']`
defaulted to 0 the entire time.

Frontend was reading `task.progress` (rendered as
"Downloading... ${task.progress}%" in `webui/static/downloads.js:3142`),
which stayed at 0. The percentComplete field that the
`/api/downloads/status` endpoint includes for SoundCloud was correct;
this only affected the cached lookup used by the V2 task tracker.

Fix: include SoundCloud in the iteration. Used `getattr` fallback to
match the same pattern I used in `core/downloads/monitor.py` so older
soulseek_client snapshots without the attribute don't AttributeError.

Bonus: also wired the SoundCloud client's `set_shutdown_check` callback
in the startup block right after HiFi's. Previously the cooperative-
cancellation hook in `_progress_hook` would never fire on shutdown
because `self.shutdown_check` was None.

Verified: full suite 1732 passed, ruff clean. yt-dlp probe confirms
fragment_index / fragment_count are populated correctly during HLS
download (164 hook calls for a 19-fragment track), so the now-
exposed progress will increment smoothly from 0 to 99.9 and then
flip to Completed.
2026-05-03 13:22:35 -07:00
Broque Thomas
8de4a186b7 Fix three SoundCloud integration gaps surfaced by smoke testing
User report: switched download source to SoundCloud and noticed:
1. Download progress % stays at 0 until "suddenly done" — no live progress
2. Sidebar status indicator next to "SoundCloud" label is red
3. Dashboard service status card still shows "Soulseek" as the source name

Fix 1 — Live progress for HLS-segmented SoundCloud downloads
(`core/soundcloud_client.py`):
- yt-dlp's `total_bytes` / `total_bytes_estimate` for HLS describes the
  CURRENT FRAGMENT, not the whole download. So the byte-based
  percentage stayed near 0 the entire time — until 'finished' fired.
- Added `_update_download_progress_fragmented` which uses
  `fragment_index` / `fragment_count` (which yt-dlp DOES populate
  accurately for HLS) to compute a meaningful percentage. Total size
  is extrapolated from per-fragment average for the bytes/remaining
  display. Time-remaining estimate uses elapsed/index seconds-per-
  fragment.
- The progress hook prefers fragment progress when both fragment_index
  and fragment_count are present; falls back to byte-based for
  non-fragmented (progressive MP3) downloads. Five new unit tests pin
  the fragment-progress math, the 99.9% cap, and the defensive
  zero-index / unknown-id paths.

Fix 2 — Sidebar status indicator stays green for SoundCloud mode
(`web_server.py`):
- The `/api/status` route's `serverless_sources` tuple decides whether
  to even probe slskd. SoundCloud (and Lidarr) were missing — so when
  the active source was SoundCloud, the route fell through to "test
  slskd, mark not-relevant", which set `connected: False` and turned
  the sidebar dot red even though SoundCloud was working.
- Added `'soundcloud'` and `'lidarr'` to the tuple. Both are
  serverless from slskd's perspective, so the dot now stays green
  whenever they're the active source.

Fix 3 — Dashboard service card title shows the active source
(`webui/static/shared-helpers.js`):
- The dashboard's "Download Source" card has its own
  `sourceNames` map at line 3351 (separate from the sidebar map I
  already updated at 3396). Missed it during the integration PR.
- Added `'lidarr'` and `'soundcloud'` so the card title now reads
  "SoundCloud" / "Lidarr" instead of falling back to "Soulseek".

Bonus — Dashboard "Test Connection" button works for SoundCloud
(`core/connection_test.py`):
- The dashboard's Test Connection button on the download-source card
  sends `service` based on the active source — so for SoundCloud it
  was sending `service='soundcloud'`. `run_service_test` had no
  branch for it, so it fell through to "Unknown service." and the
  button always failed.
- Added a `soundcloud` branch that mirrors `/api/soundcloud/status`
  behavior: confirms yt-dlp is installed, runs a real cheap probe,
  returns a meaningful pass/fail. (HiFi has the same gap but no
  user reported it; out of scope for this fix.)

Verified:
- 41 unit tests pass (5 new fragment-progress tests added)
- Full suite 1732 passed
- Ruff clean
2026-05-03 13:09:02 -07:00
Broque Thomas
75fe04907f Wire SoundCloud as a first-class download source
Plug the previously-built SoundcloudClient (PR #478, the build-and-verify
phase) into every place a download source needs to appear. Follows the
same wiring contract as Tidal/Qobuz/HiFi/Deezer/Lidarr — orchestrator
routing, hybrid-mode picker, search dispatch, queue/cancel/clear,
provenance + library history, sidebar source label, settings UI all
work plug-and-play.

Backend wiring:
- `core/download_orchestrator.py` — import SoundcloudClient, _safe_init
  it at startup, add to _client() lookup, get_source_status(),
  check_connection's sources_to_check default, search source_names map,
  search_and_download_best _streaming_sources tuple, download
  source_map + source_names, and every iteration loop in
  reload_settings download-path-update / get_all_downloads /
  get_download_status / cancel_download (route + iterate) /
  clear_all_completed_downloads / cancel_all_downloads.
- `core/downloads/monitor.py` — added SoundCloud to the per-client
  loop that fetches active downloads outside the orchestrator (uses
  getattr fallback for older soulseek_client snapshots).
- `core/downloads/task_worker.py` — added SoundCloud (and Lidarr,
  which was missing too — bonus fix) to source_clients dict for hybrid
  fallback dispatch.
- `core/downloads/validation.py` — added 'soundcloud' to
  _streaming_sources so SoundCloud results go through the matching
  engine validation path instead of the Soulseek quality-filter path.
- `core/imports/side_effects.py` — three call sites: source_map for
  download_source label written to library_history, streaming-source
  guard for the `||`-encoded stream_id parsing, and source_service
  map for provenance recording. All three now include 'soundcloud'.
- `web_server.py` — five streaming-source detection tuples updated.
  New `/api/soundcloud/status` endpoint returns
  {available, configured, reachable} mirroring the Deezer/HiFi
  status-endpoint pattern; reachability runs a real cheap yt-dlp
  search so the settings Test Connection button gives a meaningful
  pass/fail signal.
- `config/settings.py` — added empty `soundcloud_download` defaults
  block so future tier-2 OAuth (SoundCloud Go+ session) doesn't have
  to migrate existing configs.

Frontend:
- `webui/index.html` — new `<option value="soundcloud">` in the
  download-source-mode dropdown, SoundCloud added to both hidden
  legacy hybrid-source selects, new settings container with info
  text + Test Connection button.
- `webui/static/settings.js` — HYBRID_SOURCES entry (with the
  SoundCloud cloud SVG icon), _hybridSourceEnabled default,
  updateDownloadSourceUI container display, allSources for legacy
  hybrid picker, testSoundcloudConnection function (hits the new
  status endpoint, color-codes the result), saveSettings
  soundcloud_download empty block.
- `webui/static/shared-helpers.js` — sidebar source-name map
  includes SoundCloud + Lidarr (Lidarr was also missing, bonus fix).
- `webui/static/helper.js` — WHATS_NEW entry under '2.4.2' dev cycle
  describing the user-visible change in the chill terse voice.

Tests:
- `tests/test_download_orchestrator_soundcloud.py` — 14 integration
  tests verifying the wiring: client constructed at startup, _client
  lookup resolves 'soundcloud', get_source_status includes it,
  download dispatcher routes username='soundcloud' to the SoundCloud
  client (and unknown usernames still fall back to Soulseek), hybrid
  search iterates SoundCloud when in order and skips it cleanly when
  unconfigured, get_all_downloads / get_download_status / cancel /
  clear walk SoundCloud, soundcloud-only mode dispatches only to
  SoundCloud, _streaming_sources tuple in validation includes
  'soundcloud'.
- `tests/downloads/test_download_orchestrator.py` — added
  `soundcloud` to the test fixture's _build_orchestrator helper so
  the new orchestrator attribute doesn't AttributeError in pre-
  existing tests that bypass __init__.

Verified:
- Full suite green (1728 passed, 2 deselected for soundcloud_live)
- Ruff clean
- Live SoundCloud-only mode search returns 25 SoundCloud tracks for
  "kendrick lamar luther" in <2s, returning properly-shaped
  TrackResult objects with username='soundcloud' and dispatch-key
  filename ready for the download path.

Out of scope (intentional deferrals):
- SoundCloud Go+ OAuth tier (256 kbps AAC) — anonymous-only for now.
  Adding auth later is a settings-page extension, no orchestrator
  changes needed.
- Album/playlist support — SoundCloud has playlists but they don't
  map to the album model the rest of SoulSync expects. Singles only.
2026-05-03 12:54:21 -07:00
BoulderBadgeDad
e1d6e4d51f
Merge pull request #481 from Nezreka/feat/soundcloud-client
Build SoundCloud download client (not yet wired into app)
2026-05-03 12:10:44 -07:00
Broque Thomas
583c4f1e49 Build SoundCloud download client (not yet wired into app)
Discord request (Toasti): some tracks (DJ mixes, sets, removed Spotify
content) only live on SoundCloud. Add SoundCloud as an option for the
existing multi-source download dispatch.

This commit only ships the client + tests. Integration into the search
dispatch / settings UI / web_server.py routes is intentionally deferred
to a follow-up PR — the user-requested workflow is build-and-verify
in isolation first, then wire up.

`core/soundcloud_client.py`:
- SoundcloudClient class mirrors the public surface of every other
  download client (TidalDownloadClient, QobuzClient, HiFiClient,
  DeezerDownloadClient): __init__(download_path), set_shutdown_check,
  is_available / is_configured / is_authenticated, async check_connection,
  async search returning (List[TrackResult], List[AlbumResult]),
  async download returning a download_id, _download_thread_worker /
  _download_sync / _update_download_progress, async get_all_downloads /
  get_download_status / cancel_download / clear_all_completed_downloads.
- Underlying lib: yt-dlp (already in requirements.txt as 2026.3.17).
- Anonymous-only — public SoundCloud tracks at the cap quality (typically
  128 kbps MP3, occasionally 256 kbps AAC depending on the upload).
  No FLAC ever; SoundCloud doesn't expose lossless. OAuth tier for
  SoundCloud Go+ is documented in the module header as a future tier.
- Returns standard TrackResult / DownloadStatus dataclasses from
  core.soulseek_client so downstream matching/post-processing stays
  source-agnostic.
- Filename dispatch key encodes track_id + permalink_url + display_name
  so the download worker has everything without re-querying.
- Heuristic "Artist - Title" parser handles SoundCloud uploaders'
  typical title format; falls back to uploader handle as artist when
  the title doesn't have a separator.
- Defensive: search returns empty on bad input, missing yt-dlp, or any
  raised exception. Download sync rejects files under 100KB (preview
  snippets / broken responses) and cleans them up.
- Cooperative cancellation via shutdown_check inside yt-dlp's
  progress_hooks. Cancelled state survives the download thread's
  terminal-state assignment.

`tests/test_soundcloud_client.py`:
- 37 unit tests with yt-dlp stubbed: search shape correctness, the
  artist/title heuristic, the dispatch-key roundtrip, the download
  state machine (success / failure / shutdown / Cancelled-state
  preservation), the progress emitter (progress capping, time
  remaining), defensive paths (missing yt-dlp, raising yt-dlp,
  malformed entries, empty entries), and the cancel/clear ledger
  operations.
- 2 live integration tests gated behind `-m soundcloud_live` so CI
  doesn't run them by default. Run locally with:
    python -m pytest tests/test_soundcloud_client.py -m soundcloud_live -v
- All 37 unit tests pass; both live tests pass against real SoundCloud.
- Verified end-to-end with a real album download (Kendrick GNX, 12/12
  tracks, 4-7 MB each, completed under 60s per track).

`pyproject.toml`:
- Register the `soundcloud_live` pytest marker so the unknown-mark
  warning is suppressed and the live tests can be cleanly gated.

Not changed: web_server.py, settings UI, search dispatch, matching
engine, WHATS_NEW. Integration is the next PR.
2026-05-03 11:58:25 -07:00