Discord (DXP4800 NAS, 7148 tracks): library updates kept dying with "Update appears stuck — no
progress for 300s (last phase: Fetching all tracks in bulk...)". not actually hung.
root cause: the bulk track/album fetch used a single 10000-item page, so a whole library came
back in ONE request that emitted NO progress while in flight. the watchdog (database_update_health)
kills a job with no progress for 300s — so on a slow server that one silent request tripped it even
though it was alive, not stuck. raising the timeout cap only buys the silent request more rope; a
bigger library or slower disk just needs a higher number. the per-batch progress line also only ran
when there was a NEXT page, so a sub-page-size library reported nothing at all.
fix: extract a pure paginate_all_items seam (core/library/bulk_paginate.py) that pages in 1000s and
reports progress after EVERY page — so the watchdog is fed on a cadence set by page size, not library
size, and can't starve mid-fetch no matter how big the library. both Jellyfin bulk loops (tracks +
albums, same defect) now route through it. preserves the failure-shrink resilience (halve to a floor,
then give up). does NOT change what's fetched — same query, fields, items.
note: changes nothing about WHICH tracks come back; only how they're paged + that every page reports.
keep the raised cap on dev as a margin — this is the actual fix. Plex/Navidrome don't share the
pattern (checked). 9 seam tests incl. the watchdog-feed invariant (progress count scales with
N/page_size, never one call for the whole library) + the sub-page regression + failure-shrink.
467 jellyfin/library tests green, ruff clean.
Every library track was stored with disc_number=1 because the Jellyfin/Plex/Navidrome scan parsed the track number but never the disc field. Multi-disc albums collapsed onto disc 1, so disc-2+ tracks were mis-filed (shown under disc 1) and flagged 'missing' — the frontend title-fallback band-aid couldn't recover it (breaks on iTunes title mismatches).
Now the shared insert_or_update_media_track reads the disc number (Jellyfin .discNumber=ParentIndexNumber, Navidrome .discNumber, Plex .parentIndex), floors to >=1, and stores it in the INSERT + UPDATE. The disc_number column is ensured on init (it was only added by a migration that doesn't run on fresh installs, so the new INSERT would have hard-failed for new users). The enhanced album view already carries disc_number through (SELECT * -> dict), so the display fixes itself once the column is populated — a re-scan backfills existing libraries. Seam-tested across Jellyfin/Navidrome/Plex shapes + the floor-to-1 + re-scan-update cases.
Completes align across all three servers. Jellyfin reorders in place: DELETE the
extra entries (Mirror) then POST /Playlists/{id}/Items/{entryId}/Move/{index} for
each desired track in ascending order — so the playlist's poster/name/Id survive
(no delete-recreate), same as Plex/Navidrome. Mirrors the existing reconcile path's
entry-id handling (PlaylistItemId via /Playlists/{id}/Items).
- jellyfin reorder_playlist() + get_playlist_track_ids(); reuses the shared, tested
plan_align_rewrite planner (no new pure logic).
- /align endpoint + frontend gate now cover navidrome|plex|jellyfin.
UNTESTED LIVE: no Jellyfin instance to verify against (same status as the Navidrome
path). Plex is the only one confirmed working end-to-end so far.
carlosjfcasero round 2: after 6fa956d6 append stopped recreating the playlist,
but every sync re-appended ALL matched tracks — every track N times. His log
shows it plainly: "added 22 new tracks to 'Disney' (skipped 0 already present)"
on a playlist that already had those 22.
Root cause: the dedupe read `{t.id for t in get_playlist_tracks(...)}` — but
JellyfinTrack only defines `ratingKey`, never `id`, so the existing-ids set was
ALWAYS empty and everything looked new. NavidromeTrack is also ratingKey-only,
so the Navidrome append had the identical bug. Plex (plexapi ratingKey) was
fine. The existing tests were green because they mocked existing tracks as
SimpleNamespace(id=...) — encoding the same wrong assumption as the code.
Fix:
- New pure planner plan_playlist_append(current, desired) in
core/sync/playlist_edit.py (next to the reconcile planner): order-preserving,
drops already-present ids, dedupes within desired, stringifies (Emby numeric
vs string safe).
- Jellyfin/Emby: existing ids fetched from the canonical
/Playlists/{id}/Items endpoint (same as reconcile — works for Jellyfin GUIDs
and Emby numeric ids), ratingKey fallback if that request fails.
- Navidrome: dedupe on ratingKey (the attribute that actually exists).
Tests: planner (skip-present incl. the reporter's unchanged-playlist case,
desired-order, dupes-within-desired, int/str ids, empties) + the append-mode
suite rewritten to pin the REAL shapes (raw Items dicts for Jellyfin,
ratingKey objects for Navidrome) + a new fallback-path test. 524
playlist/sync/jellyfin/navidrome tests pass.
JellyfinArtist sets self.thumb (→ artists.thumb_url on scan); JellyfinAlbum
never did. So for Jellyfin/Emby the library scan read getattr(album, 'thumb',
None) == None and albums.thumb_url stayed empty for the WHOLE library. Two
downstream effects: blank album art in the web UI, and the Cover Art Filler
flagging every album as "missing cover art" (db_missing = empty thumb_url) —
making the filler the only thing that ever populated the column, which is
backwards. It should come from the scan, like artist thumbs do.
Fix: JellyfinAlbum.thumb = /Items/<id>/Images/Primary (same shape as the artist
thumb, which already normalizes + displays via fix_artist_image_url). Now the
scan stores album thumbs, the UI shows art, and the filler only flags albums
that are genuinely missing art. Navidrome + Plex already set album thumbs.
Tests: album exposes the Primary-image thumb, None without an id, same shape as
the artist thumb.
Replace mode (default) deletes + recreates the server playlist every sync,
which wipes its custom image, description, and identity. Add an opt-in
'reconcile' sync mode that edits the existing playlist in place — adds the
tracks now in the source, removes the ones gone — without destroying the
object, so the user's custom art/description survive.
- Pure planner plan_playlist_reconcile(current, desired) -> {add, remove}.
- Per-client reconcile_playlist: Plex addItems/removeItems on the same object;
Navidrome Subsonic updatePlaylist delta (songIdToAdd / descending
songIndexToRemove); Jellyfin add + remove-by-PlaylistItemId on /Playlists/{id}/Items.
- sync_service: reconcile branch with a replace FALLBACK (if a server's in-place
edit is unavailable/fails, sync still succeeds destructively — logged loudly).
- Default stays 'replace' (no behavior change). New Settings > Playlist sync mode
picker (replace/reconcile/append) backed by playlist_sync.mode; per-request
sync_mode still overrides.
- Reconcile skips the post-sync source-image push so a custom poster isn't
re-clobbered (the bug).
Tests: planner (add/remove/dedupe/order/empty) + reconcile-or-replace dispatch
(success / false-fallback / exception-fallback / no-method). Per-server in-place
API calls need dev validation against real Plex/Jellyfin/Navidrome.
NOTE: opt-in only; default behavior unchanged.
Discord report (CJFC, 2026-04-26): syncing a Spotify playlist to the
server overwrote anything manually added to the server-side playlist.
The fix adds a per-sync mode picker next to the Sync button on the
playlist details modal — Replace (default, current delete-recreate
behavior) or Append only (preserves existing tracks, only adds new
ones). Useful when the source platform caps playlist size and the
user is manually building beyond it on the server.
Implementation:
* New `append_to_playlist(name, tracks)` method on Plex / Jellyfin /
Navidrome clients. Each uses the server's NATIVE append API:
- Plex: `existing_playlist.addItems(new_tracks)`
- Jellyfin: `POST /Playlists/<id>/Items?Ids=...&UserId=...`
- Navidrome: Subsonic `updatePlaylist?songIdToAdd=...`
Falls back to `create_playlist` when the playlist doesn't exist
yet (first sync). No delete-recreate, no backup playlist created
(preserves playlist creation date + metadata + non-soulsync-managed
tracks).
* Dedup-by-server-native-id (ratingKey for Plex, GUID for Jellyfin,
song-id for Navidrome) — never re-adds a track already on the
playlist. Server-native identity, not fuzzy title+artist match,
so it can't false-collide.
* `sync_service.sync_playlist` accepts `sync_mode='replace'|'append'`
kwarg. Single if/else branch dispatches to `append_to_playlist` or
`update_playlist`. Threaded through `core/discovery/sync.run_sync_task`
and the `/api/sync/start` HTTP handler. Validation on the API rejects
unknown mode strings (defaults to 'replace').
* Frontend: per-playlist `<select id="sync-mode-${id}">` rendered next
to the Sync button in both modal renderers (sync-spotify.js for
Spotify playlists, sync-services.js for Deezer ARL playlists).
`startPlaylistSync` reads the select at click time; missing select
(other callers like discover.js) defaults to 'replace' so backward
compat preserved without per-call-site updates.
* SoulSync standalone has no playlist methods at all and the modal
hides the Sync button entirely on it via `_isSoulsyncStandalone` —
dispatch never reaches that path, no defensive fallback needed.
15 new tests pin per-server append behavior:
- missing playlist → create_playlist delegation
- dedup filtering (existing IDs skipped, only new tracks added)
- empty new-track set short-circuits without API call
- failure paths return False without raising
- contract listing (KNOWN_PER_SERVER_METHODS includes
'append_to_playlist'; Plex / Jellyfin / Navidrome all implement)
Plus tests/discovery/test_discovery_sync.py fake `sync_playlist`
fixture got `sync_mode='replace'` default to match the new signature
(was breaking after the kwarg add; now passing).
WHATS_NEW entry under new '2.6.0' block (hidden by
`_getLatestWhatsNewVersion` until next release bump).
Closes CJFC discord request.
Catches the silent excepts the awk-based earlier sweeps missed:
- Bare `except:` followed by `pass` (also swallows KeyboardInterrupt
and SystemExit — actively wrong). Upgraded to `except Exception as
e: logger.debug("...: %s", e)`. ~14 sites across connection_detect,
soulseek_client, listenbrainz_manager, watchlist_scanner,
youtube_client, navidrome_client, jellyfin_client, web_server.
- `except Exception:` + pass that the awk pattern missed (e.g.
multi-line or unusual whitespace). ~31 sites across automation_engine,
database_update_worker, music_database, spotify_client, web_server,
others.
- 14 legitimate cleanup sites left silent with explicit `# noqa: S110`
+ comment explaining why (atexit handlers, finally-block conn.close
calls). Logging during shutdown can itself crash because file handles
get torn down before the handler fires.
Also enables `S110` rule in pyproject.toml so this pattern fails CI
going forward — drift fails at PR review instead of at runtime against
a wedged worker thread. Tests path keeps S110 ignored (test fixtures
legitimately use try-except-pass for cleanup).
Adds a WHATS_NEW entry to helper.js summarizing the full #369 sweep.
Verified: `python -m ruff check .` → All checks passed.
Verified: `python -m pytest tests/` → 2188 passed.
Closes#369
Plex / Jellyfin / Navidrome each defined a near-identical
XTrackInfo (id / title / artist / album / duration / track_number /
year / rating) and XPlaylistInfo (id / title / description /
duration / leaf_count / tracks). Three classes that grew up by
copy-paste — not a real contract difference.
Lifted both to core/media_server/types.py as canonical TrackInfo +
PlaylistInfo. Per-server constructors live as classmethods on the
unified class (TrackInfo.from_plex_track, PlaylistInfo.from_plex_playlist)
matching the metadata Album.from_X_dict pattern Cin's POC uses.
Heavy plexapi imports stay lazy under TYPE_CHECKING.
- core/plex_client.py / jellyfin_client.py / navidrome_client.py:
per-server XTrackInfo / XPlaylistInfo dataclass definitions
removed; each module now imports TrackInfo + PlaylistInfo from
the neutral package and uses the shared name internally.
- core/matching_engine.py: was annotating callers with PlexTrackInfo
even though sync_service hands it Jellyfin / Navidrome instances
at runtime when those servers are active. Annotation is now the
unified TrackInfo, so signatures match the actual contract.
- services/sync_service.py: same import + annotation update.
Apply the Cin-1 / Cin-2 pattern from the download refactor PR to the
media server engine PR before review.
Cin-1 — explicit inheritance:
- PlexClient, JellyfinClient, NavidromeClient, SoulSyncClient now
explicitly inherit MediaServerClient instead of relying on
structural typing alone. Pre-change a reader of plex_client.py
had no way to know the class was supposed to satisfy the contract.
- Removed the engine + registry re-exports from
core/media_server/__init__.py to break the circular import that
the inheritance change introduced (importing the package now
triggered a chain that loaded clients before their base class
resolved). Submodules import directly: from
core.media_server.engine import MediaServerEngine, etc.
- Conformance test now also asserts isinstance() / issubclass()
against MediaServerClient — drift in any class fails at the test
boundary instead of at runtime.
Cin-2 — generic accessors + singleton:
- engine.configured_clients() — replaces the legacy per-server
`if X and X.is_connected(): clients[name] = X` chains in
web_server.py.
- engine.reload_config(name=None) — generic dispatch, so callers
pass the server name instead of reaching for plex_client.reload_config()
directly.
- get_media_server_engine() / set_media_server_engine() singleton
factory matching the get_metadata_engine() / get_download_orchestrator()
shape. web_server.py boots via set_media_server_engine(...) so
factory + global handle share state.
- 7 new tests pin the accessors + singleton behaviour.
Discord request (Samuel [KC]): show how much disk space the library
takes on the Stats page. Implementation piggybacks on the existing
deep scan — Plex/Jellyfin/Navidrome all return file size in their
track API responses, so we read it during the deep scan and store
it on the tracks row. Aggregation is then a single SQL query — no
filesystem walk, no extra I/O during the scan, no separate stat
job. SoulSync standalone gets size from os.path.getsize at insert
time (different code path; the file is local when we write the row).
Schema (`database/music_database.py`):
- New `file_size INTEGER` column on `tracks`. Migration uses the
established `try SELECT, except ALTER TABLE ADD COLUMN` pattern.
Idempotent; safe on existing installs. NULL on legacy rows so
they don't contribute to totals until next deep scan refreshes.
- Added the column to the canonical CREATE TABLE so fresh installs
get it without going through the migration path.
Track-object plumbing:
- `core/jellyfin_client.py` — JellyfinTrack reads MediaSources[0].Size
alongside existing Bitrate read. None when 0 / missing.
- `core/navidrome_client.py` — NavidromeTrack reads `size` from
the Subsonic song object (int coercion + None on parse fail).
- `core/soulsync_client.py` — SoulSyncTrack does os.path.getsize
(only "server" where size has to come from disk).
- Plex needs no client-side change: track.media[0].parts[0].size
is read directly inside insert_or_update_media_track.
Persistence — TWO separate insert paths:
(a) `database/music_database.py:insert_or_update_media_track` —
Plex/Jellyfin/Navidrome flows. Reads file_size from Plex's
MediaPart OR `track_obj.file_size` wrapper attribute (defensive
Plex-attr-not-present check + > 0 type guard).
INSERT writes the new column.
UPDATE uses COALESCE(?, file_size) so a None from the server
on a re-sync (rare Jellyfin Size omission) doesn't blank an
existing value. Pinned via test.
(b) `core/imports/side_effects.py:record_soulsync_library_entry` —
SoulSync standalone flow. Completely separate code path: the
standalone deep scan moves files to staging for auto-import
rather than calling insert_or_update_media_track. After the
auto-import processes them, side_effects writes the tracks row
directly. Reads file_size via os.path.getsize(final_path) at
insert time (file is local) and includes it in the INSERT
column list. SoulSync only does INSERT-if-not-exists (no
UPDATE path), so no COALESCE concern.
Aggregator (`database/music_database.py:get_library_disk_usage`):
- SELECT COALESCE(SUM(file_size), 0), COUNT(file_size),
COUNT(*) - COUNT(file_size) for the totals.
- Per-format breakdown done in Python via os.path.splitext over
(file_path, file_size) rows — sidesteps SQLite's first-vs-last-dot
ambiguity for paths like /music/Kendrick/M.A.A.D City/01.flac.
- Defensive: skips empty paths, paths without extension, and
implausibly long extensions (>6 chars). Returns the full
empty-shape dict (NOT a partial / undefined) when the column
doesn't exist or queries fail, so the UI's `if (!data.has_data)`
branch handles fresh installs cleanly.
API + UI:
- `core/stats/queries.py` — thin pass-through get_library_disk_usage
matching the existing query-helper convention.
- `web_server.py` — new /api/stats/library-disk-usage endpoint
mirroring the /api/stats/db-storage pattern.
- `webui/index.html` — new card in System Statistics above the
Database Storage card.
- `webui/static/stats-automations.js` — _loadLibraryDiskUsage +
_renderLibraryDiskUsage. Empty state: "Run a Deep Scan to
populate (X tracks pending)". Partial: "X measured (+Y pending)".
Full: total + format bars proportional to the largest format.
- `webui/static/style.css` — .stats-disk-* styled to match the
Database Storage card.
Backward compatibility:
- Migration is additive; existing rows get NULL file_size; the
empty-shape return from the aggregator means the UI renders
cleanly without errors before any deep scan runs.
- Old installs upgrading will see "Run a Deep Scan to populate
(N tracks pending)". Running their next deep scan fills sizes —
the existing scan flow doesn't need any changes, just consumes
the new track-wrapper attribute.
Tests:
- `tests/test_library_disk_usage.py` — 13 cases covering schema
migration, NULL defaults on legacy inserts, fresh-install empty
shape, summing with mixed NULL/known sizes, per-format breakdown,
mixed-case extensions, paths with album-name dots, missing
extensions, empty file_path, implausibly long extensions,
JellyfinTrack.file_size persistence via insert_or_update_media_track,
COALESCE preservation on null re-sync.
- `tests/imports/test_import_side_effects.py` — extended the
existing record_soulsync_library_entry test to assert
track_row['file_size'] == os.path.getsize(final_path), pinning
the SoulSync-standalone path. Test fixture's tracks schema also
updated to include the file_size column.
Verified: full suite 1813 pass (13 new, 1 existing-test extension),
ruff clean, smoke test populating + reading the column round-trips
correctly.
WHATS_NEW entry under '2.4.2' dev cycle.
PR #340 added ruff to the build-and-test.yml CI gate, which surfaced
286 pre-existing lint errors. Left unfixed, every feature branch push
fails CI. This commit resolves all of them so CI goes green and
contributors can actually land work.
Auto-fixes (248 of 286): removed unused f-string prefixes (F541),
renamed unused loop control variables with underscore prefix (B007),
removed duplicate imports (F811).
Manually fixed 10 latent bugs ruff caught (all wrapped in try/except
today, silently failing):
- music_database.py: _add_discovery_tables() called undefined
conn.commit() — would have crashed the iTunes-support migration
for existing databases. Now uses cursor.connection.commit().
- web_server.py settings GET: referenced undefined download_orchestrator
when it should be soulseek_client. Feature (_source_status on the
settings payload) was silently missing for UI auto-disable logic.
- web_server.py _process_wishlist_automatically: active_server
undefined in track-ownership check. Auto-wishlist was falling
through to the error handler and re-downloading owned tracks.
- web_server.py start_wishlist_missing_downloads: same active_server
bug in the manual wishlist path.
- web_server.py _process_failed_tracks_to_wishlist_exact: emitted
wishlist_item_added automation event with undefined artist_name
and track. Automation event silently never fired correctly.
- web_server.py discovery metadata enrichment: referenced cache
without calling get_metadata_cache() first. Track enrichment from
cached API responses was silently skipped.
- web_server.py Beatport discovery worker: wing-it fallback branch
used undefined successful_discoveries variable. Wing-it counter
never incremented correctly. Now uses state['spotify_matches']
consistently with the rest of the function.
- web_server.py _run_full_missing_tracks_process: stale import json
mid-function shadowed the module-level import, making an earlier
json.dumps() call reference an unbound local (F823).
- web_server.py discovery loop: platform loop variable shadowed
the module-level platform import (F402).
- core/watchlist_scanner.py: 7 lambda captures of loop variables
(B023 classic Python closure-in-loop bug) now bind at creation.
No existing tests had to change. Full suite stays at 263 passed.
- collapse old multi-line debug bursts into single structured rows
- remove leftover DEBUG-style prefixes from message text
- keep the app log readable without losing useful trace detail
get_playlist_tracks() had no Limit or StartIndex params, so Jellyfin
defaulted to ~100 items per response. This caused the Server Playlists
comparison to show most tracks as missing even though they were present.
Now paginates in batches of 1000 until a partial batch signals the last page.
Stripped 4,200+ emoji characters from print(), logger calls across
39 Python files. Logs are now clean text — easier to grep, more
professional, no encoding issues on terminals without Unicode support.
Seasonal config icons preserved for UI display.
After a successful playlist sync, if the source playlist has cover
art (Spotify, Tidal, Deezer, etc.), the image is downloaded and
uploaded as the playlist poster on the media server. Plex uses
uploadPoster(), Jellyfin/Emby uses POST /Items/{id}/Images/Primary.
Navidrome skipped (no playlist image API). Failure is silent — sync
result unchanged. Automation-triggered syncs and playlists without
images are unaffected.
The _is_valid_guid method only accepted 32-char hex GUIDs (Jellyfin
format) but Emby uses plain integer IDs like "12345". All matched
tracks were rejected as "invalid/empty IDs" causing playlist creation
to fail with zero tracks. Now accepts both numeric strings (Emby)
and hex GUIDs (Jellyfin).
Full stats dashboard that polls Plex/Jellyfin/Navidrome for play
history and presents it with Chart.js visualizations:
Backend:
- ListeningStatsWorker polls active server every 30 min
- listening_history DB table with dedup, play_count/last_played on tracks
- get_play_history() and get_track_play_counts() for all 3 servers
- Pre-computed cache for all time ranges (7d/30d/12m/all) rebuilt each sync
- Single cached endpoint serves all stats data instantly
- Stats query methods: top artists/albums/tracks, timeline, genres, health
Frontend:
- New Stats nav page with glassmorphic container matching dashboard style
- Overview cards (plays, time, artists, albums, tracks) with accent hover
- Listening timeline bar chart (Chart.js)
- Genre breakdown doughnut chart with legend
- Top artists visual bubbles with profile pictures + ranked list
- Top albums and tracks ranked lists with album art
- Library health: format breakdown bar, unplayed count, enrichment coverage
- Recently played timeline with relative timestamps
- Time range pills with instant switching via cache
- Sync Now button with spinner, last synced timestamp
- Clickable artist names navigate to library artist detail
- Last.fm global listeners shown alongside personal play counts
- SoulID badges on matched artists
- Empty state when no data synced yet
- Mobile responsive layout
DB migrations: listening_history table, play_count/last_played columns,
all with idempotent CREATE IF NOT EXISTS / PRAGMA checks.
Incremental database updates now detect when artists or albums have been removed from your media server (Plex, Jellyfin, or Navidrome) and automatically clean them up from SoulSync's database. Previously, deleted content would persist as ghost entries until you ran a full refresh. Removal counts are reported in the scan results. Includes safety checks to prevent accidental mass deletion if the server is unreachable or returns incomplete data.
Raise the default Jellyfin API timeout for bulk operations from 30s to 120s to better handle slow servers and large database syncs. Updated core/jellyfin_client.py to set bulk_timeout to 120, webui/index.html to show 120s in the settings input, and webui/static/script.js to use 120 as the fallback when loading settings. Aligns UI and client behavior to reduce timeout errors during heavy syncs.
Add a configurable API timeout for Jellyfin bulk requests and improve fetch retry behavior. UI: adds an "API Timeout (seconds)" field (15–300s, default 30) in webui/index.html and persists it via webui/static/script.js (load/save). Client: jellyfin_client.py now reads api_timeout from config_manager, uses it as the bulk timeout and computes a sensible non-bulk timeout (max(5, bulk_timeout//6)). Fetch loops for tracks and albums were hardened: reducing batch size now resets consecutive failure counters, log messages were clarified, and the stopping/retry thresholds were adjusted to avoid premature aborts at minimum batch sizes.
Introduces backend and frontend functionality to list and select Jellyfin music libraries. Adds API endpoints, updates the client logic, and provides a UI selector for users to choose their preferred Jellyfin music library.
