soulsync/tests
Broque Thomas 2ab460f5c4 Add Library Disk Usage card to System Statistics
Discord request (Samuel [KC]): show how much disk space the library
takes on the Stats page. Implementation piggybacks on the existing
deep scan — Plex/Jellyfin/Navidrome all return file size in their
track API responses, so we read it during the deep scan and store
it on the tracks row. Aggregation is then a single SQL query — no
filesystem walk, no extra I/O during the scan, no separate stat
job. SoulSync standalone gets size from os.path.getsize at insert
time (different code path; the file is local when we write the row).

Schema (`database/music_database.py`):
- New `file_size INTEGER` column on `tracks`. Migration uses the
  established `try SELECT, except ALTER TABLE ADD COLUMN` pattern.
  Idempotent; safe on existing installs. NULL on legacy rows so
  they don't contribute to totals until next deep scan refreshes.
- Added the column to the canonical CREATE TABLE so fresh installs
  get it without going through the migration path.

Track-object plumbing:
- `core/jellyfin_client.py` — JellyfinTrack reads MediaSources[0].Size
  alongside existing Bitrate read. None when 0 / missing.
- `core/navidrome_client.py` — NavidromeTrack reads `size` from
  the Subsonic song object (int coercion + None on parse fail).
- `core/soulsync_client.py` — SoulSyncTrack does os.path.getsize
  (only "server" where size has to come from disk).
- Plex needs no client-side change: track.media[0].parts[0].size
  is read directly inside insert_or_update_media_track.

Persistence — TWO separate insert paths:

(a) `database/music_database.py:insert_or_update_media_track` —
    Plex/Jellyfin/Navidrome flows. Reads file_size from Plex's
    MediaPart OR `track_obj.file_size` wrapper attribute (defensive
    Plex-attr-not-present check + > 0 type guard).
    INSERT writes the new column.
    UPDATE uses COALESCE(?, file_size) so a None from the server
    on a re-sync (rare Jellyfin Size omission) doesn't blank an
    existing value. Pinned via test.

(b) `core/imports/side_effects.py:record_soulsync_library_entry` —
    SoulSync standalone flow. Completely separate code path: the
    standalone deep scan moves files to staging for auto-import
    rather than calling insert_or_update_media_track. After the
    auto-import processes them, side_effects writes the tracks row
    directly. Reads file_size via os.path.getsize(final_path) at
    insert time (file is local) and includes it in the INSERT
    column list. SoulSync only does INSERT-if-not-exists (no
    UPDATE path), so no COALESCE concern.

Aggregator (`database/music_database.py:get_library_disk_usage`):
- SELECT COALESCE(SUM(file_size), 0), COUNT(file_size),
  COUNT(*) - COUNT(file_size) for the totals.
- Per-format breakdown done in Python via os.path.splitext over
  (file_path, file_size) rows — sidesteps SQLite's first-vs-last-dot
  ambiguity for paths like /music/Kendrick/M.A.A.D City/01.flac.
- Defensive: skips empty paths, paths without extension, and
  implausibly long extensions (>6 chars). Returns the full
  empty-shape dict (NOT a partial / undefined) when the column
  doesn't exist or queries fail, so the UI's `if (!data.has_data)`
  branch handles fresh installs cleanly.

API + UI:
- `core/stats/queries.py` — thin pass-through get_library_disk_usage
  matching the existing query-helper convention.
- `web_server.py` — new /api/stats/library-disk-usage endpoint
  mirroring the /api/stats/db-storage pattern.
- `webui/index.html` — new card in System Statistics above the
  Database Storage card.
- `webui/static/stats-automations.js` — _loadLibraryDiskUsage +
  _renderLibraryDiskUsage. Empty state: "Run a Deep Scan to
  populate (X tracks pending)". Partial: "X measured (+Y pending)".
  Full: total + format bars proportional to the largest format.
- `webui/static/style.css` — .stats-disk-* styled to match the
  Database Storage card.

Backward compatibility:
- Migration is additive; existing rows get NULL file_size; the
  empty-shape return from the aggregator means the UI renders
  cleanly without errors before any deep scan runs.
- Old installs upgrading will see "Run a Deep Scan to populate
  (N tracks pending)". Running their next deep scan fills sizes —
  the existing scan flow doesn't need any changes, just consumes
  the new track-wrapper attribute.

Tests:
- `tests/test_library_disk_usage.py` — 13 cases covering schema
  migration, NULL defaults on legacy inserts, fresh-install empty
  shape, summing with mixed NULL/known sizes, per-format breakdown,
  mixed-case extensions, paths with album-name dots, missing
  extensions, empty file_path, implausibly long extensions,
  JellyfinTrack.file_size persistence via insert_or_update_media_track,
  COALESCE preservation on null re-sync.
- `tests/imports/test_import_side_effects.py` — extended the
  existing record_soulsync_library_entry test to assert
  track_row['file_size'] == os.path.getsize(final_path), pinning
  the SoulSync-standalone path. Test fixture's tracks schema also
  updated to include the file_size column.

Verified: full suite 1813 pass (13 new, 1 existing-test extension),
ruff clean, smoke test populating + reading the column round-trips
correctly.

WHATS_NEW entry under '2.4.2' dev cycle.
2026-05-03 20:17:06 -07:00
..
artists Lift enhance_artist_quality to core/artists/quality.py 2026-04-29 10:05:57 -07:00
automation Surface handler-returned errors in automation last_error 2026-04-30 15:45:28 -07:00
discovery Preserve artwork in quality scanner wishlist handoff 2026-04-30 21:42:16 +03:00
downloads Wire SoundCloud as a first-class download source 2026-05-03 12:54:21 -07:00
imports Add Library Disk Usage card to System Statistics 2026-05-03 20:17:06 -07:00
library Fix Album Completeness Auto-Fill on Docker / shared-library setups (#476) 2026-05-03 10:11:06 -07:00
metadata Fix album MBID inconsistency: detector + persistent release-MBID cache 2026-05-03 17:16:39 -07:00
playlists Lift _prepare_stream_task + playlist_explorer_build_tree to core/ 2026-04-29 11:26:07 -07:00
search Search lift: pre-merge parity polish for cin's review 2026-04-27 15:32:48 -07:00
stats Lift /api/stats/* and /api/listening-stats/* into core/stats/ 2026-04-27 14:27:03 -07:00
streaming Mock streaming prep sleep in tests 2026-04-30 21:26:28 +03:00
watchlist Lift _process_watchlist_scan_automatically to core/watchlist/auto_scan.py 2026-04-29 08:17:41 -07:00
wishlist Coerce wishlist track counts before category checks 2026-04-30 21:42:43 +03:00
conftest.py Split metadata source and Spotify status 2026-05-02 22:02:00 +03:00
test_acoustid_scanner.py Fix null-pointer error in acoustid_scanner 2026-04-17 19:17:24 +03:00
test_acoustid_skip_logic.py Fix: tasks showed Completed when file was quarantined 2026-05-03 18:28:32 -07:00
test_album_completeness_job.py Fix DummyConfigManager position in album completeness job test 2026-04-27 19:55:06 +03:00
test_album_mbid_consistency.py Fix album MBID inconsistency: detector + persistent release-MBID cache 2026-05-03 17:16:39 -07:00
test_auth_usage_throttle.py test: auth last_used_at write throttle 2026-04-19 15:22:24 -07:00
test_config_save_retry.py Stop config retry tests from writing to the real DB 2026-04-30 14:07:00 -07:00
test_content_filter_regex.py Fix watchlist content filters: live false positives + auto-scan bypass 2026-04-21 19:16:25 -07:00
test_dashboard_events.py Realign test file names 2026-04-17 21:25:01 +03:00
test_discogs_track_count.py Discogs: count rows with empty type_ as real tracks too 2026-04-25 08:27:44 -07:00
test_download_orchestrator_soundcloud.py Wire SoundCloud as a first-class download source 2026-05-03 12:54:21 -07:00
test_downloads_pagination.py Lift shared runtime state into core 2026-04-27 19:54:44 +03:00
test_duplicate_detector_mount_paths.py Filter same-physical-file duplicates from duplicate detector 2026-04-30 14:06:10 -07:00
test_duplicate_detector_slskd_dedup.py Add filename-pass safety: require duration agreement or artist match 2026-05-01 13:15:57 -07:00
test_enrichment_services.py Consolidate enrichment bubble routes behind a service registry 2026-05-01 20:14:30 -07:00
test_featured_artist_completion.py Match featured-artist tracks across discography completion 2026-04-30 19:31:11 -07:00
test_hifi_instance_methods.py validate hifi instance reorder against pre-existing instances 2026-04-28 20:19:01 -06:00
test_hls_parsing.py fix: pick OS-specific ffmpeg binary in hls demux fallback test 2026-04-28 21:13:54 -07:00
test_import_singles_parallel.py Restore soulsync logger state between parallel-imports tests 2026-05-01 08:45:17 -07:00
test_integrity_failure_marks_task_failed.py Fix: tasks showed Completed when file was quarantined 2026-05-03 18:28:32 -07:00
test_library_disk_usage.py Add Library Disk Usage card to System Statistics 2026-05-03 20:17:06 -07:00
test_library_reorganize.py Fix CI test failures from incomplete dummy config and encoding 2026-04-19 13:04:55 -07:00
test_library_reorganize_orchestrator.py Reorganize: fix moved-count + status/total UX issues from PR #377 review 2026-04-25 09:07:44 -07:00
test_library_track_identity.py Make extract_external_ids recognize all source-tagging conventions 2026-05-02 18:26:12 -07:00
test_lidarr_download_client.py Fix three Lidarr bugs that prevented it from being a real download source 2026-05-03 15:49:38 -07:00
test_listening_stats_batch_queries.py test: listening stats worker batched query paths 2026-04-19 15:22:25 -07:00
test_missing_cover_art.py Fix CI test failures from incomplete dummy config and encoding 2026-04-19 13:04:55 -07:00
test_provenance_id_persistence.py Persist source IDs at download time + backfill onto tracks on sync 2026-05-02 17:44:10 -07:00
test_qobuz_credential_sync.py Sync Qobuz auth to enrichment worker after login 2026-05-02 14:00:04 -07:00
test_reorganize_db_methods.py Reorganize queue: race + dedupe fixes from kettui review 2026-04-26 08:40:24 -07:00
test_reorganize_queue.py Reorganize queue: race + dedupe fixes from kettui review 2026-04-26 08:40:24 -07:00
test_reorganize_runner.py Library reorganize: FIFO queue with live status panel 2026-04-25 18:01:32 -07:00
test_repair_worker_album_fill.py Fix CI test failures from incomplete dummy config and encoding 2026-04-19 13:04:55 -07:00
test_replaygain_summary_parse.py Fix: ReplayGain wrote same +52 dB gain to every track 2026-05-03 19:08:35 -07:00
test_request_cleanup_timer.py test: api/request periodic cleanup timer 2026-04-19 15:22:25 -07:00
test_runtime_state.py Tighten metadata and import safety 2026-04-27 20:28:05 +03:00
test_script_split_integrity.py Unify artist detail: route source artists to standalone page, retire inline Artists page 2026-04-22 17:00:52 -07:00
test_socketio_cors.py Socket.IO CORS: handle self-review nits 2026-04-26 19:24:43 -07:00
test_soundcloud_client.py Fix three SoundCloud integration gaps surfaced by smoke testing 2026-05-03 13:09:02 -07:00
test_spa_deep_linking.py Rename Search page id from 'downloads' to 'search', bump to 2.43 2026-04-22 13:22:49 -07:00
test_sync_discovery_scan_events.py Realign test file names 2026-04-17 21:25:01 +03:00
test_tidal_auth_instructions.py Show Tidal callback port (not Spotify's) in auth instructions 2026-04-30 20:54:06 -07:00
test_tidal_auth_redirect_uri.py Honor configured Tidal redirect_uri, drop request-host fallback 2026-05-02 19:20:18 -07:00
test_tidal_search_shortening.py Reject Tidal streams that silently downgrade from the requested quality 2026-04-24 13:12:30 -07:00
test_tool_progress_events.py Realign test file names 2026-04-17 21:25:01 +03:00
test_track_artist_matching.py Add fallback negative-case test for track-artist matching 2026-04-30 17:13:15 -07:00
test_track_number_repair.py Fix CI test failures from incomplete dummy config and encoding 2026-04-19 13:04:55 -07:00
test_track_search_single_query.py test: api_search_tracks single-query track search 2026-04-19 15:22:24 -07:00
test_unknown_artist_fixer.py Fix test using old deezer_track_id column name 2026-04-20 21:11:45 -07:00
test_watchlist_album_match.py Stop watchlist re-downloading compilation tracks; catch slskd dedup orphans 2026-05-01 12:57:50 -07:00
test_watchlist_bulk_add.py Bulk watchlist add: fall back through every source ID, not just active 2026-04-30 20:27:42 -07:00
test_watchlist_scanner_scan.py Move the import flows and related post-processing pipelines into separate modules 2026-04-27 19:54:43 +03:00
test_websocket_client_transport.py Repoint websocket transport test to core.js after split 2026-04-22 11:30:39 -07:00
test_websocket_infrastructure.py Split metadata source and Spotify status 2026-05-02 22:02:00 +03:00
test_wishlist_pagination.py test: wishlist SQL pagination and category filter 2026-04-19 15:22:24 -07:00
test_worker_existing_id_marks_matched.py test: enrichment worker re-processing fix and migration backfill 2026-04-19 15:22:24 -07:00
test_worker_utils_album_track_count.py Fix Album Completeness job reporting zero findings for every album 2026-04-24 12:39:41 -07:00