Discord request (Samuel [KC]): show how much disk space the library
takes on the Stats page. Implementation piggybacks on the existing
deep scan — Plex/Jellyfin/Navidrome all return file size in their
track API responses, so we read it during the deep scan and store
it on the tracks row. Aggregation is then a single SQL query — no
filesystem walk, no extra I/O during the scan, no separate stat
job. SoulSync standalone gets size from os.path.getsize at insert
time (different code path; the file is local when we write the row).
Schema (`database/music_database.py`):
- New `file_size INTEGER` column on `tracks`. Migration uses the
established `try SELECT, except ALTER TABLE ADD COLUMN` pattern.
Idempotent; safe on existing installs. NULL on legacy rows so
they don't contribute to totals until next deep scan refreshes.
- Added the column to the canonical CREATE TABLE so fresh installs
get it without going through the migration path.
Track-object plumbing:
- `core/jellyfin_client.py` — JellyfinTrack reads MediaSources[0].Size
alongside existing Bitrate read. None when 0 / missing.
- `core/navidrome_client.py` — NavidromeTrack reads `size` from
the Subsonic song object (int coercion + None on parse fail).
- `core/soulsync_client.py` — SoulSyncTrack does os.path.getsize
(only "server" where size has to come from disk).
- Plex needs no client-side change: track.media[0].parts[0].size
is read directly inside insert_or_update_media_track.
Persistence — TWO separate insert paths:
(a) `database/music_database.py:insert_or_update_media_track` —
Plex/Jellyfin/Navidrome flows. Reads file_size from Plex's
MediaPart OR `track_obj.file_size` wrapper attribute (defensive
Plex-attr-not-present check + > 0 type guard).
INSERT writes the new column.
UPDATE uses COALESCE(?, file_size) so a None from the server
on a re-sync (rare Jellyfin Size omission) doesn't blank an
existing value. Pinned via test.
(b) `core/imports/side_effects.py:record_soulsync_library_entry` —
SoulSync standalone flow. Completely separate code path: the
standalone deep scan moves files to staging for auto-import
rather than calling insert_or_update_media_track. After the
auto-import processes them, side_effects writes the tracks row
directly. Reads file_size via os.path.getsize(final_path) at
insert time (file is local) and includes it in the INSERT
column list. SoulSync only does INSERT-if-not-exists (no
UPDATE path), so no COALESCE concern.
Aggregator (`database/music_database.py:get_library_disk_usage`):
- SELECT COALESCE(SUM(file_size), 0), COUNT(file_size),
COUNT(*) - COUNT(file_size) for the totals.
- Per-format breakdown done in Python via os.path.splitext over
(file_path, file_size) rows — sidesteps SQLite's first-vs-last-dot
ambiguity for paths like /music/Kendrick/M.A.A.D City/01.flac.
- Defensive: skips empty paths, paths without extension, and
implausibly long extensions (>6 chars). Returns the full
empty-shape dict (NOT a partial / undefined) when the column
doesn't exist or queries fail, so the UI's `if (!data.has_data)`
branch handles fresh installs cleanly.
API + UI:
- `core/stats/queries.py` — thin pass-through get_library_disk_usage
matching the existing query-helper convention.
- `web_server.py` — new /api/stats/library-disk-usage endpoint
mirroring the /api/stats/db-storage pattern.
- `webui/index.html` — new card in System Statistics above the
Database Storage card.
- `webui/static/stats-automations.js` — _loadLibraryDiskUsage +
_renderLibraryDiskUsage. Empty state: "Run a Deep Scan to
populate (X tracks pending)". Partial: "X measured (+Y pending)".
Full: total + format bars proportional to the largest format.
- `webui/static/style.css` — .stats-disk-* styled to match the
Database Storage card.
Backward compatibility:
- Migration is additive; existing rows get NULL file_size; the
empty-shape return from the aggregator means the UI renders
cleanly without errors before any deep scan runs.
- Old installs upgrading will see "Run a Deep Scan to populate
(N tracks pending)". Running their next deep scan fills sizes —
the existing scan flow doesn't need any changes, just consumes
the new track-wrapper attribute.
Tests:
- `tests/test_library_disk_usage.py` — 13 cases covering schema
migration, NULL defaults on legacy inserts, fresh-install empty
shape, summing with mixed NULL/known sizes, per-format breakdown,
mixed-case extensions, paths with album-name dots, missing
extensions, empty file_path, implausibly long extensions,
JellyfinTrack.file_size persistence via insert_or_update_media_track,
COALESCE preservation on null re-sync.
- `tests/imports/test_import_side_effects.py` — extended the
existing record_soulsync_library_entry test to assert
track_row['file_size'] == os.path.getsize(final_path), pinning
the SoulSync-standalone path. Test fixture's tracks schema also
updated to include the file_size column.
Verified: full suite 1813 pass (13 new, 1 existing-test extension),
ruff clean, smoke test populating + reading the column round-trips
correctly.
WHATS_NEW entry under '2.4.2' dev cycle.
206 lines
6.5 KiB
Python
206 lines
6.5 KiB
Python
import os
|
|
import sqlite3
|
|
from types import SimpleNamespace
|
|
|
|
from core.imports import side_effects
|
|
|
|
|
|
class _FakeDB:
|
|
def __init__(self, conn):
|
|
self._conn = conn
|
|
|
|
def _get_connection(self):
|
|
return self._conn
|
|
|
|
|
|
def _make_soulsync_db():
|
|
conn = sqlite3.connect(":memory:")
|
|
conn.row_factory = sqlite3.Row
|
|
conn.execute(
|
|
"""
|
|
CREATE TABLE artists (
|
|
id TEXT PRIMARY KEY,
|
|
name TEXT,
|
|
genres TEXT,
|
|
thumb_url TEXT,
|
|
server_source TEXT,
|
|
created_at TEXT,
|
|
updated_at TEXT,
|
|
spotify_artist_id TEXT
|
|
)
|
|
"""
|
|
)
|
|
conn.execute(
|
|
"""
|
|
CREATE TABLE albums (
|
|
id TEXT PRIMARY KEY,
|
|
artist_id TEXT,
|
|
title TEXT,
|
|
year INTEGER,
|
|
thumb_url TEXT,
|
|
genres TEXT,
|
|
track_count INTEGER,
|
|
duration INTEGER,
|
|
server_source TEXT,
|
|
created_at TEXT,
|
|
updated_at TEXT,
|
|
spotify_album_id TEXT
|
|
)
|
|
"""
|
|
)
|
|
conn.execute(
|
|
"""
|
|
CREATE TABLE tracks (
|
|
id TEXT PRIMARY KEY,
|
|
album_id TEXT,
|
|
artist_id TEXT,
|
|
title TEXT,
|
|
track_number INTEGER,
|
|
duration INTEGER,
|
|
file_path TEXT,
|
|
bitrate INTEGER,
|
|
file_size INTEGER,
|
|
track_artist TEXT,
|
|
server_source TEXT,
|
|
created_at TEXT,
|
|
updated_at TEXT,
|
|
spotify_track_id TEXT
|
|
)
|
|
"""
|
|
)
|
|
return conn
|
|
|
|
|
|
def test_record_soulsync_library_entry_writes_artist_album_and_track(tmp_path, monkeypatch):
|
|
conn = _make_soulsync_db()
|
|
fake_db = _FakeDB(conn)
|
|
final_path = tmp_path / "track.flac"
|
|
final_path.write_bytes(b"audio")
|
|
|
|
monkeypatch.setattr(side_effects, "get_database", lambda: fake_db)
|
|
monkeypatch.setattr(
|
|
side_effects,
|
|
"_get_config_manager",
|
|
lambda: SimpleNamespace(get_active_media_server=lambda: "soulsync"),
|
|
)
|
|
|
|
import core.genre_filter as genre_filter
|
|
|
|
monkeypatch.setattr(genre_filter, "filter_genres", lambda genres, _cfg: [genre.upper() for genre in genres])
|
|
|
|
context = {
|
|
"source": "spotify",
|
|
"artist": {"id": "sp-artist", "name": "Artist One"},
|
|
"album": {
|
|
"id": "sp-album",
|
|
"name": "Album One",
|
|
"release_date": "2024-02-03",
|
|
"total_tracks": 12,
|
|
"image_url": "https://img.example/album.jpg",
|
|
},
|
|
"track_info": {
|
|
"id": "sp-track",
|
|
"name": "Song One",
|
|
"track_number": 7,
|
|
"duration_ms": 210000,
|
|
"artists": [{"name": "Guest Artist"}],
|
|
"_source": "spotify",
|
|
},
|
|
"original_search_result": {
|
|
"title": "Song One",
|
|
"artists": [{"name": "Guest Artist"}],
|
|
"_source": "spotify",
|
|
},
|
|
"_final_processed_path": str(final_path),
|
|
}
|
|
|
|
artist_context = {"name": "Artist One", "genres": ["rock", "indie"]}
|
|
album_info = {"is_album": True, "album_name": "Album One", "track_number": 7}
|
|
|
|
side_effects.record_soulsync_library_entry(context, artist_context, album_info)
|
|
|
|
artist_row = conn.execute("SELECT * FROM artists").fetchone()
|
|
album_row = conn.execute("SELECT * FROM albums").fetchone()
|
|
track_row = conn.execute("SELECT * FROM tracks").fetchone()
|
|
|
|
assert artist_row["name"] == "Artist One"
|
|
assert artist_row["server_source"] == "soulsync"
|
|
assert artist_row["spotify_artist_id"] == "sp-artist"
|
|
assert artist_row["genres"] == '["ROCK", "INDIE"]'
|
|
|
|
assert album_row["title"] == "Album One"
|
|
assert album_row["server_source"] == "soulsync"
|
|
assert album_row["spotify_album_id"] == "sp-album"
|
|
assert album_row["year"] == 2024
|
|
assert album_row["track_count"] == 12
|
|
assert album_row["duration"] == 210000
|
|
assert album_row["artist_id"] == artist_row["id"]
|
|
|
|
assert track_row["title"] == "Song One"
|
|
assert track_row["server_source"] == "soulsync"
|
|
assert track_row["spotify_track_id"] == "sp-track"
|
|
assert track_row["track_number"] == 7
|
|
assert track_row["duration"] == 210000
|
|
assert track_row["track_artist"] == "Guest Artist"
|
|
assert track_row["album_id"] == album_row["id"]
|
|
assert track_row["file_path"] == str(final_path)
|
|
# File size in bytes — populates the Library Disk Usage card on Stats.
|
|
# Read via os.path.getsize at insert time since SoulSync standalone is
|
|
# the only flow where the file is local at the moment we write the row.
|
|
assert track_row["file_size"] == os.path.getsize(str(final_path))
|
|
|
|
|
|
def test_record_soulsync_library_entry_ignores_numeric_spotify_ids(tmp_path, monkeypatch):
|
|
conn = _make_soulsync_db()
|
|
fake_db = _FakeDB(conn)
|
|
final_path = tmp_path / "track.flac"
|
|
final_path.write_bytes(b"audio")
|
|
|
|
monkeypatch.setattr(side_effects, "get_database", lambda: fake_db)
|
|
monkeypatch.setattr(
|
|
side_effects,
|
|
"_get_config_manager",
|
|
lambda: SimpleNamespace(get_active_media_server=lambda: "soulsync"),
|
|
)
|
|
|
|
import core.genre_filter as genre_filter
|
|
|
|
monkeypatch.setattr(genre_filter, "filter_genres", lambda genres, _cfg: genres)
|
|
|
|
context = {
|
|
"source": "spotify",
|
|
"artist": {"id": "396753", "name": "Artist One"},
|
|
"album": {
|
|
"id": "284076172",
|
|
"name": "Album One",
|
|
"release_date": "2024-02-03",
|
|
"total_tracks": 12,
|
|
},
|
|
"track_info": {
|
|
"id": "1607091752",
|
|
"name": "Song One",
|
|
"track_number": 7,
|
|
"duration_ms": 210000,
|
|
"artists": [{"name": "Guest Artist"}],
|
|
"_source": "spotify",
|
|
},
|
|
"original_search_result": {
|
|
"title": "Song One",
|
|
"artists": [{"name": "Guest Artist"}],
|
|
"_source": "spotify",
|
|
},
|
|
"_final_processed_path": str(final_path),
|
|
}
|
|
|
|
artist_context = {"name": "Artist One", "genres": ["rock"]}
|
|
album_info = {"is_album": True, "album_name": "Album One", "track_number": 7}
|
|
|
|
side_effects.record_soulsync_library_entry(context, artist_context, album_info)
|
|
|
|
artist_row = conn.execute("SELECT * FROM artists").fetchone()
|
|
album_row = conn.execute("SELECT * FROM albums").fetchone()
|
|
track_row = conn.execute("SELECT * FROM tracks").fetchone()
|
|
|
|
assert artist_row["spotify_artist_id"] is None
|
|
assert album_row["spotify_album_id"] is None
|
|
assert track_row["spotify_track_id"] is None
|