Add Library Disk Usage card to System Statistics
Discord request (Samuel [KC]): show how much disk space the library
takes on the Stats page. Implementation piggybacks on the existing
deep scan — Plex/Jellyfin/Navidrome all return file size in their
track API responses, so we read it during the deep scan and store
it on the tracks row. Aggregation is then a single SQL query — no
filesystem walk, no extra I/O during the scan, no separate stat
job. SoulSync standalone gets size from os.path.getsize at insert
time (different code path; the file is local when we write the row).
Schema (`database/music_database.py`):
- New `file_size INTEGER` column on `tracks`. Migration uses the
established `try SELECT, except ALTER TABLE ADD COLUMN` pattern.
Idempotent; safe on existing installs. NULL on legacy rows so
they don't contribute to totals until next deep scan refreshes.
- Added the column to the canonical CREATE TABLE so fresh installs
get it without going through the migration path.
Track-object plumbing:
- `core/jellyfin_client.py` — JellyfinTrack reads MediaSources[0].Size
alongside existing Bitrate read. None when 0 / missing.
- `core/navidrome_client.py` — NavidromeTrack reads `size` from
the Subsonic song object (int coercion + None on parse fail).
- `core/soulsync_client.py` — SoulSyncTrack does os.path.getsize
(only "server" where size has to come from disk).
- Plex needs no client-side change: track.media[0].parts[0].size
is read directly inside insert_or_update_media_track.
Persistence — TWO separate insert paths:
(a) `database/music_database.py:insert_or_update_media_track` —
Plex/Jellyfin/Navidrome flows. Reads file_size from Plex's
MediaPart OR `track_obj.file_size` wrapper attribute (defensive
Plex-attr-not-present check + > 0 type guard).
INSERT writes the new column.
UPDATE uses COALESCE(?, file_size) so a None from the server
on a re-sync (rare Jellyfin Size omission) doesn't blank an
existing value. Pinned via test.
(b) `core/imports/side_effects.py:record_soulsync_library_entry` —
SoulSync standalone flow. Completely separate code path: the
standalone deep scan moves files to staging for auto-import
rather than calling insert_or_update_media_track. After the
auto-import processes them, side_effects writes the tracks row
directly. Reads file_size via os.path.getsize(final_path) at
insert time (file is local) and includes it in the INSERT
column list. SoulSync only does INSERT-if-not-exists (no
UPDATE path), so no COALESCE concern.
Aggregator (`database/music_database.py:get_library_disk_usage`):
- SELECT COALESCE(SUM(file_size), 0), COUNT(file_size),
COUNT(*) - COUNT(file_size) for the totals.
- Per-format breakdown done in Python via os.path.splitext over
(file_path, file_size) rows — sidesteps SQLite's first-vs-last-dot
ambiguity for paths like /music/Kendrick/M.A.A.D City/01.flac.
- Defensive: skips empty paths, paths without extension, and
implausibly long extensions (>6 chars). Returns the full
empty-shape dict (NOT a partial / undefined) when the column
doesn't exist or queries fail, so the UI's `if (!data.has_data)`
branch handles fresh installs cleanly.
API + UI:
- `core/stats/queries.py` — thin pass-through get_library_disk_usage
matching the existing query-helper convention.
- `web_server.py` — new /api/stats/library-disk-usage endpoint
mirroring the /api/stats/db-storage pattern.
- `webui/index.html` — new card in System Statistics above the
Database Storage card.
- `webui/static/stats-automations.js` — _loadLibraryDiskUsage +
_renderLibraryDiskUsage. Empty state: "Run a Deep Scan to
populate (X tracks pending)". Partial: "X measured (+Y pending)".
Full: total + format bars proportional to the largest format.
- `webui/static/style.css` — .stats-disk-* styled to match the
Database Storage card.
Backward compatibility:
- Migration is additive; existing rows get NULL file_size; the
empty-shape return from the aggregator means the UI renders
cleanly without errors before any deep scan runs.
- Old installs upgrading will see "Run a Deep Scan to populate
(N tracks pending)". Running their next deep scan fills sizes —
the existing scan flow doesn't need any changes, just consumes
the new track-wrapper attribute.
Tests:
- `tests/test_library_disk_usage.py` — 13 cases covering schema
migration, NULL defaults on legacy inserts, fresh-install empty
shape, summing with mixed NULL/known sizes, per-format breakdown,
mixed-case extensions, paths with album-name dots, missing
extensions, empty file_path, implausibly long extensions,
JellyfinTrack.file_size persistence via insert_or_update_media_track,
COALESCE preservation on null re-sync.
- `tests/imports/test_import_side_effects.py` — extended the
existing record_soulsync_library_entry test to assert
track_row['file_size'] == os.path.getsize(final_path), pinning
the SoulSync-standalone path. Test fixture's tracks schema also
updated to include the file_size column.
Verified: full suite 1813 pass (13 new, 1 existing-test extension),
ruff clean, smoke test populating + reading the column round-trips
correctly.
WHATS_NEW entry under '2.4.2' dev cycle.
This commit is contained in:
parent
c66d307c0c
commit
2ab460f5c4
13 changed files with 620 additions and 9 deletions
|
|
@ -324,6 +324,17 @@ def record_soulsync_library_entry(context: Dict[str, Any], artist_context: Dict[
|
|||
except Exception:
|
||||
pass
|
||||
|
||||
# File size on disk (powers Library Disk Usage card on Stats).
|
||||
# SoulSync standalone is the only path where the file is local
|
||||
# at insert time, so we read it directly via os.path.getsize.
|
||||
# Mirrors what JellyfinTrack/NavidromeTrack pull from API
|
||||
# responses for the media-server flows.
|
||||
file_size = None
|
||||
try:
|
||||
file_size = os.path.getsize(final_path) or None
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
artist_id = _stable_soulsync_id(artist_name.lower().strip())
|
||||
album_id = _stable_soulsync_id(f"{artist_name}::{album_name}".lower().strip())
|
||||
track_id = _stable_soulsync_id(final_path)
|
||||
|
|
@ -410,9 +421,9 @@ def record_soulsync_library_entry(context: Dict[str, Any], artist_context: Dict[
|
|||
cursor.execute(
|
||||
"""
|
||||
INSERT INTO tracks (id, album_id, artist_id, title, track_number,
|
||||
duration, file_path, bitrate, track_artist, server_source,
|
||||
duration, file_path, bitrate, file_size, track_artist, server_source,
|
||||
created_at, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, 'soulsync', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 'soulsync', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)
|
||||
""",
|
||||
(
|
||||
track_id,
|
||||
|
|
@ -423,6 +434,7 @@ def record_soulsync_library_entry(context: Dict[str, Any], artist_context: Dict[
|
|||
duration_ms,
|
||||
final_path,
|
||||
bitrate,
|
||||
file_size,
|
||||
track_artist,
|
||||
),
|
||||
)
|
||||
|
|
|
|||
|
|
@ -116,9 +116,15 @@ class JellyfinTrack:
|
|||
|
||||
# File path and media info (used by quality scanner and DB update)
|
||||
self.path = jellyfin_data.get('Path')
|
||||
# Extract bitrate from MediaSources if available
|
||||
# Extract bitrate + file size from MediaSources if available.
|
||||
# `file_size` powers the Library Disk Usage card on the Stats
|
||||
# page — populated free during the deep scan from data Jellyfin
|
||||
# already returns in MediaSources[].
|
||||
media_sources = jellyfin_data.get('MediaSources', [])
|
||||
self.bitRate = (media_sources[0].get('Bitrate') or 0) // 1000 if media_sources else None # Convert bps to kbps
|
||||
self.file_size = (media_sources[0].get('Size') or 0) if media_sources else None
|
||||
if self.file_size == 0:
|
||||
self.file_size = None
|
||||
|
||||
def _parse_date(self, date_str: Optional[str]) -> Optional[datetime]:
|
||||
if not date_str:
|
||||
|
|
|
|||
|
|
@ -117,6 +117,14 @@ class NavidromeTrack:
|
|||
self.suffix = navidrome_data.get('suffix') # e.g. "flac", "mp3"
|
||||
self.bitRate = navidrome_data.get('bitRate') # e.g. 320
|
||||
self.path = navidrome_data.get('path') # e.g. "/music/Artist/Album/track.flac"
|
||||
# File size in bytes (Subsonic <song size="..."/>) — powers the
|
||||
# Library Disk Usage card on Stats. None when the server didn't
|
||||
# report a size (rare but possible for streaming-only nodes).
|
||||
_nv_size = navidrome_data.get('size')
|
||||
try:
|
||||
self.file_size = int(_nv_size) if _nv_size else None
|
||||
except (TypeError, ValueError):
|
||||
self.file_size = None
|
||||
|
||||
self._album_id = navidrome_data.get('albumId', '')
|
||||
self._artist_id = navidrome_data.get('artistId', '')
|
||||
|
|
|
|||
|
|
@ -98,6 +98,14 @@ class SoulSyncTrack:
|
|||
self.path = file_path
|
||||
self.bitRate = tags['bitrate']
|
||||
self.suffix = os.path.splitext(file_path)[1].lstrip('.').lower()
|
||||
# File size in bytes (powers Library Disk Usage card on Stats).
|
||||
# SoulSync standalone is the only "server" where we can read
|
||||
# size from disk directly — Plex/Jellyfin/Navidrome get theirs
|
||||
# from the API response.
|
||||
try:
|
||||
self.file_size = os.path.getsize(file_path) if os.path.exists(file_path) else None
|
||||
except OSError:
|
||||
self.file_size = None
|
||||
|
||||
def artist(self):
|
||||
return self._artist_ref
|
||||
|
|
|
|||
|
|
@ -172,6 +172,17 @@ def get_db_storage(database) -> dict:
|
|||
return database.get_db_storage_stats()
|
||||
|
||||
|
||||
def get_library_disk_usage(database) -> dict:
|
||||
"""On-disk size of the library, with per-format breakdown.
|
||||
|
||||
Backed by `tracks.file_size` populated during the deep scan from
|
||||
media-server-reported sizes (Plex MediaPart.size, Jellyfin
|
||||
MediaSources[].Size, Navidrome <song size="...">,
|
||||
SoulSync standalone os.path.getsize).
|
||||
"""
|
||||
return database.get_library_disk_usage()
|
||||
|
||||
|
||||
def get_recent_tracks(database, limit: int) -> list[dict]:
|
||||
"""Recently played tracks from listening_history."""
|
||||
conn = database._get_connection()
|
||||
|
|
|
|||
|
|
@ -294,6 +294,7 @@ class MusicDatabase:
|
|||
duration INTEGER, -- milliseconds
|
||||
file_path TEXT,
|
||||
bitrate INTEGER,
|
||||
file_size INTEGER, -- bytes; populated by deep scan from media-server API
|
||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
FOREIGN KEY (album_id) REFERENCES albums (id) ON DELETE CASCADE,
|
||||
|
|
@ -672,6 +673,23 @@ class MusicDatabase:
|
|||
except Exception:
|
||||
pass
|
||||
|
||||
# Migration: add file_size column so the Stats page can show
|
||||
# total library size on disk without having to walk the
|
||||
# filesystem on every request. Populated by the deep scan from
|
||||
# whatever the media server reports (Plex MediaPart.size,
|
||||
# Jellyfin MediaSources[].Size, Navidrome <song size="...">,
|
||||
# SoulSync standalone os.path.getsize). NULL on existing rows
|
||||
# until the next deep scan fills them in — UI handles the
|
||||
# NULL case by showing "(run a Deep Scan to populate)".
|
||||
try:
|
||||
cursor.execute("SELECT file_size FROM tracks LIMIT 1")
|
||||
except Exception:
|
||||
try:
|
||||
cursor.execute("ALTER TABLE tracks ADD COLUMN file_size INTEGER")
|
||||
logger.info("Added file_size column to tracks table")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# One-time migration: purge discovery cache entries that lack track_number.
|
||||
# Prior versions cached discovery results without track_number/disc_number/release_date,
|
||||
# causing incorrect file organization (all tracks as "01", missing album year).
|
||||
|
|
@ -3648,6 +3666,85 @@ class MusicDatabase:
|
|||
if conn:
|
||||
conn.close()
|
||||
|
||||
def get_library_disk_usage(self):
|
||||
"""Aggregate disk usage of the on-disk music library.
|
||||
|
||||
Returns:
|
||||
{
|
||||
'total_bytes': int, # sum of all known file sizes
|
||||
'tracks_with_size': int, # count of tracks with a known size
|
||||
'tracks_without_size': int, # count of tracks where size is NULL
|
||||
'by_format': { # bytes per file extension
|
||||
'flac': int, 'mp3': int, ...
|
||||
},
|
||||
'has_data': bool, # False on fresh installs / before first deep scan
|
||||
}
|
||||
|
||||
Returns the empty-shape dict when the column doesn't exist (very
|
||||
old install pre-migration) — UI shows "(run a Deep Scan)" in
|
||||
that case rather than crashing.
|
||||
"""
|
||||
empty = {
|
||||
'total_bytes': 0,
|
||||
'tracks_with_size': 0,
|
||||
'tracks_without_size': 0,
|
||||
'by_format': {},
|
||||
'has_data': False,
|
||||
}
|
||||
conn = None
|
||||
try:
|
||||
conn = self._get_connection()
|
||||
cursor = conn.cursor()
|
||||
|
||||
# Confirm column exists (defensive against fresh-install race
|
||||
# where the migration hasn't run yet).
|
||||
try:
|
||||
cursor.execute("SELECT file_size FROM tracks LIMIT 1")
|
||||
except Exception:
|
||||
return empty
|
||||
|
||||
cursor.execute(
|
||||
"SELECT COALESCE(SUM(file_size), 0), "
|
||||
" COUNT(file_size), "
|
||||
" COUNT(*) - COUNT(file_size) "
|
||||
"FROM tracks"
|
||||
)
|
||||
row = cursor.fetchone()
|
||||
total_bytes = int(row[0] or 0)
|
||||
tracks_with_size = int(row[1] or 0)
|
||||
tracks_without_size = int(row[2] or 0)
|
||||
|
||||
# Per-format breakdown via Python aggregation. Doing the
|
||||
# extension split in SQLite is fragile (paths with dots
|
||||
# before the file extension would group wrong); doing it
|
||||
# in Python is one os.path.splitext per row, which is
|
||||
# negligible cost compared to the SUM() above.
|
||||
cursor.execute(
|
||||
"SELECT file_path, file_size FROM tracks "
|
||||
"WHERE file_size IS NOT NULL AND file_path IS NOT NULL "
|
||||
" AND file_path != ''"
|
||||
)
|
||||
by_format: dict = {}
|
||||
for path, size in cursor.fetchall():
|
||||
ext = os.path.splitext(path)[1].lstrip('.').lower()
|
||||
if not ext or len(ext) > 6:
|
||||
continue
|
||||
by_format[ext] = by_format.get(ext, 0) + int(size or 0)
|
||||
|
||||
return {
|
||||
'total_bytes': total_bytes,
|
||||
'tracks_with_size': tracks_with_size,
|
||||
'tracks_without_size': tracks_without_size,
|
||||
'by_format': by_format,
|
||||
'has_data': tracks_with_size > 0,
|
||||
}
|
||||
except Exception as e:
|
||||
logger.error(f"Error getting library disk usage: {e}")
|
||||
return empty
|
||||
finally:
|
||||
if conn:
|
||||
conn.close()
|
||||
|
||||
@staticmethod
|
||||
def _listening_time_filter(time_range, alias=''):
|
||||
"""Build a WHERE clause for time-range filtering."""
|
||||
|
|
@ -4970,12 +5067,20 @@ class MusicDatabase:
|
|||
# Get file path and media info (Plex-specific, Jellyfin may not have these)
|
||||
file_path = None
|
||||
bitrate = None
|
||||
file_size = None
|
||||
if hasattr(track_obj, 'media') and track_obj.media:
|
||||
media = track_obj.media[0] if track_obj.media else None
|
||||
if media:
|
||||
if hasattr(media, 'parts') and media.parts:
|
||||
part = media.parts[0]
|
||||
file_path = getattr(part, 'file', None)
|
||||
# Plex's MediaPart exposes the file size in bytes
|
||||
# via plexapi — pull it for the Library Disk
|
||||
# Usage card on Stats. None when the server
|
||||
# didn't report a size.
|
||||
_plex_size = getattr(part, 'size', None)
|
||||
if isinstance(_plex_size, int) and _plex_size > 0:
|
||||
file_size = _plex_size
|
||||
bitrate = getattr(media, 'bitrate', None)
|
||||
|
||||
# Fallback for Navidrome/Subsonic tracks
|
||||
|
|
@ -4985,6 +5090,14 @@ class MusicDatabase:
|
|||
bitrate = track_obj.bitRate
|
||||
if file_path is None and hasattr(track_obj, 'suffix') and track_obj.suffix:
|
||||
file_path = f"{track_obj.title}.{track_obj.suffix}"
|
||||
# File size: Jellyfin / Navidrome / SoulSync-standalone
|
||||
# all set track_obj.file_size on their wrapper class.
|
||||
# Plex came in via the media.parts[0].size path above —
|
||||
# don't clobber that.
|
||||
if file_size is None and hasattr(track_obj, 'file_size'):
|
||||
_wrapper_size = getattr(track_obj, 'file_size', None)
|
||||
if isinstance(_wrapper_size, int) and _wrapper_size > 0:
|
||||
file_size = _wrapper_size
|
||||
|
||||
# Extract per-track artist for compilations/DJ mixes.
|
||||
# Only stored when it differs from the album artist.
|
||||
|
|
@ -5040,21 +5153,26 @@ class MusicDatabase:
|
|||
if is_new_track:
|
||||
cursor.execute("""
|
||||
INSERT INTO tracks
|
||||
(id, album_id, artist_id, title, track_number, duration, file_path, bitrate, server_source, track_artist, musicbrainz_recording_id, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, CURRENT_TIMESTAMP)
|
||||
""", (track_id, album_id, artist_id, title, track_number, duration, file_path, bitrate, server_source, track_artist, mbid))
|
||||
(id, album_id, artist_id, title, track_number, duration, file_path, bitrate, file_size, server_source, track_artist, musicbrainz_recording_id, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, CURRENT_TIMESTAMP)
|
||||
""", (track_id, album_id, artist_id, title, track_number, duration, file_path, bitrate, file_size, server_source, track_artist, mbid))
|
||||
else:
|
||||
# Update server-provided fields only — preserves spotify_track_id, deezer_id,
|
||||
# isrc, bpm, and all other enrichment data
|
||||
# isrc, bpm, and all other enrichment data. file_size uses
|
||||
# COALESCE(?, file_size) so a NULL from the server (e.g.
|
||||
# Jellyfin sometimes omits Size on first sync) doesn't wipe
|
||||
# an existing value.
|
||||
cursor.execute("""
|
||||
UPDATE tracks
|
||||
SET album_id = ?, artist_id = ?, title = ?, track_number = ?,
|
||||
duration = ?, file_path = ?, bitrate = ?, server_source = ?,
|
||||
duration = ?, file_path = ?, bitrate = ?,
|
||||
file_size = COALESCE(?, file_size),
|
||||
server_source = ?,
|
||||
track_artist = COALESCE(?, track_artist),
|
||||
musicbrainz_recording_id = COALESCE(?, musicbrainz_recording_id),
|
||||
updated_at = CURRENT_TIMESTAMP
|
||||
WHERE id = ?
|
||||
""", (album_id, artist_id, title, track_number, duration, file_path, bitrate, server_source, track_artist, mbid, track_id))
|
||||
""", (album_id, artist_id, title, track_number, duration, file_path, bitrate, file_size, server_source, track_artist, mbid, track_id))
|
||||
|
||||
conn.commit()
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
import os
|
||||
import sqlite3
|
||||
from types import SimpleNamespace
|
||||
|
||||
|
|
@ -58,6 +59,7 @@ def _make_soulsync_db():
|
|||
duration INTEGER,
|
||||
file_path TEXT,
|
||||
bitrate INTEGER,
|
||||
file_size INTEGER,
|
||||
track_artist TEXT,
|
||||
server_source TEXT,
|
||||
created_at TEXT,
|
||||
|
|
@ -142,6 +144,10 @@ def test_record_soulsync_library_entry_writes_artist_album_and_track(tmp_path, m
|
|||
assert track_row["track_artist"] == "Guest Artist"
|
||||
assert track_row["album_id"] == album_row["id"]
|
||||
assert track_row["file_path"] == str(final_path)
|
||||
# File size in bytes — populates the Library Disk Usage card on Stats.
|
||||
# Read via os.path.getsize at insert time since SoulSync standalone is
|
||||
# the only flow where the file is local at the moment we write the row.
|
||||
assert track_row["file_size"] == os.path.getsize(str(final_path))
|
||||
|
||||
|
||||
def test_record_soulsync_library_entry_ignores_numeric_spotify_ids(tmp_path, monkeypatch):
|
||||
|
|
|
|||
282
tests/test_library_disk_usage.py
Normal file
282
tests/test_library_disk_usage.py
Normal file
|
|
@ -0,0 +1,282 @@
|
|||
"""Tests for the Library Disk Usage stat.
|
||||
|
||||
Discord request (Samuel [KC]): show how much disk space the library
|
||||
takes on the System Statistics page. Implementation piggybacks on the
|
||||
existing deep scan — Plex/Jellyfin/Navidrome all return file size in
|
||||
their track API responses, so we read it during the deep scan and
|
||||
aggregate via SQL on demand. No filesystem walk involved.
|
||||
|
||||
Tests pin:
|
||||
- Schema migration is idempotent and backward-compatible (existing
|
||||
rows get NULL file_size; new column doesn't break old inserts).
|
||||
- Aggregator returns the empty-shape dict for fresh installs and
|
||||
walks/sums correctly when populated.
|
||||
- Per-format breakdown handles mixed extensions correctly.
|
||||
- Defensive: empty / NULL / malformed paths don't crash.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sqlite3
|
||||
import tempfile
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from database.music_database import MusicDatabase
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def db(tmp_path: Path) -> MusicDatabase:
|
||||
"""Build a fresh isolated MusicDatabase against a temp file."""
|
||||
db_path = tmp_path / 'test_library_size.db'
|
||||
return MusicDatabase(database_path=str(db_path))
|
||||
|
||||
|
||||
def _insert_track(db: MusicDatabase, *, track_id: str, file_path: str,
|
||||
file_size, album_id: str = 'a1', artist_id: str = 'ar1') -> None:
|
||||
"""Helper: seed an artist+album+track row with the given size."""
|
||||
conn = db._get_connection()
|
||||
cur = conn.cursor()
|
||||
cur.execute("INSERT OR IGNORE INTO artists (id, name) VALUES (?, ?)",
|
||||
(artist_id, 'Test Artist'))
|
||||
cur.execute("INSERT OR IGNORE INTO albums (id, artist_id, title) VALUES (?, ?, ?)",
|
||||
(album_id, artist_id, 'Test Album'))
|
||||
cur.execute(
|
||||
"INSERT INTO tracks (id, album_id, artist_id, title, file_path, file_size) "
|
||||
"VALUES (?, ?, ?, ?, ?, ?)",
|
||||
(track_id, album_id, artist_id, f'track-{track_id}', file_path, file_size),
|
||||
)
|
||||
conn.commit()
|
||||
conn.close()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Schema migration
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_file_size_column_exists_after_init(db: MusicDatabase) -> None:
|
||||
"""Fresh install should have the column from the canonical
|
||||
CREATE TABLE."""
|
||||
conn = db._get_connection()
|
||||
cur = conn.cursor()
|
||||
cur.execute("PRAGMA table_info(tracks)")
|
||||
cols = {row[1] for row in cur.fetchall()}
|
||||
conn.close()
|
||||
assert 'file_size' in cols
|
||||
|
||||
|
||||
def test_existing_tracks_have_null_file_size_after_migration(db: MusicDatabase) -> None:
|
||||
"""Backward-compat: rows inserted via the OLD schema (no file_size)
|
||||
must still be readable, and querying file_size returns NULL — not
|
||||
an error. Simulated by inserting a track without specifying
|
||||
file_size (relies on column default = NULL)."""
|
||||
conn = db._get_connection()
|
||||
cur = conn.cursor()
|
||||
cur.execute("INSERT OR IGNORE INTO artists (id, name) VALUES ('ar1', 'A')")
|
||||
cur.execute("INSERT OR IGNORE INTO albums (id, artist_id, title) VALUES ('a1', 'ar1', 'Al')")
|
||||
# Note: NOT specifying file_size — should default to NULL
|
||||
cur.execute(
|
||||
"INSERT INTO tracks (id, album_id, artist_id, title, file_path) "
|
||||
"VALUES ('legacy_t', 'a1', 'ar1', 'L', '/x/legacy.flac')"
|
||||
)
|
||||
conn.commit()
|
||||
cur.execute("SELECT file_size FROM tracks WHERE id = 'legacy_t'")
|
||||
row = cur.fetchone()
|
||||
conn.close()
|
||||
# Could be sqlite3.Row or tuple; both index by 0
|
||||
assert row[0] is None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Aggregator
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_aggregator_returns_empty_shape_for_fresh_install(db: MusicDatabase) -> None:
|
||||
"""No tracks inserted → has_data=False, total=0, no formats."""
|
||||
result = db.get_library_disk_usage()
|
||||
assert result == {
|
||||
'total_bytes': 0,
|
||||
'tracks_with_size': 0,
|
||||
'tracks_without_size': 0,
|
||||
'by_format': {},
|
||||
'has_data': False,
|
||||
}
|
||||
|
||||
|
||||
def test_aggregator_sums_known_sizes(db: MusicDatabase) -> None:
|
||||
_insert_track(db, track_id='t1', file_path='/x/song1.flac', file_size=10_000_000)
|
||||
_insert_track(db, track_id='t2', file_path='/x/song2.flac', file_size=5_000_000)
|
||||
_insert_track(db, track_id='t3', file_path='/x/song3.mp3', file_size=3_000_000)
|
||||
|
||||
result = db.get_library_disk_usage()
|
||||
assert result['total_bytes'] == 18_000_000
|
||||
assert result['tracks_with_size'] == 3
|
||||
assert result['tracks_without_size'] == 0
|
||||
assert result['has_data'] is True
|
||||
|
||||
|
||||
def test_aggregator_excludes_null_sizes_from_sum(db: MusicDatabase) -> None:
|
||||
"""Tracks without size are counted but don't contribute to total_bytes."""
|
||||
_insert_track(db, track_id='t1', file_path='/x/sized.flac', file_size=10_000_000)
|
||||
_insert_track(db, track_id='t2', file_path='/x/null.flac', file_size=None)
|
||||
|
||||
result = db.get_library_disk_usage()
|
||||
assert result['total_bytes'] == 10_000_000
|
||||
assert result['tracks_with_size'] == 1
|
||||
assert result['tracks_without_size'] == 1
|
||||
# Has data — at least one track was measured
|
||||
assert result['has_data'] is True
|
||||
|
||||
|
||||
def test_aggregator_per_format_breakdown(db: MusicDatabase) -> None:
|
||||
_insert_track(db, track_id='t1', file_path='/x/song.flac', file_size=10_000_000)
|
||||
_insert_track(db, track_id='t2', file_path='/x/other.flac', file_size=5_000_000)
|
||||
_insert_track(db, track_id='t3', file_path='/x/song.mp3', file_size=3_000_000)
|
||||
_insert_track(db, track_id='t4', file_path='/x/song.m4a', file_size=2_000_000)
|
||||
|
||||
result = db.get_library_disk_usage()
|
||||
assert result['by_format'] == {
|
||||
'flac': 15_000_000,
|
||||
'mp3': 3_000_000,
|
||||
'm4a': 2_000_000,
|
||||
}
|
||||
|
||||
|
||||
def test_aggregator_handles_mixed_case_extensions(db: MusicDatabase) -> None:
|
||||
"""Extensions get lowercased so .FLAC and .flac group together."""
|
||||
_insert_track(db, track_id='t1', file_path='/x/song.FLAC', file_size=5_000_000)
|
||||
_insert_track(db, track_id='t2', file_path='/x/other.flac', file_size=5_000_000)
|
||||
|
||||
result = db.get_library_disk_usage()
|
||||
assert result['by_format'] == {'flac': 10_000_000}
|
||||
|
||||
|
||||
def test_aggregator_handles_paths_with_dots_in_album_name(db: MusicDatabase) -> None:
|
||||
"""Albums like 'M.A.A.D City' have dots in the path. Extension
|
||||
extraction must use the LAST dot, not the first."""
|
||||
_insert_track(
|
||||
db, track_id='t1',
|
||||
file_path='/music/Kendrick Lamar/M.A.A.D City/01 - track.flac',
|
||||
file_size=10_000_000,
|
||||
)
|
||||
result = db.get_library_disk_usage()
|
||||
assert result['by_format'] == {'flac': 10_000_000}
|
||||
|
||||
|
||||
def test_aggregator_skips_paths_without_extension(db: MusicDatabase) -> None:
|
||||
"""Defensive: files without an extension don't show up in
|
||||
by_format (would otherwise produce an empty-string key or junk)."""
|
||||
_insert_track(db, track_id='t1', file_path='/x/no_extension', file_size=5_000_000)
|
||||
_insert_track(db, track_id='t2', file_path='/x/song.flac', file_size=10_000_000)
|
||||
|
||||
result = db.get_library_disk_usage()
|
||||
assert result['total_bytes'] == 15_000_000
|
||||
assert result['by_format'] == {'flac': 10_000_000}
|
||||
assert '' not in result['by_format']
|
||||
|
||||
|
||||
def test_aggregator_skips_empty_file_path(db: MusicDatabase) -> None:
|
||||
"""Empty string file_path → shouldn't appear in by_format."""
|
||||
_insert_track(db, track_id='t1', file_path='', file_size=5_000_000)
|
||||
_insert_track(db, track_id='t2', file_path='/x/song.flac', file_size=10_000_000)
|
||||
|
||||
result = db.get_library_disk_usage()
|
||||
# Total still includes the empty-path track (it was measured)
|
||||
assert result['total_bytes'] == 15_000_000
|
||||
# But by_format only has the one with a real extension
|
||||
assert result['by_format'] == {'flac': 10_000_000}
|
||||
|
||||
|
||||
def test_aggregator_skips_implausibly_long_extension(db: MusicDatabase) -> None:
|
||||
"""Extensions over 6 chars are filtered (would be junk from an
|
||||
unusual filename like 'song.somethingweird')."""
|
||||
_insert_track(db, track_id='t1', file_path='/x/song.somethingweird', file_size=5_000_000)
|
||||
_insert_track(db, track_id='t2', file_path='/x/song.flac', file_size=10_000_000)
|
||||
|
||||
result = db.get_library_disk_usage()
|
||||
assert result['by_format'] == {'flac': 10_000_000}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Backward compatibility — schema column ordering / NULL writes
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_insert_or_update_media_track_persists_size_for_object_with_file_size(db: MusicDatabase) -> None:
|
||||
"""The Jellyfin/Navidrome/SoulSync track wrappers expose
|
||||
`track_obj.file_size`. Verify insert_or_update_media_track reads
|
||||
it and persists to the new column."""
|
||||
|
||||
class _FakeTrack:
|
||||
def __init__(self):
|
||||
self.ratingKey = 'fake_track_id_1'
|
||||
self.title = 'Test Track'
|
||||
self.trackNumber = 1
|
||||
self.duration = 200000
|
||||
self.path = '/library/Artist/Album/01 - track.flac'
|
||||
self.bitRate = 1411
|
||||
self.file_size = 42_000_000
|
||||
|
||||
# Seed parent rows so FK constraints are satisfied
|
||||
conn = db._get_connection()
|
||||
cur = conn.cursor()
|
||||
cur.execute("INSERT OR IGNORE INTO artists (id, name) VALUES ('ar2', 'Artist')")
|
||||
cur.execute("INSERT OR IGNORE INTO albums (id, artist_id, title) VALUES ('al2', 'ar2', 'Album')")
|
||||
conn.commit()
|
||||
conn.close()
|
||||
|
||||
db.insert_or_update_media_track(_FakeTrack(), album_id='al2', artist_id='ar2',
|
||||
server_source='jellyfin')
|
||||
|
||||
conn = db._get_connection()
|
||||
cur = conn.cursor()
|
||||
cur.execute("SELECT file_size FROM tracks WHERE id = 'fake_track_id_1'")
|
||||
row = cur.fetchone()
|
||||
conn.close()
|
||||
assert row[0] == 42_000_000
|
||||
|
||||
|
||||
def test_insert_or_update_media_track_preserves_size_on_null_re_sync(db: MusicDatabase) -> None:
|
||||
"""If a subsequent deep scan returns no file_size for a track that
|
||||
previously had one (e.g. server hiccup, rare Jellyfin response),
|
||||
the COALESCE on UPDATE preserves the existing value rather than
|
||||
blanking it. Pin the regression — losing data on every scan would
|
||||
be worse than the original problem."""
|
||||
|
||||
class _FakeTrack:
|
||||
def __init__(self, size):
|
||||
self.ratingKey = 'fake_track_id_2'
|
||||
self.title = 'Test'
|
||||
self.trackNumber = 1
|
||||
self.duration = 200000
|
||||
self.path = '/library/Artist/Album/02 - track.flac'
|
||||
self.bitRate = 1411
|
||||
self.file_size = size
|
||||
|
||||
conn = db._get_connection()
|
||||
cur = conn.cursor()
|
||||
cur.execute("INSERT OR IGNORE INTO artists (id, name) VALUES ('ar3', 'Artist')")
|
||||
cur.execute("INSERT OR IGNORE INTO albums (id, artist_id, title) VALUES ('al3', 'ar3', 'Album')")
|
||||
conn.commit()
|
||||
conn.close()
|
||||
|
||||
# First sync — server reports 30 MB
|
||||
db.insert_or_update_media_track(_FakeTrack(size=30_000_000), album_id='al3',
|
||||
artist_id='ar3', server_source='jellyfin')
|
||||
|
||||
# Second sync — server reports None (didn't include Size in MediaSources this time)
|
||||
db.insert_or_update_media_track(_FakeTrack(size=None), album_id='al3',
|
||||
artist_id='ar3', server_source='jellyfin')
|
||||
|
||||
conn = db._get_connection()
|
||||
cur = conn.cursor()
|
||||
cur.execute("SELECT file_size FROM tracks WHERE id = 'fake_track_id_2'")
|
||||
row = cur.fetchone()
|
||||
conn.close()
|
||||
# Original size preserved
|
||||
assert row[0] == 30_000_000
|
||||
|
|
@ -32845,6 +32845,21 @@ def stats_db_storage():
|
|||
except Exception as e:
|
||||
return jsonify({'success': False, 'error': str(e)}), 500
|
||||
|
||||
@app.route('/api/stats/library-disk-usage', methods=['GET'])
|
||||
def stats_library_disk_usage():
|
||||
"""Library on-disk size + per-format breakdown.
|
||||
|
||||
Reads `tracks.file_size` populated by the deep scan from data the
|
||||
media server already returns. Returns ``has_data: false`` on fresh
|
||||
installs that haven't run a deep scan since the migration — UI
|
||||
shows "Run a Deep Scan to populate" in that case.
|
||||
"""
|
||||
try:
|
||||
data = _stats_queries.get_library_disk_usage(get_database())
|
||||
return jsonify({'success': True, **data})
|
||||
except Exception as e:
|
||||
return jsonify({'success': False, 'error': str(e)}), 500
|
||||
|
||||
@app.route('/api/stats/recent', methods=['GET'])
|
||||
def stats_recent():
|
||||
"""Get recently played tracks."""
|
||||
|
|
|
|||
|
|
@ -6055,6 +6055,18 @@
|
|||
<div class="stats-enrichment" id="stats-enrichment-coverage"></div>
|
||||
</div>
|
||||
|
||||
<!-- Library Disk Usage -->
|
||||
<div class="stats-section-card stats-full-width">
|
||||
<div class="stats-section-title">Library Disk Usage</div>
|
||||
<div class="stats-disk-usage-wrap" id="stats-disk-usage-wrap">
|
||||
<div class="stats-disk-total-row">
|
||||
<div class="stats-disk-total-value" id="stats-disk-total-value">—</div>
|
||||
<div class="stats-disk-total-meta" id="stats-disk-total-meta">Run a Deep Scan to populate</div>
|
||||
</div>
|
||||
<div class="stats-disk-formats" id="stats-disk-formats"></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Database Storage -->
|
||||
<div class="stats-section-card stats-full-width">
|
||||
<div class="stats-section-title">Database Storage</div>
|
||||
|
|
|
|||
|
|
@ -3444,6 +3444,7 @@ const WHATS_NEW = {
|
|||
'2.4.2': [
|
||||
// --- post-2.4.1 dev work — entries hidden by _getLatestWhatsNewVersion until the build version bumps ---
|
||||
{ date: 'Unreleased — 2.4.2 dev cycle' },
|
||||
{ title: 'Library Disk Usage on Stats Page', desc: 'discord request (samuel [KC]): show how much disk space the library takes. new card on stats → system statistics shows total bytes + per-format breakdown (FLAC vs MP3 vs M4A bars). data comes from `tracks.file_size` populated during deep scan from whatever the media server already returns (plex MediaPart.size, jellyfin MediaSources[].Size, navidrome song.size, soulsync standalone os.path.getsize) — zero filesystem walk overhead. existing libraries see "Run a Deep Scan to populate" until the next deep scan fills in sizes; partial coverage shown as "X tracks measured (+Y pending)". migration is additive (NULL on legacy rows) so upgrading users have nothing to do.', page: 'stats' },
|
||||
{ title: 'Fix: ReplayGain Wrote Same +52 dB Gain to Every Track', desc: 'noticed every downloaded track came out with `replaygain_track_gain: +52.00 dB` regardless of actual loudness. cause: parser used `re.search` which returned the FIRST `I:` (integrated loudness) reading from ffmpeg\'s ebur128 output. that\'s the per-window measurement at t=0.5s — almost always ~-70 LUFS because tracks start with silence/encoder padding. -18 (RG2 reference) - (-70) = +52 dB on every track. fix: parser now anchors to the `Summary:` block at the end of ffmpeg\'s output and reads the actual integrated loudness from there, not the silent-intro partial. defensive fallback uses the LAST per-window reading if Summary is missing (still better than the first). gains now reflect real per-track loudness.', page: 'downloads' },
|
||||
{ title: 'Fix: Tracks Showed Completed When File Was Quarantined', desc: 'caught downloading kendrick mr morale: three tracks (rich interlude, savior interlude, savior) showed ✅ completed in the modal but were missing on disk. two layered bugs. (1) the post-process verification wrapper had a fallback that assumed success when no `_final_processed_path` was in context — but integrity-rejected files (which get quarantined instead of moved) leave that path unset, so the wrapper marked them complete. now wrapper explicitly checks `_integrity_failure_msg` and `_race_guard_failed` markers before the assume-success fallback. failed integrity = task marked failed, batch tracker notified with success=false. (2) acoustid skip-logic was too lenient — when fingerprint confidence was very high and either title OR artist matched a bit, it skipped verification with reason "likely same song in different language/script." that fired for english-vs-english by the same artist with the word "interlude" in both — same artist + 0.55 title sim = skip = wrong file accepted. tightened: skip now requires non-ASCII chars present (real language/script case) AND artist match, OR very high title similarity (≥0.80) AND artist match. english-vs-english with very different titles by same artist no longer skipped — verification correctly returns FAIL and the wrong file gets quarantined.', page: 'downloads' },
|
||||
{ title: 'Stop Navidrome From Splitting Albums Over Inconsistent MBIDs', desc: 'discord report (samuel [KC]): tracks of the same album sometimes carry different MUSICBRAINZ_ALBUMID tags, which causes navidrome to split the album into multiple entries. two-part fix: (1) the MBID Mismatch Detector now does a second scan that groups tracks by db album, finds the consensus (most-common) album mbid, and flags dissenters — fix action rewrites the dissenter\'s tag to match. catches existing inconsistencies in your library. (2) root cause: per-track musicbrainz release lookups went through an in-memory cache that\'s capped at 4096 entries and dies on server restart, so big libraries / restarts could resolve different release ids for tracks of the same album. added a persistent sqlite-backed cache so a release mbid resolved ONCE for an album applies to every future track of that album for the install\'s lifetime. strictly additive: any failure in the persistent layer falls through to the live musicbrainz lookup exactly as before.', page: 'library' },
|
||||
|
|
|
|||
|
|
@ -200,6 +200,8 @@ async function loadStatsData() {
|
|||
|
||||
// DB storage chart (separate fetch — not part of cached stats)
|
||||
_loadDbStorageChart();
|
||||
// Library disk usage (separate fetch — populated by deep scan)
|
||||
_loadLibraryDiskUsage();
|
||||
|
||||
// Recent plays
|
||||
_renderRecentPlays(data.recent || []);
|
||||
|
|
@ -387,6 +389,70 @@ async function _loadDbStorageChart() {
|
|||
}
|
||||
}
|
||||
|
||||
async function _loadLibraryDiskUsage() {
|
||||
try {
|
||||
const resp = await fetch('/api/stats/library-disk-usage');
|
||||
const data = await resp.json();
|
||||
if (!data.success) return;
|
||||
_renderLibraryDiskUsage(data);
|
||||
} catch (e) {
|
||||
console.debug('Library disk usage load failed:', e);
|
||||
}
|
||||
}
|
||||
|
||||
function _formatBytes(n) {
|
||||
if (!n || n <= 0) return '0 B';
|
||||
const units = ['B', 'KB', 'MB', 'GB', 'TB'];
|
||||
let i = 0;
|
||||
let v = n;
|
||||
while (v >= 1024 && i < units.length - 1) { v /= 1024; i++; }
|
||||
return `${v.toFixed(v < 10 ? 2 : 1)} ${units[i]}`;
|
||||
}
|
||||
|
||||
function _renderLibraryDiskUsage(data) {
|
||||
const totalEl = document.getElementById('stats-disk-total-value');
|
||||
const metaEl = document.getElementById('stats-disk-total-meta');
|
||||
const formatsEl = document.getElementById('stats-disk-formats');
|
||||
if (!totalEl || !metaEl || !formatsEl) return;
|
||||
|
||||
if (!data.has_data || !data.total_bytes) {
|
||||
totalEl.textContent = '—';
|
||||
metaEl.textContent = data.tracks_without_size > 0
|
||||
? `Run a Deep Scan to populate (${data.tracks_without_size.toLocaleString()} tracks pending)`
|
||||
: 'No tracks in library yet';
|
||||
formatsEl.innerHTML = '';
|
||||
return;
|
||||
}
|
||||
|
||||
totalEl.textContent = _formatBytes(data.total_bytes);
|
||||
|
||||
const withSize = data.tracks_with_size || 0;
|
||||
const withoutSize = data.tracks_without_size || 0;
|
||||
const trackBits = `${withSize.toLocaleString()} tracks measured`;
|
||||
const pendingBits = withoutSize > 0
|
||||
? ` (+${withoutSize.toLocaleString()} pending next Deep Scan)`
|
||||
: '';
|
||||
metaEl.textContent = trackBits + pendingBits;
|
||||
|
||||
// Per-format bars sorted by size descending. Skip if no breakdown.
|
||||
const formats = Object.entries(data.by_format || {}).sort((a, b) => b[1] - a[1]);
|
||||
if (!formats.length) { formatsEl.innerHTML = ''; return; }
|
||||
|
||||
const max = formats[0][1] || 1;
|
||||
formatsEl.innerHTML = formats.map(([ext, bytes]) => {
|
||||
const pct = Math.max(2, Math.round((bytes / max) * 100));
|
||||
return `
|
||||
<div class="stats-disk-format-row">
|
||||
<span class="stats-disk-format-name">${ext.toUpperCase()}</span>
|
||||
<div class="stats-disk-format-bar">
|
||||
<div class="stats-disk-format-fill" style="width:${pct}%"></div>
|
||||
</div>
|
||||
<span class="stats-disk-format-size">${_formatBytes(bytes)}</span>
|
||||
</div>
|
||||
`;
|
||||
}).join('');
|
||||
}
|
||||
|
||||
function _renderDbStorageChart(tables, totalFileSize, method) {
|
||||
const canvas = document.getElementById('stats-db-storage-chart');
|
||||
if (!canvas || typeof Chart === 'undefined') return;
|
||||
|
|
|
|||
|
|
@ -39449,6 +39449,72 @@ div.artist-hero-badge {
|
|||
text-align: right;
|
||||
}
|
||||
|
||||
/* Library Disk Usage */
|
||||
.stats-disk-usage-wrap {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 14px;
|
||||
margin-top: 8px;
|
||||
}
|
||||
|
||||
.stats-disk-total-row {
|
||||
display: flex;
|
||||
align-items: baseline;
|
||||
gap: 16px;
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
|
||||
.stats-disk-total-value {
|
||||
font-size: 28px;
|
||||
font-weight: 700;
|
||||
color: rgb(var(--accent-rgb));
|
||||
}
|
||||
|
||||
.stats-disk-total-meta {
|
||||
font-size: 12px;
|
||||
color: rgba(255, 255, 255, 0.55);
|
||||
}
|
||||
|
||||
.stats-disk-formats {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 6px;
|
||||
}
|
||||
|
||||
.stats-disk-format-row {
|
||||
display: grid;
|
||||
grid-template-columns: 60px 1fr 80px;
|
||||
align-items: center;
|
||||
gap: 10px;
|
||||
font-size: 12px;
|
||||
}
|
||||
|
||||
.stats-disk-format-name {
|
||||
font-weight: 600;
|
||||
color: rgba(255, 255, 255, 0.8);
|
||||
}
|
||||
|
||||
.stats-disk-format-bar {
|
||||
height: 8px;
|
||||
background: rgba(255, 255, 255, 0.05);
|
||||
border-radius: 4px;
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
.stats-disk-format-fill {
|
||||
height: 100%;
|
||||
background: linear-gradient(90deg,
|
||||
rgb(var(--accent-rgb)) 0%,
|
||||
rgba(var(--accent-rgb), 0.6) 100%);
|
||||
border-radius: 4px;
|
||||
}
|
||||
|
||||
.stats-disk-format-size {
|
||||
text-align: right;
|
||||
color: rgba(255, 255, 255, 0.55);
|
||||
font-variant-numeric: tabular-nums;
|
||||
}
|
||||
|
||||
/* Database Storage Chart */
|
||||
.stats-db-storage-wrap {
|
||||
display: flex;
|
||||
|
|
|
|||
Loading…
Reference in a new issue