Commit graph

27 commits

Author SHA1 Message Date
Broque Thomas
967c7f7c0a Migrate album-info builders to typed Album path
Steps 2+3 of typed metadata migration. Two album-info builders now
route through Album.from_<source>_dict() when caller passes a
known source:
- _build_album_info (album-tracks lookups)
- _build_single_import_context_payload (single-track import context)

Legacy duck-typing stays as fallback for unknown source, non-dict
input, or converter errors. Pure additive — existing callers
without source kwarg unchanged.
2026-05-03 22:53:12 -07:00
Broque Thomas
2ab460f5c4 Add Library Disk Usage card to System Statistics
Discord request (Samuel [KC]): show how much disk space the library
takes on the Stats page. Implementation piggybacks on the existing
deep scan — Plex/Jellyfin/Navidrome all return file size in their
track API responses, so we read it during the deep scan and store
it on the tracks row. Aggregation is then a single SQL query — no
filesystem walk, no extra I/O during the scan, no separate stat
job. SoulSync standalone gets size from os.path.getsize at insert
time (different code path; the file is local when we write the row).

Schema (`database/music_database.py`):
- New `file_size INTEGER` column on `tracks`. Migration uses the
  established `try SELECT, except ALTER TABLE ADD COLUMN` pattern.
  Idempotent; safe on existing installs. NULL on legacy rows so
  they don't contribute to totals until next deep scan refreshes.
- Added the column to the canonical CREATE TABLE so fresh installs
  get it without going through the migration path.

Track-object plumbing:
- `core/jellyfin_client.py` — JellyfinTrack reads MediaSources[0].Size
  alongside existing Bitrate read. None when 0 / missing.
- `core/navidrome_client.py` — NavidromeTrack reads `size` from
  the Subsonic song object (int coercion + None on parse fail).
- `core/soulsync_client.py` — SoulSyncTrack does os.path.getsize
  (only "server" where size has to come from disk).
- Plex needs no client-side change: track.media[0].parts[0].size
  is read directly inside insert_or_update_media_track.

Persistence — TWO separate insert paths:

(a) `database/music_database.py:insert_or_update_media_track` —
    Plex/Jellyfin/Navidrome flows. Reads file_size from Plex's
    MediaPart OR `track_obj.file_size` wrapper attribute (defensive
    Plex-attr-not-present check + > 0 type guard).
    INSERT writes the new column.
    UPDATE uses COALESCE(?, file_size) so a None from the server
    on a re-sync (rare Jellyfin Size omission) doesn't blank an
    existing value. Pinned via test.

(b) `core/imports/side_effects.py:record_soulsync_library_entry` —
    SoulSync standalone flow. Completely separate code path: the
    standalone deep scan moves files to staging for auto-import
    rather than calling insert_or_update_media_track. After the
    auto-import processes them, side_effects writes the tracks row
    directly. Reads file_size via os.path.getsize(final_path) at
    insert time (file is local) and includes it in the INSERT
    column list. SoulSync only does INSERT-if-not-exists (no
    UPDATE path), so no COALESCE concern.

Aggregator (`database/music_database.py:get_library_disk_usage`):
- SELECT COALESCE(SUM(file_size), 0), COUNT(file_size),
  COUNT(*) - COUNT(file_size) for the totals.
- Per-format breakdown done in Python via os.path.splitext over
  (file_path, file_size) rows — sidesteps SQLite's first-vs-last-dot
  ambiguity for paths like /music/Kendrick/M.A.A.D City/01.flac.
- Defensive: skips empty paths, paths without extension, and
  implausibly long extensions (>6 chars). Returns the full
  empty-shape dict (NOT a partial / undefined) when the column
  doesn't exist or queries fail, so the UI's `if (!data.has_data)`
  branch handles fresh installs cleanly.

API + UI:
- `core/stats/queries.py` — thin pass-through get_library_disk_usage
  matching the existing query-helper convention.
- `web_server.py` — new /api/stats/library-disk-usage endpoint
  mirroring the /api/stats/db-storage pattern.
- `webui/index.html` — new card in System Statistics above the
  Database Storage card.
- `webui/static/stats-automations.js` — _loadLibraryDiskUsage +
  _renderLibraryDiskUsage. Empty state: "Run a Deep Scan to
  populate (X tracks pending)". Partial: "X measured (+Y pending)".
  Full: total + format bars proportional to the largest format.
- `webui/static/style.css` — .stats-disk-* styled to match the
  Database Storage card.

Backward compatibility:
- Migration is additive; existing rows get NULL file_size; the
  empty-shape return from the aggregator means the UI renders
  cleanly without errors before any deep scan runs.
- Old installs upgrading will see "Run a Deep Scan to populate
  (N tracks pending)". Running their next deep scan fills sizes —
  the existing scan flow doesn't need any changes, just consumes
  the new track-wrapper attribute.

Tests:
- `tests/test_library_disk_usage.py` — 13 cases covering schema
  migration, NULL defaults on legacy inserts, fresh-install empty
  shape, summing with mixed NULL/known sizes, per-format breakdown,
  mixed-case extensions, paths with album-name dots, missing
  extensions, empty file_path, implausibly long extensions,
  JellyfinTrack.file_size persistence via insert_or_update_media_track,
  COALESCE preservation on null re-sync.
- `tests/imports/test_import_side_effects.py` — extended the
  existing record_soulsync_library_entry test to assert
  track_row['file_size'] == os.path.getsize(final_path), pinning
  the SoulSync-standalone path. Test fixture's tracks schema also
  updated to include the file_size column.

Verified: full suite 1813 pass (13 new, 1 existing-test extension),
ruff clean, smoke test populating + reading the column round-trips
correctly.

WHATS_NEW entry under '2.4.2' dev cycle.
2026-05-03 20:17:06 -07:00
Broque Thomas
04a14f7e96 Fix: tasks showed Completed when file was quarantined
User caught downloading Kendrick Mr. Morale: three tracks (Rich
Interlude, Savior Interlude, Savior) showed  Completed in the modal
but were missing on disk. Log forensics revealed two layered bugs.

Bug 1 — Verification wrapper assumed success on quarantined files
(`core/imports/pipeline.py`):

The outer `post_process_matched_download_with_verification` had a
fallback at the "no `_final_processed_path` in context" branch that
marked the task completed and notified `success=True`. The inner
post-processor sets `_final_processed_path` only when the file
actually reaches its destination. Integrity-rejected files
(`_integrity_failure_msg` set) and race-guard-failed files
(`_race_guard_failed` set) get quarantined or skipped without ever
setting `_final_processed_path`, so they fell straight into the
"assume success" branch.

Confirmed in user's log:
  No _final_processed_path in context for task d5b88b84-... —
  cannot verify, assuming success

That line fired for the same task right after the integrity check
quarantined the source file. Result:  Completed in UI, file in
quarantine, never delivered.

Fix: explicit checks for `_integrity_failure_msg` and
`_race_guard_failed` markers BEFORE the assume-success fallback.
Either marker set → task status='failed' with descriptive
error_message + `_notify_download_completed(success=False)`. The
pre-existing assume-success behavior preserved when no failure
markers are set (some legitimate flows complete without setting
`_final_processed_path`).

Bug 2 — AcoustID skip-logic too lenient
(`core/acoustid_verification.py`):

The "language/script" exemption was:
  if best_score >= 0.95 and (title_sim >= 0.55 or
                              artist_sim >= ARTIST_MATCH_THRESHOLD):

The OR-clause fired for English-vs-English titles by the same artist
that share NO actual content. Confirmed in user's log: requested
"Rich (Interlude)" by Kendrick Lamar, AcoustID identified the audio
as "R.O.T.C. (interlude)" by Kendrick Lamar (a totally different
song from his 2010 mixtape) — same artist scored ≥ARTIST threshold,
shared word "interlude" pushed title_sim above 0.55, skip fired.
Verification returned SKIP instead of FAIL, the wrong file was
accepted as the answer for three different track requests.

Fix: skip now requires positive evidence the mismatch is a real
language/script case:
  (a) Non-ASCII chars present in either title AND artist matches strongly
      → real transliteration case (kanji ↔ romaji etc)
  (b) BOTH title_sim >= 0.80 AND artist_sim >= ARTIST threshold
      → minor punctuation/casing differences

English-vs-English with very different titles by the same artist no
longer skipped — verification correctly returns FAIL, the wrong file
gets quarantined, the new wrapper logic above marks the task failed.

Tests:
- `tests/test_integrity_failure_marks_task_failed.py` — 4 cases
  pinning the wrapper-level state machine: integrity marker → failed,
  race-guard marker → failed, no markers → still assumes success
  (legacy path preserved), integrity-failure-takes-priority over
  missing-final-path fallback.
- `tests/test_acoustid_skip_logic.py` — 7 cases pinning the skip
  exemption: user's R.O.T.C-vs-Rich case → FAIL (regression test),
  Savior-vs-R.O.T.C → FAIL (same bug surface), Japanese kanji →
  romaji → SKIP (real language case still works), MAAD vs M.A.A.D →
  PASS or SKIP (punctuation tolerance), low fingerprint score →
  never skipped, high score but artist mismatch → no longer skipped,
  Crown vs Crown of Thorns → no longer skipped.

Verified: full suite 1793 pass (11 new), ruff clean.

WHATS_NEW entry under '2.4.2' dev cycle.
2026-05-03 18:28:32 -07:00
Broque Thomas
75fe04907f Wire SoundCloud as a first-class download source
Plug the previously-built SoundcloudClient (PR #478, the build-and-verify
phase) into every place a download source needs to appear. Follows the
same wiring contract as Tidal/Qobuz/HiFi/Deezer/Lidarr — orchestrator
routing, hybrid-mode picker, search dispatch, queue/cancel/clear,
provenance + library history, sidebar source label, settings UI all
work plug-and-play.

Backend wiring:
- `core/download_orchestrator.py` — import SoundcloudClient, _safe_init
  it at startup, add to _client() lookup, get_source_status(),
  check_connection's sources_to_check default, search source_names map,
  search_and_download_best _streaming_sources tuple, download
  source_map + source_names, and every iteration loop in
  reload_settings download-path-update / get_all_downloads /
  get_download_status / cancel_download (route + iterate) /
  clear_all_completed_downloads / cancel_all_downloads.
- `core/downloads/monitor.py` — added SoundCloud to the per-client
  loop that fetches active downloads outside the orchestrator (uses
  getattr fallback for older soulseek_client snapshots).
- `core/downloads/task_worker.py` — added SoundCloud (and Lidarr,
  which was missing too — bonus fix) to source_clients dict for hybrid
  fallback dispatch.
- `core/downloads/validation.py` — added 'soundcloud' to
  _streaming_sources so SoundCloud results go through the matching
  engine validation path instead of the Soulseek quality-filter path.
- `core/imports/side_effects.py` — three call sites: source_map for
  download_source label written to library_history, streaming-source
  guard for the `||`-encoded stream_id parsing, and source_service
  map for provenance recording. All three now include 'soundcloud'.
- `web_server.py` — five streaming-source detection tuples updated.
  New `/api/soundcloud/status` endpoint returns
  {available, configured, reachable} mirroring the Deezer/HiFi
  status-endpoint pattern; reachability runs a real cheap yt-dlp
  search so the settings Test Connection button gives a meaningful
  pass/fail signal.
- `config/settings.py` — added empty `soundcloud_download` defaults
  block so future tier-2 OAuth (SoundCloud Go+ session) doesn't have
  to migrate existing configs.

Frontend:
- `webui/index.html` — new `<option value="soundcloud">` in the
  download-source-mode dropdown, SoundCloud added to both hidden
  legacy hybrid-source selects, new settings container with info
  text + Test Connection button.
- `webui/static/settings.js` — HYBRID_SOURCES entry (with the
  SoundCloud cloud SVG icon), _hybridSourceEnabled default,
  updateDownloadSourceUI container display, allSources for legacy
  hybrid picker, testSoundcloudConnection function (hits the new
  status endpoint, color-codes the result), saveSettings
  soundcloud_download empty block.
- `webui/static/shared-helpers.js` — sidebar source-name map
  includes SoundCloud + Lidarr (Lidarr was also missing, bonus fix).
- `webui/static/helper.js` — WHATS_NEW entry under '2.4.2' dev cycle
  describing the user-visible change in the chill terse voice.

Tests:
- `tests/test_download_orchestrator_soundcloud.py` — 14 integration
  tests verifying the wiring: client constructed at startup, _client
  lookup resolves 'soundcloud', get_source_status includes it,
  download dispatcher routes username='soundcloud' to the SoundCloud
  client (and unknown usernames still fall back to Soulseek), hybrid
  search iterates SoundCloud when in order and skips it cleanly when
  unconfigured, get_all_downloads / get_download_status / cancel /
  clear walk SoundCloud, soundcloud-only mode dispatches only to
  SoundCloud, _streaming_sources tuple in validation includes
  'soundcloud'.
- `tests/downloads/test_download_orchestrator.py` — added
  `soundcloud` to the test fixture's _build_orchestrator helper so
  the new orchestrator attribute doesn't AttributeError in pre-
  existing tests that bypass __init__.

Verified:
- Full suite green (1728 passed, 2 deselected for soundcloud_live)
- Ruff clean
- Live SoundCloud-only mode search returns 25 SoundCloud tracks for
  "kendrick lamar luther" in <2s, returning properly-shaped
  TrackResult objects with username='soundcloud' and dispatch-key
  filename ready for the download path.

Out of scope (intentional deferrals):
- SoundCloud Go+ OAuth tier (256 kbps AAC) — anonymous-only for now.
  Adding auth later is a settings-page extension, no orchestrator
  changes needed.
- Album/playlist support — SoundCloud has playlists but they don't
  map to the album model the rest of SoulSync expects. Singles only.
2026-05-03 12:54:21 -07:00
Broque Thomas
42f3026eef Reject broken downloads before tagging via universal integrity check
Discord report (fresh.dumbledore [VRN]): slskd sometimes ships broken files
(truncated transfers, corrupt FLAC, wrong file substituted on filename match).
They flowed through post-processing and only surfaced later — Plex/Jellyfin
scan failures, dead-air playback, duplicate detector tripping over the wrong
length. By that point the file was already tagged, copied, mirrored to the
media server, and recorded in provenance.

New module `core/imports/file_integrity.py`:
- `check_audio_integrity(path, expected_duration_ms=None) -> IntegrityResult`
- Three tiered checks, cheapest to most expensive:
  1. File size sanity (catches 0-byte stubs and stub transfers)
  2. Mutagen parse (catches header damage, wrong-format-with-right-extension)
  3. Duration agreement vs. metadata source's expected length, ±3s tolerance
     (5s for tracks over 10 minutes — long tracks naturally drift more)
- Returns IntegrityResult with `ok`, human-readable `reason`, and per-check
  `checks` dict for debugging
- Never raises; pathological inputs return ok=False with explanation

Pipeline integration in `core/imports/pipeline.py:post_process_matched_download`:
- Hooks between the existing file-stability wait and AcoustID verification
- On failure: quarantine via existing `move_to_quarantine` helper, mark task
  failed with descriptive error, clear matched-context, fire
  `on_download_completed(success=False)` so the slot is released for retry
- Mirrors the existing AcoustID-failure path so retry behavior stays consistent
- Wrapped in try/except so an unexpected failure inside the check itself
  cannot block downloads — logs and continues

This is intentionally tier 1: universal across formats, no external deps.
A future tier could verify FLAC STREAMINFO MD5 by decoding audio (needs
flac binary or libflac wrapper) — skipped for now since tier 1 catches the
dominant Discord-reported cases (truncated, 0-byte, wrong file).

Tests:
- `tests/imports/test_file_integrity.py` — 14 cases covering all three check
  tiers, edge cases (zero/negative expected duration, long-track wider
  tolerance, caller tolerance override), and the mutagen-unavailable
  degradation path
- `tests/imports/test_import_pipeline.py` — two existing tests use 5-byte
  fixture files that the new check would reject; they monkeypatch the
  integrity check since they're testing plumbing (notification +
  metadata_runtime forwarding), not integrity behavior

WHATS_NEW entry under '2.4.2' dev cycle.
2026-05-03 08:21:01 -07:00
Broque Thomas
34ba26f5c8 Persist source IDs at download time + backfill onto tracks on sync
Followup to fix/watchlist-external-id-match. The companion PR closed
the demand side — the watchlist scanner asks for tracks by external IDs
before falling back to fuzzy. But for users on Plex / Jellyfin /
Navidrome the supply side was still broken: tracks.spotify_track_id
(and the other ID columns) only got populated by the asynchronous
enrichment workers, sometimes hours after the file was actually
written. During that window the ID match fell through to fuzzy and
the bug returned.

We were already collecting every ID during post-processing — they
live in the `pp` dict in core/metadata/source.py:embed_source_ids and
get embedded into file tags. We just dropped the in-memory copy
afterwards.

This PR persists them and uses them:

- Schema migration adds spotify_track_id / itunes_track_id /
  deezer_track_id / tidal_track_id / qobuz_track_id /
  musicbrainz_recording_id / audiodb_id / soul_id / isrc columns +
  indexes to the existing track_downloads table (already keyed by
  file_path).
- core/metadata/source.py:embed_source_ids exposes pp["id_tags"] and
  the resolved ISRC back to the import context as _embedded_id_tags
  / _isrc.
- core/imports/side_effects.py:record_download_provenance reads those
  context fields and passes them to db.record_track_download, which
  now accepts the new ID kwargs and persists them.
- New db.get_provenance_by_file_path with exact + basename-suffix
  fallback (handles container mount-root differences between
  download-time path and media-server-reported path).
- New db.backfill_track_external_ids_from_provenance copies IDs
  from track_downloads onto a tracks row idempotently — COALESCE on
  every column preserves any value the enrichment worker already
  wrote (enrichment is more authoritative for late binding).
- database/music_database.py:insert_or_update_media_track (the
  single insertion point used by every Plex / Jellyfin / Navidrome
  sync) calls the backfill immediately after each INSERT/UPDATE.
- New core/library/track_identity.py:find_provenance_by_external_id
  used as a second-tier fallback in watchlist_scanner.is_track_missing
  _from_library — catches the window between download and media-server
  sync. Caller checks os.path.exists on the provenance file_path
  before treating it as "already in library" so a deleted file
  doesn't prevent re-download.

Effect: freshly downloaded files become ID-recognizable to the
watchlist on the very next scan, no enrichment-wait window.

19 regression tests in tests/test_provenance_id_persistence.py:
- Schema migration adds expected columns + indexes
- record_track_download persists every ID kwarg
- record_track_download backward-compat (old kwargs still work)
- get_provenance_by_file_path: exact match, basename fallback for
  mount-root differences, multi-record latest-wins, defensive None
- backfill: copies all IDs, preserves existing via COALESCE,
  no-op when no provenance exists
- find_provenance_by_external_id: per-ID lookup, ISRC cross-bridge,
  OR semantics, latest-wins on multiple matches

Out of scope: backfilling provenance for files downloaded BEFORE
this PR (their track_downloads rows don't carry the new IDs). Those
continue to wait for enrichment. Acceptable — only affects historical
files; new downloads benefit immediately.

Full pytest 1625 passed; ruff clean.
2026-05-02 17:44:10 -07:00
Broque Thomas
486116c34f Honor lossy_copy.delete_original after successful conversion
Reported case (CAL): with lossy_copy.enabled=True,
lossy_copy.delete_original=True, and codec=mp3, every download left
both the original FLAC AND the converted MP3 in the target folder.
Users opting into a lossy-only library ended up dual-format on
every import.

Root cause: ``core/imports/file_ops.py:create_lossy_copy`` reads
``lossy_copy.codec`` and ``lossy_copy.bitrate`` from config but never
reads ``lossy_copy.delete_original``. The setting is only consulted
by the pre-move source-vanished check at
``core/imports/pipeline.py:651`` (so the pipeline knows to look for
a lossy variant when the FLAC has already moved on), but no code
path actually deletes the source after conversion.

Fix: after ffmpeg returns success and the QUALITY tag is written,
check ``lossy_copy.delete_original`` and ``os.remove`` the original
when enabled. Belt-and-suspenders:

- Same-path guard (``os.path.normpath(out_path) != os.path.normpath(final_path)``)
  prevents accidentally wiping the just-converted file if a future
  codec choice somehow resolves out_path to the source path.
- ``FileNotFoundError`` is treated as success (concurrent worker /
  dedup cleanup got there first).
- Other ``OSError`` (permission denied, locked file) is logged but
  doesn't propagate — the conversion already succeeded, the user just
  has to clean up the original manually.

Failure paths skip the delete:
- ffmpeg returns non-zero → returns None, original stays
- lossy_copy.enabled=False → early return before conversion runs
- delete_original=False (default) → original stays

7 regression tests cover honored-when-enabled, kept-when-disabled,
default-keep, ffmpeg-failure-path, lossy-disabled-path, racing-delete,
and locked-file paths. Full pytest 1563 passed; ruff clean.

Note: this PR does NOT address the second bug CAL mentioned (track
re-downloaded despite already existing on disk). That symptom is
caused by stale album metadata on the user's existing files — the
library DB has the track tagged on a different album than the
metadata source reports — combined with wishlist.allow_duplicate_tracks
defaulting to True. Same class of issue partially addressed in PR
fix/watchlist-redownload-and-duplicate-detection but compilation-
album drift is the only currently-handled case. Tracking separately.
2026-05-02 14:26:46 -07:00
Antti Kettunen
2b3022f6b0
Fix Spotify source ID fallback
- Prefer real Spotify IDs when importing Spotify contexts
- Skip numeric fallback IDs so Deezer values do not leak into spotify_* columns
- Add regressions for import context and SoulSync library writes
- Keep the route test asserting the Spotify album link
2026-05-02 22:02:01 +03:00
elmerohueso
f9f47f978e fix post-download tagging, and enable tagging for hifi 2026-05-02 07:50:12 -06:00
Broque Thomas
46d8e15674 Prune slskd dedup orphans after import
slskd appends "_<19-digit unix-nanosecond timestamp>" to a downloaded
filename when the destination already contains a same-named file
(concurrent downloads of the same track, partial-file retries after a
connection drop, cancelled-then-redownloaded files, the same track
surfacing in multiple synced playlists). The file-finder code already
recognized the suffix when matching a download to its source — but
after the canonical file moved into the library, the leftover
"_<timestamp>" siblings sat orphaned in the downloads folder forever.

Reported on Discord by Shdjfgatdif.

cleanup_slskd_dedup_siblings() runs at the end of each successful
import (3 safe_move_file sites in pipeline.py) and prunes any
remaining siblings that strip down to the canonical stem with the
same extension. Conservative match (>= 18 trailing digits) keeps
legitimate filenames like "Track 5" and "Album 1995" untouched. Per-
file unlink failures are swallowed so a single locked file doesn't
block the rest.

17 regression tests cover the suffix-strip primitive, orphan removal,
no-op cases, mismatched extensions, subdirectories, and partial-failure
recovery.
2026-05-01 09:35:08 -07:00
Broque Thomas
99a38a6201 Route imported singles/EPs through album_path template
Discord-reported (winecountrygames + fresh.dumbledore): "Import only
makes Albums folder no singles or eps". Users with a
${albumtype}s/$albumartist/... album_path template saw an "Albums"
folder fill up correctly but never any "Singles" or "EPs" folder.

build_import_album_info detected an album using
``total_tracks > 1`` AND ``album_name != track_title``. Spotify
singles fail both — total_tracks is 1 and the album is usually
named after the song. The result was that staging/auto-import
routed singles through single_path, which doesn't honour
$albumtype, so the user's per-type folder layout never applied.

Now also treats the metadata source's explicit release-type
classification ("single", "ep", "compilation") as evidence that
this is an album-shaped release, so it routes through album_path
and the user's $albumtype substitution runs. The default fallback
value "album" is deliberately excluded from this check so
single-track downloads with no real metadata behave exactly as
before.

Adds 10 regression tests covering the reported scenario, EP and
compilation explicit types, and three guards: normal multi-track
albums still detected, default 'album' type falls through, and
empty/unknown types fall through.
2026-04-30 21:33:09 -07:00
BoulderBadgeDad
58a4c1905b
Merge pull request #419 from kettui/refactor/metadata-service-split-and-metadata-client-management-optimizations
Split metadata service logic into separate modules, move client management out of web_server
2026-04-29 12:36:03 -07:00
Antti Kettunen
50e1ae3a3f
Move metadata helpers into package modules
- split metadata lookup logic into core/metadata/*
- keep core/metadata_service.py as the legacy barrel
- update tests and artist-detail code to patch concrete modules
2026-04-29 11:28:42 +03:00
Broque Thomas
d97d105b97 fix: substitute \$cdnum in download paths and skip auto disc folder when template uses it
User report: multi-disc albums on the latest dev had literal "\$cdnum"
in their filenames instead of the expected "CDxx" label, plus a
redundant "Disc N" folder on top of the in-filename label.

Two bugs in core/imports/paths.py:

1. _replace_template_variables (the substitution helper used by every
   download path builder) had no handling for \$cdnum or \${cdnum}. The
   matching helper in web_server.py and core/repair_jobs/library_reorganize.py
   did the substitution; this one didn't, so production downloads passed
   the placeholder through unchanged. Added a cdnum_value computation
   (CD%02d when total_discs > 1, empty otherwise) plus the corresponding
   bracket_map entry and \$cdnum replace before \$track (matches the
   ordering in the other path builders).

2. The album-path branch of build_final_path_for_track auto-injected a
   "Disc N" folder whenever total_discs > 1, suppressed only when the
   template contained \$disc. Templates using \$cdnum (or \${disc} /
   \${discnum} / \${cdnum}) got both a "CDxx" label in the filename and
   the auto folder. Widened the user_controls_disc check to cover all
   the disc-bearing placeholders.

Bonus cleanup along the way:

- Folder-part stripping now drops a leading \$cdnum token (mirrors the
  existing \$disc / \$discnum / \$quality strip — defensive against an
  empty cdnum landing alone in a folder segment).
- Filename cleanup now strips a leading "  - " left behind when \$cdnum
  expands to empty on a single-disc album (mirrors the same regex in
  library_reorganize.py).
- album_template config access switched from the dotted-path key to the
  nested-dict access pattern used by the rest of the function — handles
  both production config_manager and the flat _Config used in tests.

Tests: 4 new under tests/imports/test_import_paths.py
- multi-disc cdnum substitution produces "CD02"
- single-disc cdnum collapses to empty
- folder-part containing only \$cdnum is dropped
- build_final_path_for_track with \$cdnum template produces no auto
  "Disc N" folder

Full suite: 1276 passing (was 1272). Ruff clean.
2026-04-28 21:32:24 -07:00
Antti Kettunen
f32fc9d56e
Extract wishlist logic into dedicated package
- add core/wishlist as the home for wishlist payload, resolution, state, processing, reporting, and selection helpers
- move wishlist-specific tests into tests/wishlist alongside the new package layout
- keep web_server.py and the import/search callers as thin adapters for now
2026-04-28 21:17:24 +03:00
Broque Thomas
313b5677a5 Drop stale post-PR378 redefs and fix B009
Lifted-then-not-deleted leftovers from the PR378 merge:
- web_server.py `_resolve_album_group` and `_build_final_path_for_track`
  were already imported at module top from `core/imports/`. Removed the
  shadowing local copies.
- Mutagen reimports (FLAC/MP4/OggVorbis) at L17736-17738 shadowed the
  top-of-file imports. Picture/MP4Cover/MP4FreeForm were unused. Dropped
  the whole block.
- core/imports/context.py: `getattr(artist, "name")` -> `artist.name`
  (B009).

Ruff clean, 667 tests pass.
2026-04-27 13:39:16 -07:00
Antti Kettunen
02305096a3
Tighten metadata and import safety
- Normalize album import track display handling so queue labels and match rows stay consistent
- Bound MusicBrainz caches and avoid caching transient lookup failures
- Stop swallowing programmer errors in source enrichment helpers
- Restore import config test seams without reintroducing lazy imports
- Guard task completion calls and fix the Windows path test expectation
- Keep file lock tracking from growing without bound
2026-04-27 20:28:05 +03:00
Antti Kettunen
9315e74bea
Broaden import and metadata test coverage
- Cover search_result fallback normalization and ambiguous album detection.
- Add staging metadata, multi-disc path, and MusicBrainz enrichment cases.
- Move the single-track context test next to the imports code it exercises.
2026-04-27 19:55:07 +03:00
Antti Kettunen
4f236baa6d
Fix import normalization and task completion locking
- Promote legacy _source into source during import normalization.
- Keep the normalized import context neutral after stripping aliases.
- Avoid re-entering tasks_lock when marking completed download tasks.
2026-04-27 19:55:07 +03:00
Antti Kettunen
4c819681a1
Move single-track resolver; fix wishlist cleanup
- keep single-track import lookup in imports/resolution.py
- normalize simple-download search_result data before wishlist matching
- run wishlist cleanup for simple-download post-processing
- keep source-only artist detail on resolved names and MB short-circuit
2026-04-27 19:55:06 +03:00
Antti Kettunen
d04573f397
Fix single import source handling
- pass the selected manual match through singles import
- keep the import context source-aware so artist and album stay correct
- avoid treating non-Spotify IDs as wishlist Spotify IDs
- make wishlist logging and local variable names source-neutral
2026-04-27 19:54:45 +03:00
Antti Kettunen
9b2b6d856f
Split runtime builders into owning modules
- Move the import pipeline runtime factory into core.imports.pipeline
- Move the metadata runtime factory into core.metadata.enrichment
- Keep the web server wiring thin and drop the shared glue module
- Add contract tests that keep the two runtime bundles separate
2026-04-27 19:54:45 +03:00
Antti Kettunen
9e496397da
Move shared metadata helpers into package
- Relocate the shared metadata helper module from core/metadata_common.py into core/metadata/common.py.
- Update the new metadata package, the import pipeline, and the web entrypoint to use the package-scoped helper.
- Keep the shared config, mutagen, file-lock, and tag-writing helpers centralized without touching unrelated files.
2026-04-27 19:54:44 +03:00
Antti Kettunen
9656dbd46a
Thread runtime through metadata enrichment
- Pass the live runtime bundle into the shared metadata facade so worker-backed source enrichment can actually run.
- Forward runtime from the import pipeline and web-server wrapper into embed_source_ids.
- Add a regression test that verifies the runtime object reaches the source-ID embedding path.
2026-04-27 19:54:44 +03:00
Antti Kettunen
8319c6679f
Move new metadata helpers into a package
- Keep existing metadata_cache and metadata_service at the top level for now
- Move the new branch-local metadata helpers under core/metadata
- Share MusicBrainz release cache state from core.metadata.source and update import sites
2026-04-27 19:54:44 +03:00
Antti Kettunen
bdef127dd6
Lift shared runtime state into core
- Move app-wide task and activity registries out of core/imports
- Share one runtime-state module across the web server, API, and import pipeline
- Keep import-specific helpers focused on context and post-processing
2026-04-27 19:54:44 +03:00
Antti Kettunen
e10df4caf2
Rehome import helpers into core/imports
- Move import flow modules into a dedicated package
- Update app and test imports to the new namespace
- Group the import-focused tests under tests/imports
2026-04-27 19:54:44 +03:00