soulsync/core
Broque Thomas 8493be207e Auto-import: SoulSync standalone library writes server-quality rows
# Background

SoulSync standalone is meant to be a full replacement for Plex /
Jellyfin / Navidrome — files imported via auto-import (or any other
import path) should land in the database with the same field richness
a media-server scan would write. They weren't.

# Gaps fixed

The auto-import worker built a context dict for each track and handed
it to `_post_process_matched_download` (the same callback the regular
download flow uses). That dict was missing three things downstream
needed:

1. **No `source` field anywhere.** `record_soulsync_library_entry`
   reads `get_import_source(context)` to pick the source-aware ID
   columns (`spotify_track_id` / `deezer_id` / `itunes_track_id` /
   etc.) on the artists / albums / tracks rows. With no source, the
   resolver returned an empty string → `get_library_source_id_columns("")`
   returned an empty dict → the `UPDATE tracks SET <source>_id = ?`
   blocks were silently skipped. Result: every auto-imported track
   landed with NULL on every source-id column. Watchlist scans
   (which match by stable source IDs to detect "this track is already
   in library") couldn't recognise these rows and would re-download
   them on the next pass.

2. **No `_download_username='auto_import'`.** Both
   `record_library_history_download` and `record_download_provenance`
   default to "Soulseek" when no `username` is in the context. Every
   staging-folder import was being labelled as a Soulseek download
   in library history + provenance — false signal in the UI.

3. **No per-recording IDs (`isrc`, `musicbrainz_recording_id`) on
   track_info.** The Navidrome scanner already writes
   `musicbrainz_recording_id` directly to the tracks row when present.
   Picard-tagged libraries always carry MBID; metadata sources
   (Spotify via MusicBrainz enrichment, Deezer, etc.) carry ISRC.
   Auto-import had access to both via the metadata-source response
   but didn't propagate them — so the soulsync row went in with
   NULL on both columns.

# Changes

**`core/auto_import_worker.py` — `_process_matches`:**
- Top-level `'source': source` (from `identification['source']`)
- `'_download_username': 'auto_import'`
- `track_info['isrc']`, `track_info['musicbrainz_recording_id']` —
  pulled from the per-track payload returned by the metadata source
- `track_info['album_id']` — back-reference so source-aware ID
  resolution works on sources whose API nests album under
  `track.album.id` rather than `track.album_id`
- `spotify_artist['id']` now correctly carries the artist's source ID
  (was `identification['album_id']`, a copy-paste bug from the
  original implementation that made artist-id resolution fall back
  to fuzzy matching)
- `spotify_album['artists'][0]['id']` carries artist source ID for
  the same resolution path

**`core/imports/side_effects.py`:**
- `record_library_history_download` source_map: add
  `"auto_import": "Auto-Import"` — tags imported tracks correctly
- `record_download_provenance` source_service: add
  `"auto_import": "auto_import"` — provenance shows real source
- `record_soulsync_library_entry` track INSERT: now includes
  `musicbrainz_recording_id` + `isrc` columns (matches
  `insert_or_update_media_track`'s shape for Navidrome /
  Plex / Jellyfin scans). Both default to NULL when not present.

# Behavior preserved

- Files still land in the same library template path (no path-build
  change)
- Other media-server flows (Plex / Jellyfin / Navidrome users)
  unaffected — `record_soulsync_library_entry` still gates on
  `get_active_media_server() == "soulsync"`. Auto-import on those
  servers continues to drop the file in the library folder + emits
  `batch_complete` for the scan-trigger automation, same as before.
- Direct downloads (search → Download button) unaffected — they
  already passed `source` + `username` correctly.

# Tests added

`tests/imports/test_auto_import_context_shape.py` (8 tests, new file):
- Worker context carries `source` for every metadata source
  (parametrised across spotify / deezer / itunes / discogs)
- `_download_username='auto_import'` set unconditionally
- ISRC + MBID propagate from track payload to track_info when present
- ISRC + MBID default to empty string when absent (downstream
  normalises to NULL at write time)
- track_info includes album-id back-reference

`tests/imports/test_import_side_effects.py` (4 new tests + 2 schema
column adds):
- `record_soulsync_library_entry` writes mbid + isrc columns when
  present in track_info
- Deezer source maps to deezer_id column (regression case for
  source-aware column resolver)
- `record_library_history_download` labels `_download_username=
  'auto_import'` as "Auto-Import" not "Soulseek"
- `record_download_provenance` registers source_service as
  "auto_import" not "soulseek"

# Verification

- 8/8 new context-shape tests pass
- 6/6 side-effects tests pass (4 new + 2 existing)
- 207 imports tests pass
- 2359 full suite passes (+12 from baseline 2347, no regressions)
- 1 pre-existing flake (`test_watchdog_warns_about_stuck_workers`,
  passes in isolation, unrelated to this change)
- Ruff clean
2026-05-09 19:25:47 -07:00
..
artists Surface silent exceptions in core/artists — 23 sites 2026-05-07 09:49:29 -07:00
automation Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
discovery Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
download_engine Add module logger + surface silent exceptions in 7 logger-less files — 12 sites 2026-05-07 10:27:04 -07:00
download_plugins Address Copilot doc-drift review 2026-05-05 15:46:48 -07:00
downloads Manual picks: stream results, don't auto-retry, fix stuck-at-0% 2026-05-08 15:12:58 -07:00
enrichment Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
imports Auto-import: SoulSync standalone library writes server-quality rows 2026-05-09 19:25:47 -07:00
library Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
media_server Final review-pass nits — class docstring, dead branch, dead imports, boot resilience 2026-05-05 22:59:33 -07:00
metadata Auto-import: MBID/ISRC fast paths + duration sanity gate 2026-05-09 09:57:33 -07:00
playlists Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
repair_jobs Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
search Cin-6: Rename soulseek_client global → download_orchestrator 2026-05-04 23:23:32 -07:00
stats Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
streaming Cin-6: Rename soulseek_client global → download_orchestrator 2026-05-04 23:23:32 -07:00
watchlist Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
wishlist Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
workers Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
acoustid_client.py Clean up 286 ruff lint errors to unblock CI and fix 10 latent bugs 2026-04-21 13:30:52 -07:00
acoustid_verification.py Reject AcoustID matches whose version disagrees with the expected track 2026-05-07 13:25:30 -07:00
album_consistency.py Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
api_call_tracker.py Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
artist_source_detail.py Move metadata helpers into package modules 2026-04-29 11:28:42 +03:00
artist_source_lookup.py Make artist_name Optional in find_library_artist_for_source 2026-04-22 22:15:03 -07:00
audiodb_client.py Add API Rate Monitor dashboard with real-time speedometer gauges 2026-04-02 10:46:43 -07:00
audiodb_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
auto_import_worker.py Auto-import: SoulSync standalone library writes server-quality rows 2026-05-09 19:25:47 -07:00
automation_engine.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
connection_detect.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
connection_test.py Cin-6: Rename soulseek_client global → download_orchestrator 2026-05-04 23:23:32 -07:00
database_update_worker.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
debug_info.py Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
deezer_client.py Add download buttons + bulk action to artist top-tracks sidebar 2026-05-07 15:44:47 -07:00
deezer_download_client.py Surface silent exceptions in metadata clients — 37 sites 2026-05-07 09:33:38 -07:00
deezer_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
discogs_client.py Surface silent exceptions in metadata clients — 37 sites 2026-05-07 09:33:38 -07:00
discogs_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
download_orchestrator.py Lift shared download dataclasses + boot via singleton factory 2026-05-05 09:08:39 -07:00
genius_client.py Add API Rate Monitor dashboard with real-time speedometer gauges 2026-04-02 10:46:43 -07:00
genius_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
genre_filter.py Expand default genre whitelist from 223 to 272 genres 2026-04-18 20:32:49 -07:00
hifi_client.py Surface engine-not-wired errors + exclude soulseek from monitor aggregation 2026-05-05 12:20:51 -07:00
hydrabase_client.py Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
hydrabase_worker.py Improve graceful shutdown and rollback safety 2026-04-12 15:17:18 +03:00
itunes_client.py Surface silent exceptions in metadata clients — 37 sites 2026-05-07 09:33:38 -07:00
itunes_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
jellyfin_client.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
lastfm_client.py Add Last.fm Track Radio to Discover page 2026-04-14 23:41:36 -07:00
lastfm_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
library_reorganize.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
lidarr_download_client.py Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
listenbrainz_client.py Remove emojis from all Python log and print statements 2026-04-11 21:11:02 -07:00
listenbrainz_manager.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
listening_stats_worker.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
lyrics_client.py Remove emojis from all Python log and print statements 2026-04-11 21:11:02 -07:00
matching_engine.py Merge remote-tracking branch 'origin/dev' into refactor/media-server-engine 2026-05-05 20:36:31 -07:00
metadata_service.py Move profile Spotify cache into registry 2026-04-29 12:36:37 +03:00
musicbrainz_client.py MusicBrainz: Resolve release-group MBIDs to a release on album click 2026-04-24 08:48:02 -07:00
musicbrainz_search.py MusicBrainz: Dedupe same-named homonyms in artist search results 2026-04-24 10:27:45 -07:00
musicbrainz_service.py Add metadata cache maintenance and health monitoring 2026-03-30 07:40:20 -07:00
musicbrainz_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
navidrome_client.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
personalized_playlists.py Discover: sharpen track selection (diversity, source-aware popularity, library dedup, SQL genre) 2026-05-08 08:49:22 -07:00
plex_client.py Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
qobuz_client.py Address Copilot doc-drift review 2026-05-05 15:46:48 -07:00
qobuz_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
reorganize_queue.py Reorganize queue: race + dedupe fixes from kettui review 2026-04-26 08:40:24 -07:00
reorganize_runner.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
repair_worker.py Surface silent exceptions in repair_worker — 16 sites 2026-05-07 09:17:51 -07:00
replaygain.py Add module logger + surface silent exceptions in 7 logger-less files — 12 sites 2026-05-07 10:27:04 -07:00
runtime_state.py Add module logger + surface silent exceptions in 7 logger-less files — 12 sites 2026-05-07 10:27:04 -07:00
seasonal_discovery.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
socketio_cors.py Socket.IO CORS: handle self-review nits 2026-04-26 19:24:43 -07:00
soulid_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
soulseek_client.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
soulsync_client.py Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
soundcloud_client.py Surface engine-not-wired errors + exclude soulseek from monitor aggregation 2026-05-05 12:20:51 -07:00
spotify_client.py Add download buttons + bulk action to artist top-tracks sidebar 2026-05-07 15:44:47 -07:00
spotify_public_scraper.py Add Spotify Link tab for public playlist/album scraping without API credentials 2026-03-15 21:25:05 -07:00
spotify_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
tag_writer.py Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
tidal_client.py Surface silent exceptions in metadata clients — 37 sites 2026-05-07 09:33:38 -07:00
tidal_download_client.py Surface engine-not-wired errors + exclude soulseek from monitor aggregation 2026-05-05 12:20:51 -07:00
tidal_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
watchlist_scanner.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
web_scan_manager.py MS Cin-5: Drop per-server globals — engine owns the clients 2026-05-05 18:05:45 -07:00
wishlist_service.py Extract wishlist logic into dedicated package 2026-04-28 21:17:24 +03:00
worker_utils.py Fix Album Completeness job reporting zero findings for every album 2026-04-24 12:39:41 -07:00
youtube_client.py Drop redundant import sys inside _auto_download_disabled 2026-05-08 15:51:16 -07:00