Merge pull request #951 from Nezreka/dev

Dev

.dockerignore

@@ -25,6 +25,13 @@ __pycache__/
 dist/
 build/
 
+# Frontend build artifacts and local dependency caches
+webui/.tanstack/
+webui/.vite/
+webui/node_modules/
+webui/test-results/
+webui/static/dist/
+
 # Virtual environments
 venv/
 env/
@@ -51,6 +58,20 @@ Incomplete/*
 artist_bubble_snapshots.json
 .spotify_cache
 
+# Auto-downloaded ffmpeg binaries — the YouTube client downloads these
+# into tools/ when system ffmpeg isn't on PATH. The Dockerfile installs
+# system ffmpeg via apt, so the container never needs the bundled
+# binaries. If a CI run leaves them in the workspace before the docker
+# build (e.g. because a test imported web_server which initialized the
+# YouTube client), they'd otherwise get baked into the image — adding
+# ~388 MB and getting duplicated again by the chown layer.
+tools/ffmpeg
+tools/ffprobe
+tools/ffmpeg.exe
+tools/ffprobe.exe
+tools/*.zip
+tools/*.tar.xz
+
 # Documentation
 *.md
 README.md

.github/workflows/docker-publish.yml

@@ -9,9 +9,9 @@ on:
   workflow_dispatch:
     inputs:
       version_tag:
-        description: 'Version tag (e.g. 2.4.1)'
+        description: 'Version tag (e.g. 2.8.2)'
         required: true
-        default: '2.4.1'
+        default: '2.8.2'
 
 jobs:
   build-and-push:

.gitignore

@@ -12,11 +12,14 @@ __pycache__/
 
 # User-specific files (auto-created by the app if missing)
 config/config.json
-database/music_library.db
-database/music_library.db-shm
-database/music_library.db-wal
-database/music_library.db.backup_*
+config/youtube_cookies.txt
+# All app databases are live user data — never commit (music_library, video_library, …)
+database/*.db
+database/*.db-shm
+database/*.db-wal
+database/*.db.backup_*
 database/api_call_history.json
+storage/image_cache/
 logs/*.log
 logs/*.log.*
 

DISCOVER_BEST_IN_CLASS_PLAN.md

@@ -0,0 +1,74 @@
+# discover page — best in class plan (#913 + full generator audit)
+
+morning notes. did the work overnight. tl;dr at top, details below, all of it `break nothing` + tested.
+
+## what i shipped tonight (done, tested, safe)
+
+### 1. listening recommendations (#913) — went from BROKEN to best-in-class
+
+the feature was silently producing **zero** recs on real data. dug in and found three stacked bugs in the generation:
+
+- **wrong id key (the killer).** `similar_artists.source_artist_id` is a *source* id (spotify/itunes/deezer), but the scanner built its id→name map from `artists.id` (the internal row id). so every edge resolved to nothing → 0 recs. proved it on your live db: internal-id join = 0 rows, spotify-id join = 71,636 rows.
+- **consensus could never fire.** it fed the ranker `get_top_similar_artists`, which does `GROUP BY similar_artist_name` + `MAX(source_artist_id)` — collapsing every similar artist down to a *single* seed. the whole point of the ranker is "artist X is similar to 3 of your seeds = strong signal," and that signal was being flattened away before it ever reached the ranker.
+- **similarity strength thrown away.** each edge stores a 1-10 closeness rank; it was ignored (everything weighted equally).
+
+the fix (all in the pure, tested core + thin scan wiring):
+- build id→name from the **source-id columns**, query the **raw per-seed edges** (consensus preserved), and thread **similarity_rank** into the score so a seed's closest matches count for more.
+- **recency-weighted seeds**: `weight = lifetime_plays + 1.5 × recent_30d_plays`. picks now track what you're into *now*, not just all-time totals.
+
+result on your actual library (simulated through the real code path): **40 recommendations, 13 with multi-seed consensus, all 40 with cached art.** top picks: Arcangel (Bad Bunny + Ozuna + J Balvin), Melanie Martinez (Ariana + Billie), Maluma, De La Ghetto — all coherent, all explainable.
+
+### 2. its own row on the discover page
+
+new row **"Based On Your Listening"** — play-weighted, consensus-ranked artist cards with a **"Because you listen to X, Y"** line. sits right above the library-driven "Recommended For You" row. purely additive: new endpoint `/api/discover/listening-recommendations`, new loader, hides itself when empty.
+
+**you need to run one watchlist scan** for the row to populate (the data regenerates during the scan — i did NOT touch your live db). before that scan the row just stays hidden; after it, it fills in.
+
+> note: this is deliberately different from the existing "Recommended For You" row. that one is driven by your *whole library / watchlist*. this one is driven by your *actual listening intensity* — the ~30 artists you really play, not the thousands you happen to own.
+
+### 3. Fresh Tape "only 5-10 tracks" — fixed
+
+root cause: `get_discovery_recent_albums` orders `release_date DESC`, so announced-but-unreleased albums sort to the *top* and ate the 50-album budget. the scanner skipped them *after* the budget was already spent → only a handful of released albums left → 5-10 tracks. fixed by fetching a generous budget (300) **and** excluding next-year albums at the query, so released albums fill the budget. the precise same-year `is_future_release` skip stays as a second guard. downstream caps (6/artist, top 75, take 50) unchanged.
+
+**tests:** 25 pure-core cases (consensus/similarity/recency) + 2 Fresh Tape regression tests, all green. full discovery suite (255) green. nothing else touched.
+
+---
+
+## best-in-class roadmap for listening recs (next phases — your call)
+
+these are the levers to take it further. ordered by value-to-risk. none are required; tonight's work stands on its own.
+
+| phase | what | value | risk | notes |
+|---|---|---|---|---|
+| **3** | **playable track row** ✅ DONE | high | low-med | shipped: "🎧 Your Listening Mix" row — a track playlist (play/queue/download/sync) right under the artist row. stored as full render-ready dicts (not pool-hydrated, so it can't shrink on pool rotation like Fresh Tape does). |
+| **4** | **direct top-tracks fetch** ✅ DONE | high | med | shipped: scan fetches each recommended artist's top tracks (Spotify/Deezer), resolving the artist id by name-search when the similar-artist row lacks one — guarded by a strict name-match so it never pulls the wrong artist. bounded (top 20 recs), per-call guarded, fail-soft to the pool. iTunes has no top-tracks API → pool-only there. needs a live scan to populate. |
+| **5** | **genre-affinity boost** | med | low | we already compute your genre breakdown. boost recs whose genres match your top genres → tighter taste alignment. pure scoring add. |
+| **6** | **adventurousness dial** | med | low | the ranker already supports `min_seed_count` (consensus floor). expose it as a "Safe ↔ Adventurous" slider on the row. |
+| **7** | **diversity pass** | low-med | low | avoid 40 recs all orbiting your single heaviest seed — cap picks-per-seed so the row spans your taste. |
+
+the core is built to absorb all of these without re-plumbing — `similarity_from_rank`, `build_recency_weighted_seeds`, and the scoring formula are all pure + tested.
+
+---
+
+## full discover-page generator audit (every soulsync-built row, excluding last.fm + listenbrainz)
+
+how each one is generated today, and whether it can be elevated. "clear win" = safe + additive. "product call" = needs your decision (changes the row's character).
+
+### curated (built during the scan, then hydrated)
+- **Fresh Tape / Release Radar** — new releases from watchlist+similar artists. **FIXED tonight** (see above). one more *clear win* available: hydration silently drops any curated id no longer in the discovery pool — could fall back to the stored `track_data_json` blob so the row can't shrink at read time.
+- **The Archives / Discovery Weekly** — strong already. nice 3-tier popularity split + serendipity scoring (boost never-played artists, penalize overplayed). same hydration-drop caveat as Fresh Tape; same cheap fallback fix.
+- **Seasonal Mix** — cleanest of the bunch. hydrates from a dedicated `seasonal_tracks` table (carries its own data), so it doesn't suffer the pool-drop problem. no bug.
+
+### discovery-pool generators (live queries)
+- **Popular Picks** — ranks by popularity DESC. solid. only nit: on iTunes (no popularity scale) it silently degrades to random — indistinguishable from Shuffle there. UI-label thing at most.
+- **Hidden Gems** — *clear win*. currently `ORDER BY RANDOM()` over low-popularity tracks — so it's "random obscure," not "*best* obscure." a light ranking (popularity just under the threshold, or genre-affinity to you) would make it feel curated instead of arbitrary. (a deeper *product call*: add personalization like Archives has — bigger lift, changes its "pure underground" character.)
+- **Genre Playlists** — good. pushes the genre match into SQL. `RANDOM()` ordering is fine for a browse; a popularity/affinity tiebreak (*clear win*) would make thin genres feel less arbitrary.
+- **Discovery Shuffle** — random by design, correct. only possible add: exclude tracks already shown in other rows this refresh (needs a cross-section seen-set — medium plumbing).
+- **Time Machine (by decade)** — *clear win, low risk*: decades are hardcoded, so a modern-only library shows 7 decade tabs, 5 empty. filter the tabs to decades that actually have pool data.
+- **Daily Mix** — the weakest row. the "50% your library" half permanently returns nothing (library tracks have no source ids to play), so each Daily Mix is really just a relabeled Genre Playlist. real fix = backfill source ids into library rows (*schema-level, higher risk*) — worth a dedicated pass, not a quick tweak. also silently falls back to "top artists as pseudo-genres" when genre data is missing → "Daily Mix 1" becomes mislabeled artist-radio. gate/label that (*clear win*).
+
+### cross-cutting
+- **hydration fragility** (Fresh Tape + Archives): both depend on curated ids still living in the pool at read time; misses are dropped silently. Seasonal already solved this with a dedicated table. giving the two spotify-style rows the same data-blob fallback is the single most robust cross-cutting fix. low risk, clear win.
+- **RANDOM-ordering pattern** (Hidden Gems, Shuffle, Genre, Decade): intentional for variety, but leaves quality signal on the table for the non-shuffle rows. adding a light ranking pass to Hidden Gems + Genre is the biggest "best-in-class" lever after tonight's work.
+
+want me to take any of these? the Hidden Gems ranking + Time Machine empty-decade filter + the Fresh Tape/Archives hydration fallback are all safe, additive, same-shape-as-tonight wins i can knock out next.

Dockerfile

@@ -1,6 +1,16 @@
 # SoulSync WebUI Dockerfile
 # Multi-architecture support for AMD64 and ARM64
 
+FROM node:24-slim AS webui-builder
+
+WORKDIR /app/webui
+
+COPY webui/package.json webui/package-lock.json ./
+RUN npm ci
+
+COPY webui/ ./
+RUN npm run build
+
 # Stage 1: Builder — install Python dependencies with compilation tools
 FROM python:3.11-slim AS builder
 
@@ -19,6 +29,16 @@ COPY requirements.txt .
 RUN pip install --no-cache-dir --upgrade pip && \
     pip install --no-cache-dir -r requirements.txt
 
+# yt-dlp must track YouTube faster than its stable channel ships — stable can
+# lag months behind a breaking YouTube change while extraction is broken
+# ("Requested format is not available"). Build images with the NIGHTLY channel.
+# COMMIT_SHA is referenced in the RUN so CI's layer cache (cache-from: gha)
+# busts on every new commit — otherwise this layer could pin a stale "nightly"
+# for months, silently defeating its purpose.
+ARG COMMIT_SHA=""
+RUN echo "yt-dlp nightly for build ${COMMIT_SHA}" && \
+    pip install --no-cache-dir -U --pre "yt-dlp[default]"
+
 # Stage 2: Runtime — only runtime dependencies, no build tools
 FROM python:3.11-slim
 
@@ -34,24 +54,61 @@ ENV PATH="/opt/venv/bin:$PATH"
 # Set working directory
 WORKDIR /app
 
-# Install runtime-only system dependencies (no gcc/build tools)
+# Install runtime-only system dependencies (no gcc/build tools).
+# unzip is needed by the Deno installer below.
 RUN apt-get update && apt-get install -y --no-install-recommends \
     curl \
     gosu \
     ffmpeg \
     libchromaprint-tools \
+    unzip \
     && rm -rf /var/lib/apt/lists/*
 
+# Deno — JavaScript runtime for yt-dlp. YouTube gates its downloadable formats
+# behind JS challenges (nsig); without a JS runtime, yt-dlp's extraction is
+# deprecated and streams / music-video downloads fail with "Requested format
+# is not available". Deno is yt-dlp's default-enabled runtime; the official
+# installer auto-detects amd64/arm64. `deno --version` fails the build early
+# if the install ever breaks.
+RUN curl -fsSL https://deno.land/install.sh | DENO_INSTALL=/usr/local sh && \
+    deno --version
+
 # Create non-root user for security
 RUN useradd --create-home --shell /bin/bash --uid 1000 soulsync
 
-# Copy application code
-COPY . .
+# Copy application code with ownership baked in.
+# Using `COPY --chown` instead of `COPY` + `chown -R /app` avoids an
+# extra image layer that duplicates the entire /app tree just to flip
+# ownership bits — Docker layers are immutable, so chown -R rewrites
+# every file into a new layer. On a clean repo that's small; if any
+# bulky workspace file slips in (e.g. auto-downloaded ffmpeg binaries
+# in tools/), it gets counted twice in the image. Cin caught this on
+# 2026-05-08 — see the .dockerignore comment for the same incident.
+COPY --chown=soulsync:soulsync . .
+COPY --chown=soulsync:soulsync --from=webui-builder /app/webui/static/dist /app/webui/static/dist
 
-# Create necessary directories with proper permissions
+# Create runtime mount-point directories the app expects to exist.
 # NOTE: /app/data is for database FILES, /app/database is the Python package
-RUN mkdir -p /app/config /app/data /app/logs /app/downloads /app/Transfer /app/MusicVideos /app/scripts && \
-    chown -R soulsync:soulsync /app
+# NOTE: /app/Staging is required even though most users bind-mount it —
+# the entrypoint mkdir runs early and is gated by `set -e`, so a missing
+# pre-baked directory would crash the container into a restart loop on
+# rootless Docker/Podman where in-container "root" can't write to /app.
+# Pre-baking the dir here makes the entrypoint mkdir a guaranteed no-op.
+# NOTE: /app/Stream is the transient single-file streaming cache used by
+# the basic-search "Play" flow (cleared per use, never persistent). It's
+# created lazily by `core/streaming/prepare.py` via `os.makedirs`, which
+# fails silently on rootless Docker where the soulsync UID can't write
+# to /app — playback then errors out with no obvious cause. Pre-baking
+# at build time (when the layer is owned by root) avoids that path.
+# NOTE: /app/storage is the PRIVATE album-bundle staging area for the
+# torrent / usenet whole-release flow (download_source.album_bundle_staging_path
+# defaults to 'storage/album_bundle_staging'). Like /app/Stream it's created
+# lazily at runtime via mkdir(parents=True); without pre-baking it owned by
+# soulsync, the album-bundle copy fails with "[Errno 13] Permission denied:
+# 'storage'" because /app itself is root-owned and the soulsync UID can't
+# create a top-level dir there.
+RUN mkdir -p /app/config /app/data /app/logs /app/downloads /app/Transfer /app/Staging /app/Stream /app/storage /app/MusicVideos /app/scripts && \
+    chown soulsync:soulsync /app/config /app/data /app/logs /app/downloads /app/Transfer /app/Staging /app/Stream /app/storage /app/MusicVideos /app/scripts
 
 # Create defaults directory and copy template files
 # These will be used by entrypoint.sh to initialize empty volumes

README.md

@@ -8,7 +8,7 @@
 
 > **IMPORTANT**: Configure file sharing in slskd to avoid Soulseek bans. Set up shared folders at `http://localhost:5030/shares`.
 
-**Community**: [Discord](https://discord.gg/wGvKqVQwmy) | [Reddit](https://old.reddit.com/r/ssync/) | **Website**: [ssync.net](https://www.ssync.net/) | **Support**: [GitHub Issues](https://github.com/Nezreka/SoulSync/issues) | **Donate**: [Ko-fi](https://ko-fi.com/boulderbadgedad)
+**Community**: [Discord](https://discord.gg/wGvKqVQwmy) | **Website**: [ssync.net](https://www.ssync.net/) | **Support**: [GitHub Issues](https://github.com/Nezreka/SoulSync/issues) | **Donate**: [Ko-fi](https://ko-fi.com/boulderbadgedad)
 
 ---
 
@@ -105,6 +105,21 @@ SoulSync bridges streaming services to your music library with automated discove
 - Catches wrong versions (live, remix, cover) even from streaming API sources
 - Fail-open design: verification errors never block downloads
 
+#### AcoustID API key
+
+AcoustID verification is opt-in. To enable it, request a free API key
+at <https://acoustid.org/new-application> and paste it into
+Settings → AcoustID. Without a key, downloads still complete but the
+verification step is skipped silently.
+
+If a track was previously tagged by AcoustID but the retag action in
+the AcoustID Scanner no longer changes anything, see issue #704 — the
+most common cause is that the file already carries a
+`MUSICBRAINZ_TRACKID` tag, which the retag step uses as a short-circuit
+and therefore never overwrites. Removing the cached
+`MUSICBRAINZ_TRACKID` (and the `ACOUSTID_ID` if present) from the file
+restores the retag.
+
 ### Metadata & Enrichment
 
 **10 Background Enrichment Workers**: Spotify, MusicBrainz, iTunes, Deezer, Discogs, AudioDB, Last.fm, Genius, Tidal, Qobuz
@@ -278,19 +293,64 @@ The template points at `boulderbadgedad/soulsync:latest` (stable) by default. To
 ```bash
 git clone https://github.com/Nezreka/SoulSync
 cd SoulSync
-pip install -r requirements.txt
+python -m pip install -r requirements.txt
+
+# Build the React WebUI bundle used by the Python server.
+# Docker does this automatically; Python installs must do it manually.
+cd webui
+npm ci
+npm run build
+cd ..
+
 gunicorn -c gunicorn.conf.py wsgi:application
 # Open http://localhost:8008
 ```
 
-For local development and tests:
+When updating a Python/no-Docker install with `git pull`, rebuild the WebUI before restarting SoulSync:
 
 ```bash
-pip install -r requirements-dev.txt
-pytest
-gunicorn -c gunicorn.dev.conf.py wsgi:application
+cd webui
+npm ci
+npm run build
+cd ..
 ```
 
+If `webui/static/dist/.vite/manifest.json` is missing or stale, React-owned routes and route handoffs may not load correctly.
+
+**YouTube streaming / music videos** need two extra things on bare-metal installs (Docker bundles both):
+
+- **Deno** — yt-dlp now requires a JavaScript runtime to unlock YouTube formats. Without it, streams and music-video downloads fail with `Requested format is not available`. Install: `winget install DenoLand.Deno` (Windows) or see [deno.com](https://docs.deno.com/runtime/), then restart SoulSync.
+- **yt-dlp nightly** — the stable release can lag months behind YouTube changes. If YouTube breaks, update with: `python -m pip install -U --pre "yt-dlp[default]"`
+
+### Local Development
+
+This is only for contributors working on the WebUI with hot reload. Normal Python/no-Docker installs should build once with `npm run build` as shown above, then run only Gunicorn.
+
+For active frontend development, use two terminals so the backend and Vite stay independent:
+
+1. Backend
+   ```bash
+   python -m pip install -r requirements-dev.txt
+   gunicorn -c gunicorn.dev.conf.py wsgi:application
+   ```
+   The dev Gunicorn config watches backend files and restarts the Python server when they change.
+2. Frontend
+   ```bash
+   cd webui
+   npm ci
+   npm run dev
+   ```
+   Vite hot reloads the React side when you change webui files.
+
+Run tests separately when needed:
+
+```bash
+python -m pytest
+```
+
+If you want a convenience launcher, `python dev.py` starts both halves together
+on any OS. `./dev.sh` remains available as a Unix shell wrapper.
+
 ---
 
 ## Setup Guide
@@ -300,6 +360,7 @@ gunicorn -c gunicorn.dev.conf.py wsgi:application
 - **slskd** running and accessible ([Download](https://github.com/slskd/slskd/releases)) — required for Soulseek downloads
 - **Spotify API** credentials ([Dashboard](https://developer.spotify.com/dashboard)) — optional but recommended for discovery
 - **Media Server** (optional): Plex, Jellyfin, or Navidrome
+- **Deno** (Python/no-Docker installs only): JavaScript runtime required by yt-dlp for YouTube streaming/music videos — `winget install DenoLand.Deno` or [deno.com](https://docs.deno.com/runtime/). Docker images bundle it.
 - **Deezer ARL token** (optional): For Deezer downloads — get from browser cookies after logging into deezer.com
 - **Tidal account** (optional): For Tidal downloads — authenticate via device flow in Settings
 - **Qobuz account** (optional): For Qobuz downloads — email/password login in Settings
@@ -413,17 +474,14 @@ SoulSync uses a `dev` → `main` flow:
 2. Branch off `dev`: `git checkout -b fix/your-change dev`
 3. Make your changes and commit
 4. Push and open a PR against **`dev`** (not `main`)
-5. CI (`build-and-test.yml`) runs ruff lint + compile + pytest on your branch — wait for green
+5. CI (`build-and-test.yml`) runs ruff lint + compile + `python -m pytest` on your branch — wait for green
 6. A maintainer reviews and merges
 
 ### Running locally
 
-```bash
-pip install -r requirements-dev.txt
-python -m ruff check .       # must be 0 errors
-python -m pytest             # all tests must pass
-gunicorn -c gunicorn.dev.conf.py wsgi:application
-```
+Use the [Local Development](#local-development) section above for the full repo-wide setup and the portable dev launcher.
+
+For web UI work, see [webui/README.md](webui/README.md). It keeps the React-side notes close to the app while this file stays the single place for repo-wide dev instructions.
 
 Ruff config lives in `pyproject.toml`. The ruleset is intentionally lenient — it catches real bugs (undefined names, import shadowing, closure-in-loop) without style nits.
 

RELEASE_2.7.9_discord.md

@@ -0,0 +1,13 @@
+**SoulSync 2.7.9** is out 🎉 a big one.
+
+🎚️ **Best-quality downloads** — downloads now follow a ranked quality profile you drag to order (FLAC 24/192 → mp3). best-quality mode grabs the highest-quality copy across *every* source; priority mode gets an opt-in rank-based order toggle. quarantine is folded into the Downloads page + safer imports (AcoustID fail-closed, silence/truncation guards).
+
+🎧 **Discover got smart** — "Based On Your Listening" ranks artists from who you *actually play*, and "Your Listening Mix" is a playable track playlist of their top tracks (works on any source). Fresh Tape fills properly now.
+
+⚡ **Wing It Pool** — a new spot next to Discovery Pool to review + re-match the tracks Wing It guessed at (they used to be invisible).
+
+🔁 **Auto-Sync redesign** — the scheduling board is now clean horizontal lanes instead of a side-scrolling column wall.
+
+🐛 **Fixes** — multi-disc albums no longer show disc-2 as "missing" (#927), playlists no longer stuck on "Never Synced" (#925), and tracks can't import while quarantined (#928). thanks @ramonskie + @nick2000713 🙏
+
+⚠️ **re-scan your library once** so the multi-disc fix can backfill disc numbers on existing tracks. enjoy! 🎶

RELEASE_2.8.0_discord.md

@@ -0,0 +1,13 @@
+**SoulSync 2.8.0** is out 🎉 a quality + reliability release.
+
+🧹 **The Unverified queue, finally under control** — if you saw thousands of "unverified" rows piling up, this is for you. the AcoustID scan stops duplicating history rows, a one-time reconcile on startup clears the existing backlog from your library (no re-scan), and a new 🧹 *Clean orphaned* button sweeps dead rows whose file is gone. (#934 — thanks @nick2000713 for #938)
+
+✂️ **Preview Clip Cleanup** — a new Tools job that finds the ~30s preview clips the HiFi source sometimes hands back instead of the full song, then deletes them and re-wishlists the real version. each finding has a ▶ Play button so you can confirm before approving.
+
+💿 **Album Completeness handles split albums** — an album split across multiple library rows no longer shows every fragment as falsely "incomplete"; it groups the validated fragments into one correct finding. (#936 — thanks @ragnarlotus)
+
+🐛 **Fixes** — pasted YouTube cookies no longer throw `unsupported browser: "custom"` on Docker (thanks HellRa1SeR); longer remasters aren't quarantined as "truncated" anymore (#937, thanks @diegocade1); "Add to Wishlist" from a discography went from ~15–30s *per track* to instant; wishlist art renders for re-downloads; and **Clear Completed** is back on the Downloads page.
+
+⚡ **Performance** — trimmed the dashboard GPU usage that was hammering Firefox/Zen (and Background Particles are OFF by default now), plus bounded the runaway memory growth that could lock the app up on big libraries. (#935 / #802)
+
+enjoy! 🎶

RELEASE_2.8.1_discord.md

@@ -0,0 +1,15 @@
+**SoulSync 2.8.1** is out 🎉 a feature + reliability release.
+
+🎧 **Export playlists to Spotify & Deezer** — the mirrored-playlist export now has **Sync to Spotify** and **Sync to Deezer** next to the ListenBrainz / JSPF options. it builds a playlist in your account from the IDs soulsync already has (the discovery cache first, then your library), so an already-discovered playlist exports **instantly with zero API calls**. re-exporting updates the same playlist instead of duplicating it, and an optional *"match missing tracks"* toggle confidently searches for the stragglers — a wrong-artist or karaoke version is left out, never guessed. the first Spotify export asks permission once. (#945)
+
+🏷️ **Library Reorganize — Rename only** — a lighter action that just **renames your files** to your naming scheme: no re-tag, no quality/AcoustID re-check, no copy-to-staging. much faster on a NAS, and only touches files whose path actually changes. pick it from the new Action dropdown. (#875 — thanks @tsoulard / @Tacobell444)
+
+💿 **Broader lossless handling** — lossy-copy now covers **all lossless formats**, not just FLAC (#941); and **DSD** (`.dsf`/`.dff`) is recognized as lossless instead of false-flagged "truncated" (#939).
+
+🐛 **Download + search fixes** — an unbalanced bracket no longer false-fails as "file not found"; a file we couldn't quarantine is left for retry instead of deleted; "file not found" errors are actionable now; pasted Qobuz/Tidal links inject the exact track into manual search (#932); the Wing It pool "Fix Match" works again.
+
+⚡ **Reduce visual effects, refined** — it no longer freezes functional motion (spinners, progress), only the expensive GPU stuff (blur, shadows, glow). worker orbs default OFF on Firefox and run at ~30fps under reduce-effects. plus a jellyfin scan watchdog fix for big libraries.
+
+🔧 **Under the hood** — settings cleanup (#943, @nick2000713), spotify oauth hardening (#942) + npm security fixes (#944, HellRa1SeR).
+
+enjoy! 🎶

RELEASE_2.8.2_discord.md

@@ -0,0 +1,9 @@
+**SoulSync 2.8.2** is out 🎉 a stability + performance release.
+
+🎧 **Spotify, reliably** — the Docker boot hang is fixed: with Spotify as your primary source, an unreachable Spotify API could block startup so the container bound `:8008` but never served the UI. auth probes are now deferred during boot + capped with a timeout. the "re-auth didn't stick" bug is fixed too (the OAuth callback and the app were reading different token caches), and **Sync to Spotify** now works — it asks for playlist-write permission once, on-demand, leaving your normal login untouched. (#949 — thanks HellRa1SeR)
+
+⚡ **The "slow after update" fix** — the post-update lag wasn't SoulSync, it was browser password managers (Bitwarden/1Password/etc.) rebuilding their autofill overlay on *every* DOM change. non-credential fields are now marked so they skip them — **~110× less main-thread blocking** in the reporter's benchmark. plus a new **Max Performance** mode (Settings → Appearance) that kills every effect for no-GPU / Docker setups. (#948 — thanks @nick2000713)
+
+📥 **Large-library imports no longer time out** — dropping a whole library into staging used to make the import page scan every file synchronously and never load. the scan runs in the background now with a live "Scanning N of M…" progress, and fills in when done. (#947 — thanks @ramonskie)
+
+enjoy! 🎶

SPEC_canonical_album_version.md

@@ -0,0 +1,144 @@
+# Spec: Canonical Album Version (fixes #765 + #767-Bug2)
+
+**Status:** design only — no code yet.
+**Goal:** Pin ONE canonical `(source, album_id)` per album, chosen by best-fit to
+the user's actual files, so the Library Reorganizer, Track Number Repair, and
+tagging/enrichment all agree on the same release. Today each re-resolves
+independently and they contradict each other (Spotify Believer=4 vs MusicBrainz
+Believer=3; standard album mislinked to a deluxe release).
+
+**Canonical-selection rule (decided):** *match the user's actual files.* The
+canonical release is the candidate whose track count + per-track durations +
+titles best fit what's on disk. Self-correcting: picks standard when you own the
+standard, deluxe when you own the deluxe.
+
+---
+
+## Hard requirement: don't disrupt the running app
+
+Every stage below is **additive and dormant until explicitly consumed**, and
+every consumer **falls back to today's behavior when no canonical is set**. So:
+- albums with no resolved canonical behave EXACTLY as they do now;
+- each stage is independently shippable and reversible;
+- nothing big-bangs.
+
+---
+
+## Stage 1 — Schema + pure scorer (ships dormant, zero behavior change)
+
+### Schema (additive, nullable → migration-safe)
+Add to `albums` (guarded `ALTER TABLE ... ADD COLUMN`, idempotent — mirror the
+existing column-exists checks; see [[db-schema-review]] migration-safety notes):
+- `canonical_source TEXT` — e.g. 'spotify' / 'itunes' / 'musicbrainz'
+- `canonical_album_id TEXT`
+- `canonical_score REAL` — best-fit score (for transparency / re-resolve gating)
+- `canonical_resolved_at TIMESTAMP`
+
+All nullable. Existing rows = NULL → "unresolved" → consumers fall back. No
+backfill in this stage. No reads in this stage.
+
+### Pure core helper (the testable heart) — `core/metadata/canonical_version.py`
+```
+score_release_against_files(file_tracks, release_tracks) -> float
+pick_canonical_release(file_tracks, candidates) -> (best, score) | (None, 0)
+```
+- `file_tracks`: list of {duration_ms, title, track_number?} read from disk.
+- `release_tracks`: a candidate release's tracklist (same shape).
+- Scoring (tunable weights):
+  - **track-count fit** — exact match strongly preferred; |Δcount| penalized.
+  - **duration alignment** — greedily match each file to its closest release
+    track by duration (within a tolerance, e.g. ±3s); reward coverage.
+  - **title overlap** — token/fuzzy overlap as a tiebreaker.
+  - **graceful degradation** — if a source gives no per-track durations, fall
+    back to count + title only (never crash, never force-pick).
+- Returns the best candidate + score, or (None, 0) when nothing clears a floor
+  (so we never pin a bad guess — leave it unresolved, consumers fall back).
+
+### Tests (extreme, like the rest of this codebase)
+- standard (11) vs deluxe (17) with 11 files on disk → picks standard.
+- same album, 17 files → picks deluxe.
+- duration disambiguation when track counts tie (e.g. radio edit vs album).
+- missing-duration source → count+title fallback still picks sanely.
+- no candidate clears the floor → (None, 0).
+- "Believer" standard(=track 3 listing) vs Spotify(=4) with the user's files →
+  whichever the files actually match.
+
+**End of Stage 1: scorer exists + tested, columns exist, NOTHING reads/writes
+them yet. Provably zero behavior change.**
+
+---
+
+## Stage 2 — Resolver populates canonical (writes, still no consumers)
+
+A function `resolve_canonical_for_album(album_id, db, ...)`:
+1. Gather on-disk file metadata for the album (durations/titles) via the
+   library's known file paths.
+2. Gather candidate releases: every source the album has an ID for
+   (spotify/itunes/deezer/discogs/soul/musicbrainz) AND — for the deluxe/standard
+   case — sibling editions discoverable from those. Fetch each tracklist
+   (cached, rate-limited).
+3. `pick_canonical_release(files, candidates)` → store `(source, album_id, score)`
+   on the album row if it clears the floor.
+
+Wiring: a small **backfill repair job** (dry-run-capable) + a hook in enrichment
+when an album is (re)enriched. Still **no tool READS canonical**, so behavior is
+unchanged — this stage only populates the new columns. Reversible: clearing the
+columns reverts to unresolved.
+
+Tests: resolver picks the right release for the standard/deluxe fixtures; stores
+nothing when below floor; idempotent re-resolve.
+
+Cost note: fetching multiple candidate releases = more API calls. Mitigate via
+cache + only-on-(re)enrich + the existing rate trackers. Surface in the job's
+progress so it's not silent.
+
+---
+
+## Stage 3 — Reorganizer reads canonical (first real behavior change, gated)
+
+In `library_reorganize._resolve_source`: if the album has
+`canonical_source`/`canonical_album_id`, use THAT first; else fall back to the
+current `get_source_priority` walk. One-line precedence change, fully gated on
+non-NULL.
+
+Tests: with canonical set → resolves to it; with canonical NULL → byte-identical
+to today. Re-run the existing reorganize battery (148 tests) — must stay green.
+
+**This alone fixes #767-Bug2** (a standard album whose files match the standard
+release pins the standard, so reorganize stops targeting the deluxe folder).
+
+---
+
+## Stage 4 — Track Number Repair reads canonical (closes #765)
+
+In `track_number_repair._resolve_album_tracklist`: add **Fallback -1** (before
+everything) — if the album has a canonical `(source, album_id)`, use it. The
+existing 6-level cascade stays as the fallback for albums with no canonical
+(preserves its all-01-album rescue ability — the regression risk we refused to
+take in the reactive fix).
+
+Now both tools resolve the SAME release → same track numbers → no contradiction.
+
+Tests: canonical present → both tools agree (shared-release test); canonical
+NULL → existing cascade unchanged.
+
+---
+
+## Risks & mitigations
+- **Extra API calls** (Stage 2 fetches multiple releases) → cache, rate-limit,
+  only-on-(re)enrich, progress-logged.
+- **Sources without per-track durations** → scorer degrades to count+title.
+- **Schema migration** → additive nullable columns only; idempotent guards.
+- **Wrong pick** → floor gate (never pin a low-confidence guess); `canonical_score`
+  stored for inspection/re-resolve; manual override possible later.
+- **Backward-compat** → every consumer falls back to today's path when NULL, so
+  un-resolved albums (incl. all existing albums until backfilled) are unaffected.
+
+## Out of scope (for now)
+- Per-album manual version override UI (can layer on later — the columns support it).
+- Merging the two tools into one (the reporter's alt suggestion) — unnecessary
+  once they share the canonical.
+
+## Suggested order to build
+1, then 2, then 3, then 4 — each shippable and verifiable on its own. We can stop
+after any stage and the app is consistent (just with fewer consumers wired).

Support/REVERSE-PROXY.md

@@ -0,0 +1,134 @@
+# Running SoulSync behind a reverse proxy (nginx / Caddy / Traefik)
+
+Putting SoulSync behind a reverse proxy lets you serve it over **HTTPS** and — the
+important part — put **authentication** in front of it before exposing it to the
+internet. This guide covers the safe setup.
+
+> **The golden rule:** the safest way to expose *any* self-hosted app publicly is
+> to require authentication at the proxy (an auth layer), **not** to rely on the
+> app's own protection. SoulSync's launch PIN is a useful fallback, but it is not
+> a substitute for a real auth layer on a public instance.
+
+---
+
+## 1. Turn on reverse-proxy mode
+
+By default SoulSync does **not** trust proxy headers (so a direct client can't spoof
+its IP or pretend the connection is HTTPS). If you're behind a proxy that
+terminates TLS, turn on **Settings → Security → "Behind a reverse proxy"** and
+**restart SoulSync** (this option applies at startup).
+
+When enabled, SoulSync:
+- trusts `X-Forwarded-For/Proto/Host/Port` from **one** proxy hop (correct client
+  IP, HTTPS detection, redirects),
+- marks its session cookie `Secure` (HTTPS-only) + `SameSite=Lax`, and
+- sends conservative security headers (`X-Content-Type-Options: nosniff`,
+  `X-Frame-Options: SAMEORIGIN`, `Strict-Transport-Security`). No CSP is set — tune
+  one at your proxy if you want it.
+
+**Leave it off if you access SoulSync directly over http:// on your LAN** — turning
+it on would make the session cookie HTTPS-only and break plain-HTTP access. With it
+off, none of the above applies and SoulSync behaves exactly as before.
+
+> The launch PIN is also brute-force limited (10 wrong attempts from an IP → a
+> short cooldown), regardless of this setting — a correct PIN is never affected.
+
+Restart SoulSync after changing it.
+
+---
+
+## 2. nginx
+
+SoulSync uses WebSockets (Socket.IO), so the `Upgrade`/`Connection` headers are
+**required** — without them live updates silently stop working.
+
+```nginx
+server {
+    listen 443 ssl;
+    server_name soulsync.example.com;
+
+    ssl_certificate     /etc/letsencrypt/live/soulsync.example.com/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/soulsync.example.com/privkey.pem;
+
+    # Large library scans / uploads
+    client_max_body_size 0;
+
+    location / {
+        proxy_pass http://127.0.0.1:8008;
+
+        proxy_set_header Host              $host;
+        proxy_set_header X-Real-IP         $remote_addr;
+        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $scheme;
+        proxy_set_header X-Forwarded-Host  $host;
+
+        # Required for Socket.IO / live updates
+        proxy_http_version 1.1;
+        proxy_set_header Upgrade    $http_upgrade;
+        proxy_set_header Connection "upgrade";
+
+        proxy_read_timeout 3600s;   # long-running scans
+        proxy_send_timeout 3600s;
+    }
+}
+```
+
+---
+
+## 3. Caddy
+
+Caddy handles TLS automatically and proxies WebSockets out of the box:
+
+```caddy
+soulsync.example.com {
+    reverse_proxy 127.0.0.1:8008
+}
+```
+
+Caddy sets `X-Forwarded-*` for you. (Add an auth provider directive if you want
+auth at the proxy — see below.)
+
+---
+
+## 4. Traefik
+
+Traefik proxies WebSockets automatically and forwards the headers. Point a router
+at the SoulSync service on port `8008` with your TLS resolver; no extra WebSocket
+config is needed.
+
+---
+
+## 5. Add authentication in front (recommended for public instances)
+
+Pick one:
+
+- **Auth proxy** — [Authelia](https://www.authelia.com/),
+  [Authentik](https://goauthentik.io/), or
+  [oauth2-proxy](https://oauth2-proxy.github.io/oauth2-proxy/). These sit in front
+  of SoulSync and force a login (with 2FA) before any request reaches it. Best
+  option for internet exposure.
+
+  SoulSync can **trust the proxy's authenticated-user header** so the launch PIN is
+  skipped once the proxy has logged you in. Set the header name in **Settings →
+  Security → "Auth proxy user header"** (e.g. `Remote-User`).
+
+  > ⚠️ **Only enable this behind a proxy you control that STRIPS any client-supplied
+  > copy of that header.** Otherwise a direct visitor could send `Remote-User: admin`
+  > and walk straight in. It's **off by default** — an unset header name means
+  > SoulSync ignores the header entirely (a spoofed one does nothing).
+- **HTTP Basic Auth** — quick and simple (nginx `auth_basic` / Caddy `basicauth`).
+  Better than nothing; weaker than an auth proxy.
+- **SoulSync launch PIN** — set an admin PIN in Settings. Enforced server-side, so
+  it can't be bypassed by hitting the API directly — but it's a shared PIN, so
+  treat it as a fallback, not your only gate.
+
+---
+
+## Troubleshooting
+
+- **Live updates / progress bars don't move** → the WebSocket `Upgrade`/`Connection`
+  headers are missing (nginx) or your proxy is buffering. Check section 2.
+- **Login won't stick / "session expired"** → you enabled `trust_reverse_proxy` but
+  are reaching SoulSync over plain `http://`. The session cookie is now HTTPS-only;
+  use `https://`, or turn the setting off for direct HTTP access.
+- **Scans time out** → raise `proxy_read_timeout` / `proxy_send_timeout`.

api/downloads.py

@@ -121,10 +121,13 @@ def register_routes(bp):
 
         try:
             from utils.async_helpers import run_async
-            soulseek = current_app.soulsync.get("soulseek_client")
+            soulseek = current_app.soulsync.get("download_orchestrator")
             if not soulseek:
                 return api_error("NOT_AVAILABLE", "Soulseek client not configured.", 503)
 
+            current_app.logger.info(
+                f"[CancelTrigger:api.public_cancel] download_id={download_id} username={username}"
+            )
             ok = run_async(soulseek.cancel_download(download_id, username, remove=True))
             if ok:
                 return api_success({"message": "Download cancelled."})
@@ -138,7 +141,7 @@ def register_routes(bp):
         """Cancel all active downloads and clear completed ones."""
         try:
             from utils.async_helpers import run_async
-            soulseek = current_app.soulsync.get("soulseek_client")
+            soulseek = current_app.soulsync.get("download_orchestrator")
             if not soulseek:
                 return api_error("NOT_AVAILABLE", "Soulseek client not configured.", 503)
 

api/request.py

@@ -104,7 +104,7 @@ def _run_search_and_download(request_id, query, notify_url):
             if request_id in _pending_requests:
                 _pending_requests[request_id]['status'] = 'searching'
 
-        soulseek = current_app._get_current_object().soulsync.get('soulseek_client')
+        soulseek = current_app._get_current_object().soulsync.get('download_orchestrator')
         if not soulseek:
             with _requests_lock:
                 if request_id in _pending_requests:

api/search.py

@@ -2,10 +2,14 @@
 Search endpoints — search external sources (Spotify, iTunes, Hydrabase).
 """
 
+import logging
+
 from flask import request, current_app
 from .auth import require_api_key
 from .helpers import api_success, api_error
 
+logger = logging.getLogger(__name__)
+
 
 def register_routes(bp):
 
@@ -38,8 +42,8 @@ def register_routes(bp):
                         if hydra_results:
                             tracks = [_serialize_track(t) for t in hydra_results]
                             return api_success({"tracks": tracks, "source": "hydrabase"})
-                except Exception:
-                    pass
+                except Exception as e:
+                    logger.debug("hydrabase search failed: %s", e)
 
             spotify = ctx.get("spotify_client")
             from core.metadata_service import get_primary_source, get_primary_client

api/serializers.py

@@ -368,6 +368,7 @@ def serialize_similar_artist(obj, fields: Optional[Set[str]] = None) -> dict:
         "source_artist_id": d.get("source_artist_id"),
         "similar_artist_spotify_id": d.get("similar_artist_spotify_id"),
         "similar_artist_itunes_id": d.get("similar_artist_itunes_id"),
+        "similar_artist_musicbrainz_id": d.get("similar_artist_musicbrainz_id"),
         "similar_artist_name": d.get("similar_artist_name"),
         "similarity_rank": d.get("similarity_rank"),
         "occurrence_count": d.get("occurrence_count"),

api/system.py

@@ -2,12 +2,15 @@
 System endpoints — status, activity feed, stats.
 """
 
+import logging
 import time
 
 from flask import current_app
 from .auth import require_api_key
 from .helpers import api_success, api_error
 
+logger = logging.getLogger(__name__)
+
 
 def register_routes(bp):
 
@@ -26,7 +29,7 @@ def register_routes(bp):
             spotify = ctx.get("spotify_client")
             spotify_ok = bool(spotify and spotify.is_authenticated())
 
-            soulseek = ctx.get("soulseek_client")
+            soulseek = ctx.get("download_orchestrator")
             soulseek_ok = bool(soulseek)
 
             hydrabase = ctx.get("hydrabase_client")
@@ -35,8 +38,8 @@ def register_routes(bp):
                 try:
                     ws, _ = hydrabase.get_ws_and_lock()
                     hydrabase_ok = ws is not None and ws.connected
-                except Exception:
-                    pass
+                except Exception as e:
+                    logger.debug("hydrabase status probe failed: %s", e)
 
             return api_success({
                 "uptime": f"{hours}h {minutes}m {seconds}s",

beatport_unified_scraper.py

@@ -2612,7 +2612,7 @@ class BeatportUnifiedScraper:
                     result['title'] = title
 
         except Exception as e:
-            pass  # Silently handle URL extraction errors
+            logger.debug("URL slug extraction: %s", e)
 
         return result
 
@@ -3186,8 +3186,8 @@ class BeatportUnifiedScraper:
                     except Exception:
                         continue
 
-        except Exception:
-            pass
+        except Exception as e:
+            logger.debug("parse release tracks failed: %s", e)
 
         return tracks
 

config/config.example.json

@@ -44,7 +44,8 @@
   },
   "metadata_enhancement": {
     "enabled": true,
-    "embed_album_art": true
+    "embed_album_art": true,
+    "single_to_album": false
   },
   "file_organization": {
     "enabled": true,
@@ -56,6 +57,11 @@
       "playlist_path": "$playlist/$artist - $title"
     }
   },
+  "import": {
+    "staging_path": "./Staging",
+    "replace_lower_quality": false,
+    "folder_artist_override": true
+  },
   "lossy_copy": {
     "enabled": false,
     "bitrate": "320",
@@ -67,4 +73,4 @@
   "listenbrainz": {
     "token": "LISTENBRAINZ_TOKEN"
   }
-}
+}

config/settings.py

@@ -66,6 +66,12 @@ class ConfigManager:
         
         self._load_config()
 
+    # Placeholder shipped to the browser in place of a configured secret
+    # (#832 follow-up). The settings UI shows it as masked dots; if it's
+    # round-tripped back on save, ``set()`` treats it as "keep existing" so the
+    # real value is never overwritten by the mask.
+    REDACTED_SENTINEL = '__redacted_unchanged__'
+
     # Dot-notation paths to sensitive config values that must be encrypted at rest.
     # Paths pointing to dicts encrypt the entire dict as a JSON blob.
     _SENSITIVE_PATHS = frozenset({
@@ -87,6 +93,10 @@ class ConfigManager:
         'soulseek.api_key',
         'deezer_download.arl',
         'lidarr_download.api_key',
+        'prowlarr.api_key',
+        'torrent_client.password',
+        'usenet_client.api_key',
+        'usenet_client.password',
         # Enrichment services
         'listenbrainz.token',
         'acoustid.api_key',
@@ -466,14 +476,71 @@ class ConfigManager:
                 "download_path": "./downloads",
                 "transfer_path": "./Transfer",
                 "max_peer_queue": 0,
-                "download_timeout": 600
+                "download_timeout": 600,
+                # Reddit report (YeloMelo95, Bell Canada): the existing
+                # 35-per-220s sliding-window cap allows all 35 searches in
+                # rapid succession before throttling — that burst trips ISP
+                # anti-abuse. This knob forces a min gap between consecutive
+                # searches even when the window cap isn't hit. 0 = disabled
+                # (preserves prior behavior).
+                "search_min_delay_seconds": 0,
             },
             "download_source": {
-                "mode": "soulseek",  # Options: "soulseek", "youtube", "tidal", "qobuz", "hifi", "hybrid"
+                "mode": "soulseek",  # Options: "soulseek", "youtube", "tidal", "qobuz", "hifi", "hybrid", "torrent", "usenet"
                 "hybrid_primary": "soulseek",  # Legacy: primary source for hybrid mode
                 "hybrid_secondary": "youtube",  # Legacy: fallback source for hybrid mode
                 "hybrid_order": [],  # Ordered list of sources for hybrid mode (overrides primary/secondary)
                 "stream_source": "youtube",  # Options: "youtube" (instant, default), "active" (use download source; falls back to youtube if soulseek)
+                # Album-bundle (torrent / usenet single-source) poll tuning.
+                # Downloader is polled every N seconds until the release
+                # lands; whole job aborts at the timeout. Defaults match
+                # the previous hard-coded constants. Users on slow private
+                # trackers / large box sets can extend the timeout without
+                # editing source.
+                "album_bundle_poll_interval_seconds": 2.0,
+                "album_bundle_timeout_seconds": 6 * 60 * 60,    # 6 hours
+                # Stalled-torrent handling (noldevin): abandon a torrent that
+                # makes zero download progress for this long (dead magnet
+                # stuck on "downloading metadata", no seeders) instead of
+                # holding the worker for the full album timeout. 0 disables.
+                "torrent_stall_timeout_seconds": 10 * 60,       # 10 minutes
+                # What to do when a torrent stalls: "abandon" (remove it +
+                # its partial data, fail the download so the next source can
+                # try) or "pause" (pause in the client, leave for the user).
+                "torrent_stall_action": "abandon",
+                # Where THIS container can read completed torrent/usenet
+                # downloads (#857). The downloader (qBit/SAB) reports a save
+                # path from inside ITS OWN container — often a category folder
+                # like /data/downloads/music — which may be mounted at a
+                # different point here. Set these to the in-container path(s)
+                # where SoulSync sees those finished downloads; the resolver
+                # then finds the release by name under them. Empty = fall back
+                # to the soulseek download/transfer dirs (the shared-volume
+                # default). See core.download_plugins.album_bundle.resolve_reported_save_path.
+                "torrent_download_path": "",
+                "usenet_download_path": "",
+                # Explicit remote→local prefix mappings for non-shared / oddly
+                # mounted layouts (Sonarr/Radarr "Remote Path Mapping" style):
+                # a list of {"from": "<client path>", "to": "<soulsync path>"}.
+                # Tried before the basename fallback above.
+                "usenet_path_mappings": [],
+            },
+            "post_processing": {
+                # When a download is quarantined (AcoustID mismatch, integrity /
+                # duration failure), retry the next-best candidate instead of
+                # failing outright. Default ON (PR #801's documented default —
+                # the monitor reads this with inline default True; this template
+                # said False, so fresh installs silently shipped with the retry
+                # engine off while existing configs got it on. CI caught the
+                # split: its fresh default config failed all 7 requeue tests).
+                "retry_next_candidate_on_mismatch": True,
+                # Opt-in exhaustive retry: budget retries PER SOURCE so every
+                # source (Soulseek, then HiFi/Tidal/…) gets its own attempts
+                # before the track gives up. Default off (single global cap).
+                "retry_exhaustive": False,
+                # Retries per search query per source in exhaustive mode. The
+                # per-source budget is query_count × this value.
+                "retries_per_query": 5,
             },
             "tidal_download": {
                 "quality": "lossless",  # Options: "low", "high", "lossless", "hires"
@@ -495,6 +562,16 @@ class ConfigManager:
             "hifi_download": {
                 "quality": "lossless",  # Options: "low", "high", "lossless", "hires"
             },
+            "hifi": {
+                "embed_tags": True,
+                "tags": {
+                    "track_id": True,
+                    "artist_id": True,
+                    "isrc": True,
+                    "bpm": True,
+                    "copyright": True,
+                }
+            },
             "lidarr_download": {
                 "url": "",
                 "api_key": "",
@@ -502,6 +579,44 @@ class ConfigManager:
                 "quality_profile": "Any",
                 "cleanup_after_import": True,
             },
+            # Prowlarr — indexer aggregator. Feeds the torrent / usenet
+            # download plugins. Not a standalone source.
+            "prowlarr": {
+                "url": "",
+                "api_key": "",
+                # Comma-separated list of indexer IDs to limit searches to.
+                # Empty = search all enabled indexers.
+                "indexer_ids": "",
+            },
+            # Torrent client — receives .torrent / magnet URIs from the
+            # torrent download plugin. ``type`` picks which adapter to
+            # instantiate (qbittorrent | transmission | deluge).
+            "torrent_client": {
+                "type": "qbittorrent",
+                "url": "",
+                "username": "",
+                "password": "",
+                "category": "soulsync",
+                "save_path": "",
+            },
+            # Usenet client — receives .nzb URLs / payloads. ``type``
+            # picks the adapter (sabnzbd | nzbget). SABnzbd uses an
+            # API key; NZBGet uses username + password.
+            "usenet_client": {
+                "type": "sabnzbd",
+                "url": "",
+                "api_key": "",
+                "username": "",
+                "password": "",
+                "category": "soulsync",
+            },
+            "soundcloud_download": {
+                # Anonymous-only for now — SoundCloud Go+ OAuth tier could be
+                # added later, with credentials living under a "session" subkey
+                # alongside Tidal/Qobuz. No quality knob: anonymous SoundCloud
+                # caps at the upload's transcoding (typically 128 kbps MP3 or
+                # AAC). yt-dlp resolves bestaudio at download time.
+            },
             "listenbrainz": {
                 "base_url": "",
                 "token": "",
@@ -528,16 +643,43 @@ class ConfigManager:
                 "path": os.environ.get('DATABASE_PATH', 'database/music_library.db'),
                 "max_workers": 5
             },
+            "image_cache": {
+                "enabled": True,
+                "path": "storage/image_cache",
+                "ttl_seconds": 2592000,
+                "failed_ttl_seconds": 21600,
+                "max_download_mb": 15
+            },
             "metadata_enhancement": {
                 "enabled": True,
                 "embed_album_art": True,
-                "post_process_order": ["musicbrainz", "deezer", "audiodb", "tidal", "qobuz", "lastfm", "genius"]
+                "post_process_order": ["musicbrainz", "deezer", "audiodb", "tidal", "qobuz", "lastfm", "genius"],
+                # Ordered preferred cover-art sources (empty = use the
+                # download's own art, i.e. today's behavior). Resolved + walked
+                # with fallback by core/metadata/art_sources.py.
+                "album_art_order": [],
+                # Minimum cover-art resolution (shortest side, px). A preferred
+                # source whose art is smaller is skipped so the next source is
+                # tried — stops a low-res Cover Art Archive upload from winning.
+                # 0 disables the size gate.
+                "min_art_size": 1000,
+                # When a track matches a SINGLE release, look up the parent ALBUM
+                # that contains it and tag it as that album, so it groups with its
+                # album-mates and gets the album cover (not the single's). Off by
+                # default — it's an extra per-import metadata lookup.
+                "single_to_album": False
             },
             "musicbrainz": {
                 "embed_tags": True
             },
             "playlist_sync": {
-                "create_backup": True
+                "create_backup": True,
+                # How a re-sync writes to the server playlist:
+                #   replace   — delete + recreate (default; today's behavior)
+                #   reconcile — edit in place (add/remove delta), preserving the
+                #               playlist's custom image, description, and identity (#792)
+                #   append    — only add new tracks, never remove
+                "mode": "replace"
             },
             "settings": {
                 "audio_quality": "flac"
@@ -563,12 +705,35 @@ class ConfigManager:
             },
             "import": {
                 "staging_path": "./Staging",
-                "replace_lower_quality": False
+                # Master toggle for quality-filtering on import. On by default:
+                # downloaded files that don't meet the quality profile are
+                # quarantined instead of imported (same gate the download
+                # pipeline uses). Off → import everything regardless of quality;
+                # the library Quality Upgrade Scanner still flags them.
+                "quality_filter_enabled": True,
+                "replace_lower_quality": False,
+                # Use the top Staging folder as the artist (Artist/Album layouts,
+                # mixtapes). On by default to preserve the long-standing import
+                # behaviour for existing users. Turn OFF if you stage a mixed pile
+                # of songs under one container folder, otherwise that folder's name
+                # overrides every metadata-identified artist (the "soulsync" case).
+                "folder_artist_override": True
             },
             "m3u_export": {
                 "enabled": False,
                 "entry_base_path": ""
             },
+            "playlists": {
+                # Where "Organize by playlist" materializes playlist folders.
+                # MUST be a separate root from the music library so the media
+                # server (and the maintenance jobs) never scan it — otherwise the
+                # same track would show up twice. Mapped separately for Docker.
+                "materialize_path": "./Playlists",
+                # "symlink" (relative links, ~zero disk) or "copy" (real
+                # duplicates for FAT/USB/DAPs that can't follow links). Symlink
+                # auto-falls back to copy when the filesystem can't link.
+                "materialize_mode": "symlink"
+            },
             "youtube": {
                 "cookies_browser": "",      # "", "chrome", "firefox", "edge", "brave", "opera", "safari"
                 "download_delay": 3,        # seconds between sequential downloads
@@ -677,7 +842,40 @@ class ConfigManager:
 
         return value
 
+    def redacted_config(self) -> Dict[str, Any]:
+        """Deep copy of the live config with every sensitive value masked.
+
+        Used for ``GET /api/settings`` so decrypted secrets never reach the
+        browser (#832 follow-up). A *set* secret becomes ``REDACTED_SENTINEL``
+        (the UI renders it as masked dots); an unset one stays empty so the UI
+        can show "not configured". Dict-valued secrets (OAuth sessions) collapse
+        to the sentinel too — the UI has no field for them anyway. The matching
+        guard in ``set()`` turns a round-tripped sentinel back into a no-op.
+        """
+        import copy
+        data = copy.deepcopy(self.config_data)
+        for path in self._SENSITIVE_PATHS:
+            keys = path.split('.')
+            parent = data
+            for k in keys[:-1]:
+                if isinstance(parent, dict) and k in parent:
+                    parent = parent[k]
+                else:
+                    parent = None
+                    break
+            if not isinstance(parent, dict):
+                continue
+            leaf = keys[-1]
+            if leaf in parent and parent[leaf] not in (None, '', {}, [], 0, False):
+                parent[leaf] = self.REDACTED_SENTINEL
+        return data
+
     def set(self, key: str, value: Any):
+        # The UI round-trips REDACTED_SENTINEL for any secret the user didn't
+        # touch — never let the mask overwrite the real value (#832 follow-up).
+        if value == self.REDACTED_SENTINEL and key in self._SENSITIVE_PATHS:
+            return
+
         keys = key.split('.')
         config = self.config_data
 
@@ -689,6 +887,20 @@ class ConfigManager:
         config[keys[-1]] = value
         self._save_config()
 
+    def resolve_secret(self, key: str, posted: Any) -> str:
+        """Resolve a secret value coming back from the settings UI.
+
+        The UI renders a saved-but-untouched secret as the REDACTED_SENTINEL (shown
+        masked); empty or that sentinel means "use the stored value", while a real
+        string is a genuine new secret. A connection-test endpoint should test the
+        EFFECTIVE secret, not the mask — otherwise testing a saved-but-untouched
+        token sends the sentinel and the source rejects it (#870)."""
+        if isinstance(posted, str):
+            posted = posted.strip()
+        if not posted or posted == self.REDACTED_SENTINEL:
+            return self.get(key, '') or ''
+        return posted
+
     def get_spotify_config(self) -> Dict[str, str]:
         return self.get('spotify', {})
 

core/acoustid_client.py

@@ -282,8 +282,9 @@ class AcoustIDClient:
 
     def test_api_key(self) -> Tuple[bool, str]:
         """
-        Validate the API key by fingerprinting a real audio file and looking it up.
-        Falls back to a direct API call if no audio files are available.
+        Validate the API key with a direct AcoustID lookup call. An invalid key
+        is reported as invalid (error code 4); any other error means the key was
+        accepted.
 
         Returns:
             Tuple of (success, message)
@@ -294,24 +295,12 @@ class AcoustIDClient:
         import requests
 
         try:
-            # Try to find a real audio file to fingerprint for an end-to-end test
-            test_file = self._find_test_audio_file()
-
-            if test_file and CHROMAPRINT_AVAILABLE:
-                logger.info(f"Testing API key with real audio file: {test_file}")
-                try:
-                    result = self.fingerprint_and_lookup(test_file)
-                    # If we get here without exception, the API key is valid
-                    # (invalid keys raise or return error before results)
-                    return True, "AcoustID API key is valid"
-                except Exception as e:
-                    error_str = str(e).lower()
-                    if 'invalid' in error_str and 'api' in error_str:
-                        return False, "Invalid AcoustID API key - get one from https://acoustid.org/new-application"
-                    # Fingerprint/lookup failed for non-key reasons, fall through to direct test
-                    logger.warning(f"Real file test failed ({e}), trying direct API call")
-
-            # Fallback: direct API call with minimal fingerprint
+            # Authoritative key check: a direct API lookup with a dummy
+            # fingerprint. AcoustID validates the client key first, so an
+            # invalid key returns error code 4 regardless of the fingerprint.
+            # (The previous real-file path trusted "no exception = valid", but
+            # fingerprint_and_lookup swallows the invalid-key error and returns
+            # None — so it reported broken keys as valid. #756-adjacent.)
             url = 'https://api.acoustid.org/v2/lookup'
             params = {
                 'client': self.api_key,
@@ -326,7 +315,6 @@ class AcoustIDClient:
             if data.get('status') == 'error':
                 error = data.get('error', {})
                 error_code = error.get('code', 0)
-                error_msg = error.get('message', 'Unknown error')
 
                 # Error code 4 is specifically "invalid API key"
                 if error_code == 4:
@@ -346,33 +334,33 @@ class AcoustIDClient:
             logger.error(f"Error testing AcoustID API key: {e}")
             return False, f"Error: {str(e)}"
 
-    def fingerprint_and_lookup(self, audio_file: str) -> Optional[Dict[str, Any]]:
-        """
-        Generate fingerprint and look up recording in AcoustID.
+    def lookup_with_status(self, audio_file: str) -> Dict[str, Any]:
+        """Fingerprint + AcoustID lookup returning a STRUCTURED result.
 
-        This is the main method - combines fingerprinting and lookup in one call.
+        Unlike fingerprint_and_lookup() (which collapses every outcome into
+        dict-or-None), this distinguishes a genuine no-match from an actual
+        error — an invalid API key, rate limit, missing chromaprint, or a
+        fingerprint failure. That distinction is what lets the UI show "AcoustID
+        Error" (something is broken — fix it) instead of a benign-looking
+        "Skipped" that silently hides a dead key.
 
-        Args:
-            audio_file: Path to the audio file
-
-        Returns:
-            Dict with:
-                'recordings': list of dicts with 'mbid', 'title', 'artist', 'score'
-                'best_score': float (highest score across all results)
-                'recording_mbids': list of unique MBIDs (for backward compat)
-            Or None on error.
+        Returns dict with:
+            'status':   'ok' | 'no_match' | 'error' | 'no_backend'
+                        | 'fingerprint_error' | 'unsupported' | 'unavailable'
+                        | 'not_found'
+            'recordings': list (meaningful only for 'ok')
+            'best_score': float
+            'recording_mbids': list
+            'error':    human-readable detail for any non-'ok' status
+            'invalid_key': bool (True when the API specifically rejected the key)
         """
         if not ACOUSTID_AVAILABLE:
-            logger.debug("Cannot lookup: pyacoustid not available")
-            return None
-
+            return {'status': 'unavailable', 'recordings': [], 'error': 'pyacoustid library not installed'}
         if not self.api_key:
-            logger.debug("Cannot lookup: no API key")
-            return None
-
+            return {'status': 'unavailable', 'recordings': [], 'error': 'No AcoustID API key configured'}
         if not os.path.isfile(audio_file):
             logger.warning(f"Cannot lookup: file not found: {audio_file}")
-            return None
+            return {'status': 'not_found', 'recordings': [], 'error': f'File not found: {audio_file}'}
 
         # Check channel count — chromaprint crashes (SIGABRT) on >2 channel files (e.g. 5.1 surround)
         try:
@@ -382,7 +370,8 @@ class AcoustIDClient:
                 channels = getattr(mf.info, 'channels', 2)
                 if channels and channels > 2:
                     logger.warning(f"Skipping AcoustID: file has {channels} channels (surround audio): {audio_file}")
-                    return None
+                    return {'status': 'unsupported', 'recordings': [],
+                            'error': f'{channels}-channel (surround) audio not supported by chromaprint'}
         except Exception as e:
             logger.debug(f"Could not check channel count, proceeding anyway: {e}")
 
@@ -392,17 +381,12 @@ class AcoustIDClient:
             api_key_preview = f"{self.api_key[:8]}..." if self.api_key and len(self.api_key) > 8 else "NOT SET"
             logger.info(f"Fingerprinting and looking up: {audio_file} (API key: {api_key_preview})")
 
-            # Use match() which handles fingerprinting + lookup + parsing
             logger.debug("Running acoustid.match()...")
             recordings = []
             seen_mbids = set()
             best_score = 0.0
 
-            for result in acoustid.match(
-                self.api_key,
-                audio_file,
-                parse=True
-            ):
+            for result in acoustid.match(self.api_key, audio_file, parse=True):
                 # match() with parse=True returns (score, recording_id, title, artist)
                 if not isinstance(result, tuple) or len(result) < 2:
                     logger.warning(f"Unexpected result format: {result}")
@@ -420,45 +404,57 @@ class AcoustIDClient:
 
                 if recording_id and recording_id not in seen_mbids:
                     seen_mbids.add(recording_id)
-                    recordings.append({
-                        'mbid': recording_id,
-                        'title': title,
-                        'artist': artist,
-                        'score': score,
-                    })
+                    recordings.append({'mbid': recording_id, 'title': title, 'artist': artist, 'score': score})
                     logger.debug(f"Found match: {title} by {artist} (MBID: {recording_id}, score: {score})")
 
             if not recordings:
                 logger.info(f"No AcoustID matches found for: {audio_file}")
-                return None
+                return {'status': 'no_match', 'recordings': [], 'best_score': best_score,
+                        'recording_mbids': [], 'error': 'Track not found in AcoustID database'}
 
             logger.info(f"AcoustID found {len(recordings)} recording(s) (best score: {best_score:.2f})")
-            return {
-                'recordings': recordings,
-                'best_score': best_score,
-                'recording_mbids': list(seen_mbids),
-            }
+            return {'status': 'ok', 'recordings': recordings, 'best_score': best_score,
+                    'recording_mbids': list(seen_mbids)}
 
         except acoustid.NoBackendError:
             logger.error("Chromaprint library not found and fpcalc not available")
-            return None
+            return {'status': 'no_backend', 'recordings': [],
+                    'error': 'Chromaprint/fpcalc not installed (install libchromaprint1)'}
         except acoustid.FingerprintGenerationError as e:
             logger.warning(f"Failed to fingerprint {audio_file}: {e}")
-            return None
+            return {'status': 'fingerprint_error', 'recordings': [], 'error': f'Could not fingerprint file: {e}'}
         except acoustid.WebServiceError as e:
-            # Log more details about the API error
             api_key_preview = f"{self.api_key[:8]}..." if self.api_key and len(self.api_key) > 8 else "???"
             logger.warning(f"AcoustID API error (key: {api_key_preview}): {e}")
-            # Check for common errors
             error_str = str(e).lower()
-            if 'invalid' in error_str or 'unknown' in error_str:
-                logger.error("API key appears to be invalid - check your AcoustID settings")
+            # Old pyacoustid reports an invalid key as the bare "status: error"
+            # (it drops the detail), so treat that as an invalid-key signal too.
+            invalid = ('invalid' in error_str or 'unknown' in error_str or 'status: error' in error_str)
+            if invalid:
+                logger.error("AcoustID API key appears to be invalid — check your AcoustID settings")
             elif 'rate' in error_str or 'limit' in error_str:
-                logger.warning("Rate limited by AcoustID - will retry later")
-            return None
+                logger.warning("Rate limited by AcoustID — will retry later")
+            return {'status': 'error', 'recordings': [], 'invalid_key': invalid,
+                    'error': f'AcoustID API error: {e}'}
         except Exception as e:
             logger.error(f"Unexpected error in AcoustID lookup: {e}", exc_info=True)
-            return None
+            return {'status': 'error', 'recordings': [], 'error': f'Unexpected error: {e}'}
+
+    def fingerprint_and_lookup(self, audio_file: str) -> Optional[Dict[str, Any]]:
+        """Legacy dict-or-None lookup. Returns the recordings dict on a confirmed
+        match, else None. Kept for callers that only need "did we identify it"
+        (library scanner, auto-import). Callers that must report WHY a lookup
+        didn't match (verification badge, key test) should use
+        ``lookup_with_status`` so an error isn't mistaken for a no-match.
+        """
+        res = self.lookup_with_status(audio_file)
+        if res.get('status') == 'ok':
+            return {
+                'recordings': res['recordings'],
+                'best_score': res.get('best_score', 0.0),
+                'recording_mbids': res.get('recording_mbids', []),
+            }
+        return None
 
     def refresh_config(self):
         """Refresh cached config values (call after settings change)."""

core/acoustid_verification.py

@@ -15,96 +15,82 @@ from typing import Optional, Dict, Any, Tuple, List
 from enum import Enum
 from utils.logging_config import get_logger
 from core.acoustid_client import AcoustIDClient
+from core.matching_engine import MusicMatchingEngine
+from core.matching.version_mismatch import is_acceptable_version_mismatch
+from core.matching.script_compat import is_cross_script_mismatch
 from core.musicbrainz_client import MusicBrainzClient
 
 logger = get_logger("acoustid.verification")
 
-# Thresholds
-MIN_ACOUSTID_SCORE = 0.80       # Minimum AcoustID fingerprint score to trust
-TITLE_MATCH_THRESHOLD = 0.70    # Title similarity needed to consider a match
-ARTIST_MATCH_THRESHOLD = 0.60   # Artist similarity needed to consider a match
+# Thresholds — single definition lives in the shared core; re-exported here so
+# existing importers keep working and the values can't drift between paths.
+from core.matching.audio_verification import (  # noqa: E402
+    MIN_ACOUSTID_SCORE, TITLE_MATCH_THRESHOLD, ARTIST_MATCH_THRESHOLD,
+)
+
+# Single matching-engine instance so version detection reuses the same patterns
+# used by the pre-download Soulseek matcher (remix / live / acoustic /
+# instrumental / etc). detect_version_type doesn't use self state, so one
+# shared instance is fine.
+_match_engine_for_version = MusicMatchingEngine()
+
+
+def _detect_title_version(title: str) -> str:
+    """Return version label for a track title.
+
+    Returns ``'original'`` when no version marker is detected, otherwise one
+    of the labels produced by ``MusicMatchingEngine.detect_version_type``
+    (``'instrumental'``, ``'live'``, ``'acoustic'``, ``'remix'``, etc).
+    """
+    if not title:
+        return 'original'
+    version_type, _ = _match_engine_for_version.detect_version_type(title)
+    return version_type
 
 
 class VerificationResult(Enum):
     """Possible outcomes of audio verification."""
     PASS = "pass"       # Title/artist match - file is correct
     FAIL = "fail"       # Title/artist mismatch - wrong file downloaded
-    SKIP = "skip"       # Could not verify (error or unavailable) - continue normally
+    SKIP = "skip"       # Genuinely couldn't verify (no match in DB) - continue normally
     DISABLED = "disabled"  # Verification not enabled
+    ERROR = "error"     # Lookup errored (invalid key / rate limit / no backend) - continue, but flag it
 
 
-def _normalize(text: str) -> str:
-    """Normalize a string for comparison: lowercase, strip parentheticals, punctuation."""
-    if not text:
-        return ""
-    s = text.lower().strip()
-    # Remove ALL parenthetical suffixes — these are metadata annotations, not core title
-    # Covers: (Live), (Remastered), (Parody of ...), (from "..." Soundtrack), (feat. ...), etc.
-    s = re.sub(r'\s*\([^)]*\)', '', s)
-    # Remove ALL square bracket suffixes: [Live], [Remastered], [Deluxe], etc.
-    s = re.sub(r'\s*\[[^\]]*\]', '', s)
-    # Remove trailing featuring info not in parentheses: "feat. ...", "ft. ...", "featuring ..."
-    s = re.sub(r'\s+(?:feat\.?|ft\.?|featuring)\s+.*$', '', s, flags=re.IGNORECASE)
-    # Remove dash-separated version tags: "- Vocal", "- Instrumental", "- Acoustic", etc.
-    s = re.sub(r'\s*-\s*(?:vocal|instrumental|acoustic|live|remix|cover|clean|explicit|radio\s*edit|original\s*mix|extended\s*mix|club\s*mix)\s*$', '', s, flags=re.IGNORECASE)
-    # Remove soundtrack/source subtitles: ' - From "..." Soundtrack', ' - from the film ...'
-    s = re.sub(r'\s*-\s*from\s+.+$', '', s, flags=re.IGNORECASE)
-    # Remove non-alphanumeric except spaces
-    s = re.sub(r'[^\w\s]', '', s)
-    # Collapse whitespace
-    s = re.sub(r'\s+', ' ', s).strip()
-    return s
+# normalize() + similarity() + the alias-aware comparison now live in the shared
+# decision core (core/matching/audio_verification.py) so import-time verification
+# and the library scan share ONE definition — the <>-strip fix, CJK handling and
+# thresholds can't drift apart again. Names kept (`_normalize` etc.) for existing
+# importers/tests.
+from core.matching.audio_verification import (  # noqa: E402
+    normalize as _normalize,
+    similarity as _similarity,
+    _alias_aware_artist_sim,
+    _find_best_title_artist_match as _core_find_best_title_artist_match,
+    evaluate as _core_evaluate,
+    Decision as _CoreDecision,
+)
 
 
-def _similarity(a: str, b: str) -> float:
-    """Calculate similarity between two strings (0.0-1.0) after normalization."""
-    na = _normalize(a)
-    nb = _normalize(b)
-    if not na or not nb:
-        return 0.0
-    if na == nb:
-        return 1.0
-    return SequenceMatcher(None, na, nb).ratio()
-
-
-def _find_best_title_artist_match(
-    recordings: List[Dict[str, Any]],
-    expected_title: str,
-    expected_artist: str,
-) -> Tuple[Optional[Dict], float, float]:
-    """
-    Find the AcoustID recording that best matches expected title/artist.
-
-    Returns:
-        (best_recording, title_similarity, artist_similarity)
-    """
-    best_rec = None
-    best_title_sim = 0.0
-    best_artist_sim = 0.0
-    best_combined = 0.0
-
-    for rec in recordings:
-        title = rec.get('title') or ''
-        artist = rec.get('artist') or ''
-
-        title_sim = _similarity(expected_title, title)
-        artist_sim = _similarity(expected_artist, artist)
-        # Weight title higher since that's the primary identifier
-        combined = (title_sim * 0.6) + (artist_sim * 0.4)
-
-        if combined > best_combined:
-            best_combined = combined
-            best_rec = rec
-            best_title_sim = title_sim
-            best_artist_sim = artist_sim
-
-    return best_rec, best_title_sim, best_artist_sim
+def _find_best_title_artist_match(recordings, expected_title, expected_artist,
+                                  expected_artist_aliases=None):
+    """Back-compat wrapper around the shared core matcher (keeps the
+    ``expected_artist_aliases`` kwarg name for existing callers/tests)."""
+    return _core_find_best_title_artist_match(
+        recordings, expected_title, expected_artist, expected_artist_aliases,
+    )
 
 
 # Shared MusicBrainz client for enrichment lookups
 _mb_client = None
 _mb_client_lock = threading.Lock()
 
+# Shared MusicBrainzService for alias lookups (issue #442). Service
+# layer wraps the raw client + adds caching + DB access — all of which
+# the alias resolution chain (library DB → cache → live MB) needs.
+_mb_service = None
+_mb_service_lock = threading.Lock()
+
 MAX_MB_ENRICHMENT_LOOKUPS = 3
 
 
@@ -118,6 +104,42 @@ def _get_mb_client() -> MusicBrainzClient:
     return _mb_client
 
 
+def _get_mb_service():
+    """Get or create a shared MusicBrainzService instance.
+
+    Used by the alias-resolution chain in `verify_audio_file`. Lazy
+    init so importing this module doesn't trigger a DB connection on
+    paths that never run AcoustID verification (test runs, dry runs).
+    """
+    global _mb_service
+    if _mb_service is None:
+        with _mb_service_lock:
+            if _mb_service is None:
+                from core.musicbrainz_service import MusicBrainzService
+                from database.music_database import get_database
+                _mb_service = MusicBrainzService(get_database())
+    return _mb_service
+
+
+def _resolve_expected_artist_aliases(expected_artist_name: str) -> List[str]:
+    """Look up alternate-spelling aliases for the expected artist.
+
+    Issue #442 — bridges cross-script artist comparisons (Japanese
+    kanji ↔ romanized, Cyrillic ↔ Latin, etc.) without forcing the
+    verifier to know about the resolution chain. Best-effort: any
+    failure (no MB service, network down, no library DB) returns
+    empty list so verification falls back to the prior direct
+    similarity check.
+    """
+    if not expected_artist_name:
+        return []
+    try:
+        return _get_mb_service().lookup_artist_aliases(expected_artist_name)
+    except Exception as e:
+        logger.debug("alias lookup failed for %r: %s", expected_artist_name, e)
+        return []
+
+
 def _enrich_recordings_from_musicbrainz(
     recordings: List[Dict[str, Any]],
 ) -> List[Dict[str, Any]]:
@@ -243,18 +265,33 @@ class AcoustIDVerification:
                 logger.debug(f"AcoustID verification skipped: {reason}")
                 return VerificationResult.SKIP, reason
 
-            # Step 2: Fingerprint and lookup in AcoustID
+            # Step 2: Fingerprint and lookup in AcoustID (structured so an
+            # actual error — invalid key / rate limit / no chromaprint — is
+            # reported distinctly from a genuine no-match, instead of both
+            # silently surfacing as "Skipped").
             logger.info(f"Fingerprinting and looking up: {audio_file_path}")
-            acoustid_result = self.acoustid_client.fingerprint_and_lookup(audio_file_path)
+            lookup = self.acoustid_client.lookup_with_status(audio_file_path) or {}
+            status = lookup.get('status')
+            # Infer status by content when absent (a caller/stub that returned
+            # just recordings): recordings => matched, none => no match.
+            if status is None:
+                status = 'ok' if lookup.get('recordings') else 'no_match'
 
-            if not acoustid_result:
-                return VerificationResult.SKIP, "Track not found in AcoustID database"
+            if status in ('error', 'no_backend', 'fingerprint_error', 'unavailable'):
+                # Something is broken (not the track's fault) — never quarantine
+                # on this; surface it so the user can fix it.
+                return VerificationResult.ERROR, lookup.get('error', 'AcoustID lookup failed')
 
+            if status != 'ok':
+                # no_match / unsupported / not_found — genuinely could not verify.
+                return VerificationResult.SKIP, lookup.get('error', 'No match in AcoustID database')
+
+            acoustid_result = lookup
             recordings = acoustid_result.get('recordings', [])
             best_score = acoustid_result.get('best_score', 0)
 
             if not recordings:
-                return VerificationResult.SKIP, "AcoustID returned no recordings"
+                return VerificationResult.SKIP, "No match in AcoustID database"
 
             logger.debug(
                 f"AcoustID returned {len(recordings)} recording(s) "
@@ -270,120 +307,55 @@ class AcoustIDVerification:
             # Enrich recordings that are missing title/artist via MusicBrainz lookup
             recordings = _enrich_recordings_from_musicbrainz(recordings)
 
-            # Step 4: Find best title/artist match among AcoustID results
-            best_rec, title_sim, artist_sim = _find_best_title_artist_match(
-                recordings, expected_track_name, expected_artist_name
-            )
+            # Issue #442 — alias resolution is LAZY. We pass a memoising
+            # thunk to the artist-comparison sites; it only fires the
+            # multi-tier lookup (library DB → cache → live MB) when
+            # direct artist similarity falls below threshold. Verifications
+            # where the direct match already passes (the common case for
+            # same-script artist names) never trigger any lookup work,
+            # so the fix doesn't add a per-verification DB query for the
+            # happy path. When the thunk DOES fire, the result is cached
+            # in the closure so the 3 comparison sites within one
+            # verification share a single resolution pass.
+            _alias_cache: Dict[str, Any] = {}
 
-            if not best_rec:
-                return VerificationResult.SKIP, "No recordings with title/artist info"
-
-            matched_title = best_rec.get('title', '?')
-            matched_artist = best_rec.get('artist', '?')
-
-            logger.info(
-                f"Best match: '{matched_title}' by '{matched_artist}' "
-                f"(title_sim={title_sim:.2f}, artist_sim={artist_sim:.2f})"
-            )
-
-            # Step 5: Decide pass/fail based on similarity
-            if title_sim >= TITLE_MATCH_THRESHOLD and artist_sim >= ARTIST_MATCH_THRESHOLD:
-                msg = (
-                    f"Audio verified: '{matched_title}' by '{matched_artist}' "
-                    f"matches expected '{expected_track_name}' by '{expected_artist_name}' "
-                    f"(title={title_sim:.0%}, artist={artist_sim:.0%})"
-                )
-                logger.info(f"AcoustID verification PASSED - {msg}")
-                return VerificationResult.PASS, msg
-
-            # Title matches but artist doesn't — could be a cover/collab OR a
-            # genuinely different track with the same name. Distinguish the
-            # two by checking whether the expected artist appears anywhere in
-            # AcoustID's returned recordings.
-            if title_sim >= TITLE_MATCH_THRESHOLD and artist_sim < ARTIST_MATCH_THRESHOLD:
-                # First: if the expected artist is present in ANY recording's
-                # metadata for this fingerprint, it's likely the right track
-                # (AcoustID's "best" match just picked the wrong variant).
-                for rec in recordings:
-                    if _similarity(expected_artist_name, rec.get('artist', '')) >= ARTIST_MATCH_THRESHOLD:
-                        msg = (
-                            f"Audio verified: found '{expected_track_name}' by '{expected_artist_name}' "
-                            f"in AcoustID results"
+            def _aliases_provider() -> List[str]:
+                if 'value' not in _alias_cache:
+                    resolved = _resolve_expected_artist_aliases(expected_artist_name)
+                    _alias_cache['value'] = resolved
+                    if resolved:
+                        logger.debug(
+                            "Resolved %d aliases for expected artist '%s'",
+                            len(resolved), expected_artist_name,
                         )
-                        logger.info(f"AcoustID verification PASSED (secondary match) - {msg}")
-                        return VerificationResult.PASS, msg
+                return _alias_cache['value']
 
-                # Expected artist wasn't found anywhere. Decide between:
-                #   - FAIL: clear mismatch, e.g. "Tom Walker" (sim ~0.2) when
-                #     expecting "Maduk" — different song with same name
-                #   - SKIP: ambiguous, e.g. collab / alt credit / formatting
-                #     difference (sim 0.3-0.6)
-                #
-                # The 0.3 cutoff catches hard mismatches while preserving the
-                # benefit of the doubt for borderline artist formatting.
-                CLEAR_MISMATCH_THRESHOLD = 0.3
-                if artist_sim < CLEAR_MISMATCH_THRESHOLD:
-                    msg = (
-                        f"Audio mismatch: file identified as '{matched_title}' by '{matched_artist}', "
-                        f"expected '{expected_track_name}' by '{expected_artist_name}' "
-                        f"(title={title_sim:.0%}, artist={artist_sim:.0%}) — "
-                        f"expected artist not found in any AcoustID recording"
-                    )
-                    logger.warning(f"AcoustID verification FAILED (clear artist mismatch) - {msg}")
-                    return VerificationResult.FAIL, msg
-
-                msg = (
-                    f"Title matches but artist unclear: "
-                    f"AcoustID='{matched_title}' by '{matched_artist}', "
-                    f"expected '{expected_track_name}' by '{expected_artist_name}' "
-                    f"(artist_sim={artist_sim:.0%} — ambiguous, could be cover/collab)"
-                )
-                logger.info(f"AcoustID verification SKIPPED - {msg}")
-                return VerificationResult.SKIP, msg
-
-            # Title doesn't match — check ALL recordings for any title/artist match
-            # (the best combined match might not be the right one if there are many results)
-            for rec in recordings:
-                t = rec.get('title') or ''
-                a = rec.get('artist') or ''
-                if (_similarity(expected_track_name, t) >= TITLE_MATCH_THRESHOLD and
-                        _similarity(expected_artist_name, a) >= ARTIST_MATCH_THRESHOLD):
-                    msg = (
-                        f"Audio verified: found '{t}' by '{a}' in AcoustID results "
-                        f"matching expected '{expected_track_name}' by '{expected_artist_name}'"
-                    )
-                    logger.info(f"AcoustID verification PASSED (scan match) - {msg}")
-                    return VerificationResult.PASS, msg
-
-            # No match found — but if fingerprint score is very high (≥0.95)
-            # AND there's partial similarity in title or artist, the mismatch is
-            # likely a language/script difference (e.g. Japanese kanji vs English).
-            # Skip rather than quarantine a correct file.
-            # But if both title AND artist similarity are very low, the download
-            # source gave us a completely wrong file — fail it.
-            if best_score >= 0.95 and (title_sim >= 0.55 or artist_sim >= ARTIST_MATCH_THRESHOLD):
-                top = recordings[0]
-                msg = (
-                    f"Title/artist mismatch but fingerprint confidence very high ({best_score:.2f}): "
-                    f"AcoustID='{top.get('title', '?')}' by '{top.get('artist', '?')}', "
-                    f"expected '{expected_track_name}' by '{expected_artist_name}' — "
-                    f"likely same song in different language/script"
-                )
-                logger.info(f"AcoustID verification SKIPPED (high confidence) - {msg}")
-                return VerificationResult.SKIP, msg
-
-            # Low fingerprint score + no metadata match — file is likely wrong
-            top = recordings[0]
-            top_title = top.get('title', '?')
-            top_artist = top.get('artist', '?')
-
-            msg = (
-                f"Audio mismatch: file identified as '{top_title}' by '{top_artist}', "
-                f"expected '{expected_track_name}' by '{expected_artist_name}' "
-                f"(title={title_sim:.0%}, artist={artist_sim:.0%})"
+            # Steps 4-5: delegate the PASS/SKIP/FAIL decision to the shared core
+            # (core/matching/audio_verification.evaluate) so import verification
+            # and the library scan apply identical logic.
+            outcome = _core_evaluate(
+                expected_track_name, expected_artist_name, recordings,
+                fingerprint_score=best_score,
+                aliases_provider=_aliases_provider,
             )
-            logger.warning(f"AcoustID verification FAILED - {msg}")
-            return VerificationResult.FAIL, msg
+            logger.info(
+                "Best match: '%s' by '%s' (title_sim=%.2f, artist_sim=%.2f) -> %s",
+                outcome.matched_title, outcome.matched_artist,
+                outcome.title_sim, outcome.artist_sim, outcome.decision.value,
+            )
+            _decision_map = {
+                _CoreDecision.PASS: VerificationResult.PASS,
+                _CoreDecision.SKIP: VerificationResult.SKIP,
+                _CoreDecision.FAIL: VerificationResult.FAIL,
+            }
+            result = _decision_map[outcome.decision]
+            if result == VerificationResult.PASS:
+                logger.info("AcoustID verification PASSED - %s", outcome.reason)
+            elif result == VerificationResult.FAIL:
+                logger.warning("AcoustID verification FAILED - %s", outcome.reason)
+            else:
+                logger.info("AcoustID verification SKIPPED - %s", outcome.reason)
+            return result, outcome.reason
 
         except Exception as e:
             # Any unexpected error -> SKIP (fail open)

core/album_consistency.py

@@ -195,8 +195,8 @@ def _find_best_release(album_name, artist_name, track_count, mb_service):
                     sr_id = sr.get('id', '')
                     if sr_id and sr_id not in candidate_mbids:
                         candidate_mbids.append(sr_id)
-            except Exception:
-                pass
+            except Exception as e:
+                logger.debug("search_release fallback failed: %s", e)
 
         if not candidate_mbids:
             logger.info(f"No MB release found for '{album_name}' by '{artist_name}'")

core/amazon_client.py

@@ -0,0 +1,796 @@
+"""Amazon Music metadata client backed by a T2Tunes proxy instance.
+
+T2Tunes exposes the Amazon Music catalog (search, album metadata, stream info)
+through a simple REST API. This module wraps those endpoints and normalises the
+responses into the same Track / Artist / Album dataclass shape used by
+DeezerClient and iTunesClient.
+
+Endpoints used:
+    GET /api/status
+    GET /api/amazon-music/search
+    GET /api/amazon-music/metadata
+    GET /api/amazon-music/media-from-asin
+
+Config keys (all optional — fall back to public defaults):
+    amazon.base_url        T2Tunes instance URL  (default: https://t2tunes.site)
+    amazon.country         ISO-3166 country code (default: US)
+    amazon.preferred_codec Preferred audio codec (default: flac)
+"""
+
+from __future__ import annotations
+
+import re
+import threading
+import time
+from dataclasses import dataclass
+from typing import Any, Dict, Iterator, List, Optional
+from urllib.parse import urljoin
+
+import requests
+
+from config.settings import config_manager
+from core.api_call_tracker import api_call_tracker
+from utils.logging_config import get_logger
+
+logger = get_logger("amazon_client")
+
+# Strips featuring credits like "Artist feat. X", "Artist ft. Y" so artist
+# deduplication works on the primary artist name only.
+_FEAT_RE = re.compile(r'\s+(?:feat(?:uring)?\.?|ft\.?)\s+.*', re.IGNORECASE)
+
+# Strips the Explicit marker — explicit is treated as the default version.
+# Clean/Edited/Censored stay in the name so users can distinguish them.
+_EDITION_RE = re.compile(r'\s*[\[\(]explicit[\]\)]', re.IGNORECASE)
+
+
+def _primary_artist(name: str) -> str:
+    return _FEAT_RE.sub('', name).strip()
+
+
+def _strip_edition(name: str) -> str:
+    return _EDITION_RE.sub('', name).strip()
+
+
+def _unslugify(name: str) -> str:
+    """Convert a slug-form artist ID (e.g. 'kendrick_lamar') to a search name."""
+    return name.replace('_', ' ')
+
+
+DEFAULT_BASE_URL = "https://t2tunes.site"
+DEFAULT_COUNTRY = "US"
+DEFAULT_CODEC = "flac"
+MIN_API_INTERVAL = 0.5  # seconds — T2Tunes has no published rate limit
+
+_last_api_call: float = 0.0
+_api_call_lock = threading.Lock()
+
+_META_CACHE_TTL = 300  # seconds
+_meta_cache: Dict[str, tuple] = {}  # asin -> (fetched_at, meta_dict)
+_meta_cache_lock = threading.Lock()
+
+
+class AmazonClientError(RuntimeError):
+    """Raised on unrecoverable T2Tunes API errors.
+
+    Carries the HTTP ``status_code`` when the failure was an HTTP error, so
+    callers (the worker's outage detection) can tell a source outage (5xx) from
+    a per-item miss without parsing the message.
+    """
+
+    def __init__(self, *args, status_code=None):
+        super().__init__(*args)
+        self.status_code = status_code
+
+
+# ---------------------------------------------------------------------------
+# Dataclasses — field layout matches DeezerClient / iTunesClient exactly
+# ---------------------------------------------------------------------------
+
+@dataclass
+class Track:
+    id: str           # Amazon track ASIN
+    name: str
+    artists: List[str]
+    album: str
+    duration_ms: int
+    popularity: int
+    preview_url: Optional[str] = None
+    external_urls: Optional[Dict[str, str]] = None
+    image_url: Optional[str] = None
+    release_date: Optional[str] = None
+    track_number: Optional[int] = None
+    disc_number: Optional[int] = None
+    album_type: Optional[str] = None
+    total_tracks: Optional[int] = None
+    isrc: Optional[str] = None
+
+    @classmethod
+    def from_search_hit(cls, doc: Dict[str, Any]) -> "Track":
+        return cls(
+            id=str(doc.get("asin") or ""),
+            name=str(doc.get("title") or ""),
+            artists=[str(doc.get("artistName") or "Unknown Artist")],
+            album=str(doc.get("albumName") or ""),
+            duration_ms=int(doc.get("duration") or 0) * 1000,
+            popularity=0,
+            isrc=str(doc.get("isrc") or "") or None,
+        )
+
+    @classmethod
+    def from_stream_info(
+        cls,
+        stream: "T2TunesStreamInfo",
+        album_meta: Optional[Dict[str, Any]] = None,
+    ) -> "Track":
+        album = album_meta or {}
+        return cls(
+            id=stream.asin,
+            name=stream.title,
+            artists=[stream.artist] if stream.artist else ["Unknown Artist"],
+            album=stream.album,
+            duration_ms=0,
+            popularity=0,
+            image_url=album.get("image"),
+            release_date=album.get("release_date"),
+            total_tracks=album.get("trackCount"),
+            isrc=stream.isrc or None,
+        )
+
+
+@dataclass
+class Artist:
+    id: str           # Slugified artist name — T2Tunes exposes no artist IDs
+    name: str
+    popularity: int
+    genres: List[str]
+    followers: int
+    image_url: Optional[str] = None
+    external_urls: Optional[Dict[str, str]] = None
+
+    @classmethod
+    def from_name(cls, name: str) -> "Artist":
+        slug = name.lower().replace(" ", "_")
+        return cls(id=slug, name=name, popularity=0, genres=[], followers=0)
+
+
+@dataclass
+class Album:
+    id: str           # Amazon album ASIN
+    name: str
+    artists: List[str]
+    release_date: str
+    total_tracks: int
+    album_type: str
+    image_url: Optional[str] = None
+    external_urls: Optional[Dict[str, str]] = None
+    explicit: Optional[bool] = None
+
+    @classmethod
+    def from_search_hit(cls, doc: Dict[str, Any]) -> "Album":
+        return cls(
+            id=str(doc.get("albumAsin") or doc.get("asin") or ""),
+            name=str(doc.get("albumName") or doc.get("title") or ""),
+            artists=[str(doc.get("artistName") or "Unknown Artist")],
+            release_date="",
+            total_tracks=0,
+            album_type="album",
+        )
+
+    @classmethod
+    def from_metadata(cls, album_meta: Dict[str, Any], asin: str = "") -> "Album":
+        return cls(
+            id=str(album_meta.get("asin") or asin or ""),
+            name=str(album_meta.get("title") or ""),
+            artists=[str(album_meta.get("artistName") or "Unknown Artist")],
+            release_date=str(album_meta.get("release_date") or album_meta.get("releaseDate") or ""),
+            total_tracks=int(album_meta.get("trackCount") or 0),
+            album_type="album",
+            image_url=album_meta.get("image"),
+            explicit=album_meta.get("explicit"),
+        )
+
+
+# ---------------------------------------------------------------------------
+# Internal dataclasses for raw T2Tunes response parsing
+# ---------------------------------------------------------------------------
+
+@dataclass(frozen=True)
+class T2TunesSearchItem:
+    asin: str
+    title: str
+    artist_name: str
+    item_type: str
+    album_name: str = ""
+    album_asin: str = ""
+    duration_seconds: int = 0
+    isrc: str = ""
+
+    @property
+    def is_album(self) -> bool:
+        return "album" in self.item_type.lower()
+
+    @property
+    def is_track(self) -> bool:
+        return "track" in self.item_type.lower()
+
+
+@dataclass(frozen=True)
+class T2TunesStreamInfo:
+    asin: str
+    streamable: bool
+    codec: str
+    format: str
+    sample_rate: Optional[int]
+    stream_url: str
+    decryption_key: Optional[str]   # hex-encoded AES key; None when stream is clear
+    title: str = ""
+    artist: str = ""
+    album: str = ""
+    isrc: str = ""
+    cover_url: str = ""
+    track_number: Optional[int] = None
+    disc_number: Optional[int] = None
+    genre: str = ""
+    label: str = ""
+    date: str = ""
+
+    @property
+    def has_decryption_key(self) -> bool:
+        return bool(self.decryption_key)
+
+
+# ---------------------------------------------------------------------------
+# Rate-limit enforcement
+# ---------------------------------------------------------------------------
+
+def _rate_limit() -> None:
+    global _last_api_call
+    with _api_call_lock:
+        elapsed = time.monotonic() - _last_api_call
+        if elapsed < MIN_API_INTERVAL:
+            time.sleep(MIN_API_INTERVAL - elapsed)
+        _last_api_call = time.monotonic()
+    api_call_tracker.record_call("amazon")
+
+
+# ---------------------------------------------------------------------------
+# Main client
+# ---------------------------------------------------------------------------
+
+class AmazonClient:
+    """T2Tunes-backed Amazon Music metadata and stream-info client."""
+
+    def __init__(
+        self,
+        base_url: Optional[str] = None,
+        country: Optional[str] = None,
+        preferred_codec: Optional[str] = None,
+        timeout: int = 30,
+        session: Optional[Any] = None,
+    ) -> None:
+        self.base_url = (base_url or config_manager.get("amazon.base_url", DEFAULT_BASE_URL)).rstrip("/")
+        self.country = (country or config_manager.get("amazon.country", DEFAULT_COUNTRY)).upper()
+        self.preferred_codec = (
+            preferred_codec or config_manager.get("amazon.preferred_codec", DEFAULT_CODEC)
+        ).lower()
+        self.timeout = timeout
+        self.session: Any = session or requests.Session()
+        if isinstance(self.session, requests.Session):
+            self.session.headers.update({
+                "Accept": "application/json",
+                "User-Agent": "SoulSync/1.0",
+                "Referer": self.base_url,
+            })
+
+    # ------------------------------------------------------------------
+    # Lifecycle
+    # ------------------------------------------------------------------
+
+    def reload_config(self) -> None:
+        self.base_url = config_manager.get("amazon.base_url", DEFAULT_BASE_URL).rstrip("/")
+        self.country = config_manager.get("amazon.country", DEFAULT_COUNTRY).upper()
+        self.preferred_codec = config_manager.get("amazon.preferred_codec", DEFAULT_CODEC).lower()
+
+    def is_authenticated(self) -> bool:
+        """Return True when the T2Tunes instance reports Amazon Music as up."""
+        try:
+            return str(self.status().get("amazonMusic", "")).lower() == "up"
+        except AmazonClientError:
+            return False
+
+    # ------------------------------------------------------------------
+    # Low-level API wrappers
+    # ------------------------------------------------------------------
+
+    def status(self) -> Dict[str, Any]:
+        return self._get_json("/api/status")
+
+    def search_raw(self, query: str, *, types: str = "track,album") -> List[T2TunesSearchItem]:
+        data = self._get_json(
+            "/api/amazon-music/search",
+            params={"query": query, "types": types, "country": self.country},
+        )
+        return list(self._iter_search_items(data))
+
+    def album_metadata(self, asin: str) -> Dict[str, Any]:
+        return self._get_json(
+            "/api/amazon-music/metadata",
+            params={"asin": asin, "country": self.country},
+        )
+
+    def media_from_asin(self, asin: str, codec: Optional[str] = None) -> List[T2TunesStreamInfo]:
+        effective_codec = (codec or self.preferred_codec).lower()
+        data = self._get_json(
+            "/api/amazon-music/media-from-asin",
+            params={"asin": asin, "country": self.country, "codec": effective_codec},
+        )
+        if isinstance(data, list):
+            return [self._parse_stream_info(item) for item in data if isinstance(item, dict)]
+        if isinstance(data, dict):
+            return [self._parse_stream_info(data)]
+        raise AmazonClientError(f"Unexpected media-from-asin response type: {type(data).__name__}")
+
+    # ------------------------------------------------------------------
+    # Metadata interface — mirrors DeezerClient / iTunesClient signatures
+    # ------------------------------------------------------------------
+
+    def search_tracks(self, query: str, limit: int = 20) -> List[Track]:
+        _rate_limit()
+        items = self.search_raw(query, types="track")
+        track_pairs: List[tuple] = []   # (Track, album_asin)
+        seen_album_asins: List[str] = []
+        for item in items:
+            if not item.is_track:
+                continue
+            track = Track.from_search_hit({
+                "asin": item.asin,
+                "title": _strip_edition(item.title),
+                "artistName": _primary_artist(item.artist_name),
+                "albumName": _strip_edition(item.album_name),
+                "albumAsin": item.album_asin,
+                "duration": item.duration_seconds,
+                "isrc": item.isrc,
+            })
+            track_pairs.append((track, item.album_asin))
+            if item.album_asin and item.album_asin not in seen_album_asins:
+                seen_album_asins.append(item.album_asin)
+            if len(track_pairs) >= limit:
+                break
+        album_metas = self._fetch_album_metas(seen_album_asins[:5])
+        tracks: List[Track] = []
+        for track, album_asin in track_pairs:
+            if album_asin and album_asin in album_metas:
+                meta = album_metas[album_asin]
+                track.image_url = meta.get("image")
+                track.release_date = str(meta.get("release_date") or meta.get("releaseDate") or "")
+                track.total_tracks = meta.get("trackCount")
+            tracks.append(track)
+        return tracks
+
+    def search_artists(self, query: str, limit: int = 20) -> List[Artist]:
+        _rate_limit()
+        items = self.search_raw(query, types="track")
+        seen: Dict[str, Artist] = {}
+        artist_album_asin: Dict[str, str] = {}  # artist name → first album ASIN seen
+        for item in items:
+            name = _primary_artist(item.artist_name)
+            if not name:
+                continue
+            if name not in seen:
+                seen[name] = Artist.from_name(name)
+            if name not in artist_album_asin and item.album_asin:
+                artist_album_asin[name] = item.album_asin
+            if len(seen) >= limit:
+                break
+        # T2Tunes has no artist images — use an album cover as stand-in.
+        unique_asins = list({v for v in artist_album_asin.values()})[:5]
+        album_metas = self._fetch_album_metas(unique_asins)
+        for name, artist in seen.items():
+            asin = artist_album_asin.get(name)
+            if asin and asin in album_metas:
+                artist.image_url = album_metas[asin].get("image")
+        return list(seen.values())
+
+    def search_albums(self, query: str, limit: int = 20) -> List[Album]:
+        _rate_limit()
+        items = self.search_raw(query, types="track")
+        album_candidates: List[tuple] = []  # (Album, asin)
+        seen_keys: set = set()
+        for item in items:
+            album_asin = item.album_asin
+            raw_name = item.album_name
+            if not raw_name:
+                continue
+            display_name = _strip_edition(raw_name)
+            artist = _primary_artist(item.artist_name)
+            # Collapse Explicit/Clean variants: same normalised name + artist = same album
+            dedup_key = (display_name.lower(), artist.lower())
+            if dedup_key in seen_keys:
+                continue
+            seen_keys.add(dedup_key)
+            album = Album.from_search_hit({
+                "albumAsin": album_asin,
+                "albumName": display_name,
+                "artistName": artist,
+            })
+            album_candidates.append((album, album_asin))
+            if len(album_candidates) >= limit:
+                break
+        album_metas = self._fetch_album_metas([a for _, a in album_candidates[:10]])
+        albums: List[Album] = []
+        for album, asin in album_candidates:
+            if asin in album_metas:
+                meta = album_metas[asin]
+                album.image_url = meta.get("image")
+                album.release_date = str(meta.get("release_date") or meta.get("releaseDate") or "")
+                album.total_tracks = int(meta.get("trackCount") or 0)
+            albums.append(album)
+        return albums
+
+    def get_track_details(self, asin: str) -> Optional[Dict[str, Any]]:
+        """Return a Spotify-compatible dict for a single track ASIN."""
+        _rate_limit()
+        try:
+            streams = self.media_from_asin(asin)
+        except AmazonClientError:
+            return None
+        if not streams:
+            return None
+        s = streams[0]
+
+        album_data: Dict[str, Any] = {}
+        try:
+            meta = self.album_metadata(asin)
+            albums = meta.get("albumList")
+            if isinstance(albums, list) and albums and isinstance(albums[0], dict):
+                album_data = albums[0]
+        except AmazonClientError:
+            pass
+
+        return {
+            "id": s.asin,
+            "name": _strip_edition(s.title),
+            "artists": [{"name": _primary_artist(s.artist), "id": ""}],
+            "album": {
+                "id": album_data.get("asin", ""),
+                "name": _strip_edition(s.album),
+                "images": [{"url": album_data["image"]}] if album_data.get("image") else [],
+                "release_date": album_data.get("release_date") or album_data.get("releaseDate") or s.date or "",
+                "total_tracks": album_data.get("trackCount", 0),
+            },
+            "duration_ms": 0,
+            "popularity": 0,
+            "external_urls": {"amazon": f"https://music.amazon.com/albums/{asin}"},
+            "track_number": s.track_number,
+            "disc_number": s.disc_number,
+            "isrc": s.isrc,
+            "is_album_track": True,
+            "raw_data": {
+                "codec": s.codec,
+                "format": s.format,
+                "sample_rate": s.sample_rate,
+                "streamable": s.streamable,
+                "has_decryption_key": s.has_decryption_key,
+            },
+        }
+
+    def get_album(self, asin: str, include_tracks: bool = True) -> Optional[Dict[str, Any]]:
+        """Return a Spotify-compatible album dict."""
+        _rate_limit()
+        try:
+            meta = self.album_metadata(asin)
+        except AmazonClientError:
+            return None
+
+        albums = meta.get("albumList")
+        if not isinstance(albums, list) or not albums:
+            return None
+        album = albums[0] if isinstance(albums[0], dict) else {}
+
+        result: Dict[str, Any] = {
+            "id": asin,
+            "name": _strip_edition(album.get("title", "")),
+            "artists": [{"name": _primary_artist(album.get("artistName", "")), "id": ""}],
+            "release_date": album.get("release_date") or album.get("releaseDate") or "",
+            "total_tracks": album.get("trackCount", 0),
+            "album_type": "album",
+            "images": [{"url": album["image"]}] if album.get("image") else [],
+            "external_urls": {"amazon": f"https://music.amazon.com/albums/{asin}"},
+            "label": album.get("label", ""),
+        }
+        if include_tracks:
+            tracks_data = self.get_album_tracks(asin) or {
+                "items": [], "total": 0, "limit": 50, "next": None,
+            }
+            result["tracks"] = tracks_data
+            # Backfill release_date from stream tags when album metadata lacks it.
+            if not result["release_date"]:
+                items = tracks_data.get("items") or []
+                for item in items:
+                    rd = item.get("release_date") or ""
+                    if rd and len(rd) >= 4:
+                        result["release_date"] = rd
+                        break
+        return result
+
+    def get_album_tracks(self, asin: str) -> Optional[Dict[str, Any]]:
+        """Return album tracks in Spotify pagination format."""
+        _rate_limit()
+        try:
+            streams = self.media_from_asin(asin)
+        except AmazonClientError:
+            return None
+
+        # media_from_asin has no duration — enrich from search results which do.
+        duration_map: Dict[str, int] = {}  # track asin → duration_ms
+        if streams:
+            album_name = _strip_edition(streams[0].album)
+            artist_name = _primary_artist(streams[0].artist)
+            try:
+                search_items = self.search_raw(
+                    f"{album_name} {artist_name}", types="track"
+                )
+                for item in search_items:
+                    if item.album_asin == asin and item.duration_seconds:
+                        duration_map[item.asin] = item.duration_seconds * 1000
+            except Exception as exc:
+                logger.debug("Duration backfill failed for ASIN %s: %s", asin, exc)
+
+        items = [
+            {
+                "id": s.asin,
+                "name": _strip_edition(s.title),
+                "artists": [{"name": _primary_artist(s.artist), "id": ""}],
+                "duration_ms": duration_map.get(s.asin, 0),
+                "track_number": s.track_number if s.track_number is not None else idx + 1,
+                "disc_number": s.disc_number,
+                "release_date": s.date or "",
+                "isrc": s.isrc,
+            }
+            for idx, s in enumerate(streams)
+        ]
+        return {"items": items, "total": len(items), "limit": 50, "next": None}
+
+    def get_artist(self, artist_name: str) -> Optional[Dict[str, Any]]:
+        """Return a Spotify-compatible artist dict inferred from search results."""
+        _rate_limit()
+        search_name = _unslugify(artist_name)
+        try:
+            items = self.search_raw(search_name, types="track")
+        except AmazonClientError:
+            return None
+        name_lower = search_name.lower()
+        match = next(
+            (i for i in items if _primary_artist(i.artist_name).lower() == name_lower),
+            next((i for i in items if name_lower in _primary_artist(i.artist_name).lower()), None),
+        )
+        if not match:
+            return None
+        return {
+            "id": match.artist_name.lower().replace(" ", "_"),
+            "name": match.artist_name,
+            "genres": [],
+            "popularity": 0,
+            "followers": {"total": 0},
+            "images": [],
+            "external_urls": {},
+        }
+
+    def get_artist_albums(
+        self,
+        artist_name: str,
+        album_type: str = "album,single",
+        limit: int = 200,
+    ) -> List[Album]:
+        """Return albums for an artist inferred from search results."""
+        _rate_limit()
+        search_name = _unslugify(artist_name)
+        try:
+            items = self.search_raw(f"{search_name} album", types="track")
+        except AmazonClientError:
+            return []
+        album_candidates: List[tuple] = []  # (Album, asin)
+        seen_asins: set = set()
+        name_lower = search_name.lower()
+        for item in items:
+            if not item.is_album:
+                continue
+            if _primary_artist(item.artist_name).lower() != name_lower:
+                continue
+            album_asin = item.album_asin or item.asin
+            if album_asin in seen_asins:
+                continue
+            seen_asins.add(album_asin)
+            album = Album.from_search_hit({
+                "albumAsin": album_asin,
+                "albumName": _strip_edition(item.album_name or item.title),
+                "artistName": _primary_artist(item.artist_name),
+            })
+            album_candidates.append((album, album_asin))
+            if len(album_candidates) >= limit:
+                break
+
+        # Fetch metadata for art, release_date, track_count, and type inference.
+        # No explicit cap here — search_raw naturally bounds results to ~20 items,
+        # so album_candidates is already small.
+        asins_to_fetch = [a for _, a in album_candidates]
+        metas = self._fetch_album_metas(asins_to_fetch)
+
+        albums: List[Album] = []
+        for album, asin in album_candidates:
+            meta = metas.get(asin, {})
+            if meta:
+                album.image_url = meta.get("image")
+                album.release_date = str(meta.get("release_date") or meta.get("releaseDate") or "")
+                total = int(meta.get("trackCount") or 0)
+                album.total_tracks = total
+                # T2Tunes doesn't expose release type — infer from track count.
+                # 1-track releases are singles; keep default "album" otherwise.
+                if total == 1:
+                    album.album_type = "single"
+            albums.append(album)
+        return albums
+
+    def get_track_features(self, track_id: str) -> Optional[Dict[str, Any]]:
+        """Not available from Amazon Music — returns None for compatibility."""
+        return None
+
+    def _get_artist_image_from_albums(self, artist_id: str) -> Optional[str]:
+        """Return an album cover as artist image stand-in (T2Tunes has no artist images)."""
+        search_name = _unslugify(artist_id)
+        try:
+            items = self.search_raw(search_name, types="track")
+        except AmazonClientError:
+            return None
+        name_lower = search_name.lower()
+        for item in items:
+            if _primary_artist(item.artist_name).lower() != name_lower:
+                continue
+            asin = item.album_asin or item.asin
+            if not asin:
+                continue
+            metas = self._fetch_album_metas([asin])
+            if asin in metas and metas[asin].get("image"):
+                return metas[asin]["image"]
+        return None
+
+    # ==================== Interface Aliases (match DeezerClient method names) ====================
+    get_album_metadata = get_album
+    get_artist_info = get_artist
+    get_artist_albums_list = get_artist_albums
+
+    # ------------------------------------------------------------------
+    # Private helpers
+    # ------------------------------------------------------------------
+
+    def _fetch_album_metas(self, asins: List[str]) -> Dict[str, Dict[str, Any]]:
+        """Parallel-fetch album metadata for up to N ASINs. Returns {asin: albumList[0]}."""
+        if not asins:
+            return {}
+        metas: Dict[str, Dict[str, Any]] = {}
+
+        def _fetch(asin: str) -> None:
+            now = time.monotonic()
+            with _meta_cache_lock:
+                entry = _meta_cache.get(asin)
+                if entry and (now - entry[0]) < _META_CACHE_TTL:
+                    metas[asin] = entry[1]
+                    return
+            _rate_limit()
+            try:
+                raw = self.album_metadata(asin)
+                lst = raw.get("albumList")
+                if isinstance(lst, list) and lst and isinstance(lst[0], dict):
+                    meta = lst[0]
+                    with _meta_cache_lock:
+                        _meta_cache[asin] = (time.monotonic(), meta)
+                    metas[asin] = meta
+            except Exception as exc:
+                logger.debug("Album metadata fetch failed for ASIN %s: %s", asin, exc)
+
+        from concurrent.futures import ThreadPoolExecutor
+        with ThreadPoolExecutor(max_workers=min(len(asins), 5)) as pool:
+            list(pool.map(_fetch, asins))
+        return metas
+
+    def _get_json(self, path: str, params: Optional[Dict[str, Any]] = None) -> Any:
+        url = urljoin(f"{self.base_url}/", path.lstrip("/"))
+        last_exc: Optional[Exception] = None
+        for attempt in range(3):
+            if attempt:
+                time.sleep(1.0 * attempt)
+            try:
+                resp = self.session.get(url, params=params, timeout=self.timeout)
+                resp.raise_for_status()
+            except requests.HTTPError as exc:
+                body = exc.response.text[:500].replace("\n", " ")
+                # T2Tunes returns 400 for transient Amazon-side failures — retry those.
+                if exc.response.status_code == 400 and "Failed to search" in exc.response.text:
+                    logger.debug("T2Tunes transient 400 on attempt %d, retrying: %s", attempt + 1, url)
+                    last_exc = AmazonClientError(
+                        f"HTTP {exc.response.status_code} for {url} — body: {body!r}"
+                    )
+                    continue
+                raise AmazonClientError(
+                    f"HTTP {exc.response.status_code} for {url} — body: {body!r}",
+                    status_code=exc.response.status_code,
+                ) from exc
+            except requests.RequestException as exc:
+                raise AmazonClientError(f"Request failed for {url}: {exc}") from exc
+            try:
+                return resp.json()
+            except ValueError as exc:
+                preview = resp.text[:200].replace("\n", " ")
+                raise AmazonClientError(f"Response not JSON for {url}: {preview!r}") from exc
+        raise last_exc  # type: ignore[misc]
+
+    @staticmethod
+    def _iter_search_items(response: Any) -> Iterator[T2TunesSearchItem]:
+        if not isinstance(response, dict):
+            raise AmazonClientError(
+                f"Unexpected search response type: {type(response).__name__}"
+            )
+        for result in response.get("results") or []:
+            if not isinstance(result, dict):
+                continue
+            for hit in result.get("hits") or []:
+                if not isinstance(hit, dict):
+                    continue
+                doc = hit.get("document")
+                if not isinstance(doc, dict):
+                    continue
+                asin = str(doc.get("asin") or "")
+                if not asin:
+                    continue
+                yield T2TunesSearchItem(
+                    asin=asin,
+                    title=str(doc.get("title") or ""),
+                    artist_name=str(doc.get("artistName") or ""),
+                    item_type=str(doc.get("__type") or ""),
+                    album_name=str(doc.get("albumName") or ""),
+                    album_asin=str(doc.get("albumAsin") or ""),
+                    duration_seconds=int(doc.get("duration") or 0),
+                    isrc=str(doc.get("isrc") or ""),
+                )
+
+    @staticmethod
+    def _parse_stream_info(item: Dict[str, Any]) -> T2TunesStreamInfo:
+        stream_info = item.get("streamInfo") if isinstance(item.get("streamInfo"), dict) else {}
+        tags = item.get("tags") if isinstance(item.get("tags"), dict) else {}
+        # T2Tunes API has a typo: "stremeable" in some responses
+        streamable = item.get("streamable")
+        if streamable is None:
+            streamable = item.get("stremeable")
+        raw_key = item.get("decryptionKey")
+        decryption_key = str(raw_key) if raw_key else None
+
+        def _int_tag(key: str) -> Optional[int]:
+            v = tags.get(key)
+            try:
+                return int(v) if v is not None else None
+            except (TypeError, ValueError):
+                return None
+
+        return T2TunesStreamInfo(
+            asin=str(item.get("asin") or ""),
+            streamable=bool(streamable),
+            codec=str(stream_info.get("codec") or ""),
+            format=str(stream_info.get("format") or ""),
+            sample_rate=(
+                stream_info.get("sampleRate")
+                if isinstance(stream_info.get("sampleRate"), int)
+                else None
+            ),
+            stream_url=str(stream_info.get("streamUrl") or ""),
+            decryption_key=decryption_key,
+            title=str(tags.get("title") or ""),
+            artist=str(tags.get("artist") or ""),
+            album=str(tags.get("album") or ""),
+            isrc=str(tags.get("isrc") or ""),
+            cover_url=str(item.get("coverUrl") or ""),
+            track_number=_int_tag("trackNumber"),
+            disc_number=_int_tag("discNumber"),
+            genre=str(tags.get("genre") or ""),
+            label=str(tags.get("label") or ""),
+            date=str(tags.get("date") or ""),
+        )

core/amazon_download_client.py

@@ -0,0 +1,466 @@
+"""Amazon Music download source plugin backed by a T2Tunes proxy.
+
+NOT yet wired into the app registry — validated in isolation only.
+See tests/tools/test_amazon_download_client.py.
+
+Filename encoding (the DownloadSourcePlugin dispatch contract):
+    "{asin}||{display_name}"
+    e.g. "B09XYZ1234||Kendrick Lamar - Not Like Us"
+
+Codec preference order: FLAC → Opus → EAC3.
+
+Download flow (from Tubifarry reference implementation):
+    1. GET stream_url → encrypted bytes on disk
+    2. FFmpeg -decryption_key <hex> to decrypt in place
+    3. Embed metadata tags (handled by the app's standard post-processing)
+"""
+
+from __future__ import annotations
+
+import os
+import shutil
+import subprocess
+import threading
+import time
+import uuid
+from pathlib import Path
+from typing import Any, Dict, List, Optional, Tuple
+
+import requests as http_requests
+
+from config.settings import config_manager
+from core.amazon_client import AmazonClient, AmazonClientError
+from core.download_plugins.base import DownloadSourcePlugin
+from core.download_plugins.types import AlbumResult, DownloadStatus, TrackResult
+from core.quality.model import AudioQuality
+from core.quality.source_map import quality_tier_for_source
+from utils.logging_config import get_logger
+
+logger = get_logger("amazon_download_client")
+
+# Quality / codec helpers
+CODEC_PREFERENCE = ["flac", "opus", "eac3"]
+
+_CODEC_EXTENSIONS: Dict[str, str] = {
+    "flac": "flac",
+    "ogg_vorbis": "ogg",
+    "opus": "opus",
+    "eac3": "eac3",
+    "mp4": "m4a",
+    "aac": "m4a",
+    "mp3": "mp3",
+}
+
+MIN_AUDIO_BYTES = 512 * 1024  # 512 KB — anything smaller is a broken stream
+
+
+def _codec_key(codec: str) -> str:
+    return codec.lower().replace("-", "_").replace(" ", "_")
+
+
+def _file_extension(codec: str) -> str:
+    return _CODEC_EXTENSIONS.get(_codec_key(codec), "bin")
+
+
+def _quality_label(codec: str, sample_rate: Optional[int] = None) -> str:
+    ck = _codec_key(codec)
+    if ck == "flac":
+        if sample_rate and sample_rate > 48000:
+            return "Hi-Res"
+        return "Lossless"
+    return "Lossy"
+
+
+class AmazonDownloadClient(DownloadSourcePlugin):
+    """DownloadSourcePlugin — Amazon Music via T2Tunes proxy."""
+
+    def __init__(self, download_path: Optional[str] = None) -> None:
+        if download_path is None:
+            download_path = config_manager.get("soulseek.download_path", "./downloads")
+        self.download_path = Path(download_path)
+        try:
+            self.download_path.mkdir(parents=True, exist_ok=True)
+        except OSError as e:
+            logger.warning(f"Could not verify download path {self.download_path}: {e}")
+
+        self._quality = quality_tier_for_source("amazon", default="flac")
+        self._allow_fallback = config_manager.get("amazon_download.allow_fallback", True)
+
+        self._client = AmazonClient(preferred_codec=self._quality)
+        self.session = http_requests.Session()
+        self.session.headers.update({
+            "User-Agent": "SoulSync/1.0",
+            "Accept": "*/*",
+        })
+
+        self._engine: Any = None
+        self.shutdown_check: Any = None
+
+    # ------------------------------------------------------------------
+    # DownloadSourcePlugin — lifecycle
+    # ------------------------------------------------------------------
+
+    def set_engine(self, engine) -> None:
+        """Engine callback — wires the central thread worker + state store."""
+        self._engine = engine
+
+    def set_shutdown_check(self, check_callable) -> None:
+        self.shutdown_check = check_callable
+
+    def is_configured(self) -> bool:
+        # T2Tunes has a public default instance; no credentials required.
+        # Return True unconditionally so the source shows as available.
+        return True
+
+    async def check_connection(self) -> bool:
+        try:
+            return self._client.is_authenticated()
+        except Exception:
+            return False
+
+    # ------------------------------------------------------------------
+    # DownloadSourcePlugin — search
+    # ------------------------------------------------------------------
+
+    async def search(
+        self,
+        query: str,
+        timeout: Optional[int] = None,
+        progress_callback: Any = None,
+    ) -> Tuple[List[TrackResult], List[AlbumResult]]:
+        try:
+            items = self._client.search_raw(query, types="track,album")
+        except AmazonClientError as exc:
+            logger.warning(f"Amazon search failed for {query!r}: {exc}")
+            return [], []
+
+        track_results: List[TrackResult] = []
+        album_map: Dict[str, AlbumResult] = {}
+        album_order: List[str] = []
+        preferred = self._client.preferred_codec
+        # Search results only carry the codec (real sample_rate arrives at
+        # stream time). Claim the format honestly — FLAC for the lossless
+        # codec, lossy otherwise — so audio_quality derives a real format
+        # instead of the display label ("Lossless"), and the post-download
+        # probe pins the actual sample_rate/bit_depth.
+        amazon_q = AudioQuality(format='flac' if _codec_key(preferred) == 'flac' else 'aac')
+
+        for item in items:
+            quality = _quality_label(preferred)
+            if item.is_track:
+                tr = TrackResult(
+                    username="amazon",
+                    filename=f"{item.asin}||{item.artist_name} - {item.title}",
+                    size=0,
+                    bitrate=None,
+                    duration=item.duration_seconds * 1000 if item.duration_seconds else None,
+                    quality=quality,
+                    free_upload_slots=999,
+                    upload_speed=999_999,
+                    queue_length=0,
+                    artist=item.artist_name,
+                    title=item.title,
+                    album=item.album_name,
+                    _source_metadata={
+                        "asin": item.asin,
+                        "album_asin": item.album_asin,
+                        "isrc": item.isrc,
+                    },
+                )
+                tr.set_quality(amazon_q)
+                track_results.append(tr)
+            elif item.is_album:
+                album_asin = item.album_asin or item.asin
+                if album_asin not in album_map:
+                    placeholder = TrackResult(
+                        username="amazon",
+                        filename=f"{item.asin}||{item.artist_name} - {item.title}",
+                        size=0,
+                        bitrate=None,
+                        duration=None,
+                        quality=quality,
+                        free_upload_slots=999,
+                        upload_speed=999_999,
+                        queue_length=0,
+                        artist=item.artist_name,
+                        title=item.title,
+                        album=item.album_name,
+                    )
+                    placeholder.set_quality(amazon_q)
+                    album_map[album_asin] = AlbumResult(
+                        username="amazon",
+                        album_path=album_asin,
+                        album_title=item.album_name or item.title,
+                        artist=item.artist_name,
+                        track_count=0,
+                        total_size=0,
+                        tracks=[placeholder],
+                        dominant_quality=quality,
+                    )
+                    album_order.append(album_asin)
+
+        return track_results, [album_map[k] for k in album_order]
+
+    # ------------------------------------------------------------------
+    # DownloadSourcePlugin — download dispatch
+    # ------------------------------------------------------------------
+
+    async def download(
+        self,
+        username: str,
+        filename: str,
+        file_size: int = 0,
+    ) -> Optional[str]:
+        if "||" not in filename:
+            logger.error(f"Invalid Amazon filename format (no '||'): {filename!r}")
+            return None
+        if self._engine is None:
+            raise RuntimeError(
+                "AmazonDownloadClient._engine is not set — "
+                "client not connected to download infrastructure"
+            )
+        asin, display_name = filename.split("||", 1)
+        asin = asin.strip()
+        display_name = display_name.strip()
+        return self._engine.worker.dispatch(
+            source_name="amazon",
+            target_id=asin,
+            display_name=display_name,
+            original_filename=filename,
+            impl_callable=self._download_sync,
+        )
+
+    def _download_sync(
+        self,
+        download_id: str,
+        target_id: str,
+        display_name: str,
+    ) -> Optional[str]:
+        asin = str(target_id)
+        codecs = CODEC_PREFERENCE if self._allow_fallback else [self._quality]
+        for codec in codecs:
+            try:
+                streams = self._client.media_from_asin(asin, codec=codec)
+            except AmazonClientError as exc:
+                logger.warning(f"media_from_asin({asin!r}, {codec}) failed: {exc}")
+                continue
+
+            stream = next(
+                (s for s in streams if s.streamable and s.stream_url),
+                None,
+            )
+            if not stream:
+                logger.debug(f"No streamable result for {asin} at codec={codec}")
+                continue
+
+            ext = _file_extension(stream.codec or codec)
+            safe = "".join(
+                ch if ch.isalnum() or ch in " -_." else "_"
+                for ch in display_name
+            )[:80]
+            # T2Tunes always serves audio in an encrypted MP4 container.
+            # The codec extension (.flac, .opus, .eac3) is only for the
+            # final decrypted output.
+            enc_ext = "mp4" if stream.decryption_key else ext
+            enc_path = self._unique_path(self.download_path / f"{safe}.enc.{enc_ext}")
+            out_path = self._unique_path(self.download_path / f"{safe}.{ext}")
+
+            if self._engine is not None:
+                self._engine.update_record(
+                    "amazon", download_id, {"state": "downloading", "progress": 0.0}
+                )
+            try:
+                downloaded = self._stream_to_file(stream.stream_url, enc_path, download_id)
+            except Exception as exc:
+                logger.warning(f"Stream download failed for {asin} ({codec}): {exc}")
+                enc_path.unlink(missing_ok=True)
+                continue
+
+            if downloaded < MIN_AUDIO_BYTES:
+                logger.warning(
+                    f"File too small ({downloaded} B) for {asin} at {codec} — trying next"
+                )
+                enc_path.unlink(missing_ok=True)
+                continue
+
+            if stream.decryption_key:
+                if self._engine is not None:
+                    self._engine.update_record(
+                        "amazon", download_id, {"state": "decrypting", "progress": 1.0}
+                    )
+                try:
+                    self._decrypt_with_ffmpeg(enc_path, out_path, stream.decryption_key)
+                    enc_path.unlink(missing_ok=True)
+                except Exception as exc:
+                    logger.error(f"Decryption failed for {asin} ({codec}): {exc}")
+                    enc_path.unlink(missing_ok=True)
+                    out_path.unlink(missing_ok=True)
+                    continue
+            else:
+                enc_path.rename(out_path)
+
+            final_size = out_path.stat().st_size
+            logger.info(
+                f"Amazon download complete ({codec}): {out_path} "
+                f"({final_size / (1024 * 1024):.1f} MB)"
+            )
+            # Sync size == transferred so the download monitor's bytes-incomplete
+            # guard doesn't block post-processing. The throttled updates in
+            # _stream_to_file leave transferred < size after the last 0.5s tick;
+            # other streaming clients avoid this by not tracking bytes at all
+            # (size stays 0, the guard is skipped). Writing the final output size
+            # here restores parity.
+            if self._engine is not None:
+                self._engine.update_record(
+                    "amazon", download_id, {'size': final_size, 'transferred': final_size}
+                )
+            return str(out_path)
+
+        logger.error(f"All codec tiers exhausted for '{display_name}' ({asin})")
+        return None
+
+    def _decrypt_with_ffmpeg(
+        self, enc_path: Path, out_path: Path, hex_key: str
+    ) -> None:
+        """Decrypt a T2Tunes encrypted audio file using FFmpeg -decryption_key.
+
+        T2Tunes uses CENC (Common Encryption) for DRM-protected tracks.
+        FFmpeg supports decryption via the -decryption_key flag when the
+        key is provided in hex.
+        """
+        ffmpeg = shutil.which("ffmpeg")
+        if not ffmpeg:
+            tools_dir = Path(__file__).parent.parent / "tools"
+            candidate = tools_dir / ("ffmpeg.exe" if os.name == "nt" else "ffmpeg")
+            if candidate.exists():
+                ffmpeg = str(candidate)
+            else:
+                raise RuntimeError(
+                    "ffmpeg is required for Amazon Music decryption. "
+                    "Install ffmpeg and ensure it is on your PATH."
+                )
+        cmd = [
+            ffmpeg,
+            "-y",
+            "-hide_banner",
+            "-loglevel", "error",
+            "-decryption_key", hex_key,
+            "-i", str(enc_path),
+            "-map", "0:a:0",   # extract first audio stream (FLAC/Opus/EAC3 inside MP4)
+            "-c", "copy",
+            str(out_path),
+        ]
+        logger.debug(f"Decrypting {enc_path.name} → {out_path.name}")
+        result = subprocess.run(cmd, capture_output=True)
+        if result.returncode != 0:
+            stderr = result.stderr.decode("utf-8", errors="replace").strip()
+            raise RuntimeError(f"FFmpeg decryption failed (exit {result.returncode}): {stderr}")
+
+    def _stream_to_file(self, url: str, out_path: Path, download_id: str) -> int:
+        resp = self.session.get(url, stream=True, timeout=60)
+        resp.raise_for_status()
+
+        total = int(resp.headers.get("content-length", 0))
+        downloaded = 0
+        last_report = time.monotonic()
+        shutdown_triggered = False
+
+        with out_path.open("wb") as fh:
+            for chunk in resp.iter_content(chunk_size=64 * 1024):
+                if not chunk:
+                    continue
+                if self.shutdown_check and self.shutdown_check():
+                    shutdown_triggered = True
+                    break
+                fh.write(chunk)
+                downloaded += len(chunk)
+                now = time.monotonic()
+                if self._engine and now - last_report >= 0.5:
+                    self._engine.update_record(
+                        "amazon",
+                        download_id,
+                        {
+                            "transferred": downloaded,
+                            "size": total,
+                            "progress": downloaded / total if total else 0.0,
+                        },
+                    )
+                    last_report = now
+
+        if shutdown_triggered:
+            out_path.unlink(missing_ok=True)
+            raise RuntimeError("Shutdown requested mid-download")
+
+        return downloaded
+
+    # ------------------------------------------------------------------
+    # DownloadSourcePlugin — status interface
+    # ------------------------------------------------------------------
+
+    async def get_all_downloads(self) -> List[DownloadStatus]:
+        if self._engine is None:
+            return []
+        return [
+            self._record_to_status(record)
+            for record in self._engine.iter_records_for_source('amazon')
+        ]
+
+    async def get_download_status(self, download_id: str) -> Optional[DownloadStatus]:
+        if self._engine is None:
+            return None
+        record = self._engine.get_record('amazon', download_id)
+        return self._record_to_status(record) if record is not None else None
+
+    async def cancel_download(
+        self,
+        download_id: str,
+        username: Optional[str] = None,
+        remove: bool = False,
+    ) -> bool:
+        if self._engine is None:
+            return False
+        if self._engine.get_record('amazon', download_id) is None:
+            return False
+        self._engine.update_record('amazon', download_id, {'state': 'Cancelled'})
+        if remove:
+            self._engine.remove_record('amazon', download_id)
+        return True
+
+    async def clear_all_completed_downloads(self) -> bool:
+        if self._engine is None:
+            return True
+        terminal = {'Completed, Succeeded', 'Cancelled', 'Errored', 'Aborted'}
+        for record in list(self._engine.iter_records_for_source('amazon')):
+            if record.get('state') in terminal:
+                self._engine.remove_record('amazon', record['id'])
+        return True
+
+    # ------------------------------------------------------------------
+    # Private helpers
+    # ------------------------------------------------------------------
+
+    @staticmethod
+    def _unique_path(path: Path) -> Path:
+        if not path.exists():
+            return path
+        stem, suffix = path.stem, path.suffix
+        for i in range(1, 100):
+            candidate = path.with_name(f"{stem} ({i}){suffix}")
+            if not candidate.exists():
+                return candidate
+        return path.with_name(f"{stem}_{uuid.uuid4().hex[:8]}{suffix}")
+
+    @staticmethod
+    def _record_to_status(rec: Dict[str, Any]) -> DownloadStatus:
+        return DownloadStatus(
+            id=str(rec.get('id', '')),
+            filename=str(rec.get('filename', '')),
+            username='amazon',
+            state=str(rec.get('state', 'queued')),
+            progress=float(rec.get('progress', 0.0)),
+            size=int(rec.get('size', 0)),
+            transferred=int(rec.get('transferred', 0)),
+            speed=int(rec.get('speed', 0)),
+            time_remaining=rec.get('time_remaining'),
+            file_path=rec.get('file_path'),
+        )

core/amazon_outage.py

@@ -0,0 +1,61 @@
+"""Amazon enrichment outage detection + back-off — pure, importable, testable.
+
+The Amazon worker enriches via a public T2Tunes proxy instance. When that
+instance is down (HTTP 5xx, "Amazon Music API is not initialized", or an
+unreachable host), the worker must NOT treat every album as an individual
+failure: doing so floods the logs with an error per item, churns network + DB
+continuously, and permanently marks the whole library ``error`` (which the
+retry tiers never re-attempt) for what is really a transient outage.
+
+Instead it recognizes "the whole source is down", leaves the item untouched so
+it's retried once the instance recovers, and backs off hard. These two pure
+helpers carry that logic so it can be unit-tested without the worker, the DB,
+or the network.
+"""
+
+from __future__ import annotations
+
+import re
+
+# HTTP statuses that mean "the source/proxy is unhealthy", not "no match".
+_OUTAGE_STATUS = {500, 502, 503, 504}
+
+# Substrings (lower-cased) in an error message that indicate a source outage
+# rather than a per-item miss: proxy not ready, gateway errors, the host being
+# unreachable, or an error page returned instead of JSON.
+_OUTAGE_PHRASES = (
+    "not initialized", "not configured", "service unavailable",
+    "bad gateway", "gateway time", "request failed", "response not json",
+    "max retries", "connection", "timed out", "temporarily unavailable",
+)
+
+# Back-off schedule while the source is down.
+_NORMAL_DELAY = 2          # seconds between items when healthy
+_OUTAGE_BASE = 30          # first back-off step
+_OUTAGE_CAP = 1800         # 30 minutes max
+
+
+def is_source_outage(exc: Exception) -> bool:
+    """True when ``exc`` indicates the Amazon source/proxy is down (transient,
+    whole-source), as opposed to a normal per-item error.
+
+    Robust to how the error is surfaced: an explicit ``status_code`` attribute,
+    an ``HTTP <code>`` prefix in the message, or an outage phrase (covers
+    connection failures and non-JSON error pages that carry no status code)."""
+    code = getattr(exc, "status_code", None)
+    if isinstance(code, int) and code in _OUTAGE_STATUS:
+        return True
+    msg = str(exc).lower()
+    m = re.search(r"http\s+(\d{3})", msg)
+    if m and int(m.group(1)) in _OUTAGE_STATUS:
+        return True
+    return any(p in msg for p in _OUTAGE_PHRASES)
+
+
+def next_poll_delay_seconds(outage_streak: int) -> int:
+    """Seconds to wait before the next item. Normal cadence when healthy;
+    escalating back-off (30s, 60s, 120s, … capped at 30 min) the longer the
+    source has been down, so a dead instance can't flood logs/CPU/DB."""
+    if outage_streak <= 0:
+        return _NORMAL_DELAY
+    return min(_OUTAGE_BASE * (2 ** min(outage_streak - 1, 6)), _OUTAGE_CAP)

core/amazon_worker.py

@@ -0,0 +1,645 @@
+import re
+import threading
+import time
+from difflib import SequenceMatcher
+from typing import Optional, Dict, Any
+from datetime import datetime, timedelta
+from utils.logging_config import get_logger
+from database.music_database import MusicDatabase
+from core.amazon_client import AmazonClient
+from core.worker_utils import interruptible_sleep, set_album_api_track_count
+from core.enrichment.manual_match_honoring import honor_stored_match
+from core.amazon_outage import is_source_outage, next_poll_delay_seconds
+
+logger = get_logger("amazon_worker")
+
+
+class AmazonWorker:
+    """Background worker for enriching library artists, albums, and tracks with Amazon Music metadata."""
+
+    def __init__(self, database: MusicDatabase):
+        self.db = database
+        self.client = AmazonClient()
+        self._amazon_schema_checked = False
+
+        self.running = False
+        self.paused = False
+        self.should_stop = False
+        self.thread = None
+        self._stop_event = threading.Event()
+
+        self.current_item = None
+
+        self.stats = {
+            'matched': 0,
+            'not_found': 0,
+            'pending': 0,
+            'errors': 0,
+        }
+
+        self.retry_days = 30
+        self.name_similarity_threshold = 0.80
+
+        # Source-outage circuit breaker: counts consecutive whole-source
+        # failures (proxy down / "not initialized" / unreachable) so the loop
+        # backs off instead of grinding the whole library item-by-item.
+        self._outage_streak = 0
+
+        logger.info("Amazon background worker initialized")
+
+    def _ensure_amazon_schema(self, cursor) -> None:
+        """Ensure upgraded installs have the Amazon enrichment columns.
+
+        MusicDatabase normally runs this migration during startup, but the
+        worker should still be defensive because it is the code path that
+        repeatedly queries these columns in the background.
+        """
+        if self._amazon_schema_checked:
+            return
+
+        table_columns = {
+            'artists': ('amazon_id', 'amazon_match_status', 'amazon_last_attempted'),
+            'albums': ('amazon_id', 'amazon_match_status', 'amazon_last_attempted'),
+            'tracks': ('amazon_id', 'amazon_match_status', 'amazon_last_attempted'),
+        }
+        for table, columns in table_columns.items():
+            cursor.execute(f"PRAGMA table_info({table})")
+            existing = {row[1] for row in cursor.fetchall()}
+            for column in columns:
+                if column not in existing:
+                    column_type = 'TIMESTAMP' if column == 'amazon_last_attempted' else 'TEXT'
+                    cursor.execute(f"ALTER TABLE {table} ADD COLUMN {column} {column_type}")
+
+        cursor.execute("CREATE INDEX IF NOT EXISTS idx_artists_amazon_id ON artists (amazon_id)")
+        cursor.execute("CREATE INDEX IF NOT EXISTS idx_artists_amazon_status ON artists (amazon_match_status)")
+        cursor.execute("CREATE INDEX IF NOT EXISTS idx_albums_amazon_id ON albums (amazon_id)")
+        cursor.execute("CREATE INDEX IF NOT EXISTS idx_albums_amazon_status ON albums (amazon_match_status)")
+        cursor.execute("CREATE INDEX IF NOT EXISTS idx_tracks_amazon_id ON tracks (amazon_id)")
+        cursor.execute("CREATE INDEX IF NOT EXISTS idx_tracks_amazon_status ON tracks (amazon_match_status)")
+        cursor.connection.commit()
+        self._amazon_schema_checked = True
+
+    def start(self):
+        if self.running:
+            logger.warning("Worker already running")
+            return
+
+        self.running = True
+        self.should_stop = False
+        self._stop_event.clear()
+        self.thread = threading.Thread(target=self._run, daemon=True)
+        self.thread.start()
+        logger.info("Amazon background worker started")
+
+    def stop(self):
+        if not self.running:
+            return
+
+        logger.info("Stopping Amazon worker...")
+        self.should_stop = True
+        self.running = False
+        self._stop_event.set()
+
+        if self.thread:
+            self.thread.join(timeout=1)
+
+        logger.info("Amazon worker stopped")
+
+    def pause(self):
+        if not self.running:
+            logger.warning("Worker not running, cannot pause")
+            return
+        self.paused = True
+        logger.info("Amazon worker paused")
+
+    def resume(self):
+        if not self.running:
+            logger.warning("Worker not running, start it first")
+            return
+        self.paused = False
+        logger.info("Amazon worker resumed")
+
+    def get_stats(self) -> Dict[str, Any]:
+        self.stats['pending'] = self._count_pending_items()
+        progress = self._get_progress_breakdown()
+        is_actually_running = self.running and (self.thread is not None and self.thread.is_alive())
+        is_idle = is_actually_running and not self.paused and self.stats['pending'] == 0 and self.current_item is None
+        return {
+            'enabled': True,
+            'running': is_actually_running and not self.paused,
+            'paused': self.paused,
+            'idle': is_idle,
+            'current_item': self.current_item,
+            'stats': self.stats.copy(),
+            'progress': progress,
+        }
+
+    def _run(self):
+        logger.info("Amazon worker thread started")
+        while not self.should_stop:
+            try:
+                if self.paused:
+                    interruptible_sleep(self._stop_event, 1)
+                    continue
+
+                self.current_item = None
+                item = self._get_next_item()
+
+                if not item:
+                    logger.debug("No pending items, sleeping...")
+                    interruptible_sleep(self._stop_event, 10)
+                    continue
+
+                self.current_item = item
+                item_id = item.get('id') or item.get('artist_id') or item.get('album_id')
+                if item_id is None:
+                    logger.warning(f"Skipping {item.get('type', 'unknown')} with NULL id: {item.get('name', '?')}")
+                    continue
+
+                self._process_item(item)
+                # Normal 2s cadence when healthy; escalating back-off (up to
+                # 30 min) while the source is in an outage streak.
+                interruptible_sleep(self._stop_event, next_poll_delay_seconds(self._outage_streak))
+
+            except Exception as e:
+                logger.error(f"Error in worker loop: {e}")
+                interruptible_sleep(self._stop_event, 5)
+
+        logger.info("Amazon worker thread finished")
+
+    def _get_next_item(self) -> Optional[Dict[str, Any]]:
+        conn = None
+        try:
+            conn = self.db._get_connection()
+            cursor = conn.cursor()
+            self._ensure_amazon_schema(cursor)
+
+            # Pinned-group override (Manage Enrichment Workers): process one
+            # entity type first, then fall through to the normal chain. Unset or
+            # exhausted ⇒ default artist→album→track order, unchanged.
+            from core.worker_utils import read_enrichment_priority, priority_pending_item
+            _prio = read_enrichment_priority('amazon')
+            if _prio:
+                _pi = priority_pending_item(cursor, 'amazon', _prio)
+                if _pi:
+                    return _pi
+
+            # Priority 1: Unattempted artists
+            cursor.execute("""
+                SELECT id, name FROM artists
+                WHERE amazon_match_status IS NULL AND id IS NOT NULL
+                ORDER BY id ASC LIMIT 1
+            """)
+            row = cursor.fetchone()
+            if row:
+                return {'type': 'artist', 'id': row[0], 'name': row[1]}
+
+            # Priority 2: Unattempted albums
+            cursor.execute("""
+                SELECT a.id, a.title, ar.name AS artist_name, ar.amazon_id AS artist_amazon_id
+                FROM albums a
+                JOIN artists ar ON a.artist_id = ar.id
+                WHERE a.amazon_match_status IS NULL AND a.id IS NOT NULL
+                ORDER BY a.id ASC LIMIT 1
+            """)
+            row = cursor.fetchone()
+            if row:
+                return {'type': 'album', 'id': row[0], 'name': row[1], 'artist': row[2], 'artist_amazon_id': row[3]}
+
+            # Priority 3: Unattempted tracks
+            cursor.execute("""
+                SELECT t.id, t.title, ar.name AS artist_name, ar.amazon_id AS artist_amazon_id
+                FROM tracks t
+                JOIN artists ar ON t.artist_id = ar.id
+                WHERE t.amazon_match_status IS NULL AND t.id IS NOT NULL
+                ORDER BY t.id ASC LIMIT 1
+            """)
+            row = cursor.fetchone()
+            if row:
+                return {'type': 'track', 'id': row[0], 'name': row[1], 'artist': row[2], 'artist_amazon_id': row[3]}
+
+            not_found_cutoff = datetime.now() - timedelta(days=self.retry_days)
+
+            # Priority 4: Retry not_found artists
+            cursor.execute("""
+                SELECT id, name FROM artists
+                WHERE amazon_match_status = 'not_found' AND amazon_last_attempted < ?
+                ORDER BY amazon_last_attempted ASC LIMIT 1
+            """, (not_found_cutoff,))
+            row = cursor.fetchone()
+            if row:
+                logger.info(f"Retrying artist '{row[1]}' (last attempted before cutoff)")
+                return {'type': 'artist', 'id': row[0], 'name': row[1]}
+
+            # Priority 5: Retry not_found albums
+            cursor.execute("""
+                SELECT a.id, a.title, ar.name AS artist_name, ar.amazon_id AS artist_amazon_id
+                FROM albums a
+                JOIN artists ar ON a.artist_id = ar.id
+                WHERE a.amazon_match_status = 'not_found' AND a.amazon_last_attempted < ?
+                ORDER BY a.amazon_last_attempted ASC LIMIT 1
+            """, (not_found_cutoff,))
+            row = cursor.fetchone()
+            if row:
+                return {'type': 'album', 'id': row[0], 'name': row[1], 'artist': row[2], 'artist_amazon_id': row[3]}
+
+            # Priority 6: Retry not_found tracks
+            cursor.execute("""
+                SELECT t.id, t.title, ar.name AS artist_name, ar.amazon_id AS artist_amazon_id
+                FROM tracks t
+                JOIN artists ar ON t.artist_id = ar.id
+                WHERE t.amazon_match_status = 'not_found' AND t.amazon_last_attempted < ?
+                ORDER BY t.amazon_last_attempted ASC LIMIT 1
+            """, (not_found_cutoff,))
+            row = cursor.fetchone()
+            if row:
+                return {'type': 'track', 'id': row[0], 'name': row[1], 'artist': row[2], 'artist_amazon_id': row[3]}
+
+            return None
+
+        except Exception as e:
+            logger.error(f"Error getting next item: {e}")
+            return None
+        finally:
+            if conn:
+                conn.close()
+
+    def _normalize_name(self, name: str) -> str:
+        name = name.lower().strip()
+        name = re.sub(r'\s+[-–—]\s+.*$', '', name)
+        name = re.sub(r'\s*\(.*?\)\s*', ' ', name)
+        name = re.sub(r'[^\w\s]', '', name)
+        name = re.sub(r'\s+', ' ', name).strip()
+        return name
+
+    def _name_matches(self, query_name: str, result_name: str) -> bool:
+        norm_query = self._normalize_name(query_name)
+        norm_result = self._normalize_name(result_name)
+        similarity = SequenceMatcher(None, norm_query, norm_result).ratio()
+        logger.debug(f"Name similarity: '{query_name}' vs '{result_name}' = {similarity:.2f}")
+        return similarity >= self.name_similarity_threshold
+
+    def _process_item(self, item: Dict[str, Any]):
+        try:
+            item_type = item['type']
+            item_id = item['id']
+            item_name = item['name']
+            logger.debug(f"Processing {item_type} #{item_id}: {item_name}")
+
+            if item_type == 'artist':
+                self._process_artist(item_id, item_name)
+            elif item_type == 'album':
+                self._process_album(item_id, item_name, item.get('artist', ''), item)
+            elif item_type == 'track':
+                self._process_track(item_id, item_name, item.get('artist', ''), item)
+
+            # The source answered (match or not_found) — clear any outage streak.
+            if self._outage_streak:
+                logger.info("Amazon source recovered after %d outage(s), resuming",
+                            self._outage_streak)
+                self._outage_streak = 0
+
+        except Exception as e:
+            if is_source_outage(e):
+                # The whole source is down (proxy 5xx / "not initialized" /
+                # unreachable). Do NOT mark the item 'error' — that would burn
+                # the entire library to a state the retry tiers never re-attempt
+                # for a transient outage. Leave it untouched so it's retried once
+                # the instance recovers, and let the loop back off. Log once per
+                # streak to avoid flooding.
+                self._outage_streak += 1
+                if self._outage_streak == 1:
+                    logger.warning("Amazon source unavailable — pausing enrichment "
+                                   "until it recovers: %s", e)
+                else:
+                    logger.debug("Amazon source still unavailable (streak=%d): %s",
+                                 self._outage_streak, e)
+                return
+            # A non-outage error means the source actually answered (e.g. a
+            # 404/parse error on a real response), so the outage is over —
+            # clear the streak and handle this as a normal per-item error.
+            self._outage_streak = 0
+            logger.error(f"Error processing {item['type']} #{item['id']}: {e}")
+            self.stats['errors'] += 1
+            try:
+                self._mark_status(item['type'], item['id'], 'error')
+            except Exception as e2:
+                logger.error(f"Error updating item status: {e2}")
+
+    def _get_existing_id(self, entity_type: str, entity_id: int) -> Optional[str]:
+        table_map = {'artist': 'artists', 'album': 'albums', 'track': 'tracks'}
+        table = table_map.get(entity_type)
+        if not table:
+            return None
+        conn = None
+        try:
+            conn = self.db._get_connection()
+            cursor = conn.cursor()
+            cursor.execute(f"SELECT amazon_id FROM {table} WHERE id = ?", (entity_id,))
+            row = cursor.fetchone()
+            return row[0] if row and row[0] else None
+        except Exception:
+            return None
+        finally:
+            if conn:
+                conn.close()
+
+    def _process_artist(self, artist_id: int, artist_name: str):
+        existing_id = self._get_existing_id('artist', artist_id)
+        if existing_id:
+            logger.debug(f"Preserving existing Amazon ID for artist '{artist_name}': {existing_id}")
+            return
+
+        results = self.client.search_artists(artist_name, limit=5)
+        if results:
+            result = results[0]
+            if self._name_matches(artist_name, result.name):
+                self._update_artist(artist_id, result)
+                self.stats['matched'] += 1
+                logger.info(f"Matched artist '{artist_name}' -> Amazon ID: {result.id}")
+            else:
+                self._mark_status('artist', artist_id, 'not_found')
+                self.stats['not_found'] += 1
+                logger.debug(f"Name mismatch for artist '{artist_name}' (got '{result.name}')")
+        else:
+            self._mark_status('artist', artist_id, 'not_found')
+            self.stats['not_found'] += 1
+            logger.debug(f"No match for artist '{artist_name}'")
+
+    def _refresh_album_via_stored_id(self, album_id, stored_id, api_data):
+        self._update_album(album_id, api_data, stored_id)
+
+    def _refresh_track_via_stored_id(self, track_id, stored_id, api_data):
+        self._update_track(track_id, api_data, stored_id)
+
+    def _process_album(self, album_id: int, album_name: str, artist_name: str, item: Dict[str, Any]):
+        if honor_stored_match(
+            db=self.db, entity_table='albums', entity_id=album_id,
+            id_column='amazon_id',
+            client_fetch_fn=lambda asin: self.client.get_album(asin, include_tracks=False),
+            on_match_fn=self._refresh_album_via_stored_id,
+            log_prefix='Amazon',
+        ):
+            self.stats['matched'] += 1
+            return
+
+        query = f"{artist_name} {album_name}"
+        results = self.client.search_albums(query, limit=10)
+        if results:
+            result = results[0]
+            if self._name_matches(album_name, result.name):
+                full_album = None
+                if result.id:
+                    try:
+                        full_album = self.client.get_album(result.id, include_tracks=False)
+                    except Exception as e:
+                        logger.warning(f"Failed to fetch full album '{album_name}' (ASIN: {result.id}): {e}")
+
+                if full_album is None:
+                    self._mark_status('album', album_id, 'error')
+                    self.stats['errors'] += 1
+                    logger.warning(f"Album '{album_name}' matched but full details unavailable, will retry")
+                    return
+
+                self._update_album(album_id, full_album, result.id)
+                self.stats['matched'] += 1
+                logger.info(f"Matched album '{album_name}' -> Amazon ASIN: {result.id}")
+            else:
+                self._mark_status('album', album_id, 'not_found')
+                self.stats['not_found'] += 1
+                logger.debug(f"Name mismatch for album '{album_name}' (got '{result.name}')")
+        else:
+            self._mark_status('album', album_id, 'not_found')
+            self.stats['not_found'] += 1
+            logger.debug(f"No match for album '{album_name}'")
+
+    def _process_track(self, track_id: int, track_name: str, artist_name: str, item: Dict[str, Any]):
+        if honor_stored_match(
+            db=self.db, entity_table='tracks', entity_id=track_id,
+            id_column='amazon_id',
+            client_fetch_fn=self.client.get_track_details,
+            on_match_fn=self._refresh_track_via_stored_id,
+            log_prefix='Amazon',
+        ):
+            self.stats['matched'] += 1
+            return
+
+        query = f"{artist_name} {track_name}"
+        results = self.client.search_tracks(query, limit=10)
+        if results:
+            result = results[0]
+            if self._name_matches(track_name, result.name):
+                full_track = None
+                if result.id:
+                    try:
+                        full_track = self.client.get_track_details(result.id)
+                    except Exception as e:
+                        logger.warning(f"Failed to fetch full track '{track_name}' (ASIN: {result.id}): {e}")
+
+                if full_track is None:
+                    self._mark_status('track', track_id, 'error')
+                    self.stats['errors'] += 1
+                    logger.warning(f"Track '{track_name}' matched but full details unavailable, will retry")
+                    return
+
+                self._update_track(track_id, full_track, result.id)
+                self.stats['matched'] += 1
+                logger.info(f"Matched track '{track_name}' -> Amazon ASIN: {result.id}")
+            else:
+                self._mark_status('track', track_id, 'not_found')
+                self.stats['not_found'] += 1
+                logger.debug(f"Name mismatch for track '{track_name}' (got '{result.name}')")
+        else:
+            self._mark_status('track', track_id, 'not_found')
+            self.stats['not_found'] += 1
+            logger.debug(f"No match for track '{track_name}'")
+
+    def _update_artist(self, artist_id: int, result):
+        """Store Amazon metadata for an artist. ``result`` is an Artist dataclass."""
+        conn = None
+        try:
+            conn = self.db._get_connection()
+            cursor = conn.cursor()
+
+            cursor.execute("""
+                UPDATE artists SET
+                    amazon_id = ?,
+                    amazon_match_status = 'matched',
+                    amazon_last_attempted = CURRENT_TIMESTAMP,
+                    updated_at = CURRENT_TIMESTAMP
+                WHERE id = ?
+            """, (str(result.id), artist_id))
+
+            # Backfill thumb_url from album cover stand-in when artist has no image
+            image_url = result.image_url
+            if not image_url:
+                try:
+                    image_url = self.client._get_artist_image_from_albums(result.id)
+                except Exception as exc:
+                    logger.debug("Artist image from albums failed for %s: %s", result.id, exc)
+            if image_url:
+                cursor.execute("""
+                    UPDATE artists SET thumb_url = ?
+                    WHERE id = ? AND (thumb_url IS NULL OR thumb_url = '')
+                """, (image_url, artist_id))
+
+            conn.commit()
+
+        except Exception as e:
+            logger.error(f"Error updating artist #{artist_id} with Amazon data: {e}")
+            raise
+        finally:
+            if conn:
+                conn.close()
+
+    def _update_album(self, album_id: int, full_data: Dict[str, Any], asin: str):
+        """Store Amazon metadata for an album. ``full_data`` is a get_album() dict."""
+        conn = None
+        try:
+            conn = self.db._get_connection()
+            cursor = conn.cursor()
+
+            cursor.execute("""
+                UPDATE albums SET
+                    amazon_id = ?,
+                    amazon_match_status = 'matched',
+                    amazon_last_attempted = CURRENT_TIMESTAMP,
+                    updated_at = CURRENT_TIMESTAMP
+                WHERE id = ?
+            """, (asin, album_id))
+
+            # Backfill label when missing
+            label = full_data.get('label')
+            if label:
+                cursor.execute("""
+                    UPDATE albums SET label = ?
+                    WHERE id = ? AND (label IS NULL OR label = '')
+                """, (label, album_id))
+
+            # Backfill thumb_url
+            images = full_data.get('images') or []
+            thumb_url = images[0].get('url') if images else None
+            if thumb_url:
+                cursor.execute("""
+                    UPDATE albums SET thumb_url = ?
+                    WHERE id = ? AND (thumb_url IS NULL OR thumb_url = '')
+                """, (thumb_url, album_id))
+
+            # Cache authoritative track count for completeness repair
+            total_tracks = full_data.get('total_tracks') or (
+                full_data.get('tracks', {}).get('total') if isinstance(full_data.get('tracks'), dict) else None
+            )
+            set_album_api_track_count(cursor, album_id, total_tracks)
+
+            conn.commit()
+
+        except Exception as e:
+            logger.error(f"Error updating album #{album_id} with Amazon data: {e}")
+            raise
+        finally:
+            if conn:
+                conn.close()
+
+    def _update_track(self, track_id: int, full_data: Dict[str, Any], asin: str):
+        """Store Amazon metadata for a track. ``full_data`` is a get_track_details() dict."""
+        conn = None
+        try:
+            conn = self.db._get_connection()
+            cursor = conn.cursor()
+
+            cursor.execute("""
+                UPDATE tracks SET
+                    amazon_id = ?,
+                    amazon_match_status = 'matched',
+                    amazon_last_attempted = CURRENT_TIMESTAMP
+                WHERE id = ?
+            """, (asin, track_id))
+
+            conn.commit()
+
+        except Exception as e:
+            logger.error(f"Error updating track #{track_id} with Amazon data: {e}")
+            raise
+        finally:
+            if conn:
+                conn.close()
+
+    def _mark_status(self, entity_type: str, entity_id: int, status: str):
+        table_map = {'artist': 'artists', 'album': 'albums', 'track': 'tracks'}
+        table = table_map.get(entity_type)
+        if not table:
+            logger.error(f"Unknown entity type: {entity_type}")
+            return
+
+        conn = None
+        try:
+            conn = self.db._get_connection()
+            cursor = conn.cursor()
+            cursor.execute(f"""
+                UPDATE {table} SET
+                    amazon_match_status = ?,
+                    amazon_last_attempted = CURRENT_TIMESTAMP,
+                    updated_at = CURRENT_TIMESTAMP
+                WHERE id = ?
+            """, (status, entity_id))
+            conn.commit()
+        except Exception as e:
+            logger.error(f"Error marking {entity_type} #{entity_id} status: {e}")
+        finally:
+            if conn:
+                conn.close()
+
+    def _count_pending_items(self) -> int:
+        conn = None
+        try:
+            conn = self.db._get_connection()
+            cursor = conn.cursor()
+            self._ensure_amazon_schema(cursor)
+            cursor.execute("""
+                SELECT
+                    (SELECT COUNT(*) FROM artists WHERE amazon_match_status IS NULL AND id IS NOT NULL) +
+                    (SELECT COUNT(*) FROM albums  WHERE amazon_match_status IS NULL AND id IS NOT NULL) +
+                    (SELECT COUNT(*) FROM tracks  WHERE amazon_match_status IS NULL AND id IS NOT NULL)
+                AS pending
+            """)
+            row = cursor.fetchone()
+            return row[0] if row else 0
+        except Exception as e:
+            logger.error(f"Error counting pending items: {e}")
+            return 0
+        finally:
+            if conn:
+                conn.close()
+
+    def _get_progress_breakdown(self) -> Dict[str, Dict[str, int]]:
+        conn = None
+        try:
+            conn = self.db._get_connection()
+            cursor = conn.cursor()
+            self._ensure_amazon_schema(cursor)
+            progress = {}
+
+            for table in ('artists', 'albums', 'tracks'):
+                cursor.execute(f"""
+                    SELECT
+                        COUNT(*) AS total,
+                        SUM(CASE WHEN amazon_match_status IS NOT NULL THEN 1 ELSE 0 END) AS processed
+                    FROM {table}
+                """)
+                row = cursor.fetchone()
+                if row:
+                    total, processed = row[0], row[1] or 0
+                    progress[table] = {
+                        'matched': processed,
+                        'total': total,
+                        'percent': int((processed / total * 100) if total > 0 else 0),
+                    }
+
+            return progress
+
+        except Exception as e:
+            logger.error(f"Error getting progress breakdown: {e}")
+            return {}
+        finally:
+            if conn:
+                conn.close()

core/api_call_tracker.py

@@ -30,6 +30,7 @@ RATE_LIMITS = {
     'tidal': 120,         # MIN_API_INTERVAL=0.5s → ~120/min
     'qobuz': 60,          # Variable throttle, ~60/min estimate
     'discogs': 60,        # MIN_API_INTERVAL=1.0s with auth → ~60/min
+    'amazon': 120,        # MIN_API_INTERVAL=0.5s → ~120/min (T2Tunes proxy)
 }
 
 # Display names for UI
@@ -44,12 +45,13 @@ SERVICE_LABELS = {
     'tidal': 'Tidal',
     'qobuz': 'Qobuz',
     'discogs': 'Discogs',
+    'amazon': 'Amazon Music',
 }
 
 # Display order
 SERVICE_ORDER = [
     'spotify', 'itunes', 'deezer', 'lastfm', 'genius',
-    'musicbrainz', 'audiodb', 'tidal', 'qobuz', 'discogs',
+    'musicbrainz', 'audiodb', 'tidal', 'qobuz', 'discogs', 'amazon',
 ]
 
 
@@ -302,8 +304,8 @@ class ApiCallTracker:
             try:
                 if os.path.exists(_PERSIST_PATH + '.tmp'):
                     os.remove(_PERSIST_PATH + '.tmp')
-            except Exception:
-                pass
+            except Exception as e:
+                logger.debug("remove stale tmp file failed: %s", e)
 
     def _load(self):
         """Restore 24h minute history from disk. Called on init."""

core/archive_pipeline.py

@@ -0,0 +1,238 @@
+"""Archive extraction + audio-file discovery for torrent / usenet downloads.
+
+The torrent and usenet download plugins need a uniform way to:
+
+1. Walk the downloader's save directory and find every audio file in it.
+2. If the directory contains an archive (``.zip`` / ``.rar`` / ``.tar`` /
+   ``.7z``), extract it first so the audio files inside become walkable.
+
+This module is intentionally narrow — no matching, no tagging, no
+import. The download plugin layer composes this with the existing
+post-processing / matching pipeline. Lidarr does NOT use this module:
+Lidarr extracts archives in its own import step before SoulSync sees
+the files at all. Usenet downloaders (SABnzbd, NZBGet) also auto-
+extract by default. Torrents are the main case where SoulSync may
+need to do the extract step itself — most music torrents ship loose,
+but some bundle the album in a ``.rar`` archive.
+
+``rarfile`` is an optional dependency. If it isn't installed, archives
+with ``.rar`` content are skipped with a single warning rather than
+crashing the download.
+"""
+
+from __future__ import annotations
+
+import tarfile
+import zipfile
+from pathlib import Path
+from typing import List, Optional
+
+from utils.logging_config import get_logger
+
+logger = get_logger("archive_pipeline")
+
+
+# Same audio-extension set as ``core/imports/file_ops.py`` ``quality_tiers``.
+# Keep them in sync — if a new format is added to file_ops, add it here too
+# or the walker will skip it and the download plugin will mark the download
+# failed even when files arrived.
+AUDIO_EXTENSIONS = frozenset([
+    # lossless
+    '.flac', '.ape', '.wav', '.alac', '.dsf', '.dff', '.aiff', '.aif',
+    # high lossy
+    '.opus', '.ogg',
+    # standard lossy
+    '.m4a', '.aac',
+    # low lossy
+    '.mp3', '.wma',
+])
+
+ARCHIVE_EXTENSIONS = frozenset(['.zip', '.rar', '.tar', '.tar.gz', '.tgz', '.7z'])
+
+
+def is_archive(path: Path) -> bool:
+    """True if the file extension looks like a supported archive.
+
+    Compound extensions (``.tar.gz``, ``.tar.bz2``) are detected by
+    checking the last two suffixes joined together — Path.suffix
+    only returns the final suffix.
+    """
+    if not path.is_file():
+        return False
+    name = path.name.lower()
+    if name.endswith(('.tar.gz', '.tar.bz2', '.tar.xz')):
+        return True
+    return path.suffix.lower() in ARCHIVE_EXTENSIONS
+
+
+def walk_audio_files(directory: Path) -> List[Path]:
+    """Recursively scan ``directory`` for audio files. Returns
+    a sorted list of absolute paths. Empty list if the directory
+    doesn't exist or contains no audio.
+    """
+    if not directory or not directory.exists() or not directory.is_dir():
+        return []
+    out: List[Path] = []
+    for child in directory.rglob('*'):
+        if not child.is_file():
+            continue
+        if child.suffix.lower() in AUDIO_EXTENSIONS:
+            out.append(child.resolve())
+    out.sort()
+    return out
+
+
+def find_archives_in_dir(directory: Path) -> List[Path]:
+    """Find every archive file directly inside ``directory`` (one
+    level deep — torrents normally put the archive at the root of
+    their folder; we don't search nested dirs to avoid extracting
+    something we shouldn't).
+    """
+    if not directory or not directory.exists() or not directory.is_dir():
+        return []
+    return sorted(p for p in directory.iterdir() if is_archive(p))
+
+
+def extract_archive(archive_path: Path, extract_to: Optional[Path] = None) -> Optional[Path]:
+    """Extract a single archive in-place (or to ``extract_to`` if
+    given). Returns the directory the archive was extracted into,
+    or ``None`` on failure.
+
+    Supports ``.zip``, ``.tar``/``.tar.gz``/``.tar.bz2``/``.tar.xz``,
+    and ``.rar`` (only when the optional ``rarfile`` library is
+    installed). ``.7z`` is recognised but extraction requires
+    ``py7zr``; without it, the call logs and returns None.
+    """
+    if not archive_path or not archive_path.exists():
+        logger.warning("archive_pipeline: %s does not exist", archive_path)
+        return None
+    dest = extract_to or archive_path.parent
+    dest.mkdir(parents=True, exist_ok=True)
+
+    name = archive_path.name.lower()
+    try:
+        if name.endswith('.zip'):
+            with zipfile.ZipFile(archive_path) as zf:
+                _safe_extract_zip(zf, dest)
+            return dest
+        if name.endswith(('.tar', '.tar.gz', '.tgz', '.tar.bz2', '.tar.xz')):
+            with tarfile.open(archive_path) as tf:
+                _safe_extract_tar(tf, dest)
+            return dest
+        if name.endswith('.rar'):
+            return _extract_rar(archive_path, dest)
+        if name.endswith('.7z'):
+            return _extract_7z(archive_path, dest)
+    except (zipfile.BadZipFile, tarfile.TarError, OSError) as e:
+        logger.error("archive_pipeline: failed to extract %s: %s", archive_path, e)
+        return None
+    logger.warning("archive_pipeline: unknown archive type for %s", archive_path)
+    return None
+
+
+def extract_all_in_dir(directory: Path) -> List[Path]:
+    """Find every archive in ``directory`` and extract each in place.
+    Returns the list of directories archives were extracted into
+    (usually all the same — ``directory`` itself). Archives that
+    failed to extract are skipped silently after a warning.
+    """
+    out: List[Path] = []
+    for archive in find_archives_in_dir(directory):
+        result = extract_archive(archive)
+        if result is not None:
+            out.append(result)
+    return out
+
+
+def collect_audio_after_extraction(directory: Path) -> List[Path]:
+    """One-shot helper for the download plugins: extract any archives
+    in the directory, then return the walked audio file list. This is
+    the common pattern — torrent / usenet plugin gets a save_path,
+    calls this, hands the resulting files to the matching pipeline.
+    """
+    extract_all_in_dir(directory)
+    return walk_audio_files(directory)
+
+
+# ---------------------------------------------------------------------------
+# Safety helpers
+# ---------------------------------------------------------------------------
+
+
+def _safe_extract_zip(zf: zipfile.ZipFile, dest: Path) -> None:
+    """Extract a zipfile after rejecting any member whose resolved
+    path escapes ``dest`` (path traversal protection).
+    """
+    dest = dest.resolve()
+    for member in zf.namelist():
+        target = (dest / member).resolve()
+        if dest not in target.parents and target != dest:
+            logger.error("archive_pipeline: refusing path-traversal member %r", member)
+            return
+    zf.extractall(dest)
+
+
+def _safe_extract_tar(tf: tarfile.TarFile, dest: Path) -> None:
+    """Same path-traversal protection for tarfiles."""
+    dest = dest.resolve()
+    for member in tf.getmembers():
+        target = (dest / member.name).resolve()
+        if dest not in target.parents and target != dest:
+            logger.error("archive_pipeline: refusing path-traversal member %r", member.name)
+            return
+    # ``filter='data'`` is the Python 3.12+ safe extractor; fall back
+    # to the legacy call on older runtimes.
+    try:
+        tf.extractall(dest, filter='data')  # type: ignore[call-arg]
+    except TypeError:
+        tf.extractall(dest)
+
+
+def _extract_rar(archive_path: Path, dest: Path) -> Optional[Path]:
+    try:
+        import rarfile  # type: ignore[import-untyped]
+    except ImportError:
+        logger.warning(
+            "archive_pipeline: cannot extract %s — rarfile library not installed. "
+            "Install with: pip install rarfile (and ensure unrar is on PATH).",
+            archive_path,
+        )
+        return None
+    try:
+        with rarfile.RarFile(archive_path) as rf:
+            dest_resolved = dest.resolve()
+            for name in rf.namelist():
+                target = (dest_resolved / name).resolve()
+                if dest_resolved not in target.parents and target != dest_resolved:
+                    logger.error("archive_pipeline: refusing path-traversal rar member %r", name)
+                    return None
+            rf.extractall(dest)
+        return dest
+    except Exception as e:
+        logger.error("archive_pipeline: rar extract failed for %s: %s", archive_path, e)
+        return None
+
+
+def _extract_7z(archive_path: Path, dest: Path) -> Optional[Path]:
+    try:
+        import py7zr  # type: ignore[import-untyped]
+    except ImportError:
+        logger.warning(
+            "archive_pipeline: cannot extract %s — py7zr library not installed. "
+            "Install with: pip install py7zr.",
+            archive_path,
+        )
+        return None
+    try:
+        with py7zr.SevenZipFile(archive_path, 'r') as sz:
+            dest_resolved = dest.resolve()
+            for name in sz.getnames():
+                target = (dest_resolved / name).resolve()
+                if dest_resolved not in target.parents and target != dest_resolved:
+                    logger.error("archive_pipeline: refusing path-traversal 7z member %r", name)
+                    return None
+            sz.extractall(path=dest)
+        return dest
+    except Exception as e:
+        logger.error("archive_pipeline: 7z extract failed for %s: %s", archive_path, e)
+        return None

core/artist_source_detail.py

@@ -43,6 +43,7 @@ def build_source_only_artist_detail(
     deezer_client: Optional[Any] = None,
     itunes_client: Optional[Any] = None,
     discogs_client: Optional[Any] = None,
+    amazon_client: Optional[Any] = None,
     lastfm_api_key: Optional[str] = None,
 ) -> Tuple[Dict[str, Any], int]:
     """Build the artist-detail payload for a source-only artist.
@@ -51,7 +52,7 @@ def build_source_only_artist_detail(
     ``jsonify`` or equivalent. Status is 200 on success, 404 when the
     source's discography lookup returned no releases.
     """
-    resolved_name = (artist_name or artist_id or "").strip()
+    resolved_name = (artist_name or "").strip()
 
     # 1. Image URL via the same helper /api/artist/<id>/image uses.
     image_url: Optional[str] = None
@@ -67,6 +68,8 @@ def build_source_only_artist_detail(
         if source == "spotify" and spotify_client is not None:
             sp_artist = spotify_client.get_artist(artist_id, allow_fallback=False)
             if sp_artist:
+                if not artist_name and sp_artist.get("name"):
+                    resolved_name = sp_artist["name"]
                 source_genres = sp_artist.get("genres") or []
                 source_followers = (sp_artist.get("followers") or {}).get("total")
                 if not image_url and sp_artist.get("images"):
@@ -74,16 +77,41 @@ def build_source_only_artist_detail(
         elif source == "deezer" and deezer_client is not None:
             dz_artist = deezer_client.get_artist_info(artist_id)
             if dz_artist:
+                if not artist_name and dz_artist.get("name"):
+                    resolved_name = dz_artist["name"]
                 source_genres = dz_artist.get("genres") or []
                 source_followers = (dz_artist.get("followers") or {}).get("total")
         elif source == "itunes" and itunes_client is not None:
             it_artist = itunes_client.get_artist(artist_id)
             if it_artist:
+                if not artist_name and it_artist.get("name"):
+                    resolved_name = it_artist["name"]
                 source_genres = it_artist.get("genres") or []
         elif source == "discogs" and discogs_client is not None:
             dc_artist = discogs_client.get_artist(artist_id)
             if dc_artist:
+                if not artist_name and dc_artist.get("name"):
+                    resolved_name = dc_artist["name"]
                 source_genres = dc_artist.get("genres") or []
+        elif source == "amazon" and amazon_client is not None:
+            az_artist = amazon_client.get_artist(resolved_name or artist_id)
+            if az_artist:
+                if not artist_name and az_artist.get("name"):
+                    resolved_name = az_artist["name"]
+                source_genres = az_artist.get("genres") or []
+                if not image_url and az_artist.get("images"):
+                    image_url = az_artist["images"][0].get("url")
+        elif source == "musicbrainz":
+            try:
+                from core.musicbrainz_search import MusicBrainzSearchClient
+                mb = MusicBrainzSearchClient()
+                mb_artist = mb.get_artist(artist_id)
+                if mb_artist:
+                    if not artist_name and mb_artist.get("name"):
+                        resolved_name = mb_artist["name"]
+                    source_genres = mb_artist.get("genres") or []
+            except Exception as e:
+                logger.debug(f"MusicBrainz artist info lookup failed for {artist_id}: {e}")
     except Exception as e:
         logger.debug(f"Source-side artist info lookup failed for {source}:{artist_id}: {e}")
 
@@ -126,7 +154,11 @@ def build_source_only_artist_detail(
             allow_fallback=True,
             skip_cache=False,
             max_pages=0,
-            limit=50,
+            # Match the Download Discography endpoint cap (200).
+            # Spotify already paginates all; Deezer / iTunes / Discogs /
+            # Hydrabase clamp at the outer limit. 200 covers prolific
+            # catalogues without exceeding iTunes/Discogs internal caps.
+            limit=200,
             artist_source_ids={source: artist_id},
             dedup_variants=False,
         ),

core/artist_source_lookup.py

@@ -21,21 +21,24 @@ from __future__ import annotations
 import logging
 from typing import Optional
 
+from core.source_ids import id_column as _artist_id_column
+
 logger = logging.getLogger("artist_source_lookup")
 
 
 SOURCE_ONLY_ARTIST_SOURCES = frozenset({
-    "spotify", "itunes", "deezer", "discogs", "hydrabase", "musicbrainz",
+    "spotify", "itunes", "deezer", "discogs", "hydrabase", "musicbrainz", "amazon",
 })
 
 
+# The per-source column on the ``artists`` table, derived from the canonical
+# source-ID registry (the single source of truth). Values are unchanged from the
+# previous hardcoded map — this just stops duplicating that knowledge here.
 SOURCE_ID_FIELD = {
-    "spotify": "spotify_artist_id",
-    "itunes": "itunes_artist_id",
-    "deezer": "deezer_id",
-    "discogs": "discogs_id",
-    "hydrabase": "soul_id",
-    "musicbrainz": "musicbrainz_id",
+    source: _artist_id_column(source, "artist")
+    for source in (
+        "spotify", "itunes", "deezer", "discogs", "hydrabase", "musicbrainz", "amazon",
+    )
 }
 
 
@@ -50,7 +53,11 @@ def find_library_artist_for_source(
 
     Lookup order:
       1. Direct match on the source-specific ID column (server-agnostic — any
-         library record with the right external ID is a hit).
+         library record with the right external ID is a hit). If that id is
+         stamped on MORE than one library artist, the mapping is corrupt /
+         ambiguous (e.g. an enrichment bug wrote one Deezer id onto several
+         artists) — we refuse to guess and fall through, so the caller can
+         show the source artist directly instead of an arbitrary wrong one.
       2. Case-insensitive name match within ``active_server`` (defaults to the
          active media server when not provided), so we don't jump the user
          across server contexts on a name collision.
@@ -64,13 +71,23 @@ def find_library_artist_for_source(
     try:
         with database._get_connection() as conn:
             cursor = conn.cursor()
+            # LIMIT 2 so we can tell a unique match from an ambiguous one.
             cursor.execute(
-                f"SELECT id, name FROM artists WHERE {column} = ? LIMIT 1",
+                f"SELECT id FROM artists WHERE {column} = ? LIMIT 2",
                 (str(source_artist_id),),
             )
-            row = cursor.fetchone()
-            if row:
-                return row[0]
+            rows = cursor.fetchall()
+            if len(rows) == 1:
+                return rows[0][0]
+            if len(rows) > 1:
+                # Same source id on multiple artists — corrupt mapping. Don't
+                # upgrade on the id; fall through to the name match (and, if
+                # that misses, let the caller render the source artist).
+                logger.warning(
+                    f"Source id {source}:{source_artist_id} maps to "
+                    f"{len(rows)}+ library artists — ambiguous, skipping "
+                    f"id-based library upgrade"
+                )
 
             if artist_name and active_server:
                 cursor.execute(

core/artists/liked_match.py

@@ -12,6 +12,7 @@ from core.metadata.registry import (
     get_deezer_client,
     get_discogs_client,
     get_itunes_client,
+    get_musicbrainz_client,
     get_spotify_client,
 )
 
@@ -33,6 +34,11 @@ def _get_discogs_client(token=None):
     return get_discogs_client(token)
 
 
+def _get_musicbrainz_client():
+    """Mirror of web_server._get_musicbrainz_client — delegates to registry."""
+    return get_musicbrainz_client()
+
+
 class _SpotifyClientProxy:
     """Resolves the global Spotify client lazily so a Spotify re-auth that
     rebinds the cached client in core.metadata.registry is visible to the
@@ -65,6 +71,7 @@ def _match_liked_artists_to_all_sources(database, profile_id: int):
         'itunes': 'itunes_artist_id',
         'deezer': 'deezer_artist_id',
         'discogs': 'discogs_artist_id',
+        'musicbrainz': 'musicbrainz_artist_id',
     }
     id_cols = list(source_cols.values())
 
@@ -89,20 +96,24 @@ def _match_liked_artists_to_all_sources(database, profile_id: int):
         search_clients['spotify'] = spotify_client
     try:
         search_clients['itunes'] = _get_itunes_client()
-    except Exception:
-        pass
+    except Exception as e:
+        logger.debug("itunes client init failed: %s", e)
     try:
         search_clients['deezer'] = _get_deezer_client()
-    except Exception:
-        pass
+    except Exception as e:
+        logger.debug("deezer client init failed: %s", e)
     try:
         dc = _get_discogs_client()
         # Only use Discogs if token is configured
         from config.settings import config_manager as _cm
         if _cm.get('discogs.token', ''):
             search_clients['discogs'] = dc
-    except Exception:
-        pass
+    except Exception as e:
+        logger.debug("discogs client init failed: %s", e)
+    try:
+        search_clients['musicbrainz'] = _get_musicbrainz_client()
+    except Exception as e:
+        logger.debug("musicbrainz client init failed: %s", e)
 
     # Reuse watchlist scanner's fuzzy matching logic
     from core.watchlist_scanner import WatchlistScanner
@@ -167,8 +178,8 @@ def _match_liked_artists_to_all_sources(database, profile_id: int):
                         harvested_ids[col] = str(r[col])
                 if _valid_image(r.get('thumb_url')):
                     best_image = r['thumb_url']
-        except Exception:
-            pass
+        except Exception as e:
+            logger.debug("library artist lookup failed: %s", e)
 
         # 2. Watchlist artists
         try:
@@ -186,8 +197,8 @@ def _match_liked_artists_to_all_sources(database, profile_id: int):
                         harvested_ids[col] = str(wl[col])
                 if _valid_image(wl.get('image_url')) and not best_image:
                     best_image = wl['image_url']
-        except Exception:
-            pass
+        except Exception as e:
+            logger.debug("watchlist artist lookup failed: %s", e)
 
         # 3. Metadata cache (all sources)
         try:
@@ -203,8 +214,8 @@ def _match_liked_artists_to_all_sources(database, profile_id: int):
                     harvested_ids[col] = row['entity_id']
                 if _valid_image(row['image_url']) and not best_image:
                     best_image = row['image_url']
-        except Exception:
-            pass
+        except Exception as e:
+            logger.debug("metadata cache lookup failed: %s", e)
 
         # --- API STRATEGIES (search each missing source) ---
         # Same pattern as watchlist scanner's _backfill_missing_ids
@@ -234,7 +245,7 @@ def _match_liked_artists_to_all_sources(database, profile_id: int):
             # Determine best active source/ID — prefer Spotify, then iTunes, Deezer, Discogs
             resolved_source = None
             resolved_id = None
-            for src in ('spotify', 'itunes', 'deezer', 'discogs'):
+            for src in ('spotify', 'itunes', 'deezer', 'discogs', 'musicbrainz'):
                 col = source_cols[src]
                 if col in harvested_ids:
                     resolved_source = src
@@ -287,8 +298,8 @@ def _backfill_liked_artist_images(database, profile_id: int, search_clients: dic
                         artist_data = sp.sp.artist(r['spotify_artist_id'])
                         if artist_data and artist_data.get('images'):
                             image_url = artist_data['images'][0]['url']
-                except Exception:
-                    pass
+                except Exception as e:
+                    logger.debug("spotify artist image fetch failed: %s", e)
 
             # Try Deezer (direct image URL from ID)
             if not image_url and r.get('deezer_artist_id'):
@@ -302,8 +313,8 @@ def _backfill_liked_artist_images(database, profile_id: int, search_clients: dic
                         (image_url, r['id'])
                     )
                     filled += 1
-                except Exception:
-                    pass
+                except Exception as e:
+                    logger.debug("liked artist image update failed: %s", e)
                 time.sleep(0.3)
 
         conn.commit()

core/artists/map.py

@@ -20,10 +20,14 @@ logger = logging.getLogger(__name__)
 
 
 def get_current_profile_id() -> int:
-    """Mirror of web_server.get_current_profile_id — uses Flask g."""
+    """Mirror of web_server.get_current_profile_id — uses Flask g.
+
+    Catches RuntimeError too because reading `g` outside a request
+    context raises that (not AttributeError) — happens when this is
+    called from background threads (sync, automation, scanners)."""
     try:
         return g.profile_id
-    except AttributeError:
+    except (AttributeError, RuntimeError):
         return 1
 
 
@@ -136,7 +140,7 @@ def get_artist_map_data():
             placeholders = ','.join(['?'] * len(watchlist_ids))
             cursor.execute(f"""
                 SELECT source_artist_id, similar_artist_name, similar_artist_spotify_id,
-                       similar_artist_itunes_id, similar_artist_deezer_id,
+                       similar_artist_itunes_id, similar_artist_deezer_id, similar_artist_musicbrainz_id,
                        similarity_rank, occurrence_count, image_url, genres, popularity
                 FROM similar_artists
                 WHERE profile_id = ? AND source_artist_id IN ({placeholders})
@@ -158,8 +162,8 @@ def get_artist_map_data():
                     if r.get('genres'):
                         try:
                             genres = json.loads(r['genres'])
-                        except Exception:
-                            pass
+                        except Exception as e:
+                            logger.debug("similar node genres parse failed: %s", e)
                     nodes.append({
                         'id': idx,
                         'name': r['similar_artist_name'],
@@ -169,6 +173,7 @@ def get_artist_map_data():
                         'spotify_id': r.get('similar_artist_spotify_id') or '',
                         'itunes_id': r.get('similar_artist_itunes_id') or '',
                         'deezer_id': r.get('similar_artist_deezer_id') or '',
+                        'musicbrainz_id': r.get('similar_artist_musicbrainz_id') or '',
                         'rank': r.get('similarity_rank', 5),
                         'occurrence': r.get('occurrence_count', 1),
                         'popularity': r.get('popularity', 0),
@@ -232,8 +237,8 @@ def get_artist_map_data():
                     if cr['genres']:
                         try:
                             genres = json.loads(cr['genres']) if isinstance(cr['genres'], str) else []
-                        except Exception:
-                            pass
+                        except Exception as e:
+                            logger.debug("backfill cache genres parse failed: %s", e)
                     cache_by_name[cn][source] = {
                         'id': cr['entity_id'],
                         'image_url': cr['image_url'] or '',
@@ -241,7 +246,7 @@ def get_artist_map_data():
                     }
 
                 # Apply cache data to nodes
-                source_id_map = {'spotify': 'spotify_id', 'itunes': 'itunes_id', 'deezer': 'deezer_id', 'discogs': 'discogs_id'}
+                source_id_map = {'spotify': 'spotify_id', 'itunes': 'itunes_id', 'deezer': 'deezer_id', 'discogs': 'discogs_id', 'musicbrainz': 'musicbrainz_id'}
                 for n in nodes:
                     nn = _norm(n['name'])
                     cached = cache_by_name.get(nn)
@@ -258,7 +263,7 @@ def get_artist_map_data():
                                 break
                     # Backfill genres if missing
                     if not n.get('genres') or len(n.get('genres', [])) == 0:
-                        for source in ('spotify', 'deezer', 'itunes', 'discogs'):
+                        for source in ('spotify', 'deezer', 'itunes', 'discogs', 'musicbrainz'):
                             if source in cached and cached[source].get('genres'):
                                 n['genres'] = cached[source]['genres'][:5]
                                 break
@@ -280,8 +285,8 @@ def get_artist_map_data():
                         an = _norm(r['artist_name'])
                         if an and an not in _album_art:
                             _album_art[an] = r['image_url']
-                except Exception:
-                    pass
+                except Exception as e:
+                    logger.debug("artist map album-art cache build failed: %s", e)
                 for n in nodes:
                     if not n.get('image_url') or not n['image_url'].startswith('http'):
                         nn = _norm(n['name'])
@@ -330,8 +335,8 @@ def get_artist_map_genre_list():
                     if g and isinstance(g, str):
                         gl = g.lower().strip()
                         genre_counts[gl] = genre_counts.get(gl, 0) + 1
-            except Exception:
-                pass
+            except Exception as e:
+                logger.debug("genre count row parse failed: %s", e)
 
         # Sort by count descending
         sorted_genres = sorted(genre_counts.items(), key=lambda x: -x[1])
@@ -365,14 +370,14 @@ def get_artist_map_genres():
         def _norm(n):
             return (n or '').lower().strip()
 
-        def _add(name, image_url=None, genres=None, spotify_id=None, itunes_id=None, deezer_id=None, discogs_id=None, source='unknown', popularity=0):
+        def _add(name, image_url=None, genres=None, spotify_id=None, itunes_id=None, deezer_id=None, discogs_id=None, musicbrainz_id=None, source='unknown', popularity=0):
             n = _norm(name)
             if not n or len(n) < 2:
                 return
             if n not in artists_by_name:
                 artists_by_name[n] = {
                     'name': name, 'image_url': '', 'genres': set(),
-                    'spotify_id': '', 'itunes_id': '', 'deezer_id': '', 'discogs_id': '',
+                    'spotify_id': '', 'itunes_id': '', 'deezer_id': '', 'discogs_id': '', 'musicbrainz_id': '',
                     'sources': set(), 'popularity': 0
                 }
             a = artists_by_name[n]
@@ -390,6 +395,8 @@ def get_artist_map_genres():
                 a['deezer_id'] = str(deezer_id)
             if discogs_id and not a['discogs_id']:
                 a['discogs_id'] = str(discogs_id)
+            if musicbrainz_id and not a['musicbrainz_id']:
+                a['musicbrainz_id'] = str(musicbrainz_id)
             if popularity > a['popularity']:
                 a['popularity'] = popularity
             a['sources'].add(source)
@@ -404,16 +411,16 @@ def get_artist_map_genres():
             if r['genres']:
                 try:
                     genres = json.loads(r['genres']) if isinstance(r['genres'], str) else []
-                except Exception:
-                    pass
-            src_map = {'spotify': 'spotify_id', 'itunes': 'itunes_id', 'deezer': 'deezer_id', 'discogs': 'discogs_id'}
+                except Exception as e:
+                    logger.debug("cache artist genres parse failed: %s", e)
+            src_map = {'spotify': 'spotify_id', 'itunes': 'itunes_id', 'deezer': 'deezer_id', 'discogs': 'discogs_id', 'musicbrainz': 'musicbrainz_id'}
             kwargs = {src_map.get(r['source'], 'spotify_id'): r['entity_id']}
             _add(r['name'], image_url=r['image_url'], genres=genres, source='cache', popularity=r['popularity'] or 0, **kwargs)
 
         # 2. Similar artists
         cursor.execute("""
             SELECT similar_artist_name, similar_artist_spotify_id, similar_artist_itunes_id,
-                   similar_artist_deezer_id, image_url, genres, popularity
+                   similar_artist_deezer_id, similar_artist_musicbrainz_id, image_url, genres, popularity
             FROM similar_artists WHERE profile_id = ?
         """, (profile_id,))
         for r in cursor.fetchall():
@@ -421,11 +428,13 @@ def get_artist_map_genres():
             if r['genres']:
                 try:
                     genres = json.loads(r['genres']) if isinstance(r['genres'], str) else []
-                except Exception:
-                    pass
+                except Exception as e:
+                    logger.debug("similar artist genres parse failed: %s", e)
             _add(r['similar_artist_name'], image_url=r['image_url'], genres=genres,
                  spotify_id=r['similar_artist_spotify_id'], itunes_id=r['similar_artist_itunes_id'],
-                 deezer_id=r['similar_artist_deezer_id'], source='similar', popularity=r['popularity'] or 0)
+                 deezer_id=r['similar_artist_deezer_id'],
+                 musicbrainz_id=r['similar_artist_musicbrainz_id'] if 'similar_artist_musicbrainz_id' in r.keys() else None,
+                 source='similar', popularity=r['popularity'] or 0)
 
         # 3. Watchlist artists
         cursor.execute("""
@@ -445,8 +454,8 @@ def get_artist_map_genres():
             if r['genres']:
                 try:
                     genres = json.loads(r['genres']) if isinstance(r['genres'], str) else []
-                except Exception:
-                    pass
+                except Exception as e:
+                    logger.debug("library artist genres parse failed: %s", e)
             img = r['thumb_url'] if r['thumb_url'] and r['thumb_url'].startswith('http') else None
             _add(r['name'], image_url=img, genres=genres, source='library')
 
@@ -479,6 +488,7 @@ def get_artist_map_genres():
                 'itunes_id': a['itunes_id'],
                 'deezer_id': a['deezer_id'],
                 'discogs_id': a['discogs_id'],
+                'musicbrainz_id': a['musicbrainz_id'],
                 'popularity': a['popularity'],
                 'type': 'watchlist' if 'watchlist' in a['sources'] else 'similar',
             })
@@ -550,8 +560,8 @@ def get_artist_map_genres():
                 nn = (r['artist_name'] or '').lower().strip()
                 if nn and nn not in _album_art_cache:
                     _album_art_cache[nn] = r['image_url']
-        except Exception:
-            pass
+        except Exception as e:
+            logger.debug("genre map cache build failed: %s", e)
 
         for n in nodes:
             img = n.get('image_url', '')
@@ -622,7 +632,7 @@ def get_artist_map_explore():
         # Find the center artist
         center_name = artist_name
         center_image = ''
-        center_ids = {'spotify_id': '', 'itunes_id': '', 'deezer_id': '', 'discogs_id': ''}
+        center_ids = {'spotify_id': '', 'itunes_id': '', 'deezer_id': '', 'discogs_id': '', 'musicbrainz_id': ''}
         center_genres = []
 
         # Search metadata cache for the center artist
@@ -644,25 +654,25 @@ def get_artist_map_explore():
             center_name = row['name']
             if row['image_url'] and row['image_url'].startswith('http'):
                 center_image = row['image_url']
-            src_map = {'spotify': 'spotify_id', 'itunes': 'itunes_id', 'deezer': 'deezer_id', 'discogs': 'discogs_id'}
+            src_map = {'spotify': 'spotify_id', 'itunes': 'itunes_id', 'deezer': 'deezer_id', 'discogs': 'discogs_id', 'musicbrainz': 'musicbrainz_id'}
             k = src_map.get(row['source'], 'spotify_id')
             center_ids[k] = row['entity_id']
             if row['genres']:
                 try:
                     center_genres = json.loads(row['genres']) if isinstance(row['genres'], str) else []
-                except Exception:
-                    pass
+                except Exception as e:
+                    logger.debug("initial center genres parse failed: %s", e)
 
         # Check watchlist + library if not in cache
         if not artist_found and not artist_id:
-            cursor.execute("SELECT artist_name, image_url, spotify_artist_id, itunes_artist_id, deezer_artist_id, discogs_artist_id FROM watchlist_artists WHERE artist_name = ? COLLATE NOCASE LIMIT 1", (artist_name,))
+            cursor.execute("SELECT artist_name, image_url, spotify_artist_id, itunes_artist_id, deezer_artist_id, discogs_artist_id, musicbrainz_artist_id FROM watchlist_artists WHERE artist_name = ? COLLATE NOCASE LIMIT 1", (artist_name,))
             wr = cursor.fetchone()
             if wr:
                 artist_found = True
                 center_name = wr['artist_name']
                 if wr['image_url'] and str(wr['image_url']).startswith('http'):
                     center_image = wr['image_url']
-                for k, col in [('spotify_id', 'spotify_artist_id'), ('itunes_id', 'itunes_artist_id'), ('deezer_id', 'deezer_artist_id'), ('discogs_id', 'discogs_artist_id')]:
+                for k, col in [('spotify_id', 'spotify_artist_id'), ('itunes_id', 'itunes_artist_id'), ('deezer_id', 'deezer_artist_id'), ('discogs_id', 'discogs_artist_id'), ('musicbrainz_id', 'musicbrainz_artist_id')]:
                     if wr[col]:
                         center_ids[k] = str(wr[col])
             else:
@@ -713,7 +723,7 @@ def get_artist_map_explore():
             WHERE entity_type = 'artist' AND name = ? COLLATE NOCASE
         """, (center_name,))
         for r in cursor.fetchall():
-            src_map = {'spotify': 'spotify_id', 'itunes': 'itunes_id', 'deezer': 'deezer_id', 'discogs': 'discogs_id'}
+            src_map = {'spotify': 'spotify_id', 'itunes': 'itunes_id', 'deezer': 'deezer_id', 'discogs': 'discogs_id', 'musicbrainz': 'musicbrainz_id'}
             k = src_map.get(r['source'], 'spotify_id')
             if not center_ids.get(k):
                 center_ids[k] = r['entity_id']
@@ -722,8 +732,8 @@ def get_artist_map_explore():
             if r['genres'] and not center_genres:
                 try:
                     center_genres = json.loads(r['genres']) if isinstance(r['genres'], str) else []
-                except Exception:
-                    pass
+                except Exception as e:
+                    logger.debug("center genres parse failed: %s", e)
 
         # Add center node
         center_idx = 0
@@ -742,7 +752,7 @@ def get_artist_map_explore():
             placeholders = ','.join(['?'] * len(id_values))
             cursor.execute(f"""
                 SELECT DISTINCT similar_artist_name, similar_artist_spotify_id,
-                       similar_artist_itunes_id, similar_artist_deezer_id,
+                       similar_artist_itunes_id, similar_artist_deezer_id, similar_artist_musicbrainz_id,
                        image_url, genres, popularity, similarity_rank
                 FROM similar_artists
                 WHERE source_artist_id IN ({placeholders}) AND profile_id = ?
@@ -753,7 +763,7 @@ def get_artist_map_explore():
         # Also search by name (the center artist might be a watchlist source)
         cursor.execute("""
             SELECT DISTINCT sa.similar_artist_name, sa.similar_artist_spotify_id,
-                   sa.similar_artist_itunes_id, sa.similar_artist_deezer_id,
+                   sa.similar_artist_itunes_id, sa.similar_artist_deezer_id, sa.similar_artist_musicbrainz_id,
                    sa.image_url, sa.genres, sa.popularity, sa.similarity_rank
             FROM similar_artists sa
             JOIN watchlist_artists wa ON sa.source_artist_id = COALESCE(wa.spotify_artist_id, wa.itunes_artist_id, CAST(wa.id AS TEXT))
@@ -785,16 +795,17 @@ def get_artist_map_explore():
                                     image_url=sa.get('image_url'),
                                     genres=sa.get('genres'),
                                     popularity=sa.get('popularity', 0),
-                                    similar_artist_deezer_id=sa.get('deezer_id')
+                                    similar_artist_deezer_id=sa.get('deezer_id'),
+                                    similar_artist_musicbrainz_id=sa.get('musicbrainz_id'),
                                 )
-                            except Exception:
-                                pass
+                            except Exception as e:
+                                logger.debug("similar artist insert failed: %s", e)
                         # Re-query from DB to get consistent format
                         if id_values:
                             placeholders = ','.join(['?'] * len(id_values))
                             cursor.execute(f"""
                                 SELECT DISTINCT similar_artist_name, similar_artist_spotify_id,
-                                       similar_artist_itunes_id, similar_artist_deezer_id,
+                                       similar_artist_itunes_id, similar_artist_deezer_id, similar_artist_musicbrainz_id,
                                        image_url, genres, popularity, similarity_rank
                                 FROM similar_artists
                                 WHERE source_artist_id IN ({placeholders}) AND profile_id = ?
@@ -805,7 +816,7 @@ def get_artist_map_explore():
                             # Fallback: query by name-based source ID
                             cursor.execute("""
                                 SELECT DISTINCT similar_artist_name, similar_artist_spotify_id,
-                                       similar_artist_itunes_id, similar_artist_deezer_id,
+                                       similar_artist_itunes_id, similar_artist_deezer_id, similar_artist_musicbrainz_id,
                                        image_url, genres, popularity, similarity_rank
                                 FROM similar_artists
                                 WHERE source_artist_id = ? AND profile_id = ?
@@ -828,8 +839,8 @@ def get_artist_map_explore():
             if r['genres']:
                 try:
                     genres = json.loads(r['genres']) if isinstance(r['genres'], str) else []
-                except Exception:
-                    pass
+                except Exception as e:
+                    logger.debug("ring1 genres parse failed: %s", e)
             img = r['image_url'] if r['image_url'] and r['image_url'].startswith('http') else ''
             nodes.append({
                 'id': idx, 'name': r['similar_artist_name'], 'image_url': img,
@@ -837,6 +848,7 @@ def get_artist_map_explore():
                 'spotify_id': r['similar_artist_spotify_id'] or '',
                 'itunes_id': r['similar_artist_itunes_id'] or '',
                 'deezer_id': r['similar_artist_deezer_id'] or '',
+                'musicbrainz_id': r['similar_artist_musicbrainz_id'] if 'similar_artist_musicbrainz_id' in r.keys() else '',
                 'discogs_id': '',
                 'popularity': r['popularity'] or 0,
                 'rank': r['similarity_rank'] or 5,
@@ -857,7 +869,8 @@ def get_artist_map_explore():
             cursor.execute(f"""
                 SELECT DISTINCT source_artist_id, similar_artist_name,
                        similar_artist_spotify_id, similar_artist_itunes_id,
-                       similar_artist_deezer_id, image_url, genres, popularity, similarity_rank
+                       similar_artist_deezer_id, similar_artist_musicbrainz_id,
+                       image_url, genres, popularity, similarity_rank
                 FROM similar_artists
                 WHERE source_artist_id IN ({placeholders}) AND profile_id = ?
                 ORDER BY similarity_rank ASC
@@ -889,8 +902,8 @@ def get_artist_map_explore():
                 if r['genres']:
                     try:
                         genres = json.loads(r['genres']) if isinstance(r['genres'], str) else []
-                    except Exception:
-                        pass
+                    except Exception as e:
+                        logger.debug("ring2 genres parse failed: %s", e)
                 img = r['image_url'] if r['image_url'] and r['image_url'].startswith('http') else ''
                 nodes.append({
                     'id': idx, 'name': r['similar_artist_name'], 'image_url': img,
@@ -898,6 +911,7 @@ def get_artist_map_explore():
                     'spotify_id': r['similar_artist_spotify_id'] or '',
                     'itunes_id': r['similar_artist_itunes_id'] or '',
                     'deezer_id': r['similar_artist_deezer_id'] or '',
+                    'musicbrainz_id': r['similar_artist_musicbrainz_id'] if 'similar_artist_musicbrainz_id' in r.keys() else '',
                     'discogs_id': '',
                     'popularity': r['popularity'] or 0,
                     'rank': r['similarity_rank'] or 5,
@@ -928,10 +942,10 @@ def get_artist_map_explore():
                 if not n['genres'] and cr['genres']:
                     try:
                         n['genres'] = json.loads(cr['genres'])[:5] if isinstance(cr['genres'], str) else []
-                    except Exception:
-                        pass
+                    except Exception as e:
+                        logger.debug("explorer node genres parse failed: %s", e)
                 # Harvest missing IDs from cache
-                src_map = {'spotify': 'spotify_id', 'itunes': 'itunes_id', 'deezer': 'deezer_id', 'discogs': 'discogs_id'}
+                src_map = {'spotify': 'spotify_id', 'itunes': 'itunes_id', 'deezer': 'deezer_id', 'discogs': 'discogs_id', 'musicbrainz': 'musicbrainz_id'}
                 k = src_map.get(cr['source'])
                 if k and not n.get(k):
                     n[k] = cr['entity_id']
@@ -951,8 +965,8 @@ def get_artist_map_explore():
                             n['image_url'] = artist_data['images'][0]['url']
                             if not n['genres'] and artist_data.get('genres'):
                                 n['genres'] = artist_data['genres'][:5]
-                except Exception:
-                    pass
+                except Exception as e:
+                    logger.debug("spotify artist image fallback failed: %s", e)
 
             # Album art fallback (iTunes artists have no artist images)
             if not n['image_url']:

core/artists/quality.py

@@ -2,8 +2,8 @@
 
 `enhance_artist_quality(artist_id, track_ids, deps)` is the route-handler
 body for the `/api/library/artist/<artist_id>/enhance` endpoint. It walks
-the user's selected tracks, finds the best Spotify (preferred) or iTunes
-(fallback) match for each, and queues high-quality re-downloads on the
+the user's selected tracks, finds the best metadata match against the
+configured primary source, and queues high-quality re-downloads on the
 wishlist with `source_type='enhance'`.
 
 Per-track flow:
@@ -12,13 +12,43 @@ Per-track flow:
    front from `database.get_artist_full_detail`).
 2. Read current quality tier from the file extension.
 3. Build `matched_track_data` for the wishlist entry, in priority order:
-   - Direct Spotify lookup via stored `spotify_track_id` (preferred).
-   - Spotify search fallback using matching_engine queries.
-   - iTunes/fallback source search.
-4. Add to wishlist via `wishlist_service.add_spotify_track_to_wishlist`
+   - **Direct lookup using stored source IDs** — for every source the
+     user has configured, if the library track has the corresponding
+     stored ID (`spotify_track_id` / `deezer_id` / `itunes_track_id` /
+     `soul_id`), call `client.get_track_details(stored_id)` and convert
+     the result to the wishlist payload. First success wins; the user's
+     configured primary source is tried first. Mirrors what Download
+     Discography does — stable IDs straight to the source's API, no
+     fuzzy text matching.
+   - **Multi-source parallel text search fallback** — if no stored ID
+     resolved, run the shared `core.metadata.multi_source_search`
+     against every configured source in parallel and pick the best
+     cross-source match (auto-accept threshold 0.7).
+4. Validate the match has non-empty title, album, and artists. Reject
+   matches with empty fields — those propagated as
+   "unknown artist - unknown album - unknown track" wishlist entries
+   pre-fix because the wishlist payload normalizer's truthy-check
+   passthrough accepted dicts with empty string fields.
+5. Add to wishlist via `wishlist_service.add_spotify_track_to_wishlist`
    with `source_type='enhance'` and a `source_context` carrying the
    original file path, format tier, bitrate, and artist name.
-5. Tally `enhanced_count` / `failed_count` / per-track failure reasons.
+6. Tally `enhanced_count` / `failed_count` / per-track failure reasons.
+
+The flow originally had Spotify-only logic with an iTunes search-only
+fallback. Two failure modes drove the rewrite:
+
+- Users with neither Spotify nor Deezer connected got silent failures
+  ("unknown artist - unknown album - unknown track" wishlist entries)
+  because iTunes's text search returned junk matches with empty fields
+  that cleared the 0.7 confidence threshold.
+- Library tracks with messy tags ("Title (Live)", featured artists in
+  the artist field, etc.) failed fuzzy text search even when a perfect
+  stored ID was available — Download Discography had no such problem
+  because it resolves albums by stable ID.
+
+Direct-lookup-via-stored-ID matches the Download Discography contract
+for every source where we have an ID column. Text search is only the
+fallback now.
 
 Returns `(payload_dict, http_status_code)` so the route wrapper can
 `jsonify()` and return.
@@ -26,28 +56,286 @@ Returns `(payload_dict, http_status_code)` so the route wrapper can
 
 from __future__ import annotations
 
-import logging
 import os
 from dataclasses import dataclass
-from typing import Any, Callable
+from typing import Any, Callable, Optional
 
-logger = logging.getLogger(__name__)
+from utils.logging_config import get_logger
+
+logger = get_logger('artists.quality')
 
 
 @dataclass
 class ArtistQualityDeps:
     """Bundle of cross-cutting deps the artist quality enhancement needs."""
-    spotify_client: Any
     matching_engine: Any
     get_database: Callable[[], Any]
     get_wishlist_service: Callable[[], Any]
     get_current_profile_id: Callable[[], int]
     get_quality_tier_from_extension: Callable
-    get_metadata_fallback_client: Callable[[], Any]
+    # Returns ``[(source_name, client), ...]`` for every metadata source
+    # the user has configured. Powers both the direct-lookup fast path
+    # (resolves stored source IDs straight from each source's API,
+    # like Download Discography) and the multi-source parallel text
+    # search fallback (shared with Track Redownload via
+    # ``core.metadata.multi_source_search``).
+    get_metadata_search_sources: Callable[[], list]
+
+
+def _has_complete_metadata(payload: Optional[dict]) -> bool:
+    """Reject matches with empty / missing core fields. Pre-fix, iTunes
+    returned matches that cleared the 0.7 confidence threshold while
+    having empty artist / album / title — those propagated as junk
+    wishlist entries displayed as 'unknown artist - unknown album -
+    unknown track'."""
+    if not payload:
+        return False
+    if not (payload.get('name') or '').strip():
+        return False
+    artists = payload.get('artists') or []
+    has_artist = any(
+        (a.get('name') or '').strip() if isinstance(a, dict) else (a or '').strip()
+        for a in artists
+    )
+    if not has_artist:
+        return False
+    album = payload.get('album') or {}
+    if isinstance(album, dict):
+        if not (album.get('name') or '').strip():
+            return False
+    elif not (album or '').strip():
+        return False
+    return True
+
+
+def _build_payload_from_track(track_obj) -> dict:
+    """Build a Spotify-shaped wishlist payload from any metadata source's
+    Track-shaped object (Spotify Track, iTunes Track, Deezer Track,
+    Discogs Track — they all have the same .id / .name / .artists /
+    .album / .duration_ms / etc shape because each client mimics
+    Spotify's surface).
+
+    The wishlist's downstream pipeline expects Spotify shape; this helper
+    is the single place that knows how to produce it. Replaces the
+    duplicated payload construction that used to live in the Spotify
+    search path AND the iTunes fallback path.
+
+    Does NOT substitute defaults for missing artists / album / title —
+    ``_has_complete_metadata`` rejects empty matches downstream so the
+    user sees a clear failure instead of a junk wishlist entry with
+    fabricated values.
+    """
+    image_url = getattr(track_obj, 'image_url', '') or ''
+    album_images = (
+        [{'url': image_url, 'height': 600, 'width': 600}]
+        if image_url else []
+    )
+    artist_names = list(getattr(track_obj, 'artists', None) or [])
+    return {
+        'id': getattr(track_obj, 'id', ''),
+        'name': getattr(track_obj, 'name', '') or '',
+        'artists': [{'name': a} for a in artist_names],
+        'album': {
+            'name': getattr(track_obj, 'album', '') or '',
+            'artists': [{'name': a} for a in artist_names],
+            'album_type': getattr(track_obj, 'album_type', None) or 'album',
+            'images': album_images,
+            'release_date': getattr(track_obj, 'release_date', '') or '',
+            'total_tracks': 1,
+        },
+        'duration_ms': getattr(track_obj, 'duration_ms', 0) or 0,
+        'track_number': getattr(track_obj, 'track_number', None) or 1,
+        'disc_number': getattr(track_obj, 'disc_number', None) or 1,
+        'popularity': getattr(track_obj, 'popularity', None) or 0,
+        'preview_url': getattr(track_obj, 'preview_url', None),
+        'external_urls': getattr(track_obj, 'external_urls', None) or {},
+    }
+
+
+# Map metadata source name → DB column on the ``tracks`` table that
+# stores that source's native track ID. Used to drive the direct-lookup
+# fast path: when a library track has a stored ID for source X and the
+# user has source X configured, skip fuzzy text search and resolve
+# straight from X's API. Mirrors what Download Discography does — stable
+# IDs all the way, no fuzzy text matching.
+#
+# Discogs is release-based and has no per-track ID column; not listed
+# here, so direct lookup never tries Discogs (search-fallback still
+# runs for Discogs as one of the parallel sources).
+_STORED_ID_COLUMNS = {
+    'spotify': 'spotify_track_id',
+    'deezer': 'deezer_id',
+    'itunes': 'itunes_track_id',
+    'hydrabase': 'soul_id',
+}
+
+
+def _enhanced_to_wishlist_payload(enhanced: dict,
+                                   fallback_title: str,
+                                   fallback_artist: str,
+                                   fallback_album: str) -> Optional[dict]:
+    """Convert a ``get_track_details`` enhanced-shape dict to the
+    Spotify-shape wishlist payload.
+
+    Every metadata source's ``get_track_details`` returns the same
+    "enhanced" intermediate shape (top-level ``id``, ``name``,
+    ``artists`` as a list of strings, ``album.artists`` as strings),
+    documented and pinned across spotify_client / itunes_client /
+    deezer_client / hydrabase_client. The wishlist downstream expects
+    Spotify's native shape (``artists`` as ``[{'name': ...}]``), so
+    this helper does the conversion in one place.
+
+    Spotify's ``raw_data`` field is already in wishlist shape (the
+    raw Spotify API response), so we return it as-is when detected,
+    preserving full ``album.images`` and ``external_urls`` that the
+    enhanced top-level fields drop. Other sources' ``raw_data`` is
+    in source-native shape and gets ignored.
+    """
+    if not enhanced:
+        return None
+    raw = enhanced.get('raw_data')
+    if isinstance(raw, dict):
+        raw_artists = raw.get('artists')
+        if (isinstance(raw_artists, list) and raw_artists
+                and isinstance(raw_artists[0], dict)):
+            return raw
+
+    artists = enhanced.get('artists') or [fallback_artist]
+    album_data = enhanced.get('album') or {}
+    album_artists = album_data.get('artists') or artists
+
+    def _to_dict_artists(seq):
+        return [a if isinstance(a, dict) else {'name': a} for a in seq]
+
+    image_url = enhanced.get('image_url') or ''
+    album_images_field = album_data.get('images')
+    if isinstance(album_images_field, list) and album_images_field:
+        album_images = album_images_field
+    elif image_url:
+        album_images = [{'url': image_url, 'height': 600, 'width': 600}]
+    else:
+        album_images = []
+
+    return {
+        'id': str(enhanced.get('id', '')),
+        'name': enhanced.get('name') or fallback_title,
+        'artists': _to_dict_artists(artists),
+        'album': {
+            'id': str(album_data.get('id', '')),
+            'name': album_data.get('name') or fallback_album,
+            'album_type': album_data.get('album_type', 'album'),
+            'release_date': album_data.get('release_date', ''),
+            'total_tracks': album_data.get('total_tracks', 1),
+            'artists': _to_dict_artists(album_artists),
+            'images': album_images,
+        },
+        'duration_ms': enhanced.get('duration_ms', 0),
+        'track_number': enhanced.get('track_number', 1),
+        'disc_number': enhanced.get('disc_number', 1),
+        'popularity': enhanced.get('popularity', 0),
+        'preview_url': enhanced.get('preview_url'),
+        'external_urls': enhanced.get('external_urls', {}),
+    }
+
+
+def _try_direct_lookup_all_sources(track: dict,
+                                    sources: list,
+                                    preferred_source: Optional[str],
+                                    title: str,
+                                    artist_name: str,
+                                    album_title: str
+                                    ) -> tuple:
+    """Try direct ID-based lookup on every source where the library
+    track has a stored ID. Returns ``(payload, source_name)`` on first
+    success, or ``(None, None)`` if no source has a stored ID with a
+    successful lookup.
+
+    Mirrors what Download Discography does — stable IDs straight to the
+    source's API, no fuzzy text matching. Avoids the failure mode where
+    library text tags don't match the source's canonical title (the
+    Discord report case: track tagged "Title (Live)" and source has
+    "Title" → fuzzy search misses, but stored ID resolves directly).
+
+    Preferred source attempted first when present in ``sources``,
+    typically the user's configured primary metadata source — so a
+    Deezer-primary user gets Deezer art / album shape on the wishlist
+    entry instead of whichever source happened to have a stored ID
+    first in iteration order.
+    """
+    def _priority(entry):
+        name = entry[0]
+        return 0 if name == preferred_source else 1
+    ordered = sorted(sources, key=_priority)
+
+    for source_name, client in ordered:
+        column = _STORED_ID_COLUMNS.get(source_name)
+        if not column:
+            continue
+        stored_id = track.get(column)
+        if not stored_id:
+            continue
+        if not hasattr(client, 'get_track_details'):
+            continue
+        try:
+            enhanced = client.get_track_details(str(stored_id))
+        except Exception as exc:
+            logger.error(
+                f"[Enhance] {source_name} direct lookup failed for "
+                f"ID {stored_id}: {exc}"
+            )
+            continue
+        if not enhanced:
+            continue
+        payload = _enhanced_to_wishlist_payload(
+            enhanced, title, artist_name, album_title,
+        )
+        if _has_complete_metadata(payload):
+            logger.info(
+                f"[Enhance] Direct lookup matched: {source_name} "
+                f"ID {stored_id} → '{payload.get('name')}'"
+            )
+            return payload, source_name
+
+    return None, None
+
+
+# Minimum match-score threshold for accepting a search-fallback match
+# without user confirmation. Mirrors the legacy threshold the enhance
+# flow has always used.
+_AUTO_ACCEPT_SCORE_THRESHOLD = 0.7
 
 
 def enhance_artist_quality(artist_id, track_ids, deps: ArtistQualityDeps):
-    """Add selected tracks to wishlist for quality enhancement re-download."""
+    """Add selected tracks to wishlist for quality enhancement re-download.
+
+    Per-track flow:
+
+    1. **Direct lookup using stored source IDs** (mirrors what Download
+       Discography does — stable IDs straight to the source's API, no
+       fuzzy text matching). For each source the user has configured,
+       if the library track has the corresponding stored ID
+       (``spotify_track_id`` / ``deezer_id`` / ``itunes_track_id`` /
+       ``soul_id``), call ``client.get_track_details(stored_id)`` and
+       convert to wishlist payload. First success wins; preferred
+       source (user's configured primary) tried first.
+
+    2. **Multi-source parallel text search fallback** (via the shared
+       ``core.metadata.multi_source_search`` module — same code path
+       Track Redownload uses) for tracks with no stored IDs / lookup
+       misses.
+
+    3. **Validation**: reject matches with empty title / album / artists
+       so the user sees a clear failure instead of an "unknown artist"
+       wishlist entry.
+
+    Pre-refactor: only Spotify had a direct-lookup fast path; everything
+    else went through fuzzy text search. Discogs / Hydrabase / Deezer-
+    primary users got far worse coverage than Download Discography
+    despite both flows asking the same question.
+    """
+    from core.metadata.multi_source_search import TrackQuery, search_all_sources
+    from core.metadata.registry import get_primary_source
+
     try:
         if not track_ids:
             return {"success": False, "error": "No track IDs provided"}, 400
@@ -73,6 +361,18 @@ def enhance_artist_quality(artist_id, track_ids, deps: ArtistQualityDeps):
                 track['_album_id'] = album.get('id')
                 track_lookup[tid] = track
 
+        # Resolve every configured metadata source up front.
+        search_sources = deps.get_metadata_search_sources()
+
+        # User's configured primary source — direct-lookup tries this
+        # first so Deezer-primary users get Deezer payloads on the
+        # wishlist entry (correct cover art / album shape) even when
+        # other sources also have stored IDs for the same track.
+        try:
+            preferred_source = get_primary_source()
+        except Exception:
+            preferred_source = None
+
         enhanced_count = 0
         failed_count = 0
         failed_tracks = []
@@ -95,200 +395,67 @@ def enhance_artist_quality(artist_id, track_ids, deps: ArtistQualityDeps):
             title = track.get('title', '') or ''
             if not title.strip():
                 title = os.path.splitext(os.path.basename(file_path))[0]
-            spotify_tid = track.get('spotify_track_id')
+            album_title = track.get('_album_title', '')
 
-            # Build Spotify track data for wishlist
             matched_track_data = None
+            chosen_source = None
 
-            if spotify_tid and deps.spotify_client:
-                # Direct lookup via stored Spotify ID — raw_data has full Spotify API format
+            # 1. Direct lookup via every stored source ID — like Download
+            # Discography. Stable IDs, no fuzzy text matching.
+            if search_sources:
+                matched_track_data, chosen_source = _try_direct_lookup_all_sources(
+                    track, search_sources, preferred_source,
+                    title, artist_name, album_title,
+                )
+
+            # 2. Multi-source parallel text search fallback — for tracks
+            # with no stored IDs / lookup misses.
+            if not matched_track_data and search_sources:
                 try:
-                    track_details = deps.spotify_client.get_track_details(spotify_tid)
-                    if track_details and track_details.get('raw_data'):
-                        matched_track_data = track_details['raw_data']
-                    elif track_details:
-                        # Enhanced format — rebuild with images for wishlist compatibility
-                        album_data = track_details.get('album', {})
-                        album_images = []
-                        # Try to get album art from a full album lookup
-                        if album_data.get('id'):
-                            try:
-                                full_album = deps.spotify_client.get_album(album_data['id'])
-                                if full_album and full_album.get('images'):
-                                    album_images = full_album['images']
-                            except Exception:
-                                pass
-                        matched_track_data = {
-                            'id': spotify_tid,
-                            'name': track_details.get('name', title),
-                            'artists': [{'name': a} for a in track_details.get('artists', [artist_name])],
-                            'album': {
-                                'id': album_data.get('id', ''),
-                                'name': album_data.get('name', track.get('_album_title', '')),
-                                'album_type': album_data.get('album_type', 'album'),
-                                'release_date': album_data.get('release_date', ''),
-                                'total_tracks': album_data.get('total_tracks', 1),
-                                'artists': [{'name': a} for a in album_data.get('artists', [artist_name])],
-                                'images': album_images,
-                            },
-                            'duration_ms': track_details.get('duration_ms', track.get('duration', 0)),
-                            'track_number': track_details.get('track_number', track.get('track_number', 1)),
-                            'disc_number': track_details.get('disc_number', 1),
-                            'popularity': 0,
-                            'preview_url': None,
-                            'external_urls': {},
-                        }
-                except Exception as e:
-                    logger.error(f"[Enhance] Spotify lookup failed for {spotify_tid}: {e}")
-
-            if not matched_track_data and deps.spotify_client:
-                # Fallback: Spotify search matching — need full track data for wishlist
-                try:
-                    temp_track = type('TempTrack', (), {
-                        'name': title, 'artists': [artist_name],
-                        'album': track.get('_album_title', '')
-                    })()
-                    search_queries = deps.matching_engine.generate_download_queries(temp_track)
-                    best_match = None
-                    best_match_raw = None
-                    best_confidence = 0.0
-
-                    for search_query in search_queries[:3]:  # Limit queries
-                        try:
-                            results = deps.spotify_client.search_tracks(search_query, limit=5)
-                            if not results:
-                                continue
-                            for sp_track in results:
-                                artist_conf = max(
-                                    (deps.matching_engine.similarity_score(
-                                        deps.matching_engine.normalize_string(artist_name),
-                                        deps.matching_engine.normalize_string(a)
-                                    ) for a in (sp_track.artists or [artist_name])),
-                                    default=0
-                                )
-                                title_conf = deps.matching_engine.similarity_score(
-                                    deps.matching_engine.normalize_string(title),
-                                    deps.matching_engine.normalize_string(sp_track.name)
-                                )
-                                combined = artist_conf * 0.5 + title_conf * 0.5
-                                # Small bonus for album tracks over singles
-                                _at = getattr(sp_track, 'album_type', None) or ''
-                                if _at == 'album':
-                                    combined += 0.02
-                                elif _at == 'ep':
-                                    combined += 0.01
-                                if combined > best_confidence and combined >= 0.7:
-                                    best_confidence = combined
-                                    best_match = sp_track
-                            if best_confidence >= 0.9:
-                                break
-                        except Exception:
-                            continue
-
-                    if best_match:
-                        # Fetch full track data from Spotify for proper wishlist format
-                        try:
-                            full_details = deps.spotify_client.get_track_details(best_match.id)
-                            if full_details and full_details.get('raw_data'):
-                                matched_track_data = full_details['raw_data']
-                            else:
-                                raise ValueError("No raw_data from get_track_details")
-                        except Exception:
-                            # Build from Track dataclass with image
-                            album_images = [{'url': best_match.image_url}] if best_match.image_url else []
-                            matched_track_data = {
-                                'id': best_match.id,
-                                'name': best_match.name,
-                                'artists': [{'name': a} for a in best_match.artists],
-                                'album': {
-                                    'name': best_match.album,
-                                    'artists': [{'name': a} for a in best_match.artists],
-                                    'album_type': 'album',
-                                    'release_date': getattr(best_match, 'release_date', '') or '',
-                                    'images': album_images,
-                                },
-                                'duration_ms': best_match.duration_ms,
-                                'popularity': best_match.popularity or 0,
-                                'preview_url': best_match.preview_url,
-                                'external_urls': best_match.external_urls or {},
-                            }
-                except Exception as e:
-                    logger.error(f"[Enhance] Search match failed for {title}: {e}")
-
-            # Fallback source when Spotify unavailable or no match found
-            if not matched_track_data:
-                try:
-                    fallback_client = deps.get_metadata_fallback_client()
-                    itunes_best = None
-                    itunes_best_conf = 0.0
-
-                    itunes_queries = deps.matching_engine.generate_download_queries(
-                        type('TempTrack', (), {
-                            'name': title, 'artists': [artist_name],
-                            'album': track.get('_album_title', '')
-                        })()
+                    track_query = TrackQuery(
+                        title=title,
+                        artist=artist_name,
+                        album=album_title,
+                        duration_ms=track.get('duration', 0) or 0,
+                        spotify_track_id=track.get('spotify_track_id'),
+                        deezer_id=track.get('deezer_id'),
                     )
+                    multi_result = search_all_sources(track_query, search_sources)
+                    if multi_result.best_match and multi_result.best_match['score'] >= _AUTO_ACCEPT_SCORE_THRESHOLD:
+                        chosen_source = multi_result.best_match['source']
+                        best_track_obj = multi_result.best_track()
+                        if best_track_obj:
+                            matched_track_data = _build_payload_from_track(best_track_obj)
+                except Exception as exc:
+                    logger.error(f"[Enhance] Multi-source search failed for {title}: {exc}")
 
-                    for search_query in itunes_queries[:3]:
-                        try:
-                            itunes_results = fallback_client.search_tracks(search_query, limit=5)
-                            if not itunes_results:
-                                continue
-                            for it_track in itunes_results:
-                                artist_conf = max(
-                                    (deps.matching_engine.similarity_score(
-                                        deps.matching_engine.normalize_string(artist_name),
-                                        deps.matching_engine.normalize_string(a)
-                                    ) for a in (it_track.artists or [artist_name])),
-                                    default=0
-                                )
-                                title_conf = deps.matching_engine.similarity_score(
-                                    deps.matching_engine.normalize_string(title),
-                                    deps.matching_engine.normalize_string(it_track.name)
-                                )
-                                combined = artist_conf * 0.5 + title_conf * 0.5
-                                # Small bonus for album tracks over singles
-                                _at = getattr(it_track, 'album_type', None) or ''
-                                if _at == 'album':
-                                    combined += 0.02
-                                elif _at == 'ep':
-                                    combined += 0.01
-                                if combined > itunes_best_conf and combined >= 0.7:
-                                    itunes_best_conf = combined
-                                    itunes_best = it_track
-                            if itunes_best_conf >= 0.9:
-                                break
-                        except Exception:
-                            continue
-
-                    if itunes_best:
-                        album_images = [{'url': itunes_best.image_url, 'height': 600, 'width': 600}] if itunes_best.image_url else []
-                        matched_track_data = {
-                            'id': itunes_best.id,
-                            'name': itunes_best.name,
-                            'artists': [{'name': a} for a in itunes_best.artists],
-                            'album': {
-                                'name': itunes_best.album,
-                                'artists': [{'name': a} for a in itunes_best.artists],
-                                'album_type': 'album',
-                                'images': album_images,
-                                'release_date': itunes_best.release_date or '',
-                                'total_tracks': 1,
-                            },
-                            'duration_ms': itunes_best.duration_ms,
-                            'track_number': itunes_best.track_number or 1,
-                            'disc_number': itunes_best.disc_number or 1,
-                            'popularity': itunes_best.popularity or 0,
-                            'preview_url': itunes_best.preview_url,
-                            'external_urls': itunes_best.external_urls or {},
-                        }
-                        logger.warning(f"[Enhance] Fallback match for {title}: {itunes_best.artists[0]} - {itunes_best.name} (conf: {itunes_best_conf:.3f})")
-                except Exception as e:
-                    logger.error(f"[Enhance] Fallback source failed for {title}: {e}")
+            # 3. Reject matches with empty / missing core fields.
+            if not _has_complete_metadata(matched_track_data):
+                if matched_track_data:
+                    logger.warning(
+                        f"[Enhance] {chosen_source} match for '{title}' rejected — "
+                        f"empty title / album / artists (would render as 'unknown')"
+                    )
+                matched_track_data = None
 
             if not matched_track_data:
                 failed_count += 1
-                failed_tracks.append({'track_id': track_id, 'title': title, 'reason': 'No Spotify or fallback match'})
+                source_list = ', '.join(name for name, _ in (search_sources or []))
+                if not source_list:
+                    reason = (
+                        'No metadata source configured — connect Spotify / '
+                        'iTunes / Deezer / Discogs / Hydrabase to enable enhance'
+                    )
+                else:
+                    reason = (
+                        f'No usable match across {source_list} — '
+                        f'try connecting an additional metadata source'
+                    )
+                failed_tracks.append({
+                    'track_id': track_id,
+                    'title': title,
+                    'reason': reason,
+                })
                 continue
 
             # Add to wishlist with enhance source

core/audiodb_worker.py

@@ -8,7 +8,7 @@ from datetime import datetime, timedelta
 from utils.logging_config import get_logger
 from database.music_database import MusicDatabase
 from core.audiodb_client import AudioDBClient
-from core.worker_utils import interruptible_sleep
+from core.worker_utils import accept_artist_match, interruptible_sleep
 
 logger = get_logger("audiodb_worker")
 
@@ -140,8 +140,8 @@ class AudioDBWorker:
                         itype = item.get('type', '')
                         table = 'artists' if 'artist' in itype else ('albums' if 'album' in itype else 'tracks')
                         # Can't mark status without an ID — just skip
-                    except Exception:
-                        pass
+                    except Exception as e:
+                        logger.debug("null id table resolve failed: %s", e)
                     continue
 
 
@@ -162,6 +162,16 @@ class AudioDBWorker:
             conn = self.db._get_connection()
             cursor = conn.cursor()
 
+            # Pinned-group override (Manage Enrichment Workers): process one
+            # entity type first, then fall through to the normal chain. Unset or
+            # exhausted ⇒ default artist→album→track order, unchanged.
+            from core.worker_utils import read_enrichment_priority, priority_pending_item
+            _prio = read_enrichment_priority('audiodb')
+            if _prio:
+                _pi = priority_pending_item(cursor, 'audiodb', _prio)
+                if _pi:
+                    return _pi
+
             # Priority 1: Unattempted artists
             cursor.execute("""
                 SELECT id, name
@@ -200,42 +210,45 @@ class AudioDBWorker:
             if row:
                 return {'type': 'track', 'id': row[0], 'name': row[1], 'artist': row[2], 'artist_audiodb_id': row[3]}
 
-            # Priority 4: Retry 'not_found' artists after retry_days
-            not_found_cutoff = datetime.now() - timedelta(days=self.retry_days)
+            # Priority 4: Retry 'not_found' OR 'error' artists after retry_days.
+            # 'error' status covers transient AudioDB outages (timeouts, 500s)
+            # that the issue-#553 fix marks rather than leaving NULL — without
+            # this retry path those rows would stay errored forever.
+            retry_cutoff = datetime.now() - timedelta(days=self.retry_days)
             cursor.execute("""
                 SELECT id, name
                 FROM artists
-                WHERE audiodb_match_status = 'not_found' AND audiodb_last_attempted < ?
+                WHERE audiodb_match_status IN ('not_found', 'error') AND audiodb_last_attempted < ?
                 ORDER BY audiodb_last_attempted ASC
                 LIMIT 1
-            """, (not_found_cutoff,))
+            """, (retry_cutoff,))
             row = cursor.fetchone()
             if row:
                 logger.info(f"Retrying artist '{row[1]}' (last attempted before cutoff)")
                 return {'type': 'artist', 'id': row[0], 'name': row[1]}
 
-            # Priority 5: Retry 'not_found' albums
+            # Priority 5: Retry 'not_found' OR 'error' albums
             cursor.execute("""
                 SELECT a.id, a.title, ar.name AS artist_name, ar.audiodb_id AS artist_audiodb_id
                 FROM albums a
                 JOIN artists ar ON a.artist_id = ar.id
-                WHERE a.audiodb_match_status = 'not_found' AND a.audiodb_last_attempted < ?
+                WHERE a.audiodb_match_status IN ('not_found', 'error') AND a.audiodb_last_attempted < ?
                 ORDER BY a.audiodb_last_attempted ASC
                 LIMIT 1
-            """, (not_found_cutoff,))
+            """, (retry_cutoff,))
             row = cursor.fetchone()
             if row:
                 return {'type': 'album', 'id': row[0], 'name': row[1], 'artist': row[2], 'artist_audiodb_id': row[3]}
 
-            # Priority 6: Retry 'not_found' tracks
+            # Priority 6: Retry 'not_found' OR 'error' tracks
             cursor.execute("""
                 SELECT t.id, t.title, ar.name AS artist_name, ar.audiodb_id AS artist_audiodb_id
                 FROM tracks t
                 JOIN artists ar ON t.artist_id = ar.id
-                WHERE t.audiodb_match_status = 'not_found' AND t.audiodb_last_attempted < ?
+                WHERE t.audiodb_match_status IN ('not_found', 'error') AND t.audiodb_last_attempted < ?
                 ORDER BY t.audiodb_last_attempted ASC
                 LIMIT 1
-            """, (not_found_cutoff,))
+            """, (retry_cutoff,))
             row = cursor.fetchone()
             if row:
                 return {'type': 'track', 'id': row[0], 'name': row[1], 'artist': row[2], 'artist_audiodb_id': row[3]}
@@ -260,8 +273,13 @@ class AudioDBWorker:
 
     def _verify_artist_id(self, item: Dict[str, Any], result: Dict[str, Any]) -> bool:
         """Verify that the result's artist ID matches the parent artist's stored AudioDB ID.
+
         If mismatched, the album/track search is more specific (uses artist+title),
-        so we trust it and correct the parent artist's audiodb_id."""
+        so we trust it and correct the parent artist's audiodb_id — BUT only when
+        the result's artist *name* matches our parent artist. Without that guard,
+        a collaboration/compilation (a track our library credits to one artist
+        that lives on another artist's album) would stamp the wrong AudioDB id
+        onto our artist. See the Deezer fix for the full write-up."""
         parent_audiodb_id = item.get('artist_audiodb_id')
         if not parent_audiodb_id:
             return True
@@ -271,6 +289,18 @@ class AudioDBWorker:
             return True
 
         if str(result_artist_id) != str(parent_audiodb_id):
+            parent_name = item.get('artist') or ''
+            result_artist_name = result.get('strArtist') or ''
+            if (result_artist_name and parent_name
+                    and not self._name_matches(parent_name, result_artist_name)):
+                logger.info(
+                    f"Skipping artist-ID correction from {item['type']} "
+                    f"'{item['name']}': result artist '{result_artist_name}' "
+                    f"≠ parent '{parent_name}' (collab/compilation, not a "
+                    f"correction)"
+                )
+                return True
+
             logger.info(
                 f"Artist ID correction from {item['type']} '{item['name']}': "
                 f"updating parent artist AudioDB ID from {parent_audiodb_id} to {result_artist_id}"
@@ -374,22 +404,40 @@ class AudioDBWorker:
                             return
                     except Exception as e:
                         logger.warning(f"Direct lookup failed for existing AudioDB ID {existing_id}: {e}")
-                    # Direct lookup failed — don't overwrite manual match
-                    logger.debug(f"Preserving manual match for {item_type} '{item_name}' (AudioDB ID: {existing_id})")
+                    # Direct lookup returned no metadata (None) or raised — don't
+                    # fall through to the name-search path below, which could
+                    # overwrite a manually-matched audiodb_id with a wrong guess.
+                    # Mark status='error' so the queue's NULL-status filter stops
+                    # re-picking this row on every tick (issue #553: AudioDB
+                    # `track.php` timeouts caused infinite enrichment loops as
+                    # the row was repeatedly picked + re-attempted because it
+                    # never left the NULL state). The error-retry priority block
+                    # in `_get_next_item` re-attempts after `retry_days` so
+                    # transient AudioDB outages still recover automatically.
+                    self._mark_status(item_type, item_id, 'error')
+                    self.stats['errors'] += 1
+                    logger.debug(
+                        f"Preserving manual match for {item_type} '{item_name}' "
+                        f"(AudioDB ID: {existing_id}); marked error pending retry"
+                    )
                     return
 
             if item_type == 'artist':
                 result = self.client.search_artist(item_name)
                 if result:
                     result_name = result.get('strArtist', '')
-                    if self._name_matches(item_name, result_name):
+                    ok, reason = accept_artist_match(
+                        self.db, 'audiodb_id', result.get('idArtist'), item_id,
+                        item_name, result_name,
+                    )
+                    if ok:
                         self._update_artist(item_id, result)
                         self.stats['matched'] += 1
                         logger.info(f"Matched artist '{item_name}' -> AudioDB ID: {result.get('idArtist')}")
                     else:
                         self._mark_status('artist', item_id, 'not_found')
                         self.stats['not_found'] += 1
-                        logger.debug(f"Name mismatch for artist '{item_name}' (got '{result_name}')")
+                        logger.debug(f"Artist '{item_name}' not matched: {reason}")
                 else:
                     self._mark_status('artist', item_id, 'not_found')
                     self.stats['not_found'] += 1

core/automation/__init__.py

@@ -1,7 +1,11 @@
-"""Automation API + progress tracking helpers package.
+"""Automation API + progress + handlers package.
 
-Lifted from web_server.py /api/automations/* routes and progress
-emitters. The action handler registration (`_register_automation_handlers`)
-stays in web_server.py because each handler closure is tightly coupled
-to other application features.
+Lifted from web_server.py:
+  - `/api/automations/*` route helpers → `api.py`
+  - block library used by the trigger/action UI → `blocks.py`
+  - progress tracker (init / update / finish) → `progress.py`
+  - cross-handler signal bus → `signals.py`
+  - per-action handler functions → `handlers/` subpackage (with
+    `deps.py` defining the dependency-injection surface so handlers
+    stay testable in isolation)
 """

core/automation/api.py

@@ -172,9 +172,11 @@ def create_automation(
         return {'error': f'Signal cycle detected: {cycle_path}. This would cause an infinite loop.'}, 400
 
     group_name = data.get('group_name') or None
+    owned_by = data.get('owned_by') or None
     auto_id = database.create_automation(
         name, trigger_type, trigger_config, action_type, action_config,
         profile_id, notify_type, notify_config, then_actions_json, group_name,
+        owned_by=owned_by,
     )
     if auto_id is None:
         return {'error': 'Failed to create automation'}, 500
@@ -217,6 +219,8 @@ def update_automation(
         update_fields['notify_config'] = json.dumps(data['notify_config'])
     if 'group_name' in data:
         update_fields['group_name'] = data['group_name'] or None
+    if 'owned_by' in data:
+        update_fields['owned_by'] = data['owned_by'] or None
 
     if not update_fields:
         return {'error': 'No fields to update'}, 400
@@ -225,6 +229,12 @@ def update_automation(
     if cycle_path:
         return {'error': f'Signal cycle detected: {cycle_path}. This would cause an infinite loop.'}, 400
 
+    # Schedule-shape changes must invalidate the stored next_run so the
+    # scheduler recomputes it; otherwise restart-survival logic keeps the
+    # leftover timestamp from the previous interval.
+    if {'trigger_type', 'trigger_config'} & update_fields.keys():
+        update_fields['next_run'] = None
+
     success = database.update_automation(automation_id, **update_fields)
     if not success:
         return {'error': 'Automation not found'}, 404

core/automation/blocks.py

@@ -146,6 +146,15 @@ ACTIONS: list[dict] = [
          {"key": "all", "type": "checkbox", "label": "Process all mirrored playlists", "default": False},
          {"key": "skip_wishlist", "type": "checkbox", "label": "Skip wishlist processing", "default": False},
      ]},
+    {"type": "personalized_pipeline", "label": "Personalized Playlist Pipeline", "icon": "sparkles",
+     "description": "Sync personalized / discover-page playlists (Hidden Gems, Time Machine, Fresh Tape, etc.) to your media server + queue missing tracks for download.",
+     "available": True,
+     "config_fields": [
+         {"key": "kinds", "type": "personalized_playlist_select", "label": "Playlists to sync",
+          "description": "Multi-select: Hidden Gems, Discovery Shuffle, Time Machine (per decade), Genre playlists, Fresh Tape, The Archives, Seasonal Mix (per season)"},
+         {"key": "refresh_first", "type": "checkbox", "label": "Refresh playlists before sync (regenerate snapshots)", "default": False},
+         {"key": "skip_wishlist", "type": "checkbox", "label": "Skip wishlist processing", "default": False},
+     ]},
     {"type": "notify_only", "label": "Notify Only", "icon": "bell", "description": "No action — just send notification", "available": True},
     # Phase 3 actions
     {"type": "start_database_update", "label": "Update Database", "icon": "database",
@@ -162,12 +171,7 @@ ACTIONS: list[dict] = [
     {"type": "update_discovery_pool", "label": "Update Discovery", "icon": "compass",
      "description": "Refresh discovery pool with new tracks", "available": True},
     {"type": "start_quality_scan", "label": "Run Quality Scan", "icon": "bar-chart",
-     "description": "Scan for low-quality audio files", "available": True,
-     "config_fields": [
-         {"key": "scope", "type": "select", "label": "Scope",
-          "options": [{"value": "watchlist", "label": "Watchlist Artists"}, {"value": "library", "label": "Full Library"}],
-          "default": "watchlist"}
-     ]},
+     "description": "Run the Quality Upgrade Finder (scope is set in Library Maintenance)", "available": True},
     {"type": "backup_database", "label": "Backup Database", "icon": "save",
      "description": "Create timestamped database backup", "available": True},
     {"type": "refresh_beatport_cache", "label": "Refresh Beatport Cache", "icon": "music",

core/automation/deps.py

@@ -0,0 +1,163 @@
+"""Dependency-injection surface for automation handlers.
+
+Each handler in ``core.automation.handlers`` is a top-level pure
+function that accepts ``(config: dict, deps: AutomationDeps)`` instead
+of reaching for module-level globals in ``web_server``. The deps
+namespace bundles every callable, client, and mutable-state container
+the handlers need.
+
+Construction happens once at app startup in ``web_server.py``:
+
+    from core.automation.deps import AutomationDeps, AutomationState
+    state = AutomationState()
+    deps = AutomationDeps(
+        engine=automation_engine,
+        state=state,
+        get_database=get_database,
+        spotify_client=spotify_client,
+        ...
+    )
+    register_all(deps)
+
+Tests construct a fake ``AutomationDeps`` with stub callables — every
+handler is then exercisable without spinning up Flask, the DB, or
+the real media-server clients.
+"""
+
+from __future__ import annotations
+
+import threading
+from dataclasses import dataclass, field
+from typing import Any, Callable, Dict, Optional
+
+
+@dataclass
+class AutomationState:
+    """Mutable flags shared across handler invocations.
+
+    Pre-refactor each was a ``global`` or ``nonlocal`` variable inside
+    the registration closure. Lifted here so handlers + their guards
+    can read/write a single object instead of importing globals.
+
+    All mutations should hold ``lock``; the helper methods below do
+    so for the common get/set patterns.
+    """
+
+    scan_library_automation_id: Optional[str] = None
+    db_update_automation_id: Optional[str] = None
+    pipeline_running: bool = False
+    lock: threading.Lock = field(default_factory=threading.Lock)
+
+    def is_scan_library_active(self) -> bool:
+        with self.lock:
+            return self.scan_library_automation_id is not None
+
+    def is_pipeline_running(self) -> bool:
+        with self.lock:
+            return self.pipeline_running
+
+    def try_start_pipeline(self) -> bool:
+        """Atomically mark the shared playlist pipeline as running."""
+        with self.lock:
+            if self.pipeline_running:
+                return False
+            self.pipeline_running = True
+            return True
+
+    def set_scan_library_id(self, automation_id: Optional[str]) -> None:
+        with self.lock:
+            self.scan_library_automation_id = automation_id
+
+    def set_pipeline_running(self, value: bool) -> None:
+        with self.lock:
+            self.pipeline_running = value
+
+
+@dataclass
+class AutomationDeps:
+    """Bundle of every callable + client an automation handler may need.
+
+    Add fields as new handlers are extracted. Every field is required
+    at construction (no defaults) so a missing dep fails loudly at
+    startup, not silently mid-handler.
+    """
+
+    # --- Engine + shared state ---
+    engine: Any                                          # AutomationEngine instance
+    state: AutomationState
+    config_manager: Any                                  # config.settings.ConfigManager singleton
+    update_progress: Callable[..., None]                 # _update_automation_progress
+    logger: Any                                          # module-level logger from utils.logging_config
+
+    # --- Service clients (each may be None depending on user config) ---
+    get_database: Callable[[], Any]                      # late-binding so tests don't need DB
+    spotify_client: Any
+    tidal_client: Any
+    web_scan_manager: Any
+
+    # --- Background-task entry points ---
+    process_wishlist_automatically: Callable[..., Any]
+    process_watchlist_scan_automatically: Callable[..., Any]
+    is_wishlist_actually_processing: Callable[[], bool]
+    is_watchlist_actually_scanning: Callable[[], bool]
+    get_watchlist_scan_state: Callable[[], dict]         # accessor returns the live mutable dict
+
+    # --- Playlist pipeline entry points ---
+    run_playlist_discovery_worker: Callable[..., Any]
+    run_sync_task: Callable[..., Any]
+    run_playlist_organize_download: Callable[..., Dict[str, Any]]
+    missing_download_executor: Any
+    load_sync_status_file: Callable[[], dict]
+    get_deezer_client: Callable[[], Any]
+    parse_youtube_playlist: Callable[[str], Any]
+    get_sync_states: Callable[[], dict]                  # accessor returns the live dict shared with the sync UI
+
+    # --- Database update + quality scanner (shared state + executors) ---
+    set_db_update_automation_id: Callable[[Optional[str]], None]  # syncs the legacy `_db_update_automation_id` global so the live DB-update progress callbacks (which still read the global directly) keep firing for the active automation
+    get_db_update_state: Callable[[], dict]
+    db_update_lock: Any                                  # threading.Lock
+    db_update_executor: Any                              # ThreadPoolExecutor
+    run_db_update_task: Callable[..., Any]
+    run_deep_scan_task: Callable[..., Any]
+    get_duplicate_cleaner_state: Callable[[], dict]
+    duplicate_cleaner_lock: Any
+    duplicate_cleaner_executor: Any
+    run_duplicate_cleaner: Callable[..., Any]
+    # Triggers a "Run Now" of a library-maintenance repair job by id (e.g.
+    # 'quality_upgrade'). Returns truthy if the job was queued. Replaces the old
+    # standalone quality-scanner executor/state (the scanner is now a repair job).
+    run_repair_job_now: Callable[[str], Any]
+
+    # --- Download orchestrator + queue accessors ---
+    download_orchestrator: Any
+    run_async: Callable[..., Any]
+    tasks_lock: Any
+    get_download_batches: Callable[[], dict]
+    get_download_tasks: Callable[[], dict]
+    sweep_empty_download_directories: Callable[[], int]
+    get_staging_path: Callable[[], str]
+
+    # --- Maintenance helpers ---
+    docker_resolve_path: Callable[[str], str]
+    get_current_profile_id: Callable[[], int]
+    get_watchlist_scanner: Callable[[Any], Any]
+    get_app: Callable[[], Any]                           # Flask app for test_client (beatport refresh)
+    get_beatport_data_cache: Callable[[], dict]
+
+    # --- Progress + history callbacks (used by register_all to wire
+    # the engine's progress callback hooks). ---
+    init_automation_progress: Callable[..., Any]
+    record_progress_history: Callable[..., Any]
+
+    # --- Personalized playlist pipeline ---
+    # Lazy builder so the pipeline handler can construct a fresh
+    # PersonalizedPlaylistManager per run (cheap accessors inside,
+    # no caching needed yet).
+    build_personalized_manager: Callable[[], Any]
+
+    # --- Unified PlaylistSource registry ---
+    # Optional so test fixtures that don't exercise refresh_mirrored
+    # can keep their existing scaffolding. Production wiring in
+    # ``web_server.py`` always populates it via
+    # ``core.playlists.sources.bootstrap.build_playlist_source_registry``.
+    playlist_source_registry: Optional[Any] = None

core/automation/handlers/__init__.py

@@ -0,0 +1,64 @@
+"""Per-action automation handlers.
+
+Each module in this subpackage exposes one top-level handler function
+(or a small cluster of related handlers) of the form::
+
+    def auto_<action_name>(config: dict, deps: AutomationDeps) -> dict
+
+The ``register_all`` helper in :mod:`registration` wires every handler
+to the engine in one place. ``web_server.py`` calls
+``register_all(deps)`` once at startup.
+"""
+
+from core.automation.handlers.process_wishlist import auto_process_wishlist
+from core.automation.handlers.scan_watchlist import auto_scan_watchlist
+from core.automation.handlers.scan_library import auto_scan_library
+from core.automation.handlers.refresh_mirrored import auto_refresh_mirrored
+from core.automation.handlers.sync_playlist import auto_sync_playlist
+from core.automation.handlers.discover_playlist import auto_discover_playlist
+from core.automation.handlers.playlist_pipeline import auto_playlist_pipeline
+from core.automation.handlers.personalized_pipeline import auto_personalized_pipeline
+from core.automation.handlers.database_update import auto_start_database_update, auto_deep_scan_library
+from core.automation.handlers.duplicate_cleaner import auto_run_duplicate_cleaner
+from core.automation.handlers.quality_scanner import auto_start_quality_scan
+from core.automation.handlers.maintenance import (
+    auto_clear_quarantine,
+    auto_cleanup_wishlist,
+    auto_update_discovery_pool,
+    auto_backup_database,
+    auto_refresh_beatport_cache,
+)
+from core.automation.handlers.download_cleanup import (
+    auto_clean_search_history,
+    auto_clean_completed_downloads,
+    auto_full_cleanup,
+)
+from core.automation.handlers.run_script import auto_run_script
+from core.automation.handlers.search_and_download import auto_search_and_download
+from core.automation.handlers.registration import register_all
+
+__all__ = [
+    'auto_process_wishlist',
+    'auto_scan_watchlist',
+    'auto_scan_library',
+    'auto_refresh_mirrored',
+    'auto_sync_playlist',
+    'auto_discover_playlist',
+    'auto_playlist_pipeline',
+    'auto_personalized_pipeline',
+    'auto_start_database_update',
+    'auto_deep_scan_library',
+    'auto_run_duplicate_cleaner',
+    'auto_start_quality_scan',
+    'auto_clear_quarantine',
+    'auto_cleanup_wishlist',
+    'auto_update_discovery_pool',
+    'auto_backup_database',
+    'auto_refresh_beatport_cache',
+    'auto_clean_search_history',
+    'auto_clean_completed_downloads',
+    'auto_full_cleanup',
+    'auto_run_script',
+    'auto_search_and_download',
+    'register_all',
+]

core/automation/handlers/_pipeline_shared.py

@@ -0,0 +1,259 @@
+"""Shared helpers between mirrored + personalized playlist pipelines.
+
+Both pipelines end in the same shape:
+1. SYNC each playlist to the active media server.
+2. WISHLIST: trigger the wishlist processor for missing tracks.
+
+The differing prefix (mirrored = REFRESH external sources + DISCOVER
+metadata; personalized = SNAPSHOT manager-backed playlists) is owned
+by each pipeline. This module owns the SYNC + WISHLIST tail so both
+pipelines stay consistent + DRY.
+"""
+
+from __future__ import annotations
+
+import time
+from typing import Any, Callable, Dict, List, Optional
+
+from core.automation.deps import AutomationDeps
+
+
+# Per-playlist sync poll cap (mirrored side already used this).
+_SYNC_PER_PLAYLIST_TIMEOUT_SECONDS = 600
+# Sync-status final-state markers.
+_SYNC_TERMINAL_STATUSES = ('finished', 'complete', 'error', 'failed')
+
+
+def run_sync_and_wishlist(
+    deps: AutomationDeps,
+    automation_id: Optional[str],
+    playlists: List[Dict[str, Any]],
+    *,
+    sync_one_fn: Callable[[Dict[str, Any]], Dict[str, Any]],
+    sync_id_for_fn: Callable[[Dict[str, Any]], str],
+    skip_wishlist: bool = False,
+    progress_start: int = 56,
+    progress_end: int = 85,
+    sync_phase_label: str = 'Phase: Syncing to server...',
+    sync_phase_start_log: str = 'Sync',
+    wishlist_phase_label: str = 'Phase: Processing wishlist...',
+    wishlist_phase_start_log: str = 'Wishlist',
+) -> Dict[str, int]:
+    """Run the SYNC + WISHLIST tail of a playlist pipeline.
+
+    The caller supplies:
+    - ``playlists``: list of playlist payload dicts. Each must have at
+      least a ``name`` key (used in progress logs). The shape beyond
+      ``name`` is opaque to the helper — ``sync_one_fn`` receives the
+      payload and returns a sync_result dict.
+    - ``sync_one_fn(payload) -> sync_result``: launches sync for one
+      playlist. Result dict must carry ``status`` ∈ ``('started',
+      'skipped', 'error')`` and may carry ``reason``.
+    - ``sync_id_for_fn(payload) -> str``: returns the sync-state key
+      the helper polls on (so we can wait for the background sync
+      thread to complete + read the matched_tracks count).
+
+    Returns ``{'synced': int, 'skipped': int, 'errors': int,
+    'wishlist_queued': int}`` so the caller can stitch it into its
+    final status.
+    """
+    deps.update_progress(
+        automation_id,
+        progress=progress_start,
+        phase=sync_phase_label,
+        log_line=sync_phase_start_log,
+        log_type='info',
+    )
+
+    total_synced = 0
+    total_skipped = 0
+    sync_errors = 0
+    sync_states = deps.get_sync_states()
+    n_playlists = max(1, len(playlists))
+    progress_span = max(1, progress_end - progress_start - 1)
+
+    for pl_idx, pl in enumerate(playlists):
+        pl_name = pl.get('name', '')
+        sync_result = sync_one_fn(pl)
+        sync_status = sync_result.get('status', '')
+
+        if sync_status == 'started':
+            sync_id = sync_id_for_fn(pl)
+            sync_poll_start = time.time()
+            timed_out = True
+            while time.time() - sync_poll_start < _SYNC_PER_PLAYLIST_TIMEOUT_SECONDS:
+                if (sync_id in sync_states
+                        and sync_states[sync_id].get('status') in _SYNC_TERMINAL_STATUSES):
+                    timed_out = False
+                    break
+                time.sleep(2)
+                elapsed = int(time.time() - sync_poll_start)
+                sub_progress = progress_start + 1 + ((pl_idx + 1) / n_playlists) * progress_span
+                deps.update_progress(
+                    automation_id,
+                    progress=min(int(sub_progress), progress_end - 1),
+                    phase=f'{sync_phase_label.rstrip(".")} — "{pl_name}" ({elapsed}s)',
+                )
+
+            ss = sync_states.get(sync_id, {})
+            final_status = ss.get('status')
+            if timed_out:
+                sync_errors += 1
+                deps.update_progress(
+                    automation_id,
+                    log_line=f'Sync timed out "{pl_name}" after {_SYNC_PER_PLAYLIST_TIMEOUT_SECONDS}s',
+                    log_type='error',
+                )
+                continue
+            if final_status in ('error', 'failed'):
+                sync_errors += 1
+                reason = ss.get('error') or ss.get('reason') or 'background sync failed'
+                deps.update_progress(
+                    automation_id,
+                    log_line=f'Sync error "{pl_name}": {reason}',
+                    log_type='error',
+                )
+                continue
+
+            ss_result = ss.get('result', ss.get('progress', {}))
+            matched = ss_result.get('matched_tracks', 0) if isinstance(ss_result, dict) else 0
+            total_synced += int(matched) if matched else 0
+            deps.update_progress(
+                automation_id,
+                log_line=f'Synced "{pl_name}": {matched} tracks matched',
+                log_type='success',
+            )
+
+        elif sync_status == 'skipped':
+            total_skipped += 1
+            reason = sync_result.get('reason', 'unchanged')
+            deps.update_progress(
+                automation_id,
+                log_line=f'Skipped "{pl_name}": {reason}',
+                log_type='skip',
+            )
+        elif sync_status == 'error':
+            sync_errors += 1
+            deps.update_progress(
+                automation_id,
+                log_line=f'Sync error "{pl_name}": {sync_result.get("reason", "unknown")}',
+                log_type='error',
+            )
+
+    deps.update_progress(
+        automation_id,
+        progress=progress_end,
+        phase=f'{sync_phase_label.rstrip(".")} complete',
+        log_line=f'Sync done: {total_synced} matched, {total_skipped} skipped, {sync_errors} errors',
+        log_type='success' if sync_errors == 0 else 'warning',
+    )
+
+    organize_playlists = [pl for pl in playlists if pl.get('organize_by_playlist')]
+    organize_started = 0
+    if organize_playlists and hasattr(deps, 'run_playlist_organize_download'):
+        for pl in organize_playlists:
+            pl_id = pl.get('id')
+            if not pl_id:
+                continue
+            pl_name = pl.get('name', '')
+            try:
+                org_result = deps.run_playlist_organize_download(
+                    mirrored_playlist_id=int(pl_id),
+                    automation_id=automation_id,
+                )
+                if org_result.get('status') == 'started':
+                    organize_started += 1
+                    deps.update_progress(
+                        automation_id,
+                        log_line=f'Organize download started for "{pl_name}"',
+                        log_type='success',
+                    )
+                elif org_result.get('status') == 'skipped':
+                    deps.update_progress(
+                        automation_id,
+                        log_line=f'Organize download skipped for "{pl_name}": {org_result.get("reason", "")}',
+                        log_type='skip',
+                    )
+            except Exception as org_err:  # noqa: BLE001
+                deps.update_progress(
+                    automation_id,
+                    log_line=f'Organize download error for "{pl_name}": {org_err}',
+                    log_type='warning',
+                )
+
+    all_organize = bool(playlists) and len(organize_playlists) == len(playlists)
+    effective_skip_wishlist = skip_wishlist or all_organize
+
+    wishlist_queued = run_wishlist_phase(
+        deps, automation_id,
+        skip=effective_skip_wishlist,
+        progress_pct=progress_end + 1,
+        wishlist_phase_label=wishlist_phase_label,
+        wishlist_phase_start_log=wishlist_phase_start_log,
+    )
+
+    return {
+        'synced': total_synced,
+        'skipped': total_skipped,
+        'errors': sync_errors,
+        'wishlist_queued': wishlist_queued,
+        'organize_downloads_started': organize_started,
+    }
+
+
+def run_wishlist_phase(
+    deps: AutomationDeps,
+    automation_id: Optional[str],
+    *,
+    skip: bool,
+    progress_pct: int,
+    wishlist_phase_label: str = 'Phase: Processing wishlist...',
+    wishlist_phase_start_log: str = 'Wishlist',
+) -> int:
+    """Trigger the wishlist processor unless skipped or already running.
+
+    Returns 1 when the processor was triggered, 0 otherwise. Errors are
+    logged but never raised — wishlist failure should not abort the
+    pipeline."""
+    if skip:
+        deps.update_progress(
+            automation_id,
+            progress=progress_pct,
+            log_line=f'{wishlist_phase_start_log}: skipped (disabled)',
+            log_type='skip',
+        )
+        return 0
+
+    deps.update_progress(
+        automation_id,
+        progress=progress_pct,
+        phase=wishlist_phase_label,
+        log_line=wishlist_phase_start_log,
+        log_type='info',
+    )
+
+    try:
+        if not deps.is_wishlist_actually_processing():
+            deps.process_wishlist_automatically(automation_id=None)
+            deps.update_progress(
+                automation_id,
+                log_line='Wishlist processing triggered',
+                log_type='success',
+            )
+            return 1
+        deps.update_progress(
+            automation_id,
+            log_line='Wishlist already running — skipped',
+            log_type='skip',
+        )
+        return 0
+    except Exception as e:  # noqa: BLE001 — wishlist failure must never abort pipeline
+        deps.update_progress(
+            automation_id,
+            log_line=f'Wishlist error: {e}',
+            log_type='warning',
+        )
+        return 0
+
+
+__all__ = ['run_sync_and_wishlist', 'run_wishlist_phase']

core/automation/handlers/database_update.py

@@ -0,0 +1,197 @@
+"""Automation handlers: ``start_database_update`` and
+``deep_scan_library`` actions.
+
+Lifted from ``web_server._register_automation_handlers`` (the
+``_auto_start_database_update`` and ``_auto_deep_scan_library``
+closures). Both share the same ``db_update_state`` / executor / lock
+infrastructure -- the only difference is which task they submit
+(``run_db_update_task`` vs ``run_deep_scan_task``).
+
+Pattern: pre-set state to running, submit task to executor, then
+poll the state dict until it transitions away from ``running``.
+Stall-detection emits a warning every 10 minutes when progress
+hasn't budged. 2-hour outer timeout caps the worst case.
+"""
+
+from __future__ import annotations
+
+import time
+from typing import Any, Dict
+
+from core.automation.deps import AutomationDeps
+
+
+# Time out on STALL (no progress), not total runtime: a large library can scan
+# for many hours while progressing fine — a hard total cap would falsely mark a
+# healthy scan 'error' (the scan thread keeps running uncancelled). We only give
+# up when progress hasn't moved for a long stretch, with a generous absolute
+# backstop against a truly stuck monitor loop.
+_STALL_WARNING_SECONDS = 600     # warn after 10 min with no progress (repeats)
+_STALL_TIMEOUT_SECONDS = 1800    # 30 min with no progress at all = genuinely stalled
+_ABSOLUTE_CAP_SECONDS = 86400    # 24h hard backstop (runaway-loop guard only)
+_POLL_INTERVAL_SECONDS = 3
+_INITIAL_DELAY_SECONDS = 1
+
+
+def scan_wait_action(
+    *,
+    status: str,
+    idle_seconds: float,
+    total_seconds: float,
+    stall_timeout_s: float = _STALL_TIMEOUT_SECONDS,
+    stall_warn_s: float = _STALL_WARNING_SECONDS,
+    abs_cap_s: float = _ABSOLUTE_CAP_SECONDS,
+) -> str:
+    """Decide what the monitor loop should do on a poll tick (pure/testable).
+
+    ``idle_seconds`` is time since progress last changed; ``total_seconds`` is
+    time since the wait began. Returns one of:
+    ``'finished'`` (task no longer running), ``'stall_timeout'`` (no progress for
+    too long → give up), ``'abs_timeout'`` (absolute backstop), ``'warn'``
+    (stalled long enough to warn but not give up), or ``'continue'``.
+
+    Crucially, an actively-progressing scan keeps resetting ``idle_seconds``, so
+    it never hits ``stall_timeout`` no matter how long the whole scan takes.
+    """
+    if status != 'running':
+        return 'finished'
+    if total_seconds >= abs_cap_s:
+        return 'abs_timeout'
+    if idle_seconds >= stall_timeout_s:
+        return 'stall_timeout'
+    if idle_seconds >= stall_warn_s:
+        return 'warn'
+    return 'continue'
+
+
+def auto_start_database_update(config: Dict[str, Any], deps: AutomationDeps) -> Dict[str, Any]:
+    """Run a full or incremental DB update via ``run_db_update_task``."""
+    return _run_with_progress(
+        config, deps,
+        task=deps.run_db_update_task,
+        task_args=(config.get('full_refresh', False), deps.config_manager.get_active_media_server()),
+        initial_phase='Initializing...',
+        stall_label='Database update',
+        finished_extras=lambda: {'full_refresh': str(config.get('full_refresh', False))},
+        timeout_label='Database update timed out after 24 hours',
+    )
+
+
+def auto_deep_scan_library(config: Dict[str, Any], deps: AutomationDeps) -> Dict[str, Any]:
+    """Run a deep library scan via ``run_deep_scan_task``."""
+    return _run_with_progress(
+        config, deps,
+        task=deps.run_deep_scan_task,
+        task_args=(deps.config_manager.get_active_media_server(),),
+        initial_phase='Deep scan: Initializing...',
+        stall_label='Deep scan',
+        finished_extras=lambda: {},
+        timeout_label='Deep scan timed out after 24 hours',
+    )
+
+
+def _run_with_progress(
+    config: Dict[str, Any],
+    deps: AutomationDeps,
+    *,
+    task,
+    task_args: tuple,
+    initial_phase: str,
+    stall_label: str,
+    finished_extras,
+    timeout_label: str,
+) -> Dict[str, Any]:
+    """Shared poll-and-wait body for both DB-update handlers."""
+    automation_id = config.get('_automation_id')
+    state = deps.get_db_update_state()
+    if state.get('status') == 'running':
+        return {'status': 'skipped', 'reason': 'Database update already running'}
+    deps.state.db_update_automation_id = automation_id
+    # Sync legacy module global so the DB-update progress callbacks
+    # (still living in web_server.py) emit against this automation.
+    deps.set_db_update_automation_id(automation_id)
+
+    with deps.db_update_lock:
+        state.update({
+            'status': 'running', 'phase': initial_phase,
+            'progress': 0, 'current_item': '', 'processed': 0, 'total': 0,
+            'error_message': '',
+        })
+    deps.db_update_executor.submit(task, *task_args)
+
+    # Monitor progress (callbacks handle card updates, we just block until done).
+    # We time out on STALL, not total runtime: ``processed`` advances on every
+    # artist, so an actively-progressing scan keeps resetting the idle clock and
+    # is never falsely failed no matter how long the whole library takes.
+    time.sleep(_INITIAL_DELAY_SECONDS)
+    poll_start = time.time()
+    last_progress_time = time.time()
+    # Any of these advancing means the scan is alive. current_item (the artist
+    # being processed) changes every artist even when the rounded progress %
+    # holds steady, so it guards against a false stall during slow stretches.
+    last_progress_val = (0, 0, '')
+    last_warn_time = 0.0
+    outcome = 'finished'
+    while True:
+        time.sleep(_POLL_INTERVAL_SECONDS)
+        now = time.time()
+        with deps.db_update_lock:
+            current_status = state.get('status', 'idle')
+            current_val = (state.get('processed', 0), state.get('progress', 0),
+                           state.get('current_item', ''))
+        if current_val != last_progress_val:
+            last_progress_val = current_val
+            last_progress_time = now
+
+        action = scan_wait_action(
+            status=current_status,
+            idle_seconds=now - last_progress_time,
+            total_seconds=now - poll_start,
+        )
+        if action in ('finished', 'stall_timeout', 'abs_timeout'):
+            outcome = action
+            break
+        if action == 'warn' and (now - last_warn_time) > _STALL_WARNING_SECONDS:
+            idle_min = int((now - last_progress_time) / 60)
+            deps.update_progress(
+                automation_id,
+                log_line=f'{stall_label} — no progress for {idle_min} min, still waiting...',
+                log_type='warning',
+            )
+            last_warn_time = now
+
+    if outcome == 'stall_timeout':
+        deps.update_progress(
+            automation_id, status='error', phase='Stalled',
+            log_line=f'{stall_label} made no progress for {_STALL_TIMEOUT_SECONDS // 60} minutes — giving up',
+            log_type='error',
+        )
+        return {'status': 'error', 'reason': 'Stalled (no progress)', '_manages_own_progress': True}
+    if outcome == 'abs_timeout':
+        deps.update_progress(
+            automation_id, status='error',
+            phase='Timed out', log_line=timeout_label, log_type='error',
+        )
+        return {'status': 'error', 'reason': 'Timed out', '_manages_own_progress': True}
+
+    # Finished/error callback already updated the card — return matching status.
+    with deps.db_update_lock:
+        final_status = state.get('status', 'unknown')
+    if final_status == 'error':
+        return {
+            'status': 'error',
+            'reason': state.get('error_message', 'Unknown error'),
+            '_manages_own_progress': True,
+        }
+    with deps.db_update_lock:
+        stats = {
+            'status': 'completed', '_manages_own_progress': True,
+            'artists': state.get('total', 0),
+            'albums': state.get('total_albums', 0),
+            'tracks': state.get('total_tracks', 0),
+            'removed_artists': state.get('removed_artists', 0),
+            'removed_albums': state.get('removed_albums', 0),
+            'removed_tracks': state.get('removed_tracks', 0),
+        }
+    stats.update(finished_extras())
+    return stats

core/automation/handlers/discover_playlist.py

@@ -0,0 +1,48 @@
+"""Automation handler: ``discover_playlist`` action.
+
+Lifted from ``web_server._register_automation_handlers`` (the
+``_auto_discover_playlist`` closure). Kicks off background discovery
+of official Spotify / iTunes metadata for mirrored playlist tracks.
+The worker runs in a daemon thread and emits its own progress; this
+handler returns immediately after launching it (``_manages_own_progress``).
+"""
+
+from __future__ import annotations
+
+import threading
+from typing import Any, Dict
+
+from core.automation.deps import AutomationDeps
+
+
+def auto_discover_playlist(config: Dict[str, Any], deps: AutomationDeps) -> Dict[str, Any]:
+    """Discover official Spotify/iTunes metadata for mirrored
+    playlist tracks. Runs the worker in a background thread."""
+    db = deps.get_database()
+    playlist_id = config.get('playlist_id')
+    discover_all = config.get('all', False)
+
+    if discover_all:
+        playlists = db.get_mirrored_playlists()
+    elif playlist_id:
+        p = db.get_mirrored_playlist(int(playlist_id))
+        playlists = [p] if p else []
+    else:
+        return {'status': 'error', 'reason': 'No playlist specified'}
+
+    if not playlists:
+        return {'status': 'error', 'reason': 'No playlists found'}
+
+    threading.Thread(
+        target=deps.run_playlist_discovery_worker,
+        args=(playlists, config.get('_automation_id')),
+        daemon=True,
+        name='auto-discover-playlist',
+    ).start()
+    names = ', '.join(p['name'] for p in playlists[:3])
+    return {
+        'status': 'started',
+        'playlist_count': str(len(playlists)),
+        'playlists': names,
+        '_manages_own_progress': True,
+    }

core/automation/handlers/download_cleanup.py

@@ -0,0 +1,267 @@
+"""Automation handlers: download-queue cleanup actions.
+
+Lifted from ``web_server._register_automation_handlers``:
+- ``clean_search_history``       → :func:`auto_clean_search_history`
+- ``clean_completed_downloads``  → :func:`auto_clean_completed_downloads`
+- ``full_cleanup``               → :func:`auto_full_cleanup`
+
+All three share the download-orchestrator + tasks_lock /
+download_batches / download_tasks accessors. ``full_cleanup`` is a
+multi-step orchestration that pulls in quarantine purge + staging
+sweep on top of the queue cleanup -- kept as one big handler since
+its phases share state-detection logic.
+"""
+
+from __future__ import annotations
+
+import os
+import shutil as _shutil
+from typing import Any, Dict
+
+from core.automation.deps import AutomationDeps
+
+
+# ─── clean_search_history ────────────────────────────────────────────
+
+
+def auto_clean_search_history(config: Dict[str, Any], deps: AutomationDeps) -> Dict[str, Any]:
+    """Remove old searches from Soulseek when configured."""
+    automation_id = config.get('_automation_id')
+    # Skip if soulseek is not the active download source or in hybrid order.
+    dl_mode = deps.config_manager.get('download_source.mode', 'hybrid')
+    hybrid_order = deps.config_manager.get(
+        'download_source.hybrid_order', ['hifi', 'youtube', 'soulseek'],
+    )
+    soulseek_active = (
+        dl_mode == 'soulseek'
+        or (dl_mode == 'hybrid' and 'soulseek' in hybrid_order)
+    )
+    # Reach the underlying SoulseekClient via the orchestrator's
+    # generic accessor.
+    slskd = deps.download_orchestrator.client('soulseek') if deps.download_orchestrator else None
+    if not soulseek_active or not slskd or not slskd.base_url:
+        deps.update_progress(automation_id, log_line='Soulseek not active — skipped', log_type='skip')
+        return {'status': 'skipped'}
+    if not deps.config_manager.get('soulseek.auto_clear_searches', True):
+        deps.update_progress(
+            automation_id, log_line='Auto-clear disabled in settings', log_type='skip',
+        )
+        return {'status': 'skipped'}
+    try:
+        success = deps.run_async(deps.download_orchestrator.maintain_search_history_with_buffer(
+            keep_searches=50, trigger_threshold=200,
+        ))
+        if success:
+            deps.update_progress(
+                automation_id,
+                log_line='Search history maintenance completed',
+                log_type='success',
+            )
+            return {'status': 'completed'}
+        else:
+            deps.update_progress(automation_id, log_line='No cleanup needed', log_type='skip')
+            return {'status': 'completed'}
+    except Exception as e:  # noqa: BLE001 — automation handlers must never raise
+        return {'status': 'error', 'error': str(e)}
+
+
+# ─── clean_completed_downloads ───────────────────────────────────────
+
+
+def auto_clean_completed_downloads(config: Dict[str, Any], deps: AutomationDeps) -> Dict[str, Any]:
+    """Clear completed downloads + sweep empty download directories.
+    Skips when active batches or post-processing is in flight."""
+    automation_id = config.get('_automation_id')
+    try:
+        has_active_batches = False
+        has_post_processing = False
+        with deps.tasks_lock:
+            batches = deps.get_download_batches()
+            for batch_data in batches.values():
+                if batch_data.get('phase') not in ['complete', 'error', 'cancelled', None]:
+                    has_active_batches = True
+                    break
+            if not has_active_batches:
+                tasks = deps.get_download_tasks()
+                for task_data in tasks.values():
+                    if task_data.get('status') == 'post_processing':
+                        has_post_processing = True
+                        break
+
+        if has_active_batches:
+            deps.update_progress(
+                automation_id, log_line='Skipped — downloads active', log_type='skip',
+            )
+            return {'status': 'completed'}
+
+        deps.run_async(deps.download_orchestrator.clear_all_completed_downloads())
+        if not has_post_processing:
+            deps.sweep_empty_download_directories()
+        deps.update_progress(
+            automation_id, log_line='Download cleanup completed', log_type='success',
+        )
+        return {'status': 'completed'}
+    except Exception as e:  # noqa: BLE001 — automation handlers must never raise
+        return {'status': 'error', 'reason': str(e)}
+
+
+# ─── full_cleanup ────────────────────────────────────────────────────
+
+
+def auto_full_cleanup(config: Dict[str, Any], deps: AutomationDeps) -> Dict[str, Any]:
+    """Run all cleanup tasks: quarantine purge → download queue clear
+    → empty-dir sweep → staging sweep → search history."""
+    automation_id = config.get('_automation_id')
+    steps = []
+
+    # --- 1. Clear quarantine ---
+    deps.update_progress(automation_id, phase='Clearing quarantine...', progress=0)
+    quarantine_path = os.path.join(
+        deps.docker_resolve_path(deps.config_manager.get('soulseek.download_path', './downloads')),
+        'ss_quarantine',
+    )
+    q_removed = 0
+    if os.path.exists(quarantine_path):
+        for f in os.listdir(quarantine_path):
+            fp = os.path.join(quarantine_path, f)
+            try:
+                if os.path.isfile(fp):
+                    os.remove(fp)
+                    q_removed += 1
+                elif os.path.isdir(fp):
+                    _shutil.rmtree(fp)
+                    q_removed += 1
+            except Exception as e:  # noqa: BLE001 — best-effort purge
+                deps.logger.debug("quarantine entry purge failed: %s", e)
+    steps.append(f'Quarantine: removed {q_removed} items')
+    deps.update_progress(
+        automation_id,
+        log_line=f'Quarantine: removed {q_removed} items',
+        log_type='success' if q_removed else 'info',
+    )
+
+    # --- 2. Clear completed/errored/cancelled downloads from Soulseek queue ---
+    deps.update_progress(automation_id, phase='Clearing download queue...', progress=20)
+    has_active_batches = False
+    has_post_processing = False
+    with deps.tasks_lock:
+        batches = deps.get_download_batches()
+        for batch_data in batches.values():
+            if batch_data.get('phase') not in ['complete', 'error', 'cancelled', None]:
+                has_active_batches = True
+                break
+        if not has_active_batches:
+            tasks = deps.get_download_tasks()
+            for task_data in tasks.values():
+                if task_data.get('status') == 'post_processing':
+                    has_post_processing = True
+                    break
+    if has_active_batches:
+        steps.append('Download queue: skipped (active batches)')
+        deps.update_progress(
+            automation_id,
+            log_line='Download queue: skipped (active batches)',
+            log_type='skip',
+        )
+    else:
+        try:
+            deps.run_async(deps.download_orchestrator.clear_all_completed_downloads())
+            steps.append('Download queue: cleared')
+            deps.update_progress(
+                automation_id, log_line='Download queue: cleared', log_type='success',
+            )
+        except Exception as e:  # noqa: BLE001 — per-step best-effort
+            steps.append(f'Download queue: error ({e})')
+            deps.update_progress(
+                automation_id,
+                log_line=f'Download queue: error ({e})',
+                log_type='error',
+            )
+
+    # --- 3. Sweep empty download directories ---
+    deps.update_progress(automation_id, phase='Sweeping empty directories...', progress=40)
+    if has_active_batches or has_post_processing:
+        reason = 'active batches' if has_active_batches else 'post-processing active'
+        steps.append(f'Empty directories: skipped ({reason})')
+        deps.update_progress(
+            automation_id,
+            log_line=f'Empty directories: skipped ({reason})',
+            log_type='skip',
+        )
+    else:
+        dirs_removed = deps.sweep_empty_download_directories()
+        steps.append(f'Empty directories: removed {dirs_removed}')
+        deps.update_progress(
+            automation_id,
+            log_line=f'Empty directories: removed {dirs_removed}',
+            log_type='success' if dirs_removed else 'info',
+        )
+
+    # --- 4. Sweep empty staging directories ---
+    deps.update_progress(automation_id, phase='Sweeping import folder...', progress=60)
+    staging_path = deps.get_staging_path()
+    s_removed = 0
+    if os.path.isdir(staging_path):
+        for dirpath, _dirnames, _filenames in os.walk(staging_path, topdown=False):
+            if os.path.normpath(dirpath) == os.path.normpath(staging_path):
+                continue
+            try:
+                entries = os.listdir(dirpath)
+            except OSError:
+                continue
+            visible = [e for e in entries if not e.startswith('.')]
+            if not visible:
+                for hidden in entries:
+                    try:
+                        os.remove(os.path.join(dirpath, hidden))
+                    except Exception as e:  # noqa: BLE001 — best-effort
+                        deps.logger.debug("hidden file cleanup failed: %s", e)
+                try:
+                    os.rmdir(dirpath)
+                    s_removed += 1
+                except OSError:
+                    pass
+    steps.append(f'Staging: removed {s_removed} empty directories')
+    deps.update_progress(
+        automation_id,
+        log_line=f'Staging: removed {s_removed} empty directories',
+        log_type='success' if s_removed else 'info',
+    )
+
+    # --- 5. Clean search history (if enabled) ---
+    deps.update_progress(automation_id, phase='Cleaning search history...', progress=80)
+    try:
+        if not deps.config_manager.get('soulseek.auto_clear_searches', True):
+            steps.append('Search cleanup: disabled in settings')
+            deps.update_progress(
+                automation_id, log_line='Search cleanup: disabled in settings', log_type='skip',
+            )
+        else:
+            deps.run_async(deps.download_orchestrator.maintain_search_history_with_buffer(
+                keep_searches=50, trigger_threshold=200,
+            ))
+            steps.append('Search history: cleaned')
+            deps.update_progress(
+                automation_id, log_line='Search history: cleaned', log_type='success',
+            )
+    except Exception as e:  # noqa: BLE001 — per-step best-effort
+        steps.append(f'Search history: error ({e})')
+        deps.update_progress(
+            automation_id, log_line=f'Search history: error ({e})', log_type='error',
+        )
+
+    total_removed = q_removed + s_removed
+    deps.update_progress(
+        automation_id, status='finished', progress=100,
+        phase='Complete',
+        log_line=f'Full cleanup complete — {total_removed} items removed',
+        log_type='success',
+    )
+    return {
+        'status': 'completed',
+        'quarantine_removed': str(q_removed),
+        'staging_removed': str(s_removed),
+        'total_removed': str(total_removed),
+        'steps': steps,
+        '_manages_own_progress': True,
+    }

core/automation/handlers/duplicate_cleaner.py

@@ -0,0 +1,87 @@
+"""Automation handler: ``run_duplicate_cleaner`` action.
+
+Lifted from ``web_server._register_automation_handlers`` (the
+``_auto_run_duplicate_cleaner`` closure). Submits the duplicate
+cleaner to its executor, then polls the shared state dict until
+the worker transitions away from ``running``.
+"""
+
+from __future__ import annotations
+
+import time
+from typing import Any, Dict
+
+from core.automation.deps import AutomationDeps
+
+
+_TIMEOUT_SECONDS = 7200          # 2 hours
+_POLL_INTERVAL_SECONDS = 3
+_INITIAL_DELAY_SECONDS = 1
+
+
+def auto_run_duplicate_cleaner(config: Dict[str, Any], deps: AutomationDeps) -> Dict[str, Any]:
+    """Kick off the duplicate cleaner and report final stats."""
+    automation_id = config.get('_automation_id')
+    state = deps.get_duplicate_cleaner_state()
+    if state.get('status') == 'running':
+        return {'status': 'skipped', 'reason': 'Duplicate cleaner already running'}
+
+    # Pre-set status before submit so the polling loop doesn't see a
+    # stale 'finished' from a previous run.
+    with deps.duplicate_cleaner_lock:
+        state['status'] = 'running'
+    deps.duplicate_cleaner_executor.submit(deps.run_duplicate_cleaner)
+    deps.update_progress(automation_id, log_line='Duplicate cleaner started', log_type='info')
+
+    # Monitor progress (max 2 hours).
+    time.sleep(_INITIAL_DELAY_SECONDS)
+    poll_start = time.time()
+    while time.time() - poll_start < _TIMEOUT_SECONDS:
+        time.sleep(_POLL_INTERVAL_SECONDS)
+        current_status = state.get('status', 'idle')
+        if current_status not in ('running',):
+            break
+        deps.update_progress(
+            automation_id,
+            phase=state.get('phase', 'Scanning...'),
+            progress=state.get('progress', 0),
+            processed=state.get('files_scanned', 0),
+            total=state.get('total_files', 0),
+        )
+    else:
+        # 2-hour timeout reached.
+        deps.update_progress(
+            automation_id, status='error',
+            phase='Timed out',
+            log_line='Duplicate cleaner timed out after 2 hours',
+            log_type='error',
+        )
+        return {'status': 'error', 'reason': 'Timed out', '_manages_own_progress': True}
+
+    # Check actual exit status (could be 'finished' or 'error').
+    final_status = state.get('status', 'idle')
+    if final_status == 'error':
+        err = state.get('error_message', 'Unknown error')
+        deps.update_progress(
+            automation_id, status='error', progress=100,
+            phase='Error', log_line=err, log_type='error',
+        )
+        return {'status': 'error', 'reason': err, '_manages_own_progress': True}
+
+    dupes = state.get('duplicates_found', 0)
+    removed = state.get('deleted', 0)
+    space_freed = state.get('space_freed', 0)
+    scanned = state.get('files_scanned', 0)
+    deps.update_progress(
+        automation_id, status='finished', progress=100,
+        phase='Complete',
+        log_line=f'Found {dupes} duplicates, removed {removed} files',
+        log_type='success',
+    )
+    return {
+        'status': 'completed', '_manages_own_progress': True,
+        'files_scanned': scanned,
+        'duplicates_found': dupes,
+        'files_deleted': removed,
+        'space_freed_mb': round(space_freed / (1024 * 1024), 1),
+    }

core/automation/handlers/maintenance.py

@@ -0,0 +1,221 @@
+"""Automation handlers: short maintenance actions.
+
+Lifted from ``web_server._register_automation_handlers``:
+- ``clear_quarantine``       → :func:`auto_clear_quarantine`
+- ``cleanup_wishlist``       → :func:`auto_cleanup_wishlist`
+- ``update_discovery_pool``  → :func:`auto_update_discovery_pool`
+- ``backup_database``        → :func:`auto_backup_database`
+- ``refresh_beatport_cache`` → :func:`auto_refresh_beatport_cache`
+
+Each is a thin wrapper around an existing service / helper. Grouped
+in one module because every body is short and they share no state
+between them — splitting into per-handler files would just add
+import noise.
+"""
+
+from __future__ import annotations
+
+import glob as _glob
+import os
+import shutil as _shutil
+import sqlite3
+import time
+from datetime import datetime
+from typing import Any, Dict
+
+from core.automation.deps import AutomationDeps
+
+
+# ─── clear_quarantine ────────────────────────────────────────────────
+
+
+def auto_clear_quarantine(config: Dict[str, Any], deps: AutomationDeps) -> Dict[str, Any]:
+    """Purge every file/folder under the configured ss_quarantine path."""
+    automation_id = config.get('_automation_id')
+    quarantine_path = os.path.join(
+        deps.docker_resolve_path(deps.config_manager.get('soulseek.download_path', './downloads')),
+        'ss_quarantine',
+    )
+    if not os.path.exists(quarantine_path):
+        deps.update_progress(automation_id, log_line='No quarantine folder found', log_type='info')
+        return {'status': 'completed', 'removed': '0'}
+    removed = 0
+    for f in os.listdir(quarantine_path):
+        fp = os.path.join(quarantine_path, f)
+        try:
+            if os.path.isfile(fp):
+                os.remove(fp)
+                removed += 1
+            elif os.path.isdir(fp):
+                _shutil.rmtree(fp)
+                removed += 1
+        except Exception as e:  # noqa: BLE001 — best-effort purge
+            deps.logger.debug("quarantine entry purge failed: %s", e)
+    deps.update_progress(
+        automation_id,
+        log_line=f'Removed {removed} quarantined items',
+        log_type='success' if removed > 0 else 'info',
+    )
+    return {'status': 'completed', 'removed': str(removed)}
+
+
+# ─── cleanup_wishlist ────────────────────────────────────────────────
+
+
+def auto_cleanup_wishlist(config: Dict[str, Any], deps: AutomationDeps) -> Dict[str, Any]:
+    """Drop duplicate entries from the wishlist for the active profile."""
+    automation_id = config.get('_automation_id')
+    db = deps.get_database()
+    removed = db.remove_wishlist_duplicates(deps.get_current_profile_id())
+    deps.update_progress(
+        automation_id,
+        log_line=f'Removed {removed or 0} duplicate wishlist entries',
+        log_type='success' if removed else 'info',
+    )
+    return {'status': 'completed', 'removed': str(removed or 0)}
+
+
+# ─── update_discovery_pool ───────────────────────────────────────────
+
+
+def auto_update_discovery_pool(config: Dict[str, Any], deps: AutomationDeps) -> Dict[str, Any]:
+    """Run an incremental refresh of the discovery pool via the
+    watchlist scanner."""
+    automation_id = config.get('_automation_id')
+    try:
+        scanner = deps.get_watchlist_scanner(deps.spotify_client)
+        deps.update_progress(automation_id, log_line='Updating discovery pool...', log_type='info')
+        scanner.update_discovery_pool_incremental(deps.get_current_profile_id())
+        deps.update_progress(
+            automation_id, status='finished', progress=100,
+            phase='Complete', log_line='Discovery pool updated', log_type='success',
+        )
+        return {'status': 'completed', '_manages_own_progress': True}
+    except Exception as e:  # noqa: BLE001 — automation handlers must never raise
+        deps.update_progress(
+            automation_id, status='error',
+            phase='Error', log_line=str(e), log_type='error',
+        )
+        return {'status': 'error', 'reason': str(e), '_manages_own_progress': True}
+
+
+# ─── backup_database ─────────────────────────────────────────────────
+
+
+_MAX_BACKUPS = 5
+
+
+def auto_backup_database(config: Dict[str, Any], deps: AutomationDeps) -> Dict[str, Any]:
+    """Create a hot SQLite backup, then prune old backups so only the
+    newest ``_MAX_BACKUPS`` remain."""
+    automation_id = config.get('_automation_id')
+    db_path = os.environ.get('DATABASE_PATH', 'database/music_library.db')
+    if not os.path.exists(db_path):
+        return {'status': 'error', 'reason': 'Database file not found'}
+
+    timestamp = datetime.now().strftime('%Y%m%d_%H%M%S')
+    backup_path = f"{db_path}.backup_{timestamp}"
+    # safe_backup verifies source + result integrity, so an automated backup
+    # can never silently snapshot a corrupt DB (the incident where every
+    # rolling backup faithfully copied the corruption).
+    from core.db_integrity import DBIntegrityError, safe_backup, prune_backups
+    try:
+        safe_backup(db_path, backup_path)
+    except DBIntegrityError as integ:
+        deps.logger.error("Auto-backup refused — DB integrity check failed: %s", integ)
+        deps.update_progress(
+            automation_id,
+            log_line=f'Backup SKIPPED — database failed integrity check: {integ}',
+            log_type='error',
+        )
+        return {'status': 'error', 'reason': f'Database integrity check failed: {integ}'}
+    size_mb = round(os.path.getsize(backup_path) / (1024 * 1024), 1)
+
+    # Rolling cleanup — never evict the most-recent verified-healthy backup.
+    existing = list(_glob.glob(f"{db_path}.backup_*"))
+    for removed in prune_backups(existing, _MAX_BACKUPS):
+        try:
+            os.remove(removed)
+        except Exception as e:  # noqa: BLE001 — best-effort cleanup
+            deps.logger.debug("rolling backup cleanup failed: %s", e)
+    deps.update_progress(
+        automation_id,
+        log_line=f'Backup created: {size_mb}MB ({os.path.basename(backup_path)})',
+        log_type='success',
+    )
+    return {'status': 'completed', 'backup_path': backup_path, 'size_mb': str(size_mb)}
+
+
+# ─── refresh_beatport_cache ──────────────────────────────────────────
+
+
+_BEATPORT_SECTIONS = (
+    ('hero_tracks', '/api/beatport/hero-tracks', 'Hero Tracks'),
+    ('new_releases', '/api/beatport/new-releases', 'New Releases'),
+    ('featured_charts', '/api/beatport/featured-charts', 'Featured Charts'),
+    ('dj_charts', '/api/beatport/dj-charts', 'DJ Charts'),
+    ('top_10_lists', '/api/beatport/homepage/top-10-lists', 'Top 10 Lists'),
+    ('top_10_releases', '/api/beatport/homepage/top-10-releases-cards', 'Top 10 Releases'),
+    ('hype_picks', '/api/beatport/hype-picks', 'Hype Picks'),
+)
+
+
+def auto_refresh_beatport_cache(config: Dict[str, Any], deps: AutomationDeps) -> Dict[str, Any]:
+    """Refresh Beatport homepage cache by calling each endpoint internally
+    via Flask's ``test_client``. Invalidates the homepage cache first
+    so endpoints re-scrape rather than returning stale data."""
+    automation_id = config.get('_automation_id')
+    cache = deps.get_beatport_data_cache()
+    # Invalidate all homepage cache timestamps so endpoints re-scrape.
+    with cache['cache_lock']:
+        for key in cache['homepage']:
+            cache['homepage'][key]['timestamp'] = 0
+            cache['homepage'][key]['data'] = None
+
+    refreshed = 0
+    errors = []
+    app = deps.get_app()
+    with app.test_client() as client:
+        for idx, (_, endpoint, label) in enumerate(_BEATPORT_SECTIONS):
+            deps.update_progress(
+                automation_id,
+                progress=(idx / len(_BEATPORT_SECTIONS)) * 100,
+                phase=f'Scraping: {label}',
+                current_item=label,
+            )
+            try:
+                resp = client.get(endpoint)
+                if resp.status_code == 200:
+                    refreshed += 1
+                    deps.update_progress(
+                        automation_id, log_line=f'{label}: cached', log_type='success',
+                    )
+                else:
+                    errors.append(label)
+                    deps.update_progress(
+                        automation_id,
+                        log_line=f'{label}: HTTP {resp.status_code}',
+                        log_type='error',
+                    )
+            except Exception as e:  # noqa: BLE001 — per-section best-effort
+                errors.append(label)
+                deps.update_progress(
+                    automation_id,
+                    log_line=f'{label}: {str(e)}',
+                    log_type='error',
+                )
+            if idx < len(_BEATPORT_SECTIONS) - 1:
+                time.sleep(2)
+
+    deps.update_progress(
+        automation_id, status='finished', progress=100,
+        phase='Complete',
+        log_line=f'Refreshed {refreshed}/{len(_BEATPORT_SECTIONS)} sections',
+        log_type='success',
+    )
+    return {
+        'status': 'completed',
+        'refreshed': str(refreshed),
+        'errors': str(len(errors)),
+        '_manages_own_progress': True,
+    }

core/automation/handlers/personalized_pipeline.py

@@ -0,0 +1,356 @@
+"""Personalized Playlist Pipeline automation handler.
+
+Sibling to ``auto_playlist_pipeline`` (mirrored). Where the mirrored
+pipeline runs REFRESH external sources → DISCOVER metadata → SYNC →
+WISHLIST, the personalized pipeline is simpler:
+
+    SNAPSHOT → SYNC → WISHLIST
+
+SNAPSHOT reads the persisted track list from
+``PersonalizedPlaylistManager``. When ``refresh_first=True`` (config),
+each playlist is refreshed BEFORE syncing — useful when the user
+wants the cron to capture a fresh-each-run view (e.g. "give me a new
+Hidden Gems set every night"). Default is to sync the existing
+snapshot, on the assumption the user / a separate cron has already
+refreshed when they wanted to.
+
+Config schema:
+    {
+        'kinds': [
+            {'kind': 'hidden_gems'},
+            {'kind': 'time_machine', 'variant': '1980s'},
+            {'kind': 'seasonal_mix', 'variant': 'halloween'},
+            ...
+        ],
+        'refresh_first': bool,    # default false
+        'skip_wishlist': bool,    # default false
+    }
+
+Each kind dict has at minimum ``kind``; ``variant`` is required for
+kinds that need it (time_machine, genre_playlist, daily_mix,
+seasonal_mix). Singleton kinds (hidden_gems, discovery_shuffle,
+popular_picks, fresh_tape, archives) ignore variant.
+
+Pipeline-running flag (``deps.state.pipeline_running``) is shared
+with the mirrored pipeline so the two can't overlap. (One sync
+queue, one wishlist worker — overlapping triggers would step on
+each other.)"""
+
+from __future__ import annotations
+
+import json
+import threading
+import time
+from typing import Any, Dict, List, Optional
+
+from core.automation.deps import AutomationDeps
+from core.automation.handlers._pipeline_shared import run_sync_and_wishlist
+
+
+# Sync state key prefix so personalized syncs don't collide with
+# mirrored ones (`auto_mirror_<id>`).
+_SYNC_ID_PREFIX = 'auto_personalized'
+
+
+def auto_personalized_pipeline(config: Dict[str, Any], deps: AutomationDeps) -> Dict[str, Any]:
+    """Run SNAPSHOT → SYNC → WISHLIST for selected personalized playlists."""
+    deps.state.set_pipeline_running(True)
+    automation_id = config.get('_automation_id')
+    pipeline_start = time.time()
+
+    try:
+        kinds_config = config.get('kinds') or []
+        if not isinstance(kinds_config, list) or not kinds_config:
+            deps.state.set_pipeline_running(False)
+            return {
+                'status': 'error',
+                'error': 'No personalized playlist kinds selected',
+            }
+
+        refresh_first = bool(config.get('refresh_first', False))
+        skip_wishlist = bool(config.get('skip_wishlist', False))
+
+        manager = deps.build_personalized_manager()
+
+        deps.update_progress(
+            automation_id,
+            progress=2,
+            phase=f'Personalized pipeline: {len(kinds_config)} playlist(s)',
+            log_line=f'Starting pipeline for {len(kinds_config)} playlist(s)',
+            log_type='info',
+        )
+
+        # ── PHASE 1: SNAPSHOT (optionally refresh) ──────────────────
+        deps.update_progress(
+            automation_id,
+            progress=3,
+            phase='Phase 1/2: Loading snapshots...' if not refresh_first
+                  else 'Phase 1/2: Refreshing snapshots...',
+            log_line='Phase 1: Snapshot' + (' (with refresh)' if refresh_first else ''),
+            log_type='info',
+        )
+
+        profile_id = deps.get_current_profile_id()
+        playload_payloads = _build_payloads_for_kinds(
+            deps, manager, kinds_config, profile_id,
+            automation_id=automation_id,
+            refresh_first=refresh_first,
+        )
+
+        if not playload_payloads:
+            deps.state.set_pipeline_running(False)
+            deps.update_progress(
+                automation_id,
+                status='finished', progress=100,
+                phase='No playlists to sync',
+                log_line='No personalized playlists had tracks to sync',
+                log_type='warning',
+            )
+            return {
+                'status': 'completed',
+                '_manages_own_progress': True,
+                'playlists_synced': '0',
+                'tracks_synced': '0',
+                'duration_seconds': str(int(time.time() - pipeline_start)),
+            }
+
+        deps.update_progress(
+            automation_id,
+            progress=50,
+            phase='Phase 1/2: Snapshot complete',
+            log_line=f'Phase 1 done: {len(playload_payloads)} playlist(s) ready to sync',
+            log_type='success',
+        )
+
+        # ── PHASE 2: SYNC + WISHLIST (shared helper) ────────────────
+        sync_summary = run_sync_and_wishlist(
+            deps,
+            automation_id,
+            playload_payloads,
+            sync_one_fn=lambda pl: _sync_personalized_playlist(deps, pl),
+            sync_id_for_fn=lambda pl: pl['sync_id'],
+            skip_wishlist=skip_wishlist,
+            progress_start=51,
+            progress_end=90,
+            sync_phase_label='Phase 2/2: Syncing to server...',
+            sync_phase_start_log='Phase 2: Sync',
+            wishlist_phase_label='Phase 2/2: Processing wishlist...',
+            wishlist_phase_start_log='Wishlist',
+        )
+
+        # ── COMPLETE ────────────────────────────────────────────────
+        duration = int(time.time() - pipeline_start)
+        deps.update_progress(
+            automation_id,
+            status='finished', progress=100,
+            phase='Pipeline complete',
+            log_line=f'Personalized pipeline finished in {duration // 60}m {duration % 60}s',
+            log_type='success',
+        )
+
+        deps.state.set_pipeline_running(False)
+        return {
+            'status': 'completed',
+            '_manages_own_progress': True,
+            'playlists_synced': str(len(playload_payloads)),
+            'tracks_synced': str(sync_summary['synced']),
+            'sync_skipped': str(sync_summary['skipped']),
+            'wishlist_queued': str(sync_summary['wishlist_queued']),
+            'duration_seconds': str(duration),
+        }
+
+    except Exception as e:  # noqa: BLE001 — automation handlers must never raise into engine
+        deps.state.set_pipeline_running(False)
+        deps.update_progress(
+            automation_id,
+            status='error', progress=100,
+            phase='Pipeline error',
+            log_line=f'Personalized pipeline failed: {e}',
+            log_type='error',
+        )
+        return {'status': 'error', 'error': str(e), '_manages_own_progress': True}
+
+
+def _build_payloads_for_kinds(
+    deps: AutomationDeps,
+    manager: Any,
+    kinds_config: List[Dict[str, Any]],
+    profile_id: int,
+    *,
+    automation_id: Optional[str],
+    refresh_first: bool,
+) -> List[Dict[str, Any]]:
+    """Resolve each requested kind+variant into a sync-payload dict.
+
+    Each payload has: ``{'name', 'kind', 'variant', 'tracks_json',
+    'image_url', 'sync_id'}``. Playlists with no tracks (e.g. a
+    seasonal mix that hasn't been populated yet) are omitted from
+    the result so the sync loop doesn't waste time on empty pushes.
+    """
+    payloads: List[Dict[str, Any]] = []
+    for entry in kinds_config:
+        if not isinstance(entry, dict):
+            continue
+        kind = entry.get('kind')
+        variant = entry.get('variant') or ''
+        if not kind:
+            continue
+
+        try:
+            # Refresh when ANY of:
+            #   - explicit user flag (cron use case: regenerate each run)
+            #   - snapshot marked stale by upstream data refresher
+            #   - playlist was never generated yet (auto-created by
+            #     ensure_playlist; track_count=0, last_generated_at=NULL).
+            #     Without this branch, a first-run pipeline reads the
+            #     empty snapshot and silently skips — user picks a kind,
+            #     hits run, gets "No tracks to sync" with no clue why.
+            if refresh_first:
+                record = manager.refresh_playlist(kind, variant, profile_id)
+            else:
+                existing = manager.ensure_playlist(kind, variant, profile_id)
+                needs_first_gen = existing.last_generated_at is None
+                if existing.is_stale or needs_first_gen:
+                    record = manager.refresh_playlist(kind, variant, profile_id)
+                else:
+                    record = existing
+        except Exception as exc:  # noqa: BLE001 — log + continue with next kind
+            deps.update_progress(
+                automation_id,
+                log_line=f'Skipping {kind}{("/" + variant) if variant else ""}: {exc}',
+                log_type='warning',
+            )
+            continue
+
+        tracks = manager.get_playlist_tracks(record.id)
+        if not tracks:
+            deps.update_progress(
+                automation_id,
+                log_line=f'No tracks in {record.name} — skipping sync',
+                log_type='skip',
+            )
+            continue
+
+        tracks_json = [_track_to_sync_shape(t) for t in tracks]
+        payloads.append({
+            'name': record.name,
+            'kind': record.kind,
+            'variant': record.variant,
+            'tracks_json': tracks_json,
+            'image_url': '',  # personalized playlists don't have a cover image yet
+            'sync_id': f'{_SYNC_ID_PREFIX}_{record.kind}_{record.variant or "_"}',
+        })
+    return payloads
+
+
+def _track_to_sync_shape(track: Any) -> Dict[str, Any]:
+    """Convert a personalized.types.Track into the dict shape
+    `_run_sync_task` expects. Mirrors what the mirrored pipeline
+    builds from extra_data.matched_data, preserving enriched metadata
+    from personalized snapshots when available."""
+    primary_id = track.spotify_track_id or track.itunes_track_id or track.deezer_track_id or ''
+    rich_data = _coerce_track_data_json(getattr(track, 'track_data_json', None))
+
+    if not rich_data:
+        album = {'name': track.album_name or ''}
+        cover_url = getattr(track, 'album_cover_url', None)
+        if cover_url:
+            album['images'] = [{'url': cover_url}]
+        return {
+            'name': track.track_name,
+            'artists': [{'name': track.artist_name}],
+            'album': album,
+            'duration_ms': int(track.duration_ms or 0),
+            'id': primary_id,
+        }
+
+    payload = dict(rich_data)
+    cover_url = (
+        getattr(track, 'album_cover_url', None)
+        or payload.get('album_cover_url')
+        or payload.get('image_url')
+    )
+    payload['id'] = payload.get('id') or primary_id
+    payload['name'] = payload.get('name') or track.track_name
+    payload['artists'] = _normalize_artists(payload.get('artists'), track.artist_name)
+    payload['album'] = _normalize_album(payload.get('album'), track, cover_url=cover_url)
+    payload['duration_ms'] = int(payload.get('duration_ms') or track.duration_ms or 0)
+    if 'popularity' not in payload and getattr(track, 'popularity', None) is not None:
+        payload['popularity'] = int(track.popularity or 0)
+
+    if cover_url and not payload.get('image_url'):
+        payload['image_url'] = cover_url
+
+    return payload
+
+
+def _coerce_track_data_json(value: Any) -> Dict[str, Any]:
+    if isinstance(value, dict):
+        return value
+    if isinstance(value, str) and value.strip():
+        try:
+            loaded = json.loads(value)
+        except (TypeError, ValueError):
+            return {}
+        return loaded if isinstance(loaded, dict) else {}
+    return {}
+
+
+def _normalize_artists(artists: Any, fallback_artist: str) -> List[Dict[str, Any]]:
+    if not artists:
+        return [{'name': fallback_artist or 'Unknown Artist'}]
+    if isinstance(artists, list):
+        normalized = []
+        for artist in artists:
+            if isinstance(artist, dict):
+                normalized.append(artist if artist.get('name') else {'name': str(artist)})
+            elif isinstance(artist, str):
+                normalized.append({'name': artist})
+            else:
+                normalized.append({'name': str(artist)})
+        return normalized or [{'name': fallback_artist or 'Unknown Artist'}]
+    if isinstance(artists, dict):
+        return [artists if artists.get('name') else {'name': str(artists)}]
+    return [{'name': str(artists)}]
+
+
+def _normalize_album(album: Any, track: Any, cover_url: Optional[str] = None) -> Dict[str, Any]:
+    if isinstance(album, dict):
+        normalized = dict(album)
+    else:
+        normalized = {'name': str(album) if album else (track.album_name or '')}
+
+    normalized['name'] = normalized.get('name') or track.album_name or ''
+    images = normalized.get('images')
+    if not isinstance(images, list):
+        images = []
+    if cover_url and not images:
+        images = [{'url': cover_url}]
+    if images:
+        normalized['images'] = images
+    return normalized
+
+
+def _sync_personalized_playlist(deps: AutomationDeps, payload: Dict[str, Any]) -> Dict[str, Any]:
+    """Launch a personalized playlist sync via _run_sync_task on a
+    daemon thread + return immediately with status='started'.
+
+    Mirrors the mirrored ``auto_sync_playlist`` return contract so the
+    shared helper can poll on ``sync_states[sync_id]`` and aggregate
+    results identically."""
+    sync_id = payload['sync_id']
+    name = payload['name']
+    tracks_json = payload['tracks_json']
+    profile_id = deps.get_current_profile_id()
+
+    threading.Thread(
+        target=deps.run_sync_task,
+        args=(sync_id, name, tracks_json, None, profile_id, payload.get('image_url', '')),
+        daemon=True,
+        name=f'auto-personalized-{sync_id}',
+    ).start()
+    return {
+        'status': 'started',
+        'playlist_name': name,
+        '_manages_own_progress': True,
+    }

core/automation/handlers/playlist_pipeline.py

@@ -0,0 +1,27 @@
+"""Automation adapter for the mirrored playlist pipeline.
+
+The actual all-in-one playlist lifecycle lives in
+``core.playlists.pipeline`` so it can be reused by non-automation UI actions.
+This module only wires automation-specific dependencies and handlers.
+"""
+
+from __future__ import annotations
+
+from typing import Any, Dict
+
+from core.automation.deps import AutomationDeps
+from core.automation.handlers._pipeline_shared import run_sync_and_wishlist
+from core.automation.handlers.refresh_mirrored import auto_refresh_mirrored
+from core.automation.handlers.sync_playlist import auto_sync_playlist
+from core.playlists.pipeline import run_mirrored_playlist_pipeline
+
+
+def auto_playlist_pipeline(config: Dict[str, Any], deps: AutomationDeps) -> Dict[str, Any]:
+    """Run REFRESH -> DISCOVER -> SYNC -> WISHLIST for mirrored playlists."""
+    return run_mirrored_playlist_pipeline(
+        config,
+        deps,
+        refresh_fn=auto_refresh_mirrored,
+        sync_one_fn=auto_sync_playlist,
+        sync_and_wishlist_fn=run_sync_and_wishlist,
+    )

core/automation/handlers/process_wishlist.py

@@ -0,0 +1,27 @@
+"""Automation handler: ``process_wishlist`` action.
+
+Lifted from ``web_server._register_automation_handlers`` (the
+``_auto_process_wishlist`` closure). Wishlist processing is async —
+the helper submits a batch to an executor and returns immediately;
+per-track stats arrive later via batch-completion callbacks.
+"""
+
+from __future__ import annotations
+
+from typing import Any, Dict
+
+from core.automation.deps import AutomationDeps
+
+
+def auto_process_wishlist(config: Dict[str, Any], deps: AutomationDeps) -> Dict[str, Any]:
+    """Kick off the wishlist processor for an automation trigger.
+
+    Returns immediately after submission; the wishlist worker emits
+    per-batch progress via its own callbacks. We only report
+    ``status: completed`` to mark the trigger fired successfully.
+    """
+    try:
+        deps.process_wishlist_automatically(automation_id=config.get('_automation_id'))
+        return {'status': 'completed'}
+    except Exception as e:  # noqa: BLE001 — automation handlers must never raise into the engine
+        return {'status': 'error', 'error': str(e)}

core/automation/handlers/progress_callbacks.py

@@ -0,0 +1,89 @@
+"""Progress + history callbacks the automation engine invokes around
+each handler run.
+
+Lifted from the closures at the bottom of
+``web_server._register_automation_handlers``:
+- ``_progress_init``               → :func:`progress_init`
+- ``_progress_finish``             → :func:`progress_finish`
+- ``_record_automation_history``   → :func:`record_history`
+- ``_on_library_scan_completed``   → :func:`on_library_scan_completed`
+
+The engine accepts four callables via
+``register_progress_callbacks(init, finish, update, history)``;
+``registration.register_all`` wires these here. The
+``library_scan_completed`` callback is registered separately on the
+``web_scan_manager`` (when one is available) -- see
+``register_library_scan_completed_emitter``.
+"""
+
+from __future__ import annotations
+
+from typing import Any, Dict
+
+from core.automation.deps import AutomationDeps
+
+
+def progress_init(aid: Any, name: str, action_type: str, deps: AutomationDeps) -> None:
+    """Initialize per-automation progress state when the engine starts
+    a handler. Thin wrapper so the engine receives a closure that
+    delegates into the live progress tracker."""
+    deps.init_automation_progress(aid, name, action_type)
+
+
+def progress_finish(aid: Any, result: Dict[str, Any], deps: AutomationDeps) -> None:
+    """Emit the final progress update when a handler returns.
+
+    Skipped for handlers that manage their own progress lifecycle
+    (they call ``update_progress(status='finished')`` themselves and
+    set ``_manages_own_progress: True`` in the returned dict).
+    Otherwise translates the handler's status into a finished/error
+    progress emit with a status-appropriate phase + log line.
+    """
+    if result.get('_manages_own_progress'):
+        return
+    result_status = result.get('status', '')
+    status = 'error' if result_status == 'error' else 'finished'
+    msg = result.get('error', result.get('reason', result_status or 'done'))
+    deps.update_progress(
+        aid,
+        status=status,
+        progress=100,
+        phase='Error' if status == 'error' else 'Complete',
+        log_line=msg,
+        log_type='error' if status == 'error' else 'success',
+    )
+
+
+def record_history(aid: Any, result: Dict[str, Any], deps: AutomationDeps) -> None:
+    """Capture progress state into run history before the engine's
+    cleanup pass clears it. Thin wrapper so the engine sees a stable
+    callable."""
+    deps.record_progress_history(aid, result, deps.get_database())
+
+
+def on_library_scan_completed(deps: AutomationDeps) -> None:
+    """Emit the ``library_scan_completed`` automation event with the
+    active media-server type. Replaces the hard-coded
+    ``scan_completion_callback → trigger_automatic_database_update``
+    chain so any automation can listen for scan completion as a
+    trigger."""
+    if not deps.engine:
+        return
+    server_type = (
+        getattr(deps.web_scan_manager, '_current_server_type', None)
+        or 'unknown'
+    )
+    deps.engine.emit('library_scan_completed', {
+        'server_type': server_type,
+    })
+
+
+def register_library_scan_completed_emitter(deps: AutomationDeps) -> None:
+    """Wire :func:`on_library_scan_completed` to the
+    ``web_scan_manager``'s scan-completion callback list. No-op when
+    no scan manager is configured (e.g. headless / test contexts)."""
+    if not deps.web_scan_manager:
+        return
+    deps.web_scan_manager.add_scan_completion_callback(
+        lambda: on_library_scan_completed(deps),
+    )

core/automation/handlers/quality_scanner.py

@@ -0,0 +1,35 @@
+"""Automation handler: ``start_quality_scan`` action.
+
+The quality scanner was redesigned from an auto-acting tool into the
+``quality_upgrade`` library-maintenance repair job (findings-based, reviewed
+before anything is wishlisted). This action now simply triggers a "Run Now" of
+that job; its progress and findings surface in Library Maintenance. The action
+name is kept so existing automation rules keep working.
+"""
+
+from __future__ import annotations
+
+from typing import Any, Dict
+
+from core.automation.deps import AutomationDeps
+
+
+def auto_start_quality_scan(config: Dict[str, Any], deps: AutomationDeps) -> Dict[str, Any]:
+    automation_id = config.get('_automation_id')
+
+    triggered = deps.run_repair_job_now('quality_upgrade')
+    if not triggered:
+        deps.update_progress(
+            automation_id, status='error', phase='Unavailable',
+            log_line='Quality Upgrade job could not be triggered (library worker unavailable)',
+            log_type='error',
+        )
+        return {'status': 'error', 'reason': 'library worker unavailable',
+                '_manages_own_progress': True}
+
+    deps.update_progress(
+        automation_id, status='finished', progress=100, phase='Triggered',
+        log_line='Quality Upgrade scan queued — findings appear in Library Maintenance',
+        log_type='success',
+    )
+    return {'status': 'completed', 'triggered': True, '_manages_own_progress': True}

core/automation/handlers/refresh_mirrored.py

@@ -0,0 +1,341 @@
+"""Automation handler: ``refresh_mirrored`` action.
+
+Re-pulls track lists from each mirrored playlist's source via the
+unified ``PlaylistSourceRegistry`` (Phase 1 of the Discover-to-Sync
+unification). The pre-extraction handler had ~190 lines of per-source
+if/elif branches; this version delegates to the adapter for each
+source, leaving the handler responsible only for:
+
+- filtering sources that can't be refreshed (``file``, ``beatport``),
+- extracting upstream URLs from the stored ``description`` for URL-
+  backed sources (``spotify_public``, ``youtube``),
+- the Spotify-public → authenticated-Spotify fallback (uses the
+  ``spotify`` adapter when the user is signed in so the mirror keeps
+  album art),
+- the Tidal-not-authenticated skip log type (vs error),
+- preserving existing per-track ``extra_data`` on tracks that survive
+  the refresh, and
+- emitting the ``playlist_changed`` automation event when the track
+  set actually shifts.
+"""
+
+from __future__ import annotations
+
+from typing import Any, Dict, List, Optional
+
+from core.automation.deps import AutomationDeps
+from core.playlists.source_refs import require_refresh_url
+from core.playlists.sources import PlaylistDetail, to_mirror_track_dict
+from core.playlists.sources.base import (
+    SOURCE_SPOTIFY,
+    SOURCE_SPOTIFY_PUBLIC,
+    SOURCE_TIDAL,
+    SOURCE_YOUTUBE,
+)
+
+
+# Sources that store the upstream URL in ``description`` (because their
+# ``source_playlist_id`` is a deterministic hash, not the native ID).
+# The refresh path has to recover the URL before calling the adapter.
+_URL_BACKED_SOURCES = {SOURCE_SPOTIFY_PUBLIC, SOURCE_YOUTUBE}
+
+
+def auto_refresh_mirrored(config: Dict[str, Any], deps: AutomationDeps) -> Dict[str, Any]:
+    """Refresh mirrored playlist(s) from source.
+
+    Returns ``{'status': 'completed', 'refreshed': '<int>',
+    'errors': '<int>'}`` on success (counts stringified to match the
+    automation engine's stat-rendering convention).
+    """
+    db = deps.get_database()
+    playlist_id = config.get('playlist_id')
+    refresh_all = config.get('all', False)
+    auto_id = config.get('_automation_id')
+    # Pipeline runs (``run_mirrored_playlist_pipeline``) set this flag
+    # because Phase 2 of the pipeline already runs the playlist
+    # discovery worker — the same matching engine ``_maybe_discover``
+    # would call here. Running both means LB tracks discover twice
+    # AND the refresh-side discovery blocks 5+ minutes with no
+    # progress emission, leaving the UI stuck on "Refreshing:" until
+    # the loop returns. Standalone callers (Sync page, registration
+    # action) leave it False so LB tracks still get matched_data on
+    # refresh.
+    skip_discovery = bool(config.get('skip_discovery', False))
+
+    if refresh_all:
+        playlists = db.get_mirrored_playlists()
+    elif playlist_id:
+        p = db.get_mirrored_playlist(int(playlist_id))
+        playlists = [p] if p else []
+    else:
+        return {'status': 'error', 'reason': 'No playlist specified'}
+
+    # Filter out sources that can't be refreshed (no external API).
+    playlists = [pl for pl in playlists if pl.get('source', '') not in ('file', 'beatport')]
+
+    refreshed = 0
+    errors: List[str] = []
+    for idx, pl in enumerate(playlists):
+        try:
+            source = pl.get('source', '')
+            source_id = pl.get('source_playlist_id', '')
+            deps.update_progress(
+                auto_id,
+                progress=(idx / max(1, len(playlists))) * 100,
+                phase=f'Refreshing: "{pl.get("name", "")}"',
+                current_item=pl.get('name', ''),
+            )
+
+            detail = _fetch_detail(source, source_id, pl, deps, auto_id)
+            if detail is None:
+                # _fetch_detail already logged the specific failure;
+                # mark the playlist as a generic refresh error so the
+                # automation result tally matches the legacy handler.
+                errors.append(f"{pl.get('name', '?')}: no tracks returned from source")
+                deps.update_progress(
+                    auto_id,
+                    log_line=f'Refresh failed: "{pl.get("name", "")}" - no tracks returned from source',
+                    log_type='error',
+                )
+                continue
+
+            # Sources that return MB-metadata-only tracks (LB, Last.fm)
+            # mark them ``needs_discovery=True``. Hand them to the
+            # adapter's matcher so the resulting mirror rows carry
+            # provider IDs + matched_data, ready for the sync pipeline.
+            #
+            # Pipeline runs skip this because Phase 2's discovery
+            # worker handles it with proper progress emission — see
+            # ``skip_discovery`` resolution at the top of this fn.
+            detail_tracks = (
+                detail.tracks if skip_discovery
+                else _maybe_discover(detail.tracks, source, deps)
+            )
+
+            tracks = [to_mirror_track_dict(t) for t in detail_tracks]
+            refreshed += _commit_refresh(pl, source, source_id, tracks, db, deps, auto_id)
+        except _SkipPlaylist:
+            # Source-specific soft-skip (e.g. Tidal not authenticated).
+            # Logging was already emitted; do not count as error.
+            continue
+        except Exception as e:
+            errors.append(f"{pl.get('name', '?')}: {str(e)}")
+            deps.update_progress(
+                auto_id,
+                log_line=f'Error: {pl.get("name", "?")} — {str(e)}',
+                log_type='error',
+            )
+    return {'status': 'completed', 'refreshed': str(refreshed), 'errors': str(len(errors))}
+
+
+def _maybe_discover(
+    tracks: List[Any],
+    source: str,
+    deps: AutomationDeps,
+) -> List[Any]:
+    """Run the adapter's ``discover_tracks`` when any track needs it.
+
+    Most sources are no-ops here (their tracks have provider IDs
+    already). LB / Last.fm are the ones that actually do work."""
+    if not tracks:
+        return tracks
+    if not any(getattr(t, "needs_discovery", False) for t in tracks):
+        return tracks
+    registry = deps.playlist_source_registry
+    if registry is None:
+        return tracks
+    adapter = registry.get_source(source)
+    if adapter is None:
+        return tracks
+    try:
+        return adapter.discover_tracks(tracks)
+    except Exception as exc:
+        deps.logger.warning(f"{source} discover_tracks failed: {exc}")
+        return tracks
+
+
+class _SkipPlaylist(Exception):
+    """Internal sentinel: source-specific soft-skip (e.g. not authed).
+
+    The per-playlist loop catches it specifically so the skip isn't
+    counted in the error tally — matches the pre-extraction behavior
+    where ``continue`` was used inline."""
+
+
+def _fetch_detail(
+    source: str,
+    source_id: str,
+    pl: Dict[str, Any],
+    deps: AutomationDeps,
+    auto_id: Optional[str],
+) -> Optional[PlaylistDetail]:
+    """Resolve the playlist's tracks through the registry.
+
+    Handler-level branches (URL extraction, Spotify-public→authed
+    fallback, Tidal not-authed skip) live here; everything else
+    delegates to the adapter."""
+    registry = deps.playlist_source_registry
+    if registry is None:
+        return None
+
+    # URL-backed sources: pull the upstream URL out of `description`.
+    playlist_input = source_id
+    if source in _URL_BACKED_SOURCES:
+        # ``require_refresh_url`` raises ValueError on missing URL.
+        # The outer try/except in the loop catches it and reports as
+        # an error — matching the pre-extraction behavior.
+        playlist_input = require_refresh_url(
+            source, pl.get('description', ''), pl.get('name', '')
+        )
+
+    # Spotify-public refresh: prefer the authenticated Spotify API
+    # when the user is signed in. Better album art, matches the
+    # pre-extraction handler. Falls through to the public scraper on
+    # auth failure or non-playlist URL types (e.g. album URLs).
+    if source == SOURCE_SPOTIFY_PUBLIC:
+        detail = _try_spotify_authed_for_public(playlist_input, deps)
+        if detail is not None:
+            return detail
+
+    # Tidal not-authed: soft-skip with a 'skip' log line, not an error.
+    if source == SOURCE_TIDAL:
+        tidal_source = registry.get_source(SOURCE_TIDAL)
+        if tidal_source is None or not tidal_source.is_authenticated():
+            deps.logger.warning(
+                f"Tidal not authenticated — skipping refresh for '{pl.get('name', '')}'"
+            )
+            deps.update_progress(
+                auto_id,
+                log_line=f'Skipped "{pl.get("name", "")}" — Tidal not authenticated',
+                log_type='skip',
+            )
+            raise _SkipPlaylist
+
+    adapter = registry.get_source(source)
+    if adapter is None:
+        return None
+    try:
+        return adapter.refresh_playlist(playlist_input)
+    except Exception as exc:
+        deps.logger.warning(
+            f"{source} playlist refresh failed for {playlist_input}: {exc}"
+        )
+        return None
+
+
+def _try_spotify_authed_for_public(
+    spotify_url: str, deps: AutomationDeps
+) -> Optional[PlaylistDetail]:
+    """Best-effort: use the authenticated Spotify adapter on a public URL.
+
+    Returns ``None`` to signal "fall through to the public-scraper
+    adapter" — never raises. Only applies to ``playlist``-type URLs;
+    album URLs fall through unconditionally."""
+    if not spotify_url:
+        return None
+    spotify_client = deps.spotify_client
+    if spotify_client is None or not spotify_client.is_spotify_authenticated():
+        return None
+    try:
+        from core.spotify_public_scraper import parse_spotify_url
+        parsed = parse_spotify_url(spotify_url)
+    except Exception:
+        return None
+    if not parsed or parsed.get('type') != 'playlist':
+        return None
+    adapter = deps.playlist_source_registry.get_source(SOURCE_SPOTIFY)
+    if adapter is None:
+        return None
+    try:
+        return adapter.refresh_playlist(parsed['id'])
+    except Exception as exc:
+        deps.logger.debug(f"Spotify authed fallback for public mirror failed: {exc}")
+        return None
+
+
+def _commit_refresh(
+    pl: Dict[str, Any],
+    source: str,
+    source_id: str,
+    tracks: List[Dict[str, Any]],
+    db: Any,
+    deps: AutomationDeps,
+    auto_id: Optional[str],
+) -> int:
+    """Persist the refreshed track list + emit playlist_changed when delta.
+
+    Returns 1 when a refresh successfully landed, 0 otherwise. The
+    caller is responsible for incrementing the running tally."""
+    old_tracks = db.get_mirrored_playlist_tracks(pl['id']) if pl.get('id') else []
+    old_ids = {t.get('source_track_id') for t in old_tracks if t.get('source_track_id')}
+    new_ids = {t.get('source_track_id') for t in tracks if t.get('source_track_id')}
+
+    # Preserve existing extra_data (matched_data + discovery state)
+    # for tracks that still exist in the refreshed snapshot, unless
+    # the adapter already provided fresh extra_data for that track.
+    old_extra_map = (
+        db.get_mirrored_tracks_extra_data_map(pl['id']) if pl.get('id') else {}
+    )
+    for t in tracks:
+        sid = t.get('source_track_id', '')
+        if sid and sid in old_extra_map and 'extra_data' not in t:
+            t['extra_data'] = old_extra_map[sid]
+
+    db.mirror_playlist(
+        source=source,
+        source_playlist_id=source_id,
+        name=pl['name'],
+        tracks=tracks,
+        profile_id=pl.get('profile_id', 1),
+        description=pl.get('description'),
+        owner=pl.get('owner'),
+        image_url=pl.get('image_url'),
+    )
+
+    # Membership just changed — if this playlist is organize-by-playlist, rebuild
+    # its folder (with prune) so a track that LEFT the playlist has its symlink
+    # cleaned up now. Gated to organized playlists, non-fatal — never disturbs
+    # the refresh. (Additions are handled by the post-download reconcile.)
+    try:
+        from core.playlists.materialize_service import rebuild_mirrored_playlist_if_organized
+        rebuild_mirrored_playlist_if_organized(
+            db, deps.config_manager, pl.get('id'), profile_id=pl.get('profile_id', 1)
+        )
+    except Exception as _mat_err:
+        deps.logger.debug(f"[Playlist Folder] mirror-refresh cleanup skipped: {_mat_err}")
+
+    if old_ids != new_ids:
+        added = len(new_ids - old_ids)
+        removed = len(old_ids - new_ids)
+        deps.logger.info(
+            f"[AUTOMATION] Playlist changed: '{pl.get('name', '')}' — "
+            f"{added} added, {removed} removed (old={len(old_ids)}, new={len(new_ids)})"
+        )
+        deps.update_progress(
+            auto_id,
+            log_line=f'"{pl.get("name", "")}" — {added} added, {removed} removed',
+            log_type='success',
+        )
+        try:
+            if deps.engine:
+                deps.engine.emit('playlist_changed', {
+                    'playlist_name': pl.get('name', ''),
+                    'playlist_id': str(pl.get('id', '')),
+                    'old_count': str(len(old_ids)),
+                    'new_count': str(len(new_ids)),
+                    'added': str(added),
+                    'removed': str(removed),
+                })
+        except Exception as e:
+            deps.logger.debug("playlist_synced automation emit failed: %s", e)
+    else:
+        deps.logger.warning(
+            f"[AUTOMATION] No changes: '{pl.get('name', '')}' (tracks={len(old_ids)})"
+        )
+        deps.update_progress(
+            auto_id,
+            log_line=f'No changes: "{pl.get("name", "")}"',
+            log_type='skip',
+        )
+
+    return 1

core/automation/handlers/registration.py

@@ -0,0 +1,184 @@
+"""One-stop registration of every extracted automation handler.
+
+``web_server`` builds the deps once at startup and calls
+:func:`register_all` here. Each new handler module gets one line in
+this file when it lands.
+"""
+
+from __future__ import annotations
+
+from core.automation.deps import AutomationDeps
+from core.automation.handlers.process_wishlist import auto_process_wishlist
+from core.automation.handlers.scan_watchlist import auto_scan_watchlist
+from core.automation.handlers.scan_library import auto_scan_library
+from core.automation.handlers.refresh_mirrored import auto_refresh_mirrored
+from core.automation.handlers.sync_playlist import auto_sync_playlist
+from core.automation.handlers.discover_playlist import auto_discover_playlist
+from core.automation.handlers.playlist_pipeline import auto_playlist_pipeline
+from core.automation.handlers.personalized_pipeline import auto_personalized_pipeline
+from core.automation.handlers.database_update import (
+    auto_start_database_update, auto_deep_scan_library,
+)
+from core.automation.handlers.duplicate_cleaner import auto_run_duplicate_cleaner
+from core.automation.handlers.quality_scanner import auto_start_quality_scan
+from core.automation.handlers.maintenance import (
+    auto_clear_quarantine,
+    auto_cleanup_wishlist,
+    auto_update_discovery_pool,
+    auto_backup_database,
+    auto_refresh_beatport_cache,
+)
+from core.automation.handlers.download_cleanup import (
+    auto_clean_search_history,
+    auto_clean_completed_downloads,
+    auto_full_cleanup,
+)
+from core.automation.handlers.run_script import auto_run_script
+from core.automation.handlers.search_and_download import auto_search_and_download
+from core.automation.handlers.progress_callbacks import (
+    progress_init,
+    progress_finish,
+    record_history,
+    register_library_scan_completed_emitter,
+)
+
+
+def register_all(deps: AutomationDeps) -> None:
+    """Wire every extracted handler to the engine.
+
+    Each ``register_action_handler`` call binds the action name (the
+    string the trigger uses to look up its action) to a thin lambda
+    that injects ``deps`` and forwards the engine-supplied config.
+    Guards stay alongside their handler so duplicate-run prevention
+    behaves identically to the pre-extraction code.
+    """
+    engine = deps.engine
+
+    # Self-guards prevent duplicate runs of the SAME operation, but
+    # different operations can run concurrently — wishlist downloads
+    # use bandwidth, watchlist scans use API calls, library scans use
+    # media-server CPU. Different resources, no contention.
+    engine.register_action_handler(
+        'process_wishlist',
+        lambda config: auto_process_wishlist(config, deps),
+        guard_fn=deps.is_wishlist_actually_processing,
+    )
+    engine.register_action_handler(
+        'scan_watchlist',
+        lambda config: auto_scan_watchlist(config, deps),
+        guard_fn=deps.is_watchlist_actually_scanning,
+    )
+    engine.register_action_handler(
+        'scan_library',
+        lambda config: auto_scan_library(config, deps),
+        deps.state.is_scan_library_active,
+    )
+
+    # Playlist lifecycle handlers. The pipeline composes refresh +
+    # sync + discover (it imports them directly), so all four ship
+    # together. The pipeline guard prevents an in-flight pipeline
+    # from being re-triggered mid-run.
+    engine.register_action_handler(
+        'refresh_mirrored',
+        lambda config: auto_refresh_mirrored(config, deps),
+    )
+    engine.register_action_handler(
+        'sync_playlist',
+        lambda config: auto_sync_playlist(config, deps),
+    )
+    engine.register_action_handler(
+        'discover_playlist',
+        lambda config: auto_discover_playlist(config, deps),
+    )
+    engine.register_action_handler(
+        'playlist_pipeline',
+        lambda config: auto_playlist_pipeline(config, deps),
+        deps.state.is_pipeline_running,
+    )
+    # Personalized pipeline shares the pipeline_running flag with the
+    # mirrored pipeline so the two can't overlap (single sync queue,
+    # single wishlist worker).
+    engine.register_action_handler(
+        'personalized_pipeline',
+        lambda config: auto_personalized_pipeline(config, deps),
+        deps.state.is_pipeline_running,
+    )
+
+    # Database update + deep scan share the db_update_state guard —
+    # only one operation can mutate that state at a time.
+    engine.register_action_handler(
+        'start_database_update',
+        lambda config: auto_start_database_update(config, deps),
+        lambda: deps.get_db_update_state().get('status') == 'running',
+    )
+    engine.register_action_handler(
+        'deep_scan_library',
+        lambda config: auto_deep_scan_library(config, deps),
+        lambda: deps.get_db_update_state().get('status') == 'running',
+    )
+    engine.register_action_handler(
+        'run_duplicate_cleaner',
+        lambda config: auto_run_duplicate_cleaner(config, deps),
+        lambda: deps.get_duplicate_cleaner_state().get('status') == 'running',
+    )
+    engine.register_action_handler(
+        'clear_quarantine',
+        lambda config: auto_clear_quarantine(config, deps),
+    )
+    engine.register_action_handler(
+        'cleanup_wishlist',
+        lambda config: auto_cleanup_wishlist(config, deps),
+    )
+    engine.register_action_handler(
+        'update_discovery_pool',
+        lambda config: auto_update_discovery_pool(config, deps),
+    )
+    engine.register_action_handler(
+        'start_quality_scan',
+        lambda config: auto_start_quality_scan(config, deps),
+        lambda: False,  # repair worker dedupes Run-Now requests itself
+    )
+    engine.register_action_handler(
+        'backup_database',
+        lambda config: auto_backup_database(config, deps),
+    )
+    engine.register_action_handler(
+        'refresh_beatport_cache',
+        lambda config: auto_refresh_beatport_cache(config, deps),
+    )
+    engine.register_action_handler(
+        'clean_search_history',
+        lambda config: auto_clean_search_history(config, deps),
+    )
+    engine.register_action_handler(
+        'clean_completed_downloads',
+        lambda config: auto_clean_completed_downloads(config, deps),
+    )
+    engine.register_action_handler(
+        'full_cleanup',
+        lambda config: auto_full_cleanup(config, deps),
+    )
+    engine.register_action_handler(
+        'run_script',
+        lambda config: auto_run_script(config, deps),
+    )
+    engine.register_action_handler(
+        'search_and_download',
+        lambda config: auto_search_and_download(config, deps),
+    )
+
+    # Progress + history callbacks: the engine invokes these around
+    # each handler run. Lift the closures from
+    # `web_server._register_automation_handlers` into thin lambdas
+    # that delegate into the extracted top-level functions.
+    engine.register_progress_callbacks(
+        lambda aid, name, action_type: progress_init(aid, name, action_type, deps),
+        lambda aid, result: progress_finish(aid, result, deps),
+        deps.update_progress,
+        lambda aid, result: record_history(aid, result, deps),
+    )
+
+    # `library_scan_completed` event: when the media-server scan
+    # manager finishes a scan, emit the event so any automation can
+    # trigger off it. No-op when no scan manager is configured.
+    register_library_scan_completed_emitter(deps)

core/automation/handlers/run_script.py

@@ -0,0 +1,103 @@
+"""Automation handler: ``run_script`` action.
+
+Lifted from ``web_server._register_automation_handlers`` (the
+``_auto_run_script`` closure). Runs a user-provided shell or Python
+script from the configured scripts directory with bounded timeout +
+captured stdout/stderr. Path-traversal guard ensures users can't
+escape the scripts directory.
+
+Environment variables exposed to the script:
+- ``SOULSYNC_EVENT``: triggering event type (when fired by an event)
+- ``SOULSYNC_AUTOMATION``: automation name
+- ``SOULSYNC_SCRIPTS_DIR``: absolute path to the scripts dir
+"""
+
+from __future__ import annotations
+
+import os
+import subprocess as _sp
+from typing import Any, Dict
+
+from core.automation.deps import AutomationDeps
+
+
+_MAX_TIMEOUT_SECONDS = 300  # Hard cap on user-supplied timeout config.
+
+
+def auto_run_script(config: Dict[str, Any], deps: AutomationDeps) -> Dict[str, Any]:
+    script_name = config.get('script_name', '')
+    timeout = min(int(config.get('timeout', 60)), _MAX_TIMEOUT_SECONDS)
+    automation_id = config.get('_automation_id')
+
+    if not script_name:
+        return {'status': 'error', 'error': 'No script selected'}
+
+    scripts_dir = deps.docker_resolve_path(deps.config_manager.get('scripts.path', './scripts'))
+    if not scripts_dir or not os.path.isdir(scripts_dir):
+        os.makedirs(scripts_dir, exist_ok=True)
+        return {
+            'status': 'error',
+            'error': 'Scripts directory is empty. Add scripts to the scripts/ folder.',
+        }
+
+    script_path = os.path.join(scripts_dir, script_name)
+    script_path = os.path.realpath(script_path)
+
+    # Security: block path traversal — script must resolve under
+    # the scripts dir, no symlinks/.. tricks allowed out.
+    if not script_path.startswith(os.path.realpath(scripts_dir)):
+        return {'status': 'error', 'error': 'Script path traversal blocked'}
+
+    if not os.path.isfile(script_path):
+        return {'status': 'error', 'error': f'Script not found: {script_name}'}
+
+    deps.update_progress(automation_id, phase=f'Running {script_name}...', progress=10)
+
+    # Build environment with SoulSync context.
+    env = os.environ.copy()
+    event_data = config.get('_event_data') or {}
+    env['SOULSYNC_EVENT'] = str(event_data.get('type', ''))
+    env['SOULSYNC_AUTOMATION'] = config.get('_automation_name', '')
+    env['SOULSYNC_SCRIPTS_DIR'] = scripts_dir
+
+    try:
+        # Determine how to run the script.
+        if script_path.endswith('.py'):
+            cmd = ['python', script_path]
+        elif script_path.endswith('.sh'):
+            cmd = ['bash', script_path]
+        else:
+            cmd = [script_path]
+
+        result = _sp.run(
+            cmd,
+            capture_output=True, text=True, timeout=timeout,
+            cwd=scripts_dir, env=env,
+        )
+
+        deps.update_progress(automation_id, phase='Script completed', progress=100)
+
+        stdout = result.stdout[:2000] if result.stdout else ''
+        stderr = result.stderr[:1000] if result.stderr else ''
+
+        if result.returncode == 0:
+            deps.logger.info(f"Script '{script_name}' completed (exit 0)")
+        else:
+            deps.logger.warning(f"Script '{script_name}' exited with code {result.returncode}")
+
+        return {
+            'status': 'completed' if result.returncode == 0 else 'error',
+            'exit_code': str(result.returncode),
+            'stdout': stdout,
+            'stderr': stderr,
+            'script': script_name,
+        }
+    except _sp.TimeoutExpired:
+        deps.update_progress(automation_id, phase='Script timed out', progress=100)
+        return {
+            'status': 'error',
+            'error': f'Script timed out after {timeout}s',
+            'script': script_name,
+        }
+    except Exception as e:  # noqa: BLE001 — automation handlers must never raise
+        return {'status': 'error', 'error': str(e), 'script': script_name}

core/automation/handlers/scan_library.py

@@ -0,0 +1,158 @@
+"""Automation handler: ``scan_library`` action.
+
+Lifted from ``web_server._register_automation_handlers`` (the
+``_auto_scan_library`` closure). The handler triggers a media-server
+scan via ``web_scan_manager``, then polls the manager's status until
+the scan completes (or a 30-minute timeout fires). Progress phases
+are emitted via :func:`AutomationDeps.update_progress` so the
+trigger card stays current throughout the run.
+
+The handler manages its own progress reporting (it sets
+``_manages_own_progress: True`` in the result) so the engine doesn't
+overwrite the live phase string with a generic 'completed' label.
+"""
+
+from __future__ import annotations
+
+import time
+from typing import Any, Dict
+
+from core.automation.deps import AutomationDeps
+
+
+# Outer poll cap — covers extreme worst case (long Plex scans on
+# huge libraries). Past this point we surface a clear timeout error
+# so users notice rather than letting the trigger hang forever.
+_SCAN_TIMEOUT_SECONDS = 1800
+
+# Per-phase poll intervals.
+_POLL_SCHEDULED_SECONDS = 2
+_POLL_SCANNING_SECONDS = 5
+_POLL_UNKNOWN_SECONDS = 2
+
+# Progress percentage waypoints.
+_PROGRESS_SCHEDULED_MAX = 14
+_PROGRESS_SCAN_START = 15
+_PROGRESS_SCAN_MAX = 95
+
+
+def auto_scan_library(config: Dict[str, Any], deps: AutomationDeps) -> Dict[str, Any]:
+    """Run a media-server library scan and stream progress to the
+    trigger card.
+
+    Returns one of:
+      - ``{'status': 'completed', '_manages_own_progress': True, ...}``
+      - ``{'status': 'skipped', 'reason': 'Scan already being tracked'}``
+      - ``{'status': 'error', 'reason': '...', '_manages_own_progress': True}``
+    """
+    automation_id = config.get('_automation_id')
+
+    if not deps.web_scan_manager:
+        return {'status': 'error', 'reason': 'Scan manager not available'}
+
+    # If another automation is already tracking the scan, just forward
+    # the request — the original tracker keeps emitting progress.
+    if deps.state.is_scan_library_active():
+        deps.web_scan_manager.request_scan('Automation trigger (additional batch)')
+        return {'status': 'skipped', 'reason': 'Scan already being tracked'}
+
+    deps.state.set_scan_library_id(automation_id)
+
+    try:
+        result = deps.web_scan_manager.request_scan('Automation trigger')
+        scan_status_val = result.get('status', 'unknown')
+
+        if scan_status_val == 'queued':
+            deps.update_progress(
+                automation_id,
+                log_line='Scan already in progress — waiting for completion',
+                log_type='info',
+            )
+        else:
+            delay = result.get('delay_seconds', 60)
+            deps.update_progress(
+                automation_id,
+                log_line=f'Scan scheduled (debounce: {delay}s)',
+                log_type='info',
+            )
+
+        # Unified polling loop — handles debounce → scanning → idle.
+        poll_start = time.time()
+        scan_started = (scan_status_val == 'queued')
+        while time.time() - poll_start < _SCAN_TIMEOUT_SECONDS:
+            status = deps.web_scan_manager.get_scan_status()
+            st = status.get('status')
+
+            if st == 'idle':
+                break  # Scan completed (or finished before we polled)
+
+            if st == 'scheduled':
+                elapsed = int(time.time() - poll_start)
+                deps.update_progress(
+                    automation_id,
+                    phase=f'Waiting for scan to start... ({elapsed}s)',
+                    progress=min(int(elapsed / 60 * 10), _PROGRESS_SCHEDULED_MAX),
+                )
+                time.sleep(_POLL_SCHEDULED_SECONDS)
+                continue
+
+            if st == 'scanning':
+                if not scan_started:
+                    scan_started = True
+                    deps.update_progress(
+                        automation_id,
+                        progress=_PROGRESS_SCAN_START,
+                        log_line='Scan triggered on media server',
+                        log_type='success',
+                    )
+                elapsed = status.get('elapsed_seconds', 0)
+                max_time = status.get('max_time_seconds', 300)
+                pct = min(_PROGRESS_SCAN_START + int(elapsed / max_time * 80), _PROGRESS_SCAN_MAX)
+                mins, secs = divmod(elapsed, 60)
+                deps.update_progress(
+                    automation_id,
+                    phase=f'Library scan in progress... ({mins}m {secs}s)',
+                    progress=pct,
+                )
+                time.sleep(_POLL_SCANNING_SECONDS)
+                continue
+
+            time.sleep(_POLL_UNKNOWN_SECONDS)
+        else:
+            # 30-min timeout reached
+            deps.update_progress(
+                automation_id,
+                status='error',
+                phase='Timed out',
+                log_line='Library scan timed out after 30 minutes',
+                log_type='error',
+            )
+            return {'status': 'error', 'reason': 'Timed out', '_manages_own_progress': True}
+
+        elapsed = round(time.time() - poll_start, 1)
+        deps.update_progress(
+            automation_id,
+            status='finished',
+            progress=100,
+            phase='Complete',
+            log_line='Library scan completed',
+            log_type='success',
+        )
+        return {
+            'status': 'completed',
+            '_manages_own_progress': True,
+            'scan_duration_seconds': elapsed,
+        }
+
+    except Exception as e:  # noqa: BLE001 — automation handlers must never raise into the engine
+        deps.update_progress(
+            automation_id,
+            status='error',
+            phase='Error',
+            log_line=str(e),
+            log_type='error',
+        )
+        return {'status': 'error', 'error': str(e), '_manages_own_progress': True}
+
+    finally:
+        deps.state.set_scan_library_id(None)

core/automation/handlers/scan_watchlist.py

@@ -0,0 +1,46 @@
+"""Automation handler: ``scan_watchlist`` action.
+
+Lifted from ``web_server._register_automation_handlers`` (the
+``_auto_scan_watchlist`` closure). The watchlist scanner returns
+summary stats for the trigger card only when a fresh scan actually
+ran — detected by snapshotting ``id(state_dict)`` before/after, since
+the live processor reassigns the dict on each new scan.
+"""
+
+from __future__ import annotations
+
+from typing import Any, Dict
+
+from core.automation.deps import AutomationDeps
+
+
+def auto_scan_watchlist(config: Dict[str, Any], deps: AutomationDeps) -> Dict[str, Any]:
+    """Run a watchlist scan when the automation triggers.
+
+    Pre-scan we capture ``id(watchlist_scan_state)`` so we can tell
+    afterwards whether the worker ran (and reassigned the state dict)
+    or short-circuited (kept the same dict). Only fresh scans report
+    summary stats — repeat triggers without an intervening run return
+    a bare ``completed``.
+    """
+    try:
+        pre_state = deps.get_watchlist_scan_state()
+        pre_state_id = id(pre_state)
+        deps.process_watchlist_scan_automatically(
+            automation_id=config.get('_automation_id'),
+            profile_id=config.get('_profile_id'),
+        )
+        post_state = deps.get_watchlist_scan_state()
+        # Fresh scan = state dict was reassigned mid-run.
+        if id(post_state) != pre_state_id:
+            summary = post_state.get('summary', {}) if isinstance(post_state, dict) else {}
+            return {
+                'status': 'completed',
+                'artists_scanned': summary.get('total_artists', 0),
+                'successful_scans': summary.get('successful_scans', 0),
+                'new_tracks_found': summary.get('new_tracks_found', 0),
+                'tracks_added_to_wishlist': summary.get('tracks_added_to_wishlist', 0),
+            }
+        return {'status': 'completed'}
+    except Exception as e:  # noqa: BLE001 — automation handlers must never raise into the engine
+        return {'status': 'error', 'error': str(e)}

core/automation/handlers/search_and_download.py

@@ -0,0 +1,57 @@
+"""Automation handler: ``search_and_download`` action.
+
+Lifted from ``web_server._register_automation_handlers`` (the
+``_auto_search_and_download`` closure). Searches for a track by
+name/artist string and dispatches the best match through the
+download orchestrator. Query can come from the trigger config
+(direct value) or from event data (e.g. webhook payload).
+"""
+
+from __future__ import annotations
+
+from typing import Any, Dict
+
+from core.automation.deps import AutomationDeps
+
+
+def auto_search_and_download(config: Dict[str, Any], deps: AutomationDeps) -> Dict[str, Any]:
+    automation_id = config.get('_automation_id')
+    query = config.get('query', '').strip()
+    # Event-triggered: pull query from event data (e.g. webhook_received).
+    if not query:
+        event_data = config.get('_event_data', {})
+        query = (event_data.get('query', '') or '').strip()
+    if not query:
+        if automation_id:
+            deps.update_progress(
+                automation_id, log_line='No search query provided', log_type='error',
+            )
+        return {'status': 'error', 'error': 'No search query provided'}
+    try:
+        if automation_id:
+            deps.update_progress(
+                automation_id, phase='Searching',
+                log_line=f'Searching: {query}', log_type='info',
+            )
+        result = deps.run_async(deps.download_orchestrator.search_and_download_best(query))
+        if result:
+            if automation_id:
+                deps.update_progress(
+                    automation_id,
+                    log_line=f'Download started for: {query}',
+                    log_type='success',
+                )
+            return {'status': 'completed', 'query': query, 'download_id': result}
+        if automation_id:
+            deps.update_progress(
+                automation_id,
+                log_line=f'No match found for: {query}',
+                log_type='warning',
+            )
+        return {'status': 'not_found', 'query': query, 'error': 'No match found'}
+    except Exception as e:  # noqa: BLE001 — automation handlers must never raise
+        if automation_id:
+            deps.update_progress(
+                automation_id, log_line=f'Error: {e}', log_type='error',
+            )
+        return {'status': 'error', 'query': query, 'error': str(e)}

core/automation/handlers/sync_playlist.py

@@ -0,0 +1,240 @@
+"""Automation handler: ``sync_playlist`` action.
+
+Lifted from ``web_server._register_automation_handlers`` (the
+``_auto_sync_playlist`` closure). Syncs a mirrored playlist to the
+configured media server, using discovered metadata when available
+and skipping undiscovered tracks. When triggered on a schedule with
+no track changes since the last sync, short-circuits with
+``status: skipped`` (saves Plex / Jellyfin / Navidrome from
+needless rewrites)."""
+
+from __future__ import annotations
+
+import hashlib
+import json
+import threading
+from typing import Any, Dict
+
+from core.automation.deps import AutomationDeps
+
+
+def auto_sync_playlist(config: Dict[str, Any], deps: AutomationDeps) -> Dict[str, Any]:
+    """Sync a mirrored playlist to the active media server.
+
+    Behavior:
+    - Tracks with discovered metadata (extra_data.discovered + matched_data)
+      are routed via the official metadata.
+    - Tracks with a Spotify hint (real Spotify ID from the embed
+      scraper) are included so they can still hit Soulseek + the
+      wishlist.
+    - Tracks with neither are counted as ``skipped_tracks``.
+    - Empty result → ``status: skipped`` with the skipped count.
+    - Same track set as last sync (matched_tracks unchanged) →
+      ``status: skipped`` (no-op).
+    - Otherwise spawns a daemon thread running ``run_sync_task`` and
+      returns ``status: started`` with ``_manages_own_progress: True``.
+    """
+    auto_id = config.get('_automation_id')
+    playlist_id = config.get('playlist_id')
+    if not playlist_id:
+        return {'status': 'error', 'reason': 'No playlist specified'}
+
+    db = deps.get_database()
+    pl = db.get_mirrored_playlist(int(playlist_id))
+    if not pl:
+        return {'status': 'error', 'reason': 'Playlist not found'}
+
+    tracks = db.get_mirrored_playlist_tracks(int(playlist_id))
+    if not tracks:
+        return {'status': 'error', 'reason': 'No tracks in playlist'}
+
+    # Convert mirrored tracks to format expected by run_sync_task.
+    # Use discovered metadata when available, fall back to Spotify
+    # hint or raw playlist fields when not.
+    tracks_json = []
+    skipped_count = 0
+
+    for t in tracks:
+        # Parse extra_data for discovery info.
+        extra = {}
+        if t.get('extra_data'):
+            try:
+                extra = json.loads(t['extra_data']) if isinstance(t['extra_data'], str) else t['extra_data']
+            except (json.JSONDecodeError, TypeError):
+                pass
+
+        if extra.get('discovered') and extra.get('matched_data'):
+            # Use official discovered metadata.
+            md = extra['matched_data']
+            album_raw = md.get('album', '')
+            album_obj = album_raw if isinstance(album_raw, dict) else {'name': album_raw or ''}
+            _track_entry = {
+                'name': md.get('name', ''),
+                'artists': md.get('artists', [{'name': t.get('artist_name', '')}]),
+                'album': album_obj,
+                'duration_ms': md.get('duration_ms', 0),
+                'id': md.get('id', ''),
+            }
+            if md.get('track_number'):
+                _track_entry['track_number'] = md['track_number']
+            if md.get('disc_number'):
+                _track_entry['disc_number'] = md['disc_number']
+            tracks_json.append(_track_entry)
+        else:
+            # NOT discovered — try to include using available metadata so
+            # the track can still be searched on Soulseek and added to
+            # wishlist. Without this, failed discovery blocks the entire
+            # download pipeline.
+            #
+            # Priority: spotify_hint (has real Spotify ID from embed
+            # scraper) > raw playlist fields (only if source_track_id
+            # is valid).
+            hint = extra.get('spotify_hint', {})
+            # Build album object with cover art from the mirrored playlist track.
+            track_image = (t.get('image_url') or '').strip()
+            album_obj = {
+                'name': (t.get('album_name') or '').strip(),
+                'images': [{'url': track_image, 'height': 300, 'width': 300}] if track_image else [],
+            }
+
+            if hint.get('id') and hint.get('name'):
+                # spotify_hint has proper Spotify track ID + metadata from embed scraper.
+                hint_artists = hint.get('artists', [])
+                if hint_artists and isinstance(hint_artists[0], str):
+                    hint_artists = [{'name': a} for a in hint_artists]
+                elif hint_artists and isinstance(hint_artists[0], dict):
+                    pass  # Already in correct format
+                else:
+                    hint_artists = [{'name': t.get('artist_name', '')}]
+                tracks_json.append({
+                    'name': hint['name'],
+                    'artists': hint_artists,
+                    'album': album_obj,
+                    'duration_ms': t.get('duration_ms', 0),
+                    'id': hint['id'],
+                })
+            elif t.get('source_track_id') and (t.get('track_name') or '').strip():
+                # Has a valid source ID and track name — usable for wishlist.
+                tracks_json.append({
+                    'name': t['track_name'].strip(),
+                    'artists': [{'name': (t.get('artist_name') or '').strip() or 'Unknown Artist'}],
+                    'album': album_obj,
+                    'duration_ms': t.get('duration_ms', 0),
+                    'id': t['source_track_id'],
+                })
+            else:
+                skipped_count += 1  # No usable ID or name — truly can't process.
+
+    if not tracks_json:
+        deps.update_progress(
+            auto_id,
+            log_line=f'No discovered tracks — {skipped_count} need discovery first',
+            log_type='skip',
+        )
+        return {
+            'status': 'skipped',
+            'reason': f'No discovered tracks to sync ({skipped_count} tracks need discovery first)',
+            'skipped_tracks': str(skipped_count),
+        }
+
+    # Preflight: hash the track list and compare against last sync.
+    # Skip if the exact same set of tracks was already synced and
+    # everything matched (no-op preserves Plex / Jellyfin / Navidrome
+    # from needless rewrites).
+    track_ids_str = ','.join(sorted(t.get('id', '') for t in tracks_json))
+    tracks_hash = hashlib.md5(track_ids_str.encode()).hexdigest()
+
+    sync_id_key = f"auto_mirror_{playlist_id}"
+    # Full mirror identity (every source_track_id on the playlist). tracks_hash
+    # only covers tracks_json — if a new mirror row is skipped (no discovery /
+    # no source id), tracks_hash stays identical to the pre-add sync and we
+    # used to no-op with "unchanged" while the new song never hit wishlist.
+    mirror_ids_str = ','.join(
+        sorted(t.get('source_track_id', '') or '' for t in tracks if t.get('source_track_id'))
+    )
+    mirror_tracks_hash = hashlib.md5(mirror_ids_str.encode()).hexdigest() if mirror_ids_str else ''
+
+    event_data = config.get('_event_data') or {}
+    try:
+        tracks_added = int(event_data.get('added') or 0)
+    except (TypeError, ValueError):
+        tracks_added = 0
+    force_sync = tracks_added > 0 or skipped_count > 0
+
+    try:
+        sync_statuses = deps.load_sync_status_file()
+        last_status = sync_statuses.get(sync_id_key, {})
+        last_hash = last_status.get('tracks_hash', '')
+        last_mirror_hash = last_status.get('mirror_tracks_hash', '')
+        last_matched = last_status.get('matched_tracks', -1)
+
+        mirror_changed = bool(mirror_tracks_hash) and mirror_tracks_hash != last_mirror_hash
+        if (
+            not force_sync
+            and not mirror_changed
+            and last_hash == tracks_hash
+            and last_matched >= len(tracks_json)
+        ):
+            # Exact same tracks, all matched last time — nothing to do.
+            deps.update_progress(
+                auto_id,
+                log_line=f'All {len(tracks_json)} tracks unchanged since last sync — skipping',
+                log_type='skip',
+            )
+            return {
+                'status': 'skipped',
+                'reason': f'All {len(tracks_json)} tracks unchanged since last sync',
+            }
+        if force_sync and last_hash == tracks_hash and last_matched >= len(tracks_json):
+            deps.update_progress(
+                auto_id,
+                log_line=(
+                    f'Forcing sync: playlist changed ({tracks_added} added) or '
+                    f'{skipped_count} track(s) need discovery'
+                ),
+                log_type='info',
+            )
+        elif mirror_changed:
+            deps.update_progress(
+                auto_id,
+                log_line='Mirror track list changed — running sync',
+                log_type='info',
+            )
+    except Exception as e:
+        deps.logger.debug("mirror sync last-status read: %s", e)
+
+    # Sync under the user's custom alias when set, else the upstream name (#865
+    # follow-up). The server-side playlist is named with this.
+    from core.playlists.naming import effective_mirrored_name
+    sync_name = effective_mirrored_name(pl) or pl.get('name') or 'Playlist'
+
+    deps.update_progress(
+        auto_id,
+        progress=50,
+        phase=f'Syncing "{sync_name}"',
+        log_line=f'{len(tracks_json)} discovered, {skipped_count} skipped',
+        log_type='info',
+    )
+
+    sync_id = f"auto_mirror_{playlist_id}"
+    deps.update_progress(
+        auto_id,
+        progress=90,
+        log_line=f'Starting sync: {len(tracks_json)} tracks',
+        log_type='success',
+    )
+    skip_wishlist_add = bool(pl.get('organize_by_playlist'))
+    threading.Thread(
+        target=deps.run_sync_task,
+        args=(sync_id, sync_name, tracks_json, auto_id, 1, pl.get('image_url', '')),
+        kwargs={'skip_wishlist_add': skip_wishlist_add},
+        daemon=True,
+        name=f'auto-sync-{playlist_id}',
+    ).start()
+    return {
+        'status': 'started',
+        'playlist_name': sync_name,
+        'discovered_tracks': str(len(tracks_json)),
+        'skipped_tracks': str(skipped_count),
+        '_manages_own_progress': True,
+    }

core/automation/progress.py

@@ -77,8 +77,8 @@ def update_progress(
             if socketio_emit is not None:
                 try:
                     socketio_emit('automation:progress', {str(automation_id): dict(state)})
-                except Exception:
-                    pass
+                except Exception as e:
+                    logger.debug("socketio progress emit: %s", e)
 
 
 def get_running_progress() -> dict[str, dict]:
@@ -121,8 +121,8 @@ def record_history(
                 t0 = datetime.fromisoformat(started_at)
                 t1 = datetime.fromisoformat(finished_at)
                 duration = (t1 - t0).total_seconds()
-            except Exception:
-                pass
+            except Exception as e:
+                logger.debug("duration parse: %s", e)
 
         r_status = result.get('status', 'completed') if result else 'completed'
         if r_status == 'error':

core/automation/schedule.py

@@ -0,0 +1,321 @@
+"""Pure functions for computing the next-run datetime of a scheduled
+automation trigger.
+
+The Auto-Sync schedule board currently exposes interval-based scheduling
+(``every N hours``) backed by ``trigger_type='schedule'``. The
+automation engine ALSO supports ``daily_time`` and ``weekly_time``
+triggers via separate ``_setup_*_trigger`` methods inline on the engine
+class. None of that logic is currently testable in isolation — the
+engine's ``_finish_run`` reaches for ``datetime.now()``, threads it
+through ``_next_weekly_occurrence``, and writes the result to the DB,
+all on the same call.
+
+This module lifts the "given a trigger config, what's the next run?"
+question out of the engine into a pure function:
+
+    next_run_at(trigger_type, trigger_config, now_utc, default_tz)
+        -> Optional[datetime]
+
+That means:
+- ``now_utc`` is INJECTED, not pulled from the system clock. Tests
+  freeze time without monkeypatching ``datetime.now``.
+- ``default_tz`` is INJECTED. Daily / weekly / monthly schedules are
+  inherently in the USER'S timezone (cron "every Monday at 9am" is
+  not UTC), and the historic engine implicitly used the server's
+  local tz via naive ``datetime.now()``. That broke for users on a
+  different tz than their server. The pure function takes the tz
+  explicitly so the caller controls it.
+- Returns an aware UTC ``datetime`` ready to serialise to the DB's
+  ``next_run`` string column, or ``None`` for unrecognised /
+  event-based triggers (engine should not store a next_run for those).
+
+PR 1 of the schedule-types feature ships ONLY this module + tests.
+The engine continues to compute next_run via its existing inline
+helpers; PR 2 collapses those into a single ``next_run_at`` call.
+Net behavior is identical until the engine is wired through — this
+PR is pure plumbing.
+
+Schedule types supported here:
+
+- ``schedule`` (interval): ``{interval: N, unit: 'minutes'|'hours'|'days'}``
+  — adds the interval to ``now_utc``; no tz needed.
+- ``daily_time``: ``{time: 'HH:MM', tz: '<IANA>'}`` — runs every day at
+  the given local time in the given timezone. ``tz`` falls back to
+  ``default_tz`` when absent.
+- ``weekly_time``: ``{time: 'HH:MM', days: ['mon','wed',...], tz: '<IANA>'}``
+  — runs on the matching weekday(s) at the given local time. Empty
+  ``days`` list means "every day" (matches the engine's existing
+  fallback in ``_next_weekly_occurrence``).
+- ``monthly_time``: ``{time: 'HH:MM', day_of_month: 1-31, tz: '<IANA>'}``
+  — runs on the given day each month. Days that don't exist in a
+  given month (Feb 30, Apr 31) clamp to the LAST valid day of that
+  month rather than skipping the run entirely; missing a whole
+  month silently because the schedule was over-eager is worse than
+  running a day early.
+"""
+
+from __future__ import annotations
+
+from datetime import datetime, timedelta, timezone
+from typing import Any, Dict, Optional
+from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
+
+from utils.logging_config import get_logger
+
+logger = get_logger("automation.schedule")
+
+
+# Unknown-tz names already warned about in this process — avoids
+# spamming the log on every poll cycle for the same misconfigured row.
+_UNKNOWN_TZ_WARNED: set = set()
+
+
+# Weekday abbreviation → ``datetime.weekday()`` index (Mon=0..Sun=6).
+# Mirrors the engine's existing ``_next_weekly_occurrence`` mapping so
+# schedules created against either implementation accept the same
+# ``days`` strings.
+_WEEKDAY_MAP = {
+    'mon': 0, 'tue': 1, 'wed': 2, 'thu': 3, 'fri': 4, 'sat': 5, 'sun': 6,
+}
+
+# Interval multipliers — kept aligned with the engine's existing
+# ``_calc_delay_seconds`` in ``core/automation_engine.py``. Adding
+# entries here without also updating the engine would silently drift:
+# this function would honour the new unit while the live engine path
+# defaults it to hours. Keep the maps in sync until PR 2 collapses the
+# engine through this function.
+_INTERVAL_MULTIPLIERS = {
+    'minutes': 60,
+    'hours':   60 * 60,
+    'days':    60 * 60 * 24,
+}
+
+
+def next_run_at(
+    trigger_type: str,
+    trigger_config: Dict[str, Any],
+    now_utc: datetime,
+    default_tz: str = 'UTC',
+) -> Optional[datetime]:
+    """Compute the next-run timestamp (UTC, aware) for a scheduled
+    trigger. Returns ``None`` for unrecognised types or event-based
+    triggers — callers should not write a next_run for those.
+
+    See module docstring for supported trigger types + config shapes.
+    """
+    if not isinstance(trigger_config, dict):
+        trigger_config = {}
+
+    if trigger_type == 'schedule':
+        return _next_interval(trigger_config, now_utc)
+    if trigger_type == 'daily_time':
+        return _next_daily(trigger_config, now_utc, default_tz)
+    if trigger_type == 'weekly_time':
+        return _next_weekly(trigger_config, now_utc, default_tz)
+    if trigger_type == 'monthly_time':
+        return _next_monthly(trigger_config, now_utc, default_tz)
+    return None
+
+
+# ---------------------------------------------------------------------------
+# Interval
+# ---------------------------------------------------------------------------
+
+
+def _next_interval(config: Dict[str, Any], now_utc: datetime) -> datetime:
+    """``{interval: N, unit: 'hours'}`` → ``now_utc + N hours``.
+
+    Mirrors the engine's existing ``_calc_delay_seconds``. Unit defaults
+    to ``hours`` for backward compat with legacy DB rows that pre-date
+    the unit field being mandatory; interval defaults to 1 so a fully
+    empty config doesn't divide-by-zero or schedule for the past."""
+    try:
+        interval = max(int(config.get('interval', 1)), 1)
+    except (TypeError, ValueError):
+        interval = 1
+    unit = config.get('unit') or 'hours'
+    seconds = interval * _INTERVAL_MULTIPLIERS.get(unit, _INTERVAL_MULTIPLIERS['hours'])
+    return _ensure_utc(now_utc) + timedelta(seconds=seconds)
+
+
+# ---------------------------------------------------------------------------
+# Daily
+# ---------------------------------------------------------------------------
+
+
+def _next_daily(
+    config: Dict[str, Any], now_utc: datetime, default_tz: str,
+) -> datetime:
+    """``{time: 'HH:MM', tz: '<IANA>'}`` → next occurrence of that
+    wall-clock time in the user's timezone, expressed as aware UTC.
+
+    DST-aware via ``zoneinfo``: when the local time falls during a
+    spring-forward gap, the ``replace`` lands on a non-existent
+    instant; ``zoneinfo`` resolves that to the gap's later side
+    (e.g. 02:30 on the DST-forward day becomes 03:30 local). Tests
+    pin both spring-forward and fall-back behaviour."""
+    tz = _resolve_tz(config.get('tz') or default_tz)
+    hour, minute = _parse_hhmm(config.get('time'))
+    now_local = _ensure_utc(now_utc).astimezone(tz)
+    target_local = now_local.replace(hour=hour, minute=minute, second=0, microsecond=0)
+    if target_local <= now_local:
+        target_local = target_local + timedelta(days=1)
+    return target_local.astimezone(timezone.utc)
+
+
+# ---------------------------------------------------------------------------
+# Weekly
+# ---------------------------------------------------------------------------
+
+
+def _next_weekly(
+    config: Dict[str, Any], now_utc: datetime, default_tz: str,
+) -> datetime:
+    """``{time: 'HH:MM', days: ['mon',...], tz: '<IANA>'}`` → next
+    occurrence of that wall-clock time on any of the listed weekdays
+    in the user's timezone.
+
+    Empty ``days`` list ≡ every day, matching the engine's existing
+    fallback. Unrecognised day abbreviations are silently dropped
+    (an empty result-set then triggers the every-day fallback)."""
+    tz = _resolve_tz(config.get('tz') or default_tz)
+    hour, minute = _parse_hhmm(config.get('time'))
+    days = _parse_weekdays(config.get('days'))
+
+    now_local = _ensure_utc(now_utc).astimezone(tz)
+    # Scan today + next 7 days; the matching day with a future
+    # local time wins. 8-day scan is enough to handle the case where
+    # today already passed the time AND today is the only allowed
+    # weekday (next occurrence is exactly one week out).
+    for offset in range(8):
+        candidate = now_local + timedelta(days=offset)
+        if candidate.weekday() not in days:
+            continue
+        target = candidate.replace(hour=hour, minute=minute, second=0, microsecond=0)
+        if target > now_local:
+            return target.astimezone(timezone.utc)
+    # Shouldn't reach: 8-day scan always finds a hit when ``days``
+    # is non-empty. Defensive fallback: next week, same weekday as today.
+    fallback = (now_local + timedelta(days=7)).replace(
+        hour=hour, minute=minute, second=0, microsecond=0,
+    )
+    return fallback.astimezone(timezone.utc)
+
+
+# ---------------------------------------------------------------------------
+# Monthly
+# ---------------------------------------------------------------------------
+
+
+def _next_monthly(
+    config: Dict[str, Any], now_utc: datetime, default_tz: str,
+) -> datetime:
+    """``{time: 'HH:MM', day_of_month: 1-31, tz: '<IANA>'}`` → next
+    occurrence in the user's timezone.
+
+    ``day_of_month`` is clamped to ``[1, 31]``. When the target day
+    doesn't exist in a given month (Feb 30, Apr 31), the schedule
+    falls back to the LAST valid day of that month — running a day
+    or two early in short months is less surprising than skipping
+    a month entirely. This matches the convention every cron
+    implementation in the wild settled on."""
+    tz = _resolve_tz(config.get('tz') or default_tz)
+    hour, minute = _parse_hhmm(config.get('time'))
+    raw_day = config.get('day_of_month', 1)
+    try:
+        target_day = max(1, min(31, int(raw_day)))
+    except (TypeError, ValueError):
+        target_day = 1
+
+    now_local = _ensure_utc(now_utc).astimezone(tz)
+    # Try this month first; if the target day has already passed
+    # (or doesn't exist this month and the clamped day is in the
+    # past), advance to next month. Loop bounded to 12 iterations
+    # so a pathologically broken config can't infinite-loop us.
+    year, month = now_local.year, now_local.month
+    for _ in range(12):
+        day = min(target_day, _days_in_month(year, month))
+        target = now_local.replace(
+            year=year, month=month, day=day,
+            hour=hour, minute=minute, second=0, microsecond=0,
+        )
+        if target > now_local:
+            return target.astimezone(timezone.utc)
+        # Roll to next month.
+        if month == 12:
+            year, month = year + 1, 1
+        else:
+            month += 1
+    # Defensive — should be unreachable.
+    return (now_local + timedelta(days=30)).astimezone(timezone.utc)
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+
+def _ensure_utc(dt: datetime) -> datetime:
+    """Coerce a possibly-naive datetime to aware UTC. Naive inputs
+    are assumed UTC (matches the convention the engine uses when
+    parsing the DB ``next_run`` column)."""
+    if dt.tzinfo is None:
+        return dt.replace(tzinfo=timezone.utc)
+    return dt.astimezone(timezone.utc)
+
+
+def _resolve_tz(name: Optional[str]):
+    """Look up an IANA tz by name. Falls back to UTC when the name is
+    unknown — ``ZoneInfoNotFoundError`` is the symptom of either a
+    typo in the tz string or ``tzdata`` missing on the host. Logged
+    once per unknown name so the user can see WHY their schedule
+    isn't running in the timezone they configured."""
+    if not name:
+        return timezone.utc
+    try:
+        return ZoneInfo(name)
+    except ZoneInfoNotFoundError:
+        if name not in _UNKNOWN_TZ_WARNED:
+            _UNKNOWN_TZ_WARNED.add(name)
+            logger.warning(
+                "Unknown timezone %r — schedule will run against UTC. "
+                "Check the spelling (IANA format like 'America/Los_Angeles') "
+                "or install the `tzdata` package on minimal hosts.",
+                name,
+            )
+        return timezone.utc
+
+
+def _parse_hhmm(time_str: Optional[str]) -> tuple:
+    """Parse ``HH:MM`` → ``(hour, minute)``. Defaults to 00:00 on
+    garbage input — same defensive shape as the engine's existing
+    daily/weekly time parsing."""
+    if not isinstance(time_str, str):
+        return 0, 0
+    try:
+        h, m = time_str.split(':', 1)
+        return max(0, min(23, int(h))), max(0, min(59, int(m)))
+    except (ValueError, AttributeError):
+        return 0, 0
+
+
+def _parse_weekdays(days) -> set:
+    """``['mon', 'wed']`` → ``{0, 2}``. Empty / missing / all-invalid
+    list returns ``set(range(7))`` ("every day"), matching the
+    engine's existing ``_next_weekly_occurrence`` fallback."""
+    if not isinstance(days, (list, tuple)):
+        return set(range(7))
+    parsed = {_WEEKDAY_MAP[d.lower()] for d in days
+              if isinstance(d, str) and d.lower() in _WEEKDAY_MAP}
+    return parsed or set(range(7))
+
+
+def _days_in_month(year: int, month: int) -> int:
+    """Last calendar day of ``year-month``. Stdlib-only — no calendar
+    module import needed; cycle through the 12 months."""
+    if month == 12:
+        next_first = datetime(year + 1, 1, 1)
+    else:
+        next_first = datetime(year, month + 1, 1)
+    last_day = next_first - timedelta(days=1)
+    return last_day.day

core/automation/signals.py

@@ -8,6 +8,9 @@ names from the saved automation set so the builder UI can autocomplete.
 from __future__ import annotations
 
 import json
+import logging
+
+logger = logging.getLogger(__name__)
 
 
 def collect_known_signals(database) -> list[str]:
@@ -38,6 +41,6 @@ def collect_known_signals(database) -> list[str]:
                             signals.add(sig)
             except (json.JSONDecodeError, TypeError):
                 pass
-    except Exception:
-        pass
+    except Exception as e:
+        logger.debug("collect known signals failed: %s", e)
     return sorted(signals)

core/automation_engine.py

@@ -18,10 +18,14 @@ import time
 import threading
 import requests
 from datetime import datetime, timedelta, timezone
+from typing import Optional
 from utils.logging_config import get_logger
 
+from core.automation.schedule import next_run_at
+
 logger = get_logger("automation_engine")
 
+
 def _utcnow():
     """Return current UTC time as timezone-aware datetime."""
     return datetime.now(timezone.utc)
@@ -34,6 +38,42 @@ def _utc_after(seconds):
     """Return UTC time N seconds from now as naive string for DB storage."""
     return (datetime.now(timezone.utc) + timedelta(seconds=seconds)).strftime('%Y-%m-%d %H:%M:%S')
 
+
+def _dt_to_db_str(dt: datetime) -> str:
+    """Convert an aware-UTC datetime to the naive-UTC string the DB
+    ``next_run`` column stores. Centralised so a tz mistake here
+    surfaces in one place, not scattered through every caller of
+    ``next_run_at``."""
+    if dt.tzinfo is None:
+        return dt.strftime('%Y-%m-%d %H:%M:%S')
+    return dt.astimezone(timezone.utc).strftime('%Y-%m-%d %H:%M:%S')
+
+
+def _resolve_system_default_tz() -> str:
+    """Return the IANA tz name the engine uses when a schedule's
+    trigger config doesn't carry an explicit ``tz`` field.
+
+    Existing daily / weekly rows pre-date the ``tz`` field — the
+    historic engine computed delays from naive ``datetime.now()``,
+    which is implicitly the server's local timezone. Falling back to
+    that same tz here preserves "every Monday at 09:00" running at
+    09:00 server local for rows that already exist in the DB.
+    Without ``tzlocal`` installed (or a system without a discoverable
+    tz), falls back to UTC."""
+    try:
+        import tzlocal
+        return tzlocal.get_localzone_name() or 'UTC'
+    except Exception:
+        return 'UTC'
+
+
+# Server-local tz cached at import time. Re-reading per-call is
+# pointless: the host's timezone doesn't change while the process is
+# running. Tests that need a different default tz inject it through
+# the engine's ``_default_tz`` attribute or via the
+# ``automation.default_timezone`` config key.
+_SYSTEM_DEFAULT_TZ = _resolve_system_default_tz()
+
 SYSTEM_AUTOMATIONS = [
     {
         'name': 'Auto-Process Wishlist',
@@ -134,11 +174,25 @@ class AutomationEngine:
         self._max_chain_depth = 5
         self._signal_cooldown_seconds = 10
 
+        # Default tz used when a schedule's ``trigger_config`` doesn't
+        # carry an explicit ``tz`` field — preserves historic behaviour
+        # for daily / weekly rows created before the field existed
+        # (engine used naive ``datetime.now()`` = server local). Reads
+        # from the ``automation.default_timezone`` config key first to
+        # let users override without touching env vars; falls back to
+        # the system-detected local tz.
+        try:
+            from config.settings import config_manager
+            self._default_tz = (config_manager.get('automation.default_timezone', '') or _SYSTEM_DEFAULT_TZ)
+        except Exception:
+            self._default_tz = _SYSTEM_DEFAULT_TZ
+
         # Trigger registry: type → setup function (schedule only — events use emit())
         self._trigger_handlers = {
             'schedule': self._setup_schedule_trigger,
             'daily_time': self._setup_daily_time_trigger,
             'weekly_time': self._setup_weekly_time_trigger,
+            'monthly_time': self._setup_monthly_time_trigger,
         }
 
     # --- Action Handler Registration ---
@@ -455,8 +509,10 @@ class AutomationEngine:
         if delay_minutes and delay_minutes > 0:
             # Initialize progress BEFORE delay so card glows during wait
             if self._progress_init_fn:
-                try: self._progress_init_fn(automation_id, auto.get('name', ''), action_type)
-                except Exception: pass
+                try:
+                    self._progress_init_fn(automation_id, auto.get('name', ''), action_type)
+                except Exception as e:
+                    logger.debug("event progress init (delay): %s", e)
                 _delay_already_inited = True
 
             delay_seconds = int(delay_minutes) * 60
@@ -486,13 +542,17 @@ class AutomationEngine:
                     logger.info(f"Event automation '{auto.get('name')}' skipped — {action_type} busy")
                     # If progress was initialized during delay, finalize it
                     if _delay_already_inited and self._progress_finish_fn:
-                        try: self._progress_finish_fn(automation_id, result)
-                        except Exception: pass
+                        try:
+                            self._progress_finish_fn(automation_id, result)
+                        except Exception as e:
+                            logger.debug("event progress finish (skipped): %s", e)
                 else:
                     # Initialize progress tracking (skip if already done during delay)
                     if not _delay_already_inited and self._progress_init_fn:
-                        try: self._progress_init_fn(automation_id, auto.get('name', ''), action_type)
-                        except Exception: pass
+                        try:
+                            self._progress_init_fn(automation_id, auto.get('name', ''), action_type)
+                        except Exception as e:
+                            logger.debug("event progress init: %s", e)
                     try:
                         result = handler_info['handler'](action_config) or {}
                         logger.info(f"Event automation '{auto.get('name')}' executed: {result.get('status', 'ok')}")
@@ -501,8 +561,10 @@ class AutomationEngine:
                         logger.error(f"Event automation '{auto.get('name')}' action failed: {e}")
                     # Finalize progress tracking
                     if self._progress_finish_fn:
-                        try: self._progress_finish_fn(automation_id, result)
-                        except Exception: pass
+                        try:
+                            self._progress_finish_fn(automation_id, result)
+                        except Exception as e:
+                            logger.debug("event progress finish: %s", e)
 
         # Merge event data into result for then-action variables
         merged = {**event_data, **result}
@@ -531,8 +593,8 @@ class AutomationEngine:
         if self._history_record_fn:
             try:
                 self._history_record_fn(automation_id, result)
-            except Exception:
-                pass
+            except Exception as e:
+                logger.debug("history record failed: %s", e)
 
     # --- Schedule Execution (timer-based) ---
 
@@ -573,6 +635,11 @@ class AutomationEngine:
         action_config['_automation_name'] = auto.get('name', '')
         if profile_id is not None:
             action_config['_profile_id'] = profile_id
+        # The profile this run acts AS: an explicit trigger profile, else the
+        # automation's owner, else admin. System + admin automations are
+        # profile 1, so this is a no-op for them — only non-admin-owned
+        # automations gain their correct identity in the background.
+        _effective_profile_id = profile_id if profile_id is not None else (auto.get('profile_id') or 1)
 
         # Action delay (skipped for manual run_now)
         delay_minutes = action_config.get('delay', 0)
@@ -580,8 +647,10 @@ class AutomationEngine:
         if not skip_delay and delay_minutes and delay_minutes > 0:
             # Initialize progress BEFORE delay so card glows during wait
             if self._progress_init_fn:
-                try: self._progress_init_fn(automation_id, auto.get('name', ''), action_type)
-                except Exception: pass
+                try:
+                    self._progress_init_fn(automation_id, auto.get('name', ''), action_type)
+                except Exception as e:
+                    logger.debug("scheduled progress init (delay): %s", e)
                 _delay_already_inited = True
 
             delay_seconds = int(delay_minutes) * 60
@@ -603,19 +672,28 @@ class AutomationEngine:
             logger.info(f"Automation '{auto['name']}' skipped — {action_type} already running")
             # If progress was initialized during delay, finalize it
             if _delay_already_inited and self._progress_finish_fn:
-                try: self._progress_finish_fn(automation_id, result)
-                except Exception: pass
-            self._finish_run(auto, automation_id, result, error=None)
+                try:
+                    self._progress_finish_fn(automation_id, result)
+                except Exception as e:
+                    logger.debug("scheduled progress finish (skipped): %s", e)
+            self._finish_run(auto, automation_id, result, error=None, retry_delay_seconds=300)
             return
 
         # Initialize progress tracking (skip if already done during delay)
         if not _delay_already_inited and self._progress_init_fn:
-            try: self._progress_init_fn(automation_id, auto.get('name', ''), action_type)
-            except Exception: pass
+            try:
+                self._progress_init_fn(automation_id, auto.get('name', ''), action_type)
+            except Exception as e:
+                logger.debug("scheduled progress init: %s", e)
 
-        # Execute the action
+        # Execute the action under the owner's profile so get_current_profile_id()
+        # (and the per-profile clients it resolves) act as the automation's owner
+        # in the background, not admin. Reset in finally so a pooled thread can't
+        # leak the override to the next job.
         error = None
         result = {}
+        from core.profile_context import set_background_profile, reset_background_profile
+        _bg_token = set_background_profile(_effective_profile_id)
         try:
             result = handler_info['handler'](action_config) or {}
             logger.info(f"Automation '{auto['name']}' (id={automation_id}) executed: {result.get('status', 'ok')}")
@@ -634,11 +712,15 @@ class AutomationEngine:
             error = str(e)
             result = {'status': 'error', 'error': error}
             logger.error(f"Automation '{auto['name']}' (id={automation_id}) failed: {e}")
+        finally:
+            reset_background_profile(_bg_token)
 
         # Finalize progress tracking
         if self._progress_finish_fn:
-            try: self._progress_finish_fn(automation_id, result)
-            except Exception: pass
+            try:
+                self._progress_finish_fn(automation_id, result)
+            except Exception as e:
+                logger.debug("scheduled progress finish: %s", e)
 
         # Execute then-actions (notifications + fire_signal)
         try:
@@ -648,7 +730,7 @@ class AutomationEngine:
 
         self._finish_run(auto, automation_id, result, error)
 
-    def _finish_run(self, auto, automation_id, result, error):
+    def _finish_run(self, auto, automation_id, result, error, retry_delay_seconds=None):
         """Update DB with run stats and reschedule."""
         next_run_str = None
         trigger_type = auto.get('trigger_type', '')
@@ -656,25 +738,25 @@ class AutomationEngine:
         if trigger_type in self._trigger_handlers:
             try:
                 trigger_config = json.loads(auto.get('trigger_config') or '{}')
-                if trigger_type == 'daily_time':
-                    # Next run is tomorrow at the configured time (compute delay from local time, store as UTC)
-                    time_str = trigger_config.get('time', '00:00')
-                    hour, minute = map(int, time_str.split(':'))
-                    now_local = datetime.now()
-                    target = now_local.replace(hour=hour, minute=minute, second=0, microsecond=0) + timedelta(days=1)
-                    next_run_str = _utc_after((target - now_local).total_seconds())
-                elif trigger_type == 'weekly_time':
-                    time_str = trigger_config.get('time', '00:00')
-                    hour, minute = map(int, time_str.split(':'))
-                    now_local = datetime.now()
-                    target = self._next_weekly_occurrence(hour, minute, trigger_config.get('days', []))
-                    next_run_str = _utc_after((target - now_local).total_seconds())
+                if retry_delay_seconds:
+                    next_run_str = _utc_after(retry_delay_seconds)
                 else:
-                    delay = self._calc_delay_seconds(trigger_config)
-                    if delay:
-                        next_run_str = _utc_after(delay)
-            except Exception:
-                pass
+                    # Single integration point with ``next_run_at``. The
+                    # helper handles every trigger type the engine
+                    # supports (interval / daily / weekly / monthly) and
+                    # returns aware-UTC; ``_dt_to_db_str`` normalises to
+                    # the naive-UTC string the DB column stores. Tests
+                    # injecting a different ``now_utc`` patch this same
+                    # path — no scattered ``datetime.now()`` calls left.
+                    next_run_dt = next_run_at(
+                        trigger_type, trigger_config,
+                        now_utc=_utcnow(),
+                        default_tz=self._default_tz,
+                    )
+                    if next_run_dt is not None:
+                        next_run_str = _dt_to_db_str(next_run_dt)
+            except Exception as e:
+                logger.debug("next run calc failed: %s", e)
 
         last_result = json.dumps(result) if result else None
         self.db.update_automation_run(automation_id, next_run=next_run_str, error=error, last_result=last_result)
@@ -682,8 +764,8 @@ class AutomationEngine:
         if self._history_record_fn:
             try:
                 self._history_record_fn(automation_id, result)
-            except Exception:
-                pass
+            except Exception as e:
+                logger.debug("history record failed: %s", e)
 
         if self._running:
             self.schedule_automation(automation_id)
@@ -746,45 +828,72 @@ class AutomationEngine:
         logger.debug(f"Scheduled automation {automation_id} in {delay:.0f}s")
 
     def _setup_daily_time_trigger(self, automation_id, config):
-        """Config: {"time": "03:00"}  — runs daily at the specified local time."""
-        time_str = config.get('time', '00:00')
-        try:
-            hour, minute = map(int, time_str.split(':'))
-        except (ValueError, AttributeError):
-            hour, minute = 0, 0
-
-        now_local = datetime.now()
-        target = now_local.replace(hour=hour, minute=minute, second=0, microsecond=0)
-        if target <= now_local:
-            target += timedelta(days=1)
-
-        delay = (target - now_local).total_seconds()
-
-        next_run_str = _utc_after(delay)
-        self.db.update_automation(automation_id, next_run=next_run_str)
-
-        timer = threading.Timer(delay, self.run_automation, args=(automation_id,))
-        timer.daemon = True
-        timer.start()
-
-        with self._lock:
-            self._timers[automation_id] = timer
-
-        logger.debug(f"Daily automation {automation_id} scheduled for {time_str} (in {delay:.0f}s)")
+        """Config: ``{"time": "03:00", "tz": "<IANA>"}`` — runs daily
+        at the specified local time. Tz defaults to ``self._default_tz``
+        when absent."""
+        self._setup_timed_trigger(automation_id, 'daily_time', config,
+                                  label=f"Daily at {config.get('time', '00:00')}")
 
     def _setup_weekly_time_trigger(self, automation_id, config):
-        """Config: {"time": "03:00", "days": ["mon", "wed", "fri"]}"""
-        time_str = config.get('time', '00:00')
-        try:
-            hour, minute = map(int, time_str.split(':'))
-        except (ValueError, AttributeError):
-            hour, minute = 0, 0
+        """Config: ``{"time": "03:00", "days": ["mon","wed","fri"], "tz": "<IANA>"}``."""
+        day_names = ', '.join(config.get('days') or []) or 'every day'
+        self._setup_timed_trigger(automation_id, 'weekly_time', config,
+                                  label=f"Weekly {config.get('time', '00:00')} on {day_names}")
 
-        target = self._next_weekly_occurrence(hour, minute, config.get('days', []))
-        delay = (target - datetime.now()).total_seconds()
+    def _setup_monthly_time_trigger(self, automation_id, config):
+        """Config: ``{"time": "09:00", "day_of_month": 15, "tz": "<IANA>"}``.
 
-        next_run_str = _utc_after(delay)
-        self.db.update_automation(automation_id, next_run=next_run_str)
+        Day clamped to [1, 31]; months too short for the target day
+        clamp to the last valid day (Feb 31 → Feb 28 / Feb 29 leap
+        year) per standard cron convention — see
+        ``core.automation.schedule._next_monthly`` for the rule."""
+        day = config.get('day_of_month', 1)
+        self._setup_timed_trigger(automation_id, 'monthly_time', config,
+                                  label=f"Monthly {config.get('time', '00:00')} on day {day}")
+
+    def _setup_timed_trigger(self, automation_id, trigger_type, config, *, label):
+        """Shared setup for daily / weekly / monthly time triggers.
+
+        All three flow through the same skeleton: compute next-run
+        via ``next_run_at``, persist to DB, arm a ``threading.Timer``
+        that fires the automation when the delay elapses. Lifting
+        these out of three near-identical methods means there's one
+        place to fix when (e.g.) timer rearm semantics need a tweak.
+
+        Honours an existing future ``next_run`` row in the DB —
+        prevents losing a hand-edited next_run when the engine
+        reschedules at startup. Same guard as the interval path."""
+        target_dt = next_run_at(
+            trigger_type, config or {},
+            now_utc=_utcnow(),
+            default_tz=self._default_tz,
+        )
+        if target_dt is None:
+            logger.warning(
+                f"Skip scheduling automation {automation_id}: next_run_at returned "
+                f"None for {trigger_type!r}",
+            )
+            return
+
+        delay = max(0.0, (target_dt - _utcnow()).total_seconds())
+
+        # If the DB already carries a future next_run, prefer it —
+        # matches the interval-path behaviour and lets manual edits
+        # or pending retries survive a process restart.
+        auto = self.db.get_automation(automation_id)
+        if auto and auto.get('next_run'):
+            try:
+                existing = datetime.strptime(
+                    auto['next_run'], '%Y-%m-%d %H:%M:%S',
+                ).replace(tzinfo=timezone.utc)
+                remaining = (existing - _utcnow()).total_seconds()
+                if remaining > 0:
+                    delay = remaining
+                    target_dt = existing
+            except (ValueError, TypeError):
+                pass
+
+        self.db.update_automation(automation_id, next_run=_dt_to_db_str(target_dt))
 
         timer = threading.Timer(delay, self.run_automation, args=(automation_id,))
         timer.daemon = True
@@ -793,25 +902,7 @@ class AutomationEngine:
         with self._lock:
             self._timers[automation_id] = timer
 
-        day_names = ', '.join(config.get('days', [])) or 'every day'
-        logger.debug(f"Weekly automation {automation_id} scheduled for {time_str} on {day_names} (in {delay:.0f}s)")
-
-    def _next_weekly_occurrence(self, hour, minute, days):
-        """Find the next datetime matching one of the given weekday abbreviations."""
-        day_map = {'mon': 0, 'tue': 1, 'wed': 2, 'thu': 3, 'fri': 4, 'sat': 5, 'sun': 6}
-        allowed = {day_map[d] for d in days if d in day_map}
-        if not allowed:
-            allowed = set(range(7))  # no days selected = every day
-
-        now = datetime.now()
-        for offset in range(8):  # check today + next 7 days
-            candidate = now + timedelta(days=offset)
-            if candidate.weekday() in allowed:
-                target = candidate.replace(hour=hour, minute=minute, second=0, microsecond=0)
-                if target > now:
-                    return target
-        # Fallback: tomorrow (shouldn't happen with 8-day scan)
-        return now.replace(hour=hour, minute=minute, second=0, microsecond=0) + timedelta(days=1)
+        logger.debug(f"{label} automation {automation_id} scheduled (in {delay:.0f}s)")
 
     # --- Then Actions (notifications + signals) ---
 
@@ -981,6 +1072,8 @@ class AutomationEngine:
         """Send message via Telegram Bot API."""
         bot_token = config.get('bot_token', '').strip()
         chat_id = config.get('chat_id', '').strip()
+        thread_id = config.get('thread_id', '').strip()
+        
         if not bot_token or not chat_id:
             raise ValueError("Bot token and chat ID are required for Telegram")
 
@@ -989,9 +1082,16 @@ class AutomationEngine:
         for key, value in variables.items():
             message = message.replace('{' + key + '}', value)
 
+        payload = {"chat_id": chat_id, "text": message, "parse_mode": "HTML"}
+        if thread_id:
+            try:
+                payload["message_thread_id"] = int(thread_id)
+            except ValueError:
+                pass  # invalid — fall back to main chat
+
         resp = requests.post(
             f'https://api.telegram.org/bot{bot_token}/sendMessage',
-            json={"chat_id": chat_id, "text": message, "parse_mode": "HTML"},
+            json=payload,
             timeout=10,
         )
         data = resp.json() if resp.status_code == 200 else {}

core/blocklist/__init__.py

@@ -0,0 +1,37 @@
+"""Artist / album / track blocklist (the "proper" blacklist).
+
+Distinct from ``download_blacklist`` (which skips one bad source file from one
+Soulseek peer — untouched here). This blocklist bans an ARTIST, ALBUM, or
+TRACK from being acquired, keyed by metadata-source IDs (Spotify / iTunes /
+Deezer / MusicBrainz) so a ban survives a source switch.
+
+Phase 1 enforces at the single ``add_to_wishlist`` chokepoint: every
+auto-acquisition path (watchlist, discography backfill, repair, manual
+wishlist add) funnels through it, so one guard covers them all.
+
+- ``matching`` — the pure decision core (no DB, no I/O): build an index from
+  blocklist rows, ask whether a candidate is blocked, with artist→album→track
+  cascade.
+"""
+
+from core.blocklist.matching import (
+    ENTITY_ALBUM,
+    ENTITY_ARTIST,
+    ENTITY_TRACK,
+    ENTITY_TYPES,
+    SOURCE_ID_FIELDS,
+    BlocklistIndex,
+    build_index,
+    candidate_block_reason,
+)
+
+__all__ = [
+    "ENTITY_ARTIST",
+    "ENTITY_ALBUM",
+    "ENTITY_TRACK",
+    "ENTITY_TYPES",
+    "SOURCE_ID_FIELDS",
+    "BlocklistIndex",
+    "build_index",
+    "candidate_block_reason",
+]

core/blocklist/backfill.py

@@ -0,0 +1,50 @@
+"""Cross-source ID backfill for blocklist entries.
+
+When a user blocks an item, the modal gives us the ID for ONE source (the one
+they searched). For the ban to survive a source switch, we resolve the OTHER
+sources' IDs too — matching the blocked artist/album/track by name on each
+source and taking a confident hit.
+
+The resolution is kept pure + injected so it tests without a network: callers
+pass a ``resolvers`` map ``{source: fn(entity_type, name, parent_name) -> id |
+None}``. ``core/blocklist/runtime.py`` wires the real metadata clients.
+
+Honest about fragility (acknowledged in design): artist matching is reliable,
+album/track cross-source matching is best-effort (editions, common titles), so
+a resolver returning None just leaves that source unmatched — the artist
+name-fallback in matching.py covers artist gaps; album/track gaps mean that
+ban only applies on sources where an ID resolved.
+"""
+
+from __future__ import annotations
+
+from typing import Any, Callable, Dict, Optional
+
+from core.blocklist.matching import SOURCE_ID_FIELDS
+
+
+def resolve_missing_ids(
+    entry: Dict[str, Any],
+    resolvers: Dict[str, Callable[..., Optional[str]]],
+) -> Dict[str, str]:
+    """Return ``{id_column: resolved_id}`` for the sources currently missing an
+    ID on ``entry``. Never raises — a resolver that errors is skipped."""
+    out: Dict[str, str] = {}
+    entity_type = entry.get("entity_type")
+    name = entry.get("name")
+    parent = entry.get("parent_name")
+    if not entity_type or not name:
+        return out
+    for source, col in SOURCE_ID_FIELDS.items():
+        if entry.get(col):
+            continue  # already known
+        fn = resolvers.get(source)
+        if not fn:
+            continue
+        try:
+            rid = fn(entity_type, name, parent)
+        except Exception:
+            rid = None
+        if rid:
+            out[col] = str(rid)
+    return out

core/blocklist/matching.py

@@ -0,0 +1,128 @@
+"""Pure blocklist matching — no DB, no I/O, fully unit-testable.
+
+The brain of the blocklist: given the stored blocklist rows and a candidate
+track being considered for the wishlist, decide whether it's blocked.
+
+Design decisions (per Boulder):
+- **ID-keyed.** Each row carries the candidate's IDs in up to four metadata
+  sources. A candidate is matched against the SAME source it came in on
+  (the wishlist payload carries active-source IDs), so a Deezer-numeric id
+  can't collide with an iTunes-numeric id of a different entity.
+- **Cascade.** Blocking an artist blocks their albums + tracks; blocking an
+  album blocks its tracks. The candidate carries its own artist/album/track
+  IDs, so the check walks track → album → artist and blocks on the first hit.
+- **Name fallback for ARTISTS only.** A blocked artist also matches by
+  case-folded name — this covers the window before the background ID-backfill
+  has resolved the active source's id. Albums/tracks do NOT fall back to name
+  (common titles like "Greatest Hits" would false-positive across artists);
+  they rely on IDs, which backfill fills in.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import Any, Dict, Iterable, List, Optional, Set, Tuple
+
+ENTITY_ARTIST = "artist"
+ENTITY_ALBUM = "album"
+ENTITY_TRACK = "track"
+ENTITY_TYPES = (ENTITY_ARTIST, ENTITY_ALBUM, ENTITY_TRACK)
+
+# Blocklist-row column → the metadata source it belongs to.
+SOURCE_ID_FIELDS = {
+    "spotify": "spotify_id",
+    "itunes": "itunes_id",
+    "deezer": "deezer_id",
+    "musicbrainz": "musicbrainz_id",
+}
+
+
+def _norm(text: Any) -> str:
+    return str(text or "").strip().casefold()
+
+
+@dataclass
+class _TypeIndex:
+    # per-source set of blocked ids, plus a case-folded name set (artists only)
+    ids: Dict[str, Set[str]] = field(default_factory=lambda: {s: set() for s in SOURCE_ID_FIELDS})
+    names: Set[str] = field(default_factory=set)
+
+    def hit(self, source: Optional[str], entity_id: Any, name: Any, use_name: bool) -> bool:
+        if entity_id and source in self.ids and str(entity_id) in self.ids[source]:
+            return True
+        if use_name and name and _norm(name) in self.names:
+            return True
+        return False
+
+
+@dataclass
+class BlocklistIndex:
+    """Membership index built once per scan from the blocklist rows."""
+    artists: _TypeIndex = field(default_factory=_TypeIndex)
+    albums: _TypeIndex = field(default_factory=_TypeIndex)
+    tracks: _TypeIndex = field(default_factory=_TypeIndex)
+
+    @property
+    def is_empty(self) -> bool:
+        for ti in (self.artists, self.albums, self.tracks):
+            if ti.names or any(ti.ids.values()):
+                return False
+        return True
+
+
+def build_index(rows: Iterable[Dict[str, Any]]) -> BlocklistIndex:
+    """Build a BlocklistIndex from blocklist DB rows.
+
+    Each row needs ``entity_type``, ``name``, and the source id columns
+    (``spotify_id`` / ``itunes_id`` / ``deezer_id`` / ``musicbrainz_id``).
+    Unknown entity types are ignored."""
+    idx = BlocklistIndex()
+    by_type = {ENTITY_ARTIST: idx.artists, ENTITY_ALBUM: idx.albums, ENTITY_TRACK: idx.tracks}
+    for row in rows or []:
+        ti = by_type.get((row.get("entity_type") or "").strip().lower())
+        if ti is None:
+            continue
+        for source, col in SOURCE_ID_FIELDS.items():
+            val = row.get(col)
+            if val:
+                ti.ids[source].add(str(val))
+        name = row.get("name")
+        if name:
+            ti.names.add(_norm(name))
+    return idx
+
+
+def candidate_block_reason(
+    index: BlocklistIndex,
+    *,
+    source: Optional[str],
+    track_id: Any = None,
+    track_name: Any = None,
+    album_id: Any = None,
+    album_name: Any = None,
+    artists: Optional[List[Dict[str, Any]]] = None,
+) -> Optional[Tuple[str, str]]:
+    """Return ``(entity_type, label)`` for the first cascade hit, else None.
+
+    ``source`` is the metadata source the candidate IDs came from (the wishlist
+    payload's provider). ``artists`` is a list of ``{'id', 'name'}`` dicts.
+    Order matters only for the returned reason — any hit blocks."""
+    if index.is_empty:
+        return None
+
+    # Track level — id only (names too ambiguous to ban across artists).
+    if index.tracks.hit(source, track_id, track_name, use_name=False):
+        return (ENTITY_TRACK, str(track_name or track_id or "track"))
+
+    # Album level — id only.
+    if index.albums.hit(source, album_id, album_name, use_name=False):
+        return (ENTITY_ALBUM, str(album_name or album_id or "album"))
+
+    # Artist level — id OR case-folded name (safe + covers the backfill window).
+    for artist in artists or []:
+        a_id = artist.get("id") if isinstance(artist, dict) else None
+        a_name = artist.get("name") if isinstance(artist, dict) else artist
+        if index.artists.hit(source, a_id, a_name, use_name=True):
+            return (ENTITY_ARTIST, str(a_name or a_id or "artist"))
+
+    return None

core/blocklist/runtime.py

@@ -0,0 +1,82 @@
+"""Wire real metadata clients to the blocklist backfill resolvers.
+
+Resolves a blocked item's ID on each metadata source by searching that source
+for the name and taking a confidently name-matched hit. Confidence = exact
+significant-token match (drops articles/punctuation) so we never hang a wrong
+ID on an entry. Albums/tracks additionally require the parent artist to match
+when both sides expose one.
+"""
+
+from __future__ import annotations
+
+import re
+from typing import Any, Callable, Dict, Optional
+
+from utils.logging_config import get_logger
+
+logger = get_logger("blocklist.runtime")
+
+_STOP = {"the", "a", "an", "feat", "ft", "featuring", "with"}
+
+
+def _tokens(text: Any) -> frozenset:
+    words = re.sub(r"[^a-z0-9]+", " ", str(text or "").lower()).split()
+    return frozenset(w for w in words if w not in _STOP)
+
+
+def _name_of(obj: Any) -> str:
+    if isinstance(obj, dict):
+        return str(obj.get("name") or obj.get("title") or "")
+    return str(getattr(obj, "name", None) or getattr(obj, "title", None) or "")
+
+
+def _id_of(obj: Any) -> Optional[str]:
+    val = obj.get("id") if isinstance(obj, dict) else getattr(obj, "id", None)
+    return str(val) if val else None
+
+
+def _confident(result_name: str, want_name: str) -> bool:
+    rt, wt = _tokens(result_name), _tokens(want_name)
+    return bool(rt) and rt == wt
+
+
+def _make_resolver(source: str) -> Callable[..., Optional[str]]:
+    def resolve(entity_type: str, name: str, parent_name: Optional[str] = None) -> Optional[str]:
+        from core.metadata.registry import get_client_for_source
+        client = get_client_for_source(source)
+        if not client:
+            return None
+        method = {
+            "artist": "search_artists",
+            "album": "search_albums",
+            "track": "search_tracks",
+        }.get(entity_type)
+        fn = getattr(client, method, None) if method else None
+        if not fn:
+            return None
+        try:
+            results = fn(name, limit=5) or []
+        except Exception as e:
+            logger.debug("%s %s search failed for %r: %s", source, entity_type, name, e)
+            return None
+        for r in results:
+            if not _confident(_name_of(r), name):
+                continue
+            # For album/track, also require the artist to line up when known.
+            if entity_type in ("album", "track") and parent_name:
+                artists = (r.get("artists") if isinstance(r, dict) else getattr(r, "artists", None)) or []
+                cand_artists = " ".join(
+                    a.get("name", "") if isinstance(a, dict) else str(a) for a in artists)
+                if _tokens(parent_name) and not (_tokens(parent_name) & _tokens(cand_artists)):
+                    continue
+            rid = _id_of(r)
+            if rid:
+                return rid
+        return None
+
+    return resolve
+
+
+def build_resolvers() -> Dict[str, Callable[..., Optional[str]]]:
+    """Source→resolver map for core.blocklist.backfill.resolve_missing_ids."""
+    return {s: _make_resolver(s) for s in ("spotify", "itunes", "deezer", "musicbrainz")}

core/boot_phase.py

@@ -0,0 +1,27 @@
+"""Boot-phase guard for non-blocking container startup.
+
+While the gunicorn worker is importing ``web_server`` (module-level client and
+worker initialization), external provider API probes must not block startup.
+Network validation is deferred until ``mark_boot_complete()`` runs at the end
+of that import pass.
+"""
+
+from __future__ import annotations
+
+import threading
+
+_boot_lock = threading.Lock()
+_boot_active = True
+
+
+def is_boot_phase() -> bool:
+    """Return True while module import must avoid blocking provider API calls."""
+    with _boot_lock:
+        return _boot_active
+
+
+def mark_boot_complete() -> None:
+    """End the boot phase — provider clients may perform network probes again."""
+    global _boot_active
+    with _boot_lock:
+        _boot_active = False

core/connection_detect.py

@@ -79,9 +79,9 @@ def run_detection(server_type):
                 api_response = requests.get(api_url, timeout=1)
                 if api_response.status_code == 200 and 'MediaContainer' in api_response.text:
                     return f"http://{ip}:{port}"
-                    
-        except:
-            pass
+
+        except Exception as e:
+            logger.debug("plex probe %s: %s", ip, e)
         return None
 
     def test_jellyfin_server(ip, port=8096):
@@ -101,9 +101,9 @@ def run_detection(server_type):
             web_response = requests.get(web_url, timeout=1)
             if web_response.status_code == 200 and 'jellyfin' in web_response.text.lower():
                 return f"http://{ip}:{port}"
-                
-        except:
-            pass
+
+        except Exception as e:
+            logger.debug("jellyfin probe %s: %s", ip, e)
         return None
 
     def test_slskd_server(ip, port=5030):
@@ -117,8 +117,8 @@ def run_detection(server_type):
             if response.status_code in [200, 401]:
                 return f"http://{ip}:{port}"
 
-        except:
-            pass
+        except Exception as e:
+            logger.debug("slskd probe %s: %s", ip, e)
         return None
 
     def test_navidrome_server(ip, port=4533):
@@ -140,8 +140,8 @@ def run_detection(server_type):
                     # Check for Subsonic/Navidrome API response structure
                     if 'subsonic-response' in data:
                         return f"http://{ip}:{port}"
-                except:
-                    pass
+                except Exception as e:
+                    logger.debug("navidrome json parse: %s", e)
 
             # Also try the web interface
             web_url = f"http://{ip}:{port}/"
@@ -149,8 +149,8 @@ def run_detection(server_type):
             if web_response.status_code == 200 and 'navidrome' in web_response.text.lower():
                 return f"http://{ip}:{port}"
 
-        except:
-            pass
+        except Exception as e:
+            logger.debug("navidrome probe %s: %s", ip, e)
         return None
 
     try:

core/connection_test.py

@@ -1,6 +1,6 @@
 """Service connection test — lifted from web_server.py.
 
-The function body is byte-identical to the original. soulseek_client,
+The function body is byte-identical to the original. download_orchestrator,
 qobuz_enrichment_worker, hydrabase_client, docker_resolve_url, and
 docker_resolve_path are injected at runtime because they live in
 web_server.py and are constructed there.
@@ -27,7 +27,7 @@ def _get_metadata_fallback_source():
 
 
 # Injected at runtime via init().
-soulseek_client = None
+download_orchestrator = None
 qobuz_enrichment_worker = None
 hydrabase_client = None
 docker_resolve_url = None
@@ -35,16 +35,16 @@ docker_resolve_path = None
 
 
 def init(
-    soulseek_client_obj,
+    download_orchestrator_obj,
     qobuz_worker,
     hydrabase_client_obj,
     docker_resolve_url_fn,
     docker_resolve_path_fn,
 ):
     """Bind web_server-side helpers/globals so the lifted body can resolve them."""
-    global soulseek_client, qobuz_enrichment_worker, hydrabase_client
+    global download_orchestrator, qobuz_enrichment_worker, hydrabase_client
     global docker_resolve_url, docker_resolve_path
-    soulseek_client = soulseek_client_obj
+    download_orchestrator = download_orchestrator_obj
     qobuz_enrichment_worker = qobuz_worker
     hydrabase_client = hydrabase_client_obj
     docker_resolve_url = docker_resolve_url_fn
@@ -82,7 +82,16 @@ def run_service_test(service, test_config):
                  if temp_client.is_spotify_authenticated():
                      return True, "Spotify connection successful!"
                  else:
-                     # Using fallback metadata source
+                     # Spotify-Free (no-auth) metadata path: officially unauthenticated,
+                     # but the no-creds source is selected and available. Report it as the
+                     # working source rather than the generic Deezer/Discogs/iTunes fallback.
+                     try:
+                         spotify_free_available = temp_client.is_spotify_metadata_available()
+                     except Exception:
+                         spotify_free_available = False
+                     if spotify_free_available:
+                         return True, "Spotify (no-auth) connection successful!"
+                     # Using a different fallback metadata source
                      fb_src = _get_metadata_fallback_source()
                      fallback_name = 'Deezer' if fb_src == 'deezer' else 'Discogs' if fb_src == 'discogs' else 'iTunes'
                      if spotify_configured:
@@ -177,13 +186,13 @@ def run_service_test(service, test_config):
             else:
                 return False, f"Output folder not found: {transfer_path}"
         elif service == "soulseek":
-            if soulseek_client is None:
+            if download_orchestrator is None:
                 return False, "Download orchestrator failed to initialize. Check server logs for startup errors."
 
             # Test the orchestrator's configured download source (not just Soulseek)
             download_mode = config_manager.get('download_source.mode', 'hybrid')
 
-            if run_async(soulseek_client.check_connection()):
+            if run_async(download_orchestrator.check_connection()):
                 # Success message based on active mode
                 mode_messages = {
                     'soulseek': "Successfully connected to Soulseek network via slskd.",
@@ -191,6 +200,12 @@ def run_service_test(service, test_config):
                     'tidal': "Tidal download source ready.",
                     'qobuz': "Qobuz download source ready.",
                     'hifi': "HiFi download source ready.",
+                    'deezer_dl': "Deezer download source ready.",
+                    'amazon': "Amazon download source ready.",
+                    'lidarr': "Lidarr download source ready.",
+                    'soundcloud': "SoundCloud download source ready.",
+                    'torrent': "Torrent download source ready.",
+                    'usenet': "Usenet download source ready.",
                     'hybrid': "Download sources ready (Hybrid mode)."
                 }
                 message = mode_messages.get(download_mode, "Download source connected.")
@@ -203,6 +218,12 @@ def run_service_test(service, test_config):
                     'tidal': "Tidal download source not available. Check authentication.",
                     'qobuz': "Qobuz download source not available. Check authentication.",
                     'hifi': "HiFi download source not available. Public API instances may be down.",
+                    'deezer_dl': "Deezer download source not available. Check authentication.",
+                    'amazon': "Amazon download source not available.",
+                    'lidarr': "Lidarr download source not available. Check Lidarr URL and API key.",
+                    'soundcloud': "SoundCloud download source not available.",
+                    'torrent': "Torrent download source not available. Check Prowlarr and torrent client settings.",
+                    'usenet': "Usenet download source not available. Check Prowlarr and usenet client settings.",
                     'hybrid': "Could not connect to download sources. Check configuration."
                 }
                 error = mode_errors.get(download_mode, "Download source connection failed.")
@@ -305,6 +326,61 @@ def run_service_test(service, test_config):
                     return False, "Invalid Genius access token."
             except Exception as e:
                 return False, f"Genius connection error: {str(e)}"
+        elif service == "usenet_client":
+            client_type = (config_manager.get('usenet_client.type', '') or '').strip().lower()
+            url = config_manager.get('usenet_client.url', '')
+            if not url:
+                return False, "Usenet client URL is required."
+            if not client_type:
+                return False, "Pick a usenet client (SABnzbd or NZBGet)."
+            try:
+                from core.usenet_clients import adapter_for_type as _usenet_adapter_for_type
+                adapter = _usenet_adapter_for_type(client_type)
+                if adapter is None:
+                    return False, f"Unknown usenet client type: {client_type}"
+                if not adapter.is_configured():
+                    if client_type == "sabnzbd":
+                        return False, "SABnzbd needs both URL and API key."
+                    return False, "NZBGet needs URL, username, and password."
+                if run_async(adapter.check_connection()):
+                    return True, f"Connected to {client_type}"
+                return False, f"{client_type} probe failed — check URL, credentials, and that the client is running."
+            except Exception as e:
+                return False, f"Usenet client connection error: {str(e)}"
+        elif service == "torrent_client":
+            client_type = (config_manager.get('torrent_client.type', '') or '').strip().lower()
+            url = config_manager.get('torrent_client.url', '')
+            if not url:
+                return False, "Torrent client URL is required."
+            if not client_type:
+                return False, "Pick a torrent client (qBittorrent, Transmission, or Deluge)."
+            try:
+                from core.torrent_clients import adapter_for_type
+                adapter = adapter_for_type(client_type)
+                if adapter is None:
+                    return False, f"Unknown torrent client type: {client_type}"
+                if not adapter.is_configured():
+                    return False, "Torrent client missing required credentials."
+                if run_async(adapter.check_connection()):
+                    return True, f"Connected to {client_type}"
+                return False, f"{client_type} probe failed — check URL, credentials, and that the client is running."
+            except Exception as e:
+                return False, f"Torrent client connection error: {str(e)}"
+        elif service == "prowlarr":
+            url = config_manager.get('prowlarr.url', '')
+            api_key = config_manager.get('prowlarr.api_key', '')
+            if not url or not api_key:
+                return False, "Prowlarr URL and API key are required."
+            try:
+                import requests as _req
+                resp = _req.get(f"{url.rstrip('/')}/api/v1/system/status",
+                                headers={'X-Api-Key': api_key}, timeout=10)
+                if resp.ok:
+                    version = resp.json().get('version', '?')
+                    return True, f"Connected to Prowlarr v{version}"
+                return False, f"Prowlarr returned HTTP {resp.status_code}"
+            except Exception as e:
+                return False, f"Prowlarr connection error: {str(e)}"
         elif service == "lidarr" or service == "lidarr_download":
             url = config_manager.get('lidarr_download.url', '')
             api_key = config_manager.get('lidarr_download.api_key', '')
@@ -379,6 +455,34 @@ def run_service_test(service, test_config):
                 return False, "Hydrabase not connected. Configure URL + API key and click Connect."
             except Exception as e:
                 return False, f"Hydrabase connection error: {str(e)}"
+        elif service == "musicbrainz":
+            try:
+                from core.metadata.registry import get_musicbrainz_client
+                mb = get_musicbrainz_client()
+                results = mb.search_artists("radiohead", limit=1)
+                if results:
+                    return True, "MusicBrainz reachable"
+                return False, "MusicBrainz returned no results — may be rate-limited or unreachable."
+            except Exception as e:
+                return False, f"MusicBrainz connection error: {str(e)}"
+        elif service == "soundcloud":
+            # Anonymous SoundCloud has no auth, so "test" really means
+            # "is yt-dlp installed and can it reach SoundCloud right now."
+            # This mirrors the /api/soundcloud/status check.
+            try:
+                from core.soundcloud_client import SoundcloudClient
+                sc = SoundcloudClient()
+                if not sc.is_available():
+                    return False, "SoundCloud unavailable — yt-dlp not installed."
+                # Run a tiny live probe via asyncio so the dashboard test
+                # gives a meaningful pass/fail.
+                import asyncio
+                reachable = asyncio.new_event_loop().run_until_complete(sc.check_connection())
+                if reachable:
+                    return True, "SoundCloud reachable (anonymous)"
+                return False, "SoundCloud unreachable — search probe failed. Try again."
+            except Exception as e:
+                return False, f"SoundCloud connection error: {str(e)}"
         return False, "Unknown service."
     except AttributeError as e:
         # This specifically catches the error you reported for Jellyfin

core/credentials/store.py

@@ -0,0 +1,85 @@
+"""Named, switchable service-credential sets — pure logic (Phase 0 foundation).
+
+Today every auth service (Spotify, Tidal, Deezer, Qobuz, Plex, Jellyfin,
+Navidrome) holds ONE credential set in config, and clients are global singletons
+built from that single slot. This module is the groundwork for letting an admin
+save MULTIPLE named credential sets per service ("pills") that each profile can
+switch between, without anyone but the admin creating them.
+
+Kept PURE — service registry, payload validation, and active-set selection,
+free of DB/Flask so it's unit-testable. Encrypted storage lives in MusicDatabase
+(service_credentials / profile_service_credentials tables); runtime client
+resolution + UI come in later phases. Nothing here changes existing behaviour;
+it's dormant capability until wired.
+"""
+
+from __future__ import annotations
+
+# Services that support multiple named credential sets, mapped to the payload
+# keys that MUST be present for a set to be usable. Extra keys (OAuth tokens,
+# redirect URIs, quality prefs) are allowed and preserved — these are only the
+# minimum required to validate a set the admin is saving.
+SERVICE_CREDENTIAL_SCHEMA = {
+    'spotify':   ('client_id', 'client_secret'),
+    'tidal':     ('access_token', 'refresh_token'),
+    'deezer':    ('arl',),
+    'qobuz':     ('user_auth_token',),
+    'plex':      ('base_url', 'token'),
+    'jellyfin':  ('base_url', 'api_key'),
+    'navidrome': ('base_url', 'username', 'password'),
+}
+
+SUPPORTED_SERVICES = frozenset(SERVICE_CREDENTIAL_SCHEMA)
+
+
+def is_supported_service(service: str) -> bool:
+    """True when the service supports named credential sets."""
+    return service in SERVICE_CREDENTIAL_SCHEMA
+
+
+def validate_credential_payload(service: str, payload):
+    """Return ``(ok, missing_keys)`` for a credential set.
+
+    Valid when every required key for the service is present and truthy. An
+    unknown service is invalid with no missing list (caller should reject it
+    as unsupported, not as "incomplete").
+    """
+    required = SERVICE_CREDENTIAL_SCHEMA.get(service)
+    if required is None:
+        return False, []
+    if not isinstance(payload, dict):
+        return False, list(required)
+
+    def _present(v):
+        # Whitespace-only strings count as missing — they'd otherwise save a
+        # blank secret that fails confusingly at the real service later.
+        return bool(v.strip()) if isinstance(v, str) else bool(v)
+
+    missing = [k for k in required if not _present(payload.get(k))]
+    return (not missing), missing
+
+
+def pick_active_credential(credentials, selected_id):
+    """From ``credentials`` (a list of dicts each carrying ``id``), return the
+    one whose id == ``selected_id``.
+
+    Returns None when there's no selection OR the selected id isn't present —
+    i.e. a stale pointer whose credential set was deleted. The caller then
+    falls back to the global/admin default, so a deleted set never breaks a
+    profile. Pure + stale-safe.
+    """
+    if not selected_id:
+        return None
+    for cred in credentials or []:
+        if cred.get('id') == selected_id:
+            return cred
+    return None
+
+
+__all__ = [
+    'SERVICE_CREDENTIAL_SCHEMA',
+    'SUPPORTED_SERVICES',
+    'is_supported_service',
+    'validate_credential_payload',
+    'pick_active_credential',
+]

core/database_update_health.py

@@ -0,0 +1,82 @@
+"""Stall detection for the database-update job.
+
+The DB updater keeps a single in-memory state dict whose ``status`` is set to
+``running`` at start and only flipped to ``finished``/``error`` by the worker's
+completion/error callbacks. If the worker thread hangs — e.g. a media-server API
+call with no timeout, a DB lock — those callbacks never fire, so ``status`` stays
+``running`` forever and the UI shows a frozen progress bar with no way to recover
+(GitHub #859).
+
+This module is the single, *pure* decision for "is a running job stalled?". It
+takes the state dict plus the current wall-clock time and a timeout, and answers
+yes/no — no DB, no globals, no clock of its own. That keeps it unit-testable and
+lets the watchdog wiring in web_server.py stay a thin call. The job carries a
+``last_progress_at`` epoch timestamp that the start path and every progress/phase
+callback bump; staleness is simply "running, and that timestamp is older than the
+timeout".
+"""
+
+from __future__ import annotations
+
+from typing import Any, Mapping
+
+# 5 minutes with zero forward progress = presumed hung. A healthy scan ticks
+# progress (per-artist) far more often than this even for large libraries, so
+# the timeout won't false-positive a slow-but-working run.
+DEFAULT_STALL_TIMEOUT_SECONDS = 300
+
+
+def is_db_update_stalled(
+    state: Mapping[str, Any],
+    now: float,
+    timeout_seconds: float = DEFAULT_STALL_TIMEOUT_SECONDS,
+) -> bool:
+    """Return True when the job is ``running`` but has made no progress within
+    ``timeout_seconds``.
+
+    Conservative by design — it only ever reports a stall it can prove:
+    - Only a ``running`` job can stall (idle/finished/error never do).
+    - With no usable ``last_progress_at`` timestamp we cannot judge, so we return
+      False rather than risk killing a job we have no clock for.
+    - A non-positive timeout is treated as "disabled" (never stalls).
+    """
+    if not isinstance(state, Mapping):
+        return False
+    if state.get("status") != "running":
+        return False
+    if timeout_seconds is None or timeout_seconds <= 0:
+        return False
+    last = state.get("last_progress_at")
+    if not last:
+        return False
+    try:
+        elapsed = float(now) - float(last)
+    except (TypeError, ValueError):
+        return False
+    return elapsed >= float(timeout_seconds)
+
+
+def stalled_error_message(state: Mapping[str, Any], now: float) -> str:
+    """Build a clear, human-facing message for a stalled job, including how long
+    it has been silent and the phase it died in."""
+    last = state.get("last_progress_at") if isinstance(state, Mapping) else None
+    phase = state.get("phase") if isinstance(state, Mapping) else None
+    try:
+        secs = int(float(now) - float(last)) if last else 0
+    except (TypeError, ValueError):
+        secs = 0
+    msg = "Update appears stuck — no progress"
+    if secs > 0:
+        msg += f" for {secs}s"
+    if phase:
+        msg += f" (last phase: {phase})"
+    msg += (". The worker may be hung on the media server. Start a new update "
+            "to try again, or restart SoulSync if it keeps stalling.")
+    return msg
+
+
+__all__ = [
+    "DEFAULT_STALL_TIMEOUT_SECONDS",
+    "is_db_update_stalled",
+    "stalled_error_message",
+]

core/database_update_worker.py

@@ -41,7 +41,19 @@ class DatabaseUpdateWorker:
         self.database_path = database_path
         self.full_refresh = full_refresh
         self.should_stop = False
-        
+
+        # Track ids of rows newly INSERTED this run (not updates). The web
+        # layer reads this to gap-fill embedded provider IDs for the new files
+        # (auto-reconcile), so newly-added music contributes its
+        # Spotify/MusicBrainz/etc. ids without a manual backfill.
+        self._new_track_ids = set()
+
+        # Optional callback(worker) run as the FINAL scan phase, immediately
+        # before the 'finished' signal — so the auto-reconcile is inside the
+        # scan's running window (automations/UI treat it as a normal phase and
+        # wait for it). Injected by the web layer (which owns path resolution).
+        self.post_scan_hook = None
+
         # Statistics tracking
         self.processed_artists = 0
         self.processed_albums = 0
@@ -79,7 +91,26 @@ class DatabaseUpdateWorker:
                 callback(*args)
             except Exception as e:
                 logger.error(f"Error in callback for {signal_name}: {e}")
-    
+
+    def _emit_finished(self, *args):
+        """Run the post-scan hook (auto-reconcile) as the final phase, THEN
+        emit 'finished'.
+
+        Running the hook before 'finished' keeps the scan's status at
+        'running' through the reconcile, so every caller (automations that
+        poll for completion, the dashboard card, the Tools page) treats it as
+        a normal scan phase and waits for it — rather than seeing 'finished'
+        and missing the tail. Best-effort: a hook failure never blocks the
+        completion signal.
+        """
+        if self.post_scan_hook:
+            try:
+                self.post_scan_hook(self)
+            except Exception as e:
+                logger.warning(f"post-scan hook failed (non-fatal): {e}")
+        self._emit_signal('finished', *args)
+
+
     def connect_callback(self, signal_name: str, callback: Callable):
         """Connect a callback for progress notifications."""
         self.callbacks.setdefault(signal_name, []).append(callback)
@@ -146,7 +177,7 @@ class DatabaseUpdateWorker:
                                 logger.info(f"Merged {merged} duplicate artists")
                         except Exception as e:
                             logger.warning(f"Could not merge duplicate artists: {e}")
-                    self._emit_signal('finished', 0, 0, 0, 0, 0)
+                    self._emit_finished(0, 0, 0, 0, 0)
                     return
                 logger.info(f"Incremental update: Found {len(artists_to_process)} artists to process")
             
@@ -230,7 +261,7 @@ class DatabaseUpdateWorker:
             self.removed_tracks = removal.get('tracks_removed', 0) if removal else 0
 
             # Emit final results
-            self._emit_signal('finished',
+            self._emit_finished(
                 self.processed_artists,
                 self.processed_albums,
                 self.processed_tracks,
@@ -331,7 +362,7 @@ class DatabaseUpdateWorker:
                         f"{self.processed_albums} albums, {self.processed_tracks} new tracks, "
                         f"{stale_removed} stale tracks removed")
 
-            self._emit_signal('finished',
+            self._emit_finished(
                 self.processed_artists,
                 self.processed_albums,
                 self.processed_tracks,
@@ -407,9 +438,14 @@ class DatabaseUpdateWorker:
                 logger.error(f"Could not connect to {self.server_type} server — check URL, credentials, and network (Docker users: use container name or host.docker.internal instead of host IP)")
                 return []
             
-            # Check for music library (Plex-specific check)
-            if self.server_type == "plex" and not self.media_client.music_library:
-                logger.error("No music library found in Plex")
+            # Check for music library (Plex-specific check). Routes
+            # through ``is_fully_configured`` so all-libraries mode (in
+            # which ``music_library`` is None but ``_all_libraries_mode``
+            # is True) counts as configured. Pre-fix this bailed out on
+            # the bare music_library None check, silently aborting the
+            # deep scan for any all-libraries-mode user.
+            if self.server_type == "plex" and not self.media_client.is_fully_configured():
+                logger.error("No music library configured in Plex")
                 return []
             
             # Check if database has enough content for incremental updates (server-specific)
@@ -875,6 +911,8 @@ class DatabaseUpdateWorker:
                                             track_success = self.database.insert_or_update_media_track(track, album_id, artist_id, server_source=self.server_type)
                                             if track_success:
                                                 total_processed_tracks += 1
+                                                if track_success == 'inserted':
+                                                    self._new_track_ids.add(str(track.ratingKey))
                                                 logger.debug(f"Processed new track: {track.title}")
                                         except Exception as e:
                                             logger.warning(f"Failed to process track '{getattr(track, 'title', 'Unknown')}': {e}")
@@ -1046,8 +1084,8 @@ class DatabaseUpdateWorker:
                             batch + [self.server_type])
                         cascade_album_ids.update(row[0] for row in cursor.fetchall())
                     removed_album_ids -= cascade_album_ids
-            except Exception:
-                pass  # If this optimization fails, double-delete is harmless
+            except Exception as e:
+                logger.debug("cascade album cleanup optimization: %s", e)
 
         if not removed_artist_ids and not removed_album_ids:
             logger.info("Removal detection: no stale content found")
@@ -1084,24 +1122,31 @@ class DatabaseUpdateWorker:
             return []
     
     def _get_recent_albums_plex(self) -> List:
-        """Get recently added and updated albums from Plex"""
+        """Get recently added and updated albums from Plex.
+
+        Routes through ``PlexClient.get_recently_added_albums`` and
+        ``get_recently_updated_albums`` so the all-libraries mode union
+        works (pre-fix this reached ``self.media_client.music_library.X``
+        directly which crashed when music_library is None in all-
+        libraries mode).
+        """
         all_recent_content = []
-        
+
         try:
-            # Get recently added albums (up to 400 to catch more recent content)  
-            try:
-                recently_added = self.media_client.music_library.recentlyAdded(libtype='album', maxresults=400)
+            # Get recently added albums (up to 400 to catch more recent content)
+            recently_added = self.media_client.get_recently_added_albums(maxresults=400, libtype='album')
+            if recently_added:
                 all_recent_content.extend(recently_added)
                 logger.info(f"Found {len(recently_added)} recently added albums")
-            except:
-                # Fallback to general recently added
-                recently_added = self.media_client.music_library.recentlyAdded(maxresults=400)
-                all_recent_content.extend(recently_added)
-                logger.info(f"Found {len(recently_added)} recently added items (mixed types)")
-            
+            else:
+                # Fallback to mixed-type recents.
+                recently_added = self.media_client.get_recently_added_albums(maxresults=400, libtype=None)
+                all_recent_content.extend(recently_added or [])
+                logger.info(f"Found {len(recently_added or [])} recently added items (mixed types)")
+
             # Get recently updated albums (catches metadata corrections)
             try:
-                recently_updated = self.media_client.music_library.search(sort='updatedAt:desc', libtype='album', limit=400)
+                recently_updated = self.media_client.get_recently_updated_albums(limit=400)
                 # Remove duplicates (items that are both recently added and updated)
                 added_keys = {getattr(item, 'ratingKey', None) for item in all_recent_content}
                 unique_updated = [item for item in recently_updated if getattr(item, 'ratingKey', None) not in added_keys]
@@ -1332,6 +1377,8 @@ class DatabaseUpdateWorker:
                                             skipped_count += 1
                                         elif track_success:
                                             track_count += 1
+                                            if track_success == 'inserted':
+                                                self._new_track_ids.add(track_id_str)
                                     except Exception as e:
                                         logger.warning(f"Failed to process track '{getattr(track, 'title', 'Unknown')}': {e}")
 

core/db_integrity.py

@@ -0,0 +1,153 @@
+"""SQLite integrity + safe-backup helpers.
+
+Born out of a real incident: a WAL-mode DB got corrupted (most likely an
+interrupted write during a hard restart), and because the backup routine
+(a) never checked integrity and (b) rotated the oldest backup out by mtime,
+every rolling backup ended up being a faithful copy of the already-corrupt
+file — so when recovery was needed, all snapshots were poisoned.
+
+This module makes that impossible:
+
+* ``quick_check(path)`` / ``is_healthy(path)`` — fast read-only integrity probe.
+* ``safe_backup(...)`` — verifies the SOURCE is healthy before copying, uses the
+  SQLite Online Backup API, then verifies the RESULT. A corrupt source never
+  produces (or keeps) a backup.
+* ``prune_backups(...)`` — rotation that NEVER deletes the most recent
+  *verified-healthy* backup, even to honor the max-count, so a run of bad
+  backups can't evict your last good one.
+
+Pure-ish: only touches sqlite3 + the filesystem paths it's given; no Flask, no
+app globals. Unit-testable with real (and deliberately-corrupted) temp DBs.
+"""
+
+from __future__ import annotations
+
+import logging
+import os
+import sqlite3
+from typing import Optional
+
+logger = logging.getLogger("db_integrity")
+
+
+def _close_quietly(conn) -> None:
+    """Best-effort close; a failure to close during cleanup must not mask the
+    real error we're handling, but we log it rather than swallow silently."""
+    if conn is None:
+        return
+    try:
+        conn.close()
+    except Exception as e:  # noqa: BLE001 — cleanup path, real error already in flight
+        logger.debug("db_integrity: connection close failed: %s", e)
+
+
+class DBIntegrityError(Exception):
+    """Raised when a database fails its integrity check."""
+
+
+def quick_check(db_path: str, *, timeout: float = 30.0) -> str:
+    """Run ``PRAGMA quick_check`` read-only and return its first result row.
+
+    Returns ``'ok'`` for a healthy DB, otherwise the first error line. Raises
+    ``DBIntegrityError`` if the file can't even be opened/read (malformed
+    header, I/O error) — i.e. unambiguously bad.
+    """
+    if not os.path.exists(db_path):
+        raise DBIntegrityError(f"Database file not found: {db_path}")
+    conn = None
+    try:
+        conn = sqlite3.connect(f"file:{db_path}?mode=ro", uri=True, timeout=timeout)
+        row = conn.execute("PRAGMA quick_check(1)").fetchone()
+        return (row[0] if row else "no result")
+    except sqlite3.DatabaseError as e:
+        # malformed header / disk image malformed / disk I/O error
+        raise DBIntegrityError(f"{db_path}: {e}") from e
+    finally:
+        _close_quietly(conn)
+
+
+def is_healthy(db_path: str, *, timeout: float = 30.0) -> bool:
+    """True iff the DB opens and ``quick_check`` reports 'ok'. Never raises."""
+    try:
+        return quick_check(db_path, timeout=timeout) == "ok"
+    except DBIntegrityError:
+        return False
+
+
+def safe_backup(src_path: str, dst_path: str, *, verify_source: bool = True,
+                verify_result: bool = True) -> None:
+    """Back up ``src_path`` to ``dst_path`` via the SQLite Online Backup API,
+    refusing to produce a backup from (or keep a backup of) a corrupt DB.
+
+    Raises ``DBIntegrityError`` and removes any partial ``dst_path`` when the
+    source is unhealthy (``verify_source``) or the produced backup fails its
+    own check (``verify_result``). On success ``dst_path`` is a verified-good
+    copy.
+    """
+    if verify_source and not is_healthy(src_path):
+        # Don't immortalize corruption — surface it so the caller can alert
+        # and, crucially, NOT rotate out the existing good backups.
+        raise DBIntegrityError(
+            f"Refusing to back up: source database failed integrity check ({src_path})"
+        )
+
+    src = dst = None
+    try:
+        src = sqlite3.connect(src_path)
+        dst = sqlite3.connect(dst_path)
+        src.backup(dst)
+    finally:
+        _close_quietly(dst)
+        _close_quietly(src)
+
+    if verify_result and not is_healthy(dst_path):
+        # The copy itself came out bad — discard it rather than keep a dud.
+        try:
+            os.remove(dst_path)
+        except OSError:
+            pass
+        raise DBIntegrityError(
+            f"Backup produced a corrupt file and was discarded ({dst_path})"
+        )
+
+
+def prune_backups(backup_paths, max_keep: int,
+                  health_check=is_healthy) -> list:
+    """Decide which backups to delete to honor ``max_keep`` WITHOUT ever
+    deleting the most-recent verified-healthy backup.
+
+    ``backup_paths`` is an iterable of paths; order does not matter (we sort by
+    mtime). Returns the list of paths that SHOULD be deleted (does not delete
+    them — the caller does the IO, so this stays pure/testable).
+
+    Rule: oldest-first deletion until <= max_keep, but the single newest
+    *healthy* backup is protected and never selected for deletion. So even if
+    the newest few backups are corrupt, the last good snapshot survives.
+    """
+    paths = [p for p in backup_paths]
+    # Newest first.
+    paths.sort(key=lambda p: _safe_mtime(p), reverse=True)
+
+    # Find the newest healthy backup — the one we must never drop.
+    protected: Optional[str] = None
+    for p in paths:
+        if health_check(p):
+            protected = p
+            break
+
+    if len(paths) <= max_keep:
+        return []
+
+    # Delete oldest-first beyond max_keep, but skip the protected one.
+    deletable = [p for p in paths if p != protected]
+    # oldest first among deletable
+    deletable.sort(key=lambda p: _safe_mtime(p))
+    num_to_delete = len(paths) - max_keep
+    return deletable[:num_to_delete]
+
+
+def _safe_mtime(path: str) -> float:
+    try:
+        return os.path.getmtime(path)
+    except OSError:
+        return 0.0

core/debug_info.py

@@ -14,7 +14,12 @@ from pathlib import Path
 from flask import jsonify, request
 
 from config.settings import config_manager
-from core.metadata.registry import get_spotify_client
+from core.metadata.registry import (
+    get_spotify_client,
+    get_primary_source,
+    is_hydrabase_enabled,
+)
+from core.metadata.status import get_spotify_status
 
 logger = logging.getLogger(__name__)
 
@@ -64,7 +69,7 @@ download_batches = None
 sync_states = None
 youtube_playlist_states = None
 tidal_discovery_states = None
-soulseek_client = None
+download_orchestrator = None
 _log_path = None
 _log_dir = None
 app = None
@@ -80,7 +85,7 @@ def init(
     sync_states_dict,
     youtube_playlist_states_dict,
     tidal_discovery_states_dict,
-    soulseek_client_obj,
+    download_orchestrator_obj,
     log_path,
     log_dir,
     flask_app,
@@ -90,7 +95,7 @@ def init(
     """Bind shared state/helpers from web_server."""
     global SOULSYNC_VERSION, _DIRECT_RUN, _status_cache, qobuz_enrichment_worker
     global download_batches, sync_states, youtube_playlist_states
-    global tidal_discovery_states, soulseek_client, _log_path, _log_dir
+    global tidal_discovery_states, download_orchestrator, _log_path, _log_dir
     global app, get_database, _get_tidal_client
     SOULSYNC_VERSION = soulsync_version
     _DIRECT_RUN = direct_run
@@ -100,7 +105,7 @@ def init(
     sync_states = sync_states_dict
     youtube_playlist_states = youtube_playlist_states_dict
     tidal_discovery_states = tidal_discovery_states_dict
-    soulseek_client = soulseek_client_obj
+    download_orchestrator = download_orchestrator_obj
     _log_path = log_path
     _log_dir = log_dir
     app = flask_app
@@ -183,21 +188,47 @@ def get_debug_info():
         info['paths']['music_videos_path'] = music_videos_path
         info['paths']['music_videos_path_exists'] = os.path.isdir(music_videos_path)
 
-    # Services from status cache
-    spotify_cache = _status_cache.get('spotify', {})
+    # Services. `_status_cache` only carries 'media_server' and 'soulseek'
+    # (no 'spotify' key) so anything we used to read from `spotify_cache`
+    # silently defaulted to the missing-value fallback — that's the
+    # "music_source: unknown" bug. Spotify status now comes from the
+    # canonical `get_spotify_status` accessor; primary metadata source
+    # comes from `get_primary_source` (which already accounts for the
+    # auth-fallback chain — Spotify drops back to Deezer when not
+    # authenticated).
     media_server_cache = _status_cache.get('media_server', {})
     soulseek_cache = _status_cache.get('soulseek', {})
+    spotify_status = _safe_check(lambda: get_spotify_status(spotify_client=spotify_client), default={})
+    if not isinstance(spotify_status, dict):
+        spotify_status = {}
     info['services'] = {
-        'music_source': spotify_cache.get('source', 'unknown'),
-        'spotify_connected': spotify_cache.get('connected', False),
-        'spotify_rate_limited': spotify_cache.get('rate_limited', False),
+        'music_source': _safe_check(get_primary_source, default='unknown') or 'unknown',
+        'spotify_connected': bool(spotify_status.get('connected', False)),
+        'spotify_rate_limited': bool(spotify_status.get('rate_limited', False)),
         'media_server_type': media_server_cache.get('type', 'none'),
         'media_server_connected': media_server_cache.get('connected', False),
         'soulseek_connected': soulseek_cache.get('connected', False),
         'download_source': config_manager.get('download_source.mode', 'hybrid'),
         'tidal_connected': _safe_check(lambda: bool(tidal_client and tidal_client.is_authenticated())),
         'qobuz_connected': _safe_check(lambda: bool(qobuz_enrichment_worker and qobuz_enrichment_worker.client and qobuz_enrichment_worker.client.is_authenticated())),
+        'hydrabase_connected': _safe_check(is_hydrabase_enabled),
+        # YouTube is URL-based via yt-dlp — no auth, always reachable as
+        # long as the binary is installed. Surfaced so the debug dump
+        # documents that YouTube is one of the available download sources
+        # rather than implying it doesn't exist.
+        'youtube_available': True,
     }
+    # HiFi instance count — separate from connection status because each
+    # instance is its own independent endpoint with its own auth state.
+    info['services']['hifi_instance_count'] = _safe_check(
+        lambda: len(get_database().get_hifi_instances()), default=0
+    )
+    # Always-available public metadata sources (no auth, no per-user
+    # connection state). Listed so the debug dump reflects the full
+    # metadata surface SoulSync queries from, not just the auth-gated ones.
+    info['services']['always_available_metadata_sources'] = [
+        'deezer', 'itunes', 'musicbrainz',
+    ]
 
     # Enrichment workers
     workers = {}
@@ -260,8 +291,8 @@ def get_debug_info():
         for _pid, st in list(tidal_discovery_states.items()):
             if st.get('phase') == 'syncing':
                 active_syncs += 1
-    except Exception:
-        pass
+    except Exception as e:
+        logger.debug("count active syncs failed: %s", e)
     info['active_downloads'] = active_downloads
     info['active_syncs'] = active_syncs
 
@@ -292,31 +323,36 @@ def get_debug_info():
 
     # Download client init failures
     info['download_client_failures'] = []
-    if soulseek_client and hasattr(soulseek_client, '_init_failures'):
-        info['download_client_failures'] = soulseek_client._init_failures
-    elif not soulseek_client:
+    if download_orchestrator and hasattr(download_orchestrator, '_init_failures'):
+        info['download_client_failures'] = download_orchestrator._init_failures
+    elif not download_orchestrator:
         info['download_client_failures'] = ['ALL (orchestrator failed to initialize)']
 
     # API rate monitor — current calls/min, 24h totals, peaks, rate limit events
     try:
         from core.api_call_tracker import api_call_tracker
+        # `get_spotify_status` is already imported at module level. A
+        # local re-import here would make Python treat the name as a
+        # function-scoped local for the WHOLE body, breaking the lambda
+        # at the top of get_debug_info that closes over the module-level
+        # binding (Python 3.12 NameError on free variables).
         rates = api_call_tracker.get_all_rates()
         info['api_rates'] = rates
         # Rich 24h debug summary with peaks, totals, per-endpoint breakdown, events
         info['api_debug_summary'] = api_call_tracker.get_debug_summary()
         # Spotify rate limit details
-        if spotify_client:
-            rl_info = spotify_client.get_rate_limit_info()
-            if rl_info:
-                info['spotify_rate_limit'] = {
-                    'active': True,
-                    'remaining_seconds': rl_info.get('remaining_seconds', 0),
-                    'retry_after': rl_info.get('retry_after', 0),
-                    'endpoint': rl_info.get('endpoint', ''),
-                    'expires_at': rl_info.get('expires_at', ''),
-                }
-            else:
-                info['spotify_rate_limit'] = {'active': False}
+        spotify_status = get_spotify_status(spotify_client=spotify_client)
+        rl_info = spotify_status.get('rate_limit')
+        if spotify_status.get('rate_limited') and rl_info:
+            info['spotify_rate_limit'] = {
+                'active': True,
+                'remaining_seconds': rl_info.get('remaining_seconds', 0),
+                'retry_after': rl_info.get('retry_after', 0),
+                'endpoint': rl_info.get('endpoint', ''),
+                'expires_at': rl_info.get('expires_at', ''),
+            }
+        else:
+            info['spotify_rate_limit'] = {'active': False}
     except Exception:
         info['api_rates'] = {}
         info['api_debug_summary'] = {}

core/deezer_client.py

@@ -6,6 +6,7 @@ from typing import Dict, List, Optional, Any
 from functools import wraps
 from dataclasses import dataclass
 from utils.logging_config import get_logger
+from core.metadata.artist_album_cache import get_cached_artist_album_items, store_artist_album_items
 from core.metadata.cache import get_metadata_cache
 
 logger = get_logger("deezer_client")
@@ -45,6 +46,119 @@ def rate_limited(func):
     return wrapper
 
 
+# Pattern matches Deezer's CDN cover/picture URL: a numeric width-x-height
+# segment in the path (e.g. ``/1000x1000-000000-80-0-0.jpg``). Captures
+# both halves so the replacement can use a single dimension and preserve
+# the rest of the path verbatim.
+_DEEZER_CDN_SIZE_PATTERN = re.compile(r'/(\d+)x(\d+)-')
+
+# Maximum size Deezer's CDN serves before returning 403. Verified
+# empirically against multiple albums — 1900 works reliably, 2000+
+# returns Forbidden. CDN serves the source-native size when it's
+# smaller than requested, so asking for 1900 is safe even on albums
+# whose source upload was lower-res (no upscaling, just same bytes).
+_DEEZER_MAX_COVER_SIZE = 1900
+
+
+def _upgrade_deezer_cover_url(url: str, target_size: int = _DEEZER_MAX_COVER_SIZE) -> str:
+    """Rewrite a Deezer CDN cover/picture URL to request a larger size.
+
+    Deezer's API returns ``cover_xl`` / ``picture_xl`` URLs at
+    1000×1000, but the underlying CDN serves up to 1900×1900 by
+    rewriting the size segment in the URL path. This helper does the
+    rewrite — same idea as ``_upgrade_spotify_image_url`` in
+    ``spotify_client`` and the ``mzstatic.com`` size-replacement in
+    ``download_cover_art``.
+
+    Defensive on every input shape:
+    - Empty / None URL → returned as-is
+    - Non-Deezer URL (no ``dzcdn`` host, no size segment) → returned as-is
+    - Already at or above target size → returned as-is (no point rewriting)
+
+    The CDN returns the source-native image bytes when source < target,
+    so asking for 1900 on an album whose source was uploaded at 600
+    just returns the 600-pixel image — no upscaling, no failure.
+    """
+    if not url or 'dzcdn' not in url:
+        return url
+    match = _DEEZER_CDN_SIZE_PATTERN.search(url)
+    if not match:
+        return url
+    current = int(match.group(1))
+    if current >= target_size:
+        return url
+    return _DEEZER_CDN_SIZE_PATTERN.sub(f'/{target_size}x{target_size}-', url, count=1)
+
+
+def _is_full_track_payload(payload: Optional[Dict[str, Any]]) -> bool:
+    """Distinguish a full `/track/<id>` cache hit from partial album-tracks data.
+
+    Three Deezer endpoints feed the per-track cache:
+      - `/track/<id>` — full record, includes both `track_position` AND
+        `contributors` (the multi-artist list the contributors-upgrade
+        path reads).
+      - `/album/<id>/tracks` — partial; includes `track_position` but
+        omits `contributors`.
+      - `/search/track` — minimal; lacks `track_position`.
+
+    Pre-fix `get_track_details` only checked `track_position`, so
+    partial album-tracks payloads were treated as full hits and the
+    contributors-upgrade silently fell back to single-artist tagging
+    whenever an album had been fetched before its individual tracks
+    were post-processed (issue #588).
+
+    `contributors` key presence is the load-bearing distinction —
+    `[]` is a valid value for genuinely single-artist tracks fetched
+    via the per-track endpoint, so test for key membership not
+    truthiness.
+    """
+    if not isinstance(payload, dict):
+        return False
+    return 'track_position' in payload and 'contributors' in payload
+
+
+def resolve_album_track_positions(session, base_url, album_ids, cache=None, sleep_s=0.2):
+    """Build ``{str(track_id): track_position}`` for a set of Deezer album ids.
+
+    Deezer PLAYLIST and SEARCH track objects (and even the album object's embedded
+    ``tracks.data``) omit ``track_position`` — only ``/album/<id>/tracks`` and
+    ``/track/<id>`` carry it. So numbering playlist tracks by their playlist index
+    silently poisons the real album track number, which then rides onto the
+    downloaded file's tag. This resolves the authoritative position per album
+    (cache-first, best-effort — a failed album just isn't in the map)."""
+    import time as _time
+    positions: Dict[str, int] = {}
+    for aid in album_ids:
+        aid = str(aid)
+        at_list = None
+        if cache:
+            try:
+                ct = cache.get_entity('deezer', 'album_tracks', aid)
+                if ct and ct.get('data'):
+                    at_list = ct['data']
+            except Exception:   # noqa: BLE001 - cache is best-effort
+                at_list = None
+        if at_list is None:
+            try:
+                if sleep_s:
+                    _time.sleep(sleep_s)   # respect Deezer rate limits
+                r = session.get(f"{base_url}/album/{aid}/tracks", params={'limit': 500}, timeout=10)
+                if getattr(r, 'ok', False):
+                    at_list = (r.json() or {}).get('data', [])
+                    if cache and at_list is not None:
+                        try:
+                            cache.store_entity('deezer', 'album_tracks', aid, {'data': at_list})
+                        except Exception as _cache_err:   # noqa: BLE001
+                            logger.debug("album_tracks cache store failed for %s: %s", aid, _cache_err)
+            except Exception:   # noqa: BLE001 - never let metadata resolution break the fetch
+                at_list = None
+        for at in (at_list or []):
+            tp = at.get('track_position')
+            if at.get('id') and tp:
+                positions[str(at['id'])] = tp
+    return positions
+
+
 # ==================== Dataclasses (match iTunesClient / SpotifyClient format) ====================
 
 @dataclass
@@ -168,6 +282,7 @@ class Album:
     album_type: str
     image_url: Optional[str] = None
     external_urls: Optional[Dict[str, str]] = None
+    explicit: Optional[bool] = None
 
     @classmethod
     def from_deezer_album(cls, album_data: Dict[str, Any]) -> 'Album':
@@ -199,7 +314,8 @@ class Album:
             total_tracks=album_data.get('nb_tracks', 0),
             album_type=album_type,
             image_url=image_url,
-            external_urls=external_urls if external_urls else None
+            external_urls=external_urls if external_urls else None,
+            explicit=bool(album_data.get('explicit_lyrics', False)),
         )
 
 
@@ -298,39 +414,136 @@ class DeezerClient:
     # can serve as a drop-in fallback metadata source in SpotifyClient.
 
     @rate_limited
-    def search_tracks(self, query: str, limit: int = 20) -> List[Track]:
-        """Search for tracks — returns Track dataclass list (metadata source interface)"""
+    def search_tracks(
+        self,
+        query: str = '',
+        limit: int = 20,
+        *,
+        track: Optional[str] = None,
+        artist: Optional[str] = None,
+        album: Optional[str] = None,
+    ) -> List[Track]:
+        """Search for tracks — returns Track dataclass list (metadata source interface).
+
+        Two call modes:
+
+        1. **Free-text** (`query='Foreigner Dirty White Boy'`) — legacy
+           shape, passes the string straight to Deezer's `q` param.
+           Same behaviour as before, kept for backward compat.
+
+        2. **Field-scoped** (`track='Dirty White Boy', artist='Foreigner'`) —
+           builds Deezer's advanced search syntax (`track:"X" artist:"Y"`).
+           Massively tighter relevance than the free-text path because
+           the API matches each term in the right field instead of
+           anywhere across title / lyrics / artist / album / contributors.
+           Without this, the Deezer ranking buries the canonical track
+           under karaoke / cover / "originally performed by" variants
+           — see issue #534.
+
+        Field-scoped form is used whenever ``track`` or ``artist`` is
+        provided. ``query`` is ignored in that case (the field params
+        are authoritative). When both are missing, falls through to
+        ``query``. The cache key is the constructed query string in
+        either case so the two paths share entries naturally.
+        """
+        # Build the actual API query — advanced syntax when callers pass
+        # field hints, raw query otherwise.
+        used_advanced = bool(track or artist or album)
+        if used_advanced:
+            api_query = self._build_advanced_query(track=track, artist=artist, album=album)
+        else:
+            api_query = query
+
+        if not api_query:
+            return []
+
+        tracks = self._search_tracks_with_query(api_query, limit)
+
+        # Safety net: Deezer's advanced syntax is `artist:"X"`-style
+        # substring match, but in practice it's brittle on artist name
+        # variants ("Foreigner [US]", "The Foreigner", etc.) and on
+        # tracks indexed under non-canonical title spellings. When the
+        # advanced query returns nothing, fall back to a free-text join
+        # so the user sees the prior (less-relevant but non-empty) result
+        # set rather than "No matches" — same behaviour as pre-fix for
+        # this edge case. Caller-side rerank still tightens the result.
+        if not tracks and used_advanced:
+            fallback_parts = [p for p in (track, artist, album) if p]
+            fallback_query = ' '.join(fallback_parts)
+            if fallback_query and fallback_query != api_query:
+                logger.debug(
+                    "[Deezer] Advanced query returned 0 results, falling back "
+                    "to free-text: %r → %r", api_query, fallback_query,
+                )
+                tracks = self._search_tracks_with_query(fallback_query, limit)
+
+        return tracks
+
+    def _search_tracks_with_query(self, api_query: str, limit: int) -> List[Track]:
+        """Cache-aware single API call. Pulled out so the
+        ``search_tracks`` orchestration can call this twice (advanced
+        query → free-text fallback) without duplicating the cache +
+        parse + store dance."""
         cache = get_metadata_cache()
-        cached_results = cache.get_search_results('deezer', 'track', query, limit)
+        cached_results = cache.get_search_results('deezer', 'track', api_query, limit)
         if cached_results is not None:
             tracks = []
             for raw in cached_results:
                 try:
                     tracks.append(Track.from_deezer_track(raw))
-                except Exception:
-                    pass
+                except Exception as e:
+                    logger.debug("Track.from_deezer_track cache parse: %s", e)
             if tracks:
                 return tracks
 
-        data = self._api_get('search/track', {'q': query, 'limit': min(limit, 100)})
+        data = self._api_get('search/track', {'q': api_query, 'limit': min(limit, 100)})
         if not data or 'data' not in data:
             return []
 
         tracks = []
         raw_items = []
         for track_data in data['data']:
-            track = Track.from_deezer_track(track_data)
-            tracks.append(track)
+            track_obj = Track.from_deezer_track(track_data)
+            tracks.append(track_obj)
             raw_items.append(track_data)
 
         entries = [(str(td.get('id', '')), td) for td in raw_items if td.get('id')]
         if entries:
             cache.store_entities_bulk('deezer', 'track', entries)
-            cache.store_search_results('deezer', 'track', query, limit,
+            cache.store_search_results('deezer', 'track', api_query, limit,
                                        [str(td.get('id', '')) for td in raw_items if td.get('id')])
 
         return tracks
 
+    @staticmethod
+    def _build_advanced_query(
+        *,
+        track: Optional[str] = None,
+        artist: Optional[str] = None,
+        album: Optional[str] = None,
+    ) -> str:
+        """Compose Deezer's advanced search syntax from field hints.
+
+        Per Deezer's docs:
+        https://developers.deezer.com/api/search
+
+            q=track:"X" artist:"Y" album:"Z"
+
+        Quotes around each value preserve multi-word phrases. Empty
+        fields are skipped. Embedded double-quotes get stripped (no
+        escape mechanism in Deezer's syntax) — rare in practice, but
+        a search for `O"Hara` would otherwise produce a malformed
+        query.
+        """
+        parts = []
+        if track:
+            parts.append(f'track:"{track.replace(chr(34), "")}"')
+        if artist:
+            parts.append(f'artist:"{artist.replace(chr(34), "")}"')
+        if album:
+            parts.append(f'album:"{album.replace(chr(34), "")}"')
+        return ' '.join(parts)
+
     @rate_limited
     def search_artists(self, query: str, limit: int = 20) -> List[Artist]:
         """Search for artists — returns Artist dataclass list (metadata source interface)"""
@@ -341,8 +554,8 @@ class DeezerClient:
             for raw in cached_results:
                 try:
                     artists.append(Artist.from_deezer_artist(raw))
-                except Exception:
-                    pass
+                except Exception as e:
+                    logger.debug("Artist.from_deezer_artist cache parse: %s", e)
             if artists:
                 return artists
 
@@ -375,8 +588,8 @@ class DeezerClient:
             for raw in cached_results:
                 try:
                     albums.append(Album.from_deezer_album(raw))
-                except Exception:
-                    pass
+                except Exception as e:
+                    logger.debug("Album.from_deezer_album cache parse: %s", e)
             if albums:
                 return albums
 
@@ -403,14 +616,9 @@ class DeezerClient:
         """Get detailed track info — returns Spotify-compatible dict (metadata source interface)"""
         cache = get_metadata_cache()
         cached = cache.get_entity('deezer', 'track', str(track_id))
-        if cached and cached.get('title'):
-            # Search results are cached with minimal data (no track_position).
-            # Only use cache if it has track_position — the key field from /track/{id}.
-            # Search results include 'isrc' and 'release_date' but NOT track_position,
-            # so those fields alone are not sufficient to distinguish full from partial data.
-            if 'track_position' in cached:
-                return self._build_enhanced_track(cached)
-            # Otherwise fall through to fetch full data from API
+        if cached and cached.get('title') and _is_full_track_payload(cached):
+            return self._build_enhanced_track(cached)
+        # Otherwise fall through to fetch full data from API
 
         data = self._api_get(f'track/{track_id}')
         if not data:
@@ -599,6 +807,74 @@ class DeezerClient:
 
         return result
 
+    def get_artist_top_tracks(self, artist_id: str, limit: int = 10) -> List[Dict[str, Any]]:
+        """Return the artist's top tracks in Spotify-compatible dict format.
+
+        Wraps Deezer's `/artist/{id}/top?limit=N`. Returns dicts with the same
+        shape Spotify's `artist_top_tracks` produces — id, name, artists, album
+        (with album_type / total_tracks / release_date / images), duration_ms,
+        track_number, disc_number — so callers don't need to branch on source.
+        """
+        if not artist_id:
+            return []
+        try:
+            limit = max(1, min(int(limit or 10), 100))
+        except (TypeError, ValueError):
+            limit = 10
+
+        data = self._api_get(f'artist/{artist_id}/top', {'limit': limit})
+        if not data or 'data' not in data:
+            return []
+
+        tracks = []
+        for track_data in data['data']:
+            if not isinstance(track_data, dict):
+                continue
+            artist_data = track_data.get('artist') or {}
+            album_data = track_data.get('album') or {}
+
+            # Build images list from any cover sizes Deezer returned for the album
+            images = []
+            if isinstance(album_data, dict):
+                for size_key, dim in [('cover_xl', 1000), ('cover_big', 500),
+                                       ('cover_medium', 250), ('cover_small', 56)]:
+                    if album_data.get(size_key):
+                        images.append({'url': album_data[size_key], 'height': dim, 'width': dim})
+
+            # Deezer `/artist/{id}/top` results don't include record_type on the
+            # nested album object; we don't have a track-count to infer from
+            # either. Default 'album' so the path-builder template variable
+            # always has something to substitute (existing behavior elsewhere).
+            album_payload = {
+                'id': str(album_data.get('id', '')) if isinstance(album_data, dict) else '',
+                'name': album_data.get('title', '') if isinstance(album_data, dict) else '',
+                'album_type': 'album',
+                'images': images,
+                'release_date': '',
+                'total_tracks': 0,
+                'artists': [{'name': artist_data.get('name', '')}] if isinstance(artist_data, dict) else [],
+            }
+
+            tracks.append({
+                'id': str(track_data.get('id', '')),
+                'name': track_data.get('title', ''),
+                'artists': [{
+                    'id': str(artist_data.get('id', '')) if isinstance(artist_data, dict) else '',
+                    'name': artist_data.get('name', '') if isinstance(artist_data, dict) else '',
+                }],
+                'album': album_payload,
+                'duration_ms': (track_data.get('duration') or 0) * 1000,  # Deezer is seconds
+                'popularity': track_data.get('rank', 0),
+                'preview_url': track_data.get('preview'),
+                'external_urls': {'deezer': track_data['link']} if track_data.get('link') else {},
+                'track_number': track_data.get('track_position'),
+                'disc_number': track_data.get('disk_number', 1),
+                'explicit': bool(track_data.get('explicit_lyrics', False)),
+                '_source': 'deezer',
+            })
+
+        return tracks
+
     def get_artist_info(self, artist_id: str) -> Optional[Dict[str, Any]]:
         """Get full artist details — returns Spotify-compatible dict (metadata source interface).
 
@@ -640,17 +916,41 @@ class DeezerClient:
 
         Matches iTunesClient.get_artist_albums() interface.
         Paginates through all results up to the requested limit."""
+        cache = get_metadata_cache()
+        cached_items = get_cached_artist_album_items(cache, 'deezer', artist_id, album_type=album_type, limit=limit)
+        if cached_items:
+            try:
+                requested_types = [t.strip() for t in album_type.split(',')]
+                cached_albums = []
+                for album_data in cached_items:
+                    album = Album.from_deezer_album(album_data)
+                    if album_type != 'album,single':
+                        if album.album_type not in requested_types:
+                            if not (album.album_type == 'ep' and 'single' in requested_types):
+                                continue
+                    cached_albums.append(album)
+                return cached_albums[:limit]
+            except Exception as e:
+                logger.debug("Deezer artist albums cache reuse failed: %s", e)
+
         albums = []
         all_raw = []
         requested_types = [t.strip() for t in album_type.split(',')]
         offset = 0
         page_size = 100  # Deezer API max per request
+        complete = True  # cleared if pagination breaks on a transient/malformed error
 
         while offset < limit:
             fetch_limit = min(page_size, limit - offset)
             data = self._api_get(f'artist/{artist_id}/albums', {'limit': fetch_limit, 'index': offset})
-            if not data or 'data' not in data or len(data['data']) == 0:
+            if not data or 'data' not in data:
+                # Malformed/transient response mid-pagination — what we have is a
+                # PARTIAL discography. Don't cache it as the full list (mirrors the
+                # Spotify truncated-fetch guard). #853 follow-up.
+                complete = False
                 break
+            if len(data['data']) == 0:
+                break  # No more albums — a clean end of pagination.
 
             for album_data in data['data']:
                 all_raw.append(album_data)
@@ -667,7 +967,6 @@ class DeezerClient:
                 break  # Last page
             offset += len(data['data'])
 
-        cache = get_metadata_cache()
         # Deezer's /artist/{id}/albums endpoint doesn't include artist info on each album.
         # Inject it so cached album entities have artist_name for discover page display.
         artist_stub = None
@@ -681,6 +980,11 @@ class DeezerClient:
                 entries.append((str(ad['id']), ad))
         if entries:
             cache.store_entities_bulk('deezer', 'album', entries, skip_if_exists=True)
+        # Only cache the artist→album-LIST when pagination finished cleanly; a
+        # partial list would otherwise serve an incomplete discography until TTL.
+        # (Individual album entities above are complete, so they cache regardless.)
+        if complete:
+            store_artist_album_items(cache, 'deezer', artist_id, all_raw, album_type=album_type, limit=limit)
 
         logger.info(f"Retrieved {len(albums)} albums for artist {artist_id}")
         return albums[:limit]
@@ -842,8 +1146,8 @@ class DeezerClient:
                 try:
                     cache = get_metadata_cache()
                     cache.store_entity('deezer', 'artist', str(result.get('id', '')), result)
-                except Exception:
-                    pass
+                except Exception as e:
+                    logger.debug("cache store_entity artist search: %s", e)
                 logger.debug(f"Found artist for query: {artist_name}")
                 return result
 
@@ -887,8 +1191,8 @@ class DeezerClient:
                 try:
                     cache = get_metadata_cache()
                     cache.store_entity('deezer', 'album', str(result.get('id', '')), result)
-                except Exception:
-                    pass
+                except Exception as e:
+                    logger.debug("cache store_entity album search: %s", e)
                 logger.debug(f"Found album for query: {artist_name} - {album_title}")
                 return result
 
@@ -932,8 +1236,8 @@ class DeezerClient:
                 try:
                     cache = get_metadata_cache()
                     cache.store_entity('deezer', 'track', str(result.get('id', '')), result)
-                except Exception:
-                    pass
+                except Exception as e:
+                    logger.debug("cache store_entity track search: %s", e)
                 logger.debug(f"Found track for query: {artist_name} - {track_title}")
                 return result
 
@@ -965,8 +1269,8 @@ class DeezerClient:
                 # Cache hit with full details (has label = was a get_album response, not just search)
                 logger.debug(f"Cache hit for album {album_id}")
                 return cached
-        except Exception:
-            pass
+        except Exception as e:
+            logger.debug("cache get_entity album: %s", e)
 
         try:
             response = self.session.get(
@@ -984,8 +1288,8 @@ class DeezerClient:
             try:
                 cache = get_metadata_cache()
                 cache.store_entity('deezer', 'album', str(album_id), data)
-            except Exception:
-                pass
+            except Exception as e:
+                logger.debug("cache store_entity album full: %s", e)
             logger.debug(f"Got full album details for ID: {album_id}")
             return data
 
@@ -1013,8 +1317,8 @@ class DeezerClient:
             if cached and cached.get('bpm'):
                 logger.debug(f"Cache hit for track {track_id}")
                 return cached
-        except Exception:
-            pass
+        except Exception as e:
+            logger.debug("cache get_entity track: %s", e)
 
         try:
             response = self.session.get(
@@ -1032,8 +1336,8 @@ class DeezerClient:
             try:
                 cache = get_metadata_cache()
                 cache.store_entity('deezer', 'track', str(track_id), data)
-            except Exception:
-                pass
+            except Exception as e:
+                logger.debug("cache store_entity track full: %s", e)
             logger.debug(f"Got full track details for ID: {track_id}")
             return data
 
@@ -1095,6 +1399,16 @@ class DeezerClient:
 
                 raw_tracks.extend(page_tracks)
 
+            # Real album track positions — playlist tracks don't carry track_position,
+            # so numbering by playlist index would poison the downloaded file's tag.
+            album_ids = {str(t.get('album', {}).get('id')) for t in raw_tracks if t.get('album', {}).get('id')}
+            try:
+                from core.metadata.cache import get_metadata_cache
+                _cache = get_metadata_cache()
+            except Exception:
+                _cache = None
+            track_positions = resolve_album_track_positions(self.session, self.BASE_URL, album_ids, _cache)
+
             # Normalize tracks
             tracks: List[Dict[str, Any]] = []
             for i, t in enumerate(raw_tracks, start=1):
@@ -1106,7 +1420,8 @@ class DeezerClient:
                     'artists': [artist_name],
                     'album': t.get('album', {}).get('title', ''),
                     'duration_ms': t.get('duration', 0) * 1000,
-                    'track_number': i,
+                    # REAL album position; the playlist index is a last resort only.
+                    'track_number': track_positions.get(str(t.get('id'))) or i,
                 })
 
             result = {

core/deezer_download_client.py

@@ -20,7 +20,8 @@ from typing import Any, Dict, List, Optional, Tuple
 
 import requests
 
-from core.soulseek_client import AlbumResult, DownloadStatus, TrackResult
+from core.download_plugins.types import AlbumResult, DownloadStatus, TrackResult
+from core.quality.source_map import quality_from_deezer, quality_tier_for_source
 from utils.logging_config import get_logger
 
 logger = get_logger("deezer_download")
@@ -79,7 +80,10 @@ def _decrypt_chunk(chunk: bytes, key: bytes) -> bytes:
             ) from exc
 
 
-class DeezerDownloadClient:
+from core.download_plugins.base import DownloadSourcePlugin
+
+
+class DeezerDownloadClient(DownloadSourcePlugin):
     """Deezer download client using ARL token authentication."""
 
     def __init__(self, download_path: str = None):
@@ -89,11 +93,14 @@ class DeezerDownloadClient:
         if download_path is None:
             download_path = config_manager.get('soulseek.download_path', './downloads')
         self.download_path = Path(download_path)
-        self.download_path.mkdir(parents=True, exist_ok=True)
+        try:
+            self.download_path.mkdir(parents=True, exist_ok=True)
+        except OSError as e:
+            logger.warning(f"Could not verify download path {self.download_path}: {e}")
 
-        # Download tracking (same pattern as Tidal/Qobuz/HiFi)
-        self.active_downloads: Dict[str, Dict[str, Any]] = {}
-        self._download_lock = threading.Lock()
+        # Engine reference is populated by set_engine() at registration
+        # time. None until orchestrator wires the registry.
+        self._engine = None
 
         # Shutdown check callback (set by web_server)
         self.shutdown_check = None
@@ -114,17 +121,27 @@ class DeezerDownloadClient:
         self._license_token = None
         self._user_data = None
         self._authenticated = False
+        self._pending_arl: Optional[str] = None
 
         # Quality preference
-        self._quality = config_manager.get('deezer_download.quality', 'flac')
+        self._quality = quality_tier_for_source('deezer', default='flac')
 
         # Try to authenticate on init if ARL is configured
         arl = config_manager.get('deezer_download.arl', '')
         if arl:
-            self._authenticate(arl)
+            from core.boot_phase import is_boot_phase
+            if is_boot_phase():
+                self._pending_arl = arl
+                logger.debug("Deezer ARL present — authentication deferred until after boot")
+            else:
+                self._authenticate(arl)
 
         logger.info(f"Deezer download client initialized (download path: {self.download_path})")
 
+    def set_engine(self, engine):
+        """Engine callback — wires the central thread worker + state store."""
+        self._engine = engine
+
     # ─── Authentication ──────────────────────────────────────────
 
     def _authenticate(self, arl: str) -> bool:
@@ -216,12 +233,66 @@ class DeezerDownloadClient:
         return self._authenticated
 
     def is_authenticated(self) -> bool:
+        if self._pending_arl and not self._authenticated:
+            from core.boot_phase import is_boot_phase
+            if not is_boot_phase():
+                self._authenticate(self._pending_arl)
+                self._pending_arl = None
         return self._authenticated
 
     async def check_connection(self) -> bool:
         loop = asyncio.get_event_loop()
         return await loop.run_in_executor(None, self.is_available)
 
+    # ─── Playlist export (#945) ──────────────────────────────────
+    #
+    # UNOFFICIAL: rides the private gw-light gateway with the ARL session already used
+    # for downloads. Deezer shut their public developer API, so this is the only write
+    # path — and it's fragile by nature (breaks when Deezer changes internals).
+
+    def create_or_update_playlist(self, name, track_ids, *, existing_id=None,
+                                  public=False, description=""):
+        """Create a Deezer playlist (or append to an existing one) from a mirrored
+        playlist's tracks. ``track_ids`` are stored ``deezer_id`` values per library track.
+        ``existing_id`` set → add to that playlist (idempotent re-export reuses the stored
+        target); unset → create a new one. Returns
+        ``{success, playlist_id, url, added, error}``."""
+        if not self._authenticated:
+            return {"success": False, "error": "Deezer is not connected (ARL)"}
+        song_ids = [str(t) for t in (track_ids or []) if t]
+        if not song_ids:
+            return {"success": False, "error": "No matching Deezer tracks to export"}
+        try:
+            songs = [[sid, i] for i, sid in enumerate(song_ids)]
+            if existing_id:
+                res = self._gw_call("playlist.addSongs",
+                                    {"playlist_id": int(existing_id), "songs": songs})
+                if res is None:
+                    return {"success": False, "error": "Deezer rejected the playlist update"}
+                playlist_id = existing_id
+            else:
+                res = self._gw_call("playlist.create", {
+                    "title": name, "description": description,
+                    "is_public": bool(public), "songs": songs,
+                })
+                if res is None:
+                    return {"success": False, "error": "Deezer rejected the playlist create"}
+                # gw 'playlist.create' returns the new playlist id (int) as `results`.
+                if isinstance(res, dict):
+                    playlist_id = res.get("PLAYLIST_ID") or res.get("id")
+                else:
+                    playlist_id = res
+                if not playlist_id:
+                    return {"success": False, "error": "Deezer did not return a playlist id"}
+            return {
+                "success": True,
+                "playlist_id": str(playlist_id),
+                "url": f"https://www.deezer.com/playlist/{playlist_id}",
+                "added": len(song_ids),
+            }
+        except Exception as e:
+            return {"success": False, "error": str(e)}
+
     def reconnect(self, arl: str = None) -> bool:
         """Re-authenticate with a new or existing ARL."""
         if arl is None:
@@ -410,6 +481,12 @@ class DeezerDownloadClient:
                 if aid:
                     album_ids.add(str(aid))
             album_release_dates = {}
+            # Deezer PLAYLIST tracks do NOT carry `track_position` (only `/track/<id>`
+            # and `/album/<id>/tracks` do), so numbering them by their playlist index
+            # poisons the real album track number — which then rides into the wishlist
+            # and onto the downloaded file's tag (e.g. 'Apologize' tagged track 1 instead
+            # of 16). Resolve the REAL position from each album's track list (cache-first).
+            track_positions: Dict[str, int] = {}   # str(track_id) -> album track_position
             try:
                 from core.metadata.cache import get_metadata_cache
                 cache = get_metadata_cache()
@@ -422,24 +499,32 @@ class DeezerDownloadClient:
                         cached = cache.get_entity('deezer', 'album', aid)
                         if cached and cached.get('release_date'):
                             album_release_dates[aid] = cached['release_date']
-                            continue
-                    except Exception:
-                        pass
+                    except Exception as e:
+                        logger.debug("cache get_entity album release_date: %s", e)
                 # Cache miss — fetch from API
-                try:
-                    time.sleep(0.3)  # Respect rate limits
-                    a_resp = self._session.get(f'https://api.deezer.com/album/{aid}', timeout=10)
-                    if a_resp.ok:
-                        a_data = a_resp.json()
-                        album_release_dates[aid] = a_data.get('release_date', '')
-                        # Store in metadata cache for future use
-                        if cache:
-                            try:
-                                cache.store_entity('deezer', 'album', aid, a_data)
-                            except Exception:
-                                pass
-                except Exception:
-                    pass
+                if aid not in album_release_dates:
+                    try:
+                        time.sleep(0.3)  # Respect rate limits
+                        a_resp = self._session.get(f'https://api.deezer.com/album/{aid}', timeout=10)
+                        if a_resp.ok:
+                            a_data = a_resp.json()
+                            album_release_dates[aid] = a_data.get('release_date', '')
+                            # Store in metadata cache for future use
+                            if cache:
+                                try:
+                                    cache.store_entity('deezer', 'album', aid, a_data)
+                                except Exception as e:
+                                    logger.debug("cache store_entity album release_date: %s", e)
+                    except Exception as e:
+                        logger.debug("fetch deezer album release_date %s: %s", aid, e)
+            # Real album track positions (separate endpoint — playlist tracks AND the
+            # album object's embedded tracks both omit track_position). Cache-first.
+            try:
+                from core.deezer_client import resolve_album_track_positions
+                track_positions = resolve_album_track_positions(
+                    self._session, 'https://api.deezer.com', album_ids, cache)
+            except Exception as e:
+                logger.debug("resolve deezer album track positions: %s", e)
 
             tracks = []
             for i, t in enumerate(raw_tracks, start=1):
@@ -460,7 +545,9 @@ class DeezerDownloadClient:
                         'id': album_id,
                     },
                     'duration_ms': t.get('duration', 0) * 1000,
-                    'track_number': i,
+                    # REAL album position (resolved above); the playlist index is a last
+                    # resort only when the album lookup failed, never the default.
+                    'track_number': track_positions.get(str(t.get('id'))) or t.get('track_position') or i,
                 })
 
             return {
@@ -574,7 +661,7 @@ class DeezerDownloadClient:
                     bitrate = 128
                     quality = 'mp3'
 
-                results.append(TrackResult(
+                tr = TrackResult(
                     username='deezer_dl',
                     filename=f"{track_id}||{artist} - {title}",
                     size=est_size,
@@ -588,7 +675,10 @@ class DeezerDownloadClient:
                     title=title,
                     album=album,
                     track_number=item.get('track_position'),
-                ))
+                )
+                # Stamp CD-quality FLAC (16/44.1) so lossless ranks correctly.
+                tr.set_quality(quality_from_deezer(self._quality))
+                results.append(tr)
 
             logger.info(f"Deezer search for '{query}' returned {len(results)} results")
             return results, []
@@ -605,87 +695,67 @@ class DeezerDownloadClient:
         if not self._authenticated:
             logger.error("Deezer not authenticated — cannot download")
             return None
+        if self._engine is None:
+            # Raise rather than return None so the orchestrator's
+            # download_with_fallback surfaces a real warning + tries
+            # the next source. Returning None silently dropped the
+            # download with no user feedback (per JohnBaumb).
+            raise RuntimeError("Deezer client has no engine reference — cannot dispatch download")
 
         # Parse filename: "track_id||display_name"
         parts = filename.split('||', 1)
         track_id = parts[0]
         display_name = parts[1] if len(parts) > 1 else f"Track {track_id}"
 
-        download_id = str(uuid.uuid4())
-
-        with self._download_lock:
-            self.active_downloads[download_id] = {
-                'id': download_id,
+        return self._engine.worker.dispatch(
+            source_name='deezer',
+            target_id=track_id,
+            display_name=display_name,
+            original_filename=filename,
+            impl_callable=self._download_sync,
+            extra_record_fields={
                 'track_id': track_id,
                 'display_name': display_name,
-                'filename': filename,
-                'username': 'deezer_dl',
-                'state': 'Initializing',
-                'progress': 0.0,
                 'size': file_size,
-                'transferred': 0,
-                'speed': 0,
-                'file_path': None,
                 'error': None,
-            }
-
-        thread = threading.Thread(
-            target=self._download_thread_worker,
-            args=(download_id, track_id, display_name),
-            daemon=True,
-            name=f'deezer-dl-{track_id}'
+            },
+            # Legacy username slot — frontend status indicators key off
+            # ``deezer_dl``, not the canonical ``deezer``.
+            username_override='deezer_dl',
+            # Diagnostic thread name for multi-thread debugging.
+            thread_name=f'deezer-dl-{track_id}',
         )
-        thread.start()
 
-        logger.info(f"Started Deezer download {download_id}: {display_name}")
-        return download_id
+    def _set_error(self, download_id: str, message: str) -> None:
+        """Helper: set the engine record's `error` slot. No-op if
+        engine isn't wired or record was already removed."""
+        if self._engine is None:
+            return
+        self._engine.update_record('deezer', download_id, {'error': message})
 
-    def _download_thread_worker(self, download_id: str, track_id: str, display_name: str):
-        """Background worker for a single download."""
-        try:
-            result_path = self._download_sync(download_id, track_id, display_name)
-            with self._download_lock:
-                if download_id in self.active_downloads:
-                    dl = self.active_downloads[download_id]
-                    if dl['state'] == 'Cancelled':
-                        return
-                    if result_path:
-                        dl['state'] = 'Completed, Succeeded'
-                        dl['progress'] = 100.0
-                        dl['file_path'] = result_path
-                        logger.info(f"Deezer download {download_id} completed: {result_path}")
-                    else:
-                        dl['state'] = 'Errored'
-                        logger.error(f"Deezer download {download_id} failed: {dl.get('error', 'unknown')}")
-        except Exception as e:
-            logger.error(f"Deezer download thread error: {e}")
-            with self._download_lock:
-                if download_id in self.active_downloads:
-                    self.active_downloads[download_id]['state'] = 'Errored'
-                    self.active_downloads[download_id]['error'] = str(e)
+    def _is_cancelled(self, download_id: str) -> bool:
+        if self._engine is None:
+            return False
+        record = self._engine.get_record('deezer', download_id)
+        return record is not None and record.get('state') == 'Cancelled'
 
     def _download_sync(self, download_id: str, track_id: str, display_name: str) -> Optional[str]:
         """Synchronous download: get URL, download, decrypt, save."""
         # Check for shutdown
         if self.shutdown_check and self.shutdown_check():
-            with self._download_lock:
-                if download_id in self.active_downloads:
-                    self.active_downloads[download_id]['state'] = 'Aborted'
+            if self._engine is not None:
+                self._engine.update_record('deezer', download_id, {'state': 'Aborted'})
             return None
 
         # Get track data from private API
         track_data = self._get_track_data(track_id)
         if not track_data:
-            with self._download_lock:
-                if download_id in self.active_downloads:
-                    self.active_downloads[download_id]['error'] = 'Failed to get track data'
+            self._set_error(download_id, 'Failed to get track data')
             return None
 
         track_token = track_data.get('TRACK_TOKEN', '')
         if not track_token:
-            with self._download_lock:
-                if download_id in self.active_downloads:
-                    self.active_downloads[download_id]['error'] = 'No track token available'
+            self._set_error(download_id, 'No track token available')
             return None
 
         # Determine quality and get media URL with fallback
@@ -695,7 +765,6 @@ class DeezerDownloadClient:
 
         if allow_fallback:
             quality_order = _QUALITY_ORDER.copy()
-            # Start from user's preferred quality
             try:
                 pref_idx = quality_order.index(self._quality)
                 quality_order = quality_order[pref_idx:] + quality_order[:pref_idx]
@@ -712,27 +781,16 @@ class DeezerDownloadClient:
                 break
 
         if not media_url:
-            with self._download_lock:
-                if download_id in self.active_downloads:
-                    self.active_downloads[download_id]['error'] = 'No media URL available (may require higher subscription tier)'
+            self._set_error(download_id, 'No media URL available (may require higher subscription tier)')
             return None
 
         if actual_quality != self._quality:
             logger.info(f"Quality fallback: {self._quality} → {actual_quality} for {display_name}")
 
-        # Determine file extension
         ext = '.flac' if actual_quality == 'flac' else '.mp3'
-
-        # Sanitize filename
         safe_name = self._sanitize_filename(display_name)
         out_path = str(self.download_path / f"{safe_name}{ext}")
 
-        # Update state
-        with self._download_lock:
-            if download_id in self.active_downloads:
-                dl = self.active_downloads[download_id]
-                dl['state'] = 'InProgress, Downloading'
-
         # Download and decrypt
         try:
             bf_key = _get_blowfish_key(track_id)
@@ -740,9 +798,8 @@ class DeezerDownloadClient:
             resp.raise_for_status()
 
             total_size = int(resp.headers.get('content-length', 0))
-            with self._download_lock:
-                if download_id in self.active_downloads:
-                    self.active_downloads[download_id]['size'] = total_size
+            if self._engine is not None:
+                self._engine.update_record('deezer', download_id, {'size': total_size})
 
             downloaded = 0
             chunk_index = 0
@@ -755,23 +812,20 @@ class DeezerDownloadClient:
 
                     # Check for cancellation/shutdown
                     if self.shutdown_check and self.shutdown_check():
-                        with self._download_lock:
-                            if download_id in self.active_downloads:
-                                self.active_downloads[download_id]['state'] = 'Aborted'
+                        if self._engine is not None:
+                            self._engine.update_record('deezer', download_id, {'state': 'Aborted'})
                         try:
                             os.remove(out_path)
                         except OSError:
                             pass
                         return None
 
-                    with self._download_lock:
-                        if download_id in self.active_downloads:
-                            if self.active_downloads[download_id]['state'] == 'Cancelled':
-                                try:
-                                    os.remove(out_path)
-                                except OSError:
-                                    pass
-                                return None
+                    if self._is_cancelled(download_id):
+                        try:
+                            os.remove(out_path)
+                        except OSError:
+                            pass
+                        return None
 
                     # Decrypt every 3rd chunk (Deezer's encryption pattern)
                     if chunk_index % 3 == 0 and len(raw_chunk) == _CHUNK_SIZE:
@@ -788,12 +842,12 @@ class DeezerDownloadClient:
                     speed = int(downloaded / elapsed) if elapsed > 0 else 0
                     progress = (downloaded / total_size * 100) if total_size > 0 else 0
 
-                    with self._download_lock:
-                        if download_id in self.active_downloads:
-                            dl = self.active_downloads[download_id]
-                            dl['transferred'] = downloaded
-                            dl['progress'] = min(progress, 99.9)
-                            dl['speed'] = speed
+                    if self._engine is not None:
+                        self._engine.update_record('deezer', download_id, {
+                            'transferred': downloaded,
+                            'progress': min(progress, 99.9),
+                            'speed': speed,
+                        })
 
             # Validate file size
             file_size = os.path.getsize(out_path)
@@ -803,9 +857,7 @@ class DeezerDownloadClient:
                     os.remove(out_path)
                 except OSError:
                     pass
-                with self._download_lock:
-                    if download_id in self.active_downloads:
-                        self.active_downloads[download_id]['error'] = f'File too small ({file_size} bytes)'
+                self._set_error(download_id, f'File too small ({file_size} bytes)')
                 return None
 
             logger.info(f"Deezer download complete: {out_path} ({file_size / 1048576:.1f} MB, {actual_quality})")
@@ -817,59 +869,58 @@ class DeezerDownloadClient:
                 os.remove(out_path)
             except OSError:
                 pass
-            with self._download_lock:
-                if download_id in self.active_downloads:
-                    self.active_downloads[download_id]['error'] = str(e)
+            self._set_error(download_id, str(e))
             return None
 
     # ─── Download Status ─────────────────────────────────────────
 
+    def _record_to_status(self, record: dict) -> DownloadStatus:
+        return DownloadStatus(
+            id=record['id'],
+            filename=record['filename'],
+            username=record['username'],
+            state=record['state'],
+            progress=record['progress'],
+            size=record.get('size', 0),
+            transferred=record.get('transferred', 0),
+            speed=record.get('speed', 0),
+            file_path=record.get('file_path'),
+        )
+
     async def get_all_downloads(self) -> List[DownloadStatus]:
-        """Return all active downloads."""
-        with self._download_lock:
-            return [self._to_status(dl) for dl in self.active_downloads.values()]
+        if self._engine is None:
+            return []
+        return [
+            self._record_to_status(record)
+            for record in self._engine.iter_records_for_source('deezer')
+        ]
 
     async def get_download_status(self, download_id: str) -> Optional[DownloadStatus]:
-        """Get status of a specific download."""
-        with self._download_lock:
-            dl = self.active_downloads.get(download_id)
-            return self._to_status(dl) if dl else None
+        if self._engine is None:
+            return None
+        record = self._engine.get_record('deezer', download_id)
+        return self._record_to_status(record) if record is not None else None
 
     async def cancel_download(self, download_id: str, username: str = None,
                               remove: bool = False) -> bool:
-        """Cancel a download."""
-        with self._download_lock:
-            dl = self.active_downloads.get(download_id)
-            if not dl:
-                return False
-            dl['state'] = 'Cancelled'
-            if remove:
-                del self.active_downloads[download_id]
+        if self._engine is None:
+            return False
+        if self._engine.get_record('deezer', download_id) is None:
+            return False
+        self._engine.update_record('deezer', download_id, {'state': 'Cancelled'})
+        if remove:
+            self._engine.remove_record('deezer', download_id)
         return True
 
     async def clear_all_completed_downloads(self) -> bool:
-        """Remove all terminal downloads."""
-        terminal_states = {'Completed, Succeeded', 'Cancelled', 'Errored', 'Aborted'}
-        with self._download_lock:
-            to_remove = [k for k, v in self.active_downloads.items() if v['state'] in terminal_states]
-            for k in to_remove:
-                del self.active_downloads[k]
+        if self._engine is None:
+            return True
+        terminal = {'Completed, Succeeded', 'Cancelled', 'Errored', 'Aborted'}
+        for record in list(self._engine.iter_records_for_source('deezer')):
+            if record.get('state') in terminal:
+                self._engine.remove_record('deezer', record['id'])
         return True
 
-    def _to_status(self, dl: dict) -> DownloadStatus:
-        """Convert internal dict to DownloadStatus."""
-        return DownloadStatus(
-            id=dl['id'],
-            filename=dl['filename'],
-            username=dl['username'],
-            state=dl['state'],
-            progress=dl['progress'],
-            size=dl['size'],
-            transferred=dl['transferred'],
-            speed=dl['speed'],
-            file_path=dl.get('file_path'),
-        )
-
     # ─── Utilities ───────────────────────────────────────────────
 
     @staticmethod

core/deezer_worker.py

@@ -8,7 +8,16 @@ from datetime import datetime, timedelta
 from utils.logging_config import get_logger
 from database.music_database import MusicDatabase
 from core.deezer_client import DeezerClient
-from core.worker_utils import interruptible_sleep, set_album_api_track_count
+from core.worker_utils import (
+    accept_artist_match,
+    artist_name_matches,
+    interruptible_sleep,
+    owned_album_titles,
+    pick_artist_by_catalog,
+    release_titles,
+    set_album_api_track_count,
+)
+from core.enrichment.manual_match_honoring import honor_stored_match
 
 logger = get_logger("deezer_worker")
 
@@ -140,8 +149,8 @@ class DeezerWorker:
                         itype = item.get('type', '')
                         table = 'artists' if 'artist' in itype else ('albums' if 'album' in itype else 'tracks')
                         # Can't mark status without an ID — just skip
-                    except Exception:
-                        pass
+                    except Exception as e:
+                        logger.debug("null id table resolve failed: %s", e)
                     continue
 
 
@@ -162,6 +171,16 @@ class DeezerWorker:
             conn = self.db._get_connection()
             cursor = conn.cursor()
 
+            # Pinned-group override (Manage Enrichment Workers): process one
+            # entity type first, then fall through to the normal chain. Unset or
+            # exhausted ⇒ default artist→album→track order, unchanged.
+            from core.worker_utils import read_enrichment_priority, priority_pending_item
+            _prio = read_enrichment_priority('deezer')
+            if _prio:
+                _pi = priority_pending_item(cursor, 'deezer', _prio)
+                if _pi:
+                    return _pi
+
             # Priority 1: Unattempted artists
             cursor.execute("""
                 SELECT id, name
@@ -267,10 +286,19 @@ class DeezerWorker:
         logger.debug(f"Name similarity: '{query_name}' vs '{result_name}' = {similarity:.2f}")
         return similarity >= self.name_similarity_threshold
 
-    def _verify_artist_id(self, item: Dict[str, Any], result_artist_id) -> bool:
+    def _verify_artist_id(self, item: Dict[str, Any], result_artist_id,
+                          result_artist_name: Optional[str] = None) -> bool:
         """Verify that the result's artist ID matches the parent artist's stored Deezer ID.
+
         If mismatched, the album/track search is more specific (uses artist+title),
-        so we trust it and correct the parent artist's deezer_id."""
+        so we trust it and correct the parent artist's deezer_id — BUT only when
+        the result's artist *name* actually matches our parent artist. Without
+        that guard, a collaboration or compilation track (e.g. a track our
+        library credits to Jorja Smith that lives on Kendrick Lamar's curated
+        "Black Panther" album) would search up to an album whose Deezer primary
+        artist is someone else (Kendrick), and we'd stamp that wrong Deezer ID
+        onto our artist — corrupting it (and causing duplicate ids shared across
+        unrelated artists)."""
         parent_deezer_id = item.get('artist_deezer_id')
         if not parent_deezer_id:
             return True
@@ -279,6 +307,20 @@ class DeezerWorker:
             return True
 
         if str(result_artist_id) != str(parent_deezer_id):
+            # Guard: only correct when the album/track's primary artist is the
+            # SAME artist by name. A mismatch means it's a collab/compilation,
+            # not a stale-id correction.
+            parent_name = item.get('artist') or ''
+            if (result_artist_name and parent_name
+                    and not self._name_matches(parent_name, result_artist_name)):
+                logger.info(
+                    f"Skipping artist-ID correction from {item['type']} "
+                    f"'{item['name']}': result artist '{result_artist_name}' "
+                    f"≠ parent '{parent_name}' (collab/compilation, not a "
+                    f"correction)"
+                )
+                return True
+
             logger.info(
                 f"Artist ID correction from {item['type']} '{item['name']}': "
                 f"updating parent artist Deezer ID from {parent_deezer_id} to {result_artist_id}"
@@ -367,27 +409,66 @@ class DeezerWorker:
             logger.debug(f"Preserving existing Deezer ID for artist '{artist_name}': {existing_id}")
             return
 
-        result = self.client.search_artist(artist_name)
+        # Multi-candidate search (was single search_artist) so same-name artists
+        # can be disambiguated: gate by name, then pick the one whose catalog
+        # overlaps the albums this library owns.
+        results = self.client.search_artists(artist_name, limit=5)
+        gated = [a for a in (results or []) if artist_name_matches(artist_name, getattr(a, 'name', ''))]
+        chosen, _overlap = pick_artist_by_catalog(
+            gated,
+            owned_album_titles(self.db, artist_id),
+            lambda a: release_titles(self.client.get_artist_albums_list(a.id)),
+        )
+
+        # search_artists returns lean Artist objects; fetch the full dict (same
+        # shape the old search_artist returned) for storage.
+        result = self.client.get_artist_info(chosen.id) if chosen else None
         if result:
             result_name = result.get('name', '')
-            if self._name_matches(artist_name, result_name):
+            ok, reason = accept_artist_match(
+                self.db, 'deezer_id', result.get('id'), artist_id,
+                artist_name, result_name,
+            )
+            if ok:
                 self._update_artist(artist_id, result)
                 self.stats['matched'] += 1
                 logger.info(f"Matched artist '{artist_name}' -> Deezer ID: {result.get('id')}")
             else:
                 self._mark_status('artist', artist_id, 'not_found')
                 self.stats['not_found'] += 1
-                logger.debug(f"Name mismatch for artist '{artist_name}' (got '{result_name}')")
+                logger.debug(f"Artist '{artist_name}' not matched: {reason}")
         else:
             self._mark_status('artist', artist_id, 'not_found')
             self.stats['not_found'] += 1
             logger.debug(f"No match for artist '{artist_name}'")
 
+    def _refresh_album_via_stored_id(self, album_id, stored_id, full_album_dict):
+        """Issue #501 callback. Stored ID exists → fetched full Deezer
+        album payload. Use it as both args to ``_update_album`` (search-
+        result and full-data shapes overlap on the fields we need —
+        artist verification skipped since manual match presumably
+        already vetted)."""
+        self._update_album(album_id, full_album_dict, full_album_dict)
+
+    def _refresh_track_via_stored_id(self, track_id, stored_id, full_track_dict):
+        """Issue #501 callback for tracks — same pattern as albums."""
+        self._update_track(track_id, full_track_dict, full_track_dict)
+
     def _process_album(self, album_id: int, album_name: str, artist_name: str, item: Dict[str, Any]):
         """Process an album: search Deezer, verify, fetch full details, store metadata"""
-        existing_id = self._get_existing_id('album', album_id)
-        if existing_id:
-            logger.debug(f"Preserving existing Deezer ID for album '{album_name}': {existing_id}")
+        # Issue #501: honor manual matches. Pre-fix this method just
+        # SKIPPED when a stored ID was present (preserved the ID but
+        # never refreshed metadata). Now it goes through the full
+        # refresh path via the stored ID, picking up label / genres /
+        # explicit updates without ever overwriting the manual match.
+        if honor_stored_match(
+            db=self.db, entity_table='albums', entity_id=album_id,
+            id_column='deezer_id',
+            client_fetch_fn=self.client.get_album_raw,
+            on_match_fn=self._refresh_album_via_stored_id,
+            log_prefix='Deezer',
+        ):
+            self.stats['matched'] += 1
             return
 
         result = self.client.search_album(artist_name, album_name)
@@ -397,7 +478,8 @@ class DeezerWorker:
                 # Verify artist ID
                 result_artist = result.get('artist', {})
                 result_artist_id = result_artist.get('id') if result_artist else None
-                self._verify_artist_id(item, result_artist_id)
+                result_artist_name = result_artist.get('name') if result_artist else None
+                self._verify_artist_id(item, result_artist_id, result_artist_name)
 
                 # Fetch full album details for label, genres, explicit
                 deezer_album_id = result.get('id')
@@ -430,9 +512,15 @@ class DeezerWorker:
 
     def _process_track(self, track_id: int, track_name: str, artist_name: str, item: Dict[str, Any]):
         """Process a track: search Deezer, verify, fetch full details for BPM, store metadata"""
-        existing_id = self._get_existing_id('track', track_id)
-        if existing_id:
-            logger.debug(f"Preserving existing Deezer ID for track '{track_name}': {existing_id}")
+        # Issue #501: honor manual matches (see _process_album).
+        if honor_stored_match(
+            db=self.db, entity_table='tracks', entity_id=track_id,
+            id_column='deezer_id',
+            client_fetch_fn=self.client.get_track_raw,
+            on_match_fn=self._refresh_track_via_stored_id,
+            log_prefix='Deezer',
+        ):
+            self.stats['matched'] += 1
             return
 
         result = self.client.search_track(artist_name, track_name)
@@ -442,7 +530,8 @@ class DeezerWorker:
                 # Verify artist ID
                 result_artist = result.get('artist', {})
                 result_artist_id = result_artist.get('id') if result_artist else None
-                self._verify_artist_id(item, result_artist_id)
+                result_artist_name = result_artist.get('name') if result_artist else None
+                self._verify_artist_id(item, result_artist_id, result_artist_name)
 
                 # Fetch full track details for BPM
                 deezer_track_id = result.get('id')

core/diagnostics/memory_tracker.py

@@ -0,0 +1,136 @@
+"""On-demand memory-growth diagnostic (issue #802: ~0.7 MiB/s RSS growth).
+
+Wraps ``tracemalloc`` so a user seeing runaway memory can capture WHERE the
+allocations come from instead of us guessing:
+
+    1. start_tracking()  — begins tracing + stores a baseline snapshot
+    2. ...reproduce the growth for a few minutes...
+    3. report()          — top allocation sites, with the DELTA since baseline
+                           (the delta is the leak; absolute sizes are mostly
+                           startup noise)
+    4. stop_tracking()   — ends tracing, frees trace memory
+
+Opt-in by design: tracemalloc costs CPU and memory while active (it shadows
+every allocation), so it must never run by default. The Flask endpoints that
+expose this live in web_server (GET /api/debug/memory/...) so a user can drive
+the whole flow from a browser.
+"""
+
+from __future__ import annotations
+
+import os
+import time
+import tracemalloc
+from typing import Any, Dict, List, Optional
+
+from utils.logging_config import get_logger
+
+logger = get_logger("diagnostics.memory")
+
+_baseline: Optional[tracemalloc.Snapshot] = None
+_started_at: Optional[float] = None
+
+# Allocation-site traces this deep give useful "who called it" context without
+# pathological overhead.
+_TRACE_FRAMES = 15
+
+
+def is_tracking() -> bool:
+    return tracemalloc.is_tracing()
+
+
+def start_tracking() -> Dict[str, Any]:
+    """Begin tracing and store the baseline snapshot. Idempotent."""
+    global _baseline, _started_at
+    if tracemalloc.is_tracing():
+        return {"tracking": True, "already_running": True, "started_at": _started_at}
+    tracemalloc.start(_TRACE_FRAMES)
+    _baseline = tracemalloc.take_snapshot()
+    _started_at = time.time()
+    logger.info("Memory tracking started (tracemalloc, %d frames)", _TRACE_FRAMES)
+    return {"tracking": True, "already_running": False, "started_at": _started_at}
+
+
+def stop_tracking() -> Dict[str, Any]:
+    """End tracing and free the trace bookkeeping."""
+    global _baseline, _started_at
+    was = tracemalloc.is_tracing()
+    if was:
+        tracemalloc.stop()
+        logger.info("Memory tracking stopped")
+    _baseline = None
+    _started_at = None
+    return {"tracking": False, "was_tracking": was}
+
+
+def _rss_mb() -> Optional[float]:
+    """Process RSS in MiB, best-effort (psutil, then /proc fallback)."""
+    try:
+        import psutil
+        return round(psutil.Process(os.getpid()).memory_info().rss / (1024 * 1024), 1)
+    except Exception:  # noqa: S110 — RSS is optional context; fall through to /proc
+        pass
+    try:
+        with open("/proc/self/status", encoding="utf-8") as fh:
+            for line in fh:
+                if line.startswith("VmRSS:"):
+                    return round(int(line.split()[1]) / 1024, 1)
+    except Exception:  # noqa: S110 — no /proc on this platform; RSS stays None
+        pass
+    return None
+
+
+def format_stat(stat: Any) -> Dict[str, Any]:
+    """Project one tracemalloc StatisticDiff/Statistic into a plain dict.
+    Duck-typed (reads size/count/size_diff/count_diff/traceback) so it's
+    unit-testable without real snapshots."""
+    tb = getattr(stat, "traceback", None)
+    frames: List[str] = []
+    if tb:
+        # Most-recent-call-last reads naturally top-down in a report.
+        for frame in list(tb)[-3:]:
+            frames.append(f"{frame.filename}:{frame.lineno}")
+    return {
+        "location": frames[-1] if frames else "?",
+        "trace": frames,
+        "size_mb": round(getattr(stat, "size", 0) / (1024 * 1024), 3),
+        "size_diff_mb": round(getattr(stat, "size_diff", 0) / (1024 * 1024), 3),
+        "count": getattr(stat, "count", 0),
+        "count_diff": getattr(stat, "count_diff", 0),
+    }
+
+
+def report(top: int = 25) -> Dict[str, Any]:
+    """Current snapshot vs the start_tracking() baseline: the top allocation
+    sites by GROWTH (size_diff). Includes traced totals + process RSS so the
+    user can see how much of the real growth tracemalloc accounts for."""
+    if not tracemalloc.is_tracing():
+        return {
+            "tracking": False,
+            "rss_mb": _rss_mb(),
+            "hint": "Start with /api/debug/memory/start, reproduce the growth "
+                    "for a few minutes, then call this again.",
+        }
+    snapshot = tracemalloc.take_snapshot()
+    # Filter the tracer's own bookkeeping out of the picture.
+    snapshot = snapshot.filter_traces((
+        tracemalloc.Filter(False, tracemalloc.__file__),
+        tracemalloc.Filter(False, "<frozen importlib._bootstrap>"),
+    ))
+    current, peak = tracemalloc.get_traced_memory()
+
+    if _baseline is not None:
+        stats = snapshot.compare_to(_baseline, "traceback")
+        stats.sort(key=lambda s: s.size_diff, reverse=True)
+    else:
+        stats = snapshot.statistics("traceback")
+
+    return {
+        "tracking": True,
+        "started_at": _started_at,
+        "elapsed_seconds": round(time.time() - _started_at, 1) if _started_at else None,
+        "traced_current_mb": round(current / (1024 * 1024), 1),
+        "traced_peak_mb": round(peak / (1024 * 1024), 1),
+        "rss_mb": _rss_mb(),
+        "top_growth": [format_stat(s) for s in stats[:top]],
+    }

core/discogs_client.py

@@ -12,6 +12,7 @@ import re
 import time
 import threading
 import requests
+from core.metadata.artist_album_cache import get_cached_artist_album_payload, store_artist_album_items
 from core.metadata.cache import get_metadata_cache
 from typing import List, Dict, Any, Optional
 from dataclasses import dataclass
@@ -61,6 +62,69 @@ def rate_limited(func):
     return wrapper
 
 
+# Discogs disambiguates duplicate artist names two ways: numeric `(N)`
+# suffix on the older convention (e.g. "Bullet (2)") and a trailing `*`
+# on the newer convention (e.g. "John Smith*"). Both are presentation-
+# only — the underlying canonical name is what users expect to see in
+# search results, on cards, and especially in import paths (otherwise
+# library folders end up named `Foo*` on disk). Strip both at every
+# point a Discogs payload becomes a name string.
+_DISCOGS_DISAMBIG_RE = re.compile(r'(?:\s*\(\d+\))?\s*\*+\s*$|\s*\(\d+\)\s*$')
+
+
+def _clean_discogs_artist_name(name: Optional[str]) -> str:
+    """Strip Discogs disambiguation suffixes — both `(N)` and trailing
+    `*` — from an artist name. Returns '' for None / empty input."""
+    if not name:
+        return ''
+    return _DISCOGS_DISAMBIG_RE.sub('', name).strip()
+
+
+# --- Discogs album ID typing -------------------------------------------------
+# Discogs has two album object types — masters (/masters/{id}) and releases
+# (/releases/{id}) — whose numeric IDs share one space, so release N and master
+# N are DIFFERENT albums. A bare numeric ID is therefore ambiguous. We tag the
+# type into the ID string ('m12345' / 'r12345') at the point we parse it, so the
+# correct endpoint can be chosen later without guessing. (Artist IDs are a single
+# namespace and stay untagged.)
+
+def _discogs_album_kind(data: Dict[str, Any]) -> str:
+    """Classify a Discogs album payload as 'master' or 'release'.
+
+    Search results and artist-discography items carry an explicit ``type``;
+    full detail responses don't, but only master detail has ``main_release``."""
+    t = (data.get('type') or '').lower()
+    if t in ('master', 'release'):
+        return t
+    return 'master' if 'main_release' in data else 'release'
+
+
+def _tag_discogs_album_id(raw_id: Any, kind: str) -> str:
+    """``'12345'`` + ``'master'`` -> ``'m12345'``; empty input -> ``''``."""
+    s = str(raw_id or '').strip()
+    if not s:
+        return ''
+    return f"{'m' if kind == 'master' else 'r'}{s}"
+
+
+def _discogs_album_endpoints(album_id: Any) -> List[str]:
+    """Map a (possibly tagged) album ID to the API path(s) to try, in order.
+
+      ``'m12345'`` -> ``['/masters/12345']``
+      ``'r12345'`` -> ``['/releases/12345']``
+      ``'12345'``  (legacy untagged) -> ``['/releases/12345', '/masters/12345']``
+
+    Legacy bare IDs are tried release-first because stored IDs originate
+    overwhelmingly from search / manual-match / collection sync (all releases);
+    this also self-heals pre-fix bad matches. Returns ``[]`` for unusable input."""
+    s = str(album_id or '').strip()
+    if len(s) > 1 and s[0] in ('m', 'r') and s[1:].isdigit():
+        return [f"/{'masters' if s[0] == 'm' else 'releases'}/{s[1:]}"]
+    if s.isdigit():
+        return [f'/releases/{s}', f'/masters/{s}']
+    return []
+
+
 # --- Shared dataclasses (same shape as iTunes/Deezer/Spotify) ---
 
 @dataclass
@@ -117,9 +181,10 @@ class Track:
         # Artists from track-level or release-level
         track_artists = []
         if track_data.get('artists'):
-            track_artists = [a.get('name', '') for a in track_data['artists'] if a.get('name')]
+            track_artists = [_clean_discogs_artist_name(a.get('name', '')) for a in track_data['artists'] if a.get('name')]
         if not track_artists and release.get('artists'):
-            track_artists = [a.get('name', '') for a in release['artists'] if a.get('name')]
+            track_artists = [_clean_discogs_artist_name(a.get('name', '')) for a in release['artists'] if a.get('name')]
+        track_artists = [a for a in track_artists if a]
         if not track_artists:
             track_artists = ['Unknown Artist']
 
@@ -190,7 +255,9 @@ class Artist:
 
         return cls(
             id=str(artist_data.get('id', '')),
-            name=artist_data.get('name', artist_data.get('title', '')),
+            name=_clean_discogs_artist_name(
+                artist_data.get('name', artist_data.get('title', ''))
+            ),
             popularity=0,
             genres=[],
             followers=0,
@@ -216,14 +283,15 @@ class Album:
         artists = []
         title = release_data.get('title', '')
         if release_data.get('artists'):
-            artists = [a.get('name', '') for a in release_data['artists'] if a.get('name')]
+            artists = [_clean_discogs_artist_name(a.get('name', '')) for a in release_data['artists'] if a.get('name')]
         elif release_data.get('artist'):
-            artists = [release_data['artist']]
+            artists = [_clean_discogs_artist_name(release_data['artist'])]
         elif ' - ' in title:
             # Search results: "Radiohead - OK Computer" → artist="Radiohead", title="OK Computer"
             parts = title.split(' - ', 1)
-            artists = [parts[0].strip()]
+            artists = [_clean_discogs_artist_name(parts[0])]
             title = parts[1].strip()
+        artists = [a for a in artists if a]
         if not artists:
             artists = ['Unknown Artist']
 
@@ -282,7 +350,7 @@ class Album:
             external_urls['discogs_api'] = release_data['resource_url']
 
         return cls(
-            id=str(release_data.get('id', '')),
+            id=_tag_discogs_album_id(release_data.get('id', ''), _discogs_album_kind(release_data)),
             name=title,
             artists=artists,
             release_date=release_date,
@@ -320,8 +388,8 @@ class DiscogsClient:
             try:
                 from config.settings import config_manager
                 self.token = config_manager.get('discogs.token', '')
-            except Exception:
-                pass
+            except Exception as e:
+                logger.debug("load discogs.token from config: %s", e)
 
         if self.token:
             self.session.headers['Authorization'] = f'Discogs token={self.token}'
@@ -369,6 +437,125 @@ class DiscogsClient:
             logger.error(f"Discogs API error ({endpoint}): {e}")
             return None
 
+    # --- User Collection (powers Your Albums Discogs source) ---
+
+    def get_authenticated_username(self) -> Optional[str]:
+        """Resolve the username for the configured personal token.
+
+        Discogs's `/oauth/identity` endpoint returns the user's
+        username when called with a valid token. Cached on the
+        instance so subsequent calls don't re-hit the API.
+        """
+        if hasattr(self, '_cached_username'):
+            return self._cached_username
+        if not self.is_authenticated():
+            self._cached_username = None
+            return None
+        data = self._api_get('/oauth/identity')
+        username = data.get('username') if data else None
+        self._cached_username = username
+        return username
+
+    def get_user_collection(self, username: Optional[str] = None,
+                            folder_id: int = 0,
+                            per_page: int = 100,
+                            max_pages: int = 50) -> List[Dict[str, Any]]:
+        """Fetch a Discogs user's collection (folder 0 = "All").
+
+        Returns a list of normalized release dicts ready for
+        ``database.upsert_liked_album``:
+            {
+                'album_name': str,
+                'artist_name': str,
+                'release_id': int,        # Discogs release id
+                'image_url': str | None,
+                'release_date': str,      # 'YYYY' (Discogs only stores year)
+                'total_tracks': int,
+            }
+
+        Pagination caps at ``max_pages`` to bound runtime — at 100/page
+        that's 5000 releases, more than enough for typical collections.
+        Authenticated calls only (Discogs collection is private).
+        """
+        if not self.is_authenticated():
+            logger.warning("Discogs collection fetch attempted without token")
+            return []
+
+        if not username:
+            username = self.get_authenticated_username()
+            if not username:
+                logger.warning("Could not resolve Discogs username for token")
+                return []
+
+        results: List[Dict[str, Any]] = []
+        page = 1
+        while page <= max_pages:
+            data = self._api_get(
+                f'/users/{username}/collection/folders/{folder_id}/releases',
+                {'page': page, 'per_page': per_page, 'sort': 'added', 'sort_order': 'desc'},
+            )
+            if not data:
+                break
+
+            releases = data.get('releases', []) or []
+            if not releases:
+                break
+
+            for entry in releases:
+                info = entry.get('basic_information') or {}
+                release_id = entry.get('id') or info.get('id')
+                if not release_id:
+                    continue
+                title = info.get('title') or ''
+                # Discogs `artists` is a list of {name, id, ...}; first is primary.
+                artists = info.get('artists') or []
+                artist_name = ''
+                if artists and isinstance(artists[0], dict):
+                    artist_name = (artists[0].get('name') or '').strip()
+                # Strip Discogs disambiguation suffixes — both `(N)` and `*`.
+                artist_name = _clean_discogs_artist_name(artist_name)
+                if not title or not artist_name:
+                    continue
+
+                # Image URLs: cover_image is the primary, also has thumb.
+                image_url = (info.get('cover_image')
+                             or info.get('thumb')
+                             or '')
+
+                year = info.get('year')
+                release_date = str(year) if year and year > 0 else ''
+
+                results.append({
+                    'album_name': title.strip(),
+                    'artist_name': artist_name,
+                    'release_id': int(release_id),
+                    'image_url': image_url or None,
+                    'release_date': release_date,
+                    'total_tracks': 0,  # Not in basic_information; populated via get_release if needed
+                })
+
+            pagination = data.get('pagination') or {}
+            if page >= int(pagination.get('pages') or 1):
+                break
+            page += 1
+
+        logger.info(f"Discogs collection: fetched {len(results)} releases for {username}")
+        return results
+
+    def get_release(self, release_id: int) -> Optional[Dict[str, Any]]:
+        """Fetch full Discogs release detail including tracklist.
+
+        Returns the raw API response so callers can render rich
+        Discogs context (year, format, label, country, tracklist).
+        """
+        if not release_id:
+            return None
+        try:
+            release_id = int(release_id)
+        except (TypeError, ValueError):
+            return None
+        return self._api_get(f'/releases/{release_id}')
+
     # --- Search Methods (same signatures as iTunes/Deezer) ---
 
     def search_artists(self, query: str, limit: int = 10) -> List[Artist]:
@@ -380,8 +567,8 @@ class DiscogsClient:
             for raw in cached_results:
                 try:
                     artists.append(Artist.from_discogs_artist(raw))
-                except Exception:
-                    pass
+                except Exception as e:
+                    logger.debug("Artist.from_discogs_artist cache parse: %s", e)
             if artists:
                 return artists
 
@@ -417,8 +604,8 @@ class DiscogsClient:
             for raw in cached_results:
                 try:
                     albums.append(Album.from_discogs_release(raw))
-                except Exception:
-                    pass
+                except Exception as e:
+                    logger.debug("Album.from_discogs_release cache parse: %s", e)
             if albums:
                 return albums
 
@@ -502,10 +689,13 @@ class DiscogsClient:
         if cached and cached.get('title'):
             data = cached
         else:
-            # Try as master first (artist discography returns master IDs)
-            data = self._api_get(f'/masters/{release_id}')
-            if not data or not data.get('title'):
-                data = self._api_get(f'/releases/{release_id}')
+            # Hit the endpoint that matches the ID's type (tag-driven, no guessing).
+            data = None
+            for path in _discogs_album_endpoints(release_id):
+                data = self._api_get(path)
+                if data and data.get('title'):
+                    break
+                data = None
             if not data:
                 return None
             cache.store_entity('discogs', 'album', release_id, data)
@@ -539,22 +729,45 @@ class DiscogsClient:
 
     def get_artist_albums(self, artist_id: str, album_type: str = 'album,single', limit: int = 50) -> List[Album]:
         """Get releases by an artist. Prefers master releases, filters features."""
-        # First get the artist name for feature filtering
-        artist_data = self._api_get(f'/artists/{artist_id}')
-        artist_name = artist_data.get('name', '').lower() if artist_data else ''
+        cache = get_metadata_cache()
+        cached_payload = get_cached_artist_album_payload(cache, 'discogs', artist_id, album_type=album_type, limit=limit)
+        releases = cached_payload.get('_releases') if cached_payload else None
+        artist_name = ''
+        if cached_payload:
+            artist_name = str(cached_payload.get('artist_name') or '').lower()
 
-        data = self._api_get(f'/artists/{artist_id}/releases', {
-            'sort': 'year', 'sort_order': 'desc', 'per_page': min(limit * 3, 200),
-        })
-        if not data or not data.get('releases'):
-            return []
+        if not isinstance(releases, list) or not releases:
+            # First get the artist name for feature filtering. Strip Discogs
+            # disambiguation suffix so feature-vs-primary matching below
+            # compares against the canonical name, not "Beyoncé*".
+            artist_data = self._api_get(f'/artists/{artist_id}')
+            artist_name = _clean_discogs_artist_name(
+                artist_data.get('name', '') if artist_data else ''
+            ).lower()
+
+            data = self._api_get(f'/artists/{artist_id}/releases', {
+                'sort': 'year', 'sort_order': 'desc', 'per_page': min(limit * 3, 200),
+            })
+            if not data or not data.get('releases'):
+                return []
+            releases = data.get('releases') or []
+            store_artist_album_items(
+                cache,
+                'discogs',
+                artist_id,
+                releases,
+                album_type=album_type,
+                limit=limit,
+                items_field='_releases',
+                extra_fields={'artist_name': artist_name},
+            )
 
         # Separate masters from individual releases — prefer masters (canonical versions)
         masters = []
         releases_no_master = []
         master_titles = set()
 
-        for item in data['releases']:
+        for item in releases:
             # Skip non-main roles
             role = item.get('role', 'Main').lower()
             if role not in ('main', ''):
@@ -622,10 +835,13 @@ class DiscogsClient:
         if cached:
             return cached
 
-        # Try as master first (master IDs are used in artist discography)
-        data = self._api_get(f'/masters/{release_id}')
-        if not data or not data.get('tracklist'):
-            data = self._api_get(f'/releases/{release_id}')
+        # Hit the endpoint that matches the ID's type (tag-driven, no guessing).
+        data = None
+        for path in _discogs_album_endpoints(release_id):
+            data = self._api_get(path)
+            if data and data.get('tracklist'):
+                break
+            data = None
         if not data or not data.get('tracklist'):
             return None
 
@@ -637,7 +853,7 @@ class DiscogsClient:
             image_url = (primary or images[0]).get('uri')
 
         album_info = {
-            'id': str(data.get('id', release_id)),
+            'id': str(release_id),
             'name': data.get('title', ''),
             'images': [{'url': image_url, 'height': 600, 'width': 600}] if image_url else [],
             'release_date': str(data.get('year', '')) if data.get('year') else '',
@@ -646,7 +862,11 @@ class DiscogsClient:
         # Get artists
         artists_list = []
         if data.get('artists'):
-            artists_list = [{'name': a.get('name', '')} for a in data['artists'] if a.get('name')]
+            artists_list = [
+                {'name': _clean_discogs_artist_name(a.get('name', ''))}
+                for a in data['artists']
+                if a.get('name') and _clean_discogs_artist_name(a.get('name', ''))
+            ]
         if not artists_list:
             artists_list = [{'name': 'Unknown Artist'}]
 
@@ -675,7 +895,11 @@ class DiscogsClient:
             # Per-track artists or fall back to release artists
             track_artists = artists_list
             if t.get('artists'):
-                track_artists = [{'name': a.get('name', '')} for a in t['artists'] if a.get('name')]
+                track_artists = [
+                    {'name': _clean_discogs_artist_name(a.get('name', ''))}
+                    for a in t['artists']
+                    if a.get('name') and _clean_discogs_artist_name(a.get('name', ''))
+                ]
 
             tracks.append({
                 'id': f"{release_id}_t{track_num}",
@@ -718,9 +942,13 @@ class DiscogsClient:
         cached = cache.get_entity('discogs', 'album', str(release_id))
         if cached and cached.get('title'):
             return cached
-        data = self._api_get(f'/masters/{release_id}')
-        if not data or not data.get('title'):
-            data = self._api_get(f'/releases/{release_id}')
+        # Hit the endpoint that matches the ID's type (tag-driven, no guessing).
+        data = None
+        for path in _discogs_album_endpoints(release_id):
+            data = self._api_get(path)
+            if data and data.get('title'):
+                break
+            data = None
         if data:
             cache.store_entity('discogs', 'album', str(release_id), data)
         return data

core/discogs_worker.py

@@ -17,8 +17,8 @@ from typing import Optional, Dict, Any
 from datetime import datetime, timedelta
 from utils.logging_config import get_logger
 from database.music_database import MusicDatabase
-from core.discogs_client import DiscogsClient
-from core.worker_utils import interruptible_sleep, set_album_api_track_count
+from core.discogs_client import DiscogsClient, _discogs_album_kind, _tag_discogs_album_id
+from core.worker_utils import accept_artist_match, interruptible_sleep, set_album_api_track_count
 
 logger = get_logger("discogs_worker")
 
@@ -174,6 +174,16 @@ class DiscogsWorker:
             conn = self.db._get_connection()
             cursor = conn.cursor()
 
+            # Pinned-group override (Manage Enrichment Workers): process one
+            # entity type first, then fall through to the normal chain. Discogs
+            # has no track endpoint, so only artist/album are honored.
+            from core.worker_utils import read_enrichment_priority, priority_pending_item
+            _prio = read_enrichment_priority('discogs')
+            if _prio in ('artist', 'album'):
+                _pi = priority_pending_item(cursor, 'discogs', _prio)
+                if _pi:
+                    return _pi
+
             # Priority 1: Unattempted artists
             cursor.execute("""
                 SELECT id, name FROM artists
@@ -298,8 +308,8 @@ class DiscogsWorker:
             self.stats['errors'] += 1
             try:
                 self._mark_status(item['type'], item['id'], 'error')
-            except Exception:
-                pass
+            except Exception as e:
+                logger.debug("mark item status error failed: %s", e)
 
     def _get_existing_id(self, entity_type: str, entity_id) -> Optional[str]:
         """Check if entity already has a discogs_id."""
@@ -322,9 +332,13 @@ class DiscogsWorker:
             self.stats['not_found'] += 1
             return
 
-        # Find best match by name similarity
+        # Find best match by name similarity (skipping ids already claimed by
+        # a differently-named artist, so we don't create a shared/duplicate id).
         for result in results:
-            if self._name_matches(artist_name, result.name):
+            ok, reason = accept_artist_match(
+                self.db, 'discogs_id', result.id, artist_id, artist_name, result.name,
+            )
+            if ok:
                 # Fetch full artist detail (uses cache)
                 data = self.client._fetch_and_cache_artist(result.id)
                 if data:
@@ -422,7 +436,9 @@ class DiscogsWorker:
             conn = self.db._get_connection()
             cursor = conn.cursor()
 
-            discogs_id = str(data.get('id', ''))
+            # Tag the ID with its Discogs type so later re-fetches hit the right
+            # endpoint (master vs release share one numeric space).
+            discogs_id = _tag_discogs_album_id(data.get('id', ''), _discogs_album_kind(data))
             genres = json.dumps(data.get('genres', []))
             styles = json.dumps(data.get('styles', []))
             labels = data.get('labels', [])

core/discovery/endpoints.py

@@ -0,0 +1,767 @@
+"""Generic, source-agnostic helpers for the playlist-discovery route layer.
+
+The discovery/sync endpoints in ``web_server.py`` were copy-pasted once per
+source (Tidal, Deezer, Qobuz, Spotify-public, iTunes-link, YouTube,
+ListenBrainz, Beatport). The per-source copies differ only by a source label
+string and which ``<source>_discovery_states`` global they read. This module
+lifts the source-agnostic pieces into importable, unit-testable helpers so the
+route functions become thin wrappers — exactly preserving behavior (1:1).
+
+Each helper is lifted verbatim from its web_server.py counterpart; any
+per-source quirk that genuinely differs (e.g. Beatport's distinct result
+shape) is intentionally NOT routed through here and stays in its own function.
+"""
+
+from __future__ import annotations
+
+import time
+from typing import Any, Dict, List, Tuple
+
+from utils.logging_config import get_logger
+
+logger = get_logger("discovery.endpoints")
+
+
+def convert_results_to_spotify_tracks(
+    discovery_results: List[Dict[str, Any]],
+    source_label: str,
+) -> List[Dict[str, Any]]:
+    """Convert a source's discovery results into the Spotify-track dicts the
+    sync pipeline expects.
+
+    Lifted verbatim from the per-source ``convert_<source>_results_to_spotify_tracks``
+    functions (and the already-generic ``_convert_link_results_to_spotify_tracks``),
+    which were byte-identical apart from the ``source_label`` used in the log
+    line. Two input shapes are supported, matching the originals exactly:
+
+    - ``spotify_data`` (manual-fix shape): copied through, preserving optional
+      ``track_number`` / ``disc_number``.
+    - ``spotify_track`` + ``status_class == 'found'`` (auto-discovery shape):
+      rebuilt from the flat ``spotify_*`` fields.
+
+    Any result matching neither shape is skipped, identical to the originals.
+
+    NOTE: Beatport deliberately does NOT use this — its converter coerces
+    artist objects to strings and emits a different track shape (``source``
+    field, album dict), so it keeps its own implementation.
+    """
+    spotify_tracks: List[Dict[str, Any]] = []
+
+    for result in discovery_results:
+        # Support both data formats: spotify_data (manual fixes) and individual
+        # fields (automatic discovery).
+        if result.get('spotify_data'):
+            spotify_data = result['spotify_data']
+            track = {
+                'id': spotify_data['id'],
+                'name': spotify_data['name'],
+                'artists': spotify_data['artists'],
+                'album': spotify_data['album'],
+                'duration_ms': spotify_data.get('duration_ms', 0),
+            }
+            if spotify_data.get('track_number'):
+                track['track_number'] = spotify_data['track_number']
+            if spotify_data.get('disc_number'):
+                track['disc_number'] = spotify_data['disc_number']
+            spotify_tracks.append(track)
+        elif result.get('spotify_track') and result.get('status_class') == 'found':
+            spotify_tracks.append({
+                'id': result.get('spotify_id', 'unknown'),
+                'name': result.get('spotify_track', 'Unknown Track'),
+                'artists': [result.get('spotify_artist', 'Unknown Artist')] if result.get('spotify_artist') else ['Unknown Artist'],
+                'album': result.get('spotify_album', 'Unknown Album'),
+                'duration_ms': 0,
+            })
+
+    logger.info(f"Converted {len(spotify_tracks)} {source_label} matches to Spotify tracks for sync")
+    return spotify_tracks
+
+
+def cancel_sync(
+    states: Dict[str, Any],
+    key: str,
+    *,
+    label: str,
+    not_found_message: str,
+    sync_lock: Any,
+    sync_states: Dict[str, Any],
+    active_sync_workers: Dict[str, Any],
+) -> Tuple[Dict[str, Any], int]:
+    """Cancel an in-progress sync for one discovery playlist.
+
+    1:1 lift of the byte-identical ``cancel_<source>_sync`` bodies (Tidal,
+    Deezer, Qobuz, Spotify-Public, iTunes-Link, YouTube, ListenBrainz). The
+    caller passes the already-resolved state key (ListenBrainz transforms it
+    via ``_lb_state_key`` first), the source ``label``, the exact 404 message
+    (iTunes-Link uses "iTunes Link not found", not "... playlist not found"),
+    and the shared sync infrastructure (so this stays free of web_server
+    globals / Flask).
+
+    Returns ``(payload_dict, status_code)``; the caller wraps in ``jsonify``.
+
+    Beatport is NOT routed here — it cancels a stored ``sync_future`` and
+    returns a different payload.
+    """
+    try:
+        if key not in states:
+            # Idempotent: the live discovery state is gone (a restart wiped the
+            # in-memory state, or it was already cancelled). Cancelling a sync
+            # that isn't running is a no-op SUCCESS, not a 404 — otherwise a
+            # mirrored playlist (e.g. a ListenBrainz weekly) whose state vanished
+            # is permanently wedged with "playlist not found" and can never be
+            # re-synced or dismissed (#702).
+            return {"success": True, "message": f"No active {label} sync to cancel"}, 200
+
+        state = states[key]
+        state['last_accessed'] = time.time()
+        sync_playlist_id = state.get('sync_playlist_id')
+
+        if sync_playlist_id:
+            with sync_lock:
+                sync_states[sync_playlist_id] = {"status": "cancelled"}
+            if sync_playlist_id in active_sync_workers:
+                del active_sync_workers[sync_playlist_id]
+
+        state['phase'] = 'discovered'
+        state['sync_playlist_id'] = None
+        state['sync_progress'] = {}
+
+        return {"success": True, "message": f"{label} sync cancelled"}, 200
+    except Exception as e:
+        logger.error(f"Error cancelling {label} sync: {e}")
+        return {"error": str(e)}, 500
+
+
+def delete_playlist_state(
+    states: Dict[str, Any],
+    key: str,
+    *,
+    label: str,
+    not_found_message: str,
+) -> Tuple[Dict[str, Any], int]:
+    """Delete a discovery playlist's state entry, cancelling any active
+    discovery first.
+
+    1:1 lift of the byte-identical ``delete_<source>_playlist`` bodies
+    (Tidal, Deezer, Qobuz, Spotify-Public). Returns ``(payload, status_code)``.
+
+    The iTunes-Link / YouTube / ListenBrainz / Beatport deletes intentionally
+    keep their own bodies — they differ in success message, info-log wording,
+    name extraction, and/or key transform.
+    """
+    try:
+        if key not in states:
+            return {"error": not_found_message}, 404
+
+        state = states[key]
+        if 'discovery_future' in state and state['discovery_future']:
+            state['discovery_future'].cancel()
+        del states[key]
+
+        logger.info(f"Deleted {label} playlist state: {key}")
+        return {"success": True, "message": "Playlist deleted"}, 200
+    except Exception as e:
+        logger.error(f"Error deleting {label} playlist: {e}")
+        return {"error": str(e)}, 500
+
+
+# --- playlist-name accessors -------------------------------------------------
+# The per-source sync-status handlers read the display name three different
+# ways. Each is reproduced verbatim so the 1:1 behavior (including which ones
+# raise vs. fall back to 'Unknown Playlist') is preserved.
+
+def playlist_name_attr_or_unknown(state: Dict[str, Any]) -> str:
+    """Tidal: playlist is an object — use ``.name`` or 'Unknown Playlist'."""
+    pl = state.get('playlist')
+    return pl.name if pl and hasattr(pl, 'name') else 'Unknown Playlist'
+
+
+def playlist_name_strict(state: Dict[str, Any]) -> str:
+    """Deezer / Qobuz / Spotify-Public / iTunes-Link: strict dict access —
+    raises (→ 500) if 'playlist' is missing, exactly like the originals."""
+    return state['playlist']['name']
+
+
+def playlist_name_safe(state: Dict[str, Any]) -> str:
+    """YouTube / ListenBrainz: safe dict access, defaulting to 'Unknown
+    Playlist'."""
+    return state.get('playlist', {}).get('name', 'Unknown Playlist')
+
+
+def playlist_name_obj(state: Dict[str, Any]) -> str:
+    """Tidal start-sync: playlist is an object — strict ``.name`` (raises if
+    absent, exactly like the original)."""
+    return state['playlist'].name
+
+
+def playlist_image_obj(state: Dict[str, Any]) -> str:
+    """Tidal: ``getattr(playlist, 'image_url', '')`` (object attribute)."""
+    return getattr(state['playlist'], 'image_url', '')
+
+
+def playlist_image_dict(state: Dict[str, Any]) -> str:
+    """Deezer/Qobuz/Spotify-Public/YouTube: ``playlist.get('image_url', '')``
+    (dict access)."""
+    return state['playlist'].get('image_url', '')
+
+
+def get_sync_status(
+    states: Dict[str, Any],
+    key: str,
+    *,
+    not_found_message: str,
+    error_label: str,
+    activity_subject: str,
+    playlist_name_getter,
+    sync_lock: Any,
+    sync_states: Dict[str, Any],
+    add_activity_item,
+) -> Tuple[Dict[str, Any], int]:
+    """Report sync status for one discovery playlist, posting an activity-feed
+    item when the sync finishes or errors.
+
+    1:1 lift of the ``get_<source>_sync_status`` bodies (Tidal, Deezer, Qobuz,
+    Spotify-Public, iTunes-Link, YouTube, ListenBrainz). Per-source variation
+    is captured by the parameters:
+
+    - ``not_found_message`` — the 404 string (iTunes-Link drops "playlist").
+    - ``error_label`` — used in the except log ("Error getting <X> sync status").
+    - ``activity_subject`` — the activity-feed prefix; note Spotify-Public uses
+      "Spotify Link playlist" while its error_label is "Spotify Public".
+    - ``playlist_name_getter`` — one of the accessors above (attr/strict/safe);
+      the strict one can raise, matching the originals (→ 500). The state's
+      phase/sync_progress are mutated BEFORE the name is read, so a raising
+      getter leaves the same partial mutation the original did.
+
+    Beatport is NOT routed here — it returns a different payload (``status``
+    not ``sync_status``, includes ``sync_id``, no lock, ``chart`` key).
+    """
+    try:
+        if key not in states:
+            return {"error": not_found_message}, 404
+
+        state = states[key]
+        state['last_accessed'] = time.time()
+        sync_playlist_id = state.get('sync_playlist_id')
+
+        if not sync_playlist_id:
+            return {"error": "No sync in progress"}, 404
+
+        with sync_lock:
+            sync_state = sync_states.get(sync_playlist_id, {})
+
+        response = {
+            'phase': state['phase'],
+            'sync_status': sync_state.get('status', 'unknown'),
+            'progress': sync_state.get('progress', {}),
+            'complete': sync_state.get('status') == 'finished',
+            'error': sync_state.get('error'),
+        }
+
+        if sync_state.get('status') == 'finished':
+            state['phase'] = 'sync_complete'
+            state['sync_progress'] = sync_state.get('progress', {})
+            playlist_name = playlist_name_getter(state)
+            add_activity_item("", "Sync Complete", f"{activity_subject} '{playlist_name}' synced successfully", "Now")
+        elif sync_state.get('status') == 'error':
+            state['phase'] = 'discovered'
+            playlist_name = playlist_name_getter(state)
+            add_activity_item("", "Sync Failed", f"{activity_subject} '{playlist_name}' sync failed", "Now")
+
+        return response, 200
+    except Exception as e:
+        logger.error(f"Error getting {error_label} sync status: {e}")
+        return {"error": str(e)}, 500
+
+
+def get_discovery_status(
+    states: Dict[str, Any],
+    key: str,
+    *,
+    not_found_message: str,
+    error_label: str,
+) -> Tuple[Dict[str, Any], int]:
+    """Report real-time discovery progress/results for one playlist.
+
+    1:1 lift of the byte-identical ``get_<source>_discovery_status`` bodies.
+    Unlike sync-status, this shape is identical for ALL eight sources —
+    Beatport included — so it folds in too. Only the 404 message
+    (".../discovery not found" vs ".../playlist not found" vs "Beatport chart
+    not found") and the except-log label vary, both passed in. The caller
+    resolves the key (ListenBrainz via ``_lb_state_key``).
+
+    Returns ``(payload, status_code)``.
+    """
+    try:
+        if key not in states:
+            return {"error": not_found_message}, 404
+
+        state = states[key]
+        state['last_accessed'] = time.time()
+
+        return {
+            'phase': state['phase'],
+            'status': state['status'],
+            'progress': state['discovery_progress'],
+            'spotify_matches': state['spotify_matches'],
+            'spotify_total': state['spotify_total'],
+            'results': state['discovery_results'],
+            'complete': state['phase'] == 'discovered',
+        }, 200
+    except Exception as e:
+        logger.error(f"Error getting {error_label} discovery status: {e}")
+        return {"error": str(e)}, 500
+
+
+def reset_playlist(
+    states: Dict[str, Any],
+    key: str,
+    *,
+    label: str,
+    not_found_message: str,
+) -> Tuple[Dict[str, Any], int]:
+    """Reset a discovery playlist back to the 'fresh' phase, clearing all
+    discovery/sync data while preserving the original playlist payload.
+
+    1:1 lift of the byte-identical ``reset_<source>_playlist`` bodies
+    (Tidal, Deezer, Qobuz, Spotify-Public). Returns ``(payload, status_code)``.
+
+    NOT folded in (genuinely divergent): YouTube (status -> 'parsed', no
+    download_process_id, logs the playlist name, "reset to fresh state"),
+    ListenBrainz (status -> 'cached', logs playlist title, returns
+    {"phase": "fresh"}), iTunes-Link (uses state.update, no info log, distinct
+    message). Those keep their own bodies.
+    """
+    try:
+        if key not in states:
+            return {"error": not_found_message}, 404
+
+        state = states[key]
+        if 'discovery_future' in state and state['discovery_future']:
+            state['discovery_future'].cancel()
+
+        state['phase'] = 'fresh'
+        state['status'] = 'fresh'
+        state['discovery_results'] = []
+        state['discovery_progress'] = 0
+        state['spotify_matches'] = 0
+        state['sync_playlist_id'] = None
+        state['converted_spotify_playlist_id'] = None
+        state['download_process_id'] = None
+        state['sync_progress'] = {}
+        state['discovery_future'] = None
+        state['last_accessed'] = time.time()
+
+        logger.info(f"Reset {label} playlist to fresh: {key}")
+        return {"success": True, "message": "Playlist reset to fresh phase"}, 200
+    except Exception as e:
+        logger.error(f"Error resetting {label} playlist: {e}")
+        return {"error": str(e)}, 500
+
+
+def get_playlist_states(
+    states: Dict[str, Any],
+    *,
+    error_label: str,
+    info_log_label: str = None,
+) -> Tuple[Dict[str, Any], int]:
+    """Return all stored discovery states for a source as a list for frontend
+    card hydration (``{"states": [...]}``).
+
+    1:1 lift of the ``get_<source>_playlist_states`` bodies (Tidal, Deezer,
+    Qobuz, Spotify-Public, iTunes-Link), which build the same per-entry dict.
+    iTunes-Link is the only one without the "Returning N ..." info log, so
+    ``info_log_label`` is optional (pass None to suppress it, as iTunes did).
+
+    NOT folded in: the YouTube/ListenBrainz ``get_all_*_playlists`` endpoints —
+    they return ``{"playlists": [...]}`` (different key + fields: url/created_at,
+    no discovery_results) and filter mirrored/profile-scoped entries.
+    """
+    try:
+        result = []
+        current_time = time.time()
+
+        for key, state in states.items():
+            state['last_accessed'] = current_time
+            result.append({
+                'playlist_id': key,
+                'phase': state['phase'],
+                'status': state['status'],
+                'discovery_progress': state['discovery_progress'],
+                'spotify_matches': state['spotify_matches'],
+                'spotify_total': state['spotify_total'],
+                'discovery_results': state['discovery_results'],
+                'converted_spotify_playlist_id': state.get('converted_spotify_playlist_id'),
+                'download_process_id': state.get('download_process_id'),
+                'last_accessed': state['last_accessed'],
+            })
+
+        if info_log_label:
+            logger.info(f"Returning {len(result)} stored {info_log_label} playlist states for hydration")
+        return {"states": result}, 200
+    except Exception as e:
+        logger.error(f"Error getting {error_label} playlist states: {e}")
+        return {"error": str(e)}, 500
+
+
+def save_bubble_snapshot(
+    get_json,
+    *,
+    payload_key: str,
+    no_data_error: str,
+    snapshot_kind: str,
+    success_noun: str,
+    log_subject: str,
+    log_noun: str,
+    get_database,
+    get_current_profile_id,
+) -> Tuple[Dict[str, Any], int]:
+    """Persist a bubble/download snapshot for cross-refresh hydration.
+
+    1:1 lift of the four structurally-identical snapshot endpoints
+    (discover_downloads, artist_bubbles, search_bubbles, beatport_bubbles),
+    which differ only by:
+
+    - ``payload_key`` ('downloads' for discover, 'bubbles' for the rest) and
+      its ``no_data_error`` message.
+    - ``snapshot_kind`` — the db.save_bubble_snapshot category.
+    - ``success_noun`` — fills "Snapshot saved with N <noun>".
+    - ``log_subject`` / ``log_noun`` — the info ("Saved <subject>: N <noun>")
+      and except ("Error saving <subject>") log lines.
+
+    Returns ``(payload, status_code)``. ``get_json`` is invoked inside the try
+    like the original ``request.json``.
+    """
+    try:
+        from datetime import datetime
+
+        data = get_json()
+        if not data or payload_key not in data:
+            return {'success': False, 'error': no_data_error}, 400
+
+        items = data[payload_key]
+
+        db = get_database()
+        db.save_bubble_snapshot(snapshot_kind, items, profile_id=get_current_profile_id())
+
+        count = len(items)
+        logger.info(f"Saved {log_subject}: {count} {log_noun}")
+
+        return {
+            'success': True,
+            'message': f'Snapshot saved with {count} {success_noun}',
+            'timestamp': datetime.now().isoformat(),
+        }, 200
+    except Exception as e:
+        logger.error(f"Error saving {log_subject}: {e}")
+        import traceback
+        traceback.print_exc()
+        return {'success': False, 'error': str(e)}, 500
+
+
+def update_playlist_phase(
+    states: Dict[str, Any],
+    key: str,
+    get_json,
+    *,
+    not_found_message: str,
+    error_label: str,
+    valid_phases: List[str],
+    apply_extra_fields: bool,
+) -> Tuple[Dict[str, Any], int]:
+    """Update a discovery playlist's phase (used when the modal closes, e.g. to
+    reset download_complete -> discovered).
+
+    1:1 lift of the ``update_<source>_playlist_phase`` bodies for the five
+    sources with the identical validation + full-message response (Tidal,
+    Deezer, Qobuz, Spotify-Public, YouTube). Per-source params:
+
+    - ``valid_phases`` — YouTube's list additionally includes 'parsed'.
+    - ``apply_extra_fields`` — Deezer/Qobuz/Spotify-Public also persist
+      download_process_id / converted_spotify_playlist_id from the body;
+      Tidal/YouTube do NOT (so pass False to keep them 1:1).
+    - ``not_found_message`` / ``error_label``; ``get_json`` invoked inside the
+      try like the original ``request.get_json()``.
+
+    Returns ``(payload, status_code)``.
+
+    NOT folded in: iTunes-Link — it uses ``data.get('phase')`` (no separate
+    "Phase not provided" 400) and returns a no-message payload.
+    """
+    try:
+        if key not in states:
+            return {"error": not_found_message}, 404
+
+        data = get_json()
+        if not data or 'phase' not in data:
+            return {"error": "Phase not provided"}, 400
+
+        new_phase = data['phase']
+        if new_phase not in valid_phases:
+            return {"error": f"Invalid phase. Must be one of: {', '.join(valid_phases)}"}, 400
+
+        state = states[key]
+        old_phase = state.get('phase', 'unknown')
+        state['phase'] = new_phase
+        state['last_accessed'] = time.time()
+
+        if apply_extra_fields:
+            if 'download_process_id' in data:
+                state['download_process_id'] = data['download_process_id']
+            if 'converted_spotify_playlist_id' in data:
+                state['converted_spotify_playlist_id'] = data['converted_spotify_playlist_id']
+
+        logger.info(f"Updated {error_label} playlist {key} phase: {old_phase} → {new_phase}")
+        return {"success": True, "message": f"Phase updated to {new_phase}", "old_phase": old_phase, "new_phase": new_phase}, 200
+    except Exception as e:
+        logger.error(f"Error updating {error_label} playlist phase: {e}")
+        return {"error": str(e)}, 500
+
+
+def first_artist_str_or_obj(original_track: Dict[str, Any]) -> str:
+    """Tidal: first artist from an artists list that may hold strings OR
+    objects ({'name': ...}); '' when empty."""
+    artists = original_track.get('artists', [])
+    if artists:
+        return artists[0] if isinstance(artists[0], str) else artists[0].get('name', '')
+    return ''
+
+
+def first_artist_plain(original_track: Dict[str, Any]) -> str:
+    """Deezer/Qobuz/Spotify-Public: first artist assuming a list of strings;
+    '' when empty."""
+    artists = original_track.get('artists', [])
+    return artists[0] if artists else ''
+
+
+def update_discovery_match(
+    states: Dict[str, Any],
+    get_json,
+    *,
+    source_log_label: str,
+    error_label: str,
+    original_track_key: str,
+    original_artist_getter,
+    join_artist_names,
+    extract_artist_name,
+    build_fix_modal_spotify_data,
+    get_discovery_cache_key,
+    get_database,
+    get_active_discovery_source,
+) -> Tuple[Dict[str, Any], int]:
+    """Apply a manually-selected Spotify track to a discovery result (the
+    fix-modal flow) and persist it to the discovery cache.
+
+    1:1 lift of the ``update_<source>_discovery_match`` bodies for the four
+    sources with the identical structure (Tidal, Deezer, Qobuz, Spotify-Public).
+    Per-source pieces are params:
+
+    - ``source_log_label`` (lowercase, e.g. "tidal") for the "Manual match
+      updated: ..." line; ``error_label`` for the except log.
+    - ``original_track_key`` — the raw-source track key on the result
+      ('tidal_track', 'deezer_track', ...).
+    - ``original_artist_getter`` — Tidal handles string-or-object artists
+      (``first_artist_str_or_obj``); the rest assume strings
+      (``first_artist_plain``).
+    - the web_server helpers (join/extract artist, build_fix_modal_spotify_data,
+      cache-key, get_database, active-discovery-source) are injected so this
+      stays free of those globals.
+    - ``get_json`` is called INSIDE the try (like the original's
+      ``request.get_json()``) so a malformed body yields the same 500.
+
+    Returns ``(payload, status_code)``.
+
+    NOT folded in: iTunes-Link (saves spotify_data directly via a different
+    cache signature), YouTube (multi-key original_track fallback), ListenBrainz
+    (entirely different unmatch-capable structure, no cache write), Beatport.
+    """
+    try:
+        data = get_json()
+        identifier = data.get('identifier')
+        track_index = data.get('track_index')
+        spotify_track = data.get('spotify_track')
+
+        if not identifier or track_index is None or not spotify_track:
+            return {'error': 'Missing required fields'}, 400
+
+        state = states.get(identifier)
+        result = None
+        if state:
+            if track_index >= len(state['discovery_results']):
+                return {'error': 'Invalid track index'}, 400
+
+            result = state['discovery_results'][track_index]
+            old_status = result.get('status')
+
+            result['status'] = 'Found'
+            result['status_class'] = 'found'
+            result['spotify_track'] = spotify_track['name']
+            result['spotify_artist'] = join_artist_names(spotify_track['artists']) if isinstance(spotify_track['artists'], list) else extract_artist_name(spotify_track['artists'])
+            result['spotify_album'] = spotify_track['album']
+            result['spotify_id'] = spotify_track['id']
+
+            duration_ms = spotify_track.get('duration_ms', 0)
+            if duration_ms:
+                minutes = duration_ms // 60000
+                seconds = (duration_ms % 60000) // 1000
+                result['duration'] = f"{minutes}:{seconds:02d}"
+            else:
+                result['duration'] = '0:00'
+
+            result['spotify_data'] = build_fix_modal_spotify_data(spotify_track)
+            result['wing_it_fallback'] = False
+            result['manual_match'] = True
+
+            if old_status != 'found' and old_status != 'Found':
+                state['spotify_matches'] = state.get('spotify_matches', 0) + 1
+
+            logger.info(f"Manual match updated: {source_log_label} - {identifier} - track {track_index}")
+            logger.info(f"   → {result['spotify_artist']} - {result['spotify_track']}")
+
+            original_track = result.get(original_track_key, {})
+            original_name = original_track.get('name', spotify_track['name'])
+            original_artist = original_artist_getter(original_track)
+        else:
+            # #843: the in-memory discovery state can be gone — a server restart,
+            # or an imported playlist that wasn't discovered in THIS process —
+            # while the card is still shown from persisted data. The DURABLE part
+            # of a manual fix (writing the match to the discovery cache so future
+            # syncs resolve it) doesn't need the in-memory state, only the original
+            # track's name + artist, which the client now sends. Fall back to those
+            # instead of 404ing the fix into uselessness.
+            original_name = (data.get('original_name') or '').strip()
+            original_artist = (data.get('original_artist') or '').strip()
+            if not original_name and not original_artist:
+                return {'error': 'Discovery state not found'}, 404
+            if not original_name:
+                original_name = spotify_track['name']
+            # Key the cache by the FIRST artist — every in-memory + sync path uses
+            # artists[0], but the client may send a joined "A, B, C" string. Without
+            # this, a multi-artist track would save under a key the sync never looks
+            # up (full string ≠ first artist), so the fix would silently never apply.
+            if original_artist:
+                original_artist = original_artist.split(',')[0].strip()
+            logger.info(
+                f"Manual match (no in-memory state) → discovery cache: "
+                f"{source_log_label} - {identifier} - '{original_name}' by '{original_artist}'"
+            )
+
+        try:
+            cache_key = get_discovery_cache_key(original_name, original_artist)
+            artists_list = spotify_track['artists']
+            if isinstance(artists_list, list):
+                artists_list = [a if isinstance(a, str) else a.get('name', '') for a in artists_list]
+            image_url = spotify_track.get('image_url') or ''
+            album_raw = spotify_track.get('album', '')
+            if isinstance(album_raw, dict):
+                album_obj = dict(album_raw)
+                if image_url and not album_obj.get('image_url'):
+                    album_obj['image_url'] = image_url
+                if image_url and not album_obj.get('images'):
+                    album_obj['images'] = [{'url': image_url}]
+            else:
+                album_obj = {'name': album_raw or ''}
+                if image_url:
+                    album_obj['image_url'] = image_url
+                    album_obj['images'] = [{'url': image_url}]
+
+            matched_data = {
+                'id': spotify_track['id'],
+                'name': spotify_track['name'],
+                'artists': artists_list,
+                'album': album_obj,
+                'duration_ms': spotify_track.get('duration_ms', 0),
+                'image_url': image_url,
+                'source': 'spotify',
+            }
+            cache_db = get_database()
+            cache_db.save_discovery_cache_match(
+                cache_key[0], cache_key[1], get_active_discovery_source(), 1.0, matched_data,
+                original_name, original_artist
+            )
+            logger.info(f"Manual fix saved to discovery cache: {original_name} by {original_artist}")
+        except Exception as cache_err:
+            logger.error(f"Error saving manual fix to discovery cache: {cache_err}")
+
+        return {'success': True, 'result': result}, 200
+    except Exception as e:
+        logger.error(f"Error updating {error_label} discovery match: {e}")
+        return {'error': str(e)}, 500
+
+
+def start_sync(
+    states: Dict[str, Any],
+    key: str,
+    *,
+    sync_id_prefix: str,
+    not_found_message: str,
+    not_ready_message: str,
+    convert_fn,
+    playlist_name_getter,
+    playlist_image_getter,
+    activity_label: str,
+    error_label: str,
+    sync_lock: Any,
+    sync_states: Dict[str, Any],
+    active_sync_workers: Dict[str, Any],
+    submit_sync_task,
+    add_activity_item,
+) -> Tuple[Dict[str, Any], int]:
+    """Kick off a playlist sync from a source's discovered Spotify matches.
+
+    1:1 lift of the ``start_<source>_sync`` bodies for the five sources with
+    the identical flow (Tidal, Deezer, Qobuz, Spotify-Public, YouTube). The
+    per-source pieces are parameters:
+
+    - ``sync_id_prefix`` — the ``f"{prefix}_{key}"`` sync id.
+    - ``convert_fn`` — the source's discovery->spotify-tracks converter.
+    - ``playlist_name_getter`` / ``playlist_image_getter`` — Tidal reads an
+      object (``.name`` / ``getattr``), the rest read a dict; lifted as the
+      ``playlist_name_obj``/``playlist_image_obj`` vs ``playlist_name_strict``/
+      ``playlist_image_dict`` accessors.
+    - ``activity_label`` vs ``error_label`` — these DIFFER for Spotify-Public:
+      activity says "Spotify Link Sync Started" while logs say "Spotify Public".
+    - ``submit_sync_task(sync_playlist_id, playlist_name, spotify_tracks,
+      playlist_image_url) -> Future`` — wraps sync_executor/_run_sync_task/
+      get_current_profile_id so this stays free of those globals.
+
+    Returns ``(payload, status_code)``.
+
+    NOT folded in: iTunes-Link (no final info log), ListenBrainz (submits the
+    task without an image arg), Beatport (extra debug logging, 'chart' key).
+    """
+    try:
+        if key not in states:
+            return {"error": not_found_message}, 404
+
+        state = states[key]
+        state['last_accessed'] = time.time()
+
+        if state['phase'] not in ['discovered', 'sync_complete', 'download_complete']:
+            return {"error": not_ready_message}, 400
+
+        spotify_tracks = convert_fn(state['discovery_results'])
+        if not spotify_tracks:
+            return {"error": "No Spotify matches found for sync"}, 400
+
+        sync_playlist_id = f"{sync_id_prefix}_{key}"
+        playlist_name = playlist_name_getter(state)
+
+        add_activity_item("", f"{activity_label} Sync Started", f"'{playlist_name}' - {len(spotify_tracks)} tracks", "Now")
+
+        state['phase'] = 'syncing'
+        state['sync_playlist_id'] = sync_playlist_id
+        state['sync_progress'] = {}
+
+        with sync_lock:
+            sync_states[sync_playlist_id] = {"status": "starting", "progress": {}}
+
+        playlist_image_url = playlist_image_getter(state)
+        future = submit_sync_task(sync_playlist_id, playlist_name, spotify_tracks, playlist_image_url)
+        active_sync_workers[sync_playlist_id] = future
+
+        logger.info(f"Started {error_label} sync for: {playlist_name} ({len(spotify_tracks)} tracks)")
+        return {"success": True, "sync_playlist_id": sync_playlist_id}, 200
+    except Exception as e:
+        logger.error(f"Error starting {error_label} sync: {e}")
+        return {"error": str(e)}, 500

core/discovery/hero.py

@@ -17,10 +17,14 @@ logger = logging.getLogger(__name__)
 
 
 def get_current_profile_id() -> int:
-    """Mirror of web_server.get_current_profile_id — uses Flask g."""
+    """Mirror of web_server.get_current_profile_id — uses Flask g.
+
+    Catches RuntimeError too because reading `g` outside a request
+    context raises that (not AttributeError) — happens when this is
+    called from background threads (sync, automation, scanners)."""
     try:
         return g.profile_id
-    except AttributeError:
+    except (AttributeError, RuntimeError):
         return 1
 
 
@@ -92,6 +96,8 @@ def get_discover_hero():
                     artist_id = artist.spotify_artist_id
                 elif active_source == 'deezer':
                     artist_id = getattr(artist, 'deezer_artist_id', None) or artist.itunes_artist_id
+                elif active_source == 'musicbrainz':
+                    artist_id = getattr(artist, 'musicbrainz_artist_id', None) or artist.itunes_artist_id
                 else:
                     artist_id = artist.itunes_artist_id
                 if not artist_id:
@@ -121,7 +127,7 @@ def get_discover_hero():
         valid_artists = list(similar_artists)
 
         # FALLBACK: If no valid artists for fallback source, try to resolve IDs on-the-fly
-        if active_source in ('itunes', 'deezer') and not valid_artists:
+        if active_source in ('itunes', 'deezer', 'musicbrainz') and not valid_artists:
             logger.warning(f"[{active_source} Fallback] No artists with {active_source} IDs found, attempting on-the-fly resolution for {len(similar_artists)} artists")
             resolved_count = 0
             for artist in similar_artists:
@@ -131,13 +137,20 @@ def get_discover_hero():
                     continue
                 # Try to resolve ID by name
                 try:
-                    search_results = itunes_client.search_artists(artist.similar_artist_name, limit=1)
+                    resolve_client = itunes_client
+                    if active_source == 'musicbrainz':
+                        from core.metadata.registry import get_musicbrainz_client
+                        resolve_client = get_musicbrainz_client()
+                    search_results = resolve_client.search_artists(artist.similar_artist_name, limit=1)
                     if search_results and len(search_results) > 0:
                         resolved_id = search_results[0].id
                         # Cache the resolved ID for future use
                         if active_source == 'deezer':
                             database.update_similar_artist_deezer_id(artist.id, resolved_id)
                             artist.similar_artist_deezer_id = resolved_id
+                        elif active_source == 'musicbrainz':
+                            database.update_similar_artist_musicbrainz_id(artist.id, resolved_id)
+                            artist.similar_artist_musicbrainz_id = resolved_id
                         else:
                             database.update_similar_artist_itunes_id(artist.id, resolved_id)
                             artist.similar_artist_itunes_id = resolved_id
@@ -169,12 +182,15 @@ def get_discover_hero():
                 artist_id = artist.similar_artist_spotify_id or artist.similar_artist_itunes_id
             elif active_source == 'deezer':
                 artist_id = getattr(artist, 'similar_artist_deezer_id', None) or artist.similar_artist_itunes_id or artist.similar_artist_spotify_id
+            elif active_source == 'musicbrainz':
+                artist_id = getattr(artist, 'similar_artist_musicbrainz_id', None) or artist.similar_artist_itunes_id or artist.similar_artist_spotify_id
             else:
                 artist_id = artist.similar_artist_itunes_id or artist.similar_artist_spotify_id
 
             artist_data = {
                 "spotify_artist_id": artist.similar_artist_spotify_id,
                 "itunes_artist_id": artist.similar_artist_itunes_id,
+                "musicbrainz_artist_id": getattr(artist, 'similar_artist_musicbrainz_id', None),
                 "artist_id": artist_id,
                 "artist_name": artist.similar_artist_name,
                 "occurrence_count": artist.occurrence_count,
@@ -203,11 +219,19 @@ def get_discover_hero():
                                     artist.id, artist_data.get('image_url'),
                                     artist_data.get('genres'), artist_data.get('popularity')
                                 )
-                    elif active_source in ('itunes', 'deezer'):
-                        fb_artist_id = getattr(artist, 'similar_artist_deezer_id', None) if active_source == 'deezer' else None
-                        fb_artist_id = fb_artist_id or artist.similar_artist_itunes_id
+                    elif active_source in ('itunes', 'deezer', 'musicbrainz'):
+                        if active_source == 'deezer':
+                            fb_artist_id = getattr(artist, 'similar_artist_deezer_id', None) or artist.similar_artist_itunes_id
+                            fetch_client = itunes_client
+                        elif active_source == 'musicbrainz':
+                            fb_artist_id = getattr(artist, 'similar_artist_musicbrainz_id', None)
+                            from core.metadata.registry import get_musicbrainz_client
+                            fetch_client = get_musicbrainz_client()
+                        else:
+                            fb_artist_id = artist.similar_artist_itunes_id
+                            fetch_client = itunes_client
                         if fb_artist_id:
-                            fb_artist_data = itunes_client.get_artist(fb_artist_id)
+                            fb_artist_data = fetch_client.get_artist(fb_artist_id)
                             if fb_artist_data:
                                 artist_data['artist_name'] = fb_artist_data.get('name', artist.similar_artist_name)
                                 artist_data['image_url'] = fb_artist_data.get('images', [{}])[0].get('url') if fb_artist_data.get('images') else None

core/discovery/listening_recommendations.py

@@ -0,0 +1,340 @@
+"""Listening-driven recommendation core (#913).
+
+PURE, side-effect-free ranking that turns "the artists you listen to most" plus
+"who's similar to each" into:
+
+  1. a consensus-ranked list of artists you'd probably love but don't own, and
+  2. an aggregated candidate-track list for a generated playlist.
+
+No DB / network / config here. The caller (the watchlist scanner) supplies the
+seeds (top-played artists), the ``similar_artists`` rows per seed, and the
+owned-artist set, then fetches top tracks for the winners. Keeping the decision
+logic in one pure place makes it fully unit-testable without the live stack and
+keeps the scan wiring thin — and additive, so it can't disturb existing flows.
+
+Scoring rationale (the "best in class" bit): a recommended artist's score is
+``Σ over the seeds that recommend it of (seed_weight × similarity)``. That single
+sum rewards all three signals at once — **consensus** (an artist endorsed by many
+of your seeds accumulates more terms), your **play weight** (heavier seeds push
+harder), and **similarity strength** — instead of a flat "appears in N lists".
+``seed_count`` is exposed separately for display ("because you like A, B, C") and
+as the adventurousness dial's lever (``min_seed_count``).
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import Dict, List, Optional, Sequence, Set
+
+
+def _norm(name: object) -> str:
+    return str(name or "").strip().lower()
+
+
+def _positive_float(value: object, default: float = 1.0) -> float:
+    try:
+        f = float(value)  # type: ignore[arg-type]
+    except (TypeError, ValueError):
+        return default
+    return f if f > 0 else default
+
+
+def _get(row: object, attr: str):
+    """Read a field from a dataclass row or a dict row."""
+    if isinstance(row, dict):
+        return row.get(attr)
+    return getattr(row, attr, None)
+
+
+def choose_mix_fetch_source(active_source: object, active_can_fetch: bool) -> str:
+    """Pick which source to fetch the "Listening Mix" top tracks from.
+
+    The mix is a list of (artist, title) pairs acquired via Soulseek, so the fetch source need
+    NOT match the user's active metadata source. Use the active source when it can fetch top
+    tracks itself (Spotify/Deezer); otherwise fall back to Deezer, whose public ``artist/{id}/top``
+    needs no auth and is available to every user — so iTunes / Discogs / MusicBrainz users still
+    get a full mix without switching sources. Pure.
+    """
+    if str(active_source or "").lower() in ("spotify", "deezer") and active_can_fetch:
+        return str(active_source).lower()
+    return "deezer"
+
+
+def names_match(a: object, b: object) -> bool:
+    """Strict artist-name equality after stripping case + non-alphanumerics.
+
+    Used to verify a name-search result before fetching that artist's top tracks, so the
+    "Listening Mix" can never pull the WRONG artist's songs (e.g. a same-name act). Exact
+    alphanumeric match: "Tyler, The Creator" == "Tyler The Creator", but "Drake" != "Drake Bell".
+    Pure.
+    """
+    def _alnum(x: object) -> str:
+        return "".join(ch for ch in str(x or "").lower() if ch.isalnum())
+    na, nb = _alnum(a), _alnum(b)
+    return bool(na) and na == nb
+
+
+def similarity_from_rank(rank: object, max_rank: int = 10) -> float:
+    """Turn a stored ``similarity_rank`` (1 = most similar … 10 = least) into a 0–1 weight.
+
+    SoulSync stores each ``(seed → similar)`` edge with a 1–10 rank (``1`` is the closest
+    match). The ranker multiplies this into the score so a seed's *closest* matches count
+    for more than its long-tail ones. Linear decay over the documented range: rank 1 → 1.0,
+    rank 5 → 0.6, rank 10 → 0.1, with a 0.1 floor so a far match still contributes. A
+    missing/garbage rank falls back to 1.0 (treat as "no rank info, full weight"). Pure.
+    """
+    try:
+        r = int(rank)
+    except (TypeError, ValueError):
+        return 1.0
+    floor = round(1.0 / max_rank, 4)
+    if r <= 1:
+        return 1.0
+    if r >= max_rank:
+        return floor
+    return round((max_rank - r + 1) / max_rank, 4)
+
+
+def build_recency_weighted_seeds(
+    top_artists: Sequence[dict],
+    recent_play_counts: Optional[Dict[str, float]] = None,
+    *,
+    recency_factor: float = 1.5,
+) -> List[dict]:
+    """Blend lifetime + recent play counts into seed weights — "what you're into NOW".
+
+    ``weight = lifetime_plays + recency_factor × recent_plays``. An artist you've played a
+    lot *recently* outranks one you played a lot years ago, so the recommendations track
+    your current taste instead of your all-time history. ``recency_factor`` is the dial
+    (0 = pure lifetime). Returns ``[{'name', 'weight'}]`` for :func:`rank_recommended_artists`.
+    Pure — the caller supplies both play-count maps from the listening history.
+    """
+    recent = {_norm(k): _positive_float(v, 0.0) for k, v in (recent_play_counts or {}).items()}
+    out: List[dict] = []
+    for a in top_artists or ():
+        name = str(a.get("name") or "").strip()
+        if not name:
+            continue
+        lifetime = _positive_float(a.get("play_count", a.get("weight", 1.0)))
+        boost = recency_factor * recent.get(_norm(name), 0.0)
+        out.append({"name": name, "weight": lifetime + boost})
+    return out
+
+
+def group_similars_by_seed(
+    seeds: Sequence[dict],
+    similar_rows: Sequence,
+    id_to_name: Dict[str, str],
+    *,
+    source_id_attr: str = "source_artist_id",
+    similar_name_attr: str = "similar_artist_name",
+    rank_attr: Optional[str] = None,
+) -> Dict[str, List[dict]]:
+    """Reshape flat ``similar_artists`` rows into ``{seed_name_lower: [{'name', 'score'?}]}``.
+
+    The stored rows key the similar artist by the SEED's source id (``source_artist_id``),
+    not its name, so :func:`rank_recommended_artists` can't consume them directly. This
+    resolves each row's source id to a name via ``id_to_name`` (``{source_artist_id:
+    artist_name}`` for the library, built by the caller) and keeps only rows that resolve
+    to one of the ``seeds``. Rows may be dataclass objects or dicts. Pure — no I/O.
+
+    ``id_to_name`` MUST be keyed by whatever id the edges actually store — for SoulSync that
+    is the artist's SOURCE id (Spotify/iTunes/Deezer/MusicBrainz), NOT the internal row id.
+    When ``rank_attr`` is given, each row's rank is converted via :func:`similarity_from_rank`
+    and carried as ``score`` so closer matches weigh more; without it every similar comes out
+    score-less (the ranker then treats similarity as 1.0 — original behavior).
+    """
+    seed_names = {_norm(s.get("name")) for s in seeds}
+    seed_names.discard("")
+    id_to_norm = {str(k): _norm(v) for k, v in (id_to_name or {}).items()}
+
+    out: Dict[str, List[dict]] = {}
+    for row in similar_rows or ():
+        seed_name = id_to_norm.get(str(_get(row, source_id_attr) or ""), "")
+        if not seed_name or seed_name not in seed_names:
+            continue
+        sim_name = str(_get(row, similar_name_attr) or "").strip()
+        if not sim_name:
+            continue
+        entry = {"name": sim_name}
+        if rank_attr is not None:
+            entry["score"] = similarity_from_rank(_get(row, rank_attr))
+        out.setdefault(seed_name, []).append(entry)
+    return out
+
+
+@dataclass
+class RecommendedArtist:
+    """One artist recommended from your listening, with the why."""
+    name: str                                       # display name (first-seen casing)
+    score: float                                    # Σ seed_weight × similarity
+    seed_count: int                                 # distinct seeds endorsing it (consensus)
+    seeds: List[str] = field(default_factory=list)  # display names of those seeds
+
+
+def rank_recommended_artists(
+    seeds: Sequence[dict],
+    similars_by_seed: Dict[str, Sequence[dict]],
+    owned_artist_names: Optional[Set[str]] = None,
+    *,
+    limit: int = 30,
+    min_seed_count: int = 1,
+) -> List[RecommendedArtist]:
+    """Rank artists similar to your most-played by consensus + play weight + similarity.
+
+    Args:
+        seeds: ``[{'name': str, 'weight': float}]`` — your top-played artists.
+            ``weight`` (play count or any positive number) defaults to 1.0.
+        similars_by_seed: ``{seed_name_lower: [{'name': str, 'score': float}]}`` — the
+            similar-artist rows for each seed. ``score`` is optional (defaults 1.0).
+        owned_artist_names: lowercased names already in the library — excluded so the
+            result is artists you DON'T have. The seeds themselves are always excluded.
+        limit: max results.
+        min_seed_count: drop recommendations endorsed by fewer than N seeds — the
+            adventurousness dial's "Safer" end raises this for higher-confidence picks.
+
+    Returns up to ``limit`` :class:`RecommendedArtist`, highest score first.
+    """
+    owned = {_norm(a) for a in (owned_artist_names or set())}
+    seed_norms = {_norm(s.get("name")) for s in seeds}
+    seed_norms.discard("")
+    exclude = owned | seed_norms
+
+    acc: Dict[str, dict] = {}
+    for seed in seeds:
+        s_name = _norm(seed.get("name"))
+        if not s_name:
+            continue
+        s_display = str(seed.get("name") or "").strip()
+        weight = _positive_float(seed.get("weight", 1.0))
+        for sim in similars_by_seed.get(s_name, ()) or ():
+            a_norm = _norm(sim.get("name"))
+            if not a_norm or a_norm in exclude:
+                continue
+            sim_score = _positive_float(sim.get("score", 1.0))
+            row = acc.setdefault(
+                a_norm, {"name": str(sim.get("name") or "").strip(), "score": 0.0, "seeds": {}}
+            )
+            row["score"] += weight * sim_score
+            row["seeds"].setdefault(s_name, s_display)   # one seed counts once
+
+    out: List[RecommendedArtist] = []
+    floor = max(1, int(min_seed_count))
+    for row in acc.values():
+        seed_count = len(row["seeds"])
+        if seed_count < floor:
+            continue
+        out.append(RecommendedArtist(
+            name=row["name"],
+            score=round(row["score"], 6),
+            seed_count=seed_count,
+            seeds=list(row["seeds"].values()),
+        ))
+    out.sort(key=lambda r: (-r.score, -r.seed_count, r.name.lower()))
+    return out[:limit]
+
+
+def aggregate_candidate_tracks(
+    recommended_artists: Sequence[RecommendedArtist],
+    top_tracks_by_artist: Dict[str, Sequence[dict]],
+    owned_track_keys: Optional[Set] = None,
+    *,
+    per_artist: int = 3,
+    limit: int = 50,
+    exclude_owned: bool = True,
+) -> List[dict]:
+    """Build the candidate track list for the generated playlist.
+
+    Takes the top ``per_artist`` tracks from each recommended artist **in artist-rank
+    order**, dedups by ``(artist, title)``, optionally drops owned tracks (the
+    "discovery" flavor) and caps at ``limit``. Each returned track dict is the source
+    track plus ``_seed_artist`` (which recommended artist it came from).
+
+    Args:
+        recommended_artists: ranked output of :func:`rank_recommended_artists`.
+        top_tracks_by_artist: ``{artist_name_lower: [track_dict, ...]}`` — fetched by
+            the caller (Last.fm / source top tracks), NOT limited to a curated pool.
+        owned_track_keys: set of ``(artist_lower, title_lower)`` already in the library.
+        exclude_owned: drop tracks in ``owned_track_keys`` (discovery flavor). Set False
+            for a "replay" playlist of tracks you already own.
+    """
+    owned = owned_track_keys or set()
+    seen: Set = set()
+    out: List[dict] = []
+    for art in recommended_artists:
+        tracks = top_tracks_by_artist.get(_norm(art.name), ()) or ()
+        taken = 0
+        for t in tracks:
+            if taken >= per_artist:
+                break
+            title = str(t.get("name") or t.get("title") or "").strip()
+            if not title:
+                continue
+            key = (_norm(art.name), _norm(title))
+            if key in seen:
+                continue
+            if exclude_owned and key in owned:
+                continue
+            seen.add(key)
+            out.append({**t, "_seed_artist": art.name})
+            taken += 1
+        if len(out) >= limit:
+            break
+    return out[:limit]
+
+
+def to_mix_track(track: object, source: str) -> Optional[dict]:
+    """Shape one source "top tracks" API dict into the flat dict the Discover compact
+    playlist row renders + syncs (the "Listening Mix" #913 playlist).
+
+    Spotify's ``artist_top_tracks`` and Deezer's ``get_artist_top_tracks`` both return the
+    same Spotify-shape object (``id, name, artists[], album{name,images[]}, duration_ms``).
+    This flattens that into the renderer's field names (``track_name/artist_name/album_name/
+    album_cover_url/duration_ms``), keeps the original under ``track_data_json`` for sync, and
+    sets the source-specific id field. Returns None for anything without a usable id/title so
+    the caller can filter. A ``name`` key is kept so :func:`aggregate_candidate_tracks` can
+    dedup by title. Pure — no I/O.
+    """
+    if not isinstance(track, dict):
+        return None
+    tid = track.get("id")
+    name = str(track.get("name") or "").strip()
+    if not tid or not name:
+        return None
+    artists = track.get("artists") or []
+    artist_name = ""
+    if artists and isinstance(artists[0], dict):
+        artist_name = str(artists[0].get("name") or "").strip()
+    album = track.get("album") if isinstance(track.get("album"), dict) else {}
+    album_name = str(album.get("name") or "").strip()
+    images = album.get("images") or []
+    cover = images[0].get("url") if images and isinstance(images[0], dict) else None
+    out = {
+        "track_id": str(tid),
+        "name": name,                  # for aggregate_candidate_tracks dedup
+        "track_name": name,            # for the renderer
+        "artist_name": artist_name,
+        "album_name": album_name,
+        "album_cover_url": cover,
+        "duration_ms": track.get("duration_ms") or 0,
+        "track_data_json": track,      # full payload for sync/download
+        "source": source,
+    }
+    id_field = {"spotify": "spotify_track_id", "deezer": "deezer_track_id",
+                "itunes": "itunes_track_id"}.get(source)
+    if id_field:
+        out[id_field] = str(tid)
+    return out
+
+
+__all__ = [
+    "RecommendedArtist",
+    "choose_mix_fetch_source",
+    "names_match",
+    "similarity_from_rank",
+    "build_recency_weighted_seeds",
+    "to_mix_track",
+    "group_similars_by_seed",
+    "rank_recommended_artists",
+    "aggregate_candidate_tracks",
+]

core/discovery/manual_match.py

@@ -0,0 +1,120 @@
+"""Helpers for Fix-popup manual match persistence.
+
+When the user manually fixes a mirrored-playlist discovery via the Fix
+popup, two questions land at the web_server route layer that are easier
+to test in isolation:
+
+1. *Which metadata source did the manual match come from?* — the popup
+   cascade queries the user's primary source first, then Spotify /
+   Deezer / iTunes / MusicBrainz as fallbacks; each search endpoint
+   stamps `source` on its rows but the MBID-paste lookup uses a lean
+   flat shape that doesn't carry it. `derive_manual_match_provider`
+   collapses the fallback chain into a single string.
+
+2. *Should the discovery layer re-run for this track when the current
+   active provider differs from the cached one?* — re-running silently
+   overwrites the user's deliberate pick with whatever the auto-search
+   ranks first, so manual matches are exempt regardless of provider
+   drift. `is_drifted_for_redo` encapsulates the decision.
+
+3. *Should the Playlist Pipeline pre-scan (re)discover this track at all?*
+   — `should_rediscover` encapsulates that gate, with the manual match
+   checked FIRST so a leftover Wing It flag can't override the user's pick.
+"""
+
+from __future__ import annotations
+
+from typing import Any, Dict, Optional
+
+
+def derive_manual_match_provider(
+    payload_track: Dict[str, Any],
+    active_provider: Optional[str],
+) -> str:
+    """Return the provider string to stamp on a manually-fixed match.
+
+    Resolution order:
+        1. ``payload_track['source']`` — every *_search_tracks endpoint
+           sets this; the MBID-paste path doesn't.
+        2. ``active_provider`` — what the user has configured as their
+           primary discovery source.
+        3. ``'spotify'`` — last-ditch default matching the historic
+           hardcode (so behaviour is identical when both upstream
+           signals are absent).
+    """
+    if not isinstance(payload_track, dict):
+        payload_track = {}
+    source = payload_track.get('source')
+    if source:
+        return source
+    if active_provider:
+        return active_provider
+    return 'spotify'
+
+
+def is_drifted_for_redo(
+    extra_data: Optional[Dict[str, Any]],
+    active_provider: Optional[str],
+) -> bool:
+    """Return True when a cached discovery entry should be treated as
+    stale because the user's active provider has changed since it was
+    cached AND the entry isn't a manual match.
+
+    Manual matches are *always* considered fresh: re-running discovery
+    against the current source would overwrite the user's deliberate
+    pick with whatever auto-search ranks first. The first Playlist
+    Pipeline run after a manual fix used to clobber it for exactly
+    this reason — the check lives here now so it's pinned by tests.
+    """
+    if not isinstance(extra_data, dict):
+        return False
+    if extra_data.get('manual_match'):
+        return False
+    cached_provider = extra_data.get('provider', 'spotify')
+    return cached_provider != active_provider
+
+
+def should_rediscover(extra_data: Optional[Dict[str, Any]]) -> bool:
+    """Return True when a mirrored track needs (re)discovery, False to skip it.
+
+    This is the gate the Playlist Pipeline pre-scan runs over every mirrored
+    track before discovering. The **ordering is the fix**: a manual match is
+    authoritative and is checked FIRST.
+
+    ``extra_data`` is *merged* on save (see ``update_mirrored_track_extra_data``),
+    so a track that was a Wing It stub and is then manually fixed still carries
+    ``wing_it_fallback: True`` alongside the new ``manual_match: True``. The old
+    pre-scan tested ``wing_it_fallback`` before ``manual_match``, so the stale
+    flag won and the pipeline re-discovered the track — silently reverting the
+    user's pick to Wing It. Checking ``manual_match`` first makes the fix stick.
+
+    Decision order:
+      * manual_match            -> skip   (authoritative; never re-discover)
+      * wing_it_fallback        -> redo   (stub — keep trying for a real match)
+      * discovered + complete   -> skip   (full metadata already stored)
+      * discovered + incomplete -> redo   (backfill track_number / album fields)
+      * unmatched_by_user       -> skip   (user deliberately removed the match)
+      * never discovered        -> redo   (first-time discovery)
+    """
+    extra = extra_data if isinstance(extra_data, dict) else {}
+
+    if extra.get('discovered'):
+        if extra.get('manual_match'):
+            return False
+        if extra.get('wing_it_fallback'):
+            return True
+        # Otherwise re-discover only when the stored match is missing the
+        # enriched fields (track_number + release_date/album.id) that older
+        # discoveries dropped via the Track dataclass.
+        matched = extra.get('matched_data')
+        matched = matched if isinstance(matched, dict) else {}
+        album = matched.get('album')
+        album = album if isinstance(album, dict) else {}
+        has_track_num = matched.get('track_number')
+        has_release = album.get('release_date')
+        has_album_id = album.get('id')
+        return not (has_track_num and (has_release or has_album_id))
+
+    if extra.get('unmatched_by_user'):
+        return False
+    return True

core/discovery/matching.py

@@ -0,0 +1,134 @@
+"""Pure helper for matching raw MusicBrainz-metadata tracks against
+Spotify / iTunes.
+
+Used by the PlaylistSource adapters whose ``get_playlist`` returns
+tracks with ``needs_discovery=True`` (ListenBrainz, Last.fm radio).
+Phase 1b ships Strategy 1 only (matching-engine queries → search →
+score → pick best ≥0.9). The richer multi-strategy +
+discovery-cache flow stays in
+``core.discovery.listenbrainz.run_listenbrainz_discovery_worker``
+for the Discover-page state-machine UI; this helper is the slimmer
+version used by the auto-refresh pipeline.
+"""
+
+from __future__ import annotations
+
+import logging
+from dataclasses import dataclass
+from typing import Any, Callable, Dict, List, Optional
+
+logger = logging.getLogger(__name__)
+
+
+@dataclass
+class MBMatchDeps:
+    """Bundle of primitives the matcher needs.
+
+    Wired up at bootstrap. Tests pass stub callables / clients."""
+
+    matching_engine: Any
+    score_candidates: Callable[..., Any]
+    spotify_client_getter: Callable[[], Any]
+    itunes_client_getter: Callable[[], Any]
+    prefer_spotify_getter: Callable[[], bool]
+    min_confidence: float = 0.9
+
+
+def match_mb_track(
+    track: Dict[str, Any], deps: MBMatchDeps
+) -> Optional[Dict[str, Any]]:
+    """Try to match a single MB-metadata track.
+
+    Input shape:
+        ``{'track_name', 'artist_name', 'album_name', 'duration_ms'}``
+
+    Returns the matched_data dict (Spotify/iTunes track projection)
+    or ``None`` when no candidate cleared the confidence threshold.
+    """
+    title = track.get("track_name") or ""
+    artist = track.get("artist_name") or ""
+    album = track.get("album_name") or ""
+    duration_ms = int(track.get("duration_ms") or 0)
+    if not title or not artist:
+        return None
+
+    spotify_client = deps.spotify_client_getter()
+    itunes_client = deps.itunes_client_getter()
+    use_spotify = bool(
+        deps.prefer_spotify_getter()
+        and spotify_client is not None
+        and getattr(spotify_client, "is_spotify_authenticated", lambda: False)()
+    )
+    if not use_spotify and itunes_client is None:
+        return None
+
+    # Strategy 1 — matching-engine query generation.
+    try:
+        temp_track = type("_TempTrack", (), {
+            "name": title,
+            "artists": [artist],
+            "album": album or None,
+        })()
+        queries = deps.matching_engine.generate_download_queries(temp_track)
+    except Exception as exc:
+        logger.debug(f"matching_engine query-gen failed: {exc}")
+        queries = [f"{artist} {title}", title]
+
+    best_match: Any = None
+    best_confidence = 0.0
+    for query in queries:
+        try:
+            if use_spotify:
+                results = spotify_client.search_tracks(query, limit=10)
+            else:
+                results = itunes_client.search_tracks(query, limit=10)
+        except Exception as exc:
+            logger.debug(f"search failed for query={query!r}: {exc}")
+            continue
+        if not results:
+            continue
+        try:
+            match, confidence, _ = deps.score_candidates(
+                title, artist, duration_ms, results
+            )
+        except Exception as exc:
+            logger.debug(f"score_candidates failed: {exc}")
+            continue
+        if match and confidence > best_confidence and confidence >= deps.min_confidence:
+            best_match = match
+            best_confidence = confidence
+        if best_confidence >= deps.min_confidence:
+            break
+
+    if not best_match:
+        return None
+
+    provider = "spotify" if use_spotify else "itunes"
+    image_url = getattr(best_match, "image_url", None) or ""
+    album_data: Dict[str, Any] = {
+        "name": getattr(best_match, "album", "") or "",
+    }
+    if image_url:
+        album_data["images"] = [{"url": image_url}]
+    return {
+        "id": getattr(best_match, "id", "") or "",
+        "name": getattr(best_match, "name", "") or "",
+        "artists": list(getattr(best_match, "artists", []) or []),
+        "album": album_data,
+        "duration_ms": int(getattr(best_match, "duration_ms", 0) or 0),
+        "image_url": image_url,
+        "source": provider,
+        "_provider": provider,
+        "_confidence": float(best_confidence),
+    }
+
+
+def match_mb_tracks(
+    tracks: List[Dict[str, Any]], deps: MBMatchDeps
+) -> List[Optional[Dict[str, Any]]]:
+    """Vectorized variant — runs ``match_mb_track`` per track.
+
+    Phase 1b is sequential. If profiling shows it's too slow on big
+    LB playlists, this becomes the natural spot to thread-pool the
+    per-track searches."""
+    return [match_mb_track(t, deps) for t in tracks]

core/discovery/playlist.py

@@ -37,9 +37,35 @@ import time
 from dataclasses import dataclass
 from typing import Any, Callable
 
+from core.discovery.manual_match import should_rediscover
+
 logger = logging.getLogger(__name__)
 
 
+def _canonical_best_score(deps, title, artist, duration_ms, results):
+    """Score search results against the source track, trying the canonicalized
+    title/artist too and keeping the better confidence (#785).
+
+    YouTube playlists have their "Artist - Title" / channel decoration stripped
+    at ingest, but file/CSV-imported playlists keep raw titles — so a track
+    titled "Arctic Monkeys - Do I Wanna Know?" scored verbatim against the
+    library's "Do I Wanna Know?" never matched. canonical_source_track is
+    conservative (only strips an "<artist> - " prefix when it equals the
+    artist), so this can only ADD a better candidate, never weaken a match.
+    Returns (match, confidence)."""
+    match, confidence, _ = deps.discovery_score_candidates(title, artist, duration_ms, results)
+    try:
+        from core.text.source_title import canonical_source_track
+        canon_title, canon_artist = canonical_source_track(title or '', artist or '')
+    except Exception:
+        return match, confidence
+    if (canon_title, canon_artist) != (title, artist):
+        alt_match, alt_conf, _ = deps.discovery_score_candidates(canon_title, canon_artist, duration_ms, results)
+        if alt_match and alt_conf > confidence:
+            return alt_match, alt_conf
+    return match, confidence
+
+
 @dataclass
 class PlaylistDiscoveryDeps:
     """Bundle of cross-cutting deps the playlist discovery worker needs."""
@@ -121,31 +147,14 @@ def run_playlist_discovery_worker(playlists, automation_id=None, deps: PlaylistD
                         existing_extra = json.loads(track['extra_data']) if isinstance(track['extra_data'], str) else track['extra_data']
                     except (json.JSONDecodeError, TypeError):
                         pass
-                if existing_extra.get('discovered'):
-                    if existing_extra.get('wing_it_fallback'):
-                        # Wing It stub — always re-attempt to find a real match
-                        undiscovered_tracks.append(track)
-                    else:
-                        # Check if matched_data is complete — old discoveries may be missing
-                        # track_number/release_date due to the Track dataclass stripping them.
-                        # Re-discover these so the enriched pipeline fills in the gaps.
-                        md = existing_extra.get('matched_data', {})
-                        album = md.get('album', {})
-                        has_track_num = md.get('track_number')
-                        has_release = album.get('release_date') if isinstance(album, dict) else None
-                        has_album_id = album.get('id') if isinstance(album, dict) else None
-                        if has_track_num and (has_release or has_album_id):
-                            pl_skipped += 1
-                            total_skipped += 1
-                        else:
-                            # Incomplete discovery — re-discover to get full metadata
-                            undiscovered_tracks.append(track)
-                elif existing_extra.get('unmatched_by_user'):
-                    # User explicitly removed this match — respect their choice
+                # `should_rediscover` is the single source of truth for this
+                # gate (manual match checked FIRST so a stale Wing It flag can't
+                # revert a user's deliberate fix — see its docstring).
+                if should_rediscover(existing_extra):
+                    undiscovered_tracks.append(track)
+                else:
                     pl_skipped += 1
                     total_skipped += 1
-                else:
-                    undiscovered_tracks.append(track)
 
             if pl_skipped > 0:
                 deps.update_automation_progress(automation_id,
@@ -196,8 +205,8 @@ def run_playlist_discovery_worker(playlists, automation_id=None, deps: PlaylistD
                             current_item=track_name,
                             log_line=f'{track_name} → {cached_match.get("name", "?")} (cache)', log_type='success')
                         continue
-                except Exception:
-                    pass
+                except Exception as e:
+                    logger.debug("discovery cache lookup failed: %s", e)
 
                 # Step 2: Generate search queries
                 try:
@@ -210,6 +219,20 @@ def run_playlist_discovery_worker(playlists, automation_id=None, deps: PlaylistD
                 except Exception:
                     search_queries = [f"{artist_name} {track_name}", track_name]
 
+                # #785: file/CSV playlists keep raw "Artist - Title" titles, so the
+                # queries above search for the artist prefix too. Also search the
+                # canonicalized title so the right candidates are actually returned
+                # (the scorer best-of then matches them).
+                try:
+                    from core.text.source_title import canonical_source_track
+                    _cq_title, _cq_artist = canonical_source_track(track_name, artist_name)
+                    if (_cq_title, _cq_artist) != (track_name, artist_name):
+                        for _q in (f"{_cq_artist} {_cq_title}", _cq_title):
+                            if _q not in search_queries:
+                                search_queries.append(_q)
+                except Exception as _cq_err:
+                    logger.debug("canonical search-query add failed: %s", _cq_err)
+
                 # Step 3: Search and score
                 best_match = None
                 best_confidence = 0.0
@@ -224,8 +247,8 @@ def run_playlist_discovery_worker(playlists, automation_id=None, deps: PlaylistD
                         if not results:
                             continue
 
-                        match, confidence, _ = deps.discovery_score_candidates(
-                            track_name, artist_name, duration_ms, results
+                        match, confidence = _canonical_best_score(
+                            deps, track_name, artist_name, duration_ms, results
                         )
 
                         if match and confidence > best_confidence:
@@ -246,14 +269,14 @@ def run_playlist_discovery_worker(playlists, automation_id=None, deps: PlaylistD
                         else:
                             extended = itunes_client_instance.search_tracks(query, limit=50)
                         if extended:
-                            match, confidence, _ = deps.discovery_score_candidates(
-                                track_name, artist_name, duration_ms, extended
+                            match, confidence = _canonical_best_score(
+                                deps, track_name, artist_name, duration_ms, extended
                             )
                             if match and confidence > best_confidence:
                                 best_confidence = confidence
                                 best_match = match
-                    except Exception:
-                        pass
+                    except Exception as e:
+                        logger.debug("extended discovery search failed: %s", e)
 
                 # Step 4: Store results
                 if best_match and best_confidence >= min_confidence:
@@ -290,9 +313,21 @@ def run_playlist_discovery_worker(playlists, automation_id=None, deps: PlaylistD
                             if _raw:
                                 track_number = _raw.get('track_number')
                                 disc_number = _raw.get('disc_number')
-                        except Exception:
-                            pass
+                        except Exception as e:
+                            logger.debug("metadata cache lookup for album enrichment failed: %s", e)
 
+                    # Always include ``track_number`` / ``disc_number``
+                    # in the matched payload — None when unknown rather
+                    # than omitting the key. Downstream consumers
+                    # (``ensure_wishlist_track_format``, post-process
+                    # pipeline) check for None to know "look this up
+                    # somewhere else"; an absent key was indistinguishable
+                    # from "value is 1" after older payload helpers
+                    # silently filled the default. Pre-fix Deezer-sourced
+                    # matches always omitted the key (Deezer's track shape
+                    # uses ``track_position`` and the cache lookup at
+                    # line 304 reads ``track_number`` literally so it
+                    # returns None for Deezer rows).
                     matched_data = {
                         'id': best_match.id if hasattr(best_match, 'id') else '',
                         'name': best_match.name if hasattr(best_match, 'name') else '',
@@ -301,11 +336,13 @@ def run_playlist_discovery_worker(playlists, automation_id=None, deps: PlaylistD
                         'duration_ms': best_match.duration_ms if hasattr(best_match, 'duration_ms') else 0,
                         'image_url': match_image,
                         'source': discovery_source,
+                        'track_number': track_number if track_number else (
+                            getattr(best_match, 'track_number', None)
+                        ),
+                        'disc_number': disc_number if disc_number else (
+                            getattr(best_match, 'disc_number', None)
+                        ),
                     }
-                    if track_number:
-                        matched_data['track_number'] = track_number
-                    if disc_number:
-                        matched_data['disc_number'] = disc_number
 
                     extra_data = {
                         'discovered': True,
@@ -323,8 +360,8 @@ def run_playlist_discovery_worker(playlists, automation_id=None, deps: PlaylistD
                             best_confidence, matched_data,
                             track_name, artist_name
                         )
-                    except Exception:
-                        pass
+                    except Exception as e:
+                        logger.debug("save discovery cache match failed: %s", e)
 
                     logger.info(f"[{i+1}/{len(undiscovered_tracks)}] {track_name} → {matched_data['name']} ({best_confidence:.2f})")
                     deps.update_automation_progress(automation_id,
@@ -366,8 +403,8 @@ def run_playlist_discovery_worker(playlists, automation_id=None, deps: PlaylistD
                     'failed_count': str(total_failed),
                     'skipped_count': str(total_skipped),
                 })
-        except Exception:
-            pass
+        except Exception as e:
+            logger.debug("discovery_completed emit failed: %s", e)
 
         logger.error(f"Playlist discovery complete: {total_discovered} discovered, {total_failed} failed, {total_skipped} skipped")
         deps.update_automation_progress(automation_id, status='finished', progress=100,

core/discovery/qobuz.py

@@ -0,0 +1,299 @@
+"""Background worker for Qobuz playlist discovery.
+
+`run_qobuz_discovery_worker(playlist_id, deps)` is the function the
+Qobuz discovery start-endpoint submits to its executor to match each
+Qobuz playlist track against Spotify (preferred) or the configured
+fallback metadata source (iTunes / Deezer / Discogs / MusicBrainz).
+
+Mirrors `core/discovery/deezer.py` exactly — Qobuz playlists arrive as
+dicts (not dataclasses) from `core/qobuz_client.py:get_playlist`, so
+this worker uses dict-style access on track data and wraps each entry
+in a SimpleNamespace before handing it to the shared
+`_search_spotify_for_tidal_track` helper.
+"""
+
+from __future__ import annotations
+
+import logging
+import time
+import types
+from dataclasses import dataclass
+from typing import Any, Callable
+
+logger = logging.getLogger(__name__)
+
+
+@dataclass
+class QobuzDiscoveryDeps:
+    """Bundle of cross-cutting deps the Qobuz discovery worker needs."""
+    qobuz_discovery_states: dict
+    spotify_client: Any
+    pause_enrichment_workers: Callable[[str], dict]
+    resume_enrichment_workers: Callable[[dict, str], None]
+    get_active_discovery_source: Callable[[], str]
+    get_metadata_fallback_client: Callable[[], Any]
+    get_discovery_cache_key: Callable
+    get_database: Callable[[], Any]
+    validate_discovery_cache_artist: Callable
+    search_spotify_for_tidal_track: Callable
+    build_discovery_wing_it_stub: Callable
+    add_activity_item: Callable
+    sync_discovery_results_to_mirrored: Callable
+
+
+def run_qobuz_discovery_worker(playlist_id, deps: QobuzDiscoveryDeps):
+    """Background worker for Qobuz discovery process (Spotify preferred, fallback metadata source)."""
+    _ew_state = {}
+    try:
+        _ew_state = deps.pause_enrichment_workers('Qobuz discovery')
+        state = deps.qobuz_discovery_states[playlist_id]
+        playlist = state['playlist']
+
+        # Determine which provider to use
+        discovery_source = deps.get_active_discovery_source()
+        use_spotify = (discovery_source == 'spotify') and deps.spotify_client and deps.spotify_client.is_spotify_authenticated()
+
+        # Initialize fallback client if needed
+        itunes_client_instance = None
+        if not use_spotify:
+            itunes_client_instance = deps.get_metadata_fallback_client()
+
+        logger.info(f"Starting Qobuz discovery for: {playlist['name']} (using {discovery_source.upper()})")
+
+        # Store discovery source in state for frontend
+        state['discovery_source'] = discovery_source
+
+        successful_discoveries = 0
+        tracks = playlist['tracks']
+
+        for i, qobuz_track in enumerate(tracks):
+            if state.get('cancelled', False):
+                break
+
+            try:
+                track_name = qobuz_track['name']
+                track_artists = qobuz_track['artists']
+                track_id = qobuz_track['id']
+                track_album = qobuz_track.get('album', '')
+                track_duration_ms = qobuz_track.get('duration_ms', 0)
+
+                logger.info(f"[{i+1}/{len(tracks)}] Searching {discovery_source.upper()}: {track_name} by {', '.join(track_artists)}")
+
+                # Check discovery cache first
+                cache_key = deps.get_discovery_cache_key(track_name, track_artists[0] if track_artists else '')
+                try:
+                    cache_db = deps.get_database()
+                    cached_match = cache_db.get_discovery_cache_match(cache_key[0], cache_key[1], discovery_source)
+                    if cached_match and deps.validate_discovery_cache_artist(track_artists[0] if track_artists else '', cached_match):
+                        logger.debug(f"CACHE HIT [{i+1}/{len(tracks)}]: {track_name} by {', '.join(track_artists)}")
+                        cached_artists = cached_match.get('artists', [])
+                        if cached_artists:
+                            cached_artist_str = ', '.join(
+                                a if isinstance(a, str) else a.get('name', '') for a in cached_artists
+                            )
+                        else:
+                            cached_artist_str = ''
+                        cached_album = cached_match.get('album', '')
+                        if isinstance(cached_album, dict):
+                            cached_album = cached_album.get('name', '')
+
+                        result = {
+                            'qobuz_track': {
+                                'id': track_id,
+                                'name': track_name,
+                                'artists': track_artists or [],
+                                'album': track_album,
+                                'duration_ms': track_duration_ms,
+                            },
+                            'spotify_data': cached_match,
+                            'match_data': cached_match,
+                            'status': 'Found',
+                            'status_class': 'found',
+                            'spotify_track': cached_match.get('name', ''),
+                            'spotify_artist': cached_artist_str,
+                            'spotify_album': cached_album,
+                            'spotify_id': cached_match.get('id', ''),
+                            'discovery_source': discovery_source,
+                            'index': i
+                        }
+                        successful_discoveries += 1
+                        state['spotify_matches'] = successful_discoveries
+                        state['discovery_results'].append(result)
+                        state['discovery_progress'] = int(((i + 1) / len(tracks)) * 100)
+                        continue
+                except Exception as cache_err:
+                    logger.error(f"Cache lookup error: {cache_err}")
+
+                # SimpleNamespace duck-type for _search_spotify_for_tidal_track
+                track_ns = types.SimpleNamespace(
+                    id=track_id,
+                    name=track_name,
+                    artists=track_artists,
+                    album=track_album,
+                    duration_ms=track_duration_ms
+                )
+
+                track_result = deps.search_spotify_for_tidal_track(
+                    track_ns,
+                    use_spotify=use_spotify,
+                    itunes_client=itunes_client_instance
+                )
+
+                result = {
+                    'qobuz_track': {
+                        'id': track_id,
+                        'name': track_name,
+                        'artists': track_artists or [],
+                        'album': track_album,
+                        'duration_ms': track_duration_ms,
+                    },
+                    'spotify_data': None,
+                    'match_data': None,
+                    'status': 'Not Found',
+                    'status_class': 'not-found',
+                    'spotify_track': '',
+                    'spotify_artist': '',
+                    'spotify_album': '',
+                    'discovery_source': discovery_source
+                }
+
+                match_confidence = 0.0
+
+                if use_spotify and isinstance(track_result, tuple):
+                    track_obj, raw_track_data, match_confidence = track_result
+                    album_obj = raw_track_data.get('album', {}) if raw_track_data else {}
+                    if isinstance(album_obj, dict) and not album_obj.get('name') and track_obj.album:
+                        album_obj['name'] = track_obj.album
+                    elif not album_obj and track_obj.album:
+                        album_obj = {'name': track_obj.album}
+                    if isinstance(album_obj, dict) and not album_obj.get('release_date'):
+                        album_obj['release_date'] = getattr(track_obj, 'release_date', '') or ''
+                    _album_images = album_obj.get('images', []) if isinstance(album_obj, dict) else []
+                    _image_url = _album_images[0].get('url', '') if _album_images else (getattr(track_obj, 'image_url', '') or '')
+
+                    match_data = {
+                        'id': track_obj.id,
+                        'name': track_obj.name,
+                        'artists': track_obj.artists,
+                        'album': album_obj,
+                        'duration_ms': track_obj.duration_ms,
+                        'external_urls': track_obj.external_urls,
+                        'image_url': _image_url,
+                        'source': 'spotify'
+                    }
+                    if raw_track_data and raw_track_data.get('track_number'):
+                        match_data['track_number'] = raw_track_data['track_number']
+                    if raw_track_data and raw_track_data.get('disc_number'):
+                        match_data['disc_number'] = raw_track_data['disc_number']
+                    result['spotify_data'] = match_data
+                    result['match_data'] = match_data
+                    result['status'] = 'Found'
+                    result['status_class'] = 'found'
+                    result['spotify_track'] = track_obj.name
+                    result['spotify_artist'] = ', '.join(track_obj.artists) if isinstance(track_obj.artists, list) else str(track_obj.artists)
+                    result['spotify_album'] = album_obj.get('name', '') if isinstance(album_obj, dict) else str(album_obj)
+                    result['spotify_id'] = track_obj.id
+                    result['confidence'] = match_confidence
+                    successful_discoveries += 1
+                    state['spotify_matches'] = successful_discoveries
+
+                elif not use_spotify and track_result and isinstance(track_result, dict):
+                    match_confidence = track_result.pop('confidence', 0.80)
+                    match_data = track_result
+                    match_data['source'] = discovery_source
+                    _fb_album = match_data.get('album', {})
+                    _fb_images = _fb_album.get('images', []) if isinstance(_fb_album, dict) else []
+                    if _fb_images and 'image_url' not in match_data:
+                        match_data['image_url'] = _fb_images[0].get('url', '')
+                    result['spotify_data'] = match_data
+                    result['match_data'] = match_data
+                    result['status'] = 'Found'
+                    result['status_class'] = 'found'
+                    result['spotify_track'] = match_data.get('name', '')
+                    itunes_artists = match_data.get('artists', [])
+                    result['spotify_artist'] = ', '.join(a if isinstance(a, str) else a.get('name', '') for a in itunes_artists) if itunes_artists else ''
+                    result['spotify_album'] = match_data.get('album', {}).get('name', '') if isinstance(match_data.get('album'), dict) else match_data.get('album', '')
+                    result['spotify_id'] = match_data.get('id', '')
+                    result['confidence'] = match_confidence
+                    successful_discoveries += 1
+                    state['spotify_matches'] = successful_discoveries
+
+                # Save to discovery cache if match found
+                if result['status_class'] == 'found' and result.get('match_data'):
+                    try:
+                        cache_db = deps.get_database()
+                        cache_db.save_discovery_cache_match(
+                            cache_key[0], cache_key[1], discovery_source, match_confidence,
+                            result['match_data'], track_name,
+                            track_artists[0] if track_artists else ''
+                        )
+                        logger.info(f"CACHE SAVED: {track_name} (confidence: {match_confidence:.3f})")
+                    except Exception as cache_err:
+                        logger.error(f"Cache save error: {cache_err}")
+
+                # Auto Wing It fallback for unmatched tracks
+                if result['status_class'] == 'not-found':
+                    qobuz_t = result.get('qobuz_track', {})
+                    stub = deps.build_discovery_wing_it_stub(
+                        qobuz_t.get('name', ''),
+                        ', '.join(qobuz_t.get('artists', [])),
+                        qobuz_t.get('duration_ms', 0)
+                    )
+                    result['status'] = 'Wing It'
+                    result['status_class'] = 'wing-it'
+                    result['spotify_data'] = stub
+                    result['match_data'] = stub
+                    result['spotify_track'] = qobuz_t.get('name', '')
+                    result['spotify_artist'] = ', '.join(qobuz_t.get('artists', []))
+                    result['wing_it_fallback'] = True
+                    result['confidence'] = 0
+                    successful_discoveries += 1
+                    state['spotify_matches'] = successful_discoveries
+                    state['wing_it_count'] = state.get('wing_it_count', 0) + 1
+
+                result['index'] = i
+                state['discovery_results'].append(result)
+                state['discovery_progress'] = int(((i + 1) / len(tracks)) * 100)
+
+                time.sleep(0.1)
+
+            except Exception as e:
+                logger.error(f"Error processing track {i+1}: {e}")
+                result = {
+                    'qobuz_track': {
+                        'name': qobuz_track.get('name', 'Unknown'),
+                        'artists': qobuz_track.get('artists', []),
+                    },
+                    'spotify_data': None,
+                    'match_data': None,
+                    'status': 'Error',
+                    'status_class': 'error',
+                    'spotify_track': '',
+                    'spotify_artist': '',
+                    'spotify_album': '',
+                    'error': str(e),
+                    'discovery_source': discovery_source,
+                    'index': i
+                }
+                state['discovery_results'].append(result)
+                state['discovery_progress'] = int(((i + 1) / len(tracks)) * 100)
+
+        # Mark as complete
+        state['phase'] = 'discovered'
+        state['status'] = 'discovered'
+        state['discovery_progress'] = 100
+
+        source_label = discovery_source.upper()
+        deps.add_activity_item("", f"Qobuz Discovery Complete ({source_label})", f"'{playlist['name']}' - {successful_discoveries}/{len(tracks)} tracks found", "Now")
+
+        logger.info(f"Qobuz discovery complete ({source_label}): {successful_discoveries}/{len(tracks)} tracks found")
+
+        deps.sync_discovery_results_to_mirrored('qobuz', playlist_id, state.get('discovery_results', []), discovery_source, profile_id=state.get('_profile_id', 1))
+
+    except Exception as e:
+        logger.error(f"Error in Qobuz discovery worker: {e}")
+        if playlist_id in deps.qobuz_discovery_states:
+            deps.qobuz_discovery_states[playlist_id]['phase'] = 'error'
+            deps.qobuz_discovery_states[playlist_id]['status'] = f'error: {str(e)}'
+    finally:
+        deps.resume_enrichment_workers(_ew_state, 'Qobuz discovery')

core/discovery/quality_scanner.py

@@ -1,55 +1,48 @@
-"""Background worker for the library quality scanner.
+"""Shared metadata match + result-normalization helpers for quality matching.
 
-`run_quality_scanner(scope, profile_id, deps)` is the function the
-quality-scanner endpoint kicks off in a thread to scan the library
-for low-quality tracks (below the user's configured quality profile)
-and add provider matches to the wishlist:
+These were the matching guts of the old auto-acting quality-scanner worker (now
+removed — quality scanning is the ``quality_upgrade`` library-maintenance repair
+job in ``core/repair_jobs/quality_upgrade.py``). They're kept here as a single
+source of truth and imported by that job:
 
-1. Reset scanner state, load quality profile + minimum acceptable tier.
-2. Load tracks from DB based on scope:
-   - 'watchlist' → tracks for watchlisted artists only.
-   - other → all library tracks.
-3. For each track:
-   - Stop-request gate (state['status'] != 'running').
-   - Quality-tier check via _get_quality_tier_from_extension(file_path).
-   - Skip tracks meeting standards (tier_num <= min_acceptable_tier).
-   - For low-quality tracks: matching_engine search query gen, score
-     candidates against the configured metadata source priority
-     (artist + title similarity, album-type bonus), pick best match >=
-     0.7 confidence.
-   - On match: add normalized track data to wishlist via
-     `wishlist_service.add_track_to_wishlist` with
-     source_type='quality_scanner' and a source_context that captures
-     original file_path, format tier, bitrate, and match confidence.
-4. After all tracks: status='finished', progress=100, activity feed
-   entry, emit `quality_scan_completed` event for automation engine.
-5. On critical exception: status='error', error message captured.
+- ``_search_tracks_for_source`` — query one metadata source's ``search_tracks``.
+- ``_normalize_track_match`` / ``_normalize_track_album`` / ``_normalize_track_artists``
+  — turn a provider track into the wishlist-ready dict (typed Album converters
+  with legacy duck-typed fallback).
+- ``_track_name`` / ``_track_artist_names`` / ``_extract_lookup_value`` — accessors.
 """
 
 from __future__ import annotations
 
 import logging
-import time
-from dataclasses import dataclass
-from datetime import datetime
-from typing import Any, Callable
+from typing import Any, Callable, Dict, Optional
 
-from core.metadata.registry import get_client_for_source, get_primary_source, get_source_priority
+from core.metadata.registry import get_client_for_source
+from core.metadata.types import Album
 from core.wishlist.payloads import ensure_wishlist_track_format
 
-logger = logging.getLogger(__name__)
+# Use the project logger namespace ("soulsync.*") so the scanner's progress and
+# diagnostics actually surface in the app log — plain getLogger(__name__) lands
+# under "core.discovery.quality_scanner", which the app log view doesn't show.
+from utils.logging_config import get_logger
+logger = get_logger("discovery.quality_scanner")
+
+
+# Per-source typed converter dispatch — same registry pattern as
+# the metadata builders. Quality-scanner result normalization routes
+# the embedded ``track.album`` blob through Album.from_<source>_dict()
+# when provider is known. Falls back to legacy duck-typed extraction.
+_TYPED_ALBUM_CONVERTERS: Dict[str, Callable[[Dict[str, Any]], Album]] = {
+    'spotify': Album.from_spotify_dict,
+    'itunes': Album.from_itunes_dict,
+    'deezer': Album.from_deezer_dict,
+    'discogs': Album.from_discogs_dict,
+    'musicbrainz': Album.from_musicbrainz_dict,
+    'hydrabase': Album.from_hydrabase_dict,
+    'qobuz': Album.from_qobuz_dict,
+}
 
 
-@dataclass
-class QualityScannerDeps:
-    """Bundle of cross-cutting deps the quality scanner needs."""
-    quality_scanner_state: dict
-    quality_scanner_lock: Any  # threading.Lock
-    QUALITY_TIERS: dict
-    matching_engine: Any
-    automation_engine: Any
-    get_quality_tier_from_extension: Callable
-    add_activity_item: Callable
 
 
 def _extract_lookup_value(value: Any, *names: str, default: Any = None) -> Any:
@@ -140,7 +133,16 @@ def _normalize_image_entries(image_value: Any) -> list[dict]:
     return normalized
 
 
-def _normalize_track_album(track_item: Any) -> dict:
+def _normalize_track_album(track_item: Any, provider: Optional[str] = None) -> dict:
+    """Normalize a track's embedded album blob into a flat dict.
+
+    When ``provider`` is provided AND maps to a registered typed Album
+    converter, routes through the typed path to seed canonical fields
+    on ``album_data`` before legacy fallback chains fill any gaps.
+    Falls back to legacy duck-typed extraction on unknown provider /
+    non-dict input / typed converter error — same pattern as the
+    metadata builders.
+    """
     album = _extract_lookup_value(track_item, 'album', default={})
     if isinstance(album, dict):
         album_data = dict(album)
@@ -152,6 +154,27 @@ def _normalize_track_album(track_item: Any) -> dict:
             'release_date': _extract_lookup_value(album, 'release_date', default='') or '',
         }
 
+    if provider and isinstance(album, dict):
+        converter = _TYPED_ALBUM_CONVERTERS.get(provider.strip().lower())
+        if converter is not None:
+            try:
+                typed_album = converter(album)
+                if typed_album.name:
+                    album_data.setdefault('name', typed_album.name)
+                if typed_album.album_type:
+                    album_data.setdefault('album_type', typed_album.album_type)
+                if typed_album.total_tracks:
+                    album_data.setdefault('total_tracks', typed_album.total_tracks)
+                if typed_album.release_date:
+                    album_data.setdefault('release_date', typed_album.release_date)
+                if typed_album.id:
+                    album_data.setdefault('id', typed_album.id)
+            except Exception as exc:
+                logger.debug(
+                    "Typed album converter failed for provider %s in quality "
+                    "scanner normalize, falling back to legacy: %s", provider, exc,
+                )
+
     album_data.setdefault('name', _extract_lookup_value(track_item, 'album_name', default='Unknown Album') or 'Unknown Album')
     album_data.setdefault('album_type', _extract_lookup_value(track_item, 'album_type', default='album') or 'album')
     album_data.setdefault('total_tracks', _extract_lookup_value(track_item, 'total_tracks', 'track_count', default=0) or 0)
@@ -189,7 +212,7 @@ def _normalize_track_match(track_item: Any, provider: str) -> dict:
         'id': _extract_lookup_value(track_item, 'id', 'track_id', default='') or '',
         'name': _extract_lookup_value(track_item, 'name', 'title', default='Unknown Track') or 'Unknown Track',
         'artists': _normalize_track_artists(track_item),
-        'album': _normalize_track_album(track_item),
+        'album': _normalize_track_album(track_item, provider=provider),
         'image_url': _extract_lookup_value(track_item, 'image_url', 'album_cover_url', default=None),
         'duration_ms': _extract_lookup_value(track_item, 'duration_ms', default=0) or 0,
         'track_number': _extract_lookup_value(track_item, 'track_number', default=1) or 1,
@@ -254,363 +277,3 @@ def _search_tracks_for_source(source: str, query: str, limit: int = 5, client: A
     except Exception as exc:
         logger.debug("Could not search %s for %s: %s", source, query, exc)
         return []
-
-
-def run_quality_scanner(scope='watchlist', profile_id=1, deps: QualityScannerDeps = None):
-    """Main quality scanner worker function"""
-    from core.wishlist_service import get_wishlist_service
-    from database.music_database import MusicDatabase
-
-    try:
-        with deps.quality_scanner_lock:
-            deps.quality_scanner_state["status"] = "running"
-            deps.quality_scanner_state["phase"] = "Initializing scan..."
-            deps.quality_scanner_state["progress"] = 0
-            deps.quality_scanner_state["processed"] = 0
-            deps.quality_scanner_state["total"] = 0
-            deps.quality_scanner_state["quality_met"] = 0
-            deps.quality_scanner_state["low_quality"] = 0
-            deps.quality_scanner_state["matched"] = 0
-            deps.quality_scanner_state["results"] = []
-            deps.quality_scanner_state["error_message"] = ""
-
-        logger.info(f"[Quality Scanner] Starting scan with scope: {scope}")
-
-        # Get database instance
-        db = MusicDatabase()
-
-        # Get quality profile to determine preferred quality
-        quality_profile = db.get_quality_profile()
-        preferred_qualities = quality_profile.get('qualities', {})
-
-        # Determine minimum acceptable tier based on enabled qualities
-        min_acceptable_tier = 999
-        for quality_name, quality_config in preferred_qualities.items():
-            if quality_config.get('enabled', False):
-                # Map quality profile names to tier names
-                tier_map = {
-                    'flac': 'lossless',
-                    'mp3_320': 'low_lossy',
-                    'mp3_256': 'low_lossy',
-                    'mp3_192': 'low_lossy'
-                }
-                tier_name = tier_map.get(quality_name)
-                if tier_name:
-                    tier_num = deps.QUALITY_TIERS[tier_name]['tier']
-                    min_acceptable_tier = min(min_acceptable_tier, tier_num)
-
-        logger.info(f"[Quality Scanner] Minimum acceptable tier: {min_acceptable_tier}")
-
-        # Get tracks to scan based on scope
-        with deps.quality_scanner_lock:
-            deps.quality_scanner_state["phase"] = "Loading tracks from database..."
-
-        if scope == 'watchlist':
-            # Get watchlist artists
-            watchlist_artists = db.get_watchlist_artists(profile_id=profile_id)
-            if not watchlist_artists:
-                with deps.quality_scanner_lock:
-                    deps.quality_scanner_state["status"] = "finished"
-                    deps.quality_scanner_state["phase"] = "No watchlist artists found"
-                    deps.quality_scanner_state["error_message"] = "Please add artists to watchlist first"
-                logger.warning("[Quality Scanner] No watchlist artists found")
-                return
-
-            # Get artist names from watchlist
-            artist_names = [artist.artist_name for artist in watchlist_artists]
-            logger.info(f"[Quality Scanner] Scanning {len(artist_names)} watchlist artists")
-
-            # Get all tracks for these artists by name
-            conn = db._get_connection()
-            placeholders = ','.join(['?' for _ in artist_names])
-            tracks_to_scan = conn.execute(
-                f"SELECT t.id, t.title, t.artist_id, t.album_id, t.file_path, t.bitrate, a.name as artist_name, al.title as album_title "
-                f"FROM tracks t "
-                f"JOIN artists a ON t.artist_id = a.id "
-                f"JOIN albums al ON t.album_id = al.id "
-                f"WHERE a.name IN ({placeholders}) AND t.file_path IS NOT NULL",
-                artist_names
-            ).fetchall()
-            conn.close()
-        else:
-            # Scan all library tracks
-            with deps.quality_scanner_lock:
-                deps.quality_scanner_state["phase"] = "Loading all library tracks..."
-
-            conn = db._get_connection()
-            tracks_to_scan = conn.execute(
-                "SELECT t.id, t.title, t.artist_id, t.album_id, t.file_path, t.bitrate, a.name as artist_name, al.title as album_title "
-                "FROM tracks t "
-                "JOIN artists a ON t.artist_id = a.id "
-                "JOIN albums al ON t.album_id = al.id "
-                "WHERE t.file_path IS NOT NULL"
-            ).fetchall()
-            conn.close()
-
-        total_tracks = len(tracks_to_scan)
-        logger.info(f"[Quality Scanner] Found {total_tracks} tracks to scan")
-
-        with deps.quality_scanner_lock:
-            deps.quality_scanner_state["total"] = total_tracks
-            deps.quality_scanner_state["phase"] = f"Scanning {total_tracks} tracks..."
-
-        source_priority = get_source_priority(get_primary_source())
-        if not source_priority:
-            with deps.quality_scanner_lock:
-                deps.quality_scanner_state["status"] = "error"
-                deps.quality_scanner_state["phase"] = "No metadata provider available"
-                deps.quality_scanner_state["error_message"] = "No metadata provider is available for quality scanning"
-            logger.info("[Quality Scanner] No metadata provider available")
-            return
-
-        logger.info("[Quality Scanner] Using metadata source priority: %s", source_priority)
-
-        wishlist_service = get_wishlist_service()
-        add_to_wishlist = getattr(wishlist_service, 'add_track_to_wishlist', None)
-        if add_to_wishlist is None:
-            add_to_wishlist = getattr(wishlist_service, 'add_spotify_track_to_wishlist', None)
-        if add_to_wishlist is None:
-            raise AttributeError("Wishlist service does not expose an add-to-wishlist method")
-
-        # Scan each track
-        for idx, track_row in enumerate(tracks_to_scan, 1):
-            # Check for stop request
-            if deps.quality_scanner_state.get('status') != 'running':
-                logger.info(f"[Quality Scanner] Stop requested, halting at track {idx}/{total_tracks}")
-                break
-
-            try:
-                track_id, title, artist_id, album_id, file_path, bitrate, artist_name, album_title = track_row
-
-                # Check quality tier
-                tier_name, tier_num = deps.get_quality_tier_from_extension(file_path)
-
-                # Update progress
-                with deps.quality_scanner_lock:
-                    deps.quality_scanner_state["processed"] = idx
-                    deps.quality_scanner_state["progress"] = (idx / total_tracks) * 100
-                    deps.quality_scanner_state["phase"] = f"Scanning: {artist_name} - {title}"
-
-                # Check if meets quality standards
-                if tier_num <= min_acceptable_tier:
-                    # Quality met
-                    with deps.quality_scanner_lock:
-                        deps.quality_scanner_state["quality_met"] += 1
-                    continue
-
-                # Low quality track found
-                with deps.quality_scanner_lock:
-                    deps.quality_scanner_state["low_quality"] += 1
-
-                logger.info(f"[Quality Scanner] Low quality: {artist_name} - {title} ({tier_name}, {file_path})")
-
-                # Attempt to match using the active metadata provider
-                matched = False
-                matched_track_data = None
-                best_source = None
-                attempted_any_provider = False
-
-                try:
-                    # Generate search queries using matching engine
-                    temp_track = type('TempTrack', (), {
-                        'name': title,
-                        'artists': [artist_name],
-                        'album': album_title
-                    })()
-
-                    search_queries = deps.matching_engine.generate_download_queries(temp_track)
-                    logger.info(f"[Quality Scanner] Generated {len(search_queries)} search queries for {artist_name} - {title}")
-
-                    # Find best match using confidence scoring
-                    best_match = None
-                    best_confidence = 0.0
-                    min_confidence = 0.7  # Match existing standard
-
-                    for _query_idx, search_query in enumerate(search_queries):
-                        try:
-                            for source in source_priority:
-                                client = get_client_for_source(source)
-                                if not client or not hasattr(client, 'search_tracks'):
-                                    continue
-
-                                attempted_any_provider = True
-                                provider_matches = _search_tracks_for_source(source, search_query, limit=5, client=client)
-                                time.sleep(0.5)  # Rate limit metadata API calls
-
-                                if not provider_matches:
-                                    continue
-
-                                # Score each result using matching engine
-                                for provider_track in provider_matches:
-                                    try:
-                                        # Calculate artist confidence
-                                        artist_confidence = 0.0
-                                        provider_artists = _track_artist_names(provider_track)
-                                        if provider_artists:
-                                            for result_artist in provider_artists:
-                                                artist_sim = deps.matching_engine.similarity_score(
-                                                    deps.matching_engine.normalize_string(artist_name),
-                                                    deps.matching_engine.normalize_string(result_artist)
-                                                )
-                                                artist_confidence = max(artist_confidence, artist_sim)
-
-                                        # Calculate title confidence
-                                        title_confidence = deps.matching_engine.similarity_score(
-                                            deps.matching_engine.normalize_string(title),
-                                            deps.matching_engine.normalize_string(_track_name(provider_track))
-                                        )
-
-                                        # Combined confidence (50% artist + 50% title)
-                                        combined_confidence = (artist_confidence * 0.5 + title_confidence * 0.5)
-
-                                        # Small bonus for album tracks over singles
-                                        _at = _extract_lookup_value(provider_track, 'album_type', default='') or ''
-                                        if _at == 'album':
-                                            combined_confidence += 0.02
-                                        elif _at == 'ep':
-                                            combined_confidence += 0.01
-
-                                        candidate_artist = provider_artists[0] if provider_artists else 'Unknown Artist'
-                                        candidate_name = _track_name(provider_track)
-                                        logger.info(
-                                            f"[Quality Scanner] Candidate ({source}): '{candidate_artist}' - "
-                                            f"'{candidate_name}' (confidence: {combined_confidence:.3f})"
-                                        )
-
-                                        # Update best match if this is better
-                                        if combined_confidence > best_confidence and combined_confidence >= min_confidence:
-                                            best_confidence = combined_confidence
-                                            best_match = provider_track
-                                            best_source = source
-                                            logger.info(
-                                                f"[Quality Scanner] New best match ({source}): {candidate_artist} - "
-                                                f"{candidate_name} (confidence: {combined_confidence:.3f})"
-                                            )
-
-                                    except Exception as e:
-                                        logger.error(f"[Quality Scanner] Error scoring result: {e}")
-                                        continue
-
-                                # If we found a very high confidence match, stop searching this query
-                                if best_confidence >= 0.9:
-                                    logger.info(f"[Quality Scanner] High confidence match found ({best_confidence:.3f}), stopping search")
-                                    break
-
-                        except Exception as e:
-                            logger.debug(f"[Quality Scanner] Error searching with query '{search_query}': {e}")
-                            continue
-
-                    if not attempted_any_provider:
-                        with deps.quality_scanner_lock:
-                            deps.quality_scanner_state["status"] = "error"
-                            deps.quality_scanner_state["phase"] = "No metadata provider available"
-                            deps.quality_scanner_state["error_message"] = "No metadata provider is available for quality scanning"
-                        logger.info("[Quality Scanner] No metadata provider available")
-                        return
-
-                    # Process best match
-                    if best_match:
-                        matched = True
-                        final_artist = _track_artist_names(best_match)[0] if _track_artist_names(best_match) else 'Unknown Artist'
-                        final_name = _track_name(best_match)
-                        final_source = best_source or 'metadata'
-                        logger.info(
-                            f"[Quality Scanner] Final match ({final_source}): {final_artist} - "
-                            f"{final_name} (confidence: {best_confidence:.3f})"
-                        )
-
-                        # Build normalized track data for wishlist
-                        matched_track_data = _normalize_track_match(best_match, final_source)
-
-                        # Add to wishlist
-                        source_context = {
-                            'quality_scanner': True,
-                            'original_file_path': file_path,
-                            'original_format': tier_name,
-                            'original_bitrate': bitrate,
-                            'match_confidence': best_confidence,
-                            'scan_date': datetime.now().isoformat()
-                        }
-
-                        success = add_to_wishlist(
-                            track_data=matched_track_data,
-                            failure_reason=f"Low quality - {tier_name.replace('_', ' ').title()} format",
-                            source_type='quality_scanner',
-                            source_context=source_context,
-                            profile_id=profile_id
-                        )
-
-                        if success:
-                            with deps.quality_scanner_lock:
-                                deps.quality_scanner_state["matched"] += 1
-                            logger.info(f"[Quality Scanner] Matched and added to wishlist: {artist_name} - {title}")
-                        else:
-                            logger.error(f"[Quality Scanner] Failed to add to wishlist: {artist_name} - {title}")
-                    else:
-                        logger.warning(
-                            f"[Quality Scanner] No suitable metadata match found "
-                            f"(best confidence: {best_confidence:.3f}, required: {min_confidence:.3f})"
-                        )
-
-                except Exception as matching_error:
-                    logger.error(f"[Quality Scanner] Matching error for {artist_name} - {title}: {matching_error}")
-
-                    # Store result
-                    result_entry = {
-                        'track_id': track_id,
-                        'title': title,
-                        'artist': artist_name,
-                        'album': album_title,
-                        'file_path': file_path,
-                        'current_format': tier_name,
-                        'bitrate': bitrate,
-                        'matched': matched,
-                        'match_id': matched_track_data['id'] if matched_track_data else None,
-                        'provider': best_source if matched else None,
-                        'spotify_id': matched_track_data['id'] if matched_track_data else None,
-                    }
-
-                    with deps.quality_scanner_lock:
-                        deps.quality_scanner_state["results"].append(result_entry)
-
-                    if not matched:
-                        logger.warning(f"[Quality Scanner] No metadata match found for: {artist_name} - {title}")
-
-            except Exception as track_error:
-                logger.error(f"[Quality Scanner] Error processing track: {track_error}")
-                continue
-
-        # Scan complete (don't overwrite if already stopped by user)
-        with deps.quality_scanner_lock:
-            was_stopped = deps.quality_scanner_state["status"] != "running"
-            deps.quality_scanner_state["status"] = "finished"
-            deps.quality_scanner_state["progress"] = 100
-            if not was_stopped:
-                deps.quality_scanner_state["phase"] = "Scan complete"
-
-        logger.info(f"[Quality Scanner] Scan {'stopped' if was_stopped else 'complete'}: {deps.quality_scanner_state['processed']} processed, "
-              f"{deps.quality_scanner_state['low_quality']} low quality, {deps.quality_scanner_state['matched']} matched to metadata providers")
-
-        # Add activity
-        deps.add_activity_item("", "Quality Scan Complete",
-                         f"{deps.quality_scanner_state['matched']} tracks added to wishlist", "Now")
-
-        try:
-            if deps.automation_engine:
-                deps.automation_engine.emit('quality_scan_completed', {
-                    'quality_met': str(deps.quality_scanner_state.get('quality_met', 0)),
-                    'low_quality': str(deps.quality_scanner_state.get('low_quality', 0)),
-                    'total_scanned': str(deps.quality_scanner_state.get('processed', 0)),
-                })
-        except Exception:
-            pass
-
-    except Exception as e:
-        logger.error(f"[Quality Scanner] Critical error: {e}")
-        import traceback
-        traceback.print_exc()
-
-        with deps.quality_scanner_lock:
-            deps.quality_scanner_state["status"] = "error"
-            deps.quality_scanner_state["error_message"] = str(e)
-            deps.quality_scanner_state["phase"] = f"Error: {str(e)}"

core/discovery/spotify_public.py

@@ -55,13 +55,17 @@ class SpotifyPublicDiscoveryDeps:
     search_spotify_for_tidal_track: Callable
     build_discovery_wing_it_stub: Callable
     add_activity_item: Callable
+    source_label: str = "Spotify Public"
+    activity_label: str = "Spotify Link"
+    original_track_key: str = "spotify_public_track"
 
 
 def run_spotify_public_discovery_worker(url_hash, deps: SpotifyPublicDiscoveryDeps):
     """Background worker for Spotify Public discovery process (Spotify preferred, iTunes fallback)"""
     _ew_state = {}
     try:
-        _ew_state = deps.pause_enrichment_workers('Spotify Public discovery')
+        worker_label = f"{deps.source_label} discovery"
+        _ew_state = deps.pause_enrichment_workers(worker_label)
         state = deps.spotify_public_discovery_states[url_hash]
         playlist = state['playlist']
 
@@ -74,7 +78,7 @@ def run_spotify_public_discovery_worker(url_hash, deps: SpotifyPublicDiscoveryDe
         if not use_spotify:
             itunes_client_instance = deps.get_metadata_fallback_client()
 
-        logger.info(f"Starting Spotify Public discovery for: {playlist['name']} (using {discovery_source.upper()})")
+        logger.info(f"Starting {deps.source_label} discovery for: {playlist['name']} (using {discovery_source.upper()})")
 
         # Store discovery source in state for frontend
         state['discovery_source'] = discovery_source
@@ -126,7 +130,7 @@ def run_spotify_public_discovery_worker(url_hash, deps: SpotifyPublicDiscoveryDe
                             cached_album = cached_album.get('name', '')
 
                         result = {
-                            'spotify_public_track': {
+                            deps.original_track_key: {
                                 'id': track_id,
                                 'name': track_name,
                                 'artists': track_artists or [],
@@ -170,7 +174,7 @@ def run_spotify_public_discovery_worker(url_hash, deps: SpotifyPublicDiscoveryDe
 
                 # Create result entry
                 result = {
-                    'spotify_public_track': {
+                    deps.original_track_key: {
                         'id': track_id,
                         'name': track_name,
                         'artists': track_artists or [],
@@ -270,7 +274,7 @@ def run_spotify_public_discovery_worker(url_hash, deps: SpotifyPublicDiscoveryDe
 
                 # Auto Wing It fallback for unmatched tracks
                 if result['status_class'] == 'not-found':
-                    sp_t = result.get('spotify_public_track', {})
+                    sp_t = result.get(deps.original_track_key, {})
                     stub = deps.build_discovery_wing_it_stub(
                         sp_t.get('name', ''),
                         ', '.join(sp_t.get('artists', [])),
@@ -299,7 +303,7 @@ def run_spotify_public_discovery_worker(url_hash, deps: SpotifyPublicDiscoveryDe
                 logger.error(f"Error processing track {i+1}: {e}")
                 # Add error result
                 result = {
-                    'spotify_public_track': {
+                    deps.original_track_key: {
                         'name': sp_track.get('name', 'Unknown'),
                         'artists': sp_track.get('artists', []),
                     },
@@ -324,14 +328,14 @@ def run_spotify_public_discovery_worker(url_hash, deps: SpotifyPublicDiscoveryDe
 
         # Add activity for discovery completion
         source_label = discovery_source.upper()
-        deps.add_activity_item("", f"Spotify Link Discovery Complete ({source_label})", f"'{playlist['name']}' - {successful_discoveries}/{len(tracks)} tracks found", "Now")
+        deps.add_activity_item("", f"{deps.activity_label} Discovery Complete ({source_label})", f"'{playlist['name']}' - {successful_discoveries}/{len(tracks)} tracks found", "Now")
 
-        logger.info(f"Spotify Public discovery complete ({source_label}): {successful_discoveries}/{len(tracks)} tracks found")
+        logger.info(f"{deps.source_label} discovery complete ({source_label}): {successful_discoveries}/{len(tracks)} tracks found")
 
     except Exception as e:
-        logger.error(f"Error in Spotify Public discovery worker: {e}")
+        logger.error(f"Error in {deps.source_label} discovery worker: {e}")
         if url_hash in deps.spotify_public_discovery_states:
             deps.spotify_public_discovery_states[url_hash]['phase'] = 'error'
             deps.spotify_public_discovery_states[url_hash]['status'] = f'error: {str(e)}'
     finally:
-        deps.resume_enrichment_workers(_ew_state, 'Spotify Public discovery')
+        deps.resume_enrichment_workers(_ew_state, f"{deps.source_label} discovery")

core/discovery/sync.py

@@ -39,8 +39,7 @@ class SyncDeps:
     """Bundle of cross-cutting deps the sync worker needs."""
     config_manager: Any
     sync_service: Any
-    plex_client: Any
-    jellyfin_client: Any
+    media_server_engine: Any
     automation_engine: Any
     run_async: Callable[..., Any]
     record_sync_history_start: Callable
@@ -48,9 +47,229 @@ class SyncDeps:
     update_and_save_sync_status: Callable
     sync_states: dict
     sync_lock: Any  # threading.Lock
+    # Optional: post-sync download follow-up for mirrored-playlist automations.
+    process_wishlist_automatically: Callable[..., Any] | None = None
+    run_playlist_organize_download: Callable[..., Any] | None = None
+    is_wishlist_actually_processing: Callable[[], bool] | None = None
 
 
-def run_sync_task(playlist_id, playlist_name, tracks_json, automation_id=None, profile_id=1, playlist_image_url='', deps: SyncDeps = None):
+def _post_sync_automation_followup(
+    deps: SyncDeps,
+    *,
+    automation_id: str,
+    playlist_id: str,
+    skip_wishlist_add: bool,
+    result: Any,
+) -> None:
+    """Queue downloads after an automation sync finishes.
+
+    Sync Playlist runs in a background thread and returns immediately, so a
+    separate scheduled "Process Wishlist" action often runs on an empty wishlist.
+    Organize-by-playlist skips sync-time wishlist adds and expects a folder
+    download batch instead — that only ran in Playlist Pipeline before this hook.
+    """
+    if not automation_id or not str(playlist_id).startswith('auto_mirror_'):
+        return
+    try:
+        mirrored_id = int(str(playlist_id).replace('auto_mirror_', '', 1))
+    except ValueError:
+        return
+
+    failed = int(getattr(result, 'failed_tracks', 0) or 0)
+    wishlist_added = int(getattr(result, 'wishlist_added_count', 0) or 0)
+
+    if skip_wishlist_add:
+        org_fn = deps.run_playlist_organize_download
+        if failed <= 0:
+            return
+        if not org_fn:
+            logger.warning(
+                "Organize-by-playlist sync left %s missing tracks but organize download is unavailable",
+                failed,
+            )
+            deps.update_automation_progress(
+                automation_id,
+                log_line=f'{failed} missing — enable Playlist Pipeline or disable Organize by Playlist',
+                log_type='warning',
+            )
+            return
+        org_result = org_fn(mirrored_playlist_id=mirrored_id, automation_id=automation_id)
+        status = org_result.get('status', 'unknown') if isinstance(org_result, dict) else 'unknown'
+        reason = org_result.get('reason', '') if isinstance(org_result, dict) else ''
+        log_type = 'success' if status == 'started' else 'warning'
+        detail = f' ({reason})' if reason and status != 'started' else ''
+        deps.update_automation_progress(
+            automation_id,
+            log_line=f'Organize download {status} for {failed} missing track(s){detail}',
+            log_type=log_type,
+        )
+        return
+
+    if wishlist_added <= 0:
+        if failed > 0:
+            deps.update_automation_progress(
+                automation_id,
+                log_line=f'{failed} missing but none added to wishlist — check logs',
+                log_type='warning',
+            )
+        return
+
+    proc_fn = deps.process_wishlist_automatically
+    if not proc_fn:
+        return
+    is_busy = deps.is_wishlist_actually_processing
+    if is_busy and is_busy():
+        deps.update_automation_progress(
+            automation_id,
+            log_line=f'Added {wishlist_added} to wishlist; download worker already running',
+            log_type='info',
+        )
+        return
+    proc_fn(automation_id=automation_id)
+    deps.update_automation_progress(
+        automation_id,
+        log_line=f'Started wishlist download for {wishlist_added} track(s)',
+        log_type='success',
+    )
+
+
+async def _database_only_find_track(spotify_track, candidate_pool=None):
+    """Database-only track matcher used when no media server is connected.
+
+    Patched onto sync_service._find_track_in_media_server. Accepts
+    ``candidate_pool`` for interface parity with the real matcher (sync_service
+    calls it with candidate_pool=...); the DB path queries the library directly
+    via check_track_exists, so it doesn't need the per-artist candidate cache —
+    but it MUST accept the kwarg or sync raises "unexpected keyword argument
+    'candidate_pool'". Module-level (not a nested closure) so it's importable
+    and unit-tested.
+    """
+    logger.info(f"Database-only search for: '{spotify_track.name}' by {spotify_track.artists}")
+    try:
+        from database.music_database import MusicDatabase
+        from config.settings import config_manager
+
+        db = MusicDatabase()
+        active_server = config_manager.get_active_media_server()
+        original_title = spotify_track.name
+        spotify_id = getattr(spotify_track, 'id', '') or ''
+
+        # --- Sync match cache fast-path ---
+        if spotify_id:
+            try:
+                cached = db.read_sync_match_cache(spotify_id, active_server)
+                if cached:
+                    db_track_check = db.get_track_by_id(cached['server_track_id'])
+                    if db_track_check:
+                        class DatabaseTrackCached:
+                            def __init__(self, db_t):
+                                self.ratingKey = db_t.id
+                                self.title = db_t.title
+                                self.id = db_t.id
+                        logger.debug(f"Sync cache hit: '{original_title}' → server track {cached['server_track_id']}")
+                        return DatabaseTrackCached(db_track_check), cached['confidence']
+                    logger.warning(f"Sync cache stale for '{original_title}' — track gone")
+            except Exception as e:
+                logger.debug("sync match cache fast-path failed: %s", e)
+        # --- End cache fast-path ---
+
+        # Durable manual library match (#787) — survives a library rescan (the
+        # sync_match_cache above does not), so a user's Find & Add pairing keeps
+        # sticking across auto-syncs instead of being re-matched from scratch (#895
+        # follow-up). Self-heals a stale library id via the stored file path.
+        if spotify_id:
+            try:
+                from core.artists.map import get_current_profile_id
+                m = db.find_manual_library_match_by_source_track_id(
+                    get_current_profile_id(), str(spotify_id), active_server)
+                if m:
+                    lib_id = m.get('library_track_id')
+                    dt = db.get_track_by_id(lib_id) if lib_id is not None else None
+                    if not dt and m.get('library_file_path'):
+                        new_id = db.find_track_id_by_file_path(m['library_file_path'])
+                        dt = db.get_track_by_id(new_id) if new_id else None
+                    if dt:
+                        class DatabaseTrackDurable:
+                            def __init__(self, db_t):
+                                self.ratingKey = db_t.id
+                                self.title = db_t.title
+                                self.id = db_t.id
+                        logger.debug(f"Durable manual match hit: '{original_title}' → {lib_id}")
+                        return DatabaseTrackDurable(dt), 1.0
+            except Exception as e:
+                logger.debug("durable manual match fast-path failed: %s", e)
+
+        # Try each artist
+        for artist in spotify_track.artists:
+            if isinstance(artist, str):
+                artist_name = artist
+            elif isinstance(artist, dict) and 'name' in artist:
+                artist_name = artist['name']
+            else:
+                artist_name = str(artist)
+
+            db_track, confidence = db.check_track_exists(
+                original_title, artist_name,
+                confidence_threshold=0.80,
+                server_source=active_server
+            )
+
+            if not (db_track and confidence >= 0.80):
+                # #785: file/CSV playlists keep raw "Artist - Title" titles (unlike
+                # YouTube, cleaned at ingest), which don't match the clean library
+                # title. Retry with the canonical form (best-of, conservative).
+                try:
+                    from core.text.source_title import canonical_source_track
+                    _canon_title, _canon_artist = canonical_source_track(original_title, artist_name)
+                    if (_canon_title, _canon_artist) != (original_title, artist_name):
+                        _alt_track, _alt_conf = db.check_track_exists(
+                            _canon_title, _canon_artist,
+                            confidence_threshold=0.80, server_source=active_server)
+                        if _alt_track and _alt_conf > confidence:
+                            db_track, confidence = _alt_track, _alt_conf
+                except Exception as _canon_err:
+                    logger.debug("canonical retry failed: %s", _canon_err)
+
+            if db_track and confidence >= 0.80:
+                logger.info(f"Database match: '{db_track.title}' (confidence: {confidence:.2f})")
+                if spotify_id:
+                    try:
+                        from core.matching_engine import MusicMatchingEngine
+                        me = MusicMatchingEngine()
+                        db.save_sync_match_cache(
+                            spotify_id, me.clean_title(original_title), me.clean_artist(artist_name),
+                            active_server, db_track.id, db_track.title, confidence
+                        )
+                    except Exception as e:
+                        logger.debug("save sync match cache failed: %s", e)
+
+                class DatabaseTrackMock:
+                    def __init__(self, db_track):
+                        self.ratingKey = db_track.id
+                        self.title = db_track.title
+                        self.id = db_track.id
+
+                return DatabaseTrackMock(db_track), confidence
+
+        logger.warning(f"No database match found for: '{original_title}'")
+        return None, 0.0
+
+    except Exception as e:
+        logger.error(f"Database search error: {e}")
+        return None, 0.0
+
+
+def run_sync_task(
+    playlist_id,
+    playlist_name,
+    tracks_json,
+    automation_id=None,
+    profile_id=1,
+    playlist_image_url='',
+    deps: SyncDeps = None,
+    sync_mode: str = 'replace',
+    skip_wishlist_add: bool = False,
+):
     """The actual sync function that runs in the background thread."""
     sync_states = deps.sync_states
     sync_lock = deps.sync_lock
@@ -88,35 +307,11 @@ def run_sync_task(playlist_id, playlist_name, tracks_json, automation_id=None, p
         # This avoids needing to re-fetch it from Spotify
         logger.info("Converting JSON tracks to SpotifyTrack objects...")
 
-        # Store original track data with full album objects (for wishlist with cover art)
-        # Normalize formats for wishlist: album must be dict {'name': ...}, artists must be [{'name': ...}]
-        # Important: copy data — don't mutate tracks_json since SpotifyTrack expects List[str] artists
-        original_tracks_map = {}
-        for t in tracks_json:
-            track_id = t.get('id', '')
-            if track_id:
-                normalized = dict(t)
-                # Normalize album to dict format, preserving images and metadata
-                raw_album = normalized.get('album', '')
-                if isinstance(raw_album, str):
-                    normalized['album'] = {
-                        'name': raw_album or normalized.get('name', 'Unknown Album'),
-                        'images': [], 'album_type': 'single', 'total_tracks': 1, 'release_date': ''
-                    }
-                elif not isinstance(raw_album, dict):
-                    normalized['album'] = {
-                        'name': str(raw_album) if raw_album else normalized.get('name', 'Unknown Album'),
-                        'images': [], 'album_type': 'single', 'total_tracks': 1, 'release_date': ''
-                    }
-                else:
-                    # Dict — ensure required keys exist
-                    raw_album.setdefault('name', 'Unknown Album')
-                    raw_album.setdefault('images', [])
-                # Normalize artists to list of dicts
-                raw_artists = normalized.get('artists', [])
-                if raw_artists and isinstance(raw_artists[0], str):
-                    normalized['artists'] = [{'name': a} for a in raw_artists]
-                original_tracks_map[track_id] = normalized
+        # Store original track data with full album objects (for wishlist with cover art).
+        # Shared with the sync-detail "re-add to wishlist" action so both build the
+        # IDENTICAL payload (album→dict + images, artists→dicts). Copy-safe.
+        from core.sync.wishlist_readd import build_original_tracks_map
+        original_tracks_map = build_original_tracks_map(tracks_json)
 
         tracks = []
         for i, t in enumerate(tracks_json):
@@ -227,8 +422,9 @@ def run_sync_task(playlist_id, playlist_name, tracks_json, automation_id=None, p
             
         # Check sync service components
         logger.info(f"   spotify_client: {sync_service.spotify_client is not None}")
-        logger.info(f"   deps.plex_client: {sync_service.plex_client is not None}")
-        logger.info(f"   deps.jellyfin_client: {sync_service.jellyfin_client is not None}")
+        _ms_engine = getattr(sync_service, '_engine', None)
+        logger.info(f"   plex_client: {(_ms_engine.client('plex') if _ms_engine else None) is not None}")
+        logger.info(f"   jellyfin_client: {(_ms_engine.client('jellyfin') if _ms_engine else None) is not None}")
         
         # Check media server connection before starting
         from config.settings import config_manager
@@ -260,89 +456,9 @@ def run_sync_task(playlist_id, playlist_name, tracks_json, automation_id=None, p
         if media_client is None or not media_client.is_connected():
             logger.info("Media client not connected - patching sync service for database-only matching")
             
-            # Store original method
-            original_find_track = sync_service._find_track_in_media_server
-            
-            # Create database-only replacement method
-            async def database_only_find_track(spotify_track):
-                logger.info(f"Database-only search for: '{spotify_track.name}' by {spotify_track.artists}")
-                try:
-                    from database.music_database import MusicDatabase
-                    from config.settings import config_manager
-
-                    db = MusicDatabase()
-                    active_server = config_manager.get_active_media_server()
-                    original_title = spotify_track.name
-                    spotify_id = getattr(spotify_track, 'id', '') or ''
-
-                    # --- Sync match cache fast-path ---
-                    if spotify_id:
-                        try:
-                            cached = db.read_sync_match_cache(spotify_id, active_server)
-                            if cached:
-                                db_track_check = db.get_track_by_id(cached['server_track_id'])
-                                if db_track_check:
-                                    class DatabaseTrackCached:
-                                        def __init__(self, db_t):
-                                            self.ratingKey = db_t.id
-                                            self.title = db_t.title
-                                            self.id = db_t.id
-                                    logger.debug(f"Sync cache hit: '{original_title}' → server track {cached['server_track_id']}")
-                                    return DatabaseTrackCached(db_track_check), cached['confidence']
-                                logger.warning(f"Sync cache stale for '{original_title}' — track gone")
-                        except Exception:
-                            pass
-                    # --- End cache fast-path ---
-
-                    # Try each artist (same logic as original)
-                    for artist in spotify_track.artists:
-                        # Extract artist name from both string and dict formats
-                        if isinstance(artist, str):
-                            artist_name = artist
-                        elif isinstance(artist, dict) and 'name' in artist:
-                            artist_name = artist['name']
-                        else:
-                            artist_name = str(artist)
-
-                        db_track, confidence = db.check_track_exists(
-                            original_title, artist_name,
-                            confidence_threshold=0.80,
-                            server_source=active_server
-                        )
-
-                        if db_track and confidence >= 0.80:
-                            logger.info(f"Database match: '{db_track.title}' (confidence: {confidence:.2f})")
-
-                            # Save to sync match cache
-                            if spotify_id:
-                                try:
-                                    from core.matching_engine import MusicMatchingEngine
-                                    me = MusicMatchingEngine()
-                                    db.save_sync_match_cache(
-                                        spotify_id, me.clean_title(original_title), me.clean_artist(artist_name),
-                                        active_server, db_track.id, db_track.title, confidence
-                                    )
-                                except Exception:
-                                    pass
-
-                            # Create mock track object for playlist creation
-                            class DatabaseTrackMock:
-                                def __init__(self, db_track):
-                                    self.ratingKey = db_track.id
-                                    self.title = db_track.title
-                                    self.id = db_track.id
-
-                            return DatabaseTrackMock(db_track), confidence
-
-                    logger.warning(f"No database match found for: '{original_title}'")
-                    return None, 0.0
-
-                except Exception as e:
-                    logger.error(f"Database search error: {e}")
-                    return None, 0.0
-            
-            # Patch the method
-            sync_service._find_track_in_media_server = database_only_find_track
+            # Patch the matcher to the module-level database-only implementation
+            # (importable + unit-tested; accepts candidate_pool for parity).
+            sync_service._find_track_in_media_server = _database_only_find_track
             logger.info("Patched sync service to use database-only matching")
 
         sync_start_time = time.time()
@@ -356,10 +472,17 @@ def run_sync_task(playlist_id, playlist_name, tracks_json, automation_id=None, p
         # Wing It mode — skip wishlist for unmatched tracks
         with sync_lock:
             is_wing_it = sync_states.get(playlist_id, {}).get('wing_it', False)
+        sync_service._skip_unmatched_wishlist = is_wing_it or skip_wishlist_add
         sync_service._skip_wishlist = is_wing_it
+        if skip_wishlist_add:
+            logger.info(
+                "[Organize by Playlist] Skipping sync-time wishlist for '%s' — "
+                "organize download + batch failure handling cover missing tracks",
+                playlist_name,
+            )
 
         # Run the sync (this is a blocking call within this thread)
-        result = deps.run_async(sync_service.sync_playlist(playlist, download_missing=False, profile_id=profile_id))
+        result = deps.run_async(sync_service.sync_playlist(playlist, download_missing=False, profile_id=profile_id, sync_mode=sync_mode))
 
         # Clear progress callback immediately to prevent race condition where a
         # late-firing progress callback overwrites the "finished" state below
@@ -397,18 +520,28 @@ def run_sync_task(playlist_id, playlist_name, tracks_json, automation_id=None, p
             }
         logger.info(f"Sync finished for {playlist_id} - state updated")
 
-        # Set playlist poster image if available (Plex, Jellyfin, Emby)
+        # Set playlist poster image if available (Plex, Jellyfin, Emby).
+        # Don't log the URL itself — it may carry an auth token (Plex
+        # X-Plex-Token / Jellyfin X-Emby-Token / Subsonic auth) that we
+        # don't want persisted to app.log.
         _synced = getattr(result, 'synced_tracks', 0)
-        logger.info(f"[PLAYLIST IMAGE] image_url={playlist_image_url!r}, synced_tracks={_synced}")
-        if playlist_image_url and _synced > 0:
+        logger.info(f"[PLAYLIST IMAGE] has_image={bool(playlist_image_url)}, synced_tracks={_synced}")
+        # Modes that edit a playlist in place (reconcile #792, append #811) must
+        # NOT push the source image — doing so re-clobbers a user's custom poster
+        # every sync, the exact bug these modes exist to avoid. Only the
+        # destructive 'replace' (recreate-from-scratch) pushes the image.
+        if sync_mode in ('reconcile', 'append'):
+            logger.info(f"[PLAYLIST IMAGE] {sync_mode} mode — preserving existing playlist image")
+        elif playlist_image_url and _synced > 0:
             try:
                 active_server = deps.config_manager.get_active_media_server()
                 logger.info(f"[PLAYLIST IMAGE] active_server={active_server}")
-                if active_server == 'plex' and deps.plex_client:
-                    ok = deps.plex_client.set_playlist_image(playlist_name, playlist_image_url)
+                _engine = deps.media_server_engine
+                if active_server == 'plex' and _engine and _engine.client('plex'):
+                    ok = _engine.client('plex').set_playlist_image(playlist_name, playlist_image_url)
                     logger.info(f"[PLAYLIST IMAGE] Plex upload result: {ok}")
-                elif active_server in ('jellyfin', 'emby') and deps.jellyfin_client:
-                    ok = deps.jellyfin_client.set_playlist_image(playlist_name, playlist_image_url)
+                elif active_server in ('jellyfin', 'emby') and _engine and _engine.client('jellyfin'):
+                    ok = _engine.client('jellyfin').set_playlist_image(playlist_name, playlist_image_url)
                     logger.info(f"[PLAYLIST IMAGE] Jellyfin upload result: {ok}")
                 # Navidrome doesn't support custom playlist images
             except Exception as img_err:
@@ -428,8 +561,8 @@ def run_sync_task(playlist_id, playlist_name, tracks_json, automation_id=None, p
                     entry = db.get_sync_history_entry(_resync_entry_id)
                     if entry:
                         target_batch_id = entry.get('batch_id', sync_batch_id)
-                except Exception:
-                    pass
+                except Exception as e:
+                    logger.debug("resync history lookup failed: %s", e)
             else:
                 db.update_sync_history_completion(sync_batch_id, matched, synced, failed)
 
@@ -451,9 +584,21 @@ def run_sync_task(playlist_id, playlist_name, tracks_json, automation_id=None, p
             matched = getattr(result, 'matched_tracks', 0)
             total = getattr(result, 'total_tracks', 0)
             failed = getattr(result, 'failed_tracks', 0)
+            wishlist_added = getattr(result, 'wishlist_added_count', 0) or 0
             deps.update_automation_progress(automation_id, status='finished', progress=100,
                 phase='Sync complete',
-                log_line=f'Done: {matched}/{total} matched, {failed} failed', log_type='success')
+                log_line=(
+                    f'Done: {matched}/{total} in library, {failed} missing'
+                    + (f', {wishlist_added} added to wishlist' if wishlist_added else '')
+                ),
+                log_type='success')
+            _post_sync_automation_followup(
+                deps,
+                automation_id=automation_id,
+                playlist_id=playlist_id,
+                skip_wishlist_add=skip_wishlist_add,
+                result=result,
+            )
 
         # Emit playlist_synced event for automation engine
         try:
@@ -465,19 +610,35 @@ def run_sync_task(playlist_id, playlist_name, tracks_json, automation_id=None, p
                     'synced_tracks': str(getattr(result, 'synced_tracks', 0)),
                     'failed_tracks': str(getattr(result, 'failed_tracks', 0)),
                 })
-        except Exception:
-            pass
+        except Exception as e:
+            logger.debug("playlist_synced emit failed: %s", e)
 
         # Save sync status with match counts and track hash for smart-skip on next scheduled sync
         import hashlib as _hl
         _track_ids_str = ','.join(sorted(t.get('id', '') for t in tracks_json))
         _tracks_hash = _hl.md5(_track_ids_str.encode()).hexdigest()
+        _mirror_tracks_hash = None
+        if str(playlist_id).startswith('auto_mirror_'):
+            try:
+                _mp_id = int(str(playlist_id).replace('auto_mirror_', '', 1))
+                from database.music_database import MusicDatabase
+                _mtracks = MusicDatabase().get_mirrored_playlist_tracks(_mp_id)
+                _mids = ','.join(
+                    sorted(t.get('source_track_id', '') or '' for t in _mtracks if t.get('source_track_id'))
+                )
+                _mirror_tracks_hash = _hl.md5(_mids.encode()).hexdigest() if _mids else ''
+            except Exception as e:
+                logger.debug("mirror_tracks_hash for sync status: %s", e)
         snapshot_id = getattr(playlist, 'snapshot_id', None)
-        deps.update_and_save_sync_status(playlist_id, playlist_name, playlist.owner, snapshot_id,
+        _status_kwargs = dict(
             matched_tracks=getattr(result, 'matched_tracks', 0),
             total_tracks=getattr(result, 'total_tracks', 0),
             discovered_tracks=len(tracks_json),
-            tracks_hash=_tracks_hash)
+            tracks_hash=_tracks_hash,
+        )
+        if _mirror_tracks_hash is not None:
+            _status_kwargs['mirror_tracks_hash'] = _mirror_tracks_hash
+        deps.update_and_save_sync_status(playlist_id, playlist_name, playlist.owner, snapshot_id, **_status_kwargs)
 
     except Exception as e:
         logger.error(f"SYNC FAILED for {playlist_id}: {e}")

core/discovery/youtube.py

@@ -32,6 +32,26 @@ from typing import Any, Callable
 
 logger = logging.getLogger(__name__)
 
+_UNKNOWN_ARTIST = 'Unknown Artist'
+
+
+def resolve_display_artist(yt_artist: str, matched_artist: str) -> str:
+    """The artist to show in the 'YT Artist' column (#909).
+
+    YouTube's flat playlist data carries no artist, so a track starts as
+    "Unknown Artist" and only gains a real name if per-video recovery succeeds.
+    When recovery comes up empty but the track still matched confidently, show
+    the matched artist instead of a misleading "Unknown Artist". Returns the
+    original ``yt_artist`` whenever it's already a real name (recovery worked) or
+    when there's no matched artist to fall back to — purely a display choice, the
+    match itself is unaffected.
+    """
+    current = (yt_artist or '').strip()
+    if current and current != _UNKNOWN_ARTIST:
+        return current                      # recovery already gave a real name — keep it
+    fallback = (matched_artist or '').strip()
+    return fallback or _UNKNOWN_ARTIST      # backfill from the match, else honest Unknown
+
 
 @dataclass
 class YoutubeDiscoveryDeps:
@@ -52,6 +72,10 @@ class YoutubeDiscoveryDeps:
     build_discovery_wing_it_stub: Callable
     get_database: Callable[[], Any]
     add_activity_item: Callable
+    # Recover a YouTube track's artist from its own video page when flat playlist
+    # extraction left it "Unknown Artist" (#863). Takes a video id, returns a raw
+    # artist string or ''. Optional — discovery still works without it.
+    recover_youtube_artist: Callable[[str], str] = None
 
 
 def run_youtube_discovery_worker(url_hash, deps: YoutubeDiscoveryDeps):
@@ -94,6 +118,30 @@ def run_youtube_discovery_worker(url_hash, deps: YoutubeDiscoveryDeps):
                 cleaned_title = track['name']
                 cleaned_artist = track['artists'][0] if track['artists'] else 'Unknown Artist'
 
+                # Recover the artist from the track's own video page if flat
+                # playlist extraction left it Unknown (#863). Done here, in the
+                # background worker, rather than in the parse request (which would
+                # block for minutes on a big playlist). Per-track cost is hidden
+                # behind the discovery progress bar; the recovered artist makes the
+                # match below actually find the song.
+                if cleaned_artist == 'Unknown Artist' and track.get('id'):
+                    if not deps.recover_youtube_artist:
+                        logger.warning("[YT Discovery] artist recovery unavailable (dep not wired) "
+                                       "— '%s' stays Unknown", cleaned_title)
+                    else:
+                        try:
+                            _rec = deps.recover_youtube_artist(track['id'])
+                        except Exception as _rec_err:
+                            logger.warning(f"[YT Discovery] artist recovery raised for {track.get('id')}: {_rec_err}")
+                            _rec = ''
+                        if _rec and _rec != 'Unknown Artist':
+                            logger.info(f"[YT Discovery] recovered artist '{_rec}' for '{cleaned_title}' ({track['id']})")
+                            cleaned_artist = _rec
+                            track['artists'] = [_rec]  # persist so retries/UI see it
+                        else:
+                            logger.info(f"[YT Discovery] artist recovery returned nothing for "
+                                        f"'{cleaned_title}' ({track['id']}) — leaving Unknown")
+
                 logger.info(f"Searching {discovery_source} for: '{cleaned_artist}' - '{cleaned_title}'")
 
                 # Check discovery cache first
@@ -103,14 +151,15 @@ def run_youtube_discovery_worker(url_hash, deps: YoutubeDiscoveryDeps):
                     cached_match = cache_db.get_discovery_cache_match(cache_key[0], cache_key[1], discovery_source)
                     if cached_match and deps.validate_discovery_cache_artist(cleaned_artist, cached_match):
                         logger.debug(f"CACHE HIT [{i+1}/{len(tracks)}]: {cleaned_artist} - {cleaned_title}")
+                        _match_artist = deps.extract_artist_name(cached_match.get('artists', [''])[0]) if cached_match.get('artists') else ''
                         result = {
                             'index': i,
                             'yt_track': cleaned_title,
-                            'yt_artist': cleaned_artist,
+                            'yt_artist': resolve_display_artist(cleaned_artist, _match_artist),
                             'status': 'Found',
                             'status_class': 'found',
                             'spotify_track': cached_match.get('name', ''),
-                            'spotify_artist': deps.extract_artist_name(cached_match.get('artists', [''])[0]) if cached_match.get('artists') else '',
+                            'spotify_artist': _match_artist,
                             'spotify_album': cached_match.get('album', {}).get('name', '') if isinstance(cached_match.get('album'), dict) else cached_match.get('album', ''),
                             'duration': f"{int(track['duration_ms']) // 60000}:{(int(track['duration_ms']) % 60000) // 1000:02d}" if track['duration_ms'] else '0:00',
                             'discovery_source': discovery_source,
@@ -237,15 +286,17 @@ def run_youtube_discovery_worker(url_hash, deps: YoutubeDiscoveryDeps):
                             best_confidence = confidence
                             logger.info(f"Strategy 4 YouTube match (extended): {match.artists[0]} - {match.name} (confidence: {confidence:.3f})")
 
-                # Create result entry
+                # Create result entry. yt_artist falls back to the matched artist when
+                # YouTube/recovery left it "Unknown Artist" but we matched confidently (#909).
+                _match_artist = deps.extract_artist_name(matched_track.artists[0]) if matched_track else ''
                 result = {
                     'index': i,
                     'yt_track': cleaned_title,
-                    'yt_artist': cleaned_artist,
+                    'yt_artist': resolve_display_artist(cleaned_artist, _match_artist),
                     'status': 'Found' if matched_track else 'Not Found',
                     'status_class': 'found' if matched_track else 'not-found',
                     'spotify_track': matched_track.name if matched_track else '',
-                    'spotify_artist': deps.extract_artist_name(matched_track.artists[0]) if matched_track else '',
+                    'spotify_artist': _match_artist,
                     'spotify_album': matched_track.album if matched_track else '',
                     'duration': f"{int(track['duration_ms']) // 60000}:{(int(track['duration_ms']) % 60000) // 1000:02d}" if track['duration_ms'] else '0:00',
                     'discovery_source': discovery_source,

core/download_engine/__init__.py

@@ -0,0 +1,31 @@
+"""Download Engine — central owner of cross-source download state,
+thread workers, search retry, rate-limits, and fallback chains.
+
+This is the second leg of the multi-source download dispatcher
+refactor (the first leg, ``core/download_plugins/``, defined the
+contract). The engine takes ownership of everything that used to
+be duplicated across the per-source clients (background thread
+workers, active_downloads dicts, search retry ladders, quality
+filtering, hybrid fallback). Clients become DUMB — just hit the
+API for their source, manage their own auth state, and let the
+engine drive everything else.
+
+This package is built up in phases (see
+``docs/download-engine-refactor-plan.md`` for the full plan):
+
+- Phase B (current) — engine skeleton + state lift.
+- Phase C — background download worker.
+- Phase D — search retry + quality filter.
+- Phase E — rate-limit pool.
+- Phase F — fallback chain.
+
+Each phase is purely additive at first (engine grows, clients
+unchanged). Migration to the new shape happens one source per
+commit so behavior never breaks across the suite.
+"""
+
+from core.download_engine.engine import DownloadEngine
+from core.download_engine.rate_limit import RateLimitPolicy
+from core.download_engine.worker import BackgroundDownloadWorker
+
+__all__ = ["DownloadEngine", "BackgroundDownloadWorker", "RateLimitPolicy"]

core/download_engine/engine.py

@@ -0,0 +1,538 @@
+"""DownloadEngine — central owner of cross-source download state.
+
+Phase B scope: skeleton only. The engine exposes a place for
+plugins to register, a single ``active_downloads`` dict keyed by
+``(source, download_id)``, and per-source RLocks that guard mutations
+without serializing workers across different sources.
+
+Subsequent phases bolt more capability on top:
+- ``dispatch_download(plugin, target_id)`` (Phase C — replaces every
+  client's ``_download_thread_worker`` boilerplate).
+- ``search(query, source_chain)`` (Phase D — replaces every client's
+  retry ladder + quality filter).
+- ``rate_limit.acquire(source)`` (Phase E — replaces every client's
+  semaphore + last-download-timestamp dance).
+- ``search_with_fallback`` / ``download_with_fallback`` (Phase F —
+  unifies hybrid mode across search and download).
+
+The engine is constructed by ``DownloadOrchestrator.__init__`` and
+each plugin from the registry is registered with it. In Phase B
+nothing in the existing code paths goes through the engine yet —
+this commit is pure additive scaffolding so subsequent commits can
+introduce engine-driven behavior one piece at a time without a
+big-bang switchover.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import threading
+from typing import Any, Dict, Iterator, List, Optional, Tuple
+
+from utils.logging_config import get_logger
+
+logger = get_logger("download_engine")
+
+
+# Type alias for the per-download state dict. Today's clients each
+# define their own slightly-different shape (see Phase A pinning
+# tests); the engine stores them as opaque dicts and the per-plugin
+# accessor preserves the source-specific fields.
+DownloadRecord = Dict[str, Any]
+
+
+class DownloadEngine:
+    """Central state for every active download across every source.
+
+    State is keyed by ``(source_name, download_id)`` so the same
+    UUID could hypothetically appear in two sources without
+    collision (in practice each source generates its own UUID4
+    so collisions are negligible — the source qualifier exists
+    so the engine can answer "which plugin owns this download" in
+    O(1) without iterating every plugin).
+
+    Thread safety: per-source lock sharding. Each source gets its own
+    RLock — progress callbacks on Deezer don't block Tidal's worker
+    and vice versa, matching the pre-refactor behavior where each
+    client owned its own download lock. Read-only accessors
+    (``get_record``, ``iter_records_for_source``) take the source's
+    lock briefly and return a SHALLOW COPY so the caller can iterate
+    without holding the lock. Callers that need to mutate a record
+    should use ``update_record`` which takes the lock and applies the
+    patch atomically.
+    """
+
+    def __init__(self) -> None:
+        # Nested dict: source_name → {download_id → record}. Replaces
+        # the original single-dict composite-key layout so
+        # ``iter_records_for_source`` is O(source_records) instead of
+        # O(total_records).
+        self._records: Dict[str, Dict[str, DownloadRecord]] = {}
+        # Per-source RLocks. Each source gets its own so progress
+        # updates on one source never block writes on another. RLock
+        # so a plugin's worker callback can re-enter while holding the
+        # lock for its own update. Lazily created via ``_source_lock``;
+        # the meta-lock guards creation against the create-race window
+        # where two threads could both miss + both create.
+        self._source_locks: Dict[str, threading.RLock] = {}
+        self._source_locks_lock = threading.Lock()
+        # Plugins that have registered with the engine. Source name
+        # → plugin instance.
+        self._plugins: Dict[str, Any] = {}
+        # Alias → canonical-name map. Lets engine resolve legacy
+        # source-name strings (e.g. ``'deezer_dl'`` for Deezer) to
+        # the canonical key in ``_plugins``. Cin's review caught
+        # that engine.cancel_download(source_hint='deezer_dl')
+        # silently fell through to Soulseek because alias resolution
+        # only existed at the registry, not on the engine.
+        self._aliases: Dict[str, str] = {}
+        # Background download worker — lives on the engine because
+        # it owns the cross-source state the worker mutates. Lazy
+        # import keeps the engine module standalone.
+        from core.download_engine.worker import BackgroundDownloadWorker
+        self.worker = BackgroundDownloadWorker(self)
+
+    # ------------------------------------------------------------------
+    # Plugin registration
+    # ------------------------------------------------------------------
+
+    def register_plugin(self, source_name: str, plugin: Any,
+                        aliases: Tuple[str, ...] = ()) -> None:
+        """Register a plugin under its canonical source name. Called
+        once per source by the orchestrator after the registry's
+        ``initialize`` builds the client instances.
+
+        ``aliases`` is the list of legacy source-name strings that
+        should resolve to this plugin (e.g. ``'deezer_dl'`` for
+        Deezer). Without alias resolution the engine couldn't route
+        cancel/lookup calls that came in with the legacy name.
+
+        If the plugin exposes ``set_engine(engine)``, the engine
+        passes a self-reference so the plugin can dispatch into
+        ``engine.worker`` / read state / etc. Plugins that haven't
+        been migrated to the engine yet simply don't define
+        ``set_engine`` — they keep their pre-engine behavior
+        unchanged.
+
+        Also reads the plugin's declared ``RateLimitPolicy`` (via
+        the ``rate_limit_policy()`` method or ``RATE_LIMIT_POLICY``
+        class attribute) and applies it to the worker. Plugins that
+        don't declare a policy get the conservative default
+        (concurrency=1, delay=0).
+        """
+        if source_name in self._plugins:
+            logger.warning("Plugin %s already registered with engine — overwriting", source_name)
+        self._plugins[source_name] = plugin
+        for alias in aliases:
+            self._aliases[alias] = source_name
+
+        # Apply the plugin's rate-limit policy BEFORE set_engine so
+        # set_engine callbacks can override per-source if they need
+        # config-driven values (e.g. YouTube's user-tunable delay).
+        from core.download_engine.rate_limit import resolve_policy
+        policy = resolve_policy(plugin)
+        self.worker.set_concurrency(source_name, policy.download_concurrency)
+        self.worker.set_delay(source_name, policy.download_delay_seconds)
+
+        set_engine = getattr(plugin, 'set_engine', None)
+        if callable(set_engine):
+            try:
+                set_engine(self)
+            except Exception as exc:
+                logger.warning(
+                    "Plugin %s set_engine callback failed: %s", source_name, exc,
+                )
+
+    def get_plugin(self, source_name: str) -> Optional[Any]:
+        """Return the plugin instance for the given source name.
+        Resolves through aliases — e.g. ``get_plugin('deezer_dl')``
+        returns the same instance as ``get_plugin('deezer')``."""
+        if source_name in self._plugins:
+            return self._plugins[source_name]
+        canonical = self._aliases.get(source_name)
+        if canonical:
+            return self._plugins.get(canonical)
+        return None
+
+    def _resolve_canonical(self, source_name: str) -> Optional[str]:
+        """Return the canonical source name for an input that may be
+        an alias. Returns None if the input matches neither a
+        canonical name nor an alias."""
+        if source_name in self._plugins:
+            return source_name
+        return self._aliases.get(source_name)
+
+    def registered_sources(self) -> List[str]:
+        return list(self._plugins.keys())
+
+    def _source_lock(self, source_name: str) -> threading.RLock:
+        """Return the per-source RLock, lazy-creating it on first use.
+        The meta-lock around the cache lookup closes the create-race
+        window where two threads both miss + both create a fresh lock.
+        """
+        with self._source_locks_lock:
+            lock = self._source_locks.get(source_name)
+            if lock is None:
+                lock = threading.RLock()
+                self._source_locks[source_name] = lock
+            return lock
+
+    # ------------------------------------------------------------------
+    # Active-downloads state — Phase B core surface
+    # ------------------------------------------------------------------
+
+    def add_record(self, source_name: str, download_id: str, record: DownloadRecord) -> None:
+        """Insert a fresh download record. Used by clients (today
+        directly via their own dicts; Phase B2 routes them through
+        here)."""
+        with self._source_lock(source_name):
+            source_bucket = self._records.setdefault(source_name, {})
+            if download_id in source_bucket:
+                logger.warning("Replacing existing download record for %s/%s", source_name, download_id)
+            source_bucket[download_id] = dict(record)
+
+    def update_record(self, source_name: str, download_id: str, patch: DownloadRecord) -> None:
+        """Apply a partial patch to an existing record. No-op if the
+        record was already removed (e.g. cancelled mid-update)."""
+        with self._source_lock(source_name):
+            existing = self._records.get(source_name, {}).get(download_id)
+            if existing is None:
+                return
+            existing.update(patch)
+
+    def update_record_unless_state(self, source_name: str, download_id: str,
+                                   patch: DownloadRecord,
+                                   skip_if_state_in: Tuple[str, ...] = ()) -> bool:
+        """Atomically check the record's state and apply ``patch`` only
+        if the current state is NOT in ``skip_if_state_in``. Returns
+        True if the patch was applied, False if it was skipped (or
+        the record didn't exist).
+
+        Used by the background download worker's ``_mark_terminal``
+        to avoid the read-then-write race Cin flagged: a cancel
+        landing between the snapshot and update could be overwritten
+        back to Errored / Completed. Holding the source's lock across
+        the check + write closes the window.
+        """
+        with self._source_lock(source_name):
+            existing = self._records.get(source_name, {}).get(download_id)
+            if existing is None:
+                return False
+            if existing.get('state') in skip_if_state_in:
+                return False
+            existing.update(patch)
+            return True
+
+    def remove_record(self, source_name: str, download_id: str) -> Optional[DownloadRecord]:
+        """Delete a record (cancellation cleanup). Returns the
+        removed record or None if not found."""
+        with self._source_lock(source_name):
+            source_bucket = self._records.get(source_name)
+            if not source_bucket:
+                return None
+            removed = source_bucket.pop(download_id, None)
+            # Drop the empty source bucket so iteration / membership
+            # checks don't see a stale source key.
+            if not source_bucket:
+                self._records.pop(source_name, None)
+            return removed
+
+    def get_record(self, source_name: str, download_id: str) -> Optional[DownloadRecord]:
+        """Return a SHALLOW COPY of the record. Caller mutations
+        don't affect engine state — use ``update_record`` for that."""
+        with self._source_lock(source_name):
+            record = self._records.get(source_name, {}).get(download_id)
+            return dict(record) if record is not None else None
+
+    def iter_records_for_source(self, source_name: str) -> Iterator[DownloadRecord]:
+        """Yield SHALLOW COPIES of every record owned by a source.
+        Holds the source's lock briefly to snapshot, then yields
+        outside the lock so callers can spend arbitrary time on each
+        record.
+
+        With the nested-dict layout this is O(source_records) — only
+        touches the bucket for the requested source, not every record
+        across every source.
+        """
+        with self._source_lock(source_name):
+            source_bucket = self._records.get(source_name, {})
+            snapshot = [dict(record) for record in source_bucket.values()]
+        for record in snapshot:
+            yield record
+
+    # ------------------------------------------------------------------
+    # Cross-source query dispatch — Phase B2 surface
+    # ------------------------------------------------------------------
+    #
+    # The orchestrator historically iterated every plugin in its own
+    # ``get_all_downloads`` / ``get_download_status`` / ``cancel_download``
+    # methods (with hand-maintained client lists, before the registry
+    # came along). That iteration logic moves into the engine here so
+    # the orchestrator becomes a thin pass-through (Phase B3).
+    #
+    # In Phase B these methods iterate the registered plugins and call
+    # their existing ``get_all_downloads`` / ``cancel_download``
+    # methods — same behavior as today, just in a new home. Phase C/D
+    # will replace plugin-iteration with direct engine-state queries
+    # once the thread worker is also lifted.
+    #
+    # All methods are async to match the per-plugin contract.
+
+    async def get_all_downloads(self, exclude: Tuple[str, ...] = ()):
+        """Aggregated view across every registered plugin's active
+        downloads. Per-plugin exceptions are swallowed (one source
+        failing shouldn't take down cross-source aggregation) but
+        logged at debug level — same defensive shape the legacy
+        orchestrator had.
+
+        ``exclude`` skips named sources entirely. The download monitor
+        passes ``('soulseek',)`` so it doesn't double-fetch slskd
+        transfers (it already pulled them via the slskd transfers
+        endpoint earlier in the same loop).
+        """
+        all_downloads = []
+        for source_name, plugin in self._plugins.items():
+            if plugin is None or source_name in exclude:
+                continue
+            try:
+                all_downloads.extend(await plugin.get_all_downloads())
+            except Exception as exc:
+                logger.debug("%s get_all_downloads failed: %s", source_name, exc)
+        return all_downloads
+
+    async def get_download_status(self, download_id: str):
+        """Find a download_id across every plugin. Returns the first
+        plugin's response or None if no plugin owns it."""
+        for source_name, plugin in self._plugins.items():
+            if plugin is None:
+                continue
+            try:
+                status = await plugin.get_download_status(download_id)
+                if status:
+                    return status
+            except Exception as exc:
+                logger.debug("%s get_download_status failed: %s", source_name, exc)
+        return None
+
+    async def cancel_download(self, download_id: str,
+                              source_hint: Optional[str] = None,
+                              remove: bool = False) -> bool:
+        """Cancel a download. ``source_hint`` is the source name (or
+        legacy alias like ``'deezer_dl'``, or a real Soulseek peer
+        username) — when provided, routes directly to that plugin.
+        When omitted, every plugin is asked in turn until one accepts.
+
+        Cin's review caught a bug here: legacy alias strings like
+        ``'deezer_dl'`` weren't resolved to the canonical ``'deezer'``
+        plugin name, so the cancel silently fell through to Soulseek.
+        Resolution now goes through ``_resolve_canonical`` first.
+        """
+        # Direct routing when the caller knows the source.
+        if source_hint:
+            canonical = self._resolve_canonical(source_hint)
+            # Streaming source names (or aliases) resolve to a
+            # registered plugin. Anything else (real Soulseek peer
+            # name not in our registry) routes to Soulseek.
+            if canonical and canonical != 'soulseek':
+                target_plugin = self._plugins.get(canonical)
+                if target_plugin is not None:
+                    try:
+                        return await target_plugin.cancel_download(
+                            download_id, source_hint, remove,
+                        )
+                    except Exception as exc:
+                        logger.debug("%s cancel_download failed: %s", canonical, exc)
+                        return False
+            soulseek = self._plugins.get('soulseek')
+            if soulseek is not None:
+                try:
+                    return await soulseek.cancel_download(download_id, source_hint, remove)
+                except Exception as exc:
+                    logger.debug("soulseek cancel_download failed: %s", exc)
+                    return False
+
+        # No hint → ask every plugin until one cancels successfully.
+        for source_name, plugin in self._plugins.items():
+            if plugin is None:
+                continue
+            try:
+                if await plugin.cancel_download(download_id, source_hint, remove):
+                    return True
+            except Exception as exc:
+                logger.debug("%s cancel_download failed: %s", source_name, exc)
+        return False
+
+    async def clear_all_completed_downloads(self) -> bool:
+        """Best-effort cleanup of every plugin's completed-downloads
+        list. Skips plugins that report not-configured (saves API
+        calls + log noise)."""
+        results = []
+        for source_name, plugin in self._plugins.items():
+            if plugin is None:
+                continue
+            if hasattr(plugin, 'is_configured') and not plugin.is_configured():
+                logger.debug("Skipping %s clear_all_completed_downloads (not configured)", source_name)
+                continue
+            try:
+                results.append(await plugin.clear_all_completed_downloads())
+            except Exception as exc:
+                logger.warning("%s clear_all_completed_downloads failed: %s", source_name, exc)
+                results.append(False)
+        return all(results) if results else True
+
+    # ------------------------------------------------------------------
+    # Hybrid fallback — Phase F surface
+    # ------------------------------------------------------------------
+
+    async def search_with_fallback(self, query: str, source_chain,
+                                   timeout=None, progress_callback=None):
+        """Try each source in ``source_chain`` until one returns
+        tracks. Skips unconfigured / unregistered sources, swallows
+        per-source exceptions. Returns the first non-empty
+        (tracks, albums) tuple, or ``([], [])`` when every source
+        in the chain is exhausted.
+
+        Priority mode is deliberately quality-AGNOSTIC at search time — source
+        order is king and the first source that returns any tracks wins, exactly
+        matching pre-quality-system behaviour byte-for-byte (#896 review #3).
+        Quality-gating the priority path would deprioritise e.g. a soulseek
+        mp3 whose bitrate slskd omitted (``bitrate=None`` → "unsatisfied"),
+        changing which source wins and adding latency for users who never opted
+        in. Cross-source quality pooling is the job of best_quality mode
+        (``search_all_sources``); final per-result ranking still happens in the
+        orchestrator's match/quality filter. RAW tracks are returned.
+
+        Replaces orchestrator's hand-rolled hybrid search loop. The
+        chain is ordered (most-preferred first).
+        """
+        for i, source_name in enumerate(source_chain):
+            plugin = self._plugins.get(source_name)
+            if plugin is None:
+                logger.info(f"Skipping {source_name} (not available)")
+                continue
+            if hasattr(plugin, 'is_configured') and not plugin.is_configured():
+                logger.info(f"Skipping {source_name} (not configured)")
+                continue
+
+            try:
+                logger.info(f"Trying {source_name} (priority {i+1}): {query}")
+                tracks, albums = await plugin.search(query, timeout, progress_callback)
+                if not tracks:
+                    continue
+                logger.info(f"{source_name} found {len(tracks)} tracks")
+                return (tracks, albums)
+            except Exception as e:
+                logger.warning(f"{source_name} search failed: {e}")
+
+        logger.warning(
+            "Hybrid search: all sources (%s) found nothing for: %s",
+            ', '.join(source_chain), query,
+        )
+        return ([], [])
+
+    async def search_all_sources(self, query: str, source_chain,
+                                 timeout=None, progress_callback=None,
+                                 exclude_sources=None):
+        """Best-quality mode: pool RAW tracks from EVERY configured source in
+        ``source_chain`` instead of stopping at the first satisfying one.
+
+        Unlike :meth:`search_with_fallback`, no source short-circuits the
+        search — the caller (orchestrator/worker) ranks the combined pool
+        best→worst by actual audio quality. ``exclude_sources`` drops sources
+        whose per-source retry budget is already spent (so their candidates
+        never re-enter the pool). Unconfigured / unregistered / raising sources
+        are skipped exactly like the fallback path. Returns
+        ``(combined_tracks, combined_albums)``.
+        """
+        excluded = {s.lower() for s in (exclude_sources or []) if s}
+        pooled_tracks = []
+        pooled_albums = []
+        # Per-source contribution for an honest pool log — e.g. a release-level
+        # source like usenet/torrent that returns nothing for a track-title
+        # query should read "usenet=0", not silently hide behind the chain name.
+        contributions = []
+
+        # Decide which sources to actually query, recording why the rest were
+        # skipped. Searches then run CONCURRENTLY so the pool waits only for the
+        # slowest source (e.g. usenet/Prowlarr, which can be slow) rather than
+        # the sum of every source's latency.
+        to_search = []  # (source_name, plugin)
+        for source_name in source_chain:
+            if source_name.lower() in excluded:
+                contributions.append(f"{source_name}=excluded")
+                continue
+            plugin = self._plugins.get(source_name)
+            if plugin is None:
+                logger.info(f"Skipping {source_name} (not available)")
+                contributions.append(f"{source_name}=unavailable")
+                continue
+            if hasattr(plugin, 'is_configured') and not plugin.is_configured():
+                logger.info(f"Skipping {source_name} (not configured)")
+                contributions.append(f"{source_name}=unconfigured")
+                continue
+            to_search.append((source_name, plugin))
+
+        async def _one(plugin):
+            return await plugin.search(query, timeout, progress_callback)
+
+        results = await asyncio.gather(
+            *[_one(plugin) for _, plugin in to_search],
+            return_exceptions=True,
+        )
+
+        for (source_name, _), result in zip(to_search, results, strict=True):
+            if isinstance(result, Exception):
+                logger.warning(f"{source_name} search failed: {result}")
+                contributions.append(f"{source_name}=error")
+                continue
+            tracks, albums = result
+            n = len(tracks) if tracks else 0
+            if tracks:
+                pooled_tracks.extend(tracks)
+            if albums:
+                pooled_albums.extend(albums)
+            contributions.append(f"{source_name}={n}")
+
+        logger.info(
+            "Best-quality pool: %d candidates [%s] for: %s",
+            len(pooled_tracks), ', '.join(contributions), query,
+        )
+        return (pooled_tracks, pooled_albums)
+
+    async def download_with_fallback(self, username: str, filename: str,
+                                     file_size: int, source_chain) -> Optional[str]:
+        """Try each source in ``source_chain`` until one accepts the
+        download (returns a non-None download_id). Fixes the legacy
+        bug where hybrid mode silently routed to a single source via
+        the username hint with no retry on failure.
+
+        ``username`` is treated as a hint when it matches a source
+        name in the chain — that source is tried FIRST regardless of
+        chain order. Anything else (e.g. a real Soulseek peer name)
+        routes through the chain in declared order.
+        """
+        # Promote a matching source-name hint to the head of the chain.
+        ordered_chain = list(source_chain)
+        if username and username in ordered_chain:
+            ordered_chain.remove(username)
+            ordered_chain.insert(0, username)
+
+        for source_name in ordered_chain:
+            plugin = self._plugins.get(source_name)
+            if plugin is None:
+                continue
+            if hasattr(plugin, 'is_configured') and not plugin.is_configured():
+                continue
+            try:
+                download_id = await plugin.download(username, filename, file_size)
+                if download_id is not None:
+                    return download_id
+                logger.info(f"{source_name} declined download — trying next in chain")
+            except Exception as e:
+                logger.warning(f"{source_name} download raised — trying next in chain: {e}")
+
+        logger.warning(
+            "Hybrid download: every source in chain (%s) refused %r",
+            ', '.join(ordered_chain), filename,
+        )
+        return None

core/download_engine/rate_limit.py

@@ -0,0 +1,76 @@
+"""Per-source rate-limit policy declarations.
+
+Today's per-source download throttling is scattered:
+
+- YouTube: ``self._download_delay = config_manager.get('youtube.download_delay', 3)``
+  set in ``__init__``, applied in ``set_engine`` via worker.set_delay.
+- Qobuz: module-level ``_qobuz_api_lock`` + ``_QOBUZ_MIN_INTERVAL`` for
+  search-side throttling, no download-side throttle.
+- Other sources: no explicit declarations — default to 0s delay /
+  concurrency=1, which works because the streaming APIs have their
+  own gateway-level rate limits.
+
+Phase E centralizes this into one place: each plugin declares a
+``RateLimitPolicy`` (either as a class attribute or returned from a
+``rate_limit_policy()`` method), and the engine reads + applies the
+policy to ``engine.worker`` at ``register_plugin`` time.
+
+Adding a new source = declaring its policy alongside the rest of
+the source's auth/config — no longer a hidden line in __init__ or a
+module-level constant in the client file.
+"""
+
+from __future__ import annotations
+
+import logging
+from dataclasses import dataclass
+
+logger = logging.getLogger(__name__)
+
+
+@dataclass(frozen=True)
+class RateLimitPolicy:
+    """Per-source download throttling policy.
+
+    Attributes:
+        download_concurrency: Max number of concurrent downloads
+            from this source. Default 1 (serial). Most streaming
+            APIs prefer serial transfers because parallel just
+            trades rate-limit errors for thread overhead.
+        download_delay_seconds: Minimum gap between successive
+            downloads from this source. YouTube uses 3s today
+            (legacy ``_download_delay`` config key) to avoid
+            yt-dlp 429s. Most other sources use 0.
+    """
+
+    download_concurrency: int = 1
+    download_delay_seconds: float = 0.0
+
+
+# Sentinel default — most plugins want this. Plugins that need
+# tighter throttling override by exposing ``RATE_LIMIT_POLICY`` as
+# a class attribute or returning a custom one from
+# ``rate_limit_policy()``.
+DEFAULT_POLICY = RateLimitPolicy()
+
+
+def resolve_policy(plugin) -> RateLimitPolicy:
+    """Read a plugin's declared rate-limit policy. Checks (in order):
+    1. ``plugin.rate_limit_policy()`` method (returns a RateLimitPolicy)
+    2. ``plugin.RATE_LIMIT_POLICY`` class attribute
+    3. ``DEFAULT_POLICY``
+    """
+    method = getattr(plugin, 'rate_limit_policy', None)
+    if callable(method):
+        try:
+            policy = method()
+            if isinstance(policy, RateLimitPolicy):
+                return policy
+        except Exception as e:
+            logger.debug("plugin rate_limit_policy() call failed: %s", e)
+
+    declared = getattr(plugin, 'RATE_LIMIT_POLICY', None)
+    if isinstance(declared, RateLimitPolicy):
+        return declared
+
+    return DEFAULT_POLICY

core/download_engine/worker.py

@@ -0,0 +1,315 @@
+"""BackgroundDownloadWorker — engine-owned thread spawning + state
+lifecycle for downloads.
+
+Today every streaming download client (YouTube, Tidal, Qobuz, HiFi,
+Deezer, SoundCloud) hand-rolls the same thread-spawn pattern:
+
+```python
+async def download(self, ...):
+    download_id = str(uuid.uuid4())
+    with self._download_lock:
+        self.active_downloads[download_id] = {...initial state...}
+    threading.Thread(
+        target=self._download_thread_worker,
+        args=(download_id, target_id, display_name, ...),
+        daemon=True,
+    ).start()
+    return download_id
+
+def _download_thread_worker(self, download_id, target_id, display_name, ...):
+    with self._download_semaphore:
+        # rate-limit sleep
+        # update state to 'InProgress, Downloading'
+        file_path = self._download_sync(...)  # the source-specific atomic op
+        # update state to 'Completed, Succeeded' / 'Errored'
+```
+
+That pattern is duplicated 6+ times across the codebase (~70 LOC
+each, ~490 total). The worker class lifts it into the engine — each
+plugin only has to provide the atomic op (``impl_callable``) and
+declare its rate-limit policy. Adding a new download source becomes
+a much smaller patch.
+
+Phase C1 scope: introduce the worker. No client migrated yet — the
+worker just exists for C2–C7 to migrate sources one at a time, each
+under a passing pinning test.
+"""
+
+from __future__ import annotations
+
+import threading
+import time
+import uuid
+from typing import Any, Callable, Dict, Optional
+
+from utils.logging_config import get_logger
+
+logger = get_logger("download_engine.worker")
+
+
+# Type aliases for clarity. ``ImplCallable`` is the per-plugin
+# atomic download operation — synchronous, returns a file path on
+# success or raises (or returns None) on failure.
+ImplCallable = Callable[[str, Any, str], Optional[str]]
+
+
+class BackgroundDownloadWorker:
+    """Engine-owned thread spawner for per-source downloads.
+
+    State-machine semantics (preserved verbatim from the legacy
+    per-client workers so consumers reading these fields keep
+    working):
+
+    - ``Initializing`` — set on dispatch, before the thread starts.
+    - ``InProgress, Downloading`` — set when the worker thread
+      acquires the semaphore and is about to call the impl.
+    - ``Completed, Succeeded`` — set when impl returns a non-None
+      file path. ``progress=100.0`` and ``file_path=<the path>``
+      also written.
+    - ``Errored`` — set when impl returns None OR raises. The
+      record is left in place so downstream consumers can inspect
+      what failed.
+
+    Per-source serialization: each source gets a ``threading.Semaphore``
+    (default size 1, configurable per-source via ``set_concurrency``).
+    Same shape the existing clients use today (each source defines
+    its own semaphore). Engine owning them centrally lets a future
+    Phase E rate-limiter swap the semaphore for a smarter pool.
+
+    Per-source delay-between-downloads: default 0 seconds (most
+    sources don't need it). YouTube currently uses 3s, Qobuz uses
+    1s — the legacy values get configured in via ``set_delay``
+    when the source registers.
+    """
+
+    def __init__(self, engine: Any) -> None:
+        self._engine = engine
+        # Per-source semaphores + delay state. The first dispatch
+        # for a source auto-creates a semaphore with concurrency=1
+        # if the source hasn't been configured explicitly.
+        self._semaphores: Dict[str, threading.Semaphore] = {}
+        self._delays: Dict[str, float] = {}
+        self._last_download_at: Dict[str, float] = {}
+        self._config_lock = threading.Lock()
+
+    # ------------------------------------------------------------------
+    # Per-source rate-limit configuration
+    # ------------------------------------------------------------------
+
+    def set_concurrency(self, source_name: str, max_concurrent: int) -> None:
+        """Set the max number of concurrent downloads for a source.
+        Default is 1 (serial). Most sources will keep the default —
+        the streaming APIs all rate-limit at the API gateway level
+        anyway, parallel downloads just trade rate-limit errors for
+        thread overhead."""
+        with self._config_lock:
+            self._semaphores[source_name] = threading.Semaphore(max_concurrent)
+
+    def set_delay(self, source_name: str, seconds: float) -> None:
+        """Set a minimum delay between successive downloads from the
+        same source. YouTube uses 3s today (avoid yt-dlp 429s),
+        Qobuz uses 1s. Other sources use 0 (no delay)."""
+        with self._config_lock:
+            self._delays[source_name] = float(seconds)
+
+    def _get_semaphore(self, source_name: str) -> threading.Semaphore:
+        with self._config_lock:
+            sem = self._semaphores.get(source_name)
+            if sem is None:
+                sem = threading.Semaphore(1)
+                self._semaphores[source_name] = sem
+            return sem
+
+    def _get_delay(self, source_name: str) -> float:
+        with self._config_lock:
+            return self._delays.get(source_name, 0.0)
+
+    # ------------------------------------------------------------------
+    # Dispatch — public API
+    # ------------------------------------------------------------------
+
+    def dispatch(
+        self,
+        source_name: str,
+        target_id: Any,
+        display_name: str,
+        original_filename: str,
+        impl_callable: ImplCallable,
+        extra_record_fields: Optional[Dict[str, Any]] = None,
+        username_override: Optional[str] = None,
+        thread_name: Optional[str] = None,
+    ) -> str:
+        """Kick off a background download.
+
+        Args:
+            source_name: Canonical source name (e.g. 'youtube',
+                'tidal'). Used as the engine state key + the
+                username slot in the record (unless overridden).
+            target_id: Source-specific identifier (track_id, video_id,
+                permalink_url, album_foreign_id, etc.). Passed
+                verbatim to ``impl_callable``.
+            display_name: Human-readable label for logs / UI.
+            original_filename: The encoded filename the orchestrator
+                received (e.g. ``'12345||Song Title'``). Stored in
+                the record's ``filename`` slot for context-key lookups.
+            impl_callable: Synchronous function that performs the
+                actual download. Signature:
+                ``impl_callable(download_id, target_id, display_name) -> Optional[str]``.
+                Returns the final file path on success or None /
+                raises on failure.
+            extra_record_fields: Per-source extras to merge into the
+                initial record (e.g. ``{'video_id': '...', 'url':
+                '...', 'title': '...'}`` for YouTube). Used to
+                preserve source-specific slots that downstream
+                consumers + status APIs read.
+            username_override: Use this instead of ``source_name``
+                in the record's ``username`` slot. Required for
+                Deezer (legacy ``'deezer_dl'``) — every other source
+                uses the canonical name.
+            thread_name: Optional thread name for diagnostics. Deezer
+                uses ``'deezer-dl-<track_id>'`` — Phase A pinning
+                tests catch any drift in this convention.
+
+        Returns:
+            download_id (UUID4 string). The orchestrator polls via
+            ``engine.get_download_status(download_id)`` for progress.
+        """
+        download_id = str(uuid.uuid4())
+
+        record: Dict[str, Any] = {
+            'id': download_id,
+            'filename': original_filename,
+            'username': username_override or source_name,
+            'state': 'Initializing',
+            'progress': 0.0,
+            'size': 0,
+            'transferred': 0,
+            'speed': 0,
+            'time_remaining': None,
+            'file_path': None,
+        }
+        if extra_record_fields:
+            record.update(extra_record_fields)
+
+        self._engine.add_record(source_name, download_id, record)
+
+        thread = threading.Thread(
+            target=self._worker_loop,
+            args=(source_name, download_id, target_id, display_name, impl_callable),
+            daemon=True,
+            name=thread_name,
+        )
+        thread.start()
+
+        return download_id
+
+    # ------------------------------------------------------------------
+    # Worker thread — the lifted boilerplate
+    # ------------------------------------------------------------------
+
+    def _worker_loop(
+        self,
+        source_name: str,
+        download_id: str,
+        target_id: Any,
+        display_name: str,
+        impl_callable: ImplCallable,
+    ) -> None:
+        """Runs on the spawned daemon thread. Handles semaphore
+        acquisition, rate-limit sleep, state lifecycle, exception
+        capture. The plugin-specific work happens entirely inside
+        ``impl_callable``."""
+        try:
+            with self._get_semaphore(source_name):
+                # Rate-limit delay against the LAST download from
+                # this source (not just this worker — semaphore
+                # ensures serial access while delay is configured).
+                delay = self._get_delay(source_name)
+                if delay > 0:
+                    last_at = self._last_download_at.get(source_name, 0.0)
+                    elapsed = time.time() - last_at
+                    if last_at > 0 and elapsed < delay:
+                        wait_time = delay - elapsed
+                        logger.info(
+                            "Rate-limit delay for %s: waiting %.1fs before next download",
+                            source_name, wait_time,
+                        )
+                        time.sleep(wait_time)
+
+                self._engine.update_record(source_name, download_id, {
+                    'state': 'InProgress, Downloading',
+                })
+
+                try:
+                    file_path = impl_callable(download_id, target_id, display_name)
+                except Exception as exc:
+                    logger.error(
+                        "%s download %s failed (impl raised): %s",
+                        source_name, download_id, exc,
+                    )
+                    self._mark_terminal(
+                        source_name, download_id,
+                        success=False, error=str(exc),
+                    )
+                    return
+
+                self._last_download_at[source_name] = time.time()
+
+                if file_path:
+                    # Atomic write — preserve Cancelled if user cancelled
+                    # between impl returning and this write. Same guard
+                    # _mark_terminal uses; Cin flagged both split sites.
+                    self._engine.update_record_unless_state(
+                        source_name, download_id,
+                        {
+                            'state': 'Completed, Succeeded',
+                            'progress': 100.0,
+                            'file_path': file_path,
+                        },
+                        skip_if_state_in=('Cancelled',),
+                    )
+                    logger.info(
+                        "%s download %s completed: %s",
+                        source_name, download_id, file_path,
+                    )
+                else:
+                    self._mark_terminal(source_name, download_id, success=False)
+                    logger.error(
+                        "%s download %s failed (impl returned None)",
+                        source_name, download_id,
+                    )
+
+        except Exception as exc:
+            # Defensive — semaphore / sleep shouldn't blow up the
+            # thread, but if they do the record needs SOME terminal
+            # state or it sits at 'Initializing' forever.
+            logger.exception(
+                "%s worker_loop crashed for download %s: %s",
+                source_name, download_id, exc,
+            )
+            self._mark_terminal(
+                source_name, download_id,
+                success=False, error=f'worker crash: {exc}',
+            )
+
+    def _mark_terminal(self, source_name: str, download_id: str,
+                       success: bool, error: Optional[str] = None) -> None:
+        """Write a terminal state, but DON'T clobber an explicit
+        'Cancelled' state set by the user via cancel_download.
+        Mirrors the legacy per-client guard
+        (``if state != 'Cancelled': state = 'Errored'``) every
+        client used to hand-roll inside its thread worker.
+
+        Uses ``update_record_unless_state`` so the check + write are
+        atomic under the engine's per-source lock. Cin caught a race
+        where a cancel landing between the read-snapshot + write
+        could overwrite Cancelled back to Errored / Completed.
+        """
+        patch: Dict[str, Any] = {
+            'state': 'Completed, Succeeded' if success else 'Errored',
+        }
+        if error is not None:
+            patch['error'] = error
+        self._engine.update_record_unless_state(
+            source_name, download_id, patch, skip_if_state_in=('Cancelled',),
+        )

core/download_orchestrator.py

@@ -1,15 +1,22 @@
 """
 Download Orchestrator
-Routes downloads between Soulseek, YouTube, Tidal, Qobuz, HiFi, and Deezer based on configuration.
+Routes downloads between Soulseek, YouTube, Tidal, Qobuz, HiFi, Deezer, and SoundCloud based on configuration.
 
-Supports seven modes:
+Supports eight modes:
 - Soulseek Only: Traditional behavior
 - YouTube Only: YouTube-exclusive downloads
 - Tidal Only: Tidal-exclusive downloads
 - Qobuz Only: Qobuz-exclusive downloads
 - HiFi Only: Free lossless downloads via public hifi-api instances
 - Deezer Only: Deezer downloads via ARL authentication
+- SoundCloud Only: Anonymous SoundCloud downloads (DJ mixes, removed/exclusive tracks)
 - Hybrid: Try primary source first, fallback to others
+
+The orchestrator dispatches through ``core.download_plugins.registry``
+instead of hardcoded per-source ``[self.soulseek, self.youtube, ...]``
+lists. External callers reach individual clients via the generic
+``orchestrator.client('<name>')`` accessor (alias-aware), not direct
+attribute access.
 """
 
 import asyncio
@@ -18,13 +25,10 @@ from pathlib import Path
 
 from utils.logging_config import get_logger
 from config.settings import config_manager
-from core.soulseek_client import SoulseekClient, TrackResult, AlbumResult, DownloadStatus
-from core.youtube_client import YouTubeClient
-from core.tidal_download_client import TidalDownloadClient
-from core.qobuz_client import QobuzClient
-from core.hifi_client import HiFiClient
-from core.deezer_download_client import DeezerDownloadClient
-from core.lidarr_download_client import LidarrDownloadClient
+from core.download_engine import DownloadEngine
+from core.download_plugins.registry import DownloadPluginRegistry, build_default_registry
+from core.download_plugins.types import TrackResult, AlbumResult, DownloadStatus
+from core.quality.selection import load_search_mode
 
 logger = get_logger("download_orchestrator")
 
@@ -37,18 +41,31 @@ class DownloadOrchestrator:
     Routes requests to the appropriate client(s) based on configured mode.
     """
 
-    def __init__(self):
-        """Initialize orchestrator with all clients.
-        Each client is initialized independently — one failing client doesn't prevent others from working."""
-        self._init_failures = []
+    def __init__(self, registry: Optional[DownloadPluginRegistry] = None,
+                 engine: Optional[DownloadEngine] = None):
+        """Initialize orchestrator with a plugin registry. Each plugin
+        is built and registered independently — one failing plugin
+        doesn't prevent others from working. The ``registry`` arg
+        exists so tests can inject a registry with mock plugins; in
+        production callers leave it None and get the default.
 
-        self.soulseek = self._safe_init('Soulseek', SoulseekClient)
-        self.youtube = self._safe_init('YouTube', YouTubeClient)
-        self.tidal = self._safe_init('Tidal', TidalDownloadClient)
-        self.qobuz = self._safe_init('Qobuz', QobuzClient)
-        self.hifi = self._safe_init('HiFi', HiFiClient)
-        self.deezer_dl = self._safe_init('Deezer', DeezerDownloadClient)
-        self.lidarr = self._safe_init('Lidarr', LidarrDownloadClient)
+        ``engine`` is the cross-source state owner. Phase B introduces
+        it as a held reference; it isn't on any code path yet — Phase
+        C/D/E/F migrate behavior into it incrementally.
+        """
+        self.registry = registry if registry is not None else build_default_registry()
+        self.registry.initialize()
+        self._init_failures = self.registry.init_failures
+
+        # Engine — owns cross-source state, threading, search retry,
+        # rate-limits, fallback. Built in subsequent phases. For Phase
+        # B it's just an empty registry of plugins so future phases
+        # can route through it without further orchestrator changes.
+        self.engine = engine if engine is not None else DownloadEngine()
+        for source_name, plugin in self.registry.all_plugins():
+            spec = self.registry.get_spec(source_name)
+            aliases = spec.aliases if spec else ()
+            self.engine.register_plugin(source_name, plugin, aliases=aliases)
 
         if self._init_failures:
             logger.warning(f"Download clients failed to initialize: {', '.join(self._init_failures)}")
@@ -68,15 +85,6 @@ class DownloadOrchestrator:
                 self.hybrid_secondary,
             )
 
-    def _safe_init(self, name, cls):
-        """Initialize a download client, returning None on failure instead of crashing."""
-        try:
-            return cls()
-        except Exception as e:
-            logger.error(f"{name} download client failed to initialize: {e}")
-            self._init_failures.append(name)
-            return None
-
     def reload_settings(self):
         """Reload settings from config (call after settings change)"""
         self.mode = config_manager.get('download_source.mode', 'soulseek')
@@ -85,22 +93,60 @@ class DownloadOrchestrator:
         self.hybrid_order = config_manager.get('download_source.hybrid_order', ['hifi', 'youtube', 'soulseek'])
 
         # Reload underlying client configs (SLSKD URL, API key, etc.)
-        if self.soulseek:
-            self.soulseek._setup_client()
+        soulseek = self.client('soulseek')
+        if soulseek:
+            soulseek._setup_client()
             logger.info("Soulseek client config reloaded")
 
         # Reconnect Deezer if ARL changed
         deezer_arl = config_manager.get('deezer_download.arl', '')
-        if deezer_arl and self.deezer_dl:
-            self.deezer_dl.reconnect(deezer_arl)
-            self.deezer_dl._quality = config_manager.get('deezer_download.quality', 'flac')
+        deezer_dl = self.client('deezer_dl')
+        if deezer_arl and deezer_dl:
+            deezer_dl.reconnect(deezer_arl)
+            from core.quality.source_map import quality_tier_for_source
+            deezer_dl._quality = quality_tier_for_source('deezer', default='flac')
 
-        # Reload download path for all clients that cache it
+        # Reload Amazon quality preference (T2Tunes needs no reconnect — public proxy)
+        amazon = self.client('amazon')
+        if amazon:
+            from core.quality.source_map import quality_tier_for_source
+            quality = quality_tier_for_source('amazon', default='flac')
+            amazon._quality = quality
+            amazon._allow_fallback = config_manager.get('amazon_download.allow_fallback', True)
+            if hasattr(amazon, '_client') and amazon._client:
+                amazon._client.preferred_codec = quality
+
+        # Let registry-backed plugins refresh any config they cache at
+        # construction time. This covers Prowlarr-backed torrent / usenet
+        # clients without rebuilding the registry and losing active downloads.
+        for name, client in self.registry.all_plugins():
+            if not hasattr(client, 'reload_settings'):
+                continue
+            try:
+                client.reload_settings()
+                logger.info("%s client settings reloaded", self.registry.display_name(name))
+            except Exception as exc:
+                logger.warning(
+                    "%s client settings reload failed: %s",
+                    self.registry.display_name(name),
+                    exc,
+                )
+
+        # Reload download path for all clients that cache it.
+        # Soulseek owns the path config and is reloaded above; every
+        # other source mirrors that path so files all land in one
+        # tree. Sources without a `download_path` attribute (e.g.
+        # Lidarr — pulls into Lidarr's own tree) silently skip.
         new_path = Path(config_manager.get('soulseek.download_path', './downloads'))
-        for client in [self.youtube, self.tidal, self.qobuz, self.hifi, self.deezer_dl]:
-            if client and hasattr(client, 'download_path') and client.download_path != new_path:
+        for name, client in self.registry.all_plugins():
+            if name == 'soulseek':
+                continue
+            if hasattr(client, 'download_path') and client.download_path != new_path:
                 client.download_path = new_path
-                client.download_path.mkdir(parents=True, exist_ok=True)
+                try:
+                    client.download_path.mkdir(parents=True, exist_ok=True)
+                except OSError as e:
+                    logger.warning(f"Could not verify download path {new_path}: {e}")
                 # YouTube also caches path in yt-dlp opts
                 if hasattr(client, 'download_opts') and 'outtmpl' in client.download_opts:
                     client.download_opts['outtmpl'] = str(new_path / '%(title)s.%(ext)s')
@@ -108,11 +154,61 @@ class DownloadOrchestrator:
 
         logger.info(f"Download Orchestrator settings reloaded - Mode: {self.mode}")
 
-    def _client(self, name):
-        """Get a client by name, returning None if not initialized."""
-        return {'soulseek': self.soulseek, 'youtube': self.youtube, 'tidal': self.tidal,
-                'qobuz': self.qobuz, 'hifi': self.hifi, 'deezer_dl': self.deezer_dl,
-                'lidarr': self.lidarr}.get(name)
+    def client(self, name):
+        """Generic accessor for a download source client by name.
+
+        Cin's review feedback: external callers should reach into
+        per-source clients via this method (``orch.client('hifi')``)
+        instead of attribute access (``orch.hifi``). Resolves both
+        canonical names (``deezer``) and legacy aliases (``deezer_dl``)
+        via the registry. Returns None if the source isn't registered
+        or failed to initialize.
+        """
+        return self.registry.get(name)
+
+    # Internal alias kept for legacy callers inside this file.
+    _client = client
+
+    def configured_clients(self) -> dict:
+        """Return ``{source_name: client}`` for every download source
+        that's both initialized AND reports is_configured() == True.
+
+        Replaces the legacy per-source iteration pattern Cin called
+        out — `if hasattr(orch, 'soulseek') and orch.soulseek and
+        orch.soulseek.is_configured(): download_clients['soulseek']
+        = orch.soulseek` repeated for each source.
+        """
+        result = {}
+        for name, client in self.registry.all_plugins():
+            try:
+                if not hasattr(client, 'is_configured') or client.is_configured():
+                    result[name] = client
+            except Exception as exc:
+                logger.debug("%s is_configured raised: %s", name, exc)
+        return result
+
+    def reload_instances(self, source: str = None) -> bool:
+        """Reload a source's instance config (e.g. HiFi instance list,
+        Qobuz session restore). Generic dispatch — caller passes the
+        source name instead of reaching for ``orch.hifi.reload_instances()``.
+
+        When ``source`` is None, reloads every source that has a
+        ``reload_instances`` method.
+        """
+        sources = [source] if source else list(self.registry.names())
+        ok = True
+        for name in sources:
+            client = self.client(name)
+            if client is None:
+                continue
+            if not hasattr(client, 'reload_instances'):
+                continue
+            try:
+                client.reload_instances()
+            except Exception as exc:
+                logger.warning("%s reload_instances failed: %s", name, exc)
+                ok = False
+        return ok
 
     def is_configured(self) -> bool:
         """
@@ -129,12 +225,18 @@ class DownloadOrchestrator:
         return False
 
     def get_source_status(self) -> dict:
-        """Return configured status for each download source."""
-        return {name: (c.is_configured() if c else False)
-                for name, c in [('soulseek', self.soulseek), ('youtube', self.youtube),
-                                ('tidal', self.tidal), ('qobuz', self.qobuz),
-                                ('hifi', self.hifi), ('deezer_dl', self.deezer_dl),
-                                ('lidarr', self.lidarr)]}
+        """Return configured status for each download source.
+
+        Keys preserve the legacy ``deezer_dl`` alias used by the
+        frontend status indicators and per-source dispatch strings,
+        so callers reading specific keys keep working unchanged.
+        """
+        status = {}
+        for name in self.registry.names():
+            client = self.registry.get(name)
+            key = 'deezer_dl' if name == 'deezer' else name
+            status[key] = client.is_configured() if client else False
+        return status
 
     async def check_connection(self) -> bool:
         """
@@ -146,7 +248,7 @@ class DownloadOrchestrator:
         if client and self.mode != 'hybrid':
             return await client.check_connection()
         elif self.mode == 'hybrid':
-            sources_to_check = self.hybrid_order if self.hybrid_order else ['soulseek', 'youtube', 'tidal', 'qobuz', 'hifi', 'deezer_dl', 'lidarr']
+            sources_to_check = self.hybrid_order if self.hybrid_order else self.registry.names()
             results = {}
             for source in sources_to_check:
                 client = self._client(source)
@@ -165,76 +267,94 @@ class DownloadOrchestrator:
 
         return False
 
-    async def search(self, query: str, timeout: int = None, progress_callback=None) -> Tuple[List[TrackResult], List[AlbumResult]]:
+    def _normalize_source_name(self, name: str) -> Optional[str]:
+        """Convert a possibly-aliased source name (e.g. legacy
+        ``'deezer_dl'``) to the canonical registry name (``'deezer'``).
+        Returns None if the input matches neither a canonical name
+        nor an alias.
+
+        Cin's review caught a bug where legacy alias values from
+        config (hybrid_order containing ``'deezer_dl'``) silently
+        dropped Deezer from hybrid mode because the canonical-name
+        membership check rejected the alias.
         """
-        Search for tracks using configured source(s).
+        spec = self.registry.get_spec(name) if name else None
+        return spec.name if spec else None
 
-        Args:
-            query: Search query
-            timeout: Search timeout (for Soulseek)
-            progress_callback: Progress callback (for Soulseek)
+    def _resolve_source_chain(self) -> List[str]:
+        """Order the configured sources for hybrid mode. Prefers
+        ``hybrid_order`` config; falls back to legacy
+        primary/secondary pair when no order set. Normalizes alias
+        names through the registry so legacy ``deezer_dl`` config
+        values resolve correctly to the canonical ``deezer`` plugin."""
+        if self.hybrid_order:
+            chain = []
+            seen = set()
+            for raw in self.hybrid_order:
+                canonical = self._normalize_source_name(raw)
+                if canonical and canonical not in seen:
+                    chain.append(canonical)
+                    seen.add(canonical)
+            return chain
+        primary = self._normalize_source_name(self.hybrid_primary) or 'soulseek'
+        secondary = self._normalize_source_name(self.hybrid_secondary) or 'soulseek'
+        if secondary == primary:
+            secondary = next(
+                (name for name in self.registry.names() if name != primary),
+                'soulseek',
+            )
+        chain = [primary, secondary]
+        if not chain:
+            chain = ['soulseek']
+        return chain
 
-        Returns:
-            Tuple of (track_results, album_results)
-        """
-        source_names = {'soulseek': 'Soulseek', 'youtube': 'YouTube', 'tidal': 'Tidal',
-                        'qobuz': 'Qobuz', 'hifi': 'HiFi', 'deezer_dl': 'Deezer', 'lidarr': 'Lidarr'}
+    async def search(self, query: str, timeout: int = None, progress_callback=None,
+                     exclude_sources=None) -> Tuple[List[TrackResult], List[AlbumResult]]:
+        """Search for tracks using configured source(s). Single-source
+        modes route directly; hybrid mode delegates to
+        ``engine.search_with_fallback`` which tries the chain in order.
 
+        ``exclude_sources`` (optional) is an iterable of source names
+        the caller wants filtered out of the hybrid chain. Used by the
+        per-track download worker to skip torrent / usenet for album-
+        context batches — those sources are release-level and don't
+        score meaningfully on per-track titles; the album-bundle flow
+        on the master worker handles them separately when they're the
+        single active source."""
         if self.mode != 'hybrid':
+            # Single-source mode is opt-in; honour the user's choice even
+            # if it's torrent/usenet on an album batch (the master worker
+            # routes those through the album-bundle flow before per-track
+            # tasks ever fire, so search() being called here would be a
+            # non-album / wishlist / basic-search use case).
             client = self._client(self.mode)
             if not client:
-                logger.error(f"{source_names.get(self.mode, self.mode)} client not available (failed to initialize)")
+                logger.error(f"{self.registry.display_name(self.mode)} client not available (failed to initialize)")
                 return [], []
-            logger.info(f"Searching {source_names.get(self.mode, self.mode)}: {query}")
+            logger.info(f"Searching {self.registry.display_name(self.mode)}: {query}")
             return await client.search(query, timeout, progress_callback)
 
-        elif self.mode == 'hybrid':
-            clients = {name: self._client(name) for name in source_names}
-
-            # Build ordered source list: prefer hybrid_order, fall back to legacy primary/secondary
-            if self.hybrid_order:
-                source_order = [s for s in self.hybrid_order if s in clients]
-            else:
-                primary = self.hybrid_primary if self.hybrid_primary in clients else 'soulseek'
-                secondary = self.hybrid_secondary if self.hybrid_secondary in clients else 'soulseek'
-                if secondary == primary:
-                    secondary = next((name for name in clients if name != primary), 'soulseek')
-                source_order = [primary, secondary]
-
-            if not source_order:
-                source_order = ['soulseek']
-
-            logger.info(f"Hybrid search ({' → '.join(source_order)}): {query}")
-
-            # Try each source in priority order (skip unconfigured/unavailable ones)
-            for i, source_name in enumerate(source_order):
-                client = clients.get(source_name)
-                if not client:
-                    logger.info(f"Skipping {source_name} (not available)")
-                    continue
-                if hasattr(client, 'is_configured') and not client.is_configured():
-                    logger.info(f"Skipping {source_name} (not configured)")
-                    continue
-
-                try:
-                    if i == 0:
-                        logger.info(f"Trying {source_name} (priority {i+1}): {query}")
-                    else:
-                        logger.info(f"Trying {source_name} (priority {i+1}): {query}")
-
-                    tracks, albums = await client.search(query, timeout, progress_callback)
-                    if tracks:
-                        logger.info(f"{source_name} found {len(tracks)} tracks")
-                        return (tracks, albums)
-                except Exception as e:
-                    logger.warning(f"{source_name} search failed: {e}")
-
-            # Nothing found from any source
-            logger.warning(f"Hybrid search: all sources ({', '.join(source_order)}) found nothing for: {query}")
-            return ([], [])
-
-        # Fallback: empty results
-        return ([], [])
+        chain = self._resolve_source_chain()
+        if exclude_sources:
+            blocked = {s.lower() for s in exclude_sources if s}
+            filtered = [s for s in chain if s.lower() not in blocked]
+            if filtered != chain:
+                logger.info(
+                    "Hybrid search: excluding %s for this query (chain %s -> %s)",
+                    sorted(blocked & {s.lower() for s in chain}),
+                    " → ".join(chain), " → ".join(filtered) if filtered else "(empty)",
+                )
+            chain = filtered
+        if not chain:
+            logger.warning("Hybrid search exhausted: no eligible sources after exclusion filter")
+            return [], []
+        if load_search_mode() == 'best_quality':
+            logger.info(f"Best-quality search ({' → '.join(chain)}): {query}")
+            return await self.engine.search_all_sources(
+                query, chain, timeout, progress_callback,
+            )
+        logger.info(f"Hybrid search ({' → '.join(chain)}): {query}")
+        return await self.engine.search_with_fallback(query, chain, timeout, progress_callback)
 
     async def search_and_download_best(self, query: str, expected_track=None) -> Optional[str]:
         """
@@ -262,7 +382,7 @@ class DownloadOrchestrator:
             return None
 
         # 2. Filter and validate results
-        _streaming_sources = ('youtube', 'tidal', 'qobuz', 'hifi', 'deezer_dl', 'lidarr')
+        _streaming_sources = ('youtube', 'tidal', 'qobuz', 'hifi', 'deezer_dl', 'lidarr', 'soundcloud', 'amazon')
         is_streaming = tracks[0].username in _streaming_sources if tracks else False
 
         if is_streaming and expected_track:
@@ -303,16 +423,25 @@ class DownloadOrchestrator:
 
             if scored:
                 scored.sort(key=lambda x: x._match_confidence, reverse=True)
-                filtered_results = scored
+                # Match filter done (right track); now prefer the best quality
+                # among the confidence-passing survivors so streaming isn't
+                # quality-blind like Soulseek already isn't. Stable ranking
+                # keeps confidence order within an equal quality tier; the
+                # `or scored` fail-safe never leaves us with nothing to try.
+                from core.quality.selection import rank_for_profile
+                ranked, _ = rank_for_profile(scored)
+                filtered_results = ranked or scored
                 logger.info(f"Streaming validation: {len(scored)}/{len(tracks)} passed "
-                            f"(best: {scored[0]._match_confidence:.2f})")
+                            f"(best: {scored[0]._match_confidence:.2f}, "
+                            f"quality pick: {filtered_results[0].audio_quality.label()})")
             else:
                 logger.warning(f"No streaming results passed validation for: {query}")
                 return None
         elif is_streaming:
             filtered_results = tracks
         else:
-            filtered_results = self.soulseek.filter_results_by_quality_preference(tracks) if self.soulseek else tracks
+            soulseek = self.client('soulseek')
+            filtered_results = soulseek.filter_results_by_quality_preference(tracks) if soulseek else tracks
 
         if not filtered_results:
             logger.warning(f"No suitable quality results found for: {query}")
@@ -335,102 +464,47 @@ class DownloadOrchestrator:
         Download a track using the appropriate client.
 
         Args:
-            username: Username (or "youtube" for YouTube)
-            filename: Filename or YouTube video ID
+            username: Source-name string for streaming sources
+                (e.g. ``'youtube'``, ``'tidal'``, ``'deezer_dl'``)
+                OR the actual slskd peer username for Soulseek.
+            filename: Filename / video ID / track ID encoding (source-specific)
             file_size: File size estimate
 
         Returns:
             download_id: Unique download ID for tracking
         """
-        # Detect which client to use based on username
-        source_map = {'youtube': self.youtube, 'tidal': self.tidal, 'qobuz': self.qobuz,
-                      'hifi': self.hifi, 'deezer_dl': self.deezer_dl, 'lidarr': self.lidarr}
-        source_names = {'youtube': 'YouTube', 'tidal': 'Tidal', 'qobuz': 'Qobuz',
-                        'hifi': 'HiFi', 'deezer_dl': 'Deezer', 'lidarr': 'Lidarr'}
-
-        if username in source_map:
-            client = source_map[username]
+        # Streaming sources are dispatched by name match; anything
+        # unrecognized falls through to Soulseek (peer username case).
+        spec = self.registry.get_spec(username) if username else None
+        if spec is not None and spec.name != 'soulseek':
+            client = self.registry.get(spec.name)
             if not client:
-                raise RuntimeError(f"{source_names[username]} download client not available (failed to initialize)")
-            logger.info(f"Downloading from {source_names[username]}: {filename}")
+                raise RuntimeError(f"{spec.display_name} download client not available (failed to initialize)")
+            logger.info(f"Downloading from {spec.display_name}: {filename}")
             return await client.download(username, filename, file_size)
-        else:
-            if not self.soulseek:
-                raise RuntimeError("Soulseek client not available (failed to initialize)")
-            logger.info(f"Downloading from Soulseek: {filename}")
-            return await self.soulseek.download(username, filename, file_size)
+
+        soulseek = self.registry.get('soulseek')
+        if not soulseek:
+            raise RuntimeError("Soulseek client not available (failed to initialize)")
+        logger.info(f"Downloading from Soulseek: {filename}")
+        return await soulseek.download(username, filename, file_size)
 
     async def get_all_downloads(self) -> List[DownloadStatus]:
-        """
-        Get all active downloads from all sources.
-
-        Returns:
-            List of DownloadStatus objects
-        """
-        # Get downloads from all available sources
-        all_downloads = []
-        for client in [self.soulseek, self.youtube, self.tidal, self.qobuz, self.hifi, self.deezer_dl, self.lidarr]:
-            if client:
-                try:
-                    all_downloads.extend(await client.get_all_downloads())
-                except Exception:
-                    pass
-        return all_downloads
+        """Aggregated view across every source. Delegates to the
+        engine, which iterates registered plugins."""
+        return await self.engine.get_all_downloads()
 
     async def get_download_status(self, download_id: str) -> Optional[DownloadStatus]:
-        """
-        Get status of a specific download.
-
-        Args:
-            download_id: Download ID to query
-
-        Returns:
-            DownloadStatus object or None if not found
-        """
-        # Try each source until we find the download
-        for client in [self.soulseek, self.youtube, self.tidal, self.qobuz, self.hifi, self.deezer_dl, self.lidarr]:
-            if not client:
-                continue
-            try:
-                status = await client.get_download_status(download_id)
-                if status:
-                    return status
-            except Exception:
-                pass
-
-        return None
+        """Find a download by id across every source. Delegates to
+        the engine."""
+        return await self.engine.get_download_status(download_id)
 
     async def cancel_download(self, download_id: str, username: str = None, remove: bool = False) -> bool:
-        """
-        Cancel an active download.
-
-        Args:
-            download_id: Download ID to cancel
-            username: Username hint (optional)
-            remove: Whether to remove from active downloads
-
-        Returns:
-            True if cancelled successfully
-        """
-        # If username is provided, route directly to that source
-        source_map = {'youtube': self.youtube, 'tidal': self.tidal, 'qobuz': self.qobuz,
-                      'hifi': self.hifi, 'deezer_dl': self.deezer_dl, 'lidarr': self.lidarr}
-        if username in source_map:
-            client = source_map[username]
-            return await client.cancel_download(download_id, username, remove) if client else False
-        elif username:
-            return await self.soulseek.cancel_download(download_id, username, remove) if self.soulseek else False
-
-        # Otherwise, try all available sources
-        for client in [self.soulseek, self.youtube, self.tidal, self.qobuz, self.hifi, self.deezer_dl, self.lidarr]:
-            if not client:
-                continue
-            try:
-                if await client.cancel_download(download_id, username, remove):
-                    return True
-            except Exception:
-                pass
-        return False
+        """Cancel an active download. Delegates to the engine, which
+        handles source-hint routing (streaming source name → direct
+        plugin, unknown name → Soulseek as peer username, no hint →
+        try every plugin)."""
+        return await self.engine.cancel_download(download_id, username, remove)
 
     async def signal_download_completion(self, download_id: str, username: str, remove: bool = True) -> bool:
         """
@@ -445,39 +519,16 @@ class DownloadOrchestrator:
             True if successful
         """
         # This is Soulseek-specific, so only call on Soulseek client
-        if not self.soulseek:
+        soulseek = self.client('soulseek')
+        if not soulseek:
             return False
-        return await self.soulseek.signal_download_completion(download_id, username, remove)
+        return await soulseek.signal_download_completion(download_id, username, remove)
 
     async def clear_all_completed_downloads(self) -> bool:
-        """
-        Clear all completed downloads from both sources.
-
-        Returns:
-            True if successful
-        """
-        results = []
-        for name, client in [
-            ("soulseek", self.soulseek),
-            ("youtube", self.youtube),
-            ("tidal", self.tidal),
-            ("qobuz", self.qobuz),
-            ("hifi", self.hifi),
-            ("deezer_dl", self.deezer_dl),
-            ("lidarr", self.lidarr),
-        ]:
-            if not client:
-                continue
-            if hasattr(client, "is_configured") and not client.is_configured():
-                logger.debug("Skipping %s clear_all_completed_downloads (not configured)", name)
-                continue
-            try:
-                results.append(await client.clear_all_completed_downloads())
-            except Exception as exc:
-                logger.warning("%s clear_all_completed_downloads failed: %s", name, exc)
-                results.append(False)
-
-        return all(results) if results else True
+        """Clear completed downloads from every source. Delegates
+        to the engine, which skips unconfigured plugins and treats
+        per-plugin failures as False (not an exception)."""
+        return await self.engine.clear_all_completed_downloads()
 
     # ===== Soulseek-specific methods (for backwards compatibility) =====
     # These are internal methods that some parts of the codebase use directly
@@ -495,9 +546,10 @@ class DownloadOrchestrator:
         Returns:
             API response
         """
-        if not self.soulseek:
+        soulseek = self.client('soulseek')
+        if not soulseek:
             raise RuntimeError("Soulseek client not available (failed to initialize)")
-        return await self.soulseek._make_request(method, endpoint, **kwargs)
+        return await soulseek._make_request(method, endpoint, **kwargs)
 
     async def _make_direct_request(self, method: str, endpoint: str, **kwargs):
         """
@@ -512,9 +564,10 @@ class DownloadOrchestrator:
         Returns:
             API response
         """
-        if not self.soulseek:
+        soulseek = self.client('soulseek')
+        if not soulseek:
             raise RuntimeError("Soulseek client not available (failed to initialize)")
-        return await self.soulseek._make_direct_request(method, endpoint, **kwargs)
+        return await soulseek._make_direct_request(method, endpoint, **kwargs)
 
     async def clear_all_searches(self) -> bool:
         """
@@ -523,7 +576,8 @@ class DownloadOrchestrator:
         Returns:
             True if successful
         """
-        return await self.soulseek.clear_all_searches() if self.soulseek else True
+        soulseek = self.client('soulseek')
+        return await soulseek.clear_all_searches() if soulseek else True
 
     async def maintain_search_history_with_buffer(self, keep_searches: int = 50, trigger_threshold: int = 200) -> bool:
         """
@@ -536,15 +590,54 @@ class DownloadOrchestrator:
         Returns:
             True if successful
         """
-        return await self.soulseek.maintain_search_history_with_buffer(keep_searches, trigger_threshold) if self.soulseek else True
+        soulseek = self.client('soulseek')
+        return await soulseek.maintain_search_history_with_buffer(keep_searches, trigger_threshold) if soulseek else True
 
     async def cancel_all_downloads(self) -> bool:
-        """Cancel and remove all downloads from all sources."""
+        """Cancel and remove all downloads from all sources.
+
+        Note: YouTube is intentionally excluded from this loop in the
+        legacy implementation — preserved here. (yt-dlp downloads
+        run as detached subprocesses and don't share the
+        ``cancel_all_downloads`` semantics the streaming sources
+        use.) Sources without ``cancel_all_downloads`` fall back to
+        ``clear_all_completed_downloads``.
+        """
         ok = True
-        for client in [self.soulseek, self.tidal, self.qobuz, self.hifi, self.deezer_dl, self.lidarr]:
-            if client:
-                try:
-                    await client.cancel_all_downloads() if hasattr(client, 'cancel_all_downloads') else await client.clear_all_completed_downloads()
-                except Exception:
-                    ok = False
+        for name, client in self.registry.all_plugins():
+            if name == 'youtube':
+                continue
+            try:
+                await client.cancel_all_downloads() if hasattr(client, 'cancel_all_downloads') else await client.clear_all_completed_downloads()
+            except Exception:
+                ok = False
         return ok
+
+
+# ---------------------------------------------------------------------------
+# Singleton accessor — mirrors Cin's metadata engine pattern
+# (``get_metadata_engine()``). Callers that don't need a custom
+# registry use this instead of instantiating DownloadOrchestrator
+# directly. web_server.py constructs the singleton at startup and
+# exposes it via the ``download_orchestrator`` global.
+# ---------------------------------------------------------------------------
+
+_default_orchestrator: Optional['DownloadOrchestrator'] = None
+
+
+def get_download_orchestrator() -> 'DownloadOrchestrator':
+    """Return (lazily creating) the process-wide DownloadOrchestrator
+    singleton. Mirrors the ``get_metadata_engine()`` pattern Cin used
+    for the metadata engine refactor."""
+    global _default_orchestrator
+    if _default_orchestrator is None:
+        _default_orchestrator = DownloadOrchestrator()
+    return _default_orchestrator
+
+
+def set_download_orchestrator(orchestrator: 'DownloadOrchestrator') -> None:
+    """Set the process-wide singleton. Used by web_server.py at boot
+    to install the orchestrator it constructs as the default for
+    callers that grab via ``get_download_orchestrator()``."""
+    global _default_orchestrator
+    _default_orchestrator = orchestrator

core/download_plugins/__init__.py

@@ -0,0 +1,34 @@
+"""Download source plugin contract + registry.
+
+This package defines the canonical interface every download source
+(Soulseek, YouTube, Tidal, Qobuz, HiFi, Deezer, Lidarr, SoundCloud,
+and future additions like Usenet) must satisfy. The orchestrator
+dispatches through this contract instead of hardcoded
+`if self.youtube ... elif self.tidal ...` chains.
+
+This is the foundation step of a multi-commit refactor. Subsequent
+commits extract shared logic (background download worker, search
+query normalization, post-processing context building) into the
+contract so adding a new source becomes a one-class plugin instead
+of a 700+ LOC copy-paste loop.
+
+See `core/download_plugins/base.py` for the protocol contract and
+`core/download_plugins/registry.py` for the dispatch entry point.
+"""
+
+from core.download_plugins.base import DownloadSourcePlugin
+
+# NOTE: DownloadPluginRegistry is intentionally NOT re-exported here.
+# Importing the registry triggers eager imports of every client class
+# (see registry.py for why eager — test fixtures inject mock modules
+# at collection time and we need real bindings before that). Clients
+# inherit from DownloadSourcePlugin (Cin's review feedback — visible
+# contract conformance), so importing the package via ``from
+# core.download_plugins import DownloadSourcePlugin`` from a client
+# file would create a circular import if registry came along for the
+# ride. Callers that need the registry import it directly:
+#     from core.download_plugins.registry import DownloadPluginRegistry
+
+__all__ = [
+    "DownloadSourcePlugin",
+]