From 31d85e59c926e7d1dc3917bf957277013747dc68 Mon Sep 17 00:00:00 2001 From: dev Date: Sun, 14 Jun 2026 17:54:42 +0200 Subject: [PATCH] docs(spec): best-quality search mode design MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Toggle to search all sources and download best→worst by actual audio quality, vs today's priority-first (first satisfying source wins). Reuses exhausted_download_sources for per-source budget removal; query generator and budget counters untouched. Independent of retry_exhaustive. Co-Authored-By: Claude Opus 4.8 --- ...6-06-14-best-quality-search-mode-design.md | 201 ++++++++++++++++++ 1 file changed, 201 insertions(+) create mode 100644 docs/superpowers/specs/2026-06-14-best-quality-search-mode-design.md diff --git a/docs/superpowers/specs/2026-06-14-best-quality-search-mode-design.md b/docs/superpowers/specs/2026-06-14-best-quality-search-mode-design.md new file mode 100644 index 00000000..3cf65627 --- /dev/null +++ b/docs/superpowers/specs/2026-06-14-best-quality-search-mode-design.md @@ -0,0 +1,201 @@ +# Best-Quality Search Mode — Design + +**Date:** 2026-06-14 +**Branch:** (new, off current `fix/import-folder-artist-override-optin`) +**Status:** Approved approach (Option A), pending spec review + +## Goal + +Add a user-toggleable download strategy. Today hybrid search is **priority-first**: +`engine.search_with_fallback` walks the source chain in priority order and accepts +the **first** source that meets a quality target — so a passable Soulseek 16-bit FLAC +"wins" even when HiFi/Qobuz/Tidal could deliver a 24-bit version of the same track. + +The new **best-quality** mode instead searches **all** configured sources, pools their +candidates, and works them **best→worst by actual audio quality**. Source priority +becomes only a tiebreaker between equally-good candidates. + +## Hard constraints (from the user) + +1. **Two independent toggles.** The new `search_mode` and the existing + `post_processing.retry_exhaustive` are orthogonal. The feature must behave + correctly in **all four** combinations (priority/best × default/exhaustive budget). +2. **Budget semantics preserved 1:1.** No change to how retries are counted: + - Default mode: single global cap `MAX_QUARANTINE_RETRIES = 5` across all sources. + - Exhaustive mode: per-source budget `query_count × retries_per_query`. + When a source **completely spends** its budget, **all of that source's candidates + are removed from the entire pooled list** (not just skipped once). +3. **Do not touch the query generator.** `matching_engine.generate_download_queries` + and the legacy query-building block in `download_track_worker` stay exactly as-is. +4. **`force_import` never fires on quality** (unchanged existing invariant — only + AcoustID mismatches can force-import). + +## Key realization: the budget-removal mechanism already exists + +When a source spends its budget, `monitor.requeue_quarantined_task_for_retry` +already adds it to the task's **`exhausted_download_sources`** set, and the worker +already passes that set as `exclude_sources` to its searches. Constraint #2's +"remove the source from the whole list" is therefore **not new logic** — best-quality +mode simply consults the *same* set when assembling/filtering its pooled candidate +list. No new budget bookkeeping is introduced. + +## Architecture + +### The one genuinely new piece: quality-dominant candidate ordering + +`attempt_download_with_candidates` (core/downloads/candidates.py:71) sorts candidates +**confidence-first, then `quality_score`**. That is correct for priority mode (never +download a high-quality *wrong* file). For best-quality mode we keep the +"correct-first" guarantee — `get_valid_candidates` already drops candidates below the +match threshold, so **every candidate in the list is correctly matched** — but among +those valid candidates we want **profile quality rank to dominate**, with confidence as +the tiebreaker. + +Add an opt-in parameter: + +```python +attempt_download_with_candidates(task_id, candidates, track, batch_id=None, + deps=None, *, quality_first=False) +``` + +- `quality_first=False` (default): today's sort, byte-for-byte unchanged. +- `quality_first=True`: sort key becomes + `(profile_quality_rank, confidence, upload_speed, -queue_length, free_slots, size)`, + i.e. profile quality dominates, all existing signals become tiebreakers. + +`profile_quality_rank` is derived from the candidate's stamped `AudioQuality` against +the user's `ranked_targets` (reusing `core.quality.model` / `filter_and_rank` ordering). +Candidates with no usable quality info sort last (rank = worst), never dropped. + +### New engine method: `search_all_sources` + +Mirror of `search_with_fallback`, but it does **not** stop at the first satisfying +source. It iterates every configured, non-excluded source in the chain, runs each +source's `search`, and returns the **combined** raw track list (each track already +quality-stamped by its client's `set_quality`, as today). Per-source exceptions are +swallowed exactly like the existing method. Excluded (`exhausted`) sources are skipped. + +```python +async def search_all_sources(self, query, source_chain, timeout=None, + progress_callback=None, exclude_sources=None) + -> Tuple[list, list] # (combined_tracks, combined_albums) +``` + +Quality ranking is **not** done here — the orchestrator/worker owns final ranking, same +division of labour as today (engine returns raw, orchestrator filters/ranks). + +### Orchestrator wiring + +`download_orchestrator.search(...)` gains awareness of `search_mode`: +- `priority` (default) → unchanged: `search_with_fallback`. +- `best_quality` **and** `mode == 'hybrid'` → `search_all_sources`. +- Single-source mode → unchanged regardless of `search_mode` (only one source exists; + its candidates are still quality-ranked downstream, so the toggle is a no-op there). + +A thin helper `load_search_mode() -> str` reads the quality profile (see Config). + +### Worker integration (the per-query loop stays intact) + +In `download_track_worker`'s existing query loop (core/downloads/task_worker.py:398), +when best-quality mode is active: + +1. The loop and the query generator are untouched. +2. `orchestrator.search(query, exclude_sources=_exclude_sources)` now returns + **pooled** results from all non-exhausted sources (via `search_all_sources`). +3. `candidates = get_valid_candidates(pooled, track, query)` — unchanged. +4. `ranked, _ = rank_for_profile(candidates)` — orders best→worst by profile quality. +5. `attempt_download_with_candidates(task_id, ranked, track, batch_id, quality_first=True)`. + +Because the pool already came from every source, the **hybrid-fallback block** +(task_worker.py:529–593, which re-tries `hybrid_order[1:]` individually) is +**redundant in best-quality mode** and is skipped — it would just re-search sources the +pool already covered. In priority mode it runs exactly as today. + +The `exhausted_download_sources` set is already folded into `_exclude_sources` +(task_worker.py:446), so a budget-spent source is excluded from `search_all_sources`'s +pool automatically — satisfying constraint #2 with no new code. A belt-and-suspenders +filter drops any straggler exhausted-source candidates before step 5. + +## Config / persistence + +`search_mode` belongs to the quality profile (it *is* a quality-driven policy). Stored +in the v3 quality-profile JSON (`database.music_database.get_quality_profile`): + +```json +{ "version": 3, "search_mode": "priority", ... } +``` + +- Default: `"priority"` (preserves today's behavior for every existing install). +- Accessor: extend `core/quality/selection.py` with `load_search_mode() -> str` + returning `'priority'` unless the profile says `'best_quality'`. +- No migration needed: a profile lacking the key reads as `'priority'`. + +## UI + +A single toggle in the existing **Quality Profile** tile (webui/index.html + +webui/static/settings.js), e.g. a labelled switch: + +> **Search strategy:** ( ) Source priority — fastest, stops at first good source +> (•) Best quality — searches all sources, picks the highest quality + +With a short note that best-quality searches **all** sources every track (slower / more +API calls). `collectQualityProfileFromUI` emits `search_mode`; `populateQualityProfileUI` +reads it. No change to the ranked-targets editor or the fallback toggle. + +## Behaviour matrix (all four combinations must hold) + +| search_mode | retry_exhaustive | Behaviour | +|---|---|---| +| priority | off | **Unchanged today.** First satisfying source wins; 5 global retries. | +| priority | on | **Unchanged today.** First source wins; per-source budgets. | +| best_quality | off | Pool all sources, best→worst; 5 **global** retries total. A source can't be "removed from the pool" because there's no per-source budget — the global cap fires first. (Documented: combine best-quality with exhaustive for per-source removal.) | +| best_quality | on | Pool all sources, best→worst; each source has `query_count × retries_per_query`; on spend → source removed from the whole pool via `exhausted_download_sources`. **This is the user's target configuration.** | + +## Error handling + +- `search_all_sources`: fail-open per source (swallow exceptions, keep going), identical + to `search_with_fallback`. If the whole pool is empty, the worker falls through to its + existing not-found handling. +- Quality-rank computation: any candidate that can't be ranked sorts last, never crashes + the walk (mirrors the engine's existing fail-open ranking). + +## Testing (TDD) + +New tests under `tests/quality/` and `tests/downloads/`: + +1. **`load_search_mode`** — defaults to `'priority'`; returns `'best_quality'` when set; + unknown value falls back to `'priority'`. +2. **`search_all_sources`** — returns combined candidates from multiple fake sources; + skips excluded/exhausted sources; swallows a raising source; empty when all empty. +3. **quality-first ordering** — `attempt_download_with_candidates(quality_first=True)` + tries a 24-bit candidate before a higher-confidence 16-bit one; `quality_first=False` + reproduces today's confidence-first order exactly (regression lock). +4. **budget-removal in pool** — given an exhausted source, its candidates never appear in + the walked list, while other sources' candidates remain (exhaustive mode). +5. **toggle independence** — orchestrator chooses `search_all_sources` only when + `search_mode == best_quality and mode == hybrid`; single-source and priority paths + call `search_with_fallback` (or single-client search) unchanged. + +Each test written first, watched fail, then minimal code to green. No production code +before a failing test. + +## Files touched + +- `core/quality/selection.py` — add `load_search_mode()`; small quality-rank helper for the sort key (or reuse model). +- `core/download_engine/engine.py` — add `search_all_sources`. +- `core/download_orchestrator.py` — branch `search()` on `search_mode`. +- `core/downloads/candidates.py` — add `quality_first` param + alternate sort key. +- `core/downloads/task_worker.py` — pass pooled results + `quality_first=True`; skip the redundant hybrid-fallback block in best-quality mode. (Query generator block untouched.) +- `database/music_database.py` — include `search_mode` default in the v3 profile. +- `webui/index.html`, `webui/static/settings.js` — the toggle. +- `tests/quality/`, `tests/downloads/` — the tests above. + +## Explicitly out of scope + +- No change to the query generator. +- No change to the budget counters / `MAX_*` constants / `requeue_quarantined_task_for_retry`. +- No change to AcoustID / force_import behavior. +- No cross-query pooling: best-quality pools across **sources** within each query of the + existing loop. The primary query already covers all sources, so this captures the best + candidate without restructuring the loop. Later queries remain per-source fallbacks for + the empty-result case.