docs(spec): best-quality search mode design
Toggle to search all sources and download best→worst by actual audio quality, vs today's priority-first (first satisfying source wins). Reuses exhausted_download_sources for per-source budget removal; query generator and budget counters untouched. Independent of retry_exhaustive. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
dev
parent
11d2fa9ad6
commit
31d85e59c9
1 changed files with 201 additions and 0 deletions
|
|
@ -0,0 +1,201 @@
|
|||
# Best-Quality Search Mode — Design
|
||||
|
||||
**Date:** 2026-06-14
|
||||
**Branch:** (new, off current `fix/import-folder-artist-override-optin`)
|
||||
**Status:** Approved approach (Option A), pending spec review
|
||||
|
||||
## Goal
|
||||
|
||||
Add a user-toggleable download strategy. Today hybrid search is **priority-first**:
|
||||
`engine.search_with_fallback` walks the source chain in priority order and accepts
|
||||
the **first** source that meets a quality target — so a passable Soulseek 16-bit FLAC
|
||||
"wins" even when HiFi/Qobuz/Tidal could deliver a 24-bit version of the same track.
|
||||
|
||||
The new **best-quality** mode instead searches **all** configured sources, pools their
|
||||
candidates, and works them **best→worst by actual audio quality**. Source priority
|
||||
becomes only a tiebreaker between equally-good candidates.
|
||||
|
||||
## Hard constraints (from the user)
|
||||
|
||||
1. **Two independent toggles.** The new `search_mode` and the existing
|
||||
`post_processing.retry_exhaustive` are orthogonal. The feature must behave
|
||||
correctly in **all four** combinations (priority/best × default/exhaustive budget).
|
||||
2. **Budget semantics preserved 1:1.** No change to how retries are counted:
|
||||
- Default mode: single global cap `MAX_QUARANTINE_RETRIES = 5` across all sources.
|
||||
- Exhaustive mode: per-source budget `query_count × retries_per_query`.
|
||||
When a source **completely spends** its budget, **all of that source's candidates
|
||||
are removed from the entire pooled list** (not just skipped once).
|
||||
3. **Do not touch the query generator.** `matching_engine.generate_download_queries`
|
||||
and the legacy query-building block in `download_track_worker` stay exactly as-is.
|
||||
4. **`force_import` never fires on quality** (unchanged existing invariant — only
|
||||
AcoustID mismatches can force-import).
|
||||
|
||||
## Key realization: the budget-removal mechanism already exists
|
||||
|
||||
When a source spends its budget, `monitor.requeue_quarantined_task_for_retry`
|
||||
already adds it to the task's **`exhausted_download_sources`** set, and the worker
|
||||
already passes that set as `exclude_sources` to its searches. Constraint #2's
|
||||
"remove the source from the whole list" is therefore **not new logic** — best-quality
|
||||
mode simply consults the *same* set when assembling/filtering its pooled candidate
|
||||
list. No new budget bookkeeping is introduced.
|
||||
|
||||
## Architecture
|
||||
|
||||
### The one genuinely new piece: quality-dominant candidate ordering
|
||||
|
||||
`attempt_download_with_candidates` (core/downloads/candidates.py:71) sorts candidates
|
||||
**confidence-first, then `quality_score`**. That is correct for priority mode (never
|
||||
download a high-quality *wrong* file). For best-quality mode we keep the
|
||||
"correct-first" guarantee — `get_valid_candidates` already drops candidates below the
|
||||
match threshold, so **every candidate in the list is correctly matched** — but among
|
||||
those valid candidates we want **profile quality rank to dominate**, with confidence as
|
||||
the tiebreaker.
|
||||
|
||||
Add an opt-in parameter:
|
||||
|
||||
```python
|
||||
attempt_download_with_candidates(task_id, candidates, track, batch_id=None,
|
||||
deps=None, *, quality_first=False)
|
||||
```
|
||||
|
||||
- `quality_first=False` (default): today's sort, byte-for-byte unchanged.
|
||||
- `quality_first=True`: sort key becomes
|
||||
`(profile_quality_rank, confidence, upload_speed, -queue_length, free_slots, size)`,
|
||||
i.e. profile quality dominates, all existing signals become tiebreakers.
|
||||
|
||||
`profile_quality_rank` is derived from the candidate's stamped `AudioQuality` against
|
||||
the user's `ranked_targets` (reusing `core.quality.model` / `filter_and_rank` ordering).
|
||||
Candidates with no usable quality info sort last (rank = worst), never dropped.
|
||||
|
||||
### New engine method: `search_all_sources`
|
||||
|
||||
Mirror of `search_with_fallback`, but it does **not** stop at the first satisfying
|
||||
source. It iterates every configured, non-excluded source in the chain, runs each
|
||||
source's `search`, and returns the **combined** raw track list (each track already
|
||||
quality-stamped by its client's `set_quality`, as today). Per-source exceptions are
|
||||
swallowed exactly like the existing method. Excluded (`exhausted`) sources are skipped.
|
||||
|
||||
```python
|
||||
async def search_all_sources(self, query, source_chain, timeout=None,
|
||||
progress_callback=None, exclude_sources=None)
|
||||
-> Tuple[list, list] # (combined_tracks, combined_albums)
|
||||
```
|
||||
|
||||
Quality ranking is **not** done here — the orchestrator/worker owns final ranking, same
|
||||
division of labour as today (engine returns raw, orchestrator filters/ranks).
|
||||
|
||||
### Orchestrator wiring
|
||||
|
||||
`download_orchestrator.search(...)` gains awareness of `search_mode`:
|
||||
- `priority` (default) → unchanged: `search_with_fallback`.
|
||||
- `best_quality` **and** `mode == 'hybrid'` → `search_all_sources`.
|
||||
- Single-source mode → unchanged regardless of `search_mode` (only one source exists;
|
||||
its candidates are still quality-ranked downstream, so the toggle is a no-op there).
|
||||
|
||||
A thin helper `load_search_mode() -> str` reads the quality profile (see Config).
|
||||
|
||||
### Worker integration (the per-query loop stays intact)
|
||||
|
||||
In `download_track_worker`'s existing query loop (core/downloads/task_worker.py:398),
|
||||
when best-quality mode is active:
|
||||
|
||||
1. The loop and the query generator are untouched.
|
||||
2. `orchestrator.search(query, exclude_sources=_exclude_sources)` now returns
|
||||
**pooled** results from all non-exhausted sources (via `search_all_sources`).
|
||||
3. `candidates = get_valid_candidates(pooled, track, query)` — unchanged.
|
||||
4. `ranked, _ = rank_for_profile(candidates)` — orders best→worst by profile quality.
|
||||
5. `attempt_download_with_candidates(task_id, ranked, track, batch_id, quality_first=True)`.
|
||||
|
||||
Because the pool already came from every source, the **hybrid-fallback block**
|
||||
(task_worker.py:529–593, which re-tries `hybrid_order[1:]` individually) is
|
||||
**redundant in best-quality mode** and is skipped — it would just re-search sources the
|
||||
pool already covered. In priority mode it runs exactly as today.
|
||||
|
||||
The `exhausted_download_sources` set is already folded into `_exclude_sources`
|
||||
(task_worker.py:446), so a budget-spent source is excluded from `search_all_sources`'s
|
||||
pool automatically — satisfying constraint #2 with no new code. A belt-and-suspenders
|
||||
filter drops any straggler exhausted-source candidates before step 5.
|
||||
|
||||
## Config / persistence
|
||||
|
||||
`search_mode` belongs to the quality profile (it *is* a quality-driven policy). Stored
|
||||
in the v3 quality-profile JSON (`database.music_database.get_quality_profile`):
|
||||
|
||||
```json
|
||||
{ "version": 3, "search_mode": "priority", ... }
|
||||
```
|
||||
|
||||
- Default: `"priority"` (preserves today's behavior for every existing install).
|
||||
- Accessor: extend `core/quality/selection.py` with `load_search_mode() -> str`
|
||||
returning `'priority'` unless the profile says `'best_quality'`.
|
||||
- No migration needed: a profile lacking the key reads as `'priority'`.
|
||||
|
||||
## UI
|
||||
|
||||
A single toggle in the existing **Quality Profile** tile (webui/index.html +
|
||||
webui/static/settings.js), e.g. a labelled switch:
|
||||
|
||||
> **Search strategy:** ( ) Source priority — fastest, stops at first good source
|
||||
> (•) Best quality — searches all sources, picks the highest quality
|
||||
|
||||
With a short note that best-quality searches **all** sources every track (slower / more
|
||||
API calls). `collectQualityProfileFromUI` emits `search_mode`; `populateQualityProfileUI`
|
||||
reads it. No change to the ranked-targets editor or the fallback toggle.
|
||||
|
||||
## Behaviour matrix (all four combinations must hold)
|
||||
|
||||
| search_mode | retry_exhaustive | Behaviour |
|
||||
|---|---|---|
|
||||
| priority | off | **Unchanged today.** First satisfying source wins; 5 global retries. |
|
||||
| priority | on | **Unchanged today.** First source wins; per-source budgets. |
|
||||
| best_quality | off | Pool all sources, best→worst; 5 **global** retries total. A source can't be "removed from the pool" because there's no per-source budget — the global cap fires first. (Documented: combine best-quality with exhaustive for per-source removal.) |
|
||||
| best_quality | on | Pool all sources, best→worst; each source has `query_count × retries_per_query`; on spend → source removed from the whole pool via `exhausted_download_sources`. **This is the user's target configuration.** |
|
||||
|
||||
## Error handling
|
||||
|
||||
- `search_all_sources`: fail-open per source (swallow exceptions, keep going), identical
|
||||
to `search_with_fallback`. If the whole pool is empty, the worker falls through to its
|
||||
existing not-found handling.
|
||||
- Quality-rank computation: any candidate that can't be ranked sorts last, never crashes
|
||||
the walk (mirrors the engine's existing fail-open ranking).
|
||||
|
||||
## Testing (TDD)
|
||||
|
||||
New tests under `tests/quality/` and `tests/downloads/`:
|
||||
|
||||
1. **`load_search_mode`** — defaults to `'priority'`; returns `'best_quality'` when set;
|
||||
unknown value falls back to `'priority'`.
|
||||
2. **`search_all_sources`** — returns combined candidates from multiple fake sources;
|
||||
skips excluded/exhausted sources; swallows a raising source; empty when all empty.
|
||||
3. **quality-first ordering** — `attempt_download_with_candidates(quality_first=True)`
|
||||
tries a 24-bit candidate before a higher-confidence 16-bit one; `quality_first=False`
|
||||
reproduces today's confidence-first order exactly (regression lock).
|
||||
4. **budget-removal in pool** — given an exhausted source, its candidates never appear in
|
||||
the walked list, while other sources' candidates remain (exhaustive mode).
|
||||
5. **toggle independence** — orchestrator chooses `search_all_sources` only when
|
||||
`search_mode == best_quality and mode == hybrid`; single-source and priority paths
|
||||
call `search_with_fallback` (or single-client search) unchanged.
|
||||
|
||||
Each test written first, watched fail, then minimal code to green. No production code
|
||||
before a failing test.
|
||||
|
||||
## Files touched
|
||||
|
||||
- `core/quality/selection.py` — add `load_search_mode()`; small quality-rank helper for the sort key (or reuse model).
|
||||
- `core/download_engine/engine.py` — add `search_all_sources`.
|
||||
- `core/download_orchestrator.py` — branch `search()` on `search_mode`.
|
||||
- `core/downloads/candidates.py` — add `quality_first` param + alternate sort key.
|
||||
- `core/downloads/task_worker.py` — pass pooled results + `quality_first=True`; skip the redundant hybrid-fallback block in best-quality mode. (Query generator block untouched.)
|
||||
- `database/music_database.py` — include `search_mode` default in the v3 profile.
|
||||
- `webui/index.html`, `webui/static/settings.js` — the toggle.
|
||||
- `tests/quality/`, `tests/downloads/` — the tests above.
|
||||
|
||||
## Explicitly out of scope
|
||||
|
||||
- No change to the query generator.
|
||||
- No change to the budget counters / `MAX_*` constants / `requeue_quarantined_task_for_retry`.
|
||||
- No change to AcoustID / force_import behavior.
|
||||
- No cross-query pooling: best-quality pools across **sources** within each query of the
|
||||
existing loop. The primary query already covers all sources, so this captures the best
|
||||
candidate without restructuring the loop. Later queries remain per-source fallbacks for
|
||||
the empty-result case.
|
||||
Loading…
Reference in a new issue