soulsync/docs/superpowers/specs/2026-06-14-best-quality-search-mode-design.md

Best-Quality Search Mode — Design

Date: 2026-06-14 Branch: (new, off current fix/import-folder-artist-override-optin) Status: Approved approach (Option A), pending spec review

Goal

Add a user-toggleable download strategy. Today hybrid search is priority-first: engine.search_with_fallback walks the source chain in priority order and accepts the first source that meets a quality target — so a passable Soulseek 16-bit FLAC "wins" even when HiFi/Qobuz/Tidal could deliver a 24-bit version of the same track.

The new best-quality mode instead searches all configured sources, pools their candidates, and works them best→worst by actual audio quality. Source priority becomes only a tiebreaker between equally-good candidates.

Hard constraints (from the user)

1. Two independent toggles. The new search_mode and the existing post_processing.retry_exhaustive are orthogonal. The feature must behave correctly in all four combinations (priority/best × default/exhaustive budget).
2. Budget semantics preserved 1:1. No change to how retries are counted:
   • Default mode: single global cap MAX_QUARANTINE_RETRIES = 5 across all sources.
   • Exhaustive mode: per-source budget query_count × retries_per_query. When a source completely spends its budget, all of that source's candidates are removed from the entire pooled list (not just skipped once).
3. Do not touch the query generator. matching_engine.generate_download_queries and the legacy query-building block in download_track_worker stay exactly as-is.
4. force_import never fires on quality (unchanged existing invariant — only AcoustID mismatches can force-import).

Key realization: the budget-removal mechanism already exists

When a source spends its budget, monitor.requeue_quarantined_task_for_retry already adds it to the task's exhausted_download_sources set, and the worker already passes that set as exclude_sources to its searches. Constraint #2's "remove the source from the whole list" is therefore not new logic — best-quality mode simply consults the same set when assembling/filtering its pooled candidate list. No new budget bookkeeping is introduced.

Architecture

The one genuinely new piece: quality-dominant candidate ordering

attempt_download_with_candidates (core/downloads/candidates.py:71) sorts candidates confidence-first, then quality_score. That is correct for priority mode (never download a high-quality wrong file). For best-quality mode we keep the "correct-first" guarantee — get_valid_candidates already drops candidates below the match threshold, so every candidate in the list is correctly matched — but among those valid candidates we want profile quality rank to dominate, with confidence as the tiebreaker.

Add an opt-in parameter:

attempt_download_with_candidates(task_id, candidates, track, batch_id=None,
                                 deps=None, *, quality_first=False)

• quality_first=False (default): today's sort, byte-for-byte unchanged.
• quality_first=True: sort key becomes (profile_quality_rank, confidence, upload_speed, -queue_length, free_slots, size), i.e. profile quality dominates, all existing signals become tiebreakers.

profile_quality_rank is derived from the candidate's stamped AudioQuality against the user's ranked_targets (reusing core.quality.model / filter_and_rank ordering). Candidates with no usable quality info sort last (rank = worst), never dropped.

New engine method: search_all_sources

Mirror of search_with_fallback, but it does not stop at the first satisfying source. It iterates every configured, non-excluded source in the chain, runs each source's search, and returns the combined raw track list (each track already quality-stamped by its client's set_quality, as today). Per-source exceptions are swallowed exactly like the existing method. Excluded (exhausted) sources are skipped.

async def search_all_sources(self, query, source_chain, timeout=None,
                             progress_callback=None, exclude_sources=None)
        -> Tuple[list, list]   # (combined_tracks, combined_albums)

Quality ranking is not done here — the orchestrator/worker owns final ranking, same division of labour as today (engine returns raw, orchestrator filters/ranks).

Orchestrator wiring

download_orchestrator.search(...) gains awareness of search_mode:

• priority (default) → unchanged: search_with_fallback.
• best_quality and mode == 'hybrid' → search_all_sources.
• Single-source mode → unchanged regardless of search_mode (only one source exists; its candidates are still quality-ranked downstream, so the toggle is a no-op there).

A thin helper load_search_mode() -> str reads the quality profile (see Config).

Worker integration (the per-query loop stays intact)

In download_track_worker's existing query loop (core/downloads/task_worker.py:398), when best-quality mode is active:

1. The loop and the query generator are untouched.
2. orchestrator.search(query, exclude_sources=_exclude_sources) now returns pooled results from all non-exhausted sources (via search_all_sources).
3. candidates = get_valid_candidates(pooled, track, query) — unchanged.
4. ranked, _ = rank_for_profile(candidates) — orders best→worst by profile quality.
5. attempt_download_with_candidates(task_id, ranked, track, batch_id, quality_first=True).

Because the pool already came from every source, the hybrid-fallback block (task_worker.py:529–593, which re-tries hybrid_order[1:] individually) is redundant in best-quality mode and is skipped — it would just re-search sources the pool already covered. In priority mode it runs exactly as today.

The exhausted_download_sources set is already folded into _exclude_sources (task_worker.py:446), so a budget-spent source is excluded from search_all_sources's pool automatically — satisfying constraint #2 with no new code. A belt-and-suspenders filter drops any straggler exhausted-source candidates before step 5.

Config / persistence

search_mode belongs to the quality profile (it is a quality-driven policy). Stored in the v3 quality-profile JSON (database.music_database.get_quality_profile):

{ "version": 3, "search_mode": "priority", ... }

• Default: "priority" (preserves today's behavior for every existing install).
• Accessor: extend core/quality/selection.py with load_search_mode() -> str returning 'priority' unless the profile says 'best_quality'.
• No migration needed: a profile lacking the key reads as 'priority'.

UI

A single toggle in the existing Quality Profile tile (webui/index.html + webui/static/settings.js), e.g. a labelled switch:

Search strategy: ( ) Source priority — fastest, stops at first good source (•) Best quality — searches all sources, picks the highest quality

With a short note that best-quality searches all sources every track (slower / more API calls). collectQualityProfileFromUI emits search_mode; populateQualityProfileUI reads it. No change to the ranked-targets editor or the fallback toggle.

Behaviour matrix (all four combinations must hold)

search_mode	retry_exhaustive	Behaviour
priority	off	Unchanged today. First satisfying source wins; 5 global retries.
priority	on	Unchanged today. First source wins; per-source budgets.
best_quality	off	Pool all sources, best→worst; 5 global retries total. A source can't be "removed from the pool" because there's no per-source budget — the global cap fires first. (Documented: combine best-quality with exhaustive for per-source removal.)
best_quality	on	Pool all sources, best→worst; each source has query_count × retries_per_query; on spend → source removed from the whole pool via exhausted_download_sources. This is the user's target configuration.

Error handling

• search_all_sources: fail-open per source (swallow exceptions, keep going), identical to search_with_fallback. If the whole pool is empty, the worker falls through to its existing not-found handling.
• Quality-rank computation: any candidate that can't be ranked sorts last, never crashes the walk (mirrors the engine's existing fail-open ranking).

Testing (TDD)

New tests under tests/quality/ and tests/downloads/:

1. load_search_mode — defaults to 'priority'; returns 'best_quality' when set; unknown value falls back to 'priority'.
2. search_all_sources — returns combined candidates from multiple fake sources; skips excluded/exhausted sources; swallows a raising source; empty when all empty.
3. quality-first ordering — attempt_download_with_candidates(quality_first=True) tries a 24-bit candidate before a higher-confidence 16-bit one; quality_first=False reproduces today's confidence-first order exactly (regression lock).
4. budget-removal in pool — given an exhausted source, its candidates never appear in the walked list, while other sources' candidates remain (exhaustive mode).
5. toggle independence — orchestrator chooses search_all_sources only when search_mode == best_quality and mode == hybrid; single-source and priority paths call search_with_fallback (or single-client search) unchanged.

Each test written first, watched fail, then minimal code to green. No production code before a failing test.

Files touched

• core/quality/selection.py — add load_search_mode(); small quality-rank helper for the sort key (or reuse model).
• core/download_engine/engine.py — add search_all_sources.
• core/download_orchestrator.py — branch search() on search_mode.
• core/downloads/candidates.py — add quality_first param + alternate sort key.
• core/downloads/task_worker.py — pass pooled results + quality_first=True; skip the redundant hybrid-fallback block in best-quality mode. (Query generator block untouched.)
• database/music_database.py — include search_mode default in the v3 profile.
• webui/index.html, webui/static/settings.js — the toggle.
• tests/quality/, tests/downloads/ — the tests above.

Explicitly out of scope

• No change to the query generator.
• No change to the budget counters / MAX_* constants / requeue_quarantined_task_for_retry.
• No change to AcoustID / force_import behavior.
• No cross-query pooling: best-quality pools across sources within each query of the existing loop. The primary query already covers all sources, so this captures the best candidate without restructuring the loop. Later queries remain per-source fallbacks for the empty-result case.