remove docs from PR
This commit is contained in:
parent
e7bc77cb04
commit
17886477cb
2 changed files with 0 additions and 531 deletions
|
|
@ -1,378 +0,0 @@
|
|||
# AcoustID Verification Unification + Status Tracking — Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Make import-time verification and the library AcoustID scan share ONE decision core, and persist a per-track verification status (DB + file tag) surfaced on the Downloads page.
|
||||
|
||||
**Architecture:** Extract a pure `core/matching/audio_verification.py` (`normalize` + `evaluate`) built on the existing shared helpers (`artist_aliases`, `script_compat`, `acoustid_candidates`, `version_mismatch`). `acoustid_verification.verify_audio_file` (import) and `acoustid_scanner._scan_file` (scan) delegate the decision to it. A new `tracks.verification_status` column + `SOULSYNC_VERIFICATION` file tag record `verified` / `unverified` / `force_imported`; the Downloads page shows a badge.
|
||||
|
||||
**Tech Stack:** Python 3.11, pytest (`.venv/bin/python -m pytest`), mutagen, SQLite, vanilla JS webui.
|
||||
|
||||
**Spec:** `docs/superpowers/specs/2026-06-10-acoustid-verification-unification-design.md`
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- **Create** `core/matching/audio_verification.py` — the shared `normalize()` + `evaluate()` decision core (pure, no I/O). Single source of truth for normalization, thresholds, alias-aware comparison, cross-script SKIP, version gate, duration guard.
|
||||
- **Create** `core/matching/verification_status.py` — the status vocabulary (`VERIFIED`, `UNVERIFIED`, `FORCE_IMPORTED`) + `status_from_outcome(decision)` + `status_from_context(context)` mapping helpers. Tiny, pure, testable.
|
||||
- **Modify** `core/acoustid_verification.py` — `_normalize`/`_similarity`/`_alias_aware_artist_sim`/`_find_best_title_artist_match` and the decision branches in `verify_audio_file` delegate to the core. Keep the import-facing wrapper (fingerprint lookup, MB alias provider, `VerificationResult`).
|
||||
- **Modify** `core/repair_jobs/acoustid_scanner.py` — drop the private `_normalize` (line 485) and the inline decision; call `normalize` + `evaluate`.
|
||||
- **Modify** `database/music_database.py` — additive migration: `tracks.verification_status TEXT`.
|
||||
- **Modify** `core/tag_writer.py` — write `SOULSYNC_VERIFICATION` in `_write_vorbis`/`_write_id3`/`_write_mp4`; read it in `read_file_tags`.
|
||||
- **Modify** `core/imports/pipeline.py` — compute + persist the status (DB + tag) after post-processing; stash on the download context.
|
||||
- **Modify** `web_server.py` + `webui/static/downloads.js` + `webui/index.html` — surface the status badge.
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Core `normalize()`
|
||||
|
||||
**Files:**
|
||||
- Create: `core/matching/audio_verification.py`
|
||||
- Test: `tests/matching/test_audio_verification_core.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
```python
|
||||
from core.matching.audio_verification import normalize
|
||||
|
||||
def test_normalize_strips_paren_bracket_angle_and_keeps_cjk():
|
||||
assert normalize("澤野弘之 <Vocal: MIKA KOBAYASHI>") == "澤野弘之"
|
||||
assert normalize("Clarity (Live at X) [Remastered]") == "clarity"
|
||||
assert normalize("Attack on Titan <TV Size>") == "attack on titan"
|
||||
|
||||
def test_normalize_strips_version_and_featuring():
|
||||
assert normalize("In My Feelings - Instrumental") == "in my feelings"
|
||||
assert normalize("Song feat. Someone") == "song"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `.venv/bin/python -m pytest tests/matching/test_audio_verification_core.py -q`
|
||||
Expected: FAIL — `ModuleNotFoundError: core.matching.audio_verification`.
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
Port the current (already-correct) body of `core/acoustid_verification.py::_normalize`
|
||||
verbatim into the new module (it already strips `()`/`[]`/`<>`/version/featuring and
|
||||
keeps CJK via `\w`). Add the thresholds as module constants for later tasks.
|
||||
|
||||
```python
|
||||
"""Shared audio-verification decision core (pure; no file/DB I/O).
|
||||
|
||||
Single source of truth for normalization + the PASS/SKIP/FAIL decision used by
|
||||
BOTH import-time verification (core/acoustid_verification.py) and the library
|
||||
scan (core/repair_jobs/acoustid_scanner.py).
|
||||
"""
|
||||
import re
|
||||
from difflib import SequenceMatcher
|
||||
|
||||
MIN_ACOUSTID_SCORE = 0.80
|
||||
TITLE_MATCH_THRESHOLD = 0.70
|
||||
ARTIST_MATCH_THRESHOLD = 0.60
|
||||
|
||||
def normalize(text: str) -> str:
|
||||
if not text:
|
||||
return ""
|
||||
s = text.lower().strip()
|
||||
s = re.sub(r'\s*\([^)]*\)', '', s)
|
||||
s = re.sub(r'\s*\[[^\]]*\]', '', s)
|
||||
s = re.sub(r'\s*<[^>]*>', '', s)
|
||||
s = re.sub(r'\s+(?:feat\.?|ft\.?|featuring)\s+.*$', '', s, flags=re.IGNORECASE)
|
||||
s = re.sub(r'\s*-\s*(?:vocal|instrumental|acoustic|live|remix|cover|clean|explicit|radio\s*edit|original\s*mix|extended\s*mix|club\s*mix)\s*$', '', s, flags=re.IGNORECASE)
|
||||
s = re.sub(r'\s*-\s*from\s+.+$', '', s, flags=re.IGNORECASE)
|
||||
s = re.sub(r'[^\w\s]', '', s)
|
||||
s = re.sub(r'\s+', ' ', s).strip()
|
||||
return s
|
||||
|
||||
def similarity(a: str, b: str) -> float:
|
||||
na, nb = normalize(a), normalize(b)
|
||||
if not na or not nb:
|
||||
return 0.0
|
||||
if na == nb:
|
||||
return 1.0
|
||||
return SequenceMatcher(None, na, nb).ratio()
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `.venv/bin/python -m pytest tests/matching/test_audio_verification_core.py -q`
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add core/matching/audio_verification.py tests/matching/test_audio_verification_core.py
|
||||
git commit -m "feat(verification): shared normalize() core for import + scan"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Core `evaluate()` decision
|
||||
|
||||
**Files:**
|
||||
- Modify: `core/matching/audio_verification.py`
|
||||
- Test: `tests/matching/test_audio_verification_core.py`
|
||||
|
||||
`evaluate` reproduces the import-path decision (it is the richer of the two), so it
|
||||
is behaviour-preserving for import and an upgrade for scan. Port the logic from
|
||||
`acoustid_verification.verify_audio_file` Steps 4b–end (version gate via
|
||||
`is_acceptable_version_mismatch`, alias-aware artist sim, secondary/scan match via
|
||||
`find_matching_recording`, cross-script SKIP via `is_cross_script_mismatch`,
|
||||
duration guard via `duration_mismatches_strongly`).
|
||||
|
||||
- [ ] **Step 1: Write the failing tests** (the three real-world cases)
|
||||
|
||||
```python
|
||||
from core.matching.audio_verification import evaluate, Decision
|
||||
|
||||
REC = lambda t, a, d=None: {"title": t, "artist": a, "duration": d}
|
||||
|
||||
def test_cross_script_artist_with_vocal_credit_skips_not_fails():
|
||||
# Sawano / 澤野弘之 <Vocal: ...> + IPA title -> SKIP, never FAIL
|
||||
out = evaluate(
|
||||
"Call Your Name", "Sawano Hiroyuki",
|
||||
[REC("call your name", "澤野弘之 <Vocal: mpi & CASG>")],
|
||||
fingerprint_score=0.95,
|
||||
aliases_provider=lambda: ["澤野弘之"],
|
||||
)
|
||||
assert out.decision == Decision.SKIP
|
||||
|
||||
def test_clean_match_passes():
|
||||
out = evaluate("Xl-Tt", "Sawano Hiroyuki",
|
||||
[REC("xl-tt", "澤野弘之")], fingerprint_score=0.95,
|
||||
aliases_provider=lambda: ["澤野弘之"])
|
||||
assert out.decision == Decision.PASS
|
||||
|
||||
def test_genuine_wrong_song_fails():
|
||||
out = evaluate("Yellow", "Coldplay",
|
||||
[REC("Rich Interlude", "Kendrick Lamar")], fingerprint_score=0.85)
|
||||
assert out.decision == Decision.FAIL
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run to verify failure**
|
||||
|
||||
Run: `.venv/bin/python -m pytest tests/matching/test_audio_verification_core.py -k evaluate -q`
|
||||
Expected: FAIL — `cannot import name 'evaluate'`.
|
||||
|
||||
- [ ] **Step 3: Implement `evaluate` + `Decision` + `Outcome`**
|
||||
|
||||
Add an `enum Decision {PASS, SKIP, FAIL}`, a dataclass `Outcome(decision, title_sim,
|
||||
artist_sim, matched_title, matched_artist, reason)`, and `_alias_aware_artist_sim`
|
||||
(ported from acoustid_verification). Port the decision sequence from
|
||||
`verify_audio_file` lines 470–703 into `evaluate`, taking `recordings` +
|
||||
`fingerprint_score` + `file_duration_s` + `aliases_provider` as params and
|
||||
returning an `Outcome` instead of `(VerificationResult, msg)`. Reuse the existing
|
||||
imports: `from core.matching.artist_aliases import artist_names_match, best_alias_match`,
|
||||
`from core.matching.script_compat import is_cross_script_mismatch`,
|
||||
`from core.matching.acoustid_candidates import find_matching_recording, duration_mismatches_strongly`,
|
||||
`from core.matching.version_mismatch import is_acceptable_version_mismatch`, and
|
||||
`MusicMatchingEngine().detect_version_type` for `_detect_title_version`.
|
||||
|
||||
- [ ] **Step 4: Run to verify pass**
|
||||
|
||||
Run: `.venv/bin/python -m pytest tests/matching/test_audio_verification_core.py -q`
|
||||
Expected: PASS (all).
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add core/matching/audio_verification.py tests/matching/test_audio_verification_core.py
|
||||
git commit -m "feat(verification): shared evaluate() PASS/SKIP/FAIL decision core"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Import path delegates to the core
|
||||
|
||||
**Files:**
|
||||
- Modify: `core/acoustid_verification.py`
|
||||
- Test (regression): `tests/test_acoustid_skip_logic.py`, `tests/test_acoustid_version_mismatch.py`, `tests/test_acoustid_normalize_angle_annotations.py`, `tests/test_acoustid_error_reporting.py`
|
||||
|
||||
- [ ] **Step 1: Run the existing suite to capture green baseline**
|
||||
|
||||
Run: `.venv/bin/python -m pytest tests/test_acoustid_skip_logic.py tests/test_acoustid_version_mismatch.py tests/test_acoustid_normalize_angle_annotations.py tests/test_acoustid_error_reporting.py -q`
|
||||
Expected: PASS (baseline before refactor).
|
||||
|
||||
- [ ] **Step 2: Refactor**
|
||||
|
||||
In `core/acoustid_verification.py`: replace the bodies of `_normalize`/`_similarity`
|
||||
with re-exports from the core (`from core.matching.audio_verification import normalize as _normalize, similarity as _similarity`). Replace the decision block in
|
||||
`verify_audio_file` (Steps 4b–end) with a single call to `evaluate(...)`, then map
|
||||
`Outcome.decision` → `VerificationResult` (PASS→PASS, SKIP→SKIP, FAIL→FAIL) and pass
|
||||
`Outcome.reason` through. Keep the existing fingerprint lookup, MB enrichment, and
|
||||
`_resolve_expected_artist_aliases` thunk (pass it as `aliases_provider`).
|
||||
|
||||
- [ ] **Step 3: Run regression suite**
|
||||
|
||||
Run: same command as Step 1.
|
||||
Expected: PASS (unchanged behaviour for import).
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add core/acoustid_verification.py
|
||||
git commit -m "refactor(verification): import path delegates to shared core"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Scanner uses the core (gains alias bridge + cross-script SKIP)
|
||||
|
||||
**Files:**
|
||||
- Modify: `core/repair_jobs/acoustid_scanner.py`
|
||||
- Test: `tests/test_acoustid_scanner.py` (regression) + new case
|
||||
|
||||
- [ ] **Step 1: Write the failing test** (cross-script track must NOT create a finding)
|
||||
|
||||
```python
|
||||
# tests/test_acoustid_scanner_cross_script.py
|
||||
def test_scanner_does_not_flag_cross_script_anime_ost(monkeypatch):
|
||||
# Build a scan over one track expected "Call Your Name" / "Sawano Hiroyuki"
|
||||
# whose fingerprint returns "澤野弘之 <Vocal: ...>"; assert findings_created == 0.
|
||||
...
|
||||
```
|
||||
|
||||
(Concrete construction mirrors `tests/test_acoustid_scanner.py` fixtures; assert the
|
||||
mismatch branch is NOT reached.)
|
||||
|
||||
- [ ] **Step 2: Run to verify failure**
|
||||
|
||||
Run: `.venv/bin/python -m pytest tests/test_acoustid_scanner_cross_script.py -q`
|
||||
Expected: FAIL — a finding IS created (current scanner strips non-ASCII, no alias).
|
||||
|
||||
- [ ] **Step 3: Refactor scanner**
|
||||
|
||||
Delete `_normalize` (line 485) and import `normalize` from the core. Replace the
|
||||
`_scan_file` similarity + decision block (the `title_sim`/`artist_sim`/finding logic)
|
||||
with a call to `evaluate(...)` passing `fp_result['recordings']`, `best_score`,
|
||||
`file_duration_s`, and an `aliases_provider` (reuse the MB alias lookup, or pass the
|
||||
DB-resolved aliases). Create a finding only when `Outcome.decision == Decision.FAIL`.
|
||||
|
||||
- [ ] **Step 4: Run new + existing scanner tests**
|
||||
|
||||
Run: `.venv/bin/python -m pytest tests/test_acoustid_scanner.py tests/test_acoustid_scanner_cross_script.py -q`
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add core/repair_jobs/acoustid_scanner.py tests/test_acoustid_scanner_cross_script.py
|
||||
git commit -m "refactor(scanner): use shared verification core; stop false-flagging cross-script"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Status vocabulary + mapping
|
||||
|
||||
**Files:**
|
||||
- Create: `core/matching/verification_status.py`
|
||||
- Test: `tests/matching/test_verification_status.py`
|
||||
|
||||
- [ ] **Step 1: Failing test**
|
||||
|
||||
```python
|
||||
from core.matching.verification_status import (
|
||||
VERIFIED, UNVERIFIED, FORCE_IMPORTED, status_from_decision, status_from_context)
|
||||
from core.matching.audio_verification import Decision
|
||||
|
||||
def test_decision_maps_to_status():
|
||||
assert status_from_decision(Decision.PASS) == VERIFIED
|
||||
assert status_from_decision(Decision.SKIP) == UNVERIFIED
|
||||
|
||||
def test_force_import_context_wins():
|
||||
assert status_from_context({"_version_mismatch_fallback": "instrumental"}) == FORCE_IMPORTED
|
||||
assert status_from_context({}) is None
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run → fail.** `.venv/bin/python -m pytest tests/matching/test_verification_status.py -q`
|
||||
|
||||
- [ ] **Step 3: Implement** the three string constants + the two pure mappers.
|
||||
|
||||
- [ ] **Step 4: Run → pass.**
|
||||
|
||||
- [ ] **Step 5: Commit** `feat(verification): status vocabulary + mappers`.
|
||||
|
||||
---
|
||||
|
||||
## Task 6: DB migration `tracks.verification_status`
|
||||
|
||||
**Files:**
|
||||
- Modify: `database/music_database.py`
|
||||
- Test: `tests/test_verification_status_migration.py`
|
||||
|
||||
- [ ] **Step 1: Failing test** — open a fresh DB, assert `verification_status` is in `PRAGMA table_info(tracks)`.
|
||||
- [ ] **Step 2: Run → fail.**
|
||||
- [ ] **Step 3: Implement** following the existing additive pattern:
|
||||
|
||||
```python
|
||||
cursor.execute("PRAGMA table_info(tracks)")
|
||||
cols = [r[1] for r in cursor.fetchall()]
|
||||
if 'verification_status' not in cols:
|
||||
cursor.execute("ALTER TABLE tracks ADD COLUMN verification_status TEXT")
|
||||
```
|
||||
(Place beside the other `tracks` ADD COLUMN migrations, e.g. near line 2372.)
|
||||
|
||||
- [ ] **Step 4: Run → pass.**
|
||||
- [ ] **Step 5: Commit** `feat(db): add tracks.verification_status`.
|
||||
|
||||
---
|
||||
|
||||
## Task 7: Write + read the `SOULSYNC_VERIFICATION` tag
|
||||
|
||||
**Files:**
|
||||
- Modify: `core/tag_writer.py`
|
||||
- Test: `tests/test_verification_tag_roundtrip.py`
|
||||
|
||||
- [ ] **Step 1: Failing test** — write a FLAC with `db_data={'verification_status': 'unverified', ...}`, read it back via `read_file_tags`, assert `tags['verification_status'] == 'unverified'`. (Build the FLAC with ffmpeg as in `/tmp` test harness; or reuse an existing tag-writer test fixture.)
|
||||
- [ ] **Step 2: Run → fail.**
|
||||
- [ ] **Step 3: Implement** — in `_write_vorbis` set `audio['SOULSYNC_VERIFICATION']=[status]`; in `_write_id3` add `TXXX(desc='SOULSYNC_VERIFICATION', text=[status])`; in `_write_mp4` set `----:com.soulsync:VERIFICATION`. Read all three back in `read_file_tags` into key `verification_status`. Only write when `db_data.get('verification_status')` is set (non-fatal on error).
|
||||
- [ ] **Step 4: Run → pass.**
|
||||
- [ ] **Step 5: Commit** `feat(tags): persist SOULSYNC_VERIFICATION tag`.
|
||||
|
||||
---
|
||||
|
||||
## Task 8: Persist status at import (DB + tag + context)
|
||||
|
||||
**Files:**
|
||||
- Modify: `core/imports/pipeline.py`
|
||||
- Test: `tests/imports/test_import_verification_status.py`
|
||||
|
||||
- [ ] **Step 1: Failing test** — drive `post_process_matched_download` (or the helper that records the library row) with a PASS verification result and assert the track row's `verification_status == 'verified'`; with `context['_version_mismatch_fallback']` set assert `'force_imported'`.
|
||||
- [ ] **Step 2: Run → fail.**
|
||||
- [ ] **Step 3: Implement** — after verification, compute `status = status_from_context(context) or status_from_decision(verification_decision)`; thread it into `db_data` (so the tag write in Task 7 picks it up) and into the library-row write (`tracks.verification_status`); also stash `context['_verification_status'] = status` for the Downloads payload.
|
||||
- [ ] **Step 4: Run → pass.**
|
||||
- [ ] **Step 5: Commit** `feat(import): record verification status (db+tag+context)`.
|
||||
|
||||
---
|
||||
|
||||
## Task 9: Downloads page badge
|
||||
|
||||
**Files:**
|
||||
- Modify: `web_server.py` (download-status payload), `webui/static/downloads.js`, `webui/index.html` (badge styles if needed)
|
||||
- Test: manual + a small JS-free assertion if a payload test exists
|
||||
|
||||
- [ ] **Step 1:** Add `verification_status` to the per-task download payload in `web_server.py` (read from `context['_verification_status']` / the task record).
|
||||
- [ ] **Step 2:** In `downloads.js`, render a badge per completed row: ✓ verified / ⚠ unverified / ⚑ force-imported, keyed on `task.verification_status`.
|
||||
- [ ] **Step 3:** Add minimal badge CSS in `index.html`/`style.css` mirroring existing status pills.
|
||||
- [ ] **Step 4:** Manual verify: import one clean + one cross-script track; confirm ✓ and ⚠ badges.
|
||||
- [ ] **Step 5: Commit** `feat(ui): show verification status badge on Downloads`.
|
||||
|
||||
---
|
||||
|
||||
## Final verification
|
||||
|
||||
- [ ] Run the full AcoustID + import suites:
|
||||
`.venv/bin/python -m pytest tests/test_acoustid_*.py tests/matching/ tests/imports/ -q`
|
||||
Expected: all PASS.
|
||||
- [ ] Build + push image (per build-deploy loop) for live testing.
|
||||
|
||||
## Self-review notes
|
||||
|
||||
- Spec coverage: unify (Tasks 1–4), status values (Task 5), DB (6), tag (7),
|
||||
import wiring (8), UI (9) — all spec sections mapped.
|
||||
- Behaviour-preserving for import (Task 3 regression-gated); intentional change for
|
||||
scan (Task 4 — the fix).
|
||||
- Tasks 4, 8, 9 reference existing fixtures/APIs the executor must read live
|
||||
(`test_acoustid_scanner.py` fixtures, the download payload shape, `downloads.js`
|
||||
row render) — flagged as such rather than guessed.
|
||||
|
|
@ -1,153 +0,0 @@
|
|||
# AcoustID Verification: Unify the Pipeline + Persist Verification Status
|
||||
|
||||
Date: 2026-06-10
|
||||
Branch: `fix/import-folder-artist-override-optin`
|
||||
|
||||
## Problem
|
||||
|
||||
Audio verification logic is **duplicated** across two code paths that have
|
||||
drifted apart, producing inconsistent results ("komische Fehler"):
|
||||
|
||||
- **Import time** — `core/acoustid_verification.py` (`verify_audio_file`,
|
||||
`_normalize` @ line 59, alias-aware artist sim, cross-script SKIP, version
|
||||
gate, duration guard). Keeps CJK characters; strips `()`/`[]`/`<>` (after the
|
||||
recent fix).
|
||||
- **Library scan** — `core/repair_jobs/acoustid_scanner.py` (its OWN `_normalize`
|
||||
@ line 485, which strips **all non-ASCII** — kanji/IPA vanish — and has NO
|
||||
cross-script alias bridge / SKIP). A full library scan therefore false-flags
|
||||
correct cross-script tracks (e.g. anime OSTs credited `澤野弘之 <Vocal: …>`).
|
||||
|
||||
Two `_normalize`s + two decision paths = the same bug must be fixed twice and
|
||||
they diverge. There is also no record of *how* a track passed verification, so a
|
||||
force-imported or skipped-but-imported track looks identical to a cleanly
|
||||
verified one.
|
||||
|
||||
## Goals
|
||||
|
||||
1. **One verification core** used by BOTH import and scan. Normalization,
|
||||
alias-aware comparison, cross-script handling, version gate, duration guard,
|
||||
and thresholds live in exactly one place.
|
||||
2. **Persist a verification status** per track in the DB **and** as a file tag,
|
||||
so it survives DB resets / file moves and can drive the UI.
|
||||
3. **Surface the status on the Downloads page** as an info badge.
|
||||
|
||||
## Consumer map (discovered 2026-06-10)
|
||||
|
||||
In scope (call the new core):
|
||||
- `core/imports/pipeline.py:340-373` — the only caller of
|
||||
`AcoustIDVerification.verify_audio_file` (import path).
|
||||
- `core/repair_jobs/acoustid_scanner.py` — the library-scan job.
|
||||
|
||||
Shared helpers (stay shared, unchanged; the core builds on them):
|
||||
- `core/matching/artist_aliases.py::artist_names_match` / `best_alias_match`
|
||||
- `core/matching/script_compat.py::is_cross_script_mismatch`
|
||||
- `core/matching/acoustid_candidates.py::find_matching_recording`,
|
||||
`duration_mismatches_strongly`
|
||||
- `core/matching/version_mismatch.py::is_acceptable_version_mismatch`
|
||||
|
||||
Status hook:
|
||||
- `core/imports/version_mismatch_fallback.py` sets
|
||||
`context["_version_mismatch_fallback"]` when it force-accepts the best
|
||||
quarantined candidate after retries are exhausted → maps to `force_imported`.
|
||||
Dispatched from `core/downloads/task_worker.py` via `try_version_mismatch_fallback`.
|
||||
|
||||
Deliberately OUT of scope (consume a shared helper but serve other purposes;
|
||||
NOT merged):
|
||||
- `core/repair_worker.py::_album_fill_artist_names_match` — Album-Completeness
|
||||
auto-fill gate.
|
||||
- `core/matching/album_context_title.py::_normalize` — trivial album-grouping
|
||||
normalize.
|
||||
- `core/downloads/task_worker.py`, `core/matching_engine.py` — pre-download
|
||||
(Soulseek candidate) matching, not post-download verification.
|
||||
|
||||
## Design
|
||||
|
||||
### 1. Shared core — `core/matching/audio_verification.py` (new)
|
||||
|
||||
The single home for the verification *decision*. Pure where possible (no file
|
||||
I/O, no DB) so it is unit-testable in isolation. Callers keep their own I/O
|
||||
(fingerprinting, quarantine, finding creation, tag writing).
|
||||
|
||||
```
|
||||
def normalize(text: str) -> str
|
||||
# lowercase; strip ()/[]/<> annotations; strip version (- Live, etc) +
|
||||
# featuring tags; KEEP CJK (\w under unicode); collapse whitespace.
|
||||
# Single source of truth — replaces acoustid_verification._normalize AND
|
||||
# acoustid_scanner._normalize.
|
||||
|
||||
def evaluate(
|
||||
expected_title, expected_artist, recordings, *,
|
||||
fingerprint_score, file_duration_s=None, aliases_provider=None,
|
||||
) -> Outcome
|
||||
# Outcome(decision, title_sim, artist_sim, matched_title, matched_artist, reason)
|
||||
# decision in {PASS, SKIP, FAIL}.
|
||||
# Encapsulates: alias-aware artist sim, version-mismatch gate, duration
|
||||
# collision guard, cross-script SKIP, thresholds. Built on the shared helpers.
|
||||
```
|
||||
|
||||
Thresholds (`TITLE_MATCH_THRESHOLD`, `ARTIST_MATCH_THRESHOLD`, `MIN_ACOUSTID_SCORE`)
|
||||
move into the core as the single definition.
|
||||
|
||||
Caller mapping:
|
||||
- **Import** (`acoustid_verification.verify_audio_file`): PASS → import +
|
||||
status `verified`; SKIP → import + status `unverified`; FAIL → quarantine.
|
||||
- **Scan** (`acoustid_scanner._scan_file`): PASS/SKIP → no finding (optionally
|
||||
refresh the stored status); FAIL → create `acoustid_mismatch` finding.
|
||||
|
||||
`acoustid_verification.py` keeps `verify_audio_file` as the import-facing wrapper
|
||||
(fingerprint lookup, MB enrichment, alias provider, returns `VerificationResult`)
|
||||
but delegates the decision to `evaluate`. `acoustid_scanner.py` drops its private
|
||||
`_normalize` and decision branches and calls `normalize` + `evaluate`.
|
||||
|
||||
### 2. Verification status
|
||||
|
||||
Values (the three the user selected; quarantined files are not imported, so they
|
||||
carry no status):
|
||||
- `verified` — clean AcoustID PASS.
|
||||
- `unverified` — SKIP: cross-script / ambiguous / no AcoustID match. Imported,
|
||||
not hard-confirmed.
|
||||
- `force_imported` — accepted via `version_mismatch_fallback` after retries
|
||||
exhausted.
|
||||
|
||||
### 3. Storage — DB + tag
|
||||
|
||||
- **DB:** new nullable column `tracks.verification_status TEXT` (idempotent
|
||||
migration in `database/music_database.py`). Set at import; refreshed by the
|
||||
scan. Mirrored onto the download record/context so the Downloads page can show
|
||||
it before a library re-scan.
|
||||
- **Tag:** `SOULSYNC_VERIFICATION=<status>` written via `core/tag_writer.py`
|
||||
(Vorbis comment / ID3 `TXXX`). Travels with the file; the scanner reads it and
|
||||
may skip re-verifying an already-`verified` file (perf bonus) and to display.
|
||||
|
||||
### 4. Downloads page UI
|
||||
|
||||
A small, info-only badge per row sourced from the status:
|
||||
- ✓ `verified` · ⚠ `unverified` · ⚑ `force_imported`.
|
||||
Wired in `webui/static/downloads.js` (+ the download-status API payload).
|
||||
|
||||
### 5. Testing (TDD)
|
||||
|
||||
- Unit-test the core `normalize` (incl. `<>`/`()`/`[]`, CJK retention) and
|
||||
`evaluate` (Sawano/IPA cross-script → SKIP not FAIL; vocal-credit artist →
|
||||
alias 100%; version mismatch → FAIL; duration collision → no FAIL).
|
||||
- Wire import + scanner to the core; their existing tests
|
||||
(`test_acoustid_skip_logic`, `test_acoustid_scanner`,
|
||||
`test_acoustid_version_mismatch`, `test_acoustid_error_reporting`,
|
||||
`test_acoustid_normalize_angle_annotations`) act as regression.
|
||||
- New tests for the status: force-import → `force_imported`; PASS → `verified`;
|
||||
SKIP → `unverified`; tag written + read back; DB column populated.
|
||||
|
||||
## Rollout / risk
|
||||
|
||||
- Pure-core extraction is behaviour-preserving for the import path (existing
|
||||
tests pin it); the scan path *changes* (gains alias bridge + cross-script SKIP)
|
||||
— that is the intended fix.
|
||||
- DB migration is additive (nullable column), safe on existing DBs.
|
||||
- Tag writing reuses the existing `tag_writer` path; failures are non-fatal.
|
||||
- Staged on the current branch with TDD; ships in the next image build.
|
||||
|
||||
## Out of scope (now)
|
||||
|
||||
Unifying the pre-download Soulseek matcher or the album-completeness gate. They
|
||||
consume shared helpers but are different decisions; folding them in would widen
|
||||
blast radius without serving this goal.
|
||||
Loading…
Reference in a new issue