2604704a27
AcoustID returns a recording's title/artist in their ORIGINAL script (e.g. "久石譲" for Joe Hisaishi) while SoulSync's expected metadata is romanized/English. A correct download then fails verification on two walls: the title can never clear the 0.70 similarity bar cross-script, and the only skip path that ignores the title required a near-perfect 0.95 fingerprint plus a resolved alias. Result: every non-English artist trips it. Two complementary fixes, per the reporter's two ideas. Graceful fix (automatic): - New pure core/matching/script_compat.py detects when two strings are in genuinely different writing systems (CJK/Hangul/Cyrillic/Greek/ Arabic/Hebrew/Thai vs Latin). Accented Latin (Beyoncé, Sigur Rós) stays Latin — no false trigger. - acoustid_verification.py: when the EXPECTED artist and the matched artist span scripts AND the artist is confirmed via the existing MusicBrainz alias bridge, SKIP instead of quarantine, without the 0.95 floor (the 0.80 trust floor already gates the fingerprint). - Deliberately narrow: keyed on the ARTIST spanning scripts + being confirmed. A same-script artist with only a cross-script title keeps the stricter 0.95 floor, so the #607 wrong-file protection (Kendrick R.O.T.C, low-fingerprint Japanese-title) is untouched. Per-request toggle (manual escape hatch): - New "Skip AcoustID verification" checkbox in the download-missing modal beside "Force Download All". - skip_acoustid threads request -> batch -> per-track track_info -> download context (same path as _playlist_folder_mode), landing on the existing _skip_quarantine_check='acoustid' bypass. No new mechanism; only the AcoustID gate is bypassed (integrity/bit-depth still run). Tests: - tests/matching/test_script_compat.py — script-boundary cases. - test_acoustid_skip_logic.py — Joe Hisaishi SKIPs at 0.85; unconfirmed cross-script artist still FAILs; same-script low-fingerprint still FAILs. - test_downloads_candidates.py — toggle injects the bypass; absent toggle keeps verification. Full suite: 5169 passed; only pre-existing soundcloud /app env failures remain. Zero regressions.
79 lines
3.1 KiB
Python
79 lines
3.1 KiB
Python
"""Tests for core/matching/script_compat.py — writing-system detection.
|
|
|
|
Issue #797 — these pin the exact boundary that the AcoustID verifier
|
|
relies on: accented Latin is still Latin (no false cross-script
|
|
trigger), but genuinely different writing systems (CJK / Hangul /
|
|
Cyrillic / Greek / Arabic / Hebrew / Thai) ARE flagged so a
|
|
romanized-vs-native artist comparison isn't treated as a real mismatch.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import pytest
|
|
|
|
from core.matching.script_compat import (
|
|
has_strong_nonlatin,
|
|
is_cross_script_mismatch,
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# has_strong_nonlatin — accented Latin must NOT count
|
|
# ---------------------------------------------------------------------------
|
|
|
|
@pytest.mark.parametrize('text', [
|
|
'Beyoncé', 'Sigur Rós', 'Mötley Crüe', 'Joe Hisaishi',
|
|
'Kendrick Lamar', 'Dmitry Yablonsky', 'AC/DC', 'P!nk',
|
|
'', ' ', '12345', 'Café del Mar',
|
|
])
|
|
def test_latin_and_accented_latin_is_not_nonlatin(text):
|
|
assert has_strong_nonlatin(text) is False
|
|
|
|
|
|
@pytest.mark.parametrize('text', [
|
|
'久石譲', # kanji (Joe Hisaishi)
|
|
'残酷な天使のテーゼ', # kana + kanji
|
|
'Дмитрий Яблонский', # Cyrillic
|
|
'방탄소년단', # Hangul (BTS)
|
|
'Σωκράτης', # Greek
|
|
'عمرو دياب', # Arabic
|
|
'שלום', # Hebrew
|
|
'ก้อง สหรัถ', # Thai
|
|
])
|
|
def test_real_nonlatin_scripts_detected(text):
|
|
assert has_strong_nonlatin(text) is True
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# is_cross_script_mismatch — the verifier's gate
|
|
# ---------------------------------------------------------------------------
|
|
|
|
def test_romanized_vs_native_is_cross_script():
|
|
# The reported case (#797): expected romanized, AcoustID native.
|
|
assert is_cross_script_mismatch('Joe Hisaishi', '久石譲') is True
|
|
assert is_cross_script_mismatch('Dmitry Yablonsky', 'Дмитрий Яблонский') is True
|
|
|
|
|
|
def test_is_symmetric():
|
|
assert is_cross_script_mismatch('久石譲', 'Joe Hisaishi') is True
|
|
|
|
|
|
def test_same_latin_both_sides_is_not_mismatch():
|
|
# English-vs-English — comparison is meaningful, no bridge. This is
|
|
# the Kendrick R.O.T.C protection surface: must stay False so the
|
|
# verifier keeps its strict FAIL path.
|
|
assert is_cross_script_mismatch('Kendrick Lamar', 'Kendrick Lamar feat. BJ') is False
|
|
assert is_cross_script_mismatch('Crown', 'Crown of Thorns') is False
|
|
|
|
|
|
def test_same_nonlatin_both_sides_is_not_mismatch():
|
|
# Both native — same-script similarity still works, don't relax.
|
|
assert is_cross_script_mismatch('久石譲', '久石譲') is False
|
|
assert is_cross_script_mismatch('久石譲', '坂本龍一') is False
|
|
|
|
|
|
def test_empty_or_letterless_other_side_is_not_mismatch():
|
|
# One side non-Latin but the other has no Latin LETTER to bridge to.
|
|
assert is_cross_script_mismatch('', '久石譲') is False
|
|
assert is_cross_script_mismatch('12345', '久石譲') is False
|
|
assert is_cross_script_mismatch('久石譲', ' ') is False
|