174513d351
Tracks NOT in the library were matched to a DIFFERENT song by the SAME artist
and reported with high confidence instead of as missing — e.g. "Dani
California" -> "Californication" (Red Hot Chili Peppers), "Under The Bridge"
-> "Around the World".
Root cause: _calculate_track_confidence scores 0.5*title + 0.5*artist. A
same-artist comparison always yields artist = 1.0, so the title score is the
only thing that can tell two of an artist's songs apart — but that score is a
SequenceMatcher CHARACTER ratio, which over-credits unrelated titles that
share a long substring ("californi…" = 0.67) or just a stopword ("the" =
0.62). With the flat 0.5 artist term, anything clearing the weak 0.6 char
floor lands at ~0.81-0.83, well over the 0.7 sync threshold. Reproduced on
dev: both reported pairs score 0.81/0.83.
Fix: new core/text/title_match.py:titles_plausibly_same, called in
_calculate_track_confidence right before the floor. It accepts a pair only
when it's near-identical char-wise (>=0.85, so typos / punctuation / casing
like "Beleive"->"Believe", "HUMBLE."->"Humble" still match) OR the titles
share at least one significant (non-stopword) word. Two different songs by the
same artist share no content word, so they're rejected and the real track is
correctly reported missing. ("the" is a stopword — that's what leaked "Under
The Bridge"/"Around the World".)
Scoped deliberately: the word-overlap test fires ONLY when at least one side
has 2+ content words. For single-word titles there is no other word to share,
so it defers to the existing char floor — otherwise legitimate stylized
spellings ("Grey"/"Gray", "Tonite"/"Tonight", "4ever"/"Forever") would become
new false-negatives. Verified those still match. The few single-word variants
that do score low (Ok/Okay, Thru/Through) were already rejected by the
pre-existing length-ratio penalty, not by this gate.
Both reported false positives now score 0.33/0.31 -> missing. Does NOT address
the harder case of two different same-artist songs that DO share a content
word (e.g. "Believe"/"Believer") — pre-existing and unworsened. Any residual
error fails safe: a false-missing is re-downloaded/wishlisted, vs the old
behavior which silently substituted the wrong song.
Tests: tests/test_title_match_guard.py (14) — pure-guard unit tests + a
13-pair battery driving the REAL _calculate_track_confidence (genuine matches
stay >=0.7, same-artist different songs drop below), plus an explicit
no-regression test for stylized single-word spellings. 292 matching/sync tests
pass.
81 lines
3.8 KiB
Python
81 lines
3.8 KiB
Python
"""Guard against char-level title false positives in track matching.
|
|
|
|
Issue #769: playlist sync matched tracks that aren't in the library to a
|
|
DIFFERENT song by the SAME artist, with high confidence — e.g. "Dani
|
|
California" -> "Californication" (Red Hot Chili Peppers), "Under The Bridge"
|
|
-> "Around the World". The confidence formula is ``0.5*title + 0.5*artist``,
|
|
and a same-artist comparison always yields ``artist = 1.0``, so the title score
|
|
is the only thing that can tell two of an artist's songs apart. But the title
|
|
score is a ``difflib.SequenceMatcher`` character ratio, which over-credits
|
|
unrelated titles that happen to share a long substring ("californi…") or only a
|
|
stopword ("the"): 0.67 and 0.62 respectively. With the flat 0.5 artist term
|
|
that lands at 0.83 / 0.81 — well over the 0.7 sync threshold.
|
|
|
|
``titles_plausibly_same`` adds a cheap word-level sanity check on top of the
|
|
char ratio: accept a pair only when it's near-identical char-wise (so typos and
|
|
punctuation/casing variants — "Beleive"/"Believe", "HUMBLE."/"Humble" — still
|
|
match) OR the two titles share at least one significant (non-stopword) token.
|
|
Two genuinely different songs by the same artist share no content word, so they
|
|
get rejected; the real track is then correctly reported missing.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import re
|
|
|
|
# Articles / prepositions / conjunctions only. Deliberately NOT pronouns
|
|
# ("you", "me", "i") — those carry meaning in song titles and dropping them
|
|
# could strip the only shared word from a real match. "the" MUST stay here:
|
|
# without it "Under The Bridge" and "Around the World" would falsely share it.
|
|
_TITLE_STOPWORDS = frozenset({
|
|
"the", "a", "an", "of", "and", "or", "to", "in", "on",
|
|
"for", "with", "at", "by", "from",
|
|
})
|
|
|
|
_TOKEN_RE = re.compile(r"[a-z0-9]+")
|
|
|
|
# Char ratio at/above which two titles are treated as the same regardless of
|
|
# shared words — covers typos, punctuation, casing, accents. Tuned so single-
|
|
# word typos ("Beleive"/"Believe" = 0.857) pass while the #769 false positives
|
|
# ("Dani California"/"Californication" = 0.667) do not.
|
|
_NEAR_IDENTICAL = 0.85
|
|
|
|
|
|
def _content_tokens(text: str) -> set[str]:
|
|
return {t for t in _TOKEN_RE.findall((text or "").lower()) if t not in _TITLE_STOPWORDS}
|
|
|
|
|
|
def titles_plausibly_same(
|
|
title_a: str,
|
|
title_b: str,
|
|
char_similarity: float,
|
|
*,
|
|
near_identical: float = _NEAR_IDENTICAL,
|
|
) -> bool:
|
|
"""Whether two titles could be the same track, given their char similarity.
|
|
|
|
``title_a`` / ``title_b`` should already be normalised/cleaned (lowercased,
|
|
brackets stripped) the same way the caller computed ``char_similarity``.
|
|
|
|
Returns ``True`` when the pair is near-identical char-wise OR shares at
|
|
least one significant (non-stopword) token. Returns ``False`` for two
|
|
titles that are only moderately char-similar and share no content word —
|
|
i.e. different songs the char ratio over-credited (#769)."""
|
|
if char_similarity >= near_identical:
|
|
return True
|
|
ta = _content_tokens(title_a)
|
|
tb = _content_tokens(title_b)
|
|
# Word-overlap is only a reliable "different song" signal when at least one
|
|
# side has 2+ content words — that's the #769 case where the char ratio
|
|
# over-credits a shared substring ("Dani California"/"Californication") or
|
|
# a stopword ("Under The Bridge"/"Around the World"). For single-word
|
|
# titles there's no other word to share, so applying it would wrongly fail
|
|
# legitimate stylized spellings ("Grey"/"Gray", "Tonite"/"Tonight",
|
|
# "Thru"/"Through") that the char ratio rightly accepts. In that case defer
|
|
# to the caller's existing char-similarity floor instead of force-failing.
|
|
if max(len(ta), len(tb)) < 2 or not ta or not tb:
|
|
return True
|
|
return not ta.isdisjoint(tb)
|
|
|
|
|
|
__all__ = ["titles_plausibly_same"]
|