Auto-import: live card updates + multi-disc + featured-artist tag fixes

The 'Live Per-Track Progress' work shipped a backend in-progress row + top-of-tab
progress text but the history cards themselves stayed visually stale during
processing — lowercase "processing" badge, neutral styling, no per-track hint.
Smoke-testing also surfaced two latent identification bugs that prevented
multi-disc rips with features (Kendrick GKMC Deluxe) from importing at all.

Card-level live progress (`webui/static/stats-automations.js`):
- Cache `/api/auto-import/status` response in `_autoImportLastStatus`; poller
  awaits status before re-rendering results so the card has the live data.
- Add 'processing' entries to statusLabels / statusIcons / statusClass.
- When card folder_name matches `current_folder`, swap the meta line to
  `track N/M: <track name>` and tag the matching row in the expanded list
  as `auto-import-track-row-active`; prior rows tag as `-row-done`.

Card styling (`webui/static/style.css`):
- `.auto-import-processing` blue left border, `.auto-import-badge-processing`
  pulse animation, active/done track-row classes.

Multi-disc enumeration (`core/auto_import_worker.py:_scan_directory`):
- Old code skipped disc folders during recursion AND only attached them to a
  parent that had its own loose audio. A folder containing only `Disc 1/`,
  `Disc 2/` was invisible. Now: when a directory has only disc subdirs and no
  loose audio, treat that directory itself as the album candidate. Disc folders
  still skipped when standing alone.
- Add `FolderCandidate.is_staging_root` flag (set when the staging dir itself
  becomes the candidate via this path) so identification can refuse to use the
  meaningless folder name.

Tag identification (`core/auto_import_worker.py:_identify_from_tags`):
- Per-track `artist` tag fragmented consensus on albums with features
  ("Kendrick Lamar" / "Kendrick Lamar, Drake" / "Kendrick Lamar, Dr. Dre"
  produced 3 separate `(album, artist)` keys for one album). Now group by
  album first, then pick the most-common artist within that album group.
- `_read_file_tags` now prefers `albumartist` over `artist` for album-level
  identity; falls back to `artist` for files without albumartist.
- Add INFO-level log when tag identification rejects, showing top albums and
  their counts so the user can diagnose multi-disc / tagging issues.

Folder-name false-match guard (`core/auto_import_worker.py:_identify_folder`):
- When `is_staging_root` is set, skip the folder-name strategy entirely. Logs
  the skip and falls through to AcoustID. Without this, dropping disc folders
  directly into staging caused the scanner to search the metadata source for
  the literal name "Staging", which false-matched against random albums (e.g.
  "Stamina, Dinos" — a French rap album — at 13% confidence).

What's New entries added under 2.4.2 dev cycle.
This commit is contained in:
Broque Thomas 2026-05-02 23:15:52 -07:00
parent 783c543c3e
commit cdd408b6f3
4 changed files with 143 additions and 32 deletions

View file

@ -36,6 +36,11 @@ class FolderCandidate:
disc_structure: Dict[int, List[str]] = field(default_factory=dict) # disc_num -> files
folder_hash: str = ''
is_single: bool = False # True for loose files in staging root
# True when the candidate "folder" is the staging root itself (user dropped
# disc folders directly into staging without an album wrapper). The name is
# meaningless ("Staging", "Music", etc.) — folder-name identification must
# be skipped or it will false-match against random albums.
is_staging_root: bool = False
def _compute_folder_hash(audio_files: List[str]) -> str:
@ -58,7 +63,11 @@ def _read_file_tags(file_path: str) -> Dict[str, Any]:
if audio and audio.tags:
tags = audio.tags
result['title'] = (tags.get('title', [''])[0] or '').strip()
result['artist'] = (tags.get('artist', [''])[0] or tags.get('albumartist', [''])[0] or '').strip()
# Prefer albumartist for album-level identification (per-track artist
# often includes features like "Kendrick Lamar, Drake" which fragment
# consensus when grouping tracks into an album). Fall back to artist
# for files that lack albumartist.
result['artist'] = (tags.get('albumartist', [''])[0] or tags.get('artist', [''])[0] or '').strip()
result['album'] = (tags.get('album', [''])[0] or '').strip()
# Date/year — try 'date' first, fall back to 'year'
date_str = (tags.get('date', [''])[0] or tags.get('year', [''])[0] or '').strip()
@ -385,10 +394,10 @@ class AutoImportWorker:
def _enumerate_folders(self, staging: str) -> List[FolderCandidate]:
"""Find album folder and single file candidates in staging directory (recursive)."""
candidates = []
self._scan_directory(staging, candidates)
self._scan_directory(staging, candidates, staging_root=staging)
return candidates
def _scan_directory(self, directory: str, candidates: List[FolderCandidate]):
def _scan_directory(self, directory: str, candidates: List[FolderCandidate], staging_root: str = ''):
"""Recursively scan a directory for album folders and loose audio files."""
try:
entries = sorted(os.listdir(directory))
@ -444,12 +453,45 @@ class AutoImportWorker:
disc_structure=disc_structure, folder_hash=folder_hash
))
else:
# No audio files here — recurse into subdirectories
for sub_name, sub_path in subdirs:
# Skip disc folders at this level (they'll be handled by the parent album)
if DISC_FOLDER_RE.match(sub_name):
continue
self._scan_directory(sub_path, candidates)
# No loose audio files. If the only subdirs are disc folders,
# treat THIS directory as the album candidate (multi-disc album
# with no album-level loose files — common when a user drops
# `Album/Disc 1/`, `Album/Disc 2/` straight into staging, or
# drops `Disc 1/`, `Disc 2/` with the staging dir itself as
# the album root).
disc_subdirs = [(n, p) for n, p in subdirs if DISC_FOLDER_RE.match(n)]
non_disc_subdirs = [(n, p) for n, p in subdirs if not DISC_FOLDER_RE.match(n)]
if disc_subdirs and not non_disc_subdirs:
disc_structure = {}
audio_files = []
for sub_name, sub_path in disc_subdirs:
disc_num = int(DISC_FOLDER_RE.match(sub_name).group(1))
try:
disc_files = [os.path.join(sub_path, f) for f in sorted(os.listdir(sub_path))
if os.path.isfile(os.path.join(sub_path, f))
and os.path.splitext(f)[1].lower() in AUDIO_EXTENSIONS]
except OSError:
disc_files = []
if disc_files:
disc_structure[disc_num] = disc_files
audio_files.extend(disc_files)
if audio_files:
folder_name = os.path.basename(directory)
folder_hash = _compute_folder_hash(audio_files)
is_staging_root = bool(staging_root) and os.path.normpath(directory) == os.path.normpath(staging_root)
candidates.append(FolderCandidate(
path=directory, name=folder_name, audio_files=audio_files,
disc_structure=disc_structure, folder_hash=folder_hash,
is_staging_root=is_staging_root,
))
return
# Otherwise recurse into non-disc subdirs (disc folders only
# ever attach to a parent album, never stand alone).
for sub_name, sub_path in non_disc_subdirs:
self._scan_directory(sub_path, candidates, staging_root=staging_root)
def _is_folder_stable(self, candidate: FolderCandidate) -> bool:
"""Check if folder contents have stopped changing."""
@ -491,10 +533,15 @@ class AutoImportWorker:
if tag_result:
return tag_result
# Strategy 2: Parse folder name
folder_result = self._identify_from_folder_name(candidate)
if folder_result:
return folder_result
# Strategy 2: Parse folder name (skip when the candidate is the staging
# root itself — the folder name is meaningless and will false-match
# against random albums in the metadata source).
if candidate.is_staging_root:
logger.info(f"[Auto-Import] Skipping folder-name identification for staging root '{candidate.name}' — would false-match. Falling through to AcoustID.")
else:
folder_result = self._identify_from_folder_name(candidate)
if folder_result:
return folder_result
# Strategy 3: AcoustID fingerprint
acoustid_result = self._identify_from_acoustid(candidate)
@ -684,29 +731,47 @@ class AutoImportWorker:
def _identify_from_tags(self, candidate: FolderCandidate) -> Optional[Dict]:
"""Try to identify album from embedded file tags."""
tags_list = []
for f in candidate.audio_files[:20]: # Cap at 20 files
sampled = candidate.audio_files[:20] # Cap at 20 files
for f in sampled:
tags = _read_file_tags(f)
if tags['album'] and tags['artist']:
tags_list.append(tags)
if len(tags_list) < max(1, len(candidate.audio_files) * 0.5):
if len(tags_list) < max(1, len(sampled) * 0.5):
logger.info(f"[Auto-Import] Tag identification rejected for '{candidate.name}' — only {len(tags_list)}/{len(sampled)} files have album+artist tags (need >=50%)")
return None # Less than 50% of files have usable tags
# Check consistency — most common album+artist
album_artist_counts = {}
# Group by album first (album-level identity). Per-track artist often
# varies due to features ("Artist", "Artist, Drake", etc.) so grouping
# by (album, artist) fragments consensus on a real album. Pick the
# dominant album, then within that album pick the most-common artist
# (which will usually be the album's primary artist).
album_counts = {}
for t in tags_list:
key = (t['album'].lower().strip(), t['artist'].lower().strip())
album_artist_counts[key] = album_artist_counts.get(key, 0) + 1
album_key = t['album'].lower().strip()
album_counts[album_key] = album_counts.get(album_key, 0) + 1
if not album_artist_counts:
if not album_counts:
return None
best_key, best_count = max(album_artist_counts.items(), key=lambda x: x[1])
if best_count < len(tags_list) * 0.6:
return None # Tags too inconsistent
best_album, best_album_count = max(album_counts.items(), key=lambda x: x[1])
if best_album_count < len(tags_list) * 0.6:
sample = ', '.join([f"'{a}' x{c}" for a, c in sorted(album_counts.items(), key=lambda x: -x[1])[:3]])
logger.info(f"[Auto-Import] Tag identification rejected for '{candidate.name}' — best album '{best_album}' only {best_album_count}/{len(tags_list)} files (need >=60%). Top albums: {sample}")
return None
album_name, artist_name = best_key
return self._search_metadata_source(artist_name, album_name, 'tags', candidate)
# Most-common artist among files matching the dominant album
artist_counts = {}
for t in tags_list:
if t['album'].lower().strip() == best_album:
a = t['artist'].lower().strip()
if a:
artist_counts[a] = artist_counts.get(a, 0) + 1
if not artist_counts:
return None
artist_name, _ = max(artist_counts.items(), key=lambda x: x[1])
return self._search_metadata_source(artist_name, best_album, 'tags', candidate)
def _identify_from_folder_name(self, candidate: FolderCandidate) -> Optional[Dict]:
"""Try to identify album from folder name."""

View file

@ -3449,7 +3449,8 @@ const WHATS_NEW = {
{ title: 'Watchlist Stops Re-Downloading Tracks That Already Exist', desc: 'a track that was already on disk got re-downloaded by the watchlist on every scan because the library had stale album metadata for it (file tagged on the wrong album by an old import) and the album fuzzy comparison declared the track missing. now the watchlist also matches by stable external IDs (spotify / itunes / deezer / tidal / qobuz / musicbrainz / audiodb / hydrabase / isrc) before falling through to the fuzzy block — so any track whose tags or DB row carry a matching ID is recognized as already present regardless of album drift. provider-neutral, falls through to existing fuzzy logic for older imports without IDs.', page: 'watchlist' },
{ title: 'Persist Source IDs at Download Time + Backfill on Sync', desc: 'every download already collects spotify/itunes/deezer/tidal/qobuz/musicbrainz/audiodb/hydrabase/isrc IDs during post-processing, but for plex/jellyfin/navidrome users they got dropped on the floor — only enrichment workers eventually wrote them onto the tracks row, hours later. now those IDs persist to the track_downloads table immediately, the media-server sync code copies them onto the new tracks row the moment it gets created, and the watchlist scanner has a second-tier fallback to query provenance directly when the tracks row hasn\'t been synced yet. closes the enrichment-wait window — freshly downloaded files are recognizable on the very next watchlist scan instead of after enrichment catches up.', page: 'library' },
{ title: 'Fix Tidal Auth Error 1002 for Docker / Remote Access', desc: 'tidal returned error 1002 ("invalid redirect URI") on every authentication attempt for users accessing soulsync from a network IP. cause: when the redirect_uri config field was empty (which it usually was, because the UI just shows the default as a placeholder without saving it), the /auth/tidal route silently overrode the constructor default with a uri built from request.host — http://192.168.x.x:8889/tidal/callback. that didn\'t match what users had registered in their tidal developer portal (http://127.0.0.1:8889/tidal/callback per the docs and UI default), so tidal rejected the authorize request before users ever saw the consent screen. fix: drop the request-host fallback entirely. empty config now falls back to the constructor default that matches the documented portal registration. the existing post-auth swap-step instructions handle the docker/remote-access case as designed.', page: 'settings' },
{ title: 'Auto-Import: Live Per-Track Progress in History', desc: 'dropping an album into the staging folder used to leave the auto-import history blank for the entire processing window — sometimes 5+ minutes for a full album — because the database row only got written after every track was post-processed. now an in-progress row gets inserted up-front (status=processing) the moment processing starts, then updated to completed/failed when done. the status indicator and progress bar also show "processing speak now — track 3/14: mine" so you can see exactly where it is. one row per album, not per track, so the history list stays clean.', page: 'import' },
{ title: 'Auto-Import: Live Per-Track Progress in History', desc: 'dropping an album into the staging folder used to leave the auto-import history blank for the entire processing window — sometimes 5+ minutes for a full album — because the database row only got written after every track was post-processed. now an in-progress row gets inserted up-front (status=processing) the moment processing starts, then updated to completed/failed when done. the status indicator + progress bar show "processing speak now — track 3/14: mine", and the history card itself gets a pulsing "Processing" badge, swaps its meta line to "track 3/14: mine", and highlights the currently-processing row in the expanded track list (with prior tracks dimmed as done). one row per album, not per track, so the history list stays clean.', page: 'import' },
{ title: 'Auto-Import: Multi-Disc Albums + Featured-Artist Tag Handling', desc: 'two longstanding auto-import gaps that surfaced when a kendrick lamar deluxe rip got dropped into staging. (1) folders containing only `Disc 1/`, `Disc 2/` subfolders (no loose audio at the parent level) used to be invisible to the scanner — disc folders were only attached to a parent when the parent had its own loose tracks. now scanner treats a parent of disc-only subfolders as the album candidate. (2) tag identification grouped files by `(album, artist)` — but per-track artist often varies on albums with features ("kendrick lamar" vs "kendrick lamar, drake" vs "kendrick lamar, dr. dre"), which fragmented the consensus and rejected real albums. now groups by album first, picks the dominant artist within that album group; also prefers `albumartist` tag over per-track `artist` since the former is the album-level identity. as a defensive bonus, when the staging folder itself becomes the candidate (raw disc folders dropped at the root with no album wrapper), the folder-name fallback gets skipped — the name "Staging" was matching against random albums in the metadata source.', page: 'import' },
{ title: 'Sidebar Library Button Shows Artist Breadcrumb', desc: 'when you open an artist detail page (from library, search, or the global search popover), the sidebar Library button now lights up and rewrites its label to "Library / Artist Name" — long names truncate with an ellipsis and the full name shows on hover. revertes to plain "Library" when you leave. purely visual, no functionality change.', page: 'library' },
{ title: 'Enrichment Bubble Routes Consolidated', desc: 'internal — every dashboard enrichment bubble (musicbrainz, spotify, itunes, deezer, discogs, audiodb, lastfm, genius, tidal, qobuz) used to hit its own per-service status / pause / resume route in web_server.py. unified them under a single registry-driven endpoint set: /api/enrichment/<service>/<action>. spotify\'s rate-limit guard, lastfm/genius yield-override behavior, and tidal/qobuz extra status fields are encoded as data on the registry. 27 new tests cover the registry behavior.' },
{ title: 'Drop Old Per-Service Enrichment Routes', desc: 'internal — followup to the registry consolidation. now that the dashboard has cut over to /api/enrichment/<service>/<action>, deleted the 30 hand-rolled per-service routes from web_server.py (musicbrainz/audiodb/discogs/deezer/spotify/itunes/lastfm/genius/tidal/qobuz status+pause+resume). ~510 lines gone from the monolith, no behavior change.' },

View file

@ -626,12 +626,13 @@ function importPageSwitchTab(tab) {
// ── Auto-Import Tab ──
let _autoImportPollInterval = null;
let _autoImportFilter = 'all';
let _autoImportLastStatus = null;
function _autoImportStartPolling() {
_autoImportStopPolling();
_autoImportPollInterval = setInterval(() => {
_autoImportPollInterval = setInterval(async () => {
if (importPageState.activeTab === 'auto') {
_autoImportLoadStatus();
await _autoImportLoadStatus();
_autoImportLoadResults();
}
}, 5000);
@ -672,6 +673,7 @@ async function _autoImportLoadStatus() {
const res = await fetch('/api/auto-import/status');
const data = await res.json();
if (!data.success) return;
_autoImportLastStatus = data;
const toggle = document.getElementById('auto-import-enabled');
const statusText = document.getElementById('auto-import-status-text');
@ -803,17 +805,30 @@ async function _autoImportLoadResults() {
'needs_identification': 'Unidentified', 'failed': 'Failed',
'scanning': 'Scanning...', 'matched': 'Matched',
'rejected': 'Dismissed', 'approved': 'Approved',
'processing': 'Processing',
};
const statusIcons = {
'completed': '\u2713', 'pending_review': '\u26A0',
'needs_identification': '\u2717', 'failed': '\u2717',
'scanning': '\u231B', 'matched': '\u2713',
'rejected': '\u2715', 'approved': '\u2713',
'processing': '\u29D7',
};
const statusLabel = statusLabels[r.status] || r.status;
const statusIcon = statusIcons[r.status] || '';
const statusClass = r.status === 'completed' ? 'completed' : r.status === 'pending_review' ? 'review' :
r.status === 'failed' || r.status === 'needs_identification' ? 'failed' : 'neutral';
r.status === 'failed' || r.status === 'needs_identification' ? 'failed' :
r.status === 'processing' ? 'processing' : 'neutral';
// Live per-track progress for the row currently being processed.
// Match by folder_name since the worker only tracks one folder at a time.
const liveStatus = _autoImportLastStatus;
const isLiveProcessing = r.status === 'processing'
&& liveStatus && liveStatus.current_status === 'processing'
&& liveStatus.current_folder === r.folder_name;
const liveTrackIdx = isLiveProcessing ? (liveStatus.current_track_index || 0) : 0;
const liveTrackTotal = isLiveProcessing ? (liveStatus.current_track_total || 0) : 0;
const liveTrackName = isLiveProcessing ? (liveStatus.current_track_name || '') : '';
// Parse match data for track details
let matchCount = 0, totalTracks = 0, trackDetails = [];
@ -832,7 +847,10 @@ async function _autoImportLoadResults() {
} catch (e) {}
}
const matchSummary = totalTracks > 0 ? `${matchCount}/${totalTracks} tracks` : `${r.total_files} files`;
let matchSummary = totalTracks > 0 ? `${matchCount}/${totalTracks} tracks` : `${r.total_files} files`;
if (isLiveProcessing && liveTrackTotal > 0) {
matchSummary = `track ${liveTrackIdx}/${liveTrackTotal}: ${liveTrackName}`;
}
const methodLabels = { tags: 'Tags', folder_name: 'Folder Name', acoustid: 'AcoustID', filename: 'Filename' };
const methodLabel = methodLabels[r.identification_method] || r.identification_method || '';
@ -864,9 +882,15 @@ async function _autoImportLoadResults() {
<div class="auto-import-track-list-header">
<span>Track</span><span>Matched File</span><span>Conf</span>
</div>
${trackDetails.map(t => {
${trackDetails.map((t, tIdx) => {
const tConfClass = t.confidence >= 90 ? 'high' : t.confidence >= 70 ? 'medium' : 'low';
return `<div class="auto-import-track-row">
// 1-based liveTrackIdx — current row glows, prior rows dim as "done".
let rowState = '';
if (isLiveProcessing && liveTrackIdx > 0) {
if (tIdx + 1 === liveTrackIdx) rowState = ' auto-import-track-row-active';
else if (tIdx + 1 < liveTrackIdx) rowState = ' auto-import-track-row-done';
}
return `<div class="auto-import-track-row${rowState}">
<span class="auto-import-track-name">${escapeHtml(t.name)}</span>
<span class="auto-import-track-file">${escapeHtml(t.file)}</span>
<span class="auto-import-track-conf auto-import-conf-${tConfClass}">${t.confidence}%</span>

View file

@ -59723,6 +59723,7 @@ body.reduce-effects *::after {
.auto-import-completed { border-left: 3px solid #4ade80; }
.auto-import-review { border-left: 3px solid #fbbf24; }
.auto-import-failed { border-left: 3px solid #f87171; }
.auto-import-processing { border-left: 3px solid #60a5fa; }
.auto-import-card-art {
width: 56px; height: 56px;
@ -59980,6 +59981,26 @@ body.reduce-effects *::after {
.auto-import-badge-review { background: rgba(251,191,36,0.1); color: #fbbf24; }
.auto-import-badge-failed { background: rgba(248,113,113,0.1); color: #f87171; }
.auto-import-badge-neutral { background: rgba(255,255,255,0.05); color: rgba(255,255,255,0.4); }
.auto-import-badge-processing {
background: rgba(96,165,250,0.12);
color: #60a5fa;
animation: auto-import-badge-pulse 1.6s ease-in-out infinite;
}
@keyframes auto-import-badge-pulse {
0%, 100% { opacity: 1; }
50% { opacity: 0.55; }
}
.auto-import-track-row-active {
background: rgba(96,165,250,0.08);
border-left: 2px solid #60a5fa;
padding-left: 6px;
margin-left: -8px;
border-radius: 3px;
}
.auto-import-track-row-active .auto-import-track-name { color: #60a5fa; font-weight: 600; }
.auto-import-track-row-done .auto-import-track-name,
.auto-import-track-row-done .auto-import-track-file { opacity: 0.4; }
.auto-import-actions {
display: flex; gap: 4px; margin-top: 4px;