- depth setting (light = core tags + matched source ids; full = same
multi-source enrichment cascade a fresh download gets, run additively
via embed_source_ids). Threaded through scan/finding/auto-apply and the
repair_worker fix handler.
- source now defaults to 'auto' (= your source priority / active source)
instead of blank.
- give native <option> popups a solid dark background (were white-on-white).
- tests for full-depth full_meta payload + enrich invocation + light no-op.
The job was the odd one out — auto_fix=False, no dry_run setting, so it never
showed the 'Dry Run' badge the other jobs do (the badge keys off
settings.dry_run === true). Aligned it to the standard pattern:
- auto_fix=True + dry_run setting defaulting True. Default behavior is unchanged
(findings only, nothing written) AND it now shows the Dry Run badge.
- Turning dry_run off makes the scan auto-apply in place (result.auto_fixed),
no finding — the opt-in 'just retag it' mode.
- Extracted a shared apply_track_plans() used by both the scan auto-apply and
the repair_worker fix handler (handler now resolves Docker paths then
delegates — one code path, no duplication).
Tests: dry_run=False auto-applies + writes + no finding; existing dry-run
finding/skip/apply tests still green. 410 passing.
Two fixes:
- The retag-tool-card removal accidentally ate the </div></div> that closed the
Metadata & Cache grid + section, so the Management section nested inside it.
Restored the close — Management is a sibling section again. (div balance back
to 1998/1998.)
- Moved the Metadata Updater card from 'Database & Scanning' into 'Metadata &
Cache' where it belongs.
Closes the kettui gap — the orchestration was unproven. Injected-fake seam
tests (temp sqlite + real empty track files, no metadata APIs / no real tag
writes):
- embed_known_source_ids: builds the right canonical id_tags from flat db keys,
honors the musicbrainz embed gate, no-ops when there's nothing to write.
- library_retag scan: produces a detailed finding with the per-track old->new
diff + stamped source ids, and skips an album that's already correct.
- _add_source_ids: per-source key mapping.
- _fix_library_retag apply: writes each track's payload, and reports failure
when files are unreachable.
476 tests pass; ruff clean.
The old per-download Retag Tool was limited (only native-pipeline downloads,
100-group cap, manual per-group) and did the wrong thing — it moved/reorganized
files instead of just tagging. It's superseded by the new Library Re-tag job
(whole-library, in-place) + the enhanced-library 'Write Tags' button.
Removed: the post-download record_retag_download ingestion hook (stops writing
retag_groups on every download), core/library/retag.py, the web_server state +
deps + /api/retag/* endpoints + the tool:retag WebSocket emit, the dashboard
card + both modals (index.html), the core.js socket handler, and the tools-page
wiring + help entry (wishlist-tools.js). Updated the import-pipeline test.
Verified: web_server parses, app + core imports OK, 392 tests pass, no live
references to removed symbols.
Left as inert (harmless) for a careful follow-up sweep: the retag_groups/
retag_tracks tables + their DB CRUD methods (no longer written/read), and the
now-orphaned retag JS helper functions (no entry point/wiring/socket calls them;
interspersed with wishlist functions, so not blind-deleted).
write_tags_to_file wrote the core fields + cover but never the source IDs
(Spotify/iTunes/MusicBrainz) the import post-process embeds. Added a focused
source.embed_known_source_ids() that writes ALREADY-KNOWN ids (from db_data)
via the canonical, Picard-compatible frame writer the import uses
(_write_embedded_metadata) — no API re-fetch, frames correct by construction.
write_tags_to_file now calls it whenever db_data carries id keys.
Fed from both paths: the enhanced-library 'Write Tags' button now carries the
track's spotify/itunes/musicbrainz ids, and the Library Re-tag job stamps the
matched album/track source ids onto each track. So both now write the full tag
set, not a subset.
Wire library_retag into the repair findings UI: a 'Re-tag' type badge, an
'Apply Tags' fix button, and an expandable detail that shows, per track, every
tag that would change as old -> new (plus source/mode/cover-action summary and
any unmatched tracks). So the dry-run finding is actually reviewable before you
apply it — the rich details_json the job stores now surfaces in the card.
New 'Library Re-tag' repair job (default-OFF, opt-in; weekly when enabled):
- Scans every source-matched album (spotify/itunes/deezer/musicbrainz album id),
pulls fresh metadata + tracklist from that source, reads each local track's
current tags, and uses the planner to compute per-field diffs.
- Dry-run by design: scan only CREATES findings — nothing touches a file. Each
finding is highly detailed: per-track old->new for every changed field, the
source used, the mode, a cover-art action, and any unmatched tracks, plus a
summary description. Settings: mode (overwrite | fill_missing), cover_art
(replace | fill_missing | skip), source override.
- Apply handler (_fix_library_retag in repair_worker): writes each track's
planned tags in place via tag_writer.write_tags_to_file (+ batch-embeds cover,
refreshes cover.jpg). Only adds/overwrites planned fields — no moves/renames/
re-match. Resolves Docker paths; read-only/unreachable files counted, never
crash. Media-server-only / unreachable tracks are skipped.
Registered in the job list + fix dispatch. The old per-download Retag Tool is
left untouched alongside this for now.
The testable core for the new library-wide re-tag job. Given a source album's
metadata + tracklist and the library tracks' current file tags, it:
- matches source tracks to library tracks (disc+track number, then title sim),
- computes the per-field diff (old -> new) for the dry-run finding,
- builds the minimal write_tags_to_file payload — only fields that actually
change under the chosen mode (overwrite vs fill-missing), so applying never
touches unrelated/unchanged tags.
No IO/network/DB — 10 unit tests cover matching, both modes, blank-source
fields, and the album-artist/track-count payload mapping.
The version-button modal renders from VERSION_MODAL_SECTIONS (the curated
highlight reel), separate from the WHATS_NEW detailed log. Its top entries were
stale (2.5/2.6.0 era), so promote the 2.6.6 highlights to the top per the file's
release process: Artist Map v2, self-explaining recommendations, the cover-art
filler file-embedding, and a Recent Fixes & Performance roundup (qBittorrent
5.2.0, organize-by-playlist #780, nav/scroll perf #783, dashboard mobile).
- Bump _SOULSYNC_BASE_VERSION 2.6.5 -> 2.6.6 (the single source of truth that
propagates to the UI, backups, and the update check).
- Add the 2.6.6 What's New block (qBittorrent 5.2.0 login fix, Cover Art Filler
on-disk detection + file embedding + stricter matching, recommendations
explainability + Discover section, organize-by-playlist #780, nav/scroll perf
#783, dashboard mobile polish).
- Finalize the 2.6.5 block: it shipped in tag 2.6.5 but was left flagged
unreleased (so its notes never displayed) — stripped the flag + dated it per
the file's own release convention.
Verification found a non-additive edge: embed_album_art_metadata uses FLAC
add_picture(), which APPENDS — so applying to an album where some tracks already
had art would have added a duplicate embedded picture. The apply now checks each
file and skips any that already carry art (shared _audio_has_art helper), so it
only ever ADDS art to files missing it. Test covers the skip (no re-embed).
Previously the filler only flagged albums whose DB thumb_url was empty and, on
apply, only updated that DB thumb_url — so albums whose files had no embedded
art and no cover.jpg (but whose DB row had a URL) were never found, and even
'applying' art never touched the files. That's the reported 'doesn't scan all
albums' gap.
New core.metadata.art_apply (reuses the post-processing standard so the user's
album_art_order is honored):
- album_has_art_on_disk(): cheap-first check — folder cover.jpg/folder.jpg
sidecar, then embedded art in a representative track (FLAC/ID3/MP4/Vorbis).
- apply_art_to_album_files(): embeds via embed_album_art_metadata + writes
cover.jpg via download_cover_art; only ADDS art (never rewrites the user's
tags); read-only/unwritable files are skipped + counted, never crash.
Scan now examines every titled album and flags it when art is missing in the DB
OR on disk. Apply embeds into the album's audio files + writes cover.jpg in
addition to the DB thumbnail (media-server-only albums fall back to DB-only).
Tests cover sidecar/embedded detection, the cheap-first short-circuit, and the
apply orchestration (embeds each file + cover.jpg; read-only failures counted).
The title/artist fallback search took results[0]'s artwork unconditionally, so
a loose full-text match returned the wrong album's cover (the 'new sources give
incorrect art' reports). Now it pulls a few results and only accepts one whose
title matches (subset, to allow Deluxe/Remaster) AND whose artist matches
exactly — the artist being the strong guard against wrong covers. Falls back to
an exact title match when a result carries no artist.
The album's own stored source-id path is unchanged (that id is authoritative).
Tests: wrong-artist rejected, skips wrong result for a matching one, + unit
coverage of the matcher (deluxe/feat/stopwords accepted, wrong artist/title
rejected).
qBittorrent 5.2.0 changed /api/v2/auth/login to return HTTP 204 (No Content)
on success instead of HTTP 200 with body 'Ok.'. The adapter required the body
to equal 'Ok.', so every login on 5.2.0+ failed with 'HTTP 204 body=' — the
connection probe and all torrent actions were broken.
Treat login as successful on the SID auth cookie and/or a success body: 'Ok.'
(<=5.1) or an empty HTTP 204 (>=5.2.0). Still reject bad creds, which
qBittorrent reports as HTTP 200 + 'Fails.' (not a 4xx).
Tests: 204-empty -> success, SID-cookie+empty-body -> success, 'Fails.' (even
with a stale cookie) -> failure.
The #rate-monitor-section equalizer had breakpoints but two narrow-bar gaps:
- The status pill ("Not configured", "Yielding") is wider than a thin
equalizer column and spilled over neighbours — now capped to the bar width
with the label truncating via ellipsis.
- Wrapped rows were left-aligned (orphan bar stranded) with no vertical gap —
now centered with a row-gap so multi-row layouts read intentionally.
Plus smaller value/name fonts at <=480px so tiny bars stay legible.
PR #783 reordered transports to websocket-first for faster connects. Reverting
to the polling-first default: it's the most compatible behind reverse proxies
that don't forward WebSocket upgrade headers (common self-hosted setups), where
websocket-first silently breaks real-time updates. The connect-time gain isn't
worth the connectivity risk. Everything else from #783 (scroll-pause, content-
visibility, dashboard parallelization, settings fixes, reduce-effects) kept.
- Stale-cache check (playlistTrackCacheIsStale) compared raw track_count to the
filtered/cached track list, so any playlist with local or unavailable tracks
always looked 'stale' and refetched + re-mirrored on every modal open. Now it
compares the upstream snapshot_id (stored at cache time in the shared fetch
choke point), and returns not-stale when no snapshot is available — explicit
invalidation on refresh still handles real changes.
- organize_download: guard executor.submit so a refused job cleans up the batch
instead of stranding it in 'analysis' (holding a limited analysis slot).
- Removed the dead, deprecated, unused mirrorSpotifyPlaylistTracks.
resolve_mirrored_playlist tried the mirrored-playlists primary key FIRST for
any all-digit ref. Deezer upstream ids are all-numeric, so a Deezer playlist id
was mistaken for the PK and the organize-by-playlist toggle resolved a wrong row
(or nothing) — the toggle silently wouldn't save / 'Open in Mirrored' missed.
Resolve by (source, source_playlist_id) first, fall back to PK only when the
source lookup misses. Thread the batch/wishlist source through the download-path
callers so numeric upstream ids resolve correctly there too. Spotify (base62
ids) is unaffected.
Seam tests: numeric Deezer id resolves by source (not PK), spotify alphanumeric
by source, PK fallback still works, profile-scoped, empty refs -> None.
The error/health state was jarring: a red ring flickering at sin(time*12) plus
stress speeding up the heartbeat, which read as the whole glow flickering. Now
it's all gradual: stress no longer changes the heartbeat speed, the red tint is
softened (never full alarm-red) and eases in/out via a small accumulating
errorHeat bump + smooth decay, and the warning ring is a single soft ring that
breathes slowly (sin*1.4) at low alpha instead of strobing.
On mobile, worker-orbs is disabled so the enrichment buttons render as real
buttons. They were a ragged centered flex-wrap with the wide 'Manage Workers'
pill jammed inline. Now (<=768px, scoped to #dashboard-page so Settings etc.
are untouched): the 44px icon buttons spread evenly across the full width in an
auto-fit grid, and the Manage Workers pill gets its own full-width row.
The expanding heartbeat ring read as a heavy circular pulse. Now: the nucleus
barely breathes (size oscillation cut ~70%), the glow holds steady instead of
pulsing, the logo no longer visibly throbs, and the heartbeat ring is a single
very-faint halo that only appears when workers are actually busy. The red
error-warning ring is unchanged (still punchy, since it only fires on real
failures).
- New 'Recommended For You' carousel section on the Discover page (between the
hero and Your Artists), so recommendations aren't buried behind a hero modal
button. Reuses the recommended-card markup/CSS, the watchlist add handler, and
primes the modal cache so 'View All' opens instantly in sync.
- Re-frames the now-stale copy: recommendations are library-wide (the similar-
artists worker feeds the whole library), not watchlist-only.
- Shows the real explanation from the backend's 'because' field —
'Because you have X & Y' (with a full-list hover tooltip) instead of just a
count — in both the section cards, the modal cards, and the hero subtitle.
- Cards lazy-enrich their images via the same endpoint the modal uses.
Adds get_recommendation_sources() — for each recommended similar artist it
resolves the polymorphic similar_artists.source_artist_id back to the display
names of the user's OWN artists (library + watchlist) that list it, by matching
against every provider-id column on both tables. The /api/discover/similar-artists
endpoint now attaches a 'because' array per recommendation so the UI can show
'because you have X, Y, Z' instead of just a count.
Seam tests cover: library + watchlist resolution across different provider-id
columns, dedup + name-sort, max_per cap, orphan source omission, profile scoping.
The hub now reads as a health gauge on top of the activity gauge. A new
decaying errorHeat (0..1) is bumped by onStatus whenever a worker reports a
real error increment, and cools over ~6s. While stressed the nucleus blends
toward red, its heartbeat quickens (agitation), and a fast-flickering red
warning ring appears — so a glance distinguishes 'busy and healthy' from
'something's actually failing'. Since 404s are classified as not_found now,
this only lights up on genuine failures (timeouts, 5xx).
The worker's WARNING observability proved the '38 errors' were almost all
MusicMap returning 404 (artist has no map page) — a genuine not-found, not a
fetch failure. But iter_musicmap_similar_artist_events flattened every
RequestException to status_code 502, and the worker maps 400/404 -> not_found
/ everything-else -> error, so these inflated the error count.
Surface the real HTTP status from the exception's response (404 stays 404),
falling back to 502 only when there's no response (timeout/connection drop,
which is correctly still an error eligible for retry).
Regression tests: 404 -> 404 (not_found), timeout -> 502 (error), 500 stays
error, plus an end-to-end worker check that a 404 result marks 'not_found'
and stores nothing.
Status pushes land every ~2s, so the previous fixed 'drain 2/frame' fired a
whole window's worth of pulses in a fraction of a second then went quiet.
Now each orb sets a release rate when a status arrives (pending / ~2s, with
a floor so a lone event still shows within ~0.75s) and the loop drips pulses
out via a fractional accumulator — so a busy worker streams a steady line up
its spoke and a slow one sends the occasional single pulse.
The inbound pulses are now event-driven instead of a random trickle:
- core.js forwards every enrichment:<id> WebSocket status to a new
window.workerOrbs.onStatus hook (extra listener, UI handlers untouched).
- onStatus diffs the cumulative stats counters (matched/not_found/repaired/
synced/scanned, and errors) between pushes and queues one pulse per real
item processed (worker's brand colour) or error (red). First sample only
sets a baseline so we never dump the whole backlog at once.
- tick() drains a couple of queued pulses per frame so bursts stagger up
the spoke; cap of 8 queued per update prevents flooding on big jumps.
- Falls back to the old ambient trickle for any orb that hasn't received a
status yet, so nothing goes dead if the socket is quiet.
Bonus perf: an idle/slow worker now emits almost nothing instead of a
constant random stream of particles.
Navigation & sidebar feedback:
- Show legacy pages optimistically on pointerdown + CSS :active so the
sidebar reacts instantly instead of waiting for the click/router cycle.
- Defer heavy per-page init via requestIdleCallback so a page becomes
scrollable before its init work runs.
Scroll smoothness:
- Cache particle canvas dimensions (no forced reflow per navigation).
- Pause particle + worker-orb canvas redraws during active scroll so the
scroll gets the full frame budget.
- content-visibility:auto on discover shelves and search/wishlist/library
list items to skip off-screen layout.
Dashboard:
- Run the independent initial loads in parallel (Promise.all) instead of
six sequential awaits, collapsing the reflow cascade.
Settings:
- Wire input listeners once instead of rescanning the ~960-node subtree
on every visit.
- Suppress auto-save while the form is programmatically populated on load,
fixing a spurious full save (4 POSTs + backend service re-init) that
fired on every Settings visit.
Reduce Visual Effects = full performance mode:
- Also halts particles, worker orbs and all filters; hides the static
sidebar aura circles that looked broken without their blur/animation.
Global search bar hidden on settings/help/issues/import pages.
Performance:
- Bake one soft glow sprite per colour into an offscreen canvas and blit
it with drawImage instead of allocating a radial gradient every frame.
This was the hot path: sparks + inbound pulses + every orb glow each
built a gradient per frame (100+/frame at 60fps). Colours quantised to
8-step buckets to bound the cache (imperceptible tint shift, keeps the
rainbow path from minting a sprite every frame).
- Cache each orb's button element at init so the 30-frame active-state
check no longer re-runs querySelector.
- Net: the pulses/glows look identical, far fewer allocations per frame.
UI:
- Enrichment manager modal topbar icon now uses the SoulSync logo
(trans2.png) instead of the helix emoji, matching the dashboard button.
- Nucleus logo now fits to the pulsing radius using the image's natural
width/height, so it no longer stretches to a square.
- Manage Workers button swaps the helix emoji for the SoulSync logo
(trans2.png) inside the existing accent badge.
The Manage Workers hub now draws /static/trans2.png (the SoulSync mark)
at its center instead of a plain colored core, scaled to the pulsing
radius and brightening slightly with energy. Energy-reactive glow, rings
and inbound pulses still surround it. Falls back to the drawn core while
the image loads.
Three upgrades to the Manage Workers nucleus:
- Energy-reactive: hub size, glow, heartbeat speed and ring count all
scale with how many workers are actually running. Calm + dim when idle,
big/bright/fast with 1-3 radiating rings when busy. The animation now
reads as a live gauge.
- Inbound pulses: active workers fire colored particles along their spokes
into the hub, so it visibly collects their output (eased to accelerate
on arrival; cleared on collapse so they don't snap).
- Orbital rotation: worker orbs get a tangential nudge around the nucleus
so the cluster slowly revolves like an atom instead of drifting randomly
(active orbs orbit a touch faster).
The 'Manage Workers' orb now acts as a central nucleus instead of just
another drifting particle:
- Settles at canvas center with strong pull + heavy damping (no jitter)
- Drawn larger and brighter with a slow breathing pulse, white core
highlight, and an expanding heartbeat ring
- Wired to every worker orb with full-length spokes (a traveling pulse
runs along each), so it visually reads as the center managing the cluster
- Other orbs repel off it, leaving a clean halo around the nucleus
The screenshot said it all — the orbs collapse into the floating particle cluster
after 7s idle, but the Manage Workers pill just sat there static. It wasn't in
worker-orbs.js's WORKER_DEFS. Added .em-manage-btn (purple) so it collapses into
a floating orb with the others and reveals on header hover — now it behaves like
the rest of the cluster instead of an out-of-place static button.
The modal opened with a plain pop — out of place next to the worker orbs. Now it
springs up from the bottom (toward the Manage Workers button) with the SAME easing
the orbs reveal with, then the worker rail assembles one-by-one: each chip springs
in staggered (scale 0.4→1) with a brief pulse of its own brand colour. Mirrors the
orb motion language AND walks your eye across every worker + its live state dot /
coverage bar as they land — cool + informative. Respects prefers-reduced-motion.
Verified against live data: 1312/1313 stored similars carry a metadata source id,
but 1 slipped through name-only (a match on a source with no id column, e.g.
discogs). Enforce the standard: process_artist now SKIPS any similar whose match
doesn't map to a storable id column (spotify/itunes/deezer/musicbrainz) instead
of writing a useless name-only row. Regression test covers discogs-match + no-id
cases. Now 100% of newly-stored similars are actionable.
The kettui move: 38/79 fetches errored on the first live run, but they were
logged at DEBUG only — invisible in app.log, so the cause (rate-limit vs
no-providers vs bug) is unprovable. process_artist now returns a (status, count,
detail) triple carrying the error reason (status code + message / exception),
and the worker logs the first 15 errors per session at WARNING (rest DEBUG) +
keeps _last_error. No blind pacing tweak — let it run, read the real reason, then
fix the proven cause. Seam tests updated + assert the reason is captured.
THE root cause of 'orb frozen, click does nothing visibly': when a socket is
connected, the orbs don't poll — update*Status() bails on socketConnected and
relies on server pushes. similar_artists was missing from BOTH the server emit
loop (_emit_enrichment_status_loop's workers dict) and the client dispatch
(core.js socket.on('enrichment:<id>')), so the orb never received status → never
updated. Clicks DID pause the backend (modal showed paused), but the orb visual
was frozen. Added the worker to the emit loop + the socket.on handler.
Root cause of 'click does nothing': I flip-flopped between inline onclick and
addEventListener. A cached index.html with my inline onclick + fresh JS with
addEventListener = the click fires the toggle TWICE (pause then resume) = no net
change. Now identical to AudioDB/Deezer/etc.: NO inline onclick on the button,
single addEventListener('click', toggle) in the init. One handler, one fire.
The addEventListener wiring evidently wasn't firing the toggle (orb showed
running but clicking didn't pause). Switched the button back to an inline
onclick=toggleSimilarArtistsEnrichment() — identical to the Amazon orb, which
works — and exposed the fn on window so the inline handler always resolves.
Toggle logic unchanged (active ? pause : resume).
Stop diverging — match toggleAmazonEnrichment/toggleSpotifyEnrichment verbatim:
contains('active') ? pause : resume. A paused orb isn't 'active', so a click
resumes it (same as every other worker). My earlier 'paused'-class variant was
what broke unpausing.
Class-based toggle had a hole: the orb may lack the 'paused' class even when the
backend is paused (before the first 2s status poll, or worker fallback), so a
click would PAUSE the already-paused worker (no-op) → 'clicking doesn't unpause'.
Now the toggle reads /status first and does the opposite of the real paused
state, so a paused worker always resumes on click.
The orb was excluded from worker-orbs.js's WORKER_DEFS list, so it never got the
shared 'collapse to floating orb after 7s idle / reveal on header hover'
animation (worker-orb-hidden / worker-orb-reveal) every other orb has. Added its
container (.similar-artists-enrich-button-container, purple) to the list.
- Orb wouldn't pause when the worker had finished its library: the toggle keyed
off classList.contains('active'), but a done worker sits in the green
'complete'/idle state, so clicking tried to resume (no-op). Now it pauses
unless already paused → pausable in any state.
- Switched from inline onclick to addEventListener (matches spotify/itunes/etc.,
the majority pattern) instead of the amazon/discogs inline style.
