Adds the dashboard status bubble (the small icon row) for the Similar Artists
worker, alongside the modal entry. Mirrors the per-source bubbles: MusicMap logo,
purple accent, spinner + active/complete/paused states, hover tooltip, and a 2s
status poll against /api/enrichment/similar_artists/status. Click toggles
pause/resume. Tooltip shows matched/pending (the worker has no artist/album/track
phases). 74 JS integrity tests pass.
Closes the gap where similar artists only existed for WATCHLIST artists: a new
background worker populates them for the whole LIBRARY, slotting into the
existing enrichment-worker pattern (bubble + Manage Enrichment Workers modal,
status/pause/resume, matched/not_found/pending/errors).
Per source-matched library artist → get_musicmap_similar_artists(name, 25)
(the same matcher the artist-detail page uses: fetches MusicMap names, matches
each to the user's source chain — primary + active fallbacks — returns only
matched artists) → store via add_or_update_similar_artist keyed by the artist's
metadata source id, the SAME key the watchlist scanner + artist map use, so the
two cooperate (idempotent upsert + retry_days window).
- core/similar_artists_worker.py: pure seams (pick_source_artist_id,
map_payload_to_store_kwargs, process_artist) + the threaded worker; skips
artists not yet source-matched; classifies not_found vs transient error
(retry after 30d).
- DB migration: similar_artists_match_status / _last_attempted on artists
(mirrors every other source worker's tracking columns).
- Registered in EnrichmentService + instantiated in web_server, DEFAULT-PAUSED
(opt-in) like Amazon — MusicMap is scraped/outage-prone + this is library-wide.
- SERVICE_ENTITY_SUPPORT['similar_artists']=('artist',) so the modal breakdown
('artists with / without similars') + Retry work; manual-match (inapplicable
to a relationship) is gated out via relationship:true.
- 10 seam tests; existing 80 enrichment tests still pass.
Note: keys under profile 1 (single-profile setups); multi-profile is future work.
The fixed hamburger (top:16 left:16, 44px, z9999) sat on top of the map's back
button on mobile. Push .artmap-nav-left right by 52px on <=760px so the back
button clears it.
- Toolbar wraps on phones (<=760px): back + title + stats and the compact tools
stay on row 1, the search drops to its own full-width row below so nothing gets
crushed. Brand text hidden, stats truncate with ellipsis.
- Island nav + canvas height now MEASURE the toolbar height instead of assuming
~50px, so the taller wrapped header doesn't overlap the nav or clip the canvas.
64 JS integrity tests pass.
- Info panel becomes a bottom SHEET on phones (<=760px): slides up when you tap
a bubble, doesn't steal map width (islands frame full-width via _artMapReservedW
= 0 on mobile). Grip/handle to dismiss; a floating menu FAB opens it to the
dashboard + top-artists. Desktop stays the right sidebar.
- Genre sidebar hidden on mobile (the top-left quick-jump nav handles genre
switching; no room for a sidebar).
- Touch tap now selects a bubble (card in the sheet) instead of opening the modal,
matching desktop click; ignores taps that were drags.
- Resize/orientation: debounced reflow that re-styles the panel for the new
breakpoint, recomputes canvas size (minus sidebar/toolbar), and re-frames the
focused island / fit. 64 JS integrity tests pass.
- Watchlist button now reflects real state: shows 'On watchlist' (filled) vs
'Watchlist' (outline), confirmed per-artist via /api/watchlist/check, and
flips instantly when you add/remove (cached in _watchSet so it stays correct
as you browse). Uses the artist's source id, works on any map.
- Debounced hover-select: the card only swaps to a bubble you've settled on for
~0.8s, so sweeping toward the panel no longer keeps changing the card on
bubbles you pass over. Clicking a bubble selects it instantly (bypasses the
debounce, pins the card) instead of auto-opening the modal — Details button
still opens it.
- Fix: panel started at top:0 and covered the navbar; now it starts below the
.artist-map-toolbar (measured) so the toolbar stays clear. 64 tests pass.
The persisted Standard/Enhanced preference was re-applied on every artist load
BEFORE the data came back — so for an artist not in the library (source-only, no
Enhanced view) it still flipped to Enhanced, which showed an empty Enhanced pane
and never rendered the discography.
Now the preference is applied inside loadArtistDetailData, after we know the
artist's status (data.artist.server_source). Only library artists honour a saved
'enhanced' choice; source-only artists always stay on Standard (discography).
A polished detail panel on the right of every map (never collides with the genre
sidebar; islands now frame in the space left of it):
- Header dashboard: view title, Artists / Watchlist / Genres stat tiles, and a
watchlist-coverage bar for the current genre/view.
- Top-artists list: the current island's biggest artists, clickable (shows
their card + ripples them on the map).
- Rich artist card on hover/click: large art (from the decoded bitmap), genre
chips, popularity bar, connection count, watchlist/discovered badge, and
actions — Explore from here, Details, Watchlist toggle, Open artist page.
Card stays pinned (no auto-revert) so you can reach its buttons; a back
button returns to the list.
64 JS integrity tests pass.
Bubbles now rise up into position (water-surfacing) with a soft ease-out-back
settle and alpha fading in a touch faster than scale. Stagger is continuous
radial + a deterministic per-bubble jitter so they fill in organically instead
of popping in visible rings/segments. 64 JS integrity tests pass.
Root cause of the 'loads as placeholder orbs, only pops in after a zoom' bug:
streamed images were cached in _artMap.images but written into the buffer via
the per-node composite path, which didn't reliably refresh in one-island /
overflow mode — so covers stayed as placeholders until a zoom forced a full
rebuild that picked up the cached bitmaps.
Now that each map's buffer is small (one focused island, or a small explore
map), a throttled FULL rebuild on image arrival is cheap and always bakes every
cached image. Dropped the composite call from the stream; art fills in by itself
as it loads. 64 JS integrity tests pass.
- Soft genre-hued halo glows behind the focused island (cached per-hue sprite →
one drawImage, no per-frame gradient) so it reads as a place on the water.
- Hover-pop: hovered bubbles scale up + get a bright hue ring + glow, even on
static genre islands (drawn on top), so hover always feels tactile/responsive.
- Genre quick-jump: click the genre name in the nav for a dropdown of every
genre island — jump straight to one instead of only prev/next.
- Decluttered: dropped the redundant in-world island titles in one-island mode
(the nav bar already names the genre, and they could clip off the top).
64 JS integrity tests pass.
- Focused islands now render from the high-res buffer (one cheap crisp blit)
instead of redrawing every bubble each frame for the bob. In one-island mode
the buffer already covers just that island at high resolution, so this is
crisp AND cheap — kills the genre lag. Bob/shove stay live only for small
views (zoomed-in subsets, explore) where per-frame redraw is cheap.
(Overflow threshold 650→140; the loop parks once the island bakes.)
- Fewer bubbles per island (maxPerIsland 500→300) — less cramped, lighter bloom.
- Island nav bar moved from bottom-center to top-left (clears the genre sidebar
+ toolbar). 64 JS integrity tests pass.
Two things from feedback:
1) Toolbar search now queries the metadata source for ANY artist (like the
discover page) and launches an exploration on click — instead of only
filtering the current map's nodes (which showed nothing for off-map names).
2) Genre + watchlist maps now frame ONE genre island at a time, with prev/next
nav (and ← / → keys) through the genres. This sidesteps the persistent
'renders small/sparse' bug entirely: only the focused island is visible, so
the buffer covers a small region at HIGH res (crisp covers, no more shrunk
images) and the live layer handles just ~hundreds of bubbles (bob works, no
overflow). Each island blooms in (drop-in-water) on focus. Explore stays
multi-island (it's small). A bottom nav bar shows genre name + i/N.
Streaming caches off-island images silently (no redraw) so navigating is
instant. 64 JS integrity tests pass.
Fixes the genre-map 'renders small/sparse after the reveal, zoom fixes it' bug.
Root cause: tighter islands (Phase D) raised the fit-zoom so nearly every bubble
crossed the live-size threshold → the buffer excluded them all (thought they
were live) but the live layer is capped, so only ~600 of 1800 drew until a zoom
rebuilt the partition.
Fix: _artMapRebuildBuffer now counts would-be-live bubbles; if more than the
live layer can draw (>450), it sets _liveOverflow and bakes EVERYTHING into the
buffer (full, correct render). The live layer + bob only take over once zoomed
in enough that few bubbles qualify. So the overview is always complete,
regardless of zoom. Trade-off: very large maps (genre 1800) render from the
buffer (no per-bubble bob, slightly softer when deeply zoomed until the
zoom-rebuild sharpens) — correctness over flourish on the crowd.
Also: whole animation loop capped at ~30fps (reveal/ripple/bob all read fine at
30) to cut the churn on dense maps; a pending rebuild (dirty) always draws so the
throttle can't skip the post-reveal bake. 64 JS integrity tests pass.
Addresses the perf + tooltip feedback:
- Hover constellation no longer clips per node every frame (images are already
pre-masked circles) — that per-node ctx.clip() was the hover-lag culprit once
the ambient loop forced continuous redraws. Now a plain drawImage + arc tint.
- Ambient buoyancy loop runs at ~30fps when idle (full 60 only during
reveal/ripple), halving redraw cost on dense zoomed-in maps while keeping the
bob smooth.
- Gloss highlight gated to bubbles >=12px on screen (skips the dense swarm) —
halves per-frame drawImage cost when zoomed in.
- Tooltip photo now paints from the already-decoded bitmap into a canvas
instead of a fresh <img src> reload — fixes the blank photo when sweeping
across dense zoomed-in bubbles (the <img> was churn-reloading faster than it
could decode). 64 JS integrity tests pass.
Clicking (or tapping) the map now drops a water ripple: a hue-tinted ring
expands from the point AND nearby bubbles get shoved radially outward at the
wavefront, then settle back as it passes and decays (_artMapNodeDisplacement —
a gaussian bump at the expanding front, world-space radial push). Ripples emit
from the clicked bubble's centre in its genre hue (or the bare click point),
and still open the artist after a beat. Replaces the old single purple ring.
Note: the physical shove acts on live-layer (zoomed-in) bubbles; at the far-out
overview the ring shows but the tiny baked bubbles don't move. 64 tests pass.
- Ambient bob: bubbles gently float (sine offset, phase varies by position so
they move in a wave, not in unison). Driven by a persistent rAF loop that runs
only while bubbles are on screen + the tab is visible, and parks when zoomed
out (_liveCount==0) or the map closes — so an idle overview costs nothing.
- Glassy specular highlight (cached sprite, cheap drawImage per bubble) so
bubbles read as glossy orbs at every size.
- Tighter island spacing (water gap 7*nodeR → 3.5*nodeR) so the settled
overview is more substantial, not thin-spread — addresses the 'mini version'
feel after the reveal ripples fade.
- Ambient resumes on zoom and on tab re-focus; stops cleanly on close.
64 JS integrity tests pass.
Islands now bloom in like drops on water instead of a flat fade:
- Each island reveals in turn (staggered by island order); within an island,
bubbles fade + scale (0.55→1, ease-out) outward from the centre by radial
distance — a drop-in-water bloom. Genre titles fade in just after.
- A hue-tinted water ripple ring expands from each island centre as it blooms
(_artMapDrawRipples — reused by click ripples in Phase E).
- During the reveal the static buffer is bypassed so EVERY bubble can animate
(live layer, cap 2200); when the bloom ends it bakes into the buffer once and
steady-state returns to the cheap two-layer path.
- aAlpha folds into the global draw-alpha multiplier so fades compose cleanly.
64 JS integrity tests pass.
All three maps (watchlist / genre / explore) now lay out as genre 'islands' on
the water via one shared engine (_artMapLayoutIslands):
- Group artists by primary genre (long tail folds into 'Other'; max 14 islands).
- Each island is a FILLED disc of covers packed centre-out (no empty donut
hole), most-popular nearest the middle, focal artists sized up + centre-most.
- Islands spread by golden spiral + push-apart with generous water between.
- Clean floating genre TITLE above each island (hue-tinted, glow) instead of
the old giant translucent label bubble.
- Per-genre accent hue tints member-bubble borders so clusters read as a family.
- Discovery edges (watchlist→similar, center→ring1→ring2) remapped to the new
node ids so the hover constellation still works across islands.
Replaces the per-artist donut clusters from the screenshots. Shared helpers:
_artMapGroupByGenre, _artMapPackDisc, _artMapRemapEdges, _artMapFitToContent.
64 JS integrity tests pass.
Addresses the screenshot feedback (mix of detailed covers + blank dots, lag,
'weird' load):
- Pre-mask each album image into a circle ONCE at load (a canvas), so every
draw is a plain drawImage instead of a per-frame ctx.clip(). Clipping was
the live-layer stutter — hundreds of clips per frame. Now free.
- Draw album art at nearly every on-screen size (only sub-2.2px fall back to a
dot), instead of detailed-vs-blank-dot tiers. Consistent 'sea of covers'.
- Reveal is now a clean ease-out-cubic fade of the whole map (buffer blit +
live layer ramp together via _drawAlphaMul) — dropped the bouncy per-node
pop that read as 'weird'. The real island ripple bloom comes in Phase C.
64 JS integrity tests pass.
Foundation for the water/ripple redesign. Splits rendering into:
- Static far-field buffer: small/distant bubbles, baked once (cheap blit).
- Live overlay layer: every bubble big enough to read (radius*zoom >= LIVE_PX)
redrawn each frame in world space, so it can scale/bob/ripple. Viewport-
culled + capped at 600 draws.
The partition is frozen at buffer-build zoom (_liveBuildZoom) so the two sets
stay exact complements even mid-zoom — no flicker, no double-draw.
Adds an idle-capable rAF loop (_artMapStartLoop/_artMapStepAnimations) that runs
only while something animates and stops when still. First payload: a reveal —
the far field fades in globally while live bubbles pop outward from the camera
centre (ease-out-back, staggered by distance). Wired into all three loaders.
Bonus: live bubbles now draw full-res at the current zoom instead of through the
4096px-capped buffer, so zoomed-in artwork is crisp (addresses the earlier
low-res complaint structurally). Engine only — the island layout, ripple
choreography and click physics build on this in B–E. 64 JS integrity tests pass.
The streaming fix in Phase 4 still rebuilt the ENTIRE offscreen buffer (~1500
nodes) on each image wave, and any hover/pan during streaming hit that same
dirty flag — so interacting while images loaded redrew the whole world over and
over (the 'laggy until all images load' jank).
Now each arriving image composites ONLY its own node into the existing buffer
(_artMapCompositeNode) and does a cheap rAF-coalesced blit — no full rebuild.
The per-node draw is extracted into _artMapDrawNodeToBuffer so the full rebuild
and the incremental compositor share identical drawing (can't drift). Falls back
to a full rebuild only if the buffer isn't built yet. Pan/hover stay at
blit-speed the entire time images stream in.
All three maps (watchlist/genre/explore) now paint instantly with placeholder
circles and stay fully interactive (pan/zoom/hover/click) while images stream
in throttled ~280ms waves and sharpen the map in place. Replaces the old
blocking 'await all N images then paint' loaders — the headline 'feels slow'
fix. Focal/large nodes fetch first; a per-open load token cancels stale streams
when you jump to another artist, so rapid click-through never piles up fetches.
- Images: decode adaptively (focal/watchlist nodes ~256-384px, small nodes
~112-150px) instead of a flat 128px — crisp where it matters, memory still
bounded (~150-250MB, not 6GB). Fixes the low-res look.
- Hover constellation: drop the activation delay 800ms → 220ms (it felt 'gone'
because nothing happened for nearly a second), and draw the connection lines
as a wide-faint halo + crisp core (a real glow) with no per-frame gradients
or shadowBlur — stays cheap.
- Backdrop: subtle cached radial glow + vignette behind the map for depth
instead of a flat fill (one cheap fillRect/frame).
JS clean; 64 integrity tests pass.
Perf telemetry was the giveaway: after the buffer cap, rebuild + draw were both
~10ms, yet fps stayed 1-3 and the browser 'locked'. Cheap draw + locked system =
memory/GPU thrash, not drawing.
Cause: artist images load at up to 1000×1000, and a dense map holds ~1500 of
them — ~1500 × 1000² × 4B ≈ 6 GB of decoded ImageBitmap memory. The browser GCs/
evicts textures constantly → systemic lag the canvas timers don't see.
Fix: decode straight to a 128px avatar via createImageBitmap resize options
(nodes render tiny anyway). ~1500 × 128² × 4B ≈ 100 MB instead of 6 GB. Falls
back to full decode on engines that ignore the resize opts.
This is the one that should actually make it smooth. Perf overlay stays on 'd'.
Perf telemetry from the genre map (2004 nodes) proved it: the offscreen buffer
was 7465×10240 (76 megapixels) — rebuilt in ~979ms on every zoom and blitted at
~150ms/frame (3 fps), with the constellation overlay piling on top. The buffer
renders the WHOLE world, and the size cap was 10240px.
Cap the max buffer dimension to 4096 (MAX_BUFFER_PX). On the dense genre map
that's ~12MP instead of 76MP → ~6x faster rebuild and blit, and more nodes drop
under the LOD dot threshold so the rebuild also draws fewer image-clips. The cap
only binds on large worlds; small watchlist/explorer maps don't reach it and
stay full-resolution.
Tunable; perf overlay ('d' → app.log) stays so we can confirm the new numbers.
The on-canvas overlay text can't be copied (and can't be grabbed mid-freeze), so
when perf mode is on ('d'), the frontend now also POSTs the render timings to
/api/discover/artist-map/perf ~1.5x/sec, which logs them as [ARTMAP-PERF] in
app.log. Lets the bottleneck be diagnosed from the server side with no manual
copying.
- REVERT the spatial-grid hit-test I added in Phase 1. It inserted each node
into every grid cell its bounding box overlaps; the genre map's huge cluster
nodes span an enormous number of cells, so the first hover/click triggered a
multi-second synchronous build → 'can't hover or click' freeze. Back to a flat
O(N) single-pass hit-test (no per-move sort) — sub-ms even for thousands of
nodes, can't lock up.
- Keep the safe Phase 1 wins (render coalescing, tooltip de-churn, solid-stroke
connection lines).
- Add a perf overlay toggled with 'd' on the map: shows node/edge counts, the
offscreen buffer size + scale, zoom, and the last buffer-rebuild + draw times.
So we can measure the real drag/zoom bottleneck (buffer rebuild) instead of
optimising blind.
JS clean; 64 integrity tests pass.
The Explorer prompt accepted any loose text and explored whatever you typed.
Now it's a proper picker: type -> debounced search of the metadata source
(reuses /api/discover/build-playlist/search-artists — Hydrabase if active,
Spotify if configured, else the active metadata source) -> shows real artist
results with images -> click one to explore that resolved artist. Enter picks
the top match (never explores raw text); Escape/Cancel/backdrop close.
Pure frontend: rebuilds _showArtistMapSearchPrompt() (same Promise<name|null>
contract, so the caller is unchanged), reusing the playlist-builder's search
endpoint + picker styling. No backend change.
Investigating 'each mode is different / not good enough' showed the engine is
already shared across all three modes (watchlist/genre/explore) and already does
LOD rendering, eased camera, and debounced zoom-rebuilds — so the inconsistency
was perception driven mostly by the (now-fixed) lag, not separate engines.
This phase surfaces more real data the map already has: the hover tooltip now
shows each artist's live connection count (computed from the map edges), shown
consistently across all three modes. Cheap (only recomputed when the hovered
artist changes, after Phase 1's de-churn). Additive + safe.
JS syntax clean.
Kills the hover/move lag on dense maps. Root causes were in the live
mouse/render path, not the layout:
- Render coalescing: _artMapRender() now just requests a single rAF; the actual
draw (_artMapDraw) runs at most once per frame. A burst of mousemove/pan/
animation calls no longer triggers many full-buffer blits per second.
- Tooltip de-churn: only rebuild the tooltip innerHTML (and reload its image)
when the hovered artist changes; a plain mousemove just repositions. Was
rebuilding innerHTML + a new <img> every pixel of movement.
- Spatial-grid hit-test: bucket nodes into a coarse world grid and test only the
cell under the cursor, instead of sorting + scanning every node each move.
Grid rebuilds only when the node set changes.
- Constellation lines: draw all connection lines as ONE solid-stroke path
instead of creating a fresh linear-gradient object per line every frame —
that per-frame gradient churn was the main 'connected lines' lag.
No layout/data/click changes; behaviour identical, just frame-bound. Pure
frontend; JS syntax clean.
- Verified end-to-end: fetch_public_playlist_full pulled all 236 tracks of the
test playlist via SpotipyFree (the library handles the client-auth that 429'd
the raw approach). Name + tracks correct.
- requirements.txt: declare spotipyFree>=1.1.2 as a normal pip dependency (like
spotDL, also MIT — aggregation, not vendored) + websockets (a transitive dep
SpotipyFree/spotapi needs that pip doesn't pull automatically). Code still
soft-imports + falls back to embed, so it's never a hard runtime requirement.
- meta fetch uses limit=1 (name/owner only) so we don't pull the whole list
twice. 9 tests green.
The in-house anonymous-token path is blocked by Spotify (429 without the web
player's rotating client-auth). Switch the full-fetch to SpotipyFree — the
maintained no-creds spotipy drop-in spotDL uses, which tracks that machinery.
- core/spotify_public_api.fetch_public_playlist_full now uses a SpotipyFree
client (playlist + playlist_items + next), normalising the spotipy-shaped
items to the embed scraper's shape. Injectable client_factory keeps it
unit-testable without the library or network. Dropped the dead in-house
token/pagination code.
- Licensing: SpotipyFree is GPL-3.0, so it is NOT bundled/required (SoulSync is
MIT). Optional, user-installed: the import is soft, and on ImportError (or any
failure) fetch_spotify_public falls back to the embed scraper (~100). So the
shipped project stays cleanly MIT and the link path never regresses.
- requirements.txt: documents it as a commented optional extra
(pip install SpotipyFree) with the GPL/MIT rationale.
- 9 tests: normalisation, pagination past 100, library-missing -> raises (->
fallback), and the embed-fallback orchestration.
Needs a live click-through with SpotipyFree installed to confirm the exact
class/method names match (SpotipyFree.Spotify / playlist / playlist_items).
The full-fetch's logs used a bare module logger that app.log doesn't capture,
so we couldn't see whether the API path succeeded or why it fell back. Route
them to 'soulsync.spotify_public' and log: token found?, embed parsed?, the
API HTTP status on a non-200, and pagination result. Lets us see the exact
failure (e.g. 401 vs 429) on the next link-tab test.
Live debugging the 'shows 100' report:
- The full playlist page no longer embeds an accessToken, and get_access_token
/ server-time now 403/404. The EMBED page (open.spotify.com/embed/playlist/{id})
still ships a usable anonymous token. Was fetching the wrong page -> no token
-> raised -> embed fallback (100). Now reads the embed page for the token.
- Confirmed live: token extraction + embed parse work; the token is accepted by
the Web API (429 rate-limit, not 401). Could not show >100 from here because
the test IP got rate-limited from probing; needs a clean-IP click-through.
While in there, made it more robust against the rate-limiting that's clearly in
play:
- Refactored scrape_spotify_embed -> reusable parse_embed_html.
- fetch_public_playlist_full now does ONE embed fetch for token + name + first
page (no separate metadata call = fewer requests = less 429 surface), then
paginates the API. If the API is unavailable/rate-limited, it keeps the embed
page's tracks (<=100) instead of raising — so the result is always >= today's
behaviour, never worse.
- 12 tests incl. the new API-fails-but-embed-tracks-survive path.
Caveat unchanged: rides Spotify's undocumented embed-page token; degrades to the
embed fallback, never crashes.
The no-auth 'add by link' path scrapes Spotify's embed widget, which only ever
contains ~100 tracks and can't paginate — so big public playlists got
truncated. This adds an in-house anonymous fetch that pulls the FULL list:
- core/spotify_public_api.py: reads the anonymous web-player accessToken Spotify
already embeds in its own open.spotify.com page HTML (no app credentials, and
no rotating TOTP secret for us to maintain), then paginates
/v1/playlists/{id}/tracks 100 at a time until the whole playlist is pulled.
Returns the embed scraper's exact shape. Pure helpers + injected http_get so
it's unit-testable without the network.
- core/spotify_public_scraper.fetch_spotify_public(): tries the full fetch for
playlists; on ANY failure (or for albums) falls back to scrape_spotify_embed.
Worst case == today's behaviour, so the link path can't regress.
- web_server: the link-tab endpoint and the authed flow's last-resort scrape
now both go through fetch_spotify_public.
Scoped entirely to the spotify_public_* (no-auth) path — the authenticated
playlist sync is untouched. 11 tests (token extraction, normalisation,
pagination past 100, and the embed-fallback orchestration).
Caveat: rides Spotify's undocumented page-embedded token — expected to break
when they change their page; it degrades to the embed fallback, never crashes.
Needs a live click-through to confirm the token path works end to end (can't
hit Spotify from the test env).
The onclick-coverage guard only scans the split modules + a hardcoded extras
list, so it flagged openEnrichmentManager() (defined in the new, loaded
enrichment-manager.js) as undefined. Add enrichment-manager.js to the scanned
non-split files. The function genuinely exists and is loaded via its script tag.
Global priority previously set order only; per-worker pin also re-queued the
group's failed items. Made global consistent: setting a group globally now also
resets that group's not_found -> pending on every supporting worker, so each
worker sweeps ALL pending + failed of the group before moving on. Toast reports
total re-queued. Workers that don't enrich the group are skipped.
- Rebuilt the modal header: gradient top bar with a glowing 🧬 icon chip,
gradient title + subtitle, and styled refresh/close — replaces the flat bar.
- Global 'process first everywhere' control in the header: Artists/Albums/
Tracks/Auto applies to every worker at once (workers that don't enrich a
group are skipped via the 400 the endpoint already returns). Sets order only.
- Match rows: replaced the loud accent-gradient artwork placeholder with a
subtle neutral chip showing the entity glyph; real images layer over it and
remove themselves on error, so missing/broken art never leaves ragged gaps.
- Removed overflow:hidden from .em-row.
Frontend only; JS syntax clean.
Addresses three pieces of UI feedback:
- Fix entity order: enrichment coverage was rendering by object-key order
(albums first). Now sorted canonically artist → album → track via
_emOrderEntities, used everywhere.
- Combine 'Processing order' and 'Enrichment coverage' into a single set of
entity cards: each card shows coverage (segmented matched/not_found/pending
bar + %) AND is the click target to pin that group to enrich first, with
live 'Now' / pinned 'First 📌' / 'Done' states and per-worker accent. Drops
the two redundant sections (and the old chain/stats renderers).
- Richer match rows: status stripe down the left edge (red=not found,
amber=pending), larger rounded artwork with a gradient placeholder, parent
context (artist/album), and a subtle slide-on-hover.
Frontend only; JS syntax clean.
Aligns the 'process this group first' behaviour with intent:
- Pinning a group now also re-queues that group's previously-failed
(not_found -> pending) items, so the worker processes ALL unmatched in the
group (pending + missing), not just never-tried ones. Safe from loops: each
is attempted once, still-unmatched return to not_found, and the pending-only
worker hook won't re-pick them. Toast reports how many were re-queued.
- The left rail now shows each worker's current group while running
('Running · albums'), so you can see what every service is on at a glance.
Frontend only; reuses the tested /priority + /retry endpoints.
- #1 Unconfigured-source banner: when a source has enabled=false, show a
notice that browsing works but matches/retries won't run until it's set up.
- #2 Rate-limit detail: when rate_limited, surface 'resumes in ~Xm' (from the
status payload) instead of just a pill.
- #3 Richer rows: unmatched items now show parent context — an album's artist,
a track's album — via a parent expression in the query (+ test).
- #4 Bulk select: per-row checkboxes + a bulk bar to retry several at once
(capped concurrency), reusing the /retry item endpoint.
- #5 Remember last worker: selection persists in localStorage and is restored
on open; openEnrichmentManager(workerId) supports future deep-linking
(bubbles left on their pause-on-click behaviour).
- #6 Keyboard nav: ArrowUp/Down moves focus between rows; actions are native
buttons (Enter/Space) and Escape closes — list isn't poll-refreshed so focus
is stable.
53 enrichment tests green; JS syntax clean.
Per-worker processing-order override + UI polish.
Feature — pin an entity group to enrich first:
- Each worker normally runs artist -> album -> track. A user can pin one
group (artist/album/track) to run first from the modal; the worker keeps
that group first until it's exhausted, then resumes the normal chain.
- core/worker_utils.py: read_enrichment_priority() (reads
<service>_enrichment_priority each loop, live) + priority_pending_item()
(shared, whitelisted query returning the worker's expected item shape;
Spotify/iTunes get album_individual/track_individual via a type map).
- A guarded ~6-line hook at the top of all 11 workers' _get_next_item.
CRITICAL: when nothing is pinned (default) the hook returns immediately,
so default enrichment order is byte-identical to before. Discogs (no track)
and Genius (no album) only honor their supported entities.
- core/enrichment/api.py: GET/POST /api/enrichment/<id>/priority (+ config_get
hook); POST validates the entity against what the source enriches.
- 14 new tests (helper shapes, exhaustion, route get/set/clear/validate).
UI:
- Refined hero header: identity + inline status left, single Pause right,
'now enriching' quiet sub-line; overall coverage % moved into the stats
section ('82% matched · 1,203 of 1,460'). Hero gently pulses while running.
- New processing-order strip: artist→album→track steps showing the live phase
(pulsing 'now'), pinned group ('first' + 📌), and done/remaining; click a
step to pin it, click again for auto.
py_compile clean across all 11 workers; 52 enrichment tests green.
Fixes a correctness bug and adds bulk re-queuing.
- Bug: per-row 'Retry' used clear-match, which sets an item to not_found
with last_attempted=NULL. The worker only retries not_found items where
last_attempted < (now - 30d), and 'NULL < cutoff' is false in SQLite, so
those items were never re-queued. Fixed by resetting match_status to NULL
(pending), which every worker's queue picks up on the next pass.
- New POST /api/enrichment/<id>/retry with scope 'item' | 'failed'
(failed = re-queue every not_found item of an entity type), backed by a
pure whitelisted build_reset_query + MusicDatabase.reset_enrichment().
- UI: per-row Retry now hits /retry; a 'Retry all failed' bulk button appears
when the current entity has not-found items (confirm + count toast); a hint
line explains retry/match/auto-retry behaviour.
- 11 new tests (38 enrichment tests total, all green).
Dashboard 'enrichment bubbles' could pause/hover but offered no way to
*manage* a worker. This adds a full management modal opened from a new
header button, covering all 11 enrichment sources.
Backend (testable core helper + seam tests; no live-DB dependency):
- core/enrichment/unmatched.py: pure, whitelisted SQL builders for the
unmatched browser. service/entity validated against a support map (never
interpolated raw); search + pagination bound as params; tracks join albums
for artwork; limit capped at 200.
- database/music_database.py: get_enrichment_unmatched() +
get_enrichment_breakdown() (the breakdown splits matched/not_found/pending,
which the existing get_stats().progress lumps together).
- core/enrichment/api.py: GET /api/enrichment/<id>/{unmatched,breakdown} on
the existing blueprint + a db_getter hook.
- web_server.py: wire db_getter=get_database.
- tests/enrichment/test_unmatched.py: 19 tests across builders, DB methods,
and Flask routes.
Frontend (vanilla, matches app conventions):
- webui/static/enrichment-manager.js: worker rail with live status + coverage
micro-bars, accent-themed detail panel (hero header, segmented matched/
not_found/pending stat cards, current item, pause/resume), and a searchable
paginated unmatched browser with inline manual match (reusing
search-service + manual-match) and retry (clear-match re-queues).
- Polish: entrance/exit motion, scroll-lock, Escape, refresh control,
flicker-free polling (in-place updates), skeleton loaders, relative
timestamps, per-worker accent theming, real dashboard logos reused at
runtime (with the same invert/circle treatment), responsive rail.
- index.html: header button + script include. style.css: full styling.
Reuses existing pause/resume, status, and manual search+assign endpoints.
Backend tests green (19 new + 11 existing enrichment tests).
Per-track import does heavy synchronous server-side enrichment (metadata,
art, lyrics) that can take 60-90s/track, far longer when external sources
are degraded. The React apiClient (ky) had no timeout, so ky's default 10s
aborted the import-process request client-side even though the server
completed the import (200) and moved the files. The import loop then counted
the aborted call as an error, so the bar stayed at 0 and flipped to 'Failed'
while files imported fine.
Give the two import-process calls (album/process, singles/process) an
explicit 5-min timeout. Scoped to import only -- every other endpoint keeps
the 10s default; bounded, not disabled. Server behavior unchanged.
Adds a test asserting both calls pass the long timeout.
- B023: default_fetch_tracklist built a per-item lambda closing over the loop
variable `it`. Replaced with a module-level _item_get(item, key, default)
helper (takes the item as a param — no closure). Behavior unchanged; the
dict/object normalization test still passes.
- S110: the two best-effort guards in the canonical job (skip-already-pinned
read, estimate_scope active-server read) now carry `# noqa: S110 — <reason>`,
matching the repo's existing convention for intentional swallow-and-continue.
ruff check passes on all canonical files + tests; 30 affected tests green.
