Pache711: a cover-art finding showed the (correct) found album art next to a
(wrong) artist image with one "Apply Art" button — no way to take one and
skip the other. Turned out "Apply Art" only ever applied ALBUM art anyway;
the artist image was display-only context, so the bundling was an illusion
the UI created.
Now the finding is genuinely multi-target:
- scan (missing_cover_art.py): also searches for an artist image (always, so
a WRONG existing one can be replaced — Boulder's call), name-matched
exactly. Stored as found_artist_url only when it differs from the current
artist thumb, so nothing is offered when there's nothing to change.
- apply (_fix_missing_cover_art): honors a target via _fix_action —
'album' (default, unchanged "Apply Art" behavior: DB thumb + embed +
cover.jpg), 'artist' (the artist's DB image), or 'both'. New _fix_artist_art
sets artists.thumb_url for the album's artist.
- UI: each found image gets its own apply button — "Use for album" /
"Use for artist". Applying either resolves the finding, so taking the
correct one and ignoring the wrong one IS "fix one, dismiss the other".
Current artist art shows as "(current)" context with no button.
Default stays album-only, so the plain Apply Art button and every existing
caller behave exactly as before. Tests: 5 on the apply targets (artist-only /
album-only / default / both / missing-url) against a real SQLite DB, plus the
existing cover-art suite updated for the new artist search. 107 repair/
cover-art/UI-integrity tests pass.
Ashh: the manual-match modal fuzzy-searches a service and shows the top 8.
When the right release isn't in those 8 (common title — their example was
"Idols", which returns 8 unrelated releases and not Yungblud's), there was no
way through. But the user usually already knows the exact MBID.
Now the modal's search box doubles as a direct-ID box. Paste a MusicBrainz
MBID (bare UUID or a musicbrainz.org URL) and SoulSync looks that exact
entity up and shows it as the single result to confirm + Match — no fighting
the search ranking.
- core/library/direct_id.py: pure detector, returns the canonical ID only
when the text unambiguously IS one (whole-query UUID, or a UUID inside a
musicbrainz.org URL). "Idols", "Yungblud Idols", a UUID buried in free
text → None, so normal search is never hijacked.
- _search_service: direct-ID fast path before the fuzzy search —
get_release (→ get_release_group fallback for albums) / get_artist /
get_recording. A pasted-but-unresolvable ID falls THROUGH to fuzzy search,
so a typo can't dead-end the modal.
- UI: MusicBrainz placeholder now says "…or paste a MusicBrainz ID/URL".
Detector is service-keyed so Spotify/iTunes/etc. direct IDs can be added
later; today only MusicBrainz has a confirmable direct lookup, matching the
reporter's ask + screenshot. 9 tests: detector truth table (bare/URL/plain/
buried/other-service) + dispatch (confirmed release, release-group fallback,
unresolvable→fuzzy, plain query skips direct lookup).
Follow-up to 5187fe5f, which shipped stall handling as config-only keys.
Boulder wanted them user-accessible, so the two knobs now render in the
Torrent Client settings section:
- "Stalled torrent timeout (minutes)" — number input. Shown in MINUTES for
friendliness, stored in SECONDS (download_source.torrent_stall_timeout_
seconds). 0 disables. Blank/NaN falls back to the 10-min default on save.
- "When a torrent stalls" — Abandon (default) / Pause select, maps to
download_source.torrent_stall_action.
Both live under download_source (already in the settings POST allowlist), so
no backend change — load converts seconds→minutes, save converts back.
Inputs/selects only (no onclick), so the script-split onclick-coverage test
stays green. settings.js syntax-checked via Windows node.
noldevin's first torrent was stuck "downloading metadata" — a dead magnet
with no peers. The poll loop would ride the full album deadline (6h default)
on it, holding the worker the whole time, with no built-in escape.
New stall handling, off the existing poll loop:
- core/download_plugins/torrent_stall.py — pure StallTracker (clock injected,
no I/O): forward byte progress resets a stall clock; once a torrent spends
the stall timeout in a working state (queued/downloading/stalled/error)
with zero progress, it's stalled. seeding/completed/paused never count.
Covers the metadata-stuck case (0 bytes, 0 progress) and a dead mid-download
swarm with one rule.
- _handle_stalled: 'abandon' (default) removes the torrent + its partial data
(a metadata stub is junk) and fails the download so the next source can try;
'pause' parks it in the client for the user. Adapter errors are swallowed —
the download still fails cleanly.
- two settings (download_source.torrent_stall_timeout_seconds = 600,
torrent_stall_action = 'abandon'); timeout 0 disables, restoring the old
ride-the-deadline behavior. Config-key driven, matching the existing
album_bundle_* tuning knobs (no UI form, same as those).
Tests: 18 on the tracker + settings (timeout trip, progress reset, idle-state
exemption, pause→resume clock restart, disable, parse tolerance) + 3 on the
plugin action path (abandon removes w/ delete_files, pause pauses, adapter
error survived). 158 torrent-family tests pass.
Follow-up to 603b7a2a (tokens in the config store): /api/settings GET
returns the whole config dict, which would now include the OAuth access +
refresh tokens. The settings UI has no field for them — strip the key from
the response. dict(config_data) is a SHALLOW copy of live state, so the
section is rebuilt rather than popped in place (verified: response clean,
live config intact, token survives a settings save since POST merges
per-key).
wolf39us: "It keeps unauthenticating... daily" — re-auth fixes it until the
next day. Mechanism: spotipy's token cache was a loose FILE at
config/.spotify_cache. /app/config is a declared VOLUME, but a compose file
that doesn't map it explicitly gets an ANONYMOUS volume — recreated empty on
every container pull. So a nightly Watchtower update kept all his settings
(config lives in the database now) while silently dropping the OAuth tokens.
His redirect-URI change won't help: callback URLs only matter during the
initial handshake, never for refresh.
New DatabaseTokenCache (spotipy CacheHandler) stores the token payload in
the same database-backed config store as every other setting — tokens now
survive exactly as long as the rest of the configuration does. The legacy
file is imported once on upgrade (no forced re-auth) and removed on logout;
a failed cache write logs and never raises (spotipy calls it mid-request).
Tests: roundtrip, JSON-string tolerance, one-time legacy import (store wins
after the file vanishes), garbage file ignored, logout clears both stores,
write failure never raises. 204 spotify tests pass.
Netti93's follow-up report (single artist at download time, correct only
after retag) reproduces as FIXED on current dev — verified live against
Deezer's API with his literal track ('VERLIEBT IN MICH', FAYAN feat.
Dalton) and his exact config, through the real tag writer onto a real MP3:
TPE1=FAYAN, TIT2 gains '(feat. Dalton)', TXXX:Artists=[FAYAN, Dalton].
His last test (2.5.6 / May-19 dev) predates the fixes that closed it
(d5de724f contributors upgrade hardening, 0769fcd5 collab-tag loss).
These tests pin the full direct-download shape so it can't quietly
regress: Deezer /search payload (one artist) + provider on the candidate
(not the context) -> contributors upgrade fires -> feat_in_title and
artist_separator both honored. Network-free (client mocked with the live
API's verified response shape).
37725457 fixed _match_to_itunes to use the real iTunes client and flagged
the cross-source corruption as a possibility. Boulder's live DB proves it
happened: 6 of his 9 watchlist "iTunes" ids EQUAL the artist's Deezer id
(Taylor Swift's "iTunes" id was her Deezer id 12246; the real one is
159260351) — written back when the misnamed MetadataService.itunes slot
held a DeezerClient. The June-4 batch (Green Day, SOAD, Vulfpeck, ...) got
NULL instead because the slot now holds the Spotify primary.
The fix alone can't heal those rows: the backfill only fills EMPTY ids, so
a wrong non-empty id is permanent. New migration clears itunes_artist_id
where it equals deezer_artist_id (the corruption signature — distinct id
spaces, so a legitimate equal pair is effectively impossible, and the worst
case is a NULL that re-matches correctly on the next scan). Idempotent by
construction; similar_artists checked clean (its backfill always used the
registry correctly).
Tests: corrupted row cleared / legit + no-deezer rows kept / idempotent —
via a real re-init with the per-process init memo cleared (an app restart).
Boulder noticed his recently added watchlist artists (June 4 batch) have no
iTunes match while Spotify/Deezer/MusicBrainz matched fine. The rotated log
has the receipt: "Cannot match to iTunes - MetadataService not available" ×8
→ "Backfilled 0/8 artists with itunes IDs", every scan.
_match_to_itunes was the only matcher with no fallback: it read the PRIVATE
_metadata_service attr, which is None whenever the scanner is constructed
from a spotify_client — the normal web_server wiring — and gave up, while a
lazy-loading metadata_service property sat right next to it and the deezer/
discogs/musicbrainz matchers all fall back to their registry clients. Bonus
landmine: even when set, metadata_service.itunes is the FALLBACK-client slot
and may actually be a DeezerClient (per _match_to_deezer's own comment), so
"iTunes" matching could have stored a Deezer artist ID as itunes_artist_id.
Now mirrors the other matchers: canonical registry get_itunes_client().
Self-healing — missing IDs are re-attempted every scan, so existing
watchlists backfill on the next run with no migration needed.
Tests: match works with _metadata_service=None (the exact production
condition), unconfident result returns None, missing client degrades
gracefully. 103 watchlist tests pass.
CI failed all 7 requeue tests that passed locally. Root cause is a real
shipping bug, not test flake: config/settings.py's default template set
retry_next_candidate_on_mismatch: False ("Default off — opt-in") while the
monitor reads it with inline default True and the PR documents it as ON.
Outcome split the userbase: a FRESH install (or CI's clean runner) gets the
template key = retry engine silently OFF; an existing config.json lacks the
key = inline True wins = engine ON. Same code, opposite behavior, decided by
install age.
- template aligned to True (the documented + approved default; existing
installs already behave this way via the inline default)
- the requeue tests now pin the toggle ON via the wiring helper instead of
reading the runner's ambient config — CI's fresh defaults vs a dev's
lived-in config.json must never decide whether they pass. _patch_config
composes (it wraps the pinned get and falls through).
64 retry-engine tests pass; fresh-default simulation confirms the toggle
resolves True.
Boulder's lock-in question caught it: the read-only pre-flight called
os.statvfs unconditionally, which doesn't exist on Windows, and the
AttributeError wasn't covered by the except OSError — the whole cover-art
apply would have crashed for every Windows install (docker images are
Linux, so the reporter was fine; the maintainer wasn't). getattr-guarded
now: no statvfs -> skip the pre-flight, per-file EROFS detection (errno is
cross-platform) still active. Test pins the no-statvfs path.
Tim (Discord): cover-art automation fails with '[Errno 30] Read-only file
system' on every file; he chmod 777'd and nothing changed — because EROFS is
the KERNEL refusing writes to a docker ':ro' volume mount, which no chmod
can fix. SoulSync's response was a wall of per-file warnings and a fix
result that still said success with a soft "(read-only?)" hint.
- apply_art_to_album_files now pre-flights the album folder with statvfs
(asks the kernel, writes nothing): a read-only mount short-circuits the
whole album instead of failing file by file. Belt: a per-file/cover EROFS
(overlay quirks where statvfs lies) still sets the flag.
- the repair worker's apply now FAILS the finding with the actual cure:
"remove ':ro' from the volume mapping and recreate the container — chmod
cannot change this". EACCES (a real permissions problem chmod CAN fix)
deliberately keeps the old soft path.
Tests: RO mount short-circuits before any file/cover write, save-time EROFS
still flagged, EACCES not conflated with EROFS. 29 art/repair tests pass.
Sokhi (continued from #806): volume-numbered series ('B小町 …キャラクター
ソングCD Vol.2' / 'Vol.2.5' / 'Vol.4' / 'Vol.4.5') got each other's art from
both normal downloads and the retag tool. Two distinct holes, one principle:
1. The art picker's _album_matches validates by significant-token SUBSET —
built to tolerate '(Deluxe)'/'- Remastered' suffixes. CJK strips out of
the normalizer entirely, so Vol.4 → {b,tv,cd,vol,4}, a clean subset of
Vol.4.5's {b,tv,cd,vol,4,5}: the wrong volume validated as "the same
album with a suffix". Affected every fuzzy art source (iTunes, Deezer,
AudioDB, Spotify) in downloads, retag, and the missing-art repair.
2. MusicBrainz match_release scores by string similarity — Vol.4 vs Vol.4.5
is 0.973, so the wrong volume could win the match outright, and its MBID
then feeds Cover Art Archive with NO downstream validation (CAA is
MBID-keyed, trusted by design). With Sokhi's MB metadata source this is
the likely path in his logs (his release-group 404s push re-matching).
The shared rule (core.text.title_match.numeric_tokens_differ): digit-bearing
tokens must be IDENTICAL between the two titles. A number on one side only —
volume, part, sequel, remaster year — is a different release, never a
suffix. '1989' vs '1989 (Deluxe)' still matches (digits shared); 'Album' vs
'Album 2' now rejects (sequels!). Art picker rejects outright (falls through
to next source / the download's own art — the designed cost of a false
reject); MB matcher halves the candidate's confidence, landing it below the
70 gate while the exact-volume result is untouched.
Tests: helper truth table, the exact reported pairs through _album_matches,
and match_release end-to-end (wrong volume alone → no match beats a wrong
MBID; exact volume beats near-identical wrong one despite lower MB score).
828 matching/metadata + 301 musicbrainz/retag/artwork tests pass.
Boulder: "Taylor Swift shows only 8 albums, nothing before 2022, no singles,
no EPs" — for every artist (actually: every WATCHLIST artist). Traced live:
get_artist_albums caches its result under an UNQUALIFIED key (no limit/page
info), and the watchlist's new-release probe (limit=5, max_pages=1 — the
April "reduce watchlist API calls ~90%" optimization) stored its truncated
single page in that same slot. The artist detail page reads the cache first,
so a watchlisted artist's page showed only the newest handful of releases —
newest-first, hence "nothing before 2022" — re-poisoned on every scan, with a
30-day TTL. When the source-priority fetch comes back tiny, the page's
fallback path quietly serves it, so the symptom looked like a discography
filter bug. Not related to the #808 matching change (that is a pure max(),
provably additive).
Three pieces:
- get_artist_albums tracks whether the fetch stopped while more pages
existed (truncated) and only caches COMPLETE discographies. Individual
albums keep their opportunistic caching — they're complete entities
regardless of pagination. A small real discography that fits one page
stays cacheable even under max_pages=1.
- MetadataCache.purge_artist_album_lists(): delete the already-poisoned
album-list entries (TTL would have kept them for weeks); lists rebuild
lazily on the next artist-page visit.
- one-time startup purge in web_server, config-guarded
(maintenance.album_cache_purge_v1), mirroring the startup-repair pattern.
Tests: truncated probe never stores the list (but still returns its page),
complete multi-page fetch caches, and a genuinely-small one-page discography
under max_pages=1 still caches. 1087 spotify/cache/watchlist/artist tests
pass.
carlosjfcasero: 'Champagne Supernova (OurVinyl Sessions)' is in the library
but the artist page shows it unowned and wishlist cleanup never removes it.
Measured with the real catalogs: Deezer/iTunes title the TRACK with the
qualifier while the library track is bare (the qualifier lives in the album
title) — and _calculate_track_confidence crushed that pair to ~0.17: the
"clean" titles keep parenthetical words, so the length-ratio penalty treats
'Champagne Supernova' vs 'Champagne Supernova (OurVinyl Sessions)' as
different songs. (Also confirmed: the OurVinyl release is absent from
Deezer's discography for the artist, so the standard page's 25-release list
not showing it is the source catalog, not a bug.)
Fix 1 — core.text.title_match.strip_redundant_context_qualifiers: a
parenthetical qualifier whose text appears (word-bounded) in the db track's
ALBUM title — or in the other title — restates release context and is
stripped for a comparison variant scored with its own length guard. Genuine
version markers keep their penalty: '(Live)' on a studio album appears in no
context and still blocks; '(Live)' on 'Live at Wembley' correctly matches —
owning the live album IS owning the live cut. Wired into
_calculate_track_confidence, so every check_track_exists consumer (wishlist
cleanup, discography dedup, repair jobs) benefits.
Fix 2 — the artist-page ownership endpoint's album gate: when album-aware
narrowing eliminates EVERY library candidate (the source's album naming just
doesn't resemble the library's — 'Jillette Johnson | OurVinyl Sessions' vs
'Champagne Supernova (OurVinyl Sessions)' ~0.5), fall back to artist-wide
title matching instead of declaring everything unowned off a failed
album-NAME comparison.
Tests: 8 — the exact reported pair end-to-end through check_track_exists,
word-boundary containment ('live' in 'alive' doesn't count), version-marker
safety both ways, and prefix songs still blocked. 1125 matching/wishlist/
library tests pass.
Review findings from PR #801, fixed as promised after merge:
- core/imports/version_mismatch_fallback.py and core/downloads/task_worker.py
used bare getLogger(__name__) — outside the soulsync.* namespace where
handlers attach, so the entire retry story (the [Modal Worker] search/retry
walk and, critically, the "accepting best quarantined candidate as last
resort" warning) never reached app.log. Same bug class as the prepare.py
fix; both moved to get_logger. A repo sweep shows 61 more modules with the
same pattern — noted as its own cleanup project.
- the full-suite run also caught a miss of MINE, not the PR's: the new
origin-history.js wasn't registered in the script-split integrity test, so
openDownloadOriginsModal failed onclick coverage. Registered — and the
onclick scan now iterates the NON_SPLIT_JS registry instead of its own
hardcoded copy, so the next standalone module can't silently skip coverage.
Merged dev verified: PR's 77 tests + 4233 full-suite tests pass (the only
exclusion is the eternal soundcloud /app file); integrity suite 64/64.
Watchlist scans add announced albums on purpose (so singles download the day
they drop), but the future-dated tracks leaked into two hot paths:
- Fresh Tape / Release Radar: future albums got NEGATIVE days_old, and the
recency score (100 - days*7) has no upper clamp — prereleases weren't just
slipping into the radar, they were mathematically FAVORED above every
released track. That's the "50% prerelease" report. The builder now skips
confidently-future albums (and clamps days_old to 0 as a belt).
- Wishlist processing: every auto cycle burned a full Soulseek search +
timeout per unreleased track (~60 tracks/cycle for the reporter). Both the
auto and manual flows now skip future-dated tracks with a counted log line.
They STAY in the wishlist and join the cycle automatically the day their
release date passes — no state, the date check is per-cycle. An explicit
manual track selection overrides the gate (the user asked for those).
The gate (core/metadata/release_dates.py, pure + tested) is conservative by
design: Spotify dates come as YYYY / YYYY-MM / YYYY-MM-DD, and a track only
gates when its date is CONFIDENTLY future at its stated precision. Release
day counts as released; garbage or missing dates never block anything
(including out-of-range months/days, which fall back a precision level).
Tests: 6 covering all precisions, the release-day boundary, garbage
tolerance, dict shapes, and ordered partitioning. 403 wishlist/watchlist
tests pass.
User ask: "a modal that lists the tracks downloaded via watchlist" — extended,
as discussed, to playlists too. One modal, two tabs, opened from the Watchlist
page (watchlist tab preselected) and the Sync page (playlists tab) — same
shared-modal-different-entry-points UX as the rest of the app.
The data: library_history recorded which SERVICE a file came from but never
what TRIGGERED it. New origin/origin_context columns (migration + index) are
written once at the import chokepoint via core/downloads/origin.py, a pure
tested deriver that reads, in priority: an explicit _dl_origin stamp (set at
batch-task creation for direct playlist batches, where the playlist context
otherwise only survived in folder mode), the wishlist provenance already
riding in track_info.source_info (watchlist_artist_name / playlist_name —
watchlist_scanner has stamped these for ages), and the folder-mode playlist
thread. Manual downloads stay unclassified by design. History starts from
now — provenance can't be conjured retroactively.
API: GET /api/download-origins?origin=watchlist|playlist (paged) and POST
/api/download-origins/delete — deletes the file on disk (resolved through the
shared container/host path resolver), the matching library track row, and the
history entries; a file that refuses deletion keeps its row and reports the
error instead of lying.
UI: webui/static/origin-history.js — tabbed modal in the revamp design
language (accent light-edge, pill tabs, entry rows reusing the
library-history-entry components), per-row delete + select-all bulk delete
with honest result toasts, empty/loading states, per-tab totals.
Tests: 8 — deriver priority/shapes (incl. the exact watchlist_scanner
source_info shape and JSON-string survival), origin filtering + counts,
row fetch/delete isolation between origins, delete-track-by-path.
The lock-in pass caught the cost hole: art is fetched PER TRACK, and the old
code never touched archive.org at all — so an archive.org outage was free,
while the new native-first chain would pay a 10s timeout on every track
(a 12-track album = +2 minutes, exactly the import-slowness class we spent
today killing). One failed original now puts originals on a 10-minute
cooldown: subsequent fetches go straight to the 1200px CDN midpoint (the
pre-#806 behavior, full speed) and recover automatically when the cooldown
expires. Locked by a test: track 1 pays the failure once, track 2 never
touches the original. (Also: the missing time import the first run caught.)
The CAA branch of _upgrade_art_url capped art at the /front-1200 thumbnail —
a deliberate flakiness trade-off, but the policy had rotted into inconsistency:
iTunes art already shipped at 3000x3000, and bare /front URLs (release-group
lookups — exactly what the Re-tag flow produces) bypassed the cap entirely,
which is how Sokhi observed retag delivering full-res while downloads got 1200.
CAA URLs now upgrade to the bare /front ORIGINAL (native res, frequently
3000px+). The flakiness concern that motivated the old cap is handled where it
belongs, in the fetch: _fetch_art_bytes now walks an attempt chain — original
-> /front-1200 midpoint -> the original sized thumbnail — so a flaky
archive.org degrades to the old 1200px behavior, never below it.
Tests updated to the new contract (+3 chain tests: native-first, flaky
degrades to 1200 not 250, full chain ends at the thumbnail). 623 metadata +
1267 art-path tests pass.
Caught live by the new lookup timing ("Genius track lookup took 242.4s"):
the 429 handler slept the backoff (30/60/120s) in the CALLING thread and then
re-raised anyway — the import pipeline waited 2x120s per track for lookups
that still failed. Worse, the pre-flight backoff wait also slept while
HOLDING the global Genius API lock, so every other Genius caller queued
serially behind the nap.
Now the backoff is a gate: a 429 opens a 30s->60s->120s window and re-raises
immediately; any call inside the window raises GeniusRateLimitedError on the
spot. The error subclasses requests.RequestException, so every existing
caller (the import's source lookups catch RequestException and skip; the
worker's per-item guards) already handles it as a one-line skip — lyrics and
Genius tags are garnish, nothing is allowed to WAIT for them.
Tests: backoff window fails fast (<0.5s vs the old full-window sleep), a 429
opens and escalates the gate without sleeping, the error is a
RequestException (the no-call-site-changes hinge), success decays the gate.
Measured during a live album download: ~4m15s per track in post-processing
(normal is ~20s), with the time vanishing silently inside embed_source_ids —
up to 5 MusicBrainz calls per track crawling against a degraded musicbrainz.org
while the MB enrichment worker kept eating the same ~1 req/s per-IP budget.
Only Spotify/Last.fm/Genius were in the yield set; MusicBrainz, Deezer, iTunes,
Discogs etc. kept grinding through downloads.
Policy (new core/enrichment/yield_policy, tested):
- downloads active -> ALL enrichment workers yield (post-processing touches
every metadata source). listening-stats (local-only) and repair
(user-scheduled) intentionally keep running.
- discovery active -> the API-contention five yield (spotify/itunes/deezer/
discogs/hydrabase) — discovery never paused anything before, despite the
pause helper literally defaulting to label='discovery'.
- user overrides and user-paused bookkeeping keep their existing semantics;
the dashboard yield_reason label now says WHICH foreground work caused it.
Observability (the 4-minute silence can never come back):
- every source lookup is timed; >2s logs a warning NAMING the source and
duration (core/metadata/source.py _call_source_lookup)
- the pipeline always logs "Metadata enhancement took X.Xs" per track
7 policy tests (incl. the motivating case: MB yields to downloads, keeps
running during discovery); 277 pipeline/enrichment tests pass.
The lock-in pass caught it before it shipped anywhere: the pill styling set
display:inline-flex on the status cells — which are <td>s — knocking them out
of table-cell layout and corrupting the row grid. The pill is now a centered
pseudo-element painted BEHIND the text (z-index -1 inside the cell's own
stacking context), so the cell's box model is untouched. State colors stay as
CSS vars on the td and cascade into the pseudo.
Also covers the secondary live-progress writer discovered in the same pass:
it stamps legacy download-downloading / download-complete classes instead of
data-state — both vocabularies now get the same pills (accent + breathe while
downloading, green when complete).
A user reports ~0.7 MiB/s RSS growth; the one theory offered so far
(connection leak) was debunked, so instead of guessing: measure. New
core/diagnostics/memory_tracker wraps tracemalloc behind three GET endpoints
the user can drive from a browser:
/api/debug/memory/start begin tracing + baseline snapshot (idempotent)
/api/debug/memory/report top allocation sites by GROWTH since the baseline
(?top=N), with traced totals + process RSS so we
can see how much of the real growth tracing
accounts for; 15-frame tracebacks name the caller
/api/debug/memory/stop end tracing, free trace bookkeeping
Opt-in by design — tracemalloc shadows every allocation while active, so it
never runs by default. RSS via psutil with a /proc fallback.
Tests: report-without-tracking returns a hint (not an error); a real
start->hog->report->stop roundtrip attributes a genuine 5MB allocation to the
test file (fun fact encoded in the test: 'x'*1000 constant-folds into ONE
shared string and traces as ~40KB — the hog must allocate at runtime); the
stat formatter is duck-typed and unit-tested.
The status cells were the last plain-text corner of the revamped modal, and
they're the most alive data in the app during a run. The renderer now stamps
data-state on the download-status cell and toggles .row-working on the row
(visual-only hooks; zero logic change). CSS turns both status columns into
state-colored pills — accent while searching/downloading, amber processing,
green completed, red failed, orange quarantined — and ONLY the actively-
working states breathe (opacity, compositor-only). The working row carries the
same accent edge treatment as hover, but earned by real work instead of the
mouse. prefers-reduced-motion respected.
- entrance: soft rise + settle, one-shot spring
- header light-sweep: the dashboard's signature strip (same keyframes, same
transform-only technique) drifting across both modal headers
- progress sheen: a light band scanning the FILL — it lives inside the fill's
clip, so zero progress shows nothing and motion is gated by real progress
- hero stats become glass chips with per-state color identity (found green,
missing amber, downloaded accent) and a top light-edge each
- download modal's close X matches the discovery one (circular ghost, rotates)
- press feel on every pill button (active scale)
- all of it honors prefers-reduced-motion; only transform/opacity animate
Both modals were functionally perfect but visually dated — flat dark panels,
heavy table grids, the discovery modal still wearing its legacy RED border.
Pure CSS override layer appended last in the cascade; markup and JS untouched.
Same design language as the dashboard pass, theme-aware via --accent-rgb:
- deep glass surface with an accent light-edge along the top
- progress bars -> rounded inset tracks with gradient accent fill + glow
- tables -> micro-label sticky headers, calm hairline rows, accent hover
glow with an inset edge bar, themed thin scrollbars, accent checkboxes
- download modal: the two stacked progress bars become side-by-side glass
cards; tracks toolbar with a pill selection counter; glass footer with
pill buttons (gradient primary, ghost secondary, soft-red danger)
- discovery modal: red border killed, kicker typography header, circular
rotating close button, carded progress + table, matching pill footer
CI builds with GHA layer caching (cache-from/to: gha), and the nightly RUN's
cache key is just the instruction text — the layer could pin a months-old
"nightly" across releases, silently defeating the channel's whole purpose.
Referencing COMMIT_SHA (already passed by docker-publish.yml) in the RUN makes
every new commit bust that one layer while everything above it stays cached.
YouTube now gates downloadable formats behind JS challenges (nsig); yt-dlp
needs a JavaScript runtime (Deno — its only default-enabled one) to solve
them, and its STABLE channel can lag months behind a breaking YouTube change.
Users hit "Requested format is not available" on every stream and music-video
download with no hint why. Neither piece is fixable via requirements.txt:
Deno isn't a Python package, and a version floor can only ever resolve stable.
- Dockerfile: install Deno in the runtime image (official installer,
auto-detects amd64/arm64, build fails early via `deno --version` if it
breaks; unzip added for the installer) and build with the yt-dlp NIGHTLY
channel (`pip install -U --pre "yt-dlp[default]"`) on top of requirements
- core/youtube_client.py: one-time startup warning when deno isn't on PATH,
naming the exact failure it causes and the install command — instead of
letting users debug a cryptic yt-dlp format error
- requirements.txt: annotate the yt-dlp line with the stable-lag caveat, the
nightly upgrade command, and the Deno requirement
- README: Deno + nightly notes in Prerequisites and the Python (No Docker)
install section; Docker bundles both automatically
The tile's liveness was wired to sync:progress / discovery:progress — both
ROOM-scoped (only clients watching a specific playlist receive them), so the
dashboard tile would basically never light. And the scheduled auto-sync runs
as an automation, reporting on automation:progress — the wrong tile.
The 1s sync emitter now also sends an UNSCOPED sync:active heartbeat while any
playlist work is running anywhere: manual per-playlist syncs (sync_states),
the UI-triggered mirrored pipeline (playlist_pipeline_progress_states), and
scheduled auto-sync pipelines (running automations whose action_type is
playlist_pipeline / sync_playlist / refresh_mirrored). Emitted only while
active; the tile's 6s freshness decay handles the off. The dashboard listens
for the heartbeat alongside the (kept) room-scoped signals.
The three bento tiles had signature background animations that were pure
decoration. Each now SURGES while its subsystem is actually working, driven by
the live socket events — idle keeps the exact calm look they always had:
- Auto-Sync: the EQ bars dance fast + brighter, the playhead sweeps quicker
and the pulse dot races while a sync/discovery pipeline is running
(sync:progress / discovery:progress)
- Tools: the gear spins up 4x and brightens while a tool, scan, db-update or
repair job is running (tool:* / scan:media / repair:progress, with a shape-
tolerant "actually running" check so the 1s idle pushes don't light it)
- Automations: the flow nodes + line signals pulse at 2.5x while an automation
is firing (automation:progress)
Tiles carry .is-live while the last matching event is <6s old; a 2s interval
handles decay (no rAF, no per-frame JS).
GPU pass on the same tiles, same visuals:
- hero playhead animated `left` (layout + paint every frame, 9s loop) -> a
full-width strip whose 1.5px line is a static background, transform-only
- flow-node pulse animated background + box-shadow x3 nodes -> bright state
painted once on a pseudo, opacity breathes; added to reduced-motion kills
Audit of every dashboard animation. Already good and untouched: orb canvas
(cached glow sprites, no shadowBlur, stops on tab-hide/page-switch/scroll),
shimmer scan, sidebar orbs, embers, rl-blink (all transform/opacity), and the
reduce-effects global kill-switch. The offenders were infinite animations of
paint-bound properties — each repaints its region every frame, forever:
- avatar halo: animated box-shadow on every active bar -> the bright state is
painted once on a wrap pseudo and only its OPACITY breathes (the wrap exists
because the avatar clips overflow)
- rate-limited warn: animated filter:brightness -> a white-wash pseudo whose
opacity breathes
- active-fill glow: animated box-shadow -> static glow at the old midpoint,
breathing moved to the tip's opacity
- header sweep: animated background-position across the full-width band (on
all four headers sharing the class) -> a real child strip translated inside
an overflow-clipped wrap; transform+opacity, zero paint
- orb canvas: renders at ~20fps while fully asleep (drift is at crawl speed —
invisible) instead of 60fps for the hours the dashboard sits idle
Visual parity throughout; peak-flash (event-driven, 0.65s one-shot) keeps its
box-shadow since its duty cycle is negligible.
1-3 tiny accent sparks per socket update drift up from each bar's fill tip,
scaled by the real (unclamped) rate — motion strictly means API calls are
happening right now. Self-removing DOM nodes with a per-bar live cap of 6,
suppressed during cooldown and under reduced-effects mode. The taste-risk one
of the set: revert this commit alone if it reads as noise.
The payload has carried daily_budget {used, limit, exhausted} forever and the
dashboard rendered none of it. The avatar disc now wears a conic progress rim
that fills as the day's real-API budget is spent — green to 70%, amber to 95%,
red after — and flips purple once the worker has bridged to Spotify Free for
the rest of the day (using_free now included in the emit payload). Tooltip
carries the exact used/limit numbers.
A banned service used to just tint red. The payload carries the seconds
remaining (rl_remaining), so the bar now locks into a cooldown state: the live
VU dims, a red column drains away as the ban ticks down (largest remaining
seen is latched as the denominator — only remaining is sent), and an m:ss
timer counts to recovery. The moment the ban expires the track flashes green
('recovered') and the VU takes the stage back. 'Back in 4:00', not
'something's red'.
A thin accent marker sticks at each bar's recent maximum, holds ~1.2s, then
falls a few percent per update until it rests on the live fill — exactly how a
hardware VU meter's peak LED behaves. A traffic burst stays readable for a few
seconds after it's over instead of vanishing with the next 1s sample. Hidden
while it sits on the fill so idle bars don't carry a stray line.
The inactive->active transition (caught at the existing 30-frame state
refresh) now jolts the orb: a random-direction velocity kick with a briefly
lifted speed cap so it actually darts, a fast shallow size wobble (~1s decay),
and three sparks. A worker waking up reads as 'it sprang into action' instead
of just getting brighter.
Zero active workers + no pulses in flight for 75s eases the whole header into
a drowse (~4s): nucleus dims to embers, logo and orbs fade ~35-45%, spokes and
links fade ~70%, drift slows to a quarter speed (velocities keep integrating
so motion stays continuous). The first sign of work wakes it in ~0.3s with
three staggered rings blooming out of the nucleus. Idle and busy finally look
DIFFERENT — the contrast is what makes activity read as alive.
Each inbound pulse draws 3 fading ghost positions behind its head. The path is
parametric (eased t), so the tail is the same easing evaluated slightly in the
past — no position history, and it naturally stretches as the pulse accelerates
into the nucleus. Makes the energy flow legible at a glance.
A telemetry pulse used to just vanish on arrival. Now it makes contact: a small
expanding ring at the rim where it hits (pulse's own color, ~0.5s fade) plus
two debris sparks splashing back along the approach direction — the nucleus
reads as absorbing the work instead of deleting it. Capped pool (24), reset
alongside sparks/inflows on page switches and collapses.
Each worker orb gets a slow z-oscillation (-1 back .. +1 front); orbs draw in
painter's order with the hub pinned at z=0, so the cluster visibly swings
behind and in front of the nucleus instead of drifting on a flat plane. Depth
scales size ±18% and dims the back arc; the physics stay 2D. Effect eases out
during the hover-expand morph so orbs land on their buttons at natural size.
On page load every orb sat at the canvas origin: orbs are created at (0,0) and
the random scatter skipped orbs that weren't visible yet — which on load is all
of them, since the header hasn't been laid out when init() runs. The whole
cluster then drifted in from the top-left corner.
scatterOrbs() -> centerOrbs(): spawn the cluster dead-center of the canvas,
positioning every orb regardless of visibility (a few px of jitter on purpose —
the separation force ignores pairs closer than 0.1px, so a perfect stack would
never split). enterOrbState() also re-centers right after resizeCanvas(), so
activations get the true center even when init ran against a hidden 0x0 header
— and returning to the dashboard replays the center-bloom intro.
"Liked Songs" isn't a real Spotify playlist — no playlist URI exists for it;
the web UI invents the virtual id 'spotify:liked-songs' and Spotify serves the
collection via the saved-tracks endpoint. The playlist DETAIL endpoint special-
cases that id, but the mirrored refresh path resolves stored ids through
get_playlist_by_id, which fed the virtual id straight into sp.playlist() ->
"http status: 400 ... Unsupported URL / URI" on every sync cycle, silently.
get_playlist_by_id now special-cases the virtual id at the client seam (every
by-id resolver benefits, not just the mirror adapter): it builds the Playlist
from the existing get_saved_tracks() pagination, with the real owner name and
track count. New LIKED_SONGS_PLAYLIST_ID constant owns the magic string.
Safety: get_saved_tracks swallows fetch errors into [] — indistinguishable
from "no likes" — and the virtual playlist is only ever offered when likes
exist. An empty result therefore resolves as a FAILED refresh (None) instead
of a valid-looking empty playlist a mirror sync might propagate by clearing
the server-side copy.
Tests: virtual id resolves from saved tracks and never touches the playlist
endpoint, real ids still do (regression), the mirrored adapter seam returns a
full PlaylistDetail, and empty saved-tracks -> None. 473 passed across the
playlist/mirror/spotify families.
On path-mapped setups (Docker mounts etc.) the scan checked a bare
os.path.isfile() on the raw DB path — false for EVERY track — while the apply
handler resolves container/host mismatches. With a cover-art mode set, the
cover_action kept the album past the "anything to do?" gate, so every album
produced a finding with an empty tracks list whose apply could only ever fail
with "No tracks to re-tag in finding".
- the scan now resolves each track path with the same resolver the apply
handler uses (resolve_library_file_path) before reachability checks and
current-tag reads; plans carry the resolved path
- a finding can never be created with zero tracks — cover-action albums with
no usable tracks are skipped, with a debug log of why (unreachable/unmatched
counts) and the counts surfaced in the finding description
- unmatched-but-reachable tracks now get an art-only plan (empty db_data) so
album cover art covers ALL the album's files, not just source-matched ones —
apply_track_plans already treats empty db_data as a pure cover embed and
counts a failed cover download as skipped, never failed (now locked by tests)
- cover-only findings are titled "(cover art, N track(s))" instead of the
puzzling "(0 track(s))"
Tests: +5 (mapped paths resolve into plans, cover-with-nothing-reachable
creates no finding, unmatched -> art-only plan, art-only plan embeds cover,
failed cover download -> skipped). 87 passed across retag/repair/tag_writer.
Since the per-listener stream sessions refactor (Phase 3b), every browser gets
its own stream session — but the 1s 'tool:stream' socket broadcast still read
the legacy GLOBAL state (the DEFAULT session no real browser uses), so it told
every client "stopped" forever. The frontend skipped HTTP polling whenever the
WebSocket was up, so it only ever saw that wrong broadcast: the backend prep
downloaded the track, moved it into the session's stream folder and sat at
"ready" while the mini player showed nothing. Proxy users whose WebSockets
don't connect fell back to HTTP polling (session-correct) and streamed fine —
which is why this hid so well.
Fix: stream status is inherently per-listener, so stop pretending a global
broadcast can carry it —
- web_server.py: remove the 'tool:stream' emit from the tool-progress loop
(the broadcast thread has no request context; it can only ever see DEFAULT)
- media-player.js: the status poller always polls /api/stream/status (resolves
the caller's own session from the cookie); drop the dead broadcast handler
- core.js: unwire the 'tool:stream' socket listener
Observability fix that made this undebuggable: core/streaming/prepare.py used
getLogger(__name__) — outside the soulsync.* namespace where handlers attach —
so every prep log line (including failures) vanished from app.log. Moved to
get_logger("streaming.prepare") + a regression test locking the namespace.
34 streaming tests pass; ruff clean; web_server compiles; JS syntax-checked.
A single enriched against the deluxe gets every source ID pointing at the deluxe,
so the organizer filed it as e.g. track 2 of a 10-track album. Root cause: the
canonical resolver only ever scored the editions already linked — the correct
single was never even a candidate, and the misfit deluxe scored so low (0.1,
below the 0.5 floor) that nothing got pinned and the priority-walk grabbed the
deluxe anyway.
Fix, in three tested layers:
- resolve_canonical_for_album gains a fetch_alternates seam: when no linked
edition clears the floor, it scores the source's OTHER editions of the same
release and re-picks by best fit (dedup, injected, pure).
- default_fetch_alternates lists the artist's editions and keeps the same-release
ones (edition-blind name match: Deluxe / - Single / [Remastered] all collapse),
returning their tracklists. Favors recall; the scorer is the precision gate.
- _resolve_source does the misfit check inline: it fit-scores the walked edition
and only on a clear misfit searches for a better edition, then persists the pin
on apply (Track Number Repair + future runs agree). Cost-neutral and behavior-
identical for well-fitting albums (no extra API calls); strict_source and the
#758 manual lock are never overridden.
Tests: +4 resolver (expand/no-expand/dedupe/back-compat), +7 alternates (name
matcher + fetcher over fake APIs + cap), +3 organizer end-to-end (misfit->single
+pin, well-fit->no-expand, strict->no-expand). 300 passed across the reorganize
+ canonical family, lint clean.
User report: "Not authenticated with Spotify" daily + workers paused. The
log was misleading — is_spotify_authenticated() returns False for five
distinct reasons (no creds / rate-limit ban / post-ban cooldown / no
cached token / probe failure), and all five API call sites logged the
same bare "Not authenticated with Spotify". So a routine rate-limit ban
read as a logout, and the real cause (logged only at DEBUG) was invisible.
New pure describe_spotify_unavailable() maps the real state to a clear
message (priority matches is_spotify_authenticated): not-configured →
rate-limited ("ban ~Nm left (not a logout)") → post-ban cooldown →
no-token ("not connected — re-authenticate") → "auth check failed (token
refresh may have failed)". A side-effect-free client method
(_auth_unavailable_reason) gathers the live state (reads the cached token,
no API probe) and the 5 sites log it.
Now a daily ban is identifiable as a ban, and a genuine logout is
identifiable as one — so reports like this are diagnosable from the log
alone. 7 tests pin the priority/messaging. Full suite clean.
User request: the re-tag diff card shows old→new metadata per track but not
the file it applies to, so a wrong match is hard to spot before applying.
The finding already carries each track's file_path (details.tracks[].file_path
from the scan) — the renderer just wasn't showing it. Now each changed track
displays its filename beneath the title, so you can eyeball that the metadata
about to be written actually belongs to that file. Skipped when the label is
already the filename (track had no title). Pure frontend display change.
