Regression in 2.6.9. The spotify_public source adapter (used by auto-sync /
refresh_mirrored) called scrape_spotify_embed() directly — the embed widget only
exposes ~100 tracks — instead of fetch_spotify_public(), the wrapper the rest of
the app uses, which pulls the full list via the paginated public API and only
falls back to the embed on failure. So initial discovery got the whole playlist
but every auto-sync re-fetch truncated it to 100.
Switched the adapter to fetch_spotify_public (same return shape — drop-in). Albums
still resolve via the embed (already whole); on any failure it falls back to the
embed exactly as before.
Test: the adapter returns all 150 tracks when the full fetch yields 150 (was
capped at 100). 29 adapter tests pass.
Pipeline-driven Auto-Sync runs against any ListenBrainz playlist
(Weekly Jams, Weekly Exploration, Top Discoveries, etc.) would sit
on ``Refreshing: "<name>"`` with no UI updates for 5-7 minutes
before the pipeline progressed. Two real bugs stacked:
1. **Double discovery.** The refresh handler called
``_maybe_discover`` (matching engine, per-track Spotify/iTunes/
Deezer matches) inline for any source returning
``needs_discovery=True`` tracks. Phase 2 of the pipeline then
ran the SAME matching engine via ``run_playlist_discovery_worker``
on the same tracks. The refresh-side run blocked the loop with
zero progress emission; Phase 2's already has the timed
progress-poll pattern. So LB tracks discovered twice, the first
time silently.
Pipeline now sets ``skip_discovery=True`` on its refresh config.
The handler honors the flag and lets Phase 2 handle discovery
end-to-end. Standalone callers (Sync-page tab, registration
action) leave the flag unset so they still get matched_data
on refresh.
2. **No targeted LB refresh.** The LB adapter's ``refresh_playlist``
called ``manager.update_all_playlists()`` — the only refresh
entry-point the manager exposed — which re-pulls every cached
LB playlist's details from the API (~12+ round-trips) even
when only one playlist needed refreshing. Wasteful;
tax-on-everyone for one-playlist work.
Added ``LBManager.refresh_playlist(mbid)`` — reads the cached
playlist_type, fetches just that playlist's details, runs the
normal ``_update_playlist`` upsert path. Defaults type to
``user`` for un-cached mbids so new-playlist discovery still
works. Skips ``_cleanup_old_playlists`` and
``_ensure_rolling_mirrors_from_cache`` (wasted work for a
single-playlist refresh).
Also: killed a silent ``except Exception: pass`` in the LB
adapter's old refresh wrapper that was masking every LB API
failure as a stale-cache hit. Refresh errors now log with full
traceback at warning level and propagate ``None`` so the outer
handler at ``refresh_mirrored.py:104`` counts the error and
surfaces it to the run-history error tally.
Pinned with 12 new unit tests across:
- ``tests/test_listenbrainz_manager.py`` (8): targeted refresh
happy path, unauthenticated guard, empty-mbid guard, upstream
``None`` return, default playlist_type for unknown mbid,
exception propagation, cost guard skipping cleanup, skipped-
when-unchanged signal
- ``tests/test_playlist_sources_adapters.py`` (3): adapter uses
targeted call (not legacy), adapter returns ``None`` on manager
error (not silent swallow), adapter resolves synthetic series
ids before calling the manager
- ``tests/automation/test_handlers_playlist.py`` (1):
skip_discovery flag bypasses ``_maybe_discover`` end-to-end
Adds ``discover_tracks(tracks) -> List[NormalizedTrack]`` to the
PlaylistSource interface. Sources whose tracks already carry
provider IDs (Spotify, Tidal, Qobuz, YouTube, Deezer, Spotify
public, iTunes link, SoulSync Discovery) inherit a no-op default;
ListenBrainz + Last.fm override to run the matching engine.
This closes the last gap before LB / Last.fm / SoulSync Discovery
can land as Sync-page mirror sources: the refresh handler now
calls ``source.discover_tracks(...)`` whenever a source returns
tracks with ``needs_discovery=True``, so mirrored LB rows arrive
already discovered + ready for the sync pipeline. Previously, LB
playlists ran through a separate state-machine worker tied to the
Discover-page UI, with results stored in ``discovery_cache``
instead of ``mirrored_playlist_tracks.extra_data``.
Changes:
- ``core/playlists/sources/base.py`` — PlaylistSource switches from
Protocol to ABC so a concrete default for ``discover_tracks``
can live on the base class. The four real-work methods stay
``@abstractmethod``; instantiating an adapter that forgets one
fails loudly at construction.
- ``core/discovery/matching.py`` (new) — pure ``match_mb_tracks``
helper that runs Strategy-1-only matching-engine queries against
Spotify (primary) or iTunes (fallback). No state machine, no
discovery-cache writes, no wing-it stub — that richer flow stays
in ``core/discovery/listenbrainz.py`` for the Discover-page UI.
- ``ListenBrainzPlaylistSource`` + ``LastFMPlaylistSource`` take
an optional ``discover_callable`` constructor arg. Last.fm reuses
the LB implementation since the track shape is identical.
- ``bootstrap.build_playlist_source_registry`` accepts a
``discover_callable`` kwarg and wires it into LB + Last.fm
adapters.
- ``web_server.py`` boot constructs the discovery callable from the
existing matching engine + ``_discovery_score_candidates`` +
Spotify / iTunes clients, passes through to the registry.
- ``refresh_mirrored.py`` adds a small ``_maybe_discover`` helper
that calls ``source.discover_tracks(...)`` between fetch and
``to_mirror_track_dict`` projection — only fires when at least
one track has ``needs_discovery=True``, so the normal Spotify /
Tidal / etc. refresh path stays a zero-cost pass-through.
Tests:
- 5 new adapter tests: default no-op pass-through, LB discovery
with mixed matches/misses, LB no-callable fallback, Last.fm
shares the LB implementation, mirror-dict spotify_hint emit.
- 1 new automation test: end-to-end LB refresh with a stub
discover_callable proves the matched_data lands in
``mirror_playlist_tracks.extra_data`` after the registry
refresh + discover hop.
225 tests across adapter + automation suites green.
Phase 1a of the Discover-to-Sync unification. The mirrored-playlist
refresh handler used to branch per-source through a ~190-line
if/elif chain (Spotify, Spotify public, Deezer, Tidal, YouTube).
Each branch hand-built its own ``extra_data`` JSON for the matched-
data block. With every new source we considered for Sync-page mirror
support (ListenBrainz, Last.fm radio, SoulSync Discovery, iTunes
link), that chain would have grown a new elif.
This commit lifts the per-source logic into the existing adapter
layer and collapses the dispatch to a registry lookup:
- ``core/playlists/sources/deezer.py`` — new adapter so the registry
covers every source the refresh handler previously branched on.
- ``core/playlists/sources/bootstrap.py`` — single helper that builds
a populated registry from injected getter callables. Both
``web_server.py`` boot and the automation test fixtures call it,
so the two construction paths can't drift.
- ``core/playlists/sources/base.py`` — ``to_mirror_track_dict``
projection helper centralises the NormalizedTrack → DB-row
conversion (including the discovered/matched_data and
spotify_hint extra_data shapes the downstream sync + wishlist
consumers already expect).
- Spotify adapter now populates ``extra['discovered']`` + an
``extra['matched_data']`` block when fetching via the authed API,
so Spotify mirrors keep landing pre-discovered (matches the
pre-refactor contract pinned by
``test_spotify_refresh_writes_to_db``).
- Spotify-public adapter populates ``extra['spotify_hint']`` so the
discovery worker can skip its search step and jump straight to
enrichment for the known track ID.
- All artist-name fields now project to first-artist-only across
every adapter — matches the pre-refactor mirror_playlist DB shape
(``t.artists[0]``).
``refresh_mirrored.py`` shrinks ~190 → ~80 lines and keeps:
- the file/beatport unrefreshable-source filter,
- URL extraction from ``description`` via ``require_refresh_url``
for spotify_public + youtube,
- the Spotify-public → authed-Spotify fallback when the user is
signed in (handler-level branch, not in any adapter),
- the Tidal-not-authenticated soft-skip log (skip, not error),
- existing-extra_data preservation across refreshes,
- the ``playlist_changed`` automation event emit on track-set delta.
Test scaffolding:
- ``_build_deps`` in ``tests/automation/test_handlers_playlist.py``
now builds a default registry from the passed clients via
``build_playlist_source_registry``, so existing refresh tests
exercise the same path without per-test changes. New tests cover
Tidal-not-authed soft-skip, Deezer refresh writes plain tracks,
YouTube refresh reads URL from description, and Spotify-public
uses authed Spotify when signed in.
- 4 new adapter tests for Deezer projection +
``to_mirror_track_dict`` (minimal track, Spotify matched_data,
Spotify-public spotify_hint).
- ``playlist_source_registry`` field on ``AutomationDeps`` defaults
to ``None`` so the other 5 automation test files (which don't
exercise refresh_mirrored) keep working unchanged.
220 tests across automation + adapter suites green.
Groundwork for unifying Discover-page playlists (ListenBrainz, Last.fm
radio, SoulSync Discovery) with Sync-page playlists (Spotify, Tidal,
Qobuz, YouTube, Spotify public, iTunes link). All nine sources now
expose the same `PlaylistSource` Protocol so callers stop having to
branch per-source.
This commit only adds the abstraction — no dispatch sites collapse to
the registry yet, no DB or UI changes. Adapters wrap existing clients
via injected getter callables to avoid eager imports of web_server.py
globals.
- core/playlists/sources/base.py — PlaylistMeta, NormalizedTrack,
PlaylistDetail dataclasses + PlaylistSource Protocol with
supports_listing / supports_refresh / requires_auth capability
flags. needs_discovery flag on NormalizedTrack marks tracks that
carry raw MB metadata (LB, Last.fm) vs tracks already matched to a
provider ID (everything else).
- core/playlists/sources/registry.py — thread-safe lazy-factory
registry with instance caching + re-register invalidation.
- nine adapters in core/playlists/sources/ wrapping SpotifyClient,
TidalClient, QobuzClient, spotify_public_scraper, the YouTube +
iTunes-link parsers (via injected callables), ListenBrainzManager,
Last.fm radio rows in the ListenBrainz cache, and
PersonalizedPlaylistManager.
- tests/test_playlist_sources_adapters.py — 18 tests covering each
adapter's field projection with fake backing clients, plus
registry lazy-construct + cache + re-register invalidation.
Phase 1 will collapse refresh_mirrored.py's per-source if/elif chain
to a registry lookup and surface ListenBrainz as a Sync-page tab.
BoulderBadgeDad