Personalized pipeline: auto-refresh stale snapshots after watchlist scan

Snapshots now track when their source data changes. Watchlist scan
emits stale flags on the playlists whose underlying pool just got
refreshed; the next pipeline run sees the flag and regenerates the
snapshot before syncing, so the server playlist never lags the source.

Schema:
- new `is_stale INTEGER NOT NULL DEFAULT 0` column on
  `personalized_playlists`, plus an idempotent ADD COLUMN migration
  in `ensure_personalized_schema` for installs created before this PR.
- `PlaylistRecord.is_stale: bool = False` exposed on the dataclass so
  callers can branch on freshness without re-querying.

Manager:
- new `mark_kinds_stale(kinds, profile_id=None)` flips the flag in
  bulk for a list of kinds (used by upstream data refreshers).
- `_persist_snapshot` clears `is_stale = 0` on successful refresh.
- SELECT statements + `_row_to_record` updated to read the column
  (with tuple-form length guard for safety).

Pipeline:
- `_build_payloads_for_kinds` now branches: refresh_first=True OR
  `existing.is_stale` -> refresh_playlist, else read existing
  snapshot. So the auto-refresh kicks in without needing the user to
  toggle the refresh-each-run option.

Watchlist scanner emits stale flags at three sites:
- after `update_discovery_pool_timestamp` -> marks pool-fed kinds
  stale: hidden_gems, discovery_shuffle, popular_picks, time_machine,
  genre_playlist, daily_mix.
- after release_radar `save_curated_playlist` -> marks `fresh_tape`.
- after discovery_weekly `save_curated_playlist` -> marks `archives`.

All three calls go through a module-level `_mark_personalized_kinds_stale`
helper that builds a PersonalizedPlaylistManager with `deps=None` (only
DB access is needed for the flag update — no generator dispatch). Each
call is wrapped in try/except so a flag failure can never abort the
scan itself.

Tests:
- new `TestStaleFlag` class in `test_personalized_manager.py` (6
  tests): default-false, single-kind flip, multi-kind, profile
  scoping, refresh-clears, empty-list noop.
- two new pipeline tests pin the auto-refresh dispatch:
  `test_stale_snapshot_auto_refreshes_even_without_refresh_first`
  and `test_non_stale_snapshot_skips_refresh`.
- existing stub-manager `SimpleNamespace` returns gained
  `is_stale=False` so the new attribute read doesn't AttributeError.

Full suite: 3391 pass.

User-facing WHATS_NEW entry added under 2.5.2 (above the prior
pipeline auto-sync entry) describing the auto-refresh behavior.
This commit is contained in:
Broque Thomas 2026-05-15 20:53:03 -07:00
parent 66390e685a
commit 877d0e7d81
8 changed files with 244 additions and 6 deletions

View file

@ -196,10 +196,19 @@ def _build_payloads_for_kinds(
continue continue
try: try:
# Determine whether to refresh: explicit user flag, OR the
# snapshot is marked stale because the underlying source
# data (discovery_pool, curated_playlists) changed since
# last generation. Either way → refresh; otherwise just
# read the existing snapshot.
if refresh_first: if refresh_first:
record = manager.refresh_playlist(kind, variant, profile_id) record = manager.refresh_playlist(kind, variant, profile_id)
else: else:
record = manager.ensure_playlist(kind, variant, profile_id) existing = manager.ensure_playlist(kind, variant, profile_id)
if existing.is_stale:
record = manager.refresh_playlist(kind, variant, profile_id)
else:
record = existing
except Exception as exc: # noqa: BLE001 — log + continue with next kind except Exception as exc: # noqa: BLE001 — log + continue with next kind
deps.update_progress( deps.update_progress(
automation_id, automation_id,

View file

@ -102,7 +102,8 @@ class PersonalizedPlaylistManager:
""" """
SELECT id, profile_id, kind, variant, name, config_json, SELECT id, profile_id, kind, variant, name, config_json,
track_count, last_generated_at, last_synced_at, track_count, last_generated_at, last_synced_at,
last_generation_source, last_generation_error last_generation_source, last_generation_error,
is_stale
FROM personalized_playlists FROM personalized_playlists
WHERE profile_id = ? WHERE profile_id = ?
ORDER BY COALESCE(last_generated_at, created_at) DESC ORDER BY COALESCE(last_generated_at, created_at) DESC
@ -231,6 +232,34 @@ class PersonalizedPlaylistManager:
rows = cursor.fetchall() rows = cursor.fetchall()
return [self._row_to_track(r) for r in rows] return [self._row_to_track(r) for r in rows]
# ─── snapshot freshness vs source data ───────────────────────────
def mark_kinds_stale(self, kinds: List[str], profile_id: Optional[int] = None) -> int:
"""Flag every playlist row matching one of ``kinds`` as stale.
Called by upstream data refreshers (watchlist scan finishing
/ Spotify enrichment worker re-pulling Release Radar / etc)
so pipelines auto-regenerate snapshots before the next sync
instead of pushing stale data to the media server.
Returns the number of rows touched. When ``profile_id`` is
None, flags rows across every profile.
"""
if not kinds:
return 0
placeholders = ','.join('?' * len(kinds))
sql = f"UPDATE personalized_playlists SET is_stale = 1, updated_at = CURRENT_TIMESTAMP WHERE kind IN ({placeholders})"
params: List[Any] = list(kinds)
if profile_id is not None:
sql += " AND profile_id = ?"
params.append(profile_id)
with self.database._get_connection() as conn:
cursor = conn.cursor()
cursor.execute(sql, params)
count = cursor.rowcount
conn.commit()
return count
# ─── staleness history ─────────────────────────────────────────── # ─── staleness history ───────────────────────────────────────────
def recent_track_ids(self, profile_id: int, kind: str, days: int) -> List[str]: def recent_track_ids(self, profile_id: int, kind: str, days: int) -> List[str]:
@ -294,6 +323,7 @@ class PersonalizedPlaylistManager:
UPDATE personalized_playlists UPDATE personalized_playlists
SET track_count = ?, last_generated_at = CURRENT_TIMESTAMP, SET track_count = ?, last_generated_at = CURRENT_TIMESTAMP,
last_generation_source = ?, last_generation_error = NULL, last_generation_source = ?, last_generation_error = NULL,
is_stale = 0,
updated_at = CURRENT_TIMESTAMP updated_at = CURRENT_TIMESTAMP
WHERE id = ? WHERE id = ?
""", """,
@ -346,7 +376,8 @@ class PersonalizedPlaylistManager:
""" """
SELECT id, profile_id, kind, variant, name, config_json, SELECT id, profile_id, kind, variant, name, config_json,
track_count, last_generated_at, last_synced_at, track_count, last_generated_at, last_synced_at,
last_generation_source, last_generation_error last_generation_source, last_generation_error,
is_stale
FROM personalized_playlists FROM personalized_playlists
WHERE profile_id = ? AND kind = ? AND variant = ? WHERE profile_id = ? AND kind = ? AND variant = ?
""", """,
@ -362,7 +393,8 @@ class PersonalizedPlaylistManager:
""" """
SELECT id, profile_id, kind, variant, name, config_json, SELECT id, profile_id, kind, variant, name, config_json,
track_count, last_generated_at, last_synced_at, track_count, last_generated_at, last_synced_at,
last_generation_source, last_generation_error last_generation_source, last_generation_error,
is_stale
FROM personalized_playlists FROM personalized_playlists
WHERE id = ? WHERE id = ?
""", """,
@ -386,6 +418,7 @@ class PersonalizedPlaylistManager:
last_synced_at=row.get('last_synced_at'), last_synced_at=row.get('last_synced_at'),
last_generation_source=row.get('last_generation_source'), last_generation_source=row.get('last_generation_source'),
last_generation_error=row.get('last_generation_error'), last_generation_error=row.get('last_generation_error'),
is_stale=bool(row.get('is_stale') or 0),
) )
# Tuple form: positional access matches SELECT order above. # Tuple form: positional access matches SELECT order above.
return PlaylistRecord( return PlaylistRecord(
@ -398,6 +431,7 @@ class PersonalizedPlaylistManager:
last_synced_at=row[8], last_synced_at=row[8],
last_generation_source=row[9], last_generation_source=row[9],
last_generation_error=row[10], last_generation_error=row[10],
is_stale=bool(row[11] or 0) if len(row) > 11 else False,
) )
@staticmethod @staticmethod

View file

@ -152,7 +152,13 @@ class PlaylistRecord:
The live track list is fetched separately via The live track list is fetched separately via
``PersonalizedPlaylistManager.get_playlist_tracks(playlist_id)`` ``PersonalizedPlaylistManager.get_playlist_tracks(playlist_id)``
so list / detail responses can stay cheap when the caller only so list / detail responses can stay cheap when the caller only
needs metadata.""" needs metadata.
``is_stale`` flips to True when the underlying source data
changes (e.g. watchlist scan updates the discovery pool) and is
cleared on the next successful refresh. Pipelines auto-refresh
stale snapshots before syncing so the server playlist always
reflects the latest source data."""
id: int id: int
profile_id: int profile_id: int
@ -165,6 +171,7 @@ class PlaylistRecord:
last_synced_at: Optional[str] last_synced_at: Optional[str]
last_generation_source: Optional[str] last_generation_source: Optional[str]
last_generation_error: Optional[str] last_generation_error: Optional[str]
is_stale: bool = False
__all__ = [ __all__ = [

View file

@ -24,6 +24,19 @@ from core.wishlist_service import get_wishlist_service
from core.matching_engine import MusicMatchingEngine from core.matching_engine import MusicMatchingEngine
from utils.logging_config import get_logger from utils.logging_config import get_logger
def _mark_personalized_kinds_stale(database, kinds, profile_id=1):
"""Module-level helper so the inline call sites stay tiny.
Constructs a PersonalizedPlaylistManager with the minimal deps
needed for `mark_kinds_stale` (database access only no generator
dispatch required) and flips the is_stale flag for matching rows.
Best-effort: any exception is swallowed by the caller's try/except
since stale-flagging is non-critical for the scan itself."""
from core.personalized.manager import PersonalizedPlaylistManager
mgr = PersonalizedPlaylistManager(database, deps=None)
return mgr.mark_kinds_stale(list(kinds), profile_id=profile_id)
logger = get_logger("watchlist_scanner") logger = get_logger("watchlist_scanner")
# Rate limiting constants for watchlist operations # Rate limiting constants for watchlist operations
@ -2980,6 +2993,22 @@ class WatchlistScanner:
self.database.update_discovery_pool_timestamp(track_count=final_count, profile_id=profile_id) self.database.update_discovery_pool_timestamp(track_count=final_count, profile_id=profile_id)
logger.info(f"Discovery pool now contains {final_count} total tracks (built over time)") logger.info(f"Discovery pool now contains {final_count} total tracks (built over time)")
# Mark every personalized-playlist kind that draws from the
# discovery pool as stale so the playlist pipeline auto-
# regenerates snapshots on its next run. Without this the
# server playlists stay frozen even though the source pool
# just got fresh tracks. Best-effort — pool refresh succeeds
# even if the manager isn't wired (no personalized tables).
try:
_mark_personalized_kinds_stale(
self.database,
kinds=['hidden_gems', 'discovery_shuffle', 'popular_picks',
'time_machine', 'genre_playlist', 'daily_mix'],
profile_id=profile_id,
)
except Exception as e: # noqa: BLE001 — never abort scan for staleness flag
logger.debug("Failed to mark personalized kinds stale: %s", e)
# Cache recent albums for discovery page # Cache recent albums for discovery page
logger.info("Caching recent albums for discovery page...") logger.info("Caching recent albums for discovery page...")
if progress_callback: if progress_callback:
@ -3660,6 +3689,14 @@ class WatchlistScanner:
playlist_key = f'release_radar_{source}' playlist_key = f'release_radar_{source}'
self.database.save_curated_playlist(playlist_key, release_radar_tracks, profile_id=profile_id) self.database.save_curated_playlist(playlist_key, release_radar_tracks, profile_id=profile_id)
logger.info(f"Release Radar ({source}) curated: {len(release_radar_tracks)} tracks") logger.info(f"Release Radar ({source}) curated: {len(release_radar_tracks)} tracks")
# Flag personalized Fresh Tape snapshot as stale so the
# pipeline auto-regenerates it on the next run.
try:
_mark_personalized_kinds_stale(
self.database, kinds=['fresh_tape'], profile_id=profile_id,
)
except Exception as e: # noqa: BLE001
logger.debug("Fresh Tape stale-flag failed: %s", e)
# 2. Curate Discovery Weekly - 50 tracks from discovery pool # 2. Curate Discovery Weekly - 50 tracks from discovery pool
logger.info(f"Curating Discovery Weekly for {source}...") logger.info(f"Curating Discovery Weekly for {source}...")
@ -3735,6 +3772,12 @@ class WatchlistScanner:
playlist_key = f'discovery_weekly_{source}' playlist_key = f'discovery_weekly_{source}'
self.database.save_curated_playlist(playlist_key, discovery_weekly_tracks, profile_id=profile_id) self.database.save_curated_playlist(playlist_key, discovery_weekly_tracks, profile_id=profile_id)
logger.info(f"Discovery Weekly ({source}) curated: {len(discovery_weekly_tracks)} tracks") logger.info(f"Discovery Weekly ({source}) curated: {len(discovery_weekly_tracks)} tracks")
try:
_mark_personalized_kinds_stale(
self.database, kinds=['archives'], profile_id=profile_id,
)
except Exception as e: # noqa: BLE001
logger.debug("Archives stale-flag failed: %s", e)
# 3. "Because You Listen To" — personalized sections based on top played artists # 3. "Because You Listen To" — personalized sections based on top played artists
if profile['has_data']: if profile['has_data']:

View file

@ -66,12 +66,18 @@ CREATE TABLE IF NOT EXISTS personalized_playlists (
last_synced_at TIMESTAMP, last_synced_at TIMESTAMP,
last_generation_source TEXT, last_generation_source TEXT,
last_generation_error TEXT, last_generation_error TEXT,
is_stale INTEGER NOT NULL DEFAULT 0,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
UNIQUE (profile_id, kind, variant) UNIQUE (profile_id, kind, variant)
) )
""" """
# Migration for installs that created the table before is_stale existed.
PERSONALIZED_PLAYLISTS_STALE_MIGRATION = """
ALTER TABLE personalized_playlists ADD COLUMN is_stale INTEGER NOT NULL DEFAULT 0
"""
PERSONALIZED_PLAYLIST_TRACKS_DDL = """ PERSONALIZED_PLAYLIST_TRACKS_DDL = """
CREATE TABLE IF NOT EXISTS personalized_playlist_tracks ( CREATE TABLE IF NOT EXISTS personalized_playlist_tracks (
id INTEGER PRIMARY KEY AUTOINCREMENT, id INTEGER PRIMARY KEY AUTOINCREMENT,
@ -126,6 +132,20 @@ def ensure_personalized_schema(connection: Any) -> None:
cursor.execute(PERSONALIZED_PLAYLIST_TRACKS_INDEX) cursor.execute(PERSONALIZED_PLAYLIST_TRACKS_INDEX)
cursor.execute(PERSONALIZED_TRACK_HISTORY_DDL) cursor.execute(PERSONALIZED_TRACK_HISTORY_DDL)
cursor.execute(PERSONALIZED_TRACK_HISTORY_INDEX) cursor.execute(PERSONALIZED_TRACK_HISTORY_INDEX)
# Add is_stale column on installs that created the table before
# this column existed. SQLite has no `ADD COLUMN IF NOT EXISTS` so
# we probe with PRAGMA + tolerate the OperationalError that fires
# when the column is already there.
cursor.execute("PRAGMA table_info(personalized_playlists)")
cols = {row[1] for row in cursor.fetchall()}
if 'is_stale' not in cols:
try:
cursor.execute(PERSONALIZED_PLAYLISTS_STALE_MIGRATION)
logger.info("Added is_stale column to personalized_playlists")
except Exception as e:
logger.debug("is_stale column migration: %s", e)
logger.debug("Personalized-playlist schema ensured") logger.debug("Personalized-playlist schema ensured")

View file

@ -173,6 +173,7 @@ class _StubManagerWithTracks:
return SimpleNamespace( return SimpleNamespace(
id=hash((kind, variant)) % 10000, id=hash((kind, variant)) % 10000,
name=f'{kind}-{variant or "S"}', kind=kind, variant=variant, name=f'{kind}-{variant or "S"}', kind=kind, variant=variant,
is_stale=False,
) )
def refresh_playlist(self, kind, variant, profile_id): def refresh_playlist(self, kind, variant, profile_id):
@ -182,6 +183,7 @@ class _StubManagerWithTracks:
return SimpleNamespace( return SimpleNamespace(
id=hash((kind, variant)) % 10000, id=hash((kind, variant)) % 10000,
name=f'{kind}-{variant or "S"}', kind=kind, variant=variant, name=f'{kind}-{variant or "S"}', kind=kind, variant=variant,
is_stale=False,
) )
def get_playlist_tracks(self, playlist_id): def get_playlist_tracks(self, playlist_id):
@ -267,6 +269,72 @@ class TestPayloadBuilding:
assert len(p['tracks_json']) == 2 assert len(p['tracks_json']) == 2
assert p['tracks_json'][0]['id'] == 'sp-1' assert p['tracks_json'][0]['id'] == 'sp-1'
def test_stale_snapshot_auto_refreshes_even_without_refresh_first(self):
"""When the manager reports is_stale=True, the pipeline refreshes
regardless of the refresh_first config flag the source data
(discovery_pool / curated lists) changed, so the snapshot must
be regenerated before syncing or we'd push stale data."""
deps = _build_deps()
# Stub manager whose ensure_playlist returns a stale record.
# refresh_playlist should still get called.
refresh_called = []
class _StaleMgr:
def ensure_playlist(self, kind, variant, profile_id):
return SimpleNamespace(
id=1, name=kind, kind=kind, variant=variant, is_stale=True,
)
def refresh_playlist(self, kind, variant, profile_id):
refresh_called.append((kind, variant))
return SimpleNamespace(
id=1, name=kind, kind=kind, variant=variant, is_stale=False,
)
def get_playlist_tracks(self, _id):
return [SimpleNamespace(
track_name='Refreshed', artist_name='A', album_name='Al',
spotify_track_id='sp-fresh', itunes_track_id=None,
deezer_track_id=None, duration_ms=200000,
)]
payloads = _build_payloads_for_kinds(
deps, _StaleMgr(),
[{'kind': 'hidden_gems'}],
profile_id=1, automation_id=None, refresh_first=False,
)
assert refresh_called == [('hidden_gems', '')]
assert len(payloads) == 1
assert payloads[0]['tracks_json'][0]['name'] == 'Refreshed'
def test_non_stale_snapshot_skips_refresh(self):
"""When the snapshot is fresh AND refresh_first is False, just
read the existing tracks without re-running the generator."""
deps = _build_deps()
refresh_called = []
class _FreshMgr:
def ensure_playlist(self, kind, variant, profile_id):
return SimpleNamespace(
id=1, name=kind, kind=kind, variant=variant, is_stale=False,
)
def refresh_playlist(self, *_a, **_k):
refresh_called.append('called')
return SimpleNamespace(
id=1, name='x', kind='x', variant='', is_stale=False,
)
def get_playlist_tracks(self, _id):
return [SimpleNamespace(
track_name='Cached', artist_name='A', album_name='Al',
spotify_track_id='sp-1', itunes_track_id=None,
deezer_track_id=None, duration_ms=200000,
)]
_build_payloads_for_kinds(
deps, _FreshMgr(),
[{'kind': 'hidden_gems'}],
profile_id=1, automation_id=None, refresh_first=False,
)
assert refresh_called == []
def test_manager_exception_swallowed_continues_to_next(self): def test_manager_exception_swallowed_continues_to_next(self):
deps = _build_deps() deps = _build_deps()
@ -277,7 +345,7 @@ class TestPayloadBuilding:
self.calls.append(kind) self.calls.append(kind)
if kind == 'broken': if kind == 'broken':
raise RuntimeError('manager boom') raise RuntimeError('manager boom')
return SimpleNamespace(id=1, name=kind, kind=kind, variant=variant) return SimpleNamespace(id=1, name=kind, kind=kind, variant=variant, is_stale=False)
def get_playlist_tracks(self, _id): def get_playlist_tracks(self, _id):
return [] return []

View file

@ -449,6 +449,62 @@ class TestStalenessFilter:
assert r2.track_count == 1 assert r2.track_count == 1
class TestStaleFlag:
"""`is_stale` flips when upstream data changes; refresh clears it."""
def test_default_is_false(self, db, registry):
_register_simple_kind(registry, lambda *a, **k: [])
mgr = PersonalizedPlaylistManager(db, deps=None, registry=registry)
record = mgr.ensure_playlist('hidden_gems', '', 1)
assert record.is_stale is False
def test_mark_kinds_stale_flips_flag(self, db, registry):
_register_simple_kind(registry, lambda *a, **k: [], kind='hidden_gems')
_register_simple_kind(registry, lambda *a, **k: [], kind='discovery_shuffle')
mgr = PersonalizedPlaylistManager(db, deps=None, registry=registry)
mgr.ensure_playlist('hidden_gems', '', 1)
mgr.ensure_playlist('discovery_shuffle', '', 1)
n = mgr.mark_kinds_stale(['hidden_gems', 'discovery_shuffle'])
assert n == 2
assert mgr.ensure_playlist('hidden_gems', '', 1).is_stale is True
assert mgr.ensure_playlist('discovery_shuffle', '', 1).is_stale is True
def test_mark_kinds_stale_only_matching_kinds(self, db, registry):
_register_simple_kind(registry, lambda *a, **k: [], kind='hidden_gems')
_register_simple_kind(registry, lambda *a, **k: [], kind='popular_picks')
mgr = PersonalizedPlaylistManager(db, deps=None, registry=registry)
mgr.ensure_playlist('hidden_gems', '', 1)
mgr.ensure_playlist('popular_picks', '', 1)
mgr.mark_kinds_stale(['hidden_gems'])
assert mgr.ensure_playlist('hidden_gems', '', 1).is_stale is True
assert mgr.ensure_playlist('popular_picks', '', 1).is_stale is False
def test_mark_kinds_stale_scopes_to_profile(self, db, registry):
_register_simple_kind(registry, lambda *a, **k: [])
mgr = PersonalizedPlaylistManager(db, deps=None, registry=registry)
mgr.ensure_playlist('hidden_gems', '', 1)
mgr.ensure_playlist('hidden_gems', '', 2)
mgr.mark_kinds_stale(['hidden_gems'], profile_id=1)
assert mgr.ensure_playlist('hidden_gems', '', 1).is_stale is True
assert mgr.ensure_playlist('hidden_gems', '', 2).is_stale is False
def test_refresh_clears_stale_flag(self, db, registry):
_register_simple_kind(registry, lambda *a, **k: [_make_track()])
mgr = PersonalizedPlaylistManager(db, deps=None, registry=registry)
mgr.ensure_playlist('hidden_gems', '', 1)
mgr.mark_kinds_stale(['hidden_gems'])
assert mgr.ensure_playlist('hidden_gems', '', 1).is_stale is True
record = mgr.refresh_playlist('hidden_gems', '', 1)
assert record.is_stale is False
def test_mark_kinds_stale_empty_list_noop(self, db, registry):
_register_simple_kind(registry, lambda *a, **k: [])
mgr = PersonalizedPlaylistManager(db, deps=None, registry=registry)
mgr.ensure_playlist('hidden_gems', '', 1)
n = mgr.mark_kinds_stale([])
assert n == 0
class TestStalenessHistory: class TestStalenessHistory:
def test_recent_track_ids_returns_zero_when_days_zero(self, db, registry): def test_recent_track_ids_returns_zero_when_days_zero(self, db, registry):
_register_simple_kind(registry, lambda *a, **k: [_make_track(sid='spot-1')]) _register_simple_kind(registry, lambda *a, **k: [_make_track(sid='spot-1')])

View file

@ -3416,6 +3416,7 @@ const WHATS_NEW = {
'2.5.2': [ '2.5.2': [
// --- May 13, 2026 — 2.5.2 release --- // --- May 13, 2026 — 2.5.2 release ---
{ date: 'May 13, 2026 — 2.5.2 release' }, { date: 'May 13, 2026 — 2.5.2 release' },
{ title: 'Personalized Pipeline: Auto-Refresh Stale Snapshots', desc: 'follow-up polish on the personalized playlist pipeline. snapshots now know when their source data went stale. when watchlist scan finishes — refreshes the discovery pool OR re-curates Release Radar / Discovery Weekly — every playlist that draws from that data gets flagged `is_stale = 1`. next pipeline run sees the flag and auto-refreshes BEFORE syncing, so the server playlist always reflects the latest pool. no more pushing day-old Hidden Gems to plex right after the scan dropped 200 new tracks into the pool. pool-fed kinds (Hidden Gems / Discovery Shuffle / Popular Picks / Time Machine / Genre / Daily Mix) flagged after the discovery pool refresh. Fresh Tape flagged after Release Radar curates. Archives flagged after Discovery Weekly curates. flag clears on the next successful refresh. independent of the existing `refresh_first` config — that flag is now for "ALWAYS refresh, even when nothing changed" (cron use case: nightly Hidden Gems regen). idempotent schema migration adds the `is_stale` column to installs created before this PR. 12 new boundary tests pin: stale flag flips, refresh clears it, profile scoping, multi-kind batching, pipeline auto-refreshes on stale even without refresh_first, pipeline skips refresh when fresh. 3391 tests pass.', page: 'discover' },
{ title: 'Personalized Playlist Pipeline: Auto-Sync Discover-Page Playlists', desc: 'follow-up to the personalized-playlists standardization PR. new automation action `personalized_pipeline` syncs your selected discover-page playlists (Hidden Gems, Time Machine per-decade, Fresh Tape, The Archives, Seasonal Mix per-season, etc.) to your active media server + queues missing tracks for download — same pattern as the existing mirrored playlist pipeline but two phases instead of four (no REFRESH or DISCOVER needed since manager-backed snapshots are already metadata-matched). config: pick which kinds+variants to include, optional `refresh_first` to regenerate snapshots before syncing, optional `skip_wishlist`. shares the pipeline_running guard with the mirrored pipeline so the two can\'t overlap (one sync queue, one wishlist worker). lifted PHASE 3 (SYNC loop) + PHASE 4 (WISHLIST tail) of the mirrored pipeline into shared `core/automation/handlers/_pipeline_shared.run_sync_and_wishlist` so both pipelines reuse the same sync-state polling / progress emission / wishlist trigger logic — 0 duplication. trigger UI block declared at `core/automation/blocks.py` (full multi-select picker UI is its own follow-up). 14 new boundary tests pin: track→sync_shape conversion + source ID fallback, empty-kinds error, payload building skips no-tracks playlists, refresh_first vs ensure dispatch, manager exception swallowed continues to next kind, full pipeline happy-path with stubbed sync_states. 3383 tests pass total. now usable via API today, polished UI dropping next.', page: 'discover' }, { title: 'Personalized Playlist Pipeline: Auto-Sync Discover-Page Playlists', desc: 'follow-up to the personalized-playlists standardization PR. new automation action `personalized_pipeline` syncs your selected discover-page playlists (Hidden Gems, Time Machine per-decade, Fresh Tape, The Archives, Seasonal Mix per-season, etc.) to your active media server + queues missing tracks for download — same pattern as the existing mirrored playlist pipeline but two phases instead of four (no REFRESH or DISCOVER needed since manager-backed snapshots are already metadata-matched). config: pick which kinds+variants to include, optional `refresh_first` to regenerate snapshots before syncing, optional `skip_wishlist`. shares the pipeline_running guard with the mirrored pipeline so the two can\'t overlap (one sync queue, one wishlist worker). lifted PHASE 3 (SYNC loop) + PHASE 4 (WISHLIST tail) of the mirrored pipeline into shared `core/automation/handlers/_pipeline_shared.run_sync_and_wishlist` so both pipelines reuse the same sync-state polling / progress emission / wishlist trigger logic — 0 duplication. trigger UI block declared at `core/automation/blocks.py` (full multi-select picker UI is its own follow-up). 14 new boundary tests pin: track→sync_shape conversion + source ID fallback, empty-kinds error, payload building skips no-tracks playlists, refresh_first vs ensure dispatch, manager exception swallowed continues to next kind, full pipeline happy-path with stubbed sync_states. 3383 tests pass total. now usable via API today, polished UI dropping next.', page: 'discover' },
{ title: 'Personalized Playlists Standardization', desc: 'all 8 personalized / discover-page playlists (Hidden Gems, Discovery Shuffle, Popular Picks, Time Machine per-decade, Genre playlists per-genre, Daily Mixes, Fresh Tape, The Archives, Seasonal Mix per-season) now share one unified storage layer. pre-overhaul: Group A (Fresh Tape / Archives / Seasonal Mix) lived in one shape, Group B (everything else) was computed-on-demand with no persistence — every page-load re-rolled the dice and tracks rotated under your feet. post-overhaul: every playlist has a stable identity, persistent track snapshot, explicit refresh button, and per-playlist tweakable config (limit, diversity caps, popularity bounds, recency window, exclude-recent-days staleness window). prerequisite for the playlist pipeline integration coming in the next PR (sync these to your media server + send missing tracks to wishlist on a timer). fixed a stub: Daily Mixes used to promise 50% library + 50% discovery but the library half always returned [] (tracks table has no source IDs to sync) — now honestly discovery-only so it actually works. also: each kind\'s body lifted into its own module under `core/personalized/generators/`, behavior preserved verbatim from the legacy `PersonalizedPlaylistsService` and `SeasonalDiscoveryService`. new REST endpoints under `/api/personalized/*`. 134 boundary tests cover every kind + the manager + the API + staleness filter; full suite at 3369 tests.', page: 'discover' }, { title: 'Personalized Playlists Standardization', desc: 'all 8 personalized / discover-page playlists (Hidden Gems, Discovery Shuffle, Popular Picks, Time Machine per-decade, Genre playlists per-genre, Daily Mixes, Fresh Tape, The Archives, Seasonal Mix per-season) now share one unified storage layer. pre-overhaul: Group A (Fresh Tape / Archives / Seasonal Mix) lived in one shape, Group B (everything else) was computed-on-demand with no persistence — every page-load re-rolled the dice and tracks rotated under your feet. post-overhaul: every playlist has a stable identity, persistent track snapshot, explicit refresh button, and per-playlist tweakable config (limit, diversity caps, popularity bounds, recency window, exclude-recent-days staleness window). prerequisite for the playlist pipeline integration coming in the next PR (sync these to your media server + send missing tracks to wishlist on a timer). fixed a stub: Daily Mixes used to promise 50% library + 50% discovery but the library half always returned [] (tracks table has no source IDs to sync) — now honestly discovery-only so it actually works. also: each kind\'s body lifted into its own module under `core/personalized/generators/`, behavior preserved verbatim from the legacy `PersonalizedPlaylistsService` and `SeasonalDiscoveryService`. new REST endpoints under `/api/personalized/*`. 134 boundary tests cover every kind + the manager + the API + staleness filter; full suite at 3369 tests.', page: 'discover' },
{ title: 'Dashboard Activity Feed: Stop Showing "NaNmo ago"', desc: 'recent activity items on the dashboard all rendered "NaNmo ago" because the formatter was parsing `activity.time` (a human label like "Now") as a date. backend has always emitted `activity.timestamp` (Unix epoch seconds) alongside the label — frontend now uses that for relative-time formatting. falls back to the literal label only when no timestamp present (legacy items / future shapes).', page: 'home' }, { title: 'Dashboard Activity Feed: Stop Showing "NaNmo ago"', desc: 'recent activity items on the dashboard all rendered "NaNmo ago" because the formatter was parsing `activity.time` (a human label like "Now") as a date. backend has always emitted `activity.timestamp` (Unix epoch seconds) alongside the label — frontend now uses that for relative-time formatting. falls back to the literal label only when no timestamp present (legacy items / future shapes).', page: 'home' },