- EPISODE downloads now show the specific episode: still + 'S02E05 · air date' + that
episode's own title + synopsis (was just the show synopsis). meta endpoint takes
?season=&episode= → engine.tmdb_season; show cast/logo stay as context.
- YOUTUBE drawer: big 16:9 thumbnail + channel · duration · views · upload date +
description. new yt-meta route → db.youtube_video_detail (cached duration/views).
- AVAILABILITY line: the chosen source's free-slot/queue/speed snapshot, stashed in
search_ctx at grab time (build_download_record) → 'Availability: ✓ free slot · queue 0
· 2.1 MB/s'.
drawer also reworked into clean youtube / episode / movie-show branches; first paint
shows 'Loading…' instead of flashing 'no synopsis'. tested.
the airing automation reads the LOCAL episodes table, which only refreshes on initial
enrichment / manual re-match / lazy on-view — so a newly-announced or rescheduled
episode could be missed indefinitely. (the deep TV scan is a SERVER scan, unrelated.)
new daily automation re-pulls TMDB episode schedules (air dates/stills) for still-airing
watchlist shows so the calendar is fresh when the airing run reads it:
- db.watchlist_continuing_shows(): effective-watchlist library shows that are still airing
(skips tmdb-only follows — no episodes — and ended/canceled shows; keeps unknown status).
- handler mirrors the standard: injected fetch_shows + refresh_show seams, live per-show
progress, _manages_own_progress; reuses engine.refresh_show_art (match + episode cascade).
- seeded daily at 23:00 (2h before the 01:00 airing run), registered, UI block, sorted in
the page right before the airing automation.
seam-level tested + a regression that it's seeded before the airing run.
both lanes now expose the post-processing phase instead of jumping downloading→completed:
- YouTube now STAGES in the shared download folder (a 'youtube' subdir) then transfers to
the library — same as movies/TV — so no partial .part files land in the Plex folder.
process_youtube_download gains stage_dir + move seams: download → 'importing' → move →
completed (import_failed if the move dies; wish kept for retry). falls back to
straight-to-library when no download folder is set.
- movies/TV: the organizer flips the row to 'importing' before the (slow) move + sidecars +
subtitles, so that phase is visible.
- 'importing' is now an active status (get_active_video_downloads + youtube concurrency count
+ the downloads page status map), so the row stays on the page + holds its worker slot.
tested (staging + import-failure paths).
after you delete a file, the scans dedup against download history so it won't
re-grab — this gives you the escape hatch:
- per-row 'Re-download' button in the History modal: forgets that grab (DELETE
/downloads/history/<id>) so the next scan re-adds + re-downloads it.
- 'Clear' button: wipes the whole history (guarded confirm).
db.delete_download_history / clear_download_history(kind?) + the two endpoints.
tested (db methods + endpoints).
a completed youtube download is removed from the wishlist (correct), but the
channel/playlist scans' downloaded_ids seam was stubbed to [] — so the next scan,
not seeing it wishlisted and not knowing it was downloaded, re-added it (still in
the channel's recent uploads / last-N net) → re-download loop for the last-N
videos every scan.
fix: db.downloaded_youtube_video_ids() pulls completed youtube grabs from the
permanent download history; both scans' _default_downloaded_ids now use it. so a
downloaded video stays gone. tested (db method + the scans already exclude
downloaded_ids in their pure logic).
storage + wiring + API for the channel settings modal (UI next):
- db.get/set_channel_settings(channel_id): per-channel {custom_name, quality}
in the settings KV store (no schema change); blanks clear the override.
- enqueue applies them: custom_name overrides the $channel folder token; a
quality override is stashed in the download row's search_ctx. the worker reads
it back (quality_override_from_download) and uses it instead of the global
youtube quality profile. global default still applies when no override.
- API GET/POST /youtube/channel/<id>/settings (GET also returns the global
default quality for the modal's 'using default' hint).
pure helpers (enqueue_ctx, quality_override_from_download) + db + api seam-tested.
the soulseek counterpart of the youtube drain — finally makes the people/airing
scans pay off. two automations (Auto-Download Movie / Episode Wishlist, hourly):
for each wished+released item, do a bounded blocking slskd search, pick the top
ACCEPTED release per the quality profile, and enqueue it exactly like a manual
grab (same add_video_download shape → the monitor finishes + organises it).
same standard as youtube: processes the WHOLE eligible wishlist (no total cap)
but searches a few at a time (max_concurrent, default 3, via a thread pool). a
busy guard skips the next hourly tick while a drain is still working. movies gate
on status='wanted' (skips monitored); episodes are all-wished. items already
downloading are skipped. quiet skip if the library folder isn't set.
reuses the real infra (build_query/slskd_search/_evaluate_hits/start_download/
download_monitor) — pure pick/select/record seam-tested; db queries added. 377
automation+video tests green.
ditch the per-run batch cap (a 200-video backlog would've taken weeks). now the
'Download YouTube Wishlist' automation queues the ENTIRE wishlist as 'queued'
rows, starts up to max_concurrent (default 3) right away, and each finished
download starts the next (one-out-one-in in the worker) so it all drains in a
controlled stream. the knob is 'max simultaneous downloads', not a total cap.
mirrors the music download worker's lesson (cap concurrency + space starts to
avoid yt-dlp 429s) but stays isolated on the video side:
- youtube_download: _pace() staggers fetch starts (3s); start_next_queued()
claims+spawns the next; run_youtube_download chains on finish.
- db.count_active_youtube_downloads() + claim_next_youtube_queued() (atomic,
race-safe). block field batch_size -> max_concurrent.
all seam-tested (pure select + pump); 680-test sweep green.
new 'Download YouTube Wishlist' automation: pushes a polite batch of wished
youtube videos into the shared video_downloads queue + spawns the yt-dlp worker
per video. skips in-flight ones (no double-grab); big backlogs drain over
several scheduled runs (batch_size, default 3). needs the youtube library folder
set.
- download_monitor: SKIP source='youtube' rows (owned by their worker thread, no
slskd transfer to match) — surgical, slskd path untouched.
- db.youtube_wishlist_to_download(): flat newest-first list of wished videos with
channel/title/date/thumb for organising.
block + registration + icon/label + drift test. all seam-tested.
two new video-side automation blocks that keep the wishlist fed:
- scan watchlist people: for each followed person, wishlist every un-owned
movie they acted in or directed (back catalog + upcoming). released ->
wanted, upcoming -> monitored (engine skips it til it's out, promotes on
release). grabs rich detail at add time (backdrop/cast/overview/etc +
provenance) into a new video_wishlist.detail_json col. fast re-runs skip
already-wishlisted + only promote.
- scan watchlist channels: for each followed youtube channel, wishlist new
long-form uploads (shorts excluded). forward-looking from follow time +
a last-N safety net (default 10). diffs against wishlisted/downloaded/
dismissed so it never dupes. pair with a 6h schedule trigger. scan-only;
fulfillment engine comes later.
both are pure handlers with injected seams + full seam tests. add_movie_to_
wishlist gains status + detail_json (promote-only upsert). no schema break,
music side untouched.
Already-matched movies predate the tmdb_collection_id column, so the collection gap rails
were empty (only newly-enriched movies got the id). Add a self-healing backfill: each
/discover/gaps load fills the franchise id for up to 20 owned movies missing it
(eng.movie_collection reuses the matcher's belongs_to_collection read), recording 0 for
movies with no franchise so they're not re-checked. The 'no-franchise' 0 is excluded from
the rails. Backfill is wrapped/isolated so it can never break the gap response. Over a few
Discover visits the whole library fills in and 'Complete the <franchise>' rails populate.
The new index on movies.tmdb_collection_id was in video_schema.sql, which executescript runs
BEFORE _ensure_columns adds the column on existing DBs — so 'CREATE INDEX ... ON
movies(tmdb_collection_id)' failed with 'no such column' and the whole video DB init aborted
(500s on every /api/video/* call). Moved the index to _POST_INDEXES (runs after the ALTERs),
matching the pattern the code comments already prescribe. The CREATE TABLE columns stay (fresh
DBs) + the ALTER migration stays (existing DBs); only the index moved.
- video_database: owned_movie_tmdb_ids (diff set), owned_movie_collections (franchises you've
started, most-invested first), top_owned_people (directors/creators you own the most).
- engine.collection(id): cached + owned-annotated franchise film list (person_detail already
gives owned-annotated filmography).
- /api/video/discover/gaps: builds 'Complete the <franchise>' rails (collection_gaps) + 'More
from <person>' rails (filmography_gaps, movies, vote-filtered) — the 'what am I missing' section.
All additive; gap diffs are the pure tested core.
Data layer for the 'complete your collections' gap engine. People/credits/genres are already
normalized + indexed, so the only missing signal was franchise membership:
- movies: + tmdb_collection_id (indexed) + tmdb_collection_name (schema + _COLUMN_MIGRATIONS,
SCHEMA_VERSION 17->18, _ENRICH_META_COLS whitelist so enrichment_apply backfills them)
- enrichment match() reads belongs_to_collection (a standard movie-detail field, no extra call)
and writes the id/name into the match metadata.
Additive + backfill-only (COALESCE), nothing existing rewired. (also noqa'd a pre-existing
OMDb S110 in the touched file to keep ruff clean.)
You can eye-add a show to the watchlist before we know its status, so ended/canceled
shows leak in (auto-airing LIBRARY shows already exclude ended ones; explicit follows
don't). Fix it as cleanup-on-process, per Boulder: the daily 'Wishlist Today's Airings'
automation now runs a watchlist-tidy pass first — scans every explicit show follow,
resolves its status (local for owned, TMDB for tmdb-only follows), and removes any
that have ended/been canceled/completed. Only prunes on a DEFINITIVE terminal status;
unknown/lookup-error → left alone. Toggle prune_ended (default on); returns shows_pruned.
DB: followed_shows(). Pure prune_ended_show_follows() with injected seams; seam tests.
video_downloads is a transient queue (hard-deleted on cleanup), so there was no record
of what SoulSync actually grabbed. Add a permanent video_download_history table +
capture: the monitor snapshots every terminal download (completed/import_failed/
cancelled/failed) into it, with rich metadata (title, year, S/E from search_ctx,
release, source, size, quality + parsed resolution/codec, dest path, poster, outcome,
timestamps). Idempotent per (download_id, outcome, dest_path).
DB methods: record_download_history, query_download_history (paged/kind/search),
download_history_detail, download_history_counts, latest_completed_download(media_type)
— the last is the probe target for the upcoming smart post-download scan. Schema v17.
Auto-added airing episodes came in metadata-empty (no synopsis, no still) — the
handler only passed season/episode/title/air_date, dropping the overview the
calendar already returns and never fetching the still URL (calendar_upcoming
only returned a has_still flag, not the URL). Now calendar_upcoming also returns
e.still_url, and the handler carries overview + still_url through. The wishlist
renders the (Plex-relative) still via the same pimg() proxy as the show poster,
so it resolves. Idempotent upsert backfills the already-added empty rows on the
next run.
'Holy Marvels with Dennis Quaid' is stored with a leading space in the shows
table, so the watchlist sort key fell to ' holy marvels…' — and a leading space
sorts before 'a', jumping it to the top. .strip() the sort key so dirty titles
sort by their real first letter.
The default sort put manual follows first (newest date_added), then airing
shows A–Z — so recently-followed shows like 'Welcome to Widows Bay' and 'From'
jumped to the top. A manual follow is no more special than an auto-added airing
show; default now sorts everything by name A–Z. 'added' (newest first) stays as
an opt-in sort.
End-to-end import for video grabs, mirroring the music side's rigor and the
Radarr/Sonarr standard, fully isolated in core/video + api/video.
- Importer: parse release -> ffprobe-verify (true resolution, reject corrupt/
samples) -> templated rename into Movie (Year)/ + Show/Season NN/ -> copy or
move, carry subtitles, upgrade-replace a worse copy.
- Library Organization settings: editable $token path templates + toggles
(transfer mode, verify, replace, carry subs, save artwork, write NFO,
download subtitles + langs). Stored in video.db; matches the music File
Organization section's look.
- Sidecar writer: movie.nfo / tvshow.nfo + full artwork set (poster, fanart,
clearlogo, season posters) from on-demand TMDB detail, and external .srt from
OpenSubtitles. Owned re-grabs resolve their library tmdb_id; tmdb_full_detail
bypasses the owned->library redirect so they enrich too.
- Import page: surfaces import_failed downloads, resolve by hand (library-first
-> TMDB picker -> force-place) or dismiss; fires a library refresh on place.
- "Grab whole season": episode-level batch (reuses searchInto + _autoPick).
- Brutalist redesign of the download modal sources + result cards.
All new logic has seam-level tests (pure parsers/planners + injected I/O);
sidecars/subtitles are best-effort and never break an import.
The wishlist page had no way to empty a tab — only per-item remove. Added a red-tinted
'Clear all' button in the toolbar that empties the ACTIVE tab in one click (after a
confirm), shown only when that tab has items.
- db.clear_wishlist(kind) maps the tab to its rows (movie→'movie', show→'episode',
youtube→'video') and deletes them; returns the count.
- POST /api/video/wishlist/clear {kind: movie|show|youtube}.
- Toolbar button + clearAll() (confirm → clear → reload) + updateClearBtn() visibility.
Tests: per-tab clear leaves the others intact, unknown-kind/empty no-ops, the endpoint,
and the frontend wiring. node --check clean.
A scan wrote server-provided fields straight over the row, so an incremental/deep
re-read wiped the TMDB-backfilled 'status' (Plex returns it blank) — clearing the
airing watchlist. Now matches the intended model: incremental (add recent) and deep
(coverage + prune) PRESERVE enrichment-owned fields the server left blank; only a FULL
scan clobbers them (an explicit reset / fresh start).
- _resilient_upsert gains preserve_enrichment (default True): on a conflict UPDATE,
enrichment-owned columns (per _ENRICH_META_COLS: status/network/ratings/air dates/…)
take the server value only when non-blank, else keep what's stored. A real server
value still wins.
- upsert_movie/upsert_show_tree thread the flag; scanner passes preserve=(mode!='full').
Tests: preserve-on-blank, server-value-wins, full-resets, and the scanner picking the
right mode. 88 DB + 18 scanner tests + isolation green.
Alongside the per-worker 'Retry all failed', the worker modal now has a topbar
'Retry all failed' that re-queues every failed/not_found item across ALL workers and
kinds in one click — one-shot recovery after an API outage left lots errored.
- db.retry_all_failed() derives the full service+kind set from the same _ENRICH /
_BACKFILL maps the workers use (tmdb/tvdb + omdb + fanart/opensubtitles/trakt/
tvmaze/anilist/wikidata + ryd/sponsorblock/dearrow), loops enrichment_retry, returns
the total re-queued. POST /api/video/enrichment/retry-all-failed.
- Topbar button (amber, text) → calls it, toasts the count, refreshes the modal.
DB test (resets across matcher + backfill + youtube service, deterministic count) +
frontend wiring test. ruff + node --check clean.
The calendar pulled every airing show in the library regardless of whether you
follow it. Now it's scoped to the EFFECTIVE watchlist by default — explicit show
follows ∪ airing library shows (not muted), same logic as the Shows watchlist tab —
so it tracks what you actually care about, and you can mute a show off it. A
'Watchlist / All library' toggle on the calendar lets you flip to everything you
own (remembered in localStorage).
- calendar_upcoming(watchlist_only=) adds the watchlist filter; /calendar takes
?scope=watchlist|all (default watchlist).
- Calendar page gets a scope toggle (defaults watchlist, persists, refetches on
change).
Tests: DB scope (followed-only / airing-default / mute drops out / all-library sees
all) + frontend wiring. node --check clean.
enrichment_retry handled ryd/sponsorblock + the keyed backfills, but DeArrow (also a
youtube_video_stats backfill, keyed on dearrow_status) fell through to nothing — so the
modal's Retry button did nothing for DeArrow. Fold dearrow into the youtube_video_stats
retry branch. Regression test re-queues failed dearrow rows, leaves matched ones.
The media server pre-matches shows/movies (tmdb_id set), so the enrichment matcher
skips them and never fetches TMDB details — leaving details-only fields like `status`
(airing vs ended) blank on almost the whole library. That's why the watchlist's
airing-shows default only ever saw the handful of shows whose detail page had been
opened (the one path that force-fetches details). Library here: 3,371 matched shows,
only 18 with status.
Fix: a one-time-per-item details backfill that runs in the enrichment worker's idle
loop (after the episode-sync pass). New `details_synced` marker column on shows+movies;
detail_backfill_next/mark_details_synced/pending_count; worker._detail_backfill_one()
re-fetches an already-matched item's TMDB details and gap-fills (never clobbers server
data), then marks it done so it's attempted once. No re-scan needed — it heals the
existing library in place, and once status is populated the airing-watchlist reflects
real TV.
It's a background gap-fill on already-matched items (like episode coverage), so it
doesn't block the worker's 'Complete' status. kettui: DB seam tests + worker tests
(fills status / enrich-by-id / marks-done-when-absent). ruff + isolation guards green;
94 enrichment tests pass.
When a grabbed release fails (transfer error / peer-cancel / never lands), the engine
now retries instead of giving up — the music-style depth:
- Grab stores the OTHER accepted results as a retry pool + the search context (schema
v16: candidates / search_ctx / tried_queries / tried_files / attempts).
- core/video/retry.py (pure, tested): plan_retry() → try the next-best candidate; when
the pool is dry, next_query() generates an ALTERNATE query (movie: drop the year; TV:
numbering variants) to re-search; budget MAX_ATTEMPTS=6. merge_candidates dedupes
against already-tried releases.
- Monitor: on failure, _fail_or_retry hops to the next candidate inline; if none, flips
the row to a new 'searching' state and a background requery thread re-searches the
alternate query, evaluates against the profile, and starts the best fresh hit — or
marks failed once truly exhausted. 'searching' rows are owned by their thread.
- Page: 'Searching' status (Trying another release…) + a 'Nx' attempt badge.
16 tests (retry engine + candidate-retry transition); ruff clean on touched files.
Grabs now carry the movie's identity so the Downloads cards are rich, not anonymous:
- video_downloads gains media_id / media_source / year / poster_url (schema v15 +
migration); grab stores them (passed from the get-modal → download view → grab).
- Cards show the POSTER in the art tile (emoji fallback), 'Title (Year)', a quality
chip (1080p · BluRay · X265), and an ↗ Open button that jumps to the movie/show
detail page (dispatches soulsync:video-open-detail). Cancel/retry unchanged.
16 tests green, ruff + balance clean.
Fixes the 'cancelled but still shows running' stuck bug and adds real depth:
- classify_state now distinguishes 'cancelled' from 'failed'.
- Monitor is robust to slskd forgetting a transfer: if it's gone, it first tries to
complete from the FILE on disk (survives the music 'Clean Completed Downloads'
auto-clear), else counts misses and fails the row after ~8 polls instead of hanging
on 'downloading' forever. Cancelled transfers → cancelled status.
- POST /downloads/cancel (slskd DELETE transfer + mark cancelled) and /downloads/retry
(re-grab the same release). get_video_download(id) added; clear includes cancelled.
process_download stays pure (fs/slskd injected); 15 tests, ruff + guard clean.
Phase B (page: tabs/queue/history/cancel+retry buttons) next.
Fifth/final service. Keyless, movies + shows, on by default → toggle in the
'Community Data (No Key)' frame.
- WikidataWorker: two-step lookup — find the entity by IMDb id (haswbstatement P345)
then read its official website (P856); stores wikidata_url. Registered in
build_backfill_workers.
- DB: wikidata_url/status/attempted on movies + shows + _BACKFILL/_BACKFILL_COLS;
both detail payloads return wikidata_url.
- config GET/POST wikidata_enabled; manager orb (green 🔗) + status poll; detail page
'Official Site' link badge alongside IMDb/TMDB/TVDB.
- 3 new tests (incl. the two-step fetch) + fixed-set/config assertions.
34 backfill tests green, ruff clean. All five new services (Trakt, TVmaze, AniList,
DeArrow, Wikidata) now have the full worker/DB/connections/orb/manager/detail parity.
Fourth service. Rides the YouTube-video enrich path (like RYD/SponsorBlock), keyless,
on by default → toggle in the YouTube-Extras frame.
- DeArrowWorker: fetches the branding API and stores the first non-original crowd
title for a cached YouTube video. apply_youtube_dearrow + youtube_video_dearrow_title
DB methods; dearrow_status added to the youtube_enrich next/breakdown whitelists.
- Schema: dearrow_title/status/attempted on youtube_video_stats (video_schema.sql +
_COLUMN_MIGRATIONS for existing DBs).
- config GET/POST dearrow_enabled; manager orb (blue 🏹, video-kind) + status poll;
YT video-detail panel shows a 'DeArrow' alt-title (payload via youtube.py + CSS).
- 4 new tests + fixed-set/config assertions.
30 backfill tests green. (My change is ruff-clean; the 12 pre-existing S110s in
api/video/youtube.py are unrelated tech debt on this branch, left untouched.)
Third service. Keyless GraphQL (new _http_post_json helper) → enable toggle in the
'Community Data (No Key)' frame, OFF by default (anime-niche + title-search match).
- AniListWorker: TV-only, searches AniList by title and stores the anime averageScore
(0-100), with a conservative normalized-title guard so a fuzzy anime search can't
attach a score to a non-anime show. anilist_enabled toggle (default off).
- DB: anilist_score/status/attempted on shows + _BACKFILL/_BACKFILL_COLS (keyed on
title); show_detail returns anilist_score.
- config GET/POST anilist_enabled (default 0); manager orb (blue 🎌) + status poll;
detail page 'AniList 85%' chip (+ CSS).
- 4 new tests (incl. the title-mismatch rejection) + fixed-set/config assertions.
27 backfill tests green, ruff clean. (Note: the youtube_status_route test still
flaps on the sandbox WSL WAL disk-I/O — environmental, unrelated.)
Second service. Keyless (no API key) → an enable toggle in a new 'Community Data
(No Key)' connections frame, mirroring the YouTube-Extras pattern.
- TVmazeWorker: TV-only (no movie DB), looks a show up by imdb/thetvdb id and
gap-fills the TVmaze community rating. On by default; tvmaze_enabled toggle.
- DB: tvmaze_rating/status/attempted on shows + _BACKFILL/_BACKFILL_COLS (show only);
show_detail returns tvmaze_rating.
- config GET/POST tvmaze_enabled; manager orb (teal 📺, show-kind) + status poll;
detail page TVmaze rating chip (+ CSS).
- 4 new tests + the 3 fixed-set/config-exact assertions updated.
Note: one UNRELATED test (youtube_status_route) flaps on a WSL WAL 'disk I/O error'
in this sandbox (pass/fail/fail across reruns of identical code) — environmental, not
this change; the TVmaze + config tests pass consistently.
First of the new enrichment services, wired to the SAME standard as the rest:
- TraktWorker backfill (core/video/enrichment/backfill.py): looks a title up by its
IMDb id (?extended=full) and gap-fills the community rating + vote count. Registered
in build_backfill_workers.
- DB: trakt_rating/trakt_votes/trakt_status/trakt_attempted columns + _BACKFILL /
_BACKFILL_COLS registration; detail payloads return trakt_rating/votes.
- Connections tab: Trakt service frame (Client ID field + Test button), wired into
video-settings.js load/save/bindings and the /enrichment/config GET+POST.
- Worker-manager modal + orb: WORKERS registry entry (red ★) + status-poll SERVICES
list, so it gets the same orb animation + per-service matched/pending/error card.
- Detail page: a 'Trakt 8.2' rating chip alongside IMDb/RT/Metacritic (+ CSS).
5 new tests + 2 fixed-set assertions updated; 249 video tests green, ruff clean.
The OpenSubtitles backfill worker already collects which subtitle languages exist
for each title (movies.subtitle_langs / shows.subtitle_langs), but nothing showed
it. Now the detail payload returns it (parsed to a list) and the hero renders a
'Subtitles: English · Spanish · …' CC-tinted chip row under the genres — so you can
tell subs exist before grabbing the file. Hidden entirely when there's no data.
(The clearlogo hero was already implemented, so this targets the one genuinely
invisible enrichment field.)
Also bound the per-iteration loop var in the ratings-breakdown counter (pre-existing
ruff B023). 154 video tests green, ruff clean.
Adds a VideoBackfillWorker base that enriches already-identified items BY id
(vs the matcher workers), with the exact same lifecycle + get_stats() shape so
the engine registry, /api/video/enrichment routes, and Manage-Workers modal
drive them identically.
Workers:
- fanart.tv (free key): gap-fills logo/clearart/banner/backdrop/poster art
- OpenSubtitles (free key): records subtitle-language availability per title
- Return YouTube Dislike (no key): like/dislike estimates on cached videos
- SponsorBlock (no key): crowd segments per video
DB: youtube_video_stats + youtube_video_segments tables; fanart/subs columns;
backfill_next/mark/breakdown + youtube_enrich_* helpers; likes/dislikes merged
into get_channel_videos. Seam tests in tests/test_video_backfill.py.
The channel card's 'N videos' was actually the count of WISHED videos (0 until you
wish some) — so every channel read '0 videos'. It now shows the REMEMBERED catalog
size (from youtube_channel_videos, which fills in as a channel is enriched/opened),
falling back to 'Channel' when nothing's cached yet. Wished count is kept as a
separate wished_count field. Playlists likewise show their video count (cached when
the playlist is followed-with-videos or opened) instead of a static 'Playlist'.
list_watchlist_channels/playlists return the remembered count; the playlist detail +
follow endpoints cache the list so the count is known. 138 tests green.
A playlist is now a first-class watchlist type alongside channels:
- core: parse_playlist_id (URL/bare id; rejects mixes RD… + personal lists) +
resolve_playlist → {playlist_id, title, channel_title, video_count, thumbnail,
videos} in the CURATOR's order (partial set, not by year).
- db: add/remove/list/watch_state for kind='playlist' video_watchlist rows
(mirrors channels; same surrogate scheme, separate kind so they don't collide).
- api: /youtube/resolve now detects playlist links; /youtube/playlist/follow +
/unfollow; the existing /youtube/playlist/<id> is upgraded to also return the
playlist meta + following (serves the channel-page expansion AND the new detail
view from one route); /youtube/channels also returns followed playlists.
Validated live (Lex Fridman playlist resolves title+owner+videos). 190 tests green.
Frontend (search detect + result card + detail view + watchlist display) next.
TV episode rows show runtime; channel video cards showed nothing per video. The
InnerTube lockup already carries both — now we capture duration ('12:34') + an
approximate view count (parsed from '2.6M views') in innertube_parse_video_items,
remember them on youtube_channel_videos (new duration/view_count cols + migration),
and render a duration badge bottom-right of the thumb + 'N views' in the card meta.
The background stream backfills them onto the recent (yt-dlp) videos too, so the
whole catalog gets them. Live-validated (MrBeast: 32:08 / 50M, 30/30). 65 tests green.
Channel pages re-fetched everything every open (re-stream + ~3s yt-dlp metadata).
Now we remember what we learn about a channel and serve it cache-first:
- schema: youtube_channel_videos (list) + youtube_channel_meta (avatar/subs/tags/
banner); dates stay in youtube_video_dates, merged on read. DB: cache_/get_
channel_videos + cache_/get_channel_meta (upserts COALESCE so refreshes never
drop fields).
- enricher: _enrich now REMEMBERS a channel — caches the full InnerTube catalog
(list, via new innertube_channel_catalog) + metadata + dates, not just dates.
Since it already sweeps followed channels, watchlisted channels get pre-warmed
in the background → opening them is instant.
- channel endpoint: cache-first. A remembered channel renders from cache with NO
network (returns from_cache:true); a miss resolves live (yt-dlp) and remembers
it. The /videos batch endpoint caches every page it streams.
- frontend: cache hit renders the full catalog instantly, then re-streams to
refresh QUIETLY (new uploads/date fixes); only the first (miss) load shows the
'loading full history' banner.
Live-validated (MrBeast catalog: 90/3 pages, all titled+dated). 190 tests green.
Channels enriched before InnerTube existed (partial coverage, e.g. CaseyNeistat
36, Good Mythical MORE 25) were locked behind the 24h coverage gate and would
never reach the full ~240-date catalog on their own. Tag each enrichment row
with the source ('innertube'|'fallback'); legacy rows (method NULL after the
additive migration) bypass the gate ONCE so they re-enrich and upgrade, then
settle under the normal window. Non-destructive: existing follows, wished videos
(24), and cached dates (1176) are untouched — only coverage refreshes on next
open/sweep. Migration is additive (method TEXT); regression test added.
- Fixes 'stuck idle': a channel marked enriched with FEW dates (proxies were
down) was locked out for 24h, so the improved yt-dlp fallback never re-ran.
channel_dates_enriched_recently now retries in 15 min when the last run got
<15 dates, and skips 24h only after a good run. So the catalog actually fills
in once a source works.
- The YouTube Dates worker now appears in the Manage Workers modal: added to
WORKERS (red ▶), youtube-aware rail subtitle, and a dedicated simple panel
(status + what-it-does + channels-enriched / dates-cached / queued counts) —
no per-kind match queue. Pause/resume works; polls every 3s like the rail.
Follow a channel → a background job fetches its FULL upload-date catalog so the
channel page's year-seasons populate fully, cached permanently (one-time per
channel, instant after).
- core/video/youtube.py: proxy_channel_dates() — no-key BULK dates via Piped/
Invidious instances (tries several, paginates); parse_proxy_dates handles both
shapes. The clean path the user wanted.
- core/video/youtube_enrichment.py: YoutubeDateEnricher daemon — enqueue(channel)
→ proxy bulk → cache; per-video yt-dlp only as a throttled fallback (cap 60,
0.4s) for wished videos when every proxy is down. Skips channels enriched in
the last 24h.
- db: youtube_channel_enrichment table + mark/check + wishlisted_video_ids_for_
channel. Enqueued from /youtube/follow and on opening a followed channel.
Scoped to FOLLOWED channels only (per request). 174 tests green; enricher imports
nothing from music.
Flat listing has no upload dates, so channels showed one 'All Videos' season.
Now real year-seasons, filled from cheap sources and cached so they grow:
- core: parse_rss_dates / channel_recent_dates — one public RSS GET dates the
~15 most-recent uploads (no yt-dlp, no bot risk).
- db: youtube_video_dates cache table + cache_video_dates / get_video_dates.
- /youtube/channel merges cached + RSS dates onto the flat videos (year-seasons)
and caches what it learns; /youtube/video caches each fetched date too — so
expanding/wishing videos progressively fills the catalog's years, instant on
repeat. Dateless tail groups as 'Earlier videos'. Missing-only toggle hidden
for channels. 84 db + 84 youtube/api tests green.
Full day-one coverage still needs the YouTube Data API's publishedAt (optional,
yt-dlp can't match it cheaply) — RSS + cache is the no-key best.
Flat channel listing doesn't always surface the avatar, so it could be stored
null → the orb fell back to plain initials (looked like a missing poster). Two
fixes:
- Orb falls back to the channel's newest video thumbnail when the avatar is
absent, so it's never blank.
- Opening the channel page (which resolves the real avatar) now backfills it
onto every wished row via set_wishlist_channel_poster — so the actual channel
avatar appears on the wishlist orb thereafter. 6 tests green.
Flat channel listing can't return descriptions, so selecting a video showed 'No
description'. Now we do a non-flat single-video extract on demand — the way the
TV nebula lazy-loads guest stars:
- core: video_detail(id) / shape_video() — description, views, likes, duration,
tags, channel, webpage_url (full extract, yt-dlp injectable for tests).
- API: GET /youtube/video/<id> — returns it AND backfills the description onto
the wishlist row (set_wishlist_video_overview), so re-opening is instant.
- Wishlist info bar (youtube): on select, lazy-fetch → real description +
eyebrow (date · duration · views) + a 'Watch on YouTube' link; cached on the
episode object. 'Loading details…' while in flight.
Mirrors the episode lazy-load + art-backfill patterns. 73 youtube+api tests
green; brace/CSS balanced; music untouched.
- Resolver: capture banner_url + separate avatar (by thumbnail id), keeping
per-video duration/views for the detail hero.
- query_youtube_wishlist reshaped to the EXACT TV-nebula shape: channel = show,
upload YEAR = season (newest first), video = episode (newest=ep1 within a
year), carrying source='youtube' + youtube_id + per-video source_id. Surrogate
int returned as tmdb_id so the nebula's int keying just works.
- New API: GET /youtube/channel/<id> (full channel detail — meta + deeper
uploads + following + per-video wished flags) and POST /youtube/wishlist/add
(per-video wish). DB: youtube_video_wish_state, remove_one_video_from_wishlist.
- Tests updated for the nebula shape + banner + detail endpoints. 82 DB + 69
api/youtube tests green.
The two new partial indexes (idx_video_wishlist_video/_channel on source_id /
parent_source_id) were in video_schema.sql, which runs via executescript()
BEFORE the column ALTERs. On a fresh DB the CREATE TABLE includes the columns so
it's fine, but on an existing (pre-bridge) DB the table already exists without
them, so 'CREATE INDEX ... ON video_wishlist(source_id)' threw 'no such column'
and the whole init rolled back — the video side wouldn't load at all.
Move those two indexes out of the schema into VideoDatabase._ensure_indexes(),
run AFTER _ensure_columns(). Regression test simulates a pre-source DB and
asserts the in-place upgrade + the youtube path both work. 81 DB tests green.
