Commit graph

3 commits

Author SHA1 Message Date
Broque Thomas
a3929b457b E1: Add RateLimitPolicy declaration mechanism
`core/download_engine/rate_limit.py` introduces a per-source
policy declaration: download_concurrency + download_delay_seconds.
Plugins declare via `RATE_LIMIT_POLICY` class attribute or a
`rate_limit_policy()` method.

Engine applies the declared policy to engine.worker at
register_plugin time — set_concurrency + set_delay get pushed
in automatically. Plugins without a declaration get the
conservative default (1 / 0). The set_engine callback fires
AFTER policy registration so config-driven sources (YouTube
reads user-tunable youtube.download_delay) can override.

Plan doc updated to reflect Phase D skip (search code is 90%
source-specific, not 60% — lifting it would be lossy or
bloated).

Pure additive — no plugin migrated yet. 8 tests pin the
resolution priority + engine wire-up + override semantics.
Suite still green (327 download tests).
2026-05-04 14:38:20 -07:00
Broque Thomas
78724861f9 C1: Add BackgroundDownloadWorker to engine
`BackgroundDownloadWorker` lives on the engine and owns the
boilerplate every streaming download client currently
hand-rolls: thread spawn, per-source semaphore, rate-limit
delay, state lifecycle (Initializing → InProgress → Completed
or Errored), exception capture.

Plugins provide only the atomic download op (`impl_callable`).
Per-source rate-limit policy (concurrency, delay) is configured
on the worker via `set_concurrency` / `set_delay`. Source-
specific record fields merge in via `extra_record_fields` so
existing consumer code that reads `video_id`, `track_id`,
`permalink_url`, etc. keeps working post-migration. Username
slot supports override (Deezer's legacy `'deezer_dl'`).

Phase C1 scope: worker exists. No client migrated yet — C2-C7
migrate sources one at a time, each gated by the Phase A
pinning tests so per-source contract drift fails fast.

10 new tests pin the worker contract: UUID id format, initial
record shape, extra-fields merge, username override, state
transitions on success / impl-returns-None / impl-raises,
semaphore serialization (default + parallel), rate-limit
delay between successive downloads.

Suite still green (308 download tests). Pure additive.
2026-05-04 13:21:41 -07:00
Broque Thomas
f40c6d3b55 B1: Add DownloadEngine skeleton
`core/download_engine/` package with the engine class that will own
cross-source state, threading, search retry, rate-limits, and
fallback chains. Orchestrator constructs an engine and registers
each plugin with it.

Phase B1 scope: skeleton only. Engine stores active_downloads
records keyed by (source, download_id), provides thread-safe
add/update/remove/iterate primitives, and holds plugin references
for later phases. NOT on any code path yet — pure additive
scaffolding so subsequent commits can introduce engine-driven
behavior one piece at a time without a big-bang switchover.

15 new tests pin the engine's state-storage contract: shallow-copy
reads, partial-patch updates, no-op-on-missing semantics,
per-source iteration, id-only find, concurrent-add safety.

Suite still 290 (download subset) green. Zero behavior change.
2026-05-04 12:39:41 -07:00