78724861f9
`BackgroundDownloadWorker` lives on the engine and owns the boilerplate every streaming download client currently hand-rolls: thread spawn, per-source semaphore, rate-limit delay, state lifecycle (Initializing → InProgress → Completed or Errored), exception capture. Plugins provide only the atomic download op (`impl_callable`). Per-source rate-limit policy (concurrency, delay) is configured on the worker via `set_concurrency` / `set_delay`. Source- specific record fields merge in via `extra_record_fields` so existing consumer code that reads `video_id`, `track_id`, `permalink_url`, etc. keeps working post-migration. Username slot supports override (Deezer's legacy `'deezer_dl'`). Phase C1 scope: worker exists. No client migrated yet — C2-C7 migrate sources one at a time, each gated by the Phase A pinning tests so per-source contract drift fails fast. 10 new tests pin the worker contract: UUID id format, initial record shape, extra-fields merge, username override, state transitions on success / impl-returns-None / impl-raises, semaphore serialization (default + parallel), rate-limit delay between successive downloads. Suite still green (308 download tests). Pure additive.
30 lines
1.2 KiB
Python
30 lines
1.2 KiB
Python
"""Download Engine — central owner of cross-source download state,
|
|
thread workers, search retry, rate-limits, and fallback chains.
|
|
|
|
This is the second leg of the multi-source download dispatcher
|
|
refactor (the first leg, ``core/download_plugins/``, defined the
|
|
contract). The engine takes ownership of everything that used to
|
|
be duplicated across the per-source clients (background thread
|
|
workers, active_downloads dicts, search retry ladders, quality
|
|
filtering, hybrid fallback). Clients become DUMB — just hit the
|
|
API for their source, manage their own auth state, and let the
|
|
engine drive everything else.
|
|
|
|
This package is built up in phases (see
|
|
``docs/download-engine-refactor-plan.md`` for the full plan):
|
|
|
|
- Phase B (current) — engine skeleton + state lift.
|
|
- Phase C — background download worker.
|
|
- Phase D — search retry + quality filter.
|
|
- Phase E — rate-limit pool.
|
|
- Phase F — fallback chain.
|
|
|
|
Each phase is purely additive at first (engine grows, clients
|
|
unchanged). Migration to the new shape happens one source per
|
|
commit so behavior never breaks across the suite.
|
|
"""
|
|
|
|
from core.download_engine.engine import DownloadEngine
|
|
from core.download_engine.worker import BackgroundDownloadWorker
|
|
|
|
__all__ = ["DownloadEngine", "BackgroundDownloadWorker"]
|