a3929b457b
`core/download_engine/rate_limit.py` introduces a per-source policy declaration: download_concurrency + download_delay_seconds. Plugins declare via `RATE_LIMIT_POLICY` class attribute or a `rate_limit_policy()` method. Engine applies the declared policy to engine.worker at register_plugin time — set_concurrency + set_delay get pushed in automatically. Plugins without a declaration get the conservative default (1 / 0). The set_engine callback fires AFTER policy registration so config-driven sources (YouTube reads user-tunable youtube.download_delay) can override. Plan doc updated to reflect Phase D skip (search code is 90% source-specific, not 60% — lifting it would be lossy or bloated). Pure additive — no plugin migrated yet. 8 tests pin the resolution priority + engine wire-up + override semantics. Suite still green (327 download tests).
31 lines
1.3 KiB
Python
31 lines
1.3 KiB
Python
"""Download Engine — central owner of cross-source download state,
|
|
thread workers, search retry, rate-limits, and fallback chains.
|
|
|
|
This is the second leg of the multi-source download dispatcher
|
|
refactor (the first leg, ``core/download_plugins/``, defined the
|
|
contract). The engine takes ownership of everything that used to
|
|
be duplicated across the per-source clients (background thread
|
|
workers, active_downloads dicts, search retry ladders, quality
|
|
filtering, hybrid fallback). Clients become DUMB — just hit the
|
|
API for their source, manage their own auth state, and let the
|
|
engine drive everything else.
|
|
|
|
This package is built up in phases (see
|
|
``docs/download-engine-refactor-plan.md`` for the full plan):
|
|
|
|
- Phase B (current) — engine skeleton + state lift.
|
|
- Phase C — background download worker.
|
|
- Phase D — search retry + quality filter.
|
|
- Phase E — rate-limit pool.
|
|
- Phase F — fallback chain.
|
|
|
|
Each phase is purely additive at first (engine grows, clients
|
|
unchanged). Migration to the new shape happens one source per
|
|
commit so behavior never breaks across the suite.
|
|
"""
|
|
|
|
from core.download_engine.engine import DownloadEngine
|
|
from core.download_engine.rate_limit import RateLimitPolicy
|
|
from core.download_engine.worker import BackgroundDownloadWorker
|
|
|
|
__all__ = ["DownloadEngine", "BackgroundDownloadWorker", "RateLimitPolicy"]
|