Commit graph

2 commits

Author SHA1 Message Date
Broque Thomas
0ee092979e Self-review fixes before opening PR
Three findings from a final review pass:

1. **Worker clobbered Cancelled with Errored when impl returned
   None / raised mid-cancel.** The legacy per-client thread workers
   each had a guard (``if state != 'Cancelled': state = 'Errored'``);
   the shared worker dropped it. Fix: new ``_mark_terminal`` helper
   in BackgroundDownloadWorker reads current state before writing
   the terminal one and leaves Cancelled alone. SoundCloud test
   updated back to the strict Cancelled-only assertion (had been
   loosened to accept Errored as a workaround). Two new pinning
   tests catch the regression.

2. **Dead code in engine.py.** ``find_record`` and
   ``iter_all_records`` had no production callers — only tests.
   Removed them. Concurrent-add stress test rewritten to use the
   per-source iterator that's actually in use.

3. **Silent ``except Exception: pass`` in cross-source query
   methods.** Faithful to legacy behavior (one source failing
   shouldn't take down aggregation) but Cin's standard is "log
   even when you swallow." Each silent-swallow site now logs at
   debug level so the source name + exception are inspectable
   without adding warning-level noise.

Suite still green (2049 passed).
2026-05-04 16:24:13 -07:00
Broque Thomas
78724861f9 C1: Add BackgroundDownloadWorker to engine
`BackgroundDownloadWorker` lives on the engine and owns the
boilerplate every streaming download client currently
hand-rolls: thread spawn, per-source semaphore, rate-limit
delay, state lifecycle (Initializing → InProgress → Completed
or Errored), exception capture.

Plugins provide only the atomic download op (`impl_callable`).
Per-source rate-limit policy (concurrency, delay) is configured
on the worker via `set_concurrency` / `set_delay`. Source-
specific record fields merge in via `extra_record_fields` so
existing consumer code that reads `video_id`, `track_id`,
`permalink_url`, etc. keeps working post-migration. Username
slot supports override (Deezer's legacy `'deezer_dl'`).

Phase C1 scope: worker exists. No client migrated yet — C2-C7
migrate sources one at a time, each gated by the Phase A
pinning tests so per-source contract drift fails fast.

10 new tests pin the worker contract: UUID id format, initial
record shape, extra-fields merge, username override, state
transitions on success / impl-returns-None / impl-raises,
semaphore serialization (default + parallel), rate-limit
delay between successive downloads.

Suite still green (308 download tests). Pure additive.
2026-05-04 13:21:41 -07:00