Commit graph

5 commits

Author SHA1 Message Date
Yiorgis Gozadinos
d781335868
Move CircuitBreaker to a shared module
Relocate CircuitBreaker from ingester/pollers to haiku/rag/circuit_breaker
so non-ingester callers (docling-serve provider) can reuse it without
depending on the ingester package.

Co-Authored-By: bryan davis <bryan@monkeytronics.org>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 11:03:30 +03:00
Chris McDonough
004d59563c Extract _stagger_start helper into BasePoller, add tests
The jitter-before-first-sleep block was duplicated verbatim in
PeriodicPoller.run() and FSPoller._sweep_loop(). Move it to
BasePoller._stagger_start() with a named _STAGGER_FRACTION constant.

This also gives a testable seam outside the pragma-no-cover
event-loop glue methods.
2026-06-01 09:32:18 -04:00
Chris McDonough
a6b3e7f1f6 Stagger periodic poll sweeps to avoid thundering herd
All pollers sharing the same poll_interval_s previously woke up and
swept at exactly the same moment after startup. With 10+ sources
this causes a coordinated spike in listing traffic (S3 LIST, WebDAV
PROPFIND, HTTP HEAD) every interval.

Add a random initial delay of 0-25% of the poll interval after the
first sweep, applied to both PeriodicPoller and FSPoller's sweep
loop. Subsequent sweeps run on the normal fixed interval, now
staggered across sources.
2026-06-01 07:02:42 -04:00
Yiorgis Gozadinos
ba623b0862
mark event-loop glue + defensive guards as no-cover 2026-05-26 11:44:46 +03:00
Yiorgis Gozadinos
75c3896588
additional config, pollers, serve 2026-05-26 11:41:54 +03:00