Commit graph

10 commits

Author SHA1 Message Date
Chris McDonough
adf03284ff
Classify IsADirectoryError and NotADirectoryError as PermanentError
Both are OSError subclasses caught by the broad timeout/io handler
and classified as transient. Pointing at a directory instead of a
file or a broken path component will never succeed on retry.
2026-06-01 17:59:23 +03:00
Chris McDonough
3c7dbf0966
Classify PermissionError as PermanentError instead of TransientError
PermissionError is a subclass of OSError, so it was caught by the
broad timeout/io handler and classified as transient. An unreadable
file would retry 5 times then DLQ — permissions don't fix themselves
without operator intervention.

Add an explicit PermissionError check before the OSError catch so
unreadable files go straight to the DLQ.
2026-06-01 15:36:37 +03:00
Chris McDonough
c9c48bc814 Classify FileNotFoundError as PermanentError instead of TransientError
FileNotFoundError is a subclass of OSError, so it was caught by the
broad timeout/io handler and classified as transient. A file deleted
between discovery and fetch would retry 5 times on a file that's
permanently gone, then DLQ with a confusing error message.

Add an explicit FileNotFoundError check before the OSError catch so
deleted files go straight to the DLQ.
2026-06-01 07:32:09 -04:00
Yiorgis Gozadinos
2c63ceda0c
Backfill ingester test gaps and drop unneeded retry clamp 2026-05-27 14:24:22 +03:00
Yiorgis Gozadinos
d7fdd61fed
Resolve worker source by source_id, not just supports(uri) 2026-05-27 13:36:03 +03:00
Yiorgis Gozadinos
b491f767f3
Let httpx errors flow through DoclingServeClient 2026-05-26 17:38:49 +03:00
Yiorgis Gozadinos
f89cc998eb
add sources_provider to WorkerPool. workers now resolve extra info through these sources 2026-05-26 11:45:35 +03:00
Yiorgis Gozadinos
ed36cc2230
typed UnsupportedSourceError replaces pipeline string-marker matching 2026-05-26 11:44:46 +03:00
Yiorgis Gozadinos
13cbadeb6f
canonical metadata keys: source_revision + content_type, bump to 0.50.0 2026-05-26 11:43:30 +03:00
Yiorgis Gozadinos
bdcc1caa49
worker pool, rety policy, pipeline 2026-05-26 11:41:54 +03:00