Yiorgis Gozadinos
8bf1b6be1a
Merge pull request #403 from mcdonc/fix/watch-change-stat-race
...
fix: watch loop crash when file deleted before stat() in _handle_watch_change
2026-06-01 15:59:50 +03:00
Yiorgis Gozadinos
bf45c4efa8
Merge pull request #406 from mcdonc/fix/permissionerror-permanent
...
fix: classify PermissionError as PermanentError
2026-06-01 15:49:47 +03:00
Chris McDonough
3c7dbf0966
Classify PermissionError as PermanentError instead of TransientError
...
PermissionError is a subclass of OSError, so it was caught by the
broad timeout/io handler and classified as transient. An unreadable
file would retry 5 times then DLQ — permissions don't fix themselves
without operator intervention.
Add an explicit PermissionError check before the OSError catch so
unreadable files go straight to the DLQ.
2026-06-01 15:36:37 +03:00
Yiorgis Gozadinos
4372b0546d
Merge pull request #400 from mcdonc/fix/filenotfound-permanent-error
...
fix: classify FileNotFoundError as PermanentError
2026-06-01 15:35:03 +03:00
Chris McDonough
42922cd2bd
Fix watch loop crash when file is deleted before stat() in _handle_watch_change
...
The expression `str(path.stat().st_mtime_ns) if path.exists() else None`
has a TOCTOU race: the file can be deleted between exists() and stat().
The resulting FileNotFoundError propagates up to _watch_loop's except
handler, which records a breaker failure and terminates the loop — no
more push events are processed until restart.
Replace with a try/except around stat() and return early on
FileNotFoundError. The deletion event from watchfiles will handle
cleanup.
2026-06-01 07:46:35 -04:00
Chris McDonough
c9c48bc814
Classify FileNotFoundError as PermanentError instead of TransientError
...
FileNotFoundError is a subclass of OSError, so it was caught by the
broad timeout/io handler and classified as transient. A file deleted
between discovery and fetch would retry 5 times on a file that's
permanently gone, then DLQ with a confusing error message.
Add an explicit FileNotFoundError check before the OSError catch so
deleted files go straight to the DLQ.
2026-06-01 07:32:09 -04:00
Yiorgis Gozadinos
d5e5733f67
Merge pull request #389 from ggozad/chore/clean-up-evaluations
...
Clean up and update evaluations.
2026-06-01 11:07:32 +03:00
Yiorgis Gozadinos
cbd9b46da6
Results for analysis skill
2026-06-01 10:40:52 +03:00
Yiorgis Gozadinos
339d231a4e
Remove repliqa from test setup
2026-06-01 10:40:52 +03:00
Yiorgis Gozadinos
e653c49fce
Remove unecessary Mean Reciprocal Rank metric
2026-06-01 10:40:52 +03:00
Yiorgis Gozadinos
00f2a40a60
Refresh benchmarks doc and remove unused eval datasets
2026-06-01 10:40:51 +03:00
Yiorgis Gozadinos
6ac7d5a4be
Add nemotron embedder to ORB benchmarks and consolidate source buckets
2026-06-01 10:40:51 +03:00
Yiorgis Gozadinos
a0029e9ab9
Merge pull request #390 from ggozad/feat/ingester-run-batch
...
Add haiku-ingester run-batch, remove run-once
2026-06-01 10:34:57 +03:00
Yiorgis Gozadinos
5a23e4eda6
Surface failed discovery sweeps in run-batch
2026-06-01 10:27:31 +03:00
Yiorgis Gozadinos
6f2a40c676
cover run-batch, serve, and _stop_pool with tests
2026-05-29 17:43:58 +03:00
Yiorgis Gozadinos
5ba7838b71
Add haiku-ingester run-batch, remove run-once
2026-05-29 17:00:59 +03:00
Yiorgis Gozadinos
62be23d130
vb
2026-05-29 11:47:12 +03:00
Yiorgis Gozadinos
33cfa11484
Merge pull request #388 from ggozad/chore/consolidate-embedder-cache
...
Always re-use the Store-owned embedder
2026-05-29 11:45:09 +03:00
Yiorgis Gozadinos
e7c7df2915
Always use the Store-owned embedder
2026-05-29 11:36:53 +03:00
Yiorgis Gozadinos
0a9a444996
Merge pull request #387 from ggozad/feat/cache_reranker
...
Cache reranker on the client instead of rebuilding per search
2026-05-29 10:50:45 +03:00
Yiorgis Gozadinos
37a78a4e9b
Cache reranker on the client instead of rebuilding per search
2026-05-29 10:33:50 +03:00
Yiorgis Gozadinos
c0c83d8037
Merge pull request #386 from ggozad/feat/pdf-containers
...
Support for embedded documents inside pdfs.
2026-05-29 10:25:05 +03:00
Yiorgis Gozadinos
d8d6727227
improve coverage
2026-05-28 18:06:47 +03:00
Yiorgis Gozadinos
402957d3e5
Fix CI flakes for PDF attachment extraction
2026-05-28 17:51:02 +03:00
Yiorgis Gozadinos
6eee09743b
Cover PDF attachment ingest through the full create_document_from_source path
2026-05-28 16:07:12 +03:00
Yiorgis Gozadinos
8e8c4433bd
Extract PDF /EmbeddedFiles attachments as child documents
2026-05-28 15:51:32 +03:00
Yiorgis Gozadinos
26bc71d6d8
Cascade delete_document to children via metadata.parent_uri
2026-05-28 15:36:00 +03:00
Yiorgis Gozadinos
e04a425d78
Merge pull request #385 from ggozad/feat/auto-prune
...
Auto-prune dead jobs when a sibling DELETE succeeds
2026-05-27 17:31:37 +03:00
Yiorgis Gozadinos
5400085147
Auto-prune dead jobs when a sibling DELETE succeeds
2026-05-27 17:17:01 +03:00
Yiorgis Gozadinos
3828a91c37
vb
2026-05-27 16:23:43 +03:00
Yiorgis Gozadinos
f6364126ec
Merge pull request #383 from ggozad/feat/ingester
...
Ingestion service for production, replaces monitor
2026-05-27 16:13:49 +03:00
Yiorgis Gozadinos
561010d1ec
Add ingester screenshot
2026-05-27 15:53:01 +03:00
Yiorgis Gozadinos
4d1b89de8f
Custom hover tooltips for truncated dashboard cells
2026-05-27 15:42:17 +03:00
Yiorgis Gozadinos
1e6dc34871
Skip FS DELETE enqueue when the file is already back
2026-05-27 15:18:46 +03:00
Yiorgis Gozadinos
15d13cab04
Probe docling-serve only when converter or chunker uses it
2026-05-27 15:18:25 +03:00
Yiorgis Gozadinos
b0d0ac588d
Split snapshot APIs and resolve_fetcher by intent
2026-05-27 14:39:54 +03:00
Yiorgis Gozadinos
2c63ceda0c
Backfill ingester test gaps and drop unneeded retry clamp
2026-05-27 14:24:22 +03:00
Yiorgis Gozadinos
cfbd0b09d6
Tighten HTTP config-removal handling
2026-05-27 14:07:45 +03:00
Yiorgis Gozadinos
7466539b4f
Emit DELETE for HTTP URLs removed from config
2026-05-27 13:39:18 +03:00
Yiorgis Gozadinos
d7fdd61fed
Resolve worker source by source_id, not just supports(uri)
2026-05-27 13:36:03 +03:00
Yiorgis Gozadinos
922d1d567d
Prevent DELETE/UPSERT race for the same URI
2026-05-27 13:30:46 +03:00
Yiorgis Gozadinos
0114653522
Docs update
2026-05-27 13:22:35 +03:00
Yiorgis Gozadinos
d059b59c1e
Adapt docker compose example to use the published image
2026-05-27 12:59:32 +03:00
Yiorgis Gozadinos
1c710433c7
Collapse worker_count and max_concurrent into worker_count
2026-05-27 12:48:14 +03:00
Yiorgis Gozadinos
6f74fe67b3
Cap claimed-row count at max_concurren
2026-05-27 12:10:42 +03:00
Yiorgis Gozadinos
87af5e5139
Boot-reap stale claims at WorkerPool start
2026-05-27 11:46:37 +03:00
Yiorgis Gozadinos
1b36452629
Surface provider reachability and per-job failure context on dashboard
2026-05-27 11:38:48 +03:00
Yiorgis Gozadinos
439307d5af
Add pool-wide circuit breaker to the worker pool
2026-05-27 10:52:48 +03:00
Yiorgis Gozadinos
b491f767f3
Let httpx errors flow through DoclingServeClient
2026-05-26 17:38:49 +03:00
Yiorgis Gozadinos
fdb73fc41f
Drop defensive branches
2026-05-26 16:56:32 +03:00
