Commit graph

1794 commits

Author SHA1 Message Date
Yiorgis Gozadinos
076916a8cc
Add CI workflow to build RAG database on release 2026-06-01 10:35:52 +03:00
Yiorgis Gozadinos
a0029e9ab9
Merge pull request #390 from ggozad/feat/ingester-run-batch
Add haiku-ingester run-batch, remove run-once
2026-06-01 10:34:57 +03:00
Yiorgis Gozadinos
5a23e4eda6
Surface failed discovery sweeps in run-batch 2026-06-01 10:27:31 +03:00
Yiorgis Gozadinos
6f2a40c676
cover run-batch, serve, and _stop_pool with tests 2026-05-29 17:43:58 +03:00
Yiorgis Gozadinos
5ba7838b71
Add haiku-ingester run-batch, remove run-once 2026-05-29 17:00:59 +03:00
Yiorgis Gozadinos
62be23d130
vb 2026-05-29 11:47:12 +03:00
Yiorgis Gozadinos
33cfa11484
Merge pull request #388 from ggozad/chore/consolidate-embedder-cache
Always re-use the Store-owned embedder
2026-05-29 11:45:09 +03:00
Yiorgis Gozadinos
e7c7df2915
Always use the Store-owned embedder 2026-05-29 11:36:53 +03:00
Yiorgis Gozadinos
0a9a444996
Merge pull request #387 from ggozad/feat/cache_reranker
Cache reranker on the client instead of rebuilding per search
2026-05-29 10:50:45 +03:00
Yiorgis Gozadinos
37a78a4e9b
Cache reranker on the client instead of rebuilding per search 2026-05-29 10:33:50 +03:00
Yiorgis Gozadinos
c0c83d8037
Merge pull request #386 from ggozad/feat/pdf-containers
Support for embedded documents inside pdfs.
2026-05-29 10:25:05 +03:00
Yiorgis Gozadinos
d8d6727227
improve coverage 2026-05-28 18:06:47 +03:00
Yiorgis Gozadinos
402957d3e5
Fix CI flakes for PDF attachment extraction 2026-05-28 17:51:02 +03:00
Yiorgis Gozadinos
6eee09743b
Cover PDF attachment ingest through the full create_document_from_source path 2026-05-28 16:07:12 +03:00
Yiorgis Gozadinos
8e8c4433bd
Extract PDF /EmbeddedFiles attachments as child documents 2026-05-28 15:51:32 +03:00
Yiorgis Gozadinos
26bc71d6d8
Cascade delete_document to children via metadata.parent_uri 2026-05-28 15:36:00 +03:00
Yiorgis Gozadinos
e04a425d78
Merge pull request #385 from ggozad/feat/auto-prune
Auto-prune dead jobs when a sibling DELETE succeeds
2026-05-27 17:31:37 +03:00
Yiorgis Gozadinos
5400085147
Auto-prune dead jobs when a sibling DELETE succeeds 2026-05-27 17:17:01 +03:00
Yiorgis Gozadinos
3828a91c37
vb 2026-05-27 16:23:43 +03:00
Yiorgis Gozadinos
f6364126ec
Merge pull request #383 from ggozad/feat/ingester
Ingestion service for production, replaces monitor
2026-05-27 16:13:49 +03:00
Yiorgis Gozadinos
561010d1ec
Add ingester screenshot 2026-05-27 15:53:01 +03:00
Yiorgis Gozadinos
4d1b89de8f
Custom hover tooltips for truncated dashboard cells 2026-05-27 15:42:17 +03:00
Yiorgis Gozadinos
1e6dc34871
Skip FS DELETE enqueue when the file is already back 2026-05-27 15:18:46 +03:00
Yiorgis Gozadinos
15d13cab04
Probe docling-serve only when converter or chunker uses it 2026-05-27 15:18:25 +03:00
Yiorgis Gozadinos
b0d0ac588d
Split snapshot APIs and resolve_fetcher by intent 2026-05-27 14:39:54 +03:00
Yiorgis Gozadinos
2c63ceda0c
Backfill ingester test gaps and drop unneeded retry clamp 2026-05-27 14:24:22 +03:00
Yiorgis Gozadinos
cfbd0b09d6
Tighten HTTP config-removal handling 2026-05-27 14:07:45 +03:00
Yiorgis Gozadinos
7466539b4f
Emit DELETE for HTTP URLs removed from config 2026-05-27 13:39:18 +03:00
Yiorgis Gozadinos
d7fdd61fed
Resolve worker source by source_id, not just supports(uri) 2026-05-27 13:36:03 +03:00
Yiorgis Gozadinos
922d1d567d
Prevent DELETE/UPSERT race for the same URI 2026-05-27 13:30:46 +03:00
Yiorgis Gozadinos
0114653522
Docs update 2026-05-27 13:22:35 +03:00
Yiorgis Gozadinos
d059b59c1e
Adapt docker compose example to use the published image 2026-05-27 12:59:32 +03:00
Yiorgis Gozadinos
1c710433c7
Collapse worker_count and max_concurrent into worker_count 2026-05-27 12:48:14 +03:00
Yiorgis Gozadinos
6f74fe67b3
Cap claimed-row count at max_concurren 2026-05-27 12:10:42 +03:00
Yiorgis Gozadinos
87af5e5139
Boot-reap stale claims at WorkerPool start 2026-05-27 11:46:37 +03:00
Yiorgis Gozadinos
1b36452629
Surface provider reachability and per-job failure context on dashboard 2026-05-27 11:38:48 +03:00
Yiorgis Gozadinos
439307d5af
Add pool-wide circuit breaker to the worker pool 2026-05-27 10:52:48 +03:00
Yiorgis Gozadinos
b491f767f3
Let httpx errors flow through DoclingServeClient 2026-05-26 17:38:49 +03:00
Yiorgis Gozadinos
fdb73fc41f
Drop defensive branches 2026-05-26 16:56:32 +03:00
Yiorgis Gozadinos
291b565b4c
Stream dashboard index.html via FileResponse 2026-05-26 16:31:41 +03:00
Yiorgis Gozadinos
29b3ccdbda
Drain orphan cancel-cleanup releases before closing the queue 2026-05-26 16:14:17 +03:00
Yiorgis Gozadinos
960bf8f3e3
Document auth_token for non-loopback 2026-05-26 16:07:53 +03:00
Yiorgis Gozadinos
3c608046e2
Bound ?limit and ?offset on /jobs and /dlq 2026-05-26 16:05:47 +03:00
Yiorgis Gozadinos
761956eb70
Extend reaper-resurrection guard to reschedule and release_if_claimed 2026-05-26 16:04:24 +03:00
Yiorgis Gozadinos
6799a4a6d4
Document worker-pool settings 2026-05-26 14:13:16 +03:00
Yiorgis Gozadinos
a822b754b7
Guard mark_succeeded/mark_dead against reaper resurrection 2026-05-26 14:07:44 +03:00
Yiorgis Gozadinos
9cb3ce40ad
Shield cancel-cleanup release in worker pool 2026-05-26 13:57:35 +03:00
Yiorgis Gozadinos
78fc5d0e05
Discover within-root symlinks in FSSource 2026-05-26 13:54:33 +03:00
Yiorgis Gozadinos
e5e0df6ade
Unlink PDF slice tempfiles when slice write fails 2026-05-26 13:46:24 +03:00
Yiorgis Gozadinos
98efd73177
Serialize JobRepo and SyncStateRepo on one shared lock 2026-05-26 13:11:01 +03:00