Commit graph

795 commits

Author SHA1 Message Date
Yiorgis Gozadinos
2fd025951c
Treat empty env vars as unset in config expansion 2026-06-25 12:41:04 +03:00
Yiorgis Gozadinos
9f4dda9254
Reuse one MarkdownDocSerializer per document in item extraction 2026-06-24 12:02:13 +03:00
Yiorgis Gozadinos
2b2b475279
Collapse docling compression to a single function 2026-06-24 10:43:27 +03:00
bryan davis
e95ac2e25d
remove redundant serialization 2026-06-24 10:42:58 +03:00
bryan davis
2ca12c0096
adjust zstd handling to avoid possible core dumps with concurrency 2026-06-23 15:26:33 -05:00
Yiorgis Gozadinos
e1fbaf8aad
Re-record remaining docling-serve cassettes against 1.25.0 2026-06-23 16:44:58 +03:00
Yiorgis Gozadinos
3cc3c98986
Store docling-serve chunk bodies via raw_text 2026-06-23 16:35:21 +03:00
Yiorgis Gozadinos
68715c9657
Request page images explicitly from docling-serve 2026-06-23 16:15:02 +03:00
Yiorgis Gozadinos
f22341f003
Fix doctor's vector-index check to match documented guidance 2026-06-23 15:39:44 +03:00
Yiorgis Gozadinos
3586ac30a7
Add Cohere multimodal embedder 2026-06-23 15:23:10 +03:00
Yiorgis Gozadinos
a1ec310bf4
Add VoyageAI multimodal embedder 2026-06-23 15:23:10 +03:00
Yiorgis Gozadinos
0ea251219c
Add explicit multimodal embedding flag, decoupled from provider name 2026-06-23 15:23:10 +03:00
Yiorgis Gozadinos
4fae733a50
Fix doctor provider checks: custom endpoints and processing models 2026-06-23 11:48:21 +03:00
Yiorgis Gozadinos
eb855f827a
Refine doctor's content and coverage checks 2026-06-23 11:31:20 +03:00
Yiorgis Gozadinos
cc1d8d1e4c
Add provider connectivity probes to haiku-rag doctor 2026-06-23 10:34:27 +03:00
Yiorgis Gozadinos
c2cb3cedf3
Add haiku-rag doctor database health check 2026-06-23 10:03:27 +03:00
Yiorgis Gozadinos
0ee36a269d
Merge pull request #460 from mcdonc/thread-processing-iterate
Thread picture chunk merging off the asyncio event loop
2026-06-23 09:05:45 +03:00
Chris McDonough
633517bf04 Widen chunk() type to accept None, add test for None guard 2026-06-22 14:24:12 -04:00
Chris McDonough
c59f88bcd6 Add test for _merge_picture_chunks no-pictures branch 2026-06-22 14:13:02 -04:00
Chris McDonough
21a8f52893 Use patch.object to satisfy type checker 2026-06-22 14:09:01 -04:00
Chris McDonough
99a200c9d2 Restore None guard, add thread-safety test for _chunk_sync 2026-06-22 14:05:56 -04:00
Chris McDonough
25b0a155b0 Remove None guard and test — type system guarantees document is not None 2026-06-22 14:04:58 -04:00
Chris McDonough
256fd66b46 Add test for chunker.chunk(None) to cover guard clause 2026-06-22 13:58:37 -04:00
Yiorgis Gozadinos
463c55673e
improve coverage 2026-06-22 12:59:44 +03:00
Yiorgis Gozadinos
baf8decb27
Show progress for run-batch drains 2026-06-22 12:51:38 +03:00
Yiorgis Gozadinos
b0706204bd
Allow resuming run-batch manifest replay 2026-06-22 12:22:40 +03:00
Yiorgis Gozadinos
68bbf94577
tighten run-batch manifest replay validation 2026-06-22 11:58:32 +03:00
Yiorgis Gozadinos
7251d104c4
Add run-batch manifest replay 2026-06-22 11:39:12 +03:00
Yiorgis Gozadinos
a3b542dba8
Add run-batch dry-run manifest output 2026-06-22 11:33:26 +03:00
Yiorgis Gozadinos
4b573bfebd
Add side-effect-free batch dry-run discovery 2026-06-22 11:33:26 +03:00
Yiorgis Gozadinos
ad3b111cf1
test: clarify off-loop thread assertions 2026-06-22 10:51:35 +03:00
Yiorgis Gozadinos
d3a1011baf
write fetched bodies off the event loop 2026-06-22 10:37:48 +03:00
Yiorgis Gozadinos
c6514c9df4
compare off-loop work against actual event-loop thread, fix ty 2026-06-22 10:17:02 +03:00
bryan davis
fe954a090e
move cpu bound actions off of main loop 2026-06-18 16:35:48 -05:00
bryan davis
faa97f8bc6
improve concurrency management for pdf attachments 2026-06-18 14:30:02 -05:00
Yiorgis Gozadinos
1e65d36766
Pass fetched FetchResult to ingester metadata providers 2026-06-16 16:37:26 +03:00
Yiorgis Gozadinos
6add2780e8
Make ingester job retry idempotent against a live sibling 2026-06-16 16:05:05 +03:00
Yiorgis Gozadinos
79a4f49387
Stop re-enqueuing permanently-failed ingester documents 2026-06-16 16:03:51 +03:00
Yiorgis Gozadinos
515f42dd5a
Add custom ingester sources via entry points 2026-06-16 10:54:24 +03:00
Yiorgis Gozadinos
7e20b47e98
Widen SQLite ingester queue pool to serve concurrent connections 2026-06-16 09:25:01 +03:00
Yiorgis Gozadinos
fac62cb347
Follow HTTP redirects in the WebDAV source 2026-06-15 09:59:13 +03:00
Yiorgis Gozadinos
cc73a8629a
Wire metadata providers into ingester ingestion 2026-06-15 08:41:26 +03:00
Yiorgis Gozadinos
5722260857
Add metadata-provider discovery for the ingester 2026-06-15 08:05:45 +03:00
Yiorgis Gozadinos
70c9c2778f
Serialize all in-process pdfium access under a shared lock 2026-06-14 10:50:42 +03:00
Yiorgis Gozadinos
885e7b7ce7
coverage 2026-06-12 10:17:19 +03:00
Yiorgis Gozadinos
2b8b9477b7
Serialize and roll back the multi-table document delete 2026-06-12 10:17:18 +03:00
Yiorgis Gozadinos
f2a5ac4246
Throttle background auto-vacuum to at most once per 5 minutes 2026-06-12 10:17:18 +03:00
Yiorgis Gozadinos
5d5d87d44c
Throttle background auto-vacuum to at most once per 5 minutes 2026-06-12 10:17:18 +03:00
Yiorgis Gozadinos
df8af54298
rebase from main 2026-06-12 10:17:18 +03:00
Yiorgis Gozadinos
3366a6d383
Split meta document attributes into a document_meta table 2026-06-12 10:17:17 +03:00