Commit graph

85 commits

Author SHA1 Message Date
bryan davis
faa97f8bc6
improve concurrency management for pdf attachments 2026-06-18 14:30:02 -05:00
Yiorgis Gozadinos
1e65d36766
Pass fetched FetchResult to ingester metadata providers 2026-06-16 16:37:26 +03:00
Yiorgis Gozadinos
70c9c2778f
Serialize all in-process pdfium access under a shared lock 2026-06-14 10:50:42 +03:00
Yiorgis Gozadinos
2b8b9477b7
Serialize and roll back the multi-table document delete 2026-06-12 10:17:18 +03:00
Yiorgis Gozadinos
f2a5ac4246
Throttle background auto-vacuum to at most once per 5 minutes 2026-06-12 10:17:18 +03:00
Yiorgis Gozadinos
5d5d87d44c
Throttle background auto-vacuum to at most once per 5 minutes 2026-06-12 10:17:18 +03:00
Yiorgis Gozadinos
3366a6d383
Split meta document attributes into a document_meta table 2026-06-12 10:17:17 +03:00
Yiorgis Gozadinos
4bc52f0710
Serialize multi-table document writes and bound update version churn 2026-06-12 10:16:39 +03:00
Yiorgis Gozadinos
8ac670714d
Tidy PDF attachment extension fix 2026-06-12 09:44:11 +03:00
bryan davis
bce24e84a0
fix for issue with parsing pdf attachment such as *.joboptions files 2026-06-11 16:52:07 -05:00
Yiorgis Gozadinos
c62166b26e
Add /database and /config endpoints to the ingester API 2026-06-09 12:11:33 +03:00
Yiorgis Gozadinos
a7ed405d75
Add HaikuRAG.import_documents for batch document import 2026-06-09 10:23:37 +03:00
Yiorgis Gozadinos
0e3d53791f
Add rebuild --set-embedder to reconcile embedder identity 2026-06-05 10:42:15 +03:00
Yiorgis Gozadinos
651b22ddcf
Fix rebuild --embed-only corrupting picture embeddings 2026-06-03 11:46:43 +03:00
Yiorgis Gozadinos
e7c7df2915
Always use the Store-owned embedder 2026-05-29 11:36:53 +03:00
Yiorgis Gozadinos
37a78a4e9b
Cache reranker on the client instead of rebuilding per search 2026-05-29 10:33:50 +03:00
Yiorgis Gozadinos
402957d3e5
Fix CI flakes for PDF attachment extraction 2026-05-28 17:51:02 +03:00
Yiorgis Gozadinos
8e8c4433bd
Extract PDF /EmbeddedFiles attachments as child documents 2026-05-28 15:51:32 +03:00
Yiorgis Gozadinos
26bc71d6d8
Cascade delete_document to children via metadata.parent_uri 2026-05-28 15:36:00 +03:00
Yiorgis Gozadinos
b0d0ac588d
Split snapshot APIs and resolve_fetcher by intent 2026-05-27 14:39:54 +03:00
Yiorgis Gozadinos
d7fdd61fed
Resolve worker source by source_id, not just supports(uri) 2026-05-27 13:36:03 +03:00
Yiorgis Gozadinos
f89cc998eb
add sources_provider to WorkerPool. workers now resolve extra info through these sources 2026-05-26 11:45:35 +03:00
Yiorgis Gozadinos
73f8349a00
Use haiku.rag as telemtry scope 2026-05-26 11:44:47 +03:00
Yiorgis Gozadinos
ed36cc2230
typed UnsupportedSourceError replaces pipeline string-marker matching 2026-05-26 11:44:46 +03:00
Yiorgis Gozadinos
6bcc2f6357
PDF split-convert-merge for memory-bound large PDFs 2026-05-26 11:44:45 +03:00
Yiorgis Gozadinos
13cbadeb6f
canonical metadata keys: source_revision + content_type, bump to 0.50.0 2026-05-26 11:43:30 +03:00
Yiorgis Gozadinos
fa672c298a
logfire spans for the ingester, nested poller/job/document traces 2026-05-26 11:42:42 +03:00
Yiorgis Gozadinos
6d15f1f247
fix URL-decode file:// paths before stat/exists checks 2026-05-26 11:42:42 +03:00
Yiorgis Gozadinos
7ea61a7b10
drop the old monitor, rename serve→mcp, add e2e tests 2026-05-26 11:42:41 +03:00
Yiorgis Gozadinos
717fc3320b
route create_document_from_source through source adapters 2026-05-26 11:41:54 +03:00
Yiorgis Gozadinos
8c57a3ca99
Drop the multi-agent research workflow 2026-05-20 12:46:48 +03:00
Yiorgis Gozadinos
e3314e36fd
search: only attach picture bytes from pre-expansion chunks 2026-05-19 15:03:22 +03:00
Yiorgis Gozadinos
6f95e2bc27
Delete the standalone QA and analysis agents 2026-05-19 11:39:20 +03:00
Yiorgis Gozadinos
947a26b391
client.analyze routes through the rag-analysis skill; drop documents= and AnalysisResult.program. Re-record all cassettes that are relevant 2026-05-19 10:46:32 +03:00
Yiorgis Gozadinos
d96d2eeb0f
client.ask routes through the rag skill; always show citations in CLI 2026-05-19 10:19:39 +03:00
Yiorgis Gozadinos
3b6bef3bfb
Rename get_captions_for_chunk() to get_text_for_refs() 2026-05-18 16:49:44 +03:00
Yiorgis Gozadinos
514b0c51f8
surface figure captions in search results, lower search.limit to 5 2026-05-18 16:49:43 +03:00
Yiorgis Gozadinos
68388032be
consolidate #/pictures/ prefix, tighten CHANGELOG, log migration exc_info 2026-05-18 16:49:43 +03:00
Yiorgis Gozadinos
f208730a5b
Merge pull request #374 from tseaver/feat-cli-search_type
feat: add '--search-type' option to CLI 'search'
2026-05-18 16:39:28 +03:00
Tres Seaver
8026693450
chore: ruff format 2026-05-18 09:31:38 -04:00
Tres Seaver
f9ce3bc401
refactor: move 'SearchType' to 'store.models.chunk' 2026-05-18 09:27:34 -04:00
Tres Seaver
d7e96f2658
fix: typo 2026-05-18 09:17:53 -04:00
Yiorgis Gozadinos
04415a9d09
bump pydantic-monty, refresh deps, cap transformers<5 2026-05-18 16:02:55 +03:00
Tres Seaver
b331dd0b76
fix: default 'search_type' to 'None' in 'client.search.search'
Apply 'hybrid' default only for text queries.
2026-05-18 06:52:03 -04:00
Tres Seaver
7953de46b9
fix: declare 'SearchType' for 'search_type' arg
fix: allow 'search_type' only for text searches
2026-05-18 06:35:51 -04:00
Yiorgis Gozadinos
57f273e000
Minor fixes, CI should build 2026-05-14 15:43:36 +03:00
Yiorgis Gozadinos
519afe6709
Add cross-encoder reranking provider 2026-05-14 15:43:35 +03:00
Yiorgis Gozadinos
dbb87e091f
Ensure schema parity for _StagingChunkRecord 2026-05-14 15:11:18 +03:00
Yiorgis Gozadinos
849eca94d0
Make rebuild --embed-only idempotent across crashes 2026-05-14 15:02:12 +03:00
Yiorgis Gozadinos
a035b0f9e4
Stream rebuild --embed-only through a staging table to bound memory 2026-05-14 15:02:12 +03:00