Commit graph

346 commits

Author SHA1 Message Date
Yiorgis Gozadinos
97a7eb246c
vb 2026-05-14 16:03:33 +03:00
Yiorgis Gozadinos
519afe6709
Add cross-encoder reranking provider 2026-05-14 15:43:35 +03:00
Yiorgis Gozadinos
dbb87e091f
Ensure schema parity for _StagingChunkRecord 2026-05-14 15:11:18 +03:00
Yiorgis Gozadinos
849eca94d0
Make rebuild --embed-only idempotent across crashes 2026-05-14 15:02:12 +03:00
Yiorgis Gozadinos
a035b0f9e4
Stream rebuild --embed-only through a staging table to bound memory 2026-05-14 15:02:12 +03:00
Yiorgis Gozadinos
862a209137
vb 2026-05-13 16:53:35 +03:00
Yiorgis Gozadinos
a98ddc14b8
Replace picture_description.enabled with processing.pictures enum 2026-05-13 16:22:44 +03:00
Yiorgis Gozadinos
bdab201c42
Merge pull request #371 from ggozad/fix/image-results
Verify picture bytes before attaching to multimodal tool returns
2026-05-13 16:19:09 +03:00
Yiorgis Gozadinos
a6e5664c78
Verify picture bytes before attaching to multimodal tool returns 2026-05-13 16:01:39 +03:00
Yiorgis Gozadinos
072edad62f
Merge pull request #368 from ggozad/feat/relax-compat-check
Relax embedding compatibility check to vector_dim only
2026-05-13 15:43:37 +03:00
Yiorgis Gozadinos
701e9d3632
Relax embedding compat check to vector_dim only 2026-05-13 14:44:18 +03:00
Yiorgis Gozadinos
8b135c4d4d
Expose embedding batch size as config 2026-05-13 14:09:47 +03:00
Yiorgis Gozadinos
c6cd847299
Add ModelConfig.extra_body for raw provider pass-through 2026-05-13 13:45:09 +03:00
Yiorgis Gozadinos
0f4c4af495
Cache the chunking tokenizer to avoid HF Hub 429 2026-05-13 13:32:24 +03:00
Yiorgis Gozadinos
d512f9fbd1
Disable Typer locals in CLI tracebacks 2026-05-13 13:24:05 +03:00
Yiorgis Gozadinos
674b80e8c9
Support streaming for markdown 2026-05-12 16:01:42 +03:00
Yiorgis Gozadinos
a206f8bfcc
Document fetch_remote_images and the docling-serve HTML gap 2026-05-12 15:50:33 +03:00
Yiorgis Gozadinos
afd4665517
Thread source_uri through URL ingest 2026-05-12 15:13:02 +03:00
Yiorgis Gozadinos
a25d9699ee
Wire per-format options across docling-local 2026-05-12 15:02:59 +03:00
Yiorgis Gozadinos
6f6ebf27c8
Document s3 storage & monitoring 2026-05-11 11:29:56 +03:00
Yiorgis Gozadinos
8b8f44e7b2
vb 2026-05-08 12:22:35 +03:00
Yiorgis Gozadinos
ab5cfdd04a
Docs & cl 2026-05-08 11:20:04 +03:00
Yiorgis Gozadinos
61a88edfd8
drop bogus deprecation warning for processing.pictures 2026-05-05 14:46:41 +03:00
Yiorgis Gozadinos
ff82d36c2f
add rebuild --descriptions: run VLM over stored picture bytes only 2026-05-05 12:02:46 +03:00
Yiorgis Gozadinos
2038d43435
collapse pictures enum to picture_description.enabled boolean 2026-05-05 11:16:08 +03:00
Yiorgis Gozadinos
65d9c74224
expose image-as-query through MCP and the CLI. 2026-05-04 11:46:45 +03:00
Yiorgis Gozadinos
ff656504d3
make client.search() polymorphic on query type: str | bytes | PIL.Image.Image. Bytes/PIL queries embed via embed_image_query and run vector-only against the chunks table 2026-05-04 11:22:22 +03:00
Yiorgis Gozadinos
21f12cd770
add vision: bool flag on ModelConfig (default False). Gate the agent search tool's BinaryContent attachment on qa.model.vision so picture bytes are only sent to vision-capable QA models. 2026-05-04 10:47:21 +03:00
Yiorgis Gozadinos
6037f26673
drop mlx 2026-05-03 19:05:49 +03:00
Yiorgis Gozadinos
c9227c649f
emit synthetic picture chunks at ingest under multimodal embedders.
processing.chunk() merges text chunks with one synthetic Chunk per PictureItem-with-bytes,
sorted by iterate_items() position so chunk.order is structural.
embed_chunks dispatches on a Chunk._picture_data PrivateAttr
(text through embed_documents, picture through embed_image_query)
2026-05-03 17:04:43 +03:00
Yiorgis Gozadinos
37215f1a00
add multimodal embedder support for vllm and mlx 2026-05-03 10:57:56 +03:00
Yiorgis Gozadinos
861c36942b
cl 2026-04-30 15:50:28 +03:00
Yiorgis Gozadinos
7831057339
add processing.pictures enum (none|description|image), replacing the implicit pair of generate_picture_images+picture_description.enabled flags 2026-04-30 14:26:14 +03:00
Yiorgis Gozadinos
b01c649684
surface picture image bytes in SearchResult and emit multimodal ToolReturn from the agent search tool 2026-04-30 12:43:05 +03:00
Yiorgis Gozadinos
6a77ce92a9
extract picture bytes to document_items.picture_data at ingest, strip them from the docling_document blob, and add 0.45.0 migration to backfill existing
databases
2026-04-30 11:26:37 +03:00
Yiorgis Gozadinos
dc16f74b58
add picture_data column to document_items 2026-04-30 11:23:29 +03:00
Yiorgis Gozadinos
29bc42514e
vb 2026-04-29 13:45:36 +03:00
Yiorgis Gozadinos
318266e681
Update benchmark 2026-04-29 13:27:32 +03:00
Yiorgis Gozadinos
8962987ff9
pin judge model to ollama:qwen3.6 2026-04-29 12:24:06 +03:00
Yiorgis Gozadinos
5e31c15907
docs 2026-04-28 14:44:27 +03:00
Yiorgis Gozadinos
6807f082ec
vb 2026-04-25 09:35:47 +03:00
Yiorgis Gozadinos
758bd3ee72
Group relative-path test with the other connect_lancedb dispatch tests 2026-04-25 09:28:46 +03:00
Yiorgis Gozadinos
81ba248483
vb 2026-04-24 17:47:57 +03:00
Yiorgis Gozadinos
821b7361e9
Push document-id filter into chunk search query 2026-04-24 16:07:07 +03:00
Yiorgis Gozadinos
cd4224a727
drop private title/embedding forwarders from HaikuRAG 2026-04-24 15:52:50 +03:00
Yiorgis Gozadinos
ce2df2e0bd
Docs 2026-04-24 14:42:52 +03:00
Yiorgis Gozadinos
82fd10e0ee
Migrate LanceDB to native async API
Convert all LanceDB operations from sync calls wrapped in async
functions to the native async API (connect_async, AsyncConnection,
AsyncTable, AsyncQuery). Database I/O no longer blocks the event loop.

- Store and HaikuRAG use async context managers (async with). Store
  initialization is deferred to __aenter__; direct construction
  without async with is no longer supported.
- Index creation uses config objects (FTS, BTree, IvfPq) instead of
  string-based index_type parameter.
- Upgrade callbacks are async.
- HaikuRAG tracks background vacuum tasks and awaits them in __aexit__
  and before destructive rebuild operations to avoid races with
  concurrent table mutations.
- temp_db_path fixture uses pytest's tmp_path for reliable async
  cleanup.
2026-04-24 14:42:52 +03:00
Yiorgis Gozadinos
cb9360ef7b
fix chat TUI citation rendering after state flattening 2026-04-24 13:48:57 +03:00
Yiorgis Gozadinos
4f058bf484
vb 2026-04-22 17:00:03 +03:00
Yiorgis Gozadinos
ae2b8461be
flatten skill state citations from list[list[str]] to list[str] 2026-04-22 16:50:13 +03:00