Yiorgis Gozadinos
3603b00513
Fix base64 images leaking into expanded search results
...
PictureItem.export_to_markdown() defaults to ImageRefMode.EMBEDDED,
which embeds base64 image data. When _extract_item_text() called this
method during context expansion, base64 data would leak into results.
Now explicitly handles PictureItem with ImageRefMode.PLACEHOLDER to
prevent base64 while still including VLM descriptions and captions.
2026-01-08 18:42:58 +02:00
Yiorgis Gozadinos
2f413e937b
Add VLM model to download_models()
2026-01-07 11:09:24 +02:00
Yiorgis Gozadinos
c9cb005d07
Use FTS on contextualized text
2025-12-29 14:50:07 +02:00
Yiorgis Gozadinos
3852e961b9
Update tests and client for explicit embed_query/embed_documents API
2025-12-26 11:26:56 +02:00
Yiorgis Gozadinos
77f9a2a1b9
before parameter in client, app, inspector, cli
2025-12-19 12:03:18 +02:00
Yiorgis Gozadinos
98727ad080
Add global --read-only mode in CLI
2025-12-18 16:43:49 +02:00
Yiorgis Gozadinos
45de9bf0d5
Flatten graph structure, simplify
2025-12-18 11:41:57 +02:00
Yiorgis Gozadinos
50b7fb4461
Chunkers now set chunk.order directly
2025-12-18 11:35:48 +02:00
Yiorgis Gozadinos
3149552a61
Add 'plain' format and fallback for text conversion
2025-12-12 18:11:25 +02:00
Yiorgis Gozadinos
d7c755f454
Add filter to simple qa agent
2025-12-11 15:28:41 +02:00
Yiorgis Gozadinos
2161727716
Add auto_vacuum config option to control automatic vacuuming
2025-12-10 11:42:14 +02:00
Yiorgis Gozadinos
802f058205
Move search related config under SearchConfig
2025-12-09 12:28:28 +02:00
Yiorgis Gozadinos
4da50873bd
Make search limit configurable
2025-12-09 11:46:43 +02:00
Yiorgis Gozadinos
a56e1ba67c
Simplify import_document, update_document
2025-12-08 18:19:12 +02:00
Yiorgis Gozadinos
67684c799b
Rebuild in batches to avoid creating multiple lancedb versions
2025-12-08 15:56:41 +02:00
Yiorgis Gozadinos
06b4f6a224
Add format parameter for text-to-DoclingDocument conversion
2025-12-08 15:56:21 +02:00
Yiorgis Gozadinos
1b14a4643d
Remove unused bounding box calculations
2025-12-08 15:56:21 +02:00
Yiorgis Gozadinos
9a5f8e4368
Add type-aware context expansion for search results
2025-12-08 15:56:21 +02:00
Yiorgis Gozadinos
2572125804
Move embedding logic from ChunkRepository to client._ensure_chunks_embedded()
2025-12-08 15:56:02 +02:00
Yiorgis Gozadinos
e97d235c98
Unify update_document, update_document_fields
2025-12-08 15:56:02 +02:00
Yiorgis Gozadinos
16ba3a9962
Remove create_chunks_for_document from repositories
2025-12-08 15:56:01 +02:00
Yiorgis Gozadinos
69b1afa534
Remove _create_and_chunk & _update_and_rechunk, keep storage independent of chunking
2025-12-08 15:55:31 +02:00
Yiorgis Gozadinos
0ab43d62cd
Refactor create_document(), create_document_from_source() to use primitives
2025-12-08 15:55:30 +02:00
Yiorgis Gozadinos
4fbb4609b6
HaikuRag.chunk() utility method
2025-12-08 15:55:30 +02:00
Yiorgis Gozadinos
ace915473c
HaikuRag.convert() utility method
2025-12-08 15:55:30 +02:00
Yiorgis Gozadinos
33e6a36290
Make document import/update handling consistent for DoclingDocument
2025-12-08 15:55:29 +02:00
Yiorgis Gozadinos
60ab864fc1
Add docling_document_json to update_document_fields() and
2025-12-08 15:55:29 +02:00
Yiorgis Gozadinos
8e30f67266
Introduce import_document(), remove chunks from create_document()
2025-12-08 15:55:29 +02:00
Yiorgis Gozadinos
910d382748
Introduce visualize_chunk(), refactor inspector to use it
2025-12-08 15:55:27 +02:00
Yiorgis Gozadinos
d808c6c425
Simplify citations in qa & graph agents
2025-12-08 15:55:27 +02:00
Yiorgis Gozadinos
16a97cc140
Properly expand context in the case of docling documents and merge chunks and meta
2025-12-08 15:55:26 +02:00
Yiorgis Gozadinos
c69b934eac
Incorporate additional docling meta in search(), expand_context() and relevant qa/research agents
2025-12-08 15:54:56 +02:00
Yiorgis Gozadinos
84aa46a10b
Update rebuild to handle DoclingDocument json
2025-12-08 15:54:31 +02:00
Yiorgis Gozadinos
ad4e79ab5d
Store DoclingDocument json
2025-12-08 15:54:31 +02:00
Yiorgis Gozadinos
80447c9024
Make download_models() a HaikuRAG client method. Update app to show progress of what happens.
2025-12-04 16:30:26 +02:00
Yiorgis Gozadinos
33b254ae3d
Always upgrade database on access. Explicitly create db by running init or HaikuRAG(path, create=True)
2025-12-02 16:49:55 +02:00
Yiorgis Gozadinos
e548a51ff7
Optimized rebuild --embed-only to use batch updates via LanceDB merge_insert instead of individual chunk updates
2025-12-01 17:53:27 +02:00
Yiorgis Gozadinos
88a4edb537
--embed-only, --rechunk options in rebuild command
2025-11-28 10:11:52 +02:00
Yiorgis Gozadinos
14bdb7cb27
async versions for convert_file() convert_text()
2025-11-26 17:46:25 +02:00
Yiorgis Gozadinos
2f54ee7d59
Replace allow_create with cleaner read_only. Elaborate on the test_info tests
2025-11-21 14:37:33 +02:00
Yiorgis Gozadinos
30a0bbcff2
Accept both single chunks and lists for batch insertion in ChunkRepository.create()
2025-11-19 12:35:02 +02:00
Yiorgis Gozadinos
5089ea4203
New update_document_fields() method for partial document updates
2025-11-19 11:37:36 +02:00
Yiorgis Gozadinos
cc09ce87c2
Increase reranking candidate retrieval multiplier from 3x to 10x for improved result quality
2025-11-18 14:35:57 +02:00
Yiorgis Gozadinos
7ba1c2376e
Introduce converters for supporting more than local docling document conversion. Transform existing FileReader and utils to "docling-local" converter
2025-11-17 13:05:05 +02:00
Yiorgis Gozadinos
4599dd1481
Drop disable_auto_create flag, simply create only on write operations
2025-11-13 13:53:11 +02:00
Yiorgis Gozadinos
908296e155
Do not create unecessary DoclingDocument when chunks are provided. Rename private _create_with_docling & _update_with_docling to _create_and_chunk & _update_and_rechunk
2025-11-13 13:16:15 +02:00
Yiorgis Gozadinos
31713e5fb6
Properly skip (and log as such) documents without changing updated_at when the md5 hash is unchanged
2025-11-07 14:00:05 +02:00
Yiorgis Gozadinos
2f9c907031
Restructure into uv workspace to support minimal and full installations
2025-11-04 17:59:12 +02:00
