Yiorgis Gozadinos
5464d24483
require explicit migrate command for database migrations
2026-01-19 13:22:09 +02:00
Yiorgis Gozadinos
5f2f180960
vb
2026-01-16 15:27:55 +02:00
Yiorgis Gozadinos
bdfc6b87e7
Rename initial_context to background_context
2026-01-16 15:03:42 +02:00
Yiorgis Gozadinos
d2fabb9f13
Update docs
2026-01-16 15:03:41 +02:00
Yiorgis Gozadinos
12e5e9ebfb
vb
2026-01-15 16:24:03 +02:00
Yiorgis Gozadinos
7639c2915e
Add AGUI_STATE_KEY for namespaced AG-UI state emission
2026-01-15 15:46:37 +02:00
Yiorgis Gozadinos
157773ef15
vb
2026-01-15 11:47:49 +02:00
Yiorgis Gozadinos
e0da58bb6c
Add tests for settings validation and app operations
2026-01-15 10:46:29 +02:00
Yiorgis Gozadinos
fc1ea82cb1
Display on info pydantic-ai and docling-document schema version
2026-01-14 18:14:53 +02:00
Yiorgis Gozadinos
8bdfbe526c
Fix embed-only rebuild with changed vector dimensions
...
When a database was created with one embedding model and rebuild
--embed-only was run with a different model, it failed with a
vector dimension validation error.
- Store reads stored vector_dim when opening existing databases
- _rebuild_embed_only recreates chunks table to handle dimension changes
- Add test for rebuild with changed vector dimensions
2026-01-14 17:42:28 +02:00
Yiorgis Gozadinos
11f7f02262
Add option to namespace state key as a "feature" for ag-ui apps that maintain their own state
2026-01-14 14:21:08 +02:00
Yiorgis Gozadinos
7b6b2e2f38
Update docs
2026-01-14 12:55:02 +02:00
Yiorgis Gozadinos
2930679dc1
vb
2026-01-13 19:47:27 +02:00
Yiorgis Gozadinos
98b3d114b3
Update to docling-document 1.9.0
2026-01-13 19:20:27 +02:00
Yiorgis Gozadinos
275b09c8c5
vb
2026-01-13 18:34:45 +02:00
Yiorgis Gozadinos
eb5d0b0622
Use tui as an optional group. vb
2026-01-13 12:30:19 +02:00
Yiorgis Gozadinos
6b3ff5ef1e
CLI app
2026-01-13 11:31:21 +02:00
Yiorgis Gozadinos
100933b36e
update cl
2026-01-12 17:21:41 +02:00
Yiorgis Gozadinos
a9ed178a90
Research graph test cassette
2026-01-12 12:43:51 +02:00
Yiorgis Gozadinos
bb61e8fab6
Remove custom AG-UI infrastructure in favor of pydantic-ai native support
2026-01-12 12:37:07 +02:00
Yiorgis Gozadinos
747acd6f6a
vb
2026-01-12 12:08:28 +02:00
Yiorgis Gozadinos
061095e855
Remove obsolete migrations
2026-01-12 12:07:49 +02:00
Yiorgis Gozadinos
29c682d8eb
Add migration
2026-01-11 11:40:57 +02:00
Yiorgis Gozadinos
ff1f19bb08
vb
2026-01-08 18:50:39 +02:00
Yiorgis Gozadinos
3603b00513
Fix base64 images leaking into expanded search results
...
PictureItem.export_to_markdown() defaults to ImageRefMode.EMBEDDED,
which embeds base64 image data. When _extract_item_text() called this
method during context expansion, base64 data would leak into results.
Now explicitly handles PictureItem with ImageRefMode.PLACEHOLDER to
prevent base64 while still including VLM descriptions and captions.
2026-01-08 18:42:58 +02:00
Yiorgis Gozadinos
92f0fd956f
vb
2026-01-08 10:25:25 +02:00
Yiorgis Gozadinos
1616891c40
Fix non-reasoning OpenAI models that do not support reasoning config
2026-01-07 18:47:31 +02:00
Yiorgis Gozadinos
ef187767ed
vb
2026-01-07 11:20:48 +02:00
Yiorgis Gozadinos
248eed8d7b
Update docs
2026-01-07 11:09:23 +02:00
Yiorgis Gozadinos
af340ac410
vb
2026-01-05 12:15:41 +02:00
Yiorgis Gozadinos
c4b1733339
vb
2025-12-29 18:45:06 +02:00
Yiorgis Gozadinos
fc3530258d
cl
2025-12-29 14:50:59 +02:00
Yiorgis Gozadinos
5844d07c5d
Add lint github action
2025-12-27 18:50:57 +02:00
Yiorgis Gozadinos
1f654917b8
vb
2025-12-26 13:56:29 +02:00
Yiorgis Gozadinos
08fb99c6f8
Add prompt customization and domain preable
2025-12-26 13:34:51 +02:00
Yiorgis Gozadinos
958fe43f2a
Remove vllm and lmstudio custom configs, they can now use the openai base
2025-12-26 12:02:55 +02:00
Yiorgis Gozadinos
d4861b6408
Docs & changelog
2025-12-26 11:57:27 +02:00
Yiorgis Gozadinos
5c7f033d2c
vb
2025-12-19 12:35:35 +02:00
Yiorgis Gozadinos
4164895514
Update critical dependencies
2025-12-19 12:21:57 +02:00
Yiorgis Gozadinos
d7b3b6859c
document time-travel
2025-12-19 12:03:19 +02:00
Yiorgis Gozadinos
8c0adb47e1
Merge branch 'main' into feat/read-only-db
2025-12-19 12:01:36 +02:00
Yiorgis Gozadinos
090be293d2
Clarify folder mounts in docker docs, check monitor paths exist in FileWatcher
2025-12-19 11:58:08 +02:00
Yiorgis Gozadinos
98727ad080
Add global --read-only mode in CLI
2025-12-18 16:43:49 +02:00
Yiorgis Gozadinos
1dbbaf6dd8
vb
2025-12-18 12:21:16 +02:00
Yiorgis Gozadinos
a1f5d944ba
Replace custom event classes with ag_ui.core types
2025-12-18 12:07:37 +02:00
Yiorgis Gozadinos
9c766846c0
Use proper ag-ui tool calls in ag-ui-example. Fuck copilotkit
2025-12-18 11:48:35 +02:00
Yiorgis Gozadinos
ff13be1210
Human-in-the-loop in ag-ui-example
2025-12-18 11:48:34 +02:00
Yiorgis Gozadinos
218126de8d
Interactive research agent through AGUI client-side tool calls in CLI
2025-12-18 11:48:34 +02:00
Yiorgis Gozadinos
557a296341
Update ag-ui example
2025-12-18 11:41:53 +02:00
Yiorgis Gozadinos
50b7fb4461
Chunkers now set chunk.order directly
2025-12-18 11:35:48 +02:00
Yiorgis Gozadinos
3149552a61
Add 'plain' format and fallback for text conversion
2025-12-12 18:11:25 +02:00
Yiorgis Gozadinos
40e0ea090d
Add option to override vacuum interval
2025-12-12 16:04:06 +02:00
Yiorgis Gozadinos
056b9ad090
Use periodic vacuum to prevent disk exhaustion with large datasets
2025-12-12 15:53:23 +02:00
Yiorgis Gozadinos
f726229f60
Reformat benchmarks and add hotpotqa placeholder
2025-12-12 12:33:31 +02:00
Yiorgis Gozadinos
f6073a0db9
vb
2025-12-12 10:48:21 +02:00
Yiorgis Gozadinos
011f527579
Use rich rendering for citations in cli
2025-12-12 10:02:30 +02:00
Yiorgis Gozadinos
abe01e4b99
Fix ag-ui-example frontend
2025-12-12 08:32:51 +02:00
Yiorgis Gozadinos
00d621e211
vb
2025-12-11 16:31:21 +02:00
Yiorgis Gozadinos
c0cbc22137
Info modal
2025-12-11 15:38:45 +02:00
Yiorgis Gozadinos
d7c755f454
Add filter to simple qa agent
2025-12-11 15:28:41 +02:00
Yiorgis Gozadinos
748a465591
vb
2025-12-10 13:10:28 +02:00
Yiorgis Gozadinos
2161727716
Add auto_vacuum config option to control automatic vacuuming
2025-12-10 11:42:14 +02:00
Yiorgis Gozadinos
5a593e928e
Set max_context_items to 10
2025-12-09 12:47:42 +02:00
Yiorgis Gozadinos
802f058205
Move search related config under SearchConfig
2025-12-09 12:28:28 +02:00
Yiorgis Gozadinos
a56e1ba67c
Simplify import_document, update_document
2025-12-08 18:19:12 +02:00
Yiorgis Gozadinos
a1162f8020
rebase from main
2025-12-08 16:07:50 +02:00
Yiorgis Gozadinos
67684c799b
Rebuild in batches to avoid creating multiple lancedb versions
2025-12-08 15:56:41 +02:00
Yiorgis Gozadinos
06b4f6a224
Add format parameter for text-to-DoclingDocument conversion
2025-12-08 15:56:21 +02:00
Yiorgis Gozadinos
9a5f8e4368
Add type-aware context expansion for search results
2025-12-08 15:56:21 +02:00
Yiorgis Gozadinos
0ea3717497
Update docs
2025-12-08 15:56:02 +02:00
Yiorgis Gozadinos
17c7147a49
Remove pre-processor, no longer needed
2025-12-08 15:56:01 +02:00
Yiorgis Gozadinos
e299aed788
contextualize() and embed_chunks() utilities in embeddings
2025-12-08 15:55:30 +02:00
Yiorgis Gozadinos
4fbb4609b6
HaikuRag.chunk() utility method
2025-12-08 15:55:30 +02:00
Yiorgis Gozadinos
ace915473c
HaikuRag.convert() utility method
2025-12-08 15:55:30 +02:00
Yiorgis Gozadinos
ab57c55eaa
Drop ChunkWithMetadata, use good old Chunk
2025-12-08 15:55:30 +02:00
Yiorgis Gozadinos
33e6a36290
Make document import/update handling consistent for DoclingDocument
2025-12-08 15:55:29 +02:00
Yiorgis Gozadinos
60ab864fc1
Add docling_document_json to update_document_fields() and
2025-12-08 15:55:29 +02:00
Yiorgis Gozadinos
8e30f67266
Introduce import_document(), remove chunks from create_document()
2025-12-08 15:55:29 +02:00
Yiorgis Gozadinos
fe551066d8
Visual grounding docs
2025-12-08 15:55:29 +02:00
Yiorgis Gozadinos
ac439b6c69
Store raw text (do not contextualize) in chunk.text. Embed with contextualized text.
2025-12-08 15:55:26 +02:00
Yiorgis Gozadinos
0d8691342a
Show bounding boxes for chunks in inspector
2025-12-08 15:55:26 +02:00
Yiorgis Gozadinos
fe4ac530be
DB migration
2025-12-08 15:54:56 +02:00
Yiorgis Gozadinos
80447c9024
Make download_models() a HaikuRAG client method. Update app to show progress of what happens.
2025-12-04 16:30:26 +02:00
Yiorgis Gozadinos
4e30637d61
vb
2025-12-03 08:57:14 +02:00
Yiorgis Gozadinos
33b254ae3d
Always upgrade database on access. Explicitly create db by running init or HaikuRAG(path, create=True)
2025-12-02 16:49:55 +02:00
Yiorgis Gozadinos
f07590bd5b
vb
2025-12-01 17:58:47 +02:00
Yiorgis Gozadinos
e548a51ff7
Optimized rebuild --embed-only to use batch updates via LanceDB merge_insert instead of individual chunk updates
2025-12-01 17:53:27 +02:00
Yiorgis Gozadinos
10aa2a7cf2
vb
2025-11-28 10:15:59 +02:00
Yiorgis Gozadinos
88a4edb537
--embed-only, --rechunk options in rebuild command
2025-11-28 10:11:52 +02:00
Yiorgis Gozadinos
8a0cc04181
Add opencv-python-headless to docling optional dependency, required for TableStructureModel which uses TableFormer for table detection.
2025-11-27 16:22:37 +02:00
Yiorgis Gozadinos
eee29578ba
vb
2025-11-27 15:24:39 +02:00
Yiorgis Gozadinos
b4602b0cab
use OcrAutoOptions for automatic OCR engine selection.
2025-11-27 15:23:17 +02:00
Yiorgis Gozadinos
3959ce39f7
vb
2025-11-27 13:06:04 +02:00
Yiorgis Gozadinos
bdbd3ce556
cl
2025-11-27 10:43:23 +02:00
Yiorgis Gozadinos
ca2445d869
vb
2025-11-26 10:33:46 +02:00
Yiorgis Gozadinos
0303e6a8cd
Support LMStudio
2025-11-26 10:31:40 +02:00
Yiorgis Gozadinos
d30ac60a75
cl
2025-11-26 10:29:00 +02:00
Yiorgis Gozadinos
c22a5bb0ce
vb
2025-11-25 13:44:48 +02:00
Yiorgis Gozadinos
1bb7b6d5bd
Add support for per-model configuration settings including thinking, temperature and max_tokens
2025-11-25 12:14:24 +02:00
Yiorgis Gozadinos
4e1752a740
Default eval db location, evaluation script
2025-11-25 11:07:20 +02:00
Yiorgis Gozadinos
13d3a366e8
Enhanced ActivitySnapshot events with richer structured data
2025-11-24 16:18:27 +02:00
Yiorgis Gozadinos
b5892699a0
Use Mean Reciprocal Rank for single document evaluation as metric. Use Mean Average Precision for variable document evaluation as metric
2025-11-24 14:49:30 +02:00
Yiorgis Gozadinos
8d0846b88a
Record useful meta in the experiment
2025-11-24 10:46:25 +02:00
Yiorgis Gozadinos
0c95a9f2e7
cl
2025-11-22 17:53:13 +02:00
Yiorgis Gozadinos
ada2b66d5b
vb
2025-11-21 16:38:52 +02:00
Yiorgis Gozadinos
8c02e865c7
Change default embedding model to qwen3-embedding:4b
2025-11-21 16:30:41 +02:00
Yiorgis Gozadinos
6d8ab2575d
Clarify refine_factor, set default to 30
2025-11-21 15:53:24 +02:00
Yiorgis Gozadinos
fdcf9a1a12
Show index stats in cli info command
2025-11-21 13:54:04 +02:00
Yiorgis Gozadinos
6a2f33b464
Create vector index from CLI, use configurable refine_factor
2025-11-21 13:54:01 +02:00
Yiorgis Gozadinos
63b374c7e9
Name evaluation runs. Use only LLMJudge evaluator.
2025-11-21 13:26:39 +02:00
Yiorgis Gozadinos
c45d507213
Allow custom config when building research/deep ask graphs. Closes #149
2025-11-21 12:37:41 +02:00
Yiorgis Gozadinos
5c4a32f1c8
Fix ACTIVITY_SNAPSHOT payloads. Closes #150
2025-11-21 12:27:54 +02:00
Yiorgis Gozadinos
09900f8e8b
vb
2025-11-19 13:08:37 +02:00
Yiorgis Gozadinos
30a0bbcff2
Accept both single chunks and lists for batch insertion in ChunkRepository.create()
2025-11-19 12:35:02 +02:00
Yiorgis Gozadinos
5089ea4203
New update_document_fields() method for partial document updates
2025-11-19 11:37:36 +02:00
Yiorgis Gozadinos
0ea5f1e10e
Update dependencies
2025-11-19 10:24:37 +02:00
Yiorgis Gozadinos
826986708d
vb
2025-11-18 15:52:52 +02:00
Yiorgis Gozadinos
0bb3f4301c
Additional options for document conversion with docling
2025-11-18 15:44:04 +02:00
Yiorgis Gozadinos
f728fea489
Stop building full docker image
2025-11-18 14:48:34 +02:00
Yiorgis Gozadinos
cc09ce87c2
Increase reranking candidate retrieval multiplier from 3x to 10x for improved result quality
2025-11-18 14:35:57 +02:00
Yiorgis Gozadinos
efbd5b92f7
vb
2025-11-17 16:30:39 +02:00
Yiorgis Gozadinos
4b0a1f0038
Update ag-ui-research example with haiku.rag-slim image
2025-11-17 14:42:14 +02:00
Yiorgis Gozadinos
da4179678b
Update changelog
2025-11-17 13:11:11 +02:00
Yiorgis Gozadinos
973522ffe2
Documentation
2025-11-17 13:07:46 +02:00
Yiorgis Gozadinos
541552215e
Switch to using HuggingFace tokenizers
2025-11-17 13:05:05 +02:00
Yiorgis Gozadinos
636d5261e6
vb
2025-11-14 22:29:37 +02:00
Yiorgis Gozadinos
adb7d9093d
Run entire evaluation dataset as one so that it appears properly in logfire
2025-11-14 11:39:58 +02:00
Yiorgis Gozadinos
6eb0d59a28
vb
2025-11-13 15:00:57 +02:00
Yiorgis Gozadinos
4599dd1481
Drop disable_auto_create flag, simply create only on write operations
2025-11-13 13:53:11 +02:00
Yiorgis Gozadinos
8e802d4a91
Default vacuum_retention_seconds increased from 60 seconds to 86400 seconds (1 day) for better version retention in typical workflows
2025-11-13 13:31:53 +02:00
Yiorgis Gozadinos
2c43f034de
Refactor ag-ui-research. Drop human-in-the-loop, use MemoryObjectSendStream to merge the graph and agent streams together
2025-11-13 13:22:53 +02:00
Yiorgis Gozadinos
0c8d2838b2
Better typing
2025-11-13 13:22:52 +02:00
Yiorgis Gozadinos
888c9e338e
Changelog
2025-11-13 13:22:51 +02:00
Yiorgis Gozadinos
908296e155
Do not create unecessary DoclingDocument when chunks are provided. Rename private _create_with_docling & _update_with_docling to _create_and_chunk & _update_and_rechunk
2025-11-13 13:16:15 +02:00
Yiorgis Gozadinos
942ad2818f
Filereader error messages now include both original exception details and file path for easier debugging
2025-11-10 18:26:25 +02:00
Yiorgis Gozadinos
34f50f37a0
vb
2025-11-07 14:44:58 +02:00
Yiorgis Gozadinos
b6650d0486
Introduce delete_orphans in monitor (default=false) to delete db entries that do not resolve in the filesystem
2025-11-07 14:40:13 +02:00
Yiorgis Gozadinos
9ee64a33e0
Remove A2A from haiku.rag, turn it into an example
2025-11-07 13:31:51 +02:00
Yiorgis Gozadinos
0b696d9629
Remove deprecated use of env variables for configuration
2025-11-07 10:54:32 +02:00
Yiorgis Gozadinos
b51283c647
Use haiku.rag.yaml as the default config filename everywhere
2025-11-07 10:42:57 +02:00
Yiorgis Gozadinos
3c557d5bfe
Make --config a global CLI parameter, add it as a param to HaikuRAGApp
2025-11-07 10:31:55 +02:00
Yiorgis Gozadinos
0e3f2ebd67
vb
2025-11-06 16:13:53 +02:00
Yiorgis Gozadinos
ce6dcb8b64
Pin critical packages
2025-11-06 16:09:31 +02:00
Yiorgis Gozadinos
8c05b76056
cl
2025-11-06 11:02:33 +02:00
Yiorgis Gozadinos
4129be5bf9
Version bump & Changelog
2025-11-05 13:15:48 +02:00
