Yiorgis Gozadinos
b1e8bdc839
Bump docling
2026-05-18 15:14:29 +03:00
Yiorgis Gozadinos
97a7eb246c
vb
2026-05-14 16:03:33 +03:00
Yiorgis Gozadinos
519afe6709
Add cross-encoder reranking provider
2026-05-14 15:43:35 +03:00
Yiorgis Gozadinos
dbb87e091f
Ensure schema parity for _StagingChunkRecord
2026-05-14 15:11:18 +03:00
Yiorgis Gozadinos
849eca94d0
Make rebuild --embed-only idempotent across crashes
2026-05-14 15:02:12 +03:00
Yiorgis Gozadinos
a035b0f9e4
Stream rebuild --embed-only through a staging table to bound memory
2026-05-14 15:02:12 +03:00
Yiorgis Gozadinos
862a209137
vb
2026-05-13 16:53:35 +03:00
Yiorgis Gozadinos
a98ddc14b8
Replace picture_description.enabled with processing.pictures enum
2026-05-13 16:22:44 +03:00
Yiorgis Gozadinos
bdab201c42
Merge pull request #371 from ggozad/fix/image-results
...
Verify picture bytes before attaching to multimodal tool returns
2026-05-13 16:19:09 +03:00
Yiorgis Gozadinos
a6e5664c78
Verify picture bytes before attaching to multimodal tool returns
2026-05-13 16:01:39 +03:00
Yiorgis Gozadinos
072edad62f
Merge pull request #368 from ggozad/feat/relax-compat-check
...
Relax embedding compatibility check to vector_dim only
2026-05-13 15:43:37 +03:00
Yiorgis Gozadinos
701e9d3632
Relax embedding compat check to vector_dim only
2026-05-13 14:44:18 +03:00
Yiorgis Gozadinos
8b135c4d4d
Expose embedding batch size as config
2026-05-13 14:09:47 +03:00
Yiorgis Gozadinos
c6cd847299
Add ModelConfig.extra_body for raw provider pass-through
2026-05-13 13:45:09 +03:00
Yiorgis Gozadinos
0f4c4af495
Cache the chunking tokenizer to avoid HF Hub 429
2026-05-13 13:32:24 +03:00
Yiorgis Gozadinos
d512f9fbd1
Disable Typer locals in CLI tracebacks
2026-05-13 13:24:05 +03:00
Yiorgis Gozadinos
674b80e8c9
Support streaming for markdown
2026-05-12 16:01:42 +03:00
Yiorgis Gozadinos
a206f8bfcc
Document fetch_remote_images and the docling-serve HTML gap
2026-05-12 15:50:33 +03:00
Yiorgis Gozadinos
afd4665517
Thread source_uri through URL ingest
2026-05-12 15:13:02 +03:00
Yiorgis Gozadinos
a25d9699ee
Wire per-format options across docling-local
2026-05-12 15:02:59 +03:00
Yiorgis Gozadinos
6f6ebf27c8
Document s3 storage & monitoring
2026-05-11 11:29:56 +03:00
Yiorgis Gozadinos
8b8f44e7b2
vb
2026-05-08 12:22:35 +03:00
Yiorgis Gozadinos
ab5cfdd04a
Docs & cl
2026-05-08 11:20:04 +03:00
Yiorgis Gozadinos
61a88edfd8
drop bogus deprecation warning for processing.pictures
2026-05-05 14:46:41 +03:00
Yiorgis Gozadinos
ff82d36c2f
add rebuild --descriptions: run VLM over stored picture bytes only
2026-05-05 12:02:46 +03:00
Yiorgis Gozadinos
2038d43435
collapse pictures enum to picture_description.enabled boolean
2026-05-05 11:16:08 +03:00
Yiorgis Gozadinos
65d9c74224
expose image-as-query through MCP and the CLI.
2026-05-04 11:46:45 +03:00
Yiorgis Gozadinos
ff656504d3
make client.search() polymorphic on query type: str | bytes | PIL.Image.Image. Bytes/PIL queries embed via embed_image_query and run vector-only against the chunks table
2026-05-04 11:22:22 +03:00
Yiorgis Gozadinos
21f12cd770
add vision: bool flag on ModelConfig (default False). Gate the agent search tool's BinaryContent attachment on qa.model.vision so picture bytes are only sent to vision-capable QA models.
2026-05-04 10:47:21 +03:00
Yiorgis Gozadinos
6037f26673
drop mlx
2026-05-03 19:05:49 +03:00
Yiorgis Gozadinos
c9227c649f
emit synthetic picture chunks at ingest under multimodal embedders.
...
processing.chunk() merges text chunks with one synthetic Chunk per PictureItem-with-bytes,
sorted by iterate_items() position so chunk.order is structural.
embed_chunks dispatches on a Chunk._picture_data PrivateAttr
(text through embed_documents, picture through embed_image_query)
2026-05-03 17:04:43 +03:00
Yiorgis Gozadinos
37215f1a00
add multimodal embedder support for vllm and mlx
2026-05-03 10:57:56 +03:00
Yiorgis Gozadinos
861c36942b
cl
2026-04-30 15:50:28 +03:00
Yiorgis Gozadinos
7831057339
add processing.pictures enum (none|description|image), replacing the implicit pair of generate_picture_images+picture_description.enabled flags
2026-04-30 14:26:14 +03:00
Yiorgis Gozadinos
b01c649684
surface picture image bytes in SearchResult and emit multimodal ToolReturn from the agent search tool
2026-04-30 12:43:05 +03:00
Yiorgis Gozadinos
6a77ce92a9
extract picture bytes to document_items.picture_data at ingest, strip them from the docling_document blob, and add 0.45.0 migration to backfill existing
...
databases
2026-04-30 11:26:37 +03:00
Yiorgis Gozadinos
dc16f74b58
add picture_data column to document_items
2026-04-30 11:23:29 +03:00
Yiorgis Gozadinos
29bc42514e
vb
2026-04-29 13:45:36 +03:00
Yiorgis Gozadinos
318266e681
Update benchmark
2026-04-29 13:27:32 +03:00
Yiorgis Gozadinos
8962987ff9
pin judge model to ollama:qwen3.6
2026-04-29 12:24:06 +03:00
Yiorgis Gozadinos
5e31c15907
docs
2026-04-28 14:44:27 +03:00
Yiorgis Gozadinos
6807f082ec
vb
2026-04-25 09:35:47 +03:00
Yiorgis Gozadinos
758bd3ee72
Group relative-path test with the other connect_lancedb dispatch tests
2026-04-25 09:28:46 +03:00
Yiorgis Gozadinos
81ba248483
vb
2026-04-24 17:47:57 +03:00
Yiorgis Gozadinos
821b7361e9
Push document-id filter into chunk search query
2026-04-24 16:07:07 +03:00
Yiorgis Gozadinos
cd4224a727
drop private title/embedding forwarders from HaikuRAG
2026-04-24 15:52:50 +03:00
Yiorgis Gozadinos
ce2df2e0bd
Docs
2026-04-24 14:42:52 +03:00
Yiorgis Gozadinos
82fd10e0ee
Migrate LanceDB to native async API
...
Convert all LanceDB operations from sync calls wrapped in async
functions to the native async API (connect_async, AsyncConnection,
AsyncTable, AsyncQuery). Database I/O no longer blocks the event loop.
- Store and HaikuRAG use async context managers (async with). Store
initialization is deferred to __aenter__; direct construction
without async with is no longer supported.
- Index creation uses config objects (FTS, BTree, IvfPq) instead of
string-based index_type parameter.
- Upgrade callbacks are async.
- HaikuRAG tracks background vacuum tasks and awaits them in __aexit__
and before destructive rebuild operations to avoid races with
concurrent table mutations.
- temp_db_path fixture uses pytest's tmp_path for reliable async
cleanup.
2026-04-24 14:42:52 +03:00
Yiorgis Gozadinos
cb9360ef7b
fix chat TUI citation rendering after state flattening
2026-04-24 13:48:57 +03:00
Yiorgis Gozadinos
4f058bf484
vb
2026-04-22 17:00:03 +03:00
Yiorgis Gozadinos
ae2b8461be
flatten skill state citations from list[list[str]] to list[str]
2026-04-22 16:50:13 +03:00
Yiorgis Gozadinos
bb9e7b28ca
vb
2026-04-22 16:18:44 +03:00
Yiorgis Gozadinos
9067b89d2f
fix convert() misreading text content that starts with a URL
2026-04-22 14:47:23 +03:00
Yiorgis Gozadinos
f0016ebcd2
scope citations, searches, and executions to the current invocation
2026-04-22 13:53:39 +03:00
Yiorgis Gozadinos
4a9dd9b49a
persist sandbox variables across execute_code calls within one invocation
2026-04-22 13:30:38 +03:00
Yiorgis Gozadinos
4e9c02afc2
open one HaikuRAG client per skill invocation via lifespan
2026-04-22 13:13:42 +03:00
Yiorgis Gozadinos
8a23108fbd
vb
2026-04-20 16:36:32 +03:00
Yiorgis Gozadinos
eb4e1721ce
remove dead code, add tool descriptions in frontend, update changelog
2026-04-20 15:38:17 +03:00
Yiorgis Gozadinos
fa87cf79c5
Docs & cl
2026-04-20 10:14:19 +03:00
Yiorgis Gozadinos
2a0e89bbe7
add --skill flag to chat TUI for rag and analysis skills
2026-04-17 18:35:16 +03:00
Yiorgis Gozadinos
a45820dbf7
add document virtual filesystem to analysis sandbox
...
Replace get_document() and get_docling_document() with a VFS at
/documents/{id}/ with metadata.json (eager), content.txt (lazy),
and items.jsonl (lazy). Keep search(), list_documents() (now returns
all), and llm() as external functions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-17 18:35:16 +03:00
Yiorgis Gozadinos
4118533db1
fold context expansion into sandbox search and remove get_context
2026-04-17 18:35:16 +03:00
Yiorgis Gozadinos
44fd0c9906
add get_context() to analysis sandbox and improve prompt
2026-04-17 18:32:45 +03:00
Yiorgis Gozadinos
d2b3ba1b59
rename RLM agent to analysis throughout the codebase
2026-04-17 18:32:01 +03:00
Yiorgis Gozadinos
499a843a43
remove unused create_analysis_toolset and AnalysisResult
2026-04-17 18:32:01 +03:00
Yiorgis Gozadinos
1049586469
vb
2026-04-17 16:24:25 +03:00
Yiorgis Gozadinos
a0a9a3410b
info: report partial stats and pending migrations on pre-migration DBs
2026-04-17 14:00:03 +03:00
Yiorgis Gozadinos
9beaf0d4ed
fix changelog
2026-04-16 12:57:22 +03:00
Yiorgis Gozadinos
75c84805b4
Improve judge prompt, QA prompt, and raise max_searches to 5
2026-04-16 12:12:23 +03:00
Yiorgis Gozadinos
b6113bf8ab
replace fixed-radius expansion with section-bounded algorithm
...
Context expansion is now automatic and structure-aware. For structured
documents, expands within the section containing the match. For sections
that exceed the budget or are too small, expands item-by-item outward
skipping noise labels. Unstructured documents use budget-based outward
expansion. Results sorted by relevance score.
2026-04-16 12:11:53 +03:00
Yiorgis Gozadinos
364b1bc509
add document_items table for fast context expansion
2026-04-16 12:11:53 +03:00
Yiorgis Gozadinos
dcde86a5f1
vb
2026-04-09 15:33:19 +03:00
Yiorgis Gozadinos
a51b922be3
fix skill analyze tool ignores state.document_filter
2026-04-09 15:29:20 +03:00
Yiorgis Gozadinos
99af4fe11d
fix skill list_documents tool ignores state.document_filter
2026-04-09 15:29:20 +03:00
Yiorgis Gozadinos
a118ee7aaa
Cleanup
2026-04-08 14:19:33 +03:00
Yiorgis Gozadinos
5e7a4ebae5
Update docs
2026-04-08 14:19:32 +03:00
Yiorgis Gozadinos
d7d0090a92
cl
2026-04-08 14:08:00 +03:00
Yiorgis Gozadinos
62a436dd9d
vb
2026-04-08 14:04:32 +03:00
Yiorgis Gozadinos
b614ef19e1
switch compression from gzip to zstd, use stdlib for python >= 3.14
2026-04-08 14:04:32 +03:00
Yiorgis Gozadinos
3d30840e9d
vb
2026-04-07 16:36:27 +03:00
Yiorgis Gozadinos
8a24606784
Strip page images from DoclingDocument before validation, unless we use visualize_chunk()
2026-04-07 12:54:23 +03:00
Yiorgis Gozadinos
32258d4e2a
Add batching to embeddings for huge documents
2026-04-07 12:06:28 +03:00
Yiorgis Gozadinos
68c3fa0f79
Add order to SearchResult, add ChunkRepository.get_chunks_in_range(), to use them in _expand_with_chunks to fetch only nearby chunks
2026-04-07 12:04:05 +03:00
Yiorgis Gozadinos
e4dc60c4f1
Update core dependencies
2026-04-03 10:14:27 +03:00
Yiorgis Gozadinos
5c4164d224
vb
2026-04-01 13:32:56 +03:00
Yiorgis Gozadinos
eb5db78026
clarify domain_preamble purpose
2026-04-01 13:13:52 +03:00
Yiorgis Gozadinos
15d94bfe7f
replace UUIDs with readable identifiers in citation text
2026-04-01 11:44:17 +03:00
Yiorgis Gozadinos
127535bc56
vb
2026-03-28 10:05:50 +02:00
Yiorgis Gozadinos
13a71e34a7
vb
2026-03-27 16:39:22 +02:00
Yiorgis Gozadinos
f46b8195ed
vb
2026-03-26 14:15:09 +02:00
Yiorgis Gozadinos
a07a64a31f
enable repeat_table_header for self-contained table chunks and docling-serve parity
2026-03-26 14:02:12 +02:00
Yiorgis Gozadinos
1c9035ebb5
Add optional db_path and config params to generated skill create_skill()
2026-03-26 13:41:58 +02:00
Yiorgis Gozadinos
9d986421a0
add visualize_chunk to generated skills and built-in RAG skill
2026-03-26 11:41:22 +02:00
Yiorgis Gozadinos
f5014f39bf
vb
2026-03-24 17:46:51 +02:00
Yiorgis Gozadinos
7d50edea86
Fix validate_metadata layering and type annotations in _tools.py
2026-03-24 17:09:14 +02:00
Yiorgis Gozadinos
7309d13317
Docs
2026-03-24 16:06:46 +02:00
Yiorgis Gozadinos
8a5188aa01
vb
2026-03-24 14:23:29 +02:00
Yiorgis Gozadinos
aa9fdf051d
Remove support for structured output
2026-03-24 14:11:43 +02:00
Yiorgis Gozadinos
ce320ca772
Update docling, docling-core and move to version 1.10.0 of DoclingDocument.
2026-03-23 17:34:06 +02:00
Yiorgis Gozadinos
fd327996b8
Configurable judge and reflect models for evaluations
2026-03-18 17:14:11 +02:00
Yiorgis Gozadinos
9243f56e1c
vb
2026-03-16 14:30:08 +02:00
Yiorgis Gozadinos
d4e34ab79f
Support for plantuml diagrams
2026-03-13 17:18:19 +02:00
Yiorgis Gozadinos
f961f00f2c
vb
2026-03-13 13:24:31 +02:00
Yiorgis Gozadinos
a0025e9203
Fix ty
2026-03-13 12:51:35 +02:00
Yiorgis Gozadinos
e8e02f341d
Update docs
2026-03-12 17:55:17 +02:00
Yiorgis Gozadinos
7696b3f98f
vb
2026-03-12 14:46:13 +02:00
Yiorgis Gozadinos
1bee8723ae
Merge pull request #307 from ggozad/feat/evals-gepa
...
Add GEPA prompt optimization for QA evaluations
2026-03-12 14:36:43 +02:00
Yiorgis Gozadinos
6ac5f277d8
Rename iterations to num_candidates
2026-03-12 13:00:51 +02:00
Yiorgis Gozadinos
8bac8b4104
Properly choose training/eval set
2026-03-12 12:05:50 +02:00
Yiorgis Gozadinos
10abfbe3d8
docs
2026-03-12 12:05:49 +02:00
Yiorgis Gozadinos
90fe624fa7
improve QA citation reliability via prompt and ID normalization
2026-03-12 11:50:46 +02:00
Yiorgis Gozadinos
b8c377da1e
Prevent read-only mode from creating tables in empty directories
2026-03-12 11:42:04 +02:00
Yiorgis Gozadinos
8d6617d63f
vb
2026-03-11 17:08:22 +02:00
Yiorgis Gozadinos
45e94ec67a
Set search limit to 10
2026-03-11 12:26:41 +02:00
Yiorgis Gozadinos
e2749ad2a6
Cap QA agent search iterations to reduce response time
2026-03-11 12:13:43 +02:00
Yiorgis Gozadinos
f366aea905
vb
2026-03-06 11:12:54 +02:00
Yiorgis Gozadinos
5a0430857f
Enable thinking by default for QA agent
2026-03-05 15:48:08 +02:00
Yiorgis Gozadinos
40ff54eac0
Set params for evaluations judge
2026-03-05 13:59:20 +02:00
Yiorgis Gozadinos
3cc2d5e19e
Set appropriate temperature and max_tokens defaults
2026-03-05 13:23:50 +02:00
Yiorgis Gozadinos
58959b9249
enable parallel test run
2026-03-05 12:23:22 +02:00
Yiorgis Gozadinos
c7b03eb9ab
Strengthen weak assertions in context enhancement and converter tests
...
- Add doc_item_refs and headings assertions to test_expand_context_docling_merges_metadata
- Add page count and markdown content assertions to test_convert_pdf_with_ocr_engine
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 11:44:51 +02:00
Yiorgis Gozadinos
878423ea2e
vb
2026-03-04 15:02:34 +02:00
Yiorgis Gozadinos
28e90e5a87
Auto-detect structured output mode from model profile. Remove the structured_output config field from ModelConfig.
2026-03-04 14:51:09 +02:00
Yiorgis Gozadinos
fdd7c21757
Add configurable structured output mode (tool vs native)
2026-03-04 13:35:32 +02:00
Yiorgis Gozadinos
3f02abbc70
Expose skill state metadata as module-level API
2026-03-04 11:38:00 +02:00
Yiorgis Gozadinos
8a859fbe3c
vb
2026-03-03 17:25:53 +02:00
Yiorgis Gozadinos
2566bf1a61
Adapt to haiku.skills streaming sub-agent ag-ui events
2026-03-03 11:16:44 +02:00
Yiorgis Gozadinos
1a3ea64aa6
Remove filter parameter from search and list_documents tools
2026-03-03 11:01:58 +02:00
Yiorgis Gozadinos
99ca3605e7
vb
2026-02-28 14:11:25 +02:00
Yiorgis Gozadinos
52ef983e75
Fix default chat model, default to config.qa.model
2026-02-28 14:00:40 +02:00
Yiorgis Gozadinos
2eea9b3bc9
Add changelog entry for haiku.skills 0.5.1 compatibility fix
...
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 13:42:42 +02:00
Yiorgis Gozadinos
50116bb49a
vb
2026-02-26 11:21:46 +02:00
Yiorgis Gozadinos
d3a1031ec4
Update docs
2026-02-26 09:09:42 +02:00
Yiorgis Gozadinos
2acb0292e4
vb
2026-02-24 13:30:01 +02:00
Yiorgis Gozadinos
6759a2e0a5
Give regex to monty as externals
2026-02-24 13:12:56 +02:00
Yiorgis Gozadinos
d134276819
Update docs
2026-02-24 12:26:04 +02:00
Yiorgis Gozadinos
459fdfca71
Add get_docling_document external function to sandbox
2026-02-24 11:51:43 +02:00
Yiorgis Gozadinos
c61ded1271
Replace Docker sandbox with pydantic-monty
2026-02-24 09:55:40 +02:00
Yiorgis Gozadinos
2cafb5c8cc
vb
2026-02-20 19:41:17 +02:00
Yiorgis Gozadinos
b23f99af06
Fix history too
2026-02-20 19:15:28 +02:00
Yiorgis Gozadinos
388e94ae80
info() should always be read-only
2026-02-20 19:13:45 +02:00
Yiorgis Gozadinos
e186e13cd2
vb
2026-02-20 19:01:16 +02:00
Yiorgis Gozadinos
69083c8cb4
Fix stale docs
2026-02-20 16:59:58 +02:00
Yiorgis Gozadinos
37f28ea8de
Clean up stale references and dead code
2026-02-20 16:37:15 +02:00
Yiorgis Gozadinos
0c270ba42f
vb
2026-02-19 14:28:13 +02:00
Yiorgis Gozadinos
08a4252c33
download-models, show actionable error message when Ollama is not running
2026-02-19 14:06:53 +02:00
Yiorgis Gozadinos
df5c8bcfaf
Add cachetools dependency
2026-02-19 13:04:07 +02:00
Yiorgis Gozadinos
d15c31b550
Fix changelog and bump version script
2026-02-19 12:32:11 +02:00
Yiorgis Gozadinos
c62fe5fce8
vb
2026-02-17 13:56:43 +02:00
Yiorgis Gozadinos
f4735667df
emit StateDeltaEvent with client-aware baseline
2026-02-17 12:24:09 +02:00
