Yiorgis Gozadinos
04415a9d09
bump pydantic-monty, refresh deps, cap transformers<5
2026-05-18 16:02:55 +03:00
Yiorgis Gozadinos
1df6530d03
Bump pydantic-ai, prepare for 2.*
2026-05-18 15:14:29 +03:00
Yiorgis Gozadinos
b1e8bdc839
Bump docling
2026-05-18 15:14:29 +03:00
Yiorgis Gozadinos
97a7eb246c
vb
2026-05-14 16:03:33 +03:00
Yiorgis Gozadinos
519afe6709
Add cross-encoder reranking provider
2026-05-14 15:43:35 +03:00
Yiorgis Gozadinos
dbb87e091f
Ensure schema parity for _StagingChunkRecord
2026-05-14 15:11:18 +03:00
Yiorgis Gozadinos
849eca94d0
Make rebuild --embed-only idempotent across crashes
2026-05-14 15:02:12 +03:00
Yiorgis Gozadinos
a035b0f9e4
Stream rebuild --embed-only through a staging table to bound memory
2026-05-14 15:02:12 +03:00
Yiorgis Gozadinos
862a209137
vb
2026-05-13 16:53:35 +03:00
Yiorgis Gozadinos
a98ddc14b8
Replace picture_description.enabled with processing.pictures enum
2026-05-13 16:22:44 +03:00
Yiorgis Gozadinos
bdab201c42
Merge pull request #371 from ggozad/fix/image-results
...
Verify picture bytes before attaching to multimodal tool returns
2026-05-13 16:19:09 +03:00
Yiorgis Gozadinos
a6e5664c78
Verify picture bytes before attaching to multimodal tool returns
2026-05-13 16:01:39 +03:00
Yiorgis Gozadinos
072edad62f
Merge pull request #368 from ggozad/feat/relax-compat-check
...
Relax embedding compatibility check to vector_dim only
2026-05-13 15:43:37 +03:00
Yiorgis Gozadinos
701e9d3632
Relax embedding compat check to vector_dim only
2026-05-13 14:44:18 +03:00
Yiorgis Gozadinos
8b135c4d4d
Expose embedding batch size as config
2026-05-13 14:09:47 +03:00
Yiorgis Gozadinos
c6cd847299
Add ModelConfig.extra_body for raw provider pass-through
2026-05-13 13:45:09 +03:00
Yiorgis Gozadinos
0f4c4af495
Cache the chunking tokenizer to avoid HF Hub 429
2026-05-13 13:32:24 +03:00
Yiorgis Gozadinos
d512f9fbd1
Disable Typer locals in CLI tracebacks
2026-05-13 13:24:05 +03:00
Yiorgis Gozadinos
674b80e8c9
Support streaming for markdown
2026-05-12 16:01:42 +03:00
Yiorgis Gozadinos
a206f8bfcc
Document fetch_remote_images and the docling-serve HTML gap
2026-05-12 15:50:33 +03:00
Yiorgis Gozadinos
afd4665517
Thread source_uri through URL ingest
2026-05-12 15:13:02 +03:00
Yiorgis Gozadinos
a25d9699ee
Wire per-format options across docling-local
2026-05-12 15:02:59 +03:00
Yiorgis Gozadinos
6f6ebf27c8
Document s3 storage & monitoring
2026-05-11 11:29:56 +03:00
Yiorgis Gozadinos
8b8f44e7b2
vb
2026-05-08 12:22:35 +03:00
Yiorgis Gozadinos
ab5cfdd04a
Docs & cl
2026-05-08 11:20:04 +03:00
Yiorgis Gozadinos
61a88edfd8
drop bogus deprecation warning for processing.pictures
2026-05-05 14:46:41 +03:00
Yiorgis Gozadinos
ff82d36c2f
add rebuild --descriptions: run VLM over stored picture bytes only
2026-05-05 12:02:46 +03:00
Yiorgis Gozadinos
2038d43435
collapse pictures enum to picture_description.enabled boolean
2026-05-05 11:16:08 +03:00
Yiorgis Gozadinos
65d9c74224
expose image-as-query through MCP and the CLI.
2026-05-04 11:46:45 +03:00
Yiorgis Gozadinos
ff656504d3
make client.search() polymorphic on query type: str | bytes | PIL.Image.Image. Bytes/PIL queries embed via embed_image_query and run vector-only against the chunks table
2026-05-04 11:22:22 +03:00
Yiorgis Gozadinos
21f12cd770
add vision: bool flag on ModelConfig (default False). Gate the agent search tool's BinaryContent attachment on qa.model.vision so picture bytes are only sent to vision-capable QA models.
2026-05-04 10:47:21 +03:00
Yiorgis Gozadinos
6037f26673
drop mlx
2026-05-03 19:05:49 +03:00
Yiorgis Gozadinos
c9227c649f
emit synthetic picture chunks at ingest under multimodal embedders.
...
processing.chunk() merges text chunks with one synthetic Chunk per PictureItem-with-bytes,
sorted by iterate_items() position so chunk.order is structural.
embed_chunks dispatches on a Chunk._picture_data PrivateAttr
(text through embed_documents, picture through embed_image_query)
2026-05-03 17:04:43 +03:00
Yiorgis Gozadinos
37215f1a00
add multimodal embedder support for vllm and mlx
2026-05-03 10:57:56 +03:00
Yiorgis Gozadinos
861c36942b
cl
2026-04-30 15:50:28 +03:00
Yiorgis Gozadinos
7831057339
add processing.pictures enum (none|description|image), replacing the implicit pair of generate_picture_images+picture_description.enabled flags
2026-04-30 14:26:14 +03:00
Yiorgis Gozadinos
b01c649684
surface picture image bytes in SearchResult and emit multimodal ToolReturn from the agent search tool
2026-04-30 12:43:05 +03:00
Yiorgis Gozadinos
6a77ce92a9
extract picture bytes to document_items.picture_data at ingest, strip them from the docling_document blob, and add 0.45.0 migration to backfill existing
...
databases
2026-04-30 11:26:37 +03:00
Yiorgis Gozadinos
dc16f74b58
add picture_data column to document_items
2026-04-30 11:23:29 +03:00
Yiorgis Gozadinos
29bc42514e
vb
2026-04-29 13:45:36 +03:00
Yiorgis Gozadinos
318266e681
Update benchmark
2026-04-29 13:27:32 +03:00
Yiorgis Gozadinos
8962987ff9
pin judge model to ollama:qwen3.6
2026-04-29 12:24:06 +03:00
Yiorgis Gozadinos
5e31c15907
docs
2026-04-28 14:44:27 +03:00
Yiorgis Gozadinos
6807f082ec
vb
2026-04-25 09:35:47 +03:00
Yiorgis Gozadinos
758bd3ee72
Group relative-path test with the other connect_lancedb dispatch tests
2026-04-25 09:28:46 +03:00
Yiorgis Gozadinos
81ba248483
vb
2026-04-24 17:47:57 +03:00
Yiorgis Gozadinos
821b7361e9
Push document-id filter into chunk search query
2026-04-24 16:07:07 +03:00
Yiorgis Gozadinos
cd4224a727
drop private title/embedding forwarders from HaikuRAG
2026-04-24 15:52:50 +03:00
Yiorgis Gozadinos
ce2df2e0bd
Docs
2026-04-24 14:42:52 +03:00
Yiorgis Gozadinos
82fd10e0ee
Migrate LanceDB to native async API
...
Convert all LanceDB operations from sync calls wrapped in async
functions to the native async API (connect_async, AsyncConnection,
AsyncTable, AsyncQuery). Database I/O no longer blocks the event loop.
- Store and HaikuRAG use async context managers (async with). Store
initialization is deferred to __aenter__; direct construction
without async with is no longer supported.
- Index creation uses config objects (FTS, BTree, IvfPq) instead of
string-based index_type parameter.
- Upgrade callbacks are async.
- HaikuRAG tracks background vacuum tasks and awaits them in __aexit__
and before destructive rebuild operations to avoid races with
concurrent table mutations.
- temp_db_path fixture uses pytest's tmp_path for reliable async
cleanup.
2026-04-24 14:42:52 +03:00
Yiorgis Gozadinos
cb9360ef7b
fix chat TUI citation rendering after state flattening
2026-04-24 13:48:57 +03:00
Yiorgis Gozadinos
4f058bf484
vb
2026-04-22 17:00:03 +03:00
Yiorgis Gozadinos
ae2b8461be
flatten skill state citations from list[list[str]] to list[str]
2026-04-22 16:50:13 +03:00
Yiorgis Gozadinos
bb9e7b28ca
vb
2026-04-22 16:18:44 +03:00
Yiorgis Gozadinos
9067b89d2f
fix convert() misreading text content that starts with a URL
2026-04-22 14:47:23 +03:00
Yiorgis Gozadinos
f0016ebcd2
scope citations, searches, and executions to the current invocation
2026-04-22 13:53:39 +03:00
Yiorgis Gozadinos
4a9dd9b49a
persist sandbox variables across execute_code calls within one invocation
2026-04-22 13:30:38 +03:00
Yiorgis Gozadinos
4e9c02afc2
open one HaikuRAG client per skill invocation via lifespan
2026-04-22 13:13:42 +03:00
Yiorgis Gozadinos
8a23108fbd
vb
2026-04-20 16:36:32 +03:00
Yiorgis Gozadinos
eb4e1721ce
remove dead code, add tool descriptions in frontend, update changelog
2026-04-20 15:38:17 +03:00
Yiorgis Gozadinos
fa87cf79c5
Docs & cl
2026-04-20 10:14:19 +03:00
Yiorgis Gozadinos
2a0e89bbe7
add --skill flag to chat TUI for rag and analysis skills
2026-04-17 18:35:16 +03:00
Yiorgis Gozadinos
a45820dbf7
add document virtual filesystem to analysis sandbox
...
Replace get_document() and get_docling_document() with a VFS at
/documents/{id}/ with metadata.json (eager), content.txt (lazy),
and items.jsonl (lazy). Keep search(), list_documents() (now returns
all), and llm() as external functions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-17 18:35:16 +03:00
Yiorgis Gozadinos
4118533db1
fold context expansion into sandbox search and remove get_context
2026-04-17 18:35:16 +03:00
Yiorgis Gozadinos
44fd0c9906
add get_context() to analysis sandbox and improve prompt
2026-04-17 18:32:45 +03:00
Yiorgis Gozadinos
d2b3ba1b59
rename RLM agent to analysis throughout the codebase
2026-04-17 18:32:01 +03:00
Yiorgis Gozadinos
499a843a43
remove unused create_analysis_toolset and AnalysisResult
2026-04-17 18:32:01 +03:00
Yiorgis Gozadinos
1049586469
vb
2026-04-17 16:24:25 +03:00
Yiorgis Gozadinos
a0a9a3410b
info: report partial stats and pending migrations on pre-migration DBs
2026-04-17 14:00:03 +03:00
Yiorgis Gozadinos
9beaf0d4ed
fix changelog
2026-04-16 12:57:22 +03:00
Yiorgis Gozadinos
75c84805b4
Improve judge prompt, QA prompt, and raise max_searches to 5
2026-04-16 12:12:23 +03:00
Yiorgis Gozadinos
b6113bf8ab
replace fixed-radius expansion with section-bounded algorithm
...
Context expansion is now automatic and structure-aware. For structured
documents, expands within the section containing the match. For sections
that exceed the budget or are too small, expands item-by-item outward
skipping noise labels. Unstructured documents use budget-based outward
expansion. Results sorted by relevance score.
2026-04-16 12:11:53 +03:00
Yiorgis Gozadinos
364b1bc509
add document_items table for fast context expansion
2026-04-16 12:11:53 +03:00
Yiorgis Gozadinos
dcde86a5f1
vb
2026-04-09 15:33:19 +03:00
Yiorgis Gozadinos
a51b922be3
fix skill analyze tool ignores state.document_filter
2026-04-09 15:29:20 +03:00
Yiorgis Gozadinos
99af4fe11d
fix skill list_documents tool ignores state.document_filter
2026-04-09 15:29:20 +03:00
Yiorgis Gozadinos
a118ee7aaa
Cleanup
2026-04-08 14:19:33 +03:00
Yiorgis Gozadinos
5e7a4ebae5
Update docs
2026-04-08 14:19:32 +03:00
Yiorgis Gozadinos
d7d0090a92
cl
2026-04-08 14:08:00 +03:00
Yiorgis Gozadinos
62a436dd9d
vb
2026-04-08 14:04:32 +03:00
Yiorgis Gozadinos
b614ef19e1
switch compression from gzip to zstd, use stdlib for python >= 3.14
2026-04-08 14:04:32 +03:00
Yiorgis Gozadinos
3d30840e9d
vb
2026-04-07 16:36:27 +03:00
Yiorgis Gozadinos
8a24606784
Strip page images from DoclingDocument before validation, unless we use visualize_chunk()
2026-04-07 12:54:23 +03:00
Yiorgis Gozadinos
32258d4e2a
Add batching to embeddings for huge documents
2026-04-07 12:06:28 +03:00
Yiorgis Gozadinos
68c3fa0f79
Add order to SearchResult, add ChunkRepository.get_chunks_in_range(), to use them in _expand_with_chunks to fetch only nearby chunks
2026-04-07 12:04:05 +03:00
Yiorgis Gozadinos
e4dc60c4f1
Update core dependencies
2026-04-03 10:14:27 +03:00
Yiorgis Gozadinos
5c4164d224
vb
2026-04-01 13:32:56 +03:00
Yiorgis Gozadinos
eb5db78026
clarify domain_preamble purpose
2026-04-01 13:13:52 +03:00
Yiorgis Gozadinos
15d94bfe7f
replace UUIDs with readable identifiers in citation text
2026-04-01 11:44:17 +03:00
Yiorgis Gozadinos
127535bc56
vb
2026-03-28 10:05:50 +02:00
Yiorgis Gozadinos
13a71e34a7
vb
2026-03-27 16:39:22 +02:00
Yiorgis Gozadinos
f46b8195ed
vb
2026-03-26 14:15:09 +02:00
Yiorgis Gozadinos
a07a64a31f
enable repeat_table_header for self-contained table chunks and docling-serve parity
2026-03-26 14:02:12 +02:00
Yiorgis Gozadinos
1c9035ebb5
Add optional db_path and config params to generated skill create_skill()
2026-03-26 13:41:58 +02:00
Yiorgis Gozadinos
9d986421a0
add visualize_chunk to generated skills and built-in RAG skill
2026-03-26 11:41:22 +02:00
Yiorgis Gozadinos
f5014f39bf
vb
2026-03-24 17:46:51 +02:00
Yiorgis Gozadinos
7d50edea86
Fix validate_metadata layering and type annotations in _tools.py
2026-03-24 17:09:14 +02:00
Yiorgis Gozadinos
7309d13317
Docs
2026-03-24 16:06:46 +02:00
Yiorgis Gozadinos
8a5188aa01
vb
2026-03-24 14:23:29 +02:00
Yiorgis Gozadinos
aa9fdf051d
Remove support for structured output
2026-03-24 14:11:43 +02:00
