Yiorgis Gozadinos
37215f1a00
add multimodal embedder support for vllm and mlx
2026-05-03 10:57:56 +03:00
Yiorgis Gozadinos
75c8e83515
Allow uri override on create_document_from_source
2026-04-30 16:36:35 +03:00
Yiorgis Gozadinos
4c9ef81fc6
drop the existing-row-not-found guard from the 0.45.0 picture-bytes migration.
2026-04-30 15:48:03 +03:00
Yiorgis Gozadinos
b290bbe82d
rename document_items repository's _LIGHT_COLUMNS to _METADATA_COLUMNS
2026-04-30 15:43:03 +03:00
Yiorgis Gozadinos
875c635a77
have rebuild --rechunk re-chunk from the stored docling blob instead of re-converting from the markdown export
2026-04-30 15:30:24 +03:00
Yiorgis Gozadinos
7831057339
add processing.pictures enum (none|description|image), replacing the implicit pair of generate_picture_images+picture_description.enabled flags
2026-04-30 14:26:14 +03:00
Yiorgis Gozadinos
b01c649684
surface picture image bytes in SearchResult and emit multimodal ToolReturn from the agent search tool
2026-04-30 12:43:05 +03:00
Yiorgis Gozadinos
6a77ce92a9
extract picture bytes to document_items.picture_data at ingest, strip them from the docling_document blob, and add 0.45.0 migration to backfill existing
...
databases
2026-04-30 11:26:37 +03:00
Yiorgis Gozadinos
191cdc636b
retrieve picture image bytes via referenced+zip path, fix docling-serve not returning images even when set to do so. Remove xfail from relevant test
2026-04-30 11:23:29 +03:00
Yiorgis Gozadinos
dc16f74b58
add picture_data column to document_items
2026-04-30 11:23:29 +03:00
Yiorgis Gozadinos
29bc42514e
vb
2026-04-29 13:45:36 +03:00
Yiorgis Gozadinos
5f08a2a8ae
Adapt rag SKILL.md in order to improve citation rate
2026-04-28 14:51:28 +03:00
Yiorgis Gozadinos
7685a592c2
bump haiku.skills to 0.16.0
2026-04-28 12:03:14 +03:00
Yiorgis Gozadinos
6807f082ec
vb
2026-04-25 09:35:47 +03:00
Tres Seaver
0d70f4aec6
fix: pass absolute 'db_path' to 'lancedb.connect_async'
...
Closes #354 .
2026-04-24 17:59:01 -04:00
Yiorgis Gozadinos
81ba248483
vb
2026-04-24 17:47:57 +03:00
Yiorgis Gozadinos
821b7361e9
Push document-id filter into chunk search query
2026-04-24 16:07:07 +03:00
Yiorgis Gozadinos
31fc22ce9d
List tables consistently
2026-04-24 15:57:07 +03:00
Yiorgis Gozadinos
cd4224a727
drop private title/embedding forwarders from HaikuRAG
2026-04-24 15:52:50 +03:00
Yiorgis Gozadinos
b23bad32c3
Check table_names() before dropping chunks in recreate_embeddings_table
2026-04-24 15:32:06 +03:00
Yiorgis Gozadinos
729899299b
Log post-rebuild vacuum failures and clarify vacuum-await scope
2026-04-24 15:31:36 +03:00
Yiorgis Gozadinos
258ffa41fa
Improve test coverage
2026-04-24 14:43:59 +03:00
Yiorgis Gozadinos
aca3ccc29f
Close connections if __aenter__ fails mid-initialization
2026-04-24 14:43:59 +03:00
Yiorgis Gozadinos
3224d1f20c
Extract document orchestration into client/documents.py
2026-04-24 14:43:59 +03:00
Yiorgis Gozadinos
cb02a2c3d8
Extract rebuild machinery into client/rebuild.py
2026-04-24 14:43:59 +03:00
Yiorgis Gozadinos
528acb79a3
Extract LLM agents into client/agents.py
2026-04-24 14:43:58 +03:00
Yiorgis Gozadinos
12939f1444
Extract search, expand_context, visualize_chunk into client/search.py
2026-04-24 14:43:58 +03:00
Yiorgis Gozadinos
d4ecebf896
Extract processing primitives into client/processing.py
2026-04-24 14:42:53 +03:00
Yiorgis Gozadinos
b5e480bfa1
Extract title generation into client/titles.py
2026-04-24 14:42:53 +03:00
Yiorgis Gozadinos
e814459e1f
Extract download_models into client/downloads.py
2026-04-24 14:42:53 +03:00
Yiorgis Gozadinos
43f2774c14
Turn client.py into a client/ package
2026-04-24 14:42:53 +03:00
Yiorgis Gozadinos
d0ed9e213f
Pin FTS search column to content_fts
2026-04-24 14:42:53 +03:00
Yiorgis Gozadinos
a74c3a6f9e
Satisfy ty by adding a typed to_pydantic helper
2026-04-24 14:42:53 +03:00
Yiorgis Gozadinos
b6ea07d2de
Track all in-flight background vacuum tasks, not just the last one
2026-04-24 14:42:53 +03:00
Yiorgis Gozadinos
57385506c1
Fix is_new_db detection when create=True is passed on an existing DB
2026-04-24 14:42:53 +03:00
Yiorgis Gozadinos
82fd10e0ee
Migrate LanceDB to native async API
...
Convert all LanceDB operations from sync calls wrapped in async
functions to the native async API (connect_async, AsyncConnection,
AsyncTable, AsyncQuery). Database I/O no longer blocks the event loop.
- Store and HaikuRAG use async context managers (async with). Store
initialization is deferred to __aenter__; direct construction
without async with is no longer supported.
- Index creation uses config objects (FTS, BTree, IvfPq) instead of
string-based index_type parameter.
- Upgrade callbacks are async.
- HaikuRAG tracks background vacuum tasks and awaits them in __aexit__
and before destructive rebuild operations to avoid races with
concurrent table mutations.
- temp_db_path fixture uses pytest's tmp_path for reliable async
cleanup.
2026-04-24 14:42:52 +03:00
Yiorgis Gozadinos
84c5391259
Merge pull request #353 from ggozad/feat/better-visual-highlight
...
Tone down the yellow colour in visual highlights
2026-04-24 14:41:27 +03:00
Yiorgis Gozadinos
cb9360ef7b
fix chat TUI citation rendering after state flattening
2026-04-24 13:48:57 +03:00
Yiorgis Gozadinos
a833e58d54
Tone down the yellow colour in visual highlights
2026-04-24 10:31:43 +03:00
Yiorgis Gozadinos
4f058bf484
vb
2026-04-22 17:00:03 +03:00
Yiorgis Gozadinos
ae2b8461be
flatten skill state citations from list[list[str]] to list[str]
2026-04-22 16:50:13 +03:00
Yiorgis Gozadinos
bb9e7b28ca
vb
2026-04-22 16:18:44 +03:00
Yiorgis Gozadinos
206c4ffcf2
address review: top-level AnalysisRunDeps import, asserts, e2e tests
...
- Move AnalysisRunDeps to the module-level import in _tools.py.
- Swap the RuntimeError guards in _require_rag and execute_code for
plain asserts with the same diagnostic messages. Apply the same
treatment to the ty-reachability guard in Sandbox._ensure_initialized.
- Add end-to-end lifespan tests for both skills that drive _run_skill
with TestModel, covering the sub-agent entry path that the existing
tool-level tests bypass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 15:04:24 +03:00
Yiorgis Gozadinos
9067b89d2f
fix convert() misreading text content that starts with a URL
2026-04-22 14:47:23 +03:00
Yiorgis Gozadinos
f0016ebcd2
scope citations, searches, and executions to the current invocation
2026-04-22 13:53:39 +03:00
Yiorgis Gozadinos
4a9dd9b49a
persist sandbox variables across execute_code calls within one invocation
2026-04-22 13:30:38 +03:00
Yiorgis Gozadinos
4e9c02afc2
open one HaikuRAG client per skill invocation via lifespan
2026-04-22 13:13:42 +03:00
Yiorgis Gozadinos
8a23108fbd
vb
2026-04-20 16:36:32 +03:00
Yiorgis Gozadinos
eb4e1721ce
remove dead code, add tool descriptions in frontend, update changelog
2026-04-20 15:38:17 +03:00
Yiorgis Gozadinos
ff8ad0879c
fix context expansion: respect section boundaries, remove max_context_items
2026-04-20 15:18:16 +03:00
Yiorgis Gozadinos
af7731f4e1
bulk-fetch items.jsonl via lazy cache to avoid per-document query timeout
2026-04-20 14:25:16 +03:00
Yiorgis Gozadinos
cd0c21c996
remove cited_chunks from analysis agent, hoist _deny_write out of loop
2026-04-20 14:05:10 +03:00
Yiorgis Gozadinos
579e609ae1
revert REPL persistence: fresh sandbox per execute_code call
2026-04-20 12:48:05 +03:00
Yiorgis Gozadinos
9d921b13fe
reset sandbox per skill invocation to prevent state leaks
2026-04-20 12:18:14 +03:00
Yiorgis Gozadinos
167c837514
repository methods, module-level executor, read-only VFS
2026-04-20 11:57:39 +03:00
Yiorgis Gozadinos
27c5defdbb
Improve analysis SKILL.md
2026-04-20 11:38:09 +03:00
Yiorgis Gozadinos
4d75943df5
use MontyRepl for persistent variables across execute_code calls
2026-04-20 10:47:50 +03:00
Yiorgis Gozadinos
68a9f191d2
Remove documents from state, fix tests
2026-04-20 09:48:18 +03:00
Yiorgis Gozadinos
d52f453c44
flatten skill architecture: replace ask/analyze/research with direct tools
2026-04-17 18:35:17 +03:00
Yiorgis Gozadinos
20e40d75f9
add collapsible program display to chat TUI
...
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-17 18:35:17 +03:00
Yiorgis Gozadinos
7d98d0ec57
add citation support to analysis agent
2026-04-17 18:35:16 +03:00
Yiorgis Gozadinos
cf89ff55cd
When setting --model, set all subagents as well
2026-04-17 18:35:16 +03:00
Yiorgis Gozadinos
aa4ce07dc9
remove filter parameter from analyze skill tool and clean up unused filter helpers
2026-04-17 18:35:16 +03:00
Yiorgis Gozadinos
2a0e89bbe7
add --skill flag to chat TUI for rag and analysis skills
2026-04-17 18:35:16 +03:00
Yiorgis Gozadinos
a45820dbf7
add document virtual filesystem to analysis sandbox
...
Replace get_document() and get_docling_document() with a VFS at
/documents/{id}/ with metadata.json (eager), content.txt (lazy),
and items.jsonl (lazy). Keep search(), list_documents() (now returns
all), and llm() as external functions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-17 18:35:16 +03:00
Yiorgis Gozadinos
bacc21b38b
expose doc_item_refs and labels in sandbox search results
2026-04-17 18:35:16 +03:00
Yiorgis Gozadinos
4118533db1
fold context expansion into sandbox search and remove get_context
2026-04-17 18:35:16 +03:00
Yiorgis Gozadinos
44fd0c9906
add get_context() to analysis sandbox and improve prompt
2026-04-17 18:32:45 +03:00
Yiorgis Gozadinos
d2b3ba1b59
rename RLM agent to analysis throughout the codebase
2026-04-17 18:32:01 +03:00
Yiorgis Gozadinos
499a843a43
remove unused create_analysis_toolset and AnalysisResult
2026-04-17 18:32:01 +03:00
Yiorgis Gozadinos
1049586469
vb
2026-04-17 16:24:25 +03:00
Yiorgis Gozadinos
a1181ff52d
extract get_database_stats and share it across info, inspector, and backend
2026-04-17 15:33:30 +03:00
Yiorgis Gozadinos
a0a9a3410b
info: report partial stats and pending migrations on pre-migration DBs
2026-04-17 14:00:03 +03:00
Yiorgis Gozadinos
75c84805b4
Improve judge prompt, QA prompt, and raise max_searches to 5
2026-04-16 12:12:23 +03:00
Yiorgis Gozadinos
d3c8ed62a8
preserve original chunk when expansion produces less content
2026-04-16 12:11:54 +03:00
Yiorgis Gozadinos
e2bac1e887
deduplicate escape_sql_string into utils
2026-04-16 12:11:53 +03:00
Yiorgis Gozadinos
3c1a33779f
use windowed fetch for context expansion instead of loading all items
2026-04-16 12:11:53 +03:00
Yiorgis Gozadinos
b6113bf8ab
replace fixed-radius expansion with section-bounded algorithm
...
Context expansion is now automatic and structure-aware. For structured
documents, expands within the section containing the match. For sections
that exceed the budget or are too small, expands item-by-item outward
skipping noise labels. Unstructured documents use budget-based outward
expansion. Results sorted by relevance score.
2026-04-16 12:11:53 +03:00
Yiorgis Gozadinos
364b1bc509
add document_items table for fast context expansion
2026-04-16 12:11:53 +03:00
Yiorgis Gozadinos
661e0d34d8
Pin docling-core to avoid surprises with version bumps
2026-04-16 12:11:53 +03:00
Yiorgis Gozadinos
e16c819e83
Adapt to new haiku.skills
2026-04-16 11:50:55 +03:00
Yiorgis Gozadinos
0d6df472af
bump ag-ui protocol deps, fix duplicate React key in chat
2026-04-14 12:50:25 +03:00
Yiorgis Gozadinos
dcde86a5f1
vb
2026-04-09 15:33:19 +03:00
Yiorgis Gozadinos
a51b922be3
fix skill analyze tool ignores state.document_filter
2026-04-09 15:29:20 +03:00
Yiorgis Gozadinos
99af4fe11d
fix skill list_documents tool ignores state.document_filter
2026-04-09 15:29:20 +03:00
Yiorgis Gozadinos
a118ee7aaa
Cleanup
2026-04-08 14:19:33 +03:00
Yiorgis Gozadinos
98641c24e4
S3 storage tests
2026-04-08 14:19:10 +03:00
Yiorgis Gozadinos
8e876ef28d
For remote stores, detect new db by checking if tables exist
2026-04-08 14:19:10 +03:00
Yiorgis Gozadinos
2522d46304
support remote storage in skill generator
2026-04-08 14:19:10 +03:00
Yiorgis Gozadinos
343bfd7199
guard app/UI filesystem checks for remote storage
2026-04-08 14:19:09 +03:00
Yiorgis Gozadinos
110accb8e7
Add ConnectionMode enum (LOCAL/CLOUD/OBJECT_STORAGE) and connect_lancedb() utility to support S3, GCS, Azure, and HDFS backends via storage_options.
2026-04-08 14:19:09 +03:00
Yiorgis Gozadinos
3fe953eccb
Minor fixes
2026-04-08 14:04:32 +03:00
Yiorgis Gozadinos
9c5bef163e
Use zstandard get_frame_parameters for max_output_size
2026-04-08 14:04:32 +03:00
Yiorgis Gozadinos
62a436dd9d
vb
2026-04-08 14:04:32 +03:00
Yiorgis Gozadinos
89e2a303e6
migration to split pages and re-compress with zstd
2026-04-08 14:04:32 +03:00
Yiorgis Gozadinos
53445b8722
Add docling_pages column to DocumentRecord/Document for separate page image storage.
2026-04-08 14:04:32 +03:00
Yiorgis Gozadinos
b614ef19e1
switch compression from gzip to zstd, use stdlib for python >= 3.14
2026-04-08 14:04:32 +03:00
Yiorgis Gozadinos
f8db7fccb6
fix: apply config.prompts.domain_preamble in generated skills
2026-04-08 13:54:53 +03:00
Yiorgis Gozadinos
3d30840e9d
vb
2026-04-07 16:36:27 +03:00
Yiorgis Gozadinos
8a24606784
Strip page images from DoclingDocument before validation, unless we use visualize_chunk()
2026-04-07 12:54:23 +03:00
