Yiorgis Gozadinos
3d30840e9d
vb
2026-04-07 16:36:27 +03:00
Yiorgis Gozadinos
8a24606784
Strip page images from DoclingDocument before validation, unless we use visualize_chunk()
2026-04-07 12:54:23 +03:00
Yiorgis Gozadinos
32258d4e2a
Add batching to embeddings for huge documents
2026-04-07 12:06:28 +03:00
Yiorgis Gozadinos
68c3fa0f79
Add order to SearchResult, add ChunkRepository.get_chunks_in_range(), to use them in _expand_with_chunks to fetch only nearby chunks
2026-04-07 12:04:05 +03:00
Yiorgis Gozadinos
6fdb15e3b2
Add DocumentRepository.get_docling_data() for lazy loading docling in expand_context
2026-04-07 11:19:04 +03:00
Yiorgis Gozadinos
52712dbdb2
Use .select() projection to fetch only id/uri/title/metadata in search
2026-04-07 11:04:40 +03:00
Yiorgis Gozadinos
e4dc60c4f1
Update core dependencies
2026-04-03 10:14:27 +03:00
Yiorgis Gozadinos
5c4164d224
vb
2026-04-01 13:32:56 +03:00
Yiorgis Gozadinos
c4c90b3930
Propagate domain_preamble to skill instructions and main agent preamble
2026-04-01 13:13:36 +03:00
Yiorgis Gozadinos
15d94bfe7f
replace UUIDs with readable identifiers in citation text
2026-04-01 11:44:17 +03:00
Yiorgis Gozadinos
127535bc56
vb
2026-03-28 10:05:50 +02:00
Yiorgis Gozadinos
ef6c69e9a8
Fix typos
2026-03-28 10:01:46 +02:00
Tres Seaver
ed69bf4262
fix: include 'db_path'/'config' in 'extras'
...
fix: 'skills.rlm.create_skill' includes extras
Closes #329 .
Closes #330 .
2026-03-27 14:20:31 -04:00
Yiorgis Gozadinos
13a71e34a7
vb
2026-03-27 16:39:22 +02:00
Yiorgis Gozadinos
f4dd2dafc3
Use haiku.skills extras for passing utility functions
2026-03-27 11:31:10 +02:00
Yiorgis Gozadinos
5eb250af05
Fix broken textual detection in chat & inspect
2026-03-26 17:42:38 +02:00
Yiorgis Gozadinos
f46b8195ed
vb
2026-03-26 14:15:09 +02:00
Yiorgis Gozadinos
a07a64a31f
enable repeat_table_header for self-contained table chunks and docling-serve parity
2026-03-26 14:02:12 +02:00
Yiorgis Gozadinos
a3a1ad9811
detect per-document failure status in docling-serve chunker
2026-03-26 14:02:11 +02:00
Yiorgis Gozadinos
f6231c0b19
Require haiku.skills>=0.11.0
2026-03-26 13:48:27 +02:00
Yiorgis Gozadinos
1c9035ebb5
Add optional db_path and config params to generated skill create_skill()
2026-03-26 13:41:58 +02:00
Yiorgis Gozadinos
9d986421a0
add visualize_chunk to generated skills and built-in RAG skill
2026-03-26 11:41:22 +02:00
Yiorgis Gozadinos
ffc70be7b0
include SKILL.md and assets in generated skill package wheels. Add README to generated skills
2026-03-26 10:49:57 +02:00
Yiorgis Gozadinos
f5014f39bf
vb
2026-03-24 17:46:51 +02:00
Yiorgis Gozadinos
75b9a172cb
Minor fixes
2026-03-24 17:45:46 +02:00
Yiorgis Gozadinos
705ce80b4f
Allow all spec-compliant names
2026-03-24 17:27:56 +02:00
Yiorgis Gozadinos
7d50edea86
Fix validate_metadata layering and type annotations in _tools.py
2026-03-24 17:09:14 +02:00
Yiorgis Gozadinos
ce95234cc8
CLI for create-skill;
2026-03-24 15:44:46 +02:00
Yiorgis Gozadinos
26da4a0624
Skill generator core
2026-03-24 15:44:24 +02:00
Yiorgis Gozadinos
a778ea68a8
Extract reusable tool functions from skill implementations
2026-03-24 14:46:28 +02:00
Yiorgis Gozadinos
8a5188aa01
vb
2026-03-24 14:23:29 +02:00
Yiorgis Gozadinos
aa9fdf051d
Remove support for structured output
2026-03-24 14:11:43 +02:00
Yiorgis Gozadinos
83ea6347b6
Fix CI failures: update tests for pydantic-ai 1.70.0 and FastMCP API changes
2026-03-24 14:02:03 +02:00
Yiorgis Gozadinos
ba83779bb2
Update skills
2026-03-24 13:30:23 +02:00
Yiorgis Gozadinos
ce320ca772
Update docling, docling-core and move to version 1.10.0 of DoclingDocument.
2026-03-23 17:34:06 +02:00
Yiorgis Gozadinos
fd327996b8
Configurable judge and reflect models for evaluations
2026-03-18 17:14:11 +02:00
Yiorgis Gozadinos
9243f56e1c
vb
2026-03-16 14:30:08 +02:00
Yiorgis Gozadinos
d4e34ab79f
Support for plantuml diagrams
2026-03-13 17:18:19 +02:00
Yiorgis Gozadinos
f961f00f2c
vb
2026-03-13 13:24:31 +02:00
Yiorgis Gozadinos
6548f1f0aa
Update haiku.skills to 0.8.0
2026-03-13 13:12:55 +02:00
Yiorgis Gozadinos
1d108b42ec
handle ActivitySnapshotEvent for skill sub-agent tool calls
2026-03-13 11:37:24 +02:00
Yiorgis Gozadinos
b09b45e8fa
Merge pull request #308 from ggozad/feat/rlm-monty-update
...
RLM monty update.
2026-03-13 11:23:39 +02:00
Tres Seaver
53481a7d7a
chore: suppress deprecation wwarning from 'lancedb'
2026-03-12 17:11:58 -04:00
Yiorgis Gozadinos
713e9087cc
Bump pydantic-monty to 0.0.8, remove regex external functions, update skill
2026-03-12 17:24:01 +02:00
Yiorgis Gozadinos
7696b3f98f
vb
2026-03-12 14:46:13 +02:00
Yiorgis Gozadinos
8c7eec8a8d
Strengthen cited chunk structured output description
2026-03-12 14:00:24 +02:00
Yiorgis Gozadinos
90fe624fa7
improve QA citation reliability via prompt and ID normalization
2026-03-12 11:50:46 +02:00
Yiorgis Gozadinos
b8c377da1e
Prevent read-only mode from creating tables in empty directories
2026-03-12 11:42:04 +02:00
Yiorgis Gozadinos
8d6617d63f
vb
2026-03-11 17:08:22 +02:00
Yiorgis Gozadinos
45e94ec67a
Set search limit to 10
2026-03-11 12:26:41 +02:00
Yiorgis Gozadinos
bf8eec68e3
Use dict-keyed counter for concurrent-safe search cap
2026-03-11 12:19:19 +02:00
Yiorgis Gozadinos
e2749ad2a6
Cap QA agent search iterations to reduce response time
2026-03-11 12:13:43 +02:00
Yiorgis Gozadinos
f366aea905
vb
2026-03-06 11:12:54 +02:00
Yiorgis Gozadinos
5a0430857f
Enable thinking by default for QA agent
2026-03-05 15:48:08 +02:00
Yiorgis Gozadinos
3cc2d5e19e
Set appropriate temperature and max_tokens defaults
2026-03-05 13:23:50 +02:00
Yiorgis Gozadinos
d460d553f3
Set repeat_table_header=False to match docling serve
2026-03-05 13:02:18 +02:00
Yiorgis Gozadinos
30ebd6694b
Update dependencies
2026-03-05 13:01:53 +02:00
Yiorgis Gozadinos
878423ea2e
vb
2026-03-04 15:02:34 +02:00
Yiorgis Gozadinos
28e90e5a87
Auto-detect structured output mode from model profile. Remove the structured_output config field from ModelConfig.
2026-03-04 14:51:09 +02:00
Yiorgis Gozadinos
fdd7c21757
Add configurable structured output mode (tool vs native)
2026-03-04 13:35:32 +02:00
Yiorgis Gozadinos
3f02abbc70
Expose skill state metadata as module-level API
2026-03-04 11:38:00 +02:00
Yiorgis Gozadinos
8a859fbe3c
vb
2026-03-03 17:25:53 +02:00
Yiorgis Gozadinos
034ee27daf
Use ToolOutput for structured output.
2026-03-03 16:14:45 +02:00
Yiorgis Gozadinos
1eca0fc308
Fix AGENT_PREAMBLE
2026-03-03 15:06:58 +02:00
Yiorgis Gozadinos
9a4cda1d65
Reinforce ask prompt in skill
2026-03-03 14:04:53 +02:00
Yiorgis Gozadinos
2566bf1a61
Adapt to haiku.skills streaming sub-agent ag-ui events
2026-03-03 11:16:44 +02:00
Yiorgis Gozadinos
1a3ea64aa6
Remove filter parameter from search and list_documents tools
2026-03-03 11:01:58 +02:00
Yiorgis Gozadinos
70273ecf5b
Remove filter from the search tool
2026-03-02 17:15:38 +02:00
Yiorgis Gozadinos
99ca3605e7
vb
2026-02-28 14:11:25 +02:00
Yiorgis Gozadinos
52ef983e75
Fix default chat model, default to config.qa.model
2026-02-28 14:00:40 +02:00
Yiorgis Gozadinos
27a71c7e01
Adapt skill usage to haiku.skills prompt builder. Closes #289
2026-02-28 13:34:54 +02:00
Yiorgis Gozadinos
50116bb49a
vb
2026-02-26 11:21:46 +02:00
Yiorgis Gozadinos
e2bcbd8eea
Update docs, and download-models for title generation
2026-02-26 11:05:48 +02:00
Yiorgis Gozadinos
b5cb2852c4
Simplify _resolve_title()
2026-02-26 10:56:28 +02:00
Yiorgis Gozadinos
41f694d377
Better exception handling
2026-02-26 08:55:53 +02:00
Yiorgis Gozadinos
49f9843db3
add generate_title(), rebuild --title-only, and --title on add
2026-02-26 08:40:07 +02:00
Yiorgis Gozadinos
ae3469b282
add automatic title generation for documents
2026-02-25 17:12:24 +02:00
Yiorgis Gozadinos
2acb0292e4
vb
2026-02-24 13:30:01 +02:00
Yiorgis Gozadinos
6759a2e0a5
Give regex to monty as externals
2026-02-24 13:12:56 +02:00
Yiorgis Gozadinos
1ee9747551
Catch MontyRuntimeError from Monty() constructor
2026-02-24 12:11:20 +02:00
Yiorgis Gozadinos
380fc6c930
Consolidate prompt
2026-02-24 12:09:11 +02:00
Yiorgis Gozadinos
459fdfca71
Add get_docling_document external function to sandbox
2026-02-24 11:51:43 +02:00
Yiorgis Gozadinos
d3c6322481
Simplify sandbox with run_monty_async, replace manual ThreadPoolExecutor start/resume loop
2026-02-24 11:35:39 +02:00
Yiorgis Gozadinos
b9fa1a061f
Update monty
2026-02-24 09:55:45 +02:00
Yiorgis Gozadinos
017712c9b1
Update prompts & docs. Remove docker sandbox workflow
2026-02-24 09:55:45 +02:00
Yiorgis Gozadinos
c61ded1271
Replace Docker sandbox with pydantic-monty
2026-02-24 09:55:40 +02:00
Yiorgis Gozadinos
2cafb5c8cc
vb
2026-02-20 19:41:17 +02:00
Yiorgis Gozadinos
b23f99af06
Fix history too
2026-02-20 19:15:28 +02:00
Yiorgis Gozadinos
388e94ae80
info() should always be read-only
2026-02-20 19:13:45 +02:00
Yiorgis Gozadinos
e186e13cd2
vb
2026-02-20 19:01:16 +02:00
Yiorgis Gozadinos
cb5935a6fa
Show in tui and nextjs app what is being requested from a skill
2026-02-20 18:14:44 +02:00
Yiorgis Gozadinos
492f5cf1e4
Avoid mutable defaults
2026-02-20 17:59:35 +02:00
Yiorgis Gozadinos
d1fdbdeb21
Refactor MCP to use proper models, test
2026-02-20 17:39:02 +02:00
Yiorgis Gozadinos
69083c8cb4
Fix stale docs
2026-02-20 16:59:58 +02:00
Yiorgis Gozadinos
339ff15018
Update rlm SKILL.md
2026-02-20 16:45:41 +02:00
Yiorgis Gozadinos
5dd8a82d1f
Exclude TUI apps from coverage
2026-02-20 16:37:16 +02:00
Yiorgis Gozadinos
a1c09e1a47
Break apart RLM analysis as a separate skill
2026-02-20 16:37:15 +02:00
Yiorgis Gozadinos
37f28ea8de
Clean up stale references and dead code
2026-02-20 16:37:15 +02:00
Yiorgis Gozadinos
eb9436eb2a
Remove unused conversational output mode from research graph
2026-02-20 16:36:50 +02:00
Yiorgis Gozadinos
e6310fc484
Fix citations typing
2026-02-20 16:36:50 +02:00
Yiorgis Gozadinos
c567842420
Rewire document filter
2026-02-20 16:36:50 +02:00
Yiorgis Gozadinos
972fb18d30
Wire AG-UI state round-tripping in app backend
2026-02-20 16:36:50 +02:00
Yiorgis Gozadinos
696f8598eb
Remove get_session_context, handle it automatically in ask()
2026-02-20 16:36:49 +02:00
Yiorgis Gozadinos
a7850e2210
Assign stable index to citations. Use a strong prompt preamble for our app agents
2026-02-20 16:36:48 +02:00
Yiorgis Gozadinos
643ed20d6f
Rewrite chat TUI and app backend with haiku.skills
2026-02-20 16:36:48 +02:00
Yiorgis Gozadinos
8176103eb0
Remove agents/chat/
2026-02-20 16:36:48 +02:00
Yiorgis Gozadinos
ed89ff0fc9
Simplify tools/ — remove AG-UI state machinery, keep core toolsets
2026-02-20 16:36:48 +02:00
Yiorgis Gozadinos
524647c501
Add unified RAG skill with session context and reuse existing tools;
2026-02-20 16:36:47 +02:00
Yiorgis Gozadinos
4855051936
Add research() to client and add haiku.skills dependency, remove --deep flag and simplify app
2026-02-20 16:36:47 +02:00
Yiorgis Gozadinos
0c270ba42f
vb
2026-02-19 14:28:13 +02:00
Yiorgis Gozadinos
08a4252c33
download-models, show actionable error message when Ollama is not running
2026-02-19 14:06:53 +02:00
Yiorgis Gozadinos
df5c8bcfaf
Add cachetools dependency
2026-02-19 13:04:07 +02:00
Yiorgis Gozadinos
c62fe5fce8
vb
2026-02-17 13:56:43 +02:00
Yiorgis Gozadinos
8e67d0212e
Merge pull request #279 from ggozad/chore/dep-update
...
Update dependencies
2026-02-17 13:54:37 +02:00
Yiorgis Gozadinos
a870052ee9
Fix coverage
2026-02-17 13:22:23 +02:00
Yiorgis Gozadinos
2d5fbd06f2
Update dependencies
2026-02-17 13:12:08 +02:00
Yiorgis Gozadinos
f4735667df
emit StateDeltaEvent with client-aware baseline
2026-02-17 12:24:09 +02:00
Yiorgis Gozadinos
9685bd533b
Expand ~ in paths
2026-02-16 17:21:35 +02:00
Yiorgis Gozadinos
53f6cd3579
vb
2026-02-16 13:12:06 +02:00
Yiorgis Gozadinos
8c76897af2
Update tui and app to use the toolkit
2026-02-16 12:54:44 +02:00
Yiorgis Gozadinos
7ff2123806
Introduce toolkit to reduce toolset creation ceremony
2026-02-16 11:51:02 +02:00
Yiorgis Gozadinos
72c8f6e1b1
Additional tests
2026-02-13 18:08:46 +02:00
Yiorgis Gozadinos
84c48e1541
Add preamble parameter to chat agent and citations_history to ChatSessionState
2026-02-13 16:59:21 +02:00
Yiorgis Gozadinos
5767158737
Paddings and ui fixes
2026-02-13 16:42:03 +02:00
Yiorgis Gozadinos
969c64d106
Add citations_history to SessionState for unified citation rendering between search and ask
2026-02-13 16:21:50 +02:00
Yiorgis Gozadinos
c0ed93da2d
Expand tests
2026-02-13 14:34:24 +02:00
Yiorgis Gozadinos
f5c562db9b
Fix double summarization, cleanup
2026-02-13 12:16:23 +02:00
Yiorgis Gozadinos
b64d9721bb
Mark with pragma no cover areas that are just wiring. Consolidate tests and bring coverage to 100%
2026-02-13 11:56:37 +02:00
Yiorgis Gozadinos
d9acdfba9d
prompt builder
2026-02-13 11:21:09 +02:00
Yiorgis Gozadinos
5f6d9e9812
handle state_key duplication between deps and ToolContext
2026-02-13 10:23:12 +02:00
Yiorgis Gozadinos
a5b8e3be7e
Add AgentDeps and prepare_context for custom agent DX
2026-02-12 17:37:37 +02:00
Yiorgis Gozadinos
8f180f707b
Decouple QA toolset from chat agent, add ToolContext state methods
2026-02-12 17:27:47 +02:00
Yiorgis Gozadinos
0c6d73409d
Move SessionContext from agents/chat/state.py to tools/session.py, change QASessionState.session_context to use it
2026-02-12 17:13:01 +02:00
Yiorgis Gozadinos
05ebd781d4
Move client and tool_context from toolset factories to RunContext.deps
2026-02-12 15:50:46 +02:00
Yiorgis Gozadinos
51f1d9cf13
Issue a full state snapshot from ask() tool to preserve server side background context
2026-02-11 17:49:09 +02:00
Yiorgis Gozadinos
58c2bd49ea
Clean up server vs client session setting priorities
2026-02-11 16:54:00 +02:00
Yiorgis Gozadinos
0434cd068b
Simplify TUI app
2026-02-11 15:10:50 +02:00
Yiorgis Gozadinos
9e63ff1ff7
Remove session_id from state layer, remove module-level caches
2026-02-11 14:55:53 +02:00
Yiorgis Gozadinos
d88f2f003a
Remove incoming_* fields, simplify delta computation
2026-02-11 14:31:37 +02:00
Yiorgis Gozadinos
d67df09cce
Introduce ToolContextCache
2026-02-11 14:18:26 +02:00
Yiorgis Gozadinos
c916356a42
Clean up toolset exports, simplify update_session_context(), fix health check
2026-02-11 12:46:50 +02:00
Yiorgis Gozadinos
87bcb7a5dc
Extract run_qa_core() from ask closure in QA toolset. Remove redundant tests and cassettes for OpenAI and Anthropic QA tools, as they are now tested in the core function tests.
2026-02-11 11:38:34 +02:00
Yiorgis Gozadinos
f85aaefed7
Extract shared snapshot helpers
2026-02-10 17:34:30 +02:00
Yiorgis Gozadinos
a592a04031
Removed unused accumulator states
2026-02-10 16:15:30 +02:00
Yiorgis Gozadinos
ce3c3e7ed5
Add feature-based chat agent & prompt composition
2026-02-10 14:38:14 +02:00
Yiorgis Gozadinos
da47e3e345
Unify get_typed() into get() with optional type parameter
...
Fix find_document regression
2026-02-10 14:38:14 +02:00
Yiorgis Gozadinos
dd7162e72e
Fix get_or_create type annotation, export compute_combined_state_delta
2026-02-10 14:38:13 +02:00
Yiorgis Gozadinos
bd3b26f78b
Extract get_session_filter helper, remove dead code and duplicate tests
2026-02-10 14:38:13 +02:00
Yiorgis Gozadinos
df57b0cf43
Remove SearchAgent & friends, consolidate duplicate models
2026-02-10 14:38:13 +02:00
Yiorgis Gozadinos
84b320765a
Fix analysis tool to use docker sandbox
2026-02-10 14:38:13 +02:00
Yiorgis Gozadinos
3ed9cbb7d3
Use the tools inside the chat agent.
2026-02-10 14:38:12 +02:00
Yiorgis Gozadinos
5bcbcc3928
Remove analyze tool from chat agent
2026-02-10 14:38:12 +02:00
Yiorgis Gozadinos
2be3389222
DocumentToolset, QAToolset, AnalysisToolset
2026-02-10 14:38:12 +02:00
Yiorgis Gozadinos
2f229f91e1
Create SearchToolset, refactor QA Agent to use it
2026-02-10 14:38:12 +02:00
Yiorgis Gozadinos
a3863882cd
Introduce tool module & ToolContext to save state
2026-02-10 14:38:12 +02:00
Yiorgis Gozadinos
a7b79c433d
TUI now generates a UUID session_id on mount and on chat clear
2026-02-10 14:21:22 +02:00
Yiorgis Gozadinos
aaa9b6a32f
vb
2026-02-10 13:29:30 +02:00
Yiorgis Gozadinos
d1a9439500
Fix session_id not persisting across AG-UI requests
2026-02-10 13:14:05 +02:00
Yiorgis Gozadinos
8c2a101a8a
Fix out-of-memory in list_documents by adding column projection
2026-02-06 14:11:36 +01:00
Yiorgis Gozadinos
17c2136b4e
vb
2026-02-06 12:48:16 +01:00
Yiorgis Gozadinos
20414ed959
Additional tests
2026-02-06 12:07:21 +01:00
Yiorgis Gozadinos
720f697a48
Cleanup
2026-02-06 12:07:21 +01:00
Yiorgis Gozadinos
f8ec511250
Return consolidated program from RLM agent instead of execution history
2026-02-06 12:07:20 +01:00
Yiorgis Gozadinos
e882831afb
Fix execute_code prompt
2026-02-06 12:07:20 +01:00
Yiorgis Gozadinos
a0c4ddbd01
Simplify deps
2026-02-06 12:07:20 +01:00
Yiorgis Gozadinos
122da834d7
Use resolve_document()
2026-02-06 12:07:19 +01:00
Yiorgis Gozadinos
764b62ee20
Fix tests
2026-02-06 12:07:19 +01:00
Yiorgis Gozadinos
b426827fc2
Docker-based sandbox. Reused within a single rlm() call for latency
2026-02-06 12:07:19 +01:00
Yiorgis Gozadinos
4241a4b09e
Security fixes for RLM. Fix type() builtin, AST validation for dict key access, sql injection
2026-02-06 12:07:19 +01:00
Yiorgis Gozadinos
fb6bbca124
Remove analyze tool from chat agent, not yet ready for integration
2026-02-06 12:07:19 +01:00
Yiorgis Gozadinos
9a480cee90
Return program in chat agent analyze;
2026-02-06 12:07:18 +01:00
Yiorgis Gozadinos
ed570633cd
Basic integration of RLM with chat agent
2026-02-06 12:07:18 +01:00
Yiorgis Gozadinos
198a8d5a88
Add pre-loaded documents support and improve RLM test coverage
2026-02-06 12:07:04 +01:00
Yiorgis Gozadinos
fa4d7731d7
Replace ask() with llm(). Add and document the programs written in tests
2026-02-06 12:07:03 +01:00
Yiorgis Gozadinos
5e7832a6ea
Fix REPL for docling document, add test
2026-02-06 12:07:03 +01:00
Yiorgis Gozadinos
b68b2393e9
Integrate with client, cli, app, mcp
2026-02-06 12:07:03 +01:00
Yiorgis Gozadinos
75de81accf
RLM agent
2026-02-06 12:07:03 +01:00
Yiorgis Gozadinos
ad416cac84
RLM sandbox environment
2026-02-06 12:07:02 +01:00
Yiorgis Gozadinos
b22ac33dde
Pass OCR options from conversion_options to docling-serve chunker
2026-02-03 11:34:46 +02:00
Yiorgis Gozadinos
1e22937482
vb
2026-01-31 11:12:38 +02:00
Yiorgis Gozadinos
ac76ecee86
Remove dead code from iterative research planning refactor
2026-01-31 10:51:34 +02:00
Yiorgis Gozadinos
57d5b1bde5
Simplify research planning: remove gather_context, always use planner
2026-01-30 22:32:08 +02:00
Yiorgis Gozadinos
109f770a2a
Refactor research graph to iterative planning approach
2026-01-30 20:46:29 +02:00
Yiorgis Gozadinos
4264cdcc24
vb
2026-01-29 15:35:42 +02:00
Yiorgis Gozadinos
77894b8c35
Clena citation handling now that we handle the total state
2026-01-29 12:12:48 +02:00
Yiorgis Gozadinos
202df368a0
TUI now syncs full session state from AG-UI events
2026-01-29 12:05:46 +02:00
Yiorgis Gozadinos
cbd94f6cc9
Always use state deltas instead of conditional snapshot/delta logic
2026-01-28 17:03:18 +02:00
Yiorgis Gozadinos
57fd08d600
Revert "Synthesis steps now select only relevant citations instead of including all"
2026-01-28 16:00:27 +02:00
Yiorgis Gozadinos
e2158c7dad
Handle snapshot deltas in TUI
2026-01-28 15:23:56 +02:00
Yiorgis Gozadinos
3741274c6c
Update ty and fix
2026-01-28 14:24:16 +02:00
Yiorgis Gozadinos
b5aeefb10f
Fix state delta computation by avoiding mutation of original state
2026-01-28 14:06:29 +02:00
Yiorgis Gozadinos
c27dde2497
Send state deltas updates instead of full state snapshots
2026-01-28 14:06:29 +02:00
Yiorgis Gozadinos
626f5cdd0e
Make list_documents return structured output and give feedback about total count and pages
2026-01-27 17:00:39 +02:00
Yiorgis Gozadinos
cca03dbe61
Give the conversational agent a list documents and a summarize tools
2026-01-27 16:39:48 +02:00
Yiorgis Gozadinos
8a587ec554
Synthesis steps now select only relevant citations instead of including all
2026-01-27 15:56:29 +02:00
Yiorgis Gozadinos
614c34b98f
Add read-only initial context for chat sessions for TUI and web app
2026-01-27 13:59:00 +02:00
Yiorgis Gozadinos
bb634e68a4
vb
2026-01-27 11:17:17 +02:00
Yiorgis Gozadinos
09aa94bfe5
Fix datetime JSON serialization in AG-UI StateSnapshotEvent
2026-01-27 11:05:01 +02:00
Yiorgis Gozadinos
ee3cb5bd87
Allow initial_context to be set for the chat agent
2026-01-27 10:38:18 +02:00
Yiorgis Gozadinos
7c9d70fa62
vb
2026-01-26 16:46:33 +02:00
Yiorgis Gozadinos
0465e1eebd
Update prompt to remove the recall tool
2026-01-26 16:43:11 +02:00
Yiorgis Gozadinos
a7d7a09c22
Integrate recall functionality into ask tool
2026-01-26 16:43:11 +02:00
Yiorgis Gozadinos
276e86a033
Fix citation selection when we call visualize
2026-01-26 16:43:10 +02:00
Yiorgis Gozadinos
7f63a7c4ab
Use recall tool in chat agent
2026-01-26 16:43:10 +02:00
Yiorgis Gozadinos
77647b08bc
Recall tool in chat agent
2026-01-26 16:42:41 +02:00
Yiorgis Gozadinos
8f35033a42
Update search(), ask() to use get_or_assign_index()
2026-01-26 16:42:41 +02:00
Yiorgis Gozadinos
af786596b1
Document filter in the TUI, use command palette instead of shortcuts
2026-01-26 16:42:40 +02:00
Yiorgis Gozadinos
30e9ff3038
Use session document filter in agent, combine with document name filter if necessary
2026-01-26 16:42:40 +02:00
Yiorgis Gozadinos
5b14038dc5
Add document_filter to ChatSessionState for session-level filtering
2026-01-26 16:42:40 +02:00
Yiorgis Gozadinos
0f0c3a076f
minor fixes
2026-01-26 16:42:39 +02:00
Yiorgis Gozadinos
729a3910d8
Fix visual grounding citation selection in TUI
2026-01-26 16:42:39 +02:00
Yiorgis Gozadinos
ff92ee8271
Skip gather_context when session context exists
2026-01-26 16:42:39 +02:00
Yiorgis Gozadinos
5aabae4d06
Remove background_context & simplify
2026-01-26 16:42:39 +02:00
Yiorgis Gozadinos
9d71ccb213
Move session context to agent, make AGUI work
2026-01-26 16:42:39 +02:00
Yiorgis Gozadinos
93737852a8
Use previous context in summarization
2026-01-26 16:42:38 +02:00
Yiorgis Gozadinos
9ed8773672
Unify Citation class and context formatting between chat and research agents
2026-01-26 16:42:38 +02:00
Yiorgis Gozadinos
74eac57ffa
Remove qa_history ranking in favor of SessionContext, planner can skip searching if context adequate
2026-01-26 16:42:38 +02:00
Yiorgis Gozadinos
3b7b7c1bbc
Cancel any running summarization tasks
2026-01-26 16:42:38 +02:00
Yiorgis Gozadinos
8441516a73
Tests, widget to show context in tui
2026-01-26 16:41:13 +02:00
Yiorgis Gozadinos
aaddf4a5a9
Dynamic session context to be used in the conversational agent
2026-01-26 16:41:12 +02:00
Yiorgis Gozadinos
47d355ed0a
Update dependencies
2026-01-23 11:57:02 +02:00
Yiorgis Gozadinos
35ebdd67e2
vb
2026-01-22 17:48:39 +02:00
Yiorgis Gozadinos
b4c158ffcf
Fix potential 0.25.0 migration crash
2026-01-22 17:29:26 +02:00
Yiorgis Gozadinos
32c64d3d11
vb
2026-01-22 12:59:32 +02:00
Yiorgis Gozadinos
ea7a4a9a8f
apply dotenv usecwd fix to remaining files
...
Apply the same find_dotenv(usecwd=True) fix from cli.py to:
- evaluations/evaluations/benchmark.py
- app/backend/main.py
Co-Authored-By: Tianyi Cui <contact@tianyicui.com>
2026-01-22 12:41:35 +02:00
Tianyi Cui
910ebbab22
fix: load .env from current working directory, not source file directory
...
The previous load_dotenv() searched for .env starting from the calling
file's directory (haiku/rag/cli.py), which fails when users run haiku-rag
from a different directory containing their .env file.
Changes:
- Move load_dotenv() before config imports (config reads env vars at import time)
- Use find_dotenv(usecwd=True) to search from current working directory
2026-01-22 12:36:55 +02:00
Tres Seaver
e9746a1785
fix: mark 'hashlib.md5' as 'usedforsecurity=False'
...
Closes #255 .
2026-01-21 12:14:56 -05:00
Yiorgis Gozadinos
b96e5a3be1
Additional prompts needing adapting. Missing cassettes.
2026-01-21 15:17:56 +02:00
Yiorgis Gozadinos
7f51c63ad7
Search results now show rank position instead of raw scores. Consolidate cassettes
2026-01-21 14:54:15 +02:00
Yiorgis Gozadinos
a1abfd9666
Jina uses AutoModel not AutoModelForSequenceClassification
2026-01-21 11:26:23 +02:00
Yiorgis Gozadinos
ce95dc47a5
Do not run coverage on download-models & code that requires integration tests
2026-01-21 11:09:35 +02:00
Yiorgis Gozadinos
36a675e1bb
Add jina, mxbai and sentence-transformers to download-models when used
2026-01-21 10:53:45 +02:00
Yiorgis Gozadinos
7ff014684a
Support for jina reranker, both local and API
2026-01-21 10:39:21 +02:00
Yiorgis Gozadinos
b8cf8f5198
Remove reranker cache
2026-01-20 17:26:24 +02:00
Yiorgis Gozadinos
e7ce9694b5
vb
2026-01-20 15:47:37 +02:00
Yiorgis Gozadinos
be82473624
Merge pull request #249 from ggozad/feat/ocr-engine
...
Add `ocr_engine` field to `ConversionOptions`
2026-01-20 13:56:23 +02:00
Yiorgis Gozadinos
7bbac66640
Fix crash when logfire is installed but user is not authenticated. Closes #247
2026-01-20 13:43:44 +02:00
Yiorgis Gozadinos
62d35d2907
Add ocr_engine field to ConversionOptions
2026-01-20 13:41:43 +02:00
Yiorgis Gozadinos
1022d5dafd
Update dependencies
2026-01-19 16:23:26 +02:00
Yiorgis Gozadinos
f05159859c
Additional fixes to remove type ignores
2026-01-19 16:00:18 +02:00
Yiorgis Gozadinos
5e55c0df54
replace pyright with ty type checker
2026-01-19 15:50:14 +02:00
Yiorgis Gozadinos
2c264f4afc
vb
2026-01-19 14:54:18 +02:00
Yiorgis Gozadinos
5464d24483
require explicit migrate command for database migrations
2026-01-19 13:22:09 +02:00
Yiorgis Gozadinos
5f2f180960
vb
2026-01-16 15:27:55 +02:00
Yiorgis Gozadinos
bdfc6b87e7
Rename initial_context to background_context
2026-01-16 15:03:42 +02:00
Yiorgis Gozadinos
2054450142
ChatDeps implements StateHandler protocol for proper AG-UI state management
2026-01-16 15:03:41 +02:00
Yiorgis Gozadinos
18362f01c2
Pass context for CLI/app
2026-01-16 15:03:06 +02:00
Yiorgis Gozadinos
d82e1f95c1
Optimize context prompts
2026-01-16 15:03:05 +02:00
Yiorgis Gozadinos
dff489f75f
Add initial_context support to chat agent
2026-01-16 15:03:05 +02:00
Yiorgis Gozadinos
e73486b7a8
Set retries to 3 for agents that did not have it.
2026-01-16 15:03:05 +02:00
Yiorgis Gozadinos
12e5e9ebfb
vb
2026-01-15 16:24:03 +02:00
Yiorgis Gozadinos
7639c2915e
Add AGUI_STATE_KEY for namespaced AG-UI state emission
2026-01-15 15:46:37 +02:00
Yiorgis Gozadinos
157773ef15
vb
2026-01-15 11:47:49 +02:00
Yiorgis Gozadinos
bb60ad127a
Add pragma nocover to optional dependency imports
2026-01-15 10:32:35 +02:00
Yiorgis Gozadinos
fae274a04d
Remove defensive checks in info()
2026-01-15 10:30:45 +02:00
Yiorgis Gozadinos
345807e699
Remove defensive app() try/except, let errors propagate if they occur
2026-01-15 10:17:26 +02:00
Yiorgis Gozadinos
affffc1013
Extract get_package_versions() for info & modals
2026-01-14 18:24:41 +02:00
Yiorgis Gozadinos
fe73267e66
Remove unecessary try/except from info()
2026-01-14 18:17:43 +02:00
Yiorgis Gozadinos
fc1ea82cb1
Display on info pydantic-ai and docling-document schema version
2026-01-14 18:14:53 +02:00
Yiorgis Gozadinos
8bdfbe526c
Fix embed-only rebuild with changed vector dimensions
...
When a database was created with one embedding model and rebuild
--embed-only was run with a different model, it failed with a
vector dimension validation error.
- Store reads stored vector_dim when opening existing databases
- _rebuild_embed_only recreates chunks table to handle dimension changes
- Add test for rebuild with changed vector dimensions
2026-01-14 17:42:28 +02:00
Yiorgis Gozadinos
11f7f02262
Add option to namespace state key as a "feature" for ag-ui apps that maintain their own state
2026-01-14 14:21:08 +02:00
Yiorgis Gozadinos
929c29250e
Additional tests for page & picture images for local and docling-serve
2026-01-14 12:48:13 +02:00
Yiorgis Gozadinos
447e94c5db
Config options for generating page images
2026-01-14 11:55:08 +02:00
Yiorgis Gozadinos
2930679dc1
vb
2026-01-13 19:47:27 +02:00
Yiorgis Gozadinos
98b3d114b3
Update to docling-document 1.9.0
2026-01-13 19:20:27 +02:00
Yiorgis Gozadinos
275b09c8c5
vb
2026-01-13 18:34:45 +02:00
Yiorgis Gozadinos
c9cbf04cc8
Reverted docling from 2.67.0 to 2.65.0 to use docling-document 1.8.0
2026-01-13 18:22:58 +02:00
Yiorgis Gozadinos
eb5d0b0622
Use tui as an optional group. vb
2026-01-13 12:30:19 +02:00
Yiorgis Gozadinos
6b3ff5ef1e
CLI app
2026-01-13 11:31:21 +02:00
Yiorgis Gozadinos
3e66d6b21d
Make qa list a FIFO with 50 max, keep a cache of embeddings
2026-01-12 17:02:23 +02:00
Yiorgis Gozadinos
51836a1c55
Add tests for chat agent & friends
2026-01-12 16:00:07 +02:00
Yiorgis Gozadinos
f4ddb0f5c6
Allow chat agent to specify the search limit
2026-01-12 15:52:40 +02:00
Yiorgis Gozadinos
2da2c6d13a
Search tool in chat agent needs no search context
2026-01-12 15:13:43 +02:00
Yiorgis Gozadinos
5adab04e50
Update prompts
2026-01-12 15:10:04 +02:00
Yiorgis Gozadinos
d43821659c
Filter low-confidence answers
2026-01-12 15:01:32 +02:00
Yiorgis Gozadinos
e4a7d86348
Rank q/a history with respect to cosine similarity to new queries. Adapt the chat agent to only pass relevant q/as
2026-01-12 14:15:36 +02:00
Yiorgis Gozadinos
7cc561d1db
Refactor to make agents a top-level module. Bring in the conversational agent from the app
2026-01-12 12:37:08 +02:00
Yiorgis Gozadinos
bb61e8fab6
Remove custom AG-UI infrastructure in favor of pydantic-ai native support
2026-01-12 12:37:07 +02:00
Yiorgis Gozadinos
26bc02ef14
Build a "conversational" research graph
2026-01-12 12:36:31 +02:00
Yiorgis Gozadinos
061095e855
Remove obsolete migrations
2026-01-12 12:07:49 +02:00
Yiorgis Gozadinos
29c682d8eb
Add migration
2026-01-11 11:40:57 +02:00
Yiorgis Gozadinos
9c4fd08c47
Update client serialization for compressed docling documents
2026-01-11 09:23:56 +02:00
Yiorgis Gozadinos
be715dd456
Revert docling core back to 1.8.0 for compatibility with docling-serve
2026-01-11 09:23:19 +02:00
Yiorgis Gozadinos
afc5b02285
Update schema to store docling_document as compressed bytes
...
- Change DocumentRecord.docling_document_json (str) to docling_document (bytes)
- Use large_binary Arrow type (64-bit offsets) to avoid 2GB column limit
- Decompress in Document.get_docling_document() using gzip
- Update DocumentRepository field mappings
2026-01-10 15:17:27 +02:00
Yiorgis Gozadinos
face6f95bc
compression utils
2026-01-10 15:11:58 +02:00
Yiorgis Gozadinos
09acc845c0
Update docling, docling-core, lancedb deps
2026-01-10 15:06:56 +02:00
Yiorgis Gozadinos
ff1f19bb08
vb
2026-01-08 18:50:39 +02:00
Yiorgis Gozadinos
3603b00513
Fix base64 images leaking into expanded search results
...
PictureItem.export_to_markdown() defaults to ImageRefMode.EMBEDDED,
which embeds base64 image data. When _extract_item_text() called this
method during context expansion, base64 data would leak into results.
Now explicitly handles PictureItem with ImageRefMode.PLACEHOLDER to
prevent base64 while still including VLM descriptions and captions.
2026-01-08 18:42:58 +02:00
Yiorgis Gozadinos
92f0fd956f
vb
2026-01-08 10:25:25 +02:00
Yiorgis Gozadinos
1616891c40
Fix non-reasoning OpenAI models that do not support reasoning config
2026-01-07 18:47:31 +02:00
Yiorgis Gozadinos
ef187767ed
vb
2026-01-07 11:20:48 +02:00
Yiorgis Gozadinos
2f413e937b
Add VLM model to download_models()
2026-01-07 11:09:24 +02:00
Yiorgis Gozadinos
f861664656
Extract shared DoclingServeClient for async workflow & use it for VLM support, chunking, converting
2026-01-07 11:09:23 +02:00
Yiorgis Gozadinos
769e0d4746
Update picture description prompt, move its config under prompts
2026-01-07 11:09:23 +02:00
Yiorgis Gozadinos
3c4116397d
Add VLM picture description support for image handling
2026-01-07 11:09:22 +02:00
Yiorgis Gozadinos
af340ac410
vb
2026-01-05 12:15:41 +02:00
Yiorgis Gozadinos
271af6cb3c
Update core dependencies
2026-01-05 11:52:38 +02:00
Yiorgis Gozadinos
5b34b9ed0e
Track potentially more than one running step in agui emitter
2026-01-05 10:41:35 +02:00
Yiorgis Gozadinos
679d0bfafd
Merge pull request #212 from ggozad/feat/contextualize-fts
...
Add additional context (headers/sections etc) to the FTS index
2025-12-29 18:42:19 +02:00
Yiorgis Gozadinos
a4d33c9e71
Delete test_agui_server.py, it was accidentally commited
2025-12-29 14:58:04 +02:00
Yiorgis Gozadinos
c9cb005d07
Use FTS on contextualized text
2025-12-29 14:50:07 +02:00
Yiorgis Gozadinos
1e9a10a235
Record cassettes for embedder, ignore huggingface while recording
2025-12-29 13:23:53 +02:00
Yiorgis Gozadinos
1f654917b8
vb
2025-12-26 13:56:29 +02:00
Yiorgis Gozadinos
08fb99c6f8
Add prompt customization and domain preable
2025-12-26 13:34:51 +02:00
Yiorgis Gozadinos
958fe43f2a
Remove vllm and lmstudio custom configs, they can now use the openai base
2025-12-26 12:02:55 +02:00
Yiorgis Gozadinos
86e31b8d29
VoyageAI embeddings, should end up as a PR for pydantic-ai
2025-12-26 11:50:47 +02:00
Yiorgis Gozadinos
3852e961b9
Update tests and client for explicit embed_query/embed_documents API
2025-12-26 11:26:56 +02:00
Yiorgis Gozadinos
132b8a36bc
Differentiate between embedding a query and a document
2025-12-24 12:43:11 +02:00
Yiorgis Gozadinos
5a30c197af
Delete obsolete embedders, rewrite get_embedder
2025-12-24 12:37:17 +02:00
Yiorgis Gozadinos
3141c91023
Add base_url to ModelConfig and EmbeddingModelConfig. Deprecate vllm and lm_studio configs, now through open ai
2025-12-24 12:33:44 +02:00
Yiorgis Gozadinos
271c50e225
pydantic-ai update
2025-12-24 12:25:22 +02:00
Yiorgis Gozadinos
5c7f033d2c
vb
2025-12-19 12:35:35 +02:00
Yiorgis Gozadinos
4164895514
Update critical dependencies
2025-12-19 12:21:57 +02:00
Yiorgis Gozadinos
2678c1aaad
history command, shows available lancedb versions per table
2025-12-19 12:03:18 +02:00
Yiorgis Gozadinos
77f9a2a1b9
before parameter in client, app, inspector, cli
2025-12-19 12:03:18 +02:00
Yiorgis Gozadinos
1e5eebfbf0
Checkout version logic and datetime utils
2025-12-19 12:03:16 +02:00
Yiorgis Gozadinos
8c0adb47e1
Merge branch 'main' into feat/read-only-db
2025-12-19 12:01:36 +02:00
Yiorgis Gozadinos
30f6bac388
Merge pull request #205 from ggozad/fix/docker-paths
...
Clarify how folder mounts work in docker docs
2025-12-19 11:59:00 +02:00
Yiorgis Gozadinos
090be293d2
Clarify folder mounts in docker docs, check monitor paths exist in FileWatcher
2025-12-19 11:58:08 +02:00
Yiorgis Gozadinos
1ff8fc1b56
Merge pull request #201 from ggozad/fix/allow-custom-config
...
Fix get_model() uses where default Config is forced
2025-12-19 11:57:30 +02:00
Yiorgis Gozadinos
fb1d3c97b3
Add additional programming language specifiers
2025-12-19 11:09:57 +02:00
Yiorgis Gozadinos
98727ad080
Add global --read-only mode in CLI
2025-12-18 16:43:49 +02:00
Yiorgis Gozadinos
1a8f72f99a
Add is_read_only property to Store. Guard all write operations in repositories
2025-12-18 15:55:24 +02:00
Yiorgis Gozadinos
e8e67d336c
Fix get_model() uses where default Config is forced
2025-12-18 15:10:31 +02:00
Yiorgis Gozadinos
1dbbaf6dd8
vb
2025-12-18 12:21:16 +02:00
Yiorgis Gozadinos
a1f5d944ba
Replace custom event classes with ag_ui.core types
2025-12-18 12:07:37 +02:00
Yiorgis Gozadinos
bf5711f7ab
Give hints in interactive cli
2025-12-18 11:48:35 +02:00
Yiorgis Gozadinos
08349e5c0e
QOL fixes for interactive cli research
2025-12-18 11:48:35 +02:00
Yiorgis Gozadinos
34092e4cdc
Wrap client side tool call in a message
2025-12-18 11:48:34 +02:00
Yiorgis Gozadinos
9818cac4eb
Fix agui tool call args being a json string
2025-12-18 11:48:34 +02:00
Yiorgis Gozadinos
218126de8d
Interactive research agent through AGUI client-side tool calls in CLI
2025-12-18 11:48:34 +02:00
Yiorgis Gozadinos
921893cff5
Remove deep ask from ag-ui server
2025-12-18 11:41:57 +02:00
Yiorgis Gozadinos
45de9bf0d5
Flatten graph structure, simplify
2025-12-18 11:41:57 +02:00
Yiorgis Gozadinos
557a296341
Update ag-ui example
2025-12-18 11:41:53 +02:00
Yiorgis Gozadinos
6e621ea34d
Provide full report on deep ask, modify prompt to not just summarize
2025-12-18 11:41:11 +02:00
Yiorgis Gozadinos
f0a7abd953
Remove deep qa graph, everything now uses the simplified research graph
2025-12-18 11:41:10 +02:00
Yiorgis Gozadinos
bdcf81774d
Remove analyze insights node, simplify research context, state, prompts & models
2025-12-18 11:41:10 +02:00
Yiorgis Gozadinos
50b7fb4461
Chunkers now set chunk.order directly
2025-12-18 11:35:48 +02:00
Yiorgis Gozadinos
3149552a61
Add 'plain' format and fallback for text conversion
2025-12-12 18:11:25 +02:00
Yiorgis Gozadinos
5c4799164c
Escape when getting by URI
2025-12-12 10:56:37 +02:00
Yiorgis Gozadinos
f6073a0db9
vb
2025-12-12 10:48:21 +02:00
Yiorgis Gozadinos
011f527579
Use rich rendering for citations in cli
2025-12-12 10:02:30 +02:00
Yiorgis Gozadinos
6f36085ac3
Fix synthesis prompt
2025-12-12 07:44:15 +02:00
Yiorgis Gozadinos
9ee8dfdc0c
Adapt prompts to clarify all lists must contain strings. Hopefully fixes LLM producing wrong output
2025-12-12 07:40:25 +02:00
Yiorgis Gozadinos
00d621e211
vb
2025-12-11 16:31:21 +02:00
Yiorgis Gozadinos
c0cbc22137
Info modal
2025-12-11 15:38:45 +02:00
Yiorgis Gozadinos
09be31f021
Lazy load chunks in inspector
2025-12-11 15:38:19 +02:00
Yiorgis Gozadinos
448d00f9b2
Add pagination to chunk repo's get_by_document_id
2025-12-11 15:38:19 +02:00
Yiorgis Gozadinos
c43c8fab22
Switch modals, do not stack them
2025-12-11 15:38:19 +02:00
Yiorgis Gozadinos
d7c755f454
Add filter to simple qa agent
2025-12-11 15:28:41 +02:00
Yiorgis Gozadinos
ecba8cfd38
Docs and cli support for filter in deep QA, Research
2025-12-11 15:23:50 +02:00
Yiorgis Gozadinos
5f4f5f70f0
Search filter in qa & research graphs
2025-12-11 14:37:43 +02:00
Yiorgis Gozadinos
7ccabf4fb9
Log to console with logfire only in development
2025-12-11 10:47:18 +02:00
Yiorgis Gozadinos
5edae13cc0
Run logfire regardless of environment
2025-12-11 10:43:25 +02:00
Yiorgis Gozadinos
2161727716
Add auto_vacuum config option to control automatic vacuuming
2025-12-10 11:42:14 +02:00
Yiorgis Gozadinos
ed809759d5
Fix mxbai rerank default model
2025-12-10 11:39:33 +02:00
Yiorgis Gozadinos
68457f251e
Do not generate base64 images by default
2025-12-09 18:42:56 +02:00
Yiorgis Gozadinos
156bba3359
Infinite scroll lazy load document list in inspector
2025-12-09 17:48:27 +02:00
Yiorgis Gozadinos
5a593e928e
Set max_context_items to 10
2025-12-09 12:47:42 +02:00
Yiorgis Gozadinos
802f058205
Move search related config under SearchConfig
2025-12-09 12:28:28 +02:00
Yiorgis Gozadinos
4da50873bd
Make search limit configurable
2025-12-09 11:46:43 +02:00
Yiorgis Gozadinos
a56e1ba67c
Simplify import_document, update_document
2025-12-08 18:19:12 +02:00
Yiorgis Gozadinos
67684c799b
Rebuild in batches to avoid creating multiple lancedb versions
2025-12-08 15:56:41 +02:00
Yiorgis Gozadinos
1b96937ec6
Update dependencies
2025-12-08 15:56:22 +02:00
Yiorgis Gozadinos
06b4f6a224
Add format parameter for text-to-DoclingDocument conversion
2025-12-08 15:56:21 +02:00
Yiorgis Gozadinos
1b14a4643d
Remove unused bounding box calculations
2025-12-08 15:56:21 +02:00
Yiorgis Gozadinos
9a5f8e4368
Add type-aware context expansion for search results
2025-12-08 15:56:21 +02:00
Yiorgis Gozadinos
dab04668bd
Dependency updates
2025-12-08 15:56:02 +02:00
Yiorgis Gozadinos
4f0f214f26
Remove unecessary tests, add no coverage pragmas where appropriate
2025-12-08 15:56:02 +02:00
Yiorgis Gozadinos
2572125804
Move embedding logic from ChunkRepository to client._ensure_chunks_embedded()
2025-12-08 15:56:02 +02:00
Yiorgis Gozadinos
e97d235c98
Unify update_document, update_document_fields
2025-12-08 15:56:02 +02:00
Yiorgis Gozadinos
16ba3a9962
Remove create_chunks_for_document from repositories
2025-12-08 15:56:01 +02:00
Yiorgis Gozadinos
17c7147a49
Remove pre-processor, no longer needed
2025-12-08 15:56:01 +02:00
Yiorgis Gozadinos
69b1afa534
Remove _create_and_chunk & _update_and_rechunk, keep storage independent of chunking
2025-12-08 15:55:31 +02:00
Yiorgis Gozadinos
0ab43d62cd
Refactor create_document(), create_document_from_source() to use primitives
2025-12-08 15:55:30 +02:00
Yiorgis Gozadinos
e299aed788
contextualize() and embed_chunks() utilities in embeddings
2025-12-08 15:55:30 +02:00
Yiorgis Gozadinos
4fbb4609b6
HaikuRag.chunk() utility method
2025-12-08 15:55:30 +02:00
Yiorgis Gozadinos
ace915473c
HaikuRag.convert() utility method
2025-12-08 15:55:30 +02:00
Yiorgis Gozadinos
ab57c55eaa
Drop ChunkWithMetadata, use good old Chunk
2025-12-08 15:55:30 +02:00
Yiorgis Gozadinos
33e6a36290
Make document import/update handling consistent for DoclingDocument
2025-12-08 15:55:29 +02:00
Yiorgis Gozadinos
60ab864fc1
Add docling_document_json to update_document_fields() and
2025-12-08 15:55:29 +02:00
Yiorgis Gozadinos
8e30f67266
Introduce import_document(), remove chunks from create_document()
2025-12-08 15:55:29 +02:00
Yiorgis Gozadinos
de14bb9f90
Avoid citation meta inside LLM context
2025-12-08 15:55:28 +02:00
Yiorgis Gozadinos
a57203065a
Fix test
2025-12-08 15:55:28 +02:00
Yiorgis Gozadinos
84f25f0627
Visual grounding in ag-ui-example
2025-12-08 15:55:28 +02:00
Yiorgis Gozadinos
6e31ea38a8
Remove bounding boxes from citations, pass sources meta info from ResearchContext
2025-12-08 15:55:28 +02:00
Yiorgis Gozadinos
75a086bae0
Visualize chunk command, format citations to show chunk id
2025-12-08 15:55:28 +02:00
Yiorgis Gozadinos
910d382748
Introduce visualize_chunk(), refactor inspector to use it
2025-12-08 15:55:27 +02:00
Yiorgis Gozadinos
bfdaa4652a
Configure logging regardsless of the env
2025-12-08 15:55:27 +02:00
Yiorgis Gozadinos
9382a6b132
expose citations in ag-ui-example
2025-12-08 15:55:27 +02:00
Yiorgis Gozadinos
d808c6c425
Simplify citations in qa & graph agents
2025-12-08 15:55:27 +02:00
Yiorgis Gozadinos
16a97cc140
Properly expand context in the case of docling documents and merge chunks and meta
2025-12-08 15:55:26 +02:00
Yiorgis Gozadinos
ac439b6c69
Store raw text (do not contextualize) in chunk.text. Embed with contextualized text.
2025-12-08 15:55:26 +02:00
Yiorgis Gozadinos
0d8691342a
Show bounding boxes for chunks in inspector
2025-12-08 15:55:26 +02:00
Yiorgis Gozadinos
6ab990b2ed
Always generate page images, default in docling-serve
2025-12-08 15:54:57 +02:00
Yiorgis Gozadinos
c69b934eac
Incorporate additional docling meta in search(), expand_context() and relevant qa/research agents
2025-12-08 15:54:56 +02:00
Yiorgis Gozadinos
fe4ac530be
DB migration
2025-12-08 15:54:56 +02:00
Yiorgis Gozadinos
d267b2c433
LRU cache for DoclingDocument
2025-12-08 15:54:31 +02:00
Yiorgis Gozadinos
a5d8af07e9
Resolve DoclingDocument
2025-12-08 15:54:31 +02:00
Yiorgis Gozadinos
84aa46a10b
Update rebuild to handle DoclingDocument json
2025-12-08 15:54:31 +02:00
Yiorgis Gozadinos
ad4e79ab5d
Store DoclingDocument json
2025-12-08 15:54:31 +02:00
Yiorgis Gozadinos
343272d6fb
Chunkers return chunks with metadata, i.e. (refs, labels, headings, page_numbers)
2025-12-08 15:54:31 +02:00
Yiorgis Gozadinos
44e8d7b340
Update Document & Chunk schemas to include DoclingDocument references.
2025-12-08 15:54:30 +02:00
Yiorgis Gozadinos
80447c9024
Make download_models() a HaikuRAG client method. Update app to show progress of what happens.
2025-12-04 16:30:26 +02:00
Yiorgis Gozadinos
78f1d0cef5
No warnings, just info on db upgrade
2025-12-03 08:51:25 +02:00
Yiorgis Gozadinos
33b254ae3d
Always upgrade database on access. Explicitly create db by running init or HaikuRAG(path, create=True)
2025-12-02 16:49:55 +02:00
Yiorgis Gozadinos
3525fae625
Use EmbeddingModelConfig similar to ModelConfig for embeddings
2025-12-02 11:55:31 +02:00
Yiorgis Gozadinos
f07590bd5b
vb
2025-12-01 17:58:47 +02:00
Yiorgis Gozadinos
e548a51ff7
Optimized rebuild --embed-only to use batch updates via LanceDB merge_insert instead of individual chunk updates
2025-12-01 17:53:27 +02:00
Yiorgis Gozadinos
3e18a5f2ff
Update project description
2025-11-29 11:18:53 +02:00
Yiorgis Gozadinos
10aa2a7cf2
vb
2025-11-28 10:15:59 +02:00
Yiorgis Gozadinos
88a4edb537
--embed-only, --rechunk options in rebuild command
2025-11-28 10:11:52 +02:00
Yiorgis Gozadinos
8a0cc04181
Add opencv-python-headless to docling optional dependency, required for TableStructureModel which uses TableFormer for table detection.
2025-11-27 16:22:37 +02:00
Yiorgis Gozadinos
eee29578ba
vb
2025-11-27 15:24:39 +02:00
Yiorgis Gozadinos
b4602b0cab
use OcrAutoOptions for automatic OCR engine selection.
2025-11-27 15:23:17 +02:00
Yiorgis Gozadinos
51104ddb5a
Use httpx for docling-serve
2025-11-27 15:06:13 +02:00
Yiorgis Gozadinos
3959ce39f7
vb
2025-11-27 13:06:04 +02:00
Yiorgis Gozadinos
ed53ec461a
Convert prefetch_models() to async
2025-11-26 18:08:48 +02:00
Yiorgis Gozadinos
e40c352026
Switch from requests to httpx.AsyncClient for docling-serve
2025-11-26 18:00:29 +02:00
Yiorgis Gozadinos
14bdb7cb27
async versions for convert_file() convert_text()
2025-11-26 17:46:25 +02:00
Yiorgis Gozadinos
ca2445d869
vb
2025-11-26 10:33:46 +02:00
Yiorgis Gozadinos
0303e6a8cd
Support LMStudio
2025-11-26 10:31:40 +02:00
Yiorgis Gozadinos
283c44867d
Generate the config from the AppConfig model directly, add tests
2025-11-26 10:17:22 +02:00
Yiorgis Gozadinos
c22a5bb0ce
vb
2025-11-25 13:44:48 +02:00
Yiorgis Gozadinos
e83978f88b
Revert embeddings to use flat config
2025-11-25 12:51:58 +02:00
Yiorgis Gozadinos
ba4dddeeaf
Make thinking false by default
2025-11-25 12:35:38 +02:00
Yiorgis Gozadinos
a9701616c8
Rename model to name under model
2025-11-25 12:28:43 +02:00
Yiorgis Gozadinos
1bb7b6d5bd
Add support for per-model configuration settings including thinking, temperature and max_tokens
2025-11-25 12:14:24 +02:00
Yiorgis Gozadinos
13d3a366e8
Enhanced ActivitySnapshot events with richer structured data
2025-11-24 16:18:27 +02:00
Yiorgis Gozadinos
03c1f356b2
Update qa agent search limit to 5
2025-11-24 15:01:31 +02:00
Yiorgis Gozadinos
0b92eae75f
Docs
2025-11-22 17:46:21 +02:00
Yiorgis Gozadinos
2c6efc55c6
Make search a screen
2025-11-21 17:06:30 +02:00
Yiorgis Gozadinos
0d126d894e
Keep documents when moving from main view to search and back
2025-11-21 16:48:35 +02:00
Yiorgis Gozadinos
59eba31267
Handle missing textual dependencies
2025-11-21 16:48:35 +02:00
Yiorgis Gozadinos
8450c07085
Fix markdown
2025-11-21 16:48:35 +02:00
Yiorgis Gozadinos
82831c8941
Search in inspector
2025-11-21 16:48:34 +02:00
Yiorgis Gozadinos
2c1c530f3a
Fix tabbing
2025-11-21 16:48:34 +02:00
Yiorgis Gozadinos
89a2668c25
inspector command that runs a textual app to browse documents and chunks in the database
2025-11-21 16:48:34 +02:00
Yiorgis Gozadinos
ada2b66d5b
vb
2025-11-21 16:38:52 +02:00
Yiorgis Gozadinos
8c02e865c7
Change default embedding model to qwen3-embedding:4b
2025-11-21 16:30:41 +02:00
Yiorgis Gozadinos
6d8ab2575d
Clarify refine_factor, set default to 30
2025-11-21 15:53:24 +02:00
Yiorgis Gozadinos
2f54ee7d59
Replace allow_create with cleaner read_only. Elaborate on the test_info tests
2025-11-21 14:37:33 +02:00
Yiorgis Gozadinos
fdcf9a1a12
Show index stats in cli info command
2025-11-21 13:54:04 +02:00
Yiorgis Gozadinos
6a2f33b464
Create vector index from CLI, use configurable refine_factor
2025-11-21 13:54:01 +02:00
Yiorgis Gozadinos
c45d507213
Allow custom config when building research/deep ask graphs. Closes #149
2025-11-21 12:37:41 +02:00
Yiorgis Gozadinos
5c4a32f1c8
Fix ACTIVITY_SNAPSHOT payloads. Closes #150
2025-11-21 12:27:54 +02:00
Yiorgis Gozadinos
09900f8e8b
vb
2025-11-19 13:08:37 +02:00
Yiorgis Gozadinos
30a0bbcff2
Accept both single chunks and lists for batch insertion in ChunkRepository.create()
2025-11-19 12:35:02 +02:00
Yiorgis Gozadinos
5089ea4203
New update_document_fields() method for partial document updates
2025-11-19 11:37:36 +02:00
Yiorgis Gozadinos
0ea5f1e10e
Update dependencies
2025-11-19 10:24:37 +02:00
Yiorgis Gozadinos
826986708d
vb
2025-11-18 15:52:52 +02:00
Yiorgis Gozadinos
0bb3f4301c
Additional options for document conversion with docling
2025-11-18 15:44:04 +02:00
Yiorgis Gozadinos
2dca72d670
Revert "Handle client-side tool calls in graph ag-ui emitter"
...
This reverts commit b1d19651d5
2025-11-18 15:19:28 +02:00
Yiorgis Gozadinos
cc09ce87c2
Increase reranking candidate retrieval multiplier from 3x to 10x for improved result quality
2025-11-18 14:35:57 +02:00
Yiorgis Gozadinos
b1d19651d5
Handle client-side tool calls in graph ag-ui emitter
2025-11-18 14:10:21 +02:00
Yiorgis Gozadinos
efbd5b92f7
vb
2025-11-17 16:30:39 +02:00
Yiorgis Gozadinos
91de82ec14
docling-serve chunker implementation. Lacks tests
2025-11-17 13:05:06 +02:00
Yiorgis Gozadinos
75cbdb47a8
Add chunker configuration options
...
- Add `chunker_type` config field to choose between "hybrid" (default) and "hierarchical" chunking strategies
- Add `chunking_merge_peers` config field for HybridChunker (default: true)
- Add `chunking_use_markdown_tables` config field to control table serialization format (default: false, matching docling's default)
2025-11-17 13:05:06 +02:00
Yiorgis Gozadinos
de5bb117fe
Chunker abstraction
2025-11-17 13:05:06 +02:00
Yiorgis Gozadinos
541552215e
Switch to using HuggingFace tokenizers
2025-11-17 13:05:05 +02:00
Yiorgis Gozadinos
fe7d0c5c24
Implement docling-serve converter
2025-11-17 13:05:05 +02:00
Yiorgis Gozadinos
92def1c6a5
Rebase after haiku.rag-slim migration
2025-11-17 13:05:05 +02:00
Yiorgis Gozadinos
eb2b65135a
Lazy load docling & friends
2025-11-17 13:05:05 +02:00
Yiorgis Gozadinos
7ba1c2376e
Introduce converters for supporting more than local docling document conversion. Transform existing FileReader and utils to "docling-local" converter
2025-11-17 13:05:05 +02:00
Yiorgis Gozadinos
636d5261e6
vb
2025-11-14 22:29:37 +02:00
Yiorgis Gozadinos
c8999a89b8
Update and lock pydantic-ai-slim
2025-11-14 11:26:09 +02:00
Yiorgis Gozadinos
6eb0d59a28
vb
2025-11-13 15:00:57 +02:00
Yiorgis Gozadinos
4599dd1481
Drop disable_auto_create flag, simply create only on write operations
2025-11-13 13:53:11 +02:00
Yiorgis Gozadinos
8e802d4a91
Default vacuum_retention_seconds increased from 60 seconds to 86400 seconds (1 day) for better version retention in typical workflows
2025-11-13 13:31:53 +02:00
Yiorgis Gozadinos
2c43f034de
Refactor ag-ui-research. Drop human-in-the-loop, use MemoryObjectSendStream to merge the graph and agent streams together
2025-11-13 13:22:53 +02:00
Yiorgis Gozadinos
4a623934b2
Log additional ag-ui events in the cli renderer
2025-11-13 13:22:52 +02:00
Yiorgis Gozadinos
0c8d2838b2
Better typing
2025-11-13 13:22:52 +02:00
Yiorgis Gozadinos
68d1127e46
Compute and emit state deltas instead of full state snapshots by default
2025-11-13 13:22:52 +02:00
Yiorgis Gozadinos
657245f686
Simplify InsightRecord, GapRecord
2025-11-13 13:22:52 +02:00
Yiorgis Gozadinos
2060898018
Use a node factor for common nodes
2025-11-13 13:22:52 +02:00
Yiorgis Gozadinos
032625b0bb
Refactor everything graph-related under the graph module
2025-11-13 13:22:03 +02:00
Yiorgis Gozadinos
8f0597e89e
Update deep ask graph
2025-11-13 13:22:03 +02:00
Yiorgis Gozadinos
d5b6ad544d
Bring back verbose
2025-11-13 13:22:02 +02:00
Yiorgis Gozadinos
60ef8fe04c
AGUI starlette server
2025-11-13 13:22:02 +02:00
Yiorgis Gozadinos
2c1f79614a
Use StateSnapshot updates
2025-11-13 13:22:02 +02:00
Yiorgis Gozadinos
663a7fac9a
Replace console/stream with AG-UI event protocol
2025-11-13 13:22:01 +02:00
Yiorgis Gozadinos
da77e0f2f9
Convert ResearchState from dataclass to Pydantic model
2025-11-13 13:22:01 +02:00
Yiorgis Gozadinos
765ebb698f
AG-UI Event-emitter for graphs. Compute state-deltas from from BaseModel pydantic state models
2025-11-13 13:22:01 +02:00
Yiorgis Gozadinos
908296e155
Do not create unecessary DoclingDocument when chunks are provided. Rename private _create_with_docling & _update_with_docling to _create_and_chunk & _update_and_rechunk
2025-11-13 13:16:15 +02:00
Yiorgis Gozadinos
942ad2818f
Filereader error messages now include both original exception details and file path for easier debugging
2025-11-10 18:26:25 +02:00
Yiorgis Gozadinos
34f50f37a0
vb
2025-11-07 14:44:58 +02:00
Yiorgis Gozadinos
b6650d0486
Introduce delete_orphans in monitor (default=false) to delete db entries that do not resolve in the filesystem
2025-11-07 14:40:13 +02:00
Yiorgis Gozadinos
31713e5fb6
Properly skip (and log as such) documents without changing updated_at when the md5 hash is unchanged
2025-11-07 14:00:05 +02:00
Yiorgis Gozadinos
9ee64a33e0
Remove A2A from haiku.rag, turn it into an example
2025-11-07 13:31:51 +02:00
Yiorgis Gozadinos
0b696d9629
Remove deprecated use of env variables for configuration
2025-11-07 10:54:32 +02:00
Yiorgis Gozadinos
b51283c647
Use haiku.rag.yaml as the default config filename everywhere
2025-11-07 10:42:57 +02:00
Yiorgis Gozadinos
3c557d5bfe
Make --config a global CLI parameter, add it as a param to HaikuRAGApp
2025-11-07 10:31:55 +02:00
Yiorgis Gozadinos
0e3f2ebd67
vb
2025-11-06 16:13:53 +02:00
Yiorgis Gozadinos
ce6dcb8b64
Pin critical packages
2025-11-06 16:09:31 +02:00
Yiorgis Gozadinos
782fbafb3d
Make graph settings part of config for both deep ask and research graphs
2025-11-06 14:31:22 +02:00
Yiorgis Gozadinos
17d6a1dfee
Bring back max_concurrency by means of asyncio.Semaphore
2025-11-06 11:02:32 +02:00
Yiorgis Gozadinos
6858760bca
Remove max_concurrency, not supported (yet) when we .map() in the beta Graph API
2025-11-06 11:02:32 +02:00
Yiorgis Gozadinos
38861da549
Go back and separate common patters in graph_common
2025-11-06 11:02:32 +02:00
Yiorgis Gozadinos
9075dac23a
Remove graph common and move its content to deep QA, research.
2025-11-06 11:02:32 +02:00
Yiorgis Gozadinos
bc71ad6fb0
Refactor research, deep ask to work with new beta pyndtic AI graph
2025-11-06 11:02:32 +02:00
Yiorgis Gozadinos
b8bc6d9b68
Let ruff know about our package structure
2025-11-05 17:47:27 +02:00
Yiorgis Gozadinos
21bb7b5aab
Version bump & Changelog
2025-11-05 13:16:03 +02:00
Yiorgis Gozadinos
74d9c60ba0
Update docling
2025-11-05 13:11:30 +02:00
Yiorgis Gozadinos
ba8e503cb1
Depend on pydantic-ai-slim with extras
2025-11-05 11:37:06 +02:00
Yiorgis Gozadinos
f06dac7102
Update READMEs
2025-11-05 10:47:00 +02:00
Yiorgis Gozadinos
be2d8ea8f4
Catch import errors seperately to raise errors about missing dependencies.
2025-11-05 10:31:07 +02:00
Yiorgis Gozadinos
5cd951a5f3
When checking for updates, use haiku.rag-slim as the package, since it is always installed
2025-11-05 10:30:59 +02:00
Yiorgis Gozadinos
b02514dc82
Make docling optional, depend on docling-core
2025-11-04 18:15:04 +02:00
Yiorgis Gozadinos
2f9c907031
Restructure into uv workspace to support minimal and full installations
2025-11-04 17:59:12 +02:00
