Commit graph

594 commits

Author SHA1 Message Date
Yiorgis Gozadinos
ad1ee52a27
Move monitor config in its own section 2025-10-29 11:23:56 +02:00
Yiorgis Gozadinos
987ddab683
Tests for filtering with pathspec 2025-10-29 10:39:00 +02:00
Yiorgis Gozadinos
cabb7748f5
Filtering based on pathspec 2025-10-29 10:27:16 +02:00
Yiorgis Gozadinos
a15f1d8cf5
Use pathspec for monitor filtering 2025-10-29 10:13:53 +02:00
Yiorgis Gozadinos
e11a5128f9
Pass config to embedders 2025-10-28 09:15:08 +02:00
Yiorgis Gozadinos
4cc25407ef
Properly skip a2a tests if fasta2a not installed 2025-10-27 11:28:35 +02:00
Yiorgis Gozadinos
469d993542
Remove provider API keys from config, rely on environment variables for those 2025-10-23 13:52:57 +03:00
Yiorgis Gozadinos
618b3ffd5f
Refactor config from flat to nested Pydantic models 2025-10-23 13:30:21 +03:00
Yiorgis Gozadinos
082811a846
tests for yaml configuration 2025-10-23 11:58:47 +03:00
Yiorgis Gozadinos
1cab0339ad
add-src in cli & create_document_from_source will recursively add directories 2025-10-16 09:55:50 +03:00
Yiorgis Gozadinos
7b371b96ed
Use TestModel from pydantic AI where appropriate 2025-10-14 13:27:20 +03:00
Yiorgis Gozadinos
c0691bc237
Remove list documents, make separate artifacts for q/a, search & get document 2025-10-13 18:01:24 +03:00
Yiorgis Gozadinos
1f2552bd6a
Make serve cli command run monitor, mcp or a2a independently 2025-10-13 18:01:24 +03:00
Yiorgis Gozadinos
6ea7d73eed
Remove deep q/a from a2a agent 2025-10-13 18:01:23 +03:00
Yiorgis Gozadinos
352637faa6
Refactor into own module 2025-10-13 18:01:23 +03:00
Yiorgis Gozadinos
1a88b0355a
deep q/a skill and skill selection 2025-10-13 18:01:22 +03:00
Yiorgis Gozadinos
5c0f0e5830
Add skills 2025-10-13 18:01:22 +03:00
Yiorgis Gozadinos
48cdb8a1db
Implement an LRUMemoryStorage for context 2025-10-13 18:01:22 +03:00
Yiorgis Gozadinos
e70fa737b7
Tests for a2a 2025-10-13 18:01:21 +03:00
Yiorgis Gozadinos
43dac018c0
Test mcp tools 2025-10-08 09:20:26 +03:00
Yiorgis Gozadinos
5e4928ccad
Make sure vacuum task has been complete before exiting the client 2025-10-07 12:27:55 +03:00
Yiorgis Gozadinos
094f0e0d44
Centralize vacuuming 2025-10-07 12:05:25 +03:00
Yiorgis Gozadinos
c6eaf9d5cd
Add vacuum retention setting 2025-10-07 10:47:28 +03:00
Yiorgis Gozadinos
a6275fcf25
Rename nodes to make it clear they are part of Q/A 2025-09-30 16:59:36 +03:00
Yiorgis Gozadinos
49eac86259
Add verbose flag to ask 2025-09-30 16:59:36 +03:00
Yiorgis Gozadinos
2dfb1882dd
Deep QA multi-agent 2025-09-30 16:59:36 +03:00
Yiorgis Gozadinos
42c54a8d25
Shuffle reusable nodes under graph/ 2025-09-30 16:59:36 +03:00
Yiorgis Gozadinos
05c382fcb5
When rebuilding if a document does not exist anymore, re-embed the chunks instead of failing 2025-09-30 16:57:02 +03:00
Yiorgis Gozadinos
57e86d00fd
Refactor evaluations so that we can perform with multiple datasets..
Introduce Wix dataset.
2025-09-30 11:31:50 +03:00
Yiorgis Gozadinos
19ad1b9d7d
improve output 2025-09-30 11:30:35 +03:00
Yiorgis Gozadinos
87f28ed96b
Update docs 2025-09-30 11:30:35 +03:00
Yiorgis Gozadinos
7cc4e13b57
Make benchmarks use pydantic evals 2025-09-30 11:30:34 +03:00
Yiorgis Gozadinos
14168db795
Fix embedding client test 2025-09-26 13:52:06 +03:00
Yiorgis Gozadinos
184fd12e42
Do not run db upgrades when creating a new db 2025-09-25 10:52:11 +03:00
Yiorgis Gozadinos
064b823020
Break Evaluation node into AnalyzeInsights & DecisionNode. Use a list of updateable insights & gaps 2025-09-24 16:26:48 +03:00
Yiorgis Gozadinos
84e94f8257
Wire gaps that were forgotten 2025-09-24 14:03:53 +03:00
Yiorgis Gozadinos
cf9d6a116f
Do not duplicate ResearchContext in ResearchState 2025-09-24 13:43:07 +03:00
Yiorgis Gozadinos
086eb56d34
Stream research 2025-09-24 11:27:05 +03:00
Yiorgis Gozadinos
8f3055d37d
Parse non-strings 2025-09-23 16:42:40 +03:00
Yiorgis Gozadinos
45d0b47a02
Meta in CLI add-src, add 2025-09-23 16:39:27 +03:00
Yiorgis Gozadinos
5ea5de274c
Make styles consistent 2025-09-23 12:27:01 +03:00
Yiorgis Gozadinos
52ea1ebb70
info command in CLI 2025-09-23 12:20:25 +03:00
Yiorgis Gozadinos
2c93f4f823
Preserve title if it exists when expanding context 2025-09-22 13:45:59 +03:00
Yiorgis Gozadinos
20993b48a7
Fix test after rebase 2025-09-22 12:53:52 +03:00
Yiorgis Gozadinos
73aff1695f
Update LLM prompts/tools to reflect we will pass title (if available) or URI 2025-09-22 12:45:32 +03:00
Yiorgis Gozadinos
4077e3ecb2
Handle title in client & CLI 2025-09-22 12:45:32 +03:00
Yiorgis Gozadinos
dce60fa461
Remove SSE transport, fix cli tests 2025-09-20 10:58:28 +03:00
Yiorgis Gozadinos
cba8ad7696
Research agent as a pydantic graph 2025-09-19 14:39:39 +03:00
Yiorgis Gozadinos
a215a64686
Refactor to give its research subagent full responsibility 2025-09-19 12:10:00 +03:00
Yiorgis Gozadinos
5bfa720864
Remove openai models from research tests 2025-09-19 12:09:59 +03:00
Yiorgis Gozadinos
d1e2470d94
Upgrades are back. Make chunk order a table column rather than a metadata property 2025-09-18 13:42:07 +03:00
Yiorgis Gozadinos
03289bb644
Fix MCP test 2025-09-18 12:54:36 +03:00
Yiorgis Gozadinos
a807f3ea36
Simplify structured output for research agents 2025-09-17 18:31:55 +03:00
Yiorgis Gozadinos
05e08c48e4
Simplify SearchAgent. Omit search results, but keep QA results. 2025-09-17 12:37:16 +03:00
Yiorgis Gozadinos
70544b5112
Research agent tests 2025-09-17 08:54:54 +03:00
Yiorgis Gozadinos
98ca5c4746
Simplify orchestration & subagents 2025-09-17 08:54:54 +03:00
Yiorgis Gozadinos
c029350d79
Make search agent more like a qa agent 2025-09-17 08:51:44 +03:00
Yiorgis Gozadinos
53905c2d69
Simplify search agent 2025-09-17 08:51:44 +03:00
Yiorgis Gozadinos
fe8e48df38
Config research provider/model, CLI 2025-09-17 08:51:44 +03:00
Yiorgis Gozadinos
86b44c47f9
Adapt clarification agent, basic orchestrator 2025-09-17 08:51:43 +03:00
Yiorgis Gozadinos
b03ee08f18
Search agent 2025-09-17 08:51:43 +03:00
Yiorgis Gozadinos
4a07444030
Rollback version if a failure occurs when creating/updating documents 2025-09-15 11:54:50 +03:00
Yiorgis Gozadinos
765a6e647d
Allow for custom pre-processing after markdown is generated and before it is chunked. 2025-09-15 11:02:34 +03:00
Yiorgis Gozadinos
c43ae62018
Fix client test on context expansion to not depend on .env settings 2025-09-12 12:38:22 +03:00
Yiorgis Gozadinos
d90dfe258c
Use logfire when running evaluations 2025-09-12 09:45:06 +03:00
Yiorgis Gozadinos
2207b7d1ed
vacuum() in store, client, cli. Removes old versions and reduces disk usage 2025-09-10 11:24:51 +03:00
Yiorgis Gozadinos
8dfb8638be
Fix SSE server setup 2025-09-08 15:31:56 +03:00
Yiorgis Gozadinos
b99f88f104
Support batch embedding 2025-09-05 10:48:31 +03:00
Yiorgis Gozadinos
bc28e60cf7
Support vLLM for QA agents 2025-09-04 15:20:37 +03:00
Yiorgis Gozadinos
f2847d0524
Reranking support for vLLM, documentation update 2025-09-04 13:13:10 +03:00
Yiorgis Gozadinos
16f7ba9d99
VLLM embedding 2025-09-04 10:44:07 +03:00
Yiorgis Gozadinos
3c235e8b6c
Suppress logging from deps 2025-09-03 11:42:08 +03:00
Yiorgis Gozadinos
b73949c784
update benchmarks 2025-09-02 15:10:57 +03:00
Yiorgis Gozadinos
9e5f11060d
Allow connecting to lancedb cloud, s3, gc, az etc 2025-09-02 12:49:17 +03:00
Yiorgis Gozadinos
fe850e778d
Remove Ollama reranker, this was a bad idea 2025-09-02 11:42:31 +03:00
Yiorgis Gozadinos
27b16bdf8d
Update basic benchmarks 2025-09-02 10:51:20 +03:00
Yiorgis Gozadinos
b05e5f5408
Set/get haiku version to db 2025-09-01 17:41:41 +03:00
Yiorgis Gozadinos
a5e087aa53
Fix scores depending on search type 2025-09-01 14:56:15 +03:00
Yiorgis Gozadinos
628bdc8151
Cleanup 2025-09-01 14:56:15 +03:00
Yiorgis Gozadinos
0592a7d122
Adapt search to lancedb 2025-09-01 14:56:14 +03:00
Yiorgis Gozadinos
b83596ff07
Basic moving to lancedb. 2025-09-01 14:56:14 +03:00
Yiorgis Gozadinos
1f61d78165
Bring back QA_MODEL and keep compatibility with previous versions 2025-08-17 15:27:03 +02:00
Yiorgis Gozadinos
d6d79d5e94
Allow any provider/model supported by pydantic AI 2025-08-17 14:55:50 +02:00
Yiorgis Gozadinos
4ac658e9dd
Convert LLM Judge 2025-08-17 14:36:51 +02:00
Yiorgis Gozadinos
79841f32ca
Do not use mocks in embeddings test 2025-08-15 22:01:57 +02:00
Yiorgis Gozadinos
26a3715902
Refactor Ollama reranker around pydantic AI 2025-08-15 21:26:16 +02:00
Yiorgis Gozadinos
390397570a
Migrate QA agent to Pydantic AI. 2025-08-15 21:13:17 +02:00
Yiorgis Gozadinos
bf8a4f22be
Make radius a parameter we can pass to expand_context() 2025-08-12 16:53:45 +02:00
Yiorgis Gozadinos
99f7401817
Merge expanded context if possible 2025-08-12 15:02:07 +02:00
Yiorgis Gozadinos
eb13426aae
expand_context() by using the adjacent chunks 2025-08-12 14:14:18 +02:00
Yiorgis Gozadinos
2f42b1a3ac
Get adjacent chunks method 2025-08-12 13:24:04 +02:00
Yiorgis Gozadinos
842e4a882d
Make citations optional, through cite parameter 2025-08-12 13:19:11 +02:00
Yiorgis Gozadinos
5103e6c2d7
Add Ollama reranker 2025-08-10 20:50:17 +02:00
Yiorgis Gozadinos
8d98eed2e2
Make mxbai reranker optional. Only use a reranker if mxbai or cohere are installed 2025-08-10 19:49:14 +02:00
Yiorgis Gozadinos
ad4b655a31
Operate on DoclingDocument instead of Document 2025-08-08 17:24:34 +02:00
Yiorgis Gozadinos
6a3c85e6ae
text_to_docling_document() util 2025-08-08 15:38:57 +02:00
Yiorgis Gozadinos
b92f25d6fb
Make FileReader return a DoclingDocument instead of a md string 2025-08-08 14:45:24 +02:00
Yiorgis Gozadinos
b428cb4c5d
Forgotten test for FileReader 2025-08-01 16:41:24 +02:00
Yiorgis Gozadinos
91f6122ef7
Basic chunking using docling 2025-08-01 16:27:44 +02:00
Yiorgis Gozadinos
74eea9284a
create_document now optionally accepts pre-computed chunks 2025-07-26 10:42:27 +03:00
Yiorgis Gozadinos
dfe23482da
Tune parameters 2025-07-20 22:45:24 +03:00
Yiorgis Gozadinos
c1d5334804
Update benchmarks with reranking 2025-07-19 19:02:47 +03:00
Yiorgis Gozadinos
2343ab7751
Return tuple (Chunk, score,) in rerank() 2025-07-19 12:16:17 +03:00
Yiorgis Gozadinos
f25416b0ff
Properly test cohere reranker 2025-07-18 20:55:28 +03:00
Yiorgis Gozadinos
044e1b0a06
mixedbread reranker as default 2025-07-18 20:29:24 +03:00
Yiorgis Gozadinos
859860318a
Cohere reranker 2025-07-17 13:17:16 +03:00
Yiorgis Gozadinos
0bf5606117
Store haiku.rag version, enabling upgrades 2025-07-16 18:21:41 +03:00
Yiorgis Gozadinos
a2e0f2cefd
Remove Claude nonsense, use context manager 2025-07-12 23:00:49 +03:00
Yiorgis Gozadinos
62a519b094
Prevent rebuild from throwing and recreate the embeddings table 2025-07-12 21:21:14 +03:00
Yiorgis Gozadinos
4444ec50da
Check if settings are compatible when loading a db 2025-07-12 20:56:49 +03:00
Yiorgis Gozadinos
a59b38158d
Settings table & repo, tests 2025-07-12 20:21:34 +03:00
Yiorgis Gozadinos
42e62f92a1
Better formatting of benchmark script 2025-07-09 10:27:32 +03:00
Yiorgis Gozadinos
9d72a158a6
Document benchmarks 2025-07-08 19:01:54 +03:00
Yiorgis Gozadinos
bea96f1b1b
Better prompts 2025-07-04 13:29:51 +03:00
Yiorgis Gozadinos
1705e7a21c
Honour QA_MODEL for anthropic and OpenAI 2025-07-04 12:14:13 +03:00
Yiorgis Gozadinos
e66c160055
Support for anthropic in Question/Answering 2025-07-04 11:55:03 +03:00
Yiorgis Gozadinos
85443ee963
Turn rebuild into a generator, track progress in command line 2025-07-02 11:10:31 +03:00
Yiorgis Gozadinos
11d9f2701e
Rebuild database in client & cli 2025-07-02 10:55:06 +03:00
Yiorgis Gozadinos
e5d8396d1b
Merge pull request #5 from ggozad/feat/qa
Question/Answering using LLMs
2025-06-28 09:41:55 +03:00
Yiorgis Gozadinos
85b106c461
OpenAI Question/Answer agent 2025-06-28 09:11:39 +03:00
Yiorgis Gozadinos
f3ed88b970
Include document uri and meta in search results 2025-06-28 09:10:23 +03:00
Yiorgis Gozadinos
ca45f8e5d4
Fix db path 2025-06-25 19:01:57 +03:00
Yiorgis Gozadinos
2855dd7c13
Use LLM-as-a-judge to test QA 2025-06-25 18:18:28 +03:00
Yiorgis Gozadinos
26900d9687
Run simple benchmarks 2025-06-25 18:18:28 +03:00
Yiorgis Gozadinos
c4c670bc40
Use proper mocks instead of setattr to satisfy typing. Fix minor typos 2025-06-25 17:50:19 +03:00
Russ Ferriday
94ff11ad25 feat(tests): Add tests for app and cli modules 2025-06-24 23:00:18 +01:00
Yiorgis Gozadinos
3c98f246b5
Merge pull request #2 from ggozad/feat/monitor
File monitoring support
2025-06-20 10:30:40 +02:00
Yiorgis Gozadinos
27f65adda7
Test FileWatcher 2025-06-20 09:35:20 +02:00
Yiorgis Gozadinos
ee57bc0a82
Use path.as_uri() instead of path.resolve() 2025-06-20 09:27:01 +02:00
Yiorgis Gozadinos
943cf3c5e0
Rename EMBDEDDING->EMBEDDINGS in env vars 2025-06-19 17:44:42 +02:00
Yiorgis Gozadinos
d89b2f1306
Test openai voyageai embedders 2025-06-19 17:39:04 +02:00
Yiorgis Gozadinos
51723308ed
Introduce embedder factory 2025-06-18 08:26:11 +02:00
Yiorgis Gozadinos
1e0b6b18aa
Add a context manager to the client 2025-06-17 12:48:26 +02:00
Yiorgis Gozadinos
6321cfb7a8
Add search to client 2025-06-17 12:41:37 +02:00
Yiorgis Gozadinos
0b43c7dc7f
Handle files/url updates and store md5 hash in the meta 2025-06-17 12:22:00 +02:00
Yiorgis Gozadinos
11fd4d75ad
Handle files/url with create_document_from_source in client 2025-06-17 11:26:02 +02:00
Yiorgis Gozadinos
a91249c575
Basic client 2025-06-17 10:48:40 +02:00
Yiorgis Gozadinos
1ff17c2f50
Add uri to Documents 2025-06-17 10:41:44 +02:00
Yiorgis Gozadinos
0b5f89ebad
FTS and hybrid search using Reciprocal Rank Fusion 2025-06-16 19:28:45 +02:00
Yiorgis Gozadinos
892d1a378d
test search 2025-06-16 18:10:03 +02:00
Yiorgis Gozadinos
5f9fb12644
Use db repositories 2025-06-16 17:31:53 +02:00
Yiorgis Gozadinos
5d1a854934
Basic store for chunks and documents 2025-06-15 22:34:54 +02:00
Yiorgis Gozadinos
4219cddea4
Ollama embeddings 2025-06-15 21:27:20 +02:00
Yiorgis Gozadinos
6c948a4afd
Chucker & tests 2025-06-15 21:08:06 +02:00