Commit graph

234 commits

Author SHA1 Message Date
Yiorgis Gozadinos
45d0b47a02
Meta in CLI add-src, add 2025-09-23 16:39:27 +03:00
Yiorgis Gozadinos
0e1bf3e00d
CLI command to prefetch models (Ollama & Docling) 2025-09-23 16:30:56 +03:00
Yiorgis Gozadinos
5369bd5dda
Show db table versions 2025-09-23 12:44:05 +03:00
Yiorgis Gozadinos
c0de22c9cd
Show package versions 2025-09-23 12:34:32 +03:00
Yiorgis Gozadinos
5ea5de274c
Make styles consistent 2025-09-23 12:27:01 +03:00
Yiorgis Gozadinos
52ea1ebb70
info command in CLI 2025-09-23 12:20:25 +03:00
Yiorgis Gozadinos
b46765e197
vb 2025-09-22 13:59:03 +03:00
Yiorgis Gozadinos
2c93f4f823
Preserve title if it exists when expanding context 2025-09-22 13:45:59 +03:00
Yiorgis Gozadinos
73aff1695f
Update LLM prompts/tools to reflect we will pass title (if available) or URI 2025-09-22 12:45:32 +03:00
Yiorgis Gozadinos
2c1e81dc6e
Display title where appropriate in cli 2025-09-22 12:45:32 +03:00
Yiorgis Gozadinos
4077e3ecb2
Handle title in client & CLI 2025-09-22 12:45:32 +03:00
Yiorgis Gozadinos
312d29333c
DB migration and schema updates to introduce title in documents 2025-09-22 12:45:32 +03:00
Yiorgis Gozadinos
c11e6da60d
Merge pull request #68 from ggozad/feat/disable-autocreation
DISABLE_DB_AUTOCREATE option to prevent auto-creation of dbs
2025-09-22 12:43:51 +03:00
Yiorgis Gozadinos
6b7f574180
Config DISABLE_DB_AUTOCREATE option to not auto-create dbs 2025-09-22 10:09:07 +03:00
Yiorgis Gozadinos
dce60fa461
Remove SSE transport, fix cli tests 2025-09-20 10:58:28 +03:00
Yiorgis Gozadinos
f582fb1be0
Lazy import when using CLI to speed up startup 2025-09-20 10:50:01 +03:00
Yiorgis Gozadinos
52febd01f9
Do not suppress logs in development 2025-09-20 10:11:31 +03:00
Yiorgis Gozadinos
fbb43c06c7
Cleanup console logging 2025-09-19 17:25:04 +03:00
Yiorgis Gozadinos
3fc461e1b1
Set default concurrency to 1 2025-09-19 16:54:06 +03:00
Yiorgis Gozadinos
42fe2201af
Only configure logfire in cli when not in development 2025-09-19 15:47:35 +03:00
Yiorgis Gozadinos
db47eb45cb
Update prompts 2025-09-19 15:33:46 +03:00
Yiorgis Gozadinos
cba8ad7696
Research agent as a pydantic graph 2025-09-19 14:39:39 +03:00
Yiorgis Gozadinos
a215a64686
Refactor to give its research subagent full responsibility 2025-09-19 12:10:00 +03:00
Yiorgis Gozadinos
9c128c1d36
Make answered question handling the responsibility of the search agent 2025-09-19 12:10:00 +03:00
Yiorgis Gozadinos
ccaae7cca1
Simplify research verbose logging 2025-09-19 12:10:00 +03:00
Yiorgis Gozadinos
5f102c8475
Set research agents to retry 3 times 2025-09-19 12:10:00 +03:00
Yiorgis Gozadinos
ad7a9ed324
Provide register_tools to the base research agent 2025-09-19 12:09:59 +03:00
Yiorgis Gozadinos
3abe669681
Simplify presearch prompt 2025-09-19 12:09:59 +03:00
Yiorgis Gozadinos
c221a35547
Upgrade fts index for phrase queries 2025-09-19 12:06:12 +03:00
Yiorgis Gozadinos
cf8b87246b
Infer vector dimension 2025-09-18 14:01:38 +03:00
Yiorgis Gozadinos
d1e2470d94
Upgrades are back. Make chunk order a table column rather than a metadata property 2025-09-18 13:42:07 +03:00
Yiorgis Gozadinos
a807f3ea36
Simplify structured output for research agents 2025-09-17 18:31:55 +03:00
Yiorgis Gozadinos
714c6c627e
Presearch Survey agent 2025-09-17 17:26:48 +03:00
Yiorgis Gozadinos
02fb8d439a
Fix typo 2025-09-17 14:51:18 +03:00
Yiorgis Gozadinos
ecad4c28b4
Use ToolOutput in order to specify 3 retries 2025-09-17 13:06:53 +03:00
Yiorgis Gozadinos
3e30059a42
Change default config to use gpt-oss for QA 2025-09-17 12:39:58 +03:00
Yiorgis Gozadinos
05e08c48e4
Simplify SearchAgent. Omit search results, but keep QA results. 2025-09-17 12:37:16 +03:00
Yiorgis Gozadinos
670e4e2254
Tweaks to research multi-agent 2025-09-17 08:54:54 +03:00
Yiorgis Gozadinos
98ca5c4746
Simplify orchestration & subagents 2025-09-17 08:54:54 +03:00
Yiorgis Gozadinos
c029350d79
Make search agent more like a qa agent 2025-09-17 08:51:44 +03:00
Yiorgis Gozadinos
9dc2e05f3f
Verbose logging 2025-09-17 08:51:44 +03:00
Yiorgis Gozadinos
53905c2d69
Simplify search agent 2025-09-17 08:51:44 +03:00
Yiorgis Gozadinos
4df5d31b18
Fix config for research 2025-09-17 08:51:44 +03:00
Yiorgis Gozadinos
fe8e48df38
Config research provider/model, CLI 2025-09-17 08:51:44 +03:00
Yiorgis Gozadinos
86b44c47f9
Adapt clarification agent, basic orchestrator 2025-09-17 08:51:43 +03:00
Yiorgis Gozadinos
49d4080240
Synthesis agent 2025-09-17 08:51:43 +03:00
Yiorgis Gozadinos
210275c6d2
Analysis and clarification agents 2025-09-17 08:51:43 +03:00
Yiorgis Gozadinos
b03ee08f18
Search agent 2025-09-17 08:51:43 +03:00
Yiorgis Gozadinos
b568293661
Base research agent & dependencies for research multi-agent 2025-09-17 08:51:43 +03:00
Yiorgis Gozadinos
b4b6c3f1ad
When MARKDOWN_PREPROCESSOR fails to load raise the exception. 2025-09-16 12:32:09 +03:00
Yiorgis Gozadinos
4a07444030
Rollback version if a failure occurs when creating/updating documents 2025-09-15 11:54:50 +03:00
Yiorgis Gozadinos
765a6e647d
Allow for custom pre-processing after markdown is generated and before it is chunked. 2025-09-15 11:02:34 +03:00
Yiorgis Gozadinos
ddaf1cb4d7
Suppress warnings in cli 2025-09-10 13:36:12 +03:00
Yiorgis Gozadinos
2207b7d1ed
vacuum() in store, client, cli. Removes old versions and reduces disk usage 2025-09-10 11:24:51 +03:00
Yiorgis Gozadinos
0a279aa12a
Fix missing vllm embeddings from embedder factory 2025-09-09 10:58:41 +03:00
Yiorgis Gozadinos
8dfb8638be
Fix SSE server setup 2025-09-08 15:31:56 +03:00
Yiorgis Gozadinos
4ad732a6f3
Fix migration script vector deserialization 2025-09-05 17:59:19 +03:00
Yiorgis Gozadinos
13bbdc2ed6
Document autocomplete, add rm alias to delete 2025-09-05 12:04:20 +03:00
Yiorgis Gozadinos
572e3605fe
Autocompletion for ids & paths 2025-09-05 12:00:40 +03:00
Yiorgis Gozadinos
d122252e16
If a document is deleted but does not exist, say so 2025-09-05 11:46:58 +03:00
Yiorgis Gozadinos
495e01a33d
Disable other logging facilities when running on cli 2025-09-05 11:39:19 +03:00
Yiorgis Gozadinos
b99f88f104
Support batch embedding 2025-09-05 10:48:31 +03:00
Yiorgis Gozadinos
bc28e60cf7
Support vLLM for QA agents 2025-09-04 15:20:37 +03:00
Yiorgis Gozadinos
f2847d0524
Reranking support for vLLM, documentation update 2025-09-04 13:13:10 +03:00
Yiorgis Gozadinos
16f7ba9d99
VLLM embedding 2025-09-04 10:44:07 +03:00
Yiorgis Gozadinos
3c235e8b6c
Suppress logging from deps 2025-09-03 11:42:08 +03:00
Yiorgis Gozadinos
450a58eb52
Update OpenAI pydantic model 2025-09-03 11:41:48 +03:00
Yiorgis Gozadinos
974451edc7
Suppress httpx logging 2025-09-03 11:27:33 +03:00
Yiorgis Gozadinos
221a00ab2f
Turn migration into a cli command 2025-09-03 11:12:23 +03:00
Yiorgis Gozadinos
ed604f6a1d
Batch operations & small optimization 2025-09-03 10:50:08 +03:00
Yiorgis Gozadinos
75e878308e
Simplify deleting all chunks 2025-09-02 15:13:31 +03:00
Yiorgis Gozadinos
ecf8894755
Fix checking settings 2025-09-02 13:01:36 +03:00
Yiorgis Gozadinos
9e5f11060d
Allow connecting to lancedb cloud, s3, gc, az etc 2025-09-02 12:49:17 +03:00
Yiorgis Gozadinos
767d36b494
Use a lock for optimize(), when bulk creating chunks optimize once. 2025-09-02 12:10:54 +03:00
Yiorgis Gozadinos
fe850e778d
Remove Ollama reranker, this was a bad idea 2025-09-02 11:42:31 +03:00
Yiorgis Gozadinos
e08fce84f8
Debounce optimize() to avoid exceeding too-many-files-open 2025-09-02 10:55:58 +03:00
Yiorgis Gozadinos
b05e5f5408
Set/get haiku version to db 2025-09-01 17:41:41 +03:00
Yiorgis Gozadinos
58a04db556
Disable reranking by default 2025-09-01 15:00:38 +03:00
Yiorgis Gozadinos
d233788400
Optimize table after insertion, update instead of recreating the index 2025-09-01 14:56:15 +03:00
Yiorgis Gozadinos
a5e087aa53
Fix scores depending on search type 2025-09-01 14:56:15 +03:00
Yiorgis Gozadinos
628bdc8151
Cleanup 2025-09-01 14:56:15 +03:00
Yiorgis Gozadinos
0592a7d122
Adapt search to lancedb 2025-09-01 14:56:14 +03:00
Yiorgis Gozadinos
b83596ff07
Basic moving to lancedb. 2025-09-01 14:56:14 +03:00
Yiorgis Gozadinos
1f61d78165
Bring back QA_MODEL and keep compatibility with previous versions 2025-08-17 15:27:03 +02:00
Yiorgis Gozadinos
d6d79d5e94
Allow any provider/model supported by pydantic AI 2025-08-17 14:55:50 +02:00
Yiorgis Gozadinos
60543caa36
Use /no_think for Ollama 2025-08-15 23:13:27 +02:00
Yiorgis Gozadinos
79841f32ca
Do not use mocks in embeddings test 2025-08-15 22:01:57 +02:00
Yiorgis Gozadinos
26a3715902
Refactor Ollama reranker around pydantic AI 2025-08-15 21:26:16 +02:00
Yiorgis Gozadinos
390397570a
Migrate QA agent to Pydantic AI. 2025-08-15 21:13:17 +02:00
Yiorgis Gozadinos
bf8a4f22be
Make radius a parameter we can pass to expand_context() 2025-08-12 16:53:45 +02:00
Yiorgis Gozadinos
99f7401817
Merge expanded context if possible 2025-08-12 15:02:07 +02:00
Yiorgis Gozadinos
e7c2a105ed
Use the expand_context when searching in the qa agent 2025-08-12 14:51:33 +02:00
Yiorgis Gozadinos
eb13426aae
expand_context() by using the adjacent chunks 2025-08-12 14:14:18 +02:00
Yiorgis Gozadinos
2f42b1a3ac
Get adjacent chunks method 2025-08-12 13:24:04 +02:00
Yiorgis Gozadinos
842e4a882d
Make citations optional, through cite parameter 2025-08-12 13:19:11 +02:00
Yiorgis Gozadinos
a0d0359618
Change prompt and tool definition to to include citations in the QA agent. 2025-08-12 13:05:37 +02:00
Yiorgis Gozadinos
6f50706e8a
Give the results of search to the qa agent as json including the document uri 2025-08-12 12:40:55 +02:00
Yiorgis Gozadinos
e39be89666
Merge pull request #33 from ggozad/feat/ollama-reranker
Add custom reranker based on Ollama.
2025-08-11 11:18:42 +02:00
Yiorgis Gozadinos
022f8677b3
Merge pull request #32 from ggozad/chore/slim-deps
Make mxbai reranker optional. Only use a reranker if mxbai or cohere are installed.
2025-08-11 11:18:27 +02:00
Yiorgis Gozadinos
9648cb97cb
Fix URL handling in add-src 2025-08-11 11:10:29 +02:00