Yiorgis Gozadinos
|
9e5f11060d
|
Allow connecting to lancedb cloud, s3, gc, az etc
|
2025-09-02 12:49:17 +03:00 |
|
Yiorgis Gozadinos
|
767d36b494
|
Use a lock for optimize(), when bulk creating chunks optimize once.
|
2025-09-02 12:10:54 +03:00 |
|
Yiorgis Gozadinos
|
fe850e778d
|
Remove Ollama reranker, this was a bad idea
|
2025-09-02 11:42:31 +03:00 |
|
Yiorgis Gozadinos
|
e08fce84f8
|
Debounce optimize() to avoid exceeding too-many-files-open
|
2025-09-02 10:55:58 +03:00 |
|
Yiorgis Gozadinos
|
b05e5f5408
|
Set/get haiku version to db
|
2025-09-01 17:41:41 +03:00 |
|
Yiorgis Gozadinos
|
58a04db556
|
Disable reranking by default
|
2025-09-01 15:00:38 +03:00 |
|
Yiorgis Gozadinos
|
d233788400
|
Optimize table after insertion, update instead of recreating the index
|
2025-09-01 14:56:15 +03:00 |
|
Yiorgis Gozadinos
|
a5e087aa53
|
Fix scores depending on search type
|
2025-09-01 14:56:15 +03:00 |
|
Yiorgis Gozadinos
|
628bdc8151
|
Cleanup
|
2025-09-01 14:56:15 +03:00 |
|
Yiorgis Gozadinos
|
0592a7d122
|
Adapt search to lancedb
|
2025-09-01 14:56:14 +03:00 |
|
Yiorgis Gozadinos
|
b83596ff07
|
Basic moving to lancedb.
|
2025-09-01 14:56:14 +03:00 |
|
Yiorgis Gozadinos
|
1f61d78165
|
Bring back QA_MODEL and keep compatibility with previous versions
|
2025-08-17 15:27:03 +02:00 |
|
Yiorgis Gozadinos
|
d6d79d5e94
|
Allow any provider/model supported by pydantic AI
|
2025-08-17 14:55:50 +02:00 |
|
Yiorgis Gozadinos
|
60543caa36
|
Use /no_think for Ollama
|
2025-08-15 23:13:27 +02:00 |
|
Yiorgis Gozadinos
|
79841f32ca
|
Do not use mocks in embeddings test
|
2025-08-15 22:01:57 +02:00 |
|
Yiorgis Gozadinos
|
26a3715902
|
Refactor Ollama reranker around pydantic AI
|
2025-08-15 21:26:16 +02:00 |
|
Yiorgis Gozadinos
|
390397570a
|
Migrate QA agent to Pydantic AI.
|
2025-08-15 21:13:17 +02:00 |
|
Yiorgis Gozadinos
|
bf8a4f22be
|
Make radius a parameter we can pass to expand_context()
|
2025-08-12 16:53:45 +02:00 |
|
Yiorgis Gozadinos
|
99f7401817
|
Merge expanded context if possible
|
2025-08-12 15:02:07 +02:00 |
|
Yiorgis Gozadinos
|
e7c2a105ed
|
Use the expand_context when searching in the qa agent
|
2025-08-12 14:51:33 +02:00 |
|
Yiorgis Gozadinos
|
eb13426aae
|
expand_context() by using the adjacent chunks
|
2025-08-12 14:14:18 +02:00 |
|
Yiorgis Gozadinos
|
2f42b1a3ac
|
Get adjacent chunks method
|
2025-08-12 13:24:04 +02:00 |
|
Yiorgis Gozadinos
|
842e4a882d
|
Make citations optional, through cite parameter
|
2025-08-12 13:19:11 +02:00 |
|
Yiorgis Gozadinos
|
a0d0359618
|
Change prompt and tool definition to to include citations in the QA agent.
|
2025-08-12 13:05:37 +02:00 |
|
Yiorgis Gozadinos
|
6f50706e8a
|
Give the results of search to the qa agent as json including the document uri
|
2025-08-12 12:40:55 +02:00 |
|
Yiorgis Gozadinos
|
e39be89666
|
Merge pull request #33 from ggozad/feat/ollama-reranker
Add custom reranker based on Ollama.
|
2025-08-11 11:18:42 +02:00 |
|
Yiorgis Gozadinos
|
022f8677b3
|
Merge pull request #32 from ggozad/chore/slim-deps
Make mxbai reranker optional. Only use a reranker if mxbai or cohere are installed.
|
2025-08-11 11:18:27 +02:00 |
|
Yiorgis Gozadinos
|
9648cb97cb
|
Fix URL handling in add-src
|
2025-08-11 11:10:29 +02:00 |
|
Yiorgis Gozadinos
|
ffe867f7bb
|
Allow to disable reranking by having an empty provider
|
2025-08-10 21:02:17 +02:00 |
|
Yiorgis Gozadinos
|
5103e6c2d7
|
Add Ollama reranker
|
2025-08-10 20:50:17 +02:00 |
|
Yiorgis Gozadinos
|
8d98eed2e2
|
Make mxbai reranker optional. Only use a reranker if mxbai or cohere are installed
|
2025-08-10 19:49:14 +02:00 |
|
Yiorgis Gozadinos
|
0b3477c46c
|
Rebuild now re-adds files.
|
2025-08-08 18:30:53 +02:00 |
|
Yiorgis Gozadinos
|
22ed9ca51e
|
Disable python warning unless in development
|
2025-08-08 17:31:42 +02:00 |
|
Yiorgis Gozadinos
|
ad4b655a31
|
Operate on DoclingDocument instead of Document
|
2025-08-08 17:24:34 +02:00 |
|
Yiorgis Gozadinos
|
6a3c85e6ae
|
text_to_docling_document() util
|
2025-08-08 15:38:57 +02:00 |
|
Yiorgis Gozadinos
|
b92f25d6fb
|
Make FileReader return a DoclingDocument instead of a md string
|
2025-08-08 14:45:24 +02:00 |
|
Yiorgis Gozadinos
|
7906b1b026
|
Use absolute paths when add-src
|
2025-08-08 14:34:13 +02:00 |
|
Yiorgis Gozadinos
|
af95b5f3eb
|
Replace loop.run_until_complete() with asyncio.run()
|
2025-08-08 12:41:50 +02:00 |
|
Yiorgis Gozadinos
|
9b4942401f
|
Fix file extension typo
|
2025-08-01 17:15:14 +02:00 |
|
Yiorgis Gozadinos
|
ec1bc21a91
|
Contextualize chunks
|
2025-08-01 16:54:51 +02:00 |
|
Yiorgis Gozadinos
|
a453853211
|
Keep using the openai tokenizer
|
2025-08-01 16:36:24 +02:00 |
|
Yiorgis Gozadinos
|
91f6122ef7
|
Basic chunking using docling
|
2025-08-01 16:27:44 +02:00 |
|
Yiorgis Gozadinos
|
75c1cdb36c
|
Merge pull request #26 from ggozad/feat/docling
Use docling for parsing documents
|
2025-08-01 16:10:34 +02:00 |
|
Yiorgis Gozadinos
|
4bfd54b2ec
|
Merge pull request #25 from ggozad/feat/sqlite_wal
Use Write-Ahead Logging jorunaling mode for SQLite concurrency
|
2025-08-01 16:10:13 +02:00 |
|
Yiorgis Gozadinos
|
14d8b18f1d
|
Wrap code files in md code blocks
|
2025-08-01 10:13:46 +02:00 |
|
Yiorgis Gozadinos
|
5a08816ed6
|
Replace markitdown with docling
|
2025-08-01 09:59:53 +02:00 |
|
Yiorgis Gozadinos
|
0f2809cb8d
|
Use Write-Ahead Logging jorunaling mode for SQLite concurrency
|
2025-08-01 09:07:31 +02:00 |
|
Yiorgis Gozadinos
|
8052442052
|
Add -v --version option to display version
|
2025-08-01 09:00:20 +02:00 |
|
Yiorgis Gozadinos
|
74eea9284a
|
create_document now optionally accepts pre-computed chunks
|
2025-07-26 10:42:27 +03:00 |
|
Yiorgis Gozadinos
|
6a7d2ff319
|
Merge pull request #20 from ggozad/fix/honour-default_data_dir
Honour DEFAULT_DATA_DIR env variable in CLI
|
2025-07-23 13:52:25 +03:00 |
|
Yiorgis Gozadinos
|
72ce212ad4
|
Honour DEFAULT_DATA_DIR env variable
|
2025-07-23 13:49:27 +03:00 |
|
Yiorgis Gozadinos
|
dfe23482da
|
Tune parameters
|
2025-07-20 22:45:24 +03:00 |
|
Yiorgis Gozadinos
|
ad9879802a
|
Fix incorrect syntax for uv pip install with extras
|
2025-07-20 22:40:12 +03:00 |
|
Yiorgis Gozadinos
|
942be2271c
|
Update docs
|
2025-07-20 11:33:09 +03:00 |
|
Yiorgis Gozadinos
|
c1d5334804
|
Update benchmarks with reranking
|
2025-07-19 19:02:47 +03:00 |
|
Yiorgis Gozadinos
|
2347612fb6
|
Optionally perform reranking when searching
|
2025-07-19 13:12:34 +03:00 |
|
Yiorgis Gozadinos
|
1437942edc
|
Make reranker a global module object to avoid re-initialization
|
2025-07-19 12:51:22 +03:00 |
|
Yiorgis Gozadinos
|
2343ab7751
|
Return tuple (Chunk, score,) in rerank()
|
2025-07-19 12:16:17 +03:00 |
|
Yiorgis Gozadinos
|
f25416b0ff
|
Properly test cohere reranker
|
2025-07-18 20:55:28 +03:00 |
|
Yiorgis Gozadinos
|
044e1b0a06
|
mixedbread reranker as default
|
2025-07-18 20:29:24 +03:00 |
|
Yiorgis Gozadinos
|
0f8124fd98
|
Simply embedders
|
2025-07-17 13:17:47 +03:00 |
|
Yiorgis Gozadinos
|
859860318a
|
Cohere reranker
|
2025-07-17 13:17:16 +03:00 |
|
Yiorgis Gozadinos
|
20beb23a38
|
Fix docstrings
|
2025-07-16 19:06:54 +03:00 |
|
Yiorgis Gozadinos
|
4d8088efa8
|
Log db upgrades
|
2025-07-16 18:32:58 +03:00 |
|
Yiorgis Gozadinos
|
0bf5606117
|
Store haiku.rag version, enabling upgrades
|
2025-07-16 18:21:41 +03:00 |
|
Yiorgis Gozadinos
|
f4aef4eb9c
|
Check pypi version on startup
|
2025-07-14 12:27:28 +03:00 |
|
Yiorgis Gozadinos
|
a2e0f2cefd
|
Remove Claude nonsense, use context manager
|
2025-07-12 23:00:49 +03:00 |
|
Yiorgis Gozadinos
|
62a519b094
|
Prevent rebuild from throwing and recreate the embeddings table
|
2025-07-12 21:21:14 +03:00 |
|
Yiorgis Gozadinos
|
4444ec50da
|
Check if settings are compatible when loading a db
|
2025-07-12 20:56:49 +03:00 |
|
Yiorgis Gozadinos
|
a59b38158d
|
Settings table & repo, tests
|
2025-07-12 20:21:34 +03:00 |
|
Yiorgis Gozadinos
|
2f261a33f5
|
Allow multiple tool calling rounds in QA agent
|
2025-07-08 12:51:58 +03:00 |
|
Yiorgis Gozadinos
|
bea96f1b1b
|
Better prompts
|
2025-07-04 13:29:51 +03:00 |
|
Yiorgis Gozadinos
|
1705e7a21c
|
Honour QA_MODEL for anthropic and OpenAI
|
2025-07-04 12:14:13 +03:00 |
|
Yiorgis Gozadinos
|
e66c160055
|
Support for anthropic in Question/Answering
|
2025-07-04 11:55:03 +03:00 |
|
Yiorgis Gozadinos
|
27e1c9fd49
|
Show settings command
|
2025-07-04 11:54:16 +03:00 |
|
Yiorgis Gozadinos
|
85443ee963
|
Turn rebuild into a generator, track progress in command line
|
2025-07-02 11:10:31 +03:00 |
|
Yiorgis Gozadinos
|
11d9f2701e
|
Rebuild database in client & cli
|
2025-07-02 10:55:06 +03:00 |
|
Yiorgis Gozadinos
|
954e96370a
|
Log document uri, meta in search results
|
2025-06-29 09:26:33 +03:00 |
|
Yiorgis Gozadinos
|
e5d8396d1b
|
Merge pull request #5 from ggozad/feat/qa
Question/Answering using LLMs
|
2025-06-28 09:41:55 +03:00 |
|
Yiorgis Gozadinos
|
4bbc23dbd4
|
Add QA to client, cli
|
2025-06-28 09:21:45 +03:00 |
|
Yiorgis Gozadinos
|
85b106c461
|
OpenAI Question/Answer agent
|
2025-06-28 09:11:39 +03:00 |
|
Yiorgis Gozadinos
|
f3ed88b970
|
Include document uri and meta in search results
|
2025-06-28 09:10:23 +03:00 |
|
Yiorgis Gozadinos
|
ca45f8e5d4
|
Fix db path
|
2025-06-25 19:01:57 +03:00 |
|
Yiorgis Gozadinos
|
095d532a7a
|
QA LLM, Ollama implementation
|
2025-06-25 18:18:28 +03:00 |
|
Yiorgis Gozadinos
|
3c98f246b5
|
Merge pull request #2 from ggozad/feat/monitor
File monitoring support
|
2025-06-20 10:30:40 +02:00 |
|
Yiorgis Gozadinos
|
03b2b20ae2
|
Handle keyboard interrupt
|
2025-06-20 10:18:07 +02:00 |
|
Yiorgis Gozadinos
|
51e7f990a0
|
Implement upsert and delete in monitor
|
2025-06-20 09:28:44 +02:00 |
|
Yiorgis Gozadinos
|
ee57bc0a82
|
Use path.as_uri() instead of path.resolve()
|
2025-06-20 09:27:01 +02:00 |
|
Yiorgis Gozadinos
|
8132a44994
|
Use MONITOR_DIRECTORIES for a list of dirs to monitor
|
2025-06-20 09:04:07 +02:00 |
|
Yiorgis Gozadinos
|
5f604d31dd
|
Add monitor
|
2025-06-20 08:37:56 +02:00 |
|
Yiorgis Gozadinos
|
943cf3c5e0
|
Rename EMBDEDDING->EMBEDDINGS in env vars
|
2025-06-19 17:44:42 +02:00 |
|
Yiorgis Gozadinos
|
91c4c30f65
|
Add support for OpenAI embeddings
|
2025-06-19 17:38:49 +02:00 |
|
Yiorgis Gozadinos
|
8fbf98466a
|
MCP server
|
2025-06-19 15:00:57 +02:00 |
|
Yiorgis Gozadinos
|
9f5ba912d7
|
Simplify repos
|
2025-06-19 08:55:34 +02:00 |
|
Yiorgis Gozadinos
|
8a5c505b86
|
Replace positional params by keyword params for clarity
|
2025-06-19 08:21:58 +02:00 |
|
Yiorgis Gozadinos
|
d7c9b41a2c
|
Console app
|
2025-06-18 16:23:39 +02:00 |
|
Yiorgis Gozadinos
|
ac5c60fecc
|
default sqlite when no path is given
|
2025-06-18 10:41:07 +02:00 |
|
Yiorgis Gozadinos
|
a1079fd1f5
|
VoyageAI embeddings
|
2025-06-18 09:45:36 +02:00 |
|
Yiorgis Gozadinos
|
51723308ed
|
Introduce embedder factory
|
2025-06-18 08:26:11 +02:00 |
|
Yiorgis Gozadinos
|
1e0b6b18aa
|
Add a context manager to the client
|
2025-06-17 12:48:26 +02:00 |
|