Commit graph

30 commits

Author SHA1 Message Date
Yiorgis Gozadinos
37a78a4e9b
Cache reranker on the client instead of rebuilding per search 2026-05-29 10:33:50 +03:00
Yiorgis Gozadinos
7fac35d2af
rename embed_image_query to embed_image; run description check on text path 2026-05-05 16:15:47 +03:00
Yiorgis Gozadinos
e8d89aa035
dedup picture-only chunks at the search-result layer 2026-05-04 13:31:15 +03:00
Yiorgis Gozadinos
ff656504d3
make client.search() polymorphic on query type: str | bytes | PIL.Image.Image. Bytes/PIL queries embed via embed_image_query and run vector-only against the chunks table 2026-05-04 11:22:22 +03:00
Yiorgis Gozadinos
d0ed9e213f
Pin FTS search column to content_fts 2026-04-24 14:42:53 +03:00
Yiorgis Gozadinos
82fd10e0ee
Migrate LanceDB to native async API
Convert all LanceDB operations from sync calls wrapped in async
functions to the native async API (connect_async, AsyncConnection,
AsyncTable, AsyncQuery). Database I/O no longer blocks the event loop.

- Store and HaikuRAG use async context managers (async with). Store
  initialization is deferred to __aenter__; direct construction
  without async with is no longer supported.
- Index creation uses config objects (FTS, BTree, IvfPq) instead of
  string-based index_type parameter.
- Upgrade callbacks are async.
- HaikuRAG tracks background vacuum tasks and awaits them in __aexit__
  and before destructive rebuild operations to avoid races with
  concurrent table mutations.
- temp_db_path fixture uses pytest's tmp_path for reliable async
  cleanup.
2026-04-24 14:42:52 +03:00
Yiorgis Gozadinos
3c7d746d57
Strengthen search tests, remove redundant title test, relocate primary label test
- Strengthen test_search_returns_search_result with chunk_id, document_id, and
  non-empty labels assertions
- Remove test_chunks_include_document_title (subsumed by strengthened test) and
  its cassette
- Rename test_chunks_include_document_info to
  test_search_chunk_includes_document_provenance; add document_title is None
  assertion for untitled documents
- Move test_search_result_get_primary_label from test_context_enhancement.py to
  test_search.py; rename, remove unnecessary @pytest.mark.vcr() decorator

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 11:39:45 +02:00
Yiorgis Gozadinos
7f51c63ad7
Search results now show rank position instead of raw scores. Consolidate cassettes 2026-01-21 14:54:15 +02:00
Yiorgis Gozadinos
5e55c0df54
replace pyright with ty type checker 2026-01-19 15:50:14 +02:00
Yiorgis Gozadinos
1e9a10a235
Record cassettes for embedder, ignore huggingface while recording 2025-12-29 13:23:53 +02:00
Yiorgis Gozadinos
4da50873bd
Make search limit configurable 2025-12-09 11:46:43 +02:00
Yiorgis Gozadinos
1b14a4643d
Remove unused bounding box calculations 2025-12-08 15:56:21 +02:00
Yiorgis Gozadinos
9a5f8e4368
Add type-aware context expansion for search results 2025-12-08 15:56:21 +02:00
Yiorgis Gozadinos
8e30f67266
Introduce import_document(), remove chunks from create_document() 2025-12-08 15:55:29 +02:00
Yiorgis Gozadinos
b5bd56e110
Fix after rebase 2025-12-08 15:55:27 +02:00
Yiorgis Gozadinos
c69b934eac
Incorporate additional docling meta in search(), expand_context() and relevant qa/research agents 2025-12-08 15:54:56 +02:00
Yiorgis Gozadinos
33b254ae3d
Always upgrade database on access. Explicitly create db by running init or HaikuRAG(path, create=True) 2025-12-02 16:49:55 +02:00
Yiorgis Gozadinos
2c554a9350
Do not use private methods in tests 2025-11-25 14:19:28 +02:00
Yiorgis Gozadinos
7ba1c2376e
Introduce converters for supporting more than local docling document conversion. Transform existing FileReader and utils to "docling-local" converter 2025-11-17 13:05:05 +02:00
Yiorgis Gozadinos
908296e155
Do not create unecessary DoclingDocument when chunks are provided. Rename private _create_with_docling & _update_with_docling to _create_and_chunk & _update_and_rechunk 2025-11-13 13:16:15 +02:00
Yiorgis Gozadinos
73aff1695f
Update LLM prompts/tools to reflect we will pass title (if available) or URI 2025-09-22 12:45:32 +03:00
Yiorgis Gozadinos
a5e087aa53
Fix scores depending on search type 2025-09-01 14:56:15 +03:00
Yiorgis Gozadinos
628bdc8151
Cleanup 2025-09-01 14:56:15 +03:00
Yiorgis Gozadinos
0592a7d122
Adapt search to lancedb 2025-09-01 14:56:14 +03:00
Yiorgis Gozadinos
b83596ff07
Basic moving to lancedb. 2025-09-01 14:56:14 +03:00
Yiorgis Gozadinos
2f42b1a3ac
Get adjacent chunks method 2025-08-12 13:24:04 +02:00
Yiorgis Gozadinos
91f6122ef7
Basic chunking using docling 2025-08-01 16:27:44 +02:00
Yiorgis Gozadinos
f3ed88b970
Include document uri and meta in search results 2025-06-28 09:10:23 +03:00
Yiorgis Gozadinos
0b5f89ebad
FTS and hybrid search using Reciprocal Rank Fusion 2025-06-16 19:28:45 +02:00
Yiorgis Gozadinos
892d1a378d
test search 2025-06-16 18:10:03 +02:00