Commit graph

25 commits

Author SHA1 Message Date
Yiorgis Gozadinos
e8f00fcff4
Make get_config the only configuration lookup
haiku.rag.config exported two configuration instances: the lazy _config
behind get_config/set_config, and Config, loaded at import time. Nothing
linked them, and eleven signatures captured Config as a default argument,
so set_config could not reach the factories, the client, the store or the
MCP server. reranking/base.py went further and snapshotted the configured
reranker name into a class attribute at import.

Config is removed. Internal defaults are config: AppConfig | None = None,
resolved through get_config() per call. RerankerBase._model is None and
CohereReranker takes its model name as an argument, like every other
reranker.

The suite patched attributes on Config while production read the instance
get_config() returns, a different object, so those patches were no-ops
waiting to happen. They now go through get_config().
2026-08-19 14:43:40 +03:00
Yiorgis Gozadinos
2da294b850
Put multi-table writes behind one transaction boundary
Four call sites repeated lock, snapshot, try/except, restore. Each used
restore_table_versions, which restores in _tables() order — documents
first, contradicting RESTORE_TABLE_ORDER — and each caught Exception, so
a cancellation mid-write skipped rollback and left the earlier table
writes committed.

Store.write_transaction() holds the lock, snapshots under it, and rolls
back through _rollback_to_snapshot: RESTORE_TABLE_ORDER, shielded from
cancellation, absorbed cancellation re-delivered, rollback failure raised
with the original as cause. The two single-table update_meta sites keep
the bare lock.

The batch documents write moves inside the guarded body; it was outside
the try, so a failure there was never rolled back. Auto-vacuum is
scheduled after the transaction rather than inside it.

restore_table_versions is removed; those four sites were its only callers.
2026-08-19 14:05:40 +03:00
Yiorgis Gozadinos
5d5d87d44c
Throttle background auto-vacuum to at most once per 5 minutes 2026-06-12 10:17:18 +03:00
Yiorgis Gozadinos
4bc52f0710
Serialize multi-table document writes and bound update version churn 2026-06-12 10:16:39 +03:00
Yiorgis Gozadinos
e7c7df2915
Always use the Store-owned embedder 2026-05-29 11:36:53 +03:00
Yiorgis Gozadinos
b6ea07d2de
Track all in-flight background vacuum tasks, not just the last one 2026-04-24 14:42:53 +03:00
Yiorgis Gozadinos
ba911a35f6
Add test for background vaccum being awaited 2026-04-24 14:42:52 +03:00
Yiorgis Gozadinos
82fd10e0ee
Migrate LanceDB to native async API
Convert all LanceDB operations from sync calls wrapped in async
functions to the native async API (connect_async, AsyncConnection,
AsyncTable, AsyncQuery). Database I/O no longer blocks the event loop.

- Store and HaikuRAG use async context managers (async with). Store
  initialization is deferred to __aenter__; direct construction
  without async with is no longer supported.
- Index creation uses config objects (FTS, BTree, IvfPq) instead of
  string-based index_type parameter.
- Upgrade callbacks are async.
- HaikuRAG tracks background vacuum tasks and awaits them in __aexit__
  and before destructive rebuild operations to avoid races with
  concurrent table mutations.
- temp_db_path fixture uses pytest's tmp_path for reliable async
  cleanup.
2026-04-24 14:42:52 +03:00
Yiorgis Gozadinos
5e55c0df54
replace pyright with ty type checker 2026-01-19 15:50:14 +02:00
Yiorgis Gozadinos
5464d24483
require explicit migrate command for database migrations 2026-01-19 13:22:09 +02:00
Yiorgis Gozadinos
1e9a10a235
Record cassettes for embedder, ignore huggingface while recording 2025-12-29 13:23:53 +02:00
Yiorgis Gozadinos
2161727716
Add auto_vacuum config option to control automatic vacuuming 2025-12-10 11:42:14 +02:00
Yiorgis Gozadinos
e97d235c98
Unify update_document, update_document_fields 2025-12-08 15:56:02 +02:00
Yiorgis Gozadinos
69b1afa534
Remove _create_and_chunk & _update_and_rechunk, keep storage independent of chunking 2025-12-08 15:55:31 +02:00
Yiorgis Gozadinos
33b254ae3d
Always upgrade database on access. Explicitly create db by running init or HaikuRAG(path, create=True) 2025-12-02 16:49:55 +02:00
Yiorgis Gozadinos
14bdb7cb27
async versions for convert_file() convert_text() 2025-11-26 17:46:25 +02:00
Yiorgis Gozadinos
2c554a9350
Do not use private methods in tests 2025-11-25 14:19:28 +02:00
Yiorgis Gozadinos
7ba1c2376e
Introduce converters for supporting more than local docling document conversion. Transform existing FileReader and utils to "docling-local" converter 2025-11-17 13:05:05 +02:00
Yiorgis Gozadinos
908296e155
Do not create unecessary DoclingDocument when chunks are provided. Rename private _create_with_docling & _update_with_docling to _create_and_chunk & _update_and_rechunk 2025-11-13 13:16:15 +02:00
Yiorgis Gozadinos
618b3ffd5f
Refactor config from flat to nested Pydantic models 2025-10-23 13:30:21 +03:00
Yiorgis Gozadinos
5e4928ccad
Make sure vacuum task has been complete before exiting the client 2025-10-07 12:27:55 +03:00
Yiorgis Gozadinos
094f0e0d44
Centralize vacuuming 2025-10-07 12:05:25 +03:00
Yiorgis Gozadinos
c6eaf9d5cd
Add vacuum retention setting 2025-10-07 10:47:28 +03:00
Yiorgis Gozadinos
184fd12e42
Do not run db upgrades when creating a new db 2025-09-25 10:52:11 +03:00
Yiorgis Gozadinos
4a07444030
Rollback version if a failure occurs when creating/updating documents 2025-09-15 11:54:50 +03:00