Commit graph

27 commits

Author SHA1 Message Date
Yiorgis Gozadinos
028217b7c0
Ask the session which database an operation works on
One session field, whichever kind it is, with the covered databases derived
from it rather than kept beside it. `_single_session` returns the database a
write works on in place of ten guards that only asserted one existed, and the
single-database operations move to the session that owns them.
2026-08-25 09:28:20 +03:00
Yiorgis Gozadinos
79a63a46d4
Give one database its own session
SingleDatabaseSession owns the store, the repositories and the vacuum
machinery, so nothing above has to ask whether it has a store. The client
keeps every name callers already use.
2026-08-24 17:13:45 +03:00
Yiorgis Gozadinos
e8f00fcff4
Make get_config the only configuration lookup
haiku.rag.config exported two configuration instances: the lazy _config
behind get_config/set_config, and Config, loaded at import time. Nothing
linked them, and eleven signatures captured Config as a default argument,
so set_config could not reach the factories, the client, the store or the
MCP server. reranking/base.py went further and snapshotted the configured
reranker name into a class attribute at import.

Config is removed. Internal defaults are config: AppConfig | None = None,
resolved through get_config() per call. RerankerBase._model is None and
CohereReranker takes its model name as an argument, like every other
reranker.

The suite patched attributes on Config while production read the instance
get_config() returns, a different object, so those patches were no-ops
waiting to happen. They now go through get_config().
2026-08-19 14:43:40 +03:00
Yiorgis Gozadinos
2da294b850
Put multi-table writes behind one transaction boundary
Four call sites repeated lock, snapshot, try/except, restore. Each used
restore_table_versions, which restores in _tables() order — documents
first, contradicting RESTORE_TABLE_ORDER — and each caught Exception, so
a cancellation mid-write skipped rollback and left the earlier table
writes committed.

Store.write_transaction() holds the lock, snapshots under it, and rolls
back through _rollback_to_snapshot: RESTORE_TABLE_ORDER, shielded from
cancellation, absorbed cancellation re-delivered, rollback failure raised
with the original as cause. The two single-table update_meta sites keep
the bare lock.

The batch documents write moves inside the guarded body; it was outside
the try, so a failure there was never rolled back. Auto-vacuum is
scheduled after the transaction rather than inside it.

restore_table_versions is removed; those four sites were its only callers.
2026-08-19 14:05:40 +03:00
Yiorgis Gozadinos
5d5d87d44c
Throttle background auto-vacuum to at most once per 5 minutes 2026-06-12 10:17:18 +03:00
Yiorgis Gozadinos
4bc52f0710
Serialize multi-table document writes and bound update version churn 2026-06-12 10:16:39 +03:00
Yiorgis Gozadinos
e7c7df2915
Always use the Store-owned embedder 2026-05-29 11:36:53 +03:00
Yiorgis Gozadinos
b6ea07d2de
Track all in-flight background vacuum tasks, not just the last one 2026-04-24 14:42:53 +03:00
Yiorgis Gozadinos
ba911a35f6
Add test for background vaccum being awaited 2026-04-24 14:42:52 +03:00
Yiorgis Gozadinos
82fd10e0ee
Migrate LanceDB to native async API
Convert all LanceDB operations from sync calls wrapped in async
functions to the native async API (connect_async, AsyncConnection,
AsyncTable, AsyncQuery). Database I/O no longer blocks the event loop.

- Store and HaikuRAG use async context managers (async with). Store
  initialization is deferred to __aenter__; direct construction
  without async with is no longer supported.
- Index creation uses config objects (FTS, BTree, IvfPq) instead of
  string-based index_type parameter.
- Upgrade callbacks are async.
- HaikuRAG tracks background vacuum tasks and awaits them in __aexit__
  and before destructive rebuild operations to avoid races with
  concurrent table mutations.
- temp_db_path fixture uses pytest's tmp_path for reliable async
  cleanup.
2026-04-24 14:42:52 +03:00
Yiorgis Gozadinos
5e55c0df54
replace pyright with ty type checker 2026-01-19 15:50:14 +02:00
Yiorgis Gozadinos
5464d24483
require explicit migrate command for database migrations 2026-01-19 13:22:09 +02:00
Yiorgis Gozadinos
1e9a10a235
Record cassettes for embedder, ignore huggingface while recording 2025-12-29 13:23:53 +02:00
Yiorgis Gozadinos
2161727716
Add auto_vacuum config option to control automatic vacuuming 2025-12-10 11:42:14 +02:00
Yiorgis Gozadinos
e97d235c98
Unify update_document, update_document_fields 2025-12-08 15:56:02 +02:00
Yiorgis Gozadinos
69b1afa534
Remove _create_and_chunk & _update_and_rechunk, keep storage independent of chunking 2025-12-08 15:55:31 +02:00
Yiorgis Gozadinos
33b254ae3d
Always upgrade database on access. Explicitly create db by running init or HaikuRAG(path, create=True) 2025-12-02 16:49:55 +02:00
Yiorgis Gozadinos
14bdb7cb27
async versions for convert_file() convert_text() 2025-11-26 17:46:25 +02:00
Yiorgis Gozadinos
2c554a9350
Do not use private methods in tests 2025-11-25 14:19:28 +02:00
Yiorgis Gozadinos
7ba1c2376e
Introduce converters for supporting more than local docling document conversion. Transform existing FileReader and utils to "docling-local" converter 2025-11-17 13:05:05 +02:00
Yiorgis Gozadinos
908296e155
Do not create unecessary DoclingDocument when chunks are provided. Rename private _create_with_docling & _update_with_docling to _create_and_chunk & _update_and_rechunk 2025-11-13 13:16:15 +02:00
Yiorgis Gozadinos
618b3ffd5f
Refactor config from flat to nested Pydantic models 2025-10-23 13:30:21 +03:00
Yiorgis Gozadinos
5e4928ccad
Make sure vacuum task has been complete before exiting the client 2025-10-07 12:27:55 +03:00
Yiorgis Gozadinos
094f0e0d44
Centralize vacuuming 2025-10-07 12:05:25 +03:00
Yiorgis Gozadinos
c6eaf9d5cd
Add vacuum retention setting 2025-10-07 10:47:28 +03:00
Yiorgis Gozadinos
184fd12e42
Do not run db upgrades when creating a new db 2025-09-25 10:52:11 +03:00
Yiorgis Gozadinos
4a07444030
Rollback version if a failure occurs when creating/updating documents 2025-09-15 11:54:50 +03:00