Commit graph

4 commits

Author SHA1 Message Date
Yiorgis Gozadinos
e49f4264d3
Update lancedb to 0.37.1
The newest release with wheels for every supported platform: 0.38.0
publishes none for x86_64 Linux or Windows. The zero-coverage FTS scan
path is still broken at 0.37.1, so the guard, prevention and repair
carry unchanged. ensure_indexes names the index it declares, keeping
replace deterministic on 0.38+, where an unnamed create_index builds a
suffixed sibling instead of replacing a different-typed index. An
invalid search filter raises ValueError.
2026-08-31 18:54:48 +03:00
Yiorgis Gozadinos
02bccb1b90
Repair an FTS index that covers no rows on write
Deleting or replacing every indexed row while unindexed rows remain
returns lance to the zero-coverage scan path. ensure_indexes rebuilds
a declared FTS index that covers none of a populated table's rows,
and delete_by_document_id runs index maintenance like the other chunk
writes. A legacy database in that state is repaired by its first write.
2026-08-31 18:54:48 +03:00
Yiorgis Gozadinos
35f8d721b1
Build the chunks FTS index on the first write instead of at table creation
An FTS index built over an empty table indexes nothing and lance never
catches it up on add. ensure_indexes skips FTS while the table is
empty; create, replace_for_document and embed-only rebuild ensure
indexes after writing, so the index always covers at least its first
rows. The first write into a fresh table writes one extra chunks table
version for the index build; a failed build fails the write.
2026-08-31 18:42:31 +03:00
Yiorgis Gozadinos
629e1ba4ea
Split the store module by responsibility
engine.py held four unrelated things: what the tables are, how to open a
connection, how to read a database's state, and the Store that coordinates
writes. At 1240 lines the Store's own concerns — locks, migrations, vacuum,
tags — were hard to find among them.

Table records, Arrow schemas, index_specs, ensure_indexes, REQUIRED_TABLES
and query_to_pydantic move to store/schema.py, which imports nothing from
haiku.rag: it describes the tables and never opens or mutates one.

gather_database_info, get_database_stats, DatabaseInfo and its result models
move to store/info.py. Nothing in Store calls them — they are read paths for
the CLI, doctor, inspector and ingester API — so info depends on engine and
not the reverse.

engine.py keeps the Store, ConnectionMode, connect_lancedb, the tag helpers
and the restore-order and retention constants. No re-exports: importers
point at the new modules.

test_app_info_uses_connect_lancedb_for_remote patched
haiku.rag.store.engine.connect_lancedb; gather_database_info now binds that
name in info.py, so the patch targets where the call is looked up.
2026-08-20 12:13:51 +03:00