haiku.rag/haiku_rag_slim/haiku/rag/ingester
Yiorgis Gozadinos 629e1ba4ea
Split the store module by responsibility
engine.py held four unrelated things: what the tables are, how to open a
connection, how to read a database's state, and the Store that coordinates
writes. At 1240 lines the Store's own concerns — locks, migrations, vacuum,
tags — were hard to find among them.

Table records, Arrow schemas, index_specs, ensure_indexes, REQUIRED_TABLES
and query_to_pydantic move to store/schema.py, which imports nothing from
haiku.rag: it describes the tables and never opens or mutates one.

gather_database_info, get_database_stats, DatabaseInfo and its result models
move to store/info.py. Nothing in Store calls them — they are read paths for
the CLI, doctor, inspector and ingester API — so info depends on engine and
not the reverse.

engine.py keeps the Store, ConnectionMode, connect_lancedb, the tag helpers
and the restore-order and retention constants. No re-exports: importers
point at the new modules.

test_app_info_uses_connect_lancedb_for_remote patched
haiku.rag.store.engine.connect_lancedb; gather_database_info now binds that
name in info.py, so the patch targets where the call is looked up.
2026-08-20 12:13:51 +03:00
..
api Split the store module by responsibility 2026-08-20 12:13:51 +03:00
pollers Move source adapters out of the ingester package 2026-08-20 11:46:55 +03:00
queue Add last_heartbeat_at lease column to the ingester queue 2026-06-25 13:25:37 +03:00
workers Move source adapters out of the ingester package 2026-08-20 11:46:55 +03:00
__init__.py
app.py Renew job leases from the worker pool; lease-based reaping 2026-06-25 13:26:09 +03:00
batch.py Add side-effect-free batch dry-run discovery 2026-06-22 11:33:26 +03:00
cli.py Move ingester telemetry setup behind argument parsing 2026-08-13 11:37:36 +03:00
exceptions.py worker pool, rety policy, pipeline 2026-05-26 11:41:54 +03:00
metadata.py Move source adapters out of the ingester package 2026-08-20 11:46:55 +03:00