Commit graph

21 commits

Author SHA1 Message Date
Yiorgis Gozadinos
34180a0fd1
One reference and one placement for a database
DatabaseRef is a name and a location. The configuration places databases
through lancedb.databases alone; with none configured the default is the
entry haiku.rag under storage.data_dir, selectable like any other.
lancedb.uri is removed, and a config carrying it fails to load with the
replacement spelled out. A path passed from Python is valid where the
configuration places nothing and raises AmbiguousDatabaseError beside
lancedb.databases; haiku-rag --db and haiku-ingester --db construct the
scope directly, so a human's override keeps working. Every database
answers to a name, and a database given as a path keeps its own errors.
2026-09-03 15:12:08 +03:00
Yiorgis Gozadinos
9baa213b34
Hand storage the database location, not the configuration that placed it
Store, connect_lancedb, gather_database_info and run_doctor take a
location, a path or a URI, and classify it with ConnectionMode.of.
SingleDatabaseSession owns the resolved DatabaseRef and passes its
location down. This removes DatabaseRef.connection(), default_db_path,
the placeholder path for URI-backed databases and the per-database
config copies, so the configuration a client holds is the one the caller
gave it. The chat hands its capabilities the scope it opened along with
the client it lends, and the v0.58.0 migration no longer checks local
free disk for a database behind a URI.
2026-09-03 15:11:37 +03:00
Yiorgis Gozadinos
afdef92b5b
Finish the comment pass, and escape document fields everywhere Rich renders
`_rich_print_document` escapes uri, title and metadata, the sibling of
the escaped search-result renderer. The remaining comments and
docstrings that narrated rejected alternatives, consequences or history
now state the current invariant. The Sandbox class docstring names the
held connection close() releases, and wrapped docs paragraphs join to
one line.
2026-08-28 15:34:47 +03:00
Yiorgis Gozadinos
09a7076b7e
State what the code does, not what it replaced
Comments and docstrings across the branch narrated rejected
alternatives, consequences and history; each now states the current
contract. Renames test_a_legacy_uri_client_keeps_its_error to
test_an_unnamed_database_keeps_its_error. Documents the Sandbox
connection paths, the citation header's database segment, both
AmbiguousDatabaseError conditions on create_app, and run_inspector's
scope parameter. Doc paragraphs added by the branch in python.md,
storage.md and cli.md are one physical line each.
2026-08-28 15:13:52 +03:00
Yiorgis Gozadinos
b2c617f9f2
Name the example databases as the documentation does
`medic` and `st` become `papers` and `notes`.
2026-08-27 11:06:34 +03:00
Yiorgis Gozadinos
d367b1eb5a
Resolve the databases a command works on, once
The CLI decides only what it knows — that --db and --database are the same
thing said twice, and whether a command reads more than one — and hands the
resolved scope down. Nothing rewrites the configuration, so a named database
keeps the name results and citations carry, and a remote one opens the URI
it was configured with rather than the local path standing in for it.

HaikuRAGApp, ChatApp and InspectorApp take that scope and nothing else.
Selection reaches the client through a private constructor, so the public
signature still takes a path or names.
2026-08-25 15:48:47 +03:00
Yiorgis Gozadinos
629e1ba4ea
Split the store module by responsibility
engine.py held four unrelated things: what the tables are, how to open a
connection, how to read a database's state, and the Store that coordinates
writes. At 1240 lines the Store's own concerns — locks, migrations, vacuum,
tags — were hard to find among them.

Table records, Arrow schemas, index_specs, ensure_indexes, REQUIRED_TABLES
and query_to_pydantic move to store/schema.py, which imports nothing from
haiku.rag: it describes the tables and never opens or mutates one.

gather_database_info, get_database_stats, DatabaseInfo and its result models
move to store/info.py. Nothing in Store calls them — they are read paths for
the CLI, doctor, inspector and ingester API — so info depends on engine and
not the reverse.

engine.py keeps the Store, ConnectionMode, connect_lancedb, the tag helpers
and the restore-order and retention constants. No re-exports: importers
point at the new modules.

test_app_info_uses_connect_lancedb_for_remote patched
haiku.rag.store.engine.connect_lancedb; gather_database_info now binds that
name in info.py, so the patch targets where the call is looked up.
2026-08-20 12:13:51 +03:00
Yiorgis Gozadinos
ce1b9ac88e
Harden history against tag annotation failures; document tag restore 2026-07-16 13:11:34 +03:00
Yiorgis Gozadinos
0813a1c980
Add tag CLI commands and history tag annotations 2026-07-16 13:11:27 +03:00
Yiorgis Gozadinos
82fd10e0ee
Migrate LanceDB to native async API
Convert all LanceDB operations from sync calls wrapped in async
functions to the native async API (connect_async, AsyncConnection,
AsyncTable, AsyncQuery). Database I/O no longer blocks the event loop.

- Store and HaikuRAG use async context managers (async with). Store
  initialization is deferred to __aenter__; direct construction
  without async with is no longer supported.
- Index creation uses config objects (FTS, BTree, IvfPq) instead of
  string-based index_type parameter.
- Upgrade callbacks are async.
- HaikuRAG tracks background vacuum tasks and awaits them in __aexit__
  and before destructive rebuild operations to avoid races with
  concurrent table mutations.
- temp_db_path fixture uses pytest's tmp_path for reliable async
  cleanup.
2026-04-24 14:42:52 +03:00
Yiorgis Gozadinos
a1181ff52d
extract get_database_stats and share it across info, inspector, and backend 2026-04-17 15:33:30 +03:00
Yiorgis Gozadinos
a0a9a3410b
info: report partial stats and pending migrations on pre-migration DBs 2026-04-17 14:00:03 +03:00
Yiorgis Gozadinos
364b1bc509
add document_items table for fast context expansion 2026-04-16 12:11:53 +03:00
Yiorgis Gozadinos
343bfd7199
guard app/UI filesystem checks for remote storage 2026-04-08 14:19:09 +03:00
Yiorgis Gozadinos
fc1ea82cb1
Display on info pydantic-ai and docling-document schema version 2026-01-14 18:14:53 +02:00
Yiorgis Gozadinos
bfae711a17
Do not run test requiring docling OCR in CI 2025-12-29 13:44:57 +02:00
Yiorgis Gozadinos
33b254ae3d
Always upgrade database on access. Explicitly create db by running init or HaikuRAG(path, create=True) 2025-12-02 16:49:55 +02:00
Yiorgis Gozadinos
3525fae625
Use EmbeddingModelConfig similar to ModelConfig for embeddings 2025-12-02 11:55:31 +02:00
Yiorgis Gozadinos
2f54ee7d59
Replace allow_create with cleaner read_only. Elaborate on the test_info tests 2025-11-21 14:37:33 +02:00
Yiorgis Gozadinos
f9b6894974
Fix info to not use old .env settings 2025-10-29 13:31:54 +02:00
Yiorgis Gozadinos
52ea1ebb70
info command in CLI 2025-09-23 12:20:25 +03:00