danvics/haiku.rag
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

State what the code does, not what it replaced

Browse source 
Comments and docstrings across the branch narrated rejected
alternatives, consequences and history; each now states the current
contract. Renames test_a_legacy_uri_client_keeps_its_error to
test_an_unnamed_database_keeps_its_error. Documents the Sandbox
connection paths, the citation header's database segment, both
AmbiguousDatabaseError conditions on create_app, and run_inspector's
scope parameter. Doc paragraphs added by the branch in python.md,
storage.md and cli.md are one physical line each.
This commit is contained in:
Yiorgis Gozadinos 2026-08-28 15:13:52 +03:00
parent d0df382ce8
commit 09a7076b7e
No known key found for this signature in database
43 changed files with 225 additions and 368 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
docs/cli.md
View file

@ -24,11 +24,7 @@ The `haiku-rag` CLI provides complete document management functionality.
haiku-rag add -h
```
With `lancedb.databases` configured, `search`, `ask`, `analyze`, and `chat`
use the full set by default. Select one database for other commands with
`--db-name` or `--db`. `settings`, `init-config`, and `download-models` do
not open a database. See
[Multiple Databases](configuration/storage.md#multiple-databases).
With `lancedb.databases` configured, `search`, `ask`, `analyze`, and `chat` use the full set by default. Select one database for other commands with `--db-name` or `--db`. `settings`, `init-config`, and `download-models` do not open a database. See [Multiple Databases](configuration/storage.md#multiple-databases).
## Document Management

133
docs/configuration/storage.md
View file

@ -68,9 +68,7 @@ This is an upstream limitation rather than a `haiku.rag` setting. Compaction bou
### Changing the Default Database Path
`storage.data_dir` holds the default database, always called
`haiku.rag.lancedb`. To put the database somewhere else for every command, give
`lancedb.uri` a local path:
`storage.data_dir` holds the default database, always called `haiku.rag.lancedb`. To put the database somewhere else for every command, give `lancedb.uri` a local path:
```yaml
lancedb:
@ -79,15 +77,9 @@ lancedb:
An explicit `--db PATH` overrides `lancedb.uri` for that invocation.
This places one database without naming it. Its `source` is `None` in search
results, citations and documents, since only [`lancedb.databases`](#multiple-databases)
assigns the names that carry provenance. A path here changes where the database
lives, not what it is called.
This places one database without naming it. Its `source` is `None` in search results, citations and documents, since only [`lancedb.databases`](#multiple-databases) assigns the names that carry provenance. A path here changes where the database lives, not what it is called.
A value with no scheme is a local path wherever it is configured, so
`haiku-rag init` creates it and every command that opens an existing database
requires it to exist. A mistyped path fails rather than becoming a new empty
database.
A value with no scheme is a local path wherever it is configured, so `haiku-rag init` creates it and every command that opens an existing database requires it to exist. A mistyped path fails rather than becoming a new empty database.
## Database Creation
@ -202,8 +194,7 @@ Each process picks up its own credentials from the AWS default chain (env vars,
## Multiple Databases
Use `lancedb.databases` to name local or remote databases that should be searched
together:
Use `lancedb.databases` to name local or remote databases that should be searched together:
```yaml
lancedb:
@ -213,119 +204,62 @@ lancedb:
notes: /data/notes.lancedb
```
A location can be a URI or local path. `databases` and `uri` are mutually
exclusive.
A location can be a URI or local path. `databases` and `uri` are mutually exclusive.
Results, documents, and citations use the configured name as `source`. An
unavailable configured database raises `SourceUnavailableError`, which names the
database and not its location, so a location never travels in an error a consumer
might render or log. A migration, configuration or read-only failure keeps its
own type, with the database named in the message. Commands that report on a
database, such as `info`, still show where it is.
Results, documents, and citations use the configured name as `source`. An unavailable configured database raises `SourceUnavailableError`, which names the database and not its location, so a location never travels in an error a consumer might render or log. A migration, configuration or read-only failure keeps its own type, with the database named in the message. Commands that report on a database, such as `info`, still show where it is.
Searches spanning multiple databases identify each result with a model-facing
`Collection:` line. Searches over one database omit it. Structured `source`
fields on results, documents, citations, and analysis dictionaries are
unchanged.
Searches spanning multiple databases identify each result with a model-facing `Collection:` line. Searches over one database omit it. Structured `source` fields on results, documents, citations, and analysis dictionaries are unchanged.
Embedding compatibility is checked against two different things.
On open, each database is compared with the current configuration. A dimension
mismatch raises `ConfigMismatchError`. A provider or model-name mismatch at the
same dimension warns in read-only mode and raises in writable mode.
On open, each database is compared with the current configuration. A dimension mismatch raises `ConfigMismatchError`. A provider or model-name mismatch at the same dimension warns in read-only mode and raises in writable mode.
Across a selection, the databases are compared with each other. Vector and
hybrid search embed the query once, so every database answering it must record
the same provider, model, and dimension. A disagreement raises
`ConfigMismatchError` in read-only mode as well. Only the databases searched
together have to agree, and full-text search embeds nothing, so it is
unaffected.
Across a selection, the databases are compared with each other. Vector and hybrid search embed the query once, so every database answering it must record the same provider, model, and dimension. A disagreement raises `ConfigMismatchError` in read-only mode as well. Only the databases searched together have to agree, and full-text search embeds nothing, so it is unaffected.
### Search and Provenance
`search`, `ask`, and `analyze` use the full set by default. Pass `sources` to
select a subset:
`search`, `ask`, and `analyze` use the full set by default. Pass `sources` to select a subset:
```python
results = await client.search("query") # every database
results = await client.search("query", sources=["papers"]) # one of them
```
Candidates are combined into one ranked list with the configured reranker, or
with reciprocal rank fusion when reranking is disabled. `SearchResult.source`,
`Citation.source`, and `Document.source` contain the configured database name.
The name is retained when a client covers only one named database. Databases
configured through `lancedb.uri` are unnamed, so their `source` is `None`.
Candidates are combined into one ranked list with the configured reranker, or with reciprocal rank fusion when reranking is disabled. `SearchResult.source`, `Citation.source`, and `Document.source` contain the configured database name. The name is retained when a client covers only one named database. Databases configured through `lancedb.uri` are unnamed, so their `source` is `None`.
The CLI labels results and citations only when the operation spans multiple
databases. A command already narrowed with `--db-name` does not repeat the name
on every result.
The CLI labels results and citations only when the operation spans multiple databases. A command already narrowed with `--db-name` does not repeat the name on every result.
#### Duplicate IDs
IDs are unique within a database, not across databases. Copies of a database
therefore retain the same IDs.
IDs are unique within a database, not across databases. Copies of a database therefore retain the same IDs.
Citation ambiguity is evaluated against evidence available to the run. A cited
chunk ID is rejected with `AmbiguousCitationError` if search returned it from
multiple databases, or it was previously cited from another database. If only
one retrieved result has the ID, that result is cited. For an ID absent from
search results, the fallback checks every selected database and rejects
multiple holders. A shared ID that nothing cites is ignored.
Citation ambiguity is evaluated against evidence available to the run. A cited chunk ID is rejected with `AmbiguousCitationError` if search returned it from multiple databases, or it was previously cited from another database. If only one retrieved result has the ID, that result is cited. For an ID absent from search results, the fallback checks every selected database and rejects multiple holders. A shared ID that nothing cites is ignored.
`get_document_by_id`, `get_chunk_by_id` and `get_picture_bytes` take an optional
`source`, and ask that database alone. A name the client does not cover raises
`UnknownDatabaseError`. Without one, the document and chunk lookups ask every
covered database and answer from the first that holds the ID; `get_picture_bytes`
requires one whenever the client covers a set.
`get_document_by_id`, `get_chunk_by_id` and `get_picture_bytes` take an optional `source`, and ask that database alone. A name the client does not cover raises `UnknownDatabaseError`. Without one, the document and chunk lookups ask every covered database and answer from the first that holds the ID; `get_picture_bytes` requires one whenever the client covers a set.
The analysis sandbox rejects shared document IDs because its mount path is
`/documents/{id}/`.
The analysis sandbox rejects shared document IDs because its mount path is `/documents/{id}/`.
The chat document filter selects by document and database: the search is narrowed to the databases the selection names, and the ID filter applies within them. An ID that copies share still matches in every selected database that holds it.
#### Ranking
Reciprocal rank fusion compares positions rather than scores, so each database
contributes top-ranked results even when another database has stronger matches. A
reranker scores the combined candidate set directly, which has been measured to
help aggregate retrieval and to hurt attribution between near-identical
documents.
Reciprocal rank fusion compares positions rather than scores, so each database contributes top-ranked results even when another database has stronger matches. A reranker scores the combined candidate set directly, which has been measured to help aggregate retrieval and to hurt attribution between near-identical documents.
Aggregate retrieval is stronger with a reranker. In a 3,045-query evaluation over
a corpus split across three databases, reranking produced retrieval MAP 0.9914,
compared with 0.9918 for the same corpus in one database. Without a reranker, MAP
was 0.6044, compared with 0.9798 in one database. Reranking cost grows with the
number of databases because each contributes candidates.
Aggregate retrieval is stronger with a reranker. In a 3,045-query evaluation over a corpus split across three databases, reranking produced retrieval MAP 0.9914, compared with 0.9918 for the same corpus in one database. Without a reranker, MAP was 0.6044, compared with 0.9798 in one database. Reranking cost grows with the number of databases because each contributes candidates.
A reranker scores the combined candidates with no notion of which database each
came from, so on near-identical text it can pick the wrong database's chunk,
where fusion keeps them apart because each database contributes its own
top-ranked result. In two nine-case acceptance runs over a synthetic corpus
holding one station in two databases under near-identical names, attribution was
weaker with reranking: citing the right database succeeded 5 of 9 and 6 of 9
times with a reranker, against 8 of 9 and 9 of 9 without.
A reranker scores the combined candidates with no notion of which database each came from, so on near-identical text it can pick the wrong database's chunk, where fusion keeps them apart because each database contributes its own top-ranked result. In two nine-case acceptance runs over a synthetic corpus holding one station in two databases under near-identical names, attribution was weaker with reranking: citing the right database succeeded 5 of 9 and 6 of 9 times with a reranker, against 8 of 9 and 9 of 9 without.
Configure a reranker where retrieval breadth matters, and measure it where
answers have to attribute between documents that read alike.
Configure a reranker where retrieval breadth matters, and measure it where answers have to attribute between documents that read alike.
Without a reranker, consider increasing `search.limit` with the number of
databases. With three complete rankings and a limit of 5, a database may
contribute only one or two results. A higher limit also sends more results to
the caller and model.
Without a reranker, consider increasing `search.limit` with the number of databases. With three complete rankings and a limit of 5, a database may contribute only one or two results. A higher limit also sends more results to the caller and model.
Image queries are vector-only and skip the reranker: there is no query text to
score a document against, so candidates keep their vector ranking and fusion
ranks by position.
Image queries are vector-only and skip the reranker: there is no query text to score a document against, so candidates keep their vector ranking and fusion ranks by position.
If a selected database is unavailable, the operation fails with
`SourceUnavailableError`, which names that database.
If a selected database is unavailable, the operation fails with `SourceUnavailableError`, which names that database.
### Python Operations
Creating, writing, rebuilding, and vacuuming require one database. Calling these
operations on a client that covers multiple raises `AmbiguousDatabaseError`.
Select one at creation time or obtain a single-database client:
Creating, writing, rebuilding, and vacuuming require one database. Calling these operations on a client that covers multiple raises `AmbiguousDatabaseError`. Select one at creation time or obtain a single-database client:
```python
async with HaikuRAG(config=config, create=True, sources=["papers"]) as papers:
@ -335,21 +269,15 @@ async with HaikuRAG(config=config) as client:
papers = (await client.clients_for(["papers"]))[0]
```
Conversion, chunking, and title generation do not access a database and remain
available on a multi-database client.
Conversion, chunking, and title generation do not access a database and remain available on a multi-database client.
### CLI Commands
Commands use database sets as follows:
- **Set-capable**: `search`, `ask`, `analyze`, and `chat` use the full
configured set, or the single database selected by `--db-name`.
- **Config-only**: `settings`, `init-config`, and `download-models` do not open
a database.
- **Single-database**: everything else — document writes, `rebuild`, `vacuum`,
`migrate`, `init`, `info`, `history`, `tag`, `doctor`, `list`, `inspect`,
`visualize`, and `mcp` — works on one database, selected with the global
`--db-name` option.
- **Set-capable**: `search`, `ask`, `analyze`, and `chat` use the full configured set, or the single database selected by `--db-name`.
- **Config-only**: `settings`, `init-config`, and `download-models` do not open a database.
- **Single-database**: everything else — document writes, `rebuild`, `vacuum`, `migrate`, `init`, `info`, `history`, `tag`, `doctor`, `list`, `inspect`, `visualize`, and `mcp` — works on one database, selected with the global `--db-name` option.
```bash
haiku-rag search "query" # every configured database
@ -357,10 +285,7 @@ haiku-rag --db-name papers list # one of them
haiku-rag --db-name papers migrate
```
`--db-name` selects an entry from `lancedb.databases`, including remote entries.
`--db` selects a local path and overrides the configured location. A
single-database command requires one of these options when multiple databases are
configured. A configured set of one is selected automatically.
`--db-name` selects an entry from `lancedb.databases`, including remote entries. `--db` selects a local path and overrides the configured location. A single-database command requires one of these options when multiple databases are configured. A configured set of one is selected automatically.
Each database is created, migrated and vacuumed on its own:

30
docs/python.md
View file

@ -24,11 +24,7 @@ async with HaikuRAG("path/to/database.lancedb", read_only=True) as client:
# await client.create_document(...) # Would raise ReadOnlyError
```
`async with` is the lifecycle. A caller that owns the client some other way
releases it with `await client.aclose()`, which does the same work for every
client shape. `client.close()` closes the connection to one database and nothing
else, since draining the background vacuum and releasing the embedder and
reranker are awaitable; it refuses a client covering several.
`async with` is the lifecycle. A caller that owns the client some other way releases it with `await client.aclose()`, which does the same work for every client shape. `client.close()` closes the connection to one database and nothing else, since draining the background vacuum and releasing the embedder and reranker are awaitable; it refuses a client covering several.
!!! note
Databases must be explicitly created with `create=True` or via `haiku-rag init` before use. Opening a nonexistent unnamed local database raises `FileNotFoundError`, naming its path; one named in `lancedb.databases` raises `SourceUnavailableError`, which names the database rather than its location.
@ -240,9 +236,7 @@ for result in results:
### Searching Multiple Databases
With [`lancedb.databases`](configuration/storage.md#multiple-databases)
configured, a client covers the full set. Use `sources` to select a subset.
Each result includes its database name:
With [`lancedb.databases`](configuration/storage.md#multiple-databases) configured, a client covers the full set. Use `sources` to select a subset. Each result includes its database name:
```python
results = await client.search("machine learning") # all of them
@ -262,21 +256,13 @@ for cite in citations:
result = await client.analyze("How many documents mention it?", sources=["papers"])
```
A scoped question can cite only the selected databases. Analysis mounts only
their documents.
A scoped question can cite only the selected databases. Analysis mounts only their documents.
`sources=None` covers every database the client covers. `sources=[]` covers
none: `search` returns no results, and `ask` and `analyze` run with no evidence
from any database.
`sources=None` covers every database the client covers. `sources=[]` covers none: `search` returns no results, and `ask` and `analyze` run with no evidence from any database.
A name no client covers raises `UnknownDatabaseError`, a `KeyError`, wherever it
is given: at construction, per query, and when placing a citation.
A name no client covers raises `UnknownDatabaseError`, a `KeyError`, wherever it is given: at construction, per query, and when placing a citation.
On the constructor `sources=[]` means something else. Passing `sources` alongside a
database path raises `AmbiguousDatabaseError` immediately, since both say which
database to open. Passing `sources=[]` alone raises `ValueError` on entering the
client: a selection of nothing to search is a legitimate question, a client over
no database is not.
On the constructor `sources=[]` means something else. Passing `sources` alongside a database path raises `AmbiguousDatabaseError` immediately, since both say which database to open. Passing `sources=[]` alone raises `ValueError` on entering the client: a selection of nothing to search is a legitimate question, a client over no database is not.
#### Inspecting the client scope
@ -289,9 +275,7 @@ owner = await client.reader_for("papers") # the client reading that databas
papers, wiki = await client.clients_for(["papers", "wiki"])
```
`reader_for` and `clients_for` open databases lazily and return borrowed clients.
They remain valid while the covering client is open and inherit its read-only
mode. The covering client owns and closes their database sessions.
`reader_for` and `clients_for` open databases lazily and return borrowed clients. They remain valid while the covering client is open and inherit its read-only mode. The covering client owns and closes their database sessions.
### Filtering Search Results

5
evaluations/evaluations/capability_runner.py
View file

@ -43,9 +43,8 @@ class CapabilityRunResult:
answer: str
cited_uris: list[str] = field(default_factory=list)
cited_chunk_ids: list[str] = field(default_factory=list)
# The database each cited chunk came from, in the order they were cited, so a
# run over several databases records which one grounded the answer. Empty
# strings where the database is unnamed, since one database names nothing.
# The database each cited chunk came from, in the order they were cited.
# Empty string where the database is unnamed.
cited_sources: list[str] = field(default_factory=list)
searched_uris: list[str] = field(default_factory=list)
n_searches: int = 0

5
evaluations/evaluations/qa.py
View file

@ -186,9 +186,8 @@ def _refusal_metrics(report_cases) -> tuple[float, float, int, int] | None:
def _filter_qa_corpus(corpus, case_ids: set[str] | None):
"""Keep only rows whose ``id`` is in ``case_ids`` (failure-subset reruns).
Returns the corpus unchanged when ``case_ids`` is None. Matching nothing
raises: a dataset keying its rows by another name leaves every case filtered
out, and a run of no cases otherwise reports 0.0000 as though it were a score.
Returns the corpus unchanged when ``case_ids`` is None; matching nothing
raises.
"""
if case_ids is None:
return corpus

6
evaluations/tests/test_benchmark.py
View file

@ -1291,8 +1291,7 @@ class TestEvaluateDatasetCaseIds:
def test_a_case_filter_matching_nothing_raises():
"""A run of no cases reports 0.0000, which reads like a score rather than a
mistake — so an id set that matches nothing fails instead."""
"""An id set that matches no case raises."""
import pytest
from datasets import Dataset
@ -1315,8 +1314,7 @@ def test_a_case_filter_that_matches_keeps_those_rows():
async def test_population_refuses_a_configured_set():
"""Population writes to one database, so a set would be ingested into a
database the run never reads."""
"""Population writes to one database and refuses a configured set."""
import pytest
from haiku.rag.config.models import AppConfig, LanceDBConfig

7
evaluations/tests/test_capability_runner.py
View file

@ -346,9 +346,8 @@ async def test_conversation_applies_document_filter(tmp_path):
async def test_conversation_carries_one_state_dict_across_turns(tmp_path):
"""Capabilities read and write state through the deps dict; carrying the
same dict across turns is what lets compaction see earlier questions'
records instead of refusing."""
"""Capabilities read and write state through the deps dict, and the same
dict is carried across every turn of a conversation."""
from evaluations.capability_runner import run_capability_conversation
deps_seen: list[object] = []
@ -488,7 +487,7 @@ def test_records_the_database_each_citation_came_from():
def test_an_unnamed_database_records_no_source():
"""One database names nothing, so the field stays empty rather than absent."""
"""One database names nothing: the field holds an empty string."""
from haiku.rag.capabilities._base import EvidenceState
from haiku.rag.capabilities.ledger import CapabilityEvidenceRecord
from haiku.rag.store.models.citation import Citation

4
haiku_rag_slim/haiku/rag/app.py
View file

@ -83,7 +83,7 @@ class HaikuRAGApp:
@property
def _is_local(self) -> bool:
"""Whether the database is a local path rather than a URI.
"""True when the database is a local path, False for a URI.
Read from the database this command resolved to, not from the
configuration: a database named in `lancedb.databases` can sit behind a
@ -166,7 +166,7 @@ class HaikuRAGApp:
tables = {t.name: t for t in info.tables}
# Per-table row counts and sizes. Missing required tables are
# reported as "absent" rather than raising.
# reported as "absent".
for name in ("documents", "document_meta", "chunks", "document_items"):
entry = tables[name]
if entry.exists:

8
haiku_rag_slim/haiku/rag/capabilities/_base.py
View file

@ -65,8 +65,7 @@ duplicating a character or a whole group stays above 0.75, so the gap is wide.
def _ambiguous_retry(error: AmbiguousCitationError) -> ModelRetry:
"""The only way out of an id that names a chunk in two databases.
The model cannot say which it meant, so it is asked for other evidence
rather than for the same id again.
The model cannot say which it meant, so the retry asks for other evidence.
"""
return ModelRetry(
f"{error}. Cite a chunk id that appears once across the databases "
@ -188,7 +187,7 @@ class RAGCapabilityBase[StateT: EvidenceState](AbstractCapability[Any]):
False on a first question, and equally on every question of a host that does
not carry state between runs. Capabilities that need the record to mean
anything across questions read it to refuse rather than act on nothing.
anything across questions refuse when this is False.
"""
async def for_run(self, ctx: RunContext[Any]) -> "RAGCapabilityBase[StateT]":
@ -618,8 +617,7 @@ class RAGCapabilityBase[StateT: EvidenceState](AbstractCapability[Any]):
"""Index the citations, numbering the ones not already registered.
The index is keyed by chunk id and outlives the question, so an id
already registered from another database is refused here rather than
silently keeping the earlier one's content and database.
already registered from another database is refused here.
"""
assert self.state is not None
state = self.state

7
haiku_rag_slim/haiku/rag/cli.py
View file

@ -72,7 +72,8 @@ def create_app(db: Path | None = None, *, covers_set: bool = False) -> "HaikuRAG
Raises:
AmbiguousDatabaseError: multiple databases are configured and this
command works on one, without `--db` or `--db-name` naming which.
command works on one, without `--db` or `--db-name` naming which;
or `--db` and `--db-name` are both given.
"""
from haiku.rag.app import HaikuRAGApp
@ -90,8 +91,8 @@ def resolve_scope(
The CLI decides only what it alone knows: that `--db` and `--db-name` are
the same thing said twice, and whether this command can read more than one.
Everything else — an unknown name, a legacy `uri`, the default location — is
`DatabaseScope.resolve`'s to answer, so there is one table and not two.
Everything else — an unknown name, a `lancedb.uri`, the default location —
is `DatabaseScope.resolve`'s to answer, so there is one table and not two.
"""
from haiku.rag.client.scope import DatabaseScope

38
haiku_rag_slim/haiku/rag/client/__init__.py
View file

@ -151,8 +151,8 @@ class HaikuRAG:
"""
self._configured = config if config is not None else get_config()
# What the caller configured, kept intact: entering derives a
# single-database configuration from it, and asking again has to see the
# same set rather than the answer from last time.
# single-database configuration from it, and every re-entry derives
# from the configured set.
self._config = self._configured
self._requested_db_path = Path(db_path) if db_path is not None else None
if self._requested_db_path is not None and sources is not None:
@ -209,9 +209,8 @@ class HaikuRAG:
None only when a client covering a set is given no name, as for evidence
recorded before databases could be named. A name this client does not
cover raises `UnknownDatabaseError`, decided by `clients_covering` so
that one database answers a wrong name the same way a set does:
provenance naming another database is wrong rather than absent.
cover raises `UnknownDatabaseError`, decided by `clients_covering`: one
database answers a wrong name the same way a set does.
"""
if source is None:
return None if self.covers_multiple else self
@ -275,7 +274,7 @@ class HaikuRAG:
def embedder(self) -> "EmbedderWrapper":
"""The embedder for the databases this client covers.
An embedder is a function of configuration rather than of a database,
An embedder is a function of configuration, not of a database,
and the databases in a selection are required to share one, so a client
covering a set has an unambiguous embedder without opening any of them.
Built on first use and owned by this client, which closes it.
@ -361,9 +360,9 @@ class HaikuRAG:
async def clients_for(self, names: list[str]) -> list["HaikuRAG"]:
"""The clients for these databases, opening any not yet open.
Opening is per query rather than at entry: a set of 25 configured
databases is typically queried a few at a time, and a database nobody
asked for must not be able to fail a query, or be opened for nothing.
Opening is per query: a set of 25 configured databases is typically
queried a few at a time, and a database the query does not cover stays
closed.
The clients returned borrow their databases from this one and are valid
only while it is open. Closing one, or entering it as a context manager,
@ -415,8 +414,7 @@ class HaikuRAG:
) -> "HaikuRAG":
"""A client over a database another session opened and will close.
`lender` is the client that opened it, whose reranker this one borrows
rather than building a second copy of the same model.
`lender` is the client that opened it, whose reranker this one borrows.
"""
client = cls(
session.db_path, config=session.config, read_only=session.read_only
@ -429,14 +427,10 @@ class HaikuRAG:
def _require_one_embedder(self, clients: "list[HaikuRAG]") -> None:
"""Fail when two of these databases were written with different embedders.
Searching a set embeds the query once, so a database written with another
model answers from a different vector space: its candidates are noise, and
rank fusion gives them slots anyway. Only databases searched together have
to agree, so this is a property of the selection rather than of the set.
Drift between a database and the *config* is a separate, softer matter —
the same model served by another stack is spelled differently — which
`SettingsRepository` reports on open.
Only databases searched together have to agree: this is a property of
the selection, not of the set. Drift between a database and the *config*
is a separate, softer matter, which `SettingsRepository` reports on
open.
"""
recorded = [
(client.source, client.store.stored_embedding)
@ -505,8 +499,7 @@ class HaikuRAG:
async def _aclose_cached(self, name: str) -> None:
"""Close a cached_property this client materialized, and discard it.
Discarded rather than left in place so that re-entering the client
builds a fresh one instead of reusing something already closed.
Re-entering the client builds a fresh one.
"""
cached = self.__dict__.pop(name, None)
if cached is not None:
@ -920,7 +913,8 @@ class HaikuRAG:
return []
results = await search(self, query, limit, search_type, filter, include_images)
# A database named in config keeps its name even when it is the only one
# this client covers. Only a legacy single `uri` leaves source unset.
# this client covers. Only an unnamed `lancedb.uri` database leaves
# source unset.
for result in results:
result.source = self.source
return results

4
haiku_rag_slim/haiku/rag/client/scope.py
View file

@ -58,8 +58,8 @@ class DatabaseRef:
class DatabaseScope:
"""The databases an operation covers.
Resolved once, from configuration plus at most one selector, and passed down
rather than re-derived. Never empty.
Resolved once, from configuration plus at most one selector, then passed
down. Never empty.
Nothing here reads the environment: ``HAIKU_RAG_DB`` is the capability entry
point's to honour, and reading it here would extend it to every caller.

16
haiku_rag_slim/haiku/rag/client/search.py
View file

@ -80,8 +80,8 @@ async def search_sources(
"""Search several databases and fuse their results into one ranked list.
Fetch, fuse, truncate, then enrich: enrichment runs on the survivors through
the database each came from, so it costs the same as a single-database search
rather than multiplying by the number searched.
the database each came from, so its cost is that of a single-database
search.
"""
if limit is None:
limit = client._config.search.limit
@ -242,10 +242,9 @@ async def _embed_query(
) -> list[float] | None:
"""The query as a vector, or None when the search needs no vector.
Computed by the caller so that searching several databases embeds once: the
databases in a selection share an embedder, and embedding per database costs
a round trip each on a remote endpoint. `search_type` is the resolved one, so
only a text query ever reaches this as full-text.
The caller computes it once for however many databases the search covers:
the databases in a selection share an embedder. `search_type` is the
resolved one, so only a text query ever reaches this as full-text.
"""
if search_type == "fts":
return None
@ -289,9 +288,8 @@ async def _rank(
async def _attach_picture_data(client: "HaikuRAG", chunks: list[Chunk]) -> None:
"""Attach picture bytes to synthetic picture chunks in-place, so a
multimodal reranker can score the pixels instead of just the chunk's
description text.
"""Attach picture bytes to synthetic picture chunks in-place; a multimodal
reranker scores the pixels.
One query however many documents the candidates span, which matters here
more than anywhere: reranking fetches `limit * 10` candidates.

31
haiku_rag_slim/haiku/rag/client/session.py
View file

@ -30,17 +30,13 @@ logger = logging.getLogger(__name__)
_VACUUM_MIN_INTERVAL_S = 300.0
# Failures whose message names the remedy and never the location, so the failing
# database is named alongside it instead of in place of it.
# Failures whose message names the remedy and never the location. `open()`
# prefixes the failing database's name.
_NAMEABLE_FAILURES = (MigrationRequiredError, ConfigMismatchError, ReadOnlyError)
async def aclose_quietly(closeable: Any, what: str) -> None:
"""Close, reporting failure to the log rather than raising.
Teardown can run while an exception unwinds, so a raising close must
neither mask that exception nor stop a sibling from being closed.
"""
"""Close; a failure is logged, never raised."""
try:
await closeable.aclose()
except Exception:
@ -59,8 +55,8 @@ class SingleDatabaseSession:
it has one. ``source`` is the configured name this database answers to, or
None where nothing names it.
``db_path``, ``config``, ``read_only`` and ``source`` are readable so that a
client borrowing this session can report them as its own.
``db_path``, ``config``, ``read_only`` and ``source`` are readable: a client
borrowing this session reports them as its own.
"""
def __init__(
@ -110,8 +106,7 @@ class SingleDatabaseSession:
self.store.close()
raise
except _NAMEABLE_FAILURES as error:
# These name the remedy and not the database, so the name is added
# rather than substituted.
# The message keeps its remedy and gains the database's name.
if self.source is None:
raise
raise type(error)(f"database {self.source!r}: {error}") from error
@ -152,10 +147,8 @@ class SingleDatabaseSession:
def schedule_vacuum(self) -> None:
"""Schedule a background vacuum, throttled to at most one per
``_VACUUM_MIN_INTERVAL_S``. Sustained writes would otherwise trigger
back-to-back compaction of the blob-bearing documents table. The throttle
only skips the background task — ``_vacuum_dirty`` still marks that a
final vacuum on close is owed."""
``_VACUUM_MIN_INTERVAL_S``. The throttle only skips the background task —
``_vacuum_dirty`` still marks that a final vacuum on close is owed."""
self._vacuum_dirty = True
now = monotonic()
if (
@ -246,8 +239,7 @@ class SingleDatabaseSession:
async def aclose(self) -> None:
"""Drain, release the embedder, and close the connection.
The store owns the embedder, so releasing it belongs here rather than
with whoever happened to hold the session.
The store owns the embedder, so this is where it is released.
"""
await self.drain_vacuum()
await aclose_quietly(self.store.embedder, "embedder")
@ -262,9 +254,8 @@ class FederatedSession:
"""Several databases, read as one. Reads only.
Composes single-database sessions and owns their teardown. They open on first
use rather than at entry: which databases a query covers is a per-query
choice, so a database nobody asked for must neither be opened for nothing nor
be able to fail a query.
use: which databases a query covers is a per-query choice, and a database the
query does not cover stays closed.
"""
def __init__(

4
haiku_rag_slim/haiku/rag/ingester/app.py
View file

@ -77,8 +77,8 @@ class IngesterApp:
from haiku.rag.store.exceptions import AmbiguousDatabaseError
self._config = config
# `--db` names the database directly, and nothing stands in for it: a
# manufactured default would override a configured `lancedb.uri`.
# `--db` is an explicit override; None leaves placement to the
# configuration.
self._scope = DatabaseScope.resolve(config, database_path=db_path)
if self._scope.covers_multiple:
raise AmbiguousDatabaseError(

3
haiku_rag_slim/haiku/rag/inspector/app.py
View file

@ -235,7 +235,8 @@ def run_inspector(
"""Run the inspector TUI.
Args:
db_path: Path to the LanceDB database. If None, uses default from config.
db_path: Path to the LanceDB database, when no scope is given.
scope: The database to inspect, resolved by the caller.
read_only: Whether to open the database in read-only mode.
"""
config = get_config()

7
haiku_rag_slim/haiku/rag/inspector/widgets/info_modal.py
View file

@ -17,11 +17,8 @@ if TYPE_CHECKING:
async def database_lines(client: "HaikuRAG") -> list[str]:
"""What one database reports about itself, without naming its location.
Reported through the connection the client already holds: a second one to the
same database would be a second open for the same statistics.
A failure is reported as a line rather than raised, so one unreachable
database does not cost the report on the others.
Reported through the connection the client already holds. A failure becomes
a line of the report, and the other databases still report.
"""
from haiku.rag.store.engine import ConnectionMode
from haiku.rag.store.info import get_database_stats

7
haiku_rag_slim/haiku/rag/mcp.py
View file

@ -33,8 +33,8 @@ def create_mcp_server(
Args:
db_path: Path to the database file, or None to let `config` place it. A
path overrides a configured `lancedb.uri`, so a URI-backed database
must pass None rather than a local stand-in.
path overrides a configured `lancedb.uri`: for a URI-backed
database, pass None.
config: Configuration to use.
read_only: If True, write tools (add_document_*, delete_document) are not registered.
"""
@ -82,8 +82,7 @@ def _covering(scope: "DatabaseScope", config: AppConfig, read_only: bool) -> Fas
@asynccontextmanager
async def lifespan(_server: FastMCP) -> AsyncIterator[None]:
# Open eagerly so an unopenable database fails startup rather than
# every tool call.
# Opened eagerly: an unopenable database fails at startup.
nonlocal client
await _client()
try:

14
haiku_rag_slim/haiku/rag/sandbox/sandbox.py
View file

@ -135,8 +135,9 @@ class Sandbox:
file callbacks are synchronous and run off that loop while ``feed_run`` is
awaited, so they bridge back to it via ``run_coroutine_threadsafe`` without
deadlocking. When a ``rag`` connection is supplied it is used for every read,
so an analysis run drives a single connection on a single loop; otherwise
each read opens an ephemeral read-only connection.
so an analysis run drives a single connection on a single loop. Otherwise a
scope covering several databases opens a federated client once and holds it
until ``close()``, and a single database is opened per read.
"""
_scope: "DatabaseScope"
@ -188,8 +189,8 @@ class Sandbox:
Internal: the public constructor takes a path and resolves it, which is
its own job. This is for callers that did the resolving, as
``HaikuRAG._covering`` is. It sets the sandbox up directly rather than
through ``__init__``, so the scope it is handed is the only one resolved.
``HaikuRAG._covering`` is. It bypasses ``__init__``: the scope it is
handed is the only one resolved.
"""
sandbox = cls.__new__(cls)
sandbox._configure(scope, config, context, rag, lock)
@ -604,9 +605,8 @@ class Sandbox:
"""Resource limits for the worker session.
Monty spends ``max_duration_secs`` across the session's whole life, and
the session is reused so variables persist between calls. Budget it for
the run rather than for one call, or the first slow call starves every
later one. ``code_timeout`` is enforced per call elsewhere: the read
the session is reused so variables persist between calls: the budget
covers the whole run. ``code_timeout`` is enforced per call elsewhere: the read
deadline in ``_run_on_loop`` bounds a call that reads, and the pool's
``request_timeout`` bounds one that computes.
"""

3
haiku_rag_slim/haiku/rag/store/exceptions.py
View file

@ -41,8 +41,7 @@ class AmbiguousCitationError(Exception):
"""A cited chunk id names a chunk in more than one database.
A citation records the id alone, so nothing downstream can say which
database it came from. Raised rather than resolved: picking one attributes
the answer to a database it may not have come from.
database it came from.
"""

5
haiku_rag_slim/haiku/rag/store/repositories/chunk.py
View file

@ -213,9 +213,8 @@ class ChunkRepository:
limit: Maximum number of results to return.
search_type: "vector", "fts", or "hybrid" (default).
filter: Optional SQL WHERE clause to filter documents before searching chunks.
query_vector: Pre-computed query embedding, used instead of embedding
``query``. Searching several databases embeds once and passes it
to each.
query_vector: Pre-computed query embedding; when supplied, ``query``
is not embedded.
Returns:
List of (chunk, score) tuples ordered by relevance.

7
haiku_rag_slim/haiku/rag/utils.py
View file

@ -394,9 +394,10 @@ async def format_citations_rich(
) -> "list[RenderableType]":
"""Format citations as Rich renderables for terminal display.
Each citation becomes a Panel with a compact header (``[N] Title (URI) — locator``),
a body holding any referenced figures followed by a truncated text preview, and
a dimmed footer that exposes the document and chunk IDs.
Each citation becomes a Panel with a compact header (``[N] Title (URI) —
locator``, with the database name before the locator when ``client`` covers
several), a body holding any referenced figures followed by a truncated text
preview, and a dimmed footer that exposes the document and chunk IDs.
When ``client`` is supplied, picture bytes for ``picture_refs`` are fetched and
rendered inline via ``textual_image``. Without a client, picture refs appear as

19
tests/capabilities/test_capabilities.py
View file

@ -121,8 +121,8 @@ def test_capability_factories_resolve_environment_and_defaults(
class TestACapabilityFollowsTheConfiguredLocation:
"""A capability nobody handed a client opens one for itself, and has to open
the database the configuration places rather than the default directory."""
"""A capability nobody handed a client opens one for itself, at the
database the configuration places."""
def _config(self, tmp_path, uri: str) -> AppConfig:
from haiku.rag.config.models import LanceDBConfig, StorageConfig
@ -133,8 +133,8 @@ class TestACapabilityFollowsTheConfiguredLocation:
)
def test_a_configured_uri_is_left_to_the_client(self, tmp_path):
"""A path overrides a configured location, so manufacturing one would
send the capability to the default directory instead of the bucket."""
"""A path overrides a configured location, so the capability passes
None and the client resolves the configured URI."""
located = tmp_path / "notes.lancedb"
for factory in (create_rag, create_analysis):
[local] = factory(
@ -208,10 +208,8 @@ def test_domain_preamble_is_added_to_capability_instructions(temp_db_path):
def _single_database_client() -> AsyncMock:
"""A stand-in for a client covering one unnamed database.
A bare AsyncMock answers every attribute with a truthy Mock, so
`covers_multiple` would read as a set of databases, `source` would reach a
validated field, and `clients_covering` would return a Mock where the code
iterates clients.
`covers_multiple`, `source` and `clients_covering` answer as one unnamed
database does; a bare AsyncMock answers every attribute with a truthy Mock.
"""
client = AsyncMock()
client.covers_multiple = False
@ -1667,9 +1665,8 @@ async def test_an_answered_question_is_no_longer_in_progress(temp_db_path):
class TestMultipleCollectionsInstructions:
"""The note follows what a run reads, not what the configuration names, so a
run over one collection is instructed exactly as it was before collections
could be named."""
"""The note follows what a run reads, not what the configuration names: a
run over one collection gets the single-collection instructions."""
@staticmethod
def _config(**databases):

27
tests/chat/test_chat_app.py
View file

@ -90,8 +90,8 @@ def test_chat_capabilities_read_the_named_database(tmp_path, monkeypatch):
run_chat(scope=DatabaseScope.resolve(config))
[covering] = chat_app.call_args.kwargs["capabilities"]
# The chat lends its own client, so this scope is the fallback: what matters
# is that it places the named database rather than the whole set.
# The chat lends its own client, so this scope is the fallback: it places
# the named database alone.
[placed] = named.scope.databases
assert placed.db_path == tmp_path / "b.lancedb"
assert named.config.lancedb.databases == {}
@ -156,8 +156,8 @@ def _make_mock_client():
mock_client = AsyncMock()
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
mock_client.__aexit__ = AsyncMock(return_value=None)
# Covers one database: a bare AsyncMock answers `covers_multiple` with a truthy
# Mock, which would send every read down the covering-a-set branch.
# Covers one database; a bare AsyncMock answers `covers_multiple` with a
# truthy Mock.
mock_client.covers_multiple = False
mock_client.source_names = ()
mock_client.source = None
@ -670,8 +670,8 @@ async def test_visual_grounding_uses_the_database_holding_the_citation(tmp_path)
class TestLendingTheClient:
@pytest.mark.asyncio
async def test_mounting_lends_its_client_to_every_capability(self, temp_db_path):
"""Capabilities are built before the client exists, so each reads through
the one the app opened rather than opening its own."""
"""Capabilities are built before the client exists, and each reads
through the one the app opened."""
client = _make_mock_client()
app, _ = _make_app(temp_db_path, client)
@ -1126,8 +1126,8 @@ class TestKeepingSelectionsReachable:
[only] = list(modal.query(DocumentCheckbox))
assert only.doc_id == "sel-0200"
# Awaited rather than posted: the handler reads the database, so
# a single pause need not have flushed it.
# Awaited directly: the handler reads the database, and a
# single pause need not have flushed it.
await modal.on_checkbox_changed(Checkbox.Changed(only, False))
await pilot.pause()
@ -1145,8 +1145,8 @@ class TestKeepingSelectionsReachable:
@pytest.mark.asyncio
async def test_the_results_listing_pages_too(self, temp_db_path: Path):
"""More documents match than one page holds, so the rest are a page
away rather than unreachable."""
"""More documents match than one page holds; the rest are a page
away."""
from textual.widgets import Button
from haiku.rag.chat.widgets.document_filter_modal import (
@ -1211,11 +1211,8 @@ class TestKeepingSelectionsReachable:
async def test_typing_without_submitting_leaves_the_listing_alone(
self, temp_db_path: Path
):
"""The listing is what the last submitted search asked for, and it pages.
Narrowing it as the user types would hide rows from the page the search
landed on while the rest of the matches stayed a page away, so the term
applies on enter and says so until then.
"""The listing is what the last submitted search asked for, and it
pages. The term applies on enter, and the footer says so until then.
"""
from textual.widgets import Button, Input, Static

4
tests/conftest.py
View file

@ -226,7 +226,7 @@ def docling_serve_url() -> str:
def writing(client: "HaikuRAG") -> "SingleDatabaseSession":
"""The database a write implementation works on, from a client holding one.
Write implementations take a session rather than a client, so a set can
Write implementations take a session, never a client, so a set can
never reach them. Tests that call one directly go through here."""
from haiku.rag.client.session import SingleDatabaseSession
@ -240,7 +240,7 @@ def for_path(
"""A scope covering one database at `db_path`.
The application layer takes the databases it works on, already resolved.
Tests that hold a path rather than a scope go through here.
Tests that hold a path and need a scope go through here.
"""
from haiku.rag.client.scope import DatabaseScope
from haiku.rag.config import get_config

13
tests/ingester/test_cli.py
View file

@ -504,9 +504,8 @@ def test_cli_entry_point_exits_on_store_state_errors(monkeypatch, capsys, error)
class TestPlacingTheIngesterDatabase:
"""The ingester writes wherever the configuration places the database, and
resolves that once. A manufactured local default would silently redirect a
remote deployment to the local disk, because a path is an explicit override
of a configured `lancedb.uri`."""
resolves that once. A path is an explicit override of a configured
`lancedb.uri`, so no local default stands in for one."""
@staticmethod
def _app(config: AppConfig, db_path=None):
@ -543,8 +542,8 @@ class TestPlacingTheIngesterDatabase:
assert not scope.covers_multiple
def test_a_set_is_refused_with_a_remedy_this_command_has(self, tmp_path):
"""The client would name `sources=[name]`, a Python argument no CLI user
can pass."""
"""The remedy in the message is `--db PATH`, an argument this command
has; `sources=` is a Python argument no CLI user can pass."""
from haiku.rag.store.exceptions import AmbiguousDatabaseError
config = AppConfig(
@ -560,8 +559,8 @@ class TestPlacingTheIngesterDatabase:
assert "sources=" not in str(raised.value)
def test_several_configured_databases_exit_cleanly(self, tmp_path, monkeypatch):
"""No selector names one of a set, so the CLI reports it rather than
printing a traceback."""
"""No selector names one of a set, so the CLI exits with the message
and no traceback."""
config_file = tmp_path / "haiku.rag.yaml"
config_file.write_text(
"lancedb:\n databases:\n"

4
tests/ingester/test_run_batch.py
View file

@ -103,8 +103,8 @@ def use_client(monkeypatch):
async def _cm(*_, **__):
yield client
# The app opens the databases its scope covers, so the double stands in
# for `_covering` rather than the public constructor.
# The app opens the databases its scope covers: the double stands in
# for `_covering`.
stub = MagicMock()
stub._covering = lambda *a, **k: _cm()
monkeypatch.setattr("haiku.rag.client.HaikuRAG", stub)

3
tests/multi_db/helpers.py
View file

@ -58,8 +58,7 @@ async def _restore_embedder(config, name, *, provider=None, model_name=None):
async def _seed_expandable(config, name, sentences):
"""One document whose chunk covers a single item, so expansion has
neighbours to pull in and rebuilds the result rather than passing it
through."""
neighbours to pull in and rebuilds the result."""
dim = get_config().embeddings.model.vector_dim
doc = DoclingDocument(name=name)
for sentence in sentences:

4
tests/multi_db/test_capabilities.py
View file

@ -226,8 +226,8 @@ class TestNamingDatabasesBeforeTheModelRuns:
@pytest.mark.asyncio
async def test_checking_a_name_opens_nothing(self, tmp_path):
"""Opening to check would open every database on an unscoped question,
and let one nobody asked about fail a run before any search."""
"""Validating a name reads the configured set; no database opens for
it."""
config = _config(tmp_path, ["alpha", "beta"])
await _seed(config, "alpha", ["alpha document about cats"])
await _seed(config, "beta", ["beta document about cats"])

15
tests/multi_db/test_citations.py
View file

@ -40,8 +40,8 @@ class TestSharedChunkIds:
]
def test_a_shared_id_cannot_be_cited(self):
"""A citation records the id alone, so resolving one held by two
databases would attribute the answer to whichever came last."""
"""A citation records the id alone, so one held by two databases
cannot say which grounded the answer."""
results = [
SearchResult(
content="alpha body",
@ -66,8 +66,8 @@ class TestSharedChunkIds:
def test_a_repeated_id_from_one_database_still_collapses(self):
"""One database cannot hold two chunks under one id, so seeing it twice
is the same chunk seen twice, and it resolves rather than raising. Which
copy supplies the content is `resolve_citations`' own rule, pinned in
is the same chunk seen twice, and it resolves. Which copy supplies the
content is `resolve_citations`' own rule, pinned in
`tests/store/test_citation.py`."""
results = [
SearchResult(
@ -122,8 +122,7 @@ class TestSharedChunkIds:
@pytest.mark.asyncio
async def test_an_unsearched_shared_id_is_refused_by_the_fallback(self, tmp_path):
"""The direct lookup is the only place a collision shows for an id no
search returned, so it has to ask every database rather than take the
first that answers."""
search returned, so it asks every database."""
import shutil
config = _config(tmp_path, ["alpha", "beta"])
@ -200,8 +199,8 @@ class TestSharedChunkIds:
async def test_cite_refuses_an_id_already_cited_from_another_database(
self, tmp_path
):
"""The citation index outlives the question, so the collision can arrive
a turn later than the search that would have shown it."""
"""The citation index outlives the question, so the collision can
arrive a turn after the search."""
capability = create_capability(
config=_config(tmp_path, ["alpha", "beta"]), defer_loading=False
)

3
tests/multi_db/test_documents.py
View file

@ -155,8 +155,7 @@ class TestLookupByIdentifier:
self, tmp_path
):
"""A database copied from another holds the same ids. A read has an
answer wherever it finds one, and which one it is has to be the
configured order rather than whichever replied first."""
answer wherever it finds one, and the configured order says which."""
import shutil
config = _config(tmp_path, ["alpha", "beta"])

30
tests/multi_db/test_lifecycle.py
View file

@ -64,9 +64,9 @@ class TestOpeningDatabases:
@pytest.mark.asyncio
async def test_a_cancelled_open_does_not_leak_the_ones_that_worked(self, tmp_path):
"""Cancellation discards the fan-out's results rather than returning them,
so a database that opened while a sibling was still pending is reachable
only because the opener recorded it."""
"""Cancellation discards the fan-out's results, so a database that
opened while a sibling was still pending is reachable only through the
opener's record."""
config = _config(tmp_path, ["alpha", "beta"])
await _seed(config, "alpha", ["alpha document about cats"])
await _seed(config, "beta", ["beta document about cats"])
@ -145,7 +145,8 @@ class TestOpeningDatabases:
@pytest.mark.asyncio
async def test_a_database_named_twice_is_opened_once(self, tmp_path):
"""Fusion would count a repeated database as two rank lists."""
"""Fusion counts rank lists per database, so a repeated name
contributes one."""
config = _config(tmp_path, ["alpha", "beta"])
await _seed(config, "alpha", ["alpha document about cats"])
await _seed(config, "beta", ["beta document about cats"])
@ -200,8 +201,8 @@ class TestReportingWhereADatabaseIs:
class TestClosingASet:
@pytest.mark.asyncio
async def test_every_database_opened_is_released(self, tmp_path):
"""A covered database owns an embedder and may owe a vacuum. Closing only
its connection would leave both behind."""
"""A covered database owns an embedder and may owe a vacuum; closing
releases both."""
config = _config(tmp_path, ["alpha", "beta"])
await _seed(config, "alpha", ["alpha one"])
await _seed(config, "beta", ["beta one"])
@ -257,9 +258,9 @@ class TestBorrowedDatabases:
@pytest.mark.asyncio
async def test_entering_a_borrowed_client_reuses_its_database(self, tmp_path):
"""`async with` on a borrowed client is a plausible thing to write.
Opening a second session would leak it, since teardown declines to close
what this client did not open."""
"""`async with` on a borrowed client is a plausible thing to write. It
reuses the borrowed session: teardown declines to close what this client
did not open."""
config = _config(tmp_path, ["alpha", "beta"])
await _seed(config, "alpha", ["alpha document about cats"])
@ -375,8 +376,8 @@ class TestReleasingAClient:
class TestSharingTheReranker:
@pytest.mark.asyncio
async def test_the_set_builds_and_closes_one_reranker(self, tmp_path, monkeypatch):
"""A local reranker loads model weights, so one per database in a set
would load the same weights that many times."""
"""A local reranker loads model weights; the set builds one and shares
it."""
import haiku.rag.client as client_module
built: list[object] = []
@ -475,8 +476,8 @@ class TestFailureNaming:
assert caught.value.__cause__ is None
@pytest.mark.asyncio
async def test_a_legacy_uri_client_keeps_its_error(self, tmp_path):
"""Nothing named it, so there is no name to report instead."""
async def test_an_unnamed_database_keeps_its_error(self, tmp_path):
"""Nothing named it, so there is no name to report."""
with pytest.raises(FileNotFoundError):
async with HaikuRAG(tmp_path / "nope.lancedb"):
pass
@ -645,8 +646,7 @@ class TestDatabaseIndependentWork:
@pytest.mark.asyncio
async def test_re_entering_a_set_builds_a_fresh_embedder(self, tmp_path):
"""Teardown closes the embedder, so keeping it would hand the next
context one that is already closed."""
"""Teardown closes the embedder; re-entry builds a fresh one."""
config = _config(tmp_path, ["alpha", "beta"])
await _seed(config, "alpha", ["alpha one"])

13
tests/multi_db/test_scope.py
View file

@ -109,8 +109,7 @@ class TestOneConfiguredLocation:
@pytest.mark.asyncio
async def test_a_local_uri_that_does_not_exist_is_refused(self, tmp_path):
"""A mistyped path fails instead of quietly becoming an empty database,
which is what a value carrying a scheme would do."""
"""A schemeless location is a local path and must exist."""
config = self._config(tmp_path / "typo.lancedb")
with pytest.raises(FileNotFoundError):
@ -253,14 +252,14 @@ class TestPlacingADatabase:
# A KeyError, so a caller treating selection as a lookup still catches it.
assert issubclass(UnknownDatabaseError, KeyError)
# ...but the message reads as a sentence rather than a missing key.
# ...but the message reads as a sentence, not as a missing key.
assert str(UnknownDatabaseError("unknown database 'typo'")) == (
"unknown database 'typo'"
)
def test_a_path_and_sources_cannot_both_choose(self, tmp_path):
"""`sources` used to be ignored beside a path, so selecting a database
that is not the one at the path opened the path anyway."""
"""A path names one database and `sources` names others; together they
are refused, whatever the selection."""
config = _config(tmp_path, ["alpha", "beta"])
for sources in ([], ["alpha"], ["nope"]):
@ -269,8 +268,8 @@ class TestPlacingADatabase:
@pytest.mark.asyncio
async def test_one_database_refuses_a_name_it_does_not_cover(self, tmp_path):
"""Answering with itself would hand back the wrong database's reader for
a citation that named another."""
"""A citation naming another database must not get this database's
reader."""
config = _config(tmp_path, ["alpha", "beta"])
await _seed(config, "alpha", ["alpha one"])
await _seed(config, "beta", ["beta one"])

20
tests/multi_db/test_search.py
View file

@ -110,8 +110,8 @@ class TestOneQueryVector:
class TestOneEmbedderAcrossTheSet:
"""A set is searched with one query vector, so a database written with
another model would answer from a different space."""
"""A set is searched with one query vector, so the databases in a
selection must share an embedder."""
@pytest.mark.asyncio
async def test_disagreeing_databases_cannot_be_searched_together(self, tmp_path):
@ -403,8 +403,8 @@ class TestOneReranker:
@pytest.mark.asyncio
async def test_the_reranker_is_closed_once(self, tmp_path, monkeypatch):
"""Handing the same object to every database and letting each close it
would close it N times, and the federator not at all."""
"""The federator owns the reranker: it hands the same object to every
database and closes it once."""
closes = []
class CountingReranker(StubReranker):
@ -431,8 +431,8 @@ class TestNarrowingToOneDatabase:
@pytest.mark.asyncio
async def test_narrowing_keeps_the_database_s_own_scores(self, tmp_path):
"""RRF scores position, so fusing one ranking would report 1/(60+rank)
where the database reported a hybrid score."""
"""RRF scores position; a selection of one keeps the database's own
hybrid scores."""
config = _config(tmp_path, ["alpha", "beta"])
await _seed(config, "alpha", ["alpha document about cats", "alpha on dogs"])
await _seed(config, "beta", ["beta document about cats"])
@ -453,7 +453,7 @@ class TestNarrowingToOneDatabase:
self, tmp_path, monkeypatch
):
"""One database embeds inside the repository, which returns early when
the filter matches no document. Fusing would have embedded first."""
the filter matches no document."""
from haiku.rag.embeddings import EmbedderWrapper
config = _config(tmp_path, ["alpha", "beta"])
@ -486,8 +486,7 @@ class TestReciprocalRankFusion:
def _lopsided(self, count: int) -> list[list[tuple[Chunk, float]]]:
"""Every native score in the first database beats every one in the
second, so anything ranking by score rather than position puts all of
one before any of the other."""
second, so score order and position order disagree."""
return [self._ranked("a", count, 0.9), self._ranked("b", count, 0.2)]
async def _fuse_over(self, tmp_path, per_source, limit):
@ -640,8 +639,7 @@ class TestOneNamedDatabase:
@pytest.mark.asyncio
async def test_selecting_nothing_means_the_same_with_one_database(self, tmp_path):
"""`sources=[]` selects nothing whether one database is configured or
several, rather than raising on one path and returning nothing on the
other."""
several."""
config = _config(tmp_path, ["alpha"])
await _seed(config, "alpha", ["alpha document about cats"])

14
tests/sandbox/test_sandbox_multi_db.py
View file

@ -47,13 +47,13 @@ class TestSerializingTheConnection:
@pytest.mark.asyncio
async def test_an_owner_is_not(self, tmp_path):
"""Serializing owner reads would queue every database's file read behind
the capability's searches, to guard state none of them touch."""
"""The lock guards the lent session's state, which owner reads do not
touch: they take no lock."""
import asyncio
class Trap(asyncio.Lock):
"""Refuses rather than waits: holding a real lock would wedge the
suite on a regression instead of failing it."""
"""Raises on acquire, failing the test at the serialization
point."""
async def acquire(self):
raise AssertionError("serialized a read on an owner's own session")
@ -282,7 +282,7 @@ class TestExecutingAcrossDatabases:
class TestSelectionOnOneDatabase:
"""A client covering a single named database answers a selection the same way
a search does, or the sandbox would mount what a search would refuse."""
a search does: the sandbox mounts what a search reaches."""
@pytest.mark.asyncio
async def test_selecting_no_database_mounts_nothing(self, tmp_path):
@ -318,8 +318,8 @@ class TestSelectionOnOneDatabase:
class TestCopiedDatabases:
@pytest.mark.asyncio
async def test_a_document_in_two_databases_is_refused(self, tmp_path):
"""Ids are unique per database, not across a copy of one: two documents
would claim one path and the last would answer for both."""
"""Ids are unique per database, not across a copy of one, so one path
cannot serve two documents."""
config = _config(tmp_path, ["alpha", "clone"])
await _seed(config, "alpha", ["alpha document about cats"])
shutil.rmtree(tmp_path / "clone.lancedb", ignore_errors=True)

7
tests/store/test_engine.py
View file

@ -9,8 +9,7 @@ class TestStoredSettings:
async def test_a_new_database_carries_the_version_it_was_created_with(
self, temp_db_path
):
"""Creating writes the settings row, so the store has to report it
rather than the emptiness it opened on."""
"""Creating writes the settings row, and the store reports it."""
from importlib import metadata
async with Store(temp_db_path, create=True) as store:
@ -20,8 +19,8 @@ class TestStoredSettings:
@pytest.mark.asyncio
async def test_an_existing_database_carries_its_stored_settings(self, temp_db_path):
"""Read once on open: reporting on a database reads them from here
instead of querying the settings table again."""
"""Read once on open: reporting on a database reads them from
here."""
async with Store(temp_db_path, create=True) as store:
written = store.stored_settings

5
tests/test_app.py
View file

@ -428,9 +428,8 @@ def test_show_settings_hides_secrets(tmp_path):
def test_show_settings_renders_the_shape_a_config_file_has(tmp_path):
"""Nesting is indented rather than flattened into one dict repr per block,
and a path is its string rather than its Python repr, so what is read here
is what would be written into `haiku.rag.yaml`."""
"""Nesting is indented and a path is its string: what is read here is what
`haiku.rag.yaml` holds."""
import yaml
config = AppConfig(

19
tests/test_cli.py
View file

@ -141,8 +141,8 @@ class TestOneDatabaseCommands:
assert "bucket" not in str(raised.value)
def test_a_configured_set_of_one_needs_no_choosing(self, monkeypatch):
"""Nothing is ambiguous about a set with one database in it, and it keeps
its name rather than being refused."""
"""Nothing is ambiguous about a set with one database in it, and it
keeps its name."""
self._install(monkeypatch, alpha="/db/a.lancedb")
assert resolve_scope(None).names == ("alpha",)
@ -189,8 +189,8 @@ class TestSelectingADatabaseByName:
set_config(AppConfig(lancedb=LanceDBConfig(databases=databases)))
def test_a_named_database_is_passed_on_by_name(self, monkeypatch):
"""Not resolved to a path: the name is what results and citations carry,
and rewriting the configuration is what used to lose it."""
"""Not resolved to a path: the name is what results and citations
carry."""
self._install(monkeypatch, papers="s3://bucket/prefix/papers.lancedb")
monkeypatch.setattr("haiku.rag.cli._db_name", "papers")
@ -271,8 +271,8 @@ class TestSelectingADatabaseByName:
def test_a_selection_does_not_outlive_its_invocation_in_process(
self, tmp_path, monkeypatch
):
"""The selection is per invocation, so a second one starts from a
freshly loaded configuration rather than inheriting the first."""
"""The selection is per invocation: a second one starts from a freshly
loaded configuration."""
import haiku.rag.cli as cli_module
import haiku.rag.config as config_module
@ -419,8 +419,7 @@ class TestConfiguredLocalUri:
assert self._reports(result, located)
def test_a_missing_configured_path_is_refused(self, tmp_path, monkeypatch):
"""A schemeless value is a local path, so a typo fails instead of
becoming a new empty database."""
"""A schemeless value is a local path and must exist."""
self._fresh(monkeypatch)
located = tmp_path / "typo.lancedb"
config_file = self._config_file(tmp_path, located)
@ -726,8 +725,8 @@ class TestAskAnalyzeImageOption:
class TestChatCoversTheSet:
"""Chat is a read verb: it answers with the same capabilities `ask` uses, so
it covers the configured set rather than demanding one database."""
"""Chat is a read verb: it answers with the same capabilities `ask` uses,
and covers the configured set."""
@staticmethod
def _config_file(tmp_path):

12
tests/test_database_scope.py
View file

@ -28,7 +28,7 @@ class TestResolution:
def test_a_path_names_one_unnamed_database(self):
"""A path says which database, not what it is called, even where the
configuration would have named it."""
configuration names one."""
config = _config(databases={"alpha": "/data/alpha.lancedb"})
scope = DatabaseScope.resolve(config, database_path=Path("/data/other.lancedb"))
@ -108,8 +108,8 @@ class TestResolution:
assert scope.databases == (DatabaseRef.at(tmp_path / "haiku.rag.lancedb"),)
def test_the_environment_is_not_consulted(self, monkeypatch, tmp_path):
"""HAIKU_RAG_DB is honoured by the capability entry point alone. Reading it
here would change what every other caller opens."""
"""HAIKU_RAG_DB is honoured by the capability entry point alone;
resolution never reads the environment."""
monkeypatch.setenv("HAIKU_RAG_DB", "/data/from-the-environment.lancedb")
config = _config(databases={"alpha": "/data/alpha.lancedb"})
@ -118,8 +118,7 @@ class TestResolution:
assert scope.names == ("alpha",)
def test_a_path_is_never_reinterpreted_as_a_uri(self):
"""A caller naming a path means that path. Sending it back through the
configured-location rules would let a scheme turn it into a URI."""
"""A caller naming a path means that path, whatever scheme it carries."""
scope = DatabaseScope.resolve(
_config(), database_path="s3://bucket/looks-like-a-uri.lancedb"
)
@ -139,8 +138,7 @@ class TestResolution:
assert ref.db_path is None
def test_a_database_is_a_uri_or_a_path(self):
"""Both would silently ignore the path; neither fails later, when the
connection is derived and there is nothing to open.
"""A ref holding both, or neither, is refused at construction.
The message names what it was given: this is a programming error raised
in the caller's own process, not one an operator or a model ever sees.

9
tests/test_info.py
View file

@ -167,8 +167,7 @@ async def test_app_info_with_vector_index(temp_db_path, capsys):
@pytest.mark.asyncio
async def test_app_info_opens_a_named_remote_database(tmp_path):
"""A database named in `lancedb.databases` can sit behind a URI while the
configuration's own `uri` is empty. Passing that configuration on would open
the local path that only stands in for it."""
configuration's own `uri` is empty; info derives and opens the URI."""
from haiku.rag.client.scope import DatabaseScope
config = AppConfig(
@ -195,8 +194,8 @@ async def test_app_info_opens_a_named_remote_database(tmp_path):
async def test_app_doctor_opens_a_named_remote_database():
"""`run_doctor` connects with the configuration it is handed, so it needs the
one derived for the database rather than the one naming the set."""
"""`run_doctor` connects with the configuration it is handed: the one
derived for the database, not the one naming the set."""
from haiku.rag.client.scope import DatabaseScope
config = AppConfig(
@ -385,7 +384,7 @@ async def test_app_init_skips_exists_check_for_remote(tmp_path):
covering.__aenter__ = AsyncMock(return_value=mock_client)
covering.__aexit__ = AsyncMock(return_value=False)
await app.init()
# Opened to create, rather than returning early on a missing local path.
# A missing local path is opened to create, not returned early on.
mock_client_cls._covering.assert_called_once()

8
tests/test_inspector.py
View file

@ -369,8 +369,8 @@ class TestReportingReusesTheConnection:
async def test_statistics_come_from_the_open_connection(
self, tmp_path, monkeypatch
):
"""The client already holds a connection to the database being reported,
so opening a second one would be an open for the same statistics."""
"""Statistics are read through the connection the client already
holds."""
from haiku.rag.inspector.widgets.info_modal import database_lines
from haiku.rag.store.engine import ConnectionMode
@ -403,8 +403,8 @@ class TestReportingReusesTheConnection:
async def test_settings_come_from_the_store_that_parsed_them(
self, tmp_path, monkeypatch
):
"""The store read and parsed the settings blob on open, so reporting
reads it from there instead of querying the settings table again."""
"""The store read and parsed the settings blob on open, and reporting
reads it from there."""
from haiku.rag.inspector.widgets.info_modal import database_lines
from haiku.rag.store.engine import ConnectionMode

9
tests/test_mcp.py
View file

@ -608,9 +608,8 @@ class TestMCPClientLifetime:
async def test_the_scope_decides_the_database_and_names_its_results(
self, mcp_db, tmp_path
):
"""The scope is the selection, so the server reads the one database it
names and results carry that name. The configuration alone would place
every database it configures."""
"""The scope is the selection: the server reads the one database it
names, and results carry that name."""
from haiku.rag.client.scope import DatabaseScope
from haiku.rag.config.models import AppConfig, LanceDBConfig
@ -670,8 +669,8 @@ class TestMCPClientLifetime:
async def test_the_command_hands_the_server_its_resolved_database(
self, monkeypatch
):
"""A path would override a configured URI, and a derived configuration
would lose the name, so `run_mcp` passes neither."""
"""`run_mcp` passes the resolved scope, not a path and not a derived
configuration: the scope keeps both the URI and the name."""
from haiku.rag.app import HaikuRAGApp
from haiku.rag.client.scope import DatabaseScope
from haiku.rag.config.models import AppConfig, LanceDBConfig

5
tests/test_s3_integration.py
View file

@ -68,9 +68,8 @@ def config():
def _remote_scope(config: AppConfig) -> DatabaseScope:
"""The configured S3 database.
These tests name no path, so the scope must resolve to the URI. A precedence
change that let a path win would otherwise move them to the local disk and
leave them passing against nothing.
These tests name no path, and the assertion pins that the scope resolved to
the URI.
"""
scope = DatabaseScope.resolve(config)
[ref] = scope.databases