danvics/haiku.rag
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

State what the code does, not what it replaced

Browse source 
Comments and docstrings across the branch narrated rejected
alternatives, consequences and history; each now states the current
contract. Renames test_a_legacy_uri_client_keeps_its_error to
test_an_unnamed_database_keeps_its_error. Documents the Sandbox
connection paths, the citation header's database segment, both
AmbiguousDatabaseError conditions on create_app, and run_inspector's
scope parameter. Doc paragraphs added by the branch in python.md,
storage.md and cli.md are one physical line each.
This commit is contained in:
Yiorgis Gozadinos 2026-08-28 15:13:52 +03:00
parent d0df382ce8
commit 09a7076b7e
No known key found for this signature in database
43 changed files with 225 additions and 368 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
docs/cli.md
View file

@ -24,11 +24,7 @@ The `haiku-rag` CLI provides complete document management functionality.
haiku-rag add -h haiku-rag add -h
``` ```
With `lancedb.databases` configured, `search`, `ask`, `analyze`, and `chat` With `lancedb.databases` configured, `search`, `ask`, `analyze`, and `chat` use the full set by default. Select one database for other commands with `--db-name` or `--db`. `settings`, `init-config`, and `download-models` do not open a database. See [Multiple Databases](configuration/storage.md#multiple-databases).
use the full set by default. Select one database for other commands with
`--db-name` or `--db`. `settings`, `init-config`, and `download-models` do
not open a database. See
[Multiple Databases](configuration/storage.md#multiple-databases).
## Document Management ## Document Management

133
docs/configuration/storage.md
View file

@ -68,9 +68,7 @@ This is an upstream limitation rather than a `haiku.rag` setting. Compaction bou
### Changing the Default Database Path ### Changing the Default Database Path
`storage.data_dir` holds the default database, always called `storage.data_dir` holds the default database, always called `haiku.rag.lancedb`. To put the database somewhere else for every command, give `lancedb.uri` a local path:
`haiku.rag.lancedb`. To put the database somewhere else for every command, give
`lancedb.uri` a local path:
```yaml ```yaml
lancedb: lancedb:
@ -79,15 +77,9 @@ lancedb:
An explicit `--db PATH` overrides `lancedb.uri` for that invocation. An explicit `--db PATH` overrides `lancedb.uri` for that invocation.
This places one database without naming it. Its `source` is `None` in search This places one database without naming it. Its `source` is `None` in search results, citations and documents, since only [`lancedb.databases`](#multiple-databases) assigns the names that carry provenance. A path here changes where the database lives, not what it is called.
results, citations and documents, since only [`lancedb.databases`](#multiple-databases)
assigns the names that carry provenance. A path here changes where the database
lives, not what it is called.
A value with no scheme is a local path wherever it is configured, so A value with no scheme is a local path wherever it is configured, so `haiku-rag init` creates it and every command that opens an existing database requires it to exist. A mistyped path fails rather than becoming a new empty database.
`haiku-rag init` creates it and every command that opens an existing database
requires it to exist. A mistyped path fails rather than becoming a new empty
database.
## Database Creation ## Database Creation
@ -202,8 +194,7 @@ Each process picks up its own credentials from the AWS default chain (env vars,
## Multiple Databases ## Multiple Databases
Use `lancedb.databases` to name local or remote databases that should be searched Use `lancedb.databases` to name local or remote databases that should be searched together:
together:
```yaml ```yaml
lancedb: lancedb:
@ -213,119 +204,62 @@ lancedb:
notes: /data/notes.lancedb notes: /data/notes.lancedb
``` ```
A location can be a URI or local path. `databases` and `uri` are mutually A location can be a URI or local path. `databases` and `uri` are mutually exclusive.
exclusive.
Results, documents, and citations use the configured name as `source`. An Results, documents, and citations use the configured name as `source`. An unavailable configured database raises `SourceUnavailableError`, which names the database and not its location, so a location never travels in an error a consumer might render or log. A migration, configuration or read-only failure keeps its own type, with the database named in the message. Commands that report on a database, such as `info`, still show where it is.
unavailable configured database raises `SourceUnavailableError`, which names the
database and not its location, so a location never travels in an error a consumer
might render or log. A migration, configuration or read-only failure keeps its
own type, with the database named in the message. Commands that report on a
database, such as `info`, still show where it is.
Searches spanning multiple databases identify each result with a model-facing Searches spanning multiple databases identify each result with a model-facing `Collection:` line. Searches over one database omit it. Structured `source` fields on results, documents, citations, and analysis dictionaries are unchanged.
`Collection:` line. Searches over one database omit it. Structured `source`
fields on results, documents, citations, and analysis dictionaries are
unchanged.
Embedding compatibility is checked against two different things. Embedding compatibility is checked against two different things.
On open, each database is compared with the current configuration. A dimension On open, each database is compared with the current configuration. A dimension mismatch raises `ConfigMismatchError`. A provider or model-name mismatch at the same dimension warns in read-only mode and raises in writable mode.
mismatch raises `ConfigMismatchError`. A provider or model-name mismatch at the
same dimension warns in read-only mode and raises in writable mode.
Across a selection, the databases are compared with each other. Vector and Across a selection, the databases are compared with each other. Vector and hybrid search embed the query once, so every database answering it must record the same provider, model, and dimension. A disagreement raises `ConfigMismatchError` in read-only mode as well. Only the databases searched together have to agree, and full-text search embeds nothing, so it is unaffected.
hybrid search embed the query once, so every database answering it must record
the same provider, model, and dimension. A disagreement raises
`ConfigMismatchError` in read-only mode as well. Only the databases searched
together have to agree, and full-text search embeds nothing, so it is
unaffected.
### Search and Provenance ### Search and Provenance
`search`, `ask`, and `analyze` use the full set by default. Pass `sources` to `search`, `ask`, and `analyze` use the full set by default. Pass `sources` to select a subset:
select a subset:
```python ```python
results = await client.search("query") # every database results = await client.search("query") # every database
results = await client.search("query", sources=["papers"]) # one of them results = await client.search("query", sources=["papers"]) # one of them
``` ```
Candidates are combined into one ranked list with the configured reranker, or Candidates are combined into one ranked list with the configured reranker, or with reciprocal rank fusion when reranking is disabled. `SearchResult.source`, `Citation.source`, and `Document.source` contain the configured database name. The name is retained when a client covers only one named database. Databases configured through `lancedb.uri` are unnamed, so their `source` is `None`.
with reciprocal rank fusion when reranking is disabled. `SearchResult.source`,
`Citation.source`, and `Document.source` contain the configured database name.
The name is retained when a client covers only one named database. Databases
configured through `lancedb.uri` are unnamed, so their `source` is `None`.
The CLI labels results and citations only when the operation spans multiple The CLI labels results and citations only when the operation spans multiple databases. A command already narrowed with `--db-name` does not repeat the name on every result.
databases. A command already narrowed with `--db-name` does not repeat the name
on every result.
#### Duplicate IDs #### Duplicate IDs
IDs are unique within a database, not across databases. Copies of a database IDs are unique within a database, not across databases. Copies of a database therefore retain the same IDs.
therefore retain the same IDs.
Citation ambiguity is evaluated against evidence available to the run. A cited Citation ambiguity is evaluated against evidence available to the run. A cited chunk ID is rejected with `AmbiguousCitationError` if search returned it from multiple databases, or it was previously cited from another database. If only one retrieved result has the ID, that result is cited. For an ID absent from search results, the fallback checks every selected database and rejects multiple holders. A shared ID that nothing cites is ignored.
chunk ID is rejected with `AmbiguousCitationError` if search returned it from
multiple databases, or it was previously cited from another database. If only
one retrieved result has the ID, that result is cited. For an ID absent from
search results, the fallback checks every selected database and rejects
multiple holders. A shared ID that nothing cites is ignored.
`get_document_by_id`, `get_chunk_by_id` and `get_picture_bytes` take an optional `get_document_by_id`, `get_chunk_by_id` and `get_picture_bytes` take an optional `source`, and ask that database alone. A name the client does not cover raises `UnknownDatabaseError`. Without one, the document and chunk lookups ask every covered database and answer from the first that holds the ID; `get_picture_bytes` requires one whenever the client covers a set.
`source`, and ask that database alone. A name the client does not cover raises
`UnknownDatabaseError`. Without one, the document and chunk lookups ask every
covered database and answer from the first that holds the ID; `get_picture_bytes`
requires one whenever the client covers a set.
The analysis sandbox rejects shared document IDs because its mount path is The analysis sandbox rejects shared document IDs because its mount path is `/documents/{id}/`.
`/documents/{id}/`.
The chat document filter selects by document and database: the search is narrowed to the databases the selection names, and the ID filter applies within them. An ID that copies share still matches in every selected database that holds it. The chat document filter selects by document and database: the search is narrowed to the databases the selection names, and the ID filter applies within them. An ID that copies share still matches in every selected database that holds it.
#### Ranking #### Ranking
Reciprocal rank fusion compares positions rather than scores, so each database Reciprocal rank fusion compares positions rather than scores, so each database contributes top-ranked results even when another database has stronger matches. A reranker scores the combined candidate set directly, which has been measured to help aggregate retrieval and to hurt attribution between near-identical documents.
contributes top-ranked results even when another database has stronger matches. A
reranker scores the combined candidate set directly, which has been measured to
help aggregate retrieval and to hurt attribution between near-identical
documents.
Aggregate retrieval is stronger with a reranker. In a 3,045-query evaluation over Aggregate retrieval is stronger with a reranker. In a 3,045-query evaluation over a corpus split across three databases, reranking produced retrieval MAP 0.9914, compared with 0.9918 for the same corpus in one database. Without a reranker, MAP was 0.6044, compared with 0.9798 in one database. Reranking cost grows with the number of databases because each contributes candidates.
a corpus split across three databases, reranking produced retrieval MAP 0.9914,
compared with 0.9918 for the same corpus in one database. Without a reranker, MAP
was 0.6044, compared with 0.9798 in one database. Reranking cost grows with the
number of databases because each contributes candidates.
A reranker scores the combined candidates with no notion of which database each A reranker scores the combined candidates with no notion of which database each came from, so on near-identical text it can pick the wrong database's chunk, where fusion keeps them apart because each database contributes its own top-ranked result. In two nine-case acceptance runs over a synthetic corpus holding one station in two databases under near-identical names, attribution was weaker with reranking: citing the right database succeeded 5 of 9 and 6 of 9 times with a reranker, against 8 of 9 and 9 of 9 without.
came from, so on near-identical text it can pick the wrong database's chunk,
where fusion keeps them apart because each database contributes its own
top-ranked result. In two nine-case acceptance runs over a synthetic corpus
holding one station in two databases under near-identical names, attribution was
weaker with reranking: citing the right database succeeded 5 of 9 and 6 of 9
times with a reranker, against 8 of 9 and 9 of 9 without.
Configure a reranker where retrieval breadth matters, and measure it where Configure a reranker where retrieval breadth matters, and measure it where answers have to attribute between documents that read alike.
answers have to attribute between documents that read alike.
Without a reranker, consider increasing `search.limit` with the number of Without a reranker, consider increasing `search.limit` with the number of databases. With three complete rankings and a limit of 5, a database may contribute only one or two results. A higher limit also sends more results to the caller and model.
databases. With three complete rankings and a limit of 5, a database may
contribute only one or two results. A higher limit also sends more results to
the caller and model.
Image queries are vector-only and skip the reranker: there is no query text to Image queries are vector-only and skip the reranker: there is no query text to score a document against, so candidates keep their vector ranking and fusion ranks by position.
score a document against, so candidates keep their vector ranking and fusion
ranks by position.
If a selected database is unavailable, the operation fails with If a selected database is unavailable, the operation fails with `SourceUnavailableError`, which names that database.
`SourceUnavailableError`, which names that database.
### Python Operations ### Python Operations
Creating, writing, rebuilding, and vacuuming require one database. Calling these Creating, writing, rebuilding, and vacuuming require one database. Calling these operations on a client that covers multiple raises `AmbiguousDatabaseError`. Select one at creation time or obtain a single-database client:
operations on a client that covers multiple raises `AmbiguousDatabaseError`.
Select one at creation time or obtain a single-database client:
```python ```python
async with HaikuRAG(config=config, create=True, sources=["papers"]) as papers: async with HaikuRAG(config=config, create=True, sources=["papers"]) as papers:
@ -335,21 +269,15 @@ async with HaikuRAG(config=config) as client:
papers = (await client.clients_for(["papers"]))[0] papers = (await client.clients_for(["papers"]))[0]
``` ```
Conversion, chunking, and title generation do not access a database and remain Conversion, chunking, and title generation do not access a database and remain available on a multi-database client.
available on a multi-database client.
### CLI Commands ### CLI Commands
Commands use database sets as follows: Commands use database sets as follows:
- **Set-capable**: `search`, `ask`, `analyze`, and `chat` use the full - **Set-capable**: `search`, `ask`, `analyze`, and `chat` use the full configured set, or the single database selected by `--db-name`.
configured set, or the single database selected by `--db-name`. - **Config-only**: `settings`, `init-config`, and `download-models` do not open a database.
- **Config-only**: `settings`, `init-config`, and `download-models` do not open - **Single-database**: everything else — document writes, `rebuild`, `vacuum`, `migrate`, `init`, `info`, `history`, `tag`, `doctor`, `list`, `inspect`, `visualize`, and `mcp` — works on one database, selected with the global `--db-name` option.
a database.
- **Single-database**: everything else — document writes, `rebuild`, `vacuum`,
`migrate`, `init`, `info`, `history`, `tag`, `doctor`, `list`, `inspect`,
`visualize`, and `mcp` — works on one database, selected with the global
`--db-name` option.
```bash ```bash
haiku-rag search "query" # every configured database haiku-rag search "query" # every configured database
@ -357,10 +285,7 @@ haiku-rag --db-name papers list # one of them
haiku-rag --db-name papers migrate haiku-rag --db-name papers migrate
``` ```
`--db-name` selects an entry from `lancedb.databases`, including remote entries. `--db-name` selects an entry from `lancedb.databases`, including remote entries. `--db` selects a local path and overrides the configured location. A single-database command requires one of these options when multiple databases are configured. A configured set of one is selected automatically.
`--db` selects a local path and overrides the configured location. A
single-database command requires one of these options when multiple databases are
configured. A configured set of one is selected automatically.
Each database is created, migrated and vacuumed on its own: Each database is created, migrated and vacuumed on its own:

30
docs/python.md
View file

@ -24,11 +24,7 @@ async with HaikuRAG("path/to/database.lancedb", read_only=True) as client:
# await client.create_document(...) # Would raise ReadOnlyError # await client.create_document(...) # Would raise ReadOnlyError
``` ```
`async with` is the lifecycle. A caller that owns the client some other way `async with` is the lifecycle. A caller that owns the client some other way releases it with `await client.aclose()`, which does the same work for every client shape. `client.close()` closes the connection to one database and nothing else, since draining the background vacuum and releasing the embedder and reranker are awaitable; it refuses a client covering several.
releases it with `await client.aclose()`, which does the same work for every
client shape. `client.close()` closes the connection to one database and nothing
else, since draining the background vacuum and releasing the embedder and
reranker are awaitable; it refuses a client covering several.
!!! note !!! note
Databases must be explicitly created with `create=True` or via `haiku-rag init` before use. Opening a nonexistent unnamed local database raises `FileNotFoundError`, naming its path; one named in `lancedb.databases` raises `SourceUnavailableError`, which names the database rather than its location. Databases must be explicitly created with `create=True` or via `haiku-rag init` before use. Opening a nonexistent unnamed local database raises `FileNotFoundError`, naming its path; one named in `lancedb.databases` raises `SourceUnavailableError`, which names the database rather than its location.
@ -240,9 +236,7 @@ for result in results:
### Searching Multiple Databases ### Searching Multiple Databases
With [`lancedb.databases`](configuration/storage.md#multiple-databases) With [`lancedb.databases`](configuration/storage.md#multiple-databases) configured, a client covers the full set. Use `sources` to select a subset. Each result includes its database name:
configured, a client covers the full set. Use `sources` to select a subset.
Each result includes its database name:
```python ```python
results = await client.search("machine learning") # all of them results = await client.search("machine learning") # all of them
@ -262,21 +256,13 @@ for cite in citations:
result = await client.analyze("How many documents mention it?", sources=["papers"]) result = await client.analyze("How many documents mention it?", sources=["papers"])
``` ```
A scoped question can cite only the selected databases. Analysis mounts only A scoped question can cite only the selected databases. Analysis mounts only their documents.
their documents.
`sources=None` covers every database the client covers. `sources=[]` covers `sources=None` covers every database the client covers. `sources=[]` covers none: `search` returns no results, and `ask` and `analyze` run with no evidence from any database.
none: `search` returns no results, and `ask` and `analyze` run with no evidence
from any database.
A name no client covers raises `UnknownDatabaseError`, a `KeyError`, wherever it A name no client covers raises `UnknownDatabaseError`, a `KeyError`, wherever it is given: at construction, per query, and when placing a citation.
is given: at construction, per query, and when placing a citation.
On the constructor `sources=[]` means something else. Passing `sources` alongside a On the constructor `sources=[]` means something else. Passing `sources` alongside a database path raises `AmbiguousDatabaseError` immediately, since both say which database to open. Passing `sources=[]` alone raises `ValueError` on entering the client: a selection of nothing to search is a legitimate question, a client over no database is not.
database path raises `AmbiguousDatabaseError` immediately, since both say which
database to open. Passing `sources=[]` alone raises `ValueError` on entering the
client: a selection of nothing to search is a legitimate question, a client over
no database is not.
#### Inspecting the client scope #### Inspecting the client scope
@ -289,9 +275,7 @@ owner = await client.reader_for("papers") # the client reading that databas
papers, wiki = await client.clients_for(["papers", "wiki"]) papers, wiki = await client.clients_for(["papers", "wiki"])
``` ```
`reader_for` and `clients_for` open databases lazily and return borrowed clients. `reader_for` and `clients_for` open databases lazily and return borrowed clients. They remain valid while the covering client is open and inherit its read-only mode. The covering client owns and closes their database sessions.
They remain valid while the covering client is open and inherit its read-only
mode. The covering client owns and closes their database sessions.
### Filtering Search Results ### Filtering Search Results

5
evaluations/evaluations/capability_runner.py
View file

@ -43,9 +43,8 @@ class CapabilityRunResult:
answer: str answer: str
cited_uris: list[str] = field(default_factory=list) cited_uris: list[str] = field(default_factory=list)
cited_chunk_ids: list[str] = field(default_factory=list) cited_chunk_ids: list[str] = field(default_factory=list)
# The database each cited chunk came from, in the order they were cited, so a # The database each cited chunk came from, in the order they were cited.
# run over several databases records which one grounded the answer. Empty # Empty string where the database is unnamed.
# strings where the database is unnamed, since one database names nothing.
cited_sources: list[str] = field(default_factory=list) cited_sources: list[str] = field(default_factory=list)
searched_uris: list[str] = field(default_factory=list) searched_uris: list[str] = field(default_factory=list)
n_searches: int = 0 n_searches: int = 0

5
evaluations/evaluations/qa.py
View file

@ -186,9 +186,8 @@ def _refusal_metrics(report_cases) -> tuple[float, float, int, int] | None:
def _filter_qa_corpus(corpus, case_ids: set[str] | None): def _filter_qa_corpus(corpus, case_ids: set[str] | None):
"""Keep only rows whose ``id`` is in ``case_ids`` (failure-subset reruns). """Keep only rows whose ``id`` is in ``case_ids`` (failure-subset reruns).
Returns the corpus unchanged when ``case_ids`` is None. Matching nothing Returns the corpus unchanged when ``case_ids`` is None; matching nothing
raises: a dataset keying its rows by another name leaves every case filtered raises.
out, and a run of no cases otherwise reports 0.0000 as though it were a score.
""" """
if case_ids is None: if case_ids is None:
return corpus return corpus

6
evaluations/tests/test_benchmark.py
View file

@ -1291,8 +1291,7 @@ class TestEvaluateDatasetCaseIds:
def test_a_case_filter_matching_nothing_raises(): def test_a_case_filter_matching_nothing_raises():
"""A run of no cases reports 0.0000, which reads like a score rather than a """An id set that matches no case raises."""
mistake — so an id set that matches nothing fails instead."""
import pytest import pytest
from datasets import Dataset from datasets import Dataset
@ -1315,8 +1314,7 @@ def test_a_case_filter_that_matches_keeps_those_rows():
async def test_population_refuses_a_configured_set(): async def test_population_refuses_a_configured_set():
"""Population writes to one database, so a set would be ingested into a """Population writes to one database and refuses a configured set."""
database the run never reads."""
import pytest import pytest
from haiku.rag.config.models import AppConfig, LanceDBConfig from haiku.rag.config.models import AppConfig, LanceDBConfig

7
evaluations/tests/test_capability_runner.py
View file

@ -346,9 +346,8 @@ async def test_conversation_applies_document_filter(tmp_path):
async def test_conversation_carries_one_state_dict_across_turns(tmp_path): async def test_conversation_carries_one_state_dict_across_turns(tmp_path):
"""Capabilities read and write state through the deps dict; carrying the """Capabilities read and write state through the deps dict, and the same
same dict across turns is what lets compaction see earlier questions' dict is carried across every turn of a conversation."""
records instead of refusing."""
from evaluations.capability_runner import run_capability_conversation from evaluations.capability_runner import run_capability_conversation
deps_seen: list[object] = [] deps_seen: list[object] = []
@ -488,7 +487,7 @@ def test_records_the_database_each_citation_came_from():
def test_an_unnamed_database_records_no_source(): def test_an_unnamed_database_records_no_source():
"""One database names nothing, so the field stays empty rather than absent.""" """One database names nothing: the field holds an empty string."""
from haiku.rag.capabilities._base import EvidenceState from haiku.rag.capabilities._base import EvidenceState
from haiku.rag.capabilities.ledger import CapabilityEvidenceRecord from haiku.rag.capabilities.ledger import CapabilityEvidenceRecord
from haiku.rag.store.models.citation import Citation from haiku.rag.store.models.citation import Citation

4
haiku_rag_slim/haiku/rag/app.py
View file

@ -83,7 +83,7 @@ class HaikuRAGApp:
@property @property
def _is_local(self) -> bool: def _is_local(self) -> bool:
"""Whether the database is a local path rather than a URI. """True when the database is a local path, False for a URI.
Read from the database this command resolved to, not from the Read from the database this command resolved to, not from the
configuration: a database named in `lancedb.databases` can sit behind a configuration: a database named in `lancedb.databases` can sit behind a
@ -166,7 +166,7 @@ class HaikuRAGApp:
tables = {t.name: t for t in info.tables} tables = {t.name: t for t in info.tables}
# Per-table row counts and sizes. Missing required tables are # Per-table row counts and sizes. Missing required tables are
# reported as "absent" rather than raising. # reported as "absent".
for name in ("documents", "document_meta", "chunks", "document_items"): for name in ("documents", "document_meta", "chunks", "document_items"):
entry = tables[name] entry = tables[name]
if entry.exists: if entry.exists:

8
haiku_rag_slim/haiku/rag/capabilities/_base.py
View file

@ -65,8 +65,7 @@ duplicating a character or a whole group stays above 0.75, so the gap is wide.
def _ambiguous_retry(error: AmbiguousCitationError) -> ModelRetry: def _ambiguous_retry(error: AmbiguousCitationError) -> ModelRetry:
"""The only way out of an id that names a chunk in two databases. """The only way out of an id that names a chunk in two databases.
The model cannot say which it meant, so it is asked for other evidence The model cannot say which it meant, so the retry asks for other evidence.
rather than for the same id again.
""" """
return ModelRetry( return ModelRetry(
f"{error}. Cite a chunk id that appears once across the databases " f"{error}. Cite a chunk id that appears once across the databases "
@ -188,7 +187,7 @@ class RAGCapabilityBase[StateT: EvidenceState](AbstractCapability[Any]):
False on a first question, and equally on every question of a host that does False on a first question, and equally on every question of a host that does
not carry state between runs. Capabilities that need the record to mean not carry state between runs. Capabilities that need the record to mean
anything across questions read it to refuse rather than act on nothing. anything across questions refuse when this is False.
""" """
async def for_run(self, ctx: RunContext[Any]) -> "RAGCapabilityBase[StateT]": async def for_run(self, ctx: RunContext[Any]) -> "RAGCapabilityBase[StateT]":
@ -618,8 +617,7 @@ class RAGCapabilityBase[StateT: EvidenceState](AbstractCapability[Any]):
"""Index the citations, numbering the ones not already registered. """Index the citations, numbering the ones not already registered.
The index is keyed by chunk id and outlives the question, so an id The index is keyed by chunk id and outlives the question, so an id
already registered from another database is refused here rather than already registered from another database is refused here.
silently keeping the earlier one's content and database.
""" """
assert self.state is not None assert self.state is not None
state = self.state state = self.state

7
haiku_rag_slim/haiku/rag/cli.py
View file

@ -72,7 +72,8 @@ def create_app(db: Path | None = None, *, covers_set: bool = False) -> "HaikuRAG
Raises: Raises:
AmbiguousDatabaseError: multiple databases are configured and this AmbiguousDatabaseError: multiple databases are configured and this
command works on one, without `--db` or `--db-name` naming which. command works on one, without `--db` or `--db-name` naming which;
or `--db` and `--db-name` are both given.
""" """
from haiku.rag.app import HaikuRAGApp from haiku.rag.app import HaikuRAGApp
@ -90,8 +91,8 @@ def resolve_scope(
The CLI decides only what it alone knows: that `--db` and `--db-name` are The CLI decides only what it alone knows: that `--db` and `--db-name` are
the same thing said twice, and whether this command can read more than one. the same thing said twice, and whether this command can read more than one.
Everything else — an unknown name, a legacy `uri`, the default location — is Everything else — an unknown name, a `lancedb.uri`, the default location —
`DatabaseScope.resolve`'s to answer, so there is one table and not two. is `DatabaseScope.resolve`'s to answer, so there is one table and not two.
""" """
from haiku.rag.client.scope import DatabaseScope from haiku.rag.client.scope import DatabaseScope

38
haiku_rag_slim/haiku/rag/client/__init__.py
View file

@ -151,8 +151,8 @@ class HaikuRAG:
""" """
self._configured = config if config is not None else get_config() self._configured = config if config is not None else get_config()
# What the caller configured, kept intact: entering derives a # What the caller configured, kept intact: entering derives a
# single-database configuration from it, and asking again has to see the # single-database configuration from it, and every re-entry derives
# same set rather than the answer from last time. # from the configured set.
self._config = self._configured self._config = self._configured
self._requested_db_path = Path(db_path) if db_path is not None else None self._requested_db_path = Path(db_path) if db_path is not None else None
if self._requested_db_path is not None and sources is not None: if self._requested_db_path is not None and sources is not None:
@ -209,9 +209,8 @@ class HaikuRAG:
None only when a client covering a set is given no name, as for evidence None only when a client covering a set is given no name, as for evidence
recorded before databases could be named. A name this client does not recorded before databases could be named. A name this client does not
cover raises `UnknownDatabaseError`, decided by `clients_covering` so cover raises `UnknownDatabaseError`, decided by `clients_covering`: one
that one database answers a wrong name the same way a set does: database answers a wrong name the same way a set does.
provenance naming another database is wrong rather than absent.
""" """
if source is None: if source is None:
return None if self.covers_multiple else self return None if self.covers_multiple else self
@ -275,7 +274,7 @@ class HaikuRAG:
def embedder(self) -> "EmbedderWrapper": def embedder(self) -> "EmbedderWrapper":
"""The embedder for the databases this client covers. """The embedder for the databases this client covers.
An embedder is a function of configuration rather than of a database, An embedder is a function of configuration, not of a database,
and the databases in a selection are required to share one, so a client and the databases in a selection are required to share one, so a client
covering a set has an unambiguous embedder without opening any of them. covering a set has an unambiguous embedder without opening any of them.
Built on first use and owned by this client, which closes it. Built on first use and owned by this client, which closes it.
@ -361,9 +360,9 @@ class HaikuRAG:
async def clients_for(self, names: list[str]) -> list["HaikuRAG"]: async def clients_for(self, names: list[str]) -> list["HaikuRAG"]:
"""The clients for these databases, opening any not yet open. """The clients for these databases, opening any not yet open.
Opening is per query rather than at entry: a set of 25 configured Opening is per query: a set of 25 configured databases is typically
databases is typically queried a few at a time, and a database nobody queried a few at a time, and a database the query does not cover stays
asked for must not be able to fail a query, or be opened for nothing. closed.
The clients returned borrow their databases from this one and are valid The clients returned borrow their databases from this one and are valid
only while it is open. Closing one, or entering it as a context manager, only while it is open. Closing one, or entering it as a context manager,
@ -415,8 +414,7 @@ class HaikuRAG:
) -> "HaikuRAG": ) -> "HaikuRAG":
"""A client over a database another session opened and will close. """A client over a database another session opened and will close.
`lender` is the client that opened it, whose reranker this one borrows `lender` is the client that opened it, whose reranker this one borrows.
rather than building a second copy of the same model.
""" """
client = cls( client = cls(
session.db_path, config=session.config, read_only=session.read_only session.db_path, config=session.config, read_only=session.read_only
@ -429,14 +427,10 @@ class HaikuRAG:
def _require_one_embedder(self, clients: "list[HaikuRAG]") -> None: def _require_one_embedder(self, clients: "list[HaikuRAG]") -> None:
"""Fail when two of these databases were written with different embedders. """Fail when two of these databases were written with different embedders.
Searching a set embeds the query once, so a database written with another Only databases searched together have to agree: this is a property of
model answers from a different vector space: its candidates are noise, and the selection, not of the set. Drift between a database and the *config*
rank fusion gives them slots anyway. Only databases searched together have is a separate, softer matter, which `SettingsRepository` reports on
to agree, so this is a property of the selection rather than of the set. open.
Drift between a database and the *config* is a separate, softer matter —
the same model served by another stack is spelled differently — which
`SettingsRepository` reports on open.
""" """
recorded = [ recorded = [
(client.source, client.store.stored_embedding) (client.source, client.store.stored_embedding)
@ -505,8 +499,7 @@ class HaikuRAG:
async def _aclose_cached(self, name: str) -> None: async def _aclose_cached(self, name: str) -> None:
"""Close a cached_property this client materialized, and discard it. """Close a cached_property this client materialized, and discard it.
Discarded rather than left in place so that re-entering the client Re-entering the client builds a fresh one.
builds a fresh one instead of reusing something already closed.
""" """
cached = self.__dict__.pop(name, None) cached = self.__dict__.pop(name, None)
if cached is not None: if cached is not None:
@ -920,7 +913,8 @@ class HaikuRAG:
return [] return []
results = await search(self, query, limit, search_type, filter, include_images) results = await search(self, query, limit, search_type, filter, include_images)
# A database named in config keeps its name even when it is the only one # A database named in config keeps its name even when it is the only one
# this client covers. Only a legacy single `uri` leaves source unset. # this client covers. Only an unnamed `lancedb.uri` database leaves
# source unset.
for result in results: for result in results:
result.source = self.source result.source = self.source
return results return results

4
haiku_rag_slim/haiku/rag/client/scope.py
View file

@ -58,8 +58,8 @@ class DatabaseRef:
class DatabaseScope: class DatabaseScope:
"""The databases an operation covers. """The databases an operation covers.
Resolved once, from configuration plus at most one selector, and passed down Resolved once, from configuration plus at most one selector, then passed
rather than re-derived. Never empty. down. Never empty.
Nothing here reads the environment: ``HAIKU_RAG_DB`` is the capability entry Nothing here reads the environment: ``HAIKU_RAG_DB`` is the capability entry
point's to honour, and reading it here would extend it to every caller. point's to honour, and reading it here would extend it to every caller.

16
haiku_rag_slim/haiku/rag/client/search.py
View file

@ -80,8 +80,8 @@ async def search_sources(
"""Search several databases and fuse their results into one ranked list. """Search several databases and fuse their results into one ranked list.
Fetch, fuse, truncate, then enrich: enrichment runs on the survivors through Fetch, fuse, truncate, then enrich: enrichment runs on the survivors through
the database each came from, so it costs the same as a single-database search the database each came from, so its cost is that of a single-database
rather than multiplying by the number searched. search.
""" """
if limit is None: if limit is None:
limit = client._config.search.limit limit = client._config.search.limit
@ -242,10 +242,9 @@ async def _embed_query(
) -> list[float] | None: ) -> list[float] | None:
"""The query as a vector, or None when the search needs no vector. """The query as a vector, or None when the search needs no vector.
Computed by the caller so that searching several databases embeds once: the The caller computes it once for however many databases the search covers:
databases in a selection share an embedder, and embedding per database costs the databases in a selection share an embedder. `search_type` is the
a round trip each on a remote endpoint. `search_type` is the resolved one, so resolved one, so only a text query ever reaches this as full-text.
only a text query ever reaches this as full-text.
""" """
if search_type == "fts": if search_type == "fts":
return None return None
@ -289,9 +288,8 @@ async def _rank(
async def _attach_picture_data(client: "HaikuRAG", chunks: list[Chunk]) -> None: async def _attach_picture_data(client: "HaikuRAG", chunks: list[Chunk]) -> None:
"""Attach picture bytes to synthetic picture chunks in-place, so a """Attach picture bytes to synthetic picture chunks in-place; a multimodal
multimodal reranker can score the pixels instead of just the chunk's reranker scores the pixels.
description text.
One query however many documents the candidates span, which matters here One query however many documents the candidates span, which matters here
more than anywhere: reranking fetches `limit * 10` candidates. more than anywhere: reranking fetches `limit * 10` candidates.

31
haiku_rag_slim/haiku/rag/client/session.py
View file

@ -30,17 +30,13 @@ logger = logging.getLogger(__name__)
_VACUUM_MIN_INTERVAL_S = 300.0 _VACUUM_MIN_INTERVAL_S = 300.0
# Failures whose message names the remedy and never the location, so the failing # Failures whose message names the remedy and never the location. `open()`
# database is named alongside it instead of in place of it. # prefixes the failing database's name.
_NAMEABLE_FAILURES = (MigrationRequiredError, ConfigMismatchError, ReadOnlyError) _NAMEABLE_FAILURES = (MigrationRequiredError, ConfigMismatchError, ReadOnlyError)
async def aclose_quietly(closeable: Any, what: str) -> None: async def aclose_quietly(closeable: Any, what: str) -> None:
"""Close, reporting failure to the log rather than raising. """Close; a failure is logged, never raised."""
Teardown can run while an exception unwinds, so a raising close must
neither mask that exception nor stop a sibling from being closed.
"""
try: try:
await closeable.aclose() await closeable.aclose()
except Exception: except Exception:
@ -59,8 +55,8 @@ class SingleDatabaseSession:
it has one. ``source`` is the configured name this database answers to, or it has one. ``source`` is the configured name this database answers to, or
None where nothing names it. None where nothing names it.
``db_path``, ``config``, ``read_only`` and ``source`` are readable so that a ``db_path``, ``config``, ``read_only`` and ``source`` are readable: a client
client borrowing this session can report them as its own. borrowing this session reports them as its own.
""" """
def __init__( def __init__(
@ -110,8 +106,7 @@ class SingleDatabaseSession:
self.store.close() self.store.close()
raise raise
except _NAMEABLE_FAILURES as error: except _NAMEABLE_FAILURES as error:
# These name the remedy and not the database, so the name is added # The message keeps its remedy and gains the database's name.
# rather than substituted.
if self.source is None: if self.source is None:
raise raise
raise type(error)(f"database {self.source!r}: {error}") from error raise type(error)(f"database {self.source!r}: {error}") from error
@ -152,10 +147,8 @@ class SingleDatabaseSession:
def schedule_vacuum(self) -> None: def schedule_vacuum(self) -> None:
"""Schedule a background vacuum, throttled to at most one per """Schedule a background vacuum, throttled to at most one per
``_VACUUM_MIN_INTERVAL_S``. Sustained writes would otherwise trigger ``_VACUUM_MIN_INTERVAL_S``. The throttle only skips the background task —
back-to-back compaction of the blob-bearing documents table. The throttle ``_vacuum_dirty`` still marks that a final vacuum on close is owed."""
only skips the background task — ``_vacuum_dirty`` still marks that a
final vacuum on close is owed."""
self._vacuum_dirty = True self._vacuum_dirty = True
now = monotonic() now = monotonic()
if ( if (
@ -246,8 +239,7 @@ class SingleDatabaseSession:
async def aclose(self) -> None: async def aclose(self) -> None:
"""Drain, release the embedder, and close the connection. """Drain, release the embedder, and close the connection.
The store owns the embedder, so releasing it belongs here rather than The store owns the embedder, so this is where it is released.
with whoever happened to hold the session.
""" """
await self.drain_vacuum() await self.drain_vacuum()
await aclose_quietly(self.store.embedder, "embedder") await aclose_quietly(self.store.embedder, "embedder")
@ -262,9 +254,8 @@ class FederatedSession:
"""Several databases, read as one. Reads only. """Several databases, read as one. Reads only.
Composes single-database sessions and owns their teardown. They open on first Composes single-database sessions and owns their teardown. They open on first
use rather than at entry: which databases a query covers is a per-query use: which databases a query covers is a per-query choice, and a database the
choice, so a database nobody asked for must neither be opened for nothing nor query does not cover stays closed.
be able to fail a query.
""" """
def __init__( def __init__(

4
haiku_rag_slim/haiku/rag/ingester/app.py
View file

@ -77,8 +77,8 @@ class IngesterApp:
from haiku.rag.store.exceptions import AmbiguousDatabaseError from haiku.rag.store.exceptions import AmbiguousDatabaseError
self._config = config self._config = config
# `--db` names the database directly, and nothing stands in for it: a # `--db` is an explicit override; None leaves placement to the
# manufactured default would override a configured `lancedb.uri`. # configuration.
self._scope = DatabaseScope.resolve(config, database_path=db_path) self._scope = DatabaseScope.resolve(config, database_path=db_path)
if self._scope.covers_multiple: if self._scope.covers_multiple:
raise AmbiguousDatabaseError( raise AmbiguousDatabaseError(

3
haiku_rag_slim/haiku/rag/inspector/app.py
View file

@ -235,7 +235,8 @@ def run_inspector(
"""Run the inspector TUI. """Run the inspector TUI.
Args: Args:
db_path: Path to the LanceDB database. If None, uses default from config. db_path: Path to the LanceDB database, when no scope is given.
scope: The database to inspect, resolved by the caller.
read_only: Whether to open the database in read-only mode. read_only: Whether to open the database in read-only mode.
""" """
config = get_config() config = get_config()

7
haiku_rag_slim/haiku/rag/inspector/widgets/info_modal.py
View file

@ -17,11 +17,8 @@ if TYPE_CHECKING:
async def database_lines(client: "HaikuRAG") -> list[str]: async def database_lines(client: "HaikuRAG") -> list[str]:
"""What one database reports about itself, without naming its location. """What one database reports about itself, without naming its location.
Reported through the connection the client already holds: a second one to the Reported through the connection the client already holds. A failure becomes
same database would be a second open for the same statistics. a line of the report, and the other databases still report.
A failure is reported as a line rather than raised, so one unreachable
database does not cost the report on the others.
""" """
from haiku.rag.store.engine import ConnectionMode from haiku.rag.store.engine import ConnectionMode
from haiku.rag.store.info import get_database_stats from haiku.rag.store.info import get_database_stats

7
haiku_rag_slim/haiku/rag/mcp.py
View file

@ -33,8 +33,8 @@ def create_mcp_server(
Args: Args:
db_path: Path to the database file, or None to let `config` place it. A db_path: Path to the database file, or None to let `config` place it. A
path overrides a configured `lancedb.uri`, so a URI-backed database path overrides a configured `lancedb.uri`: for a URI-backed
must pass None rather than a local stand-in. database, pass None.
config: Configuration to use. config: Configuration to use.
read_only: If True, write tools (add_document_*, delete_document) are not registered. read_only: If True, write tools (add_document_*, delete_document) are not registered.
""" """
@ -82,8 +82,7 @@ def _covering(scope: "DatabaseScope", config: AppConfig, read_only: bool) -> Fas
@asynccontextmanager @asynccontextmanager
async def lifespan(_server: FastMCP) -> AsyncIterator[None]: async def lifespan(_server: FastMCP) -> AsyncIterator[None]:
# Open eagerly so an unopenable database fails startup rather than # Opened eagerly: an unopenable database fails at startup.
# every tool call.
nonlocal client nonlocal client
await _client() await _client()
try: try:

14
haiku_rag_slim/haiku/rag/sandbox/sandbox.py
View file

@ -135,8 +135,9 @@ class Sandbox:
file callbacks are synchronous and run off that loop while ``feed_run`` is file callbacks are synchronous and run off that loop while ``feed_run`` is
awaited, so they bridge back to it via ``run_coroutine_threadsafe`` without awaited, so they bridge back to it via ``run_coroutine_threadsafe`` without
deadlocking. When a ``rag`` connection is supplied it is used for every read, deadlocking. When a ``rag`` connection is supplied it is used for every read,
so an analysis run drives a single connection on a single loop; otherwise so an analysis run drives a single connection on a single loop. Otherwise a
each read opens an ephemeral read-only connection. scope covering several databases opens a federated client once and holds it
until ``close()``, and a single database is opened per read.
""" """
_scope: "DatabaseScope" _scope: "DatabaseScope"
@ -188,8 +189,8 @@ class Sandbox:
Internal: the public constructor takes a path and resolves it, which is Internal: the public constructor takes a path and resolves it, which is
its own job. This is for callers that did the resolving, as its own job. This is for callers that did the resolving, as
``HaikuRAG._covering`` is. It sets the sandbox up directly rather than ``HaikuRAG._covering`` is. It bypasses ``__init__``: the scope it is
through ``__init__``, so the scope it is handed is the only one resolved. handed is the only one resolved.
""" """
sandbox = cls.__new__(cls) sandbox = cls.__new__(cls)
sandbox._configure(scope, config, context, rag, lock) sandbox._configure(scope, config, context, rag, lock)
@ -604,9 +605,8 @@ class Sandbox:
"""Resource limits for the worker session. """Resource limits for the worker session.
Monty spends ``max_duration_secs`` across the session's whole life, and Monty spends ``max_duration_secs`` across the session's whole life, and
the session is reused so variables persist between calls. Budget it for the session is reused so variables persist between calls: the budget
the run rather than for one call, or the first slow call starves every covers the whole run. ``code_timeout`` is enforced per call elsewhere: the read
later one. ``code_timeout`` is enforced per call elsewhere: the read
deadline in ``_run_on_loop`` bounds a call that reads, and the pool's deadline in ``_run_on_loop`` bounds a call that reads, and the pool's
``request_timeout`` bounds one that computes. ``request_timeout`` bounds one that computes.
""" """

3
haiku_rag_slim/haiku/rag/store/exceptions.py
View file

@ -41,8 +41,7 @@ class AmbiguousCitationError(Exception):
"""A cited chunk id names a chunk in more than one database. """A cited chunk id names a chunk in more than one database.
A citation records the id alone, so nothing downstream can say which A citation records the id alone, so nothing downstream can say which
database it came from. Raised rather than resolved: picking one attributes database it came from.
the answer to a database it may not have come from.
""" """

5
haiku_rag_slim/haiku/rag/store/repositories/chunk.py
View file

@ -213,9 +213,8 @@ class ChunkRepository:
limit: Maximum number of results to return. limit: Maximum number of results to return.
search_type: "vector", "fts", or "hybrid" (default). search_type: "vector", "fts", or "hybrid" (default).
filter: Optional SQL WHERE clause to filter documents before searching chunks. filter: Optional SQL WHERE clause to filter documents before searching chunks.
query_vector: Pre-computed query embedding, used instead of embedding query_vector: Pre-computed query embedding; when supplied, ``query``
``query``. Searching several databases embeds once and passes it is not embedded.
to each.
Returns: Returns:
List of (chunk, score) tuples ordered by relevance. List of (chunk, score) tuples ordered by relevance.

7
haiku_rag_slim/haiku/rag/utils.py
View file

@ -394,9 +394,10 @@ async def format_citations_rich(
) -> "list[RenderableType]": ) -> "list[RenderableType]":
"""Format citations as Rich renderables for terminal display. """Format citations as Rich renderables for terminal display.
Each citation becomes a Panel with a compact header (``[N] Title (URI) — locator``), Each citation becomes a Panel with a compact header (``[N] Title (URI) —
a body holding any referenced figures followed by a truncated text preview, and locator``, with the database name before the locator when ``client`` covers
a dimmed footer that exposes the document and chunk IDs. several), a body holding any referenced figures followed by a truncated text
preview, and a dimmed footer that exposes the document and chunk IDs.
When ``client`` is supplied, picture bytes for ``picture_refs`` are fetched and When ``client`` is supplied, picture bytes for ``picture_refs`` are fetched and
rendered inline via ``textual_image``. Without a client, picture refs appear as rendered inline via ``textual_image``. Without a client, picture refs appear as

19
tests/capabilities/test_capabilities.py
View file

@ -121,8 +121,8 @@ def test_capability_factories_resolve_environment_and_defaults(
class TestACapabilityFollowsTheConfiguredLocation: class TestACapabilityFollowsTheConfiguredLocation:
"""A capability nobody handed a client opens one for itself, and has to open """A capability nobody handed a client opens one for itself, at the
the database the configuration places rather than the default directory.""" database the configuration places."""
def _config(self, tmp_path, uri: str) -> AppConfig: def _config(self, tmp_path, uri: str) -> AppConfig:
from haiku.rag.config.models import LanceDBConfig, StorageConfig from haiku.rag.config.models import LanceDBConfig, StorageConfig
@ -133,8 +133,8 @@ class TestACapabilityFollowsTheConfiguredLocation:
) )
def test_a_configured_uri_is_left_to_the_client(self, tmp_path): def test_a_configured_uri_is_left_to_the_client(self, tmp_path):
"""A path overrides a configured location, so manufacturing one would """A path overrides a configured location, so the capability passes
send the capability to the default directory instead of the bucket.""" None and the client resolves the configured URI."""
located = tmp_path / "notes.lancedb" located = tmp_path / "notes.lancedb"
for factory in (create_rag, create_analysis): for factory in (create_rag, create_analysis):
[local] = factory( [local] = factory(
@ -208,10 +208,8 @@ def test_domain_preamble_is_added_to_capability_instructions(temp_db_path):
def _single_database_client() -> AsyncMock: def _single_database_client() -> AsyncMock:
"""A stand-in for a client covering one unnamed database. """A stand-in for a client covering one unnamed database.
A bare AsyncMock answers every attribute with a truthy Mock, so `covers_multiple`, `source` and `clients_covering` answer as one unnamed
`covers_multiple` would read as a set of databases, `source` would reach a database does; a bare AsyncMock answers every attribute with a truthy Mock.
validated field, and `clients_covering` would return a Mock where the code
iterates clients.
""" """
client = AsyncMock() client = AsyncMock()
client.covers_multiple = False client.covers_multiple = False
@ -1667,9 +1665,8 @@ async def test_an_answered_question_is_no_longer_in_progress(temp_db_path):
class TestMultipleCollectionsInstructions: class TestMultipleCollectionsInstructions:
"""The note follows what a run reads, not what the configuration names, so a """The note follows what a run reads, not what the configuration names: a
run over one collection is instructed exactly as it was before collections run over one collection gets the single-collection instructions."""
could be named."""
@staticmethod @staticmethod
def _config(**databases): def _config(**databases):

27
tests/chat/test_chat_app.py
View file

@ -90,8 +90,8 @@ def test_chat_capabilities_read_the_named_database(tmp_path, monkeypatch):
run_chat(scope=DatabaseScope.resolve(config)) run_chat(scope=DatabaseScope.resolve(config))
[covering] = chat_app.call_args.kwargs["capabilities"] [covering] = chat_app.call_args.kwargs["capabilities"]
# The chat lends its own client, so this scope is the fallback: what matters # The chat lends its own client, so this scope is the fallback: it places
# is that it places the named database rather than the whole set. # the named database alone.
[placed] = named.scope.databases [placed] = named.scope.databases
assert placed.db_path == tmp_path / "b.lancedb" assert placed.db_path == tmp_path / "b.lancedb"
assert named.config.lancedb.databases == {} assert named.config.lancedb.databases == {}
@ -156,8 +156,8 @@ def _make_mock_client():
mock_client = AsyncMock() mock_client = AsyncMock()
mock_client.__aenter__ = AsyncMock(return_value=mock_client) mock_client.__aenter__ = AsyncMock(return_value=mock_client)
mock_client.__aexit__ = AsyncMock(return_value=None) mock_client.__aexit__ = AsyncMock(return_value=None)
# Covers one database: a bare AsyncMock answers `covers_multiple` with a truthy # Covers one database; a bare AsyncMock answers `covers_multiple` with a
# Mock, which would send every read down the covering-a-set branch. # truthy Mock.
mock_client.covers_multiple = False mock_client.covers_multiple = False
mock_client.source_names = () mock_client.source_names = ()
mock_client.source = None mock_client.source = None
@ -670,8 +670,8 @@ async def test_visual_grounding_uses_the_database_holding_the_citation(tmp_path)
class TestLendingTheClient: class TestLendingTheClient:
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_mounting_lends_its_client_to_every_capability(self, temp_db_path): async def test_mounting_lends_its_client_to_every_capability(self, temp_db_path):
"""Capabilities are built before the client exists, so each reads through """Capabilities are built before the client exists, and each reads
the one the app opened rather than opening its own.""" through the one the app opened."""
client = _make_mock_client() client = _make_mock_client()
app, _ = _make_app(temp_db_path, client) app, _ = _make_app(temp_db_path, client)
@ -1126,8 +1126,8 @@ class TestKeepingSelectionsReachable:
[only] = list(modal.query(DocumentCheckbox)) [only] = list(modal.query(DocumentCheckbox))
assert only.doc_id == "sel-0200" assert only.doc_id == "sel-0200"
# Awaited rather than posted: the handler reads the database, so # Awaited directly: the handler reads the database, and a
# a single pause need not have flushed it. # single pause need not have flushed it.
await modal.on_checkbox_changed(Checkbox.Changed(only, False)) await modal.on_checkbox_changed(Checkbox.Changed(only, False))
await pilot.pause() await pilot.pause()
@ -1145,8 +1145,8 @@ class TestKeepingSelectionsReachable:
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_the_results_listing_pages_too(self, temp_db_path: Path): async def test_the_results_listing_pages_too(self, temp_db_path: Path):
"""More documents match than one page holds, so the rest are a page """More documents match than one page holds; the rest are a page
away rather than unreachable.""" away."""
from textual.widgets import Button from textual.widgets import Button
from haiku.rag.chat.widgets.document_filter_modal import ( from haiku.rag.chat.widgets.document_filter_modal import (
@ -1211,11 +1211,8 @@ class TestKeepingSelectionsReachable:
async def test_typing_without_submitting_leaves_the_listing_alone( async def test_typing_without_submitting_leaves_the_listing_alone(
self, temp_db_path: Path self, temp_db_path: Path
): ):
"""The listing is what the last submitted search asked for, and it pages. """The listing is what the last submitted search asked for, and it
pages. The term applies on enter, and the footer says so until then.
Narrowing it as the user types would hide rows from the page the search
landed on while the rest of the matches stayed a page away, so the term
applies on enter and says so until then.
""" """
from textual.widgets import Button, Input, Static from textual.widgets import Button, Input, Static

4
tests/conftest.py
View file

@ -226,7 +226,7 @@ def docling_serve_url() -> str:
def writing(client: "HaikuRAG") -> "SingleDatabaseSession": def writing(client: "HaikuRAG") -> "SingleDatabaseSession":
"""The database a write implementation works on, from a client holding one. """The database a write implementation works on, from a client holding one.
Write implementations take a session rather than a client, so a set can Write implementations take a session, never a client, so a set can
never reach them. Tests that call one directly go through here.""" never reach them. Tests that call one directly go through here."""
from haiku.rag.client.session import SingleDatabaseSession from haiku.rag.client.session import SingleDatabaseSession
@ -240,7 +240,7 @@ def for_path(
"""A scope covering one database at `db_path`. """A scope covering one database at `db_path`.
The application layer takes the databases it works on, already resolved. The application layer takes the databases it works on, already resolved.
Tests that hold a path rather than a scope go through here. Tests that hold a path and need a scope go through here.
""" """
from haiku.rag.client.scope import DatabaseScope from haiku.rag.client.scope import DatabaseScope
from haiku.rag.config import get_config from haiku.rag.config import get_config

13
tests/ingester/test_cli.py
View file

@ -504,9 +504,8 @@ def test_cli_entry_point_exits_on_store_state_errors(monkeypatch, capsys, error)
class TestPlacingTheIngesterDatabase: class TestPlacingTheIngesterDatabase:
"""The ingester writes wherever the configuration places the database, and """The ingester writes wherever the configuration places the database, and
resolves that once. A manufactured local default would silently redirect a resolves that once. A path is an explicit override of a configured
remote deployment to the local disk, because a path is an explicit override `lancedb.uri`, so no local default stands in for one."""
of a configured `lancedb.uri`."""
@staticmethod @staticmethod
def _app(config: AppConfig, db_path=None): def _app(config: AppConfig, db_path=None):
@ -543,8 +542,8 @@ class TestPlacingTheIngesterDatabase:
assert not scope.covers_multiple assert not scope.covers_multiple
def test_a_set_is_refused_with_a_remedy_this_command_has(self, tmp_path): def test_a_set_is_refused_with_a_remedy_this_command_has(self, tmp_path):
"""The client would name `sources=[name]`, a Python argument no CLI user """The remedy in the message is `--db PATH`, an argument this command
can pass.""" has; `sources=` is a Python argument no CLI user can pass."""
from haiku.rag.store.exceptions import AmbiguousDatabaseError from haiku.rag.store.exceptions import AmbiguousDatabaseError
config = AppConfig( config = AppConfig(
@ -560,8 +559,8 @@ class TestPlacingTheIngesterDatabase:
assert "sources=" not in str(raised.value) assert "sources=" not in str(raised.value)
def test_several_configured_databases_exit_cleanly(self, tmp_path, monkeypatch): def test_several_configured_databases_exit_cleanly(self, tmp_path, monkeypatch):
"""No selector names one of a set, so the CLI reports it rather than """No selector names one of a set, so the CLI exits with the message
printing a traceback.""" and no traceback."""
config_file = tmp_path / "haiku.rag.yaml" config_file = tmp_path / "haiku.rag.yaml"
config_file.write_text( config_file.write_text(
"lancedb:\n databases:\n" "lancedb:\n databases:\n"

4
tests/ingester/test_run_batch.py
View file

@ -103,8 +103,8 @@ def use_client(monkeypatch):
async def _cm(*_, **__): async def _cm(*_, **__):
yield client yield client
# The app opens the databases its scope covers, so the double stands in # The app opens the databases its scope covers: the double stands in
# for `_covering` rather than the public constructor. # for `_covering`.
stub = MagicMock() stub = MagicMock()
stub._covering = lambda *a, **k: _cm() stub._covering = lambda *a, **k: _cm()
monkeypatch.setattr("haiku.rag.client.HaikuRAG", stub) monkeypatch.setattr("haiku.rag.client.HaikuRAG", stub)

3
tests/multi_db/helpers.py
View file

@ -58,8 +58,7 @@ async def _restore_embedder(config, name, *, provider=None, model_name=None):
async def _seed_expandable(config, name, sentences): async def _seed_expandable(config, name, sentences):
"""One document whose chunk covers a single item, so expansion has """One document whose chunk covers a single item, so expansion has
neighbours to pull in and rebuilds the result rather than passing it neighbours to pull in and rebuilds the result."""
through."""
dim = get_config().embeddings.model.vector_dim dim = get_config().embeddings.model.vector_dim
doc = DoclingDocument(name=name) doc = DoclingDocument(name=name)
for sentence in sentences: for sentence in sentences:

4
tests/multi_db/test_capabilities.py
View file

@ -226,8 +226,8 @@ class TestNamingDatabasesBeforeTheModelRuns:
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_checking_a_name_opens_nothing(self, tmp_path): async def test_checking_a_name_opens_nothing(self, tmp_path):
"""Opening to check would open every database on an unscoped question, """Validating a name reads the configured set; no database opens for
and let one nobody asked about fail a run before any search.""" it."""
config = _config(tmp_path, ["alpha", "beta"]) config = _config(tmp_path, ["alpha", "beta"])
await _seed(config, "alpha", ["alpha document about cats"]) await _seed(config, "alpha", ["alpha document about cats"])
await _seed(config, "beta", ["beta document about cats"]) await _seed(config, "beta", ["beta document about cats"])

15
tests/multi_db/test_citations.py
View file

@ -40,8 +40,8 @@ class TestSharedChunkIds:
] ]
def test_a_shared_id_cannot_be_cited(self): def test_a_shared_id_cannot_be_cited(self):
"""A citation records the id alone, so resolving one held by two """A citation records the id alone, so one held by two databases
databases would attribute the answer to whichever came last.""" cannot say which grounded the answer."""
results = [ results = [
SearchResult( SearchResult(
content="alpha body", content="alpha body",
@ -66,8 +66,8 @@ class TestSharedChunkIds:
def test_a_repeated_id_from_one_database_still_collapses(self): def test_a_repeated_id_from_one_database_still_collapses(self):
"""One database cannot hold two chunks under one id, so seeing it twice """One database cannot hold two chunks under one id, so seeing it twice
is the same chunk seen twice, and it resolves rather than raising. Which is the same chunk seen twice, and it resolves. Which copy supplies the
copy supplies the content is `resolve_citations`' own rule, pinned in content is `resolve_citations`' own rule, pinned in
`tests/store/test_citation.py`.""" `tests/store/test_citation.py`."""
results = [ results = [
SearchResult( SearchResult(
@ -122,8 +122,7 @@ class TestSharedChunkIds:
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_an_unsearched_shared_id_is_refused_by_the_fallback(self, tmp_path): async def test_an_unsearched_shared_id_is_refused_by_the_fallback(self, tmp_path):
"""The direct lookup is the only place a collision shows for an id no """The direct lookup is the only place a collision shows for an id no
search returned, so it has to ask every database rather than take the search returned, so it asks every database."""
first that answers."""
import shutil import shutil
config = _config(tmp_path, ["alpha", "beta"]) config = _config(tmp_path, ["alpha", "beta"])
@ -200,8 +199,8 @@ class TestSharedChunkIds:
async def test_cite_refuses_an_id_already_cited_from_another_database( async def test_cite_refuses_an_id_already_cited_from_another_database(
self, tmp_path self, tmp_path
): ):
"""The citation index outlives the question, so the collision can arrive """The citation index outlives the question, so the collision can
a turn later than the search that would have shown it.""" arrive a turn after the search."""
capability = create_capability( capability = create_capability(
config=_config(tmp_path, ["alpha", "beta"]), defer_loading=False config=_config(tmp_path, ["alpha", "beta"]), defer_loading=False
) )

3
tests/multi_db/test_documents.py
View file

@ -155,8 +155,7 @@ class TestLookupByIdentifier:
self, tmp_path self, tmp_path
): ):
"""A database copied from another holds the same ids. A read has an """A database copied from another holds the same ids. A read has an
answer wherever it finds one, and which one it is has to be the answer wherever it finds one, and the configured order says which."""
configured order rather than whichever replied first."""
import shutil import shutil
config = _config(tmp_path, ["alpha", "beta"]) config = _config(tmp_path, ["alpha", "beta"])

30
tests/multi_db/test_lifecycle.py
View file

@ -64,9 +64,9 @@ class TestOpeningDatabases:
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_a_cancelled_open_does_not_leak_the_ones_that_worked(self, tmp_path): async def test_a_cancelled_open_does_not_leak_the_ones_that_worked(self, tmp_path):
"""Cancellation discards the fan-out's results rather than returning them, """Cancellation discards the fan-out's results, so a database that
so a database that opened while a sibling was still pending is reachable opened while a sibling was still pending is reachable only through the
only because the opener recorded it.""" opener's record."""
config = _config(tmp_path, ["alpha", "beta"]) config = _config(tmp_path, ["alpha", "beta"])
await _seed(config, "alpha", ["alpha document about cats"]) await _seed(config, "alpha", ["alpha document about cats"])
await _seed(config, "beta", ["beta document about cats"]) await _seed(config, "beta", ["beta document about cats"])
@ -145,7 +145,8 @@ class TestOpeningDatabases:
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_a_database_named_twice_is_opened_once(self, tmp_path): async def test_a_database_named_twice_is_opened_once(self, tmp_path):
"""Fusion would count a repeated database as two rank lists.""" """Fusion counts rank lists per database, so a repeated name
contributes one."""
config = _config(tmp_path, ["alpha", "beta"]) config = _config(tmp_path, ["alpha", "beta"])
await _seed(config, "alpha", ["alpha document about cats"]) await _seed(config, "alpha", ["alpha document about cats"])
await _seed(config, "beta", ["beta document about cats"]) await _seed(config, "beta", ["beta document about cats"])
@ -200,8 +201,8 @@ class TestReportingWhereADatabaseIs:
class TestClosingASet: class TestClosingASet:
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_every_database_opened_is_released(self, tmp_path): async def test_every_database_opened_is_released(self, tmp_path):
"""A covered database owns an embedder and may owe a vacuum. Closing only """A covered database owns an embedder and may owe a vacuum; closing
its connection would leave both behind.""" releases both."""
config = _config(tmp_path, ["alpha", "beta"]) config = _config(tmp_path, ["alpha", "beta"])
await _seed(config, "alpha", ["alpha one"]) await _seed(config, "alpha", ["alpha one"])
await _seed(config, "beta", ["beta one"]) await _seed(config, "beta", ["beta one"])
@ -257,9 +258,9 @@ class TestBorrowedDatabases:
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_entering_a_borrowed_client_reuses_its_database(self, tmp_path): async def test_entering_a_borrowed_client_reuses_its_database(self, tmp_path):
"""`async with` on a borrowed client is a plausible thing to write. """`async with` on a borrowed client is a plausible thing to write. It
Opening a second session would leak it, since teardown declines to close reuses the borrowed session: teardown declines to close what this client
what this client did not open.""" did not open."""
config = _config(tmp_path, ["alpha", "beta"]) config = _config(tmp_path, ["alpha", "beta"])
await _seed(config, "alpha", ["alpha document about cats"]) await _seed(config, "alpha", ["alpha document about cats"])
@ -375,8 +376,8 @@ class TestReleasingAClient:
class TestSharingTheReranker: class TestSharingTheReranker:
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_the_set_builds_and_closes_one_reranker(self, tmp_path, monkeypatch): async def test_the_set_builds_and_closes_one_reranker(self, tmp_path, monkeypatch):
"""A local reranker loads model weights, so one per database in a set """A local reranker loads model weights; the set builds one and shares
would load the same weights that many times.""" it."""
import haiku.rag.client as client_module import haiku.rag.client as client_module
built: list[object] = [] built: list[object] = []
@ -475,8 +476,8 @@ class TestFailureNaming:
assert caught.value.__cause__ is None assert caught.value.__cause__ is None
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_a_legacy_uri_client_keeps_its_error(self, tmp_path): async def test_an_unnamed_database_keeps_its_error(self, tmp_path):
"""Nothing named it, so there is no name to report instead.""" """Nothing named it, so there is no name to report."""
with pytest.raises(FileNotFoundError): with pytest.raises(FileNotFoundError):
async with HaikuRAG(tmp_path / "nope.lancedb"): async with HaikuRAG(tmp_path / "nope.lancedb"):
pass pass
@ -645,8 +646,7 @@ class TestDatabaseIndependentWork:
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_re_entering_a_set_builds_a_fresh_embedder(self, tmp_path): async def test_re_entering_a_set_builds_a_fresh_embedder(self, tmp_path):
"""Teardown closes the embedder, so keeping it would hand the next """Teardown closes the embedder; re-entry builds a fresh one."""
context one that is already closed."""
config = _config(tmp_path, ["alpha", "beta"]) config = _config(tmp_path, ["alpha", "beta"])
await _seed(config, "alpha", ["alpha one"]) await _seed(config, "alpha", ["alpha one"])

13
tests/multi_db/test_scope.py
View file

@ -109,8 +109,7 @@ class TestOneConfiguredLocation:
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_a_local_uri_that_does_not_exist_is_refused(self, tmp_path): async def test_a_local_uri_that_does_not_exist_is_refused(self, tmp_path):
"""A mistyped path fails instead of quietly becoming an empty database, """A schemeless location is a local path and must exist."""
which is what a value carrying a scheme would do."""
config = self._config(tmp_path / "typo.lancedb") config = self._config(tmp_path / "typo.lancedb")
with pytest.raises(FileNotFoundError): with pytest.raises(FileNotFoundError):
@ -253,14 +252,14 @@ class TestPlacingADatabase:
# A KeyError, so a caller treating selection as a lookup still catches it. # A KeyError, so a caller treating selection as a lookup still catches it.
assert issubclass(UnknownDatabaseError, KeyError) assert issubclass(UnknownDatabaseError, KeyError)
# ...but the message reads as a sentence rather than a missing key. # ...but the message reads as a sentence, not as a missing key.
assert str(UnknownDatabaseError("unknown database 'typo'")) == ( assert str(UnknownDatabaseError("unknown database 'typo'")) == (
"unknown database 'typo'" "unknown database 'typo'"
) )
def test_a_path_and_sources_cannot_both_choose(self, tmp_path): def test_a_path_and_sources_cannot_both_choose(self, tmp_path):
"""`sources` used to be ignored beside a path, so selecting a database """A path names one database and `sources` names others; together they
that is not the one at the path opened the path anyway.""" are refused, whatever the selection."""
config = _config(tmp_path, ["alpha", "beta"]) config = _config(tmp_path, ["alpha", "beta"])
for sources in ([], ["alpha"], ["nope"]): for sources in ([], ["alpha"], ["nope"]):
@ -269,8 +268,8 @@ class TestPlacingADatabase:
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_one_database_refuses_a_name_it_does_not_cover(self, tmp_path): async def test_one_database_refuses_a_name_it_does_not_cover(self, tmp_path):
"""Answering with itself would hand back the wrong database's reader for """A citation naming another database must not get this database's
a citation that named another.""" reader."""
config = _config(tmp_path, ["alpha", "beta"]) config = _config(tmp_path, ["alpha", "beta"])
await _seed(config, "alpha", ["alpha one"]) await _seed(config, "alpha", ["alpha one"])
await _seed(config, "beta", ["beta one"]) await _seed(config, "beta", ["beta one"])

20
tests/multi_db/test_search.py
View file

@ -110,8 +110,8 @@ class TestOneQueryVector:
class TestOneEmbedderAcrossTheSet: class TestOneEmbedderAcrossTheSet:
"""A set is searched with one query vector, so a database written with """A set is searched with one query vector, so the databases in a
another model would answer from a different space.""" selection must share an embedder."""
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_disagreeing_databases_cannot_be_searched_together(self, tmp_path): async def test_disagreeing_databases_cannot_be_searched_together(self, tmp_path):
@ -403,8 +403,8 @@ class TestOneReranker:
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_the_reranker_is_closed_once(self, tmp_path, monkeypatch): async def test_the_reranker_is_closed_once(self, tmp_path, monkeypatch):
"""Handing the same object to every database and letting each close it """The federator owns the reranker: it hands the same object to every
would close it N times, and the federator not at all.""" database and closes it once."""
closes = [] closes = []
class CountingReranker(StubReranker): class CountingReranker(StubReranker):
@ -431,8 +431,8 @@ class TestNarrowingToOneDatabase:
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_narrowing_keeps_the_database_s_own_scores(self, tmp_path): async def test_narrowing_keeps_the_database_s_own_scores(self, tmp_path):
"""RRF scores position, so fusing one ranking would report 1/(60+rank) """RRF scores position; a selection of one keeps the database's own
where the database reported a hybrid score.""" hybrid scores."""
config = _config(tmp_path, ["alpha", "beta"]) config = _config(tmp_path, ["alpha", "beta"])
await _seed(config, "alpha", ["alpha document about cats", "alpha on dogs"]) await _seed(config, "alpha", ["alpha document about cats", "alpha on dogs"])
await _seed(config, "beta", ["beta document about cats"]) await _seed(config, "beta", ["beta document about cats"])
@ -453,7 +453,7 @@ class TestNarrowingToOneDatabase:
self, tmp_path, monkeypatch self, tmp_path, monkeypatch
): ):
"""One database embeds inside the repository, which returns early when """One database embeds inside the repository, which returns early when
the filter matches no document. Fusing would have embedded first.""" the filter matches no document."""
from haiku.rag.embeddings import EmbedderWrapper from haiku.rag.embeddings import EmbedderWrapper
config = _config(tmp_path, ["alpha", "beta"]) config = _config(tmp_path, ["alpha", "beta"])
@ -486,8 +486,7 @@ class TestReciprocalRankFusion:
def _lopsided(self, count: int) -> list[list[tuple[Chunk, float]]]: def _lopsided(self, count: int) -> list[list[tuple[Chunk, float]]]:
"""Every native score in the first database beats every one in the """Every native score in the first database beats every one in the
second, so anything ranking by score rather than position puts all of second, so score order and position order disagree."""
one before any of the other."""
return [self._ranked("a", count, 0.9), self._ranked("b", count, 0.2)] return [self._ranked("a", count, 0.9), self._ranked("b", count, 0.2)]
async def _fuse_over(self, tmp_path, per_source, limit): async def _fuse_over(self, tmp_path, per_source, limit):
@ -640,8 +639,7 @@ class TestOneNamedDatabase:
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_selecting_nothing_means_the_same_with_one_database(self, tmp_path): async def test_selecting_nothing_means_the_same_with_one_database(self, tmp_path):
"""`sources=[]` selects nothing whether one database is configured or """`sources=[]` selects nothing whether one database is configured or
several, rather than raising on one path and returning nothing on the several."""
other."""
config = _config(tmp_path, ["alpha"]) config = _config(tmp_path, ["alpha"])
await _seed(config, "alpha", ["alpha document about cats"]) await _seed(config, "alpha", ["alpha document about cats"])

14
tests/sandbox/test_sandbox_multi_db.py
View file

@ -47,13 +47,13 @@ class TestSerializingTheConnection:
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_an_owner_is_not(self, tmp_path): async def test_an_owner_is_not(self, tmp_path):
"""Serializing owner reads would queue every database's file read behind """The lock guards the lent session's state, which owner reads do not
the capability's searches, to guard state none of them touch.""" touch: they take no lock."""
import asyncio import asyncio
class Trap(asyncio.Lock): class Trap(asyncio.Lock):
"""Refuses rather than waits: holding a real lock would wedge the """Raises on acquire, failing the test at the serialization
suite on a regression instead of failing it.""" point."""
async def acquire(self): async def acquire(self):
raise AssertionError("serialized a read on an owner's own session") raise AssertionError("serialized a read on an owner's own session")
@ -282,7 +282,7 @@ class TestExecutingAcrossDatabases:
class TestSelectionOnOneDatabase: class TestSelectionOnOneDatabase:
"""A client covering a single named database answers a selection the same way """A client covering a single named database answers a selection the same way
a search does, or the sandbox would mount what a search would refuse.""" a search does: the sandbox mounts what a search reaches."""
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_selecting_no_database_mounts_nothing(self, tmp_path): async def test_selecting_no_database_mounts_nothing(self, tmp_path):
@ -318,8 +318,8 @@ class TestSelectionOnOneDatabase:
class TestCopiedDatabases: class TestCopiedDatabases:
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_a_document_in_two_databases_is_refused(self, tmp_path): async def test_a_document_in_two_databases_is_refused(self, tmp_path):
"""Ids are unique per database, not across a copy of one: two documents """Ids are unique per database, not across a copy of one, so one path
would claim one path and the last would answer for both.""" cannot serve two documents."""
config = _config(tmp_path, ["alpha", "clone"]) config = _config(tmp_path, ["alpha", "clone"])
await _seed(config, "alpha", ["alpha document about cats"]) await _seed(config, "alpha", ["alpha document about cats"])
shutil.rmtree(tmp_path / "clone.lancedb", ignore_errors=True) shutil.rmtree(tmp_path / "clone.lancedb", ignore_errors=True)

7
tests/store/test_engine.py
View file

@ -9,8 +9,7 @@ class TestStoredSettings:
async def test_a_new_database_carries_the_version_it_was_created_with( async def test_a_new_database_carries_the_version_it_was_created_with(
self, temp_db_path self, temp_db_path
): ):
"""Creating writes the settings row, so the store has to report it """Creating writes the settings row, and the store reports it."""
rather than the emptiness it opened on."""
from importlib import metadata from importlib import metadata
async with Store(temp_db_path, create=True) as store: async with Store(temp_db_path, create=True) as store:
@ -20,8 +19,8 @@ class TestStoredSettings:
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_an_existing_database_carries_its_stored_settings(self, temp_db_path): async def test_an_existing_database_carries_its_stored_settings(self, temp_db_path):
"""Read once on open: reporting on a database reads them from here """Read once on open: reporting on a database reads them from
instead of querying the settings table again.""" here."""
async with Store(temp_db_path, create=True) as store: async with Store(temp_db_path, create=True) as store:
written = store.stored_settings written = store.stored_settings

5
tests/test_app.py
View file

@ -428,9 +428,8 @@ def test_show_settings_hides_secrets(tmp_path):
def test_show_settings_renders_the_shape_a_config_file_has(tmp_path): def test_show_settings_renders_the_shape_a_config_file_has(tmp_path):
"""Nesting is indented rather than flattened into one dict repr per block, """Nesting is indented and a path is its string: what is read here is what
and a path is its string rather than its Python repr, so what is read here `haiku.rag.yaml` holds."""
is what would be written into `haiku.rag.yaml`."""
import yaml import yaml
config = AppConfig( config = AppConfig(

19
tests/test_cli.py
View file

@ -141,8 +141,8 @@ class TestOneDatabaseCommands:
assert "bucket" not in str(raised.value) assert "bucket" not in str(raised.value)
def test_a_configured_set_of_one_needs_no_choosing(self, monkeypatch): def test_a_configured_set_of_one_needs_no_choosing(self, monkeypatch):
"""Nothing is ambiguous about a set with one database in it, and it keeps """Nothing is ambiguous about a set with one database in it, and it
its name rather than being refused.""" keeps its name."""
self._install(monkeypatch, alpha="/db/a.lancedb") self._install(monkeypatch, alpha="/db/a.lancedb")
assert resolve_scope(None).names == ("alpha",) assert resolve_scope(None).names == ("alpha",)
@ -189,8 +189,8 @@ class TestSelectingADatabaseByName:
set_config(AppConfig(lancedb=LanceDBConfig(databases=databases))) set_config(AppConfig(lancedb=LanceDBConfig(databases=databases)))
def test_a_named_database_is_passed_on_by_name(self, monkeypatch): def test_a_named_database_is_passed_on_by_name(self, monkeypatch):
"""Not resolved to a path: the name is what results and citations carry, """Not resolved to a path: the name is what results and citations
and rewriting the configuration is what used to lose it.""" carry."""
self._install(monkeypatch, papers="s3://bucket/prefix/papers.lancedb") self._install(monkeypatch, papers="s3://bucket/prefix/papers.lancedb")
monkeypatch.setattr("haiku.rag.cli._db_name", "papers") monkeypatch.setattr("haiku.rag.cli._db_name", "papers")
@ -271,8 +271,8 @@ class TestSelectingADatabaseByName:
def test_a_selection_does_not_outlive_its_invocation_in_process( def test_a_selection_does_not_outlive_its_invocation_in_process(
self, tmp_path, monkeypatch self, tmp_path, monkeypatch
): ):
"""The selection is per invocation, so a second one starts from a """The selection is per invocation: a second one starts from a freshly
freshly loaded configuration rather than inheriting the first.""" loaded configuration."""
import haiku.rag.cli as cli_module import haiku.rag.cli as cli_module
import haiku.rag.config as config_module import haiku.rag.config as config_module
@ -419,8 +419,7 @@ class TestConfiguredLocalUri:
assert self._reports(result, located) assert self._reports(result, located)
def test_a_missing_configured_path_is_refused(self, tmp_path, monkeypatch): def test_a_missing_configured_path_is_refused(self, tmp_path, monkeypatch):
"""A schemeless value is a local path, so a typo fails instead of """A schemeless value is a local path and must exist."""
becoming a new empty database."""
self._fresh(monkeypatch) self._fresh(monkeypatch)
located = tmp_path / "typo.lancedb" located = tmp_path / "typo.lancedb"
config_file = self._config_file(tmp_path, located) config_file = self._config_file(tmp_path, located)
@ -726,8 +725,8 @@ class TestAskAnalyzeImageOption:
class TestChatCoversTheSet: class TestChatCoversTheSet:
"""Chat is a read verb: it answers with the same capabilities `ask` uses, so """Chat is a read verb: it answers with the same capabilities `ask` uses,
it covers the configured set rather than demanding one database.""" and covers the configured set."""
@staticmethod @staticmethod
def _config_file(tmp_path): def _config_file(tmp_path):

12
tests/test_database_scope.py
View file

@ -28,7 +28,7 @@ class TestResolution:
def test_a_path_names_one_unnamed_database(self): def test_a_path_names_one_unnamed_database(self):
"""A path says which database, not what it is called, even where the """A path says which database, not what it is called, even where the
configuration would have named it.""" configuration names one."""
config = _config(databases={"alpha": "/data/alpha.lancedb"}) config = _config(databases={"alpha": "/data/alpha.lancedb"})
scope = DatabaseScope.resolve(config, database_path=Path("/data/other.lancedb")) scope = DatabaseScope.resolve(config, database_path=Path("/data/other.lancedb"))
@ -108,8 +108,8 @@ class TestResolution:
assert scope.databases == (DatabaseRef.at(tmp_path / "haiku.rag.lancedb"),) assert scope.databases == (DatabaseRef.at(tmp_path / "haiku.rag.lancedb"),)
def test_the_environment_is_not_consulted(self, monkeypatch, tmp_path): def test_the_environment_is_not_consulted(self, monkeypatch, tmp_path):
"""HAIKU_RAG_DB is honoured by the capability entry point alone. Reading it """HAIKU_RAG_DB is honoured by the capability entry point alone;
here would change what every other caller opens.""" resolution never reads the environment."""
monkeypatch.setenv("HAIKU_RAG_DB", "/data/from-the-environment.lancedb") monkeypatch.setenv("HAIKU_RAG_DB", "/data/from-the-environment.lancedb")
config = _config(databases={"alpha": "/data/alpha.lancedb"}) config = _config(databases={"alpha": "/data/alpha.lancedb"})
@ -118,8 +118,7 @@ class TestResolution:
assert scope.names == ("alpha",) assert scope.names == ("alpha",)
def test_a_path_is_never_reinterpreted_as_a_uri(self): def test_a_path_is_never_reinterpreted_as_a_uri(self):
"""A caller naming a path means that path. Sending it back through the """A caller naming a path means that path, whatever scheme it carries."""
configured-location rules would let a scheme turn it into a URI."""
scope = DatabaseScope.resolve( scope = DatabaseScope.resolve(
_config(), database_path="s3://bucket/looks-like-a-uri.lancedb" _config(), database_path="s3://bucket/looks-like-a-uri.lancedb"
) )
@ -139,8 +138,7 @@ class TestResolution:
assert ref.db_path is None assert ref.db_path is None
def test_a_database_is_a_uri_or_a_path(self): def test_a_database_is_a_uri_or_a_path(self):
"""Both would silently ignore the path; neither fails later, when the """A ref holding both, or neither, is refused at construction.
connection is derived and there is nothing to open.
The message names what it was given: this is a programming error raised The message names what it was given: this is a programming error raised
in the caller's own process, not one an operator or a model ever sees. in the caller's own process, not one an operator or a model ever sees.

9
tests/test_info.py
View file

@ -167,8 +167,7 @@ async def test_app_info_with_vector_index(temp_db_path, capsys):
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_app_info_opens_a_named_remote_database(tmp_path): async def test_app_info_opens_a_named_remote_database(tmp_path):
"""A database named in `lancedb.databases` can sit behind a URI while the """A database named in `lancedb.databases` can sit behind a URI while the
configuration's own `uri` is empty. Passing that configuration on would open configuration's own `uri` is empty; info derives and opens the URI."""
the local path that only stands in for it."""
from haiku.rag.client.scope import DatabaseScope from haiku.rag.client.scope import DatabaseScope
config = AppConfig( config = AppConfig(
@ -195,8 +194,8 @@ async def test_app_info_opens_a_named_remote_database(tmp_path):
async def test_app_doctor_opens_a_named_remote_database(): async def test_app_doctor_opens_a_named_remote_database():
"""`run_doctor` connects with the configuration it is handed, so it needs the """`run_doctor` connects with the configuration it is handed: the one
one derived for the database rather than the one naming the set.""" derived for the database, not the one naming the set."""
from haiku.rag.client.scope import DatabaseScope from haiku.rag.client.scope import DatabaseScope
config = AppConfig( config = AppConfig(
@ -385,7 +384,7 @@ async def test_app_init_skips_exists_check_for_remote(tmp_path):
covering.__aenter__ = AsyncMock(return_value=mock_client) covering.__aenter__ = AsyncMock(return_value=mock_client)
covering.__aexit__ = AsyncMock(return_value=False) covering.__aexit__ = AsyncMock(return_value=False)
await app.init() await app.init()
# Opened to create, rather than returning early on a missing local path. # A missing local path is opened to create, not returned early on.
mock_client_cls._covering.assert_called_once() mock_client_cls._covering.assert_called_once()

8
tests/test_inspector.py
View file

@ -369,8 +369,8 @@ class TestReportingReusesTheConnection:
async def test_statistics_come_from_the_open_connection( async def test_statistics_come_from_the_open_connection(
self, tmp_path, monkeypatch self, tmp_path, monkeypatch
): ):
"""The client already holds a connection to the database being reported, """Statistics are read through the connection the client already
so opening a second one would be an open for the same statistics.""" holds."""
from haiku.rag.inspector.widgets.info_modal import database_lines from haiku.rag.inspector.widgets.info_modal import database_lines
from haiku.rag.store.engine import ConnectionMode from haiku.rag.store.engine import ConnectionMode
@ -403,8 +403,8 @@ class TestReportingReusesTheConnection:
async def test_settings_come_from_the_store_that_parsed_them( async def test_settings_come_from_the_store_that_parsed_them(
self, tmp_path, monkeypatch self, tmp_path, monkeypatch
): ):
"""The store read and parsed the settings blob on open, so reporting """The store read and parsed the settings blob on open, and reporting
reads it from there instead of querying the settings table again.""" reads it from there."""
from haiku.rag.inspector.widgets.info_modal import database_lines from haiku.rag.inspector.widgets.info_modal import database_lines
from haiku.rag.store.engine import ConnectionMode from haiku.rag.store.engine import ConnectionMode

9
tests/test_mcp.py
View file

@ -608,9 +608,8 @@ class TestMCPClientLifetime:
async def test_the_scope_decides_the_database_and_names_its_results( async def test_the_scope_decides_the_database_and_names_its_results(
self, mcp_db, tmp_path self, mcp_db, tmp_path
): ):
"""The scope is the selection, so the server reads the one database it """The scope is the selection: the server reads the one database it
names and results carry that name. The configuration alone would place names, and results carry that name."""
every database it configures."""
from haiku.rag.client.scope import DatabaseScope from haiku.rag.client.scope import DatabaseScope
from haiku.rag.config.models import AppConfig, LanceDBConfig from haiku.rag.config.models import AppConfig, LanceDBConfig
@ -670,8 +669,8 @@ class TestMCPClientLifetime:
async def test_the_command_hands_the_server_its_resolved_database( async def test_the_command_hands_the_server_its_resolved_database(
self, monkeypatch self, monkeypatch
): ):
"""A path would override a configured URI, and a derived configuration """`run_mcp` passes the resolved scope, not a path and not a derived
would lose the name, so `run_mcp` passes neither.""" configuration: the scope keeps both the URI and the name."""
from haiku.rag.app import HaikuRAGApp from haiku.rag.app import HaikuRAGApp
from haiku.rag.client.scope import DatabaseScope from haiku.rag.client.scope import DatabaseScope
from haiku.rag.config.models import AppConfig, LanceDBConfig from haiku.rag.config.models import AppConfig, LanceDBConfig

5
tests/test_s3_integration.py
View file

@ -68,9 +68,8 @@ def config():
def _remote_scope(config: AppConfig) -> DatabaseScope: def _remote_scope(config: AppConfig) -> DatabaseScope:
"""The configured S3 database. """The configured S3 database.
These tests name no path, so the scope must resolve to the URI. A precedence These tests name no path, and the assertion pins that the scope resolved to
change that let a path win would otherwise move them to the local disk and the URI.
leave them passing against nothing.
""" """
scope = DatabaseScope.resolve(config) scope = DatabaseScope.resolve(config)
[ref] = scope.databases [ref] = scope.databases