Store, connect_lancedb, gather_database_info and run_doctor take a
location, a path or a URI, and classify it with ConnectionMode.of.
SingleDatabaseSession owns the resolved DatabaseRef and passes its
location down. This removes DatabaseRef.connection(), default_db_path,
the placeholder path for URI-backed databases and the per-database
config copies, so the configuration a client holds is the one the caller
gave it. The chat hands its capabilities the scope it opened along with
the client it lends, and the v0.58.0 migration no longer checks local
free disk for a database behind a URI.
Sibling searches emitted in one response overlap heavily (40.6% of
returned chunk slots on Glimmer ORB fan-out cases). A result whose
rendered evidence a sibling already showed keeps its rank slot but
collapses to a reference line, and a picture attaches once per response
keyed on (source, document_id, self_ref). Equivalence is the
format_for_agent rendering at neutral rank/total plus picture keys,
bucketed under the qualified chunk id, so another database's copy or a
different expansion of the same anchor formats in full.
Search state now commits only after formatting and image construction
succeed: a raising image build no longer leaves results citable that
the model never saw, notes evidence for them, or suppresses a later
sibling.
Searches a model emits in one response share a budget unit, up to
FREE_SIBLINGS_PER_ROUND (3) per unit; sequential searches pay one unit
each, as before. Grouping keys on RunContext.run_step, which pydantic-ai
increments once per model request. A budget-rejected round fails all its
remaining siblings, and tracking resets per run.
Glimmer opens most questions with a burst of ~3 rephrasings in a single
response (95.8% of its three-search ORB cases are one-response bursts),
spending 3 of 5 searches before reading anything. Pass rate at 3 calls
equals 1 call, so a burst is priced as one probe.
Retrieval scores are each database's own rank arithmetic; the databases
in a selection share an embedder, so similarity in that one space is the
signal comparable across databases by construction. Measured product to
product against score ordering: +8.3 to +16.6pp recall@5 across five
cells on two corpora, flat in collection count and corpus shape where
score ordering dips with both, closing roughly 60% of the gap to a
reranker; order-sensitivity residual 0.00pp in every cell. Exact ties
collapse from 51-81% of candidates to under 1%. Full-text-only searches
keep retrieval-score order, having no query vector. The vector column
already travels with every search result, so the similarity costs no
additional transfer; per-chunk embeddings are materialized only for the
federated path that reads them.
Rank interleaving guarantees every database slots regardless of content;
on domain-split collections it allocates no better than chance and costs
4.7pp recall@5 at four collections against score ordering (7.1pp at
eight). Hybrid scores are each database's own vector/FTS rank agreement,
which carries across databases; equal scores resolve by within-database
rank, and only a tie on both falls to configured order, leaving
permutation sensitivity at 0.02-0.26pp. Fused results carry the
candidate's own retrieval score, so the context-expansion re-sort
preserves fused order.
Disjoint corpora give every database's rank-r candidate the same RRF
score, and the stable sort resolved those ties to lancedb.databases
declaration order, discarding the retrieval scores entirely. Ties now
break on the raw retrieval score, which is uncalibrated across indexes
but only ever orders candidates within one rank tier: the databases in
a fusion share an embedder and ran the same search type, and it can
never lift a candidate above another rank. Hybrid per-database scores
are themselves rank-derived, so exact agreement still ties and keeps
configured order, deterministically. The n > limit depth quota is
unchanged, pending the retrieval eval.
`_rich_print_document` escapes uri, title and metadata, the sibling of
the escaped search-result renderer. The remaining comments and
docstrings that narrated rejected alternatives, consequences or history
now state the current invariant. The Sandbox class docstring names the
held connection close() releases, and wrapped docs paragraphs join to
one line.
Comments and docstrings across the branch narrated rejected
alternatives, consequences and history; each now states the current
contract. Renames test_a_legacy_uri_client_keeps_its_error to
test_an_unnamed_database_keeps_its_error. Documents the Sandbox
connection paths, the citation header's database segment, both
AmbiguousDatabaseError conditions on create_app, and run_inspector's
scope parameter. Doc paragraphs added by the branch in python.md,
storage.md and cli.md are one physical line each.
`_fetch_limit` had no test: a text query over-fetches `limit * 10` only with a
reranker, and an image query keeps its vector ranking either way. The sandbox
test asserted `KeyError`, which `UnknownDatabaseError` subclasses, so it could
not tell the contract from a bare one.
`uses_configured_databases` documents a mapping of one as covered; the test
named for it passed no mapping at all. Its `config` parameter is an `AppConfig`.
`test_an_analysis_capability_mounts_the_configured_set` carried a VCR marker and
no cassette, making no HTTP calls.
`resolve_document` and `find_document` selected a document through a listing,
then dropped its source and looked the id up across the set. Ids repeat between
copies of a database, so a title that matched in one could be answered by
another's document.
`get_document_by_id` and `get_chunk_by_id` join `get_picture_bytes` in taking an
optional `source`, and all three route it through `clients_covering`, so a name
the client does not cover raises `UnknownDatabaseError` rather than being
answered by the database it does cover. Without a source the reads are as they
were, answering from the first database in configured order that holds the id.
`_resolve_scope` returned a scope without keeping it, so a client re-entered
after its configuration was edited covered whatever the configuration then said.
Resolving once is what the rest of the design rests on: the scope is what names
results, citations and errors.
A configuration is still free to change before first entry.
Search results and capsule entries name the collection they came from; the
images attached beside them carried only the chunk id and reference. Two
collections can return the same picture of the same document, so the two
labels were identical and the model could place neither.
The decision is the one already made for the text: `covers_multiple` at the
generic search tool, and the flag `search_corpus` computed for the capability
tools, which it now returns.
`asyncio.gather` propagates the first failure while its siblings run on, and the
caller unwinding from that closes the set through `async with` — so a sibling
still reading reads through a closed session. Seven fan-outs were affected:
lookup, search, image enrichment, multimodal picture loading, context expansion,
document listing and counting, and the sandbox's document load.
`gather_all` cancels and drains the rest, then re-raises the original exception.
A `TaskGroup` would drain them too but raise an `ExceptionGroup`, which every
caller and both CLIs' exception handlers would have to unwrap. The two
`return_exceptions=True` gathers in session opening and teardown already drain
their children and are left alone.
Reads tagged the document they returned and writes did not, so creating in
`alpha` came back with `source=None` while reading the same document came back
with `"alpha"`. Every outward write result goes through the owning session.
`test_an_image_query_builds_no_reranker` never submitted an image query: it
opened clients and found nothing built, which is lazy construction. It sends one
now, so consulting the reranker before the query type fails it.
`TestComparingEmbedders` called `_require_one_embedder` and asserted nothing.
It rejects a disagreement and accepts an absent record, and searches through.
Three `ModelRetry` tests took any reason where the reason is the subject.
Creating re-read the settings blob and left `stored_embedding` at None, so a
client that created a database compared as though it recorded no embedder.
`_remember_settings` takes both, and the comment no longer says one follows the
other.
A name nothing covers raised `KeyError` in four places and
`AmbiguousDatabaseError` in a fifth, so a caller had to catch both and neither
name said what happened. `UnknownDatabaseError` is all of them, exported from
`haiku.rag.store` beside the other errors.
It subclasses `KeyError`, since selecting by name is a lookup, and prints its
message plainly rather than quoted as a missing key. Both CLIs turn it into the
same clean exit they already gave the others.
`close()` refused a client covering a set, which left one with no method to
call: `async with` was the only lifecycle it had. `aclose()` runs that teardown
for a caller that owns the client some other way, whatever it covers, and
nothing to release is not an error, so it is safe before entering and after
closing.
`close()` stays what it is, one connection and nothing else, and says so:
draining the background vacuum and releasing the embedder and reranker are
awaitable.
`ask(sources=["typo"])` reached the model, which discovered the name only if it
searched: requests spent on a selection that could never answer, and a run that
never searched answered anyway.
Checked by name, not by opening: a client covering a set opens a database when a
query reaches it, and validating by opening would open every one of them before
any search, letting a database nobody asked about fail the run.
Every client covering a database built its own, so a set of five loaded the same
local model weights five times over, beside the federator's. A client covering a
database for another borrows that one's, and only its owner closes it.
`_fuse` ran in tests without anything asserting its output, so reversing the
sort or dropping the truncation changed nothing. Databases interleave by rank,
the score is the reciprocal of it, ties keep configured order, and the limit
cuts the fused list.
Every native score in one database beats every one in the other, so ranking by
score rather than position fails all four.
`create_mcp_server` promised one database and accepted a scope covering a set,
where the write tools exist and fail on use. It refuses that now.
Resolving is the public factory's job, as it is `HaikuRAG`'s: `_covering` takes
a scope someone already resolved, so the configured name survives without a
`DatabaseScope` reaching the public signature.
The test that a scope decides the database asserted `all(...)` over a search
that could return nothing, which held whatever the server read. It reads the
listing instead, so alpha's documents being present and beta's absent both have
to be true.
The test searched through the capability, which embeds the query, so it passed
against a local Ollama and failed CI where there is none. What it exists to show
is that a lent client is what the capability reads through: `_ensure_rag`
returns it, and a full-text search through it names its results.
`sources` was ignored beside a database path, so `sources=["nope"]` opened the
path and read as though the selection had been honoured. A path and a name
already conflict inside `DatabaseScope.resolve`; this is the same rule where a
caller can reach it.
`reader_for` returned itself before looking at the name, so an "alpha" client
answered `reader_for("beta")` with alpha's reader. A citation naming another
database would have been read from the wrong one.
Both paths decide through `clients_covering` now, so one database refuses a
wrong name the way a set already did.
The chat citation test said mounting lends the client without showing it; a
chat test shows it and the docstring points there.
`reader_for` documented None as covering any database it could not place, but a
name outside the set raises `KeyError` like `clients_for` does: provenance
naming a database this client does not cover is wrong rather than absent. None
means one thing, a federated client given no name.
`sources=[]` means two things. On a search it selects nothing to search; on the
constructor it raises, since a client over no database can do nothing. Both are
written down now.
A capability reads through a lent client, so what a citation records is that
client's database and not the scope the capability was built with. Chat lends
one, and had no test saying so.
`sources=["alpha"]` on a client covering a set went through fusion, which
scores position: a result the database ranked at 0.6549 was reported as
1/(60+rank). Embedding also moved ahead of the repository, so a filter matching
no document embedded the query anyway.
A selection of one now runs the single-database search. Fusion reconciles
rankings from separate indexes, and one ranking has nothing to reconcile.
A reranker that returns chunks it built rather than the ones it was given loses
which database each came from, since ownership is by identity. That is named
now instead of surfacing as a KeyError, and stated on `RerankerBase._rerank`.
The documented rule was existence in more than one selected database. The rule
is narrower and does not need to be wider: a copy the search never returned
grounded nothing, so the retrieved result is the citation and its database is
recorded rather than guessed.
The page now states the whole rule, including the fallback's own check for an
id no search returned. Two retrieved copies are refused, and so is an id
already cited from another database in the conversation. The case between them
had no test.
Collision detection replaced the lookup's dict comprehension with
`setdefault`, which also flipped a chunk found by several searches from its
last occurrence to its first. The copies differ in everything the expansion
window decides, figures included, so that silently changed what a citation
renders.
The rules are separate now: a repeated (source, chunk_id) takes the later
result, a chunk_id under two sources is still refused.
`format_for_agent` named the database whenever one was named, so a search over
a single named database carried a line with nothing to distinguish. It now takes
`include_collection` from the caller, which decides from the search selection
rather than from the hits: a search that could have drawn on two collections
names them even when everything came back from one.
`Collection:` at the model boundary, database in configuration and
administration. `source` on results, documents, citations and analysis
dictionaries is unchanged.
A URI-backed database is constructed with a local `db_path` nothing connects
to, and the info modal reported that, so `--db-name` on an S3 database named a
local directory. `location` reads the configuration instead, and the modal
labels it as one.
`InfoModal`'s `db_path` argument is gone: both callers passed None, so only a
test reached the branch that used it.
`sessions_for` recorded what opened from `gather`'s results, which arrive only
when it runs to completion. Cancelling it discarded them, so a database that
opened while a sibling was still pending was never recorded and `aclose` never
closed it. `return_exceptions=True` covers a failing child, not a cancelled
parent.
`_open` now registers its own session, so what opened is reachable however the
fan-out ends.
`--database` and `--db` read as one word abbreviated but take different
things, a configured name against a filesystem path, and they sit in
different positions: `haiku-rag --db /path info` fails with `No such option`.
The CLI already says "database" to the user everywhere it prints one, so the
name it selects by should say the same. Unreleased, so no deprecation.
Also: without a reranker, raise `search.limit` with the number of databases
searched, since each contributes its best matches to a list truncated back
to the limit.
Two files of 1,405 and 814 lines become seven: scope resolution, lifecycle,
search, documents, expansion, citations and capabilities. `_config`, `_seed`
and the rest move to `helpers.py`, importable by the sandbox tests that share
them, and the package points VCR back at `tests/cassettes/multi_db/`.
