A database name, document title, uri or heading containing Rich markup
crashed search output, chat citations and the chat document filter with
MarkupError. Every interpolation into markup-parsed text now escapes.
`_fetch_limit` had no test: a text query over-fetches `limit * 10` only with a
reranker, and an image query keeps its vector ranking either way. The sandbox
test asserted `KeyError`, which `UnknownDatabaseError` subclasses, so it could
not tell the contract from a bare one.
`uses_configured_databases` documents a mapping of one as covered; the test
named for it passed no mapping at all. Its `config` parameter is an `AppConfig`.
`test_an_analysis_capability_mounts_the_configured_set` carried a VCR marker and
no cassette, making no HTTP calls.
Comments that narrated a failure mode across three or four lines say the
invariant they protect instead: the repository's late embedding, one database
keeping its hybrid scores, one over-fetch decision for a selection, and a cite
fallback that covers exactly what the question covers.
`tag_restore` reached into `HaikuRAGApp._display_path` and `_path`, and seven
places spelled out `self._is_local and not self._path.exists()`. `display_path`
and `database_missing` say both, and `database_missing` is False for a database
behind a URI, which has no path to check.
`init` keeps its own check: it asks the opposite question.
A database name, a location and an exception message all reach the info modal
as configuration-derived text, and Rich reads `[...]` in any of them as markup:
a name like `beta [prod]` disappeared, and a message carrying `[Errno 2]` or a
stray closing tag could break the line it sat in.
Two collections can hold documents with the same title, and the collapsed
citation carried only that title, so the two read as one source repeated. The
name is appended when the client covers a set, the decision already made for
search results and retrieved images.
`resolve_document` and `find_document` selected a document through a listing,
then dropped its source and looked the id up across the set. Ids repeat between
copies of a database, so a title that matched in one could be answered by
another's document.
`get_document_by_id` and `get_chunk_by_id` join `get_picture_bytes` in taking an
optional `source`, and all three route it through `clients_covering`, so a name
the client does not cover raises `UnknownDatabaseError` rather than being
answered by the database it does cover. Without a source the reads are as they
were, answering from the first database in configured order that holds the id.
`_resolve_scope` returned a scope without keeping it, so a client re-entered
after its configuration was edited covered whatever the configuration then said.
Resolving once is what the rest of the design rests on: the scope is what names
results, citations and errors.
A configuration is still free to change before first entry.
Search results and capsule entries name the collection they came from; the
images attached beside them carried only the chunk id and reference. Two
collections can return the same picture of the same document, so the two
labels were identical and the model could place neither.
The decision is the one already made for the text: `covers_multiple` at the
generic search tool, and the flag `search_corpus` computed for the capability
tools, which it now returns.
`asyncio.gather` propagates the first failure while its siblings run on, and the
caller unwinding from that closes the set through `async with` — so a sibling
still reading reads through a closed session. Seven fan-outs were affected:
lookup, search, image enrichment, multimodal picture loading, context expansion,
document listing and counting, and the sandbox's document load.
`gather_all` cancels and drains the rest, then re-raises the original exception.
A `TaskGroup` would drain them too but raise an `ExceptionGroup`, which every
caller and both CLIs' exception handlers would have to unwrap. The two
`return_exceptions=True` gathers in session opening and teardown already drain
their children and are left alone.
Three places identified a picture by document and reference alone, and one
identified a citation's images by chunk id alone. Both repeat between copies of
a database, so a search returning a figure from two collections sent one, a
capsule retained one, and a citation rendered the other collection's figures.
Keyed on the source as well: `(source, document_id, self_ref)` for search
pictures, the capability and source for retained ones, and `qualified_id` for
the chat's citation images.
A capsule replaces earlier questions' evidence with what they cited, so the
searches that carried a `Collection:` line are gone by the time the model reads
it. Cited content survived; which collection it came from did not, and a
follow-up question attributed it to whichever the model guessed.
Named the same way a search result names it: the capsule decides, and only when
it spans more than one.
`ConfigMismatchError` and `SourceUnavailableError` escaped the entry point as
tracebacks rather than a message and exit code 1. The entry-point test is
parametrized over the translated types, so the two CLIs' lists cannot drift
apart silently again.
Typing narrowed the mounted checkboxes without touching the search the listing
was built from, so a term matching more than one page hid rows from whichever
page the user happened to be on, left the rest of the matches a page away, and
paged on the previous search while the term still sat in the box. The term now
applies on enter, and the footer says so until it does.
The set shares the reranker its facades borrow, not an embedder: each covered
database builds and closes its own. Sharing one needs the lender pattern inside
`Store`, since a store's embedder also serves writes and is closed with the
session, so it is left as follow-up rather than done here.
`reported_location` returned `client.location` and nothing else; its test
asserted that a mock returns what it was given. `list_documents` names its
results through `name_all`.
Four passages said every missing database raises `FileNotFoundError` and that
errors show locations. A database named in `lancedb.databases` raises
`SourceUnavailableError` instead, naming the database and not its location,
which is the point of naming them. A path you gave keeps `FileNotFoundError` and
still shows the path.
The lock was applied to an owner as well, so every owner-backed file read queued
behind the capability's tool calls to guard state it does not touch. An owner is
a session of its own and is yielded straight through, which is what the
docstring already claimed.
Chat asked the covering client for a picture with the citation's source, which
raises when there is none, losing the answer to one figure. It resolves the
reader first and omits the bytes when the database cannot be placed, as the rich
formatter already did.
Reads tagged the document they returned and writes did not, so creating in
`alpha` came back with `source=None` while reading the same document came back
with `"alpha"`. Every outward write result goes through the owning session.
`settings` and `download-models` read the configuration and open nothing, but
resolved a scope to be constructed, so `--db-name nope` failed them over a
selection they never use. `HaikuRAGApp` takes no scope for those, and asking it
for one is an error rather than a silent default.
`--db-name` selects one database, not a subset: it is not repeatable.
The reranker recommendation was one-sided. It reports both measured effects now:
stronger aggregate retrieval across shards, and weaker attribution between
near-identical documents, where fusion keeps twins apart because each database
contributes its own top-ranked result.
Image queries are vector-only and skip the reranker.
The changelog described components of the new feature as fixes to the last
release. They are one Added entry, and the changes a user upgrading does see —
`settings` printing YAML, the document filter paging and searching, `list`
printing only the fields a document has — are listed.
`test_an_image_query_builds_no_reranker` never submitted an image query: it
opened clients and found nothing built, which is lazy construction. It sends one
now, so consulting the reranker before the query type fails it.
`TestComparingEmbedders` called `_require_one_embedder` and asserted nothing.
It rejects a disagreement and accepts an absent record, and searches through.
Three `ModelRetry` tests took any reason where the reason is the subject.
`haiku-ingester` reached the client's refusal, which names `sources=[name]`, a
Python argument no CLI user can pass. It refuses a configured set itself, with
`--db PATH` and the one-ingester-per-database layout.
`haiku-rag`'s own refusal now says where each option goes: `--db-name` is global
and precedes the command, `--db` follows it.
Creating re-read the settings blob and left `stored_embedding` at None, so a
client that created a database compared as though it recorded no embedder.
`_remember_settings` takes both, and the comment no longer says one follows the
other.
The lock claimed to serialize a whole read, while `_documents` resolves the
owners under it and then reads through them outside. That is right: resolving
the owners is an operation on the shared connection, reading through them is
not, and each owner is a session of its own. Holding the lock across a read per
database would serialize them against the capability's searches to guard state
none of them touch.
A name nothing covers raised `KeyError` in four places and
`AmbiguousDatabaseError` in a fifth, so a caller had to catch both and neither
name said what happened. `UnknownDatabaseError` is all of them, exported from
`haiku.rag.store` beside the other errors.
It subclasses `KeyError`, since selecting by name is a lookup, and prints its
message plainly rather than quoted as a missing key. Both CLIs turn it into the
same clean exit they already gave the others.
`close()` refused a client covering a set, which left one with no method to
call: `async with` was the only lifecycle it had. `aclose()` runs that teardown
for a caller that owns the client some other way, whatever it covers, and
nothing to release is not an error, so it is safe before entering and after
closing.
`close()` stays what it is, one connection and nothing else, and says so:
draining the background vacuum and releasing the embedder and reranker are
awaitable.
A selection outside the page stayed applied while its checkbox was gone, so
there was no way to remove it. Appending those documents to the page instead
loses the bound the page exists to keep, since selections accumulate across
searches.
Both listings page at `DOCUMENT_PAGE`, and `Selected` switches between them, so
the mounted widgets stay bounded whichever is showing and every selection is a
page away rather than unreachable.
The count reads the checkboxes on screen, so narrowing as the user types reports
what is visible. A listing with nothing in it says so, instead of leaving an
empty box that reads as still loading.
`ask(sources=["typo"])` reached the model, which discovered the name only if it
searched: requests spent on a selection that could never answer, and a run that
never searched answered anyway.
Checked by name, not by opening: a client covering a set opens a database when a
query reaches it, and validating by opening would open every one of them before
any search, letting a database nobody asked about fail the run.
Every client covering a database built its own, so a set of five loaded the same
local model weights five times over, beside the federator's. A client covering a
database for another borrows that one's, and only its owner closes it.
`format_citations_rich` asked a client covering a set for a picture whose
citation named no database, which raises, losing the whole answer to one figure.
Citations recorded before databases could be named carry no source, so the
figure marker the caller already renders stands in.
`_fuse` ran in tests without anything asserting its output, so reversing the
sort or dropping the truncation changed nothing. Databases interleave by rank,
the score is the reciprocal of it, ties keep configured order, and the limit
cuts the fused list.
Every native score in one database beats every one in the other, so ranking by
score rather than position fails all four.
`create_mcp_server` promised one database and accepted a scope covering a set,
where the write tools exist and fail on use. It refuses that now.
Resolving is the public factory's job, as it is `HaikuRAG`'s: `_covering` takes
a scope someone already resolved, so the configured name survives without a
`DatabaseScope` reaching the public signature.
The test that a scope decides the database asserted `all(...)` over a search
that could return nothing, which held whatever the server read. It reads the
listing instead, so alpha's documents being present and beta's absent both have
to be true.
The test searched through the capability, which embeds the query, so it passed
against a local Ollama and failed CI where there is none. What it exists to show
is that a lent client is what the capability reads through: `_ensure_rag`
returns it, and a full-text search through it names its results.
`sources` was ignored beside a database path, so `sources=["nope"]` opened the
path and read as though the selection had been honoured. A path and a name
already conflict inside `DatabaseScope.resolve`; this is the same rule where a
caller can reach it.
`run_mcp` derived a path and a configuration for the server, and deriving drops
the configured name: MCP results and citations carried `source=None` where the
same database named through any other path carried "alpha".
`create_mcp_server` takes the resolved scope and opens through it, so the name
survives. Passing a path and configuration still works and resolves to the same
place.
`reader_for` returned itself before looking at the name, so an "alpha" client
answered `reader_for("beta")` with alpha's reader. A citation naming another
database would have been read from the wrong one.
Both paths decide through `clients_covering` now, so one database refuses a
wrong name the way a set already did.
The chat citation test said mounting lends the client without showing it; a
chat test shows it and the docstring points there.
`reader_for` documented None as covering any database it could not place, but a
name outside the set raises `KeyError` like `clients_for` does: provenance
naming a database this client does not cover is wrong rather than absent. None
means one thing, a federated client given no name.
`sources=[]` means two things. On a search it selects nothing to search; on the
constructor it raises, since a client over no database can do nothing. Both are
written down now.
A capability reads through a lent client, so what a citation records is that
client's database and not the scope the capability was built with. Chat lends
one, and had no test saying so.
`run_mcp` passed `_path`, the local stand-in a URI-backed ref resolves to for
display. A path overrides `lancedb.uri`, so `--db-name` on an S3 database
served the local default. It passes the ref's own path now, None where a URI
placed the database, and `create_mcp_server` accepts that.
The client opened around the server is gone. It never served a request, and it
opened the scope rather than the derived path, so startup validated the remote
database while the server read the local one.
`sources=["alpha"]` on a client covering a set went through fusion, which
scores position: a result the database ranked at 0.6549 was reported as
1/(60+rank). Embedding also moved ahead of the repository, so a filter matching
no document embedded the query anyway.
A selection of one now runs the single-database search. Fusion reconciles
rankings from separate indexes, and one ranking has nothing to reconcile.
A reranker that returns chunks it built rather than the ones it was given loses
which database each came from, since ownership is by identity. That is named
now instead of surfacing as a KeyError, and stated on `RerankerBase._rerank`.
The documented rule was existence in more than one selected database. The rule
is narrower and does not need to be wider: a copy the search never returned
grounded nothing, so the retrieved result is the citation and its database is
recorded rather than guessed.
The page now states the whole rule, including the fallback's own check for an
id no search returned. Two retrieved copies are refused, and so is an id
already cited from another database in the conversation. The case between them
had no test.
The ingester manufactured `data_dir / haiku.rag.lancedb` whenever `--db` was
absent and passed it to HaikuRAG. A path is now an explicit override that
clears the configured URI, so a `lancedb.uri` deployment wrote to local disk
while the control plane reported on the remote.
`IngesterApp` resolves the databases it works on once, in its constructor, and
both the client it opens and the control plane read that scope. `--db` names
one directly and nothing stands in for it, so a configured set is refused
rather than guessed, and `AmbiguousDatabaseError` joins the errors `cli()`
turns into a clean exit.
Collision detection replaced the lookup's dict comprehension with
`setdefault`, which also flipped a chunk found by several searches from its
last occurrence to its first. The copies differ in everything the expansion
window decides, figures included, so that silently changed what a citation
renders.
The rules are separate now: a repeated (source, chunk_id) takes the later
result, a chunk_id under two sources is still refused.
The collection block was chosen when the capability was built, so a run narrowed
through `state.sources` to one collection was still told how to attribute across
collections it could not reach, while its results correctly carried no
`Collection:` line.
`get_instructions` composes it per run instead, from `state.sources` where the
question narrowed the conversation and from the lent client or the scope
otherwise. Order is preamble, base instructions, collection block.
`format_for_agent` named the database whenever one was named, so a search over
a single named database carried a line with nothing to distinguish. It now takes
`include_collection` from the caller, which decides from the search selection
rather than from the hits: a search that could have drawn on two collections
names them even when everything came back from one.
`Collection:` at the model boundary, database in configuration and
administration. `source` on results, documents, citations and analysis
dictionaries is unchanged.
`resolve_citations` documented the behaviour it now rejects: a shared chunk id
resolving to whichever result came last.
The storage page ran two embedding checks together as if the second explained
the first. They are separate: each database against the configuration on open,
and the databases in a selection against each other, which raises in read-only
mode too and does not apply to full-text search.
`several` becomes `multiple` where it names the feature, matching the docs and
`covers_multiple`.
`_documents` took its owner clients inside an ephemeral connection and used
them after that connection closed, then stored them for the file reads that
follow, so every standalone multi-database read went through a closed database.
A sandbox covering a set now retains the client it opened until `close()`. One
database has no owners and keeps its connection no longer than the read that
opened it, so a write from elsewhere is still visible to the next read.
A URI-backed database is constructed with a local `db_path` nothing connects
to, and the info modal reported that, so `--db-name` on an S3 database named a
local directory. `location` reads the configuration instead, and the modal
labels it as one.
`InfoModal`'s `db_path` argument is gone: both callers passed None, so only a
test reached the branch that used it.
`sessions_for` recorded what opened from `gather`'s results, which arrive only
when it runs to completion. Cancelling it discarded them, so a database that
opened while a sibling was still pending was never recorded and `aclose` never
closed it. `return_exceptions=True` covers a failing child, not a cancelled
parent.
`_open` now registers its own session, so what opened is reachable however the
fan-out ends.
