SingleDatabaseSession owns the store, the repositories and the vacuum
machinery, so nothing above has to ask whether it has a store. The client
keeps every name callers already use.
DatabaseScope.resolve reads configuration and at most one selector; a
DatabaseRef carries the configured name and a location already resolved to
a URI or a path, so a path a caller names is never reinterpreted. Nothing
consumes it yet.
An embedder is a function of configuration, not of a database, and the
databases in a selection share one, so a client covering a set builds it
on first use and closes it on teardown. Operations that need one database
say so instead of surfacing a missing store.
An image query has no text to match, so it is vector-only whatever the
caller asked for, and full-text search embeds nothing, so it needs no
agreement on embedders.
`first_found` replaces the client's private version and the capability's
sequential one. The info modal reports through the connection the client
already holds.
A title repeats within a corpus and across databases, so a substring
match on the displayed name widened the filter to documents the user did
not pick. The label names the database.
Each database owns an embedder, so embedding per database cost a round
trip each. One database still embeds inside the repository, which returns
early for a filter that matches nothing.
A blank name is falsy, so source routing reads it as absent, and a blank
location resolves to the working directory. `is_read_only` reports the
mode the client was opened with, which a client covering a set can answer
without a store.
Document.source names the configured database, as SearchResult and
Citation already do. A listing spanning databases is unreadable
without it, and `--database NAME list` could not name the one it
opened.
`get_document_by_id`, `get_document_by_uri` and `get_chunk_by_id` read
through repositories a client covering a set does not have, so a lookup by
identifier raised AttributeError and `resolve_document` with it. An
identifier says nothing about which database holds it, so every database is
asked at once and the first that has it, in configured order, answers.
On the evaluation side, `--db` overrides the configured set as the CLI
documents, and population refuses a set rather than ingesting into a
database the run would not read. A case filter matching nothing raises
instead of reporting 0.0000 as though it were a score.
A capability covering several databases received `source` on every document
and search result and never used it: asked how many documents were in each
database, the model read the titles and answered that there was one corpus
of 67,581. The instruction files enumerate what a result carries, and both
enumerations had gone stale.
The note follows what the capability opens rather than what the
configuration names, through `covers_several_databases`: an explicit
`db_path` or a lent client covering one database is instructed as before,
as is every `uri` or path deployment and every eval dataset. The analysis
note separates the three interfaces, since they differ: an
`analysis_search` result carries a `Database:` line, in-code `search` and
`list_documents` return `source`, and the mounted files carry neither.
The modal listed every document and mounted a checkbox per document, so a
corpus of tens of thousands never finished rendering: 67k sequential
mounts across two databases, and the same for one database that size.
Titles repeat at that scale too, and the ids were derived from the title,
so they collided.
It shows a page of 200 now, mounted in one call and identified by
position, and the search box asks the database for the rest on enter,
matching titles and URIs. Typing still narrows the page on screen, for
feedback while typing. `search_filter` escapes the term, which is
whatever was typed.
A federated listing takes its window across the databases rather than
filling it from the first one: concatenating hid every database after
whichever was listed first, which for a set of a thousand papers and
sixty thousand articles meant a page of papers alone. Sorting would not
have helped, since document ids and article titles sort into separate
runs, so the page is picked by interleaving and the modal sorts it for
display.
Chat answers with the same capabilities `ask` does, so it federates as
naturally as `ask` and `analyze` — but it went through the one-database
guard and refused a configured set outright, which left no way to chat
across several databases.
The guard was the visible half. `run_chat` also defaulted `db_path` to the
single default path whenever it was None, so lifting the refusal alone
would still have opened one database. It now leaves the path unresolved
when `lancedb.databases` names the set, and the client resolves it.
Listing and counting documents fan out over the set, which is what the
document filter reads, and visual grounding resolves the database holding
the cited chunk through the citation's source: chunks, pages and bounding
boxes all come from that one database. A limit on a listing means that
many documents in total, not that many per database.
The info modal reports every database it covers, each under its
configured name and without its location, since names are the only
identity that leaves the configuration. `database_lines` is what one
database reports about itself, shared by both paths, and it reports a
failure as a line so one unreachable database does not cost the report on
the others.
`inspect` stays a one-database command. It browses one database's
documents and chunks, so a set has nothing to show it.
The CLI reports this error, and importing it from the settings repository
pulled lancedb onto the CLI's import path. Deferring the import into cli()
bought nothing, since cli() runs on every invocation: it cost about 1.9
seconds on a cold start, `--help` included.
It now sits beside the other store exceptions, in a module that imports
nothing, and every importer points there.
A database whose stored embedder disagrees with the configuration raised
ConfigMismatchError through Typer, so the operator got a traceback wrapped
around the one message that says what to run, while every sibling failure
exits with its message. It joins the errors the CLI reports.
Imported inside cli() rather than at module scope: the settings module
pulls in lancedb, and importing the CLI must not pay for it.
Chunk 2 gave search a configured set to fan out over. ask and analyze
covered one database still: the RAG capability had no way to be told which
databases a question spanned, and the analysis sandbox mounted one
document tree.
The selection travels as sources on EvidenceState, beside the filter it
scopes with, so both capabilities read it the same way. clients_covering
is the one rule that turns a selection into clients, used by search, the
sandbox mount and the cite fallback, so a question scoped to some
databases cannot search, mount or cite another. Citations carry the
database they came from, and format_for_agent names it, so the model can
attribute evidence while it answers rather than only afterwards.
The sandbox keeps one flat /documents/{id}/ namespace and resolves each id
to the client holding it, which rests on ids being UUID4. A database
copied from another breaks that, so an id held twice is refused rather
than resolved to whichever arrived last.
On the CLI, search, ask and analyze cover the configured set and label
each result with its database. Every other command works on one, named
with --database NAME (a name reaches a database behind a URI, which --db
cannot) or --db PATH, and refuses a set it cannot choose from instead of
silently reading the default database. Cold databases open together, so a
first query costs the slowest open rather than their sum.
`ask(sources=[…])` scopes a question to some of the configured databases, carried
on the capability state so its search tool searches those. `Citation.source` names
the database a cited chunk came from, resolved from the search results the model
saw, which already carry it.
Context expansion routes each result through the database it came from: a
federating client has no repositories of its own.
The cite fallback, which looks up an id absent from this run's results, searches
only the selected databases. A chunk id says nothing about which database holds
it, so placing one means asking, and asking outside the selection would let a
question scoped to some databases cite another.
The loosely-specced client mocks in the capability tests now say they stand in for
a single-database client. A bare AsyncMock answers any attribute with a truthy
Mock, so `_federated` sent the fallback down the multi-database branch, and
`_source` reached a validated field.
`lancedb.databases` maps a name to a location, mutually exclusive with `uri`.
`search(sources=[…])` selects which to search, `sources=None` searches all of them
and `sources=[]` searches none; `SearchResult.source` carries the configured name,
so a path or URI never leaves the configuration. A database named in config keeps
its name even when it is the only one configured; only a legacy single `uri`
leaves `source` unset.
Databases open on first use, not at entry. Which are searched is a per-query
choice, so a set of 25 queried a few at a time opens a few, and a database nobody
asked for can neither fail a query nor be opened for nothing.
A named database that fails to open raises `SourceUnavailableError` naming it,
raised outside the handler so the original is not attached at all. A local failure
spells out the absolute path and an object-store failure can carry the bucket;
`from None` would only stop that being printed, leaving it on `__context__` for
anything that walks the chain. A legacy `uri` client has no name to report
instead, so its error passes through unchanged.
Candidates are fetched concurrently, then fused before anything is ranked. A
configured reranker scores the union, which is what makes ranking across
databases tractable: it compares query against document and does not care where a
candidate came from. Without one, reciprocal rank fusion over the per-database
rankings, since scores from separate indexes are not comparable. Enrichment then
runs on the survivors through the database each came from, concurrently, so it
costs what a single-database search costs.
The over-fetch decision and the reranker belong to the federating client alone.
Deciding per database would have each consult its own, and a local reranker loads
model weights per instance. It is built only for a text query, and closed once by
the client that owns it.
A location without a scheme is opened as a local path rather than through
`lancedb.uri`. Routing it through `uri` had `ConnectionMode` classify it as object
storage, which opens a missing database instead of reporting it.
With several databases configured, `store` and the repositories are left unset:
they have no unambiguous meaning across a set, and picking one silently would be
worse than the error.
`search` fetched, reranked and truncated in one pass, with the reranker's
over-fetch and the reranking itself interleaved in the same branch. Searching
several databases needs to fuse their candidates before anything is ranked, so
the phases have to be separable.
`_fetch` returns one database's candidates, over-fetching only when a reranker
will re-order them. `_rank` orders and cuts them, leaving an image query's vector
ranking alone since there is no text for a reranker to score against. The
over-fetch multiplier is named rather than a literal 10 at the point of use.
Both check the query type before reading `client.reranker`, which is a
cached_property that builds the reranker on first access and loads model weights
for a local one. An image query never used it and must not start.
No behaviour change: the same suite passes, and the search outputs digest
identically to before.
`Agent.from_spec` raised `TypeError` on `RAGCapability` and `AnalysisCapability`,
whose constructors take a state class, packaged instruction text and a tool-name
set, and silently omitted both from the generated spec schema. The two
zero-configuration capabilities constructed but with `id=None`, so pydantic-ai's
duplicate-id rejection no longer held and a spec could register two citation
policies, defeating the single-decision-maker invariant.
Override `from_spec` on all four, delegating to `create_capability()` so ids and
instructions come from one place. The spec surface is `db_path`, `config`,
`defer_loading`, `request_limit` and `vision`; a live `HaikuRAG` client stays out
of it, and a `config` mapping is validated through `AppConfig`.
The documented `HaikuRAG("knowledge.lancedb")` and `rag(db_path="my.lancedb")`
forms both raised `AttributeError: 'str' object has no attribute 'exists'`.
`Store.__init__` assigned its argument to a `Path`-annotated attribute without
coercing, and `resolve_db_path` returned a non-None argument unchanged. Every
runnable example wraps the path in `Path(...)`, which is why it survived.
Coerce in `Store.__init__` and `resolve_db_path`; widen the annotations on
`Store`, `HaikuRAG` and both `create_capability` factories to accept `str`.
urlparse().path keeps the leading slash in front of a Windows drive, so
file:///C:/docs/a.pdf read as \C:\docs\a.pdf and the ingester reported
"File does not exist" for every file it discovered. url2pathname is the
stdlib conversion that strips it, per platform.
Four sites each decided both "is this local" and "what path is this":
FSSource._uri_to_path and supports, resolve_adhoc_fetcher,
create_document_from_source and check_source_accessible, and convert.
is_local_uri and uri_to_path in haiku.rag.uri own those two decisions now,
which closes two more cases of the same root cause. A bare C:\docs\a.pdf
parses with scheme "c", so add-src raised "No source adapter for URI scheme
'c'" and convert silently treated the path as raw text. And convert and
check_source_accessible never percent-decoded at all, so a file named
a[b] c.md read as missing on Linux and macOS too.
A file URI's host is reattached after conversion rather than passed to
url2pathname, which as of 3.14 rejects a non-local authority off Windows.
file:////server/share is the empty-authority spelling of a UNC path, its
host being the first path segment, so that host is normalised into the
authority before conversion. Output is identical on 3.12, 3.13 and 3.14.
The ad-hoc FS fetcher roots at the path's own anchor rather than "/", which
on Windows is only the current drive.
test_uri.py runs on ubuntu, macos and windows across 3.13 and 3.14 without
the project installed: --noconftest because the repo conftest imports
dependencies that job does not need, and -o addopts= to drop the
repository's -n auto. The Windows legs are what cover the drive conversion.
Fixes#574.
cli.py carried 40 pragmas over whole command bodies and app.py a
class-level one over all 412 statements, while tests/test_cli.py already
drove 29 commands through CliRunner. The pragmas hid lines the suite
executed, so the 100% gate understated real coverage and gave new CLI code
no scrutiny.
Both are measured now. 38 CLI tests stub HaikuRAGApp and assert the parsed
arguments reach the right method; 60 app tests stub the client and record
the console, pinning what each command asks for and what it prints. The only
pragma left in either file is cli() under __main__. The omit list is back to
the two Textual TUIs.
Three defects the coverage surfaced:
haiku-rag settings masked only top-level secret-named fields, so nested ones
printed in full — lancedb.api_key, providers.docling_serve.api_key, WebDAV
source passwords. It uses redact_secrets, which walks the dump.
chat guarded the wrong thing: haiku.rag.chat imports without Textual, and
run_chat raises when it imports ChatApp, so the missing extra escaped as an
ImportError. The guard is on the call. inspector raises at module import
instead, so inspect keeps its guard on the import; each has a test that
fails the way the real installation fails.
search --limit/--search-type and history --limit default to None so the
config resolves the default. Now pinned.
CI passed --cov=haiku while pyproject declares source = ["haiku_rag_slim"];
pass --cov and let the config decide. build-docs.yml only ran on push to
main, so a broken docs build merged and failed at deploy: build on pull
requests, with configure-pages, upload-pages-artifact and deploy gated to
push, and a per-ref concurrency group.
engine.py held four unrelated things: what the tables are, how to open a
connection, how to read a database's state, and the Store that coordinates
writes. At 1240 lines the Store's own concerns — locks, migrations, vacuum,
tags — were hard to find among them.
Table records, Arrow schemas, index_specs, ensure_indexes, REQUIRED_TABLES
and query_to_pydantic move to store/schema.py, which imports nothing from
haiku.rag: it describes the tables and never opens or mutates one.
gather_database_info, get_database_stats, DatabaseInfo and its result models
move to store/info.py. Nothing in Store calls them — they are read paths for
the CLI, doctor, inspector and ingester API — so info depends on engine and
not the reverse.
engine.py keeps the Store, ConnectionMode, connect_lancedb, the tag helpers
and the restore-order and retention constants. No re-exports: importers
point at the new modules.
test_app_info_uses_connect_lancedb_for_remote patched
haiku.rag.store.engine.connect_lancedb; gather_database_info now binds that
name in info.py, so the patch targets where the call is looked up.
haiku.rag.ingester.sources was never ingester-only: one-shot client
ingestion resolves adapters through it (create_document_from_source), and
convert() now fetches through HTTPSource, so the core client imported into
the ingester package to reach them.
Move the package to haiku.rag.sources and update every import. No shims:
haiku.rag.ingester.sources is gone.
The haiku.rag.sources plugin entry-point group is unchanged, so third-party
source packages need no edit — the group name now matches the module path it
always implied.
Source unit tests move to tests/sources/. test_source_plugins.py stays in
tests/ingester/: it drives a PeriodicPoller against the job repo, so it is
plugin wiring through ingester machinery rather than a source test.
Five call sites repeated the same post-conversion preparation: store the
Docling representation and resolve a title when none was supplied.
_prepare_and_title now owns that sequence. update_document continues to
call _prepare_document_from_docling directly because an explicit update
must preserve an existing empty title.
create_document, both content-replacement branches of update_document,
and source ingestion embedded eagerly before passing chunks to a
persistence funnel that checked them again. The funnels now own
embedding, including the checks required by import_document and
import_documents for caller-supplied chunks.
Move the document.embed span into ensure_chunks_embedded after its early
return. Every path that performs embedding is now instrumented, while
operations whose chunks are already embedded emit no span.
convert() previously used its own HTTP client and temporary-file path.
Route URL conversion through HTTPSource, matching source ingestion, and
move _write_fetch_body to processing.py so both paths share temporary
file handling without an import cycle.
Add walk_files for filesystem enumeration and use it from both
FSSource.discover and one-shot directory ingestion. Symlink escape
filtering now has one implementation.
RAGState and AnalysisState each declared the same five fields, so the generic
base could not name them: StateT was bound to BaseModel, and every access
went through cast(Any, state), a getattr by string, or a loop clearing fields
by name so it could skip the one only AnalysisState has.
EvidenceState declares them once. RAGState adds nothing, AnalysisState adds
executions and overrides begin_invocation to clear them. StateT binds to
EvidenceState, which removes all ten casts and both state-shape getattrs; the
three getattr(ctx.deps, "state") probes stay, since those check a
host-supplied object rather than our own state.
discover_evidence reached into capability.state for two fields. It now asks
through evidence_record() and citation_index(), alongside the
evidence_tool_names() and cite_available accessors it already used. The eval
runner's _RagLikeState protocol and the chat app's getattr reads described
this shape from outside and are gone.
Compatibility is semantic JSON-object equivalence, not bytes: field names and
nesting are unchanged, so a dict stored by 0.75.0 loads and re-dumps equal,
but deriving from a shared base reorders AnalysisState's keys. Nothing
serializes, hashes or string-compares this state — every carry point
re-validates by key.
search.limit was documented as 10 in three places while the default is 5.
The documented way to disable reranking, provider: "", is a valid
ModelConfig, so it raised "Unknown reranking provider" — disabling means
omitting reranking.model or setting it to null. The inline provider list
named four of the six rerankers. prompts.picture_description: null fails
validation, since the field is a non-optional str.
storage.data_dir: "" coerced to Path("") — the working directory — while two
doc pages promise the platform default and soliplex's example config relies
on it. Empty or whitespace now resolves to the platform directory; an
explicit "." is still honoured, so a config that wants the working directory
says so.
Three tests keep this from drifting again: every fenced yaml block in the
docs validates against AppConfig, every value in the complete example either
equals its default or is listed as a deliberate deviation, and empty
data_dir resolves to the platform default.
init-config's test reimplemented the command body instead of invoking it,
which is why the command carried a coverage pragma. It now goes through
CliRunner, with the refuse-to-overwrite guard covered too.
Every section inherited plain BaseModel, so unknown keys were dropped
silently: providers.docling_serve.timeout was documented for months while
being ignored, and a typo in any setting took the default. Sections now
derive from ConfigModel, which forbids extras, so a stale or misspelled key
fails with its path. This already found search.context_radius in a live app
config and providers.vllm in soliplex's example.
converter, chunker and chunker_type are Literals. Sizes, limits,
dimensions, token budgets, attempt counts and breaker thresholds must be
positive; retention, delays, intervals and cooldowns non-negative;
similarity_threshold within 0-1; port within 0-65535. port 0 keeps its
OS-assigned meaning and worker_count allows 0 for an API-and-reaper-only
process.
get_reranker caught ImportError and returned None, so a configured reranker
whose extra was missing silently disappeared. It now propagates.
raise_missing_extra names the install command and re-raises when the failure
came from inside an installed package, so a broken transitive import is not
reported as a missing one. zeroentropy imported bare and now guards like the
others.
The haiku.rag package declares the jina extra. jina-local already worked
there through cross-encoder's transitive transformers and torch; the
resolved package set is unchanged, but the support is now promised rather
than inherited.
Provider fields stay unconstrained: get_model ends in a pass-through to
pydantic-ai for any provider it supports, so a Literal there would reject
valid configurations.
haiku.rag.config exported two configuration instances: the lazy _config
behind get_config/set_config, and Config, loaded at import time. Nothing
linked them, and eleven signatures captured Config as a default argument,
so set_config could not reach the factories, the client, the store or the
MCP server. reranking/base.py went further and snapshotted the configured
reranker name into a class attribute at import.
Config is removed. Internal defaults are config: AppConfig | None = None,
resolved through get_config() per call. RerankerBase._model is None and
CohereReranker takes its model name as an argument, like every other
reranker.
The suite patched attributes on Config while production read the instance
get_config() returns, a different object, so those patches were no-ops
waiting to happen. They now go through get_config().
Four call sites repeated lock, snapshot, try/except, restore. Each used
restore_table_versions, which restores in _tables() order — documents
first, contradicting RESTORE_TABLE_ORDER — and each caught Exception, so
a cancellation mid-write skipped rollback and left the earlier table
writes committed.
Store.write_transaction() holds the lock, snapshots under it, and rolls
back through _rollback_to_snapshot: RESTORE_TABLE_ORDER, shielded from
cancellation, absorbed cancellation re-delivered, rollback failure raised
with the original as cause. The two single-table update_meta sites keep
the bare lock.
The batch documents write moves inside the guarded body; it was outside
the try, so a failure there was never rolled back. Auto-vacuum is
scheduled after the transaction rather than inside it.
restore_table_versions is removed; those four sites were its only callers.
The setting was documented but did not exist on DoclingServeConfig, so it
was silently dropped, and DoclingServeClient.from_config never forwarded
the timeout parameter it already accepted. The per-request timeout was
therefore pinned at the constructor default of 300s with no way to change
it. Add the field, forward it, and reject a non-positive value.
Also parametrize over the checked-in *.yaml.example files and validate
each through AppConfig, so an example that no longer loads fails a test
rather than a user's first run.
create_document_from_source resolves a fetcher per call and never closed
it, so every one-shot URL or WebDAV ingest leaked an httpx connection
pool. Close it, but only when we built it: resolve_adhoc_fetcher returns
a caller-supplied source when one matches the URI, and the ingester keeps
those open across jobs.
Directory ingestion also yielded symlinked files resolving outside the
directory it was given. rglob does not recurse into symlinked
directories, so a symlinked file was the only way out of the tree; skip
those, as FSSource.discover already does.
The chunk repository docstring named client._ensure_chunks_embedded,
which does not exist.
FULL rebuild deleted a document before re-ingesting it from its URI, and
the handler around that logged and continued. A 404, a timeout or any
conversion error therefore removed the document permanently.
Deleting after a successful create is not an alternative:
create_document_from_source resolves the same URI to the existing
document and updates it in place, so a trailing delete would remove the
freshly rebuilt row.
Refresh in place instead. create_document_from_source takes an internal
force flag that skips the revision and MD5 short-circuits, so an
unchanged source is still re-converted, re-chunked and re-embedded into
the existing document, and the document id survives a rebuild.
A failed refresh now falls through to the stored-content path rather
than skipping the document: FULL recreates the chunks table before the
loop, so skipping left the document present but unsearchable until the
next rebuild.
The pending-batch flush moves out of the try. A failed flush is a lost
write and should abort the rebuild, not be logged and skipped.
Collapsing the caption text into `get_pictures_grouped` served the enrichment
path, which uses it, but the multimodal reranker discards the second return value
while still paying to read the column. That is the widest fan-out in the codebase,
`limit * 10` candidates, and it previously projected self_ref and picture_data
alone.
`with_text` is opt-in and off by default, so the cheap projection is what a caller
gets unless it asks for more. The reranker test asserts the projection as well as
the query count, since a count alone would not notice the column coming back.
Splitting the assembly into a passthrough pass and an expandable pass reordered
equal-scored results: the score sort that follows is stable, so the order results
arrive in is the tiebreak. Results are assembled in document_groups order again,
after the batched fetch rather than around it.
Also ports the caption negative cases the removed single-document test carried: a
table's caption and an ordinary text reference map to no picture.
`resolve_refs`, `get_items_in_range`, `get_caption_picture_refs` and
`get_all_items_grouped` have no callers left: the grouped equivalents serve every
path that used them. `get_all_items_grouped` had none even before this branch.
Tests whose subject was a removed method go with it. Tests that only used one to
fetch a fixture now use the grouped call, so what they assert is unchanged.
`_attach_picture_data` fetched picture bytes once per document, over the
`limit * 10` candidates reranking asks for, so it was the per-document fetch with
the most candidates behind it. It now issues one query however many documents the
candidates span: one for ten documents, as for one.
Removes `get_text_for_refs`, whose only caller now gets the text back with the
bytes from `get_pictures_grouped`.
`test_client_search_include_images_false_skips_lookup` returned no search
results, so asserting the picture accessor went uncalled held whatever the code
did. It now returns a picture-carrying result, making "did not fetch" the
assertion rather than "had nothing to fetch".
`expand_with_items` fetched its own inputs per document: one query to resolve
refs to positions, one for the window of items around them. A result set spanning
N documents cost 2N queries, which was 10 of the 18 measured for a limit=5 search
on a remote object-store corpus.
`expand_context` now does both fetches once for every document it is expanding,
and `expand_with_items` takes the positions and items it needs. Two queries for
one document, and two for five.
Each document keeps its own inclusive window in `get_items_in_ranges`. Positions
repeat across documents, so a shared range would splice one document's items into
another's context.
`_populate_image_data` ran its stages once per result document, so a result set
spanning N documents cost 4N `document_items` queries. Measured on a remote
object-store corpus, a limit=5 search with expansion was 18 queries, 16 of them
against `document_items`.
The stages now run once each across every document, and flat in document count:
two queries for the dependent caption-to-picture mapping when results ranked on a
caption, one for the picture bytes. Two queries for a picture-ref result set,
three at most.
Picture text comes back with the bytes rather than from a second query, since it
is on the same rows.
Predicates are per document, `(document_id = 'a' AND self_ref IN (…)) OR (…)`,
rather than `self_ref IN (union)`. self_ref and position values repeat across
documents, so a union predicate would return other documents' rows: for
picture_data that fetches blobs nobody asked for, and it can hand one document
another document's picture.
`client.ask` and `client.analyze` built their capability from a db_path, so the
capability opened a second connection to the database the client already had
open, once per call.
Ownership is now explicit rather than inferred. `rag` stays the connection the
capability opened and must close; `borrowed_rag` is a caller's, which
`_ensure_rag` prefers and `_close` never touches. Two fields rather than a flag,
so closing a borrowed connection is not expressible.
`for_run` still clears `rag` per run, since a run owns what it opens. It leaves
`borrowed_rag` alone: that connection belongs to the caller and outlives the run.
All ten tool bodies opened their own `HaikuRAG`, so every tool call paid a
connection open and, on object storage, refetched the index the previous call had
just cached. The client is now opened once, lazily so that calling a tool
function directly still works, and eagerly from the lifespan so an unopenable
database fails startup instead of every call. Teardown clears the cached client
in a finally, since `_lifespan_manager` can be re-entered and would otherwise
hand out a closed connection, including when the close itself fails.
`delete_document` no longer opens its own connection with `skip_validation=True`.
Keeping it separate broke consistency once connections became long-lived: the
delete committed on one connection while reads served from another, which with a
30s consistency interval showed the deleted document as still present. A
connection always sees its own writes, so sharing one is what makes delete
visible to the next read.
So the server no longer opts out of embedding-config validation. Drift that
validation rejects now fails MCP startup, where before the server started and
only `delete_document` worked while every read returned empty. Same-dimension
identity drift still starts a read-only server, matching every other read verb.
Delete under drift is now a CLI operation; CLAUDE.md and the CHANGELOG record it.
Opening a database ran `list_tables` three times, opened the settings table
twice, and read and parsed the same settings row three times: once for the stored
vector dimension, once for the version behind the migration check, and once for
config validation. On object storage each of those is a round trip.
`_initialize` now reads both once and threads them down. `_init_tables` and
`_check_migrations` take what it read instead of fetching their own copy, and
`validate_config_compatibility` accepts the settings it should compare against,
still reading for itself when called directly.
Passing the pre-init read to validation is equivalent: nothing between the read
and the validation rewrites `embeddings`, which is all it compares.
The settings read no longer swallows every exception. It did before, when the
only consequence was falling back to the configured vector dimension; now the
same empty result feeds the migration check, where it would read as version
0.0.0 and declare every migration pending. Only decode failures are tolerated,
and a decoded non-object normalizes to {} rather than reaching callers that
expect a mapping.
