`_rich_print_document` escapes uri, title and metadata, the sibling of
the escaped search-result renderer. The remaining comments and
docstrings that narrated rejected alternatives, consequences or history
now state the current invariant. The Sandbox class docstring names the
held connection close() releases, and wrapped docs paragraphs join to
one line.
The chat filter modal keys a selection by (database, id): copies of a
database share document ids, and checking one copy left the other
reading as selected. Applying the filter narrows the question's
`sources` to the databases the selection names. A twin id inside
another selected database still matches there: a serialized id filter
cannot carry a source.
A database name, document title, uri or heading containing Rich markup
crashed search output, chat citations and the chat document filter with
MarkupError. Every interpolation into markup-parsed text now escapes.
Comments that narrated a failure mode across three or four lines say the
invariant they protect instead: the repository's late embedding, one database
keeping its hybrid scores, one over-fetch decision for a selection, and a cite
fallback that covers exactly what the question covers.
Two collections can hold documents with the same title, and the collapsed
citation carried only that title, so the two read as one source repeated. The
name is appended when the client covers a set, the decision already made for
search results and retrieved images.
Three places identified a picture by document and reference alone, and one
identified a citation's images by chunk id alone. Both repeat between copies of
a database, so a search returning a figure from two collections sent one, a
capsule retained one, and a citation rendered the other collection's figures.
Keyed on the source as well: `(source, document_id, self_ref)` for search
pictures, the capability and source for retained ones, and `qualified_id` for
the chat's citation images.
Typing narrowed the mounted checkboxes without touching the search the listing
was built from, so a term matching more than one page hid rows from whichever
page the user happened to be on, left the rest of the matches a page away, and
paged on the previous search while the term still sat in the box. The term now
applies on enter, and the footer says so until it does.
Chat asked the covering client for a picture with the citation's source, which
raises when there is none, losing the answer to one figure. It resolves the
reader first and omits the bytes when the database cannot be placed, as the rich
formatter already did.
A selection outside the page stayed applied while its checkbox was gone, so
there was no way to remove it. Appending those documents to the page instead
loses the bound the page exists to keep, since selections accumulate across
searches.
Both listings page at `DOCUMENT_PAGE`, and `Selected` switches between them, so
the mounted widgets stay bounded whichever is showing and every selection is a
page away rather than unreachable.
The count reads the checkboxes on screen, so narrowing as the user types reports
what is visible. A listing with nothing in it says so, instead of leaving an
empty box that reads as still loading.
A URI-backed database is constructed with a local `db_path` nothing connects
to, and the info modal reported that, so `--db-name` on an S3 database named a
local directory. `location` reads the configuration instead, and the modal
labels it as one.
`InfoModal`'s `db_path` argument is gone: both callers passed None, so only a
test reached the branch that used it.
Every load looked up the loading indicator, which only the first page has,
so pressing enter in the search box raised `NoMatches`. The indicator is a
child of the list `remove_children()` already clears.
A document's id and search text travel on a `DocumentCheckbox` instead of
being assigned onto a `Checkbox` behind type suppressions. The positional
widget ids are gone; nothing queried them.
`--database` and `--db` read as one word abbreviated but take different
things, a configured name against a filesystem path, and they sit in
different positions: `haiku-rag --db /path info` fails with `No such option`.
The CLI already says "database" to the user everywhere it prints one, so the
name it selects by should say the same. Unreleased, so no deprecation.
Also: without a reranker, raise `search.limit` with the number of databases
searched, since each contributes its best matches to a list truncated back
to the limit.
`HaikuRAG.__init__` said an omitted `db_path` uses `storage.data_dir`, which
is the last of three; and that `sources` is ignored for a single `uri`, where
it raises, since only `lancedb.databases` names databases.
The name is not the only identity leaving the configuration: results,
citations, model input and errors opening a named database carry it, while
`info`, `init` and `tag` print the location. `sources=[]` returns no search
results, but `ask` and `analyze` still answer, without evidence.
Trim the comments that still narrated a failure or an alternative to the
invariant they were there for.
One sentence for the contract, one or two more only where an invariant is not
obvious. The reasons that stay are about correctness and ownership: which
database a result belongs to, who closes what, why assembly order is the
tiebreak. The ones that go narrated how the code got here.
The CLI decides only what it knows — that --db and --database are the same
thing said twice, and whether a command reads more than one — and hands the
resolved scope down. Nothing rewrites the configuration, so a named database
keeps the name results and citations carry, and a remote one opens the URI
it was configured with rather than the local path standing in for it.
HaikuRAGApp, ChatApp and InspectorApp take that scope and nothing else.
Selection reaches the client through a private constructor, so the public
signature still takes a path or names.
covers_multiple, source_names, source and reader_for replace the private
state seven modules were reading to work out how many databases they had.
The configured selection is kept intact, so entering a client twice derives
the same database rather than the last derivation.
A title repeats within a corpus and across databases, so a substring
match on the displayed name widened the filter to documents the user did
not pick. The label names the database.
The modal listed every document and mounted a checkbox per document, so a
corpus of tens of thousands never finished rendering: 67k sequential
mounts across two databases, and the same for one database that size.
Titles repeat at that scale too, and the ids were derived from the title,
so they collided.
It shows a page of 200 now, mounted in one call and identified by
position, and the search box asks the database for the rest on enter,
matching titles and URIs. Typing still narrows the page on screen, for
feedback while typing. `search_filter` escapes the term, which is
whatever was typed.
A federated listing takes its window across the databases rather than
filling it from the first one: concatenating hid every database after
whichever was listed first, which for a set of a thousand papers and
sixty thousand articles meant a page of papers alone. Sorting would not
have helped, since document ids and article titles sort into separate
runs, so the page is picked by interleaving and the modal sorts it for
display.
Chat answers with the same capabilities `ask` does, so it federates as
naturally as `ask` and `analyze` — but it went through the one-database
guard and refused a configured set outright, which left no way to chat
across several databases.
The guard was the visible half. `run_chat` also defaulted `db_path` to the
single default path whenever it was None, so lifting the refusal alone
would still have opened one database. It now leaves the path unresolved
when `lancedb.databases` names the set, and the client resolves it.
Listing and counting documents fan out over the set, which is what the
document filter reads, and visual grounding resolves the database holding
the cited chunk through the citation's source: chunks, pages and bounding
boxes all come from that one database. A limit on a listing means that
many documents in total, not that many per database.
The info modal reports every database it covers, each under its
configured name and without its location, since names are the only
identity that leaves the configuration. `database_lines` is what one
database reports about itself, shared by both paths, and it reports a
failure as a line so one unreachable database does not cost the report on
the others.
`inspect` stays a one-database command. It browses one database's
documents and chunks, so a set has nothing to show it.
Chunk 2 gave search a configured set to fan out over. ask and analyze
covered one database still: the RAG capability had no way to be told which
databases a question spanned, and the analysis sandbox mounted one
document tree.
The selection travels as sources on EvidenceState, beside the filter it
scopes with, so both capabilities read it the same way. clients_covering
is the one rule that turns a selection into clients, used by search, the
sandbox mount and the cite fallback, so a question scoped to some
databases cannot search, mount or cite another. Citations carry the
database they came from, and format_for_agent names it, so the model can
attribute evidence while it answers rather than only afterwards.
The sandbox keeps one flat /documents/{id}/ namespace and resolves each id
to the client holding it, which rests on ids being UUID4. A database
copied from another breaks that, so an id held twice is refused rather
than resolved to whichever arrived last.
On the CLI, search, ask and analyze cover the configured set and label
each result with its database. Every other command works on one, named
with --database NAME (a name reaches a database behind a URI, which --db
cannot) or --db PATH, and refuses a set it cannot choose from instead of
silently reading the default database. Cold databases open together, so a
first query costs the slowest open rather than their sum.
Sixty-three comments said what the next statement already said: # Connect to
LanceDB above connect_lancedb, # Path object above isinstance(source, Path),
# Get page numbers from provenance above the prov loop, # Clear and populate
results above list_view.clear(). They cost a read and carry nothing.
The line is whether a comment restates one statement or labels a phase. Phase
labels stay: the migrations keep # Create staging table with new schema and
# Copy from staging to final table in batches, each heading ten lines of a
long procedure. So do comments carrying a fact the code cannot: the
merge_insert update-only note on document_meta, why the poller builds sources
eagerly, why create_document_from_source returns a list for directories, that
indexes need training data, the field-group markers in the config models, and
the file:// URL-encoding note in create_document_from_source.
capabilities/ is untouched. Its docstrings sit next to prompt surface, and
changing them needs an eval to back it.
The cassette-recording docs were wrong three ways. They named
tests/test_qa.py::test_qa_anthropic, which no longer exists; they targeted
whole modules, so a rewrite would re-record cassettes for services the
recorder is not running; and they used COHERE_API_KEY where the SDK reads
CO_API_KEY. docs/development.md now names exact tests with -n0, and the keyed
example is test_cohere_reranker, which owns the one cassette recording
api.cohere.com.
RAGState and AnalysisState each declared the same five fields, so the generic
base could not name them: StateT was bound to BaseModel, and every access
went through cast(Any, state), a getattr by string, or a loop clearing fields
by name so it could skip the one only AnalysisState has.
EvidenceState declares them once. RAGState adds nothing, AnalysisState adds
executions and overrides begin_invocation to clear them. StateT binds to
EvidenceState, which removes all ten casts and both state-shape getattrs; the
three getattr(ctx.deps, "state") probes stay, since those check a
host-supplied object rather than our own state.
discover_evidence reached into capability.state for two fields. It now asks
through evidence_record() and citation_index(), alongside the
evidence_tool_names() and cite_available accessors it already used. The eval
runner's _RagLikeState protocol and the chat app's getattr reads described
this shape from outside and are gone.
Compatibility is semantic JSON-object equivalence, not bytes: field names and
nesting are unchanged, so a dict stored by 0.75.0 loads and re-dumps equal,
but deriving from a shared base reorders AnalysisState's keys. Nothing
serializes, hashes or string-compares this state — every carry point
re-validates by key.
A request can carry several of this capability's returns — a model can call search
twice in one response — and the carrier was identified by message alone, so each of
them received the whole capsule. It is selected by message and part now, so exactly
one carries it however many share the request.
The chat TUI passed its persisted state into the run, so tool synchronisation mutated
it in place while the message history was promoted only on success. A cancelled or
failed run therefore kept the evidence the tools had recorded and discarded the
messages that justified it, and the next question derived its identity from the
shorter history: behind the recorded epoch, refused as non-append-only, the
conversation unusable until cleared. The run gets a copy, promoted with the messages
or not at all.
Five decorators had been left attached to a helper by an earlier extraction, which
pytest does not collect, so the resume case they carried was silently untested. The
wire test covers both resume shapes again, no prompt and deferred results.
`_compact_old_tool_returns`, `PRIOR_TURN_NOTICE` and `turn_start` leave
`RAGCapabilityBase`, along with its `wrap_model_request` hook. The evidence
capabilities now retrieve and validate, and nothing else. Registering the compaction
capability is what rewrites a request; leaving it out sends the transcript untouched,
which was never a choice a host could make before.
The boundary is the recorded question identity rather than message shape, so a
resumption compacts what lies below the question in progress instead of switching
compaction off for the whole run. The newest earlier evidence return carries the
capsule and every other becomes a receipt, so one capsule exists by construction and
every return stays paired with its call.
Pictures of cited evidence are fetched through the capability that retrieved them and
re-attached beside the capsule with fresh labels. Ownership of a picture on the wire
requires the machine tag we write and an image directly after it, since neither
position nor prose is proof: several tools' results can arrive in one request, and a
user quoting our wording above their own picture had it removed. A picture that cannot
be fetched or decoded is emitted with neither its image nor its label.
The chat TUI and the example backend register the compactor, being multi-turn.
`client.ask`, `client.analyze` and the MCP tools do not: a single-shot question has
nothing earlier to compact.
ChatApp and InspectorApp assigned self.client before __aenter__
completed, so a failed open was masked by an AttributeError from
on_unmount tearing down the never-opened client.
Drive analysis-only chat with analysis.model (falling back to qa.model) so
the running model matches the one the analysis capability configures,
including its vision flag; RAG-bearing chats still run on qa.model.
Give the AG-UI example state-bearing deps and a final STATE_SNAPSHOT so
registered citations reach the client, mirroring the app backend.
Update chat docs: haiku-rag chat uses --capability/-c, and drop the removed
"View state" command-palette entry.
visualize_chunk re-expanded chunks from scratch to recover their refs,
which could not faithfully reproduce the original merge, scores, and
clip — so a visualization could highlight different pages than the
citation covered. Carry the cited items on Citation.doc_item_refs and
resolve bounding boxes from them directly; re-expansion remains only as
the fallback for callers with no stored context (CLI, inspector). Chat,
inspector, the app endpoint, and the frontend pass the refs through.
