Commit graph

82 commits

Author SHA1 Message Date
Yiorgis Gozadinos
a04a16c717
Remove the environment overrides and the last unnamed-database wording
HAIKU_RAG_DB and DB_PATH are gone: a capability covers what the
configuration places or the db_path it is given, and the app backend and
the AG-UI example load their configuration as the CLI does. The compose
files point HAIKU_RAG_CONFIG_PATH at the mounted haiku.rag.yaml, which
places the database at /data where DB_VOLUME is mounted; the backend
refuses a configured set since it serves one database. The chat scopes a
selection by source only over a set and names databases on filter rows
only across several. Docstrings, docs and test fixtures stop describing an
unnamed database; every database a search, listing or citation reports
carries a name.
2026-09-03 15:12:09 +03:00
Yiorgis Gozadinos
9baa213b34
Hand storage the database location, not the configuration that placed it
Store, connect_lancedb, gather_database_info and run_doctor take a
location, a path or a URI, and classify it with ConnectionMode.of.
SingleDatabaseSession owns the resolved DatabaseRef and passes its
location down. This removes DatabaseRef.connection(), default_db_path,
the placeholder path for URI-backed databases and the per-database
config copies, so the configuration a client holds is the one the caller
gave it. The chat hands its capabilities the scope it opened along with
the client it lends, and the v0.58.0 migration no longer checks local
free disk for a database behind a URI.
2026-09-03 15:11:37 +03:00
Yiorgis Gozadinos
afdef92b5b
Finish the comment pass, and escape document fields everywhere Rich renders
`_rich_print_document` escapes uri, title and metadata, the sibling of
the escaped search-result renderer. The remaining comments and
docstrings that narrated rejected alternatives, consequences or history
now state the current invariant. The Sandbox class docstring names the
held connection close() releases, and wrapped docs paragraphs join to
one line.
2026-08-28 15:34:47 +03:00
Yiorgis Gozadinos
ee835b77c2
Select a filtered document from the database that holds it
The chat filter modal keys a selection by (database, id): copies of a
database share document ids, and checking one copy left the other
reading as selected. Applying the filter narrows the question's
`sources` to the databases the selection names. A twin id inside
another selected database still matches there: a serialized id filter
cannot carry a source.
2026-08-28 15:12:55 +03:00
Yiorgis Gozadinos
688d7e708e
State what comments guarantee, not what would go wrong without them
Comments that narrated a failure mode across three or four lines say the
invariant they protect instead: the repository's late embedding, one database
keeping its hybrid scores, one over-fetch decision for a selection, and a cite
fallback that covers exactly what the question covers.
2026-08-28 14:03:05 +03:00
Yiorgis Gozadinos
7dc853606f
Name the collection a chat citation came from
Two collections can hold documents with the same title, and the collapsed
citation carried only that title, so the two read as one source repeated. The
name is appended when the client covers a set, the decision already made for
search results and retrieved images.
2026-08-28 13:30:11 +03:00
Yiorgis Gozadinos
225a37ca73
Tell one collection's pictures from another's
Three places identified a picture by document and reference alone, and one
identified a citation's images by chunk id alone. Both repeat between copies of
a database, so a search returning a figure from two collections sent one, a
capsule retained one, and a citation rendered the other collection's figures.

Keyed on the source as well: `(source, document_id, self_ref)` for search
pictures, the capability and source for retained ones, and `qualified_id` for
the chat's citation images.
2026-08-28 10:35:46 +03:00
Yiorgis Gozadinos
bd8c1a6d15
Render a citation whose pictures no database claims
Chat asked the covering client for a picture with the citation's source, which
raises when there is none, losing the answer to one figure. It resolves the
reader first and omits the bytes when the database cannot be placed, as the rich
formatter already did.
2026-08-28 08:47:36 +03:00
Yiorgis Gozadinos
9210642f65
Report where a named remote database actually is
A URI-backed database is constructed with a local `db_path` nothing connects
to, and the info modal reported that, so `--db-name` on an S3 database named a
local directory. `location` reads the configuration instead, and the modal
labels it as one.

`InfoModal`'s `db_path` argument is gone: both callers passed None, so only a
test reached the branch that used it.
2026-08-27 11:40:35 +03:00
Yiorgis Gozadinos
d367b1eb5a
Resolve the databases a command works on, once
The CLI decides only what it knows — that --db and --database are the same
thing said twice, and whether a command reads more than one — and hands the
resolved scope down. Nothing rewrites the configuration, so a named database
keeps the name results and citations carry, and a remote one opens the URI
it was configured with rather than the local path standing in for it.

HaikuRAGApp, ChatApp and InspectorApp take that scope and nothing else.
Selection reaches the client through a private constructor, so the public
signature still takes a path or names.
2026-08-25 15:48:47 +03:00
Yiorgis Gozadinos
8db447e095
Ask the client what it covers
covers_multiple, source_names, source and reader_for replace the private
state seven modules were reading to work out how many databases they had.
The configured selection is kept intact, so entering a client twice derives
the same database rather than the last derivation.
2026-08-25 14:16:22 +03:00
Yiorgis Gozadinos
307e250f29
Select documents to filter by id, not by displayed name
A title repeats within a corpus and across databases, so a substring
match on the displayed name widened the filter to documents the user did
not pick. The label names the database.
2026-08-24 15:27:59 +03:00
Yiorgis Gozadinos
bb92ccf67e
Cover the configured set in the chat TUI
Chat answers with the same capabilities `ask` does, so it federates as
naturally as `ask` and `analyze` — but it went through the one-database
guard and refused a configured set outright, which left no way to chat
across several databases.

The guard was the visible half. `run_chat` also defaulted `db_path` to the
single default path whenever it was None, so lifting the refusal alone
would still have opened one database. It now leaves the path unresolved
when `lancedb.databases` names the set, and the client resolves it.

Listing and counting documents fan out over the set, which is what the
document filter reads, and visual grounding resolves the database holding
the cited chunk through the citation's source: chunks, pages and bounding
boxes all come from that one database. A limit on a listing means that
many documents in total, not that many per database.

The info modal reports every database it covers, each under its
configured name and without its location, since names are the only
identity that leaves the configuration. `database_lines` is what one
database reports about itself, shared by both paths, and it reports a
failure as a line so one unreachable database does not cost the report on
the others.

`inspect` stays a one-database command. It browses one database's
documents and chunks, so a set has nothing to show it.
2026-08-24 10:03:46 +03:00
Yiorgis Gozadinos
fdb5710491
Ask and analyze across several databases
Chunk 2 gave search a configured set to fan out over. ask and analyze
covered one database still: the RAG capability had no way to be told which
databases a question spanned, and the analysis sandbox mounted one
document tree.

The selection travels as sources on EvidenceState, beside the filter it
scopes with, so both capabilities read it the same way. clients_covering
is the one rule that turns a selection into clients, used by search, the
sandbox mount and the cite fallback, so a question scoped to some
databases cannot search, mount or cite another. Citations carry the
database they came from, and format_for_agent names it, so the model can
attribute evidence while it answers rather than only afterwards.

The sandbox keeps one flat /documents/{id}/ namespace and resolves each id
to the client holding it, which rests on ids being UUID4. A database
copied from another breaks that, so an id held twice is refused rather
than resolved to whichever arrived last.

On the CLI, search, ask and analyze cover the configured set and label
each result with its database. Every other command works on one, named
with --database NAME (a name reaches a database behind a URI, which --db
cannot) or --db PATH, and refuses a set it cannot choose from instead of
silently reading the default database. Cold databases open together, so a
first query costs the slowest open rather than their sum.
2026-08-24 10:03:46 +03:00
Yiorgis Gozadinos
cab510d0d5
Give the evidence capabilities one typed state
RAGState and AnalysisState each declared the same five fields, so the generic
base could not name them: StateT was bound to BaseModel, and every access
went through cast(Any, state), a getattr by string, or a loop clearing fields
by name so it could skip the one only AnalysisState has.

EvidenceState declares them once. RAGState adds nothing, AnalysisState adds
executions and overrides begin_invocation to clear them. StateT binds to
EvidenceState, which removes all ten casts and both state-shape getattrs; the
three getattr(ctx.deps, "state") probes stay, since those check a
host-supplied object rather than our own state.

discover_evidence reached into capability.state for two fields. It now asks
through evidence_record() and citation_index(), alongside the
evidence_tool_names() and cite_available accessors it already used. The eval
runner's _RagLikeState protocol and the chat app's getattr reads described
this shape from outside and are gone.

Compatibility is semantic JSON-object equivalence, not bytes: field names and
nesting are unchanged, so a dict stored by 0.75.0 loads and re-dumps equal,
but deriving from a shared base reorders AnalysisState's keys. Nothing
serializes, hashes or string-compares this state — every carry point
re-validates by key.
2026-08-19 17:08:44 +03:00
Yiorgis Gozadinos
11b1bfbc94
Give the capsule to one return, and the run its own copy of state
A request can carry several of this capability's returns — a model can call search
twice in one response — and the carrier was identified by message alone, so each of
them received the whole capsule. It is selected by message and part now, so exactly
one carries it however many share the request.

The chat TUI passed its persisted state into the run, so tool synchronisation mutated
it in place while the message history was promoted only on success. A cancelled or
failed run therefore kept the evidence the tools had recorded and discarded the
messages that justified it, and the next question derived its identity from the
shorter history: behind the recorded epoch, refused as non-append-only, the
conversation unusable until cleared. The run gets a copy, promoted with the messages
or not at all.

Five decorators had been left attached to a helper by an earlier extraction, which
pytest does not collect, so the resume case they carried was silently untested. The
wire test covers both resume shapes again, no prompt and deferred results.
2026-08-13 13:00:02 +03:00
Yiorgis Gozadinos
2cd568847e
Move the wire rewrite into the compaction capability
`_compact_old_tool_returns`, `PRIOR_TURN_NOTICE` and `turn_start` leave
`RAGCapabilityBase`, along with its `wrap_model_request` hook. The evidence
capabilities now retrieve and validate, and nothing else. Registering the compaction
capability is what rewrites a request; leaving it out sends the transcript untouched,
which was never a choice a host could make before.

The boundary is the recorded question identity rather than message shape, so a
resumption compacts what lies below the question in progress instead of switching
compaction off for the whole run. The newest earlier evidence return carries the
capsule and every other becomes a receipt, so one capsule exists by construction and
every return stays paired with its call.

Pictures of cited evidence are fetched through the capability that retrieved them and
re-attached beside the capsule with fresh labels. Ownership of a picture on the wire
requires the machine tag we write and an image directly after it, since neither
position nor prose is proof: several tools' results can arrive in one request, and a
user quoting our wording above their own picture had it removed. A picture that cannot
be fetched or decoded is emitted with neither its image nor its label.

The chat TUI and the example backend register the compactor, being multi-turn.
`client.ask`, `client.analyze` and the MCP tools do not: a single-shot question has
nothing earlier to compact.
2026-08-13 13:00:02 +03:00
Yiorgis Gozadinos
93824cdee5
Surface database-open errors in the TUIs
ChatApp and InspectorApp assigned self.client before __aenter__
completed, so a failed open was masked by an AttributeError from
on_unmount tearing down the never-opened client.
2026-07-25 11:27:29 +03:00
Yiorgis Gozadinos
5f4c73f89f
Add image attachment to the chat TUI 2026-07-25 10:58:08 +03:00
Yiorgis Gozadinos
77585ef26a
fix per-question capability limits and tool isolation 2026-07-24 15:26:18 +03:00
Yiorgis Gozadinos
43c17a6777
fix capability execution limits and chat loading 2026-07-24 15:26:17 +03:00
Yiorgis Gozadinos
9e7db72997
Load capabilities eagerly for dedicated single-agent consumers 2026-07-24 15:26:17 +03:00
Yiorgis Gozadinos
9deb1f2bd4
replace haiku.skills with native Pydantic AI capabilities 2026-07-24 15:26:17 +03:00
Yiorgis Gozadinos
0e271eaf4b
Remove --before/--at time travel 2026-07-16 13:11:33 +03:00
Yiorgis Gozadinos
0813a1c980
Add tag CLI commands and history tag annotations 2026-07-16 13:11:27 +03:00
Yiorgis Gozadinos
da79f92af1
Honor OTEL_SERVICE_NAME and set service name/version for all processes 2026-07-10 11:11:46 +03:00
Yiorgis Gozadinos
995d081aa1
Visualize the exact context the model saw via Citation.doc_item_refs
visualize_chunk re-expanded chunks from scratch to recover their refs,
which could not faithfully reproduce the original merge, scores, and
clip — so a visualization could highlight different pages than the
citation covered. Carry the cited items on Citation.doc_item_refs and
resolve bounding boxes from them directly; re-expansion remains only as
the fallback for callers with no stored context (CLI, inspector). Chat,
inspector, the app endpoint, and the frontend pass the refs through.
2026-07-09 11:51:37 +03:00
Yiorgis Gozadinos
494f774473
Visualize all constituent chunks of a merged citation 2026-07-09 11:05:41 +03:00
Yiorgis Gozadinos
73f8349a00
Use haiku.rag as telemtry scope 2026-05-26 11:44:47 +03:00
Yiorgis Gozadinos
188d35023f
rotate chat TUI conversation id per launch / clear-chat 2026-05-21 14:21:51 +03:00
Yiorgis Gozadinos
f13461cdca
route figures through cite, drop show_image 2026-05-18 16:48:34 +03:00
Yiorgis Gozadinos
1df6530d03
Bump pydantic-ai, prepare for 2.* 2026-05-18 15:14:29 +03:00
Yiorgis Gozadinos
674b80e8c9
Support streaming for markdown 2026-05-12 16:01:42 +03:00
Yiorgis Gozadinos
7685a592c2
bump haiku.skills to 0.16.0 2026-04-28 12:03:14 +03:00
Yiorgis Gozadinos
cb9360ef7b
fix chat TUI citation rendering after state flattening 2026-04-24 13:48:57 +03:00
Yiorgis Gozadinos
d52f453c44
flatten skill architecture: replace ask/analyze/research with direct tools 2026-04-17 18:35:17 +03:00
Yiorgis Gozadinos
7d98d0ec57
add citation support to analysis agent 2026-04-17 18:35:16 +03:00
Yiorgis Gozadinos
2a0e89bbe7
add --skill flag to chat TUI for rag and analysis skills 2026-04-17 18:35:16 +03:00
Yiorgis Gozadinos
e16c819e83
Adapt to new haiku.skills 2026-04-16 11:50:55 +03:00
Yiorgis Gozadinos
c4c90b3930
Propagate domain_preamble to skill instructions and main agent preamble 2026-04-01 13:13:36 +03:00
Yiorgis Gozadinos
5eb250af05
Fix broken textual detection in chat & inspect 2026-03-26 17:42:38 +02:00
Yiorgis Gozadinos
1d108b42ec
handle ActivitySnapshotEvent for skill sub-agent tool calls 2026-03-13 11:37:24 +02:00
Yiorgis Gozadinos
2566bf1a61
Adapt to haiku.skills streaming sub-agent ag-ui events 2026-03-03 11:16:44 +02:00
Yiorgis Gozadinos
52ef983e75
Fix default chat model, default to config.qa.model 2026-02-28 14:00:40 +02:00
Yiorgis Gozadinos
27a71c7e01
Adapt skill usage to haiku.skills prompt builder. Closes #289 2026-02-28 13:34:54 +02:00
Yiorgis Gozadinos
c61ded1271
Replace Docker sandbox with pydantic-monty 2026-02-24 09:55:40 +02:00
Yiorgis Gozadinos
cb5935a6fa
Show in tui and nextjs app what is being requested from a skill 2026-02-20 18:14:44 +02:00
Yiorgis Gozadinos
5dd8a82d1f
Exclude TUI apps from coverage 2026-02-20 16:37:16 +02:00
Yiorgis Gozadinos
eb9436eb2a
Remove unused conversational output mode from research graph 2026-02-20 16:36:50 +02:00
Yiorgis Gozadinos
c567842420
Rewire document filter 2026-02-20 16:36:50 +02:00