From 7bdd11db39212a2872c8c87682c48bd85253f860 Mon Sep 17 00:00:00 2001 From: Yiorgis Gozadinos Date: Thu, 13 Aug 2026 14:08:09 +0300 Subject: [PATCH] Update caps documentation --- docs/capabilities/analysis.md | 14 +++-- docs/capabilities/compaction.md | 43 +++++++++++++ docs/capabilities/index.md | 105 ++++++++------------------------ docs/capabilities/policy.md | 49 +++++++++++++++ docs/capabilities/rag.md | 19 ++++-- docs/chat.md | 11 ++-- zensical.toml | 2 + 7 files changed, 151 insertions(+), 92 deletions(-) create mode 100644 docs/capabilities/compaction.md create mode 100644 docs/capabilities/policy.md diff --git a/docs/capabilities/analysis.md b/docs/capabilities/analysis.md index 5effc127..b3a58138 100644 --- a/docs/capabilities/analysis.md +++ b/docs/capabilities/analysis.md @@ -18,18 +18,24 @@ When `qa.max_searches` or `analysis.max_executions` runs out, the exhausted tool The sandbox exposes documents under `/documents/{document_id}/` with `metadata.json`, `content.txt`, `items.jsonl`, and `toc.json`. -## Compose with RAG +## Compose an agent + +Register it on its own, not alongside `RAGCapability`: it already searches and cites, +and the two together give the model duplicate tools and separate budgets. See +[Capabilities](index.md#compose-an-agent). ```python from pydantic_ai import Agent from haiku.rag.capabilities.analysis import create_capability as analysis -from haiku.rag.capabilities.rag import create_capability as rag +from haiku.rag.capabilities.compaction import create_capability as compaction +from haiku.rag.capabilities.policy import create_capability as citation_policy agent = Agent( "openai:gpt-5", capabilities=[ - rag(db_path="my.lancedb"), analysis(db_path="my.lancedb"), + compaction(), + citation_policy(), ], ) ``` @@ -48,6 +54,6 @@ async with HaikuRAG("my.lancedb") as client: When dependencies expose a state dictionary, `AnalysisState` is stored under `"analysis"`. It contains the document filter, code execution log, searches, citations, and the `evidence` record of what was retrieved and cited per question. Searches and executions are cleared when a new question starts, and a resumed question keeps them; the filter, citation index and evidence record persist. -This capability does not alter the message history either. Register the [compaction capability](index.md#multi-turn-conversations) to compact earlier questions. +This capability does not alter the message history either. Register the [compaction capability](compaction.md) to compact earlier questions. The capability lazily opens both LanceDB and the sandbox only after it is loaded and a tool requires them. Resources close at the end of the agent run. diff --git a/docs/capabilities/compaction.md b/docs/capabilities/compaction.md new file mode 100644 index 00000000..46c6a038 --- /dev/null +++ b/docs/capabilities/compaction.md @@ -0,0 +1,43 @@ +# Evidence compaction capability + +`EvidenceCompactionCapability` keeps a multi-turn conversation from carrying every +search result it ever produced. Every question adds its evidence to the history, so +requests grow turn after turn, which degrades answers and can exceed a provider's +limits. + +Register it alongside an evidence capability: + +```python +from pydantic_ai import Agent +from haiku.rag.capabilities.compaction import create_capability as compaction +from haiku.rag.capabilities.rag import create_capability as rag + +agent = Agent( + "openai:gpt-5", + capabilities=[rag(db_path="my.lancedb"), compaction()], +) +``` + +It exposes no tools and takes no configuration. Registering it is the only switch: +leave it out and the transcript reaches the model untouched. + +## What it does + +On each request, evidence from earlier questions is replaced by the evidence those +questions actually cited. Cited text and cited page images are kept in full, grouped by +the question that cited them, and stay citable by the same chunk ids. Every other +earlier evidence return becomes a short receipt. The current question is untouched. + +Compaction rewrites the request, never the stored history, so `all_messages()` still +holds everything the run gathered. + +This reduces what a request carries. It does not bound it: retained evidence still +grows with the conversation. A host that needs more aggressive pruning can compact its +own requests further, on the wire only. + +## Resuming a question + +Resuming a question (deferred tool results, an interruption, a suspension) requires the +host to carry the capability state from the run being resumed, alongside the message +history. Without it the identity of the question in progress is unknowable, and the run +fails rather than silently treating it as a new question. diff --git a/docs/capabilities/index.md b/docs/capabilities/index.md index c7b582e5..15636fff 100644 --- a/docs/capabilities/index.md +++ b/docs/capabilities/index.md @@ -6,105 +6,50 @@ haiku.rag provides native [Pydantic AI capabilities](https://ai.pydantic.dev/cap |---|---| | [`RAGCapability`](rag.md) | Grounded document search and citations. | | [`AnalysisCapability`](analysis.md) | Corpus computation and structural analysis with sandboxed Python. | -| `EvidenceCompactionCapability` | Optional. Shrinking a conversation's history to the evidence that was cited. | -| `CitationPolicyCapability` | Optional. Requiring every answer to declare what grounds it. | +| [`EvidenceCompactionCapability`](compaction.md) | Optional. Shrinking a conversation's history to the evidence that was cited. | +| [`CitationPolicyCapability`](policy.md) | Optional. Requiring every answer to declare what grounds it. | The two evidence capabilities are deferred by default. An agent initially sees only their descriptions and the standard `load_capability` tool. Instructions and tools enter the model context only when the model loads a capability. ## Compose an agent +Pick one evidence capability, and add both optional capabilities to it: + ```python from pydantic_ai import Agent -from haiku.rag.capabilities.rag import create_capability - -rag = create_capability(db_path="my.lancedb") -agent = Agent("openai:gpt-5", capabilities=[rag]) - -result = await agent.run("What does the knowledge base say about X?") -print(result.output) -``` - -Attach both capabilities when an agent should choose between retrieval and computation: - -```python -from haiku.rag.capabilities.analysis import create_capability as analysis -from haiku.rag.capabilities.rag import create_capability as rag - -agent = Agent( - "openai:gpt-5", - capabilities=[rag(db_path="my.lancedb"), analysis(db_path="my.lancedb")], -) -``` - -## Multi-turn conversations - -Every question adds its search results to the history, so requests grow turn after -turn, and can degrade answers or exceed a provider's limits as they do. Register the -compaction capability to replace earlier questions' evidence with the evidence that -was actually cited: - -```python from haiku.rag.capabilities.compaction import create_capability as compaction -from haiku.rag.capabilities.rag import create_capability as rag - -agent = Agent( - "openai:gpt-5", - capabilities=[rag(db_path="my.lancedb"), compaction()], -) -``` - -Cited text and cited page images are kept in full, grouped by the question that -cited them, and stay citable by the same chunk ids. Everything else earlier becomes a -short receipt. Registering the capability is the only switch: leave it out and the -transcript reaches the model untouched. There is nothing to configure. - -Compaction rewrites the request, never the stored history, so `all_messages()` still -holds everything the run gathered. Retained evidence still grows with the -conversation — this reduces what a request carries, it does not bound it. A host that -needs more aggressive pruning can compact its own requests further, on the wire only. - -Resuming a question (deferred tool results, an interruption, a suspension) requires -the host to carry the capability state from the run being resumed, alongside the -message history. Without it the identity of the question in progress is unknowable -and the run fails rather than silently treating it as a new question. - -## Requiring citations - -Citing is always available and always recorded, but nothing requires it. Register the -citation policy capability to make every answer declare its grounding: - -```python from haiku.rag.capabilities.policy import create_capability as citation_policy from haiku.rag.capabilities.rag import create_capability as rag agent = Agent( "openai:gpt-5", - capabilities=[rag(db_path="my.lancedb"), citation_policy()], + capabilities=[ + rag(db_path="my.lancedb"), + compaction(), + citation_policy(), + ], ) + +result = await agent.run("What does the knowledge base say about X?") +print(result.output) ``` -An empty citation is a valid declaration: a model that finds nothing relevant calls -the cite tool with an empty list, which records the answer as *ungrounded* — distinct -from an answer that declared nothing at all. That distinction is what makes requiring -a declaration possible without forcing the model to invent grounding. +Swap `rag` for `analysis` for an analysis agent. Both optional capabilities work the +same way with either one, and neither exposes tools or takes configuration. -When a question ends undeclared, the model is asked once to record what grounded the -answer it already gave. It is not asked to change the answer. If the cite tool is no -longer available by then, or the question finishes undeclared anyway, it is recorded as -a violation in `CitationPolicyState` under `"citation_policy"`, since pointing a model -at a tool that is gone costs it retries. +!!! note "Register one evidence capability, not both" -What gets enforced is every answer in a conversation that has something to declare: -either this question retrieved evidence, or the conversation has already cited -something, which stays available to later answers. So a follow-up about evidence cited -earlier is enforced even though it searched nothing — that case is the reason the -capability exists. It also means that once anything has been cited, later turns are -enforced too, a greeting included; the model satisfies the policy by citing an empty -list, at the cost of one extra request. A conversation with neither a current-question -evidence outcome nor any earlier citation is not enforced. + `RAGCapability` and `AnalysisCapability` overlap. Both search the same corpus and + both register citations, so an agent holding both must choose between two + near-identical search tools, and its citations land in whichever capability it + happened to call. Each also carries its own request limit and its own search + budget, so registering both doubles what a question may spend. -Exactly one policy capability makes the decision, however many evidence capabilities -are registered, so two of them cannot each demand a citation for one answer. + Choose by what the questions need. `RAGCapability` answers questions from retrieved + passages. `AnalysisCapability` adds a Python sandbox and a document filesystem, for + questions that compute over many documents or read their structure, and it can + search too. If you need computation, register the analysis capability alone rather + than adding it to the RAG one. ## State diff --git a/docs/capabilities/policy.md b/docs/capabilities/policy.md new file mode 100644 index 00000000..de0b56df --- /dev/null +++ b/docs/capabilities/policy.md @@ -0,0 +1,49 @@ +# Citation policy capability + +`CitationPolicyCapability` requires every answer to declare what grounds it. Citing is +always available and always recorded without it, but nothing makes the model do it. + +Register it alongside an evidence capability: + +```python +from pydantic_ai import Agent +from haiku.rag.capabilities.policy import create_capability as citation_policy +from haiku.rag.capabilities.rag import create_capability as rag + +agent = Agent( + "openai:gpt-5", + capabilities=[rag(db_path="my.lancedb"), citation_policy()], +) +``` + +It exposes no tools and takes no configuration. Exactly one policy capability makes the +decision, however many evidence capabilities are registered, so two of them cannot each +demand a citation for one answer. + +## Declaring nothing is a valid answer + +A model that finds nothing relevant calls the cite tool with an empty list. That records +the answer as *ungrounded*, which is distinct from an answer that declared nothing at +all (*missing*). The distinction is what makes a declaration requirable without forcing +the model to invent grounding. + +## What happens when a question ends undeclared + +The model is asked once to record what grounded the answer it already gave. It is not +asked to change the answer. If the cite tool is no longer available by then, or the +question finishes undeclared anyway, the question is recorded in +`CitationPolicyState.violations` under the `"citation_policy"` state key. Pointing a +model at a tool that is gone costs it retries, so the capability records the failure +instead. + +## Which answers are enforced + +Every answer in a conversation that has something to declare: either this question +retrieved evidence, or the conversation has already cited something, which stays +available to later answers. A follow-up about evidence cited earlier is enforced even +though it searched nothing, which is the case the capability exists for. + +Once anything has been cited, later turns are enforced too, a greeting included. The +model satisfies the policy by citing an empty list, at the cost of one extra request. A +conversation with neither a current-question evidence outcome nor any earlier citation +is not enforced. diff --git a/docs/capabilities/rag.md b/docs/capabilities/rag.md index 35c3fbf2..ba3c3267 100644 --- a/docs/capabilities/rag.md +++ b/docs/capabilities/rag.md @@ -13,12 +13,23 @@ The distinct `rag_` prefix lets this capability coexist with analysis and other ## Create and compose +Register it on its own rather than alongside `AnalysisCapability`, which searches and +cites as well. See [Capabilities](index.md#compose-an-agent). + ```python from pydantic_ai import Agent -from haiku.rag.capabilities.rag import create_capability +from haiku.rag.capabilities.compaction import create_capability as compaction +from haiku.rag.capabilities.policy import create_capability as citation_policy +from haiku.rag.capabilities.rag import create_capability as rag -rag = create_capability(db_path="my.lancedb") -agent = Agent("openai:gpt-5", capabilities=[rag]) +agent = Agent( + "openai:gpt-5", + capabilities=[ + rag(db_path="my.lancedb"), + compaction(), + citation_policy(), + ], +) result = await agent.run("What safety equipment does the manual require?") print(result.output) @@ -49,7 +60,7 @@ State is ordinary application state; the capability does not depend on AG-UI. An ## Context management -This capability does not alter the message history. To stop long conversations resending old retrieved content, register the [compaction capability](index.md#multi-turn-conversations) alongside it. +This capability does not alter the message history. To stop long conversations resending old retrieved content, register the [compaction capability](compaction.md) alongside it. ## Domain context and vision diff --git a/docs/chat.md b/docs/chat.md index 091a3eb1..f5efdcad 100644 --- a/docs/chat.md +++ b/docs/chat.md @@ -61,13 +61,16 @@ Retrieval stays text-based; the images are sent to the model alongside your mess The default capability is `rag`. Enable `analysis` when the question needs computation, aggregation, comparison across documents, or section-scoped reading that a single search can't deliver: ```bash -# both capabilities (the agent routes between them) -haiku-rag chat -c rag -c analysis - -# analysis only +# analysis instead of rag haiku-rag chat -c analysis + +# both, which gives the model duplicate search and cite tools +haiku-rag chat -c rag -c analysis ``` +Prefer one. `analysis` searches and cites as well as computing, so pairing it with `rag` +duplicates tools and budgets. See [Capabilities](capabilities/index.md#compose-an-agent). + The `analysis` capability mounts every document as a virtual filesystem at `/documents/{id}/` (with `metadata.json`, `content.txt`, `items.jsonl`, and `toc.json`) and runs Python in a sandboxed interpreter with `search` and `list_documents` as awaitable functions. It's the right choice for questions like: - "How many of these documents mention X?" diff --git a/zensical.toml b/zensical.toml index 9ce0ec76..fbf59e7f 100644 --- a/zensical.toml +++ b/zensical.toml @@ -23,6 +23,8 @@ nav = [ "capabilities/index.md", { "RAG capability" = "capabilities/rag.md" }, { "Analysis capability" = "capabilities/analysis.md" }, + { "Evidence compaction" = "capabilities/compaction.md" }, + { "Citation policy" = "capabilities/policy.md" }, ] }, { Configure = [ "configuration/index.md",