From cee08247447f4233be0585abd1d5bdb26963c99a Mon Sep 17 00:00:00 2001 From: Yiorgis Gozadinos Date: Wed, 12 Aug 2026 12:04:15 +0300 Subject: [PATCH] Say precisely which conversations go unenforced The documentation claimed a conversation that has never cited anything is not enforced, which reads as though a first question could answer from fresh evidence without declaring it. Enforcement needs neither condition to hold: no evidence outcome in this question, and nothing cited earlier. The structured-output test accepted a redirect or a recorded violation, so a break in ending detection would still have passed on the backstop alone. The cite tool is available in that scenario, so it asserts the redirect. --- CHANGELOG.md | 2 +- docs/capabilities/index.md | 4 ++-- tests/capabilities/test_citation_policy.py | 6 +++--- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3ac453f5..cb1d2e1f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,7 +3,7 @@ ### Added -- `CitationPolicyCapability` (`haiku.rag.capabilities.policy.create_capability`): registering it requires every answer to declare its grounding, in any conversation that has something to declare — this question retrieved evidence, or something was cited earlier. A question that ends undeclared is sent back once to record what grounded the answer already given, and is recorded in `CitationPolicyState.violations` if it finishes undeclared regardless. A conversation that has never cited anything is not enforced. +- `CitationPolicyCapability` (`haiku.rag.capabilities.policy.create_capability`): registering it requires every answer to declare its grounding, in any conversation that has something to declare — this question retrieved evidence, or something was cited earlier. A question that ends undeclared is sent back once to record what grounded the answer already given, and is recorded in `CitationPolicyState.violations` if it finishes undeclared regardless. A conversation with neither a current-question evidence outcome nor any earlier citation is not enforced. - `haiku.rag.capabilities.evidence.discover_evidence()` and `DiscoveredEvidence`, moved out of `compaction` so both optional capabilities share them. `RAGCapabilityBase.cite_available`. - `EvidenceCompactionCapability` (`haiku.rag.capabilities.compaction.create_capability`): registering it replaces earlier questions' evidence on the model request with the evidence that was cited, grouped by the question that cited it, cited page images re-attached, other earlier evidence returns reduced to a receipt. Requests only; `all_messages()` is untouched. No configuration. - `RAGState.evidence` / `AnalysisState.evidence` (`CapabilityEvidenceRecord`): which evidence a capability retrieved and cited, per question, keyed by message-count question identities and epochs. `haiku.rag.capabilities.ledger.citation_status(records, question=...)` derives `missing` / `grounded` / `ungrounded` across capabilities. diff --git a/docs/capabilities/index.md b/docs/capabilities/index.md index 817eeadb..c7b582e5 100644 --- a/docs/capabilities/index.md +++ b/docs/capabilities/index.md @@ -100,8 +100,8 @@ something, which stays available to later answers. So a follow-up about evidence earlier is enforced even though it searched nothing — that case is the reason the capability exists. It also means that once anything has been cited, later turns are enforced too, a greeting included; the model satisfies the policy by citing an empty -list, at the cost of one extra request. A conversation that has never cited anything is -not enforced at all. +list, at the cost of one extra request. A conversation with neither a current-question +evidence outcome nor any earlier citation is not enforced. Exactly one policy capability makes the decision, however many evidence capabilities are registered, so two of them cannot each demand a citation for one answer. diff --git a/tests/capabilities/test_citation_policy.py b/tests/capabilities/test_citation_policy.py index bcd1a1f9..b800f8c6 100644 --- a/tests/capabilities/test_citation_policy.py +++ b/tests/capabilities/test_citation_policy.py @@ -461,9 +461,9 @@ async def test_a_structured_output_answer_does_not_escape_enforcement(temp_db_pa with patch.object(RAGCapability, "_search", stub_search): await agent.run("what does the supervisor do?", deps=deps) - redirected = [p for p in prompts_of(sent[-1]) if CITATION_REDIRECT_TAG in p] - violations = deps.state.get("citation_policy", {}).get("violations", []) - assert redirected or violations + # The cite tool is available here, so the redirect is what must happen; the + # backstop recording a violation would pass an `or` even with detection broken. + assert [p for p in prompts_of(sent[-1]) if CITATION_REDIRECT_TAG in p] @pytest.mark.asyncio