Say precisely which conversations go unenforced

The documentation claimed a conversation that has never cited anything is not enforced,
which reads as though a first question could answer from fresh evidence without
declaring it. Enforcement needs neither condition to hold: no evidence outcome in this
question, and nothing cited earlier.

The structured-output test accepted a redirect or a recorded violation, so a break in
ending detection would still have passed on the backstop alone. The cite tool is
available in that scenario, so it asserts the redirect.
This commit is contained in:
Yiorgis Gozadinos 2026-08-12 12:04:15 +03:00
parent eb9934a6e5
commit cee0824744
No known key found for this signature in database
3 changed files with 6 additions and 6 deletions

View file

@ -3,7 +3,7 @@
### Added ### Added
- `CitationPolicyCapability` (`haiku.rag.capabilities.policy.create_capability`): registering it requires every answer to declare its grounding, in any conversation that has something to declare — this question retrieved evidence, or something was cited earlier. A question that ends undeclared is sent back once to record what grounded the answer already given, and is recorded in `CitationPolicyState.violations` if it finishes undeclared regardless. A conversation that has never cited anything is not enforced. - `CitationPolicyCapability` (`haiku.rag.capabilities.policy.create_capability`): registering it requires every answer to declare its grounding, in any conversation that has something to declare — this question retrieved evidence, or something was cited earlier. A question that ends undeclared is sent back once to record what grounded the answer already given, and is recorded in `CitationPolicyState.violations` if it finishes undeclared regardless. A conversation with neither a current-question evidence outcome nor any earlier citation is not enforced.
- `haiku.rag.capabilities.evidence.discover_evidence()` and `DiscoveredEvidence`, moved out of `compaction` so both optional capabilities share them. `RAGCapabilityBase.cite_available`. - `haiku.rag.capabilities.evidence.discover_evidence()` and `DiscoveredEvidence`, moved out of `compaction` so both optional capabilities share them. `RAGCapabilityBase.cite_available`.
- `EvidenceCompactionCapability` (`haiku.rag.capabilities.compaction.create_capability`): registering it replaces earlier questions' evidence on the model request with the evidence that was cited, grouped by the question that cited it, cited page images re-attached, other earlier evidence returns reduced to a receipt. Requests only; `all_messages()` is untouched. No configuration. - `EvidenceCompactionCapability` (`haiku.rag.capabilities.compaction.create_capability`): registering it replaces earlier questions' evidence on the model request with the evidence that was cited, grouped by the question that cited it, cited page images re-attached, other earlier evidence returns reduced to a receipt. Requests only; `all_messages()` is untouched. No configuration.
- `RAGState.evidence` / `AnalysisState.evidence` (`CapabilityEvidenceRecord`): which evidence a capability retrieved and cited, per question, keyed by message-count question identities and epochs. `haiku.rag.capabilities.ledger.citation_status(records, question=...)` derives `missing` / `grounded` / `ungrounded` across capabilities. - `RAGState.evidence` / `AnalysisState.evidence` (`CapabilityEvidenceRecord`): which evidence a capability retrieved and cited, per question, keyed by message-count question identities and epochs. `haiku.rag.capabilities.ledger.citation_status(records, question=...)` derives `missing` / `grounded` / `ungrounded` across capabilities.

View file

@ -100,8 +100,8 @@ something, which stays available to later answers. So a follow-up about evidence
earlier is enforced even though it searched nothing — that case is the reason the earlier is enforced even though it searched nothing — that case is the reason the
capability exists. It also means that once anything has been cited, later turns are capability exists. It also means that once anything has been cited, later turns are
enforced too, a greeting included; the model satisfies the policy by citing an empty enforced too, a greeting included; the model satisfies the policy by citing an empty
list, at the cost of one extra request. A conversation that has never cited anything is list, at the cost of one extra request. A conversation with neither a current-question
not enforced at all. evidence outcome nor any earlier citation is not enforced.
Exactly one policy capability makes the decision, however many evidence capabilities Exactly one policy capability makes the decision, however many evidence capabilities
are registered, so two of them cannot each demand a citation for one answer. are registered, so two of them cannot each demand a citation for one answer.

View file

@ -461,9 +461,9 @@ async def test_a_structured_output_answer_does_not_escape_enforcement(temp_db_pa
with patch.object(RAGCapability, "_search", stub_search): with patch.object(RAGCapability, "_search", stub_search):
await agent.run("what does the supervisor do?", deps=deps) await agent.run("what does the supervisor do?", deps=deps)
redirected = [p for p in prompts_of(sent[-1]) if CITATION_REDIRECT_TAG in p] # The cite tool is available here, so the redirect is what must happen; the
violations = deps.state.get("citation_policy", {}).get("violations", []) # backstop recording a violation would pass an `or` even with detection broken.
assert redirected or violations assert [p for p in prompts_of(sent[-1]) if CITATION_REDIRECT_TAG in p]
@pytest.mark.asyncio @pytest.mark.asyncio