Say precisely which conversations go unenforced

The documentation claimed a conversation that has never cited anything is not enforced,
which reads as though a first question could answer from fresh evidence without
declaring it. Enforcement needs neither condition to hold: no evidence outcome in this
question, and nothing cited earlier.

The structured-output test accepted a redirect or a recorded violation, so a break in
ending detection would still have passed on the backstop alone. The cite tool is
available in that scenario, so it asserts the redirect.
This commit is contained in:
Yiorgis Gozadinos 2026-08-12 12:04:15 +03:00
parent eb9934a6e5
commit cee0824744
No known key found for this signature in database
3 changed files with 6 additions and 6 deletions

View file

@ -3,7 +3,7 @@
### Added
- `CitationPolicyCapability` (`haiku.rag.capabilities.policy.create_capability`): registering it requires every answer to declare its grounding, in any conversation that has something to declare — this question retrieved evidence, or something was cited earlier. A question that ends undeclared is sent back once to record what grounded the answer already given, and is recorded in `CitationPolicyState.violations` if it finishes undeclared regardless. A conversation that has never cited anything is not enforced.
- `CitationPolicyCapability` (`haiku.rag.capabilities.policy.create_capability`): registering it requires every answer to declare its grounding, in any conversation that has something to declare — this question retrieved evidence, or something was cited earlier. A question that ends undeclared is sent back once to record what grounded the answer already given, and is recorded in `CitationPolicyState.violations` if it finishes undeclared regardless. A conversation with neither a current-question evidence outcome nor any earlier citation is not enforced.
- `haiku.rag.capabilities.evidence.discover_evidence()` and `DiscoveredEvidence`, moved out of `compaction` so both optional capabilities share them. `RAGCapabilityBase.cite_available`.
- `EvidenceCompactionCapability` (`haiku.rag.capabilities.compaction.create_capability`): registering it replaces earlier questions' evidence on the model request with the evidence that was cited, grouped by the question that cited it, cited page images re-attached, other earlier evidence returns reduced to a receipt. Requests only; `all_messages()` is untouched. No configuration.
- `RAGState.evidence` / `AnalysisState.evidence` (`CapabilityEvidenceRecord`): which evidence a capability retrieved and cited, per question, keyed by message-count question identities and epochs. `haiku.rag.capabilities.ledger.citation_status(records, question=...)` derives `missing` / `grounded` / `ungrounded` across capabilities.

View file

@ -100,8 +100,8 @@ something, which stays available to later answers. So a follow-up about evidence
earlier is enforced even though it searched nothing — that case is the reason the
capability exists. It also means that once anything has been cited, later turns are
enforced too, a greeting included; the model satisfies the policy by citing an empty
list, at the cost of one extra request. A conversation that has never cited anything is
not enforced at all.
list, at the cost of one extra request. A conversation with neither a current-question
evidence outcome nor any earlier citation is not enforced.
Exactly one policy capability makes the decision, however many evidence capabilities
are registered, so two of them cannot each demand a citation for one answer.

View file

@ -461,9 +461,9 @@ async def test_a_structured_output_answer_does_not_escape_enforcement(temp_db_pa
with patch.object(RAGCapability, "_search", stub_search):
await agent.run("what does the supervisor do?", deps=deps)
redirected = [p for p in prompts_of(sent[-1]) if CITATION_REDIRECT_TAG in p]
violations = deps.state.get("citation_policy", {}).get("violations", [])
assert redirected or violations
# The cite tool is available here, so the redirect is what must happen; the
# backstop recording a violation would pass an `or` even with detection broken.
assert [p for p in prompts_of(sent[-1]) if CITATION_REDIRECT_TAG in p]
@pytest.mark.asyncio