Stop a partly resolved citation asking to be retried
A cite call naming one good id and one mangled one registered the good one and then asked the model to cite again. A model that mangles ids obeys, mangles again, and the run dies on output retries with the answer lost: observed at 22 consecutive cite calls against gemma4-26b, with the citation policy registered to press for a declaration. The branch is only reachable once something has registered, so the answer already has grounding and there is nothing to ask for.
This commit is contained in:
parent
3ec72363bc
commit
21e261f608
3 changed files with 9 additions and 3 deletions
|
|
@ -22,6 +22,7 @@
|
|||
|
||||
### Fixed
|
||||
|
||||
- `rag_cite` and `analysis_cite` no longer ask for another call when a call resolved some ids and not others.
|
||||
- `cross-encoder` reranking no longer ties the scores of strongly-relevant candidates, which left their order to the sort. Scores remain 0-1.
|
||||
- `haiku-rag` and `haiku-ingester` CLI startup no longer imports `lancedb`, `pyarrow` and `pydantic_ai`.
|
||||
- `haiku.rag.store` no longer re-exports `Store`; import it from `haiku.rag.store.engine`.
|
||||
|
|
|
|||
|
|
@ -487,11 +487,14 @@ class RAGCapabilityBase[StateT: BaseModel](AbstractCapability[Any]):
|
|||
resolved = {citation.chunk_id for citation in citations}
|
||||
unresolved = [cid for cid in missing if cid not in resolved]
|
||||
if unresolved:
|
||||
# States the outcome without asking for another call. Reaching here
|
||||
# means something registered, so the answer already has grounding: a
|
||||
# model that keeps mangling ids would obey an invitation to retry
|
||||
# until the run dies on output retries.
|
||||
return (
|
||||
f"Registered {len(citations)} citation(s); "
|
||||
f"ignored {len(unresolved)} unresolvable id(s): "
|
||||
f"{unresolved}. Copy chunk_ids verbatim from search "
|
||||
"results and cite again."
|
||||
f"{unresolved}, which were not verbatim from search results."
|
||||
)
|
||||
return f"Registered {len(citations)} citation(s)."
|
||||
|
||||
|
|
|
|||
|
|
@ -362,7 +362,9 @@ async def test_cite_reports_unresolved_ids_on_partial_success(temp_db_path):
|
|||
assert "Registered 1 citation(s)" in result
|
||||
assert "6.43" in result
|
||||
assert "6.51.2" in result
|
||||
assert "verbatim" in result
|
||||
# Never ask for another call here: something did register, and a model that
|
||||
# keeps mangling ids obeys the ask until the run dies on output retries.
|
||||
assert "again" not in result
|
||||
assert capability.state.citations == ["chunk-1"]
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue