Port cite partial-success feedback to capabilities

This commit is contained in:
Yiorgis Gozadinos 2026-07-22 13:34:28 +03:00
parent 175929f23a
commit b8dcb066dc
No known key found for this signature in database
3 changed files with 54 additions and 16 deletions

View file

@ -1,6 +1,25 @@
# Changelog
## [Unreleased]
### Added
- Native deferred Pydantic AI `RAGCapability` and `AnalysisCapability` implementations under `haiku.rag.capabilities`, with namespaced host state and lazy per-run database and sandbox resources.
- Prior-turn RAG and analysis tool results are compacted before model requests while current-turn evidence remains intact.
- Per-question capability request limits force a final answer from gathered evidence by removing only the exhausted capability's tools; unrelated agent and capability tools remain available.
### Changed
- Require `pydantic-ai-slim>=2.11,<3`; the `vertexai` optional extra now installs Pydantic AI's `google` extra.
- The chat TUI consumes native Pydantic AI stream events. The web example uses the standard `AGUIAdapter` and emits one final state snapshot instead of forwarding sub-agent activity and per-tool state events.
- Chat capability selection is now `haiku-rag chat --capability/-c {rag,analysis}`. Migrate from `--skill/-s`.
- Evaluation targets are now `rag-capability` and `analysis-capability`, and the model override is `--capability-model`. Migrate from `rag-skill`, `analysis-skill`, and `--skill-model`.
### Removed
- The `haiku.skills` dependency, `haiku.rag.skills` modules, Python entry-point discovery, and sub-agent execution layer. Migrate `create_skill(...)` plus `SkillToolset` usage to `haiku.rag.capabilities.*.create_capability(...)` passed through `Agent(capabilities=[...])`.
- The `haiku-rag create-skill` package generator. Compose native capabilities directly and package application-specific instructions and data in the consuming project.
- Legacy sub-agent `ActivitySnapshotEvent` plumbing and per-tool `StateDeltaEvent` generation.
## [0.68.0] - 2026-07-24
### Added
@ -28,22 +47,6 @@
### Added
- `hotpotqa` evaluation dataset.
- Native deferred Pydantic AI `RAGCapability` and `AnalysisCapability` implementations under `haiku.rag.capabilities`, with namespaced host state and lazy per-run database and sandbox resources.
- Prior-turn RAG and analysis tool results are compacted before model requests while current-turn evidence remains intact.
- Per-question capability request limits force a final answer from gathered evidence by removing only the exhausted capability's tools; unrelated agent and capability tools remain available.
### Changed
- Require `pydantic-ai-slim>=2.11,<3`; the `vertexai` optional extra now installs Pydantic AI's `google` extra.
- The chat TUI consumes native Pydantic AI stream events. The web example uses the standard `AGUIAdapter` and emits one final state snapshot instead of forwarding sub-agent activity and per-tool state events.
- Chat capability selection is now `haiku-rag chat --capability/-c {rag,analysis}`. Migrate from `--skill/-s`.
- Evaluation targets are now `rag-capability` and `analysis-capability`, and the model override is `--capability-model`. Migrate from `rag-skill`, `analysis-skill`, and `--skill-model`.
### Removed
- The `haiku.skills` dependency, `haiku.rag.skills` modules, Python entry-point discovery, and sub-agent execution layer. Migrate `create_skill(...)` plus `SkillToolset` usage to `haiku.rag.capabilities.*.create_capability(...)` passed through `Agent(capabilities=[...])`.
- The `haiku-rag create-skill` package generator. Compose native capabilities directly and package application-specific instructions and data in the consuming project.
- Legacy sub-agent `ActivitySnapshotEvent` plumbing and per-tool `StateDeltaEvent` generation.
### Changed

View file

@ -264,6 +264,15 @@ class RAGCapabilityBase[StateT: BaseModel](AbstractCapability[Any]):
"Copy chunk_ids verbatim from search results."
)
self._register_citations(citations)
resolved = {citation.chunk_id for citation in citations}
unresolved = [cid for cid in missing if cid not in resolved]
if unresolved:
return (
f"Registered {len(citations)} citation(s); "
f"ignored {len(unresolved)} unresolvable id(s): "
f"{unresolved}. Copy chunk_ids verbatim from search "
"results and cite again."
)
return f"Registered {len(citations)} citation(s)."
def _register_citations(self, citations: list[Citation]) -> None:

View file

@ -326,6 +326,32 @@ async def test_cite_resolves_direct_chunk_ids_and_reuses_document_lookup(temp_db
client.get_document_by_id.assert_awaited_once_with("doc-1")
@pytest.mark.asyncio
async def test_cite_reports_unresolved_ids_on_partial_success(temp_db_path):
capability = create_rag(db_path=temp_db_path, config=AppConfig())
capability.state = RAGState()
client = AsyncMock()
client.get_chunk_by_id.side_effect = [
Chunk(id="chunk-1", document_id="doc-1", content="first"),
None,
None,
]
client.get_document_by_id.return_value = SimpleNamespace(
uri="test://document",
title="Document",
metadata={},
)
capability.rag = client
result = await capability._cite(["chunk-1", "6.43", "6.51.2"])
assert "Registered 1 citation(s)" in result
assert "6.43" in result
assert "6.51.2" in result
assert "verbatim" in result
assert capability.state.citations == ["chunk-1"]
@pytest.mark.asyncio
async def test_analysis_records_new_sandbox_search_results(temp_db_path):
capability = create_analysis(db_path=temp_db_path, config=AppConfig())