Port cite partial-success feedback to capabilities
This commit is contained in:
parent
175929f23a
commit
b8dcb066dc
3 changed files with 54 additions and 16 deletions
35
CHANGELOG.md
35
CHANGELOG.md
|
|
@ -1,6 +1,25 @@
|
|||
# Changelog
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
|
||||
- Native deferred Pydantic AI `RAGCapability` and `AnalysisCapability` implementations under `haiku.rag.capabilities`, with namespaced host state and lazy per-run database and sandbox resources.
|
||||
- Prior-turn RAG and analysis tool results are compacted before model requests while current-turn evidence remains intact.
|
||||
- Per-question capability request limits force a final answer from gathered evidence by removing only the exhausted capability's tools; unrelated agent and capability tools remain available.
|
||||
|
||||
### Changed
|
||||
|
||||
- Require `pydantic-ai-slim>=2.11,<3`; the `vertexai` optional extra now installs Pydantic AI's `google` extra.
|
||||
- The chat TUI consumes native Pydantic AI stream events. The web example uses the standard `AGUIAdapter` and emits one final state snapshot instead of forwarding sub-agent activity and per-tool state events.
|
||||
- Chat capability selection is now `haiku-rag chat --capability/-c {rag,analysis}`. Migrate from `--skill/-s`.
|
||||
- Evaluation targets are now `rag-capability` and `analysis-capability`, and the model override is `--capability-model`. Migrate from `rag-skill`, `analysis-skill`, and `--skill-model`.
|
||||
|
||||
### Removed
|
||||
|
||||
- The `haiku.skills` dependency, `haiku.rag.skills` modules, Python entry-point discovery, and sub-agent execution layer. Migrate `create_skill(...)` plus `SkillToolset` usage to `haiku.rag.capabilities.*.create_capability(...)` passed through `Agent(capabilities=[...])`.
|
||||
- The `haiku-rag create-skill` package generator. Compose native capabilities directly and package application-specific instructions and data in the consuming project.
|
||||
- Legacy sub-agent `ActivitySnapshotEvent` plumbing and per-tool `StateDeltaEvent` generation.
|
||||
|
||||
## [0.68.0] - 2026-07-24
|
||||
|
||||
### Added
|
||||
|
|
@ -28,22 +47,6 @@
|
|||
### Added
|
||||
|
||||
- `hotpotqa` evaluation dataset.
|
||||
- Native deferred Pydantic AI `RAGCapability` and `AnalysisCapability` implementations under `haiku.rag.capabilities`, with namespaced host state and lazy per-run database and sandbox resources.
|
||||
- Prior-turn RAG and analysis tool results are compacted before model requests while current-turn evidence remains intact.
|
||||
- Per-question capability request limits force a final answer from gathered evidence by removing only the exhausted capability's tools; unrelated agent and capability tools remain available.
|
||||
|
||||
### Changed
|
||||
|
||||
- Require `pydantic-ai-slim>=2.11,<3`; the `vertexai` optional extra now installs Pydantic AI's `google` extra.
|
||||
- The chat TUI consumes native Pydantic AI stream events. The web example uses the standard `AGUIAdapter` and emits one final state snapshot instead of forwarding sub-agent activity and per-tool state events.
|
||||
- Chat capability selection is now `haiku-rag chat --capability/-c {rag,analysis}`. Migrate from `--skill/-s`.
|
||||
- Evaluation targets are now `rag-capability` and `analysis-capability`, and the model override is `--capability-model`. Migrate from `rag-skill`, `analysis-skill`, and `--skill-model`.
|
||||
|
||||
### Removed
|
||||
|
||||
- The `haiku.skills` dependency, `haiku.rag.skills` modules, Python entry-point discovery, and sub-agent execution layer. Migrate `create_skill(...)` plus `SkillToolset` usage to `haiku.rag.capabilities.*.create_capability(...)` passed through `Agent(capabilities=[...])`.
|
||||
- The `haiku-rag create-skill` package generator. Compose native capabilities directly and package application-specific instructions and data in the consuming project.
|
||||
- Legacy sub-agent `ActivitySnapshotEvent` plumbing and per-tool `StateDeltaEvent` generation.
|
||||
|
||||
### Changed
|
||||
|
||||
|
|
|
|||
|
|
@ -264,6 +264,15 @@ class RAGCapabilityBase[StateT: BaseModel](AbstractCapability[Any]):
|
|||
"Copy chunk_ids verbatim from search results."
|
||||
)
|
||||
self._register_citations(citations)
|
||||
resolved = {citation.chunk_id for citation in citations}
|
||||
unresolved = [cid for cid in missing if cid not in resolved]
|
||||
if unresolved:
|
||||
return (
|
||||
f"Registered {len(citations)} citation(s); "
|
||||
f"ignored {len(unresolved)} unresolvable id(s): "
|
||||
f"{unresolved}. Copy chunk_ids verbatim from search "
|
||||
"results and cite again."
|
||||
)
|
||||
return f"Registered {len(citations)} citation(s)."
|
||||
|
||||
def _register_citations(self, citations: list[Citation]) -> None:
|
||||
|
|
|
|||
|
|
@ -326,6 +326,32 @@ async def test_cite_resolves_direct_chunk_ids_and_reuses_document_lookup(temp_db
|
|||
client.get_document_by_id.assert_awaited_once_with("doc-1")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cite_reports_unresolved_ids_on_partial_success(temp_db_path):
|
||||
capability = create_rag(db_path=temp_db_path, config=AppConfig())
|
||||
capability.state = RAGState()
|
||||
client = AsyncMock()
|
||||
client.get_chunk_by_id.side_effect = [
|
||||
Chunk(id="chunk-1", document_id="doc-1", content="first"),
|
||||
None,
|
||||
None,
|
||||
]
|
||||
client.get_document_by_id.return_value = SimpleNamespace(
|
||||
uri="test://document",
|
||||
title="Document",
|
||||
metadata={},
|
||||
)
|
||||
capability.rag = client
|
||||
|
||||
result = await capability._cite(["chunk-1", "6.43", "6.51.2"])
|
||||
|
||||
assert "Registered 1 citation(s)" in result
|
||||
assert "6.43" in result
|
||||
assert "6.51.2" in result
|
||||
assert "verbatim" in result
|
||||
assert capability.state.citations == ["chunk-1"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_analysis_records_new_sandbox_search_results(temp_db_path):
|
||||
capability = create_analysis(db_path=temp_db_path, config=AppConfig())
|
||||
|
|
|
|||
Loading…
Reference in a new issue