Human-in-the-loop in ag-ui-example

This commit is contained in:
Yiorgis Gozadinos 2025-12-17 12:03:42 +02:00
parent 9818cac4eb
commit ff13be1210
No known key found for this signature in database
8 changed files with 443 additions and 92 deletions

View file

@ -12,6 +12,8 @@
- New `human_decide` graph node emits AG-UI tool calls (`TOOL_CALL_START/ARGS/END`) for frontend integration
- New `emit_tool_call_start()`, `emit_tool_call_args()`, `emit_tool_call_end()` AG-UI event helpers
- New `AGUIEmitter.emit()` method for direct event emission
- **AG-UI Research Example**: Updated with interactive decision UI
- Decision panel with question editing (add/remove) at each decision point
- **HotpotQA Evaluation**: Added HotpotQA dataset adapter for multi-hop QA benchmarks
- Extracts unique documents from validation set context paragraphs
- Uses MAP for retrieval evaluation (multiple supporting documents per question)

View file

@ -219,4 +219,8 @@ In interactive mode, you can:
- Execute searches and review collected answers
- Continue researching or synthesize when ready
For a web-based interactive experience, see the [AG-UI Research Example](https://github.com/ggozad/haiku.rag/tree/main/examples/ag-ui-research).
For a web-based interactive experience with visual decision UI, see the [AG-UI Research Example](https://github.com/ggozad/haiku.rag/tree/main/examples/ag-ui-research). The web interface provides:
- Question editing panel to add/remove sub-questions at decision points
- Search and Generate Report buttons for controlling research flow
- Live state display showing answers, confidence, and progress

View file

@ -1,13 +1,13 @@
# Interactive Research Assistant
Research assistant powered by [haiku.rag](https://ggozad.github.io/haiku.rag/), [Pydantic Graph](https://ai.pydantic.dev/graph/), and [AG-UI](https://docs.ag-ui.com/). Ask complex questions and watch the research process unfold in real-time.
Research assistant powered by [haiku.rag](https://ggozad.github.io/haiku.rag/), [Pydantic Graph](https://ai.pydantic.dev/graph/), and [AG-UI](https://docs.ag-ui.com/). Ask complex questions and watch the research process unfold in real-time with human-in-the-loop control.
[Watch demo video](https://vimeo.com/1128874386)
## Features
- **Multi-iteration research graph**: Automated question decomposition and search
- **Intelligent evaluation**: Confidence-based decision making with automatic iteration until sufficient information is gathered
- **Human-in-the-loop research**: Review and modify questions at decision points, then continue searching or generate report
- **Multi-iteration research graph**: Automated question decomposition and parallel search
- **Live state synchronization**: Real-time delta updates of research progress via AG-UI protocol
- **Rich reporting**: Generates comprehensive research reports with findings, conclusions, and sources
@ -25,9 +25,7 @@ Research assistant powered by [haiku.rag](https://ggozad.github.io/haiku.rag/),
**Option A: Create a new database**
```bash
mkdir -p data
haiku-rag add "Your documents here" --db data/haiku_rag.lancedb
# Or add from files
haiku-rag init --db data/haiku_rag.lancedb
haiku-rag add-src document.pdf --db data/haiku_rag.lancedb
```
@ -63,27 +61,29 @@ Research assistant powered by [haiku.rag](https://ggozad.github.io/haiku.rag/),
DB_PATH=/path/to/your/existing/haiku_rag.lancedb # If using an existing db.
```
1. **Start the application**
4. **Start the application**
```bash
docker compose up --build
```
2. **Access the interface**
5. **Access the interface**
- Frontend: http://localhost:3000
- Backend health: http://localhost:8000/health
## How It Works
1. **Ask a question**: Type your research question in the chat
2. **Plan phase**: The research graph automatically:
- Decomposes your question into targeted sub-questions
- Gathers initial context about the topic
3. **Research iterations**: The graph autonomously:
- Searches the knowledge base for each sub-question in parallel
- Assesses confidence in gathered information
- Generates new follow-up questions if needed
- Iterates until confidence threshold is met or max iterations reached
4. **Synthesis**: Generates a comprehensive research report with:
2. **Plan phase**: The research graph decomposes your question into targeted sub-questions
3. **Decision point**: Review the proposed questions in the right panel
- Add new questions using the input field
- Remove questions you don't need
- Click **Search** to execute searches for pending questions
- Click **Generate Report** to skip to synthesis (when you have enough answers)
4. **Research iterations**: After each search cycle, you return to a decision point where you can:
- Review collected answers
- Add follow-up questions based on findings
- Continue searching or generate the final report
5. **Synthesis**: Generates a comprehensive research report with:
- Executive summary
- Main findings with supporting evidence
- Conclusions and recommendations
@ -99,32 +99,35 @@ This example demonstrates the **agent+graph** architecture pattern:
- Pydantic AI agent handles user conversations
- Decides when to invoke the research tool based on user intent
- Responds directly to greetings/casual chat without tools
- Formats research results for the user
2. **Research Graph** (haiku.rag):
2. **Interactive Research Graph** (haiku.rag):
- Multi-step research workflow invoked by the agent's tool
- Autonomous execution with plan → search → analyze → decide → synthesize flow
- Pauses at decision points waiting for human input via async queue
- Emits AG-UI events for real-time progress tracking
3. **Shared Event Stream**:
3. **Decision Endpoint** (`main.py`):
- `/v1/research/decide` receives human decisions from frontend
- Forwards decisions to the waiting graph via `HumanDecision` queue
- Supports actions: `search`, `synthesize`, `modify_questions`
4. **Shared Event Stream**:
- `AGUIEmitter` is shared between agent and graph
- Events from both flow through a single stream to the frontend
- Custom streaming endpoint (`main.py`) uses anyio memory streams for proper async handling
- `STATE_DELTA` events sync research state to frontend in real-time
### Components
- **Backend** (Python):
- Uses published `ghcr.io/ggozad/haiku.rag:latest` Docker image as base
- `agent.py`: Pydantic AI agent with `run_research` tool
- `main.py`: Custom AG-UI streaming endpoint with anyio memory object streams
- `agent.py`: Pydantic AI agent with `run_research` tool, manages `ActiveResearch` registry
- `main.py`: Custom AG-UI streaming endpoint, decision endpoint for human input
- Real-time event forwarding from emitter to SSE stream
- Filters out `ACTIVITY_SNAPSHOT` events (not yet supported by CopilotKit)
- **Frontend** (Next.js/React):
- CopilotKit for AG-UI protocol integration
- Split-pane UI: chat on left, live research state on right
- Decision UI: question editor with add/remove, search and generate report buttons
- Real-time state synchronization via Server-Sent Events (SSE)
- `StateDisplay` component with collapsible sections for questions and report
## Configuration

View file

@ -1,6 +1,7 @@
"""Research assistant agent with graph integration."""
from dataclasses import dataclass
import asyncio
from dataclasses import dataclass, field
from pathlib import Path
from typing import TYPE_CHECKING
@ -11,7 +12,7 @@ from haiku.rag.config import load_yaml_config
from haiku.rag.config.models import AppConfig
from haiku.rag.graph.research.dependencies import ResearchContext
from haiku.rag.graph.research.graph import build_research_graph
from haiku.rag.graph.research.state import ResearchDeps, ResearchState
from haiku.rag.graph.research.state import HumanDecision, ResearchDeps, ResearchState
from haiku.rag.utils import get_model
if TYPE_CHECKING:
@ -27,6 +28,20 @@ Config = (
)
@dataclass
class ActiveResearch:
"""Tracks state for active research awaiting human decision."""
queue: asyncio.Queue[HumanDecision]
sub_questions: list[str] = field(default_factory=list)
qa_responses: list[dict] = field(default_factory=list)
original_question: str = ""
# Global registry of active research by thread_id
_active_research: dict[str, ActiveResearch] = {}
@dataclass
class AgentDeps:
"""Dependencies for research agent."""
@ -34,6 +49,7 @@ class AgentDeps:
client: HaikuRAG
agui_emitter: "AGUIEmitter[ResearchState, ResearchReport] | None" = None
search_filter: str | None = None
thread_id: str | None = None
model = get_model(Config.research.model, Config)
@ -50,10 +66,10 @@ CRITICAL RULES:
4. NEVER answer substantive questions from your own knowledge - always use the tool
How to decide:
- "Hi" / "Hello" / "How are you?" Respond directly, NO tools
- "What can you do?" Respond directly, NO tools
- "How does X work in the codebase?" Use run_research tool
- "Tell me about Y" Use run_research tool
- "Hi" / "Hello" / "How are you?" -> Respond directly, NO tools
- "What can you do?" -> Respond directly, NO tools
- "How does X work in the codebase?" -> Use run_research tool
- "Tell me about Y" -> Use run_research tool
When you use run_research, the graph will decompose questions, search the knowledge base,
and generate a comprehensive report.
@ -70,23 +86,39 @@ async def run_research(ctx: RunContext[AgentDeps], question: str) -> str:
DO NOT use for greetings or casual conversation.
"""
if ctx.deps.agui_emitter:
ctx.deps.agui_emitter.log(f"🔍 Starting research on: {question}")
ctx.deps.agui_emitter.log(f"Starting research on: {question}")
graph = build_research_graph(Config)
# Create queue for human decisions
queue: asyncio.Queue[HumanDecision] = asyncio.Queue()
# Build interactive graph
graph = build_research_graph(Config, interactive=True)
context = ResearchContext(original_question=question)
state = ResearchState.from_config(context=context, config=Config)
state.search_filter = ctx.deps.search_filter
# Register active research for decision endpoint to find
thread_id = ctx.deps.thread_id
if thread_id:
_active_research[thread_id] = ActiveResearch(
queue=queue,
sub_questions=[],
qa_responses=[],
original_question=question,
)
graph_deps = ResearchDeps(
client=ctx.deps.client,
agui_emitter=ctx.deps.agui_emitter,
human_input_queue=queue,
interactive=True,
)
try:
result = await graph.run(state=state, deps=graph_deps)
if ctx.deps.agui_emitter:
ctx.deps.agui_emitter.log("✅ Research complete!")
ctx.deps.agui_emitter.log("Research complete!")
return f"""Research completed successfully!
@ -108,5 +140,9 @@ The full research report with all citations has been provided to the user.
except Exception as e:
if ctx.deps.agui_emitter:
ctx.deps.agui_emitter.log(f"Research error: {str(e)}")
ctx.deps.agui_emitter.log(f"Research error: {str(e)}")
return f"I encountered an error while researching: {str(e)}"
finally:
# Cleanup
if thread_id and thread_id in _active_research:
del _active_research[thread_id]

View file

@ -1,8 +1,9 @@
import json
import logging
import os
from pathlib import Path
from agent import AgentDeps, agent
from agent import AgentDeps, _active_research, agent
from anyio import create_memory_object_stream, create_task_group
from anyio.streams.memory import MemoryObjectSendStream
from starlette.applications import Starlette
@ -19,7 +20,7 @@ from haiku.rag.graph.agui.emitter import AGUIEmitter
from haiku.rag.graph.agui.server import RunAgentInput, format_sse_event
from haiku.rag.graph.research.dependencies import ResearchContext
from haiku.rag.graph.research.models import ResearchReport
from haiku.rag.graph.research.state import ResearchState
from haiku.rag.graph.research.state import HumanDecision, ResearchState
logging.basicConfig(
level=logging.INFO, format="%(asctime)s - %(name)s - %(levelname)s - %(message)s"
@ -79,11 +80,11 @@ async def stream_research_agent(request: Request) -> StreamingResponse:
"""Execute agent and forward emitter events to memory stream."""
async with send_stream:
try:
# Create shared emitter
# Create shared emitter (use_deltas=True for CopilotKit compatibility)
emitter: AGUIEmitter[ResearchState, ResearchReport] = AGUIEmitter(
thread_id=input_data.thread_id,
run_id=input_data.run_id,
use_deltas=False,
use_deltas=True,
)
# Get client
@ -99,11 +100,14 @@ async def stream_research_agent(request: Request) -> StreamingResponse:
ids_str = ", ".join(f"'{id}'" for id in document_ids)
search_filter = f"id IN ({ids_str})"
thread_id = input_data.thread_id
# Create agent dependencies with shared emitter
agent_deps = AgentDeps(
client=client,
agui_emitter=emitter,
search_filter=search_filter,
thread_id=thread_id,
)
# Start run with empty initial state
@ -148,6 +152,37 @@ async def stream_research_agent(request: Request) -> StreamingResponse:
await send_stream.send(format_sse_event(delta_event))
continue
# When human_decision tool starts, set awaiting_decision flag
if event_type == "TOOL_CALL_START":
tool_name = event.get("toolCallName")
if tool_name == "human_decision":
delta_event = {
"type": "STATE_DELTA",
"delta": [
{
"op": "add",
"path": "/awaiting_decision",
"value": True,
}
],
}
await send_stream.send(format_sse_event(delta_event))
# Sync state to ActiveResearch when human_decision tool call
if event_type == "TOOL_CALL_ARGS" and thread_id:
delta = event.get("delta", "{}")
args = (
json.loads(delta) if isinstance(delta, str) else delta
)
active = _active_research.get(thread_id)
if active:
active.sub_questions = list(
args.get("sub_questions", [])
)
active.qa_responses = list(args.get("qa_responses", []))
if "original_question" in args:
active.original_question = args["original_question"]
await send_stream.send(format_sse_event(event))
# Run agent and event forwarding concurrently
@ -248,10 +283,36 @@ async def visualize_chunk(request: Request) -> JSONResponse:
)
async def research_decide(request: Request) -> JSONResponse:
"""Endpoint to receive human decisions for active research."""
body = await request.json()
action = body.get("action", "search")
questions = body.get("questions", [])
# Get first active research (single-user example)
active = next(iter(_active_research.values()), None)
if not active:
return JSONResponse({"error": "No active research found"}, status_code=404)
# When "search" action is sent with questions, use "modify_questions" to update them
effective_action = (
"modify_questions" if action == "search" and questions else action
)
decision = HumanDecision(
action=effective_action,
questions=questions or None,
)
await active.queue.put(decision)
return JSONResponse({"status": "ok", "action": effective_action})
# Create Starlette app
app = Starlette(
routes=[
Route("/v1/research/stream", stream_research_agent, methods=["POST"]),
Route("/v1/research/decide", research_decide, methods=["POST"]),
Route("/api/documents", list_documents, methods=["GET"]),
Route("/api/visualize/{chunk_id}", visualize_chunk, methods=["GET"]),
Route("/health", health_check, methods=["GET"]),

View file

@ -3,6 +3,7 @@
import { CopilotKit, useCoAgent } from "@copilotkit/react-core";
import { CopilotChat } from "@copilotkit/react-ui";
import "@copilotkit/react-ui/styles.css";
import { useState, useEffect } from "react";
import DocumentSelector from "./DocumentSelector";
import StateDisplay from "./StateDisplay";
@ -58,6 +59,7 @@ interface ResearchState {
current_activity?: string;
current_activity_message?: string;
documentFilter?: string[];
awaiting_decision?: boolean;
}
function AgentContent() {
@ -75,13 +77,61 @@ function AgentContent() {
max_concurrency: 1,
last_eval: null,
documentFilter: [],
awaiting_decision: false,
},
});
const [editableQuestions, setEditableQuestions] = useState<string[]>([]);
const [newQuestion, setNewQuestion] = useState("");
const [submitting, setSubmitting] = useState(false);
// Sync editable questions when state changes
useEffect(() => {
if (state.awaiting_decision && state.context.sub_questions) {
setEditableQuestions([...state.context.sub_questions]);
}
}, [state.awaiting_decision, state.context.sub_questions]);
const handleDocumentFilterChange = (ids: string[]) => {
setState({ ...state, documentFilter: ids });
};
const handleRemoveQuestion = (index: number) => {
setEditableQuestions(editableQuestions.filter((_, i) => i !== index));
};
const handleAddQuestion = () => {
if (newQuestion.trim()) {
setEditableQuestions([...editableQuestions, newQuestion.trim()]);
setNewQuestion("");
}
};
const handleDecision = async (action: "search" | "synthesize") => {
setSubmitting(true);
try {
const response = await fetch(
`${process.env.NEXT_PUBLIC_API_URL || "http://localhost:8000"}/v1/research/decide`,
{
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
thread_id: state.context.original_question, // Use as identifier
action,
questions: editableQuestions,
}),
}
);
if (response.ok) {
setState({ ...state, awaiting_decision: false });
}
} catch (error) {
console.error("Failed to send decision:", error);
} finally {
setSubmitting(false);
}
};
return (
<>
<style>{`
@ -141,13 +191,162 @@ function AgentContent() {
</p>
</header>
<div style={{ marginBottom: "1rem" }}>
<DocumentSelector
selected={state.documentFilter || []}
onChange={handleDocumentFilterChange}
disabled={running}
/>
</div>
{/* Document filter - hidden when research is running */}
{!running && (
<div style={{ marginBottom: "1rem" }}>
<DocumentSelector
selected={state.documentFilter || []}
onChange={handleDocumentFilterChange}
/>
</div>
)}
{/* Decision UI when awaiting human input - hidden when report exists, submitting, or not running */}
{state.awaiting_decision && !submitting && !state.result && running && (
<div
style={{
marginBottom: "1rem",
background: "#f0f9ff",
border: "2px solid #0ea5e9",
borderRadius: "8px",
padding: "1rem",
}}
>
<div
style={{
fontWeight: "bold",
color: "#0369a1",
marginBottom: "0.75rem",
fontSize: "1rem",
}}
>
Research Decision Point
</div>
<div style={{ fontSize: "0.85rem", color: "#64748b", marginBottom: "0.75rem" }}>
{state.context.qa_responses?.length || 0} answers collected | Iteration {state.iterations || 0}
</div>
{/* Questions list */}
<div style={{ marginBottom: "0.75rem" }}>
<div style={{ fontSize: "0.8rem", color: "#475569", marginBottom: "0.5rem" }}>
Pending Questions ({editableQuestions.length}):
</div>
{editableQuestions.map((q, idx) => (
<div
key={idx}
style={{
display: "flex",
alignItems: "center",
gap: "0.5rem",
padding: "0.375rem 0.5rem",
background: "white",
borderRadius: "4px",
marginBottom: "0.25rem",
fontSize: "0.85rem",
}}
>
<span style={{ flex: 1 }}>{q}</span>
<button
type="button"
onClick={() => handleRemoveQuestion(idx)}
disabled={submitting}
style={{
background: "#ef4444",
color: "white",
border: "none",
borderRadius: "4px",
padding: "0.25rem 0.5rem",
cursor: submitting ? "not-allowed" : "pointer",
fontSize: "0.75rem",
}}
>
Remove
</button>
</div>
))}
</div>
{/* Add question input */}
<div style={{ display: "flex", gap: "0.5rem", marginBottom: "1rem" }}>
<input
type="text"
value={newQuestion}
onChange={(e) => setNewQuestion(e.target.value)}
placeholder="Add a new question..."
disabled={submitting}
style={{
flex: 1,
padding: "0.5rem",
border: "1px solid #cbd5e1",
borderRadius: "4px",
fontSize: "0.85rem",
}}
onKeyDown={(e) => {
if (e.key === "Enter") {
handleAddQuestion();
}
}}
/>
<button
type="button"
onClick={handleAddQuestion}
disabled={submitting}
style={{
background: "#22c55e",
color: "white",
border: "none",
borderRadius: "4px",
padding: "0.5rem 1rem",
cursor: submitting ? "not-allowed" : "pointer",
fontSize: "0.85rem",
}}
>
Add
</button>
</div>
{/* Action buttons */}
<div style={{ display: "flex", gap: "0.5rem" }}>
<button
type="button"
onClick={() => handleDecision("search")}
disabled={editableQuestions.length === 0 || submitting}
style={{
flex: 1,
background: editableQuestions.length === 0 || submitting ? "#94a3b8" : "#0ea5e9",
color: "white",
border: "none",
borderRadius: "4px",
padding: "0.75rem",
cursor: editableQuestions.length === 0 || submitting ? "not-allowed" : "pointer",
fontWeight: "bold",
fontSize: "0.9rem",
}}
>
{submitting ? "Submitting..." : `Search (${editableQuestions.length})`}
</button>
<button
type="button"
onClick={() => handleDecision("synthesize")}
disabled={submitting || (state.context.qa_responses?.length || 0) === 0}
style={{
flex: 1,
background: submitting || (state.context.qa_responses?.length || 0) === 0 ? "#94a3b8" : "#8b5cf6",
color: "white",
border: "none",
borderRadius: "4px",
padding: "0.75rem",
cursor: submitting || (state.context.qa_responses?.length || 0) === 0 ? "not-allowed" : "pointer",
fontWeight: "bold",
fontSize: "0.9rem",
}}
>
Generate Report
</button>
</div>
</div>
)}
<StateDisplay state={state} />
</div>

View file

@ -192,8 +192,8 @@ export default function StateDisplay({ state }: StateDisplayProps) {
</div>
)}
{/* Research Progress - only show when research has started */}
{(state.iterations > 0 || (state.current_activity && !state.result)) && (
{/* Research Progress - only show when research is in progress (not when complete) */}
{(state.iterations > 0 || state.current_activity) && !state.result && (
<div
style={{
background: "white",
@ -202,8 +202,8 @@ export default function StateDisplay({ state }: StateDisplayProps) {
boxShadow: "0 1px 3px rgba(0,0,0,0.1)",
}}
>
{/* Current Activity - hide when complete */}
{state.current_activity && !state.result && (
{/* Current Activity */}
{state.current_activity && (
<div
style={{
padding: "0.75rem",
@ -363,9 +363,8 @@ export default function StateDisplay({ state }: StateDisplayProps) {
</div>
)}
{/* Sub-Questions and QA Responses */}
{(state.context.sub_questions.length > 0 ||
state.context.qa_responses.length > 0) && (
{/* Answers */}
{state.context.qa_responses.length > 0 && (
<div
style={{
background: "white",
@ -392,10 +391,7 @@ export default function StateDisplay({ state }: StateDisplayProps) {
color: "#2d3748",
}}
>
<span>
Sub-Questions ({state.context.sub_questions.length}) Answers (
{state.context.qa_responses.length})
</span>
<span>Answers ({state.context.qa_responses.length})</span>
<span>{expandedSections.questions ? "▼" : "▶"}</span>
</button>
{expandedSections.questions && (
@ -408,38 +404,6 @@ export default function StateDisplay({ state }: StateDisplayProps) {
borderRadius: "0 0 4px 4px",
}}
>
{/* Show pending sub_questions */}
{state.context.sub_questions.map((question, idx) => (
<div
key={`pending-${idx}`}
style={{
marginBottom: "0.5rem",
background: "white",
borderRadius: "4px",
border: "1px solid #e2e8f0",
padding: "0.75rem",
display: "flex",
gap: "0.75rem",
alignItems: "center",
}}
>
<div
style={{
fontSize: "1.25rem",
color: "#a0aec0",
flexShrink: 0,
}}
>
</div>
<div
style={{ flex: 1, fontSize: "0.875rem", color: "#4a5568" }}
>
<Markdown content={question} />
</div>
</div>
))}
{/* Show all qa_responses (each has query + answer) */}
{state.context.qa_responses.map((qaResponse, idx) => {
const questionId = `q-${idx}`;

View file

@ -1,3 +1,5 @@
import asyncio
import pytest
from pydantic_ai.models.test import TestModel
@ -5,7 +7,7 @@ from haiku.rag.client import HaikuRAG
from haiku.rag.graph.agui.stream import stream_graph
from haiku.rag.graph.research.dependencies import ResearchContext
from haiku.rag.graph.research.graph import build_research_graph
from haiku.rag.graph.research.state import ResearchDeps, ResearchState
from haiku.rag.graph.research.state import HumanDecision, ResearchDeps, ResearchState
@pytest.mark.asyncio
@ -61,3 +63,83 @@ async def test_graph_end_to_end_with_test_model(monkeypatch, temp_db_path):
assert "STEP_STARTED" in event_types
client.close()
@pytest.mark.asyncio
async def test_interactive_graph_with_human_decision(monkeypatch, temp_db_path):
"""Test interactive research graph pauses and resumes with human decisions."""
# Mock get_model to return TestModel
def test_model_factory(_provider, _model, _config=None):
return TestModel()
monkeypatch.setattr("haiku.rag.utils.get_model", test_model_factory)
monkeypatch.setattr("haiku.rag.graph.research.graph.get_model", test_model_factory)
# Build interactive graph
graph = build_research_graph(interactive=True)
state = ResearchState(
context=ResearchContext(original_question="What is haiku.rag?"),
max_iterations=1,
confidence_threshold=0.5,
max_concurrency=2,
)
# Create human input queue
human_input_queue: asyncio.Queue[HumanDecision] = asyncio.Queue()
client = HaikuRAG(temp_db_path, create=True)
deps = ResearchDeps(
client=client,
human_input_queue=human_input_queue,
interactive=True,
)
events = []
tool_call_received = asyncio.Event()
result = None
async def run_graph():
nonlocal result
async for event in stream_graph(graph, state, deps):
events.append(event)
if event["type"] == "TOOL_CALL_START":
tool_name = event.get("toolCallName")
if tool_name == "human_decision":
tool_call_received.set()
elif event["type"] == "RUN_FINISHED":
result = event["result"]
elif event["type"] == "RUN_ERROR":
pytest.fail(f"Graph execution failed: {event['message']}")
async def send_decisions():
# Wait for first tool call (after planning)
await asyncio.wait_for(tool_call_received.wait(), timeout=30)
tool_call_received.clear()
# Send search decision
await human_input_queue.put(HumanDecision(action="search"))
# Wait for second tool call (after search cycle)
await asyncio.wait_for(tool_call_received.wait(), timeout=30)
# Send synthesize decision
await human_input_queue.put(HumanDecision(action="synthesize"))
# Run graph and decision sender concurrently
await asyncio.gather(run_graph(), send_decisions())
# Verify result
assert result is not None, (
f"No result. Events collected: {[e['type'] for e in events]}"
)
assert isinstance(result, dict)
assert "title" in result
# Verify human_decision tool calls were emitted
event_types = [e["type"] for e in events]
assert "TOOL_CALL_START" in event_types
assert "TOOL_CALL_END" in event_types
client.close()