Human-in-the-loop in ag-ui-example

This commit is contained in:
Yiorgis Gozadinos 2025-12-17 12:03:42 +02:00
parent 9818cac4eb
commit ff13be1210
No known key found for this signature in database
8 changed files with 443 additions and 92 deletions

View file

@ -12,6 +12,8 @@
- New `human_decide` graph node emits AG-UI tool calls (`TOOL_CALL_START/ARGS/END`) for frontend integration - New `human_decide` graph node emits AG-UI tool calls (`TOOL_CALL_START/ARGS/END`) for frontend integration
- New `emit_tool_call_start()`, `emit_tool_call_args()`, `emit_tool_call_end()` AG-UI event helpers - New `emit_tool_call_start()`, `emit_tool_call_args()`, `emit_tool_call_end()` AG-UI event helpers
- New `AGUIEmitter.emit()` method for direct event emission - New `AGUIEmitter.emit()` method for direct event emission
- **AG-UI Research Example**: Updated with interactive decision UI
- Decision panel with question editing (add/remove) at each decision point
- **HotpotQA Evaluation**: Added HotpotQA dataset adapter for multi-hop QA benchmarks - **HotpotQA Evaluation**: Added HotpotQA dataset adapter for multi-hop QA benchmarks
- Extracts unique documents from validation set context paragraphs - Extracts unique documents from validation set context paragraphs
- Uses MAP for retrieval evaluation (multiple supporting documents per question) - Uses MAP for retrieval evaluation (multiple supporting documents per question)

View file

@ -219,4 +219,8 @@ In interactive mode, you can:
- Execute searches and review collected answers - Execute searches and review collected answers
- Continue researching or synthesize when ready - Continue researching or synthesize when ready
For a web-based interactive experience, see the [AG-UI Research Example](https://github.com/ggozad/haiku.rag/tree/main/examples/ag-ui-research). For a web-based interactive experience with visual decision UI, see the [AG-UI Research Example](https://github.com/ggozad/haiku.rag/tree/main/examples/ag-ui-research). The web interface provides:
- Question editing panel to add/remove sub-questions at decision points
- Search and Generate Report buttons for controlling research flow
- Live state display showing answers, confidence, and progress

View file

@ -1,13 +1,13 @@
# Interactive Research Assistant # Interactive Research Assistant
Research assistant powered by [haiku.rag](https://ggozad.github.io/haiku.rag/), [Pydantic Graph](https://ai.pydantic.dev/graph/), and [AG-UI](https://docs.ag-ui.com/). Ask complex questions and watch the research process unfold in real-time. Research assistant powered by [haiku.rag](https://ggozad.github.io/haiku.rag/), [Pydantic Graph](https://ai.pydantic.dev/graph/), and [AG-UI](https://docs.ag-ui.com/). Ask complex questions and watch the research process unfold in real-time with human-in-the-loop control.
[Watch demo video](https://vimeo.com/1128874386) [Watch demo video](https://vimeo.com/1128874386)
## Features ## Features
- **Multi-iteration research graph**: Automated question decomposition and search - **Human-in-the-loop research**: Review and modify questions at decision points, then continue searching or generate report
- **Intelligent evaluation**: Confidence-based decision making with automatic iteration until sufficient information is gathered - **Multi-iteration research graph**: Automated question decomposition and parallel search
- **Live state synchronization**: Real-time delta updates of research progress via AG-UI protocol - **Live state synchronization**: Real-time delta updates of research progress via AG-UI protocol
- **Rich reporting**: Generates comprehensive research reports with findings, conclusions, and sources - **Rich reporting**: Generates comprehensive research reports with findings, conclusions, and sources
@ -25,9 +25,7 @@ Research assistant powered by [haiku.rag](https://ggozad.github.io/haiku.rag/),
**Option A: Create a new database** **Option A: Create a new database**
```bash ```bash
mkdir -p data haiku-rag init --db data/haiku_rag.lancedb
haiku-rag add "Your documents here" --db data/haiku_rag.lancedb
# Or add from files
haiku-rag add-src document.pdf --db data/haiku_rag.lancedb haiku-rag add-src document.pdf --db data/haiku_rag.lancedb
``` ```
@ -63,27 +61,29 @@ Research assistant powered by [haiku.rag](https://ggozad.github.io/haiku.rag/),
DB_PATH=/path/to/your/existing/haiku_rag.lancedb # If using an existing db. DB_PATH=/path/to/your/existing/haiku_rag.lancedb # If using an existing db.
``` ```
1. **Start the application** 4. **Start the application**
```bash ```bash
docker compose up --build docker compose up --build
``` ```
2. **Access the interface** 5. **Access the interface**
- Frontend: http://localhost:3000 - Frontend: http://localhost:3000
- Backend health: http://localhost:8000/health - Backend health: http://localhost:8000/health
## How It Works ## How It Works
1. **Ask a question**: Type your research question in the chat 1. **Ask a question**: Type your research question in the chat
2. **Plan phase**: The research graph automatically: 2. **Plan phase**: The research graph decomposes your question into targeted sub-questions
- Decomposes your question into targeted sub-questions 3. **Decision point**: Review the proposed questions in the right panel
- Gathers initial context about the topic - Add new questions using the input field
3. **Research iterations**: The graph autonomously: - Remove questions you don't need
- Searches the knowledge base for each sub-question in parallel - Click **Search** to execute searches for pending questions
- Assesses confidence in gathered information - Click **Generate Report** to skip to synthesis (when you have enough answers)
- Generates new follow-up questions if needed 4. **Research iterations**: After each search cycle, you return to a decision point where you can:
- Iterates until confidence threshold is met or max iterations reached - Review collected answers
4. **Synthesis**: Generates a comprehensive research report with: - Add follow-up questions based on findings
- Continue searching or generate the final report
5. **Synthesis**: Generates a comprehensive research report with:
- Executive summary - Executive summary
- Main findings with supporting evidence - Main findings with supporting evidence
- Conclusions and recommendations - Conclusions and recommendations
@ -99,32 +99,35 @@ This example demonstrates the **agent+graph** architecture pattern:
- Pydantic AI agent handles user conversations - Pydantic AI agent handles user conversations
- Decides when to invoke the research tool based on user intent - Decides when to invoke the research tool based on user intent
- Responds directly to greetings/casual chat without tools - Responds directly to greetings/casual chat without tools
- Formats research results for the user
2. **Research Graph** (haiku.rag): 2. **Interactive Research Graph** (haiku.rag):
- Multi-step research workflow invoked by the agent's tool - Multi-step research workflow invoked by the agent's tool
- Autonomous execution with plan → search → analyze → decide → synthesize flow - Pauses at decision points waiting for human input via async queue
- Emits AG-UI events for real-time progress tracking - Emits AG-UI events for real-time progress tracking
3. **Shared Event Stream**: 3. **Decision Endpoint** (`main.py`):
- `/v1/research/decide` receives human decisions from frontend
- Forwards decisions to the waiting graph via `HumanDecision` queue
- Supports actions: `search`, `synthesize`, `modify_questions`
4. **Shared Event Stream**:
- `AGUIEmitter` is shared between agent and graph - `AGUIEmitter` is shared between agent and graph
- Events from both flow through a single stream to the frontend - Events from both flow through a single stream to the frontend
- Custom streaming endpoint (`main.py`) uses anyio memory streams for proper async handling - `STATE_DELTA` events sync research state to frontend in real-time
### Components ### Components
- **Backend** (Python): - **Backend** (Python):
- Uses published `ghcr.io/ggozad/haiku.rag:latest` Docker image as base - Uses published `ghcr.io/ggozad/haiku.rag:latest` Docker image as base
- `agent.py`: Pydantic AI agent with `run_research` tool - `agent.py`: Pydantic AI agent with `run_research` tool, manages `ActiveResearch` registry
- `main.py`: Custom AG-UI streaming endpoint with anyio memory object streams - `main.py`: Custom AG-UI streaming endpoint, decision endpoint for human input
- Real-time event forwarding from emitter to SSE stream - Real-time event forwarding from emitter to SSE stream
- Filters out `ACTIVITY_SNAPSHOT` events (not yet supported by CopilotKit)
- **Frontend** (Next.js/React): - **Frontend** (Next.js/React):
- CopilotKit for AG-UI protocol integration - CopilotKit for AG-UI protocol integration
- Split-pane UI: chat on left, live research state on right - Split-pane UI: chat on left, live research state on right
- Decision UI: question editor with add/remove, search and generate report buttons
- Real-time state synchronization via Server-Sent Events (SSE) - Real-time state synchronization via Server-Sent Events (SSE)
- `StateDisplay` component with collapsible sections for questions and report
## Configuration ## Configuration

View file

@ -1,6 +1,7 @@
"""Research assistant agent with graph integration.""" """Research assistant agent with graph integration."""
from dataclasses import dataclass import asyncio
from dataclasses import dataclass, field
from pathlib import Path from pathlib import Path
from typing import TYPE_CHECKING from typing import TYPE_CHECKING
@ -11,7 +12,7 @@ from haiku.rag.config import load_yaml_config
from haiku.rag.config.models import AppConfig from haiku.rag.config.models import AppConfig
from haiku.rag.graph.research.dependencies import ResearchContext from haiku.rag.graph.research.dependencies import ResearchContext
from haiku.rag.graph.research.graph import build_research_graph from haiku.rag.graph.research.graph import build_research_graph
from haiku.rag.graph.research.state import ResearchDeps, ResearchState from haiku.rag.graph.research.state import HumanDecision, ResearchDeps, ResearchState
from haiku.rag.utils import get_model from haiku.rag.utils import get_model
if TYPE_CHECKING: if TYPE_CHECKING:
@ -27,6 +28,20 @@ Config = (
) )
@dataclass
class ActiveResearch:
"""Tracks state for active research awaiting human decision."""
queue: asyncio.Queue[HumanDecision]
sub_questions: list[str] = field(default_factory=list)
qa_responses: list[dict] = field(default_factory=list)
original_question: str = ""
# Global registry of active research by thread_id
_active_research: dict[str, ActiveResearch] = {}
@dataclass @dataclass
class AgentDeps: class AgentDeps:
"""Dependencies for research agent.""" """Dependencies for research agent."""
@ -34,6 +49,7 @@ class AgentDeps:
client: HaikuRAG client: HaikuRAG
agui_emitter: "AGUIEmitter[ResearchState, ResearchReport] | None" = None agui_emitter: "AGUIEmitter[ResearchState, ResearchReport] | None" = None
search_filter: str | None = None search_filter: str | None = None
thread_id: str | None = None
model = get_model(Config.research.model, Config) model = get_model(Config.research.model, Config)
@ -50,10 +66,10 @@ CRITICAL RULES:
4. NEVER answer substantive questions from your own knowledge - always use the tool 4. NEVER answer substantive questions from your own knowledge - always use the tool
How to decide: How to decide:
- "Hi" / "Hello" / "How are you?" Respond directly, NO tools - "Hi" / "Hello" / "How are you?" -> Respond directly, NO tools
- "What can you do?" Respond directly, NO tools - "What can you do?" -> Respond directly, NO tools
- "How does X work in the codebase?" Use run_research tool - "How does X work in the codebase?" -> Use run_research tool
- "Tell me about Y" Use run_research tool - "Tell me about Y" -> Use run_research tool
When you use run_research, the graph will decompose questions, search the knowledge base, When you use run_research, the graph will decompose questions, search the knowledge base,
and generate a comprehensive report. and generate a comprehensive report.
@ -70,23 +86,39 @@ async def run_research(ctx: RunContext[AgentDeps], question: str) -> str:
DO NOT use for greetings or casual conversation. DO NOT use for greetings or casual conversation.
""" """
if ctx.deps.agui_emitter: if ctx.deps.agui_emitter:
ctx.deps.agui_emitter.log(f"🔍 Starting research on: {question}") ctx.deps.agui_emitter.log(f"Starting research on: {question}")
graph = build_research_graph(Config) # Create queue for human decisions
queue: asyncio.Queue[HumanDecision] = asyncio.Queue()
# Build interactive graph
graph = build_research_graph(Config, interactive=True)
context = ResearchContext(original_question=question) context = ResearchContext(original_question=question)
state = ResearchState.from_config(context=context, config=Config) state = ResearchState.from_config(context=context, config=Config)
state.search_filter = ctx.deps.search_filter state.search_filter = ctx.deps.search_filter
# Register active research for decision endpoint to find
thread_id = ctx.deps.thread_id
if thread_id:
_active_research[thread_id] = ActiveResearch(
queue=queue,
sub_questions=[],
qa_responses=[],
original_question=question,
)
graph_deps = ResearchDeps( graph_deps = ResearchDeps(
client=ctx.deps.client, client=ctx.deps.client,
agui_emitter=ctx.deps.agui_emitter, agui_emitter=ctx.deps.agui_emitter,
human_input_queue=queue,
interactive=True,
) )
try: try:
result = await graph.run(state=state, deps=graph_deps) result = await graph.run(state=state, deps=graph_deps)
if ctx.deps.agui_emitter: if ctx.deps.agui_emitter:
ctx.deps.agui_emitter.log("✅ Research complete!") ctx.deps.agui_emitter.log("Research complete!")
return f"""Research completed successfully! return f"""Research completed successfully!
@ -108,5 +140,9 @@ The full research report with all citations has been provided to the user.
except Exception as e: except Exception as e:
if ctx.deps.agui_emitter: if ctx.deps.agui_emitter:
ctx.deps.agui_emitter.log(f"Research error: {str(e)}") ctx.deps.agui_emitter.log(f"Research error: {str(e)}")
return f"I encountered an error while researching: {str(e)}" return f"I encountered an error while researching: {str(e)}"
finally:
# Cleanup
if thread_id and thread_id in _active_research:
del _active_research[thread_id]

View file

@ -1,8 +1,9 @@
import json
import logging import logging
import os import os
from pathlib import Path from pathlib import Path
from agent import AgentDeps, agent from agent import AgentDeps, _active_research, agent
from anyio import create_memory_object_stream, create_task_group from anyio import create_memory_object_stream, create_task_group
from anyio.streams.memory import MemoryObjectSendStream from anyio.streams.memory import MemoryObjectSendStream
from starlette.applications import Starlette from starlette.applications import Starlette
@ -19,7 +20,7 @@ from haiku.rag.graph.agui.emitter import AGUIEmitter
from haiku.rag.graph.agui.server import RunAgentInput, format_sse_event from haiku.rag.graph.agui.server import RunAgentInput, format_sse_event
from haiku.rag.graph.research.dependencies import ResearchContext from haiku.rag.graph.research.dependencies import ResearchContext
from haiku.rag.graph.research.models import ResearchReport from haiku.rag.graph.research.models import ResearchReport
from haiku.rag.graph.research.state import ResearchState from haiku.rag.graph.research.state import HumanDecision, ResearchState
logging.basicConfig( logging.basicConfig(
level=logging.INFO, format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" level=logging.INFO, format="%(asctime)s - %(name)s - %(levelname)s - %(message)s"
@ -79,11 +80,11 @@ async def stream_research_agent(request: Request) -> StreamingResponse:
"""Execute agent and forward emitter events to memory stream.""" """Execute agent and forward emitter events to memory stream."""
async with send_stream: async with send_stream:
try: try:
# Create shared emitter # Create shared emitter (use_deltas=True for CopilotKit compatibility)
emitter: AGUIEmitter[ResearchState, ResearchReport] = AGUIEmitter( emitter: AGUIEmitter[ResearchState, ResearchReport] = AGUIEmitter(
thread_id=input_data.thread_id, thread_id=input_data.thread_id,
run_id=input_data.run_id, run_id=input_data.run_id,
use_deltas=False, use_deltas=True,
) )
# Get client # Get client
@ -99,11 +100,14 @@ async def stream_research_agent(request: Request) -> StreamingResponse:
ids_str = ", ".join(f"'{id}'" for id in document_ids) ids_str = ", ".join(f"'{id}'" for id in document_ids)
search_filter = f"id IN ({ids_str})" search_filter = f"id IN ({ids_str})"
thread_id = input_data.thread_id
# Create agent dependencies with shared emitter # Create agent dependencies with shared emitter
agent_deps = AgentDeps( agent_deps = AgentDeps(
client=client, client=client,
agui_emitter=emitter, agui_emitter=emitter,
search_filter=search_filter, search_filter=search_filter,
thread_id=thread_id,
) )
# Start run with empty initial state # Start run with empty initial state
@ -148,6 +152,37 @@ async def stream_research_agent(request: Request) -> StreamingResponse:
await send_stream.send(format_sse_event(delta_event)) await send_stream.send(format_sse_event(delta_event))
continue continue
# When human_decision tool starts, set awaiting_decision flag
if event_type == "TOOL_CALL_START":
tool_name = event.get("toolCallName")
if tool_name == "human_decision":
delta_event = {
"type": "STATE_DELTA",
"delta": [
{
"op": "add",
"path": "/awaiting_decision",
"value": True,
}
],
}
await send_stream.send(format_sse_event(delta_event))
# Sync state to ActiveResearch when human_decision tool call
if event_type == "TOOL_CALL_ARGS" and thread_id:
delta = event.get("delta", "{}")
args = (
json.loads(delta) if isinstance(delta, str) else delta
)
active = _active_research.get(thread_id)
if active:
active.sub_questions = list(
args.get("sub_questions", [])
)
active.qa_responses = list(args.get("qa_responses", []))
if "original_question" in args:
active.original_question = args["original_question"]
await send_stream.send(format_sse_event(event)) await send_stream.send(format_sse_event(event))
# Run agent and event forwarding concurrently # Run agent and event forwarding concurrently
@ -248,10 +283,36 @@ async def visualize_chunk(request: Request) -> JSONResponse:
) )
async def research_decide(request: Request) -> JSONResponse:
"""Endpoint to receive human decisions for active research."""
body = await request.json()
action = body.get("action", "search")
questions = body.get("questions", [])
# Get first active research (single-user example)
active = next(iter(_active_research.values()), None)
if not active:
return JSONResponse({"error": "No active research found"}, status_code=404)
# When "search" action is sent with questions, use "modify_questions" to update them
effective_action = (
"modify_questions" if action == "search" and questions else action
)
decision = HumanDecision(
action=effective_action,
questions=questions or None,
)
await active.queue.put(decision)
return JSONResponse({"status": "ok", "action": effective_action})
# Create Starlette app # Create Starlette app
app = Starlette( app = Starlette(
routes=[ routes=[
Route("/v1/research/stream", stream_research_agent, methods=["POST"]), Route("/v1/research/stream", stream_research_agent, methods=["POST"]),
Route("/v1/research/decide", research_decide, methods=["POST"]),
Route("/api/documents", list_documents, methods=["GET"]), Route("/api/documents", list_documents, methods=["GET"]),
Route("/api/visualize/{chunk_id}", visualize_chunk, methods=["GET"]), Route("/api/visualize/{chunk_id}", visualize_chunk, methods=["GET"]),
Route("/health", health_check, methods=["GET"]), Route("/health", health_check, methods=["GET"]),

View file

@ -3,6 +3,7 @@
import { CopilotKit, useCoAgent } from "@copilotkit/react-core"; import { CopilotKit, useCoAgent } from "@copilotkit/react-core";
import { CopilotChat } from "@copilotkit/react-ui"; import { CopilotChat } from "@copilotkit/react-ui";
import "@copilotkit/react-ui/styles.css"; import "@copilotkit/react-ui/styles.css";
import { useState, useEffect } from "react";
import DocumentSelector from "./DocumentSelector"; import DocumentSelector from "./DocumentSelector";
import StateDisplay from "./StateDisplay"; import StateDisplay from "./StateDisplay";
@ -58,6 +59,7 @@ interface ResearchState {
current_activity?: string; current_activity?: string;
current_activity_message?: string; current_activity_message?: string;
documentFilter?: string[]; documentFilter?: string[];
awaiting_decision?: boolean;
} }
function AgentContent() { function AgentContent() {
@ -75,13 +77,61 @@ function AgentContent() {
max_concurrency: 1, max_concurrency: 1,
last_eval: null, last_eval: null,
documentFilter: [], documentFilter: [],
awaiting_decision: false,
}, },
}); });
const [editableQuestions, setEditableQuestions] = useState<string[]>([]);
const [newQuestion, setNewQuestion] = useState("");
const [submitting, setSubmitting] = useState(false);
// Sync editable questions when state changes
useEffect(() => {
if (state.awaiting_decision && state.context.sub_questions) {
setEditableQuestions([...state.context.sub_questions]);
}
}, [state.awaiting_decision, state.context.sub_questions]);
const handleDocumentFilterChange = (ids: string[]) => { const handleDocumentFilterChange = (ids: string[]) => {
setState({ ...state, documentFilter: ids }); setState({ ...state, documentFilter: ids });
}; };
const handleRemoveQuestion = (index: number) => {
setEditableQuestions(editableQuestions.filter((_, i) => i !== index));
};
const handleAddQuestion = () => {
if (newQuestion.trim()) {
setEditableQuestions([...editableQuestions, newQuestion.trim()]);
setNewQuestion("");
}
};
const handleDecision = async (action: "search" | "synthesize") => {
setSubmitting(true);
try {
const response = await fetch(
`${process.env.NEXT_PUBLIC_API_URL || "http://localhost:8000"}/v1/research/decide`,
{
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
thread_id: state.context.original_question, // Use as identifier
action,
questions: editableQuestions,
}),
}
);
if (response.ok) {
setState({ ...state, awaiting_decision: false });
}
} catch (error) {
console.error("Failed to send decision:", error);
} finally {
setSubmitting(false);
}
};
return ( return (
<> <>
<style>{` <style>{`
@ -141,13 +191,162 @@ function AgentContent() {
</p> </p>
</header> </header>
<div style={{ marginBottom: "1rem" }}> {/* Document filter - hidden when research is running */}
<DocumentSelector {!running && (
selected={state.documentFilter || []} <div style={{ marginBottom: "1rem" }}>
onChange={handleDocumentFilterChange} <DocumentSelector
disabled={running} selected={state.documentFilter || []}
/> onChange={handleDocumentFilterChange}
</div> />
</div>
)}
{/* Decision UI when awaiting human input - hidden when report exists, submitting, or not running */}
{state.awaiting_decision && !submitting && !state.result && running && (
<div
style={{
marginBottom: "1rem",
background: "#f0f9ff",
border: "2px solid #0ea5e9",
borderRadius: "8px",
padding: "1rem",
}}
>
<div
style={{
fontWeight: "bold",
color: "#0369a1",
marginBottom: "0.75rem",
fontSize: "1rem",
}}
>
Research Decision Point
</div>
<div style={{ fontSize: "0.85rem", color: "#64748b", marginBottom: "0.75rem" }}>
{state.context.qa_responses?.length || 0} answers collected | Iteration {state.iterations || 0}
</div>
{/* Questions list */}
<div style={{ marginBottom: "0.75rem" }}>
<div style={{ fontSize: "0.8rem", color: "#475569", marginBottom: "0.5rem" }}>
Pending Questions ({editableQuestions.length}):
</div>
{editableQuestions.map((q, idx) => (
<div
key={idx}
style={{
display: "flex",
alignItems: "center",
gap: "0.5rem",
padding: "0.375rem 0.5rem",
background: "white",
borderRadius: "4px",
marginBottom: "0.25rem",
fontSize: "0.85rem",
}}
>
<span style={{ flex: 1 }}>{q}</span>
<button
type="button"
onClick={() => handleRemoveQuestion(idx)}
disabled={submitting}
style={{
background: "#ef4444",
color: "white",
border: "none",
borderRadius: "4px",
padding: "0.25rem 0.5rem",
cursor: submitting ? "not-allowed" : "pointer",
fontSize: "0.75rem",
}}
>
Remove
</button>
</div>
))}
</div>
{/* Add question input */}
<div style={{ display: "flex", gap: "0.5rem", marginBottom: "1rem" }}>
<input
type="text"
value={newQuestion}
onChange={(e) => setNewQuestion(e.target.value)}
placeholder="Add a new question..."
disabled={submitting}
style={{
flex: 1,
padding: "0.5rem",
border: "1px solid #cbd5e1",
borderRadius: "4px",
fontSize: "0.85rem",
}}
onKeyDown={(e) => {
if (e.key === "Enter") {
handleAddQuestion();
}
}}
/>
<button
type="button"
onClick={handleAddQuestion}
disabled={submitting}
style={{
background: "#22c55e",
color: "white",
border: "none",
borderRadius: "4px",
padding: "0.5rem 1rem",
cursor: submitting ? "not-allowed" : "pointer",
fontSize: "0.85rem",
}}
>
Add
</button>
</div>
{/* Action buttons */}
<div style={{ display: "flex", gap: "0.5rem" }}>
<button
type="button"
onClick={() => handleDecision("search")}
disabled={editableQuestions.length === 0 || submitting}
style={{
flex: 1,
background: editableQuestions.length === 0 || submitting ? "#94a3b8" : "#0ea5e9",
color: "white",
border: "none",
borderRadius: "4px",
padding: "0.75rem",
cursor: editableQuestions.length === 0 || submitting ? "not-allowed" : "pointer",
fontWeight: "bold",
fontSize: "0.9rem",
}}
>
{submitting ? "Submitting..." : `Search (${editableQuestions.length})`}
</button>
<button
type="button"
onClick={() => handleDecision("synthesize")}
disabled={submitting || (state.context.qa_responses?.length || 0) === 0}
style={{
flex: 1,
background: submitting || (state.context.qa_responses?.length || 0) === 0 ? "#94a3b8" : "#8b5cf6",
color: "white",
border: "none",
borderRadius: "4px",
padding: "0.75rem",
cursor: submitting || (state.context.qa_responses?.length || 0) === 0 ? "not-allowed" : "pointer",
fontWeight: "bold",
fontSize: "0.9rem",
}}
>
Generate Report
</button>
</div>
</div>
)}
<StateDisplay state={state} /> <StateDisplay state={state} />
</div> </div>

View file

@ -192,8 +192,8 @@ export default function StateDisplay({ state }: StateDisplayProps) {
</div> </div>
)} )}
{/* Research Progress - only show when research has started */} {/* Research Progress - only show when research is in progress (not when complete) */}
{(state.iterations > 0 || (state.current_activity && !state.result)) && ( {(state.iterations > 0 || state.current_activity) && !state.result && (
<div <div
style={{ style={{
background: "white", background: "white",
@ -202,8 +202,8 @@ export default function StateDisplay({ state }: StateDisplayProps) {
boxShadow: "0 1px 3px rgba(0,0,0,0.1)", boxShadow: "0 1px 3px rgba(0,0,0,0.1)",
}} }}
> >
{/* Current Activity - hide when complete */} {/* Current Activity */}
{state.current_activity && !state.result && ( {state.current_activity && (
<div <div
style={{ style={{
padding: "0.75rem", padding: "0.75rem",
@ -363,9 +363,8 @@ export default function StateDisplay({ state }: StateDisplayProps) {
</div> </div>
)} )}
{/* Sub-Questions and QA Responses */} {/* Answers */}
{(state.context.sub_questions.length > 0 || {state.context.qa_responses.length > 0 && (
state.context.qa_responses.length > 0) && (
<div <div
style={{ style={{
background: "white", background: "white",
@ -392,10 +391,7 @@ export default function StateDisplay({ state }: StateDisplayProps) {
color: "#2d3748", color: "#2d3748",
}} }}
> >
<span> <span>Answers ({state.context.qa_responses.length})</span>
Sub-Questions ({state.context.sub_questions.length}) Answers (
{state.context.qa_responses.length})
</span>
<span>{expandedSections.questions ? "▼" : "▶"}</span> <span>{expandedSections.questions ? "▼" : "▶"}</span>
</button> </button>
{expandedSections.questions && ( {expandedSections.questions && (
@ -408,38 +404,6 @@ export default function StateDisplay({ state }: StateDisplayProps) {
borderRadius: "0 0 4px 4px", borderRadius: "0 0 4px 4px",
}} }}
> >
{/* Show pending sub_questions */}
{state.context.sub_questions.map((question, idx) => (
<div
key={`pending-${idx}`}
style={{
marginBottom: "0.5rem",
background: "white",
borderRadius: "4px",
border: "1px solid #e2e8f0",
padding: "0.75rem",
display: "flex",
gap: "0.75rem",
alignItems: "center",
}}
>
<div
style={{
fontSize: "1.25rem",
color: "#a0aec0",
flexShrink: 0,
}}
>
</div>
<div
style={{ flex: 1, fontSize: "0.875rem", color: "#4a5568" }}
>
<Markdown content={question} />
</div>
</div>
))}
{/* Show all qa_responses (each has query + answer) */} {/* Show all qa_responses (each has query + answer) */}
{state.context.qa_responses.map((qaResponse, idx) => { {state.context.qa_responses.map((qaResponse, idx) => {
const questionId = `q-${idx}`; const questionId = `q-${idx}`;

View file

@ -1,3 +1,5 @@
import asyncio
import pytest import pytest
from pydantic_ai.models.test import TestModel from pydantic_ai.models.test import TestModel
@ -5,7 +7,7 @@ from haiku.rag.client import HaikuRAG
from haiku.rag.graph.agui.stream import stream_graph from haiku.rag.graph.agui.stream import stream_graph
from haiku.rag.graph.research.dependencies import ResearchContext from haiku.rag.graph.research.dependencies import ResearchContext
from haiku.rag.graph.research.graph import build_research_graph from haiku.rag.graph.research.graph import build_research_graph
from haiku.rag.graph.research.state import ResearchDeps, ResearchState from haiku.rag.graph.research.state import HumanDecision, ResearchDeps, ResearchState
@pytest.mark.asyncio @pytest.mark.asyncio
@ -61,3 +63,83 @@ async def test_graph_end_to_end_with_test_model(monkeypatch, temp_db_path):
assert "STEP_STARTED" in event_types assert "STEP_STARTED" in event_types
client.close() client.close()
@pytest.mark.asyncio
async def test_interactive_graph_with_human_decision(monkeypatch, temp_db_path):
"""Test interactive research graph pauses and resumes with human decisions."""
# Mock get_model to return TestModel
def test_model_factory(_provider, _model, _config=None):
return TestModel()
monkeypatch.setattr("haiku.rag.utils.get_model", test_model_factory)
monkeypatch.setattr("haiku.rag.graph.research.graph.get_model", test_model_factory)
# Build interactive graph
graph = build_research_graph(interactive=True)
state = ResearchState(
context=ResearchContext(original_question="What is haiku.rag?"),
max_iterations=1,
confidence_threshold=0.5,
max_concurrency=2,
)
# Create human input queue
human_input_queue: asyncio.Queue[HumanDecision] = asyncio.Queue()
client = HaikuRAG(temp_db_path, create=True)
deps = ResearchDeps(
client=client,
human_input_queue=human_input_queue,
interactive=True,
)
events = []
tool_call_received = asyncio.Event()
result = None
async def run_graph():
nonlocal result
async for event in stream_graph(graph, state, deps):
events.append(event)
if event["type"] == "TOOL_CALL_START":
tool_name = event.get("toolCallName")
if tool_name == "human_decision":
tool_call_received.set()
elif event["type"] == "RUN_FINISHED":
result = event["result"]
elif event["type"] == "RUN_ERROR":
pytest.fail(f"Graph execution failed: {event['message']}")
async def send_decisions():
# Wait for first tool call (after planning)
await asyncio.wait_for(tool_call_received.wait(), timeout=30)
tool_call_received.clear()
# Send search decision
await human_input_queue.put(HumanDecision(action="search"))
# Wait for second tool call (after search cycle)
await asyncio.wait_for(tool_call_received.wait(), timeout=30)
# Send synthesize decision
await human_input_queue.put(HumanDecision(action="synthesize"))
# Run graph and decision sender concurrently
await asyncio.gather(run_graph(), send_decisions())
# Verify result
assert result is not None, (
f"No result. Events collected: {[e['type'] for e in events]}"
)
assert isinstance(result, dict)
assert "title" in result
# Verify human_decision tool calls were emitted
event_types = [e["type"] for e in events]
assert "TOOL_CALL_START" in event_types
assert "TOOL_CALL_END" in event_types
client.close()