From 567e50be1f6fe6c761630656d7ea523892c708a2 Mon Sep 17 00:00:00 2001 From: Yiorgis Gozadinos Date: Tue, 13 Jan 2026 11:59:21 +0200 Subject: [PATCH] Update documentation --- docs/agents.md | 66 ++++++++++++- docs/apps.md | 181 +++++++++++++++++++++++++++++++++++ docs/architecture.md | 219 +++++++++++++++++++++++++++++++++++++++++++ docs/cli.md | 18 ++++ docs/index.md | 6 +- docs/inspector.md | 77 --------------- docs/tutorial.md | 1 + mkdocs.yml | 3 +- 8 files changed, 491 insertions(+), 80 deletions(-) create mode 100644 docs/apps.md create mode 100644 docs/architecture.md delete mode 100644 docs/inspector.md diff --git a/docs/agents.md b/docs/agents.md index ca082e90..0785f226 100644 --- a/docs/agents.md +++ b/docs/agents.md @@ -1,8 +1,9 @@ # Agents -Two agentic flows are provided by haiku.rag: +Three agentic flows are provided by haiku.rag: - **Simple QA Agent** — a focused question answering agent +- **Chat Agent** — multi-turn conversational RAG with session memory - **Research Graph** — a multi-step research workflow with question decomposition See [QA and Research Configuration](configuration/qa-research.md) for configuring model, iterations, concurrency, and other settings. @@ -47,6 +48,69 @@ async with HaikuRAG(path_to_db) as client: print(answer) ``` +## Chat Agent + +The chat agent enables multi-turn conversational RAG. It maintains session state including Q/A history and uses that context to improve follow-up answers. + +Key features: + +- **Session memory**: Previous Q/A pairs are used as context for follow-up questions +- **Query expansion**: SearchAgent generates multiple query variations for better recall +- **Document filtering**: Natural language document filtering ("search in document X about...") +- **Confidence filtering**: Low-confidence answers are flagged + +### Tools + +The chat agent uses three tools: + +- `search` — Hybrid search with optional document filter +- `ask` — Answer questions using the conversational research graph +- `get_document` — Retrieve a specific document by title or URI + +### CLI Usage + +```bash +haiku-rag chat +haiku-rag chat --db /path/to/database.lancedb +``` + +See [Applications](apps.md#chat-tui) for the full TUI interface guide. + +### Python Usage + +```python +from haiku.rag.client import HaikuRAG +from haiku.rag.agents.chat import create_chat_agent, ChatDeps, ChatSessionState + +async with HaikuRAG(path_to_db) as client: + # Create agent and session + agent = create_chat_agent(config) + session = ChatSessionState() + deps = ChatDeps(client=client, config=config, session_state=session) + + # First question + result = await agent.run("What is haiku.rag?", deps=deps) + print(result.output) + + # Follow-up (uses session context) + result = await agent.run("How does it handle PDFs?", deps=deps) + print(result.output) +``` + +### Session State + +The `ChatSessionState` maintains: + +- `session_id` — Unique identifier for the session +- `qa_history` — List of previous Q/A pairs (FIFO, max 50) +- `embedding_cache` — Cached embeddings for semantic ranking + +Q/A history is used to: + +1. Provide context for follow-up questions +2. Avoid repeating previous answers +3. Enable semantic ranking of relevant past answers + ## Research Graph The research workflow is implemented as a typed pydantic-graph. It plans, searches (in parallel batches), evaluates, and synthesizes into a final report. diff --git a/docs/apps.md b/docs/apps.md new file mode 100644 index 00000000..c0688691 --- /dev/null +++ b/docs/apps.md @@ -0,0 +1,181 @@ +# Applications + +Three interactive applications for working with your knowledge base. + +## Chat TUI + +Conversational RAG from the terminal with streaming responses and session memory. + +### Usage + +```bash +haiku-rag chat +haiku-rag chat --db /path/to/database.lancedb +``` + +### Interface + +The chat interface provides: + +- Streaming responses with real-time tool execution indicators +- Expandable citations showing source document, pages, and headings +- Session memory for context-aware follow-up questions +- Visual grounding to inspect chunk source locations + +### Keyboard Shortcuts + +| Key | Action | +|-----|--------| +| `Ctrl+L` | Clear chat history and reset session | +| `Ctrl+G` | Show visual grounding for selected citation | +| `Ctrl+I` | Show database info (document/chunk counts) | +| `Escape` | Focus input field or cancel processing | + +### Session Management + +- Conversation history is maintained in memory for the session +- Previous Q/A pairs are used as context for follow-up questions +- Citations are tracked per response and can be inspected +- Clearing chat (`Ctrl+L`) resets the session state + +## Web Application + +Browser-based conversational RAG with a CopilotKit frontend. + +### Quick Start + +```bash +cd app +docker compose -f docker-compose.dev.yml up -d --build +``` + +- Frontend: http://localhost:3000 +- Backend: http://localhost:8001 + +### Architecture + +- **Backend**: Starlette server with pydantic-ai `AGUIAdapter` +- **Frontend**: Next.js with CopilotKit +- **Protocol**: AG-UI for streaming chat + +### Configuration + +Create a `.env` file in the `app/` directory: + +```bash +# API Keys (at least one required) +ANTHROPIC_API_KEY=your-anthropic-key +OPENAI_API_KEY=your-openai-key + +# Database path +DB_PATH=/path/to/your/haiku.rag.lancedb + +# Optional: Ollama base URL (if using local models) +OLLAMA_BASE_URL=http://localhost:11434 + +# Optional: Logfire for observability +LOGFIRE_TOKEN=your-logfire-token +``` + +For full configuration, mount a `haiku.rag.yaml` file: + +```yaml +# app/haiku.rag.yaml +qa: + model: + provider: anthropic + name: claude-sonnet-4-20250514 +``` + +### API Endpoints + +| Endpoint | Method | Description | +|----------|--------|-------------| +| `/v1/chat/stream` | POST | AG-UI chat streaming | +| `/api/documents` | GET | List all documents | +| `/api/info` | GET | Database statistics | +| `/api/visualize/{chunk_id}` | GET | Visual grounding images (base64) | +| `/health` | GET | Health check | + +### Development + +**Hot reload**: The backend reloads automatically on file changes. For frontend changes: + +```bash +docker compose -f docker-compose.dev.yml up -d --build frontend +``` + +**Logfire debugging**: If `LOGFIRE_TOKEN` is set, LLM calls are traced and available in the Logfire dashboard. + +## Inspector + +TUI for browsing documents, chunks, and search results. + +### Installation + +```bash +# For haiku.rag-slim +pip install 'haiku.rag-slim[inspector]' + +# Already included in haiku.rag +pip install haiku.rag +``` + +### Usage + +```bash +haiku-rag inspect +haiku-rag inspect --db /path/to/database.lancedb +``` + +### Interface + +Three panels display your data: + +- **Documents** (left) - All documents in the database +- **Chunks** (top right) - Chunks for the selected document +- **Detail View** (bottom right) - Full content and metadata + +### Navigation + +| Key | Action | +|-----|--------| +| `Tab` | Cycle between panels | +| `↑` / `↓` | Navigate lists | +| `/` | Open search modal | +| `c` | Context expansion modal (when viewing a chunk) | +| `v` | Visual grounding modal (when viewing a chunk) | +| `q` | Quit | + +**Mouse**: Click to select, scroll to view content. + +### Search + +Press `/` to open the full-screen search modal: + +- Enter your query and press `Enter` to search +- **Left panel**: Search results with relevance scores `[0.95] content preview` +- **Right panel**: Full chunk content and metadata +- Use `↑` / `↓` to navigate results +- Press `Enter` on a result to navigate to that document/chunk +- Press `Esc` to close search + +Search uses hybrid (vector + full-text) search across all chunks. + +### Context Expansion + +Press `c` while viewing a chunk to see the expanded context that would be provided to the QA agent: + +- Type-aware expansion: tables, code blocks, and lists expand to their complete structures +- Text content expands based on `search.context_radius` setting +- Includes metadata like source document, content type, and relevance score + +### Visual Grounding + +Press `v` while viewing a chunk to see page images with the chunk's location highlighted: + +- Use `←` / `→` arrow keys to navigate between pages +- Requires documents processed with Docling that include page images + +!!! note + Visual grounding requires documents with a stored DoclingDocument that includes page images. Text-only documents won't have visual grounding available. diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 00000000..62a40bff --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,219 @@ +# Architecture + +High-level overview of haiku.rag components and data flow. + +## System Overview + +```mermaid +flowchart TB + subgraph Sources["Document Sources"] + Files[Files] + URLs[URLs] + Text[Text] + end + + subgraph Processing["Processing Pipeline"] + Converter[Converter] + Chunker[Chunker] + Embedder[Embedder] + end + + subgraph Storage["Storage Layer"] + LanceDB[(LanceDB)] + end + + subgraph Agents["Agent Layer"] + QA[QA Agent] + Chat[Chat Agent] + Research[Research Graph] + end + + subgraph Apps["Applications"] + CLI[CLI] + ChatTUI[Chat TUI] + WebApp[Web App] + Inspector[Inspector] + MCP[MCP Server] + end + + Sources --> Converter + Converter --> Chunker + Chunker --> Embedder + Embedder --> LanceDB + + LanceDB --> Agents + Agents --> Apps +``` + +## Core Components + +### Storage Layer + +LanceDB provides vector storage with full-text search capabilities: + +- **DocumentRecord** - Document metadata and full content +- **ChunkRecord** - Text chunks with embeddings and structural metadata +- **SettingsRecord** - Database configuration and version info + +Repositories handle CRUD operations: + +- `DocumentRepository` - Create, read, update, delete documents +- `ChunkRepository` - Chunk management and hybrid search +- `SettingsRepository` - Configuration persistence + +### Processing Pipeline + +```mermaid +flowchart LR + Source[Source] --> Converter + Converter --> DoclingDoc[DoclingDocument] + DoclingDoc --> Chunker + Chunker --> Chunks[Chunks] + Chunks --> Embedder + Embedder --> Vectors[Vectors] + Vectors --> DB[(LanceDB)] +``` + +**Converters** transform sources into DoclingDocuments: + +- `docling-local` - Local Docling processing +- `docling-serve` - Remote processing via docling-serve + +**Chunkers** split documents into semantic chunks: + +- Preserves document structure (tables, lists, code blocks) +- Maintains provenance (page numbers, headings) +- Configurable chunk size + +**Embedders** generate vector representations: + +| Provider | Models | +|----------|--------| +| Ollama | nomic-embed-text, mxbai-embed-large | +| OpenAI | text-embedding-3-small, text-embedding-3-large | +| VoyageAI | voyage-3, voyage-code-3 | +| vLLM | Any compatible model | +| LM Studio | Any compatible model | + +### Agent Layer + +Three agent types for different use cases: + +```mermaid +flowchart TB + subgraph QA["QA Agent"] + Q1[Question] --> S1[Search] + S1 --> A1[Answer] + end + + subgraph Chat["Chat Agent"] + Q2[Question] --> Expand[Query Expansion] + Expand --> S2[Search/Ask] + S2 --> A2[Answer] + A2 --> History[Session History] + History -.-> Q2 + end + + subgraph Research["Research Graph"] + Q3[Question] --> Plan[Plan] + Plan --> Batch[Get Batch] + Batch --> SearchN[Search × N] + SearchN --> Evaluate[Evaluate] + Evaluate -->|Continue| Batch + Evaluate -->|Done| Synthesize[Synthesize] + end +``` + +**QA Agent** - Single-turn question answering: + +- Searches for relevant chunks +- Expands context around results +- Generates answer with optional citations + +**Chat Agent** - Multi-turn conversational RAG: + +- Maintains session history +- Uses previous Q/A pairs as context +- Query expansion for better recall +- Natural language document filtering + +**Research Graph** - Multi-step research workflow: + +- Decomposes questions into sub-questions +- Parallel search execution +- Iterative refinement based on confidence +- Synthesizes structured research report + +### Applications + +| Application | Interface | Use Case | +|-------------|-----------|----------| +| CLI | Command line | Scripts, one-off queries, batch processing | +| Chat TUI | Terminal | Interactive conversations | +| Web App | Browser | Team collaboration, visual interface | +| Inspector | Terminal | Database exploration, debugging | +| MCP Server | Protocol | AI assistant integration | + +## Data Flow + +### Document Ingestion + +```mermaid +sequenceDiagram + participant User + participant CLI + participant Converter + participant Chunker + participant Embedder + participant DB as LanceDB + + User->>CLI: add-src document.pdf + CLI->>Converter: Convert to DoclingDocument + Converter-->>CLI: DoclingDocument + CLI->>Chunker: Split into chunks + Chunker-->>CLI: Chunks with metadata + CLI->>Embedder: Generate embeddings + Embedder-->>CLI: Vectors + CLI->>DB: Store document + chunks + DB-->>User: Document ID +``` + +### Search and QA + +```mermaid +sequenceDiagram + participant User + participant Agent + participant Embedder + participant DB as LanceDB + participant LLM + + User->>Agent: Ask question + Agent->>Embedder: Embed query + Embedder-->>Agent: Query vector + Agent->>DB: Hybrid search + DB-->>Agent: Relevant chunks + Agent->>Agent: Expand context + Agent->>LLM: Generate answer + LLM-->>Agent: Answer + citations + Agent-->>User: Response +``` + +## Configuration + +Configuration flows through the system: + +``` +CLI args → Environment variables → haiku.rag.yaml → Defaults +``` + +Key configuration areas: + +- **Storage** - Database path, vacuum settings +- **Embeddings** - Provider, model, dimensions +- **Processing** - Chunk size, converter, chunker +- **Search** - Limits, context expansion +- **QA/Research** - Model, iterations, concurrency +- **Providers** - Ollama, vLLM, docling-serve URLs + +See [Configuration](configuration/index.md) for details. diff --git a/docs/cli.md b/docs/cli.md index 13a6d8b3..445d1b69 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -161,6 +161,24 @@ Flags: - `--deep`: Decompose the question into sub-questions answered in parallel before synthesizing a final answer - `--filter`: Restrict searches to documents matching the filter (see [Filtering Search Results](python.md#filtering-search-results)) +## Chat + +Launch an interactive chat session for multi-turn conversations: + +```bash +haiku-rag chat +haiku-rag chat --db /path/to/database.lancedb +``` + +The chat interface provides: + +- Streaming responses with real-time tool execution +- Expandable citations with source metadata +- Session memory for context-aware follow-up questions +- Visual grounding to inspect chunk source locations + +See [Applications](apps.md#chat-tui) for keyboard shortcuts and features. + ## Research Run the multi-step research graph: diff --git a/docs/index.md b/docs/index.md index 542de5ea..fbfcb0b9 100644 --- a/docs/index.md +++ b/docs/index.md @@ -8,6 +8,7 @@ Agentic RAG built on [LanceDB](https://lancedb.com/), [Pydantic AI](https://ai.p - **Reranking** — MxBAI, Cohere, Zero Entropy, or vLLM - **Question answering** — QA agents with citations (page numbers, section headings) - **Research agents** — Multi-agent workflows via pydantic-graph: plan, search, evaluate, synthesize +- **Conversational RAG** — Chat TUI and web application for multi-turn conversations with session memory - **Document structure** — Stores full [DoclingDocument](https://docling-project.github.io/docling/concepts/docling_document/), enabling structure-aware context expansion and visual grounding - **Multiple providers** — Embeddings: Ollama, OpenAI, VoyageAI, LM Studio, vLLM. QA/Research: any model supported by Pydantic AI - **Local-first** — Embedded LanceDB, no servers required. Also supports S3, GCS, Azure, and LanceDB Cloud @@ -48,17 +49,20 @@ haiku-rag add "Your document content" --meta author=alice haiku-rag add-src /path/to/document.pdf --title "Q3 Financial Report" --meta source=manual haiku-rag search "query" haiku-rag ask "Who is the author of haiku.rag?" +haiku-rag chat # Interactive conversation mode ``` ## Documentation - [Getting started](tutorial.md) - Tutorial - [Installation](installation.md) - Install haiku.rag with different providers +- [Architecture](architecture.md) - System overview and data flow - [Configuration](configuration/index.md) - Environment variables and settings - [CLI](cli.md) - Command line interface usage - [Python](python.md) - Python API reference - [Custom Pipelines](custom-pipelines.md) - Build custom processing workflows -- [Agents](agents.md) - QA agent and multi-agent research +- [Agents](agents.md) - QA, chat, and research agents +- [Applications](apps.md) - Chat TUI, web app, and inspector - [Server](server.md) - File monitoring and server mode - [MCP](mcp.md) - Model Context Protocol integration - [Remote processing](remote-processing.md) - Remote document processing with docling-serve diff --git a/docs/inspector.md b/docs/inspector.md deleted file mode 100644 index a5250821..00000000 --- a/docs/inspector.md +++ /dev/null @@ -1,77 +0,0 @@ -# Database Inspector - -Interactive TUI for browsing your LanceDB database. - -## Installation - -```bash -# For haiku.rag-slim -pip install 'haiku.rag-slim[inspector]' - -# Already included in haiku.rag -pip install haiku.rag -``` - -## Usage - -```bash -haiku-rag inspect -haiku-rag inspect --db /path/to/database.lancedb -``` - -## Interface - -![Inspector with search](img/inspector-search.svg) - -Three panels display your data: - -- **Documents** (left) - All documents in the database -- **Chunks** (top right) - Chunks for the selected document -- **Detail View** (bottom right) - Full content and metadata - -## Navigation - -**Keyboard:** - -- `Tab` - Cycle between panels -- `↑` / `↓` - Navigate lists -- `/` - Open search modal -- `c` - Open context expansion modal (when viewing a chunk) -- `v` - Open visual grounding modal (when viewing a chunk) -- `q` - Quit - -**Mouse:** Click to select, scroll to view content - -## Search - -Press `/` to open the full-screen search modal: - -- Enter your query and press `Enter` to search -- **Left panel**: Search results with relevance scores `[0.95] content preview` -- **Right panel**: Full chunk content and metadata -- Use `↑` / `↓` to navigate results - detail view updates in real-time -- Press `Enter` on a result to close search and navigate to that document/chunk -- Press `Esc` to close search without selecting - -Search uses hybrid (vector + full-text) search across all chunks. Content is rendered as markdown with syntax highlighting. - -## Context Expansion - -Press `c` while viewing a chunk to open the context expansion modal: - -- Shows the expanded context that would be provided to the QA agent -- Type-aware expansion: tables, code blocks, and lists expand to their complete structures -- Text content expands based on `search.context_radius` setting -- Includes metadata like source document, content type, and relevance score -- Press `Esc` to close the modal - -## Visual Grounding - -Press `v` while viewing a chunk to open the visual grounding modal: - -- Shows page images from the source document with the chunk's location highlighted in yellow/orange -- Use `←` / `→` arrow keys to navigate between pages (when chunk spans multiple pages) -- Press `Esc` to close the modal - -!!! note - Visual grounding requires documents with a stored DoclingDocument that includes page images. Text-only documents or documents imported without DoclingDocument won't have visual grounding available. diff --git a/docs/tutorial.md b/docs/tutorial.md index cae572d8..adaa08e8 100644 --- a/docs/tutorial.md +++ b/docs/tutorial.md @@ -203,6 +203,7 @@ The following people are presenting talks at PyCon Finland 2025: ## Next Steps +- **[Chat](apps.md#chat-tui)** - Interactive conversations with `haiku-rag chat` - **[CLI Reference](cli.md)** - All available commands and options - **[Python API](python.md)** - Use haiku.rag in your Python applications - **[Agents](agents.md)** - Deep QA and multi-agent research workflows diff --git a/mkdocs.yml b/mkdocs.yml index 86a50ed6..9a4d04ce 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -59,6 +59,7 @@ nav: - index.md - Getting started: tutorial.md - Installation: installation.md + - Architecture: architecture.md - Configuration: - configuration/index.md - Providers: configuration/providers.md @@ -71,10 +72,10 @@ nav: - Custom Pipelines: custom-pipelines.md - Tuning: tuning.md - Agents: agents.md + - Applications: apps.md - Server: server.md - Remote processing: remote-processing.md - MCP: mcp.md - - Inspector: inspector.md - Benchmarks: benchmarks.md - Development: development.md - Changelog: changelog.md