danvics/haiku.rag
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge pull request #378 from ggozad/feat/better-docs

Browse source 
Improve documentation.
This commit is contained in:
Yiorgis Gozadinos 2026-05-20 14:33:14 +03:00 committed by GitHub
commit 1d3b6bb0c6
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
29 changed files with 1359 additions and 1568 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
CHANGELOG.md
View file

@ -53,6 +53,10 @@
- Chat TUI's state-edit screen syntax-highlights JSON instead of falling back to plain text. Adds `tree-sitter` + `tree-sitter-json` to the `[tui]` extra.
### Documentation
- Rework documentation
## [0.47.0] - 2026-05-14
### Added

117
docs/agents/analysis.md
View file

@ -1,117 +0,0 @@
# Analysis
The analysis skill enables complex analytical tasks by writing and executing Python code in a sandboxed environment. It solves problems that traditional RAG struggles with:
- **Aggregation**: "How many documents mention security vulnerabilities?"
- **Computation**: "What's the average revenue across all quarterly reports?"
- **Multi-document analysis**: "Compare the key findings between Report A and Report B"
- **Structured data extraction**: "Extract all dollar amounts and compute totals"
## How It Works
1. The skill receives a question
2. It writes Python code to explore the knowledge base
3. Code executes in a sandboxed Python interpreter with access to search and a virtual filesystem of documents
4. The skill iterates: run code, examine results, refine approach
5. Final answer is synthesized from the gathered data
## CLI Usage
```bash
# Basic usage
haiku-rag analyze "How many documents are in the database?"
# With document filter (restricts what the skill can access)
haiku-rag analyze "Summarize the key points" --filter "uri LIKE '%report%'"
```
## Python Usage
```python
from haiku.rag.client import HaikuRAG
async with HaikuRAG(path_to_db) as client:
result = await client.analyze("How many documents mention 'security'?")
print(result.answer)
for citation in result.citations:
print(citation.uri, citation.title)
# With filter (skill can only see filtered documents)
result = await client.analyze(
"What is the total revenue?",
filter="title LIKE '%Financial%'"
)
```
The executed Python program(s) for each turn are not on `AnalysisResult` itself; they live on `AnalysisState.executions` while the skill runs.
## Sandbox Capabilities
The agent's code runs in a sandboxed Python interpreter ([pydantic-monty](https://github.com/pydantic/monty)) with:
### Functions
| Function | Description |
|----------|-------------|
| `search(query, limit)` | Hybrid search (vector + full-text) with automatic context expansion. Returns `doc_item_refs` and `picture_refs` for cross-referencing with `items.jsonl` |
| `list_documents()` | List all documents in the knowledge base |
### Document Filesystem
All documents are mounted as a virtual filesystem at `/documents/`. The agent uses standard Python `pathlib.Path` to browse and read files:
```
/documents/{document_id}/
metadata.json # {id, title, uri, created_at}
content.txt # Full document text
items.jsonl # Structured items: position, self_ref, label, text, page_numbers
```
- **`metadata.json`** — Loaded eagerly (small). Use `Path('/documents').iterdir()` to discover documents.
- **`content.txt`** — Lazy-loaded on first read. Full document text for regex or keyword search.
- **`items.jsonl`** — Lazy-loaded on first read. One JSON object per line with structured document elements. Tables are pre-rendered as markdown. Labels include `section_header`, `text`, `table`, `list_item`, `caption`, `formula`, `picture`, `code`, `footnote`, etc.
Search results include `doc_item_refs` (e.g. `["#/texts/5", "#/tables/0"]`) that match `self_ref` values in `items.jsonl`, enabling navigation from search hits to document structure.
### Python Features
The interpreter supports a subset of Python: variables, arithmetic, strings, f-strings, lists, dicts, tuples, sets, loops, conditionals, comprehensions, functions, async/await, `filter()`, `getattr()`, try/except, file I/O via `pathlib.Path`, and the `json`, `re`, `math` modules.
Not supported: most imports (only `json`, `re`, `math`, `pathlib` are available), class definitions, generators/yield, match statements, decorators, `with` statements. For pattern matching, use `import re` or string methods.
### Security
Code executes in an isolated interpreter with:
- **Virtual filesystem only**: The `/documents/` filesystem is sandboxed — no access to the real filesystem
- **No network access**: Code cannot make HTTP requests or open sockets
- **No imports**: Only `json`, `re`, `math`, and `pathlib` modules are available
- **Execution timeout**: Configurable limit (default 60s)
- **Output truncation**: Large outputs are truncated to prevent memory issues
## Context Filter
The `filter` parameter restricts what documents the skill can access. Unlike tool parameters, the filter is applied automatically and cannot be bypassed by the LLM — both the VFS and search results are scoped to the filter:
```python
# Skill can only see documents with "confidential" in the URI
result = await client.analyze(
"Summarize all findings",
filter="uri LIKE '%confidential%'"
)
```
This is useful for scoping to specific document sets, enforcing access control, or limiting context for focused analysis.
## Configuration
Analysis settings can be configured in `haiku.rag.yaml`:
```yaml
analysis:
model:
provider: anthropic
name: claude-sonnet-4-20250514
code_timeout: 60.0 # Max seconds for code execution
max_output_chars: 50000 # Truncate output after this many chars
```

177
docs/apps.md
View file

@ -1,90 +1,34 @@
# Applications
# Web application
Three interactive applications for working with your knowledge base.
A browser-based reference implementation of conversational RAG, built on a Starlette backend with pydantic-ai's `AGUIAdapter` and a Next.js / CopilotKit frontend. It lives in the `app/` directory of the haiku.rag repository.
## Chat TUI
This is a starting point for your own deployments, not the canonical haiku.rag UX. For the day-to-day terminal experience see [Chat](chat.md).
Conversational RAG from the terminal with streaming responses and session memory.
## Features
!!! note
Requires the `tui` extra: `pip install haiku.rag-slim[tui]` (included in full `haiku.rag` package)
- Streaming chat with real-time tool execution visibility.
- Expandable citations with source documents, pages, and headings.
- Visual grounding to view chunk source locations in documents.
- Document filter to restrict searches to selected documents.
- Session state view for inspecting citations and search results.
### Usage
```bash
haiku-rag chat
haiku-rag chat --db /path/to/database.lancedb
# Enable analysis skill (code execution)
haiku-rag chat -s rag -s analysis
# Analysis only
haiku-rag chat -s analysis
```
### Interface
The chat interface provides:
- Streaming responses with real-time tool execution indicators
- Expandable citations showing source document, pages, and headings
- Session memory for context-aware follow-up questions
- Visual grounding to inspect chunk source locations
![Chat TUI interface](img/tui-qa.svg)
<div style="padding:56.25% 0 0 0;position:relative;"><iframe src="https://player.vimeo.com/video/1159658167?badge=0&amp;autopause=0&amp;player_id=0&amp;app_id=58479" frameborder="0" allow="autoplay; fullscreen; picture-in-picture; clipboard-write; encrypted-media" style="position:absolute;top:0;left:0;width:100%;height:100%;" title="haiku.rag Chat TUI demo"></iframe></div><script src="https://player.vimeo.com/api/player.js"></script>
*Demo: Chatting with an agent over 1000 arXiv papers. Shows context building (3:00), citations with visual grounding (3:20), and document listing/retrieval.*
### Command Palette
Press `Ctrl+P` to open the command palette:
| Command | Description |
|---------|-------------|
| View state | View the current session state |
| Filter documents | Select documents to restrict searches |
| Show database info | View document/chunk counts and storage info |
| Visual grounding | View chunk source location in document |
| Clear chat | Clear chat history and reset session |
### Session Management
- Conversation history is maintained in memory for the session
- Citations are tracked per response and can be inspected
- Document filter restricts all searches to selected documents
- Clearing chat resets session state
## Web Application
Browser-based conversational RAG with a CopilotKit frontend.
### Features
- Streaming chat with real-time tool execution visibility
- Expandable citations with source documents, pages, and headings
- Visual grounding to view chunk source locations in documents
- Document filter to restrict searches to selected documents
- Session state view for inspecting citations and search results
### Quick Start
## Quick start
```bash
cd app
docker compose -f docker-compose.dev.yml up -d --build
```
- Frontend: http://localhost:3000
- Backend: http://localhost:8001
- Frontend: `http://localhost:3000`
- Backend: `http://localhost:8001`
### Architecture
## Architecture
- **Backend**: Starlette server with pydantic-ai `AGUIAdapter`
- **Frontend**: Next.js with CopilotKit
- **Protocol**: AG-UI for streaming chat
- **Backend**: Starlette server with pydantic-ai `AGUIAdapter`.
- **Frontend**: Next.js with CopilotKit.
- **Protocol**: AG-UI for streaming chat.
### Configuration
## Configuration
Create a `.env` file in the `app/` directory:
@ -113,7 +57,7 @@ qa:
name: claude-sonnet-4-20250514
```
### API Endpoints
## API endpoints
| Endpoint | Method | Description |
|----------|--------|-------------|
@ -123,91 +67,12 @@ qa:
| `/api/visualize/{chunk_id}` | GET | Visual grounding images (base64) |
| `/health` | GET | Health check |
### Development
## Development
**Hot reload**: The backend reloads automatically on file changes. For frontend changes:
The backend reloads automatically on file changes. For frontend changes:
```bash
docker compose -f docker-compose.dev.yml up -d --build frontend
```
**Logfire debugging**: If `LOGFIRE_TOKEN` is set, LLM calls are traced and available in the Logfire dashboard.
## Inspector
TUI for browsing documents, chunks, and search results.
!!! note
Requires the `tui` extra: `pip install haiku.rag-slim[tui]` (included in full `haiku.rag` package)
### Usage
```bash
haiku-rag inspect
haiku-rag inspect --db /path/to/database.lancedb
```
### Interface
Three panels display your data:
- **Documents** (left) - All documents in the database
- **Chunks** (top right) - Chunks for the selected document
- **Detail View** (bottom right) - Full content and metadata
![Inspector search](img/inspector-search.svg)
### Navigation
| Key | Action |
|-----|--------|
| `Tab` | Cycle between panels |
| `↑` / `↓` | Navigate lists |
| `/` | Open search modal |
| `c` | Context expansion modal (when viewing a chunk) |
| `v` | Visual grounding modal (when viewing a chunk) |
| `q` | Quit |
**Mouse**: Click to select, scroll to view content.
### Search
Press `/` to open the full-screen search modal:
- Enter your query and press `Enter` to search
- **Left panel**: Search results with relevance scores `[0.95] content preview`
- **Right panel**: Full chunk content and metadata
- Use `↑` / `↓` to navigate results
- Press `Enter` on a result to navigate to that document/chunk
- Press `Esc` to close search
Search uses hybrid (vector + full-text) search across all chunks.
### Context Expansion
Press `c` while viewing a chunk to see the expanded context that would be provided to the rag skill:
- Section-aware expansion: expands to fill the current document section
- Noise filtering: footnotes, page headers/footers excluded from structured documents
- Includes metadata like source document, content type, and relevance score
### Visual Grounding
Visual grounding shows exactly where a chunk appears in the original document by highlighting its bounding box on the page image. This helps verify chunk boundaries and understand how content was extracted.
Press `v` while viewing a chunk to see page images with the chunk's location highlighted:
- Bounding boxes show the exact region of the page that maps to the chunk
- Use `←` / `→` arrow keys to navigate between pages when a chunk spans multiple pages
- Press `Esc` to close the modal
![Visual grounding modal](img/tui-visual-grounding.png)
#### Requirements
- **Page images**: Documents must be processed with Docling's page image extraction enabled (default for PDFs)
- **Terminal image support**: Your terminal must support inline images (e.g., iTerm2, WezTerm, Kitty). Terminals without image support will show a placeholder message.
- **DoclingDocument storage**: Text-only documents (plain text, markdown added via `add`) don't have visual grounding available
!!! tip
You can also view visual grounding from the command line with `haiku-rag visualize <chunk_id>`. See [CLI documentation](cli.md#visualize-chunk) for details.
If `LOGFIRE_TOKEN` is set, LLM calls are traced and available in the Logfire dashboard.

134
docs/benchmarks.md
View file

@ -1,14 +1,14 @@
# Benchmarks
We evaluate `haiku.rag` on several datasets to measure both retrieval quality and question-answering accuracy.
We evaluate `haiku.rag` on a small set of datasets that exercise different parts of the pipeline. Wix and OpenRAG Bench (ORB) are the two we currently track. Retrieval, QA accuracy, and citation retrieval are scored end-to-end through the rag and rag-analysis skills.
## Running Evaluations
You can run evaluations with the `evaluations` CLI:
```bash
evaluations run repliqa
evaluations run wix
evaluations run orb_text
```
The evaluation flow is orchestrated with [`pydantic-evals`](https://github.com/pydantic/pydantic-ai/tree/main/libs/pydantic-evals), which we leverage for dataset management, scoring, and report generation.
@ -19,29 +19,34 @@ Building evaluation databases from scratch can take a long time, especially for
```bash
# Download a specific dataset
evaluations download repliqa
evaluations download wix
# Download all datasets
evaluations download all
# Force re-download (overwrite existing)
evaluations download repliqa --force
evaluations download wix --force
```
Available datasets:
Active datasets:
| Dataset | Size |
|---------|------|
| `wix` | ~511MB |
| `orb_text` — OpenRAG Bench, text embedder (`qwen3-embedding:4b`) with VLM picture descriptions baked into chunk content | ~18 GB |
| `orb_multimodal` — OpenRAG Bench, multimodal embedder (`qwen3-vl-embedding-8b`); picture vectors live in the same space as text for cross-modal retrieval | ~16 GB |
Inactive (kept downloadable, not currently maintained):
| Dataset | Size |
|---------|------|
| `repliqa` | ~30MB |
| `hotpotqa` | ~331MB |
| `wix` | ~511MB |
| `orb_text` — OpenRAG Bench, text embedder (`qwen3-embedding:4b`) with VLM picture descriptions baked into chunk content | ~18 GB |
| `orb_multimodal` — OpenRAG Bench, multimodal embedder (`qwen3-vl-embedding-8b`); picture vectors live in the same space as text for cross-modal retrieval | ~16 GB |
After downloading, run benchmarks with `--skip-db` to use the pre-built database:
```bash
evaluations run repliqa --skip-db
evaluations run wix --skip-db
```
### Configuration
@ -49,7 +54,7 @@ evaluations run repliqa --skip-db
The benchmark script accepts several options:
```bash
evaluations run repliqa --config /path/to/haiku.rag.yaml --db /path/to/custom.lancedb
evaluations run wix --config /path/to/haiku.rag.yaml --db /path/to/custom.lancedb
```
**Options:**
@ -61,12 +66,21 @@ evaluations run repliqa --config /path/to/haiku.rag.yaml --db /path/to/custom.la
- `--skip-qa` - Skip QA benchmark
- `--limit N` - Limit number of test cases
- `--name NAME` - Override the evaluation name
- `--judge-model PROVIDER:NAME` - Override the LLM judge model. Defaults to `ollama:qwen3.6` so the judge stays stable when the answering model changes.
- `--target {rag-skill,analysis-skill}` - Choose which [skill](skills/index.md) to benchmark end-to-end against the same datasets and judge (default: `rag-skill`).
- `--target {rag-skill,analysis-skill}` - Choose which [skill](skills/index.md) to benchmark end-to-end (default: `rag-skill`).
- `--skill-model PROVIDER:NAME` - Override the skill model independently from the judge (default: `config.qa.model`, or `config.analysis.model` when set for `--target analysis-skill`).
If no config file is specified, the script searches standard locations: `./haiku.rag.yaml`, user config directory, then falls back to defaults.
To pin the LLM judge in YAML (rather than the default `ollama:qwen3.6`):
```yaml
evaluations:
judge:
provider: openai
name: gpt-4o-mini
base_url: http://localhost:8000/v1 # optional, for OpenAI-compatible servers (vLLM, LM Studio, etc.)
```
## Methodology
### Retrieval Metrics
@ -84,19 +98,19 @@ If no config file is specified, the script searches standard locations: `./haiku
- For each relevant document at position k, calculate precision@k = (relevant docs in top k) / k
- Average Precision (AP) = mean of these precision values / total relevant documents
- MAP is the mean of AP scores across all queries
- Range: 0 to 1; rewards ranking relevant documents higher
- Range: 0 to 1. Rewards ranking relevant documents higher
### QA Accuracy
For question-answering evaluation, `pydantic-evals` coordinates an LLM judge to determine whether answers are correct. The default judge is `ollama:qwen3.6` — pinned so changes to the skill model don't change the judge underneath. Override per run with `--judge-model provider:name`. Accuracy is the fraction of correctly answered questions.
`pydantic-evals` coordinates an LLM judge to determine whether the skill's answer is correct. The default judge is `ollama:qwen3.6`, pinned so changes to the skill model don't change the judge underneath. Set `evaluations.judge` in `haiku.rag.yaml` to override (including a custom `base_url` for any OpenAI-compatible endpoint). Accuracy is the fraction of correctly answered questions.
We picked `qwen3.6` over the previously-pinned `gpt-oss` after a 4-cell calibration (gpt-oss / qwen3.6 as both answerer and judge, with Claude Opus 4.7 as a reference). `qwen3.6` had κ ≥ 0.66 vs the reference on both same-family and cross-family answerers (vs ~0.39–0.55 for `gpt-oss`) and showed no measurable self-preference bias, while `gpt-oss` was ~10 pp more lenient on its own outputs.
### Citation Retrieval
When benchmarking a skill (`--target rag-skill` or `--target analysis-skill`), a second metric scores the URIs the skill registered via the `cite` tool against each dataset's gold `expected_uris`, using the same MRR / MAP math as raw retrieval. The score key is `cited_mrr` for single-doc datasets and `cited_map` for multi-doc. Console output also includes the cite rate (% of cases with at least one citation) and the mean number of citations per case.
Alongside QA accuracy, a second metric scores the URIs the skill registered via the `cite` tool against each dataset's gold `expected_uris`, using the same MRR / MAP math as raw retrieval. The score key is `cited_mrr` for single-doc datasets and `cited_map` for multi-doc. Console output also includes the cite rate (% of cases with at least one citation) and the mean number of citations per case.
This is computed alongside QA accuracy from the same skill run — no extra invocations. The signal complements raw retrieval: where raw retrieval measures whether the retriever surfaced the gold document at any rank, citation retrieval measures whether the skill grounded its answer on it.
This is computed alongside QA accuracy from the same skill run, no extra invocations. The signal complements raw retrieval: where raw retrieval measures whether the retriever surfaced the gold document at any rank, citation retrieval measures whether the skill grounded its answer on it.
## Current results
@ -104,25 +118,15 @@ Numbers measured under the current pinned judge (`ollama:qwen3.6`) on a recent `
### Wix
[WixQA](https://huggingface.co/datasets/Wix/WixQA) — real customer support questions paired with curated answers. 200 cases.
[WixQA](https://huggingface.co/datasets/Wix/WixQA) is real customer support questions paired with curated answers. 200 cases.
#### QA Accuracy
`evaluations run wix --target rag-skill` runs the RAG skill end-to-end and produces both QA accuracy and a citation retrieval metric (`cited_map`) computed from the URIs the skill registered via the `cite` tool against the gold `expected_uris`.
| Embedding Model | Chunk size | QA Model | Accuracy | Notes |
|----------------------|------------|-----------------------------|----------|------------------------|
| `qwen3-embedding:4b` | 256 | `gpt-oss:20b` - thinking | 0.88 | html, `chunk-radius=2` |
| Skill model | Reranker | QA accuracy | Mean `cited_map` |
|------------------------------|------------------------|-------------|------------------|
| `vllm:Gemma-4-26B-A4B-NVFP4` | `mxbai-rerank-base-v2` | 0.87 | 0.38 |
*Measured on haiku.rag v0.43.1, judged by `ollama:qwen3.6` (current default), 175 / 200 = 87.5 %.*
#### Skill QA + citation retrieval
`evaluations run wix --target rag-skill` benchmarks the RAG skill end-to-end and produces both QA accuracy and a citation retrieval metric (`cited_map`) computed from the URIs the skill registered via the `cite` tool against the gold `expected_uris`.
| Skill model | QA accuracy | Mean `cited_map` |
|------------------|-------------|------------------|
| `ollama:gpt-oss` | 0.85 | 0.40 |
*Measured on haiku.rag v0.43.1, judged by `ollama:qwen3.6` (current default), on 199 of 200 completed cases.* 28 % of cases produce a perfect citation (`cited_map` = 1.0).
*Measured on haiku.rag v0.48.0 with `qwen3-embedding:4b` (vLLM, dim 2560), `chunk_size=256`, `search.limit=5`. Judged by `vllm:Qwen3.6-35B-A3B-NVFP4` (qwen3.6 family, NVFP4 quant served via vLLM rather than the default Ollama). 172 / 198 completed cases (2 errored).*
### OpenRAG Bench (ORB)
@ -131,7 +135,7 @@ Numbers measured under the current pinned judge (`ollama:qwen3.6`) on a recent `
Two approaches are benchmarked separately:
- **Multimodal embedder** (`Qwen/Qwen3-VL-Embedding-8B`, served via vLLM): picture bytes and text live in a shared vector space, no VLM is run at ingest.
- **Text embedder + VLM picture descriptions** (`qwen3-embedding:4b` + `ollama/ministral-3`): pictures are described at ingest and the descriptions are woven into chunk text; retrieval runs over text only. See [Picture Description configuration](configuration/processing.md#picture-description-vlm).
- **Text embedder + VLM picture descriptions** (`qwen3-embedding:4b` + `ollama/ministral-3`): pictures are described at ingest and the descriptions are woven into chunk text. Retrieval runs over text only. See [Picture handling configuration](configuration/processing.md#picture-handling).
#### Multimodal embedder
@ -147,7 +151,7 @@ Two approaches are benchmarked separately:
##### QA Accuracy
| Embedding Model | QA Model | Reranker | Source bucket | Cases | Accuracy |
| Embedding Model | Skill model | Reranker | Source bucket | Cases | Accuracy |
|------------------------------|-----------------------------------|------------------------|---------------|------:|---------:|
| `Qwen/Qwen3-VL-Embedding-8B` | `ollama:qwen3.6` (vision) | none | text only | 682 | 96.9 % |
| `Qwen/Qwen3-VL-Embedding-8B` | `ollama:qwen3.6` (vision) | none | with image | 299 | 91.3 % |
@ -167,17 +171,7 @@ Two approaches are benchmarked separately:
*Measured on haiku.rag v0.45.0.*
##### QA Accuracy
| Embedding Model | VLM | QA Model | Reranker | Accuracy |
|----------------------|----------------------|-------------------------------------|------------------------|---------:|
| `qwen3-embedding:4b` | Ollama / ministral-3 | `ollama:qwen3.6` | none | 0.95 |
| `qwen3-embedding:4b` | Ollama / ministral-3 | `vllm:Gemma-4-26B-A4B-NVFP4` | none | 0.81 |
| `qwen3-embedding:4b` | Ollama / ministral-3 | `vllm:Gemma-4-26B-A4B-NVFP4` | `mxbai-rerank-base-v2` | 0.92 |
*Measured on haiku.rag v0.45.0, judged by `ollama:qwen3.6` (current default).*
##### Skill QA + citation retrieval
##### QA accuracy + citation retrieval
| Embedding Model | VLM | Skill model | QA accuracy | Mean `cited_map` |
|------------------------|----------------------|------------------------------|-------------|------------------|
@ -188,11 +182,9 @@ Two approaches are benchmarked separately:
*`vllm:Gemma-4-26B-A4B-NVFP4` row measured on haiku.rag v0.47.0, with `mxbai-rerank-base-v2`, stopped at 674 of 3045 cases (cumulative means stable from case ~200).*
*Both judged by `ollama:qwen3.6` (current default).*
## Past results
## Inactive datasets
These were measured under the prior pinned judge (`ollama:gpt-oss`). The pinned default has since switched to `ollama:qwen3.6` (see [Methodology — QA Accuracy](#qa-accuracy)) — under the new judge the QA accuracy numbers below typically shift up by ~5–10 pp.
Retrieval tables don't depend on the judge but are kept here because they were measured on the same older `haiku.rag` versions as their accompanying QA tables.
The benchmarks below are not currently maintained. Numbers were measured against earlier `haiku.rag` versions and an older pinned judge (`ollama:gpt-oss`), before the skill workflow became the only path. Retrieval tables don't depend on the judge, but the QA tables aren't reproducible against the current skill-only setup. We may revive them.
### RepliQA
@ -220,9 +212,29 @@ Retrieval tables don't depend on the judge but are kept here because they were m
Note the significant degradation when very small models are used such as `qwen3:0.6b`.
### Wix
### HotpotQA
[WixQA](https://huggingface.co/datasets/Wix/WixQA) — see description above. We benchmark both the plain text version (HTML stripped, no structure) and HTML version. Since HTML chunks are small (typically a phrase), we use `chunk_radius=2` to expand context.
[HotpotQA](https://huggingface.co/datasets/hotpotqa/hotpot_qa) is a multi-hop question answering dataset requiring reasoning over multiple Wikipedia paragraphs. Each question requires evidence from 2+ documents, making it ideal for testing retrieval and reasoning capabilities. We use MAP for retrieval evaluation since queries have multiple relevant documents.
#### Retrieval (MAP)
| Embedding Model | MAP | Reranker |
|----------------------|------|----------|
| `qwen3-embedding:4b` | 0.69 | none |
*Measured on haiku.rag v0.20.2.*
#### QA Accuracy
| Embedding Model | QA Model | Accuracy |
|----------------------|--------------------------|----------|
| `qwen3-embedding:4b` | `gpt-oss:20b` - thinking | 0.86 |
*Measured on haiku.rag v0.20.2, judged by `ollama:gpt-oss`.*
### Wix (historical, plain text and HTML)
Earlier Wix runs measured under different chunk settings and reranker combinations, against the older `gpt-oss` judge.
#### Retrieval (MAP)
@ -243,23 +255,3 @@ Note the significant degradation when very small models are used such as `qwen3:
| `qwen3-embedding:4b` | 256 | `gpt-oss:20b` - no thinking | 0.83 | html, `chunk-radius=2`, `jinaai/jina-reranker-v3` |
*Measured on haiku.rag v0.27.2, judged by `ollama:gpt-oss`.*
### HotpotQA
[HotpotQA](https://huggingface.co/datasets/hotpotqa/hotpot_qa) is a multi-hop question answering dataset requiring reasoning over multiple Wikipedia paragraphs. Each question requires evidence from 2+ documents, making it ideal for testing retrieval and reasoning capabilities. We use MAP for retrieval evaluation since queries have multiple relevant documents.
#### Retrieval (MAP)
| Embedding Model | MAP | Reranker |
|----------------------|------|----------|
| `qwen3-embedding:4b` | 0.69 | none |
*Measured on haiku.rag v0.20.2.*
#### QA Accuracy
| Embedding Model | QA Model | Accuracy |
|----------------------|--------------------------|----------|
| `qwen3-embedding:4b` | `gpt-oss:20b` - thinking | 0.86 |
*Measured on haiku.rag v0.20.2, judged by `ollama:gpt-oss`.*

79
docs/chat.md Normal file
View file

@ -0,0 +1,79 @@
# Chat
The chat TUI runs conversational RAG against your database from the terminal. Streaming responses, expandable citations with visual grounding, multi-turn sessions, and a command palette for filtering and inspection.
!!! note
Requires the `tui` extra: `pip install haiku.rag-slim[tui]` (included in the full `haiku.rag` package).
## Run it
```bash
haiku-rag chat
haiku-rag chat --db /path/to/database.lancedb
haiku-rag chat --model openai:gpt-4o
```
![Chat TUI session against the rag-analysis skill](img/chat-qa.png)
## How it works
The chat is a Pydantic AI agent with the `rag` [skill](skills/rag.md) attached by default. Each turn the agent decides which tool to call next, runs hybrid search against your documents, expands context around the hits, may issue further searches, and answers with citations. You see streaming text and a live indicator of which tool is running.
The session is in-memory for the lifetime of the TUI. Conversation history is kept across turns so follow-up questions reuse prior context. Citations are tracked per turn and inspectable via the command palette. Clearing the chat resets the session and the agent's memory.
## Citations and visual grounding
Each answer cites the chunks the agent used, with source document, page numbers, and section headings. Citations are expandable inline. Picture citations render the figure directly underneath the text snippet.
![Expanded citation with an inline figure](img/chat-citation-figure.png)
For visual grounding of a text chunk (the chunk highlighted on its source page image), open the command palette and pick "Show visual grounding". This requires:
- Documents processed via Docling with page images (default for PDFs).
- A terminal that supports inline images (iTerm2, WezTerm, Kitty).
- A stored DoclingDocument on the document. Plain text added via `haiku-rag add` doesn't have it.
You can also render visual grounding from the CLI without launching the TUI:
```bash
haiku-rag visualize <chunk_id>
```
## Command palette
`Ctrl+P` opens the palette.
| Command | What it does |
|---------|--------------|
| Clear chat | Reset session memory |
| Filter documents | Restrict searches to selected documents |
| Show visual grounding | Visual grounding for a citation |
| Database info | Document and chunk counts, storage stats |
| View state | Current session state, citations, and intermediate tool results |
## Skills
The default skill is `rag`. Enable `analysis` when the question needs computation, aggregation, comparison across documents, or section-scoped reading that a single search can't deliver:
```bash
# both skills (the agent routes between them)
haiku-rag chat -s rag -s analysis
# analysis only
haiku-rag chat -s analysis
```
The `analysis` skill mounts every document as a virtual filesystem at `/documents/{id}/` (with `metadata.json`, `content.txt`, `items.jsonl`, and `toc.json`) and runs Python in a sandboxed interpreter with `search` and `list_documents` as awaitable functions. It's the right choice for questions like:
- "How many of these documents mention X?"
- "Summarize Section 5 of paper Y."
- "Compare the experimental sections across these three reports."
- "Which section discusses the proof of Theorem 4.10?"
For everyday Q&A, the rag skill alone is faster and cheaper. Attaching both lets the agent pick. See [Analysis skill](skills/analysis.md) for the full sandbox capabilities and worked code patterns.
## Document filter
Run "Filter documents" from the command palette to restrict searches to a subset. The filter applies to every search the agent runs for the rest of the session.
Chat also honors the global `--read-only` and `--before` flags. See the [CLI reference](cli.md) for details.

460
docs/cli.md
View file

@ -26,24 +26,6 @@ The `haiku-rag` CLI provides complete document management functionality.
## Document Management
### List Documents
```bash
haiku-rag list
```
Filter documents by properties:
```bash
# Filter by URI pattern (--filter or -f)
haiku-rag list --filter "uri LIKE '%arxiv%'"
# Filter by exact title
haiku-rag list --filter "title = 'My Document'"
# Combine multiple conditions
haiku-rag list --filter "uri LIKE '%.pdf' AND title LIKE '%paper%'"
```
### Add Documents
From text:
@ -75,7 +57,7 @@ From directory (recursively adds all supported files):
haiku-rag add-src /path/to/documents/
```
From an S3 bucket (requires the `[s3]` extra — see [Server Mode → S3 / Object Storage Monitoring](server.md#s3-object-storage-monitoring)):
From an S3 bucket (requires the `[s3]` extra, see [Server Mode → S3 / Object Storage Monitoring](server.md#s3-object-storage-monitoring)):
```bash
# AWS S3 with credentials in the default chain (env vars, IAM role, AWS profile)
haiku-rag add-src s3://my-bucket/path/to/document.pdf
@ -95,6 +77,24 @@ AWS_ACCESS_KEY_ID=key AWS_SECRET_ACCESS_KEY=secret AWS_REGION=us-east-1 \
the database rolls back to the pre‑operation snapshot using LanceDB table versioning. You can optimize and
compact the database by running the [vacuum](#vacuum-optimize-and-cleanup) command.
### List Documents
```bash
haiku-rag list
```
Filter documents by properties:
```bash
# Filter by URI pattern (--filter or -f)
haiku-rag list --filter "uri LIKE '%arxiv%'"
# Filter by exact title
haiku-rag list --filter "title = 'My Document'"
# Combine multiple conditions
haiku-rag list --filter "uri LIKE '%.pdf' AND title LIKE '%paper%'"
```
### Get Document
```bash
@ -108,19 +108,6 @@ haiku-rag delete 3f4a... # document ID
haiku-rag rm 3f4a... # alias
```
## Visualize Chunk
Display visual grounding for a chunk - shows page images with highlighted bounding boxes:
```bash
haiku-rag visualize <chunk_id>
```
This renders the source document pages with the chunk's location highlighted. Useful for verifying chunk boundaries and understanding document structure.
!!! note
Requires a terminal with image support (iTerm2, Kitty, WezTerm, etc.) and documents processed with docling that have page images stored.
## Search
Basic search:
@ -176,12 +163,32 @@ Filter to specific documents:
haiku-rag ask "What are the main findings?" --filter "uri LIKE '%paper%'"
```
`ask` runs the [rag skill](skills/index.md) and always renders citations under the answer. When available, citations use the document title; otherwise they fall back to the URI.
`ask` runs the [rag skill](skills/index.md) and always renders citations under the answer. When available, citations use the document title, otherwise they fall back to the URI.
Flags:
- `--filter` / `-f`: Restrict searches to documents matching the filter (see [Filtering Search Results](python.md#filtering-search-results))
## Analyze
Answer complex analytical questions via code execution:
```bash
haiku-rag analyze "How many documents mention security?"
```
Filter to specific documents:
```bash
haiku-rag analyze "What is the total revenue?" --filter "title LIKE '%Financial%'"
```
Flags:
- `--filter` / `-f`: SQL WHERE clause to restrict document access
See [Analysis skill](skills/analysis.md) for details on capabilities and configuration.
## Chat
Launch an interactive chat session for multi-turn conversations:
@ -199,7 +206,7 @@ haiku-rag chat -s rag -s analysis
Flags:
- `--skill` / `-s`: Skills to enable — `rag` (default), `analysis`. Can be repeated for multiple skills.
- `--skill` / `-s`: Skills to enable. `rag` (default), `analysis`. Can be repeated for multiple skills.
The chat interface provides:
@ -208,7 +215,7 @@ The chat interface provides:
- Session memory for context-aware follow-up questions
- Visual grounding to inspect chunk source locations
See [Applications](apps.md#chat-tui) for keyboard shortcuts and features.
See [Chat](chat.md) for keyboard shortcuts and features.
## Inspect
@ -229,27 +236,207 @@ The inspector provides:
- Explore individual chunks
- Search and filter results
See [Applications](apps.md#inspector) for details.
See [Tuning: Inspector](tuning.md#inspector) for the full keybindings and modal flows.
## Analyze
## Visualize Chunk
Answer complex analytical questions via code execution:
Display visual grounding for a chunk - shows page images with highlighted bounding boxes:
```bash
haiku-rag analyze "How many documents mention security?"
haiku-rag visualize <chunk_id>
```
Filter to specific documents:
This renders the source document pages with the chunk's location highlighted. Useful for verifying chunk boundaries and understanding document structure.
!!! note
Requires a terminal with image support (iTerm2, Kitty, WezTerm, etc.) and documents processed with docling that have page images stored.
## Database lifecycle
### Initialize Database
Create a new database:
```bash
haiku-rag analyze "What is the total revenue?" --filter "title LIKE '%Financial%'"
haiku-rag init [--db /path/to/your.lancedb]
```
Flags:
This creates the database with the configured settings. **All other commands require an existing database** - they will fail with an informative error if the database doesn't exist.
- `--filter` / `-f`: SQL WHERE clause to restrict document access
### Info
See [Analysis](agents/analysis.md) for details on capabilities and configuration.
Display database metadata:
```bash
haiku-rag info [--db /path/to/your.lancedb]
```
Shows:
- path to the database
- stored haiku.rag version (from settings)
- embeddings provider/model and vector dimension
- number of documents and chunks (with storage sizes)
- vector index status (exists/not created, indexed/unindexed chunks)
- table versions per table (documents, chunks)
At the end, a separate "Versions" section lists runtime package versions:
- haiku.rag
- lancedb
- docling
### Migrate Database
Apply pending database migrations:
```bash
haiku-rag migrate [--db /path/to/your.lancedb]
```
When you upgrade haiku.rag to a new version that includes schema changes, the database requires migration. Opening a database with pending migrations will display an error:
```
Error: Database requires migration from 0.19.0 to 0.26.5. 3 migration(s) pending. Run 'haiku-rag migrate' to upgrade.
```
Run `haiku-rag migrate` to apply the pending migrations. The command shows which migrations were applied:
```
Applied 4 migration(s):
- 0.20.0: Add 'docling_document_json' and 'docling_version' columns
- 0.23.1: Add content_fts column for contextualized FTS search
- 0.25.0: Compress docling_document with gzip
- 0.38.0: Split docling_document pages into separate column and re-compress with zstd
Migration completed successfully.
```
!!! tip
Back up your database before running migrations. While migrations are designed to be safe, having a backup provides peace of mind for production databases.
### Download Models
Download required runtime models:
```bash
haiku-rag download-models
```
This command downloads:
- Docling OCR/conversion models
- HuggingFace tokenizer (for chunking)
- Ollama models referenced in your configuration (embeddings, QA, rerank)
Progress is displayed in real-time with download status and progress bars for Ollama model pulls.
## Maintenance
### Create Vector Index
Create a vector index on the chunks table for fast approximate nearest neighbor search:
```bash
haiku-rag create-index [--db /path/to/your.lancedb]
```
**Requirements:**
- Minimum 256 chunks required for index creation (LanceDB training data requirement)
- Creates an IVF_PQ index using the configured `search.vector_index_metric` (cosine/l2/dot)
**When to use:**
- After ingesting documents (indexes are not created automatically)
- After adding significant new data to rebuild the index
- Use `haiku-rag info` to check index status and see how many chunks are indexed/unindexed
**Search behavior:**
- Without index: Brute-force kNN search (exact nearest neighbors, slower for large datasets)
- With index: Fast ANN (approximate nearest neighbors) using IVF_PQ
- With stale index: LanceDB combines indexed results (fast ANN) + brute-force kNN on unindexed rows
- Performance degrades as more unindexed data accumulates
### Rebuild Database
Rebuild the database by re-indexing documents. Useful when switching embeddings provider/model or changing chunking settings:
```bash
# Full rebuild (default) - re-converts from source files, re-chunks, re-embeds
haiku-rag rebuild
# Re-chunk from stored content (no source file access)
haiku-rag rebuild --rechunk
# Only regenerate embeddings (fastest, keeps existing chunks)
haiku-rag rebuild --embed-only
# Only generate titles for untitled documents
haiku-rag rebuild --title-only
# Run the VLM over already-stored picture bytes and patch descriptions
# into the docling blob. Skips the docling parse entirely.
haiku-rag rebuild --descriptions
```
**Rebuild modes:**
| Mode | Flag | Use case |
|------|------|----------|
| Full | (default) | Changed converter, source files updated |
| Rechunk | `--rechunk` | Changed chunking strategy or chunk size |
| Embed only | `--embed-only` | Changed embedding model or vector dimensions |
| Title only | `--title-only` | Generate titles for documents without one |
| Descriptions | `--descriptions` | Add VLM picture descriptions to an existing database |
**`--descriptions` mode** runs the configured VLM (`processing.conversion_options.picture_description.model`) over the picture bytes already stored in `document_items.picture_data`, patches each description into the stored docling blob's `pictures[i].meta.description.text`, and re-chunks + re-embeds so chunk text reflects the new descriptions. Requires `processing.pictures: description` in the config. Idempotent: pictures that already carry a description are skipped, so the operation is safe to re-run after a partial failure. The docling parse is skipped entirely. Only the VLM time is paid.
### Vacuum (Optimize and Cleanup)
Reduce disk usage by optimizing and pruning old table versions across all tables:
```bash
haiku-rag vacuum
```
**Automatic Cleanup:** Vacuum runs automatically in the background after document operations. By default, it removes versions older than 1 day (configurable via `storage.vacuum_retention_seconds`), preserving recent versions for concurrent connections. Manual vacuum can be useful for cleanup after bulk operations or to free disk space immediately.
## Server
Start services (requires at least one flag):
```bash
# MCP server only (HTTP transport)
haiku-rag serve --mcp
# MCP server (stdio transport)
haiku-rag serve --mcp --stdio
# File monitoring only
haiku-rag serve --monitor
# Both services
haiku-rag serve --monitor --mcp
# Custom MCP port
haiku-rag serve --mcp --mcp-port 9000
# Read-only mode (excludes write MCP tools, disables monitor)
haiku-rag --read-only serve --mcp
```
See [Server Mode](server.md) for details on available services.
## Settings
View current configuration settings:
```bash
haiku-rag settings
```
### Generate Configuration File
Generate a YAML configuration file with defaults:
```bash
haiku-rag init-config [output_path]
```
If no path is specified, creates `haiku.rag.yaml` in the current directory.
## Create Skill
@ -309,191 +496,6 @@ haiku-skills chat --use-entrypoints --skill medic
└── haiku.rag.yaml # Optional config
```
## Server
Start services (requires at least one flag):
```bash
# MCP server only (HTTP transport)
haiku-rag serve --mcp
# MCP server (stdio transport)
haiku-rag serve --mcp --stdio
# File monitoring only
haiku-rag serve --monitor
# Both services
haiku-rag serve --monitor --mcp
# Custom MCP port
haiku-rag serve --mcp --mcp-port 9000
# Read-only mode (excludes write MCP tools, disables monitor)
haiku-rag --read-only serve --mcp
```
See [Server Mode](server.md) for details on available services.
## Settings
View current configuration settings:
```bash
haiku-rag settings
```
### Generate Configuration File
Generate a YAML configuration file with defaults:
```bash
haiku-rag init-config [output_path]
```
If no path is specified, creates `haiku.rag.yaml` in the current directory.
## Database Management
### Initialize Database
Create a new database:
```bash
haiku-rag init [--db /path/to/your.lancedb]
```
This creates the database with the configured settings. **All other commands require an existing database** - they will fail with an informative error if the database doesn't exist.
### Migrate Database
Apply pending database migrations:
```bash
haiku-rag migrate [--db /path/to/your.lancedb]
```
When you upgrade haiku.rag to a new version that includes schema changes, the database requires migration. Opening a database with pending migrations will display an error:
```
Error: Database requires migration from 0.19.0 to 0.26.5. 3 migration(s) pending. Run 'haiku-rag migrate' to upgrade.
```
Run `haiku-rag migrate` to apply the pending migrations. The command shows which migrations were applied:
```
Applied 4 migration(s):
- 0.20.0: Add 'docling_document_json' and 'docling_version' columns
- 0.23.1: Add content_fts column for contextualized FTS search
- 0.25.0: Compress docling_document with gzip
- 0.38.0: Split docling_document pages into separate column and re-compress with zstd
Migration completed successfully.
```
!!! tip
Back up your database before running migrations. While migrations are designed to be safe, having a backup provides peace of mind for production databases.
### Info
Display database metadata:
```bash
haiku-rag info [--db /path/to/your.lancedb]
```
Shows:
- path to the database
- stored haiku.rag version (from settings)
- embeddings provider/model and vector dimension
- number of documents and chunks (with storage sizes)
- vector index status (exists/not created, indexed/unindexed chunks)
- table versions per table (documents, chunks)
At the end, a separate "Versions" section lists runtime package versions:
- haiku.rag
- lancedb
- docling
### Create Vector Index
Create a vector index on the chunks table for fast approximate nearest neighbor search:
```bash
haiku-rag create-index [--db /path/to/your.lancedb]
```
**Requirements:**
- Minimum 256 chunks required for index creation (LanceDB training data requirement)
- Creates an IVF_PQ index using the configured `search.vector_index_metric` (cosine/l2/dot)
**When to use:**
- After ingesting documents (indexes are not created automatically)
- After adding significant new data to rebuild the index
- Use `haiku-rag info` to check index status and see how many chunks are indexed/unindexed
**Search behavior:**
- Without index: Brute-force kNN search (exact nearest neighbors, slower for large datasets)
- With index: Fast ANN (approximate nearest neighbors) using IVF_PQ
- With stale index: LanceDB combines indexed results (fast ANN) + brute-force kNN on unindexed rows
- Performance degrades as more unindexed data accumulates
### Vacuum (Optimize and Cleanup)
Reduce disk usage by optimizing and pruning old table versions across all tables:
```bash
haiku-rag vacuum
```
**Automatic Cleanup:** Vacuum runs automatically in the background after document operations. By default, it removes versions older than 1 day (configurable via `storage.vacuum_retention_seconds`), preserving recent versions for concurrent connections. Manual vacuum can be useful for cleanup after bulk operations or to free disk space immediately.
### Rebuild Database
Rebuild the database by re-indexing documents. Useful when switching embeddings provider/model or changing chunking settings:
```bash
# Full rebuild (default) - re-converts from source files, re-chunks, re-embeds
haiku-rag rebuild
# Re-chunk from stored content (no source file access)
haiku-rag rebuild --rechunk
# Only regenerate embeddings (fastest, keeps existing chunks)
haiku-rag rebuild --embed-only
# Only generate titles for untitled documents
haiku-rag rebuild --title-only
# Run the VLM over already-stored picture bytes and patch descriptions
# into the docling blob. Skips the docling parse entirely.
haiku-rag rebuild --descriptions
```
**Rebuild modes:**
| Mode | Flag | Use case |
|------|------|----------|
| Full | (default) | Changed converter, source files updated |
| Rechunk | `--rechunk` | Changed chunking strategy or chunk size |
| Embed only | `--embed-only` | Changed embedding model or vector dimensions |
| Title only | `--title-only` | Generate titles for documents without one |
| Descriptions | `--descriptions` | Add VLM picture descriptions to an existing database |
**`--descriptions` mode** runs the configured VLM (`processing.conversion_options.picture_description.model`) over the picture bytes already stored in `document_items.picture_data`, patches each description into the stored docling blob's `pictures[i].meta.description.text`, and re-chunks + re-embeds so chunk text reflects the new descriptions. Requires `processing.pictures: description` in the config. Idempotent — pictures that already carry a description are skipped, so the operation is safe to re-run after a partial failure. The docling parse is skipped entirely; only the VLM time is paid.
### Download Models
Download required runtime models:
```bash
haiku-rag download-models
```
This command downloads:
- Docling OCR/conversion models
- HuggingFace tokenizer (for chunking)
- Ollama models referenced in your configuration (embeddings, QA, rerank)
Progress is displayed in real-time with download status and progress bars for Ollama model pulls.
## Time Travel
LanceDB maintains version history for tables, enabling you to query the database as it existed at a previous point in time. This is useful for:

152
docs/configuration/processing.md
View file

@ -53,6 +53,37 @@ processing:
pictures: image # none | description | image
```
### Local vs Remote Processing
**Local processing** (default):
- Uses `docling` library locally
- No external dependencies
- Good for development and small workloads
**Remote processing** (docling-serve):
- Offloads processing to docling-serve API
- Better for heavy workloads and production
- Requires docling-serve instance (see [Remote processing setup](../remote-processing.md))
To use remote processing:
```yaml
processing:
converter: docling-serve
chunker: docling-serve
providers:
docling_serve:
base_url: http://localhost:5001
api_key: "your-api-key" # Optional
```
Conversion options work identically for both local and remote processing.
**Note:** When using `chunker: docling-serve`, OCR options (`do_ocr`, `force_ocr`, `ocr_engine`, `ocr_lang`) from `conversion_options` are passed to the chunking API. This is useful when running docling-serve in a read-only container where OCR model downloads fail. Set `do_ocr: false` to disable OCR entirely.
### Conversion Options
The `conversion_options` section allows fine-grained control over document conversion. These options work with both `docling-local` and `docling-serve` converters.
@ -104,7 +135,7 @@ conversion_options:
- **images_scale**: Scale factor for extracted images. Higher values = better quality but larger size. Typical range: 1.0-3.0.
- **generate_page_images**: When `true` (default), rendered images of each PDF page are included in the document. Required for `visualize_chunk()` to show visual grounding. When `false`, page images are excluded to reduce document size.
- **fetch_remote_images**: When `true` (default), HTML and Markdown inputs have their external `<img src="https://...">` URLs fetched and stored as picture bytes. Set `false` for air-gapped ingest. Applies only to docling-local; see [Remote processing](../remote-processing.md#html-image-fetching) for the docling-serve limitation.
- **fetch_remote_images**: When `true` (default), HTML and Markdown inputs have their external `<img src="https://...">` URLs fetched and stored as picture bytes. Set `false` for air-gapped ingest. Applies only to docling-local. See [Remote processing](../remote-processing.md#html-image-fetching) for the docling-serve limitation.
#### External image fetching
@ -115,9 +146,9 @@ For HTML and Markdown inputs, docling fetches images referenced by URL when `fet
- **Timeouts**: 5 s connect, 30 s read.
- **SVGs are skipped** (PIL cannot rasterize them).
- **`data:` URIs** are decoded inline (no network).
- **`file://` URIs** are *not* fetched — `enable_local_fetch` stays off to keep the SSRF surface narrow for arbitrary HTML/MD content.
- **`file://` URIs** are *not* fetched. `enable_local_fetch` stays off to keep the SSRF surface narrow for arbitrary HTML/MD content.
Per-image failures (404, timeout, oversized, unreadable) leave that picture as a placeholder with `picture_data=NULL` — the rest of the document still ingests.
Per-image failures (404, timeout, oversized, unreadable) leave that picture as a placeholder with `picture_data=NULL`. The rest of the document still ingests.
**Scope of conversion options across formats:**
@ -140,7 +171,7 @@ Per-image failures (404, timeout, oversized, unreadable) leave that picture as a
| `description` | on | yes | yes |
| `image` (default) | on | yes | no |
Use `none` when you don't need picture content (e.g. very large reference manuals where RAM is tight); use `description` to weave VLM-generated text into chunk content and keep bytes for later; use `image` (default) to keep bytes without paying the VLM cost. The prompt is configurable under `prompts.picture_description` — see [Prompts](prompts.md).
Use `none` when you don't need picture content (e.g. very large reference manuals where RAM is tight). Use `description` to weave VLM-generated text into chunk content and keep bytes for later. Use `image` (default) to keep bytes without paying the VLM cost. The prompt is configurable under `prompts.picture_description`. See [Prompts](prompts.md).
```yaml
processing:
@ -155,7 +186,7 @@ processing:
```
!!! warning "Breaking change"
`processing.conversion_options.picture_description.enabled` is replaced by `processing.pictures`. Map `enabled: true` → `pictures: description`, `enabled: false` → `pictures: image`. The pre-April-30 `generate_picture_images` flag also no longer exists; use `pictures: none` for the old opt-out.
`processing.conversion_options.picture_description.enabled` is replaced by `processing.pictures`. Map `enabled: true` → `pictures: description`, `enabled: false` → `pictures: image`. The pre-April-30 `generate_picture_images` flag also no longer exists. Use `pictures: none` for the old opt-out.
**Switching modes on an existing database** doesn't require reingesting when the bytes are already stored:
@ -163,7 +194,7 @@ processing:
- `description` → `image`: `haiku-rag rebuild --rechunk` recomposes chunk text from the stripped docling blob without descriptions.
- Switching to/from `none`: a full reingest is needed since the bytes either weren't stored or need to be discarded.
When using `converter: docling-serve`, the VLM is invoked from docling-serve rather than haiku.rag — see [Remote processing](../remote-processing.md#vlm-picture-description-with-docling-serve).
When using `converter: docling-serve`, the VLM is invoked from docling-serve rather than haiku.rag. See [Remote processing](../remote-processing.md#vlm-picture-description-with-docling-serve).
#### Pictures × embedder × QA model: how the pieces compose
@ -188,9 +219,9 @@ Three independent settings drive ingest, retrieval, and QA:
**What QA receives** at search time:
- `qa.model.vision: false` — text chunks only (descriptions, when present, answer figure questions in prose).
- `qa.model.vision: true` — text chunks + raw picture bytes via `BinaryContent`; the model reads figures directly. Requires `pictures != none` so the bytes exist.
- `qa.model.vision: true` — text chunks + raw picture bytes via `BinaryContent`. The model reads figures directly. Requires `pictures != none` so the bytes exist.
`qa.model.vision` is independent of ingestion — flipping it never requires reingesting. Setting `vision: true` against a text-only model causes silent acceptance and confabulation on Ollama and a 400 on OpenAI; default `false` is the safe choice.
`qa.model.vision` is independent of ingestion. Flipping it never requires reingesting. Setting `vision: true` against a text-only model causes silent acceptance and confabulation on Ollama and a 400 on OpenAI. Default `false` is the safe choice.
**Recommended combinations:**
@ -203,6 +234,39 @@ Three independent settings drive ingest, retrieval, and QA:
| Cross-modal search + vision QA | `image` or `description` | multimodal | `true` |
| Cross-modal search, text QA only | `description` | multimodal | `false` |
### Chunking Strategies
**Hybrid chunking** (default):
- Structure-aware chunking
- Respects document boundaries
- Best for most use cases
**Hierarchical chunking**:
- Creates hierarchical chunk structure
- Preserves document hierarchy
- Useful for complex documents
### Chunk Size
```yaml
processing:
chunk_size: 256 # Maximum tokens per chunk
```
Context expansion settings (for enriching search results with surrounding content) are configured in the `search` section. See [Search Settings](qa.md#search-settings).
### Table Serialization
Control how tables are represented in chunks:
```yaml
processing:
chunking_use_markdown_tables: false # Default: narrative format
```
- `false`: Tables as narrative text ("Value A, Column 2 = Value B")
- `true`: Tables as markdown (preserves table structure)
### Automatic Title Generation
Enable automatic title generation during document ingestion:
@ -218,79 +282,15 @@ processing:
When `auto_title` is enabled, haiku.rag attempts to extract a title for each document during ingestion using a two-tier approach:
1. **Structural extraction** (free, no model calls): Scans the DoclingDocument for semantic labels — HTML `<title>` tags, `<h1>` headings, PDF title blocks, and section headers
1. **Structural extraction** (free, no model calls): Scans the DoclingDocument for semantic labels (HTML `<title>` tags, `<h1>` headings, PDF title blocks, and section headers)
2. **LLM fallback**: When no structural title is found (e.g., plain text), generates a title using the configured `title_model`
Priority order: HTML `<title>` (furniture layer) → h1/PDF title (body layer) → first section header → LLM generation.
Explicit titles passed via `title=` parameter always take precedence and are never overridden. When updating documents, existing titles are preserved — auto-generation only applies to untitled documents.
Explicit titles passed via `title=` parameter always take precedence and are never overridden. When updating documents, existing titles are preserved. Auto-generation only applies to untitled documents.
To generate titles for existing untitled documents, use [`rebuild --title-only`](../cli.md#rebuild-database).
### Local vs Remote Processing
**Local processing** (default):
- Uses `docling` library locally
- No external dependencies
- Good for development and small workloads
**Remote processing** (docling-serve):
- Offloads processing to docling-serve API
- Better for heavy workloads and production
- Requires docling-serve instance (see [Remote processing setup](../remote-processing.md))
To use remote processing:
```yaml
processing:
converter: docling-serve
chunker: docling-serve
providers:
docling_serve:
base_url: http://localhost:5001
api_key: "your-api-key" # Optional
```
Conversion options work identically for both local and remote processing.
**Note:** When using `chunker: docling-serve`, OCR options (`do_ocr`, `force_ocr`, `ocr_engine`, `ocr_lang`) from `conversion_options` are passed to the chunking API. This is useful when running docling-serve in a read-only container where OCR model downloads fail—set `do_ocr: false` to disable OCR entirely.
### Chunking Strategies
**Hybrid chunking** (default):
- Structure-aware chunking
- Respects document boundaries
- Best for most use cases
**Hierarchical chunking**:
- Creates hierarchical chunk structure
- Preserves document hierarchy
- Useful for complex documents
### Table Serialization
Control how tables are represented in chunks:
```yaml
processing:
chunking_use_markdown_tables: false # Default: narrative format
```
- `false`: Tables as narrative text ("Value A, Column 2 = Value B")
- `true`: Tables as markdown (preserves table structure)
### Chunk Size
```yaml
processing:
chunk_size: 256 # Maximum tokens per chunk
```
Context expansion settings (for enriching search results with surrounding content) are configured in the `search` section. See [Search Settings](qa.md#search-settings).
## File Monitoring
Set directories to monitor for automatic indexing:
@ -379,8 +379,8 @@ monitor:
allow_http: "true"
```
Each entry is independent — own poll interval, own include/ignore patterns, own `delete_orphans` setting, own credentials. Omit `storage_options` to fall back to the AWS default credential chain (env vars, IAM role, AWS profile).
Each entry is independent: own poll interval, own include/ignore patterns, own `delete_orphans` setting, own credentials. Omit `storage_options` to fall back to the AWS default credential chain (env vars, IAM role, AWS profile).
The dict shape matches `lancedb.storage_options` — the same Rust `object_store` library is used by both, so credentials configured for the LanceDB backend can be copy-pasted here.
The dict shape matches `lancedb.storage_options`. The same Rust `object_store` library is used by both, so credentials configured for the LanceDB backend can be copy-pasted here.
See [Server Mode → S3 / Object Storage Monitoring](../server.md#s3-object-storage-monitoring) for behaviour details (ETag-based change detection, orphan-deletion scope, CLI `add-src s3://…`).

2
docs/configuration/prompts.md
View file

@ -24,7 +24,7 @@ The `domain_preamble` field provides **domain context** prepended to the rag and
- Clarify domain-specific terminology
- Provide context that helps the model interpret ambiguous queries
**Important:** `domain_preamble` is for domain context, not behavioral instructions. Descriptions of subject matter, terminology, and content scope belong here. Behavioral guidance (tone, response style, formatting rules) lives in the skill's SKILL.md — fork the skill via `haiku-rag create-skill` to customize behavior.
**Important:** `domain_preamble` is for domain context, not behavioral instructions. Descriptions of subject matter, terminology, and content scope belong here. Behavioral guidance (tone, response style, formatting rules) lives in the skill's SKILL.md. Fork the skill via `haiku-rag create-skill` to customize behavior.
**Example:**

8
docs/configuration/providers.md
View file

@ -22,7 +22,7 @@ qa:
**Available options:**
- **temperature**: Sampling temperature (0.0-1.0+). Defaults vary by task: 0.3 for QA and title generation; 0.0 for analysis and picture description.
- **temperature**: Sampling temperature (0.0-1.0+). Defaults vary by task: 0.3 for QA and title generation, 0.0 for analysis and picture description.
- Lower (0.0-0.3): Deterministic, focused responses
- Medium (0.4-0.7): Balanced
- Higher (0.8-1.0+): Creative, varied responses
@ -67,7 +67,7 @@ See the [Pydantic AI thinking documentation](https://ai.pydantic.dev/thinking/)
The `extra_body` setting takes a dict that haiku.rag forwards verbatim to the underlying model SDK as `ModelSettings.extra_body`. Use it to reach provider-specific keys that haiku.rag does not model with a dedicated field.
**Example — disable Qwen3 thinking on vLLM:**
**Example: disable Qwen3 thinking on vLLM:**
```yaml
qa:
@ -225,7 +225,7 @@ embeddings:
base_url: http://localhost:8000/v1
```
Tested with `Qwen/Qwen3-VL-Embedding-8B` (4096-dim) and `jinaai/jina-embeddings-v4` (2048-dim). Run vLLM separately; haiku.rag adds no Python ML dependencies for this path. Text inputs use the standard OpenAI `input` field; image inputs use vLLM's `messages`-with-`image_url` superset, transparently to the caller.
Tested with `Qwen/Qwen3-VL-Embedding-8B` (4096-dim) and `jinaai/jina-embeddings-v4` (2048-dim). Run vLLM separately. haiku.rag adds no Python ML dependencies for this path. Text inputs use the standard OpenAI `input` field. Image inputs use vLLM's `messages`-with-`image_url` superset, transparently to the caller.
Picture chunks for retrieval are emitted at ingest under any embedder reporting `supports_images=True`. See [Picture Handling](processing.md#picture-handling).
@ -464,7 +464,7 @@ reranking:
### Cross-Encoder (sentence-transformers)
Run any HuggingFace cross-encoder reranker in-process via `sentence-transformers` — no separate server required. Useful when you want a specific model (BGE, Qwen3-Reranker, MS-MARCO MiniLM, etc.) without running vLLM.
Run any HuggingFace cross-encoder reranker in-process via `sentence-transformers`. No separate server required. Useful when you want a specific model (BGE, Qwen3-Reranker, MS-MARCO MiniLM, etc.) without running vLLM.
Install the extra:

6
docs/configuration/qa.md
View file

@ -13,7 +13,7 @@ search:
- **limit**: Default number of search results to return when no limit is specified. Used by CLI, MCP server, and QA. Default: 10
- **max_context_chars**: Hard limit on total characters in expanded content. Default: 10000.
Context expansion is automatic and section-aware. For structured documents (with section headers), expansion includes the entire section containing the match. For sections that exceed the budget or are too small (e.g., a title+authors area), expansion grows outward item-by-item from the match center, skipping noise labels (footnotes, page headers) — this naturally crosses into adjacent sections until the budget is filled. For unstructured documents, expansion grows outward item-by-item. Results without `doc_item_refs` (e.g., custom chunks passed to `import_document`) pass through unexpanded.
Context expansion is automatic and section-aware. For structured documents (with section headers), expansion includes the entire section containing the match. For sections that exceed the budget or are too small (e.g., a title+authors area), expansion grows outward item-by-item from the match center, skipping noise labels (footnotes, page headers). This naturally crosses into adjacent sections until the budget is filled. For unstructured documents, expansion grows outward item-by-item. Results without `doc_item_refs` (e.g., custom chunks passed to `import_document`) pass through unexpanded.
!!! note "Reranking behavior"
When a reranker is configured, search automatically retrieves 10x the requested limit, then reranks to return the final count. This improves result quality without requiring you to adjust `limit`.
@ -34,7 +34,7 @@ qa:
```
- **model**: LLM configuration (see [Providers](providers.md#model-settings))
- **model.vision**: Set to `true` for vision-capable models (`qwen2.5vl`, `qwen3.6`, `gpt-4o`, `claude-sonnet`, …). The skill's `search` tool only attaches picture bytes (`BinaryContent`) to its `ToolReturn` when this is `true`; otherwise picture bytes are withheld. See [Pictures × embedder × QA model](processing.md#pictures--embedder--qa-model-how-the-pieces-compose) for the full matrix.
- **model.vision**: Set to `true` for vision-capable models (`qwen2.5vl`, `qwen3.6`, `gpt-4o`, `claude-sonnet`, …). The skill's `search` tool only attaches picture bytes (`BinaryContent`) to its `ToolReturn` when this is `true`, otherwise picture bytes are withheld. See [Pictures × embedder × QA model](processing.md#pictures-embedder-qa-model-how-the-pieces-compose) for the full matrix.
- **max_searches**: Maximum number of search tool calls the rag skill can make per question (default: 3)
## Analysis Configuration
@ -55,4 +55,4 @@ analysis:
- **code_timeout**: Maximum seconds for each code execution (default: 60)
- **max_output_chars**: Truncate code output after this many characters (default: 50000)
See [Analysis](../agents/analysis.md) for usage details.
See [Analysis skill](../skills/analysis.md) for usage details.

60
docs/configuration/storage.md
View file

@ -18,6 +18,34 @@ storage:
!!! warning "Vacuum Retention Threshold"
The `vacuum_retention_seconds` value should be larger than the typical time it takes to process and write a document. If a concurrent operation is in progress while vacuum runs, setting this value too low can cause race conditions where vacuum removes table versions that an in-flight operation still needs. The default of 86400 seconds (1 day) is conservative and safe for most use cases.
## Database Creation
Databases must be explicitly created before use:
**CLI:**
```bash
# Create in default location (see Configuration File Locations below)
haiku-rag init
# Create at custom path
haiku-rag init --db /path/to/database.lancedb
```
**Python:**
```python
# Create at custom path
async with HaikuRAG("/path/to/database.lancedb", create=True) as client:
...
# Create in default location
async with HaikuRAG(create=True) as client:
...
```
The [default location](index.md#configuration-file-locations) is platform-specific (e.g., `~/Library/Application Support/haiku.rag/` on macOS).
Operations on non-existent databases raise `FileNotFoundError`. This prevents accidental database creation from typos or misconfigured paths.
## Remote Storage
For remote storage, use the `lancedb` settings with various backends:
@ -70,13 +98,13 @@ lancedb:
- **Object storage** (`s3://`, `gs://`, `az://`, `hdfs://`): Uses `storage_options` for credentials and endpoint configuration. Authentication can also be provided via environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, etc.) or cloud provider SDK defaults (AWS CLI, Azure CLI, gcloud).
- **S3-compatible stores** (MinIO, Tigris, etc.): Set `endpoint` in `storage_options`. When using `http://` endpoints, also set `allow_http: "true"`.
The `storage_options` keys are case-insensitive and passed directly to the underlying object store library. Available keys depend on the backend — see the [LanceDB storage docs](https://lancedb.com/docs/storage/) for details.
The `storage_options` keys are case-insensitive and passed directly to the underlying object store library. Available keys depend on the backend. See the [LanceDB storage docs](https://lancedb.com/docs/storage/) for details.
**Note:** Table optimization is automatically handled by LanceDB Cloud (`db://` URIs) and is disabled for better performance. For object storage backends (S3, Azure, GCS), optimization and vector indexing are still performed normally.
### Deployment Pattern: One Writer, Many Readers
LanceDB on S3 supports **exactly one writer + N readers** per database URI. Multiple writers against the same URI can race on the manifest commit and corrupt state — this is a LanceDB property, not something `haiku.rag` enforces.
LanceDB on S3 supports **exactly one writer + N readers** per database URI. Multiple writers against the same URI can race on the manifest commit and corrupt state. This is a LanceDB property, not something `haiku.rag` enforces.
The recommended layout for production is "different buckets, same account, separate IAM roles per process":
@ -85,34 +113,6 @@ The recommended layout for production is "different buckets, same account, separ
Each process picks up its own credentials from the AWS default chain (env vars, IAM instance role, AWS profile), so no credentials are hard-coded in the configuration files.
## Database Creation
Databases must be explicitly created before use:
**CLI:**
```bash
# Create in default location (see Configuration File Locations below)
haiku-rag init
# Create at custom path
haiku-rag init --db /path/to/database.lancedb
```
**Python:**
```python
# Create at custom path
async with HaikuRAG("/path/to/database.lancedb", create=True) as client:
...
# Create in default location
async with HaikuRAG(create=True) as client:
...
```
The [default location](index.md#configuration-file-locations) is platform-specific (e.g., `~/Library/Application Support/haiku.rag/` on macOS).
Operations on non-existent databases raise `FileNotFoundError`. This prevents accidental database creation from typos or misconfigured paths.
## Vector Indexing
Configure vector search settings:

36
docs/custom-pipelines.md
View file

@ -1,6 +1,22 @@
# Custom Processing Pipelines
haiku.rag provides processing primitives that let you build custom document pipelines. Use these when you need control over conversion, chunking, or embedding—for example, to preprocess content, use external services, or implement custom chunking logic.
haiku.rag provides processing primitives that let you build custom document pipelines. Use these when you need control over conversion, chunking, or embedding (for example, to preprocess content, use external services, or implement custom chunking logic).
## When to Use Custom Pipelines
Use the primitives when you need to:
- Preprocess or clean content before chunking
- Filter or modify chunks before embedding
- Use external embedding services
- Implement custom chunking strategies
- Debug or inspect intermediate processing steps
For standard use cases, prefer the convenience methods:
- `create_document()` - Create from text content
- `create_document_from_source()` - Create from file or URL
- `import_document()` - Store pre-processed documents with custom chunks
## Processing Primitives
@ -120,7 +136,7 @@ assert embedded_chunks[0].embedding is not None
## Contextualize (for custom embedders)
`contextualize()` is a lower-level utility that prepares chunk content for embedding by prepending section headings. You only need this when implementing custom embedding logic—`embed_chunks()` already calls it internally.
`contextualize()` is a lower-level utility that prepares chunk content for embedding by prepending section headings. You only need this when implementing custom embedding logic. `embed_chunks()` already calls it internally.
```python
from haiku.rag.embeddings import contextualize
@ -233,19 +249,3 @@ async with HaikuRAG("database.lancedb", create=True) as client:
chunks=embedded_chunks,
)
```
## When to Use Custom Pipelines
Use the primitives when you need to:
- Preprocess or clean content before chunking
- Filter or modify chunks before embedding
- Use external embedding services
- Implement custom chunking strategies
- Debug or inspect intermediate processing steps
For standard use cases, prefer the convenience methods:
- `create_document()` - Create from text content
- `create_document_from_source()` - Create from file or URL
- `import_document()` - Store pre-processed documents with custom chunks

BIN
docs/img/chat-citation-figure.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 473 KiB

BIN
docs/img/chat-qa.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 470 KiB

306
docs/img/tui-qa.svg
View file

File diff suppressed because one or more lines are too long
Side by side

Before

Width:  |  Height:  |  Size: 64 KiB

83
docs/index.md
View file

@ -1,78 +1,47 @@
# haiku.rag
Agentic RAG built on [LanceDB](https://lancedb.com/), [Pydantic AI](https://ai.pydantic.dev/), and [Docling](https://docling-project.github.io/docling/).
haiku.rag is an agentic RAG that runs locally and scales to production. Index PDFs, web pages, or whole directories. Ask questions and get cited answers. Build agents, skills, and MCP integrations on top.
> **New: vision and multimodal search.** Picture-aware ingestion captures embedded figure bytes; vision-capable QA models receive them alongside text. Multimodal embedders (vLLM with `Qwen3-VL-Embedding-8B` or `jinaai/jina-embeddings-v4`) put picture vectors in the same space as text, enabling text-as-query → figure hits and image-as-query retrieval.
haiku.rag is open-source first. The defaults run open models through [Ollama](https://ollama.com/) so the full pipeline works without external API keys. Any provider Pydantic AI supports works in its place.
## Features
Built on [LanceDB](https://lancedb.com/), [Pydantic AI](https://ai.pydantic.dev/), and [Docling](https://docling-project.github.io/docling/). Embedded database, no servers required.
- **Hybrid search** — Vector + full-text with Reciprocal Rank Fusion
- **Multimodal & cross-modal search** — Multimodal embedders (vLLM) put picture vectors in the same space as text; supports text-as-query → figure hits and image-as-query
- **Question answering** — RAG skill with citations (page numbers, section headings)
- **Vision QA** — Vision-capable models receive figure bytes alongside chunk text via pydantic-ai `BinaryContent` when `qa.model.vision = true`
- **Reranking** — MxBAI, Cohere, Zero Entropy, or vLLM
- **Analysis skill** — Complex analytical tasks via sandboxed Python code execution (aggregation, computation, multi-document analysis)
- **Conversational RAG** — Chat TUI and web application for multi-turn conversations with session memory
- **Document structure** — Stores full [DoclingDocument](https://docling-project.github.io/docling/concepts/docling_document/), enabling structure-aware context expansion
- **Multiple providers** — Embeddings: Ollama, OpenAI, VoyageAI, LM Studio, vLLM (multimodal). QA: any model supported by Pydantic AI
- **Local-first** — Embedded LanceDB, no servers required. Also supports S3, GCS, Azure, and LanceDB Cloud
- **CLI & Python API** — Full functionality from command line or code
- **MCP server** — Expose as tools for AI assistants (Claude Desktop, etc.)
- **Visual grounding** — View chunks highlighted on original page images
- **File monitoring** — Watch directories and auto-index on changes
- **Time travel** — Query the database at any historical point with `--before`
- **Inspector** — TUI for browsing documents, chunks, and search results
## Quick Start
Install haiku.rag:
## See it work
```bash
uv pip install haiku.rag
ollama pull qwen3-embedding:4b
ollama pull gpt-oss
haiku-rag init
haiku-rag add-src ~/Documents/some-paper.pdf
haiku-rag chat
```
Use from Python:
The chat TUI is one way to interact with the database. `haiku-rag ask` and `haiku-rag search` cover one-shot CLI usage. Python integrations, skills, and the MCP server work against the same database.
```python
from haiku.rag.client import HaikuRAG
## What it does
async with HaikuRAG("database.lancedb", create=True) as client:
# Add a document
doc = await client.create_document("Your content here")
**Ingest.** PDFs, DOCX, HTML, images, and 40+ formats via Docling. Add files, URLs, or whole directories. Monitor folders and reindex on change.
# Search documents
results = await client.search("query")
**Search.** Hybrid retrieval (vector + full-text with reciprocal rank fusion), optional cross-encoder reranking, structure-aware context expansion. Image-as-query and cross-modal retrieval when configured with a multimodal embedder.
# Ask questions (returns answer and citations)
answer, citations = await client.ask("Who is the author of haiku.rag?")
```
**Answer.** RAG skill with citations including page numbers, section headings, and visual grounding. Vision-capable models receive figure bytes alongside chunk text. Analysis skill with a sandboxed Python interpreter for aggregation and computation across documents.
Or use the CLI:
**Integrate.** Use it from Python, the CLI, the [MCP server](mcp.md), or as composable [skills](skills/index.md) built on haiku.skills. Skills bundle tools, prompts, and state for use inside any Pydantic AI agent.
```bash
haiku-rag add "Your document content"
haiku-rag add "Your document content" --meta author=alice
haiku-rag add-src /path/to/document.pdf --title "Q3 Financial Report" --meta source=manual
haiku-rag search "query"
haiku-rag ask "Who is the author of haiku.rag?"
haiku-rag chat # Interactive conversation mode
```
**Operate.** Embedded LanceDB by default. Also runs on S3, GCS, Azure, or LanceDB Cloud. Time-travel queries via LanceDB versioning. File-monitoring mode for production deployments.
## Documentation
## Where to go next
- [Getting started](tutorial.md) - Tutorial
- [Installation](installation.md) - Install haiku.rag with different providers
- [Configuration](configuration/index.md) - Environment variables and settings
- [CLI](cli.md) - Command line interface usage
- [Python](python.md) - Python API reference
- [Custom Pipelines](custom-pipelines.md) - Build custom processing workflows
- [Skills](skills/index.md) - The RAG and analysis skills the client wraps
- [Analysis](agents/analysis.md) - Complex analytical tasks via code execution
- [Applications](apps.md) - Chat TUI, web app, and inspector
- [Server](server.md) - File monitoring and server mode
- [MCP](mcp.md) - Model Context Protocol integration
- [Remote processing](remote-processing.md) - Remote document processing with docling-serve
- [Quickstart](tutorial.md): install, index, chat.
- [Skills](skills/index.md): the rag and rag-analysis skills you compose into Pydantic AI agents.
- [Python API](python.md): use haiku.rag from code.
- [MCP server](mcp.md): expose haiku.rag to Claude Desktop or other AI assistants.
- [Tuning](tuning.md): improve retrieval quality.
- [Configuration](configuration/index.md): every setting.
## License
This project is licensed under the [MIT License](https://raw.githubusercontent.com/ggozad/haiku.rag/main/LICENSE).
MIT. Source on [GitHub](https://github.com/ggozad/haiku.rag).

2
docs/installation.md
View file

@ -15,8 +15,6 @@ The full package includes **all features and extras**:
- **All embedding providers** - VoyageAI
- **All rerankers** - MixedBread AI, Cohere, Zero Entropy
This is the easiest way to get started with all features enabled.
### Slim Package (Minimal Dependencies)
```bash

100
docs/mcp.md
View file

@ -2,6 +2,56 @@
The MCP server exposes `haiku.rag` as MCP tools for compatible MCP clients like Claude Desktop.
## Starting MCP Server
The MCP server supports Streamable HTTP and stdio transports:
```bash
# Default streamable HTTP transport on port 8001
haiku-rag serve --mcp
# Custom port
haiku-rag serve --mcp --mcp-port 9000
# stdio transport (for Claude Desktop)
haiku-rag serve --mcp --stdio
# Read-only mode (excludes write tools)
haiku-rag --read-only serve --mcp --stdio
```
**Read-only mode:** When `--read-only` is specified, write tools (`add_document_from_file`, `add_document_from_url`, `add_document_from_text`, `delete_document`) are not registered. Only search and query tools remain available.
## Claude Desktop Integration
Add to your Claude Desktop configuration (`claude_desktop_config.json`):
```json
{
"mcpServers": {
"haiku-rag": {
"command": "haiku-rag",
"args": ["serve", "--mcp", "--stdio"]
}
}
}
```
With a custom database path:
```json
{
"mcpServers": {
"haiku-rag": {
"command": "haiku-rag",
"args": ["serve", "--mcp", "--stdio", "--db", "/path/to/database.lancedb"]
}
}
}
```
After restarting Claude Desktop, you can ask Claude to search your documents, add new content, or answer questions using your knowledge base.
## Available Tools
### Document Management
@ -58,56 +108,6 @@ The MCP server exposes `haiku.rag` as MCP tools for compatible MCP clients like
- `document` (optional): Document title/ID to pre-load (can repeat)
- Best for aggregation, computation, and multi-document analysis
## Starting MCP Server
The MCP server supports Streamable HTTP and stdio transports:
```bash
# Default streamable HTTP transport on port 8001
haiku-rag serve --mcp
# Custom port
haiku-rag serve --mcp --mcp-port 9000
# stdio transport (for Claude Desktop)
haiku-rag serve --mcp --stdio
# Read-only mode (excludes write tools)
haiku-rag --read-only serve --mcp --stdio
```
**Read-only mode:** When `--read-only` is specified, write tools (`add_document_from_file`, `add_document_from_url`, `add_document_from_text`, `delete_document`) are not registered. Only search and query tools remain available.
## Claude Desktop Integration
Add to your Claude Desktop configuration (`claude_desktop_config.json`):
```json
{
"mcpServers": {
"haiku-rag": {
"command": "haiku-rag",
"args": ["serve", "--mcp", "--stdio"]
}
}
}
```
With a custom database path:
```json
{
"mcpServers": {
"haiku-rag": {
"command": "haiku-rag",
"args": ["serve", "--mcp", "--stdio", "--db", "/path/to/database.lancedb"]
}
}
}
```
After restarting Claude Desktop, you can ask Claude to search your documents, add new content, or answer questions using your knowledge base.
## Running with Other Services
Combine MCP with file monitoring:

244
docs/python.md
View file

@ -80,45 +80,6 @@ doc = await client.create_document_from_source(
)
```
### Importing Pre-Processed Documents
If you process documents externally or need custom processing, use `import_document()`:
```python
from haiku.rag.store.models.chunk import Chunk
# Convert your source to a DoclingDocument
docling_doc = await client.convert("path/to/document.pdf")
# Create chunks (embeddings optional - will be generated if missing)
chunks = [
Chunk(
content="This is the first chunk",
metadata={"section": "intro"},
order=0,
),
Chunk(
content="This is the second chunk",
metadata={"section": "body"},
embedding=[0.1] * 1024, # Optional: pre-computed embedding
order=1,
),
]
# Import document with custom chunks
doc = await client.import_document(
docling_document=docling_doc,
chunks=chunks,
uri="doc://custom",
title="Custom Document",
metadata={"source": "external-pipeline"},
)
```
The `docling_document` provides rich metadata for visual grounding, page numbers, and section headings. Content is automatically extracted from the DoclingDocument.
See [Custom Processing Pipelines](custom-pipelines.md) for building pipelines with `convert()`, `chunk()`, and `embed_chunks()`.
### Retrieving Documents
By ID:
@ -198,7 +159,7 @@ await client.update_document(document_id=doc.id, chunks=custom_chunks)
- Updates to only `metadata` or `title` skip re-chunking
- Updates to `content` trigger re-chunking and re-embedding
- Custom `chunks` with embeddings are stored as-is; missing embeddings are generated automatically
- Custom `chunks` with embeddings are stored as-is. Missing embeddings are generated automatically
### Deleting Documents
@ -206,78 +167,6 @@ await client.update_document(document_id=doc.id, chunks=custom_chunks)
await client.delete_document(doc.id)
```
### Rebuilding the Database
```python
from haiku.rag.client import RebuildMode
# Full rebuild (default) - re-converts from source files, re-chunks, re-embeds
async for doc_id in client.rebuild_database():
print(f"Processed document {doc_id}")
# Re-chunk from stored content (no source file access)
async for doc_id in client.rebuild_database(mode=RebuildMode.RECHUNK):
print(f"Processed document {doc_id}")
# Only regenerate embeddings (fastest, keeps existing chunks)
async for doc_id in client.rebuild_database(mode=RebuildMode.EMBED_ONLY):
print(f"Processed document {doc_id}")
# Add VLM picture descriptions to an existing database — runs the VLM
# over already-stored picture bytes, patches descriptions into the
# docling blob, then re-chunks + re-embeds. Requires
# processing.pictures='description' in the config.
async for doc_id in client.rebuild_database(mode=RebuildMode.DESCRIPTIONS):
print(f"Described pictures in {doc_id}")
```
**Rebuild modes:**
- `RebuildMode.FULL` - Re-convert from source files, re-chunk, re-embed (default)
- `RebuildMode.RECHUNK` - Re-chunk from existing document content, re-embed
- `RebuildMode.EMBED_ONLY` - Keep existing chunks, only regenerate embeddings
- `RebuildMode.TITLE_ONLY` - Generate titles for untitled documents (no re-chunking or re-embedding)
- `RebuildMode.DESCRIPTIONS` - Run the VLM over picture bytes already stored on `document_items.picture_data`, patch descriptions into the docling blob, re-chunk + re-embed. Skips the docling parse entirely. Idempotent — pictures already carrying `meta.description.text` are not re-described, so the operation is safe to re-run.
### Generating Titles
Generate a title for an existing document on demand:
```python
title = await client.generate_title(doc)
if title:
await client.update_document(document_id=doc.id, title=title)
```
Uses the same two-tier approach as automatic ingestion: structural extraction from DoclingDocument metadata first, with LLM fallback via `processing.title_model`. Unlike ingestion, this method does not catch exceptions — if the LLM call fails, the error propagates.
To batch-generate titles for all untitled documents, use `RebuildMode.TITLE_ONLY`:
```python
async for doc_id in client.rebuild_database(mode=RebuildMode.TITLE_ONLY):
print(f"Generated title for {doc_id}")
```
See [Automatic Title Generation](configuration/processing.md#automatic-title-generation) for configuration details.
## Maintenance
Run maintenance to optimize storage and prune old table versions:
```python
await client.vacuum()
```
This compacts tables and removes historical versions to keep disk usage in check. It’s safe to run anytime, for example after bulk imports or periodically in long‑running apps.
### Atomic Writes and Rollback
Document create and update operations take a snapshot of table versions before any write and automatically roll back to that snapshot if something fails (for example, during chunking or embedding). This restores both the `documents` and `chunks` tables to their pre‑operation state using LanceDB’s table versioning.
- Applies to: `create_document(...)`, `create_document_from_source(...)`, `update_document(...)`, and internal rebuild/update flows.
- Scope: Both document rows and all associated chunks are rolled back together.
- Vacuum: Running `vacuum()` later prunes old versions for disk efficiency; rollbacks occur immediately during the failing operation and are not impacted.
## Searching Documents
The search method performs native hybrid search (vector + full-text) using LanceDB with optional reranking for improved relevance:
@ -367,7 +256,7 @@ results = await client.search(
### Image queries
`client.search()` accepts an image instead of a text query when the configured embedder is multimodal (e.g. `provider: vllm` against a vision-language embedding model). The image is embedded once and the chunks table is searched vector-only — full-text search and reranking don't apply without a text query.
`client.search()` accepts an image instead of a text query when the configured embedder is multimodal (e.g. `provider: vllm` against a vision-language embedding model). The image is embedded once and the chunks table is searched vector-only. Full-text search and reranking don't apply without a text query.
```python
from PIL import Image
@ -402,7 +291,7 @@ for result in expanded_results:
print(f"Expanded content: {result.content}")
```
Context expansion is automatic and section-aware. For structured documents (with section headers), expansion includes the entire section containing the match. For sections that exceed the budget or are too small (e.g., a title+authors area), expansion grows outward item-by-item from the match center, skipping noise labels (footnotes, page headers) — this naturally crosses into adjacent sections until the budget is filled. For unstructured documents, expansion grows outward item-by-item. Results without `doc_item_refs` (e.g., custom chunks passed to `import_document`) pass through unexpanded.
Context expansion is automatic and section-aware. For structured documents (with section headers), expansion includes the entire section containing the match. For sections that exceed the budget or are too small (e.g., a title+authors area), expansion grows outward item-by-item from the match center, skipping noise labels (footnotes, page headers). This naturally crosses into adjacent sections until the budget is filled. For unstructured documents, expansion grows outward item-by-item. Results without `doc_item_refs` (e.g., custom chunks passed to `import_document`) pass through unexpanded.
Configuration:
@ -430,7 +319,7 @@ answer, citations = await client.ask(
)
```
`client.ask` runs the [rag skill](skills/index.md) under the hood and returns `(answer_text, list[Citation])`. Citations include page numbers, section headings, and document references.
`client.ask` runs the [rag skill](skills/index.md) and returns `(answer_text, list[Citation])`. Citations include page numbers, section headings, and document references.
The QA provider and model are configured in `haiku.rag.yaml` or can be passed directly to the client (see [Configuration](configuration/index.md)).
@ -456,28 +345,121 @@ result = await client.analyze(
`client.analyze` runs the [analysis skill](skills/index.md), which writes and executes Python code in a sandboxed environment to solve problems that traditional RAG struggles with: aggregation, computation, and multi-document analysis.
See [Analysis](agents/analysis.md) for details on capabilities and configuration.
See [Analysis skill](skills/analysis.md) for details on capabilities and configuration.
## Building Custom Agents
## Building custom agents
haiku.rag provides a RAG skill built on [haiku.skills](https://github.com/ggozad/haiku.skills) that bundles all capabilities into a composable agent:
`client.ask` and `client.analyze` are the convenience wrappers. To build your own Pydantic AI agent against the same database, attach the rag and rag-analysis skills directly with `SkillToolset`. See [Skills](skills/index.md) for the full story and worked examples.
For the low-level toolset factories under `haiku.rag.tools` (one rung below the skill abstraction), see [Toolsets](tools.md).
## Importing Pre-Processed Documents
If you process documents externally or need custom processing, use `import_document()`:
```python
from pydantic_ai import Agent
from haiku.rag.skills.rag import create_skill
from haiku.skills.agent import SkillToolset
from haiku.skills.prompts import build_system_prompt
from haiku.rag.store.models.chunk import Chunk
skill = create_skill(db_path=db_path, config=config)
toolset = SkillToolset(skills=[skill])
# Convert your source to a DoclingDocument
docling_doc = await client.convert("path/to/document.pdf")
agent = Agent(
"openai-chat:gpt-4o",
instructions=build_system_prompt(toolset.skill_catalog),
toolsets=[toolset],
# Create chunks (embeddings optional - will be generated if missing)
chunks = [
Chunk(
content="This is the first chunk",
metadata={"section": "intro"},
order=0,
),
Chunk(
content="This is the second chunk",
metadata={"section": "body"},
embedding=[0.1] * 1024, # Optional: pre-computed embedding
order=1,
),
]
# Import document with custom chunks
doc = await client.import_document(
docling_document=docling_doc,
chunks=chunks,
uri="doc://custom",
title="Custom Document",
metadata={"source": "external-pipeline"},
)
result = await agent.run("What are the main findings?")
```
See [Toolsets](tools.md) for the full API reference.
The `docling_document` provides rich metadata for visual grounding, page numbers, and section headings. Content is automatically extracted from the DoclingDocument.
See [Custom Processing Pipelines](custom-pipelines.md) for building pipelines with `convert()`, `chunk()`, and `embed_chunks()`.
## Maintenance
Run maintenance to optimize storage and prune old table versions:
```python
await client.vacuum()
```
This compacts tables and removes historical versions to keep disk usage in check. It’s safe to run anytime, for example after bulk imports or periodically in long‑running apps.
### Rebuilding the Database
```python
from haiku.rag.client import RebuildMode
# Full rebuild (default) - re-converts from source files, re-chunks, re-embeds
async for doc_id in client.rebuild_database():
print(f"Processed document {doc_id}")
# Re-chunk from stored content (no source file access)
async for doc_id in client.rebuild_database(mode=RebuildMode.RECHUNK):
print(f"Processed document {doc_id}")
# Only regenerate embeddings (fastest, keeps existing chunks)
async for doc_id in client.rebuild_database(mode=RebuildMode.EMBED_ONLY):
print(f"Processed document {doc_id}")
# Add VLM picture descriptions to an existing database. Runs the VLM
# over already-stored picture bytes, patches descriptions into the
# docling blob, then re-chunks + re-embeds. Requires
# processing.pictures='description' in the config.
async for doc_id in client.rebuild_database(mode=RebuildMode.DESCRIPTIONS):
print(f"Described pictures in {doc_id}")
```
**Rebuild modes:**
- `RebuildMode.FULL` - Re-convert from source files, re-chunk, re-embed (default)
- `RebuildMode.RECHUNK` - Re-chunk from existing document content, re-embed
- `RebuildMode.EMBED_ONLY` - Keep existing chunks, only regenerate embeddings
- `RebuildMode.TITLE_ONLY` - Generate titles for untitled documents (no re-chunking or re-embedding)
- `RebuildMode.DESCRIPTIONS` - Run the VLM over picture bytes already stored on `document_items.picture_data`, patch descriptions into the docling blob, re-chunk + re-embed. Skips the docling parse entirely. Idempotent: pictures already carrying `meta.description.text` are not re-described, so the operation is safe to re-run.
### Generating Titles
Generate a title for an existing document on demand:
```python
title = await client.generate_title(doc)
if title:
await client.update_document(document_id=doc.id, title=title)
```
Uses the same two-tier approach as automatic ingestion: structural extraction from DoclingDocument metadata first, with LLM fallback via `processing.title_model`. Unlike ingestion, this method does not catch exceptions. If the LLM call fails, the error propagates.
To batch-generate titles for all untitled documents, use `RebuildMode.TITLE_ONLY`:
```python
async for doc_id in client.rebuild_database(mode=RebuildMode.TITLE_ONLY):
print(f"Generated title for {doc_id}")
```
See [Automatic Title Generation](configuration/processing.md#automatic-title-generation) for configuration details.
### Atomic Writes and Rollback
Document create and update operations take a snapshot of table versions before any write and automatically roll back to that snapshot if something fails (for example, during chunking or embedding). This restores both the `documents` and `chunks` tables to their pre‑operation state using LanceDB’s table versioning.
- Applies to: `create_document(...)`, `create_document_from_source(...)`, `update_document(...)`, and internal rebuild/update flows.
- Scope: Both document rows and all associated chunks are rolled back together.
- Vacuum: Running `vacuum()` later prunes old versions for disk efficiency. Rollbacks occur immediately during the failing operation and are not impacted.

8
docs/remote-processing.md
View file

@ -25,14 +25,14 @@ docling-serve is a REST API service that provides:
- Processing large volumes of documents
- Working with complex PDFs requiring OCR
- Running in production environments
- Want to separate compute-intensive tasks
- Need to scale document processing independently
- Separating compute-intensive tasks
- Scaling document processing independently
## Setup
### Docker Compose (Recommended)
The easiest way to use haiku.rag with docling-serve is using the slim Docker image with docker-compose. See `examples/docker/docker-compose.yml` for a complete setup that includes both services.
The slim Docker image with docker-compose is the recommended setup. See `examples/docker/docker-compose.yml` for a complete configuration that includes both services.
### Running docling-serve Manually
@ -136,7 +136,7 @@ processing:
## HTML Image Fetching
docling-serve does **not** fetch external `<img src="https://...">` URLs in HTML inputs. The `ConvertDocumentsOptions` API exposes no equivalent of docling-local's `HTMLBackendOptions.fetch_images` / `enable_remote_fetch`, and the server-side `DoclingConverterManager` registers `format_options` only for PDF and IMAGE — HTML falls through to docling's defaults (`fetch_images=False`).
docling-serve does **not** fetch external `<img src="https://...">` URLs in HTML inputs. The `ConvertDocumentsOptions` API exposes no equivalent of docling-local's `HTMLBackendOptions.fetch_images` / `enable_remote_fetch`, and the server-side `DoclingConverterManager` registers `format_options` only for PDF and IMAGE. HTML falls through to docling's defaults (`fetch_images=False`).
Consequence: ingesting HTML with external image references through docling-serve produces picture items with `picture_data=NULL`. The same input through docling-local fetches the bytes (subject to the SSRF / size / timeout guards documented in [Configuration → External image fetching](configuration/processing.md#external-image-fetching)).

8
docs/server.md
View file

@ -151,7 +151,7 @@ monitor:
allow_http: "true"
```
Then start the server with `--monitor` — the same flag enables both local-directory and S3 watchers:
Then start the server with `--monitor` (the same flag enables both local-directory and S3 watchers):
```bash
haiku-rag serve --monitor
@ -161,12 +161,12 @@ Each entry in `monitor.s3` runs as its own polling task. On every sweep the watc
### Credentials
`storage_options` follows the same convention as `lancedb.storage_options` — the dict is passed straight to obstore (the same Rust `object_store` library LanceDB uses internally), so any keys you've configured there work here too. When `storage_options` is omitted, the watcher falls back to the AWS default credential chain (environment variables, IAM instance role, AWS profile).
`storage_options` follows the same convention as `lancedb.storage_options`. The dict is passed straight to obstore (the same Rust `object_store` library LanceDB uses internally), so any keys you've configured there work here too. When `storage_options` is omitted, the watcher falls back to the AWS default credential chain (environment variables, IAM instance role, AWS profile).
### Orphan deletion scope
`delete_orphans: true` is per-entry: a watcher only removes documents whose URI starts with that entry's `s3://bucket/prefix/`. Documents from other buckets, prefixes, or local-file sources are never touched.
### One-off ingestion
## One-off ingestion
`s3://` URIs are also a first-class source for `haiku-rag add-src` and the MCP `add_document_from_url` tool — see [CLI → Add Documents](cli.md#add-documents).
`s3://` URIs are also a first-class source for `haiku-rag add-src` and the MCP `add_document_from_url` tool. See [CLI → Add Documents](cli.md#add-documents).

191
docs/skills/analysis.md
View file

@ -1,32 +1,58 @@
# Analysis Skill
The analysis skill provides computational analysis via code execution. It writes and runs Python code in a sandboxed interpreter to answer questions that require computation, aggregation, or data traversal.
Plain RAG (search → cite → answer) works for questions whose answer sits in a chunk or two: "Who wrote this?", "What does X say about Y?". It struggles when the answer requires touching the whole corpus, reading a specific section in full, or doing arithmetic on the data.
## `create_skill(db_path?, config?)`
The analysis skill (`rag-analysis`) gives the agent a second tool (`execute_code`) that runs Python in a sandboxed interpreter against a structured view of your documents. The agent can search, read, count, slice, and compare without leaving the tool call. Citations work the same way as the rag skill.
```python
from haiku.rag.skills.analysis import create_skill
`client.analyze`, `haiku-rag analyze`, the MCP `analyze` tool, and the chat TUI (when `-s analysis` is enabled) all run through this skill.
skill = create_skill(db_path=db_path, config=config)
```
## When to use it
| Parameter | Default | Description |
|-----------|---------|-------------|
| `db_path` | `None` | Path to LanceDB database. Falls back to `HAIKU_RAG_DB` env var, then config default. |
| `config` | `None` | `AppConfig` instance. If None, uses `get_config()`. |
Reach for the analysis skill when the question needs more than a search:
- **Aggregation across the corpus.** "How many documents mention security vulnerabilities?"
- **Section-scoped reading.** "Summarize Section 5 of paper Y."
- **Structural comparison.** "Do both papers have an Experimental Results section?"
- **Computation on retrieved data.** "What's the average revenue across these quarterly reports?"
- **Multi-step chains.** Search, filter the results in Python, search again, aggregate, all in one tool call.
For everyday Q&A, the [RAG skill](rag.md) is faster and cheaper. Attach both and the agent routes.
## How it works
Two things make the agent's programs short and the resulting analyses tractable:
1. **Search and document listing are awaitable inside the code.** `await search(query)` returns the same hits the rag skill sees: chunk IDs, text, source metadata, picture refs. The agent can immediately filter, sort, count, or follow up with another search without exiting the tool call.
2. **Every document is mounted as a virtual filesystem at `/documents/{id}/`.** The agent reads four files per document: identifiers and metadata, full text, a list of structured items (paragraphs, tables, figures, headings), and a section tree built from the document's headings. The structure exposes what search alone hides. The agent can navigate from a search hit to the section it lives in, slice a single section instead of pulling the whole document, or scan a document's text directly when keyword precision matters.
A search hit is always a starting point. The agent reads structure around it, drills into the right section, and cites the chunks it actually used. Chunk IDs from search results and chunk IDs surfaced through the VFS are both accepted by `cite`.
### Sandbox guarantees
The interpreter is [pydantic-monty](https://github.com/pydantic/monty), isolated from the host:
- **Virtual filesystem only.** `/documents/` is the entire FS.
- **No network.** HTTP, sockets, and the `requests` family are unavailable.
- **Limited imports.** Only `json`, `re`, `math`, `pathlib`.
- **Execution timeout** (default 60s, configurable via `analysis.code_timeout`).
- **Output truncation** (default 50000 chars, configurable via `analysis.max_output_chars`).
Variables persist between `execute_code` calls within one invocation, so the agent can build state step by step. A fresh sandbox is built per `client.analyze` call.
## Tools
| Tool | Purpose |
|------|---------|
| `search(query, limit?)` | Hybrid search (vector + full-text) with context expansion |
| `list_documents()` | List all documents in the knowledge base |
| `execute_code(code)` | Execute Python code in a sandboxed interpreter with VFS access |
| `cite(chunk_ids)` | Register chunk IDs as citations for the current answer |
| `search(query, limit?)` | Hybrid search with context expansion. Same as the RAG skill's `search`. |
| `execute_code(code)` | Run Python in a sandboxed interpreter with VFS access. |
| `cite(chunk_ids)` | Register chunk IDs as citations. Call before producing the final answer. |
`list_documents` isn't exposed as a top-level tool but is available inside `execute_code` as `await list_documents()`.
## State
The skill manages an `AnalysisState` under the `"analysis"` namespace:
`AnalysisState` lives under the `"analysis"` namespace:
```python
class AnalysisState(BaseModel):
@ -37,32 +63,135 @@ class AnalysisState(BaseModel):
searches: dict[str, list[SearchResult]] = {}
```
- **document_filter** — SQL WHERE clause applied to `search` and `list_documents` calls. Persists across invocations as session-level configuration.
- **executions** — Each `execute_code` call appends a `CodeExecutionEntry` with code, stdout, stderr, and success status. Cleared at the start of each invocation; mirrors the sandbox lifecycle (variables persist across calls within one invocation, a fresh sandbox is built per invocation).
- **citation_index** — Citations indexed by chunk ID. Accumulates across invocations (same semantics as the RAG skill).
- **citations** — Chunk IDs cited during the current invocation. Deduplicated; cleared at the start of each invocation.
- **searches** — Search results from both the `search` tool and sandbox-internal searches. Cleared at the start of each invocation.
- **document_filter** — SQL WHERE clause applied to `search` and the VFS. The LLM can't bypass it: both views are scoped.
- **executions** — Each `execute_code` call appends an entry with code, stdout, stderr, success. Cleared at the start of each invocation.
- **citation_index** — All citations indexed by chunk ID. Accumulates across invocations.
- **citations** — Chunk IDs registered via `cite` during the current invocation. Deduplicated, cleared per-invocation.
- **searches** — Search results from both the `search` tool and sandbox-internal searches. Cleared per-invocation.
## Usage with RAG Skill
## `create_skill(db_path?, config?)`
Combine both skills to give the agent full RAG + analysis capabilities:
```python
from haiku.rag.skills.analysis import create_skill
skill = create_skill(db_path="my.lancedb")
```
| Parameter | Default | Description |
|-----------|---------|-------------|
| `db_path` | `None` | Path to LanceDB database. Falls back to `HAIKU_RAG_DB` env var, then config default. |
| `config` | `None` | `AppConfig` instance. Falls back to `get_config()`. |
## Use it
### From `client.analyze`
```python
from haiku.rag.client import HaikuRAG
async with HaikuRAG("my.lancedb") as client:
result = await client.analyze("How many documents mention 'security'?")
print(result.answer)
for citation in result.citations:
print(citation.uri, citation.title)
```
`client.analyze` runs the skill end-to-end and returns an `AnalysisResult` with `answer` and `citations`. The executed Python programs live on `AnalysisState.executions` during the run, not on the returned result.
### Combine with the RAG skill
```python
from haiku.rag.skills.rag import create_skill as create_rag_skill
from haiku.rag.skills.analysis import create_skill as create_analysis_skill
from haiku.skills.agent import SkillToolset
from haiku.skills.prompts import build_system_prompt
from pydantic_ai import Agent
rag = create_rag_skill(db_path=db_path)
analysis = create_analysis_skill(db_path=db_path)
rag = create_rag_skill(db_path="my.lancedb")
analysis = create_analysis_skill(db_path="my.lancedb")
toolset = SkillToolset(skills=[rag, analysis])
```
agent = Agent(
"openai-chat:gpt-4o",
instructions=build_system_prompt(toolset.skill_catalog),
toolsets=[toolset],
The agent routes Q&A to the rag skill and computational questions to rag-analysis.
## What the agent actually writes
You don't write these programs yourself. The agent does, inside `execute_code`. Seeing the shape helps when you tune prompts, debug a run via `AnalysisState.executions`, or design a custom skill.
**Aggregate across the corpus.** *"How many documents mention security vulnerabilities?"*
```python
hits = await search("security vulnerability", limit=50)
doc_ids = {h['document_id'] for h in hits}
print(f"{len(doc_ids)} documents mention security vulnerabilities")
# Cite the top hit per document
seen = set()
for hit in hits:
if hit['document_id'] not in seen:
seen.add(hit['document_id'])
await cite(hit['chunk_id'])
```
**Read one section in depth.** *"Summarize Section 5."*
```python
from pathlib import Path
import json
doc_id = "..." # from a prior search or list_documents
toc = json.loads(Path(f'/documents/{doc_id}/toc.json').read_text())
section = next(n for n in toc['tree'] if n['title'].startswith('5'))
start, end = section['item_range']
lines = Path(f'/documents/{doc_id}/items.jsonl').read_text().splitlines()[start:end]
for line in lines:
print(json.loads(line)['text'])
await cite(section['chunk_ids'])
```
The section node already aggregates the chunks underneath it, so the agent cites the whole section without a separate search.
**Compare structure across documents.** *"Do both papers have an Experimental Results section?"*
```python
from pathlib import Path
import json
for doc_id in ["doc-a-id", "doc-b-id"]:
toc = json.loads(Path(f'/documents/{doc_id}/toc.json').read_text())
print(f"\n=== {toc['title']} ===")
for node in toc['tree']:
if 'experiment' in node['title'].lower():
print(f" {node['title']} (pages {node['page_numbers']})")
await cite(node['chunk_ids'])
```
## Context filter
The `filter` parameter is enforced at the deps layer. The LLM can't bypass it: both the VFS and search results are scoped to the filter.
```python
result = await client.analyze(
"Summarize all findings",
filter="uri LIKE '%confidential%'"
)
```
See the [Analysis Agent](../agents/analysis.md) documentation for details on how the underlying sandbox works.
Useful for scoping to a corpus subset, enforcing access control, or restricting context.
## Configuration
```yaml
analysis:
model:
provider: anthropic
name: claude-sonnet-4-20250514
code_timeout: 60.0 # Max seconds per code execution
max_output_chars: 50000 # Truncate output after this many chars
```
When `analysis.model` is unset, the skill falls back to `qa.model`.
See [Search and question answering](../configuration/qa.md#analysis-configuration) for the full set.

124
docs/skills/custom.md Normal file
View file

@ -0,0 +1,124 @@
# Custom Skills
The two skills haiku.rag ships work against any LanceDB database. When you want a *domain-specific* skill that bundles its own data, prompt, and tool surface (for example, a "recipes" skill that knows about cooking and ships with a recipes database), generate one with `haiku-rag create-skill`.
The generated package is a regular pip-installable Python package that registers as a `haiku.skills` entry point. Any haiku.skills-aware host (haiku.skills CLI, your own agent, the AG-UI adapter) discovers it automatically.
## When to use a custom skill
- The model should consult a specific knowledge base for a specific kind of question, alongside other skills.
- You want a different instruction prompt than the generic `rag` skill (different tone, refusal style, domain rules).
- You want to ship a knowledge base plus its prompt as one distributable unit.
- You're running multiple skills against different databases in the same agent.
If you just want to point a haiku.rag database at your own model and prompt, configure `haiku.rag.yaml` and use the built-in `rag` skill. No custom package needed.
## Generate
```bash
haiku-rag create-skill \
--name recipes \
--db /path/to/recipes.lancedb \
--tools search,cite \
--description "Recipe and cooking knowledge base" \
--preamble "You are a culinary expert helping with recipes and cooking techniques."
```
Then install and use:
```bash
uv pip install -e ./recipes-skill
haiku-skills list --use-entrypoints
# recipes — Recipe and cooking knowledge base
haiku-skills chat --use-entrypoints --skill recipes
```
### Flags
| Flag | Description | Default |
|------|-------------|---------|
| `--name` | Skill name (lowercase alphanumeric and hyphens). Required. | — |
| `--db` | Path to the LanceDB database to embed. Required. | — |
| `--description` | One-line skill description. The agent reads this to decide when to invoke. | Standard RAG description |
| `--tools` | Comma-separated tool subset, or `all`. | `all` |
| `--preamble` | Custom preamble for the skill's instructions. | Standard RAG preamble |
| `--config-file` | Path to a `haiku.rag.yaml` to embed alongside the database. | None |
| `--output` / `-o` | Output directory. | Current directory |
### Available tools
`cite`, `execute_code`, `get_document`, `list_documents`, `search`.
Drop `execute_code` from `--tools` if the skill shouldn't run sandboxed Python. That gives you a search-and-cite-only skill with no analysis capabilities.
## Anatomy of a generated skill
```
{name}-skill/
├── pyproject.toml
└── {name}_skill/
├── __init__.py # create_skill() entry point
├── SKILL.md # Skill metadata and instructions
└── assets/
├── {name}.lancedb/ # The embedded database
└── haiku.rag.yaml # Optional config (only if --config-file passed)
```
- **`SKILL.md`** carries the instruction prompt the agent will follow. The frontmatter includes the skill name and description. Everything below is the prompt body. Edit this to change behavior.
- **`__init__.py`** exposes `create_skill()` (the entry point) and `visualize_chunk()` for rendering visual grounding.
- **`assets/{name}.lancedb/`** is the database, shipped inside the package.
- **`assets/haiku.rag.yaml`** (optional) pins provider settings the skill needs.
The package can be installed locally with `uv pip install -e .` or published to PyPI.
## Generating visual grounding from a custom skill
Each generated skill exposes a `visualize_chunk()` function that returns the chunk's bounding boxes rendered onto its source page:
```python
from recipes_skill import visualize_chunk
images = await visualize_chunk(chunk_id)
# images is a list of PIL.Image objects, one per page the chunk covers
images[0].save("citation.png")
```
Pass chunk IDs from skill citations or search results. Same prerequisites as elsewhere in haiku.rag: documents need stored page images, and the chunk must come from a PDF or other docling-converted source.
## Multi-skill agents
Each generated skill is self-contained with its own database and instructions. Compose multiple skills in one agent and the model routes between them via their descriptions:
```python
from recipes_skill import create_skill as create_recipes_skill
from medic_skill import create_skill as create_medic_skill
from haiku.skills.agent import SkillToolset
from haiku.skills.prompts import build_system_prompt
from pydantic_ai import Agent
recipes = create_recipes_skill()
medic = create_medic_skill()
toolset = SkillToolset(skills=[recipes, medic])
agent = Agent(
"openai-chat:gpt-4o",
instructions=build_system_prompt(toolset.skill_catalog),
toolsets=[toolset],
)
await agent.run("What's the optimal temperature for braising short ribs?")
# Routes to recipes
await agent.run("What's the field treatment for tension pneumothorax?")
# Routes to medic
```
Each skill maintains state under its own namespace (`recipes`, `medic`, …), so citations and searches don't collide.
## Writing a skill from scratch
`create-skill` is the convenience path. If you need full control over the tools, state model, or instruction loading, write the skill against [haiku.skills](https://github.com/ggozad/haiku.skills) directly. The generated package in `{name}_skill/__init__.py` is a good reference. It composes haiku.rag's `_tools` factory with a `haiku.skills.Skill` and registers under the `haiku.skills` entry point group in `pyproject.toml`.
See the haiku.skills repository for the full Skill contract.

119
docs/skills/index.md
View file

@ -1,25 +1,19 @@
# Skills
haiku.rag exposes its RAG capabilities as [haiku.skills](https://github.com/ggozad/haiku.skills) skills. Skills are self-contained units that bundle tools, instructions, and state — they can be composed into any pydantic-ai agent via `SkillToolset`.
Skills put haiku.rag in front of a model. A skill bundles tools, an instruction prompt, and managed state into a unit that drops into any Pydantic AI agent via `SkillToolset`. haiku.rag ships two skills and supports custom skills.
## Available Skills
Built on [haiku.skills](https://github.com/ggozad/haiku.skills).
| Skill | Description |
|-------|-------------|
| [`rag`](rag.md) | Search, retrieve, and answer questions from the knowledge base |
| [`rag-analysis`](analysis.md) | Computational analysis via code execution |
## Available skills
## Discovery
| Skill | What it does | Reach for it when |
|-------|--------------|-------------------|
| [`rag`](rag.md) | Search, retrieve, and cite content from a knowledge base. | The model needs to find and quote evidence from documents. |
| [`rag-analysis`](analysis.md) | Same as `rag`, plus a sandboxed Python interpreter mounting every document as a virtual filesystem. | The question requires computation, aggregation, structural traversal, or section-scoped reading. |
Skills are registered as Python entrypoints under `haiku.skills`. They are discovered automatically by `haiku.skills`:
To ship your own skill (bundled with its own database), see [Custom skills](custom.md).
```bash
haiku-skills list --use-entrypoints
# rag — Search, retrieve and analyze documents using RAG.
# rag-analysis — Analyze documents using code execution in a sandboxed interpreter.
```
## Usage
## Your first agent
```python
from haiku.rag.skills.rag import create_skill
@ -27,8 +21,8 @@ from haiku.skills.agent import SkillToolset
from haiku.skills.prompts import build_system_prompt
from pydantic_ai import Agent
skill = create_skill(db_path=db_path, config=config)
toolset = SkillToolset(skills=[skill])
rag = create_skill(db_path="my.lancedb")
toolset = SkillToolset(skills=[rag])
agent = Agent(
"openai-chat:gpt-4o",
@ -36,45 +30,37 @@ agent = Agent(
toolsets=[toolset],
)
result = await agent.run("What documents do we have?")
result = await agent.run("What does the knowledge base say about X?")
print(result.output)
```
## Generating Custom Skills
The skill searches, cites, and answers. You supply the model and the question.
Use `create-skill` to generate a standalone skill package with an embedded database:
```bash
haiku-rag create-skill \
--name recipes \
--db /path/to/recipes.lancedb \
--tools search,cite \
--description "Recipe knowledge base" \
--preamble "You are a recipe expert."
```
This generates a pip-installable package (`recipes-skill/`) that bundles the database and registers as a `haiku.skills` entry point. After installing (`uv pip install -e ./recipes-skill`), the skill is automatically discovered:
```bash
haiku-skills list --use-entrypoints
# recipes — Recipe knowledge base
haiku-skills chat --use-entrypoints --skill recipes
```
Since each generated skill is self-contained with its own database and instructions, you can generate multiple skills for different domains and run them together. The agent sees each skill's description and routes questions to the appropriate knowledge base automatically.
Generated skills also expose `visualize_chunk()` for rendering visual grounding. Use chunk IDs from citations or search results in state:
To run analysis against the same database, swap in the `rag-analysis` skill or attach both:
```python
from my_skill import visualize_chunk
from haiku.rag.skills.rag import create_skill as create_rag_skill
from haiku.rag.skills.analysis import create_skill as create_analysis_skill
images = await visualize_chunk(chunk_id)
# Returns list of PIL Images with highlighted bounding boxes
rag = create_rag_skill(db_path="my.lancedb")
analysis = create_analysis_skill(db_path="my.lancedb")
toolset = SkillToolset(skills=[rag, analysis])
```
See [CLI: Create Skill](../cli.md#create-skill) for all options.
The agent reads each skill's description and routes questions itself. See the individual skill pages for the tool surface, state model, and worked examples.
## Database Path Resolution
## State
Each skill manages its own state under a dedicated namespace. State is synced via the AG-UI protocol when using `AGUIAdapter`.
```python
rag_state = toolset.get_namespace("rag")
analysis_state = toolset.get_namespace("analysis")
```
Both state models track citations, the current document filter, and per-turn searches. Analysis state also carries the sandbox execution log. See [RAG skill: state](rag.md#state) and [Analysis skill: state](analysis.md#state).
## Database path resolution
Both skills resolve the database path in the same order:
@ -82,20 +68,9 @@ Both skills resolve the database path in the same order:
2. `HAIKU_RAG_DB` environment variable
3. Config default (`config.storage.data_dir / "haiku.rag.lancedb"`)
## State Management
## AG-UI streaming for web apps
Each skill manages its own state under a dedicated namespace. State is automatically synced via the AG-UI protocol when using `AGUIAdapter`.
```python
rag_state = toolset.get_namespace("rag")
analysis_state = toolset.get_namespace("analysis")
```
See the individual skill pages for state model details.
## AG-UI Streaming
For web applications, use pydantic-ai's `AGUIAdapter` to stream tool calls, text, and state deltas:
For browser apps, use pydantic-ai's `AGUIAdapter` to stream tool calls, text, and state deltas:
```python
from pydantic_ai.ui.ag_ui import AGUIAdapter
@ -105,4 +80,26 @@ event_stream = adapter.run_stream()
sse_event_stream = adapter.encode_stream(event_stream)
```
See the [Web Application](../apps.md#web-application) for a complete implementation.
See the [Web application](../apps.md) reference implementation.
## Exposing via MCP
To use a skill from Claude Desktop or another MCP-aware client, run the MCP server:
```bash
haiku-rag serve --mcp --stdio
```
The server exposes the skill tools (search, ask, analyze) over MCP. See [MCP](../mcp.md).
## Discovery
Skills are registered as Python entry points under `haiku.skills`. They are discovered automatically:
```bash
haiku-skills list --use-entrypoints
# rag — Search, retrieve and analyze documents using RAG.
# rag-analysis — Analyze documents using code execution in a sandboxed interpreter.
```
This is what makes custom skills installable as plain pip packages. See [Custom skills](custom.md).

173
docs/skills/rag.md
View file

@ -1,28 +1,23 @@
# RAG Skill
The RAG skill is the primary way to use haiku.rag tools. It bundles search, document browsing, and citation management into a single skill with managed state.
The `rag` skill answers questions over a knowledge base with hybrid search, structure-aware context expansion, and explicit citations. `client.ask`, `haiku-rag ask`, the MCP `ask_question` tool, and the chat TUI all run through this skill.
## `create_skill(db_path?, config?)`
## When to use it
```python
from haiku.rag.skills.rag import create_skill
- The model needs to find and quote evidence from a document corpus.
- You want citations under every answer.
- You're building a Q&A agent, a documentation chatbot, or any RAG-style integration.
skill = create_skill(db_path=db_path, config=config)
```
| Parameter | Default | Description |
|-----------|---------|-------------|
| `db_path` | `None` | Path to LanceDB database. Falls back to `HAIKU_RAG_DB` env var, then config default. |
| `config` | `None` | `AppConfig` instance. If None, uses `get_config()`. |
If the question requires *computation* over the corpus (counts, aggregates, comparisons, section-scoped reading), reach for the [Analysis skill](analysis.md) instead, or attach both.
## Tools
| Tool | Purpose |
|------|---------|
| `search(query, limit?)` | Hybrid search (vector + full-text) with context expansion |
| `list_documents()` | List all documents in the knowledge base |
| `get_document(query)` | Retrieve a document by ID, title, or URI |
| `cite(chunk_ids)` | Register chunk IDs as citations for the current answer |
| `search(query, limit?)` | Hybrid search (vector + full-text) with section-aware context expansion. Returns `chunk_id`, content, `doc_item_refs`, `picture_refs`, `picture_captions`, source metadata. |
| `list_documents()` | List all documents in the knowledge base. |
| `get_document(query)` | Fetch a document by ID, title, or URI. Partial matches work. |
| `cite(chunk_ids)` | Register chunk IDs as citations for the current answer. The agent calls this before writing the final response. |
## State
@ -36,7 +31,149 @@ class RAGState(BaseModel):
searches: dict[str, list[SearchResult]] = {}
```
- **citation_index** — All citations indexed by chunk ID. Accumulates across invocations so historical turns' chunk IDs remain resolvable in the UI scrollback.
- **citations** — Chunk IDs registered via the `cite` tool during the current invocation. Deduplicated; cleared at the start of each invocation.
- **document_filter** — SQL WHERE clause applied to `search` and `list_documents` calls. Persists across invocations as session-level configuration.
- **citation_index** — All citations indexed by chunk ID. Accumulates across invocations so historical chunk IDs stay resolvable in UI scrollback.
- **citations** — Chunk IDs registered via `cite` during the current invocation. Deduplicated, cleared at the start of each invocation.
- **document_filter** — SQL WHERE clause applied to `search` and `list_documents`. Persists across invocations.
- **searches** — Search results keyed by query string. Cleared at the start of each invocation.
## `create_skill(db_path?, config?)`
```python
from haiku.rag.skills.rag import create_skill
skill = create_skill(db_path="my.lancedb")
```
| Parameter | Default | Description |
|-----------|---------|-------------|
| `db_path` | `None` | Path to LanceDB database. Falls back to `HAIKU_RAG_DB` env var, then config default. |
| `config` | `None` | `AppConfig` instance. Falls back to `get_config()`. |
## Examples
### Minimal agent
```python
from haiku.rag.skills.rag import create_skill
from haiku.skills.agent import SkillToolset
from haiku.skills.prompts import build_system_prompt
from pydantic_ai import Agent
rag = create_skill(db_path="my.lancedb")
toolset = SkillToolset(skills=[rag])
agent = Agent(
"openai-chat:gpt-4o",
instructions=build_system_prompt(toolset.skill_catalog),
toolsets=[toolset],
)
result = await agent.run("What does the manual say about safety procedures?")
print(result.output)
# Inspect what the model cited
state = toolset.get_namespace("rag")
for chunk_id in state.citations:
citation = state.citation_index[chunk_id]
print(f"- {citation.document_title}: {citation.content[:100]}…")
```
### Domain customization
Set a domain preamble in `haiku.rag.yaml` and the skill picks it up:
```yaml
prompts:
domain_preamble: |
The knowledge base contains the operations manual for the Helios solar array.
"The array" or unqualified specs refer to Helios. Terminology like "string"
refers to a series-connected panel chain, not text.
```
To scope a session to a subset of documents, set the filter on the namespace state:
```python
state = toolset.get_namespace("rag")
state.document_filter = "uri LIKE '%helios/v4/%'"
result = await agent.run("What's the maintenance interval for the inverters?")
```
The filter applies to every `search` and `list_documents` call for the rest of the session, including the model can't bypass it from inside.
### Combining with the analysis skill
Attach both skills and the agent routes between them:
```python
from haiku.rag.skills.rag import create_skill as create_rag_skill
from haiku.rag.skills.analysis import create_skill as create_analysis_skill
rag = create_rag_skill(db_path="my.lancedb")
analysis = create_analysis_skill(db_path="my.lancedb")
toolset = SkillToolset(skills=[rag, analysis])
agent = Agent(
"openai-chat:gpt-4o",
instructions=build_system_prompt(toolset.skill_catalog),
toolsets=[toolset],
)
# Q&A → uses rag
await agent.run("What safety equipment is required on-site?")
# Computational question → uses rag-analysis
await agent.run("How many checklists mention torque specifications?")
```
### Streaming to a web frontend
Wrap the agent with `AGUIAdapter` to stream tool calls, text deltas, and state changes to a CopilotKit-style frontend:
```python
from pydantic_ai.ui.ag_ui import AGUIAdapter
adapter = AGUIAdapter(agent=agent, run_input=run_input)
sse_stream = adapter.encode_stream(adapter.run_stream())
```
See the [Web application](../apps.md) reference implementation for the full Starlette + Next.js setup.
### Exposing via MCP
To call the skill from Claude Desktop (or any MCP client), run the MCP server:
```bash
haiku-rag serve --mcp --stdio
```
The exposed `ask_question` tool runs this skill. See [MCP](../mcp.md) for the configuration block.
## Configuration
The skill picks up its model and search behavior from the standard config sections:
```yaml
qa:
model:
provider: ollama
name: gpt-oss
enable_thinking: true
temperature: 0.3
vision: false # set true for vision-capable QA models
max_searches: 3
search:
limit: 5
max_context_chars: 10000
```
See [Search and question answering](../configuration/qa.md) for every knob.
## Vision support
When `qa.model.vision: true` is set, the skill's `search` tool attaches picture bytes to its tool returns as `BinaryContent`. The model can then read figures, diagrams, and screenshots directly alongside the surrounding text. Requires `processing.pictures != none` so the bytes exist on disk. See the [pictures × embedder × QA model matrix](../configuration/processing.md#picture-handling) for the combinations that make sense.
## Customizing the skill prompt
The skill's instruction prompt lives in `SKILL.md` inside the package. For behavior changes (different phrasing, refusal style, additional rules), the supported path is to fork the skill with `haiku-rag create-skill` and edit the generated `SKILL.md`. The `domain_preamble` field above is for *what the corpus is about*, not for *how the agent should behave*. See [Custom skills](custom.md).

4
docs/tools.md
View file

@ -1,8 +1,8 @@
# Toolsets
haiku.rag exposes its RAG capabilities as [haiku.skills](https://github.com/ggozad/haiku.skills) skills. See the [Skills](skills/index.md) section for the primary way to use haiku.rag tools.
haiku.rag exposes its RAG capabilities as [haiku.skills](https://github.com/ggozad/haiku.skills) skills. For most integrations, see [Skills](skills/index.md).
For lower-level access, `haiku.rag.tools` provides individual `FunctionToolset` factories used internally by agents.
For lower-level access, `haiku.rag.tools` provides individual `FunctionToolset` factories that the skills themselves compose.
## Low-Level Toolsets

74
docs/tuning.md
View file

@ -4,13 +4,13 @@ How to adjust haiku.rag's pipeline for better retrieval and answer quality. For
## Pipeline Overview
Documents flow through: **chunking → embedding → hybrid search (vector + FTS) → reranking → context expansion → LLM generation**. Retrieval tuning (chunking through reranking) is highest-leverage — if the LLM never sees the right chunks, no prompt or model change will help.
Documents flow through: **chunking → embedding → hybrid search (vector + FTS) → reranking → context expansion → LLM generation**. Retrieval tuning (chunking through reranking) is the highest-leverage stage. If the LLM never sees the right chunks, no prompt or model change will help.
## Tuning Retrieval
### Chunking
`chunk_size` controls the granularity of retrieval. Smaller chunks match queries more precisely but carry less context each; larger chunks provide more surrounding information but dilute relevance signals. On the Wix benchmark, increasing from 256 to 512 tokens raised MAP from 0.43 to 0.45 on plain text — a modest gain that also increases token cost per result. See [Processing](configuration/processing.md#chunk-size) for configuration.
`chunk_size` controls the granularity of retrieval. Smaller chunks match queries more precisely but carry less context each. Larger chunks provide more surrounding information but dilute relevance signals. On the Wix benchmark, increasing from 256 to 512 tokens raised MAP from 0.43 to 0.45 on plain text, a modest gain that also increases token cost per result. See [Processing](configuration/processing.md#chunk-size) for configuration.
`chunker_type` selects between `hybrid` (default) and `hierarchical` chunking. Hierarchical chunking preserves the document's heading structure and works better for deeply nested or structured content. See [Chunking Strategies](configuration/processing.md#chunking-strategies).
@ -20,17 +20,17 @@ Larger embedding models produce better representations at the cost of slower ind
### Reranking
When configured, a cross-encoder reranker re-scores 10x the requested candidates and returns the top results. This adds latency but improves precision — on the Wix benchmark, adding `mxbai-rerank-base-v2` raised MAP from 0.34 to 0.39 on HTML content. See [Search Settings](configuration/qa.md#search-settings) for how reranking integrates with search.
When configured, a cross-encoder reranker re-scores 10x the requested candidates and returns the top results. This adds latency but improves precision. On the Wix benchmark, adding `mxbai-rerank-base-v2` raised MAP from 0.34 to 0.39 on HTML content. See [Search Settings](configuration/qa.md#search-settings) for how reranking integrates with search.
### Search Settings
`limit` controls how many results reach the LLM. More candidates improve recall but increase token usage. See [Search Settings](configuration/qa.md#search-settings).
Context expansion is automatic and section-aware — search results are expanded to include surrounding content from the same document section. For structured documents, expansion stays within section boundaries and filters noise (footnotes, page headers). For unstructured documents, expansion grows outward until the character budget is filled. `max_context_chars` caps expansion to prevent context bloat.
Context expansion is automatic and section-aware. Search results are expanded to include surrounding content from the same document section. For structured documents, expansion stays within section boundaries and filters noise (footnotes, page headers). For unstructured documents, expansion grows outward until the character budget is filled. `max_context_chars` caps expansion to prevent context bloat.
## Tuning Generation
Model and temperature selection affect answer quality directly — see [Providers](configuration/providers.md#model-settings) for options.
Model and temperature selection affect answer quality directly. See [Providers](configuration/providers.md#model-settings) for options.
`domain_preamble` prepends domain context to the rag and rag-analysis skill instructions. Use it to describe what the knowledge base contains and clarify domain-specific terminology. See [Prompt Customization](configuration/prompts.md).
@ -38,18 +38,74 @@ Model and temperature selection affect answer quality directly — see [Provider
| Change | Rebuild required? |
|--------|:-:|
| `chunk_size`, `chunker_type`, `chunking_merge_peers` | Yes — `haiku-rag rebuild` |
| Embedding model | Yes — `haiku-rag rebuild` |
| `chunk_size`, `chunker_type`, `chunking_merge_peers` | Yes (run `haiku-rag rebuild`) |
| Embedding model | Yes (run `haiku-rag rebuild`) |
| Search settings, reranking, prompts | No |
## Measuring Changes
## Inspector
Use the inspector for ad-hoc exploration:
The inspector shows what your model would actually receive for a given query. Run it against your database and step through the same hybrid search, context expansion, and chunk previews the rag skill uses at runtime. Press `c` on a chunk and you see the exact context the LLM would get back from a search hit.
```bash
haiku-rag inspect
haiku-rag inspect --db /path/to/database.lancedb
```
!!! note
Requires the `tui` extra: `pip install haiku.rag-slim[tui]` (included in the full `haiku.rag` package).
### Layout
Three panels:
- **Documents** (left): every document in the database.
- **Chunks** (top right): chunks for the selected document.
- **Detail view** (bottom right): full content and metadata.
![Inspector search](img/inspector-search.svg)
### Keys
| Key | Action |
|-----|--------|
| `Tab` | Cycle panels |
| `↑` / `↓` | Navigate lists |
| `/` | Search modal |
| `c` | Context expansion modal (the chunk plus what the agent would see around it) |
| `v` | Visual grounding modal (chunk highlighted on the page) |
| `q` | Quit |
Mouse: click to select, scroll to view content.
### Search
Press `/` to open the search modal. Type a query and press `Enter`. The left panel lists results with relevance scores like `[0.95] content preview`. The right panel shows the full chunk and its metadata. `↑` / `↓` navigates results, `Enter` jumps to the document and chunk, `Esc` closes the modal. Search uses the same hybrid (vector + full-text) retrieval the rag skill uses.
### Context expansion (`c`)
Press `c` on a chunk to see the expanded context that would be fed to the rag skill. This is where you find out whether your `chunk_size`, `chunker_type`, and `max_context_chars` settings actually deliver the surrounding content the model needs. The modal shows:
- The expanded text. Section-aware expansion stays within section boundaries on structured documents and fills `max_context_chars` outward on unstructured ones.
- Source document, content type, and relevance score.
- Filtered noise. Footnotes, page headers and footers are excluded from structured documents.
If `qa.model.vision = true` is set, the modal also renders the picture bytes attached to that chunk, so you see exactly what the vision model would receive.
### Visual grounding (`v`)
Press `v` to highlight the chunk's bounding box on its page image. Useful for verifying chunk boundaries and seeing how Docling carved up the document.
- `←` / `→` to navigate pages when a chunk spans multiple pages.
- `Esc` closes the modal.
![Visual grounding modal](img/tui-visual-grounding.png)
Requirements: documents must have page images (default for PDFs), and the terminal must support inline images (iTerm2, WezTerm, Kitty). Plain-text documents added via `haiku-rag add` don't have visual grounding.
You can also visualize a chunk from the CLI without launching the TUI: `haiku-rag visualize <chunk_id>`.
## Measuring Changes
For systematic measurement, use the `evaluations/` workspace which provides retrieval metrics (MRR, MAP) and LLM-judged QA accuracy via `pydantic-evals`:
```bash

211
docs/tutorial.md
View file

@ -1,211 +1,86 @@
# Tutorial
# Quickstart
This tutorial provides quickstart instructions for getting familiar with `haiku.rag`. This tutorial is intended for people who are familiar with command line and Python, but not different AI ecosystem tools.
Install haiku.rag, index a document, and chat with it.
The tutorial covers:
- RAG and embeddings basics
- Installing `haiku.rag` Python package
- Configuring `haiku.rag` with YAML
- Adding and retrieving items
- Inspecting the database
The tutorial uses OpenAI API service - no local installation needed and will work on computers with any amount of RAM and GPU. The OpenAI API is pay-as-you-go, so you need to top it up with at least ~$5 when creating the API key.
## Introduction
Retrieval-Augmented Generation (RAG) lets you give AI models access to your own documents and data. Instead of relying solely on the model's training data, RAG finds relevant information from your documents and includes it in the AI's responses.
`haiku.rag` handles the mechanics: it converts your documents into searchable embeddings, stores them locally, and retrieves relevant chunks when you ask questions. You provide the documents and questions, and it coordinates between the embedding service (like OpenAI) and the AI model to give you accurate, grounded answers.
## Setup
First, [get an OpenAI API key](https://platform.openai.com/api-keys).
Install `haiku.rag` Python package using [uv](https://docs.astral.sh/uv/getting-started/installation/) or your favourite Python package manager:
## Install
```bash
# Python 3.12+ needed
uv pip install haiku.rag
```
Configure haiku.rag to use OpenAI. Create a `haiku.rag.yaml` file:
```yaml
embeddings:
model:
provider: openai
name: text-embedding-3-small # or text-embedding-3-large
vector_dim: 1536
qa:
model:
provider: openai
name: gpt-4o-mini # or gpt-4o, gpt-4, etc.
```
Set your OpenAI API key as an environment variable (API keys should not be stored in the YAML file):
You also need [Ollama](https://ollama.com/) for the default embedding and answering models:
```bash
export OPENAI_API_KEY="<your OpenAI API key>"
ollama pull qwen3-embedding:4b
ollama pull gpt-oss
```
For the list of available OpenAI models and their vector dimensions, see the [OpenAI documentation](https://platform.openai.com/docs/guides/embeddings).
!!! note "Prefer OpenAI?"
Drop this into a `haiku.rag.yaml` next to where you'll run the CLI:
See [Configuration](configuration/index.md) for all available options.
```yaml
embeddings:
model:
provider: openai
name: text-embedding-3-small
vector_dim: 1536
## Initialize the database
qa:
model:
provider: openai
name: gpt-4o-mini
```
Before adding documents, initialize the database:
Then `export OPENAI_API_KEY="sk-..."` and continue with the rest of this page. Any provider Pydantic AI supports works the same way. See [Providers](configuration/providers.md).
## Initialize
```bash
haiku-rag init
```
This creates an empty database with the configured settings.
## Adding the first documents
Now you can add some pieces of text in the database:
This creates a LanceDB database in your platform's user directory. Pass `--db` to any subcommand to use a different path:
```bash
haiku-rag add "Python is the best programming language in the world, because it is flexible, with robust ecosystem, open source licensing and thousands of contributors"
haiku-rag add "JavaScript is a popular programming language, but has a lot of warts"
haiku-rag add "PHP is a bad programming language, because of spotted security history, horrible syntax and declining popularity"
haiku-rag init --db /tmp/test.lancedb
```
What will happen:
## Add a document
- The piece of text is sent to OpenAI `/embeddings` API service
- OpenAI translates the free form text to RAG embedding vectors needed for the retrieval
- The vector values will be stored in a local database
Now you can view your [LanceDB](https://lancedb.com/) database, and the embeddings it is configured for:
Add a file, a URL, or a whole folder:
```bash
haiku-rag info
haiku-rag add-src https://arxiv.org/pdf/2408.09134
haiku-rag add-src ~/Documents/papers/
```
You should see output similar to:
```
haiku.rag database info
path: /Users/moo/Library/Application Support/haiku.rag/haiku.rag.lancedb
haiku.rag version (db): x.y.z
embeddings: openai/text-embedding-3-small (dim: 1536)
documents: 3 (storage: 48.0 KB)
chunks: 3 (storage: 52.0 KB)
vector index: not created
──────────────────────────────────────────────────────────────────────────────────
Versions
haiku.rag: x.y.z
lancedb: ...
docling: ...
```
## Asking questions and retrieving information
Now we can use OpenAI LLMs to retrieve information from our embeddings database.
In this example, we connect to a remote OpenAI API.
Behind the scenes [pydantic-ai](https://ai.pydantic.dev/) query is created
using `OpenAIChatModel.request()`.
The easiest way to do this is `ask` CLI command:
Or paste text inline:
```bash
haiku-rag ask "What is the best programming language in the world"
haiku-rag add "Yiorgis wrote haiku.rag in 2025."
```
```
Question: What is the best programming language in the world
Each `add-src` call converts the file with Docling, splits it into chunks, embeds them, and writes everything to LanceDB. Run `haiku-rag list` to see what you've added, `haiku-rag info` for a database summary.
Answer:
According to the document, Python is considered the best programming language in the world due to its flexibility, robust ecosystem, open-source licensing, and thousands of contributors.
```
## Programmatic interaction in Python
You can interact with haiku.rag from Python. Since the API is async, we'll use IPython which supports async/await directly.
## Chat
```bash
uv pip install ipython
ipython
haiku-rag chat
```
Then run:
Ask a question. The agent searches your documents, expands context around the hits, and answers with citations pointing back to the source page and section. Citations are expandable, with visual grounding so you can see the chunk highlighted on the original page. Follow-ups continue within the same session. Start a new session when you switch topics.
```python
from haiku.rag.client import HaikuRAG
# Uses database from default location (must be initialized first)
async with HaikuRAG() as client:
answer, citations = await client.ask("What is the best programming language in the world?")
print(answer)
```
You should see:
```
According to the document, Python is considered the best programming language in the world due to its flexibility, robust ecosystem, open-source licensing, and support from thousands of contributors.
```
## Complex documents
Haiku RAG can also handle types beyond plain text, including PDF, DOCX, HTML, and 40+ other file formats.
Here we add research papers about Python from [arxiv](https://arxiv.org/search/?query=python&searchtype=all&source=header) using URL retriever.
You can also ask a single question directly from the CLI without launching the TUI:
```bash
# Better Python Programming for all: With the focus on Maintainability
haiku-rag add-src --meta collection="Interesting Python papers" "https://arxiv.org/pdf/2408.09134"
# Interoperability From OpenTelemetry to Kieker: Demonstrated as Export from the Astronomy Shop
haiku-rag add-src --meta collection="Interesting Python papers" "https://arxiv.org/pdf/2510.11179"
haiku-rag ask "Who wrote haiku.rag?"
```
Then we can query this:
## Where to go next
```bash
haiku-rag ask "Who wrote a paper about OpenTelemetry interoperability, and what was his take"
```
We should get something along the lines:
```
Answer:
David Georg Reichelt from Lancaster University wrote a paper titled "Interoperability From OpenTelemetry to Kieker: Demonstrated as Export from the Astronomy Shop." In his work, he indicates that there is a structural difference between Kieker’s synchronous traces and OpenTelemetry’s asynchronous traces, leading to limited compatibility between the two systems. This highlights the challenges of interoperability in observability frameworks.
```
We can also add offline files, like PDFs. Here we add a local file to ensure OpenAI does not cheat - a file we know that should not be very well known in Internet:
```bash
# This static file is supplied in haiku.rag repo
haiku-rag add-src "examples/samples/PyCon Finland 2025 Schedule.html"
```
And then:
```bash
haiku-rag ask "Who were presenting talks in Pycon Finland 2025? Can you give at least five different people."
```
```
The following people are presenting talks at PyCon Finland 2025:
1 Jeremy Mayeres - Talk: The Limits of Imagination: An Open Source Journey
2 Aroma Rodrigues - Talk: Python and Rust, a Perfect Pairing
3 Andreas Jung - Talk: Guillotina Volto: A New Backend for Volto
4 Daniel Vahla - Talk: Experiences with AI in Software Projects
5 Andreas Jung (also presenting another talk) - Talk: Debugging Python
```
## Next Steps
- **[Chat](apps.md#chat-tui)** - Interactive conversations with `haiku-rag chat`
- **[CLI Reference](cli.md)** - All available commands and options
- **[Python API](python.md)** - Use haiku.rag in your Python applications
- **[Skills](skills/index.md)** - The RAG and analysis skills the client wraps
- **[Configuration](configuration/index.md)** - Complete YAML configuration reference
- **[Server Mode](server.md)** - File monitoring and MCP server
- [Chat](chat.md): sessions, citations, and the full TUI.
- [CLI reference](cli.md): every command.
- [Python API](python.md): use haiku.rag in your own code.
- [Skills](skills/index.md): the rag and rag-analysis skills the client wraps.
- [Tuning](tuning.md): better retrieval.
- [Configuration](configuration/index.md): every setting.

45
mkdocs.yml
View file

@ -55,31 +55,36 @@ plugins:
# Material for MkDocs
search:
nav:
- haiku.rag:
- Get started:
- index.md
- Getting started: tutorial.md
- Quickstart: tutorial.md
- Installation: installation.md
- Configuration:
- configuration/index.md
- Providers: configuration/providers.md
- Search and Question Answering: configuration/qa.md
- Document Processing: configuration/processing.md
- Storage: configuration/storage.md
- Prompts: configuration/prompts.md
- Use it:
- CLI: cli.md
- Python: python.md
- Custom Pipelines: custom-pipelines.md
- Chat: chat.md
- Skills:
- skills/index.md
- RAG skill: skills/rag.md
- Analysis skill: skills/analysis.md
- Custom skills: skills/custom.md
- Configure:
- configuration/index.md
- Providers: configuration/providers.md
- Search and question answering: configuration/qa.md
- Document processing: configuration/processing.md
- Storage: configuration/storage.md
- Prompts: configuration/prompts.md
- Tuning: tuning.md
- Analysis: agents/analysis.md
- Skills:
- skills/index.md
- RAG: skills/rag.md
- Analysis: skills/analysis.md
- Toolsets: tools.md
- Applications: apps.md
- Production:
- Server: server.md
- Remote processing: remote-processing.md
- MCP: mcp.md
- Remote processing: remote-processing.md
- Develop:
- Python: python.md
- Custom pipelines: custom-pipelines.md
- Toolsets: tools.md
- Web app: apps.md
- Reference:
- Benchmarks: benchmarks.md
- Development: development.md
- Changelog: changelog.md
@ -94,7 +99,7 @@ markdown_extensions:
use_pygments: true
- pymdownx.inlinehilite
- pymdownx.snippets:
base_path: ['.']
base_path: ["."]
- pymdownx.superfences:
custom_fences:
- name: mermaid