diff --git a/docs/configuration/index.md b/docs/configuration/index.md index b2063844..9ab1f0c6 100644 --- a/docs/configuration/index.md +++ b/docs/configuration/index.md @@ -101,7 +101,7 @@ research: search: limit: 5 # Default number of results to return context_radius: 0 # DocItems before/after to include for text content - max_context_items: 25 # Maximum items in expanded context + max_context_items: 10 # Maximum items in expanded context max_context_chars: 10000 # Maximum characters in expanded context vector_index_metric: cosine # cosine, l2, or dot vector_refine_factor: 30 @@ -191,7 +191,7 @@ This is useful for: For detailed configuration of specific topics, see: -- **[Providers](providers.md)** - Model settings and provider-specific configuration (embeddings, QA, reranking) -- **[QA and Research](qa-research.md)** - Question answering and research workflow configuration -- **[Storage](storage.md)** - Database, remote storage, and vector indexing +- **[Providers](providers.md)** - Model settings and provider-specific configuration (embeddings, reranking) +- **[Search and Question Answering](qa-research.md)** - Search settings, question answering, and research workflows - **[Document Processing](processing.md)** - Document conversion, chunking, and file monitoring +- **[Storage](storage.md)** - Database, remote storage, and vector indexing diff --git a/docs/configuration/processing.md b/docs/configuration/processing.md index a4b43895..f57b23d7 100644 --- a/docs/configuration/processing.md +++ b/docs/configuration/processing.md @@ -139,7 +139,7 @@ processing: chunk_size: 256 # Maximum tokens per chunk ``` -Context expansion settings (for enriching search results with surrounding content) are configured in the `search` section. See [Search Settings](storage.md#search-settings). +Context expansion settings (for enriching search results with surrounding content) are configured in the `search` section. See [Search Settings](qa-research.md#search-settings). ## File Monitoring diff --git a/docs/configuration/qa-research.md b/docs/configuration/qa-research.md index 4d3f2e0a..611afa53 100644 --- a/docs/configuration/qa-research.md +++ b/docs/configuration/qa-research.md @@ -1,4 +1,26 @@ -# QA and Research Configuration +# Search and Question Answering + +## Search Settings + +Configure search behavior and context expansion: + +```yaml +search: + limit: 5 # Default number of results to return + context_radius: 0 # DocItems before/after to include for text content + max_context_items: 10 # Maximum items in expanded context + max_context_chars: 10000 # Maximum characters in expanded context +``` + +- **limit**: Default number of search results to return when no limit is specified. Used by CLI, MCP server, QA, and research workflows. Default: 5 +- **context_radius**: For text content (paragraphs), includes N DocItems before and after. Set to 0 to disable expansion (default). +- **max_context_items**: Limits how many document items (paragraphs, list items, etc.) can be included in expanded context. Default: 10. +- **max_context_chars**: Hard limit on total characters in expanded content. Default: 10000. + +Structural content (tables, code blocks, lists) uses type-aware expansion that automatically includes the complete structure regardless of how it was chunked. + +!!! note "Reranking behavior" + When a reranker is configured, search automatically retrieves 10x the requested limit, then reranks to return the final count. This improves result quality without requiring you to adjust `limit`. ## Question Answering Configuration @@ -43,29 +65,3 @@ research: - **max_concurrency**: Sub-questions searched in parallel per iteration (default: 1) The research workflow plans sub-questions, searches in parallel batches, evaluates findings, and iterates until reaching the confidence threshold or max iterations. - -## AG-UI Server Configuration - -Configure the AG-UI HTTP server for streaming graph execution events: - -```yaml -agui: - host: "0.0.0.0" - port: 8000 - cors_origins: ["*"] - cors_credentials: true - cors_methods: ["GET", "POST", "OPTIONS"] - cors_headers: ["*"] -``` - -Start the AG-UI server with: - -```bash -haiku-rag serve --agui -``` - -The server exposes: -- `GET /health` - Health check endpoint -- `POST /v1/agent/stream` - Research graph streaming endpoint (Server-Sent Events) - -See [Server Mode](../server.md) for more details. diff --git a/docs/configuration/storage.md b/docs/configuration/storage.md index 97a10c8f..54bb6d67 100644 --- a/docs/configuration/storage.md +++ b/docs/configuration/storage.md @@ -57,33 +57,17 @@ haiku.rag intelligently handles database creation based on operation type: This prevents the common mistake where a search query accidentally creates an empty database. To initialize your database, simply add your first document using `haiku-rag add` or `haiku-rag add-src`. -## Search Settings +## Vector Indexing -Configure search behavior and context expansion: +Configure vector search settings: ```yaml search: - limit: 5 # Default number of results to return - context_radius: 0 # DocItems before/after to include for text content - max_context_items: 25 # Maximum items in expanded context - max_context_chars: 10000 # Maximum characters in expanded context vector_index_metric: cosine # cosine, l2, or dot vector_refine_factor: 30 # Re-ranking factor for accuracy ``` -- **limit**: Default number of search results to return when no limit is specified. Used by CLI, MCP server, and API. Default: 5 - -### Context Expansion - -Context expansion enriches search results with surrounding content from the source document: - -- **context_radius**: For text content (paragraphs), includes N DocItems before and after. Set to 0 to disable expansion (default). -- **max_context_items**: Limits how many document items (paragraphs, list items, etc.) can be included in expanded context. -- **max_context_chars**: Hard limit on total characters in expanded content. - -Structural content (tables, code blocks, lists) uses type-aware expansion that automatically includes the complete structure regardless of how it was chunked. For example, if a table was split across multiple chunks, expansion retrieves the complete table. - -### Vector Indexing +For search behavior settings (`limit`, `context_radius`, `max_context_items`, `max_context_chars`), see [QA and Research](qa-research.md#search-settings). - **vector_index_metric**: Distance metric for vector similarity: - `cosine`: Cosine similarity (default, best for most embeddings) diff --git a/docs/server.md b/docs/server.md index d08f7b3a..f312620d 100644 --- a/docs/server.md +++ b/docs/server.md @@ -143,9 +143,16 @@ agui: port: 8000 cors_origins: ["*"] cors_credentials: true + cors_methods: ["GET", "POST", "OPTIONS"] + cors_headers: ["*"] ``` -See [Configuration](configuration/qa-research.md#ag-ui-server-configuration) for all available options. +- **host**: Bind address (default: `0.0.0.0`) +- **port**: Server port (default: `8000`) +- **cors_origins**: Allowed CORS origins (default: `["*"]`) +- **cors_credentials**: Allow credentials in CORS requests (default: `true`) +- **cors_methods**: Allowed HTTP methods (default: `["GET", "POST", "OPTIONS"]`) +- **cors_headers**: Allowed headers (default: `["*"]`) ### Using the Streaming Endpoints diff --git a/docs/tuning.md b/docs/tuning.md new file mode 100644 index 00000000..507aeb9b --- /dev/null +++ b/docs/tuning.md @@ -0,0 +1,320 @@ +# Tuning haiku.rag for Your Corpus + +This guide explains how to tune haiku.rag settings based on your document corpus characteristics. The right settings depend on your document types, query patterns, and accuracy requirements. + +## Key Concepts + +### Retrieval vs Generation + +RAG has two phases: + +1. **Retrieval**: Finding relevant chunks from your corpus +2. **Generation**: Using those chunks to answer questions + +Poor retrieval means the LLM never sees the relevant content, regardless of how good the model is. Tuning retrieval is usually more impactful than tuning generation. + +### Recall vs Precision + +- **Recall**: What fraction of relevant documents did we find? +- **Precision**: What fraction of retrieved documents are relevant? + +For RAG, recall matters more than precision. Missing a relevant chunk means wrong answers. Including an extra irrelevant chunk just wastes context tokens. + +## Search Settings + +### `search.limit` + +Default number of chunks to retrieve. + +```yaml +search: + limit: 5 # Default +``` + +**When to increase:** + +- Complex questions requiring information from multiple sources +- Broad topics spread across many documents + +**When to decrease:** + +- Simple factual questions +- Highly focused corpus where top results are usually correct +- Cost-sensitive deployments (fewer chunks = fewer tokens) + +**Typical values:** 3-10 + +### `search.context_radius` + +Number of adjacent DocItems to include when expanding search results. Only applies to text content (paragraphs). Tables, code blocks, and lists use structural expansion automatically. + +```yaml +search: + context_radius: 0 # Default: no expansion +``` + +**When to increase:** + +- Answers require surrounding context (definitions, explanations) +- Chunks are small and queries need more context +- Documents have strong local coherence (adjacent paragraphs relate) + +**When to keep at 0:** + +- Large chunks that already contain sufficient context +- Documents where adjacent content is often unrelated +- When chunk boundaries align well with semantic units + +**Typical values:** 0-3 + +### `search.max_context_items` and `search.max_context_chars` + +Safety limits on context expansion to prevent runaway expansion. + +```yaml +search: + max_context_items: 10 # Max DocItems per expanded result + max_context_chars: 10000 # Max characters per expanded result +``` + +Increase if expansion is being truncated and you need more context. Decrease if expanded results are too long for your LLM context window. + +## Processing Settings + +### `processing.chunk_size` + +Maximum tokens per chunk (using the configured tokenizer). + +```yaml +processing: + chunk_size: 256 # Default +``` + +**Trade-offs:** + +| Smaller chunks (128-256) | Larger chunks (512-1024) | +|-------------------------|-------------------------| +| More precise retrieval | Better context per chunk | +| May miss spanning content | Better recall | +| More chunks to search | Faster search | +| Better for specific queries | Better for broad queries | + +**Guidance by corpus type:** + +- **Technical documentation**: 256-512 (specific lookups) +- **Long-form articles**: 512-1024 (need context) +- **FAQs/short answers**: 128-256 (discrete answers) +- **Code documentation**: 256-512 (function-level) + +### `processing.chunker_type` + +Chunking strategy. + +```yaml +processing: + chunker_type: hybrid # Default +``` + +- **`hybrid`**: Structure-aware with token limits. Best for most documents. +- **`hierarchical`**: Preserves document hierarchy strictly. Use for highly structured documents where hierarchy matters. + + +### `processing.chunking_merge_peers` + +Whether to merge adjacent small chunks that share the same section. + +```yaml +processing: + chunking_merge_peers: true # Default +``` + +Keep `true` unless you specifically want very granular chunks. Merging improves embedding quality by ensuring chunks have sufficient context. + +## Embedding Settings + +### Model Selection + +Embedding model choice significantly impacts retrieval quality. + +```yaml +embeddings: + model: + provider: ollama + name: qwen3-embedding:4b + vector_dim: 2560 +``` + +**Considerations:** + +- Larger models generally produce better embeddings but are slower +- Match `vector_dim` to your model's actual output dimension +- Local models (Ollama) vs API models (OpenAI, VoyageAI) trade-off cost vs quality + +### Contextualizing Embeddings + +Chunks are embedded with section headings prepended (via `contextualize()`). This improves retrieval by including structural context in the embedding. + +If your documents lack clear headings, embeddings will be based on chunk content alone. + +## Reranking + +Reranking retrieves more candidates than needed, then uses a cross-encoder to re-score them. + +```yaml +reranking: + model: + provider: mxbai # or cohere, zeroentropy, vllm + name: mxbai-rerank-base-v1 +``` + +**When to use reranking:** + +- Embedding model has limited accuracy +- Queries are complex or ambiguous +- You can afford the latency (adds ~100-500ms) + +**When to skip reranking:** + +- Simple, specific queries +- High-quality embedding model +- Latency-sensitive applications + +When reranking is enabled, haiku.rag automatically retrieves 10x the requested limit, then reranks to the final count. You don't need to adjust `search.limit` for reranking. + +## Tuning Workflow + +### 1. Use the Inspector + +The inspector is your best tool for understanding how your corpus is chunked and how search behaves: + +```bash +haiku-rag inspect +``` + +**What to look for:** + +- Browse documents and their chunks to see how content is split +- Use the search modal (`/`) to test queries and see which chunks are retrieved +- Press `c` on a chunk to view expanded context - see what additional content would be included with `context_radius > 0` +- Check chunk sizes - are they too small (fragmented) or too large (unfocused)? + +### 2. Test Search Manually + +Before changing settings, run searches from the CLI to understand current behavior: + +```bash +# Search and see results +haiku-rag search "your test query" --limit 10 + +# Try the QA to see end-to-end behavior +haiku-rag ask "your question" +``` + +### 3. Identify the Bottleneck + +- **Relevant chunks not retrieved**: Try larger `search.limit`, smaller `chunk_size`, or a different embedding model +- **Too many irrelevant chunks**: Try reranking or larger `chunk_size` +- **Chunks found but answers wrong**: Try `context_radius` expansion or a better QA model + +### 4. Test One Change at a Time + +```bash +# After changing chunk_size, rebuild is required +haiku-rag rebuild + +# After changing search settings, no rebuild needed - just test again +haiku-rag search "your test query" +``` + +### 5. Build Dataset-Specific Evaluations + +For systematic tuning, create evaluations specific to your corpus. See the `evaluations/` directory in the repository for examples of how to: + +- Define test cases with questions and expected answers +- Run retrieval benchmarks (MRR, MAP) +- Run QA accuracy benchmarks with LLM judges + +Custom evaluations let you measure the impact of configuration changes objectively rather than relying on intuition. + +### 6. Consider Your Corpus + +| Corpus Type | Suggested Starting Point | +|-------------|-------------------------| +| Technical docs | `chunk_size: 256`, `limit: 10`, `context_radius: 1` | +| Legal/contracts | `chunk_size: 512`, `limit: 5`, `context_radius: 2` | +| News articles | `chunk_size: 512`, `limit: 5`, `context_radius: 0` | +| Scientific papers | `chunk_size: 256`, `limit: 5`, reranking enabled | +| FAQs | `chunk_size: 128`, `limit: 5`, `context_radius: 0` | +| Code repos | `chunk_size: 256`, `limit: 10`, `context_radius: 1` | + +## Common Issues + +### "Relevant content not being retrieved" + +1. Check chunk boundaries - is the content split awkwardly? +2. Try smaller chunks for more granular matching +3. Increase `search.limit` +4. Consider a different embedding model + +### "Retrieved chunks lack context" + +1. Increase `context_radius` for text content +2. Increase `chunk_size` for more context per chunk +3. Structural content (tables, code) expands automatically + +### "Search is slow" + +1. Create a vector index: `haiku-rag create-index` +2. Reduce `search.limit` +3. Consider a smaller embedding model + +### "QA answers are wrong despite good retrieval" + +1. Check if chunks are being truncated by LLM context limits +2. Try a more capable QA model +3. Reduce number of chunks or expansion to fit context window + +## Example Configurations + +### High-Precision Technical Documentation + +```yaml +processing: + chunk_size: 256 + chunker_type: hybrid + +search: + limit: 10 + context_radius: 1 + max_context_items: 15 + +reranking: + model: + provider: mxbai + name: mxbai-rerank-base-v1 +``` + +### Long-Form Content (Articles, Reports) + +```yaml +processing: + chunk_size: 512 + chunker_type: hybrid + +search: + limit: 5 + context_radius: 2 + max_context_items: 10 +``` + +### FAQ/Knowledge Base + +```yaml +processing: + chunk_size: 128 + chunker_type: hybrid + +search: + limit: 5 + context_radius: 0 +``` diff --git a/mkdocs.yml b/mkdocs.yml index 35ff83d2..dff5c1d2 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -62,12 +62,13 @@ nav: - Configuration: - configuration/index.md - Providers: configuration/providers.md - - QA and Research: configuration/qa-research.md + - Search and Question Answering: configuration/qa-research.md - Document Processing: configuration/processing.md - Storage: configuration/storage.md - CLI: cli.md - Python: python.md - Custom Pipelines: custom-pipelines.md + - Tuning: tuning.md - Agents: agents.md - Server: server.md - Remote processing: remote-processing.md