b374d5eb83
In Claude Code the client is the model, so the server no longer runs one. execute_code runs a Python program per call in the analysis sandbox over the selected documents and returns what it printed; the sandbox is created and closed per call so Monty's cumulative budget and a frozen mount never outlive a program. --no-agents goes with the two tools, and format_citations in haiku.rag.utils goes with its only caller. The sandbox exposes chunk metadata to code: chunk_meta on search results, metadata on list_documents rows and in metadata.json, and chunks.jsonl per document. A host-side failure inside a program, a document read or an in-code search raising, reaches the program by exception type only and is logged with its traceback. recovery_hint moves to haiku.rag.sandbox. Closes #604.
63 lines
4.3 KiB
Markdown
63 lines
4.3 KiB
Markdown
# Search and Question Answering
|
||
|
||
## Search Settings
|
||
|
||
Configure search behavior and context expansion:
|
||
|
||
```yaml
|
||
search:
|
||
limit: 5 # Default number of results to return
|
||
max_context_chars: 5000 # Maximum characters in expanded context
|
||
```
|
||
|
||
- **limit**: Default number of search results to return when no limit is specified. Used by CLI, MCP server, and QA. Default: 5
|
||
- **max_context_chars**: Hard limit on total characters in expanded content. Default: 5000.
|
||
|
||
Context expansion is automatic and section-aware. For structured documents (with section headers), expansion includes the entire section containing the match. For sections that exceed the budget or are too small (e.g., a title+authors area), expansion grows outward item-by-item from the match center, skipping noise labels (footnotes, page headers). This naturally crosses into adjacent sections until the budget is filled. Picture and table matches are exempt: they return their enclosing section as-is and never cross section boundaries. For unstructured documents, expansion grows outward item-by-item. Results without `doc_item_refs` (e.g., custom chunks passed to `import_document`) pass through unexpanded.
|
||
|
||
!!! note "Reranking behavior"
|
||
When a reranker is configured, search automatically retrieves 10x the requested limit, then reranks to return the final count. This improves result quality without requiring you to adjust `limit`.
|
||
|
||
## Question Answering Configuration
|
||
|
||
Configure the RAG capability (used by `client.ask` and `haiku-rag ask`):
|
||
|
||
```yaml
|
||
qa:
|
||
model:
|
||
provider: ollama
|
||
name: qwen3.8
|
||
enable_thinking: true
|
||
temperature: 0.3 # Default: 0.3
|
||
vision: true # Set false for text-only models
|
||
max_searches: 5 # Maximum search units per question
|
||
```
|
||
|
||
- **model**: LLM configuration (see [Providers](providers.md#model-settings))
|
||
- **model.vision**: Set to `true` for vision-capable models (`qwen2.5vl`, `qwen3.6`, `gpt-4o`, `claude-sonnet`, …). The capability's `search` tool only attaches picture bytes (`BinaryContent`) to its `ToolReturn` when this is `true`, otherwise picture bytes are withheld. See [Pictures × embedder × QA model](processing.md#pictures-embedder-qa-model-how-the-pieces-compose) for the full matrix.
|
||
- **max_searches**: Maximum number of search units a capability can spend per question (default: 5). Up to three searches emitted in the same model response share one unit, so a model that rephrases its query in one response spends one unit. A search in a later response starts a new unit, as does each further group of three within one response. Shared by the RAG and analysis capabilities. Searches in one response also deduplicate their returns: evidence a sibling search already showed collapses to a reference line, and each picture attaches once per response.
|
||
|
||
!!! note "Thinking on vLLM"
|
||
`enable_thinking` only applies to models with a pydantic-ai reasoning profile (o-series, gpt-5, gpt-oss). For other vLLM-served models such as Qwen3 or the Gemma family, the field is a silent no-op — set the chat template switch via [`extra_body`](providers.md#raw-provider-pass-through) instead.
|
||
|
||
## Analysis Configuration
|
||
|
||
Configure the analysis capability:
|
||
|
||
```yaml
|
||
analysis:
|
||
model:
|
||
provider: anthropic
|
||
name: claude-sonnet-4-20250514
|
||
temperature: 0.0 # Default: 0.0 (deterministic for code generation)
|
||
code_timeout: 60.0 # Max seconds a call may spend reading documents
|
||
max_output_chars: 50000 # Truncate output after this many chars
|
||
max_executions: 15 # Max execute_code calls per question
|
||
```
|
||
|
||
- **model**: LLM configuration (see [Providers](providers.md#model-settings)). When unset, falls back to `qa.model`.
|
||
- **code_timeout**: Seconds a single `execute_code` call may spend reading documents (default: 60). The sandbox refuses further reads past this point. Code that computes without reading is killed by the worker watchdog at the same limit. `code_timeout * max_executions` is the cumulative ceiling across all calls in one question.
|
||
- **max_output_chars**: Truncate code output after this many characters (default: 50000)
|
||
- **max_executions**: Maximum `execute_code` calls per question before the capability is told to answer from what it has (default: 15)
|
||
|
||
See [Analysis capability](../capabilities/analysis.md) for usage details.
|