117 lines
4.9 KiB
Markdown
117 lines
4.9 KiB
Markdown
# Analysis
|
|
|
|
The analysis skill enables complex analytical tasks by writing and executing Python code in a sandboxed environment. It solves problems that traditional RAG struggles with:
|
|
|
|
- **Aggregation**: "How many documents mention security vulnerabilities?"
|
|
- **Computation**: "What's the average revenue across all quarterly reports?"
|
|
- **Multi-document analysis**: "Compare the key findings between Report A and Report B"
|
|
- **Structured data extraction**: "Extract all dollar amounts and compute totals"
|
|
|
|
## How It Works
|
|
|
|
1. The skill receives a question
|
|
2. It writes Python code to explore the knowledge base
|
|
3. Code executes in a sandboxed Python interpreter with access to search and a virtual filesystem of documents
|
|
4. The skill iterates: run code, examine results, refine approach
|
|
5. Final answer is synthesized from the gathered data
|
|
|
|
## CLI Usage
|
|
|
|
```bash
|
|
# Basic usage
|
|
haiku-rag analyze "How many documents are in the database?"
|
|
|
|
# With document filter (restricts what the skill can access)
|
|
haiku-rag analyze "Summarize the key points" --filter "uri LIKE '%report%'"
|
|
```
|
|
|
|
## Python Usage
|
|
|
|
```python
|
|
from haiku.rag.client import HaikuRAG
|
|
|
|
async with HaikuRAG(path_to_db) as client:
|
|
result = await client.analyze("How many documents mention 'security'?")
|
|
print(result.answer)
|
|
for citation in result.citations:
|
|
print(citation.uri, citation.title)
|
|
|
|
# With filter (skill can only see filtered documents)
|
|
result = await client.analyze(
|
|
"What is the total revenue?",
|
|
filter="title LIKE '%Financial%'"
|
|
)
|
|
```
|
|
|
|
The executed Python program(s) for each turn are not on `AnalysisResult` itself; they live on `AnalysisState.executions` while the skill runs.
|
|
|
|
## Sandbox Capabilities
|
|
|
|
The agent's code runs in a sandboxed Python interpreter ([pydantic-monty](https://github.com/pydantic/monty)) with:
|
|
|
|
### Functions
|
|
|
|
| Function | Description |
|
|
|----------|-------------|
|
|
| `search(query, limit)` | Hybrid search (vector + full-text) with automatic context expansion. Returns `doc_item_refs` and `picture_refs` for cross-referencing with `items.jsonl` |
|
|
| `list_documents()` | List all documents in the knowledge base |
|
|
|
|
### Document Filesystem
|
|
|
|
All documents are mounted as a virtual filesystem at `/documents/`. The agent uses standard Python `pathlib.Path` to browse and read files:
|
|
|
|
```
|
|
/documents/{document_id}/
|
|
metadata.json # {id, title, uri, created_at}
|
|
content.txt # Full document text
|
|
items.jsonl # Structured items: position, self_ref, label, text, page_numbers
|
|
```
|
|
|
|
- **`metadata.json`** — Loaded eagerly (small). Use `Path('/documents').iterdir()` to discover documents.
|
|
- **`content.txt`** — Lazy-loaded on first read. Full document text for regex or keyword search.
|
|
- **`items.jsonl`** — Lazy-loaded on first read. One JSON object per line with structured document elements. Tables are pre-rendered as markdown. Labels include `section_header`, `text`, `table`, `list_item`, `caption`, `formula`, `picture`, `code`, `footnote`, etc.
|
|
|
|
Search results include `doc_item_refs` (e.g. `["#/texts/5", "#/tables/0"]`) that match `self_ref` values in `items.jsonl`, enabling navigation from search hits to document structure.
|
|
|
|
### Python Features
|
|
|
|
The interpreter supports a subset of Python: variables, arithmetic, strings, f-strings, lists, dicts, tuples, sets, loops, conditionals, comprehensions, functions, async/await, `filter()`, `getattr()`, try/except, file I/O via `pathlib.Path`, and the `json`, `re`, `math` modules.
|
|
|
|
Not supported: most imports (only `json`, `re`, `math`, `pathlib` are available), class definitions, generators/yield, match statements, decorators, `with` statements. For pattern matching, use `import re` or string methods.
|
|
|
|
### Security
|
|
|
|
Code executes in an isolated interpreter with:
|
|
|
|
- **Virtual filesystem only**: The `/documents/` filesystem is sandboxed — no access to the real filesystem
|
|
- **No network access**: Code cannot make HTTP requests or open sockets
|
|
- **No imports**: Only `json`, `re`, `math`, and `pathlib` modules are available
|
|
- **Execution timeout**: Configurable limit (default 60s)
|
|
- **Output truncation**: Large outputs are truncated to prevent memory issues
|
|
|
|
## Context Filter
|
|
|
|
The `filter` parameter restricts what documents the skill can access. Unlike tool parameters, the filter is applied automatically and cannot be bypassed by the LLM — both the VFS and search results are scoped to the filter:
|
|
|
|
```python
|
|
# Skill can only see documents with "confidential" in the URI
|
|
result = await client.analyze(
|
|
"Summarize all findings",
|
|
filter="uri LIKE '%confidential%'"
|
|
)
|
|
```
|
|
|
|
This is useful for scoping to specific document sets, enforcing access control, or limiting context for focused analysis.
|
|
|
|
## Configuration
|
|
|
|
Analysis settings can be configured in `haiku.rag.yaml`:
|
|
|
|
```yaml
|
|
analysis:
|
|
model:
|
|
provider: anthropic
|
|
name: claude-sonnet-4-20250514
|
|
code_timeout: 60.0 # Max seconds for code execution
|
|
max_output_chars: 50000 # Truncate output after this many chars
|
|
```
|