Analysis

The analysis skill enables complex analytical tasks by writing and executing Python code in a sandboxed environment. It solves problems that traditional RAG struggles with:

Aggregation: "How many documents mention security vulnerabilities?"
Computation: "What's the average revenue across all quarterly reports?"
Multi-document analysis: "Compare the key findings between Report A and Report B"
Structured data extraction: "Extract all dollar amounts and compute totals"

How It Works

The skill receives a question
It writes Python code to explore the knowledge base
Code executes in a sandboxed Python interpreter with access to search and a virtual filesystem of documents
The skill iterates: run code, examine results, refine approach
Final answer is synthesized from the gathered data

CLI Usage

# Basic usage
haiku-rag analyze "How many documents are in the database?"

# With document filter (restricts what the skill can access)
haiku-rag analyze "Summarize the key points" --filter "uri LIKE '%report%'"

Python Usage

from haiku.rag.client import HaikuRAG

async with HaikuRAG(path_to_db) as client:
    result = await client.analyze("How many documents mention 'security'?")
    print(result.answer)
    for citation in result.citations:
        print(citation.uri, citation.title)

    # With filter (skill can only see filtered documents)
    result = await client.analyze(
        "What is the total revenue?",
        filter="title LIKE '%Financial%'"
    )

The executed Python program(s) for each turn are not on AnalysisResult itself; they live on AnalysisState.executions while the skill runs.

Sandbox Capabilities

The agent's code runs in a sandboxed Python interpreter (pydantic-monty) with:

Functions

Function	Description
`search(query, limit)`	Hybrid search (vector + full-text) with automatic context expansion. Returns `doc_item_refs` and `picture_refs` for cross-referencing with `items.jsonl`
`list_documents()`	List all documents in the knowledge base

Document Filesystem

All documents are mounted as a virtual filesystem at /documents/. The agent uses standard Python pathlib.Path to browse and read files:

/documents/{document_id}/
    metadata.json    # {id, title, uri, created_at}
    content.txt      # Full document text
    items.jsonl      # Structured items: position, self_ref, label, text, page_numbers

metadata.json — Loaded eagerly (small). Use Path('/documents').iterdir() to discover documents.
content.txt — Lazy-loaded on first read. Full document text for regex or keyword search.
items.jsonl — Lazy-loaded on first read. One JSON object per line with structured document elements. Tables are pre-rendered as markdown. Labels include section_header, text, table, list_item, caption, formula, picture, code, footnote, etc.

Search results include doc_item_refs (e.g. ["#/texts/5", "#/tables/0"]) that match self_ref values in items.jsonl, enabling navigation from search hits to document structure.

Python Features

The interpreter supports a subset of Python: variables, arithmetic, strings, f-strings, lists, dicts, tuples, sets, loops, conditionals, comprehensions, functions, async/await, filter(), getattr(), try/except, file I/O via pathlib.Path, and the json, re, math modules.

Not supported: most imports (only json, re, math, pathlib are available), class definitions, generators/yield, match statements, decorators, with statements. For pattern matching, use import re or string methods.

Security

Code executes in an isolated interpreter with:

Virtual filesystem only: The /documents/ filesystem is sandboxed — no access to the real filesystem
No network access: Code cannot make HTTP requests or open sockets
No imports: Only json, re, math, and pathlib modules are available
Execution timeout: Configurable limit (default 60s)
Output truncation: Large outputs are truncated to prevent memory issues

Context Filter

The filter parameter restricts what documents the skill can access. Unlike tool parameters, the filter is applied automatically and cannot be bypassed by the LLM — both the VFS and search results are scoped to the filter:

# Skill can only see documents with "confidential" in the URI
result = await client.analyze(
    "Summarize all findings",
    filter="uri LIKE '%confidential%'"
)

This is useful for scoping to specific document sets, enforcing access control, or limiting context for focused analysis.

Configuration

Analysis settings can be configured in haiku.rag.yaml:

analysis:
  model:
    provider: anthropic
    name: claude-sonnet-4-20250514
  code_timeout: 60.0      # Max seconds for code execution
  max_output_chars: 50000 # Truncate output after this many chars

4.9 KiB Raw Blame History