haiku.rag/docs/rlm.md

RLM Agent (Recursive Language Model)

The RLM agent enables complex analytical tasks by writing and executing Python code in a sandboxed environment. It solves problems that traditional RAG struggles with:

• Aggregation: "How many documents mention security vulnerabilities?"
• Computation: "What's the average revenue across all quarterly reports?"
• Multi-document analysis: "Compare the key findings between Report A and Report B"
• Structured data extraction: "Extract all tables from the document and summarize them"

How It Works

1. The agent receives a question
2. It writes Python code to explore the knowledge base
3. Code executes in a sandboxed environment with access to haiku.rag functions
4. The agent iterates: run code, examine results, refine approach
5. Final answer is synthesized from the gathered data

CLI Usage

# Basic usage
haiku-rag rlm "How many documents are in the database?"

# With document filter (restricts what the agent can access)
haiku-rag rlm "Summarize the key points" --filter "uri LIKE '%report%'"

# Pre-load specific documents
haiku-rag rlm "Compare these two reports" --document "Q1 Report" --document "Q2 Report"

Python Usage

from haiku.rag.client import HaikuRAG

async with HaikuRAG(path_to_db) as client:
    # Basic question
    result = await client.rlm("How many documents mention 'security'?")
    print(result.answer)    # The answer
    print(result.program)   # The final consolidated program

    # With filter (agent can only see filtered documents)
    result = await client.rlm(
        "What is the total revenue?",
        filter="title LIKE '%Financial%'"
    )

    # Pre-load specific documents
    result = await client.rlm(
        "Compare the conclusions",
        documents=["Report A", "Report B"]
    )

Available Functions

Inside the sandbox, these functions are available (no imports needed):

search(query, limit=10)

Search the knowledge base using hybrid search (vector + full-text).

results = search("climate change impacts", limit=20)
for r in results:
    print(r['document_title'], r['score'])
    print(r['content'][:200])

Returns list of dicts with keys: chunk_id, content, document_id, document_title, document_uri, score, page_numbers, headings

list_documents(limit=10, offset=0)

List available documents in the knowledge base.

docs = list_documents(limit=100)
for doc in docs:
    print(doc['id'], doc['title'])

Returns list of dicts with keys: id, title, uri, created_at

get_document(id_or_title)

Get the full text content of a document by ID, title, or URI.

content = get_document("Q1 Report")
if content:
    print(len(content), "characters")

Returns the document content as a string, or None if not found.

get_docling_document(id_or_title)

Get the structured DoclingDocument object for advanced analysis of tables, figures, and document structure.

doc = get_docling_document("Technical Manual")
if doc:
    print(f"Tables: {len(doc.tables)}")
    print(f"Pictures: {len(doc.pictures)}")

    # Extract table data
    for table in doc.tables:
        for cell in table.data.table_cells:
            print(f"Row {cell.start_row_offset_idx}, Col {cell.start_col_offset_idx}: {cell.text}")

llm(prompt)

Call an LLM directly for classification, summarization, or extraction tasks.

content = get_document("Q1 Report")
sentiment = llm(f"Classify the sentiment as positive, negative, or mixed: {content}")
print(sentiment)

Use this when you have content and need LLM reasoning without RAG search.

Pre-loaded Documents

When documents are pre-loaded via the documents parameter, they're available as a documents variable:

# Available when documents are pre-loaded
for doc in documents:
    print(doc['title'], len(doc['content']))

Each document dict has keys: id, title, uri, content

Imports

The sandbox runs in a Docker container with full Python available. Any module installed in the container image can be imported:

import re
import json
from collections import Counter

# Extract and count patterns
results = search("error", limit=50)
error_types = []
for r in results:
    matches = re.findall(r'Error: (\w+)', r['content'])
    error_types.extend(matches)

print(Counter(error_types).most_common(10))

The default image (ghcr.io/ggozad/haiku.rag-slim) includes the Python standard library. Custom images can add additional packages like pandas or numpy.

Docker Sandbox

Code executes in an isolated Docker container with:

• Read-only database: The LanceDB database is mounted read-only
• Memory limits: Configurable memory limit (default 512MB)
• Execution timeout: Code times out after configurable limit (default 60s)
• Output truncation: Large outputs are truncated to prevent memory issues
• Container reuse: Within a single rlm() call, the container stays warm for multiple code executions

Context Filter

The filter parameter restricts what documents the agent can access. Unlike tool parameters, the filter is applied automatically and cannot be bypassed by the LLM:

# Agent can only see documents with "confidential" in the URI
result = await client.rlm(
    "Summarize all findings",
    filter="uri LIKE '%confidential%'"
)

This is useful for:

• Scoping to specific document sets
• Enforcing access control
• Limiting context for focused analysis

Configuration

RLM settings can be configured in haiku.rag.yaml:

rlm:
  model:
    provider: anthropic
    name: claude-sonnet-4-20250514
  code_timeout: 60.0      # Max seconds for code execution
  max_output_chars: 50000 # Truncate output after this many chars
  docker_image: "ghcr.io/ggozad/haiku.rag-slim:latest"  # Container image
  docker_memory_limit: "512m"  # Container memory limit

Custom Docker Image

To add additional Python packages, create a custom Dockerfile:

FROM ghcr.io/ggozad/haiku.rag-slim:latest
RUN pip install pandas numpy

Build and configure:

docker build -t my-rlm-image .

rlm:
  docker_image: "my-rlm-image"