Installation

Choose Your Package

haiku.rag is available in two packages:

Full Package (Recommended)

uv pip install haiku.rag

The full package includes all features and extras:

Document processing (Docling) - PDF, DOCX, PPTX, images, and 40+ file formats
All embedding providers - Ollama, OpenAI, VoyageAI, Anthropic, vLLM
All rerankers - MixedBread AI, Cohere, Zero Entropy, vLLM
A2A agent - Agent-to-Agent protocol support

This is the easiest way to get started with all features enabled.

Slim Package (Minimal Dependencies)

# Minimal installation (no document processing)
uv pip install haiku.rag-slim

# With document processing
uv pip install haiku.rag-slim[docling]

# With specific providers
uv pip install haiku.rag-slim[docling,voyageai,mxbai]

The slim package has minimal dependencies and lets you install only what you need:

docling - PDF, DOCX, PPTX, images, and other document formats
voyageai - VoyageAI embeddings
mxbai - MixedBread AI reranking
a2a - Agent-to-Agent protocol support
cohere - Cohere reranking
zeroentropy - Zero Entropy reranking

Built-in providers (no extras needed):

Ollama (default embedding provider)
OpenAI (GPT models for QA and embeddings)
Anthropic (Claude models for QA)
vLLM (high-performance local inference)

vLLM Setup

vLLM requires no additional installation - it works with the base haiku.rag package. However, you need to run vLLM servers separately:

# Install vLLM
pip install vllm

# Serve an embedding model
vllm serve mixedbread-ai/mxbai-embed-large-v1 --port 8000

# Serve a model for QA (requires tool calling support)
vllm serve Qwen/Qwen3-4B --port 8002 --enable-auto-tool-choice --tool-call-parser hermes

# Serve a model for reranking
vllm serve mixedbread-ai/mxbai-rerank-base-v2 --hf_overrides '{"architectures": ["Qwen2ForSequenceClassification"],"classifier_from_token": ["0", "1"], "method": "from_2_way_softmax"}' --port 8001

Then configure haiku.rag to use the vLLM servers. Create a haiku.rag.yaml file:

embeddings:
  provider: vllm
  model: mixedbread-ai/mxbai-embed-large-v1
  vector_dim: 512

qa:
  provider: vllm
  model: Qwen/Qwen3-4B

reranking:
  provider: vllm
  model: mixedbread-ai/mxbai-rerank-base-v2

providers:
  vllm:
    embeddings_base_url: http://localhost:8000
    qa_base_url: http://localhost:8002
    rerank_base_url: http://localhost:8001

See Configuration for all available options.

Requirements

Python 3.12+
Ollama (for default embeddings)
vLLM server (for vLLM provider)

Pre-download Models (Optional)

You can prefetch all required runtime models before first use:

haiku-rag download-models

This will download Docling models and pull any Ollama models referenced by your current configuration.

Docker

docker pull ghcr.io/ggozad/haiku.rag:latest

Run the container with all services:

docker run -p 8000:8000 -p 8001:8001 -v $(pwd)/data:/data ghcr.io/ggozad/haiku.rag:latest

This starts the MCP server on port 8001 and A2A server on port 8000, with data persisted to ./data.

3.1 KiB Raw Blame History