# Installation

## Basic Installation

```bash
uv pip install haiku.rag
```

This includes support for:
- **Ollama** (default embedding provider using `mxbai-embed-large`)
- **OpenAI** (GPT models for QA and embeddings)
- **Anthropic** (Claude models for QA)
- **vLLM** (high-performance local inference for embeddings, QA, and reranking)

## Provider-Specific Installation

For additional providers, install with extras:

### Embedding Providers

#### VoyageAI

```bash
uv pip install haiku.rag[voyageai]
```

### Reranking Providers

#### MixedBread AI

```bash
uv pip install haiku.rag[mxbai]
```

#### Cohere

```bash
uv pip install haiku.rag[cohere]
```

#### Zero Entropy

```bash
uv pip install haiku.rag[zeroentropy]
```

### vLLM Setup

vLLM requires no additional installation - it works with the base haiku.rag package. However, you need to run vLLM servers separately:

```bash
# Install vLLM
pip install vllm

# Serve an embedding model
vllm serve mixedbread-ai/mxbai-embed-large-v1 --port 8000

# Serve a model for QA (requires tool calling support)
vllm serve Qwen/Qwen3-4B --port 8002 --enable-auto-tool-choice --tool-call-parser hermes

# Serve a model for reranking
vllm serve mixedbread-ai/mxbai-rerank-base-v2 --hf_overrides '{"architectures": ["Qwen2ForSequenceClassification"],"classifier_from_token": ["0", "1"], "method": "from_2_way_softmax"}' --port 8001
```

Then configure haiku.rag to use the vLLM servers. Create a `haiku.rag.yaml` file:

```yaml
embeddings:
  provider: vllm
  model: mixedbread-ai/mxbai-embed-large-v1
  vector_dim: 512

qa:
  provider: vllm
  model: Qwen/Qwen3-4B

reranking:
  provider: vllm
  model: mixedbread-ai/mxbai-rerank-base-v2

providers:
  vllm:
    embeddings_base_url: http://localhost:8000
    qa_base_url: http://localhost:8002
    rerank_base_url: http://localhost:8001
```

See [Configuration](configuration.md) for all available options.

## Requirements

- Python 3.12+
- Ollama (for default embeddings)
- vLLM server (for vLLM provider)

## Pre-download Models (Optional)

You can prefetch all required runtime models before first use:

```bash
haiku-rag download-models
```

This will download Docling models and pull any Ollama models referenced by your current configuration.

## Docker

```bash
docker pull ghcr.io/ggozad/haiku.rag:latest
```

Run the container with all services:

```bash
docker run -p 8000:8000 -p 8001:8001 -v $(pwd)/data:/data ghcr.io/ggozad/haiku.rag:latest
```

This starts the MCP server on port 8001 and A2A server on port 8000, with data persisted to `./data`.