Update docs
This commit is contained in:
parent
106945ddc0
commit
ef5e829bdc
2 changed files with 6 additions and 49 deletions
|
|
@ -12,8 +12,8 @@ uv pip install haiku.rag
|
|||
|
||||
The full package includes **all features and extras**:
|
||||
- **Document processing** (Docling) - PDF, DOCX, PPTX, images, and 40+ file formats
|
||||
- **All embedding providers** - Ollama, OpenAI, VoyageAI, Anthropic, vLLM
|
||||
- **All rerankers** - MixedBread AI, Cohere, Zero Entropy, vLLM
|
||||
- **All embedding providers** - VoyageAI
|
||||
- **All rerankers** - MixedBread AI, Cohere, Zero Entropy
|
||||
- **A2A agent** - Agent-to-Agent protocol support
|
||||
|
||||
This is the easiest way to get started with all features enabled.
|
||||
|
|
@ -44,56 +44,13 @@ The slim package has minimal dependencies and lets you install only what you nee
|
|||
- **Ollama** (default embedding provider)
|
||||
- **OpenAI** (GPT models for QA and embeddings)
|
||||
- **Anthropic** (Claude models for QA)
|
||||
- **vLLM** (high-performance local inference)
|
||||
|
||||
### vLLM Setup
|
||||
|
||||
vLLM requires no additional installation - it works with the base haiku.rag package. However, you need to run vLLM servers separately:
|
||||
|
||||
```bash
|
||||
# Install vLLM
|
||||
pip install vllm
|
||||
|
||||
# Serve an embedding model
|
||||
vllm serve mixedbread-ai/mxbai-embed-large-v1 --port 8000
|
||||
|
||||
# Serve a model for QA (requires tool calling support)
|
||||
vllm serve Qwen/Qwen3-4B --port 8002 --enable-auto-tool-choice --tool-call-parser hermes
|
||||
|
||||
# Serve a model for reranking
|
||||
vllm serve mixedbread-ai/mxbai-rerank-base-v2 --hf_overrides '{"architectures": ["Qwen2ForSequenceClassification"],"classifier_from_token": ["0", "1"], "method": "from_2_way_softmax"}' --port 8001
|
||||
```
|
||||
|
||||
Then configure haiku.rag to use the vLLM servers. Create a `haiku.rag.yaml` file:
|
||||
|
||||
```yaml
|
||||
embeddings:
|
||||
provider: vllm
|
||||
model: mixedbread-ai/mxbai-embed-large-v1
|
||||
vector_dim: 512
|
||||
|
||||
qa:
|
||||
provider: vllm
|
||||
model: Qwen/Qwen3-4B
|
||||
|
||||
reranking:
|
||||
provider: vllm
|
||||
model: mixedbread-ai/mxbai-rerank-base-v2
|
||||
|
||||
providers:
|
||||
vllm:
|
||||
embeddings_base_url: http://localhost:8000
|
||||
qa_base_url: http://localhost:8002
|
||||
rerank_base_url: http://localhost:8001
|
||||
```
|
||||
|
||||
See [Configuration](configuration.md) for all available options.
|
||||
See [Configuration](configuration.md) for configuring providers including advanced options like vLLM.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Python 3.12+
|
||||
- Ollama (for default embeddings)
|
||||
- vLLM server (for vLLM provider)
|
||||
- Ollama (for default embeddings and QA)
|
||||
|
||||
## Pre-download Models (Optional)
|
||||
|
||||
|
|
|
|||
|
|
@ -61,9 +61,9 @@ nav:
|
|||
- Installation: installation.md
|
||||
- Configuration: configuration.md
|
||||
- CLI: cli.md
|
||||
- Server: server.md
|
||||
- Agents: agents.md
|
||||
- Python: python.md
|
||||
- Agents: agents.md
|
||||
- Server: server.md
|
||||
- MCP: mcp.md
|
||||
- A2A: a2a.md
|
||||
- Benchmarks: benchmarks.md
|
||||
|
|
|
|||
Loading…
Reference in a new issue