diff --git a/docs/installation.md b/docs/installation.md index d064e485..f088631c 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -12,8 +12,8 @@ uv pip install haiku.rag The full package includes **all features and extras**: - **Document processing** (Docling) - PDF, DOCX, PPTX, images, and 40+ file formats -- **All embedding providers** - Ollama, OpenAI, VoyageAI, Anthropic, vLLM -- **All rerankers** - MixedBread AI, Cohere, Zero Entropy, vLLM +- **All embedding providers** - VoyageAI +- **All rerankers** - MixedBread AI, Cohere, Zero Entropy - **A2A agent** - Agent-to-Agent protocol support This is the easiest way to get started with all features enabled. @@ -44,56 +44,13 @@ The slim package has minimal dependencies and lets you install only what you nee - **Ollama** (default embedding provider) - **OpenAI** (GPT models for QA and embeddings) - **Anthropic** (Claude models for QA) -- **vLLM** (high-performance local inference) -### vLLM Setup - -vLLM requires no additional installation - it works with the base haiku.rag package. However, you need to run vLLM servers separately: - -```bash -# Install vLLM -pip install vllm - -# Serve an embedding model -vllm serve mixedbread-ai/mxbai-embed-large-v1 --port 8000 - -# Serve a model for QA (requires tool calling support) -vllm serve Qwen/Qwen3-4B --port 8002 --enable-auto-tool-choice --tool-call-parser hermes - -# Serve a model for reranking -vllm serve mixedbread-ai/mxbai-rerank-base-v2 --hf_overrides '{"architectures": ["Qwen2ForSequenceClassification"],"classifier_from_token": ["0", "1"], "method": "from_2_way_softmax"}' --port 8001 -``` - -Then configure haiku.rag to use the vLLM servers. Create a `haiku.rag.yaml` file: - -```yaml -embeddings: - provider: vllm - model: mixedbread-ai/mxbai-embed-large-v1 - vector_dim: 512 - -qa: - provider: vllm - model: Qwen/Qwen3-4B - -reranking: - provider: vllm - model: mixedbread-ai/mxbai-rerank-base-v2 - -providers: - vllm: - embeddings_base_url: http://localhost:8000 - qa_base_url: http://localhost:8002 - rerank_base_url: http://localhost:8001 -``` - -See [Configuration](configuration.md) for all available options. +See [Configuration](configuration.md) for configuring providers including advanced options like vLLM. ## Requirements - Python 3.12+ -- Ollama (for default embeddings) -- vLLM server (for vLLM provider) +- Ollama (for default embeddings and QA) ## Pre-download Models (Optional) diff --git a/mkdocs.yml b/mkdocs.yml index 8b28324d..f7034495 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -61,9 +61,9 @@ nav: - Installation: installation.md - Configuration: configuration.md - CLI: cli.md - - Server: server.md - - Agents: agents.md - Python: python.md + - Agents: agents.md + - Server: server.md - MCP: mcp.md - A2A: a2a.md - Benchmarks: benchmarks.md