245 lines
7.3 KiB
Markdown
245 lines
7.3 KiB
Markdown
# Remote Processing
|
|
|
|
`haiku.rag` can use [docling-serve](https://github.com/docling-project/docling-serve) for remote document processing and chunking, offloading resource-intensive operations to a dedicated service.
|
|
|
|
## Overview
|
|
|
|
docling-serve is a REST API service that provides:
|
|
|
|
- Document conversion (PDF, DOCX, PPTX, images, etc.)
|
|
- Intelligent chunking with structure preservation
|
|
- OCR capabilities for scanned documents
|
|
- Table and figure extraction
|
|
|
|
## When to Use docling-serve
|
|
|
|
**Use local processing (default) when:**
|
|
|
|
- Working with small to medium document volumes
|
|
- Running on development machines
|
|
- Want zero external dependencies
|
|
- Processing simple document formats
|
|
|
|
**Use docling-serve when:**
|
|
|
|
- Processing large volumes of documents
|
|
- Working with complex PDFs requiring OCR
|
|
- Running in production environments
|
|
- Separating compute-intensive tasks
|
|
- Scaling document processing independently
|
|
|
|
## Setup
|
|
|
|
### Docker Compose (Recommended)
|
|
|
|
The slim Docker image with docker-compose is the recommended setup. See `examples/docker/docker-compose.yml` for a complete configuration that includes both services.
|
|
|
|
### Running docling-serve Manually
|
|
|
|
See the [official docling-serve repository](https://github.com/docling-project/docling-serve) for installation options. The quickest way is using Docker:
|
|
|
|
```bash
|
|
docker run -p 5001:5001 quay.io/docling-project/docling-serve
|
|
```
|
|
|
|
To enable the web UI for debugging:
|
|
|
|
```bash
|
|
docker run -p 5001:5001 -e DOCLING_SERVE_ENABLE_UI=true quay.io/docling-project/docling-serve
|
|
```
|
|
|
|
### Configuration
|
|
|
|
Configure haiku.rag to use docling-serve. See the [Document Processing](configuration/processing.md) guide for all available options.
|
|
|
|
```yaml
|
|
# haiku.rag.yaml
|
|
processing:
|
|
converter: docling-serve # Use remote conversion
|
|
chunker: docling-serve # Use remote chunking
|
|
|
|
providers:
|
|
docling_serve:
|
|
base_url: http://localhost:5001
|
|
api_key: "" # Optional API key for authentication
|
|
```
|
|
|
|
## Features
|
|
|
|
### Remote Document Conversion
|
|
|
|
When `converter: docling-serve` is configured, documents are sent to the docling-serve API for conversion:
|
|
|
|
```python
|
|
from haiku.rag.client import HaikuRAG
|
|
|
|
async with HaikuRAG() as client:
|
|
# PDF is processed by docling-serve
|
|
doc = await client.create_document_from_source("complex.pdf")
|
|
```
|
|
|
|
### Remote Chunking
|
|
|
|
When `chunker: docling-serve` is configured, chunking is performed remotely:
|
|
|
|
```yaml
|
|
processing:
|
|
chunker: docling-serve
|
|
chunker_type: hybrid # or hierarchical
|
|
chunk_size: 256
|
|
chunking_tokenizer: "Qwen/Qwen3-Embedding-0.6B"
|
|
chunking_merge_peers: true
|
|
chunking_use_markdown_tables: false
|
|
```
|
|
|
|
## Advanced Configuration
|
|
|
|
### Custom Tokenizers
|
|
|
|
You can use any HuggingFace tokenizer model:
|
|
|
|
```yaml
|
|
processing:
|
|
chunking_tokenizer: "bert-base-uncased" # Or any HF model
|
|
```
|
|
|
|
### Chunking Strategies
|
|
|
|
**Hybrid Chunking** (default):
|
|
|
|
- Best for most documents
|
|
- Preserves semantic boundaries
|
|
- Structure-aware splitting
|
|
|
|
**Hierarchical Chunking**:
|
|
|
|
- Maintains document hierarchy
|
|
- Better for deeply nested documents
|
|
- Preserves parent-child relationships
|
|
|
|
```yaml
|
|
processing:
|
|
chunker_type: hierarchical
|
|
```
|
|
|
|
### Table Handling
|
|
|
|
Control how tables are represented:
|
|
|
|
```yaml
|
|
processing:
|
|
chunking_use_markdown_tables: true # Preserve table structure
|
|
```
|
|
|
|
- `false` (default): Tables as narrative text
|
|
- `true`: Tables as markdown format
|
|
|
|
## HTML Image Fetching
|
|
|
|
docling-serve does **not** fetch external `<img src="https://...">` URLs in HTML inputs. The `ConvertDocumentsOptions` API exposes no equivalent of docling-local's `HTMLBackendOptions.fetch_images` / `enable_remote_fetch`, and the server-side `DoclingConverterManager` registers `format_options` only for PDF and IMAGE. HTML falls through to docling's defaults (`fetch_images=False`).
|
|
|
|
Consequence: ingesting HTML with external image references through docling-serve produces picture items with `picture_data=NULL`. The same input through docling-local fetches the bytes (subject to the SSRF / size / timeout guards documented in [Configuration → External image fetching](configuration/processing.md#external-image-fetching)).
|
|
|
|
To preserve image bytes when ingesting HTML or Markdown that references remote images, use `converter: docling-local`. The `processing.conversion_options.fetch_remote_images` flag has no effect on docling-serve and the `source_uri` kwarg on `HaikuRAG.convert()` is accepted but ignored on this path.
|
|
|
|
## VLM Picture Description with docling-serve
|
|
|
|
When using VLM picture description with docling-serve, the VLM API calls are made by the docling-serve container, not by haiku.rag. This requires additional configuration.
|
|
|
|
### Enable Remote Services
|
|
|
|
docling-serve blocks external API calls by default. To enable VLM picture description, start docling-serve with:
|
|
|
|
```bash
|
|
docker run -p 5001:5001 -e DOCLING_SERVE_ENABLE_REMOTE_SERVICES=true quay.io/docling-project/docling-serve
|
|
```
|
|
|
|
### Docker Networking
|
|
|
|
When docling-serve runs in Docker and your VLM (e.g., Ollama) runs on the host, `localhost` inside the container refers to the container itself, not your host machine.
|
|
|
|
Use `host.docker.internal` to reach host services from within Docker:
|
|
|
|
```yaml
|
|
# haiku.rag.yaml
|
|
processing:
|
|
converter: docling-serve
|
|
chunker: docling-serve
|
|
conversion_options:
|
|
picture_description:
|
|
enabled: true
|
|
model:
|
|
provider: ollama
|
|
name: ministral-3
|
|
base_url: http://host.docker.internal:11434 # NOT localhost!
|
|
```
|
|
|
|
### Complete Example
|
|
|
|
1. Start Ollama with a vision model on your host:
|
|
|
|
```bash
|
|
ollama pull ministral-3
|
|
ollama serve
|
|
```
|
|
|
|
2. Start docling-serve with remote services enabled:
|
|
|
|
```bash
|
|
docker run -p 5001:5001 -e DOCLING_SERVE_ENABLE_REMOTE_SERVICES=true quay.io/docling-project/docling-serve
|
|
```
|
|
|
|
3. Configure haiku.rag:
|
|
|
|
```yaml
|
|
# haiku.rag.yaml
|
|
processing:
|
|
converter: docling-serve
|
|
chunker: docling-serve
|
|
conversion_options:
|
|
picture_description:
|
|
enabled: true
|
|
model:
|
|
provider: ollama
|
|
name: ministral-3
|
|
base_url: http://host.docker.internal:11434
|
|
|
|
providers:
|
|
docling_serve:
|
|
base_url: http://localhost:5001
|
|
```
|
|
|
|
4. Add a document:
|
|
|
|
```bash
|
|
haiku-rag add-src document.pdf
|
|
```
|
|
|
|
## Operational notes
|
|
|
|
Long-running docling-serve containers see CPU memory grow monotonically
|
|
([docling-serve #366](https://github.com/docling-project/docling-serve/issues/366),
|
|
[#474](https://github.com/docling-project/docling-serve/issues/474)). The
|
|
underlying parser leaks are in core docling
|
|
([#2209](https://github.com/docling-project/docling/issues/2209),
|
|
[#1343](https://github.com/docling-project/docling/issues/1343)) and affect
|
|
docling-local too.
|
|
|
|
Recommended deployment shape:
|
|
|
|
- Set `mem_limit` on the docling-serve container (or `resources.limits.memory`
|
|
in Kubernetes) at a value comfortably above your largest expected job.
|
|
- Combine with `restart: unless-stopped` so the runtime restarts when the
|
|
kernel OOM-kills.
|
|
- Run multiple docling-serve replicas behind haiku.rag's round-robin
|
|
`providers.docling_serve.base_url` list (see
|
|
[Document Processing](configuration/processing.md)). A restart of one
|
|
replica doesn't stop ingest.
|
|
- In haiku.rag, set `processing.split_pages` for large-PDF workloads so each
|
|
slice is an independent docling-serve task and the per-task working set
|
|
stays bounded.
|
|
|
|
## Resources
|
|
|
|
- [docling-serve GitHub](https://github.com/docling-project/docling-serve)
|
|
- [docling-serve Documentation](https://github.com/docling-project/docling-serve#readme)
|