Clarify docs about using a VLM with docling-serve
This commit is contained in:
parent
92f0fd956f
commit
4cae2cf26a
2 changed files with 93 additions and 1 deletions
|
|
@ -174,6 +174,19 @@ When using `converter: docling-serve`, the VLM calls are made by the docling-ser
|
|||
1. Set `DOCLING_SERVE_ENABLE_REMOTE_SERVICES=true` when running docling-serve
|
||||
2. Ensure the VLM endpoint is accessible from where docling-serve is running
|
||||
|
||||
**Docker networking:** If docling-serve runs in Docker and your VLM runs on the host, use `host.docker.internal` instead of `localhost`:
|
||||
|
||||
```yaml
|
||||
picture_description:
|
||||
enabled: true
|
||||
model:
|
||||
provider: ollama
|
||||
name: ministral-3
|
||||
base_url: http://host.docker.internal:11434 # NOT localhost!
|
||||
```
|
||||
|
||||
See [VLM Picture Description with docling-serve](../remote-processing.md#vlm-picture-description-with-docling-serve) for a complete example.
|
||||
|
||||
### Local vs Remote Processing
|
||||
|
||||
**Local processing** (default):
|
||||
|
|
|
|||
|
|
@ -39,7 +39,13 @@ The easiest way to use haiku.rag with docling-serve is using the slim Docker ima
|
|||
See the [official docling-serve repository](https://github.com/docling-project/docling-serve) for installation options. The quickest way is using Docker:
|
||||
|
||||
```bash
|
||||
docker run -p 5001:5001 -e DOCLING_SERVE_ENABLE_UI=1 quay.io/docling-project/docling-serve
|
||||
docker run -p 5001:5001 quay.io/docling-project/docling-serve
|
||||
```
|
||||
|
||||
To enable the web UI for debugging:
|
||||
|
||||
```bash
|
||||
docker run -p 5001:5001 -e DOCLING_SERVE_ENABLE_UI=true quay.io/docling-project/docling-serve
|
||||
```
|
||||
|
||||
### Configuration
|
||||
|
|
@ -128,6 +134,79 @@ processing:
|
|||
- `false` (default): Tables as narrative text
|
||||
- `true`: Tables as markdown format
|
||||
|
||||
## VLM Picture Description with docling-serve
|
||||
|
||||
When using VLM picture description with docling-serve, the VLM API calls are made by the docling-serve container, not by haiku.rag. This requires additional configuration.
|
||||
|
||||
### Enable Remote Services
|
||||
|
||||
docling-serve blocks external API calls by default. To enable VLM picture description, start docling-serve with:
|
||||
|
||||
```bash
|
||||
docker run -p 5001:5001 -e DOCLING_SERVE_ENABLE_REMOTE_SERVICES=true quay.io/docling-project/docling-serve
|
||||
```
|
||||
|
||||
### Docker Networking
|
||||
|
||||
When docling-serve runs in Docker and your VLM (e.g., Ollama) runs on the host, `localhost` inside the container refers to the container itself, not your host machine.
|
||||
|
||||
Use `host.docker.internal` to reach host services from within Docker:
|
||||
|
||||
```yaml
|
||||
# haiku.rag.yaml
|
||||
processing:
|
||||
converter: docling-serve
|
||||
chunker: docling-serve
|
||||
conversion_options:
|
||||
picture_description:
|
||||
enabled: true
|
||||
model:
|
||||
provider: ollama
|
||||
name: ministral-3
|
||||
base_url: http://host.docker.internal:11434 # NOT localhost!
|
||||
```
|
||||
|
||||
### Complete Example
|
||||
|
||||
1. Start Ollama with a vision model on your host:
|
||||
|
||||
```bash
|
||||
ollama pull ministral-3
|
||||
ollama serve
|
||||
```
|
||||
|
||||
2. Start docling-serve with remote services enabled:
|
||||
|
||||
```bash
|
||||
docker run -p 5001:5001 -e DOCLING_SERVE_ENABLE_REMOTE_SERVICES=true quay.io/docling-project/docling-serve
|
||||
```
|
||||
|
||||
3. Configure haiku.rag:
|
||||
|
||||
```yaml
|
||||
# haiku.rag.yaml
|
||||
processing:
|
||||
converter: docling-serve
|
||||
chunker: docling-serve
|
||||
conversion_options:
|
||||
picture_description:
|
||||
enabled: true
|
||||
model:
|
||||
provider: ollama
|
||||
name: ministral-3
|
||||
base_url: http://host.docker.internal:11434
|
||||
|
||||
providers:
|
||||
docling_serve:
|
||||
base_url: http://localhost:5001
|
||||
```
|
||||
|
||||
4. Add a document:
|
||||
|
||||
```bash
|
||||
haiku-rag add-src document.pdf
|
||||
```
|
||||
|
||||
## Resources
|
||||
|
||||
- [docling-serve GitHub](https://github.com/docling-project/docling-serve)
|
||||
|
|
|
|||
Loading…
Reference in a new issue