Replace picture_description.enabled with processing.pictures enum

This commit is contained in:
Yiorgis Gozadinos 2026-05-13 15:29:32 +03:00
parent bdab201c42
commit a98ddc14b8
No known key found for this signature in database
17 changed files with 118 additions and 136 deletions

View file

@ -15,6 +15,7 @@
- **Chat TUI streams markdown incrementally.** Assistant messages now use Textual's `MarkdownStream` (`Markdown.get_stream`) and write per-token deltas instead of re-parsing the entire accumulated message on every token. Removes the O(n²) re-parse that visibly stuttered long responses. Bumps `textual` floor to `>=8.2.4` so `Markdown.get_stream` is reachable via the public API. - **Chat TUI streams markdown incrementally.** Assistant messages now use Textual's `MarkdownStream` (`Markdown.get_stream`) and write per-token deltas instead of re-parsing the entire accumulated message on every token. Removes the O(n²) re-parse that visibly stuttered long responses. Bumps `textual` floor to `>=8.2.4` so `Markdown.get_stream` is reachable via the public API.
- **Embedding compatibility check only raises on `vector_dim` mismatch.** `provider` and `name` drift (legitimate when the same model is served by a different stack, e.g. Ollama → vLLM-via-openai) now logs a one-time warning and updates the stored settings to match the current config. Subsequent opens are silent. Run `rebuild --embed-only` if you also want to re-embed under the new stack. - **Embedding compatibility check only raises on `vector_dim` mismatch.** `provider` and `name` drift (legitimate when the same model is served by a different stack, e.g. Ollama → vLLM-via-openai) now logs a one-time warning and updates the stored settings to match the current config. Subsequent opens are silent. Run `rebuild --embed-only` if you also want to re-embed under the new stack.
- **`processing.pictures` enum replaces `picture_description.enabled`.** Three modes: `none` (skip picture generation entirely — lower RAM, smaller DBs), `description` (generate images, run VLM, store bytes), `image` (default — generate images, store bytes, no VLM). Closes #366. Breaking change: rename `picture_description.enabled: true``pictures: description`, `picture_description.enabled: false``pictures: image`. The pre-April-30 `generate_picture_images` flag is also gone; use `pictures: none` for that opt-out.
### Fixed ### Fixed

View file

@ -497,7 +497,7 @@ haiku-rag rebuild --descriptions
| Title only | `--title-only` | Generate titles for documents without one | | Title only | `--title-only` | Generate titles for documents without one |
| Descriptions | `--descriptions` | Add VLM picture descriptions to an existing database | | Descriptions | `--descriptions` | Add VLM picture descriptions to an existing database |
**`--descriptions` mode** runs the configured VLM (`processing.conversion_options.picture_description.model`) over the picture bytes already stored in `document_items.picture_data`, patches each description into the stored docling blob's `pictures[i].meta.description.text`, and re-chunks + re-embeds so chunk text reflects the new descriptions. Requires `picture_description.enabled: true` in the config. Idempotent — pictures that already carry a description are skipped, so the operation is safe to re-run after a partial failure. The docling parse is skipped entirely; only the VLM time is paid. **`--descriptions` mode** runs the configured VLM (`processing.conversion_options.picture_description.model`) over the picture bytes already stored in `document_items.picture_data`, patches each description into the stored docling blob's `pictures[i].meta.description.text`, and re-chunks + re-embeds so chunk text reflects the new descriptions. Requires `processing.pictures: description` in the config. Idempotent — pictures that already carry a description are skipped, so the operation is safe to re-run after a partial failure. The docling parse is skipped entirely; only the VLM time is paid.
### Download Models ### Download Models

View file

@ -45,12 +45,12 @@ processing:
images_scale: 2.0 # Image scale factor images_scale: 2.0 # Image scale factor
generate_page_images: true # Include rendered page images (for visualize_chunk) generate_page_images: true # Include rendered page images (for visualize_chunk)
# VLM picture description (off by default; see "Picture Handling" below) # VLM settings used when processing.pictures == "description" (see "Picture Handling" below)
picture_description: picture_description:
enabled: false
model: model:
provider: ollama provider: ollama
name: ministral-3 name: ministral-3
pictures: image # none | description | image
``` ```
### Conversion Options ### Conversion Options
@ -121,7 +121,7 @@ Per-image failures (404, timeout, oversized, unreadable) leave that picture as a
**Scope of conversion options across formats:** **Scope of conversion options across formats:**
| Input | OCR / table options | `images_scale` / `generate_page_images` | `picture_description` | `fetch_remote_images` | | Input | OCR / table options | `images_scale` / `generate_page_images` | `pictures` | `fetch_remote_images` |
|---|---|---|---|---| |---|---|---|---|---|
| `.pdf` | ✅ | ✅ | ✅ | n/a | | `.pdf` | ✅ | ✅ | ✅ | n/a |
| `.png` / `.jpg` / `.jpeg` / `.bmp` / `.tiff` / `.webp` | ✅ | ✅ | ✅ | n/a | | `.png` / `.jpg` / `.jpeg` / `.bmp` / `.tiff` / `.webp` | ✅ | ✅ | ✅ | n/a |
@ -132,26 +132,36 @@ Per-image failures (404, timeout, oversized, unreadable) leave that picture as a
#### Picture Handling #### Picture Handling
Picture bytes (figures, diagrams) are always extracted and stored in `document_items.picture_data` for every ingested document. The single configurable knob is whether a Vision Language Model (VLM) runs at ingest to generate textual descriptions: `processing.pictures` picks one of three modes:
| Mode | Picture-image generation in docling | Bytes stored in `document_items.picture_data` | VLM runs at ingest |
|---|---|---|---|
| `none` | off | no | no |
| `description` | on | yes | yes |
| `image` (default) | on | yes | no |
Use `none` when you don't need picture content (e.g. very large reference manuals where RAM is tight); use `description` to weave VLM-generated text into chunk content and keep bytes for later; use `image` (default) to keep bytes without paying the VLM cost. The prompt is configurable under `prompts.picture_description` — see [Prompts](prompts.md).
```yaml ```yaml
processing: processing:
pictures: description # none | description | image
conversion_options: conversion_options:
picture_description: picture_description: # only consulted when pictures == "description"
enabled: true # default false
model: model:
provider: ollama # any OpenAI-compatible /v1/chat/completions provider provider: ollama # any OpenAI-compatible /v1/chat/completions provider
name: ministral-3 name: ministral-3
timeout: 90 timeout: 90
max_tokens: 200 max_tokens: 200
``` ```
When `enabled: true`, each picture's description is woven into the chunk text and is searchable via FTS. The prompt is configurable under `prompts.picture_description` — see [Prompts](prompts.md). !!! warning "Breaking change"
`processing.conversion_options.picture_description.enabled` is replaced by `processing.pictures`. Map `enabled: true``pictures: description`, `enabled: false``pictures: image`. The pre-April-30 `generate_picture_images` flag also no longer exists; use `pictures: none` for the old opt-out.
**Switching the VLM on or off on an existing database** doesn't require reingesting (the bytes are already there): **Switching modes on an existing database** doesn't require reingesting when the bytes are already stored:
- Off → on: `haiku-rag rebuild --descriptions` runs the VLM over stored bytes and re-chunks. Skips the docling parse entirely. - `image``description`: `haiku-rag rebuild --descriptions` runs the VLM over stored bytes and re-chunks. Skips the docling parse entirely.
- On → off: `haiku-rag rebuild --rechunk` recomposes chunk text from the stripped docling blob without descriptions. - `description``image`: `haiku-rag rebuild --rechunk` recomposes chunk text from the stripped docling blob without descriptions.
- Switching to/from `none`: a full reingest is needed since the bytes either weren't stored or need to be discarded.
When using `converter: docling-serve`, the VLM is invoked from docling-serve rather than haiku.rag — see [Remote processing](../remote-processing.md#vlm-picture-description-with-docling-serve). When using `converter: docling-serve`, the VLM is invoked from docling-serve rather than haiku.rag — see [Remote processing](../remote-processing.md#vlm-picture-description-with-docling-serve).
@ -161,37 +171,37 @@ Three independent settings drive ingest, retrieval, and QA:
| Setting | Question it answers | Values | | Setting | Question it answers | Values |
|---|---|---| |---|---|---|
| `picture_description.enabled` | Should a VLM weave descriptions into chunk text at ingest? | `false` (default) / `true` | | `processing.pictures` | Generate and/or describe pictures at ingest? | `none` / `description` / `image` (default) |
| `embeddings.model.provider` | Can the embedder index image content? | text-only (`ollama`, `openai`, `cohere`, `sentence-transformers`) vs `vllm` (multimodal) | | `embeddings.model.provider` | Can the embedder index image content? | text-only (`ollama`, `openai`, `cohere`, `sentence-transformers`) vs `vllm` (multimodal) |
| `qa.model.vision` | Can the QA model interpret images? | `false` (default) / `true` | | `qa.model.vision` | Can the QA model interpret images? | `false` (default) / `true` |
Picture bytes are always stored, regardless of these settings. **What gets stored** by `pictures` × embedder:
**What gets stored** by `enabled` × embedder: | `pictures` | Embedder | Text chunks | Synthetic picture chunks |
| `enabled` | Embedder | Text chunks | Synthetic picture chunks |
|---|---|---|---| |---|---|---|---|
| `false` | text-only | text only (caption/surrounding) | none | | `none` | any | text only (caption/surrounding) | none |
| `false` | multimodal | text only | one per picture, vector = image embedding | | `image` | text-only | text only (caption/surrounding) | none |
| `true` | text-only | text + descriptions | none | | `image` | multimodal | text only | one per picture, vector = image embedding |
| `true` | multimodal | text + descriptions | one per picture, vector = image embedding | | `description` | text-only | text + descriptions | none |
| `description` | multimodal | text + descriptions | one per picture, vector = image embedding |
**What QA receives** at search time: **What QA receives** at search time:
- `qa.model.vision: false` — text chunks only (descriptions, when present, answer figure questions in prose). - `qa.model.vision: false` — text chunks only (descriptions, when present, answer figure questions in prose).
- `qa.model.vision: true` — text chunks + raw picture bytes via `BinaryContent`; the model reads figures directly. - `qa.model.vision: true` — text chunks + raw picture bytes via `BinaryContent`; the model reads figures directly. Requires `pictures != none` so the bytes exist.
`qa.model.vision` is independent of ingestion — flipping it never requires reingesting. Setting `vision: true` against a text-only model causes silent acceptance and confabulation on Ollama and a 400 on OpenAI; default `false` is the safe choice. `qa.model.vision` is independent of ingestion — flipping it never requires reingesting. Setting `vision: true` against a text-only model causes silent acceptance and confabulation on Ollama and a 400 on OpenAI; default `false` is the safe choice.
**Recommended combinations:** **Recommended combinations:**
| Use case | `picture_description.enabled` | Embedder | `qa.model.vision` | | Use case | `processing.pictures` | Embedder | `qa.model.vision` |
|---|---|---|---| |---|---|---|---|
| Pure text RAG, no figures | `false` | text-only | `false` | | Pure text RAG, no figures, lowest RAM | `none` | text-only | `false` |
| Text RAG, figures answered through descriptions | `true` | text-only | `false` | | Text RAG, store figure bytes for later | `image` | text-only | `false` |
| Vision QA on figure-rich docs (no cross-modal search) | `true` or `false` | text-only | `true` | | Text RAG, figures answered through descriptions | `description` | text-only | `false` |
| Cross-modal search + vision QA | `true` or `false` | multimodal | `true` | | Vision QA on figure-rich docs (no cross-modal search) | `image` or `description` | text-only | `true` |
| Cross-modal search, text QA only | `true` | multimodal | `false` | | Cross-modal search + vision QA | `image` or `description` | multimodal | `true` |
| Cross-modal search, text QA only | `description` | multimodal | `false` |
### Automatic Title Generation ### Automatic Title Generation

View file

@ -119,7 +119,7 @@ prompts:
Be concise and factual. Be concise and factual.
``` ```
The prompt is used when `processing.conversion_options.picture_description.enabled` is `true`. See [Picture Handling](processing.md#picture-handling) for full configuration. The prompt is used when `processing.pictures` is `"description"`. See [Picture Handling](processing.md#picture-handling) for full configuration.
## Programmatic Configuration ## Programmatic Configuration

View file

@ -226,7 +226,7 @@ async for doc_id in client.rebuild_database(mode=RebuildMode.EMBED_ONLY):
# Add VLM picture descriptions to an existing database — runs the VLM # Add VLM picture descriptions to an existing database — runs the VLM
# over already-stored picture bytes, patches descriptions into the # over already-stored picture bytes, patches descriptions into the
# docling blob, then re-chunks + re-embeds. Requires # docling blob, then re-chunks + re-embeds. Requires
# picture_description.enabled=true in the config. # processing.pictures='description' in the config.
async for doc_id in client.rebuild_database(mode=RebuildMode.DESCRIPTIONS): async for doc_id in client.rebuild_database(mode=RebuildMode.DESCRIPTIONS):
print(f"Described pictures in {doc_id}") print(f"Described pictures in {doc_id}")
``` ```

View file

@ -498,7 +498,7 @@ def rebuild(
help=( help=(
"Run the VLM over already-stored picture bytes, patch descriptions " "Run the VLM over already-stored picture bytes, patch descriptions "
"into the docling blob, then re-chunk + re-embed. Skips the docling " "into the docling blob, then re-chunk + re-embed. Skips the docling "
"parse entirely. Requires picture_description.enabled=true." "parse entirely. Requires processing.pictures='description'."
), ),
), ),
): ):

View file

@ -105,7 +105,10 @@ async def download_models(
if config.reranking.model and config.reranking.model.provider == "ollama": if config.reranking.model and config.reranking.model.provider == "ollama":
required_models.add(config.reranking.model.name) required_models.add(config.reranking.model.name)
pic_desc = config.processing.conversion_options.picture_description pic_desc = config.processing.conversion_options.picture_description
if pic_desc.enabled and pic_desc.model.provider == "ollama": if (
config.processing.pictures == "description"
and pic_desc.model.provider == "ollama"
):
required_models.add(pic_desc.model.name) required_models.add(pic_desc.model.name)
if ( if (
config.processing.auto_title config.processing.auto_title

View file

@ -26,12 +26,12 @@ def _warn_if_descriptions_missing(
docling-serve swallows VLM errors (network failures, missing models, docling-serve swallows VLM errors (network failures, missing models,
etc.) and returns a successful conversion with empty descriptions. etc.) and returns a successful conversion with empty descriptions.
docling-local can do the same when the VLM endpoint is unreachable. docling-local can do the same when the VLM endpoint is unreachable.
Surface the silent failure: when ``picture_description.enabled=True`` Surface the silent failure: when ``processing.pictures="description"``
AND the document has at least one picture AND zero descriptions came AND the document has at least one picture AND zero descriptions came
back, log a clear warning so the user can fix their VLM config before back, log a clear warning so the user can fix their VLM config before
a thousand-document ingest produces an empty corpus. a thousand-document ingest produces an empty corpus.
""" """
if not config.processing.conversion_options.picture_description.enabled: if config.processing.pictures != "description":
return return
if not doc.pictures: if not doc.pictures:
return return
@ -39,7 +39,7 @@ def _warn_if_descriptions_missing(
if described == 0: if described == 0:
model = config.processing.conversion_options.picture_description.model model = config.processing.conversion_options.picture_description.model
logger.warning( logger.warning(
"picture_description.enabled is True but no descriptions came back " "processing.pictures='description' but no descriptions came back "
"for %s (%d pictures, 0 described). The VLM call likely failed " "for %s (%d pictures, 0 described). The VLM call likely failed "
"silently inside the converter. Check that the VLM at %s is " "silently inside the converter. Check that the VLM at %s is "
"reachable from the converter and that the model name '%s' " "reachable from the converter and that the model name '%s' "

View file

@ -383,10 +383,9 @@ async def _rebuild_descriptions(
""" """
from haiku.rag.embeddings import embed_chunks, get_embedder from haiku.rag.embeddings import embed_chunks, get_embedder
if not client._config.processing.conversion_options.picture_description.enabled: if client._config.processing.pictures != "description":
raise ValueError( raise ValueError(
"rebuild --descriptions requires " "rebuild --descriptions requires processing.pictures = 'description' "
"processing.conversion_options.picture_description.enabled = true "
"in your config." "in your config."
) )

View file

@ -47,30 +47,7 @@ def load_yaml_config(path: Path) -> dict:
"""Load and parse a YAML config file.""" """Load and parse a YAML config file."""
with open(path) as f: with open(path) as f:
data = yaml.safe_load(f) data = yaml.safe_load(f)
data = data or {} return data or {}
_drop_legacy_generate_picture_images(data)
return data
def _drop_legacy_generate_picture_images(data: dict) -> None:
"""Drop ``processing.conversion_options.generate_picture_images`` from
loaded YAML and log that it is no longer needed.
Picture bytes are always extracted now, so the flag has no effect.
Old YAMLs keep working; users see one log line telling them they can
remove the entry.
"""
processing = data.get("processing")
if not isinstance(processing, dict):
return
opts = processing.get("conversion_options")
if isinstance(opts, dict) and "generate_picture_images" in opts:
opts.pop("generate_picture_images", None)
logger.warning(
"Config: 'processing.conversion_options.generate_picture_images' "
"no longer has any effect (picture bytes are always extracted). "
"Remove it from your haiku.rag.yaml."
)
def generate_default_config() -> dict: def generate_default_config() -> dict:

View file

@ -129,14 +129,12 @@ class AnalysisConfig(BaseModel):
class PictureDescriptionConfig(BaseModel): class PictureDescriptionConfig(BaseModel):
"""Whether (and how) to run a VLM over each picture at ingest. """How the VLM runs over each picture when it runs at all.
When ``enabled`` is True, picture descriptions are generated by the Activation lives on ``ProcessingConfig.pictures`` these fields only
configured VLM and woven into chunk text. Picture bytes are stored describe *how* the VLM runs once ``pictures == "description"``.
regardless this flag only controls the description pass.
""" """
enabled: bool = False
model: ModelConfig = Field( model: ModelConfig = Field(
default_factory=lambda: ModelConfig( default_factory=lambda: ModelConfig(
provider="ollama", provider="ollama",
@ -177,6 +175,9 @@ class ConversionOptions(BaseModel):
) )
PicturesMode = Literal["none", "description", "image"]
class ProcessingConfig(BaseModel): class ProcessingConfig(BaseModel):
chunk_size: int = 256 chunk_size: int = 256
converter: str = "docling-local" converter: str = "docling-local"
@ -186,6 +187,21 @@ class ProcessingConfig(BaseModel):
chunking_merge_peers: bool = True chunking_merge_peers: bool = True
chunking_use_markdown_tables: bool = False chunking_use_markdown_tables: bool = False
conversion_options: ConversionOptions = Field(default_factory=ConversionOptions) conversion_options: ConversionOptions = Field(default_factory=ConversionOptions)
pictures: PicturesMode = "image"
"""How embedded pictures are handled at ingest.
- ``"none"``: docling skips picture-image generation; ``label="picture"``
rows still exist as structure but carry no bytes or description. Use
this when you don't need picture content and want to keep RAM and DB
size low on large documents.
- ``"description"``: docling generates picture images, the configured
VLM produces text descriptions woven into chunk text, AND the bytes
are retained in ``document_items.picture_data`` so a vision-capable
QA model or multimodal embedder can be enabled later without
reingesting.
- ``"image"``: docling generates picture images and stores them in
``document_items.picture_data``; no VLM runs at ingest.
"""
auto_title: bool = False auto_title: bool = False
title_model: ModelConfig = Field( title_model: ModelConfig = Field(
default_factory=lambda: ModelConfig( default_factory=lambda: ModelConfig(

View file

@ -119,14 +119,15 @@ class DoclingLocalConverter(DocumentConverter):
opts = self.config.processing.conversion_options opts = self.config.processing.conversion_options
pic_desc = opts.picture_description pic_desc = opts.picture_description
runs_vlm = pic_desc.enabled pictures = self.config.processing.pictures
runs_vlm = pictures == "description"
pipeline_options = PdfPipelineOptions( pipeline_options = PdfPipelineOptions(
do_ocr=opts.do_ocr, do_ocr=opts.do_ocr,
do_table_structure=opts.do_table_structure, do_table_structure=opts.do_table_structure,
images_scale=opts.images_scale, images_scale=opts.images_scale,
generate_page_images=opts.generate_page_images, generate_page_images=opts.generate_page_images,
generate_picture_images=True, generate_picture_images=pictures != "none",
table_structure_options=TableStructureOptions( table_structure_options=TableStructureOptions(
do_cell_matching=opts.table_cell_matching, do_cell_matching=opts.table_cell_matching,
mode=( mode=(

View file

@ -99,7 +99,8 @@ class DoclingServeConverter(DocumentConverter):
""" """
opts = self.config.processing.conversion_options opts = self.config.processing.conversion_options
pic_desc = opts.picture_description pic_desc = opts.picture_description
runs_vlm = pic_desc.enabled pictures = self.config.processing.pictures
runs_vlm = pictures == "description"
data: dict[str, str | list[str]] = { data: dict[str, str | list[str]] = {
"to_formats": "json", "to_formats": "json",
@ -111,7 +112,7 @@ class DoclingServeConverter(DocumentConverter):
"table_cell_matching": str(opts.table_cell_matching).lower(), "table_cell_matching": str(opts.table_cell_matching).lower(),
"images_scale": str(opts.images_scale), "images_scale": str(opts.images_scale),
"image_export_mode": "referenced", "image_export_mode": "referenced",
"include_images": "true", "include_images": str(pictures != "none").lower(),
"do_picture_description": str(runs_vlm).lower(), "do_picture_description": str(runs_vlm).lower(),
"target_type": "zip", "target_type": "zip",
} }

View file

@ -1,5 +1,3 @@
import logging
import pytest import pytest
import yaml import yaml
@ -9,16 +7,6 @@ from haiku.rag.config.loader import (
generate_default_config, generate_default_config,
load_yaml_config, load_yaml_config,
) )
from haiku.rag.config.loader import logger as loader_logger
class _ListHandler(logging.Handler):
def __init__(self) -> None:
super().__init__(level=logging.WARNING)
self.records: list[logging.LogRecord] = []
def emit(self, record: logging.LogRecord) -> None:
self.records.append(record)
def test_load_yaml_config(tmp_path): def test_load_yaml_config(tmp_path):
@ -239,39 +227,25 @@ def test_init_config_creates_valid_yaml(tmp_path):
assert config.environment == "production" assert config.environment == "production"
# Removed-field handling: processing.conversion_options.generate_picture_images
# was honored by earlier releases. It's dropped from loaded YAML now and a
# warning is logged so the user knows they can remove it from their config.
def _write(tmp_path, body: str): def _write(tmp_path, body: str):
p = tmp_path / "haiku.rag.yaml" p = tmp_path / "haiku.rag.yaml"
p.write_text(body) p.write_text(body)
return p return p
def test_load_yaml_drops_generate_picture_images(tmp_path): def test_pictures_field_defaults_to_image():
"""``generate_picture_images`` from earlier releases is dropped from """`processing.pictures` defaults to `'image'` — current ingest behavior
the parsed dict and a warning tells the user to remove it from the (store picture bytes, no VLM) preserved on fresh configs."""
YAML.""" cfg = AppConfig()
config_file = _write( assert cfg.processing.pictures == "image"
tmp_path,
"""
processing: def test_picture_description_has_no_enabled_field():
conversion_options: """`enabled` is gone from PictureDescriptionConfig — activation is now
generate_picture_images: true controlled by `processing.pictures == 'description'`."""
""", from haiku.rag.config.models import PictureDescriptionConfig
)
handler = _ListHandler() assert "enabled" not in PictureDescriptionConfig.model_fields
loader_logger.addHandler(handler)
try:
data = load_yaml_config(config_file)
finally:
loader_logger.removeHandler(handler)
cfg = AppConfig.model_validate(data)
assert cfg.processing.conversion_options.picture_description.enabled is False
assert "generate_picture_images" not in data["processing"]["conversion_options"]
assert any("generate_picture_images" in r.getMessage() for r in handler.records)
def test_fetch_remote_images_default_true(): def test_fetch_remote_images_default_true():

View file

@ -574,7 +574,7 @@ class TestDoclingLocalConverter:
def test_image_format_shares_pdf_pipeline_options(self, config): def test_image_format_shares_pdf_pipeline_options(self, config):
"""IMAGE FormatOption shares the same PdfPipelineOptions instance as """IMAGE FormatOption shares the same PdfPipelineOptions instance as
PDF. Without this, `do_ocr` / `picture_description.enabled` / etc. PDF. Without this, `do_ocr` / `processing.pictures='description'` / etc.
silently no-op when ingesting raw `.png` / `.jpg` files (which run silently no-op when ingesting raw `.png` / `.jpg` files (which run
through StandardPdfPipeline). End-to-end image conversion is covered through StandardPdfPipeline). End-to-end image conversion is covered
by the PDF picture test both paths feed the same pipeline class.""" by the PDF picture test both paths feed the same pipeline class."""
@ -785,7 +785,7 @@ class TestDoclingLocalConverter:
def test_picture_description_config_defaults(self, config): def test_picture_description_config_defaults(self, config):
"""Test that picture description config has correct defaults.""" """Test that picture description config has correct defaults."""
pic_desc = config.processing.conversion_options.picture_description pic_desc = config.processing.conversion_options.picture_description
assert pic_desc.enabled is False assert config.processing.pictures == "image"
assert pic_desc.model.provider == "ollama" assert pic_desc.model.provider == "ollama"
assert pic_desc.model.name == "ministral-3" assert pic_desc.model.name == "ministral-3"
assert pic_desc.timeout == 90 assert pic_desc.timeout == 90
@ -795,12 +795,12 @@ class TestDoclingLocalConverter:
def test_picture_description_config_applied(self, config): def test_picture_description_config_applied(self, config):
"""Test that picture description config is applied to converter.""" """Test that picture description config is applied to converter."""
config.processing.conversion_options.picture_description.enabled = True config.processing.pictures = "description"
config.processing.conversion_options.picture_description.timeout = 120 config.processing.conversion_options.picture_description.timeout = 120
converter = DoclingLocalConverter(config) converter = DoclingLocalConverter(config)
assert converter.config.processing.pictures == "description"
pic_desc = converter.config.processing.conversion_options.picture_description pic_desc = converter.config.processing.conversion_options.picture_description
assert pic_desc.enabled is True
assert pic_desc.timeout == 120 assert pic_desc.timeout == 120
@pytest.mark.asyncio @pytest.mark.asyncio
@ -812,7 +812,7 @@ class TestDoclingLocalConverter:
# Disable OCR (not needed for native PDF, avoids model downloads) # Disable OCR (not needed for native PDF, avoids model downloads)
config.processing.conversion_options.do_ocr = False config.processing.conversion_options.do_ocr = False
# Enable picture description with Ollama # Enable picture description with Ollama
config.processing.conversion_options.picture_description.enabled = True config.processing.pictures = "description"
config.processing.conversion_options.picture_description.model.provider = ( config.processing.conversion_options.picture_description.model.provider = (
"ollama" "ollama"
) )
@ -1221,7 +1221,7 @@ class TestDoclingServeConverterPictureDescription:
VLM is enabled.""" VLM is enabled."""
import json import json
config.processing.conversion_options.picture_description.enabled = True config.processing.pictures = "description"
config.processing.conversion_options.picture_description.model.provider = ( config.processing.conversion_options.picture_description.model.provider = (
"ollama" "ollama"
) )
@ -1332,7 +1332,7 @@ class TestDoclingServeConverterIntegration:
Note: Not using VCR because this test involves polling with changing task IDs. Note: Not using VCR because this test involves polling with changing task IDs.
""" """
pdf_path = Path("tests/data/doclaynet.pdf") pdf_path = Path("tests/data/doclaynet.pdf")
config.processing.conversion_options.picture_description.enabled = True config.processing.pictures = "description"
config.processing.conversion_options.picture_description.model.provider = ( config.processing.conversion_options.picture_description.model.provider = (
"ollama" "ollama"
) )

View file

@ -65,10 +65,10 @@ def caplog_warnings(caplog):
def test_no_warning_when_picture_description_disabled(caplog_warnings): def test_no_warning_when_picture_description_disabled(caplog_warnings):
"""Default config has picture_description.enabled=False — even a """Default config has pictures='image' — even a document full of
document full of pictures with no descriptions should not warn.""" pictures with no descriptions should not warn."""
config = AppConfig() config = AppConfig()
assert config.processing.conversion_options.picture_description.enabled is False assert config.processing.pictures == "image"
doc = _doc_with_pictures(with_descriptions=False) doc = _doc_with_pictures(with_descriptions=False)
_warn_if_descriptions_missing(config, doc, "fake.pdf") _warn_if_descriptions_missing(config, doc, "fake.pdf")
@ -80,7 +80,7 @@ def test_no_warning_when_doc_has_no_pictures(caplog_warnings):
"""A picture-less document under enabled=True shouldn't warn — there """A picture-less document under enabled=True shouldn't warn — there
was simply nothing to describe.""" was simply nothing to describe."""
config = AppConfig() config = AppConfig()
config.processing.conversion_options.picture_description.enabled = True config.processing.pictures = "description"
doc = _doc_without_pictures() doc = _doc_without_pictures()
_warn_if_descriptions_missing(config, doc, "no-pictures.txt") _warn_if_descriptions_missing(config, doc, "no-pictures.txt")
@ -89,12 +89,12 @@ def test_no_warning_when_doc_has_no_pictures(caplog_warnings):
def test_warns_when_pictures_present_but_no_descriptions(caplog_warnings): def test_warns_when_pictures_present_but_no_descriptions(caplog_warnings):
"""VLM was requested via ``picture_description.enabled = True``, the """VLM was requested via ``processing.pictures='description'``, the
doc has pictures, but the converter returned zero descriptions doc has pictures, but the converter returned zero descriptions
(docling-serve swallows VLM errors). Warn loudly so the user can fix (docling-serve swallows VLM errors). Warn loudly so the user can fix
their VLM config before a long ingest.""" their VLM config before a long ingest."""
config = AppConfig() config = AppConfig()
config.processing.conversion_options.picture_description.enabled = True config.processing.pictures = "description"
config.processing.conversion_options.picture_description.model.name = "qwen3.6" config.processing.conversion_options.picture_description.model.name = "qwen3.6"
config.processing.conversion_options.picture_description.model.base_url = ( config.processing.conversion_options.picture_description.model.base_url = (
"http://host.docker.internal:11434" "http://host.docker.internal:11434"
@ -117,7 +117,7 @@ def test_no_warning_when_at_least_one_description_came_back(caplog_warnings):
and does not warn the user might have area-threshold filtering or and does not warn the user might have area-threshold filtering or
classification gating.""" classification gating."""
config = AppConfig() config = AppConfig()
config.processing.conversion_options.picture_description.enabled = True config.processing.pictures = "description"
doc = _doc_with_pictures(with_descriptions=True) doc = _doc_with_pictures(with_descriptions=True)
_warn_if_descriptions_missing(config, doc, "fine.pdf") _warn_if_descriptions_missing(config, doc, "fine.pdf")
@ -160,7 +160,7 @@ async def test_convert_emits_warning_via_chokepoint(
) )
config = AppConfig() config = AppConfig()
config.processing.conversion_options.picture_description.enabled = True config.processing.pictures = "description"
await convert(config, pdf) await convert(config, pdf)
@ -201,7 +201,7 @@ async def test_convert_text_path_also_warns(monkeypatch, caplog_warnings):
) )
config = AppConfig() config = AppConfig()
config.processing.conversion_options.picture_description.enabled = True config.processing.pictures = "description"
# No URL scheme, no Path → drops into the convert_text branch. # No URL scheme, no Path → drops into the convert_text branch.
await convert(config, "<html><img src='...'/></html>") await convert(config, "<html><img src='...'/></html>")

View file

@ -427,12 +427,12 @@ async def test_rebuild_batch_size_flush(temp_db_path, monkeypatch):
assert len(chunks) > 0 assert len(chunks) > 0
async def test_rebuild_descriptions_requires_enabled(temp_db_path): async def test_rebuild_descriptions_requires_description_mode(temp_db_path):
"""Calling rebuild --descriptions without picture_description.enabled is """Calling rebuild --descriptions without `processing.pictures='description'`
a config error: the user has nothing to gain and the resulting state is is a config error: the user has nothing to gain and the resulting state is
indistinguishable from a plain --rechunk.""" indistinguishable from a plain --rechunk."""
async with HaikuRAG(temp_db_path, create=True) as client: async with HaikuRAG(temp_db_path, create=True) as client:
with pytest.raises(ValueError, match="picture_description.enabled"): with pytest.raises(ValueError, match="processing.pictures"):
async for _ in client.rebuild_database(mode=RebuildMode.DESCRIPTIONS): async for _ in client.rebuild_database(mode=RebuildMode.DESCRIPTIONS):
pass pass
@ -459,7 +459,7 @@ async def test_rebuild_descriptions_patches_blob_and_chunks(temp_db_path, monkey
docling_doc = _docling_doc_with_picture() docling_doc = _docling_doc_with_picture()
config = AppConfig() config = AppConfig()
config.processing.conversion_options.picture_description.enabled = True config.processing.pictures = "description"
async with HaikuRAG(temp_db_path, config=config, create=True) as rag: async with HaikuRAG(temp_db_path, config=config, create=True) as rag:
document = Document(content="x", uri="test://doc") document = Document(content="x", uri="test://doc")
@ -543,7 +543,7 @@ async def test_rebuild_descriptions_skips_already_described(temp_db_path, monkey
) )
config = AppConfig() config = AppConfig()
config.processing.conversion_options.picture_description.enabled = True config.processing.pictures = "description"
async with HaikuRAG(temp_db_path, config=config, create=True) as rag: async with HaikuRAG(temp_db_path, config=config, create=True) as rag:
document = Document(content="x", uri="test://doc") document = Document(content="x", uri="test://doc")
@ -589,7 +589,7 @@ async def test_patch_picture_descriptions_returns_zero_for_doc_without_pictures(
from haiku.rag.config import AppConfig from haiku.rag.config import AppConfig
config = AppConfig() config = AppConfig()
config.processing.conversion_options.picture_description.enabled = True config.processing.pictures = "description"
async with HaikuRAG(temp_db_path, config=config, create=True) as rag: async with HaikuRAG(temp_db_path, config=config, create=True) as rag:
doc = await rag.create_document(content="Just text, no pictures.") doc = await rag.create_document(content="Just text, no pictures.")
@ -613,7 +613,7 @@ async def test_patch_picture_descriptions_warns_on_missing_bytes(temp_db_path, c
docling_doc = _docling_doc_with_picture() docling_doc = _docling_doc_with_picture()
config = AppConfig() config = AppConfig()
config.processing.conversion_options.picture_description.enabled = True config.processing.pictures = "description"
async with HaikuRAG(temp_db_path, config=config, create=True) as rag: async with HaikuRAG(temp_db_path, config=config, create=True) as rag:
document = Document(content="x", uri="test://doc") document = Document(content="x", uri="test://doc")
@ -670,7 +670,7 @@ async def test_patch_picture_descriptions_skips_when_all_already_described(
) )
config = AppConfig() config = AppConfig()
config.processing.conversion_options.picture_description.enabled = True config.processing.pictures = "description"
async with HaikuRAG(temp_db_path, config=config, create=True) as rag: async with HaikuRAG(temp_db_path, config=config, create=True) as rag:
document = Document(content="x", uri="test://doc") document = Document(content="x", uri="test://doc")
@ -708,7 +708,7 @@ async def test_rebuild_descriptions_raises_when_blob_is_missing(
docling_doc = _docling_doc_with_picture() docling_doc = _docling_doc_with_picture()
config = AppConfig() config = AppConfig()
config.processing.conversion_options.picture_description.enabled = True config.processing.pictures = "description"
async with HaikuRAG(temp_db_path, config=config, create=True) as rag: async with HaikuRAG(temp_db_path, config=config, create=True) as rag:
document = Document(content="x", uri="test://doc") document = Document(content="x", uri="test://doc")