diff --git a/CHANGELOG.md b/CHANGELOG.md index de459832..aee4c129 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,12 +8,7 @@ - **`vision: bool` flag on `ModelConfig`.** Tracks whether a configured language model can interpret images. Default `False`. The agent's `search` tool only attaches picture bytes (as `BinaryContent`) to the `ToolReturn` when `qa.model.vision = True`. Without the gate, sending image content to a text-only model behaves inconsistently across providers — Ollama silently accepts and the model hallucinates a confident wrong answer; OpenAI returns 400; others vary. Capability detection from a probe or a model-name whitelist is unreliable, so `vision` is an explicit user-set capability declaration. Set it to `True` for vision-capable QA models (`qwen2.5vl`, `qwen3.6`, `gpt-4o`, `claude-sonnet`, ...). - **Synthetic picture chunks at ingest under multimodal embedders.** `build_picture_chunks` (in `client/processing.py`) walks a `DoclingDocument`'s `pictures` and emits one synthetic `Chunk` per `PictureItem` with available bytes. Bytes ride on a `Chunk._picture_data` PrivateAttr (not serialized) so `embed_chunks` can route them through `embed_images` while text chunks keep going through `embed_documents`. Wired into the three ingest paths (`create_document`, `_create_document_from_file`, `_create_or_update_document_from_url`, `_update_document_with_chunks`, and `_rebuild_rechunk`) — guarded by `embedder.supports_images` so text-only configurations are unaffected. Snapshot/merge with `existing_picture_data` keeps picture chunks alive across rebuild round-trips. Picture chunks land in the same `chunks` table with the same vector dim as text chunks, so cross-modal search reuses the existing hybrid+RRF pipeline. - **Multimodal embedder support (`provider="vllm"`).** `EmbedderWrapper` gains `supports_images: bool` and `embed_image_query`. The `vllm` provider talks HTTP to a vLLM server's OpenAI-compatible `/v1/embeddings` endpoint — text inputs use the standard `input` field for true server-side batching; image inputs use vLLM's `messages` superset with `image_url` content parts carrying base64 data URIs. Works with `Qwen/Qwen3-VL-Embedding-8B` and `jinaai/jina-embeddings-v4`. No Python ML deps added — uses `httpx`. Text-only providers (`ollama`, `openai`, `cohere`, `sentence-transformers`) report `supports_images=False` and raise a clear error if image methods are called. -- **Picture-handling mode enum.** `processing.pictures: "none" | "description" | "image"` (default `"none"`) replaces the legacy `generate_picture_images` + `picture_description.enabled` pair. - - `none` — no picture bytes, no VLM, just structural picture rows. - - `description` — VLM runs at ingest, descriptions woven into chunk text, bytes also retained on `document_items.picture_data` so vision QA can be turned on later without reingesting. - - `image` — bytes retained, no VLM. - - YAML loader translates legacy configs and emits a deprecation warning. - - Rebuild and update under `pictures="none"` skip the picture-bytes snapshot/merge so storage is reclaimed. +- **Picture bytes are always stored.** Every ingest captures embedded picture bytes into `document_items.picture_data`; rebuilds and updates always preserve them. The single ingest-time decision is whether to run a VLM at ingest, exposed via `processing.conversion_options.picture_description.enabled: bool` (default `False`). When enabled, descriptions are woven into chunk text; when disabled, chunks contain only their natural text. No reingest is required to flip the VLM on or off — `haiku-rag rebuild --rechunk` recomputes chunk text against the new setting. The legacy `processing.pictures: "none" | "description" | "image"` enum and `processing.conversion_options.generate_picture_images` flag are auto-translated by the YAML loader with a one-time deprecation warning per field. - **Storage column for embedded picture bytes.** `DocumentItemRecord.picture_data: bytes | None` (Arrow `large_binary`), addressable by `(document_id, self_ref)`. - New accessors `get_picture_bytes`, `get_pictures_for_chunk`, `get_all_picture_data` on `DocumentItemRepository`. - Bulk read paths project a metadata-only column set so context expansion and the analysis-sandbox `items.jsonl` build never pull picture bytes into memory. diff --git a/docs/configuration/processing.md b/docs/configuration/processing.md index b45e6a12..8c272c5e 100644 --- a/docs/configuration/processing.md +++ b/docs/configuration/processing.md @@ -28,9 +28,6 @@ processing: name: gpt-oss enable_thinking: false - # Picture handling (none / description / image) - pictures: none # See "Picture Handling" below - # Conversion options (works with both local and remote converters) conversion_options: # OCR settings @@ -48,8 +45,9 @@ processing: images_scale: 2.0 # Image scale factor generate_page_images: true # Include rendered page images (for visualize_chunk) - # VLM picture description settings (only effective when pictures: description) + # VLM picture description (off by default; see "Picture Handling" below) picture_description: + enabled: false model: provider: ollama name: ministral-3 @@ -106,27 +104,15 @@ conversion_options: - **images_scale**: Scale factor for extracted images. Higher values = better quality but larger size. Typical range: 1.0-3.0. - **generate_page_images**: When `true` (default), rendered images of each PDF page are included in the document. Required for `visualize_chunk()` to show visual grounding. When `false`, page images are excluded to reduce document size. -Embedded picture extraction is controlled by `processing.pictures` (see [Picture Handling](#picture-handling) below), not by an image-settings flag. - #### Picture Handling -`processing.pictures` is a single enum that decides how embedded picture images (figures, diagrams) are handled at ingest. Three modes: - -| Mode | VLM at ingest | `picture_data` populated | Chunk text contains description | -|---|---|---|---| -| `none` (default) | no | no | no | -| `description` | yes | yes | yes | -| `image` | no | yes | no | - -- **`none`**: docling skips picture-image generation. `label="picture"` rows still appear in the items table for structural metadata, but they carry no bytes and no description. Cheapest mode; non-vision QA is unaffected. -- **`description`**: docling generates picture images, the configured VLM produces a description woven into chunk text, and the bytes are also retained in `document_items.picture_data`. Picture-text is searchable via FTS, vision-capable QA models also receive the bytes via the agent's search tool. Bytes are kept (not just thrown away after the VLM runs) so a vision-only QA strategy can be turned on later without reingesting. -- **`image`**: docling generates picture images and stores them in `document_items.picture_data` without running the VLM. Vision-capable QA models reason directly about the figures; non-vision QA only sees the picture's caption/surrounding text. +Picture bytes (figures, diagrams) are always extracted and stored in `document_items.picture_data` for every ingested document. The single configurable knob is whether a Vision Language Model (VLM) runs at ingest time to generate textual descriptions, set via `picture_description.enabled`: ```yaml processing: - pictures: description # none / description / image conversion_options: - picture_description: # only effective when pictures: description + picture_description: + enabled: true # default false; runs the VLM at ingest model: provider: ollama # ollama, openai, or custom name: ministral-3 # VLM model name @@ -135,56 +121,55 @@ processing: max_tokens: 200 # Maximum tokens in response ``` -**Switching modes on an existing database.** No reingest is required if you only need to change between `description` and `image` — the bytes are already there. Run `haiku-rag rebuild --rechunk` after the config change so the chunk-text composition reflects the new mode. Switching *down* to `none` clears `picture_data` on rebuild, reclaiming storage. +When `enabled: false` (default), the VLM doesn't run; chunks contain only their natural text (captions, surrounding paragraphs). When `enabled: true`, each picture's description is woven into the chunk text and is searchable via FTS. -#### Pictures × embedder × QA model: how the pieces compose +**Switching the VLM on or off on an existing database.** Picture bytes are already stored, so no reingest is required. Run `haiku-rag rebuild --rechunk` after flipping `enabled` so the chunk-text composition reflects the new setting. -Three orthogonal settings drive what gets stored, what gets retrieved, and what reaches the QA model. Each setting answers one question: +#### Picture descriptions × embedder × QA model: how the pieces compose + +Three settings drive what gets stored, what gets retrieved, and what reaches the QA model: | Setting | Question it answers | Values | |---|---|---| -| `processing.pictures` | What gets captured at ingest? | `none` / `description` / `image` | +| `picture_description.enabled` | Should a VLM weave descriptions into chunk text at ingest? | `false` (default) / `true` | | `embeddings.model.provider` | Can the embedder index image content? | text-only providers (`ollama`, `openai`, `cohere`, `sentence-transformers`) vs `vllm` (multimodal) | | `qa.model.vision` | Can the QA model interpret images? | `false` (default) / `true` | -**What gets stored** for each `pictures` × embedder combination: +Picture bytes are always stored, regardless of these settings. -| `pictures` | Embedder | Text chunks contain… | `document_items.picture_data` | Synthetic picture chunks | -|---|---|---|---|---| -| `none` | text-only or multimodal | regular text only | empty | none | -| `description` | text-only | text + VLM descriptions | populated (kept for later use) | none | -| `description` | multimodal | text + VLM descriptions | populated | one per picture, content = description, vector = image embedding | -| `image` | text-only | text only (caption/surrounding) | populated (kept for later use) | none | -| `image` | multimodal | text only | populated | one per picture, content = caption/empty, vector = image embedding | +**What gets stored** for each `picture_description.enabled` × embedder combination: + +| `enabled` | Embedder | Text chunks contain… | Synthetic picture chunks | +|---|---|---|---| +| `false` | text-only | text only (caption/surrounding) | none | +| `false` | multimodal | text only | one per picture, content = caption/empty, vector = image embedding | +| `true` | text-only | text + VLM descriptions | none | +| `true` | multimodal | text + VLM descriptions | one per picture, content = description, vector = image embedding | **What QA receives** at search time, given stored state and `qa.model.vision`: -| `pictures` at ingest | `qa.model.vision` | QA receives | -|---|---|---| -| `none` | either | text chunks only | -| `description` | `false` | text chunks (descriptions in chunk text answer figure questions in prose) | -| `description` | `true` | text chunks + raw picture bytes; vision model uses both signals | -| `image` | `false` | text chunks only (caption + surrounding text); model has no figure content to draw on | -| `image` | `true` | text chunks + raw picture bytes; vision model reads the figures directly | +| `qa.model.vision` | QA receives | +|---|---| +| `false` | text chunks only (descriptions in chunk text answer figure questions in prose when `picture_description.enabled` was true) | +| `true` | text chunks + raw picture bytes; vision model reads the figures directly | A few invariants worth knowing: - **`qa.model.vision` is independent of ingestion.** It only controls whether the agent's `search` tool attaches picture bytes to its `ToolReturn`. A text-only QA model with `vision: true` won't suddenly understand images — it will silently accept the bytes and confabulate. Default `false` is the safe choice. -- **`description` mode preserves the bytes**, so flipping the QA strategy later (text-only → vision, or vice versa) doesn't require reingesting. Just change `qa.model.vision` and optionally `qa.model` itself. +- **The bytes are always there**, so flipping the QA strategy later (text-only → vision, or vice versa) doesn't require reingesting. Just change `qa.model.vision` and optionally `qa.model` itself. - **Cross-modal search** (text query → picture-chunk hits) requires a multimodal embedder. With a text-only embedder, picture-chunk vectors aren't generated; figures only surface via section-bounded expansion off matching text chunks. -- **`image` mode + text-only embedder** is rarely the right choice — pictures are stored but neither searchable as image vectors nor described in text. Picture bytes are then only reachable via expand-context's section bounds when an adjacent text chunk matches. **Recommended combinations** by use case: -| Use case | `pictures` | Embedder | `qa.model.vision` | +| Use case | `picture_description.enabled` | Embedder | `qa.model.vision` | |---|---|---|---| -| Pure text RAG, no figures | `none` | text-only | `false` | -| Text RAG, figures answered through descriptions | `description` | text-only | `false` | -| Vision QA on figure-rich docs (no cross-modal search) | `description` | text-only | `true` | -| Cross-modal search + vision QA (the full multimodal stack) | `description` or `image` | multimodal | `true` | -| Cross-modal search, text QA only | `description` | multimodal | `false` | +| Pure text RAG, no figures | `false` | text-only | `false` | +| Text RAG, figures answered through descriptions | `true` | text-only | `false` | +| Vision QA on figure-rich docs (no cross-modal search) | `true` or `false` | text-only | `true` | +| Cross-modal search + vision QA (the full multimodal stack) | `true` or `false` | multimodal | `true` | +| Cross-modal search, text QA only | `true` | multimodal | `false` | -**`picture_description.model` configuration** (used only under `pictures: description`): +**`picture_description.model` configuration** (used only when `picture_description.enabled: true`): - **model**: Standard model configuration - `provider`: `ollama` (default), `openai`, or use `base_url` for custom endpoints @@ -214,9 +199,9 @@ prompts: ```yaml processing: - pictures: description conversion_options: picture_description: + enabled: true model: provider: ollama name: ministral-3 @@ -233,9 +218,9 @@ ollama serve ```yaml processing: - pictures: description conversion_options: picture_description: + enabled: true model: provider: openai # Use OpenAI-compatible API format name: granite-vision @@ -260,9 +245,9 @@ When using `converter: docling-serve`, the VLM calls are made by the docling-ser ```yaml processing: - pictures: description conversion_options: picture_description: + enabled: true model: provider: ollama name: ministral-3 diff --git a/docs/configuration/prompts.md b/docs/configuration/prompts.md index 51bf6956..f0fac0d7 100644 --- a/docs/configuration/prompts.md +++ b/docs/configuration/prompts.md @@ -119,7 +119,7 @@ prompts: Be concise and factual. ``` -The prompt is used when `processing.pictures` is set to `description`. See [Picture Handling](processing.md#picture-handling) for full configuration. +The prompt is used when `processing.conversion_options.picture_description.enabled` is `true`. See [Picture Handling](processing.md#picture-handling) for full configuration. ## Programmatic Configuration diff --git a/haiku_rag_slim/haiku/rag/client/downloads.py b/haiku_rag_slim/haiku/rag/client/downloads.py index 62bd7611..9578cfbb 100644 --- a/haiku_rag_slim/haiku/rag/client/downloads.py +++ b/haiku_rag_slim/haiku/rag/client/downloads.py @@ -105,10 +105,7 @@ async def download_models( if config.reranking.model and config.reranking.model.provider == "ollama": required_models.add(config.reranking.model.name) pic_desc = config.processing.conversion_options.picture_description - if ( - config.processing.pictures == "description" - and pic_desc.model.provider == "ollama" - ): + if pic_desc.enabled and pic_desc.model.provider == "ollama": required_models.add(pic_desc.model.name) if ( config.processing.auto_title diff --git a/haiku_rag_slim/haiku/rag/config/loader.py b/haiku_rag_slim/haiku/rag/config/loader.py index 5205bfde..ca853b27 100644 --- a/haiku_rag_slim/haiku/rag/config/loader.py +++ b/haiku_rag_slim/haiku/rag/config/loader.py @@ -53,56 +53,54 @@ def load_yaml_config(path: Path) -> dict: def _translate_legacy_picture_fields(data: dict) -> None: - """Map legacy picture knobs onto ``processing.pictures``. + """Map legacy picture-handling knobs onto + ``processing.conversion_options.picture_description.enabled``. - Older configs expressed the same intent through two booleans on - ``conversion_options``: ``generate_picture_images`` and - ``picture_description.enabled``. Translation, in priority order: + Two earlier shapes need translating: - - ``picture_description.enabled = true`` (regardless of the image flag) - → ``pictures = "description"``. - - ``generate_picture_images = true`` (and no description) → ``"image"``. - - both false / missing → no translation; default ``"none"`` applies. + - ``processing.pictures: "description"`` → + ``picture_description.enabled = true``. The other values + (``"none"``, ``"image"``) collapse to ``false`` since the only + remaining decision is whether the VLM runs; picture bytes are + always stored. + - ``processing.conversion_options.generate_picture_images: `` is a + no-op now (docling always extracts picture bytes) and is dropped + with a one-time warning. - If ``pictures`` is already set on the loaded YAML it wins. Mutates - ``data`` in-place and emits one warning per legacy field encountered. + If ``picture_description.enabled`` is already explicitly set on the + loaded YAML, it wins. Mutates ``data`` in-place and emits one warning + per legacy field encountered. """ processing = data.get("processing") if not isinstance(processing, dict): return - if "pictures" in processing: - # User has migrated; legacy fields may still be present but should not - # override the explicit choice. Drop them silently to avoid confusion. - opts = processing.get("conversion_options") - if isinstance(opts, dict): - opts.pop("generate_picture_images", None) - pic = opts.get("picture_description") - if isinstance(pic, dict): - pic.pop("enabled", None) - return + legacy_pictures = processing.pop("pictures", None) + if legacy_pictures is not None: + opts = processing.setdefault("conversion_options", {}) + if not isinstance(opts, dict): + opts = {} + processing["conversion_options"] = opts + pic = opts.setdefault("picture_description", {}) + if not isinstance(pic, dict): + pic = {} + opts["picture_description"] = pic + if "enabled" not in pic: + pic["enabled"] = legacy_pictures == "description" + logger.warning( + "Config: 'processing.pictures' is deprecated; use " + "'processing.conversion_options.picture_description.enabled' " + "instead. Picture bytes are now always stored. Please update " + "your haiku.rag.yaml." + ) opts = processing.get("conversion_options") - if not isinstance(opts, dict): - return - - pic = opts.get("picture_description") if isinstance(opts, dict) else None - legacy_describe = pic.pop("enabled", None) if isinstance(pic, dict) else None - legacy_image = opts.pop("generate_picture_images", None) - - if legacy_describe: - processing["pictures"] = "description" + if isinstance(opts, dict) and "generate_picture_images" in opts: + opts.pop("generate_picture_images", None) logger.warning( - "Config: 'processing.conversion_options.picture_description.enabled=true' is " - "deprecated; mapped to 'processing.pictures: description'. Please update your " - "haiku.rag.yaml." - ) - elif legacy_image: - processing["pictures"] = "image" - logger.warning( - "Config: 'processing.conversion_options.generate_picture_images=true' is " - "deprecated; mapped to 'processing.pictures: image'. Please update your " - "haiku.rag.yaml." + "Config: 'processing.conversion_options.generate_picture_images' " + "is deprecated and ignored; picture bytes are always extracted. " + "Please update your haiku.rag.yaml." ) diff --git a/haiku_rag_slim/haiku/rag/config/models.py b/haiku_rag_slim/haiku/rag/config/models.py index 34404473..29d36407 100644 --- a/haiku_rag_slim/haiku/rag/config/models.py +++ b/haiku_rag_slim/haiku/rag/config/models.py @@ -112,12 +112,14 @@ class AnalysisConfig(BaseModel): class PictureDescriptionConfig(BaseModel): - """Settings for the VLM that runs at ingest under ``pictures="description"``. + """Whether (and how) to run a VLM over each picture at ingest. - Whether the VLM runs at all is decided by ``ProcessingConfig.pictures``; - these fields only describe *how* it runs once enabled. + When ``enabled`` is True, picture descriptions are generated by the + configured VLM and woven into chunk text. Picture bytes are stored + regardless — this flag only controls the description pass. """ + enabled: bool = False model: ModelConfig = Field( default_factory=lambda: ModelConfig( provider="ollama", @@ -149,15 +151,11 @@ class ConversionOptions(BaseModel): images_scale: float = 2.0 generate_page_images: bool = True - # VLM picture description (only effective when ProcessingConfig.pictures == "description") picture_description: PictureDescriptionConfig = Field( default_factory=PictureDescriptionConfig ) -PicturesMode = Literal["none", "description", "image"] - - class ProcessingConfig(BaseModel): chunk_size: int = 256 converter: str = "docling-local" @@ -167,18 +165,6 @@ class ProcessingConfig(BaseModel): chunking_merge_peers: bool = True chunking_use_markdown_tables: bool = False conversion_options: ConversionOptions = Field(default_factory=ConversionOptions) - pictures: PicturesMode = "none" - """How embedded pictures are handled at ingest. - - - ``"none"``: docling skips picture-image generation; structural - ``label="picture"`` rows still exist but carry no bytes or description. - - ``"description"``: docling generates picture images, the VLM produces - text descriptions woven into chunk text, and the bytes are also - retained in ``document_items.picture_data`` so a vision-capable QA - model can be turned on later without reingesting. - - ``"image"``: docling generates picture images and stores them in - ``document_items.picture_data``; no VLM runs at ingest. - """ auto_title: bool = False title_model: ModelConfig = Field( default_factory=lambda: ModelConfig( diff --git a/haiku_rag_slim/haiku/rag/converters/docling_local.py b/haiku_rag_slim/haiku/rag/converters/docling_local.py index 982c8b87..baf7456e 100644 --- a/haiku_rag_slim/haiku/rag/converters/docling_local.py +++ b/haiku_rag_slim/haiku/rag/converters/docling_local.py @@ -123,7 +123,7 @@ class DoclingLocalConverter(DocumentConverter): opts = self.config.processing.conversion_options pic_desc = opts.picture_description - runs_vlm = self.config.processing.pictures == "description" + runs_vlm = pic_desc.enabled pipeline_options = PdfPipelineOptions( do_ocr=opts.do_ocr, diff --git a/haiku_rag_slim/haiku/rag/converters/docling_serve.py b/haiku_rag_slim/haiku/rag/converters/docling_serve.py index f89df3cd..ea3fe173 100644 --- a/haiku_rag_slim/haiku/rag/converters/docling_serve.py +++ b/haiku_rag_slim/haiku/rag/converters/docling_serve.py @@ -99,7 +99,7 @@ class DoclingServeConverter(DocumentConverter): """ opts = self.config.processing.conversion_options pic_desc = opts.picture_description - runs_vlm = self.config.processing.pictures == "description" + runs_vlm = pic_desc.enabled data: dict[str, str | list[str]] = { "to_formats": "json", diff --git a/tests/store/test_document_items.py b/tests/store/test_document_items.py index ab7636b6..ffe4bd3c 100644 --- a/tests/store/test_document_items.py +++ b/tests/store/test_document_items.py @@ -5,7 +5,6 @@ from haiku.rag.client.documents import ( _store_document_with_chunks, _update_document_with_chunks, ) -from haiku.rag.config import AppConfig from haiku.rag.store.engine import Store from haiku.rag.store.models.document_item import ( DocumentItem, @@ -647,11 +646,7 @@ class TestPictureDataPreservedThroughRoundTrip: docling_doc = _docling_doc_with_picture() - config = AppConfig() - # Preservation only kicks in under modes that retain picture bytes. - config.processing.pictures = "image" - - async with HaikuRAG(temp_db_path, config=config, create=True) as rag: + async with HaikuRAG(temp_db_path, create=True) as rag: document = Document(content="Hello world", uri="test://doc") document.set_docling(docling_doc) created = await _store_document_with_chunks(rag, document, [], docling_doc) diff --git a/tests/test_chunker.py b/tests/test_chunker.py index e4705453..8e30e891 100644 --- a/tests/test_chunker.py +++ b/tests/test_chunker.py @@ -676,7 +676,6 @@ async def test_serve_chunker_accepts_picture_laden_docling(): from haiku.rag.converters.docling_serve import DoclingServeConverter config = AppConfig() - config.processing.pictures = "image" config.processing.conversion_options.do_ocr = False config.processing.chunk_size = 256 config.processing.chunker_type = "hybrid" diff --git a/tests/test_config.py b/tests/test_config.py index ccae77fa..61da2c7f 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -239,8 +239,10 @@ def test_init_config_creates_valid_yaml(tmp_path): assert config.environment == "production" -# Legacy picture-handling field translation (`generate_picture_images` and -# `picture_description.enabled` → `processing.pictures` enum) +# Legacy picture-handling field translation: +# processing.pictures: "description" -> picture_description.enabled = true +# processing.pictures: "none" | "image" -> picture_description.enabled = false +# processing.conversion_options.generate_picture_images: -> ignored def _write(tmp_path, body: str): @@ -249,17 +251,16 @@ def _write(tmp_path, body: str): return p -def test_load_yaml_legacy_picture_description_maps_to_description(tmp_path): - """`picture_description.enabled=true` (with or without the image flag) - maps to `processing.pictures: description`.""" +def test_load_yaml_legacy_pictures_description_maps_to_enabled(tmp_path): + """`processing.pictures: description` maps to + `picture_description.enabled = true` and warns.""" config_file = _write( tmp_path, """ processing: + pictures: description conversion_options: - generate_picture_images: false picture_description: - enabled: true timeout: 120 """, ) @@ -270,15 +271,49 @@ processing: finally: loader_logger.removeHandler(handler) cfg = AppConfig.model_validate(data) - assert cfg.processing.pictures == "description" + assert cfg.processing.conversion_options.picture_description.enabled is True assert cfg.processing.conversion_options.picture_description.timeout == 120 - assert any( - "picture_description.enabled=true" in r.getMessage() for r in handler.records + assert any("processing.pictures" in r.getMessage() for r in handler.records) + + +def test_load_yaml_legacy_pictures_image_maps_to_disabled(tmp_path): + """`processing.pictures: image` is the bytes-only mode in the old + enum; under always-store semantics it collapses to + `picture_description.enabled = false`.""" + config_file = _write( + tmp_path, + """ +processing: + pictures: image +""", ) + handler = _ListHandler() + loader_logger.addHandler(handler) + try: + data = load_yaml_config(config_file) + finally: + loader_logger.removeHandler(handler) + cfg = AppConfig.model_validate(data) + assert cfg.processing.conversion_options.picture_description.enabled is False -def test_load_yaml_legacy_generate_picture_images_maps_to_image(tmp_path): - """`generate_picture_images=true` alone maps to `pictures: image`.""" +def test_load_yaml_legacy_pictures_none_maps_to_disabled(tmp_path): + """`processing.pictures: none` maps to `picture_description.enabled = false`.""" + config_file = _write( + tmp_path, + """ +processing: + pictures: none +""", + ) + data = load_yaml_config(config_file) + cfg = AppConfig.model_validate(data) + assert cfg.processing.conversion_options.picture_description.enabled is False + + +def test_load_yaml_legacy_generate_picture_images_warns_and_drops(tmp_path): + """`generate_picture_images` is now a no-op (bytes always extracted); + it gets dropped with a deprecation warning.""" config_file = _write( tmp_path, """ @@ -294,15 +329,14 @@ processing: finally: loader_logger.removeHandler(handler) cfg = AppConfig.model_validate(data) - assert cfg.processing.pictures == "image" - assert any( - "generate_picture_images=true" in r.getMessage() for r in handler.records - ) + assert cfg.processing.conversion_options.picture_description.enabled is False + assert "generate_picture_images" not in data["processing"]["conversion_options"] + assert any("generate_picture_images" in r.getMessage() for r in handler.records) -def test_load_yaml_no_legacy_fields_keeps_default_none(tmp_path): - """Empty processing block leaves the default `none` mode untouched and - does not warn.""" +def test_load_yaml_no_legacy_fields_keeps_default_disabled(tmp_path): + """Empty processing block leaves picture_description.enabled at the + default (False) and does not warn.""" config_file = _write( tmp_path, """ @@ -317,29 +351,23 @@ processing: finally: loader_logger.removeHandler(handler) cfg = AppConfig.model_validate(data) - assert cfg.processing.pictures == "none" + assert cfg.processing.conversion_options.picture_description.enabled is False assert not handler.records -def test_load_yaml_explicit_pictures_wins_over_legacy(tmp_path): - """When the user has migrated to `pictures: ...` we keep their choice - and silently drop legacy fields if both are present (e.g. from a - half-migrated config).""" +def test_load_yaml_explicit_enabled_wins_over_legacy_pictures(tmp_path): + """If the user already set `picture_description.enabled` explicitly, + a stale `processing.pictures` value does not override it.""" config_file = _write( tmp_path, """ processing: - pictures: image + pictures: none conversion_options: - generate_picture_images: false picture_description: enabled: true """, ) data = load_yaml_config(config_file) cfg = AppConfig.model_validate(data) - assert cfg.processing.pictures == "image" - # Legacy fields scrubbed so Pydantic validation doesn't trip on extras. - opts = data["processing"]["conversion_options"] - assert "generate_picture_images" not in opts - assert "enabled" not in opts.get("picture_description", {}) + assert cfg.processing.conversion_options.picture_description.enabled is True diff --git a/tests/test_converters.py b/tests/test_converters.py index 87ddbaec..de2fc97b 100644 --- a/tests/test_converters.py +++ b/tests/test_converters.py @@ -507,30 +507,23 @@ class TestDoclingLocalConverter: def test_picture_description_config_defaults(self, config): """Test that picture description config has correct defaults.""" - assert config.processing.pictures == "none" - assert ( - config.processing.conversion_options.picture_description.model.provider - == "ollama" - ) - assert ( - config.processing.conversion_options.picture_description.model.name - == "ministral-3" - ) - assert config.processing.conversion_options.picture_description.timeout == 90 - assert ( - config.processing.conversion_options.picture_description.max_tokens == 200 - ) + pic_desc = config.processing.conversion_options.picture_description + assert pic_desc.enabled is False + assert pic_desc.model.provider == "ollama" + assert pic_desc.model.name == "ministral-3" + assert pic_desc.timeout == 90 + assert pic_desc.max_tokens == 200 # Default prompt is in PromptsConfig assert "blind user" in config.prompts.picture_description def test_picture_description_config_applied(self, config): """Test that picture description config is applied to converter.""" - config.processing.pictures = "description" + config.processing.conversion_options.picture_description.enabled = True config.processing.conversion_options.picture_description.timeout = 120 converter = DoclingLocalConverter(config) - assert converter.config.processing.pictures == "description" pic_desc = converter.config.processing.conversion_options.picture_description + assert pic_desc.enabled is True assert pic_desc.timeout == 120 @pytest.mark.asyncio @@ -542,7 +535,7 @@ class TestDoclingLocalConverter: # Disable OCR (not needed for native PDF, avoids model downloads) config.processing.conversion_options.do_ocr = False # Enable picture description with Ollama - config.processing.pictures = "description" + config.processing.conversion_options.picture_description.enabled = True config.processing.conversion_options.picture_description.model.provider = ( "ollama" ) @@ -947,15 +940,11 @@ class TestDoclingServeConverterPictureDescription: @pytest.mark.asyncio async def test_picture_description_options_passed_to_api(self, config): - """Test that picture description options are passed to docling-serve API. - - ``pictures="description"`` requires picture images for the VLM, which - routes the request through the ``target_type=zip`` path. The test - mocks the zip workflow. - """ + """Picture-description options reach the docling-serve API when the + VLM is enabled.""" import json - config.processing.pictures = "description" + config.processing.conversion_options.picture_description.enabled = True config.processing.conversion_options.picture_description.model.provider = ( "ollama" ) @@ -1066,7 +1055,7 @@ class TestDoclingServeConverterIntegration: Note: Not using VCR because this test involves polling with changing task IDs. """ pdf_path = Path("tests/data/doclaynet.pdf") - config.processing.pictures = "description" + config.processing.conversion_options.picture_description.enabled = True config.processing.conversion_options.picture_description.model.provider = ( "ollama" ) diff --git a/tests/test_picture_in_context.py b/tests/test_picture_in_context.py index 09e53d37..d52daf6d 100644 --- a/tests/test_picture_in_context.py +++ b/tests/test_picture_in_context.py @@ -178,10 +178,7 @@ async def test_rechunk_preserves_picture_data(temp_db_path): docling_doc = _docling_doc_with_picture() - config = AppConfig() - config.processing.pictures = "image" - - async with HaikuRAG(temp_db_path, config=config, create=True) as rag: + async with HaikuRAG(temp_db_path, create=True) as rag: document = Document(content="x", uri="test://doc") document.set_docling(docling_doc) created = await _store_document_with_chunks(rag, document, [], docling_doc) @@ -253,7 +250,6 @@ class _Deps: async def test_search_tool_returns_multimodal_when_picture_present(): """The agent-facing search tool must wrap text + BinaryContent in ToolReturn whenever a result carries picture image_data AND the QA model is vision-capable.""" - from haiku.rag.config import AppConfig picture_result = SearchResult( content="A diagram of the layout", @@ -352,7 +348,6 @@ async def test_chunk_interleaves_picture_in_structural_order(monkeypatch): sit where they appear in the document, not appended at the end. """ from haiku.rag.client.processing import chunk - from haiku.rag.config import AppConfig from haiku.rag.embeddings import EmbedderWrapper from haiku.rag.store.models.chunk import Chunk @@ -513,14 +508,13 @@ async def test_ingest_emits_picture_chunks_with_multimodal_embedder( docling_doc = _docling_doc_with_picture() - from haiku.rag.config import AppConfig, EmbeddingModelConfig, EmbeddingsConfig + from haiku.rag.config import EmbeddingModelConfig, EmbeddingsConfig config = AppConfig( embeddings=EmbeddingsConfig( model=EmbeddingModelConfig(provider="ollama", name="stub", vector_dim=4) ) ) - config.processing.pictures = "image" async with HaikuRAG(temp_db_path, config=config, create=True) as rag: chunks = await rag.chunk(docling_doc) @@ -546,7 +540,6 @@ async def test_search_tool_skips_binary_content_when_qa_model_is_text_only(): is text-only (``qa.model.vision = False``, the default). Sending image parts to a text-only model would cause it to hallucinate confidently — Ollama silently accepts the bytes and the model guesses.""" - from haiku.rag.config import AppConfig picture_result = SearchResult( content="A diagram of the layout",