add processing.pictures enum (none|description|image), replacing the implicit pair of generate_picture_images+picture_description.enabled flags
This commit is contained in:
parent
b01c649684
commit
7831057339
14 changed files with 316 additions and 81 deletions
|
|
@ -3,6 +3,7 @@
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
|
|
||||||
|
- **Picture-handling mode enum.** `processing.pictures: "none" | "description" | "image"` (default `"none"`) replaces the previous two-flag dance (`generate_picture_images` + `picture_description.enabled`) where flipping picture descriptions on silently forced docling to also generate picture images, leaving raw bytes inside the compressed `docling_document` blob with no user-facing knob to control it. The new enum makes intent explicit: `description` runs the VLM at ingest *and* preserves bytes (so a vision-capable QA model can be turned on later without reingesting), `image` preserves bytes without running the VLM, `none` is bytes-free. The legacy fields are removed from the schema (`ConversionOptions.generate_picture_images` and `PictureDescriptionConfig.enabled`); the YAML loader silently translates pre-existing configs to the new enum and emits a one-line deprecation warning so existing `haiku.rag.yaml` files keep working until the user updates them. No data migration is required — the 0.45.0 backfill already populates `picture_data` for every doc that had bytes inline. Rebuild and update flows now skip the picture-bytes snapshot/merge under `pictures="none"`, so users who downgrade to `none` and rebuild reclaim storage.
|
||||||
- **Storage column for embedded picture bytes.** `DocumentItemRecord` gains a `picture_data: bytes | None` column (Arrow `large_binary`) to hold per-`PictureItem` image bytes addressable by `(document_id, self_ref)`. New repository accessors `get_picture_bytes` and `get_pictures_for_chunk` expose them; the existing items read paths (`get_all_items`, `get_all_items_grouped`, `get_items_in_range`, `_record_to_item`) now project an explicit lightweight column set so context expansion and the analysis-sandbox `items.jsonl` build never pull picture bytes into memory. Existing databases pick up the column via the `0.45.0` migration alongside the picture-byte backfill (see below). Foundation for upcoming vision-in-context retrieval; not yet wired into ingestion or search.
|
- **Storage column for embedded picture bytes.** `DocumentItemRecord` gains a `picture_data: bytes | None` column (Arrow `large_binary`) to hold per-`PictureItem` image bytes addressable by `(document_id, self_ref)`. New repository accessors `get_picture_bytes` and `get_pictures_for_chunk` expose them; the existing items read paths (`get_all_items`, `get_all_items_grouped`, `get_items_in_range`, `_record_to_item`) now project an explicit lightweight column set so context expansion and the analysis-sandbox `items.jsonl` build never pull picture bytes into memory. Existing databases pick up the column via the `0.45.0` migration alongside the picture-byte backfill (see below). Foundation for upcoming vision-in-context retrieval; not yet wired into ingestion or search.
|
||||||
- **Embedded picture bytes captured at ingestion.** `extract_items` now decodes each `PictureItem.image.uri` data URI into raw bytes and writes them to `document_items.picture_data` so per-figure lookups don't require decompressing the full docling blob. The same path also surfaces VLM-generated picture descriptions (`meta.description.text`) into `DocumentItem.text` so picture-only chunks survive `expand_with_items`' text filter. The `0.45.0` migration adds the `picture_data` column to existing databases and backfills it by extracting bytes out of `docling_document`, stripping picture URIs from the structure blob in the process; `compress_docling_split` does the same for new ingests so the structure stays lean. Rebuild and update flows snapshot picture bytes via the new `DocumentItemRepository.get_all_picture_data` accessor before re-extraction so a re-chunk doesn't drop them.
|
- **Embedded picture bytes captured at ingestion.** `extract_items` now decodes each `PictureItem.image.uri` data URI into raw bytes and writes them to `document_items.picture_data` so per-figure lookups don't require decompressing the full docling blob. The same path also surfaces VLM-generated picture descriptions (`meta.description.text`) into `DocumentItem.text` so picture-only chunks survive `expand_with_items`' text filter. The `0.45.0` migration adds the `picture_data` column to existing databases and backfills it by extracting bytes out of `docling_document`, stripping picture URIs from the structure blob in the process; `compress_docling_split` does the same for new ingests so the structure stays lean. Rebuild and update flows snapshot picture bytes via the new `DocumentItemRepository.get_all_picture_data` accessor before re-extraction so a re-chunk doesn't drop them.
|
||||||
- **Picture image bytes in search results and vision-capable QA.** `SearchResult` gains an `image_data: dict[str, str] | None` field carrying base64-encoded picture bytes keyed by `self_ref` for picture-labeled chunks. `client.search()` and the MCP `search_documents` tool gain an `include_images: bool = True` flag; set False to omit the bytes for plain-text consumers. `expand_context` now preserves picture self_refs with empty text so they aren't filtered out before reaching the image-data lookup. The agent-facing search tool (`tools/search.py`) returns `pydantic_ai.messages.ToolReturn(return_value=text, content=[BinaryContent(...), ...])` when picture data is present so a vision-capable QA model sees the figures alongside the text; otherwise it returns a plain string and non-vision flows are unchanged.
|
- **Picture image bytes in search results and vision-capable QA.** `SearchResult` gains an `image_data: dict[str, str] | None` field carrying base64-encoded picture bytes keyed by `self_ref` for picture-labeled chunks. `client.search()` and the MCP `search_documents` tool gain an `include_images: bool = True` flag; set False to omit the bytes for plain-text consumers. `expand_context` now preserves picture self_refs with empty text so they aren't filtered out before reaching the image-data lookup. The agent-facing search tool (`tools/search.py`) returns `pydantic_ai.messages.ToolReturn(return_value=text, content=[BinaryContent(...), ...])` when picture data is present so a vision-capable QA model sees the figures alongside the text; otherwise it returns a plain string and non-vision flows are unchanged.
|
||||||
|
|
|
||||||
|
|
@ -28,6 +28,9 @@ processing:
|
||||||
name: gpt-oss
|
name: gpt-oss
|
||||||
enable_thinking: false
|
enable_thinking: false
|
||||||
|
|
||||||
|
# Picture handling (none / description / image)
|
||||||
|
pictures: none # See "Picture Handling" below
|
||||||
|
|
||||||
# Conversion options (works with both local and remote converters)
|
# Conversion options (works with both local and remote converters)
|
||||||
conversion_options:
|
conversion_options:
|
||||||
# OCR settings
|
# OCR settings
|
||||||
|
|
@ -44,11 +47,9 @@ processing:
|
||||||
# Image settings
|
# Image settings
|
||||||
images_scale: 2.0 # Image scale factor
|
images_scale: 2.0 # Image scale factor
|
||||||
generate_page_images: true # Include rendered page images (for visualize_chunk)
|
generate_page_images: true # Include rendered page images (for visualize_chunk)
|
||||||
generate_picture_images: false # Include embedded figure/diagram images
|
|
||||||
|
|
||||||
# VLM picture description (optional)
|
# VLM picture description settings (only effective when pictures: description)
|
||||||
picture_description:
|
picture_description:
|
||||||
enabled: false # Enable VLM image descriptions
|
|
||||||
model:
|
model:
|
||||||
provider: ollama
|
provider: ollama
|
||||||
name: ministral-3
|
name: ministral-3
|
||||||
|
|
@ -100,34 +101,44 @@ conversion_options:
|
||||||
conversion_options:
|
conversion_options:
|
||||||
images_scale: 2.0 # Image resolution scale factor
|
images_scale: 2.0 # Image resolution scale factor
|
||||||
generate_page_images: true # Include rendered page images
|
generate_page_images: true # Include rendered page images
|
||||||
generate_picture_images: false # Include embedded figure/diagram images
|
|
||||||
```
|
```
|
||||||
|
|
||||||
- **images_scale**: Scale factor for extracted images. Higher values = better quality but larger size. Typical range: 1.0-3.0.
|
- **images_scale**: Scale factor for extracted images. Higher values = better quality but larger size. Typical range: 1.0-3.0.
|
||||||
- **generate_page_images**: When `true` (default), rendered images of each PDF page are included in the document. Required for `visualize_chunk()` to show visual grounding. When `false`, page images are excluded to reduce document size.
|
- **generate_page_images**: When `true` (default), rendered images of each PDF page are included in the document. Required for `visualize_chunk()` to show visual grounding. When `false`, page images are excluded to reduce document size.
|
||||||
- **generate_picture_images**: When `true`, embedded images (figures, diagrams) are included as base64-encoded data in the document. When `false` (default), images are excluded to reduce chunk size and avoid context bloat.
|
|
||||||
|
|
||||||
**Note:** With `docling-serve`, `generate_picture_images` has limited support - picture image data may not be returned in the JSON response. Page images work correctly with both local and remote converters.
|
Embedded picture extraction is controlled by `processing.pictures` (see [Picture Handling](#picture-handling) below), not by an image-settings flag.
|
||||||
|
|
||||||
#### Picture Description (VLM)
|
#### Picture Handling
|
||||||
|
|
||||||
Use a Vision Language Model (VLM) to automatically describe images in documents. Descriptions become searchable text, improving RAG retrieval for visual content.
|
`processing.pictures` is a single enum that decides how embedded picture images (figures, diagrams) are handled at ingest. Three modes:
|
||||||
|
|
||||||
|
| Mode | VLM at ingest | `picture_data` populated | Chunk text contains description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `none` (default) | no | no | no |
|
||||||
|
| `description` | yes | yes | yes |
|
||||||
|
| `image` | no | yes | no |
|
||||||
|
|
||||||
|
- **`none`**: docling skips picture-image generation. `label="picture"` rows still appear in the items table for structural metadata, but they carry no bytes and no description. Cheapest mode; non-vision QA is unaffected.
|
||||||
|
- **`description`**: docling generates picture images, the configured VLM produces a description woven into chunk text, and the bytes are also retained in `document_items.picture_data`. Picture-text is searchable via FTS, vision-capable QA models also receive the bytes via the agent's search tool. Bytes are kept (not just thrown away after the VLM runs) so a vision-only QA strategy can be turned on later without reingesting.
|
||||||
|
- **`image`**: docling generates picture images and stores them in `document_items.picture_data` without running the VLM. Vision-capable QA models reason directly about the figures; non-vision QA only sees the picture's caption/surrounding text.
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
conversion_options:
|
processing:
|
||||||
picture_description:
|
pictures: description # none / description / image
|
||||||
enabled: true # Enable VLM picture description
|
conversion_options:
|
||||||
|
picture_description: # only effective when pictures: description
|
||||||
model:
|
model:
|
||||||
provider: ollama # ollama, openai, or custom
|
provider: ollama # ollama, openai, or custom
|
||||||
name: ministral-3 # VLM model name
|
name: ministral-3 # VLM model name
|
||||||
temperature: 0.0 # Default: 0.0 (factual descriptions)
|
temperature: 0.0
|
||||||
timeout: 90 # Request timeout in seconds
|
timeout: 90 # Request timeout in seconds
|
||||||
max_tokens: 200 # Maximum tokens in response
|
max_tokens: 200 # Maximum tokens in response
|
||||||
```
|
```
|
||||||
|
|
||||||
**Configuration options:**
|
**Switching modes on an existing database.** No reingest is required if you only need to change between `description` and `image` — the bytes are already there. Run `haiku-rag rebuild --rechunk` after the config change so the chunk-text composition reflects the new mode. Switching *down* to `none` clears `picture_data` on rebuild, reclaiming storage.
|
||||||
|
|
||||||
|
**`picture_description.model` configuration** (used only under `pictures: description`):
|
||||||
|
|
||||||
- **enabled**: When `true`, each embedded image is sent to a VLM for description. Requires `generate_picture_images` to be `true` (automatically enabled).
|
|
||||||
- **model**: Standard model configuration
|
- **model**: Standard model configuration
|
||||||
- `provider`: `ollama` (default), `openai`, or use `base_url` for custom endpoints
|
- `provider`: `ollama` (default), `openai`, or use `base_url` for custom endpoints
|
||||||
- `name`: Model name (e.g., `ministral-3`, `granite3.2-vision`, `gpt-4-vision`)
|
- `name`: Model name (e.g., `ministral-3`, `granite3.2-vision`, `gpt-4-vision`)
|
||||||
|
|
@ -155,9 +166,10 @@ prompts:
|
||||||
**Using with Ollama:**
|
**Using with Ollama:**
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
conversion_options:
|
processing:
|
||||||
|
pictures: description
|
||||||
|
conversion_options:
|
||||||
picture_description:
|
picture_description:
|
||||||
enabled: true
|
|
||||||
model:
|
model:
|
||||||
provider: ollama
|
provider: ollama
|
||||||
name: ministral-3
|
name: ministral-3
|
||||||
|
|
@ -173,9 +185,10 @@ ollama serve
|
||||||
**Using with vLLM or custom endpoints:**
|
**Using with vLLM or custom endpoints:**
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
conversion_options:
|
processing:
|
||||||
|
pictures: description
|
||||||
|
conversion_options:
|
||||||
picture_description:
|
picture_description:
|
||||||
enabled: true
|
|
||||||
model:
|
model:
|
||||||
provider: openai # Use OpenAI-compatible API format
|
provider: openai # Use OpenAI-compatible API format
|
||||||
name: granite-vision
|
name: granite-vision
|
||||||
|
|
@ -199,8 +212,10 @@ When using `converter: docling-serve`, the VLM calls are made by the docling-ser
|
||||||
**Docker networking:** If docling-serve runs in Docker and your VLM runs on the host, use `host.docker.internal` instead of `localhost`:
|
**Docker networking:** If docling-serve runs in Docker and your VLM runs on the host, use `host.docker.internal` instead of `localhost`:
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
picture_description:
|
processing:
|
||||||
enabled: true
|
pictures: description
|
||||||
|
conversion_options:
|
||||||
|
picture_description:
|
||||||
model:
|
model:
|
||||||
provider: ollama
|
provider: ollama
|
||||||
name: ministral-3
|
name: ministral-3
|
||||||
|
|
|
||||||
|
|
@ -119,7 +119,7 @@ prompts:
|
||||||
Be concise and factual.
|
Be concise and factual.
|
||||||
```
|
```
|
||||||
|
|
||||||
The prompt is used when `processing.conversion_options.picture_description.enabled` is `true`. See [Picture Description (VLM)](processing.md#picture-description-vlm) for full configuration.
|
The prompt is used when `processing.pictures` is set to `description`. See [Picture Handling](processing.md#picture-handling) for full configuration.
|
||||||
|
|
||||||
## Programmatic Configuration
|
## Programmatic Configuration
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -99,12 +99,16 @@ async def _update_document_with_chunks(
|
||||||
# Replace document items when a new DoclingDocument is provided.
|
# Replace document items when a new DoclingDocument is provided.
|
||||||
# Snapshot existing picture bytes first so they survive the
|
# Snapshot existing picture bytes first so they survive the
|
||||||
# delete-and-re-extract cycle when the live docling has already had
|
# delete-and-re-extract cycle when the live docling has already had
|
||||||
# its picture URIs stripped (rebuild / round-trip scenarios).
|
# its picture URIs stripped (rebuild / round-trip scenarios). Under
|
||||||
|
# `pictures="none"` we skip the snapshot so updates reclaim storage.
|
||||||
if docling_document is not None:
|
if docling_document is not None:
|
||||||
|
keep_picture_data = client._config.processing.pictures != "none"
|
||||||
existing_picture_data = (
|
existing_picture_data = (
|
||||||
await client.document_item_repository.get_all_picture_data(
|
await client.document_item_repository.get_all_picture_data(
|
||||||
updated_doc.id
|
updated_doc.id
|
||||||
)
|
)
|
||||||
|
if keep_picture_data
|
||||||
|
else None
|
||||||
)
|
)
|
||||||
await client.document_item_repository.delete_by_document_id(updated_doc.id)
|
await client.document_item_repository.delete_by_document_id(updated_doc.id)
|
||||||
items = extract_items(
|
items = extract_items(
|
||||||
|
|
|
||||||
|
|
@ -105,7 +105,10 @@ async def download_models(
|
||||||
if config.reranking.model and config.reranking.model.provider == "ollama":
|
if config.reranking.model and config.reranking.model.provider == "ollama":
|
||||||
required_models.add(config.reranking.model.name)
|
required_models.add(config.reranking.model.name)
|
||||||
pic_desc = config.processing.conversion_options.picture_description
|
pic_desc = config.processing.conversion_options.picture_description
|
||||||
if pic_desc.enabled and pic_desc.model.provider == "ollama":
|
if (
|
||||||
|
config.processing.pictures == "description"
|
||||||
|
and pic_desc.model.provider == "ollama"
|
||||||
|
):
|
||||||
required_models.add(pic_desc.model.name)
|
required_models.add(pic_desc.model.name)
|
||||||
if (
|
if (
|
||||||
config.processing.auto_title
|
config.processing.auto_title
|
||||||
|
|
|
||||||
|
|
@ -197,14 +197,19 @@ async def _flush_rebuild_batch(
|
||||||
|
|
||||||
# Repopulate document items from stored docling data. The stored docling
|
# Repopulate document items from stored docling data. The stored docling
|
||||||
# blob has had its picture URIs stripped (compress_docling_split), so
|
# blob has had its picture URIs stripped (compress_docling_split), so
|
||||||
# re-extracting from it would lose picture_data; snapshot the existing
|
# re-extracting from it would lose picture_data; under modes that retain
|
||||||
# bytes per document and merge them back during extraction.
|
# bytes (`description`/`image`) we snapshot the existing bytes per
|
||||||
|
# document and merge them back. Under `none`, we deliberately skip the
|
||||||
|
# snapshot so the rebuild reclaims storage.
|
||||||
|
keep_picture_data = client._config.processing.pictures != "none"
|
||||||
for doc in documents:
|
for doc in documents:
|
||||||
assert doc.id is not None
|
assert doc.id is not None
|
||||||
docling_doc = doc.get_docling_document()
|
docling_doc = doc.get_docling_document()
|
||||||
if docling_doc is not None:
|
if docling_doc is not None:
|
||||||
existing_picture_data = (
|
existing_picture_data = (
|
||||||
await client.document_item_repository.get_all_picture_data(doc.id)
|
await client.document_item_repository.get_all_picture_data(doc.id)
|
||||||
|
if keep_picture_data
|
||||||
|
else None
|
||||||
)
|
)
|
||||||
await client.document_item_repository.delete_by_document_id(doc.id)
|
await client.document_item_repository.delete_by_document_id(doc.id)
|
||||||
items = extract_items(
|
items = extract_items(
|
||||||
|
|
|
||||||
|
|
@ -1,8 +1,11 @@
|
||||||
|
import logging
|
||||||
import os
|
import os
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
import yaml
|
import yaml
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
def find_config_file(cli_path: Path | None = None) -> Path | None:
|
def find_config_file(cli_path: Path | None = None) -> Path | None:
|
||||||
"""Find the YAML config file using the search path.
|
"""Find the YAML config file using the search path.
|
||||||
|
|
@ -44,7 +47,65 @@ def load_yaml_config(path: Path) -> dict:
|
||||||
"""Load and parse a YAML config file."""
|
"""Load and parse a YAML config file."""
|
||||||
with open(path) as f:
|
with open(path) as f:
|
||||||
data = yaml.safe_load(f)
|
data = yaml.safe_load(f)
|
||||||
return data or {}
|
data = data or {}
|
||||||
|
_translate_legacy_picture_fields(data)
|
||||||
|
return data
|
||||||
|
|
||||||
|
|
||||||
|
def _translate_legacy_picture_fields(data: dict) -> None:
|
||||||
|
"""Map pre-A4 picture knobs onto ``processing.pictures``.
|
||||||
|
|
||||||
|
Pre-A4 the same intent was expressed by two booleans on
|
||||||
|
``conversion_options``: ``generate_picture_images`` and
|
||||||
|
``picture_description.enabled``. Translation, in priority order:
|
||||||
|
|
||||||
|
- ``picture_description.enabled = true`` (regardless of the image flag)
|
||||||
|
→ ``pictures = "description"``. Mirrors the original behavior where
|
||||||
|
enabling the VLM implicitly forced docling to produce picture bytes.
|
||||||
|
- ``generate_picture_images = true`` (and no description) → ``"image"``.
|
||||||
|
- both false / missing → no translation; default ``"none"`` applies.
|
||||||
|
|
||||||
|
If ``pictures`` is already set on the loaded YAML it wins — users who
|
||||||
|
have migrated keep their explicit choice. Mutates ``data`` in-place
|
||||||
|
and emits one warning per legacy field encountered.
|
||||||
|
"""
|
||||||
|
processing = data.get("processing")
|
||||||
|
if not isinstance(processing, dict):
|
||||||
|
return
|
||||||
|
|
||||||
|
if "pictures" in processing:
|
||||||
|
# User has migrated; legacy fields may still be present but should not
|
||||||
|
# override the explicit choice. Drop them silently to avoid confusion.
|
||||||
|
opts = processing.get("conversion_options")
|
||||||
|
if isinstance(opts, dict):
|
||||||
|
opts.pop("generate_picture_images", None)
|
||||||
|
pic = opts.get("picture_description")
|
||||||
|
if isinstance(pic, dict):
|
||||||
|
pic.pop("enabled", None)
|
||||||
|
return
|
||||||
|
|
||||||
|
opts = processing.get("conversion_options")
|
||||||
|
if not isinstance(opts, dict):
|
||||||
|
return
|
||||||
|
|
||||||
|
pic = opts.get("picture_description") if isinstance(opts, dict) else None
|
||||||
|
legacy_describe = pic.pop("enabled", None) if isinstance(pic, dict) else None
|
||||||
|
legacy_image = opts.pop("generate_picture_images", None)
|
||||||
|
|
||||||
|
if legacy_describe:
|
||||||
|
processing["pictures"] = "description"
|
||||||
|
logger.warning(
|
||||||
|
"Config: 'processing.conversion_options.picture_description.enabled=true' is "
|
||||||
|
"deprecated; mapped to 'processing.pictures: description'. Please update your "
|
||||||
|
"haiku.rag.yaml."
|
||||||
|
)
|
||||||
|
elif legacy_image:
|
||||||
|
processing["pictures"] = "image"
|
||||||
|
logger.warning(
|
||||||
|
"Config: 'processing.conversion_options.generate_picture_images=true' is "
|
||||||
|
"deprecated; mapped to 'processing.pictures: image'. Please update your "
|
||||||
|
"haiku.rag.yaml."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def generate_default_config() -> dict:
|
def generate_default_config() -> dict:
|
||||||
|
|
|
||||||
|
|
@ -110,9 +110,12 @@ class AnalysisConfig(BaseModel):
|
||||||
|
|
||||||
|
|
||||||
class PictureDescriptionConfig(BaseModel):
|
class PictureDescriptionConfig(BaseModel):
|
||||||
"""Configuration for VLM-based picture description."""
|
"""Settings for the VLM that runs at ingest under ``pictures="description"``.
|
||||||
|
|
||||||
|
Whether the VLM runs at all is decided by ``ProcessingConfig.pictures``;
|
||||||
|
these fields only describe *how* it runs once enabled.
|
||||||
|
"""
|
||||||
|
|
||||||
enabled: bool = False
|
|
||||||
model: ModelConfig = Field(
|
model: ModelConfig = Field(
|
||||||
default_factory=lambda: ModelConfig(
|
default_factory=lambda: ModelConfig(
|
||||||
provider="ollama",
|
provider="ollama",
|
||||||
|
|
@ -143,14 +146,16 @@ class ConversionOptions(BaseModel):
|
||||||
# Image options
|
# Image options
|
||||||
images_scale: float = 2.0
|
images_scale: float = 2.0
|
||||||
generate_page_images: bool = True
|
generate_page_images: bool = True
|
||||||
generate_picture_images: bool = False
|
|
||||||
|
|
||||||
# VLM picture description
|
# VLM picture description (only effective when ProcessingConfig.pictures == "description")
|
||||||
picture_description: PictureDescriptionConfig = Field(
|
picture_description: PictureDescriptionConfig = Field(
|
||||||
default_factory=PictureDescriptionConfig
|
default_factory=PictureDescriptionConfig
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
PicturesMode = Literal["none", "description", "image"]
|
||||||
|
|
||||||
|
|
||||||
class ProcessingConfig(BaseModel):
|
class ProcessingConfig(BaseModel):
|
||||||
chunk_size: int = 256
|
chunk_size: int = 256
|
||||||
converter: str = "docling-local"
|
converter: str = "docling-local"
|
||||||
|
|
@ -160,6 +165,18 @@ class ProcessingConfig(BaseModel):
|
||||||
chunking_merge_peers: bool = True
|
chunking_merge_peers: bool = True
|
||||||
chunking_use_markdown_tables: bool = False
|
chunking_use_markdown_tables: bool = False
|
||||||
conversion_options: ConversionOptions = Field(default_factory=ConversionOptions)
|
conversion_options: ConversionOptions = Field(default_factory=ConversionOptions)
|
||||||
|
pictures: PicturesMode = "none"
|
||||||
|
"""How embedded pictures are handled at ingest.
|
||||||
|
|
||||||
|
- ``"none"``: docling skips picture-image generation; structural
|
||||||
|
``label="picture"`` rows still exist but carry no bytes or description.
|
||||||
|
- ``"description"``: docling generates picture images, the VLM produces
|
||||||
|
text descriptions woven into chunk text, and the bytes are also
|
||||||
|
retained in ``document_items.picture_data`` so a vision-capable QA
|
||||||
|
model can be turned on later without reingesting.
|
||||||
|
- ``"image"``: docling generates picture images and stores them in
|
||||||
|
``document_items.picture_data``; no VLM runs at ingest.
|
||||||
|
"""
|
||||||
auto_title: bool = False
|
auto_title: bool = False
|
||||||
title_model: ModelConfig = Field(
|
title_model: ModelConfig = Field(
|
||||||
default_factory=lambda: ModelConfig(
|
default_factory=lambda: ModelConfig(
|
||||||
|
|
|
||||||
|
|
@ -123,13 +123,16 @@ class DoclingLocalConverter(DocumentConverter):
|
||||||
|
|
||||||
opts = self.config.processing.conversion_options
|
opts = self.config.processing.conversion_options
|
||||||
pic_desc = opts.picture_description
|
pic_desc = opts.picture_description
|
||||||
|
pictures_mode = self.config.processing.pictures
|
||||||
|
wants_picture_images = pictures_mode != "none"
|
||||||
|
runs_vlm = pictures_mode == "description"
|
||||||
|
|
||||||
pipeline_options = PdfPipelineOptions(
|
pipeline_options = PdfPipelineOptions(
|
||||||
do_ocr=opts.do_ocr,
|
do_ocr=opts.do_ocr,
|
||||||
do_table_structure=opts.do_table_structure,
|
do_table_structure=opts.do_table_structure,
|
||||||
images_scale=opts.images_scale,
|
images_scale=opts.images_scale,
|
||||||
generate_page_images=opts.generate_page_images,
|
generate_page_images=opts.generate_page_images,
|
||||||
generate_picture_images=opts.generate_picture_images or pic_desc.enabled,
|
generate_picture_images=wants_picture_images,
|
||||||
table_structure_options=TableStructureOptions(
|
table_structure_options=TableStructureOptions(
|
||||||
do_cell_matching=opts.table_cell_matching,
|
do_cell_matching=opts.table_cell_matching,
|
||||||
mode=(
|
mode=(
|
||||||
|
|
@ -139,10 +142,10 @@ class DoclingLocalConverter(DocumentConverter):
|
||||||
),
|
),
|
||||||
),
|
),
|
||||||
ocr_options=self._get_ocr_options(opts),
|
ocr_options=self._get_ocr_options(opts),
|
||||||
do_picture_description=pic_desc.enabled,
|
do_picture_description=runs_vlm,
|
||||||
)
|
)
|
||||||
|
|
||||||
if pic_desc.enabled:
|
if runs_vlm:
|
||||||
from pydantic import AnyUrl
|
from pydantic import AnyUrl
|
||||||
|
|
||||||
prompt = self.config.prompts.picture_description
|
prompt = self.config.prompts.picture_description
|
||||||
|
|
|
||||||
|
|
@ -85,13 +85,8 @@ class DoclingServeConverter(DocumentConverter):
|
||||||
raise ValueError(f"Unsupported VLM provider: {model.provider}")
|
raise ValueError(f"Unsupported VLM provider: {model.provider}")
|
||||||
|
|
||||||
def _picture_images_enabled(self) -> bool:
|
def _picture_images_enabled(self) -> bool:
|
||||||
"""Whether the conversion should produce embedded picture images.
|
"""Whether the conversion should produce embedded picture images."""
|
||||||
|
return self.config.processing.pictures != "none"
|
||||||
True if the user explicitly enabled ``generate_picture_images`` or if
|
|
||||||
``picture_description.enabled`` is on (the VLM needs the picture bytes).
|
|
||||||
"""
|
|
||||||
opts = self.config.processing.conversion_options
|
|
||||||
return opts.generate_picture_images or opts.picture_description.enabled
|
|
||||||
|
|
||||||
def _build_conversion_data(self) -> dict[str, str | list[str]]:
|
def _build_conversion_data(self) -> dict[str, str | list[str]]:
|
||||||
"""Build form data for conversion request.
|
"""Build form data for conversion request.
|
||||||
|
|
@ -108,7 +103,9 @@ class DoclingServeConverter(DocumentConverter):
|
||||||
"""
|
"""
|
||||||
opts = self.config.processing.conversion_options
|
opts = self.config.processing.conversion_options
|
||||||
pic_desc = opts.picture_description
|
pic_desc = opts.picture_description
|
||||||
picture_images_enabled = self._picture_images_enabled()
|
pictures_mode = self.config.processing.pictures
|
||||||
|
picture_images_enabled = pictures_mode != "none"
|
||||||
|
runs_vlm = pictures_mode == "description"
|
||||||
|
|
||||||
if picture_images_enabled:
|
if picture_images_enabled:
|
||||||
image_export_mode = "referenced"
|
image_export_mode = "referenced"
|
||||||
|
|
@ -128,7 +125,7 @@ class DoclingServeConverter(DocumentConverter):
|
||||||
"images_scale": str(opts.images_scale),
|
"images_scale": str(opts.images_scale),
|
||||||
"image_export_mode": image_export_mode,
|
"image_export_mode": image_export_mode,
|
||||||
"include_images": str(picture_images_enabled).lower(),
|
"include_images": str(picture_images_enabled).lower(),
|
||||||
"do_picture_description": str(pic_desc.enabled).lower(),
|
"do_picture_description": str(runs_vlm).lower(),
|
||||||
}
|
}
|
||||||
|
|
||||||
if picture_images_enabled:
|
if picture_images_enabled:
|
||||||
|
|
@ -137,7 +134,7 @@ class DoclingServeConverter(DocumentConverter):
|
||||||
if opts.ocr_lang:
|
if opts.ocr_lang:
|
||||||
data["ocr_lang"] = opts.ocr_lang
|
data["ocr_lang"] = opts.ocr_lang
|
||||||
|
|
||||||
if pic_desc.enabled:
|
if runs_vlm:
|
||||||
prompt = self.config.prompts.picture_description
|
prompt = self.config.prompts.picture_description
|
||||||
picture_description_api = {
|
picture_description_api = {
|
||||||
"url": self._get_vlm_api_url(pic_desc.model),
|
"url": self._get_vlm_api_url(pic_desc.model),
|
||||||
|
|
|
||||||
|
|
@ -431,6 +431,8 @@ class TestPictureDataStorage:
|
||||||
async with HaikuRAG(temp_db_path, create=True) as rag:
|
async with HaikuRAG(temp_db_path, create=True) as rag:
|
||||||
schema = await rag.store.document_items_table.schema()
|
schema = await rag.store.document_items_table.schema()
|
||||||
assert "picture_data" in {f.name for f in schema}
|
assert "picture_data" in {f.name for f in schema}
|
||||||
|
|
||||||
|
|
||||||
def _docling_doc_with_picture():
|
def _docling_doc_with_picture():
|
||||||
"""Build a tiny DoclingDocument with one PictureItem carrying real PNG bytes
|
"""Build a tiny DoclingDocument with one PictureItem carrying real PNG bytes
|
||||||
via ImageRef.from_pil. Used by the picture-extraction tests."""
|
via ImageRef.from_pil. Used by the picture-extraction tests."""
|
||||||
|
|
@ -634,6 +636,8 @@ class TestPictureDataPreservedThroughRoundTrip:
|
||||||
docling_doc = _docling_doc_with_picture()
|
docling_doc = _docling_doc_with_picture()
|
||||||
|
|
||||||
async with HaikuRAG(temp_db_path, create=True) as rag:
|
async with HaikuRAG(temp_db_path, create=True) as rag:
|
||||||
|
# Preservation only kicks in under modes that retain picture bytes.
|
||||||
|
rag._config.processing.pictures = "image"
|
||||||
document = Document(content="Hello world", uri="test://doc")
|
document = Document(content="Hello world", uri="test://doc")
|
||||||
document.set_docling(docling_doc)
|
document.set_docling(docling_doc)
|
||||||
created = await _store_document_with_chunks(rag, document, [], docling_doc)
|
created = await _store_document_with_chunks(rag, document, [], docling_doc)
|
||||||
|
|
|
||||||
|
|
@ -225,3 +225,92 @@ def test_init_config_creates_valid_yaml(tmp_path):
|
||||||
# Validate it
|
# Validate it
|
||||||
config = AppConfig.model_validate(loaded_data)
|
config = AppConfig.model_validate(loaded_data)
|
||||||
assert config.environment == "production"
|
assert config.environment == "production"
|
||||||
|
|
||||||
|
|
||||||
|
# A4: legacy `generate_picture_images` + `picture_description.enabled` translation
|
||||||
|
|
||||||
|
|
||||||
|
def _write(tmp_path, body: str):
|
||||||
|
p = tmp_path / "haiku.rag.yaml"
|
||||||
|
p.write_text(body)
|
||||||
|
return p
|
||||||
|
|
||||||
|
|
||||||
|
def test_load_yaml_legacy_picture_description_maps_to_description(tmp_path, caplog):
|
||||||
|
"""`picture_description.enabled=true` (with or without the image flag)
|
||||||
|
maps to `processing.pictures: description`."""
|
||||||
|
config_file = _write(
|
||||||
|
tmp_path,
|
||||||
|
"""
|
||||||
|
processing:
|
||||||
|
conversion_options:
|
||||||
|
generate_picture_images: false
|
||||||
|
picture_description:
|
||||||
|
enabled: true
|
||||||
|
timeout: 120
|
||||||
|
""",
|
||||||
|
)
|
||||||
|
with caplog.at_level("WARNING", logger="haiku.rag.config.loader"):
|
||||||
|
data = load_yaml_config(config_file)
|
||||||
|
cfg = AppConfig.model_validate(data)
|
||||||
|
assert cfg.processing.pictures == "description"
|
||||||
|
assert cfg.processing.conversion_options.picture_description.timeout == 120
|
||||||
|
assert any("picture_description.enabled=true" in m.message for m in caplog.records)
|
||||||
|
|
||||||
|
|
||||||
|
def test_load_yaml_legacy_generate_picture_images_maps_to_image(tmp_path, caplog):
|
||||||
|
"""`generate_picture_images=true` alone maps to `pictures: image`."""
|
||||||
|
config_file = _write(
|
||||||
|
tmp_path,
|
||||||
|
"""
|
||||||
|
processing:
|
||||||
|
conversion_options:
|
||||||
|
generate_picture_images: true
|
||||||
|
""",
|
||||||
|
)
|
||||||
|
with caplog.at_level("WARNING", logger="haiku.rag.config.loader"):
|
||||||
|
data = load_yaml_config(config_file)
|
||||||
|
cfg = AppConfig.model_validate(data)
|
||||||
|
assert cfg.processing.pictures == "image"
|
||||||
|
assert any("generate_picture_images=true" in m.message for m in caplog.records)
|
||||||
|
|
||||||
|
|
||||||
|
def test_load_yaml_no_legacy_fields_keeps_default_none(tmp_path, caplog):
|
||||||
|
"""Empty processing block leaves the default `none` mode untouched and
|
||||||
|
does not warn."""
|
||||||
|
config_file = _write(
|
||||||
|
tmp_path,
|
||||||
|
"""
|
||||||
|
processing:
|
||||||
|
chunk_size: 256
|
||||||
|
""",
|
||||||
|
)
|
||||||
|
with caplog.at_level("WARNING", logger="haiku.rag.config.loader"):
|
||||||
|
data = load_yaml_config(config_file)
|
||||||
|
cfg = AppConfig.model_validate(data)
|
||||||
|
assert cfg.processing.pictures == "none"
|
||||||
|
assert not caplog.records
|
||||||
|
|
||||||
|
|
||||||
|
def test_load_yaml_explicit_pictures_wins_over_legacy(tmp_path):
|
||||||
|
"""When the user has migrated to `pictures: ...` we keep their choice
|
||||||
|
and silently drop legacy fields if both are present (e.g. from a
|
||||||
|
half-migrated config)."""
|
||||||
|
config_file = _write(
|
||||||
|
tmp_path,
|
||||||
|
"""
|
||||||
|
processing:
|
||||||
|
pictures: image
|
||||||
|
conversion_options:
|
||||||
|
generate_picture_images: false
|
||||||
|
picture_description:
|
||||||
|
enabled: true
|
||||||
|
""",
|
||||||
|
)
|
||||||
|
data = load_yaml_config(config_file)
|
||||||
|
cfg = AppConfig.model_validate(data)
|
||||||
|
assert cfg.processing.pictures == "image"
|
||||||
|
# Legacy fields scrubbed so Pydantic validation doesn't trip on extras.
|
||||||
|
opts = data["processing"]["conversion_options"]
|
||||||
|
assert "generate_picture_images" not in opts
|
||||||
|
assert "enabled" not in opts.get("picture_description", {})
|
||||||
|
|
|
||||||
|
|
@ -378,7 +378,7 @@ class TestDoclingLocalConverter:
|
||||||
async def test_convert_pdf_without_picture_images(self, config):
|
async def test_convert_pdf_without_picture_images(self, config):
|
||||||
"""Test PDF conversion excludes embedded images by default."""
|
"""Test PDF conversion excludes embedded images by default."""
|
||||||
pdf_path = Path("tests/data/doclaynet.pdf")
|
pdf_path = Path("tests/data/doclaynet.pdf")
|
||||||
config.processing.conversion_options.generate_picture_images = False
|
config.processing.pictures = "none"
|
||||||
converter = DoclingLocalConverter(config)
|
converter = DoclingLocalConverter(config)
|
||||||
|
|
||||||
doc = await converter.convert_file(pdf_path)
|
doc = await converter.convert_file(pdf_path)
|
||||||
|
|
@ -387,14 +387,14 @@ class TestDoclingLocalConverter:
|
||||||
# Check that pictures don't have image data
|
# Check that pictures don't have image data
|
||||||
for picture in doc.pictures:
|
for picture in doc.pictures:
|
||||||
assert picture.image is None, (
|
assert picture.image is None, (
|
||||||
"Pictures should not have image data when generate_picture_images=False"
|
'Pictures should not have image data when pictures="none"'
|
||||||
)
|
)
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_convert_pdf_with_picture_images(self, config):
|
async def test_convert_pdf_with_picture_images(self, config):
|
||||||
"""Test PDF conversion includes embedded images when enabled."""
|
"""Test PDF conversion includes embedded images when enabled."""
|
||||||
pdf_path = Path("tests/data/doclaynet.pdf")
|
pdf_path = Path("tests/data/doclaynet.pdf")
|
||||||
config.processing.conversion_options.generate_picture_images = True
|
config.processing.pictures = "image"
|
||||||
converter = DoclingLocalConverter(config)
|
converter = DoclingLocalConverter(config)
|
||||||
|
|
||||||
doc = await converter.convert_file(pdf_path)
|
doc = await converter.convert_file(pdf_path)
|
||||||
|
|
@ -404,7 +404,7 @@ class TestDoclingLocalConverter:
|
||||||
pictures_with_images = [p for p in doc.pictures if p.image is not None]
|
pictures_with_images = [p for p in doc.pictures if p.image is not None]
|
||||||
if doc.pictures:
|
if doc.pictures:
|
||||||
assert len(pictures_with_images) > 0, (
|
assert len(pictures_with_images) > 0, (
|
||||||
"Pictures should have image data when generate_picture_images=True"
|
'Pictures should have image data when pictures="image"'
|
||||||
)
|
)
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
|
|
@ -549,7 +549,7 @@ class TestDoclingLocalConverter:
|
||||||
|
|
||||||
def test_picture_description_config_defaults(self, config):
|
def test_picture_description_config_defaults(self, config):
|
||||||
"""Test that picture description config has correct defaults."""
|
"""Test that picture description config has correct defaults."""
|
||||||
assert config.processing.conversion_options.picture_description.enabled is False
|
assert config.processing.pictures == "none"
|
||||||
assert (
|
assert (
|
||||||
config.processing.conversion_options.picture_description.model.provider
|
config.processing.conversion_options.picture_description.model.provider
|
||||||
== "ollama"
|
== "ollama"
|
||||||
|
|
@ -567,12 +567,12 @@ class TestDoclingLocalConverter:
|
||||||
|
|
||||||
def test_picture_description_config_applied(self, config):
|
def test_picture_description_config_applied(self, config):
|
||||||
"""Test that picture description config is applied to converter."""
|
"""Test that picture description config is applied to converter."""
|
||||||
config.processing.conversion_options.picture_description.enabled = True
|
config.processing.pictures = "description"
|
||||||
config.processing.conversion_options.picture_description.timeout = 120
|
config.processing.conversion_options.picture_description.timeout = 120
|
||||||
converter = DoclingLocalConverter(config)
|
converter = DoclingLocalConverter(config)
|
||||||
|
|
||||||
|
assert converter.config.processing.pictures == "description"
|
||||||
pic_desc = converter.config.processing.conversion_options.picture_description
|
pic_desc = converter.config.processing.conversion_options.picture_description
|
||||||
assert pic_desc.enabled is True
|
|
||||||
assert pic_desc.timeout == 120
|
assert pic_desc.timeout == 120
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
|
|
@ -584,7 +584,7 @@ class TestDoclingLocalConverter:
|
||||||
# Disable OCR (not needed for native PDF, avoids model downloads)
|
# Disable OCR (not needed for native PDF, avoids model downloads)
|
||||||
config.processing.conversion_options.do_ocr = False
|
config.processing.conversion_options.do_ocr = False
|
||||||
# Enable picture description with Ollama
|
# Enable picture description with Ollama
|
||||||
config.processing.conversion_options.picture_description.enabled = True
|
config.processing.pictures = "description"
|
||||||
config.processing.conversion_options.picture_description.model.provider = (
|
config.processing.conversion_options.picture_description.model.provider = (
|
||||||
"ollama"
|
"ollama"
|
||||||
)
|
)
|
||||||
|
|
@ -700,7 +700,7 @@ class TestDoclingServeConverter:
|
||||||
config.processing.conversion_options.table_cell_matching = False
|
config.processing.conversion_options.table_cell_matching = False
|
||||||
config.processing.conversion_options.do_table_structure = False
|
config.processing.conversion_options.do_table_structure = False
|
||||||
config.processing.conversion_options.images_scale = 3.0
|
config.processing.conversion_options.images_scale = 3.0
|
||||||
config.processing.conversion_options.generate_picture_images = False
|
config.processing.pictures = "none"
|
||||||
converter = DoclingServeConverter(config)
|
converter = DoclingServeConverter(config)
|
||||||
|
|
||||||
doc_json = create_mock_docling_document("test")
|
doc_json = create_mock_docling_document("test")
|
||||||
|
|
@ -754,11 +754,11 @@ class TestDoclingServeConverter:
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_picture_images_request_uses_referenced_zip(self, config):
|
async def test_picture_images_request_uses_referenced_zip(self, config):
|
||||||
"""When generate_picture_images is on, the request flips to
|
"""When pictures="image" the request flips to
|
||||||
image_export_mode=referenced + target_type=zip and consumes a zip
|
image_export_mode=referenced + target_type=zip and consumes a zip
|
||||||
response — mirrors the upstream docling-serve#576 workaround.
|
response — mirrors the upstream docling-serve#576 workaround.
|
||||||
"""
|
"""
|
||||||
config.processing.conversion_options.generate_picture_images = True
|
config.processing.pictures = "image"
|
||||||
converter = DoclingServeConverter(config)
|
converter = DoclingServeConverter(config)
|
||||||
|
|
||||||
doc_json = create_mock_docling_document("test")
|
doc_json = create_mock_docling_document("test")
|
||||||
|
|
@ -1038,13 +1038,13 @@ class TestDoclingServeConverterPictureDescription:
|
||||||
async def test_picture_description_options_passed_to_api(self, config):
|
async def test_picture_description_options_passed_to_api(self, config):
|
||||||
"""Test that picture description options are passed to docling-serve API.
|
"""Test that picture description options are passed to docling-serve API.
|
||||||
|
|
||||||
Picture descriptions force ``generate_picture_images=True`` upstream,
|
``pictures="description"`` requires picture images for the VLM, which
|
||||||
which routes the request through the ``target_type=zip`` path so the
|
routes the request through the ``target_type=zip`` path. The test
|
||||||
VLM can see the actual figures. The test mocks the zip workflow.
|
mocks the zip workflow.
|
||||||
"""
|
"""
|
||||||
import json
|
import json
|
||||||
|
|
||||||
config.processing.conversion_options.picture_description.enabled = True
|
config.processing.pictures = "description"
|
||||||
config.processing.conversion_options.picture_description.model.provider = (
|
config.processing.conversion_options.picture_description.model.provider = (
|
||||||
"ollama"
|
"ollama"
|
||||||
)
|
)
|
||||||
|
|
@ -1155,7 +1155,7 @@ class TestDoclingServeConverterIntegration:
|
||||||
Note: Not using VCR because this test involves polling with changing task IDs.
|
Note: Not using VCR because this test involves polling with changing task IDs.
|
||||||
"""
|
"""
|
||||||
pdf_path = Path("tests/data/doclaynet.pdf")
|
pdf_path = Path("tests/data/doclaynet.pdf")
|
||||||
config.processing.conversion_options.picture_description.enabled = True
|
config.processing.pictures = "description"
|
||||||
config.processing.conversion_options.picture_description.model.provider = (
|
config.processing.conversion_options.picture_description.model.provider = (
|
||||||
"ollama"
|
"ollama"
|
||||||
)
|
)
|
||||||
|
|
@ -1228,7 +1228,7 @@ class TestDoclingServeConverterIntegration:
|
||||||
async def test_convert_pdf_without_picture_images(self, config):
|
async def test_convert_pdf_without_picture_images(self, config):
|
||||||
"""Test PDF conversion excludes picture images when disabled."""
|
"""Test PDF conversion excludes picture images when disabled."""
|
||||||
pdf_path = Path("tests/data/doclaynet.pdf")
|
pdf_path = Path("tests/data/doclaynet.pdf")
|
||||||
config.processing.conversion_options.generate_picture_images = False
|
config.processing.pictures = "none"
|
||||||
converter = DoclingServeConverter(config)
|
converter = DoclingServeConverter(config)
|
||||||
|
|
||||||
doc = await converter.convert_file(pdf_path)
|
doc = await converter.convert_file(pdf_path)
|
||||||
|
|
@ -1237,7 +1237,7 @@ class TestDoclingServeConverterIntegration:
|
||||||
# Check that pictures don't have image data
|
# Check that pictures don't have image data
|
||||||
for picture in doc.pictures:
|
for picture in doc.pictures:
|
||||||
assert picture.image is None, (
|
assert picture.image is None, (
|
||||||
"Pictures should not have image data when generate_picture_images=False"
|
'Pictures should not have image data when pictures="none"'
|
||||||
)
|
)
|
||||||
|
|
||||||
@pytest.mark.vcr()
|
@pytest.mark.vcr()
|
||||||
|
|
@ -1266,7 +1266,7 @@ class TestDoclingServeConverterIntegration:
|
||||||
URIs so the result is shape-equivalent to the local converter.
|
URIs so the result is shape-equivalent to the local converter.
|
||||||
"""
|
"""
|
||||||
pdf_path = Path("tests/data/doclaynet.pdf")
|
pdf_path = Path("tests/data/doclaynet.pdf")
|
||||||
config.processing.conversion_options.generate_picture_images = True
|
config.processing.pictures = "image"
|
||||||
converter = DoclingServeConverter(config)
|
converter = DoclingServeConverter(config)
|
||||||
|
|
||||||
doc = await converter.convert_file(pdf_path)
|
doc = await converter.convert_file(pdf_path)
|
||||||
|
|
@ -1275,7 +1275,7 @@ class TestDoclingServeConverterIntegration:
|
||||||
pictures_with_images = [p for p in doc.pictures if p.image is not None]
|
pictures_with_images = [p for p in doc.pictures if p.image is not None]
|
||||||
assert doc.pictures, "doclaynet.pdf is expected to contain at least one picture"
|
assert doc.pictures, "doclaynet.pdf is expected to contain at least one picture"
|
||||||
assert len(pictures_with_images) > 0, (
|
assert len(pictures_with_images) > 0, (
|
||||||
"Pictures should have image data when generate_picture_images=True"
|
'Pictures should have image data when pictures="image"'
|
||||||
)
|
)
|
||||||
sample = pictures_with_images[0]
|
sample = pictures_with_images[0]
|
||||||
assert sample.image is not None
|
assert sample.image is not None
|
||||||
|
|
|
||||||
|
|
@ -166,6 +166,42 @@ async def test_expand_context_preserves_picture_refs_with_empty_text(temp_db_pat
|
||||||
assert "picture" in out.labels
|
assert "picture" in out.labels
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_update_clears_picture_data_when_mode_none(temp_db_path):
|
||||||
|
"""Switching to ``pictures="none"`` and re-running update_document
|
||||||
|
drops picture_data — the snapshot/merge gate is what gives users a
|
||||||
|
"rebuild reclaims storage" path when they downgrade modes."""
|
||||||
|
from haiku.rag.client.documents import (
|
||||||
|
_store_document_with_chunks,
|
||||||
|
_update_document_with_chunks,
|
||||||
|
)
|
||||||
|
from haiku.rag.store.models.document import Document
|
||||||
|
from tests.store.test_document_items import _docling_doc_with_picture
|
||||||
|
|
||||||
|
docling_doc = _docling_doc_with_picture()
|
||||||
|
|
||||||
|
async with HaikuRAG(temp_db_path, create=True) as rag:
|
||||||
|
# Ingest under "image" so picture bytes land in document_items.
|
||||||
|
rag._config.processing.pictures = "image"
|
||||||
|
document = Document(content="x", uri="test://doc")
|
||||||
|
document.set_docling(docling_doc)
|
||||||
|
created = await _store_document_with_chunks(rag, document, [], docling_doc)
|
||||||
|
assert created.id is not None
|
||||||
|
before = await rag.document_item_repository.get_all_picture_data(created.id)
|
||||||
|
assert before.get("#/pictures/0") is not None
|
||||||
|
|
||||||
|
# Downgrade to "none" and re-run update with the (already stripped)
|
||||||
|
# docling pulled from storage. The snapshot/merge must be skipped so
|
||||||
|
# picture_data is cleared on the new items rows.
|
||||||
|
rag._config.processing.pictures = "none"
|
||||||
|
from_blob = created.get_docling_document()
|
||||||
|
assert from_blob is not None
|
||||||
|
await _update_document_with_chunks(rag, created, [], from_blob)
|
||||||
|
|
||||||
|
after = await rag.document_item_repository.get_all_picture_data(created.id)
|
||||||
|
assert after.get("#/pictures/0") is None
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_expand_context_repopulates_image_data(temp_db_path):
|
async def test_expand_context_repopulates_image_data(temp_db_path):
|
||||||
"""expand_context rebuilds SearchResult objects via expand_with_items, so
|
"""expand_context rebuilds SearchResult objects via expand_with_items, so
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue