Merge pull request #370 from ggozad/fix/picture-generation-opt-out
Replace picture_description.enabled with processing.pictures enum
This commit is contained in:
Yiorgis Gozadinos
GitHub
commit
129563b319
17 changed files with 118 additions and 136 deletions
|
|
@ -15,6 +15,7 @@
|
|||
|
||||
- **Chat TUI streams markdown incrementally.** Assistant messages now use Textual's `MarkdownStream` (`Markdown.get_stream`) and write per-token deltas instead of re-parsing the entire accumulated message on every token. Removes the O(n²) re-parse that visibly stuttered long responses. Bumps `textual` floor to `>=8.2.4` so `Markdown.get_stream` is reachable via the public API.
|
||||
- **Embedding compatibility check only raises on `vector_dim` mismatch.** `provider` and `name` drift (legitimate when the same model is served by a different stack, e.g. Ollama → vLLM-via-openai) now logs a one-time warning and updates the stored settings to match the current config. Subsequent opens are silent. Run `rebuild --embed-only` if you also want to re-embed under the new stack.
|
||||
- **`processing.pictures` enum replaces `picture_description.enabled`.** Three modes: `none` (skip picture generation entirely — lower RAM, smaller DBs), `description` (generate images, run VLM, store bytes), `image` (default — generate images, store bytes, no VLM). Closes #366. Breaking change: rename `picture_description.enabled: true` → `pictures: description`, `picture_description.enabled: false` → `pictures: image`. The pre-April-30 `generate_picture_images` flag is also gone; use `pictures: none` for that opt-out.
|
||||
|
||||
### Fixed
|
||||
|
||||
|
|
|
|||
|
|
@ -497,7 +497,7 @@ haiku-rag rebuild --descriptions
|
|||
| Title only | `--title-only` | Generate titles for documents without one |
|
||||
| Descriptions | `--descriptions` | Add VLM picture descriptions to an existing database |
|
||||
|
||||
**`--descriptions` mode** runs the configured VLM (`processing.conversion_options.picture_description.model`) over the picture bytes already stored in `document_items.picture_data`, patches each description into the stored docling blob's `pictures[i].meta.description.text`, and re-chunks + re-embeds so chunk text reflects the new descriptions. Requires `picture_description.enabled: true` in the config. Idempotent — pictures that already carry a description are skipped, so the operation is safe to re-run after a partial failure. The docling parse is skipped entirely; only the VLM time is paid.
|
||||
**`--descriptions` mode** runs the configured VLM (`processing.conversion_options.picture_description.model`) over the picture bytes already stored in `document_items.picture_data`, patches each description into the stored docling blob's `pictures[i].meta.description.text`, and re-chunks + re-embeds so chunk text reflects the new descriptions. Requires `processing.pictures: description` in the config. Idempotent — pictures that already carry a description are skipped, so the operation is safe to re-run after a partial failure. The docling parse is skipped entirely; only the VLM time is paid.
|
||||
|
||||
### Download Models
|
||||
|
||||
|
|
|
|||
|
|
@ -45,12 +45,12 @@ processing:
|
|||
images_scale: 2.0 # Image scale factor
|
||||
generate_page_images: true # Include rendered page images (for visualize_chunk)
|
||||
|
||||
# VLM picture description (off by default; see "Picture Handling" below)
|
||||
# VLM settings used when processing.pictures == "description" (see "Picture Handling" below)
|
||||
picture_description:
|
||||
enabled: false
|
||||
model:
|
||||
provider: ollama
|
||||
name: ministral-3
|
||||
pictures: image # none | description | image
|
||||
```
|
||||
|
||||
### Conversion Options
|
||||
|
|
@ -121,7 +121,7 @@ Per-image failures (404, timeout, oversized, unreadable) leave that picture as a
|
|||
|
||||
**Scope of conversion options across formats:**
|
||||
|
||||
| Input | OCR / table options | `images_scale` / `generate_page_images` | `picture_description` | `fetch_remote_images` |
|
||||
| Input | OCR / table options | `images_scale` / `generate_page_images` | `pictures` | `fetch_remote_images` |
|
||||
|---|---|---|---|---|
|
||||
| `.pdf` | ✅ | ✅ | ✅ | n/a |
|
||||
| `.png` / `.jpg` / `.jpeg` / `.bmp` / `.tiff` / `.webp` | ✅ | ✅ | ✅ | n/a |
|
||||
|
|
@ -132,26 +132,36 @@ Per-image failures (404, timeout, oversized, unreadable) leave that picture as a
|
|||
|
||||
#### Picture Handling
|
||||
|
||||
Picture bytes (figures, diagrams) are always extracted and stored in `document_items.picture_data` for every ingested document. The single configurable knob is whether a Vision Language Model (VLM) runs at ingest to generate textual descriptions:
|
||||
`processing.pictures` picks one of three modes:
|
||||
|
||||
| Mode | Picture-image generation in docling | Bytes stored in `document_items.picture_data` | VLM runs at ingest |
|
||||
|---|---|---|---|
|
||||
| `none` | off | no | no |
|
||||
| `description` | on | yes | yes |
|
||||
| `image` (default) | on | yes | no |
|
||||
|
||||
Use `none` when you don't need picture content (e.g. very large reference manuals where RAM is tight); use `description` to weave VLM-generated text into chunk content and keep bytes for later; use `image` (default) to keep bytes without paying the VLM cost. The prompt is configurable under `prompts.picture_description` — see [Prompts](prompts.md).
|
||||
|
||||
```yaml
|
||||
processing:
|
||||
pictures: description # none | description | image
|
||||
conversion_options:
|
||||
picture_description:
|
||||
enabled: true # default false
|
||||
picture_description: # only consulted when pictures == "description"
|
||||
model:
|
||||
provider: ollama # any OpenAI-compatible /v1/chat/completions provider
|
||||
provider: ollama # any OpenAI-compatible /v1/chat/completions provider
|
||||
name: ministral-3
|
||||
timeout: 90
|
||||
max_tokens: 200
|
||||
```
|
||||
|
||||
When `enabled: true`, each picture's description is woven into the chunk text and is searchable via FTS. The prompt is configurable under `prompts.picture_description` — see [Prompts](prompts.md).
|
||||
!!! warning "Breaking change"
|
||||
`processing.conversion_options.picture_description.enabled` is replaced by `processing.pictures`. Map `enabled: true` → `pictures: description`, `enabled: false` → `pictures: image`. The pre-April-30 `generate_picture_images` flag also no longer exists; use `pictures: none` for the old opt-out.
|
||||
|
||||
**Switching the VLM on or off on an existing database** doesn't require reingesting (the bytes are already there):
|
||||
**Switching modes on an existing database** doesn't require reingesting when the bytes are already stored:
|
||||
|
||||
- Off → on: `haiku-rag rebuild --descriptions` runs the VLM over stored bytes and re-chunks. Skips the docling parse entirely.
|
||||
- On → off: `haiku-rag rebuild --rechunk` recomposes chunk text from the stripped docling blob without descriptions.
|
||||
- `image` → `description`: `haiku-rag rebuild --descriptions` runs the VLM over stored bytes and re-chunks. Skips the docling parse entirely.
|
||||
- `description` → `image`: `haiku-rag rebuild --rechunk` recomposes chunk text from the stripped docling blob without descriptions.
|
||||
- Switching to/from `none`: a full reingest is needed since the bytes either weren't stored or need to be discarded.
|
||||
|
||||
When using `converter: docling-serve`, the VLM is invoked from docling-serve rather than haiku.rag — see [Remote processing](../remote-processing.md#vlm-picture-description-with-docling-serve).
|
||||
|
||||
|
|
@ -161,37 +171,37 @@ Three independent settings drive ingest, retrieval, and QA:
|
|||
|
||||
| Setting | Question it answers | Values |
|
||||
|---|---|---|
|
||||
| `picture_description.enabled` | Should a VLM weave descriptions into chunk text at ingest? | `false` (default) / `true` |
|
||||
| `processing.pictures` | Generate and/or describe pictures at ingest? | `none` / `description` / `image` (default) |
|
||||
| `embeddings.model.provider` | Can the embedder index image content? | text-only (`ollama`, `openai`, `cohere`, `sentence-transformers`) vs `vllm` (multimodal) |
|
||||
| `qa.model.vision` | Can the QA model interpret images? | `false` (default) / `true` |
|
||||
|
||||
Picture bytes are always stored, regardless of these settings.
|
||||
**What gets stored** by `pictures` × embedder:
|
||||
|
||||
**What gets stored** by `enabled` × embedder:
|
||||
|
||||
| `enabled` | Embedder | Text chunks | Synthetic picture chunks |
|
||||
| `pictures` | Embedder | Text chunks | Synthetic picture chunks |
|
||||
|---|---|---|---|
|
||||
| `false` | text-only | text only (caption/surrounding) | none |
|
||||
| `false` | multimodal | text only | one per picture, vector = image embedding |
|
||||
| `true` | text-only | text + descriptions | none |
|
||||
| `true` | multimodal | text + descriptions | one per picture, vector = image embedding |
|
||||
| `none` | any | text only (caption/surrounding) | none |
|
||||
| `image` | text-only | text only (caption/surrounding) | none |
|
||||
| `image` | multimodal | text only | one per picture, vector = image embedding |
|
||||
| `description` | text-only | text + descriptions | none |
|
||||
| `description` | multimodal | text + descriptions | one per picture, vector = image embedding |
|
||||
|
||||
**What QA receives** at search time:
|
||||
|
||||
- `qa.model.vision: false` — text chunks only (descriptions, when present, answer figure questions in prose).
|
||||
- `qa.model.vision: true` — text chunks + raw picture bytes via `BinaryContent`; the model reads figures directly.
|
||||
- `qa.model.vision: true` — text chunks + raw picture bytes via `BinaryContent`; the model reads figures directly. Requires `pictures != none` so the bytes exist.
|
||||
|
||||
`qa.model.vision` is independent of ingestion — flipping it never requires reingesting. Setting `vision: true` against a text-only model causes silent acceptance and confabulation on Ollama and a 400 on OpenAI; default `false` is the safe choice.
|
||||
|
||||
**Recommended combinations:**
|
||||
|
||||
| Use case | `picture_description.enabled` | Embedder | `qa.model.vision` |
|
||||
| Use case | `processing.pictures` | Embedder | `qa.model.vision` |
|
||||
|---|---|---|---|
|
||||
| Pure text RAG, no figures | `false` | text-only | `false` |
|
||||
| Text RAG, figures answered through descriptions | `true` | text-only | `false` |
|
||||
| Vision QA on figure-rich docs (no cross-modal search) | `true` or `false` | text-only | `true` |
|
||||
| Cross-modal search + vision QA | `true` or `false` | multimodal | `true` |
|
||||
| Cross-modal search, text QA only | `true` | multimodal | `false` |
|
||||
| Pure text RAG, no figures, lowest RAM | `none` | text-only | `false` |
|
||||
| Text RAG, store figure bytes for later | `image` | text-only | `false` |
|
||||
| Text RAG, figures answered through descriptions | `description` | text-only | `false` |
|
||||
| Vision QA on figure-rich docs (no cross-modal search) | `image` or `description` | text-only | `true` |
|
||||
| Cross-modal search + vision QA | `image` or `description` | multimodal | `true` |
|
||||
| Cross-modal search, text QA only | `description` | multimodal | `false` |
|
||||
|
||||
### Automatic Title Generation
|
||||
|
||||
|
|
|
|||
|
|
@ -119,7 +119,7 @@ prompts:
|
|||
Be concise and factual.
|
||||
```
|
||||
|
||||
The prompt is used when `processing.conversion_options.picture_description.enabled` is `true`. See [Picture Handling](processing.md#picture-handling) for full configuration.
|
||||
The prompt is used when `processing.pictures` is `"description"`. See [Picture Handling](processing.md#picture-handling) for full configuration.
|
||||
|
||||
## Programmatic Configuration
|
||||
|
||||
|
|
|
|||
|
|
@ -226,7 +226,7 @@ async for doc_id in client.rebuild_database(mode=RebuildMode.EMBED_ONLY):
|
|||
# Add VLM picture descriptions to an existing database — runs the VLM
|
||||
# over already-stored picture bytes, patches descriptions into the
|
||||
# docling blob, then re-chunks + re-embeds. Requires
|
||||
# picture_description.enabled=true in the config.
|
||||
# processing.pictures='description' in the config.
|
||||
async for doc_id in client.rebuild_database(mode=RebuildMode.DESCRIPTIONS):
|
||||
print(f"Described pictures in {doc_id}")
|
||||
```
|
||||
|
|
|
|||
|
|
@ -498,7 +498,7 @@ def rebuild(
|
|||
help=(
|
||||
"Run the VLM over already-stored picture bytes, patch descriptions "
|
||||
"into the docling blob, then re-chunk + re-embed. Skips the docling "
|
||||
"parse entirely. Requires picture_description.enabled=true."
|
||||
"parse entirely. Requires processing.pictures='description'."
|
||||
),
|
||||
),
|
||||
):
|
||||
|
|
|
|||
|
|
@ -105,7 +105,10 @@ async def download_models(
|
|||
if config.reranking.model and config.reranking.model.provider == "ollama":
|
||||
required_models.add(config.reranking.model.name)
|
||||
pic_desc = config.processing.conversion_options.picture_description
|
||||
if pic_desc.enabled and pic_desc.model.provider == "ollama":
|
||||
if (
|
||||
config.processing.pictures == "description"
|
||||
and pic_desc.model.provider == "ollama"
|
||||
):
|
||||
required_models.add(pic_desc.model.name)
|
||||
if (
|
||||
config.processing.auto_title
|
||||
|
|
|
|||
|
|
@ -26,12 +26,12 @@ def _warn_if_descriptions_missing(
|
|||
docling-serve swallows VLM errors (network failures, missing models,
|
||||
etc.) and returns a successful conversion with empty descriptions.
|
||||
docling-local can do the same when the VLM endpoint is unreachable.
|
||||
Surface the silent failure: when ``picture_description.enabled=True``
|
||||
Surface the silent failure: when ``processing.pictures="description"``
|
||||
AND the document has at least one picture AND zero descriptions came
|
||||
back, log a clear warning so the user can fix their VLM config before
|
||||
a thousand-document ingest produces an empty corpus.
|
||||
"""
|
||||
if not config.processing.conversion_options.picture_description.enabled:
|
||||
if config.processing.pictures != "description":
|
||||
return
|
||||
if not doc.pictures:
|
||||
return
|
||||
|
|
@ -39,7 +39,7 @@ def _warn_if_descriptions_missing(
|
|||
if described == 0:
|
||||
model = config.processing.conversion_options.picture_description.model
|
||||
logger.warning(
|
||||
"picture_description.enabled is True but no descriptions came back "
|
||||
"processing.pictures='description' but no descriptions came back "
|
||||
"for %s (%d pictures, 0 described). The VLM call likely failed "
|
||||
"silently inside the converter. Check that the VLM at %s is "
|
||||
"reachable from the converter and that the model name '%s' "
|
||||
|
|
|
|||
|
|
@ -383,10 +383,9 @@ async def _rebuild_descriptions(
|
|||
"""
|
||||
from haiku.rag.embeddings import embed_chunks, get_embedder
|
||||
|
||||
if not client._config.processing.conversion_options.picture_description.enabled:
|
||||
if client._config.processing.pictures != "description":
|
||||
raise ValueError(
|
||||
"rebuild --descriptions requires "
|
||||
"processing.conversion_options.picture_description.enabled = true "
|
||||
"rebuild --descriptions requires processing.pictures = 'description' "
|
||||
"in your config."
|
||||
)
|
||||
|
||||
|
|
|
|||
|
|
@ -47,30 +47,7 @@ def load_yaml_config(path: Path) -> dict:
|
|||
"""Load and parse a YAML config file."""
|
||||
with open(path) as f:
|
||||
data = yaml.safe_load(f)
|
||||
data = data or {}
|
||||
_drop_legacy_generate_picture_images(data)
|
||||
return data
|
||||
|
||||
|
||||
def _drop_legacy_generate_picture_images(data: dict) -> None:
|
||||
"""Drop ``processing.conversion_options.generate_picture_images`` from
|
||||
loaded YAML and log that it is no longer needed.
|
||||
|
||||
Picture bytes are always extracted now, so the flag has no effect.
|
||||
Old YAMLs keep working; users see one log line telling them they can
|
||||
remove the entry.
|
||||
"""
|
||||
processing = data.get("processing")
|
||||
if not isinstance(processing, dict):
|
||||
return
|
||||
opts = processing.get("conversion_options")
|
||||
if isinstance(opts, dict) and "generate_picture_images" in opts:
|
||||
opts.pop("generate_picture_images", None)
|
||||
logger.warning(
|
||||
"Config: 'processing.conversion_options.generate_picture_images' "
|
||||
"no longer has any effect (picture bytes are always extracted). "
|
||||
"Remove it from your haiku.rag.yaml."
|
||||
)
|
||||
return data or {}
|
||||
|
||||
|
||||
def generate_default_config() -> dict:
|
||||
|
|
|
|||
|
|
@ -129,14 +129,12 @@ class AnalysisConfig(BaseModel):
|
|||
|
||||
|
||||
class PictureDescriptionConfig(BaseModel):
|
||||
"""Whether (and how) to run a VLM over each picture at ingest.
|
||||
"""How the VLM runs over each picture when it runs at all.
|
||||
|
||||
When ``enabled`` is True, picture descriptions are generated by the
|
||||
configured VLM and woven into chunk text. Picture bytes are stored
|
||||
regardless — this flag only controls the description pass.
|
||||
Activation lives on ``ProcessingConfig.pictures`` — these fields only
|
||||
describe *how* the VLM runs once ``pictures == "description"``.
|
||||
"""
|
||||
|
||||
enabled: bool = False
|
||||
model: ModelConfig = Field(
|
||||
default_factory=lambda: ModelConfig(
|
||||
provider="ollama",
|
||||
|
|
@ -177,6 +175,9 @@ class ConversionOptions(BaseModel):
|
|||
)
|
||||
|
||||
|
||||
PicturesMode = Literal["none", "description", "image"]
|
||||
|
||||
|
||||
class ProcessingConfig(BaseModel):
|
||||
chunk_size: int = 256
|
||||
converter: str = "docling-local"
|
||||
|
|
@ -186,6 +187,21 @@ class ProcessingConfig(BaseModel):
|
|||
chunking_merge_peers: bool = True
|
||||
chunking_use_markdown_tables: bool = False
|
||||
conversion_options: ConversionOptions = Field(default_factory=ConversionOptions)
|
||||
pictures: PicturesMode = "image"
|
||||
"""How embedded pictures are handled at ingest.
|
||||
|
||||
- ``"none"``: docling skips picture-image generation; ``label="picture"``
|
||||
rows still exist as structure but carry no bytes or description. Use
|
||||
this when you don't need picture content and want to keep RAM and DB
|
||||
size low on large documents.
|
||||
- ``"description"``: docling generates picture images, the configured
|
||||
VLM produces text descriptions woven into chunk text, AND the bytes
|
||||
are retained in ``document_items.picture_data`` so a vision-capable
|
||||
QA model or multimodal embedder can be enabled later without
|
||||
reingesting.
|
||||
- ``"image"``: docling generates picture images and stores them in
|
||||
``document_items.picture_data``; no VLM runs at ingest.
|
||||
"""
|
||||
auto_title: bool = False
|
||||
title_model: ModelConfig = Field(
|
||||
default_factory=lambda: ModelConfig(
|
||||
|
|
|
|||
|
|
@ -119,14 +119,15 @@ class DoclingLocalConverter(DocumentConverter):
|
|||
|
||||
opts = self.config.processing.conversion_options
|
||||
pic_desc = opts.picture_description
|
||||
runs_vlm = pic_desc.enabled
|
||||
pictures = self.config.processing.pictures
|
||||
runs_vlm = pictures == "description"
|
||||
|
||||
pipeline_options = PdfPipelineOptions(
|
||||
do_ocr=opts.do_ocr,
|
||||
do_table_structure=opts.do_table_structure,
|
||||
images_scale=opts.images_scale,
|
||||
generate_page_images=opts.generate_page_images,
|
||||
generate_picture_images=True,
|
||||
generate_picture_images=pictures != "none",
|
||||
table_structure_options=TableStructureOptions(
|
||||
do_cell_matching=opts.table_cell_matching,
|
||||
mode=(
|
||||
|
|
|
|||
|
|
@ -99,7 +99,8 @@ class DoclingServeConverter(DocumentConverter):
|
|||
"""
|
||||
opts = self.config.processing.conversion_options
|
||||
pic_desc = opts.picture_description
|
||||
runs_vlm = pic_desc.enabled
|
||||
pictures = self.config.processing.pictures
|
||||
runs_vlm = pictures == "description"
|
||||
|
||||
data: dict[str, str | list[str]] = {
|
||||
"to_formats": "json",
|
||||
|
|
@ -111,7 +112,7 @@ class DoclingServeConverter(DocumentConverter):
|
|||
"table_cell_matching": str(opts.table_cell_matching).lower(),
|
||||
"images_scale": str(opts.images_scale),
|
||||
"image_export_mode": "referenced",
|
||||
"include_images": "true",
|
||||
"include_images": str(pictures != "none").lower(),
|
||||
"do_picture_description": str(runs_vlm).lower(),
|
||||
"target_type": "zip",
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,5 +1,3 @@
|
|||
import logging
|
||||
|
||||
import pytest
|
||||
import yaml
|
||||
|
||||
|
|
@ -9,16 +7,6 @@ from haiku.rag.config.loader import (
|
|||
generate_default_config,
|
||||
load_yaml_config,
|
||||
)
|
||||
from haiku.rag.config.loader import logger as loader_logger
|
||||
|
||||
|
||||
class _ListHandler(logging.Handler):
|
||||
def __init__(self) -> None:
|
||||
super().__init__(level=logging.WARNING)
|
||||
self.records: list[logging.LogRecord] = []
|
||||
|
||||
def emit(self, record: logging.LogRecord) -> None:
|
||||
self.records.append(record)
|
||||
|
||||
|
||||
def test_load_yaml_config(tmp_path):
|
||||
|
|
@ -239,39 +227,25 @@ def test_init_config_creates_valid_yaml(tmp_path):
|
|||
assert config.environment == "production"
|
||||
|
||||
|
||||
# Removed-field handling: processing.conversion_options.generate_picture_images
|
||||
# was honored by earlier releases. It's dropped from loaded YAML now and a
|
||||
# warning is logged so the user knows they can remove it from their config.
|
||||
|
||||
|
||||
def _write(tmp_path, body: str):
|
||||
p = tmp_path / "haiku.rag.yaml"
|
||||
p.write_text(body)
|
||||
return p
|
||||
|
||||
|
||||
def test_load_yaml_drops_generate_picture_images(tmp_path):
|
||||
"""``generate_picture_images`` from earlier releases is dropped from
|
||||
the parsed dict and a warning tells the user to remove it from the
|
||||
YAML."""
|
||||
config_file = _write(
|
||||
tmp_path,
|
||||
"""
|
||||
processing:
|
||||
conversion_options:
|
||||
generate_picture_images: true
|
||||
""",
|
||||
)
|
||||
handler = _ListHandler()
|
||||
loader_logger.addHandler(handler)
|
||||
try:
|
||||
data = load_yaml_config(config_file)
|
||||
finally:
|
||||
loader_logger.removeHandler(handler)
|
||||
cfg = AppConfig.model_validate(data)
|
||||
assert cfg.processing.conversion_options.picture_description.enabled is False
|
||||
assert "generate_picture_images" not in data["processing"]["conversion_options"]
|
||||
assert any("generate_picture_images" in r.getMessage() for r in handler.records)
|
||||
def test_pictures_field_defaults_to_image():
|
||||
"""`processing.pictures` defaults to `'image'` — current ingest behavior
|
||||
(store picture bytes, no VLM) preserved on fresh configs."""
|
||||
cfg = AppConfig()
|
||||
assert cfg.processing.pictures == "image"
|
||||
|
||||
|
||||
def test_picture_description_has_no_enabled_field():
|
||||
"""`enabled` is gone from PictureDescriptionConfig — activation is now
|
||||
controlled by `processing.pictures == 'description'`."""
|
||||
from haiku.rag.config.models import PictureDescriptionConfig
|
||||
|
||||
assert "enabled" not in PictureDescriptionConfig.model_fields
|
||||
|
||||
|
||||
def test_fetch_remote_images_default_true():
|
||||
|
|
|
|||
|
|
@ -574,7 +574,7 @@ class TestDoclingLocalConverter:
|
|||
|
||||
def test_image_format_shares_pdf_pipeline_options(self, config):
|
||||
"""IMAGE FormatOption shares the same PdfPipelineOptions instance as
|
||||
PDF. Without this, `do_ocr` / `picture_description.enabled` / etc.
|
||||
PDF. Without this, `do_ocr` / `processing.pictures='description'` / etc.
|
||||
silently no-op when ingesting raw `.png` / `.jpg` files (which run
|
||||
through StandardPdfPipeline). End-to-end image conversion is covered
|
||||
by the PDF picture test — both paths feed the same pipeline class."""
|
||||
|
|
@ -785,7 +785,7 @@ class TestDoclingLocalConverter:
|
|||
def test_picture_description_config_defaults(self, config):
|
||||
"""Test that picture description config has correct defaults."""
|
||||
pic_desc = config.processing.conversion_options.picture_description
|
||||
assert pic_desc.enabled is False
|
||||
assert config.processing.pictures == "image"
|
||||
assert pic_desc.model.provider == "ollama"
|
||||
assert pic_desc.model.name == "ministral-3"
|
||||
assert pic_desc.timeout == 90
|
||||
|
|
@ -795,12 +795,12 @@ class TestDoclingLocalConverter:
|
|||
|
||||
def test_picture_description_config_applied(self, config):
|
||||
"""Test that picture description config is applied to converter."""
|
||||
config.processing.conversion_options.picture_description.enabled = True
|
||||
config.processing.pictures = "description"
|
||||
config.processing.conversion_options.picture_description.timeout = 120
|
||||
converter = DoclingLocalConverter(config)
|
||||
|
||||
assert converter.config.processing.pictures == "description"
|
||||
pic_desc = converter.config.processing.conversion_options.picture_description
|
||||
assert pic_desc.enabled is True
|
||||
assert pic_desc.timeout == 120
|
||||
|
||||
@pytest.mark.asyncio
|
||||
|
|
@ -812,7 +812,7 @@ class TestDoclingLocalConverter:
|
|||
# Disable OCR (not needed for native PDF, avoids model downloads)
|
||||
config.processing.conversion_options.do_ocr = False
|
||||
# Enable picture description with Ollama
|
||||
config.processing.conversion_options.picture_description.enabled = True
|
||||
config.processing.pictures = "description"
|
||||
config.processing.conversion_options.picture_description.model.provider = (
|
||||
"ollama"
|
||||
)
|
||||
|
|
@ -1221,7 +1221,7 @@ class TestDoclingServeConverterPictureDescription:
|
|||
VLM is enabled."""
|
||||
import json
|
||||
|
||||
config.processing.conversion_options.picture_description.enabled = True
|
||||
config.processing.pictures = "description"
|
||||
config.processing.conversion_options.picture_description.model.provider = (
|
||||
"ollama"
|
||||
)
|
||||
|
|
@ -1332,7 +1332,7 @@ class TestDoclingServeConverterIntegration:
|
|||
Note: Not using VCR because this test involves polling with changing task IDs.
|
||||
"""
|
||||
pdf_path = Path("tests/data/doclaynet.pdf")
|
||||
config.processing.conversion_options.picture_description.enabled = True
|
||||
config.processing.pictures = "description"
|
||||
config.processing.conversion_options.picture_description.model.provider = (
|
||||
"ollama"
|
||||
)
|
||||
|
|
|
|||
|
|
@ -65,10 +65,10 @@ def caplog_warnings(caplog):
|
|||
|
||||
|
||||
def test_no_warning_when_picture_description_disabled(caplog_warnings):
|
||||
"""Default config has picture_description.enabled=False — even a
|
||||
document full of pictures with no descriptions should not warn."""
|
||||
"""Default config has pictures='image' — even a document full of
|
||||
pictures with no descriptions should not warn."""
|
||||
config = AppConfig()
|
||||
assert config.processing.conversion_options.picture_description.enabled is False
|
||||
assert config.processing.pictures == "image"
|
||||
doc = _doc_with_pictures(with_descriptions=False)
|
||||
|
||||
_warn_if_descriptions_missing(config, doc, "fake.pdf")
|
||||
|
|
@ -80,7 +80,7 @@ def test_no_warning_when_doc_has_no_pictures(caplog_warnings):
|
|||
"""A picture-less document under enabled=True shouldn't warn — there
|
||||
was simply nothing to describe."""
|
||||
config = AppConfig()
|
||||
config.processing.conversion_options.picture_description.enabled = True
|
||||
config.processing.pictures = "description"
|
||||
doc = _doc_without_pictures()
|
||||
|
||||
_warn_if_descriptions_missing(config, doc, "no-pictures.txt")
|
||||
|
|
@ -89,12 +89,12 @@ def test_no_warning_when_doc_has_no_pictures(caplog_warnings):
|
|||
|
||||
|
||||
def test_warns_when_pictures_present_but_no_descriptions(caplog_warnings):
|
||||
"""VLM was requested via ``picture_description.enabled = True``, the
|
||||
"""VLM was requested via ``processing.pictures='description'``, the
|
||||
doc has pictures, but the converter returned zero descriptions
|
||||
(docling-serve swallows VLM errors). Warn loudly so the user can fix
|
||||
their VLM config before a long ingest."""
|
||||
config = AppConfig()
|
||||
config.processing.conversion_options.picture_description.enabled = True
|
||||
config.processing.pictures = "description"
|
||||
config.processing.conversion_options.picture_description.model.name = "qwen3.6"
|
||||
config.processing.conversion_options.picture_description.model.base_url = (
|
||||
"http://host.docker.internal:11434"
|
||||
|
|
@ -117,7 +117,7 @@ def test_no_warning_when_at_least_one_description_came_back(caplog_warnings):
|
|||
and does not warn — the user might have area-threshold filtering or
|
||||
classification gating."""
|
||||
config = AppConfig()
|
||||
config.processing.conversion_options.picture_description.enabled = True
|
||||
config.processing.pictures = "description"
|
||||
doc = _doc_with_pictures(with_descriptions=True)
|
||||
|
||||
_warn_if_descriptions_missing(config, doc, "fine.pdf")
|
||||
|
|
@ -160,7 +160,7 @@ async def test_convert_emits_warning_via_chokepoint(
|
|||
)
|
||||
|
||||
config = AppConfig()
|
||||
config.processing.conversion_options.picture_description.enabled = True
|
||||
config.processing.pictures = "description"
|
||||
|
||||
await convert(config, pdf)
|
||||
|
||||
|
|
@ -201,7 +201,7 @@ async def test_convert_text_path_also_warns(monkeypatch, caplog_warnings):
|
|||
)
|
||||
|
||||
config = AppConfig()
|
||||
config.processing.conversion_options.picture_description.enabled = True
|
||||
config.processing.pictures = "description"
|
||||
|
||||
# No URL scheme, no Path → drops into the convert_text branch.
|
||||
await convert(config, "<html><img src='...'/></html>")
|
||||
|
|
|
|||
|
|
@ -427,12 +427,12 @@ async def test_rebuild_batch_size_flush(temp_db_path, monkeypatch):
|
|||
assert len(chunks) > 0
|
||||
|
||||
|
||||
async def test_rebuild_descriptions_requires_enabled(temp_db_path):
|
||||
"""Calling rebuild --descriptions without picture_description.enabled is
|
||||
a config error: the user has nothing to gain and the resulting state is
|
||||
async def test_rebuild_descriptions_requires_description_mode(temp_db_path):
|
||||
"""Calling rebuild --descriptions without `processing.pictures='description'`
|
||||
is a config error: the user has nothing to gain and the resulting state is
|
||||
indistinguishable from a plain --rechunk."""
|
||||
async with HaikuRAG(temp_db_path, create=True) as client:
|
||||
with pytest.raises(ValueError, match="picture_description.enabled"):
|
||||
with pytest.raises(ValueError, match="processing.pictures"):
|
||||
async for _ in client.rebuild_database(mode=RebuildMode.DESCRIPTIONS):
|
||||
pass
|
||||
|
||||
|
|
@ -459,7 +459,7 @@ async def test_rebuild_descriptions_patches_blob_and_chunks(temp_db_path, monkey
|
|||
docling_doc = _docling_doc_with_picture()
|
||||
|
||||
config = AppConfig()
|
||||
config.processing.conversion_options.picture_description.enabled = True
|
||||
config.processing.pictures = "description"
|
||||
|
||||
async with HaikuRAG(temp_db_path, config=config, create=True) as rag:
|
||||
document = Document(content="x", uri="test://doc")
|
||||
|
|
@ -543,7 +543,7 @@ async def test_rebuild_descriptions_skips_already_described(temp_db_path, monkey
|
|||
)
|
||||
|
||||
config = AppConfig()
|
||||
config.processing.conversion_options.picture_description.enabled = True
|
||||
config.processing.pictures = "description"
|
||||
|
||||
async with HaikuRAG(temp_db_path, config=config, create=True) as rag:
|
||||
document = Document(content="x", uri="test://doc")
|
||||
|
|
@ -589,7 +589,7 @@ async def test_patch_picture_descriptions_returns_zero_for_doc_without_pictures(
|
|||
from haiku.rag.config import AppConfig
|
||||
|
||||
config = AppConfig()
|
||||
config.processing.conversion_options.picture_description.enabled = True
|
||||
config.processing.pictures = "description"
|
||||
|
||||
async with HaikuRAG(temp_db_path, config=config, create=True) as rag:
|
||||
doc = await rag.create_document(content="Just text, no pictures.")
|
||||
|
|
@ -613,7 +613,7 @@ async def test_patch_picture_descriptions_warns_on_missing_bytes(temp_db_path, c
|
|||
|
||||
docling_doc = _docling_doc_with_picture()
|
||||
config = AppConfig()
|
||||
config.processing.conversion_options.picture_description.enabled = True
|
||||
config.processing.pictures = "description"
|
||||
|
||||
async with HaikuRAG(temp_db_path, config=config, create=True) as rag:
|
||||
document = Document(content="x", uri="test://doc")
|
||||
|
|
@ -670,7 +670,7 @@ async def test_patch_picture_descriptions_skips_when_all_already_described(
|
|||
)
|
||||
|
||||
config = AppConfig()
|
||||
config.processing.conversion_options.picture_description.enabled = True
|
||||
config.processing.pictures = "description"
|
||||
|
||||
async with HaikuRAG(temp_db_path, config=config, create=True) as rag:
|
||||
document = Document(content="x", uri="test://doc")
|
||||
|
|
@ -708,7 +708,7 @@ async def test_rebuild_descriptions_raises_when_blob_is_missing(
|
|||
|
||||
docling_doc = _docling_doc_with_picture()
|
||||
config = AppConfig()
|
||||
config.processing.conversion_options.picture_description.enabled = True
|
||||
config.processing.pictures = "description"
|
||||
|
||||
async with HaikuRAG(temp_db_path, config=config, create=True) as rag:
|
||||
document = Document(content="x", uri="test://doc")
|
||||
|
|
|
|||
Loading…
Reference in a new issue