drop bogus deprecation warning for processing.pictures
This commit is contained in:
Yiorgis Gozadinos
parent
08696773fe
commit
61a88edfd8
3 changed files with 17 additions and 147 deletions
|
|
@ -10,7 +10,7 @@
|
|||
- **`vision: bool` flag on `ModelConfig`.** Tracks whether a configured language model can interpret images. Default `False`. The agent's `search` tool only attaches picture bytes (as `BinaryContent`) to the `ToolReturn` when `qa.model.vision = True`. Without the gate, sending image content to a text-only model behaves inconsistently across providers — Ollama silently accepts and the model hallucinates a confident wrong answer; OpenAI returns 400; others vary. Capability detection from a probe or a model-name whitelist is unreliable, so `vision` is an explicit user-set capability declaration. Set it to `True` for vision-capable QA models (`qwen2.5vl`, `qwen3.6`, `gpt-4o`, `claude-sonnet`, ...).
|
||||
- **Synthetic picture chunks at ingest under multimodal embedders.** `build_picture_chunks` (in `client/processing.py`) walks a `DoclingDocument`'s `pictures` and emits one synthetic `Chunk` per `PictureItem` with available bytes. Bytes ride on a `Chunk._picture_data` PrivateAttr (not serialized) so `embed_chunks` can route them through `embed_images` while text chunks keep going through `embed_documents`. Wired into the three ingest paths (`create_document`, `_create_document_from_file`, `_create_or_update_document_from_url`, `_update_document_with_chunks`, and `_rebuild_rechunk`) — guarded by `embedder.supports_images` so text-only configurations are unaffected. Snapshot/merge with `existing_picture_data` keeps picture chunks alive across rebuild round-trips. Picture chunks land in the same `chunks` table with the same vector dim as text chunks, so cross-modal search reuses the existing hybrid+RRF pipeline.
|
||||
- **Multimodal embedder support (`provider="vllm"`).** `EmbedderWrapper` gains `supports_images: bool` and `embed_image_query`. The `vllm` provider talks HTTP to a vLLM server's OpenAI-compatible `/v1/embeddings` endpoint — text inputs use the standard `input` field for true server-side batching; image inputs use vLLM's `messages` superset with `image_url` content parts carrying base64 data URIs. Works with `Qwen/Qwen3-VL-Embedding-8B` and `jinaai/jina-embeddings-v4`. No Python ML deps added — uses `httpx`. Text-only providers (`ollama`, `openai`, `cohere`, `sentence-transformers`) report `supports_images=False` and raise a clear error if image methods are called.
|
||||
- **Picture bytes are always stored.** Every ingest captures embedded picture bytes into `document_items.picture_data`; rebuilds and updates always preserve them. The single ingest-time decision is whether to run a VLM at ingest, exposed via `processing.conversion_options.picture_description.enabled: bool` (default `False`). When enabled, descriptions are woven into chunk text; when disabled, chunks contain only their natural text. No reingest is required to flip the VLM on or off — `haiku-rag rebuild --rechunk` recomputes chunk text against the new setting. The legacy `processing.pictures: "none" | "description" | "image"` enum and `processing.conversion_options.generate_picture_images` flag are auto-translated by the YAML loader with a one-time deprecation warning per field.
|
||||
- **Picture bytes are always stored.** Every ingest captures embedded picture bytes into `document_items.picture_data`; rebuilds and updates always preserve them. The single ingest-time decision is whether to run a VLM at ingest, exposed via `processing.conversion_options.picture_description.enabled: bool` (default `False`). When enabled, descriptions are woven into chunk text; when disabled, chunks contain only their natural text. No reingest is required to flip the VLM on or off — `haiku-rag rebuild --rechunk` recomputes chunk text against the new setting. The earlier `processing.conversion_options.generate_picture_images` flag is dropped from loaded YAMLs and a warning prints once telling the user to remove it from their config.
|
||||
- **Storage column for embedded picture bytes.** `DocumentItemRecord.picture_data: bytes | None` (Arrow `large_binary`), addressable by `(document_id, self_ref)`.
|
||||
- New accessors `get_picture_bytes`, `get_pictures_for_chunk`, `get_all_picture_data` on `DocumentItemRepository`.
|
||||
- Bulk read paths project a metadata-only column set so context expansion and the analysis-sandbox `items.jsonl` build never pull picture bytes into memory.
|
||||
|
|
|
|||
|
|
@ -48,59 +48,28 @@ def load_yaml_config(path: Path) -> dict:
|
|||
with open(path) as f:
|
||||
data = yaml.safe_load(f)
|
||||
data = data or {}
|
||||
_translate_legacy_picture_fields(data)
|
||||
_drop_legacy_generate_picture_images(data)
|
||||
return data
|
||||
|
||||
|
||||
def _translate_legacy_picture_fields(data: dict) -> None:
|
||||
"""Map legacy picture-handling knobs onto
|
||||
``processing.conversion_options.picture_description.enabled``.
|
||||
def _drop_legacy_generate_picture_images(data: dict) -> None:
|
||||
"""Drop ``processing.conversion_options.generate_picture_images`` from
|
||||
loaded YAML and log that it is no longer needed.
|
||||
|
||||
Two earlier shapes need translating:
|
||||
|
||||
- ``processing.pictures: "description"`` →
|
||||
``picture_description.enabled = true``. The other values
|
||||
(``"none"``, ``"image"``) collapse to ``false`` since the only
|
||||
remaining decision is whether the VLM runs; picture bytes are
|
||||
always stored.
|
||||
- ``processing.conversion_options.generate_picture_images: <any>`` is a
|
||||
no-op now (docling always extracts picture bytes) and is dropped
|
||||
with a one-time warning.
|
||||
|
||||
If ``picture_description.enabled`` is already explicitly set on the
|
||||
loaded YAML, it wins. Mutates ``data`` in-place and emits one warning
|
||||
per legacy field encountered.
|
||||
Picture bytes are always extracted now, so the flag has no effect.
|
||||
Old YAMLs keep working; users see one log line telling them they can
|
||||
remove the entry.
|
||||
"""
|
||||
processing = data.get("processing")
|
||||
if not isinstance(processing, dict):
|
||||
return
|
||||
|
||||
legacy_pictures = processing.pop("pictures", None)
|
||||
if legacy_pictures is not None:
|
||||
opts = processing.setdefault("conversion_options", {})
|
||||
if not isinstance(opts, dict):
|
||||
opts = {}
|
||||
processing["conversion_options"] = opts
|
||||
pic = opts.setdefault("picture_description", {})
|
||||
if not isinstance(pic, dict):
|
||||
pic = {}
|
||||
opts["picture_description"] = pic
|
||||
if "enabled" not in pic:
|
||||
pic["enabled"] = legacy_pictures == "description"
|
||||
logger.warning(
|
||||
"Config: 'processing.pictures' is deprecated; use "
|
||||
"'processing.conversion_options.picture_description.enabled' "
|
||||
"instead. Picture bytes are now always stored. Please update "
|
||||
"your haiku.rag.yaml."
|
||||
)
|
||||
|
||||
opts = processing.get("conversion_options")
|
||||
if isinstance(opts, dict) and "generate_picture_images" in opts:
|
||||
opts.pop("generate_picture_images", None)
|
||||
logger.warning(
|
||||
"Config: 'processing.conversion_options.generate_picture_images' "
|
||||
"is deprecated and ignored; picture bytes are always extracted. "
|
||||
"Please update your haiku.rag.yaml."
|
||||
"no longer has any effect (picture bytes are always extracted). "
|
||||
"Remove it from your haiku.rag.yaml."
|
||||
)
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -239,10 +239,9 @@ def test_init_config_creates_valid_yaml(tmp_path):
|
|||
assert config.environment == "production"
|
||||
|
||||
|
||||
# Legacy picture-handling field translation:
|
||||
# processing.pictures: "description" -> picture_description.enabled = true
|
||||
# processing.pictures: "none" | "image" -> picture_description.enabled = false
|
||||
# processing.conversion_options.generate_picture_images: <any> -> ignored
|
||||
# Removed-field handling: processing.conversion_options.generate_picture_images
|
||||
# was honored by earlier releases. It's dropped from loaded YAML now and a
|
||||
# warning is logged so the user knows they can remove it from their config.
|
||||
|
||||
|
||||
def _write(tmp_path, body: str):
|
||||
|
|
@ -251,69 +250,10 @@ def _write(tmp_path, body: str):
|
|||
return p
|
||||
|
||||
|
||||
def test_load_yaml_legacy_pictures_description_maps_to_enabled(tmp_path):
|
||||
"""`processing.pictures: description` maps to
|
||||
`picture_description.enabled = true` and warns."""
|
||||
config_file = _write(
|
||||
tmp_path,
|
||||
"""
|
||||
processing:
|
||||
pictures: description
|
||||
conversion_options:
|
||||
picture_description:
|
||||
timeout: 120
|
||||
""",
|
||||
)
|
||||
handler = _ListHandler()
|
||||
loader_logger.addHandler(handler)
|
||||
try:
|
||||
data = load_yaml_config(config_file)
|
||||
finally:
|
||||
loader_logger.removeHandler(handler)
|
||||
cfg = AppConfig.model_validate(data)
|
||||
assert cfg.processing.conversion_options.picture_description.enabled is True
|
||||
assert cfg.processing.conversion_options.picture_description.timeout == 120
|
||||
assert any("processing.pictures" in r.getMessage() for r in handler.records)
|
||||
|
||||
|
||||
def test_load_yaml_legacy_pictures_image_maps_to_disabled(tmp_path):
|
||||
"""`processing.pictures: image` is the bytes-only mode in the old
|
||||
enum; under always-store semantics it collapses to
|
||||
`picture_description.enabled = false`."""
|
||||
config_file = _write(
|
||||
tmp_path,
|
||||
"""
|
||||
processing:
|
||||
pictures: image
|
||||
""",
|
||||
)
|
||||
handler = _ListHandler()
|
||||
loader_logger.addHandler(handler)
|
||||
try:
|
||||
data = load_yaml_config(config_file)
|
||||
finally:
|
||||
loader_logger.removeHandler(handler)
|
||||
cfg = AppConfig.model_validate(data)
|
||||
assert cfg.processing.conversion_options.picture_description.enabled is False
|
||||
|
||||
|
||||
def test_load_yaml_legacy_pictures_none_maps_to_disabled(tmp_path):
|
||||
"""`processing.pictures: none` maps to `picture_description.enabled = false`."""
|
||||
config_file = _write(
|
||||
tmp_path,
|
||||
"""
|
||||
processing:
|
||||
pictures: none
|
||||
""",
|
||||
)
|
||||
data = load_yaml_config(config_file)
|
||||
cfg = AppConfig.model_validate(data)
|
||||
assert cfg.processing.conversion_options.picture_description.enabled is False
|
||||
|
||||
|
||||
def test_load_yaml_legacy_generate_picture_images_warns_and_drops(tmp_path):
|
||||
"""`generate_picture_images` is now a no-op (bytes always extracted);
|
||||
it gets dropped with a deprecation warning."""
|
||||
def test_load_yaml_drops_generate_picture_images(tmp_path):
|
||||
"""``generate_picture_images`` from earlier releases is dropped from
|
||||
the parsed dict and a warning tells the user to remove it from the
|
||||
YAML."""
|
||||
config_file = _write(
|
||||
tmp_path,
|
||||
"""
|
||||
|
|
@ -332,42 +272,3 @@ processing:
|
|||
assert cfg.processing.conversion_options.picture_description.enabled is False
|
||||
assert "generate_picture_images" not in data["processing"]["conversion_options"]
|
||||
assert any("generate_picture_images" in r.getMessage() for r in handler.records)
|
||||
|
||||
|
||||
def test_load_yaml_no_legacy_fields_keeps_default_disabled(tmp_path):
|
||||
"""Empty processing block leaves picture_description.enabled at the
|
||||
default (False) and does not warn."""
|
||||
config_file = _write(
|
||||
tmp_path,
|
||||
"""
|
||||
processing:
|
||||
chunk_size: 256
|
||||
""",
|
||||
)
|
||||
handler = _ListHandler()
|
||||
loader_logger.addHandler(handler)
|
||||
try:
|
||||
data = load_yaml_config(config_file)
|
||||
finally:
|
||||
loader_logger.removeHandler(handler)
|
||||
cfg = AppConfig.model_validate(data)
|
||||
assert cfg.processing.conversion_options.picture_description.enabled is False
|
||||
assert not handler.records
|
||||
|
||||
|
||||
def test_load_yaml_explicit_enabled_wins_over_legacy_pictures(tmp_path):
|
||||
"""If the user already set `picture_description.enabled` explicitly,
|
||||
a stale `processing.pictures` value does not override it."""
|
||||
config_file = _write(
|
||||
tmp_path,
|
||||
"""
|
||||
processing:
|
||||
pictures: none
|
||||
conversion_options:
|
||||
picture_description:
|
||||
enabled: true
|
||||
""",
|
||||
)
|
||||
data = load_yaml_config(config_file)
|
||||
cfg = AppConfig.model_validate(data)
|
||||
assert cfg.processing.conversion_options.picture_description.enabled is True
|
||||
|
|
|
|||
Loading…
Reference in a new issue