diff --git a/CHANGELOG.md b/CHANGELOG.md index 79da7e65..a1868cfb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,7 +10,7 @@ - **`vision: bool` flag on `ModelConfig`.** Tracks whether a configured language model can interpret images. Default `False`. The agent's `search` tool only attaches picture bytes (as `BinaryContent`) to the `ToolReturn` when `qa.model.vision = True`. Without the gate, sending image content to a text-only model behaves inconsistently across providers — Ollama silently accepts and the model hallucinates a confident wrong answer; OpenAI returns 400; others vary. Capability detection from a probe or a model-name whitelist is unreliable, so `vision` is an explicit user-set capability declaration. Set it to `True` for vision-capable QA models (`qwen2.5vl`, `qwen3.6`, `gpt-4o`, `claude-sonnet`, ...). - **Synthetic picture chunks at ingest under multimodal embedders.** `build_picture_chunks` (in `client/processing.py`) walks a `DoclingDocument`'s `pictures` and emits one synthetic `Chunk` per `PictureItem` with available bytes. Bytes ride on a `Chunk._picture_data` PrivateAttr (not serialized) so `embed_chunks` can route them through `embed_images` while text chunks keep going through `embed_documents`. Wired into the three ingest paths (`create_document`, `_create_document_from_file`, `_create_or_update_document_from_url`, `_update_document_with_chunks`, and `_rebuild_rechunk`) — guarded by `embedder.supports_images` so text-only configurations are unaffected. Snapshot/merge with `existing_picture_data` keeps picture chunks alive across rebuild round-trips. Picture chunks land in the same `chunks` table with the same vector dim as text chunks, so cross-modal search reuses the existing hybrid+RRF pipeline. - **Multimodal embedder support (`provider="vllm"`).** `EmbedderWrapper` gains `supports_images: bool` and `embed_image_query`. The `vllm` provider talks HTTP to a vLLM server's OpenAI-compatible `/v1/embeddings` endpoint — text inputs use the standard `input` field for true server-side batching; image inputs use vLLM's `messages` superset with `image_url` content parts carrying base64 data URIs. Works with `Qwen/Qwen3-VL-Embedding-8B` and `jinaai/jina-embeddings-v4`. No Python ML deps added — uses `httpx`. Text-only providers (`ollama`, `openai`, `cohere`, `sentence-transformers`) report `supports_images=False` and raise a clear error if image methods are called. -- **Picture bytes are always stored.** Every ingest captures embedded picture bytes into `document_items.picture_data`; rebuilds and updates always preserve them. The single ingest-time decision is whether to run a VLM at ingest, exposed via `processing.conversion_options.picture_description.enabled: bool` (default `False`). When enabled, descriptions are woven into chunk text; when disabled, chunks contain only their natural text. No reingest is required to flip the VLM on or off — `haiku-rag rebuild --rechunk` recomputes chunk text against the new setting. The legacy `processing.pictures: "none" | "description" | "image"` enum and `processing.conversion_options.generate_picture_images` flag are auto-translated by the YAML loader with a one-time deprecation warning per field. +- **Picture bytes are always stored.** Every ingest captures embedded picture bytes into `document_items.picture_data`; rebuilds and updates always preserve them. The single ingest-time decision is whether to run a VLM at ingest, exposed via `processing.conversion_options.picture_description.enabled: bool` (default `False`). When enabled, descriptions are woven into chunk text; when disabled, chunks contain only their natural text. No reingest is required to flip the VLM on or off — `haiku-rag rebuild --rechunk` recomputes chunk text against the new setting. The earlier `processing.conversion_options.generate_picture_images` flag is dropped from loaded YAMLs and a warning prints once telling the user to remove it from their config. - **Storage column for embedded picture bytes.** `DocumentItemRecord.picture_data: bytes | None` (Arrow `large_binary`), addressable by `(document_id, self_ref)`. - New accessors `get_picture_bytes`, `get_pictures_for_chunk`, `get_all_picture_data` on `DocumentItemRepository`. - Bulk read paths project a metadata-only column set so context expansion and the analysis-sandbox `items.jsonl` build never pull picture bytes into memory. diff --git a/haiku_rag_slim/haiku/rag/config/loader.py b/haiku_rag_slim/haiku/rag/config/loader.py index ca853b27..d1af459f 100644 --- a/haiku_rag_slim/haiku/rag/config/loader.py +++ b/haiku_rag_slim/haiku/rag/config/loader.py @@ -48,59 +48,28 @@ def load_yaml_config(path: Path) -> dict: with open(path) as f: data = yaml.safe_load(f) data = data or {} - _translate_legacy_picture_fields(data) + _drop_legacy_generate_picture_images(data) return data -def _translate_legacy_picture_fields(data: dict) -> None: - """Map legacy picture-handling knobs onto - ``processing.conversion_options.picture_description.enabled``. +def _drop_legacy_generate_picture_images(data: dict) -> None: + """Drop ``processing.conversion_options.generate_picture_images`` from + loaded YAML and log that it is no longer needed. - Two earlier shapes need translating: - - - ``processing.pictures: "description"`` → - ``picture_description.enabled = true``. The other values - (``"none"``, ``"image"``) collapse to ``false`` since the only - remaining decision is whether the VLM runs; picture bytes are - always stored. - - ``processing.conversion_options.generate_picture_images: `` is a - no-op now (docling always extracts picture bytes) and is dropped - with a one-time warning. - - If ``picture_description.enabled`` is already explicitly set on the - loaded YAML, it wins. Mutates ``data`` in-place and emits one warning - per legacy field encountered. + Picture bytes are always extracted now, so the flag has no effect. + Old YAMLs keep working; users see one log line telling them they can + remove the entry. """ processing = data.get("processing") if not isinstance(processing, dict): return - - legacy_pictures = processing.pop("pictures", None) - if legacy_pictures is not None: - opts = processing.setdefault("conversion_options", {}) - if not isinstance(opts, dict): - opts = {} - processing["conversion_options"] = opts - pic = opts.setdefault("picture_description", {}) - if not isinstance(pic, dict): - pic = {} - opts["picture_description"] = pic - if "enabled" not in pic: - pic["enabled"] = legacy_pictures == "description" - logger.warning( - "Config: 'processing.pictures' is deprecated; use " - "'processing.conversion_options.picture_description.enabled' " - "instead. Picture bytes are now always stored. Please update " - "your haiku.rag.yaml." - ) - opts = processing.get("conversion_options") if isinstance(opts, dict) and "generate_picture_images" in opts: opts.pop("generate_picture_images", None) logger.warning( "Config: 'processing.conversion_options.generate_picture_images' " - "is deprecated and ignored; picture bytes are always extracted. " - "Please update your haiku.rag.yaml." + "no longer has any effect (picture bytes are always extracted). " + "Remove it from your haiku.rag.yaml." ) diff --git a/tests/test_config.py b/tests/test_config.py index 61da2c7f..2f778e58 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -239,10 +239,9 @@ def test_init_config_creates_valid_yaml(tmp_path): assert config.environment == "production" -# Legacy picture-handling field translation: -# processing.pictures: "description" -> picture_description.enabled = true -# processing.pictures: "none" | "image" -> picture_description.enabled = false -# processing.conversion_options.generate_picture_images: -> ignored +# Removed-field handling: processing.conversion_options.generate_picture_images +# was honored by earlier releases. It's dropped from loaded YAML now and a +# warning is logged so the user knows they can remove it from their config. def _write(tmp_path, body: str): @@ -251,69 +250,10 @@ def _write(tmp_path, body: str): return p -def test_load_yaml_legacy_pictures_description_maps_to_enabled(tmp_path): - """`processing.pictures: description` maps to - `picture_description.enabled = true` and warns.""" - config_file = _write( - tmp_path, - """ -processing: - pictures: description - conversion_options: - picture_description: - timeout: 120 -""", - ) - handler = _ListHandler() - loader_logger.addHandler(handler) - try: - data = load_yaml_config(config_file) - finally: - loader_logger.removeHandler(handler) - cfg = AppConfig.model_validate(data) - assert cfg.processing.conversion_options.picture_description.enabled is True - assert cfg.processing.conversion_options.picture_description.timeout == 120 - assert any("processing.pictures" in r.getMessage() for r in handler.records) - - -def test_load_yaml_legacy_pictures_image_maps_to_disabled(tmp_path): - """`processing.pictures: image` is the bytes-only mode in the old - enum; under always-store semantics it collapses to - `picture_description.enabled = false`.""" - config_file = _write( - tmp_path, - """ -processing: - pictures: image -""", - ) - handler = _ListHandler() - loader_logger.addHandler(handler) - try: - data = load_yaml_config(config_file) - finally: - loader_logger.removeHandler(handler) - cfg = AppConfig.model_validate(data) - assert cfg.processing.conversion_options.picture_description.enabled is False - - -def test_load_yaml_legacy_pictures_none_maps_to_disabled(tmp_path): - """`processing.pictures: none` maps to `picture_description.enabled = false`.""" - config_file = _write( - tmp_path, - """ -processing: - pictures: none -""", - ) - data = load_yaml_config(config_file) - cfg = AppConfig.model_validate(data) - assert cfg.processing.conversion_options.picture_description.enabled is False - - -def test_load_yaml_legacy_generate_picture_images_warns_and_drops(tmp_path): - """`generate_picture_images` is now a no-op (bytes always extracted); - it gets dropped with a deprecation warning.""" +def test_load_yaml_drops_generate_picture_images(tmp_path): + """``generate_picture_images`` from earlier releases is dropped from + the parsed dict and a warning tells the user to remove it from the + YAML.""" config_file = _write( tmp_path, """ @@ -332,42 +272,3 @@ processing: assert cfg.processing.conversion_options.picture_description.enabled is False assert "generate_picture_images" not in data["processing"]["conversion_options"] assert any("generate_picture_images" in r.getMessage() for r in handler.records) - - -def test_load_yaml_no_legacy_fields_keeps_default_disabled(tmp_path): - """Empty processing block leaves picture_description.enabled at the - default (False) and does not warn.""" - config_file = _write( - tmp_path, - """ -processing: - chunk_size: 256 -""", - ) - handler = _ListHandler() - loader_logger.addHandler(handler) - try: - data = load_yaml_config(config_file) - finally: - loader_logger.removeHandler(handler) - cfg = AppConfig.model_validate(data) - assert cfg.processing.conversion_options.picture_description.enabled is False - assert not handler.records - - -def test_load_yaml_explicit_enabled_wins_over_legacy_pictures(tmp_path): - """If the user already set `picture_description.enabled` explicitly, - a stale `processing.pictures` value does not override it.""" - config_file = _write( - tmp_path, - """ -processing: - pictures: none - conversion_options: - picture_description: - enabled: true -""", - ) - data = load_yaml_config(config_file) - cfg = AppConfig.model_validate(data) - assert cfg.processing.conversion_options.picture_description.enabled is True