diff --git a/CHANGELOG.md b/CHANGELOG.md index 40ba2b0a..8834d6de 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,21 @@ ## [Unreleased] +### Changed + +- Default models are `ollama:qwen3.8`: `ModelConfig`, `qa.model`, + `processing.title_model` (was `ollama:gpt-oss`) and + `processing.conversion_options.picture_description.model` (was + `ollama:ministral-3`). Run `ollama pull qwen3.8`. +- `qa.model.vision` defaults to `true`, matching `qwen3.8`. Set it `false` when + pointing `qa.model` at a text-only model. +- `enable_thinking` on `provider: ollama` maps to `reasoning_effort` for every + model, not only `gpt-oss`: `false` sends `none`, `true` sends `high`. + `gpt-oss` keeps `low` for `false`. +- `processing.conversion_options.picture_description.model` defaults to + `enable_thinking: false`, and the field now reaches the VLM: docling's + picture-description request carries `reasoning_effort` in `params`. + ## [0.82.1] - 2026-09-03 ### Fixed diff --git a/docs/configuration/index.md b/docs/configuration/index.md index ada2c533..ab9c903e 100644 --- a/docs/configuration/index.md +++ b/docs/configuration/index.md @@ -61,7 +61,7 @@ embeddings: qa: model: provider: ollama - name: gpt-oss + name: qwen3.8 enable_thinking: true ``` @@ -105,7 +105,7 @@ reranking: qa: model: provider: ollama - name: gpt-oss + name: qwen3.8 enable_thinking: true temperature: 0.3 max_searches: 5 @@ -135,7 +135,7 @@ processing: auto_title: false # Auto-generate titles on ingestion title_model: provider: ollama - name: gpt-oss + name: qwen3.8 enable_thinking: false temperature: 0.3 max_tokens: 100 diff --git a/docs/configuration/processing.md b/docs/configuration/processing.md index d59ce8aa..13018928 100644 --- a/docs/configuration/processing.md +++ b/docs/configuration/processing.md @@ -30,7 +30,7 @@ processing: auto_title: false # Auto-generate titles on ingestion title_model: # LLM for title generation (fallback) provider: ollama - name: gpt-oss + name: qwen3.8 enable_thinking: false # Conversion options (works with both local and remote converters) @@ -54,7 +54,7 @@ processing: picture_description: model: provider: ollama - name: ministral-3 + name: qwen3.8 pictures: image # none | description | image ``` @@ -270,7 +270,7 @@ processing: picture_description: # only consulted when pictures == "description" model: provider: ollama # any OpenAI-compatible /v1/chat/completions provider - name: ministral-3 + name: qwen3.8 timeout: 90 max_tokens: 200 ``` @@ -294,7 +294,7 @@ Three independent settings drive ingest, retrieval, and QA: |---|---|---| | `processing.pictures` | Generate and/or describe pictures at ingest? | `none` / `description` / `image` (default) | | `embeddings.model.multimodal` | Can the embedder index image content? | `false` (default, text-only) / `true` (supported on `vllm`, `voyageai`, `cohere`) | -| `qa.model.vision` | Can the QA model interpret images? | `false` (default) / `true` | +| `qa.model.vision` | Can the QA model interpret images? | `false` / `true` (default) | The Embedder column below is driven by `embeddings.model.multimodal`, not the provider name — a vision-capable model under a text-only configuration still indexes no images, and an image-only document then produces zero chunks. See [Multimodal embedders](providers.md#multimodal-embedders). @@ -313,7 +313,7 @@ The Embedder column below is driven by `embeddings.model.multimodal`, not the pr - `qa.model.vision: false` — text chunks only (descriptions, when present, answer figure questions in prose). - `qa.model.vision: true` — text chunks + raw picture bytes via `BinaryContent`. The model reads figures directly. Requires `pictures != none` so the bytes exist. -`qa.model.vision` is independent of ingestion. Flipping it never requires reingesting. Setting `vision: true` against a text-only model causes silent acceptance and confabulation on Ollama and a 400 on OpenAI. Default `false` is the safe choice. +`qa.model.vision` is independent of ingestion. Flipping it never requires reingesting. It declares what the model can read: the default `qwen3.8` is vision-capable, so the default is `true`. Set it `false` when pointing `qa.model` at a text-only model, where `true` causes silent acceptance and confabulation on Ollama and a 400 on OpenAI. **Recommended combinations:** @@ -368,7 +368,7 @@ processing: auto_title: true title_model: provider: ollama - name: gpt-oss + name: qwen3.8 enable_thinking: false ``` diff --git a/docs/configuration/providers.md b/docs/configuration/providers.md index 5b5e5104..3a49f1d9 100644 --- a/docs/configuration/providers.md +++ b/docs/configuration/providers.md @@ -15,7 +15,7 @@ Configure model behavior for the `qa` and `analysis` capabilities. These setting qa: model: provider: ollama - name: gpt-oss + name: qwen3.8 temperature: 0.3 max_tokens: 500 ``` @@ -79,7 +79,7 @@ See the [Pydantic AI thinking documentation](https://ai.pydantic.dev/thinking/) - **Google**: Gemini models with thinking support - **Groq**: Models with reasoning capabilities - **Bedrock**: Claude, Qwen, and `gpt-oss` models. Bedrock Converse does not serve the proprietary OpenAI models, so configuring one raises an error. Reach those through `provider: bedrock-mantle`. -- **Ollama**: Models supporting reasoning (gpt-oss, etc.) +- **Ollama**: Any model with a thinking capability. `enable_thinking` maps to `reasoning_effort`: `false` sends `none` (`low` for `gpt-oss`, whose template has no `none` level), `true` sends `high`. - **vLLM**: Models with a pydantic-ai reasoning profile (gpt-oss). Qwen3, Gemma, and similar templates ignore the OpenAI `reasoning_effort` that `enable_thinking` translates to — use [`extra_body`](#raw-provider-pass-through) to drive them. - **LM Studio**: Models supporting reasoning (gpt-oss, etc.) @@ -311,7 +311,7 @@ Configure which LLM provider to use for question answering. Any provider and mod qa: model: provider: ollama - name: gpt-oss + name: qwen3.8 ``` The Ollama base URL can be configured via the `OLLAMA_BASE_URL` environment variable, config file, or defaults to `http://localhost:11434`: diff --git a/docs/configuration/qa.md b/docs/configuration/qa.md index 05e4c5a4..7b059bd8 100644 --- a/docs/configuration/qa.md +++ b/docs/configuration/qa.md @@ -26,10 +26,10 @@ Configure the RAG capability (used by `client.ask`, `haiku-rag ask`, and the MCP qa: model: provider: ollama - name: gpt-oss + name: qwen3.8 enable_thinking: true temperature: 0.3 # Default: 0.3 - vision: false # Set true for vision-capable models + vision: true # Set false for text-only models max_searches: 5 # Maximum search units per question ``` diff --git a/docs/tutorial.md b/docs/tutorial.md index 0d32c4a5..ab990730 100644 --- a/docs/tutorial.md +++ b/docs/tutorial.md @@ -12,7 +12,7 @@ You also need [Ollama](https://ollama.com/) for the default embedding and answer ```bash ollama pull qwen3-embedding:4b -ollama pull gpt-oss +ollama pull qwen3.8 ``` !!! note "Prefer OpenAI?" diff --git a/evaluations/README.md b/evaluations/README.md index 4aa92623..687d4874 100644 --- a/evaluations/README.md +++ b/evaluations/README.md @@ -49,7 +49,7 @@ datasets and judge: ```bash evaluations run hotpotqa --target rag-capability -evaluations run hotpotqa --target analysis-capability --capability-model ollama:gpt-oss +evaluations run hotpotqa --target analysis-capability --capability-model ollama:qwen3.8 ``` `--capability-model "provider:name"` overrides the capability model independently from diff --git a/haiku_rag_slim/haiku/rag/config/models.py b/haiku_rag_slim/haiku/rag/config/models.py index a0ab7716..0f83ff7e 100644 --- a/haiku_rag_slim/haiku/rag/config/models.py +++ b/haiku_rag_slim/haiku/rag/config/models.py @@ -40,7 +40,7 @@ class ModelConfig(ConfigModel): """ provider: str = "ollama" - name: str = "gpt-oss" + name: str = "qwen3.8" base_url: str | None = None api_key: str | None = None @@ -165,9 +165,10 @@ class QAConfig(ConfigModel): model: ModelConfig = Field( default_factory=lambda: ModelConfig( provider="ollama", - name="gpt-oss", + name="qwen3.8", enable_thinking=True, temperature=0.3, + vision=True, ) ) max_searches: int = Field(default=5, ge=0) @@ -216,7 +217,8 @@ class PictureDescriptionConfig(ConfigModel): model: ModelConfig = Field( default_factory=lambda: ModelConfig( provider="ollama", - name="ministral-3", + name="qwen3.8", + enable_thinking=False, temperature=0.0, ) ) @@ -303,7 +305,7 @@ class ProcessingConfig(ConfigModel): title_model: ModelConfig = Field( default_factory=lambda: ModelConfig( provider="ollama", - name="gpt-oss", + name="qwen3.8", enable_thinking=False, temperature=0.3, max_tokens=100, diff --git a/haiku_rag_slim/haiku/rag/converters/base.py b/haiku_rag_slim/haiku/rag/converters/base.py index 431ba651..d410a935 100644 --- a/haiku_rag_slim/haiku/rag/converters/base.py +++ b/haiku_rag_slim/haiku/rag/converters/base.py @@ -42,6 +42,20 @@ def vlm_api_headers(model: "ModelConfig") -> dict[str, str]: return {} +def vlm_api_params(model: "ModelConfig", max_tokens: int) -> dict[str, object]: + """Request body fields docling posts alongside the picture.""" + from haiku.rag.utils import reasoning_effort + + params: dict[str, object] = { + "model": model.name, + "max_completion_tokens": max_tokens, + } + effort = reasoning_effort(model) + if effort is not None: + params["reasoning_effort"] = effort + return params + + class DocumentConverter(ABC): """Abstract base class for document converters. diff --git a/haiku_rag_slim/haiku/rag/converters/docling_local.py b/haiku_rag_slim/haiku/rag/converters/docling_local.py index 71e97191..e0b9476c 100644 --- a/haiku_rag_slim/haiku/rag/converters/docling_local.py +++ b/haiku_rag_slim/haiku/rag/converters/docling_local.py @@ -12,6 +12,7 @@ from haiku.rag.config import AppConfig from haiku.rag.converters.base import ( DocumentConverter, vlm_api_headers, + vlm_api_params, vlm_api_url, ) from haiku.rag.converters.text_utils import TextFileHandler, docling_safe_name @@ -153,10 +154,7 @@ class DoclingLocalConverter(DocumentConverter): pipeline_options.picture_description_options = PictureDescriptionApiOptions( url=AnyUrl(vlm_api_url(self.config, pic_desc.model)), headers=vlm_api_headers(pic_desc.model), - params=dict( - model=pic_desc.model.name, - max_completion_tokens=pic_desc.max_tokens, - ), + params=vlm_api_params(pic_desc.model, pic_desc.max_tokens), prompt=self.config.prompts.picture_description, timeout=pic_desc.timeout, ) diff --git a/haiku_rag_slim/haiku/rag/converters/docling_serve.py b/haiku_rag_slim/haiku/rag/converters/docling_serve.py index 1ccfa21f..805cb81a 100644 --- a/haiku_rag_slim/haiku/rag/converters/docling_serve.py +++ b/haiku_rag_slim/haiku/rag/converters/docling_serve.py @@ -9,6 +9,7 @@ from haiku.rag.config import AppConfig from haiku.rag.converters.base import ( DocumentConverter, vlm_api_headers, + vlm_api_params, vlm_api_url, ) from haiku.rag.converters.text_utils import TextFileHandler, docling_safe_name @@ -110,10 +111,7 @@ class DoclingServeConverter(DocumentConverter): picture_description_api = { "url": vlm_api_url(self.config, pic_desc.model), "headers": vlm_api_headers(pic_desc.model), - "params": { - "model": pic_desc.model.name, - "max_completion_tokens": pic_desc.max_tokens, - }, + "params": vlm_api_params(pic_desc.model, pic_desc.max_tokens), "prompt": prompt, "timeout": pic_desc.timeout, } diff --git a/haiku_rag_slim/haiku/rag/utils.py b/haiku_rag_slim/haiku/rag/utils.py index 3ab93a9f..321ce3aa 100644 --- a/haiku_rag_slim/haiku/rag/utils.py +++ b/haiku_rag_slim/haiku/rag/utils.py @@ -4,7 +4,7 @@ import sys from collections.abc import Awaitable from importlib import metadata from pathlib import Path -from typing import TYPE_CHECKING, Any, NoReturn, cast +from typing import TYPE_CHECKING, Any, Literal, NoReturn, cast from packaging.version import Version, parse @@ -41,7 +41,7 @@ def parse_model_option(value: str) -> "ModelConfig": parts = value.split(":", 1) if len(parts) != 2 or not parts[0] or not parts[1]: raise ValueError( - f"Invalid model format '{value}'. Expected 'provider:name' (e.g. 'ollama:gpt-oss')." + f"Invalid model format '{value}'. Expected 'provider:name' (e.g. 'ollama:qwen3.8')." ) return ModelConfig(provider=parts[0], name=parts[1]) @@ -182,6 +182,20 @@ _OPENAI_COMPAT_PROFILE: "OpenAIModelProfile" = { } +def reasoning_effort( + model_config: "ModelConfig", +) -> Literal["none", "low", "high"] | None: + """OpenAI `reasoning_effort` for a model config, or None when unset. + + "low" is gpt-oss's floor; its template rejects "none". + """ + if model_config.enable_thinking is None: + return None + if model_config.enable_thinking: + return "high" + return "low" if model_config.name == "gpt-oss" else "none" + + def get_model( model_config: "ModelConfig", app_config: "AppConfig | None" = None, @@ -213,12 +227,9 @@ def get_model( if provider == "ollama": model_settings = None - # Apply thinking control for gpt-oss - if model == "gpt-oss" and model_config.enable_thinking is not None: - if model_config.enable_thinking is False: - model_settings = OpenAIChatModelSettings(openai_reasoning_effort="low") - else: - model_settings = OpenAIChatModelSettings(openai_reasoning_effort="high") + effort = reasoning_effort(model_config) + if effort is not None: + model_settings = OpenAIChatModelSettings(openai_reasoning_effort=effort) model_settings = apply_common_settings( model_settings, model_config, map_thinking=False diff --git a/tests/conftest.py b/tests/conftest.py index fda2e39c..0b35d944 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -102,7 +102,7 @@ def temp_yaml_config(tmp_path, monkeypatch): "vector_dim": 2560, } }, - "qa": {"model": {"provider": "ollama", "name": "gpt-oss"}}, + "qa": {"model": {"provider": "ollama", "name": "qwen3.8"}}, } with open(config_file, "w") as f: diff --git a/tests/test_converters.py b/tests/test_converters.py index b5158a77..60b87922 100644 --- a/tests/test_converters.py +++ b/tests/test_converters.py @@ -15,7 +15,11 @@ from docling_core.types.doc.document import DoclingDocument from haiku.rag.config import AppConfig from haiku.rag.config.models import ModelConfig from haiku.rag.converters import docling_local, get_converter -from haiku.rag.converters.base import vlm_api_headers, vlm_api_url +from haiku.rag.converters.base import ( + vlm_api_headers, + vlm_api_params, + vlm_api_url, +) from haiku.rag.converters.docling_local import DoclingLocalConverter from haiku.rag.converters.docling_serve import DoclingServeConverter from haiku.rag.converters.text_utils import TextFileHandler, docling_safe_name @@ -50,6 +54,32 @@ class TestVlmApiUrl: vlm_api_url(AppConfig(), ModelConfig(provider="unsupported", name="test")) +class TestVlmApiParams: + """Request body docling posts alongside the picture.""" + + def test_thinking_unset_sends_no_effort(self): + params = vlm_api_params(ModelConfig(provider="ollama", name="qwen3.8"), 200) + assert params == {"model": "qwen3.8", "max_completion_tokens": 200} + + def test_thinking_off_sends_none(self): + params = vlm_api_params( + ModelConfig(provider="ollama", name="qwen3.8", enable_thinking=False), 200 + ) + assert params["reasoning_effort"] == "none" + + def test_thinking_off_sends_gpt_oss_floor(self): + params = vlm_api_params( + ModelConfig(provider="ollama", name="gpt-oss", enable_thinking=False), 200 + ) + assert params["reasoning_effort"] == "low" + + def test_thinking_on_sends_high(self): + params = vlm_api_params( + ModelConfig(provider="ollama", name="qwen3.8", enable_thinking=True), 200 + ) + assert params["reasoning_effort"] == "high" + + class TestVlmApiHeaders: """Auth headers for picture-description VLM models.""" @@ -989,7 +1019,7 @@ class TestDoclingLocalConverter: pic_desc = config.processing.conversion_options.picture_description assert config.processing.pictures == "image" assert pic_desc.model.provider == "ollama" - assert pic_desc.model.name == "ministral-3" + assert pic_desc.model.name == "qwen3.8" assert pic_desc.timeout == 90 assert pic_desc.max_tokens == 200 # Default prompt is in PromptsConfig diff --git a/tests/test_doctor.py b/tests/test_doctor.py index eaa59ccb..0b91cbd0 100644 --- a/tests/test_doctor.py +++ b/tests/test_doctor.py @@ -163,7 +163,7 @@ def _stub_provider_probe(monkeypatch): { "models": [ {"name": "test"}, - {"name": "gpt-oss:latest"}, + {"name": "qwen3.8:latest"}, {"name": "qwen3-embedding:4b"}, ] }, @@ -751,14 +751,11 @@ def test_api_key_not_required_when_config_supplies_it(): def test_active_models_includes_picture_description_when_enabled(): - config = AppConfig(processing=ProcessingConfig(pictures="description")) - names = [model.name for model in _active_models(config)] - assert "ministral-3" in names - - -def test_active_models_excludes_picture_description_by_default(): - names = [model.name for model in _active_models(AppConfig())] - assert "ministral-3" not in names + base = _active_models(AppConfig()) + with_pictures = _active_models( + AppConfig(processing=ProcessingConfig(pictures="description")) + ) + assert len(with_pictures) == len(base) + 1 def test_active_models_includes_title_model_when_auto_title(): @@ -846,7 +843,7 @@ def test_provider_targets_default_groups_ollama_models(): assert len(targets) == 1 entry = next(iter(targets.values())) assert entry["kind"] == "ollama" - assert {"qwen3-embedding:4b", "gpt-oss"} <= entry["models"] + assert {"qwen3-embedding:4b", "qwen3.8"} <= entry["models"] def test_provider_targets_includes_docling_serve(): @@ -922,7 +919,7 @@ async def test_provider_check_ok_when_models_present(monkeypatch): { "models": [ {"name": "qwen3-embedding:4b"}, - {"name": "gpt-oss:latest"}, + {"name": "qwen3.8:latest"}, ] }, ) @@ -960,7 +957,7 @@ async def test_provider_check_fails_when_unreachable(monkeypatch): async def test_provider_check_reports_local_provider(monkeypatch): monkeypatch.setattr( "haiku.rag.doctor._probe_endpoint", - _fake_probe((True, None, {"models": [{"name": "gpt-oss:latest"}]})), + _fake_probe((True, None, {"models": [{"name": "qwen3.8:latest"}]})), ) config = AppConfig( embeddings=EmbeddingsConfig( @@ -981,7 +978,7 @@ async def test_run_doctor_includes_provider_results(temp_db_path, monkeypatch): monkeypatch.setattr( "haiku.rag.doctor._probe_endpoint", _fake_probe( - (True, None, {"models": [{"name": "test"}, {"name": "gpt-oss:latest"}]}) + (True, None, {"models": [{"name": "test"}, {"name": "qwen3.8:latest"}]}) ), ) report = await run_doctor(_config(), temp_db_path, {}) diff --git a/tests/test_download_models.py b/tests/test_download_models.py index 1b01f8c3..587e7161 100644 --- a/tests/test_download_models.py +++ b/tests/test_download_models.py @@ -77,8 +77,8 @@ async def test_download_models_ollama_pulls_models(mock_to_thread): async for progress in download_models(get_config()): events.append(progress) - # Default config has embeddings=qwen3-embedding:4b, qa=gpt-oss - ollama_models = {"gpt-oss", "qwen3-embedding:4b"} + # Default config has embeddings=qwen3-embedding:4b, qa=qwen3.8 + ollama_models = {"qwen3.8", "qwen3-embedding:4b"} ollama_events = [e for e in events if e.model in ollama_models] pulling_events = [e for e in ollama_events if e.status == "pulling"] done_events = [e for e in ollama_events if e.status == "done"] @@ -108,7 +108,7 @@ async def test_download_models_no_ollama_models(mock_to_thread): models = {e.model for e in events} assert "qwen3-embedding:4b" not in models - assert "gpt-oss" not in models + assert "qwen3.8" not in models @pytest.mark.parametrize( diff --git a/tests/test_picture_in_context.py b/tests/test_picture_in_context.py index 1f1cbd2d..3f12ddf6 100644 --- a/tests/test_picture_in_context.py +++ b/tests/test_picture_in_context.py @@ -837,7 +837,7 @@ async def test_ingest_emits_picture_chunks_with_multimodal_embedder( @pytest.mark.asyncio async def test_search_tool_skips_binary_content_when_qa_model_is_text_only(): """The agent search tool must NOT attach picture bytes when the QA model - is text-only (``qa.model.vision = False``, the default). Sending image + is text-only (``qa.model.vision = False``). Sending image parts to a text-only model would cause it to hallucinate confidently — Ollama silently accepts the bytes and the model guesses.""" @@ -856,8 +856,7 @@ async def test_search_tool_skips_binary_content_when_qa_model_is_text_only(): fake_client.expand_context = AsyncMock(return_value=[picture_result]) config = AppConfig() - # vision defaults to False; assert anyway so the test reads explicitly. - assert config.qa.model.vision is False + config.qa.model.vision = False toolset = create_search_toolset(config, expand_context=False) func = toolset.tools["search"].function diff --git a/tests/test_utils.py b/tests/test_utils.py index 82e2a877..ca0ae454 100644 --- a/tests/test_utils.py +++ b/tests/test_utils.py @@ -154,6 +154,14 @@ Emoji test: 🚀 ✅ 📝""" {"provider": "ollama", "name": "gpt-oss", "enable_thinking": True}, {"openai_reasoning_effort": "high"}, ), + ( + {"provider": "ollama", "name": "qwen3.8", "enable_thinking": False}, + {"openai_reasoning_effort": "none"}, + ), + ( + {"provider": "ollama", "name": "qwen3.8", "enable_thinking": True}, + {"openai_reasoning_effort": "high"}, + ), ( { "provider": "ollama", @@ -188,6 +196,8 @@ Emoji test: 🚀 ✅ 📝""" "ollama", "ollama_thinking_off", "ollama_thinking_on", + "ollama_other_thinking_off", + "ollama_other_thinking_on", "ollama_with_settings", "openai", "openai_reasoning_thinking_on",