Default to ollama:qwen3.8

Replaces gpt-oss on ModelConfig, qa.model and processing.title_model, and
ministral-3 on the picture-description model. qa.model.vision follows the
model and is now true.

enable_thinking was gated on the gpt-oss name, so it did nothing for
qwen3.8. With title_model's max_tokens of 100 the reasoning consumed the
whole budget and title generation returned an empty string. The mapping
now applies to any ollama model via reasoning_effort(): false sends
"none", true sends "high". Measured on qwen3.8:27b-mlx, "low" does not
disable thinking and "none" does; gpt-oss is the inverse, its template
has no "none" level, so it keeps "low".

Picture description bypasses get_model -- docling posts the request
itself from a params dict -- so the flag was inert on that path too.
vlm_api_params() carries reasoning_effort into both converters' request
bodies. At max_tokens 200 the description survived either way, but the
switch cut completion tokens from 141 to 45.

test_search_tool_skips_binary_content_when_qa_model_is_text_only asserted
the vision default rather than setting it; it now configures vision=False
itself.

docs/benchmarks.md keeps ministral-3: those are recorded measurements.
This commit is contained in:
Yiorgis Gozadinos 2026-09-04 12:13:32 +03:00
parent 660c86f991
commit 7f7223e0ac
No known key found for this signature in database
18 changed files with 132 additions and 58 deletions

View file

@ -2,6 +2,21 @@
## [Unreleased]
### Changed
- Default models are `ollama:qwen3.8`: `ModelConfig`, `qa.model`,
`processing.title_model` (was `ollama:gpt-oss`) and
`processing.conversion_options.picture_description.model` (was
`ollama:ministral-3`). Run `ollama pull qwen3.8`.
- `qa.model.vision` defaults to `true`, matching `qwen3.8`. Set it `false` when
pointing `qa.model` at a text-only model.
- `enable_thinking` on `provider: ollama` maps to `reasoning_effort` for every
model, not only `gpt-oss`: `false` sends `none`, `true` sends `high`.
`gpt-oss` keeps `low` for `false`.
- `processing.conversion_options.picture_description.model` defaults to
`enable_thinking: false`, and the field now reaches the VLM: docling's
picture-description request carries `reasoning_effort` in `params`.
## [0.82.1] - 2026-09-03
### Fixed

View file

@ -61,7 +61,7 @@ embeddings:
qa:
model:
provider: ollama
name: gpt-oss
name: qwen3.8
enable_thinking: true
```
@ -105,7 +105,7 @@ reranking:
qa:
model:
provider: ollama
name: gpt-oss
name: qwen3.8
enable_thinking: true
temperature: 0.3
max_searches: 5
@ -135,7 +135,7 @@ processing:
auto_title: false # Auto-generate titles on ingestion
title_model:
provider: ollama
name: gpt-oss
name: qwen3.8
enable_thinking: false
temperature: 0.3
max_tokens: 100

View file

@ -30,7 +30,7 @@ processing:
auto_title: false # Auto-generate titles on ingestion
title_model: # LLM for title generation (fallback)
provider: ollama
name: gpt-oss
name: qwen3.8
enable_thinking: false
# Conversion options (works with both local and remote converters)
@ -54,7 +54,7 @@ processing:
picture_description:
model:
provider: ollama
name: ministral-3
name: qwen3.8
pictures: image # none | description | image
```
@ -270,7 +270,7 @@ processing:
picture_description: # only consulted when pictures == "description"
model:
provider: ollama # any OpenAI-compatible /v1/chat/completions provider
name: ministral-3
name: qwen3.8
timeout: 90
max_tokens: 200
```
@ -294,7 +294,7 @@ Three independent settings drive ingest, retrieval, and QA:
|---|---|---|
| `processing.pictures` | Generate and/or describe pictures at ingest? | `none` / `description` / `image` (default) |
| `embeddings.model.multimodal` | Can the embedder index image content? | `false` (default, text-only) / `true` (supported on `vllm`, `voyageai`, `cohere`) |
| `qa.model.vision` | Can the QA model interpret images? | `false` (default) / `true` |
| `qa.model.vision` | Can the QA model interpret images? | `false` / `true` (default) |
The Embedder column below is driven by `embeddings.model.multimodal`, not the provider name — a vision-capable model under a text-only configuration still indexes no images, and an image-only document then produces zero chunks. See [Multimodal embedders](providers.md#multimodal-embedders).
@ -313,7 +313,7 @@ The Embedder column below is driven by `embeddings.model.multimodal`, not the pr
- `qa.model.vision: false` — text chunks only (descriptions, when present, answer figure questions in prose).
- `qa.model.vision: true` — text chunks + raw picture bytes via `BinaryContent`. The model reads figures directly. Requires `pictures != none` so the bytes exist.
`qa.model.vision` is independent of ingestion. Flipping it never requires reingesting. Setting `vision: true` against a text-only model causes silent acceptance and confabulation on Ollama and a 400 on OpenAI. Default `false` is the safe choice.
`qa.model.vision` is independent of ingestion. Flipping it never requires reingesting. It declares what the model can read: the default `qwen3.8` is vision-capable, so the default is `true`. Set it `false` when pointing `qa.model` at a text-only model, where `true` causes silent acceptance and confabulation on Ollama and a 400 on OpenAI.
**Recommended combinations:**
@ -368,7 +368,7 @@ processing:
auto_title: true
title_model:
provider: ollama
name: gpt-oss
name: qwen3.8
enable_thinking: false
```

View file

@ -15,7 +15,7 @@ Configure model behavior for the `qa` and `analysis` capabilities. These setting
qa:
model:
provider: ollama
name: gpt-oss
name: qwen3.8
temperature: 0.3
max_tokens: 500
```
@ -79,7 +79,7 @@ See the [Pydantic AI thinking documentation](https://ai.pydantic.dev/thinking/)
- **Google**: Gemini models with thinking support
- **Groq**: Models with reasoning capabilities
- **Bedrock**: Claude, Qwen, and `gpt-oss` models. Bedrock Converse does not serve the proprietary OpenAI models, so configuring one raises an error. Reach those through `provider: bedrock-mantle`.
- **Ollama**: Models supporting reasoning (gpt-oss, etc.)
- **Ollama**: Any model with a thinking capability. `enable_thinking` maps to `reasoning_effort`: `false` sends `none` (`low` for `gpt-oss`, whose template has no `none` level), `true` sends `high`.
- **vLLM**: Models with a pydantic-ai reasoning profile (gpt-oss). Qwen3, Gemma, and similar templates ignore the OpenAI `reasoning_effort` that `enable_thinking` translates to — use [`extra_body`](#raw-provider-pass-through) to drive them.
- **LM Studio**: Models supporting reasoning (gpt-oss, etc.)
@ -311,7 +311,7 @@ Configure which LLM provider to use for question answering. Any provider and mod
qa:
model:
provider: ollama
name: gpt-oss
name: qwen3.8
```
The Ollama base URL can be configured via the `OLLAMA_BASE_URL` environment variable, config file, or defaults to `http://localhost:11434`:

View file

@ -26,10 +26,10 @@ Configure the RAG capability (used by `client.ask`, `haiku-rag ask`, and the MCP
qa:
model:
provider: ollama
name: gpt-oss
name: qwen3.8
enable_thinking: true
temperature: 0.3 # Default: 0.3
vision: false # Set true for vision-capable models
vision: true # Set false for text-only models
max_searches: 5 # Maximum search units per question
```

View file

@ -12,7 +12,7 @@ You also need [Ollama](https://ollama.com/) for the default embedding and answer
```bash
ollama pull qwen3-embedding:4b
ollama pull gpt-oss
ollama pull qwen3.8
```
!!! note "Prefer OpenAI?"

View file

@ -49,7 +49,7 @@ datasets and judge:
```bash
evaluations run hotpotqa --target rag-capability
evaluations run hotpotqa --target analysis-capability --capability-model ollama:gpt-oss
evaluations run hotpotqa --target analysis-capability --capability-model ollama:qwen3.8
```
`--capability-model "provider:name"` overrides the capability model independently from

View file

@ -40,7 +40,7 @@ class ModelConfig(ConfigModel):
"""
provider: str = "ollama"
name: str = "gpt-oss"
name: str = "qwen3.8"
base_url: str | None = None
api_key: str | None = None
@ -165,9 +165,10 @@ class QAConfig(ConfigModel):
model: ModelConfig = Field(
default_factory=lambda: ModelConfig(
provider="ollama",
name="gpt-oss",
name="qwen3.8",
enable_thinking=True,
temperature=0.3,
vision=True,
)
)
max_searches: int = Field(default=5, ge=0)
@ -216,7 +217,8 @@ class PictureDescriptionConfig(ConfigModel):
model: ModelConfig = Field(
default_factory=lambda: ModelConfig(
provider="ollama",
name="ministral-3",
name="qwen3.8",
enable_thinking=False,
temperature=0.0,
)
)
@ -303,7 +305,7 @@ class ProcessingConfig(ConfigModel):
title_model: ModelConfig = Field(
default_factory=lambda: ModelConfig(
provider="ollama",
name="gpt-oss",
name="qwen3.8",
enable_thinking=False,
temperature=0.3,
max_tokens=100,

View file

@ -42,6 +42,20 @@ def vlm_api_headers(model: "ModelConfig") -> dict[str, str]:
return {}
def vlm_api_params(model: "ModelConfig", max_tokens: int) -> dict[str, object]:
"""Request body fields docling posts alongside the picture."""
from haiku.rag.utils import reasoning_effort
params: dict[str, object] = {
"model": model.name,
"max_completion_tokens": max_tokens,
}
effort = reasoning_effort(model)
if effort is not None:
params["reasoning_effort"] = effort
return params
class DocumentConverter(ABC):
"""Abstract base class for document converters.

View file

@ -12,6 +12,7 @@ from haiku.rag.config import AppConfig
from haiku.rag.converters.base import (
DocumentConverter,
vlm_api_headers,
vlm_api_params,
vlm_api_url,
)
from haiku.rag.converters.text_utils import TextFileHandler, docling_safe_name
@ -153,10 +154,7 @@ class DoclingLocalConverter(DocumentConverter):
pipeline_options.picture_description_options = PictureDescriptionApiOptions(
url=AnyUrl(vlm_api_url(self.config, pic_desc.model)),
headers=vlm_api_headers(pic_desc.model),
params=dict(
model=pic_desc.model.name,
max_completion_tokens=pic_desc.max_tokens,
),
params=vlm_api_params(pic_desc.model, pic_desc.max_tokens),
prompt=self.config.prompts.picture_description,
timeout=pic_desc.timeout,
)

View file

@ -9,6 +9,7 @@ from haiku.rag.config import AppConfig
from haiku.rag.converters.base import (
DocumentConverter,
vlm_api_headers,
vlm_api_params,
vlm_api_url,
)
from haiku.rag.converters.text_utils import TextFileHandler, docling_safe_name
@ -110,10 +111,7 @@ class DoclingServeConverter(DocumentConverter):
picture_description_api = {
"url": vlm_api_url(self.config, pic_desc.model),
"headers": vlm_api_headers(pic_desc.model),
"params": {
"model": pic_desc.model.name,
"max_completion_tokens": pic_desc.max_tokens,
},
"params": vlm_api_params(pic_desc.model, pic_desc.max_tokens),
"prompt": prompt,
"timeout": pic_desc.timeout,
}

View file

@ -4,7 +4,7 @@ import sys
from collections.abc import Awaitable
from importlib import metadata
from pathlib import Path
from typing import TYPE_CHECKING, Any, NoReturn, cast
from typing import TYPE_CHECKING, Any, Literal, NoReturn, cast
from packaging.version import Version, parse
@ -41,7 +41,7 @@ def parse_model_option(value: str) -> "ModelConfig":
parts = value.split(":", 1)
if len(parts) != 2 or not parts[0] or not parts[1]:
raise ValueError(
f"Invalid model format '{value}'. Expected 'provider:name' (e.g. 'ollama:gpt-oss')."
f"Invalid model format '{value}'. Expected 'provider:name' (e.g. 'ollama:qwen3.8')."
)
return ModelConfig(provider=parts[0], name=parts[1])
@ -182,6 +182,20 @@ _OPENAI_COMPAT_PROFILE: "OpenAIModelProfile" = {
}
def reasoning_effort(
model_config: "ModelConfig",
) -> Literal["none", "low", "high"] | None:
"""OpenAI `reasoning_effort` for a model config, or None when unset.
"low" is gpt-oss's floor; its template rejects "none".
"""
if model_config.enable_thinking is None:
return None
if model_config.enable_thinking:
return "high"
return "low" if model_config.name == "gpt-oss" else "none"
def get_model(
model_config: "ModelConfig",
app_config: "AppConfig | None" = None,
@ -213,12 +227,9 @@ def get_model(
if provider == "ollama":
model_settings = None
# Apply thinking control for gpt-oss
if model == "gpt-oss" and model_config.enable_thinking is not None:
if model_config.enable_thinking is False:
model_settings = OpenAIChatModelSettings(openai_reasoning_effort="low")
else:
model_settings = OpenAIChatModelSettings(openai_reasoning_effort="high")
effort = reasoning_effort(model_config)
if effort is not None:
model_settings = OpenAIChatModelSettings(openai_reasoning_effort=effort)
model_settings = apply_common_settings(
model_settings, model_config, map_thinking=False

View file

@ -102,7 +102,7 @@ def temp_yaml_config(tmp_path, monkeypatch):
"vector_dim": 2560,
}
},
"qa": {"model": {"provider": "ollama", "name": "gpt-oss"}},
"qa": {"model": {"provider": "ollama", "name": "qwen3.8"}},
}
with open(config_file, "w") as f:

View file

@ -15,7 +15,11 @@ from docling_core.types.doc.document import DoclingDocument
from haiku.rag.config import AppConfig
from haiku.rag.config.models import ModelConfig
from haiku.rag.converters import docling_local, get_converter
from haiku.rag.converters.base import vlm_api_headers, vlm_api_url
from haiku.rag.converters.base import (
vlm_api_headers,
vlm_api_params,
vlm_api_url,
)
from haiku.rag.converters.docling_local import DoclingLocalConverter
from haiku.rag.converters.docling_serve import DoclingServeConverter
from haiku.rag.converters.text_utils import TextFileHandler, docling_safe_name
@ -50,6 +54,32 @@ class TestVlmApiUrl:
vlm_api_url(AppConfig(), ModelConfig(provider="unsupported", name="test"))
class TestVlmApiParams:
"""Request body docling posts alongside the picture."""
def test_thinking_unset_sends_no_effort(self):
params = vlm_api_params(ModelConfig(provider="ollama", name="qwen3.8"), 200)
assert params == {"model": "qwen3.8", "max_completion_tokens": 200}
def test_thinking_off_sends_none(self):
params = vlm_api_params(
ModelConfig(provider="ollama", name="qwen3.8", enable_thinking=False), 200
)
assert params["reasoning_effort"] == "none"
def test_thinking_off_sends_gpt_oss_floor(self):
params = vlm_api_params(
ModelConfig(provider="ollama", name="gpt-oss", enable_thinking=False), 200
)
assert params["reasoning_effort"] == "low"
def test_thinking_on_sends_high(self):
params = vlm_api_params(
ModelConfig(provider="ollama", name="qwen3.8", enable_thinking=True), 200
)
assert params["reasoning_effort"] == "high"
class TestVlmApiHeaders:
"""Auth headers for picture-description VLM models."""
@ -989,7 +1019,7 @@ class TestDoclingLocalConverter:
pic_desc = config.processing.conversion_options.picture_description
assert config.processing.pictures == "image"
assert pic_desc.model.provider == "ollama"
assert pic_desc.model.name == "ministral-3"
assert pic_desc.model.name == "qwen3.8"
assert pic_desc.timeout == 90
assert pic_desc.max_tokens == 200
# Default prompt is in PromptsConfig

View file

@ -163,7 +163,7 @@ def _stub_provider_probe(monkeypatch):
{
"models": [
{"name": "test"},
{"name": "gpt-oss:latest"},
{"name": "qwen3.8:latest"},
{"name": "qwen3-embedding:4b"},
]
},
@ -751,14 +751,11 @@ def test_api_key_not_required_when_config_supplies_it():
def test_active_models_includes_picture_description_when_enabled():
config = AppConfig(processing=ProcessingConfig(pictures="description"))
names = [model.name for model in _active_models(config)]
assert "ministral-3" in names
def test_active_models_excludes_picture_description_by_default():
names = [model.name for model in _active_models(AppConfig())]
assert "ministral-3" not in names
base = _active_models(AppConfig())
with_pictures = _active_models(
AppConfig(processing=ProcessingConfig(pictures="description"))
)
assert len(with_pictures) == len(base) + 1
def test_active_models_includes_title_model_when_auto_title():
@ -846,7 +843,7 @@ def test_provider_targets_default_groups_ollama_models():
assert len(targets) == 1
entry = next(iter(targets.values()))
assert entry["kind"] == "ollama"
assert {"qwen3-embedding:4b", "gpt-oss"} <= entry["models"]
assert {"qwen3-embedding:4b", "qwen3.8"} <= entry["models"]
def test_provider_targets_includes_docling_serve():
@ -922,7 +919,7 @@ async def test_provider_check_ok_when_models_present(monkeypatch):
{
"models": [
{"name": "qwen3-embedding:4b"},
{"name": "gpt-oss:latest"},
{"name": "qwen3.8:latest"},
]
},
)
@ -960,7 +957,7 @@ async def test_provider_check_fails_when_unreachable(monkeypatch):
async def test_provider_check_reports_local_provider(monkeypatch):
monkeypatch.setattr(
"haiku.rag.doctor._probe_endpoint",
_fake_probe((True, None, {"models": [{"name": "gpt-oss:latest"}]})),
_fake_probe((True, None, {"models": [{"name": "qwen3.8:latest"}]})),
)
config = AppConfig(
embeddings=EmbeddingsConfig(
@ -981,7 +978,7 @@ async def test_run_doctor_includes_provider_results(temp_db_path, monkeypatch):
monkeypatch.setattr(
"haiku.rag.doctor._probe_endpoint",
_fake_probe(
(True, None, {"models": [{"name": "test"}, {"name": "gpt-oss:latest"}]})
(True, None, {"models": [{"name": "test"}, {"name": "qwen3.8:latest"}]})
),
)
report = await run_doctor(_config(), temp_db_path, {})

View file

@ -77,8 +77,8 @@ async def test_download_models_ollama_pulls_models(mock_to_thread):
async for progress in download_models(get_config()):
events.append(progress)
# Default config has embeddings=qwen3-embedding:4b, qa=gpt-oss
ollama_models = {"gpt-oss", "qwen3-embedding:4b"}
# Default config has embeddings=qwen3-embedding:4b, qa=qwen3.8
ollama_models = {"qwen3.8", "qwen3-embedding:4b"}
ollama_events = [e for e in events if e.model in ollama_models]
pulling_events = [e for e in ollama_events if e.status == "pulling"]
done_events = [e for e in ollama_events if e.status == "done"]
@ -108,7 +108,7 @@ async def test_download_models_no_ollama_models(mock_to_thread):
models = {e.model for e in events}
assert "qwen3-embedding:4b" not in models
assert "gpt-oss" not in models
assert "qwen3.8" not in models
@pytest.mark.parametrize(

View file

@ -837,7 +837,7 @@ async def test_ingest_emits_picture_chunks_with_multimodal_embedder(
@pytest.mark.asyncio
async def test_search_tool_skips_binary_content_when_qa_model_is_text_only():
"""The agent search tool must NOT attach picture bytes when the QA model
is text-only (``qa.model.vision = False``, the default). Sending image
is text-only (``qa.model.vision = False``). Sending image
parts to a text-only model would cause it to hallucinate confidently
Ollama silently accepts the bytes and the model guesses."""
@ -856,8 +856,7 @@ async def test_search_tool_skips_binary_content_when_qa_model_is_text_only():
fake_client.expand_context = AsyncMock(return_value=[picture_result])
config = AppConfig()
# vision defaults to False; assert anyway so the test reads explicitly.
assert config.qa.model.vision is False
config.qa.model.vision = False
toolset = create_search_toolset(config, expand_context=False)
func = toolset.tools["search"].function

View file

@ -154,6 +154,14 @@ Emoji test: 🚀 ✅ 📝"""
{"provider": "ollama", "name": "gpt-oss", "enable_thinking": True},
{"openai_reasoning_effort": "high"},
),
(
{"provider": "ollama", "name": "qwen3.8", "enable_thinking": False},
{"openai_reasoning_effort": "none"},
),
(
{"provider": "ollama", "name": "qwen3.8", "enable_thinking": True},
{"openai_reasoning_effort": "high"},
),
(
{
"provider": "ollama",
@ -188,6 +196,8 @@ Emoji test: 🚀 ✅ 📝"""
"ollama",
"ollama_thinking_off",
"ollama_thinking_on",
"ollama_other_thinking_off",
"ollama_other_thinking_on",
"ollama_with_settings",
"openai",
"openai_reasoning_thinking_on",