Merge pull request #562 from ggozad/docs/config-truth

Make the documented configuration match the code

CHANGELOG.md

@@ -1,17 +1,10 @@
 # Changelog
 ## [Unreleased]
 
-## [0.75.0] - 2026-08-19
-
 ### Added
 
 - The `haiku.rag` package declares the `jina` extra, so `provider: jina-local` is supported by declaration rather than through `cross-encoder`'s transitive `transformers` and `torch`. Raises the full package's torch floor to 2.0.
 - `providers.docling_serve.timeout` (default 300 seconds), forwarded to the docling-serve client's per-request timeout.
-- `evaluations run --filter/-f CLAUSE`: SQL `WHERE` clause over document columns, applied to the retrieval benchmark's searches and to every capability search during QA. Recorded as `document_filter` in experiment metadata.
-- `mtrag_clapnq` / `mtrag_clapnq_rewrite` / `mtrag_clapnq_live` / `mtrag_clapnq_live_uncompacted` evaluation datasets: multi-turn QA with gold-prefix and live-session conversation replay, live arms with and without `EvidenceCompactionCapability`, Recall@k/nDCG@k retrieval metrics, eligibility-aware citation scoring, refusal precision/recall, per-turn tool-traffic attributes, and `citation_status` / `turn_citation_status` eval attributes.
-- Raw chunk metadata is now exposed to search and citation results, through `SearchResult.chunk_meta` and `Citation.chunk_meta`. For context-expanded results, the metadata is that of the anchor chunk.
-- BTree indexes on `chunks.id`, `chunks.document_id` and `documents.id`, and a Bitmap index on `document_items.label`. Existing databases need `haiku-rag migrate`.
-- `lancedb.read_consistency_interval_seconds` (default 30), `lancedb.index_cache_size_bytes` and `lancedb.metadata_cache_size_bytes`. The LanceDB session is shared across connections in a process, so its index and metadata caches survive a connection being closed.
 
 ### Changed
 
@@ -20,6 +13,33 @@
 - Numeric settings carry bounds: sizes, limits, dimensions, token budgets, attempt counts, breaker thresholds and `min_chunks` must be positive; retention, delays, intervals and cooldowns non-negative; `doctor.duplicates.similarity_threshold` within 0-1; `ingester.api.port` within 0-65535 (0 keeps its OS-assigned meaning); `ingester.workers.worker_count` allows 0 for an API-and-reaper-only process.
 - A configured reranker whose optional dependency is missing raises instead of silently disabling reranking, and names the extra to install (`uv pip install 'haiku.rag-slim[cohere]'`). A failure raised from inside an installed dependency propagates untouched rather than being reported as a missing package.
 - Multi-table writes (document create, update, batch import, cascade delete) go through `Store.write_transaction()`. Rollback restores in `RESTORE_TABLE_ORDER` and is shielded from cancellation, so a cancelled write rolls back instead of committing part of itself. `Store.restore_table_versions()` is removed.
+
+### Removed
+
+- `haiku.rag.config.Config`, the eagerly loaded configuration instance. Use `get_config()` for the current global config, or pass an `AppConfig`. Every internal default (`get_embedder`, `get_converter`, `get_chunker`, `get_reranker`, `embed_chunks`, `HaikuRAG`, `Store`, `HaikuRAGApp`, `create_mcp_server`) now takes `config: AppConfig | None = None` and resolves it per call, so `set_config()` reaches them. `RerankerBase._model` no longer defaults to the configured reranker name; `CohereReranker` takes its model name as an argument.
+
+### Fixed
+
+- `storage.data_dir: ""` resolves to the platform data directory, as documented; it coerced to `Path("")`, so the database was created in the process's working directory. Set `data_dir: .` to keep the old placement.
+- Documented `search.limit` default is 5, was stated as 10.
+- The documented way to disable reranking is omitting `reranking.model` or setting it to `null`; the previous `provider: ""` example raised `Unknown reranking provider`.
+- `prompts.picture_description: null` is not valid; the documented example sets a string or omits the key.
+- `create_document_from_source` closes the source adapter it builds for the call; adapters passed in through `sources` are left to their owner.
+- Directory ingestion skips symlinked files resolving outside the given directory, matching `FSSource.discover`.
+- A FULL rebuild no longer deletes a source-backed document before re-ingesting it: it refreshes the document in place, so the document id is preserved and a failed fetch or conversion falls back to rebuilding from stored content instead of losing the document.
+
+## [0.75.0] - 2026-08-19
+
+### Added
+
+- `evaluations run --filter/-f CLAUSE`: SQL `WHERE` clause over document columns, applied to the retrieval benchmark's searches and to every capability search during QA. Recorded as `document_filter` in experiment metadata.
+- `mtrag_clapnq` / `mtrag_clapnq_rewrite` / `mtrag_clapnq_live` / `mtrag_clapnq_live_uncompacted` evaluation datasets: multi-turn QA with gold-prefix and live-session conversation replay, live arms with and without `EvidenceCompactionCapability`, Recall@k/nDCG@k retrieval metrics, eligibility-aware citation scoring, refusal precision/recall, per-turn tool-traffic attributes, and `citation_status` / `turn_citation_status` eval attributes.
+- Raw chunk metadata is now exposed to search and citation results, through `SearchResult.chunk_meta` and `Citation.chunk_meta`. For context-expanded results, the metadata is that of the anchor chunk.
+- BTree indexes on `chunks.id`, `chunks.document_id` and `documents.id`, and a Bitmap index on `document_items.label`. Existing databases need `haiku-rag migrate`.
+- `lancedb.read_consistency_interval_seconds` (default 30), `lancedb.index_cache_size_bytes` and `lancedb.metadata_cache_size_bytes`. The LanceDB session is shared across connections in a process, so its index and metadata caches survive a connection being closed.
+
+### Changed
+
 - Search enrichment, the multimodal reranker's picture fetch, and context expansion each issue a fixed number of `document_items` queries regardless of how many documents a result set spans, instead of one set per document. `expand_with_items` takes the items to expand from rather than fetching them, and `DocumentItemRepository` gains `resolve_refs_grouped`, `get_items_in_ranges`, `get_pictures_grouped` and `get_caption_picture_refs_grouped`. The superseded methods are removed: `resolve_refs`, `get_items_in_range`, `get_caption_picture_refs`, `get_text_for_refs` and `get_all_items_grouped`. `get_pictures_grouped` returns each picture's text alongside its bytes under `with_text`, off by default so the reranker's blob fetch does not read a column it discards.
 - Eval judge pinned to `qwen3.8`: `DEFAULT_JUDGE_MODEL` is `ollama:qwen3.8`, and the reference configs use `Inferact/Qwen3.8-27B-NVFP4` with `extra_body.chat_template_kwargs.reasoning_effort: low`. Results in `docs/benchmarks.md` were judged by `Qwen3.6-35B-A3B-NVFP4` and are not re-judged.
 - `create_capability(rag=...)` lends a capability an open client rather than having it open its own; `client.ask`/`client.analyze` now pass theirs.
@@ -30,14 +50,10 @@
 
 ### Removed
 
-- `haiku.rag.config.Config`, the eagerly loaded configuration instance. Use `get_config()` for the current global config, or pass an `AppConfig`. Every internal default (`get_embedder`, `get_converter`, `get_chunker`, `get_reranker`, `embed_chunks`, `HaikuRAG`, `Store`, `HaikuRAGApp`, `create_mcp_server`) now takes `config: AppConfig | None = None` and resolves it per call, so `set_config()` reaches them. `RerankerBase._model` no longer defaults to the configured reranker name; `CohereReranker` takes its model name as an argument.
 - `wix` evaluation dataset and its reference config `evaluations/configs/wix.yaml`.
 
 ### Fixed
 
-- `create_document_from_source` closes the source adapter it builds for the call; adapters passed in through `sources` are left to their owner.
-- Directory ingestion skips symlinked files resolving outside the given directory, matching `FSSource.discover`.
-- A FULL rebuild no longer deletes a source-backed document before re-ingesting it: it refreshes the document in place, so the document id is preserved and a failed fetch or conversion falls back to rebuilding from stored content instead of losing the document.
 - `DocumentRepository.delete_all` recreated `document_items` from `DocumentItemRecord` instead of `get_document_items_arrow_schema()`, returning `picture_data` as `binary` rather than `large_binary`.
 - `server.json` runtime arguments are `mcp --stdio`, was `serve --mcp`.
 

docs/configuration/index.md

@@ -96,9 +96,10 @@ embeddings:
     vector_dim: 2560
 
 reranking:
+  # Omit this section, or set `model: null`, to disable reranking.
   model:
-    provider: ""  # Empty to disable, or cross-encoder, cohere, zeroentropy, vllm
-    name: ""
+    provider: cross-encoder  # cross-encoder, cohere, zeroentropy, vllm, jina, jina-local
+    name: cross-encoder/ms-marco-MiniLM-L-6-v2
   multimodal: false  # vllm only: send picture chunks to the reranker as images
 
 qa:
@@ -110,7 +111,7 @@ qa:
   max_searches: 5
 
 search:
-  limit: 10                    # Default number of results to return
+  limit: 5                     # Default number of results to return
   max_context_chars: 5000     # Maximum characters in expanded context
   vector_index_metric: cosine  # cosine, l2, or dot
   vector_refine_factor: 30

docs/configuration/prompts.md

@@ -12,8 +12,10 @@ prompts:
     system, including installation manuals, maintenance procedures, and safety guidelines.
     Questions about "the system" or unqualified specs refer to the Helios panel.
 
-  # VLM prompt for image description during conversion (optional)
-  picture_description: null  # Uses default prompt
+  # VLM prompt for image description during conversion.
+  # Omit the key to use the built-in prompt.
+  picture_description: |
+    Describe this figure in two sentences, naming any axis labels and units.
 ```
 
 ## Domain Preamble

docs/configuration/providers.md

@@ -391,7 +391,7 @@ See the [Pydantic AI documentation](https://ai.pydantic.dev/models/) for the com
 
 Reranking improves search quality by re-ordering the initial search results using specialized models. When enabled, the system retrieves more candidates (10x the requested limit) and then reranks them to return the most relevant results.
 
-Reranking is **disabled by default** (`provider: ""`) for faster searches. You can enable it by configuring one of the providers below.
+Reranking is **disabled by default** for faster searches: there is no `reranking.model`. Enable it by configuring one of the providers below, and disable it again by removing the section or setting `model: null`.
 
 ### Cohere
 

docs/configuration/qa.md

@@ -6,11 +6,11 @@ Configure search behavior and context expansion:
 
 ```yaml
 search:
-  limit: 10                    # Default number of results to return
+  limit: 5                     # Default number of results to return
   max_context_chars: 5000     # Maximum characters in expanded context
 ```
 
-- **limit**: Default number of search results to return when no limit is specified. Used by CLI, MCP server, and QA. Default: 10
+- **limit**: Default number of search results to return when no limit is specified. Used by CLI, MCP server, and QA. Default: 5
 - **max_context_chars**: Hard limit on total characters in expanded content. Default: 5000.
 
 Context expansion is automatic and section-aware. For structured documents (with section headers), expansion includes the entire section containing the match. For sections that exceed the budget or are too small (e.g., a title+authors area), expansion grows outward item-by-item from the match center, skipping noise labels (footnotes, page headers). This naturally crosses into adjacent sections until the budget is filled. Picture and table matches are exempt: they return their enclosing section as-is and never cross section boundaries. For unstructured documents, expansion grows outward item-by-item. Results without `doc_item_refs` (e.g., custom chunks passed to `import_document`) pass through unexpanded.

haiku_rag_slim/haiku/rag/cli.py

@@ -414,7 +414,7 @@ def settings():  # pragma: no cover
 
 
 @_cli.command("init-config", help="Generate a YAML configuration file")
-def init_config(  # pragma: no cover
+def init_config(
     output: Path = typer.Argument(
         Path("haiku.rag.yaml"),
         help="Output path for the config file",

haiku_rag_slim/haiku/rag/config/models.py

@@ -70,6 +70,19 @@ class StorageConfig(ConfigModel):
     auto_vacuum: bool = True
     vacuum_retention_seconds: int = Field(default=86400, ge=0)
 
+    @field_validator("data_dir", mode="before")
+    @classmethod
+    def _platform_default_when_empty(cls, value: Any) -> Any:
+        """An empty data_dir means the platform default directory.
+
+        Without this it coerces to Path("") — the working directory — so a
+        config carrying `data_dir: ""` would put the database wherever the
+        process happened to start.
+        """
+        if value is None or (isinstance(value, str) and not value.strip()):
+            return get_default_data_dir()
+        return value
+
 
 class LanceDBConfig(ConfigModel):
     """LanceDB connection settings.

tests/test_config.py

@@ -1,3 +1,4 @@
+import re
 from pathlib import Path
 
 import pytest
@@ -206,29 +207,36 @@ def test_generate_default_config_completeness():
     assert config.reranking.model is None
 
 
-def test_init_config_creates_valid_yaml(tmp_path):
-    """Test that generated config can be written to YAML and loaded back."""
+def test_init_config_writes_a_loadable_config(tmp_path):
+    """`haiku-rag init-config` output has to load back as an AppConfig."""
+    from typer.testing import CliRunner
+
+    from haiku.rag.cli import _cli as cli
+
     config_file = tmp_path / "test-config.yaml"
+    result = CliRunner().invoke(cli, ["init-config", str(config_file)])
 
-    # Generate and write config
-    config_data = generate_default_config()
-
-    with open(config_file, "w") as f:
-        f.write("# haiku.rag configuration file\n")
-        f.write(
-            "# See https://ggozad.github.io/haiku.rag/configuration/ for details\n\n"
-        )
-        yaml.dump(config_data, f, default_flow_style=False, sort_keys=False)
-
-    # Load it back
-    with open(config_file) as f:
-        loaded_data = yaml.safe_load(f)
-
-    # Validate it
-    config = AppConfig.model_validate(loaded_data)
+    assert result.exit_code == 0, result.output
+    config = AppConfig.model_validate(yaml.safe_load(config_file.read_text()))
     assert config.environment == "production"
 
 
+def test_init_config_refuses_to_overwrite(tmp_path):
+    """Overwriting a config in place would lose an operator's settings."""
+    from typer.testing import CliRunner
+
+    from haiku.rag.cli import _cli as cli
+
+    config_file = tmp_path / "test-config.yaml"
+    config_file.write_text("environment: development\n")
+
+    result = CliRunner().invoke(cli, ["init-config", str(config_file)])
+
+    assert result.exit_code == 1
+    assert "already exists" in result.output
+    assert config_file.read_text() == "environment: development\n"
+
+
 def _write(tmp_path, body: str):
     p = tmp_path / "haiku.rag.yaml"
     p.write_text(body)
@@ -693,3 +701,101 @@ def test_finite_switches_reject_unknown_values(data):
 def test_out_of_range_numbers_are_rejected(data):
     with pytest.raises(ValidationError):
         AppConfig.model_validate(data)
+
+
+_DOCS_ROOT = Path(__file__).resolve().parent.parent
+
+
+def _documented_config_blocks() -> list[tuple[str, int, str]]:
+    """Every fenced yaml block in the docs that looks like an AppConfig fragment."""
+    known = set(AppConfig.model_fields)
+    blocks = []
+    sources = sorted(_DOCS_ROOT.glob("docs/**/*.md")) + [
+        _DOCS_ROOT / "README.md",
+        _DOCS_ROOT / "haiku_rag_slim" / "README.md",
+    ]
+    for path in sources:
+        if not path.exists():
+            continue
+        text = path.read_text()
+        for match in re.finditer(r"```yaml\n(.*?)```", text, re.S):
+            data = yaml.safe_load(match.group(1))
+            if not isinstance(data, dict) or not (set(data) & known):
+                continue
+            line = text[: match.start()].count("\n") + 1
+            blocks.append((str(path.relative_to(_DOCS_ROOT)), line, match.group(1)))
+    return blocks
+
+
+def test_documented_config_blocks_found():
+    """Guard against the regex silently matching nothing."""
+    assert len(_documented_config_blocks()) > 20
+
+
+@pytest.mark.parametrize(
+    "rel_path, line, block",
+    _documented_config_blocks(),
+    ids=[f"{rel}:{line}" for rel, line, _ in _documented_config_blocks()],
+)
+def test_documented_config_block_validates(rel_path, line, block):
+    """A config example a reader can copy has to load. Unknown keys, wrong types
+    and removed settings all fail here rather than on their first run."""
+    AppConfig.model_validate(yaml.safe_load(block))
+
+
+def test_documented_search_limit_matches_the_default():
+    """Prose that states a default drifts silently; pin the ones that are stated."""
+    qa_doc = (_DOCS_ROOT / "docs" / "configuration" / "qa.md").read_text()
+    assert f"Default: {AppConfig().search.limit}" in qa_doc
+
+
+@pytest.mark.parametrize("value", ["", " "])
+def test_empty_data_dir_means_the_platform_default(value):
+    """Both doc pages promise this, and soliplex's example config relies on it.
+    Without it the value coerces to Path("") and the database lands in whatever
+    directory the process started from."""
+    from haiku.rag.utils import get_default_data_dir
+
+    config = AppConfig.model_validate({"storage": {"data_dir": value}})
+
+    assert config.storage.data_dir == get_default_data_dir()
+
+
+def test_explicit_relative_data_dir_is_kept():
+    """`.` is a deliberate choice and must not be rewritten."""
+    config = AppConfig.model_validate({"storage": {"data_dir": "."}})
+
+    assert config.storage.data_dir == Path(".")
+
+
+# Values the complete example shows deliberately rather than as defaults.
+_EXAMPLE_DEVIATIONS = {"storage.data_dir", "ingester.sources"}
+
+
+def _flatten(data: dict, prefix: str = "") -> dict:
+    flat = {}
+    for key, value in (data or {}).items():
+        path = f"{prefix}{key}"
+        if isinstance(value, dict):
+            flat.update(_flatten(value, path + "."))
+        else:
+            flat[path] = value
+    return flat
+
+
+def test_complete_example_matches_the_defaults():
+    """The complete configuration example doubles as the default reference, so
+    every value in it either is the default or is listed as a deliberate
+    deviation. This is what catches `limit: 10` when the default is 5."""
+    text = (_DOCS_ROOT / "docs" / "configuration" / "index.md").read_text()
+    blocks = re.findall(r"```yaml\n(.*?)```", text, re.S)
+    documented = _flatten(yaml.safe_load(max(blocks, key=len)))
+    defaults = _flatten(AppConfig().model_dump(mode="json"))
+
+    drifted = {
+        key: (value, defaults[key])
+        for key, value in documented.items()
+        if key in defaults and defaults[key] != value and key not in _EXAMPLE_DEVIATIONS
+    }
+
+    assert not drifted, f"documented value != default: {drifted}"