Make the documented configuration match the code

search.limit was documented as 10 in three places while the default is 5.
The documented way to disable reranking, provider: "", is a valid
ModelConfig, so it raised "Unknown reranking provider" — disabling means
omitting reranking.model or setting it to null. The inline provider list
named four of the six rerankers. prompts.picture_description: null fails
validation, since the field is a non-optional str.

storage.data_dir: "" coerced to Path("") — the working directory — while two
doc pages promise the platform default and soliplex's example config relies
on it. Empty or whitespace now resolves to the platform directory; an
explicit "." is still honoured, so a config that wants the working directory
says so.

Three tests keep this from drifting again: every fenced yaml block in the
docs validates against AppConfig, every value in the complete example either
equals its default or is listed as a deliberate deviation, and empty
data_dir resolves to the platform default.

init-config's test reimplemented the command body instead of invoking it,
which is why the command carried a coverage pragma. It now goes through
CliRunner, with the refuse-to-overwrite guard covered too.
This commit is contained in:
Yiorgis Gozadinos 2026-08-19 15:52:50 +03:00
parent 058a820e14
commit ed5519b38d
No known key found for this signature in database
8 changed files with 153 additions and 27 deletions

View file

@ -35,6 +35,10 @@
### Fixed
- `storage.data_dir: ""` resolves to the platform data directory, as documented; it coerced to `Path("")`, so the database was created in the process's working directory. Set `data_dir: .` to keep the old placement.
- Documented `search.limit` default is 5, was stated as 10.
- The documented way to disable reranking is omitting `reranking.model` or setting it to `null`; the previous `provider: ""` example raised `Unknown reranking provider`.
- `prompts.picture_description: null` is not valid; the documented example sets a string or omits the key.
- `create_document_from_source` closes the source adapter it builds for the call; adapters passed in through `sources` are left to their owner.
- Directory ingestion skips symlinked files resolving outside the given directory, matching `FSSource.discover`.
- A FULL rebuild no longer deletes a source-backed document before re-ingesting it: it refreshes the document in place, so the document id is preserved and a failed fetch or conversion falls back to rebuilding from stored content instead of losing the document.

View file

@ -96,9 +96,10 @@ embeddings:
vector_dim: 2560
reranking:
# Omit this section, or set `model: null`, to disable reranking.
model:
provider: "" # Empty to disable, or cross-encoder, cohere, zeroentropy, vllm
name: ""
provider: cross-encoder # cross-encoder, cohere, zeroentropy, vllm, jina, jina-local
name: cross-encoder/ms-marco-MiniLM-L-6-v2
multimodal: false # vllm only: send picture chunks to the reranker as images
qa:
@ -110,7 +111,7 @@ qa:
max_searches: 5
search:
limit: 10 # Default number of results to return
limit: 5 # Default number of results to return
max_context_chars: 5000 # Maximum characters in expanded context
vector_index_metric: cosine # cosine, l2, or dot
vector_refine_factor: 30

View file

@ -12,8 +12,10 @@ prompts:
system, including installation manuals, maintenance procedures, and safety guidelines.
Questions about "the system" or unqualified specs refer to the Helios panel.
# VLM prompt for image description during conversion (optional)
picture_description: null # Uses default prompt
# VLM prompt for image description during conversion.
# Omit the key to use the built-in prompt.
picture_description: |
Describe this figure in two sentences, naming any axis labels and units.
```
## Domain Preamble

View file

@ -391,7 +391,7 @@ See the [Pydantic AI documentation](https://ai.pydantic.dev/models/) for the com
Reranking improves search quality by re-ordering the initial search results using specialized models. When enabled, the system retrieves more candidates (10x the requested limit) and then reranks them to return the most relevant results.
Reranking is **disabled by default** (`provider: ""`) for faster searches. You can enable it by configuring one of the providers below.
Reranking is **disabled by default** for faster searches: there is no `reranking.model`. Enable it by configuring one of the providers below, and disable it again by removing the section or setting `model: null`.
### Cohere

View file

@ -6,11 +6,11 @@ Configure search behavior and context expansion:
```yaml
search:
limit: 10 # Default number of results to return
limit: 5 # Default number of results to return
max_context_chars: 5000 # Maximum characters in expanded context
```
- **limit**: Default number of search results to return when no limit is specified. Used by CLI, MCP server, and QA. Default: 10
- **limit**: Default number of search results to return when no limit is specified. Used by CLI, MCP server, and QA. Default: 5
- **max_context_chars**: Hard limit on total characters in expanded content. Default: 5000.
Context expansion is automatic and section-aware. For structured documents (with section headers), expansion includes the entire section containing the match. For sections that exceed the budget or are too small (e.g., a title+authors area), expansion grows outward item-by-item from the match center, skipping noise labels (footnotes, page headers). This naturally crosses into adjacent sections until the budget is filled. Picture and table matches are exempt: they return their enclosing section as-is and never cross section boundaries. For unstructured documents, expansion grows outward item-by-item. Results without `doc_item_refs` (e.g., custom chunks passed to `import_document`) pass through unexpanded.

View file

@ -414,7 +414,7 @@ def settings(): # pragma: no cover
@_cli.command("init-config", help="Generate a YAML configuration file")
def init_config( # pragma: no cover
def init_config(
output: Path = typer.Argument(
Path("haiku.rag.yaml"),
help="Output path for the config file",

View file

@ -70,6 +70,19 @@ class StorageConfig(ConfigModel):
auto_vacuum: bool = True
vacuum_retention_seconds: int = Field(default=86400, ge=0)
@field_validator("data_dir", mode="before")
@classmethod
def _platform_default_when_empty(cls, value: Any) -> Any:
"""An empty data_dir means the platform default directory.
Without this it coerces to Path("") the working directory so a
config carrying `data_dir: ""` would put the database wherever the
process happened to start.
"""
if value is None or (isinstance(value, str) and not value.strip()):
return get_default_data_dir()
return value
class LanceDBConfig(ConfigModel):
"""LanceDB connection settings.

View file

@ -1,3 +1,4 @@
import re
from pathlib import Path
import pytest
@ -206,29 +207,36 @@ def test_generate_default_config_completeness():
assert config.reranking.model is None
def test_init_config_creates_valid_yaml(tmp_path):
"""Test that generated config can be written to YAML and loaded back."""
def test_init_config_writes_a_loadable_config(tmp_path):
"""`haiku-rag init-config` output has to load back as an AppConfig."""
from typer.testing import CliRunner
from haiku.rag.cli import _cli as cli
config_file = tmp_path / "test-config.yaml"
result = CliRunner().invoke(cli, ["init-config", str(config_file)])
# Generate and write config
config_data = generate_default_config()
with open(config_file, "w") as f:
f.write("# haiku.rag configuration file\n")
f.write(
"# See https://ggozad.github.io/haiku.rag/configuration/ for details\n\n"
)
yaml.dump(config_data, f, default_flow_style=False, sort_keys=False)
# Load it back
with open(config_file) as f:
loaded_data = yaml.safe_load(f)
# Validate it
config = AppConfig.model_validate(loaded_data)
assert result.exit_code == 0, result.output
config = AppConfig.model_validate(yaml.safe_load(config_file.read_text()))
assert config.environment == "production"
def test_init_config_refuses_to_overwrite(tmp_path):
"""Overwriting a config in place would lose an operator's settings."""
from typer.testing import CliRunner
from haiku.rag.cli import _cli as cli
config_file = tmp_path / "test-config.yaml"
config_file.write_text("environment: development\n")
result = CliRunner().invoke(cli, ["init-config", str(config_file)])
assert result.exit_code == 1
assert "already exists" in result.output
assert config_file.read_text() == "environment: development\n"
def _write(tmp_path, body: str):
p = tmp_path / "haiku.rag.yaml"
p.write_text(body)
@ -693,3 +701,101 @@ def test_finite_switches_reject_unknown_values(data):
def test_out_of_range_numbers_are_rejected(data):
with pytest.raises(ValidationError):
AppConfig.model_validate(data)
_DOCS_ROOT = Path(__file__).resolve().parent.parent
def _documented_config_blocks() -> list[tuple[str, int, str]]:
"""Every fenced yaml block in the docs that looks like an AppConfig fragment."""
known = set(AppConfig.model_fields)
blocks = []
sources = sorted(_DOCS_ROOT.glob("docs/**/*.md")) + [
_DOCS_ROOT / "README.md",
_DOCS_ROOT / "haiku_rag_slim" / "README.md",
]
for path in sources:
if not path.exists():
continue
text = path.read_text()
for match in re.finditer(r"```yaml\n(.*?)```", text, re.S):
data = yaml.safe_load(match.group(1))
if not isinstance(data, dict) or not (set(data) & known):
continue
line = text[: match.start()].count("\n") + 1
blocks.append((str(path.relative_to(_DOCS_ROOT)), line, match.group(1)))
return blocks
def test_documented_config_blocks_found():
"""Guard against the regex silently matching nothing."""
assert len(_documented_config_blocks()) > 20
@pytest.mark.parametrize(
"rel_path, line, block",
_documented_config_blocks(),
ids=[f"{rel}:{line}" for rel, line, _ in _documented_config_blocks()],
)
def test_documented_config_block_validates(rel_path, line, block):
"""A config example a reader can copy has to load. Unknown keys, wrong types
and removed settings all fail here rather than on their first run."""
AppConfig.model_validate(yaml.safe_load(block))
def test_documented_search_limit_matches_the_default():
"""Prose that states a default drifts silently; pin the ones that are stated."""
qa_doc = (_DOCS_ROOT / "docs" / "configuration" / "qa.md").read_text()
assert f"Default: {AppConfig().search.limit}" in qa_doc
@pytest.mark.parametrize("value", ["", " "])
def test_empty_data_dir_means_the_platform_default(value):
"""Both doc pages promise this, and soliplex's example config relies on it.
Without it the value coerces to Path("") and the database lands in whatever
directory the process started from."""
from haiku.rag.utils import get_default_data_dir
config = AppConfig.model_validate({"storage": {"data_dir": value}})
assert config.storage.data_dir == get_default_data_dir()
def test_explicit_relative_data_dir_is_kept():
"""`.` is a deliberate choice and must not be rewritten."""
config = AppConfig.model_validate({"storage": {"data_dir": "."}})
assert config.storage.data_dir == Path(".")
# Values the complete example shows deliberately rather than as defaults.
_EXAMPLE_DEVIATIONS = {"storage.data_dir", "ingester.sources"}
def _flatten(data: dict, prefix: str = "") -> dict:
flat = {}
for key, value in (data or {}).items():
path = f"{prefix}{key}"
if isinstance(value, dict):
flat.update(_flatten(value, path + "."))
else:
flat[path] = value
return flat
def test_complete_example_matches_the_defaults():
"""The complete configuration example doubles as the default reference, so
every value in it either is the default or is listed as a deliberate
deviation. This is what catches `limit: 10` when the default is 5."""
text = (_DOCS_ROOT / "docs" / "configuration" / "index.md").read_text()
blocks = re.findall(r"```yaml\n(.*?)```", text, re.S)
documented = _flatten(yaml.safe_load(max(blocks, key=len)))
defaults = _flatten(AppConfig().model_dump(mode="json"))
drifted = {
key: (value, defaults[key])
for key, value in documented.items()
if key in defaults and defaults[key] != value and key not in _EXAMPLE_DEVIATIONS
}
assert not drifted, f"documented value != default: {drifted}"