haiku.rag/evaluations/tests
Yiorgis Gozadinos 2bfb661c10
Pin the over-fetch rule, and assert the type a lookup raises
`_fetch_limit` had no test: a text query over-fetches `limit * 10` only with a
reranker, and an image query keeps its vector ranking either way. The sandbox
test asserted `KeyError`, which `UnknownDatabaseError` subclasses, so it could
not tell the contract from a bare one.

`uses_configured_databases` documents a mapping of one as covered; the test
named for it passed no mapping at all. Its `config` parameter is an `AppConfig`.

`test_an_analysis_capability_mounts_the_configured_set` carried a VCR marker and
no cassette, making no HTTP calls.
2026-08-28 14:03:13 +03:00
..
__init__.py Tests for gepa 2026-03-12 12:05:25 +02:00
test_benchmark.py Find a document in whichever database holds it 2026-08-24 10:03:47 +03:00
test_capability_runner.py Record the database each citation came from 2026-08-24 10:03:47 +03:00
test_citation_evaluators.py Add MTRAG ClapNQ multi-turn evaluation 2026-08-17 10:53:16 +03:00
test_config.py Pin the over-fetch rule, and assert the type a lookup raises 2026-08-28 14:03:13 +03:00
test_conversation_evaluator.py Add MTRAG ClapNQ multi-turn evaluation 2026-08-17 10:53:16 +03:00
test_datasets.py Add stable ids to FRAMES question rows 2026-08-24 09:03:44 +03:00
test_evaluators.py Add MTRAG ClapNQ multi-turn evaluation 2026-08-17 10:53:16 +03:00
test_mtrag.py Simplify the eval harness and share the embed-fill path. 2026-08-17 10:56:26 +03:00
test_numbers.py Normalize unicode signs 2026-06-06 14:52:04 +03:00
test_reference_configs.py Pin the eval judge to qwen3.8 2026-08-18 14:28:16 +03:00
test_submission.py Add T²-RAGBench leaderboard submission exporter 2026-06-08 11:51:02 +03:00