haiku.rag/evaluations/tests
Yiorgis Gozadinos 9b2ae347d2
Use filters that match the datasets they document
The `--filter` examples used `uri LIKE '%arxiv%'`, which matches no
`orb_text` document: its URIs are bare arXiv ids such as `2407.01528v3`.
A clause that matches nothing scores MAP 0 rather than erroring, so the
example failed silently.

`await_args` is typed `_Call | None`, so subscripting it fails
`ty check`; `call_args` carries the same call for an AsyncMock.
2026-08-17 10:19:51 +03:00
..
__init__.py
test_benchmark.py Use filters that match the datasets they document 2026-08-17 10:19:51 +03:00
test_capability_runner.py Scope the limit notice and spend the cite window on own turns only 2026-07-30 19:14:14 +03:00
test_citation_evaluators.py Remove unecessary Mean Reciprocal Rank metric 2026-06-01 10:40:52 +03:00
test_config.py remove dataset-specific system prompts 2026-04-28 14:33:25 +03:00
test_datasets.py Remove the wix evaluation dataset 2026-08-14 14:46:51 +03:00
test_evaluators.py Match the numeric scale convention in Number-Match 2026-06-06 14:52:05 +03:00
test_numbers.py Normalize unicode signs 2026-06-06 14:52:04 +03:00
test_reference_configs.py Pin the eval judge sampling and standardise on Qwen3-Reranker 2026-08-06 13:17:58 +03:00
test_submission.py Add T²-RAGBench leaderboard submission exporter 2026-06-08 11:51:02 +03:00