16a6881454
`mtrag_federated` partitions one domain by article title, which is round-robin fusion's friendliest case: no collection is ever off-topic for a query, so the guaranteed-slot waste that hurts a real deployment is never exercised. Every fusion conclusion measured on it is therefore provisional. `mtrag_pooled` pools all four MTRAG domains, so a query belongs to one and the rest are genuinely off-topic. `collection_of` gains `alpha`, which now means something: 0 keeps a collection to one domain, 1 ignores the domain and shards titles uniformly. Domains map onto collections proportionally, subdividing by title where there are more collections than domains and grouping where there are fewer. Passage ids are checked for collisions across domains, since gold is uri-keyed and a shared id would make it ambiguous. Claude-Session: https://claude.ai/code/session_01WhudUtZm6qqiuv8Y1sbwSc |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| test_benchmark.py | ||
| test_capability_runner.py | ||
| test_citation_evaluators.py | ||
| test_config.py | ||
| test_conversation_evaluator.py | ||
| test_datasets.py | ||
| test_evaluators.py | ||
| test_mtrag.py | ||
| test_mtrag_federated.py | ||
| test_numbers.py | ||
| test_reference_configs.py | ||
| test_submission.py | ||