danvics/haiku.rag
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

rename open_rag_bench eval DBs to text/multimodal variants

Browse source
This commit is contained in:
Yiorgis Gozadinos 2026-05-05 12:55:06 +03:00
parent dca3188a48
commit 5c3a864a0b
No known key found for this signature in database
2 changed files with 3 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
docs/benchmarks.md
View file

@ -35,7 +35,8 @@ Available datasets:
| `repliqa` | ~30MB | | `repliqa` | ~30MB |
| `hotpotqa` | ~331MB | | `hotpotqa` | ~331MB |
| `wix` | ~511MB | | `wix` | ~511MB |
| `open_rag_bench` | ~14GB | | `open_rag_bench` (text embedder + VLM picture descriptions) | ~15GB |
| `open_rag_bench` (multimodal embedder, override with `--db`) | ~16GB |
After downloading, run benchmarks with `--skip-db` to use the pre-built database: After downloading, run benchmarks with `--skip-db` to use the pre-built database:

2
evaluations/evaluations/datasets/open_rag_bench.py
View file

@ -218,7 +218,7 @@ def is_multimodal_query(source: str) -> bool:
OPEN_RAG_BENCH_SPEC = DatasetSpec( OPEN_RAG_BENCH_SPEC = DatasetSpec(
key="orb", key="orb",
db_filename="open_rag_bench.lancedb", db_filename="open_rag_bench_text.lancedb",
document_loader=load_orb_corpus, document_loader=load_orb_corpus,
document_mapper=map_orb_document, document_mapper=map_orb_document,
qa_loader=load_orb_qa, qa_loader=load_orb_qa,