danvics/haiku.rag
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add ORB multimodal-reranker retrieval result to benchmarks

Browse source
This commit is contained in:
Yiorgis Gozadinos 2026-07-24 13:25:07 +03:00
parent 71737095ea
commit a72c94aa77
No known key found for this signature in database
1 changed files with 7 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

11
docs/benchmarks.md
View file

@ -120,10 +120,13 @@ Two approaches are benchmarked separately:
##### Retrieval (MAP)
| Embedding Model | Cases | MAP |
|------------------------------------------|------:|-------:|
| `Qwen/Qwen3-VL-Embedding-8B` | 3045 | 0.9774 |
| `nvidia/llama-nemotron-embed-vl-1b-v2` | 3045 | 0.9709 |
| Embedding Model | Reranker | Cases | MAP |
|------------------------------------------|------------------------------------------------------|------:|-------:|
| `Qwen/Qwen3-VL-Embedding-8B` | none | 3045 | 0.9774 |
| `nvidia/llama-nemotron-embed-vl-1b-v2` | none | 3045 | 0.9709 |
| `nvidia/llama-nemotron-embed-vl-1b-v2` | `nvidia/llama-nemotron-rerank-vl-1b-v2` (multimodal) | 3045 | 0.9913 |
*The reranked row uses `reranking.multimodal: true`: picture chunks reach the vision reranker as images alongside their description text. Measured on haiku.rag main post-v0.67.3 (multimodal reranking ships in the next release).*
##### QA accuracy + citation retrieval