danvics/haiku.rag
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Benchmark update

Browse source
This commit is contained in:
Yiorgis Gozadinos 2026-05-12 17:20:48 +03:00
parent 45dfd4b382
commit 2189002974
No known key found for this signature in database
1 changed files with 16 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

16
docs/benchmarks.md
View file

@ -156,6 +156,22 @@ The text-vs-image gap on retrieval is small (0.81 pp) but on QA it widens to ~5.
#### Text embedder + VLM picture descriptions
##### Retrieval (MAP)
| Embedding Model | VLM | Source bucket | Cases | MAP |
|----------------------|----------------------|---------------|------:|-----------:|
| `qwen3-embedding:4b` | Ollama / ministral-3 | **all** | 3045 | **0.9722** |
*Measured on haiku.rag v0.45.0.*
##### QA Accuracy
| Embedding Model | VLM | QA Model | Accuracy |
|----------------------|----------------------|------------------|---------:|
| `qwen3-embedding:4b` | Ollama / ministral-3 | `ollama:qwen3.6` | 0.95 |
*Measured on haiku.rag v0.45.0, judged by `ollama:qwen3.6` (current default).*
##### Skill QA + citation retrieval
| Embedding Model | VLM | Skill model | QA accuracy | Mean `cited_map` |