danvics/haiku.rag
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

update orb benchmark

Browse source
This commit is contained in:
Yiorgis Gozadinos 2026-05-21 14:23:41 +03:00
parent 188d35023f
commit e881ce285d
No known key found for this signature in database
1 changed files with 2 additions and 5 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
docs/benchmarks.md
View file

@ -175,12 +175,9 @@ Two approaches are benchmarked separately:
| Embedding Model | VLM | Skill model | QA accuracy | Mean `cited_map` |
|------------------------|----------------------|------------------------------|-------------|------------------|
| `qwen3-embedding:4b` | Ollama / ministral-3 | `ollama:gpt-oss` | 0.94 | 0.86 |
| `qwen3-embedding:4b` | Ollama / ministral-3 | `vllm:Gemma-4-26B-A4B-NVFP4` | 0.90 | 0.89 |
| `qwen3-embedding:4b` | Ollama / ministral-3 | `vllm:Gemma-4-26B-A4B-NVFP4` | 0.88 | 0.89 |
*`ollama:gpt-oss` row measured on haiku.rag v0.44.0, on 2992 of 3044 completed cases.*
*`vllm:Gemma-4-26B-A4B-NVFP4` row measured on haiku.rag v0.47.0, with `mxbai-rerank-base-v2`, stopped at 674 of 3045 cases (cumulative means stable from case ~200).*
*Both judged by `ollama:qwen3.6` (current default).*
*Measured on haiku.rag v0.48.0, with `mxbai-rerank-base-v2`, on all 3045 cases. Judged by `vllm:Qwen3.6-35B-A3B-NVFP4`.*
## Inactive datasets