danvics/haiku.rag
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

add vllm Gemma-4-26B skill QA row to wix table

Browse source
This commit is contained in:
Yiorgis Gozadinos 2026-05-18 09:43:13 +03:00
parent c1923b85cd
commit f588e1150b
No known key found for this signature in database
1 changed files with 7 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

11
docs/benchmarks.md
View file

@ -179,11 +179,14 @@ Two approaches are benchmarked separately:
##### Skill QA + citation retrieval
| Embedding Model | VLM | Skill model | QA accuracy | Mean `cited_map` |
|------------------------|----------------------|------------------|-------------|------------------|
| `qwen3-embedding:4b` | Ollama / ministral-3 | `ollama:gpt-oss` | 0.94 | 0.86 |
| Embedding Model | VLM | Skill model | QA accuracy | Mean `cited_map` |
|------------------------|----------------------|------------------------------|-------------|------------------|
| `qwen3-embedding:4b` | Ollama / ministral-3 | `ollama:gpt-oss` | 0.94 | 0.86 |
| `qwen3-embedding:4b` | Ollama / ministral-3 | `vllm:Gemma-4-26B-A4B-NVFP4` | 0.90 | 0.89 |
*Measured on haiku.rag v0.44.0, judged by `ollama:qwen3.6` (current default), on 2992 of 3044 completed cases.*
*`ollama:gpt-oss` row measured on haiku.rag v0.44.0, on 2992 of 3044 completed cases.*
*`vllm:Gemma-4-26B-A4B-NVFP4` row measured on haiku.rag v0.47.0, with `mxbai-rerank-base-v2`, stopped at 674 of 3045 cases (cumulative means stable from case ~200).*
*Both judged by `ollama:qwen3.6` (current default).*
## Past results