danvics/haiku.rag
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Update benchmarks

Browse source
This commit is contained in:
Yiorgis Gozadinos 2026-01-30 13:05:47 +02:00
parent 4264cdcc24
commit 5f6488e116
No known key found for this signature in database
1 changed files with 3 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
docs/benchmarks.md
View file

@ -134,7 +134,7 @@ Note the significant degradation when very small models are used such as `qwen3:
We benchmark both the plain text version (HTML stripped, no structure) and HTML version. Since HTML chunks are small (typically a phrase), we use `chunk_radius=2` to expand context. We benchmark both the plain text version (HTML stripped, no structure) and HTML version. Since HTML chunks are small (typically a phrase), we use `chunk_radius=2` to expand context.
*Results from v0.20.0* *Results from v0.27.2*
### Retrieval (MAP) ### Retrieval (MAP)
@ -149,9 +149,8 @@ We benchmark both the plain text version (HTML stripped, no structure) and HTML
| Embedding Model | Chunk size | QA Model | Accuracy | Notes | | Embedding Model | Chunk size | QA Model | Accuracy | Notes |
|----------------------|------------|-----------------------------|----------|------------------------------| |----------------------|------------|-----------------------------|----------|------------------------------|
| `qwen3-embedding:4b` | 256 | `gpt-oss:20b` - no thinking | 0.74 | plain text, `chunk-radius=0` | | `qwen3-embedding:4b` | 256 | `gpt-oss:20b` - thinking | 0.82 | html, `chunk-radius=2` |
| `qwen3-embedding:4b` | 256 | `gpt-oss:20b` - thinking | 0.79 | html, `chunk-radius=2` | | `qwen3-embedding:4b` | 256 | `gpt-oss:20b` - no thinking | 0.80 | html, `chunk-radius=2` |
| `qwen3-embedding:4b` | 256 | `gpt-oss:20b` - thinking | 0.80 | html, `chunk-radius=2`, reranker=`mxbai-rerank-base-v2` |
## HotpotQA ## HotpotQA