26 commits

Author	SHA1	Message	Date
Yiorgis Gozadinos	b5892699a0	Use Mean Reciprocal Rank for single document evaluation as metric. Use Mean Average Precision for variable document evaluation as metric	2025-11-24 14:49:30 +02:00
Yiorgis Gozadinos	fe14e73514	Benchmarks for win qwen3-embedding:4b mxbai-rerank-base-v2	2025-11-21 18:57:07 +02:00
Yiorgis Gozadinos	9a1e7f211c	Update benchmarks	2025-11-21 15:31:36 +02:00
Yiorgis Gozadinos	63b374c7e9	Name evaluation runs. Use only LLMJudge evaluator.	2025-11-21 13:26:39 +02:00
Yiorgis Gozadinos	dcba9d9a0e	Benchmarks for qwen3-embedding:0.6b	2025-11-14 15:36:36 +02:00
Yiorgis Gozadinos	2f9c907031	Restructure into uv workspace to support minimal and full installations	2025-11-04 17:59:12 +02:00
Yiorgis Gozadinos	a7fd41d24f	Document passing config files when running evaluations	2025-10-30 15:17:02 +02:00
Yiorgis Gozadinos	6f181e07ff	Add Success@K metrics to retrieval benchmarks	2025-10-06 14:54:17 +03:00
Yiorgis Gozadinos	43b9cd50ba	Move evaluations to src/	2025-09-30 13:21:53 +03:00
Yiorgis Gozadinos	06dbdbd474	Update docs for wix benchmarks	2025-09-30 11:31:52 +03:00
Yiorgis Gozadinos	6f20d9fd6d	Update recall of qwen3 embeddings with mxbai reranking	2025-09-30 11:31:51 +03:00
Yiorgis Gozadinos	842e166041	Adapt how we measure recall when using datasets with multiple sources	2025-09-30 11:31:51 +03:00
Yiorgis Gozadinos	57e86d00fd	Refactor evaluations so that we can perform with multiple datasets.. ... Introduce Wix dataset.	2025-09-30 11:31:50 +03:00
Yiorgis Gozadinos	87f28ed96b	Update docs	2025-09-30 11:30:35 +03:00
Yiorgis Gozadinos	f0333e33ac	Change default embedding to qwen3-embedding. Update benchmarks using qwen3-embedding and gpt-oss	2025-09-26 13:23:38 +03:00
Yiorgis Gozadinos	d90dfe258c	Use logfire when running evaluations	2025-09-12 09:45:06 +03:00
Yiorgis Gozadinos	b73949c784	update benchmarks	2025-09-02 15:10:57 +03:00
Yiorgis Gozadinos	fe850e778d	Remove Ollama reranker, this was a bad idea	2025-09-02 11:42:31 +03:00
Yiorgis Gozadinos	27b16bdf8d	Update basic benchmarks	2025-09-02 10:51:20 +03:00
Yiorgis Gozadinos	b05e5f5408	Set/get haiku version to db	2025-09-01 17:41:41 +03:00
Yiorgis Gozadinos	4908ea9fcc	Update benchmarks	2025-07-20 22:45:24 +03:00
Yiorgis Gozadinos	c1d5334804	Update benchmarks with reranking	2025-07-19 19:02:47 +03:00
Yiorgis Gozadinos	eefd51ead6	Update mxbai recall stats	2025-07-09 12:50:39 +03:00
Yiorgis Gozadinos	42e62f92a1	Better formatting of benchmark script	2025-07-09 10:27:32 +03:00
Yiorgis Gozadinos	9d72a158a6	Document benchmarks	2025-07-08 19:01:54 +03:00
Yiorgis Gozadinos	2f261a33f5	Allow multiple tool calling rounds in QA agent	2025-07-08 12:51:58 +03:00

Renamed from BENCHMARKS.md (Browse further)