Commit graph

20 commits

Author SHA1 Message Date
Yiorgis Gozadinos
06b4f6a224
Add format parameter for text-to-DoclingDocument conversion 2025-12-08 15:56:21 +02:00
Yiorgis Gozadinos
9a5f8e4368
Add type-aware context expansion for search results 2025-12-08 15:56:21 +02:00
Yiorgis Gozadinos
a57203065a
Fix test 2025-12-08 15:55:28 +02:00
Yiorgis Gozadinos
d808c6c425
Simplify citations in qa & graph agents 2025-12-08 15:55:27 +02:00
Yiorgis Gozadinos
ace58a7034
Update evaluations and a2a 2025-12-08 15:54:56 +02:00
Yiorgis Gozadinos
3525fae625
Use EmbeddingModelConfig similar to ModelConfig for embeddings 2025-12-02 11:55:31 +02:00
Yiorgis Gozadinos
8a8005ead0
Add qa/judge models meta to experiment meta 2025-11-25 13:09:29 +02:00
Yiorgis Gozadinos
f843fb8140
Use non-thinking judge in evals 2025-11-25 13:00:25 +02:00
Yiorgis Gozadinos
e83978f88b
Revert embeddings to use flat config 2025-11-25 12:51:58 +02:00
Yiorgis Gozadinos
a9701616c8
Rename model to name under model 2025-11-25 12:28:43 +02:00
Yiorgis Gozadinos
1bb7b6d5bd
Add support for per-model configuration settings including thinking, temperature and max_tokens 2025-11-25 12:14:24 +02:00
Yiorgis Gozadinos
4e1752a740
Default eval db location, evaluation script 2025-11-25 11:07:20 +02:00
Yiorgis Gozadinos
b5892699a0
Use Mean Reciprocal Rank for single document evaluation as metric. Use Mean Average Precision for variable document evaluation as metric 2025-11-24 14:49:30 +02:00
Yiorgis Gozadinos
8d0846b88a
Record useful meta in the experiment 2025-11-24 10:46:25 +02:00
Yiorgis Gozadinos
63b374c7e9
Name evaluation runs. Use only LLMJudge evaluator. 2025-11-21 13:26:39 +02:00
Yiorgis Gozadinos
adb7d9093d
Run entire evaluation dataset as one so that it appears properly in logfire 2025-11-14 11:39:58 +02:00
Yiorgis Gozadinos
f55b8c8f19
Load .env in evals 2025-11-14 11:00:58 +02:00
Yiorgis Gozadinos
b8bc6d9b68
Let ruff know about our package structure 2025-11-05 17:47:27 +02:00
Yiorgis Gozadinos
009e529869
No need to have evaluations in the docker image 2025-11-05 12:35:37 +02:00
Yiorgis Gozadinos
2f9c907031
Restructure into uv workspace to support minimal and full installations 2025-11-04 17:59:12 +02:00