Yiorgis Gozadinos
|
b5892699a0
|
Use Mean Reciprocal Rank for single document evaluation as metric. Use Mean Average Precision for variable document evaluation as metric
|
2025-11-24 14:49:30 +02:00 |
|
Yiorgis Gozadinos
|
8d0846b88a
|
Record useful meta in the experiment
|
2025-11-24 10:46:25 +02:00 |
|
Yiorgis Gozadinos
|
ada2b66d5b
|
vb
|
2025-11-21 16:38:52 +02:00 |
|
Yiorgis Gozadinos
|
63b374c7e9
|
Name evaluation runs. Use only LLMJudge evaluator.
|
2025-11-21 13:26:39 +02:00 |
|
Yiorgis Gozadinos
|
09900f8e8b
|
vb
|
2025-11-19 13:08:37 +02:00 |
|
Yiorgis Gozadinos
|
0ea5f1e10e
|
Update dependencies
|
2025-11-19 10:24:37 +02:00 |
|
Yiorgis Gozadinos
|
826986708d
|
vb
|
2025-11-18 15:52:52 +02:00 |
|
Yiorgis Gozadinos
|
efbd5b92f7
|
vb
|
2025-11-17 16:30:39 +02:00 |
|
Yiorgis Gozadinos
|
636d5261e6
|
vb
|
2025-11-14 22:29:37 +02:00 |
|
Yiorgis Gozadinos
|
adb7d9093d
|
Run entire evaluation dataset as one so that it appears properly in logfire
|
2025-11-14 11:39:58 +02:00 |
|
Yiorgis Gozadinos
|
c8999a89b8
|
Update and lock pydantic-ai-slim
|
2025-11-14 11:26:09 +02:00 |
|
Yiorgis Gozadinos
|
f55b8c8f19
|
Load .env in evals
|
2025-11-14 11:00:58 +02:00 |
|
Yiorgis Gozadinos
|
6eb0d59a28
|
vb
|
2025-11-13 15:00:57 +02:00 |
|
Yiorgis Gozadinos
|
34f50f37a0
|
vb
|
2025-11-07 14:44:58 +02:00 |
|
Yiorgis Gozadinos
|
0e3f2ebd67
|
vb
|
2025-11-06 16:13:53 +02:00 |
|
Yiorgis Gozadinos
|
8da62e3868
|
Known-first party for ruff evaluations
|
2025-11-06 11:01:12 +02:00 |
|
Yiorgis Gozadinos
|
b8bc6d9b68
|
Let ruff know about our package structure
|
2025-11-05 17:47:27 +02:00 |
|
Yiorgis Gozadinos
|
21bb7b5aab
|
Version bump & Changelog
|
2025-11-05 13:16:03 +02:00 |
|
Yiorgis Gozadinos
|
009e529869
|
No need to have evaluations in the docker image
|
2025-11-05 12:35:37 +02:00 |
|
Yiorgis Gozadinos
|
ba8e503cb1
|
Depend on pydantic-ai-slim with extras
|
2025-11-05 11:37:06 +02:00 |
|
Yiorgis Gozadinos
|
f06dac7102
|
Update READMEs
|
2025-11-05 10:47:00 +02:00 |
|
Yiorgis Gozadinos
|
2f9c907031
|
Restructure into uv workspace to support minimal and full installations
|
2025-11-04 17:59:12 +02:00 |
|
Yiorgis Gozadinos
|
43b9cd50ba
|
Move evaluations to src/
|
2025-09-30 13:21:53 +03:00 |
|
Yiorgis Gozadinos
|
abfc796c42
|
Use gpt-oss for evaluation LLMJudge, allow it to retry if it fails
|
2025-09-30 11:31:51 +03:00 |
|
Yiorgis Gozadinos
|
9a859c6ee5
|
Add option to skip db in evals
|
2025-09-30 11:31:51 +03:00 |
|
Yiorgis Gozadinos
|
842e166041
|
Adapt how we measure recall when using datasets with multiple sources
|
2025-09-30 11:31:51 +03:00 |
|
Yiorgis Gozadinos
|
eb2fc67e1c
|
When populating the eval db, check if chunks for the document have been created. Protects against interrupts
|
2025-09-30 11:31:50 +03:00 |
|
Yiorgis Gozadinos
|
57e86d00fd
|
Refactor evaluations so that we can perform with multiple datasets..
Introduce Wix dataset.
|
2025-09-30 11:31:50 +03:00 |
|