Yiorgis Gozadinos
b5c0ea630c
vb
2026-07-16 13:50:40 +03:00
Yiorgis Gozadinos
5dd310a9b7
vb
2026-07-14 16:06:50 +03:00
Yiorgis Gozadinos
144900d385
Remove the mxbai reranking provider
2026-07-14 11:09:55 +03:00
Yiorgis Gozadinos
5c41f8a0ed
vb
2026-07-10 13:38:24 +03:00
Yiorgis Gozadinos
271fdf9b5a
Add Logfire debugging skills and worker-breaker event
2026-07-10 13:23:17 +03:00
Yiorgis Gozadinos
a82673a900
Route eval scripts through the shared telemetry configure
2026-07-10 11:38:33 +03:00
Yiorgis Gozadinos
da0748916b
vb
2026-07-09 15:58:58 +03:00
Yiorgis Gozadinos
a1cf405cae
Rename document_meta identity column document_id to id
2026-07-08 16:32:55 +03:00
Yiorgis Gozadinos
9a9bea86d3
vb
2026-07-03 13:35:10 +03:00
Yiorgis Gozadinos
9a85632cf3
vb
2026-06-29 12:36:36 +03:00
Yiorgis Gozadinos
325f4517ba
Update t2_finqa config
2026-06-29 12:22:58 +03:00
Yiorgis Gozadinos
3cb229d2e0
Add t2_finqa pre-built evaluation database reference config and docs
2026-06-29 10:52:23 +03:00
Yiorgis Gozadinos
4d03b1e669
Add nemotron-vl multimodal eval database and reference configs
2026-06-29 10:14:53 +03:00
Yiorgis Gozadinos
b2abdfaffb
vb
2026-06-28 10:57:26 +03:00
Yiorgis Gozadinos
c7c3f75508
vb
2026-06-27 09:59:21 +03:00
Yiorgis Gozadinos
cd1a73b389
vb
2026-06-26 17:13:03 +03:00
Yiorgis Gozadinos
197d58a602
vb
2026-06-24 18:41:05 +03:00
Yiorgis Gozadinos
ab1775dbc6
vb
2026-06-24 16:45:19 +03:00
Yiorgis Gozadinos
65f1339aa0
vb
2026-06-23 16:59:00 +03:00
Yiorgis Gozadinos
0fcf91fcf7
vb
2026-06-22 15:18:35 +03:00
Yiorgis Gozadinos
5c4b37df89
vb
2026-06-19 11:12:45 +03:00
Yiorgis Gozadinos
99011755fb
vb
2026-06-16 16:48:36 +03:00
Yiorgis Gozadinos
df8af54298
rebase from main
2026-06-12 10:17:18 +03:00
Yiorgis Gozadinos
88696cc6f1
vb
2026-06-11 09:50:33 +03:00
Yiorgis Gozadinos
bdad908a60
vb
2026-06-09 15:39:14 +03:00
Yiorgis Gozadinos
46b8fa3fef
vb
2026-06-08 12:24:04 +03:00
Yiorgis Gozadinos
295570797c
Disable Logfire scrubbing in evaluations
2026-06-08 11:51:13 +03:00
Yiorgis Gozadinos
e9bd56467c
Add T²-RAGBench leaderboard submission exporter
2026-06-08 11:51:02 +03:00
Yiorgis Gozadinos
a3a73f1331
Add --filter-ids to run QA on a case-id subset
2026-06-08 09:36:01 +03:00
Yiorgis Gozadinos
1d7d540270
Match the numeric scale convention in Number-Match
2026-06-06 14:52:05 +03:00
Yiorgis Gozadinos
d5b0aafa2b
Add T²-RAGBench TAT-DQA subset; generalize subset layout
2026-06-06 14:52:05 +03:00
Yiorgis Gozadinos
de9731d5f3
Score Number-Match on the declared ANSWER line, magnitude and ×100 scale
2026-06-06 14:52:05 +03:00
Yiorgis Gozadinos
390deb4203
Normalize unicode signs
2026-06-06 14:52:04 +03:00
Yiorgis Gozadinos
80594cd38c
Handle percent/decimal convention in Number-Match
2026-06-06 14:52:04 +03:00
Yiorgis Gozadinos
c2a48b55c0
Add deterministic Number-Match QA scoring for T²-RAGBench
2026-06-06 14:52:04 +03:00
Yiorgis Gozadinos
9f1bd9940a
Add T²-RAGBench FinQA evaluation dataset
2026-06-06 14:52:04 +03:00
Yiorgis Gozadinos
916b84bff4
vb
2026-06-05 12:09:57 +03:00
Yiorgis Gozadinos
e70f271ac7
vb
2026-06-04 14:13:55 +03:00
Yiorgis Gozadinos
4eae86225e
vb
2026-06-03 14:26:58 +03:00
Yiorgis Gozadinos
eb74525084
Lift analysis-skill citation rate via SKILL.md tightening
2026-06-01 18:58:52 +03:00
Yiorgis Gozadinos
8dbcd4bd7a
vb, changelog
2026-06-01 18:54:22 +03:00
Yiorgis Gozadinos
e653c49fce
Remove unecessary Mean Reciprocal Rank metric
2026-06-01 10:40:52 +03:00
Yiorgis Gozadinos
00f2a40a60
Refresh benchmarks doc and remove unused eval datasets
2026-06-01 10:40:51 +03:00
Yiorgis Gozadinos
62be23d130
vb
2026-05-29 11:47:12 +03:00
Yiorgis Gozadinos
3828a91c37
vb
2026-05-27 16:23:43 +03:00
Yiorgis Gozadinos
14d9951016
vb
2026-05-21 14:32:13 +03:00
Yiorgis Gozadinos
392c74039b
bump pydantic-ai-slim to 1.100 and haiku.skills to 0.17
...
Migrate off two APIs slated for removal in pydantic-ai 2.0:
- Agent(tool_retries=, output_retries=) -> Agent(retries={"tools": ...,
"output": ...}) in the LLM-as-judge evaluator.
- Evaluator.evaluation_name class attribute -> overriding
get_default_evaluation_name() on the citation MRR / MAP evaluators.
haiku.skills 0.17.0 already migrated its internal AGUIAdapter,
MCPToolset and ProcessEventStream usage; no further changes needed on
our side beyond the pin bumps.
2026-05-21 13:20:54 +03:00
Yiorgis Gozadinos
98b0c068e8
vb
2026-05-20 14:34:33 +03:00
Yiorgis Gozadinos
f9392cee46
evaluations: judge model moves to config.evaluations.judge
2026-05-20 12:56:22 +03:00
Yiorgis Gozadinos
8c57a3ca99
Drop the multi-agent research workflow
2026-05-20 12:46:48 +03:00
