Yiorgis Gozadinos
83ea6347b6
Fix CI failures: update tests for pydantic-ai 1.70.0 and FastMCP API changes
2026-03-24 14:02:03 +02:00
Yiorgis Gozadinos
fd327996b8
Configurable judge and reflect models for evaluations
2026-03-18 17:14:11 +02:00
Yiorgis Gozadinos
9243f56e1c
vb
2026-03-16 14:30:08 +02:00
Yiorgis Gozadinos
f961f00f2c
vb
2026-03-13 13:24:31 +02:00
Yiorgis Gozadinos
a0025e9203
Fix ty
2026-03-13 12:51:35 +02:00
Yiorgis Gozadinos
7696b3f98f
vb
2026-03-12 14:46:13 +02:00
Yiorgis Gozadinos
6ac5f277d8
Rename iterations to num_candidates
2026-03-12 13:00:51 +02:00
Yiorgis Gozadinos
5b3ad9bae7
Refactor QAPromptAdapter to accept QA agent directly, rewrite mock-heavy tests
2026-03-12 12:35:39 +02:00
Yiorgis Gozadinos
24275d2ef9
Fixes
2026-03-12 12:19:21 +02:00
Yiorgis Gozadinos
c785a54f5f
Respect config.prompts.qa in evaluations benchmark and optimization
2026-03-12 12:05:51 +02:00
Yiorgis Gozadinos
6812435ba0
cleanup
2026-03-12 12:05:50 +02:00
Yiorgis Gozadinos
7c399d1ff1
Remove evaluation ceremony from gepa judge
2026-03-12 12:05:50 +02:00
Yiorgis Gozadinos
8bac8b4104
Properly choose training/eval set
2026-03-12 12:05:50 +02:00
Yiorgis Gozadinos
66f88f7872
Test evaluations
2026-03-12 12:05:50 +02:00
Yiorgis Gozadinos
45f7ac8d1b
run_optimization should be sync
2026-03-12 12:05:25 +02:00
Yiorgis Gozadinos
adc83e5f81
Tests for gepa
2026-03-12 12:05:25 +02:00
Yiorgis Gozadinos
60b1d3c013
Add GEPA prompt optimization for QA evaluations
2026-03-12 12:05:25 +02:00
Yiorgis Gozadinos
8d6617d63f
vb
2026-03-11 17:08:22 +02:00
Yiorgis Gozadinos
e2749ad2a6
Cap QA agent search iterations to reduce response time
2026-03-11 12:13:43 +02:00
Yiorgis Gozadinos
f366aea905
vb
2026-03-06 11:12:54 +02:00
Yiorgis Gozadinos
40ff54eac0
Set params for evaluations judge
2026-03-05 13:59:20 +02:00
Yiorgis Gozadinos
30ebd6694b
Update dependencies
2026-03-05 13:01:53 +02:00
Yiorgis Gozadinos
878423ea2e
vb
2026-03-04 15:02:34 +02:00
Yiorgis Gozadinos
8a859fbe3c
vb
2026-03-03 17:25:53 +02:00
Yiorgis Gozadinos
99ca3605e7
vb
2026-02-28 14:11:25 +02:00
Yiorgis Gozadinos
50116bb49a
vb
2026-02-26 11:21:46 +02:00
Yiorgis Gozadinos
2acb0292e4
vb
2026-02-24 13:30:01 +02:00
Yiorgis Gozadinos
2cafb5c8cc
vb
2026-02-20 19:41:17 +02:00
Yiorgis Gozadinos
e186e13cd2
vb
2026-02-20 19:01:16 +02:00
Yiorgis Gozadinos
4855051936
Add research() to client and add haiku.skills dependency, remove --deep flag and simplify app
2026-02-20 16:36:47 +02:00
Yiorgis Gozadinos
0c270ba42f
vb
2026-02-19 14:28:13 +02:00
Yiorgis Gozadinos
c62fe5fce8
vb
2026-02-17 13:56:43 +02:00
Yiorgis Gozadinos
2d5fbd06f2
Update dependencies
2026-02-17 13:12:08 +02:00
Yiorgis Gozadinos
53f6cd3579
vb
2026-02-16 13:12:06 +02:00
Yiorgis Gozadinos
aaa9b6a32f
vb
2026-02-10 13:29:30 +02:00
Yiorgis Gozadinos
17c2136b4e
vb
2026-02-06 12:48:16 +01:00
Yiorgis Gozadinos
1e22937482
vb
2026-01-31 11:12:38 +02:00
Yiorgis Gozadinos
42f2d0fd0e
docs
2026-01-31 10:13:22 +02:00
Yiorgis Gozadinos
4264cdcc24
vb
2026-01-29 15:35:42 +02:00
Yiorgis Gozadinos
2a99cb09bf
Evaluate using ask --deep
2026-01-29 14:22:18 +02:00
Yiorgis Gozadinos
39f4937916
Handle missing datasets from huggingface
2026-01-29 13:31:31 +02:00
Yiorgis Gozadinos
bb634e68a4
vb
2026-01-27 11:17:17 +02:00
Yiorgis Gozadinos
7c9d70fa62
vb
2026-01-26 16:46:33 +02:00
Yiorgis Gozadinos
369b3a4bf7
No zip
2026-01-26 16:28:19 +02:00
Yiorgis Gozadinos
9b166969e2
Introduce huggingface dataset for sharing evaluation dbs
2026-01-26 15:53:37 +02:00
Yiorgis Gozadinos
c7a9ad8583
Customize orb QA prompt to not use LaTeX as gpt-oss Ollama implementation fails to parse it properly
2026-01-24 13:48:09 +02:00
Yiorgis Gozadinos
47d355ed0a
Update dependencies
2026-01-23 11:57:02 +02:00
Yiorgis Gozadinos
35ebdd67e2
vb
2026-01-22 17:48:39 +02:00
Yiorgis Gozadinos
0d3ac5142d
Fix answer in orb
2026-01-22 15:08:33 +02:00
Yiorgis Gozadinos
de022639b6
Correctly lookup docs for the orb dataset
2026-01-22 14:17:09 +02:00
Yiorgis Gozadinos
e6b239bd6c
Add evaluation dataset for multi-modal q&a
2026-01-22 14:17:09 +02:00
Yiorgis Gozadinos
32c64d3d11
vb
2026-01-22 12:59:32 +02:00
Yiorgis Gozadinos
ea7a4a9a8f
apply dotenv usecwd fix to remaining files
...
Apply the same find_dotenv(usecwd=True) fix from cli.py to:
- evaluations/evaluations/benchmark.py
- app/backend/main.py
Co-Authored-By: Tianyi Cui <contact@tianyicui.com>
2026-01-22 12:41:35 +02:00
Yiorgis Gozadinos
b96e5a3be1
Additional prompts needing adapting. Missing cassettes.
2026-01-21 15:17:56 +02:00
Yiorgis Gozadinos
e7ce9694b5
vb
2026-01-20 15:47:37 +02:00
Yiorgis Gozadinos
1022d5dafd
Update dependencies
2026-01-19 16:23:26 +02:00
Yiorgis Gozadinos
5e55c0df54
replace pyright with ty type checker
2026-01-19 15:50:14 +02:00
Yiorgis Gozadinos
2c264f4afc
vb
2026-01-19 14:54:18 +02:00
Yiorgis Gozadinos
5f2f180960
vb
2026-01-16 15:27:55 +02:00
Yiorgis Gozadinos
12e5e9ebfb
vb
2026-01-15 16:24:03 +02:00
Yiorgis Gozadinos
157773ef15
vb
2026-01-15 11:47:49 +02:00
Yiorgis Gozadinos
2930679dc1
vb
2026-01-13 19:47:27 +02:00
Yiorgis Gozadinos
275b09c8c5
vb
2026-01-13 18:34:45 +02:00
Yiorgis Gozadinos
eb5d0b0622
Use tui as an optional group. vb
2026-01-13 12:30:19 +02:00
Yiorgis Gozadinos
7cc561d1db
Refactor to make agents a top-level module. Bring in the conversational agent from the app
2026-01-12 12:37:08 +02:00
Yiorgis Gozadinos
747acd6f6a
vb
2026-01-12 12:08:28 +02:00
Yiorgis Gozadinos
ff1f19bb08
vb
2026-01-08 18:50:39 +02:00
Yiorgis Gozadinos
92f0fd956f
vb
2026-01-08 10:25:25 +02:00
Yiorgis Gozadinos
ef187767ed
vb
2026-01-07 11:20:48 +02:00
Yiorgis Gozadinos
af340ac410
vb
2026-01-05 12:15:41 +02:00
Yiorgis Gozadinos
271af6cb3c
Update core dependencies
2026-01-05 11:52:38 +02:00
Yiorgis Gozadinos
c9cb005d07
Use FTS on contextualized text
2025-12-29 14:50:07 +02:00
Yiorgis Gozadinos
1f654917b8
vb
2025-12-26 13:56:29 +02:00
Yiorgis Gozadinos
5c7f033d2c
vb
2025-12-19 12:35:35 +02:00
Yiorgis Gozadinos
e8e67d336c
Fix get_model() uses where default Config is forced
2025-12-18 15:10:31 +02:00
Yiorgis Gozadinos
1dbbaf6dd8
vb
2025-12-18 12:21:16 +02:00
Yiorgis Gozadinos
40e0ea090d
Add option to override vacuum interval
2025-12-12 16:04:06 +02:00
Yiorgis Gozadinos
056b9ad090
Use periodic vacuum to prevent disk exhaustion with large datasets
2025-12-12 15:53:23 +02:00
Yiorgis Gozadinos
7526735059
hotpotqa adapter
2025-12-12 10:56:37 +02:00
Yiorgis Gozadinos
f6073a0db9
vb
2025-12-12 10:48:21 +02:00
Yiorgis Gozadinos
00d621e211
vb
2025-12-11 16:31:21 +02:00
Yiorgis Gozadinos
802f058205
Move search related config under SearchConfig
2025-12-09 12:28:28 +02:00
Yiorgis Gozadinos
06b4f6a224
Add format parameter for text-to-DoclingDocument conversion
2025-12-08 15:56:21 +02:00
Yiorgis Gozadinos
9a5f8e4368
Add type-aware context expansion for search results
2025-12-08 15:56:21 +02:00
Yiorgis Gozadinos
dab04668bd
Dependency updates
2025-12-08 15:56:02 +02:00
Yiorgis Gozadinos
a57203065a
Fix test
2025-12-08 15:55:28 +02:00
Yiorgis Gozadinos
d808c6c425
Simplify citations in qa & graph agents
2025-12-08 15:55:27 +02:00
Yiorgis Gozadinos
ace58a7034
Update evaluations and a2a
2025-12-08 15:54:56 +02:00
Yiorgis Gozadinos
fe4ac530be
DB migration
2025-12-08 15:54:56 +02:00
Yiorgis Gozadinos
3525fae625
Use EmbeddingModelConfig similar to ModelConfig for embeddings
2025-12-02 11:55:31 +02:00
Yiorgis Gozadinos
f07590bd5b
vb
2025-12-01 17:58:47 +02:00
Yiorgis Gozadinos
10aa2a7cf2
vb
2025-11-28 10:15:59 +02:00
Yiorgis Gozadinos
eee29578ba
vb
2025-11-27 15:24:39 +02:00
Yiorgis Gozadinos
3959ce39f7
vb
2025-11-27 13:06:04 +02:00
Yiorgis Gozadinos
ca2445d869
vb
2025-11-26 10:33:46 +02:00
Yiorgis Gozadinos
c22a5bb0ce
vb
2025-11-25 13:44:48 +02:00
Yiorgis Gozadinos
8a8005ead0
Add qa/judge models meta to experiment meta
2025-11-25 13:09:29 +02:00
Yiorgis Gozadinos
f843fb8140
Use non-thinking judge in evals
2025-11-25 13:00:25 +02:00
Yiorgis Gozadinos
e83978f88b
Revert embeddings to use flat config
2025-11-25 12:51:58 +02:00
Yiorgis Gozadinos
a9701616c8
Rename model to name under model
2025-11-25 12:28:43 +02:00
