Move evaluations to src/
This commit is contained in:
parent
6bcc251e5d
commit
43b9cd50ba
10 changed files with 3 additions and 3 deletions
2
.gitignore
vendored
2
.gitignore
vendored
|
|
@ -11,7 +11,7 @@ wheels/
|
|||
|
||||
# tests
|
||||
.coverage*
|
||||
evaluations/data/
|
||||
src/evaluations/data/
|
||||
tests/data/
|
||||
.pytest_cache/
|
||||
.ruff_cache/
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@
|
|||
We use the [repliqa](https://huggingface.co/datasets/ServiceNow/repliqa) dataset for the evaluation of `haiku.rag`.
|
||||
|
||||
You can perform your own evaluations with the Typer CLI in
|
||||
`evaluations/benchmark.py`, for example `python -m evaluations.benchmark repliqa`.
|
||||
`src/evaluations/benchmark.py`, for example `cd src && python -m evaluations.benchmark repliqa`.
|
||||
The evaluation flow is orchestrated with
|
||||
[`pydantic-evals`](https://github.com/pydantic/pydantic-ai/tree/main/libs/pydantic-evals),
|
||||
which we leverage for dataset management, scoring, and report generation.
|
||||
|
|
|
|||
|
|
@ -49,7 +49,7 @@ requires = ["hatchling"]
|
|||
build-backend = "hatchling.build"
|
||||
|
||||
[tool.hatch.build]
|
||||
exclude = ["/docs", "/evaluations", "/tests", "/.github"]
|
||||
exclude = ["/docs", "/tests", "/.github"]
|
||||
|
||||
[tool.hatch.build.targets.wheel]
|
||||
packages = ["src/haiku"]
|
||||
|
|
|
|||
Loading…
Reference in a new issue