haiku.rag/.claude/skills
Yiorgis Gozadinos 86f254d5f2
Document the Logfire HTTP query path in the eval-debugging skill
The skill assumed the Logfire MCP is loaded. It is not loaded in every
session, which left no way to inspect a run at all. Document the HTTP query
API as the fallback, including the mandatory min_timestamp, the API keys
replacing read tokens, and the project-scoping trap: a key for the wrong
project authenticates and returns zero rows rather than erroring.

Also record three things that produced wrong readings in practice:
assertions/scores/metrics are keys inside the attributes column rather than
columns, one exception is emitted once per span level so failures must be
counted at case level, and assertion_pass_rate drops unjudged cases from its
denominator so judged and floor rates have to be quoted together.

Add a section for monitoring a run that is still in flight, since an eval
prints nothing until it finishes: case-span progress, the serial-execution
check that makes an ETA valid, and the per-case diagnostic attributes.

Vendor the query helper next to the skill so it does not point at a path
outside the repo, and derive its region from the key prefix.
2026-08-24 00:08:59 +03:00
..
debug-evals Document the Logfire HTTP query path in the eval-debugging skill 2026-08-24 00:08:59 +03:00
debug-ingestion Update debug skills for the remote Logfire MCP tool names 2026-07-15 19:24:56 +03:00