The skill assumed the Logfire MCP is loaded. It is not loaded in every
session, which left no way to inspect a run at all. Document the HTTP query
API as the fallback, including the mandatory min_timestamp, the API keys
replacing read tokens, and the project-scoping trap: a key for the wrong
project authenticates and returns zero rows rather than erroring.
Also record three things that produced wrong readings in practice:
assertions/scores/metrics are keys inside the attributes column rather than
columns, one exception is emitted once per span level so failures must be
counted at case level, and assertion_pass_rate drops unjudged cases from its
denominator so judged and floor rates have to be quoted together.
Add a section for monitoring a run that is still in flight, since an eval
prints nothing until it finishes: case-span progress, the serial-execution
check that makes an ETA valid, and the per-case diagnostic attributes.
Vendor the query helper next to the skill so it does not point at a path
outside the repo, and derive its region from the key prefix.
Retrieval and QA only read from the database, but the benchmark opened it
writable, where an embedder identity differing from the stored one aborts
instead of warning. Running a pre-built database against a different
serving stack then needed a `rebuild --set-embedder` first.
Correct the debug-evals skill alongside it: the pydantic-ai span names are
`execute_tool {tool_name}` and `invoke_agent agent`, targets are
`{rag,analysis}-capability`, and no `skill_model` metadata key exists.
