27ed0b3bb3
The notice told the model to answer from what it had the moment qa.max_searches ran out, while up to 15 code executions remained and in-code search() does not count against that budget. It now names the spent tool and points at whichever evidence tool still has budget, falling back to answer-and-cite only when none do. Also count RetryPromptPart in n_failed_tools: _cite rejects with ModelRetry, so a run whose every cite attempt was refused reported zero failures. And note that n_requests is the run's request count, which tracks a capability's own budget only while it stays loaded. |
||
|---|---|---|
| .. | ||
| datasets | ||
| evaluators | ||
| __init__.py | ||
| benchmark.py | ||
| capability_runner.py | ||
| config.py | ||
| numbers.py | ||
| submission.py | ||