danvics/haiku.rag
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Bound a compute-only call at code_timeout

Browse source 
The pool watchdog counts only the time a worker spends running code, so a read
that blocks the worker never trips it. The margin above code_timeout therefore
guarded a race that cannot happen, and only bought a runaway call 90s where 60s
was configured. Pass code_timeout straight through. The two limits are disjoint:
the watchdog bounds a call that computes, and the read deadline bounds a call
that reads.

Drop the ordering test, which was true for any positive margin. The containment
test no longer sets code_timeout to zero, because that value now also disables
the watchdog and kills the worker before the read guard can refuse. It patches
the guard instead.

Monty 0.0.19 runs a plain class and a class with __enter__ and __exit__. Only
inheritance and metaclasses raise. Say that in the instructions.

MemoryFile is no longer constructed, so drop the import, the annotation, and the
stale references in the sandbox docstring and CLAUDE.md. That CLAUDE.md line
also still claimed a ThreadPoolExecutor, _run_async, and a fresh interpreter per
call.
This commit is contained in:
Yiorgis Gozadinos 2026-07-28 18:11:33 +03:00
parent d9e4d1e57c
commit 221c90af72
No known key found for this signature in database
4 changed files with 20 additions and 37 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
docs/configuration/qa.md
View file

@ -56,7 +56,7 @@ analysis:
```
- **model**: LLM configuration (see [Providers](providers.md#model-settings)). When unset, falls back to `qa.model`.
- **code_timeout**: Seconds a single `execute_code` call may spend reading documents (default: 60). The sandbox refuses further reads past this point. Code that computes without reading is bounded instead by the session budget of `code_timeout * max_executions`.
- **code_timeout**: Seconds a single `execute_code` call may spend reading documents (default: 60). The sandbox refuses further reads past this point. Code that computes without reading is killed by the worker watchdog at the same limit. `code_timeout * max_executions` is the cumulative ceiling across all calls in one question.
- **max_output_chars**: Truncate code output after this many characters (default: 50000)
- **max_executions**: Maximum `execute_code` calls per question before the capability is told to answer from what it has (default: 15)

2
haiku_rag_slim/haiku/rag/capabilities/instructions/analysis.md
View file

@ -17,7 +17,7 @@ Inside the code, these functions are available (use `await`):
- `await list_documents()` → list of dicts with keys: id, title, uri, created_at
Available modules: `json`, `re`, `math`, `pathlib`
Not supported: class definitions, generators/yield, match statements, decorators, `collections`, iterating a file object (`for line in f`)
Not supported: class inheritance and metaclasses, generators/yield, match statements, decorators, `collections`, iterating a file object (`for line in f`)
### analysis_search
Search the knowledge base directly (outside code execution). Each result has a `Type:` (paragraph, table, code, list_item, picture). When the Type is `picture`, the corresponding figure may also be attached to the tool response as an image alongside the text — use it directly to answer questions about figures, diagrams, charts, screenshots.

26
haiku_rag_slim/haiku/rag/sandbox/sandbox.py
View file

@ -12,7 +12,6 @@ from pydantic_monty import (
AsyncMonty,
AsyncMontySession,
CallbackFile,
MemoryFile,
OSAccess,
ResourceLimits,
)
@ -28,10 +27,6 @@ if TYPE_CHECKING:
from haiku.rag.client import HaikuRAG
_REQUEST_TIMEOUT_MARGIN_S = 30.0
"""Grace above ``code_timeout`` before the pool watchdog kills the worker."""
@dataclass
class SandboxResult:
"""Result of executing code in the sandbox."""
@ -304,12 +299,12 @@ class Sandbox:
"""Build the virtual filesystem with document data.
Mounts per-document directories with:
- metadata.json: MemoryFile (eager, small)
- metadata.json: CallbackFile (eager, small)
- content.txt: CallbackFile (lazy, can be large)
- items.jsonl: CallbackFile (lazy, bulk-cached)
- toc.json: CallbackFile (lazy, bulk-cached)
"""
files: list[MemoryFile | CallbackFile] = []
files: list[CallbackFile] = []
def _deny_write(_path: "PurePosixPath", _content: str | bytes) -> None:
raise PermissionError(f"Document files are read-only: {_path}")
@ -471,17 +466,6 @@ class Sandbox:
return OSAccess(files)
def _request_timeout(self) -> float:
"""Hard per-call deadline for the pool watchdog.
The read deadline refuses the next read at ``code_timeout`` and keeps the
session, so it wins for code that reads. This watchdog is the fallback
for code that computes without reading: it kills the worker, which loses
the variables the session held. Leave room for one read that is already
in flight, so the graceful refusal wins the race.
"""
return self._config.analysis.code_timeout + _REQUEST_TIMEOUT_MARGIN_S
def _session_limits(self) -> ResourceLimits:
"""Resource limits for the worker session.
@ -500,7 +484,11 @@ class Sandbox:
if self._vfs is None:
self._vfs = await self._build_vfs()
if self._pool is None:
pool = AsyncMonty(request_timeout=self._request_timeout())
# The watchdog counts only time the worker spends running code, so a
# read that blocks the worker never trips it. That leaves the two
# limits disjoint: this one bounds a call that computes, and the read
# deadline bounds a call that reads.
pool = AsyncMonty(request_timeout=self._config.analysis.code_timeout)
await pool.__aenter__()
self._pool = pool
if self._session is None:

27
tests/sandbox/test_sandbox.py
View file

@ -826,16 +826,12 @@ class TestSandboxReadDeadline:
assert sb._session_limits() == {"max_duration_secs": 15.0}
@pytest.mark.asyncio
async def test_document_read_past_the_deadline_fails_the_execution(
self, temp_db_path
):
"""The overrun surfaces as a failed result, not a raised exception."""
async def test_refused_read_fails_the_execution(self, temp_db_path, monkeypatch):
"""The refusal surfaces as a failed result, not a raised exception."""
from docling_core.types.doc.document import DoclingDocument
from docling_core.types.doc.labels import DocItemLabel
config = AppConfig()
config.analysis.code_timeout = 0.0
docling = DoclingDocument(name="d")
docling.add_text(label=DocItemLabel.TEXT, text="Foxes and dogs.")
@ -852,6 +848,13 @@ class TestSandboxReadDeadline:
uri="test://deadline",
)
def _past_deadline(*_args, **_kwargs):
raise TimeoutError(
"time limit exceeded: no further document reads after 60.0s"
)
monkeypatch.setattr(Sandbox, "_run_on_loop", _past_deadline)
sb = Sandbox(db_path=temp_db_path, config=config, context=AnalysisContext())
try:
result = await sb.execute(
@ -906,25 +909,17 @@ class TestSandboxWorkerCrash:
class TestSandboxRequestTimeout:
"""The pool watchdog bounds a call that never reads."""
def test_request_timeout_sits_above_the_read_deadline(self, temp_db_path):
"""A read refusal keeps the session, so it must win the race."""
config = AppConfig()
config.analysis.code_timeout = 5.0
sb = Sandbox(db_path=temp_db_path, config=config, context=AnalysisContext())
assert sb._request_timeout() > config.analysis.code_timeout
@pytest.mark.asyncio
async def test_runaway_compute_is_killed_and_the_next_call_recovers(
self, temp_db_path, monkeypatch
self, temp_db_path
):
"""Code that never reads escapes the read deadline. The watchdog kills it."""
config = AppConfig()
config.analysis.code_timeout = 1.0
async with HaikuRAG(temp_db_path, create=True):
pass
sb = Sandbox(db_path=temp_db_path, config=config, context=AnalysisContext())
monkeypatch.setattr(sb, "_request_timeout", lambda: 1.0)
try:
runaway = await sb.execute(
"x = 0\nfor i in range(500000000):\n x += i\nprint(x)"