8103460e9c
Adds the `docling-agent` reasoning-trace viewer as a Studio tunnel, per `docs/design/reasoning-trace.md`. Users pick an analyzed document, import a RAGResult JSON, and the iterations are overlaid on the document graph. Graph source is decoupled from Neo4j: a new pure builder (`infra/docling_graph.build_graph_payload`) reads `document_json` from SQLite and emits the same Cytoscape-shaped payload that `fetch_graph` returns from Neo4j. Neo4j stays exclusive to the Maintain ingestion pipeline. Shared DoclingDocument helpers live in `infra/docling_tree.py` so TreeWriter and the builder can't drift on label taxonomy or tree walks. Also removes the Cytoscape minimap (cytoscape-navigator) from GraphView: second render instance hurt perf on large documents for no UX win. Backend - new `GET /api/documents/:id/reasoning-graph` (SQLite-only) - new `infra/docling_tree.py`, `infra/docling_graph.py` - `analysis_repo.find_latest_completed_by_document` - tests: `test_docling_graph.py` (builder), `test_graph_api.py` (endpoint) Frontend - `features/reasoning/` — store, overlay, types, panel, import dialog, workspace, doc picker - new `ReasoningPage` + `/reasoning` and `/reasoning/:docId` routes - `GraphView` gains a `fetcher` prop so reasoning can inject the SQLite-backed fetcher while Maintain keeps using the Neo4j one - drops minimap (nav container, dep, CSS) - legend filters + section parenting extracted for reuse - i18n base strings (FR + EN)
139 lines
4.1 KiB
Markdown
139 lines
4.1 KiB
Markdown
# Reasoning Trace — R&D sandbox
|
||
|
||
Goal: run `docling-agent`'s RAG loop against a document already ingested in
|
||
Docling-Studio, capture the `RAGResult` (per-iteration reasoning trace), and
|
||
inspect what the agent does.
|
||
|
||
Fully **isolated** from the Studio backend: no deps added to
|
||
`document-parser/`, no services modified. Just a script + uv inline deps.
|
||
|
||
---
|
||
|
||
## What it does
|
||
|
||
1. Reads the pre-parsed `DoclingDocument` directly from Studio's SQLite
|
||
(`analysis_jobs.document_json`) — no PDF re-conversion.
|
||
2. Instantiates `DoclingRAGAgent` against a local Ollama model.
|
||
3. Calls `agent._rag_loop()` directly (the public `.run()` method discards the
|
||
`RAGResult`; we need the iterations to see the reasoning trace).
|
||
4. Dumps the full `RAGResult` as JSON to `output/`.
|
||
|
||
---
|
||
|
||
## Prerequisites
|
||
|
||
### 1. Ollama running
|
||
```sh
|
||
# If not already running as a service:
|
||
ollama serve # in another terminal
|
||
```
|
||
|
||
### 2. A model pulled
|
||
Recommended (Peter Staar's default, ~3B params, good JSON adherence):
|
||
```sh
|
||
ollama pull granite4:micro-h
|
||
```
|
||
|
||
Alternative already on your machine (2 GB, may struggle with strict JSON
|
||
rejection sampling):
|
||
```
|
||
llama3.2:3b
|
||
```
|
||
|
||
Bigger/more reliable but slower (20B):
|
||
```sh
|
||
ollama pull gpt-oss:20b
|
||
```
|
||
|
||
### 3. Pick an analysis job id
|
||
Any `COMPLETED` row from `analysis_jobs` with a non-null `document_json`:
|
||
|
||
```sh
|
||
sqlite3 document-parser/data/docling_studio.db \
|
||
"SELECT aj.id, d.filename, length(aj.document_json)
|
||
FROM analysis_jobs aj JOIN documents d ON d.id=aj.document_id
|
||
WHERE aj.document_json IS NOT NULL AND aj.status='COMPLETED'
|
||
ORDER BY length(aj.document_json) DESC LIMIT 5;"
|
||
```
|
||
|
||
On this machine, the biggest one right now is:
|
||
```
|
||
722d5631-0089-44a3-a64a-7ce5b99579d3 — CCI - Conférence IA - Offre Commerciale v1.0
|
||
```
|
||
|
||
---
|
||
|
||
## Run
|
||
|
||
```sh
|
||
uv run experiments/reasoning-trace/inspect_doc.py \
|
||
--job-id 722d5631-0089-44a3-a64a-7ce5b99579d3 \
|
||
--query "Quels sont les livrables principaux proposés ?" \
|
||
--model granite4:micro-h
|
||
```
|
||
|
||
Flags:
|
||
- `--job-id` — required, analysis_jobs.id
|
||
- `--query` — required, the question
|
||
- `--model` — either a mellea catalog constant (`IBM_GRANITE_4_HYBRID_MICRO`)
|
||
or a raw Ollama tag (`granite4:micro-h`, `llama3.2:3b`). Default:
|
||
`granite4:micro-h`.
|
||
- `--max-iters` — default 5 (agent's own default)
|
||
- `--quiet` — disable the rich panels during the loop
|
||
|
||
First run will take ~1–2 min: `uv` solves the `docling-agent` env (pulls
|
||
docling-core, mellea, pydantic, rich, …) into a cached virtualenv. Subsequent
|
||
runs are instant.
|
||
|
||
---
|
||
|
||
## Output
|
||
|
||
`experiments/reasoning-trace/output/<job-id-prefix>_<utc>.json`
|
||
|
||
Schema:
|
||
```json
|
||
{
|
||
"job_id": "…",
|
||
"filename": "…",
|
||
"query": "…",
|
||
"model": { "ollama_name": "…", "hf_model_name": "…" },
|
||
"max_iterations": 5,
|
||
"result": {
|
||
"answer": "…",
|
||
"converged": true,
|
||
"iterations": [
|
||
{ "iteration": 1, "section_ref": "#/texts/3",
|
||
"reason": "…", "section_text_length": 412,
|
||
"can_answer": false, "response": "…" },
|
||
…
|
||
]
|
||
}
|
||
}
|
||
```
|
||
|
||
This is the artifact the v1 Studio endpoint (`POST /api/rag/inspect`) will
|
||
import — so anything that works here should work there.
|
||
|
||
---
|
||
|
||
## Things to check on first run
|
||
|
||
- **Do we actually get a trace?** `iterations` list should have ≥ 1 entries
|
||
(empty means "no section headers found" fallback — bad sign for the viz idea).
|
||
- **Are `section_ref` values `#/texts/N` paths or `#/groups/N`?** Determines
|
||
how the resolver walks the tree.
|
||
- **Reasoning quality**: does `reason` actually explain the pick, or is it
|
||
LLM filler? That affects whether the trace is worth surfacing visually.
|
||
- **Convergence rate**: with `max_iters=5`, does a small model converge at all,
|
||
or hit the cap and return a partial answer?
|
||
- **Latency**: per-iteration wall-clock on your M-series machine with granite4.
|
||
|
||
---
|
||
|
||
## Next step (if the above looks promising)
|
||
|
||
Resolve each `iteration.section_ref` → `(page_no, bbox)` using the same
|
||
`DoclingDocument` that was loaded here. That's the `reasoning_service.py`
|
||
resolver described in `docs/design/reasoning-trace.md` §3.2 — implement it in
|
||
a second script here (`resolve_trace.py`) before touching Studio.
|