8103460e9c
Adds the `docling-agent` reasoning-trace viewer as a Studio tunnel, per `docs/design/reasoning-trace.md`. Users pick an analyzed document, import a RAGResult JSON, and the iterations are overlaid on the document graph. Graph source is decoupled from Neo4j: a new pure builder (`infra/docling_graph.build_graph_payload`) reads `document_json` from SQLite and emits the same Cytoscape-shaped payload that `fetch_graph` returns from Neo4j. Neo4j stays exclusive to the Maintain ingestion pipeline. Shared DoclingDocument helpers live in `infra/docling_tree.py` so TreeWriter and the builder can't drift on label taxonomy or tree walks. Also removes the Cytoscape minimap (cytoscape-navigator) from GraphView: second render instance hurt perf on large documents for no UX win. Backend - new `GET /api/documents/:id/reasoning-graph` (SQLite-only) - new `infra/docling_tree.py`, `infra/docling_graph.py` - `analysis_repo.find_latest_completed_by_document` - tests: `test_docling_graph.py` (builder), `test_graph_api.py` (endpoint) Frontend - `features/reasoning/` — store, overlay, types, panel, import dialog, workspace, doc picker - new `ReasoningPage` + `/reasoning` and `/reasoning/:docId` routes - `GraphView` gains a `fetcher` prop so reasoning can inject the SQLite-backed fetcher while Maintain keeps using the Neo4j one - drops minimap (nav container, dep, CSS) - legend filters + section parenting extracted for reuse - i18n base strings (FR + EN)
253 lines
12 KiB
Markdown
253 lines
12 KiB
Markdown
# Reasoning Trace Viewer — Docling-Studio v0.6.0 (R&D preview)
|
||
|
||
Design doc for the `docling-agent` reasoning trace viewer.
|
||
Targeted release: **v0.6.0** (R&D branched from `release/0.5.0` in parallel to the
|
||
0.5 build, so the Neo4j foundation can be leveraged without blocking the hackathon deliverable).
|
||
|
||
Positioning one-liner:
|
||
> Studio becomes the **reference viewer for any `docling-agent` run** — not another
|
||
> chatbot. The PDF is the debug surface.
|
||
|
||
---
|
||
|
||
## 1. Context
|
||
|
||
### Upstream trigger
|
||
Peter Staar (IBM) suggested surfacing the LLM reasoning trace as `docling-agent`
|
||
walks a `DoclingDocument` outline (chunkless RAG, new IBM repo).
|
||
|
||
### `docling-agent` in one paragraph
|
||
`DoclingRAGAgent(model_id, tools, max_iterations=5).run(task, sources=[doc])` returns
|
||
a `RAGResult` with `answer`, `converged`, and `iterations: list[RAGIteration]`.
|
||
Each `RAGIteration` carries: `iteration`, `section_ref` (JSON-pointer, e.g. `#/texts/3`),
|
||
`reason`, `can_answer`, `response`, `section_text_length`. No bbox — must be resolved
|
||
through `DoclingDocument.<items>[i].prov[0].bbox` + `page_no`. Runs on Mellea
|
||
(Ollama / OpenAI / HF / WatsonX / LiteLLM / Bedrock). Observability is stdout logs only.
|
||
|
||
### What Studio already brings
|
||
- **Neo4j graph of the document** (0.5.0, just landed) — every `Element` is keyed by
|
||
`(doc_id, self_ref)`, Cytoscape node id is `elem::${self_ref}`. **This is the
|
||
killer enabler.** `RAGIteration.section_ref → node` is a string concat, no resolver.
|
||
- `GraphView.vue` (Cytoscape + dagre) already handles styles via selectors
|
||
(`selector: 'edge[type = "NEXT"]'`, `selector: 'node[kind = "section"]'`) — adding
|
||
a `visited` class + `REASONING_NEXT` synthetic edge type is ~20 LOC of style.
|
||
- `analysis_jobs.document_json` in SQLite → DoclingDocument available for the sidecar
|
||
runner (no PDF re-conversion). Not used by the viewer itself.
|
||
|
||
### Personas
|
||
- **v1 (this plan)**: dev / integrator of `docling-agent` debugging a run that went wrong.
|
||
- **v2 (roadmap)**: live runner with synchronized demo UX.
|
||
- v3+ (non-goals here): business analyst for semantic navigation, batch QA.
|
||
|
||
---
|
||
|
||
## 2. Scope split — **debug first, demo second**
|
||
|
||
Rendering surface pivoted: **the trace is drawn on the Neo4j graph**, not on the PDF.
|
||
See §1. This kills the whole bbox-resolution stack from v1.
|
||
|
||
| Phase | Value | Runtime deps | Surface |
|
||
|---|---|---|---|
|
||
| **v1 — Debug (this plan)** | Import externally-produced `RAGResult` JSON, overlay trace on the existing GraphView | **None** server-side. Pure frontend. | GraphView: visited nodes highlighted in order, synthetic `REASONING_NEXT` edges |
|
||
| **v2 — Demo (follow-up)** | Run the agent live against a loaded document | Ollama + Mellea + `docling-agent` (new opt-dep group `rag`) | Same GraphView + SSE streaming of iterations, staggered reveal |
|
||
|
||
Building v1 first de-risks the **graph-trace UX** on real runs (produced by the
|
||
R&D sidecar — see `experiments/reasoning-trace/`) before wiring the live runner.
|
||
Code shared between v1 and v2 is the GraphView overlay itself — 100 % reused.
|
||
|
||
**Prerequisite for v1**: the target document must have been processed through the
|
||
"Maintain" step (Neo4j pipeline). Otherwise the graph is empty and the trace has
|
||
nowhere to render — surface an explicit "Run the Maintain step first" empty state.
|
||
|
||
---
|
||
|
||
## 3. v1 — Debug mode (frontend-only)
|
||
|
||
### 3.1 No backend changes in v1
|
||
|
||
The GraphView already loads nodes keyed by `self_ref` via `GET /api/documents/{doc_id}/graph`.
|
||
Iteration `section_ref` → Cytoscape node id is `` `elem::${section_ref}` `` — a client-side
|
||
string concat. Nothing to compute server-side.
|
||
|
||
Consequences:
|
||
- No new router, no new service, no new pydantic model, no new migration.
|
||
- No dependency on `docling-agent` in `document-parser/requirements.txt`.
|
||
- `RAGResult` JSON (as produced by `experiments/reasoning-trace/`) is consumed
|
||
entirely by the frontend.
|
||
|
||
### 3.2 Frontend — feature folder
|
||
|
||
New `frontend/src/features/reasoning/`:
|
||
|
||
```
|
||
reasoning/
|
||
├── store/reasoningStore.ts # Pinia: trace, activeIteration, importDialogOpen
|
||
├── ui/
|
||
│ ├── ReasoningPanel.vue # Side panel: query, answer, iteration list
|
||
│ ├── IterationCard.vue # Single iteration row (reason + can_answer badge)
|
||
│ ├── ImportTraceDialog.vue # Drag-drop / paste RAGResult JSON
|
||
│ └── GraphReasoningOverlay.ts # NOT a component — a plugin that decorates cy
|
||
└── types.ts # RAGIteration, RAGResult mirror types
|
||
```
|
||
|
||
### 3.3 Graph overlay — how it's drawn
|
||
|
||
`GraphReasoningOverlay` takes the existing `cy` Cytoscape instance (exposed from
|
||
`GraphView.vue` via `defineExpose`) and:
|
||
|
||
1. For each `iteration[i].section_ref`, find node `` `elem::${section_ref}` ``. If
|
||
missing, tag as `resolution_status: "not_in_graph"` and show a warning in the panel
|
||
(common cause: doc not processed through Maintain, or agent returned a ref that
|
||
points at a non-Element node).
|
||
2. Add class `visited` + data attribute `visitOrder: i+1` on matched nodes.
|
||
3. Insert **synthetic edges** between successive visited nodes with `type: "REASONING_NEXT"`
|
||
and `data: { order: i }`. These edges are UI-only, never written to Neo4j.
|
||
4. On import, fit viewport to the visited subgraph (`cy.fit(cy.$('.visited'), 80)`).
|
||
5. On iteration card click → `cy.$(`#elem::${ref}`).flashClass('pulse', 800)` +
|
||
centered pan.
|
||
|
||
Cytoscape styles (append to the existing stylesheet array in `GraphView.vue`):
|
||
|
||
```js
|
||
{ selector: 'node.visited',
|
||
style: { 'border-color': '#EA580C', 'border-width': 3, 'overlay-opacity': 0 } },
|
||
{ selector: 'node.visited[visitOrder]',
|
||
style: { label: 'data(visitOrder)', 'text-valign': 'top',
|
||
'text-background-color': '#EA580C', 'text-background-opacity': 1,
|
||
'color': '#FFFFFF', 'font-weight': 700 } },
|
||
{ selector: 'edge[type = "REASONING_NEXT"]',
|
||
style: { 'line-color': '#EA580C', 'target-arrow-color': '#EA580C',
|
||
'target-arrow-shape': 'triangle', 'curve-style': 'bezier',
|
||
width: 2, 'z-index': 99 } },
|
||
```
|
||
|
||
Color ramp: single warm color (`#EA580C`) for v1. Gradient cold→warm is v2 polish.
|
||
|
||
### 3.4 Integration points
|
||
|
||
- `StudioPage.vue` → "Maintain" tab gains an **"Import reasoning trace"** action
|
||
(don't add a 3rd mode — the viz lives inside the graph view, not a new workspace).
|
||
- `GraphView.vue` → add `defineExpose({ cy })` + a `<slot name="overlay" :cy="cy"/>`
|
||
that the parent can populate with `<ReasoningPanel>`.
|
||
- `ReasoningPanel` appears as a right rail when a trace is loaded; collapsible.
|
||
|
||
### 3.5 Empty / error states
|
||
|
||
- **Graph empty for this doc** → "Run the Maintain step first. Neo4j has no graph for
|
||
this document yet." (the Maintain button is literally next to it.)
|
||
- **All `section_ref`s unresolved in graph** → "None of the visited sections exist in
|
||
the graph. The agent may have been run against a different document, or the doc was
|
||
re-analyzed since. Re-run Maintain or re-run the agent."
|
||
- **Some resolved, some not** → show trace with the missing ones greyed out in the panel.
|
||
|
||
### 3.6 Tests
|
||
|
||
No backend tests in v1 (no backend code).
|
||
|
||
Frontend (Vitest):
|
||
- `reasoningStore.test.ts` — import trace, active iteration transitions, reset on doc change.
|
||
- `graphReasoningOverlay.test.ts` — given a mock `cy` (`cytoscape({ headless: true })`)
|
||
with a known node set, verify `visited` class applied to the right ids and the
|
||
correct synthetic edges added.
|
||
- `ReasoningPanel.test.ts` — empty / loaded / partial-resolution states.
|
||
|
||
### 3.7 Out of scope for v1
|
||
- Live agent runner (v2).
|
||
- Multi-doc queries — reject import if `RAGResult` was produced against `len(sources) > 1`.
|
||
- Phrase-level attribution — `docling-agent` doesn't emit it.
|
||
- Persisting traces in Neo4j — see §7.
|
||
- PDF highlighting — dropped from v1. Could come back as v2.5 if demand exists.
|
||
|
||
---
|
||
|
||
## 4. File inventory (v1)
|
||
|
||
**New — R&D sidecar** (already scaffolded on this branch)
|
||
- `experiments/reasoning-trace/inspect_doc.py` — self-contained `uv run` script.
|
||
- `experiments/reasoning-trace/README.md`
|
||
- `experiments/reasoning-trace/.gitignore`
|
||
|
||
**New — frontend**
|
||
- `frontend/src/features/reasoning/**` (see §3.2)
|
||
- Vitest siblings under `**/*.test.ts`
|
||
|
||
**Touched**
|
||
- `frontend/src/features/analysis/ui/GraphView.vue` — `defineExpose({ cy })` +
|
||
`<slot name="overlay">` + 3 new style selectors.
|
||
- `frontend/src/pages/StudioPage.vue` — "Import reasoning trace" action in the
|
||
Maintain tab rail.
|
||
|
||
**Untouched**
|
||
- Entire `document-parser/` backend — no new router, service, schema, or dep.
|
||
- `pyproject.toml` / `requirements.txt` — **no new runtime dep in v1**.
|
||
- Neo4j schema — synthetic edges are client-side Cytoscape only.
|
||
- OpenSearch / ingestion — untouched.
|
||
- SQLite schema — no migration.
|
||
|
||
---
|
||
|
||
## 5. Risks & mitigations
|
||
|
||
| Risk | Mitigation |
|
||
|---|---|
|
||
| `RAGResult` schema drifts in `docling-agent` | `schema_version` discriminator; strict pydantic; one canonical fixture from Peter pinned in CI. |
|
||
| `section_ref` variants (`#/texts/3` vs `#/body/texts/3`) | Normalize in parser; regex test matrix. |
|
||
| Synthetic groups without `prov` | Documented child-walk fallback + `resolved_via_child` status surfaced in UI. |
|
||
| Large `RAGResult` (hundreds of iterations) | Hard-cap `iterations` at 50 in v1 (Peter's agent uses `max_iterations=5` by default) — return 413 above. |
|
||
| `document_json` blob large (some docs > 5 MB) | `analysis_repo` already handles it; but **do not** log the blob. Add redaction test. |
|
||
| Section ref not in graph (doc not through Maintain, or re-analyzed) | Explicit empty-state in `ReasoningPanel` with a link to the Maintain tab. Partial resolution shown as grey in the trace list. |
|
||
| Feature creeping into 0.5.0 | This branch targets **v0.6.0**. Do not merge into `release/0.5.0`. Rebase onto the next release branch when cut. |
|
||
|
||
---
|
||
|
||
## 6. Spec anchoring
|
||
|
||
Pin the `RAGResult` shape to **docling-agent commit SHA at the time of v1 merge** in
|
||
a short ADR `docs/architecture/adrs/ADR-002-rag-result-schema.md`. The schema is
|
||
upstream, unversioned, and will move — this doc freezes the contract Studio imports.
|
||
|
||
---
|
||
|
||
## 7. v2 preview — demo mode (not in this plan)
|
||
|
||
Kept here to constrain v1 interfaces so nothing needs rewriting:
|
||
- `POST /api/rag/answer` — server-side runner. Accepts `{doc_id, question, model_id}`.
|
||
Streams iterations via SSE. Frontend consumes the stream with the same
|
||
`GraphReasoningOverlay` used by v1 import — iterations appear one by one with
|
||
staggered reveal (~400 ms) as the SSE stream drips them in.
|
||
- Ollama wired through `Mellea` — new optional dep group `rag`.
|
||
- Persist traces in Neo4j as `(:ReasoningRun {id, query, converged})-[:VISITED {order,
|
||
reason, can_answer}]->(:Element)` for replay + cross-run analytics. Leverages
|
||
`TreeWriter` pattern already present. This is where the synthetic UI edges become
|
||
real graph edges.
|
||
- Cross-run comparison view: overlay multiple runs on the same graph, diff the paths.
|
||
|
||
---
|
||
|
||
## 8. Branch & workflow
|
||
|
||
- Branch: **`feature/reasoning-trace`** off `origin/release/0.5.0`.
|
||
- Merge target: **next release branch (`release/0.6.0`)** once cut — *not* `0.5.0`.
|
||
- Until then: live on the feature branch; rebase onto `release/0.5.0` periodically to
|
||
absorb Neo4j fixes.
|
||
- Issues: one umbrella + one per §4 subsystem (resolver, endpoint, UI panel, overlay,
|
||
import dialog, tests). Commit with `Closes #NNN` per project convention.
|
||
- PR: opened against `release/0.6.0` when available; draft in the meantime.
|
||
|
||
---
|
||
|
||
## 9. Open questions (answered by the sidecar first run)
|
||
|
||
1. Are emitted `section_ref`s reachable as `elem::${ref}` in the Neo4j graph built
|
||
by `TreeWriter`? I.e. is the `self_ref` the agent sees the same `self_ref` we
|
||
wrote to the graph? (Expected yes — both come from the same `DoclingDocument` —
|
||
but the sidecar on a real doc from SQLite will confirm in one run.)
|
||
2. Hit rate of the agent: with `max_iterations=5` and `granite4:micro-h`, does it
|
||
converge, and how many sections does it actually visit? Determines if the overlay
|
||
ever has more than 1–2 marked nodes (and whether `REASONING_NEXT` edges are worth
|
||
the effort vs just node markers).
|
||
3. Quality of `iteration.reason` — is it substantive enough to show in the panel, or
|
||
LLM filler we should hide? Sidecar output will tell.
|
||
4. Fallback when no section headers exist (`RAGResult(iterations=[], converged=True,
|
||
answer=<full md>)` — see rag.py): what does the panel show? Probably a degraded
|
||
"no trace available, full-doc answer" state.
|