Commit graph

34 commits

Author SHA1 Message Date
Pier-Jean Malandrino
77fcb32e7f fix(reasoning): re-scroll PDF when re-clicking the active iteration
The watch-based plumbing from iteration click to PDF scroll relied on a
"flip via null" pattern (assign null then the value) to coerce Vue into
re-firing the watcher. Vue 3 collapses synchronous mutations of the same
ref and only delivers the final value, so the trick was a no-op: a second
click on the same iteration left the document view stuck on the previous
page. The bug only showed when the trace had a single iteration — with
several, the user naturally clicks different ones and the value really
changes.

Replace the watch chain with imperative dispatch. ReasoningPanel now just
emits iterationFocus; ReasoningWorkspace handles it by calling the graph
focus and the new StructureViewer.scrollToFocused method directly. Both
side effects fire on every click regardless of state.
2026-04-27 11:48:32 +02:00
Pier-Jean Malandrino
9ec64961fc feat(reasoning): bidirectional PDF ↔ graph focus + DocumentView mode
Propagate Docling `self_ref` through PageElement so bboxes and graph nodes
share a stable identity. Add a Document/Graph mode switch to the reasoning
workspace; selecting a node highlights its bbox (numbered badge, focus ring,
optional dim of non-visited) and clicking a bbox re-centers the graph.
2026-04-23 22:07:58 +02:00
Pier-Jean Malandrino
bef7ec4686 feat(reasoning): live docling-agent runner + UX polish
Backend — live runner
- New `POST /api/documents/:id/rag` endpoint. Loads `document_json` from
  SQLite, reconstructs the DoclingDocument, wraps the model id in
  `ModelIdentifier(ollama_name=...)`, and calls `agent._rag_loop`
  off-thread (blocking sync call). Returns a `RAGResult` in the shape
  the existing v1 import path already consumes, so the frontend overlay
  is fully reused.
- `_rag_loop` is private upstream; we call it because `run()` wraps the
  answer in a synthetic DoclingDocument and drops the iteration trace.
- Settings: `RAG_ENABLED`, `OLLAMA_HOST`, `RAG_MODEL_ID`. Router mounts
  unconditionally; handler 503s when the flag is off or deps aren't
  installed. `rag_available` surfaced in `/api/health`.
- Maps known docling-agent bugs to readable HTTP errors: 502 with
  "the model couldn't produce a parseable answer" when `_rag_loop`
  raises `IndexError` from `find_json_dicts([])[0]` after 3 + 3
  rejection-sampling retries (model-dependent).
- Tests: 11 cases (flag off, query empty, no analysis, happy path,
  model_id wrap, Ollama env, IndexError → 502, other errors → 500,
  deps missing → 503).

Backend — bug fix
- Default `BATCH_PAGE_SIZE` flipped from `10` to `0` to match the
  dataclass default. The old default silently dropped `document_json`
  (see `domain/services.merge_results`) for any doc > 10 pages, which
  broke the reasoning tunnel. Set `BATCH_PAGE_SIZE>0` explicitly on
  memory-constrained deploys if batching is wanted.

Frontend — runner UX
- `features/reasoning/api.ts:runReasoning()` — POST wrapper.
- `RunReasoningDialog.vue` — query textarea + optional model_id
  override. Blocks close while running, 20-40s loading state,
  synthesises a sidecar-shaped envelope so the panel surfaces query +
  model the same way an imported trace would.
- `ReasoningWorkspace.vue` — primary "Run reasoning" button; "Import
  trace" relegated to ghost secondary.
- Store: `runDialogOpen`, `running`, `setRunning`.

Frontend — answer polish
- Answer rendered through `marked` + DOMPurify (models emit markdown
  lists; `pre-wrap` rendered them as plain "1. …" strings).
- Dedicated answer block with orange border, "ANSWER" label, "Copy"
  button (clipboard + "Copied ✓" feedback).
- IterationCard: drop the duplicate `response` block (the main answer
  is authoritative); style reasons equal to `"fallback"` (docling-agent
  `select_from_failure` placeholder) as italic muted "— no structured
  rationale".

Frontend — node details contents
- Clicking a SectionHeader (or any node with compound children) lists
  its contained elements in `NodeDetailsPanel` under a new "Contents"
  block. Children come from the same `parentMap` used for Cytoscape
  compound parenting (explicit PARENT_OF + synthetic section scope),
  inverted once and cached as a computed.
- Click a child row → pan the viewport to it + swap the selection.

Housekeeping
- `cytoscape-navigator` removed from `package-lock.json` (follow-up
  from the minimap removal in the previous commit).
2026-04-21 17:11:54 +02:00
Pier-Jean Malandrino
1f02274ac4 feat(reasoning): reasoning-trace viewer v1 with SQLite-backed graph
Adds the `docling-agent` reasoning-trace viewer as a Studio tunnel, per
`docs/design/reasoning-trace.md`. Users pick an analyzed document, import
a RAGResult JSON, and the iterations are overlaid on the document graph.

Graph source is decoupled from Neo4j: a new pure builder
(`infra/docling_graph.build_graph_payload`) reads `document_json` from
SQLite and emits the same Cytoscape-shaped payload that `fetch_graph`
returns from Neo4j. Neo4j stays exclusive to the Maintain ingestion
pipeline. Shared DoclingDocument helpers live in `infra/docling_tree.py`
so TreeWriter and the builder can't drift on label taxonomy or tree walks.

Also removes the Cytoscape minimap (cytoscape-navigator) from GraphView:
second render instance hurt perf on large documents for no UX win.

Backend
- new `GET /api/documents/:id/reasoning-graph` (SQLite-only)
- new `infra/docling_tree.py`, `infra/docling_graph.py`
- `analysis_repo.find_latest_completed_by_document`
- tests: `test_docling_graph.py` (builder), `test_graph_api.py` (endpoint)

Frontend
- `features/reasoning/` — store, overlay, types, panel, import dialog,
  workspace, doc picker
- new `ReasoningPage` + `/reasoning` and `/reasoning/:docId` routes
- `GraphView` gains a `fetcher` prop so reasoning can inject the
  SQLite-backed fetcher while Maintain keeps using the Neo4j one
- drops minimap (nav container, dep, CSS)
- legend filters + section parenting extracted for reuse
- i18n base strings (FR + EN)
2026-04-21 11:30:47 +02:00
Pier-Jean Malandrino
aa60fbb768 feat(ui): add Maintain step with graph visualization
Move the graph view from a Verify-tab (where it sat post-analysis, off
the main flow) to a dedicated Maintain step after Ingest, so the graph
result is visible at the natural end of the Configure → Verify →
Prepare → Ingest → Maintain pipeline.

- StudioPage: new 'maintain' mode toggle + panel rendering GraphView
- ResultTabs: remove obsolete graph tab
- i18n: add studio.maintain (fr + en)
- GraphView: fix init order — flip loading off and await nextTick before
  renderGraph so the canvas <div> is mounted when cytoscape reads its
  container ref (previous code bailed silently on null ref)
2026-04-20 10:13:18 +02:00
Pier-Jean Malandrino
c9359f60e1 feat(neo4j): Day 3 — ChunkWriter, graph API, GraphView, README
ChunkWriter mirrors chunks into Neo4j after OpenSearch indexing, creating
HAS_CHUNK edges and DERIVED_FROM back-references to the source Elements
(via doc_items propagated from the local chunker).

Graph API: GET /api/documents/{id}/graph returns a cytoscape-shaped
payload with nodes + edges for Document / Element / Page / Chunk.
Hard cap at 200 pages returns HTTP 413 per design §8.4.

Frontend: new Graph tab in Studio results, rendered with Cytoscape.js +
dagre layout (lazy-loaded, ~175 KB gz). Legend, node styling per element
label, directional edges styled per edge type.

README gains a Neo4j section with the schema, three demo Cypher
queries, and env vars. Backend tests skip cleanly when the neo4j python
package is not installed locally.

Refs #186
2026-04-20 10:13:18 +02:00
Pier-Jean Malandrino
c4057bb010 fix(e2e): update chunks synchronously in store to prevent rechunk timeout
Some checks failed
Auto-close issues on release branch merge / Close referenced issues (push) Has been cancelled
Release Docker Images / Build & push — local (push) Has been cancelled
Release Docker Images / Build & push — remote (push) Has been cancelled
ChunkPanel emitted 'rechunked' which triggered an async re-fetch via
analysisStore.select() — but Vue does not await async emit handlers,
so chunk-cards never appeared before the E2E 30s timeout.

Now rechunk/edit/delete write returned chunks directly into the
analysis store via updateChunks(), removing the async round-trip.
2026-04-14 17:06:34 +02:00
Pier-Jean Malandrino
3a09c18c59 fix(decoupling): eliminate cross-feature imports and add typed health endpoint
Frontend decoupling:
- Create shared/appConfig.ts as reactive bridge (locale, maxFileSizeMb,
  maxPageCount) eliminating shared→features and feature→feature imports
- Give history feature its own Pinia store and API layer (was re-export
  of analysis store)
- Give chunking feature its own Pinia store and API layer (was importing
  from analysis)
- ChunkPanel receives analysis data via props instead of cross-feature
  store import
- document/store reads maxFileSizeMb from shared config instead of
  importing feature-flags store
- shared/i18n reads locale from shared config instead of importing
  settings store

Backend:
- Add HealthResponse Pydantic schema for /api/health endpoint

Closes #140, closes #141, closes #142, closes #143
2026-04-10 13:11:29 +02:00
Pier-Jean Malandrino
f6992d91bc feat: segmented batch progress bar with ring indicator
Replace the basic progress bar with a ring percentage indicator,
segmented batch bar with pulse animation, and an inline mini
progress bar in the top banner visible from any tab.
2026-04-09 15:51:46 +02:00
Pier-Jean Malandrino
fdbab2f49e feat: add Karate UI e2e tests with data-e2e selectors (#124)
- Add e2e/ui/ as peer project to e2e/api/ (own pom.xml, runner, config)
- 5 critical UI journeys: upload, delete, analysis, batch-progress, rechunk
- 4 local-only tests: sidebar, i18n, error-states, pipeline-options
- 1 full happy path workflow covering all modes
- Add data-e2e attributes on all tested Vue components (decoupled from CSS)
- Add CONVENTIONS.md with 7 golden rules for writing Karate UI tests
- Update CI with dedicated e2e-ui job (Chrome headless, --no-sandbox)
- Update docs/contributing.md with UI test instructions

Closes #124
2026-04-08 17:53:52 +02:00
Pier-Jean Malandrino
fc866ce229 feat: batch large documents with page_range and progress reporting
Use Docling's native page_range parameter to split large PDFs into
sequential batches, preventing memory exhaustion and timeouts.
Progress is reported via existing polling mechanism.

Closes #56
2026-04-07 17:54:40 +02:00
Pier-Jean Malandrino
4b1a15f49a fix: add retry tolerance to frontend polling (H3)
A single transient network error (502, timeout) no longer kills the
polling loop. The frontend now tolerates up to 3 consecutive errors
before abandoning. Successful fetches reset the counter.

Also aligns frontend polling timeout (15 min) with backend timeout.

Ref #57 (H3, M5)
2026-04-07 15:32:38 +02:00
Pier-Jean Malandrino
76e8c55b7c Fix BboxOverlay inline handler breaking production build
Vue template compiler in production mode rejects multi-statement
inline handlers without semicolons. Extract to a named function.
2026-04-03 16:17:23 +02:00
Pier-Jean Malandrino
226ae5994e Format all frontend files with Prettier
Batch reformat to eliminate 23 files with inconsistent formatting
that were accumulating across feature branches.
2026-04-03 15:50:32 +02:00
Pier-Jean Malandrino
1ed9552224 Dim non-highlighted bboxes in Verify mode for visual consistency
Apply the same dimming strategy used in Prepare mode: when an element
is highlighted, reduce stroke and fill opacity of all other bboxes
so the focused element stands out clearly.
2026-04-02 16:19:43 +02:00
Pier-Jean Malandrino
4af6b5b231 Add chunk-to-bbox hover highlighting in Prepare mode
Extract bounding boxes from chunk doc_items provenance in the chunker,
propagate through domain/service/API layers, and render highlighted
bboxes on canvas when hovering a chunk card. Reset highlights on
mode and page changes to prevent stale visual state.
2026-04-02 14:47:31 +02:00
Pier-Jean Malandrino
ebdf9ae1be Add frontend chunking feature with Prepare mode
Chunk/ChunkingOptions types, rechunkAnalysis API, store actions
(currentChunks, rechunk), ChunkPanel UI, Prepare tab in StudioPage
gated by chunking feature flag, i18n keys (FR/EN), and tests.
2026-04-02 12:33:07 +02:00
Pier-Jean Malandrino
001cf6807c Fix audit findings: security, robustness, and dead code
- Remove dead get_analysis_service() function in main.py
- Scope file deletion to UPLOAD_DIR to prevent path traversal
- Parse datetime strings back to datetime objects in repos
- Add 10-minute polling timeout in frontend analysis store
- Accept .pdf extension (not just MIME type) on drag-and-drop
- Guard localStorage access for private browsing compatibility

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 16:52:36 +02:00
Pier-Jean Malandrino
1edd0668d1 Fix Ruff errror, update node and fix buttton breakline 2026-03-23 10:13:24 +01:00
pjmalandrino
b45bb87339 COnsolidation of bbox pipelin and add tests 2026-03-21 19:45:04 +01:00
pjmalandrino
8871c9162c Migrate frontend in typescript 2026-03-21 15:58:43 +01:00
pjmalandrino
577225d10e Add qualityt check and contributing doc 2026-03-21 15:34:54 +01:00
pjmalandrino
2db251182f Refacto and add cp paste btn 2026-03-21 15:03:10 +01:00
pjmalandrino
7bb4f8e298 Split the results of OCR 2026-03-20 21:45:26 +01:00
pjmalandrino
ad1f1a81d3 Mke some fixes and refacto 2026-03-20 12:36:26 +01:00
pjmalandrino
e69ff90650 Connect options with model configuration 2026-03-20 08:59:20 +01:00
pjmalandrino
049ef443cc Add a stronger testing strategy 2026-03-18 14:46:37 +01:00
pjmalandrino
ac1b68ac18 Final fixes before first release 2026-03-17 17:09:34 +01:00
pjmalandrino
4c6aff3029 Work on full Docker integration 2026-03-17 13:33:36 +01:00
pjmalandrino
f10e011830 Add page classification on MD panel 2026-03-17 10:59:01 +01:00
pjmalandrino
f1aa39b75f Fix display error on visual mode 2026-03-17 10:28:50 +01:00
pjmalandrino
9c2e42ace8 Include pretty visual mode 2026-03-17 09:19:46 +01:00
pjmalandrino
88acda3a48 Imrpove sreen chaining and visual, add configurability 2026-03-17 09:00:31 +01:00
pjmalandrino
50ba1d43dd Make a nex project base, managing new Docling version 2026-03-17 08:43:00 +01:00