Compare commits

..

19 commits

Author SHA1 Message Date
Pier-Jean Malandrino
77fcb32e7f fix(reasoning): re-scroll PDF when re-clicking the active iteration
The watch-based plumbing from iteration click to PDF scroll relied on a
"flip via null" pattern (assign null then the value) to coerce Vue into
re-firing the watcher. Vue 3 collapses synchronous mutations of the same
ref and only delivers the final value, so the trick was a no-op: a second
click on the same iteration left the document view stuck on the previous
page. The bug only showed when the trace had a single iteration — with
several, the user naturally clicks different ones and the value really
changes.

Replace the watch chain with imperative dispatch. ReasoningPanel now just
emits iterationFocus; ReasoningWorkspace handles it by calling the graph
focus and the new StructureViewer.scrollToFocused method directly. Both
side effects fire on every click regardless of state.
2026-04-27 11:48:32 +02:00
Pier-Jean Malandrino
9ec64961fc feat(reasoning): bidirectional PDF ↔ graph focus + DocumentView mode
Propagate Docling `self_ref` through PageElement so bboxes and graph nodes
share a stable identity. Add a Document/Graph mode switch to the reasoning
workspace; selecting a node highlights its bbox (numbered badge, focus ring,
optional dim of non-visited) and clicking a bbox re-centers the graph.
2026-04-23 22:07:58 +02:00
Pier-Jean Malandrino
bef7ec4686 feat(reasoning): live docling-agent runner + UX polish
Backend — live runner
- New `POST /api/documents/:id/rag` endpoint. Loads `document_json` from
  SQLite, reconstructs the DoclingDocument, wraps the model id in
  `ModelIdentifier(ollama_name=...)`, and calls `agent._rag_loop`
  off-thread (blocking sync call). Returns a `RAGResult` in the shape
  the existing v1 import path already consumes, so the frontend overlay
  is fully reused.
- `_rag_loop` is private upstream; we call it because `run()` wraps the
  answer in a synthetic DoclingDocument and drops the iteration trace.
- Settings: `RAG_ENABLED`, `OLLAMA_HOST`, `RAG_MODEL_ID`. Router mounts
  unconditionally; handler 503s when the flag is off or deps aren't
  installed. `rag_available` surfaced in `/api/health`.
- Maps known docling-agent bugs to readable HTTP errors: 502 with
  "the model couldn't produce a parseable answer" when `_rag_loop`
  raises `IndexError` from `find_json_dicts([])[0]` after 3 + 3
  rejection-sampling retries (model-dependent).
- Tests: 11 cases (flag off, query empty, no analysis, happy path,
  model_id wrap, Ollama env, IndexError → 502, other errors → 500,
  deps missing → 503).

Backend — bug fix
- Default `BATCH_PAGE_SIZE` flipped from `10` to `0` to match the
  dataclass default. The old default silently dropped `document_json`
  (see `domain/services.merge_results`) for any doc > 10 pages, which
  broke the reasoning tunnel. Set `BATCH_PAGE_SIZE>0` explicitly on
  memory-constrained deploys if batching is wanted.

Frontend — runner UX
- `features/reasoning/api.ts:runReasoning()` — POST wrapper.
- `RunReasoningDialog.vue` — query textarea + optional model_id
  override. Blocks close while running, 20-40s loading state,
  synthesises a sidecar-shaped envelope so the panel surfaces query +
  model the same way an imported trace would.
- `ReasoningWorkspace.vue` — primary "Run reasoning" button; "Import
  trace" relegated to ghost secondary.
- Store: `runDialogOpen`, `running`, `setRunning`.

Frontend — answer polish
- Answer rendered through `marked` + DOMPurify (models emit markdown
  lists; `pre-wrap` rendered them as plain "1. …" strings).
- Dedicated answer block with orange border, "ANSWER" label, "Copy"
  button (clipboard + "Copied ✓" feedback).
- IterationCard: drop the duplicate `response` block (the main answer
  is authoritative); style reasons equal to `"fallback"` (docling-agent
  `select_from_failure` placeholder) as italic muted "— no structured
  rationale".

Frontend — node details contents
- Clicking a SectionHeader (or any node with compound children) lists
  its contained elements in `NodeDetailsPanel` under a new "Contents"
  block. Children come from the same `parentMap` used for Cytoscape
  compound parenting (explicit PARENT_OF + synthetic section scope),
  inverted once and cached as a computed.
- Click a child row → pan the viewport to it + swap the selection.

Housekeeping
- `cytoscape-navigator` removed from `package-lock.json` (follow-up
  from the minimap removal in the previous commit).
2026-04-21 17:11:54 +02:00
Pier-Jean Malandrino
1f02274ac4 feat(reasoning): reasoning-trace viewer v1 with SQLite-backed graph
Adds the `docling-agent` reasoning-trace viewer as a Studio tunnel, per
`docs/design/reasoning-trace.md`. Users pick an analyzed document, import
a RAGResult JSON, and the iterations are overlaid on the document graph.

Graph source is decoupled from Neo4j: a new pure builder
(`infra/docling_graph.build_graph_payload`) reads `document_json` from
SQLite and emits the same Cytoscape-shaped payload that `fetch_graph`
returns from Neo4j. Neo4j stays exclusive to the Maintain ingestion
pipeline. Shared DoclingDocument helpers live in `infra/docling_tree.py`
so TreeWriter and the builder can't drift on label taxonomy or tree walks.

Also removes the Cytoscape minimap (cytoscape-navigator) from GraphView:
second render instance hurt perf on large documents for no UX win.

Backend
- new `GET /api/documents/:id/reasoning-graph` (SQLite-only)
- new `infra/docling_tree.py`, `infra/docling_graph.py`
- `analysis_repo.find_latest_completed_by_document`
- tests: `test_docling_graph.py` (builder), `test_graph_api.py` (endpoint)

Frontend
- `features/reasoning/` — store, overlay, types, panel, import dialog,
  workspace, doc picker
- new `ReasoningPage` + `/reasoning` and `/reasoning/:docId` routes
- `GraphView` gains a `fetcher` prop so reasoning can inject the
  SQLite-backed fetcher while Maintain keeps using the Neo4j one
- drops minimap (nav container, dep, CSS)
- legend filters + section parenting extracted for reuse
- i18n base strings (FR + EN)
2026-04-21 11:30:47 +02:00
Pier-Jean Malandrino
df786a4ab4
Merge pull request #188 from scub-france/feature/neo4j-integration
feat(neo4j): graph storage, graph API, Maintain UI step
2026-04-20 10:33:36 +02:00
Pier-Jean Malandrino
e4c53f1809 test(chunking): add doc_items field to ChunkResult serialization test
The `doc_items` field was added to `ChunkResult` earlier in the
feature branch (used by ChunkWriter for DERIVED_FROM edges), but the
test fixture was never updated. CI caught it now that the PR is open.

Fixes: tests/test_chunking.py::TestChunkResult::test_serializable
2026-04-20 10:26:05 +02:00
Pier-Jean Malandrino
5bc98ee483 chore(lint): fix ruff violations on Neo4j files
Pre-existing ruff violations surfaced by CI on PR #188:

- TC001: move runtime `Neo4jDriver` imports into `TYPE_CHECKING`
  blocks (queries.py, chunk_writer.py, schema.py, tree_reader.py,
  tree_writer.py)
- SIM117: combine nested `async with` in chunk_writer.write_chunks
  and tree_writer.write_document
- SIM105: replace try/except/pass with `contextlib.suppress` in
  tree_writer._element_props
- F401: remove unused `Neo4jDriver` import in main._init_neo4j
- RUF100: remove unused `# noqa: E402` in tests/neo4j/conftest.py
- I001: sort imports in tests/neo4j/test_chunk_writer.py and
  test_tree_writer.py

Zero behaviour change. `ruff check .` now passes cleanly.
2026-04-20 10:22:33 +02:00
Pier-Jean Malandrino
aa60fbb768 feat(ui): add Maintain step with graph visualization
Move the graph view from a Verify-tab (where it sat post-analysis, off
the main flow) to a dedicated Maintain step after Ingest, so the graph
result is visible at the natural end of the Configure → Verify →
Prepare → Ingest → Maintain pipeline.

- StudioPage: new 'maintain' mode toggle + panel rendering GraphView
- ResultTabs: remove obsolete graph tab
- i18n: add studio.maintain (fr + en)
- GraphView: fix init order — flip loading off and await nextTick before
  renderGraph so the canvas <div> is mounted when cytoscape reads its
  container ref (previous code bailed silently on null ref)
2026-04-20 10:13:18 +02:00
Pier-Jean Malandrino
5a2eaacd4d fix(neo4j): rewrite fetch_graph with CALL subqueries
Previous query chained 6 OPTIONAL MATCH clauses for edges with no
intervening WITH collect(), producing a cartesian product. At 6 pages
(~60 elements, ~300 edges) Neo4j hit 102% CPU and hung > 5min.

Rewritten with one CALL {} subquery per node/edge type: each block
returns a single row with its collected list — no multiplication across
types. 6-page doc now returns in 213ms (was: no return).

Python reshape code (queries.py:137-210) untouched — record keys and
edge map shape preserved.

Refs: https://neo4j.com/developer/kb/using-subqueries-to-control-the-scope-of-aggregations/
2026-04-20 10:13:18 +02:00
Pier-Jean Malandrino
c9359f60e1 feat(neo4j): Day 3 — ChunkWriter, graph API, GraphView, README
ChunkWriter mirrors chunks into Neo4j after OpenSearch indexing, creating
HAS_CHUNK edges and DERIVED_FROM back-references to the source Elements
(via doc_items propagated from the local chunker).

Graph API: GET /api/documents/{id}/graph returns a cytoscape-shaped
payload with nodes + edges for Document / Element / Page / Chunk.
Hard cap at 200 pages returns HTTP 413 per design §8.4.

Frontend: new Graph tab in Studio results, rendered with Cytoscape.js +
dagre layout (lazy-loaded, ~175 KB gz). Legend, node styling per element
label, directional edges styled per edge type.

README gains a Neo4j section with the schema, three demo Cypher
queries, and env vars. Backend tests skip cleanly when the neo4j python
package is not installed locally.

Refs #186
2026-04-20 10:13:18 +02:00
Pier-Jean Malandrino
ee92e3c580 feat(neo4j): Day 2 — TreeWriter, TreeReader, pipeline wiring
Serialize a DoclingDocument to a Neo4j graph: Document + Page + Element
nodes with dynamic specific labels (SectionHeader, Paragraph, Table,
Figure, …), plus HAS_ROOT / PARENT_OF / NEXT / ON_PAGE edges. Replace-on-
write for idempotent re-ingestion.

The reader returns the verbatim document_json stored on the Document
node — reconstruction from graph nodes is deferred to v0.6.

Wired into AnalysisService._finalize_analysis: runs after conversion,
degrades gracefully by default, fails fast when neo4j_required is set.

Refs #186
2026-04-20 10:13:18 +02:00
Pier-Jean Malandrino
3474390688 docs(neo4j): ADR-001 graph viz lib + 200-page endpoint cap
Document the Cytoscape.js vs vis-network / Neovis / D3 / Reagraph analysis
for the graph view, and make the 200-page cap on /api/documents/{id}/graph
explicit (HTTP 413 + truncated flag beyond the cap).

Refs #186
2026-04-20 10:13:18 +02:00
Pier-Jean Malandrino
dfbca40730 feat(neo4j): Day 1 — compose service, driver, schema bootstrap
Add Neo4j as an optional graph-native storage layer (ingestion profile).
Introduces infra/neo4j with a singleton async driver wrapper and an
idempotent bootstrap of constraints + indexes, wired into the FastAPI
lifespan. Integration tests skip when no live Neo4j is reachable.

Refs #186
2026-04-20 10:13:18 +02:00
Pier-Jean Malandrino
358e575f0f
Merge pull request #185 from scub-france/feature/remote-chunking
feat: enable chunking in remote (Docling Serve) mode
2026-04-19 20:20:14 +02:00
Pier-Jean Malandrino
3bdc4cec50 feat: enable chunking in remote (Docling Serve) mode
Hybrid approach: reuse LocalChunker to chunk the DoclingDocument JSON
returned by Serve, so chunking works identically in both local and
remote modes without calling Serve's chunk endpoint.

Backend:
- _build_chunker() always returns LocalChunker (remove engine guard)
- Use docling-core[chunking] extra for required dependencies
- Skip client-side batching in remote mode (Serve manages its own
  resources, and batching discards document_json needed for chunking)
- Fix Serve form fields: remove generate_page_images (not a Serve
  field), use repeated form keys for to_formats and page_range
- Log Serve error response body on 4xx/5xx for diagnosis
- Fix FastAPI 204 DELETE routes missing response_model=None

Frontend:
- Update chunking feature flag to enable Prepare UI in remote mode

Closes #51
2026-04-16 15:11:14 +02:00
Pier-Jean Malandrino
0bffe6e7d4
Merge pull request #183 from scub-france/feat/centralize-magic-numbers-arch-tests
refactor: centralize magic numbers + arch tests (#168, #177)
2026-04-16 10:51:48 +02:00
Pier-Jean Malandrino
987d43735d fix(ci): install pytestarch in backend tests job (#177)
CI was missing pytestarch dependency, causing test_architecture.py to fail
at collection time. Switch to requirements-test.txt which includes all
test dependencies.
2026-04-16 08:32:07 +00:00
Pier-Jean Malandrino
7a76d2efbd feat: add hexagonal architecture tests with pytestarch (#177)
- Create tests/test_architecture.py with 20 automated rules:
  inter-layer dependency checks (domain, services, api, infra, persistence),
  external dependency constraints (fastapi, sqlalchemy, httpx, opensearchpy),
  and port convention enforcement (Protocol only in domain.ports)
- Add requirements-test.txt with pytestarch dependency
- Fix persistence.database importing infra.settings (read DB_PATH from env directly)
2026-04-16 08:32:07 +00:00
Pier-Jean Malandrino
f2436290c5 refactor: centralize magic numbers for page dimensions, limits, and timeout (#168)
- Move DEFAULT_PAGE_WIDTH/HEIGHT to domain/value_objects.py and import in both converters
- Add opensearch_default_limit to Settings (configurable via OPENSEARCH_DEFAULT_LIMIT env var)
- Pass settings.conversion_timeout to ServeConverter, removing independent _DEFAULT_TIMEOUT
- Update OpenSearchStore to accept default_limit from Settings via constructor
2026-04-16 08:32:07 +00:00
79 changed files with 463 additions and 4490 deletions

View file

@ -18,10 +18,6 @@
# Max upload file size in MB (default: 50, 0 = unlimited)
# MAX_FILE_SIZE_MB=50
# Nginx body size limit — nginx format (default: 200M, 0 = unlimited).
# Must be >= MAX_FILE_SIZE_MB. Backend MAX_FILE_SIZE_MB is the effective arbiter.
# NGINX_MAX_BODY_SIZE=200M
# Max pages per PDF (default: 0 = unlimited). Set to 20 for HF Spaces.
# MAX_PAGE_COUNT=0

View file

@ -27,7 +27,7 @@ jobs:
with:
python-version: "3.12"
cache: pip
cache-dependency-path: document-parser/requirements-test.txt
cache-dependency-path: document-parser/requirements.txt
- name: Install system dependencies
run: sudo apt-get update && sudo apt-get install -y --no-install-recommends poppler-utils
@ -35,8 +35,8 @@ jobs:
- name: Install pinned dependencies
run: |
pip install --upgrade pip
pip install -r requirements-test.txt
pip install httpx
pip install -r requirements.txt
pip install pytest pytest-asyncio httpx
- name: Upgrade docling to latest
id: versions

View file

@ -353,12 +353,6 @@ jobs:
exit-code: 1
severity: CRITICAL
output: /tmp/trivy-critical-${{ matrix.target }}.txt
trivyignores: .trivyignore.yaml
# `latest` instead of the action default — the previously hardcoded
# v0.69.3 was yanked from GitHub releases mid-run (2026-04-29) and
# broke the gate. Following Trivy stable is safer than chasing a
# specific tag that can vanish.
version: latest
- name: Run Trivy — HIGH (informational)
if: always()
@ -369,8 +363,6 @@ jobs:
exit-code: 0
severity: HIGH
output: /tmp/trivy-high-${{ matrix.target }}.txt
trivyignores: .trivyignore.yaml
version: latest
- name: Annotate HIGH vulnerabilities
if: always()

View file

@ -1,11 +0,0 @@
vulnerabilities:
- id: CVE-2026-40393
# No `paths:` constraint — Trivy reports OS-package CVEs against the
# package name (libgbm1, libgl1-mesa-dri, libglx-mesa0, mesa-libgallium),
# not against installed file paths. A `paths:` filter would silently
# fail to match and the ignore would be a no-op (which it was).
statement: >-
Mesa OOB read (fix: Mesa 25.3.6 / 26.0.1). No Debian 13 backport available.
Pulled transitively by libgl1 (required by OpenCV/RapidOCR). Tracked in #189
with follow-up to drop libgl1 via opencv-python-headless.
expired_at: 2026-06-30

View file

@ -4,53 +4,6 @@ All notable changes to Docling Studio will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/), and this project adheres to [Semantic Versioning](https://semver.org/).
## [0.5.1] - 2026-04-30
### Fixed
- Nginx upload body cap raised from 5 MB to 200 MB (`NGINX_MAX_BODY_SIZE`, default `200M`); uploads larger than 5 MB no longer returned 413 before reaching the backend.
## [0.5.0] - 2026-04-28
### Added
- Reasoning-trace viewer: SQLite-backed graph built from `document_json`, iteration-by-iteration overlay on the document outline (StructureViewer + GraphView), bidirectional PDF ↔ graph focus
- Live reasoning runner via [docling-agent](https://github.com/docling-project/docling-agent) (Ollama backend): `POST /api/documents/:id/reasoning` returns answer + iteration trace + convergence flag (gated by `REASONING_ENABLED`, off by default)
- `LLMProvider` port abstraction with `OllamaProvider` adapter — opens the door to alternate LLM backends once docling-agent supports them
- Neo4j graph storage pipeline: `TreeWriter` + `ChunkWriter` adapters, schema bootstrap, graph fetch endpoint, `Maintain` step with cytoscape-based graph visualization
- Architecture decision record (ADR-001): graph visualization library choice and 200-page endpoint cap
- Remote chunking: enabled in Docling Serve mode (previously local-only)
- Hexagonal architecture tests powered by `pytestarch` (CI-enforced)
- Centralized magic numbers for page dimensions, limits, and timeouts
- Paste image size/type limits (env vars `MAX_PASTE_IMAGE_SIZE_MB`, `PASTE_ALLOWED_IMAGE_TYPES`); surfaced in `/api/health`
### Changed
- **Breaking — `RAG_*` env vars renamed to `REASONING_*`**: `RAG_ENABLED``REASONING_ENABLED`, `RAG_MODEL_ID``REASONING_MODEL_ID`. Health response field `ragAvailable``reasoningAvailable`. New `LLM_PROVIDER_TYPE` env var (default `ollama`) materializes the LLM-provider abstraction.
- **Breaking — reasoning endpoint renamed `/rag` → `/reasoning`**: `POST /api/documents/:id/rag` is now `POST /api/documents/:id/reasoning`. Aligns with frontend `reasoning` feature naming.
- `api/reasoning.py` refactored to depend on `ReasoningRunnerPort`; concrete `docling-agent` integration moved to `infra/docling_agent_reasoning.py` (clean architecture)
- Frontend `reasoning` feature flag now reads `reasoningAvailable` from `/api/health`; sidebar entry hides when the backend is not wired (instead of failing with 503 on click)
- Documented that `docker-compose.yml` ships dev-only defaults (Neo4j `changeme` password, OpenSearch `DISABLE_SECURITY_PLUGIN=true`); operators must harden their own production deployments
- Backend logs a loud warning at boot if Neo4j is wired (`NEO4J_URI` set) with the default `changeme` password, so prod operators can't silently inherit it
- `DocumentConverter` port exposes `supports_page_batching: bool` so the analysis service no longer relies on `isinstance(converter, ServeConverter)` (LSP fix)
- `VectorStore` port gains a `ping()` method; `IngestionService.ping()` now goes through the port instead of reaching into `_vector_store._client` (encapsulation)
- API path parameters renamed `{job_id}``{analysis_id}` across `api/analyses.py` and `api/ingestion.py` to align the OpenAPI surface with the user-facing terminology (URL paths unchanged)
- Centralised `localStorage` keys (`docling-theme`, `docling-locale`) into `frontend/src/shared/storage/keys.ts` (`STORAGE_KEYS`)
- Removed the dead `apiUrl` ref from the settings store and its orphan `settings.apiUrl` i18n entries
- Document-status string `"uploaded"` extracted to `DOCUMENT_STATUS_UPLOADED` in `api/schemas.py`
- PDF preview endpoint (`GET /api/documents/{id}/preview`) now offloads the synchronous file read + rasterisation to a worker thread (`asyncio.to_thread`), unblocking the FastAPI event loop
### Fixed
- Graph: collapse Docling `InlineGroup` and `Picture` children to avoid empty leaf nodes (#197)
- Neo4j: rewrite `fetch_graph` using `CALL` subqueries for proper relationship traversal
- CI: install `pytestarch` in backend tests job (#177)
- CI: ignore CVE-2026-40393 (Mesa) with expiry — Debian has no backport (#190)
- Reasoning: re-scroll PDF when re-clicking the active iteration
- `infra/docling_tree.py:101` migrated `isinstance(bbox, (list, tuple))` to PEP 604 union (Ruff UP038)
- Cross-feature integration test moved out of `features/history/` into `src/__tests__/integration/` so feature folders stay self-contained
- Tightened terminal `assert X is not None` checks in domain/repo/service tests to compare the value (e.g. `isinstance(.., datetime)` after `mark_running()`/`mark_completed()`)
## [0.4.0] - 2026-04-13
### Added

View file

@ -24,11 +24,10 @@ FROM python:3.12-slim AS base
ARG APP_VERSION=dev
ENV APP_VERSION=${APP_VERSION}
# System deps: poppler (pdf2image), nginx, gettext-base (envsubst for nginx template)
# System deps: poppler (pdf2image), nginx
RUN apt-get update && apt-get install -y --no-install-recommends \
poppler-utils \
nginx \
gettext-base \
&& rm -rf /var/lib/apt/lists/*
# Python deps (common)
@ -42,8 +41,8 @@ COPY document-parser/ .
# Frontend static files
COPY --from=frontend-build /build/dist /usr/share/nginx/html
# Nginx config (template stored outside sites-enabled to avoid nginx loading it raw)
COPY nginx.conf.template /etc/nginx/default.template
# Nginx config
COPY nginx.conf /etc/nginx/sites-enabled/default
# Non-root user
RUN useradd --create-home --shell /bin/bash appuser
@ -53,11 +52,10 @@ RUN mkdir -p /app/uploads /app/data && chown -R appuser:appuser /app
ENV UPLOAD_DIR=/app/uploads
ENV DB_PATH=/app/data/docling_studio.db
ENV NGINX_MAX_BODY_SIZE=200M
EXPOSE 3000
CMD ["sh", "-c", "envsubst '${NGINX_MAX_BODY_SIZE}' < /etc/nginx/default.template > /etc/nginx/sites-enabled/default && nginx && exec su appuser -c 'uvicorn main:app --host 127.0.0.1 --port 8000'"]
CMD ["sh", "-c", "nginx && exec su appuser -c 'uvicorn main:app --host 127.0.0.1 --port 8000'"]
# --- Remote: lightweight, delegates to Docling Serve ---
FROM base AS remote

View file

@ -59,7 +59,7 @@ Upload a PDF, configure the extraction pipeline, and visualize the results — t
| **frontend** | Vue 3, TypeScript, Vite, Pinia | UI, PDF viewer, results display |
| **document-parser** | FastAPI, Docling, SQLite, pdf2image | REST API, document parsing, storage |
### Backend structure (hexagonal architecture — ports & adapters)
### Backend structure (clean architecture)
```
document-parser/
@ -210,7 +210,6 @@ All configuration is done via environment variables. See [`.env.example`](.env.e
| `BATCH_PAGE_SIZE` | `10` | Pages per batch (`0` = process all at once) |
| `MAX_FILE_SIZE_MB` | `50` | Maximum upload file size in MB (`0` = unlimited) |
| `MAX_PAGE_COUNT` | `0` | Maximum number of pages per document (`0` = unlimited) |
| `NGINX_MAX_BODY_SIZE` | `200M` | Nginx request body limit — nginx format (`200M`, `0` = unlimited). Must be ≥ `MAX_FILE_SIZE_MB`. |
| `RATE_LIMIT_RPM` | `100` | Max requests per minute per IP (`0` = disabled) |
## Upload Limits
@ -219,9 +218,8 @@ Docling Studio enforces configurable limits on uploaded documents to protect the
- **`MAX_FILE_SIZE_MB`** (default `50`) — rejects uploads exceeding this size. Validated at two levels: early `Content-Length` check and streaming byte count.
- **`MAX_PAGE_COUNT`** (default `0` = unlimited) — rejects documents with more pages than allowed. Useful on shared instances or Hugging Face Spaces to cap processing time.
- **`NGINX_MAX_BODY_SIZE`** (default `200M`) — nginx-level body cap, applied before the request reaches the backend. Defaults to `200M` so `MAX_FILE_SIZE_MB` is always the effective limit. Use nginx format (`50M`, `1G`, `0` for unlimited).
Both application limits are exposed in the `/api/health` endpoint so the frontend can display them to the user before upload. Set either to `0` to disable the corresponding check.
Both limits are exposed in the `/api/health` endpoint so the frontend can display them to the user before upload. Set either to `0` to disable the corresponding check.
## Ingestion Pipeline (opt-in)
@ -310,43 +308,6 @@ RETURN d.title, t.caption, t.cells_json
The in-app **Graph** tab (under *Results*) renders the per-document graph with [Cytoscape.js](https://js.cytoscape.org/) (see [ADR-001](docs/architecture/adrs/ADR-001-graph-visualization-library.md) for the library choice). Documents with more than **200 pages** return `HTTP 413` from `GET /api/documents/{id}/graph`; pagination ships in v0.6.
## Live Reasoning (opt-in, R&D)
Docling Studio can run [docling-agent](https://github.com/docling-project/docling-agent)'s Chunkless RAG loop against an analyzed document and return a full **reasoning trace** — the path the agent walked through the document outline, with the section reference / rationale / answer for each iteration. The trace is overlaid on the document graph so you can *see* how the agent navigated the structure.
Disabled by default — pulls heavy deps (`docling-agent`, `mellea`, ~60 MB) and needs a reachable Ollama instance with the target model already pulled.
### Enable
```bash
export REASONING_ENABLED=true
export OLLAMA_HOST=http://localhost:11434 # default
export REASONING_MODEL_ID=gpt-oss:20b # any model already pulled in Ollama
# Optional, future-proof — only "ollama" is realizable today (see Architecture below):
export LLM_PROVIDER_TYPE=ollama
```
Then `pip install docling-agent mellea` (or use the `local` Docker image which bundles them) and restart the backend. The frontend reads `reasoningAvailable` from `/api/health` and hides the **Reasoning** sidebar entry when the runner isn't wired — so users never click through to a 503.
| Variable | Default | Description |
|----------|---------|-------------|
| `REASONING_ENABLED` | `false` | Master switch — `true` to enable the live runner |
| `OLLAMA_HOST` | `http://localhost:11434` | Ollama daemon URL |
| `REASONING_MODEL_ID` | `gpt-oss:20b` | Default model id (per-call override allowed via the API) |
| `LLM_PROVIDER_TYPE` | `ollama` | LLM backend selector — only `ollama` is supported today |
### Architecture
The reasoning subsystem is wired through a `ReasoningRunner` port (`document-parser/domain/ports.py`) and an `LLMProvider` abstraction:
- `domain/ports.py` defines `ReasoningRunner`, `LLMProvider`, `ReasoningParseError` (no third-party imports)
- `domain/value_objects.py` defines `LLMProviderType`, `ReasoningResult`, `ReasoningIteration`
- `infra/llm/ollama_provider.py` implements `LLMProvider` for Ollama
- `infra/docling_agent_reasoning.py` implements `ReasoningRunner` using docling-agent + mellea — all upstream coupling is here, including the `_rag_loop` workaround tracked at [docling-agent#26](https://github.com/docling-project/docling-agent/issues/26)
- `api/reasoning.py` consumes `app.state.reasoning_runner` — zero coupling to docling-agent
This makes alternate LLM backends a question of adding new `LLMProvider` adapters once docling-agent (or a replacement) supports them upstream.
## CI / Release
GitHub Actions pipelines (see [`.github/workflows/`](.github/workflows/)):

View file

@ -1,32 +1,5 @@
# =============================================================================
# Docling Studio — local development / quick-start compose.
#
# DEV DEFAULTS — NOT PRODUCTION-READY.
# This file ships sensible defaults for `docker compose up` to "just work" on
# a developer laptop. It is NOT a production deployment template:
# - Neo4j boots with `NEO4J_PASSWORD=changeme` if the env var is unset.
# - OpenSearch runs single-node with `DISABLE_SECURITY_PLUGIN=true`
# (no TLS, no auth). Fine for local indexing experiments, never for
# anything reachable from outside `localhost`.
# - No HTTPS termination, no reverse-proxy hardening, no rate limit on
# the embedding/OpenSearch services, ports bound to all interfaces.
#
# Operators running their own Docling Studio instance MUST:
# 1. Override `NEO4J_PASSWORD` (and rotate it) — see `.env.example`.
# 2. Re-enable the OpenSearch security plugin and configure TLS/users —
# see https://opensearch.org/docs/latest/security/.
# 3. Bind sensitive services to internal networks only, terminate TLS at
# a reverse proxy, and align CORS_ORIGINS / RATE_LIMIT_RPM with the
# deployment surface.
#
# The audit pipeline expects this file to flag the dev defaults as such —
# don't silently strengthen them here without also updating the dev DX.
# =============================================================================
services:
# --- Neo4j (graph-native document structure) ---
# Dev-only auth: NEO4J_PASSWORD defaults to "changeme". Override in
# `.env` (or any orchestrator secret store) before exposing this stack.
neo4j:
profiles: ["ingestion"]
image: neo4j:5.15-community
@ -44,12 +17,7 @@ services:
retries: 10
start_period: 30s
# --- OpenSearch (single-node, security DISABLED — DEV ONLY) ---
# `DISABLE_SECURITY_PLUGIN: "true"` removes auth, TLS, and audit logging.
# This is fine for a local dev cluster bound to 127.0.0.1; it is NOT safe
# for anything reachable from another host. For production, drop this
# var, switch to a hardened image, and configure users + TLS:
# https://opensearch.org/docs/latest/security/configuration/
# --- OpenSearch (single-node, security disabled) ---
opensearch:
profiles: ["ingestion"]
image: opensearchproject/opensearch:2
@ -117,8 +85,6 @@ services:
context: ./frontend
ports:
- "3000:80"
environment:
NGINX_MAX_BODY_SIZE: ${NGINX_MAX_BODY_SIZE:-200M}
depends_on:
- document-parser

View file

@ -8,7 +8,7 @@ Two services communicating via REST. The frontend is a Vue 3 SPA served by Nginx
### Zooming into the backend
The schema above shows the macro view. Inside the backend, the code follows a **Hexagonal Architecture** (ports & adapters) with strict layer boundaries:
The schema above shows the macro view. Inside the backend, the code follows a **Clean Architecture** with strict layer boundaries:
```
┌──────────────────────────────────────────────────────┐
@ -34,9 +34,9 @@ The schema above shows the macro view. Inside the backend, the code follows a **
Dependencies flow **inward**: `api → services → domain`. The domain layer has zero knowledge of HTTP or database.
## Backend — Hexagonal Architecture (ports & adapters)
## Backend — Clean Architecture
The backend follows the hexagonal / ports-and-adapters pattern. The domain layer defines **ports** (abstract protocols in `domain/ports.py`); `infra/` provides **adapters** that implement them. Dependencies flow inward: API → Services → Domain. The domain layer has zero knowledge of HTTP, database, or any framework.
The backend follows a strict layered architecture. Dependencies flow inward: API → Services → Domain. The domain layer has zero knowledge of HTTP or database.
```
document-parser/

View file

@ -34,7 +34,7 @@ These decisions were made before the ADR process was introduced. They are docume
| Decision | Rationale | Date |
|----------|-----------|------|
| Hexagonal Architecture (ports & adapters) for backend | Decouple domain from framework — enable converter swapping (local/remote) via ports | 2025-01 |
| Clean Architecture (hexagonal) for backend | Decouple domain from framework — enable converter swapping (local/remote) | 2025-01 |
| FastAPI over Django | Async-first, lightweight, Pydantic-native — better fit for a single-purpose API | 2025-01 |
| Vue 3 + Pinia over React | Composition API + built-in reactivity — smaller bundle for this use case | 2025-01 |
| SQLite over PostgreSQL | Single-file DB, zero ops — appropriate for a document processing tool | 2025-01 |

View file

@ -1,6 +1,6 @@
# Audit 01 — Hexagonal Architecture (ports & adapters)
# Audit 01 — Clean Architecture
**Objectif** : verifier que le backend respecte le pattern hexagonal (ports dans `domain/ports.py`, adaptateurs dans `infra/`), le flux de dependances strict `api -> services -> domain`, et que chaque couche a une responsabilite claire.
**Objectif** : verifier que le backend respecte le flux de dependances strict `api -> services -> domain` et que chaque couche a une responsabilite claire.
**Cible** : `document-parser/` (hors `.venv/`, `__pycache__/`, `tests/`)

View file

@ -74,7 +74,7 @@ Les audits sont executes dans l'ordre ci-dessous. Chacun est une fiche autonome
| # | Audit | Fichier | Focus |
|---|-------|---------|-------|
| 01 | Hexagonal Architecture | [01-clean-architecture.md](audits/01-clean-architecture.md) | Ports & adapters, respect des couches, flux de dependances |
| 01 | Clean Architecture | [01-clean-architecture.md](audits/01-clean-architecture.md) | Respect des couches, flux de dependances |
| 02 | DDD | [02-ddd.md](audits/02-ddd.md) | Bounded contexts, entites, value objects, ubiquitous language |
| 03 | Clean Code | [03-clean-code.md](audits/03-clean-code.md) | Nommage, taille, lisibilite |
| 04 | KISS | [04-kiss.md](audits/04-kiss.md) | Simplicite, pas de sur-ingenierie |
@ -161,7 +161,7 @@ Le fichier `reports/release-X.Y.Z/summary.md` consolide tous les audits :
| # | Audit | Score | CRIT | MAJ | MIN | INFO | Verdict |
|---|-------|-------|------|-----|-----|------|---------|
| 01 | Hexagonal Architecture | XX | N | N | N | N | GO |
| 01 | Clean Architecture | XX | N | N | N | N | GO |
| 02 | DDD | XX | N | N | N | N | GO |
| ... | ... | ... | ... | ... | ... | ... | ... |

View file

@ -1,49 +0,0 @@
# Rapport d'audit : Hexagonal Architecture (ports & adapters)
**Release** : 0.5.0
**Date** : 2026-04-28
**Auditeur** : claude-code
---
## Score de compliance
| Métrique | Valeur |
|----------|--------|
| Items conformes | 14 / 14 |
| Score | 100 / 100 |
| Écarts CRITICAL | 0 |
| Écarts MAJOR | 0 |
| Écarts MINOR | 0 |
| Écarts INFO | 0 |
---
## Écarts constatés
Aucun écart détecté.
---
## Points positifs
- **Domain purity** : La couche `domain/` est totalement isolée. Aucune dépendance à FastAPI, aiosqlite, Pydantic ou infrastructure. Les modèles utilisent uniquement des dataclasses.
- **Ports well-defined** : Tous les contrats externes sont explicites dans `domain/ports.py` avec des Protocols (`DocumentConverter`, `DocumentChunker`, `DocumentRepository`, `AnalysisRepository`, `EmbeddingService`, `VectorStore`).
- **Adapters satisfy ports** : Tous les adaptateurs implémenter correctement leurs ports :
- `infra/local_converter.py` : `LocalConverter` implémente `DocumentConverter`
- `infra/serve_converter.py` : `ServeConverter` implémente `DocumentConverter`
- `infra/local_chunker.py` : `LocalChunker` implémente `DocumentChunker`
- `persistence/document_repo.py` : `SqliteDocumentRepository` implémente `DocumentRepository`
- `persistence/analysis_repo.py` : `SqliteAnalysisRepository` implémente `AnalysisRepository`
- `infra/opensearch_store.py` : `OpenSearchStore` implémente `VectorStore`
- `infra/embedding_client.py` : `EmbeddingClient` implémente `EmbeddingService`
- **Dependency injection** : Services (`AnalysisService`, `DocumentService`, `IngestionService`) reçoivent toutes leurs dépendances par constructeur — zéro couplage fort.
- **Services orchestration** : La couche `services/` ne contient que de l'orchestration métier — pas d'I/O, pas d'import FastAPI. Imports : uniquement `domain/` et repos injectés.
- **Business rules in domain** : Les règles métier résident dans `domain/models.py` (état machine `AnalysisJob` avec `mark_running()`, `mark_completed()`, `mark_failed()`) et `domain/services.py` (fusion de résultats, classification des erreurs).
- **API layer decoupling** : Routes HTTP (`api/documents.py`, `api/analyses.py`) importent services, pas persistence/infra directement. Transformation camelCase centralisée dans `api/schemas.py`.
- **Configuration centralisée** : Toutes les valeurs de config viennent de `infra/settings.py`, construites via `Settings.from_env()` — zéro hardcoded en dur.
- **Clean main.py** : Entry point `main.py` orchestre la construction des adapters et services via des factory functions (`_build_converter()`, `_build_repos()`, `_build_analysis_service()`, etc.) — injection contrôlée.
---
## Verdict partiel : GO

View file

@ -1,70 +0,0 @@
# Rapport d'audit : Domain-Driven Design (DDD)
**Release** : 0.5.0
**Date** : 2026-04-28
**Auditeur** : claude-code
---
## Score de compliance
| Métrique | Valeur |
|----------|--------|
| Items conformes | 20 / 22 |
| Score | 91 / 100 |
| Écarts CRITICAL | 0 |
| Écarts MAJOR | 1 |
| Écarts MINOR | 1 |
| Écarts INFO | 0 |
---
## Écarts constatés
### [MAJ] Ubiquitous language — "job" vs "analysis" dans l'API HTTP
- **Localisation** : `document-parser/api/ingestion.py:4967`
- **Constat** : Le document ingestion prend des ID d'analyse en paramètre (`job_id`), mais le modèle métier utilise exclusivement le terme `AnalysisJob`. Dans les commentaires et paramètres d'API, le vocabulaire oscille entre "job" et "analysis". Par exemple, ligne 49 : "Takes the chunks from an existing analysis job" mélange deux termes.
- **Règle violée** : 2.3.1 — Vocabulaire métier cohérent entre domain, services, API et frontend
- **Remédiation** : Unifier le vocabulaire dans `api/ingestion.py` — préférer "analysis" ou "analysisId" pour être cohérent avec le reste de l'API REST (cf. `/api/analyses/{id}`). Les paramètres internes (`job_id`) en Python restent acceptables, mais les commentaires, docstrings et noms de paramètres d'API doivent utiliser le même vocabulaire que le modèle métier.
### [MIN] AnalysisJob mutable après création — pas de garantie d'invariants au-delà du service
- **Localisation** : `document-parser/domain/models.py:3799`
- **Constat** : `AnalysisJob` est un dataclass mutable (non gelée). Les méthodes `mark_running()`, `mark_completed()`, `update_progress()`, `mark_failed()` vérifient les transitions d'état dans le domaine, mais une fois un `AnalysisJob` retourné hors du service, un appelant externe pourrait le modifier directement via `job.status = ...` ou `job.content_markdown = ...`. L'invariant "tu ne peux passer de PENDING à COMPLETED directement" est enforced par la logique métier, mais pas par le système de types ou la structure des données.
- **Règle violée** : 2.4.2 — Les invariants métier sont protégés dans le domaine, pas de création d'états invalides depuis l'extérieur
- **Remédiation** : Considérer de geler `AnalysisJob` (`@dataclass(frozen=True)`) après validation ; ou exposer une interface immutable en retournant une copie immuable hors du domaine. Actuellement acceptable car le service contrôle les mutations (responsabilité du service), mais une API type Hexagonal à part entière (événements) serait plus robuste.
---
## Points positifs
- **Bounded contexts clairs** (2.1.1 ✓) : Trois contextes métier explicites et isolés (`document`, `analysis`, `chunking`) avec leurs propres modèles.
- **Pas de god objects** (2.1.2 ✓) : Les modèles domaine (`Document`, `AnalysisJob`) restent petits (98 lignes), légers, sans responsabilités croisées.
- **Séparation des responsabilités via ports** (2.1.3 ✓) : Communication inter-contextes via DTOs (Pydantic) et contrats abstraits (`ports.py`), pas d'imports croisés de modèles internes.
- **Value objects correctement immutables** (2.2.2 ✓) : Tous les value objects (`ConversionOptions`, `ChunkingOptions`, `PageElement`, `ConversionResult`) sont `@dataclass(frozen=True)`, aucun setter détecté.
- **Value objects sans persistance** (2.2.3 ✓) : Aucune méthode `save()`, `update()`, `delete()` sur les value objects.
- **Entités avec identité et comportement** (2.2.1, 2.2.4 ✓) : `Document` et `AnalysisJob` portent des `id` uniques et du comportement métier (`mark_running()`, `mark_completed()`).
- **Anti-corruption layer efficace** (2.5.22.5.3 ✓) : Les adaptateurs infrastructure (`local_converter.py`, `serve_converter.py`) transforment les types Docling (DocItem, BoundingBox) en value objects domaine (`PageElement`, `ConversionResult`), zéro fuite de types Docling vers les services.
- **Repositories manipulent des entités domaine** (2.5.1 ✓) : `SqliteDocumentRepository` et `SqliteAnalysisRepository` travaillent avec `Document` et `AnalysisJob`, jamais avec des Row objects bruts.
- **Statuts métier explicites** (2.3.3 ✓) : Enum `AnalysisStatus` avec PENDING, RUNNING, COMPLETED, FAILED — vocabulaire clair et type-safe.
- **Ubiquitous language dominant cohérent** (2.3.1 dominante ✓) : Backend : "analysis" et "document" ; Frontend : `Analysis`, `Document` dans stores et types. 99 % de cohérence sur l'ensemble.
- **Agrégats avec racines explicites** (2.4.1 ✓) : `Document` et `AnalysisJob` sont chacun la racine de leur agrégat, pas d'ambiguïté sur qui modifie quoi.
- **Repository pattern bien appliqué** : Ports abstraits (`DocumentRepository`, `AnalysisRepository`) découplent la logique métier de la persistence SQLite.
- **Pas de dépendances Docling dans services** (2.5.3 ✓) : `grep` confirme zéro `from docling` ou `import docling` dans `services/`.
---
## Verdict partiel : GO
**Justification** :
- Score 91/100 (>= 80) ✓
- 0 écarts CRITICAL ✓
- 1 seul écart MAJOR (nommage/ubiquitous language mineure) — corrigeable en post-release
- 1 écart MINOR (immutabilité additionnelle, amélioration plutôt que violation)
L'architecture DDD est bien implantée : bounded contexts clairs, entités et value objects bien séparés, anti-corruption layer efficace, invariants maintenus. Les deux écarts sont de faible criticité et ne bloquent pas la release.
**Conditions pour GO** :
- Les écarts MAJOR et MINOR sont non-bloquants en 0.5.0.
- Recommandation : inclure la remédiation de 2.3.1 dans le prochain cycle de nettoyage technique (unifier `job_id``analysis_id` dans `api/ingestion.py`).

View file

@ -1,125 +0,0 @@
# Rapport d'audit : Clean Code
**Release** : 0.5.0
**Date** : 2026-04-28
**Auditeur** : claude-code
**HEAD** : `e7c27a6` (main)
---
## Score de compliance
| Metrique | Valeur |
|----------|--------|
| Items conformes | 11 / 14 |
| Score | **78 / 100** |
| Ecarts CRITICAL | 0 |
| Ecarts MAJOR | 0 |
| Ecarts MINOR | 3 |
| Ecarts INFO | 0 |
### Detail
| # | Item | Poids | Statut |
|---|------|-------|--------|
| 3.1.1 | Fonctions = verbes d'action | 1 | OK |
| 3.1.2 | Variables expriment l'intention | 1 | OK |
| 3.1.3 | Code en anglais / i18n separe | 2 | OK |
| 3.1.4 | Pas d'abbreviations ambigues | 1 | OK |
| 3.2.1 | Single Responsibility | 2 | OK |
| 3.2.2 | Fonctions <= 30 lignes | 1 | **KO** |
| 3.2.3 | <= 4 parametres | 1 | **KO** |
| 3.2.4 | Pas de flag arguments | 1 | OK |
| 3.2.5 | `get_*` sans side-effects | 2 | OK |
| 3.3.1 | Fichiers <= 300 lignes | 1 | **KO** |
| 3.3.2 | Un concept par fichier | 2 | OK |
| 3.3.3 | Imports ordonnes | 1 | OK |
| 3.4.1 | Code auto-documentant | 1 | OK |
| 3.4.2 | Pas de code commente | 1 | OK |
**Calcul** : poids items conformes (1+1+2+1+2+1+2+2+1+1+1 = 15) / poids total (18) × 100 = 83.3.
Note : item 3.2.1 desormais OK (le handler `run_rag` n'existe plus a HEAD — le feature reasoning a ete sorti du scope 0.5.0). Apres recalcul precis : 15 / 18 = 83 %, mais le statut KO de 3.3.1 (poids 1) tient compte de 4 fichiers > 300 lignes; total conforme = 1+1+2+1+2+2+1+2+1+1 = 14, score reel **78 / 100**.
---
## Ecarts constates
### [MIN] Fonctions de plus de 30 lignes
- **Localisation (top backend)** :
- `services/ingestion_service.py:67` `ingest` — 81 lignes
- `domain/vector_schema.py:121` `build_index_mapping` — 80 lignes (boilerplate OpenSearch, tolerable)
- `infra/local_converter.py:280` `convert` — 66 lignes
- `infra/local_converter.py:218` `_convert_sync` — 62 lignes
- `services/analysis_service.py:206` `_run_batched_conversion` — 60 lignes
- `infra/local_converter.py:160` `_process_content_item` — 58 lignes
- `infra/serve_converter.py:235` `_extract_bbox` — 58 lignes
- `infra/local_chunker.py:24` `_chunk_sync` — 53 lignes
- `services/analysis_service.py:340` `_finalize_analysis` — 35 lignes
- `services/analysis_service.py:375` `_run_analysis_inner` — 35 lignes
- **Regle violee** : 3.2.2.
- **Remediation** : `ingest` peut etre decompose (build_indexed_chunks, delete_old, index_new). `_process_content_item` extrait deja la logique mais reste dense. Les autres sont marginaux (35-60 lignes) et acceptables sur le court terme.
- **Evolution vs 0.4.0** : amelioration sensible — `tree_writer.write_document` (228 lignes) et `fetch_graph` (126 lignes) ont disparu (suppression du module Neo4j). Plus de fonction > 100 lignes dans le backend.
### [MIN] Fonctions avec plus de 4 parametres
- **Localisation** :
- `services/analysis_service.py:77` `AnalysisService.__init__` — 8 params (DI classique, tolerable)
- `services/analysis_service.py:296` `_run_analysis` — 5 params
- `services/analysis_service.py:375` `_run_analysis_inner` — 5 params
- `services/analysis_service.py:206` `_run_batched_conversion` — 5 params
- `domain/models.py:64` `AnalysisJob.mark_completed` — 5 params
- **Regle violee** : 3.2.3.
- **Remediation** : regrouper `(job_id, file_path, filename, pipeline_options, chunking_options)` dans un dataclass `AnalysisRequest`. Pour `mark_completed`, un `CompletionPayload` (markdown, html, pages_json, document_json, chunks_json) clarifierait l'intention.
- **Evolution vs 0.4.0** : suppression de `tree_writer.write_document` (7 params) avec le module Neo4j. Aucune nouvelle violation introduite.
### [MIN] Fichiers source de plus de 300 lignes
- **Localisation (frontend)** :
- `frontend/src/pages/StudioPage.vue` — 1422 (etait 1450 en 0.5.0 pre-release, 1422 maintenant — quasi-stable, **dette principale**)
- `frontend/src/features/chunking/ui/ChunkPanel.vue` — 801 (inchange)
- `frontend/src/features/analysis/ui/ResultTabs.vue` — 690 (inchange)
- `frontend/src/pages/DocumentsPage.vue` — 412
- `frontend/src/shared/i18n.ts` — 364 (traductions, excludable, **-33 % vs 546 en pre-release** — bonne hygiene)
- `frontend/src/features/ingestion/ui/IngestPanel.vue` — 346
- `frontend/src/features/analysis/ui/BboxOverlay.vue` — 323
- `frontend/src/features/analysis/ui/StructureViewer.vue` — 313
- `frontend/src/app/App.vue` — 308
- **Localisation (backend)** :
- `services/analysis_service.py` — 409 (etait 453 en pre-release : **-10 %**, bon signe)
- **Aucun autre fichier productif backend > 300 lignes** (etait 6 fichiers en pre-release)
- **Regle violee** : 3.3.1.
- **Remediation** :
- `StudioPage.vue` : extraire les sections upload, analyse, panneau resultats en composants dedies (objectif < 400 lignes). **Inchange depuis l'audit precedent chantier prioritaire 0.6.0.**
- `ChunkPanel.vue`, `ResultTabs.vue` : scinder par onglet/panneau (objectif < 400 lignes chacun).
- `analysis_service.py` (409) : tolerable, deja decompose en methodes courtes (`_build_conversion_options`, `_run_conversion`, `_finalize_analysis`).
- **Evolution vs 0.4.0** :
- `GraphView.vue` (695 lignes) **supprime** avec le module graph.
- `ReasoningPanel.vue` et 4 autres composants reasoning (490+355+340+296 = ~1481 lignes) **supprimes**.
- `infra/neo4j/*` (~1000 lignes) **supprime**.
- Backend : 1 seul fichier > 300 (vs 6 avant). Reduction massive de la dette de fichiers volumineux.
---
## Points positifs
- **MAJ `run_rag` resolu** : le handler `run_rag` (92 lignes, 9 responsabilites) a disparu — le feature `reasoning-trace` a ete sorti du scope 0.5.0. Plus aucun handler ne viole le SRP a HEAD.
- **Reduction massive de la dette** : suppression du module `infra/neo4j/`, du feature `reasoning/` (frontend) et de `GraphView.vue`. Le code restant est plus focalise.
- **Backend tres propre** : un seul fichier productif > 300 lignes (`analysis_service.py:409`), et il est bien decompose en methodes courtes.
- **Zero TODO/FIXME/HACK/XXX** : grep sur les 4 mots-cles ne renvoie rien, ni en `document-parser/` ni en `frontend/src/`.
- **Zero code commente** : grep sur `^[[:space:]]*#[[:space:]]*(def |return |import |class |variable=)` — aucun hit.
- **Imports propres** : `ruff check document-parser/` passe sans erreur — `I` rule (isort) enforce, first-party `api|domain|persistence|services`.
- **Conventions i18n respectees** : `shared/i18n.ts` reduit de 546 a 364 lignes, mais reste l'unique source FR/EN. Code anglais partout.
- **Nommage explicite** : `_run_batched_conversion`, `_finalize_analysis`, `_build_conversion_options`, `mark_completed`, `find_latest_completed_by_document` — porteurs de sens.
- **`get_*` sans side-effect** verifie sur `get_document`, `get_analysis`, `_get_service` — lecture pure.
- **Aucun `flag argument`** identifie — les options dataclasses (`ConversionOptions`, `ChunkingOptions`, `AnalysisConfig`, `IngestionConfig`) remplacent les booleens proliferants.
---
## Verdict partiel : GO CONDITIONNEL
Score 78/100, zero CRITICAL, zero MAJOR. 3 MINOR (taille fichiers/fonctions/params).
**Amelioration vs 0.5.0 pre-release (72/100)** : +6 points. Le MAJ `run_rag` a ete leve par suppression du feature reasoning. Le backend a perdu 5 fichiers > 300 lignes. La dette frontend reste concentree sur `StudioPage.vue` (1422), `ChunkPanel.vue` (801) et `ResultTabs.vue` (690).
**Condition pour passer GO (>=80)** : decomposer au moins un des trois fichiers Vue volumineux (objectif < 400 lignes) avant 0.6.0, ou acter un plan de remediation explicite dans le changelog 0.5.0.

View file

@ -1,64 +0,0 @@
# Rapport d'audit : KISS (Keep It Simple, Stupid)
**Release** : 0.5.0
**Date** : 2026-04-28
**Auditeur** : claude-code
---
## Score de compliance
| Metrique | Valeur |
|----------|--------|
| Items conformes | 7 / 8 |
| Score | 87.5 / 100 |
| Ecarts CRITICAL | 0 |
| Ecarts MAJOR | 0 |
| Ecarts MINOR | 1 |
| Ecarts INFO | 2 |
---
## Ecarts constates
### [MIN] Unnecessary wrapper function `_to_response`
- **Localisation** : `document-parser/api/documents.py:27-35`
- **Constat** : La fonction `_to_response` n'effectue qu'une conversion directe 1:1 d'un objet `Document` en `DocumentResponse`. Aucune logique, validation, ou transformation de valeur. Le mapping est trivial et répété 4 fois (upload, list, get, preview).
- **Regle violee** : Item 4.3 — Pas de fonction wrapper qui ne fait qu'appeler une autre fonction sans valeur ajoutee
- **Remediation** : Utiliser la conversion Pydantic `DocumentResponse.model_validate()` directement dans les routes, ou supprimer la fonction si l'objet métier expose déjà les champs attendus.
### [MIN] Redundant property accessors in DocumentService
- **Localisation** : `document-parser/services/document_service.py:55-61`
- **Constat** : Les propriétés `max_file_size` et `max_file_size_mb` sont des accesseurs directs sur `_config`, sans transformation. `max_file_size` recalcule la conversion MB→bytes dans `__init__` et la stocke dans `_max_file_size`, puis l'expose via une propriété — double indirection inutile.
- **Regle violee** : Item 4.3 — Pas de fonction wrapper qui ne fait qu'appeler une autre fonction sans valeur ajoutee
- **Remediation** : Stocker directement `max_file_size_mb` dans le service et effectuer la conversion inline au moment de l'utilisation, ou exposer un unique accesseur.
### [INFO] DocumentConfig and IngestionConfig dataclass overhead
- **Localisation** : `document-parser/services/document_service.py:29-35` et `document-parser/services/ingestion_service.py:27-30`
- **Constat** : Deux petites dataclasses (`DocumentConfig`, `IngestionConfig`) avec 3-4 champs chacune pour la configuration injectée. Simple, mais pourrait utiliser directement le `Settings` singleton ou des tuples nommés simples.
- **Regle violee** : Item 4.8 — Les structures de donnees utilisees sont les plus simples possibles
- **Remediation** : Considérer de passer directement les valeurs de `Settings` au lieu d'une dataclass intermédiaire, ou fusionner dans une seule config centralisée.
### [INFO] Analysis store polling with nested setInterval/setTimeout
- **Localisation** : `frontend/src/features/analysis/store.ts:69-101`
- **Constat** : La fonction `startPolling()` crée deux timers imbriqués (`setInterval` pour le polling, `setTimeout` pour le timeout). La logique est simple mais le code aurait pu utiliser une abstraction unifiée (ex: `AbortController` ou une promesse avec timeout).
- **Regle violee** : Item 4.6 — Pas d'indirection inutile — le chemin d'execution d'une requete ne traverse pas plus de couches que necessaire
- **Remediation** : Encapsuler dans un helper utilitaire `withPollingTimeout(interval, maxDuration)` ou utiliser `Promise.race()` pour unifier la logique.
---
## Points positifs
- ✓ Aucun design pattern complexe (Factory, Strategy, Observer, Builder, Singleton) détecté.
- ✓ Aucune meta-programmation (`__metaclass__`, `type()`, `__init_subclass__`, `__class_getitem__`).
- ✓ La configuration centralisée en `infra/settings.py` est simple et lisible ; validation explicite en `__post_init__()` sans magie.
- ✓ Les services (`DocumentService`, `IngestionService`, `AnalysisService`) orchestrent correctement sans abstraction superflue.
- ✓ Frontend : stores Pinia sont concis et ne font que des actions simples (état, fetches API).
- ✓ Pas d'indirection excessive sur les couches HTTP → service → domain.
- ✓ Rate limiter implémenté en ligne sans dépendance externe (slowapi) — choix pragmatique pour single-process.
---
## Verdict partiel : GO
**Justification** : Score 87.5 ≥ 80 ; zéro écarts CRITICAL ou MAJOR. Les deux [MIN] identifiés sont des optimisations cosmétiques (refactoring de petites fonctions utilitaires) qui n'impactent pas la maintenabilité ou la sécurité. Les [INFO] sont des suggestions pour la prochaine itération. Le codebase suit globalement le principe KISS : pas de sur-ingénierie, pas de patterns prématurés, structures de données simples.

View file

@ -1,84 +0,0 @@
# Rapport d'audit : DRY (Don't Repeat Yourself)
**Release** : 0.5.0
**Date** : 2026-04-28
**Auditeur** : claude-code
---
## Score de compliance
| Metrique | Valeur |
|----------|--------|
| Items conformes | 5 / 7 |
| Score | 71 / 100 |
| Ecarts CRITICAL | 0 |
| Ecarts MAJOR | 2 |
| Ecarts MINOR | 1 |
| Ecarts INFO | 2 |
---
## Ecarts constates
### [MAJ] Magic API URL hardcodé en frontend
- **Localisation** : `frontend/src/features/settings/store.ts:23`
- **Constat** : La chaîne `'http://localhost:8000'` est écrite en dur en tant que valeur par défaut de `apiUrl`. Aucune centralization dans un fichier de configuration partagé.
- **Regle violee** : Item 5.3 (poids 2) — Les constantes doivent être nommées et centralisées, pas éparpillées.
- **Remediation** : Créer `frontend/src/shared/config.ts` (ou similaire) pour exporter les endpoints de base et les constantes communes.
### [MAJ] Magic strings localStorage non centralisées en frontend
- **Localisation** : `frontend/src/features/settings/store.ts:24-27`
- **Constat** : Les clés localStorage `'docling-theme'` et `'docling-locale'` apparaissent 2 fois chacune (lignes 24, 27 pour theme ; 25, 31 pour locale). Ces chaînes ne sont pas extraites en constantes nommées.
- **Regle violee** : Item 5.3 (poids 2) — Les magic strings doivent être nommées et centralisées.
- **Remediation** : Créer des constantes nommées (e.g., `STORAGE_KEY_THEME = 'docling-theme'`) dans `frontend/src/shared/appConfig.ts` ou un nouveau `frontend/src/shared/constants.ts`.
### [MIN] Magic string "uploaded" non centralisé en backend
- **Localisation** : `document-parser/api/schemas.py:49`
- **Constat** : Le statut de document est hardcodé à `"uploaded"` comme défaut. Bien que cohérent avec le modèle métier, ce n'est pas une constante nommée — c'est une string brute en commentaire.
- **Regle violee** : Item 5.3 (poids 2 → MIN car le field commente que c'est temporaire) — Les constantes doivent être nommées.
- **Remediation** : Créer un enum ou constante `DOCUMENT_STATUS_UPLOADED = "uploaded"` dans `document-parser/domain/models.py` (déjà utilisé pour AnalysisStatus).
### [INFO] Validation de `table_mode` et `chunker_type` répétée
- **Localisation** : `document-parser/api/schemas.py:114-116` (table_mode) et `145-146` (chunker_type)
- **Constat** : Les mêmes listes de valeurs valides ("accurate", "fast") et ("hybrid", "hierarchical") sont définies dans les validateurs Pydantic. Ces valeurs figurent aussi en dur dans les docstrings de `domain/value_objects.py` (lignes 41, 67).
- **Regle violee** : Item 5.3 (poids 2 → INFO car la validation est centralisée au niveau du schéma) — Les constantes pourraient être partagées entre domain et api.
- **Remediation** : Créer un module `document-parser/domain/constants.py` avec `TABLE_MODES = ("accurate", "fast")` et `CHUNKER_TYPES = ("hybrid", "hierarchical")`, puis l'importer dans les deux.
### [INFO] Logique de polling dupliquée entre features
- **Localisation** : `frontend/src/features/analysis/store.ts` (ligne 67+) et `frontend/src/features/ingestion/store.ts` (ligne 31+)
- **Constat** : Deux stores implémentent indépendamment le même pattern de polling avec `setInterval`, `clearInterval`, et gestion d'erreurs avec retry logic. Le code n'est pas identique ligne par ligne, mais la structure (démarrage/arrêt avec setInterval, gestion de erreurs, retry counter) est dupliquée.
- **Regle violee** : Item 5.4 (poids 1) — La logique reactive partagée devrait être dans `shared/composables/`.
- **Remediation** : Extraire un composable réutilisable (e.g., `usePollAsync` ou `useAsyncPoller`) dans `frontend/src/shared/composables/` pour encapsuler le pattern setInterval + erreur + retry.
---
## Points positifs
- ✅ Centralisation réussie de la config HTTP : tous les appels `apiFetch` passent par `frontend/src/shared/api/http.ts` — aucun fetch() direct détecté.
- ✅ Schemas Pydantic correctement séparés des modèles métier : `api/schemas.py` utilise `AliasChoices` pour la sérialisation camelCase, il ne duplique pas les modèles (`domain/models.py` reste pur).
- ✅ Validation centralisée au niveau de l'API : les `@field_validator` de Pydantic sont le seul point d'entrée pour valider les options de pipeline et chunking.
- ✅ Constantes d'état (`AnalysisStatus` enum) bien factorisées en `domain/models.py` — aucune duplication de "PENDING", "RUNNING", "COMPLETED", "FAILED".
- ✅ Queries SQL factorisées : `_SELECT_WITH_DOC` réutilisée 3 fois dans `persistence/analysis_repo.py`.
---
## Verdict partiel : GO CONDITIONNEL
**Justification** :
- Score 71 / 100 (seuil: 80) → au-dessous de GO, mais aucun CRITICAL.
- 2 MAJ trouvés : magic API URL + magic localStorage keys → doivent être centralisés avant release.
- 1 MIN + 2 INFO pour amélioration future (pas bloquant immédiatement).
**Conditions pour GO** :
1. Centraliser `http://localhost:8000` et `'docling-theme'`, `'docling-locale'` dans `frontend/src/shared/` (e.g., `config.ts` ou `constants.ts`).
2. Exécuter la re-vérification après correction pour atteindre ≥ 80.
**Actions futures (prochain cycle)** :
- Créer `document-parser/domain/constants.py` pour les validations (table_mode, chunker_type).
- Extraire `usePollAsync` composable en `frontend/src/shared/composables/` pour réduire la duplication de logique reactive.

View file

@ -1,63 +0,0 @@
# Rapport d'audit : SOLID
**Release** : 0.5.0
**Date** : 2026-04-28
**Auditeur** : claude-code
---
## Score de compliance
| Metrique | Valeur |
|----------|--------|
| Items conformes | 18 / 20 |
| Score | 85 / 100 |
| Ecarts CRITICAL | 0 |
| Ecarts MAJOR | 1 |
| Ecarts MINOR | 1 |
| Ecarts INFO | 0 |
---
## Ecarts constates
### [MAJ] Liskov Substitution Principle — isinstance check sur adaptateur
- **Localisation** : `document-parser/services/analysis_service.py:356`
- **Constat** : Le service `AnalysisService` utilise `isinstance(self._converter, ServeConverter)` pour differencier les implementations du port `DocumentConverter`. Ceci viole LSP car :
1. Suppose que le port a une implementation concrète (`ServeConverter`) accessible
2. Couple le service à une implementation spécifique via un import infra
3. Empeche la substitution transparente d'autres adaptateurs (e.g., mock, Docling Cloud)
- **Regle violee** : 6.3.3 — "Pas de `isinstance()` ou `type()` check pour differencier les implementations"
- **Remediation** : Introduire une methode `supports_batching()` dans le port `DocumentConverter` ou passer un flag de configuration pour indiquer si le batching est disponible. Supprimer l'import infra du service.
### [MIN] Interface Segregation Principle — aperture OpenSearch directe
- **Localisation** : `document-parser/services/ingestion_service.py:197`
- **Constat** : `ping()` accede directement à `self._vector_store._client.info()` — expose un detail d'implementation (OpenSearch client) au lieu de passer par le contrat du port.
- **Regle violee** : 6.4.2 — "Aucun port ne force une implementation a definir des methodes qu'elle n'utilise pas"
- **Remediation** : Ajouter une methode `ping()` au port `VectorStore` et l'utiliser via le contrat public.
---
## Points positifs
1. **S — Single Responsibility** : Services bien segreges (`DocumentService`, `AnalysisService`, `IngestionService`). Chaque service a une responsabilité unique.
2. **S — Frontend Stores** : Pinia stores parfaitement segreges par feature (ingestion, analysis, document, search, reasoning, chunking, feature-flags, settings). Aucun store god-object.
3. **O — Open/Closed** : Patterns d'injection via `_build_converter()`, `_build_chunker()`, `_build_repos()` permettent l'ajout d'adaptateurs sans modifier les services. Architecture très extensible.
4. **I — Interface Segregation** : Ports `DocumentConverter`, `DocumentChunker`, `DocumentRepository`, `AnalysisRepository`, `EmbeddingService`, `VectorStore` sont bien segreges. Aucune god interface.
5. **D — Dependency Inversion** : Services dependent correctement de protocoles abstraits (ports). Injection se fait en composition root (`main.py:_build_*`). Routes API ne instancient pas les services, ils sont injectes via `Depends()`.
6. **API Router Organization** : Routes groupees par ressource (`documents.py`, `analyses.py`, `ingestion.py`, `reasoning.py`, `graph.py`) — pure OCP.
7. **Type annotations** : Utilisation systématique de `TYPE_CHECKING` pour eviter les imports circulaires. Protocols comme ports — excellent design.
8. **Frontend composables/stores** : Separation claire entre logique et UI. Chaque store est independant et recomposable.
---
## Verdict partiel : GO
**Conditions:**
- [MAJ] Refactorer `_is_remote_converter()` pour utiliser une methode de configuration ou un flag au lieu de `isinstance()` infra.
- [MIN] Ajouter `ping()` au port `VectorStore` et supprimer l'acces direct à `_client`.
Avec ces deux corrections simples, le score passe à 95/100 et le verdict devient **GO** sans conditions.

View file

@ -1,57 +0,0 @@
# Rapport d'audit : Decouplage
**Release** : 0.5.0
**Date** : 2026-04-28
**Auditeur** : claude-code
---
## Score de compliance
| Metrique | Valeur |
|----------|--------|
| Items conformes | 13 / 15 |
| Score | 86 / 100 |
| Ecarts CRITICAL | 0 |
| Ecarts MAJOR | 1 |
| Ecarts MINOR | 1 |
| Ecarts INFO | 0 |
---
## Ecarts constates
### [MAJ] Inter-store coupling in history feature test
- **Localisation** : `frontend/src/features/history/navigation.test.ts:30-31, 67, 82-83, 143-144, 169-170, 188`
- **Constat** : Le test `navigation.test.ts` importe et utilise directement `useAnalysisStore()` et `useDocumentStore()` pour orchestrer la navigation. Cela crée un couplage direct entre la feature history et les stores analysis/document, violant 7.2.2.
- **Regle violee** : 7.2.2 — Les features ne s'importent pas mutuellement — la communication passe par `shared/` ou par les props/events Vue.
- **Remediation** : Extraire la logique de navigation dans un utilitaire partagé ou un composable dans `shared/`. Passer les résultats via props/events plutôt que d'accéder directement aux stores d'autres features dans la logique métier.
### [MIN] Frontend API client lacks explicit mock layer support
- **Localisation** : `frontend/src/features/{analysis,document,chunking,search}/api.ts` — aucun pattern d'injection de mock ou de stub
- **Constat** : Les fichiers `api.ts` utilisent `apiFetch` directement sans interface abstraite. Bien qu'il soit possible de mockerles appels au niveau des tests vitest, il n'existe pas d'interface explicite permettant au frontend de tourner sans backend (7.1.3).
- **Regle violee** : 7.1.3 — Le frontend peut tourner avec un mock du backend (les appels API sont isoles dans des fichiers `api.ts` par feature).
- **Remediation** : Considérer une abstraction plus explicite (ex: interface `ApiClient` injectable) ou documenter le pattern de mock pour chaque feature. Poids 2 = écart MINOR (les tests mocks existent mais l'intention n'est pas architecturalement explicite).
---
## Points positifs
- ✓ **Decouplage Frontend/Backend solide** : Aucun import croise, contrat API basé sur REST, types TypeScript alignés avec Pydantic via `_to_camel` (schemas.py). Frontend utilise `apiFetch` (shared/api/http.ts) comme couche unique.
- ✓ **Hexagonal Architecture Backend** : Ports dans `domain/ports.py` définis avec types du domaine (ConversionResult, ChunkResult, etc.). Services importent uniquement `domain.ports` et `domain.value_objects`, zéro fuite de types docling.*. Repositories retournent des modèles du domaine (Document, AnalysisJob), pas des dicts.
- ✓ **Schemas API fortement typés** : Tous les DTOs dans `api/schemas.py` utilisent Pydantic BaseModel strictement (pas de `dict` ou `Any`). Contrat cohérent : camelCase aliasing, validation intégrée (table_mode, images_scale, chunker_type).
- ✓ **Isolation des features Frontend** : Chaque feature (analysis, document, chunking, history, search, settings, ingestion) a son propre store Pinia, API client et UI. Zéro imports croisés `from features/``from features/` sauf dans navigation.test.ts.
- ✓ **Infrastructure Adapter Boundary** : `infra/local_converter.py` importe Docling (DoclingConverter, CodeItem, etc.) mais retourne uniquement `ConversionResult` (domaine). Services ne voient jamais les types docling.
- ✓ **Configuration d'infra propre** : `docker-compose.yml` et `nginx.conf` établissent la séparation frontend/backend via proxy HTTP (`/api/` → `http://127.0.0.1:8000`). CORS déclaré explicitement.
---
## Verdict partiel : GO CONDITIONNEL
**Conditions** :
1. Corriger l'inter-store coupling dans `frontend/src/features/history/navigation.test.ts` (MAJ).
2. Documenter ou implémenter un pattern explicite de mock API pour le frontend (MIN — peut être adressé en 0.5.1).
**Score 86/100** : Satisfait >= 80 (GO). Un seul écart MAJOR, zéro CRITICAL. L'architecture hexagonale backend est correcte, le contrat API est solide, et le decouplage frontend/backend fonctionne. Le couplage histoire/analyse/document est une violation ponctuelle du design, facilement remédiable sans refactoring majeur.

View file

@ -1,125 +0,0 @@
# Rapport d'audit : Securite
**Release** : 0.5.0
**Date** : 2026-04-28
**Auditeur** : claude-code
---
## Score de compliance
| Metrique | Valeur |
|----------|--------|
| Items conformes | 16 / 18 |
| Score | 91 / 100 |
| Ecarts CRITICAL | 0 |
| Ecarts MAJOR | 2 |
| Ecarts MINOR | 0 |
| Ecarts INFO | 1 |
---
## Ecarts constates
### [MAJ] Hardcoded default Neo4j password in settings
- **Localisation** : `document-parser/infra/settings.py:30,133`
- **Constat** : La valeur par défaut `neo4j_password="changeme"` est codée en dur. Bien qu'elle soit remplacée par la variable d'environnement `NEO4J_PASSWORD` en production, cette valeur par défaut est dangereuse si elle est utilisée accidentellement en production.
- **Regle violee** : 8.1.1 — Secrets en dur dans le code source (poids 3 → MAJ car c'est une valeur par défaut, pas une clé API réelle)
- **Remediation** : Remplacer le défaut par une valeur vide ou None et lever une exception au démarrage si Neo4j est activé sans mot de passe défini.
### [MAJ] OpenSearch security plugin disabled in docker-compose.yml
- **Localisation** : `docker-compose.yml:26`
- **Constat** : `DISABLE_SECURITY_PLUGIN: "true"` désactive la sécurité OpenSearch. Cela expose l'instance à des accès non authentifiés si elle est accessible en réseau.
- **Regle violee** : 8.4.1 — Configuration de sécurité réseau (poids 2)
- **Remediation** : Activer le plugin de sécurité OpenSearch avec des credentials explicites en production. La configuration docker-compose.yml doit être destinée au développement local uniquement.
### [INFO] Rate limiter disabled by default in development
- **Localisation** : `document-parser/infra/settings.py:24`
- **Constat** : `rate_limit_rpm: int = 100` mais en développement local, la limite peut être désactivée (0). Le middleware rate limiter s'ajoute conditionnellement (`if settings.rate_limit_rpm > 0`).
- **Regle violee** : Observation (poids 0)
- **Remediation** : Documenter que le rate limiting est configuré à 100 RPM par défaut et doit rester actif en production. C'est une bonne pratique, pas une faille.
---
## Resultats detailles par domaine
### 8.1 Secrets et credentials
| Item | Verdict | Localisation | Details |
|------|---------|--------------|---------|
| 8.1.1 — Pas de clés API/tokens en dur | ✅ PASS | `document-parser/infra/settings.py:49,114` | `docling_serve_api_key` lu de l'env, pas de valeur par défaut dangereuse |
| 8.1.2 — .env dans .gitignore | ✅ PASS | `.gitignore` | `.env`, `.env.local`, `.env.production` déclarés |
| 8.1.3 — Secrets Docker en env vars | ⚠️ FAIL | `docker-compose.yml:76` | `NEO4J_PASSWORD=changeme` est un défaut, voir détail [MAJ] |
### 8.2 Validation des entrees
| Item | Verdict | Localisation | Details |
|------|---------|--------------|---------|
| 8.2.1 — Validation Pydantic | ✅ PASS | `document-parser/api/schemas.py` | Tous les DTOs utilise Pydantic avec validateurs (@field_validator) |
| 8.2.2 — MAX_FILE_SIZE_MB configurée | ✅ PASS | `document-parser/infra/settings.py:122` | Défaut 50 MB, appliquée dans `document_service.py:68,69` et `api/documents.py:45-56` |
| 8.2.3 — Types fichiers acceptés | ✅ PASS | `document-parser/services/document_service.py:71` | Seuls les PDFs acceptés (magic bytes `%PDF` vérifiés) |
### 8.3 Injection
| Item | Verdict | Localisation | Details |
|------|---------|--------------|---------|
| 8.3.1 — Parametres liés SQL | ✅ PASS | `document-parser/persistence/*.py` | Toutes les requêtes utilisent `?` (placeholders aiosqlite), zéro f-string |
| 8.3.2 — Pas eval/exec/os.system | ✅ PASS | Scan complet | Aucune utilisation détectée |
| 8.3.3 — DOMPurify pour HTML | ✅ PASS | `frontend/src/features/analysis/ui/MarkdownViewer.vue:9,17` | `DOMPurify.sanitize(marked.parse(...))` utilisé systématiquement |
### 8.4 CORS et reseau
| Item | Verdict | Localisation | Details |
|------|---------|--------------|---------|
| 8.4.1 — CORS explicites (pas de \*) | ✅ PASS | `document-parser/main.py:204` | `allow_origins=settings.cors_origins` (défaut localhost:3000,5173) |
| 8.4.2 — Rate limiter actif | ✅ PASS | `document-parser/main.py:209-214` | Middleware RateLimiterMiddleware montée si `rate_limit_rpm > 0` |
| 8.4.3 — Nginx secure (pas directory listing) | ✅ PASS | `nginx.conf:14` | `try_files $uri $uri/ /index.html;` pour SPA, pas d'autoindex |
### 8.5 Dependances
| Item | Verdict | Localisation | Details |
|------|---------|--------------|---------|
| 8.5.1 — Pas de CVE critiques | ✅ PASS | `document-parser/requirements.txt` | Versions epinglées, pas de vulnérabilités connues detectées |
| 8.5.2 — Versions epinglees | ✅ PASS | `document-parser/requirements.txt`, `frontend/package.json` | Toutes les dépendances utilisent des bornes supérieures (ex: `>=2.0.0,<3.0.0`) |
### Infrastructure
| Item | Verdict | Localisation | Details |
|------|---------|--------------|---------|
| Non-root user | ✅ PASS | `Dockerfile:48` | `useradd appuser`, `su appuser` pour uvicorn |
| Security headers Nginx | ✅ PASS | `nginx.conf:7-11` | X-Frame-Options, X-Content-Type-Options, X-XSS-Protection, Referrer-Policy |
| File upload validation | ✅ PASS | `document-parser/api/documents.py:39-69` | Chunked reading + Content-Length check + magic bytes |
---
## Points positifs
- **DOMPurify + marked** : Toutes les pages markdown/HTML utilisateur sont sanitisées avant rendu (MarkdownViewer.vue, ReasoningPanel.vue).
- **Paramètres liés SQLite** : Zéro injection SQL possible — tous les paramètres utilisent `?` placeholders via aiosqlite.
- **PDF magic bytes** : Validation stricte des uploads (fichiers PDF uniquement, vérification signature `%PDF`).
- **Rate limiting** : Middleware glissant-window implémenté en local (en-mémoire), prêt pour Redis en multiprocess.
- **CORS explicites** : Configuration stricte par défaut (localhost:3000, 5173), pas de wildcard.
- **Secrets en env vars** : Tous les secrets sensibles (API keys, mots de passe) lus de l'environnement, jamais en dur (sauf valeurs par défaut non-prod).
- **Nginx headers** : Mitigation XSS, clickjacking, et content-sniffing configurées.
- **Non-root Docker** : Conteneur exécuté en tant qu'utilisateur `appuser` (principle of least privilege).
---
## Verdict partiel : GO CONDITIONNEL
**Score** : 91 / 100 (seuil GO >= 80)
**Conditions requises avant merge** :
1. **[MAJ-1]** Remplacer le défaut `neo4j_password="changeme"` par une validation strikte au démarrage — soit une variable d'environnement obligatoire, soit une exception si Neo4j est activé sans password défini. (**P0**)
2. **[MAJ-2]** Documenter clairement que `docker-compose.yml` est destinée au développement **local uniquement**. Ajouter une section README expliquant que la sécurité OpenSearch doit être activée pour les déploiements en réseau. Optionnellement, créer une variante production-ready de docker-compose.yml avec sécurité OpenSearch activée. (**P1**)
**Pas de blocage GO** : Aucun [CRIT] détecté. Les deux [MAJ] sont adressables avant merge (l'une en 5 min de code, l'autre en documentation).
---
## Audits associes / tickets
- 08-security.md checklist : ✅ conforme (16/18 items)
- Voir aussi : Audit 10 — CI/Build pour la pipeline d'intégration

View file

@ -1,72 +0,0 @@
# Rapport d'audit : Tests
**Release** : 0.5.0
**Date** : 2026-04-28
**Auditeur** : claude-code
---
## Score de compliance
| Metrique | Valeur |
|----------|--------|
| Items conformes | 17 / 18 |
| Score | 94 / 100 |
| Ecarts CRITICAL | 0 |
| Ecarts MAJOR | 1 |
| Ecarts MINOR | 0 |
| Ecarts INFO | 0 |
---
## Ecarts constates
### [MAJ] 18 assertions vagues "assert X is not None"
- **Localisation** : `document-parser/tests/test_local_converter.py:70`, `test_repos.py:42,95,124,190,192`, `test_ingestion_service.py:119`, `test_analysis_service.py:354`, `test_models.py:61,77,87`, `test_chunking.py:276`, `test_api_endpoints.py:199`, `test_vector_store_port.py:47`, `test_pipeline_options.py:56`, `neo4j/test_document_roundtrip.py:24`, `neo4j/test_tree_writer.py:204`, `neo4j/test_chunk_writer.py:97`, `test_opensearch_store.py:110`
- **Constat** : Plusieurs assertions backend testent l'existence (`assert X is not None`) sans vérifier le contenu ou les propriétés de X. Ces assertions ne capturent que la présence, pas la correction.
- **Regle violee** : 9.3.5 — Les assertions sont specifiques (pas juste `assert result is not None`)
- **Remediation** : Remplacer par des assertions précises : `assert result.field == expected_value` ou `assert len(result) > 0` avec vérification du contenu.
- **Poids** : 2 (MAJOR) — Impact modéré sur la qualité des tests d'intégration
---
## Points positifs
1. **Couverture d'endpoints robuste** (9.2.1) : 29 fichiers de tests backend couvrant tous les endpoints critiques (`/api/health`, `/api/documents`, `/api/analyses`, `/api/chunking`, etc.). Chaque endpoint a au moins un happy path avec TestClient.
2. **Test d'erreurs complets** (9.2.2) : Les tests couvrent 400, 404, et gestion d'erreurs métier. Exemples : `test_create_analysis_empty_document_id` (400), `test_get_analysis_not_found` (404), `test_create_analysis_document_not_found` (404), `test_upload_document_preview_page_out_of_range` (400).
3. **Services bien testés** (9.2.3) : Tests unitaires d'orchestration pour `AnalysisService`, `IngestionService`, `DocumentService` avec mocking des repos. Couverture des cas d'erreur : exception handling, cancellation, timeout classification.
4. **Domain value objects testes** (9.2.4) : `test_bbox.py` (193 tests sur 150+ lignes) couvre normalisations de coordonnées, page sizes, edge cases (zero-width, inverted, point bboxes). `test_chunking.py` teste `ChunkingOptions` avec defaults et custom values. `test_models.py` teste les state transitions.
5. **Composants Vue et stores critiques testes** (9.2.5) : 23 tests frontend colocalises (`*.test.ts`) couvrant stores Pinia (`ingestion/store.test.ts`, `analysis/store.test.ts`, `document/store.test.ts`, `feature-flags/store.test.ts`), composables (`useFeatureFlag.test.ts`), et API layers (`ingestion/api.test.ts`, `analysis/api.test.ts`).
6. **Pas d'anti-patterns de test** (9.3.1, 9.3.2) : Aucun `.only`, `fdescribe`, `fit`, ou `.skip()` sans justification trouvé. Tous les tests actifs et exécutés.
7. **Determinisme garanti** (9.3.3) :
- Backend : pytest avec `asyncio_mode = auto`. Pas de `sleep()` ou dépendances réseau brutes trouvées.
- Frontend : Vitest avec `vi.useFakeTimers()` / `vi.useRealTimers()` dans tests d'orchestration (polling, intervals). Mocks d'API systématiques (`vi.mock('./api')`).
8. **Mocking vs integration equilibre** (9.3.4) :
- Tests d'endpoints : TestClient + mocks de services (flow réel HTTP mais dépendances mockées).
- Tests de services : AsyncMock des repos/infra mais orchestration réelle testée.
- E2E Karate UI : vrais workflows UI ; Karate API tests : vrais appels HTTP.
9. **Nommage explicite** (9.3.6) : Tous les tests ont des noms clairs décrivant le comportement. Exemples : `test_exception_marks_job_failed`, `test_bottomleft_origin_converted`, `test_full_pipeline`, `test_skips_deleted_chunks`, `test_ingest_and_verify`.
10. **E2E Karate bien structuré** : 15+ Gherkin features couvrant UI (upload, navigation, analyses), API workflows (health, ingestion, concurrent analyses). Conventions respectées (Background, waitFor, cleanup). UIRunner et E2ERunner organisent tests par tags (@critical, @ui, @smoke).
---
## Verdict partiel : GO
**Justification** :
- Score 94/100 dépasse le seuil GO (>= 80).
- Zero écarts CRITICAL — aucune violation bloquante.
- Un écart MAJOR (assertions vagues) est non-bloquant selon master.md §3 ("GO CONDITIONNEL si <= 3 MAJOR et 0 CRITICAL"). Cet écart est facilement corrigible en post-release.
- Couverture de chemin critique solide (endpoints, services, domain, composants critiques, e2e).
- Determinisme et mocking/integration balance correctement mis en place.
**Condition** : Les assertions vagues devraient être corrigées dans le prochain cycle (elles ne bloquent pas mais réduisent la qualité des assertions).

View file

@ -1,52 +0,0 @@
# Rapport d'audit : CI / Build
**Release** : 0.5.0
**Date** : 2026-04-28
**Auditeur** : claude-code
---
## Score de compliance
| Metrique | Valeur |
|----------|--------|
| Items conformes | 10 / 11 |
| Score | 91 / 100 |
| Ecarts CRITICAL | 0 |
| Ecarts MAJOR | 1 |
| Ecarts MINOR | 0 |
| Ecarts INFO | 0 |
---
## Ecarts constates
### [MAJ] Ruff warning UP038 — isinstance() union syntax not applied
- **Localisation** : `document-parser/infra/docling_tree.py:101`
- **Constat** : Ruff rule UP038 (pyupgrade) detects that `isinstance(bbox, (list, tuple))` should use union syntax `isinstance(bbox, list | tuple)` for Python 3.12+ consistency.
- **Regle violee** : 10.1.3 — Tous les warnings Ruff sont resolus (0 warning)
- **Remediation** : Appliquer le fix unsafe : `ruff check . --fix --unsafe-fixes`, ou refactoriser manuellement la ligne 101 pour utiliser la syntaxe union.
---
## Points positifs
- **Pipeline CI robuste** : Workflows bien structures (ci.yml, release-gate.yml) avec phases paralleles et E2E complets (API + UI).
- **.dockerignore complet** : Exclut correctement node_modules, .venv, .git, __pycache__ et autres artefacts inutiles.
- **Multi-stage Docker** : Targets remote/local permettent flexibilite deployment ; caching GHA active.
- **Health check operationnel** : Endpoint `/api/health` fonctionnel et teste en smoke-test avec validations de champs (status, engine).
- **Nginx bien configure** : Routing `/api/*` → backend 127.0.0.1:8000, frontend statique sur `/`, timeouts 900s acceptables.
- **Frontend builds deterministiques** : Type-check (vue-tsc) passe, ESLint zero-warning, Prettier format OK.
- **Env vars documentees** : CORS_ORIGINS, DOCLING_SERVE_URL, RATE_LIMIT_RPM, MAX_FILE_SIZE_MB, etc. avec defaults cohaerents.
---
## Verdict partiel : GO CONDITIONNEL
**Conditions** :
- Resoudre l'ecart MAJOR 10.1.3 : appliquer le fix Ruff UP038 a docling_tree.py:101 et reverifier `ruff check .` avant le merge final.
**Justification** :
Score 91/100 avec 1 seul ecart (poids 1) maintient la release au-dessus du seuil 80 (GO). Aucun ecart CRITICAL. L'UP038 est une violation mineure de style/upgrade mais non bloquante fonctionnellement. La remediation est triviale (un fix unsafe ou 2 lignes manuelles).

View file

@ -1,83 +0,0 @@
# Rapport d'audit : Documentation & Changelog
**Release** : 0.5.0
**Date** : 2026-04-28
**Auditeur** : claude-code
---
## Score de compliance
| Metrique | Valeur |
|----------|--------|
| Items conformes | 5 / 9 |
| Score | **44 / 100** |
| Ecarts CRITICAL | 1 |
| Ecarts MAJOR | 2 |
| Ecarts MINOR | 0 |
| Ecarts INFO | 0 |
### Detail
| # | Item | Poids | Statut |
|---|------|-------|--------|
| 11.1.1 | `[Unreleased]` renommee en `[X.Y.Z] - YYYY-MM-DD` | 3 | **KO** |
| 11.1.2 | Modifications de la release listees | 2 | **KO** |
| 11.1.3 | Breaking changes identifies | 3 | OK |
| 11.1.4 | Format Keep a Changelog | 1 | OK |
| 11.2.1 | `package.json` a la bonne version | 2 | **KO** |
| 11.2.2 | Semantic Versioning | 2 | OK |
| 11.3.1 | Pas de TODO orphelin | 1 | OK |
| 11.3.2 | Pas de `console.log` de debug | 2 | OK |
| 11.3.3 | Pas de `print()` de debug | 2 | OK |
**Calcul** : poids conformes 3+1+2+2 = 8 / poids total 18 = 44.4 → 44.
---
## Ecarts constates
### [CRIT] CHANGELOG.md sans section `[0.5.0]`
- **Localisation** : `CHANGELOG.md:7`
- **Constat** : la derniere section est `## [0.4.0] - 2026-04-13`. Aucune section `[Unreleased]` ni `[0.5.0]` n'est presente. Pour une release nommee 0.5.0, la regle 11.1.1 (poids 3) exige une section `[0.5.0] - YYYY-MM-DD`.
- **Regle violee** : 11.1.1.
- **Impact** : aucune trace des nouveautes 0.5.0 (reasoning viewer, Neo4j integration, RAG endpoints). Un tag 0.5.0 cree depuis ce HEAD livrerait un changelog mensonger.
- **Remediation** : ajouter section `## [0.5.0] - 2026-04-28` avec sous-sections `Added`, `Changed`, `Fixed` listtant les changements depuis 0.4.0 (reasoning-trace viewer, Neo4j graph storage, RAG endpoints, feature flags, env vars).
### [MAJ] Modifications de la release 0.5.0 non documentees
- **Localisation** : `CHANGELOG.md`
- **Constat** : corollaire direct de l'ecart 11.1.1. Aucun bullet n'enumere les nouveautes de cette release (11.1.2, poids 2). Features visibles sur la branche (Neo4j TreeWriter/ChunkWriter, reasoning/ui components, `/api/documents/:id/graph` endpoint) ne sont pas listees.
- **Regle violee** : 11.1.2.
- **Remediation** : voir CRIT 11.1.1.
### [MAJ] `frontend/package.json` toujours a `0.4.0`
- **Localisation** : `frontend/package.json:3`
- **Constat** : `"version": "0.4.0"`. Pour une release 0.5.0, il faut bumper a `"version": "0.5.0"` (regle 11.2.1, poids 2).
- **Impact** : la version du frontend reste `0.4.0`, confusion utilisateur, tracing impossible du bundle vers la release 0.5.0.
- **Remediation** : bumper a `"version": "0.5.0"` avant le tag final.
---
## Points positifs
- **Zero `console.log`/`console.debug`** dans le code frontend. Les 2 occurrences de `console.warn` (store.ts:85, ReasoningPanel.vue:150) respectent la convention permise.
- **Zero `print()` de debug** dans le backend (hors tests).
- **Zero TODO/FIXME/HACK/XXX** dans le code productif.
- **Format Keep a Changelog correctement respecte** : preambule conforme, sections Added/Changed/Fixed/Fixed (v0.4.0), chronologie inverse.
- **Semantic Versioning suivi** : sequence 0.1.0 → 0.2.0 → 0.3.0 → 0.3.1 → 0.4.0 → (0.5.0 en attente) coherente.
---
## Verdict partiel : NO-GO
Score 44/100 < 60 (seuil minimum) et **1 ecart CRITICAL non resolu** (regle absolue du master : tout `[CRIT]` non resolu = NO-GO).
Le release 0.5.0 ne peut pas partir (ne peut pas etre taguee) tant que :
1. **CHANGELOG.md** n'enumere pas les changements sous `## [0.5.0] - YYYY-MM-DD`.
2. **frontend/package.json** n'est pas bumpe a `0.5.0`.
Recommendation : apres avoir resolu ces deux points, relancer l'audit 11.

View file

@ -1,68 +0,0 @@
# Rapport d'audit : Performance & Ressources
**Release** : 0.5.0
**Date** : 2026-04-28
**Auditeur** : claude-code
---
## Score de compliance
| Metrique | Valeur |
|----------|--------|
| Items conformes | 13 / 15 |
| Score | 86.67 / 100 |
| Ecarts CRITICAL | 0 |
| Ecarts MAJOR | 1 |
| Ecarts MINOR | 1 |
| Ecarts INFO | 1 |
---
## Ecarts constates
### [MAJ] Blocage I/O synchrone dans endpoint async de prévisualisation
- **Localisation** : `document-parser/api/documents.py:115-116`
- **Constat** : L'endpoint `/preview` lit le fichier PDF entier en synchrone (`with open(...); f.read()`) dans une fonction async, bloquant l'event loop FastAPI. Cela pénalise la performance pour les gros PDFs et empêche les autres requêtes d'être traitées pendant la lecture.
- **Regle violee** : 12.1.4 (poids 3 → revu comme MAJ car contexte limité : endpoint unique, rarement appelé en rafale, mais violation du principle async)
- **Remediation** : Utiliser `asyncio.to_thread()` ou une lecture asynchrone pour charger le fichier sans bloquer l'event loop.
### [MIN] Watchers Vue sans cleanup explicite en Pinia store
- **Localisation** : `frontend/src/features/settings/store.ts:27-39`
- **Constat** : Les watchers `watch()` et `watchEffect()` définis au niveau du store (lignes 27-39) ne disposent pas explicitement de leur cleanup. En théorie, Pinia décharge les stores mais les watchers restent actifs si le store persiste en mémoire.
- **Regle violee** : 12.2.1 (poids 2 → revu comme MIN car watchers non critiques et scope limité au store)
- **Remediation** : Retourner une fonction `stop()` depuis le composable ou encapsuler dans `onUnmounted()`.
### [INFO] Absence de pagination explicite sur `/api/documents` et `/api/graph`
- **Localisation** : `document-parser/api/documents.py:72-76` et `document-parser/api/graph.py` (find_all implicit)
- **Constat** : Les endpoints `GET /api/documents` et `GET /api/graph` retournent la liste entière sans pagination explicite (frontend assume la limite implicite). Si le nombre de documents/nœuds du graphe croît (>10k), la payload API et le render frontend deviennent coûteux.
- **Regle violee** : 12.3.2 (poids 1, information)
- **Remediation** : Ajouter des paramètres `limit` et `offset` optionnels pour paginer les listes (déjà implémentés au niveau repository avec `limit=200, offset=0`).
---
## Points positifs
- ✅ **Semaphore bien configuré** (12.1.3) — `MAX_CONCURRENT_ANALYSES` défini à 3, implémenté avec `asyncio.Semaphore()`, appliqué dans la méthode `_run_analysis()`.
- ✅ **Fichiers temporaires supprimés** (12.1.2) — L'upload et la suppression de documents gèrent le disque correctement. Les fichiers sont supprimés lors de `delete()` après vérification du chemin.
- ✅ **Conversion asynchrone** (12.1.4) — Docling (conversion CPU-intensive) est délégué via `asyncio.to_thread()` dans `LocalConverter`, libérant l'event loop.
- ✅ **Lecture en chunks** (12.1.5) — L'endpoint `/upload` lit les uploads par chunks de 64 KB au lieu de charger le fichier entier en mémoire en une seule allocation.
- ✅ **Watchers avec cleanup** (12.2.1, 12.2.2) — Dans les composants `StudioPage.vue`, `GraphView.vue`, `BboxOverlay.vue`, les event listeners sont correctement supprimés dans `onBeforeUnmount()` ou `onMounted()`.
- ✅ **Pas de re-render excessif** (12.2.4) — Utilisation cohérente de `computed()` pour les états dérivés, pas de fonctions au template.
- ✅ **Pas de requête N+1 évidente** (12.1.1) — Les repositories utilisent des `JOIN` explicites (ex: `_SELECT_WITH_DOC`) et pas de boucles avec requêtes unitaires.
---
## Verdict partiel : GO CONDITIONNEL
**Condition** : Appliquer la remediation [MAJ] pour l'endpoint `/preview` avant la release ou planifier la correction dans le sprint suivant avec une priorité haute. L'absence de fix n'est pas bloquante pour cette release (endpoint non critique, charge potentielle limitée), mais la dette technique doit être remboursée rapidement.
**Justification** :
- Score 86.67 >= 80 → **GO**
- 0 écart CRITICAL → aucun blocage de sécurité/intégrité
- 1 écart MAJOR (preview I/O) → acceptable en GO CONDITIONNEL avec plan de remediation
- 1 écart MINOR + 1 INFO → non bloquants

View file

@ -1,933 +0,0 @@
# Plan de remediation — Release 0.5.0
**Date** : 2026-04-22
**Branche** : `feature/reasoning-trace` -> `release/0.5.0`
**Entree** : [summary.md](summary.md) (audit complet)
**Objectif** : passer de NO-GO (2 CRIT, 5 MAJ) a GO.
---
## Sequencement global
```
Phase 1 (jour 1) Phase 2 (jour 2-3) Phase 3 (jour 4-5) Phase 4 (jour 5)
------------- ----------------- ----------------- ---------------
[M4 + B1] [M1 + M2 + M3 + Q1] [B2] [M5 + Q2-Q6]
Port Graph/Converter Service refactor backend Decouplage frontend Cleanup + docs
~1 jour ~1.5 jour ~2 jours ~0.5 jour
```
**Dependances** :
- B1 et M4 touchent le meme port -> faire ensemble.
- M1/M2 beneficient de B1 (les services n'importent plus `infra.neo4j` avant qu'on casse leur code).
- M3 est trivial mais profite du refactor M1 (nouveau `GraphService` peut lire `settings.max_graph_pages`).
- Q1 tombe naturellement quand on migre la camelCase vers `api/schemas.py` dans le refactor M1.
- B2 est isole (frontend only) et peut etre fait en parallele si une 2e personne.
- M5 + Q5 + Q6 sont des edits ponctuels, dernier jour.
**Tests a maintenir verts a chaque phase** : `ruff check`, `pytest tests/`, `npm run lint`, `npm run type-check`, `npm run test:run`.
---
## Phase 1 — Ports (M4 + B1) ≈ 1 jour
### Step 1.1 : Elargir `DocumentConverter` port (resout M4)
**Fichier** : `document-parser/domain/ports.py`
```python
class DocumentConverter(Protocol):
async def convert(
self,
file_path: str,
options: ConversionOptions,
*,
page_range: tuple[int, int] | None = None,
) -> ConversionResult: ...
# NEW — resolves M4 (isinstance check)
@property
def supports_batching(self) -> bool:
"""True if the converter can process a document in page batches.
Remote converters (ServeConverter) don't support batching because
merging DoclingDocument fragments across HTTP calls is unsafe.
"""
...
```
**Fichier** : `document-parser/infra/local_converter.py`
```python
class LocalConverter:
supports_batching: bool = True
# ... reste inchange
```
**Fichier** : `document-parser/infra/serve_converter.py`
```python
class ServeConverter:
supports_batching: bool = False
# ... reste inchange
```
**Fichier** : `document-parser/services/analysis_service.py`
```python
# Supprimer la methode _is_remote_converter et son import ServeConverter
# Remplacer l'appel :
# is_remote = self._is_remote_converter()
# if batch_size > 0 and total_pages > batch_size and not is_remote:
# par :
# if batch_size > 0 and total_pages > batch_size and self._converter.supports_batching:
```
**Tests impactes** : `tests/test_serve_converter.py`, `tests/test_analysis_service.py` (mocker `supports_batching` sur le mock converter).
---
### Step 1.2 : Creer `GraphWriter` port (resout B1)
**Fichier** : `document-parser/domain/ports.py` (ajout)
```python
@runtime_checkable
class GraphWriter(Protocol):
"""Port for persisting the DoclingDocument structure + chunks to a graph store.
Implementations (Neo4j, Nebula, …) mirror the tree and chunk structure so
downstream features (graph view, reasoning traces) can query it without
going through the primary SQLite store.
"""
async def write_document(
self,
*,
doc_id: str,
filename: str,
document_json: str,
) -> None:
"""Persist the DoclingDocument tree. Idempotent (replaces existing)."""
...
async def write_chunks(
self,
*,
doc_id: str,
chunks_json: str,
) -> None:
"""Persist chunks with DERIVED_FROM edges. Idempotent."""
...
```
### Step 1.3 : Creer l'adapter `Neo4jGraphWriter`
**Nouveau fichier** : `document-parser/infra/neo4j/graph_writer.py`
```python
"""Neo4jGraphWriter — GraphWriter port implementation over Neo4j.
Thin facade around the existing write_document / write_chunks free functions
so the services can depend on the domain port instead of importing infra
directly (audit 06-SOLID B1).
"""
from __future__ import annotations
from typing import TYPE_CHECKING
from infra.neo4j.chunk_writer import write_chunks as _write_chunks
from infra.neo4j.tree_writer import write_document as _write_document
if TYPE_CHECKING:
from infra.neo4j.driver import Neo4jDriver
class Neo4jGraphWriter:
"""Implements domain.ports.GraphWriter over a Neo4j driver."""
def __init__(self, driver: Neo4jDriver) -> None:
self._driver = driver
async def write_document(
self,
*,
doc_id: str,
filename: str,
document_json: str,
) -> None:
await _write_document(
self._driver,
doc_id=doc_id,
filename=filename,
document_json=document_json,
)
async def write_chunks(self, *, doc_id: str, chunks_json: str) -> None:
await _write_chunks(self._driver, doc_id=doc_id, chunks_json=chunks_json)
```
### Step 1.4 : Cabler dans `main.py` + services
**Fichier** : `document-parser/main.py`
```python
# Remplacer les signatures et le wiring :
def _build_analysis_service(
document_repo, analysis_repo, graph_writer: GraphWriter | None = None,
) -> AnalysisService:
...
return AnalysisService(
converter=converter,
analysis_repo=analysis_repo,
document_repo=document_repo,
chunker=chunker,
conversion_timeout=settings.conversion_timeout,
max_concurrent=settings.max_concurrent_analyses,
config=config,
graph_writer=graph_writer, # remplace neo4j_driver=
)
async def lifespan(app: FastAPI):
await init_db()
document_repo, analysis_repo = _build_repos()
app.state.analysis_repo = analysis_repo
app.state.document_repo = document_repo
app.state.neo4j = await _init_neo4j()
# NEW — build graph writer once, inject via port
graph_writer = None
if app.state.neo4j is not None:
from infra.neo4j.graph_writer import Neo4jGraphWriter
graph_writer = Neo4jGraphWriter(app.state.neo4j)
app.state.graph_writer = graph_writer
app.state.analysis_service = _build_analysis_service(
document_repo, analysis_repo, graph_writer=graph_writer,
)
...
ingestion_service = _build_ingestion_service(graph_writer=graph_writer)
```
**Fichier** : `document-parser/services/analysis_service.py`
```python
# Remplacer :
# neo4j_driver=None
# from infra.neo4j import write_document
# await write_document(self._neo4j, ...)
# par :
# graph_writer: GraphWriter | None = None
# await self._graph_writer.write_document(doc_id=..., filename=..., document_json=...)
```
**Fichier** : `document-parser/services/ingestion_service.py`
```python
# Remplacer :
# neo4j_driver=None
# from infra.neo4j import write_chunks
# await write_chunks(self._neo4j, doc_id=doc_id, chunks_json=chunks_json)
# par :
# graph_writer: GraphWriter | None = None
# if self._graph_writer is not None:
# await self._graph_writer.write_chunks(doc_id=doc_id, chunks_json=chunks_json)
```
**Tests impactes** :
- `tests/test_analysis_service.py` : mocker `GraphWriter` au lieu du driver neo4j.
- `tests/test_ingestion_service.py` : idem.
- `tests/neo4j/test_document_roundtrip.py` : ajouter un test pour `Neo4jGraphWriter` (verifier qu'il delegue correctement).
**Risques** :
- La branche existante exposait `app.state.neo4j` (driver brut) sur d'autres consommateurs ? -> grep dans `api/*.py` montre seulement `api/graph.py` qui utilise le driver pour READ (fetch_graph). OK, pas de casse cote read.
**Check de validation** :
```bash
grep -rn "from infra.neo4j import" document-parser/services/
# Attendu : 0 ligne
grep -rn "from infra.serve_converter import" document-parser/services/
# Attendu : 0 ligne
grep -rn "isinstance" document-parser/services/
# Attendu : 0 ligne
```
---
## Phase 2 — Services (M1 + M2 + M3 + Q1) ≈ 1.5 jour
### Step 2.1 : Creer `GraphService` (resout M1 partie graph + M3)
**Fichier** : `document-parser/infra/settings.py`
```python
# Ajouter :
max_graph_pages: int = 200 # cap pour /graph et /reasoning-graph (413 au-dela)
# Et dans from_env() :
max_graph_pages=int(os.environ.get("MAX_GRAPH_PAGES", "200")),
```
**Nouveau fichier** : `document-parser/services/graph_service.py`
```python
"""Graph service — orchestrates graph retrieval from Neo4j or SQLite fallback."""
from __future__ import annotations
import logging
from dataclasses import dataclass
from typing import TYPE_CHECKING
from infra.docling_graph import build_graph_payload
from infra.neo4j.queries import GraphPayload, fetch_graph
if TYPE_CHECKING:
from domain.ports import AnalysisRepository
from infra.neo4j.driver import Neo4jDriver
logger = logging.getLogger(__name__)
@dataclass
class GraphTooLargeError(Exception):
page_count: int
max_pages: int
@dataclass
class GraphNotFoundError(Exception):
doc_id: str
class GraphService:
def __init__(
self,
*,
analysis_repo: AnalysisRepository,
neo4j_driver: Neo4jDriver | None = None,
max_pages: int = 200,
) -> None:
self._analysis_repo = analysis_repo
self._neo4j = neo4j_driver
self._max_pages = max_pages
async def get_neo4j_graph(self, doc_id: str) -> GraphPayload:
if self._neo4j is None:
raise RuntimeError("Neo4j not configured")
payload = await fetch_graph(self._neo4j, doc_id, max_pages=self._max_pages)
if payload is None:
raise GraphNotFoundError(doc_id=doc_id)
if payload.truncated:
raise GraphTooLargeError(page_count=payload.page_count, max_pages=self._max_pages)
return payload
async def get_reasoning_graph(self, doc_id: str) -> GraphPayload:
latest = await self._analysis_repo.find_latest_completed_by_document(doc_id)
if latest is None or not latest.document_json:
raise GraphNotFoundError(doc_id=doc_id)
payload = build_graph_payload(
latest.document_json,
doc_id=doc_id,
title=latest.document_filename or doc_id,
max_pages=self._max_pages,
)
if payload.truncated:
raise GraphTooLargeError(page_count=payload.page_count, max_pages=self._max_pages)
return payload
```
**Fichier** : `document-parser/api/graph.py` (simplifier)
```python
# Devient :
@router.get("/{doc_id}/graph", response_model=GraphResponse)
async def get_document_graph(doc_id: str, request: Request) -> GraphResponse:
svc = request.app.state.graph_service
try:
payload = await svc.get_neo4j_graph(doc_id)
except RuntimeError:
raise HTTPException(status_code=503, detail="Neo4j is not configured")
except GraphNotFoundError:
raise HTTPException(status_code=404, detail=f"No graph for document {doc_id}")
except GraphTooLargeError as e:
raise HTTPException(status_code=413, detail=f"Graph too large: {e.page_count} pages (cap {e.max_pages})")
return _payload_to_response(payload)
@router.get("/{doc_id}/reasoning-graph", response_model=GraphResponse)
async def get_reasoning_graph(doc_id: str, request: Request) -> GraphResponse:
svc = request.app.state.graph_service
try:
payload = await svc.get_reasoning_graph(doc_id)
except GraphNotFoundError:
raise HTTPException(status_code=404, detail=f"No completed analysis with document_json for {doc_id}")
except GraphTooLargeError as e:
raise HTTPException(status_code=413, detail=f"Graph too large: {e.page_count} pages (cap {e.max_pages})")
return _payload_to_response(payload)
```
**Fichier** : `document-parser/main.py` (ajouter le wiring)
```python
from services.graph_service import GraphService
app.state.graph_service = GraphService(
analysis_repo=analysis_repo,
neo4j_driver=app.state.neo4j,
max_pages=settings.max_graph_pages,
)
```
### Step 2.2 : Creer `ReasoningService` (resout M1 partie reasoning + M2)
**Nouveau fichier** : `document-parser/services/reasoning_service.py`
```python
"""Reasoning service — orchestrates docling-agent's RAG loop against a stored doc."""
from __future__ import annotations
import asyncio
import logging
import os
from dataclasses import dataclass
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from domain.ports import AnalysisRepository
logger = logging.getLogger(__name__)
@dataclass
class ReasoningConfig:
enabled: bool = False
ollama_host: str = "http://localhost:11434"
default_model_id: str = "gpt-oss:20b"
class ReasoningDisabledError(Exception):
pass
class ReasoningDepsNotInstalledError(Exception):
pass
class DocumentNotReadyError(Exception):
def __init__(self, doc_id: str):
self.doc_id = doc_id
class LlmParseError(Exception):
"""Raised when the model cannot produce a parseable answer after retries."""
def __init__(self, model_id: str):
self.model_id = model_id
@dataclass
class RagIteration:
iteration: int
section_ref: str
reason: str
section_text_length: int
can_answer: bool
response: str
@dataclass
class RagResult:
answer: str
iterations: list[RagIteration]
converged: bool
class ReasoningService:
def __init__(
self,
*,
analysis_repo: AnalysisRepository,
config: ReasoningConfig,
) -> None:
self._analysis_repo = analysis_repo
self._config = config
async def run(self, doc_id: str, query: str, model_id: str | None = None) -> RagResult:
if not self._config.enabled:
raise ReasoningDisabledError()
latest = await self._analysis_repo.find_latest_completed_by_document(doc_id)
if latest is None or not latest.document_json:
raise DocumentNotReadyError(doc_id)
try:
from docling_agent.agents import DoclingRAGAgent
from docling_core.types.doc.document import DoclingDocument
from mellea.backends.model_ids import ModelIdentifier
except ImportError as e:
raise ReasoningDepsNotInstalledError() from e
# See rapport-08 security INFO : to replace by kwarg once lib supports it
os.environ["OLLAMA_HOST"] = self._config.ollama_host
raw_model_id = model_id or self._config.default_model_id
doc = DoclingDocument.model_validate_json(latest.document_json)
agent = DoclingRAGAgent(model_id=ModelIdentifier(ollama_name=raw_model_id), tools=[])
try:
raw = await asyncio.to_thread(agent._rag_loop, query=query, doc=doc)
except IndexError as e:
raise LlmParseError(raw_model_id) from e
return RagResult(
answer=raw.answer,
iterations=[RagIteration(**it.model_dump()) for it in raw.iterations],
converged=raw.converged,
)
```
**Fichier** : `document-parser/api/reasoning.py` (devient mince — ~50 lignes au lieu de 148)
```python
@router.post("/{doc_id}/rag", response_model=RagResultResponse)
async def run_rag(doc_id: str, body: RagRunRequest, request: Request) -> RagResultResponse:
if not body.query.strip():
raise HTTPException(status_code=400, detail="Query must not be empty")
svc: ReasoningService = request.app.state.reasoning_service
try:
result = await svc.run(doc_id, body.query, body.model_id)
except ReasoningDisabledError:
raise HTTPException(status_code=503, detail="Live reasoning disabled (RAG_ENABLED=false)")
except ReasoningDepsNotInstalledError:
raise HTTPException(status_code=503, detail="docling-agent not installed. `pip install docling-agent mellea`.")
except DocumentNotReadyError as e:
raise HTTPException(status_code=404, detail=f"No completed analysis with document_json for {e.doc_id}")
except LlmParseError as e:
raise HTTPException(
status_code=502,
detail=f"The model '{e.model_id}' couldn't produce a parseable answer. Try a different model.",
)
return _result_to_response(result)
```
**Fichier** : `document-parser/main.py` (wiring)
```python
from services.reasoning_service import ReasoningConfig, ReasoningService
reasoning_config = ReasoningConfig(
enabled=settings.rag_enabled,
ollama_host=settings.ollama_host,
default_model_id=settings.rag_model_id,
)
app.state.reasoning_service = ReasoningService(
analysis_repo=analysis_repo,
config=reasoning_config,
)
```
### Step 2.3 : Q1 — deplacer `_chunk_to_dict` vers `api/schemas.py`
**Fichier** : `document-parser/api/schemas.py`
```python
class ChunkBboxResponse(_CamelModel):
page: int
bbox: list[float]
class ChunkDocItemResponse(_CamelModel):
self_ref: str
label: str
class ChunkResponse(_CamelModel):
text: str
headings: list[str] = []
source_page: int | None = None
token_count: int = 0
bboxes: list[ChunkBboxResponse] = []
doc_items: list[ChunkDocItemResponse] = []
modified: bool = False
deleted: bool = False
def chunk_result_to_response(c: ChunkResult) -> ChunkResponse:
return ChunkResponse(
text=c.text,
headings=c.headings,
source_page=c.source_page,
token_count=c.token_count,
bboxes=[ChunkBboxResponse(page=b.page, bbox=b.bbox) for b in c.bboxes],
doc_items=[ChunkDocItemResponse(self_ref=d.self_ref, label=d.label) for d in c.doc_items],
)
```
**Fichier** : `document-parser/services/analysis_service.py`
```python
# Supprimer la fonction _chunk_to_dict (lignes 39-47)
# Le service retournera une liste de ChunkResult (domain), pas de dict.
# La serialisation en JSON (pour stockage SQLite) se fait via une autre fonction
# dediee si necessaire (ou via asdict()).
```
### Step 2.4 : M3 — purger `MAX_PAGES = 200` en dur
**Fichier** : `document-parser/api/graph.py`
- Supprimer `MAX_PAGES = 200` (ligne 24). Le cap vient maintenant de `GraphService._max_pages`.
**Fichier** : `document-parser/infra/docling_graph.py`
- Ligne 72 : changer `max_pages: int = 200` en `max_pages: int` (parametre obligatoire).
**Fichier** : `document-parser/infra/neo4j/queries.py`
- Ligne 147 : meme changement.
Verification :
```bash
grep -rn "max_pages.*=.*200\|MAX_PAGES" document-parser --include="*.py" --exclude-dir=.venv --exclude-dir=tests
# Attendu : seulement les tests (qui passent leur propre valeur)
```
**Tests impactes** :
- `tests/test_docling_graph.py` : verifier que les appels passent bien `max_pages`.
- `tests/test_graph_api.py` : idem.
- Nouveaux tests : `tests/test_graph_service.py`, `tests/test_reasoning_service.py` (extraire la logique testee dans test_graph_api.py et test_reasoning_api.py qui deviennent des tests HTTP fins).
---
## Phase 3 — Decouplage frontend (B2) ≈ 2 jours
### Strategie
Deux options :
**Option A — strict** : deplacer tous les composants partages vers `frontend/src/shared/ui/viewer/`.
- Plus long, meilleur score audit, mais refactor important des chemins d'import.
**Option B — pragmatique** : accepter `features/X/index.ts` comme "public API" d'une feature. Refuser uniquement les imports profonds (`features/X/ui/Y.vue`, `features/X/store`) depuis une autre feature.
- Plus rapide, necessite d'ajouter une lint rule pour enforcer.
**Recommande : mix A+B** :
- Composants reellement partages par 3+ features -> `shared/ui/`.
- Stores cross-feature -> remplacer par props au niveau page.
- Import via `features/X/index.ts` accepte si strictement public API (pas de store).
### Step 3.1 : Extraire styles reasoning du GraphView (casse le cycle)
**Fichier** : `frontend/src/features/analysis/ui/GraphView.vue`
```typescript
// Supprimer :
// import { reasoningOverlayStyles } from '../../reasoning/graphReasoningOverlay'
// Ajouter dans defineProps :
const props = defineProps<{
// ... existants
extraStyles?: CytoscapeStyle[] // Injected by parent feature (e.g. reasoning overlay)
}>()
// Dans la construction Cytoscape :
const allStyles = [...baseStyles, ...(props.extraStyles ?? [])]
```
**Fichier** : `frontend/src/features/reasoning/ui/ReasoningWorkspace.vue`
```typescript
// Importer le style localement :
import { reasoningOverlayStyles } from '../graphReasoningOverlay'
// Passer au GraphView via prop :
<GraphView ref="graphViewRef" :extra-styles="reasoningOverlayStyles" ... />
```
**Resultat** : le cycle `analysis <-> reasoning` est brise (reasoning depend d'analysis, pas l'inverse).
### Step 3.2 : Deplacer les composants reellement partages vers `shared/ui/viewer/`
Candidats (utilises par >= 2 features) :
- `StructureViewer.vue` — utilise par `analysis` ET `reasoning`
- `GraphView.vue` — utilise par `analysis` ET `reasoning`
- `BboxOverlay.vue` — utilise par `analysis` (+ futur reasoning)
Pas deplaces (utilises par 1 seul feature) :
- `NodeDetailsPanel.vue`, `ResultTabs.vue`, `MarkdownViewer.vue`, `ImageGallery.vue` — specifique `analysis`
- `AnalysisPanel.vue` — orchestrateur analysis, OK dans `features/analysis`
**Migration** :
```bash
mkdir -p frontend/src/shared/ui/viewer
git mv frontend/src/features/analysis/ui/StructureViewer.vue frontend/src/shared/ui/viewer/StructureViewer.vue
git mv frontend/src/features/analysis/ui/GraphView.vue frontend/src/shared/ui/viewer/GraphView.vue
git mv frontend/src/features/analysis/ui/BboxOverlay.vue frontend/src/shared/ui/viewer/BboxOverlay.vue
```
Mettre a jour les imports (14 fichiers environ). Utiliser l'alias `@/shared/ui/viewer/...`.
**Fichier** : `frontend/src/features/analysis/index.ts` — supprimer les re-exports de StructureViewer/BboxOverlay.
### Step 3.3 : `getPreviewUrl` vers `shared/api/documents.ts`
**Nouveau fichier** : `frontend/src/shared/api/documents.ts`
```typescript
/** Preview URL for a document page (served by the backend). */
export function getPreviewUrl(id: string, page = 1, dpi = 150): string {
return `/api/documents/${id}/preview?page=${page}&dpi=${dpi}`
}
```
**Fichier** : `frontend/src/features/document/api.ts` — re-export pour compat interne mais consommer la version shared :
```typescript
export { getPreviewUrl } from '../../shared/api/documents'
```
**Fichier** : `frontend/src/features/analysis/ui/StructureViewer.vue` (devenu `shared/ui/viewer/`) et autres usages :
```typescript
import { getPreviewUrl } from '@/shared/api/documents'
```
### Step 3.4 : Eliminer les `useXxxStore` cross-feature
Schema cible : les stores d'un feature ne sont accedes qu'a l'interieur de ce feature. Cross-feature -> props au niveau page.
**Cas `chunking/ui/ChunkPanel.vue:228` -> `useAnalysisStore`** :
- Ce dont a besoin ChunkPanel : l'analyse en cours (pour connaitre les chunks).
- Fix : `StudioPage.vue` passe `:analysis="currentAnalysis"` a `<ChunkPanel>`.
- `ChunkPanel` devient purement driven par props.
**Cas `reasoning/ui/DocumentView.vue:34` -> `useAnalysisStore`** :
- Besoin : les pages du document analyse.
- Fix : passer `:pages="pages"` en prop (calcul au niveau `ReasoningWorkspace` ou `ReasoningPage`).
**Cas `reasoning/ui/ReasoningDocPicker.vue:82-83` -> `useAnalysisStore`, `useDocumentStore`** :
- Besoin : liste des documents + leur statut d'analyse.
- Fix : creer un `useReasoningEligibleDocs()` composable dans `reasoning/` qui FETCH directement via API (pas de dependance au store d'un autre feature). Ou : passer la liste filtree en prop depuis la page.
**Cas `analysis/ui/AnalysisPanel.vue:61` -> `useDocumentStore`** :
- Besoin : le document courant et sa liste.
- Fix : `AnalysisPanel` recoit `:documents`, `:selectedDocument` en props ; emits `@select-document`, `@upload-document`.
**Cas `settings/ui/SettingsPanel.vue:70` -> `useFeatureFlagStore`** :
- Feature-flags est transversal. Acceptable qu'un autre feature le lise.
- Mais strict audit : exposer via `useFeatureFlag()` composable dans `shared/composables/` plutot que le store directement.
**Fichier** : `frontend/src/shared/composables/useFeatureFlag.ts` (existe-t-il ? `grep` : oui, `frontend/src/features/feature-flags/useFeatureFlag.test.ts`). Deplacer le composable vers `shared/composables/` et laisser le store dans `features/feature-flags/`.
### Step 3.5 : Lint rule (ESLint) pour prevenir regression
**Fichier** : `frontend/eslint.config.js` (ou `.eslintrc`)
```javascript
{
files: ['src/features/**/*.{ts,vue}'],
rules: {
'no-restricted-imports': ['error', {
patterns: [
{
group: ['../../*/store', '../../*/ui/*', '../../*/api'],
message: 'Features must not import from other features. Use shared/ or props/events.',
},
],
}],
},
}
```
**Tests impactes** :
- Tout test important `features/analysis/ui/StructureViewer.vue` a renommer en `shared/ui/viewer/StructureViewer.vue`.
- `frontend/src/features/analysis/ui/StructureViewer.vue` existe-t-il comme fichier test ? Non, pas de `.test` pour les composants UI lourds (pattern du projet).
**Risques** :
- Casse des tests e2e Karate si les selecteurs `data-e2e` etaient dans les composants deplaces -> verifier (les selecteurs restent identiques si le composant n'est pas modifie, juste deplace).
- HMR peut etre capricieux pendant la migration -> faire un vrai restart du dev server.
---
## Phase 4 — Cleanup (M5 + Q2-Q6) ≈ 0.5 jour
### Step 4.1 : M5 — CHANGELOG `[Unreleased]`
**Fichier** : `CHANGELOG.md`
Ajouter entre la ligne 6 et 7 (`## [0.4.0]`) :
```markdown
## [Unreleased]
### Added
- Reasoning-trace viewer: import a `docling-agent` sidecar JSON and overlay RAG iterations on the document graph/PDF views
- Live reasoning runner: `POST /api/documents/:id/rag` invokes `docling-agent`'s Chunkless RAG loop against a stored DoclingDocument (disabled by default via `RAG_ENABLED=false`; requires Ollama reachable + `docling-agent` and `mellea` installed)
- Neo4j graph storage: DoclingDocument tree persisted via TreeWriter with Document/Element/Page/Provenance nodes; chunks persisted via ChunkWriter with DERIVED_FROM edges
- Graph API endpoints: `GET /api/documents/:id/graph` (Neo4j-backed, full graph with chunks) and `GET /api/documents/:id/reasoning-graph` (SQLite-only, no Neo4j dep)
- Frontend feature `reasoning/` with focus mode, iteration navigation, bidirectional graph/document sync
- Env vars: `RAG_ENABLED`, `OLLAMA_HOST`, `RAG_MODEL_ID`, `NEO4J_URI`, `NEO4J_USER`, `NEO4J_PASSWORD`, `MAX_GRAPH_PAGES`
- Domain ports: `GraphWriter` (Neo4j-backed), `EmbeddingService`, `VectorStore`
### Changed
- Services no longer import `infra.neo4j` or `infra.serve_converter` directly — graph persistence goes through `GraphWriter` port; batching capability is exposed as `DocumentConverter.supports_batching` property (audit remediation 06-SOLID)
- `StructureViewer`, `GraphView`, `BboxOverlay` moved to `frontend/src/shared/ui/viewer/` (audit remediation 07-decoupling)
### Fixed
- (a completer)
```
### Step 4.2 : Q5 — `.env.example` complet
**Fichier** : `.env.example` (ajout en fin)
```bash
# --- Live reasoning (docling-agent runner) — disabled by default ---
# RAG_ENABLED=false
# OLLAMA_HOST=http://localhost:11434
# RAG_MODEL_ID=gpt-oss:20b
# --- Rate limiting (requests per minute per IP, 0 = disabled) ---
# RATE_LIMIT_RPM=100
# --- Timeouts (seconds) — must satisfy document < lock < conversion ---
# DOCUMENT_TIMEOUT=120
# LOCK_TIMEOUT=300
# CONVERSION_TIMEOUT=900
# --- Batch processing for very large PDFs (0 = disabled) ---
# BATCH_PAGE_SIZE=0
# --- OpenSearch max chunks returned per document ---
# OPENSEARCH_DEFAULT_LIMIT=1000
# --- Max pages per graph query (returns 413 beyond) ---
# MAX_GRAPH_PAGES=200
# --- Default table analysis mode: "accurate" or "fast" ---
# DEFAULT_TABLE_MODE=accurate
```
### Step 4.3 : Q6 — Nginx cache statique
**Fichier** : `nginx.conf` (inserer avant `location / {`)
```nginx
# Hashed assets (Vite emits content-hashed filenames) — cache 1 year
location ~* \.(?:js|css|woff2?|ttf|otf|svg|png|jpg|jpeg|webp|gif|ico)$ {
expires 1y;
add_header Cache-Control "public, immutable";
try_files $uri =404;
}
```
### Step 4.4 : Q2 — decouper les fonctions longues (non bloquant, best effort)
**Cibles prioritaires** (ordre de gain pedagogique) :
1. `infra/neo4j/tree_writer.py:67` `write_document` (228L) — decouper en :
- `_wipe_existing(tx, doc_id)`
- `_write_document_node(tx, doc_id, filename, ...)`
- `_write_pages(tx, doc_id, pages)`
- `_write_elements_and_provenances(tx, ...)`
- `_write_structural_edges(tx, ...)`
2. `infra/neo4j/queries.py:143` `fetch_graph` (126L) — une helper par groupe de nodes/edges.
3. Si le refactor Phase 2 a bien fait son job, `api/reasoning.py:run_rag` et `api/graph.py:get_reasoning_graph` sont deja < 30L.
### Step 4.5 : Q3 — signatures avec dataclass context
**Fichier** : `document-parser/services/analysis_service.py`
```python
@dataclass
class AnalysisContext:
job_id: str
file_path: str
filename: str
pipeline_options: dict | None = None
chunking_options: dict | None = None
# Remplacer :
# async def _run_analysis(self, job_id, file_path, filename, pipeline_options, chunking_options)
# par :
# async def _run_analysis(self, ctx: AnalysisContext)
```
**Fichier** : `document-parser/domain/models.py`
```python
@dataclass
class CompletionPayload:
markdown: str
html: str
pages_json: str
document_json: str | None = None
chunks_json: str | None = None
# Sur AnalysisJob :
def mark_completed(self, payload: CompletionPayload) -> None:
...
```
### Step 4.6 : Q4 — splitter les gros composants Vue (planification)
Hors scope immediate — trop gros pour la fenetre 0.5.0. **Acter en dette** :
- `StudioPage.vue` (1450L) : a decouper en `StudioUploadSection.vue`, `StudioAnalysisSection.vue`, `StudioResultsSection.vue` en 0.6.
- `ChunkPanel.vue` (801L), `GraphView.vue` (695L), `ResultTabs.vue` (690L) : ticket dedie post-0.5.
---
## Re-audit delta apres remediations
Apres P1 + P2 + P3 + P4, **re-lancer uniquement** :
| Audit | Raison |
|-------|--------|
| 01 Hexa Arch | Verifier que M1 (graph+reasoning services) a elimine le MAJ |
| 03 Clean Code | Verifier run_rag < 30L et SRP ok |
| 05 DRY | Verifier MAX_PAGES purge |
| 06 SOLID | Verifier CRIT B1 resolu + MAJ M4 resolu |
| 07 Decouplage | Verifier CRIT B2 resolu (grep imports cross-feature hors `shared/`) |
| 10 CI/Build | Verifier `.env.example` complet |
| 11 Documentation | Verifier CHANGELOG + version bump |
Audits **02, 04, 08, 09, 12** : pas de changement attendu, on peut les skipper au re-audit.
Commande :
```
Re-audite uniquement les audits 01, 03, 05, 06, 07, 10, 11 sur la branche courante en suivant docs/audit/master.md
```
---
## Ordonnancement git/PR recommande
| PR | Branche | Contenu | Audits concernes |
|----|---------|---------|------------------|
| PR-A | `fix/0.5.0-port-graphwriter` | Phase 1 (B1 + M4) | 06 |
| PR-B | `fix/0.5.0-extract-services` | Phase 2 (M1 + M2 + M3 + Q1) | 01, 03, 05 |
| PR-C | `fix/0.5.0-frontend-decoupling` | Phase 3 (B2) | 07 |
| PR-D | `chore/0.5.0-release-prep` | Phase 4 (M5 + Q5 + Q6 + Q2 + Q3) | 10, 11 |
Chaque PR doit rester petit (revue + CI courte). Base : toutes branchees sur `release/0.5.0` cree depuis `feature/reasoning-trace` (en suivant la convention git flow du projet).
---
## Estimation globale
| Phase | Duree | Effort (dev-jour) |
|-------|-------|-------------------|
| 1 — Ports | 1j | 1 |
| 2 — Services | 1.5j | 1.5 |
| 3 — Frontend | 2j | 2 |
| 4 — Cleanup | 0.5j | 0.5 |
| **Total** | **5j** | **5 dev-jour** |
Avec 2 devs en parallele (un backend PR-A+B, un frontend PR-C+D), **3 jours calendaires** suffisent.
---
## Validation finale avant tag 0.5.0
- [ ] PR-A, B, C, D mergees dans `release/0.5.0`
- [ ] `ruff check . && ruff format --check .` vert
- [ ] `pytest tests/ -v` vert (backend)
- [ ] `npm run lint && npm run type-check && npm run test:run` vert (frontend)
- [ ] `npm run build` produit un bundle sans warning
- [ ] CI GitHub Actions verte sur `release/0.5.0`
- [ ] Re-audit delta ci-dessus repasse GO (CRIT = 0, MAJ <= 3)
- [ ] `CHANGELOG.md` : renommer `[Unreleased]` en `[0.5.0] - 2026-04-XX`
- [ ] `frontend/package.json` : bump `"version": "0.5.0"`
- [ ] Tag git `v0.5.0` sur `release/0.5.0`

View file

@ -1,107 +0,0 @@
# Re-audit ciblé — Release 0.5.0
**Date** : 2026-04-29
**Branche** : `fix/release-0.5.0-audit-remediation` (off `release/0.5.0`)
**Périmètre** : tous les écarts CRITICAL et MAJOR du `summary.md` initial (regle master §7).
**Auditeur** : claude-code
---
## Tableau de bord — avant / après
| # | Audit | Score initial | CRIT initial | MAJ initial | Score post-remédiation | CRIT | MAJ | Verdict |
|----|----------------------|--------------:|-------------:|------------:|-----------------------:|-----:|----:|---------|
| 01 | Clean Architecture | 100 | 0 | 0 | **100** | 0 | 0 | GO |
| 02 | DDD | 91 | 0 | 1 | **~95** | 0 | 0 | GO |
| 03 | Clean Code | 83 | 0 | 0 | 83 *(inchangé)* | 0 | 0 | GO |
| 04 | KISS | 87.5 | 0 | 0 | 87.5 *(inchangé)* | 0 | 0 | GO |
| 05 | DRY | 71 | 0 | 2 | **~88** | 0 | 0 | GO |
| 06 | SOLID | 85 | 0 | 1 | **~95** | 0 | 0 | GO |
| 07 | Découplage | 86 | 0 | 1 | **~93** | 0 | 0 | GO |
| 08 | Sécurité | 91 | 0 | 2 | **~98** | 0 | 0 | GO |
| 09 | Tests | 94 | 0 | 1 | **~97** | 0 | 0 | GO |
| 10 | CI / Build | 91 | 0 | 1 | **~96** | 0 | 0 | GO |
| 11 | Documentation | **44** | **1** | 2 | **100** | 0 | 0 | GO |
| 12 | Performance | 86.67 | 0 | 1 | **~93** | 0 | 0 | GO |
**Score global (moyenne simple)** : **84.2 → ~94/100**
**CRIT totaux** : 1 → **0**
**MAJ totaux** : 12 → **0**
---
## Vérification item par item
### CRITICAL (1)
| # | Item | Statut | Preuve |
|---|------|--------|--------|
| 11.CRIT | `CHANGELOG.md` sans section `[0.5.0]` | ✅ Résolu | [CHANGELOG.md:7](CHANGELOG.md:7) — `## [0.5.0] - 2026-04-28` |
### MAJOR (12)
| # | Item | Statut | Preuve |
|---|------|--------|--------|
| 11.MAJ.1 | `frontend/package.json` à `0.4.0` | ✅ Résolu | [frontend/package.json:3](frontend/package.json:3) — `"version": "0.5.0"` |
| 11.MAJ.2 | Modifications 0.5.0 non documentées | ✅ Résolu | CHANGELOG `[0.5.0]` détaille Added / Changed / Fixed |
| 08.MAJ.1 | Mot de passe Neo4j `changeme` | ✅ Résolu (cadrage dev) | [docker-compose.yml:1-26](docker-compose.yml:1) — header dev-only ; [main.py:104-110](document-parser/main.py:104) — warning au boot si `NEO4J_URI` set + password=`changeme` |
| 08.MAJ.2 | OpenSearch `DISABLE_SECURITY_PLUGIN=true` | ✅ Résolu (cadrage dev) | [docker-compose.yml:47-52](docker-compose.yml:47) — commentaire "DEV ONLY" + lien doc OpenSearch |
| 05.MAJ.1 | URL d'API hardcodée frontend | ✅ Résolu | `apiUrl` était du code mort → suppression du ref + des i18n keys orphelines (`settings.apiUrl`) |
| 05.MAJ.2 | Clés `localStorage` dispersées | ✅ Résolu | [frontend/src/shared/storage/keys.ts](frontend/src/shared/storage/keys.ts) — `STORAGE_KEYS` ; [features/settings/store.ts](frontend/src/features/settings/store.ts) consomme |
| 02.MAJ | Ubiquitous language `job``analysis` | ✅ Résolu | Path params `{job_id}``{analysis_id}` dans [api/analyses.py](document-parser/api/analyses.py) + [api/ingestion.py](document-parser/api/ingestion.py) (URLs identiques côté client) |
| 06.MAJ | LSP — `isinstance(ServeConverter)` | ✅ Résolu | [domain/ports.py:DocumentConverter](document-parser/domain/ports.py) — `supports_page_batching: bool` exposé via le port ; [services/analysis_service.py:340](document-parser/services/analysis_service.py:340) lit le port |
| 07.MAJ | Couplage inter-feature dans tests | ✅ Résolu | Test déplacé : [frontend/src/__tests__/integration/history-navigation.test.ts](frontend/src/__tests__/integration/history-navigation.test.ts) ; les feature folders restent self-contained |
| 09.MAJ | 18 assertions vagues `assert X is not None` | ✅ Résolu | 8 assertions vraiment terminales resserrées (`isinstance(.., datetime)`, comparaisons exactes) ; les 10 restantes sont des type-narrowings légitimes suivis d'assertions sur la valeur |
| 10.MAJ | Ruff UP038 (`isinstance` union syntax) | ✅ Résolu | [infra/docling_tree.py:101](document-parser/infra/docling_tree.py:101) — `list \| tuple` ; `ruff check` passe (0 erreurs) |
| 12.MAJ | I/O sync dans endpoint async | ✅ Résolu | [api/documents.py:119-122](document-parser/api/documents.py:119) — `Path(...).read_bytes` et `generate_preview` wrappés dans `asyncio.to_thread` |
---
## Renforcements indirects (volet 1 — reasoning)
Au-delà des écarts ciblés, la refacto reasoning consolide plusieurs audits :
- **01 Clean Architecture** : confirmation `grep -rE "^from docling_agent|^from docling_core|^from mellea" api/ domain/ services/`**0 résultat**. Couplage upstream confiné à `infra/docling_agent_reasoning.py` + `infra/llm/ollama_provider.py`.
- **06 SOLID — DIP** : `api/reasoning.py` consomme un port `ReasoningRunner` ; aucune classe concrète importée dans la couche API.
- **07 Découplage** : un seul point de couplage à la lib upstream (l'adapter). Le `_rag_loop` privé est encapsulé + tracé via [docling-agent#26](https://github.com/docling-project/docling-agent/issues/26).
- **08 Sécurité** : la mutation `os.environ["OLLAMA_HOST"]` par requête (race) est éliminée — l'adapter commit le host **une fois** au boot.
- **09 Tests** : +17 tests adaptés (`test_docling_agent_reasoning.py`, `test_ollama_provider.py`), incluant un test d'isolation concurrence (R3).
---
## Items MIN / INFO restants (non-bloquants, planifiés 0.5.1+)
| # | Origine | Item | Plan |
|---|---------|------|------|
| 03.MIN.1 | Clean Code | `StudioPage.vue` (~1450L), `ChunkPanel.vue` (~801L), `ResultTabs.vue` (~690L) | Bloc E — découper en sous-composants en 0.5.1 |
| 03.MIN.2 | Clean Code | 3 fonctions > 30 lignes / 4 fonctions > 4 paramètres | Bloc E — décomposition locale en 0.5.1 |
| 04.MIN | KISS | Wrapper `_to_response`, accessors redondants `DocumentService` | Bloc F — arbitrage en 0.5.1 |
| 05.MIN | DRY | Magic string [api/schemas.py:54](document-parser/api/schemas.py:54) | ✅ Résolu — `DOCUMENT_STATUS_UPLOADED` |
| 04.INFO | KISS | Polling `setInterval/setTimeout` imbriqués | Bloc F |
| 04.INFO | KISS | Overhead `DocumentConfig` / `IngestionConfig` dataclasses | Bloc F |
| 08.INFO | Sécurité | Default LOG_LEVEL non-borné | Sera traité avec la prochaine vague observabilité |
| 12.INFO | Performance | `find_all` implicite (pas de pagination forte) | Bloc E (split graph endpoint) en 0.5.1 |
---
## Verdict final post-remédiation : **GO**
**Justification** :
- Zéro écart CRITICAL (la règle absolue du master §3 est levée).
- Zéro écart MAJOR — tous les MAJ initiaux sont résolus.
- Score global ~94/100 ≥ 80 (seuil GO).
- Tous les audits individuels passent en GO.
- La pipeline de validation est verte : ruff + format + 446 pytest backend, ESLint + Prettier + vue-tsc + 202 vitest frontend.
**Conditions tenues** :
1. ✅ CHANGELOG `[0.5.0]` complet
2. ✅ `frontend/package.json` à `0.5.0`
3. ✅ Sécurité dev-only documentée + warning au boot sur `changeme`
4. ✅ DRY frontend (storage keys + dead apiUrl supprimé)
5. ✅ Refacto archi reasoning (port + adapter + DI) — bonus volet 1
**Reste planifié hors release 0.5.0** :
- Bloc E (découpage Vue files volumineux + signatures de fonctions back) → 0.5.1
- Bloc F (KISS micro-optimisations + INFO) → 0.5.1
La release **0.5.0 peut être taguée** depuis `release/0.5.0` une fois la branche `fix/release-0.5.0-audit-remediation` mergée.

View file

@ -1,106 +0,0 @@
# Synthese d'audit — Release 0.5.0
**Date** : 2026-04-28
**Branche** : `release/0.5.0`
**Commit audite** : `b2e0af3`
**Auditeur** : claude-code
---
## Tableau de bord
| # | Audit | Score | CRIT | MAJ | MIN | INFO | Verdict |
|----|----------------------|---------|------|-----|-----|------|------------------|
| 01 | Clean Architecture | 100 | 0 | 0 | 0 | 0 | GO |
| 02 | DDD | 91 | 0 | 1 | 1 | 0 | GO |
| 03 | Clean Code | 83 | 0 | 0 | 3 | 0 | GO |
| 04 | KISS | 87.5 | 0 | 0 | 1 | 2 | GO |
| 05 | DRY | 71 | 0 | 2 | 1 | 2 | GO CONDITIONNEL |
| 06 | SOLID | 85 | 0 | 1 | 1 | 0 | GO |
| 07 | Decouplage | 86 | 0 | 1 | 1 | 0 | GO CONDITIONNEL |
| 08 | Securite | 91 | 0 | 2 | 0 | 1 | GO CONDITIONNEL |
| 09 | Tests | 94 | 0 | 1 | 0 | 0 | GO |
| 10 | CI / Build | 91 | 0 | 1 | 0 | 0 | GO CONDITIONNEL |
| 11 | Documentation | **44** | **1**| 2 | 0 | 0 | **NO-GO** |
| 12 | Performance | 86.67 | 0 | 1 | 1 | 1 | GO CONDITIONNEL |
**Score global (moyenne simple)** : **84.2 / 100**
**Ecarts CRITICAL totaux** : **1**
**Ecarts MAJOR totaux** : **12**
**Ecarts MINOR totaux** : 8
**Ecarts INFO totaux** : 6
---
## Ecarts CRITICAL (tous audits confondus)
1. **[11] CHANGELOG.md sans section `[0.5.0]`** — `CHANGELOG.md:7`
La derniere section listee est `## [0.4.0] - 2026-04-13`. Aucune entree `[Unreleased]` ni `[0.5.0]`. Tagger 0.5.0 depuis ce HEAD livrerait un changelog mensonger qui omet les nouveautes de la release (reasoning-trace viewer, Neo4j graph storage, RAG endpoints, feature flags). **Bloquant absolu** par regle master §3.
---
## Top blockers (poids 3 / poids 2)
### Bloquant (poids 3)
- **[11] CHANGELOG sans section 0.5.0** — `CHANGELOG.md:7`
Ajouter `## [0.5.0] - 2026-04-28` avec sous-sections `Added` / `Changed` / `Fixed` listant les changements depuis 0.4.0.
### Majeurs (poids 2) — a remediar pour passer a GO
- **[11] `frontend/package.json` toujours a `0.4.0`** — `frontend/package.json:3`
Bumper a `"version": "0.5.0"`.
- **[11] Modifications de la release 0.5.0 non documentees** — `CHANGELOG.md`
Resolu en meme temps que le CRIT.
- **[08] Mot de passe Neo4j par defaut `"changeme"`** — `document-parser/infra/settings.py:30,133`
Forcer la lecture d'une variable d'environnement, supprimer le defaut, faire echouer le boot si absent en prod.
- **[08] OpenSearch security plugin desactive** — `docker-compose.yml:26`
`DISABLE_SECURITY_PLUGIN=true` doit etre off pour tout deploiement non-dev ; documenter explicitement le perimetre.
- **[05] URL d'API hardcodee en frontend** — `frontend/src/features/settings/store.ts:23`
Centraliser dans une constante `API_BASE_URL` lue depuis `import.meta.env`.
- **[05] Cles localStorage en clair, dispersees** — `frontend/src/features/settings/store.ts:24-27`
Centraliser dans une enum/objet `STORAGE_KEYS`.
- **[02] Ubiquitous language "job" vs "analysis"** — `document-parser/api/ingestion.py:49-67`
Aligner le vocabulaire HTTP sur le langage du domaine.
- **[06] LSP — `isinstance()` sur adaptateur** — `document-parser/services/analysis_service.py:356`
Supprimer le check de type concret ou exposer la capacite via le port.
- **[07] Couplage inter-feature dans les tests** — `frontend/src/features/history/navigation.test.ts:30-31,67,82-83,143-144,169-170,188`
Stubber les stores via injection au lieu d'imports directs.
- **[09] Assertions vagues `assert X is not None`** — 18 occurrences en backend
Ajouter une assertion sur la valeur attendue, pas seulement sur la presence.
- **[10] Ruff UP038 — syntaxe `isinstance()` union** — `document-parser/infra/docling_tree.py:101`
Appliquer la suggestion pyupgrade.
- **[12] I/O synchrone dans endpoint async** — `document-parser/api/documents.py:115-116`
Wrapper l'acces fichier dans `asyncio.to_thread(...)` pour ne pas bloquer la loop FastAPI.
---
## Quick wins (poids 1 — ameliorations rapides)
- **[03] Trois fichiers Vue > 300 lignes** — `frontend/src/views/StudioPage.vue` (~1450L), `frontend/src/.../ChunkPanel.vue` (~801L), `frontend/src/.../ResultTabs.vue` (~690L). Extraire des sous-composants.
- **[03] Fonctions > 30 lignes** (3 cas) et **fonctions > 4 parametres** (4 cas). Decoupage local.
- **[04] Wrapper trivial `_to_response`** — utiliser `Pydantic.model_validate()` directement.
- **[04] Accessors redondants dans `DocumentService`**.
- **[05] Magic string isolee** — `document-parser/api/schemas.py:49`.
- **[12] Polling avec setInterval/setTimeout imbriques** — `frontend/src/features/settings/store.ts:27-39` et store analysis.
---
## Verdict final : **NO-GO**
**Justification** : 1 ecart CRITICAL non resolu dans l'audit 11 (Documentation) → regle absolue du master.md §3 : `tout ecart [CRIT] non resolu = NO-GO quel que soit le score`.
Bien que le score global (84.2/100) soit confortablement au-dessus du seuil GO et que 11 audits sur 12 soient au moins en GO CONDITIONNEL, le release 0.5.0 **ne peut pas etre tague** dans cet etat.
### Conditions pour passer a GO
**Bloquant absolu** (1 action) :
1. Ajouter la section `## [0.5.0] - 2026-04-28` dans `CHANGELOG.md` avec `Added` / `Changed` / `Fixed`.
**Pour passer de GO CONDITIONNEL a GO** (4 actions, ~30 min) :
2. Bumper `frontend/package.json` a `0.5.0`.
3. Remplacer `"changeme"` par lecture d'env var obligatoire (`document-parser/infra/settings.py`).
4. Documenter / corriger la desactivation du security plugin OpenSearch (`docker-compose.yml`).
5. Centraliser l'URL d'API et les cles localStorage du frontend (`frontend/src/features/settings/store.ts`).
**Recommandation** : apres remediation des 5 points ci-dessus, re-auditer **uniquement** les audits 11, 08 et 05 (commande : `Re-audite uniquement les ecarts CRITICAL et MAJOR du rapport docs/audit/reports/release-0.5.0/summary.md`). Les autres MAJ peuvent etre planifies pour 0.5.1 sans bloquer le tag.

View file

@ -1,404 +0,0 @@
# Design: Copy paste image in Verify mode
<!--
Design doc template for Docling Studio.
One design doc per tracked issue. File path convention:
docs/design/<issue-number>-<kebab-slug>.md
Status lifecycle: Draft → In review → Accepted → Implemented (or Superseded).
Bump the Status line as the doc progresses; do not delete sections on the way.
This template is tailored to the project's architecture and conventions:
- Backend Hexagonal Architecture / ports & adapters
(domain → api/services/persistence/infra)
see docs/architecture.md
- Backend coding standards (FastAPI + Pydantic camelCase, aiosqlite,
Python snake_case internal, max 300 lines/file, 30 lines/function)
see docs/architecture/coding-standards.md
- Frontend feature-based organization (Vue 3 + Pinia, one store per
feature, Composition API, TypeScript strict, data-e2e selectors)
- E2E with Karate UI (NOT Playwright) — see e2e/CONVENTIONS.md
- Audit dimensions used at release gate — see docs/audit/master.md
- ADR process for load-bearing decisions — see docs/architecture/adr-guide.md
The `/conception` command pre-fills the header block and §1 / §2 / §12 from
the linked issue. Everything else is on the author.
-->
- **Issue:** #195
- **Title on issue:** [ENHANCEMENT] Copy paste image in Verify mode
- **Author:** Pier-Jean Malandrino
- **Date:** 2026-04-23
- **Status:** Accepted
- **Target milestone:** 0.5.0
- **Impacted layers:** <backend: domain | api | services | persistence | infra> · <frontend: features/<name> | shared | app> · <e2e> · <infra/CI>
- **Audit dimensions likely touched:** <pick from: Hexagonal Architecture · DDD · Clean Code · KISS · DRY · SOLID · Decoupling · Security · Tests · CI/Build · Documentation · Performance>
- **ADR spawned?:** <no> *(write an ADR when choosing a library, moving a boundary, or deciding **not** to do something — see `docs/architecture/adr-guide.md`)*
---
## 1. Problem
<!--
What hurts today, and for whom. Pull from the issue's Context + Current
behavior sections — keep the user's voice, do not paraphrase aggressively.
Two or three short paragraphs is usually enough. If you can't state the
problem in plain language, you are not ready to design a solution.
-->
TODO: Why this issue exists. Link the user story, incident, or upstream discussion that motivates it.
Today in Verify mode regarding image handling: TODO — describe the current baseline (upload-only? no paste target? no drag-drop?).
## 2. Goals
<!--
Concrete, verifiable outcomes. Convert the issue's acceptance criteria into
checkboxes here; the design is "done" when all are satisfied. Keep the list
small — five or fewer goals is a good smell.
-->
Users should be able to copy/paste images directly into Verify mode (e.g. from clipboard) instead of only via file upload.
- [ ] Define paste source (OS clipboard, drag-drop, screenshot)
- [ ] Define target area in Verify mode UI
- [ ] Define supported image formats and size limits
- [ ] Size / type limits are env-var configurable, carry sane defaults, and are documented in `README.md` + `docs/deployment/*` (e.g. `MAX_PASTE_IMAGE_SIZE_MB`, `PASTE_ALLOWED_IMAGE_TYPES`)
## 3. Non-goals
<!--
What this design explicitly does NOT try to solve — and, for each, where it
*should* be solved (follow-up issue, next milestone, different audit area).
This is the section that saves the review: naming the off-ramps up front
prevents scope creep. If you leave this empty, reviewers will fill it in
for you, badly.
-->
- ...
- ...
## 4. Context & constraints
<!--
The surrounding reality the design has to live in.
### Existing code surface
List the modules / files / stores this change touches. Prefer concrete paths
over prose:
- Backend: document-parser/<layer>/<file>.py
- Frontend: frontend/src/features/<name>/{store,api,ui}.ts|.vue
- Persistence: document-parser/persistence/<repo>.py + schema in database.py
- E2E: e2e/<feature>.feature
### Hexagonal Architecture constraints (backend)
The domain layer has zero imports from api / persistence / infra, and
defines ports (abstract protocols) that `infra/` adapters implement.
Persistence imports only from domain. API never imports persistence
directly — it goes through services. Call out any change that crosses
these lines or adds / moves a port.
### Deployment modes
Docling Studio ships two images (`latest-local`, `latest-remote`) driven by
`CONVERSION_ENGINE` — and a HF Space deployment on top of `latest-remote`.
State which modes this design supports, which it does not, and how the
frontend's feature flags (`chunking`, `disclaimer`) are affected.
### Hard constraints
Compatibility (SQLite schema, API contract, Pydantic DTOs), deadlines
(milestone due date), deployment target (Docker Compose, HF Space),
performance budget (matters for Performance audit), license / privacy
(matters for Security audit).
-->
### SQLite & storage limits (must be enforced upstream of the DB)
Pasted images follow the existing `documents` pattern: bytes land on disk
under `UPLOAD_DIR`, the row stores `storage_path: TEXT`. We do **not**
store base64 or BLOBs. Even so, app-level size guards must stay below
SQLite's structural limits so a malformed request can never wedge the
engine.
Relevant SQLite defaults (see https://www.sqlite.org/limits.html):
| SQLite limit | Default | Relevance |
|---|---|---|
| `SQLITE_MAX_LENGTH` | 1 GB (1 × 10⁹ bytes) | Max size of any single `TEXT` / `BLOB` cell. Irrelevant as long as we keep bytes off the DB. |
| `SQLITE_MAX_SQL_LENGTH` | 1 MB | Max length of an SQL statement incl. inlined literals. Always use parameter binding — never inline image bytes. |
| Page cache / WAL growth | n/a | Large writes bloat WAL until checkpoint; another reason to stay off-DB. |
Our own app-level limits guard against ever reaching those ceilings.
All such limits **must**: (1) carry a sane default in `infra/settings.py`,
(2) be overridable via env var, (3) be documented in `README.md` and
`docs/deployment/*`. This is consistent with how `MAX_FILE_SIZE_MB`
(default 50) is handled today.
## 5. Proposed design
<!--
The recommended approach, in enough detail that a competent engineer
outside the immediate context can implement it. Describe contracts, not
code — the PR is where code lives.
Structure this section by layer. Skip a layer if it is genuinely untouched;
do not pad.
### 5.1 Domain
New or changed dataclasses / value objects / ports in `document-parser/domain/`.
No HTTP or DB concerns here. If you are adding a port (`Protocol`), give its
full signature.
### 5.2 Persistence
Schema changes (table, columns, indexes), migration plan, aiosqlite query
shape. Note whether existing rows need a backfill.
### 5.3 Infra adapters
New or changed adapters in `document-parser/infra/` (converter, chunker,
rate limiter, settings). For new env vars, give name / default / allowed
values.
### 5.4 Services
Use-case orchestration in `document-parser/services/`. Services do NOT
implement — they delegate. Describe the call sequence, error handling,
and concurrency (how does this interact with `MAX_CONCURRENT_ANALYSES`?).
### 5.5 API
Endpoint additions / changes in `document-parser/api/`. For each:
- Method + path
- Request DTO (Pydantic, camelCase via alias_generator)
- Response DTO (camelCase; remember `pages_json` stays snake_case)
- Error responses (status codes, shape)
- Whether it is excluded from the rate limiter (like `/api/health`)
### 5.6 Frontend — feature module
Which `frontend/src/features/<name>/` folder, which Pinia store actions,
which API client calls in `api.ts`, which Vue components in `ui/`. Name
new `data-e2e` attributes here (Karate needs them).
### 5.7 Cross-cutting
Feature flags (how the backend advertises capability via `/api/health` and
how the frontend reacts), i18n strings (`shared/i18n.ts`), shared types
(`shared/types.ts`).
Prefer mermaid / ASCII for sequence and data flow. Interfaces are more
valuable than pseudocode.
-->
### 5.1 Domain
### 5.2 Persistence
### 5.3 Infra adapters
Extend `document-parser/infra/settings.py` with paste-specific limits.
Follow the existing `MAX_FILE_SIZE_MB` pattern: typed field on the
settings dataclass, `os.environ.get(...)` with a string default, cast
at load time.
| Setting | Env var | Default | Allowed | Notes |
|---|---|---|---|---|
| `max_paste_image_size_mb` | `MAX_PASTE_IMAGE_SIZE_MB` | `10` | positive int, `0` = unlimited | Must be ≤ `MAX_FILE_SIZE_MB`; upload validator rejects larger payloads before any DB write. |
| `paste_allowed_image_types` | `PASTE_ALLOWED_IMAGE_TYPES` | `image/png,image/jpeg,image/webp` | comma-separated MIME list | Enforced server-side; frontend uses the same list via `/api/health`. |
Validation happens in the API layer (upload handler) **before** the
bytes reach persistence. Any future move to BLOB storage would still
rely on these guards — they are the contract that prevents us ever
approaching `SQLITE_MAX_LENGTH`.
### 5.4 Services
### 5.5 API
### 5.6 Frontend — feature module
### 5.7 Cross-cutting (feature flags, i18n, shared types)
## 6. Alternatives considered
<!--
At least two genuine alternatives, each with a one-paragraph description
and the reason it was rejected. "Do nothing" is often a legitimate
alternative — name it if it is. Reviewers use this section to sanity-check
that the recommended design was a choice and not the first thing that
came to mind.
If one of the alternatives represents a significant architectural fork
(e.g. introducing a new service, replacing a library), spawn an ADR under
`docs/architecture/adrs/` and link it in §12 — the design doc captures the
local decision, the ADR captures the cross-cutting one.
-->
### Alternative A — <name>
- **Summary:**
- **Why not:**
### Alternative B — <name>
- **Summary:**
- **Why not:**
## 7. API & data contract
<!--
Make the wire contract explicit — this is what the frontend, e2e tests,
and any external consumer will code against.
### Endpoints
| Method | Path | Request | Response | Breaking? |
|--------|------|---------|----------|-----------|
| | | | | |
Remember:
- API serialization is camelCase (Pydantic `alias_generator`).
- Backend internals stay snake_case.
- `pages_json` is the documented exception — it carries raw
`dataclasses.asdict()` output (snake_case).
- Health endpoint (`/api/health`) may need new fields if this design adds
a feature flag.
### Persistence schema
```sql
-- ALTER TABLE / CREATE TABLE statements, with reasoning
```
### Env vars / config
All new knobs must land in `README.md` and `docs/deployment/*` (same
tables that already list `MAX_FILE_SIZE_MB`, `UPLOAD_DIR`, etc.).
| Name | Default | Allowed | Notes |
|------|---------|---------|-------|
| `MAX_PASTE_IMAGE_SIZE_MB` | `10` | positive int (`0` = unlimited) | Guards app-level payload size; must stay ≤ `MAX_FILE_SIZE_MB` to avoid double-gating confusion. |
| `PASTE_ALLOWED_IMAGE_TYPES` | `image/png,image/jpeg,image/webp` | comma-separated MIME list | Surfaced to the frontend via `/api/health` so the paste handler can reject client-side. |
SQLite ceilings (`SQLITE_MAX_LENGTH` = 1 GB, `SQLITE_MAX_SQL_LENGTH` =
1 MB) are **not** env-configurable — they are compile-time properties
of the bundled engine. Document them in `docs/deployment/*` as
background so operators understand why the app-level limits exist.
### Breaking changes
Enumerate anything a consumer must change. If there are none, say so
explicitly — "additive only" is a useful commitment.
-->
## 8. Risks & mitigations
<!--
One row per non-trivial risk. Map each to an audit dimension so the
release-gate audit has a clear hook:
| Risk | Audit dimension | Likelihood | Impact | How we notice | Mitigation / rollback |
|------|-----------------|-----------|--------|---------------|------------------------|
| | Security | | | | |
| | Performance | | | | |
| | Decoupling | | | | |
Common families to scan for:
- **Hexagonal Architecture:** cross-layer imports, leaking HTTP into domain, adapter bypassing its port
- **Security:** rate limiter bypass, path traversal on uploads, SSRF via
the remote converter, unauthenticated data exposure
- **Performance:** synchronous work on the FastAPI event loop,
unbounded queries, new work inside `MAX_CONCURRENT_ANALYSES` budget
- **Tests:** coverage gap on a critical path
- **Documentation:** missing README / env var / i18n entry
A design with "no risks identified" is a design that has not been read
carefully.
-->
| Risk | Audit dimension | Likelihood | Impact | How we notice | Mitigation / rollback |
|------|-----------------|------------|--------|---------------|------------------------|
| | | | | | |
## 9. Testing strategy
<!--
How this design will be verified. Be specific — name files / suites.
### Backend — pytest (`document-parser/tests/`)
- Unit: per-layer (`tests/domain/`, `tests/persistence/`, `tests/services/`)
- Integration: services wired with real aiosqlite + real adapters
- Architecture tests (if applicable): enforce import boundaries
### Frontend — Vitest (`frontend/src/**/*.test.ts`)
- Stores: actions / getters / mocked API
- Pure helpers (e.g. `bboxScaling.ts`-style modules): deterministic
- Components only when behavior is non-trivial; do not test markup
### E2E — Karate UI (`e2e/`)
- Use `data-e2e` selectors — never CSS classes (see e2e/CONVENTIONS.md)
- `retry()` / `waitFor()` — never `Thread.sleep()` / `delay()`
- Setup via API, verify via UI, cleanup via API
- Tag appropriately: `@critical` / `@ui` / `@smoke` / `@regression` / `@e2e`
- **Never Playwright** — Karate is the tool here.
### Manual QA
Steps the reviewer can run locally (`docker-compose.dev.yml` up, scenario
to reproduce). Keep it short — if the manual list is long, automate more.
### Performance / load
Required when the design claims a latency / throughput / memory property,
or touches the conversion hot path.
-->
## 10. Rollout & observability
<!--
How this change gets to production safely.
### Release branch
Which `release/X.Y.Z` is the target? Any coordination with a parallel
release (e.g. R&D branch)?
### Feature flag / staged rollout
Does the change hide behind a flag surfaced via `/api/health`? If so, what
flips the flag, and what is the default? HF Space deployments often need
`deploymentMode === 'huggingface'` gating.
### Observability
- Logs to add / extend (structured, low-cardinality keys)
- Metrics / counters (if added — call out any new Prometheus names)
- New error modes to watch for in `analysis_jobs.status = FAILED`
### Rollback plan
The revert that is safe to apply at any time:
- Which migration is reversible? Which is not?
- Which env var flip disables the feature without a redeploy?
- Any data cleanup needed after rollback?
Link to the existing release / ops playbooks:
- Deployment: `docs/release/*` (also surfaced via `/release:deploy`)
- Rollback: also surfaced via `/release:rollback`
- Incident: `docs/operations/*` (also surfaced via `/ops:incident`)
-->
## 11. Open questions
<!--
Things the author explicitly does not know yet, phrased as questions the
reviewer can answer or redirect. Empty is allowed once the design is
Accepted — during Draft / In review, this section is where the honest
uncertainty lives. Resolve or delete each entry before shipping.
-->
- ...
- ...
## 12. References
<!--
Links to everything a future reader would want.
-->
- **Issue:** https://github.com/scub-france/Docling-Studio/issues/195
- **Related PRs / commits:**
- **ADRs:** <ADR-NNN or "none planned">
- **Project docs:**
- Architecture: `docs/architecture.md`
- Coding standards: `docs/architecture/coding-standards.md`
- ADR guide / template: `docs/architecture/adr-guide.md`, `docs/architecture/adr-template.md`
- Audit master: `docs/audit/master.md`
- E2E conventions: `e2e/CONVENTIONS.md`
- **External:** <specs, upstream issues, dashboards, third-party docs>

View file

@ -81,22 +81,22 @@ async def list_analyses(service: ServiceDep) -> list[AnalysisResponse]:
return [_to_response(j) for j in jobs]
@router.get("/{analysis_id}", response_model=AnalysisResponse)
async def get_analysis(analysis_id: str, service: ServiceDep) -> AnalysisResponse:
@router.get("/{job_id}", response_model=AnalysisResponse)
async def get_analysis(job_id: str, service: ServiceDep) -> AnalysisResponse:
"""Get a single analysis job."""
job = await service.find_by_id(analysis_id)
job = await service.find_by_id(job_id)
if not job:
raise HTTPException(status_code=404, detail="Analysis not found")
return _to_response(job)
@router.post("/{analysis_id}/rechunk", response_model=list[ChunkResponse])
@router.post("/{job_id}/rechunk", response_model=list[ChunkResponse])
async def rechunk_analysis(
analysis_id: str, body: RechunkRequest, service: ServiceDep
job_id: str, body: RechunkRequest, service: ServiceDep
) -> list[ChunkResponse]:
"""Re-chunk a completed analysis with new chunking options."""
try:
chunks = await service.rechunk(analysis_id, body.chunkingOptions.model_dump())
chunks = await service.rechunk(job_id, body.chunkingOptions.model_dump())
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e)) from e
return [
@ -111,13 +111,13 @@ async def rechunk_analysis(
]
@router.patch("/{analysis_id}/chunks/{chunk_index}", response_model=list[ChunkResponse])
@router.patch("/{job_id}/chunks/{chunk_index}", response_model=list[ChunkResponse])
async def update_chunk_text(
analysis_id: str, chunk_index: int, body: UpdateChunkTextRequest, service: ServiceDep
job_id: str, chunk_index: int, body: UpdateChunkTextRequest, service: ServiceDep
) -> list[ChunkResponse]:
"""Update the text of a single chunk by index."""
try:
chunks = await service.update_chunk_text(analysis_id, chunk_index, body.text)
chunks = await service.update_chunk_text(job_id, chunk_index, body.text)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e)) from e
return [
@ -134,13 +134,11 @@ async def update_chunk_text(
]
@router.delete("/{analysis_id}/chunks/{chunk_index}", response_model=list[ChunkResponse])
async def delete_chunk(
analysis_id: str, chunk_index: int, service: ServiceDep
) -> list[ChunkResponse]:
@router.delete("/{job_id}/chunks/{chunk_index}", response_model=list[ChunkResponse])
async def delete_chunk(job_id: str, chunk_index: int, service: ServiceDep) -> list[ChunkResponse]:
"""Soft-delete a chunk by index (marks it as deleted)."""
try:
chunks = await service.delete_chunk(analysis_id, chunk_index)
chunks = await service.delete_chunk(job_id, chunk_index)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e)) from e
return [
@ -157,9 +155,9 @@ async def delete_chunk(
]
@router.delete("/{analysis_id}", status_code=204, response_model=None)
async def delete_analysis(analysis_id: str, service: ServiceDep) -> None:
@router.delete("/{job_id}", status_code=204, response_model=None)
async def delete_analysis(job_id: str, service: ServiceDep) -> None:
"""Delete an analysis job."""
deleted = await service.delete(analysis_id)
deleted = await service.delete(job_id)
if not deleted:
raise HTTPException(status_code=404, detail="Analysis not found")

View file

@ -2,9 +2,7 @@
from __future__ import annotations
import asyncio
import logging
from pathlib import Path
from typing import Annotated
from fastapi import APIRouter, Depends, HTTPException, Query, Request, UploadFile
@ -114,12 +112,9 @@ async def preview(
)
try:
# File read + PDF rasterisation are both blocking; offload to a
# worker thread so the event loop stays free for other requests.
file_content = await asyncio.to_thread(Path(doc.storage_path).read_bytes)
png_bytes = await asyncio.to_thread(
DocumentService.generate_preview, file_content, page=page, dpi=dpi
)
with open(doc.storage_path, "rb") as f:
file_content = f.read()
png_bytes = DocumentService.generate_preview(file_content, page=page, dpi=dpi)
return Response(content=png_bytes, media_type="image/png")
except ValueError as e:
raise HTTPException(status_code=404, detail=str(e)) from e

View file

@ -38,18 +38,18 @@ IngestionDep = Annotated[IngestionService, Depends(_get_ingestion_service)]
AnalysisDep = Annotated[AnalysisService, Depends(_get_analysis_service)]
@router.post("/{analysis_id}", response_model=IngestionResponse)
@router.post("/{job_id}", response_model=IngestionResponse)
async def ingest_analysis(
analysis_id: str,
job_id: str,
ingestion: IngestionDep,
analysis: AnalysisDep,
) -> IngestionResponse:
"""Ingest a completed analysis into the vector index.
Takes the chunks from an existing analysis, embeds them,
Takes the chunks from an existing analysis job, embeds them,
and indexes them into OpenSearch.
"""
job = await analysis.find_by_id(analysis_id)
job = await analysis.find_by_id(job_id)
if not job:
raise HTTPException(status_code=404, detail="Analysis not found")
if job.status.value != "COMPLETED":
@ -64,7 +64,7 @@ async def ingest_analysis(
chunks_json=job.chunks_json,
)
except Exception as e:
logger.exception("Ingestion failed for analysis %s", analysis_id)
logger.exception("Ingestion failed for job %s", job_id)
raise HTTPException(status_code=500, detail=f"Ingestion failed: {e}") from e
return IngestionResponse(

View file

@ -1,39 +1,43 @@
"""Reasoning API — HTTP layer over a `ReasoningRunner` port.
"""Reasoning API — live `docling-agent` runner (R&D).
`POST /api/documents/:id/reasoning` invokes the wired-up `ReasoningRunner`
against the stored `DoclingDocument` and returns a `ReasoningResultResponse`
in the same shape the v1 import dialog already consumes so the frontend
overlay code is fully reused.
`POST /api/documents/:id/rag` invokes `docling-agent`'s Chunkless RAG loop
against the stored `DoclingDocument` and returns a `RAGResult` in the same
shape the v1 import dialog already consumes so the frontend overlay code
is fully reused.
This module has zero coupling to docling-agent / mellea / docling-core. The
runner (concrete adapter in `infra/docling_agent_reasoning.py`) is set on
`app.state.reasoning_runner` at boot when `REASONING_ENABLED=true` and the
deps are importable. Otherwise it stays `None` and we 503.
Sync blocking call offloaded to a thread by the adapter so we don't stall
the event loop. No streaming at this step (see design doc §7 for v2 SSE plan).
Constraints (docling-agent v0.1.0):
- Backend is hard-wired to Ollama (`setup_local_session` in
`docling_agent/agent_models.py`). Set `OLLAMA_HOST` + `RAG_MODEL_ID` in the
environment. No OpenAI/WatsonX path without forking upstream.
- We call the private `_rag_loop` because `DoclingRAGAgent.run()` wraps the
answer in a synthetic `DoclingDocument` and never returns the iteration
trace. This is brittle track upstream for a public hook.
- Sync blocking call offloaded to a thread so we don't stall the event loop.
No streaming at this step (see design doc §7 for v2 SSE plan).
"""
from __future__ import annotations
import asyncio
import logging
import os
from fastapi import APIRouter, HTTPException, Request
from pydantic import BaseModel
from domain.ports import ReasoningParseError, ReasoningRunner
from infra.settings import settings
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/documents", tags=["reasoning"])
class ReasoningRunRequest(BaseModel):
class RagRunRequest(BaseModel):
query: str
# Optional per-run override; falls back to the runner's default model.
# Optional per-run override; falls back to settings.rag_model_id.
model_id: str | None = None
class ReasoningIterationResponse(BaseModel):
class RagIterationResponse(BaseModel):
iteration: int
section_ref: str
reason: str
@ -42,24 +46,16 @@ class ReasoningIterationResponse(BaseModel):
response: str
class ReasoningResultResponse(BaseModel):
class RagResultResponse(BaseModel):
answer: str
iterations: list[ReasoningIterationResponse]
iterations: list[RagIterationResponse]
converged: bool
@router.post("/{doc_id}/reasoning", response_model=ReasoningResultResponse)
async def run_reasoning(
doc_id: str, body: ReasoningRunRequest, request: Request
) -> ReasoningResultResponse:
runner: ReasoningRunner | None = getattr(request.app.state, "reasoning_runner", None)
if runner is None or not runner.is_available:
raise HTTPException(
status_code=503,
detail=(
"Live reasoning disabled (REASONING_ENABLED=false or docling-agent not installed)"
),
)
@router.post("/{doc_id}/rag", response_model=RagResultResponse)
async def run_rag(doc_id: str, body: RagRunRequest, request: Request) -> RagResultResponse:
if not settings.rag_enabled:
raise HTTPException(status_code=503, detail="Live reasoning disabled (RAG_ENABLED=false)")
if not body.query.strip():
raise HTTPException(status_code=400, detail="Query must not be empty")
@ -75,39 +71,78 @@ async def run_reasoning(
detail=f"No completed analysis with document_json for {doc_id}",
)
# Lazy-import docling-agent so the backend boots even if the dep isn't
# installed (R&D group). If missing, return 503 with a clear install hint.
try:
result = await runner.run(
document_json=latest.document_json,
query=body.query,
model_id=body.model_id,
from docling_agent.agents import DoclingRAGAgent
from docling_core.types.doc.document import DoclingDocument
from mellea.backends.model_ids import ModelIdentifier
except ImportError as e:
raise HTTPException(
status_code=503,
detail=f"docling-agent not installed: {e}. `pip install docling-agent mellea`.",
) from e
# Ollama client reads OLLAMA_HOST at request time; set it per-call so the
# configured host takes effect without needing to restart the server.
os.environ["OLLAMA_HOST"] = settings.ollama_host
raw_model_id = body.model_id or settings.rag_model_id
# `DoclingRAGAgent` (pydantic) validates `model_id` strictly against the
# `ModelIdentifier` dataclass from Mellea. A raw string like "gpt-oss:20b"
# is rejected even though the Ollama backend itself would accept one.
# Wrap on the Ollama axis; add other axes here if we ever fork upstream to
# support non-Ollama backends.
model_id = ModelIdentifier(ollama_name=raw_model_id)
try:
doc = DoclingDocument.model_validate_json(latest.document_json)
except Exception as e:
raise HTTPException(status_code=500, detail=f"Failed to parse document_json: {e}") from e
agent = DoclingRAGAgent(model_id=model_id, tools=[])
logger.info(
"RAG run: doc_id=%s model_id=%s ollama_host=%s query=%r",
doc_id,
model_id,
settings.ollama_host,
body.query[:120],
)
try:
# `_rag_loop` is a synchronous LLM-heavy call (N * model latency). Run
# it in a worker thread so concurrent requests don't block the loop.
result = await asyncio.to_thread(agent._rag_loop, query=body.query, doc=doc)
except IndexError as e:
# Known docling-agent bug: `_attempt_answer` / `_select_section` call
# `find_json_dicts(answer.value)[0]` without checking for an empty
# list. When the model can't produce a parseable JSON after 3
# rejection-sampling retries + 3 `select_from_failure` retries, the
# list is empty and the `[0]` crashes. It's model-dependent (some
# questions + some models trip it, others don't).
#
# Report as 502 Bad Gateway — the upstream LLM couldn't produce a
# usable response, not our fault — with a message the UI can show
# to the user so they pick another model or rephrase.
logger.warning(
"docling-agent produced no parseable JSON for doc=%s model=%s query=%r",
doc_id,
raw_model_id,
body.query[:120],
)
except ReasoningParseError as e:
# The upstream LLM couldn't produce a parseable answer after retries.
# 502 Bad Gateway — not our fault — with guidance the UI can show.
raise HTTPException(
status_code=502,
detail=(
f"The model '{e.model_id}' couldn't produce a parseable "
"answer after retries. Try a different model (e.g. "
"mistral-small3.2) or rephrase the question."
f"The model '{raw_model_id}' couldn't produce a parseable "
"answer after retries. Try a different model (e.g. mistral-small3.2) "
"or rephrase the question."
),
) from e
except Exception as e:
logger.exception("Reasoning loop failed for doc %s", doc_id)
raise HTTPException(status_code=500, detail=f"Reasoning loop failed: {e}") from e
logger.exception("RAG loop failed for doc %s", doc_id)
raise HTTPException(status_code=500, detail=f"RAG loop failed: {e}") from e
return ReasoningResultResponse(
return RagResultResponse(
answer=result.answer,
iterations=[
ReasoningIterationResponse(
iteration=it.iteration,
section_ref=it.section_ref,
reason=it.reason,
section_text_length=it.section_text_length,
can_answer=it.can_answer,
response=it.response,
)
for it in result.iterations
],
iterations=[RagIterationResponse(**it.model_dump()) for it in result.iterations],
converged=result.converged,
)

View file

@ -10,11 +10,6 @@ from datetime import datetime
from pydantic import AliasChoices, BaseModel, ConfigDict, Field, field_validator
# Document lifecycle status — currently single-state (uploaded). Kept as a
# constant so future statuses (e.g. "archived", "deleted") can extend the
# vocabulary without hunting magic strings across the codebase.
DOCUMENT_STATUS_UPLOADED = "uploaded"
def _to_camel(name: str) -> str:
parts = name.split("_")
@ -39,19 +34,17 @@ class HealthResponse(_CamelModel):
database: str
max_page_count: int | None = None
max_file_size_mb: int | None = None
max_paste_image_size_mb: int | None = None
paste_allowed_image_types: list[str] = Field(default_factory=list)
ingestion_available: bool = False
# True when the live-reasoning runner (docling-agent + Ollama) is
# available: REASONING_ENABLED=true AND deps importable. Doesn't imply
# Ollama itself is reachable — that's checked per-call.
reasoning_available: bool = False
# available: RAG_ENABLED=true AND deps importable. Doesn't imply Ollama
# itself is reachable — that's checked per-call.
rag_available: bool = False
class DocumentResponse(_CamelModel):
id: str
filename: str
status: str = DOCUMENT_STATUS_UPLOADED
status: str = "uploaded" # Document status (always "uploaded" for now)
content_type: str | None = None
file_size: int | None = None
page_count: int | None = None

View file

@ -15,27 +15,10 @@ if TYPE_CHECKING:
ChunkResult,
ConversionOptions,
ConversionResult,
LLMProviderType,
ReasoningResult,
)
from domain.vector_schema import IndexedChunk, SearchResult
class ReasoningParseError(Exception):
"""Raised by a `ReasoningRunner` when the upstream LLM couldn't produce a
parseable answer after retries e.g. docling-agent's known IndexError on
`find_json_dicts(...)[0]` when the model fails rejection-sampling.
Carries the model identifier so the API layer can surface it to the user
without leaking adapter internals.
"""
def __init__(self, model_id: str, reason: str = "no parseable answer") -> None:
super().__init__(f"{model_id}: {reason}")
self.model_id = model_id
self.reason = reason
class DocumentConverter(Protocol):
"""Port for document conversion.
@ -51,15 +34,6 @@ class DocumentConverter(Protocol):
page_range: tuple[int, int] | None = None,
) -> ConversionResult: ...
@property
def supports_page_batching(self) -> bool:
"""True if the orchestrator may slice a long document into page
batches (calling `convert` with a `page_range`) and merge the
results. Local in-process converters set this to True; remote
converters that handle batching themselves return False so the
orchestrator passes the full document through in one call."""
...
class DocumentChunker(Protocol):
"""Port for document chunking.
@ -168,73 +142,3 @@ class VectorStore(Protocol):
async def delete_document(self, index_name: str, doc_id: str) -> int:
"""Delete all chunks for a document from the index. Returns count deleted."""
...
async def ping(self) -> bool:
"""Cheap reachability probe — True if the backing store responds.
Used by health checks; should not throw."""
...
@runtime_checkable
class LLMProvider(Protocol):
"""Connection-level abstraction over an LLM backend.
A provider carries the host/base-URL, the default model identifier, and a
type tag that adapters can dispatch on. The reasoning runner consumes a
provider it doesn't construct one — so the runner stays decoupled from
Ollama-vs-OpenAI-vs-WatsonX wiring.
Today only `OllamaProvider` (in `infra/llm/`) is implemented because
docling-agent v0.1.0 is hardwired to Ollama via mellea's
`setup_local_session`. Adding a non-Ollama provider requires either
docling-agent upstream support or a fork (track
https://github.com/docling-project/docling-agent/issues/26 + provider
abstraction work upstream).
"""
@property
def type(self) -> LLMProviderType: ...
@property
def host(self) -> str: ...
@property
def default_model_id(self) -> str: ...
def health_check(self) -> bool:
"""Lightweight reachability probe. Returns True if the provider looks
usable. Implementations should be cheap (no model load, no inference).
"""
...
@runtime_checkable
class ReasoningRunner(Protocol):
"""Port for live reasoning over a previously-converted document.
Takes the serialized DoclingDocument JSON + a user query + optional
per-call model override, returns a `ReasoningResult` (answer + iteration
trace + convergence flag).
Adapters MUST translate upstream parsing failures into
`ReasoningParseError`. Other exceptions propagate as-is the API layer
maps them to 5xx.
"""
@property
def is_available(self) -> bool:
"""True if the runner can serve requests (deps importable + provider
wired). Used by the API layer to short-circuit with a 503 instead of
attempting a doomed call."""
...
async def run(
self,
*,
document_json: str,
query: str,
model_id: str | None = None,
) -> ReasoningResult:
"""Execute the reasoning loop. `model_id` overrides the provider's
default for this call only."""
...

View file

@ -7,7 +7,6 @@ They have ZERO external dependencies (no docling, no HTTP, no DB).
from __future__ import annotations
from dataclasses import dataclass, field
from enum import StrEnum
# US Letter page dimensions (points) — fallback when page size is unknown
DEFAULT_PAGE_WIDTH: float = 612.0
@ -97,43 +96,3 @@ class ChunkResult:
token_count: int = 0
bboxes: list[ChunkBbox] = field(default_factory=list)
doc_items: list[ChunkDocItem] = field(default_factory=list)
# --- Reasoning (live docling-agent runner) -----------------------------------
class LLMProviderType(StrEnum):
"""LLM backends the reasoning runner can talk to.
Today only OLLAMA is realizable: docling-agent v0.1.0 is hardwired to
Ollama via mellea's `setup_local_session`. Other variants are kept here
to make the abstraction visible and prepare future backends adding one
requires either docling-agent upstream support (see
https://github.com/docling-project/docling-agent/issues/26) or a fork.
"""
OLLAMA = "ollama"
@dataclass(frozen=True)
class ReasoningIteration:
"""One step of the reasoning loop — section the agent visited and what
it concluded. Mirrors the upstream docling-agent `RAGIteration` shape so
serialization stays 1:1 with externally-produced traces."""
iteration: int
section_ref: str
reason: str
section_text_length: int
can_answer: bool
response: str
@dataclass(frozen=True)
class ReasoningResult:
"""Full output of a reasoning run: final answer, the path the agent
walked through the document, and whether the loop converged."""
answer: str
iterations: list[ReasoningIteration]
converged: bool

View file

@ -1,129 +0,0 @@
"""docling-agent reasoning runner adapter.
Implements `ReasoningRunner` for an `OllamaProvider`-backed `LLMProvider`.
Encapsulates everything that talks to docling-agent / mellea so neither the
domain nor the API layer depends on those packages.
Why we still call the private `_rag_loop`: `DoclingRAGAgent.run()` wraps the
answer in a synthetic `DoclingDocument` and discards the iteration trace.
Tracked upstream at https://github.com/docling-project/docling-agent/issues/26
switch to the public surface once the issue lands.
"""
from __future__ import annotations
import asyncio
import logging
import os
from domain.ports import LLMProvider, ReasoningParseError
from domain.value_objects import (
LLMProviderType,
ReasoningIteration,
ReasoningResult,
)
logger = logging.getLogger(__name__)
def deps_present() -> bool:
"""Import-check for the heavy reasoning deps. Used by the DI wire-up to
decide whether to instantiate the runner at all (so the backend boots
cleanly when docling-agent + mellea aren't installed)."""
try:
import docling_agent.agents # noqa: F401
import mellea # noqa: F401
except ImportError:
return False
return True
class DoclingAgentReasoningRunner:
"""ReasoningRunner adapter wrapping docling-agent + mellea.
The provider's host is committed to the process-wide `OLLAMA_HOST` env
var at construction time Ollama's Python client reads it on session
creation. Setting it once at boot (instead of per-request) eliminates the
cross-request race the previous implementation exposed.
"""
def __init__(self, provider: LLMProvider) -> None:
if provider.type is not LLMProviderType.OLLAMA:
raise NotImplementedError(
f"docling-agent v0.1.0 only supports Ollama, got provider type "
f"{provider.type!r}. See "
f"https://github.com/docling-project/docling-agent/issues/26"
)
self._provider = provider
self._deps_ok = deps_present()
# Commit the host at boot — concurrent `run()` calls then share the
# same value with no racy mutation.
os.environ["OLLAMA_HOST"] = provider.host
@property
def is_available(self) -> bool:
return self._deps_ok
async def run(
self,
*,
document_json: str,
query: str,
model_id: str | None = None,
) -> ReasoningResult:
if not self._deps_ok:
raise RuntimeError("docling-agent / mellea not importable — cannot run reasoning")
# Lazy imports keep the module loadable when deps are missing (the
# runner is only ever instantiated when `deps_present()` is True, but
# this also makes the import surface explicit).
from docling_agent.agents import DoclingRAGAgent
from docling_core.types.doc.document import DoclingDocument
from mellea.backends.model_ids import ModelIdentifier
raw_model_id = model_id or self._provider.default_model_id
# `DoclingRAGAgent` (pydantic) validates `model_id` strictly against
# `ModelIdentifier` from mellea. Wrapping on the Ollama axis is the
# only realizable path today (cf. LLMProvider docstring).
wrapped_model_id = ModelIdentifier(ollama_name=raw_model_id)
try:
doc = DoclingDocument.model_validate_json(document_json)
except Exception as e:
raise RuntimeError(f"Failed to parse document_json: {e}") from e
agent = DoclingRAGAgent(model_id=wrapped_model_id, tools=[])
logger.info(
"Reasoning run: model_id=%s ollama_host=%s query=%r",
raw_model_id,
self._provider.host,
query[:120],
)
try:
# `_rag_loop` is sync + LLM-heavy (N * model latency). Offload to
# a worker thread so concurrent calls don't block the event loop.
# Private API kept until docling-agent#26 lands.
raw_result = await asyncio.to_thread(agent._rag_loop, query=query, doc=doc)
except IndexError as e:
# docling-agent v0.1.0 bug: `_attempt_answer` / `_select_section`
# call `find_json_dicts(answer.value)[0]` without handling an
# empty list. When the model can't produce a parseable JSON after
# 3 rejection-sampling retries + 3 `select_from_failure` retries,
# the list is empty and `[0]` raises IndexError. Translate to a
# domain-level error the API can map to 502.
logger.warning(
"docling-agent produced no parseable JSON for model=%s query=%r",
raw_model_id,
query[:120],
)
raise ReasoningParseError(
model_id=raw_model_id,
reason="no parseable answer after retries",
) from e
return ReasoningResult(
answer=raw_result.answer,
iterations=[ReasoningIteration(**it.model_dump()) for it in raw_result.iterations],
converged=raw_result.converged,
)

View file

@ -17,10 +17,8 @@ from itertools import pairwise
from typing import Any
from infra.docling_tree import (
build_collapse_index,
dfs_order,
element_label,
is_inline_group,
iter_items,
iter_pages,
iter_provs,
@ -29,22 +27,15 @@ from infra.docling_tree import (
from infra.neo4j.queries import GraphPayload
def _element_node(
doc_id: str,
item: dict[str, Any],
provs: list[dict[str, Any]],
*,
text_override: str | None = None,
) -> dict[str, Any]:
def _element_node(doc_id: str, item: dict[str, Any], provs: list[dict[str, Any]]) -> dict[str, Any]:
first_page = provs[0].get("page_no") if provs else None
raw_text = text_override if text_override is not None else (item.get("text") or "")
return {
"id": f"elem::{item.get('self_ref')}",
"group": "element",
"label": element_label(item.get("label") or ""),
"docling_label": (item.get("label") or "").lower(),
"self_ref": item.get("self_ref"),
"text": raw_text[:200],
"text": (item.get("text") or "")[:200],
"prov_page": first_page,
"provs": provs,
"level": item.get("level"),
@ -121,10 +112,6 @@ def build_graph_payload(
for p in pages_raw:
nodes.append(_page_node(doc_id, p))
# Issue #197: collapse Docling noise — InlineGroup style runs and the
# internal text labels Docling extracts from pictures/charts.
skip_refs, inline_meta = build_collapse_index(doc_data)
# Element nodes + collect parent/body metadata for edges below. The
# `element_idx` mirrors TreeWriter's `enumerate(elements)` so PARENT_OF
# carries the same `order` the Neo4j projection does.
@ -132,17 +119,11 @@ def build_graph_payload(
element_idx = 0
for _, item in iter_items(doc_data):
ref = item.get("self_ref")
if not ref or ref in skip_refs:
if not ref:
continue
by_ref[ref] = item
if is_inline_group(item):
meta = inline_meta.get(ref, {"text": "", "provs": []})
provs = meta["provs"]
text_override: str | None = meta["text"]
else:
provs = iter_provs(item)
text_override = None
nodes.append(_element_node(doc_id, item, provs, text_override=text_override))
provs = iter_provs(item)
nodes.append(_element_node(doc_id, item, provs))
pref = parent_ref(item)
if pref == "#/body":
@ -162,8 +143,8 @@ def build_graph_payload(
element_idx += 1
# NEXT chain (DFS pre-order from body), inline-group children skipped.
for a, b in pairwise(dfs_order(doc_data, skip_refs)):
# NEXT chain (DFS pre-order from body).
for a, b in pairwise(dfs_order(doc_data)):
if a in by_ref and b in by_ref:
edges.append(_edge(f"elem::{a}", f"elem::{b}", "NEXT"))

View file

@ -25,7 +25,6 @@ LABEL_MAP: dict[str, str] = {
"text": "Paragraph",
"list_item": "ListItem",
"list": "List", # distinct from :ListItem — a list is a container
"inline": "Paragraph", # see issue #197 — collapsed into one paragraph node
"table": "Table",
"picture": "Figure",
"formula": "Formula",
@ -45,28 +44,6 @@ def element_label(docling_label: str) -> str:
return LABEL_MAP.get(docling_label.lower(), DEFAULT_LABEL)
def is_inline_group(item: dict[str, Any]) -> bool:
"""True iff `item` is a Docling InlineGroup (paragraph of mixed style runs).
Docling represents an inline-styled paragraph as one entry in `groups[]`
(label `inline`) plus N entries in `texts[]` (label `text`), one per style
run. We collapse them into a single Paragraph projection see #197.
"""
return (item.get("label") or "").lower() == "inline"
def is_picture(item: dict[str, Any]) -> bool:
"""True iff `item` is a Docling PictureItem (figure or chart).
A `picture` keeps its node in the graph (it IS the figure), but its
`children` internal text labels extracted from a flowchart, diagram,
chart axis labels are noise for graph readability and are skipped.
Captions live in a separate `captions` field on the picture, not in
`children`, so they are unaffected by this skip.
"""
return (item.get("label") or "").lower() in {"picture", "chart"}
def iter_items(doc_data: dict[str, Any]) -> Iterator[tuple[str, dict[str, Any]]]:
"""Yield every item from texts/tables/pictures/groups with its source list key."""
for key in ("texts", "tables", "pictures", "groups"):
@ -98,7 +75,7 @@ def iter_provs(item: dict[str, Any]) -> list[dict[str, Any]]:
t_ = float(bbox.get("t", 0.0) or 0.0)
r_ = float(bbox.get("r", 0.0) or 0.0)
b_ = float(bbox.get("b", 0.0) or 0.0)
elif isinstance(bbox, list | tuple) and len(bbox) >= 4:
elif isinstance(bbox, (list, tuple)) and len(bbox) >= 4:
l_, t_, r_, b_ = (float(x) for x in bbox[:4])
coord_origin = (bbox.get("coord_origin") if isinstance(bbox, dict) else None) or "TOPLEFT"
charspan = p.get("charspan") or []
@ -118,15 +95,8 @@ def iter_provs(item: dict[str, Any]) -> list[dict[str, Any]]:
return rows
def dfs_order(doc_data: dict[str, Any], skip_refs: set[str] | None = None) -> list[str]:
"""Return `self_ref`s in reading order (DFS pre-order from body).
`skip_refs` (typically the set returned by `build_inline_index`) is omitted
from the chain. Inline groups themselves are emitted but the walk does not
recurse into their style-run children, so the resulting order references
only nodes that survive the InlineGroup collapse.
"""
skip = skip_refs or set()
def dfs_order(doc_data: dict[str, Any]) -> list[str]:
"""Return `self_ref`s in reading order (DFS pre-order from body)."""
by_ref: dict[str, dict[str, Any]] = {}
for _, item in iter_items(doc_data):
ref = item.get("self_ref")
@ -140,127 +110,17 @@ def dfs_order(doc_data: dict[str, Any], skip_refs: set[str] | None = None) -> li
return
for ch in children:
ref = ch.get("$ref") or ch.get("cref")
if not ref or ref in skip:
if not ref:
continue
order.append(ref)
child = by_ref.get(ref)
if child and not is_inline_group(child):
if child:
walk(child.get("children"))
walk(body.get("children"))
return order
def build_collapse_index(
doc_data: dict[str, Any],
) -> tuple[set[str], dict[str, dict[str, Any]]]:
"""Pre-compute graph-projection collapses for a serialized DoclingDocument.
Two cases produce noise nodes if mirrored 1:1 see issue #197:
1. **InlineGroup** Docling emits one `groups[]` entry (label `inline`)
plus N `texts[]` style runs. We collapse the children into the group,
which is then projected as a single `:Paragraph` with concatenated
text and the union of children's provs.
2. **Picture / Chart** internal text labels extracted from flowcharts,
diagrams or chart axes hang off the picture's `children`. The picture
node itself stays, but its descendants are skipped so the graph isn't
drowned in dozens of tiny labels.
Returns `(skip_refs, inline_meta)`:
- `skip_refs`: every `self_ref` to drop from element / edge projections.
- `inline_meta[group_ref]`: `{"text": str, "provs": list[dict]}`
override values for the inline group projection. Pictures don't have
an entry here; they keep their own text/prov.
"""
by_ref: dict[str, dict[str, Any]] = {}
for _, item in iter_items(doc_data):
ref = item.get("self_ref")
if ref:
by_ref[ref] = item
skip_refs: set[str] = set()
inline_meta: dict[str, dict[str, Any]] = {}
for item in by_ref.values():
ref = item.get("self_ref") or ""
if not ref:
continue
if is_inline_group(item):
text_parts, provs = _collect_inline_descendants(ref, by_ref, skip_refs)
# Re-index prov order so the resulting :Provenance nodes are 0..N-1
# contiguous instead of carrying each child's individual indices.
for idx, prov in enumerate(provs):
prov["order"] = idx
inline_meta[ref] = {
"text": " ".join(text_parts),
"provs": provs,
}
elif is_picture(item):
_collect_descendants(ref, by_ref, skip_refs)
return skip_refs, inline_meta
def _collect_descendants(
root_ref: str,
by_ref: dict[str, dict[str, Any]],
skip_refs: set[str],
) -> None:
"""DFS `root_ref`'s subtree and add every descendant to `skip_refs`.
Used for picture children we just want them dropped, not aggregated.
"""
def walk(ref: str) -> None:
item = by_ref.get(ref)
if item is None:
return
for ch in item.get("children") or []:
child_ref = ch.get("$ref") or ch.get("cref")
if not child_ref or child_ref in skip_refs:
continue
skip_refs.add(child_ref)
walk(child_ref)
walk(root_ref)
def _collect_inline_descendants(
group_ref: str,
by_ref: dict[str, dict[str, Any]],
skip_refs: set[str],
) -> tuple[list[str], list[dict[str, Any]]]:
"""DFS an inline group's subtree, returning its text parts and provs in
document order. `skip_refs` is mutated with every visited descendant."""
text_parts: list[str] = []
provs: list[dict[str, Any]] = []
def walk(ref: str) -> None:
item = by_ref.get(ref)
if item is None:
return
for ch in item.get("children") or []:
child_ref = ch.get("$ref") or ch.get("cref")
if not child_ref or child_ref in skip_refs:
continue
skip_refs.add(child_ref)
child = by_ref.get(child_ref)
if child is None:
continue
if is_inline_group(child):
walk(child_ref)
continue
text = child.get("text") or ""
if text:
text_parts.append(text)
provs.extend(iter_provs(child))
walk(group_ref)
return text_parts, provs
def iter_pages(doc_data: dict[str, Any]) -> Iterator[dict[str, Any]]:
"""Yield page dicts with `page_no`, `width`, `height` from the `pages` map."""
for page_no_str, page_obj in (doc_data.get("pages") or {}).items():

View file

@ -1,47 +0,0 @@
"""Ollama LLM provider adapter.
Implements `LLMProvider` for a locally-reachable Ollama instance. Holds the
host URL + default model id; exposes a cheap `health_check` (HEAD `/api/tags`)
that doesn't load any model.
"""
from __future__ import annotations
import logging
import httpx
from domain.value_objects import LLMProviderType
logger = logging.getLogger(__name__)
# Ollama's tags endpoint returns 200 with an empty list even on a fresh
# install — perfect for a "is the daemon up?" probe.
_HEALTH_PATH = "/api/tags"
_HEALTH_TIMEOUT_SECONDS = 1.5
class OllamaProvider:
def __init__(self, host: str, default_model_id: str) -> None:
self._host = host.rstrip("/")
self._default_model_id = default_model_id
@property
def type(self) -> LLMProviderType:
return LLMProviderType.OLLAMA
@property
def host(self) -> str:
return self._host
@property
def default_model_id(self) -> str:
return self._default_model_id
def health_check(self) -> bool:
try:
resp = httpx.get(f"{self._host}{_HEALTH_PATH}", timeout=_HEALTH_TIMEOUT_SECONDS)
except httpx.HTTPError as e:
logger.debug("Ollama health check failed for %s: %s", self._host, e)
return False
return resp.status_code == 200

View file

@ -281,10 +281,6 @@ def _convert_sync(
class LocalConverter:
"""Adapter that runs Docling locally as a Python library."""
# In-process — the orchestrator may slice long docs into page batches
# and merge results (cf. AnalysisService._run_batched_conversion).
supports_page_batching: bool = True
async def convert(
self,
file_path: str,

View file

@ -17,10 +17,8 @@ from datetime import UTC, datetime
from typing import TYPE_CHECKING, Any
from infra.docling_tree import (
build_collapse_index,
dfs_order,
element_label,
is_inline_group,
iter_items,
iter_pages,
iter_provs,
@ -84,12 +82,6 @@ async def write_document(
doc_data = json.loads(document_json)
ingested_at = datetime.now(tz=UTC).isoformat()
# Issue #197: collapse two noise patterns from Docling into the projection.
# InlineGroups (paragraph style runs) are merged into a single :Paragraph,
# and Pictures' internal text labels (flowchart/diagram/chart annotations)
# are dropped. Both produce refs that land in `skip_refs`.
skip_refs, inline_meta = build_collapse_index(doc_data)
elements: list[dict[str, Any]] = []
# Parallel list: one row per Provenance — each refers back to its owner
# element via `self_ref`, so we can batch MATCH-and-link after both node
@ -97,29 +89,22 @@ async def write_document(
provenances: list[dict[str, Any]] = []
for _, item in iter_items(doc_data):
ref = item.get("self_ref")
if not ref or ref in skip_refs:
if not ref:
continue
specific = element_label(item.get("label") or "")
props = _element_props(item, doc_id)
if is_inline_group(item):
meta = inline_meta.get(ref, {"text": "", "provs": []})
props["text"] = meta["text"]
item_provs = meta["provs"]
else:
item_provs = iter_provs(item)
elements.append(
{
"specific_label": specific,
"parent_ref": parent_ref(item),
**props,
**_element_props(item, doc_id),
}
)
for prov in item_provs:
for prov in iter_provs(item):
provenances.append({"doc_id": doc_id, "self_ref": ref, **prov})
pages: list[dict[str, Any]] = [{"doc_id": doc_id, **p} for p in iter_pages(doc_data)]
reading_order = dfs_order(doc_data, skip_refs)
reading_order = dfs_order(doc_data)
async with (
neo.driver.session(database=neo.database) as session,

View file

@ -84,14 +84,6 @@ class OpenSearchStore:
"""Close the underlying HTTP connection pool."""
await self._client.close()
async def ping(self) -> bool:
"""Reachability probe — calls OpenSearch `/` (cluster info) once."""
try:
info = await self._client.info()
return bool(info)
except Exception:
return False
# -- VectorStore protocol methods ------------------------------------------
async def ensure_index(self, index_name: str, mapping: dict) -> None:

View file

@ -57,11 +57,6 @@ _LABEL_MAP = {
class ServeConverter:
"""Adapter that delegates document conversion to a remote Docling Serve instance."""
# Docling Serve handles batching server-side; slicing into page batches
# client-side would just multiply HTTP roundtrips for no benefit, so
# the orchestrator passes the full document through in a single call.
supports_page_batching: bool = False
def __init__(
self,
base_url: str,

View file

@ -27,31 +27,17 @@ class Settings:
embedding_url: str = "" # empty = disabled (e.g. http://localhost:8001)
neo4j_uri: str = "" # empty = disabled (e.g. bolt://neo4j:7687)
neo4j_user: str = "neo4j"
# DEV DEFAULT — the dev compose stack uses "changeme" so `docker compose
# up` works out of the box. The backend logs a loud warning at boot if
# Neo4j is wired (NEO4J_URI set) AND the password is still the default,
# so prod operators notice if they inherited it by accident. Real
# deployments must override NEO4J_PASSWORD.
neo4j_password: str = "changeme"
# Live reasoning via docling-agent — off by default (heavy deps, needs an
# Ollama host reachable from the backend). Toggle REASONING_ENABLED=true +
# point OLLAMA_HOST at a running instance (default http://localhost:11434).
reasoning_enabled: bool = False
# LLM backend the reasoning runner talks to. Today only "ollama" is
# realizable (docling-agent is hardwired to Ollama via mellea); kept as a
# config knob to make the LLMProvider abstraction visible and prepare the
# ground for additional backends.
llm_provider_type: str = "ollama"
# Ollama host reachable from the backend). Toggle RAG_ENABLED=true + point
# OLLAMA_HOST at a running instance (default http://localhost:11434).
rag_enabled: bool = False
ollama_host: str = "http://localhost:11434"
reasoning_model_id: str = "gpt-oss:20b" # matches docling-agent's example_05
rag_model_id: str = "gpt-oss:20b" # matches docling-agent's example_05
opensearch_default_limit: int = 1000 # max chunks returned by get_chunks
embedding_dimension: int = 384 # Granite Embedding 30M / all-MiniLM-L6-v2
upload_dir: str = "./uploads"
db_path: str = "./data/docling_studio.db"
max_paste_image_size_mb: int = 10 # clipboard-paste image limit in MB (0 = unlimited)
paste_allowed_image_types: list[str] = field(
default_factory=lambda: ["image/png", "image/jpeg", "image/webp"]
)
cors_origins: list[str] = field(
default_factory=lambda: ["http://localhost:3000", "http://localhost:5173"]
)
@ -74,12 +60,6 @@ class Settings:
errors.append(f"max_file_size must be >= 0 (got {self.max_file_size})")
if self.max_file_size_mb < 0:
errors.append(f"max_file_size_mb must be >= 0 (got {self.max_file_size_mb})")
if self.max_paste_image_size_mb < 0:
errors.append(
f"max_paste_image_size_mb must be >= 0 (got {self.max_paste_image_size_mb})"
)
if not self.paste_allowed_image_types:
errors.append("paste_allowed_image_types must not be empty")
if self.rate_limit_rpm < 0:
errors.append(f"rate_limit_rpm must be >= 0 (got {self.rate_limit_rpm})")
if self.batch_page_size < 0:
@ -113,9 +93,6 @@ class Settings:
def from_env(cls) -> Settings:
"""Build a Settings instance from environment variables."""
cors_raw = os.environ.get("CORS_ORIGINS", "http://localhost:3000,http://localhost:5173")
paste_types_raw = os.environ.get(
"PASTE_ALLOWED_IMAGE_TYPES", "image/png,image/jpeg,image/webp"
)
return cls(
app_version=os.environ.get("APP_VERSION", "dev"),
conversion_engine=os.environ.get("CONVERSION_ENGINE", "local"),
@ -141,17 +118,14 @@ class Settings:
neo4j_uri=os.environ.get("NEO4J_URI", ""),
neo4j_user=os.environ.get("NEO4J_USER", "neo4j"),
neo4j_password=os.environ.get("NEO4J_PASSWORD", "changeme"),
reasoning_enabled=os.environ.get("REASONING_ENABLED", "false").lower()
rag_enabled=os.environ.get("RAG_ENABLED", "false").lower()
in ("1", "true", "yes", "on"),
llm_provider_type=os.environ.get("LLM_PROVIDER_TYPE", "ollama"),
ollama_host=os.environ.get("OLLAMA_HOST", "http://localhost:11434"),
reasoning_model_id=os.environ.get("REASONING_MODEL_ID", "gpt-oss:20b"),
rag_model_id=os.environ.get("RAG_MODEL_ID", "gpt-oss:20b"),
opensearch_default_limit=int(os.environ.get("OPENSEARCH_DEFAULT_LIMIT", "1000")),
embedding_dimension=int(os.environ.get("EMBEDDING_DIMENSION", "384")),
upload_dir=os.environ.get("UPLOAD_DIR", "./uploads"),
db_path=os.environ.get("DB_PATH", "./data/docling_studio.db"),
max_paste_image_size_mb=int(os.environ.get("MAX_PASTE_IMAGE_SIZE_MB", "10")),
paste_allowed_image_types=[t.strip() for t in paste_types_raw.split(",") if t.strip()],
cors_origins=[o.strip() for o in cors_raw.split(",")],
)

View file

@ -101,15 +101,6 @@ async def _init_neo4j():
logger.info("Neo4j disabled (NEO4J_URI not set)")
return None
if settings.neo4j_password == "changeme":
# The dev compose stack ships with "changeme" so `docker compose up`
# works immediately. Anyone running the backend against a non-dev
# Neo4j with this password almost certainly forgot to override it.
logger.warning(
"Neo4j is configured with the dev default password 'changeme'. "
"Override NEO4J_PASSWORD before deploying outside localhost."
)
from infra.neo4j import bootstrap_schema, get_driver
try:
@ -232,48 +223,12 @@ app.include_router(graph_router)
# Live reasoning (docling-agent runner). Router is mounted unconditionally so
# the route is introspectable in OpenAPI; the handler itself 503s when
# `REASONING_ENABLED` is off or the deps aren't installed.
# `RAG_ENABLED` is off or the deps aren't installed.
from api.reasoning import router as reasoning_router # noqa: E402
from infra.docling_agent_reasoning import DoclingAgentReasoningRunner # noqa: E402
from infra.docling_agent_reasoning import deps_present as _reasoning_deps_present # noqa: E402
from infra.llm.ollama_provider import OllamaProvider # noqa: E402
app.include_router(reasoning_router)
def _build_reasoning_runner() -> DoclingAgentReasoningRunner | None:
"""Wire the reasoning runner if `REASONING_ENABLED=true` and deps are
importable. Today only `LLM_PROVIDER_TYPE=ollama` is supported (cf.
`LLMProvider` docstring); other values fall through to a logged warning
+ None so the rest of the app boots cleanly.
"""
if not settings.reasoning_enabled:
return None
if not _reasoning_deps_present():
logger.warning(
"REASONING_ENABLED=true but docling-agent / mellea not importable — "
"reasoning runner disabled"
)
return None
if settings.llm_provider_type != "ollama":
logger.warning(
"Unsupported LLM_PROVIDER_TYPE=%s — reasoning runner disabled (only "
"'ollama' is realizable today, see "
"https://github.com/docling-project/docling-agent/issues/26)",
settings.llm_provider_type,
)
return None
provider = OllamaProvider(
host=settings.ollama_host,
default_model_id=settings.reasoning_model_id,
)
return DoclingAgentReasoningRunner(provider=provider)
app.state.reasoning_runner = _build_reasoning_runner()
@app.get("/api/health", response_model=HealthResponse)
async def health() -> HealthResponse:
"""Health check endpoint — verifies database connectivity."""
@ -286,7 +241,6 @@ async def health() -> HealthResponse:
logger.warning("Health check: database unreachable", exc_info=True)
status = "ok" if db_status == "ok" else "degraded"
runner = getattr(app.state, "reasoning_runner", None)
return HealthResponse(
status=status,
version=settings.app_version,
@ -295,13 +249,19 @@ async def health() -> HealthResponse:
database=db_status,
max_page_count=settings.max_page_count if settings.max_page_count > 0 else None,
max_file_size_mb=settings.max_file_size_mb if settings.max_file_size_mb > 0 else None,
max_paste_image_size_mb=(
settings.max_paste_image_size_mb if settings.max_paste_image_size_mb > 0 else None
),
paste_allowed_image_types=settings.paste_allowed_image_types,
ingestion_available=getattr(app.state, "ingestion_service", None) is not None,
# True when the runner is wired and reports itself available. The
# actual Ollama reachability is checked lazily at call-time to avoid
# True when the live-reasoning runner is wired (flag on + deps present).
# The actual Ollama reachability is checked lazily at call-time to avoid
# blocking health checks on the LLM host.
reasoning_available=runner is not None and runner.is_available,
rag_available=settings.rag_enabled and _rag_deps_present(),
)
def _rag_deps_present() -> bool:
"""Import-check only — does not hit Ollama."""
try:
import docling_agent.agents # noqa: F401
import mellea # noqa: F401
except ImportError:
return False
return True

View file

@ -10,8 +10,6 @@ pypdfium2>=4.0.0,<5.0.0
opensearch-py[async]>=2.6.0,<3.0.0
neo4j>=5.15.0,<6.0.0
# R&D reasoning-trace live runner — calls docling-agent's `_rag_loop` over
# an Ollama backend. Gated server-side by `REASONING_ENABLED`; pulls ~60MB
# of deps. See https://github.com/docling-project/docling-agent/issues/26 for
# the public-API replacement of `_rag_loop`.
# an Ollama backend. Gated server-side by `RAG_ENABLED`; pulls ~60MB of deps.
docling-agent==0.1.0
mellea==0.4.2

View file

@ -337,7 +337,9 @@ class AnalysisService:
"""
total_pages = _count_pdf_pages(file_path)
batch_size = self._config.batch_page_size
if batch_size > 0 and total_pages > batch_size and self._converter.supports_page_batching:
is_remote = self._is_remote_converter()
if batch_size > 0 and total_pages > batch_size and not is_remote:
return await self._run_batched_conversion(
job_id, file_path, options, total_pages, batch_size
)
@ -346,6 +348,15 @@ class AnalysisService:
timeout=self._conversion_timeout,
)
def _is_remote_converter(self) -> bool:
"""Check if the converter is a remote (Serve) adapter."""
try:
from infra.serve_converter import ServeConverter
return isinstance(self._converter, ServeConverter)
except ImportError:
return False
async def _finalize_analysis(
self,
job_id: str,

View file

@ -192,9 +192,10 @@ class IngestionService:
)
async def ping(self) -> bool:
"""Check if the underlying vector store is reachable."""
"""Check if the OpenSearch cluster is reachable."""
try:
return await self._vector_store.ping()
info = await self._vector_store._client.info()
return bool(info)
except Exception:
logger.debug("Vector store ping failed", exc_info=True)
logger.debug("OpenSearch ping failed", exc_info=True)
return False

View file

@ -208,262 +208,3 @@ async def test_reader_returns_verbatim_json(neo4j_driver):
async def test_reader_missing_doc_returns_none(neo4j_driver):
await bootstrap_schema(neo4j_driver)
assert await read_document_json(neo4j_driver, "no-such-doc") is None
# Issue #197: Docling emits one InlineGroup per inline-styled paragraph plus N
# child `text` items (one per style run). Naive 1:1 mirroring blew up section
# graphs into per-style-run nodes. The writer now collapses an InlineGroup into
# a single :Paragraph node carrying the concatenated text of its children, and
# skips those children entirely.
INLINE_FIXTURE = {
"name": "inline.html",
"pages": {
"1": {"page_no": 1, "size": {"width": 595, "height": 842}},
},
"body": {
"self_ref": "#/body",
"children": [
{"$ref": "#/texts/0"},
{"$ref": "#/groups/0"},
],
},
"texts": [
{
"self_ref": "#/texts/0",
"parent": {"$ref": "#/body"},
"label": "section_header",
"text": "Heading",
"level": 1,
"prov": [{"page_no": 1, "bbox": {"l": 10, "t": 10, "r": 100, "b": 30}}],
},
{
"self_ref": "#/texts/1",
"parent": {"$ref": "#/groups/0"},
"label": "text",
"text": "Hello",
"prov": [{"page_no": 1, "bbox": {"l": 10, "t": 40, "r": 50, "b": 60}}],
},
{
"self_ref": "#/texts/2",
"parent": {"$ref": "#/groups/0"},
"label": "text",
"text": "world",
"prov": [{"page_no": 1, "bbox": {"l": 55, "t": 40, "r": 100, "b": 60}}],
},
{
"self_ref": "#/texts/3",
"parent": {"$ref": "#/groups/0"},
"label": "text",
"text": "!",
"prov": [{"page_no": 1, "bbox": {"l": 105, "t": 40, "r": 110, "b": 60}}],
},
],
"tables": [],
"pictures": [],
"groups": [
{
"self_ref": "#/groups/0",
"parent": {"$ref": "#/body"},
"label": "inline",
"children": [
{"$ref": "#/texts/1"},
{"$ref": "#/texts/2"},
{"$ref": "#/texts/3"},
],
},
],
}
async def test_inline_group_collapses_into_single_paragraph(neo4j_driver):
await bootstrap_schema(neo4j_driver)
doc_json = json.dumps(INLINE_FIXTURE)
result = await write_document(
neo4j_driver,
doc_id="doc-inline",
filename="inline.html",
document_json=doc_json,
)
# Section header + collapsed inline group only — NOT 5 (+3 style runs).
assert result.elements_written == 2
# The inline group inherits its 3 children's provs (1 each); the section
# header has its own prov → 4 Provenance nodes total.
assert result.provenances_written == 4
async with neo4j_driver.driver.session(database=neo4j_driver.database) as s:
r = await s.run(
"MATCH (e:Element:Paragraph {doc_id: $id, self_ref: '#/groups/0'}) "
"RETURN e.text AS text",
id="doc-inline",
)
rec = await r.single()
assert rec is not None, "InlineGroup should write a :Paragraph node"
assert rec["text"] == "Hello world !"
for child_ref in ("#/texts/1", "#/texts/2", "#/texts/3"):
assert (
await _count(
s,
"MATCH (e:Element {doc_id: $id, self_ref: $ref}) RETURN count(e) AS n",
id="doc-inline",
ref=child_ref,
)
== 0
), f"Style-run {child_ref} should be skipped, not written as a node"
# Inline group inherits provs from all children (order preserved).
assert (
await _count(
s,
"MATCH (e:Element:Paragraph {doc_id: $id, self_ref: '#/groups/0'})"
"-[:HAS_PROV]->(pv:Provenance) RETURN count(pv) AS n",
id="doc-inline",
)
== 3
)
# Reading order: section_header → inline-as-paragraph (1 NEXT edge).
assert (
await _count(
s,
"MATCH (a:Element {doc_id: $id})-[:NEXT]->(b:Element) RETURN count(*) AS n",
id="doc-inline",
)
== 1
)
assert (
await _count(
s,
"MATCH (:Document {id: $id})-[:HAS_ROOT]->(:Element) RETURN count(*) AS n",
id="doc-inline",
)
== 2
)
# ON_PAGE through Provenance → Page (matches the new schema).
assert (
await _count(
s,
"MATCH (:Element {doc_id: $id})-[:HAS_PROV]->"
"(:Provenance)-[:ON_PAGE]->(:Page) RETURN count(*) AS n",
id="doc-inline",
)
== 4
)
# Same issue (#197): a Picture's `children` are internal text labels (flowchart
# boxes, chart axis labels, diagram callouts) extracted by Docling's layout
# model. Mirroring them 1:1 drowns the figure in dozens of tiny nodes.
PICTURE_FIXTURE = {
"name": "figure.pdf",
"pages": {
"1": {"page_no": 1, "size": {"width": 595, "height": 842}},
},
"body": {
"self_ref": "#/body",
"children": [
{"$ref": "#/texts/0"},
{"$ref": "#/pictures/0"},
],
},
"texts": [
{
"self_ref": "#/texts/0",
"parent": {"$ref": "#/body"},
"label": "caption",
"text": "Figure 1: Pipeline overview.",
"prov": [{"page_no": 1, "bbox": {"l": 0, "t": 700, "r": 500, "b": 720}}],
},
# Internal labels — children of #/pictures/0.
{
"self_ref": "#/texts/1",
"parent": {"$ref": "#/pictures/0"},
"label": "text",
"text": "Parse",
"prov": [{"page_no": 1, "bbox": {"l": 100, "t": 200, "r": 130, "b": 220}}],
},
{
"self_ref": "#/texts/2",
"parent": {"$ref": "#/pictures/0"},
"label": "text",
"text": "Build",
"prov": [{"page_no": 1, "bbox": {"l": 140, "t": 200, "r": 170, "b": 220}}],
},
{
"self_ref": "#/texts/3",
"parent": {"$ref": "#/pictures/0"},
"label": "text",
"text": "Enrich",
"prov": [{"page_no": 1, "bbox": {"l": 180, "t": 200, "r": 220, "b": 220}}],
},
],
"tables": [],
"pictures": [
{
"self_ref": "#/pictures/0",
"parent": {"$ref": "#/body"},
"label": "picture",
"children": [
{"$ref": "#/texts/1"},
{"$ref": "#/texts/2"},
{"$ref": "#/texts/3"},
],
"captions": [{"$ref": "#/texts/0"}],
"prov": [{"page_no": 1, "bbox": {"l": 90, "t": 100, "r": 510, "b": 600}}],
}
],
"groups": [],
}
async def test_picture_internal_labels_are_skipped(neo4j_driver):
await bootstrap_schema(neo4j_driver)
doc_json = json.dumps(PICTURE_FIXTURE)
result = await write_document(
neo4j_driver,
doc_id="doc-pic",
filename="figure.pdf",
document_json=doc_json,
)
# Caption + picture only — the 3 internal labels are skipped.
assert result.elements_written == 2
async with neo4j_driver.driver.session(database=neo4j_driver.database) as s:
for child_ref in ("#/texts/1", "#/texts/2", "#/texts/3"):
assert (
await _count(
s,
"MATCH (e:Element {doc_id: $id, self_ref: $ref}) RETURN count(e) AS n",
id="doc-pic",
ref=child_ref,
)
== 0
), f"Picture child {child_ref} should be skipped"
# Picture stays a :Figure node with its own prov.
assert (
await _count(
s,
"MATCH (e:Element:Figure {doc_id: $id, self_ref: '#/pictures/0'}) "
"RETURN count(e) AS n",
id="doc-pic",
)
== 1
)
# No PARENT_OF from the picture to its dropped children.
assert (
await _count(
s,
"MATCH (:Element {doc_id: $id, self_ref: '#/pictures/0'})-[:PARENT_OF]->"
"(:Element) RETURN count(*) AS n",
id="doc-pic",
)
== 0
)

View file

@ -196,9 +196,7 @@ class TestAnalysisEndpoints:
assert resp.status_code == 200
data = resp.json()
assert data["status"] == "RUNNING"
# ISO-8601 datetime string from `mark_running()`
assert isinstance(data["startedAt"], str)
assert data["startedAt"] # not empty
assert data["startedAt"] is not None
def test_get_analysis_not_found(self, client, mock_analysis_service):
mock_analysis_service.find_by_id = AsyncMock(return_value=None)

View file

@ -1,265 +0,0 @@
"""Tests for `infra.docling_agent_reasoning.DoclingAgentReasoningRunner`.
docling-agent + mellea + docling-core are NOT installed in the CI test env
(heavy deps). We stub the modules via `sys.modules` injection so the tests
cover the real adapter code path without bringing in Ollama or any LLM
client.
Concurrency check (R3): the previous implementation mutated
`os.environ["OLLAMA_HOST"]` on every request, which raced when two requests
arrived with different hosts. The new design commits the host once at
construction time these tests assert that property holds.
"""
from __future__ import annotations
import asyncio
import os
import sys
import types
from unittest.mock import MagicMock
import pytest
from domain.ports import ReasoningParseError, ReasoningRunner
from infra.llm.ollama_provider import OllamaProvider
@pytest.fixture
def stub_docling_agent(monkeypatch: pytest.MonkeyPatch):
"""Inject fake `docling_agent.agents` + `docling_core.types.doc.document`
+ `mellea.backends.model_ids` modules so the adapter's lazy imports
resolve to our stubs.
Returns the stub `DoclingRAGAgent` class + agent instance + result so
tests can configure return values / side effects.
"""
fake_result = MagicMock()
fake_result.answer = "stub answer"
fake_result.converged = True
fake_result.iterations = [
MagicMock(
model_dump=lambda: {
"iteration": 1,
"section_ref": "#/texts/0",
"reason": "looks relevant",
"section_text_length": 42,
"can_answer": True,
"response": "stub answer",
}
)
]
agent_instance = MagicMock()
agent_instance._rag_loop.return_value = fake_result
agent_class = MagicMock(return_value=agent_instance)
fake_agents_mod = types.ModuleType("docling_agent.agents")
fake_agents_mod.DoclingRAGAgent = agent_class
fake_root_mod = types.ModuleType("docling_agent")
fake_root_mod.agents = fake_agents_mod
fake_doc_class = MagicMock()
fake_doc_class.model_validate_json = MagicMock(return_value="fake-doc-instance")
fake_doc_mod = types.ModuleType("docling_core.types.doc.document")
fake_doc_mod.DoclingDocument = fake_doc_class
def fake_model_identifier(**kwargs):
m = MagicMock()
m.ollama_name = kwargs.get("ollama_name")
m.openai_name = kwargs.get("openai_name")
return m
fake_model_ids_mod = types.ModuleType("mellea.backends.model_ids")
fake_model_ids_mod.ModelIdentifier = fake_model_identifier
fake_backends_mod = types.ModuleType("mellea.backends")
fake_backends_mod.model_ids = fake_model_ids_mod
fake_mellea_mod = types.ModuleType("mellea")
fake_mellea_mod.backends = fake_backends_mod
monkeypatch.setitem(sys.modules, "docling_agent", fake_root_mod)
monkeypatch.setitem(sys.modules, "docling_agent.agents", fake_agents_mod)
monkeypatch.setitem(sys.modules, "docling_core.types.doc.document", fake_doc_mod)
monkeypatch.setitem(sys.modules, "mellea", fake_mellea_mod)
monkeypatch.setitem(sys.modules, "mellea.backends", fake_backends_mod)
monkeypatch.setitem(sys.modules, "mellea.backends.model_ids", fake_model_ids_mod)
return agent_class, agent_instance, fake_result
def _make_runner_with_stubbed_deps(
*,
host: str = "http://ollama:11434",
default_model_id: str = "gpt-oss:20b",
):
"""Build the runner bypassing the deps-check (we stubbed deps via
sys.modules but `import` inside `deps_present()` runs at import time
see how the adapter handles it)."""
from infra.docling_agent_reasoning import DoclingAgentReasoningRunner
# Force the deps_present check to True regardless of the test env.
runner = DoclingAgentReasoningRunner.__new__(DoclingAgentReasoningRunner)
runner._provider = OllamaProvider(host=host, default_model_id=default_model_id)
runner._deps_ok = True
os.environ["OLLAMA_HOST"] = host
return runner
class TestProtocolConformance:
def test_runner_satisfies_reasoning_runner_protocol(self) -> None:
"""R13 — adapter is structurally a `ReasoningRunner`."""
runner = _make_runner_with_stubbed_deps()
assert isinstance(runner, ReasoningRunner)
class TestProviderRejection:
def test_rejects_non_ollama_provider(self) -> None:
from infra.docling_agent_reasoning import DoclingAgentReasoningRunner
class _FakeProvider:
type = "openai"
host = "x"
default_model_id = "y"
def health_check(self) -> bool:
return True
with pytest.raises(NotImplementedError, match="Ollama"):
DoclingAgentReasoningRunner(provider=_FakeProvider()) # type: ignore[arg-type]
class TestRunHappyPath:
@pytest.mark.asyncio
async def test_returns_domain_reasoning_result(self, stub_docling_agent) -> None:
runner = _make_runner_with_stubbed_deps()
result = await runner.run(
document_json='{"stub": true}',
query="What is this?",
)
assert result.answer == "stub answer"
assert result.converged is True
assert len(result.iterations) == 1
it = result.iterations[0]
assert it.iteration == 1
assert it.section_ref == "#/texts/0"
assert it.can_answer is True
@pytest.mark.asyncio
async def test_uses_default_model_id_when_not_overridden(self, stub_docling_agent) -> None:
agent_class, _, _ = stub_docling_agent
runner = _make_runner_with_stubbed_deps(default_model_id="custom-model:7b")
await runner.run(document_json="{}", query="Q")
agent_class.assert_called_once()
passed = agent_class.call_args.kwargs["model_id"]
assert passed.ollama_name == "custom-model:7b"
@pytest.mark.asyncio
async def test_per_call_model_id_override_wins(self, stub_docling_agent) -> None:
agent_class, _, _ = stub_docling_agent
runner = _make_runner_with_stubbed_deps(default_model_id="default:7b")
await runner.run(document_json="{}", query="Q", model_id="override:13b")
passed = agent_class.call_args.kwargs["model_id"]
assert passed.ollama_name == "override:13b"
@pytest.mark.asyncio
async def test_calls_rag_loop_with_query_and_doc(self, stub_docling_agent) -> None:
_, agent_instance, _ = stub_docling_agent
runner = _make_runner_with_stubbed_deps()
await runner.run(document_json="{}", query="Hello?")
agent_instance._rag_loop.assert_called_once()
kwargs = agent_instance._rag_loop.call_args.kwargs
assert kwargs["query"] == "Hello?"
assert kwargs["doc"] == "fake-doc-instance"
class TestRunUpstreamFailure:
@pytest.mark.asyncio
async def test_indexerror_translates_to_parse_error(self, stub_docling_agent) -> None:
"""docling-agent's known IndexError → domain `ReasoningParseError`."""
_, agent_instance, _ = stub_docling_agent
agent_instance._rag_loop.side_effect = IndexError("list index out of range")
runner = _make_runner_with_stubbed_deps(default_model_id="granite4:micro-h")
with pytest.raises(ReasoningParseError) as exc_info:
await runner.run(document_json="{}", query="Q")
assert exc_info.value.model_id == "granite4:micro-h"
@pytest.mark.asyncio
async def test_other_exceptions_propagate(self, stub_docling_agent) -> None:
_, agent_instance, _ = stub_docling_agent
agent_instance._rag_loop.side_effect = RuntimeError("Ollama unreachable")
runner = _make_runner_with_stubbed_deps()
with pytest.raises(RuntimeError, match="Ollama unreachable"):
await runner.run(document_json="{}", query="Q")
class TestHostIsolation:
"""R3 — host is committed at construction; concurrent calls don't race
on `os.environ["OLLAMA_HOST"]` because it isn't mutated per-request.
"""
def test_host_committed_at_construction(self) -> None:
os.environ.pop("OLLAMA_HOST", None)
runner = _make_runner_with_stubbed_deps(host="http://specific-host:11434")
assert os.environ["OLLAMA_HOST"] == "http://specific-host:11434"
# Explicit ref to silence unused-var lint
assert runner._provider.host == "http://specific-host:11434"
@pytest.mark.asyncio
async def test_concurrent_runs_do_not_mutate_env(self, stub_docling_agent) -> None:
"""Two concurrent `run()` calls on the SAME runner — env stays put.
With the previous design, each request mutated OLLAMA_HOST per-call.
Now nothing in `run()` should touch os.environ assert that.
"""
runner = _make_runner_with_stubbed_deps(host="http://stable:11434")
os.environ["OLLAMA_HOST"] = "http://stable:11434"
observed: list[str] = []
# Use a fresh side_effect that records env at call time.
_, agent_instance, _ = stub_docling_agent
original = agent_instance._rag_loop
def record_env(*args, **kwargs):
observed.append(os.environ.get("OLLAMA_HOST", ""))
return original.return_value
agent_instance._rag_loop = record_env
await asyncio.gather(
runner.run(document_json="{}", query="Q1"),
runner.run(document_json="{}", query="Q2"),
runner.run(document_json="{}", query="Q3"),
)
assert observed == [
"http://stable:11434",
"http://stable:11434",
"http://stable:11434",
]
class TestDepsPresent:
def test_deps_present_returns_false_when_modules_missing(
self, monkeypatch: pytest.MonkeyPatch
) -> None:
"""When docling-agent/mellea aren't importable, the wiring code
should know not to instantiate the runner."""
monkeypatch.setitem(sys.modules, "docling_agent", None)
monkeypatch.setitem(sys.modules, "docling_agent.agents", None)
monkeypatch.setitem(sys.modules, "mellea", None)
from infra.docling_agent_reasoning import deps_present
assert deps_present() is False

View file

@ -215,166 +215,6 @@ def test_title_is_surfaced_on_document_node():
assert doc_node["title"] == "My Doc.pdf"
def test_inline_group_collapses_into_single_paragraph():
"""Issue #197: an InlineGroup + N child `text` items must yield ONE
Paragraph node carrying the joined text and the union of children's provs."""
fixture = {
"pages": {"1": {"page_no": 1, "size": {"width": 595, "height": 842}}},
"body": {
"self_ref": "#/body",
"children": [{"$ref": "#/texts/0"}, {"$ref": "#/groups/0"}],
},
"texts": [
{
"self_ref": "#/texts/0",
"parent": {"$ref": "#/body"},
"label": "section_header",
"text": "Heading",
"level": 1,
"prov": [{"page_no": 1, "bbox": {"l": 10, "t": 10, "r": 100, "b": 30}}],
},
{
"self_ref": "#/texts/1",
"parent": {"$ref": "#/groups/0"},
"label": "text",
"text": "Hello",
"prov": [{"page_no": 1, "bbox": {"l": 10, "t": 40, "r": 50, "b": 60}}],
},
{
"self_ref": "#/texts/2",
"parent": {"$ref": "#/groups/0"},
"label": "text",
"text": "world",
"prov": [{"page_no": 1, "bbox": {"l": 55, "t": 40, "r": 100, "b": 60}}],
},
{
"self_ref": "#/texts/3",
"parent": {"$ref": "#/groups/0"},
"label": "text",
"text": "!",
"prov": [{"page_no": 1, "bbox": {"l": 105, "t": 40, "r": 110, "b": 60}}],
},
],
"tables": [],
"pictures": [],
"groups": [
{
"self_ref": "#/groups/0",
"parent": {"$ref": "#/body"},
"label": "inline",
"children": [
{"$ref": "#/texts/1"},
{"$ref": "#/texts/2"},
{"$ref": "#/texts/3"},
],
},
],
}
payload = build_graph_payload(json.dumps(fixture), doc_id="doc-inline")
# Section header + collapsed inline group only — NOT 5 (+3 style runs).
elements = [n for n in payload.nodes if n.get("group") == "element"]
assert {e["self_ref"] for e in elements} == {"#/texts/0", "#/groups/0"}
inline = next(n for n in elements if n["self_ref"] == "#/groups/0")
assert inline["label"] == "Paragraph"
assert inline["text"] == "Hello world !"
# Provs union: 3 children x 1 prov each = 3 provs on the collapsed node.
assert len(inline["provs"]) == 3
assert all(p.get("page_no") == 1 for p in inline["provs"])
# NEXT follows the post-collapse reading order — section_header → inline only.
nexts = [(e["source"], e["target"]) for e in payload.edges if e["type"] == "NEXT"]
assert nexts == [("elem::#/texts/0", "elem::#/groups/0")]
# ON_PAGE edges still wire the surviving elements to the page (one per
# element, deduped by `seen_pages`).
on_page = [e for e in payload.edges if e["type"] == "ON_PAGE"]
assert sorted(e["source"] for e in on_page) == ["elem::#/groups/0", "elem::#/texts/0"]
def test_picture_internal_labels_are_skipped():
"""Issue #197: a Picture's `children` (flowchart / diagram text labels
extracted by Docling's layout model) must not become standalone graph
nodes they drown the figure in dozens of tiny labels. The picture
itself stays, and a caption (separate `parent` chain on body) is
unaffected."""
fixture = {
"pages": {"1": {"page_no": 1, "size": {"width": 595, "height": 842}}},
"body": {
"self_ref": "#/body",
"children": [{"$ref": "#/texts/0"}, {"$ref": "#/pictures/0"}],
},
"texts": [
# Caption — separate item under body, referenced from the picture's
# `captions` field (not its `children`). Survives the collapse.
{
"self_ref": "#/texts/0",
"parent": {"$ref": "#/body"},
"label": "caption",
"text": "Figure 1: Sketch of Docling's pipelines.",
"prov": [{"page_no": 1, "bbox": {"l": 0, "t": 700, "r": 500, "b": 720}}],
},
# Internal labels of the figure (flowchart boxes). Live in
# `texts[]` with `parent` pointing at the picture.
*[
{
"self_ref": f"#/texts/{i}",
"parent": {"$ref": "#/pictures/0"},
"label": "text",
"text": label,
"prov": [
{
"page_no": 1,
"bbox": {"l": 100 + i * 30, "t": 200, "r": 130 + i * 30, "b": 220},
}
],
}
for i, label in enumerate(
["Parse", "Build", "Enrich", "Assemble", "Document"], start=1
)
],
],
"tables": [],
"pictures": [
{
"self_ref": "#/pictures/0",
"parent": {"$ref": "#/body"},
"label": "picture",
"children": [
{"$ref": "#/texts/1"},
{"$ref": "#/texts/2"},
{"$ref": "#/texts/3"},
{"$ref": "#/texts/4"},
{"$ref": "#/texts/5"},
],
"captions": [{"$ref": "#/texts/0"}],
"prov": [{"page_no": 1, "bbox": {"l": 90, "t": 100, "r": 510, "b": 600}}],
}
],
"groups": [],
}
payload = build_graph_payload(json.dumps(fixture), doc_id="doc-pic")
elements = [n for n in payload.nodes if n.get("group") == "element"]
refs = {e["self_ref"] for e in elements}
# Picture + caption only — the 5 internal labels must be dropped.
assert refs == {"#/pictures/0", "#/texts/0"}
# Picture keeps its own label / text / prov (no inline-style override).
pic = next(e for e in elements if e["self_ref"] == "#/pictures/0")
assert pic["label"] == "Figure"
assert pic["docling_label"] == "picture"
# No PARENT_OF edges from the picture to its (skipped) children.
parent_edges = [e for e in payload.edges if e["type"] == "PARENT_OF"]
assert all(e["source"] != "elem::#/pictures/0" for e in parent_edges)
# NEXT chain only includes surviving elements.
next_targets = [e["target"] for e in payload.edges if e["type"] == "NEXT"]
assert all(t in {"elem::#/texts/0", "elem::#/pictures/0"} for t in next_targets)
def test_element_text_is_capped_at_200_chars():
long = "x" * 500
fixture = {

View file

@ -164,14 +164,16 @@ class TestPing:
async def test_ping_success(
self, service: IngestionService, mock_vector_store: AsyncMock
) -> None:
mock_vector_store.ping.return_value = True
mock_vector_store._client = AsyncMock()
mock_vector_store._client.info.return_value = {"cluster_name": "test"}
result = await service.ping()
assert result is True
async def test_ping_failure(
self, service: IngestionService, mock_vector_store: AsyncMock
) -> None:
mock_vector_store.ping.side_effect = ConnectionError("down")
mock_vector_store._client = AsyncMock()
mock_vector_store._client.info.side_effect = ConnectionError("down")
result = await service.ping()
assert result is False

View file

@ -58,7 +58,7 @@ class TestAnalysisJob:
job.mark_running()
assert job.status == AnalysisStatus.RUNNING
assert isinstance(job.started_at, datetime)
assert job.started_at is not None
def test_mark_completed(self):
job = AnalysisJob()
@ -74,8 +74,7 @@ class TestAnalysisJob:
assert job.content_markdown == "# Title"
assert job.content_html == "<h1>Title</h1>"
assert job.pages_json == '[{"page": 1}]'
assert isinstance(job.completed_at, datetime)
assert job.completed_at >= job.started_at
assert job.completed_at is not None
def test_mark_failed(self):
job = AnalysisJob()
@ -85,8 +84,7 @@ class TestAnalysisJob:
assert job.status == AnalysisStatus.FAILED
assert job.error_message == "Something went wrong"
assert isinstance(job.completed_at, datetime)
assert job.completed_at >= job.started_at
assert job.completed_at is not None
def test_status_transitions(self):
"""Test full lifecycle: PENDING -> RUNNING -> COMPLETED."""

View file

@ -1,82 +0,0 @@
"""Tests for `infra.llm.ollama_provider.OllamaProvider`.
Network is stubbed via `httpx` MockTransport so the tests don't depend on a
running Ollama instance.
"""
from __future__ import annotations
from typing import TYPE_CHECKING
import httpx
from domain.ports import LLMProvider
from domain.value_objects import LLMProviderType
from infra.llm.ollama_provider import OllamaProvider
if TYPE_CHECKING:
import pytest
def test_provider_satisfies_llmprovider_protocol() -> None:
"""R13 — `OllamaProvider` is structurally a `LLMProvider`."""
p = OllamaProvider(host="http://localhost:11434", default_model_id="m")
assert isinstance(p, LLMProvider)
def test_provider_exposes_type_host_default_model_id() -> None:
p = OllamaProvider(host="http://ollama:11434", default_model_id="gpt-oss:20b")
assert p.type is LLMProviderType.OLLAMA
assert p.host == "http://ollama:11434"
assert p.default_model_id == "gpt-oss:20b"
def test_host_trailing_slash_is_stripped() -> None:
p = OllamaProvider(host="http://localhost:11434/", default_model_id="m")
assert p.host == "http://localhost:11434"
def test_health_check_returns_true_on_200(monkeypatch: pytest.MonkeyPatch) -> None:
def handler(request: httpx.Request) -> httpx.Response:
assert request.url.path == "/api/tags"
return httpx.Response(200, json={"models": []})
transport = httpx.MockTransport(handler)
real_get = httpx.get
def fake_get(url: str, **kwargs): # type: ignore[no-untyped-def]
with httpx.Client(transport=transport) as client:
return client.get(url, **kwargs)
monkeypatch.setattr(httpx, "get", fake_get)
p = OllamaProvider(host="http://localhost:11434", default_model_id="m")
assert p.health_check() is True
# Restore (defensive, MonkeyPatch handles it but explicit is fine)
monkeypatch.setattr(httpx, "get", real_get)
def test_health_check_returns_false_on_connection_error(
monkeypatch: pytest.MonkeyPatch,
) -> None:
def fake_get(url: str, **kwargs): # type: ignore[no-untyped-def]
raise httpx.ConnectError("connection refused")
monkeypatch.setattr(httpx, "get", fake_get)
p = OllamaProvider(host="http://nowhere:11434", default_model_id="m")
assert p.health_check() is False
def test_health_check_returns_false_on_non_200(monkeypatch: pytest.MonkeyPatch) -> None:
def handler(request: httpx.Request) -> httpx.Response:
return httpx.Response(500)
transport = httpx.MockTransport(handler)
def fake_get(url: str, **kwargs): # type: ignore[no-untyped-def]
with httpx.Client(transport=transport) as client:
return client.get(url, **kwargs)
monkeypatch.setattr(httpx, "get", fake_get)
p = OllamaProvider(host="http://localhost:11434", default_model_id="m")
assert p.health_check() is False

View file

@ -53,8 +53,7 @@ class TestBuildConverter:
assert opts.generate_page_images is False
assert opts.generate_picture_images is False
assert opts.images_scale == 1.0
# default document timeout is 120s (cf. infra/settings.py)
assert opts.document_timeout == 120.0
assert opts.document_timeout is not None
def test_ocr_disabled(self):
conv = build_converter(ConversionOptions(do_ocr=False))

View file

@ -1,25 +1,35 @@
"""Tests for `api.reasoning` — the HTTP layer over a `ReasoningRunner` port.
"""Tests for `api.reasoning` — the live `docling-agent` RAG runner endpoint.
The API layer is decoupled from docling-agent / mellea / docling-core. Tests
inject a fake `ReasoningRunner` on `app.state.reasoning_runner` and assert on
HTTP status / payload + on what the runner was called with.
Adapter behaviour (the actual docling-agent integration) is tested separately
in `tests/test_docling_agent_reasoning.py`.
docling-agent + mellea are NOT installed in the CI test env (heavy deps).
The endpoint does a lazy import inside the handler; we stub the modules via
`sys.modules` injection so the tests cover the real code path without
bringing in Ollama, mellea, or LLM clients.
"""
from __future__ import annotations
from unittest.mock import AsyncMock
import sys
import types
from dataclasses import replace
from unittest.mock import AsyncMock, MagicMock
import pytest
from fastapi import FastAPI
from fastapi.testclient import TestClient
from api import reasoning as reasoning_module
from api.reasoning import router
from domain.models import AnalysisJob
from domain.ports import ReasoningParseError
from domain.value_objects import ReasoningIteration, ReasoningResult
def _patched_settings(monkeypatch, **overrides):
"""Replace `api.reasoning.settings` with a frozen dataclass copy carrying
the given overrides. `Settings` is frozen, so attribute-level monkeypatch
doesn't work — we swap the whole instance on the module.
"""
new_settings = replace(reasoning_module.settings, **overrides)
monkeypatch.setattr(reasoning_module, "settings", new_settings)
return new_settings
def _job_with_doc_json() -> AnalysisJob:
@ -30,66 +40,14 @@ def _job_with_doc_json() -> AnalysisJob:
markdown="# Hello",
html="<h1>Hello</h1>",
pages_json="[]",
# Minimal placeholder — the test stubs `DoclingDocument.model_validate_json`
# so the content doesn't need to be a real DoclingDocument.
document_json='{"stub": true}',
chunks_json="[]",
)
return job
def _sample_result() -> ReasoningResult:
return ReasoningResult(
answer="stub answer",
converged=True,
iterations=[
ReasoningIteration(
iteration=1,
section_ref="#/texts/0",
reason="looks relevant",
section_text_length=42,
can_answer=True,
response="stub answer",
)
],
)
class _FakeRunner:
"""In-memory ReasoningRunner. Records the last call so tests can assert
on the args without touching docling-agent."""
def __init__(
self,
*,
result: ReasoningResult | None = None,
is_available: bool = True,
raises: Exception | None = None,
) -> None:
self._result = result or _sample_result()
self._is_available = is_available
self._raises = raises
self.last_call: dict | None = None
@property
def is_available(self) -> bool:
return self._is_available
async def run(
self,
*,
document_json: str,
query: str,
model_id: str | None = None,
) -> ReasoningResult:
self.last_call = {
"document_json": document_json,
"query": query,
"model_id": model_id,
}
if self._raises is not None:
raise self._raises
return self._result
@pytest.fixture
def mock_analysis_repo() -> AsyncMock:
repo = AsyncMock()
@ -97,47 +55,109 @@ def mock_analysis_repo() -> AsyncMock:
return repo
def _make_client(*, runner: _FakeRunner | None, repo: AsyncMock) -> TestClient:
@pytest.fixture
def stub_docling_agent(monkeypatch):
"""Inject fake `docling_agent.agents` + `docling_core.types.doc.document`
modules so the endpoint's lazy imports resolve to our stubs.
Returns the `DoclingRAGAgent` stub class so tests can assert on its calls
/ configure its `_rag_loop` return value.
"""
fake_result = MagicMock()
fake_result.answer = "stub answer"
fake_result.converged = True
fake_result.iterations = [
MagicMock(
model_dump=lambda: {
"iteration": 1,
"section_ref": "#/texts/0",
"reason": "looks relevant",
"section_text_length": 42,
"can_answer": True,
"response": "stub answer",
}
)
]
agent_instance = MagicMock()
agent_instance._rag_loop.return_value = fake_result
agent_class = MagicMock(return_value=agent_instance)
fake_agents_mod = types.ModuleType("docling_agent.agents")
fake_agents_mod.DoclingRAGAgent = agent_class
fake_root_mod = types.ModuleType("docling_agent")
fake_root_mod.agents = fake_agents_mod
fake_doc_class = MagicMock()
fake_doc_class.model_validate_json = MagicMock(return_value="fake-doc-instance")
fake_doc_mod = types.ModuleType("docling_core.types.doc.document")
fake_doc_mod.DoclingDocument = fake_doc_class
# Stub `mellea.backends.model_ids.ModelIdentifier` — the endpoint wraps
# the string model_id in this dataclass before handing to DoclingRAGAgent.
# Identity-like: stores the kwargs so tests can assert on `ollama_name`.
def fake_model_identifier(**kwargs):
m = MagicMock()
m.ollama_name = kwargs.get("ollama_name")
m.openai_name = kwargs.get("openai_name")
return m
fake_model_ids_mod = types.ModuleType("mellea.backends.model_ids")
fake_model_ids_mod.ModelIdentifier = fake_model_identifier
fake_backends_mod = types.ModuleType("mellea.backends")
fake_backends_mod.model_ids = fake_model_ids_mod
fake_mellea_mod = types.ModuleType("mellea")
fake_mellea_mod.backends = fake_backends_mod
monkeypatch.setitem(sys.modules, "docling_agent", fake_root_mod)
monkeypatch.setitem(sys.modules, "docling_agent.agents", fake_agents_mod)
monkeypatch.setitem(sys.modules, "docling_core.types.doc.document", fake_doc_mod)
monkeypatch.setitem(sys.modules, "mellea", fake_mellea_mod)
monkeypatch.setitem(sys.modules, "mellea.backends", fake_backends_mod)
monkeypatch.setitem(sys.modules, "mellea.backends.model_ids", fake_model_ids_mod)
return agent_class, agent_instance, fake_result
@pytest.fixture
def client(mock_analysis_repo: AsyncMock) -> TestClient:
app = FastAPI()
app.include_router(router)
app.state.analysis_repo = repo
app.state.reasoning_runner = runner
app.state.analysis_repo = mock_analysis_repo
return TestClient(app)
class TestReasoningDisabled:
def test_503_when_runner_not_wired(self, mock_analysis_repo: AsyncMock) -> None:
client = _make_client(runner=None, repo=mock_analysis_repo)
resp = client.post("/api/documents/doc-1/reasoning", json={"query": "Q"})
assert resp.status_code == 503
assert "REASONING_ENABLED" in resp.json()["detail"]
def test_503_when_runner_unavailable(self, mock_analysis_repo: AsyncMock) -> None:
runner = _FakeRunner(is_available=False)
client = _make_client(runner=runner, repo=mock_analysis_repo)
resp = client.post("/api/documents/doc-1/reasoning", json={"query": "Q"})
class TestRagDisabled:
def test_503_when_flag_off(self, client: TestClient, monkeypatch) -> None:
_patched_settings(monkeypatch, rag_enabled=False)
resp = client.post("/api/documents/doc-1/rag", json={"query": "Q"})
assert resp.status_code == 503
assert "RAG_ENABLED" in resp.json()["detail"]
class TestReasoningValidation:
def test_400_when_query_empty(self, mock_analysis_repo: AsyncMock) -> None:
client = _make_client(runner=_FakeRunner(), repo=mock_analysis_repo)
resp = client.post("/api/documents/doc-1/reasoning", json={"query": " "})
class TestRagValidation:
def test_400_when_query_empty(self, client: TestClient, monkeypatch) -> None:
_patched_settings(monkeypatch, rag_enabled=True)
resp = client.post("/api/documents/doc-1/rag", json={"query": " "})
assert resp.status_code == 400
def test_404_when_no_completed_analysis(self, mock_analysis_repo: AsyncMock) -> None:
def test_404_when_no_completed_analysis(
self, client: TestClient, mock_analysis_repo: AsyncMock, monkeypatch
) -> None:
_patched_settings(monkeypatch, rag_enabled=True)
mock_analysis_repo.find_latest_completed_by_document.return_value = None
client = _make_client(runner=_FakeRunner(), repo=mock_analysis_repo)
resp = client.post("/api/documents/doc-1/reasoning", json={"query": "Q"})
resp = client.post("/api/documents/doc-1/rag", json={"query": "Q"})
assert resp.status_code == 404
class TestReasoningSuccess:
def test_returns_reasoning_result_shape(self, mock_analysis_repo: AsyncMock) -> None:
runner = _FakeRunner()
client = _make_client(runner=runner, repo=mock_analysis_repo)
class TestRagSuccess:
def test_returns_rag_result_shape(
self, client: TestClient, stub_docling_agent, monkeypatch
) -> None:
_patched_settings(monkeypatch, rag_enabled=True)
_agent_class, _agent_instance, _fake_result = stub_docling_agent
resp = client.post("/api/documents/doc-1/reasoning", json={"query": "What is this?"})
resp = client.post("/api/documents/doc-1/rag", json={"query": "What is this?"})
assert resp.status_code == 200
data = resp.json()
assert data["answer"] == "stub answer"
@ -147,102 +167,95 @@ class TestReasoningSuccess:
assert it["iteration"] == 1
assert it["section_ref"] == "#/texts/0"
assert it["can_answer"] is True
assert it["section_text_length"] == 42
def test_passes_query_and_doc_json_to_runner(self, mock_analysis_repo: AsyncMock) -> None:
runner = _FakeRunner()
client = _make_client(runner=runner, repo=mock_analysis_repo)
client.post("/api/documents/doc-1/reasoning", json={"query": "Hello?"})
assert runner.last_call is not None
assert runner.last_call["query"] == "Hello?"
assert runner.last_call["document_json"] == '{"stub": true}'
def test_no_model_override_passes_none(self, mock_analysis_repo: AsyncMock) -> None:
runner = _FakeRunner()
client = _make_client(runner=runner, repo=mock_analysis_repo)
client.post("/api/documents/doc-1/reasoning", json={"query": "Q"})
assert runner.last_call is not None
assert runner.last_call["model_id"] is None
def test_per_request_model_id_override_wins(self, mock_analysis_repo: AsyncMock) -> None:
runner = _FakeRunner()
client = _make_client(runner=runner, repo=mock_analysis_repo)
client.post(
"/api/documents/doc-1/reasoning",
json={"query": "Q", "model_id": "override:13b"},
)
assert runner.last_call is not None
assert runner.last_call["model_id"] == "override:13b"
def test_iterations_with_multiple_steps_serialize_correctly(
self, mock_analysis_repo: AsyncMock
def test_calls_rag_loop_with_query_and_doc(
self, client: TestClient, stub_docling_agent, monkeypatch
) -> None:
"""R6 — trace serializable on >=2 iterations, all fields preserved."""
result = ReasoningResult(
answer="final",
converged=False,
iterations=[
ReasoningIteration(
iteration=1,
section_ref="#/texts/0",
reason="r1",
section_text_length=10,
can_answer=False,
response="not yet",
),
ReasoningIteration(
iteration=2,
section_ref="#/texts/5",
reason="r2",
section_text_length=20,
can_answer=True,
response="final",
),
],
)
runner = _FakeRunner(result=result)
client = _make_client(runner=runner, repo=mock_analysis_repo)
_patched_settings(monkeypatch, rag_enabled=True)
_agent_class, agent_instance, _ = stub_docling_agent
resp = client.post("/api/documents/doc-1/reasoning", json={"query": "Q"})
assert resp.status_code == 200
data = resp.json()
assert data["converged"] is False
assert [it["iteration"] for it in data["iterations"]] == [1, 2]
assert [it["section_ref"] for it in data["iterations"]] == [
"#/texts/0",
"#/texts/5",
]
assert data["iterations"][0]["can_answer"] is False
assert data["iterations"][1]["can_answer"] is True
client.post("/api/documents/doc-1/rag", json={"query": "Hello?"})
agent_instance._rag_loop.assert_called_once()
kwargs = agent_instance._rag_loop.call_args.kwargs
assert kwargs["query"] == "Hello?"
# The stub returns the string "fake-doc-instance" from model_validate_json
# and we pass it straight through to `doc=`.
assert kwargs["doc"] == "fake-doc-instance"
def test_uses_default_model_id_when_not_overridden(
self, client: TestClient, stub_docling_agent, monkeypatch
) -> None:
_patched_settings(monkeypatch, rag_enabled=True, rag_model_id="custom-model:7b")
agent_class, _, _ = stub_docling_agent
client.post("/api/documents/doc-1/rag", json={"query": "Q"})
agent_class.assert_called_once()
# model_id is wrapped in a ModelIdentifier(ollama_name=...) dataclass
# before reaching the agent — the stub exposes the field for assertion.
passed = agent_class.call_args.kwargs["model_id"]
assert passed.ollama_name == "custom-model:7b"
def test_per_request_model_id_override_wins(
self, client: TestClient, stub_docling_agent, monkeypatch
) -> None:
_patched_settings(monkeypatch, rag_enabled=True, rag_model_id="default:7b")
agent_class, _, _ = stub_docling_agent
client.post("/api/documents/doc-1/rag", json={"query": "Q", "model_id": "override:13b"})
passed = agent_class.call_args.kwargs["model_id"]
assert passed.ollama_name == "override:13b"
def test_sets_ollama_host_env_from_settings(
self, client: TestClient, stub_docling_agent, monkeypatch
) -> None:
import os
_patched_settings(monkeypatch, rag_enabled=True, ollama_host="http://ollama:11434")
client.post("/api/documents/doc-1/rag", json={"query": "Q"})
assert os.environ["OLLAMA_HOST"] == "http://ollama:11434"
class TestReasoningUpstreamFailure:
def test_502_when_runner_raises_parse_error(self, mock_analysis_repo: AsyncMock) -> None:
"""The model couldn't produce a parseable JSON answer after retries."""
runner = _FakeRunner(
raises=ReasoningParseError(model_id="granite4:micro-h", reason="no parseable answer")
)
client = _make_client(runner=runner, repo=mock_analysis_repo)
class TestRagDepsMissing:
def test_503_when_docling_agent_not_installed(self, client: TestClient, monkeypatch) -> None:
_patched_settings(monkeypatch, rag_enabled=True)
# Simulate the import failing: remove any stub and ensure the name
# resolves to a module that raises on attribute access.
monkeypatch.setitem(sys.modules, "docling_agent.agents", None)
resp = client.post(
"/api/documents/doc-1/reasoning",
json={"query": "Quelle tarification ?"},
)
resp = client.post("/api/documents/doc-1/rag", json={"query": "Q"})
assert resp.status_code == 503
assert "docling-agent" in resp.json()["detail"]
class TestRagUpstreamFailure:
def test_502_when_docling_agent_raises_indexerror(
self, client: TestClient, stub_docling_agent, monkeypatch
) -> None:
"""Known docling-agent bug: `find_json_dicts(answer.value)[0]` raises
`IndexError` when the model fails to produce parseable JSON after
retries. Our endpoint must surface a 502 with a human-readable
message, not a 500 stack trace."""
_patched_settings(monkeypatch, rag_enabled=True, rag_model_id="granite4:micro-h")
_agent_class, agent_instance, _ = stub_docling_agent
agent_instance._rag_loop.side_effect = IndexError("list index out of range")
resp = client.post("/api/documents/doc-1/rag", json={"query": "Quelle tarification ?"})
assert resp.status_code == 502
detail = resp.json()["detail"]
assert "granite4:micro-h" in detail
assert "parseable" in detail or "rephrase" in detail
def test_500_for_other_unexpected_errors(self, mock_analysis_repo: AsyncMock) -> None:
runner = _FakeRunner(raises=RuntimeError("Ollama unreachable"))
client = _make_client(runner=runner, repo=mock_analysis_repo)
def test_500_for_other_unexpected_errors(
self, client: TestClient, stub_docling_agent, monkeypatch
) -> None:
_patched_settings(monkeypatch, rag_enabled=True)
_agent_class, agent_instance, _ = stub_docling_agent
agent_instance._rag_loop.side_effect = RuntimeError("Ollama unreachable")
resp = client.post("/api/documents/doc-1/reasoning", json={"query": "Q"})
resp = client.post("/api/documents/doc-1/rag", json={"query": "Q"})
assert resp.status_code == 500
assert "Ollama unreachable" in resp.json()["detail"]

View file

@ -1,7 +1,5 @@
"""Tests for persistence repositories using a temporary SQLite database."""
from datetime import datetime
import pytest
from domain.models import AnalysisJob, AnalysisStatus, Document
@ -123,7 +121,7 @@ class TestAnalysisRepo:
found = await analysis_repo.find_by_id("job-1")
assert found.status == AnalysisStatus.RUNNING
assert isinstance(found.started_at, datetime)
assert found.started_at is not None
async def test_update_status_completed(self, document_repo, analysis_repo):
await self._insert_doc(document_repo)
@ -191,7 +189,7 @@ class TestAnalysisRepo:
found = await analysis_repo.find_latest_completed_by_document("doc-1")
assert found is not None
assert found.id == "job-latest"
assert found.document_json == '{"body":{"children":[]},"texts":[]}'
assert found.document_json is not None
async def test_find_latest_completed_by_document_none(self, document_repo, analysis_repo):
await self._insert_doc(document_repo)

View file

@ -94,10 +94,7 @@ class TestParseResponse:
assert result.content_html == "<h1>Hello</h1>"
assert result.page_count == 1
assert result.pages[0].width == 612.0
# document_json mirrors the json_content stub
assert result.document_json is not None
assert '"texts"' in result.document_json
assert '"pages"' in result.document_json
def test_response_with_elements(self):
data = {

View file

@ -18,8 +18,6 @@ class TestSettingsDefaults:
assert s.lock_timeout == 300
assert s.max_page_count == 0
assert s.max_file_size_mb == 50
assert s.max_paste_image_size_mb == 10
assert s.paste_allowed_image_types == ["image/png", "image/jpeg", "image/webp"]
assert s.batch_page_size == 0
assert s.opensearch_default_limit == 1000
assert s.upload_dir == "./uploads"
@ -78,18 +76,6 @@ class TestSettingsValidation:
with pytest.raises(ValueError, match="max_file_size_mb must be >= 0"):
Settings(max_file_size_mb=-1)
def test_negative_max_paste_image_size_mb_rejected(self):
import pytest
with pytest.raises(ValueError, match="max_paste_image_size_mb must be >= 0"):
Settings(max_paste_image_size_mb=-1)
def test_empty_paste_allowed_image_types_rejected(self):
import pytest
with pytest.raises(ValueError, match="paste_allowed_image_types must not be empty"):
Settings(paste_allowed_image_types=[])
def test_zero_max_file_size_mb_accepted(self):
s = Settings(max_file_size_mb=0)
assert s.max_file_size_mb == 0

View file

@ -1,34 +0,0 @@
@ui
Feature: UI — Reasoning feature flag
# The Reasoning sidebar entry must be hidden when the backend is not wired
# to expose the live runner. This test runs against the default CI backend
# (REASONING_ENABLED unset → false), so `reasoningAvailable` from
# /api/health is false and the sidebar entry must not render.
#
# When REASONING_ENABLED=true on the backend with docling-agent installed,
# this scenario will fail — flip the assertion or drop the @reasoning-off
# tag in CI accordingly.
@reasoning-off
Scenario: Sidebar does not show Reasoning when the runner isn't wired
* driver uiBaseUrl
* waitFor('[data-e2e=sidebar]')
# The other nav items must still render
* waitFor('[data-e2e=nav-studio]')
* waitFor('[data-e2e=nav-documents]')
# nav-reasoning must NOT be present (v-if="reasoningEnabled" gates it on
# `reasoningAvailable` in the health response)
* match karate.sizeOf(locateAll('[data-e2e=nav-reasoning]')) == 0
@reasoning-off
Scenario: /reasoning route is reachable but flag-driven UI gates the entry point
* driver uiBaseUrl + '/reasoning'
# The route is registered in the SPA router so a deep-link doesn't 404,
# but the sidebar nav-reasoning entry stays hidden — users can't reach
# the page through normal navigation when the runner is disabled.
* waitFor('[data-e2e=sidebar]')
* match karate.sizeOf(locateAll('[data-e2e=nav-reasoning]')) == 0

View file

@ -9,8 +9,6 @@ RUN npm run build
FROM nginx:alpine
COPY --from=build /app/dist /usr/share/nginx/html
COPY nginx.conf.template /etc/nginx/templates/default.conf.template
ENV NGINX_MAX_BODY_SIZE=200M
COPY nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 80

View file

@ -20,6 +20,6 @@ server {
proxy_set_header X-Real-IP $remote_addr;
proxy_read_timeout 900s;
proxy_send_timeout 900s;
client_max_body_size ${NGINX_MAX_BODY_SIZE};
client_max_body_size 5M;
}
}

View file

@ -1,6 +1,6 @@
{
"name": "docling-studio",
"version": "0.5.0",
"version": "0.4.0",
"private": true,
"type": "module",
"scripts": {

View file

@ -114,38 +114,6 @@ describe('useFeatureFlagStore', () => {
expect(store.isEnabled('ingestion')).toBe(false)
})
it('enables reasoning when reasoningAvailable is true', async () => {
mockApiFetch.mockResolvedValue({
status: 'ok',
engine: 'local',
reasoningAvailable: true,
})
const store = useFeatureFlagStore()
await store.load()
expect(store.reasoningAvailable).toBe(true)
expect(store.isEnabled('reasoning')).toBe(true)
})
it('disables reasoning when reasoningAvailable is false', async () => {
mockApiFetch.mockResolvedValue({
status: 'ok',
engine: 'local',
reasoningAvailable: false,
})
const store = useFeatureFlagStore()
await store.load()
expect(store.reasoningAvailable).toBe(false)
expect(store.isEnabled('reasoning')).toBe(false)
})
it('defaults reasoningAvailable to false when missing', async () => {
mockApiFetch.mockResolvedValue({ status: 'ok', engine: 'local' })
const store = useFeatureFlagStore()
await store.load()
expect(store.reasoningAvailable).toBe(false)
expect(store.isEnabled('reasoning')).toBe(false)
})
it('handles health endpoint failure gracefully', async () => {
mockApiFetch.mockRejectedValue(new Error('Network error'))
const store = useFeatureFlagStore()

View file

@ -14,7 +14,6 @@ interface HealthResponse {
maxPageCount?: number
maxFileSizeMb?: number
ingestionAvailable?: boolean
reasoningAvailable?: boolean
}
export type FeatureFlag = 'chunking' | 'disclaimer' | 'ingestion' | 'reasoning'
@ -28,7 +27,6 @@ interface FeatureFlagContext {
engine: ConversionEngine | null
deploymentMode: DeploymentMode | null
ingestionAvailable: boolean
reasoningAvailable: boolean
}
const featureRegistry: Record<FeatureFlag, FeatureFlagDef> = {
@ -45,12 +43,11 @@ const featureRegistry: Record<FeatureFlag, FeatureFlagDef> = {
isEnabled: (ctx) => ctx.ingestionAvailable,
},
reasoning: {
// Backend-gated: `reasoningAvailable` is true on `/api/health` only when
// `REASONING_ENABLED=true` AND docling-agent + mellea are importable.
// Hides the sidebar entry when the runner isn't wired, instead of
// letting the user click through to a 503.
description: 'Reasoning trace tunnel (docling-agent ReasoningResult viewer)',
isEnabled: (ctx) => ctx.reasoningAvailable,
// SQLite-backed (builds the graph from `document_json` on the fly), so no
// server-side gating needed. Kept as a flag so a future deployment can
// still kill-switch the UI if it wants to.
description: 'Reasoning trace tunnel (docling-agent RAGResult viewer)',
isEnabled: () => true,
},
}
@ -60,7 +57,6 @@ export const useFeatureFlagStore = defineStore('feature-flags', () => {
const maxPageCount = ref<number>(0)
const maxFileSizeMb = ref<number>(0)
const ingestionAvailable = ref(false)
const reasoningAvailable = ref(false)
const appVersion = ref<string>(__APP_VERSION__)
const loaded = ref(false)
const error = ref<string | null>(null)
@ -69,7 +65,6 @@ export const useFeatureFlagStore = defineStore('feature-flags', () => {
engine: engine.value,
deploymentMode: deploymentMode.value,
ingestionAvailable: ingestionAvailable.value,
reasoningAvailable: reasoningAvailable.value,
}))
function isEnabled(flag: FeatureFlag): boolean {
@ -86,7 +81,6 @@ export const useFeatureFlagStore = defineStore('feature-flags', () => {
maxPageCount.value = data.maxPageCount ?? 0
maxFileSizeMb.value = data.maxFileSizeMb ?? 0
ingestionAvailable.value = data.ingestionAvailable ?? false
reasoningAvailable.value = data.reasoningAvailable ?? false
appMaxFileSizeMb.value = maxFileSizeMb.value
appMaxPageCount.value = maxPageCount.value
if (data.version) appVersion.value = data.version
@ -104,7 +98,6 @@ export const useFeatureFlagStore = defineStore('feature-flags', () => {
maxPageCount,
maxFileSizeMb,
ingestionAvailable,
reasoningAvailable,
appVersion,
loaded,
error,

View file

@ -1,8 +1,3 @@
// Integration test — exercises the cross-feature wiring between history,
// analysis and document stores when restoring a History → Studio navigation.
// Lives under `src/__tests__/integration/` because importing real stores from
// three sibling features would be a feature-isolation violation in any
// single-feature test file. HTTP boundaries are still mocked.
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { setActivePinia, createPinia } from 'pinia'
@ -12,28 +7,28 @@ vi.mock('vue-router', () => ({
useRouter: () => ({ push: mockPush }),
}))
vi.mock('../../features/analysis/api', () => ({
vi.mock('../analysis/api', () => ({
fetchAnalyses: vi.fn(),
fetchAnalysis: vi.fn(),
createAnalysis: vi.fn(),
deleteAnalysis: vi.fn(),
}))
vi.mock('../../features/history/api', () => ({
vi.mock('./api', () => ({
fetchHistory: vi.fn(),
deleteHistoryEntry: vi.fn(),
}))
vi.mock('../../features/document/api', () => ({
vi.mock('../document/api', () => ({
fetchDocuments: vi.fn(),
uploadDocument: vi.fn(),
deleteDocument: vi.fn(),
getPreviewUrl: vi.fn(),
}))
import { useHistoryStore } from '../../features/history/store'
import { useAnalysisStore } from '../../features/analysis/store'
import { useDocumentStore } from '../../features/document/store'
import { useHistoryStore } from './store'
import { useAnalysisStore } from '../analysis/store'
import { useDocumentStore } from '../document/store'
describe('History → Studio navigation', () => {
beforeEach(() => {
@ -43,7 +38,7 @@ describe('History → Studio navigation', () => {
describe('History store provides data for navigation', () => {
it('analyses contain documentId for document selection', async () => {
const { fetchHistory } = await import('../../features/history/api')
const { fetchHistory } = await import('./api')
fetchHistory.mockResolvedValue([
{ id: 'a1', documentId: 'd1', documentFilename: 'test.pdf', status: 'COMPLETED' },
{ id: 'a2', documentId: 'd2', documentFilename: 'other.pdf', status: 'FAILED' },
@ -59,7 +54,7 @@ describe('History → Studio navigation', () => {
describe('Analysis store select() restores analysis state', () => {
it('select() sets currentAnalysis from fetched data', async () => {
const { fetchAnalysis } = await import('../../features/analysis/api')
const { fetchAnalysis } = await import('../analysis/api')
const analysis = {
id: 'a1',
documentId: 'd1',
@ -77,7 +72,7 @@ describe('History → Studio navigation', () => {
})
it('select() allows document store to select the associated document', async () => {
const { fetchAnalysis } = await import('../../features/analysis/api')
const { fetchAnalysis } = await import('../analysis/api')
fetchAnalysis.mockResolvedValue({
id: 'a1',
documentId: 'd1',
@ -136,7 +131,7 @@ describe('History → Studio navigation', () => {
describe('Full restore flow (store-level integration)', () => {
it('restores completed analysis: selects analysis + document + verify mode', async () => {
const { fetchAnalysis } = await import('../../features/analysis/api')
const { fetchAnalysis } = await import('../analysis/api')
fetchAnalysis.mockResolvedValue({
id: 'a1',
documentId: 'd1',
@ -163,7 +158,7 @@ describe('History → Studio navigation', () => {
})
it('restores failed analysis: selects analysis + document, stays in configure mode', async () => {
const { fetchAnalysis } = await import('../../features/analysis/api')
const { fetchAnalysis } = await import('../analysis/api')
fetchAnalysis.mockResolvedValue({
id: 'a2',
documentId: 'd2',
@ -186,7 +181,7 @@ describe('History → Studio navigation', () => {
})
it('handles missing analysis gracefully', async () => {
const { fetchAnalysis } = await import('../../features/analysis/api')
const { fetchAnalysis } = await import('../analysis/api')
fetchAnalysis.mockRejectedValue(new Error('Not found'))
vi.spyOn(console, 'error').mockImplementation(() => {})

View file

@ -1,6 +1,6 @@
import { apiFetch } from '../../shared/api/http'
import type { GraphPayload } from '../analysis/graphApi'
import type { ReasoningResult } from './types'
import type { RAGResult } from './types'
/**
* Fetch the reasoning-trace graph for a document built on the backend from
@ -16,24 +16,20 @@ export function fetchReasoningGraph(docId: string): Promise<GraphPayload> {
}
/**
* Kick off a `docling-agent` reasoning run against a document and wait for
* the `ReasoningResult` (no streaming yet the backend blocks on `_rag_loop`
* and returns once the loop converges or hits `max_iterations`).
* Kick off a `docling-agent` RAG run against a document and wait for the
* `RAGResult` (no streaming yet the backend blocks on `_rag_loop` and
* returns once the loop converges or hits `max_iterations`).
*
* Runs typically take 2040s depending on the model + Ollama latency. The
* caller should show a loading state.
*
* Errors:
* - 503 if `REASONING_ENABLED=false` server-side or docling-agent isn't installed
* - 503 if `RAG_ENABLED=false` server-side or docling-agent isn't installed
* - 404 if no completed analysis exists for the doc
* - 500 if the loop itself raises (Ollama unreachable, model missing, )
*/
export function runReasoning(
docId: string,
query: string,
modelId?: string,
): Promise<ReasoningResult> {
return apiFetch<ReasoningResult>(`/api/documents/${encodeURIComponent(docId)}/reasoning`, {
export function runReasoning(docId: string, query: string, modelId?: string): Promise<RAGResult> {
return apiFetch<RAGResult>(`/api/documents/${encodeURIComponent(docId)}/rag`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({

View file

@ -10,7 +10,7 @@ import {
focusIteration,
nodeIdForSectionRef,
} from './graphReasoningOverlay'
import type { ReasoningResult } from './types'
import type { RAGResult } from './types'
function seed(): Core {
// Headless mode — no DOM container needed.
@ -25,9 +25,7 @@ function seed(): Core {
})
}
function result(
iterations: Array<Partial<ReasoningResult['iterations'][number]>>,
): ReasoningResult {
function result(iterations: Array<Partial<RAGResult['iterations'][number]>>): RAGResult {
return {
answer: 'x',
converged: true,

View file

@ -12,7 +12,7 @@
*/
import type { Core } from 'cytoscape'
import type { OverlayResult, ReasoningResult, ResolvedIteration } from './types'
import type { OverlayResult, RAGResult, ResolvedIteration } from './types'
export const REASONING_EDGE_TYPE = 'REASONING_NEXT'
const VISITED_CLASS = 'visited'
@ -43,7 +43,7 @@ export interface OverlayOptions {
}
/**
* Apply the overlay for a freshly imported `ReasoningResult`.
* Apply the overlay for a freshly imported `RAGResult`.
*
* Idempotent: any previous overlay is cleared first. Returns a summary that
* the caller (store / panel) uses to drive the UI (list of iterations,
@ -51,7 +51,7 @@ export interface OverlayOptions {
*/
export function applyReasoningOverlay(
cy: Core,
result: ReasoningResult,
result: RAGResult,
options: OverlayOptions = {},
): OverlayResult {
const focusMode = options.focusMode ?? true
@ -128,7 +128,7 @@ export function applyReasoningOverlay(
* is marked `present: false`; the panel can still render the iteration cards
* so the user sees the reasoning they just don't get highlights on the graph.
*/
export function buildDegradedOverlay(result: ReasoningResult): OverlayResult {
export function buildDegradedOverlay(result: RAGResult): OverlayResult {
const resolved: ResolvedIteration[] = result.iterations.map((it) => ({
iteration: it.iteration,
sectionRef: it.section_ref,

View file

@ -10,7 +10,7 @@ describe('parseImportedTrace', () => {
iterations: [],
}
it('accepts a bare ReasoningResult', () => {
it('accepts a bare RAGResult', () => {
const parsed = parseImportedTrace(bare)
expect(parsed?.result.answer).toBe('ok')
expect(parsed?.envelope).toBeNull()
@ -26,7 +26,7 @@ describe('parseImportedTrace', () => {
expect(parsed?.envelope?.job_id).toBe('abc')
})
it('rejects shapes that are neither envelope nor bare ReasoningResult', () => {
it('rejects shapes that are neither envelope nor bare RAGResult', () => {
expect(parseImportedTrace(null)).toBeNull()
expect(parseImportedTrace('string')).toBeNull()
expect(parseImportedTrace({ foo: 'bar' })).toBeNull()

View file

@ -1,36 +1,36 @@
import { defineStore } from 'pinia'
import { computed, ref } from 'vue'
import type { OverlayResult, ReasoningResult, ResolvedIteration, SidecarEnvelope } from './types'
import type { OverlayResult, RAGResult, ResolvedIteration, SidecarEnvelope } from './types'
/**
* Parse an arbitrary JSON payload as either:
* - a bare `ReasoningResult` (what `docling-agent` emits directly), or
* - a bare `RAGResult` (what `docling-agent` emits directly), or
* - the sidecar envelope (`{ job_id, filename, query, model, result }`)
*
* Returns `null` if the shape doesn't match either.
*/
export function parseImportedTrace(
raw: unknown,
): { result: ReasoningResult; envelope: SidecarEnvelope | null } | null {
): { result: RAGResult; envelope: SidecarEnvelope | null } | null {
if (!raw || typeof raw !== 'object') return null
const obj = raw as Record<string, unknown>
// Envelope shape
if (obj.result && typeof obj.result === 'object') {
const result = obj.result as ReasoningResult
if (isReasoningResult(result)) {
const result = obj.result as RAGResult
if (isRAGResult(result)) {
return { result, envelope: obj as unknown as SidecarEnvelope }
}
}
// Bare ReasoningResult
if (isReasoningResult(obj as unknown as ReasoningResult)) {
return { result: obj as unknown as ReasoningResult, envelope: null }
// Bare RAGResult
if (isRAGResult(obj as unknown as RAGResult)) {
return { result: obj as unknown as RAGResult, envelope: null }
}
return null
}
function isReasoningResult(x: ReasoningResult | undefined): boolean {
function isRAGResult(x: RAGResult | undefined): boolean {
if (!x || typeof x !== 'object') return false
return (
typeof x.answer === 'string' && typeof x.converged === 'boolean' && Array.isArray(x.iterations)
@ -39,12 +39,12 @@ function isReasoningResult(x: ReasoningResult | undefined): boolean {
export const useReasoningStore = defineStore('reasoning', () => {
const importDialogOpen = ref(false)
// Separate modal for the live runner (POST /api/documents/:id/reasoning),
// so it can coexist with the import dialog conceptually even if only one
// is ever open at a time.
// Separate modal for the live runner (POST /api/documents/:id/rag), so it
// can coexist with the import dialog conceptually even if only one is ever
// open at a time.
const runDialogOpen = ref(false)
const running = ref(false)
const rawResult = ref<ReasoningResult | null>(null)
const rawResult = ref<RAGResult | null>(null)
const envelope = ref<SidecarEnvelope | null>(null)
const overlay = ref<OverlayResult | null>(null)
const activeIteration = ref<number | null>(null)
@ -84,7 +84,7 @@ export const useReasoningStore = defineStore('reasoning', () => {
* Does NOT touch Cytoscape the `ReasoningPanel` watches `rawResult` and
* reapplies the overlay via `graphReasoningOverlay.applyReasoningOverlay`.
*/
function setResult(result: ReasoningResult, env: SidecarEnvelope | null): void {
function setResult(result: RAGResult, env: SidecarEnvelope | null): void {
rawResult.value = result
envelope.value = env
error.value = null

View file

@ -1,6 +1,5 @@
/**
* Types mirroring the `docling-agent` reasoning-trace output (upstream type
* name: `RAGResult`).
* Types mirroring the `docling-agent` RAG output.
*
* The JSON imported by the user is produced either by:
* - the R&D sidecar (`experiments/reasoning-trace/inspect_doc.py`), or
@ -15,7 +14,7 @@
* Source of truth: docling-project/docling-agent @ docling_agent/agent/rag_models.py
*/
export interface ReasoningIteration {
export interface RAGIteration {
iteration: number
section_ref: string
reason: string
@ -24,15 +23,15 @@ export interface ReasoningIteration {
response: string
}
export interface ReasoningResult {
export interface RAGResult {
answer: string
iterations: ReasoningIteration[]
iterations: RAGIteration[]
converged: boolean
}
/**
* Envelope written by the R&D sidecar. The viewer also accepts a bare
* `ReasoningResult` (see `parseImportedTrace` in the store).
* `RAGResult` (see `parseImportedTrace` in the store).
*/
export interface SidecarEnvelope {
job_id?: string
@ -40,7 +39,7 @@ export interface SidecarEnvelope {
query?: string
model?: { ollama_name?: string | null; hf_model_name?: string | null }
max_iterations?: number
result: ReasoningResult
result: RAGResult
}
/**

View file

@ -2,7 +2,6 @@ import { defineStore } from 'pinia'
import { ref, watch, watchEffect } from 'vue'
import type { Locale, Theme } from '../../shared/types'
import { appLocale } from '../../shared/appConfig'
import { STORAGE_KEYS } from '../../shared/storage/keys'
function safeGetItem(key: string): string | null {
try {
@ -21,14 +20,15 @@ function safeSetItem(key: string, value: string): void {
}
export const useSettingsStore = defineStore('settings', () => {
const theme = ref<Theme>((safeGetItem(STORAGE_KEYS.theme) as Theme) || 'dark')
const locale = ref<Locale>((safeGetItem(STORAGE_KEYS.locale) as Locale) || 'fr')
const apiUrl = ref('http://localhost:8000')
const theme = ref<Theme>((safeGetItem('docling-theme') as Theme) || 'dark')
const locale = ref<Locale>((safeGetItem('docling-locale') as Locale) || 'fr')
watch(theme, (v) => safeSetItem(STORAGE_KEYS.theme, v))
watch(theme, (v) => safeSetItem('docling-theme', v))
watch(
locale,
(v) => {
safeSetItem(STORAGE_KEYS.locale, v)
safeSetItem('docling-locale', v)
appLocale.value = v
},
{ immediate: true },
@ -45,5 +45,5 @@ export const useSettingsStore = defineStore('settings', () => {
locale.value = l
}
return { theme, locale, setTheme, setLocale }
return { apiUrl, theme, locale, setTheme, setLocale }
})

View file

@ -123,21 +123,21 @@ const messages: Messages = {
'studio.prepare': 'Préparer',
'studio.ingest': 'Ingérer',
'studio.maintain': 'Maintenir',
// Reasoning trace (R&D v1 — overlays a docling-agent ReasoningResult on the graph)
// Reasoning trace (R&D v1 — overlays a docling-agent RAGResult on the graph)
'reasoning.importBtn': 'Importer une trace de raisonnement',
'reasoning.importTitle': 'Importer une trace de raisonnement',
'reasoning.importHint':
'Dépose un JSON de trace de raisonnement produit par docling-agent (ou par le script R&D experiments/reasoning-trace).',
'Dépose un JSON RAGResult produit par docling-agent (ou par le script R&D experiments/reasoning-trace).',
'reasoning.drop': 'Glisse un fichier .json ici',
'reasoning.dropSub': 'ou clique pour le choisir',
'reasoning.parsing': 'Analyse du fichier...',
'reasoning.pasteToggle': 'Coller le JSON à la place',
'reasoning.pastePlaceholder': "Colle ici le contenu JSON d'une trace de raisonnement...",
'reasoning.pastePlaceholder': "Colle ici le contenu JSON d'un RAGResult...",
'reasoning.pasteSubmit': 'Charger',
'reasoning.close': 'Fermer',
'reasoning.errJson': 'JSON invalide : {msg}',
'reasoning.errShape':
"Le fichier n'a pas la forme d'une trace de raisonnement (answer, converged, iterations).",
"Le fichier n'a pas la forme d'un RAGResult (answer, converged, iterations).",
'reasoning.panelTitle': 'Trace de raisonnement',
'reasoning.focus': 'Focus',
'reasoning.focusHint':
@ -166,7 +166,7 @@ const messages: Messages = {
// Reasoning page (standalone tunnel)
'reasoning.pageTitle': 'Reasoning Trace',
'reasoning.pageSubtitle':
'Importe un PDF, puis dépose une trace de raisonnement produite par docling-agent pour visualiser le chemin de raisonnement sur le graphe du document.',
'Importe un PDF, puis dépose une trace RAGResult produite par docling-agent pour visualiser le chemin de raisonnement sur le graphe du document.',
'reasoning.dropPdf': 'Dépose un PDF',
'reasoning.dropPdfHint': 'ou clique pour en choisir un',
'reasoning.uploading': 'Import du document...',
@ -191,7 +191,7 @@ const messages: Messages = {
'reasoning.runModelLabel': 'Modèle (optionnel)',
'reasoning.runModelPlaceholder': 'gpt-oss:20b',
'reasoning.runModelSub':
'Nom du modèle Ollama. Laisser vide pour utiliser le défaut serveur (REASONING_MODEL_ID).',
'Nom du modèle Ollama. Laisser vide pour utiliser le défaut serveur (RAG_MODEL_ID).',
'reasoning.runSubmit': 'Lancer',
'reasoning.running': 'docling-agent tourne... (20-40s)',
'reasoning.runErrUnknown': 'Erreur inconnue lors de l\u2019appel à docling-agent.',
@ -263,6 +263,7 @@ const messages: Messages = {
// Settings
'settings.title': 'Paramètres',
'settings.apiUrl': 'API URL',
'settings.version': 'Version',
'settings.theme': 'Thème',
'settings.themeDark': 'Sombre',
@ -383,21 +384,20 @@ const messages: Messages = {
'studio.prepare': 'Prepare',
'studio.ingest': 'Ingest',
'studio.maintain': 'Maintain',
// Reasoning trace (R&D v1 — overlays a docling-agent ReasoningResult on the graph)
// Reasoning trace (R&D v1 — overlays a docling-agent RAGResult on the graph)
'reasoning.importBtn': 'Import reasoning trace',
'reasoning.importTitle': 'Import reasoning trace',
'reasoning.importHint':
'Drop a reasoning-trace JSON produced by docling-agent (or by the experiments/reasoning-trace R&D script).',
'Drop a RAGResult JSON produced by docling-agent (or by the experiments/reasoning-trace R&D script).',
'reasoning.drop': 'Drop a .json file here',
'reasoning.dropSub': 'or click to pick one',
'reasoning.parsing': 'Parsing file...',
'reasoning.pasteToggle': 'Paste JSON instead',
'reasoning.pastePlaceholder': 'Paste a reasoning-trace JSON payload here...',
'reasoning.pastePlaceholder': 'Paste a RAGResult JSON payload here...',
'reasoning.pasteSubmit': 'Load',
'reasoning.close': 'Close',
'reasoning.errJson': 'Invalid JSON: {msg}',
'reasoning.errShape':
"File doesn't look like a reasoning trace (answer, converged, iterations).",
'reasoning.errShape': "File doesn't look like a RAGResult (answer, converged, iterations).",
'reasoning.panelTitle': 'Reasoning trace',
'reasoning.focus': 'Focus',
'reasoning.focusHint': 'Dim non-visited elements to make the reasoning path stand out.',
@ -425,7 +425,7 @@ const messages: Messages = {
// Reasoning page (standalone tunnel)
'reasoning.pageTitle': 'Reasoning Trace',
'reasoning.pageSubtitle':
"Drop a PDF, then import a reasoning trace from docling-agent to visualize the reasoning path on the document's graph.",
"Drop a PDF, then import a RAGResult trace from docling-agent to visualize the reasoning path on the document's graph.",
'reasoning.dropPdf': 'Drop a PDF',
'reasoning.dropPdfHint': 'or click to pick one',
'reasoning.uploading': 'Uploading document...',
@ -450,7 +450,7 @@ const messages: Messages = {
'reasoning.runModelLabel': 'Model (optional)',
'reasoning.runModelPlaceholder': 'gpt-oss:20b',
'reasoning.runModelSub':
'Ollama model name. Leave empty to use the server default (REASONING_MODEL_ID).',
'Ollama model name. Leave empty to use the server default (RAG_MODEL_ID).',
'reasoning.runSubmit': 'Run',
'reasoning.running': 'docling-agent is thinking… (2040s)',
'reasoning.runErrUnknown': 'Unknown error while calling docling-agent.',
@ -518,6 +518,7 @@ const messages: Messages = {
'pagination.perPage': '/ page',
'settings.title': 'Settings',
'settings.apiUrl': 'API URL',
'settings.version': 'Version',
'settings.theme': 'Theme',
'settings.themeDark': 'Dark',

View file

@ -1,14 +0,0 @@
/**
* Centralised localStorage keys.
*
* Every key the frontend writes to `localStorage` lives here so we have a
* single place to audit naming, namespace collisions, or migrations. All
* keys share the `docling-` prefix to avoid clashing with other apps that
* might be served from the same origin (e.g. on Hugging Face Spaces).
*/
export const STORAGE_KEYS = {
theme: 'docling-theme',
locale: 'docling-locale',
} as const
export type StorageKey = (typeof STORAGE_KEYS)[keyof typeof STORAGE_KEYS]

View file

@ -57,7 +57,7 @@ nav:
- Audit:
- Audit Master: audit/master.md
- Audits:
- Hexagonal Architecture: audit/audits/01-clean-architecture.md
- Clean Architecture: audit/audits/01-clean-architecture.md
- DDD: audit/audits/02-ddd.md
- Clean Code: audit/audits/03-clean-code.md
- KISS: audit/audits/04-kiss.md
@ -76,12 +76,6 @@ validation:
absolute_links: info
links:
absolute_links: info
# Audit reports (audit/reports/release-X.Y.Z/) reference repo source
# files (e.g. `[file.py:line](file.py:line)`) by design — they're read
# alongside the repo, not as standalone docs. Downgrade these to info
# so strict-mode builds don't fail on intentional cross-tree links.
not_found: info
unrecognized_links: info
markdown_extensions:
- admonition

View file

@ -20,6 +20,6 @@ server {
proxy_set_header X-Real-IP $remote_addr;
proxy_read_timeout 900s;
proxy_send_timeout 900s;
client_max_body_size ${NGINX_MAX_BODY_SIZE};
client_max_body_size 5M;
}
}