Backend — live runner
- New `POST /api/documents/:id/rag` endpoint. Loads `document_json` from
SQLite, reconstructs the DoclingDocument, wraps the model id in
`ModelIdentifier(ollama_name=...)`, and calls `agent._rag_loop`
off-thread (blocking sync call). Returns a `RAGResult` in the shape
the existing v1 import path already consumes, so the frontend overlay
is fully reused.
- `_rag_loop` is private upstream; we call it because `run()` wraps the
answer in a synthetic DoclingDocument and drops the iteration trace.
- Settings: `RAG_ENABLED`, `OLLAMA_HOST`, `RAG_MODEL_ID`. Router mounts
unconditionally; handler 503s when the flag is off or deps aren't
installed. `rag_available` surfaced in `/api/health`.
- Maps known docling-agent bugs to readable HTTP errors: 502 with
"the model couldn't produce a parseable answer" when `_rag_loop`
raises `IndexError` from `find_json_dicts([])[0]` after 3 + 3
rejection-sampling retries (model-dependent).
- Tests: 11 cases (flag off, query empty, no analysis, happy path,
model_id wrap, Ollama env, IndexError → 502, other errors → 500,
deps missing → 503).
Backend — bug fix
- Default `BATCH_PAGE_SIZE` flipped from `10` to `0` to match the
dataclass default. The old default silently dropped `document_json`
(see `domain/services.merge_results`) for any doc > 10 pages, which
broke the reasoning tunnel. Set `BATCH_PAGE_SIZE>0` explicitly on
memory-constrained deploys if batching is wanted.
Frontend — runner UX
- `features/reasoning/api.ts:runReasoning()` — POST wrapper.
- `RunReasoningDialog.vue` — query textarea + optional model_id
override. Blocks close while running, 20-40s loading state,
synthesises a sidecar-shaped envelope so the panel surfaces query +
model the same way an imported trace would.
- `ReasoningWorkspace.vue` — primary "Run reasoning" button; "Import
trace" relegated to ghost secondary.
- Store: `runDialogOpen`, `running`, `setRunning`.
Frontend — answer polish
- Answer rendered through `marked` + DOMPurify (models emit markdown
lists; `pre-wrap` rendered them as plain "1. …" strings).
- Dedicated answer block with orange border, "ANSWER" label, "Copy"
button (clipboard + "Copied ✓" feedback).
- IterationCard: drop the duplicate `response` block (the main answer
is authoritative); style reasons equal to `"fallback"` (docling-agent
`select_from_failure` placeholder) as italic muted "— no structured
rationale".
Frontend — node details contents
- Clicking a SectionHeader (or any node with compound children) lists
its contained elements in `NodeDetailsPanel` under a new "Contents"
block. Children come from the same `parentMap` used for Cytoscape
compound parenting (explicit PARENT_OF + synthetic section scope),
inverted once and cached as a computed.
- Click a child row → pan the viewport to it + swap the selection.
Housekeeping
- `cytoscape-navigator` removed from `package-lock.json` (follow-up
from the minimap removal in the previous commit).
Adds the `docling-agent` reasoning-trace viewer as a Studio tunnel, per
`docs/design/reasoning-trace.md`. Users pick an analyzed document, import
a RAGResult JSON, and the iterations are overlaid on the document graph.
Graph source is decoupled from Neo4j: a new pure builder
(`infra/docling_graph.build_graph_payload`) reads `document_json` from
SQLite and emits the same Cytoscape-shaped payload that `fetch_graph`
returns from Neo4j. Neo4j stays exclusive to the Maintain ingestion
pipeline. Shared DoclingDocument helpers live in `infra/docling_tree.py`
so TreeWriter and the builder can't drift on label taxonomy or tree walks.
Also removes the Cytoscape minimap (cytoscape-navigator) from GraphView:
second render instance hurt perf on large documents for no UX win.
Backend
- new `GET /api/documents/:id/reasoning-graph` (SQLite-only)
- new `infra/docling_tree.py`, `infra/docling_graph.py`
- `analysis_repo.find_latest_completed_by_document`
- tests: `test_docling_graph.py` (builder), `test_graph_api.py` (endpoint)
Frontend
- `features/reasoning/` — store, overlay, types, panel, import dialog,
workspace, doc picker
- new `ReasoningPage` + `/reasoning` and `/reasoning/:docId` routes
- `GraphView` gains a `fetcher` prop so reasoning can inject the
SQLite-backed fetcher while Maintain keeps using the Neo4j one
- drops minimap (nav container, dep, CSS)
- legend filters + section parenting extracted for reuse
- i18n base strings (FR + EN)
ChunkWriter mirrors chunks into Neo4j after OpenSearch indexing, creating
HAS_CHUNK edges and DERIVED_FROM back-references to the source Elements
(via doc_items propagated from the local chunker).
Graph API: GET /api/documents/{id}/graph returns a cytoscape-shaped
payload with nodes + edges for Document / Element / Page / Chunk.
Hard cap at 200 pages returns HTTP 413 per design §8.4.
Frontend: new Graph tab in Studio results, rendered with Cytoscape.js +
dagre layout (lazy-loaded, ~175 KB gz). Legend, node styling per element
label, directional edges styled per edge type.
README gains a Neo4j section with the schema, three demo Cypher
queries, and env vars. Backend tests skip cleanly when the neo4j python
package is not installed locally.
Refs #186
Hybrid approach: reuse LocalChunker to chunk the DoclingDocument JSON
returned by Serve, so chunking works identically in both local and
remote modes without calling Serve's chunk endpoint.
Backend:
- _build_chunker() always returns LocalChunker (remove engine guard)
- Use docling-core[chunking] extra for required dependencies
- Skip client-side batching in remote mode (Serve manages its own
resources, and batching discards document_json needed for chunking)
- Fix Serve form fields: remove generate_page_images (not a Serve
field), use repeated form keys for to_formats and page_range
- Log Serve error response body on 4xx/5xx for diagnosis
- Fix FastAPI 204 DELETE routes missing response_model=None
Frontend:
- Update chunking feature flag to enable Prepare UI in remote mode
Closes#51
- Conditionally mount ingestion router only when OpenSearch + embedding are configured
- Add `ingestionAvailable` field to /api/health response
- Add `ingestion` feature flag to frontend (hides Search nav, Ingest button,
OpenSearch badge, indexed badges/filters when disabled)
- Skip ingestion polling when flag is off
- Make OpenSearch + embedding optional in docker-compose via profiles
- Add docker-compose.ingestion.yml override for full-stack ingestion
- Set BATCH_PAGE_SIZE=5 default in Docker local image
- Lead Quick Start with one-liner docker run command
- Document ingestion as opt-in with dedicated section
- Add BATCH_PAGE_SIZE, MAX_FILE_SIZE_MB, MAX_PAGE_COUNT, RATE_LIMIT_RPM to config tables
- Update test counts (380 backend, 159 frontend)
- Date CHANGELOG 0.4.0, bump frontend version to 0.4.0
- Sync CONTRIBUTING.md with E2E Karate test sections
Closes#180
Backend: GET /api/ingestion/search?q=…&doc_id=… endpoint with
SearchResponse schema. Frontend: search bar in Documents page, results
with filename, page, chunk index, relevance score. 3 new API tests.
Frontend decoupling:
- Create shared/appConfig.ts as reactive bridge (locale, maxFileSizeMb,
maxPageCount) eliminating shared→features and feature→feature imports
- Give history feature its own Pinia store and API layer (was re-export
of analysis store)
- Give chunking feature its own Pinia store and API layer (was importing
from analysis)
- ChunkPanel receives analysis data via props instead of cross-feature
store import
- document/store reads maxFileSizeMb from shared config instead of
importing feature-flags store
- shared/i18n reads locale from shared config instead of importing
settings store
Backend:
- Add HealthResponse Pydantic schema for /api/health endpoint
Closes#140, closes#141, closes#142, closes#143
- Add DocumentRepository and AnalysisRepository protocols in domain/ports.py (#128)
- Refactor persistence repos from module functions to SqliteDocumentRepository
and SqliteAnalysisRepository classes
- Inject repos into AnalysisService and new DocumentService class via
constructor, removing direct imports of persistence and infra.settings (#129)
- Move _merge_results, _classify_error, _extract_html_body to domain/services.py (#130)
- Update main.py composition root to build and wire all dependencies
- Switch api/documents.py to Depends pattern matching api/analyses.py
- Update all tests to use injected mocks instead of module-level patches
Closes#128, closes#129, closes#130
Replace hardcoded 5 MB upload limit with a configurable setting.
Backend exposes the value via /api/health, frontend reads it
dynamically for validation and UI messages.
Closes#48
Use Docling's native page_range parameter to split large PDFs into
sequential batches, preventing memory exhaustion and timeouts.
Progress is reported via existing polling mechanism.
Closes#56
Prevents PyTorch/Docling pipeline crashes on HF Spaces CPU by:
- Reducing max file size from 50 MB to 5 MB
- Adding configurable MAX_PAGE_COUNT setting (env var, default unlimited)
- Increasing conversion timeout from 600s to 900s
- Adding frontend upload validation with explicit error messages
- Exposing maxPageCount via /api/health for dynamic UI hints
Adapt test expectations to external changes: upload returns 200,
ValueError yields 400, and schemas now accept both snake_case and
camelCase via AliasChoices.
Read uploaded files in fixed-size chunks instead of a single file.read()
to reduce peak memory pressure. Also enforces the size limit during
streaming so oversized payloads are rejected before fully buffered.
All endpoint functions, lifespan context manager, and get_connection now
have explicit return types. Upload endpoint returns 201 Created instead
of 200 to follow REST conventions.
Replace broad `except Exception` with specific types (FileNotFoundError,
PermissionError, OSError) so errors are properly categorized in logs.
Users reporting bugs will get actionable messages instead of generic ones.
Extract bounding boxes from chunk doc_items provenance in the chunker,
propagate through domain/service/API layers, and render highlighted
bboxes on canvas when hovering a chunk card. Reset highlights on
mode and page changes to prevent stale visual state.
AnalysisService gains rechunk() and inline chunking during conversion.
ChunkingOptionsRequest/ChunkResponse schemas, POST rechunk endpoint,
and conditional chunker injection in main.py (local engine only).
- Delete domain/parsing.py (broke hexagonal layering by importing infra)
- Migrate all tests to import directly from domain.value_objects and
infra.local_converter
- Rewrite ServeConverter to match real Docling Serve v1 API contract:
options sent as individual form fields (not JSON blob), response
parsed from document.json_content (DoclingDocument), proper bbox
coord_origin handling (TOPLEFT/BOTTOMLEFT)
- Transmit all conversion options including generate_picture_images
- Replace fragile lazy import circular dep with FastAPI Depends() +
app.state for AnalysisService injection
- Add frontend file size validation (50MB) before upload
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extract domain value objects and ports from parsing.py, move Docling-specific
code to infra/local_converter.py, and convert analysis_service to a class
with injected DocumentConverter. This prepares the codebase for plugging in
alternative conversion backends (e.g. Docling Serve) via the Protocol pattern.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
