Commit graph

12 commits

Author SHA1 Message Date
Pier-Jean Malandrino
efc27932dd refactor(audit): remediate 0.5.0 audit findings — clean architecture, security, DRY, SOLID, perf
Closes the 12 MAJ raised by the release/0.5.0 audit pipeline (cf.
docs/audit/reports/release-0.5.0/summary.md → summary-reaudit.md).

Volet 1 — Reasoning architecture (audits 01/02/06/07 strengthening)
  * Domain ports: LLMProvider, ReasoningRunner, ReasoningParseError
  * Domain DTOs: LLMProviderType, ReasoningResult, ReasoningIteration
  * infra/llm/ollama_provider.py — OllamaProvider with health_check
  * infra/docling_agent_reasoning.py — runner adapter, encapsulates the
    private _rag_loop call (tracked at docling-project/docling-agent#26),
    commits OLLAMA_HOST once at boot (eliminates the per-request env race),
    translates upstream IndexError into ReasoningParseError
  * api/reasoning.py — zero coupling to docling-agent / mellea / docling-core,
    consumes app.state.reasoning_runner via the port
  * main.py — DI wires OllamaProvider + DoclingAgentReasoningRunner at boot
    when REASONING_ENABLED=true and deps are importable
  * Rename RAG_* env vars → REASONING_*, endpoint /rag → /reasoning,
    type RAGResult → ReasoningResult, frontend feature flag wiring,
    i18n strings, tests, docs (BREAKING — pre-1.0 surface, no external
    consumers in production)
  * 17 new tests: adapter unit tests with sys.modules stubs, OllamaProvider
    httpx tests, R3 concurrent-host isolation, R6 multi-iteration trace
    serialization, R13 Protocol conformance via isinstance
  * E2E Karate scenario: nav-reasoning hidden when REASONING_ENABLED=false
  * README — Live Reasoning section (env vars, archi, link to issue #26)

Bloc B — Security (audit 08, dev-only context)
  * docker-compose.yml — DEV DEFAULTS header, OpenSearch DISABLE_SECURITY_PLUGIN
    flagged as dev-only with link to OpenSearch security docs
  * main.py — boot warning if NEO4J_URI is set with the default 'changeme'
    password, so prod operators can't silently inherit it

Bloc C — DRY frontend (audit 05)
  * shared/storage/keys.ts — STORAGE_KEYS centralised (theme, locale)
  * features/settings/store.ts — dead apiUrl ref + orphan i18n keys removed
  * api/schemas.py — DOCUMENT_STATUS_UPLOADED constant

Bloc D — Quality (audits 02/06/07/09/10/12)
  * domain/ports.py — DocumentConverter.supports_page_batching property
    (LSP fix, replaces isinstance(ServeConverter) check)
  * domain/ports.py — VectorStore.ping() (encapsulation, replaces
    _vector_store._client.info() reach-around)
  * api/analyses.py + api/ingestion.py — path params {job_id} → {analysis_id}
    aligned with the user-facing terminology (URLs unchanged)
  * api/documents.py — Path.read_bytes() + generate_preview() wrapped in
    asyncio.to_thread, unblocks the FastAPI event loop on /preview
  * infra/docling_tree.py — PEP 604 union for isinstance (Ruff UP038)
  * src/__tests__/integration/ — cross-feature integration test relocated
    out of features/history/ so feature folders stay self-contained
  * Tightened terminal `assert X is not None` checks (isinstance(.., datetime),
    exact value comparisons)

Validation
  * 446 backend pytest, 202 frontend vitest — all green
  * ruff + ruff format + ESLint + Prettier + vue-tsc clean
  * Re-audit verdict: 0 CRIT / 0 MAJ, score ~94/100, GO

Closes #200
2026-04-29 14:00:00 +02:00
Pier-Jean Malandrino
e0f8e81b63 feat: enable chunking in remote (Docling Serve) mode
Hybrid approach: reuse LocalChunker to chunk the DoclingDocument JSON
returned by Serve, so chunking works identically in both local and
remote modes without calling Serve's chunk endpoint.

Backend:
- _build_chunker() always returns LocalChunker (remove engine guard)
- Use docling-core[chunking] extra for required dependencies
- Skip client-side batching in remote mode (Serve manages its own
  resources, and batching discards document_json needed for chunking)
- Fix Serve form fields: remove generate_page_images (not a Serve
  field), use repeated form keys for to_formats and page_range
- Log Serve error response body on 4xx/5xx for diagnosis
- Fix FastAPI 204 DELETE routes missing response_model=None

Frontend:
- Update chunking feature flag to enable Prepare UI in remote mode

Closes #51
2026-04-29 14:00:00 +02:00
Pier-Jean Malandrino
fc866ce229 feat: batch large documents with page_range and progress reporting
Use Docling's native page_range parameter to split large PDFs into
sequential batches, preventing memory exhaustion and timeouts.
Progress is reported via existing polling mechanism.

Closes #56
2026-04-07 17:54:40 +02:00
Pier-Jean Malandrino
1c82a80389 Guard all docling-dependent tests for lightweight CI
local_converter imports docling at module level, so any test that
touches it crashes when docling is not installed. Skip those tests
with importorskip / skipif so the CI (which only has docling-core)
passes cleanly.
2026-04-03 15:46:29 +02:00
Pier-Jean Malandrino
2cbef859e7 Fix generate_page_images option silently dropped in remote mode
The form data builder was not forwarding generate_page_images to
Docling Serve, so page image generation was silently ignored.
2026-04-03 13:01:50 +02:00
Pier-Jean Malandrino
e1636e710d Fix ServeConverter not returning document_json for chunking
The remote converter never set document_json in ConversionResult,
silently preventing chunking from working in remote mode.
2026-04-03 13:00:43 +02:00
Pier-Jean Malandrino
05de0cc774 Add chunking tests and update existing test assertions
16 new chunking tests (domain, schemas, API endpoints).
Update existing tests for chunkingOptions parameter and document_json mock.
2026-04-02 12:33:07 +02:00
Pier-Jean Malandrino
1b8cfe0a6b Fix Serve API contract: send to_formats as repeated form fields
Docling Serve expects array fields (to_formats) as repeated multipart
keys (to_formats=md&to_formats=html&to_formats=json), not a JSON
string. Changed _build_form_data to return list[tuple] so httpx sends
repeated keys correctly. Fixes 422 Unprocessable Entity on convert.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 16:52:36 +02:00
Pier-Jean Malandrino
0af3b81b8f Fix zombie jobs and unprotected JSON parse
Bug #1: _on_task_done now receives job_id via functools.partial and
calls _mark_failed when the background task raises or is cancelled,
preventing jobs from being stuck in RUNNING state forever.

Bug #5: _parse_response wraps json.loads in try/except JSONDecodeError
so malformed json_content strings fall back gracefully instead of crashing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 16:52:36 +02:00
Pier-Jean Malandrino
a63af61b73 Fix lint ans test errors 2026-03-31 16:35:48 +02:00
Pier-Jean Malandrino
fe4e792885 Fix audit findings: remove domain→infra violation, align Serve API, fix DI
- Delete domain/parsing.py (broke hexagonal layering by importing infra)
- Migrate all tests to import directly from domain.value_objects and
  infra.local_converter
- Rewrite ServeConverter to match real Docling Serve v1 API contract:
  options sent as individual form fields (not JSON blob), response
  parsed from document.json_content (DoclingDocument), proper bbox
  coord_origin handling (TOPLEFT/BOTTOMLEFT)
- Transmit all conversion options including generate_picture_images
- Replace fragile lazy import circular dep with FastAPI Depends() +
  app.state for AnalysisService injection
- Add frontend file size validation (50MB) before upload

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 10:58:58 +02:00
Pier-Jean Malandrino
a3486a8501 Add ServeConverter adapter for remote Docling Serve integration
Implement the HTTP client adapter that delegates document conversion
to a remote Docling Serve instance via its /v1/convert/file endpoint.
Switchable via CONVERSION_ENGINE=remote env var. Includes health check,
API key auth, response parsing, and 30 new tests covering parsing,
type mapping, HTTP calls, and DI wiring.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 10:36:35 +02:00