Commit graph

15 commits

Author SHA1 Message Date
Pier-Jean Malandrino
efc27932dd refactor(audit): remediate 0.5.0 audit findings — clean architecture, security, DRY, SOLID, perf
Closes the 12 MAJ raised by the release/0.5.0 audit pipeline (cf.
docs/audit/reports/release-0.5.0/summary.md → summary-reaudit.md).

Volet 1 — Reasoning architecture (audits 01/02/06/07 strengthening)
  * Domain ports: LLMProvider, ReasoningRunner, ReasoningParseError
  * Domain DTOs: LLMProviderType, ReasoningResult, ReasoningIteration
  * infra/llm/ollama_provider.py — OllamaProvider with health_check
  * infra/docling_agent_reasoning.py — runner adapter, encapsulates the
    private _rag_loop call (tracked at docling-project/docling-agent#26),
    commits OLLAMA_HOST once at boot (eliminates the per-request env race),
    translates upstream IndexError into ReasoningParseError
  * api/reasoning.py — zero coupling to docling-agent / mellea / docling-core,
    consumes app.state.reasoning_runner via the port
  * main.py — DI wires OllamaProvider + DoclingAgentReasoningRunner at boot
    when REASONING_ENABLED=true and deps are importable
  * Rename RAG_* env vars → REASONING_*, endpoint /rag → /reasoning,
    type RAGResult → ReasoningResult, frontend feature flag wiring,
    i18n strings, tests, docs (BREAKING — pre-1.0 surface, no external
    consumers in production)
  * 17 new tests: adapter unit tests with sys.modules stubs, OllamaProvider
    httpx tests, R3 concurrent-host isolation, R6 multi-iteration trace
    serialization, R13 Protocol conformance via isinstance
  * E2E Karate scenario: nav-reasoning hidden when REASONING_ENABLED=false
  * README — Live Reasoning section (env vars, archi, link to issue #26)

Bloc B — Security (audit 08, dev-only context)
  * docker-compose.yml — DEV DEFAULTS header, OpenSearch DISABLE_SECURITY_PLUGIN
    flagged as dev-only with link to OpenSearch security docs
  * main.py — boot warning if NEO4J_URI is set with the default 'changeme'
    password, so prod operators can't silently inherit it

Bloc C — DRY frontend (audit 05)
  * shared/storage/keys.ts — STORAGE_KEYS centralised (theme, locale)
  * features/settings/store.ts — dead apiUrl ref + orphan i18n keys removed
  * api/schemas.py — DOCUMENT_STATUS_UPLOADED constant

Bloc D — Quality (audits 02/06/07/09/10/12)
  * domain/ports.py — DocumentConverter.supports_page_batching property
    (LSP fix, replaces isinstance(ServeConverter) check)
  * domain/ports.py — VectorStore.ping() (encapsulation, replaces
    _vector_store._client.info() reach-around)
  * api/analyses.py + api/ingestion.py — path params {job_id} → {analysis_id}
    aligned with the user-facing terminology (URLs unchanged)
  * api/documents.py — Path.read_bytes() + generate_preview() wrapped in
    asyncio.to_thread, unblocks the FastAPI event loop on /preview
  * infra/docling_tree.py — PEP 604 union for isinstance (Ruff UP038)
  * src/__tests__/integration/ — cross-feature integration test relocated
    out of features/history/ so feature folders stay self-contained
  * Tightened terminal `assert X is not None` checks (isinstance(.., datetime),
    exact value comparisons)

Validation
  * 446 backend pytest, 202 frontend vitest — all green
  * ruff + ruff format + ESLint + Prettier + vue-tsc clean
  * Re-audit verdict: 0 CRIT / 0 MAJ, score ~94/100, GO

Closes #200
2026-04-29 14:00:00 +02:00
Pier-Jean Malandrino
8694353d8b feat(reasoning): bidirectional PDF ↔ graph focus + DocumentView mode
Propagate Docling `self_ref` through PageElement so bboxes and graph nodes
share a stable identity. Add a Document/Graph mode switch to the reasoning
workspace; selecting a node highlights its bbox (numbered badge, focus ring,
optional dim of non-visited) and clicking a bbox re-centers the graph.
2026-04-29 14:00:00 +02:00
Pier-Jean Malandrino
d62b4ad32e refactor: centralize magic numbers for page dimensions, limits, and timeout (#168)
- Move DEFAULT_PAGE_WIDTH/HEIGHT to domain/value_objects.py and import in both converters
- Add opensearch_default_limit to Settings (configurable via OPENSEARCH_DEFAULT_LIMIT env var)
- Pass settings.conversion_timeout to ServeConverter, removing independent _DEFAULT_TIMEOUT
- Update OpenSearchStore to accept default_limit from Settings via constructor
2026-04-29 14:00:00 +02:00
Pier-Jean Malandrino
d8937ad916 fix: remove dead store and no-effect await flagged by CodeQL
- local_converter.py: remove redundant `_default_converter = None` in
  except block of `_ensure_default_converter` (variable was already None,
  re-raised immediately — dead store)
- test_analysis_service.py: replace bare `await task` with
  `await asyncio.gather(task)` to satisfy static analysis
2026-04-10 14:37:51 +02:00
Pier-Jean Malandrino
3d199cb783 fix(clean-code): English mode strings, SRP extraction, rename getter
- Replace French mode strings (configurer/verifier/preparer) with English
  equivalents (configure/verify/prepare) in StudioPage.vue and tests
- Extract _build_conversion_options, _run_conversion, _finalize_analysis
  from _run_analysis_inner to respect Single Responsibility Principle
- Rename _get_default_converter to _ensure_default_converter to reflect
  its lazy-init side effect

Closes #136, closes #137, closes #138
2026-04-10 13:45:13 +02:00
Pier-Jean Malandrino
fc866ce229 feat: batch large documents with page_range and progress reporting
Use Docling's native page_range parameter to split large PDFs into
sequential batches, preventing memory exhaustion and timeouts.
Progress is reported via existing polling mechanism.

Closes #56
2026-04-07 17:54:40 +02:00
Pier-Jean Malandrino
1657bce1f0 fix: make converter lock timeout configurable via LOCK_TIMEOUT env var (#61)
Replace hardcoded _LOCK_TIMEOUT=300 with settings.lock_timeout,
readable from LOCK_TIMEOUT environment variable.
2026-04-07 17:01:10 +02:00
Pier-Jean Malandrino
f89dc51661 fix: reset _default_converter on init failure (H5)
If the lazy-init of the default converter fails (e.g. model download
error), the singleton was left as None but subsequent calls would not
retry. Now the failed state is cleared so the next request retries.

Ref #57 (H5)
2026-04-07 15:32:38 +02:00
Pier-Jean Malandrino
6327b13614 fix: pass max_num_pages and max_file_size to Docling convert (C3)
Defense-in-depth: even if upload validation passes, Docling itself
now enforces page count and file size limits. Configurable via
MAX_PAGE_COUNT and MAX_FILE_SIZE env vars (0 = unlimited).

Ref #57 (C3)
2026-04-07 15:32:38 +02:00
Pier-Jean Malandrino
c0fb128718 fix: replace _converter_lock with timeout-based acquisition (C2)
A frozen conversion holding the lock indefinitely blocks all subsequent
jobs. Using lock.acquire(timeout=300) fails fast with a clear error
instead of waiting forever.

Ref #57 (C2)
2026-04-07 15:32:38 +02:00
Pier-Jean Malandrino
4c76101142 fix: add document_timeout to PdfPipelineOptions (C1)
Docling's native document_timeout is the only mechanism that can
interrupt processing inside a blocked thread (OCR, table extraction).
Without it, asyncio.wait_for cannot stop a frozen conversion.

Configurable via DOCUMENT_TIMEOUT env var (default: 120s).

Closes #57 (C1)
2026-04-07 15:32:38 +02:00
Pier-Jean Malandrino
503b7e8a40 Fix import ordering violations (ruff I001)
Reorder first-party imports in local_converter and local_chunker to
satisfy isort rules: domain imports before infra imports.
2026-04-03 14:05:38 +02:00
Pier-Jean Malandrino
ab6d42aecd Move bbox.py from domain/ to infra/ and unify coordinate logic
domain/ must be pure with no external dependencies. bbox.py imports
docling_core and belongs in infra/. Also refactor ServeConverter to
use the canonical to_topleft_list via BoundingBox instead of
duplicated manual coordinate conversion. Move docling-core to base
requirements since it is now needed in both modes.
2026-04-03 13:17:26 +02:00
Pier-Jean Malandrino
4b1ec364f4 Add LocalChunker adapter and DoclingDocument serialization
LocalChunker implements DocumentChunker port using docling-core chunkers.
LocalConverter now serializes DoclingDocument to JSON for re-chunking support.
2026-04-02 12:33:07 +02:00
Pier-Jean Malandrino
3743ed4ca8 Refactor backend to hexagonal architecture for converter extensibility
Extract domain value objects and ports from parsing.py, move Docling-specific
code to infra/local_converter.py, and convert analysis_service to a class
with injected DocumentConverter. This prepares the codebase for plugging in
alternative conversion backends (e.g. Docling Serve) via the Protocol pattern.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 10:34:07 +02:00