Commit graph

19 commits

Author SHA1 Message Date
Pier-Jean Malandrino
27e3323bff feat(#257): surface gating via STUDIO_MODE + RAG_PIPELINE master flags
Some checks failed
Auto-close issues on release branch merge / Close referenced issues (push) Has been cancelled
Introduces two master feature flags that select which UI surface is
exposed, replacing the previous "delete legacy pages" approach with a
softer isolation:

- STUDIO_MODE_ENABLED  (default false) — legacy OCR-debug surface
- RAG_PIPELINE_ENABLED (default true)  — new doc-centric ingestion + viz

At least one master must be enabled (validated server-side at startup).
Sub-flags (inspect / linked / ask) are effective only when the RAG
pipeline master is on.

CHUNKS_MODE_ENABLED renamed to LINKED_MODE_ENABLED in anticipation of
T3 (Linked view replaces the Chunks tab). The DocMode union value
'chunks' is preserved for now and will be renamed in T3 alongside the
route segment, to keep this PR scoped.

Router-level guard added: requests to a route whose surface is disabled
are redirected to the other surface's landing page (or /home as a
defensive fallback). Logic extracted into a pure resolveSurface helper
with full test coverage.

i18n strings that pointed users to "Studio" rewritten to be surface-
agnostic ("from the library" / "depuis la bibliothèque") since Studio
is hidden by default in 0.6.1.

Backend:
- infra/settings.py: add studio_mode_enabled + rag_pipeline_enabled;
  rename chunks_mode_enabled → linked_mode_enabled; add at-least-one
  master validation in __post_init__
- api/schemas.py: HealthResponse exposes both master flags + renamed
  sub-flag
- main.py: health endpoint wires the new fields
- tests: surface-flag + renamed sub-flag assertions

Frontend:
- features/feature-flags/store: add studioMode + ragPipeline registry
  entries; rename chunksMode → linkedMode; sub-flags now require
  ragPipeline enabled; modeFlags() maps linkedModeEnabled → key 'chunks'
  (transitional)
- shared/routing/resolveSurface: pure helper + tests
- app/router: beforeEach guard consumes resolveSurface
- shared/i18n: Studio-pointing strings rewritten (en + fr) + test sync
- features/reasoning: stale "from StudioPage" comment generalized
2026-05-11 16:18:34 +02:00
Pier-Jean Malandrino
fa2c7d7525 fix(analyses): filter list endpoint by documentId — prevent cross-doc bbox leak
GET /api/analyses?documentId=X retournait toutes les analyses (le query
param était ignoré). Le frontend prenait alors la première COMPLETED de
la réponse, qui pouvait être celle d'un autre doc → bboxes d'un autre
doc projetées sur l'image en cours.

Backend
- analysis_repo: nouvelle méthode find_by_document(document_id, limit, offset)
- analysis_service: expose find_by_document
- api/analyses: GET /api/analyses accepte ?documentId=... (alias Pydantic
  pour respecter la règle ruff N803). Si présent, filtre via la nouvelle
  méthode; sinon comportement inchangé.
- test: test_list_analyses_filtered_by_document vérifie le routing vers
  find_by_document quand le query param est fourni.

Frontend (defensive)
- DocInspectTab / DocAskTab: filtre client-side
  analyses.find(a => a.documentId === requestedId && a.status === 'COMPLETED')
  pour rester safe même si un backend antérieur ignore le param.
2026-05-11 15:00:57 +02:00
Pier-Jean Malandrino
7b6d64c434 feat(#210): feature-flag mode gating + deep-link redirect
Backend
- HealthResponse exposes inspectModeEnabled / chunksModeEnabled /
  askModeEnabled (additive; defaults true). main.py /api/health
  populates them from settings.
- infra/settings.py: three new env-var-driven booleans (defaults true)
  parsed in from_env() like the existing reasoning_enabled flag.
- tests/test_api_endpoints.py: extra assertion that /api/health
  surfaces the three new fields with their defaults.

Frontend — flag store
- features/feature-flags/store.ts: FeatureFlag union extended with
  inspectMode / chunksMode / askMode. New entries in featureRegistry
  are gated on context fields populated from health. Missing fields
  fall back to true so a frontend pointed at an older backend keeps
  every mode visible.
- store gains a modeFlags() helper returning Record<DocMode, boolean>
  so the routing guard does not need to know the FeatureFlag union.

Frontend — routing
- shared/routing/resolveMode.ts: pure resolver. If the requested mode
  is enabled, return it; else first enabled in priority ask > chunks
  > inspect; else null.
- app/router/index.ts: beforeEach guard scoped to ROUTES.DOC_WORKSPACE.
  Disabled mode → rewrite ?mode= to the first enabled one. All three
  off → redirect to /docs?reason=no-mode-enabled.

Frontend — flash
- pages/DocsLibraryPage.vue: shows a banner when ?reason=no-mode-enabled
  is set. #211 will move this into the proper library page banner.
- i18n flags.allModesDisabled added in fr + en.

Tests
- shared/routing/resolveMode.test.ts (6 cases): every (requested,
  enabled) combination including all-disabled, priority order,
  missing requested.
- features/feature-flags/store.test.ts: three new cases covering the
  new fields in /api/health, fall-back-to-true on missing fields, and
  modeFlags() shape.

Refs #210
2026-05-05 09:38:39 +02:00
Pier-Jean Malandrino
efc27932dd refactor(audit): remediate 0.5.0 audit findings — clean architecture, security, DRY, SOLID, perf
Closes the 12 MAJ raised by the release/0.5.0 audit pipeline (cf.
docs/audit/reports/release-0.5.0/summary.md → summary-reaudit.md).

Volet 1 — Reasoning architecture (audits 01/02/06/07 strengthening)
  * Domain ports: LLMProvider, ReasoningRunner, ReasoningParseError
  * Domain DTOs: LLMProviderType, ReasoningResult, ReasoningIteration
  * infra/llm/ollama_provider.py — OllamaProvider with health_check
  * infra/docling_agent_reasoning.py — runner adapter, encapsulates the
    private _rag_loop call (tracked at docling-project/docling-agent#26),
    commits OLLAMA_HOST once at boot (eliminates the per-request env race),
    translates upstream IndexError into ReasoningParseError
  * api/reasoning.py — zero coupling to docling-agent / mellea / docling-core,
    consumes app.state.reasoning_runner via the port
  * main.py — DI wires OllamaProvider + DoclingAgentReasoningRunner at boot
    when REASONING_ENABLED=true and deps are importable
  * Rename RAG_* env vars → REASONING_*, endpoint /rag → /reasoning,
    type RAGResult → ReasoningResult, frontend feature flag wiring,
    i18n strings, tests, docs (BREAKING — pre-1.0 surface, no external
    consumers in production)
  * 17 new tests: adapter unit tests with sys.modules stubs, OllamaProvider
    httpx tests, R3 concurrent-host isolation, R6 multi-iteration trace
    serialization, R13 Protocol conformance via isinstance
  * E2E Karate scenario: nav-reasoning hidden when REASONING_ENABLED=false
  * README — Live Reasoning section (env vars, archi, link to issue #26)

Bloc B — Security (audit 08, dev-only context)
  * docker-compose.yml — DEV DEFAULTS header, OpenSearch DISABLE_SECURITY_PLUGIN
    flagged as dev-only with link to OpenSearch security docs
  * main.py — boot warning if NEO4J_URI is set with the default 'changeme'
    password, so prod operators can't silently inherit it

Bloc C — DRY frontend (audit 05)
  * shared/storage/keys.ts — STORAGE_KEYS centralised (theme, locale)
  * features/settings/store.ts — dead apiUrl ref + orphan i18n keys removed
  * api/schemas.py — DOCUMENT_STATUS_UPLOADED constant

Bloc D — Quality (audits 02/06/07/09/10/12)
  * domain/ports.py — DocumentConverter.supports_page_batching property
    (LSP fix, replaces isinstance(ServeConverter) check)
  * domain/ports.py — VectorStore.ping() (encapsulation, replaces
    _vector_store._client.info() reach-around)
  * api/analyses.py + api/ingestion.py — path params {job_id} → {analysis_id}
    aligned with the user-facing terminology (URLs unchanged)
  * api/documents.py — Path.read_bytes() + generate_preview() wrapped in
    asyncio.to_thread, unblocks the FastAPI event loop on /preview
  * infra/docling_tree.py — PEP 604 union for isinstance (Ruff UP038)
  * src/__tests__/integration/ — cross-feature integration test relocated
    out of features/history/ so feature folders stay self-contained
  * Tightened terminal `assert X is not None` checks (isinstance(.., datetime),
    exact value comparisons)

Validation
  * 446 backend pytest, 202 frontend vitest — all green
  * ruff + ruff format + ESLint + Prettier + vue-tsc clean
  * Re-audit verdict: 0 CRIT / 0 MAJ, score ~94/100, GO

Closes #200
2026-04-29 14:00:00 +02:00
Pier-Jean Malandrino
ba54427445 feat(#180): feature-flag ingestion pipeline and add brainless one-liner Quick Start
- Conditionally mount ingestion router only when OpenSearch + embedding are configured
- Add `ingestionAvailable` field to /api/health response
- Add `ingestion` feature flag to frontend (hides Search nav, Ingest button,
  OpenSearch badge, indexed badges/filters when disabled)
- Skip ingestion polling when flag is off
- Make OpenSearch + embedding optional in docker-compose via profiles
- Add docker-compose.ingestion.yml override for full-stack ingestion
- Set BATCH_PAGE_SIZE=5 default in Docker local image
- Lead Quick Start with one-liner docker run command
- Document ingestion as opt-in with dedicated section
- Add BATCH_PAGE_SIZE, MAX_FILE_SIZE_MB, MAX_PAGE_COUNT, RATE_LIMIT_RPM to config tables
- Update test counts (380 backend, 159 frontend)
- Date CHANGELOG 0.4.0, bump frontend version to 0.4.0
- Sync CONTRIBUTING.md with E2E Karate test sections

Closes #180
2026-04-13 11:18:56 +02:00
Pier-Jean Malandrino
9907b7f040 fix(arch): inject repositories and extract domain logic
- Add DocumentRepository and AnalysisRepository protocols in domain/ports.py (#128)
- Refactor persistence repos from module functions to SqliteDocumentRepository
  and SqliteAnalysisRepository classes
- Inject repos into AnalysisService and new DocumentService class via
  constructor, removing direct imports of persistence and infra.settings (#129)
- Move _merge_results, _classify_error, _extract_html_body to domain/services.py (#130)
- Update main.py composition root to build and wire all dependencies
- Switch api/documents.py to Depends pattern matching api/analyses.py
- Update all tests to use injected mocks instead of module-level patches

Closes #128, closes #129, closes #130
2026-04-10 11:44:33 +02:00
Pier-Jean Malandrino
704ba550d4 feat: make document max size configurable via MAX_FILE_SIZE_MB env var
Replace hardcoded 5 MB upload limit with a configurable setting.
Backend exposes the value via /api/health, frontend reads it
dynamically for validation and UI messages.

Closes #48
2026-04-08 10:27:38 +02:00
Pier-Jean Malandrino
09145206d2 fix: limit upload to 5 MB / 20 pages max and increase conversion timeout
Prevents PyTorch/Docling pipeline crashes on HF Spaces CPU by:
- Reducing max file size from 50 MB to 5 MB
- Adding configurable MAX_PAGE_COUNT setting (env var, default unlimited)
- Increasing conversion timeout from 600s to 900s
- Adding frontend upload validation with explicit error messages
- Exposing maxPageCount via /api/health for dynamic UI hints
2026-04-07 10:56:16 +02:00
Pier-Jean Malandrino
fefd406dc8 Align tests with schema refactoring (AliasChoices, status codes)
Adapt test expectations to external changes: upload returns 200,
ValueError yields 400, and schemas now accept both snake_case and
camelCase via AliasChoices.
2026-04-03 14:21:30 +02:00
Pier-Jean Malandrino
8ee470f1aa Add database ping to health check endpoint
Health endpoint now verifies SQLite connectivity and reports a
"degraded" status when the database is unreachable, instead of
blindly returning "ok".
2026-04-03 13:53:01 +02:00
Pier-Jean Malandrino
7bd7d0f793 Validate page bounds on preview endpoint
Reject requests where page exceeds the document page count with a clear
400 error, instead of letting pdf2image fail with an opaque exception.
2026-04-03 13:50:32 +02:00
Pier-Jean Malandrino
661568f2cb Add return type annotations and use 201 for upload
All endpoint functions, lifespan context manager, and get_connection now
have explicit return types. Upload endpoint returns 201 Created instead
of 200 to follow REST conventions.
2026-04-03 13:50:07 +02:00
Pier-Jean Malandrino
abe59c3cb7 Fix health endpoint not reachable through nginx proxy
Move /health to /api/health on backend and update the frontend
feature flag store to match. Without this, the combined Docker
image nginx proxy could not reach the endpoint and feature flags
(chunking/prepare mode) failed to load.
2026-04-03 12:14:15 +02:00
Pier-Jean Malandrino
05de0cc774 Add chunking tests and update existing test assertions
16 new chunking tests (domain, schemas, API endpoints).
Update existing tests for chunkingOptions parameter and document_json mock.
2026-04-02 12:33:07 +02:00
Pier-Jean Malandrino
fe4e792885 Fix audit findings: remove domain→infra violation, align Serve API, fix DI
- Delete domain/parsing.py (broke hexagonal layering by importing infra)
- Migrate all tests to import directly from domain.value_objects and
  infra.local_converter
- Rewrite ServeConverter to match real Docling Serve v1 API contract:
  options sent as individual form fields (not JSON blob), response
  parsed from document.json_content (DoclingDocument), proper bbox
  coord_origin handling (TOPLEFT/BOTTOMLEFT)
- Transmit all conversion options including generate_picture_images
- Replace fragile lazy import circular dep with FastAPI Depends() +
  app.state for AnalysisService injection
- Add frontend file size validation (50MB) before upload

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 10:58:58 +02:00
Pier-Jean Malandrino
3743ed4ca8 Refactor backend to hexagonal architecture for converter extensibility
Extract domain value objects and ports from parsing.py, move Docling-specific
code to infra/local_converter.py, and convert analysis_service to a class
with injected DocumentConverter. This prepares the codebase for plugging in
alternative conversion backends (e.g. Docling Serve) via the Protocol pattern.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 10:34:07 +02:00
pjmalandrino
577225d10e Add qualityt check and contributing doc 2026-03-21 15:34:54 +01:00
pjmalandrino
e69ff90650 Connect options with model configuration 2026-03-20 08:59:20 +01:00
pjmalandrino
049ef443cc Add a stronger testing strategy 2026-03-18 14:46:37 +01:00