Commit graph

48 commits

Author SHA1 Message Date
Pier-Jean Malandrino
ab6d42aecd Move bbox.py from domain/ to infra/ and unify coordinate logic
domain/ must be pure with no external dependencies. bbox.py imports
docling_core and belongs in infra/. Also refactor ServeConverter to
use the canonical to_topleft_list via BoundingBox instead of
duplicated manual coordinate conversion. Move docling-core to base
requirements since it is now needed in both modes.
2026-04-03 13:17:26 +02:00
Pier-Jean Malandrino
dbcd9dfa4c Fix naive datetime fallbacks in repository layer
Use UTC-aware datetimes consistently to prevent TypeError when
comparing aware and naive datetime objects.
2026-04-03 13:04:44 +02:00
Pier-Jean Malandrino
8856daf3c9 Force .pdf extension on uploaded files regardless of user input
The file content is already validated as PDF, so the user-supplied
extension should not be trusted or written to disk.
2026-04-03 13:03:21 +02:00
Pier-Jean Malandrino
2cbef859e7 Fix generate_page_images option silently dropped in remote mode
The form data builder was not forwarding generate_page_images to
Docling Serve, so page image generation was silently ignored.
2026-04-03 13:01:50 +02:00
Pier-Jean Malandrino
e1636e710d Fix ServeConverter not returning document_json for chunking
The remote converter never set document_json in ConversionResult,
silently preventing chunking from working in remote mode.
2026-04-03 13:00:43 +02:00
Pier-Jean Malandrino
abe59c3cb7 Fix health endpoint not reachable through nginx proxy
Move /health to /api/health on backend and update the frontend
feature flag store to match. Without this, the combined Docker
image nginx proxy could not reach the endpoint and feature flags
(chunking/prepare mode) failed to load.
2026-04-03 12:14:15 +02:00
Pier-Jean Malandrino
48c5e4ea6b Fix RapidOCR model download permission in local image
Grant appuser write access to rapidocr/models directory so OCR
models can be downloaded at runtime.
2026-04-03 10:44:20 +02:00
Pier-Jean Malandrino
48b7d5d3e8 Use CPU-only torch in local image to reduce size from 5.8GB to 1.9GB
Install torch and torchvision from the CPU-only index before docling
to avoid pulling CUDA/nvidia/triton dependencies. Update documentation
with measured image sizes (270MB remote, 1.9GB local).
2026-04-03 09:52:22 +02:00
Pier-Jean Malandrino
9ac0840607 Add multi-target Dockerfiles for remote and local builds
Both root and backend Dockerfiles now use a shared base stage with
two targets: remote (lightweight, ~300MB) and local (full Docling,
~2-3GB). docker-compose uses CONVERSION_MODE to select the target.
2026-04-03 09:24:41 +02:00
Pier-Jean Malandrino
8dd917b6ed Split requirements into base and local variants
Remote mode only needs FastAPI, httpx and lightweight deps.
Local mode adds docling and docling-core for in-process conversion.
2026-04-03 09:24:41 +02:00
Pier-Jean Malandrino
1ff8c5108f Add release process, dynamic versioning, and changelog catch-up
Replace hardcoded version strings with build-time injection:
- Frontend: Vite __APP_VERSION__ from env or package.json
- Backend: APP_VERSION env var exposed via /health endpoint
- Docker: build arg propagated through both stages
- CI: release workflow extracts version from git tag

Document branching strategy and release process in CONTRIBUTING.md.
Catch up CHANGELOG with v0.2.0 and Unreleased sections.
Sync package.json version to 0.3.0.
2026-04-02 16:08:37 +02:00
Pier-Jean Malandrino
82059ec0b1 Add tests for ChunkBbox domain, schema, and API serialization
Cover ChunkBbox construction and serialization, ChunkBboxResponse
camelCase output, rechunk endpoint bbox propagation, and frontend
store test data alignment with the new bboxes field.
2026-04-02 14:47:40 +02:00
Pier-Jean Malandrino
4af6b5b231 Add chunk-to-bbox hover highlighting in Prepare mode
Extract bounding boxes from chunk doc_items provenance in the chunker,
propagate through domain/service/API layers, and render highlighted
bboxes on canvas when hovering a chunk card. Reset highlights on
mode and page changes to prevent stale visual state.
2026-04-02 14:47:31 +02:00
Pier-Jean Malandrino
05de0cc774 Add chunking tests and update existing test assertions
16 new chunking tests (domain, schemas, API endpoints).
Update existing tests for chunkingOptions parameter and document_json mock.
2026-04-02 12:33:07 +02:00
Pier-Jean Malandrino
a9517d38eb Add chunking service orchestration, API endpoints, and wiring
AnalysisService gains rechunk() and inline chunking during conversion.
ChunkingOptionsRequest/ChunkResponse schemas, POST rechunk endpoint,
and conditional chunker injection in main.py (local engine only).
2026-04-02 12:33:07 +02:00
Pier-Jean Malandrino
4b1ec364f4 Add LocalChunker adapter and DoclingDocument serialization
LocalChunker implements DocumentChunker port using docling-core chunkers.
LocalConverter now serializes DoclingDocument to JSON for re-chunking support.
2026-04-02 12:33:07 +02:00
Pier-Jean Malandrino
f5b31f809f Add document_json and chunks_json persistence layer
Schema migration for new columns, repository read/write support,
and dedicated update_chunks method for re-chunking operations.
2026-04-02 12:33:07 +02:00
Pier-Jean Malandrino
78c27b087d Add chunking domain objects and DocumentChunker port
ChunkingOptions and ChunkResult value objects, document_json/chunks_json
fields on AnalysisJob, and DocumentChunker protocol for adapter injection.
2026-04-02 12:33:07 +02:00
Pier-Jean Malandrino
d42c97160d Format backend codebase with ruff 2026-04-02 12:33:07 +02:00
Pier-Jean Malandrino
2b991108f7 Rebase sync 2026-03-31 16:55:17 +02:00
Pier-Jean Malandrino
1b8cfe0a6b Fix Serve API contract: send to_formats as repeated form fields
Docling Serve expects array fields (to_formats) as repeated multipart
keys (to_formats=md&to_formats=html&to_formats=json), not a JSON
string. Changed _build_form_data to return list[tuple] so httpx sends
repeated keys correctly. Fixes 422 Unprocessable Entity on convert.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 16:52:36 +02:00
Pier-Jean Malandrino
001cf6807c Fix audit findings: security, robustness, and dead code
- Remove dead get_analysis_service() function in main.py
- Scope file deletion to UPLOAD_DIR to prevent path traversal
- Parse datetime strings back to datetime objects in repos
- Add 10-minute polling timeout in frontend analysis store
- Accept .pdf extension (not just MIME type) on drag-and-drop
- Guard localStorage access for private browsing compatibility

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 16:52:36 +02:00
Pier-Jean Malandrino
0af3b81b8f Fix zombie jobs and unprotected JSON parse
Bug #1: _on_task_done now receives job_id via functools.partial and
calls _mark_failed when the background task raises or is cancelled,
preventing jobs from being stuck in RUNNING state forever.

Bug #5: _parse_response wraps json.loads in try/except JSONDecodeError
so malformed json_content strings fall back gracefully instead of crashing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 16:52:36 +02:00
Pier-Jean Malandrino
a63af61b73 Fix lint ans test errors 2026-03-31 16:35:48 +02:00
Pier-Jean Malandrino
fe4e792885 Fix audit findings: remove domain→infra violation, align Serve API, fix DI
- Delete domain/parsing.py (broke hexagonal layering by importing infra)
- Migrate all tests to import directly from domain.value_objects and
  infra.local_converter
- Rewrite ServeConverter to match real Docling Serve v1 API contract:
  options sent as individual form fields (not JSON blob), response
  parsed from document.json_content (DoclingDocument), proper bbox
  coord_origin handling (TOPLEFT/BOTTOMLEFT)
- Transmit all conversion options including generate_picture_images
- Replace fragile lazy import circular dep with FastAPI Depends() +
  app.state for AnalysisService injection
- Add frontend file size validation (50MB) before upload

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 10:58:58 +02:00
Pier-Jean Malandrino
a3486a8501 Add ServeConverter adapter for remote Docling Serve integration
Implement the HTTP client adapter that delegates document conversion
to a remote Docling Serve instance via its /v1/convert/file endpoint.
Switchable via CONVERSION_ENGINE=remote env var. Includes health check,
API key auth, response parsing, and 30 new tests covering parsing,
type mapping, HTTP calls, and DI wiring.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 10:36:35 +02:00
Pier-Jean Malandrino
3743ed4ca8 Refactor backend to hexagonal architecture for converter extensibility
Extract domain value objects and ports from parsing.py, move Docling-specific
code to infra/local_converter.py, and convert analysis_service to a class
with injected DocumentConverter. This prepares the codebase for plugging in
alternative conversion backends (e.g. Docling Serve) via the Protocol pattern.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 10:34:07 +02:00
Pier-Jean Malandrino
40cba0f036 Fix type in emit 2026-03-24 08:38:55 +01:00
Pier-Jean Malandrino
d22f2e973e Add GIF in README 2026-03-23 10:55:54 +01:00
Pier-Jean Malandrino
1edd0668d1 Fix Ruff errror, update node and fix buttton breakline 2026-03-23 10:13:24 +01:00
pjmalandrino
b45bb87339 COnsolidation of bbox pipelin and add tests 2026-03-21 19:45:04 +01:00
pjmalandrino
577225d10e Add qualityt check and contributing doc 2026-03-21 15:34:54 +01:00
pjmalandrino
2db251182f Refacto and add cp paste btn 2026-03-21 15:03:10 +01:00
pjmalandrino
7bb4f8e298 Split the results of OCR 2026-03-20 21:45:26 +01:00
pjmalandrino
c3c0149532 Add docker push
Some checks failed
Release Docker Image / Build & push Docker image (push) Has been cancelled
2026-03-20 18:26:20 +01:00
pjmalandrino
d28dfd6d81 Add slide on side bar in Studio mode 2026-03-20 16:24:38 +01:00
pjmalandrino
20e79f4361 Change document tab place and add landing page 2026-03-20 13:57:05 +01:00
pjmalandrino
ad1f1a81d3 Mke some fixes and refacto 2026-03-20 12:36:26 +01:00
pjmalandrino
ea71f7247c Add tests 2026-03-20 09:42:14 +01:00
pjmalandrino
7821cd397b Allow History consultaiton on click, add documents management tab 2026-03-20 09:37:40 +01:00
pjmalandrino
d77fe2d989 Add configuration descriptions 2026-03-20 09:23:26 +01:00
pjmalandrino
e69ff90650 Connect options with model configuration 2026-03-20 08:59:20 +01:00
pjmalandrino
049ef443cc Add a stronger testing strategy 2026-03-18 14:46:37 +01:00
pjmalandrino
5fff141045 Radical architecture change, migration to a more lightweight 2026-03-17 16:06:27 +01:00
pjmalandrino
4c6aff3029 Work on full Docker integration 2026-03-17 13:33:36 +01:00
pjmalandrino
f1aa39b75f Fix display error on visual mode 2026-03-17 10:28:50 +01:00
pjmalandrino
9c2e42ace8 Include pretty visual mode 2026-03-17 09:19:46 +01:00
pjmalandrino
50ba1d43dd Make a nex project base, managing new Docling version 2026-03-17 08:43:00 +01:00