Commit graph

227 commits

Author SHA1 Message Date
Pier-Jean Malandrino
d62af058ed
Merge pull request #116 from scub-france/ci/auto-close-issues
ci: auto-close issues on merge into release/* branches
2026-04-08 10:21:57 +02:00
Pier-Jean Malandrino
1f85d5b9cb ci: auto-close issues on merge into release/* branches
GitHub only auto-closes issues when PRs target the default branch.
This workflow scans commit messages for Closes/Fixes patterns on
push to release/** and closes the referenced issues via gh CLI.

Closes #115
2026-04-08 10:18:49 +02:00
Pier-Jean Malandrino
ea78632070
Merge pull request #117 from scub-france/fix/ci-missing-pypdfium2
fix: add pypdfium2 to requirements.txt for CI
2026-04-08 10:17:46 +02:00
Pier-Jean Malandrino
459a720e77 fix: add pypdfium2 to requirements.txt for CI
pypdfium2 is imported in analysis_service for PDF page counting
(batch feature #56) but was missing from requirements.txt, causing
CI collection errors on all 4 test modules.
2026-04-08 10:14:39 +02:00
Pier-Jean Malandrino
a1ed48fb6f
Merge pull request #67 from scub-france/feature/batch-page-range
feat: batch large documents with page_range and progress reporting
2026-04-08 10:11:31 +02:00
Pier-Jean Malandrino
fc866ce229 feat: batch large documents with page_range and progress reporting
Use Docling's native page_range parameter to split large PDFs into
sequential batches, preventing memory exhaustion and timeouts.
Progress is reported via existing polling mechanism.

Closes #56
2026-04-07 17:54:40 +02:00
Pier-Jean Malandrino
18f3b91af4
Merge pull request #66 from scub-france/fix/audit-robustness
[FIX] Robustness audit — settings validation, timeout cascade, cancellation
2026-04-07 17:10:03 +02:00
Pier-Jean Malandrino
af10c200f0 fix: cancel running conversion when analysis job is deleted (#64)
Track background tasks per job ID. On delete, cancel the task to release
the converter lock instead of letting phantom jobs run to completion.
2026-04-07 17:07:55 +02:00
Pier-Jean Malandrino
302b24cc4c fix: route _on_task_done errors through _classify_error (#63)
Unhandled task exceptions now produce user-friendly messages instead of
raw Python tracebacks in the error_message field.
2026-04-07 17:06:16 +02:00
Pier-Jean Malandrino
8ea0cebc16 fix: validate timeout cascade ordering in Settings (#62)
Enforce document_timeout < lock_timeout < conversion_timeout at startup.
Prevents inverted timeout configurations that cause unpredictable behavior.
2026-04-07 17:04:09 +02:00
Pier-Jean Malandrino
1657bce1f0 fix: make converter lock timeout configurable via LOCK_TIMEOUT env var (#61)
Replace hardcoded _LOCK_TIMEOUT=300 with settings.lock_timeout,
readable from LOCK_TIMEOUT environment variable.
2026-04-07 17:01:10 +02:00
Pier-Jean Malandrino
2c254382c8 fix: add __post_init__ validation to Settings dataclass (#65)
Reject invalid configuration at startup: negative timeouts, zero
concurrency, bad table mode. Reports all errors at once.
2026-04-07 16:59:37 +02:00
Pier-Jean Malandrino
24cfd567f2
Merge pull request #58 from scub-france/fix/pipeline-robustness
Fix/pipeline robustness
2026-04-07 15:41:00 +02:00
Pier-Jean Malandrino
f04e5369ef style: use single import style for infra.local_converter in tests
Use `import infra.local_converter as lc_mod` consistently instead of
mixing `import` and `from ... import` for the same module.

Addresses CodeQL review comment on PR #58.
2026-04-07 15:39:02 +02:00
Pier-Jean Malandrino
6177452de1 test: add robustness tests for pipeline failure scenarios
25 tests covering all backend robustness fixes:
- TestClassifyError (9): user-friendly error message mapping
- TestDefaultTableMode (4): settings.default_table_mode injection
- TestDocumentTimeout (2): document_timeout wired into pipeline
- TestConverterLockTimeout (3): lock timeout + release guarantees
- TestConvertSyncLimits (4): max_num_pages/max_file_size forwarding
- TestGetDefaultConverterReset (3): converter singleton reset on failure

Ref #57
2026-04-07 15:32:38 +02:00
Pier-Jean Malandrino
c281b6c551 fix: classify pipeline errors into user-friendly messages (M1)
Raw PyTorch/Docling stack traces are no longer shown to the user.
Common failures (missing compiler, OOM, lock contention, corrupted
document) are mapped to actionable messages. Unknown errors are
truncated to 200 chars.

Ref #57 (M1)
2026-04-07 15:32:38 +02:00
Pier-Jean Malandrino
f89dc51661 fix: reset _default_converter on init failure (H5)
If the lazy-init of the default converter fails (e.g. model download
error), the singleton was left as None but subsequent calls would not
retry. Now the failed state is cleared so the next request retries.

Ref #57 (H5)
2026-04-07 15:32:38 +02:00
Pier-Jean Malandrino
4b1a15f49a fix: add retry tolerance to frontend polling (H3)
A single transient network error (502, timeout) no longer kills the
polling loop. The frontend now tolerates up to 3 consecutive errors
before abandoning. Successful fetches reset the counter.

Also aligns frontend polling timeout (15 min) with backend timeout.

Ref #57 (H3, M5)
2026-04-07 15:32:38 +02:00
Pier-Jean Malandrino
f58b563a13 fix: make default table_mode configurable via DEFAULT_TABLE_MODE (H1)
TableFormerMode.ACCURATE is very expensive on CPU (~3-5x slower).
The default can now be set to "fast" on resource-constrained
environments (HF Spaces) via the DEFAULT_TABLE_MODE env var.

User-specified table_mode in the request still takes precedence.

Ref #57 (H1)
2026-04-07 15:32:38 +02:00
Pier-Jean Malandrino
6327b13614 fix: pass max_num_pages and max_file_size to Docling convert (C3)
Defense-in-depth: even if upload validation passes, Docling itself
now enforces page count and file size limits. Configurable via
MAX_PAGE_COUNT and MAX_FILE_SIZE env vars (0 = unlimited).

Ref #57 (C3)
2026-04-07 15:32:38 +02:00
Pier-Jean Malandrino
c0fb128718 fix: replace _converter_lock with timeout-based acquisition (C2)
A frozen conversion holding the lock indefinitely blocks all subsequent
jobs. Using lock.acquire(timeout=300) fails fast with a clear error
instead of waiting forever.

Ref #57 (C2)
2026-04-07 15:32:38 +02:00
Pier-Jean Malandrino
4c76101142 fix: add document_timeout to PdfPipelineOptions (C1)
Docling's native document_timeout is the only mechanism that can
interrupt processing inside a blocked thread (OCR, table extraction).
Without it, asyncio.wait_for cannot stop a frozen conversion.

Configurable via DOCUMENT_TIMEOUT env var (default: 120s).

Closes #57 (C1)
2026-04-07 15:32:38 +02:00
Pier-Jean Malandrino
5b6f138164
Update README.md 2026-04-07 12:37:17 +02:00
Pier-Jean Malandrino
05ed49b893 Add issue templates, GitHub stars badge, fix health endpoint type hint 2026-04-07 11:57:29 +02:00
Pier-Jean Malandrino
09145206d2 fix: limit upload to 5 MB / 20 pages max and increase conversion timeout
Prevents PyTorch/Docling pipeline crashes on HF Spaces CPU by:
- Reducing max file size from 50 MB to 5 MB
- Adding configurable MAX_PAGE_COUNT setting (env var, default unlimited)
- Increasing conversion timeout from 600s to 900s
- Adding frontend upload validation with explicit error messages
- Exposing maxPageCount via /api/health for dynamic UI hints
2026-04-07 10:56:16 +02:00
Pier-Jean Malandrino
11eaeb4cd6
Merge pull request #29 from scub-france/release/0.3.0
Some checks failed
Release Docker Images / Build & push — local (push) Has been cancelled
Release Docker Images / Build & push — remote (push) Has been cancelled
Release/0.3.0
2026-04-07 09:00:02 +02:00
Pier-Jean Malandrino
2919687202
Merge pull request #45 from scub-france/feature/github-tag
Feature/GitHub tag
2026-04-07 08:58:28 +02:00
Pier-Jean Malandrino
af4400b1f7 Add Github badge 2026-04-07 08:57:39 +02:00
Pier-Jean Malandrino
098ad30596 Freeze CHANGELOG for 0.3.0 release 2026-04-07 08:53:04 +02:00
Pier-Jean Malandrino
e1ccc7a6e0 Add document size limit (50 MB) to disclaimer banner 2026-04-07 08:35:39 +02:00
Pier-Jean Malandrino
4d84c4615b
Merge pull request #43 from scub-france/feature/update-doc
Feature/update doc
2026-04-05 10:53:58 +02:00
Pier-Jean Malandrino
09c5e5e89f Replace architecture diagram with new global/docker schemas 2026-04-05 10:53:21 +02:00
Pier-Jean Malandrino
36be9027c1 Document infra layer, feature flags, rate limiting, analysis lifecycle, local vs remote modes 2026-04-05 10:45:25 +02:00
Pier-Jean Malandrino
3bbff10d1a Add chunking options and missing env vars to getting-started docs 2026-04-05 10:45:21 +02:00
Pier-Jean Malandrino
ff7010bd6b Add missing features to docs index (chunking, feature flags, rate limiting, deployment modes) 2026-04-05 10:45:17 +02:00
Pier-Jean Malandrino
2fdb41e79e Revert "Update frontend test count in CLAUDE.md (129 tests)"
This reverts commit aa73c209bc.
2026-04-05 10:34:14 +02:00
Pier-Jean Malandrino
1b359b8faf Fix remaining stale test count in README backend structure (99 → 199) 2026-04-05 10:27:58 +02:00
Pier-Jean Malandrino
18316f513e Fix Docker image tag in docs quick start (latest → latest-local) 2026-04-05 10:25:57 +02:00
Pier-Jean Malandrino
526e3571f1 Add missing env vars to .env.example (DEPLOYMENT_MODE, MAX_CONCURRENT_ANALYSES, APP_VERSION) 2026-04-05 10:25:48 +02:00
Pier-Jean Malandrino
aa73c209bc Update frontend test count in CLAUDE.md (129 tests) 2026-04-05 10:25:38 +02:00
Pier-Jean Malandrino
2ca0d0a984 Update test counts in README and CONTRIBUTING (199 backend, 129 frontend) 2026-04-05 10:25:26 +02:00
Pier-Jean Malandrino
ee57b04f84 Add disclaimer banner for shared deployments (HuggingFace)
Feature-flagged via DEPLOYMENT_MODE env var and /api/health endpoint.
Disabled by default (self-hosted), enabled when deploymentMode=huggingface.
2026-04-04 21:06:38 +02:00
Pier-Jean Malandrino
76e8c55b7c Fix BboxOverlay inline handler breaking production build
Vue template compiler in production mode rejects multi-statement
inline handlers without semicolons. Extract to a named function.
2026-04-03 16:17:23 +02:00
Pier-Jean Malandrino
226ae5994e Format all frontend files with Prettier
Batch reformat to eliminate 23 files with inconsistent formatting
that were accumulating across feature branches.
2026-04-03 15:50:32 +02:00
Pier-Jean Malandrino
1c82a80389 Guard all docling-dependent tests for lightweight CI
local_converter imports docling at module level, so any test that
touches it crashes when docling is not installed. Skip those tests
with importorskip / skipif so the CI (which only has docling-core)
passes cleanly.
2026-04-03 15:46:29 +02:00
Pier-Jean Malandrino
41863152cb
Merge pull request #42 from scub-france/feature/quality-improvements
Skip pipeline options tests when docling is not installed
2026-04-03 15:44:12 +02:00
Pier-Jean Malandrino
d8f87c34bc Skip pipeline options tests when docling is not installed
The docling library (torch, transformers) is too heavy for lightweight
CI environments that only install docling-core. Use importorskip so
these tests are cleanly skipped instead of crashing collection.
2026-04-03 15:42:14 +02:00
Pier-Jean Malandrino
4e6fa0908d
Merge pull request #41 from scub-france/feature/quality-improvements
Feature/quality improvements
2026-04-03 15:22:48 +02:00
Pier-Jean Malandrino
fefd406dc8 Align tests with schema refactoring (AliasChoices, status codes)
Adapt test expectations to external changes: upload returns 200,
ValueError yields 400, and schemas now accept both snake_case and
camelCase via AliasChoices.
2026-04-03 14:21:30 +02:00
Pier-Jean Malandrino
503b7e8a40 Fix import ordering violations (ruff I001)
Reorder first-party imports in local_converter and local_chunker to
satisfy isort rules: domain imports before infra imports.
2026-04-03 14:05:38 +02:00