- Replace French mode strings (configurer/verifier/preparer) with English
equivalents (configure/verify/prepare) in StudioPage.vue and tests
- Extract _build_conversion_options, _run_conversion, _finalize_analysis
from _run_analysis_inner to respect Single Responsibility Principle
- Rename _get_default_converter to _ensure_default_converter to reflect
its lazy-init side effect
Closes#136, closes#137, closes#138
Frontend decoupling:
- Create shared/appConfig.ts as reactive bridge (locale, maxFileSizeMb,
maxPageCount) eliminating shared→features and feature→feature imports
- Give history feature its own Pinia store and API layer (was re-export
of analysis store)
- Give chunking feature its own Pinia store and API layer (was importing
from analysis)
- ChunkPanel receives analysis data via props instead of cross-feature
store import
- document/store reads maxFileSizeMb from shared config instead of
importing feature-flags store
- shared/i18n reads locale from shared config instead of importing
settings store
Backend:
- Add HealthResponse Pydantic schema for /api/health endpoint
Closes#140, closes#141, closes#142, closes#143
Add state-machine guard clauses to AnalysisJob transition methods
(mark_running, mark_completed, mark_failed, update_progress) to prevent
invalid status transitions. Make all domain value objects immutable with
frozen=True. Add 11 tests covering guard clause behavior.
Closes#132, closes#133
- Add DocumentRepository and AnalysisRepository protocols in domain/ports.py (#128)
- Refactor persistence repos from module functions to SqliteDocumentRepository
and SqliteAnalysisRepository classes
- Inject repos into AnalysisService and new DocumentService class via
constructor, removing direct imports of persistence and infra.settings (#129)
- Move _merge_results, _classify_error, _extract_html_body to domain/services.py (#130)
- Update main.py composition root to build and wire all dependencies
- Switch api/documents.py to Depends pattern matching api/analyses.py
- Update all tests to use injected mocks instead of module-level patches
Closes#128, closes#129, closes#130
Re-read the job from DB before mark_completed so that
progress_current/progress_total written during batched conversion
are not overwritten by the stale in-memory object.
Add regression unit test and e2e assertion on final progress values.
Set up a full E2E test suite (39 scenarios) using Karate against
the real API stack. Hybrid architecture: domain-based features +
cross-domain workflows, with data-driven testing and callable helpers.
Structure:
- e2e/pom.xml: Maven + karate-core 1.5
- 3 helpers (upload, analyze+poll, cleanup)
- 3 JSON schemas (health, document, analysis)
- 12 feature files across health, documents, analyses, workflows
- Tags: @smoke (2), @regression (35), @e2e (2)
- generate-test-data.py: fpdf2-based PDF generation (no binaries)
Also adds:
- RATE_LIMIT_RPM env var to make rate limiter configurable (0=disabled)
- CI job e2e with needs: [backend, frontend]
- e2e/ in .dockerignore
Closes#119
Replace hardcoded 5 MB upload limit with a configurable setting.
Backend exposes the value via /api/health, frontend reads it
dynamically for validation and UI messages.
Closes#48
pypdfium2 is imported in analysis_service for PDF page counting
(batch feature #56) but was missing from requirements.txt, causing
CI collection errors on all 4 test modules.
Use Docling's native page_range parameter to split large PDFs into
sequential batches, preventing memory exhaustion and timeouts.
Progress is reported via existing polling mechanism.
Closes#56
Use `import infra.local_converter as lc_mod` consistently instead of
mixing `import` and `from ... import` for the same module.
Addresses CodeQL review comment on PR #58.
Raw PyTorch/Docling stack traces are no longer shown to the user.
Common failures (missing compiler, OOM, lock contention, corrupted
document) are mapped to actionable messages. Unknown errors are
truncated to 200 chars.
Ref #57 (M1)
If the lazy-init of the default converter fails (e.g. model download
error), the singleton was left as None but subsequent calls would not
retry. Now the failed state is cleared so the next request retries.
Ref #57 (H5)
TableFormerMode.ACCURATE is very expensive on CPU (~3-5x slower).
The default can now be set to "fast" on resource-constrained
environments (HF Spaces) via the DEFAULT_TABLE_MODE env var.
User-specified table_mode in the request still takes precedence.
Ref #57 (H1)
Defense-in-depth: even if upload validation passes, Docling itself
now enforces page count and file size limits. Configurable via
MAX_PAGE_COUNT and MAX_FILE_SIZE env vars (0 = unlimited).
Ref #57 (C3)
A frozen conversion holding the lock indefinitely blocks all subsequent
jobs. Using lock.acquire(timeout=300) fails fast with a clear error
instead of waiting forever.
Ref #57 (C2)
Docling's native document_timeout is the only mechanism that can
interrupt processing inside a blocked thread (OCR, table extraction).
Without it, asyncio.wait_for cannot stop a frozen conversion.
Configurable via DOCUMENT_TIMEOUT env var (default: 120s).
Closes#57 (C1)
Prevents PyTorch/Docling pipeline crashes on HF Spaces CPU by:
- Reducing max file size from 50 MB to 5 MB
- Adding configurable MAX_PAGE_COUNT setting (env var, default unlimited)
- Increasing conversion timeout from 600s to 900s
- Adding frontend upload validation with explicit error messages
- Exposing maxPageCount via /api/health for dynamic UI hints
local_converter imports docling at module level, so any test that
touches it crashes when docling is not installed. Skip those tests
with importorskip / skipif so the CI (which only has docling-core)
passes cleanly.
The docling library (torch, transformers) is too heavy for lightweight
CI environments that only install docling-core. Use importorskip so
these tests are cleanly skipped instead of crashing collection.
Adapt test expectations to external changes: upload returns 200,
ValueError yields 400, and schemas now accept both snake_case and
camelCase via AliasChoices.
Cover PDF validation, file size limits, preview generation, page
counting, file deletion with path traversal protection, and the
not-found case — all previously untested code paths.
Lightweight sliding-window per-IP rate limiter (100 req/min default)
with no external dependency. Health endpoint is excluded. Returns 429
with Retry-After header when exceeded. Sufficient for single-process
SQLite deployments; document the Redis upgrade path for scale.
Domain model state transitions, repository CRUD operations, and service
methods now have descriptive docstrings to lower the contribution
barrier for an open-source project.
Read uploaded files in fixed-size chunks instead of a single file.read()
to reduce peak memory pressure. Also enforces the size limit during
streaming so oversized payloads are rejected before fully buffered.
Unbounded asyncio.create_task calls could exhaust CPU and memory on
modest hardware. Add a configurable semaphore (MAX_CONCURRENT_ANALYSES,
default 3) so excess jobs queue instead of running all at once.
All endpoint functions, lifespan context manager, and get_connection now
have explicit return types. Upload endpoint returns 201 Created instead
of 200 to follow REST conventions.
Replace broad `except Exception` with specific types (FileNotFoundError,
PermissionError, OSError) so errors are properly categorized in logs.
Users reporting bugs will get actionable messages instead of generic ones.
UPLOAD_DIR and DB_PATH were read directly from os.environ, bypassing
the Settings dataclass. This caused an inconsistency where overriding
Settings had no effect on these values. Now all modules import from
infra.settings.settings.
domain/ must be pure with no external dependencies. bbox.py imports
docling_core and belongs in infra/. Also refactor ServeConverter to
use the canonical to_topleft_list via BoundingBox instead of
duplicated manual coordinate conversion. Move docling-core to base
requirements since it is now needed in both modes.
Move /health to /api/health on backend and update the frontend
feature flag store to match. Without this, the combined Docker
image nginx proxy could not reach the endpoint and feature flags
(chunking/prepare mode) failed to load.
Install torch and torchvision from the CPU-only index before docling
to avoid pulling CUDA/nvidia/triton dependencies. Update documentation
with measured image sizes (270MB remote, 1.9GB local).
Both root and backend Dockerfiles now use a shared base stage with
two targets: remote (lightweight, ~300MB) and local (full Docling,
~2-3GB). docker-compose uses CONVERSION_MODE to select the target.
