- Conditionally mount ingestion router only when OpenSearch + embedding are configured
- Add `ingestionAvailable` field to /api/health response
- Add `ingestion` feature flag to frontend (hides Search nav, Ingest button,
OpenSearch badge, indexed badges/filters when disabled)
- Skip ingestion polling when flag is off
- Make OpenSearch + embedding optional in docker-compose via profiles
- Add docker-compose.ingestion.yml override for full-stack ingestion
- Set BATCH_PAGE_SIZE=5 default in Docker local image
- Lead Quick Start with one-liner docker run command
- Document ingestion as opt-in with dedicated section
- Add BATCH_PAGE_SIZE, MAX_FILE_SIZE_MB, MAX_PAGE_COUNT, RATE_LIMIT_RPM to config tables
- Update test counts (380 backend, 159 frontend)
- Date CHANGELOG 0.4.0, bump frontend version to 0.4.0
- Sync CONTRIBUTING.md with E2E Karate test sections
Closes#180
Backend: GET /api/ingestion/search?q=…&doc_id=… endpoint with
SearchResponse schema. Frontend: search bar in Documents page, results
with filename, page, chunk index, relevance score. 3 new API tests.
- local_converter.py: remove redundant `_default_converter = None` in
except block of `_ensure_default_converter` (variable was already None,
re-raised immediately — dead store)
- test_analysis_service.py: replace bare `await task` with
`await asyncio.gather(task)` to satisfy static analysis
- Replace French mode strings (configurer/verifier/preparer) with English
equivalents (configure/verify/prepare) in StudioPage.vue and tests
- Extract _build_conversion_options, _run_conversion, _finalize_analysis
from _run_analysis_inner to respect Single Responsibility Principle
- Rename _get_default_converter to _ensure_default_converter to reflect
its lazy-init side effect
Closes#136, closes#137, closes#138
Add state-machine guard clauses to AnalysisJob transition methods
(mark_running, mark_completed, mark_failed, update_progress) to prevent
invalid status transitions. Make all domain value objects immutable with
frozen=True. Add 11 tests covering guard clause behavior.
Closes#132, closes#133
- Add DocumentRepository and AnalysisRepository protocols in domain/ports.py (#128)
- Refactor persistence repos from module functions to SqliteDocumentRepository
and SqliteAnalysisRepository classes
- Inject repos into AnalysisService and new DocumentService class via
constructor, removing direct imports of persistence and infra.settings (#129)
- Move _merge_results, _classify_error, _extract_html_body to domain/services.py (#130)
- Update main.py composition root to build and wire all dependencies
- Switch api/documents.py to Depends pattern matching api/analyses.py
- Update all tests to use injected mocks instead of module-level patches
Closes#128, closes#129, closes#130
Re-read the job from DB before mark_completed so that
progress_current/progress_total written during batched conversion
are not overwritten by the stale in-memory object.
Add regression unit test and e2e assertion on final progress values.
Replace hardcoded 5 MB upload limit with a configurable setting.
Backend exposes the value via /api/health, frontend reads it
dynamically for validation and UI messages.
Closes#48
Use Docling's native page_range parameter to split large PDFs into
sequential batches, preventing memory exhaustion and timeouts.
Progress is reported via existing polling mechanism.
Closes#56
Use `import infra.local_converter as lc_mod` consistently instead of
mixing `import` and `from ... import` for the same module.
Addresses CodeQL review comment on PR #58.
Docling's native document_timeout is the only mechanism that can
interrupt processing inside a blocked thread (OCR, table extraction).
Without it, asyncio.wait_for cannot stop a frozen conversion.
Configurable via DOCUMENT_TIMEOUT env var (default: 120s).
Closes#57 (C1)
Prevents PyTorch/Docling pipeline crashes on HF Spaces CPU by:
- Reducing max file size from 50 MB to 5 MB
- Adding configurable MAX_PAGE_COUNT setting (env var, default unlimited)
- Increasing conversion timeout from 600s to 900s
- Adding frontend upload validation with explicit error messages
- Exposing maxPageCount via /api/health for dynamic UI hints
local_converter imports docling at module level, so any test that
touches it crashes when docling is not installed. Skip those tests
with importorskip / skipif so the CI (which only has docling-core)
passes cleanly.
The docling library (torch, transformers) is too heavy for lightweight
CI environments that only install docling-core. Use importorskip so
these tests are cleanly skipped instead of crashing collection.
Adapt test expectations to external changes: upload returns 200,
ValueError yields 400, and schemas now accept both snake_case and
camelCase via AliasChoices.
Cover PDF validation, file size limits, preview generation, page
counting, file deletion with path traversal protection, and the
not-found case — all previously untested code paths.
Lightweight sliding-window per-IP rate limiter (100 req/min default)
with no external dependency. Health endpoint is excluded. Returns 429
with Retry-After header when exceeded. Sufficient for single-process
SQLite deployments; document the Redis upgrade path for scale.
Unbounded asyncio.create_task calls could exhaust CPU and memory on
modest hardware. Add a configurable semaphore (MAX_CONCURRENT_ANALYSES,
default 3) so excess jobs queue instead of running all at once.
All endpoint functions, lifespan context manager, and get_connection now
have explicit return types. Upload endpoint returns 201 Created instead
of 200 to follow REST conventions.
UPLOAD_DIR and DB_PATH were read directly from os.environ, bypassing
the Settings dataclass. This caused an inconsistency where overriding
Settings had no effect on these values. Now all modules import from
infra.settings.settings.
domain/ must be pure with no external dependencies. bbox.py imports
docling_core and belongs in infra/. Also refactor ServeConverter to
use the canonical to_topleft_list via BoundingBox instead of
duplicated manual coordinate conversion. Move docling-core to base
requirements since it is now needed in both modes.
Move /health to /api/health on backend and update the frontend
feature flag store to match. Without this, the combined Docker
image nginx proxy could not reach the endpoint and feature flags
(chunking/prepare mode) failed to load.
Cover ChunkBbox construction and serialization, ChunkBboxResponse
camelCase output, rechunk endpoint bbox propagation, and frontend
store test data alignment with the new bboxes field.
Docling Serve expects array fields (to_formats) as repeated multipart
keys (to_formats=md&to_formats=html&to_formats=json), not a JSON
string. Changed _build_form_data to return list[tuple] so httpx sends
repeated keys correctly. Fixes 422 Unprocessable Entity on convert.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Bug #1: _on_task_done now receives job_id via functools.partial and
calls _mark_failed when the background task raises or is cancelled,
preventing jobs from being stuck in RUNNING state forever.
Bug #5: _parse_response wraps json.loads in try/except JSONDecodeError
so malformed json_content strings fall back gracefully instead of crashing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Delete domain/parsing.py (broke hexagonal layering by importing infra)
- Migrate all tests to import directly from domain.value_objects and
infra.local_converter
- Rewrite ServeConverter to match real Docling Serve v1 API contract:
options sent as individual form fields (not JSON blob), response
parsed from document.json_content (DoclingDocument), proper bbox
coord_origin handling (TOPLEFT/BOTTOMLEFT)
- Transmit all conversion options including generate_picture_images
- Replace fragile lazy import circular dep with FastAPI Depends() +
app.state for AnalysisService injection
- Add frontend file size validation (50MB) before upload
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
