The `doc_items` field was added to `ChunkResult` earlier in the
feature branch (used by ChunkWriter for DERIVED_FROM edges), but the
test fixture was never updated. CI caught it now that the PR is open.
Fixes: tests/test_chunking.py::TestChunkResult::test_serializable
ChunkWriter mirrors chunks into Neo4j after OpenSearch indexing, creating
HAS_CHUNK edges and DERIVED_FROM back-references to the source Elements
(via doc_items propagated from the local chunker).
Graph API: GET /api/documents/{id}/graph returns a cytoscape-shaped
payload with nodes + edges for Document / Element / Page / Chunk.
Hard cap at 200 pages returns HTTP 413 per design §8.4.
Frontend: new Graph tab in Studio results, rendered with Cytoscape.js +
dagre layout (lazy-loaded, ~175 KB gz). Legend, node styling per element
label, directional edges styled per edge type.
README gains a Neo4j section with the schema, three demo Cypher
queries, and env vars. Backend tests skip cleanly when the neo4j python
package is not installed locally.
Refs #186
Serialize a DoclingDocument to a Neo4j graph: Document + Page + Element
nodes with dynamic specific labels (SectionHeader, Paragraph, Table,
Figure, …), plus HAS_ROOT / PARENT_OF / NEXT / ON_PAGE edges. Replace-on-
write for idempotent re-ingestion.
The reader returns the verbatim document_json stored on the Document
node — reconstruction from graph nodes is deferred to v0.6.
Wired into AnalysisService._finalize_analysis: runs after conversion,
degrades gracefully by default, fails fast when neo4j_required is set.
Refs #186
Add Neo4j as an optional graph-native storage layer (ingestion profile).
Introduces infra/neo4j with a singleton async driver wrapper and an
idempotent bootstrap of constraints + indexes, wired into the FastAPI
lifespan. Integration tests skip when no live Neo4j is reachable.
Refs #186
Hybrid approach: reuse LocalChunker to chunk the DoclingDocument JSON
returned by Serve, so chunking works identically in both local and
remote modes without calling Serve's chunk endpoint.
Backend:
- _build_chunker() always returns LocalChunker (remove engine guard)
- Use docling-core[chunking] extra for required dependencies
- Skip client-side batching in remote mode (Serve manages its own
resources, and batching discards document_json needed for chunking)
- Fix Serve form fields: remove generate_page_images (not a Serve
field), use repeated form keys for to_formats and page_range
- Log Serve error response body on 4xx/5xx for diagnosis
- Fix FastAPI 204 DELETE routes missing response_model=None
Frontend:
- Update chunking feature flag to enable Prepare UI in remote mode
Closes#51
- Move DEFAULT_PAGE_WIDTH/HEIGHT to domain/value_objects.py and import in both converters
- Add opensearch_default_limit to Settings (configurable via OPENSEARCH_DEFAULT_LIMIT env var)
- Pass settings.conversion_timeout to ServeConverter, removing independent _DEFAULT_TIMEOUT
- Update OpenSearchStore to accept default_limit from Settings via constructor
- Conditionally mount ingestion router only when OpenSearch + embedding are configured
- Add `ingestionAvailable` field to /api/health response
- Add `ingestion` feature flag to frontend (hides Search nav, Ingest button,
OpenSearch badge, indexed badges/filters when disabled)
- Skip ingestion polling when flag is off
- Make OpenSearch + embedding optional in docker-compose via profiles
- Add docker-compose.ingestion.yml override for full-stack ingestion
- Set BATCH_PAGE_SIZE=5 default in Docker local image
- Lead Quick Start with one-liner docker run command
- Document ingestion as opt-in with dedicated section
- Add BATCH_PAGE_SIZE, MAX_FILE_SIZE_MB, MAX_PAGE_COUNT, RATE_LIMIT_RPM to config tables
- Update test counts (380 backend, 159 frontend)
- Date CHANGELOG 0.4.0, bump frontend version to 0.4.0
- Sync CONTRIBUTING.md with E2E Karate test sections
Closes#180
Backend: GET /api/ingestion/search?q=…&doc_id=… endpoint with
SearchResponse schema. Frontend: search bar in Documents page, results
with filename, page, chunk index, relevance score. 3 new API tests.
- local_converter.py: remove redundant `_default_converter = None` in
except block of `_ensure_default_converter` (variable was already None,
re-raised immediately — dead store)
- test_analysis_service.py: replace bare `await task` with
`await asyncio.gather(task)` to satisfy static analysis
- Replace French mode strings (configurer/verifier/preparer) with English
equivalents (configure/verify/prepare) in StudioPage.vue and tests
- Extract _build_conversion_options, _run_conversion, _finalize_analysis
from _run_analysis_inner to respect Single Responsibility Principle
- Rename _get_default_converter to _ensure_default_converter to reflect
its lazy-init side effect
Closes#136, closes#137, closes#138
Add state-machine guard clauses to AnalysisJob transition methods
(mark_running, mark_completed, mark_failed, update_progress) to prevent
invalid status transitions. Make all domain value objects immutable with
frozen=True. Add 11 tests covering guard clause behavior.
Closes#132, closes#133
- Add DocumentRepository and AnalysisRepository protocols in domain/ports.py (#128)
- Refactor persistence repos from module functions to SqliteDocumentRepository
and SqliteAnalysisRepository classes
- Inject repos into AnalysisService and new DocumentService class via
constructor, removing direct imports of persistence and infra.settings (#129)
- Move _merge_results, _classify_error, _extract_html_body to domain/services.py (#130)
- Update main.py composition root to build and wire all dependencies
- Switch api/documents.py to Depends pattern matching api/analyses.py
- Update all tests to use injected mocks instead of module-level patches
Closes#128, closes#129, closes#130
Re-read the job from DB before mark_completed so that
progress_current/progress_total written during batched conversion
are not overwritten by the stale in-memory object.
Add regression unit test and e2e assertion on final progress values.
Replace hardcoded 5 MB upload limit with a configurable setting.
Backend exposes the value via /api/health, frontend reads it
dynamically for validation and UI messages.
Closes#48
Use Docling's native page_range parameter to split large PDFs into
sequential batches, preventing memory exhaustion and timeouts.
Progress is reported via existing polling mechanism.
Closes#56
Use `import infra.local_converter as lc_mod` consistently instead of
mixing `import` and `from ... import` for the same module.
Addresses CodeQL review comment on PR #58.
Docling's native document_timeout is the only mechanism that can
interrupt processing inside a blocked thread (OCR, table extraction).
Without it, asyncio.wait_for cannot stop a frozen conversion.
Configurable via DOCUMENT_TIMEOUT env var (default: 120s).
Closes#57 (C1)
Prevents PyTorch/Docling pipeline crashes on HF Spaces CPU by:
- Reducing max file size from 50 MB to 5 MB
- Adding configurable MAX_PAGE_COUNT setting (env var, default unlimited)
- Increasing conversion timeout from 600s to 900s
- Adding frontend upload validation with explicit error messages
- Exposing maxPageCount via /api/health for dynamic UI hints
local_converter imports docling at module level, so any test that
touches it crashes when docling is not installed. Skip those tests
with importorskip / skipif so the CI (which only has docling-core)
passes cleanly.
The docling library (torch, transformers) is too heavy for lightweight
CI environments that only install docling-core. Use importorskip so
these tests are cleanly skipped instead of crashing collection.
Adapt test expectations to external changes: upload returns 200,
ValueError yields 400, and schemas now accept both snake_case and
camelCase via AliasChoices.
Cover PDF validation, file size limits, preview generation, page
counting, file deletion with path traversal protection, and the
not-found case — all previously untested code paths.
Lightweight sliding-window per-IP rate limiter (100 req/min default)
with no external dependency. Health endpoint is excluded. Returns 429
with Retry-After header when exceeded. Sufficient for single-process
SQLite deployments; document the Redis upgrade path for scale.
Unbounded asyncio.create_task calls could exhaust CPU and memory on
modest hardware. Add a configurable semaphore (MAX_CONCURRENT_ANALYSES,
default 3) so excess jobs queue instead of running all at once.
All endpoint functions, lifespan context manager, and get_connection now
have explicit return types. Upload endpoint returns 201 Created instead
of 200 to follow REST conventions.
UPLOAD_DIR and DB_PATH were read directly from os.environ, bypassing
the Settings dataclass. This caused an inconsistency where overriding
Settings had no effect on these values. Now all modules import from
infra.settings.settings.
domain/ must be pure with no external dependencies. bbox.py imports
docling_core and belongs in infra/. Also refactor ServeConverter to
use the canonical to_topleft_list via BoundingBox instead of
duplicated manual coordinate conversion. Move docling-core to base
requirements since it is now needed in both modes.
Move /health to /api/health on backend and update the frontend
feature flag store to match. Without this, the combined Docker
image nginx proxy could not reach the endpoint and feature flags
(chunking/prepare mode) failed to load.
