Merge pull request #181 from scub-france/feature/feature-flag-ingestion

feat(#180): feature-flag ingestion pipeline and brainless one-liner Quick Start
This commit is contained in:
Pier-Jean Malandrino 2026-04-14 16:30:36 +02:00 committed by GitHub
commit 2477a2af4b
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
26 changed files with 317 additions and 425 deletions

3
.gitignore vendored
View file

@ -44,5 +44,8 @@ hs_err_pid*
# Docker
docker-compose.override.yml
# Audit profiles (internal tooling)
profiles/
# E2E tests — Maven build outputs & Chrome user data
e2e/**/target/

View file

@ -4,7 +4,7 @@ All notable changes to Docling Studio will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/), and this project adheres to [Semantic Versioning](https://semver.org/).
## [0.4.0] - Unreleased
## [0.4.0] - 2026-04-13
### Added

View file

@ -96,15 +96,43 @@ npx prettier --write src/ # auto-format
## Running Tests
```bash
# Backend (199 tests)
# Backend (377 tests)
cd document-parser
pytest tests/ -v
# Frontend (129 tests)
# Frontend (156 tests)
cd frontend
npm run test:run
```
### E2E API (Karate)
```bash
# Generate test PDFs + start stack
python e2e/generate-test-data.py
docker compose up -d --wait
# Run all API tests
mvn test -f e2e/api/pom.xml
# Or by tag: @smoke, @regression, @e2e
mvn test -f e2e/api/pom.xml -Dkarate.options="--tags @smoke"
```
### E2E UI (Karate UI)
```bash
# Generate test PDFs + start stack (if not already running)
python e2e/generate-test-data.py
docker compose up -d --wait
# Run critical UI tests (CI scope)
mvn test -f e2e/ui/pom.xml -Dkarate.options="--tags @critical"
# Run all UI tests (local scope)
mvn test -f e2e/ui/pom.xml -Dkarate.options="--tags @ui"
```
All tests must pass before submitting a PR.
## Submitting Changes

View file

@ -31,9 +31,13 @@ Upload a PDF, configure the extraction pipeline, and visualize the results — t
- **Configurable Docling pipeline** — OCR, table extraction, code/formula enrichment, picture classification & description, image generation
- **Bounding box visualization** — color-coded element overlay directly on the PDF
- **Per-page results** — right panel syncs with the current PDF page
- **Chunking** — split extracted content into semantic chunks (hierarchical, hybrid, or page-based) with configurable token limits and inline editing
- **Ingestion pipeline** — Docling → chunking → embedding → OpenSearch vector indexing (one-click from Studio)
- **Markdown & HTML export** of extracted content
- **Document management** — upload, list, delete
- **Document management** — upload, list, delete, search, filter by indexing status
- **Analysis history** — re-visit and open past analyses
- **Upload limits** — configurable max file size and max page count per document
- **Rate limiting** — configurable requests per minute per IP
- **Dark / Light theme** and **FR / EN** localization
@ -74,7 +78,7 @@ document-parser/
├── services/ # Use case orchestration
│ ├── document_service.py # Upload, delete, preview
│ └── analysis_service.py # Async Docling processing
└── tests/ # 199 tests (pytest)
└── tests/ # 377 tests (pytest)
```
### Frontend structure (feature-based)
@ -98,14 +102,24 @@ frontend/src/
## Quick Start
Docling Studio ships two Docker image variants:
One command, nothing else to install:
```bash
docker run -p 3000:3000 ghcr.io/scub-france/docling-studio:latest-local
```
Open [http://localhost:3000](http://localhost:3000), upload a PDF, and get results. That's it.
> **Note:** The first analysis takes longer as Docling downloads its ML models (~400 MB). Subsequent runs are fast.
### Image variants
| Variant | Image tag | Size | Description |
|---------|-----------|------|-------------|
| **local** | `latest-local` | ~1.9 GB | Full — runs Docling in-process, CPU-only |
| **remote** | `latest-remote` | ~270 MB | Lightweight — delegates to an external [Docling Serve](https://github.com/DS4SD/docling-serve) instance |
| **local** | `latest-local` | ~1.9 GB | Full — runs Docling in-process, CPU-only (downloads ML models on first run) |
### Docker — remote mode (fastest)
For remote mode:
```bash
docker run -p 3000:3000 \
@ -113,27 +127,17 @@ docker run -p 3000:3000 \
ghcr.io/scub-france/docling-studio:latest-remote
```
### Docker — local mode (self-contained)
```bash
docker run -p 3000:3000 ghcr.io/scub-france/docling-studio:latest-local
```
> **Note:** The first analysis takes longer as Docling downloads its ML models (~400 MB). Subsequent runs are fast.
Open [http://localhost:3000](http://localhost:3000)
### Docker Compose (for development)
### Docker Compose
```bash
git clone https://github.com/scub-france/Docling-Studio.git
cd Docling-Studio
# Local mode (default)
# Simple mode (backend + frontend only)
docker compose up --build
# Remote mode
CONVERSION_MODE=remote DOCLING_SERVE_URL=http://your-docling-serve:5001 docker compose up --build
# With ingestion pipeline (OpenSearch + embeddings)
docker compose --profile ingestion -f docker-compose.yml -f docker-compose.ingestion.yml up --build
```
### Local Development
@ -162,12 +166,12 @@ npm run dev
### Running Tests
```bash
# Backend (199 tests)
# Backend (377 tests)
cd document-parser
pip install pytest pytest-asyncio httpx
pytest tests/ -v
# Frontend (129 tests)
# Frontend (156 tests)
cd frontend
npm run test:run
```
@ -202,6 +206,43 @@ All configuration is done via environment variables. See [`.env.example`](.env.e
| `UPLOAD_DIR` | `./uploads` | File storage directory |
| `DB_PATH` | `./data/docling_studio.db` | SQLite database path |
| `CONVERSION_TIMEOUT` | `600` | Max seconds for a single Docling conversion |
| `BATCH_PAGE_SIZE` | `10` | Pages per batch (`0` = process all at once) |
| `MAX_FILE_SIZE_MB` | `50` | Maximum upload file size in MB (`0` = unlimited) |
| `MAX_PAGE_COUNT` | `0` | Maximum number of pages per document (`0` = unlimited) |
| `RATE_LIMIT_RPM` | `100` | Max requests per minute per IP (`0` = disabled) |
## Upload Limits
Docling Studio enforces configurable limits on uploaded documents to protect the server against oversized files and long-running analyses:
- **`MAX_FILE_SIZE_MB`** (default `50`) — rejects uploads exceeding this size. Validated at two levels: early `Content-Length` check and streaming byte count.
- **`MAX_PAGE_COUNT`** (default `0` = unlimited) — rejects documents with more pages than allowed. Useful on shared instances or Hugging Face Spaces to cap processing time.
Both limits are exposed in the `/api/health` endpoint so the frontend can display them to the user before upload. Set either to `0` to disable the corresponding check.
## Ingestion Pipeline (opt-in)
Docling Studio can optionally index extracted chunks into [OpenSearch](https://opensearch.org/) for vector and full-text search. This requires two additional services (OpenSearch + embedding) and is **disabled by default**.
To enable ingestion with Docker Compose:
```bash
docker compose --profile ingestion \
-f docker-compose.yml -f docker-compose.ingestion.yml \
up --build
```
When ingestion is enabled, the UI shows:
- An **Ingest** button in Studio to push chunks to OpenSearch
- An **OpenSearch** connection status badge in the sidebar
- **Indexed / Not indexed** filters on the Documents page
- A **Search** page for full-text and vector search across indexed documents
| Variable | Default | Description |
|----------|---------|-------------|
| `OPENSEARCH_URL` | — | OpenSearch endpoint (empty = ingestion disabled) |
| `EMBEDDING_URL` | — | Embedding service endpoint (empty = ingestion disabled) |
| `EMBEDDING_DIMENSION` | `384` | Vector dimension (must match embedding model) |
## CI / Release

View file

@ -74,7 +74,7 @@ services:
DOCLING_SERVE_API_KEY: ${DOCLING_SERVE_API_KEY:-}
RATE_LIMIT_RPM: ${RATE_LIMIT_RPM:-100}
MAX_FILE_SIZE_MB: ${MAX_FILE_SIZE_MB:-50}
BATCH_PAGE_SIZE: ${BATCH_PAGE_SIZE:-0}
BATCH_PAGE_SIZE: ${BATCH_PAGE_SIZE:-10}
OPENSEARCH_URL: http://opensearch:9200
EMBEDDING_URL: http://embedding:8001
command: ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000", "--reload"]

View file

@ -0,0 +1,19 @@
# Override to enable the ingestion pipeline (OpenSearch + embeddings).
#
# Usage:
# docker compose --profile ingestion -f docker-compose.yml -f docker-compose.ingestion.yml up --build
#
# This wires the backend to the OpenSearch and embedding services started
# by the "ingestion" profile and ensures they are healthy before the
# backend starts.
services:
document-parser:
environment:
OPENSEARCH_URL: http://opensearch:9200
EMBEDDING_URL: http://embedding:8001
depends_on:
opensearch:
condition: service_healthy
embedding:
condition: service_healthy

View file

@ -1,6 +1,7 @@
services:
# --- OpenSearch (single-node, security disabled) ---
opensearch:
profiles: ["ingestion"]
image: opensearchproject/opensearch:2
environment:
discovery.type: single-node
@ -16,6 +17,7 @@ services:
# --- Embedding service (sentence-transformers) ---
embedding:
profiles: ["ingestion"]
build:
context: ./embedding-service
environment:
@ -48,14 +50,9 @@ services:
DOCLING_SERVE_API_KEY: ${DOCLING_SERVE_API_KEY:-}
RATE_LIMIT_RPM: ${RATE_LIMIT_RPM:-100}
MAX_FILE_SIZE_MB: ${MAX_FILE_SIZE_MB:-50}
BATCH_PAGE_SIZE: ${BATCH_PAGE_SIZE:-0}
OPENSEARCH_URL: http://opensearch:9200
EMBEDDING_URL: http://embedding:8001
depends_on:
opensearch:
condition: service_healthy
embedding:
condition: service_healthy
BATCH_PAGE_SIZE: ${BATCH_PAGE_SIZE:-10}
OPENSEARCH_URL: ${OPENSEARCH_URL:-}
EMBEDDING_URL: ${EMBEDDING_URL:-}
deploy:
resources:
limits:

View file

@ -1,15 +1,28 @@
# Getting Started
Docling Studio ships two Docker image variants:
## Quick Start
| Variant | Image tag | Size | Description |
|---------|-----------|------|-------------|
| **remote** | `latest-remote` | ~270 MB | Lightweight — delegates to an external [Docling Serve](https://github.com/DS4SD/docling-serve) instance |
| **local** | `latest-local` | ~1.9 GB | Full — runs Docling in-process, CPU-only (downloads ML models on first run) |
One command, nothing else to install:
```bash
docker run -p 3000:3000 ghcr.io/scub-france/docling-studio:latest-local
```
Open [http://localhost:3000](http://localhost:3000), upload a PDF, and get results. That's it.
!!! note
The first analysis takes longer as Docling downloads its ML models (~400 MB). Subsequent runs are fast.
![Docker architecture](images/docker.png){ width="600" }
## Docker — remote mode (fastest)
## Image Variants
| Variant | Image tag | Size | Description |
|---------|-----------|------|-------------|
| **local** | `latest-local` | ~1.9 GB | Full — runs Docling in-process, CPU-only |
| **remote** | `latest-remote` | ~270 MB | Lightweight — delegates to an external [Docling Serve](https://github.com/DS4SD/docling-serve) instance |
For remote mode:
```bash
docker run -p 3000:3000 \
@ -17,27 +30,19 @@ docker run -p 3000:3000 \
ghcr.io/scub-france/docling-studio:latest-remote
```
## Docker — local mode (self-contained)
```bash
docker run -p 3000:3000 ghcr.io/scub-france/docling-studio:latest-local
```
> **Note:** The first analysis takes longer as Docling downloads its ML models (~400 MB). Subsequent runs are fast.
Open [http://localhost:3000](http://localhost:3000).
## Docker Compose (recommended for development)
## Docker Compose
```bash
git clone https://github.com/scub-france/Docling-Studio.git
cd Docling-Studio
# Local mode (default)
# Simple mode (backend + frontend only)
docker compose up --build
# Remote mode
CONVERSION_MODE=remote DOCLING_SERVE_URL=http://your-docling-serve:5001 docker compose up --build
# With ingestion pipeline (OpenSearch + embeddings)
docker compose --profile ingestion \
-f docker-compose.yml -f docker-compose.ingestion.yml \
up --build
```
## Local Development
@ -136,10 +141,48 @@ All configuration is done via environment variables:
| `UPLOAD_DIR` | `./uploads` | File storage directory |
| `DB_PATH` | `./data/docling_studio.db` | SQLite database path |
| `CONVERSION_TIMEOUT` | `600` | Max seconds per Docling conversion |
| `BATCH_PAGE_SIZE` | `10` | Pages per batch (`0` = process all at once) |
| `MAX_CONCURRENT_ANALYSES` | `3` | Maximum parallel analysis jobs |
| `DEPLOYMENT_MODE` | `self-hosted` | `self-hosted` or `huggingface` (shows disclaimer banner) |
| `MAX_FILE_SIZE_MB` | `50` | Maximum upload file size in MB (`0` = unlimited) |
| `MAX_PAGE_COUNT` | `0` | Maximum number of pages per document (`0` = unlimited) |
| `RATE_LIMIT_RPM` | `100` | Max requests per minute per IP (`0` = disabled) |
| `APP_VERSION` | `dev` | Application version (set automatically by CI/Docker) |
## Upload Limits
Docling Studio enforces configurable limits on uploaded documents to protect the server against oversized files and long-running analyses:
- **`MAX_FILE_SIZE_MB`** (default `50`) — rejects uploads exceeding this size. Validated at two levels: early `Content-Length` check and streaming byte count.
- **`MAX_PAGE_COUNT`** (default `0` = unlimited) — rejects documents with more pages than allowed. Useful on shared instances or Hugging Face Spaces to cap processing time.
Both limits are exposed in the `/api/health` endpoint so the frontend can display them to the user before upload. Set either to `0` to disable the corresponding check.
## Ingestion Pipeline (opt-in)
Docling Studio can optionally index extracted chunks into [OpenSearch](https://opensearch.org/) for vector and full-text search. This requires two additional services (OpenSearch + embedding) and is **disabled by default**.
To enable ingestion with Docker Compose:
```bash
docker compose --profile ingestion \
-f docker-compose.yml -f docker-compose.ingestion.yml \
up --build
```
When ingestion is enabled, the UI shows:
- An **Ingest** button in Studio to push chunks to OpenSearch
- An **OpenSearch** connection status badge in the sidebar
- **Indexed / Not indexed** filters on the Documents page
- A **Search** page for full-text and vector search across indexed documents
| Variable | Default | Description |
|----------|---------|-------------|
| `OPENSEARCH_URL` | — | OpenSearch endpoint (empty = ingestion disabled) |
| `EMBEDDING_URL` | — | Embedding service endpoint (empty = ingestion disabled) |
| `EMBEDDING_DIMENSION` | `384` | Vector dimension (must match embedding model) |
## System Requirements
| | Remote image | Local image |

View file

@ -18,7 +18,8 @@ Upload a PDF, configure the extraction pipeline, and visualize the results — t
- **Document management** — upload, list, delete
- **Analysis history** — re-visit and open past analyses
- **Feature flags** — capabilities adapt to the conversion engine (local vs remote)
- **Rate limiting** — 60 requests per minute per IP to protect the backend
- **Upload limits** — configurable max file size (`MAX_FILE_SIZE_MB`) and max page count (`MAX_PAGE_COUNT`) per document
- **Rate limiting** — configurable requests per minute per IP (`RATE_LIMIT_RPM`)
- **Deployment modes** — self-hosted (default) or HuggingFace Spaces (with disclaimer banner)
- **Health endpoint**`/api/health` reports engine type, deployment mode, and database status
- **Dark / Light theme** and **FR / EN** localization

View file

@ -34,6 +34,7 @@ class HealthResponse(_CamelModel):
database: str
max_page_count: int | None = None
max_file_size_mb: int | None = None
ingestion_available: bool = False
class DocumentResponse(_CamelModel):

View file

@ -94,7 +94,7 @@ class Settings:
max_file_size=int(os.environ.get("MAX_FILE_SIZE", "0")),
max_file_size_mb=int(os.environ.get("MAX_FILE_SIZE_MB", "50")),
rate_limit_rpm=int(os.environ.get("RATE_LIMIT_RPM", "100")),
batch_page_size=int(os.environ.get("BATCH_PAGE_SIZE", "0")),
batch_page_size=int(os.environ.get("BATCH_PAGE_SIZE", "10")),
opensearch_url=os.environ.get("OPENSEARCH_URL", ""),
embedding_url=os.environ.get("EMBEDDING_URL", ""),
embedding_dimension=int(os.environ.get("EMBEDDING_DIMENSION", "384")),

View file

@ -138,7 +138,11 @@ async def lifespan(app: FastAPI) -> AsyncIterator[None]:
document_repo, analysis_repo = _build_repos()
app.state.analysis_service = _build_analysis_service(document_repo, analysis_repo)
app.state.document_service = _build_document_service(document_repo, analysis_repo)
app.state.ingestion_service = _build_ingestion_service()
ingestion_service = _build_ingestion_service()
app.state.ingestion_service = ingestion_service
if ingestion_service is not None:
app.include_router(ingestion_router)
logger.info("Ingestion router mounted")
logger.info("Docling Studio backend ready (engine=%s)", settings.conversion_engine)
yield
@ -165,7 +169,6 @@ if settings.rate_limit_rpm > 0:
app.include_router(documents_router)
app.include_router(analyses_router)
app.include_router(ingestion_router)
@app.get("/api/health", response_model=HealthResponse)
@ -188,4 +191,5 @@ async def health() -> HealthResponse:
database=db_status,
max_page_count=settings.max_page_count if settings.max_page_count > 0 else None,
max_file_size_mb=settings.max_file_size_mb if settings.max_file_size_mb > 0 else None,
ingestion_available=getattr(app.state, "ingestion_service", None) is not None,
)

View file

@ -51,6 +51,23 @@ class TestHealthEndpoint:
assert "maxFileSizeMb" in data
assert data["maxFileSizeMb"] == 50
def test_health_exposes_ingestion_available_false(self, client):
original = getattr(app.state, "ingestion_service", None)
app.state.ingestion_service = None
resp = client.get("/api/health")
app.state.ingestion_service = original
data = resp.json()
assert "ingestionAvailable" in data
assert data["ingestionAvailable"] is False
def test_health_exposes_ingestion_available_true(self, client):
original = getattr(app.state, "ingestion_service", None)
app.state.ingestion_service = MagicMock()
resp = client.get("/api/health")
app.state.ingestion_service = original
data = resp.json()
assert data["ingestionAvailable"] is True
class TestDocumentEndpoints:
def test_list_documents(self, client, mock_document_service):

View file

@ -104,7 +104,18 @@ class TestIngestionStatus:
class TestIngestionDisabled:
def test_returns_503_when_disabled(self) -> None:
def test_router_not_mounted_returns_404(self) -> None:
"""When ingestion service is None, router should not be mounted → 404."""
app = FastAPI()
# Do NOT include ingestion router — simulates main.py conditional mount
app.state.ingestion_service = None
app.state.analysis_service = AsyncMock()
tc = TestClient(app)
resp = tc.post("/api/ingestion/job-1")
assert resp.status_code == 404
def test_status_still_returns_503_when_router_mounted_but_service_none(self) -> None:
"""If router is mounted but service is None, endpoints return 503."""
app = FastAPI()
app.include_router(router)
app.state.ingestion_service = None

View file

@ -1,12 +1,12 @@
{
"name": "docling-studio",
"version": "0.3.0",
"version": "0.3.1",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "docling-studio",
"version": "0.3.0",
"version": "0.3.1",
"dependencies": {
"dompurify": "^3.3.3",
"marked": "^17.0.4",

View file

@ -1,6 +1,6 @@
{
"name": "docling-studio",
"version": "0.3.1",
"version": "0.4.0",
"private": true,
"type": "module",
"scripts": {

View file

@ -202,7 +202,10 @@
</div>
</div>
<div class="chunk-empty" v-else-if="!chunkingStore.rechunking">
<div
class="chunk-empty"
v-if="!pageChunks.length && !chunkingStore.rechunking && deleteConfirmIdx === -1"
>
<p>
{{ chunks.length ? t('chunking.noChunksOnPage') : t('chunking.noChunks') }}
</p>

View file

@ -82,6 +82,38 @@ describe('useFeatureFlagStore', () => {
expect(store.maxFileSizeMb).toBe(0)
})
it('enables ingestion when ingestionAvailable is true', async () => {
mockApiFetch.mockResolvedValue({
status: 'ok',
engine: 'local',
ingestionAvailable: true,
})
const store = useFeatureFlagStore()
await store.load()
expect(store.ingestionAvailable).toBe(true)
expect(store.isEnabled('ingestion')).toBe(true)
})
it('disables ingestion when ingestionAvailable is false', async () => {
mockApiFetch.mockResolvedValue({
status: 'ok',
engine: 'local',
ingestionAvailable: false,
})
const store = useFeatureFlagStore()
await store.load()
expect(store.ingestionAvailable).toBe(false)
expect(store.isEnabled('ingestion')).toBe(false)
})
it('defaults ingestionAvailable to false when missing', async () => {
mockApiFetch.mockResolvedValue({ status: 'ok', engine: 'local' })
const store = useFeatureFlagStore()
await store.load()
expect(store.ingestionAvailable).toBe(false)
expect(store.isEnabled('ingestion')).toBe(false)
})
it('handles health endpoint failure gracefully', async () => {
mockApiFetch.mockRejectedValue(new Error('Network error'))
const store = useFeatureFlagStore()

View file

@ -13,9 +13,10 @@ interface HealthResponse {
deploymentMode?: DeploymentMode
maxPageCount?: number
maxFileSizeMb?: number
ingestionAvailable?: boolean
}
export type FeatureFlag = 'chunking' | 'disclaimer'
export type FeatureFlag = 'chunking' | 'disclaimer' | 'ingestion'
interface FeatureFlagDef {
description: string
@ -25,6 +26,7 @@ interface FeatureFlagDef {
interface FeatureFlagContext {
engine: ConversionEngine | null
deploymentMode: DeploymentMode | null
ingestionAvailable: boolean
}
const featureRegistry: Record<FeatureFlag, FeatureFlagDef> = {
@ -36,6 +38,10 @@ const featureRegistry: Record<FeatureFlag, FeatureFlagDef> = {
description: 'Show shared-instance disclaimer banner',
isEnabled: (ctx) => ctx.deploymentMode === 'huggingface',
},
ingestion: {
description: 'OpenSearch ingestion pipeline (embedding + vector indexing)',
isEnabled: (ctx) => ctx.ingestionAvailable,
},
}
export const useFeatureFlagStore = defineStore('feature-flags', () => {
@ -43,6 +49,7 @@ export const useFeatureFlagStore = defineStore('feature-flags', () => {
const deploymentMode = ref<DeploymentMode | null>(null)
const maxPageCount = ref<number>(0)
const maxFileSizeMb = ref<number>(0)
const ingestionAvailable = ref(false)
const appVersion = ref<string>(__APP_VERSION__)
const loaded = ref(false)
const error = ref<string | null>(null)
@ -50,6 +57,7 @@ export const useFeatureFlagStore = defineStore('feature-flags', () => {
const context = computed<FeatureFlagContext>(() => ({
engine: engine.value,
deploymentMode: deploymentMode.value,
ingestionAvailable: ingestionAvailable.value,
}))
function isEnabled(flag: FeatureFlag): boolean {
@ -65,6 +73,7 @@ export const useFeatureFlagStore = defineStore('feature-flags', () => {
deploymentMode.value = data.deploymentMode ?? 'self-hosted'
maxPageCount.value = data.maxPageCount ?? 0
maxFileSizeMb.value = data.maxFileSizeMb ?? 0
ingestionAvailable.value = data.ingestionAvailable ?? false
appMaxFileSizeMb.value = maxFileSizeMb.value
appMaxPageCount.value = maxPageCount.value
if (data.version) appVersion.value = data.version
@ -81,6 +90,7 @@ export const useFeatureFlagStore = defineStore('feature-flags', () => {
deploymentMode,
maxPageCount,
maxFileSizeMb,
ingestionAvailable,
appVersion,
loaded,
error,

View file

@ -9,7 +9,7 @@
class="search-input"
:placeholder="t('ingestion.search')"
/>
<div class="filter-group">
<div v-if="ingestionEnabled" class="filter-group">
<button
v-for="f in filters"
:key="f.value"
@ -54,17 +54,19 @@
</div>
</div>
<div class="doc-row-actions">
<span
v-if="ingestionStore.ingestedDocs[doc.id]"
class="status-badge indexed"
:title="t('ingestion.chunksIndexed', { n: ingestionStore.ingestedDocs[doc.id] })"
>
{{ t('ingestion.indexed') }}
<span class="badge-count">{{ ingestionStore.ingestedDocs[doc.id] }}</span>
</span>
<span v-else class="status-badge not-indexed">
{{ t('ingestion.notIndexed') }}
</span>
<template v-if="ingestionEnabled">
<span
v-if="ingestionStore.ingestedDocs[doc.id]"
class="status-badge indexed"
:title="t('ingestion.chunksIndexed', { n: ingestionStore.ingestedDocs[doc.id] })"
>
{{ t('ingestion.indexed') }}
<span class="badge-count">{{ ingestionStore.ingestedDocs[doc.id] }}</span>
</span>
<span v-else class="status-badge not-indexed">
{{ t('ingestion.notIndexed') }}
</span>
</template>
<button
class="action-btn"
:title="t('ingestion.openInStudio')"
@ -77,7 +79,7 @@
</svg>
</button>
<button
v-if="ingestionStore.ingestedDocs[doc.id]"
v-if="ingestionEnabled && ingestionStore.ingestedDocs[doc.id]"
class="action-btn unindex"
:title="t('ingestion.deleteIndex')"
@click="confirmRemoveFromIndex(doc.id)"
@ -110,13 +112,16 @@
import { computed, onMounted, ref } from 'vue'
import { useRouter } from 'vue-router'
import { useDocumentStore } from '../features/document/store'
import { useFeatureFlagStore } from '../features/feature-flags/store'
import { useIngestionStore } from '../features/ingestion/store'
import { useI18n } from '../shared/i18n'
import { formatSize } from '../shared/format'
import type { Document } from '../shared/types'
const docStore = useDocumentStore()
const featureStore = useFeatureFlagStore()
const ingestionStore = useIngestionStore()
const ingestionEnabled = computed(() => featureStore.isEnabled('ingestion'))
const router = useRouter()
const { t } = useI18n()
@ -173,7 +178,7 @@ function confirmRemoveFromIndex(docId: string) {
}
async function handleDelete(docId: string) {
if (ingestionStore.ingestedDocs[docId]) {
if (ingestionEnabled.value && ingestionStore.ingestedDocs[docId]) {
await ingestionStore.deleteIngested(docId)
}
await docStore.remove(docId)
@ -181,7 +186,9 @@ async function handleDelete(docId: string) {
onMounted(() => {
docStore.load()
ingestionStore.checkAvailability()
if (ingestionEnabled.value) {
ingestionStore.checkAvailability()
}
})
</script>

View file

@ -4,7 +4,7 @@
<h1 class="page-title">{{ t('nav.search') }}</h1>
</div>
<div v-if="!ingestionStore.available" class="tab-empty">
<div v-if="!ingestionEnabled || !ingestionStore.available" class="tab-empty">
{{ t('ingestion.unavailable') }}
</div>
@ -50,13 +50,16 @@
</template>
<script setup lang="ts">
import { onMounted, ref } from 'vue'
import { computed, onMounted, ref } from 'vue'
import { useSearchStore } from '../features/search/store'
import { useFeatureFlagStore } from '../features/feature-flags/store'
import { useIngestionStore } from '../features/ingestion/store'
import { useI18n } from '../shared/i18n'
const searchStore = useSearchStore()
const featureStore = useFeatureFlagStore()
const ingestionStore = useIngestionStore()
const ingestionEnabled = computed(() => featureStore.isEnabled('ingestion'))
const { t } = useI18n()
const searchInput = ref('')
@ -70,7 +73,9 @@ function runSearch() {
}
onMounted(() => {
ingestionStore.checkAvailability()
if (ingestionEnabled.value) {
ingestionStore.checkAvailability()
}
})
</script>

View file

@ -67,7 +67,7 @@
{{ t('studio.prepare') }}
</button>
<button
v-if="chunkingEnabled && ingestionStore.available"
v-if="chunkingEnabled && ingestionEnabled && ingestionStore.available"
class="toggle-btn"
data-e2e="toggle-btn"
:class="{ active: mode === 'ingest' }"
@ -504,6 +504,7 @@ const analysisStore = useAnalysisStore()
const ingestionStore = useIngestionStore()
const { t } = useI18n()
const chunkingEnabled = useFeatureFlag('chunking')
const ingestionEnabled = useFeatureFlag('ingestion')
const mode = ref('configure')
const currentPage = ref(1)
@ -636,7 +637,9 @@ watch(
onMounted(async () => {
await documentStore.load()
analysisStore.load()
ingestionStore.checkAvailability()
if (ingestionEnabled.value) {
ingestionStore.checkAvailability()
}
// Restore analysis from history via query param
const analysisId = route.query.analysisId

View file

@ -130,7 +130,7 @@ const messages: Messages = {
'chunking.deleteConfirm':
'Supprimer ce chunk ? Il sera marqué comme supprimé jusqu\u2019à la prochaine synchronisation.',
'chunking.batchNotice':
'Le chunking n\u2019est pas disponible pour cette analyse. Les documents volumineux trait\u00e9s par batch ne g\u00e9n\u00e8rent pas la structure interne n\u00e9cessaire au d\u00e9coupage.',
'Le chunking n\u2019est pas disponible pour cette analyse. Les documents volumineux trait\u00e9s par batch ne g\u00e9n\u00e8rent pas la structure interne n\u00e9cessaire au d\u00e9coupage. Coming soon !',
// Search
'nav.search': 'Recherche',
@ -299,7 +299,7 @@ const messages: Messages = {
'chunking.deleteConfirm':
'Delete this chunk? It will be marked as deleted until the next sync.',
'chunking.batchNotice':
'Chunking is not available for this analysis. Large documents processed in batch mode do not generate the internal structure required for chunking.',
'Chunking is not available for this analysis. Large documents processed in batch mode do not generate the internal structure required for chunking. Coming soon!',
'nav.search': 'Search',
'search.hint': 'Enter a term to search through indexed chunks.',

View file

@ -46,6 +46,7 @@
</RouterLink>
<RouterLink
v-if="ingestionEnabled"
to="/search"
class="nav-item"
data-e2e="nav-search"
@ -96,7 +97,7 @@
<div class="sidebar-footer">
<div
v-if="ingestionStore.available"
v-if="ingestionEnabled && ingestionStore.available"
class="opensearch-status"
:title="
ingestionStore.opensearchConnected
@ -136,6 +137,7 @@ import { useIngestionStore } from '../../features/ingestion/store'
const featureStore = useFeatureFlagStore()
const ingestionStore = useIngestionStore()
const ingestionEnabled = computed(() => featureStore.isEnabled('ingestion'))
const version = computed(() => featureStore.appVersion)
const route = useRoute()
const { t } = useI18n()
@ -145,8 +147,10 @@ defineProps({
})
onMounted(() => {
ingestionStore.checkAvailability()
ingestionStore.startPolling(30_000)
if (ingestionEnabled.value) {
ingestionStore.checkAvailability()
ingestionStore.startPolling(30_000)
}
})
onBeforeUnmount(() => {

View file

@ -1,267 +0,0 @@
#!/usr/bin/env bash
# ============================================================================
# Docling Studio — Automated Audit Checks (FastAPI + Vue 3 profile)
# ============================================================================
# Runs verification commands for each of the 12 release audits.
# Usage: bash profiles/fastapi-vue/commands.sh
# Run from the repository root.
# ============================================================================
set -uo pipefail
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'
PASS=0
WARN=0
FAIL=0
pass() { echo -e " ${GREEN}PASS${NC} $1"; ((PASS++)); }
warn() { echo -e " ${YELLOW}WARN${NC} $1"; ((WARN++)); }
fail() { echo -e " ${RED}FAIL${NC} $1"; ((FAIL++)); }
# ── 01. Clean Architecture ─────────────────────────────────────────────────
echo ""
echo "== 01. Clean Architecture =="
# Domain must not import from api, persistence, infra
if grep -rn "from api\|from persistence\|from infra\|import fastapi\|import aiosqlite" document-parser/domain/ 2>/dev/null; then
fail "Domain layer imports forbidden modules"
else
pass "Domain layer has no forbidden imports"
fi
# API must not import from persistence directly
if grep -rn "from persistence" document-parser/api/ 2>/dev/null; then
fail "API layer imports directly from persistence"
else
pass "API layer does not import from persistence"
fi
# ── 02. DDD ────────────────────────────────────────────────────────────────
echo ""
echo "== 02. DDD =="
# Domain models exist
if [ -f document-parser/domain/models.py ] && [ -f document-parser/domain/ports.py ]; then
pass "Domain models and ports exist"
else
fail "Missing domain/models.py or domain/ports.py"
fi
# Value objects exist
if [ -f document-parser/domain/value_objects.py ]; then
pass "Value objects defined"
else
warn "No value_objects.py in domain"
fi
# ── 03. Clean Code ─────────────────────────────────────────────────────────
echo ""
echo "== 03. Clean Code =="
# Check for files > 300 lines (backend)
LARGE_FILES=$(find document-parser -name "*.py" ! -path "*/.venv/*" ! -path "*/__pycache__/*" ! -path "*/tests/*" -exec awk 'END{if(NR>300) print FILENAME": "NR" lines"}' {} \;)
if [ -n "$LARGE_FILES" ]; then
warn "Large Python files (>300 lines):"
echo "$LARGE_FILES"
else
pass "No Python files exceed 300 lines"
fi
# Check for files > 300 lines (frontend)
LARGE_VUE=$(find frontend/src -name "*.vue" -o -name "*.ts" | xargs awk 'END{if(NR>300) print FILENAME": "NR" lines"}' 2>/dev/null)
if [ -n "$LARGE_VUE" ]; then
warn "Large frontend files (>300 lines):"
echo "$LARGE_VUE"
else
pass "No frontend files exceed 300 lines"
fi
# ── 04. KISS ───────────────────────────────────────────────────────────────
echo ""
echo "== 04. KISS =="
# Check for overly complex patterns
if grep -rn "type:\s*ignore" document-parser/ --include="*.py" ! -path "*/.venv/*" 2>/dev/null; then
warn "Found type: ignore comments (review if justified)"
else
pass "No unjustified type: ignore"
fi
# ── 05. DRY ────────────────────────────────────────────────────────────────
echo ""
echo "== 05. DRY =="
# Check for magic numbers in backend
if grep -rn "[^a-zA-Z_][0-9]\{3,\}[^0-9]" document-parser/ --include="*.py" ! -path "*/.venv/*" ! -path "*/tests/*" 2>/dev/null | grep -v "port\|version\|status\|#\|MAX_\|DEFAULT_\|LIMIT_" | head -5; then
warn "Possible magic numbers found (review above)"
else
pass "No obvious magic numbers"
fi
# ── 06. SOLID ──────────────────────────────────────────────────────────────
echo ""
echo "== 06. SOLID =="
# Check that ports (interfaces) exist
if grep -l "Protocol\|ABC\|abstractmethod" document-parser/domain/ports.py 2>/dev/null; then
pass "Domain ports use Protocol/ABC (Dependency Inversion)"
else
fail "No abstract ports found in domain"
fi
# ── 07. Decoupling ─────────────────────────────────────────────────────────
echo ""
echo "== 07. Decoupling =="
# Frontend should not hardcode backend URLs (except in config)
if grep -rn "localhost:8000\|127.0.0.1:8000" frontend/src/ --include="*.ts" --include="*.vue" 2>/dev/null | grep -v "config\|env\|http.ts"; then
fail "Frontend hardcodes backend URL outside config"
else
pass "Frontend backend URL is configurable"
fi
# ── 08. Security ───────────────────────────────────────────────────────────
echo ""
echo "== 08. Security =="
# Check for hardcoded secrets
if grep -rni "password\s*=\s*['\"].\+['\"\|secret\s*=\s*['\"].\+['\"\|api_key\s*=\s*['\"].\+['\"]" document-parser/ --include="*.py" ! -path "*/.venv/*" ! -path "*/tests/*" 2>/dev/null; then
fail "Possible hardcoded secrets found"
else
pass "No hardcoded secrets detected"
fi
# Check for eval/exec
if grep -rn "\beval(\|exec(" document-parser/ --include="*.py" ! -path "*/.venv/*" 2>/dev/null; then
fail "eval() or exec() found in backend"
else
pass "No eval/exec in backend"
fi
# Check CORS configuration exists
if grep -rn "CORSMiddleware" document-parser/ --include="*.py" ! -path "*/.venv/*" 2>/dev/null > /dev/null; then
pass "CORS middleware is configured"
else
warn "No CORS middleware found"
fi
# ── 09. Tests ──────────────────────────────────────────────────────────────
echo ""
echo "== 09. Tests =="
# Backend tests exist
BACKEND_TESTS=$(find document-parser/tests -name "test_*.py" 2>/dev/null | wc -l)
if [ "$BACKEND_TESTS" -gt 0 ]; then
pass "Backend: $BACKEND_TESTS test files found"
else
fail "No backend test files found"
fi
# Frontend tests exist
FRONTEND_TESTS=$(find frontend/src -name "*.test.*" 2>/dev/null | wc -l)
if [ "$FRONTEND_TESTS" -gt 0 ]; then
pass "Frontend: $FRONTEND_TESTS test files found"
else
fail "No frontend test files found"
fi
# E2E tests exist
E2E_TESTS=$(find e2e -name "*.feature" 2>/dev/null | wc -l)
if [ "$E2E_TESTS" -gt 0 ]; then
pass "E2E: $E2E_TESTS feature files found"
else
warn "No e2e feature files found"
fi
# Check for skipped tests
if grep -rn "@skip\|@ignore\|xit(\|xdescribe(\|pytest.mark.skip" document-parser/tests/ frontend/src/ 2>/dev/null | grep -v "helpers"; then
warn "Skipped tests found (review if intentional)"
else
pass "No skipped tests"
fi
# ── 10. CI / Build ────────────────────────────────────────────────────────
echo ""
echo "== 10. CI / Build =="
# CI workflow exists
if [ -f .github/workflows/ci.yml ]; then
pass "CI workflow exists"
else
fail "No CI workflow found"
fi
# Dockerfile exists
if [ -f Dockerfile ]; then
pass "Dockerfile exists"
else
fail "No Dockerfile found"
fi
# Health check in docker-compose
if grep -q "healthcheck" docker-compose.yml 2>/dev/null; then
pass "Docker Compose has health check"
else
warn "No health check in docker-compose.yml"
fi
# ── 11. Documentation ─────────────────────────────────────────────────────
echo ""
echo "== 11. Documentation =="
# CHANGELOG exists and has content
if [ -f CHANGELOG.md ] && [ -s CHANGELOG.md ]; then
pass "CHANGELOG.md exists and is not empty"
else
fail "CHANGELOG.md missing or empty"
fi
# README exists
if [ -f README.md ]; then
pass "README.md exists"
else
fail "README.md missing"
fi
# Check for TODO/FIXME without issue reference
TODOS=$(grep -rn "TODO\|FIXME" document-parser/ frontend/src/ --include="*.py" --include="*.ts" --include="*.vue" ! -path "*/.venv/*" ! -path "*/node_modules/*" 2>/dev/null | grep -v "#[0-9]" | head -5)
if [ -n "$TODOS" ]; then
warn "TODO/FIXME without issue reference:"
echo "$TODOS"
else
pass "No orphaned TODO/FIXME"
fi
# ── 12. Performance ───────────────────────────────────────────────────────
echo ""
echo "== 12. Performance =="
# Check for synchronous file I/O in async context
if grep -rn "open(" document-parser/api/ document-parser/services/ --include="*.py" 2>/dev/null | grep -v "aiofiles\|async\|#"; then
warn "Synchronous file I/O in async code (review above)"
else
pass "No synchronous file I/O in async endpoints"
fi
# Check for N+1 patterns (loop with DB call)
if grep -rn "for.*in.*:" document-parser/services/ --include="*.py" -A5 2>/dev/null | grep "await.*repo\|await.*db"; then
warn "Possible N+1 query pattern (review above)"
else
pass "No obvious N+1 patterns"
fi
# ── Summary ────────────────────────────────────────────────────────────────
echo ""
echo "============================================"
echo -e " ${GREEN}PASS${NC}: $PASS"
echo -e " ${YELLOW}WARN${NC}: $WARN"
echo -e " ${RED}FAIL${NC}: $FAIL"
echo "============================================"
if [ "$FAIL" -gt 0 ]; then
exit 1
fi

View file

@ -1,70 +0,0 @@
# Stack Profile — FastAPI + Vue 3
Profile for running the 12 release audits on Docling Studio.
## Layer Mapping
| Generic Layer | Docling Studio Path | Language |
|---------------|---------------------|----------|
| **Domain** | `document-parser/domain/` | Python |
| **Services** | `document-parser/services/` | Python |
| **API** | `document-parser/api/` | Python |
| **Infrastructure** | `document-parser/infra/` | Python |
| **Persistence** | `document-parser/persistence/` | Python |
| **Frontend** | `frontend/src/` | TypeScript / Vue |
| **Tests (backend)** | `document-parser/tests/` | Python |
| **Tests (frontend)** | `frontend/src/**/*.test.*` | TypeScript |
| **Tests (e2e API)** | `e2e/api/` | Karate (Gherkin) |
| **Tests (e2e UI)** | `e2e/ui/` | Karate UI (Gherkin) |
| **CI/CD** | `.github/workflows/` | YAML |
| **Docker** | `Dockerfile`, `docker-compose.yml`, `nginx.conf` | Docker / Nginx |
## Excluded Paths
These paths are excluded from audits:
- `document-parser/.venv/`
- `document-parser/__pycache__/`
- `frontend/node_modules/`
- `frontend/dist/`
- `e2e/**/target/`
## Framework Detection
Imports that should NOT appear in the domain layer:
```python
# Forbidden in document-parser/domain/
from fastapi import ...
from pydantic import ... # except BaseModel for value objects
import aiosqlite
from infra import ...
from persistence import ...
from api import ...
```
Imports that should NOT cross feature boundaries in the frontend:
```typescript
// features/analysis/ should NOT import from features/chunking/store
// features/document/ should NOT import from features/analysis/store
// Cross-feature communication goes through shared/ or events
```
## Tools & Commands
| Task | Command |
|------|---------|
| Backend lint | `cd document-parser && ruff check .` |
| Backend format | `cd document-parser && ruff format --check .` |
| Backend tests | `cd document-parser && pytest tests/ -v` |
| Frontend lint | `cd frontend && npx eslint src/` |
| Frontend type-check | `cd frontend && npm run type-check` |
| Frontend format | `cd frontend && npx prettier --check src/` |
| Frontend tests | `cd frontend && npm run test:run` |
| E2E API tests | `mvn test -f e2e/api/pom.xml -Dkarate.options="--tags @smoke"` |
| E2E UI tests | `mvn test -f e2e/ui/pom.xml -Dkarate.options="--tags @critical"` |
| Docker build | `docker compose build` |
| Docker health | `curl -s http://localhost:3000/api/health` |
| Dependency audit (Python) | `cd document-parser && pip audit` |
| Dependency audit (Node) | `cd frontend && npm audit` |