Build true detailed documentaiton

2026-03-22 08:56:00 +01:00 · 2026-03-22 08:56:00 +01:00 · e2fe935b63
commit e2fe935b63
parent b45bb87339
8 changed files with 586 additions and 0 deletions
--- a/.github/workflows/docs.yml
+++ b/.github/workflows/docs.yml
@ -0,0 +1,44 @@
+name: Deploy docs
+
+on:
+  push:
+    branches: [main]
+    paths:
+      - "docs/**"
+      - "mkdocs.yml"
+
+permissions:
+  pages: write
+  id-token: write
+
+concurrency:
+  group: docs
+  cancel-in-progress: true
+
+jobs:
+  deploy:
+    name: Build & deploy docs
+    runs-on: ubuntu-latest
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Install MkDocs Material
+        run: pip install mkdocs-material
+
+      - name: Build
+        run: mkdocs build --strict
+
+      - uses: actions/upload-pages-artifact@v3
+        with:
+          path: site/
+
+      - id: deployment
+        uses: actions/deploy-pages@v4
--- a/.gitignore
+++ b/.gitignore
@ -4,6 +4,7 @@ frontend/node_modules/

 # Build outputs
 frontend/dist/
+site/

 # IDE & local tooling
 .idea/
--- a/docs/architecture.md
+++ b/docs/architecture.md
@ -0,0 +1,107 @@
+# Architecture
+
+## Overview
+
+```
+┌────────────┐         ┌───────────────────────┐
+│  Frontend   │────────▶│   Document Parser      │
+│  Vue 3 + TS │  /api/* │ FastAPI + Docling       │
+│  port 3000  │         │ SQLite + file storage   │
+└────────────┘         │   port 8000             │
+                        └───────────────────────┘
+```
+
+Two services communicating via REST. The frontend is a Vue 3 SPA served by Nginx in production. The backend is a FastAPI app that wraps Docling's document conversion engine.
+
+## Backend — Clean Architecture
+
+The backend follows a strict layered architecture. Dependencies flow inward: API → Services → Domain. The domain layer has zero knowledge of HTTP or database.
+
+```
+document-parser/
+├── main.py                   # FastAPI app, CORS, lifespan
+│
+├── domain/                   # Pure domain — no HTTP, no DB
+│   ├── models.py             # Document, AnalysisJob dataclasses
+│   ├── parsing.py            # Docling conversion & page extraction
+│   └── bbox.py               # Bounding box coordinate normalization
+│
+├── api/                      # HTTP layer (FastAPI routers)
+│   ├── schemas.py            # Pydantic DTOs (camelCase serialization)
+│   ├── documents.py          # /api/documents endpoints
+│   └── analyses.py           # /api/analyses endpoints
+│
+├── persistence/              # Data layer (SQLite via aiosqlite)
+│   ├── database.py           # Connection management, schema init
+│   ├── document_repo.py      # Document CRUD
+│   └── analysis_repo.py      # AnalysisJob CRUD
+│
+├── services/                 # Use case orchestration
+│   ├── document_service.py   # Upload, delete, preview
+│   └── analysis_service.py   # Async Docling processing
+│
+└── tests/                    # pytest
+```
+
+### Layer responsibilities
+
+| Layer | Role | Depends on |
+|-------|------|------------|
+| **domain** | Dataclasses, bbox math, Docling conversion | Nothing (pure Python) |
+| **persistence** | SQLite CRUD, aiosqlite | domain (models) |
+| **services** | Orchestrate use cases, call Docling | domain + persistence |
+| **api** | HTTP endpoints, Pydantic DTOs, error handling | services |
+
+### API contract
+
+The API uses **camelCase** serialization (via Pydantic `alias_generator`), while the backend uses **snake_case** internally. The `pages_json` field contains raw `dataclasses.asdict()` output, so page data uses **snake_case** (`page_number`, not `pageNumber`).
+
+## Frontend — Feature-Based
+
+The frontend is organized by feature, each with its own store, API client, and UI components.
+
+```
+frontend/src/
+├── app/                      # App shell, router, global styles
+├── pages/                    # Route-level pages
+│   ├── HomePage.vue
+│   ├── StudioPage.vue        # PDF viewer + config + results
+│   ├── DocumentsPage.vue
+│   ├── HistoryPage.vue
+│   └── SettingsPage.vue
+│
+├── features/                 # Feature modules
+│   ├── analysis/             # Analysis store, API, bbox scaling, UI
+│   │   ├── store.ts
+│   │   ├── api.ts
+│   │   ├── bboxScaling.ts    # Pure math: page coords → pixel coords
+│   │   └── ui/
+│   │       ├── BboxOverlay.vue
+│   │       ├── AnalysisPanel.vue
+│   │       ├── StructureViewer.vue
+│   │       └── ...
+│   ├── document/             # Document store, API, upload
+│   ├── history/              # History store, navigation
+│   └── settings/             # Theme, locale, API URL
+│
+└── shared/                   # Cross-feature utilities
+    ├── types.ts              # All shared TypeScript interfaces
+    ├── i18n.ts               # FR/EN translations
+    ├── format.ts             # Date/size formatters
+    └── api/http.ts           # HTTP client (fetch wrapper)
+```
+
+### Data flow
+
+```
+User action → Pinia store action → API client (fetch) → Backend REST endpoint
+                                                              │
+Backend response → Pinia store state → Vue reactivity → UI update
+```
+
+### Key design decisions
+
+- **Pinia stores** per feature, not global. Each feature owns its state.
+- **TypeScript strict mode** with shared interfaces in `shared/types.ts`.
+- **No component library** — custom CSS with CSS variables for theming.
+- **vue-tsc** in CI to catch type errors before merge.
--- a/docs/bbox-pipeline.md
+++ b/docs/bbox-pipeline.md
@ -0,0 +1,172 @@
+# Bounding Box Pipeline
+
+The bbox pipeline is the core of Docling Studio's visual overlay. It transforms Docling's raw bounding box coordinates into pixel rectangles drawn on the canvas.
+
+## The 3 Coordinate Spaces
+
+```
+ SPACE 1 — Docling (PDF points)         SPACE 2 — Normalized (PDF points)       SPACE 3 — Canvas (pixels)
+ Variable origin per PDF                Always TOPLEFT                           CSS pixels × devicePixelRatio
+
+ BOTTOMLEFT        TOPLEFT
+                                              (0,0) ──────→ x                     (0,0) ──────────→ x
+ y ↑                (0,0) ──→ x                │                                    │
+   │  ┌───┐ t=700    │  ┌───┐ t=92             │  ┌───┐ t=92                        │  ┌─────┐ y=105
+   │  │   │           │  │   │                  │  │   │                             │  │     │
+   │  └───┘ b=600     │  └───┘ b=192            │  └───┘ b=192                       │  └─────┘ y=219
+   │                  ↓ y                       ↓ y                                  ↓ y
+ ──┴──────→ x
+ (0,0)              Unit: pt PDF               Unit: pt PDF                         Unit: CSS px
+```
+
+### Space 1 — Docling Output
+
+Docling's `BoundingBox` has 4 values `(l, t, r, b)` and a `coord_origin`:
+
+- **BOTTOMLEFT** (standard PDF): `y=0` at the bottom of the page. `t > b` because "top" is further from origin.
+- **TOPLEFT** (some extractors): `y=0` at the top. `t < b` as expected.
+
+Unit: **PDF points** (1 pt = 1/72 inch). US Letter = 612 × 792 pt, A4 = 595 × 842 pt.
+
+### Space 2 — Normalized (TOPLEFT)
+
+The backend normalizes all bboxes to TOPLEFT before sending to the frontend. This is what arrives in the JSON `pages` payload.
+
+### Space 3 — Canvas Pixels
+
+The frontend converts PDF points to CSS pixels, then the canvas renders at `devicePixelRatio` for Retina sharpness.
+
+## Transformation 1 — `to_topleft_list()`
+
+**File:** `document-parser/domain/bbox.py`
+
+Normalizes any Docling bbox to `[left, top, right, bottom]` in TOPLEFT coordinates.
+
+```python
+def to_topleft_list(bbox: BoundingBox, page_height: float) -> list[float]:
+    normalized = bbox.to_top_left_origin(page_height)
+    left, top, right, bottom = normalized.l, normalized.t, normalized.r, normalized.b
+
+    # Degenerate bbox: zero or negative dimensions — skip silently.
+    if right <= left or bottom <= top:
+        return list(EMPTY_BBOX)  # [0, 0, 0, 0]
+
+    return [left, top, right, bottom]
+```
+
+**Math (BOTTOMLEFT → TOPLEFT):**
+
+```
+new_top    = page_height - old_top
+new_bottom = page_height - old_bottom
+```
+
+**Example** (US Letter page, 792pt):
+
+```
+Input:  l=50, t=700, r=200, b=600  (BOTTOMLEFT)
+
+new_top    = 792 - 700 = 92     ← near the top of the page
+new_bottom = 792 - 600 = 192    ← below the element
+
+Output: [50, 92, 200, 192]      (TOPLEFT, t < b ✓)
+```
+
+!!! warning "Fallback page dimensions"
+    If Docling doesn't report page dimensions (corrupted PDF), the backend falls back to US Letter (612 × 792 pt). A warning is logged. This may cause slight bbox misalignment on A4 or other formats.
+
+## Transformation 2 — `computeScale()` + `bboxToRect()`
+
+**File:** `frontend/src/features/analysis/bboxScaling.ts`
+
+Maps PDF points to CSS pixels based on the displayed image size.
+
+### Step 2a — Scale factors
+
+```typescript
+function computeScale(displayWidth, displayHeight, pageWidth, pageHeight): Scale {
+  return {
+    sx: displayWidth / pageWidth,    // CSS pixels per PDF point (X axis)
+    sy: displayHeight / pageHeight,  // CSS pixels per PDF point (Y axis)
+  }
+}
+```
+
+**Example:** image rendered at 700px wide for a 612pt page:
+
+```
+sx = 700 / 612 ≈ 1.1438
+sy = 907 / 792 ≈ 1.1451    (≈ same ratio when aspect is preserved)
+```
+
+### Step 2b — Bbox to pixel rectangle
+
+```typescript
+function bboxToRect(bbox: [l, t, r, b], scale: Scale): Rect {
+  return {
+    x: l × sx,         // left edge in pixels
+    y: t × sy,         // top edge in pixels
+    w: (r - l) × sx,   // width in pixels
+    h: (b - t) × sy,   // height in pixels
+  }
+}
+```
+
+**Example** with bbox `[50, 92, 200, 192]` and `sx ≈ sy ≈ 1.14`:
+
+```
+x = 50  × 1.14 =  57 px
+y = 92  × 1.14 = 105 px
+w = 150 × 1.14 = 171 px
+h = 100 × 1.14 = 114 px
+```
+
+## Transformation 3 — Retina Rendering
+
+**File:** `frontend/src/features/analysis/ui/BboxOverlay.vue`
+
+The canvas backing store is scaled by `devicePixelRatio` for crisp rendering on HiDPI screens:
+
+```typescript
+const dpr = window.devicePixelRatio || 1
+
+// Backing store at device resolution
+canvas.width  = displayWidth × dpr    // e.g. 700 × 2 = 1400
+canvas.height = displayHeight × dpr
+
+// CSS size stays the same
+canvas.style.width  = displayWidth + 'px'
+canvas.style.height = displayHeight + 'px'
+
+// Scale the drawing context
+ctx.setTransform(dpr, 0, 0, dpr, 0, 0)
+```
+
+After `setTransform`, all drawing commands use CSS pixel coordinates. The canvas automatically renders them at device resolution.
+
+```
+ctx.strokeRect(57, 105, 171, 114)
+→ Actual pixels on Retina 2x: (114, 210, 342, 228)
+→ Visually identical but 2× sharper
+```
+
+## Complete Pipeline Summary
+
+```
+Docling BoundingBox        bbox.py              bboxScaling.ts         BboxOverlay.vue
+(l, t, r, b)         → to_topleft_list() →  [l, t, r, b]  →  {x, y, w, h}  →  canvas
+BOTTOMLEFT or TOPLEFT   flip Y if needed     PDF points       CSS pixels       device pixels
+unit: PDF points        + validation         TOPLEFT          × (sx, sy)       × dpr
+```
+
+## Validation & Edge Cases
+
+Both backend and frontend guard against degenerate bboxes:
+
+| Check | Backend (`bbox.py`) | Frontend (`bboxScaling.ts`) |
+|-------|--------------------|-----------------------------|
+| Zero/negative width | Returns `[0,0,0,0]` | Returns `EMPTY_RECT` |
+| Zero/negative height | Returns `[0,0,0,0]` | Returns `EMPTY_RECT` |
+| Zero page dimensions | N/A | `computeScale` returns `{1,1}` |
+
+A degenerate bbox results in a zero-area rectangle that the canvas doesn't draw and hit-testing ignores.
--- a/docs/contributing.md
+++ b/docs/contributing.md
@ -0,0 +1,84 @@
+# Contributing
+
+## Getting Started
+
+1. **Fork** the repository
+2. **Clone** your fork:
+   ```bash
+   git clone https://github.com/<your-username>/Docling-Studio.git
+   cd Docling-Studio
+   ```
+3. **Create a branch:**
+   ```bash
+   git checkout -b feature/my-feature
+   ```
+
+## Development Setup
+
+=== "Backend (Python 3.12+)"
+
+    ```bash
+    cd document-parser
+    python -m venv .venv && source .venv/bin/activate
+    pip install -r requirements.txt
+    pip install ruff pytest pytest-asyncio httpx
+    uvicorn main:app --reload --port 8000
+    ```
+
+=== "Frontend (Node 20+)"
+
+    ```bash
+    cd frontend
+    npm install
+    npm run dev
+    ```
+
+## Code Quality
+
+### Backend — Ruff
+
+```bash
+cd document-parser
+ruff check .          # lint
+ruff check . --fix    # auto-fix
+ruff format .         # format
+```
+
+### Frontend — TypeScript + ESLint + Prettier
+
+```bash
+cd frontend
+npm run type-check          # vue-tsc strict mode
+npx eslint src/             # lint
+npx prettier --check src/   # check formatting
+npx prettier --write src/   # auto-format
+```
+
+## Running Tests
+
+=== "Backend"
+
+    ```bash
+    cd document-parser
+    pytest tests/ -v
+    ```
+
+=== "Frontend"
+
+    ```bash
+    cd frontend
+    npm run test:run
+    ```
+
+All tests must pass before submitting a PR.
+
+## Pull Request Guidelines
+
+- Keep PRs focused — one feature or fix per PR
+- Add tests for new functionality
+- Update documentation if behavior changes
+- Ensure CI passes (lint + type-check + tests + build)
+
+## License
+
+By contributing, you agree that your contributions will be licensed under the [MIT License](https://github.com/scub-france/Docling-Studio/blob/main/LICENSE).
--- a/docs/getting-started.md
+++ b/docs/getting-started.md
@ -0,0 +1,86 @@
+# Getting Started
+
+## Docker Compose (recommended)
+
+```bash
+git clone https://github.com/scub-france/Docling-Studio.git
+cd Docling-Studio
+docker compose up --build
+```
+
+Open [http://localhost:3000](http://localhost:3000).
+
+## Local Development
+
+### Backend (Python 3.12+)
+
+```bash
+cd document-parser
+python -m venv .venv && source .venv/bin/activate
+pip install -r requirements.txt
+uvicorn main:app --reload --port 8000
+```
+
+### Frontend (Node 20+)
+
+```bash
+cd frontend
+npm install
+npm run dev
+```
+
+The frontend runs on `http://localhost:3000` and proxies API calls to `http://localhost:8000`.
+
+## Running Tests
+
+=== "Backend"
+
+    ```bash
+    cd document-parser
+    pip install pytest pytest-asyncio httpx
+    pytest tests/ -v
+    ```
+
+=== "Frontend"
+
+    ```bash
+    cd frontend
+    npm run test:run
+    ```
+
+## Pipeline Options
+
+These options map directly to Docling's [`PdfPipelineOptions`](https://docling-project.github.io/docling/usage/).
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `do_ocr` | `true` | OCR for scanned pages and embedded images |
+| `do_table_structure` | `true` | Table detection and row/column reconstruction |
+| `table_mode` | `accurate` | `accurate` (TableFormer) or `fast` |
+| `do_code_enrichment` | `false` | Specialized OCR for code blocks |
+| `do_formula_enrichment` | `false` | Math formula recognition (LaTeX output) |
+| `do_picture_classification` | `false` | Classify images by type |
+| `do_picture_description` | `false` | Generate image descriptions via VLM |
+| `generate_picture_images` | `false` | Extract detected images as separate files |
+| `generate_page_images` | `false` | Rasterize each page as an image |
+| `images_scale` | `1.0` | Scale factor for generated images (0.1–10) |
+
+## Configuration
+
+All configuration is done via environment variables:
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `CORS_ORIGINS` | `http://localhost:3000,...` | CORS allowed origins |
+| `UPLOAD_DIR` | `./uploads` | File storage directory |
+| `DB_PATH` | `./data/docling_studio.db` | SQLite database path |
+| `CONVERSION_TIMEOUT` | `600` | Max seconds per Docling conversion |
+
+## System Requirements
+
+| Resource | Minimum | Recommended |
+|----------|---------|-------------|
+| Memory   | 6 GB    | 8 GB+       |
+| CPUs     | 4       | 8+          |
+
+All Docker images are multi-arch (`linux/amd64` + `linux/arm64`). No GPU required.
--- a/docs/index.md
+++ b/docs/index.md
@ -0,0 +1,44 @@
+# Docling Studio
+
+A visual document analysis studio powered by [Docling](https://github.com/DS4SD/docling).
+
+Upload a PDF, configure the extraction pipeline, and visualize the results — text, tables, images, formulas, bounding boxes — all from your browser.
+
+![Docling Studio — Execution Result](screenshots/DS-execution-result.png)
+
+## Features
+
+- **PDF viewer** with page navigation, bounding box overlay, and resizable results panel
+- **Configurable Docling pipeline** — OCR, table extraction, code/formula enrichment, picture classification & description
+- **Bounding box visualization** — color-coded element overlay directly on the PDF
+- **Markdown & HTML export** of extracted content
+- **Document management** — upload, list, delete
+- **Analysis history** — re-visit and open past analyses
+- **Dark / Light theme** and **FR / EN** localization
+
+## Tech Stack
+
+| Layer | Stack |
+|-------|-------|
+| **Frontend** | Vue 3, TypeScript, Vite, Pinia |
+| **Backend** | FastAPI, Docling 2.x, SQLite (aiosqlite) |
+| **CI** | GitHub Actions (lint, type-check, test, build) |
+| **Infra** | Docker Compose + Nginx |
+
+## Quick Start
+
+```bash
+# Docker (fastest)
+docker run -p 3000:3000 ghcr.io/scub-france/docling-studio:latest
+```
+
+Open [http://localhost:3000](http://localhost:3000) and upload a PDF.
+
+!!! note
+    The first analysis takes longer as Docling downloads its ML models (~400 MB). Subsequent runs are fast.
+
+See [Getting Started](getting-started.md) for local development setup.
+
+## License
+
+[MIT](https://github.com/scub-france/Docling-Studio/blob/main/LICENSE) — Pier-Jean Malandrino
--- a/mkdocs.yml
+++ b/mkdocs.yml
@ -0,0 +1,48 @@
+site_name: Docling Studio
+site_description: Visual document analysis studio powered by Docling
+site_url: https://scub-france.github.io/Docling-Studio/
+repo_url: https://github.com/scub-france/Docling-Studio
+repo_name: scub-france/Docling-Studio
+
+theme:
+  name: material
+  palette:
+    - media: "(prefers-color-scheme: light)"
+      scheme: default
+      primary: deep orange
+      accent: orange
+      toggle:
+        icon: material/brightness-7
+        name: Dark mode
+    - media: "(prefers-color-scheme: dark)"
+      scheme: slate
+      primary: deep orange
+      accent: orange
+      toggle:
+        icon: material/brightness-4
+        name: Light mode
+  features:
+    - navigation.sections
+    - navigation.expand
+    - content.code.copy
+    - toc.integrate
+  icon:
+    repo: fontawesome/brands/github
+
+nav:
+  - Home: index.md
+  - Getting Started: getting-started.md
+  - Architecture: architecture.md
+  - Bbox Pipeline: bbox-pipeline.md
+  - Contributing: contributing.md
+
+markdown_extensions:
+  - admonition
+  - pymdownx.highlight:
+      anchor_linenums: true
+  - pymdownx.superfences
+  - pymdownx.tabbed:
+      alternate_style: true
+  - tables
+  - toc:
+      permalink: true