Update documentation
This commit is contained in:
parent
463f84a758
commit
2e1518d26d
2 changed files with 87 additions and 53 deletions
140
README.md
140
README.md
|
|
@ -2,8 +2,9 @@
|
|||
|
||||

|
||||

|
||||

|
||||

|
||||
|
||||

|
||||
|
||||
A visual document analysis studio powered by [Docling](https://github.com/DS4SD/docling).
|
||||
Upload a PDF, configure the extraction pipeline, and visualize the results — text, tables, images, formulas, bounding boxes — all from your browser.
|
||||
|
|
@ -12,13 +13,15 @@ Upload a PDF, configure the extraction pipeline, and visualize the results — t
|
|||
|
||||
## Features
|
||||
|
||||
- **PDF viewer** with page navigation and visual overlay toggle
|
||||
- **Configurable Docling pipeline** — OCR on/off, table extraction mode (fast/accurate)
|
||||
- **Bounding box visualization** — overlay extracted elements directly on the PDF with color-coded types
|
||||
- **Home page** with quick upload and recent documents
|
||||
- **PDF viewer** with page navigation, bounding box overlay, and resizable results panel
|
||||
- **Configurable Docling pipeline** — OCR, table extraction, code/formula enrichment, picture classification & description, image generation
|
||||
- **Bounding box visualization** — color-coded element overlay directly on the PDF
|
||||
- **Per-page results** — right panel syncs with the current PDF page
|
||||
- **Document hierarchy** — heading levels and structure preserved from Docling's `iterate_items()` API
|
||||
- **Markdown & HTML export** of extracted content
|
||||
- **Analysis history** — re-visit past analyses
|
||||
- **Document management** — upload, list, delete
|
||||
- **Analysis history** — re-visit and open past analyses
|
||||
- **Dark / Light theme** and **FR / EN** localization
|
||||
|
||||
<details>
|
||||
<summary>More screenshots</summary>
|
||||
|
|
@ -43,27 +46,48 @@ Upload a PDF, configure the extraction pipeline, and visualize the results — t
|
|||
| Service | Stack | Role |
|
||||
|---------|-------|------|
|
||||
| **frontend** | Vue 3, Vite, Pinia | UI, PDF viewer, results display |
|
||||
| **document-parser** | FastAPI, Docling, SQLite, pdf2image | REST API, document parsing, storage, persistence |
|
||||
| **document-parser** | FastAPI, Docling, SQLite, pdf2image | REST API, document parsing, storage |
|
||||
|
||||
### Python project structure (clean architecture)
|
||||
### Backend structure (clean architecture)
|
||||
|
||||
```
|
||||
document-parser/
|
||||
├── main.py # FastAPI app, CORS, lifespan
|
||||
├── domain/ # Pure domain models & Docling logic
|
||||
├── domain/ # Pure domain — no HTTP, no DB
|
||||
│ ├── models.py # Document, AnalysisJob dataclasses
|
||||
│ └── parsing.py # Docling conversion & page extraction
|
||||
│ ├── parsing.py # Docling conversion & page extraction
|
||||
│ └── bbox.py # Bounding box coordinate normalization
|
||||
├── api/ # HTTP layer (FastAPI routers)
|
||||
│ ├── schemas.py # Pydantic DTOs (camelCase serialization)
|
||||
│ ├── documents.py # /api/documents endpoints
|
||||
│ └── analyses.py # /api/analyses endpoints
|
||||
├── persistence/ # Data layer (SQLite)
|
||||
├── persistence/ # Data layer (SQLite via aiosqlite)
|
||||
│ ├── database.py # Connection management, schema init
|
||||
│ ├── document_repo.py # Document CRUD
|
||||
│ └── analysis_repo.py # AnalysisJob CRUD
|
||||
└── services/ # Use case orchestration
|
||||
├── document_service.py # Upload, delete, preview
|
||||
└── analysis_service.py # Async Docling processing
|
||||
├── services/ # Use case orchestration
|
||||
│ ├── document_service.py # Upload, delete, preview
|
||||
│ └── analysis_service.py # Async Docling processing
|
||||
└── tests/ # 99 tests (pytest)
|
||||
```
|
||||
|
||||
### Frontend structure (feature-based)
|
||||
|
||||
```
|
||||
frontend/src/
|
||||
├── app/ # App shell, router, global styles
|
||||
├── pages/ # Route-level pages
|
||||
│ ├── HomePage.vue # Landing page with upload & stats
|
||||
│ ├── StudioPage.vue # PDF viewer + config + results
|
||||
│ ├── DocumentsPage.vue # Document management
|
||||
│ ├── HistoryPage.vue # Past analyses
|
||||
│ └── SettingsPage.vue # Theme, language, API URL
|
||||
├── features/ # Feature modules
|
||||
│ ├── analysis/ # Analysis store, API, bbox, UI components
|
||||
│ ├── document/ # Document store, API, upload, list
|
||||
│ ├── history/ # History store, API, navigation
|
||||
│ └── settings/ # Settings store
|
||||
└── shared/ # Shared utilities (i18n, http, format)
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
|
@ -71,24 +95,18 @@ document-parser/
|
|||
### Docker Compose (recommended)
|
||||
|
||||
```bash
|
||||
# Clone the repo
|
||||
git clone https://github.com/scub-france/docling-studio.git
|
||||
cd docling-studio
|
||||
|
||||
# (Optional) customize settings
|
||||
cp .env.example .env
|
||||
|
||||
# Start all services
|
||||
git clone https://github.com/scub-france/Docling-Studio.git
|
||||
cd Docling-Studio
|
||||
docker compose up --build
|
||||
```
|
||||
|
||||
Open [http://localhost:3000](http://localhost:3000)
|
||||
|
||||
> **Note:** The first analysis takes a bit longer as Docling downloads and caches its ML models (~400 MB). Subsequent runs are fast.
|
||||
> **Note:** The first analysis takes longer as Docling downloads its ML models (~400 MB). Subsequent runs are fast.
|
||||
|
||||
### Local Development
|
||||
|
||||
**Document Parser** (Python 3.12+):
|
||||
**Backend** (Python 3.12+):
|
||||
```bash
|
||||
cd document-parser
|
||||
python -m venv .venv && source .venv/bin/activate
|
||||
|
|
@ -103,66 +121,82 @@ npm install
|
|||
npm run dev
|
||||
```
|
||||
|
||||
## Docling Integration
|
||||
### Running Tests
|
||||
|
||||
The document parser wraps [Docling](https://github.com/DS4SD/docling) with configurable pipeline options exposed as query parameters on the `/parse` endpoint:
|
||||
```bash
|
||||
# Backend (99 tests)
|
||||
cd document-parser
|
||||
pip install pytest pytest-asyncio httpx
|
||||
pytest tests/ -v
|
||||
|
||||
| Parameter | Default | Description |
|
||||
|-----------|---------|-------------|
|
||||
| `do_ocr` | `true` | Enable OCR for scanned documents |
|
||||
| `do_table_structure` | `true` | Enable table structure extraction |
|
||||
| `table_mode` | `accurate` | Table extraction mode: `accurate` or `fast` |
|
||||
# Frontend (87 tests)
|
||||
cd frontend
|
||||
npm run test:run
|
||||
```
|
||||
|
||||
Element types are detected using `isinstance()` checks against Docling's type hierarchy (`TextItem`, `TableItem`, `PictureItem`, `SectionHeaderItem`, etc.) and the document tree depth from `iterate_items()` is preserved for heading-level reconstruction.
|
||||
## Pipeline Options
|
||||
|
||||
These options map directly to Docling's [`PdfPipelineOptions`](https://docling-project.github.io/docling/usage/). See the [Docling documentation](https://docling-project.github.io/docling/) for details on each feature.
|
||||
|
||||
| Option | Default | Description |
|
||||
|--------|---------|-------------|
|
||||
| `do_ocr` | `true` | OCR for scanned pages and embedded images |
|
||||
| `do_table_structure` | `true` | Table detection and row/column reconstruction |
|
||||
| `table_mode` | `accurate` | `accurate` (TableFormer) or `fast` |
|
||||
| `do_code_enrichment` | `false` | Specialized OCR for code blocks |
|
||||
| `do_formula_enrichment` | `false` | Math formula recognition (LaTeX output) |
|
||||
| `do_picture_classification` | `false` | Classify images by type (chart, photo, diagram…) |
|
||||
| `do_picture_description` | `false` | Generate image descriptions via VLM |
|
||||
| `generate_picture_images` | `false` | Extract detected images as separate files |
|
||||
| `generate_page_images` | `false` | Rasterize each page as an image |
|
||||
| `images_scale` | `1.0` | Scale factor for generated images (0.1–10) |
|
||||
|
||||
## Configuration
|
||||
|
||||
All configuration is done via environment variables. See [`.env.example`](.env.example) for available options.
|
||||
All configuration is done via environment variables. See [`.env.example`](.env.example).
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `CORS_ORIGINS` | `http://localhost:3000,...` | CORS allowed origins (comma-separated) |
|
||||
| `UPLOAD_DIR` | `./uploads` | File storage directory |
|
||||
| `DB_PATH` | `./data/docling_studio.db` | SQLite database path |
|
||||
| `CONVERSION_TIMEOUT` | `600` | Max seconds for a single Docling conversion |
|
||||
|
||||
## CI
|
||||
|
||||
GitHub Actions runs on every push to `main` and on pull requests:
|
||||
|
||||
- **Backend job** — Python 3.12, pytest (99 tests)
|
||||
- **Frontend job** — Node 20, vitest (87 tests) + vite build
|
||||
|
||||
Both jobs run in parallel. See [`.github/workflows/ci.yml`](.github/workflows/ci.yml).
|
||||
|
||||
## Performance & System Requirements
|
||||
|
||||
Docling leverages optimized ML models (layout analysis, OCR, table structure) that run efficiently on CPU. The first analysis takes slightly longer as models are downloaded and cached (~400 MB). Subsequent runs are fast, even on large documents.
|
||||
|
||||
| Document type | Pages | Approx. time (CPU) |
|
||||
|---------------|-------|---------------------|
|
||||
| Simple report | 5-10 | ~30s-1 min |
|
||||
| Research paper | 10-30 | ~1-2 min |
|
||||
| Large document | 100+ | ~2-5 min |
|
||||
| Simple report | 5–10 | ~30s–1 min |
|
||||
| Research paper | 10–30 | ~1–2 min |
|
||||
| Large document | 100+ | ~2–5 min |
|
||||
|
||||
### Docker Desktop settings
|
||||
|
||||
The document parser needs **at least 4 GB of RAM**. Recommended Docker Desktop allocation:
|
||||
The document parser needs **at least 4 GB of RAM**:
|
||||
|
||||
| Resource | Minimum | Recommended |
|
||||
|----------|---------|-------------|
|
||||
| Memory | 6 GB | 8 GB+ |
|
||||
| CPUs | 4 | 8+ |
|
||||
|
||||
> On **macOS**: Docker Desktop > Settings > Resources
|
||||
> On **Windows**: Docker Desktop > Settings > Resources > WSL 2
|
||||
|
||||
### Platform support
|
||||
|
||||
All Docker images are **multi-arch** (linux/amd64 + linux/arm64). All processing runs on **CPU** — no GPU required.
|
||||
|
||||
| Platform | Architecture |
|
||||
|----------|-------------|
|
||||
| **macOS Apple Silicon** (M1/M2/M3/M4) | arm64 |
|
||||
| **macOS Intel** | amd64 |
|
||||
| **Linux x86_64** | amd64 |
|
||||
| **Linux ARM** (Raspberry Pi 5, Ampere) | arm64 |
|
||||
| **Windows + WSL2** | amd64 |
|
||||
All Docker images are multi-arch (linux/amd64 + linux/arm64). No GPU required.
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Frontend**: Vue 3 + Vite + Pinia
|
||||
- **Backend**: FastAPI + Docling 2.x + SQLite + pdf2image
|
||||
- **Frontend**: Vue 3, Vite, Pinia, DOMPurify
|
||||
- **Backend**: FastAPI, Docling 2.x, SQLite (aiosqlite), pdf2image
|
||||
- **CI**: GitHub Actions
|
||||
- **Infra**: Docker Compose + Nginx
|
||||
|
||||
## Contributing
|
||||
|
|
|
|||
Binary file not shown.
|
Before Width: | Height: | Size: 517 KiB After Width: | Height: | Size: 655 KiB |
Loading…
Reference in a new issue