Add live documentaiton and change architecture diagram
This commit is contained in:
parent
765b3aa4b8
commit
c2aeec3092
5 changed files with 126 additions and 8 deletions
95
.github/workflows/docling-compat.yml
vendored
Normal file
95
.github/workflows/docling-compat.yml
vendored
Normal file
|
|
@ -0,0 +1,95 @@
|
||||||
|
name: Docling compatibility
|
||||||
|
|
||||||
|
# Runs daily and on manual trigger.
|
||||||
|
# Installs the LATEST docling + docling-core (ignoring version pins)
|
||||||
|
# and runs the backend tests. Opens an issue if something breaks.
|
||||||
|
|
||||||
|
on:
|
||||||
|
schedule:
|
||||||
|
- cron: "30 6 * * *" # Every day at 06:30 UTC
|
||||||
|
workflow_dispatch: # Manual trigger from GitHub UI
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
compat:
|
||||||
|
name: Test against latest Docling
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
defaults:
|
||||||
|
run:
|
||||||
|
working-directory: document-parser
|
||||||
|
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- uses: actions/setup-python@v5
|
||||||
|
with:
|
||||||
|
python-version: "3.12"
|
||||||
|
cache: pip
|
||||||
|
cache-dependency-path: document-parser/requirements.txt
|
||||||
|
|
||||||
|
- name: Install system dependencies
|
||||||
|
run: sudo apt-get update && sudo apt-get install -y --no-install-recommends poppler-utils
|
||||||
|
|
||||||
|
- name: Install pinned dependencies
|
||||||
|
run: |
|
||||||
|
pip install --upgrade pip
|
||||||
|
pip install -r requirements.txt
|
||||||
|
pip install pytest pytest-asyncio httpx
|
||||||
|
|
||||||
|
- name: Upgrade docling to latest
|
||||||
|
id: versions
|
||||||
|
run: |
|
||||||
|
pip install --upgrade docling docling-core
|
||||||
|
echo "docling=$(pip show docling | grep Version | cut -d' ' -f2)" >> "$GITHUB_OUTPUT"
|
||||||
|
echo "docling_core=$(pip show docling-core | grep Version | cut -d' ' -f2)" >> "$GITHUB_OUTPUT"
|
||||||
|
|
||||||
|
- name: Run tests
|
||||||
|
run: pytest tests/ -v
|
||||||
|
|
||||||
|
- name: Open issue on failure
|
||||||
|
if: failure()
|
||||||
|
uses: actions/github-script@v7
|
||||||
|
with:
|
||||||
|
script: |
|
||||||
|
const docling = '${{ steps.versions.outputs.docling }}';
|
||||||
|
const doclingCore = '${{ steps.versions.outputs.docling_core }}';
|
||||||
|
const run = `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
|
||||||
|
|
||||||
|
// Don't open duplicate issues
|
||||||
|
const { data: issues } = await github.rest.issues.listForRepo({
|
||||||
|
owner: context.repo.owner,
|
||||||
|
repo: context.repo.repo,
|
||||||
|
state: 'open',
|
||||||
|
labels: 'docling-compat',
|
||||||
|
});
|
||||||
|
|
||||||
|
const title = `Docling compatibility break: docling ${docling} / docling-core ${doclingCore}`;
|
||||||
|
|
||||||
|
if (issues.some(i => i.title === title)) {
|
||||||
|
console.log('Issue already exists, skipping.');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
await github.rest.issues.create({
|
||||||
|
owner: context.repo.owner,
|
||||||
|
repo: context.repo.repo,
|
||||||
|
title,
|
||||||
|
labels: ['docling-compat', 'bug'],
|
||||||
|
body: [
|
||||||
|
`## Docling compatibility failure`,
|
||||||
|
``,
|
||||||
|
`The daily compatibility check failed with the latest Docling releases:`,
|
||||||
|
``,
|
||||||
|
`| Package | Version |`,
|
||||||
|
`|---------|---------|`,
|
||||||
|
`| \`docling\` | \`${docling}\` |`,
|
||||||
|
`| \`docling-core\` | \`${doclingCore}\` |`,
|
||||||
|
``,
|
||||||
|
`**Workflow run:** [View logs](${run})`,
|
||||||
|
``,
|
||||||
|
`### Next steps`,
|
||||||
|
`1. Check the failing tests in the workflow logs`,
|
||||||
|
`2. Identify breaking changes in the Docling changelog`,
|
||||||
|
`3. Update the code and version pins in \`requirements.txt\``,
|
||||||
|
`4. Close this issue once fixed`,
|
||||||
|
].join('\n'),
|
||||||
|
});
|
||||||
BIN
docs/Sans-titre-2026-03-10-1116.png
Normal file
BIN
docs/Sans-titre-2026-03-10-1116.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 55 KiB |
|
|
@ -2,17 +2,38 @@
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
```
|
{ width="700" }
|
||||||
┌────────────┐ ┌───────────────────────┐
|
|
||||||
│ Frontend │────────▶│ Document Parser │
|
|
||||||
│ Vue 3 + TS │ /api/* │ FastAPI + Docling │
|
|
||||||
│ port 3000 │ │ SQLite + file storage │
|
|
||||||
└────────────┘ │ port 8000 │
|
|
||||||
└───────────────────────┘
|
|
||||||
```
|
|
||||||
|
|
||||||
Two services communicating via REST. The frontend is a Vue 3 SPA served by Nginx in production. The backend is a FastAPI app that wraps Docling's document conversion engine.
|
Two services communicating via REST. The frontend is a Vue 3 SPA served by Nginx in production. The backend is a FastAPI app that wraps Docling's document conversion engine.
|
||||||
|
|
||||||
|
### Zooming into the backend
|
||||||
|
|
||||||
|
The schema above shows the macro view. Inside the backend, the code follows a **Clean Architecture** with strict layer boundaries:
|
||||||
|
|
||||||
|
```
|
||||||
|
┌──────────────────────────────────────────────────────┐
|
||||||
|
│ Backend │
|
||||||
|
│ │
|
||||||
|
│ ┌──────────┐ │
|
||||||
|
│ │ api/ │ ← HTTP (FastAPI routes, Pydantic) │
|
||||||
|
│ └────┬─────┘ │
|
||||||
|
│ │ calls │
|
||||||
|
│ ┌────▼─────┐ │
|
||||||
|
│ │services/ │ ← Use case orchestration │
|
||||||
|
│ └──┬────┬──┘ │
|
||||||
|
│ │ │ │
|
||||||
|
│ ┌───▼──┐ ┌▼───────────┐ │
|
||||||
|
│ │domain│ │persistence/ │ │
|
||||||
|
│ │ │ │ │ │
|
||||||
|
│ │bbox │ │ SQLite CRUD │ ← Storage (your blue box) │
|
||||||
|
│ │parse │ │ file store │ │
|
||||||
|
│ └──────┘ └─────────────┘ │
|
||||||
|
│ ↑ pure Python, no deps ↑ aiosqlite │
|
||||||
|
└──────────────────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
Dependencies flow **inward**: `api → services → domain`. The domain layer has zero knowledge of HTTP or database.
|
||||||
|
|
||||||
## Backend — Clean Architecture
|
## Backend — Clean Architecture
|
||||||
|
|
||||||
The backend follows a strict layered architecture. Dependencies flow inward: API → Services → Domain. The domain layer has zero knowledge of HTTP or database.
|
The backend follows a strict layered architecture. Dependencies flow inward: API → Services → Domain. The domain layer has zero knowledge of HTTP or database.
|
||||||
|
|
|
||||||
BIN
docs/images/archi.png
Normal file
BIN
docs/images/archi.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 55 KiB |
|
|
@ -4,6 +4,8 @@ A visual document analysis studio powered by [Docling](https://github.com/DS4SD/
|
||||||
|
|
||||||
Upload a PDF, configure the extraction pipeline, and visualize the results — text, tables, images, formulas, bounding boxes — all from your browser.
|
Upload a PDF, configure the extraction pipeline, and visualize the results — text, tables, images, formulas, bounding boxes — all from your browser.
|
||||||
|
|
||||||
|
{ width="600" }
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
## Features
|
## Features
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue