Add live documentaiton and change architecture diagram
This commit is contained in:
Pier-Jean Malandrino
parent
765b3aa4b8
commit
c2aeec3092
5 changed files with 126 additions and 8 deletions
95
.github/workflows/docling-compat.yml
vendored
Normal file
95
.github/workflows/docling-compat.yml
vendored
Normal file
|
|
@ -0,0 +1,95 @@
|
|||
name: Docling compatibility
|
||||
|
||||
# Runs daily and on manual trigger.
|
||||
# Installs the LATEST docling + docling-core (ignoring version pins)
|
||||
# and runs the backend tests. Opens an issue if something breaks.
|
||||
|
||||
on:
|
||||
schedule:
|
||||
- cron: "30 6 * * *" # Every day at 06:30 UTC
|
||||
workflow_dispatch: # Manual trigger from GitHub UI
|
||||
|
||||
jobs:
|
||||
compat:
|
||||
name: Test against latest Docling
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
working-directory: document-parser
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
cache: pip
|
||||
cache-dependency-path: document-parser/requirements.txt
|
||||
|
||||
- name: Install system dependencies
|
||||
run: sudo apt-get update && sudo apt-get install -y --no-install-recommends poppler-utils
|
||||
|
||||
- name: Install pinned dependencies
|
||||
run: |
|
||||
pip install --upgrade pip
|
||||
pip install -r requirements.txt
|
||||
pip install pytest pytest-asyncio httpx
|
||||
|
||||
- name: Upgrade docling to latest
|
||||
id: versions
|
||||
run: |
|
||||
pip install --upgrade docling docling-core
|
||||
echo "docling=$(pip show docling | grep Version | cut -d' ' -f2)" >> "$GITHUB_OUTPUT"
|
||||
echo "docling_core=$(pip show docling-core | grep Version | cut -d' ' -f2)" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Run tests
|
||||
run: pytest tests/ -v
|
||||
|
||||
- name: Open issue on failure
|
||||
if: failure()
|
||||
uses: actions/github-script@v7
|
||||
with:
|
||||
script: |
|
||||
const docling = '${{ steps.versions.outputs.docling }}';
|
||||
const doclingCore = '${{ steps.versions.outputs.docling_core }}';
|
||||
const run = `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
|
||||
|
||||
// Don't open duplicate issues
|
||||
const { data: issues } = await github.rest.issues.listForRepo({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
state: 'open',
|
||||
labels: 'docling-compat',
|
||||
});
|
||||
|
||||
const title = `Docling compatibility break: docling ${docling} / docling-core ${doclingCore}`;
|
||||
|
||||
if (issues.some(i => i.title === title)) {
|
||||
console.log('Issue already exists, skipping.');
|
||||
return;
|
||||
}
|
||||
|
||||
await github.rest.issues.create({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
title,
|
||||
labels: ['docling-compat', 'bug'],
|
||||
body: [
|
||||
`## Docling compatibility failure`,
|
||||
``,
|
||||
`The daily compatibility check failed with the latest Docling releases:`,
|
||||
``,
|
||||
`| Package | Version |`,
|
||||
`|---------|---------|`,
|
||||
`| \`docling\` | \`${docling}\` |`,
|
||||
`| \`docling-core\` | \`${doclingCore}\` |`,
|
||||
``,
|
||||
`**Workflow run:** [View logs](${run})`,
|
||||
``,
|
||||
`### Next steps`,
|
||||
`1. Check the failing tests in the workflow logs`,
|
||||
`2. Identify breaking changes in the Docling changelog`,
|
||||
`3. Update the code and version pins in \`requirements.txt\``,
|
||||
`4. Close this issue once fixed`,
|
||||
].join('\n'),
|
||||
});
|
||||
BIN
docs/Sans-titre-2026-03-10-1116.png
Normal file
BIN
docs/Sans-titre-2026-03-10-1116.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 55 KiB |
|
|
@ -2,17 +2,38 @@
|
|||
|
||||
## Overview
|
||||
|
||||
```
|
||||
┌────────────┐ ┌───────────────────────┐
|
||||
│ Frontend │────────▶│ Document Parser │
|
||||
│ Vue 3 + TS │ /api/* │ FastAPI + Docling │
|
||||
│ port 3000 │ │ SQLite + file storage │
|
||||
└────────────┘ │ port 8000 │
|
||||
└───────────────────────┘
|
||||
```
|
||||
{ width="700" }
|
||||
|
||||
Two services communicating via REST. The frontend is a Vue 3 SPA served by Nginx in production. The backend is a FastAPI app that wraps Docling's document conversion engine.
|
||||
|
||||
### Zooming into the backend
|
||||
|
||||
The schema above shows the macro view. Inside the backend, the code follows a **Clean Architecture** with strict layer boundaries:
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────┐
|
||||
│ Backend │
|
||||
│ │
|
||||
│ ┌──────────┐ │
|
||||
│ │ api/ │ ← HTTP (FastAPI routes, Pydantic) │
|
||||
│ └────┬─────┘ │
|
||||
│ │ calls │
|
||||
│ ┌────▼─────┐ │
|
||||
│ │services/ │ ← Use case orchestration │
|
||||
│ └──┬────┬──┘ │
|
||||
│ │ │ │
|
||||
│ ┌───▼──┐ ┌▼───────────┐ │
|
||||
│ │domain│ │persistence/ │ │
|
||||
│ │ │ │ │ │
|
||||
│ │bbox │ │ SQLite CRUD │ ← Storage (your blue box) │
|
||||
│ │parse │ │ file store │ │
|
||||
│ └──────┘ └─────────────┘ │
|
||||
│ ↑ pure Python, no deps ↑ aiosqlite │
|
||||
└──────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
Dependencies flow **inward**: `api → services → domain`. The domain layer has zero knowledge of HTTP or database.
|
||||
|
||||
## Backend — Clean Architecture
|
||||
|
||||
The backend follows a strict layered architecture. Dependencies flow inward: API → Services → Domain. The domain layer has zero knowledge of HTTP or database.
|
||||
|
|
|
|||
BIN
docs/images/archi.png
Normal file
BIN
docs/images/archi.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 55 KiB |
|
|
@ -4,6 +4,8 @@ A visual document analysis studio powered by [Docling](https://github.com/DS4SD/
|
|||
|
||||
Upload a PDF, configure the extraction pipeline, and visualize the results — text, tables, images, formulas, bounding boxes — all from your browser.
|
||||
|
||||
{ width="600" }
|
||||
|
||||
|
||||
|
||||
## Features
|
||||
|
|
|
|||
Loading…
Reference in a new issue