Adapt documentation and fix containerisation
This commit is contained in:
parent
0d833771ff
commit
822c7dd06f
5 changed files with 108 additions and 6 deletions
|
|
@ -1,4 +1,6 @@
|
||||||
FROM python:3.12
|
FROM python:3.12
|
||||||
|
# Install poppler for PDF processing
|
||||||
|
RUN apt-get update && apt-get install -y poppler-utils && rm -rf /var/lib/apt/lists/*
|
||||||
WORKDIR /app
|
WORKDIR /app
|
||||||
COPY backend/requirements.txt ./backend/requirements.txt
|
COPY backend/requirements.txt ./backend/requirements.txt
|
||||||
RUN pip install --no-cache-dir -r backend/requirements.txt
|
RUN pip install --no-cache-dir -r backend/requirements.txt
|
||||||
|
|
|
||||||
103
README.md
103
README.md
|
|
@ -1,9 +1,108 @@
|
||||||
# DocTags Analyzer and Visualizer
|
# DocTags Analyzer and Visualizer
|
||||||
|
|
||||||
Simple command to process PDF pages with DocTags:
|
AI-powered document analysis and visualization tool for extracting structured content from PDFs.
|
||||||
|
|
||||||
|
## 🚀 Quick Start with Docker
|
||||||
|
|
||||||
|
### Prerequisites
|
||||||
|
- Docker and Docker Compose installed
|
||||||
|
- At least 4GB of free memory
|
||||||
|
- ~500MB disk space for the AI model
|
||||||
|
|
||||||
|
### Running with Docker
|
||||||
|
|
||||||
|
1. **Clone the repository**
|
||||||
|
```bash
|
||||||
|
git clone <repository-url>
|
||||||
|
cd doc-analyzer
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Place your PDF files in the project directory**
|
||||||
|
```bash
|
||||||
|
cp /path/to/your/document.pdf ./
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Start the application**
|
||||||
|
```bash
|
||||||
|
docker-compose up -d --build
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **Access the web interface**
|
||||||
|
- Open http://localhost:8080 in your browser
|
||||||
|
- Select a PDF from the dropdown
|
||||||
|
- Process your documents through the three-step workflow
|
||||||
|
|
||||||
|
### First Run Notice
|
||||||
|
⚠️ **Important**: The first analysis will take 5-10 minutes as the AI model (SmolDocling-256M) needs to be downloaded (~500MB). Subsequent runs will be much faster (30-60 seconds).
|
||||||
|
|
||||||
|
## 📋 Features
|
||||||
|
|
||||||
|
- **Document Analysis**: Extract comprehensive document structure using AI
|
||||||
|
- **Visualization**: Generate visual overlays showing document elements
|
||||||
|
- **Image Extraction**: Automatically extract and catalog embedded images
|
||||||
|
- **Web Interface**: User-friendly interface for document processing
|
||||||
|
|
||||||
|
## 🛠️ Manual Usage
|
||||||
|
|
||||||
|
Process PDF pages with DocTags:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python analyzer.py --image document.pdf --page 8 && python visualizer.py --doctags results/output.doctags.txt --pdf document.pdf --page 8 --adjust && python picture_extractor.py --doctags results/output.doctags.txt --pdf document.pdf --page 8 --adjust
|
python analyzer.py --image document.pdf --page 8 && python visualizer.py --doctags results/output.doctags.txt --pdf document.pdf --page 8 --adjust && python picture_extractor.py --doctags results/output.doctags.txt --pdf document.pdf --page 8 --adjust
|
||||||
```
|
```
|
||||||
|
|
||||||
All output files will be automatically stored in the `results` folder.
|
## 🐛 Troubleshooting
|
||||||
|
|
||||||
|
### Docker Issues
|
||||||
|
|
||||||
|
1. **Container won't start**
|
||||||
|
- Check logs: `docker-compose logs analyser`
|
||||||
|
- Ensure ports aren't in use: `lsof -i :8080`
|
||||||
|
|
||||||
|
2. **"No module named 'docling_core'" error**
|
||||||
|
- Rebuild the container: `docker-compose down && docker-compose up -d --build`
|
||||||
|
|
||||||
|
3. **Analysis stuck on "Running..."**
|
||||||
|
- First run downloads the AI model (~500MB), this can take 5-10 minutes
|
||||||
|
- Check progress: `docker-compose exec analyser du -sh /root/.cache/huggingface/`
|
||||||
|
- Monitor CPU usage: `docker-compose exec analyser ps aux | grep analyzer`
|
||||||
|
|
||||||
|
4. **PDF not loading**
|
||||||
|
- Ensure poppler is installed (already included in Dockerfile)
|
||||||
|
- Place PDFs in the project root directory
|
||||||
|
- PDFs must have `.pdf` extension
|
||||||
|
|
||||||
|
### Performance Tips
|
||||||
|
|
||||||
|
- First analysis is slow due to model download
|
||||||
|
- Subsequent analyses are much faster (model is cached)
|
||||||
|
- Processing time depends on PDF complexity and page size
|
||||||
|
- Monitor memory usage: `docker-compose exec analyser free -h`
|
||||||
|
|
||||||
|
## 📁 Project Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
doc-analyzer/
|
||||||
|
├── backend/
|
||||||
|
│ ├── page_treatment/ # Core processing scripts
|
||||||
|
│ │ ├── analyzer.py # AI-powered document analysis
|
||||||
|
│ │ ├── visualizer.py # Visualization generator
|
||||||
|
│ │ └── picture_extractor.py # Image extraction
|
||||||
|
│ ├── app.py # Flask web application
|
||||||
|
│ └── requirements.txt # Python dependencies
|
||||||
|
├── frontend/ # Web interface
|
||||||
|
├── results/ # Output directory (auto-created)
|
||||||
|
├── Dockerfile # Docker configuration
|
||||||
|
└── docker-compose.yml # Docker Compose setup
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🔧 Development
|
||||||
|
|
||||||
|
To modify the application:
|
||||||
|
|
||||||
|
1. Make changes to the code
|
||||||
|
2. Rebuild the Docker image: `docker-compose up -d --build`
|
||||||
|
3. Check logs for errors: `docker-compose logs -f analyser`
|
||||||
|
|
||||||
|
## 📄 License
|
||||||
|
|
||||||
|
This project is open source and available under the MIT License.
|
||||||
|
|
@ -12,7 +12,7 @@ APP_VERSION = "1.0.0"
|
||||||
DEBUG = os.environ.get('DEBUG', 'False').lower() == 'true'
|
DEBUG = os.environ.get('DEBUG', 'False').lower() == 'true'
|
||||||
|
|
||||||
# Server settings
|
# Server settings
|
||||||
HOST = '127.0.0.1'
|
HOST = '0.0.0.0'
|
||||||
PORT = 5000
|
PORT = 5000
|
||||||
MAX_CONTENT_LENGTH = 100 * 1024 * 1024 # 100MB
|
MAX_CONTENT_LENGTH = 100 * 1024 * 1024 # 100MB
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -6,3 +6,4 @@ pdf2image
|
||||||
pillow
|
pillow
|
||||||
requests
|
requests
|
||||||
flask
|
flask
|
||||||
|
docling_core
|
||||||
|
|
@ -1,7 +1,7 @@
|
||||||
services:
|
services:
|
||||||
analyser:
|
analyser:
|
||||||
image: ddd
|
build: .
|
||||||
ports:
|
ports:
|
||||||
- "8080:5000"
|
- "8080:5000"
|
||||||
volumes:
|
volumes:
|
||||||
- ./input:/
|
- ./:/app
|
||||||
Loading…
Reference in a new issue