From ecc4f64ad56256ffa3fceb211166481beffba594 Mon Sep 17 00:00:00 2001 From: Daniel Date: Fri, 3 Apr 2026 20:48:07 +0200 Subject: [PATCH] Update README with embedding config, admin UI, cancellation, and deployment notes Co-Authored-By: Claude Sonnet 4.6 --- README.md | 178 +++++++++++++++++++++++++++++++++--------------------- 1 file changed, 110 insertions(+), 68 deletions(-) diff --git a/README.md b/README.md index 3092567..3c27db3 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# 🩺 PedQuiz +# PedQuiz AI-powered pediatric knowledge quiz platform. Upload PDF study materials, automatically extract MCQ questions with AI, and take quizzes with text-to-speech support and semantic search. @@ -6,12 +6,14 @@ AI-powered pediatric knowledge quiz platform. Upload PDF study materials, automa - **PDF → Quiz**: Upload PREP PDFs, AI extracts questions, answers, and explanations - **Quiz Modes**: Study (instant feedback) and Exam (timed, scored) -- **Text-to-Speech**: OpenAI TTS, AWS Polly, ElevenLabs — voice selection per quiz -- **Semantic Search**: pgvector + AWS Titan Embed — finds questions by meaning, not just keywords +- **Text-to-Speech**: OpenAI TTS, AWS Polly, ElevenLabs, Google Cloud — voice selection per quiz +- **Semantic Search**: pgvector embeddings — finds questions by meaning, not just keywords +- **Embedding Model**: Configurable via Admin UI or env — supports any LiteLLM proxy model or AWS Bedrock +- **Job Cancellation**: Cancel running extractions from the web UI - **Email Verification**: Required before first login; password reset via email - **Role system**: Admin / Moderator / User -- **Nextcloud Integration**: Browse and import PDFs from your Nextcloud in Upload page -- **Themes**: Default and Markdown (GitHub-style) +- **Nextcloud Integration**: Browse and import PDFs from your Nextcloud +- **Themes**: Default and Warm Brown ## Stack @@ -19,12 +21,12 @@ AI-powered pediatric knowledge quiz platform. Upload PDF study materials, automa |---|---| | Frontend | React + React Router, plain CSS, Nginx | | Backend | FastAPI, SQLAlchemy, PostgreSQL 16 + pgvector | -| AI/LLM | LiteLLM proxy (Claude, Gemini, GPT, Bedrock) | -| Embeddings | Configurable via Admin UI or `LITELLM_EMBEDDING_MODEL` env (default: `ge-gemini-embedding-001`) | +| AI/LLM | LiteLLM proxy (Claude, Gemini, GPT, and more) | +| Embeddings | Configurable — any LiteLLM proxy model or direct AWS Bedrock (1024-dim, set via Admin UI or env) | | Document vectors | ChromaDB | -| TTS | OpenAI, AWS Polly, ElevenLabs | +| TTS | OpenAI (direct), AWS Polly, ElevenLabs, Google Cloud TTS | | Queue | Celery + Redis | -| Email | SMTP (smtp2go) | +| Email | SMTP (smtp2go or any SMTP server) | ## Quick Start @@ -52,16 +54,19 @@ SECRET_KEY= # Redis REDIS_URL=redis://redis:6379/0 -# LLM (for question extraction) -LITELLM_MODEL=openai/claude-haiku-4.5 +# LLM — for question extraction (requires LiteLLM proxy or direct OpenAI) +LITELLM_MODEL=openai/claude-haiku-4.5 # prefix with openai/ when using proxy LITELLM_API_KEY= -LITELLM_API_BASE=https://your-litellm-proxy.com # or leave empty for direct OpenAI -LITELLM_EMBEDDING_MODEL=ge-gemini-embedding-001 # model name as proxy knows it, no prefix needed +LITELLM_API_BASE=https://your-litellm-proxy.com # leave empty for direct OpenAI -# OpenAI (for TTS — uses api.openai.com directly, not proxy) +# Embedding model — use the model name exactly as your proxy lists it (no prefix needed) +# Can also be changed live via Admin → More settings without redeploying +LITELLM_EMBEDDING_MODEL=ge-gemini-embedding-001 + +# OpenAI (for TTS — calls api.openai.com directly, not the proxy) OPENAI_API_KEY= -# AWS (for Polly TTS + Bedrock embedding fallback) +# AWS (for Polly TTS + optional direct Bedrock embedding fallback) AWS_ACCESS_KEY_ID= AWS_SECRET_ACCESS_KEY= AWS_REGION=us-east-1 @@ -90,80 +95,102 @@ CHROMA_PERSIST_DIR=/app/chroma_data ## Rebuild & Restart +Frontend and backend are built into Docker images — code changes require a `build` before they take effect. `.env` changes only need a `restart`. + ```bash # Rebuild and restart everything docker compose build && docker compose up -d -# Rebuild and restart a single service (backend, frontend, or celery) +# Rebuild a single service docker compose build backend && docker compose up -d backend docker compose build frontend && docker compose up -d frontend docker compose build celery && docker compose up -d celery -# Restart without rebuilding (picks up .env changes, not code changes) +# Restart without rebuilding (for .env changes only) docker compose restart backend -docker compose restart frontend # View logs docker compose logs backend --tail=50 docker compose logs celery --tail=50 ``` -> **Note:** Frontend and backend are built into Docker images — code changes require a `build` before they take effect. Only `.env` changes and volume-mounted data (uploads, chroma, postgres) are picked up by a plain `restart`. +## Admin Dashboard + +Accessible at `/admin` for admin users. Three tabs: + +### AI Models +- **Search models** from your LiteLLM proxy — click any result to pre-fill the add form +- Configure models per task: `extraction` (PDF → questions), `tts` (voices), `general` +- Set a default model per task; enable/disable individual models +- Extraction models from the proxy don't need an `openai/` prefix — the backend adds it automatically + +### Users +- Create users directly (email auto-verified) +- Change user roles: admin / moderator / user + +### More Settings +- **Public Registration** — enable/disable new user sign-ups +- **AWS Polly** — global enable/disable toggle for all Polly voices (individual voices still manageable in AI Models tab) +- **Embedding Model** — set the model used for semantic search vectors: + - Type a model name and click **Save**, or click **Search LiteLLM** to browse proxy models + - Click **Test** to verify the model works and returns the correct dimensions (must be 1024) + - Setting is stored in Redis and takes effect immediately — no restart needed + - Priority: Admin UI setting → `LITELLM_EMBEDDING_MODEL` env → direct AWS Bedrock fallback + - Model name should match exactly what your proxy lists (no prefix required) + +#### Switching embedding models + +To switch to a different embedding model: +1. Go to Admin → More → click **Search LiteLLM**, find your model, click **Select** +2. Click **Test** to confirm it returns 1024 dimensions +3. Run `docker compose exec backend python manage.py reembed` to regenerate all existing question embeddings with the new model + +To revert to AWS Bedrock (when available): +- Set model to the Bedrock model ID as configured on your proxy, or +- Clear the Redis key to fall back to env: `docker compose exec redis redis-cli DEL settings:embedding_model` +- Then set `LITELLM_EMBEDDING_MODEL=` to empty in `.env` to use direct Bedrock ## CLI Management -All commands run inside the backend container: - ```bash # ── User management ────────────────────────────────────────────────────────── -# Reset a locked-out admin password docker compose exec backend python manage.py reset-password admin@example.com NewPassword123 - -# List all users with email verification status docker compose exec backend python manage.py list-users # ── Quiz extraction ─────────────────────────────────────────────────────────── -# 1. Find your document ID and section ID +# 1. Find document and section IDs docker compose exec backend python manage.py list-sections -# Output example: -# Doc 3: prep-PREP2012.pdf (ready, 767 pages) -# Section 6: 'ALL' pages 1–767 -# Doc 4: prep-PREP2013.pdf (ready, 227 pages) -# Section 7: 'Questions 1-100' pages 1–100 - -# 2a. Extract in background (Celery) — returns immediately, monitor via navbar badge +# 2a. Extract in background (monitor via navbar jobs badge) docker compose exec backend python manage.py extract 6 --bg docker compose exec backend python manage.py extract 6 --bg --title "PREP 2012 Full" --mode timed -# 2b. Extract inline (blocking) — shows live output in terminal +# 2b. Extract inline (blocking, live output in terminal) docker compose exec backend python manage.py extract 6 -# 3. Check job status (shows progress, skipped questions, errors) +# 3. Check job status docker compose exec backend python manage.py jobs docker compose exec backend python manage.py jobs --user admin@example.com # CLI extract options: -# --title "My Quiz" Custom quiz title (default: auto-generated from section name) +# --title "My Quiz" Custom quiz title # --mode timed|learning Quiz mode (default: timed) -# --user email Which user owns the quiz (default: first admin) +# --user email Owner of the quiz (default: first admin) # --bg Run in background via Celery # ── Embeddings ─────────────────────────────────────────────────────────────── -# Regenerate all question embeddings (e.g. after switching embedding model) +# Regenerate all question embeddings (run after switching embedding model) docker compose exec backend python manage.py reembed ``` ### How extraction works -1. **Upload PDF** via the web UI (Upload PDF page) — the system extracts text and stores it -2. **Create a section** on the document page (define page range, e.g. pages 1–767) -3. **Extract quiz** — either from the web UI or CLI: - - The system auto-detects the PDF format: - - **Inline answers** (PREP 2012): "Correct Answer: X" after each question → standard extraction - - **Separate answer key** (PREP 2013): "Preferred Response: X" in a dedicated answer section → two-phase extraction (questions first, then answer key, then matched) +1. **Upload PDF** via the web UI — text is extracted and stored +2. **Create a section** on the document page (define page range) +3. **Extract quiz** from the web UI or CLI: - Large sections are split into 50-page chunks automatically - - Progress shown live in the web UI extraction panel + - Progress shown live in the web UI; running jobs can be cancelled from the Extraction Jobs page + - Supports multiple extraction modes: standard, two-step (separate answer key), questions-only, regex, AI-decides 4. Questions land in the **Question Bank** and can be assigned to categories ## Architecture @@ -175,30 +202,41 @@ Browser Nginx (frontend + API proxy) ├─► React SPA (static) └─► FastAPI backend - ├─ PostgreSQL (pgvector) ← users, quizzes, questions + embeddings - ├─ ChromaDB ← document page chunks for quiz generation - ├─ Redis ← Celery task queue + ├─ PostgreSQL (pgvector) ← users, quizzes, questions + 1024-dim embeddings + ├─ ChromaDB ← document page chunks for semantic search + ├─ Redis ← Celery queue + runtime settings (embedding model, toggles) ├─ Celery workers ← background PDF processing, emails - ├─ LiteLLM proxy ← Claude/Gemini/GPT for extraction + embeddings - ├─ AWS Bedrock ← Polly TTS, Titan embed fallback + ├─ LiteLLM proxy ← Claude/Gemini/GPT for extraction; embedding models + ├─ AWS Bedrock ← Polly TTS; embedding fallback └─ OpenAI ← TTS (direct, not via proxy) ``` +### Embeddings + +Question embeddings are 1024-dimensional vectors stored in PostgreSQL via pgvector with an HNSW index for fast cosine similarity search. The embedding model is called **directly via the LiteLLM proxy HTTP API** (not the Python library) so the `dimensions=1024` parameter is forwarded correctly. + +Priority chain: +1. Model set in Admin UI (stored in Redis `settings:embedding_model`) +2. `LITELLM_EMBEDDING_MODEL` env var +3. Direct AWS Bedrock (`amazon.titan-embed-text-v2:0`) — automatic fallback if AWS credentials are present + ### Search -Quiz search uses **hybrid retrieval**: -1. **Semantic** — embed the query with Titan Embed V2, cosine similarity against all questions via pgvector HNSW index +Quiz search uses hybrid retrieval: +1. **Semantic** — embed the query, cosine similarity via pgvector HNSW index 2. **Keyword** — PostgreSQL `ILIKE` on question text and options -3. Results merged and ranked — semantic matches shown first by score, keyword-only matches appended +3. Results merged — semantic matches ranked first by score, keyword-only matches appended ### TTS Providers -| Provider | Model IDs | Key Needed | +| Provider | Model ID format | Key needed | |---|---|---| -| OpenAI | `tts-1:alloy`, `tts-1:nova`, `tts-1:echo`, `tts-1:shimmer`, `tts-1:onyx`, `tts-1:fable`, `tts-1-hd:*` | `OPENAI_API_KEY` | +| OpenAI | `tts-1:alloy`, `tts-1:nova`, `tts-1:echo`, `tts-1:shimmer`, `tts-1:onyx`, `tts-1:fable`, `tts-1-hd:*` | `OPENAI_API_KEY` (calls api.openai.com directly) | | AWS Polly | `polly/Joanna`, `polly/Matthew`, `polly/Amy`, `polly/Brian` | `AWS_ACCESS_KEY_ID` + `polly:SynthesizeSpeech` IAM permission | | ElevenLabs | `elevenlabs/` | `ELEVENLABS_API_KEY` | -| Google Cloud | `google/` | `GOOGLE_TTS_API_KEY` | +| Google Cloud | `google/` e.g. `google/en-US-Wavenet-D` | `GOOGLE_TTS_API_KEY` | + +AWS Polly can be globally disabled/enabled via Admin → More settings without removing the individual voice configs. ## Project Structure @@ -206,23 +244,26 @@ Quiz search uses **hybrid retrieval**: ├── backend/ │ ├── app/ │ │ ├── main.py # App startup, seeding, pgvector setup -│ │ ├── config.py # Settings (pydantic-settings) -│ │ ├── models/ # SQLAlchemy models +│ │ ├── config.py # Settings (pydantic-settings, reads .env) +│ │ ├── models/ # SQLAlchemy ORM models │ │ ├── routers/ # FastAPI route handlers │ │ ├── services/ -│ │ │ ├── ai_service.py # LLM extraction + TTS -│ │ │ ├── embedding_service.py # pgvector embeddings -│ │ │ ├── vector_service.py # ChromaDB (document pages) -│ │ │ ├── quiz_service.py # Quiz creation pipeline -│ │ │ └── email_service.py # Email templates + sending +│ │ │ ├── ai_service.py # LLM extraction + TTS routing +│ │ │ ├── embedding_service.py # pgvector embeddings (httpx → proxy) +│ │ │ ├── vector_service.py # ChromaDB document page chunks +│ │ │ ├── quiz_service.py # Quiz creation pipeline +│ │ │ └── email_service.py # Email templates + sending +│ │ ├── tasks/ +│ │ │ ├── quiz_tasks.py # Celery: PDF extraction with cancellation support +│ │ │ └── pdf_tasks.py # Celery: PDF text extraction │ │ └── utils/ │ ├── manage.py # CLI: reset-password, list-users, reembed │ ├── requirements.txt │ └── Dockerfile ├── frontend/ │ ├── src/ -│ │ ├── pages/ # Dashboard, Quiz, Results, Settings, Search … -│ │ ├── components/ # Navbar, LineChart +│ │ ├── pages/ # All page components +│ │ ├── components/ # Navbar (with jobs badge), shared UI │ │ └── context/ # AuthContext, ThemeContext │ ├── nginx.conf │ └── Dockerfile @@ -231,7 +272,8 @@ Quiz search uses **hybrid retrieval**: ## Deployment Notes -- The frontend Nginx only binds to `127.0.0.1:8081` — put a reverse proxy (Caddy/Nginx) in front for HTTPS -- PostgreSQL data is persisted in the `postgres_data` Docker volume — back it up regularly -- Uploads live in `uploads_data` volume — includes extracted question images -- Set `APP_URL` to your public domain so verification/reset email links work +- The frontend Nginx binds to `127.0.0.1:8081` — put Caddy or Nginx in front for HTTPS +- PostgreSQL data persists in the `postgres_data` Docker volume — back it up regularly +- Uploads live in the `uploads_data` volume — includes extracted question images +- Redis data persists in `redis_data` volume — holds runtime settings and job state +- Set `APP_URL` to your public domain so email verification and password reset links work