Extraction fixes:
- OCR normalization: 'Pref erred' → 'Preferred', 'ltem' → 'Item' applied to boundary
scan, Phase 1 questions, and Phase 2 answer key content before AI processing
- Chunk boundary: Phase 1 chunks now capped at (answer_section_start - 1) so no
chunk bleeds into the answer section — (51, 100) becomes (51, 55) for PREP 2013
- Result: Phase 1 gets 2 clean chunks (1-50 and 51-55), Phase 2 gets pages 56-227
Category creation in DocumentDetailPage:
- Replaced window.prompt() with inline input form (more reliable, no browser quirks)
- Fixed option value type: String(c.id) ensures consistent string comparison with
selectedQuestionCategoryId state (prevents type mismatch in controlled select)
- "+ New" button toggles inline form; Enter key or Add button submits
Deletion safety (confirmed):
- Deleting a quiz: questions detached to bank if exclusive, kept if shared — NEVER deleted
- Deleting a question category: questions uncategorized or moved — NEVER deleted
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Two-phase extraction improvements:
- Auto-detect answer section boundary by scanning in 10-page steps for
'Preferred Response:' — finds exact page where questions end and answers begin
(PREP 2013 answers start at page ~68, not at the end of the file)
- Restrict Phase 1 question chunks to pages BEFORE the answer section
- Extract answer key from answer section in CHUNKS (50 pages each) to handle
large answer sections — accumulates all item→letter mappings
- Previous version used last 40% which missed items 1-~135 for PREP 2013
README: full CLI extraction documentation:
- list-sections: find document and section IDs
- extract <section_id> [--bg] [--title] [--mode] [--user]
- jobs / jobs --user <email>
- Explanation of auto-format detection (inline vs separate answer key)
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Two-phase extraction:
- Detects end-of-document answer key format by scanning last 40 pages for
"Preferred Response:" (PREP 2013, 2014 etc use this vs PREP 2012 inline "Correct Answer:")
- Phase 1: Extract questions with item_number field, allow null correct_answer
- Phase 2: Extract answer key (item_number → letter) from last 40% of document
- Phase 3: Match questions to answers by item number, resolve letter → full option text
- Unmatched questions go to skipped list with reason shown in Jobs page
- Standard inline format (PREP 2012) unchanged
Updated extraction prompts:
- item_number field added to all extractions for cross-referencing
- Image content rule: "Item CXXXB" figure references must NOT be treated as new questions
- Recognises both "Correct Answer: X" and "Preferred Response: X"
- ANSWER_KEY_PROMPT: dedicated prompt for extracting answer key tables
Quiz navigation scroll:
- Clicking Next, Previous, or question number now scrolls the question card
into view (smooth scroll to start of question-card div)
Code: extract_questions_no_answers(), extract_answer_key(), _call_model() added to ai_service.py
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Jobs (cross-browser/cross-session):
- POST /quizzes/ stores job_id in Redis under user key (extraction:user_jobs:{uid})
- GET /quizzes/jobs returns all recent jobs for current user from any browser/session
- Navbar JobsBadge polls /quizzes/jobs API every 4s (not localStorage)
- Shows all recent jobs with status badges; links to quiz when complete
- Badge visible even after extraction completes so you can always get back
Mobile navbar fix:
- .navbar .container height was overriding dropdown to 52px (clipping all links)
- Fixed by using .navbar-inner class for the header row only
CLI extract command:
python manage.py list-sections [doc_id] — list docs + sections with IDs
python manage.py extract <section_id> — inline blocking extraction
python manage.py extract <section_id> --bg — background via Celery
python manage.py jobs — show all extraction jobs in Redis
python manage.py jobs --user <email> — filter by user
Quiz delete + question bank:
- When a quiz is deleted, questions that belong ONLY to that quiz are deleted
- Questions shared with other quizzes (via junction) are kept in the bank
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Extraction is now fully async via Celery — UI shows a live progress panel,
job continues even if page is closed. Large documents are processed in
50-page chunks to extract all questions (not just first ~50 pages).
Backend:
- app/tasks/quiz_tasks.py: new Celery task 'extract_quiz'
- Writes step-by-step progress to Redis (extraction:steps:{job_id})
- Splits large page ranges into 50-page chunks, processes each separately
- Reports per-chunk results and running total
- Falls back to synchronous if Celery/Redis unavailable
- POST /quizzes/ now returns {job_id, status:"pending"} immediately
- GET /quizzes/job/{job_id} polls progress: steps[], status, quiz_id on completion
- Celery task list updated to include quiz_tasks
Frontend (DocumentDetailPage):
- ExtractionProgress modal component: monospace step log, auto-scrolls, spinner
- Polls job status every 2 seconds via /quizzes/job/{job_id}
- "Open Quiz →" button appears when done
- "✕ closes — job continues in background" shown while running
- beforeunload warning when job is active (preventing accidental close)
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
- FastAPI backend with JWT auth, roles (admin/moderator/user)
- PDF upload (up to 500MB) with streaming, PyMuPDF text extraction
- ChromaDB vectorization per page with metadata
- LiteLLM AI question extraction from PDF (not generation)
- Image extraction from PDF pages, graceful fallback
- Quiz modes: timed (countdown timer) + learning (answers shown inline)
- Page-by-page question navigation with dot navigator
- TTS endpoint using LiteLLM (Google Vertex / OpenAI voices)
- Admin dashboard: AI model management per task, user role management
- Moderator role: upload PDFs, create sections, generate quizzes
- Spaced repetition reminders via SMTP email (SM-2 intervals)
- APScheduler daily reminder jobs
- Celery + Redis for background PDF processing
- React frontend with all pages
- Docker Compose deployment (nginx + backend + celery + redis)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
