Commit graph

58 commits

Author SHA1 Message Date
Daniel
9f97218f39 Add persistent MyNote for quizzes
All checks were successful
Mobile Android Release / android-release (push) Successful in 1m32s
2026-05-12 17:51:23 +02:00
Daniel
fdebda993c Improve quiz TTS and Android release build
Some checks failed
Mobile Android Release / android-release (push) Failing after 1s
2026-05-12 01:01:01 +02:00
Daniel
d59c8bed6f Remove coach modes and improve quiz audio 2026-05-10 01:24:06 +02:00
Daniel
2a0dd56f95 Add ai_answer extraction + broader ai_decide sampling + flashcard title edit
ai_decide now samples 4 points across the section (start, 1/3, 2/3, end)
instead of just the first 30 + last 20 pages. This gives accurate strategy
detection on large documents where the answer format might be deeper in.

New ai_answer extraction mode:
- Extracts questions from Q&A-format PDFs that have no answer key
- AI picks the correct option from each question's choices
- Generates explanation using document context + medical knowledge
- Useful for PDFs like practice tests where answers were never included
- Available manually and as an ai_decide strategy

Flashcard decks can now be renamed:
- PATCH /flashcards/{deck_id} updates title
- Inline edit on FlashcardsPage with responsive layout (input full-width,
  buttons wrap under it so Cancel never overflows the card)
- Title truncates with ellipsis when not editing

Note: generate mode (textbook -> MCQs) is unchanged per user request.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-19 20:17:49 +02:00
Daniel
5398342e3d Suspend pauses timer; hide timer-expired attempts from history
Suspend now pauses the timer instead of letting it run out:
- 'Suspend & Leave' sends suspended=true with time_left to backend
- On resume, backend re-anchors started_at to now with held time_left
- Closing tab without suspending continues to run the timer (unchanged)

Timer-expired auto-submits are marked with expired=1 and excluded from:
- Attempt history (GET /attempts/history)
- Dashboard stats (quiz count, total attempts, average score)
- Attempt list (GET /attempts)
- DDL: ALTER TABLE quiz_attempts ADD COLUMN expired INTEGER DEFAULT 0

Course-quiz decoupling is preserved — these changes only touch
non-course quizzes (Quiz.course_id IS NULL).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-19 19:27:46 +02:00
Daniel
56fdc57389 fix: gateway-agnostic URL handling for TTS and embeddings, docs cleanup
- Fix double /v1 in TTS audio/speech URL when LITELLM_API_BASE includes /v1
- Fix double /v1 in embedding service and vector service URLs
- Clean up docs: remove second-person language in deployment, frontend, migrations

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-19 02:17:35 +02:00
Daniel
0f3a267fb6 Wire Alembic to live Postgres and fix startup DDL race
- alembic.ini: remove hardcoded sqlite URL
- alembic/env.py: inject DATABASE_URL from container env
- main.py: serialize Base.metadata.create_all() + setup_pgvector() behind
  a Postgres advisory lock (_run_startup_ddl). Previously all 4 uvicorn
  workers ran the DDL in parallel and occasionally deadlocked each other
  on ALTER TABLE ordering, killing one worker at startup.
- CLAUDE.md: add Database migrations (Alembic) section

DB was stamped at 9bac7bf02e38; no schema changes in this commit.
2026-04-14 04:49:23 +02:00
Daniel
b39e09f393 Persist reminders_disabled in Postgres and harden reminder scheduler
- Add reminders_disabled boolean column to users (DB source of truth).
- Scheduler reads the DB column directly; Redis no longer used for
  opt-out checks.
- Scheduler now deactivates reminders when a user has no completed
  attempts left for a quiz (e.g., after deleting their attempts).
- Settings API: GET returns DB value for reminders_disabled; PUT
  persists that key to DB and keeps the rest of the blob in Redis.

Rollback point for Alembic wiring.
2026-04-14 04:31:03 +02:00
Daniel
d0518d0737 Add comprehensive structured logging with Loki + Grafana
Backend logging:
- Centralized JSON logging config with LOG_LEVEL env var
- Request logging middleware: user, method, path, status, duration, request_id
- Fixed all 9 silent except:pass blocks to log warnings with tracebacks
- Celery workers use same structured JSON format

Infrastructure:
- Loki 3.3.2 for log storage (30-day retention)
- Promtail 3.3.2 for Docker container log shipping
- Grafana 10.3.1 with auto-provisioned Loki datasource
- Grafana on port 3002 (admin/pedshub_grafana)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 02:53:54 +02:00
Daniel
d44ad883e9 Add SCORM data persistence across sessions
SCORM runtime data (bookmarks, scores, lesson position) now saved to
Redis via GET/PUT /courses/{id}/lessons/{id}/scorm/data endpoints.
Data loads before iframe renders and saves on Commit/Finish/Terminate.
90-day TTL on stored data.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 01:53:53 +02:00
Daniel
db98c6ae69 Add certificate generation, SCORM support, QTI export/import
- Certificate: PDF generated on course completion, download from course page
- SCORM: upload ZIP packages as lesson type, served in iframe, manifest parsed
- QTI 2.1: export selected/all questions as XML, import QTI files into bank
- Uses fpdf2 for certificate PDF generation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 01:30:09 +02:00
Daniel
699cbabfcb Add OIDC/SSO login, reminder fixes, user notification settings
- Add generic OIDC/SSO support (configurable via env vars)
- Admin can enable SSO-only mode (disables password login)
- SSO callback auto-creates and verifies users
- Login page shows SSO button when configured, hides password form in SSO-only mode
- Fix reminders: skip course quizzes and deleted quizzes
- Don't create reminders for course quiz attempts
- Add user reminder opt-out toggle in Settings > Notifications
- Scheduler checks user opt-out before sending emails

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 04:10:15 +02:00
Daniel
695392f022 Remove HIBP breached password warnings
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 02:24:24 +02:00
Daniel
197181f6f9 Fix CLI argument handling for all commands
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 20:57:58 +02:00
Daniel
affd2fc116 Add admin guide, document CLI commands, consolidate account into settings
- Add list-docs, fix-stuck-docs, reprocess-doc CLI commands
- Create ADMIN.md with deployment, CLI, database, and troubleshooting docs
- Add HIBP warning to Settings page password change
- Remove separate Account nav link (already in Settings)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 20:39:58 +02:00
Daniel
8137d5b20c Decouple course quizzes, replace passlib, add security hardening
- Fully decouple course quizzes from main quiz system (hidden from
  dashboard stats, history, search, results page)
- Course quiz results show "Back to Course" instead of retake/delete
- Add allow_review toggle for course creators to control answer review
- Show quiz title on course page, hide pool size from students
- Add course thumbnails to browse cards
- Replace passlib with bcrypt directly (compatible with existing hashes)
- Add HIBP breached password warnings on register/reset/change password
- Add CLI management tools (reset-password, set-role, stats, etc.)
- Fix quiz PATCH endpoint: ownership check instead of moderator-only
- Add max_length validation on course/module/lesson titles
- Fix score display bug on results page (0 of N when review disabled)
- Fix question count on course quiz start (show per-attempt, not pool)
- Improve suspend warning for timed course quizzes with max attempts
- Clean up validation error messages (show "Invalid email" not Pydantic dump)
- Add DDL migration for allow_review column

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 21:31:05 +02:00
Daniel
bc1a88a954 Fix course editor: field mismatches, TinyMCE, AI without save
Backend fixes:
- Remove nonexistent fields: category, thumbnail_url, file_path, sort_order
- Map to actual model fields: thumbnail_path, local_file_path, position
- Add standalone AI content endpoint (no lesson save required)
- Add missing lesson fields to course detail response

Frontend fixes:
- Fix lesson_type (was "type"), live_session_url (was "live_url")
- Fix lesson save/delete URLs to match backend routes
- Add TinyMCE rich text editor for text lessons (self-hosted, GPL)
- AI Generate works before saving lesson (uses course-level endpoint)
- Proper input styling: larger padding, border-radius, consistent sizing
- Module title input properly styled

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 02:49:43 +02:00
Daniel
594bbfaf26 Fix course router: remove nonexistent model fields
- Remove category, thumbnail_url references (not on Course model)
- Fix sort_order → position (matching actual column name)
- Fix CourseCreate schema: only title + description
- Fix tab labels: "My Created" → "Created by Me"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 02:26:41 +02:00
Daniel
a2a2da7da0 Add LMS, user quiz creation, manual questions, course system
LMS / Course System:
- Course → Module → Lesson hierarchy (any user can create)
- Lesson types: text, video (Vimeo/YouTube/local), document, quiz, live_session
- Video provider auto-detection from URL
- BBB API integration (create/join meetings) with env config
- Course enrollment with per-lesson progress tracking
- AI content generation/refinement for text lessons
- Draft/published/archived status workflow
- Subscription gate placeholder (requires_subscription flag)
- Thumbnail upload support
- Module/lesson reorder with up/down controls

User Quiz Creation:
- Any user can create quizzes from question bank (was moderator-only)
- User quizzes: is_published=0, is_shared=0 (private by default)
- Fixed section_id fallback: None instead of hardcoded 1

Manual Question Creation:
- POST /questions/create endpoint for manual MCQ entry
- CreateQuestionModal on QuestionBankPage with options, radio for correct answer
- Auto-generates embedding on creation

Frontend:
- CoursesPage: Browse/My Courses/Created tabs with search and pagination
- CourseDetailPage: Student view with module accordion, lesson viewer, progress
- CourseEditorPage: Full course builder with AI generate, question bank browser
- Courses link in Navbar
- Create Question button on Question Bank (available to all users)

Backend:
- 5 new tables: courses, course_modules, course_lessons, course_enrollments, course_lesson_progress
- Course model + schemas + router (22 endpoints)
- BBB_SERVER_URL + BBB_SECRET config
- Updated CLAUDE.md with LMS documentation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 02:07:04 +02:00
Daniel
be7efd73d2 Fix shared decks route ordering, fix corrupted middot character
- Move GET /shared before GET /{deck_id} so FastAPI matches it correctly
  (was being caught by /{deck_id} and failing int parse on "shared")
- Remove duplicate /shared endpoint left after move
- Fix corrupted middot character (U+FFFD) in My Decks card count display

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 23:45:13 +02:00
Daniel
18d0a74348 Add deck sharing, star ratings, and shared deck browsing
- Users can share/unshare their flashcard decks (toggle button)
- Shared tab: paginated list of all shared decks sorted by avg rating
- Star rating system (1-5): click stars to rate, updates average live
- Each user can rate each shared deck once (upsert on re-rate)
- Admin can force-unshare any deck (remove from shared listing)
- Shared decks are study-able by anyone (read-only access)
- Owner name displayed on shared deck cards
- Load more button for shared decks (12 per page)
- New DB table: flashcard_deck_ratings (user_id+deck_id unique)
- New column: flashcard_decks.is_shared (default 0)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 23:37:40 +02:00
Daniel
800de50fc9 Flashcard ownership, trash, delete cards, fix job status
- Fix: flashcard task used wrong Redis key prefix (job: vs extraction:)
  causing progress to stay on "pending" after completion
- Flashcards are now user-scoped: each user sees only their own decks
- Soft-delete decks: DELETE moves to trash, ?permanent=true to destroy
- Trash tab on flashcards page: restore or permanently delete decks
- Delete individual cards with inline confirmation in browse view
- Card edit/delete now allowed for deck owner (not just moderators)
- ExtractionProgress label prop: shows "Generating Flashcards" not
  "Extracting Questions" for flashcard jobs
- Added deleted_at column to flashcard_decks

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 23:24:48 +02:00
Daniel
3e37bf2128 Add AI-powered flashcard generation system
New feature: generate flashcards from PDF sections using AI, completely
separate from the existing quiz system.

Backend:
- FlashcardDeck + Flashcard models with cascade deletes
- flashcard_tag_links table for tag classification (reuses question_tags)
- /api/flashcards/ router: CRUD for decks, browse/search cards, tag filtering
- generate_flashcard_deck Celery task with chunked processing + progress
- FLASHCARD_PROMPT in extraction_modes.py (15 cards per chunk)
- "flashcard" added to admin model task types

Frontend:
- FlashcardsPage: deck grid + card browser with search/filter
- FlashcardStudyPage: flip cards, mark known/review, keyboard nav,
  shuffle, progress bar, completion screen
- DocumentDetailPage: "Create Flashcards" button alongside "Extract Quiz"
- Navbar: Flashcards link
- AdminPage: flashcard in model task dropdown

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 22:59:21 +02:00
Daniel
2cbbfe00c3 Tag filtering, multi-category, bug fixes, image validation, docs
- Fix tag filtering (sa_text import shadowing caused UnboundLocalError)
- Add TagBrowser component with per-section search
- Multi-category selection (OR within categories, AND with tags)
- AI image validation: has_figure field in extraction prompt
- Skip known branding images by MD5 hash + dimension filters
- Fix quiz timer auto-submit (wrong useEffect dependency)
- Fix QuizResponse schema: section_id nullable
- Fix Question.quiz_id → source_quiz_id attribute name
- Fix SQL injection in quizzes.py vector search
- Add PDF processing progress steps via Redis
- Add delete user from admin panel
- Admin page: no spinner flash on data refresh
- Upload progress: axios 1.x e.progress, remove manual Content-Type
- Duplicate model error: 409 with clear message
- Backend startup: retry DDL migration on lock timeout
- Replace all silent except:pass with warning logs
- Comprehensive multi-page documentation (docs/)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 22:48:26 +02:00
Daniel
10b9b1b7cf Rebrand to PedsHub, fix CSP for fonts and Turnstile
- index.html: title → PedsHub, icon → 🏥, add meta description
- nginx.conf: CSP extended for Google Fonts, Cloudflare Turnstile (script/connect/frame/img)
- Navbar, LoginPage, ForgotPassword, ResetPassword, email_service, main.py: PedQuiz → PedsHub, 🩺🏥

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 01:56:53 +02:00
Daniel
5403d78a3e Fix landing page routing, add logged-in nav integration
- App.jsx: simplify to single Routes block with Outlet layout pattern — fixes 404
- App.jsx: /home always accessible regardless of auth state
- LandingPage: show Dashboard/Quizzes/Question Bank nav links + Go to App CTA when logged in
- LandingPage: hero shows app navigation buttons when signed in
- LandingPage: fix apostrophe JSX syntax error in textarea placeholder

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 01:41:39 +02:00
Daniel
ee47fb07a2 Add PedsHub landing page with contact form and AI Scribe section
Frontend:
- LandingPage.jsx: hero, 6 feature cards, AI Scribe section (links peds.danvics.com), contact form
- Contact form: name/email/type toggle (question vs moderator app), Turnstile anti-bot (optional)
- App.jsx: /home route for unauthenticated users, auth pages reachable pre-login, logged-in /home → /

Backend:
- contact.py: POST /api/contact (public), stores in DB, emails admin; GET /submissions (admin)
- main.py: create contact_submissions table, register contact router
- main.py: add pedshub.com + www.pedshub.com to CORS origins
- config.py: TURNSTILE_SECRET_KEY, ADMIN_EMAIL settings

Env vars to set:
  VITE_TURNSTILE_SITE_KEY=<from cloudflare.com/turnstile>  (frontend)
  TURNSTILE_SECRET_KEY=<secret>  (backend .env)
  ADMIN_EMAIL=you@example.com  (backend .env)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 01:34:44 +02:00
Daniel
4cbaa529f0 Workers x4, follow-up suggestions, singleton lock, admin error details
- docker-compose.yml: --workers 4 on uvicorn
- main.py: Redis singleton lock so only one worker runs scheduler + backfill (prevents 4x cron jobs)
- teach.py: system prompt requests 3 follow-up suggestions ("> " prefix format)
- teach.py: parse and return suggestions[] separately from reply
- TeachChat.jsx: render clickable suggestion chips after last assistant message (click fills input)
- admin.py: restore full error details for admin endpoints (ElevenLabs, Polly, LiteLLM)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 01:27:03 +02:00
Daniel
4c48a5cf94 Security hardening, async TeachChat, rate limit UX, unthrottle
Security:
- admin.py: move api_key from URL query params to POST body (litellm/models, tts/voices) — prevents key logging
- admin.py: sanitize exception messages in voice discovery — log internally, return generic errors
- teach.py: log LLM errors server-side, show friendly message to user
- nextcloud.py: normalize path with posixpath.normpath to prevent ../ traversal
- auth.py: check_rate_limit now accepts user param — admins/moderators/unthrottled always exempt

Performance:
- teach.py: make /chat endpoint async, use litellm.acompletion() — no longer blocks a uvicorn thread per request

Features:
- users: add is_unthrottled column (DB migration in setup_pgvector)
- admin.py: PUT /users/{id}/unthrottle endpoint
- AdminPage.jsx: Unlimited/Throttle toggle button per user, shows "unlimited" badge
- Rate limit messages improved with user-friendly context

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 01:09:42 +02:00
Daniel
7f1f14537b Security: fix SQL injection, add rate limiting, markdown in TeachChat
- teach.py, questions.py: replace f-string SQL with parameterized CAST(:vec AS vector) queries
- auth.py: add reusable check_rate_limit() Redis helper
- teach.py: rate limit /chat to 30 req/10min per user
- tts.py: rate limit /speak to 60 req/hr per user
- teach.py: stronger system prompt — no clarifying questions, use markdown, answer directly
- TeachChat.jsx: render assistant messages with ReactMarkdown (already in package.json)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 00:45:28 +02:00
Daniel
975a31fb01 TeachChat: model selector, proxy routing fix, titan seed cleanup
- teach.py: use _proxy_model() + pass api_key/api_base from settings (fixes LiteLLM provider error for openrouter/bedrock models)
- teach.py: accept model_id in ChatRequest so frontend can select model
- main.py: remove titan-embed-v2 from general seed, auto-delete legacy entry on startup
- main.py: kill stale idle-in-transaction DB connections at startup to prevent DDL lock hangs
- main.py: set lock_timeout=10s on DDL connection as fast-fail safety net
- TeachChat.jsx: fetch /teach/models, show selector dropdown in header when >1 model available

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 00:37:35 +02:00
Daniel
12d99d3609 Add switchable embedding model, Polly toggle, job cancellation, and UI fixes
Embedding:
- Embedding model now configurable via Admin UI (More tab) or LITELLM_EMBEDDING_MODEL env
- Calls LiteLLM proxy directly via httpx (bypasses LiteLLM library param validation)
- Passes dimensions=1024 to proxy; Redis setting overrides env var
- Default model: ge-gemini-embedding-001 (Gemini AI Studio, 1024-dim)
- Test button in admin UI to verify model works
- Fixed vector_service to use httpx + Redis model (was broken with non-prefixed model names)

Polly:
- Global enable/disable toggle in Admin → More settings (stored in Redis)
- /tts/voices filters out polly/* when disabled
- /tts/speak rejects polly requests when disabled

Job cancellation:
- POST /quizzes/job/{job_id}/cancel endpoint
- Cancel button on JobsPage for running jobs
- Celery task checks Redis status at each chunk boundary and exits cleanly
- Fixes DB lock on restart caused by cancelled jobs leaving open transactions

Admin UI:
- Settings tab renamed to "More" (heading: More Settings)
- Model row overflow fixed (minWidth: 0 + ellipsis on model_id)
- Embedding model search shows all proxy models (no auto-filter by "embed")
- Navbar correctly excludes cancelled/failed jobs from "extracting" count

README:
- Added Rebuild & Restart section with commands
- Updated embedding model reference

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-03 20:44:11 +02:00
Daniel
c94640c890 CRITICAL: Remove cascade delete from Section→Quiz relationship
Root cause of lost PREP 2016 quiz: deleting a section (e.g. to recreate
with different page ranges) cascade-deleted ALL quizzes attached to it,
destroying the quiz AND all its questions permanently.

Fixes:
- Section.quizzes relationship: removed cascade="all, delete-orphan"
  Deleting a section no longer touches quizzes at all
- Quiz.section_id FK: changed to ON DELETE SET NULL, nullable=True
  If a section is deleted, quizzes keep working (section_id becomes null)
- DB migration: drops old FK constraint and recreates with SET NULL

Also answers the question about deleting a bank question in a quiz:
- quiz_question_links FK has ON DELETE CASCADE — deleting a question
  silently removes it from all quizzes
- This needs a warning UI (not yet implemented — no delete button in bank)

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 18:26:22 +02:00
Daniel
24fb00aef2 Fix jobs showing double: lpush was in both router AND Celery task
The job ID was pushed to extraction:user_jobs:{uid} twice:
1. quizzes.py router when dispatching the job
2. quiz_tasks.py Celery task when it starts running

Removed the push from the Celery task. Router handles it.
Also cleaned 13 duplicate entries from Redis.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 18:00:20 +02:00
Daniel
1343eb5801 Fix mobile touch, hide docs for users, timer save/restore, Nextcloud server-side
Mobile touch fix:
- Added touch-action: manipulation and -webkit-tap-highlight-color: transparent
  to .question-card .option CSS — prevents mobile browsers from interfering
  with tap events (double-tap zoom, highlight flash, delayed clicks)

Documents hidden for regular users:
- Dashboard: stats card hides Documents count for non-moderators
- Dashboard: document list section completely hidden for non-moderators
- Documents API not called for non-moderators (saves a request)

Timer improvements:
- Custom timer input on ModeSelectScreen — set any number of minutes
  for exam mode (overrides the quiz default, or add time to untimed quizzes)
- timeLeft saved to Redis progress every 1.5s (alongside answers, mode, voice)
- On resume: timer restarts from the saved remaining time
- Timer auto-submits when it reaches 0 (already worked, now also works on resume)
- Progress bar shows remaining time relative to what was left at resume

Nextcloud settings — server-side storage:
- GET /auth/me/settings — load user settings from Redis
- PUT /auth/me/settings — save settings to Redis (linked to user_id)
- SettingsPage loads Nextcloud config from server on mount
- Saves to BOTH server (cross-browser persistent) and localStorage (for UploadPage)
- Clear function wipes both server and localStorage
- Switching browsers and logging in restores Nextcloud settings

Also fixed: saved.attemptId → saved.attempt_id in ModeSelectScreen resume check

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 17:39:43 +02:00
Daniel
365aaf3df0 Fix option selection, submit failure, and slow load
Root causes:
1. joinedload(Quiz.questions) was SLOWER (129ms vs 75ms) with the custom
   secondary relationship — removed, back to lazy load
2. Redis progress could have stale attempt_id from a completed submission:
   GET /attempts/progress now validates the attempt exists and is not completed
   before returning; stale entries are auto-deleted
   POST /attempts/{id}/submit now clears Redis progress server-side on submit
3. Navbar jobs polling was running every 4s even with no active jobs,
   causing unnecessary requests — now polls every 30s when idle,
   every 3s when jobs are actively running

Also cleared 3 stale Redis progress entries that were pointing to
already-completed attempts (which caused silent submit failures).

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 15:29:38 +02:00
Daniel
1f95c4d1ea Explanation: capture ALL content verbatim until next question
Updated extraction prompt to be explicit that explanation must include
every section between the correct answer line and the next question:
- Explanation paragraph
- Critique section (if present)
- Learning Points (if present)
- Content Specifications / American Board of Pediatrics specs
- Suggested Reading references
- Any other content

No summarizing, skipping, or shortening of any text.
Keep all section headers verbatim.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 13:16:43 +02:00
Daniel
77da7aea6c Restore original verbatim extraction prompt; switch to Claude Opus 4.6
Extraction prompt:
- Restored full original prompt with strict verbatim instructions
- Explanation field: must include full explanation paragraph + Critique
  verbatim + Content Specifications verbatim (no summarization)
- Correct answer: explicit step-by-step with example showing letter→full text
- Options: extract full text only, no letter prefix
- All 5 rules numbered and explicit

Model switch:
- Default extraction model changed from Claude Haiku 4.5 → Claude Opus 4.6
- Haiku tends to compress/summarize long explanation text
- Opus 4.6 follows verbatim instructions reliably and handles
  the structured PREP format (explanation + Critique + Content Specs)
  without losing content
- Haiku 4.5 still available in the Extraction Model dropdown for
  cost-sensitive use cases

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 12:57:28 +02:00
Daniel
b859d441eb Add extraction modes; fix resume; hide Nextcloud/Upload for users; delete PREP 2013 questions
Extraction modes (no restart needed — code ready for next Celery deploy):
- New QuizCreate.extraction_mode field: standard|questions_only|two_step|regex|ai_decide
- extraction_modes.py: independent implementations that don't touch standard path
  - questions_only: extract Q+options, correct_answer="PENDING" for manual fill
  - two_step: separate answer key section scan + phase1/2/3 matching
  - regex: AI detects answer pattern, generates regex, applies to full doc
  - ai_decide: AI reads samples from start+end and picks strategy
- DocumentDetailPage: Extraction Mode dropdown with description per mode
- quiz_tasks.py: routes to correct mode, standard path completely unchanged

Database:
- Deleted 11 orphaned questions from PREP 2013 extraction (quiz 12 was already deleted)
- 268 questions remaining (all PREP 2012)

UI fixes:
- Nextcloud section in Settings now only shown to moderators/admins
  (regular users can't upload PDFs so they don't need Nextcloud)
- Upload PDF already hidden in navbar for non-moderators (confirmed correct)
- Resume quiz: now async — study mode quiz data loaded BEFORE showing quiz
  so correct_answer is available immediately for feedback
- Resume saves and restores voice selection
- voice field added to ProgressSave schema and Redis storage
- Progress save dependency includes selectedVoice

Attempts:
- POST /attempts/start: reuses existing incomplete attempt by default (fresh=false)

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 12:43:24 +02:00
Daniel
84c4917d91 In-progress quizzes panel, navigation warning, attempt reuse, progress via Redis
In-progress quizzes section (Quizzes page):
- Shows all incomplete (not submitted) attempts in an amber ⏸ panel
- Each shows quiz title, start date, question count
- Resume button → navigates to quiz (which loads saved answers from Redis)
- Delete button (ConfirmButton) → deletes attempt + clears reminders

Quiz navigation warning:
- beforeunload handler warns when closing browser tab/window mid-quiz
- Leave confirmation modal when navigating away via React Router
  (shows "Your progress is saved — resume later" + Stay/Leave buttons)

Quiz progress:
- Scroll on Next/Previous REMOVED per user request — page no longer scrolls
- Progress saved to Redis on every answer/navigation (1.5s debounce)
- Works across browsers, survives logout
- ModeSelectScreen loads saved progress from API and shows Resume button

Attempt management:
- POST /attempts/start now REUSES existing incomplete attempt (no more
  creating duplicate incomplete attempts on every quiz start)
  Pass fresh=true to force a new attempt
- GET /attempts/in-progress — list incomplete attempts for current user
- DELETE /attempts/{id} — delete attempt + reminders

Cleaned up 10 duplicate incomplete attempts from the database

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 05:16:14 +02:00
Daniel
92fdf55cd9 Revert extraction to simple standard mode; PREP 2013 will be separate option later
- Completely reverted quiz_tasks.py to simple standard extraction with chunking
  (no more two-phase detection that broke PREP 2012/2014)
- OCR normalization kept: 'Pref erred'→'Preferred', 'ltem'→'Item'
- The extraction prompt already handles both 'Correct Answer: X' and
  'Preferred Response: X' inline formats — no special detection needed
- PREP 2013 (separate answer key) will be implemented as a separate option
  user selects at extraction time, not automatic detection

Also in this commit:
- Fixed quiz delete 500 error (source_quiz_id attribute name)
- Added trash bin (soft delete, restore, permanent delete)
- Added hide/publish toggle per quiz (moderators see all, users see published only)
- Quiz progress saved to Redis — survives logout, works cross-browser
- Resume in-progress quiz from any browser
- ConfirmButton component replaces all window.confirm/prompt
- Delete own attempts endpoint
- TrashPage, AdminPage trash/jobs links in Settings

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 05:04:22 +02:00
Daniel
45add79db3 Fix quiz delete (500 error), add trash bin, quiz resume, hide/publish, remove JS dialogs
Critical bug fix:
- DELETE /quizzes/{id} was returning 500 due to {\"quiz_id\": None} update using
  the DB column name instead of Python attribute name (source_quiz_id).
  Fixed with {QuestionModel.source_quiz_id: None, synchronize_session=False}

Extraction fix:
- Two-phase extraction was incorrectly triggering for PREP 2012/2014 format
  documents that have 'Preferred Response:' in their explanation text.
  Fix: check for inline 'Correct Answer:' first — if found, always use standard
  extraction regardless of 'Preferred Response:' appearing elsewhere.

Quiz trash bin:
- DELETE /quizzes/{id} now soft-deletes (sets deleted_at)
- GET /quizzes/trash — list deleted quizzes (moderator)
- PATCH /quizzes/{id}/restore — restore from trash
- DELETE /quizzes/{id}/permanent — permanent delete (must be in trash first)
- TrashPage.jsx — accessible via Settings → Admin → Trash

Hide/publish quizzes:
- is_published column on quizzes (1=visible, 0=hidden)
- PATCH /quizzes/{id}/publish?published=false — hide from regular users
- Moderators see all quizzes; regular users only see published
- 👁/🙈 toggle button per quiz card (moderators only)

Quiz progress resume (cross-browser via Redis):
- POST /attempts/progress — save {answers, current_idx, mode} to Redis (7 days)
- GET /attempts/progress?quiz_id=N — retrieve saved progress
- DELETE /attempts/progress/{quiz_id} — clear on submit
- QuizPage auto-saves to Redis every 1.5s (debounced) while in progress
- ModeSelectScreen loads saved progress from server, shows Resume button
- Works across browsers, devices, and after logout

Delete attempt:
- DELETE /attempts/{id} — user can delete own attempt + clears reminders for that quiz

ConfirmButton component:
- Replaces all window.confirm() / window.prompt() across the app
- Double-click pattern: first click shows [Confirm] [Cancel] inline
- Applied to: QuizzesPage, DocumentDetailPage, QuizEditPage, QuestionBankPage

Category delete (QuestionBankPage):
- window.prompt() replaced with inline modal dialog with select dropdown
- User chooses where to move questions before deletion

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 04:51:41 +02:00
Daniel
5a2fc7e187 Fix display: update n_chunks after chunk boundary filtering
The 'Chunk X/5' message was showing 5 even when only 2 chunks were
being processed (pages 1-50 and 51-55). n_chunks was not updated
after filtering chunks to stay before answer_section_start.
Cosmetic fix — the actual extraction was already correct.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 04:13:50 +02:00
Daniel
8a220bb12e Fix PREP 2013 extraction: OCR normalization, correct chunk boundary; category UI fix
Extraction fixes:
- OCR normalization: 'Pref erred' → 'Preferred', 'ltem' → 'Item' applied to boundary
  scan, Phase 1 questions, and Phase 2 answer key content before AI processing
- Chunk boundary: Phase 1 chunks now capped at (answer_section_start - 1) so no
  chunk bleeds into the answer section — (51, 100) becomes (51, 55) for PREP 2013
- Result: Phase 1 gets 2 clean chunks (1-50 and 51-55), Phase 2 gets pages 56-227

Category creation in DocumentDetailPage:
- Replaced window.prompt() with inline input form (more reliable, no browser quirks)
- Fixed option value type: String(c.id) ensures consistent string comparison with
  selectedQuestionCategoryId state (prevents type mismatch in controlled select)
- "+ New" button toggles inline form; Enter key or Add button submits

Deletion safety (confirmed):
- Deleting a quiz: questions detached to bank if exclusive, kept if shared — NEVER deleted
- Deleting a question category: questions uncategorized or moved — NEVER deleted

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 04:09:55 +02:00
Daniel
d07be64f59 Fix two-phase extraction boundary detection; README CLI docs; scroll fix
Two-phase extraction improvements:
- Auto-detect answer section boundary by scanning in 10-page steps for
  'Preferred Response:' — finds exact page where questions end and answers begin
  (PREP 2013 answers start at page ~68, not at the end of the file)
- Restrict Phase 1 question chunks to pages BEFORE the answer section
- Extract answer key from answer section in CHUNKS (50 pages each) to handle
  large answer sections — accumulates all item→letter mappings
- Previous version used last 40% which missed items 1-~135 for PREP 2013

README: full CLI extraction documentation:
- list-sections: find document and section IDs
- extract <section_id> [--bg] [--title] [--mode] [--user]
- jobs / jobs --user <email>
- Explanation of auto-format detection (inline vs separate answer key)

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 03:36:12 +02:00
Daniel
e5e31f6eba Two-phase extraction for end-of-document answer keys (PREP 2013); scroll fix; bug audit
Two-phase extraction:
- Detects end-of-document answer key format by scanning last 40 pages for
  "Preferred Response:" (PREP 2013, 2014 etc use this vs PREP 2012 inline "Correct Answer:")
- Phase 1: Extract questions with item_number field, allow null correct_answer
- Phase 2: Extract answer key (item_number → letter) from last 40% of document
- Phase 3: Match questions to answers by item number, resolve letter → full option text
- Unmatched questions go to skipped list with reason shown in Jobs page
- Standard inline format (PREP 2012) unchanged

Updated extraction prompts:
- item_number field added to all extractions for cross-referencing
- Image content rule: "Item CXXXB" figure references must NOT be treated as new questions
- Recognises both "Correct Answer: X" and "Preferred Response: X"
- ANSWER_KEY_PROMPT: dedicated prompt for extracting answer key tables

Quiz navigation scroll:
- Clicking Next, Previous, or question number now scrolls the question card
  into view (smooth scroll to start of question-card div)

Code: extract_questions_no_answers(), extract_answer_key(), _call_model() added to ai_service.py

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 03:22:26 +02:00
Daniel
287d3a72f4 Fix question bank: select-all, bulk category, page size, category delete
Bugs fixed:
- 'Select all' now fetches ALL matching IDs from server (GET /questions/bank/ids)
  not just the currently loaded page — works for 268 questions
- Bulk category assign: fixed 'Remove category' value ('0' → 'remove'),
  now shows error message on failure instead of silent fail
- Search mode (Keyword/Semantic/Hybrid) moved to a clearly labelled dropdown,
  visually separated from category filter chips — eliminates confusion
- Category chips now have a 'Filter:' label to distinguish from other buttons

New features:
- Page size selector: 50 / 100 / 200 / All — loads the requested count in one shot
- Bulk actions moved to its own clearly labelled bar (only shown for moderators)
- Category delete with questions: prompts user to choose target category before deleting
  (uses new move_to param on DELETE /question-categories/{id})
- GET /questions/bank/ids endpoint for server-side select-all

Backend:
- DELETE /question-categories/{id}?move_to={id}: move questions to another category
  before deletion instead of always leaving them uncategorized

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 02:55:25 +02:00
Daniel
0bc336a7d0 Quiz delete: detach questions to bank instead of deleting them
When deleting a quiz:
- Questions shared with other quizzes stay linked to those quizzes
- Questions exclusive to this quiz are detached (source_quiz_id cleared)
  and remain in the Question Bank as orphaned questions, not deleted
- This preserves the question bank integrity

Also improves manage.py CLI with extract, list-sections, and jobs commands

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 02:36:25 +02:00
Daniel
871206d891 Cross-browser job tracking, hamburger fix, CLI extract, quiz delete keeps bank questions
Jobs (cross-browser/cross-session):
- POST /quizzes/ stores job_id in Redis under user key (extraction:user_jobs:{uid})
- GET /quizzes/jobs returns all recent jobs for current user from any browser/session
- Navbar JobsBadge polls /quizzes/jobs API every 4s (not localStorage)
- Shows all recent jobs with status badges; links to quiz when complete
- Badge visible even after extraction completes so you can always get back

Mobile navbar fix:
- .navbar .container height was overriding dropdown to 52px (clipping all links)
- Fixed by using .navbar-inner class for the header row only

CLI extract command:
  python manage.py list-sections [doc_id]   — list docs + sections with IDs
  python manage.py extract <section_id>     — inline blocking extraction
  python manage.py extract <section_id> --bg  — background via Celery
  python manage.py jobs                    — show all extraction jobs in Redis
  python manage.py jobs --user <email>     — filter by user

Quiz delete + question bank:
- When a quiz is deleted, questions that belong ONLY to that quiz are deleted
- Questions shared with other quizzes (via junction) are kept in the bank

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 02:30:47 +02:00
Daniel
96a1a259f0 Async extraction with live progress + chunked large PDFs
Extraction is now fully async via Celery — UI shows a live progress panel,
job continues even if page is closed. Large documents are processed in
50-page chunks to extract all questions (not just first ~50 pages).

Backend:
- app/tasks/quiz_tasks.py: new Celery task 'extract_quiz'
  - Writes step-by-step progress to Redis (extraction:steps:{job_id})
  - Splits large page ranges into 50-page chunks, processes each separately
  - Reports per-chunk results and running total
  - Falls back to synchronous if Celery/Redis unavailable
- POST /quizzes/ now returns {job_id, status:"pending"} immediately
- GET /quizzes/job/{job_id} polls progress: steps[], status, quiz_id on completion
- Celery task list updated to include quiz_tasks

Frontend (DocumentDetailPage):
- ExtractionProgress modal component: monospace step log, auto-scrolls, spinner
- Polls job status every 2 seconds via /quizzes/job/{job_id}
- "Open Quiz →" button appears when done
- "✕ closes — job continues in background" shown while running
- beforeunload warning when job is active (preventing accidental close)

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 01:55:24 +02:00