Commit graph

36 commits

Author SHA1 Message Date
Daniel
4d0cdc8f2f feat: section-level index so retrieval can cite a section, not a whole article
An article embedded as a single vector, which finds the article but not the
paragraph — so a citation could only ever point at the top of a page. Sections
live in a JSON column and cannot carry a vector or a full-text index, so they are
now projected into `article_section_index`: one row per section with its own
embedding and weighted tsvector (migration x6d7e8f9a0b1).

- Rows are keyed by section id, so editing a section updates it, removing one
  deletes it, and an unchanged section is not re-embedded on every save.
- `article_section` joins the embeddable kinds, so the retry task, the full
  regeneration and the health report cover it without further changes.
- `hybrid_ids(db, query, "article_section")` searches it like any other corpus.

This is the groundwork for grouped search results (article, then the sections
that matched) and for AI citations that deep-link to the right section.

Tests: 113 backend green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PpfzbZ1QTLMeVYxM2kyq8m
2026-09-10 02:18:53 +02:00
Daniel
48f3ded222 feat: exams as real data, AI-mode matching, and a continue-study panel
Exams (migration v4b5c6d7e8f9)
"Pediatrics Boards" was a hardcoded checkbox that filtered nothing. Exams are now
rows: Pediatrics Boards and USMLE Step 2 CK ship seeded, and everything already
in the bank is linked to the boards. Membership is a link table, not a column,
because one paediatric cardiology question can count towards several exams.

The learner's choice lives on `users.active_exam_id`, so it follows them between
devices instead of sitting in one browser's storage. Choosing an exam scopes the
bank; a question with no exam links stays visible, since unlinked content is
unclassified rather than excluded. A switcher sits in the navbar.

AI mode — matching, never generating
Both entry points build a test from the educator-reviewed questions that already
exist, ranked against the request. Nothing is invented:
- POST /questions/builder/describe turns "what I want to study" into a test.
- POST /questions/builder/from-upload matches a document against the bank. The
  file is read in memory and never stored — it is a search query, not a source
  of questions, so there is nothing to retain or expire. 10 MB cap, 30 questions.

Handing a whole document to `websearch_to_tsquery` builds one enormous
conjunction that matches nothing, so text over 300 characters is reduced to its
most distinctive terms, OR-joined, before it reaches the lexical ranker.

Continue your study (migration w5c6d7e8f9a0)
A dashboard panel with the sessions in flight and the articles most recently
opened. `article_views` records one row per learner and article, written best
effort so a reading page never fails because a bookkeeping write did.

Tests: 5 new exam tests (active exams and counts, choice persisted and cleared,
unknown/inactive refused, bank scoping including unlinked questions, moderator-only
creation) and 7 for AI-mode matching (no questions created, invisible questions
excluded, no-match reported rather than an empty test, upload limits enforced).
Full suites green: 113 backend, 136 frontend, build clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PpfzbZ1QTLMeVYxM2kyq8m
2026-09-10 02:08:38 +02:00
Daniel
25109d756d feat: hybrid search for articles and cards; full-page question editor
Retrieval generalised beyond questions
`_text_for_question`, `embed_question` and `hybrid_question_ids` all hardcoded
the questions table, so there was nothing to call for an article or a card. That
layer is now corpus-agnostic:
- `Embeddable` mixin gives articles and flashcards the same embedding,
  embedding_model and embedded_at columns questions have, plus a weighted
  full-text vector (migration u3a4b5c6d7e8).
- `embed_record(row, kind)` is one code path for all three — they share an
  embedding space, so they must share the model and provenance rules too.
- `hybrid_ids(db, query, kind)` ranks any corpus; `hybrid_question_ids` stays as
  a thin alias for existing callers.
- Article and flashcard search moved off `ILIKE '%term%'`, which could not find
  a jaundice article from "yellow newborn".
- The retry task and full regeneration now sweep every corpus, and the health
  report breaks down current/stale/missing per kind.
- Articles embed on create and on edit, with failures left to the retry task.

Quoted phrases replace the keyword-only mode
`websearch_to_tsquery` already gives "absence seizure" exact-phrase semantics,
and the semantic ranker sits out a quoted query. That covers the one case a
keyword-only toggle was for — exact lookup — per query rather than as a sticky
setting whose every position returns a subset of the default.

Full-page question editor (/questions/new, /questions/:id)
Editing happened in a cramped modal. There is now a page with room for the stem,
per-option explanations, a searchable category picker with primary plus extras,
difficulty, and images. It shows the question's id with a copy button, and
Duplicate creates a variant without retyping the stem. `GET /questions/detail/{id}`
backs it, pathed under /detail/ so it cannot shadow the static routes.

Question bank filter bar restyled — the toggle and count read as one control
instead of two grey pills crowding the result count.

Tests: 101 backend green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PpfzbZ1QTLMeVYxM2kyq8m
2026-09-10 02:01:35 +02:00
Daniel
9b51a17507 feat: flashcard decks share the question and article category tree
Questions and articles both pointed at `question_categories`; decks had no
category at all, so the three content types could not be filtered together and a
topic's cards were unreachable from its category.

- `flashcard_decks.category_id` references the same tree (migration
  t2f3a4b5c697), so one category now spans questions, articles and cards.
- `GET /flashcards/` takes `category_id` and includes descendants, so a parent
  category picks up everything filed beneath it.
- `PATCH /flashcards/{id}` files or unfiles a deck, refusing a category id that
  does not exist rather than storing a dangling reference.
- The cards page shows each deck's category as a selector.

Tests: 5 new backend (all three types resolve to the same id, descendant
filtering, file and unfile, unknown category refused, renaming leaves the
category alone) and 1 new frontend. Full suites green: 101 backend,
136 frontend, build clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PpfzbZ1QTLMeVYxM2kyq8m
2026-09-10 01:40:03 +02:00
Daniel
519f2e572a feat: hybrid search on BGE-M3, with embedding provenance and a retry job
Search
- Retrieval was hybrid in name only: the keyword filter was applied to the SQL
  query, so results were the *intersection* of the two rankers. A question that
  matched the meaning but not the literal string could never be returned. It is
  now a union, fused with Reciprocal Rank Fusion (a text rank and a cosine
  distance are not on comparable scales, so RRF uses only their orderings).
- Added a generated `search_vector` tsvector + GIN index, so the lexical half is
  ranked full text rather than ILIKE substring matching.
- Chose Postgres + pgvector over OpenSearch/Elasticsearch: a search cluster
  would add a second datastore to keep in sync and a JVM on this host, to
  replace an index Postgres maintains inside the same transaction.
- Removed the keyword-only mode. It looks precise but silently drops the
  question that asks the same thing in different words.

Embeddings — measured on 500 real questions, using each question's own
explanation as a paraphrase query (known answer, no hand labelling):
  bge-small (local CPU, 384d)  R@1 0.840  R@5 0.953  186ms/query
  bge-m3 (LiteLLM proxy, 1024d) R@1 0.847  R@5 0.973   93ms/query
BGE-M3 wins on both quality and latency and needs no extra credential, since
llm.danvics.com already serves `openrouter-bge-m3`.

Three gaps this exposed, all fixed:
- Nothing recorded which model produced a stored vector, so changing models
  silently mixed incomparable spaces. `embedding_model` / `embedded_at` now
  stamp every vector, `GET /admin/embedding/health` reports current vs stale vs
  missing, and regeneration defaults to stale-only.
- The generator read the model from env while the stamp read a Redis override,
  so a vector could be labelled with a model that did not produce it. Both now
  resolve through one function, with a regression test.
- Embedding at creation is best effort, and a failure left a question invisible
  to semantic search forever. `retry_missing_embeddings` runs every 15 minutes
  via Celery beat and backfills missing or stale rows.
- Query embeddings are cached in Redis per model, so typing is not a network
  round-trip per keystroke.

`dimensions` is only sent to OpenAI's embedding-3 family; BGE-M3 rejects it.

Tests: 8 new backend tests (union not intersection, fusion ordering, per-ranker
failure degradation, provenance stamping, stale/missing accounting, generator
and stamp agreement). Full suites green: 95 backend, 127 frontend, build clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014yhHB8Pc7oQqyqn2Vo9DXA
2026-09-09 23:45:33 +02:00
Daniel
16dc431066 feat: analysis recommendations, category grants, compact question header
Analysis / recommendations (AMBOSS parity, verified on next.amboss.com):
- GET /study-tools/recommendations ranks focus areas by the study time most
  likely to raise the score. Readiness is the learner's accuracy in a category
  shrunk toward their own overall accuracy in proportion to sample size, so two
  unlucky answers do not read as a knowledge gap; it unlocks after 40 answers.
  Relevance is the share of the bank a category holds. Counts roll up through
  the category tree, so a system inherits its children's questions.
  It is deliberately not called EPC and does not claim to predict an exam.
- New /analysis page: Performance and Recommendations tabs, readiness summary,
  adaptive-session box, and expandable focus rows showing questions seen,
  answered correctly, the linked article and a per-topic practice action.

Per-category educator grants:
- category_grants table (migration p8b9c0d1e253) plus utils/category_grants.py
  resolving a grant to the category and all of its descendants.
- Question create, edit, delete, bulk and the manager summary now accept a
  moderator OR an educator granted the affected categories, and refuse moves
  that would push a question out of the holder's scope. Summary counts are
  scoped to the grant.
- Moderator endpoints to list, add and revoke grants, plus /my-grants driving
  the nav link and the manager's scope banner; grantable-users avoids handing
  moderators the admin-only user list.
- GrantsPanel in the question manager: grant, list and revoke with inline
  confirmation.

Question page:
- The category trail was a fixed 78px band that wrapped into several rows and
  pushed the stem down the page, followed by three more stacked strips. It is
  now one scrollable meta line (breadcrumb + difficulty + type) and a single
  AMBOSS-style action bar (Mark / Listen / Listen through / Clear) between the
  stem and the options. Difficulty is exposed on the runner payload.

Deploy fix: index.html shipped with no cache header, so browsers kept serving
the previous bundle references and a release looked like nothing had changed.
nginx now sends no-cache for HTML and immutable long-cache for hashed assets.

Tests: 16 new backend (recommendation shrinkage, roll-up, locking, grant scope
across create/edit/delete/bulk/summary, moderator gate) and 10 new frontend.
Full suites green: 88 backend, 116 frontend, build clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014yhHB8Pc7oQqyqn2Vo9DXA
2026-09-09 18:47:32 +02:00
Daniel
73ef007e0a feat: facet filters, personal libraries, adaptive shortcuts, restart, rename support
Create/bank pages use AMBOSS-style facets: Exams, Disciplines, Symptoms, Systems, Articles, Saved. Personal question libraries with add-to-library in study modal. Adaptive session shortcuts from performance (including weakest topics). Quiz restart with fresh attempt. Category counts computed with two grouped queries. Migration n7a8b9c0d142. 63 backend and 97 frontend tests pass.
2026-09-09 04:31:40 +02:00
Daniel
91ec6a501c feat: key points smart links, difficulty tags, adaptive sessions, educator-only question management
Key points on questions link into article sections (AMBOSS-style) with samples; difficulty tagging with builder/bank filters; adaptive session algorithm prefers unanswered questions then recycles older incorrect ones, weakest categories first with damping; question create/edit is now admin/educator only; expired exams no longer auto-submit on resume; exam suspend messaging updated. Migrations k4f5a6b7c819, l5a6b7c8d920, m6a7b8c9d031. 63 backend and 97 frontend tests pass.
2026-09-09 02:26:45 +02:00
Daniel
179e51d143 feat: per-option explanations
Questions support an explanation per option, edited in the question dialog and shown in study feedback. Keys must match current options. Migration j3e4f5a6b708. 60 backend and 95 frontend tests pass.
2026-09-08 19:29:21 +02:00
Daniel
f5a084e49a feat: Orthobullets-style lab panel with source deep links and card links
Lab references deep-link to article sections or external sources, show linked cards with study links, and educators can attach cards and article targets. Grouped panel layout. Migration i2d3e4f5a607. 57 backend and 93 frontend tests pass.
2026-09-08 16:20:55 +02:00
Daniel
cdb1ab7468 feat: multi-subcategory questions, statistics hardening, seeded lab values
Questions can belong to additional subcategories (junction table, counts, builder/bank filters, edit UI chips); response statistics dedupe duplicate options, match case-insensitively and exclude obsolete answers; AI question classification is command-line only (UI trigger removed); lab reference seed script with cited public pediatric ranges. Migration h1c2d3e4f506. 55 backend and 88 frontend tests pass.
2026-09-08 14:43:22 +02:00
Daniel
0fa8d0a689 feat: shareable quiz links, moderated comments and AI authoring
Quiz share links replace the PIN copy with a public /share/{token} landing page; owners can enable/revoke without showing the full link. Moderated article/question comments with approval flow, bounds and rate limits. Educator AI article drafts/refine and private card generation with Celery job polling. Migrations f2a1c9d4e801 and g4b7e2f5a903. 50 backend and 85 frontend tests pass.
2026-09-07 17:10:08 +02:00
Daniel
325f7f400f feat: linked topic articles and card associations
Draft/published article library with stable section IDs, breadcrumbs, link remediation, question and card associations, manual card creation and side-by-side/mobile-drawer reading. Migration e8d4f1a27c93. Verified 42 deployed-image backend tests, 72 frontend tests/build and PostgreSQL migration round-trip.
2026-09-07 15:16:48 +02:00
Daniel
c630b0270d fix: preserve saved quiz progress across resume failures
Block Start after failed resume, retry saved state, re-read reused-attempt progress, surface cache failures and prevent leaving on failed save. Handle nullable review settings and register lab model with Alembic. Verified 25 deployed-image backend tests, 53 frontend tests/build and PostgreSQL schema round-trip.
2026-09-07 03:34:55 +02:00
Daniel
a3a6ef7995 feat: redesign quiz runner and add study tools
Add Orthobullets-inspired numbered-answer UI, explicit study response confirmation, response statistics, review navigation, safe calculator, keyboard controls and sourced educator lab references. Persist attempt mode to prevent query-flag exam disclosure. Combined deployed-image backend suite (22), frontend suite (48), build and synthetic desktop/mobile browser checks pass. PostgreSQL round-trip and independent review remain release gates; no production deployment.
2026-09-07 03:10:23 +02:00
Daniel
affd7177b5 feat: add category-based custom tests and permission-safe sharing 2026-09-07 01:17:42 +02:00
Daniel
3cf21d4438 chore: preserve pre-revamp local changes
Recovery snapshot of the existing worktree before the Orthobullets-inspired revamp. Includes explanation images, classification snapshots, quiz visibility/resume fixes, quiz codes, TTS options and bot formatting. Secret heuristic and Python syntax checks passed; not a release or full behavioral validation.
2026-09-07 00:50:35 +02:00
Daniel
9f97218f39 Add persistent MyNote for quizzes
All checks were successful
Mobile Android Release / android-release (push) Successful in 1m32s
2026-05-12 17:51:23 +02:00
Daniel
5398342e3d Suspend pauses timer; hide timer-expired attempts from history
Suspend now pauses the timer instead of letting it run out:
- 'Suspend & Leave' sends suspended=true with time_left to backend
- On resume, backend re-anchors started_at to now with held time_left
- Closing tab without suspending continues to run the timer (unchanged)

Timer-expired auto-submits are marked with expired=1 and excluded from:
- Attempt history (GET /attempts/history)
- Dashboard stats (quiz count, total attempts, average score)
- Attempt list (GET /attempts)
- DDL: ALTER TABLE quiz_attempts ADD COLUMN expired INTEGER DEFAULT 0

Course-quiz decoupling is preserved — these changes only touch
non-course quizzes (Quiz.course_id IS NULL).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-19 19:27:46 +02:00
Daniel
b39e09f393 Persist reminders_disabled in Postgres and harden reminder scheduler
- Add reminders_disabled boolean column to users (DB source of truth).
- Scheduler reads the DB column directly; Redis no longer used for
  opt-out checks.
- Scheduler now deactivates reminders when a user has no completed
  attempts left for a quiz (e.g., after deleting their attempts).
- Settings API: GET returns DB value for reminders_disabled; PUT
  persists that key to DB and keeps the rest of the blob in Redis.

Rollback point for Alembic wiring.
2026-04-14 04:31:03 +02:00
Daniel
8137d5b20c Decouple course quizzes, replace passlib, add security hardening
- Fully decouple course quizzes from main quiz system (hidden from
  dashboard stats, history, search, results page)
- Course quiz results show "Back to Course" instead of retake/delete
- Add allow_review toggle for course creators to control answer review
- Show quiz title on course page, hide pool size from students
- Add course thumbnails to browse cards
- Replace passlib with bcrypt directly (compatible with existing hashes)
- Add HIBP breached password warnings on register/reset/change password
- Add CLI management tools (reset-password, set-role, stats, etc.)
- Fix quiz PATCH endpoint: ownership check instead of moderator-only
- Add max_length validation on course/module/lesson titles
- Fix score display bug on results page (0 of N when review disabled)
- Fix question count on course quiz start (show per-attempt, not pool)
- Improve suspend warning for timed course quizzes with max attempts
- Clean up validation error messages (show "Invalid email" not Pydantic dump)
- Add DDL migration for allow_review column

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 21:31:05 +02:00
Daniel
a2a2da7da0 Add LMS, user quiz creation, manual questions, course system
LMS / Course System:
- Course → Module → Lesson hierarchy (any user can create)
- Lesson types: text, video (Vimeo/YouTube/local), document, quiz, live_session
- Video provider auto-detection from URL
- BBB API integration (create/join meetings) with env config
- Course enrollment with per-lesson progress tracking
- AI content generation/refinement for text lessons
- Draft/published/archived status workflow
- Subscription gate placeholder (requires_subscription flag)
- Thumbnail upload support
- Module/lesson reorder with up/down controls

User Quiz Creation:
- Any user can create quizzes from question bank (was moderator-only)
- User quizzes: is_published=0, is_shared=0 (private by default)
- Fixed section_id fallback: None instead of hardcoded 1

Manual Question Creation:
- POST /questions/create endpoint for manual MCQ entry
- CreateQuestionModal on QuestionBankPage with options, radio for correct answer
- Auto-generates embedding on creation

Frontend:
- CoursesPage: Browse/My Courses/Created tabs with search and pagination
- CourseDetailPage: Student view with module accordion, lesson viewer, progress
- CourseEditorPage: Full course builder with AI generate, question bank browser
- Courses link in Navbar
- Create Question button on Question Bank (available to all users)

Backend:
- 5 new tables: courses, course_modules, course_lessons, course_enrollments, course_lesson_progress
- Course model + schemas + router (22 endpoints)
- BBB_SERVER_URL + BBB_SECRET config
- Updated CLAUDE.md with LMS documentation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 02:07:04 +02:00
Daniel
18d0a74348 Add deck sharing, star ratings, and shared deck browsing
- Users can share/unshare their flashcard decks (toggle button)
- Shared tab: paginated list of all shared decks sorted by avg rating
- Star rating system (1-5): click stars to rate, updates average live
- Each user can rate each shared deck once (upsert on re-rate)
- Admin can force-unshare any deck (remove from shared listing)
- Shared decks are study-able by anyone (read-only access)
- Owner name displayed on shared deck cards
- Load more button for shared decks (12 per page)
- New DB table: flashcard_deck_ratings (user_id+deck_id unique)
- New column: flashcard_decks.is_shared (default 0)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 23:37:40 +02:00
Daniel
800de50fc9 Flashcard ownership, trash, delete cards, fix job status
- Fix: flashcard task used wrong Redis key prefix (job: vs extraction:)
  causing progress to stay on "pending" after completion
- Flashcards are now user-scoped: each user sees only their own decks
- Soft-delete decks: DELETE moves to trash, ?permanent=true to destroy
- Trash tab on flashcards page: restore or permanently delete decks
- Delete individual cards with inline confirmation in browse view
- Card edit/delete now allowed for deck owner (not just moderators)
- ExtractionProgress label prop: shows "Generating Flashcards" not
  "Extracting Questions" for flashcard jobs
- Added deleted_at column to flashcard_decks

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 23:24:48 +02:00
Daniel
3e37bf2128 Add AI-powered flashcard generation system
New feature: generate flashcards from PDF sections using AI, completely
separate from the existing quiz system.

Backend:
- FlashcardDeck + Flashcard models with cascade deletes
- flashcard_tag_links table for tag classification (reuses question_tags)
- /api/flashcards/ router: CRUD for decks, browse/search cards, tag filtering
- generate_flashcard_deck Celery task with chunked processing + progress
- FLASHCARD_PROMPT in extraction_modes.py (15 cards per chunk)
- "flashcard" added to admin model task types

Frontend:
- FlashcardsPage: deck grid + card browser with search/filter
- FlashcardStudyPage: flip cards, mark known/review, keyboard nav,
  shuffle, progress bar, completion screen
- DocumentDetailPage: "Create Flashcards" button alongside "Extract Quiz"
- Navbar: Flashcards link
- AdminPage: flashcard in model task dropdown

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 22:59:21 +02:00
Daniel
2cbbfe00c3 Tag filtering, multi-category, bug fixes, image validation, docs
- Fix tag filtering (sa_text import shadowing caused UnboundLocalError)
- Add TagBrowser component with per-section search
- Multi-category selection (OR within categories, AND with tags)
- AI image validation: has_figure field in extraction prompt
- Skip known branding images by MD5 hash + dimension filters
- Fix quiz timer auto-submit (wrong useEffect dependency)
- Fix QuizResponse schema: section_id nullable
- Fix Question.quiz_id → source_quiz_id attribute name
- Fix SQL injection in quizzes.py vector search
- Add PDF processing progress steps via Redis
- Add delete user from admin panel
- Admin page: no spinner flash on data refresh
- Upload progress: axios 1.x e.progress, remove manual Content-Type
- Duplicate model error: 409 with clear message
- Backend startup: retry DDL migration on lock timeout
- Replace all silent except:pass with warning logs
- Comprehensive multi-page documentation (docs/)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 22:48:26 +02:00
Daniel
4c48a5cf94 Security hardening, async TeachChat, rate limit UX, unthrottle
Security:
- admin.py: move api_key from URL query params to POST body (litellm/models, tts/voices) — prevents key logging
- admin.py: sanitize exception messages in voice discovery — log internally, return generic errors
- teach.py: log LLM errors server-side, show friendly message to user
- nextcloud.py: normalize path with posixpath.normpath to prevent ../ traversal
- auth.py: check_rate_limit now accepts user param — admins/moderators/unthrottled always exempt

Performance:
- teach.py: make /chat endpoint async, use litellm.acompletion() — no longer blocks a uvicorn thread per request

Features:
- users: add is_unthrottled column (DB migration in setup_pgvector)
- admin.py: PUT /users/{id}/unthrottle endpoint
- AdminPage.jsx: Unlimited/Throttle toggle button per user, shows "unlimited" badge
- Rate limit messages improved with user-friendly context

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 01:09:42 +02:00
Daniel
975a31fb01 TeachChat: model selector, proxy routing fix, titan seed cleanup
- teach.py: use _proxy_model() + pass api_key/api_base from settings (fixes LiteLLM provider error for openrouter/bedrock models)
- teach.py: accept model_id in ChatRequest so frontend can select model
- main.py: remove titan-embed-v2 from general seed, auto-delete legacy entry on startup
- main.py: kill stale idle-in-transaction DB connections at startup to prevent DDL lock hangs
- main.py: set lock_timeout=10s on DDL connection as fast-fail safety net
- TeachChat.jsx: fetch /teach/models, show selector dropdown in header when >1 model available

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 00:37:35 +02:00
Daniel
12d99d3609 Add switchable embedding model, Polly toggle, job cancellation, and UI fixes
Embedding:
- Embedding model now configurable via Admin UI (More tab) or LITELLM_EMBEDDING_MODEL env
- Calls LiteLLM proxy directly via httpx (bypasses LiteLLM library param validation)
- Passes dimensions=1024 to proxy; Redis setting overrides env var
- Default model: ge-gemini-embedding-001 (Gemini AI Studio, 1024-dim)
- Test button in admin UI to verify model works
- Fixed vector_service to use httpx + Redis model (was broken with non-prefixed model names)

Polly:
- Global enable/disable toggle in Admin → More settings (stored in Redis)
- /tts/voices filters out polly/* when disabled
- /tts/speak rejects polly requests when disabled

Job cancellation:
- POST /quizzes/job/{job_id}/cancel endpoint
- Cancel button on JobsPage for running jobs
- Celery task checks Redis status at each chunk boundary and exits cleanly
- Fixes DB lock on restart caused by cancelled jobs leaving open transactions

Admin UI:
- Settings tab renamed to "More" (heading: More Settings)
- Model row overflow fixed (minWidth: 0 + ellipsis on model_id)
- Embedding model search shows all proxy models (no auto-filter by "embed")
- Navbar correctly excludes cancelled/failed jobs from "extracting" count

README:
- Added Rebuild & Restart section with commands
- Updated embedding model reference

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-03 20:44:11 +02:00
Daniel
c94640c890 CRITICAL: Remove cascade delete from Section→Quiz relationship
Root cause of lost PREP 2016 quiz: deleting a section (e.g. to recreate
with different page ranges) cascade-deleted ALL quizzes attached to it,
destroying the quiz AND all its questions permanently.

Fixes:
- Section.quizzes relationship: removed cascade="all, delete-orphan"
  Deleting a section no longer touches quizzes at all
- Quiz.section_id FK: changed to ON DELETE SET NULL, nullable=True
  If a section is deleted, quizzes keep working (section_id becomes null)
- DB migration: drops old FK constraint and recreates with SET NULL

Also answers the question about deleting a bank question in a quiz:
- quiz_question_links FK has ON DELETE CASCADE — deleting a question
  silently removes it from all quizzes
- This needs a warning UI (not yet implemented — no delete button in bank)

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 18:26:22 +02:00
Daniel
45add79db3 Fix quiz delete (500 error), add trash bin, quiz resume, hide/publish, remove JS dialogs
Critical bug fix:
- DELETE /quizzes/{id} was returning 500 due to {\"quiz_id\": None} update using
  the DB column name instead of Python attribute name (source_quiz_id).
  Fixed with {QuestionModel.source_quiz_id: None, synchronize_session=False}

Extraction fix:
- Two-phase extraction was incorrectly triggering for PREP 2012/2014 format
  documents that have 'Preferred Response:' in their explanation text.
  Fix: check for inline 'Correct Answer:' first — if found, always use standard
  extraction regardless of 'Preferred Response:' appearing elsewhere.

Quiz trash bin:
- DELETE /quizzes/{id} now soft-deletes (sets deleted_at)
- GET /quizzes/trash — list deleted quizzes (moderator)
- PATCH /quizzes/{id}/restore — restore from trash
- DELETE /quizzes/{id}/permanent — permanent delete (must be in trash first)
- TrashPage.jsx — accessible via Settings → Admin → Trash

Hide/publish quizzes:
- is_published column on quizzes (1=visible, 0=hidden)
- PATCH /quizzes/{id}/publish?published=false — hide from regular users
- Moderators see all quizzes; regular users only see published
- 👁/🙈 toggle button per quiz card (moderators only)

Quiz progress resume (cross-browser via Redis):
- POST /attempts/progress — save {answers, current_idx, mode} to Redis (7 days)
- GET /attempts/progress?quiz_id=N — retrieve saved progress
- DELETE /attempts/progress/{quiz_id} — clear on submit
- QuizPage auto-saves to Redis every 1.5s (debounced) while in progress
- ModeSelectScreen loads saved progress from server, shows Resume button
- Works across browsers, devices, and after logout

Delete attempt:
- DELETE /attempts/{id} — user can delete own attempt + clears reminders for that quiz

ConfirmButton component:
- Replaces all window.confirm() / window.prompt() across the app
- Double-click pattern: first click shows [Confirm] [Cancel] inline
- Applied to: QuizzesPage, DocumentDetailPage, QuizEditPage, QuestionBankPage

Category delete (QuestionBankPage):
- window.prompt() replaced with inline modal dialog with select dropdown
- User chooses where to move questions before deletion

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 04:51:41 +02:00
Daniel
f2d15c3c57 Junction table for shared questions; verification blocking; semantic bank search; bug fixes
Question sharing (edits now propagate):
- quiz_question_links junction table: quizzes reference questions, not own copies
- Existing questions migrated to junction (idempotent ON CONFLICT DO NOTHING)
- from-bank and from-category create quiz by adding junction links (no copying)
- Editing a question via QuizEditPage or bank Edit modal updates the ONE record
  visible in all quizzes that reference it
- Delete from quiz: only deletes Question record if no other quiz references it
- quiz_service: fixed position counter (was using len() incorrectly, now enumerate)
- question.quiz_id renamed to source_quiz_id in Python (DB column still quiz_id)
- All attempts/quizzes/search code updated to use junction helper or source_quiz_id

Bug fixes:
- PostgreSQL en_US.utf8 collation B-tree index corruption on users.email
  Fixed by ALTERing column to COLLATE "C" in migration + immediate repair
- get_current_user now blocks unverified users on ALL endpoints (403 + X-Unverified header)
  api/client.js intercepts X-Unverified=true and auto-logs out to /login
- Search (quiz + bank): correct_answer and explanation now included in matching_questions
  so study modals work correctly
- Keyword search: order_by updated to use source_quiz_id
- Bulk category: changed from PATCH with body array to POST with Pydantic body
  (fixes null category_id removal)
- Question bank: edit button added, answer highlighting removed from list view
- QuestionBankPage: Keyword/Semantic/Hybrid search toggle
- Login: login error message stays visible (removed immediate reload)

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 00:55:13 +02:00
Daniel
a7a5bdff62 Proper question bank system with question categories
Architecture:
- Questions are primary objects in a bank, tagged with question categories
- QuestionCategory is a separate taxonomy from QuizCategory (different concepts)
- Extraction → questions added to bank, optionally tagged to a question category
- Quizzes can be created from: individual question selection, question category, or PDF extraction

Backend:
- QuestionCategory model + question_categories table
- question_category_id column on questions table (nullable, SET NULL on delete)
- GET/POST/PATCH/DELETE /api/question-categories/
- POST /api/question-categories/{id}/create-quiz — create quiz from all questions in a category
- PATCH /api/questions/{id}/category — assign single question to category
- PATCH /api/questions/bulk-category — assign multiple questions at once
- GET /api/questions/bank?category_id=&uncategorized= — filter by category
- QuizCreate schema now accepts question_category_id for extraction
- quiz_service.create_quiz_from_section accepts question_category_id param

Frontend:
- DocumentDetailPage: Add to Bank Category dropdown in Quiz Settings (optional)
  Labels extracted questions with the selected category on creation
- QuestionBankPage: full rewrite
  - Category chips for filtering (All / Uncategorized / named categories)
  - Create category button inline
  - Checkbox multi-select with bulk category assignment
  - Create Quiz modal: choose from selected questions OR all from a category
  - Each question shows its category badge and quiz source
  - Study modal with instant answer feedback

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 21:34:39 +02:00
Daniel
9af63e67b5 Major: categories, question bank, security fixes, mobile layout, UX improvements
Security:
- Nginx: X-Frame-Options, X-Content-Type-Options, X-XSS-Protection, CSP, Referrer-Policy, Permissions-Policy headers
- Redis-backed login rate limiting (survives container restarts)
- Admin litellm/models endpoint: api_key moved from GET query param to POST body
- Nextcloud credentials moved from localStorage to sessionStorage (cleared on tab close)

UX / Layout:
- Login: unverified users see inline "Resend verification email" option
- QuizPage mobile: TTS Listen button on its own row below question text
- QuizPage mobile: Voice selector on its own row in header card (not squashed with timer)
- QuizEditPage: scroll position preserved after saving a question edit

Quiz Categories:
- New QuizCategory model + quiz_categories table
- category_id column added to quizzes table
- GET/POST/DELETE /api/categories endpoints
- Quizzes grouped by category in QuizzesPage; moderators can assign via 🏷 menu
- Uncategorized section shown when categories exist

Question Bank:
- GET /api/questions/bank — search all questions across quizzes
- POST /api/questions/from-bank — create new quiz from selected questions (copies, originals untouched)
- QuestionBankPage: search, checkbox select, study modal, create quiz form
- "Question Bank" link added to Navbar

Search:
- "View all N questions →" button expands to full question list
- Each question has a Study button opening in-place modal with study mode
- Summary view shows 2 questions per quiz with Study button

Extraction prompt:
- Stronger emphasis on correct_answer field with step-by-step letter → full text example
- Explicit instruction never to store just the letter

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 20:08:05 +02:00
Daniel
47ba213ae3 Major platform update: pgvector search, multi-provider TTS, settings page, CLI
Features:
- Hybrid semantic + keyword quiz search (pgvector HNSW + PostgreSQL ILIKE)
- AWS Bedrock Titan Embed V2 embeddings via LiteLLM proxy (0.71 cosine sim)
- Multi-provider TTS: OpenAI, AWS Polly (neural), ElevenLabs, Google Cloud TTS
- Unified Settings page (profile, theme, Nextcloud integration, admin shortcuts)
- Good morning/afternoon greeting on dashboard
- manage.py CLI: reset-password, list-users, reembed
- Email verification enforced: register no longer returns JWT for unverified users
- Quiz search with debounced input, semantic/keyword/title modes, highlighted snippets
- TTS button: loading/playing states, voice selector locked during playback
- TTS auto-stops when navigating between questions
- Footer added; mobile quiz nav overflow fixed; markdown theme body selector fixed
- OpenAI Alloy as default TTS voice; favicon added
- SMTP configured via smtp2go; password reset rate limiting (3/hour)
- PostgreSQL upgraded to pgvector/pgvector:pg16

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 18:03:10 +02:00
ifedan-ed
b876f13fac Initial commit: PDF Quiz Generator app
- FastAPI backend with JWT auth, roles (admin/moderator/user)
- PDF upload (up to 500MB) with streaming, PyMuPDF text extraction
- ChromaDB vectorization per page with metadata
- LiteLLM AI question extraction from PDF (not generation)
- Image extraction from PDF pages, graceful fallback
- Quiz modes: timed (countdown timer) + learning (answers shown inline)
- Page-by-page question navigation with dot navigator
- TTS endpoint using LiteLLM (Google Vertex / OpenAI voices)
- Admin dashboard: AI model management per task, user role management
- Moderator role: upload PDFs, create sections, generate quizzes
- Spaced repetition reminders via SMTP email (SM-2 intervals)
- APScheduler daily reminder jobs
- Celery + Redis for background PDF processing
- React frontend with all pages
- Docker Compose deployment (nginx + backend + celery + redis)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 20:04:53 +00:00