pdf-quiz-generator/docs/TODO.md
Daniel 9beafff0bf feat: AI Mode — a chat that cannot cite what it did not find
The design settled earlier, built as described: retrieval decides what may be
cited, and the server enforces it.

The model is handed a shortlist of at most fourteen sources from the learner's
own library and told to cite them by marker. Afterwards every citation it wrote
is checked against that shortlist and anything else is deleted before it is
stored or shown. A hallucinated citation is not unlikely here, it is impossible
— surviving is not a decision the model gets to make. A URL it invents is not a
citation either: only the marker form counts, so a plausible-looking link stays
in the prose citing nothing.

Retrieval reuses the hybrid search already in place, and each corpus keeps its
own visibility rules — the bank predicate and exam scope for questions, the
draft rule for articles, deck ownership for cards. A question source carries the
stem only: a chat that printed the answer would hand away the practice it exists
to prepare you for.

Curated links do the job they were built for. A retrieved row an educator tied
to another retrieved row is boosted, because two things somebody already linked
surfacing for one query is evidence rather than coincidence. Nothing is stored
for this; the boost lives only in that ordering, and the answer marks those
sources so the reader knows which claim rests on an educator's judgement rather
than on a ranking.

Citations are stored with the answer as filtered, so reopening a thread shows
the links it showed at the time rather than a fresh retrieval that may now rank
differently. In the page the markers become numbers and each number opens its
source; a section citation deep-links into that section.

Two smaller decisions worth naming: a question appears in the thread the moment
you send it and is handed back to the input if the answer fails, because typed
words are not something to lose on a 502; and someone else's thread returns 404
rather than 403, since whether it exists is not your business either.

182 backend, 206 frontend green — 16 of the backend tests are the citation
contract and the retrieval boundary.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
2026-09-10 15:19:24 +02:00

12 KiB

PedsHub — outstanding work

Everything requested and not yet delivered. Ordered roughly by dependency, not priority — say which to take and I'll reorder.

Updated 2026-09-10.


Design agreed, not built

  • AI Mode (RAG chat) — done 2026-09-10. conversations and conversation_messages (migration a5b6c7d8e9f0), retrieval across sections, articles, questions and cards with the curated-link boost, the ID-citation contract enforced server-side, and /ai with a thread rail. Answers cite by number and every source opens; a section citation deep-links into its section. Question sources carry the stem only, so a chat cannot hand away the practice it is meant to prepare for.
  • Global search page — done 2026-09-10. GET /search runs every corpus at once and /search groups the answer by kind, with section hits listed under the article they belong to and linked to that section. A header box offers "go to this article" or "search everything". Each corpus keeps its own visibility rules — bank predicate and exam scope for questions, the draft rule for articles, deck ownership for cards, library grants for images. The Search / AI Mode toggle waits on AI Mode.

UI fixes raised 2026-09-10

  • Quiz/test categories removed — done.
  • Sessions list shows only a few — done, with a link to full history.
  • Analysis session rail full-height — done.
  • Articles page layout — done 2026-09-10. The page is the column browser itself: topics and the articles filed under them share a column, separated by icon, one column per level opened.
  • Systems facet duplicates — done 2026-09-10. 491 redundant "(Parent)" suffixes stripped and 3 sibling pairs merged. Nesting still to do.
  • "⚙ Filters2948 questions" — done. The stylesheet was never imported.
  • Category page relationships — direct vs rolled-up counts, empty-leaf badge, and a desktop hint on small screens.
  • Newly created categories now appear — the bare path 307-redirected to http://, which the browser blocks as mixed content, so the call failed silently. Trailing slash added.
  • Image/media page — done 2026-09-10. Tags and captions show in the picker as well as the bank, libraries have their own rail, and ids sit on every thumbnail rather than only on hover.

Content and editing

  • Study plans have a front end at all — done 2026-09-10. 13 plans were seeded with an API to serve them and no page that called it. /study-plans lists them with progress in blocks; /study-plans/:id is one plan.
  • Admin can edit study plans — done 2026-09-10. Create (as a draft), rename, publish/unpublish, delete; add, rename, reorder and remove blocks; move questions between blocks of the same plan; attach and detach reading. Editing is inline on the learner's own page, so there is no second layout to keep in step.
  • Study plan blocks carry articles — done 2026-09-10. Each block shows Articles with a reversible Mark as read, then Sessions with Study and Exam mode. Reading progress is per learner and separate from article_views: opening an article is not the claim that you finished it.
  • Admin settings page revamp — currently ugly; needs restructuring.
  • Image libraries — done 2026-09-10. Libraries, per-library grants, tags on the shared vocabulary, and MinIO behind a storage service.
  • Media management page (frontend) — done 2026-09-10. /media browses the bank by library, searches by what an image shows, and edits title, caption, alt text, tags and library in place; ids are on every thumbnail because that is what a question refers to, and an image with no caption is called out as one nobody will find. ImagePicker replaces the type-a-filename field on the question editor, and an image uploaded from it lands on the question directly.
  • Question folders — collect questions into folders for assignment and access, alongside category grants.

Article reading

  • Nested sections and per-section collapse — done 2026-09-10. A section may sit under an earlier top-level one (parent_id on the section JSON), the contents rail lists sub-sections under their parent, and an article opens as headings only, each expanding where it sits. Deep links open the target section and its parent.
  • Cross-references with previews — done 2026-09-10. [[slug]] or [[Label|slug]] in article prose becomes an in-app link that shows title, excerpt and section count on hover, from GET /articles/preview/{slug}. One fetch per article per page; no card on touch, where there is no hover.
  • Library browsed column by column — done 2026-09-10. The articles page is now the column browser itself: topics and the articles filed under them share a column, separated by icon. Search still answers with a flat list.
  • References — a section titled "References" is pinned last and styled, but the numbered list with superscript markers linking down to it is not built.
  • Per-section notes and feedback — a learner's own note attached to a section, and a feedback channel to the educator.
  • High-yield / key-exam-info toggles — mark spans and let the reader show or hide them.

Quiz runner

  • Per-question notes in study mode, replacing the global notes tab that is currently on the quiz page.
  • Per-question feedback to the educator.
  • Tutorial mode — first-run coach marks ("Step 2 of 6", Skip / Next).

Analysis

  • Per-question performance table — number, stem excerpt, difficulty, time per question, percentile; sortable, paginated.
  • Session analysis tab — per-session results with study recommendations grouped by Articles / Disciplines / Systems.

Dashboard

  • Overview page for signed-in users — search hero with Search / AI Mode toggle, "Continue your study", and a study-analysis donut. The current dashboard becomes this; a separate signed-out landing page comes later.

Taxonomy

  • Systems subsystems — done 2026-09-10. 69 subsystems created, 305 conditions nested; the tree is now System → Subsystem → Condition.
  • Cross-system duplicate conditions merged — done 2026-09-10. 216 names folded to one home each (347 rows removed, 1075 → 786), keeping the other systems as question_category_links rather than duplicate rows.
  • 114 tied conditions need a human call — an even split across systems has no evidence-based winner, so nothing was merged. Run scripts/merge_duplicate_conditions.py to list them; the largest are Pneumonia, Behavioral Disorder, Intestinal Obstruction, Acute Rheumatic Fever, Hypoxic-Ischemic Encephalopathy.
  • Primary categories pushed down — done 2026-09-10. 1,244 questions promoted from system to condition; 2,730 → 1,486 filed at system level. The system stays as a cross-link so no filter narrows.
  • Unfiled questions classified — done 2026-09-10. 1,170 of 1,486 filed by retrieval + a chooser constrained to the shortlist. 316 remain on a bare system: 301 the chooser declined and 15 with no viable candidate.
  • Every question filed below its system — done 2026-09-10. The last 316 were forced onto their nearest candidate; 0 remain on a bare system.
  • Review the classifier's work — roughly 3 in 4 were right on a spot check, so expect some wrong. The original system is kept as a cross-link, so a wrong pick never loses a question. Worth an editorial pass.
  • 31 questions tie between equally specific conditions — left alone.
  • Real organ systems, and symptoms grouped under them — done 2026-09-10. The 27 "systems" were disciplines; that tree is now the Topics facet, and Systems is a new flat axis of 16 organ systems. question_tags gained parent_id/sort_order (migration e3f4a5b6c7d8), and 726 symptoms are filed under the system they present in. 3,536 stay top-level rather than being forced into an approximate bucket.
  • Taxonomy editing screen — done 2026-09-10. /categories is now Taxonomy: five tabs (Topics, Systems, Symptoms, Diseases, Subjects), each with create / rename / reparent / delete-with-move, and "+ Questions" to attach questions from a bank search. Everything added lands in the question bank and quiz builder immediately.
  • Article section index backfilled — done 2026-09-10. The index was empty because _rebuild_section_index only runs on save, so articles written before it existed had no rows. 10 articles / 28 sections indexed and embedded; section-scoped question links already worked.
  • Grant dimensions — done 2026-09-10. A grant names any combination of exam, discipline and category, and covers what matches all of them.
  • Exam-scoped disciplines — done 2026-09-10. A tag may be shared (exam_id NULL, e.g. Cardiology) or scoped to one exam. Boards Pediatrics and a future Step 1 Pediatrics are separate rows over different content, rather than one label spanning both. Pediatrics is bound to Pediatrics Boards and now covers all 2,948 questions. Facet counts are computed within the active exam and empty facets are hidden.

AI Mode design

Retrieval decides what the model may cite; the model only writes prose.

  1. Embed the learner's message, search every corpus (hybrid_ids already covers questions, articles, sections, cards, media).
  2. Put the retrieved rows in the prompt as the only permitted sources, each with its kind and id.
  3. The model cites by id from that list — [[article:7#features]] — never a URL.
  4. The server rewrites citations to links and drops any id that was not retrieved. A citation the model invented cannot survive.

That last step is the safety property, and it is enforced by the system rather than by the model behaving well — the same discipline as the article page no longer printing answers.

Settled: two kinds of link, handled differently.

Curated links already exist as tables — QuestionArticleLink, FlashcardArticleLink, FlashcardQuestionLink, LabReferenceCardLink — and carry a section id. An educator asserted them, so they are trusted and stored.

Retrieved links are computed per query by hybrid_ids and are not stored at all. They are ranked guesses.

The AI uses both: retrieval finds candidates, and a candidate carrying a curated link to another candidate is boosted, because an educator has already tied those two together. Nothing new is stored for citations. The only new tables AI Mode needs are conversations and messages.


Done this session

Hybrid search (full text + BGE-M3, RRF-fused) · embedding provenance and retry job · articles, cards, sections and media as searchable corpora · exams as real data with a per-user active exam · AI-mode matching from description or upload · category management page · tag vocabulary sanitised · question manager with bulk editing · per-category educator grants · full-page question editor · session rail with gradual reveal · articles read as one page · practise-this-topic · continue-study panel · PREP study plans · question version history · exam-scoped disciplines.