pdf-quiz-generator/docs/adaptive-sessions.md

Adaptive sessions — what "prioritised by impact" means here

Code: backend/app/services/quiz_builder.py, class CandidateRanking (reached by adaptive_select, the algorithm: "adaptive" branch of generate_test), and backend/app/services/prepared_session.py. Reached from: the Ready for you card on the dashboard (frontend/src/components/PreparedSession.jsx), the Adaptive toggle on the custom-session builder (frontend/src/pages/CustomQuizPage.jsx), and the Next step: adaptive session card on Analysis (frontend/src/pages/AnalysisPage.jsx). Tests: backend/tests/test_prepared_session.py, backend/tests/test_quiz_builder.py (AdaptiveSelectionTests).

There is no language model anywhere in this. It is arithmetic over the learner's own answers, and it has to stay that way: the whole product here is that the session can be explained before it is sat, and a model that cannot show its working could not do that.

What decides which questions

Five rules, then one multiplier.

1. Unseen material is most of the session. A question never met teaches more than one already answered, so it takes every slot review is not holding.

2. Review holds up to 40% of the session, and only what is due. MAX_REVIEW_SHARE. Due means recall has decayed below DUE_RECALL — which is everything ever answered wrongly, and everything answered correctly more than about three and a half weeks ago. This used to be "recycle when the unanswered run out", which on a bank of nearly three thousand questions meant a learner never saw a repeat: no spaced repetition at all. The cap is the other half of it — somebody handed twenty questions they have already answered does not come back.

3. Within either pool, highest value first. For unseen material that is the topic's impact, (1 − accuracy) × blueprint weight. For review it is (1 − recall) × blueprint weight.

4. Each pick halves its topic's priority. CATEGORY_DAMPING. Without it a session of twenty becomes twenty questions from the single worst subject — and a learner with no history at all, whose topics are all equally unknown, gets handed the heaviest domain entire rather than a spread.

5. And how hard the question is follows how the learner is doing on that topic. difficulty_fit(). Readiness under 0.45 wants easy, up to 0.78 wants medium, above that hard; a question at the target level scores in full, one step away keeps 0.72 of its value, two steps 0.5. An unlabelled question keeps 0.92 — unknown is not wrong.

Three things about this are deliberate.

Per topic, not per learner. Somebody strong on growth and weak on arrhythmias is the normal case, and one number across the two describes nobody. The target is computed from the same shrunk accuracy that ranks the topic.

A multiplier, not a filter. Thinning the bank to one level makes it three times smaller, and on a narrow topic that is the same eight questions every time. Nothing is ever excluded for being the wrong level; it is preferred against — the same reasoning as the reranker being a permutation.

The middle is the default. A topic with no evidence sits at exactly NEUTRAL_RECALL, and the first band's edge is 0.45 rather than 0.55 precisely so that untouched topics get medium questions. Handing somebody the easiest questions in a subject nobody has measured them on is a poor way to find out what they know and a slow way to start.

This could not exist until 2026-09-12, when classify_question_difficulty labelled the bank: the column had been NULL on all 2,924 rows since it was added, so difficulty was a filter a learner could set and nothing the session ever did on its own.

And all of it is scaled by the topic's share of the real paper. Weakness alone said that being weak at something worth 5% of the exam and something worth 1% were the same problem. They are not. exam_blueprints.weight holds what the examining board publishes and blueprint_weights() loads it. A topic the blueprint does not cover takes the median of the published weights: a zero would make unmapped material unreachable, the highest would make it the priority, and neither is a claim the blueprint supports.

With no study objective set, or an objective carrying no weights, the multiplier is absent and selection is about weakness alone.

Only completed, non-expired, non-course attempts count. The pool is scoped to the learner's active exam, the same scope the bank and search use.

How time enters it

EVIDENCE_HALF_LIFE_DAYS = 30. Every answer's weight is 0.5 ** (age_days / 30).

Why exponential. A fixed window was the obvious thing and is wrong in a way that shows: it makes an answer twenty-nine days old count in full and one thirty-one days old count for nothing, so a topic crosses a cliff overnight and the ranking lurches without the learner having done anything. Exponential decay is also memoryless — an answer's weight depends only on its own age, not on what has been answered since — which is what keeps two consecutive sessions consistent with each other. A power law fits very long retention slightly better, but it needs an arbitrary offset to avoid a singularity at age zero and a second parameter nothing here could justify. One named half-life describes the whole curve.

Why thirty days. About the turn of a revision cycle. A ninety-day-old answer keeps an eighth of the weight of a fresh one, so what was missed last week clearly outranks what was missed in spring, while a topic revised last month is not written off as forgotten.

Two things decay:

• A question's recall. NEUTRAL_RECALL + (settled − NEUTRAL_RECALL) × decay, where settled is 0.85 after a correct answer and 0.25 after a wrong one. Both decay towards a coin flip, not towards zero: forgetting a right answer does not turn it into a wrong one, and time does not turn a wrong answer into a right one. Both end up saying nothing, which is exactly when the question is worth asking again.
• A topic's accuracy. Each answer counts for decay(age) of an answer, and PRIOR_ANSWERS = 2 answers' worth of "no idea" is mixed in. Without that prior a single correct answer made a topic 100% known and it never came back, which is the one thing a ranking claiming to decay must not do.

The prepared session

prepared_session.prepare_session — one action that produces a session tuned to this learner and a plain statement of why, before anything is written.

• GET /questions/builder/prepared?count= returns the plan. Nothing is written, and the question ids are withheld: the plan is for reading, and handing out stem identifiers is how a preview becomes a way to enumerate the bank.
• POST /questions/builder/prepared recomputes the plan, builds the test from that plan's own question ids, and returns the plan alongside the created quiz. So the account handed back describes the session by construction, not because two calls happened to agree. A learner who changed the length gets the plan for the length they chose.

The preview is a forecast, and it is accurate because the ranking is deterministic: candidates are scanned in question-id order and every tie resolves to the lowest id. The cold-start blueprint draw, which is random, is seeded on (user, length, date) for the same reason.

What the plan says. A one-line summary, a length and why that length, and one row per discipline — count, how much of it is new versus review, the learner's accuracy there, the topic's share of the paper, and one line of reasoning. Rows are grouped by top-level category because that is the vocabulary Analysis already reports weaknesses in; leaf topics would be twenty rows of one question each. Every count in the plan is tallied from the questions actually chosen, never predicted from the ranking — see test_the_plan_describes_the_questions_actually_put_in_the_session.

The reason is the strongest true fact, in this order: the blueprint share (cold start), a weak area, due for review, not attempted yet, the blueprint share. A weak topic is usually also due, and the accuracy is the more useful of the two. "Weak area" needs WEAK_EVIDENCE_ANSWERS = 3 decayed answers behind it — below that the prior is most of the number, and a topic last answered correctly in the spring would otherwise be reported as a weakness on no evidence.

Length is the learner's own median finished session, clamped to 5–60, counted as answers recorded rather than questions offered. The median rather than the mean: one abandoned session and one marathon pull a mean somewhere neither of them is. Until SESSIONS_BEFORE_LENGTH_IS_PERSONAL = 3 sessions are finished it is 20, and the plan says so.

No history degrades honestly. A learner who has finished nothing gets a paper drawn to the exam blueprint — exam_blueprint.sample, the same code behind algorithm: "blueprint" — and is told in as many words that this is a spread across the exam rather than a personalisation. No exam or no blueprint falls back to the damped impact spread over the whole bank, and says that instead.

What it is still not

Weight scales weakness; it does not replace it. A topic you are certain of does not surface because it is worth 5% of the paper — (1 − accuracy) is near zero and no multiplier rescues it. That is deliberate: this picks what to study, not what the paper is made of.

Dealing a paper shaped like the real exam is a different algorithm — algorithm: "blueprint", _blueprint_test → services/exam_blueprint.py.

Refinements still open

In the order they are worth doing:

• Adapt the difficulty, not only the count and the mix. Question.difficulty is on every row and nothing in the ranking reads it. A learner at 40% on a topic and one at 80% get the same questions from it.
• Use every category a question is filed under, not only its primary. The ranking reads question_category_id alone, so a question's extra links do not influence which topic it counts as. The plan's grouping has the same limit.
• Let time answer. A question answered correctly in ten seconds is not the same as one answered correctly in four minutes, and seconds_spent is recorded on every answer already.
• Let the half-life be per learner. Thirty days is a defensible constant and a worse fit than a rate fitted to somebody's own forgetting, which their answer history could estimate once there is enough of it.