Measured first, by the ped-ai session, fifteen runs of five prompts with
the gateway cache bypassed. Retrieval was already deterministic:
identical shortlist and identical scores every time, and the citation
checker stripped none of the 45 markers written — invented citations are
not the problem here. Generation was the whole variance. At temperature
0.3 the same sources and the same prompt gave answers differing by
15-70% of their text; one differential swung between a 35-word uncited
paraphrase and a 180-word cited list.
So temperature 0 and a seed. Temperature 0 alone was not enough — three
runs still differed — and temperature 0 with a fixed seed came back
byte-identical. The seed is derived from the question, normalised for
case and spacing, so two people asking the same thing get the same
answer and a different question is not pinned to the same sample.
An empty reply is asked once more before it becomes a 502. One in
fifteen came back empty from a healthy model in 4.9 seconds — not a
refusal, not an error, just nothing.
A short query that finds almost nothing is retried against the nearest
article title. "kawasaki criteria" finds fourteen sources; "kawasaki
critera" found none — the lexical ranker cannot match a token that is in
no index, and the embedding of a misspelling is not near the embedding
of the word. Trigrams do not care: that typo scores 0.36 against
"Kawasaki disease" with the next article at 0.11, and the gap is what
makes it safe to act on. pg_trgm is created at startup beside vector,
with a migration for the record.
And an answer drawn from the library must cite it. Not a hallucination
guard — nothing was stripped in fifteen runs — but one answer used the
sources and cited none of them, which leaves the learner an assertion
and nowhere to check it.
Also, article drafts are weighted towards mechanism, in the wording the
ped-ai rewriter is using, so the two lanes read alike: why the body does
what it does, with features and management explained through it rather
than listed. Figure lines and cross-references survive a refine.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
Both were this app doing an identity provider's work. Sign-in and
sign-up happen at sso.pedshub.com now: it takes the address, sends the
code, checks it, and knows about second factors — none of which belongs
here, and two of which were never done here at all.
Gone: services/invites.py, services/login_codes.py,
routers/login_code.py, the two models, the three admin invite routes,
the invite_only flag and its switch, the invite field on both sign-up
forms, and the code half of the sign-in page — which was the primary way
in and is now a button that says "Sign in with PedsHub SSO". The
password form stays for a site with no provider configured.
Migration r7b8c9d0e1f2 drops invite_codes (three spent rows) and
login_codes (empty). The dump beside it has both.
585 tests, and the contract snapshot is 320 routes — five fewer, all
five named in the diff so the removal is reviewable rather than
discovered later by a client.
Also: "Make a deck" in the signed-in menu and on the landing page, going
to the scribe's My Resources at app.pedshub.com/#resources. Same
sign-in on both sides; the arrow says it leaves.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
571 categories, 21 uploaded documents, 14 articles, 8 card decks, 30
shared tests and 2 questions carried somebody's name — mostly
daniel@danvics.com, which is not even the working administrator any
more. So "who may edit this" partly depended on who happened to create
it, and handing the site to somebody else would have meant rewriting
every one of those rows.
Migration q6a7b8c9d0e1 empties those owner columns and makes them
nullable, because ownerless is now a legitimate state and a NOT NULL
owner is exactly what forced a name onto every row. Nothing is deleted
and nothing moves. What keeps its owner, deliberately: attempts, notes,
favourites, collections, folders, study-plan progress, and the quizzes
that are somebody's own sittings rather than shared bank tests.
study_plans needed nothing — it never had an owner column.
Then the code, so it cannot grow back. Authorship is no longer a way in
anywhere: may_edit_question and can_edit_article ask the role and the
grants and nothing else; the article draft, status and delete paths lost
their "or you wrote it" arm; decks are the bank's, so an educator
reaches any of them and a learner reaches the shared ones; documents are
the corpus, so they are editors-only rather than "mine"; and every
creation path writes user_id NULL. The bank listing's "mine" facet went
with it — it counted nothing and could only ever count nothing.
Verified against production as a real learner account: every bank write
403s, admin settings 403, documents empty. As an admin, everything
opens.
Also: a category grant no longer offers Editorial in the menu. It offers
Questions and Images, which is what a grant covers; Editorial is the
whole library's review queue and its route is moderator-only, so the
entry was a door that answered "Not yours to open".
Six tests changed rather than deleted — they asserted the old model, and
each now asserts the new one: writing an article does not make it yours,
writing a question does not make it yours, an answer image is not opened
by authorship, the tutor is not opened by authorship.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
**The API.** Every route now lives under `/api/v1`, with `/api/...` rewritten
onto it — one route, two spellings, so they cannot drift and the OpenAPI
document describes each endpoint once. Errors carry an `error` object with a
stable code, one human sentence and, for a validation failure, the fields that
were wrong; `detail` is untouched so nothing that reads it breaks. The whole
surface — 320 routes, their parameters and their status codes — is checked in
as `backend/tests/api-contract.json`, and a test fails on any difference,
naming the routes that moved. `docs/api.md` is the contract in prose.
**Refresh tokens**, so an app can stay signed in without keeping a password.
Rows rather than signatures: listable, withdrawable, stored as hashes, rotated
on every use. A spent token coming back ends the whole session, because a theft
and a replay look identical from the server and the safe reading is the unsafe
one. A browser is not given one — it has nowhere to put it and a person to ask.
**An end-to-end stack**: `docker-compose.test.yml` with its own Postgres and
Redis, `e2e/seed.py` for the smallest world the tests name, and Playwright with
five projects — desktop, iPhone, Pixel, iPad and a browserless API project.
Devices because every bug reported this week was a phone bug found by a person
looking at a screenshot; a desktop-only suite would have passed through all of
them. Forty tests, five clean runs.
It found four things in its first hour:
- **A fresh deploy could not start.** `create_all()` ran before
`CREATE EXTENSION vector`, so any database that had never had pgvector
installed died on the first table with a vector column. Invisible here
because this one has had the extension for a year.
- **A figure in a published article was a 404 for everyone but an admin.**
Media in the library is nobody's to read by default, and nothing made an
exception for a drawing an article actually shows — so every illustration
added this week was an empty box for every real user.
- **Every rate limit was one bucket for the whole site.** The backend saw
nginx's address for every request, so ten bad passwords from anybody locked
out everybody, and no log line could say who. nginx now takes the real
address from the proxy and overwrites the header on the way in; uvicorn runs
with --proxy-headers.
- **The reading page's breakpoints disagreed** — 1150px in the component,
820px in the stylesheet. Between them the menu button claimed the contents
drawer and then toggled a class on a rail that was still in the layout: the
contents did not open and the site menu did not either. The button was dead
on every tablet.
And two smaller ones: the login limiter counted successful sign-ins, so eleven
people behind one hospital NAT locked each other out — it is cleared by a
correct password now; and `/uploads/{path}` served GET and HEAD from one route
with one operation id, which makes every OpenAPI client generator refuse the
document.
The first admin's password is generated and printed once at first start when
`DEFAULT_ADMIN_PASSWORD` is blank, rather than the account not existing:
`docker compose logs backend | grep -A3 "FIRST ADMIN"`.
CI (`.forgejo/workflows/tests.yml`) runs the backend suite, the contract, the
frontend suite and the build on every push to dev, main or master, and the
end-to-end stack on those branches and on pull requests into them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
There will be no courses. What was there: one draft called "jk" with two empty
lessons, and 4,000 lines of code around it — courses, modules, lessons,
enrolments, per-lesson progress, SCORM, BigBlueButton, completion certificates,
three React pages, a router, two models.
Its real cost was everywhere else. Every query that measured practice had to
remember `Quiz.course_id.is_(None)`, and forgetting it in one place would have
silently mixed course attempts into a learner's analytics; the bank predicate
carried a subquery to exclude a course's own questions from every search,
recommendation and share; quiz access had a second, parallel rule about
enrolment. All of that is gone, so the remaining rules say what they mean.
`quizzes.allow_review` goes with it. It was only ever enforced for a course
quiz, so it had become a promise nothing keeps — the public session page was
still offering "no answer review" about sessions that review fine.
The fixtures' question 5 lived in a course quiz and stood for "a question that
exists but is not in your bank". There is no such thing now — a question is in
the bank unless it is deleted — so the counts it kept out of the numbers are
back in, and the tests that turned on it now turn on deletion or on the
attempt that actually holds a question.
Files the LMS uploaded stay on disk and stay protected: LEGACY_LMS_PREFIXES in
app/utils/upload_access.py is what keeps them unreachable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
Three things the card system did not have.
**Spaced repetition.** There was none. "Known" and "to review" were React state
that vanished on reload, so a deck of two hundred was two hundred cards every
time and the only spacing was whichever cards a learner remembered to skip.
Verdicts are now kept, and the deck comes back in the order the learner's own
history calls for: due first, most decayed first, then never seen, then the
rest — because somebody who has met the whole deck recently should still get a
deck rather than a screen saying come back on Thursday.
It borrows the question player's arithmetic rather than choosing its own.
`recall_probability`, `DUE_RECALL`, the thirty-day half-life: two schedulers
with two ideas of "due", in one product that shows a learner one readiness
number, is how the number stops meaning anything. Two outcomes and no
four-point scale — a scale asks a learner to rate their own recall in units
they have never calibrated, and the extra resolution is noise.
**Cards are prose.** Both faces go through the same renderer as everything
else, so a card can carry `[[264|respiratory failure]]`, a `==key point==`, a
teaching tip or a figure. That is most of what "link cards to things" turns out
to mean.
**A deck is reachable from the question.** Beside the topic-reading chip under
the correct answer, one chip per linked deck. Read from the question's end
only, deliberately: a card that listed the questions it belongs to would hand a
learner revising the deck the shape of the exam, and the answer with it.
Migration m3d4e5f6a7b8.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
A 2,000px radiograph written into an article rendered at whatever width it
happened to be — a wall of greyscale in the middle of a sentence, four
megabytes to draw it, and no way to look at it properly. Every image in prose
is now a 256px thumbnail with the author's label under it, and a click gives it
the screen.
The viewer puts what is known about the figure beside it: its title, what it
shows, and where it came from. `media_assets` gains `source` and `source_url`
for that — a citation belongs to the file, because the same figure used in
three articles is cited the same way in all three, and a licence that turns out
to be wrong is one row to fix rather than three paragraphs to find. Asked for
when the figure is opened, not when the page is drawn.
And `overlay`: the regions an educator has marked, as vector shapes in
normalised coordinates on the unit square, so one drawing is correct in a
thumbnail, in the viewer and on a projector. Off until the learner turns it on
— marks shown before they have looked answer the question for them. Vectors
rather than a second burnt-in picture, for four reasons written down in
docs/image-overlays.md. The tool that draws them is next; this is the storage,
the contract and the reader's half.
On a narrow screen the description stacks above the image rather than beside
it, where it can be read before scrolling to the picture.
Also here: `classify_question_difficulty`, which labelled all 2,924 questions
in batches of twenty-five against a written rubric — 622 easy, 1,634 medium,
668 hard, no failed batches. The column had been NULL on every row since it
existed. Migration l2c3d4e5f6a7.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
"Questions filed there later are not added" was the honest description of what
the previous commit built, and it was the wrong thing to build. "The Cardiology
article covers the Cardiology questions" is a standing statement about the
material, not a snapshot of who happened to be filed where on the afternoon
somebody pressed a button — and a copy stops being true the first time a
question is added, silently, with nothing on any screen to say so.
So the claim is now stored, and it is what writes the links:
* `question_article_links` is still the **only** table anything reads. No count,
no QBank button, no mirror panel on a question, no AI Mode boost learns a
second question to ask.
* `article_topic_claims` records *why* some of those rows exist, and is the one
place that makes them — when the claim is staked, when a question is filed
into the category (single, bulk, or on create), and on a half-hourly sweep
that catches whatever bypassed both.
A link made this way is an ordinary row and can still be deleted by hand; a
sweep puts it back, which is the honest consequence of a standing claim.
Dropping the claim is how you stop it, and the panel now lists what an article
follows with two ways out — stop following and keep the links, or stop and
remove them.
Migration k1b2c3d4e5f6.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
There was no way to delete an article from anywhere in the interface. The API
had one; the only route to it was curl.
Now there is a control at the foot of the editor, and it does one of two things
depending on the article's history — and says which before it is pressed:
* A draft that was **never published** is deleted outright. There is nothing to
restore, and a trash full of abandoned stubs is a second list to maintain.
* Anything that has been published, even once, is **marked** and appears in the
trash on Editorial, restorable exactly as it was. Somewhere there is a
learner's note against one of its sections, a question linked to it, and a
link somebody sent a colleague; a DELETE typed in the afternoon should not
settle any of that.
`first_published_at` is what decides, stamped on the first publish and never
cleared — unpublishing does not make an article unseen, so it does not make
deleting it safe either. Backfilled from `reviewed_at` for everything currently
published, because an article with a null stamp reads to the rule as a
never-published draft.
A binned article is out of the listing, the editorial queue, every slug and id
lookup, and — immediately — the search index, so it cannot still answer a
learner's question from the trash.
Also on Editorial, because a hundred rows is a queue you work through and not a
page you scroll past on the way to the next queue: each bucket keeps its own
box, its own scrollbar and its own filter.
And the editor finally has a way out that is not Save: Back and Discard, with
an inline confirmation when there are unsaved changes. The way out was the
browser's back button, which throws the sitting away without saying so.
Migration j0a1b2c3d4e5.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
Four things that share a spine, so they arrive together.
**Folders.** A hand-picked set of questions, and the fourth thing a grant can
name beside exam, discipline and category. Deliberately not `user_collections`
with a sharing flag: a library is a consequence of access — you save what you
can already see — while a folder is a source of it, and one table holding
thousands of private lists beside a handful that confer permission is one
mistake away from a leak. Built from the question manager, granted on /access.
Membership stays with the owner and moderators so a grantee cannot widen their
own reach, and deleting a folder takes its grants with it.
Two live constraints had to be rewritten to accept it: `ck_grant_has_a_dimension`
and `uq_grant_dimensions` both predate `folder_id`, so a folder-only grant
failed the check and two folder grants collided on the unique index.
**Per-question feedback.** The learner's half already existed. What was wrong
was who could read it: any grant at all let an educator list and delete reports
about the whole bank. Reports are now scoped by `question_scope_predicate`, the
same predicate that decides which questions that educator can see, and a reply
thread makes the report a conversation the learner can follow rather than a
form that swallows what they said.
**Per-section notes and article feedback.** Two tables on purpose:
`article_section_notes` is private to whoever wrote it, `article_feedback` goes
to whoever maintains the article. Both point at the section id inside
`articles.sections` rather than at `article_section_index`, whose rows are
dropped on unpublish — a cascade from there would delete a learner's writing
because an educator took an article down for an afternoon. A rename keeps a
note attached; a deleted section leaves it marked orphaned under the heading it
was written on, for its writer alone to remove.
The header's feedback badge covers both, because questions and reading are the
same job to whoever is doing it.
Migration i9f0a1b2c3d4. 556 backend and 572 frontend tests pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
Single sign-on wrote a random string nobody would ever know. That reads as
"has a password" to everything that asks — so Settings demanded a current
password before it would let those accounts set their first, and the only way
through was to click "forgot password" for a password they never had. The same
trap was waiting for anybody who only ever signs in with a code.
Null says the true thing. Signing in refuses an account with no password the
way it refuses a wrong one, because which accounts have one is not a question
that endpoint answers. Setting a first password asks for no current one;
changing an existing password still does. `/auth/me` reports whether there is
one at all and nothing about it, because Settings has to choose between "Set a
password" and "Change password" and cannot tell from the outside.
The random strings already written are left alone. They are unguessable, so
nothing can sign in with them, and clearing them would mean deciding from
outside which accounts were meant to have one.
Identity is the email address throughout, so the three ways in are three ways
into the same account: single sign-on, a code, or a password — and a person may
acquire or drop the third at any point without losing the other two.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
A password is a thing to remember and a thing to lose. Somebody who can read
their own mail can now sign in without one: ask, receive six characters, type
them into the page that is already open.
A code rather than a link, and the difference is not cosmetic. The token in a
link was 256 bits, unguessable however long it lived, so its length, its expiry
and its rate limit were three independent decisions. Six characters is 2^30,
and the three stop being independent — so they are argued together:
* six characters of the invite alphabet, imported rather than copied, because
there should be one answer to which characters a person may be asked to
retype and that one already drops O/0 and I/1;
* a code answers five guesses and is then retired, not slowed — whoever is
typing has lost the mail or does not own it, and both are one click from a
new one;
* one code live per person, since several would mean one guess tested against
all of them;
* ten verify attempts per address per fifteen minutes, so nobody buys five
fresh guesses at a time by asking again.
Tens of guesses an hour against a billion, and the victim gets a mail for every
code burned. Eight characters would buy a thousandfold against an attack the
guess budget has already ended, and cost every person two more characters.
The attempt count lives in the row, not the cache. The Redis limiter fails open
when Redis is down, which is right for what it usually guards and wrong for the
only thing standing between a patient stranger and six characters.
Verifying is scoped to the address. A short code looked up on its own would be
tried against every code live on the site at once — the short code's one real
weakness, closed by knowing whose code it should be before comparing.
Fifteen minutes, because a first mail between strangers is routinely greylisted
five to ten and a code that expires before it arrives is not a sign-in method.
Shortening it buys nothing: one code is live and it answers five guesses
however long it sits there.
Nothing distinguishes an address with an account from one without — same
message, same status, same duration, and both rate limits counted before the
account is looked up, so a 429 cannot become the tell. Redis keys are
fingerprints, and the table holds a fingerprint rather than the code.
SSO stays first where it is configured, and a password is still one click away
for anybody who has one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
Three things landed together; the message names all of them, because a commit
that mentions one is a commit nobody finds the other two in.
**Figures.** Thirty-four JPEG 2000 files — 21 on questions, the rest unattached
in the media library — are WebP now, with `questions.image_path`,
`questions.explanation_image_path` and `media_assets.path` repointed together.
Serving already converted them on the way out, so nothing was broken; this
removes the step and makes what is stored the same thing that is served. The
originals stay: they are the only copy of what came out of the PDF, they cost a
few megabytes between them, and a conversion nobody can undo is not one to run
against a live bank. Paths are found by what the columns say rather than by
listing a bucket, because three tables record them and updating two would be
worse than none.
**The openai SDK is gone.** Ten call sites — one more than the map said, the
Celery article drafter — every one of them a POST with a JSON body, and not one
reading usage, cost, tool calls or logprobs. Every other call to the same proxy
was already plain httpx: embeddings, the ChromaDB embedding function, speech
both ways, model discovery, the vision probe. So this deletes an abstraction
rather than swapping one for another, and leaves one HTTP client instead of
two. `chat()` and `achat()` return the message content; a `ProxyError` carries
the status and the first 500 characters of the body, which is where the proxy
explains itself.
Behaviour is preserved deliberately, including a 600-second fallback timeout
for the four call sites that were running on the SDK's ten-minute default.
Lowering that is a real change and belongs in its own commit.
Proved against the live proxy on both services rather than only against mocks:
a completion, an async completion, a real 400 the vision probe still classifies
as a refusal, 407 models read from the catalogue, and a word read off an image.
**Voice.** A chosen voice is honoured whatever serves it. The prefix check only
accepted a locally served one, so a site adding a hosted voice would offer it
in Settings, save the learner's choice, and then quietly read every question in
the default voice. The list has always come from the database — adding a voice
is a row in Settings → AI models, never a code change.
And the sign-in page stops offering a locked door: `signup-policy` reports
whether registration is open at all, and the Sign up link goes when it is not.
The switch existed and the only way to discover it was to fill the form in.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
The voice picker was a dropdown in the quiz player, beside the question — the
one control on that screen with nothing to do with answering it, and one a
learner sets once and never touches. It is a setting now, on the user rather
than in a Redis blob, with a play button beside each voice because a voice is
worth hearing before it is chosen. Choosing nothing stays a real choice: it
means whatever an administrator marked default, so a site that changes its
default reaches everybody without a row being edited.
The tutor reads figures from `question_media` rather than the two legacy path
columns. Those agree exactly today, so nothing was being lost — the first
question given a second figure in the editor would have been the one that
broke it, silently and only for the tutor. The legacy columns remain as a
fallback for anything not projected into that table yet.
And the retrieval thresholds are written down in docs/retrieval-thresholds.md:
the three answers, the sixteen queries they were measured against, why they are
deliberately not the retrieval floor, and how to re-measure when the corpus
grows. Worth keeping the headline in mind — "discuss love" scores 0.491,
alongside "tell me a joke". A number in the 0.4s is noise, not a weak signal.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
`search_vector` weighted title, summary and `content` — but `content` is NULL
for 323 of 331 articles, because everything the generator writes goes into the
`sections` JSON and only the eight hand-seeded samples ever used the column. For
98% of the library the body contributed nothing to full-text search, so a term
that appears only in a section — a drug name, a diagnostic criterion, an
eponym — returned nothing, and did so silently.
A generated column cannot contain a subquery, so the extraction is an IMMUTABLE
function it can call, and `content` stays in the expression for the eight that
use it. Proved rather than assumed: "supraglottoplasty" appears in no title or
summary in the corpus and now finds Laryngomalacia; before this it found
nothing.
Uploads are capped at 2 MB rather than 10. A document here is a query, never
content — read once to find matching questions in the bank and then
discarded — so the cap is about how much text is worth reading, and past two
megabytes somebody is uploading a textbook.
The previous commit's message covers only the litellm removal; it also carried
the 36 rewritten article summaries and the prompt rule behind them, which were
finished in the same window.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
A library holds both articles and questions now. It held only questions, so the
bookmark on an article had nowhere to write and stood in for the questions
filed under the topic instead — which is not what a reader who saved the
reading asked for, and left a topic with no questions unsaveable. Its own
table rather than a nullable column beside `question_id`: that shape allows a
row with both or neither, and every read then has to say which kind it is
looking at.
Which libraries already hold an article is now asked of the server, as one
question. It was kept on the device because the API could not answer, which was
wrong on the second machine and silently so. Putting one back is the same
control rather than an undo somewhere else.
"Short" is called Summary, because that is what the section is called, and it
is a toggle rather than one tab of three — the whole topic, or the part of it
worth revising, which is a different kind of choice from Long versus Clinical.
It names its own state, so a reader can tell why two thirds of the contents are
not there. The stored variant stays `short`: renaming it would be a data
migration to change a word on a button.
Also: `litellm==1.28.13` has been withdrawn from PyPI, so requirements.txt
could not be edited at all without the pip layer failing to rebuild — which is
what blocked pinning Pillow. Repinned to 1.53.1, the nearest still published;
the three things we use are unchanged in it, and both suites pass on the new
set. Pillow is pinned properly now rather than arriving through PyMuPDF.
One consequence, handled: `litellm.utils.get_valid_models()` now returns
nothing unless a provider's own API key is in the environment, and ours is a
proxy. That branch is only reached when no proxy is configured, and it now says
so instead of answering with an empty list that reads as "this site has no
models".
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
`Question.is_shared` defaulted to 1 and was only ever set by a route nothing
called, so in practice it divided the bank into "everything" and "everything,
plus your own private ones" — a distinction that cost every recommendation
denominator a join and never changed an answer. Who may reach the bank is the
site's own access rules; who may manage a question is the category grant tree.
So the two predicates the whole bank was built on are now the same thing, and
say what they actually mean: a question is out of reach if it has been deleted
or belongs to a course. Nothing else. The column is dropped, the route that set
it is gone, the bulk "share" action with it, and the Private tile and pill go
from the question manager.
The tests that turned on it have been rewritten rather than deleted, because
the rule they were really about survives: revoking a question still revokes
every session carrying it — by deleting it, which is the only revocation left.
Several others named a category holding exactly two reachable questions and
then answered two particular ids; that category holds four now, so they name
the pair instead. A session's own sharing flag is untouched — that is a
different thing, and it is still how a session is handed to somebody.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
A question reached a system through a symptom keyword it happened to
mention — question → keyword → parent system — and only 726 of 4,281
keywords had ever been given a parent. The Systems tab saw 1,492 of 2,924
questions while Disciplines saw all of them.
The system now sits on the category: question_categories.system_id. Every
question has a category, so every question reaches a system. 2,919 of
2,924, and all sixteen buckets have real content.
It stays a third way of asking rather than the discipline tree relabelled
because a topic's system is assigned separately from where it sits in the
tree. scripts/assign_category_systems takes the discipline as a default
and lets the topic's own name overrule it, which is exactly the case that
makes the axis worth having: conjunctivitis is filed under Infectious
Disease and is an eye, osteomyelitis is filed there and is a bone. 110 of
660 topics were decided that way.
Two regex traps caught in the dry run and fixed before applying:
"adRENAL" matched the kidney rule, and "Abnormal Uterine Bleeding" matched
the bleeding rule. Both now have a specific rule above the general one.
I first tried to fix this by parenting the orphan keywords to systems,
deriving each keyword's system from the questions carrying it. The dry run
showed why that was the wrong shape: it reached only 534 of 3,555 orphans,
and inherited every coarse edge of the discipline map — conjunctivitis came
out as Multisystem because conjunctivitis questions are filed under
Infectious Disease. That script is left in place, unapplied, as the record
of a measurement worth keeping.
No ForeignKey on system_id in the model: question_tags is a raw-SQL table
with no ORM class, and declaring one leaves every metadata build unable to
resolve it. The constraint is real in Postgres.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
Favorites and the question libraries in one place. Card and Table views
with the choice remembered, sort by last used / created / name / size
with a direction control, a count line, and a search over name and date.
Favorites leads as a fixed row: it is the one shelf nobody made and
everybody has, so it cannot be renamed or deleted.
Sorted by when each was last used, not when it was made — the order
things were created in is nobody's mental model of their own shelf. A
library nobody has opened falls back to its age, because it is newer to
the learner than it is to the database. That needed
`user_collections.last_used_at`: null on every existing row, since
backfilling from created_at would invent a use that never happened.
A shelf opens in place rather than linking away. The obvious link would
have been /questions?collection=N, and there is no page there that reads
it — the old bank browser was dismantled — so the card would have led
nowhere. Questions can be taken back out from the open shelf, and any
shelf can be sat as a session through the existing explicit_ids builder.
The ⋯ menu moved out of QuizPage into components/MoreMenu; the player
keeps its own look and its own children through className props. It no
longer closes on any click inside, which the player's feedback form and
share dialog were relying on by accident.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
Opening a tip before answering is a nudge. The answer that follows is
still right — it is counted as right, and the percentage is not docked —
but it is not the same as right, so it keeps its own arc on the donut and
its own line in the legend: "3 correct after a tip".
attempt_answers.used_hint records it. The player reports which questions
had a tip opened before the answer went in; a tip read afterwards is
revision and does not count, which is the difference two of the tests
turn on. Both endings agree about it — an explicit submit carries the
list, and an exam that runs out takes it from the saved progress, so a
tab closing cannot launder a score.
Found while wiring this: RichText declared its component overrides inline
in the render, so every one was a fresh component type and React
remounted the whole rendered tree on each render. An open tip closed
itself every time the exam clock ticked. The map is memoised now.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
Sitting the same questions again is practice, not a new measurement. You
have already seen the answers, so getting them right the second time
says nothing about whether you knew them — and it cannot be allowed to
raise a figure that means "how much of this do you know". A repeated
session is titled "(repetition)", analysed in full on its own page, and
left out of every aggregate: the overall accuracy, the per-quiz history,
the averages, and the readiness that drives recommendations.
Deleting a single session is gone — control, endpoint, tests and all. A
session is a record of work done, and removing one edits the history
every figure on the analysis is computed from, which turns a measurement
into a number somebody chose. Starting again is still offered whole,
under Settings, Your data, which takes everything rather than the parts
that flatter.
Two layout bugs behind that. The category tree kept its appearance in
QuestionBankPage.css, so it looked right on the bank and took whatever
the host page did to a label everywhere else — in the question editor
that centred the name, leaving it adrift with the count at the far
right; it owns its own stylesheet now. And the editor's grid collapsed
to `1fr` below 900px, whose automatic minimum lets one unshrinkable
child push the column past the window: the page had padding down its
left and none down its right because the right was off the screen.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
Extraction wrote straight into `questions`, so a machine's first attempt
took a permanent id the moment it was produced. Ids come from a sequence
and are never reissued: every rejected draft burned one, and every draft
that needed fixing was sitting in the bank while it was being fixed.
A run now lands in a batch of drafts with their own table and their own
sequence. They are read, corrected and decided there, and `accept` is
the only place a Question is created — a copy rather than a translation,
because every field a draft holds is a field a question has, so nothing
is lost at the moment of acceptance.
Accepting is all or nothing, and everything is checked before anything
is created: a call that reports failure must not leave questions behind
from the drafts it got through first. My own test caught that — the
first question existed before the second draft was refused.
Readiness is reported for every draft rather than only on the attempt to
accept it, so a reviewer sees what needs work before opening anything.
A decided draft keeps its row and records what it became, so a batch
reads as a history of what was decided rather than emptying as it is
worked through. An acceptance cannot be undone from here: the question
exists, and deciding twice would make a second one.
No embeddings for drafts. A vector is for finding a question in the
bank, and a draft is not in the bank.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
The number beside a folder in Topic reading was a question count while
the browser lists articles, so "Hyperinflammatory Sepsis 4" meant four
questions and opened onto no reading at all. It counts what it opens
now, rolled up over the subtree, and a branch with nothing to read in it
is not offered — a folder with a number on it is a promise.
The hover card could not be reached. Its body was pointer-events: none,
on the idea that a hint should not sit between the reader and the link —
but the card is offset below the link and never covered it, while the
pointer travelling down to Split view crossed a body it could not enter,
so no mouseenter fired and the hide timer closed it on the way. The card
takes the pointer now, with a bridge across the gap.
And clicking the words opens the card rather than the article. A
cross-reference is read mid-sentence, and navigating away to find out
whether it was worth following is the thing that breaks the thread; the
card's two controls — beside what you are reading, or a tab for later —
are how you go. That also gives touch a route, where hover has none.
Modified and middle clicks are still the browser's.
The listing sent content and sections for all 331 articles, 214KB of
prose a list never renders. It sends what a list needs, which is 21KB.
The footer sat wherever the content stopped, so a page still loading put
it halfway up the screen with background below it. The shell is a column
the height of the window.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
Four gaps, one change.
Articles could not belong to an exam at all — an article reached one only
by inference through its category, which cannot say that the same article
belongs to a basic-science step and a clinical one showing different
views in each. article_exam_links says whether it is in the group;
Exam.article_views already decided what is shown once you are there.
POST /exams/ wrote name, slug, sort order and active, and silently
dropped family, description and article views, so a new objective landed
in "Other" showing everything whatever was asked for. It writes what it
is given now, and PATCH can change it afterwards.
Membership was one link row at a time, which nobody would do for three
thousand questions. POST /exams/{id}/assign takes whole topics with
everything beneath them — questions and articles both — and is
idempotent, so widening a selection and running it again adds only what
is new.
And the point of all of it: a real paper is not a uniform draw. The ABP
publishes that 12% of a general paediatrics exam is preventive care and
2% is rheumatology; forty questions drawn evenly is forty coin flips.
exam_blueprints holds a board's published outline — its own numbering,
its headings, its weights — and blueprint_category_links maps it onto
our taxonomy rather than bending the tree to fit, because their outline
is arranged for examining and ours for studying.
The sampler uses largest-remainder, so twenty-four percentages still come
to forty questions, and a domain that cannot supply its share gives the
shortfall back to be spread over those that can — the paper keeps its
length and loses only accuracy, and the working is returned so the
shortfall is visible rather than silent.
Seeded from the ABP General Pediatrics Content Outline (Oct 2024):
structure and published weights only, no exam material. 120 lines, 22 of
24 domains mapped; Psychosocial Issues and Child Abuse and Neglect have
no category of ours and are reported rather than hidden.
Creating an objective is now an administrator's rather than a
moderator's: it appears in everyone's picker and scopes the whole bank,
which is site configuration, and it sits with the other site switches a
moderator cannot reach.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
Question ids come from a sequence and are never reissued, and fourteen
tables point at them — attempts, quiz membership, exam membership,
media, article links, notes, favourites, feedback. Deleting the row took
all of that with it, so "restore" could only ever have meant typing the
text in again as a different question.
DELETE now sets deleted_at. The question leaves the bank, the builder,
search and every share path at once, because the exclusion lives in
general_question_predicate rather than at each call site. Restoring puts
back the same id, so everything that pointed at it still does. Erasing
for real requires the trash first and a moderator, and the confirmation
says what goes with it.
The trash page holds questions instead of tests. A test is a selection
you can remake in a minute; nobody wanted those back.
Used and withdrawn invite codes can be removed — an unused one is still
withdrawn rather than deleted, so it stays visible as having been issued
and stopped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
Settings linked to a second dashboard with its own tab bar and its own
visual language. The admin sections are rendered in Settings now, under
headings that say who they are for — You, Content, The site — and each
has its own address, so People, AI models, Safety and Search are links.
/admin redirects into Settings for anyone who bookmarked it. AdminPage
takes a `section` prop and drops its tab row when embedded; it is loaded
lazily, so it is not in a learner's download.
Comments are gone: router, model, table and the half of the test file
that covered them. They were a discussion thread nobody was obliged to
answer, and feedback replaced them with a message addressed to whoever
maintains the question. The table was empty, so nothing was lost —
verified before dropping it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
The quiz player is a box the height of the window. The question used to
scroll the whole page, which took the session rail and the navigation off
screen exactly when you wanted them; now each column scrolls on its own
and the bar — Exit session, Previous, Next, Review — stays put.
Two site-wide switches, together under Settings → Site policy because
both are the administrator's and both apply to everyone:
* Sharing can be turned off. That stops new links being made; one
already handed to somebody keeps working, since revoking it would
break something a learner has already given away.
* Sign-up can be made invite-only, with single-use codes carrying a
note of who each is for and, afterwards, who it let in. A spent code
is kept rather than deleted — that record is the point of invite-only.
The alphabet has no O/0 or I/1/l, because these get read aloud.
The registration form asks for a code only when the site needs one, via
an unauthenticated policy endpoint — it has to know before there is an
account to ask with. It never says whether a given code is valid before
the account exists, which would make it somewhere to guess them. The
first account is always allowed, or a new install would lock itself out
before an administrator existed to issue a code.
Flags fall back to their defaults when Redis is down, in the safe
direction each way: sharing keeps working, sign-up does not silently
open.
Found on the way: the registration form's three labels named nothing —
no `for`, no wrapping — so a screen reader announced unlabelled boxes.
Backend 261/261, frontend 328/328.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
Comments are gone. A thread under every question was a discussion nobody
moderated, and what it was used for was telling an educator something was
wrong. That is now feedback: a private report, carrying the question id,
that someone is expected to act on.
* Give feedback sits in the question bar's new "more" menu, beside Save
and Share — occasional actions, folded away rather than each taking a
slot in a bar read on every question.
* An educator gets a badge of what is outstanding. Each row names the
question and opens its editor, where the report sits beside the field
it is about; reply, resolve, reopen or delete from there.
* Resolving keeps the report. A question with a history of the same
complaint should visibly have one; deleting is for the ones that were
never about the question.
* A granted educator sees only their own branch. The badge answers
quietly with zero for someone with no access, so the header can ask
without first working out who is asking.
The question bank is now the Qbank: create a session, and the last three
with Resume. Its facets, tag tree and create-a-quiz were a second copy of
the custom-session page; marking and folders belong in the player while
you are sitting a question. Import and export moved to the question
manager, which is the one place questions are managed, and which now has
a Preview that opens over the list instead of a page you have to come
back from.
Fixed while there: a session in progress analysed as 0/0 with an empty
table, because the analysis read attempt_answers — written on submit —
while the session list counted the saved progress. They read the same
thing now. The category trail is gone from the player: it named the
answer's own topic and led out of a session part-way through. An option's
reasoning opens on click and closes on the next one.
Backend 253/253, frontend 323/323.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
The page at /quizzes showed the same fifteen rows twice — once under a
"Sessions" tab as a list, once under a "Library" tab as cards — with
nothing distinguishing them. The navbar carried the duplication too,
with "Sessions" and "History" both pointing at the same page.
Board Review I-XII already exist as study plans. The Library tab was
showing the bulk quizzes those plans were built from, so the same twelve
titles appeared in both systems. Those quizzes are now origin='plan':
still real, still the parent of their questions via source_quiz_id, but
no longer offered as something to pick off a list. Once a learner has
actually sat one it is history, so the session list keeps it.
- QuizzesPage deleted; /sessions is the only listing
- /quizzes/* redirects to /sessions/*, preserving path and query
- submitting a session lands on its analysis, not the old score page
- the answer review drops its score hero, which the analysis owns and
stated differently; a course quiz keeps its card, having no analysis
- delete-attempt moves to the analysis page, where the session lives
Backend 208/208, frontend 246/246.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
The linking was the gap
The marker system was built weeks ago — resolves by id, survives a rename, shows
a preview on hover — and not one of 333 articles used it. Every article was
written in isolation, so a piece on croup named stridor and epiglottitis and
offered no way to reach either. `scripts/link_articles.py` reads what is written
and links it: 3,718 cross-references across 307 articles, by id, so a later
rename cannot break them.
Conservative on purpose, because a wrong link is worse than a missing one: only
the first mention in a section, whole words, longest title first so "Otitis media
with effusion" beats "Otitis media", never inside an existing link, marker,
heading, code span or table, and never an article to itself.
That exposed a second thing: the reading view had its own Markdown pipeline with
its own cross-reference regex, and it only understood the old slug form. It would
have printed every one of those 3,718 links as literal brackets. Article prose
now goes through the same renderer as the rest of the site.
Short and Clinical looked empty
Both are usually a single section, and everything starts collapsed, so the tab
showed one heading over blank space. A view of one section is not a contents
page; it opens.
Removed
Quiz reminders — emailed nudges to retake anything under 75%, with a scheduler
that existed solely to send them: the model, the service, the scheduler, the
email, the table. Article comments. The dashboard's in-progress list and its
stat cards, both of which the analysis page now answers better.
One mistake worth recording: the first pass at removing the reminder cleanup used
a regex that took 109 lines with it, including an unrelated endpoint. The test
suite caught it (`/attempts/quiz/{id}/in-progress` returning 404 instead of 403),
and the file was restored and edited by exact match instead.
208 backend, 249 frontend green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
Figures
A question could carry exactly one stem image and one explanation image, each a
bare path with no title, no legend, and no way for the prose to refer to it.
`question_media` makes a figure a row: it points at an image already in the bank,
carries a role, a label the text can name ("Figure 1"), a caption and an order,
and there can be as many as the question needs. The same radiograph can serve two
questions without being stored twice.
The 346 existing paths were backfilled into figure records and retitled —
`page_339_img_0.png` says where a file came from and nothing about what it shows,
so the filename moved into the caption where it is still searchable, and the
title became something a person can read.
On the editor question: no new platform needed. Milkdown is already installed —
ProseMirror-based, MIT, GFM tables, code blocks, LaTeX — and already used for
articles, courses and the quick question modal. Only the question *page* still
has plain textareas, and that swap is written down rather than rushed, because
the stem carries manual-highlight offsets and a WYSIWYG rewrite would move them.
Fewer hints during a quiz
The category trail and the difficulty pill were shown beside every stem. Being
told a question is filed under Neonatology, or that it is "hard", narrows the
answer before the stem has been read. Both now wait until the answer is in,
where the trail becomes a way to more of the same topic.
The dashboard is about questions
Quizzes and attempts describe how the material happens to be packaged. What a
learner is working through is questions: how many of the bank they have seen,
how many they have answered correctly, and their average. The old per-quiz
performance card — which needed two attempts before it showed anything — is
gone, superseded by the session analysis. The greeting sits above "continue your
study" rather than below it, where it read as a heading for the wrong section.
208 backend, 249 frontend green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
The results page showed a score and a wall of explanations. What a learner needs
afterwards is where the time went and what to go back to, so
/analysis/session/:attemptId gives them: a rail of recent sessions, the four
figures they act on — correct, completed, time per question, total time — a
donut, the weakest topics, and a paginated table of every question with its
status, difficulty, time and how peers did on it.
Time per question was not recorded at all, so it could not be reported. It is
now (`attempt_answers.seconds_spent`), banked when you leave a question and
including the one still open at submission — without that the last question of
every session would show nothing. Answers from before this read "—" rather than
claiming zero, and a question nobody else has answered has no peer rate rather
than 0%, which would read as everyone having failed it.
Also in this pass, from the review:
* quiz categories are gone from the library — a second taxonomy beside the
real one, putting a heading above every test;
* the board review sets are numbered rather than dated, in both the quizzes
and the study plans built from the same material, so a learner does not meet
2019 in one place and VII in another;
* the footer's standing note is one clause, and the gap above it no longer
looks like the page ended early.
Everything else asked for today is written down in docs/TODO.md rather than
half-built: resume instead of restart, an unsuspended exam that keeps running,
deleting a session's data, reset-all-data with a warning, recommendations split
by article/discipline/system, and the adaptive session. Two questions I owe
answers to are in there too.
208 backend, 249 frontend green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
The question toolbar now carries what a learner actually reaches for. An
attending tip — one sentence of the kind said at the bedside, stored separately
from the explanation because it is read before the answer is known and must not
give it away. A note of their own on that question, replacing a single global
note that was one page for everything and so was never about the question in
front of you. Saving to a folder, which the collections API has supported all
along with nothing in the player able to call it. And the share link, which
previously only appeared on the start screen.
Panels open one at a time under the toolbar; two at once would push the options
off screen.
Reset question resets one question, not the attempt: a misclick should cost the
answer you just gave, not the nineteen before it.
The clock shows session time, time on this question and the running average, in
study mode as well as exam mode — four minutes on one question is the number
that says whether you are learning or stuck, countdown or no countdown. It
pauses, because time spent making tea is not time spent thinking.
208 backend, 249 frontend green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
The objective did almost nothing
It scoped question counts and nothing else, which is why changing it appeared to
have no effect. An exam now carries a family (USMLE, COMLEX, boards), a
description, and the article views it offers, and `/exams/` reports what the
current objective actually changes rather than leaving the learner to guess.
Reading follows from it: an article returns only the views its objective allows,
so someone revising a basic-science step is never shown bedside dosing they must
not act on — a view you can open but must never use is worse than one you were
never offered. An editor still gets the whole article, because they cannot edit
what they cannot see. An objective configured to show nothing falls back to all
three; that is a configuration mistake, not a preference worth honouring.
Unused figures deleted, at the user's request
3,262 figures — 334 MB — that nothing had ever used. "Unused" was defined by
exclusion and every exclusion was checked rather than assumed: kept if any
question uses it as a stem or explanation image, if any question version
mentions it, or if it appears in article prose or a flashcard. 440 kept, and
five question figures spot-checked as still readable afterwards. MinIO is now
596 objects, 520 MB, down from 3,858 and 854 MB.
This is not reversible from the application; the nightly borg backup of the
volume is the only way back, and that is stated in the script rather than
assumed.
For the record, since it was asked: the extraction is PyMuPDF, with an MD5 skip
list for repeated branding images. It pulled every embedded image from all 18
source PDFs, which is why one 767-page document alone produced 908 of them.
208 backend tests green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
Standardises cross-references the way we agreed, and puts a CMS around articles
so hundreds of generated drafts are reviewable rather than merely present.
Links, made rename-proof
`[[7|Febrile seizures]]` resolves by id and displays the text — the id is the
part that must not change, the text is what keeps prose readable while you write
it. `[[old-slug]]` still resolves and is rewritten to the id form on save, not in
a migration: an article nobody has touched is not broken, and rewriting prose no
one asked to change is how an editor stops trusting the editor. Every slug an
article has ever had is kept, so a rename redirects instead of 404ing, and a save
reports markers pointing at nothing — at the moment the person who wrote the link
is still looking at it.
Three views of one topic
The full article to study from, the key points to revise from, the clinical view
to act from, with doses. They are views of one article rather than three
articles, so the numbers cannot drift apart and a question linked to the topic
still means one thing. Each section carries its variant; articles written before
this are the long view, unchanged.
CMS
draft → in review → published, with an author able to submit and only a
moderator able to publish. Every save snapshots what was there, restorable, and
restoring is itself snapshotted or the way back from a mistaken restore is gone.
The editorial queue is work rather than inventory: waiting for review, generated
and unread, published without sources, published with nothing to practise,
barely written. An empty bucket is drawn as good news, not as an alert.
Articles from the clinical library
The library index is 1.8M chunks of reference texts embedded with bge-m3 — the
same model PedsHub already uses, so our query vectors are directly comparable and
nothing had to be re-indexed. Retrieval supplies the facts and the provenance;
the model supplies the prose. References are built from the metadata of the
passages actually retrieved, never from the model, so a reference cannot be
invented — the same property that makes an AI Mode citation trustworthy. A topic
with fewer than three grounding passages is skipped rather than written from
memory. Everything lands as a draft.
Two things worth naming. The generated text is original writing grounded in those
books, not extracts from them: their facts are usable, their sentences are their
publishers'. And there are two Milvus servers on this host — the collection with
the data is the one reached as `milvus`, not the similarly named one on the other
stack, which I wired up first and which silently refused.
Also fixed along the way: `litellm==1.28.13` has been withdrawn from PyPI, so
requirements.txt could no longer be resolved from scratch and the image only
built because of a cached layer. Later additions go in their own layer until the
pins are refreshed.
182 backend, 223 frontend green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
The design settled earlier, built as described: retrieval decides what may be
cited, and the server enforces it.
The model is handed a shortlist of at most fourteen sources from the learner's
own library and told to cite them by marker. Afterwards every citation it wrote
is checked against that shortlist and anything else is deleted before it is
stored or shown. A hallucinated citation is not unlikely here, it is impossible
— surviving is not a decision the model gets to make. A URL it invents is not a
citation either: only the marker form counts, so a plausible-looking link stays
in the prose citing nothing.
Retrieval reuses the hybrid search already in place, and each corpus keeps its
own visibility rules — the bank predicate and exam scope for questions, the
draft rule for articles, deck ownership for cards. A question source carries the
stem only: a chat that printed the answer would hand away the practice it exists
to prepare you for.
Curated links do the job they were built for. A retrieved row an educator tied
to another retrieved row is boosted, because two things somebody already linked
surfacing for one query is evidence rather than coincidence. Nothing is stored
for this; the boost lives only in that ordering, and the answer marks those
sources so the reader knows which claim rests on an educator's judgement rather
than on a ranking.
Citations are stored with the answer as filtered, so reopening a thread shows
the links it showed at the time rather than a fresh retrieval that may now rank
differently. In the page the markers become numbers and each number opens its
source; a section citation deep-links into that section.
Two smaller decisions worth naming: a question appears in the thread the moment
you send it and is handed back to the input if the answer fails, because typed
words are not something to lose on a 502; and someone else's thread returns 404
rather than 403, since whether it exists is not your business either.
182 backend, 206 frontend green — 16 of the backend tests are the citation
contract and the retrieval boundary.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
Thirteen plans were seeded with an API to serve them and nothing that called it,
so the whole feature existed only in the database. Two pages and the editing
endpoints it was missing.
/study-plans lists the plans with progress stated in blocks — "3 of 6 blocks"
is something you can act on, where "50%" only tells you how you feel about it.
/study-plans/:id is one plan: each block shows Articles, then Sessions, in that
order, because that is the order the block is meant to be done in.
Reading is now part of a block (migration f4a5b6c7d8e9). "Mark as read" is the
learner's own claim and reversible — someone who ticks the wrong row should be
able to fix it without an educator, and progress nobody can correct stops being
trusted and then stops being used. It is a separate table from `article_views`
on purpose: opening an article is not the same claim as having finished with it.
A draft article attached to a block is listed for the educator who can open it
and left out for everyone else, rather than offered as a dead link.
Editing is inline on the learner's own page rather than a separate builder, so
the thing being changed and the thing a learner sees are the same object.
Moderators create (as a draft — an empty plan is not something to put in front
of anyone), rename, publish, delete; add, rename, reorder and remove blocks;
move questions between blocks of one plan; attach reading found by searching
rather than by id.
Two places where the obvious implementation leaves the data wrong, both tested:
deleting a block out of the middle shuffles the survivors down, or the next
insert collides with a position nothing occupies; and reordering parks every row
outside the range before writing the real positions, because (plan_id, position)
is unique and the first move would otherwise collide with a position still held.
A partial order is refused rather than half-applied.
166 backend, 188 frontend green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XeFQJXJTfHKTfbfsdxv57Z
Systems were never systems
The 27 top-level rows were disciplines and care settings — Cardiology,
Emergency Medicine, Neonatology, and a stray condition (Sepsis) — not organ
systems. Cardiology is a discipline; Cardiovascular System is a system. So the
facet was mislabelled, and there was no organ-system axis at all.
Both fixes, as asked:
* that tree is now the "Topics" facet, which is what it always was;
* "Systems" is a new flat axis of 16 organ systems, matching how AMBOSS keeps
Systems flat while nesting Disciplines and Symptoms.
Tags can nest (migration e3f4a5b6c7d8)
`question_tags` gains parent_id and sort_order. A tag may sit under one of the
same kind (Surgery > Hand surgery) or under a system, which is how symptoms are
grouped by where they present. 726 symptoms are now filed under the system they
appear in; the remaining 3,536 stay top-level rather than being forced into an
approximate bucket. A false positive the dry run caught: "vision" was matching
"Health Supervision" — the same trap as erythema/erythematosus earlier, fixed
with a word boundary.
Admin can grow the taxonomy without a migration
POST /tags creates a top-level entry or a child; PATCH renames, reorders and
reparents, refusing a cycle; DELETE reparents children to the deleted tag's
parent rather than orphaning them, and can move its questions elsewhere;
POST /tags/{id}/questions attaches questions. Everything appears in every picker
immediately, because they all read the same endpoint.
Article sections were indexed but empty — `_rebuild_section_index` only runs on
save, so articles written before it existed had no rows. Backfilled: 10 articles,
28 sections, now embedded and searchable. Section-scoped question links already
worked (7 of 34 links name a section).
Tests: 10 new backend covering the tree shape, adding top-level and child
entries, kind rules, duplicate refusal, cycle refusal, rename/reparent, question
attachment, delete-reparents-children, delete-with-move, and the moderator gate.
141 backend, 136 frontend green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017acfNLsJpnkvH3sCZSjMJM
Storage
Media now goes through `storage_service`, which has two backends: the container
volume, and S3/MinIO. A volume can only be mounted by one host, has no presigned
URLs and no lifecycle rules, none of which suits ~860 MB of media. Reads fall
back to the volume when an object is missing, so the existing uploads keep
working and files can migrate gradually rather than in one risky pass.
A row stores the object key, never a URL: a URL embeds the backend, so a row
holding `http://minio:9000/...` breaks the moment the backend changes.
MinIO publishes no host ports — the backend reaches it over the compose network,
and 9000/9001 are already taken on this host by other stacks.
Image libraries (migration d2e3f4a5b6c7)
An image belongs to a library, and a person is granted a library the way they are
granted a category, so access can be given to some images without giving away all
of them. Tags reuse the shared `question_tags` vocabulary rather than inventing a
media-only one. Uploads are type- and size-checked, stored through the service,
and embedded so an image can be found by what it shows.
Classification finished
The 316 questions the chooser had declined are now filed with `--force`, which
takes the nearest candidate from the same shortlist the chooser saw. 306 were
forced, 10 the chooser accepted on this pass. No question sits on a bare system
any more:
system only 2,730 -> 0
condition/subsystem 214 -> 1,782
full depth 4 -> 1,166
A forced match is a weaker signal than a chosen one, so expect more errors among
those 306 — but the original system stays as a cross-link, so nothing is lost and
they can be corrected by hand.
Tests: 8 new backend covering library scoping, edit confinement, shared-vocabulary
tags, storage indirection on upload, and type/size limits. 131 backend green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WgRcMaScVEL7TBLpnAoSV9
An admin can now say "you edit Step 1 Cardiology" rather than only "you edit this
category". Each dimension on a grant is nullable and means "any"; a grant covers
the questions matching all the dimensions it sets, and holding several grants is
the union of their coverage (migration c1d2e3f4a5b6). A check constraint refuses
a grant that names nothing, which would otherwise mean "everything".
Permission checks now run against a predicate over Question rather than a set of
category ids, so the exam and discipline dimensions actually take effect on edit,
delete and bulk actions instead of being silently ignored.
Pediatrics is unbound back to a global tag. With counts already scoped by the
learner's active exam, one global row gives the right number per exam, so
scoping the row bought nothing and duplicating a 6,740-tag vocabulary per exam
would have to be repeated for every rename and merge.
Tests: 123 backend green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01365DYKu14YtsBKv2ycW6eG
A discipline may now belong to one exam. `question_tags.exam_id` NULL keeps a tag
shared — Cardiology means the same thing whichever exam you sit — while a set
exam_id scopes it. Boards Pediatrics and a future Step 1 Pediatrics are therefore
separate rows over genuinely different bodies of content, not one label stretched
across both. Uniqueness moves from (name, type) to (name, type, exam) to allow it
(migration b0c1d2e3f4a5).
`scripts/bind_exam_tags.py` binds Pediatrics to Pediatrics Boards and tags the
884 questions in that exam that were missing it — the whole bank is paediatrics,
so it now reads 2,948.
Facet counts are computed within the learner's active exam, and a tag scoped to a
different exam is left out: an unscoped list offered disciplines that could not
match anything they were studying. With no exam chosen, everything is offered as
before.
Tests: 4 new backend (same name once per exam, unscoped list offers all, choosing
an exam scopes counts and hides other exams' tags, switching exam switches which
Pediatrics is offered). Full suite green: 123 backend.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PpfzbZ1QTLMeVYxM2kyq8m
Editing a question now snapshots its previous state. The last 5 are kept — the
value is undoing a recent mistake, not an audit trail, and an uncapped history of
full question bodies grows without bound (migration a9b0c1d2e3f4).
A restore snapshots the current state first, so the restore is itself undoable.
History is gated by the same per-category grant that gates editing, so it cannot
be read by someone who could not have made the edit. The question editor shows
the versions with their dates and a Restore action.
Also added docs/TODO.md tracking everything requested and not yet delivered:
AI Mode and its citation contract, global search, study-plan editing and
articles-in-blocks, admin settings revamp, image libraries and question folders,
media management, nested article sections with references and per-section notes
and feedback, per-question notes and feedback in the runner, tutorial mode, the
per-question performance table, the Overview dashboard, systems subsystems, and
dropping "Pediatrics" as a discipline.
Tests: 6 new backend (snapshot on edit, cap at five newest-first, restore,
restore is undoable, refused without edit rights, unknown version). Full suites
green: 119 backend, 136 frontend, build clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PpfzbZ1QTLMeVYxM2kyq8m
The PREP sets were loose admin-generated quizzes. They are now study plans: one
per year, split into blocks of 50 numbered "Block 1", "Block 2", plus a
"PREP Mixed" plan of 300 drawn at random across every year. 12 plans, 2,821
questions, applied to production.
Block membership is snapshotted rather than stored as a filter — a plan you are
part-way through must not reshuffle between visits. Re-running the seeder updates
years whose questions changed and leaves the mixed draw alone unless --reshuffle.
Starting a block reuses the learner's existing quiz for it; without that,
reopening a block would create a duplicate test each time and scatter the
attempts across them. Only questions the learner may see are included.
Tests: 113 backend green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PpfzbZ1QTLMeVYxM2kyq8m
Images were findable only by the filename someone typed. `media_assets` gives
them a title, caption, alt text, a category on the shared tree and tags, with a
weighted tsvector so they are searchable now (migration y7e8f9a0b1c2).
The embedding column is filled from the caption today. A vision-capable model can
fill it from the image itself later without another migration — and because
`embedding_model` stamps every vector, a text-embedded caption and a
vision-embedded image stay distinguishable instead of being silently mixed in one
index. Adding "media" to the embeddable kinds is all the retry task, the full
regeneration and the health report needed.
`media_tag_links.tag_id` carries no ORM-level foreign key: `question_tags` is
created by raw DDL rather than a model, so the constraint lives in the migration
where the table actually exists.
Tests: 113 backend green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PpfzbZ1QTLMeVYxM2kyq8m
An article embedded as a single vector, which finds the article but not the
paragraph — so a citation could only ever point at the top of a page. Sections
live in a JSON column and cannot carry a vector or a full-text index, so they are
now projected into `article_section_index`: one row per section with its own
embedding and weighted tsvector (migration x6d7e8f9a0b1).
- Rows are keyed by section id, so editing a section updates it, removing one
deletes it, and an unchanged section is not re-embedded on every save.
- `article_section` joins the embeddable kinds, so the retry task, the full
regeneration and the health report cover it without further changes.
- `hybrid_ids(db, query, "article_section")` searches it like any other corpus.
This is the groundwork for grouped search results (article, then the sections
that matched) and for AI citations that deep-link to the right section.
Tests: 113 backend green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PpfzbZ1QTLMeVYxM2kyq8m
Exams (migration v4b5c6d7e8f9)
"Pediatrics Boards" was a hardcoded checkbox that filtered nothing. Exams are now
rows: Pediatrics Boards and USMLE Step 2 CK ship seeded, and everything already
in the bank is linked to the boards. Membership is a link table, not a column,
because one paediatric cardiology question can count towards several exams.
The learner's choice lives on `users.active_exam_id`, so it follows them between
devices instead of sitting in one browser's storage. Choosing an exam scopes the
bank; a question with no exam links stays visible, since unlinked content is
unclassified rather than excluded. A switcher sits in the navbar.
AI mode — matching, never generating
Both entry points build a test from the educator-reviewed questions that already
exist, ranked against the request. Nothing is invented:
- POST /questions/builder/describe turns "what I want to study" into a test.
- POST /questions/builder/from-upload matches a document against the bank. The
file is read in memory and never stored — it is a search query, not a source
of questions, so there is nothing to retain or expire. 10 MB cap, 30 questions.
Handing a whole document to `websearch_to_tsquery` builds one enormous
conjunction that matches nothing, so text over 300 characters is reduced to its
most distinctive terms, OR-joined, before it reaches the lexical ranker.
Continue your study (migration w5c6d7e8f9a0)
A dashboard panel with the sessions in flight and the articles most recently
opened. `article_views` records one row per learner and article, written best
effort so a reading page never fails because a bookkeeping write did.
Tests: 5 new exam tests (active exams and counts, choice persisted and cleared,
unknown/inactive refused, bank scoping including unlinked questions, moderator-only
creation) and 7 for AI-mode matching (no questions created, invisible questions
excluded, no-match reported rather than an empty test, upload limits enforced).
Full suites green: 113 backend, 136 frontend, build clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PpfzbZ1QTLMeVYxM2kyq8m
Retrieval generalised beyond questions
`_text_for_question`, `embed_question` and `hybrid_question_ids` all hardcoded
the questions table, so there was nothing to call for an article or a card. That
layer is now corpus-agnostic:
- `Embeddable` mixin gives articles and flashcards the same embedding,
embedding_model and embedded_at columns questions have, plus a weighted
full-text vector (migration u3a4b5c6d7e8).
- `embed_record(row, kind)` is one code path for all three — they share an
embedding space, so they must share the model and provenance rules too.
- `hybrid_ids(db, query, kind)` ranks any corpus; `hybrid_question_ids` stays as
a thin alias for existing callers.
- Article and flashcard search moved off `ILIKE '%term%'`, which could not find
a jaundice article from "yellow newborn".
- The retry task and full regeneration now sweep every corpus, and the health
report breaks down current/stale/missing per kind.
- Articles embed on create and on edit, with failures left to the retry task.
Quoted phrases replace the keyword-only mode
`websearch_to_tsquery` already gives "absence seizure" exact-phrase semantics,
and the semantic ranker sits out a quoted query. That covers the one case a
keyword-only toggle was for — exact lookup — per query rather than as a sticky
setting whose every position returns a subset of the default.
Full-page question editor (/questions/new, /questions/:id)
Editing happened in a cramped modal. There is now a page with room for the stem,
per-option explanations, a searchable category picker with primary plus extras,
difficulty, and images. It shows the question's id with a copy button, and
Duplicate creates a variant without retyping the stem. `GET /questions/detail/{id}`
backs it, pathed under /detail/ so it cannot shadow the static routes.
Question bank filter bar restyled — the toggle and count read as one control
instead of two grey pills crowding the result count.
Tests: 101 backend green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PpfzbZ1QTLMeVYxM2kyq8m
Questions and articles both pointed at `question_categories`; decks had no
category at all, so the three content types could not be filtered together and a
topic's cards were unreachable from its category.
- `flashcard_decks.category_id` references the same tree (migration
t2f3a4b5c697), so one category now spans questions, articles and cards.
- `GET /flashcards/` takes `category_id` and includes descendants, so a parent
category picks up everything filed beneath it.
- `PATCH /flashcards/{id}` files or unfiles a deck, refusing a category id that
does not exist rather than storing a dangling reference.
- The cards page shows each deck's category as a selector.
Tests: 5 new backend (all three types resolve to the same id, descendant
filtering, file and unfile, unknown category refused, renaming leaves the
category alone) and 1 new frontend. Full suites green: 101 backend,
136 frontend, build clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PpfzbZ1QTLMeVYxM2kyq8m
Search
- Retrieval was hybrid in name only: the keyword filter was applied to the SQL
query, so results were the *intersection* of the two rankers. A question that
matched the meaning but not the literal string could never be returned. It is
now a union, fused with Reciprocal Rank Fusion (a text rank and a cosine
distance are not on comparable scales, so RRF uses only their orderings).
- Added a generated `search_vector` tsvector + GIN index, so the lexical half is
ranked full text rather than ILIKE substring matching.
- Chose Postgres + pgvector over OpenSearch/Elasticsearch: a search cluster
would add a second datastore to keep in sync and a JVM on this host, to
replace an index Postgres maintains inside the same transaction.
- Removed the keyword-only mode. It looks precise but silently drops the
question that asks the same thing in different words.
Embeddings — measured on 500 real questions, using each question's own
explanation as a paraphrase query (known answer, no hand labelling):
bge-small (local CPU, 384d) R@1 0.840 R@5 0.953 186ms/query
bge-m3 (LiteLLM proxy, 1024d) R@1 0.847 R@5 0.973 93ms/query
BGE-M3 wins on both quality and latency and needs no extra credential, since
llm.danvics.com already serves `openrouter-bge-m3`.
Three gaps this exposed, all fixed:
- Nothing recorded which model produced a stored vector, so changing models
silently mixed incomparable spaces. `embedding_model` / `embedded_at` now
stamp every vector, `GET /admin/embedding/health` reports current vs stale vs
missing, and regeneration defaults to stale-only.
- The generator read the model from env while the stamp read a Redis override,
so a vector could be labelled with a model that did not produce it. Both now
resolve through one function, with a regression test.
- Embedding at creation is best effort, and a failure left a question invisible
to semantic search forever. `retry_missing_embeddings` runs every 15 minutes
via Celery beat and backfills missing or stale rows.
- Query embeddings are cached in Redis per model, so typing is not a network
round-trip per keystroke.
`dimensions` is only sent to OpenAI's embedding-3 family; BGE-M3 rejects it.
Tests: 8 new backend tests (union not intersection, fusion ordering, per-ranker
failure degradation, provenance stamping, stale/missing accounting, generator
and stamp agreement). Full suites green: 95 backend, 127 frontend, build clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014yhHB8Pc7oQqyqn2Vo9DXA
Analysis / recommendations (AMBOSS parity, verified on next.amboss.com):
- GET /study-tools/recommendations ranks focus areas by the study time most
likely to raise the score. Readiness is the learner's accuracy in a category
shrunk toward their own overall accuracy in proportion to sample size, so two
unlucky answers do not read as a knowledge gap; it unlocks after 40 answers.
Relevance is the share of the bank a category holds. Counts roll up through
the category tree, so a system inherits its children's questions.
It is deliberately not called EPC and does not claim to predict an exam.
- New /analysis page: Performance and Recommendations tabs, readiness summary,
adaptive-session box, and expandable focus rows showing questions seen,
answered correctly, the linked article and a per-topic practice action.
Per-category educator grants:
- category_grants table (migration p8b9c0d1e253) plus utils/category_grants.py
resolving a grant to the category and all of its descendants.
- Question create, edit, delete, bulk and the manager summary now accept a
moderator OR an educator granted the affected categories, and refuse moves
that would push a question out of the holder's scope. Summary counts are
scoped to the grant.
- Moderator endpoints to list, add and revoke grants, plus /my-grants driving
the nav link and the manager's scope banner; grantable-users avoids handing
moderators the admin-only user list.
- GrantsPanel in the question manager: grant, list and revoke with inline
confirmation.
Question page:
- The category trail was a fixed 78px band that wrapped into several rows and
pushed the stem down the page, followed by three more stacked strips. It is
now one scrollable meta line (breadcrumb + difficulty + type) and a single
AMBOSS-style action bar (Mark / Listen / Listen through / Clear) between the
stem and the options. Difficulty is exposed on the runner payload.
Deploy fix: index.html shipped with no cache header, so browsers kept serving
the previous bundle references and a release looked like nothing had changed.
nginx now sends no-cache for HTML and immutable long-cache for hashed assets.
Tests: 16 new backend (recommendation shrinkage, roll-up, locking, grant scope
across create/edit/delete/bulk/summary, moderator gate) and 10 new frontend.
Full suites green: 88 backend, 116 frontend, build clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014yhHB8Pc7oQqyqn2Vo9DXA
Daniel