An HPV vaccination question was carrying a table of craniofacial reconstruction
by age. Extraction takes figures off the page a question was printed on, which
is usually right and occasionally catastrophically wrong — and the mistake is
invisible to a text review, because nothing about the stem or the filename says
the picture is about something else. My earlier pass read stems and answers and
could not have caught it.
So this looks. Every figure goes to a vision model with its question and is
asked whether it belongs: strict about "no", generous about "unsure", because
detaching a figure a question needs is worse than leaving a stray one attached
for somebody to notice. A clear mismatch is detached and the image stays in the
bank with a note saying what happened, so nothing is lost and the decision can
be checked.
Ran over all 341: **30 detached, 2 unsure, 309 kept.** Among the thirty — the
rule of nines on a hepatitis B newborn, a molluscum table on a collapsed
athlete, a textbook citation page on a drowning, and the craniofacial table
that started this.
It also describes each figure while it is looking at it, and keeps the
description. 435 assets said "Figure from question #1206", which is a filename
with extra steps and is why the image bank could not be searched; 335 now say
what they show, and none of the placeholders remain.
The model is named in the task rather than read from the admin's `tool`
setting: this is a one-off audit, and it should not quietly become the thing
that decides what the tool model is.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
9658ddb207