danvics/pdf-quiz-generator
1
0
Fork 0
Code Issues Pull requests Projects Releases 3 Packages Wiki Activity Actions

fix: put the disciplines and diseases back, and order Performance as shown

Browse source 
Retiring three vocabularies at once was my call and the wrong one. Keyword
had to go — it was the old route to an organ system, which a topic now
carries, and that took Systems from half the bank to all of it. Subject
and disease went with it on the argument that the topic tree says the same
thing. It mostly does, and "mostly" is not a reason to remove the
vocabulary people had learned to filter by.

203 subjects and 2,275 diseases are back, with their 14,029 links, and the
Disciplines and Diseases pickers with them. Keywords stay retired.

The backup I wrote before deleting was not where I said it was:
`./backups` is mounted on db-backup, not on backend, so the file went with
the next container rebuild. The rows came from the nightly dump instead,
which is what that dump is for. scripts/restore_subject_disease_tags reads
a pg_dump extract, is idempotent, and resets the sequence afterwards so
the next tag created by hand does not collide with a restored one.

/tags serves subjects and diseases from their own links again, and systems
through the topics that carry them.

The Performance tab is ordered as the reference has it: the trend beside
the split it is a trend in, and Completion's four figures underneath.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
This commit is contained in:
Daniel 2026-09-12 05:39:22 +02:00
parent a16e45a7f1
commit 6240a1c9da
6 changed files with 182 additions and 28 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

45
backend/app/routers/tags.py
View file

@ -1,8 +1,8 @@
"""Organ systems: the vocabulary, and how many questions reach each one.
"""The vocabularies a question can be filtered by, and how many it holds.
Once this served four vocabularies a model wrote per question — subjects,
diseases, keywords, systems. The first three became the category tree and have
been retired; what is left is the sixteen systems, which a topic points at.
Three of the four a model once wrote per question survive: subjects, diseases
and the sixteen organ systems. Keywords are retired — they were the old route
to a system, which a topic now carries directly.
The table is created via raw SQL in setup_pgvector() to avoid race conditions
with multiple workers, so there is no ORM class for it.
@ -36,10 +36,14 @@ def get_tags(db: Session = Depends(get_db), current_user: User = Depends(get_cur
so the number here is the number the filter will return.
It used to come from a symptom keyword the question happened to mention.
Those keywords are retired, and with them the subject and disease
vocabularies that had already become the topic tree. The shape of the
response is kept so nothing that reads it has to be rewritten to find an
empty list where it expected one.
Those keywords are retired — the topic carries the system now, which took
the Systems axis from about half the bank to all of it.
Subjects and diseases are not retired. They were taken down with the
keywords on the argument that the topic tree says the same thing; it mostly
does, but they are the vocabulary people had learned to filter by, and
"mostly" is not a reason to remove somebody's tools. They are counted here
the way they always were, from the links a question carries directly.
"""
exam_id = getattr(current_user, "active_exam_id", None)
systems = db.execute(text(
@ -85,8 +89,31 @@ def get_tags(db: Session = Depends(get_db), current_user: User = Depends(get_cur
if system:
counts[system].add(qid)
# The two vocabularies that survived retirement, counted the way they
# always were: a question carries them directly.
vocab: dict[str, list[dict]] = {"subjects": [], "diseases": []}
of_type = {"subject": "subjects", "disease": "diseases"}
rows = db.execute(text("""
SELECT t.id, t.name, t.type, t.parent_id,
COUNT(DISTINCT tl.question_id) AS count
FROM question_tags t
LEFT JOIN question_tag_links tl ON tl.tag_id = t.id
WHERE t.type IN ('subject', 'disease')
AND (t.exam_id IS NULL OR :exam_id IS NULL OR t.exam_id = :exam_id)
GROUP BY t.id, t.name, t.type, t.parent_id
ORDER BY count DESC, t.name
"""), {"exam_id": exam_id}).fetchall()
for row in rows:
if exam_id and row.count == 0:
continue
vocab[of_type[row.type]].append({
"id": row.id, "name": row.name, "count": row.count, "parent_id": row.parent_id})
return {
"subjects": [], "diseases": [], "keywords": [],
**vocab,
# Retired: this was the route to an organ system before a topic carried
# one, and nothing reads it now.
"keywords": [],
"systems": [{"id": row.id, "name": row.name, "count": len(counts.get(row.id, ())),
"parent_id": None} for row in systems],
}

94
backend/scripts/restore_subject_disease_tags.py Normal file
View file

@ -0,0 +1,94 @@
"""Put the discipline and disease vocabularies back.
`retire_question_tags` took three vocabularies down together: subject, disease
and keyword. Keyword was the one that had to go — it was the old route to an
organ system, which the topic tree now carries properly. Subject and disease
were removed on the argument that the tree already says the same thing, and
that argument was mine to make and not mine to act on alone.
So this restores those two and leaves keyword retired. Read from a plain
tab-separated extract of the two tables — the shape `pg_dump` writes — so it
works from a nightly dump as well as from a purpose-made file.
docker compose exec backend python -m scripts.restore_subject_disease_tags \\
--tags /backups/tags.tsv --links /backups/links.tsv
…same, with --apply
Idempotent: a tag id that is already present is left alone, and so is a link
that already exists.
"""
import argparse
import sys
from sqlalchemy import text as sa_text
from app.database import SessionLocal
RESTORING = ("subject", "disease")
#: id, name, type, created_at, exam_id, parent_id, sort_order — pg_dump's order.
COLUMNS = ("id", "name", "type", "created_at", "exam_id", "parent_id", "sort_order")
def read(path: str) -> list[list[str]]:
with open(path, encoding="utf-8") as handle:
return [line.rstrip("\n").split("\t") for line in handle if line.strip()]
def value(raw: str):
"""`pg_dump` writes an unquoted \\N for null."""
return None if raw == "\\N" else raw
def main(tags_path: str, links_path: str, apply: bool) -> int:
wanted = [row for row in read(tags_path) if row[2] in RESTORING]
if not wanted:
print(f"! no {' or '.join(RESTORING)} rows in {tags_path}")
return 1
ids = {row[0] for row in wanted}
links = [row for row in read(links_path) if row[1] in ids]
db = SessionLocal()
try:
present = {str(r[0]) for r in db.execute(sa_text(
"SELECT id FROM question_tags WHERE type = ANY(:t)"), {"t": list(RESTORING)}).all()}
new_tags = [row for row in wanted if row[0] not in present]
counts = {kind: sum(1 for row in new_tags if row[2] == kind) for kind in RESTORING}
print(f"in the extract {len(wanted)} tags, {len(links)} links")
print(f"already present {len(wanted) - len(new_tags)}")
for kind, n in counts.items():
print(f" {kind:<9} {n} to restore")
if not apply:
print("\ndry run. Pass --apply to write.")
return 0
for row in new_tags:
db.execute(sa_text(
"INSERT INTO question_tags (id, name, type, created_at, exam_id, parent_id, sort_order) "
"VALUES (:id, :name, :type, :created_at, :exam_id, :parent_id, :sort_order) "
"ON CONFLICT (id) DO NOTHING"),
{col: value(raw) for col, raw in zip(COLUMNS, row)})
for question_id, tag_id in links:
db.execute(sa_text(
"INSERT INTO question_tag_links (question_id, tag_id) VALUES (:q, :t) "
"ON CONFLICT DO NOTHING"), {"q": int(question_id), "t": int(tag_id)})
# The sequence has to clear the ids just written, or the next tag
# created by hand collides with a restored one.
db.execute(sa_text(
"SELECT setval(pg_get_serial_sequence('question_tags', 'id'), "
"GREATEST((SELECT MAX(id) FROM question_tags), 1))"))
db.commit()
print(f"\nrestored {len(new_tags)} tags and {len(links)} links.")
return 0
finally:
db.close()
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("--tags", required=True)
parser.add_argument("--links", required=True)
parser.add_argument("--apply", action="store_true")
args = parser.parse_args()
sys.exit(main(args.tags, args.links, args.apply))

12
backend/tests/test_tag_hierarchy.py
View file

@ -83,12 +83,14 @@ class TagHierarchyTests(unittest.TestCase):
self.assertEqual(systems["Nervous System & Special Senses"], 2)
self.assertEqual(systems["Respiratory System"], 1)
def test_the_retired_vocabularies_answer_empty_rather_than_missing(self):
def test_keywords_are_retired_and_answer_empty(self):
body = self.client.get("/tags").json()
# Subjects, diseases and keywords became the topic tree. The shape of
# the response is kept so nothing reading it has to be rewritten.
for gone in ("subjects", "diseases", "keywords"):
self.assertEqual(body[gone], [])
# Keywords were the old route to an organ system, which a topic now
# carries. Subjects and diseases are still served — they are simply
# not in this fixture. The key stays so nothing reading it breaks.
self.assertEqual(body["keywords"], [])
for kept in ("subjects", "diseases", "systems"):
self.assertIn(kept, body)
def test_a_moderator_can_add_a_system(self):
made = self.client.post("/tags/", json={"name": "Endocrine System", "type": "system"})

2
frontend/src/pages/AnalysisPage.css
View file

@ -141,7 +141,7 @@
display: grid; grid-template-columns: minmax(0, 1fr) minmax(0, 1fr);
gap: 18px; align-items: start;
}
.an-perf-top .an-completion { margin: 24px 0 0; }
.an-perf-top > * { margin-top: 24px; }
.an-split-card {
margin-top: 24px; padding: 16px 18px 18px;
border: 1px solid var(--border); border-radius: 12px; background: var(--card-bg);

7
frontend/src/pages/AnalysisPage.jsx
View file

@ -524,11 +524,14 @@ export default function AnalysisPage() {
) : (
<>
<Readiness />
<OverTime />
{/* The chart and the donut read together — the trend beside the
split it is a trend in — and Completion is four plain figures
underneath, which is the order the page is read in. */}
<div className="an-perf-top">
<Completion />
<OverTime />
<AnswerSplit />
</div>
<Completion />
<CategoryPerformance />
</>
)}

50
frontend/src/pages/CustomQuizPage.jsx
View file

@ -139,11 +139,16 @@ export default function CustomQuizPage() {
}, [categories])
const nameById = useMemo(() => Object.fromEntries(categories.map(c => [c.id, c.name])), [categories])
const tagName = (id) => (tags.systems || []).find(t => t.id === id)?.name
const subjectTags = tags.subjects || []
const diseaseTags = tags.diseases || []
const systemTags = tags.systems || []
const tagName = (id) => [...subjectTags, ...diseaseTags, ...systemTags]
.find(t => t.id === id)?.name
const systems = summarise(categoryIds.map(id => nameById[id]).filter(Boolean))
const organSystems = summarise(tagIds.map(tagName).filter(n => n && systemTags.some(t => t.name === n)))
const disciplines = summarise(tagIds.map(tagName).filter(n => n && subjectTags.some(t => t.name === n)))
const diseases = summarise(tagIds.map(tagName).filter(n => n && diseaseTags.some(t => t.name === n)))
const articleSummary = summarise(articleIds.map(id => articles.find(a => a.id === id)?.title).filter(Boolean))
const savedNames = [
...(state === 'bookmarked' ? ['Bookmarked questions'] : []),
@ -197,9 +202,14 @@ export default function CustomQuizPage() {
out.push({ key: `c${cat.id}`, name: cat.name, facet: 'Topics', on: categoryIds.includes(cat.id), toggle: on => toggleCategory(cat.id, on) })
}
}
for (const tag of systemTags) {
if (tag.name.toLowerCase().includes(query)) {
out.push({ key: `y${tag.id}`, name: tag.name, facet: 'Systems', on: tagIds.includes(tag.id), toggle: on => toggleTag(tag.id, on) })
for (const [group, facet, prefix] of [[systemTags, 'Systems', 'y'],
[subjectTags, 'Disciplines', 's'],
[diseaseTags, 'Diseases', 'd']]) {
for (const tag of group) {
if (tag.name.toLowerCase().includes(query)) {
out.push({ key: `${prefix}${tag.id}`, name: tag.name, facet,
on: tagIds.includes(tag.id), toggle: on => toggleTag(tag.id, on) })
}
}
}
for (const article of articles) {
@ -208,7 +218,8 @@ export default function CustomQuizPage() {
}
}
return out.slice(0, 40)
}, [globalSearch, categories, systemTags, articles, categoryIds, tagIds, articleIds])
}, [globalSearch, categories, systemTags, subjectTags, diseaseTags, articles,
categoryIds, tagIds, articleIds])
const resetTopics = () => {
setCategoryIds([]); setTagIds([]); setArticleIds([]); setPresetIds([])
@ -266,6 +277,8 @@ export default function CustomQuizPage() {
</div>
<FacetRow label="Systems" {...organSystems} onOpen={() => setOpenFacet('organ-systems')} />
<FacetRow label="Topics" {...systems} onOpen={() => setOpenFacet('systems')} />
<FacetRow label="Disciplines" {...disciplines} onOpen={() => setOpenFacet('disciplines')} />
<FacetRow label="Diseases" {...diseases} onOpen={() => setOpenFacet('diseases')} />
<FacetRow label="Articles" {...articleSummary} onOpen={() => setOpenFacet('articles')} />
<FacetRow label="Saved questions" {...saved} onOpen={() => setOpenFacet('saved')} />
</div>
@ -405,11 +418,26 @@ export default function CustomQuizPage() {
)}
</FacetPicker>
{/* Disciplines and Symptoms & keywords were two more pickers here, over
the 203 subject tags and 4,281 keyword tags a model had written per
question. Both vocabularies were converted into Topics — 673 of them,
in a tree, which is the same axis said once and said better — and
have been retired. Topics is where a discipline is chosen. */}
{/* Symptoms & keywords used to sit here too, over 4,281 keyword tags.
Those were the old route to an organ system, which a topic now
carries directly, and they are retired. Disciplines and Diseases are
not: the topic tree mostly says the same thing, and "mostly" is not a
reason to take away the vocabulary people filter by. */}
<FacetPicker title="Disciplines" open={openFacet === 'disciplines'} onClose={() => setOpenFacet(null)}
onReset={() => setTagIds(ids => ids.filter(id => !subjectTags.some(t => t.id === id)))}
selected={subjectTags.filter(t => tagIds.includes(t.id)).map(t => ({ id: t.id, label: t.name }))}
onRemove={id => toggleTag(id, false)}
helper="By default, all disciplines are included unless filters are selected.">
{query => checkList(subjectTags, t => tagIds.includes(t.id), (t, on) => toggleTag(t.id, on), query)}
</FacetPicker>
<FacetPicker title="Diseases" open={openFacet === 'diseases'} onClose={() => setOpenFacet(null)}
onReset={() => setTagIds(ids => ids.filter(id => !diseaseTags.some(t => t.id === id)))}
selected={diseaseTags.filter(t => tagIds.includes(t.id)).map(t => ({ id: t.id, label: t.name }))}
onRemove={id => toggleTag(id, false)}
helper="Named conditions. By default all are included.">
{query => checkList(diseaseTags, t => tagIds.includes(t.id), (t, on) => toggleTag(t.id, on), query)}
</FacetPicker>
<FacetPicker title="Articles" open={openFacet === 'articles'} onClose={() => setOpenFacet(null)}
onReset={() => setArticleIds([])}