9d307fd442
Some checks failed
Forgejo Android APK / Root app tests (push) Successful in 50s
Forgejo Docker Build / Root app tests (push) Successful in 1m2s
Forgejo Android APK / Build signed APK (push) Successful in 2m6s
Forgejo Docker Build / Build Docker image (push) Successful in 37s
Forgejo Docker Build / Deploy to the host (push) Failing after 0s
The model that writes a deck never sees it. It cannot tell that slide four overflowed, that a nine-item list would read better in two columns, or that two labelled groups want to be a comparison — those are facts about the rendered page, not about the text. So each generated deck is now rendered to PDF through Gotenberg, rasterised to one image per slide with pdftoppm, and shown to a vision model. Off unless an administrator names a reviewer, in its own admin card because it is the one setting that spends money on every generation without a user having asked for anything. One pass, on generation only: a second pass costs as much as the first and fixes far less, and refining is a text edit. It returns a patch, not a deck. Asking for the corrected deck back put the reply in proportion to the deck rather than to the number of problems, and a fourteen-slide deck came back cut off mid-object at every output budget the provider would honour — measured twice before changing shape. The patch is better for a second reason. The reviewer names a slide and an action — two columns, one column, split after bullet N, compare with these two labels — and the server moves the text it already has. The words never pass through the model, so a review cannot reword, drop or invent a single bullet. That is a stronger guarantee than instructing it not to and checking afterwards. The check runs anyway, because a bug in applyChanges would be as bad as a model rewriting the words and worse for being trusted: body text must come out the same multiset, figures the same set, and a heading may only be reused or extended. A continuation heading is the reviewer's one piece of text and is replaced when it does not continue anything. Nothing here can fail a generation — no reviewer, an unreachable one, an unparseable reply, a deck too long to look at, or a patch that applies to nothing each return the deck that was written. Verified end to end against a deck with a deliberately overloaded slide: three slides rendered and sent, one change returned, ten bullets split into five and five under "Stepwise Management … (continued)", text intact. Left switched off; enable it under Admin. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU
223 lines
10 KiB
Markdown
223 lines
10 KiB
Markdown
# My Resources
|
|
|
|
Teaching material a signed-in user generates for themselves — a deck for
|
|
tomorrow's session, a handout, a summary — kept privately and exported as
|
|
PowerPoint, Word or PDF.
|
|
|
|
Deliberately separate from the Learning Hub. That is moderator-owned content
|
|
published into categories for everyone; this needs no role beyond being signed
|
|
in, and nothing here is shared. Every statement filters on `user_id`, and there
|
|
is no route that returns another person's work. Sharing, if it is ever wanted,
|
|
should be a deliberate feature rather than something that leaks out of a
|
|
forgotten `WHERE` clause.
|
|
|
|
## What a resource is
|
|
|
|
| Column | |
|
|
|---|---|
|
|
| `markdown` | the readable artifact — what Word renders and what a text edit edits |
|
|
| `deck` | for a presentation, the slide structure the model designed (see below) |
|
|
| `image_ids` | the illustration jobs belonging to this resource, in the order they were made |
|
|
| `topic`, `grounded_count` | what it was asked for and how many library excerpts it was written from |
|
|
|
|
`MAX_PER_USER` caps how many a person may keep.
|
|
|
|
## Sources
|
|
|
|
One function, `gatherSources()`, answers "what is this written from" for both
|
|
generating and modifying, so the two cannot drift into offering different things
|
|
or searching them differently.
|
|
|
|
- **The clinical library** — on by default. Semantic retrieval over the indexed
|
|
corpus. Budgets are in [retrieval-tuning.md](retrieval-tuning.md).
|
|
- **PubMed** — admin-enabled, optional API key. Returns structured records so a
|
|
reference carries a PMID somebody can look up.
|
|
- **The web** — admin-enabled, provider-configurable (Tavily, Serper, Brave,
|
|
SearXNG).
|
|
- **Illustrations** — see below.
|
|
|
|
Each option hides itself when an administrator has not enabled it, so nothing
|
|
appears that a person could tick and then be refused.
|
|
|
|
Nothing here may fail a generation. A retrieval or search that comes back empty
|
|
is reported as a reason and the resource is written from what is available.
|
|
|
|
### Searching is the route's job, not the model's
|
|
|
|
Both searches run up front on the topic, and their results go into the prompt as
|
|
findings. They are **not** offered as tools.
|
|
|
|
They were, once. Tested live against a question explicitly about recent trials,
|
|
the model never called them — with or without corpus grounding, and no matter
|
|
how the tool description was worded, because the prompt ends "Output ONLY Pandoc
|
|
markdown" and a model told to output only markdown does not emit a tool call.
|
|
Calling `callAI` with the tool directly produced a correct call, so the plumbing
|
|
was never the problem. The search only ever needed the topic, and the route
|
|
knows the topic before it calls the model.
|
|
|
|
### Keyword searches get the topic, retrieval gets more
|
|
|
|
PubMed ANDs every mapped term, so one unrecognised word takes a query to zero:
|
|
`"febrile seizures"` returns six results and `"febrile seizures in under-fives"`
|
|
returns none. A query that finds nothing is retried against progressively
|
|
shorter versions of itself, longest first, and the response says which query
|
|
actually worked. Those retries are spaced — three `esearch` calls back to back
|
|
trips NCBI's three-a-second limit without an API key.
|
|
|
|
When modifying, the library search gets the topic *plus* the instruction —
|
|
retrieval is semantic and benefits from context — while PubMed and the web get
|
|
the topic alone.
|
|
|
|
### Empty searches must not invite invention
|
|
|
|
When a search was asked for and came back empty, the prompt says not to invent a
|
|
citation, a PMID or a URL to fill the gap. Without that the model supplies them
|
|
from memory, and a fabricated PMID is indistinguishable from a real one.
|
|
|
|
## Presentations are designed, not written
|
|
|
|
A presentation is described as a **deck**: the model returns JSON naming a
|
|
layout per slide and the prompt for each figure it wants. Articles stay
|
|
markdown, which is what prose wants.
|
|
|
|
Markdown could express about five of the things the renderer can draw, so the
|
|
model had no way to say "put this figure beside these three bullets" or "make
|
|
this a comparison with two labelled columns" — the parser inferred a layout from
|
|
the shape of a list, and inferring is what made every deck look the same.
|
|
|
|
Layouts: `title`, `section`, `bullets`, `two`, `compare`, `table`, `callout`,
|
|
`figure`, `image`. See `src/utils/deckSchema.js` for what each accepts.
|
|
|
|
Markdown is still produced, serialised from the deck, so Word export and text
|
|
editing keep working and the stored artifact stays readable by a person. The
|
|
deck is stored alongside it because that serialisation is lossy by design:
|
|
round-tripping through markdown would discard the layout choices.
|
|
|
|
Nothing costs more than the thing that went wrong — a reply that is not a deck
|
|
falls back to asking for markdown; a malformed slide degrades to bullets; a
|
|
comparison with one column is not a comparison; JSON wrapped in fences or a
|
|
covering sentence is read rather than refused.
|
|
|
|
## Illustrations
|
|
|
|
`resourceImages.js`, **not** the shared `imageTool.dispatch` — that one permits
|
|
exactly one image per request, which is right for a chat reply and wrong for a
|
|
twelve-slide deck, and the clinical assistant and Learning Hub depend on that
|
|
rule. Same queue, same storage, same `my_resources` workflow, same asset
|
|
endpoint; only the number differs, bounded at `MAX_IMAGES` because each figure
|
|
is a paid request.
|
|
|
|
"Use 3 diagrams" in the instructions is read as the number it is. Writing
|
|
"include a diagram of the airway" switches the illustration option on and says
|
|
why, rather than the request being dropped in silence; switching it off by hand
|
|
sticks.
|
|
|
|
`my_resources` is its own image workflow rather than a reuse of `learning_hub`
|
|
because `generated_image_links` only accepts `learning_hub` assets — that is the
|
|
barrier keeping a private illustration out of published content.
|
|
|
|
### Getting a model to illustrate at all
|
|
|
|
Two things had to be right, both measured:
|
|
|
|
1. The illustration guidance is the **last** thing in the prompt. Placed before
|
|
the output rules it lost: the model returned 3297 characters of markdown and
|
|
zero tool calls, while the same tool and wording in a shorter prompt produced
|
|
three calls.
|
|
2. Even last, it loses to a prompt carrying thirty library excerpts —
|
|
deterministically: library off → three calls, library on → none. So when the
|
|
author names a number the call is **required** rather than offered. With no
|
|
number named the choice stays the model's.
|
|
|
|
A model that has just made three tool calls also tends to sign off instead of
|
|
writing — `"I'll create the presentation and the three teaching diagrams."` was
|
|
once returned as the resource, 61 characters, because only a completely empty
|
|
body counted as missing. A body with no title block and no heading is now
|
|
treated as missing whatever its length.
|
|
|
|
## Export
|
|
|
|
| Format | Built by |
|
|
|---|---|
|
|
| `pptx` | `scripts/render_pptx.py` (python-pptx) from the stored deck |
|
|
| `docx` | pandoc, from the markdown |
|
|
| `pdf` | Gotenberg (LibreOffice), from whichever office file above |
|
|
|
|
Pandoc's pptx writer was the ceiling on how good a deck could be, and the model
|
|
on top made no difference to it: a handful of reference layouts, no per-slide
|
|
layout, no positioning, no control over how large an image is drawn. It also
|
|
leaves a bare `<a:bodyPr/>` on every shape, so slides overflowed until autofit
|
|
was injected into its emitted OOXML by hand.
|
|
|
|
The renderer sizes text to fit before writing the file rather than trusting
|
|
autofit — LibreOffice ignores `<a:normAutofit/>` when converting to PDF, which
|
|
is how slides were being cut off mid-sentence. Wrapped bullet lines hang under
|
|
the text. Images are drawn at their own aspect ratio.
|
|
|
|
If the renderer fails for any reason, pandoc still produces a deck: a plainer
|
|
deck beats a failed download. The log line is
|
|
`deck renderer failed, falling back to pandoc`.
|
|
|
|
Figures are fetched to a scratch directory for the renderer and removed
|
|
afterwards. One that cannot be fetched is left out rather than failing a
|
|
download that works without it.
|
|
|
|
**Runtime dependency:** the image carries `python3`, `py3-lxml`, `py3-pillow`
|
|
(apk — both are C extensions with no Alpine wheels) and `python-pptx` pinned at
|
|
1.0.2 from pip. Roughly 58MB. Unpinned, a rebuild from the same commit could
|
|
produce different decks.
|
|
|
|
## Slide review
|
|
|
|
Off unless an administrator names a model, in **Admin → Slide review**.
|
|
|
|
The model that writes a deck never sees it, so overflow, a figure on the wrong
|
|
slide and a nine-item list that wants two columns are invisible to it. With a
|
|
reviewer configured, each generated deck is rendered to PDF through Gotenberg,
|
|
rasterised to one PNG per slide with `pdftoppm`, and shown to a vision model.
|
|
|
|
One pass, on generation only. A second pass costs as much as the first and fixes
|
|
far less, and refining is a text edit.
|
|
|
|
### It returns a patch, not a deck
|
|
|
|
```json
|
|
{"changes":[
|
|
{"slide":2,"action":"two"},
|
|
{"slide":4,"action":"split","after":3,"heading":"Management (continued)"},
|
|
{"slide":6,"action":"compare","at":3,"labels":["MILD","SEVERE"]}
|
|
]}
|
|
```
|
|
|
|
Asking for the corrected deck back put the reply in proportion to the *deck*
|
|
rather than to the number of problems — a fourteen-slide deck came back cut off
|
|
mid-object every time, at any output budget the provider would honour.
|
|
|
|
The patch is better for a second reason. The reviewer names a slide and an
|
|
action; the server moves the text it already has. The words never pass through
|
|
the model at all, so a review cannot reword, drop or invent a single bullet —
|
|
which is a stronger guarantee than instructing it not to and checking
|
|
afterwards. The check still runs: body text must come out the same multiset,
|
|
figures the same set, and a heading may only be reused or extended. A
|
|
continuation heading is the reviewer's one piece of text, and it is replaced
|
|
with `"<original> (continued)"` if it does not continue anything.
|
|
|
|
Nothing here can fail a generation. No reviewer, an unreachable one, an
|
|
unparseable reply, a deck longer than `MAX_SLIDES`, or a patch that applies to
|
|
nothing — each returns the deck that was written.
|
|
|
|
### Cost
|
|
|
|
One image per slide on every presentation generated. Pick a cheap capable vision
|
|
model rather than the best one available; `openrouter-gemini-3.8-flash` is a
|
|
reasonable default. Measured on a three-slide deck: three images in, one change
|
|
out.
|
|
|
|
## Modify
|
|
|
|
`POST /api/my-resources/:id/refine` rewrites a resource in place, keeping its
|
|
id, its downloads and its References section. It offers the same four sources as
|
|
generating — it had none, so "add what the 2024 trial showed" was answered from
|
|
the model's memory rather than by looking anything up.
|
|
|
|
The previous version is replaced, not versioned.
|