9 commits

Author	SHA1	Message	Date
Daniel	03621752e8	feat: image fallback chains for every workflow, and a library worth looking at ... **Fallbacks.** One image model meant a refusal, a rate limit or a model the gateway had since dropped ended as a missing picture. Every workflow now tries its model, then each fallback in order, stopping at the first that produces an image. Primary plus two, capped: each hop is a paid request, and a chain long enough to need a cap is long enough to surprise someone. My Resources previously had no fallback at all — only the Clinical Assistant did, and only one. That is backwards: a missing figure is most visible in a deck, where it leaves a hole in a slide. The retry rule is now a classifier that says *why*, rather than a boolean. Transient faults, a 404 for a model the gateway does not have, and a content refusal all move to the next model — a refusal because policy is a vendor decision, not a fact about the request. 401/403 stop immediately (one gateway, one set of credentials, the next model fails identically), as do 413 and any other 4xx, which are malformed everywhere. Refusals are recognised from the message: no provider sends a machine-readable reason and the status varies. Each hop re-leases the job, so a chain cannot outlive its claim and let a second worker repeat the same paid work, and the row records the model actually being paid for so a picture made by the third model is not attributed to the first. The old singular `fallback_image_model` is still read, so an existing configuration keeps working without anyone re-entering it. **Library.** Documents/Images tabs in My Resources, with a real grid: fixed aspect tiles so the rows line up whatever shape the pictures are, a source badge on the picture, two-line prompt, hover lift, shimmer skeletons while thumbnails land, and a lightbox that closes on Escape or the backdrop and restores focus. Actions are hidden on hover only behind `@media (hover:hover)` — hiding delete behind :hover would put it out of reach on touch and keyboard. Downloads go through privateImageBlob rather than a bare `<a download href>`: a mobile client's session is a bearer token an anchor cannot send, and these assets are served no-store on purpose. The gallery lives in My Resources only. Assistant images appear in it, which was the point; the assistant page does not grow a gallery of its own, and a test asserts no assistant module lists the endpoint. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU	2026-09-12 14:53:04 +02:00
Daniel	259b4858be	feat: an image library — every picture this account has made ... A figure outlives the deck it was drawn for: the deck gets replaced, the diagram is still good. Until now a generated image could only be seen in the resource it was made for, and there was no way to find one again or to delete it. Library → Images is a grid of every finished image the account owns, across all three workflows, newest first. GET /api/generated-images is scoped by owner_id in the statement rather than filtered after, returns only finished jobs — an unfinished one is a broken frame in a gallery — and pages by keyset, because a gallery that grows while you scroll repeats or skips rows under OFFSET. Most of this already existed. Thumbnails were already rendered at 256 and 640 by sharp and already served by ?w=, with their own checksum so the client's tamper check passes on a derived copy; hydrateImage already handles auth, the account boundary and caching. The tiles ask for the 256px preview, so thirty of them cost a few kB each rather than thirty full-size downloads, and the prompt is decrypted for the caption because it is the only human-readable label an image has. Deleting needed new work. The storage interface had no remove at all, so a delete that dropped the row would have left the object and both previews in the bucket — paid for, and still readable by anything with credentials. Storage now removes all three keys, and the bytes go before the row: a row pointing at a missing object is a broken image in a gallery, while an object without its row is only wasted space, and unreachable storage refuses the delete outright rather than reporting a success that left the picture behind. THUMB_WIDTHS now has one definition, in generatedImageStorage. Two copies drift, and the drift that matters is a width that gets written and never deleted. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU	2026-09-12 14:36:30 +02:00
Daniel	e244ee5240	feat: modifying a deck can see it ... The model editing a deck could not see it, which made most of what people actually ask for unanswerable: "that slide is too crowded", "the diagram is in the wrong place", "this one looks empty" are facts about the rendered page, not about the JSON. When a vision model is configured, modifying now renders the current deck — with its figures, unlike the review pass, which runs while they are still being drawn — and hands the model one image per slide alongside the JSON. Same pipeline as review, reused rather than reimplemented: pptx, Gotenberg, PDF, pdftoppm, capped at MAX_SLIDES. The vision model then does the editing, which is a second and separately measured benefit. On a real 20-slide deck, ds-deepseek-v4-flash returned the deck unchanged for "make it better" — the echo reported yesterday — while openrouter-gemini-3.8-flash applied it. So the stronger model fixes the echo even without sight. A model the author picks explicitly still wins over both. The result is rendered and reviewed again. Generation-only was the old rule, on the reasoning that refining is a text edit; it is not. The edit is made against how the deck looked before it, so a slide that gains two bullets only overflows once it is rendered again. The reviewer may reposition but is held to the same words, so a verification pass cannot quietly undo what was just asked for. Whether an instruction achieved anything is judged on the model's edit, before the reviewer runs, or a reviewer nudging a slide into two columns would mask an instruction that did nothing. Sight is an upgrade, never a dependency: no vision model, Gotenberg down, a render that fails — each falls through to editing blind, and a test covers each of those paths. Verified against two mutations: keeping the text model when images are attached, and dropping the verification pass, each fail exactly one test. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU	2026-09-12 06:55:16 +02:00
Daniel	973f9d554f	feat: a text-only model cannot be set as the slide reviewer ... The reviewer is shown rendered images of the deck, so a text-only model there fails on every generation — at request time, long after the one moment an administrator could have picked differently. Nothing checked it. Saving my_resources.review_model now asks the gateway what it reports for that model and refuses only an explicit supports_vision === false. Three answers, not two. Most of this roster carries no supports_vision at all (every openrouter-* entry here), and refusing unknowns would block the reviewer this deployment already runs on. An unreachable gateway is not evidence about a model either, so it never blocks the save. Empty means review is off and skips the lookup entirely. The capability is read from the gateway, never inferred from the model id. Verified against a mutation: relaxing `canSee === false` to `canSee !== true` fails the two tests that say unknown must stay allowed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU	2026-09-12 06:42:44 +02:00
Daniel	4f8e686907	feat: record what a deck wanted and could not have ... The shape vocabulary is deliberately small, which leaves the question of what to add next. Rather than guess, it now records demand. Two signals, because a model asks both ways. It can say so outright — {"kind":"unsupported","need":"a SmartArt cycle of four stages"}, which draws nothing and is told about in the same file that validates it — or it can reach for a kind, chart type or slide type that does not exist, which is the more common way of asking and just as much of a signal. Both produce a log line naming what was wanted and the topic it came up on, and increment ped_ai_deck_vocabulary_gap_total{wanted}, so it can be counted over time in Grafana rather than noticed once and forgotten. Deduplicated per generation and capped at twelve: a model that asks for a hundred things it cannot have should not write a hundred log lines. It can never fail a generation — it is a note to whoever decides what to build next. This is also the answer to whether to run model-authored code in a sandbox instead. The log will say whether the gap is real. Some of it is not closeable by any sandbox, being python-pptx's own ceiling — no SmartArt, no animations or transitions, limited chart types — and a sandbox would only let a model write code against the same library and hit the same wall. Documented in docs/my-resources.md, which the in-app Docs tab serves directly. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU	2026-09-11 22:51:15 +02:00
Daniel	af2e09c1de	feat: a slide can be drawn from primitives when the named layouts have no word for it ... The nine layouts are a fixed vocabulary and a good default, but "lay the three severity levels out left to right with arrows between them" had no expression in them at all. A "custom" slide now carries a list of shapes: positioned text, eight autoshape families, lines, images, tables, and native PowerPoint charts — column, bar, line, pie, doughnut. Coordinates are percentages of the slide rather than EMU, because a model reasons about "the left half" and not about 12192000. Shapes draw in array order, so a later one sits on top. The model never emits Python. It names shapes and the renderer draws them: running model-authored code to lay out a slide would be an enormous amount of trust to buy a feature, on a server holding clinical data and secrets. Validation lives beside the text that teaches the vocabulary, in one file, so what the model is told about is exactly what is accepted. Kinds are an allowlist, colours must be six hex digits, coordinates are clamped inside the slide — a shape at x=95 w=30 is cut to the edge rather than drawn half off it — counts are capped, a pie is held to one series, and anything that cannot be understood is dropped. A custom slide that loses every shape becomes a plain one rather than a heading over an empty frame, and one bad shape is caught in the renderer so it cannot cost the slide it sits on. A figure on a custom slide is requested through an image shape, drawn by the same path as any other, and attached by job id. Word renders a custom slide as its words in reading order with its tables and figures — lossy, and better than dropping the slide. Verified live end to end: asked to "lay the three severity levels out left to right as coloured boxes with arrows between them", the model produced [rect arrow rect arrow rect], chose green/amber/red itself, and the rendered slide was looked at. A column chart beside its commentary renders with real axes and gridlines. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU	2026-09-11 22:12:26 +02:00
Daniel	154b896d5b	feat: Word is built by python-docx from the same typed source as the deck ... Pandoc reads markdown, so every Word export had to flatten the resource to markdown first — and a deck flattened to markdown stops being one. A comparison became two headings and two lists, a callout became bold text, and a figure became nothing at all, because markdown has nowhere to put it. src/utils/docSpec.js reduces either source to the same blocks: a stored deck where there is one, the markdown where there is not. scripts/render_docx.py draws them. A comparison comes out as a labelled two-column table, a callout as a shaded box, a table as a real table, a figure embedded at its own aspect ratio with its caption, and speaker notes as muted indented text. The deck wins over the markdown beside it, because that markdown is a serialisation of the deck and reading it instead would be reading a lossy copy of what is right there. Word now carries the figures too. The export route skipped fetching them for docx, which was correct when pandoc could not place them and wrong the moment this could. Pandoc stays installed and stays the fallback: a plainer document beats a failed download. Both renderers now share one spawn helper. Verified end to end: a deck with two figures exported as a six-page Word document with both images embedded (537KB, two files in word/media), rendered to PDF and looked at — the comparison is a labelled table, the figure sits at its true aspect ratio, and the notes read as notes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU	2026-09-11 21:58:53 +02:00
Daniel	9d307fd442	feat: a vision model looks at the rendered deck and fixes the layout ... The model that writes a deck never sees it. It cannot tell that slide four overflowed, that a nine-item list would read better in two columns, or that two labelled groups want to be a comparison — those are facts about the rendered page, not about the text. So each generated deck is now rendered to PDF through Gotenberg, rasterised to one image per slide with pdftoppm, and shown to a vision model. Off unless an administrator names a reviewer, in its own admin card because it is the one setting that spends money on every generation without a user having asked for anything. One pass, on generation only: a second pass costs as much as the first and fixes far less, and refining is a text edit. It returns a patch, not a deck. Asking for the corrected deck back put the reply in proportion to the deck rather than to the number of problems, and a fourteen-slide deck came back cut off mid-object at every output budget the provider would honour — measured twice before changing shape. The patch is better for a second reason. The reviewer names a slide and an action — two columns, one column, split after bullet N, compare with these two labels — and the server moves the text it already has. The words never pass through the model, so a review cannot reword, drop or invent a single bullet. That is a stronger guarantee than instructing it not to and checking afterwards. The check runs anyway, because a bug in applyChanges would be as bad as a model rewriting the words and worse for being trusted: body text must come out the same multiset, figures the same set, and a heading may only be reused or extended. A continuation heading is the reviewer's one piece of text and is replaced when it does not continue anything. Nothing here can fail a generation — no reviewer, an unreachable one, an unparseable reply, a deck too long to look at, or a patch that applies to nothing each return the deck that was written. Verified end to end against a deck with a deliberately overloaded slide: three slides rendered and sent, one change returned, ten bullets split into five and five under "Stepwise Management … (continued)", text intact. Left switched off; enable it under Admin. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU	2026-09-11 21:38:18 +02:00
Daniel	fa2e7523d6	docs: My Resources, sign-in codes, invitations, and what the image carries ... Nothing documented My Resources, the slide renderer, PubMed or web search, and the authentication doc predated both sign-in codes and registration invitations. docs/my-resources.md is new and covers the feature end to end: what a resource is, where its material comes from, why both searches run in the route rather than as tools the model never called, why keyword engines get the topic while retrieval gets the instruction too, how a presentation is designed as a deck rather than written as markdown, the separate multi-image path, and what the export pipeline is made of. docs/authentication.md gains sign-in codes — storage, lifetime, reuse, supersession, guessing, and that two-factor still applies — and registration invitations, including the exact condition that decides when a code may be deleted and why it is written to match the status the list displays. Both new rate limits are in the table, with a note that Express matches app.use paths on segment boundaries, so a new sign-in endpoint needs its own limiter or it has none at all. docs/deployment.md now says what the runtime image carries and why — pandoc for Word, python3 with apk-installed lxml and pillow for the slide renderer, python-pptx pinned, and that PDF conversion is not in the image at all but goes to Gotenberg, so Word and PowerPoint still work when it is down. docs/configuration.md picks up LOGIN_RATE_LIMIT_MAX, LOGIN_CODE_RATE_LIMIT_MAX and GOTENBERG_URL, none of which were listed. README gains a My Resources section and indexes the two new docs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU	2026-09-11 20:49:10 +02:00