Commit graph

21 commits

Author SHA1 Message Date
Daniel
8a6a4df121 refactor: citations are a markdown-it token, and the numbers you see are display order
The old renderer rewrote the text: it found "[n]" with regexes, renumbered
them, and swapped the result back in — which broke inside `arr[2][1]`, inside
HTML attributes, and whenever two turns disagreed about what "[3]" meant. It
also had a fallback markdown renderer of its own for when the rewrite
produced something markdown-it would not parse.

Now "[n]" is an inline rule registered on the same markdown-it instance that
renders everything else. The parser decides what is prose and what is code, a
link, or a URL, so the rule never sees "[1]" inside a code span, and it steps
aside for "[1](url)". Math is two more rules on the same parser instead of a
regex pre-pass, so "$" inside a URL is no longer math.

Identity vs display: the stored "[n]" and each card's id are the source's
identity (sourceNumber) and are never rewritten. The number a reader sees is
the order of first appearance, computed at render time from the token stream
(orderSourcesByCitation), so "one, then seven" cannot happen and a saved chat
re-opens pointing at the same cards it was saved with. Stored messages and
sources are untouched; export and the modal resolve by identity.

Translated HTML gets the same links through a TreeWalker over text nodes
(linkCitationsInHtml) rather than a regex over markup.

Deleted: renderCitationLinks, normalizeAdjacentCitationClusters, the
fallback renderer (fallbackMarkdown/renderMixedList/renderFallbackTable),
renderLatexText, CITATION_SCAN. Tests that asserted rewritten text now assert
token output; harnesses that render for real are given a parser.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU
2026-09-13 04:41:40 +02:00
Daniel
0e1e74882e feat: sources are numbered in the order the answer cites them
Borrowed from the quiz app's AI Mode, where validating citations and
ordering them fall out of the same pass: it collects the sources an
answer actually used into an insertion-ordered map, so the list comes
back in first-citation order for free.

Ours listed sources in retrieval order — an order the reader never sees
and has no way to follow. An answer whose first citation was [7] opened
a list that began at [1], so matching a marker to a source meant
hunting. Reference lists in published writing are numbered by first
appearance for exactly this reason.

Cited sources now come first, renumbered by first appearance, and the
markers in the text are rewritten to match. Anything retrieved and not
cited keeps its place after them, labelled "not cited" — the panel is
also a view of what the search returned, which is worth keeping, but it
should not sit among the numbers the answer used.

The marker itself now shows its number instead of the word "src". Every
citation read identically, so the only way to tell one from another was
to hover it — which made the numbered list beneath useless to match
against. The export has shown numbers since the day "src" was
introduced, with no recorded reason for the difference.

Renumbering happens once the whole answer is known, never while
streaming: the order is the order of first citation, so a citation that
has not arrived yet cannot take its place, and numbers would shuffle
under the reader mid-sentence. The text is rewritten in a single pass —
number by number would turn 2 into 1 and then that 1 into whatever 1
maps to.

An invented citation reserves no position and is left exactly as it was.
It is still not turned into a link, and citation_audit still records it;
what matters here is that it cannot push a real source down the list.

Accuracy was already held: a marker with no matching source never
becomes a link. This changes what a reader can do with the ones that are
real.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU
2026-09-13 03:26:17 +02:00
Daniel
89e71b3b86 fix: stream the assistant's answer a block at a time
Markdown only means anything once a block is finished. Half a table is a
row of pipes; half a fence is a stray ```. Re-parsing the whole partial
answer every 180ms therefore flickered between a broken parse and the
real thing, and the previous answer to that was to abandon markdown
past 3500 characters or eight pipe rows and dump raw text into a <pre>
that had no CSS at all — inheriting the browser's black monospace
default. That is the dark block people saw while a table streamed.

Now the text is split at the last finished block — a blank line outside
a code fence — and everything before it is rendered once and *appended*.
Only the unfinished tail is plain text, styled as prose. Settled content
is never re-parsed and never rebuilt, so a diagram or chart that has
already drawn is not thrown away by the next token.

Two blank lines are not boundaries: the gap inside a loose list, which
would render one list as two each restarting at 1, and one inside
indented code. Telling the first from the perfectly good boundary
between a sentence and the list it introduces takes looking at both
sides of the gap, not just ahead.

renderEmbeddedBlocks now marks what it has drawn. mermaid.render is
async and replaces the element; a second Chart on one canvas throws.
Marked before the await, not after, or two frames race the same element.

Streaming is a preview: 'done' still renders the whole answer from
scratch, so anything transient here is settled by the final pass. That
is what makes appending safe.

Mutation-tested: removing fence tracking, the list guard, the backward
half of that guard, the append, the drawn marker, or the state reset in
fillMessageBubble each fail a test.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU
2026-09-12 21:36:51 +02:00
Daniel
050a7d5241 feat: citation quality tracking, and the SSO settings fit a phone
Citation quality
- A citation naming a source that never came back is never rendered as a
  link, so it appears as plain text and nobody learns it happened. It is now
  measured on the server, where the answer and the sources both exist, so it
  is seen whether or not a browser rendered it.
- Four Prometheus counters feed a Grafana dashboard (Ped-AI Citation
  Quality): answers, citations written, answers affected, and individual
  unresolved markers. Only answers with at least one unresolved citation are
  stored, with the question and the titles retrieval returned, so an operator
  can judge whether retrieval came back thin or the model over-cited. Rows
  expire after 30 days: this is a quality signal, not a transcript log.
- Both answer paths are covered. /chat/stream is normal; /chat is the
  fallback the client uses when streaming fails, so auditing only the first
  would have hidden exactly the answers produced under failure.
- The tracker is resolved on demand and allowed to be absent. Seven test
  files load this route with a hand-built list of permitted imports, and
  adding a hard dependency would mean editing all seven — and the eighth
  written later would break. Observation must never be able to fail an
  answer, so a missing module simply means no tracking.
- Metric registration reuses an already-registered counter, because this
  module can legitimately load twice in one process.

SSO settings on mobile
- Six rows were laid out inline: flex with a 160px label and an input that
  would not shrink, so on a phone the row was wider than the screen with
  nothing to scroll and no way to reach the rest. They use .admin-row now,
  which already stacks below 640px. Verified at 390px and 360px: nothing
  off-screen, no sideways overflow.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU
2026-09-10 23:44:43 +02:00
Daniel
db83255c58 feat: display-only sources toggle, signed-out preview, and a composer that carries the toolbar
Sources (correcting what I built earlier)
The previous toggle branched the SYSTEM PROMPT, so the same question could get a
different answer depending on a display setting — the bias this was meant to
avoid. The prompt is now unconditional: buildSystemPrompt takes no display
argument and is byte-identical either way. Hiding sources happens on the way out
— the server omits them and strips the now-orphaned [n] markers from the copy it
sends. The answer is generated, stored and exported with citations intact, so
turning the setting back on restores them without re-asking anything. Renamed to
clinical_assistant.show_sources; the old key is still honoured.

Signed-out preview (admin opt-in, default off)
A visitor may try the assistant; reaching for the workspace asks them to sign in.
Deliberately narrow:
- Reachable paths are an exact allow-list, not a pattern, so a new endpoint is
  private unless someone adds it on purpose.
- A preview visitor gets no identity at all (id: null), so nothing can be owned,
  saved, billed or addressed to them.
- The image tool is withheld rather than left to fail on a null owner, and no
  audit rows are written.
- A caller presenting a token is authenticated normally, so preview can never
  downgrade a real session; if the setting cannot be read, authentication is
  required.
- Actions needing an account are hidden rather than offered and refused.

Composer
The bar above the transcript is gone. Patient take home, Export PDF, Download
transcript and Attach images moved into a + menu in the composer, and the model
selector moved beside send — shown only when there is more than one model, as
before. Both views now start at the same top edge, so switching modes cannot
nudge the page up or down. On an empty transcript the tiled ground runs behind
and below the composer, which floats on it above centre.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GmpYHPSLGmXGZMyLpn2Lbe
2026-09-10 04:12:36 +02:00
Daniel
b992c6600b feat: admin can turn citations off
New setting clinical_assistant.citations_enabled (default true, admin checkbox).
With it off, retrieval, grounding and every other rule are unchanged — answers
are still built only from retrieved sources — but:

- buildSystemPrompt swaps only the citation block: the "cite factual claims with
  [1]" rules are replaced with "do not include citations, source numbers or
  bracketed markers", and the note that the sourcing requirement itself is
  unchanged. Grounding, scope, table formatting and tone rules are byte-identical
  between the two modes.
- The server strips any stray [n] the model emits anyway, from the stored answer
  rather than only the view, so saved chats and exports match what was shown.
- No sources are sent to the client at all, and the status endpoint reports the
  mode so the UI hides the Sources panel and gives its 330px column back to the
  chat instead of showing an empty rail.

Validated as a boolean in adminConfig, like the feature.* keys.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkfrkQwA4YGrGw9LZSpeAq
2026-09-09 23:06:13 +02:00
Daniel
653d6d6ed6 fix: strip [src]/[source] placeholder tokens from rendered answers instead of showing src
All checks were successful
Forgejo Android APK / Root app tests (push) Successful in 59s
Forgejo Android APK / Build signed APK (push) Successful in 2m15s
2026-09-09 04:22:15 +02:00
Daniel
491f5e02b7 Render escaped clinical citations as links
All checks were successful
Forgejo Android APK / Build signed APK (push) Successful in 1m49s
2026-09-01 19:19:11 +02:00
Daniel
e4dacbaf09 Rejoin detached clinical citations
All checks were successful
Forgejo Android APK / Build signed APK (push) Successful in 1m50s
2026-09-01 19:11:18 +02:00
Daniel
34eef6ec6b Keep prose citations inline
All checks were successful
Forgejo Android APK / Build signed APK (push) Successful in 1m50s
2026-09-01 18:51:39 +02:00
Daniel
e71a7e22e1 Keep clinical citations inline
All checks were successful
Forgejo Android APK / Build signed APK (push) Successful in 1m54s
2026-09-01 18:45:44 +02:00
Daniel
3d4a95fea4 Revert the clinical assistant markdown changes and fix mermaid rendering only
All checks were successful
Forgejo Android APK / Build signed APK (push) Successful in 1m50s
Reverts 126d7928 and 2d292d12 in full. The markdown normaliser and the trailing
emphasis and dollar handling return to exactly the state tagged
pre-citation-fixes-20260828, which has been rendering acceptably in use.

The one change kept is mermaid. DOMPurify 3.1.6 strips an attribute whose value
contains "-->", and every mermaid flowchart contains one, so the sanitiser
removed data-mermaid and querySelectorAll('[data-mermaid]') never matched: the
diagram sat on "Rendering graph..." forever. Reproduced against the exact pinned
build from cdnjs; sequence diagrams using "->>" were unaffected, which is why
only flowcharts failed.

The diagram source is now percent-encoded into the attribute and decoded when
read, so no arrow ever appears in an attribute value. Nothing about the
sanitiser configuration changes and no markup is newly allowed. The reader
tolerates an unencoded value, so anything already in flight still renders.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 19:55:03 +02:00
Daniel
2d292d12af fix: make the markdown repairs line aware so tables and ranges survive
All checks were successful
Forgejo Android APK / Build signed APK (push) Successful in 1m58s
The earlier fix corrected two of the repair rules and left the rest with the same
flaw, and the punctuation boundary it introduced let the table case back in. A
review found the remainder.

Every repair rule inserted newlines with no idea whether it was inside a markdown
table row. A dose row such as "| Dexamethasone | 0.6 mg/kg PO (max 16 mg) - single
dose | 1 |" was split mid-row, which drops that row's citation and every row
below it out of the table. A "#" column destroyed the table outright. The rules
now skip table rows, dividers and headers entirely.

Ranges were still split in three other places: in a heading, so "### Dexamethasone
0.15 - 0.6 mg/kg" rendered as a heading reading "Dexamethasone 0.15"; after a
citation; and beside a hash, where "Room # 4" became a heading. A hyphen now
starts a list item only when a number does not follow, and a run-together heading
is recognised by the capital letter after the hashes rather than by position.

Adjacent citation merging could also cross a paragraph break, turning "[1]\n\n[2]
Vancomycin only if MRSA" into "[1, 2]Vancomycin only if MRSA" -- joining two
paragraphs and moving a citation onto a claim it never supported. Merging is now
limited to citations on the same line.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 19:49:01 +02:00
Daniel
126d7928a2 fix: stop the markdown normaliser mangling doses, bold and dollar amounts
All checks were successful
Forgejo Android APK / Build signed APK (push) Successful in 1m52s
Three rendering defects in the clinical assistant, all confirmed by running the
code rather than reading it.

A spaced hyphen anywhere in a sentence was rewritten as a list item, so every
numeric range written that way was split in two:

    "Give dexamethasone 0.15 - 0.6 mg/kg orally."
      -> "Give dexamethasone 0.15\n- 0.6 mg/kg orally."

which renders as a truncated sentence followed by a bullet, and a dose range
therefore reads as a different dose. The same applied to SpO2 targets, pH ranges
and age ranges. A hyphen now starts a list item only at the beginning of a line
or after sentence punctuation, which still catches the case the rule was written
for.

An answer ending in a bold phrase lost its closing marker, because trailing
emphasis was stripped unconditionally and the strip ran twice. Only an unpaired
marker is removed now.

Inline maths swallowed dollar amounts: "Costs $5 to $10 per dose" rendered the
text between the signs as an equation and dropped both signs. A $...$ span is
now treated as maths only when it contains something mathematical, so subscripts
and fractions still render.

Citation handling is unchanged and covered by the same tests: clusters still
merge and sort, unknown source numbers stay literal, links still resolve by index,
and bare numbers in a table's Source column still become bracketed tokens.

The state before these fixes is tagged pre-citation-fixes-20260828.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 19:20:39 +02:00
Daniel
97ddd87449 fix mobile assistant table streaming
Some checks failed
Forgejo Android APK / Build signed APK (push) Has been cancelled
2026-05-11 16:15:45 +02:00
Daniel
a176e1b014 protect code blocks during citation repair 2026-05-09 20:54:05 +02:00
Daniel
83d9a77160 fix table source citation links 2026-05-09 20:50:54 +02:00
Daniel
a8f364f177 stabilize assistant and add diagrams 2026-05-08 00:23:00 +02:00
Daniel
c8436d5e4c refactor clinical assistant prompt handling 2026-05-07 22:52:46 +02:00
Daniel
21fb631fb5 feat: improve clinical assistant export and citations 2026-05-07 17:15:26 +02:00
Daniel
f134d0e80c feat: improve clinical assistant visual sources 2026-05-07 02:45:17 +02:00