Commit graph

9 commits

Author SHA1 Message Date
Daniel
bd8e413bc7 fix: an assistant attachment must be the image type it claims to be
Some checks failed
Forgejo Android APK / Root app tests (push) Successful in 59s
Forgejo Docker Build / Root app tests (push) Successful in 50s
Forgejo Android APK / Build signed APK (push) Successful in 1m56s
Forgejo Docker Build / Build Docker image (push) Successful in 9s
Forgejo Docker Build / Deploy to the host (push) Failing after 0s
The MIME type was taken on trust here. Anything at all could be posted as
image/png: it passed the size and base64 checks, was stored in the saved chat,
and was handed to a provider as a data URI. Documents and S3 uploads have always
been sniffed by fileType.js; this was the one upload path that was not.

Now sniffed with the same helper, so there is one idea of what a PNG looks like.
A PHP payload, a shell script, an ELF or PE binary, a zip, or a real PDF
labelled image/png are all refused with a message that says what is wrong.

What this does not claim: bytes hidden after a valid PNG header still make a
valid PNG, and no sniffer can promise otherwise. The protection is that the file
is never executed and never served as anything but an image.

Existing fixtures used buffers of 0x07 as stand-in images, which are correctly
refused now. They carry real file headers instead — a fixture should be the
thing it claims to be, exactly like a real upload.

Also adds the deck theme system: five palettes in assets/deck-themes.json,
render_pptx.py rebinding its palette from the theme rather than hardcoding it,
the theme carried on the deck and validated against the same catalogue the
renderer reads, a picker on the generate form, and PUT /my-resources/:id/theme
to re-skin a stored deck with no model call.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU
2026-09-12 19:00:10 +02:00
Daniel
1f06a19007 feat: a text-only model can ask a model that can see; and the image regex is gone
Some checks failed
Forgejo Android APK / Root app tests (push) Successful in 50s
Forgejo Docker Build / Root app tests (push) Successful in 48s
Forgejo Android APK / Build signed APK (push) Successful in 2m8s
Forgejo Docker Build / Build Docker image (push) Successful in 18s
Forgejo Docker Build / Deploy to the host (push) Failing after 0s
**The regex is gone.** The route ran a pattern over the user's message and
enqueued an image from the answer text when the model had not called the tool.
It was a compatibility path for models without tool calling and it did more harm
than good: it decided in English only, it could not see the conversation, and
"image summary" fell through it while reading as an obvious image request to the
model itself — which was measured, not assumed. A second and worse
decision-maker sitting behind the first. Whether a message deserves a picture is
now the model's call, made from the tool description, which is the only place it
ever belonged.

**Lending eyes.** The same shape, for a different capability. When someone
attaches a photograph and the chat model cannot accept image input, the
attachment was either refused by the provider or silently dropped — an answer
about a picture nobody had looked at, which is worse than a refusal.

The chat model is now offered look_at_image beside the image tool and decides
when to use it. The attachment goes to clinical_assistant.vision_model, whose
description comes back as a tool result, and the chat model answers in its own
voice with its own sources. Only the seeing is delegated; the clinical reasoning
stays with the model an administrator chose. The seeing model is told to report
and not to diagnose, because it has a picture and no context and an opinion from
it would carry weight it has not earned.

Delegation triggers only on an explicit supports_vision: false from the gateway.
An unknown is left alone — most of a roster reports nothing, and treating
silence as blindness would route good models through a detour. The capability
lookup moved to its own module, is cached for five minutes because it runs on
exactly the requests that are already slowest, and is never inferred from the
model id. liteLLMBaseUrl moved from the admin route to litellm.js, where the
other gateway helpers live.

The new setting is guarded like the slide reviewer: a model the gateway calls
text-only cannot be saved as the one that looks at images.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU
2026-09-12 15:43:36 +02:00
Daniel
4d92488f0c fix: translated answers keep their source chips; the patient take home can be translated
Translation
- Stop scrubbing markdown before sending it to LibreTranslate. The scrub
  deleted ordered-list numbering ("1. Give amoxicillin" -> "Give amoxicillin"),
  flattened tables into ambiguous whitespace and ate underscores inside
  identifiers. Raw markdown now goes to the translator unchanged.
- Render the translation through the same markdown pipeline as the original
  bubble, with the message's own sources, so [n] markers come back as the usual
  clickable .assistant-cite chips instead of escaped literal text. Headings,
  lists and tables survive with them.
- When the translator drops citation markers, surface the affected sources in a
  recovery block rather than letting the evidence disappear.
- Image cards are live nodes: they are now re-attached on every path out of a
  translation (success, failure and Show original), so a failed translation no
  longer silently removes a generating image from the message.

Patient take home
- Add a language selector to the take-home modal, reusing the existing
  /translate endpoint and offering only what the local LibreTranslate reports.
- Copy, Export and Email carry what the caregiver is actually reading; the
  original stays canonical behind "Original".

Conversation budget
- The admin field no longer prefills with the environment value, which turned
  the next Save into an accidental override and made the documented "leave
  empty to use the environment" path unreachable. The effective limit is shown
  as a placeholder instead.
- Report source 'default' honestly instead of naming an unset env var.
- The load-failure notice now lands on the <p> instead of an <input>'s
  textContent, where it rendered nothing.
- One validator for the budget everywhere: conversationLimit() replaces a
  parseInt that accepted "120000abc".

Other
- /assistant is addressed by its URL, not by ped_last_tab, so "/" no longer
  reopens the assistant; the URL follows tab changes and Back leaves it.
- Remove the dead DeepL path (it referenced an undefined DEEPL_BASES) and stop
  offering admins a provider the server silently ignores.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkfrkQwA4YGrGw9LZSpeAq
2026-09-09 18:40:23 +02:00
Daniel
fe63014a1f feat: image generation everywhere — deterministic fallback when the model writes text, unbounded completion poll, No-chat default with description-only generation, context passed with every image job; stop-icon send button; attach/mic bottom-left composer; single-model selectors hidden; admin image allowlist from discovered models; PDF take-home export; new-chat confirm removed; sources same-height column with internal scroll 2026-09-09 04:04:00 +02:00
Daniel
73955dd5f9 feat: model selection in chat (composer pill + popup image model), deterministic image dispatch when the model writes text instead of calling the tool, markdown-rendered email, OWUI stacked composer, left mobile drawer, Go back in topbar, DeepL removed, image history inside the Create image popup 2026-09-09 02:55:28 +02:00
Daniel
982a089fcb fix: authHeaders must include Content-Type — Express never parsed JSON bodies, causing empty "Question is required" on every chat send. Also: Learning Hub back in the main menu, sources-only right column, image gallery as the user library with poll+preview, queued job reference handling 2026-09-09 02:13:17 +02:00
Daniel
da86c7db79 fix: OWUI-style polish — fullscreen workspace replaces main menu, row-click saved chats (delete stays as hover), markdown-rendered patient handout, admin-only raw transcript, /api TTS URL, scrollable unclipped tables, action buttons hidden in exports, autosave-only copy 2026-09-08 22:41:08 +02:00
Daniel
09a4e02e39 feat: patient take home (copy/export/email), composer mic dictation, voice conversation mode, provider-aware translate languages 2026-09-08 20:01:46 +02:00
Daniel
8b072496e2 feat: Open WebUI-style assistant workspace — 3-column layout, markdown/math/code/tables, autosave with images, translation (LibreTranslate+DeepL), citation modal, Learning Hub moved in, handoff removed 2026-09-08 18:52:35 +02:00