67e416c6d9
Some checks failed
Forgejo Android APK / Root app tests (push) Successful in 47s
Forgejo Docker Build / Root app tests (push) Successful in 48s
Forgejo Android APK / Build signed APK (push) Successful in 2m38s
Forgejo Docker Build / Build Docker image (push) Successful in 12s
Forgejo Docker Build / Deploy to the host (push) Failing after 2s
Three pairs of docs described the same thing twice, and the copies had drifted
apart. Merged each into one file, keeping the unique content from both:
- ARCHITECTURE.md -> architecture.md (its operational map: ownership, request
flow, runtime boundaries, source of truth, deployment shape)
- DEVELOPMENT.md -> developer-guide.md (change workflow, Clinical Assistant
high-risk areas, frontend rendering rules, deployment checks)
- transcription-options.md -> speech.md (the clinic setup table, and the list
of browser-Whisper paths that must stay removed)
Then audited what remained against the code and the live database rather than
against the previous docs. Corrected:
- Google Vertex was still documented as a provider across nine files. The SDK
is gone; AI_PROVIDER=vertex now logs an advisory and falls back to
OpenRouter, and Gemini is reached through LiteLLM. Fixed the provider
selection order to match src/utils/ai.js, which starts from LITELLM_API_BASE.
- promptSafe was documented on 8 routes; it is on 13.
- Node 20 -> 24, "24 vanilla JS modules" -> no fixed count, and
transcribe.js/tts.js -> sttProvider.js/ttsProvider.js, which is what exists.
- STT/TTS are LiteLLM-only; README listed direct Google, AWS Transcribe and
ElevenLabs paths that are not in the runtime.
- Learning Hub PPTX export was documented as pptxgenjs, which is not a
dependency. It is pandoc against a reference deck.
- POST /api/admin/milestones/seed does not exist; it is /bulk-import.
- NEXTCLOUD_URL and NTFY_TOPIC are not read anywhere. Nextcloud is per-user in
the users table, and the ntfy topic is derived as pedscribe-{userId}.
- A prose paragraph sat inside the Clinical Assistant settings table, so half
the rows rendered as text.
Filled the gaps the audit exposed:
- database.md was missing 12 of 29 tables, including user_resources,
personal_notes, login_codes, registration_invites and generated_image_jobs.
- developer-guide.md was missing 11 routers and 10 frontend modules.
- api-reference.md detailed 121 of 244 endpoints and said so, but whole
features were absent. Added an endpoint index covering Clinical Assistant,
My Resources, Notes, Diagrams, ED Encounters, invites and sign-in codes.
- configuration.md was missing METRICS_TOKEN, REDIS_URL, API_RATE_LIMIT_MAX,
the LITELLM_* model variables, the DB_* ones maintenance.js reads, and the
per-purpose S3 resolution scheme.
- clinical-assistant.md documented 2 of its 17 environment variables.
- features-explained.md had no entry for My Resources or Clinical Assistant.
Renamed the three remaining SHOUTING filenames to kebab-case, which is what the
docs viewer's prettyName() was working around, and rewrote README's index,
which listed architecture.md twice and omitted nine files.
Noted but not changed: the Turnstile site key is hardcoded in index.html rather
than read from TURNSTILE_SITE_KEY, and /api/health/detailed can report
tts: 'elevenlabs' though no ElevenLabs path exists.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU
103 lines
3.8 KiB
Markdown
103 lines
3.8 KiB
Markdown
# AI, Speech, And Post-Note Helpers
|
|
|
|
This doc summarizes the current AI/STT/TTS pipeline without line-number
|
|
citations. For exact behavior, read `src/utils/ai.js`, `src/routes/transcribe.js`,
|
|
`src/routes/tts.js`, and the relevant frontend scripts.
|
|
|
|
## Text Generation
|
|
|
|
All text-generation routes call `callAI(messages, options)` from
|
|
`src/utils/ai.js`.
|
|
|
|
Supported providers:
|
|
|
|
- LiteLLM or another OpenAI-compatible gateway.
|
|
- OpenRouter.
|
|
- AWS Bedrock.
|
|
- Azure OpenAI.
|
|
|
|
Google Vertex is no longer a provider of its own; the Google SDK was removed
|
|
and Gemini is reached as a LiteLLM-configured model.
|
|
|
|
`AI_PROVIDER` can explicitly choose the provider. If unset, the startup loader
|
|
initializes configured clients and the final active provider follows the current
|
|
load order described in [`../ai-providers.md`](../ai-providers.md). Route
|
|
handlers do not call provider SDKs directly.
|
|
|
|
## Model Allowlist
|
|
|
|
`callAI()` rejects model IDs outside the active server-side allowlist unless a
|
|
specific admin test path opts out. The allowlist is assembled from built-in
|
|
provider models, `models.disabled`, and `models.custom` in `app_settings`.
|
|
|
|
The default model comes from the configured provider/model settings. Admins can
|
|
set defaults and custom models from the Admin Panel.
|
|
|
|
## Prompt Safety
|
|
|
|
Clinical routes should build prompts with:
|
|
|
|
- canonical templates from `src/utils/prompts.js`
|
|
- optional DB prompt overrides through `app_settings` keys `prompt.*`
|
|
- `INJECTION_GUARD`
|
|
- `wrapUserText(label, text)` around user-derived text
|
|
|
|
User-derived text includes transcripts, dictated notes, pasted chart data,
|
|
refine instructions, template preferences, and patient education source notes.
|
|
|
|
## User Templates
|
|
|
|
`getUserMemoryContext()` fetches `/api/memories/context` and passes the returned
|
|
template/preference context as `physicianMemories`. Server routes wrap that block
|
|
as low-priority style/template context. `custom` memories and legacy
|
|
`correction_*` rows are not prompt context.
|
|
|
|
## Speech-To-Text
|
|
|
|
`POST /api/transcribe` accepts one audio file up to 25 MB. Provider selection:
|
|
|
|
- explicit `TRANSCRIBE_PROVIDER=litellm`, or
|
|
- auto mode when `LITELLM_API_BASE` is configured.
|
|
|
|
Direct Google, AWS, local Whisper, and OpenAI Whisper branches are not part of the runtime. Browser Whisper/browser-local model downloads are also absent.
|
|
|
|
## Browser Web Speech
|
|
|
|
Browser-native Web Speech is an explicit opt-in preview. It may rely on browser
|
|
vendor cloud services and must not be treated as the final clinical transcript.
|
|
Final transcription should come from the configured server-side STT provider.
|
|
|
|
## Audio Backup
|
|
|
|
Failed transcription attempts can create encrypted 24-hour audio backups through
|
|
`src/routes/audioBackups.js`. The user can retry or delete backups from
|
|
Settings. Browser fallback storage is only for cases where the server cannot
|
|
store the failed recording.
|
|
|
|
## Text-To-Speech
|
|
|
|
`POST /api/text-to-speech` returns audio from LiteLLM and marks the LiteLLM
|
|
model in `X-TTS-Provider`. Voices are LiteLLM-compatible strings configured by
|
|
`LITELLM_TTS_VOICES`.
|
|
|
|
## Post-Note Helpers
|
|
|
|
Generated note outputs can expose helper panels:
|
|
|
|
- `refineDocument` for editing/refining/shortening generated text.
|
|
- `suggestBillingCodes` for clinician-facing ICD/CPT suggestions.
|
|
- `suggestDontMiss` for clinician-facing safety review.
|
|
- `attachPatientEducation` for parent-facing handout drafts.
|
|
|
|
These helpers are authenticated API-backed actions. They should treat the edited
|
|
note as the source of truth and keep the clinician in the review loop.
|
|
|
|
## Change Checklist
|
|
|
|
When changing this area:
|
|
|
|
1. Keep provider-specific code inside utility/provider modules.
|
|
2. Wrap all user-derived text with `wrapUserText` before AI calls.
|
|
3. Do not add browser-local Whisper back without a new design review.
|
|
4. Do not add clinical answer response caching.
|
|
5. Run touched-file `node --check` commands and `npm test`.
|