857ed341f5
- Drop first/second-person voice; reference-style prose throughout - Remove stale information; align with current code (argon2id primary, hybrid cookie/Bearer auth, sliding 24h idle, AES-256-GCM PHI at rest, backup codes, node-pg-migrate, collation-drift guard, multi-arch Docker, auto-version pipeline) - Preserve all technical accuracy and code examples - Remove any remaining references to separate PedsHub Quiz app - Keep consistent tone across files (tables + code blocks, imperatives where needed) - api-reference.md and developer-guide.md route tables expanded to reflect current routes (billing, sessions)
3.4 KiB
Speech: STT, TTS, audio backup
Transcription (speech-to-text)
Overview
POST /api/transcribe accepts multipart/form-data with a single audio
file (≤ 25 MB). Provider is TRANSCRIBE_PROVIDER env var, or auto-detected
(google > aws > openai) from available credentials. Each user may override
via users.stt_model; admin-wide default via stt.model in app_settings.
Providers
| Provider | Transport | HIPAA (with BAA) |
|---|---|---|
| Google Gemini | Inline audio in generateContent call. Default model gemini-2.0-flash. |
Yes |
| Amazon Transcribe | Streaming. AWS_TRANSCRIBE_MEDICAL=true + AWS_TRANSCRIBE_SPECIALTY switches to Transcribe Medical. Specialties: PRIMARYCARE, CARDIOLOGY, NEUROLOGY, ONCOLOGY, RADIOLOGY, UROLOGY. |
Yes |
| Local Whisper | Spawns whisper.cpp or faster-whisper via WHISPER_BINARY. Fully offline. Model sizes tiny/base/small/medium/large. |
N/A (nothing leaves host) |
| OpenAI Whisper | whisper-1 via /v1/audio/transcriptions. Medical-context prompt prepended: "Medical patient encounter. Pediatric." |
No |
| LiteLLM | Inline audio via LiteLLM's chat.completions endpoint (not the /audio/transcriptions path). Model from LITELLM_STT_MODEL. |
Depends on LiteLLM backend |
Browser Whisper (fully offline)
Runs entirely in the browser via WebAssembly. Zero network. Suitable when no external transcription is acceptable.
- Runtime:
@xenova/transformers(WASM). - Models (bundled in the Docker image, no CDN fetch):
whisper-tiny.en— 39 MBwhisper-base.en— 74 MBwhisper-small.en— 244 MB
- Executes in a dedicated Web Worker; UI thread is never blocked.
- Models cached in IndexedDB after first load.
- Per-user toggle. On browser transcription failure, the client falls back to server-side transcription without user intervention.
Live speech preview
Chrome / Edge webkitSpeechRecognition streams interim text to the UI during
recording. Used for real-time preview only — not for final transcription.
The actual transcript comes from the configured STT provider after recording
ends.
Text-to-speech
Overview
POST /api/text-to-speech. Returns audio/mpeg. X-TTS-Provider response
header identifies the provider used. 5000-character limit per request. Each
user may override via users.tts_voice; admin-wide default via tts.voice.
Providers
| Provider | Notes |
|---|---|
| Google Cloud TTS | @google-cloud/text-to-speech. Voice families: Journey, Studio, Neural2. |
| LiteLLM | Configured via LITELLM_TTS_MODEL + LITELLM_TTS_VOICE. Backend-agnostic. |
| ElevenLabs | eleven_turbo_v2_5. Not HIPAA-compliant. |
Audio backup
Raw audio is saved to Postgres only when transcription fails, providing a retry window without persisting every recording.
Storage
- Gzip-compressed, then AES-256-GCM encrypted (0x01 version byte prefix).
BYTEAcolumn inaudio_backups.- 24-hour
expires_at, swept hourly. - Legacy rows (gzip magic
0x1Fas first byte, no encryption envelope) decompress as-is — detection is deterministic because0x1F ≠ 0x01.
Retry UI
Settings → Audio Backups:
- List: module, size, created, expiry.
- Retry — resubmits to
POST /api/transcribe. - Delete — purge now.
Browser fallback
If the server-side save fails (network, 500, etc.), the client stores the audio in IndexedDB so it can retry later. Cleared after successful submission.