# Speech: STT, TTS, audio backup

## Transcription (speech-to-text)

### Overview

`POST /api/transcribe` accepts `multipart/form-data` with a single audio
file (≤ 25 MB). Provider is `TRANSCRIBE_PROVIDER` env var, or auto-detected
(`google > aws > openai`) from available credentials. Each user may override
via `users.stt_model`; admin-wide default via `stt.model` in `app_settings`.

### Providers

| Provider | Transport | HIPAA (with BAA) |
|---|---|---|
| **Google Gemini** | Inline audio in `generateContent` call. Default model `gemini-2.0-flash`. | Yes |
| **Amazon Transcribe** | Streaming. `AWS_TRANSCRIBE_MEDICAL=true` + `AWS_TRANSCRIBE_SPECIALTY` switches to Transcribe Medical. Specialties: `PRIMARYCARE`, `CARDIOLOGY`, `NEUROLOGY`, `ONCOLOGY`, `RADIOLOGY`, `UROLOGY`. | Yes |
| **Local Whisper** | Spawns `whisper.cpp` or `faster-whisper` via `WHISPER_BINARY`. Fully offline. Model sizes `tiny`/`base`/`small`/`medium`/`large`. | N/A (nothing leaves host) |
| **OpenAI Whisper** | `whisper-1` via `/v1/audio/transcriptions`. Medical-context prompt prepended: `"Medical patient encounter. Pediatric."` | No |
| **LiteLLM** | Inline audio via LiteLLM's `chat.completions` endpoint (not the `/audio/transcriptions` path). Model from `LITELLM_STT_MODEL`. | Depends on LiteLLM backend |

## Browser Whisper (fully offline)

Runs entirely in the browser via WebAssembly. Zero network. Suitable when
no external transcription is acceptable.

- Runtime: `@xenova/transformers` (WASM).
- Models (bundled in the Docker image, no CDN fetch):
  - `whisper-tiny.en` — 39 MB
  - `whisper-base.en` — 74 MB
  - `whisper-small.en` — 244 MB
- Executes in a dedicated Web Worker; UI thread is never blocked.
- Models cached in IndexedDB after first load.
- Per-user toggle. On browser transcription failure, the client falls back to
  server-side transcription without user intervention.

## Live speech preview

Chrome / Edge `webkitSpeechRecognition` streams interim text to the UI during
recording. Used for real-time preview only — **not** for final transcription.
The actual transcript comes from the configured STT provider after recording
ends.

## Text-to-speech

### Overview

`POST /api/text-to-speech`. Returns `audio/mpeg`. `X-TTS-Provider` response
header identifies the provider used. 5000-character limit per request. Each
user may override via `users.tts_voice`; admin-wide default via `tts.voice`.

### Providers

| Provider | Notes |
|---|---|
| **Google Cloud TTS** | `@google-cloud/text-to-speech`. Voice families: Journey, Studio, Neural2. |
| **LiteLLM** | Configured via `LITELLM_TTS_MODEL` + `LITELLM_TTS_VOICE`. Backend-agnostic. |
| **ElevenLabs** | `eleven_turbo_v2_5`. **Not HIPAA-compliant**. |

## Audio backup

Raw audio is saved to Postgres **only when transcription fails**, providing a
retry window without persisting every recording.

### Storage

- Gzip-compressed, then AES-256-GCM encrypted (0x01 version byte prefix).
- `BYTEA` column in `audio_backups`.
- 24-hour `expires_at`, swept hourly.
- Legacy rows (gzip magic `0x1F` as first byte, no encryption envelope)
  decompress as-is — detection is deterministic because `0x1F ≠ 0x01`.

### Retry UI

Settings → Audio Backups:
- List: module, size, created, expiry.
- **Retry** — resubmits to `POST /api/transcribe`.
- **Delete** — purge now.

### Browser fallback

If the server-side save fails (network, 500, etc.), the client stores the audio
in IndexedDB so it can retry later. Cleared after successful submission.