# AI, Speech, And Post-Note Helpers

This doc summarizes the current AI/STT/TTS pipeline without line-number
citations. For exact behavior, read `src/utils/ai.js`, `src/routes/transcribe.js`,
`src/routes/tts.js`, and the relevant frontend scripts.

## Text Generation

All text-generation routes call `callAI(messages, options)` from
`src/utils/ai.js`.

Supported providers:

- OpenRouter.
- AWS Bedrock.
- Azure OpenAI.
- Google Vertex AI.
- LiteLLM or another OpenAI-compatible gateway.

`AI_PROVIDER` can explicitly choose the provider. If unset, the startup loader
initializes configured clients and the final active provider follows the current
load order described in [`../ai-providers.md`](../ai-providers.md). Route
handlers do not call provider SDKs directly.

## Model Allowlist

`callAI()` rejects model IDs outside the active server-side allowlist unless a
specific admin test path opts out. The allowlist is assembled from built-in
provider models, `models.disabled`, and `models.custom` in `app_settings`.

The default model comes from the configured provider/model settings. Admins can
set defaults and custom models from the Admin Panel.

## Prompt Safety

Clinical routes should build prompts with:

- canonical templates from `src/utils/prompts.js`
- optional DB prompt overrides through `app_settings` keys `prompt.*`
- `INJECTION_GUARD`
- `wrapUserText(label, text)` around user-derived text

User-derived text includes transcripts, dictated notes, pasted chart data,
refine instructions, template preferences, and patient education source notes.

## User Templates

`getUserMemoryContext()` fetches `/api/memories/context` and passes the returned
template/preference context as `physicianMemories`. Server routes wrap that block
as low-priority style/template context. `custom` memories and legacy
`correction_*` rows are not prompt context.

## Speech-To-Text

`POST /api/transcribe` accepts one audio file up to 25 MB. Provider selection:

- explicit `TRANSCRIBE_PROVIDER=litellm`, or
- auto mode when `LITELLM_API_BASE` is configured.

Direct Google, AWS, local Whisper, and OpenAI Whisper branches are not part of the runtime. Browser Whisper/browser-local model downloads are also absent.

## Browser Web Speech

Browser-native Web Speech is an explicit opt-in preview. It may rely on browser
vendor cloud services and must not be treated as the final clinical transcript.
Final transcription should come from the configured server-side STT provider.

## Audio Backup

Failed transcription attempts can create encrypted 24-hour audio backups through
`src/routes/audioBackups.js`. The user can retry or delete backups from
Settings. Browser fallback storage is only for cases where the server cannot
store the failed recording.

## Text-To-Speech

`POST /api/text-to-speech` returns audio from LiteLLM and marks the LiteLLM
model in `X-TTS-Provider`. Voices are LiteLLM-compatible strings configured by
`LITELLM_TTS_VOICES`.

## Post-Note Helpers

Generated note outputs can expose helper panels:

- `refineDocument` for editing/refining/shortening generated text.
- `suggestBillingCodes` for clinician-facing ICD/CPT suggestions.
- `suggestDontMiss` for clinician-facing safety review.
- `attachPatientEducation` for parent-facing handout drafts.

These helpers are authenticated API-backed actions. They should treat the edited
note as the source of truth and keep the clinician in the review loop.

## Change Checklist

When changing this area:

1. Keep provider-specific code inside utility/provider modules.
2. Wrap all user-derived text with `wrapUserText` before AI calls.
3. Do not add browser-local Whisper back without a new design review.
4. Do not add clinical answer response caching.
5. Run touched-file `node --check` commands and `npm test`.