# AI Providers

This document covers the AI provider system, model management, prompt configuration, and usage logging for the Pediatric AI Scribe application.

---

## Provider Selection

The active AI provider is determined in one of two ways:

1. **Explicit**: Set the `AI_PROVIDER` environment variable to one of the supported provider names.
2. **Auto-detect**: If `AI_PROVIDER` is not set, the system checks for provider-specific credentials in the environment and selects the first match using this priority order:

   `bedrock` > `azure` > `vertex` > `litellm` > `openrouter`

All providers expose a unified `callAI()` interface defined in `src/utils/ai.js`. Calling code does not need to know which backend is active.

---

## Provider Details

### 1. AWS Bedrock (HIPAA-eligible with BAA)

- **SDK**: `@aws-sdk/client-bedrock-runtime`
- Uses **inference profiles** for newer models, enabling cross-region routing.
- **Available model families**: vendor model (Anthropic), Amazon Nova, Llama (Meta), Mistral, DeepSeek, Cohere.
- **Default temperature**: 0.3

### 2. Azure OpenAI (HIPAA-eligible with BAA)

- **SDK**: OpenAI SDK configured to point at an Azure endpoint.
- Requires a **deployment name** that maps to the desired model.
- **Available model families**: GPT-4o family, GPT-4.1 family.

### 3. Google Vertex AI (HIPAA-eligible with BAA)

- **SDK**: `@google-cloud/vertexai`
- In addition to text generation, Vertex AI handles:
  - **Speech-to-Text (STT)**: via Gemini inline audio capabilities.
  - **Text-to-Speech (TTS)**: via Vertex AI TTS endpoint.
- **Available model families**: Gemini 2.5/2.0, vendor model on Vertex (Anthropic models hosted on Google Cloud), Llama.

### 4. LiteLLM Proxy (self-hosted)

- **SDK**: OpenAI SDK pointed at the `LITELLM_API_BASE` URL.
- Acts as a proxy that routes requests to any backend configured within LiteLLM.
- **Model discovery**: queries `/v1/models` on the LiteLLM instance to populate the available model list.
- Also supports **TTS and STT** passthrough.
- Model names are used **as-configured in LiteLLM** -- the application does not auto-prefix or transform them.

### 5. OpenRouter (NOT HIPAA-compliant)

- **SDK**: OpenAI SDK pointed at `https://openrouter.ai`.
- **Cost discovery**: queries the OpenRouter API to retrieve per-model pricing.
- Offers the **cheapest option** and the **widest model selection** across many providers.
- Not suitable for environments that require HIPAA compliance.

---

## Model Management

### Model Definitions

Models are defined in `src/utils/models.js` and organized into four categories:

| Category  | Description                                    |
|-----------|------------------------------------------------|
| `free`    | No-cost models (typically smaller or rate-limited) |
| `fast`    | Low-latency models optimized for speed         |
| `smart`   | Balanced models with strong reasoning ability  |
| `premium` | Top-tier models with the highest capability    |

### Admin Controls

Administrators manage models from **Admin Panel > Models**:

- **Enable/Disable** -- toggle visibility of any model for users (`PUT /api/admin/config/models/toggle`). Disabled models are stored in the `models.disabled` setting as a JSON array of model IDs.
- **Set Default** -- choose which model is pre-selected for new users (`PUT /api/admin/config/models/default`). Stored in `models.default` setting.
- **Add Custom Models** -- manually add any model not in the built-in list (`POST /api/admin/config/models/custom`). Each custom model has:
  - `id` -- the model identifier as the provider expects it (e.g., `openai-gpt-4.1-mini` for LiteLLM, `anthropic.agent-config-3-haiku` for Bedrock)
  - `name` -- display name shown to users
  - `cost` -- cost string shown in the UI (e.g., `~$0.002`, `FREE`)
  - `category` -- one of `free`, `fast`, `smart`, `premium` (determines grouping in dropdown)
- **Delete Custom Models** -- remove a manually added model (`DELETE /api/admin/config/models/custom/:modelId`)
- **Clear All** -- remove all custom models and reset the disabled list (`POST /api/admin/config/models/clear`)

Custom models are stored in the `models.custom` setting as a JSON array in the `app_settings` table.

### Model Discovery

The **Discover** button (`GET /api/admin/config/models/discover`) queries the active provider's API:

- **LiteLLM**: calls `/v1/models` on the LiteLLM proxy
- **OpenRouter**: calls `https://openrouter.ai/api/v1/models` (includes pricing data)
- **Bedrock**: uses `ListFoundationModelsCommand`

Discovered models can be added individually via `POST /api/admin/config/models/add-discovered`, which merges them into the custom models list.

For **LiteLLM** specifically, since models are manually configured in the LiteLLM proxy, the model IDs returned by discovery are the exact names to use -- no provider prefix is added.

### Frontend Display

The model selector dropdown (present on every tab) groups models by category:

- Free -- no-cost models
- Fast & Cheap -- low-latency, low-cost
- Smart -- balanced capability
- Premium -- highest quality

Each model shows its display name and cost string. The dropdown is populated from `GET /api/models` which merges built-in models, custom models, and respects the enabled/disabled list.

---

## AI Prompts

### Storage and Override

- All default prompts are defined in `src/utils/prompts.js`.
- Prompts can be **overridden via the database** using the `app_settings` table with keys following the pattern `prompt.{name}`.
- Administrators can view and edit all prompts directly from the Admin Panel.

### Physician Memories

When a physician saves corrections or preferences (referred to as "memories"), these are injected into the prompt as **low-priority style hints**. This allows the AI to adapt its output to the physician's preferred documentation style without overriding the core clinical prompt.

---

## Logging

Every AI call is recorded in the `api_log` database table with the following fields:

| Field      | Description                                         |
|------------|-----------------------------------------------------|
| `model`    | The model identifier used for the request           |
| `tokens`   | Input and output token counts                       |
| `cost`     | Estimated cost of the call                          |
| `duration` | Wall-clock time for the request in milliseconds     |

Cost estimates are calculated from **hardcoded per-model rates** defined in the codebase. For OpenRouter, rates may also be fetched from the OpenRouter pricing API.