danvics/pediatric-ai-scribe-v3
1
0
Fork 0
Code Issues Pull requests Projects Releases 14 Packages Wiki Activity Actions
pediatric-ai-scribe-v3/docs/ai-providers.md
Daniel a36235c646 v6.1: Turnstile bot protection, LiteLLM provider, PPTX tables, audio backup fixes, docs 
- Add Cloudflare Turnstile to login, register, and password reset forms
- Switch AI provider to LiteLLM, transcription to OpenAI Whisper
- Change domain to scribe.pedshub.com
- Fix PPTX export: add tables, bold/italic, numbered lists, code blocks, blockquotes
- Fix announcement banner close button (CSP was blocking inline onclick)
- Fix auth middleware: empty Bearer token now falls through to cookie auth
- Fix audio backups: only save on transcription failure, stop auto-deleting on success
- Soften AI correction injection to prevent model hallucination from correction history
- Fix LiteLLM TTS model name handling (no incorrect openai/ prefix)
- Expand AI instructions textarea in Learning Hub CMS
- Update README for v6 with all features and providers
- Add comprehensive docs/: architecture, API reference, database schema,
  authentication, AI providers, speech, learning hub, configuration, deployment
2026-04-04 22:56:24 +02:00

4.8 KiB
Raw Blame History

AI Providers

This document covers the AI provider system, model management, prompt configuration, and usage logging for the Pediatric AI Scribe application.

Provider Selection

The active AI provider is determined in one of two ways:

  1. Explicit: Set the AI_PROVIDER environment variable to one of the supported provider names.

  2. Auto-detect: If AI_PROVIDER is not set, the system checks for provider-specific credentials in the environment and selects the first match using this priority order:

    bedrock > azure > vertex > litellm > openrouter

All providers expose a unified callAI() interface defined in src/utils/ai.js. Calling code does not need to know which backend is active.

Provider Details

1. AWS Bedrock (HIPAA-eligible with BAA)

  • SDK: @aws-sdk/client-bedrock-runtime
  • Uses inference profiles for newer models, enabling cross-region routing.
  • Available model families: vendor model (Anthropic), Amazon Nova, Llama (Meta), Mistral, DeepSeek, Cohere.
  • Default temperature: 0.3

2. Azure OpenAI (HIPAA-eligible with BAA)

  • SDK: OpenAI SDK configured to point at an Azure endpoint.
  • Requires a deployment name that maps to the desired model.
  • Available model families: GPT-4o family, GPT-4.1 family.

3. Google Vertex AI (HIPAA-eligible with BAA)

  • SDK: @google-cloud/vertexai
  • In addition to text generation, Vertex AI handles:
    • Speech-to-Text (STT): via Gemini inline audio capabilities.
    • Text-to-Speech (TTS): via Vertex AI TTS endpoint.
  • Available model families: Gemini 2.5/2.0, vendor model on Vertex (Anthropic models hosted on Google Cloud), Llama.

4. LiteLLM Proxy (self-hosted)

  • SDK: OpenAI SDK pointed at the LITELLM_API_BASE URL.
  • Acts as a proxy that routes requests to any backend configured within LiteLLM.
  • Model discovery: queries /v1/models on the LiteLLM instance to populate the available model list.
  • Also supports TTS and STT passthrough.
  • Model names are used as-configured in LiteLLM -- the application does not auto-prefix or transform them.

5. OpenRouter (NOT HIPAA-compliant)

  • SDK: OpenAI SDK pointed at https://openrouter.ai.
  • Cost discovery: queries the OpenRouter API to retrieve per-model pricing.
  • Offers the cheapest option and the widest model selection across many providers.
  • Not suitable for environments that require HIPAA compliance.

Model Management

Model Definitions

Models are defined in src/utils/models.js and organized into four categories:

Category Description
free No-cost models (typically smaller or rate-limited)
fast Low-latency models optimized for speed
smart Balanced models with strong reasoning ability
premium Top-tier models with the highest capability

Admin Controls

  • Administrators can enable or disable individual models from the Admin Panel.
  • A default model can be set for new users.
  • Custom models can be added with a user-defined name and cost per token.

Model Discovery

The system queries the active provider's API to retrieve the list of models that are actually available. This list is intersected with the configured model definitions to determine what appears in the UI.

Frontend Display

The model selector dropdown groups models by category (free, fast, smart, premium) for easy navigation.

AI Prompts

Storage and Override

  • All default prompts are defined in src/utils/prompts.js.
  • Prompts can be overridden via the database using the app_settings table with keys following the pattern prompt.{name}.
  • Administrators can view and edit all prompts directly from the Admin Panel.

Physician Memories

When a physician saves corrections or preferences (referred to as "memories"), these are injected into the prompt as low-priority style hints. This allows the AI to adapt its output to the physician's preferred documentation style without overriding the core clinical prompt.

Logging

Every AI call is recorded in the api_log database table with the following fields:

Field Description
model The model identifier used for the request
tokens Input and output token counts
cost Estimated cost of the call
duration Wall-clock time for the request in milliseconds

Cost estimates are calculated from hardcoded per-model rates defined in the codebase. For OpenRouter, rates may also be fetched from the OpenRouter pricing API.