From aeb31a2d154fa071c7fd99500037609268663c26 Mon Sep 17 00:00:00 2001 From: Daniel Date: Fri, 8 May 2026 19:15:54 +0200 Subject: [PATCH] scrub vendor model docs --- docs/ai-providers.md | 4 ++-- docs/configuration.md | 4 ++-- docs/developer-guide-extended.md | 4 ++-- docs/logic/ai-and-voice.md | 27 +++++++++------------------ 4 files changed, 15 insertions(+), 24 deletions(-) diff --git a/docs/ai-providers.md b/docs/ai-providers.md index e169394..c817371 100644 --- a/docs/ai-providers.md +++ b/docs/ai-providers.md @@ -15,7 +15,7 @@ Provider is selected once at startup and is transparent to callers. - SDK: `@aws-sdk/client-bedrock-runtime`. - Uses Bedrock **inference profiles** for newer models (cross-region routing). -- Model families: vendor model (Anthropic), Amazon Nova, Llama (Meta), Mistral, DeepSeek, Cohere. +- Model families: Amazon Nova, Llama (Meta), Mistral, DeepSeek, Cohere, and other Bedrock-hosted families. ### Azure OpenAI (BAA-eligible) @@ -27,7 +27,7 @@ Provider is selected once at startup and is transparent to callers. - SDK: `@google-cloud/vertexai`. - Also serves STT (Gemini inline audio) and TTS (Vertex TTS endpoint). -- Families: Gemini 2.5 / 2.0, vendor model on Vertex (Anthropic via GCP), Llama. +- Families: Gemini 2.5 / 2.0 and Llama. ### LiteLLM proxy (self-hosted) diff --git a/docs/configuration.md b/docs/configuration.md index 9c0d8fb..fec22d2 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -178,8 +178,8 @@ OpenAI-compatible gateway — LiteLLM, Bifrost, or other proxies. 3. **Update model names** — Different gateways use different naming conventions. Bifrost requires `provider/model` format - (e.g., `openrouter/vendor-model-sonnet-4.6`), while LiteLLM uses aliases - (e.g., `openrouter-vendor-model-sonnet-4.6`). Update model names in: + (e.g., `openrouter/gpt-4.1`), while LiteLLM can use deployment aliases + (e.g., `openrouter-gpt-4.1`). Update model names in: - Admin Panel → Models (chat models) - Admin Panel → Settings → `stt.model` (speech-to-text) - Admin Panel → Settings → `tts.model` (text-to-speech) diff --git a/docs/developer-guide-extended.md b/docs/developer-guide-extended.md index 328b80c..94694b3 100644 --- a/docs/developer-guide-extended.md +++ b/docs/developer-guide-extended.md @@ -542,11 +542,11 @@ const result = await callAI( ### Bedrock Model Notes -**Inference Profiles:** Most newer models (Anthropic vendor model 4.x, Meta Llama 4, DeepSeek R1, Amazon Nova, Writer) require cross-region inference profiles. These use a `us.` prefix on the model ID (e.g. `us.anthropic.agent-config-sonnet-4-6`). Direct model IDs will return "on-demand throughput not supported" errors. +**Inference Profiles:** Most newer Bedrock models require cross-region inference profiles. These use a `us.` prefix on the model ID (for example, `us.amazon.nova-pro-v1:0`). Direct model IDs may return "on-demand throughput not supported" errors. **Max Output Tokens:** Some models have low output limits (Cohere Command R/R+: 4096, AI21 Jamba: 4096). The `maxOut` field in `models.js` auto-clamps `maxTokens` in `callBedrock()`. -**JSON Sanitization:** Some models (notably vendor model Sonnet 4.6, Opus 4.6) output literal newline characters inside JSON string values. `learningAI.js` includes a `sanitizeJsonString()` function that escapes these before parsing. +**JSON Sanitization:** Some models output literal newline characters inside JSON string values. `learningAI.js` includes a `sanitizeJsonString()` function that escapes these before parsing. ### Prompt System Prompts are defined in `src/utils/prompts.js`. Admins can override any prompt via the Admin panel (`/admin/config/prompts`). Overrides are stored in `app_settings` table and loaded into memory on startup (with 3s grace period for DB readiness). diff --git a/docs/logic/ai-and-voice.md b/docs/logic/ai-and-voice.md index d0c21e6..9bd0883 100644 --- a/docs/logic/ai-and-voice.md +++ b/docs/logic/ai-and-voice.md @@ -175,7 +175,7 @@ Lines 478–514. On primary-provider error: ("Pediatric AI Scribe") — this is what shows up in the OpenRouter dashboard's app attribution. - Model IDs are passed straight through (`google/gemini-2.5-flash`, - `anthropic/vendor-model-sonnet-4-6`, etc.). + `openai/gpt-4.1`, etc.). - **Not BAA-eligible.** The system enforces nothing about that — the operator must just not select it for PHI. @@ -183,27 +183,18 @@ Lines 478–514. On primary-provider error: - Auth: standard AWS SDK credentials (env or instance profile). Region comes from `AWS_BEDROCK_REGION`. -- Two API paths: - - **Anthropic models** (line 223 — detected by `modelId.indexOf('anthropic.')`) - use the native `InvokeModel` Messages API (lines 225–273). System - message is hoisted out of the messages array into the top-level - `system` field — Anthropic's Messages API takes it separately, not as - a `role: system` entry. - - **All other models** (Nova, Llama, Mistral, DeepSeek, Cohere, etc.) use - the unified Bedrock **Converse API** (lines 276–313). System message - becomes `system: [{ text: ... }]`; chat content is wrapped as - `content: [{ text: ... }]`. +- Bedrock models use either native provider payloads or the unified Bedrock + **Converse API**. For Converse requests, the system message becomes + `system: [{ text: ... }]`; chat content is wrapped as `content: [{ text: ... }]`. - **Inference profile mapping.** `getBedrockModelId(model)` in - `models.js` translates the friendly id (`anthropic/vendor-model-sonnet-4-6`) - into the AWS-required cross-region inference profile id - (`us.anthropic.agent-config-sonnet-4-6`). Most newer models *require* the - inference profile; using the raw foundation model id will fail. + `models.js` translates the friendly id into the AWS-required cross-region + inference profile id. Most newer models require the inference profile; + using the raw foundation model id can fail. - **Output token clamping** (lines 209–210). `getBedrockMaxOut` returns the per-model output limit (e.g. Cohere is 4096) so we don't ask for 4000 output and get a 400 from the SDK. -- **Multi-block content extraction** (lines 245–263). vendor model Opus 4.6+ may - return multiple content blocks (e.g. a thinking block + a text block). - We iterate, keep only `type === 'text'`, and concatenate. +- **Multi-block content extraction.** Some provider-native responses may return + multiple content blocks. We iterate, keep only `type === 'text'`, and concatenate. - Usage is normalised: AWS's `input_tokens` / `output_tokens` → `prompt_tokens` / `completion_tokens` to match the OpenAI shape.