Commit graph

27 commits

Author SHA1 Message Date
Daniel
1d031af5d6 refactor: Google models go through LiteLLM; the Vertex SDK is gone
Some checks failed
Forgejo Android APK / Root app tests (push) Successful in 49s
Forgejo Docker Build / Root app tests (push) Successful in 58s
Forgejo Android APK / Build signed APK (push) Successful in 2m9s
Forgejo Docker Build / Build Docker image (push) Successful in 12s
Forgejo Docker Build / Deploy to the host (push) Failing after 0s
npm audit reports 0 vulnerabilities. It reported 2 high and 2 moderate this
morning.

@google-cloud/vertexai was the last source of findings — gaxios and a uuid with
a missing buffer bounds check, neither reachable in this deployment because
GOOGLE_VERTEX_PROJECT is unset and the require sits inside that check. Dormant
is not the same as gone, and the provider is available through the gateway
anyway, so the direct path has been removed rather than left to rot:

  - the SDK client and callVertex, which without the package could never run
  - the dispatch and discovery branches that reached them
  - VERTEX_MODELS, a list of ids nothing could route any more, and the two
    places in adminConfig that concatenated it into the built-in set
  - the health endpoint's vertex line, and the env vars documented for it

AI_PROVIDER=vertex now says where to configure the model instead of quietly
becoming something else. The Google STT and TTS paths keyed off the same
variable are untouched; neither ever used this SDK.

Verified after deploy: provider litellm, the assistant answers with 8 sources,
/api/models returns 10, and @aws-sdk/s3-request-presigner — which documents.js
needs for presigned MinIO URLs — is still declared and resolvable.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU
2026-09-11 13:51:22 +02:00
Daniel
6c9ac58ce3 chore: log image parts attached to provider requests
All checks were successful
Forgejo Android APK / Root app tests (push) Successful in 22s
Forgejo Android APK / Build signed APK (push) Successful in 2m17s
2026-09-08 16:45:07 +02:00
Daniel
94210ac0a1 feat: attach images to clinical search with strict validation and multimodal routing 2026-09-08 15:53:05 +02:00
Daniel
6be2d1375a feat: integrate durable image jobs/private assets into current core 2026-09-07 16:53:42 +02:00
Daniel
cfaf8e957b feat: ship reviewed prompt history, conversation limits and account protections 2026-09-07 04:01:01 +02:00
Daniel
2c02e6eca7 Send reasoning options only to supported model
All checks were successful
Forgejo Android APK / Build signed APK (push) Successful in 1m53s
2026-08-27 21:25:38 +02:00
Daniel
e710b1c7bd Configure clinical assistant reasoning profile
All checks were successful
Forgejo Android APK / Build signed APK (push) Successful in 1m58s
2026-08-27 20:47:59 +02:00
Daniel
795ad9ffae harden clinical assistant source handling 2026-05-09 19:59:04 +02:00
Daniel
1b3ea569b7 simplify speech and embeddings through litellm 2026-05-09 05:09:02 +02:00
Daniel
d48a19a891 honor admin default model selections 2026-05-08 06:49:50 +02:00
Daniel
10f39fc4ff restore assistant streaming 2026-05-08 00:26:10 +02:00
Daniel
a8f364f177 stabilize assistant and add diagrams 2026-05-08 00:23:00 +02:00
Daniel
569363754f fix assistant stream truncation handling 2026-05-07 23:02:26 +02:00
Daniel
c8436d5e4c refactor clinical assistant prompt handling 2026-05-07 22:52:46 +02:00
Daniel
b9270414b0 feat: stream clinical assistant responses 2026-05-07 17:26:10 +02:00
Daniel
e5f7167b8d fix: auth/API logging to Loki, TTS voice auto-detection, STT ElevenLabs support
- Add logger.audit/access calls to auth route (login, login_failed,
  login_blocked, register, password_changed, 2fa_backup_code_used,
  2fa_backup_codes_regenerated) — these previously only wrote to DB
  via raw SQL, bypassing Loki shipper
- Replace logger.info with logger.apiCall in callAI() so every AI call
  ships to Loki with model, tokens, cost, duration
- Add device identifier (parsed user agent) to audit and access logs
- Fix TTS voice/model provider mismatch: auto-detect Vertex voices
  (Puck, Charon, Kore, etc.) and ElevenLabs voice IDs, override model
  to match provider regardless of what model was previously set
- Fix TTS discovery: model IDs saved to tts.voice are detected and
  redirected to tts.model (regex for openai-tts, elevenlabs, vertex-tts)
- Fix STT transcription route: add scribe/elevenlabs/transcri to the
  isTranscriptionModel regex so ElevenLabs Scribe uses /audio/transcriptions
  endpoint instead of chat completions
- Remove OpenObserve/SigNoz code from logger (reverted to Loki-only)
2026-04-19 21:26:49 +02:00
Daniel
2f6e5a7d8f feat: neonatal calculator, DOCX/PPTX/ODT/EPUB support, gateway-agnostic URL helper, TTS/STT fixes
- Add neonatal assessment calculator: GA classification (extremely preterm through
  post term), weight-for-GA percentile (AGA/SGA/LGA) using Fenton 2013 LMS data,
  birth weight category (ELBW/VLBW/LBW/normal/macrosomia)
- Add DOCX support via mammoth, PPTX/ODT/EPUB via jszip in Learning Hub content
  generator file upload
- Add gatewayUrl() helper for consistent API URL construction — handles
  LITELLM_API_BASE with or without /v1 suffix, works with any OpenAI-compatible
  gateway (LiteLLM, Bifrost, etc.)
- Fix TTS model/voice separation: discovery now tags items as MODEL or VOICE,
  auto-detects provider from voice name (Vertex, ElevenLabs, OpenAI)
- Fix STT discovery to include ElevenLabs Scribe and Chirp models
- Fix TTS discovery to include ElevenLabs and Vertex voices alongside models
- Fix admin model test to bypass allowlist check (skipAllowlistCheck) so
  discovered models can be tested before adding
- Fix Nextcloud token decryption in learningAI.js WebDAV browse and file import
- Fix admin embedding test to show DB model name instead of hardcoded default
- Fix admin STT test to use correct endpoint for Whisper models
- Add AI gateway migration guide to configuration docs
- Add Grafana dashboard JSON for Loki log visualization
2026-04-19 02:17:06 +02:00
Daniel
7a06a4aa63 Batch of security + scale fixes
Age parser (src/routes/billing.js):
  - Now sums year + month + week + day matches so "4 yr 11 mo"
    (59 months) correctly maps to the 5-11y billing bracket instead
    of being billed as 1-4y. Added bounds sanity check.

Graceful SIGTERM shutdown (server.js):
  - Closes the HTTP listener first, then drains batched audit queues,
    then ends the Postgres pool. 9-second hard deadline to beat
    Docker's 10-second SIGKILL. Previously an in-flight note save
    during a container restart could truncate the write.

Explicit LLM fallback opt-in (src/utils/ai.js):
  - The OpenRouter / LiteLLM silent fallback now requires admin
    setting `ai.allow_model_fallback = true` (default: false). If
    primary fails and fallback is disabled, the error is surfaced
    to the caller. Prevents silent spillover from a BAA-covered
    primary to a non-covered fallback.

Prompt injection delimiters (src/utils/promptSafe.js):
  - Wraps user transcripts, dictations, refine-instructions, and
    pasted documents in <UNTRUSTED_*>...</UNTRUSTED_*> tags and
    appends an explicit system instruction telling the model to
    treat the wrapped content as data rather than commands.
  - Applied to soap.js, hpi.js, refine.js. Extend to other AI
    routes incrementally.

Cross-tab logout sync (public/js/authFetch.js, auth.js):
  - BroadcastChannel('pedscribe-auth') — logout in one tab posts
    a message; all sibling tabs clear state and reload, dropping
    any PHI-containing UI immediately.

Backup code race-free consumption (src/routes/auth.js):
  - tryConsumeBackupCode() now uses a Postgres transaction with
    SELECT ... FOR UPDATE so concurrent login attempts using the
    same code serialize. First wins, second sees the already-
    shortened array.

Optimistic encounter locking (migrations/...add-encounter-version):
  - saved_encounters.version INTEGER NOT NULL DEFAULT 1
  - POST /api/encounters/saved accepts an expected_version and
    rejects with 409 if the row has advanced. Falls back to
    last-write-wins if the client doesn't pass one (backward compat).

Audit log batching (src/utils/auditQueue.js):
  - Audit / api_log / access_log writes are buffered in memory and
    flushed every 1s or every 50 entries via one multi-row INSERT.
    Under load this reduces DB pressure by ~50x. On SIGTERM the
    shutdown path drains the queue before exiting.
2026-04-14 05:24:40 +02:00
Daniel
8db25f39be Enforce server-side LLM model whitelist + scope idle timeout to writes
Two findings from review:

1. callAI() previously accepted any model string from the client.
   POST /api/hpi with { model: "openai/o1" } would call the reasoning
   model regardless of whether the operator enabled it. Added
   getAllowedModelIds() in src/utils/models.js (60s TTL DB-backed
   cache) and a guard at the top of callAI() that rejects with
   "model_not_permitted" when the requested ID isn't in the active
   roster. No model supplied → silent fallback to DEFAULT_MODEL.

2. Middleware was updating user_sessions.last_activity on every
   request, including GETs. Client-side polling (/api/auth/me
   heartbeats, dashboard refreshes, log tail calls) kept sessions
   alive indefinitely, defeating the 24h sliding idle policy. Now
   only POST/PUT/DELETE/PATCH count as "user activity". GETs are
   read-only and often automated — they no longer extend the
   session. Idle enforcement still runs on every method, so a
   24h-idle user still gets kicked on their next GET.
2026-04-14 05:15:55 +02:00
ifedan-ed
783679a3f7 feat: Add model search, testing, and TTS/STT/embedding management to admin
- Fix model search for all providers: Bedrock now falls back to built-in
  list (with live ListFoundationModels attempt), Azure returns built-in list
- Add Test button on every model row (built-in, discovered, custom) that
  sends a live prompt and shows response + latency in a toast
- Add TTS management section: search voices from provider API (Google TTS
  voices.list, LiteLLM /v1/models, ElevenLabs /v1/voices), Set as Default
  writes tts.voice/tts.model to DB, runtime respects DB override
- Add STT management section: search models from provider (Gemini, Whisper,
  LiteLLM, OpenAI, local), Set as Default writes stt.model to DB, runtime
  respects DB override in transcribe.js
- Add Embedding models section: search from provider (LiteLLM, Vertex,
  OpenAI), Set as Default writes embeddings.model+dimensions to DB,
  embeddings.js respects DB override
- Add record-and-transcribe STT test (browser MediaRecorder)
- Add TTS synthesize-and-play test (returns base64 audio)
- Add embedding generate test (shows dims + vector sample)
- Expand PUT /config/:key(*) whitelist to include tts., stt., embeddings.
- Add @aws-sdk/client-bedrock as optional dependency for live Bedrock discovery
2026-04-03 19:55:11 +00:00
Daniel Onyejesi
29f1a9b860 v9.1: Add Google Vertex AI + LiteLLM support, admin model management panel
- Add Vertex AI provider (Gemini models via @google-cloud/vertexai SDK)
- Add LiteLLM proxy support (OpenAI-compatible, routes to any provider)
- Admin panel: model search/discover from provider API, enable/disable, custom models, set default
- New endpoints: /config/models/discover, /config/models/add-discovered, /config/models/default
- Updated models.js with VERTEX_MODELS and LITELLM_MODELS lists
- Updated health endpoint with vertex + litellm status
2026-03-29 10:32:45 -04:00
Daniel Onyejesi
986ee76489 Comprehensive Bedrock fix: inference profiles + maxTokens clamping
- ALL models that need inference profiles now use us. prefix IDs:
  Anthropic (all 8), Nova (3), Meta Llama (3), DeepSeek R1, Writer
- Models with low max output tokens (Cohere 4096, Jamba 4096) now
  have maxOut field; callBedrock clamps maxTokens automatically
- Removed Qwen3 235B (no us-east-1 support, no inference profile)
- Added better debug logging for response block structure
2026-03-24 18:26:43 -04:00
Daniel Onyejesi
6932dddbc1 Fix Anthropic models: use cross-region inference profiles (us. prefix)
Opus 4.6, Opus 4.5, Sonnet 4.6, Haiku 4.5 do NOT support direct
on-demand invocation — they require cross-region inference profile IDs
(us.anthropic.* prefix). Updated all affected bedrockIds. Also fixed
isAnthropic check to handle us.anthropic.* prefix.

Sonnet 4.5, Sonnet 4, 3.5 Haiku also updated to use profiles for
consistency (profiles work everywhere, direct IDs are limited).
2026-03-24 18:17:36 -04:00
Daniel Onyejesi
c2646f0384 Fix Opus 4.6 thinking blocks, re-add Amazon Nova models
- Opus 4.6 returns multiple content blocks (thinking + text); was only
  reading content[0] which was the thinking block (just '{')
- Now iterates all blocks and extracts only type:'text' blocks
- Added debug logging for Bedrock response structure
- Re-added Nova Pro, Nova Lite, Nova Micro with correct region metadata
2026-03-24 18:05:27 -04:00
Daniel
d2ed08cd46 v3.0: Add multi-provider Bedrock models, newborn milestones, Converse API
- Bedrock: Add 14 non-vendor model models (Amazon Nova, Meta Llama 4, DeepSeek R1/V3,
  Mistral Large 3, Cohere Command R+, AI21 Jamba) with verified AWS model IDs
- Bedrock: Implement Converse API for non-Anthropic models (unified cross-model API),
  keep native Messages API for vendor model models (best performance)
- Milestones: Add Newborn / 1 month developmental milestones (Gross Motor, Fine Motor,
  Language, Social/Emotional, Cognitive) — previously started at 2 months
- Bump version to 3.0.0, docker-compose tag to v3.0
2026-03-23 22:13:19 +01:00
ifedan-ed
ac8e7bb890 v3.0.0: Auth, admin panel, security fixes, per-tab model selector
- Add authMiddleware to all AI/transcribe routes (were unauthenticated)
- Add full admin panel: user management, registration toggle, stats
- Fix XSS in email verification (escape user.name in HTML)
- Fix missing APP_URL fallback in password reset email
- Add per-tab model selector (respects OpenRouter/Bedrock/Azure lists)
- Fix transcribeAudio to send Authorization header
- Fix labs input: textarea instead of single-line input
- Add structured logging: audit_log, api_log, access_log tables
- Add admin CLI (admin-cli.js) for Docker exec management
- Fix duplicate var duration declaration in ai.js catch block
- Fix RETURNING check case-sensitivity in database.js
2026-03-21 19:25:51 -04:00
ifedan-ed
565bec9ca8 v2.0.0: Pediatric AI Scribe
Features:
- Live encounter recording → HPI (outpatient/inpatient)
- Voice dictation → HPI / SOAP note
- Hospital course generator (prose/day-by-day/organ system/psych)
- Chart review / precharting (outpatient/subspecialty/ED)
- SOAP note generator (full/subjective only)
- Developmental milestones (AAP/Nelson) with narrative + 3-sentence summary
- AI refine & shorten for all outputs
- Ask AI what's missing (clarification)
- Authentication (email/password, email verification, 2FA)
- Nextcloud integration with auto date folders
- PWA support (installable on phone)
- 18+ AI models via OpenRouter
- HIPAA compliance guidance
- Docker support
2026-03-21 16:55:50 -04:00