harden logging and observability
This commit is contained in:
Daniel
parent
467593a109
commit
a08524d95f
24 changed files with 332 additions and 1698 deletions
420
README.md
420
README.md
|
|
@ -1,78 +1,99 @@
|
|||
# Pediatric AI Scribe v6
|
||||
# Ped-AI
|
||||
|
||||
AI-powered clinical documentation platform for pediatric medicine. Generates HPIs, hospital courses, chart reviews, SOAP notes, well/sick visit notes, and developmental milestone assessments from voice recordings or dictation.
|
||||
Ped-AI is a pediatric clinical documentation, education, and bedside decision-support app. This fork has moved well beyond the original scribe app: it now combines encounter documentation, clinical workflows, Learning Hub CMS, admin controls, MCP-backed clinical assistant integration, Redis-backed operational state, and hardened deployment defaults.
|
||||
|
||||
## Features
|
||||
The app runs as an authenticated Express/Postgres service with a browser frontend and optional integrations for LiteLLM, Vertex/Gemini, AWS, OpenAI-compatible APIs, Nextcloud WebDAV, S3-compatible storage, OpenBao, Redis, OIDC, TOTP, and Cloudflare Turnstile.
|
||||
|
||||
## Current Scope
|
||||
|
||||
### Clinical Documentation
|
||||
- **Live Encounter** — record doctor-patient conversations, AI generates structured OLDCARTS HPI
|
||||
- **Voice Dictation** — dictate narrative, AI cleans and restructures
|
||||
- **Hospital Course** — paste progress notes, generates prose, day-by-day, organ-system (ICU), or psych format
|
||||
- **Chart Review / Precharting** — summarize outpatient, subspecialty, and ED notes
|
||||
- **SOAP Notes** — full SOAP or subjective-only from dictation
|
||||
- **Well Visit** — AAP 2025 Bright Futures periodicity with vaccines, screenings, billing codes, SSHADESS (12+), milestones
|
||||
- **Sick Visit** — quick documentation with auto-suggested ROS and PE from chief complaint
|
||||
- **Developmental Milestones** — AAP/Nelson tracker (birth-11y) with narrative/structured/summary output
|
||||
|
||||
### AI & Speech
|
||||
- **5 AI Providers** — OpenRouter, AWS Bedrock, Azure OpenAI, Google Vertex AI, LiteLLM
|
||||
- **5 STT Providers** — Google Gemini, Amazon Transcribe (Medical), OpenAI Whisper, Local Whisper, LiteLLM
|
||||
- **3 TTS Providers** — Google Cloud TTS, LiteLLM (OpenAI), ElevenLabs
|
||||
- **Browser Whisper** — fully offline in-browser transcription via WebAssembly (HIPAA-safe)
|
||||
- **Per-tab model selector** — choose fast vs. smart vs. premium models per task
|
||||
- **Physician memory system** — Dragon-like learning from your corrections
|
||||
- Live encounter capture with structured pediatric HPI generation.
|
||||
- Dictation cleanup for narrative notes.
|
||||
- SOAP, sick visit, well visit, hospital course, chart review, precharting, and ED encounter workflows.
|
||||
- Pediatric developmental milestone tooling.
|
||||
- Templates, physician memory, and per-tab model overrides.
|
||||
- Server-side speech-to-text routing through configured providers.
|
||||
|
||||
### Bedside Tools
|
||||
|
||||
- Pediatric calculators and emergency dosing helpers.
|
||||
- PE guide and clinical reference content.
|
||||
- Vaccines, catch-up schedules, growth/vitals, bilirubin, BSA, GCS, equipment, and resuscitation helpers.
|
||||
- Mobile-friendly PWA layout for bedside use.
|
||||
|
||||
### Learning Hub
|
||||
- **Content Management** — articles, clinical pearls, quizzes, presentations
|
||||
- **AI Content Generation** — generate from topics, uploaded PDFs, or Nextcloud files
|
||||
- **Marp Presentations** — slide editor with preview and PPTX export
|
||||
- **Semantic Search** — vector-based search via pgvector embeddings
|
||||
- **Quiz System** — MCQ, multi-select, true/false with scoring and progress tracking
|
||||
|
||||
### Platform
|
||||
- **Multi-user with roles** — admin, moderator, user
|
||||
- **OIDC/SSO** — Azure AD, Okta, Keycloak, PocketID, Google
|
||||
- **2FA** — TOTP-based two-factor authentication
|
||||
- **Cloudflare Turnstile** — bot protection on login, register, password reset
|
||||
- **Email verification** — with customizable templates
|
||||
- **Nextcloud integration** — WebDAV export
|
||||
- **S3 Document Storage** — AWS S3, Backblaze B2, MinIO
|
||||
- **PWA** — installable, works on mobile
|
||||
- **Admin Panel** — user management, settings, prompt editor, model configuration, logs
|
||||
- CMS for articles, clinical pearls, quizzes, and presentations.
|
||||
- Tiptap article editor, quiz builder, category management, and draft/publish flow.
|
||||
- AI-assisted content generation from topic text, uploaded files, or connected Nextcloud WebDAV files.
|
||||
- Marp slide editing with preview and PPTX export.
|
||||
- Keyword, semantic, and hybrid search using Postgres/pgvector where configured.
|
||||
|
||||
---
|
||||
### Clinical Assistant
|
||||
|
||||
- Optional MCP-backed clinical assistant integration.
|
||||
- Prompt suggestions backed by Redis operational cache.
|
||||
- No clinical answer response caching.
|
||||
- Designed to retrieve from indexed clinical material while keeping provider selection explicit.
|
||||
|
||||
### Admin And Security
|
||||
|
||||
- Local auth, role-based access, TOTP 2FA, OIDC/SSO, email verification, and optional Turnstile.
|
||||
- Admin panel for users, settings, prompts, models, logs, and Learning Hub content.
|
||||
- Audit, API, access, and client-error logs with redaction hardening.
|
||||
- OpenBao secret loading support at container startup.
|
||||
- S3-compatible document storage support.
|
||||
|
||||
## Removed Browser STT
|
||||
|
||||
Browser Whisper has been removed from the runtime. The app should not ship browser Whisper workers, browser-local Whisper model downloads, Transformers.js browser STT, or Browser Whisper setup docs.
|
||||
|
||||
Speech-to-text is handled server-side through configured providers such as Google/Gemini, AWS Transcribe, LiteLLM, or OpenAI Whisper. Browser-native Web Speech remains gated behind an explicit user setting when present in the browser.
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Configure
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
docker compose up -d --build
|
||||
```
|
||||
|
||||
Edit `.env` — at minimum set:
|
||||
The default compose exposes the app on `127.0.0.1:3552` and starts:
|
||||
|
||||
```env
|
||||
AI_PROVIDER=litellm # or openrouter, bedrock, azure, vertex
|
||||
LITELLM_API_BASE=https://your-litellm.example.com
|
||||
LITELLM_API_KEY=sk-...
|
||||
- `pediatric-ai-scribe` for the Node app.
|
||||
- `pedscribe-db` for Postgres with pgvector.
|
||||
- `ped-ai-redis` for operational Redis state.
|
||||
|
||||
OPENAI_API_KEY=sk-... # for Whisper transcription (if not using LiteLLM STT)
|
||||
|
||||
JWT_SECRET=<64-char random> # openssl rand -hex 32
|
||||
DB_PASSWORD=<strong password>
|
||||
APP_URL=https://your-domain.com
|
||||
```
|
||||
|
||||
### 2. Start
|
||||
Health check:
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
curl -fsS http://127.0.0.1:3552/api/health
|
||||
```
|
||||
|
||||
App runs on **port 3552**. First user to register becomes admin.
|
||||
The first registered user becomes an admin unless registration has already been configured differently.
|
||||
|
||||
### 3. Admin CLI
|
||||
## Core Environment
|
||||
|
||||
Set real values in `.env` before production use.
|
||||
|
||||
```env
|
||||
APP_URL=https://your-domain.example
|
||||
JWT_SECRET=<64-char-random-secret>
|
||||
DB_PASSWORD=<strong-database-password>
|
||||
|
||||
AI_PROVIDER=litellm
|
||||
LITELLM_API_BASE=https://your-litellm.example/v1
|
||||
LITELLM_API_KEY=<key>
|
||||
|
||||
TRANSCRIBE_PROVIDER=litellm
|
||||
LITELLM_STT_MODEL=whisper-1
|
||||
|
||||
REDIS_URL=redis://ped-ai-redis:6379
|
||||
```
|
||||
|
||||
Supported text AI providers include LiteLLM, OpenRouter, AWS Bedrock, Azure OpenAI, and Google Vertex AI. Supported STT routing includes Google/Gemini, AWS Transcribe, OpenAI Whisper, and LiteLLM. Supported TTS routing includes Google Cloud TTS, LiteLLM/OpenAI-compatible audio, and ElevenLabs where configured.
|
||||
|
||||
## Admin CLI
|
||||
|
||||
```bash
|
||||
docker exec pediatric-ai-scribe node admin-cli.js list-users
|
||||
|
|
@ -83,293 +104,64 @@ docker exec pediatric-ai-scribe node admin-cli.js toggle-registration
|
|||
docker exec pediatric-ai-scribe node admin-cli.js stats
|
||||
```
|
||||
|
||||
---
|
||||
## Maintenance
|
||||
|
||||
## AI Provider Configuration
|
||||
|
||||
Switch providers by setting `AI_PROVIDER` in `.env`. No code changes needed.
|
||||
|
||||
| Provider | HIPAA | Config |
|
||||
|----------|-------|--------|
|
||||
| **LiteLLM** | Depends on backend | `LITELLM_API_BASE`, `LITELLM_API_KEY` |
|
||||
| **AWS Bedrock** | Yes (with BAA) | `AWS_BEDROCK_REGION`, `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` |
|
||||
| **Azure OpenAI** | Yes (with BAA) | `AZURE_OPENAI_ENDPOINT`, `AZURE_OPENAI_API_KEY`, `AZURE_DEPLOYMENT_NAME` |
|
||||
| **Google Vertex AI** | Yes (with BAA) | `GOOGLE_VERTEX_PROJECT`, `GOOGLE_VERTEX_LOCATION` |
|
||||
| **OpenRouter** | No | `OPENROUTER_API_KEY` |
|
||||
|
||||
---
|
||||
|
||||
## Transcription (Speech-to-Text)
|
||||
|
||||
Set `TRANSCRIBE_PROVIDER` or let the app auto-detect.
|
||||
|
||||
| Provider | HIPAA | Config |
|
||||
|----------|-------|--------|
|
||||
| **Google Gemini** | Yes | `GOOGLE_VERTEX_PROJECT`, `GOOGLE_STT_MODEL` |
|
||||
| **Amazon Transcribe** | Yes | AWS creds + `TRANSCRIBE_PROVIDER=aws` |
|
||||
| **Amazon Transcribe Medical** | Yes | `AWS_TRANSCRIBE_MEDICAL=true`, `AWS_TRANSCRIBE_SPECIALTY=PRIMARYCARE` |
|
||||
| **Local Whisper** | Yes (offline) | `TRANSCRIBE_PROVIDER=local`, `WHISPER_BINARY`, `WHISPER_MODEL_SIZE` |
|
||||
| **OpenAI Whisper** | No | `OPENAI_API_KEY` |
|
||||
| **LiteLLM** | Depends | `TRANSCRIBE_PROVIDER=litellm`, `LITELLM_STT_MODEL` |
|
||||
| **Browser Whisper** | Yes (client-side) | No config needed — toggle in user settings |
|
||||
|
||||
---
|
||||
|
||||
## Text-to-Speech
|
||||
|
||||
| Provider | HIPAA | Config |
|
||||
|----------|-------|--------|
|
||||
| **Google Cloud TTS** | Yes | `GOOGLE_VERTEX_PROJECT`, `GOOGLE_TTS_VOICE` |
|
||||
| **LiteLLM** | Depends | `LITELLM_TTS_MODEL`, `LITELLM_TTS_VOICE` |
|
||||
| **ElevenLabs** | No | `ELEVENLABS_API_KEY` |
|
||||
|
||||
---
|
||||
|
||||
## OpenID Connect / SSO
|
||||
|
||||
Supports Azure AD, Okta, Keycloak, PocketID, Google, and any OIDC-compliant provider.
|
||||
|
||||
1. Register callback URL: `https://your-domain.com/api/auth/oidc/callback`
|
||||
2. Admin Panel > Settings > Configure OIDC (Issuer URL, Client ID, Client Secret)
|
||||
3. Users are auto-created and linked by email on first SSO login
|
||||
|
||||
See [docs/openid-setup.md](docs/openid-setup.md) for provider-specific guides.
|
||||
|
||||
---
|
||||
|
||||
## Cloudflare Turnstile (Bot Protection)
|
||||
|
||||
Optional CAPTCHA on login, registration, and password reset forms.
|
||||
|
||||
```env
|
||||
TURNSTILE_SITE_KEY=0x4AAA...
|
||||
TURNSTILE_SECRET_KEY=0x4AAA...
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Email
|
||||
|
||||
Without SMTP, email verification is skipped and users are auto-verified.
|
||||
|
||||
```env
|
||||
SMTP_HOST=smtp.gmail.com
|
||||
SMTP_PORT=587
|
||||
SMTP_USER=your-email@gmail.com
|
||||
SMTP_PASS=your-app-password
|
||||
SMTP_FROM=noreply@yourdomain.com
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Maintenance CLI
|
||||
|
||||
After a Postgres image upgrade (major version bump or silent base-layer change),
|
||||
btree indexes on text columns can become inconsistent with the new ICU/glibc
|
||||
library. The app auto-detects this at startup and reindexes on drift, but you
|
||||
can also trigger it manually:
|
||||
The app checks Postgres collation drift on startup and can reindex text indexes after image or OS-library changes.
|
||||
|
||||
```bash
|
||||
# Health check — no writes
|
||||
docker exec pediatric-ai-scribe npm run maint:check
|
||||
|
||||
# Rebuild all indexes + refresh collation + ANALYZE
|
||||
docker exec pediatric-ai-scribe npm run maint:reindex
|
||||
```
|
||||
|
||||
Run `maint:reindex` any time after:
|
||||
|
||||
- Upgrading the Postgres image (major or minor)
|
||||
- Restoring from a dump created on a different Linux distro
|
||||
- Seeing "invalid credentials" on credentials you know are correct
|
||||
- Seeing `0 rows` returned from a lookup that should match
|
||||
|
||||
The reindex takes seconds on a small DB and a minute or two on larger ones.
|
||||
Safe to run while the app is serving traffic, though queries may slow briefly.
|
||||
|
||||
---
|
||||
|
||||
## Docker Hub
|
||||
|
||||
```bash
|
||||
docker pull danielonyejesi/pediatric-ai-scribe-v3:latest
|
||||
```
|
||||
|
||||
Minimal compose without building:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
app:
|
||||
image: danielonyejesi/pediatric-ai-scribe-v3:latest
|
||||
ports:
|
||||
- "3552:3000"
|
||||
env_file: .env
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
restart: unless-stopped
|
||||
|
||||
postgres:
|
||||
image: pgvector/pgvector:pg16
|
||||
environment:
|
||||
POSTGRES_DB: pedscribe
|
||||
POSTGRES_USER: pedscribe
|
||||
POSTGRES_PASSWORD: ${DB_PASSWORD}
|
||||
volumes:
|
||||
- pgdata:/var/lib/postgresql/data
|
||||
restart: unless-stopped
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U pedscribe"]
|
||||
interval: 10s
|
||||
retries: 5
|
||||
|
||||
volumes:
|
||||
pgdata:
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## HIPAA Notice
|
||||
|
||||
This application processes data through third-party AI APIs.
|
||||
|
||||
- All connections use HTTPS/TLS
|
||||
- Authentication required for all AI endpoints
|
||||
- 2FA and SSO available
|
||||
- Cloudflare Turnstile bot protection
|
||||
- **AWS Bedrock**, **Azure OpenAI**, and **Google Vertex AI** offer BAAs
|
||||
- **OpenRouter** and **ElevenLabs** do NOT offer BAAs
|
||||
- **Browser Whisper** and **Local Whisper** keep audio fully private
|
||||
|
||||
**Do not use real PHI without executed BAAs with all providers in your deployment.**
|
||||
|
||||
---
|
||||
|
||||
## Documentation
|
||||
|
||||
See [docs/](docs/) for the full documentation set.
|
||||
|
||||
### Application logic — start here if you're new to the codebase
|
||||
|
||||
[**docs/logic/**](docs/logic/) is a deep, dev-friendly walkthrough of how each
|
||||
part of the app actually works. ~8,300 lines of "how it works and why" — read
|
||||
the index first to know what's there:
|
||||
|
||||
- [docs/logic/README.md](docs/logic/README.md) — index + recommended reading order
|
||||
- [docs/logic/architecture.md](docs/logic/architecture.md) — frontend IIFE pattern, lazy tab loading, backend route convention, schema, encryption, sacred zones
|
||||
- [docs/logic/clinical-notes.md](docs/logic/clinical-notes.md) — every note tab (HPI, dictation, sick, well, SOAP, hospital, chart, notes) with the shared record→generate→save lifecycle
|
||||
- [docs/logic/ed-encounters.md](docs/logic/ed-encounters.md) — multi-stage ED notes, per-stage don't-miss, consolidate→MDM finalize. Worked example of how a clinical workflow is composed in this codebase.
|
||||
- [docs/logic/bedside-and-calculators.md](docs/logic/bedside-and-calculators.md) — Bedside emergencies module (ES-module pocket of the frontend), pediatric calculators, PE Guide, suture selector. Lists every clinical formula that must NOT be modified without test vectors.
|
||||
- [docs/logic/ai-and-voice.md](docs/logic/ai-and-voice.md) — `callAI` 5-provider routing, prompt centralization with DB overrides, `wrapUserText`+`INJECTION_GUARD`, server STT routing, browser Whisper, the helper trio (refine/billing/don't-miss).
|
||||
- [docs/logic/auth-admin-learning.md](docs/logic/auth-admin-learning.md) — local + OIDC auth, 2FA, sessions, OpenBao secret loading, Admin panel, Learning Hub.
|
||||
|
||||
### Operational + reference
|
||||
|
||||
- [Architecture Overview](docs/architecture.md) — high-level (the deep version is in docs/logic/architecture.md)
|
||||
- [API Reference](docs/api-reference.md)
|
||||
- [Database Schema](docs/database.md)
|
||||
- [Authentication & Security](docs/authentication.md)
|
||||
- [AI Providers & Models](docs/ai-providers.md)
|
||||
- [Speech (STT/TTS)](docs/speech.md)
|
||||
- [Learning Hub & CMS](docs/learning-hub.md)
|
||||
- [Configuration Reference](docs/configuration.md)
|
||||
- [Deployment Guide](docs/deployment.md)
|
||||
- [Developer Guide (short)](docs/developer-guide.md)
|
||||
- [Developer Guide (extended)](docs/developer-guide-extended.md)
|
||||
- [Browser Whisper Setup](docs/browser-whisper-setup.md) · [Troubleshooting](docs/browser-whisper-troubleshooting.md)
|
||||
- [Embeddings Setup](docs/embeddings-setup.md)
|
||||
- [OpenID Connect Setup](docs/openid-setup.md)
|
||||
- [Transcription Options](docs/transcription-options.md)
|
||||
- [Features Explained](docs/features-explained.md)
|
||||
- [Improvement Roadmap](docs/improvements.md)
|
||||
|
||||
---
|
||||
|
||||
## Development
|
||||
|
||||
```bash
|
||||
npm install
|
||||
cp .env.example .env # edit with your keys
|
||||
# Requires PostgreSQL with pgvector
|
||||
node server.js
|
||||
```
|
||||
|
||||
---
|
||||
Run the reindex command after major Postgres image changes, restoring a dump from another distro, or seeing lookup behavior that suggests collation/index drift.
|
||||
|
||||
## Testing
|
||||
|
||||
Two layers, both zero-config after the initial setup.
|
||||
|
||||
### Unit tests — pure dose math (Node built-in)
|
||||
Run the Node test suite:
|
||||
|
||||
```bash
|
||||
npm test
|
||||
```
|
||||
|
||||
Runs `node --test test/` against `public/js/calc-math.js` — pure functions for
|
||||
APLS / Best Guess weight, Parkland, Holliday-Segar 4-2-1, PRAM, Westley,
|
||||
epi (anaphylaxis vs arrest vs NRP, different concentrations), RSI drugs,
|
||||
min SBP, ETT sizing, Lund-Browder TBSA. **36 assertions, no dependencies.**
|
||||
|
||||
### End-to-end tests — Playwright smoke suite
|
||||
|
||||
Runs a headless Chromium against the live app. **128 tests** covering every
|
||||
calculator tab, every Bedside sub-pill + widget, auth-gated pages (encounter,
|
||||
well visit, charts, vaccines, catch-up, learning hub, dictation, settings,
|
||||
FAQ), at **both desktop and mobile (Pixel 5) viewports**.
|
||||
Run syntax checks for touched files when doing focused backend work:
|
||||
|
||||
```bash
|
||||
# First-time setup: spin up the auth-less test container (port 3553)
|
||||
docker compose -f docker-compose.yml -f docker-compose.e2e.yml up -d pediatric-scribe-e2e
|
||||
node --check server.js
|
||||
node --check src/routes/transcribe.js
|
||||
```
|
||||
|
||||
# Then run the full suite (runs inside an official Playwright container)
|
||||
Run the Playwright smoke suite against the e2e compose stack:
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.yml -f docker-compose.e2e.yml up -d pediatric-scribe-e2e
|
||||
npm run e2e
|
||||
```
|
||||
|
||||
The runner script (`scripts/e2e.sh`) uses `mcr.microsoft.com/playwright` so you
|
||||
don't need Node or browsers on the host.
|
||||
## Deployment Notes
|
||||
|
||||
**Test environment:**
|
||||
- Put the app behind HTTPS before clinical use.
|
||||
- Use only AI/STT/TTS providers covered by your BAA and data-processing requirements.
|
||||
- Configure OIDC/SSO and 2FA for production users.
|
||||
- Keep `JWT_SECRET`, database credentials, provider keys, S3 keys, SMTP credentials, and OpenBao tokens out of git.
|
||||
- Treat logs as sensitive operational data even with redaction enabled.
|
||||
- Use the Caddy/reverse-proxy layer to expose only intended public routes.
|
||||
|
||||
- `pediatric-ai-scribe` (port 3552) — your normal app
|
||||
- `pediatric-ai-scribe-e2e` (port 3553) — identical image, but with
|
||||
`TURNSTILE_SECRET_KEY=""` and `SMTP_HOST=""` so Playwright can log in
|
||||
without a bot challenge. Shares the same Postgres + pgdata volume.
|
||||
- Test user: `e2e-user@ped-ai.test` (auto-verified on first register)
|
||||
- Harness page: `public/e2e-harness.html` loads the calculators component
|
||||
without the auth wall for smoke tests that don't need a logged-in session.
|
||||
## Documentation
|
||||
|
||||
**Viewing failures** — Playwright writes `e2e/test-results/<test-name>/`
|
||||
with:
|
||||
Primary references:
|
||||
|
||||
- `test-failed-1.png` — screenshot at the point of failure
|
||||
- `trace.zip` — full action trace (replay with `npx playwright show-trace`)
|
||||
- `error-context.md` — DOM snapshot and console logs
|
||||
- `docs/architecture.md` for high-level architecture.
|
||||
- `docs/api-reference.md` for API routes.
|
||||
- `docs/authentication.md` for auth, OIDC, and security configuration.
|
||||
- `docs/ai-providers.md` for model/provider setup.
|
||||
- `docs/speech.md` for server-side STT/TTS setup.
|
||||
- `docs/learning-hub.md` for the CMS and education workflow.
|
||||
- `docs/configuration.md` for environment variables.
|
||||
- `docs/deployment.md` for production deployment.
|
||||
- `docs/logic/README.md` for the deeper code walkthrough.
|
||||
|
||||
Everything but the specs and config is gitignored under `e2e/`.
|
||||
Some deep `docs/logic/` files still describe historical implementation details. Prefer runtime code and tests when documentation conflicts with current behavior.
|
||||
|
||||
**Files:**
|
||||
## Clinical Safety
|
||||
|
||||
- `e2e/tests/bedside-smoke.spec.js` — 26 tests for the Bedside module
|
||||
- `e2e/tests/top-calculators.spec.js` — 27 tests for BP / BMI / Growth /
|
||||
Bili / Vitals / BSA / Dose / Resus / GCS / Equipment
|
||||
- `e2e/tests/auth-gated-smoke.spec.js` — 11 tests for the auth-gated tabs
|
||||
- `e2e/playwright.config.js` — runs all the above under both `chromium`
|
||||
(Desktop Chrome) and `mobile-chrome` (Pixel 5) projects
|
||||
|
||||
**Writing a new test:**
|
||||
|
||||
```js
|
||||
const { test, expect } = require('@playwright/test');
|
||||
|
||||
test('my new smoke test', async ({ page }) => {
|
||||
await page.goto('/e2e-harness.html'); // bypasses auth for calculators
|
||||
await page.waitForFunction(() => window.__harnessReady === true);
|
||||
await page.click('button.calc-nav-pill[data-calc="bedside"]');
|
||||
await expect(page.locator('#calc-bedside')).toBeVisible();
|
||||
});
|
||||
```
|
||||
|
||||
For auth-gated routes, use the login fixture in `auth-gated-smoke.spec.js`
|
||||
as a template — it caches the token at module scope so you don't hit the
|
||||
login rate-limit.
|
||||
Ped-AI is documentation and education support software. It does not replace clinical judgment, local policy, medication verification, or attending review. Validate generated notes, calculations, and recommendations before use in patient care.
|
||||
|
|
|
|||
|
|
@ -9,6 +9,7 @@ services:
|
|||
environment:
|
||||
CLINICAL_ASSISTANT_MCP_URL: http://mcp:8000/mcp
|
||||
REDIS_URL: redis://ped-ai-redis:6379
|
||||
LOKI_URL: http://monitoring-loki:3100
|
||||
volumes:
|
||||
- scribe-logs:/app/data/logs
|
||||
- clinical-assistant-mcp-data:/app/mcp-data:ro
|
||||
|
|
@ -22,6 +23,7 @@ services:
|
|||
networks:
|
||||
- default
|
||||
- mcp-server_default
|
||||
- monitoring_default
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "--spider", "-q", "http://localhost:3000/api/health"]
|
||||
interval: 30s
|
||||
|
|
@ -76,3 +78,5 @@ volumes:
|
|||
networks:
|
||||
mcp-server_default:
|
||||
external: true
|
||||
monitoring_default:
|
||||
external: true
|
||||
|
|
|
|||
|
|
@ -1,174 +0,0 @@
|
|||
# Browser Whisper Self-Hosted Setup
|
||||
|
||||
## Overview
|
||||
|
||||
As of v3, Browser Whisper is **fully self-hosted** with **zero CDN dependencies**. All models and libraries are bundled with the application and served from your own server.
|
||||
|
||||
## What Changed
|
||||
|
||||
**Before (v2 and earlier):**
|
||||
- Loaded transformers.js from `cdn.jsdelivr.net`
|
||||
- Downloaded models from `cdn-lfs.huggingface.co`
|
||||
- Failed in corporate/clinical networks with firewall restrictions
|
||||
|
||||
**Now (v3+):**
|
||||
- Transformers.js library (v2.6.2) bundled at `/models/transformers.min.js` (760KB)
|
||||
- Whisper model bundled at `/models/Xenova/whisper-tiny.en/` (42MB)
|
||||
- Everything served from your own server
|
||||
- **Works in any network environment** (firewalled, air-gapped, offline)
|
||||
|
||||
## Files Included
|
||||
|
||||
```
|
||||
public/models/
|
||||
├── transformers.min.js (760KB) - Transformers.js v2.6.2 (worker-compatible)
|
||||
└── Xenova/
|
||||
└── whisper-tiny.en/ (42MB total)
|
||||
├── config.json
|
||||
├── tokenizer.json
|
||||
├── preprocessor_config.json
|
||||
├── generation_config.json
|
||||
└── onnx/
|
||||
├── encoder_model_quantized.onnx
|
||||
└── decoder_model_merged_quantized.onnx
|
||||
```
|
||||
|
||||
## How It Works
|
||||
|
||||
1. **Worker loads transformers.js locally:**
|
||||
```javascript
|
||||
importScripts('/models/transformers.min.js');
|
||||
```
|
||||
|
||||
2. **Transformers.js configured for local models:**
|
||||
```javascript
|
||||
T.env.localModelPath = '/models/';
|
||||
T.env.allowRemoteModels = false;
|
||||
```
|
||||
|
||||
3. **Models load from your server:**
|
||||
- Browser requests: `GET /models/Xenova/whisper-tiny.en/config.json`
|
||||
- Served by Express static middleware
|
||||
- No external network calls
|
||||
|
||||
## Docker Build
|
||||
|
||||
Models are downloaded **during Docker build** (not runtime):
|
||||
|
||||
```dockerfile
|
||||
RUN curl -sL -o onnx/encoder_model_quantized.onnx \
|
||||
https://huggingface.co/Xenova/whisper-tiny.en/resolve/main/onnx/encoder_model_quantized.onnx
|
||||
```
|
||||
|
||||
This means:
|
||||
- Docker image is ~200MB larger (one-time cost)
|
||||
- Runtime has zero dependencies
|
||||
- Works in air-gapped environments (after image is pulled)
|
||||
|
||||
## Development Setup
|
||||
|
||||
If you're running locally (not Docker), download models:
|
||||
|
||||
```bash
|
||||
cd public/models
|
||||
mkdir -p Xenova/whisper-tiny.en/onnx
|
||||
|
||||
# Download transformers.js
|
||||
curl -L -o transformers.min.js \
|
||||
https://cdn.jsdelivr.net/npm/@xenova/transformers@2.17.2/dist/transformers.min.js
|
||||
|
||||
# Download model files
|
||||
cd Xenova/whisper-tiny.en
|
||||
curl -L -o config.json \
|
||||
https://huggingface.co/Xenova/whisper-tiny.en/resolve/main/config.json
|
||||
curl -L -o tokenizer.json \
|
||||
https://huggingface.co/Xenova/whisper-tiny.en/resolve/main/tokenizer.json
|
||||
curl -L -o preprocessor_config.json \
|
||||
https://huggingface.co/Xenova/whisper-tiny.en/resolve/main/preprocessor_config.json
|
||||
curl -L -o generation_config.json \
|
||||
https://huggingface.co/Xenova/whisper-tiny.en/resolve/main/generation_config.json
|
||||
curl -L -o onnx/encoder_model_quantized.onnx \
|
||||
https://huggingface.co/Xenova/whisper-tiny.en/resolve/main/onnx/encoder_model_quantized.onnx
|
||||
curl -L -o onnx/decoder_model_merged_quantized.onnx \
|
||||
https://huggingface.co/Xenova/whisper-tiny.en/resolve/main/onnx/decoder_model_merged_quantized.onnx
|
||||
```
|
||||
|
||||
Or use the helper script:
|
||||
|
||||
```bash
|
||||
./scripts/download-whisper-models.sh
|
||||
```
|
||||
|
||||
## Adding More Models
|
||||
|
||||
To add base or small models:
|
||||
|
||||
1. **Create directory:**
|
||||
```bash
|
||||
mkdir -p public/models/Xenova/whisper-base.en/onnx
|
||||
```
|
||||
|
||||
2. **Download from HuggingFace:**
|
||||
- https://huggingface.co/Xenova/whisper-base.en
|
||||
- https://huggingface.co/Xenova/whisper-small.en
|
||||
|
||||
3. **Update UI in `settings.html`:**
|
||||
```html
|
||||
<option value="Xenova/whisper-base.en">Base (~74MB, better quality)</option>
|
||||
```
|
||||
|
||||
4. **Update Dockerfile** to download during build
|
||||
|
||||
## Benefits
|
||||
|
||||
✅ **Works everywhere** - No firewall/CDN issues
|
||||
✅ **Privacy-first** - Audio never leaves browser
|
||||
✅ **Offline capable** - After initial page load
|
||||
✅ **No API costs** - Zero transcription expenses
|
||||
✅ **Predictable** - Same model, same results
|
||||
✅ **Fast** - Local processing, no network latency
|
||||
|
||||
## Limitations
|
||||
|
||||
- Docker image is larger (~200MB vs ~150MB)
|
||||
- Only tiny model included by default (base/small optional)
|
||||
- Slower than cloud APIs for long recordings
|
||||
- Requires modern browser with WebAssembly support
|
||||
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
# 1. Start server
|
||||
docker-compose up -d
|
||||
|
||||
# 2. Open browser DevTools → Network tab
|
||||
# 3. Go to Settings → Browser Transcription
|
||||
# 4. Click "Pre-download model"
|
||||
# 5. Watch for requests to /models/* (should all be 200 OK from your server)
|
||||
# 6. NO requests to cdn.jsdelivr.net or huggingface.co
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**Issue: "Failed to load transformers library"**
|
||||
- Check: `GET /models/transformers.min.js` returns 200 OK
|
||||
- Verify file exists: `ls public/models/transformers.min.js`
|
||||
|
||||
**Issue: "Model load failed"**
|
||||
- Check: `GET /models/Xenova/whisper-tiny.en/config.json` returns 200 OK
|
||||
- Verify files exist: `ls public/models/Xenova/whisper-tiny.en/`
|
||||
|
||||
**Issue: Still seeing CDN requests**
|
||||
- Clear browser cache (Ctrl+Shift+R)
|
||||
- Check you're running v18+ (`/api/health` should show version)
|
||||
|
||||
## Migration from v17
|
||||
|
||||
If upgrading from v17:
|
||||
|
||||
1. Pull new Docker image: `docker-compose pull`
|
||||
2. Restart: `docker-compose up -d`
|
||||
3. Clear browser cache
|
||||
4. Test: Settings → Browser Transcription → Pre-download
|
||||
|
||||
No configuration changes needed - it just works!
|
||||
|
|
@ -1,240 +0,0 @@
|
|||
# Browser Whisper Troubleshooting
|
||||
|
||||
## 🎙️ What is Browser Whisper?
|
||||
|
||||
Browser Whisper is an **optional** client-side transcription feature that runs entirely in your browser using WebAssembly. It provides:
|
||||
- ✅ Zero network transmission (HIPAA-safe)
|
||||
- ✅ No API costs
|
||||
- ✅ Works offline
|
||||
- ✅ Privacy-first (audio never leaves device)
|
||||
|
||||
**However**, it requires downloading AI models from CDN servers.
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ Common Issue: CDN Blocked
|
||||
|
||||
### Error Message:
|
||||
```
|
||||
NetworkError: Failed to execute 'importScripts' on 'WorkerGlobalScope':
|
||||
The script at 'https://cdn.jsdelivr.net/npm/@xenova/transformers@2.17.2' failed to load.
|
||||
```
|
||||
|
||||
### What This Means:
|
||||
Your network/firewall is blocking access to:
|
||||
- `cdn.jsdelivr.net` (JavaScript library CDN)
|
||||
- `cdn-lfs.huggingface.co` (AI model files)
|
||||
|
||||
### Why It Happens:
|
||||
1. **Corporate firewall** - Many organizations block CDN domains
|
||||
2. **Browser extensions** - Ad blockers, privacy tools may block CDN
|
||||
3. **Network proxy** - Company proxy might filter JavaScript CDN
|
||||
4. **CSP restrictions** - Very strict Content Security Policy
|
||||
|
||||
---
|
||||
|
||||
## ✅ Solutions
|
||||
|
||||
### Option 1: Use Server Transcription (Recommended)
|
||||
|
||||
**Browser Whisper is optional!** The app works perfectly fine with server-side transcription.
|
||||
|
||||
**Server transcription providers:**
|
||||
- Google Gemini (via Vertex AI) - HIPAA-eligible
|
||||
- AWS Transcribe - HIPAA-eligible
|
||||
- OpenAI Whisper - Fast, accurate
|
||||
- LiteLLM - Routes to any provider
|
||||
|
||||
**To use server transcription:**
|
||||
1. Go to Settings → Browser Transcription
|
||||
2. **Leave it disabled** (or if stuck, disable it)
|
||||
3. Record audio normally - will use server
|
||||
|
||||
**Advantages:**
|
||||
- More accurate (larger models)
|
||||
- No download needed
|
||||
- Works immediately
|
||||
- Professional grade
|
||||
|
||||
### Option 2: Whitelist CDN Domains
|
||||
|
||||
If you control your network/firewall, whitelist these domains:
|
||||
|
||||
```
|
||||
cdn.jsdelivr.net
|
||||
cdn-lfs.huggingface.co
|
||||
cdn-lfs-us-1.huggingface.co
|
||||
cdn-lfs-us-2.huggingface.co
|
||||
huggingface.co
|
||||
```
|
||||
|
||||
**For corporate IT:**
|
||||
- These are legitimate AI/JavaScript CDNs
|
||||
- Used by major companies worldwide
|
||||
- No security risk (public CDN content)
|
||||
- Required only for browser-based AI features
|
||||
|
||||
### Option 3: Disable Browser Extensions
|
||||
|
||||
Try disabling:
|
||||
- Ad blockers (uBlock Origin, AdBlock Plus)
|
||||
- Privacy extensions (Privacy Badger, Ghostery)
|
||||
- Script blockers (NoScript, ScriptSafe)
|
||||
|
||||
Then refresh and try again.
|
||||
|
||||
### Option 4: Try Different Browser
|
||||
|
||||
Some browsers have stricter security:
|
||||
- ✅ **Chrome** - Best compatibility
|
||||
- ✅ **Edge** - Works well
|
||||
- ⚠️ **Firefox** - May block CDN
|
||||
- ❌ **Safari** - Limited WebAssembly support
|
||||
|
||||
---
|
||||
|
||||
## 🧪 How to Test If It's Working
|
||||
|
||||
### Test 1: Check CDN Access
|
||||
```bash
|
||||
# From your computer, run:
|
||||
curl -I https://cdn.jsdelivr.net/npm/@xenova/transformers@2.17.2
|
||||
|
||||
# Should return: HTTP/2 200
|
||||
# If 403 or timeout: CDN is blocked
|
||||
```
|
||||
|
||||
### Test 2: Browser Console
|
||||
1. Open DevTools (F12)
|
||||
2. Go to Console tab
|
||||
3. Settings → Browser Transcription
|
||||
4. Click "Pre-download model"
|
||||
5. Watch for:
|
||||
```
|
||||
✅ [WhisperWorker] Transformers library loaded successfully
|
||||
OR
|
||||
❌ NetworkError: Failed to load
|
||||
```
|
||||
|
||||
### Test 3: Network Tab
|
||||
1. Open DevTools (F12)
|
||||
2. Go to Network tab
|
||||
3. Click "Pre-download model"
|
||||
4. Look for requests to:
|
||||
- `cdn.jsdelivr.net` (should be 200 OK)
|
||||
- `cdn-lfs.huggingface.co` (should be 200 OK)
|
||||
5. If blocked: Status will show "failed" or "blocked"
|
||||
|
||||
---
|
||||
|
||||
## 📊 When to Use Each Option
|
||||
|
||||
| Scenario | Recommendation | Why |
|
||||
|----------|---------------|-----|
|
||||
| Corporate network | **Server transcription** | CDN likely blocked |
|
||||
| Home network | **Browser Whisper** | Fast, free, private |
|
||||
| Mobile device | **Server transcription** | Limited storage/memory |
|
||||
| Offline use needed | **Browser Whisper** | Works without internet (after initial download) |
|
||||
| High accuracy needed | **Server transcription** | Larger models available |
|
||||
| Maximum privacy | **Browser Whisper** | Audio never leaves device |
|
||||
| Can't access CDN | **Server transcription** | No choice - CDN blocked |
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Technical Details
|
||||
|
||||
### What Gets Downloaded (First Time Only):
|
||||
|
||||
**Tiny model** (~39 MB):
|
||||
- onnx-runtime.wasm (~10 MB)
|
||||
- whisper-tiny.en model files (~29 MB)
|
||||
- Cached in browser IndexedDB (permanent)
|
||||
|
||||
**Base model** (~74 MB):
|
||||
- Larger model, better accuracy
|
||||
|
||||
**Small model** (~244 MB):
|
||||
- Best quality, slower processing
|
||||
|
||||
### Where It's Stored:
|
||||
- **Location:** Browser IndexedDB
|
||||
- **Persistence:** Permanent (until you clear browser data)
|
||||
- **Shared:** Across all tabs/windows for this domain
|
||||
- **Size:** Selected model size (39/74/244 MB)
|
||||
|
||||
### Performance:
|
||||
- **Tiny:** 2-3 seconds per 30-second clip
|
||||
- **Base:** 3-5 seconds per 30-second clip
|
||||
- **Small:** 6-10 seconds per 30-second clip
|
||||
|
||||
---
|
||||
|
||||
## ❓ FAQ
|
||||
|
||||
**Q: Is Browser Whisper required?**
|
||||
A: No! It's completely optional. Server transcription works great.
|
||||
|
||||
**Q: Why doesn't it work on my corporate network?**
|
||||
A: Most corporate firewalls block CDN domains for security. Use server transcription instead.
|
||||
|
||||
**Q: Can I download the models manually?**
|
||||
A: Not easily - they're optimized for CDN delivery. Use server transcription if CDN is blocked.
|
||||
|
||||
**Q: Will server transcription cost money?**
|
||||
A: Depends on your provider:
|
||||
- Google Vertex AI: ~$0.005 per minute
|
||||
- AWS Transcribe: ~$0.024 per minute
|
||||
- OpenAI: $0.006 per minute
|
||||
- Very affordable for typical use
|
||||
|
||||
**Q: Is server transcription HIPAA-safe?**
|
||||
A: Yes, if using:
|
||||
- Google Vertex AI (with BAA)
|
||||
- AWS Transcribe (with BAA)
|
||||
- Azure OpenAI (with BAA)
|
||||
|
||||
OpenAI Whisper direct is NOT HIPAA-eligible.
|
||||
|
||||
**Q: Can I use both?**
|
||||
A: Yes! Enable Browser Whisper in Settings. If it fails (CDN blocked), it automatically falls back to server transcription.
|
||||
|
||||
**Q: How do I know which one is being used?**
|
||||
A: Check the toast notification after recording:
|
||||
- "Transcribed locally" = Browser Whisper
|
||||
- "Transcribed via google-gemini/aws/openai" = Server
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Recommended Setup
|
||||
|
||||
### For Maximum Privacy (Home Network):
|
||||
1. Enable Browser Whisper
|
||||
2. Choose "Tiny" model (fast, good enough for dictation)
|
||||
3. Pre-download model
|
||||
4. Use offline
|
||||
|
||||
### For Corporate/Clinical Use:
|
||||
1. Keep Browser Whisper **disabled**
|
||||
2. Configure server transcription:
|
||||
```bash
|
||||
# In .env:
|
||||
TRANSCRIBE_PROVIDER=google
|
||||
GOOGLE_VERTEX_PROJECT=your-project
|
||||
```
|
||||
3. Use with BAA for HIPAA compliance
|
||||
|
||||
### For Best Accuracy:
|
||||
1. Use server transcription
|
||||
2. Configure Google Gemini 2.0 Flash or AWS Transcribe Medical
|
||||
3. Audio quality + large models = best results
|
||||
|
||||
---
|
||||
|
||||
## 🛠️ Still Having Issues?
|
||||
|
||||
1. **Check console logs:** DevTools → Console → Look for `[BrowserWhisper]` errors
|
||||
2. **Check network logs:** DevTools → Network → Filter by `jsdelivr` or `huggingface`
|
||||
3. **Verify server transcription works:** Just disable Browser Whisper and record
|
||||
4. **Contact IT:** Ask to whitelist CDN domains (if you need Browser Whisper)
|
||||
|
||||
**Remember:** Browser Whisper is a nice-to-have feature. Server transcription is the primary, production-ready method that works everywhere!
|
||||
|
|
@ -1,347 +1,73 @@
|
|||
# Features Explained - Pediatric AI Scribe v14
|
||||
# Features Explained
|
||||
|
||||
## 🎙️ **Audio Backups**
|
||||
This file is a practical operator-oriented overview of major Ped-AI features. It intentionally describes the current fork, not historical browser Whisper behavior.
|
||||
|
||||
### How It Works:
|
||||
Audio backups happen **automatically every time you record**, regardless of transcription success/failure.
|
||||
## Clinical Documentation
|
||||
|
||||
**Flow:**
|
||||
1. You press "Stop" on recording
|
||||
2. Audio is immediately saved **before** transcription starts
|
||||
3. Server-side backup (PostgreSQL, gzip compressed) attempted first
|
||||
4. If server fails → fallback to browser IndexedDB
|
||||
5. After successful transcription → audio backup is deleted
|
||||
6. If transcription fails → audio backup remains for retry
|
||||
Ped-AI generates pediatric clinical notes from typed input, dictation, or recorded audio. Major workflows include live encounters, dictation cleanup, sick visits, well visits, SOAP notes, hospital courses, chart review, ED documentation, and developmental milestones.
|
||||
|
||||
**Location:**
|
||||
- Server: PostgreSQL `audio_backups` table (auto-deleted after 24 hours)
|
||||
- Browser: IndexedDB `PedScribeAudioBackup` database (manual cleanup)
|
||||
Model selection is available per task where the UI exposes a tab-level selector. Admin defaults provide the baseline model and user/task choices can override that baseline.
|
||||
|
||||
**Purpose:**
|
||||
- Retry transcription if it fails
|
||||
- Recover audio if browser crashes
|
||||
- Audit trail (24 hour retention)
|
||||
## Speech
|
||||
|
||||
**Access:**
|
||||
Settings → Audio Backups section shows:
|
||||
- Date/time of recording
|
||||
- Module (encounter, dictation, etc.)
|
||||
- File size
|
||||
- "Retry Transcription" button (if transcription failed)
|
||||
- "Delete" button
|
||||
Final transcription is server-side. Configure Google/Gemini, AWS Transcribe, LiteLLM, or OpenAI Whisper according to your deployment requirements.
|
||||
|
||||
**Cost:**
|
||||
Server backups are compressed (gzip) to ~1/10 original size. A 2MB recording becomes ~200KB in database.
|
||||
Browser-native Web Speech is only an explicit opt-in preview path. It is not the final clinical transcript and may use browser-vendor cloud services.
|
||||
|
||||
---
|
||||
Browser Whisper and browser-local model workers are removed. Do not expect a pre-download model button, public Whisper worker, or bundled Xenova model path.
|
||||
|
||||
## 🌐 **S3 Document Storage**
|
||||
## Text To Speech
|
||||
|
||||
### How It Works:
|
||||
Upload documents (PDFs, images, Word docs, text files) to S3-compatible storage.
|
||||
The voice preview button calls the configured TTS provider and plays the returned audio in the browser. If preview is silent, check that a voice is selected, a provider is configured, the user is authenticated, and browser autoplay has not blocked playback.
|
||||
|
||||
**Supported Providers:**
|
||||
- AWS S3 (default)
|
||||
- Backblaze B2
|
||||
- MinIO (self-hosted)
|
||||
- Any S3-compatible service
|
||||
## Learning Hub
|
||||
|
||||
**Configuration (.env):**
|
||||
```bash
|
||||
# AWS S3 (uses Bedrock credentials if available)
|
||||
S3_BUCKET=your-bucket-name
|
||||
S3_REGION=us-east-1
|
||||
S3_PREFIX=documents/ # Optional: folder prefix
|
||||
Learning Hub is both a learner-facing content area and an admin/moderator CMS.
|
||||
|
||||
# Backblaze B2
|
||||
S3_BUCKET=your-bucket-name
|
||||
S3_ENDPOINT=https://s3.us-west-004.backblazeb2.com
|
||||
S3_REGION=us-west-004
|
||||
S3_ACCESS_KEY_ID=your-b2-application-key-id
|
||||
S3_SECRET_ACCESS_KEY=your-b2-application-key
|
||||
- Articles and pearls render sanitized content.
|
||||
- Quizzes support single-answer, multi-select, and true/false questions.
|
||||
- Presentations use Marp-style markdown with preview and PPTX export.
|
||||
- AI generation can use topic text, uploaded source files, or connected Nextcloud WebDAV files.
|
||||
- Categories can organize content without deleting the content when category assignments change.
|
||||
|
||||
# MinIO (self-hosted)
|
||||
S3_BUCKET=your-bucket
|
||||
S3_ENDPOINT=http://minio:9000
|
||||
S3_REGION=us-east-1
|
||||
S3_ACCESS_KEY_ID=minio-access-key
|
||||
S3_SECRET_ACCESS_KEY=minio-secret-key
|
||||
S3_FORCE_PATH_STYLE=true # Required for MinIO
|
||||
```
|
||||
## Nextcloud WebDAV
|
||||
|
||||
**Features:**
|
||||
- ✅ 10 MB file size limit
|
||||
- ✅ AES-256 server-side encryption
|
||||
- ✅ Per-user folder organization (`documents/{userId}/{uuid}/filename`)
|
||||
- ✅ Metadata stored in PostgreSQL (filename, mime type, size, description)
|
||||
- ✅ Presigned URLs for secure access (1 hour expiry)
|
||||
Users can connect a Nextcloud account with an app password. Learning Hub AI generation can browse files from the connected WebDAV account, and users can set a default browse path to avoid repeatedly navigating to the same clinical content folder.
|
||||
|
||||
**Allowed File Types:**
|
||||
- PDF (`.pdf`)
|
||||
- Images (`.jpg`, `.jpeg`, `.png`, `.gif`)
|
||||
- Word documents (`.doc`, `.docx`)
|
||||
- Text files (`.txt`, `.csv`)
|
||||
## Documents And S3
|
||||
|
||||
**Access:**
|
||||
Settings → Documents section
|
||||
Document upload is optional and depends on S3-compatible storage configuration. Treat uploaded documents as PHI unless you have a separate deployment reason not to.
|
||||
|
||||
**Status Check:**
|
||||
If S3 is not configured, the Documents section shows empty with message: "S3 not configured"
|
||||
## Audio Backups
|
||||
|
||||
---
|
||||
Audio backups exist to recover failed transcription attempts.
|
||||
|
||||
## 📚 **Learning Hub - Default Browse Path**
|
||||
- They are created when transcription fails.
|
||||
- They are encrypted before persistent storage.
|
||||
- They expire automatically.
|
||||
- Users can retry or delete them from Settings.
|
||||
|
||||
### What It Is:
|
||||
A user preference that sets the **starting folder** when browsing Nextcloud files for AI content generation.
|
||||
## Admin Panel
|
||||
|
||||
### When It's Used:
|
||||
Only in the **Learning Hub AI Content Generator** (Admin/Moderator feature).
|
||||
Admins can manage users, roles, registration, security settings, model defaults, prompts, logs, and Learning Hub content. Production deployments should enable SSO/2FA and restrict admin access.
|
||||
|
||||
**Scenario:**
|
||||
1. Admin/Moderator wants to create AI-generated learning content
|
||||
2. They choose "Upload from Nextcloud"
|
||||
3. File browser opens
|
||||
4. Instead of starting at root `/`, it opens at the configured path
|
||||
## Feature Status
|
||||
|
||||
**Example:**
|
||||
```
|
||||
Default path: /Medical-Resources
|
||||
↓
|
||||
When you click "Browse Nextcloud", it opens:
|
||||
/Medical-Resources/
|
||||
├── Pediatric-Guidelines/
|
||||
├── Clinical-Protocols/
|
||||
└── Research-Papers/
|
||||
| Feature | Status | Notes |
|
||||
|---|---|---|
|
||||
| Clinical note generation | Active | Provider depends on `AI_PROVIDER`. |
|
||||
| Server transcription | Active | Google/AWS/LiteLLM/OpenAI paths. |
|
||||
| Browser Web Speech preview | Optional | Explicit opt-in only. |
|
||||
| Browser Whisper | Removed | No public worker or model download path. |
|
||||
| Learning Hub CMS | Active | Articles, pearls, quizzes, presentations. |
|
||||
| Nextcloud WebDAV | Active | Used for file browsing/content import. |
|
||||
| Audio backups | Active | Failure recovery only. |
|
||||
| TTS preview | Active | Depends on configured provider. |
|
||||
|
||||
Instead of:
|
||||
/
|
||||
├── Personal/
|
||||
├── Photos/
|
||||
├── Medical-Resources/ ← you'd have to navigate here every time
|
||||
└── ...
|
||||
```
|
||||
## Troubleshooting
|
||||
|
||||
**Configuration:**
|
||||
Settings → Nextcloud Integration → "Learning Hub — Default Browse Path"
|
||||
|
||||
**Examples:**
|
||||
- `/Medical-Resources` - Opens in Medical Resources folder
|
||||
- `/Shared/Clinical-Content` - Opens in shared clinical content
|
||||
- `/` (empty) - Opens at root (default behavior)
|
||||
|
||||
**Who Can Use This:**
|
||||
- Any authenticated user (not just moderators)
|
||||
- It's a personal preference per user
|
||||
- Only affects Learning Hub AI file picker
|
||||
|
||||
**Why This Exists:**
|
||||
If you store learning resources in a specific Nextcloud folder, you don't want to navigate there every single time you generate content. Set it once, it remembers.
|
||||
|
||||
---
|
||||
|
||||
## 🎤 **Browser Whisper Pre-Download**
|
||||
|
||||
### Issue You Reported:
|
||||
"Pre-download models works, stuck at starting download"
|
||||
|
||||
### What's Happening:
|
||||
The download **is actually working** but progress updates are slow because:
|
||||
1. HuggingFace CDN serves large files (39-244 MB)
|
||||
2. Progress callbacks are not granular (reported per-file, not per-chunk)
|
||||
3. Initial ONNX runtime download has no progress tracking
|
||||
|
||||
### Fixed:
|
||||
- ✅ Added console logging to track progress
|
||||
- ✅ Added 30-second timeout warning (doesn't stop download)
|
||||
- ✅ Better error messages
|
||||
|
||||
### How to Test:
|
||||
1. Open browser DevTools (F12) → Console tab
|
||||
2. Click "Pre-download model"
|
||||
3. Watch console for progress logs:
|
||||
```
|
||||
[BrowserWhisper] Starting preload...
|
||||
[BrowserWhisper] Progress: onnx-runtime 0%
|
||||
[BrowserWhisper] Progress: model.bin 23%
|
||||
[BrowserWhisper] Progress: model.bin 47%
|
||||
...
|
||||
[BrowserWhisper] Progress: 100%
|
||||
```
|
||||
|
||||
### Expected Download Times:
|
||||
- **Tiny** (39 MB): 5-15 seconds (fast connection)
|
||||
- **Base** (74 MB): 10-30 seconds
|
||||
- **Small** (244 MB): 30-90 seconds
|
||||
|
||||
### If Still Stuck:
|
||||
**Check these:**
|
||||
1. Open DevTools → Network tab
|
||||
2. Filter by "HuggingFace"
|
||||
3. Look for downloads from `cdn-lfs-us-1.huggingface.co`
|
||||
4. Check if files are actually downloading
|
||||
|
||||
**Common issues:**
|
||||
- Slow internet connection (244 MB takes time!)
|
||||
- Corporate firewall blocking HuggingFace CDN
|
||||
- Browser IndexedDB quota exceeded
|
||||
|
||||
**Workaround:**
|
||||
Just enable it and record audio - the model will download on first use (same as pre-download, but triggered automatically).
|
||||
|
||||
---
|
||||
|
||||
## 🔊 **TTS Voice Preview**
|
||||
|
||||
### Issue You Reported:
|
||||
"Preview button next to TTS seems to do nothing"
|
||||
|
||||
### Fixed:
|
||||
- ✅ Added error logging to console
|
||||
- ✅ Better validation (checks for empty selection)
|
||||
- ✅ Clear user feedback messages
|
||||
|
||||
### How to Use:
|
||||
1. Go to Settings → Voice Preferences
|
||||
2. Select a voice from "Text-to-Speech Voice" dropdown
|
||||
3. Click "Preview" button
|
||||
4. Wait 2-3 seconds
|
||||
5. Audio should play automatically
|
||||
|
||||
### If Nothing Happens:
|
||||
**Check browser console for errors:**
|
||||
- Open DevTools (F12) → Console tab
|
||||
- Click Preview
|
||||
- Look for `[VoicePrefs] Preview error:` message
|
||||
|
||||
**Common issues:**
|
||||
1. **No voice selected** → Select from dropdown first
|
||||
2. **TTS not configured** → Check `.env` has `GOOGLE_VERTEX_PROJECT` or `LITELLM_API_BASE`
|
||||
3. **Network error** → Check server logs for TTS API errors
|
||||
4. **Browser autoplay policy** → Some browsers block autoplay, click page first
|
||||
|
||||
### Testing Checklist:
|
||||
```bash
|
||||
# 1. Check TTS is configured
|
||||
curl http://localhost:3000/api/health | grep tts
|
||||
|
||||
# 2. Test TTS endpoint directly
|
||||
curl -X POST http://localhost:3000/api/text-to-speech \
|
||||
-H "Authorization: Bearer YOUR_JWT" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"text":"Test"}' \
|
||||
--output test.mp3
|
||||
|
||||
# 3. Play the audio file
|
||||
mpg123 test.mp3 # or open in browser
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📋 **Summary of User Settings**
|
||||
|
||||
### Voice Preferences
|
||||
**Location:** Settings → Voice Preferences (top section)
|
||||
|
||||
| Setting | Options | Default | Purpose |
|
||||
|---------|---------|---------|---------|
|
||||
| **STT Model** | gemini-2.0-flash-exp, gemini-2.0-flash, gemini-1.5-flash, gemini-1.5-pro, whisper-1 | Server default | Controls transcription accuracy |
|
||||
| **TTS Voice** | Journey-F/D, Studio-O/M, Neural2 series, alloy, echo, fable, onyx, nova, shimmer | Server default | Controls read-aloud voice |
|
||||
|
||||
### Browser Whisper
|
||||
**Location:** Settings → Browser Transcription (Local Whisper)
|
||||
|
||||
| Setting | Options | Default | Purpose |
|
||||
|---------|---------|---------|---------|
|
||||
| **Enable** | On/Off | Off | Local transcription (HIPAA-safe) |
|
||||
| **Model** | Tiny, Base, Small | Tiny | Accuracy vs speed tradeoff |
|
||||
|
||||
### Nextcloud
|
||||
**Location:** Settings → Nextcloud Integration
|
||||
|
||||
| Setting | Purpose |
|
||||
|---------|---------|
|
||||
| **Nextcloud URL** | Your Nextcloud instance |
|
||||
| **Username** | Nextcloud username |
|
||||
| **App Password** | Generate in Nextcloud → Security |
|
||||
| **Default Browse Path** | Starting folder for Learning Hub AI picker |
|
||||
|
||||
### Documents (S3)
|
||||
**Location:** Settings → Documents
|
||||
|
||||
Shows list of uploaded documents if S3 is configured. Upload limit: 10 MB per file.
|
||||
|
||||
### Audio Backups
|
||||
**Location:** Settings → Audio Backups
|
||||
|
||||
Shows last 24 hours of recordings. Can retry transcription or delete.
|
||||
|
||||
---
|
||||
|
||||
## 🔧 **Troubleshooting Guide**
|
||||
|
||||
### Pre-Download Stuck
|
||||
1. ✅ Open browser console (F12)
|
||||
2. ✅ Look for `[BrowserWhisper] Progress:` logs
|
||||
3. ✅ Check Network tab for HuggingFace downloads
|
||||
4. ✅ Wait - 244 MB takes time!
|
||||
5. ✅ If truly stuck (no network activity): refresh page, try again
|
||||
|
||||
### Preview Button Silent
|
||||
1. ✅ Check voice is selected in dropdown
|
||||
2. ✅ Open console for error messages
|
||||
3. ✅ Test TTS endpoint directly (curl command above)
|
||||
4. ✅ Check server logs for TTS provider errors
|
||||
5. ✅ Verify `.env` has TTS provider configured
|
||||
|
||||
### S3 Not Working
|
||||
1. ✅ Check `.env` has `S3_BUCKET` set
|
||||
2. ✅ Verify credentials: `S3_ACCESS_KEY_ID` + `S3_SECRET_ACCESS_KEY`
|
||||
3. ✅ Test bucket access from server:
|
||||
```bash
|
||||
aws s3 ls s3://your-bucket/ --region us-east-1
|
||||
```
|
||||
4. ✅ Check server logs for S3 errors when uploading
|
||||
|
||||
### Audio Backups Not Showing
|
||||
1. ✅ Record audio first (they're created on recording, not transcription)
|
||||
2. ✅ Check database: `SELECT COUNT(*) FROM audio_backups;`
|
||||
3. ✅ Verify IndexedDB in browser: DevTools → Application → IndexedDB → `PedScribeAudioBackup`
|
||||
4. ✅ Backups auto-delete after 24 hours
|
||||
|
||||
### Learning Hub Path Not Working
|
||||
1. ✅ This only affects **AI content generator file picker**
|
||||
2. ✅ It does NOT affect manual Nextcloud document browsing
|
||||
3. ✅ Path must exist in your Nextcloud
|
||||
4. ✅ Path format: `/Folder/Subfolder` (starts with `/`)
|
||||
|
||||
---
|
||||
|
||||
## 📊 **Feature Status Matrix**
|
||||
|
||||
| Feature | Status | Config Required | HIPAA-Safe | Notes |
|
||||
|---------|--------|-----------------|------------|-------|
|
||||
| **Audio Backups** | ✅ Working | None (auto) | ✅ Yes | Server + IndexedDB |
|
||||
| **S3 Documents** | ✅ Working | S3_BUCKET | ✅ Yes (AWS) | Optional feature |
|
||||
| **Browser Whisper** | ✅ Working | None (optional) | ✅ Yes | Client-side only |
|
||||
| **Voice Preferences** | ✅ Working | Provider config | Depends | Google/AWS = yes |
|
||||
| **Learning Hub Path** | ✅ Working | Nextcloud config | ✅ Yes | User preference |
|
||||
| **TTS Preview** | ✅ Fixed | TTS provider | Depends | Check logs if fails |
|
||||
| **Embeddings** | ✅ Working | Vertex/LiteLLM | ✅ Yes | Requires pgvector |
|
||||
|
||||
---
|
||||
|
||||
## 🚀 **Next Steps**
|
||||
|
||||
1. **Push v14 to Docker** (in progress via GitHub Actions)
|
||||
2. **Test features after deployment**
|
||||
3. **Check browser console for any errors**
|
||||
4. **Verify TTS preview works with your provider**
|
||||
5. **Test browser whisper download with different models**
|
||||
|
||||
---
|
||||
|
||||
**Questions? Check the logs:**
|
||||
- Browser: F12 → Console tab
|
||||
- Server: `docker logs pediatric-ai-scribe -f`
|
||||
- Database: `psql -d pedscribe -c "SELECT COUNT(*) FROM audio_backups;"`
|
||||
- Check browser console for frontend errors.
|
||||
- Check `docker logs pediatric-ai-scribe -f` for backend errors.
|
||||
- Check `/api/health` for service status.
|
||||
- Check provider credentials and model names before debugging UI state.
|
||||
- For Learning Hub file import failures, verify Nextcloud URL, username, app password, and folder path.
|
||||
|
|
|
|||
|
|
@ -52,7 +52,7 @@ This is the highest-impact improvement for adoption but also the most complex to
|
|||
|
||||
### 5. Offline Mode
|
||||
|
||||
**Current state:** The app requires an internet connection for AI generation and cloud-based transcription. Browser Whisper works offline for transcription only.
|
||||
**Current state:** The app requires configured server-side providers for AI generation and final transcription. Browser Whisper has been removed from the runtime.
|
||||
|
||||
**Improvement:** Add a local AI model option (e.g., a small medical LLM running on the device or local server) so the entire workflow — record, transcribe, generate note — can happen without any network calls. This would be valuable for:
|
||||
- Rural clinics with unreliable internet
|
||||
|
|
|
|||
|
|
@ -1,83 +1,41 @@
|
|||
# Speech: STT, TTS, audio backup
|
||||
# Speech: STT, TTS, Audio Backup
|
||||
|
||||
## Transcription (speech-to-text)
|
||||
## Transcription
|
||||
|
||||
### Overview
|
||||
`POST /api/transcribe` accepts `multipart/form-data` with one audio file up to 25 MB. The provider is selected by `TRANSCRIBE_PROVIDER`, or auto-detected from available credentials.
|
||||
|
||||
`POST /api/transcribe` accepts `multipart/form-data` with a single audio
|
||||
file (≤ 25 MB). Provider is `TRANSCRIBE_PROVIDER` env var, or auto-detected
|
||||
(`google > aws > openai`) from available credentials. Each user may override
|
||||
via `users.stt_model`; admin-wide default via `stt.model` in `app_settings`.
|
||||
Provider priority in auto mode is Google/Gemini, AWS Transcribe, LiteLLM, then OpenAI Whisper when configured.
|
||||
|
||||
### Providers
|
||||
|
||||
| Provider | Transport | HIPAA (with BAA) |
|
||||
| Provider | Notes | HIPAA posture |
|
||||
|---|---|---|
|
||||
| **Google Gemini** | Inline audio in `generateContent` call. Default model `gemini-2.0-flash`. | Yes |
|
||||
| **Amazon Transcribe** | Streaming. `AWS_TRANSCRIBE_MEDICAL=true` + `AWS_TRANSCRIBE_SPECIALTY` switches to Transcribe Medical. Specialties: `PRIMARYCARE`, `CARDIOLOGY`, `NEUROLOGY`, `ONCOLOGY`, `RADIOLOGY`, `UROLOGY`. | Yes |
|
||||
| **Local Whisper** | Spawns `whisper.cpp` or `faster-whisper` via `WHISPER_BINARY`. Fully offline. Model sizes `tiny`/`base`/`small`/`medium`/`large`. | N/A (nothing leaves host) |
|
||||
| **OpenAI Whisper** | `whisper-1` via `/v1/audio/transcriptions`. Medical-context prompt prepended: `"Medical patient encounter. Pediatric."` | No |
|
||||
| **LiteLLM** | Inline audio via LiteLLM's `chat.completions` endpoint (not the `/audio/transcriptions` path). Model from `LITELLM_STT_MODEL`. | Depends on LiteLLM backend |
|
||||
| Google/Gemini | Uses the configured Vertex/Gemini STT model. | Eligible with the correct Google Cloud agreement. |
|
||||
| AWS Transcribe | Supports standard and Medical mode. | Eligible with the correct AWS agreement. |
|
||||
| LiteLLM | Sends audio through the configured LiteLLM `/audio/transcriptions` backend. | Depends on the selected upstream. |
|
||||
| OpenAI Whisper | Uses `whisper-1` directly when `OPENAI_API_KEY` is configured. | Not HIPAA eligible unless your own agreement says otherwise. |
|
||||
|
||||
## Browser Whisper (fully offline)
|
||||
Browser Whisper and browser-local Whisper workers are not part of the runtime. Do not add browser model downloads or Transformers.js STT back into the public app.
|
||||
|
||||
Runs entirely in the browser via WebAssembly. Zero network. Suitable when
|
||||
no external transcription is acceptable.
|
||||
## Web Speech Preview
|
||||
|
||||
- Runtime: `@xenova/transformers` (WASM).
|
||||
- Models (bundled in the Docker image, no CDN fetch):
|
||||
- `whisper-tiny.en` — 39 MB
|
||||
- `whisper-base.en` — 74 MB
|
||||
- `whisper-small.en` — 244 MB
|
||||
- Executes in a dedicated Web Worker; UI thread is never blocked.
|
||||
- Models cached in IndexedDB after first load.
|
||||
- Per-user toggle. On browser transcription failure, the client falls back to
|
||||
server-side transcription without user intervention.
|
||||
Browser-native Web Speech can show interim text when the user explicitly enables it. It is browser/vendor dependent, may send audio to browser-provider cloud services, and should not be treated as the final clinical transcript.
|
||||
|
||||
## Live speech preview
|
||||
## Text To Speech
|
||||
|
||||
Chrome / Edge `webkitSpeechRecognition` streams interim text to the UI during
|
||||
recording. Used for real-time preview only — **not** for final transcription.
|
||||
The actual transcript comes from the configured STT provider after recording
|
||||
ends.
|
||||
|
||||
## Text-to-speech
|
||||
|
||||
### Overview
|
||||
|
||||
`POST /api/text-to-speech`. Returns `audio/mpeg`. `X-TTS-Provider` response
|
||||
header identifies the provider used. 5000-character limit per request. Each
|
||||
user may override via `users.tts_voice`; admin-wide default via `tts.voice`.
|
||||
|
||||
### Providers
|
||||
`POST /api/text-to-speech` returns `audio/mpeg`. The `X-TTS-Provider` response header identifies the provider used. Requests are limited to 5000 characters.
|
||||
|
||||
| Provider | Notes |
|
||||
|---|---|
|
||||
| **Google Cloud TTS** | `@google-cloud/text-to-speech`. Voice families: Journey, Studio, Neural2. |
|
||||
| **LiteLLM** | Configured via `LITELLM_TTS_MODEL` + `LITELLM_TTS_VOICE`. Backend-agnostic. |
|
||||
| **ElevenLabs** | `eleven_turbo_v2_5`. **Not HIPAA-compliant**. |
|
||||
| Google Cloud TTS | Uses Google Cloud voices when configured. |
|
||||
| LiteLLM | Uses `LITELLM_TTS_MODEL` and `LITELLM_TTS_VOICE`. |
|
||||
| ElevenLabs | Available when configured; not HIPAA eligible by default. |
|
||||
|
||||
## Audio backup
|
||||
## Audio Backup
|
||||
|
||||
Raw audio is saved to Postgres **only when transcription fails**, providing a
|
||||
retry window without persisting every recording.
|
||||
Failed transcription submissions can be stored for retry instead of being silently lost.
|
||||
|
||||
### Storage
|
||||
- Audio backups are compressed and encrypted before storage.
|
||||
- Backups expire automatically.
|
||||
- The Settings audio backup UI can retry or delete saved items.
|
||||
- Browser fallback storage is used only when the server cannot save the failed audio.
|
||||
|
||||
- Gzip-compressed, then AES-256-GCM encrypted (0x01 version byte prefix).
|
||||
- `BYTEA` column in `audio_backups`.
|
||||
- 24-hour `expires_at`, swept hourly.
|
||||
- Legacy rows (gzip magic `0x1F` as first byte, no encryption envelope)
|
||||
decompress as-is — detection is deterministic because `0x1F ≠ 0x01`.
|
||||
|
||||
### Retry UI
|
||||
|
||||
Settings → Audio Backups:
|
||||
- List: module, size, created, expiry.
|
||||
- **Retry** — resubmits to `POST /api/transcribe`.
|
||||
- **Delete** — purge now.
|
||||
|
||||
### Browser fallback
|
||||
|
||||
If the server-side save fails (network, 500, etc.), the client stores the audio
|
||||
in IndexedDB so it can retry later. Cleared after successful submission.
|
||||
Treat audio backups as sensitive clinical data even when encrypted.
|
||||
|
|
|
|||
|
|
@ -1,279 +1,59 @@
|
|||
# Transcription Options Guide
|
||||
# Transcription Options
|
||||
|
||||
## Overview
|
||||
Ped-AI currently supports server-side transcription plus an explicit browser Web Speech preview option. Browser Whisper was removed and should not be offered in settings, documentation, public workers, or model download scripts.
|
||||
|
||||
Pediatric AI Scribe v2+ offers **three transcription methods**, allowing you to choose between **privacy**, **speed**, and **real-time feedback**.
|
||||
## Recommended Clinical Setup
|
||||
|
||||
---
|
||||
Use a server-side provider covered by your compliance requirements.
|
||||
|
||||
## 📊 Comparison Table
|
||||
|
||||
| Feature | Browser Whisper | Server Transcription | Web Speech API |
|
||||
|---------|----------------|---------------------|----------------|
|
||||
| **Privacy** | ⭐⭐⭐⭐⭐ 100% offline | ⭐⭐⭐⭐ (with BAA) | ⭐ Sends to cloud |
|
||||
| **Accuracy** | ⭐⭐⭐⭐⭐ Whisper | ⭐⭐⭐⭐⭐ Gemini/AWS | ⭐⭐⭐ Browser-dependent |
|
||||
| **Speed** | ⭐⭐⭐ 2-10s | ⭐⭐⭐⭐⭐ ~1s | ⭐⭐⭐⭐⭐ Instant |
|
||||
| **Real-time** | ❌ Batch mode | ❌ Batch mode | ✅ Live streaming |
|
||||
| **HIPAA** | ✅ Yes | ✅ (Vertex/AWS) | ❌ No |
|
||||
| **Cost** | Free | ~$0.005/min | Free |
|
||||
| **Internet** | ❌ Not required | ✅ Required | ✅ Required |
|
||||
| **Setup** | None (bundled) | API keys | None (built-in) |
|
||||
|
||||
---
|
||||
|
||||
## Option 1: Browser Whisper (Offline, Private) ⭐ RECOMMENDED
|
||||
|
||||
### What It Is
|
||||
- Runs **OpenAI Whisper** entirely in your browser using WebAssembly
|
||||
- Audio **never leaves your device** - 100% offline after initial page load
|
||||
- Models bundled in Docker image (self-hosted, no CDN)
|
||||
|
||||
### When to Use
|
||||
- ✅ Clinical documentation (HIPAA-compliant)
|
||||
- ✅ Maximum privacy required
|
||||
- ✅ Offline/air-gapped environments
|
||||
- ✅ No API costs
|
||||
- ✅ Zero vendor dependency
|
||||
|
||||
### How to Enable
|
||||
1. Settings → Browser Transcription
|
||||
2. Toggle "Enable browser transcription" ON
|
||||
3. (Optional) Click "Pre-download model" if you want to cache it first
|
||||
4. Start recording - transcription happens automatically after recording
|
||||
|
||||
### Models Available
|
||||
- **Tiny** (~39MB) - Fast, good for short clips (2-3 seconds)
|
||||
- **Base** (~74MB) - Balanced accuracy and speed (3-5 seconds)
|
||||
- **Small** (~244MB) - Best quality, slower (6-10 seconds)
|
||||
|
||||
### Performance
|
||||
- Transcribes ~30-second clip in 2-10 seconds (depending on model)
|
||||
- First run may be slower (model loading)
|
||||
- Subsequent runs are instant (cached)
|
||||
|
||||
### Privacy
|
||||
- ✅ Audio never transmitted
|
||||
- ✅ Models run locally in WASM
|
||||
- ✅ No network calls during transcription
|
||||
- ✅ HIPAA-compliant
|
||||
|
||||
---
|
||||
|
||||
## Option 2: Server Transcription (Cloud, Fast)
|
||||
|
||||
### What It Is
|
||||
- Sends audio to your configured AI provider
|
||||
- Uses Google Gemini, AWS Transcribe, OpenAI Whisper, or LiteLLM
|
||||
|
||||
### When to Use
|
||||
- ✅ Maximum speed (~1 second for 30-second clip)
|
||||
- ✅ Best accuracy (cloud models)
|
||||
- ✅ Long recordings (Browser Whisper can be slow for 5+ minutes)
|
||||
- ✅ HIPAA-compliant with BAA providers
|
||||
|
||||
### HIPAA-Eligible Providers
|
||||
- **Google Vertex AI** (with BAA) ✅
|
||||
- **AWS Transcribe** (with BAA) ✅
|
||||
- **Azure OpenAI** (with BAA) ✅
|
||||
- **OpenAI Whisper Direct** ❌ Not HIPAA-eligible
|
||||
|
||||
### How to Enable
|
||||
- Configured via environment variables (`.env`)
|
||||
- No user action needed - just works if API keys present
|
||||
- Falls back automatically if Browser Whisper fails
|
||||
|
||||
### Cost
|
||||
- Google Gemini: ~$0.005/minute
|
||||
- AWS Transcribe: ~$0.024/minute
|
||||
- OpenAI: $0.006/minute
|
||||
|
||||
---
|
||||
|
||||
## Option 3: Web Speech API (Real-Time, Experimental) ⚠️
|
||||
|
||||
### What It Is
|
||||
- Uses your browser's built-in speech recognition
|
||||
- Shows transcription **in real-time** as you speak (streaming)
|
||||
- Chrome/Edge → Google Cloud Speech
|
||||
- Safari → Apple Speech Recognition
|
||||
|
||||
### ⚠️ PRIVACY WARNING
|
||||
- **Audio IS sent to cloud servers** (Google, Apple, etc.)
|
||||
- **NOT HIPAA-compliant**
|
||||
- Only use for non-clinical, personal use
|
||||
|
||||
### When to Use
|
||||
- ✅ Personal notes (non-clinical)
|
||||
- ✅ Want real-time feedback while speaking
|
||||
- ✅ Demonstration/testing
|
||||
- ❌ **NEVER for patient data**
|
||||
|
||||
### How to Enable
|
||||
1. Settings → Real-Time Streaming Transcription
|
||||
2. Read privacy warning carefully
|
||||
3. Toggle "Enable real-time streaming" ON
|
||||
4. Confirm warning dialog
|
||||
5. Grants microphone permission
|
||||
6. Start recording - see words appear live
|
||||
|
||||
### Limitations
|
||||
- Not available in all browsers (requires Web Speech API)
|
||||
- Accuracy varies by browser
|
||||
- Requires internet connection
|
||||
- May have usage limits
|
||||
|
||||
---
|
||||
|
||||
## Choosing the Right Option
|
||||
|
||||
### For Clinical Use (HIPAA Required)
|
||||
**Use:** Browser Whisper (offline) OR Server (Vertex AI/AWS with BAA)
|
||||
- Browser Whisper: Maximum privacy, no costs
|
||||
- Server: Faster, better for long recordings
|
||||
|
||||
### For Personal Use (Non-HIPAA)
|
||||
**Use:** Any option
|
||||
- Browser Whisper: Best balance of privacy and accuracy
|
||||
- Server: Fastest
|
||||
- Web Speech: Real-time feedback
|
||||
|
||||
### Decision Tree
|
||||
|
||||
```
|
||||
Is this clinical/patient data?
|
||||
├─ YES → Use Browser Whisper or Server (Vertex/AWS)
|
||||
│ ├─ Need offline? → Browser Whisper
|
||||
│ ├─ Need speed? → Server (Vertex AI)
|
||||
│ └─ Want free? → Browser Whisper
|
||||
│
|
||||
└─ NO → Any option
|
||||
├─ Want real-time? → Web Speech API
|
||||
├─ Want privacy? → Browser Whisper
|
||||
└─ Want speed? → Server
|
||||
```
|
||||
|
||||
---
|
||||
| Need | Recommended provider |
|
||||
|---|---|
|
||||
| HIPAA-eligible cloud STT | Google/Gemini through Vertex AI or AWS Transcribe with a BAA. |
|
||||
| OpenAI-compatible routing | LiteLLM with a compliant upstream. |
|
||||
| Direct OpenAI Whisper | Only when acceptable for your deployment. |
|
||||
| Real-time draft preview | Browser Web Speech only with explicit user opt-in and privacy warning. |
|
||||
|
||||
## Configuration
|
||||
|
||||
### Browser Whisper
|
||||
```bash
|
||||
# No configuration needed - bundled in Docker image
|
||||
# Models at: /app/public/models/Xenova/whisper-tiny.en/
|
||||
```env
|
||||
TRANSCRIBE_PROVIDER=litellm
|
||||
LITELLM_API_BASE=https://your-litellm.example/v1
|
||||
LITELLM_API_KEY=<key>
|
||||
LITELLM_STT_MODEL=whisper-1
|
||||
```
|
||||
|
||||
### Server Transcription
|
||||
```bash
|
||||
# .env file
|
||||
TRANSCRIBE_PROVIDER=google # google, aws, openai, litellm
|
||||
Other provider examples:
|
||||
|
||||
# Google Vertex AI
|
||||
```env
|
||||
# Google/Gemini
|
||||
TRANSCRIBE_PROVIDER=google
|
||||
GOOGLE_VERTEX_PROJECT=your-project-id
|
||||
GOOGLE_APPLICATION_CREDENTIALS=/path/to/key.json
|
||||
GOOGLE_STT_MODEL=gemini-2.0-flash
|
||||
|
||||
# AWS Transcribe
|
||||
TRANSCRIBE_PROVIDER=aws
|
||||
AWS_BEDROCK_REGION=us-east-1
|
||||
AWS_ACCESS_KEY_ID=your-key
|
||||
AWS_SECRET_ACCESS_KEY=your-secret
|
||||
AWS_TRANSCRIBE_MEDICAL=true
|
||||
AWS_TRANSCRIBE_SPECIALTY=PRIMARYCARE
|
||||
|
||||
# OpenAI
|
||||
OPENAI_API_KEY=sk-...
|
||||
|
||||
# LiteLLM (proxy)
|
||||
LITELLM_API_BASE=http://localhost:4000
|
||||
LITELLM_API_KEY=optional
|
||||
# Direct OpenAI Whisper
|
||||
TRANSCRIBE_PROVIDER=openai
|
||||
OPENAI_API_KEY=<key>
|
||||
```
|
||||
|
||||
### Web Speech API
|
||||
```bash
|
||||
# No configuration - uses browser built-in
|
||||
# Privacy warning shown in Settings UI
|
||||
```
|
||||
## Failure Handling
|
||||
|
||||
---
|
||||
- Server transcription failures can create encrypted audio backups for retry.
|
||||
- Users can retry or delete failed backups from Settings.
|
||||
- Web Speech interim text is not a substitute for a server transcription response.
|
||||
|
||||
## FAQ
|
||||
## Removed Paths
|
||||
|
||||
### Q: Which is most accurate?
|
||||
**A:** Browser Whisper and Server (Gemini/Whisper) are equally accurate. Web Speech is slightly less accurate.
|
||||
These should remain absent unless the project intentionally reintroduces browser-local STT with a new design review:
|
||||
|
||||
### Q: Which is fastest?
|
||||
**A:** Server transcription (~1s) > Web Speech (real-time) > Browser Whisper (2-10s)
|
||||
|
||||
### Q: Which is most private?
|
||||
**A:** Browser Whisper (100% offline) > Server (with BAA) > Web Speech (not private)
|
||||
|
||||
### Q: Can I use multiple at once?
|
||||
**A:** No. Priority: Web Speech > Browser Whisper > Server (whichever is enabled first)
|
||||
|
||||
### Q: What if transcription fails?
|
||||
**A:** Automatic fallback chain:
|
||||
1. Browser Whisper (if enabled)
|
||||
2. Falls back to Server (if configured)
|
||||
3. Falls back to live transcript (if available)
|
||||
|
||||
### Q: Is Browser Whisper really offline?
|
||||
**A:** Yes! Models are bundled in the Docker image. After the page loads once, transcription works with zero network access.
|
||||
|
||||
### Q: Does Web Speech work offline?
|
||||
**A:** No. It requires internet to send audio to cloud servers.
|
||||
|
||||
### Q: Can I train/customize the models?
|
||||
**A:** No. Browser Whisper uses pre-trained models. Server transcription uses cloud models. No custom training available.
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Browser Whisper stuck at "Initializing"
|
||||
- **Cause:** Models not loaded or network blocked during initial download
|
||||
- **Fix:** See [browser-whisper-troubleshooting.md](browser-whisper-troubleshooting.md)
|
||||
|
||||
### Server transcription returns "No provider"
|
||||
- **Cause:** API keys not configured
|
||||
- **Fix:** Set environment variables in `.env`
|
||||
|
||||
### Web Speech says "Not supported"
|
||||
- **Cause:** Browser doesn't support Web Speech API
|
||||
- **Fix:** Use Chrome, Edge, or Safari
|
||||
|
||||
### Transcription is slow
|
||||
- **Browser Whisper:** Try switching to "Tiny" model
|
||||
- **Server:** Check API provider status
|
||||
- **Web Speech:** Check internet connection
|
||||
|
||||
---
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Clinical Documentation
|
||||
1. Use Browser Whisper for all patient data
|
||||
2. Enable audio backups (automatic in v2)
|
||||
3. Keep recordings under 5 minutes for faster processing
|
||||
4. Use "Tiny" model for quick notes, "Base" for detailed documentation
|
||||
|
||||
### Personal Use
|
||||
1. Web Speech for quick, informal notes
|
||||
2. Browser Whisper for anything you want private
|
||||
3. Server for long recordings
|
||||
|
||||
### Performance Optimization
|
||||
1. Pre-download Browser Whisper model before first use
|
||||
2. Use shorter clips (30-60 seconds) for fastest results
|
||||
3. Clear browser cache if models seem corrupted
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
| Need | Recommendation |
|
||||
|------|---------------|
|
||||
| Clinical/HIPAA | Browser Whisper (offline) |
|
||||
| Fast transcription | Server (Vertex AI) |
|
||||
| Real-time feedback | Web Speech (non-clinical only) |
|
||||
| Maximum privacy | Browser Whisper |
|
||||
| Zero cost | Browser Whisper |
|
||||
| Long recordings | Server (faster for 5+ min clips) |
|
||||
| Offline use | Browser Whisper |
|
||||
|
||||
**Default recommendation:** Browser Whisper for 95% of use cases. It's private, accurate, free, and offline. Only use alternatives when you have specific needs for speed or real-time feedback.
|
||||
- `public/js/browserWhisper.js`
|
||||
- `public/js/whisperWorker.js`
|
||||
- `public/js/whisperWorkerV2.js`
|
||||
- `public/models/Xenova/*`
|
||||
- Browser Whisper setup/troubleshooting docs
|
||||
- Whisper model download scripts for public browser models
|
||||
|
|
|
|||
|
|
@ -27,7 +27,6 @@ const USE_REAL_AI = process.env.E2E_USE_REAL_AI === '1' || process.env.E2E_USE_R
|
|||
// message matches one of these patterns it does NOT fail the test.
|
||||
const CONSOLE_ERROR_ALLOWLIST = [
|
||||
/favicon/i,
|
||||
/Failed to load resource.*models\/Xenova/i, // Browser Whisper models lazy-loaded on demand
|
||||
/\/api\/models/i, // When no AI provider configured yet
|
||||
/Cross-Origin-Opener-Policy/i, // Chrome warning on non-HTTPS e2e server
|
||||
/Failed to load resource.*(400|401|403|404|500|502|503)/i, // Any HTTP error on subsidiary fetches — smoke tests only verify UI renders, deeper integration tests validate endpoint contracts separately
|
||||
|
|
|
|||
42
package-lock.json
generated
42
package-lock.json
generated
|
|
@ -1,12 +1,12 @@
|
|||
{
|
||||
"name": "pediatric-ai-scribe",
|
||||
"version": "7.7.0",
|
||||
"version": "7.10.1",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "pediatric-ai-scribe",
|
||||
"version": "7.7.0",
|
||||
"version": "7.10.1",
|
||||
"dependencies": {
|
||||
"@marp-team/marp-cli": "^4.3.1",
|
||||
"@marp-team/marp-core": "^4.3.0",
|
||||
|
|
@ -38,6 +38,7 @@
|
|||
"pdf-parse": "^1.1.1",
|
||||
"pg": "^8.13.0",
|
||||
"pptxgenjs": "^4.0.1",
|
||||
"prom-client": "^15.1.3",
|
||||
"qrcode": "^1.5.4",
|
||||
"redis": "^4.7.1",
|
||||
"speakeasy": "^2.0.0"
|
||||
|
|
@ -2279,6 +2280,15 @@
|
|||
"node": ">= 0.6"
|
||||
}
|
||||
},
|
||||
"node_modules/@opentelemetry/api": {
|
||||
"version": "1.9.1",
|
||||
"resolved": "https://registry.npmjs.org/@opentelemetry/api/-/api-1.9.1.tgz",
|
||||
"integrity": "sha512-gLyJlPHPZYdAk1JENA9LeHejZe1Ti77/pTeFm/nMXmQH/HFZlcS/O2XJB+L8fkbrNSqhdtlvjBVjxwUYanNH5Q==",
|
||||
"license": "Apache-2.0",
|
||||
"engines": {
|
||||
"node": ">=8.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@phc/format": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/@phc/format/-/format-1.0.0.tgz",
|
||||
|
|
@ -4161,6 +4171,12 @@
|
|||
"node": "*"
|
||||
}
|
||||
},
|
||||
"node_modules/bintrees": {
|
||||
"version": "1.0.2",
|
||||
"resolved": "https://registry.npmjs.org/bintrees/-/bintrees-1.0.2.tgz",
|
||||
"integrity": "sha512-VOMgTMwjAaUG580SXn3LacVgjurrbMme7ZZNYGSSV7mmtY6QQRh0Eg3pwIcntQ77DErK1L0NxkbetjcoXzVwKw==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/bluebird": {
|
||||
"version": "3.4.7",
|
||||
"resolved": "https://registry.npmjs.org/bluebird/-/bluebird-3.4.7.tgz",
|
||||
|
|
@ -7375,6 +7391,19 @@
|
|||
"node": ">=0.4.0"
|
||||
}
|
||||
},
|
||||
"node_modules/prom-client": {
|
||||
"version": "15.1.3",
|
||||
"resolved": "https://registry.npmjs.org/prom-client/-/prom-client-15.1.3.tgz",
|
||||
"integrity": "sha512-6ZiOBfCywsD4k1BN9IX0uZhF+tJkV8q8llP64G5Hajs4JOeVLPCwpPVcpXy3BwYiUGgyJzsJJQeOIv7+hDSq8g==",
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"@opentelemetry/api": "^1.4.0",
|
||||
"tdigest": "^0.1.1"
|
||||
},
|
||||
"engines": {
|
||||
"node": "^16 || ^18 || >=20"
|
||||
}
|
||||
},
|
||||
"node_modules/prosemirror-changeset": {
|
||||
"version": "2.4.0",
|
||||
"resolved": "https://registry.npmjs.org/prosemirror-changeset/-/prosemirror-changeset-2.4.0.tgz",
|
||||
|
|
@ -8469,6 +8498,15 @@
|
|||
"streamx": "^2.15.0"
|
||||
}
|
||||
},
|
||||
"node_modules/tdigest": {
|
||||
"version": "0.1.2",
|
||||
"resolved": "https://registry.npmjs.org/tdigest/-/tdigest-0.1.2.tgz",
|
||||
"integrity": "sha512-+G0LLgjjo9BZX2MfdvPfH+MKLCrxlXSYec5DaPYP1fe6Iyhf0/fSmJ0bFiZ1F8BT6cGXl2LpltQptzjXKWEkKA==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"bintrees": "1.0.2"
|
||||
}
|
||||
},
|
||||
"node_modules/teex": {
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/teex/-/teex-1.0.1.tgz",
|
||||
|
|
|
|||
|
|
@ -46,6 +46,7 @@
|
|||
"pdf-parse": "^1.1.1",
|
||||
"pg": "^8.13.0",
|
||||
"pptxgenjs": "^4.0.1",
|
||||
"prom-client": "^15.1.3",
|
||||
"qrcode": "^1.5.4",
|
||||
"redis": "^4.7.1",
|
||||
"speakeasy": "^2.0.0"
|
||||
|
|
|
|||
|
|
@ -1,88 +0,0 @@
|
|||
// ============================================================
|
||||
// WHISPER WORKER — runs @xenova/transformers in a Web Worker
|
||||
// Audio never leaves the device. Model cached in IndexedDB.
|
||||
// ============================================================
|
||||
|
||||
console.log('[WhisperWorker] Starting worker initialization');
|
||||
|
||||
// Load transformers.js from our server (bundled, no external CDN dependency)
|
||||
importScripts('/models/transformers.min.js');
|
||||
|
||||
var env = self.transformers.env;
|
||||
var pipeline = self.transformers.pipeline;
|
||||
|
||||
// Configure transformers.js to load models from our local server
|
||||
env.allowLocalModels = false;
|
||||
env.useBrowserCache = true;
|
||||
env.allowRemoteModels = false;
|
||||
env.localModelPath = '/models/';
|
||||
|
||||
console.log('[WhisperWorker] Transformers loaded, models path: /models/');
|
||||
|
||||
var _pipe = null;
|
||||
var _loadedModel = null;
|
||||
|
||||
async function load(modelName) {
|
||||
if (_pipe && _loadedModel === modelName) {
|
||||
console.log('[WhisperWorker] Model already loaded:', modelName);
|
||||
return;
|
||||
}
|
||||
console.log('[WhisperWorker] Loading model:', modelName);
|
||||
self.postMessage({ type: 'loading', model: modelName });
|
||||
|
||||
try {
|
||||
_pipe = await pipeline('automatic-speech-recognition', modelName, {
|
||||
progress_callback: function(p) {
|
||||
console.log('[WhisperWorker] Progress:', p.status, p.file, p.progress);
|
||||
if (p.status === 'downloading' || p.status === 'progress') {
|
||||
self.postMessage({ type: 'progress', file: p.file || '', progress: Math.round(p.progress || 0) });
|
||||
}
|
||||
if (p.status === 'done' || p.status === 'ready') {
|
||||
self.postMessage({ type: 'progress', file: p.file || '', progress: 100 });
|
||||
}
|
||||
}
|
||||
});
|
||||
_loadedModel = modelName;
|
||||
console.log('[WhisperWorker] Model loaded successfully');
|
||||
self.postMessage({ type: 'ready', model: modelName });
|
||||
} catch (err) {
|
||||
console.error('[WhisperWorker] Load error:', err);
|
||||
self.postMessage({ type: 'error', message: 'Model load failed: ' + err.message });
|
||||
}
|
||||
}
|
||||
|
||||
self.addEventListener('message', async function(e) {
|
||||
var d = e.data;
|
||||
console.log('[WhisperWorker] Message received:', d.type);
|
||||
|
||||
if (d.type === 'load') {
|
||||
try {
|
||||
await load(d.model || 'Xenova/whisper-tiny.en');
|
||||
} catch(err) {
|
||||
console.error('[WhisperWorker] Load error:', err);
|
||||
self.postMessage({ type: 'error', message: 'Load failed: ' + err.message });
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
if (d.type === 'transcribe') {
|
||||
try {
|
||||
await load(d.model || 'Xenova/whisper-tiny.en');
|
||||
console.log('[WhisperWorker] Starting transcription...');
|
||||
var result = await _pipe(d.audio, {
|
||||
language: 'english',
|
||||
task: 'transcribe',
|
||||
chunk_length_s: 30,
|
||||
stride_length_s: 5,
|
||||
return_timestamps: false
|
||||
});
|
||||
console.log('[WhisperWorker] Transcription complete');
|
||||
self.postMessage({ type: 'result', text: result.text.trim() });
|
||||
} catch(err) {
|
||||
console.error('[WhisperWorker] Transcribe error:', err);
|
||||
self.postMessage({ type: 'error', message: 'Transcription failed: ' + err.message });
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
console.log('[WhisperWorker] Worker initialized');
|
||||
|
|
@ -1,135 +0,0 @@
|
|||
// ============================================================
|
||||
// WHISPER WORKER V2 — Alternative loading using dynamic import
|
||||
// Loads transformers.js via script tag method instead of importScripts
|
||||
// ============================================================
|
||||
|
||||
console.log('[WhisperWorker] V2 Starting initialization');
|
||||
|
||||
// Alternative loading method that works with CSP
|
||||
var TRANSFORMERS_URL = 'https://cdn.jsdelivr.net/npm/@xenova/transformers@2.17.2/dist/transformers.min.js';
|
||||
|
||||
var _transformers = null;
|
||||
var _pipe = null;
|
||||
var _loadedModel = null;
|
||||
var _loading = false;
|
||||
|
||||
// Load transformers library using alternative method
|
||||
async function initTransformers() {
|
||||
if (_transformers) return _transformers;
|
||||
if (_loading) {
|
||||
// Wait for current load to complete
|
||||
while (_loading) {
|
||||
await new Promise(resolve => setTimeout(resolve, 100));
|
||||
}
|
||||
return _transformers;
|
||||
}
|
||||
|
||||
_loading = true;
|
||||
console.log('[WhisperWorker] V2 Loading transformers via importScripts fallback...');
|
||||
|
||||
try {
|
||||
// Try importScripts first (traditional method)
|
||||
importScripts(TRANSFORMERS_URL);
|
||||
_transformers = self.transformers || transformers;
|
||||
console.log('[WhisperWorker] V2 Loaded via importScripts');
|
||||
} catch (err) {
|
||||
console.error('[WhisperWorker] V2 importScripts failed, trying alternative:', err.message);
|
||||
|
||||
// Fallback: Notify main thread to use server-side transcription
|
||||
self.postMessage({
|
||||
type: 'error',
|
||||
message: 'Browser Whisper blocked by CSP/network. Using server transcription instead.'
|
||||
});
|
||||
_loading = false;
|
||||
throw new Error('CDN blocked - use server transcription');
|
||||
}
|
||||
|
||||
if (!_transformers) {
|
||||
_loading = false;
|
||||
throw new Error('Transformers library did not load');
|
||||
}
|
||||
|
||||
_transformers.env.allowLocalModels = false;
|
||||
_transformers.env.useBrowserCache = true;
|
||||
_loading = false;
|
||||
|
||||
console.log('[WhisperWorker] V2 Transformers ready');
|
||||
return _transformers;
|
||||
}
|
||||
|
||||
async function loadModel(modelName) {
|
||||
if (_pipe && _loadedModel === modelName) {
|
||||
console.log('[WhisperWorker] V2 Model already loaded:', modelName);
|
||||
return;
|
||||
}
|
||||
|
||||
console.log('[WhisperWorker] V2 Loading model:', modelName);
|
||||
self.postMessage({ type: 'loading', model: modelName });
|
||||
|
||||
try {
|
||||
var T = await initTransformers();
|
||||
|
||||
_pipe = await T.pipeline('automatic-speech-recognition', modelName, {
|
||||
progress_callback: function(p) {
|
||||
console.log('[WhisperWorker] V2 Progress:', p.status, p.file, p.progress);
|
||||
if (p.status === 'downloading' || p.status === 'progress') {
|
||||
self.postMessage({
|
||||
type: 'progress',
|
||||
file: p.file || '',
|
||||
progress: Math.round(p.progress || 0)
|
||||
});
|
||||
}
|
||||
if (p.status === 'done' || p.status === 'ready') {
|
||||
self.postMessage({ type: 'progress', file: p.file || '', progress: 100 });
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
_loadedModel = modelName;
|
||||
console.log('[WhisperWorker] V2 Model loaded successfully');
|
||||
self.postMessage({ type: 'ready', model: modelName });
|
||||
} catch (err) {
|
||||
console.error('[WhisperWorker] V2 Load error:', err);
|
||||
self.postMessage({
|
||||
type: 'error',
|
||||
message: 'Model load failed: ' + err.message + '. Try server transcription instead.'
|
||||
});
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
self.addEventListener('message', async function(e) {
|
||||
var d = e.data;
|
||||
console.log('[WhisperWorker] V2 Message received:', d.type);
|
||||
|
||||
try {
|
||||
if (d.type === 'load') {
|
||||
await loadModel(d.model || 'Xenova/whisper-tiny.en');
|
||||
return;
|
||||
}
|
||||
|
||||
if (d.type === 'transcribe') {
|
||||
await loadModel(d.model || 'Xenova/whisper-tiny.en');
|
||||
console.log('[WhisperWorker] V2 Starting transcription...');
|
||||
|
||||
var result = await _pipe(d.audio, {
|
||||
language: 'english',
|
||||
task: 'transcribe',
|
||||
chunk_length_s: 30,
|
||||
stride_length_s: 5,
|
||||
return_timestamps: false
|
||||
});
|
||||
|
||||
console.log('[WhisperWorker] V2 Transcription complete');
|
||||
self.postMessage({ type: 'result', text: result.text.trim() });
|
||||
}
|
||||
} catch (err) {
|
||||
console.error('[WhisperWorker] V2 Error:', err);
|
||||
self.postMessage({
|
||||
type: 'error',
|
||||
message: err.message || 'Worker error'
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
console.log('[WhisperWorker] V2 Worker initialized');
|
||||
|
|
@ -1,69 +0,0 @@
|
|||
#!/bin/bash
|
||||
# Download Browser Whisper models for local development
|
||||
# These are bundled during Docker build, but need manual download for dev
|
||||
|
||||
set -e
|
||||
|
||||
cd "$(dirname "$0")/.."
|
||||
MODELS_DIR="public/models"
|
||||
|
||||
echo "🎙️ Downloading Browser Whisper models..."
|
||||
echo ""
|
||||
|
||||
# Create directories
|
||||
mkdir -p "$MODELS_DIR/Xenova/whisper-tiny.en/onnx"
|
||||
|
||||
# Download transformers.js (worker-compatible build)
|
||||
echo "📦 Downloading transformers.js..."
|
||||
curl -L --progress-bar -o "$MODELS_DIR/transformers.min.js" \
|
||||
"https://cdn.jsdelivr.net/npm/@xenova/transformers@2.0.0/dist/transformers.min.js"
|
||||
echo "✅ transformers.min.js ($(du -h $MODELS_DIR/transformers.min.js | cut -f1))"
|
||||
echo ""
|
||||
|
||||
# Download model config files
|
||||
echo "📝 Downloading model configs..."
|
||||
cd "$MODELS_DIR/Xenova/whisper-tiny.en"
|
||||
|
||||
curl -sL -o config.json \
|
||||
"https://huggingface.co/Xenova/whisper-tiny.en/resolve/main/config.json"
|
||||
echo "✅ config.json ($(du -h config.json | cut -f1))"
|
||||
|
||||
curl -sL -o tokenizer.json \
|
||||
"https://huggingface.co/Xenova/whisper-tiny.en/resolve/main/tokenizer.json"
|
||||
echo "✅ tokenizer.json ($(du -h tokenizer.json | cut -f1))"
|
||||
|
||||
curl -sL -o preprocessor_config.json \
|
||||
"https://huggingface.co/Xenova/whisper-tiny.en/resolve/main/preprocessor_config.json"
|
||||
echo "✅ preprocessor_config.json ($(du -h preprocessor_config.json | cut -f1))"
|
||||
|
||||
curl -sL -o generation_config.json \
|
||||
"https://huggingface.co/Xenova/whisper-tiny.en/resolve/main/generation_config.json"
|
||||
echo "✅ generation_config.json ($(du -h generation_config.json | cut -f1))"
|
||||
echo ""
|
||||
|
||||
# Download ONNX models (large files)
|
||||
echo "🧠 Downloading encoder model..."
|
||||
curl -L --progress-bar -o onnx/encoder_model_quantized.onnx \
|
||||
"https://huggingface.co/Xenova/whisper-tiny.en/resolve/main/onnx/encoder_model_quantized.onnx"
|
||||
echo "✅ encoder_model_quantized.onnx ($(du -h onnx/encoder_model_quantized.onnx | cut -f1))"
|
||||
echo ""
|
||||
|
||||
echo "🧠 Downloading decoder model..."
|
||||
curl -L --progress-bar -o onnx/decoder_model_merged_quantized.onnx \
|
||||
"https://huggingface.co/Xenova/whisper-tiny.en/resolve/main/onnx/decoder_model_merged_quantized.onnx"
|
||||
echo "✅ decoder_model_merged_quantized.onnx ($(du -h onnx/decoder_model_merged_quantized.onnx | cut -f1))"
|
||||
echo ""
|
||||
|
||||
# Show summary
|
||||
cd - > /dev/null
|
||||
echo "════════════════════════════════════════"
|
||||
echo "✅ Browser Whisper models downloaded!"
|
||||
echo "════════════════════════════════════════"
|
||||
echo "Total size: $(du -sh $MODELS_DIR | cut -f1)"
|
||||
echo ""
|
||||
echo "Files:"
|
||||
ls -lh "$MODELS_DIR/Xenova/whisper-tiny.en/" | tail -n +2
|
||||
echo ""
|
||||
ls -lh "$MODELS_DIR/Xenova/whisper-tiny.en/onnx/" | tail -n +2
|
||||
echo ""
|
||||
echo "Models are ready for use. Start server with: npm start"
|
||||
|
|
@ -108,7 +108,6 @@ const DYNAMIC_ID_PREFIXES = [
|
|||
'cms-', // content manager
|
||||
'fpa-', // forgot-password accept screen
|
||||
'rsp-', // reset password
|
||||
'browser-whisper-', // whisper settings
|
||||
'admin-', // admin panel
|
||||
'adminms-', // admin milestones
|
||||
'2fa-', // 2FA UI
|
||||
|
|
|
|||
|
|
@ -8,6 +8,7 @@ const cookieParser = require('cookie-parser');
|
|||
const path = require('path');
|
||||
const http = require('http');
|
||||
const loggingMiddleware = require('./src/middleware/logging');
|
||||
const { metricsHandler, metricsMiddleware } = require('./src/utils/metrics');
|
||||
|
||||
const app = express();
|
||||
const server = http.createServer(app);
|
||||
|
|
@ -72,6 +73,7 @@ app.use('/api', cors({
|
|||
}));
|
||||
|
||||
app.use(cookieParser());
|
||||
app.use(metricsMiddleware);
|
||||
|
||||
// ============================================================
|
||||
// BODY LIMITS — 10mb for JSON (chart review with many notes can be large),
|
||||
|
|
@ -198,6 +200,7 @@ app.get(['/', '/index.html'], function(req, res) {
|
|||
|
||||
// Public endpoint for cache-bust debugging + build-info
|
||||
app.get('/api/build', function(req, res) { res.json({ buildId: BUILD_ID }); });
|
||||
app.get('/metrics', metricsHandler);
|
||||
|
||||
app.use(loggingMiddleware);
|
||||
app.use('/vendor/markdown-it', express.static(path.join(__dirname, 'node_modules', 'markdown-it', 'dist'), {
|
||||
|
|
|
|||
|
|
@ -106,7 +106,7 @@ router.post('/documents/upload', upload.single('file'), async function(req, res)
|
|||
);
|
||||
|
||||
res.json({ success: true, id: result.lastInsertRowid, filename: req.file.originalname });
|
||||
logger.audit(req.user.id, 'document_upload', 'Uploaded: ' + (req.file ? req.file.originalname : 'unknown'), req, { category: 'documents' });
|
||||
logger.audit(req.user.id, 'document_upload', 'Uploaded document id:' + result.lastInsertRowid, req, { category: 'documents' });
|
||||
} catch (e) { logger.error('POST /documents/upload', e.message); res.status(500).json({ error: 'Upload failed' }); }
|
||||
});
|
||||
|
||||
|
|
|
|||
|
|
@ -61,7 +61,7 @@ router.get('/encounters/saved/:id', async function(req, res) {
|
|||
try { row.transcript = cryptoUtil.decryptString(row.transcript); } catch (e) {}
|
||||
try { row.generated_note = cryptoUtil.decryptString(row.generated_note); } catch (e) {}
|
||||
try { row.partial_data = cryptoUtil.decryptString(row.partial_data); } catch (e) {}
|
||||
logger.audit(req.user.id, 'encounter_load', 'Loaded encounter: ' + (row.label || 'unlabeled') + ' (id:' + req.params.id + ')', req, { category: 'clinical' });
|
||||
logger.audit(req.user.id, 'encounter_load', 'Loaded encounter id:' + req.params.id, req, { category: 'clinical' });
|
||||
res.json({ success: true, encounter: row });
|
||||
} catch (e) { logger.error('GET /encounters/saved/:id', e.message); res.status(500).json({ error: 'Request failed' }); }
|
||||
});
|
||||
|
|
@ -103,7 +103,7 @@ router.post('/encounters/saved', async function(req, res) {
|
|||
return res.status(409).json({ error: 'Encounter changed under you. Reload and retry.' });
|
||||
}
|
||||
res.json({ success: true, id: id, version: newVersion });
|
||||
logger.audit(req.user.id, 'encounter_save', 'Saved encounter: ' + (label || 'unlabeled'), req, { category: 'clinical' });
|
||||
logger.audit(req.user.id, 'encounter_save', 'Saved encounter id:' + id, req, { category: 'clinical' });
|
||||
} else {
|
||||
// Enforce unique label per user (within active/non-expired encounters)
|
||||
if (label && label.trim()) {
|
||||
|
|
@ -134,7 +134,7 @@ router.post('/encounters/saved', async function(req, res) {
|
|||
dup.id, req.user.id
|
||||
]
|
||||
);
|
||||
logger.audit(req.user.id, 'encounter_save', 'Saved encounter: ' + (label || 'unlabeled'), req, { category: 'clinical' });
|
||||
logger.audit(req.user.id, 'encounter_save', 'Saved encounter id:' + dup.id, req, { category: 'clinical' });
|
||||
return res.json({ success: true, id: dup.id });
|
||||
}
|
||||
}
|
||||
|
|
@ -154,7 +154,7 @@ router.post('/encounters/saved', async function(req, res) {
|
|||
]
|
||||
);
|
||||
res.json({ success: true, id: result.lastInsertRowid });
|
||||
logger.audit(req.user.id, 'encounter_save', 'Saved encounter: ' + (label || 'unlabeled'), req, { category: 'clinical' });
|
||||
logger.audit(req.user.id, 'encounter_save', 'Saved encounter id:' + result.lastInsertRowid, req, { category: 'clinical' });
|
||||
}
|
||||
} catch (e) { logger.error('POST /encounters/saved', e.message); res.status(500).json({ error: 'Request failed' }); }
|
||||
});
|
||||
|
|
|
|||
|
|
@ -11,6 +11,7 @@ var { callAI } = require('../utils/ai');
|
|||
var { authMiddleware, moderatorMiddleware } = require('../middleware/auth');
|
||||
var db = require('../db/database');
|
||||
var cryptoUtil = require('../utils/crypto');
|
||||
var { assertSafeHttpsUrl } = require('../utils/urlSafety');
|
||||
|
||||
router.use(authMiddleware);
|
||||
router.use(moderatorMiddleware);
|
||||
|
|
@ -275,7 +276,7 @@ router.post('/ai-generate', upload.array('files', 10), async function(req, res)
|
|||
allTexts.push('### Source File: ' + file.originalname + '\n\n' + text);
|
||||
fileCount++;
|
||||
} catch (e) {
|
||||
console.error('[LearningAI] Failed to extract text from ' + file.originalname + ':', e.message);
|
||||
console.error('[LearningAI] Failed to extract uploaded file:', e.message);
|
||||
// Continue with other files even if one fails
|
||||
}
|
||||
}
|
||||
|
|
@ -291,14 +292,17 @@ router.post('/ai-generate', upload.array('files', 10), async function(req, res)
|
|||
if (!user || !user.nextcloud_url) {
|
||||
return res.status(400).json({ error: 'Nextcloud not connected. Go to Settings first.' });
|
||||
}
|
||||
await assertSafeHttpsUrl(user.nextcloud_url, 'Nextcloud URL');
|
||||
var ncPassword;
|
||||
try { ncPassword = cryptoUtil.decryptString(user.nextcloud_token); }
|
||||
catch (decErr) { return res.status(400).json({ error: 'Nextcloud credentials invalid. Please reconnect.' }); }
|
||||
var fileUrl = user.nextcloud_url + '/remote.php/dav/files/' + user.nextcloud_user + webdavPath;
|
||||
if (!webdavPath.startsWith('/')) webdavPath = '/' + webdavPath;
|
||||
var fileUrl = user.nextcloud_url + '/remote.php/dav/files/' + encodeURIComponent(user.nextcloud_user) + webdavPath;
|
||||
var response = await axios.get(fileUrl, {
|
||||
auth: { username: user.nextcloud_user, password: ncPassword },
|
||||
responseType: 'arraybuffer',
|
||||
timeout: 30000
|
||||
timeout: 30000,
|
||||
maxRedirects: 0
|
||||
});
|
||||
var mimeType = response.headers['content-type'] || 'text/plain';
|
||||
var fname = path.basename(webdavPath);
|
||||
|
|
@ -375,7 +379,7 @@ router.post('/ai-generate', upload.array('files', 10), async function(req, res)
|
|||
try {
|
||||
parsed = JSON.parse(raw);
|
||||
} catch (e) {
|
||||
console.error('[LearningAI] Direct parse failed:', e.message, '| Pos:', e.message.match(/position (\d+)/)?.[1], '| Char context:', raw.substring(parseInt(e.message.match(/position (\d+)/)?.[1] || 0) - 40, parseInt(e.message.match(/position (\d+)/)?.[1] || 0) + 40));
|
||||
console.error('[LearningAI] Direct parse failed:', e.message, '| Pos:', e.message.match(/position (\d+)/)?.[1]);
|
||||
// Attempt 2: sanitize control chars and retry
|
||||
try { parsed = JSON.parse(sanitizeJsonString(raw)); }
|
||||
catch (e2) {
|
||||
|
|
@ -451,6 +455,7 @@ router.get('/webdav-browse', async function(req, res) {
|
|||
if (!user || !user.nextcloud_url) {
|
||||
return res.status(400).json({ error: 'Nextcloud not connected' });
|
||||
}
|
||||
await assertSafeHttpsUrl(user.nextcloud_url, 'Nextcloud URL');
|
||||
|
||||
var ncPassword;
|
||||
try { ncPassword = cryptoUtil.decryptString(user.nextcloud_token); }
|
||||
|
|
@ -460,7 +465,7 @@ router.get('/webdav-browse', async function(req, res) {
|
|||
// Ensure it starts with /
|
||||
if (!browsePath.startsWith('/')) browsePath = '/' + browsePath;
|
||||
|
||||
var davUrl = user.nextcloud_url + '/remote.php/dav/files/' + user.nextcloud_user + browsePath;
|
||||
var davUrl = user.nextcloud_url + '/remote.php/dav/files/' + encodeURIComponent(user.nextcloud_user) + browsePath;
|
||||
|
||||
var davResponse = await axios({
|
||||
method: 'PROPFIND',
|
||||
|
|
@ -468,7 +473,8 @@ router.get('/webdav-browse', async function(req, res) {
|
|||
auth: { username: user.nextcloud_user, password: ncPassword },
|
||||
headers: { Depth: '1', 'Content-Type': 'application/xml' },
|
||||
data: `<?xml version="1.0"?><d:propfind xmlns:d="DAV:"><d:prop><d:displayname/><d:getcontenttype/><d:getcontentlength/><d:resourcetype/></d:prop></d:propfind>`,
|
||||
timeout: 15000
|
||||
timeout: 15000,
|
||||
maxRedirects: 0
|
||||
});
|
||||
|
||||
// Parse WebDAV XML response
|
||||
|
|
|
|||
|
|
@ -2,10 +2,23 @@ var express = require('express');
|
|||
var router = express.Router();
|
||||
var { authMiddleware } = require('../middleware/auth');
|
||||
var logger = require('../utils/logger');
|
||||
var { redact } = require('../utils/redact');
|
||||
|
||||
function clampLimit(value, fallback, max) {
|
||||
var n = parseInt(value, 10);
|
||||
if (!Number.isFinite(n)) n = fallback;
|
||||
return Math.min(Math.max(n, 1), max);
|
||||
}
|
||||
|
||||
function trimField(value, max) {
|
||||
if (value == null) return '';
|
||||
var s = String(value);
|
||||
return s.length > max ? s.slice(0, max) + '…' : s;
|
||||
}
|
||||
|
||||
router.get('/logs/usage', authMiddleware, async function(req, res) {
|
||||
try {
|
||||
var days = parseInt(req.query.days) || 30;
|
||||
var days = clampLimit(req.query.days, 30, 365);
|
||||
var summary = await logger.getUsageSummary(req.user.id, days);
|
||||
res.json({ success: true, summary: summary, period: days + ' days' });
|
||||
} catch (err) { res.status(500).json({ error: 'Request failed' }); }
|
||||
|
|
@ -13,21 +26,21 @@ router.get('/logs/usage', authMiddleware, async function(req, res) {
|
|||
|
||||
router.get('/logs/audit', authMiddleware, async function(req, res) {
|
||||
try {
|
||||
var logs = await logger.getAuditLogs(req.user.id, parseInt(req.query.limit) || 50);
|
||||
var logs = await logger.getAuditLogs(req.user.id, clampLimit(req.query.limit, 50, 200));
|
||||
res.json({ success: true, logs: logs });
|
||||
} catch (err) { res.status(500).json({ error: 'Request failed' }); }
|
||||
});
|
||||
|
||||
router.get('/logs/api', authMiddleware, async function(req, res) {
|
||||
try {
|
||||
var logs = await logger.getApiLogs(req.user.id, parseInt(req.query.limit) || 50);
|
||||
var logs = await logger.getApiLogs(req.user.id, clampLimit(req.query.limit, 50, 200));
|
||||
res.json({ success: true, logs: logs });
|
||||
} catch (err) { res.status(500).json({ error: 'Request failed' }); }
|
||||
});
|
||||
|
||||
router.get('/logs/access', authMiddleware, async function(req, res) {
|
||||
try {
|
||||
var logs = await logger.getAccessLogs(req.user.id, parseInt(req.query.limit) || 50);
|
||||
var logs = await logger.getAccessLogs(req.user.id, clampLimit(req.query.limit, 50, 200));
|
||||
res.json({ success: true, logs: logs });
|
||||
} catch (err) { res.status(500).json({ error: 'Request failed' }); }
|
||||
});
|
||||
|
|
@ -36,17 +49,17 @@ router.get('/logs/access', authMiddleware, async function(req, res) {
|
|||
router.post('/logs/client-error', function(req, res) {
|
||||
try {
|
||||
var e = req.body || {};
|
||||
console.error('[CLIENT ERROR]', JSON.stringify({
|
||||
type: e.type,
|
||||
message: e.message,
|
||||
source: e.source,
|
||||
logger.warn('client_error', {
|
||||
type: trimField(e.type, 80),
|
||||
message: redact(trimField(e.message, 500)),
|
||||
source: redact(trimField(e.source, 300)),
|
||||
line: e.line,
|
||||
col: e.col,
|
||||
stack: e.stack,
|
||||
stack: redact(trimField(e.stack, 1200)),
|
||||
ip: req.ip,
|
||||
ua: req.headers['user-agent'],
|
||||
ua: trimField(req.headers['user-agent'], 255),
|
||||
time: new Date().toISOString()
|
||||
}));
|
||||
});
|
||||
res.status(204).end();
|
||||
} catch(err) { res.status(204).end(); }
|
||||
});
|
||||
|
|
|
|||
|
|
@ -6,6 +6,11 @@ var db = require('../db/database');
|
|||
var logger = require('../utils/logger');
|
||||
var cryptoUtil = require('../utils/crypto');
|
||||
var { serverError } = require('../utils/errors');
|
||||
var { assertSafeHttpsUrl } = require('../utils/urlSafety');
|
||||
|
||||
function davRoot(baseUrl, username) {
|
||||
return baseUrl + '/remote.php/dav/files/' + encodeURIComponent(username);
|
||||
}
|
||||
|
||||
router.post('/nextcloud/connect', authMiddleware, async function(req, res) {
|
||||
try {
|
||||
|
|
@ -14,9 +19,10 @@ router.post('/nextcloud/connect', authMiddleware, async function(req, res) {
|
|||
|
||||
var cleanUrl = nextcloudUrl.replace(/\/+$/, '');
|
||||
var targetFolder = (folder || '/PediatricScribe').replace(/\/+$/, '');
|
||||
await assertSafeHttpsUrl(cleanUrl, 'Nextcloud URL');
|
||||
|
||||
try {
|
||||
await axios({ method: 'PROPFIND', url: cleanUrl + '/remote.php/dav/files/' + username + '/', auth: { username: username, password: appPassword }, headers: { Depth: '0' } });
|
||||
await axios({ method: 'PROPFIND', url: davRoot(cleanUrl, username) + '/', auth: { username: username, password: appPassword }, headers: { Depth: '0' }, timeout: 15000, maxRedirects: 0 });
|
||||
} catch (e) {
|
||||
return res.status(400).json({ error: 'Cannot connect. Check credentials.' });
|
||||
}
|
||||
|
|
@ -25,7 +31,7 @@ router.post('/nextcloud/connect', authMiddleware, async function(req, res) {
|
|||
var current = '';
|
||||
for (var i = 0; i < parts.length; i++) {
|
||||
current += '/' + parts[i];
|
||||
try { await axios({ method: 'MKCOL', url: cleanUrl + '/remote.php/dav/files/' + username + current + '/', auth: { username: username, password: appPassword } }); } catch (e) {}
|
||||
try { await axios({ method: 'MKCOL', url: davRoot(cleanUrl, username) + current + '/', auth: { username: username, password: appPassword }, timeout: 15000, maxRedirects: 0 }); } catch (e) {}
|
||||
}
|
||||
|
||||
await db.run('UPDATE users SET nextcloud_url = ?, nextcloud_user = ?, nextcloud_token = ?, nextcloud_folder = ? WHERE id = ?',
|
||||
|
|
@ -41,6 +47,7 @@ router.post('/nextcloud/export', authMiddleware, async function(req, res) {
|
|||
var { content, filename, type } = req.body;
|
||||
var user = await db.get('SELECT nextcloud_url, nextcloud_user, nextcloud_token, nextcloud_folder FROM users WHERE id = ?', [req.user.id]);
|
||||
if (!user || !user.nextcloud_url) return res.status(400).json({ error: 'Nextcloud not connected. Go to Settings.' });
|
||||
await assertSafeHttpsUrl(user.nextcloud_url, 'Nextcloud URL');
|
||||
|
||||
var baseFolder = user.nextcloud_folder || '/PediatricScribe';
|
||||
var today = new Date().toISOString().split('T')[0];
|
||||
|
|
@ -54,14 +61,14 @@ router.post('/nextcloud/export', authMiddleware, async function(req, res) {
|
|||
var current = '';
|
||||
for (var i = 0; i < parts.length; i++) {
|
||||
current += '/' + parts[i];
|
||||
try { await axios({ method: 'MKCOL', url: user.nextcloud_url + '/remote.php/dav/files/' + user.nextcloud_user + current + '/', auth: { username: user.nextcloud_user, password: pw } }); } catch (e) {}
|
||||
try { await axios({ method: 'MKCOL', url: davRoot(user.nextcloud_url, user.nextcloud_user) + current + '/', auth: { username: user.nextcloud_user, password: pw }, timeout: 15000, maxRedirects: 0 }); } catch (e) {}
|
||||
}
|
||||
|
||||
var time = new Date().toTimeString().split(' ')[0].replace(/:/g, '-');
|
||||
var safeName = (filename || type + '-' + time).replace(/[^a-zA-Z0-9-_]/g, '_');
|
||||
var filePath = user.nextcloud_url + '/remote.php/dav/files/' + user.nextcloud_user + targetPath + '/' + safeName + '.txt';
|
||||
var filePath = davRoot(user.nextcloud_url, user.nextcloud_user) + targetPath + '/' + safeName + '.txt';
|
||||
|
||||
await axios({ method: 'PUT', url: filePath, data: content, auth: { username: user.nextcloud_user, password: pw }, headers: { 'Content-Type': 'text/plain; charset=utf-8' } });
|
||||
await axios({ method: 'PUT', url: filePath, data: content, auth: { username: user.nextcloud_user, password: pw }, headers: { 'Content-Type': 'text/plain; charset=utf-8' }, timeout: 30000, maxRedirects: 0 });
|
||||
|
||||
// Migrate legacy plaintext token to encrypted form on first successful use.
|
||||
if (!cryptoUtil.isEncrypted(user.nextcloud_token)) {
|
||||
|
|
|
|||
|
|
@ -6,8 +6,9 @@
|
|||
|
||||
function serverError(res, scope, err, publicMessage) {
|
||||
try {
|
||||
var logger = require('./logger');
|
||||
var msg = err && err.message ? err.message : String(err);
|
||||
console.error('[' + scope + ']', msg, err && err.stack ? '\n' + err.stack : '');
|
||||
logger.error('[' + scope + '] ' + msg, err && err.stack ? err.stack : '');
|
||||
} catch (e) {}
|
||||
return res.status(500).json({ error: publicMessage || 'Request failed. Please try again.' });
|
||||
}
|
||||
|
|
|
|||
|
|
@ -11,6 +11,14 @@ function redact(text) {
|
|||
if (text == null) return text;
|
||||
var s = typeof text === 'string' ? text : JSON.stringify(text);
|
||||
|
||||
// Credentials and tokens. These patterns intentionally run before PHI
|
||||
// redaction so accidental pasted headers/URLs are removed from logs.
|
||||
s = s.replace(/Authorization\s*:\s*Bearer\s+[A-Za-z0-9._~+/=-]+/gi, 'Authorization: Bearer [REDACTED]');
|
||||
s = s.replace(/Bearer\s+[A-Za-z0-9._~+/=-]{20,}/gi, 'Bearer [REDACTED]');
|
||||
s = s.replace(/\b[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\b/g, '[JWT]');
|
||||
s = s.replace(/\b(api[_-]?key|password|secret|token|app[_-]?password)(\s*[=:]\s*)([^\s&"']+)/gi, '$1$2[REDACTED]');
|
||||
s = s.replace(/(https?:\/\/)([^\s/@:]+):([^\s/@]+)@/gi, '$1[REDACTED]@');
|
||||
|
||||
// SSN
|
||||
s = s.replace(/\b\d{3}-\d{2}-\d{4}\b/g, '[SSN]');
|
||||
// US phone numbers
|
||||
|
|
|
|||
|
|
@ -41,6 +41,11 @@ test('note refine corrections still call AI without storing learning memories',
|
|||
|
||||
test('browser Whisper is removed from public runtime and user settings', () => {
|
||||
assert.equal(fs.existsSync(path.join(root, 'public/js/browserWhisper.js')), false);
|
||||
assert.equal(fs.existsSync(path.join(root, 'public/js/whisperWorker.js')), false);
|
||||
assert.equal(fs.existsSync(path.join(root, 'public/js/whisperWorkerV2.js')), false);
|
||||
assert.equal(fs.existsSync(path.join(root, 'docs/browser-whisper-setup.md')), false);
|
||||
assert.equal(fs.existsSync(path.join(root, 'docs/browser-whisper-troubleshooting.md')), false);
|
||||
assert.equal(fs.existsSync(path.join(root, 'scripts/download-whisper-models.sh')), false);
|
||||
const publicRuntimeFiles = [
|
||||
'public/index.html',
|
||||
'public/js/app.js',
|
||||
|
|
|
|||
Loading…
Reference in a new issue