503f5afaad
Three concurrent themes from this session:
═══════════════════════════════════════════════════════════════════
ED ENCOUNTERS — per-stage cards + consolidate→MDM finalize
═══════════════════════════════════════════════════════════════════
UX redesign per Daniel's feedback ("every stage note should be shown,
if AI is told to modify that particular note then the modified version
is used in final mdm"):
- Each generated stage stays on screen as its own editable card with
its own embedded "Don't Miss" panel. No more single rolling note
element that gets replaced on each generation.
- gatherCurrentNotes() reads contenteditable text from each stage card
before any operation (advance, finalize, persist) so inline edits
flow into the next AI call and the final consolidate.
- Stage badge is now state-accurate. "Stage N (recording)" with yellow
background after Add-more before generation; "Stage N" with gray
after generation. Fixes the bug where the badge flipped to Stage 2
the moment Add-more was clicked.
- Save & Done now runs TWO server-side AI calls in /finalize:
1. edConsolidate (new prompt) → polished single final note that
integrates every stage chronologically (HPI / ROS / PE / ED Course /
A&P with disposition).
2. edFinalize (rewritten with full inline 2023 AMA E/M element
rubric — problems / data / risk definitions, level mapping with
concrete examples) → MDM JSON.
- Two new cards render after finalize: blue-bordered Final Consolidated
Note + green-bordered MDM. Stage cards become read-only.
- partial_data on the saved row now stores {stages, finalNote, mdm,
finalized} so resume re-renders the full state.
Why two-call finalize: a single combined prompt makes the model cut
corners on one task. Two focused calls cost ~2× latency at the very end
of an encounter — acceptable since finalize is a one-time terminal
action, not a per-stage hot path.
Files: public/components/ed-encounter.html, public/js/ed-encounters.js,
src/routes/edEncounters.js, src/utils/prompts.js (edConsolidate added,
edFinalize rewritten).
═══════════════════════════════════════════════════════════════════
EXTENSIONS / PAGERS — visual polish
═══════════════════════════════════════════════════════════════════
Multiple iterations based on Daniel's feedback:
- Layout: align-items:flex-start so action buttons stay pinned top-right
when long numbers wrap (was align-items:center → buttons drifted into
the text area, causing visible overlap).
- Number: word-break:break-all + min-width:0 + font-feature-settings:tnum
so long numbers wrap within their column instead of pushing under the
buttons. Click-to-copy with a 0.55s green flash + ✓ copied badge.
- Phone/pager Font Awesome icon next to the number in the type color —
at-a-glance type signal (replacing an earlier 3px left stripe that
Daniel found visually bulky).
- Name: font-weight 700, font-size 14.5px, color g900, letter-spacing
-0.012em — scan-target headline typography for long lists.
- Alternating subtle backgrounds by index (white vs #fafbfc) so a long
list reads as distinct rows.
- Hover: card lifts 1px with a soft shadow; action buttons fade from
55% to 100% opacity. Cubic-bezier transition on transform.
- Entrance: staggered fade-up animation per card (35ms × index, capped
at 12). prefers-reduced-motion media query disables motion.
- Empty state: 48px FA icon + heading instead of plain gray text.
Files: public/js/extensions.js, public/css/styles.css.
═══════════════════════════════════════════════════════════════════
DOCS REORGANIZATION + APPLICATION-LOGIC DOCS + ADMIN VIEWER
═══════════════════════════════════════════════════════════════════
Document moves (preserving git history via git mv):
BROWSER_WHISPER_SETUP.md → docs/browser-whisper-setup.md
BROWSER_WHISPER_TROUBLESHOOTING.md → docs/browser-whisper-troubleshooting.md
DEVELOPER_GUIDE.md → docs/developer-guide-extended.md
EMBEDDINGS_SETUP.md → docs/embeddings-setup.md
FEATURES_EXPLAINED.md → docs/features-explained.md
IMPROVEMENTS.md → docs/improvements.md
OPENID_SETUP.md → docs/openid-setup.md
TRANSCRIPTION_OPTIONS.md → docs/transcription-options.md
README.md updated with the new paths + a Documentation section that
links to docs/logic/ at the top.
New application-logic doc series (~8,300 lines total) at docs/logic/.
Built with 5 parallel doc-writing agents per Daniel's "use multiple
agents" directive. Each doc explains how a part of the app actually
works — application logic, data flow, design decisions, sacred zones,
how-to-extend recipes — at a depth that lets a new dev (or an AI
assistant) modify the code confidently.
docs/logic/README.md — index + recommended reading order
docs/logic/architecture.md (2166 L) — frontend IIFE pattern, lazy tab
load, backend route convention,
schema, encryption, deployment
docs/logic/clinical-notes.md (1546L) — every note tab + helper trio
docs/logic/bedside-and-calculators.md (1373L) — bedside ES module
pocket + calculators + PE Guide
+ suture selector
docs/logic/auth-admin-learning.md (1281L) — auth (local+OIDC+2FA) +
admin panel + Learning Hub
(Quiz engine logic at sub-detail
only — TODO follow-up)
docs/logic/ai-and-voice.md (1128 L) — callAI 5-provider routing,
prompts, voice/STT, helper trio
docs/logic/ed-encounters.md (821 L) — multi-stage ED + MDM (this
session's worked example)
Admin-only docs viewer:
- New route /api/admin/docs/{tree,file}: recursively walks docs/, returns
the tree as JSON; /file?path=X validates path stays inside docs/ and
renders markdown via marked. Both gated by req.user.role==='admin'.
- New tab "Docs" (book icon) in the sidebar, hidden by default and
revealed in auth.js when user.role==='admin' (same pattern as the
existing Admin and CMS tabs).
- New component public/components/admin-docs.html: split-pane layout
with a tree sidebar + filter input + a markdown reader pane.
- New module public/js/admin-docs.js: lazy-loads the tree on first tab
activation, renders collapsible folders, persists expanded state and
last-opened path via UIState. Server-rendered HTML so no client
markdown parser needed.
- CSS for the viewer (responsive split-pane, code-block styling, table
scrolling, etc.).
- Mounted at /api/admin/docs (NOT /api) — important: mounting a router
with router.use(authMiddleware) at /api accidentally 401s every other
/api/* path (caught and fixed during testing — /api/health was 401'ing).
Files: docs/* (moved + new), README.md, public/components/admin-docs.html
(new), public/js/admin-docs.js (new), src/routes/adminDocs.js (new),
public/index.html (tab + section + script), public/js/auth.js (admin
gate + logout cleanup), public/css/styles.css (viewer styles), server.js
(mount).
═══════════════════════════════════════════════════════════════════
KNOWN GAPS (TODO follow-ups)
═══════════════════════════════════════════════════════════════════
- Learning Hub quiz engine (MCQ / multi-select / T-F scoring + attempt
tracking + progress dashboard) is covered at the architectural level
in docs/logic/auth-admin-learning.md but not drilled into the quiz
data model and scoring flow. Worth a focused follow-up doc.
- ED finalize: if MDM step JSON parse fails, server returns 502 with
the consolidated finalNote in the error payload, but client doesn't
surface the partial result. Add a "MDM failed, retry" affordance.
- No e2e Playwright coverage for ED encounters or the new docs viewer.
279 lines
8.3 KiB
Markdown
279 lines
8.3 KiB
Markdown
# Transcription Options Guide
|
|
|
|
## Overview
|
|
|
|
Pediatric AI Scribe v2+ offers **three transcription methods**, allowing you to choose between **privacy**, **speed**, and **real-time feedback**.
|
|
|
|
---
|
|
|
|
## 📊 Comparison Table
|
|
|
|
| Feature | Browser Whisper | Server Transcription | Web Speech API |
|
|
|---------|----------------|---------------------|----------------|
|
|
| **Privacy** | ⭐⭐⭐⭐⭐ 100% offline | ⭐⭐⭐⭐ (with BAA) | ⭐ Sends to cloud |
|
|
| **Accuracy** | ⭐⭐⭐⭐⭐ Whisper | ⭐⭐⭐⭐⭐ Gemini/AWS | ⭐⭐⭐ Browser-dependent |
|
|
| **Speed** | ⭐⭐⭐ 2-10s | ⭐⭐⭐⭐⭐ ~1s | ⭐⭐⭐⭐⭐ Instant |
|
|
| **Real-time** | ❌ Batch mode | ❌ Batch mode | ✅ Live streaming |
|
|
| **HIPAA** | ✅ Yes | ✅ (Vertex/AWS) | ❌ No |
|
|
| **Cost** | Free | ~$0.005/min | Free |
|
|
| **Internet** | ❌ Not required | ✅ Required | ✅ Required |
|
|
| **Setup** | None (bundled) | API keys | None (built-in) |
|
|
|
|
---
|
|
|
|
## Option 1: Browser Whisper (Offline, Private) ⭐ RECOMMENDED
|
|
|
|
### What It Is
|
|
- Runs **OpenAI Whisper** entirely in your browser using WebAssembly
|
|
- Audio **never leaves your device** - 100% offline after initial page load
|
|
- Models bundled in Docker image (self-hosted, no CDN)
|
|
|
|
### When to Use
|
|
- ✅ Clinical documentation (HIPAA-compliant)
|
|
- ✅ Maximum privacy required
|
|
- ✅ Offline/air-gapped environments
|
|
- ✅ No API costs
|
|
- ✅ Zero vendor dependency
|
|
|
|
### How to Enable
|
|
1. Settings → Browser Transcription
|
|
2. Toggle "Enable browser transcription" ON
|
|
3. (Optional) Click "Pre-download model" if you want to cache it first
|
|
4. Start recording - transcription happens automatically after recording
|
|
|
|
### Models Available
|
|
- **Tiny** (~39MB) - Fast, good for short clips (2-3 seconds)
|
|
- **Base** (~74MB) - Balanced accuracy and speed (3-5 seconds)
|
|
- **Small** (~244MB) - Best quality, slower (6-10 seconds)
|
|
|
|
### Performance
|
|
- Transcribes ~30-second clip in 2-10 seconds (depending on model)
|
|
- First run may be slower (model loading)
|
|
- Subsequent runs are instant (cached)
|
|
|
|
### Privacy
|
|
- ✅ Audio never transmitted
|
|
- ✅ Models run locally in WASM
|
|
- ✅ No network calls during transcription
|
|
- ✅ HIPAA-compliant
|
|
|
|
---
|
|
|
|
## Option 2: Server Transcription (Cloud, Fast)
|
|
|
|
### What It Is
|
|
- Sends audio to your configured AI provider
|
|
- Uses Google Gemini, AWS Transcribe, OpenAI Whisper, or LiteLLM
|
|
|
|
### When to Use
|
|
- ✅ Maximum speed (~1 second for 30-second clip)
|
|
- ✅ Best accuracy (cloud models)
|
|
- ✅ Long recordings (Browser Whisper can be slow for 5+ minutes)
|
|
- ✅ HIPAA-compliant with BAA providers
|
|
|
|
### HIPAA-Eligible Providers
|
|
- **Google Vertex AI** (with BAA) ✅
|
|
- **AWS Transcribe** (with BAA) ✅
|
|
- **Azure OpenAI** (with BAA) ✅
|
|
- **OpenAI Whisper Direct** ❌ Not HIPAA-eligible
|
|
|
|
### How to Enable
|
|
- Configured via environment variables (`.env`)
|
|
- No user action needed - just works if API keys present
|
|
- Falls back automatically if Browser Whisper fails
|
|
|
|
### Cost
|
|
- Google Gemini: ~$0.005/minute
|
|
- AWS Transcribe: ~$0.024/minute
|
|
- OpenAI: $0.006/minute
|
|
|
|
---
|
|
|
|
## Option 3: Web Speech API (Real-Time, Experimental) ⚠️
|
|
|
|
### What It Is
|
|
- Uses your browser's built-in speech recognition
|
|
- Shows transcription **in real-time** as you speak (streaming)
|
|
- Chrome/Edge → Google Cloud Speech
|
|
- Safari → Apple Speech Recognition
|
|
|
|
### ⚠️ PRIVACY WARNING
|
|
- **Audio IS sent to cloud servers** (Google, Apple, etc.)
|
|
- **NOT HIPAA-compliant**
|
|
- Only use for non-clinical, personal use
|
|
|
|
### When to Use
|
|
- ✅ Personal notes (non-clinical)
|
|
- ✅ Want real-time feedback while speaking
|
|
- ✅ Demonstration/testing
|
|
- ❌ **NEVER for patient data**
|
|
|
|
### How to Enable
|
|
1. Settings → Real-Time Streaming Transcription
|
|
2. Read privacy warning carefully
|
|
3. Toggle "Enable real-time streaming" ON
|
|
4. Confirm warning dialog
|
|
5. Grants microphone permission
|
|
6. Start recording - see words appear live
|
|
|
|
### Limitations
|
|
- Not available in all browsers (requires Web Speech API)
|
|
- Accuracy varies by browser
|
|
- Requires internet connection
|
|
- May have usage limits
|
|
|
|
---
|
|
|
|
## Choosing the Right Option
|
|
|
|
### For Clinical Use (HIPAA Required)
|
|
**Use:** Browser Whisper (offline) OR Server (Vertex AI/AWS with BAA)
|
|
- Browser Whisper: Maximum privacy, no costs
|
|
- Server: Faster, better for long recordings
|
|
|
|
### For Personal Use (Non-HIPAA)
|
|
**Use:** Any option
|
|
- Browser Whisper: Best balance of privacy and accuracy
|
|
- Server: Fastest
|
|
- Web Speech: Real-time feedback
|
|
|
|
### Decision Tree
|
|
|
|
```
|
|
Is this clinical/patient data?
|
|
├─ YES → Use Browser Whisper or Server (Vertex/AWS)
|
|
│ ├─ Need offline? → Browser Whisper
|
|
│ ├─ Need speed? → Server (Vertex AI)
|
|
│ └─ Want free? → Browser Whisper
|
|
│
|
|
└─ NO → Any option
|
|
├─ Want real-time? → Web Speech API
|
|
├─ Want privacy? → Browser Whisper
|
|
└─ Want speed? → Server
|
|
```
|
|
|
|
---
|
|
|
|
## Configuration
|
|
|
|
### Browser Whisper
|
|
```bash
|
|
# No configuration needed - bundled in Docker image
|
|
# Models at: /app/public/models/Xenova/whisper-tiny.en/
|
|
```
|
|
|
|
### Server Transcription
|
|
```bash
|
|
# .env file
|
|
TRANSCRIBE_PROVIDER=google # google, aws, openai, litellm
|
|
|
|
# Google Vertex AI
|
|
GOOGLE_VERTEX_PROJECT=your-project-id
|
|
GOOGLE_APPLICATION_CREDENTIALS=/path/to/key.json
|
|
|
|
# AWS Transcribe
|
|
AWS_BEDROCK_REGION=us-east-1
|
|
AWS_ACCESS_KEY_ID=your-key
|
|
AWS_SECRET_ACCESS_KEY=your-secret
|
|
|
|
# OpenAI
|
|
OPENAI_API_KEY=sk-...
|
|
|
|
# LiteLLM (proxy)
|
|
LITELLM_API_BASE=http://localhost:4000
|
|
LITELLM_API_KEY=optional
|
|
```
|
|
|
|
### Web Speech API
|
|
```bash
|
|
# No configuration - uses browser built-in
|
|
# Privacy warning shown in Settings UI
|
|
```
|
|
|
|
---
|
|
|
|
## FAQ
|
|
|
|
### Q: Which is most accurate?
|
|
**A:** Browser Whisper and Server (Gemini/Whisper) are equally accurate. Web Speech is slightly less accurate.
|
|
|
|
### Q: Which is fastest?
|
|
**A:** Server transcription (~1s) > Web Speech (real-time) > Browser Whisper (2-10s)
|
|
|
|
### Q: Which is most private?
|
|
**A:** Browser Whisper (100% offline) > Server (with BAA) > Web Speech (not private)
|
|
|
|
### Q: Can I use multiple at once?
|
|
**A:** No. Priority: Web Speech > Browser Whisper > Server (whichever is enabled first)
|
|
|
|
### Q: What if transcription fails?
|
|
**A:** Automatic fallback chain:
|
|
1. Browser Whisper (if enabled)
|
|
2. Falls back to Server (if configured)
|
|
3. Falls back to live transcript (if available)
|
|
|
|
### Q: Is Browser Whisper really offline?
|
|
**A:** Yes! Models are bundled in the Docker image. After the page loads once, transcription works with zero network access.
|
|
|
|
### Q: Does Web Speech work offline?
|
|
**A:** No. It requires internet to send audio to cloud servers.
|
|
|
|
### Q: Can I train/customize the models?
|
|
**A:** No. Browser Whisper uses pre-trained models. Server transcription uses cloud models. No custom training available.
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
### Browser Whisper stuck at "Initializing"
|
|
- **Cause:** Models not loaded or network blocked during initial download
|
|
- **Fix:** See [browser-whisper-troubleshooting.md](browser-whisper-troubleshooting.md)
|
|
|
|
### Server transcription returns "No provider"
|
|
- **Cause:** API keys not configured
|
|
- **Fix:** Set environment variables in `.env`
|
|
|
|
### Web Speech says "Not supported"
|
|
- **Cause:** Browser doesn't support Web Speech API
|
|
- **Fix:** Use Chrome, Edge, or Safari
|
|
|
|
### Transcription is slow
|
|
- **Browser Whisper:** Try switching to "Tiny" model
|
|
- **Server:** Check API provider status
|
|
- **Web Speech:** Check internet connection
|
|
|
|
---
|
|
|
|
## Best Practices
|
|
|
|
### Clinical Documentation
|
|
1. Use Browser Whisper for all patient data
|
|
2. Enable audio backups (automatic in v2)
|
|
3. Keep recordings under 5 minutes for faster processing
|
|
4. Use "Tiny" model for quick notes, "Base" for detailed documentation
|
|
|
|
### Personal Use
|
|
1. Web Speech for quick, informal notes
|
|
2. Browser Whisper for anything you want private
|
|
3. Server for long recordings
|
|
|
|
### Performance Optimization
|
|
1. Pre-download Browser Whisper model before first use
|
|
2. Use shorter clips (30-60 seconds) for fastest results
|
|
3. Clear browser cache if models seem corrupted
|
|
|
|
---
|
|
|
|
## Summary
|
|
|
|
| Need | Recommendation |
|
|
|------|---------------|
|
|
| Clinical/HIPAA | Browser Whisper (offline) |
|
|
| Fast transcription | Server (Vertex AI) |
|
|
| Real-time feedback | Web Speech (non-clinical only) |
|
|
| Maximum privacy | Browser Whisper |
|
|
| Zero cost | Browser Whisper |
|
|
| Long recordings | Server (faster for 5+ min clips) |
|
|
| Offline use | Browser Whisper |
|
|
|
|
**Default recommendation:** Browser Whisper for 95% of use cases. It's private, accurate, free, and offline. Only use alternatives when you have specific needs for speed or real-time feedback.
|