503f5afaad
Three concurrent themes from this session:
═══════════════════════════════════════════════════════════════════
ED ENCOUNTERS — per-stage cards + consolidate→MDM finalize
═══════════════════════════════════════════════════════════════════
UX redesign per Daniel's feedback ("every stage note should be shown,
if AI is told to modify that particular note then the modified version
is used in final mdm"):
- Each generated stage stays on screen as its own editable card with
its own embedded "Don't Miss" panel. No more single rolling note
element that gets replaced on each generation.
- gatherCurrentNotes() reads contenteditable text from each stage card
before any operation (advance, finalize, persist) so inline edits
flow into the next AI call and the final consolidate.
- Stage badge is now state-accurate. "Stage N (recording)" with yellow
background after Add-more before generation; "Stage N" with gray
after generation. Fixes the bug where the badge flipped to Stage 2
the moment Add-more was clicked.
- Save & Done now runs TWO server-side AI calls in /finalize:
1. edConsolidate (new prompt) → polished single final note that
integrates every stage chronologically (HPI / ROS / PE / ED Course /
A&P with disposition).
2. edFinalize (rewritten with full inline 2023 AMA E/M element
rubric — problems / data / risk definitions, level mapping with
concrete examples) → MDM JSON.
- Two new cards render after finalize: blue-bordered Final Consolidated
Note + green-bordered MDM. Stage cards become read-only.
- partial_data on the saved row now stores {stages, finalNote, mdm,
finalized} so resume re-renders the full state.
Why two-call finalize: a single combined prompt makes the model cut
corners on one task. Two focused calls cost ~2× latency at the very end
of an encounter — acceptable since finalize is a one-time terminal
action, not a per-stage hot path.
Files: public/components/ed-encounter.html, public/js/ed-encounters.js,
src/routes/edEncounters.js, src/utils/prompts.js (edConsolidate added,
edFinalize rewritten).
═══════════════════════════════════════════════════════════════════
EXTENSIONS / PAGERS — visual polish
═══════════════════════════════════════════════════════════════════
Multiple iterations based on Daniel's feedback:
- Layout: align-items:flex-start so action buttons stay pinned top-right
when long numbers wrap (was align-items:center → buttons drifted into
the text area, causing visible overlap).
- Number: word-break:break-all + min-width:0 + font-feature-settings:tnum
so long numbers wrap within their column instead of pushing under the
buttons. Click-to-copy with a 0.55s green flash + ✓ copied badge.
- Phone/pager Font Awesome icon next to the number in the type color —
at-a-glance type signal (replacing an earlier 3px left stripe that
Daniel found visually bulky).
- Name: font-weight 700, font-size 14.5px, color g900, letter-spacing
-0.012em — scan-target headline typography for long lists.
- Alternating subtle backgrounds by index (white vs #fafbfc) so a long
list reads as distinct rows.
- Hover: card lifts 1px with a soft shadow; action buttons fade from
55% to 100% opacity. Cubic-bezier transition on transform.
- Entrance: staggered fade-up animation per card (35ms × index, capped
at 12). prefers-reduced-motion media query disables motion.
- Empty state: 48px FA icon + heading instead of plain gray text.
Files: public/js/extensions.js, public/css/styles.css.
═══════════════════════════════════════════════════════════════════
DOCS REORGANIZATION + APPLICATION-LOGIC DOCS + ADMIN VIEWER
═══════════════════════════════════════════════════════════════════
Document moves (preserving git history via git mv):
BROWSER_WHISPER_SETUP.md → docs/browser-whisper-setup.md
BROWSER_WHISPER_TROUBLESHOOTING.md → docs/browser-whisper-troubleshooting.md
DEVELOPER_GUIDE.md → docs/developer-guide-extended.md
EMBEDDINGS_SETUP.md → docs/embeddings-setup.md
FEATURES_EXPLAINED.md → docs/features-explained.md
IMPROVEMENTS.md → docs/improvements.md
OPENID_SETUP.md → docs/openid-setup.md
TRANSCRIPTION_OPTIONS.md → docs/transcription-options.md
README.md updated with the new paths + a Documentation section that
links to docs/logic/ at the top.
New application-logic doc series (~8,300 lines total) at docs/logic/.
Built with 5 parallel doc-writing agents per Daniel's "use multiple
agents" directive. Each doc explains how a part of the app actually
works — application logic, data flow, design decisions, sacred zones,
how-to-extend recipes — at a depth that lets a new dev (or an AI
assistant) modify the code confidently.
docs/logic/README.md — index + recommended reading order
docs/logic/architecture.md (2166 L) — frontend IIFE pattern, lazy tab
load, backend route convention,
schema, encryption, deployment
docs/logic/clinical-notes.md (1546L) — every note tab + helper trio
docs/logic/bedside-and-calculators.md (1373L) — bedside ES module
pocket + calculators + PE Guide
+ suture selector
docs/logic/auth-admin-learning.md (1281L) — auth (local+OIDC+2FA) +
admin panel + Learning Hub
(Quiz engine logic at sub-detail
only — TODO follow-up)
docs/logic/ai-and-voice.md (1128 L) — callAI 5-provider routing,
prompts, voice/STT, helper trio
docs/logic/ed-encounters.md (821 L) — multi-stage ED + MDM (this
session's worked example)
Admin-only docs viewer:
- New route /api/admin/docs/{tree,file}: recursively walks docs/, returns
the tree as JSON; /file?path=X validates path stays inside docs/ and
renders markdown via marked. Both gated by req.user.role==='admin'.
- New tab "Docs" (book icon) in the sidebar, hidden by default and
revealed in auth.js when user.role==='admin' (same pattern as the
existing Admin and CMS tabs).
- New component public/components/admin-docs.html: split-pane layout
with a tree sidebar + filter input + a markdown reader pane.
- New module public/js/admin-docs.js: lazy-loads the tree on first tab
activation, renders collapsible folders, persists expanded state and
last-opened path via UIState. Server-rendered HTML so no client
markdown parser needed.
- CSS for the viewer (responsive split-pane, code-block styling, table
scrolling, etc.).
- Mounted at /api/admin/docs (NOT /api) — important: mounting a router
with router.use(authMiddleware) at /api accidentally 401s every other
/api/* path (caught and fixed during testing — /api/health was 401'ing).
Files: docs/* (moved + new), README.md, public/components/admin-docs.html
(new), public/js/admin-docs.js (new), src/routes/adminDocs.js (new),
public/index.html (tab + section + script), public/js/auth.js (admin
gate + logout cleanup), public/css/styles.css (viewer styles), server.js
(mount).
═══════════════════════════════════════════════════════════════════
KNOWN GAPS (TODO follow-ups)
═══════════════════════════════════════════════════════════════════
- Learning Hub quiz engine (MCQ / multi-select / T-F scoring + attempt
tracking + progress dashboard) is covered at the architectural level
in docs/logic/auth-admin-learning.md but not drilled into the quiz
data model and scoring flow. Worth a focused follow-up doc.
- ED finalize: if MDM step JSON parse fails, server returns 502 with
the consolidated finalNote in the error payload, but client doesn't
surface the partial result. Add a "MDM failed, retry" affordance.
- No e2e Playwright coverage for ED encounters or the new docs viewer.
8.3 KiB
8.3 KiB
Transcription Options Guide
Overview
Pediatric AI Scribe v2+ offers three transcription methods, allowing you to choose between privacy, speed, and real-time feedback.
📊 Comparison Table
| Feature | Browser Whisper | Server Transcription | Web Speech API |
|---|---|---|---|
| Privacy | ⭐⭐⭐⭐⭐ 100% offline | ⭐⭐⭐⭐ (with BAA) | ⭐ Sends to cloud |
| Accuracy | ⭐⭐⭐⭐⭐ Whisper | ⭐⭐⭐⭐⭐ Gemini/AWS | ⭐⭐⭐ Browser-dependent |
| Speed | ⭐⭐⭐ 2-10s | ⭐⭐⭐⭐⭐ ~1s | ⭐⭐⭐⭐⭐ Instant |
| Real-time | ❌ Batch mode | ❌ Batch mode | ✅ Live streaming |
| HIPAA | ✅ Yes | ✅ (Vertex/AWS) | ❌ No |
| Cost | Free | ~$0.005/min | Free |
| Internet | ❌ Not required | ✅ Required | ✅ Required |
| Setup | None (bundled) | API keys | None (built-in) |
Option 1: Browser Whisper (Offline, Private) ⭐ RECOMMENDED
What It Is
- Runs OpenAI Whisper entirely in your browser using WebAssembly
- Audio never leaves your device - 100% offline after initial page load
- Models bundled in Docker image (self-hosted, no CDN)
When to Use
- ✅ Clinical documentation (HIPAA-compliant)
- ✅ Maximum privacy required
- ✅ Offline/air-gapped environments
- ✅ No API costs
- ✅ Zero vendor dependency
How to Enable
- Settings → Browser Transcription
- Toggle "Enable browser transcription" ON
- (Optional) Click "Pre-download model" if you want to cache it first
- Start recording - transcription happens automatically after recording
Models Available
- Tiny (~39MB) - Fast, good for short clips (2-3 seconds)
- Base (~74MB) - Balanced accuracy and speed (3-5 seconds)
- Small (~244MB) - Best quality, slower (6-10 seconds)
Performance
- Transcribes ~30-second clip in 2-10 seconds (depending on model)
- First run may be slower (model loading)
- Subsequent runs are instant (cached)
Privacy
- ✅ Audio never transmitted
- ✅ Models run locally in WASM
- ✅ No network calls during transcription
- ✅ HIPAA-compliant
Option 2: Server Transcription (Cloud, Fast)
What It Is
- Sends audio to your configured AI provider
- Uses Google Gemini, AWS Transcribe, OpenAI Whisper, or LiteLLM
When to Use
- ✅ Maximum speed (~1 second for 30-second clip)
- ✅ Best accuracy (cloud models)
- ✅ Long recordings (Browser Whisper can be slow for 5+ minutes)
- ✅ HIPAA-compliant with BAA providers
HIPAA-Eligible Providers
- Google Vertex AI (with BAA) ✅
- AWS Transcribe (with BAA) ✅
- Azure OpenAI (with BAA) ✅
- OpenAI Whisper Direct ❌ Not HIPAA-eligible
How to Enable
- Configured via environment variables (
.env) - No user action needed - just works if API keys present
- Falls back automatically if Browser Whisper fails
Cost
- Google Gemini: ~$0.005/minute
- AWS Transcribe: ~$0.024/minute
- OpenAI: $0.006/minute
Option 3: Web Speech API (Real-Time, Experimental) ⚠️
What It Is
- Uses your browser's built-in speech recognition
- Shows transcription in real-time as you speak (streaming)
- Chrome/Edge → Google Cloud Speech
- Safari → Apple Speech Recognition
⚠️ PRIVACY WARNING
- Audio IS sent to cloud servers (Google, Apple, etc.)
- NOT HIPAA-compliant
- Only use for non-clinical, personal use
When to Use
- ✅ Personal notes (non-clinical)
- ✅ Want real-time feedback while speaking
- ✅ Demonstration/testing
- ❌ NEVER for patient data
How to Enable
- Settings → Real-Time Streaming Transcription
- Read privacy warning carefully
- Toggle "Enable real-time streaming" ON
- Confirm warning dialog
- Grants microphone permission
- Start recording - see words appear live
Limitations
- Not available in all browsers (requires Web Speech API)
- Accuracy varies by browser
- Requires internet connection
- May have usage limits
Choosing the Right Option
For Clinical Use (HIPAA Required)
Use: Browser Whisper (offline) OR Server (Vertex AI/AWS with BAA)
- Browser Whisper: Maximum privacy, no costs
- Server: Faster, better for long recordings
For Personal Use (Non-HIPAA)
Use: Any option
- Browser Whisper: Best balance of privacy and accuracy
- Server: Fastest
- Web Speech: Real-time feedback
Decision Tree
Is this clinical/patient data?
├─ YES → Use Browser Whisper or Server (Vertex/AWS)
│ ├─ Need offline? → Browser Whisper
│ ├─ Need speed? → Server (Vertex AI)
│ └─ Want free? → Browser Whisper
│
└─ NO → Any option
├─ Want real-time? → Web Speech API
├─ Want privacy? → Browser Whisper
└─ Want speed? → Server
Configuration
Browser Whisper
# No configuration needed - bundled in Docker image
# Models at: /app/public/models/Xenova/whisper-tiny.en/
Server Transcription
# .env file
TRANSCRIBE_PROVIDER=google # google, aws, openai, litellm
# Google Vertex AI
GOOGLE_VERTEX_PROJECT=your-project-id
GOOGLE_APPLICATION_CREDENTIALS=/path/to/key.json
# AWS Transcribe
AWS_BEDROCK_REGION=us-east-1
AWS_ACCESS_KEY_ID=your-key
AWS_SECRET_ACCESS_KEY=your-secret
# OpenAI
OPENAI_API_KEY=sk-...
# LiteLLM (proxy)
LITELLM_API_BASE=http://localhost:4000
LITELLM_API_KEY=optional
Web Speech API
# No configuration - uses browser built-in
# Privacy warning shown in Settings UI
FAQ
Q: Which is most accurate?
A: Browser Whisper and Server (Gemini/Whisper) are equally accurate. Web Speech is slightly less accurate.
Q: Which is fastest?
A: Server transcription (~1s) > Web Speech (real-time) > Browser Whisper (2-10s)
Q: Which is most private?
A: Browser Whisper (100% offline) > Server (with BAA) > Web Speech (not private)
Q: Can I use multiple at once?
A: No. Priority: Web Speech > Browser Whisper > Server (whichever is enabled first)
Q: What if transcription fails?
A: Automatic fallback chain:
- Browser Whisper (if enabled)
- Falls back to Server (if configured)
- Falls back to live transcript (if available)
Q: Is Browser Whisper really offline?
A: Yes! Models are bundled in the Docker image. After the page loads once, transcription works with zero network access.
Q: Does Web Speech work offline?
A: No. It requires internet to send audio to cloud servers.
Q: Can I train/customize the models?
A: No. Browser Whisper uses pre-trained models. Server transcription uses cloud models. No custom training available.
Troubleshooting
Browser Whisper stuck at "Initializing"
- Cause: Models not loaded or network blocked during initial download
- Fix: See browser-whisper-troubleshooting.md
Server transcription returns "No provider"
- Cause: API keys not configured
- Fix: Set environment variables in
.env
Web Speech says "Not supported"
- Cause: Browser doesn't support Web Speech API
- Fix: Use Chrome, Edge, or Safari
Transcription is slow
- Browser Whisper: Try switching to "Tiny" model
- Server: Check API provider status
- Web Speech: Check internet connection
Best Practices
Clinical Documentation
- Use Browser Whisper for all patient data
- Enable audio backups (automatic in v2)
- Keep recordings under 5 minutes for faster processing
- Use "Tiny" model for quick notes, "Base" for detailed documentation
Personal Use
- Web Speech for quick, informal notes
- Browser Whisper for anything you want private
- Server for long recordings
Performance Optimization
- Pre-download Browser Whisper model before first use
- Use shorter clips (30-60 seconds) for fastest results
- Clear browser cache if models seem corrupted
Summary
| Need | Recommendation |
|---|---|
| Clinical/HIPAA | Browser Whisper (offline) |
| Fast transcription | Server (Vertex AI) |
| Real-time feedback | Web Speech (non-clinical only) |
| Maximum privacy | Browser Whisper |
| Zero cost | Browser Whisper |
| Long recordings | Server (faster for 5+ min clips) |
| Offline use | Browser Whisper |
Default recommendation: Browser Whisper for 95% of use cases. It's private, accurate, free, and offline. Only use alternatives when you have specific needs for speed or real-time feedback.