FINAL WORKING SOLUTION:
Previous attempts failed because:
- transformers.js v2.17.2 is ES module-only
- Module workers require complex CSP and external imports
- importScripts() doesn't work with ES modules
Solution:
- Use transformers.js v2.6.2 (has worker-compatible UMD build)
- Bundle library + models, serve entirely from our server
- Classic worker with importScripts() - no CSP issues
What's self-hosted:
- ✅ transformers.min.js (760KB) - at /models/transformers.min.js
- ✅ Whisper models (42MB) - at /models/Xenova/whisper-tiny.en/
Worker loads:
1. importScripts('/models/transformers.min.js') - OUR SERVER
2. Loads models from /models/ - OUR SERVER
3. ZERO external network calls
4. Works in any network (firewalled, air-gapped, etc.)
This is the production-ready, truly offline solution.
Issue: transformers.js is an ES module package and cannot be loaded
with importScripts() in classic workers.
Solution:
- Changed to module worker (type: 'module')
- Import transformers.js from CDN as ES module
- Models (42MB) still served from local server at /models/
Trade-off:
- Library (900KB): Loads from cdn.jsdelivr.net once, cached
- Models (42MB): Self-hosted, served from /models/ (no CDN)
This is necessary because:
1. @xenova/transformers is ES module-only (package.json: "type": "module")
2. ES modules cannot use importScripts()
3. Module workers require HTTPS for imports
4. CDN is HTTPS and cacheable
If CDN is blocked:
- Use Web Speech API (with privacy warnings)
- OR use Server Transcription (Vertex AI/AWS)
Models remain self-hosted as they're 40MB+ and contain the AI.
Provides two transcription options:
1. Browser Whisper (Offline, Batch) - RECOMMENDED
- 100% offline, zero network calls
- HIPAA-compliant, audio never leaves device
- Highest accuracy (Whisper)
- Processes after recording (batch mode)
- Models self-hosted, bundled in v2
2. Web Speech API (Real-time, Streaming) - EXPERIMENTAL
- Real-time transcription (see words as you speak)
- Uses browser's built-in speech recognition
- ⚠️ Sends audio to cloud (Chrome/Edge → Google)
- ⚠️ NOT HIPAA-compliant
- Requires user consent with clear warnings
Features:
- Settings UI for both options
- Clear privacy warnings for Web Speech
- Mutual exclusion (only one active at a time)
- Browser detection shows which provider is used
- Confirmation dialog before enabling Web Speech
Use Cases:
- Clinical/HIPAA: Use Browser Whisper only
- Personal/Non-clinical: Can use Web Speech for real-time feedback
- Maximum privacy: Browser Whisper (offline)
- Maximum speed: Web Speech (if privacy not required)
Implementation:
- speechRecognition.js: Web Speech API wrapper
- transcriptionSettings.js: Settings UI handler
- Privacy info displayed per browser
User can choose based on their privacy vs. speed preference.
Features:
- Admin can add, edit, and delete developmental milestones via dashboard
- Milestones stored in PostgreSQL (developmental_milestones table)
- Client-side loads milestones from API instead of static file
- Import script to migrate existing static data to database
- Organized by age group and domain
- Supports sorting and filtering
Admin UI:
- New section in Admin panel for milestone management
- Filter by age group
- Add/Edit modal with validation
- Delete with confirmation
- Auto-complete for age groups and domains
API Endpoints:
- GET /api/milestones-data - Public endpoint for authenticated users
- GET /api/admin/milestones - List all milestones (admin only)
- GET /api/admin/milestones/meta - Get age groups and domains
- POST /api/admin/milestones - Create milestone
- PUT /api/admin/milestones/:id - Update milestone
- DELETE /api/admin/milestones/:id - Delete milestone
- POST /api/admin/milestones/bulk-import - Bulk import
Usage:
1. Run import script: node scripts/import-milestones.js
2. Access Admin dashboard → Developmental Milestones section
3. Add/Edit/Delete milestones as needed
BREAKING FIX: Browser Whisper now fully self-contained
Previous issue:
- Loaded transformers.js from cdn.jsdelivr.net
- Downloaded models from cdn-lfs.huggingface.co
- Failed in corporate/clinical networks with firewall
- Stuck at "Initializing..." with no progress
Solution:
- Bundle transformers.js library (~876KB)
- Bundle Whisper tiny.en model (~42MB)
- Serve everything from local server
- Works in ANY network environment
Changes:
- whisperWorker.js: Load transformers from /models/ instead of CDN
- Dockerfile: Download models during Docker build
- Add download script for local dev
- Add comprehensive setup documentation
Docker image size: +~42MB (one-time cost, runtime benefit)
Tested: Works on unrestricted and firewalled networks
REALITY CHECK: Browser Whisper CDN loading cannot work in all environments
- Corporate firewalls block cdn.jsdelivr.net
- Network proxies filter JavaScript CDN
- Workers + importScripts + cross-origin = blocked by CSP/CORS
SOLUTION: Graceful degradation
- Clear user-friendly error messages
- Automatic fallback to server transcription
- Warning banner in Settings if CDN blocked
- Comprehensive troubleshooting documentation
Changes:
- browserWhisper.js: Show toast on worker error, fallback gracefully
- app.js: Display CSP warning banner on preload failure
- settings.html: Add warning about network/firewall requirements
- BROWSER_WHISPER_TROUBLESHOOTING.md: Complete guide for users
Key Message:
Browser Whisper is OPTIONAL. Server transcription (Google/AWS/OpenAI)
is the primary method and works everywhere. Browser Whisper is a
privacy-focused bonus feature that requires CDN access.
User Experience:
- If CDN works: Great! Browser Whisper available
- If CDN blocked: No problem! Server transcription works perfectly
- Clear messaging: User knows what to expect
- Allow empty voice value to preview server default
- Display 'server default' in preview text
- Clears user preference (sets to null) when testing default
ROOT CAUSES FOUND AND FIXED:
1. TTS Preview not working: voicePreferences.js listening for wrong event
- Was: 'tab-loaded' (never dispatched)
- Now: 'tabChanged' (correct event name used by app.js)
- Added immediate init if page already loaded
- Added 500ms delay for DOM readiness
2. Browser Whisper CDN blocked: CSP too restrictive
- Added 'unsafe-eval' to scriptSrc (required by transformers.js)
- Added cdn.jsdelivr.net to connectSrc (worker importScripts)
- Added childSrc directive for worker script loading
- Better error messages in worker
3. Worker loading errors: Now logged with specific reasons
- importScripts wrapped in try-catch
- Posts error message to main thread
- Verifies transformers object exists after load
Testing:
- TTS Preview should now work when clicking Settings tab
- Browser Whisper should load from CDN (or show specific error)
- Console logs will show exact init sequence
The button was always returning null because it searched for onclick=speakText
but all output cards use data-action="speak" data-target="id". Now checks
data-action first so the button correctly toggles to Stop during playback.
- Raise express.json limit from 1MB to 10MB — handles large chart reviews
with many notes (50 full clinic notes ≈ 600KB, well within new limit)
- Client-side: warn user if payload >8MB, friendly toast if >30 notes
- Bump to v13.0.0
- TTS response now includes X-TTS-Provider header (google-tts, litellm/model, elevenlabs)
- Frontend reads header and shows actual provider in toast instead of hardcoded "Adam/ElevenLabs"
- CORS exposes X-TTS-Provider header so frontend can access it
- Updated .env.example: clarify that LITELLM_TTS_MODEL and LITELLM_STT_MODEL
can be either the model_name alias OR the full provider/model path depending
on your LiteLLM config (important for BAA compliance routing)
- TTS: switch from OpenAI SDK to axios (same fix as STT), drop voice
param since it's configured inside LiteLLM per model
- Fix 'ElevenLabs unavailable' toast shown even when provider is LiteLLM
- Add dedicated red Stop button to encounter recording UI
- LITELLM_MODELS = [] — no hardcoded models, global selector now only
shows what admin has actually added via Search API
- getAvailableModelsWithOverrides: for LiteLLM returns only custom list
- Remove toggle safety check — admin can disable any/all models freely
- Admin panel always reloads on tab open (was cached, showing stale data)
- Add 'Clear all models' button for LiteLLM to wipe and start fresh
- Add POST /config/models/clear-all endpoint
- Root cause: PUT /config/:key(*) wildcard was registered before
/config/models/toggle and /config/models/default, intercepting them
and returning "value is required" (body had modelId not value)
- Fix: move all model-specific PUT routes before the wildcard
- LiteLLM: return empty built-in list with discovery hint (hardcoded
models don't match user's proxy — must use Search API)
- After adding a discovered model: auto-select it in the default dropdown
- GET /config/models now returns defaultModel so dropdown pre-selects it
- Add Vertex AI provider (Gemini models via @google-cloud/vertexai SDK)
- Add LiteLLM proxy support (OpenAI-compatible, routes to any provider)
- Admin panel: model search/discover from provider API, enable/disable, custom models, set default
- New endpoints: /config/models/discover, /config/models/add-discovered, /config/models/default
- Updated models.js with VERTEX_MODELS and LITELLM_MODELS lists
- Updated health endpoint with vertex + litellm status
Audio Backups:
- New audio_backups table in PostgreSQL (bytea, gzip compressed)
- POST /api/audio-backups — upload with gzip compression (level 6)
- GET /api/audio-backups — list user's backups
- GET /api/audio-backups/:id/audio — download decompressed audio
- DELETE /api/audio-backups/:id — delete backup
- Auto-cleanup every hour (24h expiry)
- Frontend saves to server first, falls back to IndexedDB
- Settings shows source badge (server/local) per backup
AI Corrections:
- Corrections list is now expandable — click to view original vs corrected
- Shows red "Original" and green "Corrected to" sections
- Click arrow to expand/collapse each correction
- Date shown on each correction
The admin model management (enable/disable, custom models, default override)
had persistent rendering issues. Removed the UI panel — models are managed
via the global model selector in the header, which works reliably. Backend
API endpoints for model config are retained for future use.
- Add GET /api/transcribe/status endpoint — returns whether any server
transcription provider (Whisper/AWS/Local) is configured
- Frontend checks status on login via checkTranscribeStatus()
- When no provider configured: recording stops instantly, keeps live
Web Speech API text, shows friendly toast — no error, no upload wait
- Works in encounter, dictation, and SOAP tabs
- App now works fully out-of-the-box with just an AI provider key
- AWS Transcribe: remove 10ms delay between chunks (was adding ~1.25s/MB),
increase chunk size from 8KB to 32KB (AWS max per frame)
- Add detailed timing logs (ffmpeg, streaming, total) for diagnostics
- OpenAI Whisper: use response_format='text' for faster response parsing
- Frontend: show transcription time in toast, request 16kHz sample rate,
increase bitrate to 32kbps Opus (better quality, still small files)
- Return duration in API response for all providers
- Full model management UI in admin panel: toggle models on/off, add custom
model IDs (any OpenRouter/Bedrock ID), set admin-configured default model
- /api/models now returns admin-set default model, frontend respects it
- Toggle switch CSS for clean enable/disable UX
- Backend already had the API endpoints, this adds the missing UI
- OIDC callback now passes only ?sso=ok flag, token stays in
httpOnly cookie (prevents token leaking to logs/referrer/history)
- Frontend auth.js uses cookie-based auth for SSO flow
- Add [PHYSICIAN TEMPLATES] boundary markers around physicianMemories
in all 5 generation routes to mitigate prompt injection
- Consistent boundary format across wellVisit, sickVisit, hpi, soap,
hospitalCourse
OIDC/SSO:
- New /api/auth/oidc route with PKCE for secure authorization
- Supports Azure AD, Okta, Keycloak, PocketID, Google, any OIDC provider
- Admin configurable: issuer, client ID/secret, button label
- Option to disable local auth (force SSO only)
- Auto-creates users on first SSO login, links existing by email
- SSO button on login page, hidden until admin enables OIDC
Firefox:
- Show info toast on first recording that live preview requires
Chrome/Edge; server-side transcription still works in all browsers
Previously only well-visit and sick-visit used saved physician
templates. Now HPI encounter, HPI dictation, SOAP, and hospital
course all fetch getUserMemoryContext() and pass physicianMemories
to the backend so the AI learns from saved templates/preferences.
- Add deduplication logic to prevent Chrome Speech API from repeating
sentences during long recording sessions (all 4 recording modules)
- Enable AWS Transcribe Medical with PRIMARYCARE specialty in .env
- Bump version to 8.0.0
- Fix word repetition: use sessionFinals pattern so each browser SR
session starts fresh; no overlap when recognition auto-restarts
- Fix HTML injection / '>' parse error: escape < > & in live transcript
innerHTML before inserting speech recognition text
- Add 24 MB blob guard: fall back to live SR transcript if audio file
is too large for Whisper API (long sessions)
- Bump version to 7.0.0, update docker-compose image tag to v7
- Add type="button" to all CMS action buttons to prevent default submit
- Change overflow:hidden to overflow:visible on .cms-main to prevent clipping
- Add z-index to .cms-editor-actions for reliable click targeting
- Replace AI "Topic" text input with a descriptive textarea context box
so users can tell AI what to generate (AI creates its own title)
- Rename AI tab from "By Topic" to "Describe Content"
Search (was completely broken):
- wireSearch() registered inside tabChanged for 'learning' tab — runs
after component HTML is in the DOM (lh-search existed but listener
was never attached because element was null at IIFE load time)
- Proper debounced search with 350ms delay, Escape key to clear
- Search results show count, 'X results for "query"', Clear Search button
- Empty state with helpful message; loading state while fetching
Content viewer showing feed items beneath it (CSS cascade bug):
- .lh-feed{display:flex} and .lh-category-bar{display:flex} defined
at CSS lines 513/507 — after .hidden{display:none} at line 242
So classList.add('hidden') was silently overridden
- showFeed() and showViewer() now use element.style.display with
explicit values ('flex'/'block'/'none') which always win over class rules
- viewer started hidden via class="hidden"; now shown with style.display='block'
- New GET /api/learning/content/:slug/slides (authMiddleware only)
Renders Marp markdown via marp-core, returns {css, slides[]} for
any authenticated user — no moderator access required
- showViewer: when content_type === 'presentation', hides the body card
and shows a presentation card with title, slide count, and View Slides button
Auto-opens the slide modal immediately on load; button allows re-opening
- openSlidesFromSlug(): fetches slides from user-accessible endpoint
then opens the existing slide preview modal (arrow keys, swipe, dots)
- Viewer already correctly hides feed/categories/search when viewing content;
"Back to Feed" button returns to the main list
Login flash:
- Auth screen now hidden by CSS default (#auth-screen{display:none !important})
- auth.js shows it only when there is no valid token or token is expired
- No more flash of login page on refresh for logged-in users
Presentation quiz option (AI Generate panel):
- Presentation type now shows optional quiz question toggle (same as article)
- Backend: when questionCount>0, returns JSON {marpMarkdown, questions[]}
instead of plain Marp markdown
- applyAiContent fills both the Marp textarea and question blocks for presentations
Viewer revert:
- Removed the viewer presentation card change (user wants markdown as-is)
- Removed dangling viewer button wiring
Feed/CMS list:
- Presentation type shows correct icon (fa-presentation-screen) and
label "Slides" in the CMS content list
pdf-parse (was v2, broken API):
- Downgraded to v1.1.1 — default export is a function again
- pdfParse is not a function error fixed
WebDAV file selection (style.display bug, same cascade issue):
- lh-ai-webdav-selected had inline style="display:flex" always visible
- selectWebdavFile() / deselectWebdavFile() now use element.style.display
- On select: browser div hides, selected indicator shows with file name + X
- On deselect (X): indicator hides, browser shows again
Topic context for Upload and Nextcloud tabs:
- Both tabs now have optional "Topic / context" field
- Sent to backend as 'topic' param to help AI focus the generated content
Inline refine bar (replaces window.prompt):
- Refine Body button shows/hides an amber inline input bar below generate buttons
- Enter instructions, press Apply — no browser dialog, works in all contexts
CSP: remove unsafe-inline from scriptSrc (security issue #2):
- Converted all 26 onclick= handlers in component HTML files to
data-action / data-target / data-label attributes
- Added delegated click handler in app.js for copy/speak/nc-export actions
- script-src now 'self' only; script-src-attr 'none'
WebDAV path endpoint (security issue #5):
- New POST /api/user/webdav-path (authMiddleware only, not moderator)
- nextcloud.js updated to use new endpoint
nodemailer: already at 6.10.1 (vulnerability fixed).
Bug: classList.toggle('hidden') was silently ignored because
.lh-ai-opt-field{display:flex} is defined at CSS line 758, after
.hidden{display:none} at line 242 — same specificity, later rule wins.
Fix: updateAiOptions() now uses element.style.display directly (inline
styles always override class rules).
Also fix initial state: lh-ai-slides-field now uses style="display:none"
instead of class="hidden" so it's correctly hidden on load.
Panel syncs content type from editor on open (editorType → ctype select
→ updateAiOptions called immediately).
Per type:
- Quiz: word count hidden, slides hidden, quiz card visible (no toggle)
- Presentation: word count hidden, slides visible, quiz card hidden
- Article/Pearl: word count visible, slides hidden, quiz card with toggle
Quiz card redesign: gradient blue/indigo background, white header with
icon, clean body — replaces the plain grey rectangle.
REVERT: httpOnly cookie auth removed — reverts to JWT in localStorage.
The cookie implementation broke the login page (always-hidden auth screen).
localStorage token approach is restored fully:
- Login/register return token in JSON body
- getAuthHeaders() sends Authorization: Bearer header
- Session check reads localStorage token on load
- clearSession() removes localStorage token
Fix: slide preview padding — section elements now have 40px 48px padding
so text is not flush against the frame edges.
Kept from v3.14: AI panel context-aware options (word count, slide count,
quiz question toggle), delete confirmation wording per content type.
AI Generate panel:
- Content type synced from editor on open
- Article/Pearl: optional word count field
- Presentation: optional slide count field
- Questions row: checkbox toggle for article/pearl (optional), always on for quiz,
hidden for presentation; count input shown only when enabled
- Backend: passes wordCount/slideCount to prompt; questionCount=0 skips questions
Delete confirmation:
- Wording built dynamically from content type
- Presentation: no mention of questions
- Quiz: "All quiz questions will be permanently removed"
- Article/pearl: "Any quiz questions attached will also be removed"
Auth refactor — JWT → httpOnly cookie:
- Backend: setAuthCookie() sets ped_auth as httpOnly, Secure, SameSite=Lax, 7d
- Backend: clearAuthCookie() on POST /api/auth/logout (new endpoint)
- Backend: auth middleware reads ped_auth cookie first, falls back to Bearer header
- Login/register: no longer return token in JSON body; cookie is the session
- Frontend: getAuthHeaders() returns only Content-Type (no Authorization header)
- Frontend: session check uses /api/auth/me via cookie (no localStorage token)
- Frontend: clearSession() calls /api/auth/logout to clear cookie server-side
- Frontend: no token in localStorage; window.AUTH_TOKEN removed
- All FormData fetches use credentials:'same-origin' (cookie sent automatically)
- index.html: optimistic auth-screen hide (removed localStorage token check)
Tested: login, logout, /api/auth/me, admin routes, FormData uploads, cookie httpOnly
flag, 401 without cookie, cookie cleared on logout.
Delete bar bug:
- Root cause: .lh-delete-confirm-bar{display:flex} defined after .hidden{display:none}
so flex overrode hidden (no !important on global .hidden rule)
- Fix: remove display:flex from base rule, use .lh-delete-confirm-bar:not(.hidden){display:flex}
so hidden class always wins regardless of CSS order
Slide preview (replace window.open + document.write approach):
- Backend /preview-slides now returns JSON: {css, slides[]} (individual <section> HTML)
- In-page full-screen modal with dark overlay, white slide frame
- Prev/Next arrow buttons, dot indicators, slide counter
- Keyboard navigation (arrow keys, Escape to close)
- Touch/swipe support for mobile
- Overlay click closes modal
- No popups required, works in all browsers
Marp textarea:
- Placeholder updated to guide users: "Click AI Generate above to create slides"
- Makes it clear AI is the primary authoring path; textarea for review/edits
Delete confirmation:
- Replaced browser confirm() popup with inline red warning bar in editor
- Shows content title, Cancel and Delete buttons inline in the editor header
- Category delete uses double-click pattern with toast feedback
Login screen:
- Changed from dark blue/purple gradient to light gray background (var(--g50))
- Card now matches 404 page style: white, subtle blue shadow, indigo border
Presentation content type:
- New 'Presentation' button in CMS toolbar (green style)
- Marp markdown textarea editor (dark code editor style) shown when type=presentation
- Body/Tiptap editor hidden; Marp section shown (toggleEditorMode)
- Preview button: renders Marp HTML via @marp-team/marp-core, opens in new tab
- Download PPTX: pure-JS pptxgenjs (no Chromium), parses slides, exports .pptx
- AI generate: returns Marp markdown directly when contentType=presentation
- Body column stores Marp markdown for presentations
Backend:
- POST /api/admin/learning/generate-pptx — pptxgenjs PPTX from Marp markdown
- POST /api/admin/learning/preview-slides — Marp HTML preview
GitHub repo set to private via API.
Backend (src/routes/learningAI.js):
- POST /api/admin/learning/ai-generate
* Accepts: topic text, uploaded file (PDF/TXT/MD/HTML), or Nextcloud WebDAV path
* Extracts text from PDFs via pdf-parse; plain text read directly
* Fetches WebDAV files using stored Nextcloud credentials
* Prompts AI to return structured JSON: title, subject, body (HTML), questions[]
* Strips code fences, falls back to regex JSON extraction if needed
- POST /api/admin/learning/ai-refine — refine existing body with instructions
- GET /api/admin/learning/webdav-browse — PROPFIND-based Nextcloud file browser
- POST /api/admin/learning/webdav-path — save user's default browse path
Frontend:
- AI Generate button in CMS editor header (gradient purple→blue)
- Panel with 3 source tabs: By Topic / Upload File / Nextcloud
- Drag-and-drop file dropzone with DataTransfer API support
- WebDAV file browser: navigate folders, select files, breadcrumb path
- Options: question count, content type (article/quiz/pearl), model selector, instructions
- Refine Current Body: prompt-driven AI rewrite of existing content
- Generated content auto-fills title, subject, type, Tiptap body, quiz questions
Settings:
- Nextcloud section: "Learning Hub Default Browse Path" field (shown when connected)
- Saves to webdav_learning_path column via /api/admin/learning/webdav-path
DB: ALTER TABLE users ADD COLUMN IF NOT EXISTS webdav_learning_path TEXT
- Bundle Tiptap 2 (StarterKit, Link, Underline) with esbuild into
public/vendor/tiptap.bundle.js — fully self-hosted, no CDN needed
- Remove Quill (quill.min.js, quill.snow.css, patchQuillTooltip hack)
- Headless Tiptap editor: custom toolbar (bold/italic/underline/strike,
headings, lists, blockquote, code, link) with active-state highlighting
- Link insertion: inline link bar below toolbar (no popup, no tooltip)
— type URL, press Apply or Enter; Remove button to unset
- Mini editor for question text and option text (same toggle UX)
- ProseMirror content styles: headings, lists, blockquote, code, links
Quill tooltip: move tooltip element to document.body via patchQuillTooltip()
so overflow:hidden on .cms-main cannot clip it. Recalculates position from
container-relative to viewport (fixed) coordinates with viewport clamping.
Removes overflow:hidden from quill wrappers; applies border-radius to toolbar/container.
Registration: hide #show-register by default in HTML; reveal only after
registration-status fetch confirms enabled=true. Endpoint already returns 403
when disabled — no backend change needed.
- wellVisit.js now handles tabChanged for vaxschedule and catchup tabs
so renderFullSchedule() and renderCatchUp() fire when those tabs open
- Replace Quill CDN with local /vendor/quill.min.js + quill.snow.css
(CDN was unreachable in local/offline environments)
- Replace textarea+toolbar with Quill.js (free, MIT) for body editor
- Full WYSIWYG: headings, bold/italic, lists, blockquote, code, link (inline, no popup)
- Rich Text toggle button on each question text field and option text field
- Plain textarea by default; toggle activates mini Quill editor per field
- New 'Multiple Select' question type: checkboxes, all correct must be chosen
- Backend scoring: allCorrectChosen && noWrongChosen
- Results show all correct options highlighted, wrong selections flagged
- quiz display uses sanitizeHtml for question/option text (supports formatted HTML)
- sanitizeHtml allows Quill class attrs (ql-indent, etc.)
All JS modules were running immediately (IIFE/DOMContentLoaded) but
tab HTML is lazy-loaded from /components/*.html, causing null errors
on every addEventListener call. Each module now listens for tabChanged
with its tab name and initializes once after the DOM is ready.
Also: quiz question/explanation boxes changed to textarea for
multi-line support; quiz display uses sanitizeHtml() so formatted
text (bold, lists, etc.) renders correctly.
