FINAL WORKING SOLUTION:
Previous attempts failed because:
- transformers.js v2.17.2 is ES module-only
- Module workers require complex CSP and external imports
- importScripts() doesn't work with ES modules
Solution:
- Use transformers.js v2.6.2 (has worker-compatible UMD build)
- Bundle library + models, serve entirely from our server
- Classic worker with importScripts() - no CSP issues
What's self-hosted:
- ✅ transformers.min.js (760KB) - at /models/transformers.min.js
- ✅ Whisper models (42MB) - at /models/Xenova/whisper-tiny.en/
Worker loads:
1. importScripts('/models/transformers.min.js') - OUR SERVER
2. Loads models from /models/ - OUR SERVER
3. ZERO external network calls
4. Works in any network (firewalled, air-gapped, etc.)
This is the production-ready, truly offline solution.
Issue: transformers.js is an ES module package and cannot be loaded
with importScripts() in classic workers.
Solution:
- Changed to module worker (type: 'module')
- Import transformers.js from CDN as ES module
- Models (42MB) still served from local server at /models/
Trade-off:
- Library (900KB): Loads from cdn.jsdelivr.net once, cached
- Models (42MB): Self-hosted, served from /models/ (no CDN)
This is necessary because:
1. @xenova/transformers is ES module-only (package.json: "type": "module")
2. ES modules cannot use importScripts()
3. Module workers require HTTPS for imports
4. CDN is HTTPS and cacheable
If CDN is blocked:
- Use Web Speech API (with privacy warnings)
- OR use Server Transcription (Vertex AI/AWS)
Models remain self-hosted as they're 40MB+ and contain the AI.
Provides two transcription options:
1. Browser Whisper (Offline, Batch) - RECOMMENDED
- 100% offline, zero network calls
- HIPAA-compliant, audio never leaves device
- Highest accuracy (Whisper)
- Processes after recording (batch mode)
- Models self-hosted, bundled in v2
2. Web Speech API (Real-time, Streaming) - EXPERIMENTAL
- Real-time transcription (see words as you speak)
- Uses browser's built-in speech recognition
- ⚠️ Sends audio to cloud (Chrome/Edge → Google)
- ⚠️ NOT HIPAA-compliant
- Requires user consent with clear warnings
Features:
- Settings UI for both options
- Clear privacy warnings for Web Speech
- Mutual exclusion (only one active at a time)
- Browser detection shows which provider is used
- Confirmation dialog before enabling Web Speech
Use Cases:
- Clinical/HIPAA: Use Browser Whisper only
- Personal/Non-clinical: Can use Web Speech for real-time feedback
- Maximum privacy: Browser Whisper (offline)
- Maximum speed: Web Speech (if privacy not required)
Implementation:
- speechRecognition.js: Web Speech API wrapper
- transcriptionSettings.js: Settings UI handler
- Privacy info displayed per browser
User can choose based on their privacy vs. speed preference.
Features:
- Admin can add, edit, and delete developmental milestones via dashboard
- Milestones stored in PostgreSQL (developmental_milestones table)
- Client-side loads milestones from API instead of static file
- Import script to migrate existing static data to database
- Organized by age group and domain
- Supports sorting and filtering
Admin UI:
- New section in Admin panel for milestone management
- Filter by age group
- Add/Edit modal with validation
- Delete with confirmation
- Auto-complete for age groups and domains
API Endpoints:
- GET /api/milestones-data - Public endpoint for authenticated users
- GET /api/admin/milestones - List all milestones (admin only)
- GET /api/admin/milestones/meta - Get age groups and domains
- POST /api/admin/milestones - Create milestone
- PUT /api/admin/milestones/:id - Update milestone
- DELETE /api/admin/milestones/:id - Delete milestone
- POST /api/admin/milestones/bulk-import - Bulk import
Usage:
1. Run import script: node scripts/import-milestones.js
2. Access Admin dashboard → Developmental Milestones section
3. Add/Edit/Delete milestones as needed
BREAKING FIX: Browser Whisper now fully self-contained
Previous issue:
- Loaded transformers.js from cdn.jsdelivr.net
- Downloaded models from cdn-lfs.huggingface.co
- Failed in corporate/clinical networks with firewall
- Stuck at "Initializing..." with no progress
Solution:
- Bundle transformers.js library (~876KB)
- Bundle Whisper tiny.en model (~42MB)
- Serve everything from local server
- Works in ANY network environment
Changes:
- whisperWorker.js: Load transformers from /models/ instead of CDN
- Dockerfile: Download models during Docker build
- Add download script for local dev
- Add comprehensive setup documentation
Docker image size: +~42MB (one-time cost, runtime benefit)
Tested: Works on unrestricted and firewalled networks
REALITY CHECK: Browser Whisper CDN loading cannot work in all environments
- Corporate firewalls block cdn.jsdelivr.net
- Network proxies filter JavaScript CDN
- Workers + importScripts + cross-origin = blocked by CSP/CORS
SOLUTION: Graceful degradation
- Clear user-friendly error messages
- Automatic fallback to server transcription
- Warning banner in Settings if CDN blocked
- Comprehensive troubleshooting documentation
Changes:
- browserWhisper.js: Show toast on worker error, fallback gracefully
- app.js: Display CSP warning banner on preload failure
- settings.html: Add warning about network/firewall requirements
- BROWSER_WHISPER_TROUBLESHOOTING.md: Complete guide for users
Key Message:
Browser Whisper is OPTIONAL. Server transcription (Google/AWS/OpenAI)
is the primary method and works everywhere. Browser Whisper is a
privacy-focused bonus feature that requires CDN access.
User Experience:
- If CDN works: Great! Browser Whisper available
- If CDN blocked: No problem! Server transcription works perfectly
- Clear messaging: User knows what to expect
- Allow empty voice value to preview server default
- Display 'server default' in preview text
- Clears user preference (sets to null) when testing default
ROOT CAUSES FOUND AND FIXED:
1. TTS Preview not working: voicePreferences.js listening for wrong event
- Was: 'tab-loaded' (never dispatched)
- Now: 'tabChanged' (correct event name used by app.js)
- Added immediate init if page already loaded
- Added 500ms delay for DOM readiness
2. Browser Whisper CDN blocked: CSP too restrictive
- Added 'unsafe-eval' to scriptSrc (required by transformers.js)
- Added cdn.jsdelivr.net to connectSrc (worker importScripts)
- Added childSrc directive for worker script loading
- Better error messages in worker
3. Worker loading errors: Now logged with specific reasons
- importScripts wrapped in try-catch
- Posts error message to main thread
- Verifies transformers object exists after load
Testing:
- TTS Preview should now work when clicking Settings tab
- Browser Whisper should load from CDN (or show specific error)
- Console logs will show exact init sequence
The button was always returning null because it searched for onclick=speakText
but all output cards use data-action="speak" data-target="id". Now checks
data-action first so the button correctly toggles to Stop during playback.
- Raise express.json limit from 1MB to 10MB — handles large chart reviews
with many notes (50 full clinic notes ≈ 600KB, well within new limit)
- Client-side: warn user if payload >8MB, friendly toast if >30 notes
- Bump to v13.0.0
- TTS response now includes X-TTS-Provider header (google-tts, litellm/model, elevenlabs)
- Frontend reads header and shows actual provider in toast instead of hardcoded "Adam/ElevenLabs"
- CORS exposes X-TTS-Provider header so frontend can access it
- Updated .env.example: clarify that LITELLM_TTS_MODEL and LITELLM_STT_MODEL
can be either the model_name alias OR the full provider/model path depending
on your LiteLLM config (important for BAA compliance routing)
- TTS: switch from OpenAI SDK to axios (same fix as STT), drop voice
param since it's configured inside LiteLLM per model
- Fix 'ElevenLabs unavailable' toast shown even when provider is LiteLLM
- Add dedicated red Stop button to encounter recording UI
Old pedscribe-v11 cache was serving stale admin.js to browsers
even after server updates. New cache name forces old SW to
deactivate and all clients to get fresh JS on next load.
Also switch JS/CSS from stale-while-revalidate to network-first
so code fixes are picked up immediately.
- LITELLM_MODELS = [] — no hardcoded models, global selector now only
shows what admin has actually added via Search API
- getAvailableModelsWithOverrides: for LiteLLM returns only custom list
- Remove toggle safety check — admin can disable any/all models freely
- Admin panel always reloads on tab open (was cached, showing stale data)
- Add 'Clear all models' button for LiteLLM to wipe and start fresh
- Add POST /config/models/clear-all endpoint
- Root cause: PUT /config/:key(*) wildcard was registered before
/config/models/toggle and /config/models/default, intercepting them
and returning "value is required" (body had modelId not value)
- Fix: move all model-specific PUT routes before the wildcard
- LiteLLM: return empty built-in list with discovery hint (hardcoded
models don't match user's proxy — must use Search API)
- After adding a discovered model: auto-select it in the default dropdown
- GET /config/models now returns defaultModel so dropdown pre-selects it
- Add Vertex AI provider (Gemini models via @google-cloud/vertexai SDK)
- Add LiteLLM proxy support (OpenAI-compatible, routes to any provider)
- Admin panel: model search/discover from provider API, enable/disable, custom models, set default
- New endpoints: /config/models/discover, /config/models/add-discovered, /config/models/default
- Updated models.js with VERTEX_MODELS and LITELLM_MODELS lists
- Updated health endpoint with vertex + litellm status
Audio Backups:
- New audio_backups table in PostgreSQL (bytea, gzip compressed)
- POST /api/audio-backups — upload with gzip compression (level 6)
- GET /api/audio-backups — list user's backups
- GET /api/audio-backups/:id/audio — download decompressed audio
- DELETE /api/audio-backups/:id — delete backup
- Auto-cleanup every hour (24h expiry)
- Frontend saves to server first, falls back to IndexedDB
- Settings shows source badge (server/local) per backup
AI Corrections:
- Corrections list is now expandable — click to view original vs corrected
- Shows red "Original" and green "Corrected to" sections
- Click arrow to expand/collapse each correction
- Date shown on each correction
The admin model management (enable/disable, custom models, default override)
had persistent rendering issues. Removed the UI panel — models are managed
via the global model selector in the header, which works reliably. Backend
API endpoints for model config are retained for future use.
- Add GET /api/transcribe/status endpoint — returns whether any server
transcription provider (Whisper/AWS/Local) is configured
- Frontend checks status on login via checkTranscribeStatus()
- When no provider configured: recording stops instantly, keeps live
Web Speech API text, shows friendly toast — no error, no upload wait
- Works in encounter, dictation, and SOAP tabs
- App now works fully out-of-the-box with just an AI provider key
- AWS Transcribe: remove 10ms delay between chunks (was adding ~1.25s/MB),
increase chunk size from 8KB to 32KB (AWS max per frame)
- Add detailed timing logs (ffmpeg, streaming, total) for diagnostics
- OpenAI Whisper: use response_format='text' for faster response parsing
- Frontend: show transcription time in toast, request 16kHz sample rate,
increase bitrate to 32kbps Opus (better quality, still small files)
- Return duration in API response for all providers
- APK: Add WAKE_LOCK, BOOT_COMPLETED, ACCESS_NETWORK_STATE permissions
- APK: Disable allowBackup for medical data security
- APK: AudioRecordingService now acquires wake lock, has stop action in notification
- Serve /.well-known/assetlinks.json for TWA domain verification
- Service worker: cache app shell, stale-while-revalidate for assets, network-first for API
- Admin model management: validate model ID format, prevent built-in conflicts, audit toggle actions, prevent disabling all models
- Bump version to v9.0.0, Docker tag to v9
- Full model management UI in admin panel: toggle models on/off, add custom
model IDs (any OpenRouter/Bedrock ID), set admin-configured default model
- /api/models now returns admin-set default model, frontend respects it
- Toggle switch CSS for clean enable/disable UX
- Backend already had the API endpoints, this adds the missing UI
- Add local Whisper (whisper.cpp / faster-whisper) as transcription provider
Set TRANSCRIBE_PROVIDER=local with configurable model size and binary path
- Upgrade all refine/instruction inputs to resizable textareas across
encounter, dictation, hospital course, chart review, well visit, sick visit
- Make AI memory injection flexible: physician preferences and corrections
are now actively applied (not just "formatting reference"), while still
overridable by current prompt instructions
- OIDC callback now passes only ?sso=ok flag, token stays in
httpOnly cookie (prevents token leaking to logs/referrer/history)
- Frontend auth.js uses cookie-based auth for SSO flow
- Add [PHYSICIAN TEMPLATES] boundary markers around physicianMemories
in all 5 generation routes to mitigate prompt injection
- Consistent boundary format across wellVisit, sickVisit, hpi, soap,
hospitalCourse
OIDC/SSO:
- New /api/auth/oidc route with PKCE for secure authorization
- Supports Azure AD, Okta, Keycloak, PocketID, Google, any OIDC provider
- Admin configurable: issuer, client ID/secret, button label
- Option to disable local auth (force SSO only)
- Auto-creates users on first SSO login, links existing by email
- SSO button on login page, hidden until admin enables OIDC
Firefox:
- Show info toast on first recording that live preview requires
Chrome/Edge; server-side transcription still works in all browsers
Previously only well-visit and sick-visit used saved physician
templates. Now HPI encounter, HPI dictation, SOAP, and hospital
course all fetch getUserMemoryContext() and pass physicianMemories
to the backend so the AI learns from saved templates/preferences.
- Add deduplication logic to prevent Chrome Speech API from repeating
sentences during long recording sessions (all 4 recording modules)
- Enable AWS Transcribe Medical with PRIMARYCARE specialty in .env
- Bump version to 8.0.0
- Fix word repetition: use sessionFinals pattern so each browser SR
session starts fresh; no overlap when recognition auto-restarts
- Fix HTML injection / '>' parse error: escape < > & in live transcript
innerHTML before inserting speech recognition text
- Add 24 MB blob guard: fall back to live SR transcript if audio file
is too large for Whisper API (long sessions)
- Bump version to 7.0.0, update docker-compose image tag to v7
- Add type="button" to all CMS action buttons to prevent default submit
- Change overflow:hidden to overflow:visible on .cms-main to prevent clipping
- Add z-index to .cms-editor-actions for reliable click targeting
- Replace AI "Topic" text input with a descriptive textarea context box
so users can tell AI what to generate (AI creates its own title)
- Rename AI tab from "By Topic" to "Describe Content"
Search (was completely broken):
- wireSearch() registered inside tabChanged for 'learning' tab — runs
after component HTML is in the DOM (lh-search existed but listener
was never attached because element was null at IIFE load time)
- Proper debounced search with 350ms delay, Escape key to clear
- Search results show count, 'X results for "query"', Clear Search button
- Empty state with helpful message; loading state while fetching
Content viewer showing feed items beneath it (CSS cascade bug):
- .lh-feed{display:flex} and .lh-category-bar{display:flex} defined
at CSS lines 513/507 — after .hidden{display:none} at line 242
So classList.add('hidden') was silently overridden
- showFeed() and showViewer() now use element.style.display with
explicit values ('flex'/'block'/'none') which always win over class rules
- viewer started hidden via class="hidden"; now shown with style.display='block'
- New GET /api/learning/content/:slug/slides (authMiddleware only)
Renders Marp markdown via marp-core, returns {css, slides[]} for
any authenticated user — no moderator access required
- showViewer: when content_type === 'presentation', hides the body card
and shows a presentation card with title, slide count, and View Slides button
Auto-opens the slide modal immediately on load; button allows re-opening
- openSlidesFromSlug(): fetches slides from user-accessible endpoint
then opens the existing slide preview modal (arrow keys, swipe, dots)
- Viewer already correctly hides feed/categories/search when viewing content;
"Back to Feed" button returns to the main list
Login flash:
- Auth screen now hidden by CSS default (#auth-screen{display:none !important})
- auth.js shows it only when there is no valid token or token is expired
- No more flash of login page on refresh for logged-in users
Presentation quiz option (AI Generate panel):
- Presentation type now shows optional quiz question toggle (same as article)
- Backend: when questionCount>0, returns JSON {marpMarkdown, questions[]}
instead of plain Marp markdown
- applyAiContent fills both the Marp textarea and question blocks for presentations
Viewer revert:
- Removed the viewer presentation card change (user wants markdown as-is)
- Removed dangling viewer button wiring
Feed/CMS list:
- Presentation type shows correct icon (fa-presentation-screen) and
label "Slides" in the CMS content list
pdf-parse (was v2, broken API):
- Downgraded to v1.1.1 — default export is a function again
- pdfParse is not a function error fixed
WebDAV file selection (style.display bug, same cascade issue):
- lh-ai-webdav-selected had inline style="display:flex" always visible
- selectWebdavFile() / deselectWebdavFile() now use element.style.display
- On select: browser div hides, selected indicator shows with file name + X
- On deselect (X): indicator hides, browser shows again
Topic context for Upload and Nextcloud tabs:
- Both tabs now have optional "Topic / context" field
- Sent to backend as 'topic' param to help AI focus the generated content
Inline refine bar (replaces window.prompt):
- Refine Body button shows/hides an amber inline input bar below generate buttons
- Enter instructions, press Apply — no browser dialog, works in all contexts
CSP: remove unsafe-inline from scriptSrc (security issue #2):
- Converted all 26 onclick= handlers in component HTML files to
data-action / data-target / data-label attributes
- Added delegated click handler in app.js for copy/speak/nc-export actions
- script-src now 'self' only; script-src-attr 'none'
WebDAV path endpoint (security issue #5):
- New POST /api/user/webdav-path (authMiddleware only, not moderator)
- nextcloud.js updated to use new endpoint
nodemailer: already at 6.10.1 (vulnerability fixed).
Bug: classList.toggle('hidden') was silently ignored because
.lh-ai-opt-field{display:flex} is defined at CSS line 758, after
.hidden{display:none} at line 242 — same specificity, later rule wins.
Fix: updateAiOptions() now uses element.style.display directly (inline
styles always override class rules).
Also fix initial state: lh-ai-slides-field now uses style="display:none"
instead of class="hidden" so it's correctly hidden on load.
Panel syncs content type from editor on open (editorType → ctype select
→ updateAiOptions called immediately).
Per type:
- Quiz: word count hidden, slides hidden, quiz card visible (no toggle)
- Presentation: word count hidden, slides visible, quiz card hidden
- Article/Pearl: word count visible, slides hidden, quiz card with toggle
Quiz card redesign: gradient blue/indigo background, white header with
icon, clean body — replaces the plain grey rectangle.
REVERT: httpOnly cookie auth removed — reverts to JWT in localStorage.
The cookie implementation broke the login page (always-hidden auth screen).
localStorage token approach is restored fully:
- Login/register return token in JSON body
- getAuthHeaders() sends Authorization: Bearer header
- Session check reads localStorage token on load
- clearSession() removes localStorage token
Fix: slide preview padding — section elements now have 40px 48px padding
so text is not flush against the frame edges.
Kept from v3.14: AI panel context-aware options (word count, slide count,
quiz question toggle), delete confirmation wording per content type.
AI Generate panel:
- Content type synced from editor on open
- Article/Pearl: optional word count field
- Presentation: optional slide count field
- Questions row: checkbox toggle for article/pearl (optional), always on for quiz,
hidden for presentation; count input shown only when enabled
- Backend: passes wordCount/slideCount to prompt; questionCount=0 skips questions
Delete confirmation:
- Wording built dynamically from content type
- Presentation: no mention of questions
- Quiz: "All quiz questions will be permanently removed"
- Article/pearl: "Any quiz questions attached will also be removed"
Auth refactor — JWT → httpOnly cookie:
- Backend: setAuthCookie() sets ped_auth as httpOnly, Secure, SameSite=Lax, 7d
- Backend: clearAuthCookie() on POST /api/auth/logout (new endpoint)
- Backend: auth middleware reads ped_auth cookie first, falls back to Bearer header
- Login/register: no longer return token in JSON body; cookie is the session
- Frontend: getAuthHeaders() returns only Content-Type (no Authorization header)
- Frontend: session check uses /api/auth/me via cookie (no localStorage token)
- Frontend: clearSession() calls /api/auth/logout to clear cookie server-side
- Frontend: no token in localStorage; window.AUTH_TOKEN removed
- All FormData fetches use credentials:'same-origin' (cookie sent automatically)
- index.html: optimistic auth-screen hide (removed localStorage token check)
Tested: login, logout, /api/auth/me, admin routes, FormData uploads, cookie httpOnly
flag, 401 without cookie, cookie cleared on logout.
Delete bar bug:
- Root cause: .lh-delete-confirm-bar{display:flex} defined after .hidden{display:none}
so flex overrode hidden (no !important on global .hidden rule)
- Fix: remove display:flex from base rule, use .lh-delete-confirm-bar:not(.hidden){display:flex}
so hidden class always wins regardless of CSS order
Slide preview (replace window.open + document.write approach):
- Backend /preview-slides now returns JSON: {css, slides[]} (individual <section> HTML)
- In-page full-screen modal with dark overlay, white slide frame
- Prev/Next arrow buttons, dot indicators, slide counter
- Keyboard navigation (arrow keys, Escape to close)
- Touch/swipe support for mobile
- Overlay click closes modal
- No popups required, works in all browsers
Marp textarea:
- Placeholder updated to guide users: "Click AI Generate above to create slides"
- Makes it clear AI is the primary authoring path; textarea for review/edits
Delete confirmation:
- Replaced browser confirm() popup with inline red warning bar in editor
- Shows content title, Cancel and Delete buttons inline in the editor header
- Category delete uses double-click pattern with toast feedback
Login screen:
- Changed from dark blue/purple gradient to light gray background (var(--g50))
- Card now matches 404 page style: white, subtle blue shadow, indigo border
Presentation content type:
- New 'Presentation' button in CMS toolbar (green style)
- Marp markdown textarea editor (dark code editor style) shown when type=presentation
- Body/Tiptap editor hidden; Marp section shown (toggleEditorMode)
- Preview button: renders Marp HTML via @marp-team/marp-core, opens in new tab
- Download PPTX: pure-JS pptxgenjs (no Chromium), parses slides, exports .pptx
- AI generate: returns Marp markdown directly when contentType=presentation
- Body column stores Marp markdown for presentations
Backend:
- POST /api/admin/learning/generate-pptx — pptxgenjs PPTX from Marp markdown
- POST /api/admin/learning/preview-slides — Marp HTML preview
GitHub repo set to private via API.
Backend (src/routes/learningAI.js):
- POST /api/admin/learning/ai-generate
* Accepts: topic text, uploaded file (PDF/TXT/MD/HTML), or Nextcloud WebDAV path
* Extracts text from PDFs via pdf-parse; plain text read directly
* Fetches WebDAV files using stored Nextcloud credentials
* Prompts AI to return structured JSON: title, subject, body (HTML), questions[]
* Strips code fences, falls back to regex JSON extraction if needed
- POST /api/admin/learning/ai-refine — refine existing body with instructions
- GET /api/admin/learning/webdav-browse — PROPFIND-based Nextcloud file browser
- POST /api/admin/learning/webdav-path — save user's default browse path
Frontend:
- AI Generate button in CMS editor header (gradient purple→blue)
- Panel with 3 source tabs: By Topic / Upload File / Nextcloud
- Drag-and-drop file dropzone with DataTransfer API support
- WebDAV file browser: navigate folders, select files, breadcrumb path
- Options: question count, content type (article/quiz/pearl), model selector, instructions
- Refine Current Body: prompt-driven AI rewrite of existing content
- Generated content auto-fills title, subject, type, Tiptap body, quiz questions
Settings:
- Nextcloud section: "Learning Hub Default Browse Path" field (shown when connected)
- Saves to webdav_learning_path column via /api/admin/learning/webdav-path
DB: ALTER TABLE users ADD COLUMN IF NOT EXISTS webdav_learning_path TEXT
