danvics/pediatric-ai-scribe-v3
1
0
Fork 0
Code Issues Pull requests Projects Releases 14 Packages Wiki Activity Actions
pediatric-ai-scribe-v3/docs/developer-guide.md
Daniel 09193538fb v6.2: Session management, password change, audit logging, refine context, UI fixes 
Security:
- Add session management: users can view/revoke active sessions in Settings
- Add password change in Settings (requires current password, HIBP check)
- Force logout all sessions on password reset
- Fix logout to destroy server-side session (was only clearing cookie)
- Add trust proxy for correct client IP in rate limiting and audit logs
- Add CORS support for multiple domains (CORS_ORIGINS env var)
- Add HIBP breach check endpoint and inline warnings on password fields

Audit logging:
- Add audit logging to all 24 PHI-handling endpoints across 13 route files
- Covers: generation, transcription, TTS, refine, encounters, documents, Nextcloud
- All fire-and-forget (no response delay)

AI improvements:
- Refine now includes original source material (transcript, notes, labs)
  so AI can reference the full input when modifying output
- Add correction tracking (trackAIOutput) to sick visit and well visit tabs
- Fix sickvisit missing from encounter save noteIdMap

UI fixes:
- Non-blocking busy bar for transcription and AI generation (replaces full-screen overlay)
- Fix encounter recording: hide record button during recording (was showing two stop buttons)
- Fix ROS/PE "All WNL" stacking duplicate event handlers; add Clear buttons
- Enlarge AI instructions textarea in Learning Hub CMS

Domain:
- Primary domain now app.pedshub.com, with scribe.pedshub.com and peds.danvics.com as CORS origins
2026-04-08 20:27:45 +02:00

467 lines
19 KiB
Markdown
Raw Blame History

# Developer Guide
This guide explains how the codebase works so any developer can understand, modify, and extend the Pediatric AI Scribe platform.
---
## Project Structure
```
server.js -- Express app entry point, middleware stack, route registration
src/
db/database.js -- PostgreSQL pool, schema init, query helpers, auto-cleanup
middleware/
auth.js -- JWT/cookie auth, admin/moderator role checks
logging.js -- Request logging middleware
utils/
ai.js -- Multi-provider AI client (callAI), model discovery
models.js -- Built-in model definitions per provider
prompts.js -- All AI system prompts (overridable via DB)
config.js -- DB-backed settings with 2-minute cache
logger.js -- Audit, API, access logging to DB + files
embeddings.js -- Vector embedding generation (Vertex/LiteLLM/OpenAI)
transcribeAWS.js -- Amazon Transcribe client
transcribeGoogle.js -- Google Gemini STT
transcribeLocal.js -- Local whisper.cpp / faster-whisper
ttsGoogle.js -- Google Cloud TTS
routes/ -- 27 route files (see below)
public/
index.html -- Main SPA shell, auth forms, script tags
sw.js -- Service worker (cache shell, network-first API)
manifest.json -- PWA manifest
components/ -- HTML fragments loaded into tabs
js/ -- 20+ vanilla JS modules
css/styles.css -- All styles in one file
icons/ -- PWA icons
models/ -- Self-hosted Whisper WASM model files
```
---
## How the Frontend Works
### SPA Architecture
This is a **vanilla JavaScript SPA** -- no React, Vue, or framework. The approach:
1. `index.html` is the only HTML page. It contains two top-level divs:
- `#auth-screen` -- login/register/forgot forms (hidden when authenticated)
- `#main-app` -- the actual application (hidden until authenticated)
2. **Tabs** are managed by `app.js`. The sidebar has tab buttons. Clicking a tab calls `activateTab(tabName)` which:
- Fetches `/components/{tabName}.html` via `loadComponent()`
- Injects the HTML into `.app-body`
- Dispatches a `CustomEvent('tabChanged', { detail: { tab: tabName } })`
- Other modules listen for this event to initialize their UI
3. **Module communication** uses `window` globals and `CustomEvent`:
- Functions exposed on `window` (e.g., `window.saveAudioBackup`, `window.getAuthHeaders`, `window.showToast`)
- Events dispatched on `document` (e.g., `recording-started`, `recording-stopped`, `tabChanged`)
4. **Script loading**: all JS files use `defer` attribute, loaded in dependency order defined in `index.html` (lines 302-328). `audioBackup.js` before `app.js` before `auth.js` etc.
### Auth Flow
`auth.js` runs on DOMContentLoaded:
1. Checks for SSO redirect (`?sso=ok` URL param)
2. Tries to restore session from localStorage token or cookie
3. Calls `/api/auth/me` to validate
4. If valid: hides auth screen, shows main app, loads default tab
5. If invalid: shows auth screen
Token is stored in both `localStorage` (for Bearer header) and `ped_auth` cookie (for SSO/cookie-based auth). The auth middleware accepts either.
### Component Lifecycle
When a tab is activated:
1. HTML is fetched and injected into `.app-body`
2. The `tabChanged` event fires
3. Each module has a listener that initializes when its tab is active:
```javascript
document.addEventListener('tabChanged', function(e) {
if (e.detail.tab === 'settings') {
loadMemories();
renderAudioBackups();
}
});
```
4. Modules query the DOM for elements inside the injected component HTML
### Common Patterns
**API calls**: Always use `getAuthHeaders()` for JSON requests, or manually add Bearer token for FormData uploads. Include `credentials: 'same-origin'` when cookie auth may be needed.
**Toast notifications**: `showToast(message, type)` where type is `success`, `error`, `info`, or `warning`.
**Loading overlay**: `showLoading(message)` and `hideLoading()`.
**Model selection**: `getSelectedModel()` returns the currently selected model ID from the tab's dropdown.
---
## How the Backend Works
### Middleware Stack (server.js)
Requests flow through this chain in order:
```
Request
-> Helmet (CSP headers)
-> CORS (restrict to APP_URL)
-> cookieParser
-> express.json (10MB limit)
-> Rate limiters (per-endpoint)
-> Static file serving (public/)
-> Route handlers
-> 404 fallback (serves index.html for SPA routes)
```
### Database Layer (src/db/database.js)
The database module provides:
- `db.get(sql, params)` -- single row (returns object or null)
- `db.all(sql, params)` -- multiple rows (returns array)
- `db.run(sql, params)` -- INSERT/UPDATE/DELETE (returns `{ lastInsertRowid, changes }`)
- `db.query(sql, params)` -- raw pg query
- `db.getSetting(key)` -- read from app_settings
- `db.setSetting(key, value)` -- write to app_settings
SQL uses `?` placeholders which are auto-converted to PostgreSQL `$1, $2, ...` by `convertPlaceholders()`. You can also use `$N` directly.
For INSERT statements, `RETURNING id` is auto-appended if not already present.
**Schema migration**: all tables use `CREATE TABLE IF NOT EXISTS` and `ALTER TABLE ADD COLUMN IF NOT EXISTS`. Migrations run on every startup -- no separate migration tool needed. Just add new columns/tables to `initDatabase()`.
### Authentication Middleware (src/middleware/auth.js)
Three middleware functions:
- `authMiddleware` -- verifies JWT from Bearer header or `ped_auth` cookie. If Bearer header is present but empty, falls through to cookie. Sets `req.user`.
- `adminMiddleware` -- requires `req.user.role === 'admin'` (use after authMiddleware)
- `moderatorMiddleware` -- requires admin or moderator role
### AI Integration (src/utils/ai.js)
The `callAI(messages, options)` function is the single entry point for all AI calls:
```javascript
var result = await callAI([
{ role: 'system', content: systemPrompt },
{ role: 'user', content: userContent }
], { model: selectedModel });
// result = { text: "...", usage: { input, output } }
```
Internally, `callAI` routes to the active provider:
- **Bedrock**: uses `InvokeModelCommand` with Converse API
- **Azure/OpenRouter/LiteLLM**: uses OpenAI SDK `chat.completions.create()`
- **Vertex**: uses `@google-cloud/vertexai` GenerativeModel
Provider is selected once at startup. The `model` param in options overrides the default.
### Settings System (src/utils/config.js)
Database-backed configuration with in-memory caching:
```javascript
var config = require('../utils/config');
var value = await config.get('feature.read_aloud', 'true'); // key, default
await config.set('registration_enabled', 'false');
```
Cache TTL is 2 minutes. Settings are stored in the `app_settings` table. Environment variables take precedence for provider credentials, but most app settings are DB-backed.
### Prompt System (src/utils/prompts.js)
All AI prompts are defined as a `PROMPTS` object:
```javascript
module.exports = {
hpiEncounter: "You are a pediatric physician...",
soapFull: "Generate a complete SOAP note...",
// ... etc
};
```
On startup, DB overrides are loaded from `app_settings` where `key LIKE 'prompt.%'`. Admin can edit prompts from the Admin Panel without restarting.
### Logging (src/utils/logger.js)
```javascript
var logger = require('../utils/logger');
logger.audit(userId, 'action_name', 'details', req, { category: 'auth' });
logger.apiCall(userId, endpoint, { model, tokens_input, tokens_output, duration_ms });
logger.access(userId, 'login', req, true);
```
All log entries go to both the database and daily log files at `/data/logs/YYYY-MM-DD.log`.
---
## Adding a New Feature
### Adding a New AI Endpoint
1. Create a route file in `src/routes/`:
```javascript
var express = require('express');
var router = express.Router();
var { callAI } = require('../utils/ai');
var { authMiddleware } = require('../middleware/auth');
var PROMPTS = require('../utils/prompts');
router.post('/my-feature', authMiddleware, async function(req, res) {
try {
var { transcript, model } = req.body;
var result = await callAI([
{ role: 'system', content: PROMPTS.myFeature },
{ role: 'user', content: transcript }
], { model });
res.json({ success: true, text: result.text });
} catch (err) {
res.status(500).json({ error: err.message });
}
});
module.exports = router;
```
2. Register it in `server.js`:
```javascript
app.use('/api', require('./src/routes/myFeature'));
```
3. Add the prompt to `src/utils/prompts.js`:
```javascript
myFeature: "You are a pediatric physician. Generate..."
```
4. Create a frontend component in `public/components/myfeature.html`
5. Add a tab button in `public/index.html` sidebar
6. Create `public/js/myFeature.js` with a `tabChanged` listener
### Adding a New Database Table
Add the `CREATE TABLE IF NOT EXISTS` statement inside `initDatabase()` in `src/db/database.js`:
```javascript
try { await client.query(`
CREATE TABLE IF NOT EXISTS my_table (
id SERIAL PRIMARY KEY,
user_id INTEGER REFERENCES users(id) ON DELETE CASCADE,
data TEXT NOT NULL,
created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_my_table_user ON my_table(user_id);
`); } catch(e) {}
```
No migration files needed. The `IF NOT EXISTS` pattern is idempotent.
### Adding a New Setting
1. Add the default in the `defaults` array in `initDatabase()`:
```javascript
['my_setting.key', 'default_value'],
```
2. Read it in route handlers:
```javascript
var value = await db.getSetting('my_setting.key');
```
3. If it should be admin-editable, ensure the admin config route handles it (the generic `POST /api/admin/config` already saves any key-value pair).
---
## Key Design Decisions
### Why Vanilla JS (No Framework)
The frontend uses plain JavaScript instead of React/Vue because:
- Simpler deployment (no build step, no bundler)
- Components are HTML fragments loaded via fetch
- State is managed via DOM elements and window globals
- Works with aggressive CSP (no eval needed for templates)
- Easy to modify any part without understanding a framework's lifecycle
### Why PostgreSQL Placeholders Are Auto-Converted
The codebase was originally SQLite, then migrated to PostgreSQL. The `convertPlaceholders()` function in `database.js` converts `?` to `$1, $2, ...` so existing queries work unchanged. New code can use either style.
### Why Prompts Are DB-Overridable
Clinicians have specific documentation preferences. Making prompts editable from the admin panel means the team can tune AI output without redeploying. The `prompt.*` keys in `app_settings` override the hardcoded defaults in `prompts.js`.
### Why Audio Backups Use PostgreSQL (Not Filesystem)
Audio is stored as gzip-compressed BYTEA in PostgreSQL because:
- Works in containerized environments without persistent volumes for temp files
- Auto-expires via SQL (`expires_at` column + hourly cleanup)
- Per-user access control is handled by the same auth system
- No orphaned files if the container restarts
### Why Corrections Are Low-Priority Style Hints
The physician memory/correction system injects past edits into AI prompts. Originally these were labeled "APPLY these preferences" which caused smaller models to hallucinate content from the correction examples instead of the current transcript. The injection was changed to `[STYLE HINTS (low priority)]` with truncated 200-char snippets to prevent this.
---
## AI Learning System (Correction Tracker)
The app learns from physician edits over time, similar to Dragon Medical's adaptive learning. Here is how it works:
### Flow
1. **Track**: When AI generates a note, `trackAIOutput(elementId, originalText)` stores the original AI output in memory (`correctionTracker.js`).
2. **Edit**: The physician edits the generated note directly in the contenteditable output area.
3. **Save**: When the physician clicks Save, `saveCorrection(elementId, section)` compares the current text against the stored original.
4. **Store**: If there is a meaningful difference (more than 2 words or 20 characters changed), the before/after diff is sent to `POST /api/memories/correction` and stored in the `user_memories` table with category `correction_{section}`.
5. **Apply**: On future generations, the last 10 corrections per category are fetched via `GET /api/memories/context` and injected into the AI prompt as low-priority style hints.
### Which tabs support it
| Tab | trackAIOutput | saveCorrection (on Save) |
|-----|---------------|--------------------------|
| Live Encounter | Yes (`enc-hpi-text`) | Yes |
| SOAP | Yes (`soap-text`) | Yes |
| Dictation | Yes (`dict-hpi-text`) | Yes |
| Sick Visit | Yes (`sick-note-text`) | Yes |
| Well Visit | Yes (`wv-note-text`) | Yes |
| Hospital Course | No (output varies by format) | Yes (if tracked) |
| Chart Review | No (output varies by input) | Yes (if tracked) |
### Important notes
- Corrections are only captured when the user clicks **Save**. Editing without saving does not trigger learning.
- The system keeps a maximum of 20 corrections per category, auto-deleting the oldest.
- Corrections are injected as `[STYLE HINTS (low priority)]` with 200-character snippets to avoid confusing smaller AI models.
- Users can view and delete their corrections in Settings > AI Corrections.
### Why Auth Middleware Checks Cookie After Bearer
The auth middleware first checks the `Authorization: Bearer` header, then falls back to the `ped_auth` cookie. If a Bearer header is present but the token is empty (which happens with SSO-only users who have no localStorage token), the middleware now correctly treats it as absent and falls through to the cookie. This was a bug fix -- previously, an empty Bearer token would block cookie auth entirely.
---
## Route File Reference
| File | Mount Point | Auth | Purpose |
|------|------------|------|---------|
| `auth.js` | `/api/auth` | Public | Registration, login, 2FA, email verification, password reset |
| `oidc.js` | `/api/auth` | Public | OpenID Connect SSO flow |
| `hpi.js` | `/api` | Auth | HPI generation (encounter + dictation) |
| `soap.js` | `/api` | Auth | SOAP note generation |
| `chartReview.js` | `/api` | Auth | Chart review / precharting |
| `hospitalCourse.js` | `/api` | Auth | Hospital course generation |
| `wellVisit.js` | `/api` | Auth | Well visit + SSHADESS |
| `sickVisit.js` | `/api` | Auth | Sick visit documentation |
| `milestones.js` | `/api` | Auth | Developmental milestone narratives |
| `refine.js` | `/api` | Auth | Refine, shorten, clarify documents |
| `transcribe.js` | `/api` | Auth | Speech-to-text (5 providers) |
| `tts.js` | `/api` | Auth | Text-to-speech (3 providers) |
| `encounters.js` | `/api` | Auth | Save/load/delete encounters |
| `memories.js` | `/api` | Auth | Physician templates + corrections |
| `audioBackups.js` | `/api` | Auth | Audio backup storage |
| `documents.js` | `/api` | Auth | S3 document management |
| `userPreferences.js` | `/api` | Auth | STT/TTS preferences |
| `nextcloud.js` | `/api` | Auth | WebDAV integration |
| `logs.js` | `/api` | Auth | Usage and audit logs |
| `admin.js` | `/api/admin` | Admin | User management |
| `adminConfig.js` | `/api/admin` | Admin | Settings, prompts, models, SMTP, OIDC |
| `adminMilestones.js` | `/api/admin` | Admin | Milestone data management |
| `learningHub.js` | `/api/learning` | Auth | Learning content delivery + quizzes |
| `learningAdmin.js` | `/api/admin/learning` | Moderator | Learning CMS CRUD |
| `learningAI.js` | `/api/admin/learning` | Moderator | AI content generation, PPTX, slides |
---
## Frontend JS File Reference
| File | Loads After | Purpose |
|------|-------------|---------|
| `app.js` | audioBackup, correctionTracker | Tab navigation, model selector, AudioRecorder, transcription |
| `auth.js` | app.js | Login, register, SSO, session management, Turnstile |
| `liveEncounter.js` | auth.js | Recording UI, speech recognition, live transcript |
| `soap.js` | auth.js | SOAP note tab |
| `hospitalCourse.js` | auth.js | Hospital course tab |
| `chartReview.js` | auth.js | Chart review tab |
| `wellVisit.js` | auth.js | Well visit tab |
| `sickVisit.js` | auth.js | Sick visit tab |
| `encounters.js` | auth.js | Save/load/resume encounters |
| `milestones.js` | milestonesData.js | Milestone selection and generation |
| `shadess.js` | auth.js | SSHADESS adolescent assessment |
| `learningHub.js` | auth.js | Learning Hub + CMS (1843 lines) |
| `memories.js` | auth.js | Physician templates + corrections UI |
| `documents.js` | auth.js | S3 document upload/download |
| `admin.js` | auth.js | Admin panel (users, settings, prompts, models) |
| `audioBackup.js` | (early) | Audio backup save/list/retry/delete |
| `correctionTracker.js` | (early) | Track AI output edits for learning |
| `browserWhisper.js` | (early) | In-browser Whisper via WebAssembly |
| `speechRecognition.js` | (early) | Web Speech API wrapper |
| `voicePreferences.js` | auth.js | STT/TTS model/voice selection |
| `nextcloud.js` | auth.js | Nextcloud connection and export |
---
## Testing Locally
```bash
# Start just the database
docker compose up -d postgres
# Install dependencies
npm install
# Copy and configure env
cp .env.example .env
# Edit .env with your provider keys
# Start the app
node server.js
```
The app runs on `http://localhost:3000`. Without `APP_URL` set, CORS allows all origins (development mode).
## Common Tasks
### Change the default AI temperature
Edit the `callAI` function in `src/utils/ai.js`. The default temperature is `0.3` for most providers.
### Add a new AI prompt
1. Add the prompt text to `src/utils/prompts.js`
2. Use it in your route: `var PROMPTS = require('../utils/prompts'); ... PROMPTS.myPrompt`
3. It becomes admin-editable automatically via `prompt.myPrompt` in the DB
### Override a prompt without code changes
In the admin panel, go to Settings > Prompts. Edit any prompt. The override is stored in `app_settings` with key `prompt.{name}` and takes effect immediately (no restart needed).
### Add a model to the dropdown
From the admin panel, go to Models > Add Custom Model. Enter:
- **Model ID**: the exact string the provider expects (e.g., `gemini-2.5-flash` for LiteLLM)
- **Display Name**: what users see
- **Cost**: price string (e.g., `~$0.001`)
- **Category**: determines dropdown group (free/fast/smart/premium)
The model appears immediately for all users.
### Debug an AI call
Check `docker compose logs -f pediatric-scribe` for lines like:
```
[AI] bedrock/anthropic.agent-config-3-haiku... 1247 tokens in 2.3s
```
Or query the `api_log` table for detailed metrics:
```sql
SELECT endpoint, model_used, tokens_input, tokens_output, duration_ms, cost_estimate
FROM api_log ORDER BY timestamp DESC LIMIT 20;
```
Reference in a new issue View git blame Copy permalink