Daniel 1b3ea569b7 simplify speech and embeddings through litellm

2026-05-09 05:09:02 +02:00

14 KiB

Raw Permalink Blame History

Developer guide

How the codebase is organized, how the main subsystems work, and where to extend them.

Layout

server.js                        Express entry, middleware stack, route mount
Dockerfile                       node:20-alpine, argon2 native compile deps
docker-compose.yml               app + postgres services

migrations/                      node-pg-migrate versioned schema changes
scripts/
  maintenance.js                 REINDEX / drift CLI
  release.sh                     semver bump + tag + push
  import-milestones.js           one-off seed import

src/
  db/
    database.js                  pool, baseline init, query helpers
    migrate.js                   programmatic node-pg-migrate runner
  middleware/
    auth.js                      JWT + session-table validation + sliding idle
    logging.js                   request log
  utils/
    ai.js                        callAI() multi-provider router + model whitelist
    models.js                    built-in model registry
    prompts.js                   templates (DB-overridable)
    promptSafe.js                <UNTRUSTED_*> wrapping + INJECTION_GUARD
    crypto.js                    AES-256-GCM (PHI at rest)
    passwords.js                 argon2id + bcrypt fallback + rehash
    sessions.js                  token hash, UA parse, session id
    platform.js                  isMobileClient()
    redact.js                    PHI redactor for audit details
    auditQueue.js                batched audit/api/access writer
    fileType.js                  magic-byte upload verifier
    errors.js                    generic 500 responder
    logger.js                    audit + api + access + Loki shipper
    embeddings.js                LiteLLM embeddings
    notify.js                    ntfy push
    transcribe.js, tts.js        LiteLLM STT / TTS routes
  routes/                        Express routers for auth, AI workflows, education, logs, and user data

public/
  index.html                     SPA shell, version-stamped asset refs
  sw.js                          cache shell, network-first API
  manifest.json                  PWA
  js/                            24 vanilla JS modules (no bundler)
  components/                    per-tab HTML fragments loaded on demand
  css/styles.css
  template-guide.md              downloadable user template guide

mobile/                          Capacitor 6 wrapper (Android + iOS)
.github/workflows/               CI (auto-version, APK, docker)

Backend

Middleware stack (`server.js`)

request
  → helmet            CSP, HSTS, X-Content-Type-Options
  → CORS              fail-closed in prod if APP_URL/CORS_ORIGINS missing
  → cookieParser
  → express.json      10 MB cap
  → rate limiters     200/min general; 10/15min login; 20/15min sensitive
  → static            public/ with per-filetype Cache-Control; ?v=BUILD_ID cache-busts HTML on deploy
  → route handlers
  → 404 fallback      serves index.html for SPA routes

On boot:

APP_VERSION read from package.json, printed + returned by /api/health/detailed.
BUILD_ID = short git HEAD SHA (or random on non-git deploys). Rewritten into HTML at startup.
JWT_SECRET / DATA_ENCRYPTION_KEY fail-fast if missing in production.
initDatabase() → runMigrations() → collation drift check.
SIGTERM / SIGINT handler drains the audit queue and closes the pool.

DB helpers (`src/db/database.js`)

await db.get(sql, params);     // first row or null
await db.all(sql, params);     // array of rows
await db.run(sql, params);     // { lastInsertRowid, changes }
await db.query(sql, params);   // raw pg result
await db.getSetting(key);      // app_settings value (2 min cache)
await db.setSetting(key, v);   // writes + invalidates cache
db.pool                        // pg.Pool instance for transactions

SQL uses ? placeholders (auto-converted to $1, $2, ...) OR native $N. INSERTs without explicit RETURNING get RETURNING id appended.

Auth middleware (`src/middleware/auth.js`)

var { authMiddleware, adminMiddleware, moderatorMiddleware } = require('../middleware/auth');
router.post('/thing', authMiddleware, handler);            // requires auth
router.post('/admin-thing', authMiddleware, adminMiddleware, handler);

Sets req.user with { id, email, name, role, totp_enabled, disabled } and req.sessionId.

AI (`src/utils/ai.js`)

var { callAI } = require('../utils/ai');
var result = await callAI([
  { role: 'system', content: PROMPTS.hpiEncounter + INJECTION_GUARD },
  { role: 'user',   content: wrapUserText('transcript', transcript) }
], { model, maxTokens: 4000 });
// result = { content, model, usage }

Throws { code: 'model_not_permitted' } if the requested model isn't in the active allowlist.

Prompts (`src/utils/prompts.js`)

Flat PROMPTS object. Any template can be overridden live by writing a prompt.{name} row in app_settings. Admin Panel's Prompt Editor is the UX for that.

Settings (`src/utils/config.js`)

var v = await config.get('feature.read_aloud', 'false');   // key, default
await config.set('registration_enabled', 'true');

2-minute in-memory cache. Writes invalidate immediately.

Logger (`src/utils/logger.js`)

logger.audit(userId, 'action', 'details', req, { category: 'auth' });
logger.apiCall(userId, endpoint, { model, tokensInput, tokensOutput, duration });
logger.access(userId, 'login', req, true);
logger.error('scope', err.message);

Writes go to the database (batched), the daily JSONL file, and Loki (if configured).

Frontend

No framework, no bundler, no build step. public/index.html is the only document. Modules communicate through window.* globals and CustomEvent on document.

Tab system

app.js owns activateTab(name):

Fetch /components/{name}.html (browser-cached for 1 h, bust by ?v=BUILD_ID that the server injects).
Inject into .app-body.
Dispatch CustomEvent('tabChanged', { detail: { tab: name } }).
Feature modules (soap.js, encounters.js, etc.) listen for their own tab name and initialize DOM references inside the just-injected fragment.

Auth model (client)

auth.js branches on isNativeApp():

Web — no localStorage token; relies on the ped_auth httpOnly cookie set by the server. /api/auth/me on boot verifies the session; failed verification shows the login screen.
Mobile — token lives in capacitor-secure-storage-plugin (iOS Keychain / Android EncryptedSharedPreferences). getAuthHeaders() emits Authorization: Bearer <token>.

authFetch.js wraps window.fetch to catch 401 on authenticated /api/* requests and force re-login. BroadcastChannel('pedscribe-auth') propagates logout to sibling tabs.

Module load order

Fixed in index.html:

secureStorage → authFetch → auth → (feature modules)

All <script defer>. Dependencies enforced by declaration order.

Adding things

A new AI endpoint

Create src/routes/myFeature.js:

var express = require('express');
var router = express.Router();
var { callAI } = require('../utils/ai');
var { authMiddleware } = require('../middleware/auth');
var PROMPTS = require('../utils/prompts');
var { wrapUserText, INJECTION_GUARD } = require('../utils/promptSafe');
var logger = require('../utils/logger');

router.post('/my-feature', authMiddleware, async (req, res) => {
  try {
    var { transcript, model } = req.body;
    var result = await callAI([
      { role: 'system', content: PROMPTS.myFeature + INJECTION_GUARD },
      { role: 'user',   content: wrapUserText('transcript', transcript) }
    ], { model });
    res.json({ success: true, text: result.content });
    logger.audit(req.user.id, 'generate_my_feature', 'Generated', req, { category: 'clinical' });
  } catch (err) {
    res.status(500).json({ error: 'Request failed' });
  }
});
module.exports = router;

Mount in server.js:

app.use('/api', require('./src/routes/myFeature'));

Add the template to src/utils/prompts.js.
Add a component under public/components/myfeature.html.
Add public/js/myFeature.js with a tabChanged listener.
Register the tab button in public/index.html.

A new table / column

New changes go in a migration file. See docs/migrations.md.

docker exec -w /app pediatric-ai-scribe npm run migrate:new -- add_my_table
# edit the generated file

A new setting

Optional — add a default in the defaults array inside initDatabase() (only needed if the app should seed it on fresh installs).
Read at runtime: await db.getSetting('my_key').
Admin-editable automatically through PUT /api/admin/config which accepts arbitrary keys.

Physician Templates And Preferences

Settings saves user templates/preferences through /api/memories into user_memories.
New rows encrypt name and content with the shared enc1: string format.
GET /api/memories/context decrypts rows and returns only AI-context categories: physical_exam, ros, encounter_format, family_history, assessment_plan, template_soap, template_hpi, template_wellvisit, template_sickvisit, and template_ed.
custom rows remain visible in settings but are not included in prompt context.
Legacy correction_* rows from the removed correction-learning feature are filtered out rather than deleted.

Route reference

File	Mount	Auth	Purpose
`auth.js`	`/api/auth`	Public	Register, login, 2FA, email verify, password reset, backup codes
`oidc.js`	`/api/auth`	Public	OIDC SSO (Authorization Code + PKCE)
`sessions.js`	`/api/sessions`	Auth	Active sessions list + revoke
`hpi.js`	`/api`	Auth	HPI from encounter or dictation
`soap.js`	`/api`	Auth	SOAP generation
`chartReview.js`	`/api`	Auth	Chart review / pre-charting
`hospitalCourse.js`	`/api`	Auth	Hospital course
`wellVisit.js`	`/api`	Auth	Well visit + SSHADESS
`sickVisit.js`	`/api`	Auth	Sick visit
`milestones.js`	`/api`	Auth	Developmental milestone narratives
`refine.js`	`/api`	Auth	Refine / shorten / clarify
`transcribe.js`	`/api`	Auth	LiteLLM STT
`tts.js`	`/api`	Auth	LiteLLM TTS
`encounters.js`	`/api`	Auth	Save / load / optimistic-lock encounters
`memories.js`	`/api`	Auth	Templates + prompt preferences
`audioBackups.js`	`/api`	Auth	Encrypted audio retry store
`documents.js`	`/api`	Auth	S3 documents (magic-byte checked)
`userPreferences.js`	`/api`	Auth	Per-user STT/TTS choice
`nextcloud.js`	`/api`	Auth	Encrypted WebDAV tokens
`billing.js`	`/api`	Auth	ICD-10 / CPT code suggestion
`logs.js`	`/api`	Auth	Usage + audit dump (admin-only endpoints gated further)
`admin.js`	`/api/admin`	Admin	User management
`adminConfig.js`	`/api/admin`	Admin	Settings, prompts, models, SMTP, OIDC
`adminMilestones.js`	`/api/admin`	Admin	Milestone data management
`learningHub.js`	`/api/learning`	Auth	Content delivery + quizzes
`learningAdmin.js`	`/api/admin/learning`	Moderator	CMS CRUD
`learningAI.js`	`/api/admin/learning`	Moderator	AI content gen, PPTX export

Frontend JS module reference

File	Purpose
`secureStorage.js`	Platform-branched token storage (Keychain / localStorage)
`authFetch.js`	Global fetch wrapper (401 → logout + reload)
`auth.js`	Login / register / SSO / session management / Turnstile / backup-code modal
`app.js`	Tab navigation, model selector, audio recorder, transcription orchestration
`liveEncounter.js`	Live recording UI + live preview
`soap.js`, `hpi.js` (none — in liveEncounter), `sickVisit.js`, `wellVisit.js`, `hospitalCourse.js`, `chartReview.js`	Clinical tabs
`milestones.js` + `milestonesData.js`	Milestones tab
`shadess.js`	SSHADESS adolescent assessment
`encounters.js`	Save / load / resume with optimistic lock
`memories.js`	Physician templates and prompt preferences UI
`speechRecognition.js`	Explicit opt-in browser Web Speech support
`voicePreferences.js`	Per-user STT/TTS override
`audioBackup.js`	Server + IndexedDB backup retries
`nextcloud.js`	Connect / export
`documents.js`	S3 upload / download
`calculators.js`	Pediatric calculators (BP, BMI, growth, bilirubin, vitals, etc.)
`learningHub.js`	Content browser + CMS editor
`admin.js`	Admin panel (users, settings, prompts, models)

Common tasks

Change default temperature

Edit the default in callAI() in src/utils/ai.js. Per-call options.temperature overrides.

Override a prompt without deploying

Admin Panel → Prompts → pick the template → edit → save. Takes effect immediately; cache is invalidated on write.

Admin Panel → Models → Add Custom Model. Exact provider ID, display name, cost string, category (free / fast / smart / premium). Appears immediately for all users.

Trace an AI call

docker compose logs -f pediatric-scribe | grep '\[AI\]'

Or:

SELECT endpoint, model_used, tokens_input, tokens_output, duration_ms, cost_estimate, error
FROM api_log
ORDER BY timestamp DESC
LIMIT 20;

Force a re-index

docker exec -w /app pediatric-ai-scribe npm run maint:reindex

Runs after any Postgres image upgrade if the startup drift check didn't catch it.

Local development

docker compose up -d postgres      # just the DB
npm install
cp .env.example .env               # set JWT_SECRET, DATA_ENCRYPTION_KEY, provider credentials
node server.js

App binds http://localhost:3000. Without APP_URL, production-mode guards relax (open CORS, non-secure cookies) — never deploy like this.

14 KiB Raw Permalink Blame History