diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..b1d11e7 --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,34 @@ +name: CI + +# Runs root app tests on every PR and push to main. + +on: + pull_request: + push: + branches: [main] + +# Cancel superseded runs on the same ref to save minutes. +concurrency: + group: ci-${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + +jobs: + test: + name: Root app tests + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Setup Node 22 + uses: actions/setup-node@v4 + with: + node-version: '22' + cache: 'npm' + cache-dependency-path: package-lock.json + + - name: Install + run: npm install + + - name: Unit tests + run: npm test diff --git a/.github/workflows/security.yml b/.github/workflows/security.yml new file mode 100644 index 0000000..6dd3d93 --- /dev/null +++ b/.github/workflows/security.yml @@ -0,0 +1,30 @@ +name: Security audit + +# Weekly npm audit at high+ severity for the root app. Reports to the job summary; does NOT fail the build +# (advisories appear constantly and a red checkmark train would just get +# muted). Re-run on demand via workflow_dispatch. + +on: + schedule: + - cron: '0 6 * * 1' # Mondays 06:00 UTC + workflow_dispatch: + +jobs: + audit: + name: npm audit (high+) + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Setup Node 22 + uses: actions/setup-node@v4 + with: + node-version: '22' + + - name: Audit root app + run: | + echo '## Root app advisories' >> "$GITHUB_STEP_SUMMARY" + npm audit --audit-level=high --json > legacy-audit.json || true + node -e "const a=require('./legacy-audit.json');const m=a.metadata?.vulnerabilities||{};console.log('high:'+(m.high||0)+' critical:'+(m.critical||0));" >> "$GITHUB_STEP_SUMMARY" + continue-on-error: true diff --git a/.gitignore b/.gitignore index fc083cf..aa682b3 100644 --- a/.gitignore +++ b/.gitignore @@ -41,7 +41,3 @@ e2e/playwright-report/ .firecrawl/ # Refactored test stack stays local for now -.env.refactored -.env.refactored.example -docker-compose.refactored.yml -refactored-ped-ai/ diff --git a/docs/api-reference.md b/docs/api-reference.md index 2ae436e..3d16a4d 100644 --- a/docs/api-reference.md +++ b/docs/api-reference.md @@ -1231,25 +1231,69 @@ Export active entries as `pedscribe-extensions.zip`. The ZIP contains `pedscribe ### POST /api/extensions/import -Import entries from a JSON request body. Exact duplicates already owned by the user are skipped. +Import entries from a JSON request body. Exact active duplicates are skipped. Exact trashed matches can be restored when requested. Possible duplicates are identified by loose matching (`location + number` or `number + type`) and skipped unless explicitly imported. - **Auth required:** Yes -- **Request body:** Export payload object or an array of entry objects. +- **Request body:** Export payload object, an array of entry objects, or an object with `items` and `options`. + ```json + { + "items": [ + { + "location": "ED", + "name": "Charge Nurse", + "number": "1234", + "type": "extension", + "notes": "Optional note" + } + ], + "options": { + "restoreTrashed": true, + "importPossibleDuplicates": false + } + } + ``` - **Response:** ```json { "success": true, "imported": 2, + "restored": 1, "skipped": 1, + "possibleSkipped": 1, "total": 3 } ``` --- +### POST /api/extensions/import/preview + +Preview a JSON import before committing it. The preview categorizes entries as new, exact active duplicates, exact trashed matches, or possible duplicates. + +- **Auth required:** Yes +- **Request body:** Same JSON shapes accepted by `POST /api/extensions/import`. +- **Response:** + ```json + { + "success": true, + "preview": { + "summary": { + "total": 3, + "new": 1, + "exactActive": 1, + "exactTrashed": 0, + "possible": 1 + }, + "entries": [] + } + } + ``` + +--- + ### POST /api/extensions/import-file -Import entries from a multipart file upload. Accepts a PedScribe JSON export or a ZIP export containing JSON. +Import entries from a multipart file upload. Accepts a PedScribe JSON export or a ZIP export containing JSON. Form fields `restoreTrashed=true` and `importPossibleDuplicates=true` control the same duplicate handling options as JSON import. - **Auth required:** Yes - **Request body:** `multipart/form-data` with field `file`. @@ -1258,6 +1302,16 @@ Import entries from a multipart file upload. Accepts a PedScribe JSON export or --- +### POST /api/extensions/import-file/preview + +Preview a multipart JSON or ZIP import before committing it. + +- **Auth required:** Yes +- **Request body:** `multipart/form-data` with field `file`. +- **Response:** Same preview shape as `POST /api/extensions/import/preview`. + +--- + ## Nextcloud Integration ### POST /api/nextcloud/connect diff --git a/docs/authentication.md b/docs/authentication.md index a95224e..3d29e43 100644 --- a/docs/authentication.md +++ b/docs/authentication.md @@ -146,7 +146,7 @@ Helmet defaults plus: - `Strict-Transport-Security: max-age=31536000; includeSubDomains; preload` - Content-Security-Policy: - `script-src 'self' 'wasm-unsafe-eval' 'unsafe-eval' cdn.jsdelivr.net cdnjs.cloudflare.com challenges.cloudflare.com` - (`unsafe-eval` is required by @xenova/transformers for in-browser Whisper) + (do not add `unsafe-eval` unless a reviewed dependency requires it) - `script-src-attr 'none'` (blocks inline event handlers) - `frame-src 'self' challenges.cloudflare.com` - `object-src 'none'` diff --git a/docs/developer-guide.md b/docs/developer-guide.md index 0e13b0a..5fc7efe 100644 --- a/docs/developer-guide.md +++ b/docs/developer-guide.md @@ -309,7 +309,7 @@ Maximum 20 corrections retained per category (oldest deleted). | `encounters.js` | Save / load / resume with optimistic lock | | `memories.js` | Physician templates + corrections UI | | `correctionTracker.js` | Captures AI-output edits | -| `browserWhisper.js` | In-browser WASM Whisper | +| `speechRecognition.js` | Explicit opt-in browser Web Speech support | | `speechRecognition.js` | Web Speech API preview | | `voicePreferences.js` | Per-user STT/TTS override | | `audioBackup.js` | Server + IndexedDB backup retries | diff --git a/docs/logic/README.md b/docs/logic/README.md index 58b52d1..dc6165e 100644 --- a/docs/logic/README.md +++ b/docs/logic/README.md @@ -39,11 +39,10 @@ For someone brand new to the codebase: Includes the suture selector. **Important:** lists every clinical formula that must NOT be modified without test vectors. -5. **[ai-and-voice.md](ai-and-voice.md)** — The 5-provider AI routing +5. **[ai-and-voice.md](ai-and-voice.md)** — AI provider routing (`callAI`), the centralized `PROMPTS` object with DB overrides, the `wrapUserText` + `INJECTION_GUARD` safety pattern, server-side STT - routing (Whisper / AWS Transcribe / Vertex / LiteLLM), browser - Whisper, the AudioRecorder. Voice/STT plumbing is **sacred** — the + routing, server-side STT, TTS, and the AudioRecorder. Voice/STT plumbing is **sacred** — the doc describes it without proposing changes. 6. **[auth-admin-learning.md](auth-admin-learning.md)** — Authentication @@ -82,19 +81,11 @@ from Daniel. The full sacred-zone roster: | Zone | Why | |---|---| | `public/js/encounters.js` save/load/idempotency | Save/version/idempotency logic has been carefully tuned; refactors keep silently breaking it. | -| Voice/STT plumbing (`audioBackup.js`, `speechRecognition.js`, `browserWhisper.js`, `voicePreferences.js`, `transcriptionSettings.js`, recorder paths in each clinical tab) | Recording UX has been hardened against many edge cases; refactor only with smallest-diff bug fixes. | +| Voice/STT plumbing (`audioBackup.js`, `speechRecognition.js`, `voicePreferences.js`, `transcriptionSettings.js`, recorder paths in each clinical tab) | Recording UX has been hardened against many edge cases; refactor only with smallest-diff bug fixes. | | Validated clinical formulas (BP percentile LMS, Fenton 2013, bilirubin AAP 2022, Bhutani, APLS / Best-Guess weight, PE Guide SCALES) | Validated against peditools / AAP tables; modifying without test vectors risks miscoding patient care. | | Auth + crypto (`crypto.js`, `passwords.js`, `sessions.js`, `auth.js`, `oidc.js`) | Security; changes without security review are unsafe. | | MDM rubric in `PROMPTS.edFinalize` | Load-bearing for billing accuracy; trim only with explicit AMA/coding source citation. | -## Total size - -~8,300 lines of new application-logic documentation across 6 files. If -that feels like a lot, remember: the codebase is ~33,000 lines of -frontend JS + ~14,000 lines of backend JS. The docs are dense by design -— "200% detailed" was the explicit ask. Search them like a reference; -don't try to read end to end. - ## Cross-cutting topics A few topics span multiple docs. Use these as your jump-off points: @@ -108,7 +99,7 @@ A few topics span multiple docs. Use these as your jump-off points: | `wrapUserText` + `INJECTION_GUARD` prompt-injection defense | ai-and-voice.md §5 | | `saveEncounter` API + optimistic locking + idempotency keys | architecture.md §13, clinical-notes.md §4, ed-encounters.md §5 | | `cryptoUtil.encryptString` / `encryptBuffer` "enc1:" format | architecture.md §12 | -| 5-provider AI routing (`callAI`) | ai-and-voice.md §2-3 | +| AI provider routing (`callAI`) | ai-and-voice.md §2-3 | | 2023 AMA E/M MDM rubric | ed-encounters.md §6 | | User templates (`user_memories` table, `template_*` categories) | clinical-notes.md §6, ed-encounters.md §9 | diff --git a/docs/logic/ai-and-voice.md b/docs/logic/ai-and-voice.md index 9bd0883..afc950b 100644 --- a/docs/logic/ai-and-voice.md +++ b/docs/logic/ai-and-voice.md @@ -510,94 +510,9 @@ say, `gemini-2.5-flash` even if the deployment default is --- -## 7. Browser-side Whisper (fully offline) +## 7. Browser-side Whisper -Source: `public/js/browserWhisper.js` + `public/js/whisperWorker.js` (and -the alternative loader `whisperWorkerV2.js`). - -### 7.1 Why it exists - -Three reasons stack on top of each other: - -1. **Privacy.** Audio never leaves the device. Genuine HIPAA-safe regardless - of which AI backend is configured for text generation. -2. **Zero CDN dependency.** The Dockerfile bundles - `transformers.min.js` and the model weights at image-build time. After - the page loads once, transcription works with zero network — even on an - air-gapped LAN. -3. **No vendor lock-in.** No AWS/GCP/Azure account needed for STT. - -### 7.2 The bundle layout - -(From `whisperWorker.js` lines 9–18) - -```js -importScripts('/models/transformers.min.js'); -env.allowLocalModels = false; -env.useBrowserCache = true; -env.allowRemoteModels = false; // no CDN fetch -env.localModelPath = '/models/'; -``` - -The Dockerfile pulls `@xenova/transformers` and the -`Xenova/whisper-tiny.en` (and optionally `whisper-base.en` / -`whisper-small.en`) model directories into `/app/public/models/` at image -build. Setting `allowRemoteModels = false` is what makes this airgap-safe -— transformers.js can only load from `/models/`. - -### 7.3 The orchestrator - -`browserWhisper.js` exposes `window.BrowserWhisper`: - -- `isSupported()` — checks for Worker, AudioContext, WebAssembly. -- `isEnabled()` / `setEnabled(val)` — localStorage toggle - (`ped_browser_whisper`). -- `getModel()` / `setModel(m)` — model id (default - `Xenova/whisper-tiny.en`). -- `preload(onProgress)` — pre-warm in background. Called from the settings - page when the user toggles Browser Whisper on, so the first recording - doesn't pay the load cost. -- `transcribe(blob)` — the main entry. - -`transcribe(blob)` flow (lines 58–81): - -1. `_blobToFloat32(blob)` decodes the WebM/Opus blob into a Float32Array - at 16 kHz mono using `AudioContext` (lines 142–153). 16 kHz mono is - what Whisper expects. -2. If the worker is already loaded with the same model, post the - Float32Array directly (zero-copy via `transferable`). -3. Otherwise initialise the worker, then post. - -The worker (`whisperWorker.js`): - -- Listens for `{type: 'load' | 'transcribe'}` messages. -- `load(modelName)` invokes `pipeline('automatic-speech-recognition', - modelName, { progress_callback })` which is what triggers the model - load from `/models/` (or hits the IndexedDB cache after first run). -- `transcribe` calls the pipeline with `{ language: 'english', task: - 'transcribe', chunk_length_s: 30, stride_length_s: 5, return_timestamps: - false }`. -- Posts back `{type: 'progress'}` events during model download, - `{type: 'ready'}` when loaded, `{type: 'result', text}` when done, - `{type: 'error', message}` on failure. - -Notable: `whisperWorkerV2.js` is an alternative loader that tries -`importScripts(CDN_URL)` first and falls back to telling the main thread -to use server transcription. It exists for the case where the bundled -worker can't be loaded (CSP or weird network), but the production path is -the v1 worker that imports from `/models/`. - -### 7.4 Failure handling - -If the worker fires an `error`, `browserWhisper.js` (lines 114–124): -- Logs the message. -- Toasts the user: *"Browser Whisper blocked by network/firewall. Using - server transcription."* -- Rejects the pending promise, which `transcribeAudio` (in `app.js`) - catches and falls back to `_serverTranscribe(blob)`. - -This is the silent-fallback chain that keeps recording usable when one -layer breaks. +Browser-side Whisper has been removed from the runtime. Do not re-add public Whisper workers, Transformers.js browser STT, browser-local model downloads, or settings that imply offline browser Whisper support. Current transcription should route through configured server-side providers, with browser Web Speech available only as an explicit opt-in preview where supported. --- @@ -729,8 +644,6 @@ is gone — and the physician has to re-dictate. The backup feature gives paths only**: - `data.success === false` from the server (line 777). - `fetch` rejection / network error (line 786). - - Browser Whisper failure before falling back to server (line 752 — - label `'browser-whisper-failed'`). So the database stores audio only when something went wrong, not on every recording. @@ -1047,25 +960,19 @@ Per `MEMORY.md` rules `feedback_voice_stt_sacred` and the cross-cutting - `public/js/audioBackup.js` — server-first, IndexedDB-fallback save path. Touch only if a specific bug is reported. -- `public/js/browserWhisper.js` + `whisperWorker.js` (and v2) — the - in-browser WASM Whisper bundle. The `transformers.js` env config and - the `chunk_length_s/stride_length_s` are tuned; don't change them - speculatively. - `public/js/speechRecognition.js` — the live-streaming preview wrapper (and `app.js`'s `createSpeechRecognition`). Both exist for a reason (one is a real opt-in transcript path, one is just a visual preview); unifying them would break tabs. - `public/js/voicePreferences.js` — STT model + TTS voice picker UI. -- `public/js/transcriptionSettings.js` — Browser Whisper + Web Speech - toggles. Mutual exclusion (turning Web Speech on disables Browser - Whisper) is intentional and survives in this file only. +- `public/js/transcriptionSettings.js` — explicit Web Speech setting and + transcription preference UI. - `public/js/app.js` — the `AudioRecorder` class (lines 659–683) and the `transcribeAudio` / `_serverTranscribe` chain (lines 739–795). Specifically: do **not** propose: - Refactoring `transcribeAudio` to use a "TranscriptionRouter" class. - Splitting `AudioRecorder` into separate start/stop modules. -- Replacing the v1/v2 worker pair with a single worker. - Changing the 32 kbps Opus / 1-second slice / mono / 16 kHz config. - Centralising the audio-backup save logic with the upload logic. diff --git a/docs/logic/architecture.md b/docs/logic/architecture.md index 58a7099..1fd4a97 100644 --- a/docs/logic/architecture.md +++ b/docs/logic/architecture.md @@ -72,7 +72,7 @@ the same Postgres schema with row-level `user_id` scoping. | Auth | JWT (HS256) signed locally + DB-backed `user_sessions` table | | Secrets | argon2id (current), bcrypt (legacy / rehash on next login) | | AI | Provider-agnostic via `src/utils/ai.js` → OpenAI SDK shape | -| STT | OpenAI Whisper API / Google Gemini / AWS Transcribe / local whisper.cpp / browser Whisper (transformers.js) | +| STT | Server-side providers such as LiteLLM/OpenAI-compatible transcription, Google/Gemini, AWS Transcribe, or local server STT where configured | | TTS | Google Cloud TTS / ElevenLabs / LiteLLM-routed | | Frontend | **Vanilla JS, no framework, no bundler.** Plain `