docs: recording, 24-hour retention, and the STT picker
- docs/speech.md: a new Recording section (wake lock and why it is re-taken on visibility, idempotent start, what happens when a recording ends by itself, what signing out does), the retention rules, and step-by-step instructions for moving audio backups to MinIO. - Records that the model for a transcription is the user's choice first, which is why the picker must only ever offer models the gateway has, and that LITELLM_STT_MODELS is a fallback rather than a list known to work. - api-reference: /api/transcribe takes `module` and returns `backupId`, and the Audio Backups group is no longer failure-only. - configuration.md and .env.example document AUDIO_BACKUPS_S3_*, preferring the _FILE credential variants. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU
This commit is contained in:
parent
523926ab17
commit
8e4c70aaa8
5 changed files with 116 additions and 8 deletions
16
.env.example
16
.env.example
|
|
@ -181,3 +181,19 @@ EMBEDDING_DIMENSIONS=768
|
|||
# ============================================================
|
||||
DATABASE_URL=postgresql://pedscribe:<password>@postgres:5432/pedscribe
|
||||
DB_PASSWORD=pedscribe_secret_change_me
|
||||
|
||||
# ── Audio backups (optional) ────────────────────────────────────────────────
|
||||
# Every recording is kept for 24 hours, whether its transcription succeeded or
|
||||
# not. Unset, the audio lives in the audio_backups table; set, it goes to a
|
||||
# bucket instead and only metadata stays in Postgres. The generated-images key
|
||||
# is scoped to that bucket and cannot be reused here — create a bucket and a
|
||||
# user of its own, and give the bucket a 24h expiry rule as a backstop.
|
||||
# Audio is gzipped and AES-256-GCM encrypted before it is stored, either way.
|
||||
# AUDIO_BACKUPS_S3_ENDPOINT=http://assets:9000
|
||||
# AUDIO_BACKUPS_S3_BUCKET=audio-backups
|
||||
# AUDIO_BACKUPS_S3_REGION=us-east-1
|
||||
# Prefer the _FILE variants: credentials then stay out of the process environment.
|
||||
# AUDIO_BACKUPS_S3_ACCESS_KEY_FILE=/run/secrets/audio-backups-access-key
|
||||
# AUDIO_BACKUPS_S3_SECRET_KEY_FILE=/run/secrets/audio-backups-secret-key
|
||||
# AUDIO_BACKUPS_S3_ACCESS_KEY=
|
||||
# AUDIO_BACKUPS_S3_SECRET_KEY=
|
||||
|
|
|
|||
|
|
@ -607,15 +607,23 @@ Transcribe an audio file to text. Accepts multipart form data with the audio fil
|
|||
| Field | Type | Description |
|
||||
|---------|------|--------------------|
|
||||
| `audio` | file | Audio file to transcribe |
|
||||
| `module` | string | Optional. What produced the recording; recorded on the backup. Defaults to `recording`. |
|
||||
- **Response:**
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"text": "string",
|
||||
"provider": "string",
|
||||
"duration": "number (seconds)"
|
||||
"duration": "number (seconds)",
|
||||
"backupId": "number | null"
|
||||
}
|
||||
```
|
||||
- The audio is kept for 24 hours whether or not transcription succeeds, so this
|
||||
endpoint does not need a second upload to `/api/audio-backups`. `backupId` is
|
||||
`null` when the copy could not be stored — the transcription still returns,
|
||||
because losing the transcript would be worse than losing the copy.
|
||||
- The model is the caller's `users.stt_model`, then the `stt.model` setting,
|
||||
then `LITELLM_STT_MODEL`.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -886,7 +894,14 @@ generation prompts. `custom` rows and legacy `correction_*` rows are excluded.
|
|||
|
||||
## Audio Backups
|
||||
|
||||
Temporary encrypted audio backup storage with automatic 24-hour expiry.
|
||||
Encrypted 24-hour storage for recordings. Every recording made through
|
||||
`/api/transcribe` is kept automatically; this group is for browser-held copies
|
||||
(saved when the server could not be reached at all), and for listing,
|
||||
downloading and deleting.
|
||||
|
||||
Payload lives in object storage when `AUDIO_BACKUPS_S3_*` is configured and in
|
||||
the `audio_backups` column otherwise; metadata is always in Postgres, so these
|
||||
endpoints behave identically either way. See `docs/speech.md`.
|
||||
|
||||
### POST /api/audio-backups
|
||||
|
||||
|
|
|
|||
|
|
@ -15,6 +15,8 @@ keys):
|
|||
| `APP_URL` | Public base URL. Enables production mode — fail-closed CORS, HSTS, secure cookies. |
|
||||
| `JWT_SECRET` | HMAC key for JWT signing and OIDC state. Server refuses to start without it in production. |
|
||||
| `DATA_ENCRYPTION_KEY` | AES-256-GCM key for PHI at rest (Nextcloud tokens, audio backups). 64 hex chars (`openssl rand -hex 32`). Refuses to start without it in production. |
|
||||
| `AUDIO_BACKUPS_S3_ENDPOINT`, `_BUCKET`, `_REGION` | Optional. Keeps the 24-hour recording copies in object storage instead of the `audio_backups` column; metadata stays in Postgres either way. See `docs/speech.md`. |
|
||||
| `AUDIO_BACKUPS_S3_ACCESS_KEY_FILE`, `_SECRET_KEY_FILE` | Credentials for the above, read from files so they never appear in the process environment. `AUDIO_BACKUPS_S3_ACCESS_KEY` / `_SECRET_KEY` are the inline fallback. |
|
||||
| `DB_PASSWORD` / `DATABASE_URL` | Postgres password or full connection string. |
|
||||
| `PORT` | HTTP listen port (default 3000). |
|
||||
| `NODE_ENV` | `production` forces prod-only guards on even without `APP_URL`. |
|
||||
|
|
|
|||
|
|
@ -6,6 +6,14 @@
|
|||
|
||||
Set `TRANSCRIBE_PROVIDER=litellm`, `LITELLM_API_BASE`, and `LITELLM_STT_MODEL`. Auto mode also uses LiteLLM when the gateway is configured.
|
||||
|
||||
The model for a request is `users.stt_model` (the person's own choice), then
|
||||
the `stt.model` setting, then `LITELLM_STT_MODEL`. A user's choice wins, so the
|
||||
Settings picker must never offer a model the gateway does not have: it lists
|
||||
what `/model/info` advertises as `audio_transcription`
|
||||
(`discoverSTTModels()` in `src/utils/sttProvider.js`, cached five minutes).
|
||||
`LITELLM_STT_MODELS` is a fallback for when discovery fails and is not a list
|
||||
of models known to work — its ids do not resolve on every gateway.
|
||||
|
||||
| Provider | Notes | HIPAA posture |
|
||||
|---|---|---|
|
||||
| LiteLLM | Sends audio through the configured LiteLLM `/audio/transcriptions` backend. | Depends on the selected upstream. |
|
||||
|
|
@ -26,13 +34,79 @@ Browser-native Web Speech can show interim text when the user explicitly enables
|
|||
|
||||
The admin/user voice pickers read available LiteLLM-compatible voices from `LITELLM_TTS_VOICES`.
|
||||
|
||||
## Recording
|
||||
|
||||
A recording holds a screen wake lock for as long as it runs, so the screen
|
||||
going to sleep cannot suspend it. Browsers release the lock whenever the page
|
||||
is hidden, so it is requested again when the page becomes visible; a lock is
|
||||
never requested while hidden, because that request is rejected. The count is
|
||||
shared, so two recorders cannot release each other's lock. A browser that
|
||||
denies or lacks the API keeps recording without one.
|
||||
|
||||
Recording continues while the user moves around the workspace — the app is a
|
||||
single page and switching tabs does not touch the recorder. Signing out stops
|
||||
it and releases the lock; nothing is sent, because the session that owned the
|
||||
audio is gone.
|
||||
|
||||
`AudioRecorder.start()` on a running recorder is a no-op. Calling it again
|
||||
would replace the `MediaRecorder` and drop everything captured so far.
|
||||
|
||||
A recording can stop without anyone pressing Stop: the recorder can error, and
|
||||
the microphone can be claimed by another app, unplugged, or revoked. Both are
|
||||
reported once and dispatch `audio-recorder-failed`, which runs the same path as
|
||||
Stop — so the audio is transcribed and stored rather than left in a tab that
|
||||
still claims to be recording. Whatever was captured before the failure is kept.
|
||||
|
||||
## Audio Backup
|
||||
|
||||
Failed transcription submissions can be stored for retry instead of being silently lost.
|
||||
Every recording is kept for 24 hours, whether its transcription succeeded or
|
||||
not. `POST /api/transcribe` already holds the audio, so keeping it costs no
|
||||
second upload; a storage failure there is logged and the transcription still
|
||||
returns, because losing the transcript someone is waiting for would be worse
|
||||
than losing the copy.
|
||||
|
||||
- Audio backups are compressed and encrypted before storage.
|
||||
- Backups expire automatically.
|
||||
- The Settings audio backup UI can retry or delete saved items.
|
||||
- Browser fallback storage is used only when the server cannot save the failed audio.
|
||||
`src/utils/audioBackupStore.js` is the only place that knows how a recording is
|
||||
kept, shared by `/api/transcribe` and `/api/audio-backups` so the two cannot
|
||||
drift apart.
|
||||
|
||||
- Audio is gzipped, then AES-256-GCM encrypted (`DATA_ENCRYPTION_KEY`), before
|
||||
it is stored — in either backend.
|
||||
- **Object storage** is used when `AUDIO_BACKUPS_S3_*` is set; otherwise the
|
||||
payload goes in the `audio_backups.audio_data` column. Metadata (owner,
|
||||
module, sizes, expiry) is always in Postgres, so listing, ownership and
|
||||
expiry behave the same either way.
|
||||
- Object keys are `recordings/<user id>/<timestamp>-<random>`, scoped to their
|
||||
owner, so a leaked id cannot address someone else's audio.
|
||||
- Reads carry `user_id` and `expires_at > NOW()` in the query, so an expired or
|
||||
borrowed id reads as missing rather than as another person's recording.
|
||||
- The expiry sweep (`cleanupExpired` in `src/db/database.js`) deletes each
|
||||
object with its row, so audio cannot outlive its 24 hours in the bucket.
|
||||
- Rows written before encryption was added are passed through unencrypted on
|
||||
read, so old backups still play.
|
||||
- The Settings list can retry, **download** or delete a recording. Download is
|
||||
how a copy leaves the app — onto a phone's Files, a shared drive, an external
|
||||
recorder. It works for both server-side and browser-fallback copies, and a
|
||||
browser copy is only handed to the account that owns it.
|
||||
- Browser fallback storage (IndexedDB) is used only when the server cannot be
|
||||
reached at all.
|
||||
|
||||
Treat audio backups as sensitive clinical data even when encrypted.
|
||||
|
||||
### Switching audio backups to MinIO
|
||||
|
||||
Storing every recording, rather than only the failures, makes object storage
|
||||
the better home. It is off by default because it needs a bucket and its own
|
||||
credentials; the existing `generated-images` key is scoped to that bucket and
|
||||
cannot reach another.
|
||||
|
||||
1. Create an `audio-backups` bucket and a user with read/write/delete on it.
|
||||
2. Give the bucket a 24-hour expiry lifecycle rule, as a backstop for objects
|
||||
the sweep could not delete.
|
||||
3. Set `AUDIO_BACKUPS_S3_ENDPOINT`, `AUDIO_BACKUPS_S3_BUCKET`,
|
||||
`AUDIO_BACKUPS_S3_REGION`, and either `AUDIO_BACKUPS_S3_ACCESS_KEY_FILE` /
|
||||
`AUDIO_BACKUPS_S3_SECRET_KEY_FILE` (preferred — credentials stay out of the
|
||||
process environment) or `AUDIO_BACKUPS_S3_ACCESS_KEY` /
|
||||
`AUDIO_BACKUPS_S3_SECRET_KEY`.
|
||||
|
||||
New recordings then go to the bucket. Rows already in Postgres keep working:
|
||||
a row without `storage_key` is read from the column.
|
||||
|
|
|
|||
|
|
@ -24,7 +24,8 @@ LITELLM_STT_MODEL=local-parakeet-v3
|
|||
|
||||
## Failure Handling
|
||||
|
||||
- Server transcription failures can create encrypted audio backups for retry.
|
||||
- Every recording is kept, encrypted, for 24 hours — not only the ones whose
|
||||
transcription failed — and can be downloaded from Settings.
|
||||
- Users can retry or delete failed backups from Settings.
|
||||
- Web Speech interim text is not a substitute for a server transcription response.
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue