pediatric-ai-scribe-v3/docs/features-explained.md

Features Explained

This file is a practical operator-oriented overview of major Ped-AI features. It intentionally describes the current fork, not historical browser Whisper behavior.

Clinical Documentation

Ped-AI generates pediatric clinical notes from typed input, dictation, or recorded audio. Major workflows include live encounters, dictation cleanup, sick visits, well visits, SOAP notes, hospital courses, chart review, ED documentation, and developmental milestones.

Model selection is available per task where the UI exposes a tab-level selector. Admin defaults provide the baseline model and user/task choices can override that baseline.

Generated notes can expose post-note helper panels. Billing suggestions and don't-miss review are clinician-facing. Patient education handouts are parent-facing drafts generated from the edited note, with optional diagnosis, medication, and preferred-language context. The clinician must verify the handout before sharing it.

Phone Extensions And Pagers

The bedside tools include a per-user phone extension and pager directory. Entries support active/trash views, search, soft delete/restore, permanent purge, ZIP export, and JSON/ZIP import. Import preview flags exact active duplicates, exact trashed matches that can be restored, and possible duplicates before committing changes.

Speech

Final transcription is server-side through LiteLLM. Configure upstream STT providers in LiteLLM rather than in Ped-AI.

Browser-native Web Speech is only an explicit opt-in preview path. It is not the final clinical transcript and may use browser-vendor cloud services.

Browser Whisper and browser-local model workers are removed. Do not expect a pre-download model button, public Whisper worker, or bundled Xenova model path.

Signing in

Sign-in is SSO-only: the sign-in screen has one button, "Sign in with PedsHub", which goes to sso.pedshub.com (Authentik). There a person types their email and the code that is sent to it — no password. New people are invited with a sign-up link (authentik-pedshub/invite.py on the host mints one); they enter a name and email, confirm with a code, and land in the pedshub-members group, which is what both PedsHub apps admit. The same account signs into the quiz app at pedshub.com. Local password sign-in, registration, reset and the app's own emailed codes are switched off (oidc.disable_local_auth); a local account with the same email is the same account.

Text To Speech

Read Aloud uses a roster of speech models (Admin → Models → Speech): each model brings the voices it accepts, and the default is a model-and-voice pair. Users choose any voice of any roster model under Settings and preview it. If a preview is silent, check that a default pair is set, the gateway is configured, and browser autoplay has not blocked playback.

Nextcloud WebDAV

Two ways to connect. Sign in with Nextcloud uses Nextcloud's own Login Flow v2: the person enters their server address, signs in on Nextcloud itself the way they normally do — SSO, 2FA, a password manager — and Nextcloud generates an app password for this app. We never see their real password and they never have to find the app-password screen. An app password can still be pasted in directly, under Use an app password instead.

The poll token Nextcloud issues is a credential, so the polling happens on the server; the browser holds only an opaque handle bound to its own account. Both URLs the server hands back are checked and must be on the host the person typed.

Once connected, generated notes and rendered resources can be saved to it. A resource is sent as the rendered file — a PowerPoint or Word document, exactly what the download would have produced — not as text.

Documents And S3

Document upload is optional and depends on S3-compatible storage configuration. Treat uploaded documents as PHI unless you have a separate deployment reason not to.

Audio Backups

Audio backups exist to recover failed transcription attempts.

• They are created when transcription fails.
• They are encrypted before persistent storage.
• They expire automatically.
• Users can retry or delete them from Settings.

My Resources

Personal teaching material a user generates for themselves: a presentation or a document, built from a subject and optional instructions, optionally grounded in PubMed, web search, and the clinical corpus, and optionally carrying generated figures.

Decks are stored as a typed deck rather than as markdown, and scripts/render_pptx.py (python-pptx) renders that deck to PowerPoint; scripts/render_docx.py (python-docx) renders the same typed source to Word. Export also offers PDF, via Gotenberg, and plain markdown. An optional vision review pass renders the deck, looks at the slides, and returns layout fixes; it is off by default.

Full detail in my-resources.md.

Clinical Assistant

Answers clinical questions against the indexed corpus rather than from model memory: Ped-AI asks the MCP retrieval service for sources, builds a grounded prompt, and renders the answer with citations and source cards. Answers are deliberately never cached. Full detail in clinical-assistant.md.

Admin Panel

Admins can manage users, roles, registration, security settings, model defaults, prompts and logs. Production deployments should enable SSO/2FA and restrict admin access.

Feature Status

Feature	Status	Notes
Clinical note generation	Active	Provider depends on AI_PROVIDER, defaulting to LiteLLM when a gateway is configured.
Server transcription	Active	Routed through LiteLLM; the upstream vendor is a gateway choice.
Browser Web Speech preview	Optional	Explicit opt-in only.
Browser Whisper	Removed	No public worker or model download path.
Nextcloud WebDAV	Active	Used for file browsing/content import.
Patient handouts	Active	Parent-facing, note-derived, preferred-language draft.
Extension transfer	Active	ZIP export plus JSON/ZIP import preview.
Audio backups	Active	Failure recovery only.
TTS preview	Active	Depends on configured provider.
Clinical Assistant	Active	Grounded answers over MCP retrieval, with citations and source cards.
My Resources	Active	Personal teaching decks and documents, rendered by python-pptx / python-docx.
Deck vision review	Optional	Off by default; one pass, admin-configurable model.
Sign-in	SSO-only	sso.pedshub.com: email → code. Invitation links create accounts.
Personal notes	Active	Including dictation, with trash and restore.

Troubleshooting

• Check browser console for frontend errors.
• Check docker logs pediatric-ai-scribe -f for backend errors.
• Check /api/health for service status.
• Check provider credentials and model names before debugging UI state.
• For Nextcloud export failures, verify the URL, username, app password, and folder path.