danvics/pediatric-ai-scribe-v3
1
0
Fork 0
Code Issues Pull requests Projects Releases 16 Packages Wiki Activity Actions
pediatric-ai-scribe-v3/docs/scaling.md
Daniel ceccd18387
Some checks failed
Forgejo Docker Build / Root app tests (push) Successful in 47s
Details
Forgejo Docker Build / Build Docker image (push) Successful in 22s
Details
Forgejo Docker Build / Deploy to the host (push) Failing after 1s
Details
docs: take the Learning Hub out of the docs 
The feature was removed; the docs still described it as live. Worst of
it was api-reference.md, which documented nine /api/learning endpoints
and fourteen /api/admin/learning CMS endpoints — routes that answer 404
— plus POST /api/user/webdav-path, whose column was dropped by
migration. Anyone reading them was reading fiction.

Checked against the running system rather than assumed: no learning
table exists, users.webdav_learning_path is gone, generated_image_links
is gone, and no route mounts /api/learning or /api/admin/learning.

Two things that look like Learning Hub and are not, so they stay:

- learningRetrieval.js is live — My Resources uses it. Its settings keep
  the learning.* names because renaming them would orphan whatever an
  administrator has already set. retrieval-tuning.md now says so instead
  of listing the rows under two different feature names.
- the moderator role is still assignable. It gated the CMS and now
  grants nothing; authentication.md says that rather than implying
  powers it does not have. moderatorMiddleware has no callers left,
  which is worth removing on its own.

auth-admin-learning.md is now auth-admin.md.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU
2026-09-12 22:24:29 +02:00

3.6 KiB
Raw Blame History

Scaling

This document describes how Ped-AI should scale without becoming harder to debug or maintain.

Current Scaling Model

Ped-AI is currently a single app container backed by PostgreSQL and Redis. That is acceptable for self-hosted use, but the code should keep moving toward a shape where multiple app containers can run safely.

reverse proxy
  -> pediatric-ai-scribe replica 1
  -> pediatric-ai-scribe replica 2
  -> shared PostgreSQL
  -> shared Redis
  -> LiteLLM
  -> MCP

Horizontal Scaling Requirements

Requirement Why It Matters
Session state in PostgreSQL/Redis Any app replica can handle the next request
No clinical state only in memory Restarting or scaling containers should not lose required state
Shared uploads/storage if files grow Local container disk does not scale across replicas
Idempotent migrations Deploying more than one app container should not corrupt schema state
Request timeouts Slow providers should not exhaust Node workers
Queue for slow jobs Long work should not block interactive requests
Readiness endpoint Load balancer should only send traffic to ready replicas

What Can Stay In Memory

Small process-local caches are acceptable when they are optional and short-lived.

Examples:

  • settings cache with short TTL,
  • provider model metadata cache,
  • static configuration derived at boot.

Do not store required user workflow state only in memory if the action must survive restart or run across replicas.

Redis Use

Redis is appropriate for:

  • prompt suggestion pools,
  • rate-limit coordination if needed,
  • queues and job status,
  • short-lived provider metadata,
  • operational locks.

Redis should not be used for final clinical answer response caching. Clinical answers should be generated live from current retrieval context.

Queue Candidates

Consider moving these to a queue when latency or concurrency becomes a problem:

  • long transcription jobs,
  • file import/export,
  • My Resources generation from large files,
  • image generation,
  • bulk document operations,
  • provider metadata refresh,
  • long-running admin maintenance actions.

BullMQ with Redis is a natural fit if a queue is added.

Readiness And Health

Keep /api/health fast and simple for liveness.

Add a separate readiness endpoint when scaling:

GET /api/ready

It should check:

  • PostgreSQL query works,
  • Redis ping works if Redis is required for this deployment,
  • core settings can be read,
  • MCP health is reachable if Clinical Assistant is enabled,
  • LiteLLM metadata or configured model endpoint is reachable if AI features are enabled.

Database Scaling

Priorities:

  • confirm indexes on hot user/session/settings/log tables,
  • keep migrations explicit and reversible where practical,
  • monitor slow queries,
  • cap admin log queries with safe limits,
  • keep audit/log writes batched where possible,
  • avoid long transactions around provider calls.

Provider Scaling

LiteLLM and MCP can become the bottlenecks before Ped-AI does.

Track:

  • LiteLLM request latency,
  • LiteLLM error rate by model,
  • MCP search latency,
  • MCP timeout/error rate,
  • queue depth if async jobs are added,
  • Postgres connections,
  • app container memory and event-loop delay.

Scaling Order

  1. Add request IDs across browser, Ped-AI, MCP, and LiteLLM calls.
  2. Add /api/ready for dependency readiness.
  3. Ensure sessions and settings are not process-local.
  4. Add a queue for slow jobs if interactive requests block.
  5. Run a second app replica behind the reverse proxy in a staging/test environment.
  6. Add metrics and alerts around latency, errors, and resource saturation.