# Scaling

This document describes how Ped-AI should scale without becoming harder to debug or maintain.

## Current Scaling Model

Ped-AI is currently a single app container backed by PostgreSQL and Redis. That is acceptable for self-hosted use, but the code should keep moving toward a shape where multiple app containers can run safely.

```txt
reverse proxy
  -> pediatric-ai-scribe replica 1
  -> pediatric-ai-scribe replica 2
  -> shared PostgreSQL
  -> shared Redis
  -> LiteLLM
  -> MCP
```

## Horizontal Scaling Requirements

| Requirement | Why It Matters |
|---|---|
| Session state in PostgreSQL/Redis | Any app replica can handle the next request |
| No clinical state only in memory | Restarting or scaling containers should not lose required state |
| Shared uploads/storage if files grow | Local container disk does not scale across replicas |
| Idempotent migrations | Deploying more than one app container should not corrupt schema state |
| Request timeouts | Slow providers should not exhaust Node workers |
| Queue for slow jobs | Long work should not block interactive requests |
| Readiness endpoint | Load balancer should only send traffic to ready replicas |

## What Can Stay In Memory

Small process-local caches are acceptable when they are optional and short-lived.

Examples:

- settings cache with short TTL,
- provider model metadata cache,
- static configuration derived at boot.

Do not store required user workflow state only in memory if the action must survive restart or run across replicas.

## Redis Use

Redis is appropriate for:

- prompt suggestion pools,
- rate-limit coordination if needed,
- queues and job status,
- short-lived provider metadata,
- operational locks.

Redis should not be used for final clinical answer response caching. Clinical answers should be generated live from current retrieval context.

## Queue Candidates

Consider moving these to a queue when latency or concurrency becomes a problem:

- long transcription jobs,
- file import/export,
- Learning Hub AI generation from large files,
- image generation,
- bulk document operations,
- provider metadata refresh,
- long-running admin maintenance actions.

BullMQ with Redis is a natural fit if a queue is added.

## Readiness And Health

Keep `/api/health` fast and simple for liveness.

Add a separate readiness endpoint when scaling:

```txt
GET /api/ready
```

It should check:

- PostgreSQL query works,
- Redis ping works if Redis is required for this deployment,
- core settings can be read,
- MCP health is reachable if Clinical Assistant is enabled,
- LiteLLM metadata or configured model endpoint is reachable if AI features are enabled.

## Database Scaling

Priorities:

- confirm indexes on hot user/session/settings/log tables,
- keep migrations explicit and reversible where practical,
- monitor slow queries,
- cap admin log queries with safe limits,
- keep audit/log writes batched where possible,
- avoid long transactions around provider calls.

## Provider Scaling

LiteLLM and MCP can become the bottlenecks before Ped-AI does.

Track:

- LiteLLM request latency,
- LiteLLM error rate by model,
- MCP search latency,
- MCP timeout/error rate,
- queue depth if async jobs are added,
- Postgres connections,
- app container memory and event-loop delay.

## Scaling Order

1. Add request IDs across browser, Ped-AI, MCP, and LiteLLM calls.
2. Add `/api/ready` for dependency readiness.
3. Ensure sessions and settings are not process-local.
4. Add a queue for slow jobs if interactive requests block.
5. Run a second app replica behind the reverse proxy in a staging/test environment.
6. Add metrics and alerts around latency, errors, and resource saturation.