ceccd18387
The feature was removed; the docs still described it as live. Worst of it was api-reference.md, which documented nine /api/learning endpoints and fourteen /api/admin/learning CMS endpoints — routes that answer 404 — plus POST /api/user/webdav-path, whose column was dropped by migration. Anyone reading them was reading fiction. Checked against the running system rather than assumed: no learning table exists, users.webdav_learning_path is gone, generated_image_links is gone, and no route mounts /api/learning or /api/admin/learning. Two things that look like Learning Hub and are not, so they stay: - learningRetrieval.js is live — My Resources uses it. Its settings keep the learning.* names because renaming them would orphan whatever an administrator has already set. retrieval-tuning.md now says so instead of listing the rows under two different feature names. - the moderator role is still assignable. It gated the CMS and now grants nothing; authentication.md says that rather than implying powers it does not have. moderatorMiddleware has no callers left, which is worth removing on its own. auth-admin-learning.md is now auth-admin.md. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU
119 lines
3.6 KiB
Markdown
119 lines
3.6 KiB
Markdown
# Scaling
|
|
|
|
This document describes how Ped-AI should scale without becoming harder to debug or maintain.
|
|
|
|
## Current Scaling Model
|
|
|
|
Ped-AI is currently a single app container backed by PostgreSQL and Redis. That is acceptable for self-hosted use, but the code should keep moving toward a shape where multiple app containers can run safely.
|
|
|
|
```txt
|
|
reverse proxy
|
|
-> pediatric-ai-scribe replica 1
|
|
-> pediatric-ai-scribe replica 2
|
|
-> shared PostgreSQL
|
|
-> shared Redis
|
|
-> LiteLLM
|
|
-> MCP
|
|
```
|
|
|
|
## Horizontal Scaling Requirements
|
|
|
|
| Requirement | Why It Matters |
|
|
|---|---|
|
|
| Session state in PostgreSQL/Redis | Any app replica can handle the next request |
|
|
| No clinical state only in memory | Restarting or scaling containers should not lose required state |
|
|
| Shared uploads/storage if files grow | Local container disk does not scale across replicas |
|
|
| Idempotent migrations | Deploying more than one app container should not corrupt schema state |
|
|
| Request timeouts | Slow providers should not exhaust Node workers |
|
|
| Queue for slow jobs | Long work should not block interactive requests |
|
|
| Readiness endpoint | Load balancer should only send traffic to ready replicas |
|
|
|
|
## What Can Stay In Memory
|
|
|
|
Small process-local caches are acceptable when they are optional and short-lived.
|
|
|
|
Examples:
|
|
|
|
- settings cache with short TTL,
|
|
- provider model metadata cache,
|
|
- static configuration derived at boot.
|
|
|
|
Do not store required user workflow state only in memory if the action must survive restart or run across replicas.
|
|
|
|
## Redis Use
|
|
|
|
Redis is appropriate for:
|
|
|
|
- prompt suggestion pools,
|
|
- rate-limit coordination if needed,
|
|
- queues and job status,
|
|
- short-lived provider metadata,
|
|
- operational locks.
|
|
|
|
Redis should not be used for final clinical answer response caching. Clinical answers should be generated live from current retrieval context.
|
|
|
|
## Queue Candidates
|
|
|
|
Consider moving these to a queue when latency or concurrency becomes a problem:
|
|
|
|
- long transcription jobs,
|
|
- file import/export,
|
|
- My Resources generation from large files,
|
|
- image generation,
|
|
- bulk document operations,
|
|
- provider metadata refresh,
|
|
- long-running admin maintenance actions.
|
|
|
|
BullMQ with Redis is a natural fit if a queue is added.
|
|
|
|
## Readiness And Health
|
|
|
|
Keep `/api/health` fast and simple for liveness.
|
|
|
|
Add a separate readiness endpoint when scaling:
|
|
|
|
```txt
|
|
GET /api/ready
|
|
```
|
|
|
|
It should check:
|
|
|
|
- PostgreSQL query works,
|
|
- Redis ping works if Redis is required for this deployment,
|
|
- core settings can be read,
|
|
- MCP health is reachable if Clinical Assistant is enabled,
|
|
- LiteLLM metadata or configured model endpoint is reachable if AI features are enabled.
|
|
|
|
## Database Scaling
|
|
|
|
Priorities:
|
|
|
|
- confirm indexes on hot user/session/settings/log tables,
|
|
- keep migrations explicit and reversible where practical,
|
|
- monitor slow queries,
|
|
- cap admin log queries with safe limits,
|
|
- keep audit/log writes batched where possible,
|
|
- avoid long transactions around provider calls.
|
|
|
|
## Provider Scaling
|
|
|
|
LiteLLM and MCP can become the bottlenecks before Ped-AI does.
|
|
|
|
Track:
|
|
|
|
- LiteLLM request latency,
|
|
- LiteLLM error rate by model,
|
|
- MCP search latency,
|
|
- MCP timeout/error rate,
|
|
- queue depth if async jobs are added,
|
|
- Postgres connections,
|
|
- app container memory and event-loop delay.
|
|
|
|
## Scaling Order
|
|
|
|
1. Add request IDs across browser, Ped-AI, MCP, and LiteLLM calls.
|
|
2. Add `/api/ready` for dependency readiness.
|
|
3. Ensure sessions and settings are not process-local.
|
|
4. Add a queue for slow jobs if interactive requests block.
|
|
5. Run a second app replica behind the reverse proxy in a staging/test environment.
|
|
6. Add metrics and alerts around latency, errors, and resource saturation.
|