pediatric-ai-scribe-v3/docs/embeddings-setup.md

# Embeddings And Semantic Search Setup

This guide explains how to set up and use the new vector-based semantic search for the Learning Hub.

## What This Enables

- **Semantic search** - Find content by meaning, not just keywords
- **3 search modes**:
  - **Keyword** (`/api/learning/search`) - Traditional text matching
  - **Semantic** (`/api/learning/search/semantic`) - AI-powered vector similarity
  - **Hybrid** (`/api/learning/search/hybrid`) - Combines both for best results
- **Auto-embedding** - Content is automatically vectorized when created/updated
- **Gateway-routed** - Uses LiteLLM embeddings so provider policy stays in one place

## Prerequisites

### 1. Install pgvector Extension

The database needs the `pgvector` extension for vector operations:

```bash
# For PostgreSQL 16 on Ubuntu/Debian
sudo apt-get install postgresql-16-pgvector

# For PostgreSQL 15
sudo apt-get install postgresql-15-pgvector

# For Docker (add to Dockerfile or docker-compose)
# The postgres:16-alpine base image doesn't include pgvector by default
# You'll need to use a custom image or install at runtime
```

**For Docker deployments**, use this postgres image instead:
```yaml
postgres:
  image: pgvector/pgvector:pg16
  # ... rest of your config
```

### 2. Configure LiteLLM Embeddings

Add to your `.env` file:

```bash
LITELLM_API_BASE=http://localhost:4000
LITELLM_API_KEY=your-key
EMBEDDING_MODEL=openai-text-embedding-3-large
EMBEDDING_DIMENSIONS=3072
```

## Available Embedding Models

The Admin embedding search reads LiteLLM `/model/info` and only shows models with `model_info.mode = "embedding"`. Do not add app-side built-in Vertex/OpenAI embedding lists; configure those choices in LiteLLM.

The local LiteLLM instance currently exposes examples such as `openai-text-embedding-3-large`, `openai-text-embedding-3-small`, and Mistral embedding models. Dimensions are read from LiteLLM metadata when available.

## Setup Steps

### 1. Database Migration

The database will automatically:
- Enable the `pgvector` extension
- Add `embedding vector(768)` column to `learning_content`
- Create IVFFLAT index for fast similarity search (after 10+ embeddings)

Just restart your server after installing pgvector.

### 2. Generate Embeddings for Existing Content

Two options:

**Option A: Admin API (recommended)**
```bash
curl -X POST http://localhost:3000/api/admin/learning/embeddings/generate \
  -H "Authorization: Bearer YOUR_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"regenerateAll": false}'
```

**Option B: Via Admin Panel**
- Go to Admin → Learning Hub → Settings
- Click "Generate Embeddings" button
- Check status at `/api/admin/learning/embeddings/status`

### 3. Verify Setup

Check embedding status:
```bash
curl http://localhost:3000/api/admin/learning/embeddings/status \
  -H "Authorization: Bearer YOUR_JWT_TOKEN"
```

Response:
```json
{
  "success": true,
  "enabled": true,
  "total": 50,
  "withEmbeddings": 50,
  "missing": 0,
  "model": "openai-text-embedding-3-large",
  "dimensions": 3072
}
```

## Using Semantic Search

### Keyword Search (existing)
```bash
GET /api/learning/search?q=pneumonia
```
Returns exact text matches in title/subject/body.

### Semantic Search (new)
```bash
GET /api/learning/search/semantic?q=childhood breathing problems&limit=10&threshold=0.5
```
Returns content similar by **meaning** (e.g., finds "pediatric asthma" articles).

**Parameters:**
- `q` (required) - Search query
- `limit` (optional, default 10, max 50) - Max results
- `threshold` (optional, default 0.5) - Similarity threshold (0-1, higher = more similar)
- `contentType` (optional) - Filter by type: article, quiz, pearl, presentation

### Hybrid Search (recommended)
```bash
GET /api/learning/search/hybrid?q=fever management
```
Combines keyword + semantic for best results. Automatically deduplicates and ranks by relevance.

## How It Works

1. **Content Creation/Update**:
   - Text is extracted from `title`, `subject`, and `body` (HTML stripped)
   - Sent to the configured LiteLLM embedding model
   - Returns an embedding vector
   - Stored in `learning_content.embedding` column

2. **Semantic Search**:
   - Query text → embedding vector
   - PostgreSQL pgvector computes cosine similarity
   - Returns top N most similar documents
   - Similarity score 0-1 (1 = identical, 0 = unrelated)

3. **Hybrid Search**:
   - Runs both keyword + semantic searches in parallel
   - Merges results (semantic first for quality)
   - Deduplicates by content ID
   - Sorts by relevance score

## Cost Estimate

Embedding cost depends on the upstream configured in LiteLLM.

## Troubleshooting

### "pgvector extension not available"
- Install: `apt-get install postgresql-16-pgvector`
- For Docker: Use `pgvector/pgvector:pg16` image

### "Embeddings not configured"
- Verify `.env` has `LITELLM_API_BASE`
- Test: `curl http://localhost:3000/api/admin/learning/embeddings/status`

### "Embedding generation failed"
- Check logs for API errors
- Verify LiteLLM `/model/info` shows the selected model with `mode: embedding`
- Check content isn't empty (skips empty bodies)

### "No results from semantic search"
- Check if embeddings exist: `/api/admin/learning/embeddings/status`
- Lower threshold: `?threshold=0.3` (default 0.5)
- Verify pgvector index exists: `\di` in psql

## Performance

- **Embedding generation**: latency depends on the LiteLLM upstream
- **Search latency**:
  - Keyword: 10-50ms
  - Semantic: 20-100ms (with IVFFLAT index)
  - Hybrid: 30-150ms
- **Index build time**: ~1-5 seconds per 1,000 articles

## Security And Compliance

- **Compliance**: controlled by the upstream provider configured in LiteLLM
- **Data retention**: Embeddings stored in your database only
- **No PHI**: Only article content (not patient data) is embedded
- **Encryption**: TLS in transit, at-rest encryption via PostgreSQL

## Example Queries

**Before (keyword):**
```
Query: "fever in babies"
Results: Only articles with exact words "fever" or "babies"
```

**After (semantic):**
```
Query: "fever in babies"
Results:
- Infant hyperthermia management (similarity: 0.89)
- Pediatric fever evaluation (similarity: 0.87)
- Febrile seizures in toddlers (similarity: 0.82)
- Neonatal temperature regulation (similarity: 0.78)
```

**Hybrid (best):**
```
Query: "asthma"
Results:
- Childhood asthma management (keyword + semantic: 1.0)
- Pediatric breathing difficulties (semantic: 0.91)
- Reactive airway disease (semantic: 0.86)
- Bronchiolitis vs asthma (keyword: 1.0)
```

## API Reference

### Admin Endpoints

- `POST /api/admin/learning/embeddings/generate` - Backfill embeddings
- `GET /api/admin/learning/embeddings/status` - Check status
- `GET /api/admin/learning/stats` - Includes embedding count

### User Endpoints

- `GET /api/learning/search` - Keyword search
- `GET /api/learning/search/semantic` - Semantic search
- `GET /api/learning/search/hybrid` - Hybrid search (recommended)

All endpoints require authentication (JWT token).

---

**Questions?** Check logs for detailed error messages, or review the code in:
- `/src/utils/embeddings.js` - Core embedding logic
- `/src/routes/learningHub.js` - Search endpoints
- `/src/routes/learningAdmin.js` - Admin management