pediatric-ai-scribe-v3/docs/embeddings-setup.md

# Embeddings And Semantic Search Setup

This guide explains how to set up and use the new vector-based semantic search for the Learning Hub.

## What This Enables

- **Semantic search** - Find content by meaning, not just keywords
- **3 search modes**:
  - **Keyword** (`/api/learning/search`) - Traditional text matching
  - **Semantic** (`/api/learning/search/semantic`) - AI-powered vector similarity
  - **Hybrid** (`/api/learning/search/hybrid`) - Combines both for best results
- **Auto-embedding** - Content is automatically vectorized when created/updated
- **HIPAA-compliant** - Uses Vertex AI embeddings (BAA available)

## Prerequisites

### 1. Install pgvector Extension

The database needs the `pgvector` extension for vector operations:

```bash
# For PostgreSQL 16 on Ubuntu/Debian
sudo apt-get install postgresql-16-pgvector

# For PostgreSQL 15
sudo apt-get install postgresql-15-pgvector

# For Docker (add to Dockerfile or docker-compose)
# The postgres:16-alpine base image doesn't include pgvector by default
# You'll need to use a custom image or install at runtime
```

**For Docker deployments**, use this postgres image instead:
```yaml
postgres:
  image: pgvector/pgvector:pg16
  # ... rest of your config
```

### 2. Configure Embedding Provider

Add to your `.env` file:

```bash
# Option 1: Vertex AI (HIPAA-eligible, recommended)
EMBEDDING_MODEL=vertex_ai/text-embedding-005
EMBEDDING_DIMENSIONS=768
VERTEX_PROJECT=your-gcp-project-id
VERTEX_LOCATION=us-central1
GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json

# Option 2: LiteLLM Proxy (routes to any provider)
LITELLM_API_BASE=http://localhost:4000
LITELLM_API_KEY=your-key
EMBEDDING_MODEL=text-embedding-005  # LiteLLM will route to configured provider

# Option 3: OpenAI (NOT HIPAA-eligible, fallback only)
OPENAI_API_KEY=sk-your-key
# Uses text-embedding-3-small and truncates to the 768-dimension column
```

## Available Vertex AI Embedding Models

Tested and working via LiteLLM:

| Model | Dimensions | Use Case | HIPAA |
|-------|-----------|----------|-------|
| **vertex_ai/text-embedding-005** | 768 | English + code (recommended) | Yes |
| **vertex_ai/gemini-embedding-001** | 768-3072 | Multilingual + code, best quality | Yes |
| **vertex_ai/text-multilingual-embedding-002** | 768 | Multilingual focus | Yes |

## Setup Steps

### 1. Database Migration

The database will automatically:
- Enable the `pgvector` extension
- Add `embedding vector(768)` column to `learning_content`
- Create IVFFLAT index for fast similarity search (after 10+ embeddings)

Just restart your server after installing pgvector.

### 2. Generate Embeddings for Existing Content

Two options:

**Option A: Admin API (recommended)**
```bash
curl -X POST http://localhost:3000/api/admin/learning/embeddings/generate \
  -H "Authorization: Bearer YOUR_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"regenerateAll": false}'
```

**Option B: Via Admin Panel**
- Go to Admin → Learning Hub → Settings
- Click "Generate Embeddings" button
- Check status at `/api/admin/learning/embeddings/status`

### 3. Verify Setup

Check embedding status:
```bash
curl http://localhost:3000/api/admin/learning/embeddings/status \
  -H "Authorization: Bearer YOUR_JWT_TOKEN"
```

Response:
```json
{
  "success": true,
  "enabled": true,
  "total": 50,
  "withEmbeddings": 50,
  "missing": 0,
  "model": "vertex_ai/text-embedding-005",
  "dimensions": 768
}
```

## Using Semantic Search

### Keyword Search (existing)
```bash
GET /api/learning/search?q=pneumonia
```
Returns exact text matches in title/subject/body.

### Semantic Search (new)
```bash
GET /api/learning/search/semantic?q=childhood breathing problems&limit=10&threshold=0.5
```
Returns content similar by **meaning** (e.g., finds "pediatric asthma" articles).

**Parameters:**
- `q` (required) - Search query
- `limit` (optional, default 10, max 50) - Max results
- `threshold` (optional, default 0.5) - Similarity threshold (0-1, higher = more similar)
- `contentType` (optional) - Filter by type: article, quiz, pearl, presentation

### Hybrid Search (recommended)
```bash
GET /api/learning/search/hybrid?q=fever management
```
Combines keyword + semantic for best results. Automatically deduplicates and ranks by relevance.

## How It Works

1. **Content Creation/Update**:
   - Text is extracted from `title`, `subject`, and `body` (HTML stripped)
   - Sent to embedding model (Vertex AI)
   - Returns 768-dimensional vector
   - Stored in `learning_content.embedding` column

2. **Semantic Search**:
   - Query text → embedding vector
   - PostgreSQL pgvector computes cosine similarity
   - Returns top N most similar documents
   - Similarity score 0-1 (1 = identical, 0 = unrelated)

3. **Hybrid Search**:
   - Runs both keyword + semantic searches in parallel
   - Merges results (semantic first for quality)
   - Deduplicates by content ID
   - Sorts by relevance score

## Cost Estimate

**Google Vertex AI pricing:**
- text-embedding-005: $0.025 per 1M characters
- Average article: 10,000 chars = $0.00025
- 1,000 articles: ~**$0.25 one-time**
- Search queries: ~$0.0000125 per query

## Troubleshooting

### "pgvector extension not available"
- Install: `apt-get install postgresql-16-pgvector`
- For Docker: Use `pgvector/pgvector:pg16` image

### "Embeddings not configured"
- Verify `.env` has `VERTEX_PROJECT` / `GOOGLE_CLOUD_PROJECT`, `LITELLM_API_BASE`, or `OPENAI_API_KEY`
- Check service account credentials: `GOOGLE_APPLICATION_CREDENTIALS`
- Test: `curl http://localhost:3000/api/admin/learning/embeddings/status`

### "Embedding generation failed"
- Check logs for API errors
- Verify Vertex AI API is enabled in GCP
- Verify service account has `aiplatform.endpoints.predict` permission
- Check content isn't empty (skips empty bodies)

### "No results from semantic search"
- Check if embeddings exist: `/api/admin/learning/embeddings/status`
- Lower threshold: `?threshold=0.3` (default 0.5)
- Verify pgvector index exists: `\di` in psql

## Performance

- **Embedding generation**: ~500ms per article (Vertex AI)
- **Search latency**:
  - Keyword: 10-50ms
  - Semantic: 20-100ms (with IVFFLAT index)
  - Hybrid: 30-150ms
- **Index build time**: ~1-5 seconds per 1,000 articles

## Security And Compliance

- **HIPAA-eligible**: Vertex AI supports BAA (Business Associate Agreement)
- **Data retention**: Embeddings stored in your database only
- **No PHI**: Only article content (not patient data) is embedded
- **Encryption**: TLS in transit, at-rest encryption via PostgreSQL

## Example Queries

**Before (keyword):**
```
Query: "fever in babies"
Results: Only articles with exact words "fever" or "babies"
```

**After (semantic):**
```
Query: "fever in babies"
Results:
- Infant hyperthermia management (similarity: 0.89)
- Pediatric fever evaluation (similarity: 0.87)
- Febrile seizures in toddlers (similarity: 0.82)
- Neonatal temperature regulation (similarity: 0.78)
```

**Hybrid (best):**
```
Query: "asthma"
Results:
- Childhood asthma management (keyword + semantic: 1.0)
- Pediatric breathing difficulties (semantic: 0.91)
- Reactive airway disease (semantic: 0.86)
- Bronchiolitis vs asthma (keyword: 1.0)
```

## API Reference

### Admin Endpoints

- `POST /api/admin/learning/embeddings/generate` - Backfill embeddings
- `GET /api/admin/learning/embeddings/status` - Check status
- `GET /api/admin/learning/stats` - Includes embedding count

### User Endpoints

- `GET /api/learning/search` - Keyword search
- `GET /api/learning/search/semantic` - Semantic search
- `GET /api/learning/search/hybrid` - Hybrid search (recommended)

All endpoints require authentication (JWT token).

---

**Questions?** Check logs for detailed error messages, or review the code in:
- `/src/utils/embeddings.js` - Core embedding logic
- `/src/routes/learningHub.js` - Search endpoints
- `/src/routes/learningAdmin.js` - Admin management