pediatric-ai-scribe-v3/docs/embeddings-setup.md

Embeddings & Semantic Search Setup

This guide explains how to set up and use the new vector-based semantic search for the Learning Hub.

🎯 What's New

• Semantic search - Find content by meaning, not just keywords
• 3 search modes:
  • Keyword (/api/learning/search) - Traditional text matching
  • Semantic (/api/learning/search/semantic) - AI-powered vector similarity
  • Hybrid (/api/learning/search/hybrid) - Combines both for best results
• Auto-embedding - Content is automatically vectorized when created/updated
• HIPAA-compliant - Uses Vertex AI embeddings (BAA available)

📋 Prerequisites

1. Install pgvector Extension

The database needs the pgvector extension for vector operations:

# For PostgreSQL 16 on Ubuntu/Debian
sudo apt-get install postgresql-16-pgvector

# For PostgreSQL 15
sudo apt-get install postgresql-15-pgvector

# For Docker (add to Dockerfile or docker-compose)
# The postgres:16-alpine base image doesn't include pgvector by default
# You'll need to use a custom image or install at runtime

For Docker deployments, use this postgres image instead:

postgres:
  image: pgvector/pgvector:pg16
  # ... rest of your config

2. Configure Embedding Provider

Add to your .env file:

# Option 1: Vertex AI (HIPAA-eligible, recommended)
EMBEDDING_MODEL=vertex_ai/text-embedding-005
EMBEDDING_DIMENSIONS=768
VERTEX_PROJECT=your-gcp-project-id
VERTEX_LOCATION=us-central1
GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json

# Option 2: LiteLLM Proxy (routes to any provider)
LITELLM_API_BASE=http://localhost:4000
LITELLM_API_KEY=your-key
EMBEDDING_MODEL=text-embedding-005  # LiteLLM will route to configured provider

# Option 3: OpenAI (NOT HIPAA-eligible, fallback only)
OPENAI_API_KEY=sk-your-key
# Uses text-embedding-3-small automatically

🚀 Available Vertex AI Embedding Models

Tested and working via LiteLLM:

Model	Dimensions	Use Case	HIPAA
vertex_ai/text-embedding-005	768	English + code (recommended)	✅ Yes
vertex_ai/gemini-embedding-001	768-3072	Multilingual + code, best quality	✅ Yes
vertex_ai/text-multilingual-embedding-002	768	Multilingual focus	✅ Yes

🔧 Setup Steps

1. Database Migration

The database will automatically:

• Enable the pgvector extension
• Add embedding vector(768) column to learning_content
• Create IVFFLAT index for fast similarity search (after 10+ embeddings)

Just restart your server after installing pgvector.

2. Generate Embeddings for Existing Content

Two options:

Option A: Admin API (recommended)

curl -X POST http://localhost:3000/api/admin/learning/embeddings/generate \
  -H "Authorization: Bearer YOUR_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"regenerateAll": false}'

Option B: Via Admin Panel

• Go to Admin → Learning Hub → Settings
• Click "Generate Embeddings" button
• Check status at /api/admin/learning/embeddings/status

3. Verify Setup

Check embedding status:

curl http://localhost:3000/api/admin/learning/embeddings/status \
  -H "Authorization: Bearer YOUR_JWT_TOKEN"

Response:

{
  "success": true,
  "enabled": true,
  "total": 50,
  "withEmbeddings": 50,
  "missing": 0,
  "model": "vertex_ai/text-embedding-005",
  "dimensions": 768
}

🔍 Using Semantic Search

Keyword Search (existing)

GET /api/learning/search?q=pneumonia

Returns exact text matches in title/subject/body.

Semantic Search (new)

GET /api/learning/search/semantic?q=childhood breathing problems&limit=10&threshold=0.5

Returns content similar by meaning (e.g., finds "pediatric asthma" articles).

Parameters:

• q (required) - Search query
• limit (optional, default 10, max 50) - Max results
• threshold (optional, default 0.5) - Similarity threshold (0-1, higher = more similar)
• contentType (optional) - Filter by type: article, quiz, pearl, presentation

Hybrid Search (recommended)

GET /api/learning/search/hybrid?q=fever management

Combines keyword + semantic for best results. Automatically deduplicates and ranks by relevance.

🔬 How It Works

1. Content Creation/Update:

   • Text is extracted from title, subject, and body (HTML stripped)
   • Sent to embedding model (Vertex AI)
   • Returns 768-dimensional vector
   • Stored in learning_content.embedding column

2. Semantic Search:

   • Query text → embedding vector
   • PostgreSQL pgvector computes cosine similarity
   • Returns top N most similar documents
   • Similarity score 0-1 (1 = identical, 0 = unrelated)

3. Hybrid Search:

   • Runs both keyword + semantic searches in parallel
   • Merges results (semantic first for quality)
   • Deduplicates by content ID
   • Sorts by relevance score

💰 Cost Estimate (Vertex AI)

Titan Text Embeddings (AWS) pricing:

• ~$0.10 per 1M tokens
• Average article: 2,000 words (~2,700 tokens) = $0.00027
• 1,000 articles: ~$0.27 one-time
• Search queries: ~500 tokens = $0.00005 per query

Google Vertex AI pricing:

• text-embedding-005: $0.025 per 1M characters
• Average article: 10,000 chars = $0.00025
• 1,000 articles: ~$0.25 one-time
• Search queries: ~$0.0000125 per query

🐛 Troubleshooting

"pgvector extension not available"

• Install: apt-get install postgresql-16-pgvector
• For Docker: Use pgvector/pgvector:pg16 image

"Embeddings not configured"

• Verify .env has VERTEX_PROJECT or LITELLM_API_BASE or OPENAI_API_KEY
• Check service account credentials: GOOGLE_APPLICATION_CREDENTIALS
• Test: curl http://localhost:3000/api/admin/learning/embeddings/status

"Embedding generation failed"

• Check logs for API errors
• Verify Vertex AI API is enabled in GCP
• Verify service account has aiplatform.endpoints.predict permission
• Check content isn't empty (skips empty bodies)

"No results from semantic search"

• Check if embeddings exist: /api/admin/learning/embeddings/status
• Lower threshold: ?threshold=0.3 (default 0.5)
• Verify pgvector index exists: \di in psql

📊 Performance

• Embedding generation: ~500ms per article (Vertex AI)
• Search latency:
  • Keyword: 10-50ms
  • Semantic: 20-100ms (with IVFFLAT index)
  • Hybrid: 30-150ms
• Index build time: ~1-5 seconds per 1,000 articles

🔐 Security & Compliance

• HIPAA-eligible: Vertex AI supports BAA (Business Associate Agreement)
• Data retention: Embeddings stored in your database only
• No PHI: Only article content (not patient data) is embedded
• Encryption: TLS in transit, at-rest encryption via PostgreSQL

🎓 Example Queries

Before (keyword):

Query: "fever in babies"
Results: Only articles with exact words "fever" or "babies"

After (semantic):

Query: "fever in babies"
Results:
- Infant hyperthermia management (similarity: 0.89)
- Pediatric fever evaluation (similarity: 0.87)
- Febrile seizures in toddlers (similarity: 0.82)
- Neonatal temperature regulation (similarity: 0.78)

Hybrid (best):

Query: "asthma"
Results:
- Childhood asthma management (keyword + semantic: 1.0)
- Pediatric breathing difficulties (semantic: 0.91)
- Reactive airway disease (semantic: 0.86)
- Bronchiolitis vs asthma (keyword: 1.0)

📚 API Reference

Admin Endpoints

• POST /api/admin/learning/embeddings/generate - Backfill embeddings
• GET /api/admin/learning/embeddings/status - Check status
• GET /api/admin/learning/stats - Includes embedding count

User Endpoints

• GET /api/learning/search - Keyword search
• GET /api/learning/search/semantic - Semantic search
• GET /api/learning/search/hybrid - Hybrid search (recommended)

All endpoints require authentication (JWT token).

---

Questions? Check logs for detailed error messages, or review the code in:

• /src/utils/embeddings.js - Core embedding logic
• /src/routes/learningHub.js - Search endpoints
• /src/routes/learningAdmin.js - Admin management