danvics/pediatric-ai-scribe-v3
1
0
Fork 0
Code Issues Pull requests Projects Releases 14 Packages Wiki Activity Actions
pediatric-ai-scribe-v3/docs/embeddings-setup.md

7.1 KiB
Raw Permalink Blame History

Embeddings And Semantic Search Setup

This guide explains how to set up and use the new vector-based semantic search for the Learning Hub.

What This Enables

  • Semantic search - Find content by meaning, not just keywords
  • 3 search modes:
    • Keyword (/api/learning/search) - Traditional text matching
    • Semantic (/api/learning/search/semantic) - AI-powered vector similarity
    • Hybrid (/api/learning/search/hybrid) - Combines both for best results
  • Auto-embedding - Content is automatically vectorized when created/updated
  • Gateway-routed - Uses LiteLLM embeddings so provider policy stays in one place

Prerequisites

1. Install pgvector Extension

The database needs the pgvector extension for vector operations:

# For PostgreSQL 16 on Ubuntu/Debian
sudo apt-get install postgresql-16-pgvector

# For PostgreSQL 15
sudo apt-get install postgresql-15-pgvector

# For Docker (add to Dockerfile or docker-compose)
# The postgres:16-alpine base image doesn't include pgvector by default
# You'll need to use a custom image or install at runtime

For Docker deployments, use this postgres image instead:

postgres:
  image: pgvector/pgvector:pg16
  # ... rest of your config

2. Configure LiteLLM Embeddings

Add to your .env file:

LITELLM_API_BASE=http://localhost:4000
LITELLM_API_KEY=your-key
EMBEDDING_MODEL=openai-text-embedding-3-large
EMBEDDING_DIMENSIONS=3072

Available Embedding Models

The Admin embedding search reads LiteLLM /model/info and only shows models with model_info.mode = "embedding". Do not add app-side built-in Vertex/OpenAI embedding lists; configure those choices in LiteLLM.

The local LiteLLM instance currently exposes examples such as openai-text-embedding-3-large, openai-text-embedding-3-small, and Mistral embedding models. Dimensions are read from LiteLLM metadata when available.

Setup Steps

1. Database Migration

The database will automatically:

  • Enable the pgvector extension
  • Add embedding vector(768) column to learning_content
  • Create IVFFLAT index for fast similarity search (after 10+ embeddings)

Just restart your server after installing pgvector.

2. Generate Embeddings for Existing Content

Two options:

Option A: Admin API (recommended)

curl -X POST http://localhost:3000/api/admin/learning/embeddings/generate \
  -H "Authorization: Bearer YOUR_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"regenerateAll": false}'

Option B: Via Admin Panel

  • Go to Admin → Learning Hub → Settings
  • Click "Generate Embeddings" button
  • Check status at /api/admin/learning/embeddings/status

3. Verify Setup

Check embedding status:

curl http://localhost:3000/api/admin/learning/embeddings/status \
  -H "Authorization: Bearer YOUR_JWT_TOKEN"

Response:

{
  "success": true,
  "enabled": true,
  "total": 50,
  "withEmbeddings": 50,
  "missing": 0,
  "model": "openai-text-embedding-3-large",
  "dimensions": 3072
}

Using Semantic Search

Keyword Search (existing)

GET /api/learning/search?q=pneumonia

Returns exact text matches in title/subject/body.

Semantic Search (new)

GET /api/learning/search/semantic?q=childhood breathing problems&limit=10&threshold=0.5

Returns content similar by meaning (e.g., finds "pediatric asthma" articles).

Parameters:

  • q (required) - Search query
  • limit (optional, default 10, max 50) - Max results
  • threshold (optional, default 0.5) - Similarity threshold (0-1, higher = more similar)
  • contentType (optional) - Filter by type: article, quiz, pearl, presentation

Hybrid Search (recommended)

GET /api/learning/search/hybrid?q=fever management

Combines keyword + semantic for best results. Automatically deduplicates and ranks by relevance.

How It Works

  1. Content Creation/Update:

    • Text is extracted from title, subject, and body (HTML stripped)
    • Sent to the configured LiteLLM embedding model
    • Returns an embedding vector
    • Stored in learning_content.embedding column

  2. Semantic Search:

    • Query text → embedding vector
    • PostgreSQL pgvector computes cosine similarity
    • Returns top N most similar documents
    • Similarity score 0-1 (1 = identical, 0 = unrelated)

  3. Hybrid Search:

    • Runs both keyword + semantic searches in parallel
    • Merges results (semantic first for quality)
    • Deduplicates by content ID
    • Sorts by relevance score

Cost Estimate

Embedding cost depends on the upstream configured in LiteLLM.

Troubleshooting

"pgvector extension not available"

  • Install: apt-get install postgresql-16-pgvector
  • For Docker: Use pgvector/pgvector:pg16 image

"Embeddings not configured"

  • Verify .env has LITELLM_API_BASE
  • Test: curl http://localhost:3000/api/admin/learning/embeddings/status

"Embedding generation failed"

  • Check logs for API errors
  • Verify LiteLLM /model/info shows the selected model with mode: embedding
  • Check content isn't empty (skips empty bodies)

"No results from semantic search"

  • Check if embeddings exist: /api/admin/learning/embeddings/status
  • Lower threshold: ?threshold=0.3 (default 0.5)
  • Verify pgvector index exists: \di in psql

Performance

  • Embedding generation: latency depends on the LiteLLM upstream
  • Search latency:
    • Keyword: 10-50ms
    • Semantic: 20-100ms (with IVFFLAT index)
    • Hybrid: 30-150ms
  • Index build time: ~1-5 seconds per 1,000 articles

Security And Compliance

  • Compliance: controlled by the upstream provider configured in LiteLLM
  • Data retention: Embeddings stored in your database only
  • No PHI: Only article content (not patient data) is embedded
  • Encryption: TLS in transit, at-rest encryption via PostgreSQL

Example Queries

Before (keyword):

Query: "fever in babies"
Results: Only articles with exact words "fever" or "babies"

After (semantic):

Query: "fever in babies"
Results:
- Infant hyperthermia management (similarity: 0.89)
- Pediatric fever evaluation (similarity: 0.87)
- Febrile seizures in toddlers (similarity: 0.82)
- Neonatal temperature regulation (similarity: 0.78)

Hybrid (best):

Query: "asthma"
Results:
- Childhood asthma management (keyword + semantic: 1.0)
- Pediatric breathing difficulties (semantic: 0.91)
- Reactive airway disease (semantic: 0.86)
- Bronchiolitis vs asthma (keyword: 1.0)

API Reference

Admin Endpoints

  • POST /api/admin/learning/embeddings/generate - Backfill embeddings
  • GET /api/admin/learning/embeddings/status - Check status
  • GET /api/admin/learning/stats - Includes embedding count

User Endpoints

  • GET /api/learning/search - Keyword search
  • GET /api/learning/search/semantic - Semantic search
  • GET /api/learning/search/hybrid - Hybrid search (recommended)

All endpoints require authentication (JWT token).

Questions? Check logs for detailed error messages, or review the code in:

  • /src/utils/embeddings.js - Core embedding logic
  • /src/routes/learningHub.js - Search endpoints
  • /src/routes/learningAdmin.js - Admin management