pediatric-ai-scribe-v3/BROWSER_WHISPER_SETUP.md

# Browser Whisper Self-Hosted Setup

## Overview

As of v3, Browser Whisper is **fully self-hosted** with **zero CDN dependencies**. All models and libraries are bundled with the application and served from your own server.

## What Changed

**Before (v2 and earlier):**
- Loaded transformers.js from `cdn.jsdelivr.net`
- Downloaded models from `cdn-lfs.huggingface.co`
- Failed in corporate/clinical networks with firewall restrictions

**Now (v3+):**
- Transformers.js library (v2.6.2) bundled at `/models/transformers.min.js` (760KB)
- Whisper model bundled at `/models/Xenova/whisper-tiny.en/` (42MB)
- Everything served from your own server
- **Works in any network environment** (firewalled, air-gapped, offline)

## Files Included

```
public/models/
├── transformers.min.js                    (760KB) - Transformers.js v2.6.2 (worker-compatible)
└── Xenova/
    └── whisper-tiny.en/                   (42MB total)
        ├── config.json
        ├── tokenizer.json
        ├── preprocessor_config.json
        ├── generation_config.json
        └── onnx/
            ├── encoder_model_quantized.onnx
            └── decoder_model_merged_quantized.onnx
```

## How It Works

1. **Worker loads transformers.js locally:**
   ```javascript
   importScripts('/models/transformers.min.js');
   ```

2. **Transformers.js configured for local models:**
   ```javascript
   T.env.localModelPath = '/models/';
   T.env.allowRemoteModels = false;
   ```

3. **Models load from your server:**
   - Browser requests: `GET /models/Xenova/whisper-tiny.en/config.json`
   - Served by Express static middleware
   - No external network calls

## Docker Build

Models are downloaded **during Docker build** (not runtime):

```dockerfile
RUN curl -sL -o onnx/encoder_model_quantized.onnx \
    https://huggingface.co/Xenova/whisper-tiny.en/resolve/main/onnx/encoder_model_quantized.onnx
```

This means:
- Docker image is ~200MB larger (one-time cost)
- Runtime has zero dependencies
- Works in air-gapped environments (after image is pulled)

## Development Setup

If you're running locally (not Docker), download models:

```bash
cd public/models
mkdir -p Xenova/whisper-tiny.en/onnx

# Download transformers.js
curl -L -o transformers.min.js \
  https://cdn.jsdelivr.net/npm/@xenova/transformers@2.17.2/dist/transformers.min.js

# Download model files
cd Xenova/whisper-tiny.en
curl -L -o config.json \
  https://huggingface.co/Xenova/whisper-tiny.en/resolve/main/config.json
curl -L -o tokenizer.json \
  https://huggingface.co/Xenova/whisper-tiny.en/resolve/main/tokenizer.json
curl -L -o preprocessor_config.json \
  https://huggingface.co/Xenova/whisper-tiny.en/resolve/main/preprocessor_config.json
curl -L -o generation_config.json \
  https://huggingface.co/Xenova/whisper-tiny.en/resolve/main/generation_config.json
curl -L -o onnx/encoder_model_quantized.onnx \
  https://huggingface.co/Xenova/whisper-tiny.en/resolve/main/onnx/encoder_model_quantized.onnx
curl -L -o onnx/decoder_model_merged_quantized.onnx \
  https://huggingface.co/Xenova/whisper-tiny.en/resolve/main/onnx/decoder_model_merged_quantized.onnx
```

Or use the helper script:

```bash
./scripts/download-whisper-models.sh
```

## Adding More Models

To add base or small models:

1. **Create directory:**
   ```bash
   mkdir -p public/models/Xenova/whisper-base.en/onnx
   ```

2. **Download from HuggingFace:**
   - https://huggingface.co/Xenova/whisper-base.en
   - https://huggingface.co/Xenova/whisper-small.en

3. **Update UI in `settings.html`:**
   ```html
   <option value="Xenova/whisper-base.en">Base (~74MB, better quality)</option>
   ```

4. **Update Dockerfile** to download during build

## Benefits

✅ **Works everywhere** - No firewall/CDN issues
✅ **Privacy-first** - Audio never leaves browser
✅ **Offline capable** - After initial page load
✅ **No API costs** - Zero transcription expenses
✅ **Predictable** - Same model, same results
✅ **Fast** - Local processing, no network latency

## Limitations

- Docker image is larger (~200MB vs ~150MB)
- Only tiny model included by default (base/small optional)
- Slower than cloud APIs for long recordings
- Requires modern browser with WebAssembly support

## Testing

```bash
# 1. Start server
docker-compose up -d

# 2. Open browser DevTools → Network tab
# 3. Go to Settings → Browser Transcription
# 4. Click "Pre-download model"
# 5. Watch for requests to /models/* (should all be 200 OK from your server)
# 6. NO requests to cdn.jsdelivr.net or huggingface.co
```

## Troubleshooting

**Issue: "Failed to load transformers library"**
- Check: `GET /models/transformers.min.js` returns 200 OK
- Verify file exists: `ls public/models/transformers.min.js`

**Issue: "Model load failed"**
- Check: `GET /models/Xenova/whisper-tiny.en/config.json` returns 200 OK
- Verify files exist: `ls public/models/Xenova/whisper-tiny.en/`

**Issue: Still seeing CDN requests**
- Clear browser cache (Ctrl+Shift+R)
- Check you're running v18+ (`/api/health` should show version)

## Migration from v17

If upgrading from v17:

1. Pull new Docker image: `docker-compose pull`
2. Restart: `docker-compose up -d`
3. Clear browser cache
4. Test: Settings → Browser Transcription → Pre-download

No configuration changes needed - it just works!