danvics/openreader
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs(deploy): update compute worker deployment guide for clarity and conciseness

Browse source 
Revise the compute worker deployment documentation to clarify usage
scenarios, streamline environment variable instructions, and improve
separation between embedded and standalone deployment modes. Remove
outdated references and reformat for better readability.
This commit is contained in:
Richard R 2026-06-01 19:47:55 -06:00
parent ebe384c698
commit 06206ad5af
1 changed files with 82 additions and 171 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

253
docs-site/docs/deploy/compute-worker.md
View file

@ -1,221 +1,132 @@
title: Compute Worker (NATS JetStream) ---
title: Compute Worker
description: Deploy the standalone worker used for Whisper alignment and PDF layout parsing.
--- ---
Use this guide when compute-worker runs as a standalone service outside the Next.js app server. Use this guide when OpenReader runs compute as a separate service. For the default embedded/local flow (`pnpm dev` or `pnpm start` without `COMPUTE_WORKER_URL`), configure the root `.env` instead and see [Local Development](./local-development).
For embedded/local startup (`pnpm dev` / `pnpm start` without `COMPUTE_WORKER_URL`), use root `.env` instead.
## Overview ## What the worker does
The compute worker handles: - Runs Whisper word alignment jobs
- Runs PDF layout parsing jobs
- Stores durable job state in NATS JetStream and NATS KV
- Whisper word alignment operations The app server submits work to `POST /ops` and listens for updates on `GET /ops/:opId/events`.
- PDF layout parsing operations
The app server submits operations to `POST /ops`, reuses in-flight work via required `opKey`, and consumes status updates via `GET /ops/:opId/events` (SSE). Queue durability and retries are backed by NATS JetStream WorkQueue consumers and NATS KV. ## When to use it
## Published image - Required for Vercel-style deployments where heavy compute must run outside the app server
- Useful when you want a dedicated compute host
- Not needed for the default embedded local flow
- App server image: `ghcr.io/richardr1126/openreader` ## Container image
- Compute worker image: `ghcr.io/richardr1126/openreader-compute-worker`
- Compute worker image (example pinned tag): `ghcr.io/richardr1126/openreader-compute-worker:refactor-ppdoclayoutv3-onnx-layout-parsing`
## Worker environment variables - `ghcr.io/richardr1126/openreader-compute-worker:latest`
Required: ## Worker environment
- `COMPUTE_WORKER_TOKEN`: bearer token expected by worker routes Required worker variables:
- `NATS_URL`: NATS server connection string (JetStream enabled)
- `S3_BUCKET`
- `S3_REGION`
- `S3_ACCESS_KEY_ID`
- `S3_SECRET_ACCESS_KEY`
> [!IMPORTANT]
> This file (`compute/worker/.env*`) is only for standalone worker deployments.
> In embedded/local startup, app entrypoint spawns worker with the already-resolved root `.env` values.
> In standalone external worker mode:
> - App server env (root `.env` or platform env): `COMPUTE_WORKER_URL`, `COMPUTE_WORKER_TOKEN`, optional shared timeout/stale overrides.
> - Worker service env (`compute/worker/.env*` or platform env): worker runtime values (`NATS_*`, `S3_*`, model base URLs, worker tuning).
> For standalone worker deployments, keep shared app/worker values aligned:
> - `COMPUTE_WORKER_TOKEN`
> - shared object storage settings (`S3_*`)
> - shared timeout/stale settings (`COMPUTE_WHISPER_TIMEOUT_MS`, `COMPUTE_PDF_TIMEOUT_MS`, `COMPUTE_OP_STALE_MS`)
Common optional:
- `NATS_CREDS`: raw user credentials file content (JWT + private key), ideal for cloud container environments where mounting files is difficult.
- `NATS_CREDS_FILE`: path to a `.creds` file on the server.
- `S3_ENDPOINT` (for non-AWS S3-compatible storage)
- `S3_FORCE_PATH_STYLE=true` (for many S3-compatible providers)
- `S3_PREFIX=openreader`
- `COMPUTE_WORKER_HOST=0.0.0.0`
- `PORT=8081` (local/manual; on Railway platform injects this)
- `LOG_FORMAT=pretty` (default) or `json`
Advanced tuning (usually leave unset unless you need overrides):
- `COMPUTE_PREWARM_MODELS=true`
- `COMPUTE_JOB_CONCURRENCY=1` (shared total compute jobs across whisper + PDF)
- `COMPUTE_WHISPER_TIMEOUT_MS=30000`
- `COMPUTE_PDF_TIMEOUT_MS=300000`
- `WHISPER_MODEL_BASE_URL=https://huggingface.co/onnx-community/whisper-base_timestamped/resolve/main` (optional override, q4 defaults)
- `PDF_LAYOUT_MODEL_BASE_URL=https://huggingface.co/Bei0001/PP-DocLayoutV3-ONNX/resolve/main` (optional override)
- `COMPUTE_PDF_JOB_ATTEMPTS=1` (PDF layout retry attempts)
- `COMPUTE_JOBS_STREAM_MAX_BYTES=268435456` (256MB JetStream jobs stream cap)
- `COMPUTE_JOB_STATES_MAX_BYTES=67108864` (64MB JetStream KV bucket cap)
- `COMPUTE_NATS_REPLICAS=1` (JetStream stream + KV replicas; valid: `1`, `3`, `5`)
- `COMPUTE_OP_STALE_MS=1800000` (stale op replacement window)
## App server environment variables
Set on the Next.js app server:
```env ```env
# Local worker example: COMPUTE_WORKER_TOKEN=...
# COMPUTE_WORKER_URL=http://localhost:8081 NATS_URL=nats://...
# Cloud worker example (Railway): S3_BUCKET=...
COMPUTE_WORKER_URL=https://<railway-worker-domain> S3_REGION=...
S3_ACCESS_KEY_ID=...
S3_SECRET_ACCESS_KEY=...
```
:::important
`compute/worker/.env*` is only for standalone worker deployments.
- Embedded/local mode: configure the root `.env` only.
- External worker mode: set `COMPUTE_WORKER_URL` and `COMPUTE_WORKER_TOKEN` on the app, and worker runtime values on the worker service.
- Keep shared values aligned across app and worker: `COMPUTE_WORKER_TOKEN`, `S3_*`, `COMPUTE_WHISPER_TIMEOUT_MS`, `COMPUTE_PDF_TIMEOUT_MS`, and `COMPUTE_OP_STALE_MS`.
:::
Common optional variables:
- `NATS_CREDS` or `NATS_CREDS_FILE`
- `S3_ENDPOINT`, `S3_FORCE_PATH_STYLE=true`, `S3_PREFIX=openreader`
- `COMPUTE_WORKER_HOST=0.0.0.0`
- `PORT=8081` for local/manual runs. Platforms like Railway usually inject `PORT`.
- `LOG_FORMAT=json` and `COMPUTE_LOG_LEVEL=info`
- `COMPUTE_PREWARM_MODELS=true`
- `COMPUTE_JOB_CONCURRENCY=1`
- `COMPUTE_WHISPER_TIMEOUT_MS=30000`
- `COMPUTE_PDF_TIMEOUT_MS=300000`
- `COMPUTE_PDF_JOB_ATTEMPTS=1`
- `COMPUTE_JOBS_STREAM_MAX_BYTES=268435456`
- `COMPUTE_EVENTS_STREAM_MAX_BYTES=134217728`
- `COMPUTE_JOB_STATES_MAX_BYTES=67108864`
- `COMPUTE_NATS_REPLICAS=1`
- `COMPUTE_OP_STALE_MS=1800000`
- `WHISPER_MODEL_BASE_URL`
- `PDF_LAYOUT_MODEL_BASE_URL`
If you need the broader app config reference, see [Environment Variables](../reference/environment-variables).
## App server environment
Set these on the Next.js app server:
```env
COMPUTE_WORKER_URL=https://worker.example.com
COMPUTE_WORKER_TOKEN=<same-token-as-worker> COMPUTE_WORKER_TOKEN=<same-token-as-worker>
# Optional shared timeout overrides (keep equal to worker service values): # Optional shared overrides:
# COMPUTE_WHISPER_TIMEOUT_MS=30000 # COMPUTE_WHISPER_TIMEOUT_MS=30000
# COMPUTE_PDF_TIMEOUT_MS=300000 # COMPUTE_PDF_TIMEOUT_MS=300000
# COMPUTE_OP_STALE_MS=1800000 # COMPUTE_OP_STALE_MS=1800000
``` ```
Model artifact overrides (`WHISPER_MODEL_BASE_URL`, `PDF_LAYOUT_MODEL_BASE_URL`) are worker runtime variables and should be set on the compute worker service environment. Current Whisper defaults expect q4 artifacts (`encoder_model_q4.onnx`, `decoder_model_merged_q4.onnx`, `decoder_with_past_model_q4.onnx`) under that base URL. Notes:
`COMPUTE_OP_STALE_MS` is shared by both services in worker mode: - Model artifact overrides (`WHISPER_MODEL_BASE_URL`, `PDF_LAYOUT_MODEL_BASE_URL`) belong on the worker service, not the app server.
- There is no app-local compute fallback once `COMPUTE_WORKER_URL` is set. If the worker is unavailable, worker-backed requests fail.
- Worker: opKey stale replacement window in compute op state. ## Deployment notes
- App server: stale PDF parse-state healing window (`/api/documents/[id]/parsed*`).
Set the same value on app + worker envs. - App and worker must share the same object storage.
- Embedded `weed mini` is not supported for external worker mode.
There is no app-local compute fallback. If worker is unavailable, affected requests fail. - Protect `COMPUTE_WORKER_TOKEN` and do not expose worker routes without auth.
- The worker connects to NATS lazily and disconnects after 120 seconds of full idle time. That allows platforms like Railway to sleep the service, but the first request after a cold start will be slower.
## Config ownership summary
- Embedded/local startup (`pnpm dev` / `pnpm start`, no `COMPUTE_WORKER_URL`):
- Configure root `.env` only.
- `compute/worker/.env*` is ignored.
- Standalone external worker service:
- Configure app root `.env` with `COMPUTE_WORKER_URL` + `COMPUTE_WORKER_TOKEN`.
- Configure worker service env (`compute/worker/.env*` or platform env).
- Keep shared values aligned (`COMPUTE_WORKER_TOKEN`, `S3_*`, timeout/stale values).
## Production notes
- Worker mode assumes shared object storage is reachable by both app server and worker.
- Non-exposed embedded `weed mini` is not supported with external worker mode.
- Protect `COMPUTE_WORKER_TOKEN` and avoid exposing worker routes publicly without auth.
## Railway sleep & idle behavior
The worker connects to NATS lazily (on the first request needing the queue/KV) and
disconnects after **120s** of full idle — no in-flight request, SSE stream, job, or
queued work. This stops outbound pull polling and keepalive PINGs so Railway can sleep
it; the next inbound request transparently reconnects, re-ensures the stream/consumers
and KV (idempotent), and drains anything pending. No separate mode, no extra env vars,
and the `/ops*` contract is unchanged.
Caveats: inbound HTTP is the wake signal (in OpenReader the app server only enqueues via
`POST /ops`, so this is always satisfied); a continuous external `/health/*` probe keeps
it awake and prevents sleep; and the first request after a cold start re-runs model
prewarm, so it's slower.
## Health endpoints ## Health endpoints
- `GET /health/live` — liveness; always returns `{ ok: true }`. - `GET /health/live` returns `{ ok: true }`.
- `GET /health/ready` — returns `{ ok: true, natsConnected }`. It does not probe NATS (that - `GET /health/ready` returns `{ ok: true, natsConnected }` and reflects the current NATS session without forcing a reconnect.
would reconnect and prevent idle sleep); `natsConnected` just reflects the current session.
## Synadia Cloud + Railway Setup (Complete Guide) ## Railway + Synadia example
Use this end-to-end guide when your queue backend is Synadia Cloud (NGS) and your worker runs on Railway. Deploy the worker image to Railway and set worker env vars similar to:
### 1. Create Synadia account and credentials
1. Create a Synadia Cloud account and create/select your NGS environment.
2. Create a user or service account for OpenReader compute worker access.
3. Download the generated credentials file (usually `<name>.creds`) and keep it secure.
You will use:
- `NATS_URL=tls://connect.ngs.global:4222`
- The full `.creds` file content
### 2. Deploy compute worker on Railway
Create a Railway service from:
```text
ghcr.io/richardr1126/openreader-compute-worker:refactor-ppdoclayoutv3-onnx-layout-parsing
```
Railway injects a dynamic `PORT` env var and routes traffic there.
Do not hardcode Railway ingress to `8081`; keep service networking enabled and use the public Railway URL.
### 3. Configure Railway worker environment variables
Set these in the Railway worker service:
```env ```env
COMPUTE_WORKER_HOST=0.0.0.0 COMPUTE_WORKER_HOST=0.0.0.0
# Local/manual only: COMPUTE_WORKER_TOKEN=<shared-token>
# PORT=8081
# Railway: rely on injected PORT
COMPUTE_WORKER_TOKEN=<long-random-shared-token>
# Optional advanced tuning overrides (defaults shown):
# COMPUTE_PREWARM_MODELS=true
# COMPUTE_JOB_CONCURRENCY=1
# COMPUTE_WHISPER_TIMEOUT_MS=30000
# COMPUTE_PDF_TIMEOUT_MS=300000
# WHISPER_MODEL_BASE_URL=https://huggingface.co/onnx-community/whisper-base_timestamped/resolve/main
# # Expects q4 files at that base:
# # - onnx/encoder_model_q4.onnx
# # - onnx/decoder_model_merged_q4.onnx
# # - onnx/decoder_with_past_model_q4.onnx
# PDF_LAYOUT_MODEL_BASE_URL=https://huggingface.co/Bei0001/PP-DocLayoutV3-ONNX/resolve/main
# COMPUTE_PDF_JOB_ATTEMPTS=1
# COMPUTE_JOBS_STREAM_MAX_BYTES=268435456
# COMPUTE_JOB_STATES_MAX_BYTES=67108864
# COMPUTE_NATS_REPLICAS=1
NATS_URL=tls://connect.ngs.global:4222 NATS_URL=tls://connect.ngs.global:4222
NATS_CREDS="-----BEGIN NATS USER JWT----- NATS_CREDS="-----BEGIN NATS USER JWT-----
... ...
------END USER NKEY SEED------" ------END USER NKEY SEED------"
S3_BUCKET=<bucket> S3_BUCKET=<bucket>
S3_REGION=<region> S3_REGION=<region>
S3_ACCESS_KEY_ID=<key> S3_ACCESS_KEY_ID=<key>
S3_SECRET_ACCESS_KEY=<secret> S3_SECRET_ACCESS_KEY=<secret>
S3_ENDPOINT=<optional-for-s3-compatible-providers> # Optional:
S3_FORCE_PATH_STYLE=true # S3_ENDPOINT=https://...
S3_PREFIX=openreader # S3_FORCE_PATH_STYLE=true
# S3_PREFIX=openreader
``` ```
Notes: If your platform supports mounted files, you can use `NATS_CREDS_FILE` instead of `NATS_CREDS`.
- `NATS_CREDS` should be the full Synadia `.creds` file content, including begin/end markers. Set these on the OpenReader app server:
- Keep `COMPUTE_WORKER_TOKEN` identical between app server and worker.
- On Railway, leave `PORT` managed by the platform.
- If your platform supports mounted files, you can use `NATS_CREDS_FILE` instead of `NATS_CREDS`.
- `COMPUTE_JOBS_STREAM_MAX_BYTES` and `COMPUTE_JOB_STATES_MAX_BYTES` are optional; defaults are `268435456` (256MiB) and `67108864` (64MiB).
- `COMPUTE_NATS_REPLICAS` is optional; default is `1`. Valid values are `1`, `3`, `5`.
### 4. Configure the OpenReader app server
Set these env vars on the app server:
```env ```env
COMPUTE_WORKER_URL=https://<railway-worker-domain> COMPUTE_WORKER_URL=https://<railway-worker-domain>
COMPUTE_WORKER_TOKEN=<same-token-as-worker> COMPUTE_WORKER_TOKEN=<same-token-as-worker>
``` ```
### 5. Verify health Verify the worker after deploy:
After deploy, check:
- `GET https://<railway-worker-domain>/health/live` - `GET https://<railway-worker-domain>/health/live`
- `GET https://<railway-worker-domain>/health/ready` - `GET https://<railway-worker-domain>/health/ready`