danvics/openreader
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs(deploy): update compute worker deployment guide for clarity and conciseness

Browse source 
Revise the compute worker deployment documentation to clarify usage
scenarios, streamline environment variable instructions, and improve
separation between embedded and standalone deployment modes. Remove
outdated references and reformat for better readability.
This commit is contained in:
Richard R 2026-06-01 19:47:55 -06:00
parent ebe384c698
commit 06206ad5af
1 changed files with 82 additions and 171 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

253
docs-site/docs/deploy/compute-worker.md
View file

@ -1,221 +1,132 @@
title: Compute Worker (NATS JetStream)
---
title: Compute Worker
description: Deploy the standalone worker used for Whisper alignment and PDF layout parsing.
---
Use this guide when compute-worker runs as a standalone service outside the Next.js app server.
For embedded/local startup (`pnpm dev` / `pnpm start` without `COMPUTE_WORKER_URL`), use root `.env` instead.
Use this guide when OpenReader runs compute as a separate service. For the default embedded/local flow (`pnpm dev` or `pnpm start` without `COMPUTE_WORKER_URL`), configure the root `.env` instead and see [Local Development](./local-development).
## Overview
## What the worker does
The compute worker handles:
- Runs Whisper word alignment jobs
- Runs PDF layout parsing jobs
- Stores durable job state in NATS JetStream and NATS KV
- Whisper word alignment operations
- PDF layout parsing operations
The app server submits work to `POST /ops` and listens for updates on `GET /ops/:opId/events`.
The app server submits operations to `POST /ops`, reuses in-flight work via required `opKey`, and consumes status updates via `GET /ops/:opId/events` (SSE). Queue durability and retries are backed by NATS JetStream WorkQueue consumers and NATS KV.
## When to use it
## Published image
- Required for Vercel-style deployments where heavy compute must run outside the app server
- Useful when you want a dedicated compute host
- Not needed for the default embedded local flow
- App server image: `ghcr.io/richardr1126/openreader`
- Compute worker image: `ghcr.io/richardr1126/openreader-compute-worker`
- Compute worker image (example pinned tag): `ghcr.io/richardr1126/openreader-compute-worker:refactor-ppdoclayoutv3-onnx-layout-parsing`
## Container image
## Worker environment variables
- `ghcr.io/richardr1126/openreader-compute-worker:latest`
Required:
## Worker environment
- `COMPUTE_WORKER_TOKEN`: bearer token expected by worker routes
- `NATS_URL`: NATS server connection string (JetStream enabled)
- `S3_BUCKET`
- `S3_REGION`
- `S3_ACCESS_KEY_ID`
- `S3_SECRET_ACCESS_KEY`
> [!IMPORTANT]
> This file (`compute/worker/.env*`) is only for standalone worker deployments.
> In embedded/local startup, app entrypoint spawns worker with the already-resolved root `.env` values.
> In standalone external worker mode:
> - App server env (root `.env` or platform env): `COMPUTE_WORKER_URL`, `COMPUTE_WORKER_TOKEN`, optional shared timeout/stale overrides.
> - Worker service env (`compute/worker/.env*` or platform env): worker runtime values (`NATS_*`, `S3_*`, model base URLs, worker tuning).
> For standalone worker deployments, keep shared app/worker values aligned:
> - `COMPUTE_WORKER_TOKEN`
> - shared object storage settings (`S3_*`)
> - shared timeout/stale settings (`COMPUTE_WHISPER_TIMEOUT_MS`, `COMPUTE_PDF_TIMEOUT_MS`, `COMPUTE_OP_STALE_MS`)
Common optional:
- `NATS_CREDS`: raw user credentials file content (JWT + private key), ideal for cloud container environments where mounting files is difficult.
- `NATS_CREDS_FILE`: path to a `.creds` file on the server.
- `S3_ENDPOINT` (for non-AWS S3-compatible storage)
- `S3_FORCE_PATH_STYLE=true` (for many S3-compatible providers)
- `S3_PREFIX=openreader`
- `COMPUTE_WORKER_HOST=0.0.0.0`
- `PORT=8081` (local/manual; on Railway platform injects this)
- `LOG_FORMAT=pretty` (default) or `json`
Advanced tuning (usually leave unset unless you need overrides):
- `COMPUTE_PREWARM_MODELS=true`
- `COMPUTE_JOB_CONCURRENCY=1` (shared total compute jobs across whisper + PDF)
- `COMPUTE_WHISPER_TIMEOUT_MS=30000`
- `COMPUTE_PDF_TIMEOUT_MS=300000`
- `WHISPER_MODEL_BASE_URL=https://huggingface.co/onnx-community/whisper-base_timestamped/resolve/main` (optional override, q4 defaults)
- `PDF_LAYOUT_MODEL_BASE_URL=https://huggingface.co/Bei0001/PP-DocLayoutV3-ONNX/resolve/main` (optional override)
- `COMPUTE_PDF_JOB_ATTEMPTS=1` (PDF layout retry attempts)
- `COMPUTE_JOBS_STREAM_MAX_BYTES=268435456` (256MB JetStream jobs stream cap)
- `COMPUTE_JOB_STATES_MAX_BYTES=67108864` (64MB JetStream KV bucket cap)
- `COMPUTE_NATS_REPLICAS=1` (JetStream stream + KV replicas; valid: `1`, `3`, `5`)
- `COMPUTE_OP_STALE_MS=1800000` (stale op replacement window)
## App server environment variables
Set on the Next.js app server:
Required worker variables:
```env
# Local worker example:
# COMPUTE_WORKER_URL=http://localhost:8081
# Cloud worker example (Railway):
COMPUTE_WORKER_URL=https://<railway-worker-domain>
COMPUTE_WORKER_TOKEN=...
NATS_URL=nats://...
S3_BUCKET=...
S3_REGION=...
S3_ACCESS_KEY_ID=...
S3_SECRET_ACCESS_KEY=...
```
:::important
`compute/worker/.env*` is only for standalone worker deployments.
- Embedded/local mode: configure the root `.env` only.
- External worker mode: set `COMPUTE_WORKER_URL` and `COMPUTE_WORKER_TOKEN` on the app, and worker runtime values on the worker service.
- Keep shared values aligned across app and worker: `COMPUTE_WORKER_TOKEN`, `S3_*`, `COMPUTE_WHISPER_TIMEOUT_MS`, `COMPUTE_PDF_TIMEOUT_MS`, and `COMPUTE_OP_STALE_MS`.
:::
Common optional variables:
- `NATS_CREDS` or `NATS_CREDS_FILE`
- `S3_ENDPOINT`, `S3_FORCE_PATH_STYLE=true`, `S3_PREFIX=openreader`
- `COMPUTE_WORKER_HOST=0.0.0.0`
- `PORT=8081` for local/manual runs. Platforms like Railway usually inject `PORT`.
- `LOG_FORMAT=json` and `COMPUTE_LOG_LEVEL=info`
- `COMPUTE_PREWARM_MODELS=true`
- `COMPUTE_JOB_CONCURRENCY=1`
- `COMPUTE_WHISPER_TIMEOUT_MS=30000`
- `COMPUTE_PDF_TIMEOUT_MS=300000`
- `COMPUTE_PDF_JOB_ATTEMPTS=1`
- `COMPUTE_JOBS_STREAM_MAX_BYTES=268435456`
- `COMPUTE_EVENTS_STREAM_MAX_BYTES=134217728`
- `COMPUTE_JOB_STATES_MAX_BYTES=67108864`
- `COMPUTE_NATS_REPLICAS=1`
- `COMPUTE_OP_STALE_MS=1800000`
- `WHISPER_MODEL_BASE_URL`
- `PDF_LAYOUT_MODEL_BASE_URL`
If you need the broader app config reference, see [Environment Variables](../reference/environment-variables).
## App server environment
Set these on the Next.js app server:
```env
COMPUTE_WORKER_URL=https://worker.example.com
COMPUTE_WORKER_TOKEN=<same-token-as-worker>
# Optional shared timeout overrides (keep equal to worker service values):
# Optional shared overrides:
# COMPUTE_WHISPER_TIMEOUT_MS=30000
# COMPUTE_PDF_TIMEOUT_MS=300000
# COMPUTE_OP_STALE_MS=1800000
```
Model artifact overrides (`WHISPER_MODEL_BASE_URL`, `PDF_LAYOUT_MODEL_BASE_URL`) are worker runtime variables and should be set on the compute worker service environment. Current Whisper defaults expect q4 artifacts (`encoder_model_q4.onnx`, `decoder_model_merged_q4.onnx`, `decoder_with_past_model_q4.onnx`) under that base URL.
Notes:
`COMPUTE_OP_STALE_MS` is shared by both services in worker mode:
- Model artifact overrides (`WHISPER_MODEL_BASE_URL`, `PDF_LAYOUT_MODEL_BASE_URL`) belong on the worker service, not the app server.
- There is no app-local compute fallback once `COMPUTE_WORKER_URL` is set. If the worker is unavailable, worker-backed requests fail.
- Worker: opKey stale replacement window in compute op state.
- App server: stale PDF parse-state healing window (`/api/documents/[id]/parsed*`).
## Deployment notes
Set the same value on app + worker envs.
There is no app-local compute fallback. If worker is unavailable, affected requests fail.
## Config ownership summary
- Embedded/local startup (`pnpm dev` / `pnpm start`, no `COMPUTE_WORKER_URL`):
- Configure root `.env` only.
- `compute/worker/.env*` is ignored.
- Standalone external worker service:
- Configure app root `.env` with `COMPUTE_WORKER_URL` + `COMPUTE_WORKER_TOKEN`.
- Configure worker service env (`compute/worker/.env*` or platform env).
- Keep shared values aligned (`COMPUTE_WORKER_TOKEN`, `S3_*`, timeout/stale values).
## Production notes
- Worker mode assumes shared object storage is reachable by both app server and worker.
- Non-exposed embedded `weed mini` is not supported with external worker mode.
- Protect `COMPUTE_WORKER_TOKEN` and avoid exposing worker routes publicly without auth.
## Railway sleep & idle behavior
The worker connects to NATS lazily (on the first request needing the queue/KV) and
disconnects after **120s** of full idle — no in-flight request, SSE stream, job, or
queued work. This stops outbound pull polling and keepalive PINGs so Railway can sleep
it; the next inbound request transparently reconnects, re-ensures the stream/consumers
and KV (idempotent), and drains anything pending. No separate mode, no extra env vars,
and the `/ops*` contract is unchanged.
Caveats: inbound HTTP is the wake signal (in OpenReader the app server only enqueues via
`POST /ops`, so this is always satisfied); a continuous external `/health/*` probe keeps
it awake and prevents sleep; and the first request after a cold start re-runs model
prewarm, so it's slower.
- App and worker must share the same object storage.
- Embedded `weed mini` is not supported for external worker mode.
- Protect `COMPUTE_WORKER_TOKEN` and do not expose worker routes without auth.
- The worker connects to NATS lazily and disconnects after 120 seconds of full idle time. That allows platforms like Railway to sleep the service, but the first request after a cold start will be slower.
## Health endpoints
- `GET /health/live` — liveness; always returns `{ ok: true }`.
- `GET /health/ready` — returns `{ ok: true, natsConnected }`. It does not probe NATS (that
would reconnect and prevent idle sleep); `natsConnected` just reflects the current session.
- `GET /health/live` returns `{ ok: true }`.
- `GET /health/ready` returns `{ ok: true, natsConnected }` and reflects the current NATS session without forcing a reconnect.
## Synadia Cloud + Railway Setup (Complete Guide)
## Railway + Synadia example
Use this end-to-end guide when your queue backend is Synadia Cloud (NGS) and your worker runs on Railway.
### 1. Create Synadia account and credentials
1. Create a Synadia Cloud account and create/select your NGS environment.
2. Create a user or service account for OpenReader compute worker access.
3. Download the generated credentials file (usually `<name>.creds`) and keep it secure.
You will use:
- `NATS_URL=tls://connect.ngs.global:4222`
- The full `.creds` file content
### 2. Deploy compute worker on Railway
Create a Railway service from:
```text
ghcr.io/richardr1126/openreader-compute-worker:refactor-ppdoclayoutv3-onnx-layout-parsing
```
Railway injects a dynamic `PORT` env var and routes traffic there.
Do not hardcode Railway ingress to `8081`; keep service networking enabled and use the public Railway URL.
### 3. Configure Railway worker environment variables
Set these in the Railway worker service:
Deploy the worker image to Railway and set worker env vars similar to:
```env
COMPUTE_WORKER_HOST=0.0.0.0
# Local/manual only:
# PORT=8081
# Railway: rely on injected PORT
COMPUTE_WORKER_TOKEN=<long-random-shared-token>
# Optional advanced tuning overrides (defaults shown):
# COMPUTE_PREWARM_MODELS=true
# COMPUTE_JOB_CONCURRENCY=1
# COMPUTE_WHISPER_TIMEOUT_MS=30000
# COMPUTE_PDF_TIMEOUT_MS=300000
# WHISPER_MODEL_BASE_URL=https://huggingface.co/onnx-community/whisper-base_timestamped/resolve/main
# # Expects q4 files at that base:
# # - onnx/encoder_model_q4.onnx
# # - onnx/decoder_model_merged_q4.onnx
# # - onnx/decoder_with_past_model_q4.onnx
# PDF_LAYOUT_MODEL_BASE_URL=https://huggingface.co/Bei0001/PP-DocLayoutV3-ONNX/resolve/main
# COMPUTE_PDF_JOB_ATTEMPTS=1
# COMPUTE_JOBS_STREAM_MAX_BYTES=268435456
# COMPUTE_JOB_STATES_MAX_BYTES=67108864
# COMPUTE_NATS_REPLICAS=1
COMPUTE_WORKER_TOKEN=<shared-token>
NATS_URL=tls://connect.ngs.global:4222
NATS_CREDS="-----BEGIN NATS USER JWT-----
...
------END USER NKEY SEED------"
S3_BUCKET=<bucket>
S3_REGION=<region>
S3_ACCESS_KEY_ID=<key>
S3_SECRET_ACCESS_KEY=<secret>
S3_ENDPOINT=<optional-for-s3-compatible-providers>
S3_FORCE_PATH_STYLE=true
S3_PREFIX=openreader
# Optional:
# S3_ENDPOINT=https://...
# S3_FORCE_PATH_STYLE=true
# S3_PREFIX=openreader
```
Notes:
If your platform supports mounted files, you can use `NATS_CREDS_FILE` instead of `NATS_CREDS`.
- `NATS_CREDS` should be the full Synadia `.creds` file content, including begin/end markers.
- Keep `COMPUTE_WORKER_TOKEN` identical between app server and worker.
- On Railway, leave `PORT` managed by the platform.
- If your platform supports mounted files, you can use `NATS_CREDS_FILE` instead of `NATS_CREDS`.
- `COMPUTE_JOBS_STREAM_MAX_BYTES` and `COMPUTE_JOB_STATES_MAX_BYTES` are optional; defaults are `268435456` (256MiB) and `67108864` (64MiB).
- `COMPUTE_NATS_REPLICAS` is optional; default is `1`. Valid values are `1`, `3`, `5`.
### 4. Configure the OpenReader app server
Set these env vars on the app server:
Set these on the OpenReader app server:
```env
COMPUTE_WORKER_URL=https://<railway-worker-domain>
COMPUTE_WORKER_TOKEN=<same-token-as-worker>
```
### 5. Verify health
After deploy, check:
Verify the worker after deploy:
- `GET https://<railway-worker-domain>/health/live`
- `GET https://<railway-worker-domain>/health/ready`