diff --git a/.env.example b/.env.example index 99232c5..0171ff3 100644 --- a/.env.example +++ b/.env.example @@ -77,29 +77,29 @@ RUN_FS_MIGRATIONS= IMPORT_LIBRARY_DIR= IMPORT_LIBRARY_DIRS= -# Heavy compute backend mode for ONNX whisper alignment + PDF layout parsing. -# local = run compute in-process (default) -# worker = external durable worker mode (requires NATS JetStream-backed compute-worker service) -# Parse progress transport behavior: -# - local mode: in-process memory event bus inside app server -# - worker mode: app proxies compute-worker op events via SSE -# No app-layer NATS configuration is required for parse progress. -COMPUTE_MODE=local -# Required when COMPUTE_MODE=worker +# Heavy compute is always worker-backed. +# App server calls compute-worker over HTTP for ONNX whisper alignment + PDF layout parsing. +# In single-container self-host setups, run compute-worker + NATS alongside the app and +# point COMPUTE_WORKER_URL at that internal worker endpoint. +# Required: # COMPUTE_WORKER_URL=http://localhost:8081 # COMPUTE_WORKER_TOKEN=local-compute-token +# Optional embedded worker stack controls (used by scripts/openreader-entrypoint.mjs): +# START_EMBEDDED_COMPUTE_WORKER=true +# EMBEDDED_COMPUTE_WORKER_PORT=8081 +# EMBEDDED_NATS_PORT=4222 +# EMBEDDED_NATS_MONITOR_PORT=8222 +# EMBEDDED_NATS_STORE_DIR=docstore/nats/jetstream +# NATS_URL=nats://127.0.0.1:4222 # Optional shared compute timeouts used by: -# - local compute runtime # - worker compute runtime # - worker client wait budgets -# Optional local compute concurrency (in-process compute only). -# Worker mode uses compute-worker service `COMPUTE_JOB_CONCURRENCY` instead. +# Worker-side concurrency is configured in compute-worker service via `COMPUTE_JOB_CONCURRENCY`. # COMPUTE_JOB_CONCURRENCY=1 # COMPUTE_WHISPER_TIMEOUT_MS=30000 # COMPUTE_PDF_TIMEOUT_MS=300000 # COMPUTE_OP_STALE_MS=1800000 # Worker mode requires worker-reachable shared object storage. -# Non-exposed embedded weed mini is not supported in worker mode. # Optional Whisper ONNX base URL override (must contain all expected files) # WHISPER_MODEL_BASE_URL=https://huggingface.co/onnx-community/whisper-base_timestamped/resolve/main diff --git a/README.md b/README.md index 1e6b1ad..44ab3b7 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,7 @@ OpenReader is an open source, self-host-friendly text-to-speech document reader ## ✨ Highlights - 🧱 **Layout-aware PDF parsing** with PP-DocLayoutV3 (ONNX) — structured block detection, cross-page stitching, and geometry-based highlighting for precise read-along sync. -- ⏱️ **Word-by-word highlighting** via built-in ONNX Whisper alignment — no external dependencies, runs in-process (`COMPUTE_MODE=local`) or offloaded to a scalable NATS JetStream-backed worker (`COMPUTE_MODE=worker`). +- ⏱️ **Word-by-word highlighting** via ONNX Whisper alignment through the compute worker control plane (NATS JetStream-backed). - ⚡ **Segment-based read-along** for EPUB, PDF, TXT, MD, and DOCX — sentence-aware TTS with cached audio segments, background preloading, and resumable playback. - 🎯 **Multi-provider TTS** — self-hosted OpenAI-compatible servers (Kokoro-FastAPI, KittenTTS-FastAPI, Orpheus-FastAPI) or cloud APIs (OpenAI, Replicate, DeepInfra). - 🎧 **Audiobook export** in `m4b`/`mp3` with resumable chapter processing. diff --git a/docs-site/docs/configure/admin-panel.md b/docs-site/docs/configure/admin-panel.md index a887d4b..3d36571 100644 --- a/docs-site/docs/configure/admin-panel.md +++ b/docs-site/docs/configure/admin-panel.md @@ -78,7 +78,7 @@ Runtime-editable settings, one row per key: | `enableDocxConversion` | Accept .docx uploads (converted to PDF server-side). | | `enableDestructiveDeleteActions` | Show "Delete all data" buttons in the Documents tab (auth-disabled mode). | -Word-by-word highlighting and PDF layout parsing capability are controlled by `COMPUTE_MODE` (server env), not an admin runtime flag. +Word-by-word highlighting and PDF layout parsing capability are controlled by compute-worker server env configuration, not an admin runtime flag. Each row shows a source badge: diff --git a/docs-site/docs/deploy/compute-worker.md b/docs-site/docs/deploy/compute-worker.md index 21171c5..4dff615 100644 --- a/docs-site/docs/deploy/compute-worker.md +++ b/docs-site/docs/deploy/compute-worker.md @@ -1,7 +1,7 @@ title: Compute Worker (NATS JetStream) --- -Use this guide for `COMPUTE_MODE=worker` deployments where heavy compute runs outside the Next.js app server. +Use this guide for worker-backed deployments where heavy compute runs outside the Next.js app server. ## Overview @@ -59,12 +59,11 @@ Advanced tuning (usually leave unset unless you need overrides): - `COMPUTE_NATS_REPLICAS=1` (JetStream stream + KV replicas; valid: `1`, `3`, `5`) - `COMPUTE_OP_STALE_MS=1800000` (stale op replacement window) -## App server environment variables (worker mode) +## App server environment variables Set on the Next.js app server: ```env -COMPUTE_MODE=worker # Local worker example: # COMPUTE_WORKER_URL=http://localhost:8081 # Cloud worker example (Railway): @@ -85,7 +84,7 @@ Model artifact overrides (`WHISPER_MODEL_BASE_URL`, `PDF_LAYOUT_MODEL_BASE_URL`) Set the same value on app + worker envs. -`COMPUTE_MODE=worker` has no local fallback. If worker is unavailable, affected requests fail. +There is no app-local compute fallback. If worker is unavailable, affected requests fail. ## Production notes @@ -184,12 +183,11 @@ Notes: - `COMPUTE_JOBS_STREAM_MAX_BYTES` and `COMPUTE_JOB_STATES_MAX_BYTES` are optional; defaults are `268435456` (256MiB) and `67108864` (64MiB). - `COMPUTE_NATS_REPLICAS` is optional; default is `1`. Valid values are `1`, `3`, `5`. -### 4. Configure the OpenReader app server (worker mode) +### 4. Configure the OpenReader app server Set these env vars on the app server: ```env -COMPUTE_MODE=worker COMPUTE_WORKER_URL=https:// COMPUTE_WORKER_TOKEN= ``` diff --git a/docs-site/docs/deploy/local-development.md b/docs-site/docs/deploy/local-development.md index 22703cf..44da475 100644 --- a/docs-site/docs/deploy/local-development.md +++ b/docs-site/docs/deploy/local-development.md @@ -118,7 +118,7 @@ sudo apt install -y libreoffice No extra native Whisper CLI build step is required. -Set `COMPUTE_MODE=local` to enable built-in ONNX word alignment in-process. +Word-by-word highlighting and PDF layout parsing are worker-backed in current releases. If you need mirrors or pinned artifact locations, set `WHISPER_MODEL_BASE_URL` in `.env`. @@ -148,7 +148,6 @@ pnpm dev For app server worker mode, set: ```env -COMPUTE_MODE=worker COMPUTE_WORKER_URL=http://localhost:8081 COMPUTE_WORKER_TOKEN= ``` @@ -183,6 +182,13 @@ cp .env.example .env Then edit `.env`. +Required compute worker connectivity (all modes): + +```env +COMPUTE_WORKER_URL=http://localhost:8081 +COMPUTE_WORKER_TOKEN= +``` + Use one of these `.env` mode templates: @@ -191,7 +197,6 @@ Use one of these `.env` mode templates: ```env API_BASE=http://host.docker.internal:8880/v1 API_KEY=none -COMPUTE_MODE=local # Leave BASE_URL and AUTH_SECRET unset to keep auth disabled. # (Admin panel is unavailable without auth.) # API_BASE/API_KEY seed a shared default provider if you want shared mode. @@ -203,7 +208,6 @@ COMPUTE_MODE=local ```env API_BASE=http://host.docker.internal:8880/v1 API_KEY=none -COMPUTE_MODE=local BASE_URL=http://localhost:3003 AUTH_SECRET= # Optional when you need multiple local origins: @@ -218,7 +222,6 @@ AUTH_SECRET= # on first boot, then no longer read. Manage them in Settings → Admin afterwards. API_BASE=http://host.docker.internal:8880/v1 API_KEY=none -COMPUTE_MODE=local BASE_URL=http://localhost:3003 AUTH_SECRET= # Comma-separated emails to auto-promote to admin on signin. @@ -231,7 +234,6 @@ ADMIN_EMAILS=you@example.com ```env API_BASE=http://host.docker.internal:8880/v1 API_KEY=none -COMPUTE_MODE=local USE_EMBEDDED_WEED_MINI=false S3_BUCKET=your-bucket S3_REGION=us-east-1 @@ -248,7 +250,6 @@ S3_SECRET_ACCESS_KEY=your-secret-key ```env API_BASE=http://host.docker.internal:8880/v1 API_KEY=none -COMPUTE_MODE=worker COMPUTE_WORKER_URL=http://localhost:8081 COMPUTE_WORKER_TOKEN= USE_EMBEDDED_WEED_MINI=false diff --git a/docs-site/docs/deploy/vercel-deployment.md b/docs-site/docs/deploy/vercel-deployment.md index b587d1b..5218e81 100644 --- a/docs-site/docs/deploy/vercel-deployment.md +++ b/docs-site/docs/deploy/vercel-deployment.md @@ -8,7 +8,7 @@ This guide covers deploying OpenReader to Vercel with external Postgres and S3-c - Documents (PDF/EPUB/TXT/MD) work with `POSTGRES_URL` + external S3 storage. - Audiobook routes work on Node.js serverless functions using `ffmpeg-static`. -- Heavy compute features (Whisper alignment + PDF layout parsing) work through `COMPUTE_MODE=worker` with an external compute worker service. +- Heavy compute features (Whisper alignment + PDF layout parsing) run through an external compute worker service. - For worker setup details and worker-specific env vars, see [Compute Worker (NATS JetStream)](./compute-worker). :::warning DOCX Conversion Limitation @@ -37,10 +37,7 @@ BASE_URL=https://your-app.vercel.app AUTH_SECRET=... ADMIN_EMAILS=you@example.com # comma-separated; admins manage TTS + features in-app -# Heavy compute (recommended on Vercel in v1) -# local = requires native binaries/models in-process (not recommended on Vercel) -# worker = external durable compute worker (recommended) -COMPUTE_MODE=worker +# Heavy compute (required on Vercel in current releases) COMPUTE_WORKER_URL=https:// COMPUTE_WORKER_TOKEN=... @@ -145,4 +142,4 @@ Adjust memory per route if your files are larger or your plan differs. 1. Upload and read a PDF/EPUB document. 2. Confirm sync/blob fetch works across refreshes/devices. 3. Generate at least one audiobook chapter and play/download it. -4. Verify worker-backed word highlighting and PDF parsing in `COMPUTE_MODE=worker`. +4. Verify worker-backed word highlighting and PDF parsing. diff --git a/docs-site/docs/introduction.md b/docs-site/docs/introduction.md index 4759768..6d00019 100644 --- a/docs-site/docs/introduction.md +++ b/docs-site/docs/introduction.md @@ -15,7 +15,7 @@ It supports multiple TTS providers including OpenAI, Replicate, DeepInfra, and c - 🧱 **Layout-aware PDF Parsing** - PP-DocLayoutV3 (ONNX) detects structured blocks with cross-page stitching and geometry-based highlighting for precise read-along sync and clean TTS segmentation - ⏱️ **Word-by-word Highlighting** via ONNX Whisper alignment - - No external dependencies — runs in-process (`COMPUTE_MODE=local`) or offloaded to a scalable NATS JetStream-backed worker (`COMPUTE_MODE=worker`) + - Powered by the external compute worker control plane (NATS JetStream-backed) - ⚡ **Segment-based TTS Playback** - Sentence-aware generation with cached audio segments, background preloading, and resumable playback across EPUB, PDF, TXT, MD, and DOCX - 🎯 **Multi-Provider TTS Support** diff --git a/docs-site/docs/reference/environment-variables.md b/docs-site/docs/reference/environment-variables.md index 07d4ec1..a04de17 100644 --- a/docs-site/docs/reference/environment-variables.md +++ b/docs-site/docs/reference/environment-variables.md @@ -53,12 +53,11 @@ For auth-enabled deployments, use **Settings → Admin** as the primary source o | `RUN_FS_MIGRATIONS` | Storage migrations | `true` | Set `false` to skip startup filesystem -> S3/DB migration pass | | `IMPORT_LIBRARY_DIR` | Library import | `docstore/library` fallback | Set a single server library root | | `IMPORT_LIBRARY_DIRS` | Library import | unset | Set multiple roots (comma/colon/semicolon separated) | -| `COMPUTE_MODE` | Heavy compute backend | `local` | Select `local` (in-process) or `worker` (external worker service) | -| `COMPUTE_WORKER_URL` | Heavy compute backend | unset | Required when `COMPUTE_MODE=worker`; base URL for external compute worker | +| `COMPUTE_WORKER_URL` | Heavy compute backend | unset | Required; base URL for external compute worker | | `COMPUTE_WORKER_TOKEN` | Heavy compute backend | unset | Required bearer token for external compute worker auth | -| `COMPUTE_JOB_CONCURRENCY` | Heavy compute backend | `1` | Local in-process compute concurrency cap; set worker service concurrency in compute-worker env/docs | -| `COMPUTE_WHISPER_TIMEOUT_MS` | Heavy compute backend | `30000` | Shared whisper alignment timeout budget (local + worker + worker client wait budget) | -| `COMPUTE_PDF_TIMEOUT_MS` | Heavy compute backend | `300000` | Shared PDF idle-timeout budget (local + worker + worker client wait budget) | +| `COMPUTE_JOB_CONCURRENCY` | Heavy compute backend | `1` | Worker-side shared compute concurrency cap | +| `COMPUTE_WHISPER_TIMEOUT_MS` | Heavy compute backend | `30000` | Shared whisper alignment timeout budget (worker + worker client wait budget) | +| `COMPUTE_PDF_TIMEOUT_MS` | Heavy compute backend | `300000` | Shared PDF idle-timeout budget (worker + worker client wait budget) | | `COMPUTE_OP_STALE_MS` | Heavy compute backend | `max(30m, 4x max compute timeout)` | Shared stale window for worker op replacement and app-side stale PDF parse-state healing | | `PDF_LAYOUT_MODEL_BASE_URL` | PDF layout model | PP-DocLayoutV3 ONNX base URL | Optional base URL override for `ensureModel()` | | `WHISPER_MODEL_BASE_URL` | Whisper ONNX model | onnx-community defaults | Optional base URL override for ONNX whisper-base_timestamped int8 downloads | @@ -354,40 +353,26 @@ Multiple library roots for server library import. ## Audio Tooling and Alignment -### COMPUTE_MODE - -Selects the backend for heavy compute features (ONNX word alignment + PDF layout parsing). - -- Default: `local` -- Supported in v1: - - `local`: run compute in-process on the app server - - `worker`: enqueue async jobs in an external durable compute worker (NATS JetStream + NATS KV) -- `worker` requires `COMPUTE_WORKER_URL` and `COMPUTE_WORKER_TOKEN` -- `worker` assumes the external worker can directly reach shared object storage (S3-compatible endpoint) -- `worker` is not compatible with non-exposed embedded `weed mini` storage topologies -- Worker service env vars are documented in [Compute Worker (NATS JetStream)](../deploy/compute-worker) - ### COMPUTE_WORKER_URL Base URL for external compute worker mode. -- Used only when `COMPUTE_MODE=worker` +- Required - Example: `http://localhost:8081` ### COMPUTE_WORKER_TOKEN Bearer token for external compute worker auth. -- Used only when `COMPUTE_MODE=worker` +- Required - Must match worker service `COMPUTE_WORKER_TOKEN` ### COMPUTE_JOB_CONCURRENCY -In-process compute concurrency cap for app-server local compute mode. +Worker-side shared compute concurrency cap. - Default: `1` -- Applies when `COMPUTE_MODE=local` -- For `COMPUTE_MODE=worker`, set worker-side `COMPUTE_JOB_CONCURRENCY` in [Compute Worker (NATS JetStream)](../deploy/compute-worker) +- Set on the compute-worker service environment ### COMPUTE_WHISPER_TIMEOUT_MS @@ -395,8 +380,7 @@ Shared whisper alignment timeout budget in milliseconds. - Default: `30000` - Used by: - - Local compute whisper runtime (`COMPUTE_MODE=local`) - - Worker compute whisper runtime (`COMPUTE_MODE=worker`) + - Worker compute whisper runtime - App server worker-client wait budget (SSE wait timeout) ### COMPUTE_PDF_TIMEOUT_MS @@ -405,7 +389,6 @@ Shared PDF idle-timeout budget in milliseconds. - Default: `300000` (5 minutes) - Used by: - - Local compute PDF runtime (idle timeout) - Worker compute PDF runtime (idle timeout) - App server worker-client wait budget (SSE wait timeout) @@ -418,7 +401,7 @@ Shared stale window in milliseconds. - Worker op reuse/replacement guard (`/ops` opKey stale detection) - App-server PDF parse-state stale healing in `/api/documents/[id]/parsed*` - If a parse row is stuck in `pending`/`running` past this window, app routes mark it failed so retries/reparse can proceed. -- In `COMPUTE_MODE=worker`, keep this value aligned on both app-server and worker service envs. +- Keep this value aligned on both app-server and worker service envs. ### PDF_LAYOUT_MODEL_BASE_URL @@ -430,7 +413,7 @@ Optional base URL override for PP-DocLayoutV3 artifacts downloaded by `ensureMod - `PP-DocLayoutV3.onnx.data` - `config.json` - `preprocessor_config.json` -- In `COMPUTE_MODE=worker`, configure this on the worker service env (not only the app server env) +- Configure this on the worker service env (not only the app server env) ### WHISPER_MODEL_BASE_URL @@ -439,7 +422,7 @@ Optional base URL override for the built-in ONNX Whisper alignment model downloa - Default: `https://huggingface.co/onnx-community/whisper-base_timestamped/resolve/main` - Default model variant: int8 (`encoder_model_int8.onnx`, `decoder_model_merged_int8.onnx`, `decoder_with_past_model_int8.onnx`) - The base URL must host all expected manifest files under the same relative paths. -- In `COMPUTE_MODE=worker`, configure this on the worker service env (not only the app server env) +- Configure this on the worker service env (not only the app server env) ### FFMPEG_BIN diff --git a/docs-site/docs/reference/stack.md b/docs-site/docs/reference/stack.md index 5d1d538..aa20370 100644 --- a/docs-site/docs/reference/stack.md +++ b/docs-site/docs/reference/stack.md @@ -59,7 +59,7 @@ Monorepo packages under `compute/`: - Storage: AWS SDK v3 S3 client for reading/writing blobs - Logging: [Pino](https://getpino.io/) - Validation: [Zod](https://zod.dev/) -- Compute mode is controlled by `COMPUTE_MODE` env var: `local` (in-process) or `worker` (remote queue via HTTP + NATS) +- Heavy compute is worker-backed via `COMPUTE_WORKER_URL` + `COMPUTE_WORKER_TOKEN` (remote queue via HTTP + NATS) ## Tooling and testing