openreader/docs-site/docs/reference/environment-variables.md
Richard R f0e0daae77
feat: improve PDF parsing recovery and reader UI (#98)
* refactor(ui): replace anchor tags with next/link in SidebarNavLink and UserMenu

Update SidebarNavLink to use next/link for navigation instead of anchor tags,
ensuring proper routing and improved accessibility in Next.js. Refactor UserMenu
to remove legacy Link wrappers and directly use SidebarNavLink for signin and
signup links. This streamlines navigation components and aligns with Next.js
best practices.

* fix(pdf): handle non-zero viewport origins and improve layout model ordering

Update normalizeTextItemsForLayout to correctly apply viewport transforms,
including non-zero page origins, ensuring accurate mapping of PDF text items
to top-left coordinates. Refactor runLayoutModel to implement a custom order
sequence builder for layout regions using model order logits, improving
region ordering consistency with model semantics. Update related tests to
cover viewport transform edge cases.

* refactor(pdf): improve text normalization with font ascent and vertical overlap logic

Enhance text normalization by incorporating font ascent and descent data to more accurately position glyphs, especially for decorative initials. Update merge logic to better detect line membership using vertical overlap, ensuring drop caps and overlapping glyphs are merged correctly. Extend tests to cover these layout scenarios.

* style(range): redesign slider with precision gauge and ruler ticks

Revamp the range input to feature a minimalist "precision gauge" style.
Introduce a hairline rail, ruler notches for discrete steps, and a slim
needle thumb. Add CSS variables and logic for per-instance tick sizing
and coloring. Remove bulky inline class-based styling in favor of
centralized CSS for improved maintainability and visual clarity.

* refactor(ui): modularize PDF loader and range slider visuals

Move PDF layout scan visualization and range slider styles into dedicated CSS
modules, isolating their styles from the global scope. Integrate PdfLayoutScan
component into the PDF viewer loader UI for animated parse progress. Refactor
progress bars to use a reusable progress-fill class with animated sheen effect.
Update range input to use CSS module for precision gauge styling.

* style(reader): remove grid overlay from PdfLayoutScan visualization

* refactor(api): add staleness detection for inflight worker operation states

Integrate isWorkerOperationStateStale checks into document parse API endpoints to ensure inflight worker operation states are not reused if stale. Introduce helper for staleness detection and corresponding unit tests. Enhance SSE event streaming with keepalive intervals and improve progress acknowledgment error handling.

* feat(worker): recover and fail stale in-flight pdf ops on startup

Add orphaned operation recovery logic to detect and mark stale in-flight pdf_layout jobs as failed during worker startup. Extend OperationStateStore with listOpStates for state enumeration. Update tests and documentation to cover recovery behavior and new environment variable COMPUTE_PDF_JOB_ATTEMPTS.

* refactor(worker): distinguish staleness thresholds for running and queued pdf ops

Update orphan recovery logic to apply separate timeouts for 'running' and
'queued' pdf_layout operations. Adjust tests to verify that only stale
'running' operations are failed, while stale 'queued' operations remain
untouched.

* feat(worker): extend orphan recovery to handle whisper_align ops and improve logging

Update orphan recovery to detect and fail stale 'running' whisper_align operations
in addition to pdf_layout. Refactor recovery logic to generalize staleness checks
across operation kinds and enhance log output with detailed operation info.
Expand tests to verify correct handling of both whisper_align and pdf_layout
operations in running and queued states.

* refactor(control-plane): introduce revision-based CAS for operation state updates

Add revision tracking and compare-and-set (CAS) semantics to operation state
stores, enabling atomic state transitions and preventing lost updates. Extend
the OperationStateStore interface with getOpStateRecord and compareAndSetOpState
methods. Update orchestrator and worker runtime to utilize CAS for marking
operations as failed only if the state is unchanged. Enhance in-memory,
JetStream, and test control plane implementations to support revision logic.

This change improves concurrency safety and correctness of operation state
management across distributed components.

* feat(ui): add parse failure state to PDF layout scan animation

Display a distinct "parse halted" visual state in the PDF layout scan
component and PDF viewer page when parsing fails. The loader animation
is replaced by a static, dimmed page with an alert glyph and updated
styling, ensuring users are not misled by an active animation after a
failure. CSS and component logic updated to support the new state.

* feat(worker): extract orphaned operation recovery to module with periodic sweep

Move orphaned operation recovery logic into a dedicated orphan-recovery module,
introducing a periodic sweep timer that triggers recovery every 15 seconds while
the worker is connected. Refactor runtime to delegate orphan detection and
handling to the new module, improving modularity and maintainability. Add
unit tests for orphan-recovery to ensure correctness.

* fix(ui): adjust PDF viewer layout and update parse loader description

* refactor(pdf): streamline layout model region extraction and update test coverage

- Replace custom order sequence logic with softmax-based class selection in runLayoutModel
- Remove unused sigmoid and buildOrderSequence functions
- Simplify detection loop to filter and map regions directly
- Add targeted tests for layout model extraction logic
- Update CSS animation naming for consistency
- Clarify test description and add inline comments for orphan recovery scenario

* fix(pdf): add strict validation for layout model output shapes and extend test coverage

Add explicit error handling for invalid or inconsistent pred_boxes and logits array lengths in runLayoutModel to prevent silent failures. Expand test suite to verify correct region filtering and error scenarios, ensuring only labeled regions are returned and malformed outputs are handled robustly.
2026-06-03 04:44:55 -06:00

14 KiB

title toc_max_heading_level
Environment Variables 3

This page is the source-of-truth reference for OpenReader environment variables.

:::note Recommended configuration path Use Settings → Admin as the primary source of truth for shared providers and runtime site features. API_BASE / API_KEY are optional one-time provider bootstrap seeds. Runtime site features are seeded with RUNTIME_SEED_JSON / RUNTIME_SEED_JSON_PATH. :::

Quick Reference Table

Variable Area Default When to set
LOG_FORMAT Runtime logging pretty Set json for structured logs
LOG_LEVEL Runtime logging info Set app server log level
API_BASE TTS provider bootstrap seed unset Optional first-boot base URL for default-openai
API_KEY TTS provider bootstrap seed unset Optional first-boot API key for default-openai
BASE_URL Auth unset Required at startup
AUTH_SECRET Auth unset Required at startup
AUTH_TRUSTED_ORIGINS Auth empty Add extra allowed origins
USE_ANONYMOUS_AUTH_SESSIONS Auth false Set true to allow anonymous auth sessions
GITHUB_CLIENT_ID Auth/OAuth unset Set with GITHUB_CLIENT_SECRET to enable GitHub sign-in
GITHUB_CLIENT_SECRET Auth/OAuth unset Set with GITHUB_CLIENT_ID to enable GitHub sign-in
ADMIN_EMAILS Admin empty Comma-separated emails auto-promoted to admin
POSTGRES_URL Database unset (SQLite mode) Set to switch metadata/auth DB to Postgres
USE_EMBEDDED_WEED_MINI Storage true when unset Set false to use external S3-compatible storage only
WEED_MINI_DIR Storage docstore/seaweedfs Override embedded SeaweedFS data directory
WEED_MINI_WAIT_SEC Storage 20 Tune SeaweedFS startup wait timeout
S3_ACCESS_KEY_ID Storage auto-generated in embedded mode Set explicitly for stable/external credentials
S3_SECRET_ACCESS_KEY Storage auto-generated in embedded mode Set explicitly for stable/external credentials
S3_BUCKET Storage openreader-documents in embedded mode Required for external S3-compatible storage
S3_REGION Storage us-east-1 in embedded mode Required for external S3-compatible storage
S3_ENDPOINT Storage derived in embedded mode Set for S3-compatible providers (MinIO/SeaweedFS/R2/etc.)
S3_FORCE_PATH_STYLE Storage true in embedded mode Set per provider requirement
S3_PREFIX Storage openreader Customize object key prefix
IMPORT_LIBRARY_DIR Library import docstore/library fallback Set a single server library root
IMPORT_LIBRARY_DIRS Library import unset Set multiple roots (comma/colon/semicolon separated)
EMBEDDED_COMPUTE_WORKER_PORT Compute 8081 Override embedded worker bind port
EMBEDDED_NATS_PORT Compute 4222 Override embedded NATS client port
EMBEDDED_NATS_MONITOR_PORT Compute 8222 Override embedded NATS monitor port
EMBEDDED_NATS_STORE_DIR Compute docstore/nats/jetstream Override embedded JetStream storage directory
NATS_URL Compute nats://127.0.0.1:4222 in embedded startup Override embedded startup or set standalone worker URL
COMPUTE_LOG_LEVEL Compute info Compute worker log level
COMPUTE_JOB_CONCURRENCY Compute 1 Shared compute concurrency cap
COMPUTE_WHISPER_TIMEOUT_MS Compute 30000 Whisper alignment timeout budget
COMPUTE_PDF_TIMEOUT_MS Compute 300000 PDF parse timeout budget
COMPUTE_PDF_JOB_ATTEMPTS Compute 1 Max JetStream deliveries for PDF layout jobs
COMPUTE_OP_STALE_MS Compute max(30m, 4x max compute timeout) Shared stale window for compute op replacement
WHISPER_MODEL_BASE_URL Compute model source onnx-community default Override Whisper ONNX model base URL
PDF_LAYOUT_MODEL_BASE_URL Compute model source PP-DocLayoutV3 default Override PDF layout ONNX model base URL
COMPUTE_WORKER_URL External compute mode unset Set only for standalone external worker mode
COMPUTE_WORKER_TOKEN External compute mode unset Required for standalone external worker auth
FFMPEG_BIN Audio runtime auto-detected (ffmpeg-static) Override ffmpeg binary path
DISABLE_AUTH_RATE_LIMIT Auth request throttling false Set true to disable Better Auth request rate limiting
ENABLE_TEST_NAMESPACE Testing/CI unset Honor x-openreader-test-namespace header in production builds
RUN_DRIZZLE_MIGRATIONS DB migrations true Set false to skip startup Drizzle migrations
RUN_FS_MIGRATIONS Storage migrations true Set false to skip startup filesystem -> S3/DB migration pass
RUNTIME_SEED_JSON_PATH Runtime JSON seed unset Absolute path to first-boot JSON seed document
RUNTIME_SEED_JSON Runtime JSON seed unset Inline first-boot JSON seed document

Runtime Logging

LOG_FORMAT

Controls log output format for server-side Pino loggers.

  • Default: pretty
  • Allowed values: pretty, json
  • Applies to app server and compute worker

LOG_LEVEL

App server log level.

  • Default: info

TTS Provider and Request Behavior

API_BASE

Optional first-boot bootstrap base URL for the auto-created default-openai shared provider.

  • Example: http://host.docker.internal:8880/v1
  • Read only for provider bootstrap when shared providers are empty and API_KEY is set.
  • After bootstrap, provider configuration is DB-backed and managed in Settings → Admin → Shared providers.

API_KEY

Optional first-boot bootstrap API key for the auto-created default-openai shared provider.

  • Read only for provider bootstrap when shared providers are empty.
  • Stored encrypted at rest after bootstrap.
  • After bootstrap, provider configuration is DB-backed and managed in Settings → Admin → Shared providers.

TTS Daily Rate Limiting (Runtime Settings)

Managed as runtime config in Settings → Admin → Site features.

  • disableTtsRateLimit default: true (daily TTS limits disabled)
  • ttsDailyLimitAnonymous default: 50000
  • ttsDailyLimitAuthenticated default: 500000
  • ttsIpDailyLimitAnonymous default: 100000
  • ttsIpDailyLimitAuthenticated default: 1000000

TTS Upstream Settings (Runtime Settings)

Managed as runtime config in Settings → Admin → Site features → TTS upstream.

  • ttsUpstreamMaxRetries default: 2
  • ttsUpstreamTimeoutMs default: 285000
  • ttsCacheMaxSizeBytes default: 268435456 (256 MB)
  • ttsCacheTtlMs default: 1800000 (30 minutes)

There are no dedicated env vars for these runtime settings.

Auth and Identity

BASE_URL

Required external base URL for this OpenReader instance.

  • Required at startup
  • Example: http://localhost:3003 or https://reader.example.com

AUTH_SECRET

Required secret key used by auth/session handling.

  • Required at startup
  • Generate with openssl rand -hex 32

AUTH_TRUSTED_ORIGINS

Additional allowed origins for auth requests.

  • Comma-separated list
  • BASE_URL origin is trusted automatically

USE_ANONYMOUS_AUTH_SESSIONS

Controls whether auth-enabled deployments can create/use anonymous sessions.

  • Default: false

GITHUB_CLIENT_ID

GitHub OAuth client ID.

  • Set with GITHUB_CLIENT_SECRET

GITHUB_CLIENT_SECRET

GitHub OAuth client secret.

  • Set with GITHUB_CLIENT_ID

ADMIN_EMAILS

Comma-separated list of email addresses auto-promoted to admin.

  • Requires auth to be enabled
  • Admins can manage shared providers and runtime site features in-app

Database and Object Blob Storage

POSTGRES_URL

Switches metadata/auth storage from SQLite to Postgres.

  • Unset: SQLite at docstore/sqlite3.db
  • Set: Postgres mode

USE_EMBEDDED_WEED_MINI

Controls embedded SeaweedFS startup.

  • Default behavior: treated as enabled when unset
  • Set false to rely on external S3-compatible storage

WEED_MINI_DIR

Data directory for embedded SeaweedFS (weed mini).

  • Default: docstore/seaweedfs

WEED_MINI_WAIT_SEC

Max wait time for embedded SeaweedFS startup.

  • Default: 20

S3_ACCESS_KEY_ID

S3 access key.

  • Optional in embedded mode (auto-generated when unset)
  • Required for external S3 mode

S3_SECRET_ACCESS_KEY

S3 secret key.

  • Optional in embedded mode (auto-generated when unset)
  • Required for external S3 mode

S3_BUCKET

S3 bucket name.

  • Embedded default: openreader-documents
  • Required for external S3 mode

S3_REGION

S3 region.

  • Embedded default: us-east-1
  • Required for external S3 mode

S3_ENDPOINT

Custom endpoint for S3-compatible providers.

  • Optional for AWS
  • Typical for MinIO/SeaweedFS/R2

S3_FORCE_PATH_STYLE

Force path-style S3 URLs.

  • Embedded default: true

S3_PREFIX

Object key prefix.

  • Default: openreader

Library Import

IMPORT_LIBRARY_DIR

Single library source directory.

IMPORT_LIBRARY_DIRS

Multiple library roots.

  • Supports comma, colon, or semicolon-separated values

Compute Worker and Model Configuration

EMBEDDED_COMPUTE_WORKER_PORT

Embedded compute worker port.

  • Default: 8081

EMBEDDED_NATS_PORT

Embedded NATS client port.

  • Default: 4222

EMBEDDED_NATS_MONITOR_PORT

Embedded NATS monitor port.

  • Default: 8222

EMBEDDED_NATS_STORE_DIR

Embedded NATS JetStream data directory.

  • Default: docstore/nats/jetstream

NATS_URL

NATS URL used by compute services.

  • Embedded startup default: nats://127.0.0.1:4222

COMPUTE_LOG_LEVEL

Compute worker log level.

  • Default: info

COMPUTE_JOB_CONCURRENCY

Max concurrent compute jobs per worker.

  • Default: 1

COMPUTE_WHISPER_TIMEOUT_MS

Whisper alignment timeout budget.

  • Default: 30000

COMPUTE_PDF_TIMEOUT_MS

PDF parse timeout budget.

  • Default: 300000

COMPUTE_PDF_JOB_ATTEMPTS

Max JetStream deliveries for PDF layout jobs.

  • Default: 1
  • In embedded worker mode, set this in the root .env

COMPUTE_OP_STALE_MS

Stale operation window before worker/app cleanup logic can replace an op.

  • Default: max(30m, 4x max compute timeout)

WHISPER_MODEL_BASE_URL

Base URL for Whisper ONNX model downloads.

PDF_LAYOUT_MODEL_BASE_URL

Base URL for PDF layout model downloads.

COMPUTE_WORKER_URL

External compute worker URL.

  • Leave unset for embedded worker mode

COMPUTE_WORKER_TOKEN

Shared token for app-to-external-worker requests.

Compute PDF Parsing Rate Limiting (Runtime Settings)

Managed as runtime config in Settings → Admin → Site features.

  • disableComputeRateLimit default: true
  • computeParseBurstMax default: 8
  • computeParseBurstWindowSec default: 60
  • computeParseSustainedMax default: 24
  • computeParseSustainedWindowSec default: 600
  • maxUploadMb default: 200

There are no dedicated env vars for these runtime settings.

Audio Runtime

FFMPEG_BIN

Override ffmpeg binary path used for audio processing.

  • Used by audiobook processing routes and compute worker Whisper audio decode.

Testing and CI

DISABLE_AUTH_RATE_LIMIT

Disables Better Auth request rate limiting.

  • Default: false

ENABLE_TEST_NAMESPACE

Enables the x-openreader-test-namespace header path in production builds.

Migration Controls

RUN_DRIZZLE_MIGRATIONS

Controls startup Drizzle schema migrations.

  • Default: true
  • Set false to skip startup migration run

RUN_FS_MIGRATIONS

Controls startup filesystem-to-S3/DB migration pass.

  • Default: true
  • Set false to skip startup storage migration run

Runtime JSON Seed (v4)

RUNTIME_SEED_JSON_PATH

Path-based first-boot seed document.

  • If both RUNTIME_SEED_JSON_PATH and RUNTIME_SEED_JSON are set, path wins.
  • Value must point to a JSON file readable by the app process.

RUNTIME_SEED_JSON

Inline first-boot seed document.

  • Used only when RUNTIME_SEED_JSON_PATH is unset.
  • Must be a JSON object with version: 1.

Supported top-level keys:

  • version (required, must be 1)
  • runtimeConfig (optional object, strict-validated against runtime schema)
  • providers (optional array of shared provider seed entries)

Example:

{
  "version": 1,
  "runtimeConfig": {
    "enableUserSignups": true,
    "restrictUserApiKeys": true,
    "defaultTtsProvider": "custom-openai",
    "enableTtsProvidersTab": true,
    "enableAudiobookExport": true,
    "enableDocxConversion": true,
    "showAllProviderModels": true,
    "disableTtsRateLimit": true,
    "ttsDailyLimitAnonymous": 50000,
    "ttsDailyLimitAuthenticated": 500000,
    "ttsIpDailyLimitAnonymous": 100000,
    "ttsIpDailyLimitAuthenticated": 1000000,
    "ttsCacheMaxSizeBytes": 268435456,
    "ttsCacheTtlMs": 1800000,
    "ttsUpstreamMaxRetries": 2,
    "ttsUpstreamTimeoutMs": 285000,
    "disableComputeRateLimit": true,
    "computeParseBurstMax": 8,
    "computeParseBurstWindowSec": 60,
    "computeParseSustainedMax": 24,
    "computeParseSustainedWindowSec": 600,
    "maxUploadMb": 200,
    "changelogFeedUrl": "https://docs.openreader.richardr.dev/changelog/manifest.json"
  },
  "providers": [
    {
      "slug": "default-openai",
      "displayName": "Default (seeded)",
      "providerType": "custom-openai",
      "baseUrl": "http://localhost:8880/v1",
      "apiKey": "api_key_optional",
      "defaultModel": "kokoro",
      "enabled": true
    }
  ]
}

Provider fallback behavior:

  • If the JSON seed includes providers (including an empty array), API_BASE / API_KEY fallback is skipped.
  • If the JSON seed does not include a providers key, the legacy API_BASE / API_KEY bootstrap fallback can still create default-openai when provider rows are empty.

Precedence summary:

  • Runtime reads: admin DB runtime rows override built-in defaults.
  • Seed input (RUNTIME_SEED_JSON*) only populates missing runtime rows on first boot; it does not overwrite existing/admin-edited rows.
  • Provider bootstrap order: JSON providers section > API_BASE/API_KEY fallback > no provider bootstrap.