danvics/openreader
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
openreader/docs-site/docs/reference/environment-variables.md
Richard R f0e0daae77
feat: improve PDF parsing recovery and reader UI (#98) 
* refactor(ui): replace anchor tags with next/link in SidebarNavLink and UserMenu

Update SidebarNavLink to use next/link for navigation instead of anchor tags,
ensuring proper routing and improved accessibility in Next.js. Refactor UserMenu
to remove legacy Link wrappers and directly use SidebarNavLink for signin and
signup links. This streamlines navigation components and aligns with Next.js
best practices.

* fix(pdf): handle non-zero viewport origins and improve layout model ordering

Update normalizeTextItemsForLayout to correctly apply viewport transforms,
including non-zero page origins, ensuring accurate mapping of PDF text items
to top-left coordinates. Refactor runLayoutModel to implement a custom order
sequence builder for layout regions using model order logits, improving
region ordering consistency with model semantics. Update related tests to
cover viewport transform edge cases.

* refactor(pdf): improve text normalization with font ascent and vertical overlap logic

Enhance text normalization by incorporating font ascent and descent data to more accurately position glyphs, especially for decorative initials. Update merge logic to better detect line membership using vertical overlap, ensuring drop caps and overlapping glyphs are merged correctly. Extend tests to cover these layout scenarios.

* style(range): redesign slider with precision gauge and ruler ticks

Revamp the range input to feature a minimalist "precision gauge" style.
Introduce a hairline rail, ruler notches for discrete steps, and a slim
needle thumb. Add CSS variables and logic for per-instance tick sizing
and coloring. Remove bulky inline class-based styling in favor of
centralized CSS for improved maintainability and visual clarity.

* refactor(ui): modularize PDF loader and range slider visuals

Move PDF layout scan visualization and range slider styles into dedicated CSS
modules, isolating their styles from the global scope. Integrate PdfLayoutScan
component into the PDF viewer loader UI for animated parse progress. Refactor
progress bars to use a reusable progress-fill class with animated sheen effect.
Update range input to use CSS module for precision gauge styling.

* style(reader): remove grid overlay from PdfLayoutScan visualization

* refactor(api): add staleness detection for inflight worker operation states

Integrate isWorkerOperationStateStale checks into document parse API endpoints to ensure inflight worker operation states are not reused if stale. Introduce helper for staleness detection and corresponding unit tests. Enhance SSE event streaming with keepalive intervals and improve progress acknowledgment error handling.

* feat(worker): recover and fail stale in-flight pdf ops on startup

Add orphaned operation recovery logic to detect and mark stale in-flight pdf_layout jobs as failed during worker startup. Extend OperationStateStore with listOpStates for state enumeration. Update tests and documentation to cover recovery behavior and new environment variable COMPUTE_PDF_JOB_ATTEMPTS.

* refactor(worker): distinguish staleness thresholds for running and queued pdf ops

Update orphan recovery logic to apply separate timeouts for 'running' and
'queued' pdf_layout operations. Adjust tests to verify that only stale
'running' operations are failed, while stale 'queued' operations remain
untouched.

* feat(worker): extend orphan recovery to handle whisper_align ops and improve logging

Update orphan recovery to detect and fail stale 'running' whisper_align operations
in addition to pdf_layout. Refactor recovery logic to generalize staleness checks
across operation kinds and enhance log output with detailed operation info.
Expand tests to verify correct handling of both whisper_align and pdf_layout
operations in running and queued states.

* refactor(control-plane): introduce revision-based CAS for operation state updates

Add revision tracking and compare-and-set (CAS) semantics to operation state
stores, enabling atomic state transitions and preventing lost updates. Extend
the OperationStateStore interface with getOpStateRecord and compareAndSetOpState
methods. Update orchestrator and worker runtime to utilize CAS for marking
operations as failed only if the state is unchanged. Enhance in-memory,
JetStream, and test control plane implementations to support revision logic.

This change improves concurrency safety and correctness of operation state
management across distributed components.

* feat(ui): add parse failure state to PDF layout scan animation

Display a distinct "parse halted" visual state in the PDF layout scan
component and PDF viewer page when parsing fails. The loader animation
is replaced by a static, dimmed page with an alert glyph and updated
styling, ensuring users are not misled by an active animation after a
failure. CSS and component logic updated to support the new state.

* feat(worker): extract orphaned operation recovery to module with periodic sweep

Move orphaned operation recovery logic into a dedicated orphan-recovery module,
introducing a periodic sweep timer that triggers recovery every 15 seconds while
the worker is connected. Refactor runtime to delegate orphan detection and
handling to the new module, improving modularity and maintainability. Add
unit tests for orphan-recovery to ensure correctness.

* fix(ui): adjust PDF viewer layout and update parse loader description

* refactor(pdf): streamline layout model region extraction and update test coverage

- Replace custom order sequence logic with softmax-based class selection in runLayoutModel
- Remove unused sigmoid and buildOrderSequence functions
- Simplify detection loop to filter and map regions directly
- Add targeted tests for layout model extraction logic
- Update CSS animation naming for consistency
- Clarify test description and add inline comments for orphan recovery scenario

* fix(pdf): add strict validation for layout model output shapes and extend test coverage

Add explicit error handling for invalid or inconsistent pred_boxes and logits array lengths in runLayoutModel to prevent silent failures. Expand test suite to verify correct region filtering and error scenarios, ensuring only labeled regions are returned and malformed outputs are handled robustly.
2026-06-03 04:44:55 -06:00

473 lines
14 KiB
Markdown
Raw Blame History

---
title: Environment Variables
toc_max_heading_level: 3
---
This page is the source-of-truth reference for OpenReader environment variables.
:::note Recommended configuration path
Use **Settings → Admin** as the primary source of truth for shared providers and runtime site features.
`API_BASE` / `API_KEY` are optional one-time provider bootstrap seeds.
Runtime site features are seeded with `RUNTIME_SEED_JSON` / `RUNTIME_SEED_JSON_PATH`.
:::
## Quick Reference Table
| Variable | Area | Default | When to set |
| --- | --- | --- | --- |
| `LOG_FORMAT` | Runtime logging | `pretty` | Set `json` for structured logs |
| `LOG_LEVEL` | Runtime logging | `info` | Set app server log level |
| `API_BASE` | TTS provider bootstrap seed | unset | Optional first-boot base URL for `default-openai` |
| `API_KEY` | TTS provider bootstrap seed | unset | Optional first-boot API key for `default-openai` |
| `BASE_URL` | Auth | unset | Required at startup |
| `AUTH_SECRET` | Auth | unset | Required at startup |
| `AUTH_TRUSTED_ORIGINS` | Auth | empty | Add extra allowed origins |
| `USE_ANONYMOUS_AUTH_SESSIONS` | Auth | `false` | Set `true` to allow anonymous auth sessions |
| `GITHUB_CLIENT_ID` | Auth/OAuth | unset | Set with `GITHUB_CLIENT_SECRET` to enable GitHub sign-in |
| `GITHUB_CLIENT_SECRET` | Auth/OAuth | unset | Set with `GITHUB_CLIENT_ID` to enable GitHub sign-in |
| `ADMIN_EMAILS` | Admin | empty | Comma-separated emails auto-promoted to admin |
| `POSTGRES_URL` | Database | unset (SQLite mode) | Set to switch metadata/auth DB to Postgres |
| `USE_EMBEDDED_WEED_MINI` | Storage | `true` when unset | Set `false` to use external S3-compatible storage only |
| `WEED_MINI_DIR` | Storage | `docstore/seaweedfs` | Override embedded SeaweedFS data directory |
| `WEED_MINI_WAIT_SEC` | Storage | `20` | Tune SeaweedFS startup wait timeout |
| `S3_ACCESS_KEY_ID` | Storage | auto-generated in embedded mode | Set explicitly for stable/external credentials |
| `S3_SECRET_ACCESS_KEY` | Storage | auto-generated in embedded mode | Set explicitly for stable/external credentials |
| `S3_BUCKET` | Storage | `openreader-documents` in embedded mode | Required for external S3-compatible storage |
| `S3_REGION` | Storage | `us-east-1` in embedded mode | Required for external S3-compatible storage |
| `S3_ENDPOINT` | Storage | derived in embedded mode | Set for S3-compatible providers (MinIO/SeaweedFS/R2/etc.) |
| `S3_FORCE_PATH_STYLE` | Storage | `true` in embedded mode | Set per provider requirement |
| `S3_PREFIX` | Storage | `openreader` | Customize object key prefix |
| `IMPORT_LIBRARY_DIR` | Library import | `docstore/library` fallback | Set a single server library root |
| `IMPORT_LIBRARY_DIRS` | Library import | unset | Set multiple roots (comma/colon/semicolon separated) |
| `EMBEDDED_COMPUTE_WORKER_PORT` | Compute | `8081` | Override embedded worker bind port |
| `EMBEDDED_NATS_PORT` | Compute | `4222` | Override embedded NATS client port |
| `EMBEDDED_NATS_MONITOR_PORT` | Compute | `8222` | Override embedded NATS monitor port |
| `EMBEDDED_NATS_STORE_DIR` | Compute | `docstore/nats/jetstream` | Override embedded JetStream storage directory |
| `NATS_URL` | Compute | `nats://127.0.0.1:4222` in embedded startup | Override embedded startup or set standalone worker URL |
| `COMPUTE_LOG_LEVEL` | Compute | `info` | Compute worker log level |
| `COMPUTE_JOB_CONCURRENCY` | Compute | `1` | Shared compute concurrency cap |
| `COMPUTE_WHISPER_TIMEOUT_MS` | Compute | `30000` | Whisper alignment timeout budget |
| `COMPUTE_PDF_TIMEOUT_MS` | Compute | `300000` | PDF parse timeout budget |
| `COMPUTE_PDF_JOB_ATTEMPTS` | Compute | `1` | Max JetStream deliveries for PDF layout jobs |
| `COMPUTE_OP_STALE_MS` | Compute | `max(30m, 4x max compute timeout)` | Shared stale window for compute op replacement |
| `WHISPER_MODEL_BASE_URL` | Compute model source | onnx-community default | Override Whisper ONNX model base URL |
| `PDF_LAYOUT_MODEL_BASE_URL` | Compute model source | PP-DocLayoutV3 default | Override PDF layout ONNX model base URL |
| `COMPUTE_WORKER_URL` | External compute mode | unset | Set only for standalone external worker mode |
| `COMPUTE_WORKER_TOKEN` | External compute mode | unset | Required for standalone external worker auth |
| `FFMPEG_BIN` | Audio runtime | auto-detected (`ffmpeg-static`) | Override ffmpeg binary path |
| `DISABLE_AUTH_RATE_LIMIT` | Auth request throttling | `false` | Set `true` to disable Better Auth request rate limiting |
| `ENABLE_TEST_NAMESPACE` | Testing/CI | unset | Honor `x-openreader-test-namespace` header in production builds |
| `RUN_DRIZZLE_MIGRATIONS` | DB migrations | `true` | Set `false` to skip startup Drizzle migrations |
| `RUN_FS_MIGRATIONS` | Storage migrations | `true` | Set `false` to skip startup filesystem -> S3/DB migration pass |
| `RUNTIME_SEED_JSON_PATH` | Runtime JSON seed | unset | Absolute path to first-boot JSON seed document |
| `RUNTIME_SEED_JSON` | Runtime JSON seed | unset | Inline first-boot JSON seed document |
## Runtime Logging
### LOG_FORMAT
Controls log output format for server-side Pino loggers.
- Default: `pretty`
- Allowed values: `pretty`, `json`
- Applies to app server and compute worker
### LOG_LEVEL
App server log level.
- Default: `info`
## TTS Provider and Request Behavior
### API_BASE
Optional first-boot bootstrap base URL for the auto-created `default-openai` shared provider.
- Example: `http://host.docker.internal:8880/v1`
- Read only for provider bootstrap when shared providers are empty and `API_KEY` is set.
- After bootstrap, provider configuration is DB-backed and managed in **Settings → Admin → Shared providers**.
### API_KEY
Optional first-boot bootstrap API key for the auto-created `default-openai` shared provider.
- Read only for provider bootstrap when shared providers are empty.
- Stored encrypted at rest after bootstrap.
- After bootstrap, provider configuration is DB-backed and managed in **Settings → Admin → Shared providers**.
### TTS Daily Rate Limiting (Runtime Settings)
Managed as runtime config in **Settings → Admin → Site features**.
- `disableTtsRateLimit` default: `true` (daily TTS limits disabled)
- `ttsDailyLimitAnonymous` default: `50000`
- `ttsDailyLimitAuthenticated` default: `500000`
- `ttsIpDailyLimitAnonymous` default: `100000`
- `ttsIpDailyLimitAuthenticated` default: `1000000`
### TTS Upstream Settings (Runtime Settings)
Managed as runtime config in **Settings → Admin → Site features → TTS upstream**.
- `ttsUpstreamMaxRetries` default: `2`
- `ttsUpstreamTimeoutMs` default: `285000`
- `ttsCacheMaxSizeBytes` default: `268435456` (256 MB)
- `ttsCacheTtlMs` default: `1800000` (30 minutes)
There are no dedicated env vars for these runtime settings.
## Auth and Identity
### BASE_URL
Required external base URL for this OpenReader instance.
- Required at startup
- Example: `http://localhost:3003` or `https://reader.example.com`
### AUTH_SECRET
Required secret key used by auth/session handling.
- Required at startup
- Generate with `openssl rand -hex 32`
### AUTH_TRUSTED_ORIGINS
Additional allowed origins for auth requests.
- Comma-separated list
- `BASE_URL` origin is trusted automatically
### USE_ANONYMOUS_AUTH_SESSIONS
Controls whether auth-enabled deployments can create/use anonymous sessions.
- Default: `false`
### GITHUB_CLIENT_ID
GitHub OAuth client ID.
- Set with `GITHUB_CLIENT_SECRET`
### GITHUB_CLIENT_SECRET
GitHub OAuth client secret.
- Set with `GITHUB_CLIENT_ID`
### ADMIN_EMAILS
Comma-separated list of email addresses auto-promoted to admin.
- Requires auth to be enabled
- Admins can manage shared providers and runtime site features in-app
## Database and Object Blob Storage
### POSTGRES_URL
Switches metadata/auth storage from SQLite to Postgres.
- Unset: SQLite at `docstore/sqlite3.db`
- Set: Postgres mode
### USE_EMBEDDED_WEED_MINI
Controls embedded SeaweedFS startup.
- Default behavior: treated as enabled when unset
- Set `false` to rely on external S3-compatible storage
### WEED_MINI_DIR
Data directory for embedded SeaweedFS (`weed mini`).
- Default: `docstore/seaweedfs`
### WEED_MINI_WAIT_SEC
Max wait time for embedded SeaweedFS startup.
- Default: `20`
### S3_ACCESS_KEY_ID
S3 access key.
- Optional in embedded mode (auto-generated when unset)
- Required for external S3 mode
### S3_SECRET_ACCESS_KEY
S3 secret key.
- Optional in embedded mode (auto-generated when unset)
- Required for external S3 mode
### S3_BUCKET
S3 bucket name.
- Embedded default: `openreader-documents`
- Required for external S3 mode
### S3_REGION
S3 region.
- Embedded default: `us-east-1`
- Required for external S3 mode
### S3_ENDPOINT
Custom endpoint for S3-compatible providers.
- Optional for AWS
- Typical for MinIO/SeaweedFS/R2
### S3_FORCE_PATH_STYLE
Force path-style S3 URLs.
- Embedded default: `true`
### S3_PREFIX
Object key prefix.
- Default: `openreader`
## Library Import
### IMPORT_LIBRARY_DIR
Single library source directory.
### IMPORT_LIBRARY_DIRS
Multiple library roots.
- Supports comma, colon, or semicolon-separated values
## Compute Worker and Model Configuration
### EMBEDDED_COMPUTE_WORKER_PORT
Embedded compute worker port.
- Default: `8081`
### EMBEDDED_NATS_PORT
Embedded NATS client port.
- Default: `4222`
### EMBEDDED_NATS_MONITOR_PORT
Embedded NATS monitor port.
- Default: `8222`
### EMBEDDED_NATS_STORE_DIR
Embedded NATS JetStream data directory.
- Default: `docstore/nats/jetstream`
### NATS_URL
NATS URL used by compute services.
- Embedded startup default: `nats://127.0.0.1:4222`
### COMPUTE_LOG_LEVEL
Compute worker log level.
- Default: `info`
### COMPUTE_JOB_CONCURRENCY
Max concurrent compute jobs per worker.
- Default: `1`
### COMPUTE_WHISPER_TIMEOUT_MS
Whisper alignment timeout budget.
- Default: `30000`
### COMPUTE_PDF_TIMEOUT_MS
PDF parse timeout budget.
- Default: `300000`
### COMPUTE_PDF_JOB_ATTEMPTS
Max JetStream deliveries for PDF layout jobs.
- Default: `1`
- In embedded worker mode, set this in the root `.env`
### COMPUTE_OP_STALE_MS
Stale operation window before worker/app cleanup logic can replace an op.
- Default: `max(30m, 4x max compute timeout)`
### WHISPER_MODEL_BASE_URL
Base URL for Whisper ONNX model downloads.
### PDF_LAYOUT_MODEL_BASE_URL
Base URL for PDF layout model downloads.
### COMPUTE_WORKER_URL
External compute worker URL.
- Leave unset for embedded worker mode
### COMPUTE_WORKER_TOKEN
Shared token for app-to-external-worker requests.
## Compute PDF Parsing Rate Limiting (Runtime Settings)
Managed as runtime config in **Settings → Admin → Site features**.
- `disableComputeRateLimit` default: `true`
- `computeParseBurstMax` default: `8`
- `computeParseBurstWindowSec` default: `60`
- `computeParseSustainedMax` default: `24`
- `computeParseSustainedWindowSec` default: `600`
- `maxUploadMb` default: `200`
There are no dedicated env vars for these runtime settings.
## Audio Runtime
### FFMPEG_BIN
Override ffmpeg binary path used for audio processing.
- Used by audiobook processing routes and compute worker Whisper audio decode.
## Testing and CI
### DISABLE_AUTH_RATE_LIMIT
Disables Better Auth request rate limiting.
- Default: `false`
### ENABLE_TEST_NAMESPACE
Enables the `x-openreader-test-namespace` header path in production builds.
## Migration Controls
### RUN_DRIZZLE_MIGRATIONS
Controls startup Drizzle schema migrations.
- Default: `true`
- Set `false` to skip startup migration run
### RUN_FS_MIGRATIONS
Controls startup filesystem-to-S3/DB migration pass.
- Default: `true`
- Set `false` to skip startup storage migration run
## Runtime JSON Seed (v4)
### RUNTIME_SEED_JSON_PATH
Path-based first-boot seed document.
- If both `RUNTIME_SEED_JSON_PATH` and `RUNTIME_SEED_JSON` are set, path wins.
- Value must point to a JSON file readable by the app process.
### RUNTIME_SEED_JSON
Inline first-boot seed document.
- Used only when `RUNTIME_SEED_JSON_PATH` is unset.
- Must be a JSON object with `version: 1`.
Supported top-level keys:
- `version` (required, must be `1`)
- `runtimeConfig` (optional object, strict-validated against runtime schema)
- `providers` (optional array of shared provider seed entries)
Example:
```json
{
"version": 1,
"runtimeConfig": {
"enableUserSignups": true,
"restrictUserApiKeys": true,
"defaultTtsProvider": "custom-openai",
"enableTtsProvidersTab": true,
"enableAudiobookExport": true,
"enableDocxConversion": true,
"showAllProviderModels": true,
"disableTtsRateLimit": true,
"ttsDailyLimitAnonymous": 50000,
"ttsDailyLimitAuthenticated": 500000,
"ttsIpDailyLimitAnonymous": 100000,
"ttsIpDailyLimitAuthenticated": 1000000,
"ttsCacheMaxSizeBytes": 268435456,
"ttsCacheTtlMs": 1800000,
"ttsUpstreamMaxRetries": 2,
"ttsUpstreamTimeoutMs": 285000,
"disableComputeRateLimit": true,
"computeParseBurstMax": 8,
"computeParseBurstWindowSec": 60,
"computeParseSustainedMax": 24,
"computeParseSustainedWindowSec": 600,
"maxUploadMb": 200,
"changelogFeedUrl": "https://docs.openreader.richardr.dev/changelog/manifest.json"
},
"providers": [
{
"slug": "default-openai",
"displayName": "Default (seeded)",
"providerType": "custom-openai",
"baseUrl": "http://localhost:8880/v1",
"apiKey": "api_key_optional",
"defaultModel": "kokoro",
"enabled": true
}
]
}
```
Provider fallback behavior:
- If the JSON seed includes `providers` (including an empty array), `API_BASE` / `API_KEY` fallback is skipped.
- If the JSON seed does not include a `providers` key, the legacy `API_BASE` / `API_KEY` bootstrap fallback can still create `default-openai` when provider rows are empty.
Precedence summary:
- Runtime reads: admin DB runtime rows override built-in defaults.
- Seed input (`RUNTIME_SEED_JSON*`) only populates missing runtime rows on first boot; it does not overwrite existing/admin-edited rows.
- Provider bootstrap order: JSON `providers` section > `API_BASE`/`API_KEY` fallback > no provider bootstrap.
## Related
- [Admin Panel](../configure/admin-panel)
- [TTS Providers](../configure/tts-providers)
- [Local Development](../deploy/local-development)
- [Vercel Deployment](../deploy/vercel-deployment)
Reference in a new issue View git blame Copy permalink