danvics/openreader
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
openreader/docs-site/docs/docker-quick-start.md
Richard R e670c38523
Refactor user data cleanup and TTS storage flows (#105) 
* refactor(user): overhaul user data cleanup and export for cascading deletes and TTS segment support

Revise user data cleanup logic to ensure proper cascading deletion of user-related
database rows and S3 objects, including shared document and preview artifacts.
Introduce explicit checks for last ownership before removing shared resources.
Add TTS segment cache and audio cleanup to document and user deletion flows.
Expand user data export to include TTS segment entries and audio, job events,
document settings, and linked auth sessions. Update schema with ON DELETE CASCADE
for userTtsChars and userJobEvents. Add new migration scripts and comprehensive
unit tests for cleanup and export scenarios.

* test(user): expand cleanup and export coverage for shared docs and TTS segment files

Add tests for user data cleanup with shared document ownership and for TTS segment
variant export scenarios, including cases with duplicate audio keys and storage
disabled. Refactor cleanup logic to use document row types and transactional
ownership removal with last-owner checks. Update TTS segment cache clearing to
report deleted segment count by unique entry. Adjust export logic to always
include all TTS segment variants and conditionally export file buckets based on
storage availability. Update migration to use NOT VALID/VALIDATE for new
cascading constraints.

* refactor(documents): centralize owned document deletion and add mutation locking

Introduce `deleteOwnedDocument` utility to encapsulate all logic for removing a user's ownership of a document, including TTS segment cache cleanup, preview artifact removal, and S3 blob deletion for last-owner cases. Add `withDocumentMutationLock` to serialize concurrent mutations on the same document, using advisory locks in Postgres and a local queue fallback. Refactor all API routes and user data flows to use these utilities, ensuring transactional safety and preventing race conditions during document deletion or transfer. Update tests for new flows and add coverage for document cleanup sequencing and locking behavior.

* feat(data): enhance anonymous claim to support document settings and TTS segment copy

Expand the anonymous data claim process to include document settings transfer and TTS segment S3 prefix copying. Remove foreign key constraints from user_tts_chars to allow non-user buckets. Update claim modal and onboarding flow to display claimed document settings. Refactor TTS char count transfer to merge all dates and fix upsert logic. Add S3 copy utility for TTS segments and corresponding tests. Update migrations and schema to reflect relaxed constraints.

* refactor(data): improve error handling and rollback for document and TTS segment operations

Enhance robustness of document deletion and TTS segment transfer by improving
error logging, partial rollback, and degraded state reporting. Add best-effort
cleanup and recovery mechanisms to prevent orphaned data and ensure diagnostic
information is captured for unexpected failures. Update user data cleanup to
log and swallow restoration errors without masking primary failures. Refine
claim logic to handle unmapped TTS audio keys safely.

* refactor(data): unify document mutation locking and transaction handling

Replace mutation-lock with document-lock to centralize document mutation
serialization and database transaction management. Introduce runInDbTransaction
utility to abstract SQLite/Postgres transaction differences. Update document
deletion, user data cleanup, and rate limiter logic to delegate transaction
handling and locking to shared helpers. Remove dialect-specific branching and
inline transaction logic for improved maintainability and testability.

BREAKING CHANGE: withDocumentMutationLock is removed in favor of withDocumentLock and runInDbTransaction

* fix(documents): improve rollback error handling in deleteOwnedDocument

Enhance error handling during document deletion by ensuring that failures
in restoring document ownership do not obscure the original error. Log
degraded events when ownership restoration fails after a deletion error,
providing additional context for debugging and monitoring. This change
improves reliability and traceability of document deletion operations.

* feat(tasks): introduce scheduled task engine and admin UI for background jobs

Add a general-purpose scheduled task system with a persistent registry and status tracking, supporting background maintenance jobs such as orphaned blob reaping, expired upload cleanup, job event pruning, and TTS usage retention. Implement a new `scheduled_tasks` table, task engine, and handlers for each maintenance operation. Integrate an admin UI panel for monitoring, manual runs, and configuration of tasks. Update document and user data cleanup flows to delegate shared blob and preview deletion to the scheduled reaper. Add Vercel cron integration for serverless environments.

BREAKING CHANGE: Document and user storage cleanup now relies on background scheduled tasks for shared blob and preview deletion; immediate inline deletion is no longer performed.

* refactor(user): streamline document and TTS segment transfer logic

Simplify user document transfer by consolidating storage and metadata handling. Remove inline deletion of shared blobs; only metadata is moved, and TTS segment transfer is controlled via options. Add `skipStorage` option for test scenarios to bypass storage operations. Update tests to reflect new transfer behavior.

* chore(instrumentation): delegate node-specific setup to separate module

Move Node.js-specific instrumentation logic to a dedicated file. Update
registration to dynamically import the node module only when running in a
Node.js environment. This separation clarifies environment-specific behavior
and improves maintainability.

* ci(config): update scheduled task cron to run daily at midnight

Change cron schedule for /api/admin/tasks/tick from hourly to once daily at
midnight to reduce task frequency and align with updated operational
requirements.

* refactor(tasks): enforce positive interval and improve scheduled task updates

Add database-level check constraints to ensure scheduled task intervals are always
positive for both Postgres and SQLite. Update admin API and UI to support sub-minute
intervals and stricter validation. Refactor scheduled task update logic to upsert
rows, ensuring tasks can be updated even if not yet present in the database.
Improve temporary upload cleanup to delete in paginated batches. Enhance tests to
cover new constraints and update behaviors.

* chore(docker): remove ffmpeg-static from runtime dependencies in image build

Eliminate ffmpeg-static from the production node_modules during Docker image
assembly to streamline the deployment artifact and avoid bundling unused binaries.

* refactor(admin): redesign task panel UI with Card layout and running indicator

Replace the bordered div layout in AdminTasksPanel with the Card component for
improved visual hierarchy. Add a dynamic running indicator using a custom
RunningDot component to clearly show active tasks. Update control alignment and
button states for better usability. Remove Badge-based status display in favor
of a more streamlined appearance.

* feat(tasks): add document blob lease for safe orphan reaping and improve scheduled task robustness

Introduce a document blob lease mechanism to prevent race conditions between document registration and orphaned blob cleanup. The new `document_blob_leases` table ensures that only one process can claim a document blob for mutation or deletion at a time. Update the orphan reaper to acquire a lease before deleting blobs and to re-check for ownership after acquiring the lease, avoiding accidental deletion of in-flight uploads.

Enhance scheduled task infrastructure with per-task fencing tokens to prevent stale runners from overwriting newer results, enforce runtime limits with abort signals, and expose scheduler mode and minimum interval to the admin panel and API. Adjust task handlers to accept a context with abort support, and update documentation and environment variable references for the new cron secret and scheduling behavior.

BREAKING CHANGE: Scheduled tasks now require a `document_blob_leases` table and updated handler signatures. Vercel deployments must set `CRON_SECRET` for scheduled maintenance.

* refactor(admin): add loading skeleton to tasks panel for improved UX

Introduce a TasksSkeleton component to display animated placeholders while scheduled tasks are loading. Replace direct rendering of empty task rows with the skeleton when data is pending, enhancing perceived responsiveness and user experience in the admin tasks panel.

* test: clean up playwright anonymous users

* test: make blob lease expiry deterministic

* fix(documents): improve blob lease release error handling and test stale lease scenario

Handle errors during blob lease release by logging warnings instead of allowing them to mask original results. Update unit tests to verify that releasing a stale lease does not affect a replacement lease.

* refactor(documents): implement exponential backoff with jitter for blob lease retries

Replace fixed retry delay with capped exponential backoff and jitter to reduce
contention and thundering herd effect when acquiring document blob leases. This
improves lease acquisition fairness and efficiency under high load.
2026-06-07 13:33:20 -06:00

7.5 KiB
Raw Blame History

title
Docker Quick Start

import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem';

Prerequisites

:::warning SeaweedFS Compatibility Note (April 16, 2026) OpenReader currently pins embedded SeaweedFS to 4.18 in CI and Docker builds. 4.19 introduced intermittent InternalError responses on S3 PutObject in our upload flow. :::

Published images

  • App server: ghcr.io/richardr1126/openreader:latest
  • Compute worker (Optional): ghcr.io/richardr1126/openreader-compute-worker:latest
  • Legacy app alias: ghcr.io/richardr1126/openreader-webui:latest

1. Start the Docker container

Persistent storage, embedded SeaweedFS weed mini, required auth, optional library mount:

docker run --name openreader \
  --restart unless-stopped \
  -p 3003:3003 \
  -p 8333:8333 \
  -v openreader_docstore:/app/docstore \
  -e API_BASE=http://host.docker.internal:8880/v1 \
  -e BASE_URL=http://localhost:3003 \
  -e AUTH_SECRET=$(openssl rand -hex 32) \
  -e ADMIN_EMAILS=you@example.com \
  ghcr.io/richardr1126/openreader:latest

What this command enables:

  • -p 3003:3003: exposes the OpenReader web app/API.
  • -p 8333:8333: exposes embedded SeaweedFS S3 endpoint for direct browser presigned upload/download.
  • -v openreader_docstore:/app/docstore: persists SQLite metadata, SeaweedFS blob data, and migration/runtime state.
  • -e API_BASE=... / optional -e API_KEY=...: first-boot seed only. On the first container start, these are auto-migrated into a default-openai admin shared provider stored in the DB (key encrypted at rest when provided). After that, the running app no longer reads them — manage the provider from Settings → Admin → Shared providers. See Admin Panel.
  • -e BASE_URL=... and -e AUTH_SECRET=...: required for v4+ auth/session startup.
  • -e ADMIN_EMAILS=...: (optional, requires auth) comma-separated emails auto-promoted to admin. Admins see the Admin tab in Settings.

Use this when the app should be reachable from other devices on your LAN:

docker run --name openreader \
  --restart unless-stopped \
  -p 3003:3003 \
  -p 8333:8333 \
  -v openreader_docstore:/app/docstore \
  -e API_BASE=http://host.docker.internal:8880/v1 \
  -e BASE_URL=http://<YOUR_LAN_IP>:3003 \
  -e AUTH_SECRET=$(openssl rand -hex 32) \
  -e AUTH_TRUSTED_ORIGINS=http://localhost:3003,http://127.0.0.1:3003 \
  -e USE_ANONYMOUS_AUTH_SESSIONS=true \
  -e ADMIN_EMAILS=you@example.com \
  ghcr.io/richardr1126/openreader:latest

Replace YOUR_LAN_IP with the Docker host IP address on your local network to allow access from other devices.

What this command enables:

  • LAN access from phones/tablets/other computers via http://<YOUR_LAN_IP>:3003.
  • BASE_URL points auth/session cookies and callbacks at your LAN URL.
  • AUTH_TRUSTED_ORIGINS allows localhost loopback origins in addition to your primary LAN origin.
  • USE_ANONYMOUS_AUTH_SESSIONS=true allows guest sessions while auth is enabled.
  • API_BASE seeds the default TTS endpoint into the admin-managed default-openai shared provider on first boot. Edit it from Settings → Admin → Shared providers after that.
  • API_KEY optionally seeds the default provider's key (encrypted at rest). Omit it for an upstream that does not require authentication.
  • ADMIN_EMAILS=... (optional) auto-promotes the listed email(s) to admin so they can manage shared providers and site feature flags from the UI.
  • openreader_docstore volume keeps data persistent across restarts.

Auth required, embedded storage ephemeral, no library import:

docker run --name openreader \
  --restart unless-stopped \
  -p 3003:3003 \
  -p 8333:8333 \
  -e BASE_URL=http://localhost:3003 \
  -e AUTH_SECRET=$(openssl rand -hex 32) \
  ghcr.io/richardr1126/openreader:latest

What this command enables:

  • Fast startup with only the required auth env vars.
  • No persistent volume (/app/docstore stays container-local), so data is ephemeral unless you add a mount.
  • The app still requires BASE_URL + AUTH_SECRET in v4+, so include them even in minimal mode.
  • No TTS provider preset by default. Configure API_BASE and, when required, API_KEY on first boot if you want a seeded shared provider, or run auth+admin mode and manage providers from the admin panel.

:::tip Quick Tips

  • Set API_BASE on first boot to a TTS endpoint the container can reach (host.docker.internal works for host-local services). After first boot, manage providers in Settings → Admin → Shared providers.
  • BASE_URL and AUTH_SECRET are required in v4+. The admin panel requires auth.
  • Set ADMIN_EMAILS to your email if you want the Admin tab in Settings.
  • restrictUserApiKeys controls shared-provider-only mode. For per-user BYOK, toggle it off in Settings → Admin → Site features or seed runtimeConfig.restrictUserApiKeys=false via runtime seed JSON.
  • Use a /app/docstore mount if you want data to survive container/image replacement.
  • Startup automatically runs DB/storage migrations via the shared entrypoint.
  • Scheduled maintenance tasks run in-process and can be managed from Settings → Admin → Scheduled tasks; Docker/self-hosted deployments do not need CRON_SECRET. :::

:::warning Port 8333 Exposure Expose 8333 for direct browser presigned upload/download with embedded SeaweedFS.

If 8333 is not reachable from the browser, direct presigned access is unavailable. Uploads can still fall back to /api/documents/blob/upload/fallback, and document reads/downloads continue through /api/documents/blob. :::

2. Configure settings in the app UI

Visit http://localhost:3003 after startup.

  • If you set ADMIN_EMAILS, sign in with that email and open Settings → Admin to manage shared TTS providers and site feature flags for all users.
  • Per-user: set TTS provider/model in Settings → TTS Provider. API key/base URL inputs are shown only when restrictUserApiKeys=false.
  • Select the model voice from the voice dropdown.

3. Update Docker image

Legacy image compatibility: ghcr.io/richardr1126/openreader-webui:latest remains available as an alias. For external compute mode image details, see Compute Worker (NATS JetStream).

docker stop openreader || true && \
docker rm openreader || true && \
docker image rm ghcr.io/richardr1126/openreader:latest || true && \
docker pull ghcr.io/richardr1126/openreader:latest

:::tip If you use a mounted volume for /app/docstore, your persisted data remains after image updates. :::

:::info Related Docs