Compare commits

..

730 commits
v0.1.2 ... main

Author SHA1 Message Date
github-actions[bot]
2b27c16367 docs: snapshot v4.3.0
Some checks failed
Docs Deploy / build (push) Has been cancelled
Playwright Tests / e2e-testing (push) Has been cancelled
Vitest Tests / vitest-testing (push) Has been cancelled
Docs Deploy / deploy (push) Has been cancelled
2026-06-13 21:27:08 +00:00
Richard R
dd2c33b01e chore(release): bump version to v4.3.0
Some checks failed
Version Docs on Tag / version-docs (push) Has been cancelled
Create and publish Docker images / prepare (push) Has been cancelled
Create and publish Docker images / build (amd64, ., ./Dockerfile, web, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (amd64, ., ./packages/compute-worker/Dockerfile, compute-worker, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (arm64, ., ./Dockerfile, web, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / build (arm64, ., ./packages/compute-worker/Dockerfile, compute-worker, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / merge (compute-worker, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / merge (web, ubuntu-24.04) (push) Has been cancelled
2026-06-13 15:25:43 -06:00
Richard R
b322339580
Merge pull request #123 from richardr1126/fix/issue-119
Restore PDF Text Highlights After Viewport Resize (Fixes #119)
2026-06-13 15:15:01 -06:00
Richard R
9ee77408fe
Merge pull request #122 from richardr1126/refactor/compute-worker
refactor: modularize monorepo packages, decouple compute-worker, and isolate ML dependencies
2026-06-13 15:14:34 -06:00
Richard R
d4d8f0a590 refactor(bootstrap): set child process cwd to workspace root in CLI spawn
Update CLI command spawning to explicitly set the working directory to the workspace root. This ensures subprocesses execute in a consistent environment, improving reliability in monorepo setups.
2026-06-13 14:38:32 -06:00
Richard R
ee979205b3 fix(storage): unify workspace root resolution for docstore and migrations
Adopt a consistent findWorkspaceRoot helper across bootstrap, storage, and database modules to reliably locate the monorepo root. Update environment file loading and docstore path construction to use the workspace root, ensuring correct file resolution in multi-package setups. This addresses issues with relative paths and improves compatibility in monorepo and containerized environments.
2026-06-13 14:35:27 -06:00
Richard R
74a96c0d15 refactor(compute-worker): improve docstore directory resolution logic
Update resolveDocstoreDir to handle multiple environments:
- Prefer monorepo root for local development
- Check process.env.PWD for containerized bootstrap
- Fallback to standard /app/docstore path
- Default to process cwd

Enhances robustness of docstore path resolution across deployment scenarios.
2026-06-13 14:18:45 -06:00
Richard R
6b13a3394b docs(deploy): add local-slim and local-full Docker Compose examples and update docs
Introduce compose.local-slim.yml and compose.local-full.yml for local builds of
OpenReader and compute-worker. Update deployment documentation to describe new
compose variants and usage. Adjust package.json scripts to support local-slim
and local-full workflows. Rename compose files and update stack names for
clarity.
2026-06-13 14:09:37 -06:00
Richard R
9520e26a00 fix(compute-worker): wrap state store methods to ensure correct invocation
Fix/refactor reconciliation logic to wrap getOpStateRecord and listOpStates methods
from the state store, ensuring they are invoked as functions rather than accessed
as properties. This improves type safety and prevents potential runtime errors
from improper method usage.
2026-06-13 14:03:54 -06:00
Richard R
358a56cc9a refactor: implement modular monorepo packages
- Extract database schema and migrations into @openreader/database

- Extract startup and orchestration scripts into @openreader/bootstrap

- Move compute-worker into packages/compute-worker

- Remove runtime dependency on drizzle-kit

- Update Dockerfile to deploy isolated packages without merging node_modules
2026-06-13 13:51:18 -06:00
Richard R
2abadde949 docs(deploy): add unified Docker Compose guide and update dev workflow
Introduce a comprehensive Docker Compose documentation page covering slim, full, and local-build stack examples. Add example Compose files under docker/examples/ for streamlined deployment. Update local development docs to reference the new Compose guide. Revise package.json scripts to support the new Compose workflows and remove legacy compute-worker compose commands. Update sidebar to include the Docker Compose documentation for improved discoverability.
2026-06-13 13:21:18 -06:00
Richard R
5403aa00cc refactor(compute-worker): modularize operation contracts and API, update protocol naming
- Move operation contracts from api/contracts to operations/contracts for clearer separation
- Delete legacy API modules: contracts.ts, operation-keys.ts, public-operation.ts
- Add new modular files: compute-operation.ts, http-hooks.ts, routes.ts, operations/keys.ts, operations/reconciliation.ts
- Refactor all imports to use new operations/contracts and keys modules
- Rename protocol types from PublicOperation to ComputeOperation for clarity
- Update OpenAPI schema and all references to use ComputeOperation/Event naming
- Extract whisper decoder helpers to inference/whisper/decoder.ts
- Add new infra: nats-session.ts for NATS session management
- Update tests and storage logic to use new contracts and keys modules
- Refactor worker loop and orchestrator to use modular job definitions
- Update client and protocol in src/lib/server/compute-worker to use new ComputeOperation types

This refactor modularizes compute-worker operation logic, clarifies type naming, and improves maintainability by separating contracts, keys, and reconciliation logic. No breaking changes to external API, but all internal references and tests are updated for the new structure.
2026-06-13 05:04:24 -06:00
Richard R
190b67c1d2 Restore PDF text highlights after viewport resize 2026-06-13 04:27:53 -06:00
Richard R
6e69b48103 chore(worker): restructure inference modules and centralize config
- Remove legacy inference and platform files from `src/inference/` and migrate responsibilities to new modular locations under `src/infrastructure/`, `src/inference/pdf/`, and `src/inference/whisper/`
- Consolidate environment/config logic into `src/infrastructure/config.ts`
- Move docstore and ffmpeg platform utilities to `src/infrastructure/platform.ts`
- Refactor PDF and Whisper inference code to use new document layout, layout model, and timestamp utilities
- Update API contracts to define and export types and constants previously scattered in inference/types
- Adjust all imports in jobs, storage, and tests to reference new module structure and type locations
- Inline parser version and encoding logic into API contracts
- Remove obsolete files and update test fixtures for new type locations

This update improves codebase modularity, maintainability, and separation of concerns. No changes to inference or orchestration functionality.

BREAKING CHANGE: inference module structure, type imports, and config utilities have changed; downstream code must update imports and integration points.
2026-06-12 14:16:13 -06:00
Richard R
3d5dfd2a88 refactor(compute-worker): migrate compute logic to modular architecture
- Remove legacy compute-worker monolith under `src/compute/`, including PDF and Whisper inference, control-plane, and runtime orchestration
- Move PDF and Whisper inference logic to new `src/inference/` module
- Move control-plane, orchestrator, and state machine logic to `src/operations/`
- Move NATS/JetStream adapters to `src/infrastructure/`
- Move job orchestration, progress, and artifact persistence to `src/jobs/`
- Update imports throughout tests and main app to reference new module structure
- Update Dockerfile and scripts to use new asset paths
- Add new entrypoints: `src/api/app.ts` and `src/api/contracts.ts`
- Remove obsolete files and update `.gitignore` for new dev artifacts

This refactor modularizes the compute-worker, improving maintainability and separation of concerns. No functional changes to inference or orchestration logic.

BREAKING CHANGE: compute-worker internal APIs, directory structure, and imports have changed; downstream code must update imports and integration points.
2026-06-12 14:07:15 -06:00
Richard R
a56aaa2be9 Refactor PDF parsing and rendering logic; migrate to compute-worker protocol
- Removed deprecated functions related to document parsing and blob storage in blobstore.ts.
- Introduced new PDF rendering logic in pdf-preview-renderer.ts and pdf-preview-pdfjs-runtime.ts.
- Updated previews-render.ts to utilize the new PDF rendering functions.
- Refactored user-whisper-align-job.ts to use the compute-worker client for alignment requests.
- Enhanced artifact.ts and operation.ts to validate parsed PDF artifacts and resolve current PDF parses.
- Updated snapshot.ts to align with new worker operation types.
- Adjusted runtime-config.ts to check for compute-worker availability.
- Modified types in parsed-pdf.ts and tts.ts to reflect changes in the compute-worker protocol.
- Added unit tests for PDF artifact validation and compute-worker client contract.
- Removed obsolete pdf-op-key.vitest.spec.ts test file.
2026-06-12 13:43:33 -06:00
Richard R
4bfd020125 fix(auth): prevent unwanted anonymous session after logout by tracking session state
Track previous session existence to avoid minting a new anonymous user when a
session ends and the user is redirected to the sign-in page. This prevents
clobbering account-scoped preferences by ensuring bootstrapping does not occur
during the logout redirect flow.
2026-06-12 08:30:23 -06:00
Richard R
b8621e64fb feat(worker): add detailed idle status logging and activity reason tracking
Add periodic idle status logs with activity context to improve monitoring and
debugging of worker idle/disconnect behavior. Track last activity reason and
include it in status logs and idle disconnect eligibility checks. Update
activity marking calls to specify reason for more granular observability.
2026-06-12 08:30:07 -06:00
Richard R
d216ecb713
Merge pull request #118 from btpod/btpod/speech-sdk-provider
feat(tts): add Speech SDK provider for ElevenLabs, Cartesia, Deepgram, Google, Inworld, and more
2026-06-11 18:18:40 -06:00
Bilal Tahir
6da2ba2a9a fix(tts): apply upstream timeout and validate provider/model form in Speech SDK path
Review-driven: thread ttsUpstreamTimeoutMs into the speech-sdk branch as an
overall budget across SDK retries, and fail fast on model ids missing the
provider/model form instead of falling through to provider defaults.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 12:19:31 -07:00
Bilal Tahir
801f0dcca2 feat(tts): add Speech SDK provider for ElevenLabs, Cartesia, Deepgram, Google, Inworld, and more
Adds 'speech-sdk' as a fifth built-in TTS provider backed by @speech-sdk/core
(Apache 2.0). Models use the provider/model format; requests go from the
server directly to the provider's API with the user's own key. Existing
providers, defaults, and config are untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 10:39:20 -07:00
github-actions[bot]
a2e08976b6 docs: snapshot v4.2.2 2026-06-10 20:39:04 +00:00
Richard R
8ab3a0a159 chore(release): bump version to v4.2.2
Some checks failed
Create and publish Docker images / prepare (push) Has been cancelled
Version Docs on Tag / version-docs (push) Has been cancelled
Create and publish Docker images / build (amd64, ., ./Dockerfile, web, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (amd64, ., ./compute/worker/Dockerfile, compute-worker, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (arm64, ., ./Dockerfile, web, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / build (arm64, ., ./compute/worker/Dockerfile, compute-worker, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / merge (compute-worker, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / merge (web, ubuntu-24.04) (push) Has been cancelled
2026-06-10 14:37:14 -06:00
Richard R
a2f312dc35 fix(uploader): Fix bug introduced with new document uploader menu where "Click to select" box was not allowing clicks 2026-06-10 14:34:00 -06:00
Richard R
3258c04ac6 build(config): add custom externals mapping for @napi-rs/canvas in server webpack
Configure webpack to treat @napi-rs/canvas as an external dependency under
the 'canvas' module name when building the server bundle. This adjustment
ensures proper module resolution and avoids bundling native dependencies
directly, improving deployment reliability for environments requiring
@napi-rs/canvas.
2026-06-10 14:26:40 -06:00
Richard R
2d205008ef fix(tts): restrict audio output URLs to replicate.delivery to prevent SSRF
Add explicit host validation to only allow audio output URLs from
replicate.delivery and its subdomains, blocking potential SSRF attacks
from malicious model outputs. Update tests to cover allowed and
disallowed host scenarios.
2026-06-10 13:57:57 -06:00
Richard R
cecb2489e1
Merge pull request #116 from richardr1126/feat/ui-improvements
feat(documents): add web URL import, markdown/text creator, and unified sidebar modal dialog
2026-06-10 13:53:53 -06:00
Richard R
7814517b35 style(doclist): enhance preview background with color-mix and radial mask
Update DocumentPreview text preview to use a CSS color-mix radial pattern
with increased opacity and a masked vignette effect. Improves visual
clarity and consistency with design tokens.
2026-06-10 13:44:08 -06:00
Richard R
08b8b06f27 refactor(documents): replace custom SSRF checks with AntiSSRF library
Switch SSRF protection in web-loader to use @microsoft/antissrf, removing
custom IP range logic. This centralizes and future-proofs SSRF mitigation by
leveraging maintained policy sets and DNS-aware enforcement. Error handling in
the import-url API route is updated to recognize new policy error messages.
Dependency added to package.json.
2026-06-10 13:29:18 -06:00
Richard R
11b2f18691 fix(documents): expand private IP detection and improve URL resolution
Add detection for CGNAT (100.64.0.0/10) and benchmarking (198.18.0.0/15) ranges
in IPv4 private IP checks. Update IPv6 site-local detection to include full
range. Refactor fetchWithLimit to return both HTML and the resolved final URL,
ensuring relative URLs are resolved against the actual fetched location.
2026-06-10 13:13:34 -06:00
Richard R
56305d5c1b refactor(documents): improve SSRF protection and filename sanitization
Enhance SSRF mitigation in web-loader by expanding private IP detection for both
IPv4 and IPv6, including checks for multicast and broadcast addresses. Update
filename sanitization logic in UploadMenuDialog to collapse consecutive
disallowed characters and trim underscores, ensuring cleaner filenames. Remove
redundant userId check in import-url API route for streamlined authentication.
2026-06-10 12:59:14 -06:00
Richard R
c2ff4320bf feat(documents): allow custom document title for web imports
Add input field for specifying a custom document title when importing from a web URL.
Use the provided title for the generated markdown file name and success message,
falling back to the scraped title if left empty. Reset the title field after import
completion for improved UX.
2026-06-10 11:19:15 -06:00
Richard R
d86df06f9b style(ui): unify modal dialogs with SidebarDialog component
Replace ModalFrame and related modal patterns in settings and upload dialogs with
the new SidebarDialog component to establish a consistent sidebar-based dialog
layout. Refactor SettingsModal and UploadMenuDialog to use SidebarDialog, updating
header, navigation, and content handling accordingly. Update DocumentList to use
DocumentUploader in compact mode for sidebar upload action. Export SidebarDialog
from the UI module. Minor fixes in web-loader for IP parsing and option naming.

This change standardizes dialog UI structure and improves maintainability.
2026-06-10 11:14:33 -06:00
Richard R
5a44a59ed1 feat(documents): add web import and creation dialog with SSRF-safe loader
Introduce a unified upload menu dialog enabling users to upload files,
create markdown/text documents, or import content directly from web URLs.
Implement a secure server-side loader for fetching and parsing web pages
with SSRF protection, content-type checks, and size limits. Add new API
route for importing documents from URLs, and update dependencies to
support HTML parsing and markdown conversion.

- Add UploadMenuDialog component with file, create, and URL import tabs
- Implement SSRF-mitigated web-loader using linkedom, readability, turndown
- Add /api/documents/import-url endpoint for web imports
- Update DocumentList to use new upload dialog
- Add BrowserIcon and supporting UI elements
- Update dependencies: @mozilla/readability, linkedom, turndown
- Expose importUrl client API

This improves document onboarding flexibility and enhances security for
web-based imports.
2026-06-10 10:57:51 -06:00
Richard R
ea4bcb5cf0 style(ui): enhance changelog panel link appearance and hover interaction 2026-06-10 10:19:30 -06:00
Richard R
c799a9ece5
Merge pull request #115 from richardr1126/fix/worker-sleep-railway
fix(worker): stop a silent SSE stream from pinning the compute worker awake
2026-06-10 10:00:53 -06:00
Richard R
9f9e612071
Merge pull request #114 from richardr1126/fix/text-cleanup
fix(tts): strip bullet/decorative glyphs and centralize audio-text cleaning
2026-06-10 09:59:20 -06:00
Richard R
fd8d462d6e test(nlp): add case to ensure Catalan middle dot is preserved in audio text normalization
Add unit test verifying that U+00B7 middle dot is not stripped from Catalan
words during audio text preprocessing, reflecting multilingual requirements.
2026-06-10 09:38:41 -06:00
Richard R
5736419bdb refactor(audio-text): centralize audio text normalization rules and update consumers
Move canonical audio text preprocessing logic to new shared module `audio-text.ts`
to ensure consistent normalization across worker, client, and NLP code. Update
all consumers to import and use the shared rules, eliminating duplication and
preventing drift. Expand glyph stripping to handle additional decorative
characters. Add unit tests for new cases and update documentation accordingly.
2026-06-10 09:00:13 -06:00
Richard R
a284ee852e feat(worker): throttle EventSource reconnects after idle disconnect
Add SSE `retry:` directive to suggest a 2-minute reconnect delay when the worker
enters idle sleep and drops silent EventSource streams. This prevents clients
from immediately reconnecting and re-waking the container, reducing unnecessary
resource usage. Update SSE encoding to support the `retry` field and add tests
for correct behavior. Improve idle disconnect logging for better observability.
2026-06-10 07:05:34 -06:00
github-actions[bot]
92e93d4231 docs: snapshot v4.2.1 2026-06-10 00:01:38 +00:00
Richard R
511eeb735c chore(release): bump version to v4.2.1
Some checks failed
Create and publish Docker images / prepare (push) Has been cancelled
Version Docs on Tag / version-docs (push) Has been cancelled
Create and publish Docker images / build (amd64, ., ./Dockerfile, web, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (amd64, ., ./compute/worker/Dockerfile, compute-worker, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (arm64, ., ./Dockerfile, web, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / build (arm64, ., ./compute/worker/Dockerfile, compute-worker, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / merge (compute-worker, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / merge (web, ubuntu-24.04) (push) Has been cancelled
2026-06-09 17:59:47 -06:00
Richard R
5e8d6a6fa8
Merge pull request #112 from richardr1126/fix/epub-text-highlight-2
fix(highlight): unify EPUB/HTML word highlighting on shared token-sequence aligner
2026-06-09 17:54:33 -06:00
Richard R
c192d6d729 fix(epub): improve word-region cache invalidation for highlighting
Update the EPUB highlighting cache key to include aligned word texts, ensuring
that changes in alignment or word content invalidate stale cached spans even if
word counts remain unchanged. Clear the word-region cache when rendered text
maps are replaced or reset, preventing incorrect highlights after remapping.

Also add locale tracking to HTML sentence highlighting state to support
language-aware word segmentation during alignment.
2026-06-09 15:35:39 -06:00
Richard R
8131b2bee9 refactor(epub): unify word-to-region alignment for text highlighting
Replace EPUB-specific word highlight mapping with shared token-sequence alignment
logic used by HTML and PDF viewers. Remove bespoke alignment code and tests in
favor of a single primitive (`locateAlignmentWordSpans`) that robustly maps
spoken words to rendered text regions, tolerant of transcription and formatting
differences. Update EPUB highlighting to cache per-segment word-region spans for
efficient re-use. Add unit tests for the new alignment logic.
2026-06-09 14:57:27 -06:00
Richard R
65c9f03976 fix(tts): write mp3 to temp file to ensure accurate VBR duration metadata
Switch mp3 transcoding to output a seekable temp file instead of piping to stdout.
This allows ffmpeg to backpatch the Xing/Info header with correct total-duration
metadata, which is required for HTML5 audio elements to handle VBR mp3 playback
accurately. Prevents playback issues caused by missing or incorrect duration,
such as premature or stalled `ended` events.
2026-06-09 11:19:11 -06:00
Richard R
34b94534cf docs(tts): clarify fallback voice set logic for OpenAI-compatible providers
Update documentation to specify that OpenReader uses the Kokoro voice set
for Kokoro models and the standard OpenAI voices for other models when no
voice listing endpoint is available. Improves clarity on provider fallback
behavior.
2026-06-09 11:00:51 -06:00
Richard R
0eec914090 fix(voices): unify default voice set for custom-openai non-kokoro models
Align non-kokoro custom-openai models to use the canonical OpenAI voice set
instead of a separate custom set. Updates logic and tests to reflect this
standardization for improved consistency across OpenAI-compatible providers.
2026-06-09 10:56:54 -06:00
Richard R
d581df6f88 fix(auth): prevent anonymous session creation on auth pages
Avoid bootstrapping anonymous sessions when users are on sign-in or sign-up
pages to prevent overwriting account-scoped preferences after login. Adjust
loading logic to account for this scenario.
2026-06-09 10:53:01 -06:00
Richard R
335b78b435 docs(tts): clarify OpenAI-compatible provider requirements and supported formats
Update documentation to specify that only the speech endpoint is required for
OpenAI-compatible TTS providers, with voice listing now optional and auto-discovered.
Document support for multiple audio formats (mp3, wav, ogg, flac) and note that
API keys are optional for local or unauthenticated servers. Add Supertonic to
the list of compatible implementations and improve troubleshooting guidance.
2026-06-09 10:44:16 -06:00
Richard R
3f07716b31
Merge pull request #111 from richardr1126/feat/support-wav-as-mp3
feat(tts): support OpenAI-compatible servers that emit wav instead of mp3

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

* **New Features**
  * Normalize all incoming TTS audio to MP3 so stored/served audio is consistently MP3.
  * Broader discovery of voices from OpenAI‑compatible endpoints by probing multiple voice routes and respecting provider defaults.
* **Reliability**
  * Retry flow when providers reject explicit response format to improve compatibility.
  * Improved audio format detection, safer transcoding with cancellation and cleanup, and logging when normalization occurs.
* **Tests**
  * Added unit tests covering audio format detection across multiple codecs.
* **Documentation**
  * Added a Supertonic TTS provider setup guide.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-09 10:36:38 -06:00
Richard R
116b191c9a fix(tts): allow OpenAI-compatible TTS with empty apiKey by using placeholder
Update OpenAI SDK instantiation to provide a placeholder apiKey when none is
supplied, enabling support for local or unauthenticated OpenAI-compatible
TTS servers. Ensure Authorization header is cleared when not needed.
2026-06-09 10:32:11 -06:00
Richard R
c1e2b5a5f0 docs(supertonic): add Supertonic TTS provider guide and update docs navigation
Include documentation for configuring the Supertonic TTS provider. Update
the main TTS providers list and sidebar to reference the new guide, ensuring
users can easily find setup instructions for Supertonic.
2026-06-09 10:27:33 -06:00
Richard R
e85641bcfe refactor(tts): enhance mp3 format detection and improve ffmpeg abort handling
Update audio format sniffing to require a full 10-byte ID3 header for mp3 detection and add stricter validation of MPEG frame headers to avoid false positives. Refactor ffmpeg process spawning to register abort listeners before module import, ensuring abort signals are handled promptly and preventing orphaned processes. Expand tests to cover edge cases in mp3 header detection.
2026-06-09 09:58:40 -06:00
Richard R
f09d14549a fix(tts): encode normalized audio at VBR q2 instead of 64k to reduce fatigue
The wav->mp3 normalization path re-encoded pristine high-fidelity TTS (e.g.
Supertonic's 44.1 kHz wav) at 64k CBR, which lowpasses hard and introduces
high-frequency artifacts that cause listening fatigue. Switch the live/segment
transcode to libmp3lame VBR -q:a 2 (~170-210 kbps for speech), effectively
transparent. The audiobook export still re-encodes to 64k for file size.
2026-06-09 09:52:37 -06:00
Richard R
6c12a47f74
Merge pull request #110 from richardr1126/fix/epub-text-highlight
fix(epub/text): native offset word-by-word highlighting
2026-06-09 09:19:52 -06:00
Richard R
5a0b40658a fix(tts): honor explicit empty voices list in custom-openai probing
extractVoiceNames collapsed an explicit `{ voices: [] }` response to null,
which broke the existing contract of treating an empty list as a valid
"no voices" answer. Distinguish a genuinely empty array (honored as []) from
an array whose items yielded no usable names (unrecognized shape -> null, so
the next endpoint or defaults apply).
2026-06-09 09:17:10 -06:00
Richard R
a2cdc575ef fix(highlight): improve URL replacement spread and sentence alignment fallback
Distribute replacement characters across original URL spans to preserve positional mapping in highlights. In sentence alignment, add fallback to avoid misaligned word highlights when the sentence cannot be located, ensuring robustness in TTS word highlighting.
2026-06-09 09:01:48 -06:00
Richard R
ce811f7636 feat(tts): add mp3 normalization for wav/ogg responses and enhance voice endpoint compatibility
Introduce `normalizeToMp3` utility to convert wav/ogg audio buffers from TTS providers
to mp3, ensuring consistent mp3-only handling in cache and pipelines. Add logic to
detect and adapt to OpenAI-compatible servers that do not support explicit
`response_format`, falling back to default output and normalizing as needed.

Refactor voice resolution to probe multiple common endpoints and robustly extract
voice/style names from various payload shapes, improving compatibility with custom
OpenAI-compatible servers.

- Add `audio-format.ts` for audio normalization utilities
- Add tests for audio format normalization
- Update TTS provider request logic to apply normalization and handle response format negotiation
- Enhance voice endpoint probing and extraction logic for custom servers
2026-06-09 08:52:41 -06:00
Richard R
ff5a6b834d fix(epub): unify TTS word highlight mapping and normalization
Refactor EPUB and HTML word highlighting to use a shared, position-preserving
text normalization pipeline for mapping TTS alignment offsets to DOM positions.
Replace the previous fuzzy token-matching logic with a direct char→DOM map
based on the canonical "audio" text form, ensuring highlights stay in sync
across languages and with Unicode-aware hyphenation handling.

- Add `highlight-char-map.ts` for shared normalization logic
- Update normalization to handle Unicode hyphenation consistently across
  compute, client, and shared layers
- Remove unused fuzzy token mapping and cache logic from EPUB highlighting
- Update tests for new highlight mapping and normalization behavior

BREAKING CHANGE: Removes legacy fuzzy token-based word highlight mapping in favor of direct char offset mapping; highlight cache and tokenization APIs are removed.
2026-06-09 08:20:53 -06:00
Richard R
7d99011cf4
Merge pull request #109 from richardr1126/fix/epub-canonical
fix(epub): canonical per-chapter TTS segmentation (deterministic page turns, highlighting, resize)
2026-06-08 19:07:40 -06:00
Richard R
4714227913 fix(epub): improve canonical window stability and error handling
Enhance EPUB TTS canonical windowing by guarding against stale async completions
during rapid page turns, preventing outdated segment/highlight state from
overwriting the active page. Update SegmentsSidebar to robustly handle errors
in canonicalization, ensuring sidebar state is cleared on failure and avoiding
residual mappings from previous pages. Refine canonical window documentation to
clarify non-overlapping segment partitioning per page. Add a language-aware
cache test to verify correct canonical plan isolation across languages.
2026-06-08 19:05:13 -06:00
Richard R
cb9ca37500 refactor(epub): unify TTS segment planning with canonical windowing
Consolidate EPUB TTS segment planning around a single canonical windowing model.
Introduce `epub-canonical-window.ts` to provide stable, viewport-independent
segment windows for both playback and sidebar. Refactor segment planning,
prefetching, and playback handoff to use canonical ordinals and keys, ensuring
that segments straddling page breaks are uniquely identified and never replayed
across pages. Update TTSContext, SegmentsSidebar, and related hooks to support
windowed canonical segments and precise ordinal continuity. Extend types and
interfaces to carry canonical segment metadata and windowing context.

Add comprehensive unit tests for canonical window selection, segment
materialization, and cache behavior.

BREAKING CHANGE: EPUB TTS segment planning, playback, and sidebar now rely on
canonical windowing, affecting segment key generation and playback logic.
2026-06-08 18:35:18 -06:00
Richard R
293be5bd28 fix(worker): include .npmrc in compute worker Docker build context 2026-06-07 14:34:34 -06:00
github-actions[bot]
2f5179f10b docs: snapshot v4.2.0 2026-06-07 20:10:43 +00:00
Richard R
d7a1025977 chore(release): bump version to v4.2.0
Some checks failed
Create and publish Docker images / prepare (push) Has been cancelled
Version Docs on Tag / version-docs (push) Has been cancelled
Create and publish Docker images / build (arm64, ., ./Dockerfile, web, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / build (arm64, ., ./compute/worker/Dockerfile, compute-worker, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / merge (compute-worker, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / merge (web, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (amd64, ., ./Dockerfile, web, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (amd64, ., ./compute/worker/Dockerfile, compute-worker, linux/amd64, ubuntu-24.04) (push) Has been cancelled
2026-06-07 14:08:41 -06:00
Richard R
0f6f9cc407 docs(security): update secret key generation instructions to use base64 encoding
Replace all references to `openssl rand -hex 32` with `openssl rand -base64 32`
in environment variable examples and documentation. This change standardizes
the recommended method for generating strong random secrets, improving
compatibility and clarity for users configuring authentication and cron secrets.
2026-06-07 14:07:45 -06:00
Richard R
e670c38523
Refactor user data cleanup and TTS storage flows (#105)
* refactor(user): overhaul user data cleanup and export for cascading deletes and TTS segment support

Revise user data cleanup logic to ensure proper cascading deletion of user-related
database rows and S3 objects, including shared document and preview artifacts.
Introduce explicit checks for last ownership before removing shared resources.
Add TTS segment cache and audio cleanup to document and user deletion flows.
Expand user data export to include TTS segment entries and audio, job events,
document settings, and linked auth sessions. Update schema with ON DELETE CASCADE
for userTtsChars and userJobEvents. Add new migration scripts and comprehensive
unit tests for cleanup and export scenarios.

* test(user): expand cleanup and export coverage for shared docs and TTS segment files

Add tests for user data cleanup with shared document ownership and for TTS segment
variant export scenarios, including cases with duplicate audio keys and storage
disabled. Refactor cleanup logic to use document row types and transactional
ownership removal with last-owner checks. Update TTS segment cache clearing to
report deleted segment count by unique entry. Adjust export logic to always
include all TTS segment variants and conditionally export file buckets based on
storage availability. Update migration to use NOT VALID/VALIDATE for new
cascading constraints.

* refactor(documents): centralize owned document deletion and add mutation locking

Introduce `deleteOwnedDocument` utility to encapsulate all logic for removing a user's ownership of a document, including TTS segment cache cleanup, preview artifact removal, and S3 blob deletion for last-owner cases. Add `withDocumentMutationLock` to serialize concurrent mutations on the same document, using advisory locks in Postgres and a local queue fallback. Refactor all API routes and user data flows to use these utilities, ensuring transactional safety and preventing race conditions during document deletion or transfer. Update tests for new flows and add coverage for document cleanup sequencing and locking behavior.

* feat(data): enhance anonymous claim to support document settings and TTS segment copy

Expand the anonymous data claim process to include document settings transfer and TTS segment S3 prefix copying. Remove foreign key constraints from user_tts_chars to allow non-user buckets. Update claim modal and onboarding flow to display claimed document settings. Refactor TTS char count transfer to merge all dates and fix upsert logic. Add S3 copy utility for TTS segments and corresponding tests. Update migrations and schema to reflect relaxed constraints.

* refactor(data): improve error handling and rollback for document and TTS segment operations

Enhance robustness of document deletion and TTS segment transfer by improving
error logging, partial rollback, and degraded state reporting. Add best-effort
cleanup and recovery mechanisms to prevent orphaned data and ensure diagnostic
information is captured for unexpected failures. Update user data cleanup to
log and swallow restoration errors without masking primary failures. Refine
claim logic to handle unmapped TTS audio keys safely.

* refactor(data): unify document mutation locking and transaction handling

Replace mutation-lock with document-lock to centralize document mutation
serialization and database transaction management. Introduce runInDbTransaction
utility to abstract SQLite/Postgres transaction differences. Update document
deletion, user data cleanup, and rate limiter logic to delegate transaction
handling and locking to shared helpers. Remove dialect-specific branching and
inline transaction logic for improved maintainability and testability.

BREAKING CHANGE: withDocumentMutationLock is removed in favor of withDocumentLock and runInDbTransaction

* fix(documents): improve rollback error handling in deleteOwnedDocument

Enhance error handling during document deletion by ensuring that failures
in restoring document ownership do not obscure the original error. Log
degraded events when ownership restoration fails after a deletion error,
providing additional context for debugging and monitoring. This change
improves reliability and traceability of document deletion operations.

* feat(tasks): introduce scheduled task engine and admin UI for background jobs

Add a general-purpose scheduled task system with a persistent registry and status tracking, supporting background maintenance jobs such as orphaned blob reaping, expired upload cleanup, job event pruning, and TTS usage retention. Implement a new `scheduled_tasks` table, task engine, and handlers for each maintenance operation. Integrate an admin UI panel for monitoring, manual runs, and configuration of tasks. Update document and user data cleanup flows to delegate shared blob and preview deletion to the scheduled reaper. Add Vercel cron integration for serverless environments.

BREAKING CHANGE: Document and user storage cleanup now relies on background scheduled tasks for shared blob and preview deletion; immediate inline deletion is no longer performed.

* refactor(user): streamline document and TTS segment transfer logic

Simplify user document transfer by consolidating storage and metadata handling. Remove inline deletion of shared blobs; only metadata is moved, and TTS segment transfer is controlled via options. Add `skipStorage` option for test scenarios to bypass storage operations. Update tests to reflect new transfer behavior.

* chore(instrumentation): delegate node-specific setup to separate module

Move Node.js-specific instrumentation logic to a dedicated file. Update
registration to dynamically import the node module only when running in a
Node.js environment. This separation clarifies environment-specific behavior
and improves maintainability.

* ci(config): update scheduled task cron to run daily at midnight

Change cron schedule for /api/admin/tasks/tick from hourly to once daily at
midnight to reduce task frequency and align with updated operational
requirements.

* refactor(tasks): enforce positive interval and improve scheduled task updates

Add database-level check constraints to ensure scheduled task intervals are always
positive for both Postgres and SQLite. Update admin API and UI to support sub-minute
intervals and stricter validation. Refactor scheduled task update logic to upsert
rows, ensuring tasks can be updated even if not yet present in the database.
Improve temporary upload cleanup to delete in paginated batches. Enhance tests to
cover new constraints and update behaviors.

* chore(docker): remove ffmpeg-static from runtime dependencies in image build

Eliminate ffmpeg-static from the production node_modules during Docker image
assembly to streamline the deployment artifact and avoid bundling unused binaries.

* refactor(admin): redesign task panel UI with Card layout and running indicator

Replace the bordered div layout in AdminTasksPanel with the Card component for
improved visual hierarchy. Add a dynamic running indicator using a custom
RunningDot component to clearly show active tasks. Update control alignment and
button states for better usability. Remove Badge-based status display in favor
of a more streamlined appearance.

* feat(tasks): add document blob lease for safe orphan reaping and improve scheduled task robustness

Introduce a document blob lease mechanism to prevent race conditions between document registration and orphaned blob cleanup. The new `document_blob_leases` table ensures that only one process can claim a document blob for mutation or deletion at a time. Update the orphan reaper to acquire a lease before deleting blobs and to re-check for ownership after acquiring the lease, avoiding accidental deletion of in-flight uploads.

Enhance scheduled task infrastructure with per-task fencing tokens to prevent stale runners from overwriting newer results, enforce runtime limits with abort signals, and expose scheduler mode and minimum interval to the admin panel and API. Adjust task handlers to accept a context with abort support, and update documentation and environment variable references for the new cron secret and scheduling behavior.

BREAKING CHANGE: Scheduled tasks now require a `document_blob_leases` table and updated handler signatures. Vercel deployments must set `CRON_SECRET` for scheduled maintenance.

* refactor(admin): add loading skeleton to tasks panel for improved UX

Introduce a TasksSkeleton component to display animated placeholders while scheduled tasks are loading. Replace direct rendering of empty task rows with the skeleton when data is pending, enhancing perceived responsiveness and user experience in the admin tasks panel.

* test: clean up playwright anonymous users

* test: make blob lease expiry deterministic

* fix(documents): improve blob lease release error handling and test stale lease scenario

Handle errors during blob lease release by logging warnings instead of allowing them to mask original results. Update unit tests to verify that releasing a stale lease does not affect a replacement lease.

* refactor(documents): implement exponential backoff with jitter for blob lease retries

Replace fixed retry delay with capped exponential backoff and jitter to reduce
contention and thundering herd effect when acquiring document blob leases. This
improves lease acquisition fairness and efficiency under high load.
2026-06-07 13:33:20 -06:00
Richard R
5185974eb3
Merge pull request #103 from richardr1126/fix/provider-resolution
fix(tts): make user/shared provider selection reliable and default to inheriting the admin provider
2026-06-06 16:45:47 -06:00
Richard R
92df3f907c fix(tts): correct language input handling for Replicate Kokoro model
Adjust language input resolution in TTS generation to ensure Replicate Kokoro
model receives the appropriate language code mapping even when the language
property is absent or falsy. Update tests to cover this scenario and verify
correct input construction.
2026-06-06 16:18:40 -06:00
Richard R
9c685d7590 docs(env): remove runtime-only TTS and compute rate limit settings from env var docs
Eliminate documentation of TTS and compute PDF parsing rate limiting options from the environment variables reference, as these are managed exclusively via runtime configuration in the admin UI and do not correspond to environment variables. This clarifies the distinction between environment-based and runtime-managed settings.
2026-06-06 16:02:42 -06:00
Richard R
a224efd9e3 refactor(admin): support keyless provider seeding and clarify API_KEY usage
Update provider seeding logic and documentation to allow creation of admin/shared providers without requiring an API key. Adjust environment variable handling so that a blank or missing API_KEY is valid when API_BASE is set, enabling support for upstream TTS providers that do not require authentication. Remove defaulting to 'none' for API keys in API routes and ensure headers are omitted when no key is present. Add and update tests to verify correct handling of keyless providers.

BREAKING CHANGE: Providers can now be seeded without an API key; API_KEY is no longer required if API_BASE is set. Existing deployments relying on a non-empty API_KEY for seeding should review their environment configuration.
2026-06-06 16:02:36 -06:00
Richard R
23b9417567 fix(config): prevent arrays from being treated as records in preference normalization
Update preference normalization logic to explicitly reject arrays when
validating record types, ensuring only plain objects are accepted. Adjust
ConfigProvider effect conditions for clarity and correctness. Add unit tests
verifying that arrays are not accepted as valid preference records or saved
voices.
2026-06-06 15:55:54 -06:00
Richard R
f8182c97df feat(tts): add Replicate Kokoro language code resolution and input mapping
Implement logic to resolve and inject Replicate Kokoro model-specific language
codes based on language tag and voice, using new mapping utilities. Update
`buildReplicateInput` to set `language_code` for the built-in Kokoro model.
Expand language utilities with code mapping and expose `resolveReplicateKokoroLanguageCode`.
Enhance tests to cover language code resolution and input mapping for Replicate
Kokoro, and verify language input key discovery for built-in models.
2026-06-06 15:51:02 -06:00
Richard R
925c995274 refactor(config): centralize provider default resolution and simplify user preference inheritance
Move provider normalization logic into a dedicated module to ensure consistent
handling of provider defaults and user preference inheritance across client and
server. Update layout components to mount ConfigProvider only at the shared
layout level, preventing unnecessary remounts and hydration issues during
navigation. Adjust app config defaults so user provider settings are empty by
default, always inheriting the admin-configured provider unless explicitly set.
Add tests for normalization and sync logic to ensure robust provider resolution.
2026-06-06 15:25:10 -06:00
Richard R
3d1ef1fd41
Merge pull request #102 from feat/multilingual-tier3-support
Add multilingual reader and TTS support

- Detect, normalize, persist, and expose document language across EPUB, PDF, HTML, TTS segment, and audiobook workflows.
- Forward supported language inputs to TTS providers, warn on voice/document language mismatches, and prevent incompatible Kokoro voice selections.
- Centralize Unicode-aware token alignment so EPUB, HTML, and PDF highlighting handles multilingual text, including Japanese matching.
- Standardize shared select, checkbox, button, input, menu, and popover primitives across reader, settings, admin, authentication, and public UI surfaces.
- Update README, introduction docs, and landing-page copy to describe multilingual narration and highlighting.
2026-06-06 13:58:11 -06:00
Richard R
f7c93c90a6 fix(player): prevent selection reset during voice refetch by deriving voices from props
Remove internal selectedVoices state and use a memoized value based on the voice
prop to determine the selected voices. Avoid side effects on selection display,
ensuring that voice selection persists even when available voices are refetched.
This addresses issues where transient backend states could cause unwanted resets
to the first voice option.
2026-06-06 13:07:51 -06:00
Richard R
379e5082f0 docs(intro): clarify open-source status and multilingual support in introduction
Update introduction documentation and README to consistently refer to
OpenReader as "open-source" and highlight its multilingual capabilities.
Refine language segmentation example in multilingual sample text and
expand test coverage for language settings. Adjust TTS segment manifest
to normalize language tags, improve document language update logic, and
fix provider/model compatibility checks for instructions and language
input. Enhance reliability of language support detection for TTS
generation and Replicate provider integration.
2026-06-06 12:19:28 -06:00
Richard R
4b226858b1 perf(highlight): optimize exact token sequence matching for large documents
Add a fast linear scan to resolve exact token sequence highlights before
falling back to the fuzzy window search, significantly improving
performance for large documents. Update unit tests to verify that
exact matches are found efficiently without invoking expensive
comparisons.
2026-06-06 11:41:47 -06:00
Richard R
fed782eb0b docs(intro): update documentation for multilingual support
Expand README and introduction to highlight multilingual features,
including language-aware narration and highlighting. Update landing
page to reflect multilingual capabilities in descriptions, keywords,
and UI labels.
2026-06-06 11:32:41 -06:00
Richard R
922bc32462 refactor(ui): enforce shared control primitives 2026-06-06 11:15:48 -06:00
Richard R
8071ae1a61 refactor(ui): centralize standard select controls 2026-06-06 11:05:36 -06:00
Richard R
112dbc0da9 fix(voices): prevent mixed-language Kokoro selections 2026-06-06 10:50:05 -06:00
Richard R
81bd355bd1 refactor(highlight): share multilingual token alignment 2026-06-06 10:45:19 -06:00
Richard R
3f15636e0e fix(highlight): support Japanese token matching 2026-06-06 10:37:55 -06:00
Richard R
facbda2477 feat(tts): forward language to capable providers 2026-06-06 09:38:14 -06:00
Richard R
696d3e488f feat(tts): detect EPUB language and warn on voice mismatch 2026-06-06 09:29:39 -06:00
Richard R
07e27430c2 feat(tts): add multilingual reader foundation 2026-06-06 06:18:50 -06:00
Richard R
328dbbbb62 ci(playwright): simplify workflow by removing test matrix for browsers
Eliminate the Playwright test matrix to run all browser projects in a single job.
Consolidate report artifact naming and streamline workflow configuration for
faster and more maintainable CI execution.
2026-06-05 10:40:45 -06:00
Richard R
0e29450b0d refactor(pdf): centralize pdf.js runtime configuration and asset resolution
Move pdf.js worker and standard font resolution logic into dedicated
utility functions for improved maintainability and clarity. Replace
inline workerSrc setup with a reusable configurePdfjsNodeRuntime helper.
Update Next.js config to use a shared asset trace list for pdf.js runtime
files, reducing duplication and simplifying future updates.
2026-06-05 10:40:38 -06:00
Richard R
2f99b3987e ci: update Playwright workflow to improve test execution and reporting 2026-06-05 10:20:49 -06:00
github-actions[bot]
df7e0a5737 docs: snapshot v4.1.2 2026-06-05 04:08:03 +00:00
Richard R
f89adb6477 chore(release): bump version to v4.1.2
Some checks failed
Create and publish Docker images / prepare (push) Has been cancelled
Version Docs on Tag / version-docs (push) Has been cancelled
Create and publish Docker images / build (amd64, ., ./Dockerfile, web, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (amd64, ., ./compute/worker/Dockerfile, compute-worker, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (arm64, ., ./Dockerfile, web, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / build (arm64, ., ./compute/worker/Dockerfile, compute-worker, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / merge (compute-worker, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / merge (web, ubuntu-24.04) (push) Has been cancelled
2026-06-04 22:06:05 -06:00
Richard R
b594824a8f chore(config): exclude pdfjs-dist from bundle and update asset tracing for standard fonts
Add 'pdfjs-dist' to serverExternalPackages in Next.js config to prevent bundling,
ensuring runtime resolution of on-disk assets like standard_fonts. Update
tracing includes for API preview route to explicitly cover worker and font
resources loaded dynamically.
2026-06-04 21:54:32 -06:00
Richard R
2898e7e5cc
Merge pull request #101 from richardr1126/clean/job-worker
Simplify PDF parsing to worker-owned SSE and drop stale DB parse state
2026-06-04 20:51:28 -06:00
Richard R
46c4be614d refactor(pdf-parse): extract encodeParserVersion utility and strengthen artifact readiness
Move encodeParserVersion to a dedicated module for unified access and remove local duplicates. Update all consumers to import from the new entrypoint. Tighten artifact readiness checks in both client and API route to ensure 'ready' is only reported when the artifact is accessible, with retries and stricter validation. Add tests for edge cases and operation state validation.
2026-06-04 20:46:02 -06:00
Richard R
c05a60a228 chore(pdf-parse): update PDF_PARSER_VERSION import to use api-contracts entrypoint
Standardize import of PDF_PARSER_VERSION across server modules to reference
@openreader/compute-core/api-contracts. This clarifies versioning boundaries
and improves maintainability by consolidating contract exports.
2026-06-04 20:30:56 -06:00
Richard R
25342371a4 refactor(db): remove legacy PDF parse fields from documents schema
Eliminate parseState and parsedJsonKey columns from the documents table in both
Postgres and SQLite schemas, including related migration scripts and type
removal in document registration logic. This aligns the schema with the new
worker-based PDF parse model and reduces legacy field clutter.
2026-06-04 20:17:23 -06:00
Richard R
9db30742f8 test(pdf-parse): add unit tests for client lifecycle and worker route flows
Introduce new unit tests covering the PDF parse client lifecycle and worker-based
API routes. Tests verify client behavior for not-ready and ready parse states,
operation initiation, SSE event handling, and route validation. Also add
coverage for worker event proxy and worker flow routes. Remove all legacy
parseStatus and parsedJsonKey fields from document types, API, and gallery
view components to align with the new worker-owned PDF parse model.
2026-06-04 20:04:30 -06:00
Richard R
7a86ce6a94 refactor(pdf-parse): migrate PDF parse state to worker-owned model
This change removes legacy server-side PDF parse state management and transitions
to a fully worker-owned model for PDF parsing operations. Key updates include:

- Deletes all code related to server-managed parse state, including:
  - parse-state.ts, parse-state-backfill.ts, parse-state-healing.ts,
    parsed-pdf-reuse.ts, pdf-parse-operation.ts, and related job logic
- Removes the user-pdf-layout-job queue and associated job logic
- Refactors API routes for parsed PDF documents and events to use the new
  worker-owned PDF parse operation flow under src/lib/server/pdf-parse/
- Updates S3 parsed PDF artifact keying to include parser version for
  deduplication and compatibility
- Refactors client API and hooks to handle new error and progress reporting
- Removes all tests for the legacy parse state and job system, updating remaining
  tests to mock the new worker-owned flow

BREAKING CHANGE: Server no longer manages per-user PDF parse state or jobs.
All PDF parsing is now managed by the compute worker and new artifact keying.
Legacy parse state and jobs are no longer supported. Existing parsed PDFs may
need to be reprocessed for compatibility with the new model.
2026-06-04 20:00:21 -06:00
Richard R
833f38059c
Merge pull request #100 from richardr1126/fix/document-dnd
fix(dnd): unify document drag-and-drop on a single touch backend and fix mobile DnD
2026-06-04 18:26:28 -06:00
Richard R
9da21a8004 test(folders): use escaped regex for file name matching in drag-and-drop tests
Update folder tests to use an exported escapeRegExp helper when constructing
regex patterns for file name matching. This prevents false positives or errors
when file names contain special regex characters, improving test reliability.
2026-06-04 18:21:40 -06:00
Richard R
e2c0240add
Merge pull request #99 from richardr1126/pdf/parse-fixes
## Summary by CodeRabbit

* **New Features**
  * Expose PDF parser version system-wide; parser-version-aware parsing, reuse, and caching. Added tokenized temp upload flow, presign/finalize endpoints, and server-side DOCX→PDF conversion.
  * Playback readiness propagated to TTS players/pages; unified upload/delete APIs and simplified uploader/sidebar UX.

* **Bug Fixes**
  * Normalize stored parse state to current parser version to avoid stale readiness and improve reuse.
  * Prevent premature playback starts and redundant preparation.

* **Tests**
  * Added unit and E2E tests for parser-version handling, op-key inclusion, parse-state behavior, and canonical upload reuse.
2026-06-04 18:11:31 -06:00
Richard R
2e74e79e2c refactor(dnd): unify drag-and-drop backend and add custom drag layer for documents
Replace dual HTML5/Touch backend logic with a single TouchBackend configured to
handle both mouse and touch events, ensuring consistent drag-and-drop behavior
across all devices. Introduce a custom DocumentDragLayer to render drag previews
universally, as the touch backend does not provide a native drag image.

Update document tile, gallery, and list views to suppress native long-press
previews on iOS and prevent accidental document opening after drag actions.
Revise drag-and-drop test helpers to simulate pointer gestures compatible with
the new backend, improving reliability of automated tests.
2026-06-04 18:09:36 -06:00
Richard R
c59052a03e fix(documents): allow all accepted files for upload and add docx conversion timeout
Update DocumentUploader to upload all accepted files without filtering by type,
enabling consistent handling of DOCX and other supported formats. Add a default
timeout for DOCX-to-PDF conversion to prevent hanging LibreOffice processes.
This improves reliability and ensures uploads do not silently stall.
2026-06-04 13:20:47 -06:00
Richard R
2ecd281f75 refactor(upload): unify docx-to-pdf conversion into finalize handler and remove legacy route
Remove the separate docx-to-pdf upload API and integrate DOCX conversion directly into the blob upload finalize route. Update client and uploader logic to treat DOCX as a supported upload type and streamline batch state handling. Adjust document caching and test helpers to align with the new unified flow. Add server-side DOCX-to-PDF conversion utility and corresponding unit tests.

This change simplifies the upload pipeline, consolidates conversion logic, and improves maintainability.
2026-06-04 13:00:05 -06:00
Richard R
7344fcbc51 fix(pdf): prevent redundant parse jobs and improve docx-to-pdf upload reuse
Avoid enqueuing duplicate parse jobs for PDFs that are already parsed or in terminal states. Update docx-to-pdf upload to reuse existing parsed PDF results when possible, skipping unnecessary parse operations and job enqueuing. Adjust parse-state-backfill to only enqueue jobs for pending or running states. Refine client upload logic to better infer document type from MIME when file names are missing. Enhance usePdfDocument to handle terminal parse states more robustly.

This change ensures idempotent parse job creation and reduces unnecessary worker load, while improving document upload and parse state handling.
2026-06-04 12:47:37 -06:00
Richard R
62c913e96d feat(pdf): refactor parse operation orchestration and client readiness flow
Introduce startPdfParseOperation to encapsulate worker operation creation, job rate tracking, and parse state initialization for PDFs. Replace scattered operation setup logic in register-upload, docx-to-pdf upload, and parse-state-backfill with this unified helper. Update job enqueuing to propagate initial opId/jobId/status for accurate state tracking.

Change getParsedPdfDocument to throw on non-ready states and remove polling logic from usePdfDocument, shifting readiness detection to event streams. Switch API responses for non-ready parses from 202 to 409 to clarify client expectations and simplify error handling.

Add targeted tests to verify legacy backfill behavior and ensure no worker operation is created for pure data fetches of pending legacy PDFs.

BREAKING CHANGE: API now returns 409 Conflict for non-ready parsed PDFs instead of 202, and clients must subscribe to event streams for parse progress.
2026-06-04 10:52:49 -06:00
Richard R
d489356b00 refactor(documents): unify document upload and deletion logic
Replace type-specific document add/remove functions with generalized uploadDocuments and deleteDocument methods. Consolidate file upload handling in DocumentUploader for improved maintainability. Introduce evictCachedDocument to streamline cache eviction across document types. Update DocumentContext to support batch uploads and single-method deletion, simplifying context API and reducing duplication.
2026-06-03 23:31:18 -06:00
Richard R
cc868511d9 fix(documents): handle precondition failures and optimize upload finalization
Add detection for precondition failures during blob copy to prevent race conditions when finalizing uploads. Refactor upload finalization to use Promise.all for concurrent processing. Extract parse state and parsed JSON key logic to variables for clarity and to avoid duplication in document registration.
2026-06-03 20:50:44 -06:00
Richard R
6c86720926 feat(upload): implement presigned token-based upload with finalize step
Transition document upload flow to use presigned token-based temporary uploads. Add finalize endpoint to register uploaded documents after blob transfer, decoupling document ID assignment from upload initiation. Update client and server logic to operate on tokens, improve security, and enable canonical ID reuse for identical uploads. Update tests to verify deduplication behavior.
2026-06-03 20:14:18 -06:00
Richard R
58ad6212c9 fix(html): reset TTS text and playback state on document changes
Ensure TTS text and playback readiness state are properly reset when
the current document data or name changes, or when clearing or setting
the current document. Update effect dependencies and logic to prevent
stale playback state and improve user experience when switching or
removing documents.
2026-06-03 19:32:46 -06:00
Richard R
2e4f36f5c5 feat(pdf): introduce parser versioning and playback readiness state
Add PDF_PARSER_VERSION constant and propagate parser versioning throughout
the PDF parsing, job, and API layers. Implement normalization of parse state
to ensure compatibility with the current parser version, and enable reuse of
parsed PDF results when possible. Add isPlaybackReady state to document hooks
and TTS player, improving playback UX by disabling controls until content is
ready. Update tests to reflect new playback readiness logic.

BREAKING CHANGE: PDF parse state and job logic now require explicit parserVersion;
older parse states may be treated as pending until reprocessed.
2026-06-03 18:51:29 -06:00
Richard R
17c69f074f docs(docker): document API_KEY usage in quick start guide
Add explanation and example for the API_KEY environment variable in the
Docker quick start instructions. Clarify its role in seeding the default
provider's key and post-boot key management.
2026-06-03 15:14:29 -06:00
github-actions[bot]
36671f77bb docs: snapshot v4.1.1 2026-06-03 20:54:07 +00:00
Richard R
224b3ef107 chore(release): bump version to v4.1.1
Some checks failed
Create and publish Docker images / prepare (push) Has been cancelled
Version Docs on Tag / version-docs (push) Has been cancelled
Create and publish Docker images / build (amd64, ., ./Dockerfile, web, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (amd64, ., ./compute/worker/Dockerfile, compute-worker, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (arm64, ., ./Dockerfile, web, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / build (arm64, ., ./compute/worker/Dockerfile, compute-worker, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / merge (compute-worker, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / merge (web, ubuntu-24.04) (push) Has been cancelled
2026-06-03 14:51:20 -06:00
Richard R
e0cd9bc270 feat(user): remove legacy local file deletion and add cleanup for claimed FS sources
Eliminate the unused --delete-local option and related code from the
filesystem migration script to prevent accidental local data removal.
Introduce a cleanupClaimedLegacyFsSources utility for removing legacy
filesystem sources after claim, and integrate it into the user claim
flow. Add unit tests to verify the cleanup logic.
2026-06-03 14:35:27 -06:00
Richard R
985c7aaeba test(api): add regression tests and backfill handler for legacy PDF parse state
Introduce a backfill handler to restore missing PDF parse operations for
documents in legacy 'pending' or 'running' states lacking an opId.
Update parsed route handlers to invoke this logic, ensuring correct
operation association for affected documents. Add regression tests to
validate backfill behavior for legacy scenarios.
2026-06-03 14:27:37 -06:00
Richard R
bb61d0f502 refactor(pdf): simplify standard font path resolution for pdfjs-dist
Replace dynamic require.resolve lookup with direct path construction to
standard_fonts directory under node_modules. Removes use of createRequire
and streamlines runtime font path resolution logic for PDF parsing.
2026-06-03 14:17:43 -06:00
Richard R
1074a60462 feat(build): improve Docker image layering and embed migration/worker tools
Refactor Dockerfile to optimize layering, embed entrypoint migration tools and compute worker as deployable bundles, and merge only required dependencies for runtime scripts. Add docker/entrypoint-migration-tools package and update workspace configuration. Refactor PDF parsing to resolve standard font data path robustly at runtime. Update dependencies and output config for Next.js standalone mode. Enhance entrypoint script to launch embedded compute worker from deployed bundle.

- Dockerfile now deploys migration tools and compute worker as separate bundles
- compute/core/pdf uses new pdfjs-runtime utility for robust font path resolution
- docker/entrypoint-migration-tools added for migration script dependencies
- scripts/openreader-entrypoint.mjs launches embedded worker from deployed bundle
- next.config.ts enables standalone output and includes standard_fonts in tracing
- Dependency updates across package.json and compute/worker/package.json

BREAKING CHANGE: Docker image structure and runtime entrypoint logic have changed; custom deployment scripts or Docker overrides may require updates.
2026-06-03 14:08:33 -06:00
github-actions[bot]
4317ef03f5 docs: snapshot v4.1.0 2026-06-03 17:12:28 +00:00
Richard R
2ccc64166f chore(release): bump version to v4.1.0
Some checks failed
Create and publish Docker images / prepare (push) Has been cancelled
Version Docs on Tag / version-docs (push) Has been cancelled
Create and publish Docker images / build (amd64, ., ./Dockerfile, web, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (amd64, ., ./compute/worker/Dockerfile, compute-worker, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (arm64, ., ./Dockerfile, web, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / build (arm64, ., ./compute/worker/Dockerfile, compute-worker, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / merge (compute-worker, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / merge (web, ubuntu-24.04) (push) Has been cancelled
2026-06-03 11:10:01 -06:00
Richard R
f0e0daae77
feat: improve PDF parsing recovery and reader UI (#98)
* refactor(ui): replace anchor tags with next/link in SidebarNavLink and UserMenu

Update SidebarNavLink to use next/link for navigation instead of anchor tags,
ensuring proper routing and improved accessibility in Next.js. Refactor UserMenu
to remove legacy Link wrappers and directly use SidebarNavLink for signin and
signup links. This streamlines navigation components and aligns with Next.js
best practices.

* fix(pdf): handle non-zero viewport origins and improve layout model ordering

Update normalizeTextItemsForLayout to correctly apply viewport transforms,
including non-zero page origins, ensuring accurate mapping of PDF text items
to top-left coordinates. Refactor runLayoutModel to implement a custom order
sequence builder for layout regions using model order logits, improving
region ordering consistency with model semantics. Update related tests to
cover viewport transform edge cases.

* refactor(pdf): improve text normalization with font ascent and vertical overlap logic

Enhance text normalization by incorporating font ascent and descent data to more accurately position glyphs, especially for decorative initials. Update merge logic to better detect line membership using vertical overlap, ensuring drop caps and overlapping glyphs are merged correctly. Extend tests to cover these layout scenarios.

* style(range): redesign slider with precision gauge and ruler ticks

Revamp the range input to feature a minimalist "precision gauge" style.
Introduce a hairline rail, ruler notches for discrete steps, and a slim
needle thumb. Add CSS variables and logic for per-instance tick sizing
and coloring. Remove bulky inline class-based styling in favor of
centralized CSS for improved maintainability and visual clarity.

* refactor(ui): modularize PDF loader and range slider visuals

Move PDF layout scan visualization and range slider styles into dedicated CSS
modules, isolating their styles from the global scope. Integrate PdfLayoutScan
component into the PDF viewer loader UI for animated parse progress. Refactor
progress bars to use a reusable progress-fill class with animated sheen effect.
Update range input to use CSS module for precision gauge styling.

* style(reader): remove grid overlay from PdfLayoutScan visualization

* refactor(api): add staleness detection for inflight worker operation states

Integrate isWorkerOperationStateStale checks into document parse API endpoints to ensure inflight worker operation states are not reused if stale. Introduce helper for staleness detection and corresponding unit tests. Enhance SSE event streaming with keepalive intervals and improve progress acknowledgment error handling.

* feat(worker): recover and fail stale in-flight pdf ops on startup

Add orphaned operation recovery logic to detect and mark stale in-flight pdf_layout jobs as failed during worker startup. Extend OperationStateStore with listOpStates for state enumeration. Update tests and documentation to cover recovery behavior and new environment variable COMPUTE_PDF_JOB_ATTEMPTS.

* refactor(worker): distinguish staleness thresholds for running and queued pdf ops

Update orphan recovery logic to apply separate timeouts for 'running' and
'queued' pdf_layout operations. Adjust tests to verify that only stale
'running' operations are failed, while stale 'queued' operations remain
untouched.

* feat(worker): extend orphan recovery to handle whisper_align ops and improve logging

Update orphan recovery to detect and fail stale 'running' whisper_align operations
in addition to pdf_layout. Refactor recovery logic to generalize staleness checks
across operation kinds and enhance log output with detailed operation info.
Expand tests to verify correct handling of both whisper_align and pdf_layout
operations in running and queued states.

* refactor(control-plane): introduce revision-based CAS for operation state updates

Add revision tracking and compare-and-set (CAS) semantics to operation state
stores, enabling atomic state transitions and preventing lost updates. Extend
the OperationStateStore interface with getOpStateRecord and compareAndSetOpState
methods. Update orchestrator and worker runtime to utilize CAS for marking
operations as failed only if the state is unchanged. Enhance in-memory,
JetStream, and test control plane implementations to support revision logic.

This change improves concurrency safety and correctness of operation state
management across distributed components.

* feat(ui): add parse failure state to PDF layout scan animation

Display a distinct "parse halted" visual state in the PDF layout scan
component and PDF viewer page when parsing fails. The loader animation
is replaced by a static, dimmed page with an alert glyph and updated
styling, ensuring users are not misled by an active animation after a
failure. CSS and component logic updated to support the new state.

* feat(worker): extract orphaned operation recovery to module with periodic sweep

Move orphaned operation recovery logic into a dedicated orphan-recovery module,
introducing a periodic sweep timer that triggers recovery every 15 seconds while
the worker is connected. Refactor runtime to delegate orphan detection and
handling to the new module, improving modularity and maintainability. Add
unit tests for orphan-recovery to ensure correctness.

* fix(ui): adjust PDF viewer layout and update parse loader description

* refactor(pdf): streamline layout model region extraction and update test coverage

- Replace custom order sequence logic with softmax-based class selection in runLayoutModel
- Remove unused sigmoid and buildOrderSequence functions
- Simplify detection loop to filter and map regions directly
- Add targeted tests for layout model extraction logic
- Update CSS animation naming for consistency
- Clarify test description and add inline comments for orphan recovery scenario

* fix(pdf): add strict validation for layout model output shapes and extend test coverage

Add explicit error handling for invalid or inconsistent pred_boxes and logits array lengths in runLayoutModel to prevent silent failures. Expand test suite to verify correct region filtering and error scenarios, ensuring only labeled regions are returned and malformed outputs are handled robustly.
2026-06-03 04:44:55 -06:00
github-actions[bot]
6f89307af4 docs: snapshot v4.0.3 2026-06-03 03:08:17 +00:00
Richard R
88ed58a6a8 chore(release): bump version to v4.0.3
Some checks failed
Create and publish Docker images / prepare (push) Has been cancelled
Version Docs on Tag / version-docs (push) Has been cancelled
Create and publish Docker images / build (arm64, ., ./Dockerfile, web, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / build (arm64, ., ./compute/worker/Dockerfile, compute-worker, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / merge (compute-worker, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / merge (web, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (amd64, ., ./Dockerfile, web, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (amd64, ., ./compute/worker/Dockerfile, compute-worker, linux/amd64, ubuntu-24.04) (push) Has been cancelled
2026-06-02 21:06:32 -06:00
Alex Errant
beea1961bf
Fix container crash on startup: onnxruntime-node requires glibc (switch Alpine → Debian slim) (#97)
* docker fixes

* fix(worker): default COMPUTE_PREWARM_MODELS to false and update docs, env, and compose

Set COMPUTE_PREWARM_MODELS to false by default in runtime, docker-compose,
and .env.example to avoid pre-downloading ONNX models during worker startup.
Update Dockerfile and deployment documentation to reflect this change and
clarify configuration for model prewarming.

* fix(docker): downgrade pnpm version to 10.33.4 for compatibility

---------

Co-authored-by: Richard R <me@richardr.dev>
2026-06-02 20:50:37 -06:00
github-actions[bot]
470898627d docs: snapshot v4.0.2 2026-06-02 02:43:25 +00:00
Richard R
782a0c3270 chore(release): bump version to v4.0.2
Some checks failed
Create and publish Docker images / prepare (push) Has been cancelled
Version Docs on Tag / version-docs (push) Has been cancelled
Create and publish Docker images / build (amd64, ., ./Dockerfile, web, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (amd64, ., ./compute/worker/Dockerfile, compute-worker, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (arm64, ., ./Dockerfile, web, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / build (arm64, ., ./compute/worker/Dockerfile, compute-worker, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / merge (compute-worker, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / merge (web, ubuntu-24.04) (push) Has been cancelled
2026-06-01 20:41:27 -06:00
Richard R
b41d09c94f Merge branch 'main' of https://github.com/richardr1126/OpenReader-WebUI 2026-06-01 20:38:59 -06:00
Richard R
28f8d50c05 refactor(settings): remove enableDestructiveDeleteActions runtime flag and related UI
Eliminate the enableDestructiveDeleteActions feature flag from runtime
configuration, admin panel, environment docs, and user settings modal.
Remove all references, toggles, and documentation for this flag. This
simplifies the runtime config and user interface by consolidating
destructive actions under account deletion only.

BREAKING CHANGE: The enableDestructiveDeleteActions runtime flag is no longer supported. Any configuration or code depending on this flag must be updated.
2026-06-01 20:38:54 -06:00
github-actions[bot]
68bbd7bc94 docs: snapshot v4.0.1 2026-06-02 01:57:31 +00:00
Richard R
e295ee4103 chore: update version to 4.0.1 in package.json
Some checks failed
Version Docs on Tag / version-docs (push) Has been cancelled
Create and publish Docker images / prepare (push) Has been cancelled
Create and publish Docker images / merge (compute-worker, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / merge (web, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (amd64, ., ./Dockerfile, web, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (amd64, ., ./compute/worker/Dockerfile, compute-worker, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (arm64, ., ./Dockerfile, web, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / build (arm64, ., ./compute/worker/Dockerfile, compute-worker, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
2026-06-01 19:53:53 -06:00
Richard R
06206ad5af docs(deploy): update compute worker deployment guide for clarity and conciseness
Revise the compute worker deployment documentation to clarify usage
scenarios, streamline environment variable instructions, and improve
separation between embedded and standalone deployment modes. Remove
outdated references and reformat for better readability.
2026-06-01 19:47:55 -06:00
Richard R
ebe384c698 style(player): add relative positioning to TTS control buttons
Apply "relative" class to IconButton components in TTSPlayer to enable
positioning of child elements or overlays. This prepares the UI for
potential enhancements such as loading indicators or tooltips.
2026-06-01 19:41:48 -06:00
github-actions[bot]
9325f9e467 docs: snapshot v4.0.0 2026-06-02 01:04:40 +00:00
Richard R
1231b37ef3 build(docker): set build-time AUTH_SECRET and BASE_URL env vars for Docker
Some checks failed
Create and publish Docker images / prepare (push) Has been cancelled
Version Docs on Tag / version-docs (push) Has been cancelled
Create and publish Docker images / build (amd64, ., ./Dockerfile, web, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (amd64, ., ./compute/worker/Dockerfile, compute-worker, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (arm64, ., ./Dockerfile, web, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / build (arm64, ., ./compute/worker/Dockerfile, compute-worker, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / merge (compute-worker, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / merge (web, ubuntu-24.04) (push) Has been cancelled
2026-06-01 18:56:55 -06:00
Richard R
f2c7760381
Redesign: shared UI design system + app-wide migration (#96)
* phase 0: token foundation

* phase 1: motion language

* phase 2: primitives and semantic tokens

* phase 3: depth and rhythm polish

* phase 4: enforce design system lint rules

* phase 5: split ui primitives into modules

* phase 7: refactor app surfaces to ui layer

* phase 9: enforce ui architecture imports

* phase 10: add ui system harness

* fix compact reader auth control

* fix pdf loader flash

* Converge sidebar and reader controls

* refactor: remove initial loader state and related logic from PDFViewerPage

* Remove legacy UI shim imports

* Converge modal and drawer frames

* Migrate secondary modals to shared frame

* Move settings modal onto shared frame

* Use shared cards in audiobook settings

* Converge choice and popover surfaces

* Converge reader navigation buttons

* Refactor UI components to use consistent button and icon styles across the application

* refactor(ui): unify button usage in settings, admin, and doclist components

Replace native button elements with shared Button, ChoiceTile, and IconButton
components for consistent UI behavior and styling. Update classNames and
props to match new component APIs. Adjust FinderSidebar to use utility
function for conditional class merging.

* feat(ui): introduce shared Listbox components for unified select and dropdown styling

Replace direct usage of Headless UI Listbox primitives with new SharedListboxButton,
SharedListboxOption, and SharedListboxOptions components across AudiobookExportModal,
FinderToolbar, VoicesControlBase, and select UI. Refactor related imports and classNames
to centralize dropdown styling and logic. Simplify UserMenu button markup for improved
consistency.

This change consolidates dropdown/select UI patterns, reduces duplication, and
improves maintainability by providing a single source of truth for Listbox styling
and behavior.

* refactor(ui): consolidate button, menu, popover, and range primitives for unified usage

Remove legacy UI harness and dev/demo files. Replace scattered button, menu, popover, and range input utilities with shared, composable primitives: Button, ButtonLink, ButtonAnchor, MenuActionItem, MenuItemsSurface, PopoverSurface, PopoverTrigger, and RangeInput. Update all usages across app, admin, player, and document components to use these new primitives, eliminating duplicated class logic and improving consistency. Remove obsolete utility files and class exports. This change streamlines UI code, centralizes styling, and reduces maintenance overhead.

* feat(ui): redesign range input with dynamic progress styling and improved accessibility

Revamp the range input component to support dynamic progress indication using CSS custom properties and linear gradients. Add logic to compute and set the progress percentage based on current value, min, and max. Refine focus and disabled states for better accessibility and usability. Update styling for both WebKit and Mozilla engines to ensure consistent appearance. This change enhances visual feedback and modernizes the range slider UI.

* style(ui): update range input to use secondary accent color for progress

Switch range input progress styling from primary to secondary accent color
for both WebKit and Mozilla engines. Remove drop shadow from slider thumb
for a cleaner appearance. This change aligns the component with the updated
design palette and simplifies visual effects.

* refactor(app): remove unused Link imports from public pages and components

Eliminate redundant imports of the Link component from Next.js in several
public-facing pages and components. These imports were no longer in use
after recent UI refactoring and consolidation of navigation elements.
This cleanup reduces bundle size and improves code clarity.

* chore(ui): remove unused export of segmented control classes from select component

Eliminate unnecessary export statements for segmentedButtonClass and
segmentedGroupClass in the select component to streamline the module's
public API and reduce potential confusion.

* test(accessibility): improve confirm dialog test coverage and refactor media state helper

Expand accessibility tests for ConfirmDialog to assert dialog semantics,
ARIA attributes, and visible destructive actions using test IDs. Refactor
expectMediaState helper to check both UI control state and underlying
media signals for more robust playback state detection.

* refactor(ui): improve segmented control accessibility and update danger color tokens

Update SegmentedControl to support full keyboard navigation and focus management,
enhancing accessibility. Replace string indicator in Select with icon, and update
button danger variant to use new --danger-strong variable for hover states. Add
danger-strong token to Tailwind config and globals. Refine dropzone disabled
behavior, adjust focus ring for better contrast, and apply minor UI consistency
tweaks across components.

* feat(ui): convert SidebarNavLink to forwardRef component

Refactor SidebarNavLink to use React.forwardRef, enabling parent components
to access the underlying anchor element's ref. Update prop typing and
function signature accordingly for improved composability and integration
with higher-order components.
2026-06-01 16:17:12 -06:00
Richard R
3d0367bc5b fix(tests): update heading text in landing routing test 2026-05-31 17:06:04 -06:00
Richard R
e3044278b1
Merge pull request #95 from richardr1126/redesign/landing-privacy
Redesign public landing & privacy pages
2026-05-31 16:56:05 -06:00
Richard R
3c08b9e406 feat(shared): centralize origin validation logic and update metadata usage
Move origin extraction to shared utility for consistent validation across
server and client modules. Update layout metadata to use the shared origin
helper, improving maintainability and deployment flexibility. Add unit tests
for layout metadata to ensure correct origin handling.
2026-05-31 16:41:47 -06:00
Richard R
27267e5ffb refactor(public): use environment variables for base URL and improve accessibility
Switch landing and layout metadata to use BASE_URL from environment for
deployment flexibility. Refine privacy page environment check for robustness.
Update ARIA attributes on landing preview for better accessibility. Adjust
terminal and visually hidden styles for improved formatting and screen reader
support.
2026-05-31 16:29:40 -06:00
Richard R
0b1e293638 style(landing): revise feature copy and docker instructions for clarity
Update landing page descriptions to emphasize native file rendering and
layout-aware parsing. Adjust Docker quick start instructions to remove
redundant volume mount and highlight required environment variables for
self-hosting.
2026-05-31 16:09:31 -06:00
Richard R
535d562a0b style(public): overhaul landing and privacy pages with new layout, hero, and theme effects
Redesign the public landing and privacy pages with a new layout structure,
introducing an aurora background, grain overlay, and updated navigation.
Revise hero section with animated word-level highlighting and refreshed
copy. Update CSS to use semantic theme variables, color-mix, and new
background effects. Adjust navigation, footer, and action button order for
clarity. Update privacy page button label for consistency.
2026-05-31 15:55:41 -06:00
Richard R
14807a0e8c style(landing): update provider and export descriptions on landing page and adjust hero panel glow
Revise landing page metadata to specify Replicate TTS and clarify export as
audiobook workflows. Simplify hero panel glow effect in CSS for visual
consistency.
2026-05-31 14:36:58 -06:00
Richard R
e3818db294 refactor(privacy): restructure policy sections and clarify provider, analytics, and encryption details
Reorganize privacy policy content to improve clarity on service provider roles,
analytics consent, and encryption limitations. Update section headings and
descriptions for accuracy. Enhance CookieConsentBanner and PrivacyModal to
reflect optional analytics and clarify storage, operator access, and password
handling. Improve CSS for policy section headings and lists for better
readability.
2026-05-31 14:28:59 -06:00
Richard R
50234d34a9 refactor(ui): centralize button primitives and update imports across app
Move all button-related class utilities and types from formPrimitives to a new
ui/buttonPrimitives module. Update all component imports to reference the new
location, ensuring consistent button styling and easier future maintenance.
Remove redundant button code from formPrimitives and re-export as needed for
admin UI compatibility. Add public.css for landing and privacy layout styles.
2026-05-31 14:10:59 -06:00
Richard R
da1cb1e2be
Merge pull request #94 from richardr1126/refactor/clean-env
refactor(v4): land JSON runtime seeder and remove no-auth mode
2026-05-31 13:31:46 -06:00
Richard R
2a2ba35454 Apply cached TTS location before remote progress 2026-05-31 13:23:52 -06:00
Richard R
eebb54b018 refactor(auth): remove redundant authEnabled flag from contexts and components
Eliminate the unused authEnabled property from context providers, hooks,
components, and API responses. All logic and UI now assume authentication is
required, simplifying prop signatures and reducing branching. Update related
types, context values, and function calls to reflect this change. This streamlines
the authentication flow and removes unnecessary configuration.
2026-05-31 13:10:01 -06:00
Richard R
936aa50f9a refactor(api): remove all legacy unclaimed user scope logic and enforce strict userId scoping
Eliminate all code, tests, and utilities related to the legacy 'unclaimed' user
scope. All API endpoints, document and audiobook storage, and access logic now
require a valid authenticated userId and only operate on resources owned by that
user. Remove related helper functions, test cases, and conditional flows for
anonymous/unclaimed data. Update types, client APIs, and UI logic to reflect
that only 'user' scope is supported. This simplifies ownership checks and
removes ambiguity around document and audiobook access.
2026-05-31 13:01:55 -06:00
Richard R
20fa820952 clean(tests): remove CI test script from package.json 2026-05-31 12:10:57 -06:00
Richard R
5f28be5d58 refactor(env): remove legacy no-auth mode and enforce required auth env vars
Eliminate all code paths, configuration, and documentation related to running
without authentication. Require AUTH_SECRET and BASE_URL at startup, updating
middleware, server logic, and runtime checks to assume auth is always enabled.
Simplify onboarding, settings, and test helpers to reflect mandatory auth.
Update environment examples, Docker and deployment docs, and CI/test configs.
Remove no-auth-specific UI flows, test cases, and feature toggles.
2026-05-31 12:09:37 -06:00
Richard R
4bcd90274c Handle JSON provider seed encryption failures per provider 2026-05-31 00:55:25 -06:00
Richard R
bfe9567310 Clarify compute worker token docs 2026-05-31 00:21:20 -06:00
Richard R
1763a84ffc docs(env): clarify FFMPEG_BIN usage for audio processing in docs and env example
Update .env.example and environment variable documentation to clarify that
FFMPEG_BIN can override the ffmpeg binary path for general audio processing,
not just audiobooks. Note that it is also used by compute worker Whisper audio
decode in addition to audiobook routes.
2026-05-31 00:16:28 -06:00
Richard R
83aa1152cb docs(config): update environment variable docs for runtime JSON seed and remove legacy RUNTIME_SEED_* usage
Remove all references to legacy RUNTIME_SEED_* environment variables from documentation and codebase. Update docs and .env.example to document the new RUNTIME_SEED_JSON and RUNTIME_SEED_JSON_PATH variables for first-boot runtime config and provider seeding. Refactor admin seed logic and runtime config schema to eliminate env-var-based seeding in favor of JSON-based initialization. Update admin panel UI and badges to reflect new seed sources. Remove obsolete env parsing logic and tests for RUNTIME_SEED_* flags. Add new tests for JSON seed behavior.

BREAKING CHANGE: RUNTIME_SEED_* environment variables are no longer supported; use RUNTIME_SEED_JSON or RUNTIME_SEED_JSON_PATH for first-boot runtime config and provider seeding.
2026-05-31 00:13:03 -06:00
Richard R
f0800df745 refactor(tts): migrate TTS tuning from env vars to runtime config with admin controls
Remove TTS cache and upstream tuning environment variables in favor of admin-managed
runtime settings. Add new admin panel controls for TTS retry attempts, upstream timeout,
audio cache size, and cache TTL. Update API routes and TTS generation logic to consume
these runtime-configurable values, enabling live adjustment without redeploy. Update
documentation to reflect the removal of related env vars and the new admin workflow.
2026-05-30 20:47:55 -06:00
Richard R
f8c2f27e64
Merge pull request #93 from richardr1126/fix/content-views-mobile
fix(doclist): improve gallery + list view layout on small screens
2026-05-30 20:14:06 -06:00
Richard R
1b075a35d3 style(doclist): adjust skeleton grid columns for improved mobile responsiveness
Update grid column widths in DocumentListSkeleton to better fit smaller screens,
enhancing the mobile viewing experience and alignment consistency.
2026-05-30 20:12:27 -06:00
Richard R
b359d66352 refactor(doclist): update GalleryView keyboard handling and icon visuals
- Add Delete/Backspace keyboard shortcut for document deletion in GalleryView
- Use buttonClass utility for consistent button styling
- Simplify GalleryViewIcon SVG for improved clarity and alignment

These changes streamline user interactions and enhance visual consistency in the document gallery view.
2026-05-30 18:06:12 -06:00
Richard R
e6df984646 fix(ui): improve mobile layout and spacing for document list and gallery views
- Adjust GalleryView container structure and paddings for better mobile scrolling
- Refine document preview and details sizing for smaller screens
- Update ListView grid columns and icon sizing for improved mobile fit
- Ensure icons do not shrink and maintain alignment on all devices

These changes enhance usability and visual consistency of document views on mobile devices.
2026-05-30 17:59:16 -06:00
Richard R
3daf4b6453
Merge pull request #92 from richardr1126/refactor/pre-release-cleanup
refactor: pre-release cleanup — rate limiting, worker modularization, vitest migration
2026-05-30 17:32:53 -06:00
Richard R
aa7700e844 refactor(core): improve streaming, validation, and rate limit logic
- Stream request body in blob upload fallback route to enforce size limits
  without buffering entire payload
- Enhance admin provider panel error handling for multi-status responses
- Adjust positive integer validation to require value >= 1
- Use full rate limit config for job event recording and prune by largest window
- Fix PDF layout job event userId usage in docx-to-pdf upload route
- Add missing windows array fallback in documents register route
- Minor CI workflow and env example corrections
- Update audiobooks blobstore test to use beforeAll directly

These changes improve efficiency, correctness, and maintainability across API, admin, rate limiting, and test modules.
2026-05-30 16:41:17 -06:00
Richard R
36fe5f66a3 ci(workflow): add migration step to vitest CI pipeline
include drizzle migration before running Vitest to ensure database schema
is up to date during test execution
2026-05-30 15:45:42 -06:00
Richard R
8060ae9944 fix(config): update vitest reporter for CI to github-actions 2026-05-30 15:40:34 -06:00
Richard R
0448835892 style(ui): improve bottom padding for safe area compatibility
update bottom padding on banners, status bars, and player controls to use
env(safe-area-inset-bottom) for better support on devices with display
cutouts or home indicators. also add viewportFit: cover to layout for
full-viewport rendering on mobile browsers.
2026-05-30 15:35:02 -06:00
Richard R
60c1540744 refactor(ui): decouple settings modal trigger from modal logic
extract SettingsTrigger component to separate modal open control from SettingsModal.
update HomeContent to manage modal state and pass open/onOpenChange props.
remove redundant trigger button from SettingsModal for improved composability.

removes embedded trigger logic from SettingsModal, enabling external state
management and better integration with various UI layouts. also cleans up
playwright config by centralizing testIgnore for unit tests.
2026-05-30 15:13:15 -06:00
Richard R
8b45a4f6cd refactor(admin): group rate limit and upload settings in dedicated section
move TTS and PDF parsing rate limit toggles and max upload size input into a new
"Rate limiting" section in the admin panel for improved organization. update
documentation to reflect the new grouping and clarify the relationship between
these settings.
2026-05-30 14:59:45 -06:00
Richard R
3de33e2f1e docs(config): update compute rate limit defaults to disabled for first boot
change RUNTIME_SEED_DISABLE_COMPUTE_LIMIT default to true in env example,
documentation, and runtime config schema to align with TTS rate limit behavior.
clarify admin panel as the source of truth for enabling PDF parsing rate limits.
2026-05-30 14:39:04 -06:00
Richard R
fc3d05d65b feat(rate-limit): add per-user PDF parsing rate limiting and job event ledger
implement a generic user_job_events table for tracking compute job creation
enforce configurable burst and sustained limits for PDF layout parsing
add admin panel controls for compute rate limiting and max upload size
update API routes to apply and record rate checks for PDF parse jobs
document new environment variables and admin settings for compute limits
improve IP extraction logic for rate limiting accuracy
add tests for request IP extraction and test namespace gating

This change introduces a robust mechanism to throttle expensive compute operations, such as PDF parsing, on a per-user basis. It provides both burst and sustained rate controls, with admin-tunable parameters and clear user feedback on throttling. The job event ledger enables accurate concurrency and rate enforcement, while new documentation and tests ensure maintainability and clarity.
2026-05-30 14:17:49 -06:00
Richard R
3307477548 test(unit): complete vitest migration for remaining unit suites 2026-05-30 11:17:18 -06:00
Richard R
22a1b5de57 test(unit): migrate batch2 document and parser suites to vitest 2026-05-30 11:15:19 -06:00
Richard R
7570181b8a test(unit): migrate batch1 core contracts to vitest 2026-05-30 11:11:44 -06:00
Richard R
b16328abe9 fix(vitest): update test command from 'test:vitest' to 'test:unit' 2026-05-30 11:01:44 -06:00
Richard R
e1483ab80a refactor(worker): modularize worker entrypoint and add vitest infra
move worker startup logic from server.ts to runtime.ts for better modularity
add worker-loop-policy.ts for loop control abstraction
delete legacy unit tests under tests/unit/ related to compute worker and control-plane
add vitest config and test directories for compute-core and compute-worker
add vitest workflow for CI
update package.json scripts for new test commands and add vitest as dev dependency

BREAKING CHANGE: worker entrypoint is now compute/worker/src/runtime.ts instead of server.ts; legacy Playwright-based unit tests for compute worker are removed in favor of Vitest
2026-05-30 11:00:01 -06:00
Richard R
b0a4bccf4c refactor(admin): remove unused EditableRow and EditIcon components
- Delete EditableRow from formPrimitives and its usage in AdminFeaturesPanel
- Remove EditIcon from icon set as it is no longer referenced
- Simplify AdminFeaturesPanel state and draft handling
- Update AdminProvidersPanel to use distinct query key for default provider
- Ensure query invalidation covers new default provider key

This streamlines the admin codebase by eliminating obsolete editing UI and related icon, and clarifies provider query logic.
2026-05-30 10:33:11 -06:00
Richard R
267e2ce411 fix admin features tab reset/flash on tab switches 2026-05-30 10:05:24 -06:00
Richard R
a0f2d008e0 refactor(tts): transition rate limiting to runtime config and admin panel
- Remove legacy TTS rate limit environment variables and migrate all related configuration to runtime settings managed via the Admin UI
- Add runtime config keys for TTS rate limiting enable/disable and per-user/IP daily quotas
- Refactor rate limiter and API routes to use runtime config for thresholds and enablement
- Update AdminFeaturesPanel to allow editing TTS rate limiting and quota values in the UI
- Add shared-provider-selection utility for consistent provider selection logic
- Update documentation to reflect new runtime/admin configuration and remove obsolete env var docs
- Add unit tests for rate limit runtime settings and provider selection

BREAKING CHANGE: TTS rate limiting is now controlled via Admin → Site features; environment variables for TTS rate limiting are no longer supported and will be ignored.
2026-05-30 09:55:16 -06:00
Richard R
e262f93f90
Merge pull request #91 from richardr1126/redesign/app-view-documents-list 2026-05-29 22:28:09 -06:00
Richard R
2695c7055b Fix narrow-screen initial render in FinderWindow 2026-05-29 22:27:30 -06:00
Richard R
d120719abd refactor(doclist): implement upload batch progress tracking in sidebar
Add SidebarUploadLoader component to DocumentList for displaying real-time
upload progress in the sidebar. Extend DocumentUploader to emit detailed
batch state updates via onUploadBatchChange, enabling granular feedback
on current file, phase, and completion status. Improves user awareness of
document upload and conversion processes.
2026-05-29 22:11:27 -06:00
Richard R
b295d2f38d feat(doclist): add mobile sidebar close handler to FinderWindow and DocumentList
Introduce onRequestSidebarClose prop to FinderWindow and implement its
usage in DocumentList to enable closing the sidebar on mobile via
backdrop tap or Esc key. Enhances mobile usability by allowing users to
dismiss the sidebar more intuitively.
2026-05-29 21:58:45 -06:00
Richard R
503319d46e style(ui): adjust sidebar hover effects and document skeleton timeout
Update FinderSidebar to remove background color on inactive hover for
improved visual clarity. Extend the DocumentSkeleton warning timeout to
10 seconds to allow more time before displaying the alert. Refine test
description for icons grid layout to clarify auto-fit behavior.
2026-05-29 21:49:13 -06:00
Richard R
e62923fb91 feat(ui): improve icon grid layout responsiveness and add single row stretch suppression
Enhance the document icons grid to better handle single-row layouts by
dynamically suppressing column stretching when the number of documents fits
in a single row. Introduce maxColumnsForIconGrid and related logic to
calculate optimal column counts based on container width and icon size.
Update grid style computation to allow optional suppression of single row
stretch, improving alignment and appearance for small document sets.

Add unit tests for icon grid layout calculation and column logic to ensure
correctness across different icon sizes and container widths.
2026-05-29 21:33:01 -06:00
Richard R
fe59685454 feat(worker): add granular PDF progress reporting and abort error handling
Introduce per-page progress events for PDF parsing by adding onPageStarted
hooks throughout the compute worker and core PDF pipeline. Add
compute/worker/src/pdf-progress.ts to encapsulate progress event builders.
Update server logic to emit page start and page parsed events, improving
client feedback during long-running PDF jobs.

Add src/lib/server/compute/abort-like-error.ts and integrate isAbortLikeError
to robustly detect and suppress expected aborts in API event streaming.
Update API route to ignore abort-like errors on worker proxy crash.

Add unit tests for abort-like error detection and PDF progress event
generation. Refactor PDF parse logic to allow empty merged regions without
throwing, ensuring downstream flows remain resilient to edge cases.

These changes improve observability, error handling, and reliability for
document parsing and event streaming.
2026-05-29 20:13:59 -06:00
Richard R
dcec74a4a3 refactor(package): remove unused compute worker watch script 2026-05-29 19:58:56 -06:00
Richard R
5428a21b67 refactor(onboarding): modularize onboarding flow logic and improve testability
Extract onboarding flow step resolution and async runner logic into a
dedicated module for better separation of concerns and maintainability.
Refactor context/provider to use new helpers and simplify settings modal
control. Add unit tests for onboarding step resolution and async runner
coalescing. Update e2e helpers and selectors for onboarding modals.
Adjust environment file handling for CI and Docker contexts.

- Add .dockerignore for build hygiene
- Refactor onboarding context and modal logic
- Add src/lib/client/onboarding-flow.ts with core onboarding helpers
- Add onboarding-flow unit tests
- Update Playwright config and scripts for CI env loading
- Improve test selectors for claim modal
- Update .gitignore to exclude all .env* except .env.example

This change increases onboarding logic modularity and reliability, and improves CI/test environment handling.
2026-05-29 18:03:17 -06:00
Richard R
c6047ffaa4 fix(onboarding): prevent no-auth auto settings modal races 2026-05-29 09:20:34 -06:00
Richard R
c5b39f5312 test(e2e): dismiss blocking overlays after folder reload 2026-05-28 23:13:34 -06:00
Richard R
8566206911 test(e2e): disambiguate settings and uploader selectors 2026-05-28 22:46:55 -06:00
Richard R
dca1a9b35d feat(doclist): unify document identity handling and improve drag-and-drop robustness
Introduce a document identity key combining type and id for consistent
identification across document operations, including selection, drag-and-drop,
and folder management. Refactor drag item structures to use identity objects
instead of plain ids, preventing cross-type collisions and improving merge
accuracy. Update Dexie recently opened map to use identity keys. Enhance
document size formatting for small files and improve error feedback in the
uploader. Clean up redundant props and standardize icon SVG handling.
2026-05-28 22:27:15 -06:00
Richard R
2f1eafeb79 test(e2e): scope settings Documents tab selector to modal 2026-05-28 22:26:49 -06:00
Richard R
5a684214c8 fix(doclist): mobile drawer close behavior and recents toolbar polish 2026-05-28 17:45:05 -06:00
Richard R
98736bf6e3 refactor(doclist): remove columns view and update text labeling
Eliminate the deprecated "columns" view mode from the document list, toolbar,
icons, and skeleton components. Update all related types and logic to only
support "icons", "list", and "gallery" views. Standardize user-facing labels
from "HTML" to "Text" for improved clarity across the sidebar, segment
locator labels, and document counts. Clean up associated code and types to
reflect these changes.
2026-05-28 17:20:02 -06:00
Richard R
c12f795026 feat(ui): move app actions to sidebar and add sidebar user menu variant
Shift app-level actions (settings and user menu) from the top bar to the bottom
of the sidebar for improved navigation consistency. Introduce a new "sidebar"
variant for the UserMenu component with tailored styling and behavior for
sidebar placement. Update SettingsModal to support a customizable trigger label
and styling for sidebar integration. Refactor FinderSidebar to accept an
optional bottomSlot, enabling flexible placement of sidebar controls.
2026-05-28 14:58:03 -06:00
Richard R
eea143abbf refactor(doclist): centralize icons grid layout logic into reusable utility
Move grid style and template calculation for icon views into a shared
iconsGrid.ts utility. Replace inline grid logic in IconsView and DocumentListSkeleton
with calls to the new iconsGridStyle function for consistency and maintainability.
2026-05-28 14:51:48 -06:00
Richard R
e4b0c8358e test(documents): update selectors and assertions for document actions and accessibility
Revise test helpers and specs to use stricter role-based selectors for document
actions, including delete buttons and document links. Improve accessibility and
upload tests by asserting document presence via explicit file lists instead of
generic text checks. Remove redundant summary assertion from delete flow.
2026-05-28 12:34:29 -06:00
Richard R
b9490a53ac refactor(doclist): remove loading state and unify navigation prefetch behavior
Eliminate unused loading state and related effects from document tile and list
row components. Standardize navigation by disabling prefetch on all document
open links in doclist views for consistent client-side routing. Cache
document list state on load to improve state restoration and initialization.
2026-05-28 09:15:18 -06:00
Richard R
eff25ad8df feat(doclist): add bulk folder removal with sidebar menu and confirmation
Introduce a "Remove All Folders" action accessible via a new sidebar menu,
allowing users to clear all folders at once. Add a confirmation dialog to
prevent accidental removal. Update the FinderSidebar to include a menu button
with the new action, and implement supporting state and handlers in the
document list. Add DotsHorizontalIcon for menu UI. Expand folder tests to
cover the new bulk removal flow.
2026-05-28 05:18:53 -06:00
Richard R
f569c8ded2 style(doclist): improve gallery view tile visuals and image preview readiness
Update gallery view document tiles with refined border, shadow, and background
styles for better visual feedback and accessibility. Enhance image preview
component to immediately reveal cached images by checking decode state on mount,
preventing opacity glitches on remount. Adjust gallery rail scrolling to reset
on document list changes for consistent navigation.
2026-05-28 04:51:04 -06:00
Richard R
85aff2c1be refactor(doclist): flatten foldered document structure and remove folder UI
Eliminate folder grouping logic and related UI from all document list views.
Switch to a flat document array model for rendering, selection, and filtering.
Remove folder collapse/expand state and folder drag-and-drop handling. Update
sidebar and toolbar components to remove folder creation and management
controls. Simplify props and internal state across all views for consistency.
2026-05-28 04:36:08 -06:00
Richard R
d59e911ca2 Fix flaky document preview blob URL reuse 2026-05-28 04:23:12 -06:00
Richard R
495c197c4c feat(doclist): add recently opened sorting and metadata to documents list
Introduce recentlyOpenedAt field to document types and fetch per-document
recently opened timestamps from local cache. Update the "Recently Opened"
sidebar filter to sort by this value and show only documents with recent
activity. Adjust grid layout logic for icons view to better handle small
numbers of documents. Update sidebar label for clarity.
2026-05-27 23:13:53 -06:00
Richard R
13e0647b47 refactor(doclist): convert icon size controls to dropdown for icons view toggle
Replace inline icon size buttons with a dropdown menu that appears when
the icons view toggle is active. This change improves toolbar clarity by
grouping related controls and enhances accessibility with better focus
handling. The update also streamlines the button structure for view mode
selection.
2026-05-27 21:10:01 -06:00
Richard R
8104afa3e3 feat(doclist): implement dynamic grid layout for icons view using CSS variables
Replace fixed Tailwind grid column classes with a responsive CSS grid
that leverages tile width variables per icon size. Grid columns now
adapt automatically via `auto-fit` and `minmax`, and grid gaps are
centralized for consistency. This update enhances layout flexibility,
maintainability, and responsiveness in the icons view.
2026-05-27 20:52:52 -06:00
Richard R
53993aa3c6 feat(doclist): improve sidebar responsiveness and toolbar interaction in document list
Enhance mobile and desktop sidebar behavior by introducing a separate state for mobile sidebar visibility and ensuring it does not persist from desktop state. Update FinderToolbar to visually indicate sidebar open state and make the toolbar sticky for better accessibility. Refine FinderSidebar width handling for improved responsiveness. Remove redundant border in mobile sidebar dialog for a cleaner appearance. These changes provide a more consistent and intuitive sidebar experience across viewports.
2026-05-27 20:11:43 -06:00
Richard R
0470f044ef refactor(doclist): restructure loading and empty states, update skeletons and columns UI
Simplify document list state handling by centralizing loading and empty logic into DocumentList.
Remove redundant state management from HomeContent. Refactor DocumentListSkeleton to support all view modes and icon sizes for improved visual consistency during loading. Enhance ColumnsView styling for folder and document rows, improving selection, hover, and drag feedback. Use useMemo for visible order calculation to optimize rendering.

These changes streamline the document list rendering flow, unify skeleton loading visuals, and improve the clarity and responsiveness of the columns view interface.
2026-05-27 19:55:23 -06:00
Richard R
46d5bab647 cleanup doclist formatting helpers and view props 2026-05-27 19:40:12 -06:00
Richard R
eb8820d888 feat(doclist): enhance document list UI with overlay uploader and improved file size formatting
Integrate DocumentUploader in overlay mode to wrap all document views, enabling drag-and-drop uploads across the entire list area. Refine file size formatting to support KB, MB, and GB units in all views and tiles. Standardize button scaling and padding for consistent UI rhythm. Update DocumentPreview and detail panels to display both type and human-readable size. Add overlay drag visual feedback for uploads.

These changes improve usability, visual clarity, and consistency in the document list interface.
2026-05-27 19:35:03 -06:00
Richard R
7a88dabc53 refactor(doclist): redesign document list with new views, DnD, and windowed UI
Revamp the document list experience by introducing a Finder-style window
interface with multiple views (icons, list, columns, gallery) and a new
sidebar filter system. Remove legacy folder and list item components in
favor of modular, windowed views. Integrate react-dnd-touch-backend and
custom DnD context for improved drag-and-drop, including mobile support.
Update uploader and dialog styles for consistency. Extend document types
to support new view modes, icon sizing, and sidebar state.
2026-05-27 18:59:13 -06:00
Richard R
218e248cc2
Merge pull request #90: Refactor compute architecture with workerized ONNX PDF/Whisper pipeline
- Monorepo Architecture: Separates processing into compute/core and compute/worker Fastify service.
- NATS JetStream: Migrates processing queues to NATS JetStream with lazy connection and 120s idle sleep for Railway.
- ONNX PDF Layout: Upgrades to PP-DocLayout-V3 with block overlay UI, force reparse, and block-level skip playback rules.
- ONNX Whisper Alignment: Replaces whisper.cpp with a pure JS ONNX alignment decoder.
- Onboarding Coordinator: Integrates OnboardingFlowContext to manage sequential modals (Privacy, Claim Guest Data, Dexie Cloud Migration).
- Database Schema: Adds documentSettings table and tracks parsing stages (parseState, parsedJsonKey) in the documents table.
- Error Contracts: Standardizes API endpoints on ServerAppError responses.
2026-05-27 18:00:01 -06:00
Richard R
3db193e012 feat(hooks): introduce useUnmountCleanupRef for effect cleanup abstraction
Add useUnmountCleanupRef custom hook to centralize and simplify unmount
cleanup logic in document page components. Refactor EPUB, HTML, and PDF
page components to utilize this hook, replacing repetitive ref and effect
patterns for invoking cleanup functions on unmount. This reduces code
duplication and improves maintainability.
2026-05-27 16:22:14 -06:00
Richard R
139a227aa4 fix(html): use ref to ensure latest clearCurrDoc on unmount 2026-05-27 16:12:09 -06:00
Richard R
f27b5e6504 fix(epub): prevent stale clearCurrDoc reference on unmount in EPUB page 2026-05-27 16:03:34 -06:00
Richard R
a00caa9052 refactor(ui): streamline document cleanup and navigation logic in doc pages
Remove redundant clearCurrDoc calls from navigation links and ensure document cleanup occurs on component unmount for EPUB, HTML, and PDF pages. Simplify PDF back navigation by eliminating sidebar delay logic. Refactor DocumentListItem to avoid unnecessary router usage and consolidate document link handling. These changes improve maintainability and consistency in document lifecycle management across the UI.
2026-05-27 15:48:46 -06:00
Richard R
5305b896c6 refactor: complete phase 7 error-contract rollout 2026-05-27 08:02:31 -06:00
Richard R
00ce39ed74 refactor: roll out phase 7 server error contract 2026-05-27 07:24:07 -06:00
Richard R
3da13c2970 chore(logging): remove redundant errorCode fields from server logs and error contracts
Eliminate the errorCode property from server logger calls, error contract types, and API error bodies throughout the codebase. This reduces duplication and streamlines error reporting, relying on structured event names and error classes for identification. All affected API routes, server logic, and logging utilities have been updated for consistency. Documentation directory added for future reference.
2026-05-27 06:33:14 -06:00
Richard R
5de9069c20 refactor(logging): finalize phase 6 error contract and pretty output 2026-05-27 06:17:25 -06:00
Richard R
f78d066434 refactor(api): enhance tts segment error handling and logging patterns
Update TTS segment ensure route to distinguish aborted and failed segment generation, providing more granular logging and context. Replace ambiguous message variables with clearer detail fields, and improve log event naming for abort scenarios. Adjust ESLint logger call selectors for maintainability and consistency, consolidating selector logic and enforcing stricter server logger usage patterns. These changes improve observability and error traceability in TTS segment operations.
2026-05-27 05:58:48 -06:00
Richard R
034c1f03a4 logging: complete phase 5 contract and enforce via eslint rules 2026-05-27 04:54:52 -06:00
Richard R
29cc4fa42d logging: apply phase 4 warn/error level tuning 2026-05-27 04:28:35 -06:00
Richard R
59d0101c0c feat(server): implement unified structured logging with pino and enforce usage
Introduce `serverLogger` utility based on pino for consistent, structured logging across all server and API modules. Replace direct console logging with `serverLogger` and add request-scoped logging helpers. Update environment variable handling, documentation, and deployment guides to reflect new logging configuration (`LOG_FORMAT`, `LOG_LEVEL`, `COMPUTE_LOG_LEVEL`). Enforce no-console in server code via ESLint and add pino/pino-pretty dependencies.

This change standardizes log output, improves observability, and prepares the codebase for ingestion by log platforms.
2026-05-27 04:20:17 -06:00
Richard R
7732426963 feat(pdf): enable opId tracking and propagation for layout parsing operations
Integrate opId handling throughout PDF layout parsing by updating client hooks, API routes, and event streams to support operation ID propagation. Add server utility for worker operation creation to centralize opId management. Enhance polling and SSE subscription utilities to accept and return opId, improving traceability and coordination of parse operations across the client and server.
2026-05-27 02:35:46 -06:00
Richard R
1ec9632d4b feat(api): set maxDuration for document parsed events route
Define maxDuration as 300 seconds for the parsed events API route to explicitly
control execution timeouts during long-running PDF layout processing. This
ensures more predictable serverless behavior for streaming operations.
2026-05-27 01:41:29 -06:00
Richard R
956ecf5f4a refactor(api): add structured logging for worker op and SSE error states
Introduce detailed console logging across document parsing API routes and worker
operation state fetches. Logs now capture stream openings, error conditions,
worker state unavailability, and invalid responses, improving observability and
debuggability for PDF parsing and ONNX layout processing flows.
2026-05-27 01:32:14 -06:00
Richard R
5c24ea5763 docs(env): clarify env var ownership split for external compute worker mode
Expand documentation and example env files to explicitly distinguish which
environment variables belong in the app/root `.env` versus the `compute/worker/.env*`
files when deploying in external worker mode. This clarifies routing/auth
settings versus worker runtime configuration, reducing misconfiguration risk.
2026-05-27 01:19:37 -06:00
Richard R
93ae9f3082 orchestrate privacy modal in onboarding flow 2026-05-27 00:50:10 -06:00
Richard R
22f132bbc8 hard-cut onboarding modal flow orchestration 2026-05-27 00:40:12 -06:00
Richard R
14b1b5602f fix(parse-progress): persist opId only after worker op exists 2026-05-27 00:08:42 -06:00
Richard R
c717f7e33e fix(storage): restore main-like upload path without pre-presign bucket scan 2026-05-26 20:08:02 -06:00
Richard R
ebc1dbcda0 ci(playwright): install nats-server from linux tarball asset 2026-05-26 19:48:55 -06:00
Richard R
16e94a1587 build(docker): include compute/core and worker package manifests for deps
Add compute/core/package.json and compute/worker/package.json to Docker build
context to ensure all workspace dependencies are correctly installed during
image creation. This change prevents missing dependency issues for subpackages
when building in isolated environments.
2026-05-26 19:33:41 -06:00
Richard R
cd530a365d refactor(whisper): migrate default ONNX model references to q4 variant
Update all configuration, manifest, and documentation references to use the q4
Whisper ONNX model files instead of int8. Adjust expected file names, hashes,
and environment variable descriptions to reflect this new default. This aligns
runtime, deployment, and developer documentation with the updated model artifact
expectations for improved consistency.
2026-05-26 19:22:07 -06:00
Richard R
b2cf4aa7aa ci(playwright): install nats for embedded worker startup 2026-05-26 19:00:07 -06:00
Richard R
1de7ed67b4 refactor(worker): extract op helpers and shared parse mapping 2026-05-26 18:54:11 -06:00
Richard R
0b1ce1c8dc refactor(worker): remove unused job_state kv writes 2026-05-26 18:47:53 -06:00
Richard R
e6e029cf8a cleanup compute core/server dead paths 2026-05-26 18:43:05 -06:00
Richard R
6f68c169b6 feat(compute): integrate worker operation state polling for document parsing
Add worker operation state polling to document parse routes, enabling real-time
status and progress updates from the ONNX layout worker. Introduce
`fetchWorkerOperationState` utility and status mapping logic to synchronize
parse status with worker job state, improving accuracy for pending, running,
and failed operations.
2026-05-26 18:38:50 -06:00
Richard R
bfea95aa35 fix(entrypoint): ensure fatal shutdown on critical embedded service exit
Add robust fatal shutdown handling for embedded services (weed mini,
nats-server, compute-worker) to ensure all services are terminated if any
critical process exits unexpectedly. Introduce flags to prevent duplicate
shutdowns and improve error reporting for better reliability in process
management.
2026-05-26 18:05:57 -06:00
Richard R
e5ee621288 refactor(docstore): improve docstore directory resolution for monorepo setups
Refactored docstore directory logic to dynamically locate the monorepo
root by searching for a pnpm-workspace.yaml marker. If found, the
docstore directory is anchored at the monorepo root; otherwise, it
defaults to the current working directory. This enhances consistency
when running in different environments.

Also updated process shutdown and SeaweedFS launch arguments in the
entrypoint script for improved reliability and log filtering.
2026-05-26 18:05:45 -06:00
Richard R
02979a98af refactor(config): remove START_EMBEDDED_COMPUTE_WORKER env and enforce nats-server for embedded mode
Eliminates the START_EMBEDDED_COMPUTE_WORKER environment variable from configuration,
documentation, and entrypoint logic. Embedded compute worker startup is now strictly
determined by the absence of COMPUTE_WORKER_URL. If embedded mode is triggered,
the presence of the nats-server binary is mandatory; otherwise, an error is thrown.
Documentation and example env files have been updated to reflect this streamlined
behavior and clarify requirements for both embedded and external compute worker setups.
2026-05-26 17:37:18 -06:00
Richard R
e3ac1b1dac docs/env: clarify embedded vs external worker env ownership 2026-05-26 17:16:38 -06:00
Richard R
a073fb63a3 runtime: default embedded worker startup when worker url is unset 2026-05-26 17:00:41 -06:00
Richard R
943e547836 test: fix JetStream mock typing for tsc 2026-05-26 16:39:40 -06:00
Richard R
0e6f966e3a hard-cut batch C: worker projection ownership finalized 2026-05-26 16:38:22 -06:00
Richard R
012cb63de6 hard-cut batch F: remove COMPUTE_MODE docs and env references 2026-05-26 16:28:40 -06:00
Richard R
c0f0438fff hard-cut batch E: embedded nats+worker startup in single-container entrypoint 2026-05-26 16:25:39 -06:00
Richard R
fb0c95dfb1 hard-cut batch D: enforce worker bundle isolation and CI guard 2026-05-26 16:00:26 -06:00
Richard R
f77ee8cfd4 hard-cut batch B: worker-proxy parsed SSE and remove app progress bus 2026-05-26 15:58:14 -06:00
Richard R
f5eb593554 hard-cut batch A: worker-only app compute backend 2026-05-26 15:56:39 -06:00
Richard R
821d0fdc6b build(tsconfig): add path alias for compute-core control-plane module 2026-05-26 15:45:19 -06:00
Richard R
df05a7d7a3 refactor(control-plane): hard-cut worker events and shared SSE helpers 2026-05-26 11:41:33 -06:00
Richard R
084cbdfac1 refactor(worker-control-plane): route ops through core orchestrator 2026-05-26 11:16:54 -06:00
Richard R
00fc2d5e36 feat(control-plane): add shared orchestrator and in-memory adapters 2026-05-26 11:09:43 -06:00
Richard R
cd06201f5c refactor(parse-progress): rely on sse reconnect and document mode behavior 2026-05-25 21:37:43 -06:00
Richard R
be7ec2c254 feat(worker-events): stream op events via jetstream and proxy in app 2026-05-25 21:37:27 -06:00
Richard R
2df49b54de feat(parse-progress): add local in-memory realtime bus 2026-05-25 21:06:18 -06:00
Richard R
52083c6a09 build(config): add path alias for api-contracts in compute-core
Update tsconfig.json to include a path mapping for
@openreader/compute-core/api-contracts, enabling cleaner imports and
improving modularity for API contract usage across the codebase.
2026-05-25 12:00:31 -06:00
Richard R
e70b3619e0 tests: harden pdf readiness waits and extend tts/pdf timeouts 2026-05-24 20:58:57 -06:00
Richard R
3f17874fe5 tts ensure: skip whisper alignment in test namespaces 2026-05-24 20:58:48 -06:00
Richard R
b2517bc91d pdf parse: dedupe shared jobs and stop enqueue-on-read 2026-05-24 20:58:34 -06:00
Richard R
12c586d7be refactor(core,server): migrate contracts to api-contracts and modularize job handlers
Move shared type contracts from `contracts` to new `api-contracts` module in `compute/core` for clearer API boundaries. Remove legacy `contracts`, `pdf`, and `whisper` index files. Update all imports to use `api-contracts`. Refactor server job logic by replacing `parsePdfJob.ts` with modular `user-pdf-layout-job.ts` and `user-whisper-align-job.ts`, updating all relevant API routes and compute integration. This improves maintainability and separation of concerns across compute and server layers.
2026-05-24 11:39:25 -06:00
Richard R
e75114a943 refactor(onboarding): introduce onboarding state coordinator and central registry
Add useOnboardingCoordinator hook and onboarding-state registry to manage
onboarding-related state such as privacy acceptance and first-visit tracking.
Refactor SettingsModal to utilize the new onboarding state abstractions,
removing legacy privacy gating logic and improving maintainability.

Includes unit tests for onboarding-state registry to ensure correctness.
2026-05-23 17:07:50 -06:00
Richard R
df321b8832 fix(jobs): make TTS segment cache invalidation non-blocking in parsePdfJob
Move TTS segment cache invalidation to a fire-and-forget promise after
document update to prevent cache errors from blocking parse readiness.
Log warnings for both cache invalidation failures and warnings, but do not
interrupt the main job flow. This improves job robustness and avoids
unnecessary failures due to cache issues.
2026-05-23 05:55:22 -06:00
Richard R
65d25e3ce5 refactor(ui): centralize privacy gating logic for settings modal
Consolidate privacy acceptance checks into reusable hooks and callbacks within
SettingsModal. Replace scattered gating with a single entry point for opening
settings or changelog views, ensuring consistent enforcement. Update test
helpers to robustly dismiss onboarding and settings overlays by avoiding
state race conditions.
2026-05-23 05:50:54 -06:00
Richard R
3dcda2b4b7 refactor(logging): enhance logger configuration and streamline event logging 2026-05-22 13:11:51 -06:00
Richard R
ad82aede73 feat(worker,whisper): improve logging for operation lifecycle and remove align mutex
Enhance worker operation handling by adding detailed logging for operation
acceptance, reuse, replacement, and stale detection, including opKey hashes and
reasons for replacement. Remove the alignMutex from Whisper alignment logic to
simplify concurrency control and reduce unnecessary serialization of alignment
requests. These changes improve observability and throughput for alignment and
layout parsing jobs.
2026-05-22 12:49:33 -06:00
Richard R
653dbedaee refactor(worker): add configurable NATS JetStream replica count
Introduce COMPUTE_NATS_REPLICAS environment variable to control the number of
replicas for JetStream streams and key-value buckets. Only values 1, 3, or 5
are accepted, defaulting to 1 for other inputs. Update worker server logic and
deployment documentation to describe and support this configuration.
2026-05-22 09:32:25 -06:00
Richard R
64a68b1011 docs(worker): document and implement idle-based NATS disconnect for Railway sleep
Add lazy NATS connection lifecycle and idle disconnect logic to the worker,
enabling automatic disconnection after 120s of inactivity to support Railway
container sleep. Update deployment docs to describe this behavior, including
caveats and reconnection flow.
2026-05-22 09:01:14 -06:00
Richard R
0fcf4e2dab style(config): update eslint import rules for compute-core/types subpath
Allow '@openreader/compute-core/types' as a permitted subpath in ESLint
import rules. Update the lint message to reflect the new allowed subpath,
ensuring consistency with recent type centralization efforts.
2026-05-22 02:01:52 -06:00
Richard R
b6be71e3b1 refactor(types,api): centralize PDF and TTS type exports to compute-core/types
Move and re-export all PDF parsing and TTS-related types from
@openreader/compute-core/types, eliminating local type duplication.
Update imports throughout the codebase to use the new centralized type
module. Remove obsolete worker-contract file and update tsconfig paths
and compute-core exports for the new types entry point. Refactor API
routes and job logic to use the new type imports and shared parse-state
healing utility. This streamlines type management and improves
consistency between compute and app layers.
2026-05-22 01:51:51 -06:00
Richard R
09e8898683 feat(pdf,config): implement stale parse-state healing and unify op stale window config
Introduce automatic healing for stale PDF parse states in API routes. Add
`isDocumentParseStateStale` utility and use a shared `COMPUTE_OP_STALE_MS`
environment variable (with config getter) to control the stale window for both
worker op replacement and app-side parse-state healing. Update documentation and
environment variable references to reflect new config.

Enables automatic marking of stuck parses as failed, improving reliability and
retry behavior in distributed compute environments.
2026-05-22 01:21:18 -06:00
Richard R
1d4a301a5d fix(pdf): correct parse status handling and loader display logic
Update PDF viewer and document hook to improve parse status management:
- Set parseStatus to null on clear instead of 'pending' for accurate state
- Explicitly set initial parseStatus and reset parseProgress when loading a new document
- Adjust loader display logic to check for resolved parse status before showing expanded loader

Prevents UI inconsistencies and ensures parse progress is reflected correctly during document transitions.
2026-05-22 00:17:15 -06:00
Richard R
50c4330ab1 refactor(core,config,tests): update ONNX model manifest loading and test imports
- Replace static JSON imports with runtime manifest loading in pdf and whisper model modules for compatibility with ESM and bundlers
- Refactor Next.js config to clarify compute mode logic and improve worker bundling conditions
- Update test imports to use package entrypoints instead of relative paths
- Remove redundant whisper alignment/model tests now covered elsewhere or by integration
- No breaking changes to public API or model handling logic
2026-05-21 23:49:56 -06:00
Richard R
b2bb36911a test(whisper): fix playwright ESM loading via dynamic imports 2026-05-21 21:53:55 -06:00
Richard R
2fd9f05652 fix(config): load webpack DefinePlugin via createRequire 2026-05-21 21:49:03 -06:00
Richard R
7a49ff9896 chore(lint): fix eslint violations in worker config and e2e tests 2026-05-21 21:44:58 -06:00
Richard R
057d2a4e32 chore(eslint): restrict compute-core subpath imports to local-runtime 2026-05-21 21:42:35 -06:00
Richard R
55acf05ad6 refactor(compute-core): rename pdf and whisper modules to clean capability layout 2026-05-21 21:41:37 -06:00
Richard R
3a2c6ffa42 refactor(compute-core): split runtime into config/platform and move contracts barrel 2026-05-21 21:40:16 -06:00
Richard R
37b6999c2c refactor(compute-core): collapse to root/local-runtime entrypoints 2026-05-21 21:30:44 -06:00
Richard R
37d11bf9b8 refactor(compute): remove legacy app whisper duplicate and unify core whisper boundary 2026-05-21 21:15:10 -06:00
Richard R
1612e6694b refactor(compute-core): capability-boundary exports for runtime and pdf-layout 2026-05-21 21:11:28 -06:00
Richard R
d8c6d00de6 docs(env): clarify local vs worker compute env ownership 2026-05-21 20:58:32 -06:00
Richard R
a29e6fc250 docs(compute): align worker env docs and model override vars 2026-05-21 20:57:10 -06:00
Richard R
754fbee227 refactor(compute-core): centralize runtime cpu config exports 2026-05-21 20:54:22 -06:00
Richard R
51fbfa117d worker: avoid idle consumer blocking shared compute gate 2026-05-21 20:06:24 -06:00
Richard R
4664806ba0 worker: enrich single-line request logs with status and opId 2026-05-21 19:48:46 -06:00
Richard R
bcdff35200 worker: simplify request logs to single-line entries 2026-05-21 19:44:25 -06:00
Richard R
661af71fa4 refactor(whisper): implement dynamic timeout configuration for alignment and decoding
Replace static timeout constants with functions that derive alignment and ffmpeg
decode timeouts from compute timeout config. This enables adaptive timeout
behavior based on runtime configuration, improving robustness under varying
resource constraints. Update error handling and internal APIs to propagate
timeout values accordingly.

Remove unused Node.js engine constraint from package.json for broader compatibility.
2026-05-21 19:29:03 -06:00
Richard R
7cdd89b26d compute-core: resolve pdf layout license asset by module path 2026-05-21 19:09:01 -06:00
Richard R
1f13ecaed9 worker: use default JetStream long-poll for consumer next 2026-05-21 19:04:16 -06:00
Richard R
4d12c6913a test(export): streamline export cleanup and improve test reliability
Remove unnecessary UI reload and modal interactions during audiobook export
cleanup to reduce flakiness under load. Set explicit timeout for long-running
MP3 export test to ensure consistent execution. These changes enhance test
stability and maintainability.
2026-05-21 18:50:11 -06:00
Richard R
384369f48b refactor: fix tests + remove deprecated PDF layout processing files and update imports
- Deleted mergeTextWithRegions.ts, LICENSE.txt, manifest.json, parsePdf.ts, renderPage.ts, runLayoutModel.ts, stitchCrossPageBlocks.ts, and types.ts as they are no longer needed.
- Updated unit tests to reflect new import paths from @openreader/compute-core.
- Adjusted tsconfig.json to include new paths for pdf-layout module.
2026-05-21 18:48:17 -06:00
Richard R
3ebcb3e6fd tts: make ensure route abort-aware and reduce abort log noise 2026-05-21 16:46:47 -06:00
Richard R
3c25c93e8b tts: remove redundant route-level alignment timeout wrappers 2026-05-21 16:33:55 -06:00
Richard R
c9dad9c23c compute: reuse shared timeout helpers across worker and local 2026-05-21 16:32:24 -06:00
Richard R
d70c6aa5c6 docs: document shared compute timeout env vars across app and worker 2026-05-21 16:27:03 -06:00
Richard R
e384b66b7d compute: align local and worker-client timeout behavior 2026-05-21 16:17:47 -06:00
Richard R
fbb6f484ce compute: centralize timeout config in compute-core and use in worker 2026-05-21 16:15:34 -06:00
Richard R
0d494cf5e5 docs(config): update PDF job attempts default to 1 for layout parsing
Align COMPUTE_PDF_JOB_ATTEMPTS default value with new ONNX-based PDF layout
parsing approach by setting it to 1 in environment examples, worker config,
and deployment documentation. This reflects improved reliability and reduces
unnecessary retries for PDF processing jobs.
2026-05-21 16:04:17 -06:00
Richard R
f8672b073e refactor(compute): unify job concurrency and CPU thread allocation logic
Standardize compute job concurrency configuration by introducing a shared
COMPUTE_JOB_CONCURRENCY setting, replacing separate PDF and Whisper concurrency
controls. Add cross-platform CPU core detection and thread budgeting utilities,
and update both ONNX model execution and concurrency limiters to use dynamic
thread allocation per job. Refactor environment variable docs and examples to
reflect unified concurrency management. Streamline PDF.js font path resolution
for improved reliability across environments.
2026-05-21 15:58:29 -06:00
Richard R
671d5c6adb feat(pdf,worker): enhance PDF layout parsing UX and add robust timeout controls
Introduce idle and hard cap timeouts for PDF layout parsing in the compute worker,
ensuring long-running jobs are capped and progress is tracked with explicit
timeouts. Refactor NATS JetStream client initialization to use a configurable
API timeout. Update PDF viewer UI to treat parse state as 'pending' by default,
improving clarity during document preparation and parsing. Refine parse loader
labels and progress logic for more accurate user feedback.
2026-05-21 13:50:40 -06:00
Richard R
a2c714c2a5 refactor(worker,tts): unify PDF and Whisper retry config, add alignment timeouts
Update compute worker to use separate environment variable for PDF job attempts
(`COMPUTE_PDF_JOB_ATTEMPTS`) and set Whisper job max deliveries to 1, clarifying
and separating retry logic between job types. Adjust `.env.example` and deployment
docs to comment out defaults and document new/renamed variables for advanced tuning.

In TTS segment ensure route, introduce explicit timeouts for Whisper alignment
operations, with configurable durations for local and worker modes, improving
robustness and error handling for long-running alignments.
2026-05-21 12:37:21 -06:00
Richard R
fe9104e8fc docs(pdf,spinner,config): update parse state UI, spinner theming, and worker env docs
Revise PDF viewer to introduce an explicit 'unknown' parse state and improve
loader and status messaging for better user clarity. Refactor Spinner to use
'currentColor' for border styling and simplify conditional rendering based on
className. Expand compute worker .env.example with explicit concurrency,
timeout, and job attempt variables for enhanced configuration transparency.
2026-05-21 11:59:12 -06:00
Richard R
08df437e61 chore: add .npmrc for npm configuration 2026-05-21 11:18:41 -06:00
Richard R
bde8d74a97 refactor(pdf): improve parse loader UX and PDF viewer readiness signaling
Enhance PDF parse loader transitions by introducing a delayed expansion for detailed progress, reducing UI flicker during parse-to-render handoff. Refactor PDFViewer to ensure document readiness is signaled only once per load, preventing redundant callbacks and improving synchronization between parsing and rendering states.
2026-05-21 10:54:07 -06:00
Richard R
dd494d50aa Add force-reparse confirmation flow in PDF UI 2026-05-21 10:34:29 -06:00
Richard R
10748c7fcd Add force-token cache bust for worker PDF op keys 2026-05-21 10:32:18 -06:00
Richard R
3aad8a51e4 Add local compute concurrency limiter with tests 2026-05-21 10:30:36 -06:00
Richard R
4497f610c0 feat(pdf): implement granular PDF parse progress tracking and migrate to parseState
Introduce detailed progress reporting for PDF layout parsing, exposing phase and page-level updates to clients. Replace legacy parseStatus with a structured parseState field in the database schema, updating all relevant backend and API logic. Add SSE endpoint for real-time parse progress updates. Update client hooks and UI to reflect granular progress and improve user feedback during document parsing. Includes migration scripts and new parse-state utility module.
2026-05-21 10:27:16 -06:00
Richard R
a7b955e1ca refactor(worker): adopt operation-based API and opKey deduplication for compute flows
Migrate worker and backend contract to an operation-centric model, introducing opKey-based deduplication for Whisper alignment and PDF layout parsing. Add new operation request/state types and update worker endpoints to accept generic operation requests and expose operation status via SSE. Refactor client and server logic to support operation flows. Update deployment docs, environment examples, and Dockerfile to reflect new architecture and dependency management.
2026-05-21 09:05:26 -06:00
Richard R
8190b57ff2 feat(documents): support parsed document retrieval by key and configurable worker wait timeout
Add ability to fetch parsed documents using a stored S3 key, enabling more flexible blob access patterns. Update API route to prefer `parsedJsonKey` if present, falling back to legacy lookup. Introduce `COMPUTE_WORKER_WAIT_TIMEOUT_MS` environment variable for customizing worker job wait time, with a new default of 120 seconds. Update documentation and examples to reflect these changes.
2026-05-20 18:07:32 -06:00
Richard R
49d9735237 refactor(worker): unify port environment variable usage and update deployment docs
Align compute worker to use PORT environment variable consistently across
Dockerfile, docker-compose, and server code, replacing COMPUTE_WORKER_PORT.
Update .env.example and deployment documentation to clarify platform-specific
behavior, especially for Railway and similar environments. This simplifies
configuration and reduces platform-specific issues.
2026-05-20 17:05:35 -06:00
Richard R
086329505f fix(worker): improve environment variable handling and URL normalization
Update worker server to prioritize platform-specific PORT environment variable
when setting the compute worker port. Add robust normalization for
COMPUTE_WORKER_URL, ensuring proper scheme and validation, and trim trailing
slashes for consistency. This enhances reliability in diverse deployment
environments.
2026-05-20 17:01:14 -06:00
Richard R
91d9fe47a5 refactor(compute): standardize compute mode detection for backend initialization
Update backend creation logic to consistently determine compute mode using
a local variable, ensuring correct handling when the mode is undefined. This
improves clarity and robustness in backend selection.
2026-05-20 16:39:25 -06:00
Richard R
29b7cfafb9 chore(deps): update node engine requirement and clean server externals
Remove 'ffmpeg-static' from server external packages in Next.js config and
specify Node.js 22.x as required engine in package.json to align with
deployment environment.
2026-05-20 16:26:07 -06:00
Richard R
fb32dea73c build(config): adjust output and file tracing for Vercel and local compute modes
Update Next.js configuration to conditionally set 'output' only when not on
Vercel, and always define 'outputFileTracingExcludes' with dynamic includes
based on compute mode. This ensures compatibility with Vercel deployments and
improves local/server build behavior.
2026-05-20 15:53:26 -06:00
Richard R
944b8246e5 refactor(context): expose provider config type and instruction flags in TTS value
Update TTS context to include configProviderType, supportsInstructions, and
ttsInstructions in the provided value, improving downstream access to provider
capabilities and instruction handling.
2026-05-20 15:42:45 -06:00
Richard R
88809b826b fix(compute): unify PDF layout result handling with object key support
Update compute backend interfaces and job logic to consistently handle parsed
PDF layout results as either direct data or S3 object key references. Refactor
result types, local and worker backends, and job processing to support this
pattern. Improves flexibility for large document parsing and object storage
integration.
2026-05-20 15:38:24 -06:00
Richard R
4ddedee8ec docs(worker): document and expose JetStream storage limits for jobs and KV state
Add `COMPUTE_JOBS_STREAM_MAX_BYTES` and `COMPUTE_JOB_STATES_MAX_BYTES` to
environment example, worker server configuration, and deployment docs. These
variables allow tuning JetStream resource caps for job queue and state storage
to better fit deployment requirements.
2026-05-20 15:34:17 -06:00
Richard R
1bd13c02fe refactor(compute): modularize core contracts and runtime, enable flexible backend resolution
Restructure compute core by extracting job contracts and runtime logic into separate
modules (`contracts.ts`, `local-runtime.ts`) for improved modularity and clearer
API boundaries. Update exports and imports across worker, server, and local
compute backends to use the new modules. Enhance Next.js config to support
dynamic backend selection via build-time constants and aliasing, enabling
seamless switching between local and worker compute modes. Adjust TypeScript
paths and Dockerfile entrypoint to align with the new structure.
2026-05-20 15:07:22 -06:00
Richard R
8310091816 docs(deploy): update compute worker and Vercel deployment guides for Synadia + Railway
Add a comprehensive Synadia Cloud and Railway setup guide to the compute worker
deployment docs, including step-by-step environment variable configuration and
example image tags. Update Vercel deployment instructions with a quick start for
external compute worker integration via Railway and Synadia NGS. Document new
`.env.prod` pattern in .gitignore for production environment management.
2026-05-20 14:17:07 -06:00
Richard R
3a1dd41d91 refactor(build): update Dockerfiles for improved dependency isolation and workspace handling
Remove legacy .npmrc config. Revise root and worker Dockerfiles to enhance
layer caching, explicitly copy only required build artifacts, and streamline
runtime dependencies. Switch to pnpm workspace-aware install for compute worker
to ensure correct dependency resolution. Set Next.js output to standalone mode
for optimized server builds. Update docker-compose sync targets for consistency.
2026-05-20 13:21:12 -06:00
Richard R
b30304a119 docs(worker): document NATS credentials options and update authentication support
Expand compute worker documentation to explain NATS authentication using
either a credentials file or raw credentials string, including Synadia Cloud
integration steps. Update environment variable examples and server connection
logic to support both `NATS_CREDS_FILE` and `NATS_CREDS`. Add `.creds` to
.gitignore to avoid accidental credential leaks.

This improves deployment flexibility for cloud and on-prem environments by
supporting multiple authentication methods for NATS JetStream.
2026-05-20 12:25:13 -06:00
Richard R
4b26531172 chore(worker): migrate compute worker from Redis/BullMQ to NATS JetStream
Replace Redis and BullMQ with NATS JetStream and KV for job queuing and state management
in the compute worker service. Update environment variables, Docker Compose, and documentation
to reflect the new message queue backend. Adjust dependencies and code to use NATS-based
work queue and key-value storage for durable job processing.

This change improves scalability and reliability of distributed compute workloads by leveraging
NATS JetStream's work queue and persistence features. Documentation and configuration examples
are updated to guide deployments using the new backend.
2026-05-20 07:38:03 -06:00
Richard R
502c98f11f refactor(tts): add segment cache clearing and improve sidebar scroll handling
Introduce clearSegmentCaches to TTS context for explicit cache invalidation after segment clearing. Enhance SegmentsSidebar scroll logic to better distinguish user-initiated and programmatic scrolls, improving UX during segment updates.
2026-05-20 04:59:09 -06:00
Richard R
b679bf736c chore(config): remove smartSentenceSplitting option and related code paths
Eliminate the smartSentenceSplitting configuration flag from application state, UI, and all TTS and audiobook adapter logic. Consolidate TTS segment planning to always include context units, streamlining code and reducing conditional branches. Update tests and type definitions to reflect the removal of this feature toggle.
2026-05-20 04:51:10 -06:00
Richard R
bb3eb40966 refactor(tts): extend segment prefetching to support source unit arrays and enhance cache key granularity
Update TTS segment prefetching logic to handle arrays of CanonicalTtsSourceUnit for upcoming and next locations, enabling more precise TTS segment mapping. Expand cache key construction to include providerType and instructions, improving cache differentiation for TTS requests. Adjust related types and usages to support richer segment metadata propagation.
2026-05-20 04:44:47 -06:00
Richard R
9d1f9d5707 fix(pdf): readd source unit extraction for TTS integration in page parsing
Introduce sourceUnitsFromParsedPage helper to extract CanonicalTtsSourceUnit
objects from parsed PDF pages, filtering by block kind and text content. Update
TTS state setter to include extracted source units when available, enabling
improved TTS segment mapping and future extensibility.
2026-05-20 03:53:25 -06:00
Richard R
0798cd845f refactor(export): extract download trigger helper and add retry logic to full download
Move download logic into reusable downloadViaTrigger function. Add retries and
button enabled checks to downloadFullAudiobook for improved robustness against
transient UI or backend delays.
2026-05-20 03:44:09 -06:00
Richard R
8439216dd8 test(pdf-upload): update assertions for PDF viewer heading and page navigation
Replace text-based assertion for PDF title with role-based heading check and
add assertion for visible page navigation button. Improves test robustness
against UI changes and ensures accessibility compliance.
2026-05-19 23:19:53 -06:00
Richard R
84c3b22e5a refactor(pdf-layout): streamline viewer layout settling and harden pdf document loading
Unify and improve layout settling logic across EPUB, HTML, and PDF viewers by
adding delayed recomputation and guarding against zero-height container during
transient states. Refactor PDF document loading to include retry logic with
success detection, ensuring robust handling of transient failures. Update
Playwright test helpers to provide more reliable PDF viewer readiness checks
and introduce request retry utilities for backend API calls. Add targeted unit
tests for single-section chapter fallback in the PDF audiobook adapter. Update
test files and helpers to align with new layout and export logic.
2026-05-19 23:02:41 -06:00
Richard R
01cb95d8e7 fix(compute): migrate compute backend initialization to async and update consumers
Refactor compute backend initialization to use asynchronous loading and promise caching, replacing previous synchronous singleton pattern. Update all consumers—including TTS segment alignment and PDF layout parsing jobs—to await compute backend resolution before invoking methods. This change improves compatibility with dynamic imports and future-proofs backend selection logic.
2026-05-19 19:38:36 -06:00
Richard R
e52b754106 refactor(pdf): remove legacy text extraction and update parse status normalization
Eliminate unused PDF text extraction logic and margin-based filtering from the client library. Refactor API routes to generalize parse status normalization, removing explicit references to deprecated "unsupported" status and type variants. Update usages to accept string parse statuses for improved flexibility. Simplify document state dependency tracking in the PDF document hook.
2026-05-19 19:11:51 -06:00
Richard R
f5d7408f17 refactor(pdf-layout): remove compute "none" mode and unsupported parse status
Eliminate the "none" compute mode and all related code paths, including the NoneComputeBackend, "unsupported" parse status, and PDF margin settings. Parsing is now always available if the app starts successfully, and configuration is limited to "local" or "worker" compute modes. Update types, API routes, client adapters, and documentation to reflect this simplification.

BREAKING CHANGE: "none" is no longer a valid COMPUTE_MODE; only "local" and "worker" are supported. "unsupported" parse status and PDF margin settings are removed.
2026-05-19 18:25:27 -06:00
Richard R
bab3416a36 refactor(compute): add timing metrics to worker job results and status
Introduce WorkerJobTiming interface to capture queue wait, S3 fetch, and compute
durations for compute worker jobs. Update job result and status response types
to include timing data. Enhance worker implementation to record and attach
timing metrics for alignment jobs, improving observability of job processing
latency. Update exports and type usage to support new timing fields.
2026-05-19 15:55:35 -06:00
Richard R
ea5611c882 docs(intro): update highlights and backend details in docs and README
Revise project highlights in README and introduction to emphasize ONNX-based
PDF layout parsing (PP-DocLayoutV3), segment-based TTS playback, and flexible
backend/storage options. Clarify compute modes and self-hosting features.
Update TTS provider descriptions for accuracy and conciseness.

Also expand tsconfig exclude list to omit compute worker sources from main build.
2026-05-19 15:32:18 -06:00
Richard R
f1aa1c3e3b feat(compute): add external compute worker backend and integration
Introduce support for external compute worker mode (`COMPUTE_MODE=worker`) using a new `WorkerComputeBackend`. This enables offloading heavy ONNX Whisper alignment and PDF layout parsing to a standalone worker service (Redis + BullMQ), improving scalability and compatibility with serverless/limited environments.

- Add `@openreader/compute-core` as a shared package for ONNX inference and PDF parsing logic.
- Implement `WorkerComputeBackend` and worker contract/types for remote job execution.
- Update compute backend selection logic and remove previous worker mode guards.
- Extend `WhisperAlignInput` and `PdfLayoutInput` types to support object keys for remote data access.
- Refactor local compute backend to use `@openreader/compute-core` and support both buffer and object key inputs.
- Update job runner, TTS segment alignment, and PDF layout parsing flows to use new compute backend APIs.
- Add scripts, Docker workflow, and documentation for deploying and running the compute worker.
- Update environment variable docs and examples for worker mode, including storage requirements and configuration.
- Document published images and stack changes to reflect the new compute worker architecture.

BREAKING CHANGE: `COMPUTE_MODE=worker` now requires an external compute worker service and S3-compatible object storage. Embedded SeaweedFS (`weed mini`) is not supported in worker mode. See the new documentation for deployment and configuration details.
2026-05-19 15:21:25 -06:00
Richard R
3a21f2a5f5 refactor(config): migrate compute and runtime env vars to new naming scheme
- Replace `OPENREADER_COMPUTE_MODE` and related variables with `COMPUTE_MODE`
- Replace `OPENREADER_*` PDF/Whisper model URLs with `PDF_LAYOUT_MODEL_BASE_URL` and `WHISPER_MODEL_BASE_URL`
- Remove legacy `NEXT_PUBLIC_*` runtime config seeds in favor of `RUNTIME_SEED_*`
- Update documentation, code, and environment references to match new variable names
- Remove deprecated `scripts/fetch-models.mjs` and related npm script
- Update runtime config SSR injection from `window.__OPENREADER_RUNTIME_CONFIG__` to `window.__RUNTIME_CONFIG__`
- Adjust Next.js config to use new compute mode env var and optimize output file tracing for ONNX dependencies

BREAKING CHANGE: Environment variable names for compute mode, model URLs, and runtime config seeding have changed. Update `.env` files and deployment configs to use `COMPUTE_MODE`, `PDF_LAYOUT_MODEL_BASE_URL`, `WHISPER_MODEL_BASE_URL`, and `RUNTIME_SEED_*` as appropriate. Legacy `OPENREADER_*` and `NEXT_PUBLIC_*` variables are no longer supported.
2026-05-19 13:27:07 -06:00
Richard R
874e5ef359 refactor(whisper): migrate word alignment to ONNX backend and remove whisper.cpp integration
Replace the previous whisper.cpp-based word alignment with a fully ONNX-based
implementation using onnxruntime-node and @huggingface/tokenizers. Add new
Whisper ONNX model management, alignment mapping, and spectral analysis modules.
Remove all code and documentation referencing whisper.cpp, update environment
variables, Dockerfile, and docs to reflect ONNX-only alignment. Add unit tests
for alignment and ONNX model logic.
2026-05-19 13:00:21 -06:00
Richard R
6ab5c230c2 feat(pdf): improve cross-page block stitching and enforce TTS segment source boundaries
Enhance PDF cross-page block stitching to move only sentence continuations and
preserve remaining text on the following page. Update TTS segment planning to
support strict source boundary enforcement, ensuring segments do not cross
block or paragraph-title boundaries. Add new tests for both features and
refactor PDF text item normalization to filter out skewed or rotated runs.
2026-05-18 13:21:43 -06:00
Richard R
90e7caac43 refactor(previews): streamline preview generation and update image variant to 400px
Remove temporary working directory usage and in-place file writes from preview
generation, simplifying logic for both PDF and EPUB previews. Change default
preview image variant from 480px to 400px width, updating related constants and
cache schema version to maintain consistency across client and server code.
2026-05-18 12:37:26 -06:00
Richard R
8af4e857b7 feat(pdf): add block geometry-based highlighting using parsed document and locator
Implement block-level highlighting in PDF viewer by leveraging parsed document
structure and TTS segment locator. Extend highlightPattern API to accept options
for parsedDocument, locator, and block geometry mode. Update PDFViewer to use
block geometry highlighting when available, improving accuracy and alignment of
sentence and word highlights with ONNX-based DocLayoutV3 parsing. Refactor
highlighting logic to support both legacy span-based and new geometry-based
approaches.
2026-05-18 12:25:41 -06:00
Richard R
1544a2d969 refactor(previews): update preview variant defaults, cache schema, and image handling
Remove hardcoded defaults for preview variant and width in database schemas to allow more flexible preview generation. Bump preview cache schema version and add versioning to cache rows for consistency. Switch preview image generation to 480px JPEGs with updated file naming and content type. Refactor PDF preview rendering to support configurable output format and quality, and update all preview consumers to use unified "image" buffer property instead of "png". Add migration scripts and update metadata for both Postgres and SQLite.

BREAKING CHANGE: Preview variant and width defaults removed from database; preview cache and image handling updated across system.
2026-05-18 05:20:20 -06:00
Richard R
37a90b734d feat(pdf): migrate to PP-DocLayoutV3 ONNX model and update PDF block taxonomy
Switch PDF layout parsing to use the PP-DocLayoutV3 ONNX model, replacing the previous Docling-based approach. Update all environment variables, model fetching, and manifest handling for the new model and its artifacts. Refactor block kind taxonomy throughout the codebase, tests, and UI to align with PP-DocLayoutV3 labels, including expanded and renamed block types. Revise document settings, block filtering, and stitching logic to support the new set of block kinds. Update documentation and environment variable references to reflect the model transition.

BREAKING CHANGE: PDF parsing now requires PP-DocLayoutV3 ONNX model and updated environment variables; block kind names and settings have changed throughout the system.
2026-05-18 04:26:39 -06:00
Richard R
766c04d08d refactor(pdf): implement ONNX-based Docling layout parsing and block-level TTS for PDFs
Migrate PDF parsing to use ONNX Docling layout model for structured block extraction, enabling improved TTS segmentation and chaptering. Add compute backend abstraction for heavy tasks (alignment, layout parsing) with configuration via `OPENREADER_COMPUTE_MODE`. Integrate block-level locators and document settings for per-document PDF parsing options. Update S3 storage for parsed PDF JSON, add migration and schema changes, and extend client and server APIs for parsed data and settings. Remove legacy word highlight flag in favor of compute capability detection.

- Add ONNX model download, local/none compute modes, and `onnxruntime-node` dependency
- Update PDF viewer and TTS pipeline to use parsed blocks and block-level locators
- Add document settings storage and APIs for per-document PDF options
- Update S3 storage layout for parsed PDF JSON
- Add admin/config/docs updates for new compute and parsing features
- Remove obsolete word highlight runtime flag and UI

BREAKING CHANGE: PDF parsing and word highlighting now require `OPENREADER_COMPUTE_MODE=local` and ONNX model; document settings and S3 layout updated; legacy word highlight flag removed.
2026-05-17 21:18:51 -06:00
Richard R
d90e48aaf3 feat(html): add HTML/TXT/MD sentence and word highlighting, settings, and audiobook export support
Introduce sentence and word highlighting for HTML/TXT/MD documents, including new config options (`htmlHighlightEnabled`, `htmlWordHighlightEnabled`) and synced user preferences. Update the HTML viewer to support these highlights using custom spans, and add corresponding CSS styles. Extend document settings to allow toggling these features. Implement block-level locator sorting and display improvements. Add audiobook export support for HTML documents, including adapter and pipeline integration.

- Add `htmlHighlightEnabled` and `htmlWordHighlightEnabled` to config, user state, and Dexie storage
- Update HTML viewer to apply and manage highlights for sentences and words
- Add CSS for HTML highlight classes and scroll margin
- Extend document settings UI for HTML highlight toggles
- Support HTML audiobook export in modal and pipeline
- Improve block locator sorting and sidebar display for HTML
- Add HTML audiobook adapter and block parsing utilities
2026-05-17 11:14:02 -06:00
Richard R
92762fe858 ci(docs): separate release-triggered deploy and enhance changelog feed sync
Extract release-based deploy triggers to new workflow for clarity and
reliability. Update docs-deploy.yml to streamline dispatch inputs and
remove release/workflow_run triggers. Extend build-changelog-feed.mjs to
support incremental changelog sync via workflow_dispatch with release tag
and action, enabling precise upserts and deletions of changelog entries.
2026-05-14 17:14:43 -06:00
github-actions[bot]
3645b7a327 docs: snapshot v3.0.0 2026-05-14 22:32:52 +00:00
Richard R
ff62cc702e chore(config): downgrade pnpm version and refine allowBuilds exclusions
Some checks failed
Create and publish Docker images / prepare (push) Has been cancelled
Version Docs on Tag / version-docs (push) Has been cancelled
Create and publish Docker images / build (amd64, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (arm64, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / merge (push) Has been cancelled
Revert packageManager to pnpm@10.33.4 in package.json for compatibility.
Expand allowBuilds in pnpm-workspace.yaml to explicitly exclude canvas,
core-js, es5-ext, esbuild, sharp, and unrs-resolver from workspace builds.
These adjustments enhance build reliability and prevent issues with
problematic dependencies.
2026-05-14 16:22:07 -06:00
Richard R
64182e6c8d chore(workspace): streamline monorepo and CI with pnpm workspace, dependency, and workflow updates
Update GitHub Actions workflows to latest action versions and improve pnpm usage by introducing per-package workspace configs. Add docs-site/pnpm-workspace.yaml and update Dockerfile/package.json for better monorepo support. Upgrade dependencies across main and docs-site packages for compatibility and security. Refine TypeScript types and hooks for improved type safety and maintainability.
2026-05-14 15:44:22 -06:00
Richard R
288af0f370 build(deps): update pnpm to v11.1.2 and enforce workspace settings
Upgrade pnpm version to 11.1.2 across Dockerfile, workflows, and package.json.
Add pnpm-workspace.yaml to enable workspace features. Update all docs-related
scripts and CI jobs to use --ignore-workspace for isolated docs-site installs.
Set packageManager field in package.json for explicit version control.

This ensures consistent dependency management and prepares for multi-package
workspace support.
2026-05-14 14:25:29 -06:00
Richard R
80f9386a6c docs(admin-panel): clarify admin provider key handling and AUTH_SECRET rotation
Expand admin panel documentation to detail the transition from live env var
reading to one-shot seeding for TTS providers in v3.0.0. Add warnings about
the impact of rotating AUTH_SECRET on encrypted admin provider keys and the
required manual steps after rotation.

docs(migrations): add schema migration history table

Document the sequence and purpose of each migration introduced in v3.0.0,
clarifying upgrade behavior from v2.2.0 and summarizing schema changes.

chore(env): remove commented default TTS model example from .env.example
2026-05-14 13:56:53 -06:00
Richard R
ca4228970b fix(server): enforce server-only usage for runtime config resolvers
Add explicit runtime checks to prevent runtime config functions from being called on the client. This guards against accidental client-side invocation by throwing errors if executed outside the server environment.
2026-05-14 13:07:43 -06:00
Richard R
fb39723089 fix(config): extract RSC-compatible runtime config resolver
Move runtime config resolution logic to a new `runtime-config-rsc.ts` module for use in React Server Components. Update public layout and landing page to consume the RSC-specific resolver, improving compatibility and separation of concerns.
2026-05-14 12:50:16 -06:00
Richard R
522540452c feat(auth): add runtime toggle for user sign-ups and enforce signup policy
Introduce `enableUserSignups` runtime setting to allow administrators to control
whether new accounts can be created. Update environment variable and documentation
references to support this feature. UI elements for account creation are now
conditionally rendered based on this flag. Signup attempts are blocked server-side
when disabled, including email, OAuth, and anonymous upgrades.

Add `assertUserSignupAllowed` utility for consistent enforcement and corresponding
unit tests to verify policy behavior.
2026-05-14 12:47:05 -06:00
Richard R
fef3775ad4 style(ui): update modal gradient backgrounds and adjust settings header layout
Refine radial gradient backgrounds in Settings and DocumentSettings modals to use
CSS color-mix with accent variable for improved theming. Adjust Settings modal
header spacing and layout for better alignment and consistency.
2026-05-14 12:14:55 -06:00
Richard R
c3fb3af3d3 refactor(components): unify button styling using buttonClass utility across multiple components 2026-05-14 12:06:21 -06:00
Richard R
205d8f37e4 fix(tests): enhance onboarding modal dismissal logic in helpers 2026-05-14 12:06:12 -06:00
Richard R
1f548f71ea feat(audiobook): integrate settings coercion utility and migrate metadata handling
Refactor chapter API to use new coerceAudiobookGenerationSettings utility for
validating and migrating audiobook metadata. Add src/lib/server/audiobooks/settings.ts
with shared logic and introduce corresponding unit tests to ensure correct
settings migration and validation.
2026-05-14 11:34:16 -06:00
Richard R
0dee3c7b60 chore(release): bump version to 3.0.0
Update package version to 3.0.0 to reflect recent feature additions and
refactorings.
2026-05-14 11:05:47 -06:00
Richard R
0787d2e88a refactor(settings): extract changelog check logic to reusable utility
Move changelog version check effect from SettingsModal to a new
scheduleChangelogCheck utility for improved modularity and testability.
Add unit tests for changelog check scheduling behavior.
2026-05-14 10:58:32 -06:00
Richard R
beb854cb39 feat(preferences): add changelog version tracking and payload meta support
Integrate changelog version check on settings open, triggering the changelog
panel when a new app version is detected. Refactor user preferences storage to
support metadata via payload utilities, enabling version tracking and future
extensibility. Add API endpoint for changelog version check and related tests
for changelog and preferences payload logic.
2026-05-14 10:53:38 -06:00
Richard R
5efb29cda9 refactor(ui): restructure settings modal navigation and changelog panel logic
Update SettingsModal to use conditional rendering for the changelog panel,
displaying the main settings navigation only when the changelog is closed.
Refactor mobile and desktop navigation layout to improve maintainability.
Add missing ChevronRightIcon import for icon consistency.
2026-05-14 10:01:51 -06:00
Richard R
9e09f80067 feat(changelog): add changelog feed support and settings integration
Introduce changelog feed manifest support, including client and shared
utilities for fetching and parsing changelog data. Add a new
Settings modal panel for viewing changelog entries, with version
detection based on the current app version. Expose a configurable
changelog feed URL in both environment variables and admin panel
runtime settings. Update documentation and deployment workflow to
support changelog feed generation and consumption. Include unit tests
for changelog utilities.
2026-05-14 09:45:04 -06:00
Richard R
f300dfb01f Retry errored TTS segments with cooldown and retry-after 2026-05-14 08:31:40 -06:00
Richard R
df841210a3 Remove dead segment audio presign route and legacy helpers 2026-05-14 08:01:53 -06:00
Richard R
513128b802 Stream fallback segment audio and support byte-range requests 2026-05-14 07:55:48 -06:00
Richard R
da629b3ecd Enable private caching headers for segment audio routes 2026-05-13 23:44:13 -06:00
Richard R
ef2ee09064 Warm segment audio during preload before playback 2026-05-13 23:39:29 -06:00
Richard R
16bf52f889 Use direct signed segment audio URLs in ensure/manifest 2026-05-13 23:33:00 -06:00
Richard R
89391410da refactor(ui): replace loading text with skeleton placeholders across panels
Update multiple UI panels to use skeleton components instead of loading text or spinners.
Standardize loading states in AudiobookExportModal, AdminFeaturesPanel, AdminProvidersPanel,
DocumentSelectionModal, and SegmentsSidebar for improved visual consistency and user feedback.
2026-05-13 23:17:47 -06:00
Richard R
d5eebd3b11 chore(config): remove Deepinfra model gating and related admin setting
Eliminate the showAllDeepInfraModels runtime/admin config and all code paths
that allowed restricting Deepinfra's model catalog. Deepinfra now always
shows the full model list regardless of API key or environment variable.
Update documentation, environment examples, admin panels, runtime config,
provider catalog logic, and tests to reflect this change.
2026-05-13 18:42:40 -06:00
Richard R
e27d20419d refactor(ui): unify form primitives and simplify document selection modal
Move shared button, input, and listbox styles to a new formPrimitives module
used across admin panels, settings, and document settings. Update admin and
settings components to import these styles from the new location. Refactor
DocumentSelectionModal to use a direct files prop with loading and error
states, removing fetcher logic and internal fetch state. Add a custom hook
for library document queries. Clean up legacy admin/ui primitives and
standardize segmented controls and section layouts.
2026-05-13 18:42:31 -06:00
Richard R
a296d6fb13 feat(reader,settings): add react-query caching for library import and TTS segments
Integrate @tanstack/react-query in SettingsModal to cache and prefetch library
documents, improving import modal responsiveness. Refactor SegmentsSidebar to
use infiniteQuery for TTS segments manifest, enabling pagination, cache
invalidation, and consistent data fetching. Remove legacy fetch state logic in
favor of react-query state management.
2026-05-13 17:39:04 -06:00
Richard R
fdeeb60f7e refactor epub reader hooks and add unit coverage 2026-05-13 16:56:13 -06:00
Richard R
fd6a06e467 refactor(epub): move reader state to route-local hook 2026-05-13 14:22:24 -06:00
Richard R
2f067c0a90 refactor(pdf): move reader state to route-local hook 2026-05-13 14:08:47 -06:00
Richard R
41036e1341 refactor(html): move reader state to route-local hook 2026-05-13 13:40:17 -06:00
Richard R
3a0ed999ac refactor(reader,epub,providers): unify layout structure and improve resize handling
Adopt new layout files for app, epub, html, and pdf routes to standardize
Next.js nested layout structure. Refactor SegmentsSidebar to accept an explicit
epubBookRef prop, decoupling it from internal context. Enhance EPUB resize hook
to ignore initial baseline rect and avoid unnecessary TTS interruptions on load.
Simplify Providers by removing legacy context nesting and conditional logic.
2026-05-13 13:32:37 -06:00
Richard R
c8a35e505f feat(admin,providers,contexts): integrate react-query for admin panels and shared provider state
Adopt @tanstack/react-query for data fetching and cache management in admin
settings, provider management, shared provider hooks, and context providers.
Replace legacy useState/useEffect data loading with react-query's useQuery and
useMutation patterns. Refactor document, rate-limit, and shared provider
contexts to use query keys and cache invalidation for consistent state across
the app. Add QueryClientProvider to root providers. Update package.json to
include react-query dependency.
2026-05-13 12:40:58 -06:00
Richard R
913c6d5d76 refactor(tts,config,admin): update provider resolution logic and ordering for consistency
Remove fallbackProviderRef usage from settings and config context to streamline
provider resolution. Adjust normalizeLegacyProviderRef to retain 'default-openai'
unless a non-default fallback is explicitly provided. Update provider listing to
enforce deterministic ordering by updatedAt, createdAt, and slug. Expand unit tests
to verify provider ordering, legacy ref normalization, and selection logic.
2026-05-13 12:08:39 -06:00
Richard R
b4f4d43d6a feat(tts,config,types): migrate to providerRef/providerType model and add centralized TTS provider policy
Transition all TTS-related logic, types, and UI to use the new providerRef/providerType model in place of legacy ttsProvider fields. Introduce a centralized tts-provider-policy module to encapsulate provider/model capability checks, default value resolution, and compatibility logic. Update all API routes, contexts, hooks, components, and tests to use providerRef and providerType, ensuring consistent handling of built-in and shared TTS providers. Remove legacy defaultTtsModel config in favor of per-provider defaults and shared provider admin control. Add the showAllProviderModels runtime flag to restrict users to provider default models when desired.

BREAKING CHANGE: ttsProvider fields are replaced by providerRef/providerType throughout the codebase; defaultTtsModel config is removed in favor of per-provider defaults.
2026-05-13 10:11:05 -06:00
Richard R
dae97b2afc docs(admin,config,tts): document admin panel for runtime TTS provider and feature config
Add a dedicated Admin Panel documentation page detailing management of shared TTS providers and site features via the admin UI. Update all TTS provider guides, environment variable references, and deployment docs to clarify the new runtime configuration model: environment variables serve as first-boot seeds only, with ongoing management handled through the admin interface. Revise sidebars and cross-links to include the new admin panel docs and clarify the distinction between legacy env-based and admin-managed configuration.
2026-05-12 22:39:37 -06:00
Richard R
876ca7d774 feat(admin,config,ui,db): introduce runtime-editable site config and admin-managed TTS providers
Add database-backed runtime configuration for feature flags and TTS provider credentials, editable via a new admin UI panel. Replace static NEXT_PUBLIC_* and TTS provider env vars with admin-managed settings stored in the database and injected at SSR for client access. Implement admin-only panels for managing shared TTS provider credentials (with encrypted API keys) and live site feature flags. Add schema migrations, API routes, React contexts, and hooks for SSR-injected runtime config and live updates. Update client and server logic to resolve configuration from the database at runtime, enforcing admin restrictions and supporting migration from legacy env-based config.

BREAKING CHANGE: Feature flags and TTS provider credentials are now managed at runtime via the admin UI. Environment variables are only used for initial seeding and are ignored after first boot. Existing deployments must migrate configuration to the admin panel.
2026-05-12 22:39:24 -06:00
Richard R
c648d2bf6a refactor(api,tts,types): remove legacy /api/tts endpoint and update error handling
Eliminate the deprecated `/api/tts` route and its client usage, consolidating TTS generation logic under chapter and segment endpoints. Update client API utilities to remove direct TTS buffer fetching and improve error propagation for chapter generation failures. Extend TTS segment error metadata to include upstream rate limit and retry details, and standardize error typing in shared types. Adjust tests and helpers to reflect the new TTS mocking approach and endpoint structure. This streamlines TTS API surface and enhances error reporting consistency.
2026-05-12 18:40:46 -06:00
Richard R
d7e6a7b7ba fix(api,rate-limit): centralize daily quota exceeded response and update imports
Move daily quota exceeded problem response logic to a dedicated utility in `lib/server/rate-limit/problem-response`. Refactor API routes to use the new builder and remove duplicated formatting code. Update all relevant imports for consistency. This improves maintainability and ensures uniform error responses across TTS endpoints.
2026-05-12 18:24:39 -06:00
Richard R
08d9218ce4 refactor(client,tts,html): migrate EPUB TTS utilities to client and unify HTML progress logic
Move `epub-word-highlight`, `tts-epub-handoff`, and `tts-epub-preload` from shared to client/epub to clarify their usage and improve bundling. Update all imports to use new client paths. Extend TTS and sidebar logic to support HTML document progress tracking and restoration, introducing a new location format for HTML and consistent handling for non-EPUB readers.

BREAKING CHANGE: EPUB TTS utilities relocated to `lib/client/epub`; update import paths. HTML progress persistence now uses a new format incompatible with previous versions.
2026-05-12 17:51:17 -06:00
Richard R
8488ad37e1 refactor(tts,db,manifest): split tts_segments into normalized entry and variant tables
Redesign TTS segment storage by decomposing the `tts_segments` table into two normalized tables: `tts_segment_entries` for unique segment identity and locator projection, and `tts_segment_variants` for per-settings audio variants. Update schema, queries, and API routes to use the new structure, including manifest pagination and segment resolution logic. Refactor segment locator handling to use stable projections and manifest cursors. Migrate tests and data cleanup utilities to support the new model.

BREAKING CHANGE: Database schema for TTS segments is now split into entry and variant tables; all code and consumers must use the new structure. Persisted data and APIs relying on the old `tts_segments` table are incompatible.
2026-05-12 17:44:57 -06:00
Richard R
f9adba791d feat(tts,epub,db): enforce stable EPUB segment locators and add segmentKey identity
Implement strict validation for EPUB TTS segment locators, requiring stable spine coordinates (`spineHref`, `spineIndex`, `charOffset`) and rejecting legacy CFI-only locators. Introduce `segmentKey` as a normalized identity derived from segment text, used for robust merging and deduplication of synthesized and persisted segments. Update database schemas, API routes, and manifest logic to support the new locator format and segmentKey. Add helpers for locator resolution and segmentKey construction, with comprehensive unit tests for identity, normalization, and manifest grouping.

BREAKING CHANGE: Persisted EPUB TTS segment locators must now include stable spine coordinates; legacy CFI-only locators are no longer accepted. Consumers must handle the new `segmentKey` field for segment identity and merging.
2026-05-12 16:18:36 -06:00
Richard R
e8d4c11434 refactor(tts,epub,locator): introduce canonical segment keys and locator comparison utilities
Refactor TTS segment identification to support canonical segment keys, decoupling segment IDs from locator and index for more robust deduplication and handoff. Add `segmentKey` to segment types, manifest, and ensure route. Implement natural sorting for EPUB CFI locations and segment locators using new comparison utilities. Update sidebar and manifest helpers to use locator-aware sorting. Add foundational modules for EPUB word highlighting, TTS segment planning, and EPUB handoff logic. Expand types and tests to cover new canonical segment and locator behaviors.

BREAKING CHANGE: TTS segment IDs now support canonical segment keys; APIs and manifest consumers must handle `segmentKey` and updated locator comparison logic.
2026-05-11 05:17:41 -06:00
Richard R
265e1cb56a feat(tts,config,ui): add advanced segment preloading and block length controls
Introduce configurable segment preloading depth, sentence lookahead, and TTS segment max block length for both PDF and EPUB readers. Add new settings to user preferences, config context, and document settings UI. Refactor TTS segment splitting logic to support per-user max block length and propagate these options through PDF/EPUB adapters and TTS segment generation. Update API, manifest pagination, and segment deduplication for improved performance and correctness. Add tests for new NLP options and manifest logic.

BREAKING CHANGE: TTS segment splitting and manifest APIs now require max block length and preloading parameters; user config schema updated.
2026-05-07 12:26:08 -06:00
Richard R
3862d0b0ad feat(tts): add leading context support for smart sentence splitting
Introduce logic to provide previous text context for TTS sentence splitting,
enabling more accurate prefix stripping and improved handling of segment
boundaries. EPUB and PDF providers now supply previous segment text, and
TTSContext leverages this context to avoid redundant continuation logic.
This enhances TTS playback continuity and reduces repeated or truncated
sentences across segment transitions.
2026-05-04 20:59:14 -06:00
Richard R
45ae2f5000 fix(tts): improve segment grouping and sidebar selection for locator variants
Update segment manifest API to group segments by both index and locator
attributes, ensuring distinct segment variants are not merged incorrectly.
Adjust SegmentsSidebar logic to select the most relevant segment per index
based on locator match and recency, improving sidebar accuracy when
multiple locator variants exist for a segment.
2026-05-04 20:08:51 -06:00
Richard R
4cb482c3f2 refactor(tts): enhance pause handling and centralize TTS test helpers
Update TTSContext to track manual pause epochs, ensuring explicit user
pauses are respected during async sentence splitting and playback state
restoration. Adjust TTSPlayer button disabling logic to allow pausing
while processing. Move and improve pauseTTSAndVerify helper for robust
cross-browser TTS pause assertions, replacing local test implementations.
Update expectDocumentListed to use polling for document link visibility.
2026-05-04 19:38:23 -06:00
Richard R
62103ea752 style(ui): update aria-labels for clarity and consistency in sidebar controls
Standardize aria-labels in DocumentHeaderMenu and ReaderSidebarShell to use
"Hide" and "Close" phrasing for improved accessibility and user experience.
2026-05-04 15:23:48 -06:00
Richard R
71e1472650 feat(reader,db): add segments sidebar and manifest API; update TTS segment schema
Introduce SegmentsSidebar component and supporting reader UI for segment-level
navigation and inspection. Add TTS segments manifest and clear API endpoints to
enable efficient retrieval and management of segment variants. Update TTS
segment schema to include settings_hash for improved lookup and indexing,
and adjust indexes to use settings_hash instead of settings_json. Update
contexts and types to support segment variant display and interaction.

This enhances document navigation and TTS segment management for end users.
2026-05-04 15:02:05 -06:00
Richard R
71cef7b540 refactor(db): migrate tts_segments to settings_json and update indexes
Switch tts_segments schema to use settings_json (jsonb for Postgres, text for SQLite)
instead of settings_hash. Update related indexes and migration SQL. Regenerate
Drizzle snapshots to reflect the new structure. This improves flexibility for
storing TTS settings and enables more robust segment lookups.
2026-05-04 13:22:58 -06:00
Richard R
222d221e81 fix(tts): prevent stale playback by tracking playback run state in TTS context
Introduce `playbackRunIdRef` and `invalidatePlaybackRun` to ensure that playback
state is correctly reset on abort and content changes. Update abort and navigation
handlers to call `invalidatePlaybackRun`, preventing duplicate or stale playback
runs, especially in scenarios with rapid restarts or browser-specific audio event
quirks. This addresses issues where playback could become stuck or behave
unexpectedly due to outdated in-flight guards.
2026-05-04 12:26:29 -06:00
Richard R
7d7e933323 refactor(tts): replace manifest and audio endpoints with presigned and fallback URLs
Remove legacy TTS segment manifest and audio endpoints in favor of new presigned and fallback audio URL approach. Update segment manifest structure to include `audioPresignUrl` and `audioFallbackUrl` fields, deprecating the previous `audioUrl`. Refactor segment ensure logic and TTS context to utilize the new playback source model. Add server utilities for generating presigned S3 URLs and introduce new API routes for presign and fallback audio retrieval. Clean up obsolete client API functions and types to reflect the new manifest format.
2026-05-04 12:04:24 -06:00
Richard R
0a7d392d77 docs(tts,storage,dev): update documentation for TTS segment storage, schema, and local setup
Expand and clarify documentation for TTS segment storage, including key layout, cleanup, and CLI access in object storage. Add details on the `tts_segments` schema and migration process. Revise local development and Docker quick start guides with improved prerequisites, platform-specific setup instructions, and updated TTS provider references. Document new usage of `AUTH_SECRET` for TTS segment text fingerprinting.
2026-05-04 11:29:02 -06:00
Richard R
9999cafec1 feat(tts): add TTS segments API, schema, storage, and manifest endpoints
Introduce TTS segment support with new database tables and migrations for both Postgres and SQLite. Implement segment ensure, audio, and manifest API endpoints, segment storage in S3, and segment authorization logic. Update TTS context and client API to use segment-based synthesis and retrieval. Add server-side helpers for segment ID, settings hash, locator normalization, and audio probing. Extend user data cleanup and teardown logic to remove TTS segment objects. Add unit tests for segment helpers.

BREAKING CHANGE: TTS synthesis now uses segment-based storage and APIs; database migrations required.
2026-05-04 10:18:56 -06:00
github-actions[bot]
f2989efe4d docs: snapshot v2.2.0 2026-04-18 17:40:34 +00:00
Richard R
6fdcee4f25 chore: update version to 2.2.0 in package.json
Some checks failed
Create and publish Docker images / prepare (push) Has been cancelled
Version Docs on Tag / version-docs (push) Has been cancelled
Create and publish Docker images / build (amd64, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (arm64, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / merge (push) Has been cancelled
2026-04-18 11:37:36 -06:00
Richard R
21030f28eb refactor: update document section title from 'Local Documents' to 'Your Documents' 2026-04-16 20:13:01 -06:00
Richard R
5ea9aa7474 docs: add warning about SeaweedFS 4.19 upload issues and pinning to 4.18
Document observed S3 `InternalError` regressions with SeaweedFS 4.19 in
object storage, local development, and Docker quick start guides. Clarify that
version 4.18 is pinned in CI and Docker builds for reliability while
compatibility is investigated.
2026-04-16 17:13:24 -06:00
Richard R
8349ef7ae8 build(deps): pin SeaweedFS Docker image to 4.18 for stability
Update CI workflow and Dockerfile to use chrislusf/seaweedfs:4.18 instead of
latest. This avoids upload regressions observed with version 4.19 and ensures
consistent builds.
2026-04-16 16:32:48 -06:00
Richard R
0c7a845952 feat(tts): add pauseActiveHowl helper and improve playback pause/resume handling
Introduce pauseActiveHowl to centralize audio pause logic, ensuring consistent
cleanup and media session state updates. Update pause and togglePlay to use the
new helper, enabling accurate resume of paused audio and improved error
handling. Enhance Howl onpause event to synchronize playback state and clear
timeouts.
2026-04-16 15:47:39 -06:00
Richard R
0db53fbd4e refactor(tts): extract voice resolution logic to server module and update imports
Move resolveVoices and resolveReplicateVoiceInputKey to a new server-side
voice-resolution module to improve separation of concerns and reduce shared
bundle size. Update all imports and related tests to use the new module.
Remove unused LRUMap and related caches from shared catalog. Update Replicate
cooldown logic to use per-scope LRU cache for improved concurrency handling.
2026-04-16 15:46:55 -06:00
Richard R
67097109ed
Merge pull request #87 from richardr1126/replicate
replicate
2026-04-16 13:13:03 -06:00
Richard R
3db3fd19ce refactor(tts): centralize upstream response helpers and improve settings validation
Move getUpstreamStatus and getUpstreamRetryAfterSeconds to a shared utility
module for consistent upstream error handling across TTS endpoints. Strengthen
audiobook chapter API by introducing runtime validation for incoming settings
payloads, ensuring type safety and error reporting for malformed requests.
Replace in-memory Map caches with LRUMap for Replicate voice and schema
lookups, improving memory management and eviction logic.
2026-04-16 13:07:31 -06:00
Richard R
893f74f038 refactor(tts): update Replicate model defaults and voice resolution
Align default Replicate model to the versioned Kokoro model across UI, docs,
and server logic. Refactor TTS settings merging for audiobooks to ensure
consistent normalization. Improve Retry-After header handling for upstream
rate limits. Expand Replicate voice resolution to use model schemas when
available, with test coverage for custom and built-in models.

Update documentation to reflect new Replicate defaults, model selection,
and configuration guidance.
2026-04-16 12:52:08 -06:00
Richard R
d0c0ab7420 feat(replicate): add support for custom Replicate model selection 2026-04-16 11:24:19 -06:00
Richard R
9da9232d39 feat: add support for Replicate TTS provider and models
- Updated environment variables documentation to include Replicate as a TTS provider option.
- Added Replicate to the sidebar for TTS provider guides.
- Included Replicate as a dependency in package.json and pnpm-lock.yaml.
- Enhanced audiobook chapter generation to normalize native speed settings based on the TTS provider.
- Improved error handling in TTS API routes to provide retry information for rate-limited responses.
- Updated AudiobookExportModal to reflect native speed support for Replicate models.
- Modified SettingsModal to set default model for Replicate.
- Enhanced SpeedControl component to conditionally render native speed controls based on provider support.
- Updated TTSContext to utilize effective native speed for TTS requests.
- Implemented Replicate request handling in the TTS generation logic.
- Added new documentation for configuring Replicate as a TTS provider.
2026-04-16 11:20:38 -06:00
github-actions[bot]
6606b0ee4a docs: snapshot v2.1.2 2026-04-07 07:17:37 +00:00
Richard R
8ea4beac9c chore(config): bump package version to 2.1.2
Some checks failed
Create and publish Docker images / prepare (push) Has been cancelled
Version Docs on Tag / version-docs (push) Has been cancelled
Create and publish Docker images / build (amd64, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (arm64, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / merge (push) Has been cancelled
2026-04-07 01:15:39 -06:00
Richard R
b7cc2436d2 feat(theme): add persistent custom theme with live color editing
Add support for a persistent "custom" theme that users can edit and preview in settings.

- add CustomThemeColors type and storage helpers (getCustomThemeColors, setCustomThemeColors)
- apply custom CSS variables at runtime and provide applyCustomColors to live-update variables
- include fallback CSS for html.custom to avoid flash before hydration
- add ColorPicker component with curated swatches for each color role
- extend SettingsModal to support editing, previewing, and persisting custom theme colors; exclude 'custom' from regular theme lists and read custom colors when rendering previews
- tighten theme detection logic to treat custom background luminance as light/dark when needed

This enables users to define, persist, and see live updates for custom color palettes.
2026-04-07 01:09:08 -06:00
Richard R
f9237f9270 feat(tts): add TTS_UPSTREAM_TIMEOUT_MS for configurable request timeout 2026-04-06 17:48:56 -06:00
github-actions[bot]
4505684108 docs: snapshot v2.1.1 2026-04-06 17:54:36 +00:00
Richard R
6d33de83c2
Merge pull request #84 from richardr1126/clean-code-refactor
Some checks failed
Version Docs on Tag / version-docs (push) Has been cancelled
Create and publish Docker images / prepare (push) Has been cancelled
Create and publish Docker images / build (amd64, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (arm64, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / merge (push) Has been cancelled
Centralized TTS provider/catalog and resolver utilities.
New audiobook generation pipeline with EPUB and PDF source adapters.
TTS settings view model and config-update helpers for synced preferences.
Improvements

Generalized TTS instruction support and provider/model-aware voice defaults.
Audiobook export delegated to pipeline for consistent progress, retry, and cancel handling.
UI test IDs added to privacy and migration modals.
Tests

New unit tests for TTS catalog and config helpers; export readiness and onboarding dismissal added to tests.
Chores

Package version bumped to 2.1.1.
2026-04-06 11:48:23 -06:00
Richard R
bf139d9a67 refactor(tts): ensure timeout cleanup and validate remote voice payloads
Make voice-fetching more robust by always clearing the request timeout and stricter type-checking of remote data.

- move clearTimeout into finally blocks for Deepinfra and custom OpenAI voice fetches to guarantee the timeout is cleared on success, failure, or abort
- validate that custom endpoint returns an array of strings before accepting voices; otherwise fall back to defaults

This reduces potential timer leaks and avoids accepting malformed voice lists from custom endpoints.
2026-04-06 11:41:42 -06:00
Richard R
09944ec4e4 refactor(audiobooks,tts,config): tighten abort handling, progress accounting, equality checks, and voice fetch timeout
Improve robustness across audiobook pipeline, TTS catalog, and preferences comparison.

- audiobooks: centralize abort creation with createAudiobookAbortError and replace repeated ad-hoc throws with it; avoid reprocessing non-completed chapters when collecting existing indices; update progress accounting by advancing processedLength and emitting onProgress after saving a chapter.
- tts: add a 10s AbortController timeout when fetching custom OpenAI voices, wire the signal to fetch, and ensure the timeout is cleared on success or failure; fall back cleanly when endpoint doesn't support voices.
- config: harden deepEqual by checking mismatched array vs object shapes early to avoid incorrect comparisons.

No user-visible behavior changes besides improved cancellation, progress reporting, and more resilient remote voice discovery.
2026-04-06 11:28:18 -06:00
Richard R
7d9d8de967 test(ui): stabilize onboarding/modal dismissal in e2e helpers
Refactor test helpers to reliably dismiss onboarding and migration modals during
UI flows. Replace ad-hoc Settings-only dismissal with a unified dismissOnboardingModals
routine that targets privacy, migration, and settings dialogs via testids,
interacts with visible controls (checkbox, continue/skip/save buttons) and waits
for dialogs to hide. Increase resilience by repeating checks, adding short
settle delays, and tightening timeouts.

Also add data-testid attributes to PrivacyModal, SettingsModal and
DexieMigrationModal to make the selectors deterministic and reduce flakiness in
export/upload tests.

No production behavior changes.
2026-04-06 11:16:54 -06:00
Richard R
dbd202569f undo(tests): Undo bad changes to tests/helpers.ts suggested by CodeRabbit.ai 2026-04-06 10:34:59 -06:00
Richard R
97dba6d987 refactor(tts,audiobook,tests): centralize TTS catalog, harden epub & UI flows
- Move TTS provider resolution and voice-fetch logic into shared catalog and make
  remote fetches more resilient (timeouts, abort handling, baseUrl normalization,
  treat explicit empty custom-openai responses as valid).
- Add continuation guard to TTS feature set and update API voices endpoint error
  logging to better surface auth/session failures.
- Extract and stabilize ebook pipeline and epub adapter:
  - Cache prepared EPUB chapters to avoid repeated parsing and handle missing/empty
    chapter titles safely.
  - Fail early when created chapter lacks a bookId.
- Improve config/preferences handling:
  - Replace JSON stringify comparisons with a deepEqual utility to compare defaults.
  - Tighten ConfigContext effect dependencies and consolidate synced preference
    queuing logic to include additional keys.
- Harden end-to-end test helpers and unit tests:
  - Make settings dialog dismissal deterministic (use Enter/Escape and explicit
    visibility/enabled checks) and simplify upload retry behavior.
  - Add unit test to ensure custom-openai empty voice lists are preserved.

Why: Reduce runtime and test flakiness, centralize TTS logic for easier maintenance,
and make EPUB processing and preference diffing more robust. Breaking changes: none.
2026-04-06 10:28:00 -06:00
Richard R
e373d268d2 chore(package): bump version to v2.1.1 2026-04-06 10:00:25 -06:00
Richard R
52e512f17d test(ui,tests): stabilize audiobook export tests and harden settings modal interactions
- Add waitForBackendDownloadReady helper to tests/export.spec.ts to wait for backend
  chapter metadata and enabled full-download UI before attempting combines/downloads.
  Use it in two export tests to reduce flakiness related to eventual backend storage visibility.
- Improve uploadAndDisplay helper in tests/helpers.ts to retry link click when transient
  "intercepts pointer events" errors occur, ensuring test clicks succeed under flaky UI conditions.
  Add dismissSettingsModalIfVisible to detect and close the Settings dialog before interacting
  with page links (click Save or press Escape, wait for dialog to hide).
- Update SettingsModal.tsx useEffect dependencies to avoid referencing checkFirstVist in the
  local state sync effect and add separate effect for checkFirstVist for clearer lifecycle.

Why: Reduce end-to-end test flakiness by waiting for backend readiness and making UI interactions
more robust; simplify SettingsModal effect dependencies to prevent unnecessary re-runs. Breaking changes: none.
2026-04-06 09:59:37 -06:00
Richard R
d6ae2baa6f refactor(tts,config,audiobook): centralize TTS provider catalog and extract audiobook pipeline
- Introduce a shared tts-provider-catalog to centralize provider/model/voice defaults
  and feature flags (supportsTtsInstructions, providerSupportsCustomModel, resolveProviderModels,
  getDefaultVoices, etc.). Replace ad-hoc provider logic across server and client with catalog calls.
- Replace inline voice/model UI logic in SettingsModal, AudiobookExportModal, TTSContext,
  generate server code, and voice hook to use catalog helpers (resolveTtsSettingsViewModel,
  supportsTtsInstructions, getDefaultVoices).
- Extract audiobook generation responsibilities into a small client-side pipeline:
  - Add createEpubAudiobookSourceAdapter and createPdfAudiobookSourceAdapter adapter modules
    to prepare chapters from EPUB/PDF sources.
  - Add runAudiobookGeneration and regenerateAudiobookChapter pipeline functions to handle
    chapter preparation, progress tracking, request header construction, and retry/abort behavior.
  - Wire adapters/pipeline into EPUBContext and PDFContext to simplify and centralize audiobook flow.
- Move config preference helpers out of ConfigContext into dedicated client modules:
  - buildSyncedPreferencePatch (lib/client/config/preferences.ts)
  - applyConfigUpdate and getVoicePreferenceKey (lib/client/config/updates.ts)
  - Use these helpers in ConfigContext to keep update logic small and testable.
- Add TTS settings view model resolver (lib/client/settings/tts-settings.ts) and tests
  for the tts-provider catalog and config helpers (tests/unit/tts-provider-catalog.spec.ts).
- Minor usages updated: AudiobookExportModal, SettingsModal, useVoiceManagement, TTSContext,
  server tts generation, and various context files to consume the new modules.

Why: Reduce duplication, improve testability, and separate concerns for provider metadata,
UI view-model logic, and audiobook generation flow. Breaking changes: none.
2026-04-04 17:53:57 -06:00
Richard R
4a985b3bf1 feat(tts): add continuation loop guard and block fingerprinting
Introduce safeguards for continuation handling in TTS flow to prevent looping and over-carrying text. Add constants to limit carried characters and words, and loop-guard thresholds. Implement normalizeBlockFingerprint to create compact fingerprints of page-first blocks and store them in pageFirstBlockFingerprintRef. Use fingerprint comparison and progress threshold to skip to next page when a loop is detected. Cap continuation slices by characters and words and ensure carried text matches the returned addition.

test(ui): make settings save button click more robust

In tests/helpers.ts wait for the Settings Save button to become visible (with a short timeout) before attempting to open Settings, and click the Settings button with force to avoid intermittent pointer-interception failures. Improve stability of setupTest by handling enter transitions and overlays.
2026-04-04 14:50:34 -06:00
Richard R
b600b67252
Merge pull request #83 from othrayte/pg-migration-error-user-is-reserved
Fix SQL syntax for user table insert statement
2026-04-04 12:30:32 -06:00
Adrian Cowan
d2f1fe65dd
Fix SQL syntax for user table insert statement
user is a reserved keyword in postgres
2026-04-04 18:31:09 +11:00
Richard R
e8b5452863 fix(config): preserve voice selection when resetting TTS provider settings
Add a test to verify selected voice persists instead of resetting to default.
Also refactors export tests to be UI-driven rather than backend-dependent.
2026-03-19 13:41:37 -06:00
github-actions[bot]
0480a52c83 docs: snapshot v2.1.0 2026-03-19 18:40:29 +00:00
Richard R
617f459bb7 chore(package): update version to v2.1.0
Some checks failed
Create and publish Docker images / prepare (push) Has been cancelled
Version Docs on Tag / version-docs (push) Has been cancelled
Create and publish Docker images / build (amd64, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (arm64, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / merge (push) Has been cancelled
2026-03-19 12:37:48 -06:00
Richard R
ab9968f75b feat(config): implement reset logic for TTS provider settings based on feature flag 2026-03-19 12:35:01 -06:00
Richard R
b2a623ba2a feat(ui): add feature flag for TTS Providers tab and update related documentation 2026-03-19 12:25:42 -06:00
Richard R
e41a5ba481 refactor(audiobooks): Refactor TTS generation and audiobook chapter creation
- Removed unused imports and types from TTS route.
- Consolidated TTS buffer generation logic into a new module.
- Implemented caching for TTS audio buffers using LRU cache.
- Updated TTS request handling to support instructions for specific models.
- Refactored audiobook chapter creation to use new TTS generation logic.
- Simplified error handling and response management in TTS API.
- Enhanced client-side logic to manage TTS requests and retries.
- Updated types to reflect changes in TTS request payload structure.
2026-03-19 11:54:33 -06:00
Richard R
df0e4341ff fix(ui): update SettingsModal and test helpers for improved onboarding flow in non-auth mode 2026-03-19 11:33:20 -06:00
Richard R
b6852da2a2 feat(ui): enhance AudiobookExportModal and ConfirmDialog with improved layout and styling; update VoicesControlBase for better dropdown handling 2026-03-19 11:16:24 -06:00
Richard R
3c37b0c59e docs: refactor TTS provider setup documentation 2026-03-19 10:26:05 -06:00
Richard R
1deb8c21f7 feat(ui): add five new light themes and redesign settings modal
Add five new light theme options (lavender, rose, sand, sky, slate) with dedicated color palettes. Redesign Settings modal with sidebar navigation replacing tabs, add visual theme color selector showing preview swatches, and introduce new icons (KeyIcon, PaletteIcon, UserIcon, DocumentIcon). Also update PrivacyModal to render conditionally based on authEnabled and add privacy policy link. Update tests to reflect new UI structure.
2026-03-19 10:24:16 -06:00
Richard R
fe05de9fec fix(issue #81): Add synchronous guard to prevent duplicate audio playback in TTSProvider 2026-03-18 01:57:01 -06:00
Richard R
cf7cac6ebb feat: Add SeaweedFS license retrieval and third-party license report generation in Dockerfile 2026-02-28 14:56:49 -07:00
Richard R
49c5b794ea feat: Implement sequential lookahead preloading for multiple upcoming TTS sentences. 2026-02-22 03:31:03 -07:00
Richard R
6e442e8dba feat: Explicitly set and default TTS response format to MP3. 2026-02-22 03:21:27 -07:00
Richard R
f5cd3b262a feat: Refine TTS retry logic to handle client-side transport failures and enhance voice management within the TTS context. 2026-02-22 03:08:16 -07:00
Richard R
cbd8aa6240 feat: Optimize audiobook export by attempting audio stream copying, adding duration validation, refining chapter duration calculation, and synchronizing export settings with playback controls. 2026-02-21 22:57:59 -07:00
Richard R
b2d9ab5395 chore: bump version to v2.0.1 in package.json 2026-02-21 20:28:48 -07:00
github-actions[bot]
22541f2302 docs: snapshot v2.0.1 2026-02-22 03:21:45 +00:00
Richard R
7810a6329a feat: add KittenTTS-FastAPI support and update documentation
Some checks failed
Create and publish Docker images / prepare (push) Has been cancelled
Version Docs on Tag / version-docs (push) Has been cancelled
Create and publish Docker images / build (amd64, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (arm64, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / merge (push) Has been cancelled
2026-02-21 20:19:00 -07:00
Richard R
230b614b90 feat: enhance docs deployment workflow with conditional execution for workflow runs 2026-02-19 13:02:18 -07:00
github-actions[bot]
78fa6c3940 docs: snapshot v2.0.0 2026-02-19 19:55:17 +00:00
Richard R
092fc3ae0d feat: enhance workflow for versioning documentation with optional inputs and improved commit handling 2026-02-19 12:53:01 -07:00
Richard R
bc4c888866 feat: implement US-only availability check for production instance in middleware and privacy page
Some checks failed
Create and publish Docker images / prepare (push) Has been cancelled
Version Docs on Tag / version-docs (push) Has been cancelled
Create and publish Docker images / build (amd64, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (arm64, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / merge (push) Has been cancelled
2026-02-19 12:28:30 -07:00
Richard R
ffcd120f68 fix: replace getS3Client with getS3ProxyClient in audiobook functions 2026-02-19 11:46:23 -07:00
Richard R
cfd5a7679c fix: update Docker run commands and improve local network setup instructions in documentation 2026-02-19 11:34:19 -07:00
Richard R
af5373bf26 feat: update links in README and add local network setup instructions in Docker Quick Start 2026-02-19 11:25:45 -07:00
Richard R
d4bad59007 feat: add loopback support for S3 endpoint in storage migration 2026-02-19 11:20:51 -07:00
Richard R
5ffacb251f fix: remove legacy image name from Docker outputs in workflow 2026-02-19 11:05:42 -07:00
Richard R
fe567782e8 feat: update version from v1.3.0 to v2.0.0 in package.json 2026-02-19 11:00:40 -07:00
Richard R
7f9659fd99 chore: Remove unnecessary Pages configuration step from docs deployment workflow 2026-02-19 10:32:28 -07:00
Richard R
3555561d3b
Merge pull request #80 from richardr1126/better-auth
Major overhaul: implement auth, sqldb for metadata, s3-like storage, and lessen reliance on fs

New Features

Added user authentication system with sign-in, sign-up, and account management
Added TTS rate-limiting with per-user character limits and daily resets
Added user data export functionality
Added support for PostgreSQL database and external S3-compatible object storage
Documentation

Launched comprehensive documentation site with deployment guides, configuration references, and TTS provider integration guides
Chores

Rebranded project from "OpenReader-WebUI" to "OpenReader"
2026-02-19 10:28:51 -07:00
Richard R
7482a4265e feat: Enhance authentication handling by adding error logging and improving session checks in the API and ClaimDataModal component 2026-02-19 02:13:43 -07:00
Richard R
0fbd9d5c38 feat: Update Docker image name to openreader while maintaining openreader-webui as a legacy alias and updating documentation. 2026-02-18 02:54:01 -07:00
Richard R
a0cc189a0f refactor: rename project from OpenReader WebUI to OpenReader across all relevant files. 2026-02-18 02:49:47 -07:00
Richard R
f7af557073 feat: Implement S3 proxy client, enhance session handling, and improve Drizzle migration script. 2026-02-18 02:39:54 -07:00
Richard R
30ad62aa41 feat: Remove explicit background state and implementing text prefetching for smoother page/section transitions. 2026-02-18 00:05:41 -07:00
Richard R
3ede092adc feat: Make word highlighting enabled by default by inverting its environment variable logic, updating documentation, and removing NEXT_PUBLIC_NODE_ENV from Playwright tests. 2026-02-17 22:12:11 -07:00
Richard R
021533223b refactor: Standardize database timestamp fields to bigint epoch milliseconds and update schema migrations. 2026-02-17 21:00:36 -07:00
Richard R
c2188b476f feat: Refactor server-side document, audiobook, and authentication logic, introduce new client-side APIs and caching, and update related tests. 2026-02-17 19:30:12 -07:00
Richard R
8e1b6b4009 refactor: Move document-related components into a dedicated documents subdirectory and update imports. 2026-02-17 18:29:54 -07:00
Richard R
8f5e87ed4a feat: Replace NEXT_PUBLIC_NODE_ENV with explicit feature flags for granular control over DOCX conversion, destructive actions, TTS defaults, and DeepInfra model visibility. 2026-02-17 18:26:03 -07:00
Richard R
eeeceb8e54 feat(privacy): implement data export and encryption with updated policy
Add ZIP-based user data download endpoint, enable AES-256 server-side S3 encryption for all stored documents and audiobooks. Rewrite privacy policy with detailed CCPA categories, service provider disclosures (Vercel/Neon/Railway), and user rights sections. Replace privacy modal with checkbox agreement flow.
2026-02-17 17:16:58 -07:00
Richard R
8dc836cbb7 feat(auth): implement route protection and security headers
Add Next.js middleware to handle session-based access control for protected
routes while maintaining access to public paths. Configure comprehensive
security headers including CSP, HSTS, and frame options in next.config.ts.

Also, reduce session cookie cache duration to 5 minutes to ensure frequent
revalidation against the database and fix a navigation issue in the EPUB
context when jumping to specific CFI locations.
2026-02-16 19:06:24 -07:00
Richard R
f58a2b690d refactor(db): remove user_id foreign key from user_tts_chars
Remove the foreign key constraint referencing the user table in both
Postgres and SQLite schemas. This decouples the usage tracking from
the primary user table.
2026-02-16 17:29:26 -07:00
Richard R
fc24b31cc3 perf: lazy load heavy dependencies and db drivers
Move database drivers and heavy server-side modules to be loaded
lazily via require() and dynamic import(). This reduces the initial
memory footprint and improves cold start performance for serverless
functions by avoiding loading unused dependencies on every request.
2026-02-16 16:10:03 -07:00
Richard R
9931c7d0d8 fix(api): add retry for document blob head checks
Implement a retry mechanism for document blob HEAD requests to handle
S3 read-after-write propagation delays. This prevents 409 errors
when the client calls the endpoint immediately after performing a
direct upload to S3 via a presigned URL.
2026-02-16 15:51:01 -07:00
Richard R
596c04d4d3 feat(preview): improve failure recovery and configuration
Update the client-side preview status logic to gracefully handle failed
states and return fallback URLs. Add PDF worker tracing to the build
configuration for preview-related API endpoints and include server-side
error logging.
2026-02-16 15:46:30 -07:00
Richard R
deaacbd6f0 feat(auth): add conditional GitHub authentication support
Implement a check for GitHub OAuth credentials to conditionally render
the GitHub sign-in button. This ensures the UI accurately reflects
available authentication methods based on environment configuration.

- Add `isGithubAuthEnabled` utility to verify server-side credentials
- Propagate GitHub auth status through context providers to the UI
- Conditionally display the GitHub sign-in button in the sign-in page
- Update documentation and examples to use hex encoding for `AUTH_SECRET`
- Add unit tests for the new GitHub authentication configuration logic
2026-02-16 14:21:25 -07:00
Richard R
94d24c4687 test(export): increase timeout for resume test
Increase timeout to 60 seconds for the audiobook resume test to prevent flakiness when processing PDF exports.
2026-02-16 10:19:54 -07:00
Richard R
80b2ae2676 test: improve test reliability and adjust timeouts
Increase timeouts for export and navigation tests to 60 seconds to
prevent flakiness. Remove brittle assertions checking for visible
overlays in setup helpers to ensure more stable test execution.
2026-02-16 01:50:43 -07:00
Richard R
fa406a8cb2 refactor(deps): remove ffprobe-static and fix playwright flakiness
Replace ffprobe-static with ffmpeg-static for metadata extraction in
migration scripts. This removes the requirement for ffprobe binaries
across Docker, CI, and Vercel environments, simplifying the dependency
graph.
2026-02-16 01:41:54 -07:00
Richard R
da0fce708f refactor: improve database reliability and clean up code
Enable WAL mode and set a busy timeout for SQLite to better handle
concurrent access and prevent 500 errors. Add error handling to
async calls in UI components and remove unused imports across several
files.
2026-02-15 15:44:20 -07:00
Richard R
4d13b5b821 refactor: improve async state and session reliability
Implement AbortController in ConfigContext, PDFContext, and preference sync to prevent race conditions during session changes or rapid navigation. Refactor document preview routes into shared utilities and add LRU caching for text previews to optimize memory. Update documentation for Better Auth schema ownership and migration workflows. Fix issues with Postgres system user seeding and better-sqlite3 configuration.
2026-02-15 15:13:25 -07:00
Richard R
8e4f0b0ca9 build(config): remove better-sqlite3 from serverExternalPackages 2026-02-15 14:02:42 -07:00
Richard R
8cb2929116 feat(db): add system user seeding and storage cleanup
- Implement ensureSystemUserExists to handle foreign key constraints for
  unclaimed data when authentication is disabled.
- Add missing foreign key relationship between audiobook chapters and
  audiobooks with cascade delete.
- Integrate a beforeDelete hook in auth config to trigger storage
  cleanup when a user is removed.
- Regenerate initial migrations for both PostgreSQL and SQLite to
  reflect schema updates.
2026-02-15 14:00:35 -07:00
Richard R
d3c7831f70 fix: address hook violations and improve resource reliability
- Refactor useAuthSession to call useSession unconditionally, adhering to React Rules of Hooks by using a stub client when auth is disabled.
- Update migration script to insert a system user to satisfy foreign key constraints during database transitions.
- Implement AbortController and timeouts in entrypoint endpoint polling to prevent hanging fetch requests.
- Use useMemo in PDFViewer to provide a stable file object, preventing unnecessary re-renders and warnings.
- Enhance state cleanup in EPUBContext and AuthLoader to handle missing metadata or disallowed sessions gracefully.
2026-02-15 13:31:06 -07:00
Richard R
3eebfe6c6c ci(config): remove vercel configuration file 2026-02-15 13:06:34 -07:00
Richard R
bedbbb026a feat(db): add foreign keys and cascade deletes
Enforce referential integrity by linking application tables to the auth user table across both PostgreSQL and SQLite schemas.

- Add foreign key references with ON DELETE CASCADE to all user-owned tables.
- Re-generate initial migrations for Postgres and SQLite providers.
- Update migration script to ensure the SQLite database directory exists.
- Remove explicit SSL configuration from the PostgreSQL connection pool.
2026-02-15 13:02:42 -07:00
Richard R
bb853acaf6 Refactor authentication schema and database handling
- Updated SQLite journal metadata with new tag and timestamp.
- Refactored account deletion API to utilize Better Auth's built-in user deletion method for cascading cleanup.
- Enhanced database initialization to include authentication schemas for both SQLite and Postgres.
- Removed user, session, account, and verification tables from the Drizzle schema, as they are now managed by Better Auth.
- Created separate schema files for authentication in both Postgres and SQLite.
- Added SQL migration scripts for creating necessary tables in both database systems.
2026-02-15 12:49:23 -07:00
Richard R
f9366dbfa2 refactor(audio): replace ffprobe with ffmpeg for metadata extraction
- Remove ffprobe-static dependency and associated configuration
- Reimplement ffprobeAudio to parse duration and title tags from ffmpeg output
- Update CI workflow to install SeaweedFS binary and set required environment variables
- Enhance E2E test setup to handle auth-protected endpoints during document cleanup
2026-02-15 12:13:29 -07:00
Richard R
260513ebaa ci(config): configure CI environment and optimize function memory
- Set FFMPEG_BIN and FFPROBE_BIN environment variables in Playwright workflow
- Reduce memory allocation for audiobook and whisper API routes to 1024MB
2026-02-15 11:33:46 -07:00
Richard R
1f4794a706 feat(auth): implement session gatekeeping and architectural reorganization
- Introduce `USE_ANONYMOUS_AUTH_SESSIONS` to make guest access an opt-in feature.
- Transition root-level pages into `(app)` and `(public)` route groups for improved access control.
- Adopt Figtree as the primary typeface and implement a pre-render theme initialization script.
- Decouple audiobook chapter processing into a standalone API route and harden resource ownership logic.
- Replace the legacy footer with a unified layout and add skeleton loading states for document lists.
- Update environment documentation and test suites to align with the revised routing and auth flows.

BREAKING CHANGE: The audiobook generation API has been restructured, moving specific chapter logic to `/api/audiobook/chapter`.
2026-02-15 11:12:42 -07:00
Richard R
9f25e05cce feat(previews): implement document preview generation and caching
- Add document preview caching logic with in-memory and persisted storage.
- Implement S3 blobstore functions for managing document previews.
- Create rendering functions for PDF and EPUB cover images to JPEG format.
- Introduce database schema and functions for managing document preview metadata.
- Add unit tests for rendering PDF and EPUB previews.
2026-02-12 16:05:03 -07:00
Richard R
82f82a990e feat: add user preferences and document progress syncing
- Implemented user preferences management with a new API for GET and PUT requests.
- Added user document progress tracking with a new API for retrieving and updating progress.
- Introduced database schema changes for user preferences and document progress.
- Enhanced EPUB and TTS contexts to support syncing user preferences and document progress.
- Added functions to handle transferring user preferences and progress during account linking.
- Updated client-side logic to schedule syncing of user preferences and document progress.
2026-02-11 13:58:16 -07:00
Richard R
852092c558 docs: reorganize documentation structure and fix word highlight behavior 2026-02-11 06:03:51 -07:00
Richard R
63483e055e refactor(docs): reorganize and enhance documentation structure
- Rename configuration.md to auth.md for clarity
- Rename storage-and-blob-behavior.md to object-blob-storage.md
- Split server library import into dedicated documentation page
- Rename intro.md to introduction.md
- Merge support.md into support-and-contributing.md
- Add interactive tabs to configuration and setup guides
- Improve cross-references and navigation between documentation pages
- Update sidebar structure and Docusaurus config for new file paths
2026-02-11 04:16:02 -07:00
Richard R
4e2d18962d fix(config): update ffmpeg tracing for audiobook APIs
Configure Next.js output tracing to properly include ffmpeg and ffprobe
static binaries for audiobook processing endpoints. This ensures the
required binaries are bundled correctly during deployment across all
audiobook API routes.
2026-02-11 03:20:22 -07:00
Richard R
9b9206f50d refactor(audiobooks): migrate audiobook pipeline to object storage
This commit restructures the audiobook generation and serving layer to rely exclusively on S3-compatible blob storage, removing the dependency on local filesystem paths.

- Bundle `ffmpeg` and `ffprobe` binaries via npm to ensure portability across environments.
- Update API routes to stream audio directly from blob storage instead of local disk.
- Remove legacy migration endpoints and filesystem-based indexing logic.
- Add startup scripts to facilitate the transition from local to remote storage.

BREAKING CHANGE: Audiobook functionality is now strictly dependent on S3 configuration. The previous filesystem-based storage method has been removed.
2026-02-11 02:44:34 -07:00
Richard R
91bcf232e1 fix(docs): simplify docs versioning workflow 2026-02-10 15:15:44 -07:00
Richard R
6d5fb65a72 feat(docs): add configurable rate limiting and external docs
Add environment variables for fine-grained control over TTS rate limiting
and Better Auth behavior. Move documentation to external Docusaurus site
with automated deployment workflows.

- TTS rate limiting can now be enabled/disabled via TTS_ENABLE_RATE_LIMIT
- Customizable daily limits for anonymous/authenticated users and IP backstops
- Better Auth rate limiting can be disabled via DISABLE_AUTH_RATE_LIMIT
- Rename library import env vars to IMPORT_LIBRARY_DIRS/DIR
- Add docs-site with Docusaurus and GitHub Actions workflows
- Update README to reference external documentation
2026-02-10 15:11:37 -07:00
Richard R
81d249ed52 feat(storage): implement S3/SeaweedFS blob storage for documents
Replaces local filesystem document storage with S3-compatible object storage.
Adds embedded SeaweedFS 'weed mini' for local development and Docker deployments.
Updates document upload flow to use presigned URLs with a server fallback proxy.
Refactors auth configuration to use BASE_URL and AUTH_SECRET.

BREAKING CHANGE: Renamed BETTER_AUTH_URL to BASE_URL and BETTER_AUTH_SECRET to AUTH_SECRET. Removed legacy document upload/content endpoints. Requires S3 environment variables (auto-configured for embedded SeaweedFS).
2026-02-10 12:29:15 -07:00
Richard R
4a5f3060f2 refactor(documents): migrate from IndexedDB to server-backed storage with caching
- Replace client-side IndexedDB with server-first document storage
- Add document caching layer for offline capability and performance
- Implement document transfer during anonymous-to-authenticated account linking
- Add database indexes for improved query performance
- Update document contexts to use new caching system
- Refactor document upload and deletion APIs
- Add audiobook pruning for missing files
- Improve test isolation with namespace support
- Add unit tests for document cache and transfer functions
2026-02-08 11:15:57 -07:00
Richard R
2c23dc9043 refactor(db): remove conditional database usage from API layer
Ensure all metadata operations consistently use the database layer,
removing filesystem-only fallback paths and conditional DB checks.

- Simplified migration scripts to always run on startup
- Updated document/audiobook APIs to always query DB
- Added ensureDbIndexed() calls across all routes
- Extracted test namespace utilities to dedicated module
- Removed migration-manager.ts (functionality consolidated)
- Updated rate limiter to assume DB is always available

BREAKING CHANGE: Database is now required in all configurations.
When auth is disabled, SQLite is used by default at /app/docstore/sqlite3.db.
2026-02-03 12:17:30 -07:00
Richard R
c24710b2ca refactor(audiobooks): implement user-specific storage with composite primary keys
- Change audiobooks and audiobookChapters tables to use composite PK (id, userId)
- Migrate audiobooks from flat storage to user-specific directories under audiobooks_users
- Add support for claiming unclaimed audiobooks on account creation
- Improve auth rate limiting with retry logic and DISABLE_AUTH_RATE_LIMIT for tests
- Fix iOS/Safari audio playback with unlock mechanism and playback rate watchdog
- Update API routes to handle user-scoped audiobook access with fallback to unclaimed
- Transfer audiobooks when linking anonymous accounts to real accounts
- Remove foreign key constraint from audiobookChapters to support composite PK
- Add cascade delete to account and session foreign keys
2026-02-03 12:17:06 -07:00
Richard R
30ce65e747 refactor(auth): bootstrap anonymous sessions via AuthLoader
Replace privacy popup with PrivacyModal and reorganize auth UI components
Remove signed-out persistence and centralize session handling in useAuthSession
Move document viewers and Kokoro voice helpers into dedicated modules
2026-01-28 02:39:44 -07:00
Richard R
20111b1e36 fix(tests): try to get tests working again 2026-01-28 01:04:01 -07:00
Richard R
6089d2c445 chore(db): move drizzle scripts and update commands
Relocate migration scripts to `drizzle/scripts`, update `start` and
`migrate`
to use the new path, and add a `generate` script for contributors.
Update README migration guidance for SQLite vs Postgres workflows.

BREAKING CHANGE: remove `migrate:force` and change migration script
paths; consumers relying on `scripts/migrate*.mjs` or `pnpm
migrate:force` must update to the new commands.
2026-01-27 17:46:29 -07:00
Richard R
297752e83e fix(db): select drizzle config for migrations
Add dedicated Drizzle config files for sqlite and postgres and generate
new migration outputs for each dialect.

Update migrate scripts to load env files and auto-pick the correct
config unless explicitly provided, and tighten API auth handling by
using shared auth context helpers and enforcing userId checks when auth
is enabled.
2026-01-27 16:53:58 -07:00
Richard R
8189087ad9 fix: reduce pdf flicker and harden tts
- Smooth react-pdf page turns with stable canvas staging and fade-in
- Retry transient empty page text extraction during fast page turns
- Avoid noisy TTS preload toasts, guard requests when quota is exhausted,
  and improve Howler retry/unload behavior to prevent pool exhaustion
- Enforce auth FK cascade deletes and tighten rate-limiter bucket updates
  using affected-row checks
2026-01-26 17:01:51 -07:00
Richard R
c57b913cb5 feat(db/auth): migrate auth storage to Drizzle and add claimable data flow
- replace custom DB adapter with Drizzle setup (SQLite/Postgres schemas) and add `drizzle.config.ts` plus migrations in `drizzle/` and `drizzle_pg/`
- switch better-auth to drizzleAdapter, add auth helpers (`getAuthContext`, `requireAuthContext`, `requireAudiobookOwned`), and make `useAuth`/`useAuthSession` safe no-ops when auth is disabled
- persist documents/audiobooks in DB with ownership checks, unclaimed fallback, and ref-counted deletes; add FS scan helper for no-auth mode
- gate docx-to-pdf, library, voices, whisper, and migration endpoints behind auth when enabled
- add unclaimed data scan/claim flow with `/api/user/claim`, `ClaimDataModal`, and server-side scan/claim helpers
- refactor rate limiting and account deletion to use Drizzle tables (`user_tts_chars`, `user`)
- run migrations via `scripts/migrate-if-auth.mjs` (auto on `pnpm start` + `pnpm migrate`), remove Docker entrypoint and old better-auth migration file
- update README and lockfile for the new migration workflow and dependencies
2026-01-26 17:01:36 -07:00
Richard R
9561d5a816 fix(ui): adjust modal positioning for responsive design
Update modal components to start from top on mobile devices and center on larger screens, improving usability across screen sizes. Also refine transition handling in privacy popup for cleaner unmounting.
2026-01-25 15:13:02 -07:00
Richard R
fb8fc41994 fix(ui): adjust dropdown positioning and scrolling in voices control and improve the ui label for speed controls 2026-01-25 14:55:45 -07:00
Richard R
c345628200 fix: remove unused imports and fix privacy popup text
- Remove unused SettingsIcon and UserMenu imports from epub and pdf pages
- Remove unused UserMenu import from home page
- Remove unused isDev const from home page
- Update SettingsModal to only import clearSignedOut
- Fix apostrophe encoding in privacy popup description
2026-01-25 14:41:49 -07:00
Richard R
50c3829f98 update(auth): enhance rate limiting with device/IP backstops and improve UI
- Add device ID and IP-based rate limiting to prevent abuse
- Refactor UI components for better header menus and settings
- Update privacy popup with detailed data usage info
- Improve PDF handling and caching to prevent react-pdf warnings
- Update README with new Docker instructions and environment variables
2026-01-25 14:38:31 -07:00
Richard R
4689f3676f refactor(auth): consolidate authentication contexts and enhance rate limit UI integration
Streamline auth client factory and remove redundant session manager
Introduce unified auto rate limit context for seamless status updates
Implement problem details in TTS responses for improved error clarity
Add rate limit pause controls and banners across document viewers
Refine PDF scaling logic with height-aware resize handling
Update privacy popup triggers and first-visit modal behavior

BREAKING CHANGE: Daily character limits adjusted (anonymous: 250K → 50K, authenticated: 1M → 500K)
2026-01-25 11:12:19 -07:00
Richard R
9ba20c8a9e feat(auth): add user authentication and rate limiting
- Implement user sign-in, sign-up, and account management using better-auth
- Add rate limiting for TTS API with daily character limits for authenticated and anonymous users
- Integrate SQLite and PostgreSQL database support for user sessions and data persistence
- Update UI components to include authentication flows, user menu, and privacy popup
- Modify Dockerfile and package.json for new dependencies and entrypoint script
- Bump version to v1.3.0
2026-01-24 17:36:11 -07:00
Richard R
64825b626a chore: Update various dependencies and introduce pnpm overrides for specific packages. 2026-01-24 12:33:48 -07:00
Richard R
56004877c0
Merge pull request #75 from richardr1126/v1.2.1
Some checks failed
Create and publish Docker images / prepare (push) Has been cancelled
Create and publish Docker images / build (amd64, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (arm64, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / merge (push) Has been cancelled
* **Bug Fixes**
  * More reliable PDF sentence and word highlighting during layout changes and rapid updates, preventing stale or overlapping highlights.

* **Enhancements**
  * Revamped text normalization and TTS block splitting for better handling of long/complex content, EPUB texts, dialogue/quotes, and PDF artifacts.

* **Tests**
  * Expanded unit tests covering block-splitting, EPUB behavior, oversized inputs, and boundary cases.

* **Chores**
  * Package version bumped; test runner config updated to ignore unit tests for select browser projects.
2026-01-21 14:45:15 -07:00
Richard R
ddf8912092 chore(config): ignore unit tests in firefox and webkit browsers 2026-01-21 14:42:13 -07:00
Richard R
54b9691524 refactor(nlp): enhance EPUB text splitting for oversized sentences and remove unused functions
- Add logic to split oversized sentences in `splitTextToTtsBlocksEPUB` using `splitOversizedText` to ensure blocks stay within `MAX_BLOCK_LENGTH`
- Remove `extractRawSentences` and `processTextWithMapping` functions as they are no longer needed
- Update tests to reflect the changes, including a new test for oversized sentence splitting and removal of related test suites
2026-01-21 14:27:20 -07:00
Richard R
efe6ffec86 fix(nlp): add EPUB-specific text splitting for TTS blocks
Introduce `splitTextToTtsBlocksEPUB` function to handle EPUB text splitting,
treating single newlines as paragraph boundaries for precise highlighting.
Update TTS context to conditionally use EPUB splitting based on document type.
Enhance PDFViewer to clear highlights when current sentence is null.
Add comprehensive tests for the new functionality and refactor existing ones.
2026-01-21 14:04:19 -07:00
Richard R
63c316fc4b fix(pdfviewer): improve sentence highlight cleanup when current sentence is null
Handle case where currentSentence is null by canceling retry loops and clearing stale highlights to prevent lingering highlights. Remove redundant check inside the highlight function.
2026-01-21 13:32:04 -07:00
Richard R
5a40e4b00a fix: improve word highlight cleanup in PDFViewer and update test
- Add cleanup function to prevent memory leaks in word highlighting
- Update test to allow blocks ending with ., !, or ? for better accuracy
2026-01-21 13:17:57 -07:00
Richard R
c28c074e43 refactor(nlp): improve text splitting and normalization for TTS blocks
- Rename and enhance text processing functions in nlp.ts for better handling of oversized texts, sentence boundaries, and PDF artifacts
- Update PDFViewer to add layout-aware highlighting with retry logic for sentence and word highlights
- Adjust PDFContext and TTSContext to use new normalized text functions
- Expand unit tests for new splitting behaviors, including long texts and punctuation preferences
2026-01-21 12:09:59 -07:00
Richard R
54145e2550 fix: Docker whisper cli
Some checks failed
Create and publish Docker images / prepare (push) Has been cancelled
Create and publish Docker images / build (amd64, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (arm64, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / merge (push) Has been cancelled
2026-01-19 21:08:13 -07:00
Richard R
6016481e0a ci: enhance docker publish workflow for multi-arch support
Add support for building and publishing multi-architecture Docker images (amd64 and arm64) using a matrix strategy. Introduce a new 'prepare' job for metadata computation and a 'merge' job for creating manifest lists. Enable manual workflow dispatch with optional latest tagging. Improve caching and provenance handling for better efficiency and reliability.
2026-01-19 20:58:47 -07:00
Richard R
736cd0aa78
Merge pull request #74 from richardr1126/v1.2.0
refactor: External library imports, document selection modal, and file system upgrades with migration logic
2026-01-19 18:09:09 -07:00
Richard R
409b25977c tests: Add NLP text processing, audiobook chapter encoding/decoding, and SHA256 hashing testing. 2026-01-19 17:45:55 -07:00
Richard R
8dbf904a21 feat: Implement FFmpeg metadata escaping and robust chapter index validation for audiobook generation, alongside unit test reorganization. 2026-01-19 17:33:29 -07:00
Richard R
9c941b57ed fix: Enhance path sanitization in getAudiobooksDir to prevent directory traversal and update getMigratedDocumentFileName tests to assert truncated- prefix and hash suffix. 2026-01-19 17:17:51 -07:00
Richard R
47d838039a feat: Add bookId validation to API routes and implement robust, length-limited filename generation with unit tests. 2026-01-19 17:04:02 -07:00
Richard R
33d49d6966 fix: Enhance audiobook API validation and prevent critical meta file deletion 2026-01-19 16:40:04 -07:00
Richard R
f947ece01a feat: Implement document selection modal to allow users to select specific documents for sync, load, and import operations. 2026-01-19 15:53:29 -07:00
Richard R
741126c0b4 feat: Display a toast notification after library migration, powered by new migration status in the API response. 2026-01-19 15:27:07 -07:00
Richard R
f11c015f7e docs: Refactor README to remove *(New)* labels, introduce server-side sync and external library import, and reorder feature descriptions. 2026-01-19 15:02:35 -07:00
Richard R
53b225b976 test: remove audiobook status API test 2026-01-19 14:45:11 -07:00
Richard R
57ac17fb34 refactor(api): force dynamic rendering for API routes
Add 'force-dynamic' export to audiobook and documents API routes to ensure
dynamic rendering. Remove 'nodejs' runtime export from TTS route for consistency.
2026-01-19 14:33:49 -07:00
Richard R
9c56914542 chore: bump version to v1.2.0 2026-01-19 14:24:03 -07:00
Richard R
199b937a25 refactor: overhaul document storage and audiobook architecture
- refactor(documents): implement stable SHA256-based document IDs
- feat(library): add support for external document libraries
- refactor(audiobook): consolidate server logic and migrate to V1 storage layout
- feat(audiobook): add reset functionality and improve chapter management
- fix(ui): enhance audiobook export modal with download/regenerate controls
- test: add coverage for upload hashing and export reset flow
2026-01-19 14:22:21 -07:00
Richard R
ad59f3c1ed docs: add third-party license notices
Include license and copyright notices for pdfjs-dist, Dexie.js, and
dexie-react-hooks as required by the Apache License, Version 2.0.
2026-01-15 11:04:59 -07:00
Richard Roberson
d5ec96d395 feat(ui): implement better document grid with file previews
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
- Introduces `DocumentPreview` component to display rich previews for documents.
- PDFs generate a thumbnail of their first page.
- EPUBs extract and display their cover image.
- HTML, TXT, and Markdown files show a text snippet.
- Refactors document list and folder views to a responsive grid layout.
- Expands main content areas to accommodate the new grid.
- Updates Dockerfile to use Node.js LTS image.
- Updates various development and runtime dependencies.
2025-12-12 13:13:49 -07:00
Richard Roberson
30ade1d79a fix(ui): fix chapter list overflow and menu positioning
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
- Add scroll functionality to the chapter list to prevent content
  overflow.
- Conditionally adjust the positioning of chapter action menus
  to ensure they remain visible within the modal, preventing
  them from being cut off for chapters at the top of the list.
2025-11-22 18:37:38 -07:00
Richard Roberson
d2189541e8 fix(tailwind): fix footer layout and tailwind config
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
- Remove unused `isDev` variable from `Footer.tsx`.
- Add `inline` and `inline-flex` classes to footer elements for improved layout.
- Move custom `xs` breakpoint definition to `theme.extend` in `tailwind.config.ts`.
2025-11-22 16:32:03 -07:00
Richard Roberson
1c6f628c10
Merge pull request #67 from richardr1126/v1.1.0
v1.1.0
2025-11-22 15:38:13 -07:00
Richard Roberson
a755a0b3f9 refactor(ui): make UI labels and descriptions more concise 2025-11-22 15:27:04 -07:00
Richard Roberson
e86782bf38 feat: improve document list and self-hosting clarity
- Introduce a toggleable grid/list view for the document list, enhancing organization and display flexibility. This involved updates across `DocumentList`, `DocumentFolder`, `DocumentListItem`, and `SortControls`.
- Add a new `CodeBlock` component and integrate detailed Docker-based self-hosting instructions into the footer.
- Enhance privacy policy popover with clearer details on Deepinfra usage and strongly recommend self-hosting for secure experience.
- Implement conditional rendering and feature gating based on the `isDev` environment variable, differentiating features between the production demo and self-hosted instances. Affected components include `page.tsx`, `DocumentSettings.tsx`, `SettingsModal.tsx`, and `config.ts`.
- Clarify that advanced features like audiobook export and word-by-word highlighting via `whisper.cpp` are exclusive to self-hosted setups and are disabled in the demo.
- Expand the list of supported document types on the homepage to include MD and TXT.
- Integrate new `ListIcon`, `GridIcon`, and `CopyIcon` to support UI enhancements.
- Add a custom `xs` breakpoint in `tailwind.config.ts` for improved responsive design.
2025-11-22 15:18:46 -07:00
Richard Roberson
e39a5b8bcf refactor(player): simplify popover input focus logic
Removed `isPopoverOpen` state and associated `useEffect` hook.
Auto-focus and select logic for the input are now directly handled
within `handlePopoverOpen`, reducing state management complexity.
2025-11-22 02:06:28 -07:00
Richard Roberson
7046666b3f fix(tts): ensure robust caching for audio and alignments
- New buildCacheKey function creates unique identifiers for TTS cache entries.
- Cache keys now include sentence, voice, speed, provider, and model parameters.
- Prevents serving cached audio or alignment data that mismatches current TTS parameters.
- Removes redundant audioCache.clear() calls when voice or speed change, as entries are now distinct.
2025-11-22 01:57:27 -07:00
Richard Roberson
773778ee90 feat(epub): add custom reader navigation and TOC
- Implement custom previous/next page buttons for EPUB viewer.
- Display current page number out of total pages.
- Introduce an in-viewer, toggleable table of contents (TOC) for quick chapter navigation.
- Hide default `react-reader` navigation arrows and title bar to prevent redundancy.
- Adjust `react-reader` styles to optimize content area and ensure custom controls are visible.
- Update description for EPUB theme setting in document settings.
- Add `ChevronLeftIcon` and `ChevronRightIcon` to icon library.
- Update Playwright tests to reflect new navigation button labels.
2025-11-22 01:33:53 -07:00
Richard Roberson
372c65f23e feat(whisper): integrate binary with build and docs
The Dockerfile has been refactored to a multi-stage build, allowing the `whisper.cpp` CLI binary to be compiled and embedded within the application's runtime image. This enables word-by-word highlighting functionality when deployed via Docker. The `README.md` has been updated to include installation and configuration instructions for `whisper.cpp` when running locally. Additionally, the `WHISPER_CPP_BIN` environment variable has been added to `template.env` and the package version has been bumped to v1.1.0.
2025-11-22 00:25:13 -07:00
Richard Roberson
b576910523 refactor(client): centralize client-side API calls and refine types
Abstracted direct fetch calls across components and contexts into new functions within `src/lib/client.ts`. This provides a consistent and centralized interface for interacting with backend APIs.

- Introduced `src/lib/client.ts` to encapsulate API request logic.
- Standardized audio buffer types (`TTSAudioBuffer`, `TTSAudioBytes`) in `src/types/tts.ts`.
- Moved client-specific request types (`TTSRequestPayload`, `TTSRequestHeaders`, `TTSRetryOptions`) to `src/types/client.ts`.
- Updated API routes and consumer components/contexts to leverage the new client library functions and type definitions.
- Removed `src/utils/audio.ts` as its utility functions are now part of `src/lib/client.ts`.
2025-11-21 23:33:41 -07:00
Richard Roberson
7a29f73d07 refactor(api): move audiobook endpoints to new path
- Relocated all audiobook-related API routes from `/api/audio/convert/*` to `/api/audiobook/*`.
- This change affects endpoints for chapter conversion, retrieval, deletion, and overall audiobook status.
- Updated client-side calls in `AudiobookExportModal.tsx`, `EPUBContext.tsx`, and `PDFContext.tsx` to reflect the new API paths.
- Modified API tests (`api.spec.ts`, `export.spec.ts`) to target the restructured endpoints.
- The new API structure provides better organization and a clearer, more consistent interface for audiobook functionality.
2025-11-21 19:29:49 -07:00
Richard Roberson
733b4180eb feat(tts): add word-by-word highlighting with whisper.cpp
- Introduce `/api/whisper` endpoint which uses `whisper.cpp` (via a `WHISPER_CPP_BIN` executable) and `ffmpeg` to generate word-level audio alignments from provided audio and text.
- Integrate word-level alignments into `TTSContext`, tracking the currently spoken word based on audio seek position and provided timestamps. Alignments are cached in-memory and fetched asynchronously.
- Add new configuration options (`pdfWordHighlightEnabled`, `epubWordHighlightEnabled`) to `ConfigContext` and `Dexie` for enabling/disabling the feature.
- Implement visual word highlighting in both `PDFViewer` and `EPUBViewer` by mapping TTS-aligned words to rendered text elements.
- Enhance `EPUBContext` and `PDFContext` with new `highlightWordIndex` and `clearWordHighlights` functions, utilizing fuzzy string matching (`cmpstr`) to robustly align spoken words with displayed text for accurate highlighting.
- Update `DocumentSettings` to include user-facing toggles for the new highlighting modes.
2025-11-21 19:22:54 -07:00
Richard Roberson
ac0b47debb Update README.md 2025-11-21 12:11:49 -07:00
Richard Roberson
be1e50f0dc Bump version
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
2025-11-21 10:16:45 -07:00
Richard Roberson
eb2fc4eb0d
Merge pull request #66 from richardr1126/v1.0.1
v1.0.1
2025-11-21 10:15:40 -07:00
Richard Roberson
2950314ac7 fix(test): Fix export test after export modal description change 2025-11-18 13:54:28 -07:00
Richard Roberson
ac194ed749 refactor(audiobook): update audiobook export modal generation message to include TTS options. 2025-11-18 12:39:23 -07:00
Richard Roberson
50538bfaba feat(epub): Add EPUB text highlighting for the current TTS sentence with a new configuration option 2025-11-18 12:31:07 -07:00
Richard Roberson
264f3c10d7 docs(README): refine details and update TTS provider information 2025-11-17 12:31:48 -07:00
Richard Roberson
cc0372405f chore(env): hide audiobook export button in production 2025-11-16 21:41:25 -07:00
Richard Roberson
d7a94a6cb0
Merge pull request #62 from richardr1126/version1.0.0
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
Merge v1.0.0 to main branch
2025-11-16 21:22:27 -07:00
Richard Roberson
fb4ede1249 chore(release): bump project version to v1.0.0 2025-11-16 21:13:37 -07:00
Richard Roberson
ff150eec01 refactor(audiobook): standardize chapter regeneration APIs and UX
- Removed `onProgress` callback from `regenerateChapter` and related functions across `TTS`, `EPUB`, and `PDF` contexts to simplify the chapter regeneration API.
- Updated `AudiobookExportModal` to align with the refined regeneration API, including removing granular chapter progress display and adding a hint about TTS caching behavior.
- Introduced `TTSAudiobookChapter` interface and renamed `ContinuationMergeResult` to `TTSSmartMergeResult` and `PageTurnEstimate` to `TTSPageTurnEstimate` for better type consistency and clarity.
- Applied minor styling adjustments to buttons and listbox components in modals for visual consistency.
- Added `'use client'` directive to several client-side components for Next.js 13+ compatibility.
- Updated Dockerfile build command from `pnpm run build` to `pnpm build`.
- Added Playwright tests to verify backend chapter state after regeneration.
2025-11-16 21:10:32 -07:00
Richard Roberson
04def62ff5 refactor(core): standardize module structure and enhance TTS types
- Reorganized utility modules from `src/utils` to `src/lib` for clearer separation of concerns.
- Introduced new, dedicated type definitions in `src/types` for improved type safety in configuration and TTS API interactions.
  - Replaced `src/types/appConfig.ts` with `src/types/config.ts`.
  - Added `src/types/tts.ts` for TTS request payloads, error structures, and retry options.
- Updated module imports across several contexts (`Config`, `EPUB`, `HTML`, `PDF`, `TTS`) and components to reflect the new `lib` and `types` locations.
- Enhanced TTS API request and error handling in `src/app/api/tts/route.ts` and TTS-consuming contexts with explicit types.
- Simplified `ProgressCard`, `ProgressPopup`, and `AudiobookExportModal` components by removing the `isProcessing` prop, centralizing processing state management.
- Streamlined `HTMLContext` by removing `createFullAudioBook` and `isAudioCombining` properties, focusing its scope.
2025-11-16 15:12:35 -07:00
Richard Roberson
ad4faa3ddc docs(readme): update readme with reading experience enhancements 2025-11-15 18:00:56 -07:00
Richard Roberson
7271afa2f5 refactor(config): centralize app config in Dexie
Migrates scattered configuration items and last document locations
from the legacy `config` table to new, dedicated Dexie stores.

- Introduces a singleton `app-config` table (`AppConfigRow`) to standardize
  and structure all application settings in a single object.
- Creates a `last-locations` table for efficient storage of document read positions.
- Updates `ConfigContext` to utilize `dexie-react-hooks`' `useLiveQuery`
  for reactive and simplified state management.
- Implements a database upgrade path (from DB_VERSION 4 to 5) to migrate
  all existing user settings seamlessly.
- Simplifies config access and updates with new `getAppConfig` and `updateAppConfig`
  utility functions.
- Removes the deprecated `config` table after successful migration.
- Updates consumers like `SettingsModal` and test helpers to align with
  the new config structure.
2025-11-15 17:29:56 -07:00
Richard Roberson
d1dd3bd351 ci(playwright): add ffmpeg installation and verification 2025-11-15 15:12:06 -07:00
Richard Roberson
071d967f8f feat(pdf): modernize text highlighting and parsing for fluidity
- Offload computationally intensive text matching for real-time highlighting to a dedicated Web Worker, ensuring the main thread remains responsive during playback.
- Implement a new overlay-based highlighting system that renders independently of the PDF's text layer, providing smoother and more reliable visual feedback without interfering with document rendering.
- Introduce a new setting allowing users to enable or disable real-time text highlighting in PDFs, offering personalized control over the reading interface.
- Upgrade the underlying text comparison algorithm to Dice similarity for more accurate and context-aware matching of spoken words to on-screen text, improving synchronization precision.
- Improve sentence boundary detection, especially for quoted dialogue and complex structures, by enhancing the NLP processing logic, leading to a more natural audio-text flow.
2025-11-15 14:06:44 -07:00
Richard Roberson
2f39e8f014 perf(test): optimize test execution and error handling
- Reduce Playwright worker allocation from 75% to 50% for better resource management
- Disable parallel test execution temporarily to isolate flaky test behavior
- Enhance audio utility to fail fast on user cancellation, preventing unnecessary retries
- Consolidate test-specific utilities within individual test suites for better isolation
- Introduce export functionality test coverage with new export.spec.ts suite
2025-11-14 18:52:49 -07:00
Richard Roberson
5316596a1b refactor(db): migrate to Dexie with reactive queries and simplify data layer
Replaces custom IndexedDB implementation with Dexie ORM, eliminating 850+ lines of
boilerplate code and introducing reactive live queries across all document types.
Transforms document management from imperative refresh patterns to automatic
reactive updates using dexie-react-hooks.

Simplifies TTS backend by removing concurrency semaphore while maintaining
request de-duplication through in-flight tracking. Streamlines document hooks
by removing manual state management and refresh methods.

Updates package dependencies and type definitions to support new database
architecture while maintaining full backward compatibility for existing
documents and settings.

BREAKING CHANGE: Document hooks no longer expose refresh() methods as updates
are now reactive through live queries.
2025-11-14 16:14:00 -07:00
Richard Roberson
e3799e40aa feat(tts): add smart sentence continuation
introduce configurable smart sentence splitting with persisted state,
UI toggle, and EPUB/PDF contexts that send continuation metadata.
enhance TTS pipeline to merge cross-page sentences, manage carryover,
and trigger visual page changes during playback for smoother narration.

update README with kokoro quick-start guidance, remove the legacy
issues mapping doc, and add deterministic TTS mocks plus sample audio
for Playwright tests.
2025-11-14 12:00:03 -07:00
Richard Roberson
d7ef0fa8bb feat(api): implement concurrency control and request de-duplication for TTS audio processing 2025-11-13 00:27:04 -07:00
Richard Roberson
70723f4c1d perf(api): add LRU cache for TTS audio responses
Introduce an in-memory LRU cache for TTS audio with configurable
size and TTL via TTS_CACHE_MAX_SIZE_BYTES and TTS_CACHE_TTL_MS.
Return X-Cache headers (HIT/MISS) and set route runtime to nodejs.
Cache key includes provider, model, voice, speed, format, text,
and optional instructions.

Normalize non-Kokoro multi-voice input to the first token while
preserving full voice string in the cache key. Default Deepinfra
model to hexgrad/Kokoro-82M when none is provided.

Fix Deepinfra Kokoro behavior by enforcing single-voice selection:
- ui: only enable multi-select when provider supports >1 voices
- voice utils: Deepinfra max voices set to 1
- tests: gate provider selection and multi-voice tests by CI and
  increase timeout for stability
2025-11-12 23:28:05 -07:00
Richard Roberson
1dcffc82d8 feat(tts,ui,api): add Kokoro multi-voice selection and SDK support
- Introduce voice utils (model detection, voice parsing/weights, limits)
- Enable Kokoro multi-voice strings across OpenAI/Deepinfra/custom providers
- Normalize non-Kokoro voices to single token for SDK calls
- Expose full Kokoro voice list for custom-openai Kokoro models
- Update TTS API to return audio as ArrayBuffer and improve logging
- Add multi-select UI for Kokoro voices with provider-based clamping
- Preserve Kokoro voice strings in TTSContext and coalesce restarts
- Merge multi-sentence quoted dialogue in NLP sentence splitter
- Update tests for single and multi-voice selection flows
2025-11-12 20:46:45 -07:00
Richard Roberson
21870ed576 fix(api): stabilize DOCX to PDF conversion and cleanup
- Isolate concurrent LibreOffice runs with per-job profile directories
  (soffice -env:UserInstallation) and per-job temp folders
- Poll for generated PDF and verify stable file size before reading
- Consolidate temp artifacts under docstore/tmp and clean up via rm -r
- Add headless/nologo flags and improve error handling

ui(accessibility):
- ConfirmDialog exposes proper dialog roles
- DocumentFolder toggle adds type, aria-expanded, aria-controls, and title;
  associate content panel with an id
- HTMLViewer wraps content in .html-container for HTML/TXT views

test: add comprehensive Playwright specs and helpers
- Accessibility, API health, deletion flows, folders, navigation, playback,
  and upload scenarios
- Add sample.md and unsupported.xyz; update sample.pdf; extend helpers

ci: run Playwright workflow on version1.0.0 branch
2025-11-12 16:21:11 -07:00
Richard Roberson
42665884d7 feat(audiobook): add chapter-based export with UI and API
Introduce end-to-end chapterized audiobook generation with persistent
storage, resumable workflows, and MP3/M4B support.

API:
- add /api/audio/convert/chapter (GET/DELETE) for per-chapter ops
- add /api/audio/convert/chapters (GET/DELETE) for listing/reset
- enhance /api/audio/convert:
  - accept mp3|m4b, stream combined file, cache complete output
  - robust chapter indexing, docstore persistence, list concat
  - AbortSignal-aware ffmpeg/ffprobe, 499 on cancel

UI/UX:
- add AudiobookExportModal with progress, resume, regenerate, download
- add ProgressCard, enhance ProgressPopup (click-to-focus, richer info)
- add Header, ZoomControl; move TTSPlayer to sticky bottom bar
- redesign EPUB/HTML/PDF pages for full-height layout and controls
- add HomeContent; compact uploader variant; document list polish

TTS/Contexts:
- EPUB/PDF contexts now generate per-chapter to disk, return bookId
- support chapter regeneration; progress/cancel propagation
- pass provider/model/instructions; standardize MP3 TTS output
- HTML context updates for model/instructions

Styling:
- globals: overlay-dim, scrollbar styles, prism gradient utilities
- theme vars: secondary-accent, prism-gradient; tailwind color addition

Misc:
- fix PDF scale calc to use container height
- EPUB theme reader area fills height
- time estimation update cadence stab
- audio util passes format to combine endpoint
2025-11-11 13:32:30 -07:00
Richard Roberson
e7ce1a34ce Update docs 2025-11-10 15:30:23 -07:00
Richard Roberson
452bc2adfe feat: Add issue triage and mapping documentation for OpenReader v1
- Created `issues-to-components.md` to summarize open issues and their mappings to components for v1 development.
- Documented key features, bugs, and proposed solutions for issues #59, #48, #47, #44, and #40.
- Outlined global guardrails and cross-cutting improvements for the v1 architecture.

chore: Establish v1 todo and planning framework

- Created `todo.md` to capture the 1.0 rewrite plan, including scope, architecture overview, phased milestones, and a master checklist.
- Defined action items, dependency ordering, and issue mapping alignment for efficient development.

chore: Update pnpm workspace configuration

- Added `ignoredBuiltDependencies` for `canvas` in `pnpm-workspace.yaml` to prevent build issues.
2025-11-10 13:19:45 -07:00
Richard Roberson
77c955bfae fix(settings): restrict model selection for Deepinfra in production without API key 2025-11-09 17:19:37 -07:00
Richard Roberson
4d3c2cf3cf fix(settings): adjust Deepinfra options and placeholder text based on prod or not 2025-11-09 17:16:12 -07:00
Richard Roberson
4050170160 fix(footer): update Deepinfra link in footer component 2025-11-09 17:09:50 -07:00
Richard Roberson
a54d83eda5 fix(settings): set default TTS provider to Deepinfra in production 2025-11-09 17:05:23 -07:00
Richard Roberson
1614bf46bc fix(tts): add missing ttsProvider dependency to useCallback hooks
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
2025-11-09 16:43:13 -07:00
Richard Roberson
dad1dfd26f chore(ci): update Playwright workflow to use pnpm and remove .npmrc 2025-11-09 15:56:09 -07:00
Richard Roberson
1d57004e59 fix(docs): readme 2025-11-09 15:54:36 -07:00
Richard Roberson
b21bdb3a21 feat(tts): add multi-provider TTS support with Deepinfra and custom OpenAI-compatible endpoints
Add comprehensive multi-provider TTS support enabling users to choose between OpenAI, Deepinfra, and custom OpenAI-compatible endpoints. Implement provider-specific voice management with automatic voice restoration per provider-model combination, and migrate package manager to pnpm for improved dependency handling.

Key changes:
- Add TTS provider selection (OpenAI, Deepinfra, custom-openai) in settings UI
- Implement provider-specific model and voice lists with dynamic fetching
- Add voice persistence per provider-model combination in savedVoices
- Support Deepinfra models: Kokoro-82M, Orpheus-3B, Sesame-1B with their voice libraries
- Migrate to pnpm with frozen lockfile for reproducible builds
- Update Docker configuration to use pnpm and Deepinfra API defaults
- Add migration logic for existing users to infer provider from stored baseUrl
- Update test helpers and Playwright configuration for Deepinfra API
- Add example docker-compose.yml with Kokoro-FastAPI integration

BREAKING CHANGE: Voice selection is now provider-model specific. Previously saved voices will be migrated to the new savedVoices structure, but users may need to reselect voices if switching providers.
2025-11-09 15:47:17 -07:00
Richard Roberson
e56736fda8 Update README.md 2025-11-09 11:22:30 -07:00
Richard Roberson
d2477dc9d8
Merge pull request #58 from RobbyV2/feat/SaveLoadProgress
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
2025-07-20 12:13:07 -06:00
RobbyV2
7b703f015f Add cancelText parameter 2025-07-19 21:19:12 -07:00
RobbyV2
32f5cdc494 Saving and Loading Progress Bar 2025-07-19 20:22:50 -07:00
Richard Roberson
f5776e5494
Merge pull request #56 from RobbyV2/DocSelectLoad 2025-07-19 21:16:41 -06:00
Richard Roberson
ba461b20c0
Merge pull request #57 from RobbyV2/feat/MultiUpload 2025-07-19 21:16:29 -06:00
RobbyV2
772aa83f6a Upload multiple files 2025-07-19 19:22:21 -07:00
RobbyV2
0bcae0dee7 Add document selection loading icon 2025-07-19 18:58:08 -07:00
Richard Roberson
593dd7d1a1
Merge pull request #55 from RobbyV2/AutoSelect
Auto select page number when navigating
2025-07-19 13:53:45 -06:00
Richard Roberson
4f633c1359
Merge pull request #54 from RobbyV2/PDF-Click
Allow clicking of internal links
2025-07-19 13:53:25 -06:00
RobbyV2
12e30f4ecf Allow clicking of internal links 2025-07-19 12:46:05 -07:00
RobbyV2
41ea2cdc52 Auto select navigation 2025-07-19 12:40:01 -07:00
Richard Roberson
6b03d9c946
Merge pull request #53 from RobbyV2/main
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
2025-07-17 19:00:32 -06:00
RobbyV2
0aac3f4816 Move button closure higher in SettingsModal 2025-07-17 16:49:52 -07:00
Richard Roberson
b56584cbed Change go to page + add Vercel Analytics integration 2025-06-30 12:10:56 -06:00
Richard Roberson
08480f07ed
Merge pull request #52 from thepycoder/feature/better-speed-control 2025-06-29 15:09:05 -06:00
Richard Roberson
b4e538ca3b
Merge pull request #51 from thepycoder/feature/remember-last-location 2025-06-29 15:08:14 -06:00
Victor Sonck
89833d596b Allow for player speed changes as well as native voice speed changes. Kokoro for example swallows parts of words when asked to go faster than maybe 1.3x. This fixes that, by simply speeding up the player instead. Since we're already using the html5 version of howler, by default it will pitch-correct the audio and it sounds quite good! 2025-06-23 14:09:51 +02:00
Victor Sonck
ece8bcd35a Added saving and restoring last location in PDFs 2025-06-23 13:57:27 +02:00
Richard Roberson
e6cd1d0264 Bump version + fix build error
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
2025-06-22 13:47:23 -06:00
Richard Roberson
ee27330569 Merge branch 'thepycoder-feature/merge_nlp_implementations' 2025-06-22 13:41:40 -06:00
Richard Roberson
c9f8a05137 Merge branch 'feature/merge_nlp_implementations' of https://github.com/thepycoder/OpenReader-WebUI into thepycoder-feature/merge_nlp_implementations 2025-06-22 13:25:10 -06:00
Richard Roberson
8cf685633d Merge branch 'feature/merge_nlp_implementations' of https://github.com/thepycoder/OpenReader-WebUI into thepycoder-feature/merge_nlp_implementations 2025-06-22 13:23:46 -06:00
Richard Roberson
967ffcf074 Merge branch 'thepycoder-feature/editable_pagenumber' 2025-06-22 13:19:33 -06:00
Richard Roberson
8d17c69374 Change width and text color 2025-06-22 13:17:26 -06:00
Richard Roberson
d933ca93c7 Merge branch 'feature/editable_pagenumber' of https://github.com/thepycoder/OpenReader-WebUI into thepycoder-feature/editable_pagenumber 2025-06-22 13:01:18 -06:00
Richard Roberson
555a8c8a9d Merge branch 'main' of https://github.com/richardr1126/OpenReader-WebUI 2025-06-22 12:58:16 -06:00
Richard Roberson
b9ce1d6a8f Change tts url for tests 2025-06-22 12:58:11 -06:00
Victor Sonck
4c92b9875c Remove the NLP route, since it is no longer used. 2025-06-22 15:48:33 +02:00
Victor Sonck
740c5e17d3 Add editable page number in PDF navigator view, to make it much easier to jump to specific pages 2025-06-20 08:32:53 +02:00
Victor Sonck
f594ca836b Refactored NLP into a single, shared function for splitting to sentences.
now both handleClick and global sentences are the exact same, so the sentenceIndex will always be correct.
2025-06-20 08:25:29 +02:00
Richard Roberson
df649656da
Update README.md 2025-05-25 22:05:12 -06:00
Richard Roberson
17ab2d5d48 Update README 2025-04-15 11:51:18 -06:00
Richard Roberson
a1465f1680 Increase size of .md viewer 2025-04-15 11:47:35 -06:00
Richard Roberson
5892383a61 Fix mardown viewer text color
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
2025-04-15 11:41:26 -06:00
Richard Roberson
8cd6158bc4 Update README 2025-04-15 02:47:33 -06:00
Richard Roberson
bb8f8c23d4 Bump version
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
2025-04-15 02:37:41 -06:00
Richard Roberson
82bb91a850 Add TXT and MD support with and HTML viewer with react markdown + many refactors to support it 2025-04-15 02:28:51 -06:00
Richard Roberson
b0a3e2f632 Update readme 2025-04-15 00:32:23 -06:00
Richard Roberson
43a44abcb2 Fix tests 2025-03-30 21:26:56 -06:00
Richard Roberson
e03c374632 Bump version
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
2025-03-30 21:23:22 -06:00
Richard Roberson
fc780746aa Bug fixes and readme update 2025-03-30 21:17:16 -06:00
Richard Roberson
5edb990324
Update bug_report.md 2025-03-23 19:35:46 -06:00
Richard Roberson
7ba807fd95 Pause on page changes + Fix ePub bug when reloading 2025-03-23 17:18:21 -06:00
Richard Roberson
1bbd77f93b Improve space detection logic in PDF text extraction 2025-03-22 12:10:17 -06:00
Richard Roberson
4ed6b17e98 Support gpt-4o-mini-tts with instructions
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
2025-03-21 11:32:42 -06:00
Richard Roberson
f464b059e4 Update API settings 2025-03-13 21:00:34 -06:00
Richard Roberson
12b46cc8e3 Use smaller libreoffice-writer 2025-03-04 23:47:18 -07:00
Richard Roberson
4a8c194c88 Update README.md 2025-03-04 23:08:27 -07:00
Richard Roberson
520236b95c add DOCX support (requires libreoffice) + upload test 2025-03-04 22:47:37 -07:00
Richard Roberson
4c5866c02b Add core-js for promise-with-resolvers support and update version to 0.2.4-patch.4
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
2025-03-04 18:49:08 -07:00
Richard Roberson
84bd878822 New patch
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
2025-03-04 17:18:22 -07:00
Richard Roberson
d10d4410e2 Update PDF worker paths and remove setup script 2025-03-04 17:10:57 -07:00
Richard Roberson
4e857a75f0 withResolvers() try again
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
2025-03-04 14:27:05 -07:00
Richard Roberson
b411a87f08 Update README.md 2025-03-04 13:45:42 -07:00
Richard Roberson
50877d6543 Use legacy PDF worker for lower Safari versions
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
2025-03-04 13:37:41 -07:00
Richard Roberson
652a89790d Update issue template 2025-03-03 23:39:08 -07:00
Richard Roberson
a45608b9db Bump version
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
2025-03-03 23:11:42 -07:00
Richard Roberson
ce9df9a5ab implement retry logic for TTS API calls and clean up code 2025-03-03 23:03:52 -07:00
Richard Roberson
2f2442e5a2 Refactor tests: update Play/Pause functionality and document upload tests; replace e2e.spec.ts with helper functions 2025-03-02 23:40:55 -07:00
Richard Roberson
54dea5da36 Add onloaderror retries 2025-03-02 22:13:08 -07:00
Richard Roberson
4e07e737f0 Stop using audio blobs 2025-03-02 21:16:24 -07:00
Richard Roberson
4aaa74414f Update tests 2025-03-02 20:19:52 -07:00
Richard Roberson
521aafe8e0 Add tests for playing and pausing 2025-03-02 15:56:53 -07:00
Richard Roberson
95c763fb48 Update README.md 2025-03-01 23:56:04 -07:00
Richard Roberson
a704fcbec6
Merge pull request #34 from richardr1126/apple-webkit
Fix Apple WebKit (iOS) PDF Upload
2025-03-01 23:46:53 -07:00
Richard Roberson
136fdb9652 Fix Apple WebKit PDF Upload 2025-03-01 23:36:25 -07:00
Richard Roberson
28a3804843 Setup playwright e2e initial test 2025-03-01 22:44:30 -07:00
Richard Roberson
3568f3c8f9 Update README.md 2025-03-01 19:52:06 -07:00
Richard Roberson
8dfecbab73 Bump version
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
2025-03-01 18:29:23 -07:00
Richard Roberson
b89d1700e9 Fix for longer audiobooks 2025-03-01 18:21:51 -07:00
Richard Roberson
a937e0ee07 Update README.md badges 2025-02-26 06:26:41 -07:00
Richard Roberson
991304d134 Use github packages
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
2025-02-26 05:42:57 -07:00
Richard Roberson
7776333168 Patch version v0.2.2 in v0.2.2-patch.2
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
2025-02-26 04:06:13 -07:00
Richard Roberson
e3370f7526 Further audiobook extraction optimizations 2025-02-26 04:03:05 -07:00
Richard Roberson
568c9f2095 Make Docker image smaller
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
2025-02-26 00:32:30 -07:00
Richard Roberson
9ae7c39f00 Update README.md 2025-02-26 00:12:55 -07:00
Richard Roberson
43b54ba122 Bump version
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
2025-02-25 23:51:36 -07:00
Richard Roberson
129e89b491
Merge pull request #31 from richardr1126/export-audiobook
Export audiobook + PDF extraction margins
2025-02-25 23:49:53 -07:00
Richard Roberson
b67aea6b27 Refactor audio conversion API to improve memory management and add streaming response 2025-02-25 23:38:29 -07:00
Richard Roberson
4a2d75e155 Export to m4b from WAV efficiently 2025-02-25 20:01:15 -07:00
Richard Roberson
79d2e7f613 Working m4b 2025-02-25 19:47:30 -07:00
Richard Roberson
9357dddf3a Implement PDF audiobook generation with configurable margins for text extraction 2025-02-25 16:11:08 -07:00
Richard Roberson
92220de5ea Refactor PDF text extraction to support configurable margins for header, footer, left, and right 2025-02-25 15:53:45 -07:00
Richard Roberson
f948601e70 Add configurable text extraction margin for PDF processing 2025-02-25 03:26:37 -07:00
Richard Roberson
b87b83310b Rely on TTS API for splitting 2025-02-25 00:58:32 -07:00
Richard Roberson
1db906f8bc [ePub only] First working export audiobook 2025-02-24 22:41:38 -07:00
Richard Roberson
61953a761a Change speed menu to slider 2025-02-23 19:03:52 -07:00
Richard Roberson
0da77fffbf Bump version
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
2025-02-23 13:29:50 -07:00
Richard Roberson
96376e2324 Environment changes, set in Docker run 2025-02-23 13:20:42 -07:00
Richard Roberson
9ad1f5f117 Pause in background 2025-02-23 04:46:20 -07:00
Richard Roberson
0be34a09fe Re-add AbortController API, fixes duplicate playback 2025-02-23 01:29:10 -07:00
Richard Roberson
cd00ad2b9b
Update README.md 2025-02-22 14:42:16 -07:00
Richard Roberson
5649312255 Update README.md 2025-02-22 14:04:22 -07:00
Richard Roberson
e7c40b6a0d
Merge pull request #25 from richardr1126/server-tts
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
Server TTS
2025-02-22 13:44:46 -07:00
Richard Roberson
67f3452f31 Bump version 2025-02-22 13:41:25 -07:00
Richard Roberson
0251aae04b Huge iOS playback fix 2025-02-22 13:36:45 -07:00
Richard Roberson
99868f29b2 EPUB resize debounce working perfectly 2025-02-21 12:01:52 -07:00
Richard Roberson
868bb88988 Update docker publish flow
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
2025-02-21 11:30:06 -07:00
Richard Roberson
9d6632d2d0 Use server instead of client for open ai request + large refactors and bug fixes 2025-02-21 11:24:07 -07:00
Richard Roberson
86be935737 Fully unified navigation system 2025-02-19 15:19:37 -07:00
Richard Roberson
d447132859 Refactors to fix blank sections 2025-02-19 15:10:11 -07:00
Richard Roberson
8acf3f6ba6 Fix initial system theme 2025-02-19 12:32:16 -07:00
Richard Roberson
d170b3e792 Bump version
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
2025-02-18 13:39:09 -07:00
Richard Roberson
edff480e0b
Update README.md 2025-02-18 13:38:35 -07:00
Richard Roberson
0ce14f3aec Refactors + epub location fixes 2025-02-18 02:31:30 -07:00
Richard Roberson
9ae6dd9e82 Fix epub use theme and saved location 2025-02-17 23:03:28 -07:00
Richard Roberson
737cf5c9f5 Fix + debounce epub text extraction on window resize 2025-02-17 11:14:00 -07:00
Richard Roberson
36fdbc276d Debounce PDF scale and extraction 2025-02-17 10:57:39 -07:00
Richard Roberson
ebff52d4e6 Better Folder UI 2025-02-14 13:39:02 -07:00
Richard Roberson
d773f8254c EPUB saved location fix 2025-02-14 03:50:53 -07:00
Richard Roberson
1dcf1b09e3 Setting for themed epubs 2025-02-14 01:51:33 -07:00
Richard Roberson
09188ecab2 Update nlp max length 2025-02-13 17:17:29 -07:00
Richard Roberson
74318174b4 Bump version
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
2025-02-13 16:02:17 -07:00
Richard Roberson
14932ef45f Use done button in settings 2025-02-13 15:49:21 -07:00
Richard Roberson
df8772de37 Fix drag and drop on collapsed folders 2025-02-13 15:49:15 -07:00
Richard Roberson
a1f069c0ae Fix sort menu dropdown 2025-02-13 04:29:58 -07:00
Richard Roberson
005ac66d7e More themes 2025-02-13 04:25:49 -07:00
Richard Roberson
80bc0b0006 Refactor new doc list 2025-02-13 04:12:16 -07:00
Richard Roberson
1f2ebb7ba2 Fix theme flickering on Chromium 2025-02-13 00:56:48 -07:00
Richard Roberson
bc790df89e Cleanup imports and ssr cabable components 2025-02-13 00:09:49 -07:00
Richard Roberson
1d418e1444 Update README.md 2025-02-12 22:59:00 -07:00
Richard Roberson
1e5a7d54d8 Bump version
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
2025-02-12 22:46:03 -07:00
Richard Roberson
c8d9d1d952 Add folders hint + edit settings modal 2025-02-12 22:44:25 -07:00
Richard Roberson
61049d6855 Fix doculist state loss on back navigation 2025-02-12 22:07:57 -07:00
Richard Roberson
c21302815c Change folder creation popover 2025-02-12 21:12:49 -07:00
Richard Roberson
cc654f5f1d Save document list state to confid db + add unified document types 2025-02-12 20:49:37 -07:00
Richard Roberson
a6915ee51d Merge branch 'main' into advanced-doclist 2025-02-12 20:29:27 -07:00
Richard Roberson
9d4377cdf9 switch settings location 2025-02-12 05:25:46 -07:00
Richard Roberson
ca895109f3 Fix lint errors 2025-02-12 02:42:13 -07:00
Richard Roberson
b7f64a28b1 Merge branch 'main' into advanced-doclist 2025-02-12 02:39:18 -07:00
Richard Roberson
2a9510aa55 Another nlp optimization 2025-02-12 02:19:57 -07:00
Richard Roberson
f16116cab4 Huge PDF text extraction optimization 2025-02-12 02:15:21 -07:00
Richard Roberson
521eda6cff Move nlp processing to next server 2025-02-12 02:14:56 -07:00
Richard Roberson
f441b030cf DocumentList: Drag to create folders + sorting 2025-02-12 00:14:57 -07:00
Richard Roberson
99dbcd004c Fix check mark in SettingsModal 2025-02-11 21:31:11 -07:00
Richard Roberson
8b7a3195b2 Add gitattributes 2025-02-11 15:40:57 -07:00
Richard Roberson
1326ac0e7e Setup GitHub features 2025-02-11 15:21:09 -07:00
Richard Roberson
d8a5e1e068 Update README.md
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
2025-02-11 14:43:43 -07:00
Richard Roberson
d839d3cfa4 App settings upgrade + new themes 2025-02-11 14:16:47 -07:00
1132 changed files with 191760 additions and 68152 deletions

8
.dockerignore Normal file
View file

@ -0,0 +1,8 @@
.env
.env.*
**/*.creds
README.md
.next
node_modules
**/node_modules
docstore

95
.env.example Normal file
View file

@ -0,0 +1,95 @@
# Logging
# pretty (default): human-readable logs
# json: structured logs for log platforms
# LOG_FORMAT=pretty
# LOG_LEVEL=info
# Local / OpenAI TTS API Configuration (default)
# Suggest using https://github.com/remsky/Kokoro-FastAPI
#
# NOTE: On first boot, the server auto-seeds these values into a "default-openai"
# admin-managed shared provider (DB-backed, encrypted at rest). After that, the
# admin UI is authoritative and changing these env vars has no effect. You may
# remove them once the seed has run. Auth must be configured for this seed path.
# See Settings → Admin → Shared providers.
API_BASE=http://localhost:8880/v1
API_KEY=
# Auth configuration (required in v4+)
BASE_URL=http://localhost:3003 # Externally facing URL for this app (set to LAN IP for access from other devices on the network)
AUTH_SECRET=some_random_secret_key # Generate with `openssl rand -base64 32`
AUTH_TRUSTED_ORIGINS=http://localhost:3003,http://127.0.0.1:3003 # Additional trusted origins (BASE_URL is always trusted)
# (Optional) Allow anonymous auth sessions (default: `false`)
# USE_ANONYMOUS_AUTH_SESSIONS=false
# (Optional) Sign in w/ GitHub Configuration
# GITHUB_CLIENT_ID=
# GITHUB_CLIENT_SECRET=
# (Optional) Comma-separated list of emails that are auto-promoted to admin.
ADMIN_EMAILS=
# (Required for the Vercel scheduled-task cron route; Vercel sends it as a bearer token.)
# Self-hosted deployments run scheduled tasks in-process and do not require this.
# CRON_SECRET=
# (Optional) Backend DB used for server-side metadata (documents/audiobooks) and auth tables.
# Defaults to SQLite at docstore/sqlite3.db when not set.
# POSTGRES_URL=
# Embedded SeaweedFS weed mini config
# (Optional) Enable embedded weed mini for local S3-compatible storage (defaults shown below)
# USE_EMBEDDED_WEED_MINI=true
# WEED_MINI_DIR=docstore/seaweedfs
# WEED_MINI_WAIT_SEC=20
# S3 storage config (use with embedded weed mini or external S3-compatible storage)
# (Optional) For embedded weed mini, set explicit keys if you want stable credentials across restarts.
S3_ACCESS_KEY_ID=
S3_SECRET_ACCESS_KEY=
S3_BUCKET=
# S3_REGION=
# (Optional) If empty in embedded mode, OpenReader uses BASE_URL host (when set) or detected LAN host.
# S3_ENDPOINT=
# S3_FORCE_PATH_STYLE=
# S3_PREFIX=
# (Optional) Library import mode directory (uses /docstore/library by default)
# IMPORT_LIBRARY_DIR=
# IMPORT_LIBRARY_DIRS=
# Compute worker configuration
# (Optional) Embedded compute worker (automatic startup) for local compute tasks (defaults shown below)
# EMBEDDED_COMPUTE_WORKER_PORT=8081
# EMBEDDED_NATS_PORT=4222
# EMBEDDED_NATS_MONITOR_PORT=8222
# EMBEDDED_NATS_STORE_DIR=docstore/nats/jetstream
# NATS_URL=nats://127.0.0.1:4222
# COMPUTE_LOG_LEVEL=info
# COMPUTE_JOB_CONCURRENCY=1
# COMPUTE_WHISPER_TIMEOUT_MS=30000
# COMPUTE_PDF_TIMEOUT_MS=300000
# COMPUTE_PDF_JOB_ATTEMPTS=1
# COMPUTE_OP_STALE_MS=1800000
# WHISPER_MODEL_BASE_URL=https://huggingface.co/onnx-community/whisper-base_timestamped/resolve/main
# PDF_LAYOUT_MODEL_BASE_URL=https://huggingface.co/Bei0001/PP-DocLayoutV3-ONNX/resolve/main
# (Optional) External compute worker config (see docs/deploy/compute-worker)
# COMPUTE_WORKER_URL=http://localhost:8081
# COMPUTE_WORKER_TOKEN=local-compute-token
# (Optional) Override ffmpeg binary path used for audio processing
# FFMPEG_BIN=
# (Optional) Test/dev overrides
# DISABLE_AUTH_RATE_LIMIT=false
# ENABLE_TEST_NAMESPACE=false
# (Optional) Skip automatic startup migrations when set to `false` (default: `true`)
# RUN_DRIZZLE_MIGRATIONS=true
# (Optional) Skip automatic filesystem->S3/DB migration pass when set to `false` (default: `true`)
# RUN_FS_MIGRATIONS=true
# (Optional) v4 JSON seed for first-boot runtime config + shared providers.
# If both are set, RUNTIME_SEED_JSON_PATH is used.
# RUNTIME_SEED_JSON_PATH=/absolute/path/to/openreader-seed.json
# RUNTIME_SEED_JSON={"version":1,"runtimeConfig":{"enableUserSignups":true,"restrictUserApiKeys":true,"defaultTtsProvider":"custom-openai","enableTtsProvidersTab":true,"enableAudiobookExport":true,"enableDocxConversion":true,"showAllProviderModels":true,"disableTtsRateLimit":true,"ttsDailyLimitAnonymous":50000,"ttsDailyLimitAuthenticated":500000,"ttsIpDailyLimitAnonymous":100000,"ttsIpDailyLimitAuthenticated":1000000,"ttsCacheMaxSizeBytes":268435456,"ttsCacheTtlMs":1800000,"ttsUpstreamMaxRetries":2,"ttsUpstreamTimeoutMs":285000,"disableComputeRateLimit":true,"computeParseBurstMax":8,"computeParseBurstWindowSec":60,"computeParseSustainedMax":24,"computeParseSustainedWindowSec":600,"maxUploadMb":200,"changelogFeedUrl":"https://docs.openreader.richardr.dev/changelog/manifest.json"},"providers":[{"slug":"default-openai","displayName":"Default (seeded)","providerType":"custom-openai","baseUrl":"http://localhost:8880/v1","defaultModel":"kokoro","enabled":true}]}

View file

@ -1,9 +0,0 @@
# OpenAI API Key for Text-to-Speech functionality
NEXT_PUBLIC_OPENAI_API_KEY=your_openai_api_key_here
# OpenAI API Base URL (default)
# To use a local TTS model server, I suggest using https://github.com/remsky/Kokoro-FastAPI
NEXT_PUBLIC_OPENAI_API_BASE=https://api.openai.com/v1
# Add other environment variables below as needed
NEXT_PUBLIC_NODE_ENV=development

1
.gitattributes vendored Normal file
View file

@ -0,0 +1 @@
*.tsx linguist-language=TypeScript

15
.github/FUNDING.yml vendored Normal file
View file

@ -0,0 +1,15 @@
# These are supported funding model platforms
github: # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2]
patreon: # Replace with a single Patreon username
open_collective: # Replace with a single Open Collective username
ko_fi: # Replace with a single Ko-fi username
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
liberapay: # Replace with a single Liberapay username
issuehunt: # Replace with a single IssueHunt username
lfx_crowdfunding: # Replace with a single LFX Crowdfunding project-name e.g., cloud-foundry
polar: # Replace with a single Polar username
buy_me_a_coffee: rich2556
thanks_dev: # Replace with a single thanks.dev username
custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2']

24
.github/ISSUE_TEMPLATE/bug_report.md vendored Normal file
View file

@ -0,0 +1,24 @@
---
name: Bug report
about: Please report bugs in this format (provide error message or detailed description at the minimum)
title: ''
labels: bug
assignees: richardr1126
---
**Describe the bug**
A description of what the bug is, which explains how you got there.
**Environment info:**
- OpenReader version: [e.g. v0.2.5 (found in the terminal when starting)]
- Using Docker: yes/no
- Browser: [e.g. Safari, Firefox, Chromium]
- Device: [e.g. iPhone16, Macbook Pro, Intel/AMD PC]
- OS (need if iOS): [e.g. iOS17.2]
**Additional context**
Add any other context about the problem here.
**Screenshots**
If applicable, add screenshots to help explain your problem.

5
.github/ISSUE_TEMPLATE/config.yml vendored Normal file
View file

@ -0,0 +1,5 @@
blank_issues_enabled: false
contact_links:
- name: Q&A
url: https://bsky.app/profile/richardr.dev
about: Ask qustions by reaching out to me on Bluesky

View file

@ -1,53 +1,205 @@
name: Build and Publish Docker Image name: Create and publish Docker images
on: on:
push: push:
tags: tags:
- 'v*.*.*' - 'v*.*.*'
workflow_dispatch:
inputs:
use_latest_tag:
description: 'Tag as latest instead of branch name'
required: false
type: boolean
default: false
env: env:
REGISTRY: docker.io REGISTRY: ghcr.io
IMAGE_NAME: richardr1126/openreader-webui
permissions:
contents: read
jobs: jobs:
prepare:
runs-on: ubuntu-24.04
outputs:
web_image_name: ${{ steps.image-name.outputs.web_image_name }}
web_legacy_image_name: ${{ steps.image-name.outputs.web_legacy_image_name }}
compute_worker_image_name: ${{ steps.image-name.outputs.compute_worker_image_name }}
steps:
- name: Compute Docker image names
id: image-name
run: |
owner="${GITHUB_REPOSITORY_OWNER,,}"
echo "web_image_name=${owner}/openreader" >> "$GITHUB_OUTPUT"
echo "web_legacy_image_name=${owner}/openreader-webui" >> "$GITHUB_OUTPUT"
echo "compute_worker_image_name=${owner}/openreader-compute-worker" >> "$GITHUB_OUTPUT"
build: build:
runs-on: ubuntu-latest needs: prepare
runs-on: ${{ matrix.runner }}
permissions: permissions:
actions: write
contents: read contents: read
packages: write packages: write
strategy:
fail-fast: false
matrix:
include:
- image_target: web
arch: amd64
platform: linux/amd64
runner: ubuntu-24.04
context: .
dockerfile: ./Dockerfile
- image_target: web
arch: arm64
platform: linux/arm64
runner: ubuntu-24.04-arm
context: .
dockerfile: ./Dockerfile
- image_target: compute-worker
arch: amd64
platform: linux/amd64
runner: ubuntu-24.04
context: .
dockerfile: ./packages/compute-worker/Dockerfile
- image_target: compute-worker
arch: arm64
platform: linux/arm64
runner: ubuntu-24.04-arm
context: .
dockerfile: ./packages/compute-worker/Dockerfile
steps: steps:
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@v4 uses: actions/checkout@v5
with:
- name: Set up QEMU ref: ${{ github.sha }}
uses: docker/setup-qemu-action@v3 fetch-depth: 0
- name: Set up Docker Buildx - name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3 uses: docker/setup-buildx-action@v4
- name: Login to Docker Hub - name: Login to GitHub Container Registry
uses: docker/login-action@v3 uses: docker/login-action@v4
with: with:
username: ${{ secrets.DOCKERHUB_USERNAME }} registry: ${{ env.REGISTRY }}
password: ${{ secrets.DOCKERHUB_TOKEN }} username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata for Docker - name: Select image name
id: image
run: |
if [ "${{ matrix.image_target }}" = "web" ]; then
echo "image_name=${{ needs.prepare.outputs.web_image_name }}" >> "$GITHUB_OUTPUT"
else
echo "image_name=${{ needs.prepare.outputs.compute_worker_image_name }}" >> "$GITHUB_OUTPUT"
fi
- name: Build and push Docker image (${{ matrix.image_target }} / ${{ matrix.arch }})
id: build
uses: docker/build-push-action@v7
with:
context: ${{ matrix.context }}
file: ${{ matrix.dockerfile }}
platforms: ${{ matrix.platform }}
cache-from: type=gha,scope=${{ matrix.image_target }}-${{ matrix.arch }}
cache-to: type=gha,mode=max,scope=${{ matrix.image_target }}-${{ matrix.arch }}
provenance: false
outputs: type=image,name=${{ env.REGISTRY }}/${{ steps.image.outputs.image_name }},push-by-digest=true,name-canonical=true,push=true
- name: Export digest
run: |
mkdir -p /tmp/digests/${{ matrix.image_target }}
digest="${{ steps.build.outputs.digest }}"
touch "/tmp/digests/${{ matrix.image_target }}/${digest#sha256:}"
- name: Upload digest
uses: actions/upload-artifact@v7
with:
name: digests-${{ matrix.image_target }}-${{ matrix.arch }}
path: /tmp/digests/${{ matrix.image_target }}
if-no-files-found: error
retention-days: 1
merge:
needs: [prepare, build]
runs-on: ${{ matrix.runner }}
permissions:
actions: read
contents: read
packages: write
strategy:
fail-fast: false
matrix:
include:
- image_target: web
runner: ubuntu-24.04
- image_target: compute-worker
runner: ubuntu-24.04
steps:
- name: Download digests
uses: actions/download-artifact@v8
with:
path: /tmp/digests/${{ matrix.image_target }}
pattern: digests-${{ matrix.image_target }}-*
merge-multiple: true
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Login to GitHub Container Registry
uses: docker/login-action@v4
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Select image names
id: image
run: |
if [ "${{ matrix.image_target }}" = "web" ]; then
echo "image_name=${{ needs.prepare.outputs.web_image_name }}" >> "$GITHUB_OUTPUT"
echo "metadata_images<<EOF" >> "$GITHUB_OUTPUT"
echo "${{ env.REGISTRY }}/${{ needs.prepare.outputs.web_image_name }}" >> "$GITHUB_OUTPUT"
echo "${{ env.REGISTRY }}/${{ needs.prepare.outputs.web_legacy_image_name }}" >> "$GITHUB_OUTPUT"
echo "EOF" >> "$GITHUB_OUTPUT"
else
echo "image_name=${{ needs.prepare.outputs.compute_worker_image_name }}" >> "$GITHUB_OUTPUT"
echo "metadata_images=${{ env.REGISTRY }}/${{ needs.prepare.outputs.compute_worker_image_name }}" >> "$GITHUB_OUTPUT"
fi
- name: Extract metadata (tags, labels) for Docker
id: meta id: meta
uses: docker/metadata-action@v5 uses: docker/metadata-action@v6
with: with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} images: ${{ steps.image.outputs.metadata_images }}
tags: | tags: |
type=raw,value=latest type=raw,value=latest,enable=${{ (github.event_name == 'push' && !contains(github.ref, '-pre')) || (github.event_name == 'workflow_dispatch' && inputs.use_latest_tag == true) }},priority=1000
type=semver,pattern={{version}} type=semver,pattern={{version}}
type=ref,event=tag
type=ref,event=branch,enable=${{ github.event_name == 'workflow_dispatch' && inputs.use_latest_tag != true }}
- name: Build and push Docker image - name: Create manifest list and push
uses: docker/build-push-action@v5 working-directory: /tmp/digests/${{ matrix.image_target }}
with: run: |
context: . docker buildx imagetools create \
platforms: linux/amd64,linux/arm64,linux/arm/v7 $(echo "$TAGS" | xargs -I {} echo -t {}) \
push: true $(printf '${{ env.REGISTRY }}/${{ steps.image.outputs.image_name }}@sha256:%s ' *)
tags: ${{ steps.meta.outputs.tags }} env:
labels: ${{ steps.meta.outputs.labels }} TAGS: ${{ steps.meta.outputs.tags }}
cache-from: type=gha
cache-to: type=gha,mode=max - name: Output build information
run: |
echo "✅ Docker image built and pushed successfully!"
echo "🐋 Target: ${{ matrix.image_target }}"
echo "🐋 Images:"
echo '${{ steps.meta.outputs.tags }}' | sed 's/^/ - /'
echo "📝 Event: ${{ github.event_name }}"
if [ "${{ github.event_name }}" == "workflow_dispatch" ]; then
echo "📝 Triggered by manual workflow dispatch on branch: ${{ github.ref_name }}"
echo "📝 use_latest_tag: ${{ inputs.use_latest_tag }}"
else
echo "📝 Triggered by tag: ${{ github.ref_name }}"
fi

37
.github/workflows/docs-check.yml vendored Normal file
View file

@ -0,0 +1,37 @@
name: Docs Check
on:
pull_request:
branches: [main, master]
paths:
- 'docs-site/**'
- '.github/workflows/docs-check.yml'
- '.github/workflows/docs-deploy.yml'
- 'README.md'
- 'package.json'
jobs:
docs-build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v5
- name: Setup pnpm
uses: pnpm/action-setup@v6
- name: Setup Node.js
uses: actions/setup-node@v5
with:
node-version: lts/*
cache: pnpm
cache-dependency-path: docs-site/pnpm-lock.yaml
- name: Install docs dependencies
working-directory: docs-site
run: pnpm install --frozen-lockfile
- name: Build docs
working-directory: docs-site
run: pnpm build

View file

@ -0,0 +1,34 @@
name: Docs Deploy On Release
on:
release:
types: [published, edited, deleted]
permissions:
actions: write
contents: read
concurrency:
group: docs-release-relay-${{ github.event.release.tag_name }}
cancel-in-progress: true
jobs:
relay:
runs-on: ubuntu-latest
steps:
- name: Dispatch Docs Deploy on main
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REPO: ${{ github.repository }}
TAG: ${{ github.event.release.tag_name }}
ACTION: ${{ github.event.action }}
run: |
set -euo pipefail
gh api \
-X POST \
-H "Accept: application/vnd.github+json" \
"repos/${REPO}/actions/workflows/docs-deploy.yml/dispatches" \
-f ref=main \
-f "inputs[full_reconcile]=false" \
-f "inputs[release_tag]=${TAG}" \
-f "inputs[release_action]=${ACTION}"

81
.github/workflows/docs-deploy.yml vendored Normal file
View file

@ -0,0 +1,81 @@
name: Docs Deploy
on:
push:
branches: [main]
workflow_dispatch:
inputs:
full_reconcile:
description: 'Run full changelog reconcile from GitHub Releases'
required: false
default: false
type: boolean
release_tag:
description: 'Optional release tag for incremental changelog sync (for release relay)'
required: false
type: string
release_action:
description: 'Optional release action (published, edited, deleted) for incremental sync'
required: false
type: string
permissions:
contents: read
pages: write
id-token: write
concurrency:
group: docs-pages
cancel-in-progress: true
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v5
with:
ref: main
- name: Setup pnpm
uses: pnpm/action-setup@v6
- name: Setup Node.js
uses: actions/setup-node@v5
with:
node-version: lts/*
cache: pnpm
cache-dependency-path: docs-site/pnpm-lock.yaml
- name: Install docs dependencies
working-directory: docs-site
run: pnpm install --frozen-lockfile
- name: Build changelog feed
env:
CHANGELOG_PUBLIC_BASE_URL: https://docs.openreader.richardr.dev
CHANGELOG_FORCE_FULL: ${{ github.event_name == 'workflow_dispatch' && inputs.full_reconcile && '1' || '0' }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: node scripts/build-changelog-feed.mjs
- name: Build docs
working-directory: docs-site
run: pnpm build
- name: Upload Pages artifact
uses: actions/upload-pages-artifact@v5
with:
path: docs-site/build
deploy:
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
needs: build
runs-on: ubuntu-latest
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v5

View file

@ -0,0 +1,87 @@
name: Version Docs on Tag
on:
push:
tags:
- 'v*.*.*'
workflow_dispatch:
inputs:
version:
description: 'Version label to snapshot (e.g. v1.2.3)'
required: true
type: string
source_ref:
description: 'Optional git ref to build from (tag, branch, or SHA)'
required: false
type: string
permissions:
contents: write
jobs:
version-docs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v5
with:
ref: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.source_ref || github.sha }}
fetch-depth: 0
- name: Setup pnpm
uses: pnpm/action-setup@v6
- name: Setup Node.js
uses: actions/setup-node@v5
with:
node-version: lts/*
cache: pnpm
cache-dependency-path: docs-site/pnpm-lock.yaml
- name: Install docs dependencies
working-directory: docs-site
run: pnpm install --frozen-lockfile
- name: Snapshot docs version
working-directory: docs-site
env:
VERSION: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.version || github.ref_name }}
run: |
set -euo pipefail
if [ -f versions.json ] && grep -q "\"${VERSION}\"" versions.json; then
echo "Version ${VERSION} already exists in versions.json; skipping docs:version"
else
pnpm docusaurus docs:version "${VERSION}"
fi
- name: Build docs
working-directory: docs-site
run: pnpm build
- name: Commit docs snapshot
id: commit_snapshot
env:
VERSION: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.version || github.ref_name }}
run: |
set -euo pipefail
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add docs-site/versions.json docs-site/versioned_docs docs-site/versioned_sidebars
if git diff --cached --quiet; then
echo "No docs snapshot changes to commit"
echo "committed=false" >> "$GITHUB_OUTPUT"
exit 0
fi
git commit -m "docs: snapshot ${VERSION}"
echo "committed=true" >> "$GITHUB_OUTPUT"
- name: Push docs snapshot to main
if: steps.commit_snapshot.outputs.committed == 'true'
run: |
set -euo pipefail
SNAPSHOT_COMMIT="$(git rev-parse HEAD)"
git fetch origin main
git checkout -B docs-version-publish origin/main
git cherry-pick "${SNAPSHOT_COMMIT}"
git push origin HEAD:main

68
.github/workflows/playwright.yml vendored Normal file
View file

@ -0,0 +1,68 @@
name: Playwright Tests
on:
push:
branches: [ main, master ]
pull_request:
branches: [ main, master ]
jobs:
e2e-testing:
timeout-minutes: 30
runs-on: ubuntu-latest
env:
USE_EMBEDDED_WEED_MINI: true
ENABLE_TEST_NAMESPACE: true
BASE_URL: http://127.0.0.1:3003
AUTH_SECRET: ci-auth-secret-change-me
USE_ANONYMOUS_AUTH_SESSIONS: true
S3_ENDPOINT: http://127.0.0.1:8333
steps:
- uses: actions/checkout@v5
- uses: actions/setup-node@v5
with:
node-version: lts/*
package-manager-cache: false
- uses: pnpm/action-setup@v6
- name: Install system dependencies
run: |
sudo apt-get update
sudo apt-get install -y libreoffice-writer ffmpeg
- name: Install SeaweedFS weed binary
run: |
WEED_PATH=$(docker run --rm --entrypoint sh chrislusf/seaweedfs:4.18 -lc 'command -v weed')
CID=$(docker create chrislusf/seaweedfs:4.18)
docker cp "${CID}:${WEED_PATH}" ./weed
docker rm "${CID}"
sudo install -m 0755 ./weed /usr/local/bin/weed
rm -f ./weed
weed version
- name: Install NATS server binary
run: |
curl -fsSL -o /tmp/nats-server.tar.gz \
https://github.com/nats-io/nats-server/releases/download/v2.12.1/nats-server-v2.12.1-linux-amd64.tar.gz
tar -xzf /tmp/nats-server.tar.gz -C /tmp
sudo install -m 0755 /tmp/nats-server-v2.12.1-linux-amd64/nats-server /usr/local/bin/nats-server
nats-server -v
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Build app and enforce bundle guard
run: |
pnpm build
pnpm build:bundle-guard
- name: Verify ffprobe
run: ffprobe -version
- name: Install Playwright Browsers
run: pnpm exec playwright install --with-deps
- name: Run Playwright tests
env:
API_BASE: https://api.deepinfra.com/v1/openai
API_KEY: ${{ secrets.DEEPINFRA_API_KEY }}
run: pnpm exec playwright test --max-failures=5 --reporter=list,github,html
- uses: actions/upload-artifact@v7
if: ${{ always() }}
with:
name: playwright-report
path: |
playwright-report/
tests/results/
retention-days: 30

29
.github/workflows/vitest.yml vendored Normal file
View file

@ -0,0 +1,29 @@
name: Vitest Tests
on:
push:
branches: [main, master]
pull_request:
branches: [main, master]
jobs:
vitest-testing:
timeout-minutes: 20
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
with:
persist-credentials: false
- uses: actions/setup-node@v5
with:
node-version: lts/*
package-manager-cache: false
- uses: pnpm/action-setup@v6
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Apply drizzle migrations (SQLite)
run: pnpm migrate
- name: Run Vitest suites
run: pnpm test:unit
- name: Verify compute worker contract and boundary
run: pnpm compute:openapi:check && pnpm check:compute-boundary

21
.gitignore vendored
View file

@ -2,6 +2,7 @@
# dependencies # dependencies
/node_modules /node_modules
/.pnpm-store
/.pnp /.pnp
.pnp.* .pnp.*
.yarn/* .yarn/*
@ -31,8 +32,9 @@ yarn-error.log*
.pnpm-debug.log* .pnpm-debug.log*
# env files (can opt-in for committing if needed) # env files (can opt-in for committing if needed)
.env .env*
.dockerignore !.env.example
*.creds
# vercel # vercel
.vercel .vercel
@ -43,3 +45,18 @@ next-env.d.ts
# documents # documents
/docstore /docstore
# Playwright
node_modules/
/tests/results/
/playwright-report/
/blob-report/
/playwright/.cache/
# vscode
.vscode
# Agents
.agents
.codex
.claude

1
.npmrc Normal file
View file

@ -0,0 +1 @@
node-linker=hoisted

View file

@ -1,23 +1,96 @@
# Use Node.js slim image # Stage 1: extract seaweedfs weed binary (for optional embedded weed mini)
FROM node:slim # Pin to 4.18 because CI observed upload regressions on 4.19.
FROM chrislusf/seaweedfs:4.18 AS seaweedfs-builder
RUN cp "$(command -v weed)" /tmp/weed && \
(wget -qO /tmp/SeaweedFS-LICENSE.txt "https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/LICENSE" || \
wget -qO /tmp/SeaweedFS-LICENSE.txt "https://raw.githubusercontent.com/seaweedfs/seaweedfs/main/LICENSE")
# Stage 1b: extract nats-server binary for embedded single-container worker mode.
FROM nats:2.11-alpine AS nats-builder
RUN cp "$(command -v nats-server)" /tmp/nats-server
# Stage 2: build the Next.js app
FROM node:lts-slim AS app-builder
# Install pnpm globally
RUN npm install -g pnpm@10.33.4
# Create app directory # Create app directory
WORKDIR /app WORKDIR /app
# Copy package files # Copy workspace manifests needed for dependency installation
COPY package*.json ./ COPY package.json pnpm-lock.yaml pnpm-workspace.yaml ./
COPY packages/bootstrap/package.json ./packages/bootstrap/package.json
COPY packages/compute-worker/package.json ./packages/compute-worker/package.json
COPY packages/database/package.json ./packages/database/package.json
# Install dependencies # Install dependencies
RUN npm install RUN pnpm install --frozen-lockfile
# Copy project files # Copy project files
COPY . . COPY . .
# Build the Next.js application # Build the Next.js application
RUN npm run build RUN pnpm exec next telemetry disable
RUN AUTH_SECRET=build-placeholder-secret-value-32chars!! BASE_URL=http://localhost:3003 pnpm build
RUN pnpm --config.inject-workspace-packages=true --filter @openreader/bootstrap deploy /opt/openreader/bootstrap
RUN pnpm --dir /opt/openreader/bootstrap rebuild better-sqlite3 ffmpeg-static
RUN pnpm --config.inject-workspace-packages=true --filter @openreader/compute-worker deploy /opt/openreader/embedded-compute-worker
# Generate third-party dependency license report plus copied license files.
RUN mkdir -p /app/THIRD_PARTY_LICENSES && \
pnpm dlx license-checker-rseidelsohn@4.3.0 \
--production \
--json \
--relativeLicensePath \
--out /app/THIRD_PARTY_LICENSES/licenses.json \
--files /app/THIRD_PARTY_LICENSES/files
# Stage 3: minimal runtime image
FROM node:lts-slim AS runner
# Add runtime OS dependencies:
# - libreoffice-writer: required for DOCX → PDF conversion
# ffmpeg is provided by ffmpeg-static from node_modules.
RUN apt-get update && \
apt-get install -y --no-install-recommends ca-certificates libreoffice-writer && \
rm -rf /var/lib/apt/lists/*
# App runtime directory
WORKDIR /app
# Copy only the standalone Next runtime and assets.
COPY --from=app-builder /app/.next/standalone ./
COPY --from=app-builder /app/.next/static ./.next/static
COPY --from=app-builder /app/public ./public
# Ship startup orchestration and the embedded worker as independent deployed bundles.
COPY --from=app-builder /opt/openreader/bootstrap /opt/openreader/bootstrap
COPY --from=app-builder /opt/openreader/embedded-compute-worker /opt/openreader/embedded-compute-worker
# Include third-party license report and copied license texts at a stable path in the image.
COPY --from=app-builder /app/THIRD_PARTY_LICENSES /licenses
# Include SeaweedFS license text for the copied weed binary.
COPY --from=seaweedfs-builder /tmp/SeaweedFS-LICENSE.txt /licenses/SeaweedFS-LICENSE.txt
# Include static model notices for runtime-downloaded assets.
COPY --from=app-builder /app/packages/compute-worker/src/inference/pdf/assets/LICENSE.txt /licenses/pp-doclayoutv3-LICENSE.txt
# Copy seaweedfs weed binary for optional embedded local S3.
COPY --from=seaweedfs-builder /tmp/weed /usr/local/bin/weed
RUN chmod +x /usr/local/bin/weed
# Copy nats-server binary for embedded local JetStream.
COPY --from=nats-builder /tmp/nats-server /usr/local/bin/nats-server
RUN chmod +x /usr/local/bin/nats-server
# Include OpenAI Whisper license text for runtime-downloaded ONNX artifacts.
COPY --from=app-builder /app/packages/compute-worker/src/inference/whisper/assets/LICENSE.txt /licenses/openai-whisper-LICENSE.txt
# Match the app's historical container port now that standalone server.js
# is started directly instead of `next start -p 3003`.
ENV PORT=3003
# Expose the port the app runs on # Expose the port the app runs on
EXPOSE 3003 EXPOSE 3003
# Start the application # Start the application
CMD ["npm", "start"] ENTRYPOINT ["node", "/opt/openreader/bootstrap/src/cli.mjs", "--"]
CMD ["node", "server.js"]

185
README.md
View file

@ -1,157 +1,54 @@
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](#license) [![GitHub Release](https://img.shields.io/github/v/release/richardr1126/openreader)](https://github.com/richardr1126/openreader/releases)
[![Contributions Welcome](https://img.shields.io/badge/Contributions-Welcome-orange.svg)](../../pulls) [![License](https://img.shields.io/github/license/richardr1126/openreader)](LICENSE)
[![Docs](https://img.shields.io/badge/docs-openreader-0a6c74)](https://docs.openreader.richardr.dev/)
[![Playwright Tests](https://github.com/richardr1126/openreader/actions/workflows/playwright.yml/badge.svg)](https://github.com/richardr1126/openreader/actions/workflows/playwright.yml)
[![Docs Check](https://github.com/richardr1126/openreader/actions/workflows/docs-check.yml/badge.svg)](https://github.com/richardr1126/openreader/actions/workflows/docs-check.yml)
# OpenReader WebUI [![GitHub Stars](https://img.shields.io/github/stars/richardr1126/openreader)](https://github.com/richardr1126/openreader/stargazers)
[![GitHub Forks](https://img.shields.io/github/forks/richardr1126/openreader)](https://github.com/richardr1126/openreader/network/members)
[![Discussions](https://img.shields.io/badge/Discussions-Ask%20a%20Question-blue)](https://github.com/richardr1126/openreader/discussions)
OpenReader WebUI is web-based reader Text-to-Speech capabilities, offering a TTS read along experience with narration for both PDF and EPUB documents. It can use any OpenAI compatible TTS endpoint, including [Kokoro-FastAPI](https://github.com/remsky/Kokoro-FastAPI/tree/v0.0.5post1-stable). # 📄🔊 OpenReader
- 🎯 **TTS API Integration**: Compatible with OpenAI API TTS and Kokoro FastAPI TTS, enabling high-quality voice narration OpenReader is an open-source, self-host-friendly text-to-speech document reader built with Next.js for **EPUB, PDF, TXT, MD, and DOCX** with multilingual, synchronized read-along playback.
- 💾 **Local-First Architecture**: Secure document handling with IndexedDB browser storage - no server uploads required
- 🔍 **Smart Text Processing**: Real-time sentence detection and synchronized text highlighting during playback
- 📚 **EPUB Support**: Read EPUB files with table of contents and synchronized text
- 📄 **PDF Support**: Read PDF files with text extraction and page navigation
- ⚡ **Modern Tech Stack**: Built with Next.js, React, and Tailwind CSS
- 🎨 **Customizable Experience**:
- Adjustable playback speed
- Multiple voice options
- Dark/light/system theme support
- Persistent user settings
- 📱 **Cross-Platform**: Responsive design works seamlessly across desktop and mobile devices
## **Demo** > Previously named **OpenReader-WebUI**.
> **Get started in the [docs](https://docs.openreader.richardr.dev/)**.
## ✨ Highlights
https://github.com/user-attachments/assets/1bfc5fc4-8d66-4c71-a3c1-f3ec5b4f4b56 - 🧱 **Layout-aware PDF parsing** with PP-DocLayoutV3 (ONNX) — structured block detection, cross-page stitching, and geometry-based highlighting for precise read-along sync.
- ⏱️ **Word-by-word highlighting** via ONNX Whisper alignment through the compute worker control plane (NATS JetStream-backed).
- ⚡ **Segment-based read-along** for EPUB, PDF, TXT, MD, and DOCX — sentence-aware TTS with cached audio segments, background preloading, and resumable playback.
- 🎯 **Multi-provider TTS** — self-hosted OpenAI-compatible servers (Kokoro-FastAPI, KittenTTS-FastAPI, Orpheus-FastAPI) or cloud APIs (OpenAI, Replicate, DeepInfra).
- 🌐 **Multilingual support** — choose a document language for language-aware narration and highlighting. Available languages depend on the configured TTS provider and voice.
- 🎧 **Audiobook export** in `m4b`/`mp3` with resumable chapter processing.
- 🗂️ **Flexible backend** — embedded SeaweedFS or S3-compatible storage, SQLite or Postgres, server library import, and device sync.
- 🐳 **Self-host friendly** — Docker (amd64/arm64), built-in auth/session support, and automatic startup migrations.
## 🚀 Start Here
| Goal | Link |
| --- | --- |
| Run with Docker | [Docker Quick Start](https://docs.openreader.richardr.dev/docker-quick-start) |
| Deploy on Vercel | [Vercel Deployment](https://docs.openreader.richardr.dev/deploy/vercel-deployment) |
| Deploy external compute worker | [Compute Worker (NATS JetStream)](https://docs.openreader.richardr.dev/deploy/compute-worker) |
| Develop locally | [Local Development](https://docs.openreader.richardr.dev/deploy/local-development) |
| Configure auth | [Auth](https://docs.openreader.richardr.dev/configure/auth) |
| Configure SQL database | [Database and Migrations](https://docs.openreader.richardr.dev/configure/database) |
| Configure object storage | [Object / Blob Storage](https://docs.openreader.richardr.dev/configure/object-blob-storage) |
| Configure TTS providers | [TTS Providers](https://docs.openreader.richardr.dev/configure/tts-providers) |
| Run Kokoro locally | [Kokoro-FastAPI](https://docs.openreader.richardr.dev/configure/tts-provider-guides/kokoro-fastapi) |
| Run KittenTTS locally | [KittenTTS-FastAPI](https://docs.openreader.richardr.dev/configure/tts-provider-guides/kitten-tts-fastapi) |
| Get support or contribute | [Support and Contributing](https://docs.openreader.richardr.dev/about/support-and-contributing) |
## 🐳 Docker Quick Start ## 🧭 Community
```bash - Questions and ideas: [GitHub Discussions](https://github.com/richardr1126/openreader/discussions)
docker run --name openreader-webui -p 3003:3003 richardr1126/openreader-webui:latest - Bug reports: [GitHub Issues](https://github.com/richardr1126/openreader/issues)
``` - Contributions: open a pull request
or with server-side doc storage:
```bash ## 📜 License
docker run --name openreader-webui \
-p 3003:3003 \
-v openreader_docstore:/app/docstore \
richardr1126/openreader-webui:latest
```
Visit [http://localhost:3003](http://localhost:3003) to run the app.
### Using Docker Compose MIT. See [LICENSE](LICENSE).
Create or add to a `docker-compose.yml`:
```yaml
volumes:
openreader_docstore:
services:
openreader-webui:
container_name: openreader-webui
image: richardr1126/openreader-webui:latest
ports:
- "3003:3003"
volumes:
- openreader_docstore:/app/docstore
restart: unless-stopped
```
## Dev Installation
### Prerequisites
- Node.js & npm (recommended: use [nvm](https://github.com/nvm-sh/nvm))
### Steps
1. Clone the repository:
```bash
git clone https://github.com/richardr1126/OpenReader-WebUI.git
cd OpenReader-WebUI
```
2. Install dependencies:
```bash
npm install
```
3. Configure the environment:
```bash
cp .env.template .env
# Edit .env with your configuration settings
```
4. Start the development server:
```bash
npm run dev
```
or build and run the production server:
```bash
npm run build
npm start
```
Visit [http://localhost:3003](http://localhost:3003) to run the app.
> Dev server runs on port 3000 by default, while the production server runs on port 3003.
#### Docker Supported Architectures
- linux/amd64 (x86_64)
- linux/arm64 (Apple Silicon)
- linux/arm/v7 (Raspberry Pi)
## **Features**
- Real-time sentence by sentence text-to-speech via OpenAI API TTS, [Kokoro FastAPI TTS](https://github.com/remsky/Kokoro-FastAPI), or others compatible with the `/v1/audio/speech` endpoint
- IndexedDB-powered local storage
- Synchronized text highlighting during playback (using string similarity for best match on the page)
- Configurable playback speed and voice options, which checks `/v1/audio/voices` for available voices
- Click-to-skip on the same page for navigation
- Responsive design with light, dark, and system themes
- All configuration settings saved in IndexedDB
## Stack
- **Framework:** Next.js (React)
- **Storage:** IndexedDB (in browser db store)
- **PDF Processing:**
- [react-pdf](https://github.com/wojtekmaj/react-pdf)
- [pdf.js](https://mozilla.github.io/pdf.js/)
- Compromise for text analysis
- **EPUB Processing:**
- [react-reader](https://github.com/happyr/react-reader)
- [epubjs](https://github.com/futurepress/epub.js/)
- **UI Components:**
- Headless UI
- Tailwind CSS
- **TTS Integration:** anything you want
## Project Structure
```
src/
├── app/ // Next.js app router
├── components/ // Reusable UI components
├── contexts/ // State management contexts
└── services/ // Utility functions & integrations
```
## Contributing
Contributions are welcome! Fork the repository and submit a pull request with your changes. For significant alterations, please open an issue first.
## License
This project is licensed under the MIT License.
## Acknowledgements
- [react-pdf](https://github.com/wojtekmaj/react-pdf)
- [react-reader](https://github.com/happyr/react-reader)
- [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) for text-to-speech
- [Kokoro-FastAPI](https://github.com/remsky/Kokoro-FastAPI) for the API wrapper
## Support
If you encounter issues or have suggestions, please open an issue on GitHub.

View file

@ -0,0 +1,124 @@
name: openreader-full
services:
openreader:
image: ghcr.io/richardr1126/openreader:latest
# Keep localhost:8333 valid for both app requests and browser-facing presigned URLs.
network_mode: service:seaweedfs
depends_on:
kokoro-tts:
condition: service_started
postgres:
condition: service_healthy
seaweedfs:
condition: service_healthy
environment:
BASE_URL: ${BASE_URL:-http://localhost:3003}
AUTH_SECRET: ${AUTH_SECRET:-local-openreader-auth-secret-change-me}
POSTGRES_URL: postgres://openreader:openreader@postgres:5432/openreader
COMPUTE_WORKER_URL: http://compute-worker:8081
COMPUTE_WORKER_TOKEN: ${COMPUTE_WORKER_TOKEN:-local-compute-token}
USE_EMBEDDED_WEED_MINI: "false"
S3_ENDPOINT: ${S3_ENDPOINT:-http://localhost:8333} # Used for internal endpoint and public facing presigned URLs.
S3_BUCKET: ${S3_BUCKET:-openreader-documents}
S3_REGION: ${S3_REGION:-us-east-1}
S3_ACCESS_KEY_ID: ${S3_ACCESS_KEY_ID:-devkey}
S3_SECRET_ACCESS_KEY: ${S3_SECRET_ACCESS_KEY:-devsecret}
S3_FORCE_PATH_STYLE: "true"
S3_PREFIX: ${S3_PREFIX:-openreader}
RUNTIME_SEED_JSON: |-
{
"version": 1,
"runtimeConfig": {
"defaultTtsProvider": "kokoro"
},
"providers": [
{
"slug": "kokoro",
"displayName": "Kokoro",
"providerType": "custom-openai",
"baseUrl": "http://kokoro-tts:8880/v1",
"defaultModel": "kokoro",
"enabled": true
}
]
}
volumes:
- openreader-docstore:/app/docstore
kokoro-tts:
image: ghcr.io/remsky/kokoro-fastapi-cpu:v0.2.4
environment:
ONNX_NUM_THREADS: 8
ONNX_INTER_OP_THREADS: 4
ONNX_EXECUTION_MODE: parallel
ONNX_OPTIMIZATION_LEVEL: all
ONNX_MEMORY_PATTERN: "true"
ONNX_ARENA_EXTEND_STRATEGY: kNextPowerOfTwo
API_LOG_LEVEL: DEBUG
ports:
- "8880:8880"
seaweedfs:
image: chrislusf/seaweedfs:4.18
command: ["mini", "-dir=/data"]
environment:
AWS_ACCESS_KEY_ID: ${S3_ACCESS_KEY_ID:-devkey}
AWS_SECRET_ACCESS_KEY: ${S3_SECRET_ACCESS_KEY:-devsecret}
healthcheck:
test: ["CMD-SHELL", "wget -qO- http://127.0.0.1:9333/cluster/status >/dev/null"]
interval: 2s
timeout: 2s
retries: 30
ports:
- "3003:3003"
- "8333:8333"
volumes:
- seaweedfs-data:/data
nats:
image: nats:2.14-alpine
command: ["-js", "-sd", "/data"]
volumes:
- nats-data:/data
postgres:
image: postgres:17-alpine
environment:
POSTGRES_USER: openreader
POSTGRES_PASSWORD: openreader
POSTGRES_DB: openreader
healthcheck:
test: ["CMD-SHELL", "pg_isready -U openreader -d openreader"]
interval: 2s
timeout: 2s
retries: 30
volumes:
- postgres-data:/var/lib/postgresql/data
compute-worker:
image: ghcr.io/richardr1126/openreader-compute-worker:latest
depends_on:
nats:
condition: service_started
seaweedfs:
condition: service_healthy
environment:
NATS_URL: nats://nats:4222
COMPUTE_WORKER_HOST: 0.0.0.0
PORT: 8081
COMPUTE_WORKER_TOKEN: ${COMPUTE_WORKER_TOKEN:-local-compute-token}
S3_ENDPOINT: http://seaweedfs:8333
S3_BUCKET: ${S3_BUCKET:-openreader-documents}
S3_REGION: ${S3_REGION:-us-east-1}
S3_ACCESS_KEY_ID: ${S3_ACCESS_KEY_ID:-devkey}
S3_SECRET_ACCESS_KEY: ${S3_SECRET_ACCESS_KEY:-devsecret}
S3_FORCE_PATH_STYLE: "true"
S3_PREFIX: ${S3_PREFIX:-openreader}
COMPUTE_PREWARM_MODELS: ${COMPUTE_PREWARM_MODELS:-false}
volumes:
openreader-docstore:
seaweedfs-data:
nats-data:
postgres-data:

View file

@ -0,0 +1,130 @@
name: openreader-local-full-build
services:
openreader:
image: openreader:local
build:
context: ../..
dockerfile: Dockerfile
# Keep localhost:8333 valid for both app requests and browser-facing presigned URLs.
network_mode: service:seaweedfs
depends_on:
kokoro-tts:
condition: service_started
postgres:
condition: service_healthy
seaweedfs:
condition: service_healthy
environment:
BASE_URL: ${BASE_URL:-http://localhost:3003}
AUTH_SECRET: ${AUTH_SECRET:-local-openreader-auth-secret-change-me}
POSTGRES_URL: postgres://openreader:openreader@postgres:5432/openreader
COMPUTE_WORKER_URL: http://compute-worker:8081
COMPUTE_WORKER_TOKEN: ${COMPUTE_WORKER_TOKEN:-local-compute-token}
USE_EMBEDDED_WEED_MINI: "false"
S3_ENDPOINT: ${S3_ENDPOINT:-http://localhost:8333} # Used for internal endpoint and public facing presigned URLs.
S3_BUCKET: ${S3_BUCKET:-openreader-documents}
S3_REGION: ${S3_REGION:-us-east-1}
S3_ACCESS_KEY_ID: ${S3_ACCESS_KEY_ID:-devkey}
S3_SECRET_ACCESS_KEY: ${S3_SECRET_ACCESS_KEY:-devsecret}
S3_FORCE_PATH_STYLE: "true"
S3_PREFIX: ${S3_PREFIX:-openreader}
RUNTIME_SEED_JSON: |-
{
"version": 1,
"runtimeConfig": {
"defaultTtsProvider": "kokoro"
},
"providers": [
{
"slug": "kokoro",
"displayName": "Kokoro",
"providerType": "custom-openai",
"baseUrl": "http://kokoro-tts:8880/v1",
"defaultModel": "kokoro",
"enabled": true
}
]
}
volumes:
- openreader-docstore:/app/docstore
kokoro-tts:
image: ghcr.io/remsky/kokoro-fastapi-cpu:v0.2.4
environment:
ONNX_NUM_THREADS: 8
ONNX_INTER_OP_THREADS: 4
ONNX_EXECUTION_MODE: parallel
ONNX_OPTIMIZATION_LEVEL: all
ONNX_MEMORY_PATTERN: "true"
ONNX_ARENA_EXTEND_STRATEGY: kNextPowerOfTwo
API_LOG_LEVEL: DEBUG
ports:
- "8880:8880"
seaweedfs:
image: chrislusf/seaweedfs:4.18
command: ["mini", "-dir=/data"]
environment:
AWS_ACCESS_KEY_ID: ${S3_ACCESS_KEY_ID:-devkey}
AWS_SECRET_ACCESS_KEY: ${S3_SECRET_ACCESS_KEY:-devsecret}
healthcheck:
test: ["CMD-SHELL", "wget -qO- http://127.0.0.1:9333/cluster/status >/dev/null"]
interval: 2s
timeout: 2s
retries: 30
ports:
- "3003:3003"
- "8333:8333"
volumes:
- seaweedfs-data:/data
nats:
image: nats:2.14-alpine
command: ["-js", "-sd", "/data"]
volumes:
- nats-data:/data
postgres:
image: postgres:17-alpine
environment:
POSTGRES_USER: openreader
POSTGRES_PASSWORD: openreader
POSTGRES_DB: openreader
healthcheck:
test: ["CMD-SHELL", "pg_isready -U openreader -d openreader"]
interval: 2s
timeout: 2s
retries: 30
volumes:
- postgres-data:/var/lib/postgresql/data
compute-worker:
image: openreader-compute-worker:local
build:
context: ../..
dockerfile: packages/compute-worker/Dockerfile
depends_on:
nats:
condition: service_started
seaweedfs:
condition: service_healthy
environment:
NATS_URL: nats://nats:4222
COMPUTE_WORKER_HOST: 0.0.0.0
PORT: 8081
COMPUTE_WORKER_TOKEN: ${COMPUTE_WORKER_TOKEN:-local-compute-token}
S3_ENDPOINT: http://seaweedfs:8333
S3_BUCKET: ${S3_BUCKET:-openreader-documents}
S3_REGION: ${S3_REGION:-us-east-1}
S3_ACCESS_KEY_ID: ${S3_ACCESS_KEY_ID:-devkey}
S3_SECRET_ACCESS_KEY: ${S3_SECRET_ACCESS_KEY:-devsecret}
S3_FORCE_PATH_STYLE: "true"
S3_PREFIX: ${S3_PREFIX:-openreader}
COMPUTE_PREWARM_MODELS: ${COMPUTE_PREWARM_MODELS:-false}
volumes:
openreader-docstore:
seaweedfs-data:
nats-data:
postgres-data:

View file

@ -0,0 +1,52 @@
name: openreader-local-slim-build
services:
openreader:
image: openreader:local
build:
context: ../..
dockerfile: Dockerfile
depends_on:
kokoro-tts:
condition: service_started
environment:
BASE_URL: ${BASE_URL:-http://localhost:3003}
AUTH_SECRET: ${AUTH_SECRET:-local-openreader-auth-secret-change-me}
RUNTIME_SEED_JSON: |-
{
"version": 1,
"runtimeConfig": {
"defaultTtsProvider": "kokoro"
},
"providers": [
{
"slug": "kokoro",
"displayName": "Kokoro",
"providerType": "custom-openai",
"baseUrl": "http://kokoro-tts:8880/v1",
"defaultModel": "kokoro",
"enabled": true
}
]
}
ports:
- "3003:3003"
- "8333:8333"
volumes:
- openreader-docstore:/app/docstore
kokoro-tts:
image: ghcr.io/remsky/kokoro-fastapi-cpu:v0.2.4
environment:
ONNX_NUM_THREADS: 8
ONNX_INTER_OP_THREADS: 4
ONNX_EXECUTION_MODE: parallel
ONNX_OPTIMIZATION_LEVEL: all
ONNX_MEMORY_PATTERN: "true"
ONNX_ARENA_EXTEND_STRATEGY: kNextPowerOfTwo
API_LOG_LEVEL: DEBUG
ports:
- "8880:8880"
volumes:
openreader-docstore:

View file

@ -0,0 +1,49 @@
name: openreader-slim
services:
openreader:
image: ghcr.io/richardr1126/openreader:latest
depends_on:
kokoro-tts:
condition: service_started
environment:
BASE_URL: ${BASE_URL:-http://localhost:3003}
AUTH_SECRET: ${AUTH_SECRET:-local-openreader-auth-secret-change-me}
RUNTIME_SEED_JSON: |-
{
"version": 1,
"runtimeConfig": {
"defaultTtsProvider": "kokoro"
},
"providers": [
{
"slug": "kokoro",
"displayName": "Kokoro",
"providerType": "custom-openai",
"baseUrl": "http://kokoro-tts:8880/v1",
"defaultModel": "kokoro",
"enabled": true
}
]
}
ports:
- "3003:3003"
- "8333:8333"
volumes:
- openreader-docstore:/app/docstore
kokoro-tts:
image: ghcr.io/remsky/kokoro-fastapi-cpu:v0.2.4
environment:
ONNX_NUM_THREADS: 8
ONNX_INTER_OP_THREADS: 4
ONNX_EXECUTION_MODE: parallel
ONNX_OPTIMIZATION_LEVEL: all
ONNX_MEMORY_PATTERN: "true"
ONNX_ARENA_EXTEND_STRATEGY: kNextPowerOfTwo
API_LOG_LEVEL: DEBUG
ports:
- "8880:8880"
volumes:
openreader-docstore:

4
docs-site/.gitignore vendored Normal file
View file

@ -0,0 +1,4 @@
.docusaurus
build
node_modules
static/changelog

134
docs-site/custom.css Normal file
View file

@ -0,0 +1,134 @@
:root {
/* OpenReader default light theme colors */
--ifm-color-primary: #ef4444;
--ifm-color-primary-dark: #dc2626;
--ifm-color-primary-darker: #b91c1c;
--ifm-color-primary-darkest: #991b1b;
--ifm-color-primary-light: #f87171;
--ifm-color-primary-lighter: #fca5a5;
--ifm-color-primary-lightest: #fecaca;
--ifm-background-color: #ffffff;
--ifm-background-surface-color: #f7fafc;
--ifm-navbar-background-color: #f7fafc;
--ifm-footer-background-color: #f7fafc;
--ifm-font-color-base: #2d3748;
--ifm-color-content: #2d3748;
--ifm-color-content-secondary: #718096;
--ifm-toc-border-color: #e2e8f0;
--ifm-hr-background-color: #e2e8f0;
--ifm-code-background: #e2e8f0;
--ifm-code-font-size: 95%;
}
[data-theme='dark'] {
/* OpenReader default dark theme colors */
--ifm-color-primary: #f87171;
--ifm-color-primary-dark: #ef4444;
--ifm-color-primary-darker: #dc2626;
--ifm-color-primary-darkest: #b91c1c;
--ifm-color-primary-light: #fb8c8c;
--ifm-color-primary-lighter: #fca5a5;
--ifm-color-primary-lightest: #fecaca;
--ifm-background-color: #111111;
--ifm-background-surface-color: #171717;
--ifm-navbar-background-color: #171717;
--ifm-footer-background-color: #171717;
--ifm-font-color-base: #ededed;
--ifm-color-content: #ededed;
--ifm-color-content-secondary: #a3a3a3;
--ifm-toc-border-color: #343434;
--ifm-hr-background-color: #343434;
--ifm-code-background: #343434;
}
.navbar {
border-bottom: 1px solid var(--ifm-toc-border-color);
}
.footer {
border-top: 1px solid var(--ifm-toc-border-color);
background: var(--ifm-background-surface-color);
}
.footer__title {
color: var(--ifm-font-color-base);
font-weight: 600;
}
.footer__link-item {
color: var(--ifm-color-content-secondary);
}
.footer__link-item:hover {
color: var(--ifm-color-primary);
text-decoration: none;
}
.footer__copyright {
border-top: 1px solid var(--ifm-toc-border-color);
color: var(--ifm-color-content-secondary);
margin-top: 1rem;
padding-top: 1rem;
}
/* Force override Infima defaults (which use high-specificity selectors). */
:root:not(#\#):not(#\#):not(#\#) {
--ifm-color-primary: #ef4444 !important;
--ifm-color-primary-dark: #dc2626 !important;
--ifm-color-primary-darker: #b91c1c !important;
--ifm-color-primary-darkest: #991b1b !important;
--ifm-color-primary-light: #f87171 !important;
--ifm-color-primary-lighter: #fca5a5 !important;
--ifm-color-primary-lightest: #fecaca !important;
--ifm-background-color: #ffffff !important;
--ifm-background-surface-color: #f7fafc !important;
--ifm-navbar-background-color: #f7fafc !important;
--ifm-footer-background-color: #f7fafc !important;
--ifm-font-color-base: #2d3748 !important;
--ifm-color-content: #2d3748 !important;
--ifm-color-content-secondary: #718096 !important;
--ifm-toc-border-color: #e2e8f0 !important;
--ifm-hr-background-color: #e2e8f0 !important;
--ifm-code-background: #e2e8f0 !important;
}
html[data-theme='dark']:not(#\#):not(#\#):not(#\#) {
--ifm-color-primary: #f87171 !important;
--ifm-color-primary-dark: #ef4444 !important;
--ifm-color-primary-darker: #dc2626 !important;
--ifm-color-primary-darkest: #b91c1c !important;
--ifm-color-primary-light: #fb8c8c !important;
--ifm-color-primary-lighter: #fca5a5 !important;
--ifm-color-primary-lightest: #fecaca !important;
--ifm-background-color: #111111 !important;
--ifm-background-surface-color: #171717 !important;
--ifm-navbar-background-color: #171717 !important;
--ifm-footer-background-color: #171717 !important;
--ifm-font-color-base: #ededed !important;
--ifm-color-content: #ededed !important;
--ifm-color-content-secondary: #a3a3a3 !important;
--ifm-toc-border-color: #343434 !important;
--ifm-hr-background-color: #343434 !important;
--ifm-code-background: #343434 !important;
}
/* Search/Ask-AI plugin accents should follow OpenReader accent colors. */
:root:not(#\#):not(#\#):not(#\#) {
--search-local-highlight-color: var(--ifm-color-primary) !important;
--search-local-hit-active-color: var(--ifm-color-primary) !important;
--search-local-input-active-border-color: var(--ifm-color-primary) !important;
}
.ask-ai:not(#\#):not(#\#) {
--ask-ai-primary: var(--ifm-color-primary) !important;
--ask-ai-primary-hover: var(--ifm-color-primary-dark) !important;
}
html[data-theme='dark'] .ask-ai:not(#\#):not(#\#) {
--ask-ai-primary: var(--ifm-color-primary) !important;
--ask-ai-primary-hover: var(--ifm-color-primary-dark) !important;
}

View file

@ -0,0 +1,16 @@
---
title: Acknowledgements
---
This project is built with support from the following open-source projects and tools:
- [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M)
- [Kokoro-FastAPI](https://github.com/remsky/Kokoro-FastAPI)
- [Better Auth](https://www.better-auth.com/)
- [SQLite](https://www.sqlite.org/)
- [PostgreSQL](https://www.postgresql.org/)
- [SeaweedFS](https://github.com/seaweedfs/seaweedfs)
- [OpenAI Whisper](https://github.com/openai/whisper)
- [ffmpeg](https://ffmpeg.org)
- [react-pdf](https://github.com/wojtekmaj/react-pdf)
- [react-reader](https://github.com/happyr/react-reader)

View file

@ -0,0 +1,7 @@
---
title: License
---
OpenReader is licensed under the MIT License.
- Repository license file: [LICENSE](https://github.com/richardr1126/openreader/blob/main/LICENSE)

View file

@ -0,0 +1,19 @@
---
title: Support and Contributing
---
## Feature requests
Use [GitHub Discussions](https://github.com/richardr1126/openreader/discussions) for feature requests and ideas.
## Issues and support
If you encounter a bug, open an issue in [GitHub Issues](https://github.com/richardr1126/openreader/issues).
## Contributing
Contributions are welcome.
- Fork the repository
- Create your branch
- Open a pull request with your changes

View file

@ -0,0 +1,159 @@
---
title: Admin Panel
---
The admin panel lets a designated set of users manage **shared TTS providers** and **site-wide feature flags** directly from the Settings modal — without touching env vars or redeploying.
It is gated behind authentication, so you must have auth enabled to use it ([Auth](./auth)).
## Designating admins
Set `ADMIN_EMAILS` to a comma-separated list of emails:
```env
AUTH_SECRET=... # required for auth
BASE_URL=... # required for auth
ADMIN_EMAILS=alice@example.com,bob@example.com
```
On every session resolution the server compares the user's email against this list and writes `user.is_admin = true` (or `false` for emails removed from the list). No restart is required to demote — the next page load picks it up.
When the logged-in user is an admin, an **Admin** tab appears in **Settings → sidebar** with two sub-tabs:
- **Shared providers** — server-side TTS provider instances visible to all users.
- **Site features** — runtime-editable replacements for what were previously build-time public env flags.
## Shared TTS providers
Each shared provider is one named instance bound to one of the four built-in provider types (`custom-openai`, `openai`, `replicate`, `deepinfra`). The admin form has:
| Field | Notes |
| --- | --- |
| **Slug** | URL-safe identifier exposed to users (e.g. `kokoro-prod`). Must not collide with a built-in id. Lowercase alphanumeric + hyphens. |
| **Display name** | Shown in the user's provider dropdown, suffixed with "(shared)". |
| **Provider type** | One of the four built-ins. Determines voice/model resolution. |
| **Base URL** | Optional. Falls through to the provider type's default when blank. |
| **API key** | Encrypted at rest with AES-256-GCM (key derived from `AUTH_SECRET` via scrypt). On edit, leave blank to keep the existing key. |
| **Default model** | Optional. Used as the initial model when a user selects this provider. |
| **Enabled** | Toggle to hide the provider from non-admin users without deleting it. |
When a non-admin user picks a shared provider in **Settings → TTS Provider**:
- The API key / base URL fields are hidden — those credentials never leave the server.
- The TTS request still goes through the user's browser, but the server replaces the slug with the matching admin row's decrypted key and base URL before calling the upstream provider.
- The user's per-request `x-openai-key` / `x-openai-base-url` headers are ignored for shared slugs.
Whether users can supply their own personal built-in provider keys is controlled by the site feature `restrictUserApiKeys`:
- `true`: users are restricted to shared providers only.
- `false`: users may also use per-user BYOK credentials for built-in providers.
### Auto-seeded "default-openai"
On first boot, if `admin_providers` is empty and `API_BASE` or `API_KEY` is set, OpenReader creates a single shared provider with:
- slug `default-openai`, displayName `Default (from env)`, providerType `custom-openai`
- baseUrl from `API_BASE`, apiKey from `API_KEY` when provided (blank keys are supported)
- defaultModel set to `kokoro` (you can edit it in Admin → Shared providers)
After this seed runs, the legacy `API_KEY` / `API_BASE` env vars are no longer read by the TTS routes — the DB row is authoritative. You can rename, edit, disable, or delete this row like any other from the admin UI, and remove the env vars from your `.env` when convenient.
:::warning Upgrading from v2.2.0
In v2.2.0 and earlier, `API_KEY` / `API_BASE` were read live by the TTS routes on every request. As of v3.0.0 they are **one-shot seeds** consumed only on the first boot where `admin_providers` is empty. After upgrading, boot the app once and confirm a `default-openai` row exists in **Settings → Admin → Shared providers** with the correct base URL. If it is missing or wrong (e.g. the env vars were not set on first boot, or the table was already non-empty from a pre-release), create or edit the shared provider manually — TTS will not fall back to the env vars.
:::
## Site features
Runtime-editable settings, one row per key:
| Key | What it controls |
| --- | --- |
| `defaultTtsProvider` | Default provider id new users start with (built-in id or shared slug). |
| `changelogFeedUrl` | Public changelog manifest URL used by the Settings modal changelog panel. |
| `enableUserSignups` | Controls whether new accounts can be created. Existing accounts can still sign in when this is `false`. |
| `restrictUserApiKeys` | Restrict user-supplied API keys/base URLs; when `true`, only admin shared providers are allowed. |
| `enableTtsProvidersTab` | Whether the user-facing TTS Provider tab in Settings is shown. |
| `showAllProviderModels` | When `false`, users are restricted to each provider's default model (shared provider `defaultModel` or built-in provider default). |
| `enableAudiobookExport` | Show the audiobook export entry points on PDF/EPUB pages. |
| `enableDocxConversion` | Accept .docx uploads (converted to PDF server-side). |
Word-by-word highlighting and PDF layout parsing capability are controlled by compute-worker server env configuration, not an admin runtime flag.
Each row shows a source badge:
- **from seed** — the value was seeded on first boot (from `RUNTIME_SEED_JSON` / `RUNTIME_SEED_JSON_PATH`).
- **admin** — explicit admin override. Use **Reset** on the row to clear it back to built-in default behavior.
- **default** — no seed/admin row exists; built-in default is active.
:::warning Security note for `restrictUserApiKeys`
Turning `restrictUserApiKeys` off allows user-supplied API keys to flow through this server. Use this only for trusted/self-hosted deployments where that tradeoff is acceptable.
:::
## Rate limiting
A dedicated **Rate limiting** group (within the same admin panel) collects the daily quotas, the PDF parsing throttle, and the upload size cap:
| Key | What it controls |
| --- | --- |
| `disableTtsRateLimit` | Disable the per-user/IP daily TTS character limits. When `false`, the daily-limit fields below it apply. |
| `disableComputeRateLimit` | Disable per-user PDF parsing rate limiting. When `false`, the burst/sustained limit fields below it apply. |
| `maxUploadMb` | Maximum size (MB) accepted for a single document upload. Enforced server-side and signed into the presigned S3 PUT. |
The **Disable TTS daily rate limiting** and **Disable PDF parsing rate limiting** toggles each reveal a collapsible group of numeric inputs when set to `false`:
- TTS: anonymous/authenticated per-user daily limits and anonymous/authenticated IP daily backstops.
- PDF parsing: burst limit + window (seconds) and sustained limit + window (seconds). The sustained window doubles as a concurrency cap.
## TTS upstream
At the end of the **Site features** tab, a dedicated **TTS upstream** group controls server-side request and cache tuning (DB-backed runtime settings, not env vars):
| Key | What it controls |
| --- | --- |
| `ttsUpstreamMaxRetries` | Maximum retry attempts for upstream TTS 429/5xx responses. |
| `ttsUpstreamTimeoutMs` | Upstream request timeout for OpenAI-compatible TTS calls. |
| `ttsCacheMaxSizeBytes` | Maximum size of the in-memory TTS audio cache. |
| `ttsCacheTtlMs` | Time-to-live for cached TTS audio buffers. |
In v4 these settings are admin-only and are no longer configurable through environment variables.
## Scheduled tasks
The **Scheduled tasks** section controls background maintenance jobs such as expired-upload cleanup, orphaned-blob reaping, and rate-limit ledger pruning.
- Enable or disable each task, adjust its interval, or run it immediately.
- Runs use database-backed leases so multiple app instances do not normally execute the same task concurrently.
- A task that exceeds four minutes is aborted and recorded as failed. A crashed run can be reclaimed after its stale lease expires.
- Failures and the latest successful summary appear on the task card and in server logs.
Self-hosted Node.js deployments tick the scheduler in-process once per minute. Vercel uses the authenticated `/api/admin/tasks/tick` cron route; the checked-in Vercel Hobby schedule runs once daily, so intervals shorter than one day are unavailable there. See [Vercel Deployment](../deploy/vercel-deployment#5-scheduled-maintenance-tasks).
## Migrating off env vars
In v4, runtime site features are managed by admin settings and optional JSON seed. To minimize env surface area:
1. Deploy this version with your existing env values in place.
2. Boot the app once. Open Settings → Admin and verify:
- Seeded settings appear as **from seed** (if you supplied a runtime JSON seed).
- A `default-openai` row exists in **Shared providers** (if you had `API_BASE` or `API_KEY` set).
3. Remove any bootstrap env vars you no longer need from `.env`.
4. Redeploy. Behavior is unchanged — the DB is now the source of truth.
You can keep `API_BASE` / `API_KEY` if you intentionally want bootstrap fallback behavior on empty provider tables.
## How keys are protected
- API keys are encrypted in the `admin_providers` table with AES-256-GCM. The encryption key is derived from `AUTH_SECRET` via `scrypt`.
- The masked-list view (`GET /api/admin/providers`, used by the admin UI itself) returns `••••` + last-4 only — never plaintext or ciphertext.
- The public list endpoint (`GET /api/tts/shared-providers`, called by every user's browser) returns only `{ slug, displayName, providerType, defaultModel }`. Keys and base URLs are never exposed to the client.
- Non-admin users cannot enumerate admin providers' credentials or base URLs through any API.
:::danger Rotating `AUTH_SECRET` invalidates all stored admin provider keys
Because the encryption key for `admin_providers` is derived from `AUTH_SECRET`, changing `AUTH_SECRET` makes every stored API key undecryptable. After rotating it, shared providers will fail to authenticate upstream until you re-enter each provider's API key in **Settings → Admin → Shared providers** (edit the row and paste the key again). There is no automated re-encryption path. If you must rotate `AUTH_SECRET`, plan to re-enter admin provider keys immediately afterward.
:::
## Related
- [Auth](./auth) — required to use the admin panel.
- [TTS Providers](./tts-providers) — built-in provider catalog and per-user behavior.
- [Environment Variables](../reference/environment-variables) — `ADMIN_EMAILS`, provider bootstrap vars, and runtime JSON seed.

View file

@ -0,0 +1,64 @@
---
title: Auth
---
This page covers application-level configuration for provider access and authentication.
## Auth behavior
- `BASE_URL` and `AUTH_SECRET` are required at startup in v4+.
- Keep `AUTH_TRUSTED_ORIGINS` empty to trust only `BASE_URL`.
- Anonymous auth sessions are disabled by default.
- Set `USE_ANONYMOUS_AUTH_SESSIONS=true` to enable anonymous session flows.
## Runtime modes
OpenReader has two common runtime modes:
- **Auth enabled, non-admin user**: user account/session features are available, but no admin controls.
- **Auth enabled, admin user**: full **Settings → Admin** access (shared providers + site features).
## Admin role
You can designate one or more users as admins via the `ADMIN_EMAILS` env var:
```env
ADMIN_EMAILS=alice@example.com,bob@example.com
```
Admins see a new **Admin** tab in **Settings** with two sub-tabs:
- **Shared TTS providers** — server-managed TTS provider instances with encrypted keys, visible to all users.
- **Site features** — runtime overrides for what were previously build-time public env flags (including account signup availability, default TTS provider, audiobook export, etc.).
Admin assignment is reconciled on every session resolution, so removing an email from `ADMIN_EMAILS` demotes the user on next login without a restart. See [Admin Panel](./admin-panel) for the full reference.
## Route behavior
- `/` is a public landing/onboarding page and remains indexable.
- `/app` is the protected app home (document list and uploader UI).
- If a valid session exists (including anonymous), visiting `/` redirects to `/app`.
- Protected app routes continue to require auth; when anonymous sessions are disabled and no session exists, users are redirected to `/signin`.
## Related docs
- For auth environment variables: [Environment Variables](../reference/environment-variables#auth-and-identity)
- For admin role and shared TTS provider config: [Admin Panel](./admin-panel)
- For TTS character limits and quota behavior: [TTS Rate Limiting](./tts-rate-limiting)
- For provider-specific guidance: [TTS Providers](./tts-providers)
- For storage/S3/SeaweedFS behavior: [Object / Blob Storage](./object-blob-storage)
- For database mode: [Database](./database)
- For migration behavior and commands: [Migrations](./migrations)
## Sync notes
### Auth enabled
- Settings and reading progress are saved to the server.
- Updates are not instant push-based sync; they use normal client polling/refresh behavior.
- If two devices change the same item around the same time, the newest update wins.
## Claim modal note
- You may still see old anonymous settings/progress available to claim from older deployments.
- Legacy `unclaimed` data is only surfaced through the claim flow; normal authenticated routes are scoped to your current user id.

View file

@ -0,0 +1,58 @@
---
title: Database
---
This page covers database mode selection for OpenReader.
## Scope of this page
- Focus: SQL metadata, state, and relational tables.
- Not covered here: object key layout and blob transport details (see [Object / Blob Storage](./object-blob-storage)).
## Database mode
- SQLite (default): embedded DB at `docstore/sqlite3.db`; good for local/self-host single-instance setups.
- Postgres: enabled when `POSTGRES_URL` is set; recommended for production/distributed deployments.
## What the database stores
- Document and audiobook metadata/state used by server routes.
- Auth/session tables (`user`, `session`, `account`, `verification`) when auth is enabled — schema is auto-generated by Better Auth.
- TTS character usage counters (`user_tts_chars`) for daily rate limiting (when enabled).
- User settings preferences (`user_preferences`) when auth is enabled.
- User reading progress (`user_document_progress`) when auth is enabled.
- Document preview job/asset metadata (`document_previews`) for server-side PDF/EPUB thumbnails.
- TTS segment metadata (`tts_segments`) for server-side playback caching:
- Segment identity + settings hash
- Audio object key and duration
- Optional alignment payload for word highlighting
- Status/error state
- Text fingerprint/hash (plaintext segment text is not stored)
App-specific tables are manually maintained in Drizzle schema files, while auth tables are generated by the Better Auth CLI. Both are migrated together via Drizzle. See [Migrations](./migrations) for details.
## What the database does not store
- Raw document file bytes
- Audiobook audio bytes
- TTS segment audio bytes
- Generated preview image bytes
Those payloads live in object storage. SQL stores the metadata, references, and status.
## Related variables
- `POSTGRES_URL`
For database variable behavior, see [Environment Variables](../reference/environment-variables#database-and-object-blob-storage).
## Related docs
- [Migrations](./migrations)
- [Object / Blob Storage](./object-blob-storage)
- [Auth](./auth)
## State sync summary
- Settings and reading progress are stored in SQL and synced from the app.
- Sync is currently request-based (not realtime push invalidation).

View file

@ -0,0 +1,147 @@
---
title: Migrations
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
This page covers migration behavior for both database schema and storage data in OpenReader.
## Runtime ownership
- `@openreader/database` owns database clients, schemas, SQL migration files, and programmatic
migration execution for SQLite and PostgreSQL.
- `@openreader/bootstrap` owns startup orchestration, storage migration, and optional embedded
SeaweedFS, NATS, and compute-worker processes.
- The Next.js app imports `@openreader/database` directly, but does not orchestrate migrations or
child processes.
Docker deploys bootstrap as an isolated runtime bundle under `/opt/openreader/bootstrap`; it does
not merge migration dependencies into the standalone Next.js app under `/app`.
## Startup migration behavior
By default, the shared entrypoint runs migrations automatically before app startup in:
- Docker container startup
- `pnpm dev`
- `pnpm start`
Startup migration phases:
- DB schema migrations (`pnpm migrate`)
- Storage/data migration (`pnpm migrate-fs`) for legacy filesystem content into S3 + DB rows
:::info
In most setups, you do not need to run migration commands manually because startup handles this automatically.
:::
### Schema history
Migrations are applied in order. All of the following ship in v3.0.0; an instance upgrading from v2.2.0 applies `0001``0004` in a single startup pass.
| Migration | Dialects | What it does |
| --- | --- | --- |
| `0001_tts_segments` | SQLite + Postgres | Creates the original single-table `tts_segments` used by server-side TTS segment caching. |
| `0002_add_segment_key_to_tts_segments` | SQLite + Postgres | Adds the `segment_key` column to `tts_segments` for stable locator-independent segment identity. |
| `0003_tts_segments_v2_split` | SQLite + Postgres | Replaces `tts_segments` with a normalized two-table model: `tts_segment_entries` (one row per document segment + locator identity) and `tts_segment_variants` (one row per settings combination, holding the cached audio key, status, and alignment). Drops the original `tts_segments` table — no released build (v2.2.0 or earlier) ever populated it, so there is no production data to migrate. |
| `0004_admin_panel` | SQLite + Postgres | Creates `admin_providers` (encrypted shared TTS provider rows) and `admin_settings` (runtime site-feature config), and adds the `is_admin` column to the `user` table. Backs the [Admin Panel](./admin-panel). |
To skip automatic startup migrations:
- Set `RUN_DRIZZLE_MIGRATIONS=false`
- Set `RUN_FS_MIGRATIONS=false`
:::warning
If you disable startup migrations, ensure your deployment process runs migrations before serving traffic.
:::
## Apply migrations
In most cases, you do not need manual migration commands because startup runs migrations automatically.
`pnpm migrate` applies migrations for one database target:
- Postgres when `POSTGRES_URL` is set
- SQLite when `POSTGRES_URL` is unset
```bash
# Run pending migrations for one target:
# - Postgres if POSTGRES_URL is set
# - SQLite if POSTGRES_URL is unset
pnpm migrate
# Run storage migration (filesystem -> S3 + DB)
pnpm migrate-fs
# Dry-run storage migration without uploading/deleting
pnpm migrate-fs:dry-run
```
`pnpm migrate` uses the programmatic Drizzle migrator from `@openreader/database`. Drizzle Kit is
not a production or startup dependency; it is used only to generate new migration files.
## Generate migrations
`pnpm generate` is a two-phase script for contributors and schema changes:
1. **Better Auth schema generation** — runs the Better Auth CLI twice (once for SQLite, once for Postgres) to produce auto-generated Drizzle schema files for auth tables (`user`, `session`, `account`, `verification`).
2. **Drizzle migration generation** — runs `drizzle-kit generate` for both configs in `packages/database`, producing SQL migration files from all schema files (app + auth).
:::note
Most users do not need to run `pnpm generate`. Use it when contributing or when you have changed Drizzle schema files and need new migration files.
:::
### Schema ownership
Auth tables are owned by Better Auth. Their Drizzle schema definitions are auto-generated and should **not** be hand-edited:
- `packages/database/src/schema_auth_sqlite.ts`
- `packages/database/src/schema_auth_postgres.ts`
App-specific tables are manually maintained in the standard Drizzle schema files:
- `packages/database/src/schema_sqlite.ts`
- `packages/database/src/schema_postgres.ts`
Both sets of schema files are included in the Drizzle generation configs. Runtime migration
execution is owned by `@openreader/database`.
When app schema changes (for example `tts_segments`), keep these in sync:
- `packages/database/src/schema_sqlite.ts`
- `packages/database/src/schema_postgres.ts`
- `packages/database/migrations/sqlite/*.sql` + `packages/database/migrations/sqlite/meta/_journal.json`
- `packages/database/migrations/postgres/*.sql` + `packages/database/migrations/postgres/meta/_journal.json`
<Tabs groupId="generate-migration-commands">
<TabItem value="project-script" label="Project Script" default>
```bash
# Full pipeline: Better Auth CLI + Drizzle generate (both dialects)
pnpm generate
```
</TabItem>
<TabItem value="drizzle-direct" label="Manual Drizzle Cmd">
```bash
# Generate SQLite migrations only (skips Better Auth CLI)
pnpm exec drizzle-kit generate --config packages/database/drizzle.config.sqlite.ts
# Generate Postgres migrations only (skips Better Auth CLI)
pnpm exec drizzle-kit generate --config packages/database/drizzle.config.pg.ts
```
:::warning
Running `drizzle-kit generate` directly skips the Better Auth CLI step. If auth schema has changed upstream (e.g. after a Better Auth version bump), run `pnpm generate` instead to regenerate the auth schema files first.
:::
</TabItem>
</Tabs>
## Related docs
- [Database](./database)
- [Object / Blob Storage](./object-blob-storage)
- [Migration Environment Variables](../reference/environment-variables#migration-controls)

View file

@ -0,0 +1,167 @@
---
title: Object / Blob Storage
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
This page documents storage backends, blob upload routing, and core Docker mount behavior.
## Scope of this page
- Focus: object/blob backends, keyspaces, upload/read paths, and storage debugging.
- Not covered here: relational metadata tables and SQL state modeling (see [Database](./database)).
## Storage backends
- Embedded (default): embedded SeaweedFS (`weed mini`) blob storage.
- External: external S3-compatible object storage.
Metadata database mode (SQLite vs Postgres) is configured separately in [Database](./database).
:::warning SeaweedFS Compatibility Note (April 16, 2026)
OpenReader currently pins embedded SeaweedFS to `4.18` in CI and Docker builds.
`4.19` introduced intermittent `InternalError` responses on S3 `PutObject` in our upload flow.
:::
Storage variables are documented in [Environment Variables](../reference/environment-variables#database-and-object-blob-storage).
## Ports
- `3003`: OpenReader app and API routes
- `8333`: Embedded SeaweedFS S3 endpoint for direct browser blob access
:::info
`8333` is only needed for direct browser presigned access to embedded SeaweedFS.
:::
## Upload behavior
- Primary path: browser uploads to presigned URL from `/api/documents/blob/upload/presign`.
- Fallback path: `/api/documents/blob/upload/fallback` when direct upload fails/unreachable.
- Read/download path: blob/content serving route `/api/documents/blob` (not the upload fallback route).
- Preview path: `/api/documents/blob/preview` (returns `202` while a preview is generating; serves/redirects when ready).
## Document previews
- PDF/EPUB previews are generated server-side and stored in object storage under `document_previews_v1`.
- Preview generation is triggered on upload registration and also backfills on first preview request for older docs.
- Preview artifacts are temporary-cache friendly and can be regenerated from the source document blob.
## FS / Volume Mounts
### App data mount
- Target: `/app/docstore`
- Recommended: yes, for persistence
- Purpose: persists SeaweedFS blob data, SQLite metadata DB, migrations, and local runtime temp state
- Mount string: `-v openreader_docstore:/app/docstore`
### Library source mount (optional)
- Target: `/app/docstore/library`
- Recommended: optional, use read-only (`:ro`)
- Purpose: exposes host files as a source for server library import
- Mount string: `-v /path/to/your/library:/app/docstore/library:ro`
- Details: [Server Library Import](./server-library-import)
## Private blob endpoint mode
If `8333` is not published externally:
- Document uploads still work through upload fallback proxy
- Reads/snippets continue through app API routes
- Direct presigned browser upload/download to embedded endpoint is unavailable
:::warning
Without `8333`, expect higher app-server traffic because uploads/downloads go through API routes instead of direct object endpoint access.
:::
## Audiobook Storage Debug Commands
Audiobook assets are stored in object storage under the `audiobooks_v1` keyspace. Use these commands to inspect and download objects for debugging.
<Tabs groupId="audiobook-storage-access-cli">
<TabItem value="aws-s3" label="AWS S3" default>
```bash
# List all audiobook objects
aws s3 ls "s3://$S3_BUCKET/$S3_PREFIX/audiobooks_v1/" --recursive
# Filter to one book id (replace <book-id>)
aws s3 ls "s3://$S3_BUCKET/$S3_PREFIX/audiobooks_v1/" --recursive | grep "<book-id>-audiobook/"
# Download one object by full key
aws s3 cp "s3://$S3_BUCKET/$S3_PREFIX/audiobooks_v1/<path>/<file>.m4b" "./audiobook.m4b"
```
</TabItem>
<TabItem value="s3-compatible" label="Embedded / MinIO / R2 / etc">
```bash
# List all audiobook objects
aws s3 ls "s3://$S3_BUCKET/$S3_PREFIX/audiobooks_v1/" --recursive --endpoint-url "$S3_ENDPOINT"
# Filter to one book id (replace <book-id>)
aws s3 ls "s3://$S3_BUCKET/$S3_PREFIX/audiobooks_v1/" --recursive --endpoint-url "$S3_ENDPOINT" | grep "<book-id>-audiobook/"
# Download one object by full key
aws s3 cp "s3://$S3_BUCKET/$S3_PREFIX/audiobooks_v1/<path>/<file>.m4b" "./audiobook.m4b" --endpoint-url "$S3_ENDPOINT"
```
Embedded default example: `S3_ENDPOINT=http://127.0.0.1:8333` (or your mapped host/port).
</TabItem>
</Tabs>
## TTS Segment Storage
Server-side TTS segment audio is stored in object storage under the `tts_segments_v1` keyspace.
Typical key layout:
- `${S3_PREFIX}/tts_segments_v1/users/<url-encoded-user-id>/docs/<document-id>/<document-version>/<settings-hash>/<segment-id>.mp3`
- `${S3_PREFIX}/tts_segments_v1/ns/<test-namespace>/users/<url-encoded-user-id>/docs/...` (test namespace mode)
Notes:
- For the corresponding SQL metadata model (`tts_segments`), see [Database](./database).
## Account Deletion Cleanup
Account deletion performs best-effort object cleanup:
- Document blobs + preview artifacts
- Audiobook blobs
- TTS segment blobs under `tts_segments_v1`
If object deletion fails, account deletion still proceeds and orphaned objects may require manual cleanup.
## TTS Segment Storage Debug Commands
Use these commands to inspect segment objects.
<Tabs groupId="tts-segment-storage-access-cli">
<TabItem value="aws-s3" label="AWS S3" default>
```bash
# List all TTS segment objects
aws s3 ls "s3://$S3_BUCKET/$S3_PREFIX/tts_segments_v1/" --recursive
# Filter to one document id (replace <document-id>)
aws s3 ls "s3://$S3_BUCKET/$S3_PREFIX/tts_segments_v1/" --recursive | grep "/docs/<document-id>/"
```
</TabItem>
<TabItem value="s3-compatible" label="Embedded / MinIO / R2 / etc">
```bash
# List all TTS segment objects
aws s3 ls "s3://$S3_BUCKET/$S3_PREFIX/tts_segments_v1/" --recursive --endpoint-url "$S3_ENDPOINT"
# Filter to one document id (replace <document-id>)
aws s3 ls "s3://$S3_BUCKET/$S3_PREFIX/tts_segments_v1/" --recursive --endpoint-url "$S3_ENDPOINT" | grep "/docs/<document-id>/"
```
</TabItem>
</Tabs>

View file

@ -0,0 +1,68 @@
---
title: Server Library Import
---
This page documents how server library import works and how to configure it.
## What it does
Server library import lets you browse files from one or more server directories and import selected files into OpenReader.
- Import is user-driven via a selection modal
- Only selected files are imported
- Imported files become normal OpenReader documents
## FS / Volume Mounts
### App data mount
- Target: `/app/docstore`
- Recommended: yes, for persistence
- Purpose: stores app runtime data, metadata DB, and embedded storage state
- Mount string: `-v openreader_docstore:/app/docstore`
### Library source mount
- Target: `/app/docstore/library`
- Recommended: yes for this feature, use read-only (`:ro`)
- Purpose: exposes host files as import candidates in Server Library Import
- Mount string: `-v /path/to/your/library:/app/docstore/library:ro`
## Import flow
1. Open **Settings -> Documents -> Server Library Import**.
2. Select files in the modal.
3. Click **Import**.
Selected files are fetched from the server library endpoint and imported into OpenReader storage.
:::warning Shared Library Roots
Library roots are configured at the server level (not per-user). Any user who can access Server Library Import can browse/import from the same configured roots.
Imported documents are still saved to the importing user's document scope.
:::
## Supported file types
- `.pdf`
- `.epub`
- `.html`, `.htm`
- `.txt`
- `.md`, `.mdown`, `.markdown`
## Optional: Configure Library Roots
You only need this when the default mounted path is not what you want.
By default, OpenReader uses `docstore/library` as the import root. You can override that with environment variables:
- `IMPORT_LIBRARY_DIRS` (takes precedence): multiple roots separated by comma, colon, or semicolon
- `IMPORT_LIBRARY_DIR`: single root
See [Environment Variables](../reference/environment-variables#library-import) for variable details.
## Notes
- Library listing is capped per request (up to 10,000 files).
- When auth is enabled, library import endpoints require a valid session.
- The mounted library is a source; removing it does not delete already imported documents.

View file

@ -0,0 +1,42 @@
---
title: DeepInfra
---
Use DeepInfra's hosted TTS models as your provider.
## Setup
**Recommended (auth + admin): Settings → Admin → Shared providers**
1. Add a shared provider with type `deepinfra`.
2. Keep base URL as `https://api.deepinfra.com/v1/openai`.
3. Enter your API key.
4. Set your preferred default model/voice.
**Legacy bootstrap seed (optional, first boot only):**
```env
API_BASE=https://api.deepinfra.com/v1/openai
API_KEY=your-deepinfra-key
```
**Per-user Settings → TTS Provider (only when `restrictUserApiKeys=false`):**
1. Set provider to `Deepinfra`.
2. The base URL is pre-filled, no changes needed.
3. Enter your `API_KEY`.
4. Choose a model and voice.
See [TTS Providers](../tts-providers) for admin-shared vs per-user behavior.
## Notes
- Available models include `hexgrad/Kokoro-82M` and `canopylabs/orpheus-3b-0.1-ft`.
- Without an API key, only the free-tier model (`hexgrad/Kokoro-82M`) is shown in the dropdown.
- TTS requests are sent from the server, not the browser. The API key is never exposed to clients.
## References
- [DeepInfra TTS models](https://deepinfra.com/models/text-to-speech)
- [TTS Providers](../tts-providers)
- [TTS Environment Variables](../../reference/environment-variables#tts-provider-and-request-behavior)

View file

@ -0,0 +1,47 @@
---
title: KittenTTS-FastAPI
---
Run [KittenTTS-FastAPI](https://github.com/richardr1126/KittenTTS-FastAPI) locally and connect it to OpenReader using the `Custom OpenAI-Like` provider. Lightweight and CPU-friendly.
## Run KittenTTS
```bash
docker run -it --rm \
--name kittentts-fastapi \
-e KITTEN_MODEL_REPO_ID="KittenML/kitten-tts-nano-0.8-fp32" \
-p 8005:8005 \
ghcr.io/richardr1126/kittentts-fastapi-cpu
```
## Connect to OpenReader
**Recommended (auth + admin): Settings → Admin → Shared providers**
1. Add a shared provider with type `custom-openai`.
2. Set base URL to your KittenTTS endpoint (e.g. `http://kittentts-fastapi:8005/v1`).
3. Leave API key blank unless required by your deployment.
4. Set default model to `kitten-tts` (or your backend model id).
**Legacy bootstrap seed (optional, first boot only):**
```env
API_BASE=http://kittentts-fastapi:8005/v1
```
> Use `kittentts-fastapi` if that's the container name, or `host.docker.internal` if not.
**Or in-app via Settings → TTS Provider:**
1. Set provider to `Custom OpenAI-Like`.
2. Set `API_BASE` to your KittenTTS endpoint (e.g. `http://kittentts-fastapi:8005/v1`).
3. Leave `API_KEY` blank unless your deployment requires one.
4. Choose model `kitten-tts` (or the model your deployment exposes).
See [TTS Providers](../tts-providers) for admin-shared vs per-user behavior.
## References
- [richardr1126/KittenTTS-FastAPI](https://github.com/richardr1126/KittenTTS-FastAPI)
- [TTS Providers](../tts-providers)
- [TTS Environment Variables](../../reference/environment-variables#tts-provider-and-request-behavior)

View file

@ -0,0 +1,75 @@
---
title: Kokoro-FastAPI
---
Run [Kokoro-FastAPI](https://github.com/remsky/Kokoro-FastAPI) locally and connect it to OpenReader using the `Custom OpenAI-Like` provider.
:::warning
For Kokoro issues and support, use the upstream repository: [remsky/Kokoro-FastAPI](https://github.com/remsky/Kokoro-FastAPI).
:::
## Run Kokoro
**CPU:**
```bash
docker run --name kokoro-tts \
--restart unless-stopped \
-d \
-p 8880:8880 \
-e ONNX_NUM_THREADS=8 \
-e ONNX_INTER_OP_THREADS=4 \
-e ONNX_EXECUTION_MODE=parallel \
-e ONNX_OPTIMIZATION_LEVEL=all \
-e ONNX_MEMORY_PATTERN=true \
-e ONNX_ARENA_EXTEND_STRATEGY=kNextPowerOfTwo \
-e API_LOG_LEVEL=DEBUG \
ghcr.io/remsky/kokoro-fastapi-cpu:v0.2.4
```
**GPU (NVIDIA):**
```bash
docker run --name kokoro-tts \
--restart unless-stopped \
-d \
--gpus all \
--user 1001:1001 \
-p 8880:8880 \
-e USE_GPU=true \
-e PYTHONUNBUFFERED=1 \
-e API_LOG_LEVEL=DEBUG \
ghcr.io/remsky/kokoro-fastapi-gpu:v0.2.4
```
## Connect to OpenReader
**Recommended (auth + admin): Settings → Admin → Shared providers**
1. Add a shared provider with type `custom-openai`.
2. Set base URL to your Kokoro endpoint (e.g. `http://kokoro-tts:8880/v1`).
3. Leave API key blank unless required by your deployment.
4. Set default model to `Kokoro`.
**Legacy bootstrap seed (optional, first boot only):**
```env
API_BASE=http://kokoro-tts:8880/v1
```
> Use `kokoro-tts` if that's the container name, or `host.docker.internal` if not.
**Or in-app via Settings → TTS Provider:**
1. Set provider to `Custom OpenAI-Like`.
2. Set `API_BASE` to your Kokoro endpoint (e.g. `http://kokoro-tts:8880/v1`).
3. Leave `API_KEY` blank unless your deployment requires one.
4. Choose model `Kokoro`.
See [TTS Providers](../tts-providers) for admin-shared vs per-user behavior.
## References
- [remsky/Kokoro-FastAPI](https://github.com/remsky/Kokoro-FastAPI)
- [TTS Providers](../tts-providers)
- [TTS Environment Variables](../../reference/environment-variables#tts-provider-and-request-behavior)

View file

@ -0,0 +1,41 @@
---
title: OpenAI
---
Use the OpenAI TTS API as your provider.
## Setup
**Recommended (auth + admin): Settings → Admin → Shared providers**
1. Add a shared provider with type `openai`.
2. Keep base URL as `https://api.openai.com/v1`.
3. Enter your API key.
4. Set your preferred default model/voice.
**Legacy bootstrap seed (optional, first boot only):**
```env
API_BASE=https://api.openai.com/v1
API_KEY=sk-...
```
**Per-user Settings → TTS Provider (only when `restrictUserApiKeys=false`):**
1. Set provider to `OpenAI`.
2. The base URL is pre-filled, no changes needed.
3. Enter your `API_KEY`.
4. Choose a model and voice.
See [TTS Providers](../tts-providers) for admin-shared vs per-user behavior.
## Notes
- Models: `tts-1`, `tts-1-hd`, `gpt-4o-mini-tts`
- TTS requests are sent from the server, not the browser. The API key is never exposed to clients.
## References
- [OpenAI TTS pricing](https://platform.openai.com/docs/pricing#transcription-and-speech)
- [TTS Providers](../tts-providers)
- [TTS Environment Variables](../../reference/environment-variables#tts-provider-and-request-behavior)

View file

@ -0,0 +1,41 @@
---
title: Orpheus-FastAPI
---
Run [Orpheus-FastAPI](https://github.com/Lex-au/Orpheus-FastAPI) locally and connect it to OpenReader using the `Custom OpenAI-Like` provider.
## Run Orpheus
Refer to the upstream repository for Docker instructions: [Lex-au/Orpheus-FastAPI](https://github.com/Lex-au/Orpheus-FastAPI).
## Connect to OpenReader
**Recommended (auth + admin): Settings → Admin → Shared providers**
1. Add a shared provider with type `custom-openai`.
2. Set base URL to your Orpheus endpoint (e.g. `http://orpheus:8000/v1`).
3. Leave API key blank unless required by your deployment.
4. Set default model to `Orpheus` (or your backend model id).
**Legacy bootstrap seed (optional, first boot only):**
```env
API_BASE=http://orpheus:8000/v1
```
> Use the container name if that's how it's named, or `host.docker.internal` if not.
**Or in-app via Settings → TTS Provider:**
1. Set provider to `Custom OpenAI-Like`.
2. Set `API_BASE` to your Orpheus endpoint (e.g. `http://orpheus:8000/v1`).
3. Leave `API_KEY` blank unless your deployment requires one.
4. Choose model `Orpheus` (or the model your deployment exposes).
See [TTS Providers](../tts-providers) for admin-shared vs per-user behavior.
## References
- [Lex-au/Orpheus-FastAPI](https://github.com/Lex-au/Orpheus-FastAPI)
- [TTS Providers](../tts-providers)
- [TTS Environment Variables](../../reference/environment-variables#tts-provider-and-request-behavior)

View file

@ -0,0 +1,54 @@
---
title: Other
---
Use any OpenAI-compatible TTS service with OpenReader, including self-hosted servers not covered by a dedicated guide.
## Requirements
Your service only needs an OpenAI-compatible speech endpoint:
- `POST /v1/audio/speech`**required**.
- Voice listing is **optional** and auto-discovered from `/v1/audio/voices`, `/v1/voices`, or `/v1/styles`. If none respond, OpenReader falls back to default voices — the Kokoro voice set for Kokoro models, otherwise the standard OpenAI voices (`alloy`, `echo`, `fable`, `onyx`, `nova`, `shimmer`).
The endpoint may return `mp3`, `wav`, `ogg`, or `flac` — OpenReader normalizes non-mp3 audio to mp3 automatically. An API key is optional.
Known compatible implementations: [Kokoro-FastAPI](./kokoro-fastapi), [KittenTTS-FastAPI](./kitten-tts-fastapi), [Orpheus-FastAPI](./orpheus-fastapi), [Supertonic](./supertonic).
## Setup
**Recommended (auth + admin): Settings → Admin → Shared providers**
1. Add a shared provider with type `custom-openai`.
2. Set `API_BASE` to your service base URL (typically ending in `/v1`).
3. Set API key if your service requires authentication.
4. Set a default model/voice supported by your backend.
**Legacy bootstrap seed (optional, first boot only):**
```env
API_BASE=http://your-tts-server/v1
# API_KEY=optional-key-if-required
```
**Or in-app via Settings → TTS Provider:**
1. Set provider to `Custom OpenAI-Like`.
2. Set `API_BASE` to your service's base URL (typically ending in `/v1`).
3. Set `API_KEY` if your service requires authentication.
4. Choose a model and voice supported by your backend.
See [TTS Providers](../tts-providers) for admin-shared vs per-user behavior.
:::warning TTS requests are server-side
`API_BASE` must be reachable from the **Next.js server**, not just the browser. In Docker, use container names or `host.docker.internal`.
:::
## Troubleshooting
If voices don't load, confirm the server is reachable from the Next.js runtime and that at least one of `/v1/audio/voices`, `/v1/voices`, or `/v1/styles` returns a valid response. If none do, OpenReader falls back to default voices — synthesis still works as long as `POST /v1/audio/speech` succeeds.
## References
- [TTS Providers](../tts-providers)
- [TTS Environment Variables](../../reference/environment-variables#tts-provider-and-request-behavior)

View file

@ -0,0 +1,48 @@
---
title: Replicate
---
Use Replicate's hosted TTS models as your provider.
## Setup
**Recommended (auth + admin): Settings → Admin → Shared providers**
1. Add a shared provider with type `replicate`.
2. Enter your API key.
3. Set default model to:
`alphanumericuser/kokoro-82m:89b6fa84e4fa2dd6bd3a96be3e1f12827a3516c9fda8fddbac7a0be131c9a6f5` (or your preferred model).
**Legacy bootstrap seed (optional, first boot only):**
```env
API_KEY=r8_...
```
Then update the shared provider's **Default model** in **Settings → Admin → Shared providers**.
**Per-user Settings → TTS Provider (only when `restrictUserApiKeys=false`):**
1. Set provider to `Replicate`.
2. Enter your `API_KEY`.
3. Choose a model and voice.
See [TTS Providers](../tts-providers) for admin-shared vs per-user behavior.
## Notes
- Built-in Replicate models:
- `alphanumericuser/kokoro-82m:89b6fa84e4fa2dd6bd3a96be3e1f12827a3516c9fda8fddbac7a0be131c9a6f5`
- `google/gemini-3.1-flash-tts`
- `minimax/speech-2.8-turbo`
- `qwen/qwen3-tts`
- `inworld/tts-1.5-mini`
- You can also choose `Other` and enter any Replicate model ID (for example `owner/model-name` or `owner/model-name:version`).
- Native model speed is not available on all Replicate models; OpenReader hides/disables native speed controls where unsupported.
- TTS requests are sent from the server, not the browser. The API key is never exposed to clients.
## References
- [Replicate](https://replicate.com/explore)
- [TTS Providers](../tts-providers)
- [TTS Environment Variables](../../reference/environment-variables#tts-provider-and-request-behavior)

View file

@ -0,0 +1,63 @@
---
title: Speech SDK
---
Use [speech-sdk](https://github.com/Jellypod-Inc/speech-sdk) (Apache 2.0) to reach additional cloud TTS providers (ElevenLabs, Cartesia, Hume, Deepgram, Google Gemini TTS, Inworld, and more) with your own provider API keys. Requests go from the OpenReader server directly to the provider's API; no extra account or proxy is involved.
Models use the `provider/model` format. The API key you enter belongs to the provider named by the model prefix: for `elevenlabs/eleven_multilingual_v2` enter an ElevenLabs key, for `cartesia/sonic-3.5` a Cartesia key, and so on.
## Setup
**Recommended (auth + admin): Settings → Admin → Shared providers**
1. Add a shared provider with type `speech-sdk`.
2. Enter the API key for the provider you want to use.
3. Set default model to a matching `provider/model` (for example `elevenlabs/eleven_multilingual_v2`).
**Per-user Settings → TTS Provider (only when `restrictUserApiKeys=false`):**
1. Set provider to `Speech SDK`.
2. Choose a model; enter the API key for that model's provider.
3. Choose a voice.
See [TTS Providers](../tts-providers) for admin-shared vs per-user behavior.
## Built-in models
- `openai/gpt-4o-mini-tts` (works with your existing OpenAI API key)
- `elevenlabs/eleven_multilingual_v2`
- `cartesia/sonic-3.5`
- `deepgram/aura-2`
- `google/gemini-2.5-flash-preview-tts`
- `inworld/inworld-tts-1.5-max`
You can also choose `Other` and enter any `provider/model` the SDK supports. Recognized prefixes: `openai`, `elevenlabs`, `cartesia`, `hume`, `deepgram`, `google`, `inworld`, `minimax`, `fish-audio`, `murf`, `resemble`, `fal-ai`, `mistral`, `xai`, `smallest-ai`.
## Voice IDs
ElevenLabs and Cartesia identify voices by opaque IDs. The built-in lists map to these shared library voices:
| ElevenLabs ID | Name | | Cartesia ID | Name |
| --- | --- | --- | --- | --- |
| `JBFqnCBsd6RMkjVDRZzb` | George | | `a0e99841-438c-4a64-b679-ae501e7d6091` | Barbershop Man |
| `IKne3meq5aSn9XLyUdCD` | Charlie | | `156fb8d2-335b-4950-9cb3-a2d33f0c0c2a` | British Lady |
| `XB0fDUnXU5powFXDhCwa` | Charlotte | | `694f9389-aac1-45b6-b726-9d9369183238` | California Girl |
| `Xb7hH8MSUJpSbSDYk0k2` | Alice | | `87748186-23bb-4571-8b8b-a73da9bf9c4f` | Commercial Lady |
| `iP95p4xoKVk53GoZ742B` | Chris | | `ee7ea9f8-c0c1-498c-9f62-dc2da49a6f98` | Friendly Reading Man |
| `nPczCjzI2devNBz1zQrb` | Brian | | `248be419-c632-4f23-adf1-5324ed7dbf1d` | Hannah |
| `onwK4e9ZLuTAKqWW03F9` | Daniel | | | |
| `pFZP5JQG7iQjIQuC4Bku` | Lily | | | |
| `pqHfZKP75CvOlQylNhV4` | Bill | | | |
## Notes
- One voice per request; Kokoro-style multi-voice mixing does not apply to this provider.
- Playback speed is applied client-side, so cached audio segments stay valid when you change speed.
- Providers without a built-in voice list fall back to a `default` entry, which lets the provider pick its default voice.
- Word-by-word highlighting works the same as with every other provider (alignment runs in OpenReader, not the provider).
- TTS requests are sent from the server, not the browser. The API key is never exposed to clients.
## References
- [speech-sdk on GitHub](https://github.com/Jellypod-Inc/speech-sdk)
- [TTS Providers](../tts-providers)

View file

@ -0,0 +1,68 @@
---
title: Supertonic
---
Run [Supertonic](https://github.com/supertone-inc/supertonic-py) locally and connect it to OpenReader using the `Custom OpenAI-Like` provider. Supertonic is a fast, on-device TTS engine that ships its own OpenAI-compatible HTTP server.
:::note No Docker image
Supertonic does not publish a Docker image — it installs as a Python package and runs as a local HTTP server. These instructions assume OpenReader itself runs in Docker (the common case); see [Running OpenReader directly on the host](#running-openreader-directly-on-the-host) if you don't.
:::
## Run Supertonic
Install with `pip` (or `pipx` for an isolated install) and start the server:
```bash
pipx install 'supertonic[serve]' # or: pip install 'supertonic[serve]'
supertonic serve # defaults; loopback only
```
The first run downloads the model (~400MB). Once it's up, the OpenAI-compatible endpoint is at `http://127.0.0.1:7788/v1/audio/speech` and interactive docs are at `http://127.0.0.1:7788/docs`.
- **Models:** `supertonic-3` (default) or `supertonic-2`.
- **Voices:** built-ins `M1``M5` and `F1``F5`, plus any custom voices you import. OpenReader discovers them automatically via the `/v1/styles` endpoint.
- **Audio format:** Supertonic emits `wav` by default; OpenReader transcodes it to mp3 transparently, so no extra configuration is needed.
## Connect to OpenReader
From a Docker container, your host machine is reachable at `host.docker.internal`, so the base URL is `http://host.docker.internal:7788/v1`. On Docker Desktop (macOS/Windows) this reaches the loopback-bound server above with no extra setup.
:::note Linux (native Docker Engine)
On native Linux Docker, `host.docker.internal` needs `--add-host=host.docker.internal:host-gateway` on the OpenReader container (or the equivalent `extra_hosts` entry in `docker-compose.yml`), and it routes to the host's bridge interface rather than loopback. Pick one:
- Run the OpenReader container with `--network host` (or `network_mode: host`), keep Supertonic on `--host 127.0.0.1`, and use `http://127.0.0.1:7788/v1` as the base URL.
- Or start Supertonic with `--host 0.0.0.0` so the bridge can reach it — keep it on a trusted network or behind a firewall.
:::
**Recommended (auth + admin): Settings → Admin → Shared providers**
1. Add a shared provider with type `custom-openai`.
2. Set base URL to `http://host.docker.internal:7788/v1`.
3. Leave API key blank — `supertonic serve` does not require one.
4. Set default model to `supertonic-3` (or `supertonic-2`).
**Legacy bootstrap seed (optional, first boot only):**
```env
API_BASE=http://host.docker.internal:7788/v1
```
**Or in-app via Settings → TTS Provider:**
1. Set provider to `Custom OpenAI-Like`.
2. Set `API_BASE` to `http://host.docker.internal:7788/v1`.
3. Leave `API_KEY` blank.
4. Choose model `supertonic-3` (or the model your deployment exposes).
See [TTS Providers](../tts-providers) for admin-shared vs per-user behavior.
## Running OpenReader directly on the host
If OpenReader runs on the same machine (e.g. `pnpm dev`) rather than in Docker, skip `host.docker.internal` and use `http://127.0.0.1:7788/v1` as the base URL everywhere above.
## References
- [supertone-inc/supertonic-py](https://github.com/supertone-inc/supertonic-py)
- [Supported Languages](https://github.com/supertone-inc/supertonic-py#supported-languages)
- [TTS Providers](../tts-providers)
- [TTS Environment Variables](../../reference/environment-variables#tts-provider-and-request-behavior)

View file

@ -0,0 +1,63 @@
---
title: TTS Providers
---
OpenReader routes all TTS requests through the Next.js server to an OpenAI-compatible API. There are three places provider configuration can live:
**Admin-managed shared providers** (Settings > Admin > Shared providers): DB-backed instances configured by an admin and visible to all users. Keys are encrypted at rest and never exposed to the client. Available only when [auth is enabled](./auth) and your account is in `ADMIN_EMAILS`. See [Admin Panel](./admin-panel).
**Per-user Settings modal** (Settings > TTS Provider): provider + API key stored in the user's browser and sent with every TTS request. This path is available only when the admin/runtime setting `restrictUserApiKeys=false`.
**Environment variables**: `API_KEY` and `API_BASE` exist as a one-shot first-boot seed that auto-creates a `default-openai` admin shared provider. After the first boot they are no longer read by the running app.
:::tip
If you're running a private/self-hosted instance and want per-user BYOK behavior, turn off **Settings → Admin → Site features → Restrict user API keys**. For first-boot automation, set `runtimeConfig.restrictUserApiKeys=false` in `RUNTIME_SEED_JSON` / `RUNTIME_SEED_JSON_PATH`.
:::
## Providers
- **OpenAI**: Cloud. Base URL pre-filled (`https://api.openai.com/v1`). API key required.
- **Replicate**: Cloud. Base URL managed internally by OpenReader. API key required.
- **DeepInfra**: Cloud. Base URL pre-filled (`https://api.deepinfra.com/v1/openai`). API key required.
- **Speech SDK**: Cloud. Reaches additional providers (ElevenLabs, Cartesia, Hume, Deepgram, Google, Inworld, and more) directly with your own provider API keys via [speech-sdk](./tts-provider-guides/speech-sdk). No base URL. API key required (the key for the model's provider).
- **Custom OpenAI-Like**: Self-hosted or any custom endpoint. `API_BASE` must be set manually (typically ending in `/v1`). API key optional.
For `OpenAI`, `DeepInfra`, and `Replicate` you only need to supply an API key. For `Custom OpenAI-Like` you must also set `API_BASE`.
## Built-in model catalogs
- **Replicate** models: `alphanumericuser/kokoro-82m`, `google/gemini-3.1-flash-tts`, `minimax/speech-2.8-turbo`, `qwen/qwen3-tts`, `inworld/tts-1.5-mini` (or choose `Other` and enter any Replicate model ID, such as `owner/model` or `owner/model:version`)
- **OpenAI** models: `tts-1`, `tts-1-hd`, `gpt-4o-mini-tts`
- **DeepInfra** models: includes `hexgrad/Kokoro-82M` and additional hosted models (depending on API key / feature flags)
- **Speech SDK** models: `openai/gpt-4o-mini-tts`, `elevenlabs/eleven_multilingual_v2`, `cartesia/sonic-3.5`, `deepgram/aura-2`, `google/gemini-2.5-flash-preview-tts`, `inworld/inworld-tts-1.5-max` (or choose `Other` and enter any `provider/model` the SDK supports)
## Custom provider requirements
Self-hosted or custom providers only need an OpenAI-compatible speech endpoint:
- `POST /v1/audio/speech`**required**.
- Voice listing is **optional** and auto-discovered: OpenReader probes `/v1/audio/voices`, `/v1/voices`, then `/v1/styles`. If none respond, it falls back to default voices — the Kokoro set for Kokoro models, otherwise the standard OpenAI voices (`alloy`, `echo`, `fable`, `onyx`, `nova`, `shimmer`).
The speech endpoint may return any common audio format — `mp3`, `wav`, `ogg`, or `flac`. OpenReader detects the format and transcodes non-mp3 audio to mp3 automatically, so your server does not need to honor `response_format: mp3`. An API key is optional; keyless servers work.
:::warning TTS requests are server-side
TTS requests originate from the **Next.js server**, not the browser. `API_BASE` must be reachable from the server runtime. In Docker, use container names or `host.docker.internal` rather than `localhost`.
:::
## Provider guides
- [Kokoro-FastAPI](./tts-provider-guides/kokoro-fastapi)
- [KittenTTS-FastAPI](./tts-provider-guides/kitten-tts-fastapi)
- [Orpheus-FastAPI](./tts-provider-guides/orpheus-fastapi)
- [Supertonic](./tts-provider-guides/supertonic)
- [Replicate](./tts-provider-guides/replicate)
- [DeepInfra](./tts-provider-guides/deepinfra)
- [OpenAI](./tts-provider-guides/openai)
- [Speech SDK](./tts-provider-guides/speech-sdk)
- [Other](./tts-provider-guides/other)
## Related
- [Admin Panel](./admin-panel) — DB-backed shared providers with encrypted keys
- [TTS Environment Variables](../reference/environment-variables#tts-provider-and-request-behavior)
- [TTS Rate Limiting](./tts-rate-limiting)

View file

@ -0,0 +1,42 @@
---
title: TTS Rate Limiting
---
This page explains OpenReader's TTS character rate limiting controls.
## Overview
- TTS rate limiting is disabled by default.
- Primary control is **Settings → Admin → Site features → Disable TTS daily rate limiting**.
- Limits are enforced per day in UTC.
- Enforcement applies only when auth is enabled.
## How enforcement works
When enabled, OpenReader enforces:
- Per-user daily character limits.
- IP backstop daily character limits.
- Anonymous device backstop tracking (cookie-based) to reduce limit resets.
If a request exceeds the active limit, the TTS API returns `429` with reset metadata for the next UTC day.
## Required auth behavior
- Auth must be enabled (`BASE_URL` + `AUTH_SECRET`) for TTS char limits to apply.
- If auth is disabled, TTS character limits are effectively unlimited.
- `DISABLE_AUTH_RATE_LIMIT` only affects Better Auth's own request throttling.
- `DISABLE_AUTH_RATE_LIMIT` does not disable TTS character limits.
## Runtime config
- `disableTtsRateLimit` default: `true`
- Per-user and IP backstop limit values are configured in **Settings → Admin → Site features** and stored in DB runtime settings.
- Optional first-boot seeding can be done via `RUNTIME_SEED_JSON` / `RUNTIME_SEED_JSON_PATH` (`runtimeConfig.disableTtsRateLimit`).
## Related docs
- TTS/rate-limit environment variables: [Environment Variables](../reference/environment-variables#tts-provider-and-request-behavior)
- PDF parsing rate limiting (separate, compute-side throttle): [Admin Panel → Rate limiting](./admin-panel#rate-limiting)
- Auth configuration: [Auth](./auth)
- Provider setup: [TTS Providers](./tts-providers)

View file

@ -0,0 +1,134 @@
---
title: Compute Worker
description: Deploy the standalone worker used for Whisper alignment and PDF layout parsing.
---
Use this guide when OpenReader runs compute as a separate service. For the default embedded/local flow (`pnpm dev` or `pnpm start` without `COMPUTE_WORKER_URL`), configure the root `.env` instead and see [Local Development](./local-development).
## What the worker does
- Runs Whisper word alignment jobs
- Runs PDF layout parsing jobs
- Stores durable job state in NATS JetStream and NATS KV
The app server submits resource-specific operations under `/v1` and listens for updates on
`GET /v1/operations/:opId/events`.
## When to use it
- Required for Vercel-style deployments where heavy compute must run outside the app server
- Useful when you want a dedicated compute host
- Not needed for the default embedded local flow
## Container image
- `ghcr.io/richardr1126/openreader-compute-worker:latest`
## Worker environment
Required worker variables:
```env
COMPUTE_WORKER_TOKEN=...
NATS_URL=nats://...
S3_BUCKET=...
S3_REGION=...
S3_ACCESS_KEY_ID=...
S3_SECRET_ACCESS_KEY=...
```
:::important
`compute-worker/.env*` is only for standalone worker deployments.
- Embedded/local mode: configure the root `.env` only.
- External worker mode: set `COMPUTE_WORKER_URL` and `COMPUTE_WORKER_TOKEN` on the app, and worker runtime values on the worker service.
- Keep shared values aligned across app and worker: `COMPUTE_WORKER_TOKEN`, `S3_*`, `COMPUTE_WHISPER_TIMEOUT_MS`, `COMPUTE_PDF_TIMEOUT_MS`, `COMPUTE_PDF_JOB_ATTEMPTS`, and `COMPUTE_OP_STALE_MS`.
:::
Common optional variables:
- `NATS_CREDS` or `NATS_CREDS_FILE`
- `S3_ENDPOINT`, `S3_FORCE_PATH_STYLE=true`, `S3_PREFIX=openreader`
- `COMPUTE_WORKER_HOST=0.0.0.0`
- `PORT=8081` for local/manual runs. Platforms like Railway usually inject `PORT`.
- `LOG_FORMAT=json` and `COMPUTE_LOG_LEVEL=info`
- `COMPUTE_PREWARM_MODELS=false` by default. Set it to `true` to pre-download ONNX models during worker startup.
- `COMPUTE_JOB_CONCURRENCY=1`
- `COMPUTE_WHISPER_TIMEOUT_MS=30000`
- `COMPUTE_PDF_TIMEOUT_MS=300000`
- `COMPUTE_PDF_JOB_ATTEMPTS=1`
- `COMPUTE_JOBS_STREAM_MAX_BYTES=268435456`
- `COMPUTE_EVENTS_STREAM_MAX_BYTES=134217728`
- `COMPUTE_JOB_STATES_MAX_BYTES=67108864`
- `COMPUTE_NATS_REPLICAS=1`
- `COMPUTE_OP_STALE_MS=1800000`
- `WHISPER_MODEL_BASE_URL`
- `PDF_LAYOUT_MODEL_BASE_URL`
If you need the broader app config reference, see [Environment Variables](../reference/environment-variables).
## App server environment
Set these on the Next.js app server:
```env
COMPUTE_WORKER_URL=https://worker.example.com
COMPUTE_WORKER_TOKEN=<same-token-as-worker>
# Optional shared overrides:
# COMPUTE_WHISPER_TIMEOUT_MS=30000
# COMPUTE_PDF_TIMEOUT_MS=300000
# COMPUTE_PDF_JOB_ATTEMPTS=1
# COMPUTE_OP_STALE_MS=1800000
```
Notes:
- Model artifact overrides (`WHISPER_MODEL_BASE_URL`, `PDF_LAYOUT_MODEL_BASE_URL`) belong on the worker service, not the app server.
- There is no app-local compute fallback once `COMPUTE_WORKER_URL` is set. If the worker is unavailable, worker-backed requests fail.
## Deployment notes
- App and worker must share the same object storage.
- Embedded `weed mini` is not supported for external worker mode.
- Protect `COMPUTE_WORKER_TOKEN` and do not expose worker routes without auth.
- The worker connects to NATS lazily and disconnects after 120 seconds of full idle time. That allows platforms like Railway to sleep the service, but the first request after a cold start will be slower.
## Health endpoints
- `GET /health/live` returns `{ ok: true }`.
- `GET /health/ready` returns `{ ok: true, natsConnected }` and reflects the current NATS session without forcing a reconnect.
## Railway + Synadia example
Deploy the worker image to Railway and set worker env vars similar to:
```env
COMPUTE_WORKER_HOST=0.0.0.0
COMPUTE_WORKER_TOKEN=<shared-token>
NATS_URL=tls://connect.ngs.global:4222
NATS_CREDS="-----BEGIN NATS USER JWT-----
...
------END USER NKEY SEED------"
S3_BUCKET=<bucket>
S3_REGION=<region>
S3_ACCESS_KEY_ID=<key>
S3_SECRET_ACCESS_KEY=<secret>
# Optional:
# S3_ENDPOINT=https://...
# S3_FORCE_PATH_STYLE=true
# S3_PREFIX=openreader
```
If your platform supports mounted files, you can use `NATS_CREDS_FILE` instead of `NATS_CREDS`.
Set these on the OpenReader app server:
```env
COMPUTE_WORKER_URL=https://<railway-worker-domain>
COMPUTE_WORKER_TOKEN=<same-token-as-worker>
```
Verify the worker after deploy:
- `GET https://<railway-worker-domain>/health/live`
- `GET https://<railway-worker-domain>/health/ready`

View file

@ -0,0 +1,175 @@
---
title: Docker Compose
description: Run OpenReader with the slim, full, local-slim, or local-full Docker Compose examples.
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
Use these examples to run OpenReader with Kokoro-FastAPI and persistent storage. Choose the slim
stack for the simplest deployment, or the full stack when you want PostgreSQL, SeaweedFS, NATS,
and the compute worker as separate containers. Local build variants are also available for both slim
and full stacks to build the application from your current checkout.
## Prerequisites
- A recent Docker version with Docker Compose
- A clone of the OpenReader repository
```bash
git clone https://github.com/richardr1126/openreader.git
cd openreader
```
## Choose a stack
<Tabs groupId="docker-compose-stack">
<TabItem value="slim" label="Slim" default>
The default slim example runs:
- OpenReader with embedded SeaweedFS, NATS, compute worker, and SQLite
- Kokoro-FastAPI as a companion container
```bash
docker compose -f docker/examples/compose.yml up
# Repository convenience command: pnpm compose
```
Compose file: [`docker/examples/compose.yml`](https://github.com/richardr1126/openreader/blob/main/docker/examples/compose.yml)
</TabItem>
<TabItem value="full" label="Full">
The full example runs OpenReader, Kokoro-FastAPI, PostgreSQL, SeaweedFS, NATS, and the compute
worker as separate containers using published images.
```bash
docker compose -f docker/examples/compose.full.yml up
# Repository convenience command: pnpm compose:full
```
Compose file: [`docker/examples/compose.full.yml`](https://github.com/richardr1126/openreader/blob/main/docker/examples/compose.full.yml)
For details about running the worker separately, see
[Compute Worker](./compute-worker).
</TabItem>
<TabItem value="local-slim" label="Local Slim">
The local-slim example runs a slim setup (OpenReader and Kokoro-FastAPI), but builds the OpenReader app image from the current checkout.
```bash
docker compose -f docker/examples/compose.local-slim.yml up --build
# Repository convenience command: pnpm compose:local
```
Compose file: [`docker/examples/compose.local-slim.yml`](https://github.com/richardr1126/openreader/blob/main/docker/examples/compose.local-slim.yml)
</TabItem>
<TabItem value="local-full" label="Local Full">
The local-full example uses the full multi-container layout, but builds the OpenReader app and compute-worker images from the current checkout.
```bash
docker compose -f docker/examples/compose.local-full.yml up --build
# Repository convenience command: pnpm compose:local:full
```
Compose file: [`docker/examples/compose.local-full.yml`](https://github.com/richardr1126/openreader/blob/main/docker/examples/compose.local-full.yml)
</TabItem>
</Tabs>
## Included services
| Service | Slim | Full | Local Slim | Local Full |
| --- | --- | --- | --- | --- |
| OpenReader | Published image | Published image | Local build | Local build |
| Kokoro-FastAPI | Container | Container | Container | Container |
| Database | Embedded SQLite | PostgreSQL container | Embedded SQLite | PostgreSQL container |
| SeaweedFS | Embedded | Container | Embedded | Container |
| NATS | Embedded | Container | Embedded | Container |
| Compute worker | Embedded | Published image | Embedded | Local build |
On first boot, `RUNTIME_SEED_JSON` creates an enabled Kokoro shared provider and selects it as the
default TTS provider.
## Endpoints
- OpenReader: `http://localhost:3003`
- SeaweedFS S3: `http://localhost:8333`
- Kokoro-FastAPI: `http://localhost:8880`
In the full examples, PostgreSQL, the compute worker, and NATS remain internal to the Compose
network.
## LAN access
Set `BASE_URL` and `S3_ENDPOINT` to the Docker host's LAN IP so browser-facing app and presigned
S3 URLs are reachable from other devices:
<Tabs groupId="docker-compose-lan-stack">
<TabItem value="slim" label="Slim" default>
```bash
BASE_URL=http://192.168.0.XXX:3003 \
S3_ENDPOINT=http://192.168.0.XXX:8333 \
docker compose -f docker/examples/compose.yml up
# Repository convenience command: pnpm compose
```
</TabItem>
<TabItem value="full" label="Full">
```bash
BASE_URL=http://192.168.0.XXX:3003 \
S3_ENDPOINT=http://192.168.0.XXX:8333 \
docker compose -f docker/examples/compose.full.yml up
# Repository convenience command: pnpm compose:full
```
</TabItem>
<TabItem value="local-slim" label="Local Slim">
```bash
BASE_URL=http://192.168.0.XXX:3003 \
S3_ENDPOINT=http://192.168.0.XXX:8333 \
docker compose -f docker/examples/compose.local-slim.yml up --build
# Repository convenience command: pnpm compose:local
```
</TabItem>
<TabItem value="local-full" label="Local Full">
```bash
BASE_URL=http://192.168.0.XXX:3003 \
S3_ENDPOINT=http://192.168.0.XXX:8333 \
docker compose -f docker/examples/compose.local-full.yml up --build
# Repository convenience command: pnpm compose:local:full
```
</TabItem>
</Tabs>
Replace `192.168.0.XXX` with your Docker host's LAN IP and allow inbound TCP ports `3003` and
`8333` through its firewall.
:::info Internal full-stack endpoint
The full and local-full compute workers continue using `http://seaweedfs:8333` internally.
`S3_ENDPOINT` configures the app endpoint and browser-facing presigned URLs.
:::
## Configuration
The examples use local-only default credentials. Override existing `${VARIABLE}` values through
your shell environment before using them beyond local development.
:::warning Protect public deployments
Replace the default `AUTH_SECRET`, PostgreSQL credentials, S3 credentials, and compute-worker
token before exposing a stack outside your trusted local network.
:::
For the complete configuration reference, see
[Environment Variables](../reference/environment-variables). See [Database](../configure/database)
for PostgreSQL and SQLite behavior.

View file

@ -0,0 +1,338 @@
---
title: Local Development
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
## Prerequisites
<details>
<summary><strong>Node.js + pnpm (required)</strong></summary>
<Tabs groupId="local-dev-node-pnpm-os">
<TabItem value="macos" label="macOS" default>
```bash
brew install nvm pnpm
mkdir -p ~/.nvm
echo 'export NVM_DIR="$HOME/.nvm"' >> ~/.zshrc
echo '[ -s "$(brew --prefix nvm)/nvm.sh" ] && . "$(brew --prefix nvm)/nvm.sh"' >> ~/.zshrc
source ~/.zshrc
nvm install --lts
nvm use --lts
node -v
pnpm -v
```
</TabItem>
<TabItem value="linux" label="Linux">
```bash
# Debian/Ubuntu example
sudo apt update
sudo apt install -y curl
curl -fsSL https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.3/install.sh | bash
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"
nvm install --lts
nvm use --lts
corepack enable
corepack prepare pnpm@latest --activate
node -v
pnpm -v
```
</TabItem>
</Tabs>
</details>
<details>
<summary><strong>SeaweedFS <code>weed</code> binary (required unless using external S3)</strong></summary>
<Tabs groupId="local-dev-seaweed-os">
<TabItem value="macos" label="macOS" default>
```bash
brew install seaweedfs
weed version
```
:::warning SeaweedFS Compatibility Note (April 16, 2026)
If you see intermittent S3 `InternalError` upload failures with embedded storage, use SeaweedFS `4.18`.
OpenReader currently pins `4.18` in CI and Docker builds while `4.19` compatibility is investigated.
:::
</TabItem>
<TabItem value="linux" label="Linux">
```bash
# Linux amd64 example (pin 4.18)
mkdir -p "$HOME/.local/bin"
curl -fsSL -o /tmp/seaweedfs.tar.gz \
https://github.com/seaweedfs/seaweedfs/releases/download/4.18/linux_amd64.tar.gz
tar -xzf /tmp/seaweedfs.tar.gz -C /tmp weed
install -m 0755 /tmp/weed "$HOME/.local/bin/weed"
echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc
export PATH="$HOME/.local/bin:$PATH"
weed version
```
:::warning SeaweedFS Compatibility Note (April 16, 2026)
If you see intermittent S3 `InternalError` upload failures with embedded storage, use SeaweedFS `4.18`.
OpenReader currently pins `4.18` in CI and Docker builds while `4.19` compatibility is investigated.
:::
</TabItem>
</Tabs>
</details>
<details>
<summary><strong>NATS Server <code>nats-server</code> (required for embedded compute mode)</strong></summary>
If `COMPUTE_WORKER_URL` is unset, startup launches embedded compute worker + NATS, so `nats-server` must be available on host PATH.
If you always use an external worker (`COMPUTE_WORKER_URL` set), this is not required.
<Tabs groupId="local-dev-nats-os">
<TabItem value="macos" label="macOS" default>
```bash
brew install nats-server
nats-server -v
```
</TabItem>
<TabItem value="linux" label="Linux">
```bash
# Linux amd64 example
mkdir -p "$HOME/.local/bin"
curl -fsSL -o /tmp/nats-server.zip \
https://github.com/nats-io/nats-server/releases/latest/download/nats-server-v2.12.1-linux-amd64.zip
unzip -j /tmp/nats-server.zip '*/nats-server' -d /tmp
install -m 0755 /tmp/nats-server "$HOME/.local/bin/nats-server"
echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc
export PATH="$HOME/.local/bin:$PATH"
nats-server -v
```
</TabItem>
</Tabs>
</details>
<details>
<summary><strong>LibreOffice (optional, for DOCX conversion)</strong></summary>
<Tabs groupId="local-dev-libreoffice-os">
<TabItem value="macos" label="macOS" default>
```bash
brew install libreoffice
```
</TabItem>
<TabItem value="linux" label="Linux">
```bash
# Debian/Ubuntu example
sudo apt update
sudo apt install -y libreoffice
```
</TabItem>
</Tabs>
</details>
<details>
<summary><strong>Word-by-word highlighting (optional)</strong></summary>
No extra native Whisper CLI build step is required.
Word-by-word highlighting and PDF layout parsing are worker-backed in current releases.
If you need mirrors or pinned artifact locations, set `WHISPER_MODEL_BASE_URL` in `.env` (current defaults expect q4 Whisper files at that base URL).
</details>
:::tip Docker Compose
To run OpenReader and Kokoro-FastAPI with Docker Compose, including slim, full, and local-build
options, see [Docker Compose](./docker-compose).
:::
## Steps
### Required flow
1. Clone the repository.
```bash
git clone https://github.com/richardr1126/openreader.git
cd openreader
```
2. Install dependencies.
```bash
pnpm i
```
3. Configure the environment.
```bash
cp .env.example .env
```
Then edit `.env`.
Default embedded worker flow (no external worker URL):
```env
# Leave COMPUTE_WORKER_URL unset.
# Entry point auto-starts embedded worker+NATS when available.
```
External worker flow:
```env
COMPUTE_WORKER_URL=http://localhost:8081
COMPUTE_WORKER_TOKEN=<same-token-used-by-worker>
```
Use the same ownership split:
- root `.env`: app routing/auth (`COMPUTE_WORKER_URL`, `COMPUTE_WORKER_TOKEN`) plus optional shared timeout/stale overrides
- `compute-worker/.env*` (or worker platform env): worker runtime variables (`NATS_*`, `S3_*`, model base URLs, worker tuning)
Use one of these `.env` mode templates:
<Tabs groupId="local-env-modes">
<TabItem value="auth-enabled" label="Auth Enabled" default>
```env
API_BASE=http://host.docker.internal:8880/v1
BASE_URL=http://localhost:3003
AUTH_SECRET=<generate-with-openssl-rand-base64-32>
# Optional when you need multiple local origins:
# AUTH_TRUSTED_ORIGINS=http://localhost:3003,http://127.0.0.1:3003
```
</TabItem>
<TabItem value="auth-with-admin" label="Auth + Admin Panel">
```env
# API_BASE and optional API_KEY are seeded into the admin "default-openai" shared provider
# on first boot, then no longer read. Manage them in Settings → Admin afterwards.
API_BASE=http://host.docker.internal:8880/v1
BASE_URL=http://localhost:3003
AUTH_SECRET=<generate-with-openssl-rand-base64-32>
# Comma-separated emails to auto-promote to admin on signin.
ADMIN_EMAILS=you@example.com
```
</TabItem>
<TabItem value="external-s3" label="External S3">
```env
API_BASE=http://host.docker.internal:8880/v1
USE_EMBEDDED_WEED_MINI=false
BASE_URL=http://localhost:3003
AUTH_SECRET=<generate-with-openssl-rand-base64-32>
S3_BUCKET=your-bucket
S3_REGION=us-east-1
S3_ACCESS_KEY_ID=your-access-key
S3_SECRET_ACCESS_KEY=your-secret-key
# Optional for non-AWS providers:
# S3_ENDPOINT=https://your-s3-compatible-endpoint
# S3_FORCE_PATH_STYLE=true
```
</TabItem>
<TabItem value="worker-mode" label="External Worker Service">
```env
API_BASE=http://host.docker.internal:8880/v1
BASE_URL=http://localhost:3003
AUTH_SECRET=<generate-with-openssl-rand-base64-32>
COMPUTE_WORKER_URL=http://localhost:8081
COMPUTE_WORKER_TOKEN=<same-token-used-by-worker>
USE_EMBEDDED_WEED_MINI=false
S3_BUCKET=your-bucket
S3_REGION=us-east-1
S3_ACCESS_KEY_ID=your-access-key
S3_SECRET_ACCESS_KEY=your-secret-key
# Optional for non-AWS providers:
# S3_ENDPOINT=https://your-s3-compatible-endpoint
# S3_FORCE_PATH_STYLE=true
```
</TabItem>
</Tabs>
:::note Env vars vs. admin panel
On first boot, `API_KEY` / `API_BASE` can bootstrap `default-openai`, and `RUNTIME_SEED_JSON` / `RUNTIME_SEED_JSON_PATH` can seed runtime config + providers. After that, the admin UI is authoritative and editing bootstrap env vars no longer changes app behavior. See [Admin Panel](../configure/admin-panel).
:::
:::note User BYOK restriction default
If you want each user to enter personal provider credentials, set `restrictUserApiKeys=false` (from **Settings → Admin**, or by seeding `runtimeConfig.restrictUserApiKeys=false` in runtime seed JSON).
:::
:::info
For all environment variables, see [Environment Variables](../reference/environment-variables).
:::
See [Auth](../configure/auth) for app/auth behavior.
See [Admin Panel](../configure/admin-panel) for the shared-provider and feature-flag management UI.
Storage configuration details are in [Object / Blob Storage](../configure/object-blob-storage).
Refer to [Database](../configure/database) for database modes.
Learn about migration behavior and commands in [Migrations](../configure/migrations).
:::info Scheduled maintenance tasks
Local and self-hosted Node.js deployments start the scheduled-task loop in-process and check for due work once per minute. No `CRON_SECRET` is required unless you intentionally invoke the cron HTTP route yourself. Manage task intervals and inspect failures from **Settings → Admin → Scheduled tasks**.
:::
4. Start the app.
<Tabs groupId="local-run-mode">
<TabItem value="dev" label="Dev (recommended)" default>
```bash
pnpm dev
```
If you use embedded worker startup (no `COMPUTE_WORKER_URL`) and the host is missing `nats-server`,
install `nats-server` locally or switch to external worker mode.
</TabItem>
<TabItem value="prod" label="Build + Start">
```bash
pnpm build
pnpm start
```
</TabItem>
</Tabs>
:::warning API Base Reachability
`API_BASE` must be reachable from the Next.js server process, not just your browser.
:::
Visit [http://localhost:3003](http://localhost:3003).
### Optional workflows
Run manual DB migrations only for troubleshooting or explicit migration workflows:
- Migrations run automatically on startup through the shared entrypoint for both `pnpm dev` and `pnpm start`.
```bash
pnpm migrate
```
:::info
If `POSTGRES_URL` is set, migrations target Postgres; otherwise local SQLite is used. To disable automatic startup migrations, set `RUN_DRIZZLE_MIGRATIONS=false` and/or `RUN_FS_MIGRATIONS=false`. You can run storage migration manually with `pnpm migrate-fs`.
:::

View file

@ -0,0 +1,163 @@
---
title: Vercel Deployment
---
This guide covers deploying OpenReader to Vercel with external Postgres and S3-compatible object storage.
## What works on Vercel
- Documents (PDF/EPUB/TXT/MD) work with `POSTGRES_URL` + external S3 storage.
- Audiobook routes work on Node.js serverless functions using `ffmpeg-static`.
- Heavy compute features (Whisper alignment + PDF layout parsing) run through an external compute worker service.
- For worker setup details and worker-specific env vars, see [Compute Worker (NATS JetStream)](./compute-worker).
:::warning DOCX Conversion Limitation
`docx` conversion requires `soffice` (LibreOffice), which is not available in a standard Vercel runtime.
:::
## 1. Environment Variables
Recommended production setup (auth enabled, admin panel enabled):
```bash
# Infrastructure
POSTGRES_URL=postgres://...
USE_EMBEDDED_WEED_MINI=false
S3_ACCESS_KEY_ID=...
S3_SECRET_ACCESS_KEY=...
S3_BUCKET=...
S3_REGION=us-east-1
S3_PREFIX=openreader
# Optional (non-AWS S3-compatible providers):
# S3_ENDPOINT=https://...
# S3_FORCE_PATH_STYLE=true
# Auth (required for the admin panel)
BASE_URL=https://your-app.vercel.app
AUTH_SECRET=...
ADMIN_EMAILS=you@example.com # comma-separated; admins manage TTS + features in-app
CRON_SECRET=... # generate with: openssl rand -base64 32
# Heavy compute (required on Vercel in current releases)
COMPUTE_WORKER_URL=https://<railway-worker-domain>
COMPUTE_WORKER_TOKEN=...
# Logging (recommended for Vercel log ingestion)
LOG_FORMAT=json
LOG_LEVEL=info
# First-boot seed for the TTS shared provider (optional; manage in-app afterwards)
# API_KEY=your_replicate_key
# API_BASE only needed for OpenAI-compatible self-hosted providers
```
If you also run an external worker service (for example Railway), set these there too:
- `LOG_FORMAT=json`
- `COMPUTE_LOG_LEVEL=info`
:::note Env vars vs. admin panel (important for Vercel)
`API_KEY` / `API_BASE` are one-shot bootstrap seeds on first deploy. After boot, manage providers and site features in **Settings → Admin**. Changes there apply on refresh without a redeploy. See [Admin Panel](../configure/admin-panel).
:::
## 1a. Railway + Synadia quick start (worker mode)
If your Vercel app uses an external compute worker on Railway with Synadia Cloud (NGS):
1. Deploy a Railway service from:
- `ghcr.io/richardr1126/openreader-compute-worker:refactor-ppdoclayoutv3-onnx-layout-parsing`
2. Enable public networking on that Railway service and set:
- `COMPUTE_WORKER_URL=https://<railway-worker-domain>` (in Vercel)
3. Use the same `COMPUTE_WORKER_TOKEN` value in both Vercel and Railway worker env vars.
For complete Railway worker env vars (`NATS_*`, `S3_*`, health checks, and Synadia `.creds` guidance), see [Compute Worker (NATS JetStream)](./compute-worker).
## 2. First-run admin configuration (recommended)
After the first successful deploy and admin login, open **Settings → Admin** and configure:
- **Shared providers**: create/edit your provider key(s) here (encrypted at rest).
- **Site features**:
- `enableDocxConversion=false` on Vercel (`soffice` unavailable).
- `enableTtsProvidersTab=false` if you want shared-provider-only UX.
- `enableUserSignups=true` unless you explicitly want an invite-only deployment.
- `restrictUserApiKeys=true` to block user BYOK through the hosted server.
- `defaultTtsProvider=replicate` (or your preferred shared slug).
- `showAllProviderModels=false` if you want users locked to each provider's default model.
- `enableAudiobookExport=true`.
## 3. Runtime JSON seed (optional)
If you must pre-seed site features/providers at deploy time, use `RUNTIME_SEED_JSON` or `RUNTIME_SEED_JSON_PATH` (versioned JSON seed document). Prefer the admin panel for ongoing management.
See [Environment Variables](../reference/environment-variables#runtime-json-seed-v4) for schema and examples.
:::warning Auth recommendation
Set both `BASE_URL` and `AUTH_SECRET` — they are required in v4+ and also required for the admin panel and for encrypting admin-stored TTS credentials.
:::
:::warning Rotating AUTH_SECRET invalidates admin-stored keys
Admin-managed TTS provider keys are encrypted with a key derived from `AUTH_SECRET`. If you rotate `AUTH_SECRET` after the first deploy, you must re-enter each admin shared provider's API key from the UI.
:::
:::tip
For all variables and defaults, see [Environment Variables](../reference/environment-variables).
:::
## 4. Database and data migrations
Vercel deployments do not run the `@openreader/bootstrap` process, so automatic startup migrations do not run there.
- Run `pnpm migrate` in a controlled environment to apply Drizzle schema migrations to your Postgres DB.
- Run `pnpm migrate-fs` only when migrating legacy local filesystem data (`docstore/documents_v1`, `docstore/audiobooks_v1`) into object storage + DB rows. Fresh Vercel deployments usually do not need this.
## 5. Scheduled maintenance tasks
The repository configures `/api/admin/tasks/tick` as a Vercel Cron route. Set `CRON_SECRET`; requests without the matching bearer token are rejected.
The checked-in Hobby-compatible schedule invokes the route once daily. The admin task panel therefore prevents selecting intervals shorter than one day on Vercel, even though self-hosted deployments can run tasks more frequently.
Each due task is claimed with a database-backed lease, due tasks start independently, and individual runs are aborted and marked failed after four minutes. Review failures and run tasks manually from **Settings → Admin → Scheduled tasks**.
## 6. FFmpeg packaging in Vercel functions
`ffmpeg-static` binaries must be included in function traces. This repo already does that in `next.config.ts` via `outputFileTracingIncludes` for:
- `/api/audiobook`
- `/api/audiobook/chapter`
- `/api/tts/segments/ensure`
:::info
`serverExternalPackages` should include `ffmpeg-static` so package paths resolve at runtime instead of being bundled into route output.
:::
If you change route paths or split handlers, update `outputFileTracingIncludes` accordingly.
## 7. Function memory sizing
FFmpeg workloads benefit from more memory/CPU. This repo includes:
```json
{
"$schema": "https://openapi.vercel.sh/vercel.json",
"functions": {
"app/api/audiobook/route.ts": { "memory": 3009 },
"app/api/tts/segments/ensure/route.ts": { "memory": 3009 }
}
}
```
Adjust memory per route if your files are larger or your plan differs.
## 8. Runtime expectations and caveats
- Audiobook APIs require S3 configuration; otherwise they return `503`.
- For production Vercel deploys, use `POSTGRES_URL` instead of SQLite.
## 9. Smoke test after deploy
1. Upload and read a PDF/EPUB document.
2. Confirm sync/blob fetch works across refreshes/devices.
3. Generate at least one audiobook chapter and play/download it.
4. Verify worker-backed word highlighting and PDF parsing.
5. Open **Settings → Admin → Scheduled tasks**, run one task manually, and confirm the next daily cron invocation succeeds.

View file

@ -0,0 +1,165 @@
---
title: Docker Quick Start
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
## Prerequisites
- A recent Docker version installed
- A TTS API server that OpenReader can reach:
- [Kokoro-FastAPI](./configure/tts-provider-guides/kokoro-fastapi)
- [KittenTTS-FastAPI](./configure/tts-provider-guides/kitten-tts-fastapi)
- [Orpheus-FastAPI](./configure/tts-provider-guides/orpheus-fastapi)
- [Replicate](./configure/tts-provider-guides/replicate)
- [DeepInfra](./configure/tts-provider-guides/deepinfra)
- [OpenAI](./configure/tts-provider-guides/openai)
- [Other OpenAI-compatible providers](./configure/tts-provider-guides/other)
:::warning SeaweedFS Compatibility Note (April 16, 2026)
OpenReader currently pins embedded SeaweedFS to `4.18` in CI and Docker builds.
`4.19` introduced intermittent `InternalError` responses on S3 `PutObject` in our upload flow.
:::
## Published images
- App server: `ghcr.io/richardr1126/openreader:latest`
- Compute worker (Optional): `ghcr.io/richardr1126/openreader-compute-worker:latest`
- Legacy app alias: `ghcr.io/richardr1126/openreader-webui:latest`
## 1. Start the Docker container
<Tabs groupId="docker-start-mode">
<TabItem value="localhost" label="Localhost" default>
Persistent storage, embedded SeaweedFS `weed mini`, required auth, optional library mount:
```bash
docker run --name openreader \
--restart unless-stopped \
-p 3003:3003 \
-p 8333:8333 \
-v openreader_docstore:/app/docstore \
-e API_BASE=http://host.docker.internal:8880/v1 \
-e BASE_URL=http://localhost:3003 \
-e AUTH_SECRET=$(openssl rand -base64 32) \
-e ADMIN_EMAILS=you@example.com \
ghcr.io/richardr1126/openreader:latest
```
What this command enables:
- `-p 3003:3003`: exposes the OpenReader web app/API.
- `-p 8333:8333`: exposes embedded SeaweedFS S3 endpoint for direct browser presigned upload/download.
- `-v openreader_docstore:/app/docstore`: persists SQLite metadata, SeaweedFS blob data, and migration/runtime state.
- `-e API_BASE=...` / optional `-e API_KEY=...`: **first-boot seed only.** On the first container start, these are auto-migrated into a `default-openai` admin shared provider stored in the DB (key encrypted at rest when provided). After that, the running app no longer reads them — manage the provider from **Settings → Admin → Shared providers**. See [Admin Panel](./configure/admin-panel).
- `-e BASE_URL=...` and `-e AUTH_SECRET=...`: required for v4+ auth/session startup.
- `-e ADMIN_EMAILS=...`: (optional, requires auth) comma-separated emails auto-promoted to admin. Admins see the **Admin** tab in Settings.
</TabItem>
<TabItem value="local-network" label="LAN Host">
Use this when the app should be reachable from other devices on your LAN:
```bash
docker run --name openreader \
--restart unless-stopped \
-p 3003:3003 \
-p 8333:8333 \
-v openreader_docstore:/app/docstore \
-e API_BASE=http://host.docker.internal:8880/v1 \
-e BASE_URL=http://<YOUR_LAN_IP>:3003 \
-e AUTH_SECRET=$(openssl rand -base64 32) \
-e AUTH_TRUSTED_ORIGINS=http://localhost:3003,http://127.0.0.1:3003 \
-e USE_ANONYMOUS_AUTH_SESSIONS=true \
-e ADMIN_EMAILS=you@example.com \
ghcr.io/richardr1126/openreader:latest
```
Replace `YOUR_LAN_IP` with the Docker host IP address on your local network to allow access from other devices.
What this command enables:
- LAN access from phones/tablets/other computers via `http://<YOUR_LAN_IP>:3003`.
- `BASE_URL` points auth/session cookies and callbacks at your LAN URL.
- `AUTH_TRUSTED_ORIGINS` allows localhost loopback origins in addition to your primary LAN origin.
- `USE_ANONYMOUS_AUTH_SESSIONS=true` allows guest sessions while auth is enabled.
- `API_BASE` seeds the default TTS endpoint into the admin-managed `default-openai` shared provider on first boot. Edit it from **Settings → Admin → Shared providers** after that.
- `API_KEY` optionally seeds the default provider's key (encrypted at rest). Omit it for an upstream that does not require authentication.
- `ADMIN_EMAILS=...` (optional) auto-promotes the listed email(s) to admin so they can manage shared providers and site feature flags from the UI.
- `openreader_docstore` volume keeps data persistent across restarts.
</TabItem>
<TabItem value="minimal" label="Minimal">
Auth required, embedded storage ephemeral, no library import:
```bash
docker run --name openreader \
--restart unless-stopped \
-p 3003:3003 \
-p 8333:8333 \
-e BASE_URL=http://localhost:3003 \
-e AUTH_SECRET=$(openssl rand -base64 32) \
ghcr.io/richardr1126/openreader:latest
```
What this command enables:
- Fast startup with only the required auth env vars.
- No persistent volume (`/app/docstore` stays container-local), so data is ephemeral unless you add a mount.
- The app still requires `BASE_URL` + `AUTH_SECRET` in v4+, so include them even in minimal mode.
- No TTS provider preset by default. Configure `API_BASE` and, when required, `API_KEY` on first boot if you want a seeded shared provider, or run auth+admin mode and manage providers from the admin panel.
</TabItem>
</Tabs>
:::tip Quick Tips
- Set `API_BASE` on first boot to a TTS endpoint the container can reach (`host.docker.internal` works for host-local services). After first boot, manage providers in **Settings → Admin → Shared providers**.
- `BASE_URL` and `AUTH_SECRET` are required in v4+. The admin panel requires auth.
- Set `ADMIN_EMAILS` to your email if you want the **Admin** tab in Settings.
- `restrictUserApiKeys` controls shared-provider-only mode. For per-user BYOK, toggle it off in **Settings → Admin → Site features** or seed `runtimeConfig.restrictUserApiKeys=false` via runtime seed JSON.
- Use a `/app/docstore` mount if you want data to survive container/image replacement.
- Startup automatically runs DB/storage migrations via the shared entrypoint.
- Scheduled maintenance tasks run in-process and can be managed from **Settings → Admin → Scheduled tasks**; Docker/self-hosted deployments do not need `CRON_SECRET`.
:::
:::warning Port `8333` Exposure
Expose `8333` for direct browser presigned upload/download with embedded SeaweedFS.
If `8333` is not reachable from the browser, direct presigned access is unavailable. Uploads can still fall back to `/api/documents/blob/upload/fallback`, and document reads/downloads continue through `/api/documents/blob`.
:::
## 2. Configure settings in the app UI
Visit [http://localhost:3003](http://localhost:3003) after startup.
- If you set `ADMIN_EMAILS`, sign in with that email and open **Settings → Admin** to manage shared TTS providers and site feature flags for all users.
- Per-user: set TTS provider/model in **Settings → TTS Provider**. API key/base URL inputs are shown only when `restrictUserApiKeys=false`.
- Select the model voice from the voice dropdown.
## 3. Update Docker image
Legacy image compatibility: `ghcr.io/richardr1126/openreader-webui:latest` remains available as an alias.
For external compute mode image details, see [Compute Worker (NATS JetStream)](./deploy/compute-worker).
```bash
docker stop openreader || true && \
docker rm openreader || true && \
docker image rm ghcr.io/richardr1126/openreader:latest || true && \
docker pull ghcr.io/richardr1126/openreader:latest
```
:::tip
If you use a mounted volume for `/app/docstore`, your persisted data remains after image updates.
:::
:::info Related Docs
- [Environment Variables](./reference/environment-variables)
- [Auth](./configure/auth)
- [Admin Panel](./configure/admin-panel)
- [Database](./configure/database)
- [Object / Blob Storage](./configure/object-blob-storage)
- [Migrations](./configure/migrations)
:::

View file

@ -0,0 +1,47 @@
---
id: intro
title: Introduction
slug: /
---
OpenReader is an open-source text-to-speech document reader built with Next.js. It provides a multilingual read-along experience with narration for **EPUB, PDF, TXT, MD, and DOCX documents**.
> Previously named **OpenReader-WebUI**.
It supports multiple TTS providers including OpenAI, Replicate, DeepInfra, and custom OpenAI-compatible endpoints such as [Kokoro-FastAPI](https://github.com/remsky/Kokoro-FastAPI), [KittenTTS-FastAPI](https://github.com/richardr1126/KittenTTS-FastAPI), and [Orpheus-FastAPI](https://github.com/Lex-au/Orpheus-FastAPI).
## ✨ Highlights
- 🧱 **Layout-aware PDF Parsing**
- PP-DocLayoutV3 (ONNX) detects structured blocks with cross-page stitching and geometry-based highlighting for precise read-along sync and clean TTS segmentation
- ⏱️ **Word-by-word Highlighting** via ONNX Whisper alignment
- Powered by the external compute worker control plane (NATS JetStream-backed)
- ⚡ **Segment-based TTS Playback**
- Sentence-aware generation with cached audio segments, background preloading, and resumable playback across EPUB, PDF, TXT, MD, and DOCX
- 🎯 **Multi-Provider TTS Support**
- Self-hosted: [**Kokoro-FastAPI**](https://github.com/remsky/Kokoro-FastAPI) (multi-voice combinations), [**KittenTTS-FastAPI**](https://github.com/richardr1126/KittenTTS-FastAPI), [**Orpheus-FastAPI**](https://github.com/Lex-au/Orpheus-FastAPI), or any custom OpenAI-compatible endpoint
- Cloud: [**OpenAI**](https://platform.openai.com/docs/pricing#transcription-and-speech) (`tts-1`, `tts-1-hd`, `gpt-4o-mini-tts`), [**Replicate**](https://replicate.com/explore) (built-in catalog + any model ID), [**DeepInfra**](https://deepinfra.com/models/text-to-speech) (Kokoro-82M and others)
- 🌐 **Multilingual Support**
- Choose a document language for language-aware narration and highlighting
- Available languages depend on the configured provider, model, and voice
- 🎧 **Audiobook Export** in `m4b`/`mp3` with resumable chapter generation
- 🗂️ **Flexible Backend** — embedded SeaweedFS or S3-compatible storage, SQLite or Postgres, server library import, and device sync
- 🔐 **Auth and User Isolation** — auth is required in v4+, with optional anonymous auth sessions for guest flows
- 🎨 **Customizable** — 13 built-in themes (light and dark palettes), per-user TTS settings, and document handling controls
## 🧭 Key Docs
- [Docker Quick Start](./docker-quick-start)
- [Local Development](./deploy/local-development)
- [Vercel Deployment](./deploy/vercel-deployment)
- [Environment Variables](./reference/environment-variables)
- [Auth](./configure/auth)
- [Database](./configure/database)
- [Object / Blob Storage](./configure/object-blob-storage)
- [Migrations](./configure/migrations)
- [Server Library Import](./configure/server-library-import)
- [TTS Providers](./configure/tts-providers)
## Source Repository
- GitHub: [richardr1126/openreader](https://github.com/richardr1126/openreader)

View file

@ -0,0 +1,448 @@
---
title: Environment Variables
toc_max_heading_level: 3
---
This page is the source-of-truth reference for OpenReader environment variables.
:::note Recommended configuration path
Use **Settings → Admin** as the primary source of truth for shared providers and runtime site features.
`API_BASE` / `API_KEY` are optional one-time provider bootstrap seeds.
Runtime site features are seeded with `RUNTIME_SEED_JSON` / `RUNTIME_SEED_JSON_PATH`.
:::
## Quick Reference Table
| Variable | Area | Default | When to set |
| --- | --- | --- | --- |
| `LOG_FORMAT` | Runtime logging | `pretty` | Set `json` for structured logs |
| `LOG_LEVEL` | Runtime logging | `info` | Set app server log level |
| `API_BASE` | TTS provider bootstrap seed | unset | Optional first-boot base URL for `default-openai` |
| `API_KEY` | TTS provider bootstrap seed | unset | Optional first-boot API key for `default-openai` |
| `BASE_URL` | Auth | unset | Required at startup |
| `AUTH_SECRET` | Auth | unset | Required at startup |
| `AUTH_TRUSTED_ORIGINS` | Auth | empty | Add extra allowed origins |
| `USE_ANONYMOUS_AUTH_SESSIONS` | Auth | `false` | Set `true` to allow anonymous auth sessions |
| `GITHUB_CLIENT_ID` | Auth/OAuth | unset | Set with `GITHUB_CLIENT_SECRET` to enable GitHub sign-in |
| `GITHUB_CLIENT_SECRET` | Auth/OAuth | unset | Set with `GITHUB_CLIENT_ID` to enable GitHub sign-in |
| `ADMIN_EMAILS` | Admin | empty | Comma-separated emails auto-promoted to admin |
| `CRON_SECRET` | Scheduled tasks | unset | Required for Vercel cron invocations |
| `POSTGRES_URL` | Database | unset (SQLite mode) | Set to switch metadata/auth DB to Postgres |
| `USE_EMBEDDED_WEED_MINI` | Storage | `true` when unset | Set `false` to use external S3-compatible storage only |
| `WEED_MINI_DIR` | Storage | `docstore/seaweedfs` | Override embedded SeaweedFS data directory |
| `WEED_MINI_WAIT_SEC` | Storage | `20` | Tune SeaweedFS startup wait timeout |
| `S3_ACCESS_KEY_ID` | Storage | auto-generated in embedded mode | Set explicitly for stable/external credentials |
| `S3_SECRET_ACCESS_KEY` | Storage | auto-generated in embedded mode | Set explicitly for stable/external credentials |
| `S3_BUCKET` | Storage | `openreader-documents` in embedded mode | Required for external S3-compatible storage |
| `S3_REGION` | Storage | `us-east-1` in embedded mode | Required for external S3-compatible storage |
| `S3_ENDPOINT` | Storage | derived in embedded mode | Set for S3-compatible providers (MinIO/SeaweedFS/R2/etc.) |
| `S3_FORCE_PATH_STYLE` | Storage | `true` in embedded mode | Set per provider requirement |
| `S3_PREFIX` | Storage | `openreader` | Customize object key prefix |
| `IMPORT_LIBRARY_DIR` | Library import | `docstore/library` fallback | Set a single server library root |
| `IMPORT_LIBRARY_DIRS` | Library import | unset | Set multiple roots (comma/colon/semicolon separated) |
| `EMBEDDED_COMPUTE_WORKER_PORT` | Compute | `8081` | Override embedded worker bind port |
| `EMBEDDED_NATS_PORT` | Compute | `4222` | Override embedded NATS client port |
| `EMBEDDED_NATS_MONITOR_PORT` | Compute | `8222` | Override embedded NATS monitor port |
| `EMBEDDED_NATS_STORE_DIR` | Compute | `docstore/nats/jetstream` | Override embedded JetStream storage directory |
| `NATS_URL` | Compute | `nats://127.0.0.1:4222` in embedded startup | Override embedded startup or set standalone worker URL |
| `COMPUTE_LOG_LEVEL` | Compute | `info` | Compute worker log level |
| `COMPUTE_JOB_CONCURRENCY` | Compute | `1` | Shared compute concurrency cap |
| `COMPUTE_WHISPER_TIMEOUT_MS` | Compute | `30000` | Whisper alignment timeout budget |
| `COMPUTE_PDF_TIMEOUT_MS` | Compute | `300000` | PDF parse timeout budget |
| `COMPUTE_PDF_JOB_ATTEMPTS` | Compute | `1` | Max JetStream deliveries for PDF layout jobs |
| `COMPUTE_OP_STALE_MS` | Compute | `max(30m, 4x max compute timeout)` | Shared stale window for compute op replacement |
| `WHISPER_MODEL_BASE_URL` | Compute model source | onnx-community default | Override Whisper ONNX model base URL |
| `PDF_LAYOUT_MODEL_BASE_URL` | Compute model source | PP-DocLayoutV3 default | Override PDF layout ONNX model base URL |
| `COMPUTE_WORKER_URL` | External compute mode | unset | Set only for standalone external worker mode |
| `COMPUTE_WORKER_TOKEN` | External compute mode | unset | Required for standalone external worker auth |
| `FFMPEG_BIN` | Audio runtime | auto-detected (`ffmpeg-static`) | Override ffmpeg binary path |
| `DISABLE_AUTH_RATE_LIMIT` | Auth request throttling | `false` | Set `true` to disable Better Auth request rate limiting |
| `ENABLE_TEST_NAMESPACE` | Testing/CI | unset | Honor `x-openreader-test-namespace` header in production builds |
| `RUN_DRIZZLE_MIGRATIONS` | DB migrations | `true` | Set `false` to skip startup Drizzle migrations |
| `RUN_FS_MIGRATIONS` | Storage migrations | `true` | Set `false` to skip startup filesystem -> S3/DB migration pass |
| `RUNTIME_SEED_JSON_PATH` | Runtime JSON seed | unset | Absolute path to first-boot JSON seed document |
| `RUNTIME_SEED_JSON` | Runtime JSON seed | unset | Inline first-boot JSON seed document |
## Runtime Logging
### LOG_FORMAT
Controls log output format for server-side Pino loggers.
- Default: `pretty`
- Allowed values: `pretty`, `json`
- Applies to app server and compute worker
### LOG_LEVEL
App server log level.
- Default: `info`
## TTS Provider and Request Behavior
### API_BASE
Optional first-boot bootstrap base URL for the auto-created `default-openai` shared provider.
- Example: `http://host.docker.internal:8880/v1`
- Read only for provider bootstrap when shared providers are empty. Setting `API_BASE` is sufficient; `API_KEY` may be blank.
- After bootstrap, provider configuration is DB-backed and managed in **Settings → Admin → Shared providers**.
### API_KEY
Optional first-boot bootstrap API key for the auto-created `default-openai` shared provider.
- Read only for provider bootstrap when shared providers are empty.
- Stored encrypted at rest after bootstrap.
- After bootstrap, provider configuration is DB-backed and managed in **Settings → Admin → Shared providers**.
## Auth and Identity
### BASE_URL
Required external base URL for this OpenReader instance.
- Required at startup
- Example: `http://localhost:3003` or `https://reader.example.com`
### AUTH_SECRET
Required secret key used by auth/session handling.
- Required at startup
- Generate with `openssl rand -base64 32`
### AUTH_TRUSTED_ORIGINS
Additional allowed origins for auth requests.
- Comma-separated list
- `BASE_URL` origin is trusted automatically
### USE_ANONYMOUS_AUTH_SESSIONS
Controls whether auth-enabled deployments can create/use anonymous sessions.
- Default: `false`
### GITHUB_CLIENT_ID
GitHub OAuth client ID.
- Set with `GITHUB_CLIENT_SECRET`
### GITHUB_CLIENT_SECRET
GitHub OAuth client secret.
- Set with `GITHUB_CLIENT_ID`
### ADMIN_EMAILS
Comma-separated list of email addresses auto-promoted to admin.
- Requires auth to be enabled
- Admins can manage shared providers and runtime site features in-app
### CRON_SECRET
Bearer-token secret for `GET /api/admin/tasks/tick`.
- Required on Vercel so scheduled maintenance tasks can run from the configured Vercel Cron.
- Vercel automatically sends `Authorization: Bearer <CRON_SECRET>` on cron invocations.
- Generate a strong random value, for example with `openssl rand -base64 32`.
- Self-hosted Node.js deployments run the scheduler in-process and do not require this variable.
## Database and Object Blob Storage
### POSTGRES_URL
Switches metadata/auth storage from SQLite to Postgres.
- Unset: SQLite at `docstore/sqlite3.db`
- Set: Postgres mode
### USE_EMBEDDED_WEED_MINI
Controls embedded SeaweedFS startup.
- Default behavior: treated as enabled when unset
- Set `false` to rely on external S3-compatible storage
### WEED_MINI_DIR
Data directory for embedded SeaweedFS (`weed mini`).
- Default: `docstore/seaweedfs`
### WEED_MINI_WAIT_SEC
Max wait time for embedded SeaweedFS startup.
- Default: `20`
### S3_ACCESS_KEY_ID
S3 access key.
- Optional in embedded mode (auto-generated when unset)
- Required for external S3 mode
### S3_SECRET_ACCESS_KEY
S3 secret key.
- Optional in embedded mode (auto-generated when unset)
- Required for external S3 mode
### S3_BUCKET
S3 bucket name.
- Embedded default: `openreader-documents`
- Required for external S3 mode
### S3_REGION
S3 region.
- Embedded default: `us-east-1`
- Required for external S3 mode
### S3_ENDPOINT
Custom endpoint for S3-compatible providers.
- Optional for AWS
- Typical for MinIO/SeaweedFS/R2
### S3_FORCE_PATH_STYLE
Force path-style S3 URLs.
- Embedded default: `true`
### S3_PREFIX
Object key prefix.
- Default: `openreader`
## Library Import
### IMPORT_LIBRARY_DIR
Single library source directory.
### IMPORT_LIBRARY_DIRS
Multiple library roots.
- Supports comma, colon, or semicolon-separated values
## Compute Worker and Model Configuration
### EMBEDDED_COMPUTE_WORKER_PORT
Embedded compute worker port.
- Default: `8081`
### EMBEDDED_NATS_PORT
Embedded NATS client port.
- Default: `4222`
### EMBEDDED_NATS_MONITOR_PORT
Embedded NATS monitor port.
- Default: `8222`
### EMBEDDED_NATS_STORE_DIR
Embedded NATS JetStream data directory.
- Default: `docstore/nats/jetstream`
### NATS_URL
NATS URL used by compute services.
- Embedded startup default: `nats://127.0.0.1:4222`
### COMPUTE_LOG_LEVEL
Compute worker log level.
- Default: `info`
### COMPUTE_JOB_CONCURRENCY
Max concurrent compute jobs per worker.
- Default: `1`
### COMPUTE_WHISPER_TIMEOUT_MS
Whisper alignment timeout budget.
- Default: `30000`
### COMPUTE_PDF_TIMEOUT_MS
PDF parse timeout budget.
- Default: `300000`
### COMPUTE_PDF_JOB_ATTEMPTS
Max JetStream deliveries for PDF layout jobs.
- Default: `1`
- In embedded worker mode, set this in the root `.env`
### COMPUTE_OP_STALE_MS
Stale operation window before worker/app cleanup logic can replace an op.
- Default: `max(30m, 4x max compute timeout)`
### WHISPER_MODEL_BASE_URL
Base URL for Whisper ONNX model downloads.
### PDF_LAYOUT_MODEL_BASE_URL
Base URL for PDF layout model downloads.
### COMPUTE_WORKER_URL
External compute worker URL.
- Leave unset for embedded worker mode
### COMPUTE_WORKER_TOKEN
Shared token for app-to-external-worker requests.
## Audio Runtime
### FFMPEG_BIN
Override ffmpeg binary path used for audio processing.
- Used by audiobook processing routes and compute worker Whisper audio decode.
## Testing and CI
### DISABLE_AUTH_RATE_LIMIT
Disables Better Auth request rate limiting.
- Default: `false`
### ENABLE_TEST_NAMESPACE
Enables the `x-openreader-test-namespace` header path in production builds.
## Migration Controls
### RUN_DRIZZLE_MIGRATIONS
Controls startup Drizzle schema migrations.
- Default: `true`
- Set `false` to skip startup migration run
### RUN_FS_MIGRATIONS
Controls startup filesystem-to-S3/DB migration pass.
- Default: `true`
- Set `false` to skip startup storage migration run
## Runtime JSON Seed (v4)
### RUNTIME_SEED_JSON_PATH
Path-based first-boot seed document.
- If both `RUNTIME_SEED_JSON_PATH` and `RUNTIME_SEED_JSON` are set, path wins.
- Value must point to a JSON file readable by the app process.
### RUNTIME_SEED_JSON
Inline first-boot seed document.
- Used only when `RUNTIME_SEED_JSON_PATH` is unset.
- Must be a JSON object with `version: 1`.
Supported top-level keys:
- `version` (required, must be `1`)
- `runtimeConfig` (optional object, strict-validated against runtime schema)
- `providers` (optional array of shared provider seed entries)
Example:
```json
{
"version": 1,
"runtimeConfig": {
"enableUserSignups": true,
"restrictUserApiKeys": true,
"defaultTtsProvider": "custom-openai",
"enableTtsProvidersTab": true,
"enableAudiobookExport": true,
"enableDocxConversion": true,
"showAllProviderModels": true,
"disableTtsRateLimit": true,
"ttsDailyLimitAnonymous": 50000,
"ttsDailyLimitAuthenticated": 500000,
"ttsIpDailyLimitAnonymous": 100000,
"ttsIpDailyLimitAuthenticated": 1000000,
"ttsCacheMaxSizeBytes": 268435456,
"ttsCacheTtlMs": 1800000,
"ttsUpstreamMaxRetries": 2,
"ttsUpstreamTimeoutMs": 285000,
"disableComputeRateLimit": true,
"computeParseBurstMax": 8,
"computeParseBurstWindowSec": 60,
"computeParseSustainedMax": 24,
"computeParseSustainedWindowSec": 600,
"maxUploadMb": 200,
"changelogFeedUrl": "https://docs.openreader.richardr.dev/changelog/manifest.json"
},
"providers": [
{
"slug": "default-openai",
"displayName": "Default (seeded)",
"providerType": "custom-openai",
"baseUrl": "http://localhost:8880/v1",
"defaultModel": "kokoro",
"enabled": true
}
]
}
```
Provider fallback behavior:
- If the JSON seed includes `providers` (including an empty array), `API_BASE` / `API_KEY` fallback is skipped.
- If the JSON seed does not include a `providers` key, the legacy `API_BASE` / `API_KEY` bootstrap fallback can still create `default-openai` when provider rows are empty. `API_BASE` alone is sufficient for an upstream that does not require authentication.
Precedence summary:
- Runtime reads: admin DB runtime rows override built-in defaults.
- Seed input (`RUNTIME_SEED_JSON*`) only populates missing runtime rows on first boot; it does not overwrite existing/admin-edited rows.
- Provider bootstrap order: JSON `providers` section > `API_BASE`/`API_KEY` fallback > no provider bootstrap.
## Related
- [Admin Panel](../configure/admin-panel)
- [TTS Providers](../configure/tts-providers)
- [Local Development](../deploy/local-development)
- [Vercel Deployment](../deploy/vercel-deployment)

View file

@ -0,0 +1,69 @@
---
title: Stack
---
## Framework
- [Next.js](https://nextjs.org/) 15 (App Router, Turbopack in dev)
- [React](https://react.dev/) 19
- [TypeScript](https://www.typescriptlang.org/)
- [pnpm](https://pnpm.io/) workspaces monorepo
## Containerization and runtime
- [Docker](https://www.docker.com/) (linux/amd64 and linux/arm64)
- Shared entrypoint that runs DB migrations by default and can bootstrap embedded SeaweedFS before app startup
## Next.js client
- UI: [Tailwind CSS](https://tailwindcss.com), [Headless UI](https://headlessui.com), [@tailwindcss/typography](https://tailwindcss.com/docs/typography-plugin)
- Interactions: `react-dnd`, `react-dropzone`
- Server state: [TanStack Query](https://tanstack.com/query) (React Query v5)
- Authentication: [Better Auth](https://www.better-auth.com/) client SDK
- Local storage/cache: [Dexie.js](https://dexie.org/) (IndexedDB)
- Audio playback: [Howler.js](https://howlerjs.com/)
- Notifications: `react-hot-toast`
- Document rendering:
- PDF: [react-pdf](https://github.com/wojtekmaj/react-pdf), [pdf.js](https://mozilla.github.io/pdf.js/)
- EPUB: [react-reader](https://github.com/gerhardsletten/react-reader), [epubjs](https://github.com/futurepress/epub.js/)
- Markdown/Text: [react-markdown](https://github.com/remarkjs/react-markdown), [remark-gfm](https://github.com/remarkjs/remark-gfm)
- Text preprocessing/matching: [compromise](https://github.com/spencermountain/compromise), [cmpstr](https://github.com/remsky/cmpstr)
- Analytics: [Vercel Analytics](https://vercel.com/analytics)
## Next.js server
- APIs: Route Handlers for sync, blob/content access, migrations, audiobook export, TTS/Whisper proxying
- State sync: request-based today (not realtime push updates)
- Authentication: [Better Auth](https://www.better-auth.com/) server handlers/adapters with anonymous session support
- Metadata DB: [Drizzle ORM](https://orm.drizzle.team/) with SQLite (`better-sqlite3`) by default and optional Postgres (`pg`)
- App tables are manually maintained in Drizzle schema files
- Auth tables are auto-generated by the [Better Auth](https://www.better-auth.com/) CLI and migrated alongside app tables via Drizzle
- Blob storage: embedded [SeaweedFS](https://github.com/seaweedfs/seaweedfs) (`weed mini`) by default, or external S3-compatible storage via AWS SDK v3
- TTS providers: OpenAI-compatible API (`openai` SDK), [Replicate](https://replicate.com/) (`replicate` client), DeepInfra, and custom OpenAI-compatible endpoints — credentials are encrypted at rest
- Audio pipeline: [ffmpeg](https://ffmpeg.org/) (`ffmpeg-static`) for audiobook assembly, `archiver` for export packaging
- Utilities: `lru-cache` for in-process caching, `fast-xml-parser` for EPUB/XML parsing, `uuid` for identifier generation, `zod` for schema validation
## External compute worker (optional)
Standalone worker package:
- **`@openreader/compute-worker`** — standalone Node.js compute service containing its private inference and queue runtime
- ONNX runtime: `onnxruntime-node` with `@huggingface/tokenizers`
- Whisper alignment: `onnx-community/whisper-base_timestamped` (q4) for word-level timestamps
- PDF layout: `Bei0001/PP-DocLayoutV3-ONNX` for document block detection and layout parsing
- PDF rendering: `pdfjs-dist`, `@napi-rs/canvas` for server-side page rasterization
- Utilities: `jszip`, `ffmpeg-static`
- HTTP server: [Fastify](https://fastify.dev/) v5 with a versioned OpenAPI contract
- Job queue + state: [NATS](https://nats.io/) JetStream WorkQueue pull consumers + NATS KV (`jobs.whisper`, `jobs.layout`)
- Storage: AWS SDK v3 S3 client for reading/writing blobs
- Logging: [Pino](https://getpino.io/)
- Validation: [Zod](https://zod.dev/)
- The Next.js app communicates with the worker only through the versioned HTTP API generated from OpenAPI.
## Tooling and testing
- ESLint
- TypeScript
- [Playwright](https://playwright.dev/) end-to-end tests
- Drizzle migration/generation scripts
- [Docusaurus](https://docusaurus.io/) documentation site (`docs-site/`)

View file

@ -0,0 +1,123 @@
import { themes as prismThemes } from 'prism-react-renderer';
import type { Config } from '@docusaurus/types';
import type * as Preset from '@docusaurus/preset-classic';
const config: Config = {
title: 'OpenReader Docs',
tagline: 'Docs for OpenReader',
favicon: 'favicon.ico',
future: {
v4: true,
},
url: 'https://docs.openreader.richardr.dev',
baseUrl: '/',
organizationName: 'richardr1126',
projectName: 'OpenReader',
onBrokenLinks: 'throw',
markdown: {
hooks: {
onBrokenMarkdownLinks: 'throw',
},
},
i18n: {
defaultLocale: 'en',
locales: ['en'],
},
presets: [
[
'classic',
{
docs: {
routeBasePath: '/',
sidebarPath: './sidebars.ts',
editUrl: 'https://github.com/richardr1126/openreader/tree/main/docs-site/',
// lastVersion: 'current',
// versions: {
// current: {
// label: 'Current',
// },
// },
},
blog: false,
theme: {
customCss: './custom.css',
},
} satisfies Preset.Options,
],
],
plugins: [
[
'@easyops-cn/docusaurus-search-local',
{
indexDocs: true,
indexBlog: false,
docsRouteBasePath: '/',
language: ['en'],
hashed: true,
explicitSearchResultPath: true,
highlightSearchTermsOnTargetPage: true,
},
],
],
themeConfig: {
colorMode: {
defaultMode: 'light',
disableSwitch: false,
respectPrefersColorScheme: true,
},
navbar: {
title: 'OpenReader',
items: [
{
type: 'docSidebar',
sidebarId: 'tutorialSidebar',
position: 'left',
label: 'Docs',
},
{
type: 'docsVersionDropdown',
position: 'left',
},
{
href: 'https://github.com/richardr1126/openreader',
label: 'GitHub',
position: 'right',
},
],
},
footer: {
links: [
{
title: 'Community',
items: [
{ label: 'Support', to: '/about/support-and-contributing' },
{ label: 'GitHub Discussions', href: 'https://github.com/richardr1126/openreader/discussions' },
{ label: 'Issues', href: 'https://github.com/richardr1126/openreader/issues' },
],
},
{
title: 'Project',
items: [
{ label: 'GitHub', href: 'https://github.com/richardr1126/openreader' },
{ label: 'Releases', href: 'https://github.com/richardr1126/openreader/releases' },
],
},
],
copyright: `Copyright © ${new Date().getFullYear()} OpenReader contributors.`,
},
prism: {
theme: prismThemes.github,
darkTheme: prismThemes.dracula,
},
} satisfies Preset.ThemeConfig,
};
export default config;

33
docs-site/package.json Normal file
View file

@ -0,0 +1,33 @@
{
"name": "openreader-docs",
"version": "0.0.0",
"private": true,
"scripts": {
"start": "docusaurus start --host 0.0.0.0 --port 3004",
"build": "docusaurus build",
"serve": "docusaurus serve",
"clear": "docusaurus clear",
"docusaurus": "docusaurus",
"swizzle": "docusaurus swizzle",
"deploy": "docusaurus deploy"
},
"dependencies": {
"@mdx-js/react": "^3.1.1",
"@docusaurus/core": "^3.9.2",
"@docusaurus/preset-classic": "^3.9.2",
"@easyops-cn/docusaurus-search-local": "^0.54.1",
"clsx": "^2.1.1",
"prism-react-renderer": "^2.4.1",
"react": "^18.3.1",
"react-dom": "^18.3.1"
},
"devDependencies": {
"@docusaurus/module-type-aliases": "^3.9.2",
"@docusaurus/tsconfig": "^3.9.2",
"@docusaurus/types": "^3.9.2",
"typescript": "^5.9.3"
},
"engines": {
"node": ">=20"
}
}

12630
docs-site/pnpm-lock.yaml Normal file

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,6 @@
packages:
- .
allowBuilds:
core-js: true
core-js-pure: true

81
docs-site/sidebars.ts Normal file
View file

@ -0,0 +1,81 @@
import type { SidebarsConfig } from '@docusaurus/plugin-content-docs';
const sidebars: SidebarsConfig = {
tutorialSidebar: [
'intro',
{
type: 'doc',
id: 'docker-quick-start',
label: '🐳 Docker Quick Start',
},
{
type: 'category',
label: '⚙️ Configure',
items: [
{
type: 'category',
label: '🔊 TTS Providers',
link: {
type: 'doc',
id: 'configure/tts-providers',
},
items: [
'configure/tts-provider-guides/kokoro-fastapi',
'configure/tts-provider-guides/kitten-tts-fastapi',
'configure/tts-provider-guides/orpheus-fastapi',
'configure/tts-provider-guides/supertonic',
'configure/tts-provider-guides/replicate',
'configure/tts-provider-guides/deepinfra',
'configure/tts-provider-guides/openai',
'configure/tts-provider-guides/speech-sdk',
'configure/tts-provider-guides/other',
],
},
{
type: 'doc',
id: 'configure/auth',
label: '🔐 Auth',
},
{
type: 'doc',
id: 'configure/admin-panel',
label: '🛡️ Admin Panel',
},
{
type: 'doc',
id: 'configure/server-library-import',
label: '📥 Server Library Import',
},
'configure/tts-rate-limiting',
'configure/database',
'configure/object-blob-storage',
'configure/migrations',
],
},
{
type: 'category',
label: '🚀 Deploy',
items: [
'deploy/local-development',
'deploy/docker-compose',
'deploy/compute-worker',
'deploy/vercel-deployment',
],
},
{
type: 'category',
label: 'Reference',
items: [
'reference/environment-variables',
'reference/stack',
],
},
{
type: 'category',
label: 'About',
items: ['about/support-and-contributing', 'about/acknowledgements', 'about/license'],
},
],
};
export default sidebars;

1
docs-site/static/CNAME Normal file
View file

@ -0,0 +1 @@
docs.openreader.richardr.dev

Binary file not shown.

After

Width:  |  Height:  |  Size: 15 KiB

View file

@ -0,0 +1,2 @@
User-agent: *
Allow: /

6
docs-site/tsconfig.json Normal file
View file

@ -0,0 +1,6 @@
{
"extends": "@docusaurus/tsconfig",
"compilerOptions": {
"baseUrl": "."
}
}

View file

@ -0,0 +1,16 @@
---
title: Acknowledgements
---
This project is built with support from the following open-source projects and tools:
- [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M)
- [Kokoro-FastAPI](https://github.com/remsky/Kokoro-FastAPI)
- [Better Auth](https://www.better-auth.com/)
- [SQLite](https://www.sqlite.org/)
- [PostgreSQL](https://www.postgresql.org/)
- [SeaweedFS](https://github.com/seaweedfs/seaweedfs)
- [whisper.cpp](https://github.com/ggerganov/whisper.cpp)
- [ffmpeg](https://ffmpeg.org)
- [react-pdf](https://github.com/wojtekmaj/react-pdf)
- [react-reader](https://github.com/happyr/react-reader)

View file

@ -0,0 +1,7 @@
---
title: License
---
OpenReader is licensed under the MIT License.
- Repository license file: [LICENSE](https://github.com/richardr1126/openreader/blob/main/LICENSE)

View file

@ -0,0 +1,19 @@
---
title: Support and Contributing
---
## Feature requests
Use [GitHub Discussions](https://github.com/richardr1126/openreader/discussions) for feature requests and ideas.
## Issues and support
If you encounter a bug, open an issue in [GitHub Issues](https://github.com/richardr1126/openreader/issues).
## Contributing
Contributions are welcome.
- Fork the repository
- Create your branch
- Open a pull request with your changes

View file

@ -0,0 +1,46 @@
---
title: Auth
---
This page covers application-level configuration for provider access and authentication.
## Auth behavior
- Auth is enabled only when both `BASE_URL` and `AUTH_SECRET` are set.
- Remove either value to disable auth.
- Keep `AUTH_TRUSTED_ORIGINS` empty to trust only `BASE_URL`.
- Anonymous auth sessions are disabled by default.
- Set `USE_ANONYMOUS_AUTH_SESSIONS=true` to enable anonymous session flows.
## Route behavior
- `/` is a public landing/onboarding page and remains indexable.
- `/app` is the protected app home (document list and uploader UI).
- If auth is enabled and a valid session exists (including anonymous), visiting `/` redirects to `/app`.
- Protected app routes continue to require auth; when anonymous sessions are disabled and no session exists, users are redirected to `/signin`.
## Related docs
- For auth environment variables: [Environment Variables](../reference/environment-variables#auth-and-identity)
- For TTS character limits and quota behavior: [TTS Rate Limiting](./tts-rate-limiting)
- For provider-specific guidance: [TTS Providers](./tts-providers)
- For storage/S3/SeaweedFS behavior: [Object / Blob Storage](./object-blob-storage)
- For database mode: [Database](./database)
- For migration behavior and commands: [Migrations](./migrations)
## Sync notes
### Auth enabled
- Settings and reading progress are saved to the server.
- Updates are not instant push-based sync; they use normal client polling/refresh behavior.
- If two devices change the same item around the same time, the newest update wins.
### Auth disabled
- Settings and reading progress stay local in the browser (Dexie/IndexedDB).
- This avoids no-auth cross-browser conflicts, but there is no cross-device sync.
## Claim modal note
- You may still see old anonymous settings/progress available to claim from older deployments.

View file

@ -0,0 +1,39 @@
---
title: Database
---
This page covers database mode selection for OpenReader.
## Database mode
- SQLite (default): embedded DB at `docstore/sqlite3.db`; good for local/self-host single-instance setups.
- Postgres: enabled when `POSTGRES_URL` is set; recommended for production/distributed deployments.
## What the database stores
- Document and audiobook metadata/state used by server routes.
- Auth/session tables (`user`, `session`, `account`, `verification`) when auth is enabled — schema is auto-generated by Better Auth.
- TTS character usage counters (`user_tts_chars`) for daily rate limiting (when enabled).
- User settings preferences (`user_preferences`) when auth is enabled.
- User reading progress (`user_document_progress`) when auth is enabled.
- Document preview job/asset metadata (`document_previews`) for server-side PDF/EPUB thumbnails.
App-specific tables are manually maintained in Drizzle schema files, while auth tables are generated by the Better Auth CLI. Both are migrated together via Drizzle. See [Migrations](./migrations) for details.
## Related variables
- `POSTGRES_URL`
For database variable behavior, see [Environment Variables](../reference/environment-variables#database-and-object-blob-storage).
## Related docs
- [Migrations](./migrations)
- [Object / Blob Storage](./object-blob-storage)
- [Auth](./auth)
## State sync summary
- With auth enabled, settings and reading progress are stored in SQL and synced from the app.
- With auth disabled, settings and reading progress remain local in the browser.
- Sync is currently request-based (not realtime push invalidation).

View file

@ -0,0 +1,132 @@
---
title: Migrations
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
This page covers migration behavior for both database schema and storage data in OpenReader.
## Startup migration behavior
By default, the shared entrypoint runs migrations automatically before app startup in:
- Docker container startup
- `pnpm dev`
- `pnpm start`
Startup migration phases:
- DB schema migrations (`pnpm migrate`)
- Storage/data migration (`pnpm migrate-fs`) for legacy filesystem content into S3 + DB rows
:::info
In most setups, you do not need to run migration commands manually because startup handles this automatically.
:::
To skip automatic startup migrations:
- Set `RUN_DRIZZLE_MIGRATIONS=false`
- Set `RUN_FS_MIGRATIONS=false`
:::warning
If you disable startup migrations, ensure your deployment process runs migrations before serving traffic.
:::
## Apply migrations
In most cases, you do not need manual migration commands because startup runs migrations automatically.
`pnpm migrate` applies migrations for one database target:
- Postgres when `POSTGRES_URL` is set
- SQLite when `POSTGRES_URL` is unset
You can always override the target explicitly with `--config`.
<Tabs groupId="apply-migration-commands">
<TabItem value="project-scripts" label="Project Scripts" default>
```bash
# Run pending migrations for one target:
# - Postgres if POSTGRES_URL is set
# - SQLite if POSTGRES_URL is unset
pnpm migrate
# Run storage migration (filesystem -> S3 + DB)
pnpm migrate-fs
# Dry-run storage migration without uploading/deleting
pnpm migrate-fs:dry-run
```
</TabItem>
<TabItem value="drizzle-direct" label="Manual Drizzle Cmd">
```bash
# Migrate SQLite
pnpm exec drizzle-kit migrate --config drizzle.config.sqlite.ts
# Migrate Postgres
pnpm exec drizzle-kit migrate --config drizzle.config.pg.ts
```
</TabItem>
</Tabs>
## Generate migrations
`pnpm generate` is a two-phase script for contributors and schema changes:
1. **Better Auth schema generation** — runs the Better Auth CLI twice (once for SQLite, once for Postgres) to produce auto-generated Drizzle schema files for auth tables (`user`, `session`, `account`, `verification`).
2. **Drizzle migration generation** — runs `drizzle-kit generate` for both `drizzle.config.sqlite.ts` and `drizzle.config.pg.ts`, producing SQL migration files from all schema files (app + auth).
:::note
Most users do not need to run `pnpm generate`. Use it when contributing or when you have changed Drizzle schema files and need new migration files.
:::
### Schema ownership
Auth tables are owned by Better Auth. Their Drizzle schema definitions are auto-generated and should **not** be hand-edited:
- `src/db/schema_auth_sqlite.ts`
- `src/db/schema_auth_postgres.ts`
App-specific tables are manually maintained in the standard Drizzle schema files:
- `src/db/schema_sqlite.ts`
- `src/db/schema_postgres.ts`
Both sets of schema files are included in the Drizzle configs, so `drizzle-kit generate` and `drizzle-kit migrate` handle all tables together.
<Tabs groupId="generate-migration-commands">
<TabItem value="project-script" label="Project Script" default>
```bash
# Full pipeline: Better Auth CLI + Drizzle generate (both dialects)
pnpm generate
```
</TabItem>
<TabItem value="drizzle-direct" label="Manual Drizzle Cmd">
```bash
# Generate SQLite migrations only (skips Better Auth CLI)
pnpm exec drizzle-kit generate --config drizzle.config.sqlite.ts
# Generate Postgres migrations only (skips Better Auth CLI)
pnpm exec drizzle-kit generate --config drizzle.config.pg.ts
```
:::warning
Running `drizzle-kit generate` directly skips the Better Auth CLI step. If auth schema has changed upstream (e.g. after a Better Auth version bump), run `pnpm generate` instead to regenerate the auth schema files first.
:::
</TabItem>
</Tabs>
## Related docs
- [Database](./database)
- [Object / Blob Storage](./object-blob-storage)
- [Migration Environment Variables](../reference/environment-variables#migration-controls)

View file

@ -0,0 +1,103 @@
---
title: Object / Blob Storage
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
This page documents storage backends, blob upload routing, and core Docker mount behavior.
## Storage backends
- Embedded (default): SQLite metadata + embedded SeaweedFS (`weed mini`) blobs.
- External: Postgres + external S3-compatible object storage.
Storage variables are documented in [Environment Variables](../reference/environment-variables#database-and-object-blob-storage).
## Ports
- `3003`: OpenReader app and API routes
- `8333`: Embedded SeaweedFS S3 endpoint for direct browser blob access
:::info
`8333` is only needed for direct browser presigned access to embedded SeaweedFS.
:::
## Upload behavior
- Primary path: browser uploads to presigned URL from `/api/documents/blob/upload/presign`.
- Fallback path: `/api/documents/blob/upload/fallback` when direct upload fails/unreachable.
- Read/download path: blob/content serving route `/api/documents/blob` (not the upload fallback route).
- Preview path: `/api/documents/blob/preview` (returns `202` while a preview is generating; serves/redirects when ready).
## Document previews
- PDF/EPUB previews are generated server-side and stored in object storage under `document_previews_v1`.
- Preview generation is triggered on upload registration and also backfills on first preview request for older docs.
- Preview artifacts are temporary-cache friendly and can be regenerated from the source document blob.
## FS / Volume Mounts
### App data mount
- Target: `/app/docstore`
- Recommended: yes, for persistence
- Purpose: persists SeaweedFS blob data, SQLite metadata DB, migrations, and local runtime temp state
- Mount string: `-v openreader_docstore:/app/docstore`
### Library source mount (optional)
- Target: `/app/docstore/library`
- Recommended: optional, use read-only (`:ro`)
- Purpose: exposes host files as a source for server library import
- Mount string: `-v /path/to/your/library:/app/docstore/library:ro`
- Details: [Server Library Import](./server-library-import)
## Private blob endpoint mode
If `8333` is not published externally:
- Document uploads still work through upload fallback proxy
- Reads/snippets continue through app API routes
- Direct presigned browser upload/download to embedded endpoint is unavailable
:::warning
Without `8333`, expect higher app-server traffic because uploads/downloads go through API routes instead of direct object endpoint access.
:::
## Audiobook Storage Debug Commands
Audiobook assets are stored in object storage under the `audiobooks_v1` keyspace. Use these commands to inspect and download objects for debugging.
<Tabs groupId="audiobook-storage-access-cli">
<TabItem value="aws-s3" label="AWS S3" default>
```bash
# List all audiobook objects
aws s3 ls "s3://$S3_BUCKET/$S3_PREFIX/audiobooks_v1/" --recursive
# Filter to one book id (replace <book-id>)
aws s3 ls "s3://$S3_BUCKET/$S3_PREFIX/audiobooks_v1/" --recursive | grep "<book-id>-audiobook/"
# Download one object by full key
aws s3 cp "s3://$S3_BUCKET/$S3_PREFIX/audiobooks_v1/<path>/<file>.m4b" "./audiobook.m4b"
```
</TabItem>
<TabItem value="s3-compatible" label="Embedded / MinIO / R2 / etc">
```bash
# List all audiobook objects
aws s3 ls "s3://$S3_BUCKET/$S3_PREFIX/audiobooks_v1/" --recursive --endpoint-url "$S3_ENDPOINT"
# Filter to one book id (replace <book-id>)
aws s3 ls "s3://$S3_BUCKET/$S3_PREFIX/audiobooks_v1/" --recursive --endpoint-url "$S3_ENDPOINT" | grep "<book-id>-audiobook/"
# Download one object by full key
aws s3 cp "s3://$S3_BUCKET/$S3_PREFIX/audiobooks_v1/<path>/<file>.m4b" "./audiobook.m4b" --endpoint-url "$S3_ENDPOINT"
```
Embedded default example: `S3_ENDPOINT=http://127.0.0.1:8333` (or your mapped host/port).
</TabItem>
</Tabs>

View file

@ -0,0 +1,68 @@
---
title: Server Library Import
---
This page documents how server library import works and how to configure it.
## What it does
Server library import lets you browse files from one or more server directories and import selected files into OpenReader.
- Import is user-driven via a selection modal
- Only selected files are imported
- Imported files become normal OpenReader documents
## FS / Volume Mounts
### App data mount
- Target: `/app/docstore`
- Recommended: yes, for persistence
- Purpose: stores app runtime data, metadata DB, and embedded storage state
- Mount string: `-v openreader_docstore:/app/docstore`
### Library source mount
- Target: `/app/docstore/library`
- Recommended: yes for this feature, use read-only (`:ro`)
- Purpose: exposes host files as import candidates in Server Library Import
- Mount string: `-v /path/to/your/library:/app/docstore/library:ro`
## Import flow
1. Open **Settings -> Documents -> Server Library Import**.
2. Select files in the modal.
3. Click **Import**.
Selected files are fetched from the server library endpoint and imported into OpenReader storage.
:::warning Shared Library Roots
Library roots are configured at the server level (not per-user). Any user who can access Server Library Import can browse/import from the same configured roots.
Imported documents are still saved to the importing user's document scope.
:::
## Supported file types
- `.pdf`
- `.epub`
- `.html`, `.htm`
- `.txt`
- `.md`, `.mdown`, `.markdown`
## Optional: Configure Library Roots
You only need this when the default mounted path is not what you want.
By default, OpenReader uses `docstore/library` as the import root. You can override that with environment variables:
- `IMPORT_LIBRARY_DIRS` (takes precedence): multiple roots separated by comma, colon, or semicolon
- `IMPORT_LIBRARY_DIR`: single root
See [Environment Variables](../reference/environment-variables#library-import) for variable details.
## Notes
- Library listing is capped per request (up to 10,000 files).
- When auth is enabled, library import endpoints require a valid session.
- The mounted library is a source; removing it does not delete already imported documents.

View file

@ -0,0 +1,40 @@
---
title: Custom OpenAI
---
Use any custom OpenAI-compatible TTS service with OpenReader.
Use this integration when your endpoint is not directly covered by built-in dropdown defaults.
## Provider
- Provider: `Custom OpenAI-Like`
- `API_BASE`: required (your service base URL)
- `API_KEY`: set if required by your service
Custom providers should expose:
- `GET /v1/audio/voices`
- `POST /v1/audio/speech`
## OpenReader setup
1. In OpenReader Settings, choose provider `Custom OpenAI-Like`.
2. Set `API_BASE` to your service base URL (typically ending with `/v1`).
3. Set `API_KEY` if your service requires authentication.
4. Choose a model and voice supported by your backend.
## Notes
:::warning Compatibility required
Custom providers must implement OpenAI-compatible TTS endpoints, including `GET /v1/audio/voices` and `POST /v1/audio/speech`.
:::
:::info Voice troubleshooting
If voices do not load, verify the `/v1/audio/voices` response shape and that the endpoint is reachable from the OpenReader server.
:::
## References
- [TTS Providers](../tts-providers)
- [TTS Environment Variables](../../reference/environment-variables#tts-provider-and-request-behavior)

View file

@ -0,0 +1,33 @@
---
title: Deepinfra
---
Use Deepinfra as a hosted OpenAI-compatible TTS provider.
## Provider
- Provider: `Deepinfra`
- Default endpoint: `https://api.deepinfra.com/v1/openai` (auto-filled)
- `API_KEY`: required for authenticated DeepInfra usage
## OpenReader setup
1. In OpenReader Settings, choose provider `Deepinfra`.
2. Keep the default `API_BASE`.
3. Set `API_KEY`.
4. Choose your model and voice.
## Notes
:::tip Built-in endpoint
`Deepinfra` is a built-in provider, so OpenReader auto-fills the default `API_BASE`.
:::
:::info Model support
DeepInfra exposes multiple TTS models, including Kokoro-family options.
:::
## References
- [TTS Providers](../tts-providers)
- [TTS Environment Variables](../../reference/environment-variables#tts-provider-and-request-behavior)

View file

@ -0,0 +1,68 @@
---
title: Kokoro-FastAPI
---
You can run the Kokoro TTS API server directly with Docker.
:::warning
For Kokoro issues and support, use the upstream repository: [remsky/Kokoro-FastAPI](https://github.com/remsky/Kokoro-FastAPI).
:::
## Provider
- Provider: `Custom OpenAI-Like`
- Typical model: `Kokoro`
- `API_BASE`: required (typically your Kokoro URL ending with `/v1`)
- `API_KEY`: set only if your deployment requires one
## Run Kokoro (CPU)
```bash
docker run --name kokoro-tts \
--restart unless-stopped \
-d \
-p 8880:8880 \
-e ONNX_NUM_THREADS=8 \
-e ONNX_INTER_OP_THREADS=4 \
-e ONNX_EXECUTION_MODE=parallel \
-e ONNX_OPTIMIZATION_LEVEL=all \
-e ONNX_MEMORY_PATTERN=true \
-e ONNX_ARENA_EXTEND_STRATEGY=kNextPowerOfTwo \
-e API_LOG_LEVEL=DEBUG \
ghcr.io/remsky/kokoro-fastapi-cpu:v0.2.4
```
## Run Kokoro (GPU)
```bash
docker run --name kokoro-tts \
--restart unless-stopped \
-d \
--gpus all \
--user 1001:1001 \
-p 8880:8880 \
-e USE_GPU=true \
-e PYTHONUNBUFFERED=1 \
-e API_LOG_LEVEL=DEBUG \
ghcr.io/remsky/kokoro-fastapi-gpu:v0.2.4
```
## OpenReader setup
1. Start Kokoro using either the CPU or GPU image.
2. In OpenReader Settings, choose provider `Custom OpenAI-Like`.
3. Set `API_BASE` to your Kokoro endpoint (for Docker Compose, commonly `http://kokoro-tts:8880/v1`).
4. Set `API_KEY` only if your deployment requires one.
5. Choose model `Kokoro`.
## Notes
:::tip Runtime guidance
GPU mode requires NVIDIA Docker support and is best on NVIDIA hardware. CPU mode is a good default on Apple Silicon and modern x86 CPUs.
:::
## References
- [remsky/Kokoro-FastAPI](https://github.com/remsky/Kokoro-FastAPI)
- [TTS Providers](../tts-providers)
- [TTS Environment Variables](../../reference/environment-variables#tts-provider-and-request-behavior)

View file

@ -0,0 +1,33 @@
---
title: OpenAI
---
Use OpenAI directly as an OpenAI-compatible TTS provider.
## Provider
- Provider: `OpenAI`
- Default endpoint: `https://api.openai.com/v1` (auto-filled)
- `API_KEY`: required for OpenAI access
## OpenReader setup
1. In OpenReader Settings, choose provider `OpenAI`.
2. Keep the default `API_BASE`.
3. Set `API_KEY`.
4. Choose your model and voice.
## Notes
:::tip Built-in endpoint
`OpenAI` is a built-in provider, so OpenReader auto-fills the default `API_BASE`.
:::
:::info Server-side requests
OpenReader sends TTS requests from the server runtime, not directly from the browser.
:::
## References
- [TTS Providers](../tts-providers)
- [TTS Environment Variables](../../reference/environment-variables#tts-provider-and-request-behavior)

View file

@ -0,0 +1,36 @@
---
title: Orpheus-FastAPI
---
Use Orpheus-FastAPI as an OpenAI-compatible TTS backend for OpenReader.
## Provider
- Provider: `Custom OpenAI-Like`
- Typical model: `Orpheus`
- `API_BASE`: required (usually your Orpheus URL ending with `/v1`)
- `API_KEY`: set only if your deployment requires one
## OpenReader setup
1. Start your Orpheus-FastAPI server.
2. In OpenReader Settings, choose provider `Custom OpenAI-Like`.
3. Set `API_BASE` to your Orpheus base URL (typically ending with `/v1`).
4. Set `API_KEY` only if your Orpheus deployment requires one.
5. Choose model `Orpheus` (or another model exposed by your deployment).
## Notes
:::info OpenAI-compatible API
OpenReader expects OpenAI-compatible audio endpoints when using Orpheus through `Custom OpenAI-Like`.
:::
:::tip Endpoint shape
Use an `API_BASE` that points at the Orpheus API root (typically ending with `/v1`).
:::
## References
- [Lex-au/Orpheus-FastAPI](https://github.com/Lex-au/Orpheus-FastAPI)
- [TTS Providers](../tts-providers)
- [TTS Environment Variables](../../reference/environment-variables#tts-provider-and-request-behavior)

View file

@ -0,0 +1,79 @@
---
title: TTS Providers
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
OpenReader supports OpenAI-compatible TTS providers through a common API shape.
:::tip
If you are running a self-hosted TTS server (Kokoro/Orpheus/etc.), use **Custom OpenAI-Like** in Settings.
:::
## Quick Setup by Provider
<Tabs groupId="tts-provider-setup">
<TabItem value="openai" label="OpenAI" default>
1. In Settings, choose provider: `OpenAI`.
2. Keep the default `API_BASE` (auto-filled).
3. Set `API_KEY` to your OpenAI key.
4. Choose model/voice.
</TabItem>
<TabItem value="deepinfra" label="DeepInfra">
1. In Settings, choose provider: `Deepinfra`.
2. Keep the default `API_BASE` (auto-filled).
3. Set `API_KEY` to your DeepInfra key.
4. Choose model/voice.
</TabItem>
<TabItem value="custom" label="Custom OpenAI-Like">
1. In Settings, choose provider: `Custom OpenAI-Like`.
2. Set `API_BASE` to your endpoint (example: `http://host.docker.internal:8880/v1`).
3. Set `API_KEY` if your provider requires one.
4. Choose model/voice.
</TabItem>
</Tabs>
## Provider Dropdown Behavior
In Settings, provider options include:
- `OpenAI`
- `Deepinfra`
- `Custom OpenAI-Like` (Kokoro, Orpheus, and other OpenAI-compatible endpoints)
`API_BASE` guidance:
- `OpenAI` and `Deepinfra` auto-fill default endpoints.
- `Custom OpenAI-Like` requires setting `API_BASE` manually.
:::info OpenAI-Compatible API Shape
Custom providers should expose:
- `GET /v1/audio/voices`
- `POST /v1/audio/speech`
:::
:::warning Server-Reachable API Base
TTS requests are sent from the Next.js server, not directly from the browser. `API_BASE` must be reachable from the server runtime.
:::
## Provider Guides
- [Kokoro-FastAPI](./tts-provider-guides/kokoro-fastapi)
- [Orpheus-FastAPI](./tts-provider-guides/orpheus-fastapi)
- [DeepInfra](./tts-provider-guides/deepinfra)
- [OpenAI](./tts-provider-guides/openai)
- [Custom OpenAI-Like](./tts-provider-guides/custom-openai)
## Related Configuration
- [TTS Environment Variables](../reference/environment-variables#tts-provider-and-request-behavior)
- [TTS Rate Limiting](./tts-rate-limiting)
- [Auth](./auth)

View file

@ -0,0 +1,51 @@
---
title: TTS Rate Limiting
---
This page explains OpenReader's TTS character rate limiting controls.
## Overview
- TTS rate limiting is disabled by default.
- To enable it, set `TTS_ENABLE_RATE_LIMIT=true`.
- Limits are enforced per day in UTC.
- Enforcement applies only when auth is enabled.
## How enforcement works
When enabled, OpenReader enforces:
- Per-user daily character limits.
- IP backstop daily character limits.
- Anonymous device backstop tracking (cookie-based) to reduce limit resets.
If a request exceeds the active limit, the TTS API returns `429` with reset metadata for the next UTC day.
## Required auth behavior
- Auth must be enabled (`BASE_URL` + `AUTH_SECRET`) for TTS char limits to apply.
- If auth is disabled, TTS character limits are effectively unlimited.
- `DISABLE_AUTH_RATE_LIMIT` only affects Better Auth's own request throttling.
- `DISABLE_AUTH_RATE_LIMIT` does not disable TTS character limits.
## Environment variables
Enable/disable:
- `TTS_ENABLE_RATE_LIMIT` (default: `false`)
Per-user daily limits:
- `TTS_DAILY_LIMIT_ANONYMOUS` (default: `50000`)
- `TTS_DAILY_LIMIT_AUTHENTICATED` (default: `500000`)
IP backstop daily limits:
- `TTS_IP_DAILY_LIMIT_ANONYMOUS` (default: `100000`)
- `TTS_IP_DAILY_LIMIT_AUTHENTICATED` (default: `1000000`)
## Related docs
- TTS/rate-limit environment variables: [Environment Variables](../reference/environment-variables#tts-provider-and-request-behavior)
- Auth configuration: [Auth](./auth)
- Provider setup: [TTS Providers](./tts-providers)

View file

@ -0,0 +1,138 @@
---
title: Local Development
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
## Prerequisites
- Node.js (recommended with [nvm](https://github.com/nvm-sh/nvm))
- `pnpm` (recommended) or `npm`
```bash
npm install -g pnpm
```
- A reachable TTS API server
- [SeaweedFS](https://github.com/seaweedfs/seaweedfs) `weed` binary (required unless using external S3 storage)
<Tabs groupId="seaweedfs-install">
<TabItem value="macos" label="macOS" default>
```bash
brew install seaweedfs
```
</TabItem>
<TabItem value="linux" label="Linux">
Install the `weed` binary from the [SeaweedFS releases](https://github.com/seaweedfs/seaweedfs/releases) and ensure it is available on `PATH`.
</TabItem>
</Tabs>
Optional, depending on features:
- [libreoffice](https://www.libreoffice.org) (required for DOCX conversion)
```bash
brew install libreoffice
```
- [whisper.cpp](https://github.com/ggml-org/whisper.cpp) (optional, for word-by-word highlighting)
```bash
# clone and build whisper.cpp (no model download needed OpenReader handles that)
git clone https://github.com/ggml-org/whisper.cpp.git
cd whisper.cpp
cmake -B build
cmake --build build -j --config Release
# point OpenReader to the compiled whisper-cli binary
echo WHISPER_CPP_BIN="$(pwd)/build/bin/whisper-cli"
```
:::tip
Set `WHISPER_CPP_BIN` in your `.env` to enable word-by-word highlighting.
:::
## Steps
1. Clone the repository.
```bash
git clone https://github.com/richardr1126/openreader.git
cd openreader
```
2. Install dependencies.
```bash
pnpm i
```
3. Configure the environment.
```bash
cp .env.example .env
```
Then edit `.env`.
- No auth mode: leave `BASE_URL` or `AUTH_SECRET` unset.
- Auth enabled mode: set both `BASE_URL` (typically `http://localhost:3003`) and `AUTH_SECRET` (generate with `openssl rand -hex 32`).
Optional:
- `AUTH_TRUSTED_ORIGINS=http://localhost:3003,http://192.168.0.116:3003`
- Stable S3 credentials via `S3_ACCESS_KEY_ID` and `S3_SECRET_ACCESS_KEY`
- External S3 storage by setting `USE_EMBEDDED_WEED_MINI=false` and related S3 vars
:::info
For all environment variables, see [Environment Variables](../reference/environment-variables).
:::
See [Auth](../configure/auth) for app/auth behavior.
Storage configuration details are in [Object / Blob Storage](../configure/object-blob-storage).
Refer to [Database](../configure/database) for database modes.
Learn about migration behavior and commands in [Migrations](../configure/migrations).
4. Run DB migrations.
- Migrations run automatically on startup through the shared entrypoint for both `pnpm dev` and `pnpm start`.
- You only need manual migration commands for one-off troubleshooting or explicit migration workflows:
```bash
pnpm migrate
```
:::info
If `POSTGRES_URL` is set, migrations target Postgres; otherwise local SQLite is used. To disable automatic startup migrations, set `RUN_DRIZZLE_MIGRATIONS=false` and/or `RUN_FS_MIGRATIONS=false`. You can run storage migration manually with `pnpm migrate-fs`.
:::
5. Start the app.
<Tabs groupId="local-run-mode">
<TabItem value="dev" label="Dev" default>
```bash
pnpm dev
```
</TabItem>
<TabItem value="prod" label="Build + Start">
```bash
pnpm build
pnpm start
```
</TabItem>
</Tabs>
:::warning API Base Reachability
`API_BASE` must be reachable from the Next.js server process, not just your browser.
:::
Visit [http://localhost:3003](http://localhost:3003).

View file

@ -0,0 +1,106 @@
---
title: Vercel Deployment
---
This guide covers deploying OpenReader to Vercel with external Postgres and S3-compatible object storage.
## What works on Vercel
- Documents (PDF/EPUB/TXT/MD) work with `POSTGRES_URL` + external S3 storage.
- Audiobook routes work on Node.js serverless functions using `ffmpeg-static`.
:::warning DOCX Conversion Limitation
`docx` conversion requires `soffice` (LibreOffice), which is not available in a standard Vercel runtime.
:::
## 1. Environment Variables
Recommended production setup (auth enabled):
```bash
API_BASE=https://api.deepinfra.com/v1/openai
API_KEY=your_deepinfra_key
POSTGRES_URL=postgres://...
USE_EMBEDDED_WEED_MINI=false
S3_ACCESS_KEY_ID=...
S3_SECRET_ACCESS_KEY=...
S3_BUCKET=...
S3_REGION=us-east-1
S3_PREFIX=openreader
BASE_URL=https://your-app.vercel.app
AUTH_SECRET=...
# Optional client/runtime feature defaults:
NEXT_PUBLIC_ENABLE_DOCX_CONVERSION=false
NEXT_PUBLIC_ENABLE_DESTRUCTIVE_DELETE_ACTIONS=false
NEXT_PUBLIC_DEFAULT_TTS_PROVIDER=deepinfra
NEXT_PUBLIC_DEFAULT_TTS_MODEL=hexgrad/Kokoro-82M
NEXT_PUBLIC_ENABLE_AUDIOBOOK_EXPORT=true
NEXT_PUBLIC_ENABLE_WORD_HIGHLIGHT=false
# Optional (non-AWS S3-compatible providers):
# S3_ENDPOINT=https://...
# S3_FORCE_PATH_STYLE=true
```
:::info Production Configuration & Feature Flags
We recommend setting these defaults for a production-like environment:
- `NEXT_PUBLIC_ENABLE_DOCX_CONVERSION=false`: Disables DOCX upload (requires external tools anyway)
- `NEXT_PUBLIC_ENABLE_DESTRUCTIVE_DELETE_ACTIONS=false`: Hides destructive "Delete All" actions
- `NEXT_PUBLIC_DEFAULT_TTS_PROVIDER=deepinfra`: Points default TTS to a scalable provider
- `NEXT_PUBLIC_DEFAULT_TTS_MODEL=hexgrad/Kokoro-82M`: Uses a high-quality default model
- `NEXT_PUBLIC_ENABLE_AUDIOBOOK_EXPORT=true`: (Optional) Controls audiobook export UI
- `NEXT_PUBLIC_ENABLE_WORD_HIGHLIGHT=false`: (Optional) Controls word highlighting UI (requires timestamp backend)
:::
:::warning Auth recommendation
For internet-exposed Vercel deployments, set both `BASE_URL` and `AUTH_SECRET`. Running without auth is possible, but not recommended for public environments.
:::
:::tip
For all variables and defaults, see [Environment Variables](../reference/environment-variables).
:::
## 2. FFmpeg packaging in Vercel functions
`ffmpeg-static` binaries must be included in function traces. This repo already does that in `next.config.ts` via `outputFileTracingIncludes` for:
- `/api/audiobook`
- `/api/audiobook/chapter`
- `/api/audiobook/status`
- `/api/whisper`
:::info
`serverExternalPackages` should include `ffmpeg-static` so package paths resolve at runtime instead of being bundled into route output.
:::
If you change route paths or split handlers, update `outputFileTracingIncludes` accordingly.
## 3. Function memory sizing
FFmpeg workloads benefit from more memory/CPU. This repo includes:
```json
{
"$schema": "https://openapi.vercel.sh/vercel.json",
"functions": {
"app/api/audiobook/route.ts": { "memory": 3009 },
"app/api/whisper/route.ts": { "memory": 3009 }
}
}
```
Adjust memory per route if your files are larger or your plan differs.
## 4. Runtime expectations and caveats
- Audiobook APIs require S3 configuration; otherwise they return `503`.
- For production Vercel deploys, use `POSTGRES_URL` instead of SQLite.
- Filesystem-to-object-store migrations run via server scripts/entrypoint (`scripts/migrate-fs-v2.mjs`), not API routes.
- Vercel deployments do not run `scripts/openreader-entrypoint.mjs`, so run `pnpm migrate-fs` in a controlled environment when migrating legacy filesystem data.
## 5. Smoke test after deploy
1. Upload and read a PDF/EPUB document.
2. Confirm sync/blob fetch works across refreshes/devices.
3. Generate at least one audiobook chapter and play/download it.
4. If using word highlighting, verify timestamps are produced and rendered.

View file

@ -0,0 +1,121 @@
---
title: Docker Quick Start
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
## Prerequisites
- A recent Docker version installed
- A TTS API server that OpenReader can reach (Kokoro-FastAPI, Orpheus-FastAPI, DeepInfra, OpenAI, or equivalent)
:::note
If you have suitable hardware, you can run Kokoro locally with Docker. See [Kokoro-FastAPI](./configure/tts-provider-guides/kokoro-fastapi).
:::
## 1. Start the Docker container
<Tabs groupId="docker-start-mode">
<TabItem value="minimal" label="Minimal" default>
Auth disabled, embedded storage ephemeral, no library import:
```bash
docker run --name openreader \
--restart unless-stopped \
-p 3003:3003 \
-p 8333:8333 \
ghcr.io/richardr1126/openreader:latest
```
</TabItem>
<TabItem value="localhost" label="Localhost">
Persistent storage, embedded SeaweedFS `weed mini`, optional auth, optional library mount:
```bash
docker run --name openreader \
--restart unless-stopped \
-p 3003:3003 \
-p 8333:8333 \
-v openreader_docstore:/app/docstore \
-v /path/to/your/library:/app/docstore/library:ro \
-e API_BASE=http://host.docker.internal:8880/v1 \
-e API_KEY=none \
-e BASE_URL=http://localhost:3003 \
-e AUTH_SECRET=$(openssl rand -hex 32) \
ghcr.io/richardr1126/openreader:latest
```
</TabItem>
<TabItem value="local-network" label="LAN Host">
Use this when the app should be reachable from other devices on your LAN:
```bash
docker run --name openreader \
--restart unless-stopped \
-p 3003:3003 \
-p 8333:8333 \
-v openreader_docstore:/app/docstore \
-e API_BASE=http://host.docker.internal:8880/v1 \
-e BASE_URL=http://<YOUR_LAN_IP>:3003 \
-e AUTH_SECRET=$(openssl rand -hex 32) \
-e AUTH_TRUSTED_ORIGINS=http://localhost:3003,http://127.0.0.1:3003 \
-e USE_ANONYMOUS_AUTH_SESSIONS=true \
ghcr.io/richardr1126/openreader:latest
```
Replace `<YOUR_LAN_IP>` with the Docker host IP address on your local network to allow access from other devices.
</TabItem>
</Tabs>
:::tip Quick Tips
- Remove `/app/docstore/library` if you do not need server library import.
- Remove either `BASE_URL` or `AUTH_SECRET` to keep auth disabled.
- Set `API_BASE` to your reachable TTS server base URL.
:::
:::warning Port `8333` Exposure
Expose `8333` for direct browser presigned upload/download with embedded SeaweedFS.
If `8333` is not reachable from the browser, direct presigned access is unavailable. Uploads can still fall back to `/api/documents/blob/upload/fallback`, and document reads/downloads continue through `/api/documents/blob`.
:::
:::info Auth and Migrations
- Auth is enabled only when both `BASE_URL` and `AUTH_SECRET` are set.
- DB/storage migrations run automatically at container startup via the shared entrypoint.
:::
:::info Related Docs
- [Environment Variables](./reference/environment-variables)
- [Auth](./configure/auth)
- [Database](./configure/database)
- [Object / Blob Storage](./configure/object-blob-storage)
- [Migrations](./configure/migrations)
:::
## 2. Configure settings in the app UI
- Set TTS provider and model in Settings
- Set TTS API base URL and API key if needed
- Select the model voice from the voice dropdown
## 3. Update Docker image
Legacy image compatibility: `ghcr.io/richardr1126/openreader-webui:latest` remains available as an alias.
```bash
docker stop openreader || true && \
docker rm openreader || true && \
docker image rm ghcr.io/richardr1126/openreader:latest || true && \
docker pull ghcr.io/richardr1126/openreader:latest
```
:::tip
If you use a mounted volume for `/app/docstore`, your persisted data remains after image updates.
:::
Visit [http://localhost:3003](http://localhost:3003) after startup.

View file

@ -0,0 +1,51 @@
---
id: intro
title: Introduction
slug: /
---
OpenReader is an open source text-to-speech document reader built with Next.js. It provides a read-along experience with narration for **EPUB, PDF, TXT, MD, and DOCX documents**.
> Previously named **OpenReader-WebUI**.
It supports multiple TTS providers including OpenAI, DeepInfra, and custom OpenAI-compatible endpoints such as [Kokoro-FastAPI](https://github.com/remsky/Kokoro-FastAPI) and [Orpheus-FastAPI](https://github.com/Lex-au/Orpheus-FastAPI).
## ✨ Highlights
- 🎯 **Multi-Provider TTS Support**
- [**Kokoro-FastAPI**](https://github.com/remsky/Kokoro-FastAPI): supports multi-voice combinations (for example `af_heart+af_bella`)
- [**Orpheus-FastAPI**](https://github.com/Lex-au/Orpheus-FastAPI)
- **Custom OpenAI-compatible**: any TTS API with `/v1/audio/voices` and `/v1/audio/speech` endpoints
- **Cloud TTS providers**:
- [**DeepInfra**](https://deepinfra.com/models/text-to-speech): Kokoro-82M and other hosted models
- [**OpenAI API**](https://platform.openai.com/docs/pricing#transcription-and-speech): `tts-1`, `tts-1-hd`, and `gpt-4o-mini-tts`
- 🛜 **Server-side Document Storage**
- Documents are persisted in server blob/object storage for consistent access
- 📚 **External Library Import**
- Import documents from server-mounted folders
- 🎧 **Server-side Audiobook Export** in `m4b`/`mp3` with resumable chapter generation
- 📖 **Read Along Experience**
- Real-time highlighting for PDF/EPUB, with optional word-level [whisper.cpp](https://github.com/ggml-org/whisper.cpp) timestamps
- 🔐 **Auth Optional by Design**
- Run no-auth for local use, or enable auth with user isolation and claim flow
- 🗂️ **Flexible Storage and Database Modes** with embedded defaults or external S3/Postgres
- 🚀 **Production-ready Server Behavior** with TTS caching/retries/rate limits and startup migrations
- 🎨 **Customizable Experience**
- Theme, TTS, and document handling controls
## 🧭 Key Docs
- [Docker Quick Start](./docker-quick-start)
- [Local Development](./deploy/local-development)
- [Vercel Deployment](./deploy/vercel-deployment)
- [Environment Variables](./reference/environment-variables)
- [Auth](./configure/auth)
- [Database](./configure/database)
- [Object / Blob Storage](./configure/object-blob-storage)
- [Migrations](./configure/migrations)
- [Server Library Import](./configure/server-library-import)
- [TTS Providers](./configure/tts-providers)
## Source Repository
- GitHub: [richardr1126/openreader](https://github.com/richardr1126/openreader)

View file

@ -0,0 +1,396 @@
---
title: Environment Variables
toc_max_heading_level: 3
---
This is the single reference page for OpenReader environment variables.
## Quick Reference Table
| Variable | Area | Default | When to set |
| --- | --- | --- | --- |
| `NEXT_PUBLIC_ENABLE_DOCX_CONVERSION` | Client feature flags | `true` unless set to `false` | Set `false` to hide DOCX support |
| `NEXT_PUBLIC_ENABLE_DESTRUCTIVE_DELETE_ACTIONS` | Client feature flags | `true` unless set to `false` | Set `false` to hide destructive actions |
| `NEXT_PUBLIC_DEFAULT_TTS_PROVIDER` | Client feature flags | `custom-openai` | Override default TTS provider |
| `NEXT_PUBLIC_DEFAULT_TTS_MODEL` | Client feature flags | `kokoro` | Override default TTS model |
| `NEXT_PUBLIC_ENABLE_AUDIOBOOK_EXPORT` | Client feature flags | `true` unless set to `false` | Set `false` to hide audiobook export UI |
| `NEXT_PUBLIC_ENABLE_WORD_HIGHLIGHT` | Client feature flags | `true` unless set to `false` | Set `false` to disable word highlight + alignment |
| `API_BASE` | TTS provider | none | Point to your OpenAI-compatible TTS base URL |
| `API_KEY` | TTS provider | `none` fallback in TTS route | Set when provider requires auth |
| `TTS_CACHE_MAX_SIZE_BYTES` | TTS caching | `268435456` (256 MB) | Tune in-memory TTS cache size |
| `TTS_CACHE_TTL_MS` | TTS caching | `1800000` (30 min) | Tune in-memory TTS cache TTL |
| `TTS_MAX_RETRIES` | TTS retry | `2` | Tune retry attempts for upstream 429/5xx |
| `TTS_RETRY_INITIAL_MS` | TTS retry | `250` | Tune initial retry delay |
| `TTS_RETRY_MAX_MS` | TTS retry | `2000` | Tune max retry delay |
| `TTS_RETRY_BACKOFF` | TTS retry | `2` | Tune exponential backoff factor |
| `TTS_UPSTREAM_TIMEOUT_MS` | TTS request timeout | `285000` | Set max upstream TTS request duration before fail-fast |
| `TTS_ENABLE_RATE_LIMIT` | Rate limiting | `false` | Set `true` to enable TTS per-user/IP daily character limits |
| `TTS_DAILY_LIMIT_ANONYMOUS` | Rate limiting | `50000` | Override anonymous per-user daily character limit |
| `TTS_DAILY_LIMIT_AUTHENTICATED` | Rate limiting | `500000` | Override authenticated per-user daily character limit |
| `TTS_IP_DAILY_LIMIT_ANONYMOUS` | Rate limiting | `100000` | Override anonymous IP backstop daily limit |
| `TTS_IP_DAILY_LIMIT_AUTHENTICATED` | Rate limiting | `1000000` | Override authenticated IP backstop daily limit |
| `BASE_URL` | Auth | unset | Required (with `AUTH_SECRET`) to enable auth |
| `AUTH_SECRET` | Auth | unset | Required (with `BASE_URL`) to enable auth |
| `AUTH_TRUSTED_ORIGINS` | Auth | empty | Add extra allowed origins |
| `USE_ANONYMOUS_AUTH_SESSIONS` | Auth | `false` | Set `true` to enable anonymous auth sessions |
| `GITHUB_CLIENT_ID` | Auth/OAuth | unset | Set with `GITHUB_CLIENT_SECRET` to enable GitHub sign-in |
| `GITHUB_CLIENT_SECRET` | Auth/OAuth | unset | Set with `GITHUB_CLIENT_ID` to enable GitHub sign-in |
| `DISABLE_AUTH_RATE_LIMIT` | Rate limiting | `false` | Set `true` to disable auth-layer rate limiting |
| `POSTGRES_URL` | Database | unset (SQLite mode) | Set to switch metadata/auth DB to Postgres |
| `USE_EMBEDDED_WEED_MINI` | Storage | `true` when unset | Set `false` to use external S3-compatible storage only |
| `WEED_MINI_DIR` | Storage | `docstore/seaweedfs` | Override embedded SeaweedFS data directory |
| `WEED_MINI_WAIT_SEC` | Storage | `20` | Tune SeaweedFS startup wait timeout |
| `S3_ACCESS_KEY_ID` | Storage | auto-generated in embedded mode | Set explicitly for stable/external credentials |
| `S3_SECRET_ACCESS_KEY` | Storage | auto-generated in embedded mode | Set explicitly for stable/external credentials |
| `S3_BUCKET` | Storage | `openreader-documents` in embedded mode | Required for external S3-compatible storage |
| `S3_REGION` | Storage | `us-east-1` in embedded mode | Required for external S3-compatible storage |
| `S3_ENDPOINT` | Storage | derived in embedded mode | Set for S3-compatible providers (MinIO/SeaweedFS/R2/etc.) |
| `S3_FORCE_PATH_STYLE` | Storage | `true` in embedded mode | Set per provider requirement |
| `S3_PREFIX` | Storage | `openreader` | Customize object key prefix |
| `RUN_DRIZZLE_MIGRATIONS` | Database migrations | `true` | Set `false` to skip startup Drizzle schema migrations |
| `RUN_FS_MIGRATIONS` | Storage migrations | `true` | Set `false` to skip startup filesystem -> S3/DB migration pass |
| `IMPORT_LIBRARY_DIR` | Library import | `docstore/library` fallback | Set a single server library root |
| `IMPORT_LIBRARY_DIRS` | Library import | unset | Set multiple roots (comma/colon/semicolon separated) |
| `WHISPER_CPP_BIN` | Word timing | unset | Set to enable `whisper.cpp` timestamps |
| `FFMPEG_BIN` | Audio runtime | auto-detected (`ffmpeg-static`) | Override ffmpeg binary path |
## TTS Provider and Request Behavior
### API_BASE
Base URL for OpenAI-compatible TTS API requests.
- Example: `http://host.docker.internal:8880/v1`
- Can be overridden per request from UI settings
- Related docs: [TTS Providers](../configure/tts-providers)
### API_KEY
Default API key for TTS provider requests.
- Example: `none` or your provider token
- Can be overridden by request headers from app settings
- Related docs: [TTS Providers](../configure/tts-providers)
### TTS_CACHE_MAX_SIZE_BYTES
Maximum in-memory TTS audio cache size in bytes.
- Default: `268435456` (256 MB)
### TTS_CACHE_TTL_MS
In-memory TTS audio cache TTL in milliseconds.
- Default: `1800000` (30 minutes)
### TTS_MAX_RETRIES
Maximum retries for upstream TTS failures (429/5xx).
- Default: `2`
### TTS_RETRY_INITIAL_MS
Initial retry delay in milliseconds for TTS upstream requests.
- Default: `250`
### TTS_RETRY_MAX_MS
Maximum retry delay in milliseconds.
- Default: `2000`
### TTS_RETRY_BACKOFF
Exponential backoff multiplier between retries.
- Default: `2`
### TTS_UPSTREAM_TIMEOUT_MS
Maximum upstream TTS request timeout in milliseconds.
- Default: `285000` (285 seconds)
- Applies to outbound provider calls from server routes using shared TTS generation
- Increase for slower providers/models; decrease to fail fast and surface retryable errors sooner
### TTS_ENABLE_RATE_LIMIT
Controls TTS character rate limiting in the TTS API.
- Default: `false` (TTS char limits disabled)
- Set to `true` to enforce `TTS_DAILY_LIMIT_*` and `TTS_IP_DAILY_LIMIT_*`
- For behavior details and examples, see [TTS Rate Limiting](../configure/tts-rate-limiting)
### TTS_DAILY_LIMIT_ANONYMOUS
Anonymous per-user daily character limit.
- Default: `50000`
### TTS_DAILY_LIMIT_AUTHENTICATED
Authenticated per-user daily character limit.
- Default: `500000`
### TTS_IP_DAILY_LIMIT_ANONYMOUS
Anonymous IP backstop daily character limit.
- Default: `100000`
### TTS_IP_DAILY_LIMIT_AUTHENTICATED
Authenticated IP backstop daily character limit.
- Default: `1000000`
## Auth and Identity
### BASE_URL
External base URL for this OpenReader instance.
- Required with `AUTH_SECRET` to enable auth
- Example: `http://localhost:3003` or `https://reader.example.com`
- Related docs: [Auth](../configure/auth)
### AUTH_SECRET
Secret key used by auth/session handling.
- Required with `BASE_URL` to enable auth
- Generate with `openssl rand -hex 32`
- Related docs: [Auth](../configure/auth)
### AUTH_TRUSTED_ORIGINS
Additional allowed origins for auth requests.
- Comma-separated list
- `BASE_URL` origin is always trusted automatically
- Related docs: [Auth](../configure/auth)
### USE_ANONYMOUS_AUTH_SESSIONS
Controls whether auth-enabled deployments can create/use anonymous sessions.
- Default: `false` (anonymous sessions disabled)
- Set `true` to allow anonymous sessions and guest-style flows
- When `false`, users must sign in or sign up with an account
- Related docs: [Auth](../configure/auth)
### GITHUB_CLIENT_ID
GitHub OAuth client ID.
- Enable only with `GITHUB_CLIENT_SECRET`
### GITHUB_CLIENT_SECRET
GitHub OAuth client secret.
- Enable only with `GITHUB_CLIENT_ID`
### DISABLE_AUTH_RATE_LIMIT
Controls Better Auth rate limiting.
- Default behavior: auth-layer rate limiting enabled
- Set to `true` to disable auth-layer rate limiting
- This does not affect TTS character rate limiting
- Related docs: [Auth](../configure/auth)
## Database and Object Blob Storage
### POSTGRES_URL
Switches metadata/auth storage from SQLite to Postgres.
- Unset: SQLite at `docstore/sqlite3.db`
- Set: Postgres mode
- Related docs: [Database](../configure/database)
### USE_EMBEDDED_WEED_MINI
Controls embedded SeaweedFS startup.
- Default behavior: treated as enabled when unset
- Set `false` to rely on external S3-compatible storage
- Related docs: [Object / Blob Storage](../configure/object-blob-storage)
### WEED_MINI_DIR
Data directory for embedded SeaweedFS (`weed mini`).
- Default: `docstore/seaweedfs`
- Related docs: [Object / Blob Storage](../configure/object-blob-storage)
### WEED_MINI_WAIT_SEC
Maximum seconds to wait for embedded SeaweedFS startup.
- Default: `20`
- Related docs: [Object / Blob Storage](../configure/object-blob-storage)
### S3_ACCESS_KEY_ID
Access key for S3-compatible storage.
- Auto-generated in embedded mode if unset
- Set explicitly for stable credentials or external providers
- Related docs: [Object / Blob Storage](../configure/object-blob-storage)
### S3_SECRET_ACCESS_KEY
Secret key for S3-compatible storage.
- Auto-generated in embedded mode if unset
- Set explicitly for stable credentials or external providers
- Related docs: [Object / Blob Storage](../configure/object-blob-storage)
### S3_BUCKET
Bucket name used for document blobs.
- Default in embedded mode: `openreader-documents`
- Required for external S3-compatible storage
- Related docs: [Object / Blob Storage](../configure/object-blob-storage)
### S3_REGION
Region used by the S3 client.
- Default in embedded mode: `us-east-1`
- Related docs: [Object / Blob Storage](../configure/object-blob-storage)
### S3_ENDPOINT
Endpoint URL for S3-compatible storage.
- In embedded mode, defaults to `http://<BASE_URL host>:8333` (or detected host)
- For AWS S3, usually leave unset
- For MinIO/SeaweedFS/R2/B2-style APIs, typically set explicitly
- Related docs: [Object / Blob Storage](../configure/object-blob-storage)
### S3_FORCE_PATH_STYLE
Path-style S3 addressing toggle.
- Default in embedded mode: `true`
- Set according to provider requirements
- Related docs: [Object / Blob Storage](../configure/object-blob-storage)
### S3_PREFIX
Prefix prepended to stored object keys.
- Default: `openreader`
- Related docs: [Object / Blob Storage](../configure/object-blob-storage)
## Migration Controls
### RUN_DRIZZLE_MIGRATIONS
Controls startup migration execution in shared entrypoint.
- Default: `true`
- Set `false` to skip automatic startup Drizzle schema migrations
- Related docs: [Migrations](../configure/migrations), [Database](../configure/database)
### RUN_FS_MIGRATIONS
Controls startup filesystem-to-object-store migration execution in shared entrypoint.
- Default: `true`
- Runs `scripts/migrate-fs-v2.mjs` at startup after DB migrations
- Set `false` to skip automatic storage migration pass
- Related docs: [Migrations](../configure/migrations), [Database](../configure/database), [Object / Blob Storage](../configure/object-blob-storage)
## Library Import
### IMPORT_LIBRARY_DIR
Single directory root for server library import.
- Used when `IMPORT_LIBRARY_DIRS` is unset
- Default fallback root: `docstore/library`
- Related docs: [Server Library Import](../configure/server-library-import)
### IMPORT_LIBRARY_DIRS
Multiple library roots for server library import.
- Separator: comma, colon, or semicolon
- Takes precedence over `IMPORT_LIBRARY_DIR`
- Related docs: [Server Library Import](../configure/server-library-import)
## Audio Tooling and Alignment
### WHISPER_CPP_BIN
Absolute path to compiled `whisper.cpp` binary for word-level timestamps.
- Example: `/whisper.cpp/build/bin/whisper-cli`
- Required only for optional word-by-word highlighting
### FFMPEG_BIN
Absolute path or executable name for the ffmpeg binary used by audiobook/processing routes.
- Resolution order: `FFMPEG_BIN` -> `ffmpeg-static`
- Example: `/var/task/node_modules/ffmpeg-static/ffmpeg`
## Client Runtime and Feature Flags
### NEXT_PUBLIC_ENABLE_DOCX_CONVERSION
Controls whether the experimental DOCX-to-PDF conversion and upload feature is enabled.
- Default: `true` (enabled)
- Set `false` to hide DOCX support in the upload UI
### NEXT_PUBLIC_ENABLE_DESTRUCTIVE_DELETE_ACTIONS
Controls whether the "Delete all user docs" and other bulk-delete buttons are shown in Settings.
- Default: `true` (enabled)
- Set `false` to hide destructive actions (recommended for production)
### NEXT_PUBLIC_DEFAULT_TTS_PROVIDER
Sets the default TTS provider for new users.
- Default: `custom-openai`
- Example values: `deepinfra`, `openai`, `custom-openai`
### NEXT_PUBLIC_DEFAULT_TTS_MODEL
Sets the default TTS model for new users.
- Default: `kokoro`
- Example values: `hexgrad/Kokoro-82M`, `tts-1`
### NEXT_PUBLIC_ENABLE_AUDIOBOOK_EXPORT
Controls whether audiobook export UI/actions are shown in the client.
- Default behavior: enabled unless explicitly set to `false`
- Applies in both development and production
- Affects export entry points in PDF/EPUB pages and document settings UI
### NEXT_PUBLIC_ENABLE_WORD_HIGHLIGHT
Controls word-by-word highlighting UI and timestamp-alignment behavior.
- Default behavior: enabled unless explicitly set to `false`
- Applies in both development and production
- Requires working timestamp generation (for example `WHISPER_CPP_BIN`)
- Affects:
- Word-highlight toggles in document settings
- Alignment requests during TTS playback

View file

@ -0,0 +1,44 @@
---
title: Stack
---
## Framework
- [Next.js](https://nextjs.org/) 15 (App Router)
- [React](https://react.dev/) 19
- [TypeScript](https://www.typescriptlang.org/)
## Containerization and runtime
- [Docker](https://www.docker.com/) (linux/amd64 and linux/arm64)
- Shared entrypoint that runs DB migrations by default and can bootstrap embedded SeaweedFS before app startup
## Next.js client
- UI: [Tailwind CSS](https://tailwindcss.com), [Headless UI](https://headlessui.com), [@tailwindcss/typography](https://tailwindcss.com/docs/typography-plugin)
- Interactions: `react-dnd`, `react-dropzone`
- Authentication: [Better Auth](https://www.better-auth.com/) client SDK
- Local storage/cache: [Dexie.js](https://dexie.org/) (IndexedDB)
- Document rendering:
- PDF: [react-pdf](https://github.com/wojtekmaj/react-pdf), [pdf.js](https://mozilla.github.io/pdf.js/)
- EPUB: [react-reader](https://github.com/gerhardsletten/react-reader), [epubjs](https://github.com/futurepress/epub.js/)
- Markdown/Text: [react-markdown](https://github.com/remarkjs/react-markdown), [remark-gfm](https://github.com/remarkjs/remark-gfm)
- Text preprocessing/matching: [compromise](https://github.com/spencermountain/compromise), [cmpstr](https://github.com/remsky/cmpstr)
## Next.js server
- APIs: Route Handlers for sync, blob/content access, migrations, audiobook export, TTS/Whisper proxying
- State sync: request-based today (not realtime push updates)
- Authentication: [Better Auth](https://www.better-auth.com/) server handlers/adapters
- Metadata DB: [Drizzle ORM](https://orm.drizzle.team/) with SQLite (`better-sqlite3`) by default and optional Postgres (`pg`)
- App tables are manually maintained in Drizzle schema files
- Auth tables are auto-generated by the [Better Auth](https://www.better-auth.com/) CLI and migrated alongside app tables via Drizzle
- Blob storage: embedded [SeaweedFS](https://github.com/seaweedfs/seaweedfs) (`weed mini`) by default, or external S3-compatible storage via AWS SDK v3
- Audio/processing pipeline: OpenAI-compatible TTS providers, [ffmpeg](https://ffmpeg.org/) for audiobook assembly, optional [whisper.cpp](https://github.com/ggerganov/whisper.cpp) for word timestamps
## Tooling and testing
- ESLint
- TypeScript
- [Playwright](https://playwright.dev/) end-to-end tests
- Drizzle migration/generation scripts

View file

@ -0,0 +1,16 @@
---
title: Acknowledgements
---
This project is built with support from the following open-source projects and tools:
- [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M)
- [Kokoro-FastAPI](https://github.com/remsky/Kokoro-FastAPI)
- [Better Auth](https://www.better-auth.com/)
- [SQLite](https://www.sqlite.org/)
- [PostgreSQL](https://www.postgresql.org/)
- [SeaweedFS](https://github.com/seaweedfs/seaweedfs)
- [whisper.cpp](https://github.com/ggerganov/whisper.cpp)
- [ffmpeg](https://ffmpeg.org)
- [react-pdf](https://github.com/wojtekmaj/react-pdf)
- [react-reader](https://github.com/happyr/react-reader)

View file

@ -0,0 +1,7 @@
---
title: License
---
OpenReader is licensed under the MIT License.
- Repository license file: [LICENSE](https://github.com/richardr1126/openreader/blob/main/LICENSE)

View file

@ -0,0 +1,19 @@
---
title: Support and Contributing
---
## Feature requests
Use [GitHub Discussions](https://github.com/richardr1126/openreader/discussions) for feature requests and ideas.
## Issues and support
If you encounter a bug, open an issue in [GitHub Issues](https://github.com/richardr1126/openreader/issues).
## Contributing
Contributions are welcome.
- Fork the repository
- Create your branch
- Open a pull request with your changes

View file

@ -0,0 +1,46 @@
---
title: Auth
---
This page covers application-level configuration for provider access and authentication.
## Auth behavior
- Auth is enabled only when both `BASE_URL` and `AUTH_SECRET` are set.
- Remove either value to disable auth.
- Keep `AUTH_TRUSTED_ORIGINS` empty to trust only `BASE_URL`.
- Anonymous auth sessions are disabled by default.
- Set `USE_ANONYMOUS_AUTH_SESSIONS=true` to enable anonymous session flows.
## Route behavior
- `/` is a public landing/onboarding page and remains indexable.
- `/app` is the protected app home (document list and uploader UI).
- If auth is enabled and a valid session exists (including anonymous), visiting `/` redirects to `/app`.
- Protected app routes continue to require auth; when anonymous sessions are disabled and no session exists, users are redirected to `/signin`.
## Related docs
- For auth environment variables: [Environment Variables](../reference/environment-variables#auth-and-identity)
- For TTS character limits and quota behavior: [TTS Rate Limiting](./tts-rate-limiting)
- For provider-specific guidance: [TTS Providers](./tts-providers)
- For storage/S3/SeaweedFS behavior: [Object / Blob Storage](./object-blob-storage)
- For database mode: [Database](./database)
- For migration behavior and commands: [Migrations](./migrations)
## Sync notes
### Auth enabled
- Settings and reading progress are saved to the server.
- Updates are not instant push-based sync; they use normal client polling/refresh behavior.
- If two devices change the same item around the same time, the newest update wins.
### Auth disabled
- Settings and reading progress stay local in the browser (Dexie/IndexedDB).
- This avoids no-auth cross-browser conflicts, but there is no cross-device sync.
## Claim modal note
- You may still see old anonymous settings/progress available to claim from older deployments.

View file

@ -0,0 +1,39 @@
---
title: Database
---
This page covers database mode selection for OpenReader.
## Database mode
- SQLite (default): embedded DB at `docstore/sqlite3.db`; good for local/self-host single-instance setups.
- Postgres: enabled when `POSTGRES_URL` is set; recommended for production/distributed deployments.
## What the database stores
- Document and audiobook metadata/state used by server routes.
- Auth/session tables (`user`, `session`, `account`, `verification`) when auth is enabled — schema is auto-generated by Better Auth.
- TTS character usage counters (`user_tts_chars`) for daily rate limiting (when enabled).
- User settings preferences (`user_preferences`) when auth is enabled.
- User reading progress (`user_document_progress`) when auth is enabled.
- Document preview job/asset metadata (`document_previews`) for server-side PDF/EPUB thumbnails.
App-specific tables are manually maintained in Drizzle schema files, while auth tables are generated by the Better Auth CLI. Both are migrated together via Drizzle. See [Migrations](./migrations) for details.
## Related variables
- `POSTGRES_URL`
For database variable behavior, see [Environment Variables](../reference/environment-variables#database-and-object-blob-storage).
## Related docs
- [Migrations](./migrations)
- [Object / Blob Storage](./object-blob-storage)
- [Auth](./auth)
## State sync summary
- With auth enabled, settings and reading progress are stored in SQL and synced from the app.
- With auth disabled, settings and reading progress remain local in the browser.
- Sync is currently request-based (not realtime push invalidation).

View file

@ -0,0 +1,132 @@
---
title: Migrations
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
This page covers migration behavior for both database schema and storage data in OpenReader.
## Startup migration behavior
By default, the shared entrypoint runs migrations automatically before app startup in:
- Docker container startup
- `pnpm dev`
- `pnpm start`
Startup migration phases:
- DB schema migrations (`pnpm migrate`)
- Storage/data migration (`pnpm migrate-fs`) for legacy filesystem content into S3 + DB rows
:::info
In most setups, you do not need to run migration commands manually because startup handles this automatically.
:::
To skip automatic startup migrations:
- Set `RUN_DRIZZLE_MIGRATIONS=false`
- Set `RUN_FS_MIGRATIONS=false`
:::warning
If you disable startup migrations, ensure your deployment process runs migrations before serving traffic.
:::
## Apply migrations
In most cases, you do not need manual migration commands because startup runs migrations automatically.
`pnpm migrate` applies migrations for one database target:
- Postgres when `POSTGRES_URL` is set
- SQLite when `POSTGRES_URL` is unset
You can always override the target explicitly with `--config`.
<Tabs groupId="apply-migration-commands">
<TabItem value="project-scripts" label="Project Scripts" default>
```bash
# Run pending migrations for one target:
# - Postgres if POSTGRES_URL is set
# - SQLite if POSTGRES_URL is unset
pnpm migrate
# Run storage migration (filesystem -> S3 + DB)
pnpm migrate-fs
# Dry-run storage migration without uploading/deleting
pnpm migrate-fs:dry-run
```
</TabItem>
<TabItem value="drizzle-direct" label="Manual Drizzle Cmd">
```bash
# Migrate SQLite
pnpm exec drizzle-kit migrate --config drizzle.config.sqlite.ts
# Migrate Postgres
pnpm exec drizzle-kit migrate --config drizzle.config.pg.ts
```
</TabItem>
</Tabs>
## Generate migrations
`pnpm generate` is a two-phase script for contributors and schema changes:
1. **Better Auth schema generation** — runs the Better Auth CLI twice (once for SQLite, once for Postgres) to produce auto-generated Drizzle schema files for auth tables (`user`, `session`, `account`, `verification`).
2. **Drizzle migration generation** — runs `drizzle-kit generate` for both `drizzle.config.sqlite.ts` and `drizzle.config.pg.ts`, producing SQL migration files from all schema files (app + auth).
:::note
Most users do not need to run `pnpm generate`. Use it when contributing or when you have changed Drizzle schema files and need new migration files.
:::
### Schema ownership
Auth tables are owned by Better Auth. Their Drizzle schema definitions are auto-generated and should **not** be hand-edited:
- `src/db/schema_auth_sqlite.ts`
- `src/db/schema_auth_postgres.ts`
App-specific tables are manually maintained in the standard Drizzle schema files:
- `src/db/schema_sqlite.ts`
- `src/db/schema_postgres.ts`
Both sets of schema files are included in the Drizzle configs, so `drizzle-kit generate` and `drizzle-kit migrate` handle all tables together.
<Tabs groupId="generate-migration-commands">
<TabItem value="project-script" label="Project Script" default>
```bash
# Full pipeline: Better Auth CLI + Drizzle generate (both dialects)
pnpm generate
```
</TabItem>
<TabItem value="drizzle-direct" label="Manual Drizzle Cmd">
```bash
# Generate SQLite migrations only (skips Better Auth CLI)
pnpm exec drizzle-kit generate --config drizzle.config.sqlite.ts
# Generate Postgres migrations only (skips Better Auth CLI)
pnpm exec drizzle-kit generate --config drizzle.config.pg.ts
```
:::warning
Running `drizzle-kit generate` directly skips the Better Auth CLI step. If auth schema has changed upstream (e.g. after a Better Auth version bump), run `pnpm generate` instead to regenerate the auth schema files first.
:::
</TabItem>
</Tabs>
## Related docs
- [Database](./database)
- [Object / Blob Storage](./object-blob-storage)
- [Migration Environment Variables](../reference/environment-variables#migration-controls)

View file

@ -0,0 +1,103 @@
---
title: Object / Blob Storage
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
This page documents storage backends, blob upload routing, and core Docker mount behavior.
## Storage backends
- Embedded (default): SQLite metadata + embedded SeaweedFS (`weed mini`) blobs.
- External: Postgres + external S3-compatible object storage.
Storage variables are documented in [Environment Variables](../reference/environment-variables#database-and-object-blob-storage).
## Ports
- `3003`: OpenReader app and API routes
- `8333`: Embedded SeaweedFS S3 endpoint for direct browser blob access
:::info
`8333` is only needed for direct browser presigned access to embedded SeaweedFS.
:::
## Upload behavior
- Primary path: browser uploads to presigned URL from `/api/documents/blob/upload/presign`.
- Fallback path: `/api/documents/blob/upload/fallback` when direct upload fails/unreachable.
- Read/download path: blob/content serving route `/api/documents/blob` (not the upload fallback route).
- Preview path: `/api/documents/blob/preview` (returns `202` while a preview is generating; serves/redirects when ready).
## Document previews
- PDF/EPUB previews are generated server-side and stored in object storage under `document_previews_v1`.
- Preview generation is triggered on upload registration and also backfills on first preview request for older docs.
- Preview artifacts are temporary-cache friendly and can be regenerated from the source document blob.
## FS / Volume Mounts
### App data mount
- Target: `/app/docstore`
- Recommended: yes, for persistence
- Purpose: persists SeaweedFS blob data, SQLite metadata DB, migrations, and local runtime temp state
- Mount string: `-v openreader_docstore:/app/docstore`
### Library source mount (optional)
- Target: `/app/docstore/library`
- Recommended: optional, use read-only (`:ro`)
- Purpose: exposes host files as a source for server library import
- Mount string: `-v /path/to/your/library:/app/docstore/library:ro`
- Details: [Server Library Import](./server-library-import)
## Private blob endpoint mode
If `8333` is not published externally:
- Document uploads still work through upload fallback proxy
- Reads/snippets continue through app API routes
- Direct presigned browser upload/download to embedded endpoint is unavailable
:::warning
Without `8333`, expect higher app-server traffic because uploads/downloads go through API routes instead of direct object endpoint access.
:::
## Audiobook Storage Debug Commands
Audiobook assets are stored in object storage under the `audiobooks_v1` keyspace. Use these commands to inspect and download objects for debugging.
<Tabs groupId="audiobook-storage-access-cli">
<TabItem value="aws-s3" label="AWS S3" default>
```bash
# List all audiobook objects
aws s3 ls "s3://$S3_BUCKET/$S3_PREFIX/audiobooks_v1/" --recursive
# Filter to one book id (replace <book-id>)
aws s3 ls "s3://$S3_BUCKET/$S3_PREFIX/audiobooks_v1/" --recursive | grep "<book-id>-audiobook/"
# Download one object by full key
aws s3 cp "s3://$S3_BUCKET/$S3_PREFIX/audiobooks_v1/<path>/<file>.m4b" "./audiobook.m4b"
```
</TabItem>
<TabItem value="s3-compatible" label="Embedded / MinIO / R2 / etc">
```bash
# List all audiobook objects
aws s3 ls "s3://$S3_BUCKET/$S3_PREFIX/audiobooks_v1/" --recursive --endpoint-url "$S3_ENDPOINT"
# Filter to one book id (replace <book-id>)
aws s3 ls "s3://$S3_BUCKET/$S3_PREFIX/audiobooks_v1/" --recursive --endpoint-url "$S3_ENDPOINT" | grep "<book-id>-audiobook/"
# Download one object by full key
aws s3 cp "s3://$S3_BUCKET/$S3_PREFIX/audiobooks_v1/<path>/<file>.m4b" "./audiobook.m4b" --endpoint-url "$S3_ENDPOINT"
```
Embedded default example: `S3_ENDPOINT=http://127.0.0.1:8333` (or your mapped host/port).
</TabItem>
</Tabs>

View file

@ -0,0 +1,68 @@
---
title: Server Library Import
---
This page documents how server library import works and how to configure it.
## What it does
Server library import lets you browse files from one or more server directories and import selected files into OpenReader.
- Import is user-driven via a selection modal
- Only selected files are imported
- Imported files become normal OpenReader documents
## FS / Volume Mounts
### App data mount
- Target: `/app/docstore`
- Recommended: yes, for persistence
- Purpose: stores app runtime data, metadata DB, and embedded storage state
- Mount string: `-v openreader_docstore:/app/docstore`
### Library source mount
- Target: `/app/docstore/library`
- Recommended: yes for this feature, use read-only (`:ro`)
- Purpose: exposes host files as import candidates in Server Library Import
- Mount string: `-v /path/to/your/library:/app/docstore/library:ro`
## Import flow
1. Open **Settings -> Documents -> Server Library Import**.
2. Select files in the modal.
3. Click **Import**.
Selected files are fetched from the server library endpoint and imported into OpenReader storage.
:::warning Shared Library Roots
Library roots are configured at the server level (not per-user). Any user who can access Server Library Import can browse/import from the same configured roots.
Imported documents are still saved to the importing user's document scope.
:::
## Supported file types
- `.pdf`
- `.epub`
- `.html`, `.htm`
- `.txt`
- `.md`, `.mdown`, `.markdown`
## Optional: Configure Library Roots
You only need this when the default mounted path is not what you want.
By default, OpenReader uses `docstore/library` as the import root. You can override that with environment variables:
- `IMPORT_LIBRARY_DIRS` (takes precedence): multiple roots separated by comma, colon, or semicolon
- `IMPORT_LIBRARY_DIR`: single root
See [Environment Variables](../reference/environment-variables#library-import) for variable details.
## Notes
- Library listing is capped per request (up to 10,000 files).
- When auth is enabled, library import endpoints require a valid session.
- The mounted library is a source; removing it does not delete already imported documents.

View file

@ -0,0 +1,42 @@
---
title: Custom OpenAI
---
Use any custom OpenAI-compatible TTS service with OpenReader.
Use this integration when your endpoint is not directly covered by built-in dropdown defaults.
Known compatible examples include Kokoro-FastAPI, KittenTTS-FastAPI, and Orpheus-FastAPI.
## Provider
- Provider: `Custom OpenAI-Like`
- `API_BASE`: required (your service base URL)
- `API_KEY`: set if required by your service
Custom providers should expose:
- `GET /v1/audio/voices`
- `POST /v1/audio/speech`
## OpenReader setup
1. In OpenReader Settings, choose provider `Custom OpenAI-Like`.
2. Set `API_BASE` to your service base URL (typically ending with `/v1`).
3. Set `API_KEY` if your service requires authentication.
4. Choose a model and voice supported by your backend.
## Notes
:::warning Compatibility required
Custom providers must implement OpenAI-compatible TTS endpoints, including `GET /v1/audio/voices` and `POST /v1/audio/speech`.
:::
:::info Voice troubleshooting
If voices do not load, verify the `/v1/audio/voices` response shape and that the endpoint is reachable from the OpenReader server.
:::
## References
- [TTS Providers](../tts-providers)
- [TTS Environment Variables](../../reference/environment-variables#tts-provider-and-request-behavior)

View file

@ -0,0 +1,33 @@
---
title: Deepinfra
---
Use Deepinfra as a hosted OpenAI-compatible TTS provider.
## Provider
- Provider: `Deepinfra`
- Default endpoint: `https://api.deepinfra.com/v1/openai` (auto-filled)
- `API_KEY`: required for authenticated DeepInfra usage
## OpenReader setup
1. In OpenReader Settings, choose provider `Deepinfra`.
2. Keep the default `API_BASE`.
3. Set `API_KEY`.
4. Choose your model and voice.
## Notes
:::tip Built-in endpoint
`Deepinfra` is a built-in provider, so OpenReader auto-fills the default `API_BASE`.
:::
:::info Model support
DeepInfra exposes multiple TTS models, including Kokoro-family options.
:::
## References
- [TTS Providers](../tts-providers)
- [TTS Environment Variables](../../reference/environment-variables#tts-provider-and-request-behavior)

View file

@ -0,0 +1,46 @@
---
title: KittenTTS-FastAPI
---
Use [KittenTTS-FastAPI](https://github.com/richardr1126/KittenTTS-FastAPI) as an OpenAI-compatible TTS backend for OpenReader.
## Provider
- Provider: `Custom OpenAI-Like`
- Typical model: `kitten-tts`
- `API_BASE`: required (usually your KittenTTS URL ending with `/v1`)
- `API_KEY`: set only if your deployment requires one
## Run KittenTTS (CPU)
```bash
docker run -it --rm \
--name kittentts-fastapi \
-e KITTEN_MODEL_REPO_ID="KittenML/kitten-tts-nano-0.8-fp32" \
-p 8005:8005 \
ghcr.io/richardr1126/kittentts-fastapi-cpu
```
## OpenReader setup
1. Start your KittenTTS-FastAPI server.
2. In OpenReader Settings, choose provider `Custom OpenAI-Like`.
3. Set `API_BASE` to your KittenTTS base URL (typically ending with `/v1`).
4. Set `API_KEY` only if your deployment requires one.
5. Choose model `kitten-tts` (or another model exposed by your deployment).
## Notes
:::info OpenAI-compatible API
OpenReader expects OpenAI-compatible audio endpoints when using KittenTTS through `Custom OpenAI-Like`.
:::
:::tip Endpoint shape
Use an `API_BASE` that points at the KittenTTS API root (typically ending with `/v1`).
:::
## References
- [richardr1126/KittenTTS-FastAPI](https://github.com/richardr1126/KittenTTS-FastAPI)
- [TTS Providers](../tts-providers)
- [TTS Environment Variables](../../reference/environment-variables#tts-provider-and-request-behavior)

View file

@ -0,0 +1,68 @@
---
title: Kokoro-FastAPI
---
You can run the Kokoro TTS API server directly with Docker.
:::warning
For Kokoro issues and support, use the upstream repository: [remsky/Kokoro-FastAPI](https://github.com/remsky/Kokoro-FastAPI).
:::
## Provider
- Provider: `Custom OpenAI-Like`
- Typical model: `Kokoro`
- `API_BASE`: required (typically your Kokoro URL ending with `/v1`)
- `API_KEY`: set only if your deployment requires one
## Run Kokoro (CPU)
```bash
docker run --name kokoro-tts \
--restart unless-stopped \
-d \
-p 8880:8880 \
-e ONNX_NUM_THREADS=8 \
-e ONNX_INTER_OP_THREADS=4 \
-e ONNX_EXECUTION_MODE=parallel \
-e ONNX_OPTIMIZATION_LEVEL=all \
-e ONNX_MEMORY_PATTERN=true \
-e ONNX_ARENA_EXTEND_STRATEGY=kNextPowerOfTwo \
-e API_LOG_LEVEL=DEBUG \
ghcr.io/remsky/kokoro-fastapi-cpu:v0.2.4
```
## Run Kokoro (GPU)
```bash
docker run --name kokoro-tts \
--restart unless-stopped \
-d \
--gpus all \
--user 1001:1001 \
-p 8880:8880 \
-e USE_GPU=true \
-e PYTHONUNBUFFERED=1 \
-e API_LOG_LEVEL=DEBUG \
ghcr.io/remsky/kokoro-fastapi-gpu:v0.2.4
```
## OpenReader setup
1. Start Kokoro using either the CPU or GPU image.
2. In OpenReader Settings, choose provider `Custom OpenAI-Like`.
3. Set `API_BASE` to your Kokoro endpoint (for Docker Compose, commonly `http://kokoro-tts:8880/v1`).
4. Set `API_KEY` only if your deployment requires one.
5. Choose model `Kokoro`.
## Notes
:::tip Runtime guidance
GPU mode requires NVIDIA Docker support and is best on NVIDIA hardware. CPU mode is a good default on Apple Silicon and modern x86 CPUs.
:::
## References
- [remsky/Kokoro-FastAPI](https://github.com/remsky/Kokoro-FastAPI)
- [TTS Providers](../tts-providers)
- [TTS Environment Variables](../../reference/environment-variables#tts-provider-and-request-behavior)

View file

@ -0,0 +1,33 @@
---
title: OpenAI
---
Use OpenAI directly as an OpenAI-compatible TTS provider.
## Provider
- Provider: `OpenAI`
- Default endpoint: `https://api.openai.com/v1` (auto-filled)
- `API_KEY`: required for OpenAI access
## OpenReader setup
1. In OpenReader Settings, choose provider `OpenAI`.
2. Keep the default `API_BASE`.
3. Set `API_KEY`.
4. Choose your model and voice.
## Notes
:::tip Built-in endpoint
`OpenAI` is a built-in provider, so OpenReader auto-fills the default `API_BASE`.
:::
:::info Server-side requests
OpenReader sends TTS requests from the server runtime, not directly from the browser.
:::
## References
- [TTS Providers](../tts-providers)
- [TTS Environment Variables](../../reference/environment-variables#tts-provider-and-request-behavior)

View file

@ -0,0 +1,36 @@
---
title: Orpheus-FastAPI
---
Use Orpheus-FastAPI as an OpenAI-compatible TTS backend for OpenReader.
## Provider
- Provider: `Custom OpenAI-Like`
- Typical model: `Orpheus`
- `API_BASE`: required (usually your Orpheus URL ending with `/v1`)
- `API_KEY`: set only if your deployment requires one
## OpenReader setup
1. Start your Orpheus-FastAPI server.
2. In OpenReader Settings, choose provider `Custom OpenAI-Like`.
3. Set `API_BASE` to your Orpheus base URL (typically ending with `/v1`).
4. Set `API_KEY` only if your Orpheus deployment requires one.
5. Choose model `Orpheus` (or another model exposed by your deployment).
## Notes
:::info OpenAI-compatible API
OpenReader expects OpenAI-compatible audio endpoints when using Orpheus through `Custom OpenAI-Like`.
:::
:::tip Endpoint shape
Use an `API_BASE` that points at the Orpheus API root (typically ending with `/v1`).
:::
## References
- [Lex-au/Orpheus-FastAPI](https://github.com/Lex-au/Orpheus-FastAPI)
- [TTS Providers](../tts-providers)
- [TTS Environment Variables](../../reference/environment-variables#tts-provider-and-request-behavior)

View file

@ -0,0 +1,80 @@
---
title: TTS Providers
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
OpenReader supports OpenAI-compatible TTS providers through a common API shape.
:::tip
If you are running a self-hosted TTS server (Kokoro/KittenTTS/Orpheus/etc.), use **Custom OpenAI-Like** in Settings.
:::
## Quick Setup by Provider
<Tabs groupId="tts-provider-setup">
<TabItem value="openai" label="OpenAI" default>
1. In Settings, choose provider: `OpenAI`.
2. Keep the default `API_BASE` (auto-filled).
3. Set `API_KEY` to your OpenAI key.
4. Choose model/voice.
</TabItem>
<TabItem value="deepinfra" label="DeepInfra">
1. In Settings, choose provider: `Deepinfra`.
2. Keep the default `API_BASE` (auto-filled).
3. Set `API_KEY` to your DeepInfra key.
4. Choose model/voice.
</TabItem>
<TabItem value="custom" label="Custom OpenAI-Like">
1. In Settings, choose provider: `Custom OpenAI-Like`.
2. Set `API_BASE` to your endpoint (example: `http://host.docker.internal:8880/v1`).
3. Set `API_KEY` if your provider requires one.
4. Choose model/voice.
</TabItem>
</Tabs>
## Provider Dropdown Behavior
In Settings, provider options include:
- `OpenAI`
- `Deepinfra`
- `Custom OpenAI-Like` (Kokoro, KittenTTS-FastAPI, Orpheus, and other OpenAI-compatible endpoints)
`API_BASE` guidance:
- `OpenAI` and `Deepinfra` auto-fill default endpoints.
- `Custom OpenAI-Like` requires setting `API_BASE` manually.
:::info OpenAI-Compatible API Shape
Custom providers should expose:
- `GET /v1/audio/voices`
- `POST /v1/audio/speech`
:::
:::warning Server-Reachable API Base
TTS requests are sent from the Next.js server, not directly from the browser. `API_BASE` must be reachable from the server runtime.
:::
## Provider Guides
- [Kokoro-FastAPI](./tts-provider-guides/kokoro-fastapi)
- [KittenTTS-FastAPI](./tts-provider-guides/kitten-tts-fastapi)
- [Orpheus-FastAPI](./tts-provider-guides/orpheus-fastapi)
- [DeepInfra](./tts-provider-guides/deepinfra)
- [OpenAI](./tts-provider-guides/openai)
- [Custom OpenAI-Like](./tts-provider-guides/custom-openai)
## Related Configuration
- [TTS Environment Variables](../reference/environment-variables#tts-provider-and-request-behavior)
- [TTS Rate Limiting](./tts-rate-limiting)
- [Auth](./auth)

View file

@ -0,0 +1,51 @@
---
title: TTS Rate Limiting
---
This page explains OpenReader's TTS character rate limiting controls.
## Overview
- TTS rate limiting is disabled by default.
- To enable it, set `TTS_ENABLE_RATE_LIMIT=true`.
- Limits are enforced per day in UTC.
- Enforcement applies only when auth is enabled.
## How enforcement works
When enabled, OpenReader enforces:
- Per-user daily character limits.
- IP backstop daily character limits.
- Anonymous device backstop tracking (cookie-based) to reduce limit resets.
If a request exceeds the active limit, the TTS API returns `429` with reset metadata for the next UTC day.
## Required auth behavior
- Auth must be enabled (`BASE_URL` + `AUTH_SECRET`) for TTS char limits to apply.
- If auth is disabled, TTS character limits are effectively unlimited.
- `DISABLE_AUTH_RATE_LIMIT` only affects Better Auth's own request throttling.
- `DISABLE_AUTH_RATE_LIMIT` does not disable TTS character limits.
## Environment variables
Enable/disable:
- `TTS_ENABLE_RATE_LIMIT` (default: `false`)
Per-user daily limits:
- `TTS_DAILY_LIMIT_ANONYMOUS` (default: `50000`)
- `TTS_DAILY_LIMIT_AUTHENTICATED` (default: `500000`)
IP backstop daily limits:
- `TTS_IP_DAILY_LIMIT_ANONYMOUS` (default: `100000`)
- `TTS_IP_DAILY_LIMIT_AUTHENTICATED` (default: `1000000`)
## Related docs
- TTS/rate-limit environment variables: [Environment Variables](../reference/environment-variables#tts-provider-and-request-behavior)
- Auth configuration: [Auth](./auth)
- Provider setup: [TTS Providers](./tts-providers)

View file

@ -0,0 +1,138 @@
---
title: Local Development
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
## Prerequisites
- Node.js (recommended with [nvm](https://github.com/nvm-sh/nvm))
- `pnpm` (recommended) or `npm`
```bash
npm install -g pnpm
```
- A reachable TTS API server
- [SeaweedFS](https://github.com/seaweedfs/seaweedfs) `weed` binary (required unless using external S3 storage)
<Tabs groupId="seaweedfs-install">
<TabItem value="macos" label="macOS" default>
```bash
brew install seaweedfs
```
</TabItem>
<TabItem value="linux" label="Linux">
Install the `weed` binary from the [SeaweedFS releases](https://github.com/seaweedfs/seaweedfs/releases) and ensure it is available on `PATH`.
</TabItem>
</Tabs>
Optional, depending on features:
- [libreoffice](https://www.libreoffice.org) (required for DOCX conversion)
```bash
brew install libreoffice
```
- [whisper.cpp](https://github.com/ggml-org/whisper.cpp) (optional, for word-by-word highlighting)
```bash
# clone and build whisper.cpp (no model download needed OpenReader handles that)
git clone https://github.com/ggml-org/whisper.cpp.git
cd whisper.cpp
cmake -B build
cmake --build build -j --config Release
# point OpenReader to the compiled whisper-cli binary
echo WHISPER_CPP_BIN="$(pwd)/build/bin/whisper-cli"
```
:::tip
Set `WHISPER_CPP_BIN` in your `.env` to enable word-by-word highlighting.
:::
## Steps
1. Clone the repository.
```bash
git clone https://github.com/richardr1126/openreader.git
cd openreader
```
2. Install dependencies.
```bash
pnpm i
```
3. Configure the environment.
```bash
cp .env.example .env
```
Then edit `.env`.
- No auth mode: leave `BASE_URL` or `AUTH_SECRET` unset.
- Auth enabled mode: set both `BASE_URL` (typically `http://localhost:3003`) and `AUTH_SECRET` (generate with `openssl rand -hex 32`).
Optional:
- `AUTH_TRUSTED_ORIGINS=http://localhost:3003,http://192.168.0.116:3003`
- Stable S3 credentials via `S3_ACCESS_KEY_ID` and `S3_SECRET_ACCESS_KEY`
- External S3 storage by setting `USE_EMBEDDED_WEED_MINI=false` and related S3 vars
:::info
For all environment variables, see [Environment Variables](../reference/environment-variables).
:::
See [Auth](../configure/auth) for app/auth behavior.
Storage configuration details are in [Object / Blob Storage](../configure/object-blob-storage).
Refer to [Database](../configure/database) for database modes.
Learn about migration behavior and commands in [Migrations](../configure/migrations).
4. Run DB migrations.
- Migrations run automatically on startup through the shared entrypoint for both `pnpm dev` and `pnpm start`.
- You only need manual migration commands for one-off troubleshooting or explicit migration workflows:
```bash
pnpm migrate
```
:::info
If `POSTGRES_URL` is set, migrations target Postgres; otherwise local SQLite is used. To disable automatic startup migrations, set `RUN_DRIZZLE_MIGRATIONS=false` and/or `RUN_FS_MIGRATIONS=false`. You can run storage migration manually with `pnpm migrate-fs`.
:::
5. Start the app.
<Tabs groupId="local-run-mode">
<TabItem value="dev" label="Dev" default>
```bash
pnpm dev
```
</TabItem>
<TabItem value="prod" label="Build + Start">
```bash
pnpm build
pnpm start
```
</TabItem>
</Tabs>
:::warning API Base Reachability
`API_BASE` must be reachable from the Next.js server process, not just your browser.
:::
Visit [http://localhost:3003](http://localhost:3003).

View file

@ -0,0 +1,106 @@
---
title: Vercel Deployment
---
This guide covers deploying OpenReader to Vercel with external Postgres and S3-compatible object storage.
## What works on Vercel
- Documents (PDF/EPUB/TXT/MD) work with `POSTGRES_URL` + external S3 storage.
- Audiobook routes work on Node.js serverless functions using `ffmpeg-static`.
:::warning DOCX Conversion Limitation
`docx` conversion requires `soffice` (LibreOffice), which is not available in a standard Vercel runtime.
:::
## 1. Environment Variables
Recommended production setup (auth enabled):
```bash
API_BASE=https://api.deepinfra.com/v1/openai
API_KEY=your_deepinfra_key
POSTGRES_URL=postgres://...
USE_EMBEDDED_WEED_MINI=false
S3_ACCESS_KEY_ID=...
S3_SECRET_ACCESS_KEY=...
S3_BUCKET=...
S3_REGION=us-east-1
S3_PREFIX=openreader
BASE_URL=https://your-app.vercel.app
AUTH_SECRET=...
# Optional client/runtime feature defaults:
NEXT_PUBLIC_ENABLE_DOCX_CONVERSION=false
NEXT_PUBLIC_ENABLE_DESTRUCTIVE_DELETE_ACTIONS=false
NEXT_PUBLIC_DEFAULT_TTS_PROVIDER=deepinfra
NEXT_PUBLIC_DEFAULT_TTS_MODEL=hexgrad/Kokoro-82M
NEXT_PUBLIC_ENABLE_AUDIOBOOK_EXPORT=true
NEXT_PUBLIC_ENABLE_WORD_HIGHLIGHT=false
# Optional (non-AWS S3-compatible providers):
# S3_ENDPOINT=https://...
# S3_FORCE_PATH_STYLE=true
```
:::info Production Configuration & Feature Flags
We recommend setting these defaults for a production-like environment:
- `NEXT_PUBLIC_ENABLE_DOCX_CONVERSION=false`: Disables DOCX upload (requires external tools anyway)
- `NEXT_PUBLIC_ENABLE_DESTRUCTIVE_DELETE_ACTIONS=false`: Hides destructive "Delete All" actions
- `NEXT_PUBLIC_DEFAULT_TTS_PROVIDER=deepinfra`: Points default TTS to a scalable provider
- `NEXT_PUBLIC_DEFAULT_TTS_MODEL=hexgrad/Kokoro-82M`: Uses a high-quality default model
- `NEXT_PUBLIC_ENABLE_AUDIOBOOK_EXPORT=true`: (Optional) Controls audiobook export UI
- `NEXT_PUBLIC_ENABLE_WORD_HIGHLIGHT=false`: (Optional) Controls word highlighting UI (requires timestamp backend)
:::
:::warning Auth recommendation
For internet-exposed Vercel deployments, set both `BASE_URL` and `AUTH_SECRET`. Running without auth is possible, but not recommended for public environments.
:::
:::tip
For all variables and defaults, see [Environment Variables](../reference/environment-variables).
:::
## 2. FFmpeg packaging in Vercel functions
`ffmpeg-static` binaries must be included in function traces. This repo already does that in `next.config.ts` via `outputFileTracingIncludes` for:
- `/api/audiobook`
- `/api/audiobook/chapter`
- `/api/audiobook/status`
- `/api/whisper`
:::info
`serverExternalPackages` should include `ffmpeg-static` so package paths resolve at runtime instead of being bundled into route output.
:::
If you change route paths or split handlers, update `outputFileTracingIncludes` accordingly.
## 3. Function memory sizing
FFmpeg workloads benefit from more memory/CPU. This repo includes:
```json
{
"$schema": "https://openapi.vercel.sh/vercel.json",
"functions": {
"app/api/audiobook/route.ts": { "memory": 3009 },
"app/api/whisper/route.ts": { "memory": 3009 }
}
}
```
Adjust memory per route if your files are larger or your plan differs.
## 4. Runtime expectations and caveats
- Audiobook APIs require S3 configuration; otherwise they return `503`.
- For production Vercel deploys, use `POSTGRES_URL` instead of SQLite.
- Filesystem-to-object-store migrations run via server scripts/entrypoint (`scripts/migrate-fs-v2.mjs`), not API routes.
- Vercel deployments do not run `scripts/openreader-entrypoint.mjs`, so run `pnpm migrate-fs` in a controlled environment when migrating legacy filesystem data.
## 5. Smoke test after deploy
1. Upload and read a PDF/EPUB document.
2. Confirm sync/blob fetch works across refreshes/devices.
3. Generate at least one audiobook chapter and play/download it.
4. If using word highlighting, verify timestamps are produced and rendered.

Some files were not shown because too many files have changed in this diff Show more