Introduce idle and hard cap timeouts for PDF layout parsing in the compute worker,
ensuring long-running jobs are capped and progress is tracked with explicit
timeouts. Refactor NATS JetStream client initialization to use a configurable
API timeout. Update PDF viewer UI to treat parse state as 'pending' by default,
improving clarity during document preparation and parsing. Refine parse loader
labels and progress logic for more accurate user feedback.
Update compute worker to use separate environment variable for PDF job attempts
(`COMPUTE_PDF_JOB_ATTEMPTS`) and set Whisper job max deliveries to 1, clarifying
and separating retry logic between job types. Adjust `.env.example` and deployment
docs to comment out defaults and document new/renamed variables for advanced tuning.
In TTS segment ensure route, introduce explicit timeouts for Whisper alignment
operations, with configurable durations for local and worker modes, improving
robustness and error handling for long-running alignments.
Revise PDF viewer to introduce an explicit 'unknown' parse state and improve
loader and status messaging for better user clarity. Refactor Spinner to use
'currentColor' for border styling and simplify conditional rendering based on
className. Expand compute worker .env.example with explicit concurrency,
timeout, and job attempt variables for enhanced configuration transparency.
Enhance PDF parse loader transitions by introducing a delayed expansion for detailed progress, reducing UI flicker during parse-to-render handoff. Refactor PDFViewer to ensure document readiness is signaled only once per load, preventing redundant callbacks and improving synchronization between parsing and rendering states.
Introduce detailed progress reporting for PDF layout parsing, exposing phase and page-level updates to clients. Replace legacy parseStatus with a structured parseState field in the database schema, updating all relevant backend and API logic. Add SSE endpoint for real-time parse progress updates. Update client hooks and UI to reflect granular progress and improve user feedback during document parsing. Includes migration scripts and new parse-state utility module.
Migrate worker and backend contract to an operation-centric model, introducing opKey-based deduplication for Whisper alignment and PDF layout parsing. Add new operation request/state types and update worker endpoints to accept generic operation requests and expose operation status via SSE. Refactor client and server logic to support operation flows. Update deployment docs, environment examples, and Dockerfile to reflect new architecture and dependency management.
Add ability to fetch parsed documents using a stored S3 key, enabling more flexible blob access patterns. Update API route to prefer `parsedJsonKey` if present, falling back to legacy lookup. Introduce `COMPUTE_WORKER_WAIT_TIMEOUT_MS` environment variable for customizing worker job wait time, with a new default of 120 seconds. Update documentation and examples to reflect these changes.
Align compute worker to use PORT environment variable consistently across
Dockerfile, docker-compose, and server code, replacing COMPUTE_WORKER_PORT.
Update .env.example and deployment documentation to clarify platform-specific
behavior, especially for Railway and similar environments. This simplifies
configuration and reduces platform-specific issues.
Update worker server to prioritize platform-specific PORT environment variable
when setting the compute worker port. Add robust normalization for
COMPUTE_WORKER_URL, ensuring proper scheme and validation, and trim trailing
slashes for consistency. This enhances reliability in diverse deployment
environments.
Update backend creation logic to consistently determine compute mode using
a local variable, ensuring correct handling when the mode is undefined. This
improves clarity and robustness in backend selection.
Remove 'ffmpeg-static' from server external packages in Next.js config and
specify Node.js 22.x as required engine in package.json to align with
deployment environment.
Update Next.js configuration to conditionally set 'output' only when not on
Vercel, and always define 'outputFileTracingExcludes' with dynamic includes
based on compute mode. This ensures compatibility with Vercel deployments and
improves local/server build behavior.
Update TTS context to include configProviderType, supportsInstructions, and
ttsInstructions in the provided value, improving downstream access to provider
capabilities and instruction handling.
Update compute backend interfaces and job logic to consistently handle parsed
PDF layout results as either direct data or S3 object key references. Refactor
result types, local and worker backends, and job processing to support this
pattern. Improves flexibility for large document parsing and object storage
integration.
Add `COMPUTE_JOBS_STREAM_MAX_BYTES` and `COMPUTE_JOB_STATES_MAX_BYTES` to
environment example, worker server configuration, and deployment docs. These
variables allow tuning JetStream resource caps for job queue and state storage
to better fit deployment requirements.
Restructure compute core by extracting job contracts and runtime logic into separate
modules (`contracts.ts`, `local-runtime.ts`) for improved modularity and clearer
API boundaries. Update exports and imports across worker, server, and local
compute backends to use the new modules. Enhance Next.js config to support
dynamic backend selection via build-time constants and aliasing, enabling
seamless switching between local and worker compute modes. Adjust TypeScript
paths and Dockerfile entrypoint to align with the new structure.
Add a comprehensive Synadia Cloud and Railway setup guide to the compute worker
deployment docs, including step-by-step environment variable configuration and
example image tags. Update Vercel deployment instructions with a quick start for
external compute worker integration via Railway and Synadia NGS. Document new
`.env.prod` pattern in .gitignore for production environment management.
Remove legacy .npmrc config. Revise root and worker Dockerfiles to enhance
layer caching, explicitly copy only required build artifacts, and streamline
runtime dependencies. Switch to pnpm workspace-aware install for compute worker
to ensure correct dependency resolution. Set Next.js output to standalone mode
for optimized server builds. Update docker-compose sync targets for consistency.
Expand compute worker documentation to explain NATS authentication using
either a credentials file or raw credentials string, including Synadia Cloud
integration steps. Update environment variable examples and server connection
logic to support both `NATS_CREDS_FILE` and `NATS_CREDS`. Add `.creds` to
.gitignore to avoid accidental credential leaks.
This improves deployment flexibility for cloud and on-prem environments by
supporting multiple authentication methods for NATS JetStream.
Replace Redis and BullMQ with NATS JetStream and KV for job queuing and state management
in the compute worker service. Update environment variables, Docker Compose, and documentation
to reflect the new message queue backend. Adjust dependencies and code to use NATS-based
work queue and key-value storage for durable job processing.
This change improves scalability and reliability of distributed compute workloads by leveraging
NATS JetStream's work queue and persistence features. Documentation and configuration examples
are updated to guide deployments using the new backend.
Introduce clearSegmentCaches to TTS context for explicit cache invalidation after segment clearing. Enhance SegmentsSidebar scroll logic to better distinguish user-initiated and programmatic scrolls, improving UX during segment updates.
Eliminate the smartSentenceSplitting configuration flag from application state, UI, and all TTS and audiobook adapter logic. Consolidate TTS segment planning to always include context units, streamlining code and reducing conditional branches. Update tests and type definitions to reflect the removal of this feature toggle.
Update TTS segment prefetching logic to handle arrays of CanonicalTtsSourceUnit for upcoming and next locations, enabling more precise TTS segment mapping. Expand cache key construction to include providerType and instructions, improving cache differentiation for TTS requests. Adjust related types and usages to support richer segment metadata propagation.
Introduce sourceUnitsFromParsedPage helper to extract CanonicalTtsSourceUnit
objects from parsed PDF pages, filtering by block kind and text content. Update
TTS state setter to include extracted source units when available, enabling
improved TTS segment mapping and future extensibility.
Move download logic into reusable downloadViaTrigger function. Add retries and
button enabled checks to downloadFullAudiobook for improved robustness against
transient UI or backend delays.
Replace text-based assertion for PDF title with role-based heading check and
add assertion for visible page navigation button. Improves test robustness
against UI changes and ensures accessibility compliance.
Unify and improve layout settling logic across EPUB, HTML, and PDF viewers by
adding delayed recomputation and guarding against zero-height container during
transient states. Refactor PDF document loading to include retry logic with
success detection, ensuring robust handling of transient failures. Update
Playwright test helpers to provide more reliable PDF viewer readiness checks
and introduce request retry utilities for backend API calls. Add targeted unit
tests for single-section chapter fallback in the PDF audiobook adapter. Update
test files and helpers to align with new layout and export logic.
Refactor compute backend initialization to use asynchronous loading and promise caching, replacing previous synchronous singleton pattern. Update all consumers—including TTS segment alignment and PDF layout parsing jobs—to await compute backend resolution before invoking methods. This change improves compatibility with dynamic imports and future-proofs backend selection logic.
Eliminate unused PDF text extraction logic and margin-based filtering from the client library. Refactor API routes to generalize parse status normalization, removing explicit references to deprecated "unsupported" status and type variants. Update usages to accept string parse statuses for improved flexibility. Simplify document state dependency tracking in the PDF document hook.
Eliminate the "none" compute mode and all related code paths, including the NoneComputeBackend, "unsupported" parse status, and PDF margin settings. Parsing is now always available if the app starts successfully, and configuration is limited to "local" or "worker" compute modes. Update types, API routes, client adapters, and documentation to reflect this simplification.
BREAKING CHANGE: "none" is no longer a valid COMPUTE_MODE; only "local" and "worker" are supported. "unsupported" parse status and PDF margin settings are removed.
Introduce WorkerJobTiming interface to capture queue wait, S3 fetch, and compute
durations for compute worker jobs. Update job result and status response types
to include timing data. Enhance worker implementation to record and attach
timing metrics for alignment jobs, improving observability of job processing
latency. Update exports and type usage to support new timing fields.
Revise project highlights in README and introduction to emphasize ONNX-based
PDF layout parsing (PP-DocLayoutV3), segment-based TTS playback, and flexible
backend/storage options. Clarify compute modes and self-hosting features.
Update TTS provider descriptions for accuracy and conciseness.
Also expand tsconfig exclude list to omit compute worker sources from main build.
Introduce support for external compute worker mode (`COMPUTE_MODE=worker`) using a new `WorkerComputeBackend`. This enables offloading heavy ONNX Whisper alignment and PDF layout parsing to a standalone worker service (Redis + BullMQ), improving scalability and compatibility with serverless/limited environments.
- Add `@openreader/compute-core` as a shared package for ONNX inference and PDF parsing logic.
- Implement `WorkerComputeBackend` and worker contract/types for remote job execution.
- Update compute backend selection logic and remove previous worker mode guards.
- Extend `WhisperAlignInput` and `PdfLayoutInput` types to support object keys for remote data access.
- Refactor local compute backend to use `@openreader/compute-core` and support both buffer and object key inputs.
- Update job runner, TTS segment alignment, and PDF layout parsing flows to use new compute backend APIs.
- Add scripts, Docker workflow, and documentation for deploying and running the compute worker.
- Update environment variable docs and examples for worker mode, including storage requirements and configuration.
- Document published images and stack changes to reflect the new compute worker architecture.
BREAKING CHANGE: `COMPUTE_MODE=worker` now requires an external compute worker service and S3-compatible object storage. Embedded SeaweedFS (`weed mini`) is not supported in worker mode. See the new documentation for deployment and configuration details.
- Replace `OPENREADER_COMPUTE_MODE` and related variables with `COMPUTE_MODE`
- Replace `OPENREADER_*` PDF/Whisper model URLs with `PDF_LAYOUT_MODEL_BASE_URL` and `WHISPER_MODEL_BASE_URL`
- Remove legacy `NEXT_PUBLIC_*` runtime config seeds in favor of `RUNTIME_SEED_*`
- Update documentation, code, and environment references to match new variable names
- Remove deprecated `scripts/fetch-models.mjs` and related npm script
- Update runtime config SSR injection from `window.__OPENREADER_RUNTIME_CONFIG__` to `window.__RUNTIME_CONFIG__`
- Adjust Next.js config to use new compute mode env var and optimize output file tracing for ONNX dependencies
BREAKING CHANGE: Environment variable names for compute mode, model URLs, and runtime config seeding have changed. Update `.env` files and deployment configs to use `COMPUTE_MODE`, `PDF_LAYOUT_MODEL_BASE_URL`, `WHISPER_MODEL_BASE_URL`, and `RUNTIME_SEED_*` as appropriate. Legacy `OPENREADER_*` and `NEXT_PUBLIC_*` variables are no longer supported.
Replace the previous whisper.cpp-based word alignment with a fully ONNX-based
implementation using onnxruntime-node and @huggingface/tokenizers. Add new
Whisper ONNX model management, alignment mapping, and spectral analysis modules.
Remove all code and documentation referencing whisper.cpp, update environment
variables, Dockerfile, and docs to reflect ONNX-only alignment. Add unit tests
for alignment and ONNX model logic.
Enhance PDF cross-page block stitching to move only sentence continuations and
preserve remaining text on the following page. Update TTS segment planning to
support strict source boundary enforcement, ensuring segments do not cross
block or paragraph-title boundaries. Add new tests for both features and
refactor PDF text item normalization to filter out skewed or rotated runs.
Remove temporary working directory usage and in-place file writes from preview
generation, simplifying logic for both PDF and EPUB previews. Change default
preview image variant from 480px to 400px width, updating related constants and
cache schema version to maintain consistency across client and server code.
Implement block-level highlighting in PDF viewer by leveraging parsed document
structure and TTS segment locator. Extend highlightPattern API to accept options
for parsedDocument, locator, and block geometry mode. Update PDFViewer to use
block geometry highlighting when available, improving accuracy and alignment of
sentence and word highlights with ONNX-based DocLayoutV3 parsing. Refactor
highlighting logic to support both legacy span-based and new geometry-based
approaches.
Remove hardcoded defaults for preview variant and width in database schemas to allow more flexible preview generation. Bump preview cache schema version and add versioning to cache rows for consistency. Switch preview image generation to 480px JPEGs with updated file naming and content type. Refactor PDF preview rendering to support configurable output format and quality, and update all preview consumers to use unified "image" buffer property instead of "png". Add migration scripts and update metadata for both Postgres and SQLite.
BREAKING CHANGE: Preview variant and width defaults removed from database; preview cache and image handling updated across system.
Switch PDF layout parsing to use the PP-DocLayoutV3 ONNX model, replacing the previous Docling-based approach. Update all environment variables, model fetching, and manifest handling for the new model and its artifacts. Refactor block kind taxonomy throughout the codebase, tests, and UI to align with PP-DocLayoutV3 labels, including expanded and renamed block types. Revise document settings, block filtering, and stitching logic to support the new set of block kinds. Update documentation and environment variable references to reflect the model transition.
BREAKING CHANGE: PDF parsing now requires PP-DocLayoutV3 ONNX model and updated environment variables; block kind names and settings have changed throughout the system.
Migrate PDF parsing to use ONNX Docling layout model for structured block extraction, enabling improved TTS segmentation and chaptering. Add compute backend abstraction for heavy tasks (alignment, layout parsing) with configuration via `OPENREADER_COMPUTE_MODE`. Integrate block-level locators and document settings for per-document PDF parsing options. Update S3 storage for parsed PDF JSON, add migration and schema changes, and extend client and server APIs for parsed data and settings. Remove legacy word highlight flag in favor of compute capability detection.
- Add ONNX model download, local/none compute modes, and `onnxruntime-node` dependency
- Update PDF viewer and TTS pipeline to use parsed blocks and block-level locators
- Add document settings storage and APIs for per-document PDF options
- Update S3 storage layout for parsed PDF JSON
- Add admin/config/docs updates for new compute and parsing features
- Remove obsolete word highlight runtime flag and UI
BREAKING CHANGE: PDF parsing and word highlighting now require `OPENREADER_COMPUTE_MODE=local` and ONNX model; document settings and S3 layout updated; legacy word highlight flag removed.
Introduce sentence and word highlighting for HTML/TXT/MD documents, including new config options (`htmlHighlightEnabled`, `htmlWordHighlightEnabled`) and synced user preferences. Update the HTML viewer to support these highlights using custom spans, and add corresponding CSS styles. Extend document settings to allow toggling these features. Implement block-level locator sorting and display improvements. Add audiobook export support for HTML documents, including adapter and pipeline integration.
- Add `htmlHighlightEnabled` and `htmlWordHighlightEnabled` to config, user state, and Dexie storage
- Update HTML viewer to apply and manage highlights for sentences and words
- Add CSS for HTML highlight classes and scroll margin
- Extend document settings UI for HTML highlight toggles
- Support HTML audiobook export in modal and pipeline
- Improve block locator sorting and sidebar display for HTML
- Add HTML audiobook adapter and block parsing utilities
Extract release-based deploy triggers to new workflow for clarity and
reliability. Update docs-deploy.yml to streamline dispatch inputs and
remove release/workflow_run triggers. Extend build-changelog-feed.mjs to
support incremental changelog sync via workflow_dispatch with release tag
and action, enabling precise upserts and deletions of changelog entries.
Revert packageManager to pnpm@10.33.4 in package.json for compatibility.
Expand allowBuilds in pnpm-workspace.yaml to explicitly exclude canvas,
core-js, es5-ext, esbuild, sharp, and unrs-resolver from workspace builds.
These adjustments enhance build reliability and prevent issues with
problematic dependencies.
Update GitHub Actions workflows to latest action versions and improve pnpm usage by introducing per-package workspace configs. Add docs-site/pnpm-workspace.yaml and update Dockerfile/package.json for better monorepo support. Upgrade dependencies across main and docs-site packages for compatibility and security. Refine TypeScript types and hooks for improved type safety and maintainability.
