Introduce per-page progress events for PDF parsing by adding onPageStarted
hooks throughout the compute worker and core PDF pipeline. Add
compute/worker/src/pdf-progress.ts to encapsulate progress event builders.
Update server logic to emit page start and page parsed events, improving
client feedback during long-running PDF jobs.
Add src/lib/server/compute/abort-like-error.ts and integrate isAbortLikeError
to robustly detect and suppress expected aborts in API event streaming.
Update API route to ignore abort-like errors on worker proxy crash.
Add unit tests for abort-like error detection and PDF progress event
generation. Refactor PDF parse logic to allow empty merged regions without
throwing, ensuring downstream flows remain resilient to edge cases.
These changes improve observability, error handling, and reliability for
document parsing and event streaming.
Extract onboarding flow step resolution and async runner logic into a
dedicated module for better separation of concerns and maintainability.
Refactor context/provider to use new helpers and simplify settings modal
control. Add unit tests for onboarding step resolution and async runner
coalescing. Update e2e helpers and selectors for onboarding modals.
Adjust environment file handling for CI and Docker contexts.
- Add .dockerignore for build hygiene
- Refactor onboarding context and modal logic
- Add src/lib/client/onboarding-flow.ts with core onboarding helpers
- Add onboarding-flow unit tests
- Update Playwright config and scripts for CI env loading
- Improve test selectors for claim modal
- Update .gitignore to exclude all .env* except .env.example
This change increases onboarding logic modularity and reliability, and improves CI/test environment handling.
Add useOnboardingCoordinator hook and onboarding-state registry to manage
onboarding-related state such as privacy acceptance and first-visit tracking.
Refactor SettingsModal to utilize the new onboarding state abstractions,
removing legacy privacy gating logic and improving maintainability.
Includes unit tests for onboarding-state registry to ensure correctness.
- Replace static JSON imports with runtime manifest loading in pdf and whisper model modules for compatibility with ESM and bundlers
- Refactor Next.js config to clarify compute mode logic and improve worker bundling conditions
- Update test imports to use package entrypoints instead of relative paths
- Remove redundant whisper alignment/model tests now covered elsewhere or by integration
- No breaking changes to public API or model handling logic
- Deleted mergeTextWithRegions.ts, LICENSE.txt, manifest.json, parsePdf.ts, renderPage.ts, runLayoutModel.ts, stitchCrossPageBlocks.ts, and types.ts as they are no longer needed.
- Updated unit tests to reflect new import paths from @openreader/compute-core.
- Adjusted tsconfig.json to include new paths for pdf-layout module.
Introduce detailed progress reporting for PDF layout parsing, exposing phase and page-level updates to clients. Replace legacy parseStatus with a structured parseState field in the database schema, updating all relevant backend and API logic. Add SSE endpoint for real-time parse progress updates. Update client hooks and UI to reflect granular progress and improve user feedback during document parsing. Includes migration scripts and new parse-state utility module.
Eliminate the smartSentenceSplitting configuration flag from application state, UI, and all TTS and audiobook adapter logic. Consolidate TTS segment planning to always include context units, streamlining code and reducing conditional branches. Update tests and type definitions to reflect the removal of this feature toggle.
Update TTS segment prefetching logic to handle arrays of CanonicalTtsSourceUnit for upcoming and next locations, enabling more precise TTS segment mapping. Expand cache key construction to include providerType and instructions, improving cache differentiation for TTS requests. Adjust related types and usages to support richer segment metadata propagation.
Unify and improve layout settling logic across EPUB, HTML, and PDF viewers by
adding delayed recomputation and guarding against zero-height container during
transient states. Refactor PDF document loading to include retry logic with
success detection, ensuring robust handling of transient failures. Update
Playwright test helpers to provide more reliable PDF viewer readiness checks
and introduce request retry utilities for backend API calls. Add targeted unit
tests for single-section chapter fallback in the PDF audiobook adapter. Update
test files and helpers to align with new layout and export logic.
Eliminate the "none" compute mode and all related code paths, including the NoneComputeBackend, "unsupported" parse status, and PDF margin settings. Parsing is now always available if the app starts successfully, and configuration is limited to "local" or "worker" compute modes. Update types, API routes, client adapters, and documentation to reflect this simplification.
BREAKING CHANGE: "none" is no longer a valid COMPUTE_MODE; only "local" and "worker" are supported. "unsupported" parse status and PDF margin settings are removed.
Replace the previous whisper.cpp-based word alignment with a fully ONNX-based
implementation using onnxruntime-node and @huggingface/tokenizers. Add new
Whisper ONNX model management, alignment mapping, and spectral analysis modules.
Remove all code and documentation referencing whisper.cpp, update environment
variables, Dockerfile, and docs to reflect ONNX-only alignment. Add unit tests
for alignment and ONNX model logic.
Enhance PDF cross-page block stitching to move only sentence continuations and
preserve remaining text on the following page. Update TTS segment planning to
support strict source boundary enforcement, ensuring segments do not cross
block or paragraph-title boundaries. Add new tests for both features and
refactor PDF text item normalization to filter out skewed or rotated runs.
Switch PDF layout parsing to use the PP-DocLayoutV3 ONNX model, replacing the previous Docling-based approach. Update all environment variables, model fetching, and manifest handling for the new model and its artifacts. Refactor block kind taxonomy throughout the codebase, tests, and UI to align with PP-DocLayoutV3 labels, including expanded and renamed block types. Revise document settings, block filtering, and stitching logic to support the new set of block kinds. Update documentation and environment variable references to reflect the model transition.
BREAKING CHANGE: PDF parsing now requires PP-DocLayoutV3 ONNX model and updated environment variables; block kind names and settings have changed throughout the system.
Migrate PDF parsing to use ONNX Docling layout model for structured block extraction, enabling improved TTS segmentation and chaptering. Add compute backend abstraction for heavy tasks (alignment, layout parsing) with configuration via `OPENREADER_COMPUTE_MODE`. Integrate block-level locators and document settings for per-document PDF parsing options. Update S3 storage for parsed PDF JSON, add migration and schema changes, and extend client and server APIs for parsed data and settings. Remove legacy word highlight flag in favor of compute capability detection.
- Add ONNX model download, local/none compute modes, and `onnxruntime-node` dependency
- Update PDF viewer and TTS pipeline to use parsed blocks and block-level locators
- Add document settings storage and APIs for per-document PDF options
- Update S3 storage layout for parsed PDF JSON
- Add admin/config/docs updates for new compute and parsing features
- Remove obsolete word highlight runtime flag and UI
BREAKING CHANGE: PDF parsing and word highlighting now require `OPENREADER_COMPUTE_MODE=local` and ONNX model; document settings and S3 layout updated; legacy word highlight flag removed.
Introduce sentence and word highlighting for HTML/TXT/MD documents, including new config options (`htmlHighlightEnabled`, `htmlWordHighlightEnabled`) and synced user preferences. Update the HTML viewer to support these highlights using custom spans, and add corresponding CSS styles. Extend document settings to allow toggling these features. Implement block-level locator sorting and display improvements. Add audiobook export support for HTML documents, including adapter and pipeline integration.
- Add `htmlHighlightEnabled` and `htmlWordHighlightEnabled` to config, user state, and Dexie storage
- Update HTML viewer to apply and manage highlights for sentences and words
- Add CSS for HTML highlight classes and scroll margin
- Extend document settings UI for HTML highlight toggles
- Support HTML audiobook export in modal and pipeline
- Improve block locator sorting and sidebar display for HTML
- Add HTML audiobook adapter and block parsing utilities
Introduce `enableUserSignups` runtime setting to allow administrators to control
whether new accounts can be created. Update environment variable and documentation
references to support this feature. UI elements for account creation are now
conditionally rendered based on this flag. Signup attempts are blocked server-side
when disabled, including email, OAuth, and anonymous upgrades.
Add `assertUserSignupAllowed` utility for consistent enforcement and corresponding
unit tests to verify policy behavior.
Refactor chapter API to use new coerceAudiobookGenerationSettings utility for
validating and migrating audiobook metadata. Add src/lib/server/audiobooks/settings.ts
with shared logic and introduce corresponding unit tests to ensure correct
settings migration and validation.
Move changelog version check effect from SettingsModal to a new
scheduleChangelogCheck utility for improved modularity and testability.
Add unit tests for changelog check scheduling behavior.
Introduce changelog feed manifest support, including client and shared
utilities for fetching and parsing changelog data. Add a new
Settings modal panel for viewing changelog entries, with version
detection based on the current app version. Expose a configurable
changelog feed URL in both environment variables and admin panel
runtime settings. Update documentation and deployment workflow to
support changelog feed generation and consumption. Include unit tests
for changelog utilities.
Eliminate the showAllDeepInfraModels runtime/admin config and all code paths
that allowed restricting Deepinfra's model catalog. Deepinfra now always
shows the full model list regardless of API key or environment variable.
Update documentation, environment examples, admin panels, runtime config,
provider catalog logic, and tests to reflect this change.
Remove fallbackProviderRef usage from settings and config context to streamline
provider resolution. Adjust normalizeLegacyProviderRef to retain 'default-openai'
unless a non-default fallback is explicitly provided. Update provider listing to
enforce deterministic ordering by updatedAt, createdAt, and slug. Expand unit tests
to verify provider ordering, legacy ref normalization, and selection logic.
Transition all TTS-related logic, types, and UI to use the new providerRef/providerType model in place of legacy ttsProvider fields. Introduce a centralized tts-provider-policy module to encapsulate provider/model capability checks, default value resolution, and compatibility logic. Update all API routes, contexts, hooks, components, and tests to use providerRef and providerType, ensuring consistent handling of built-in and shared TTS providers. Remove legacy defaultTtsModel config in favor of per-provider defaults and shared provider admin control. Add the showAllProviderModels runtime flag to restrict users to provider default models when desired.
BREAKING CHANGE: ttsProvider fields are replaced by providerRef/providerType throughout the codebase; defaultTtsModel config is removed in favor of per-provider defaults.
Add database-backed runtime configuration for feature flags and TTS provider credentials, editable via a new admin UI panel. Replace static NEXT_PUBLIC_* and TTS provider env vars with admin-managed settings stored in the database and injected at SSR for client access. Implement admin-only panels for managing shared TTS provider credentials (with encrypted API keys) and live site feature flags. Add schema migrations, API routes, React contexts, and hooks for SSR-injected runtime config and live updates. Update client and server logic to resolve configuration from the database at runtime, enforcing admin restrictions and supporting migration from legacy env-based config.
BREAKING CHANGE: Feature flags and TTS provider credentials are now managed at runtime via the admin UI. Environment variables are only used for initial seeding and are ignored after first boot. Existing deployments must migrate configuration to the admin panel.
Move daily quota exceeded problem response logic to a dedicated utility in `lib/server/rate-limit/problem-response`. Refactor API routes to use the new builder and remove duplicated formatting code. Update all relevant imports for consistency. This improves maintainability and ensures uniform error responses across TTS endpoints.
Redesign TTS segment storage by decomposing the `tts_segments` table into two normalized tables: `tts_segment_entries` for unique segment identity and locator projection, and `tts_segment_variants` for per-settings audio variants. Update schema, queries, and API routes to use the new structure, including manifest pagination and segment resolution logic. Refactor segment locator handling to use stable projections and manifest cursors. Migrate tests and data cleanup utilities to support the new model.
BREAKING CHANGE: Database schema for TTS segments is now split into entry and variant tables; all code and consumers must use the new structure. Persisted data and APIs relying on the old `tts_segments` table are incompatible.
Implement strict validation for EPUB TTS segment locators, requiring stable spine coordinates (`spineHref`, `spineIndex`, `charOffset`) and rejecting legacy CFI-only locators. Introduce `segmentKey` as a normalized identity derived from segment text, used for robust merging and deduplication of synthesized and persisted segments. Update database schemas, API routes, and manifest logic to support the new locator format and segmentKey. Add helpers for locator resolution and segmentKey construction, with comprehensive unit tests for identity, normalization, and manifest grouping.
BREAKING CHANGE: Persisted EPUB TTS segment locators must now include stable spine coordinates; legacy CFI-only locators are no longer accepted. Consumers must handle the new `segmentKey` field for segment identity and merging.
