Commit graph

126 commits

Author SHA1 Message Date
Richard R
2f1eafeb79 test(e2e): scope settings Documents tab selector to modal 2026-05-28 22:26:49 -06:00
Richard R
e4b0c8358e test(documents): update selectors and assertions for document actions and accessibility
Revise test helpers and specs to use stricter role-based selectors for document
actions, including delete buttons and document links. Improve accessibility and
upload tests by asserting document presence via explicit file lists instead of
generic text checks. Remove redundant summary assertion from delete flow.
2026-05-28 12:34:29 -06:00
Richard R
eff25ad8df feat(doclist): add bulk folder removal with sidebar menu and confirmation
Introduce a "Remove All Folders" action accessible via a new sidebar menu,
allowing users to clear all folders at once. Add a confirmation dialog to
prevent accidental removal. Update the FinderSidebar to include a menu button
with the new action, and implement supporting state and handlers in the
document list. Add DotsHorizontalIcon for menu UI. Expand folder tests to
cover the new bulk removal flow.
2026-05-28 05:18:53 -06:00
Richard R
5305b896c6 refactor: complete phase 7 error-contract rollout 2026-05-27 08:02:31 -06:00
Richard R
00ce39ed74 refactor: roll out phase 7 server error contract 2026-05-27 07:24:07 -06:00
Richard R
e6e029cf8a cleanup compute core/server dead paths 2026-05-26 18:43:05 -06:00
Richard R
943e547836 test: fix JetStream mock typing for tsc 2026-05-26 16:39:40 -06:00
Richard R
f77ee8cfd4 hard-cut batch B: worker-proxy parsed SSE and remove app progress bus 2026-05-26 15:58:14 -06:00
Richard R
df05a7d7a3 refactor(control-plane): hard-cut worker events and shared SSE helpers 2026-05-26 11:41:33 -06:00
Richard R
084cbdfac1 refactor(worker-control-plane): route ops through core orchestrator 2026-05-26 11:16:54 -06:00
Richard R
00fc2d5e36 feat(control-plane): add shared orchestrator and in-memory adapters 2026-05-26 11:09:43 -06:00
Richard R
2df49b54de feat(parse-progress): add local in-memory realtime bus 2026-05-25 21:06:18 -06:00
Richard R
e70b3619e0 tests: harden pdf readiness waits and extend tts/pdf timeouts 2026-05-24 20:58:57 -06:00
Richard R
e75114a943 refactor(onboarding): introduce onboarding state coordinator and central registry
Add useOnboardingCoordinator hook and onboarding-state registry to manage
onboarding-related state such as privacy acceptance and first-visit tracking.
Refactor SettingsModal to utilize the new onboarding state abstractions,
removing legacy privacy gating logic and improving maintainability.

Includes unit tests for onboarding-state registry to ensure correctness.
2026-05-23 17:07:50 -06:00
Richard R
65d25e3ce5 refactor(ui): centralize privacy gating logic for settings modal
Consolidate privacy acceptance checks into reusable hooks and callbacks within
SettingsModal. Replace scattered gating with a single entry point for opening
settings or changelog views, ensuring consistent enforcement. Update test
helpers to robustly dismiss onboarding and settings overlays by avoiding
state race conditions.
2026-05-23 05:50:54 -06:00
Richard R
50c4330ab1 refactor(core,config,tests): update ONNX model manifest loading and test imports
- Replace static JSON imports with runtime manifest loading in pdf and whisper model modules for compatibility with ESM and bundlers
- Refactor Next.js config to clarify compute mode logic and improve worker bundling conditions
- Update test imports to use package entrypoints instead of relative paths
- Remove redundant whisper alignment/model tests now covered elsewhere or by integration
- No breaking changes to public API or model handling logic
2026-05-21 23:49:56 -06:00
Richard R
b2bb36911a test(whisper): fix playwright ESM loading via dynamic imports 2026-05-21 21:53:55 -06:00
Richard R
2fd9f05652 fix(config): load webpack DefinePlugin via createRequire 2026-05-21 21:49:03 -06:00
Richard R
7a49ff9896 chore(lint): fix eslint violations in worker config and e2e tests 2026-05-21 21:44:58 -06:00
Richard R
55acf05ad6 refactor(compute-core): rename pdf and whisper modules to clean capability layout 2026-05-21 21:41:37 -06:00
Richard R
37b6999c2c refactor(compute-core): collapse to root/local-runtime entrypoints 2026-05-21 21:30:44 -06:00
Richard R
37d11bf9b8 refactor(compute): remove legacy app whisper duplicate and unify core whisper boundary 2026-05-21 21:15:10 -06:00
Richard R
1612e6694b refactor(compute-core): capability-boundary exports for runtime and pdf-layout 2026-05-21 21:11:28 -06:00
Richard R
4d12c6913a test(export): streamline export cleanup and improve test reliability
Remove unnecessary UI reload and modal interactions during audiobook export
cleanup to reduce flakiness under load. Set explicit timeout for long-running
MP3 export test to ensure consistent execution. These changes enhance test
stability and maintainability.
2026-05-21 18:50:11 -06:00
Richard R
384369f48b refactor: fix tests + remove deprecated PDF layout processing files and update imports
- Deleted mergeTextWithRegions.ts, LICENSE.txt, manifest.json, parsePdf.ts, renderPage.ts, runLayoutModel.ts, stitchCrossPageBlocks.ts, and types.ts as they are no longer needed.
- Updated unit tests to reflect new import paths from @openreader/compute-core.
- Adjusted tsconfig.json to include new paths for pdf-layout module.
2026-05-21 18:48:17 -06:00
Richard R
dd494d50aa Add force-reparse confirmation flow in PDF UI 2026-05-21 10:34:29 -06:00
Richard R
10748c7fcd Add force-token cache bust for worker PDF op keys 2026-05-21 10:32:18 -06:00
Richard R
3aad8a51e4 Add local compute concurrency limiter with tests 2026-05-21 10:30:36 -06:00
Richard R
4497f610c0 feat(pdf): implement granular PDF parse progress tracking and migrate to parseState
Introduce detailed progress reporting for PDF layout parsing, exposing phase and page-level updates to clients. Replace legacy parseStatus with a structured parseState field in the database schema, updating all relevant backend and API logic. Add SSE endpoint for real-time parse progress updates. Update client hooks and UI to reflect granular progress and improve user feedback during document parsing. Includes migration scripts and new parse-state utility module.
2026-05-21 10:27:16 -06:00
Richard R
b679bf736c chore(config): remove smartSentenceSplitting option and related code paths
Eliminate the smartSentenceSplitting configuration flag from application state, UI, and all TTS and audiobook adapter logic. Consolidate TTS segment planning to always include context units, streamlining code and reducing conditional branches. Update tests and type definitions to reflect the removal of this feature toggle.
2026-05-20 04:51:10 -06:00
Richard R
bb3eb40966 refactor(tts): extend segment prefetching to support source unit arrays and enhance cache key granularity
Update TTS segment prefetching logic to handle arrays of CanonicalTtsSourceUnit for upcoming and next locations, enabling more precise TTS segment mapping. Expand cache key construction to include providerType and instructions, improving cache differentiation for TTS requests. Adjust related types and usages to support richer segment metadata propagation.
2026-05-20 04:44:47 -06:00
Richard R
0798cd845f refactor(export): extract download trigger helper and add retry logic to full download
Move download logic into reusable downloadViaTrigger function. Add retries and
button enabled checks to downloadFullAudiobook for improved robustness against
transient UI or backend delays.
2026-05-20 03:44:09 -06:00
Richard R
8439216dd8 test(pdf-upload): update assertions for PDF viewer heading and page navigation
Replace text-based assertion for PDF title with role-based heading check and
add assertion for visible page navigation button. Improves test robustness
against UI changes and ensures accessibility compliance.
2026-05-19 23:19:53 -06:00
Richard R
84c3b22e5a refactor(pdf-layout): streamline viewer layout settling and harden pdf document loading
Unify and improve layout settling logic across EPUB, HTML, and PDF viewers by
adding delayed recomputation and guarding against zero-height container during
transient states. Refactor PDF document loading to include retry logic with
success detection, ensuring robust handling of transient failures. Update
Playwright test helpers to provide more reliable PDF viewer readiness checks
and introduce request retry utilities for backend API calls. Add targeted unit
tests for single-section chapter fallback in the PDF audiobook adapter. Update
test files and helpers to align with new layout and export logic.
2026-05-19 23:02:41 -06:00
Richard R
f5d7408f17 refactor(pdf-layout): remove compute "none" mode and unsupported parse status
Eliminate the "none" compute mode and all related code paths, including the NoneComputeBackend, "unsupported" parse status, and PDF margin settings. Parsing is now always available if the app starts successfully, and configuration is limited to "local" or "worker" compute modes. Update types, API routes, client adapters, and documentation to reflect this simplification.

BREAKING CHANGE: "none" is no longer a valid COMPUTE_MODE; only "local" and "worker" are supported. "unsupported" parse status and PDF margin settings are removed.
2026-05-19 18:25:27 -06:00
Richard R
874e5ef359 refactor(whisper): migrate word alignment to ONNX backend and remove whisper.cpp integration
Replace the previous whisper.cpp-based word alignment with a fully ONNX-based
implementation using onnxruntime-node and @huggingface/tokenizers. Add new
Whisper ONNX model management, alignment mapping, and spectral analysis modules.
Remove all code and documentation referencing whisper.cpp, update environment
variables, Dockerfile, and docs to reflect ONNX-only alignment. Add unit tests
for alignment and ONNX model logic.
2026-05-19 13:00:21 -06:00
Richard R
6ab5c230c2 feat(pdf): improve cross-page block stitching and enforce TTS segment source boundaries
Enhance PDF cross-page block stitching to move only sentence continuations and
preserve remaining text on the following page. Update TTS segment planning to
support strict source boundary enforcement, ensuring segments do not cross
block or paragraph-title boundaries. Add new tests for both features and
refactor PDF text item normalization to filter out skewed or rotated runs.
2026-05-18 13:21:43 -06:00
Richard R
37a90b734d feat(pdf): migrate to PP-DocLayoutV3 ONNX model and update PDF block taxonomy
Switch PDF layout parsing to use the PP-DocLayoutV3 ONNX model, replacing the previous Docling-based approach. Update all environment variables, model fetching, and manifest handling for the new model and its artifacts. Refactor block kind taxonomy throughout the codebase, tests, and UI to align with PP-DocLayoutV3 labels, including expanded and renamed block types. Revise document settings, block filtering, and stitching logic to support the new set of block kinds. Update documentation and environment variable references to reflect the model transition.

BREAKING CHANGE: PDF parsing now requires PP-DocLayoutV3 ONNX model and updated environment variables; block kind names and settings have changed throughout the system.
2026-05-18 04:26:39 -06:00
Richard R
766c04d08d refactor(pdf): implement ONNX-based Docling layout parsing and block-level TTS for PDFs
Migrate PDF parsing to use ONNX Docling layout model for structured block extraction, enabling improved TTS segmentation and chaptering. Add compute backend abstraction for heavy tasks (alignment, layout parsing) with configuration via `OPENREADER_COMPUTE_MODE`. Integrate block-level locators and document settings for per-document PDF parsing options. Update S3 storage for parsed PDF JSON, add migration and schema changes, and extend client and server APIs for parsed data and settings. Remove legacy word highlight flag in favor of compute capability detection.

- Add ONNX model download, local/none compute modes, and `onnxruntime-node` dependency
- Update PDF viewer and TTS pipeline to use parsed blocks and block-level locators
- Add document settings storage and APIs for per-document PDF options
- Update S3 storage layout for parsed PDF JSON
- Add admin/config/docs updates for new compute and parsing features
- Remove obsolete word highlight runtime flag and UI

BREAKING CHANGE: PDF parsing and word highlighting now require `OPENREADER_COMPUTE_MODE=local` and ONNX model; document settings and S3 layout updated; legacy word highlight flag removed.
2026-05-17 21:18:51 -06:00
Richard R
d90e48aaf3 feat(html): add HTML/TXT/MD sentence and word highlighting, settings, and audiobook export support
Introduce sentence and word highlighting for HTML/TXT/MD documents, including new config options (`htmlHighlightEnabled`, `htmlWordHighlightEnabled`) and synced user preferences. Update the HTML viewer to support these highlights using custom spans, and add corresponding CSS styles. Extend document settings to allow toggling these features. Implement block-level locator sorting and display improvements. Add audiobook export support for HTML documents, including adapter and pipeline integration.

- Add `htmlHighlightEnabled` and `htmlWordHighlightEnabled` to config, user state, and Dexie storage
- Update HTML viewer to apply and manage highlights for sentences and words
- Add CSS for HTML highlight classes and scroll margin
- Extend document settings UI for HTML highlight toggles
- Support HTML audiobook export in modal and pipeline
- Improve block locator sorting and sidebar display for HTML
- Add HTML audiobook adapter and block parsing utilities
2026-05-17 11:14:02 -06:00
Richard R
522540452c feat(auth): add runtime toggle for user sign-ups and enforce signup policy
Introduce `enableUserSignups` runtime setting to allow administrators to control
whether new accounts can be created. Update environment variable and documentation
references to support this feature. UI elements for account creation are now
conditionally rendered based on this flag. Signup attempts are blocked server-side
when disabled, including email, OAuth, and anonymous upgrades.

Add `assertUserSignupAllowed` utility for consistent enforcement and corresponding
unit tests to verify policy behavior.
2026-05-14 12:47:05 -06:00
Richard R
205d8f37e4 fix(tests): enhance onboarding modal dismissal logic in helpers 2026-05-14 12:06:12 -06:00
Richard R
1f548f71ea feat(audiobook): integrate settings coercion utility and migrate metadata handling
Refactor chapter API to use new coerceAudiobookGenerationSettings utility for
validating and migrating audiobook metadata. Add src/lib/server/audiobooks/settings.ts
with shared logic and introduce corresponding unit tests to ensure correct
settings migration and validation.
2026-05-14 11:34:16 -06:00
Richard R
0787d2e88a refactor(settings): extract changelog check logic to reusable utility
Move changelog version check effect from SettingsModal to a new
scheduleChangelogCheck utility for improved modularity and testability.
Add unit tests for changelog check scheduling behavior.
2026-05-14 10:58:32 -06:00
Richard R
9e09f80067 feat(changelog): add changelog feed support and settings integration
Introduce changelog feed manifest support, including client and shared
utilities for fetching and parsing changelog data. Add a new
Settings modal panel for viewing changelog entries, with version
detection based on the current app version. Expose a configurable
changelog feed URL in both environment variables and admin panel
runtime settings. Update documentation and deployment workflow to
support changelog feed generation and consumption. Include unit tests
for changelog utilities.
2026-05-14 09:45:04 -06:00
Richard R
f300dfb01f Retry errored TTS segments with cooldown and retry-after 2026-05-14 08:31:40 -06:00
Richard R
df841210a3 Remove dead segment audio presign route and legacy helpers 2026-05-14 08:01:53 -06:00
Richard R
513128b802 Stream fallback segment audio and support byte-range requests 2026-05-14 07:55:48 -06:00
Richard R
da629b3ecd Enable private caching headers for segment audio routes 2026-05-13 23:44:13 -06:00
Richard R
ef2ee09064 Warm segment audio during preload before playback 2026-05-13 23:39:29 -06:00