No description
Find a file
Richard R f0e0daae77
feat: improve PDF parsing recovery and reader UI (#98)
* refactor(ui): replace anchor tags with next/link in SidebarNavLink and UserMenu

Update SidebarNavLink to use next/link for navigation instead of anchor tags,
ensuring proper routing and improved accessibility in Next.js. Refactor UserMenu
to remove legacy Link wrappers and directly use SidebarNavLink for signin and
signup links. This streamlines navigation components and aligns with Next.js
best practices.

* fix(pdf): handle non-zero viewport origins and improve layout model ordering

Update normalizeTextItemsForLayout to correctly apply viewport transforms,
including non-zero page origins, ensuring accurate mapping of PDF text items
to top-left coordinates. Refactor runLayoutModel to implement a custom order
sequence builder for layout regions using model order logits, improving
region ordering consistency with model semantics. Update related tests to
cover viewport transform edge cases.

* refactor(pdf): improve text normalization with font ascent and vertical overlap logic

Enhance text normalization by incorporating font ascent and descent data to more accurately position glyphs, especially for decorative initials. Update merge logic to better detect line membership using vertical overlap, ensuring drop caps and overlapping glyphs are merged correctly. Extend tests to cover these layout scenarios.

* style(range): redesign slider with precision gauge and ruler ticks

Revamp the range input to feature a minimalist "precision gauge" style.
Introduce a hairline rail, ruler notches for discrete steps, and a slim
needle thumb. Add CSS variables and logic for per-instance tick sizing
and coloring. Remove bulky inline class-based styling in favor of
centralized CSS for improved maintainability and visual clarity.

* refactor(ui): modularize PDF loader and range slider visuals

Move PDF layout scan visualization and range slider styles into dedicated CSS
modules, isolating their styles from the global scope. Integrate PdfLayoutScan
component into the PDF viewer loader UI for animated parse progress. Refactor
progress bars to use a reusable progress-fill class with animated sheen effect.
Update range input to use CSS module for precision gauge styling.

* style(reader): remove grid overlay from PdfLayoutScan visualization

* refactor(api): add staleness detection for inflight worker operation states

Integrate isWorkerOperationStateStale checks into document parse API endpoints to ensure inflight worker operation states are not reused if stale. Introduce helper for staleness detection and corresponding unit tests. Enhance SSE event streaming with keepalive intervals and improve progress acknowledgment error handling.

* feat(worker): recover and fail stale in-flight pdf ops on startup

Add orphaned operation recovery logic to detect and mark stale in-flight pdf_layout jobs as failed during worker startup. Extend OperationStateStore with listOpStates for state enumeration. Update tests and documentation to cover recovery behavior and new environment variable COMPUTE_PDF_JOB_ATTEMPTS.

* refactor(worker): distinguish staleness thresholds for running and queued pdf ops

Update orphan recovery logic to apply separate timeouts for 'running' and
'queued' pdf_layout operations. Adjust tests to verify that only stale
'running' operations are failed, while stale 'queued' operations remain
untouched.

* feat(worker): extend orphan recovery to handle whisper_align ops and improve logging

Update orphan recovery to detect and fail stale 'running' whisper_align operations
in addition to pdf_layout. Refactor recovery logic to generalize staleness checks
across operation kinds and enhance log output with detailed operation info.
Expand tests to verify correct handling of both whisper_align and pdf_layout
operations in running and queued states.

* refactor(control-plane): introduce revision-based CAS for operation state updates

Add revision tracking and compare-and-set (CAS) semantics to operation state
stores, enabling atomic state transitions and preventing lost updates. Extend
the OperationStateStore interface with getOpStateRecord and compareAndSetOpState
methods. Update orchestrator and worker runtime to utilize CAS for marking
operations as failed only if the state is unchanged. Enhance in-memory,
JetStream, and test control plane implementations to support revision logic.

This change improves concurrency safety and correctness of operation state
management across distributed components.

* feat(ui): add parse failure state to PDF layout scan animation

Display a distinct "parse halted" visual state in the PDF layout scan
component and PDF viewer page when parsing fails. The loader animation
is replaced by a static, dimmed page with an alert glyph and updated
styling, ensuring users are not misled by an active animation after a
failure. CSS and component logic updated to support the new state.

* feat(worker): extract orphaned operation recovery to module with periodic sweep

Move orphaned operation recovery logic into a dedicated orphan-recovery module,
introducing a periodic sweep timer that triggers recovery every 15 seconds while
the worker is connected. Refactor runtime to delegate orphan detection and
handling to the new module, improving modularity and maintainability. Add
unit tests for orphan-recovery to ensure correctness.

* fix(ui): adjust PDF viewer layout and update parse loader description

* refactor(pdf): streamline layout model region extraction and update test coverage

- Replace custom order sequence logic with softmax-based class selection in runLayoutModel
- Remove unused sigmoid and buildOrderSequence functions
- Simplify detection loop to filter and map regions directly
- Add targeted tests for layout model extraction logic
- Update CSS animation naming for consistency
- Clarify test description and add inline comments for orphan recovery scenario

* fix(pdf): add strict validation for layout model output shapes and extend test coverage

Add explicit error handling for invalid or inconsistent pred_boxes and logits array lengths in runLayoutModel to prevent silent failures. Expand test suite to verify correct region filtering and error scenarios, ensuring only labeled regions are returned and malformed outputs are handled robustly.
2026-06-03 04:44:55 -06:00
.github refactor(env): remove legacy no-auth mode and enforce required auth env vars 2026-05-31 12:09:37 -06:00
compute feat: improve PDF parsing recovery and reader UI (#98) 2026-06-03 04:44:55 -06:00
docs-site feat: improve PDF parsing recovery and reader UI (#98) 2026-06-03 04:44:55 -06:00
drizzle feat(rate-limit): add per-user PDF parsing rate limiting and job event ledger 2026-05-30 14:17:49 -06:00
public update(auth): enhance rate limiting with device/IP backstops and improve UI 2026-01-25 14:38:31 -07:00
scripts refactor(env): remove legacy no-auth mode and enforce required auth env vars 2026-05-31 12:09:37 -06:00
src feat: improve PDF parsing recovery and reader UI (#98) 2026-06-03 04:44:55 -06:00
tests feat: improve PDF parsing recovery and reader UI (#98) 2026-06-03 04:44:55 -06:00
.dockerignore refactor(onboarding): modularize onboarding flow logic and improve testability 2026-05-29 18:03:17 -06:00
.env.example feat: improve PDF parsing recovery and reader UI (#98) 2026-06-03 04:44:55 -06:00
.gitattributes Fix check mark in SettingsModal 2025-02-11 21:31:11 -07:00
.gitignore refactor(onboarding): modularize onboarding flow logic and improve testability 2026-05-29 18:03:17 -06:00
.npmrc chore: add .npmrc for npm configuration 2026-05-21 11:18:41 -06:00
Dockerfile Fix container crash on startup: onnxruntime-node requires glibc (switch Alpine → Debian slim) (#97) 2026-06-02 20:50:37 -06:00
drizzle.config.pg.ts Refactor authentication schema and database handling 2026-02-15 12:49:23 -07:00
drizzle.config.sqlite.ts Refactor authentication schema and database handling 2026-02-15 12:49:23 -07:00
empty-module.ts chore(lint): fix eslint violations in worker config and e2e tests 2026-05-21 21:44:58 -06:00
eslint.config.mjs Redesign: shared UI design system + app-wide migration (#96) 2026-06-01 16:17:12 -06:00
LICENSE Initial commit 2025-01-18 00:36:14 -07:00
next.config.ts hard-cut batch D: enforce worker bundle isolation and CI guard 2026-05-26 16:00:26 -06:00
package.json chore(release): bump version to v4.0.3 2026-06-02 21:06:32 -06:00
playwright.config.ts refactor(env): remove legacy no-auth mode and enforce required auth env vars 2026-05-31 12:09:37 -06:00
pnpm-lock.yaml refactor(worker): modularize worker entrypoint and add vitest infra 2026-05-30 11:00:01 -06:00
pnpm-workspace.yaml feat(compute): add external compute worker backend and integration 2026-05-19 15:21:25 -06:00
postcss.config.mjs First push 2025-01-18 04:58:57 -07:00
README.md refactor(env): remove legacy no-auth mode and enforce required auth env vars 2026-05-31 12:09:37 -06:00
tailwind.config.ts Redesign: shared UI design system + app-wide migration (#96) 2026-06-01 16:17:12 -06:00
tsconfig.json build(tsconfig): add path alias for compute-core control-plane module 2026-05-26 15:45:19 -06:00
vitest.config.ts refactor(env): remove legacy no-auth mode and enforce required auth env vars 2026-05-31 12:09:37 -06:00

GitHub Release License Docs Playwright Tests Docs Check

GitHub Stars GitHub Forks Discussions

📄🔊 OpenReader

OpenReader is an open source, self-host-friendly text-to-speech document reader built with Next.js for EPUB, PDF, TXT, MD, and DOCX with synchronized read-along playback.

Previously named OpenReader-WebUI.

Get started in the docs.

Highlights

  • 🧱 Layout-aware PDF parsing with PP-DocLayoutV3 (ONNX) — structured block detection, cross-page stitching, and geometry-based highlighting for precise read-along sync.
  • ⏱️ Word-by-word highlighting via ONNX Whisper alignment through the compute worker control plane (NATS JetStream-backed).
  • Segment-based read-along for EPUB, PDF, TXT, MD, and DOCX — sentence-aware TTS with cached audio segments, background preloading, and resumable playback.
  • 🎯 Multi-provider TTS — self-hosted OpenAI-compatible servers (Kokoro-FastAPI, KittenTTS-FastAPI, Orpheus-FastAPI) or cloud APIs (OpenAI, Replicate, DeepInfra).
  • 🎧 Audiobook export in m4b/mp3 with resumable chapter processing.
  • 🗂️ Flexible backend — embedded SeaweedFS or S3-compatible storage, SQLite or Postgres, server library import, and device sync.
  • 🐳 Self-host friendly — Docker (amd64/arm64), built-in auth/session support, and automatic startup migrations.

🚀 Start Here

Goal Link
Run with Docker Docker Quick Start
Deploy on Vercel Vercel Deployment
Deploy external compute worker Compute Worker (NATS JetStream)
Develop locally Local Development
Configure auth Auth
Configure SQL database Database and Migrations
Configure object storage Object / Blob Storage
Configure TTS providers TTS Providers
Run Kokoro locally Kokoro-FastAPI
Run KittenTTS locally KittenTTS-FastAPI
Get support or contribute Support and Contributing

🧭 Community

📜 License

MIT. See LICENSE.