This change removes legacy server-side PDF parse state management and transitions
to a fully worker-owned model for PDF parsing operations. Key updates include:
- Deletes all code related to server-managed parse state, including:
- parse-state.ts, parse-state-backfill.ts, parse-state-healing.ts,
parsed-pdf-reuse.ts, pdf-parse-operation.ts, and related job logic
- Removes the user-pdf-layout-job queue and associated job logic
- Refactors API routes for parsed PDF documents and events to use the new
worker-owned PDF parse operation flow under src/lib/server/pdf-parse/
- Updates S3 parsed PDF artifact keying to include parser version for
deduplication and compatibility
- Refactors client API and hooks to handle new error and progress reporting
- Removes all tests for the legacy parse state and job system, updating remaining
tests to mock the new worker-owned flow
BREAKING CHANGE: Server no longer manages per-user PDF parse state or jobs.
All PDF parsing is now managed by the compute worker and new artifact keying.
Legacy parse state and jobs are no longer supported. Existing parsed PDFs may
need to be reprocessed for compatibility with the new model.
Update folder tests to use an exported escapeRegExp helper when constructing
regex patterns for file name matching. This prevents false positives or errors
when file names contain special regex characters, improving test reliability.
## Summary by CodeRabbit
* **New Features**
* Expose PDF parser version system-wide; parser-version-aware parsing, reuse, and caching. Added tokenized temp upload flow, presign/finalize endpoints, and server-side DOCX→PDF conversion.
* Playback readiness propagated to TTS players/pages; unified upload/delete APIs and simplified uploader/sidebar UX.
* **Bug Fixes**
* Normalize stored parse state to current parser version to avoid stale readiness and improve reuse.
* Prevent premature playback starts and redundant preparation.
* **Tests**
* Added unit and E2E tests for parser-version handling, op-key inclusion, parse-state behavior, and canonical upload reuse.
Replace dual HTML5/Touch backend logic with a single TouchBackend configured to
handle both mouse and touch events, ensuring consistent drag-and-drop behavior
across all devices. Introduce a custom DocumentDragLayer to render drag previews
universally, as the touch backend does not provide a native drag image.
Update document tile, gallery, and list views to suppress native long-press
previews on iOS and prevent accidental document opening after drag actions.
Revise drag-and-drop test helpers to simulate pointer gestures compatible with
the new backend, improving reliability of automated tests.
Update DocumentUploader to upload all accepted files without filtering by type,
enabling consistent handling of DOCX and other supported formats. Add a default
timeout for DOCX-to-PDF conversion to prevent hanging LibreOffice processes.
This improves reliability and ensures uploads do not silently stall.
Remove the separate docx-to-pdf upload API and integrate DOCX conversion directly into the blob upload finalize route. Update client and uploader logic to treat DOCX as a supported upload type and streamline batch state handling. Adjust document caching and test helpers to align with the new unified flow. Add server-side DOCX-to-PDF conversion utility and corresponding unit tests.
This change simplifies the upload pipeline, consolidates conversion logic, and improves maintainability.
Avoid enqueuing duplicate parse jobs for PDFs that are already parsed or in terminal states. Update docx-to-pdf upload to reuse existing parsed PDF results when possible, skipping unnecessary parse operations and job enqueuing. Adjust parse-state-backfill to only enqueue jobs for pending or running states. Refine client upload logic to better infer document type from MIME when file names are missing. Enhance usePdfDocument to handle terminal parse states more robustly.
This change ensures idempotent parse job creation and reduces unnecessary worker load, while improving document upload and parse state handling.
Introduce startPdfParseOperation to encapsulate worker operation creation, job rate tracking, and parse state initialization for PDFs. Replace scattered operation setup logic in register-upload, docx-to-pdf upload, and parse-state-backfill with this unified helper. Update job enqueuing to propagate initial opId/jobId/status for accurate state tracking.
Change getParsedPdfDocument to throw on non-ready states and remove polling logic from usePdfDocument, shifting readiness detection to event streams. Switch API responses for non-ready parses from 202 to 409 to clarify client expectations and simplify error handling.
Add targeted tests to verify legacy backfill behavior and ensure no worker operation is created for pure data fetches of pending legacy PDFs.
BREAKING CHANGE: API now returns 409 Conflict for non-ready parsed PDFs instead of 202, and clients must subscribe to event streams for parse progress.
Replace type-specific document add/remove functions with generalized uploadDocuments and deleteDocument methods. Consolidate file upload handling in DocumentUploader for improved maintainability. Introduce evictCachedDocument to streamline cache eviction across document types. Update DocumentContext to support batch uploads and single-method deletion, simplifying context API and reducing duplication.
Add detection for precondition failures during blob copy to prevent race conditions when finalizing uploads. Refactor upload finalization to use Promise.all for concurrent processing. Extract parse state and parsed JSON key logic to variables for clarity and to avoid duplication in document registration.
Transition document upload flow to use presigned token-based temporary uploads. Add finalize endpoint to register uploaded documents after blob transfer, decoupling document ID assignment from upload initiation. Update client and server logic to operate on tokens, improve security, and enable canonical ID reuse for identical uploads. Update tests to verify deduplication behavior.
Ensure TTS text and playback readiness state are properly reset when
the current document data or name changes, or when clearing or setting
the current document. Update effect dependencies and logic to prevent
stale playback state and improve user experience when switching or
removing documents.
Add PDF_PARSER_VERSION constant and propagate parser versioning throughout
the PDF parsing, job, and API layers. Implement normalization of parse state
to ensure compatibility with the current parser version, and enable reuse of
parsed PDF results when possible. Add isPlaybackReady state to document hooks
and TTS player, improving playback UX by disabling controls until content is
ready. Update tests to reflect new playback readiness logic.
BREAKING CHANGE: PDF parse state and job logic now require explicit parserVersion;
older parse states may be treated as pending until reprocessed.
Add explanation and example for the API_KEY environment variable in the
Docker quick start instructions. Clarify its role in seeding the default
provider's key and post-boot key management.
Eliminate the unused --delete-local option and related code from the
filesystem migration script to prevent accidental local data removal.
Introduce a cleanupClaimedLegacyFsSources utility for removing legacy
filesystem sources after claim, and integrate it into the user claim
flow. Add unit tests to verify the cleanup logic.
Introduce a backfill handler to restore missing PDF parse operations for
documents in legacy 'pending' or 'running' states lacking an opId.
Update parsed route handlers to invoke this logic, ensuring correct
operation association for affected documents. Add regression tests to
validate backfill behavior for legacy scenarios.
Replace dynamic require.resolve lookup with direct path construction to
standard_fonts directory under node_modules. Removes use of createRequire
and streamlines runtime font path resolution logic for PDF parsing.
Refactor Dockerfile to optimize layering, embed entrypoint migration tools and compute worker as deployable bundles, and merge only required dependencies for runtime scripts. Add docker/entrypoint-migration-tools package and update workspace configuration. Refactor PDF parsing to resolve standard font data path robustly at runtime. Update dependencies and output config for Next.js standalone mode. Enhance entrypoint script to launch embedded compute worker from deployed bundle.
- Dockerfile now deploys migration tools and compute worker as separate bundles
- compute/core/pdf uses new pdfjs-runtime utility for robust font path resolution
- docker/entrypoint-migration-tools added for migration script dependencies
- scripts/openreader-entrypoint.mjs launches embedded worker from deployed bundle
- next.config.ts enables standalone output and includes standard_fonts in tracing
- Dependency updates across package.json and compute/worker/package.json
BREAKING CHANGE: Docker image structure and runtime entrypoint logic have changed; custom deployment scripts or Docker overrides may require updates.
* refactor(ui): replace anchor tags with next/link in SidebarNavLink and UserMenu
Update SidebarNavLink to use next/link for navigation instead of anchor tags,
ensuring proper routing and improved accessibility in Next.js. Refactor UserMenu
to remove legacy Link wrappers and directly use SidebarNavLink for signin and
signup links. This streamlines navigation components and aligns with Next.js
best practices.
* fix(pdf): handle non-zero viewport origins and improve layout model ordering
Update normalizeTextItemsForLayout to correctly apply viewport transforms,
including non-zero page origins, ensuring accurate mapping of PDF text items
to top-left coordinates. Refactor runLayoutModel to implement a custom order
sequence builder for layout regions using model order logits, improving
region ordering consistency with model semantics. Update related tests to
cover viewport transform edge cases.
* refactor(pdf): improve text normalization with font ascent and vertical overlap logic
Enhance text normalization by incorporating font ascent and descent data to more accurately position glyphs, especially for decorative initials. Update merge logic to better detect line membership using vertical overlap, ensuring drop caps and overlapping glyphs are merged correctly. Extend tests to cover these layout scenarios.
* style(range): redesign slider with precision gauge and ruler ticks
Revamp the range input to feature a minimalist "precision gauge" style.
Introduce a hairline rail, ruler notches for discrete steps, and a slim
needle thumb. Add CSS variables and logic for per-instance tick sizing
and coloring. Remove bulky inline class-based styling in favor of
centralized CSS for improved maintainability and visual clarity.
* refactor(ui): modularize PDF loader and range slider visuals
Move PDF layout scan visualization and range slider styles into dedicated CSS
modules, isolating their styles from the global scope. Integrate PdfLayoutScan
component into the PDF viewer loader UI for animated parse progress. Refactor
progress bars to use a reusable progress-fill class with animated sheen effect.
Update range input to use CSS module for precision gauge styling.
* style(reader): remove grid overlay from PdfLayoutScan visualization
* refactor(api): add staleness detection for inflight worker operation states
Integrate isWorkerOperationStateStale checks into document parse API endpoints to ensure inflight worker operation states are not reused if stale. Introduce helper for staleness detection and corresponding unit tests. Enhance SSE event streaming with keepalive intervals and improve progress acknowledgment error handling.
* feat(worker): recover and fail stale in-flight pdf ops on startup
Add orphaned operation recovery logic to detect and mark stale in-flight pdf_layout jobs as failed during worker startup. Extend OperationStateStore with listOpStates for state enumeration. Update tests and documentation to cover recovery behavior and new environment variable COMPUTE_PDF_JOB_ATTEMPTS.
* refactor(worker): distinguish staleness thresholds for running and queued pdf ops
Update orphan recovery logic to apply separate timeouts for 'running' and
'queued' pdf_layout operations. Adjust tests to verify that only stale
'running' operations are failed, while stale 'queued' operations remain
untouched.
* feat(worker): extend orphan recovery to handle whisper_align ops and improve logging
Update orphan recovery to detect and fail stale 'running' whisper_align operations
in addition to pdf_layout. Refactor recovery logic to generalize staleness checks
across operation kinds and enhance log output with detailed operation info.
Expand tests to verify correct handling of both whisper_align and pdf_layout
operations in running and queued states.
* refactor(control-plane): introduce revision-based CAS for operation state updates
Add revision tracking and compare-and-set (CAS) semantics to operation state
stores, enabling atomic state transitions and preventing lost updates. Extend
the OperationStateStore interface with getOpStateRecord and compareAndSetOpState
methods. Update orchestrator and worker runtime to utilize CAS for marking
operations as failed only if the state is unchanged. Enhance in-memory,
JetStream, and test control plane implementations to support revision logic.
This change improves concurrency safety and correctness of operation state
management across distributed components.
* feat(ui): add parse failure state to PDF layout scan animation
Display a distinct "parse halted" visual state in the PDF layout scan
component and PDF viewer page when parsing fails. The loader animation
is replaced by a static, dimmed page with an alert glyph and updated
styling, ensuring users are not misled by an active animation after a
failure. CSS and component logic updated to support the new state.
* feat(worker): extract orphaned operation recovery to module with periodic sweep
Move orphaned operation recovery logic into a dedicated orphan-recovery module,
introducing a periodic sweep timer that triggers recovery every 15 seconds while
the worker is connected. Refactor runtime to delegate orphan detection and
handling to the new module, improving modularity and maintainability. Add
unit tests for orphan-recovery to ensure correctness.
* fix(ui): adjust PDF viewer layout and update parse loader description
* refactor(pdf): streamline layout model region extraction and update test coverage
- Replace custom order sequence logic with softmax-based class selection in runLayoutModel
- Remove unused sigmoid and buildOrderSequence functions
- Simplify detection loop to filter and map regions directly
- Add targeted tests for layout model extraction logic
- Update CSS animation naming for consistency
- Clarify test description and add inline comments for orphan recovery scenario
* fix(pdf): add strict validation for layout model output shapes and extend test coverage
Add explicit error handling for invalid or inconsistent pred_boxes and logits array lengths in runLayoutModel to prevent silent failures. Expand test suite to verify correct region filtering and error scenarios, ensuring only labeled regions are returned and malformed outputs are handled robustly.
* docker fixes
* fix(worker): default COMPUTE_PREWARM_MODELS to false and update docs, env, and compose
Set COMPUTE_PREWARM_MODELS to false by default in runtime, docker-compose,
and .env.example to avoid pre-downloading ONNX models during worker startup.
Update Dockerfile and deployment documentation to reflect this change and
clarify configuration for model prewarming.
* fix(docker): downgrade pnpm version to 10.33.4 for compatibility
---------
Co-authored-by: Richard R <me@richardr.dev>
Eliminate the enableDestructiveDeleteActions feature flag from runtime
configuration, admin panel, environment docs, and user settings modal.
Remove all references, toggles, and documentation for this flag. This
simplifies the runtime config and user interface by consolidating
destructive actions under account deletion only.
BREAKING CHANGE: The enableDestructiveDeleteActions runtime flag is no longer supported. Any configuration or code depending on this flag must be updated.
Revise the compute worker deployment documentation to clarify usage
scenarios, streamline environment variable instructions, and improve
separation between embedded and standalone deployment modes. Remove
outdated references and reformat for better readability.
Apply "relative" class to IconButton components in TTSPlayer to enable
positioning of child elements or overlays. This prepares the UI for
potential enhancements such as loading indicators or tooltips.
* phase 0: token foundation
* phase 1: motion language
* phase 2: primitives and semantic tokens
* phase 3: depth and rhythm polish
* phase 4: enforce design system lint rules
* phase 5: split ui primitives into modules
* phase 7: refactor app surfaces to ui layer
* phase 9: enforce ui architecture imports
* phase 10: add ui system harness
* fix compact reader auth control
* fix pdf loader flash
* Converge sidebar and reader controls
* refactor: remove initial loader state and related logic from PDFViewerPage
* Remove legacy UI shim imports
* Converge modal and drawer frames
* Migrate secondary modals to shared frame
* Move settings modal onto shared frame
* Use shared cards in audiobook settings
* Converge choice and popover surfaces
* Converge reader navigation buttons
* Refactor UI components to use consistent button and icon styles across the application
* refactor(ui): unify button usage in settings, admin, and doclist components
Replace native button elements with shared Button, ChoiceTile, and IconButton
components for consistent UI behavior and styling. Update classNames and
props to match new component APIs. Adjust FinderSidebar to use utility
function for conditional class merging.
* feat(ui): introduce shared Listbox components for unified select and dropdown styling
Replace direct usage of Headless UI Listbox primitives with new SharedListboxButton,
SharedListboxOption, and SharedListboxOptions components across AudiobookExportModal,
FinderToolbar, VoicesControlBase, and select UI. Refactor related imports and classNames
to centralize dropdown styling and logic. Simplify UserMenu button markup for improved
consistency.
This change consolidates dropdown/select UI patterns, reduces duplication, and
improves maintainability by providing a single source of truth for Listbox styling
and behavior.
* refactor(ui): consolidate button, menu, popover, and range primitives for unified usage
Remove legacy UI harness and dev/demo files. Replace scattered button, menu, popover, and range input utilities with shared, composable primitives: Button, ButtonLink, ButtonAnchor, MenuActionItem, MenuItemsSurface, PopoverSurface, PopoverTrigger, and RangeInput. Update all usages across app, admin, player, and document components to use these new primitives, eliminating duplicated class logic and improving consistency. Remove obsolete utility files and class exports. This change streamlines UI code, centralizes styling, and reduces maintenance overhead.
* feat(ui): redesign range input with dynamic progress styling and improved accessibility
Revamp the range input component to support dynamic progress indication using CSS custom properties and linear gradients. Add logic to compute and set the progress percentage based on current value, min, and max. Refine focus and disabled states for better accessibility and usability. Update styling for both WebKit and Mozilla engines to ensure consistent appearance. This change enhances visual feedback and modernizes the range slider UI.
* style(ui): update range input to use secondary accent color for progress
Switch range input progress styling from primary to secondary accent color
for both WebKit and Mozilla engines. Remove drop shadow from slider thumb
for a cleaner appearance. This change aligns the component with the updated
design palette and simplifies visual effects.
* refactor(app): remove unused Link imports from public pages and components
Eliminate redundant imports of the Link component from Next.js in several
public-facing pages and components. These imports were no longer in use
after recent UI refactoring and consolidation of navigation elements.
This cleanup reduces bundle size and improves code clarity.
* chore(ui): remove unused export of segmented control classes from select component
Eliminate unnecessary export statements for segmentedButtonClass and
segmentedGroupClass in the select component to streamline the module's
public API and reduce potential confusion.
* test(accessibility): improve confirm dialog test coverage and refactor media state helper
Expand accessibility tests for ConfirmDialog to assert dialog semantics,
ARIA attributes, and visible destructive actions using test IDs. Refactor
expectMediaState helper to check both UI control state and underlying
media signals for more robust playback state detection.
* refactor(ui): improve segmented control accessibility and update danger color tokens
Update SegmentedControl to support full keyboard navigation and focus management,
enhancing accessibility. Replace string indicator in Select with icon, and update
button danger variant to use new --danger-strong variable for hover states. Add
danger-strong token to Tailwind config and globals. Refine dropzone disabled
behavior, adjust focus ring for better contrast, and apply minor UI consistency
tweaks across components.
* feat(ui): convert SidebarNavLink to forwardRef component
Refactor SidebarNavLink to use React.forwardRef, enabling parent components
to access the underlying anchor element's ref. Update prop typing and
function signature accordingly for improved composability and integration
with higher-order components.
Move origin extraction to shared utility for consistent validation across
server and client modules. Update layout metadata to use the shared origin
helper, improving maintainability and deployment flexibility. Add unit tests
for layout metadata to ensure correct origin handling.
Switch landing and layout metadata to use BASE_URL from environment for
deployment flexibility. Refine privacy page environment check for robustness.
Update ARIA attributes on landing preview for better accessibility. Adjust
terminal and visually hidden styles for improved formatting and screen reader
support.
Update landing page descriptions to emphasize native file rendering and
layout-aware parsing. Adjust Docker quick start instructions to remove
redundant volume mount and highlight required environment variables for
self-hosting.
Redesign the public landing and privacy pages with a new layout structure,
introducing an aurora background, grain overlay, and updated navigation.
Revise hero section with animated word-level highlighting and refreshed
copy. Update CSS to use semantic theme variables, color-mix, and new
background effects. Adjust navigation, footer, and action button order for
clarity. Update privacy page button label for consistency.
Revise landing page metadata to specify Replicate TTS and clarify export as
audiobook workflows. Simplify hero panel glow effect in CSS for visual
consistency.
Reorganize privacy policy content to improve clarity on service provider roles,
analytics consent, and encryption limitations. Update section headings and
descriptions for accuracy. Enhance CookieConsentBanner and PrivacyModal to
reflect optional analytics and clarify storage, operator access, and password
handling. Improve CSS for policy section headings and lists for better
readability.
Move all button-related class utilities and types from formPrimitives to a new
ui/buttonPrimitives module. Update all component imports to reference the new
location, ensuring consistent button styling and easier future maintenance.
Remove redundant button code from formPrimitives and re-export as needed for
admin UI compatibility. Add public.css for landing and privacy layout styles.
Eliminate the unused authEnabled property from context providers, hooks,
components, and API responses. All logic and UI now assume authentication is
required, simplifying prop signatures and reducing branching. Update related
types, context values, and function calls to reflect this change. This streamlines
the authentication flow and removes unnecessary configuration.
