implement a generic user_job_events table for tracking compute job creation
enforce configurable burst and sustained limits for PDF layout parsing
add admin panel controls for compute rate limiting and max upload size
update API routes to apply and record rate checks for PDF parse jobs
document new environment variables and admin settings for compute limits
improve IP extraction logic for rate limiting accuracy
add tests for request IP extraction and test namespace gating
This change introduces a robust mechanism to throttle expensive compute operations, such as PDF parsing, on a per-user basis. It provides both burst and sustained rate controls, with admin-tunable parameters and clear user feedback on throttling. The job event ledger enables accurate concurrency and rate enforcement, while new documentation and tests ensure maintainability and clarity.
- Delete EditableRow from formPrimitives and its usage in AdminFeaturesPanel
- Remove EditIcon from icon set as it is no longer referenced
- Simplify AdminFeaturesPanel state and draft handling
- Update AdminProvidersPanel to use distinct query key for default provider
- Ensure query invalidation covers new default provider key
This streamlines the admin codebase by eliminating obsolete editing UI and related icon, and clarifies provider query logic.
- Remove legacy TTS rate limit environment variables and migrate all related configuration to runtime settings managed via the Admin UI
- Add runtime config keys for TTS rate limiting enable/disable and per-user/IP daily quotas
- Refactor rate limiter and API routes to use runtime config for thresholds and enablement
- Update AdminFeaturesPanel to allow editing TTS rate limiting and quota values in the UI
- Add shared-provider-selection utility for consistent provider selection logic
- Update documentation to reflect new runtime/admin configuration and remove obsolete env var docs
- Add unit tests for rate limit runtime settings and provider selection
BREAKING CHANGE: TTS rate limiting is now controlled via Admin → Site features; environment variables for TTS rate limiting are no longer supported and will be ignored.
Add SidebarUploadLoader component to DocumentList for displaying real-time
upload progress in the sidebar. Extend DocumentUploader to emit detailed
batch state updates via onUploadBatchChange, enabling granular feedback
on current file, phase, and completion status. Improves user awareness of
document upload and conversion processes.
Introduce onRequestSidebarClose prop to FinderWindow and implement its
usage in DocumentList to enable closing the sidebar on mobile via
backdrop tap or Esc key. Enhances mobile usability by allowing users to
dismiss the sidebar more intuitively.
Update FinderSidebar to remove background color on inactive hover for
improved visual clarity. Extend the DocumentSkeleton warning timeout to
10 seconds to allow more time before displaying the alert. Refine test
description for icons grid layout to clarify auto-fit behavior.
Enhance the document icons grid to better handle single-row layouts by
dynamically suppressing column stretching when the number of documents fits
in a single row. Introduce maxColumnsForIconGrid and related logic to
calculate optimal column counts based on container width and icon size.
Update grid style computation to allow optional suppression of single row
stretch, improving alignment and appearance for small document sets.
Add unit tests for icon grid layout calculation and column logic to ensure
correctness across different icon sizes and container widths.
Introduce per-page progress events for PDF parsing by adding onPageStarted
hooks throughout the compute worker and core PDF pipeline. Add
compute/worker/src/pdf-progress.ts to encapsulate progress event builders.
Update server logic to emit page start and page parsed events, improving
client feedback during long-running PDF jobs.
Add src/lib/server/compute/abort-like-error.ts and integrate isAbortLikeError
to robustly detect and suppress expected aborts in API event streaming.
Update API route to ignore abort-like errors on worker proxy crash.
Add unit tests for abort-like error detection and PDF progress event
generation. Refactor PDF parse logic to allow empty merged regions without
throwing, ensuring downstream flows remain resilient to edge cases.
These changes improve observability, error handling, and reliability for
document parsing and event streaming.
Extract onboarding flow step resolution and async runner logic into a
dedicated module for better separation of concerns and maintainability.
Refactor context/provider to use new helpers and simplify settings modal
control. Add unit tests for onboarding step resolution and async runner
coalescing. Update e2e helpers and selectors for onboarding modals.
Adjust environment file handling for CI and Docker contexts.
- Add .dockerignore for build hygiene
- Refactor onboarding context and modal logic
- Add src/lib/client/onboarding-flow.ts with core onboarding helpers
- Add onboarding-flow unit tests
- Update Playwright config and scripts for CI env loading
- Improve test selectors for claim modal
- Update .gitignore to exclude all .env* except .env.example
This change increases onboarding logic modularity and reliability, and improves CI/test environment handling.
Introduce a document identity key combining type and id for consistent
identification across document operations, including selection, drag-and-drop,
and folder management. Refactor drag item structures to use identity objects
instead of plain ids, preventing cross-type collisions and improving merge
accuracy. Update Dexie recently opened map to use identity keys. Enhance
document size formatting for small files and improve error feedback in the
uploader. Clean up redundant props and standardize icon SVG handling.
Eliminate the deprecated "columns" view mode from the document list, toolbar,
icons, and skeleton components. Update all related types and logic to only
support "icons", "list", and "gallery" views. Standardize user-facing labels
from "HTML" to "Text" for improved clarity across the sidebar, segment
locator labels, and document counts. Clean up associated code and types to
reflect these changes.
Shift app-level actions (settings and user menu) from the top bar to the bottom
of the sidebar for improved navigation consistency. Introduce a new "sidebar"
variant for the UserMenu component with tailored styling and behavior for
sidebar placement. Update SettingsModal to support a customizable trigger label
and styling for sidebar integration. Refactor FinderSidebar to accept an
optional bottomSlot, enabling flexible placement of sidebar controls.
Move grid style and template calculation for icon views into a shared
iconsGrid.ts utility. Replace inline grid logic in IconsView and DocumentListSkeleton
with calls to the new iconsGridStyle function for consistency and maintainability.
Eliminate unused loading state and related effects from document tile and list
row components. Standardize navigation by disabling prefetch on all document
open links in doclist views for consistent client-side routing. Cache
document list state on load to improve state restoration and initialization.
Introduce a "Remove All Folders" action accessible via a new sidebar menu,
allowing users to clear all folders at once. Add a confirmation dialog to
prevent accidental removal. Update the FinderSidebar to include a menu button
with the new action, and implement supporting state and handlers in the
document list. Add DotsHorizontalIcon for menu UI. Expand folder tests to
cover the new bulk removal flow.
Update gallery view document tiles with refined border, shadow, and background
styles for better visual feedback and accessibility. Enhance image preview
component to immediately reveal cached images by checking decode state on mount,
preventing opacity glitches on remount. Adjust gallery rail scrolling to reset
on document list changes for consistent navigation.
Eliminate folder grouping logic and related UI from all document list views.
Switch to a flat document array model for rendering, selection, and filtering.
Remove folder collapse/expand state and folder drag-and-drop handling. Update
sidebar and toolbar components to remove folder creation and management
controls. Simplify props and internal state across all views for consistency.
Introduce recentlyOpenedAt field to document types and fetch per-document
recently opened timestamps from local cache. Update the "Recently Opened"
sidebar filter to sort by this value and show only documents with recent
activity. Adjust grid layout logic for icons view to better handle small
numbers of documents. Update sidebar label for clarity.
Replace inline icon size buttons with a dropdown menu that appears when
the icons view toggle is active. This change improves toolbar clarity by
grouping related controls and enhances accessibility with better focus
handling. The update also streamlines the button structure for view mode
selection.
Replace fixed Tailwind grid column classes with a responsive CSS grid
that leverages tile width variables per icon size. Grid columns now
adapt automatically via `auto-fit` and `minmax`, and grid gaps are
centralized for consistency. This update enhances layout flexibility,
maintainability, and responsiveness in the icons view.
Enhance mobile and desktop sidebar behavior by introducing a separate state for mobile sidebar visibility and ensuring it does not persist from desktop state. Update FinderToolbar to visually indicate sidebar open state and make the toolbar sticky for better accessibility. Refine FinderSidebar width handling for improved responsiveness. Remove redundant border in mobile sidebar dialog for a cleaner appearance. These changes provide a more consistent and intuitive sidebar experience across viewports.
Simplify document list state handling by centralizing loading and empty logic into DocumentList.
Remove redundant state management from HomeContent. Refactor DocumentListSkeleton to support all view modes and icon sizes for improved visual consistency during loading. Enhance ColumnsView styling for folder and document rows, improving selection, hover, and drag feedback. Use useMemo for visible order calculation to optimize rendering.
These changes streamline the document list rendering flow, unify skeleton loading visuals, and improve the clarity and responsiveness of the columns view interface.
Integrate DocumentUploader in overlay mode to wrap all document views, enabling drag-and-drop uploads across the entire list area. Refine file size formatting to support KB, MB, and GB units in all views and tiles. Standardize button scaling and padding for consistent UI rhythm. Update DocumentPreview and detail panels to display both type and human-readable size. Add overlay drag visual feedback for uploads.
These changes improve usability, visual clarity, and consistency in the document list interface.
Revamp the document list experience by introducing a Finder-style window
interface with multiple views (icons, list, columns, gallery) and a new
sidebar filter system. Remove legacy folder and list item components in
favor of modular, windowed views. Integrate react-dnd-touch-backend and
custom DnD context for improved drag-and-drop, including mobile support.
Update uploader and dialog styles for consistency. Extend document types
to support new view modes, icon sizing, and sidebar state.
Add useUnmountCleanupRef custom hook to centralize and simplify unmount
cleanup logic in document page components. Refactor EPUB, HTML, and PDF
page components to utilize this hook, replacing repetitive ref and effect
patterns for invoking cleanup functions on unmount. This reduces code
duplication and improves maintainability.
Remove redundant clearCurrDoc calls from navigation links and ensure document cleanup occurs on component unmount for EPUB, HTML, and PDF pages. Simplify PDF back navigation by eliminating sidebar delay logic. Refactor DocumentListItem to avoid unnecessary router usage and consolidate document link handling. These changes improve maintainability and consistency in document lifecycle management across the UI.
Eliminate the errorCode property from server logger calls, error contract types, and API error bodies throughout the codebase. This reduces duplication and streamlines error reporting, relying on structured event names and error classes for identification. All affected API routes, server logic, and logging utilities have been updated for consistency. Documentation directory added for future reference.
Update TTS segment ensure route to distinguish aborted and failed segment generation, providing more granular logging and context. Replace ambiguous message variables with clearer detail fields, and improve log event naming for abort scenarios. Adjust ESLint logger call selectors for maintainability and consistency, consolidating selector logic and enforcing stricter server logger usage patterns. These changes improve observability and error traceability in TTS segment operations.
Introduce `serverLogger` utility based on pino for consistent, structured logging across all server and API modules. Replace direct console logging with `serverLogger` and add request-scoped logging helpers. Update environment variable handling, documentation, and deployment guides to reflect new logging configuration (`LOG_FORMAT`, `LOG_LEVEL`, `COMPUTE_LOG_LEVEL`). Enforce no-console in server code via ESLint and add pino/pino-pretty dependencies.
This change standardizes log output, improves observability, and prepares the codebase for ingestion by log platforms.
Integrate opId handling throughout PDF layout parsing by updating client hooks, API routes, and event streams to support operation ID propagation. Add server utility for worker operation creation to centralize opId management. Enhance polling and SSE subscription utilities to accept and return opId, improving traceability and coordination of parse operations across the client and server.
Define maxDuration as 300 seconds for the parsed events API route to explicitly
control execution timeouts during long-running PDF layout processing. This
ensures more predictable serverless behavior for streaming operations.
Introduce detailed console logging across document parsing API routes and worker
operation state fetches. Logs now capture stream openings, error conditions,
worker state unavailability, and invalid responses, improving observability and
debuggability for PDF parsing and ONNX layout processing flows.
Add worker operation state polling to document parse routes, enabling real-time
status and progress updates from the ONNX layout worker. Introduce
`fetchWorkerOperationState` utility and status mapping logic to synchronize
parse status with worker job state, improving accuracy for pending, running,
and failed operations.
Move shared type contracts from `contracts` to new `api-contracts` module in `compute/core` for clearer API boundaries. Remove legacy `contracts`, `pdf`, and `whisper` index files. Update all imports to use `api-contracts`. Refactor server job logic by replacing `parsePdfJob.ts` with modular `user-pdf-layout-job.ts` and `user-whisper-align-job.ts`, updating all relevant API routes and compute integration. This improves maintainability and separation of concerns across compute and server layers.
Add useOnboardingCoordinator hook and onboarding-state registry to manage
onboarding-related state such as privacy acceptance and first-visit tracking.
Refactor SettingsModal to utilize the new onboarding state abstractions,
removing legacy privacy gating logic and improving maintainability.
Includes unit tests for onboarding-state registry to ensure correctness.
Move TTS segment cache invalidation to a fire-and-forget promise after
document update to prevent cache errors from blocking parse readiness.
Log warnings for both cache invalidation failures and warnings, but do not
interrupt the main job flow. This improves job robustness and avoids
unnecessary failures due to cache issues.
Consolidate privacy acceptance checks into reusable hooks and callbacks within
SettingsModal. Replace scattered gating with a single entry point for opening
settings or changelog views, ensuring consistent enforcement. Update test
helpers to robustly dismiss onboarding and settings overlays by avoiding
state race conditions.
Move and re-export all PDF parsing and TTS-related types from
@openreader/compute-core/types, eliminating local type duplication.
Update imports throughout the codebase to use the new centralized type
module. Remove obsolete worker-contract file and update tsconfig paths
and compute-core exports for the new types entry point. Refactor API
routes and job logic to use the new type imports and shared parse-state
healing utility. This streamlines type management and improves
consistency between compute and app layers.
Introduce automatic healing for stale PDF parse states in API routes. Add
`isDocumentParseStateStale` utility and use a shared `COMPUTE_OP_STALE_MS`
environment variable (with config getter) to control the stale window for both
worker op replacement and app-side parse-state healing. Update documentation and
environment variable references to reflect new config.
Enables automatic marking of stuck parses as failed, improving reliability and
retry behavior in distributed compute environments.
Update PDF viewer and document hook to improve parse status management:
- Set parseStatus to null on clear instead of 'pending' for accurate state
- Explicitly set initial parseStatus and reset parseProgress when loading a new document
- Adjust loader display logic to check for resolved parse status before showing expanded loader
Prevents UI inconsistencies and ensures parse progress is reflected correctly during document transitions.
- Deleted mergeTextWithRegions.ts, LICENSE.txt, manifest.json, parsePdf.ts, renderPage.ts, runLayoutModel.ts, stitchCrossPageBlocks.ts, and types.ts as they are no longer needed.
- Updated unit tests to reflect new import paths from @openreader/compute-core.
- Adjusted tsconfig.json to include new paths for pdf-layout module.
Standardize compute job concurrency configuration by introducing a shared
COMPUTE_JOB_CONCURRENCY setting, replacing separate PDF and Whisper concurrency
controls. Add cross-platform CPU core detection and thread budgeting utilities,
and update both ONNX model execution and concurrency limiters to use dynamic
thread allocation per job. Refactor environment variable docs and examples to
reflect unified concurrency management. Streamline PDF.js font path resolution
for improved reliability across environments.
Introduce idle and hard cap timeouts for PDF layout parsing in the compute worker,
ensuring long-running jobs are capped and progress is tracked with explicit
timeouts. Refactor NATS JetStream client initialization to use a configurable
API timeout. Update PDF viewer UI to treat parse state as 'pending' by default,
improving clarity during document preparation and parsing. Refine parse loader
labels and progress logic for more accurate user feedback.
Update compute worker to use separate environment variable for PDF job attempts
(`COMPUTE_PDF_JOB_ATTEMPTS`) and set Whisper job max deliveries to 1, clarifying
and separating retry logic between job types. Adjust `.env.example` and deployment
docs to comment out defaults and document new/renamed variables for advanced tuning.
In TTS segment ensure route, introduce explicit timeouts for Whisper alignment
operations, with configurable durations for local and worker modes, improving
robustness and error handling for long-running alignments.
Revise PDF viewer to introduce an explicit 'unknown' parse state and improve
loader and status messaging for better user clarity. Refactor Spinner to use
'currentColor' for border styling and simplify conditional rendering based on
className. Expand compute worker .env.example with explicit concurrency,
timeout, and job attempt variables for enhanced configuration transparency.
Enhance PDF parse loader transitions by introducing a delayed expansion for detailed progress, reducing UI flicker during parse-to-render handoff. Refactor PDFViewer to ensure document readiness is signaled only once per load, preventing redundant callbacks and improving synchronization between parsing and rendering states.
Introduce detailed progress reporting for PDF layout parsing, exposing phase and page-level updates to clients. Replace legacy parseStatus with a structured parseState field in the database schema, updating all relevant backend and API logic. Add SSE endpoint for real-time parse progress updates. Update client hooks and UI to reflect granular progress and improve user feedback during document parsing. Includes migration scripts and new parse-state utility module.
Migrate worker and backend contract to an operation-centric model, introducing opKey-based deduplication for Whisper alignment and PDF layout parsing. Add new operation request/state types and update worker endpoints to accept generic operation requests and expose operation status via SSE. Refactor client and server logic to support operation flows. Update deployment docs, environment examples, and Dockerfile to reflect new architecture and dependency management.
Add ability to fetch parsed documents using a stored S3 key, enabling more flexible blob access patterns. Update API route to prefer `parsedJsonKey` if present, falling back to legacy lookup. Introduce `COMPUTE_WORKER_WAIT_TIMEOUT_MS` environment variable for customizing worker job wait time, with a new default of 120 seconds. Update documentation and examples to reflect these changes.
Update worker server to prioritize platform-specific PORT environment variable
when setting the compute worker port. Add robust normalization for
COMPUTE_WORKER_URL, ensuring proper scheme and validation, and trim trailing
slashes for consistency. This enhances reliability in diverse deployment
environments.
Update backend creation logic to consistently determine compute mode using
a local variable, ensuring correct handling when the mode is undefined. This
improves clarity and robustness in backend selection.
Update TTS context to include configProviderType, supportsInstructions, and
ttsInstructions in the provided value, improving downstream access to provider
capabilities and instruction handling.
Update compute backend interfaces and job logic to consistently handle parsed
PDF layout results as either direct data or S3 object key references. Refactor
result types, local and worker backends, and job processing to support this
pattern. Improves flexibility for large document parsing and object storage
integration.
Restructure compute core by extracting job contracts and runtime logic into separate
modules (`contracts.ts`, `local-runtime.ts`) for improved modularity and clearer
API boundaries. Update exports and imports across worker, server, and local
compute backends to use the new modules. Enhance Next.js config to support
dynamic backend selection via build-time constants and aliasing, enabling
seamless switching between local and worker compute modes. Adjust TypeScript
paths and Dockerfile entrypoint to align with the new structure.
Introduce clearSegmentCaches to TTS context for explicit cache invalidation after segment clearing. Enhance SegmentsSidebar scroll logic to better distinguish user-initiated and programmatic scrolls, improving UX during segment updates.
Eliminate the smartSentenceSplitting configuration flag from application state, UI, and all TTS and audiobook adapter logic. Consolidate TTS segment planning to always include context units, streamlining code and reducing conditional branches. Update tests and type definitions to reflect the removal of this feature toggle.
Update TTS segment prefetching logic to handle arrays of CanonicalTtsSourceUnit for upcoming and next locations, enabling more precise TTS segment mapping. Expand cache key construction to include providerType and instructions, improving cache differentiation for TTS requests. Adjust related types and usages to support richer segment metadata propagation.
Introduce sourceUnitsFromParsedPage helper to extract CanonicalTtsSourceUnit
objects from parsed PDF pages, filtering by block kind and text content. Update
TTS state setter to include extracted source units when available, enabling
improved TTS segment mapping and future extensibility.
Unify and improve layout settling logic across EPUB, HTML, and PDF viewers by
adding delayed recomputation and guarding against zero-height container during
transient states. Refactor PDF document loading to include retry logic with
success detection, ensuring robust handling of transient failures. Update
Playwright test helpers to provide more reliable PDF viewer readiness checks
and introduce request retry utilities for backend API calls. Add targeted unit
tests for single-section chapter fallback in the PDF audiobook adapter. Update
test files and helpers to align with new layout and export logic.
Refactor compute backend initialization to use asynchronous loading and promise caching, replacing previous synchronous singleton pattern. Update all consumers—including TTS segment alignment and PDF layout parsing jobs—to await compute backend resolution before invoking methods. This change improves compatibility with dynamic imports and future-proofs backend selection logic.
Eliminate unused PDF text extraction logic and margin-based filtering from the client library. Refactor API routes to generalize parse status normalization, removing explicit references to deprecated "unsupported" status and type variants. Update usages to accept string parse statuses for improved flexibility. Simplify document state dependency tracking in the PDF document hook.
