Commit graph

336 commits

Author SHA1 Message Date
Richard R
71cef7b540 refactor(db): migrate tts_segments to settings_json and update indexes
Switch tts_segments schema to use settings_json (jsonb for Postgres, text for SQLite)
instead of settings_hash. Update related indexes and migration SQL. Regenerate
Drizzle snapshots to reflect the new structure. This improves flexibility for
storing TTS settings and enables more robust segment lookups.
2026-05-04 13:22:58 -06:00
Richard R
222d221e81 fix(tts): prevent stale playback by tracking playback run state in TTS context
Introduce `playbackRunIdRef` and `invalidatePlaybackRun` to ensure that playback
state is correctly reset on abort and content changes. Update abort and navigation
handlers to call `invalidatePlaybackRun`, preventing duplicate or stale playback
runs, especially in scenarios with rapid restarts or browser-specific audio event
quirks. This addresses issues where playback could become stuck or behave
unexpectedly due to outdated in-flight guards.
2026-05-04 12:26:29 -06:00
Richard R
7d7e933323 refactor(tts): replace manifest and audio endpoints with presigned and fallback URLs
Remove legacy TTS segment manifest and audio endpoints in favor of new presigned and fallback audio URL approach. Update segment manifest structure to include `audioPresignUrl` and `audioFallbackUrl` fields, deprecating the previous `audioUrl`. Refactor segment ensure logic and TTS context to utilize the new playback source model. Add server utilities for generating presigned S3 URLs and introduce new API routes for presign and fallback audio retrieval. Clean up obsolete client API functions and types to reflect the new manifest format.
2026-05-04 12:04:24 -06:00
Richard R
9999cafec1 feat(tts): add TTS segments API, schema, storage, and manifest endpoints
Introduce TTS segment support with new database tables and migrations for both Postgres and SQLite. Implement segment ensure, audio, and manifest API endpoints, segment storage in S3, and segment authorization logic. Update TTS context and client API to use segment-based synthesis and retrieval. Add server-side helpers for segment ID, settings hash, locator normalization, and audio probing. Extend user data cleanup and teardown logic to remove TTS segment objects. Add unit tests for segment helpers.

BREAKING CHANGE: TTS synthesis now uses segment-based storage and APIs; database migrations required.
2026-05-04 10:18:56 -06:00
Richard R
21030f28eb refactor: update document section title from 'Local Documents' to 'Your Documents' 2026-04-16 20:13:01 -06:00
Richard R
0c7a845952 feat(tts): add pauseActiveHowl helper and improve playback pause/resume handling
Introduce pauseActiveHowl to centralize audio pause logic, ensuring consistent
cleanup and media session state updates. Update pause and togglePlay to use the
new helper, enabling accurate resume of paused audio and improved error
handling. Enhance Howl onpause event to synchronize playback state and clear
timeouts.
2026-04-16 15:47:39 -06:00
Richard R
0db53fbd4e refactor(tts): extract voice resolution logic to server module and update imports
Move resolveVoices and resolveReplicateVoiceInputKey to a new server-side
voice-resolution module to improve separation of concerns and reduce shared
bundle size. Update all imports and related tests to use the new module.
Remove unused LRUMap and related caches from shared catalog. Update Replicate
cooldown logic to use per-scope LRU cache for improved concurrency handling.
2026-04-16 15:46:55 -06:00
Richard R
3db3fd19ce refactor(tts): centralize upstream response helpers and improve settings validation
Move getUpstreamStatus and getUpstreamRetryAfterSeconds to a shared utility
module for consistent upstream error handling across TTS endpoints. Strengthen
audiobook chapter API by introducing runtime validation for incoming settings
payloads, ensuring type safety and error reporting for malformed requests.
Replace in-memory Map caches with LRUMap for Replicate voice and schema
lookups, improving memory management and eviction logic.
2026-04-16 13:07:31 -06:00
Richard R
893f74f038 refactor(tts): update Replicate model defaults and voice resolution
Align default Replicate model to the versioned Kokoro model across UI, docs,
and server logic. Refactor TTS settings merging for audiobooks to ensure
consistent normalization. Improve Retry-After header handling for upstream
rate limits. Expand Replicate voice resolution to use model schemas when
available, with test coverage for custom and built-in models.

Update documentation to reflect new Replicate defaults, model selection,
and configuration guidance.
2026-04-16 12:52:08 -06:00
Richard R
d0c0ab7420 feat(replicate): add support for custom Replicate model selection 2026-04-16 11:24:19 -06:00
Richard R
9da9232d39 feat: add support for Replicate TTS provider and models
- Updated environment variables documentation to include Replicate as a TTS provider option.
- Added Replicate to the sidebar for TTS provider guides.
- Included Replicate as a dependency in package.json and pnpm-lock.yaml.
- Enhanced audiobook chapter generation to normalize native speed settings based on the TTS provider.
- Improved error handling in TTS API routes to provide retry information for rate-limited responses.
- Updated AudiobookExportModal to reflect native speed support for Replicate models.
- Modified SettingsModal to set default model for Replicate.
- Enhanced SpeedControl component to conditionally render native speed controls based on provider support.
- Updated TTSContext to utilize effective native speed for TTS requests.
- Implemented Replicate request handling in the TTS generation logic.
- Added new documentation for configuring Replicate as a TTS provider.
2026-04-16 11:20:38 -06:00
Richard R
b7cc2436d2 feat(theme): add persistent custom theme with live color editing
Add support for a persistent "custom" theme that users can edit and preview in settings.

- add CustomThemeColors type and storage helpers (getCustomThemeColors, setCustomThemeColors)
- apply custom CSS variables at runtime and provide applyCustomColors to live-update variables
- include fallback CSS for html.custom to avoid flash before hydration
- add ColorPicker component with curated swatches for each color role
- extend SettingsModal to support editing, previewing, and persisting custom theme colors; exclude 'custom' from regular theme lists and read custom colors when rendering previews
- tighten theme detection logic to treat custom background luminance as light/dark when needed

This enables users to define, persist, and see live updates for custom color palettes.
2026-04-07 01:09:08 -06:00
Richard R
f9237f9270 feat(tts): add TTS_UPSTREAM_TIMEOUT_MS for configurable request timeout 2026-04-06 17:48:56 -06:00
Richard R
bf139d9a67 refactor(tts): ensure timeout cleanup and validate remote voice payloads
Make voice-fetching more robust by always clearing the request timeout and stricter type-checking of remote data.

- move clearTimeout into finally blocks for Deepinfra and custom OpenAI voice fetches to guarantee the timeout is cleared on success, failure, or abort
- validate that custom endpoint returns an array of strings before accepting voices; otherwise fall back to defaults

This reduces potential timer leaks and avoids accepting malformed voice lists from custom endpoints.
2026-04-06 11:41:42 -06:00
Richard R
09944ec4e4 refactor(audiobooks,tts,config): tighten abort handling, progress accounting, equality checks, and voice fetch timeout
Improve robustness across audiobook pipeline, TTS catalog, and preferences comparison.

- audiobooks: centralize abort creation with createAudiobookAbortError and replace repeated ad-hoc throws with it; avoid reprocessing non-completed chapters when collecting existing indices; update progress accounting by advancing processedLength and emitting onProgress after saving a chapter.
- tts: add a 10s AbortController timeout when fetching custom OpenAI voices, wire the signal to fetch, and ensure the timeout is cleared on success or failure; fall back cleanly when endpoint doesn't support voices.
- config: harden deepEqual by checking mismatched array vs object shapes early to avoid incorrect comparisons.

No user-visible behavior changes besides improved cancellation, progress reporting, and more resilient remote voice discovery.
2026-04-06 11:28:18 -06:00
Richard R
7d9d8de967 test(ui): stabilize onboarding/modal dismissal in e2e helpers
Refactor test helpers to reliably dismiss onboarding and migration modals during
UI flows. Replace ad-hoc Settings-only dismissal with a unified dismissOnboardingModals
routine that targets privacy, migration, and settings dialogs via testids,
interacts with visible controls (checkbox, continue/skip/save buttons) and waits
for dialogs to hide. Increase resilience by repeating checks, adding short
settle delays, and tightening timeouts.

Also add data-testid attributes to PrivacyModal, SettingsModal and
DexieMigrationModal to make the selectors deterministic and reduce flakiness in
export/upload tests.

No production behavior changes.
2026-04-06 11:16:54 -06:00
Richard R
97dba6d987 refactor(tts,audiobook,tests): centralize TTS catalog, harden epub & UI flows
- Move TTS provider resolution and voice-fetch logic into shared catalog and make
  remote fetches more resilient (timeouts, abort handling, baseUrl normalization,
  treat explicit empty custom-openai responses as valid).
- Add continuation guard to TTS feature set and update API voices endpoint error
  logging to better surface auth/session failures.
- Extract and stabilize ebook pipeline and epub adapter:
  - Cache prepared EPUB chapters to avoid repeated parsing and handle missing/empty
    chapter titles safely.
  - Fail early when created chapter lacks a bookId.
- Improve config/preferences handling:
  - Replace JSON stringify comparisons with a deepEqual utility to compare defaults.
  - Tighten ConfigContext effect dependencies and consolidate synced preference
    queuing logic to include additional keys.
- Harden end-to-end test helpers and unit tests:
  - Make settings dialog dismissal deterministic (use Enter/Escape and explicit
    visibility/enabled checks) and simplify upload retry behavior.
  - Add unit test to ensure custom-openai empty voice lists are preserved.

Why: Reduce runtime and test flakiness, centralize TTS logic for easier maintenance,
and make EPUB processing and preference diffing more robust. Breaking changes: none.
2026-04-06 10:28:00 -06:00
Richard R
52e512f17d test(ui,tests): stabilize audiobook export tests and harden settings modal interactions
- Add waitForBackendDownloadReady helper to tests/export.spec.ts to wait for backend
  chapter metadata and enabled full-download UI before attempting combines/downloads.
  Use it in two export tests to reduce flakiness related to eventual backend storage visibility.
- Improve uploadAndDisplay helper in tests/helpers.ts to retry link click when transient
  "intercepts pointer events" errors occur, ensuring test clicks succeed under flaky UI conditions.
  Add dismissSettingsModalIfVisible to detect and close the Settings dialog before interacting
  with page links (click Save or press Escape, wait for dialog to hide).
- Update SettingsModal.tsx useEffect dependencies to avoid referencing checkFirstVist in the
  local state sync effect and add separate effect for checkFirstVist for clearer lifecycle.

Why: Reduce end-to-end test flakiness by waiting for backend readiness and making UI interactions
more robust; simplify SettingsModal effect dependencies to prevent unnecessary re-runs. Breaking changes: none.
2026-04-06 09:59:37 -06:00
Richard R
d6ae2baa6f refactor(tts,config,audiobook): centralize TTS provider catalog and extract audiobook pipeline
- Introduce a shared tts-provider-catalog to centralize provider/model/voice defaults
  and feature flags (supportsTtsInstructions, providerSupportsCustomModel, resolveProviderModels,
  getDefaultVoices, etc.). Replace ad-hoc provider logic across server and client with catalog calls.
- Replace inline voice/model UI logic in SettingsModal, AudiobookExportModal, TTSContext,
  generate server code, and voice hook to use catalog helpers (resolveTtsSettingsViewModel,
  supportsTtsInstructions, getDefaultVoices).
- Extract audiobook generation responsibilities into a small client-side pipeline:
  - Add createEpubAudiobookSourceAdapter and createPdfAudiobookSourceAdapter adapter modules
    to prepare chapters from EPUB/PDF sources.
  - Add runAudiobookGeneration and regenerateAudiobookChapter pipeline functions to handle
    chapter preparation, progress tracking, request header construction, and retry/abort behavior.
  - Wire adapters/pipeline into EPUBContext and PDFContext to simplify and centralize audiobook flow.
- Move config preference helpers out of ConfigContext into dedicated client modules:
  - buildSyncedPreferencePatch (lib/client/config/preferences.ts)
  - applyConfigUpdate and getVoicePreferenceKey (lib/client/config/updates.ts)
  - Use these helpers in ConfigContext to keep update logic small and testable.
- Add TTS settings view model resolver (lib/client/settings/tts-settings.ts) and tests
  for the tts-provider catalog and config helpers (tests/unit/tts-provider-catalog.spec.ts).
- Minor usages updated: AudiobookExportModal, SettingsModal, useVoiceManagement, TTSContext,
  server tts generation, and various context files to consume the new modules.

Why: Reduce duplication, improve testability, and separate concerns for provider metadata,
UI view-model logic, and audiobook generation flow. Breaking changes: none.
2026-04-04 17:53:57 -06:00
Richard R
4a985b3bf1 feat(tts): add continuation loop guard and block fingerprinting
Introduce safeguards for continuation handling in TTS flow to prevent looping and over-carrying text. Add constants to limit carried characters and words, and loop-guard thresholds. Implement normalizeBlockFingerprint to create compact fingerprints of page-first blocks and store them in pageFirstBlockFingerprintRef. Use fingerprint comparison and progress threshold to skip to next page when a loop is detected. Cap continuation slices by characters and words and ensure carried text matches the returned addition.

test(ui): make settings save button click more robust

In tests/helpers.ts wait for the Settings Save button to become visible (with a short timeout) before attempting to open Settings, and click the Settings button with force to avoid intermittent pointer-interception failures. Improve stability of setupTest by handling enter transitions and overlays.
2026-04-04 14:50:34 -06:00
Richard R
e8b5452863 fix(config): preserve voice selection when resetting TTS provider settings
Add a test to verify selected voice persists instead of resetting to default.
Also refactors export tests to be UI-driven rather than backend-dependent.
2026-03-19 13:41:37 -06:00
Richard R
ab9968f75b feat(config): implement reset logic for TTS provider settings based on feature flag 2026-03-19 12:35:01 -06:00
Richard R
b2a623ba2a feat(ui): add feature flag for TTS Providers tab and update related documentation 2026-03-19 12:25:42 -06:00
Richard R
e41a5ba481 refactor(audiobooks): Refactor TTS generation and audiobook chapter creation
- Removed unused imports and types from TTS route.
- Consolidated TTS buffer generation logic into a new module.
- Implemented caching for TTS audio buffers using LRU cache.
- Updated TTS request handling to support instructions for specific models.
- Refactored audiobook chapter creation to use new TTS generation logic.
- Simplified error handling and response management in TTS API.
- Enhanced client-side logic to manage TTS requests and retries.
- Updated types to reflect changes in TTS request payload structure.
2026-03-19 11:54:33 -06:00
Richard R
df0e4341ff fix(ui): update SettingsModal and test helpers for improved onboarding flow in non-auth mode 2026-03-19 11:33:20 -06:00
Richard R
b6852da2a2 feat(ui): enhance AudiobookExportModal and ConfirmDialog with improved layout and styling; update VoicesControlBase for better dropdown handling 2026-03-19 11:16:24 -06:00
Richard R
1deb8c21f7 feat(ui): add five new light themes and redesign settings modal
Add five new light theme options (lavender, rose, sand, sky, slate) with dedicated color palettes. Redesign Settings modal with sidebar navigation replacing tabs, add visual theme color selector showing preview swatches, and introduce new icons (KeyIcon, PaletteIcon, UserIcon, DocumentIcon). Also update PrivacyModal to render conditionally based on authEnabled and add privacy policy link. Update tests to reflect new UI structure.
2026-03-19 10:24:16 -06:00
Richard R
fe05de9fec fix(issue #81): Add synchronous guard to prevent duplicate audio playback in TTSProvider 2026-03-18 01:57:01 -06:00
Richard R
49c5b794ea feat: Implement sequential lookahead preloading for multiple upcoming TTS sentences. 2026-02-22 03:31:03 -07:00
Richard R
6e442e8dba feat: Explicitly set and default TTS response format to MP3. 2026-02-22 03:21:27 -07:00
Richard R
f5cd3b262a feat: Refine TTS retry logic to handle client-side transport failures and enhance voice management within the TTS context. 2026-02-22 03:08:16 -07:00
Richard R
cbd8aa6240 feat: Optimize audiobook export by attempting audio stream copying, adding duration validation, refining chapter duration calculation, and synchronizing export settings with playback controls. 2026-02-21 22:57:59 -07:00
Richard R
7810a6329a feat: add KittenTTS-FastAPI support and update documentation
Some checks failed
Create and publish Docker images / prepare (push) Has been cancelled
Version Docs on Tag / version-docs (push) Has been cancelled
Create and publish Docker images / build (amd64, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (arm64, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / merge (push) Has been cancelled
2026-02-21 20:19:00 -07:00
Richard R
bc4c888866 feat: implement US-only availability check for production instance in middleware and privacy page
Some checks failed
Create and publish Docker images / prepare (push) Has been cancelled
Version Docs on Tag / version-docs (push) Has been cancelled
Create and publish Docker images / build (amd64, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (arm64, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / merge (push) Has been cancelled
2026-02-19 12:28:30 -07:00
Richard R
ffcd120f68 fix: replace getS3Client with getS3ProxyClient in audiobook functions 2026-02-19 11:46:23 -07:00
Richard R
7482a4265e feat: Enhance authentication handling by adding error logging and improving session checks in the API and ClaimDataModal component 2026-02-19 02:13:43 -07:00
Richard R
a0cc189a0f refactor: rename project from OpenReader WebUI to OpenReader across all relevant files. 2026-02-18 02:49:47 -07:00
Richard R
f7af557073 feat: Implement S3 proxy client, enhance session handling, and improve Drizzle migration script. 2026-02-18 02:39:54 -07:00
Richard R
30ad62aa41 feat: Remove explicit background state and implementing text prefetching for smoother page/section transitions. 2026-02-18 00:05:41 -07:00
Richard R
3ede092adc feat: Make word highlighting enabled by default by inverting its environment variable logic, updating documentation, and removing NEXT_PUBLIC_NODE_ENV from Playwright tests. 2026-02-17 22:12:11 -07:00
Richard R
021533223b refactor: Standardize database timestamp fields to bigint epoch milliseconds and update schema migrations. 2026-02-17 21:00:36 -07:00
Richard R
c2188b476f feat: Refactor server-side document, audiobook, and authentication logic, introduce new client-side APIs and caching, and update related tests. 2026-02-17 19:30:12 -07:00
Richard R
8e1b6b4009 refactor: Move document-related components into a dedicated documents subdirectory and update imports. 2026-02-17 18:29:54 -07:00
Richard R
8f5e87ed4a feat: Replace NEXT_PUBLIC_NODE_ENV with explicit feature flags for granular control over DOCX conversion, destructive actions, TTS defaults, and DeepInfra model visibility. 2026-02-17 18:26:03 -07:00
Richard R
eeeceb8e54 feat(privacy): implement data export and encryption with updated policy
Add ZIP-based user data download endpoint, enable AES-256 server-side S3 encryption for all stored documents and audiobooks. Rewrite privacy policy with detailed CCPA categories, service provider disclosures (Vercel/Neon/Railway), and user rights sections. Replace privacy modal with checkbox agreement flow.
2026-02-17 17:16:58 -07:00
Richard R
8dc836cbb7 feat(auth): implement route protection and security headers
Add Next.js middleware to handle session-based access control for protected
routes while maintaining access to public paths. Configure comprehensive
security headers including CSP, HSTS, and frame options in next.config.ts.

Also, reduce session cookie cache duration to 5 minutes to ensure frequent
revalidation against the database and fix a navigation issue in the EPUB
context when jumping to specific CFI locations.
2026-02-16 19:06:24 -07:00
Richard R
f58a2b690d refactor(db): remove user_id foreign key from user_tts_chars
Remove the foreign key constraint referencing the user table in both
Postgres and SQLite schemas. This decouples the usage tracking from
the primary user table.
2026-02-16 17:29:26 -07:00
Richard R
fc24b31cc3 perf: lazy load heavy dependencies and db drivers
Move database drivers and heavy server-side modules to be loaded
lazily via require() and dynamic import(). This reduces the initial
memory footprint and improves cold start performance for serverless
functions by avoiding loading unused dependencies on every request.
2026-02-16 16:10:03 -07:00
Richard R
9931c7d0d8 fix(api): add retry for document blob head checks
Implement a retry mechanism for document blob HEAD requests to handle
S3 read-after-write propagation delays. This prevents 409 errors
when the client calls the endpoint immediately after performing a
direct upload to S3 via a presigned URL.
2026-02-16 15:51:01 -07:00
Richard R
596c04d4d3 feat(preview): improve failure recovery and configuration
Update the client-side preview status logic to gracefully handle failed
states and return fallback URLs. Add PDF worker tracing to the build
configuration for preview-related API endpoints and include server-side
error logging.
2026-02-16 15:46:30 -07:00
Richard R
deaacbd6f0 feat(auth): add conditional GitHub authentication support
Implement a check for GitHub OAuth credentials to conditionally render
the GitHub sign-in button. This ensures the UI accurately reflects
available authentication methods based on environment configuration.

- Add `isGithubAuthEnabled` utility to verify server-side credentials
- Propagate GitHub auth status through context providers to the UI
- Conditionally display the GitHub sign-in button in the sign-in page
- Update documentation and examples to use hex encoding for `AUTH_SECRET`
- Add unit tests for the new GitHub authentication configuration logic
2026-02-16 14:21:25 -07:00
Richard R
da0fce708f refactor: improve database reliability and clean up code
Enable WAL mode and set a busy timeout for SQLite to better handle
concurrent access and prevent 500 errors. Add error handling to
async calls in UI components and remove unused imports across several
files.
2026-02-15 15:44:20 -07:00
Richard R
4d13b5b821 refactor: improve async state and session reliability
Implement AbortController in ConfigContext, PDFContext, and preference sync to prevent race conditions during session changes or rapid navigation. Refactor document preview routes into shared utilities and add LRU caching for text previews to optimize memory. Update documentation for Better Auth schema ownership and migration workflows. Fix issues with Postgres system user seeding and better-sqlite3 configuration.
2026-02-15 15:13:25 -07:00
Richard R
8cb2929116 feat(db): add system user seeding and storage cleanup
- Implement ensureSystemUserExists to handle foreign key constraints for
  unclaimed data when authentication is disabled.
- Add missing foreign key relationship between audiobook chapters and
  audiobooks with cascade delete.
- Integrate a beforeDelete hook in auth config to trigger storage
  cleanup when a user is removed.
- Regenerate initial migrations for both PostgreSQL and SQLite to
  reflect schema updates.
2026-02-15 14:00:35 -07:00
Richard R
d3c7831f70 fix: address hook violations and improve resource reliability
- Refactor useAuthSession to call useSession unconditionally, adhering to React Rules of Hooks by using a stub client when auth is disabled.
- Update migration script to insert a system user to satisfy foreign key constraints during database transitions.
- Implement AbortController and timeouts in entrypoint endpoint polling to prevent hanging fetch requests.
- Use useMemo in PDFViewer to provide a stable file object, preventing unnecessary re-renders and warnings.
- Enhance state cleanup in EPUBContext and AuthLoader to handle missing metadata or disallowed sessions gracefully.
2026-02-15 13:31:06 -07:00
Richard R
bedbbb026a feat(db): add foreign keys and cascade deletes
Enforce referential integrity by linking application tables to the auth user table across both PostgreSQL and SQLite schemas.

- Add foreign key references with ON DELETE CASCADE to all user-owned tables.
- Re-generate initial migrations for Postgres and SQLite providers.
- Update migration script to ensure the SQLite database directory exists.
- Remove explicit SSL configuration from the PostgreSQL connection pool.
2026-02-15 13:02:42 -07:00
Richard R
bb853acaf6 Refactor authentication schema and database handling
- Updated SQLite journal metadata with new tag and timestamp.
- Refactored account deletion API to utilize Better Auth's built-in user deletion method for cascading cleanup.
- Enhanced database initialization to include authentication schemas for both SQLite and Postgres.
- Removed user, session, account, and verification tables from the Drizzle schema, as they are now managed by Better Auth.
- Created separate schema files for authentication in both Postgres and SQLite.
- Added SQL migration scripts for creating necessary tables in both database systems.
2026-02-15 12:49:23 -07:00
Richard R
f9366dbfa2 refactor(audio): replace ffprobe with ffmpeg for metadata extraction
- Remove ffprobe-static dependency and associated configuration
- Reimplement ffprobeAudio to parse duration and title tags from ffmpeg output
- Update CI workflow to install SeaweedFS binary and set required environment variables
- Enhance E2E test setup to handle auth-protected endpoints during document cleanup
2026-02-15 12:13:29 -07:00
Richard R
1f4794a706 feat(auth): implement session gatekeeping and architectural reorganization
- Introduce `USE_ANONYMOUS_AUTH_SESSIONS` to make guest access an opt-in feature.
- Transition root-level pages into `(app)` and `(public)` route groups for improved access control.
- Adopt Figtree as the primary typeface and implement a pre-render theme initialization script.
- Decouple audiobook chapter processing into a standalone API route and harden resource ownership logic.
- Replace the legacy footer with a unified layout and add skeleton loading states for document lists.
- Update environment documentation and test suites to align with the revised routing and auth flows.

BREAKING CHANGE: The audiobook generation API has been restructured, moving specific chapter logic to `/api/audiobook/chapter`.
2026-02-15 11:12:42 -07:00
Richard R
9f25e05cce feat(previews): implement document preview generation and caching
- Add document preview caching logic with in-memory and persisted storage.
- Implement S3 blobstore functions for managing document previews.
- Create rendering functions for PDF and EPUB cover images to JPEG format.
- Introduce database schema and functions for managing document preview metadata.
- Add unit tests for rendering PDF and EPUB previews.
2026-02-12 16:05:03 -07:00
Richard R
82f82a990e feat: add user preferences and document progress syncing
- Implemented user preferences management with a new API for GET and PUT requests.
- Added user document progress tracking with a new API for retrieving and updating progress.
- Introduced database schema changes for user preferences and document progress.
- Enhanced EPUB and TTS contexts to support syncing user preferences and document progress.
- Added functions to handle transferring user preferences and progress during account linking.
- Updated client-side logic to schedule syncing of user preferences and document progress.
2026-02-11 13:58:16 -07:00
Richard R
852092c558 docs: reorganize documentation structure and fix word highlight behavior 2026-02-11 06:03:51 -07:00
Richard R
9b9206f50d refactor(audiobooks): migrate audiobook pipeline to object storage
This commit restructures the audiobook generation and serving layer to rely exclusively on S3-compatible blob storage, removing the dependency on local filesystem paths.

- Bundle `ffmpeg` and `ffprobe` binaries via npm to ensure portability across environments.
- Update API routes to stream audio directly from blob storage instead of local disk.
- Remove legacy migration endpoints and filesystem-based indexing logic.
- Add startup scripts to facilitate the transition from local to remote storage.

BREAKING CHANGE: Audiobook functionality is now strictly dependent on S3 configuration. The previous filesystem-based storage method has been removed.
2026-02-11 02:44:34 -07:00
Richard R
6d5fb65a72 feat(docs): add configurable rate limiting and external docs
Add environment variables for fine-grained control over TTS rate limiting
and Better Auth behavior. Move documentation to external Docusaurus site
with automated deployment workflows.

- TTS rate limiting can now be enabled/disabled via TTS_ENABLE_RATE_LIMIT
- Customizable daily limits for anonymous/authenticated users and IP backstops
- Better Auth rate limiting can be disabled via DISABLE_AUTH_RATE_LIMIT
- Rename library import env vars to IMPORT_LIBRARY_DIRS/DIR
- Add docs-site with Docusaurus and GitHub Actions workflows
- Update README to reference external documentation
2026-02-10 15:11:37 -07:00
Richard R
81d249ed52 feat(storage): implement S3/SeaweedFS blob storage for documents
Replaces local filesystem document storage with S3-compatible object storage.
Adds embedded SeaweedFS 'weed mini' for local development and Docker deployments.
Updates document upload flow to use presigned URLs with a server fallback proxy.
Refactors auth configuration to use BASE_URL and AUTH_SECRET.

BREAKING CHANGE: Renamed BETTER_AUTH_URL to BASE_URL and BETTER_AUTH_SECRET to AUTH_SECRET. Removed legacy document upload/content endpoints. Requires S3 environment variables (auto-configured for embedded SeaweedFS).
2026-02-10 12:29:15 -07:00
Richard R
4a5f3060f2 refactor(documents): migrate from IndexedDB to server-backed storage with caching
- Replace client-side IndexedDB with server-first document storage
- Add document caching layer for offline capability and performance
- Implement document transfer during anonymous-to-authenticated account linking
- Add database indexes for improved query performance
- Update document contexts to use new caching system
- Refactor document upload and deletion APIs
- Add audiobook pruning for missing files
- Improve test isolation with namespace support
- Add unit tests for document cache and transfer functions
2026-02-08 11:15:57 -07:00
Richard R
2c23dc9043 refactor(db): remove conditional database usage from API layer
Ensure all metadata operations consistently use the database layer,
removing filesystem-only fallback paths and conditional DB checks.

- Simplified migration scripts to always run on startup
- Updated document/audiobook APIs to always query DB
- Added ensureDbIndexed() calls across all routes
- Extracted test namespace utilities to dedicated module
- Removed migration-manager.ts (functionality consolidated)
- Updated rate limiter to assume DB is always available

BREAKING CHANGE: Database is now required in all configurations.
When auth is disabled, SQLite is used by default at /app/docstore/sqlite3.db.
2026-02-03 12:17:30 -07:00
Richard R
c24710b2ca refactor(audiobooks): implement user-specific storage with composite primary keys
- Change audiobooks and audiobookChapters tables to use composite PK (id, userId)
- Migrate audiobooks from flat storage to user-specific directories under audiobooks_users
- Add support for claiming unclaimed audiobooks on account creation
- Improve auth rate limiting with retry logic and DISABLE_AUTH_RATE_LIMIT for tests
- Fix iOS/Safari audio playback with unlock mechanism and playback rate watchdog
- Update API routes to handle user-scoped audiobook access with fallback to unclaimed
- Transfer audiobooks when linking anonymous accounts to real accounts
- Remove foreign key constraint from audiobookChapters to support composite PK
- Add cascade delete to account and session foreign keys
2026-02-03 12:17:06 -07:00
Richard R
30ce65e747 refactor(auth): bootstrap anonymous sessions via AuthLoader
Replace privacy popup with PrivacyModal and reorganize auth UI components
Remove signed-out persistence and centralize session handling in useAuthSession
Move document viewers and Kokoro voice helpers into dedicated modules
2026-01-28 02:39:44 -07:00
Richard R
20111b1e36 fix(tests): try to get tests working again 2026-01-28 01:04:01 -07:00
Richard R
297752e83e fix(db): select drizzle config for migrations
Add dedicated Drizzle config files for sqlite and postgres and generate
new migration outputs for each dialect.

Update migrate scripts to load env files and auto-pick the correct
config unless explicitly provided, and tighten API auth handling by
using shared auth context helpers and enforcing userId checks when auth
is enabled.
2026-01-27 16:53:58 -07:00
Richard R
8189087ad9 fix: reduce pdf flicker and harden tts
- Smooth react-pdf page turns with stable canvas staging and fade-in
- Retry transient empty page text extraction during fast page turns
- Avoid noisy TTS preload toasts, guard requests when quota is exhausted,
  and improve Howler retry/unload behavior to prevent pool exhaustion
- Enforce auth FK cascade deletes and tighten rate-limiter bucket updates
  using affected-row checks
2026-01-26 17:01:51 -07:00
Richard R
c57b913cb5 feat(db/auth): migrate auth storage to Drizzle and add claimable data flow
- replace custom DB adapter with Drizzle setup (SQLite/Postgres schemas) and add `drizzle.config.ts` plus migrations in `drizzle/` and `drizzle_pg/`
- switch better-auth to drizzleAdapter, add auth helpers (`getAuthContext`, `requireAuthContext`, `requireAudiobookOwned`), and make `useAuth`/`useAuthSession` safe no-ops when auth is disabled
- persist documents/audiobooks in DB with ownership checks, unclaimed fallback, and ref-counted deletes; add FS scan helper for no-auth mode
- gate docx-to-pdf, library, voices, whisper, and migration endpoints behind auth when enabled
- add unclaimed data scan/claim flow with `/api/user/claim`, `ClaimDataModal`, and server-side scan/claim helpers
- refactor rate limiting and account deletion to use Drizzle tables (`user_tts_chars`, `user`)
- run migrations via `scripts/migrate-if-auth.mjs` (auto on `pnpm start` + `pnpm migrate`), remove Docker entrypoint and old better-auth migration file
- update README and lockfile for the new migration workflow and dependencies
2026-01-26 17:01:36 -07:00
Richard R
9561d5a816 fix(ui): adjust modal positioning for responsive design
Update modal components to start from top on mobile devices and center on larger screens, improving usability across screen sizes. Also refine transition handling in privacy popup for cleaner unmounting.
2026-01-25 15:13:02 -07:00
Richard R
fb8fc41994 fix(ui): adjust dropdown positioning and scrolling in voices control and improve the ui label for speed controls 2026-01-25 14:55:45 -07:00
Richard R
c345628200 fix: remove unused imports and fix privacy popup text
- Remove unused SettingsIcon and UserMenu imports from epub and pdf pages
- Remove unused UserMenu import from home page
- Remove unused isDev const from home page
- Update SettingsModal to only import clearSignedOut
- Fix apostrophe encoding in privacy popup description
2026-01-25 14:41:49 -07:00
Richard R
50c3829f98 update(auth): enhance rate limiting with device/IP backstops and improve UI
- Add device ID and IP-based rate limiting to prevent abuse
- Refactor UI components for better header menus and settings
- Update privacy popup with detailed data usage info
- Improve PDF handling and caching to prevent react-pdf warnings
- Update README with new Docker instructions and environment variables
2026-01-25 14:38:31 -07:00
Richard R
4689f3676f refactor(auth): consolidate authentication contexts and enhance rate limit UI integration
Streamline auth client factory and remove redundant session manager
Introduce unified auto rate limit context for seamless status updates
Implement problem details in TTS responses for improved error clarity
Add rate limit pause controls and banners across document viewers
Refine PDF scaling logic with height-aware resize handling
Update privacy popup triggers and first-visit modal behavior

BREAKING CHANGE: Daily character limits adjusted (anonymous: 250K → 50K, authenticated: 1M → 500K)
2026-01-25 11:12:19 -07:00
Richard R
9ba20c8a9e feat(auth): add user authentication and rate limiting
- Implement user sign-in, sign-up, and account management using better-auth
- Add rate limiting for TTS API with daily character limits for authenticated and anonymous users
- Integrate SQLite and PostgreSQL database support for user sessions and data persistence
- Update UI components to include authentication flows, user menu, and privacy popup
- Modify Dockerfile and package.json for new dependencies and entrypoint script
- Bump version to v1.3.0
2026-01-24 17:36:11 -07:00
Richard R
64825b626a chore: Update various dependencies and introduce pnpm overrides for specific packages. 2026-01-24 12:33:48 -07:00
Richard R
54b9691524 refactor(nlp): enhance EPUB text splitting for oversized sentences and remove unused functions
- Add logic to split oversized sentences in `splitTextToTtsBlocksEPUB` using `splitOversizedText` to ensure blocks stay within `MAX_BLOCK_LENGTH`
- Remove `extractRawSentences` and `processTextWithMapping` functions as they are no longer needed
- Update tests to reflect the changes, including a new test for oversized sentence splitting and removal of related test suites
2026-01-21 14:27:20 -07:00
Richard R
efe6ffec86 fix(nlp): add EPUB-specific text splitting for TTS blocks
Introduce `splitTextToTtsBlocksEPUB` function to handle EPUB text splitting,
treating single newlines as paragraph boundaries for precise highlighting.
Update TTS context to conditionally use EPUB splitting based on document type.
Enhance PDFViewer to clear highlights when current sentence is null.
Add comprehensive tests for the new functionality and refactor existing ones.
2026-01-21 14:04:19 -07:00
Richard R
63c316fc4b fix(pdfviewer): improve sentence highlight cleanup when current sentence is null
Handle case where currentSentence is null by canceling retry loops and clearing stale highlights to prevent lingering highlights. Remove redundant check inside the highlight function.
2026-01-21 13:32:04 -07:00
Richard R
5a40e4b00a fix: improve word highlight cleanup in PDFViewer and update test
- Add cleanup function to prevent memory leaks in word highlighting
- Update test to allow blocks ending with ., !, or ? for better accuracy
2026-01-21 13:17:57 -07:00
Richard R
c28c074e43 refactor(nlp): improve text splitting and normalization for TTS blocks
- Rename and enhance text processing functions in nlp.ts for better handling of oversized texts, sentence boundaries, and PDF artifacts
- Update PDFViewer to add layout-aware highlighting with retry logic for sentence and word highlights
- Adjust PDFContext and TTSContext to use new normalized text functions
- Expand unit tests for new splitting behaviors, including long texts and punctuation preferences
2026-01-21 12:09:59 -07:00
Richard R
409b25977c tests: Add NLP text processing, audiobook chapter encoding/decoding, and SHA256 hashing testing. 2026-01-19 17:45:55 -07:00
Richard R
8dbf904a21 feat: Implement FFmpeg metadata escaping and robust chapter index validation for audiobook generation, alongside unit test reorganization. 2026-01-19 17:33:29 -07:00
Richard R
9c941b57ed fix: Enhance path sanitization in getAudiobooksDir to prevent directory traversal and update getMigratedDocumentFileName tests to assert truncated- prefix and hash suffix. 2026-01-19 17:17:51 -07:00
Richard R
47d838039a feat: Add bookId validation to API routes and implement robust, length-limited filename generation with unit tests. 2026-01-19 17:04:02 -07:00
Richard R
33d49d6966 fix: Enhance audiobook API validation and prevent critical meta file deletion 2026-01-19 16:40:04 -07:00
Richard R
f947ece01a feat: Implement document selection modal to allow users to select specific documents for sync, load, and import operations. 2026-01-19 15:53:29 -07:00
Richard R
741126c0b4 feat: Display a toast notification after library migration, powered by new migration status in the API response. 2026-01-19 15:27:07 -07:00
Richard R
57ac17fb34 refactor(api): force dynamic rendering for API routes
Add 'force-dynamic' export to audiobook and documents API routes to ensure
dynamic rendering. Remove 'nodejs' runtime export from TTS route for consistency.
2026-01-19 14:33:49 -07:00
Richard R
199b937a25 refactor: overhaul document storage and audiobook architecture
- refactor(documents): implement stable SHA256-based document IDs
- feat(library): add support for external document libraries
- refactor(audiobook): consolidate server logic and migrate to V1 storage layout
- feat(audiobook): add reset functionality and improve chapter management
- fix(ui): enhance audiobook export modal with download/regenerate controls
- test: add coverage for upload hashing and export reset flow
2026-01-19 14:22:21 -07:00
Richard Roberson
d5ec96d395 feat(ui): implement better document grid with file previews
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
- Introduces `DocumentPreview` component to display rich previews for documents.
- PDFs generate a thumbnail of their first page.
- EPUBs extract and display their cover image.
- HTML, TXT, and Markdown files show a text snippet.
- Refactors document list and folder views to a responsive grid layout.
- Expands main content areas to accommodate the new grid.
- Updates Dockerfile to use Node.js LTS image.
- Updates various development and runtime dependencies.
2025-12-12 13:13:49 -07:00
Richard Roberson
30ade1d79a fix(ui): fix chapter list overflow and menu positioning
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
- Add scroll functionality to the chapter list to prevent content
  overflow.
- Conditionally adjust the positioning of chapter action menus
  to ensure they remain visible within the modal, preventing
  them from being cut off for chapters at the top of the list.
2025-11-22 18:37:38 -07:00
Richard Roberson
d2189541e8 fix(tailwind): fix footer layout and tailwind config
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
- Remove unused `isDev` variable from `Footer.tsx`.
- Add `inline` and `inline-flex` classes to footer elements for improved layout.
- Move custom `xs` breakpoint definition to `theme.extend` in `tailwind.config.ts`.
2025-11-22 16:32:03 -07:00
Richard Roberson
a755a0b3f9 refactor(ui): make UI labels and descriptions more concise 2025-11-22 15:27:04 -07:00
Richard Roberson
e86782bf38 feat: improve document list and self-hosting clarity
- Introduce a toggleable grid/list view for the document list, enhancing organization and display flexibility. This involved updates across `DocumentList`, `DocumentFolder`, `DocumentListItem`, and `SortControls`.
- Add a new `CodeBlock` component and integrate detailed Docker-based self-hosting instructions into the footer.
- Enhance privacy policy popover with clearer details on Deepinfra usage and strongly recommend self-hosting for secure experience.
- Implement conditional rendering and feature gating based on the `isDev` environment variable, differentiating features between the production demo and self-hosted instances. Affected components include `page.tsx`, `DocumentSettings.tsx`, `SettingsModal.tsx`, and `config.ts`.
- Clarify that advanced features like audiobook export and word-by-word highlighting via `whisper.cpp` are exclusive to self-hosted setups and are disabled in the demo.
- Expand the list of supported document types on the homepage to include MD and TXT.
- Integrate new `ListIcon`, `GridIcon`, and `CopyIcon` to support UI enhancements.
- Add a custom `xs` breakpoint in `tailwind.config.ts` for improved responsive design.
2025-11-22 15:18:46 -07:00
Richard Roberson
e39a5b8bcf refactor(player): simplify popover input focus logic
Removed `isPopoverOpen` state and associated `useEffect` hook.
Auto-focus and select logic for the input are now directly handled
within `handlePopoverOpen`, reducing state management complexity.
2025-11-22 02:06:28 -07:00