Commit graph

344 commits

Author SHA1 Message Date
Richard R
8e1b6b4009 refactor: Move document-related components into a dedicated documents subdirectory and update imports. 2026-02-17 18:29:54 -07:00
Richard R
8f5e87ed4a feat: Replace NEXT_PUBLIC_NODE_ENV with explicit feature flags for granular control over DOCX conversion, destructive actions, TTS defaults, and DeepInfra model visibility. 2026-02-17 18:26:03 -07:00
Richard R
eeeceb8e54 feat(privacy): implement data export and encryption with updated policy
Add ZIP-based user data download endpoint, enable AES-256 server-side S3 encryption for all stored documents and audiobooks. Rewrite privacy policy with detailed CCPA categories, service provider disclosures (Vercel/Neon/Railway), and user rights sections. Replace privacy modal with checkbox agreement flow.
2026-02-17 17:16:58 -07:00
Richard R
8dc836cbb7 feat(auth): implement route protection and security headers
Add Next.js middleware to handle session-based access control for protected
routes while maintaining access to public paths. Configure comprehensive
security headers including CSP, HSTS, and frame options in next.config.ts.

Also, reduce session cookie cache duration to 5 minutes to ensure frequent
revalidation against the database and fix a navigation issue in the EPUB
context when jumping to specific CFI locations.
2026-02-16 19:06:24 -07:00
Richard R
f58a2b690d refactor(db): remove user_id foreign key from user_tts_chars
Remove the foreign key constraint referencing the user table in both
Postgres and SQLite schemas. This decouples the usage tracking from
the primary user table.
2026-02-16 17:29:26 -07:00
Richard R
fc24b31cc3 perf: lazy load heavy dependencies and db drivers
Move database drivers and heavy server-side modules to be loaded
lazily via require() and dynamic import(). This reduces the initial
memory footprint and improves cold start performance for serverless
functions by avoiding loading unused dependencies on every request.
2026-02-16 16:10:03 -07:00
Richard R
9931c7d0d8 fix(api): add retry for document blob head checks
Implement a retry mechanism for document blob HEAD requests to handle
S3 read-after-write propagation delays. This prevents 409 errors
when the client calls the endpoint immediately after performing a
direct upload to S3 via a presigned URL.
2026-02-16 15:51:01 -07:00
Richard R
596c04d4d3 feat(preview): improve failure recovery and configuration
Update the client-side preview status logic to gracefully handle failed
states and return fallback URLs. Add PDF worker tracing to the build
configuration for preview-related API endpoints and include server-side
error logging.
2026-02-16 15:46:30 -07:00
Richard R
deaacbd6f0 feat(auth): add conditional GitHub authentication support
Implement a check for GitHub OAuth credentials to conditionally render
the GitHub sign-in button. This ensures the UI accurately reflects
available authentication methods based on environment configuration.

- Add `isGithubAuthEnabled` utility to verify server-side credentials
- Propagate GitHub auth status through context providers to the UI
- Conditionally display the GitHub sign-in button in the sign-in page
- Update documentation and examples to use hex encoding for `AUTH_SECRET`
- Add unit tests for the new GitHub authentication configuration logic
2026-02-16 14:21:25 -07:00
Richard R
94d24c4687 test(export): increase timeout for resume test
Increase timeout to 60 seconds for the audiobook resume test to prevent flakiness when processing PDF exports.
2026-02-16 10:19:54 -07:00
Richard R
80b2ae2676 test: improve test reliability and adjust timeouts
Increase timeouts for export and navigation tests to 60 seconds to
prevent flakiness. Remove brittle assertions checking for visible
overlays in setup helpers to ensure more stable test execution.
2026-02-16 01:50:43 -07:00
Richard R
fa406a8cb2 refactor(deps): remove ffprobe-static and fix playwright flakiness
Replace ffprobe-static with ffmpeg-static for metadata extraction in
migration scripts. This removes the requirement for ffprobe binaries
across Docker, CI, and Vercel environments, simplifying the dependency
graph.
2026-02-16 01:41:54 -07:00
Richard R
da0fce708f refactor: improve database reliability and clean up code
Enable WAL mode and set a busy timeout for SQLite to better handle
concurrent access and prevent 500 errors. Add error handling to
async calls in UI components and remove unused imports across several
files.
2026-02-15 15:44:20 -07:00
Richard R
4d13b5b821 refactor: improve async state and session reliability
Implement AbortController in ConfigContext, PDFContext, and preference sync to prevent race conditions during session changes or rapid navigation. Refactor document preview routes into shared utilities and add LRU caching for text previews to optimize memory. Update documentation for Better Auth schema ownership and migration workflows. Fix issues with Postgres system user seeding and better-sqlite3 configuration.
2026-02-15 15:13:25 -07:00
Richard R
8e4f0b0ca9 build(config): remove better-sqlite3 from serverExternalPackages 2026-02-15 14:02:42 -07:00
Richard R
8cb2929116 feat(db): add system user seeding and storage cleanup
- Implement ensureSystemUserExists to handle foreign key constraints for
  unclaimed data when authentication is disabled.
- Add missing foreign key relationship between audiobook chapters and
  audiobooks with cascade delete.
- Integrate a beforeDelete hook in auth config to trigger storage
  cleanup when a user is removed.
- Regenerate initial migrations for both PostgreSQL and SQLite to
  reflect schema updates.
2026-02-15 14:00:35 -07:00
Richard R
d3c7831f70 fix: address hook violations and improve resource reliability
- Refactor useAuthSession to call useSession unconditionally, adhering to React Rules of Hooks by using a stub client when auth is disabled.
- Update migration script to insert a system user to satisfy foreign key constraints during database transitions.
- Implement AbortController and timeouts in entrypoint endpoint polling to prevent hanging fetch requests.
- Use useMemo in PDFViewer to provide a stable file object, preventing unnecessary re-renders and warnings.
- Enhance state cleanup in EPUBContext and AuthLoader to handle missing metadata or disallowed sessions gracefully.
2026-02-15 13:31:06 -07:00
Richard R
3eebfe6c6c ci(config): remove vercel configuration file 2026-02-15 13:06:34 -07:00
Richard R
bedbbb026a feat(db): add foreign keys and cascade deletes
Enforce referential integrity by linking application tables to the auth user table across both PostgreSQL and SQLite schemas.

- Add foreign key references with ON DELETE CASCADE to all user-owned tables.
- Re-generate initial migrations for Postgres and SQLite providers.
- Update migration script to ensure the SQLite database directory exists.
- Remove explicit SSL configuration from the PostgreSQL connection pool.
2026-02-15 13:02:42 -07:00
Richard R
bb853acaf6 Refactor authentication schema and database handling
- Updated SQLite journal metadata with new tag and timestamp.
- Refactored account deletion API to utilize Better Auth's built-in user deletion method for cascading cleanup.
- Enhanced database initialization to include authentication schemas for both SQLite and Postgres.
- Removed user, session, account, and verification tables from the Drizzle schema, as they are now managed by Better Auth.
- Created separate schema files for authentication in both Postgres and SQLite.
- Added SQL migration scripts for creating necessary tables in both database systems.
2026-02-15 12:49:23 -07:00
Richard R
f9366dbfa2 refactor(audio): replace ffprobe with ffmpeg for metadata extraction
- Remove ffprobe-static dependency and associated configuration
- Reimplement ffprobeAudio to parse duration and title tags from ffmpeg output
- Update CI workflow to install SeaweedFS binary and set required environment variables
- Enhance E2E test setup to handle auth-protected endpoints during document cleanup
2026-02-15 12:13:29 -07:00
Richard R
260513ebaa ci(config): configure CI environment and optimize function memory
- Set FFMPEG_BIN and FFPROBE_BIN environment variables in Playwright workflow
- Reduce memory allocation for audiobook and whisper API routes to 1024MB
2026-02-15 11:33:46 -07:00
Richard R
1f4794a706 feat(auth): implement session gatekeeping and architectural reorganization
- Introduce `USE_ANONYMOUS_AUTH_SESSIONS` to make guest access an opt-in feature.
- Transition root-level pages into `(app)` and `(public)` route groups for improved access control.
- Adopt Figtree as the primary typeface and implement a pre-render theme initialization script.
- Decouple audiobook chapter processing into a standalone API route and harden resource ownership logic.
- Replace the legacy footer with a unified layout and add skeleton loading states for document lists.
- Update environment documentation and test suites to align with the revised routing and auth flows.

BREAKING CHANGE: The audiobook generation API has been restructured, moving specific chapter logic to `/api/audiobook/chapter`.
2026-02-15 11:12:42 -07:00
Richard R
9f25e05cce feat(previews): implement document preview generation and caching
- Add document preview caching logic with in-memory and persisted storage.
- Implement S3 blobstore functions for managing document previews.
- Create rendering functions for PDF and EPUB cover images to JPEG format.
- Introduce database schema and functions for managing document preview metadata.
- Add unit tests for rendering PDF and EPUB previews.
2026-02-12 16:05:03 -07:00
Richard R
82f82a990e feat: add user preferences and document progress syncing
- Implemented user preferences management with a new API for GET and PUT requests.
- Added user document progress tracking with a new API for retrieving and updating progress.
- Introduced database schema changes for user preferences and document progress.
- Enhanced EPUB and TTS contexts to support syncing user preferences and document progress.
- Added functions to handle transferring user preferences and progress during account linking.
- Updated client-side logic to schedule syncing of user preferences and document progress.
2026-02-11 13:58:16 -07:00
Richard R
852092c558 docs: reorganize documentation structure and fix word highlight behavior 2026-02-11 06:03:51 -07:00
Richard R
63483e055e refactor(docs): reorganize and enhance documentation structure
- Rename configuration.md to auth.md for clarity
- Rename storage-and-blob-behavior.md to object-blob-storage.md
- Split server library import into dedicated documentation page
- Rename intro.md to introduction.md
- Merge support.md into support-and-contributing.md
- Add interactive tabs to configuration and setup guides
- Improve cross-references and navigation between documentation pages
- Update sidebar structure and Docusaurus config for new file paths
2026-02-11 04:16:02 -07:00
Richard R
4e2d18962d fix(config): update ffmpeg tracing for audiobook APIs
Configure Next.js output tracing to properly include ffmpeg and ffprobe
static binaries for audiobook processing endpoints. This ensures the
required binaries are bundled correctly during deployment across all
audiobook API routes.
2026-02-11 03:20:22 -07:00
Richard R
9b9206f50d refactor(audiobooks): migrate audiobook pipeline to object storage
This commit restructures the audiobook generation and serving layer to rely exclusively on S3-compatible blob storage, removing the dependency on local filesystem paths.

- Bundle `ffmpeg` and `ffprobe` binaries via npm to ensure portability across environments.
- Update API routes to stream audio directly from blob storage instead of local disk.
- Remove legacy migration endpoints and filesystem-based indexing logic.
- Add startup scripts to facilitate the transition from local to remote storage.

BREAKING CHANGE: Audiobook functionality is now strictly dependent on S3 configuration. The previous filesystem-based storage method has been removed.
2026-02-11 02:44:34 -07:00
Richard R
91bcf232e1 fix(docs): simplify docs versioning workflow 2026-02-10 15:15:44 -07:00
Richard R
6d5fb65a72 feat(docs): add configurable rate limiting and external docs
Add environment variables for fine-grained control over TTS rate limiting
and Better Auth behavior. Move documentation to external Docusaurus site
with automated deployment workflows.

- TTS rate limiting can now be enabled/disabled via TTS_ENABLE_RATE_LIMIT
- Customizable daily limits for anonymous/authenticated users and IP backstops
- Better Auth rate limiting can be disabled via DISABLE_AUTH_RATE_LIMIT
- Rename library import env vars to IMPORT_LIBRARY_DIRS/DIR
- Add docs-site with Docusaurus and GitHub Actions workflows
- Update README to reference external documentation
2026-02-10 15:11:37 -07:00
Richard R
81d249ed52 feat(storage): implement S3/SeaweedFS blob storage for documents
Replaces local filesystem document storage with S3-compatible object storage.
Adds embedded SeaweedFS 'weed mini' for local development and Docker deployments.
Updates document upload flow to use presigned URLs with a server fallback proxy.
Refactors auth configuration to use BASE_URL and AUTH_SECRET.

BREAKING CHANGE: Renamed BETTER_AUTH_URL to BASE_URL and BETTER_AUTH_SECRET to AUTH_SECRET. Removed legacy document upload/content endpoints. Requires S3 environment variables (auto-configured for embedded SeaweedFS).
2026-02-10 12:29:15 -07:00
Richard R
4a5f3060f2 refactor(documents): migrate from IndexedDB to server-backed storage with caching
- Replace client-side IndexedDB with server-first document storage
- Add document caching layer for offline capability and performance
- Implement document transfer during anonymous-to-authenticated account linking
- Add database indexes for improved query performance
- Update document contexts to use new caching system
- Refactor document upload and deletion APIs
- Add audiobook pruning for missing files
- Improve test isolation with namespace support
- Add unit tests for document cache and transfer functions
2026-02-08 11:15:57 -07:00
Richard R
2c23dc9043 refactor(db): remove conditional database usage from API layer
Ensure all metadata operations consistently use the database layer,
removing filesystem-only fallback paths and conditional DB checks.

- Simplified migration scripts to always run on startup
- Updated document/audiobook APIs to always query DB
- Added ensureDbIndexed() calls across all routes
- Extracted test namespace utilities to dedicated module
- Removed migration-manager.ts (functionality consolidated)
- Updated rate limiter to assume DB is always available

BREAKING CHANGE: Database is now required in all configurations.
When auth is disabled, SQLite is used by default at /app/docstore/sqlite3.db.
2026-02-03 12:17:30 -07:00
Richard R
c24710b2ca refactor(audiobooks): implement user-specific storage with composite primary keys
- Change audiobooks and audiobookChapters tables to use composite PK (id, userId)
- Migrate audiobooks from flat storage to user-specific directories under audiobooks_users
- Add support for claiming unclaimed audiobooks on account creation
- Improve auth rate limiting with retry logic and DISABLE_AUTH_RATE_LIMIT for tests
- Fix iOS/Safari audio playback with unlock mechanism and playback rate watchdog
- Update API routes to handle user-scoped audiobook access with fallback to unclaimed
- Transfer audiobooks when linking anonymous accounts to real accounts
- Remove foreign key constraint from audiobookChapters to support composite PK
- Add cascade delete to account and session foreign keys
2026-02-03 12:17:06 -07:00
Richard R
30ce65e747 refactor(auth): bootstrap anonymous sessions via AuthLoader
Replace privacy popup with PrivacyModal and reorganize auth UI components
Remove signed-out persistence and centralize session handling in useAuthSession
Move document viewers and Kokoro voice helpers into dedicated modules
2026-01-28 02:39:44 -07:00
Richard R
20111b1e36 fix(tests): try to get tests working again 2026-01-28 01:04:01 -07:00
Richard R
6089d2c445 chore(db): move drizzle scripts and update commands
Relocate migration scripts to `drizzle/scripts`, update `start` and
`migrate`
to use the new path, and add a `generate` script for contributors.
Update README migration guidance for SQLite vs Postgres workflows.

BREAKING CHANGE: remove `migrate:force` and change migration script
paths; consumers relying on `scripts/migrate*.mjs` or `pnpm
migrate:force` must update to the new commands.
2026-01-27 17:46:29 -07:00
Richard R
297752e83e fix(db): select drizzle config for migrations
Add dedicated Drizzle config files for sqlite and postgres and generate
new migration outputs for each dialect.

Update migrate scripts to load env files and auto-pick the correct
config unless explicitly provided, and tighten API auth handling by
using shared auth context helpers and enforcing userId checks when auth
is enabled.
2026-01-27 16:53:58 -07:00
Richard R
8189087ad9 fix: reduce pdf flicker and harden tts
- Smooth react-pdf page turns with stable canvas staging and fade-in
- Retry transient empty page text extraction during fast page turns
- Avoid noisy TTS preload toasts, guard requests when quota is exhausted,
  and improve Howler retry/unload behavior to prevent pool exhaustion
- Enforce auth FK cascade deletes and tighten rate-limiter bucket updates
  using affected-row checks
2026-01-26 17:01:51 -07:00
Richard R
c57b913cb5 feat(db/auth): migrate auth storage to Drizzle and add claimable data flow
- replace custom DB adapter with Drizzle setup (SQLite/Postgres schemas) and add `drizzle.config.ts` plus migrations in `drizzle/` and `drizzle_pg/`
- switch better-auth to drizzleAdapter, add auth helpers (`getAuthContext`, `requireAuthContext`, `requireAudiobookOwned`), and make `useAuth`/`useAuthSession` safe no-ops when auth is disabled
- persist documents/audiobooks in DB with ownership checks, unclaimed fallback, and ref-counted deletes; add FS scan helper for no-auth mode
- gate docx-to-pdf, library, voices, whisper, and migration endpoints behind auth when enabled
- add unclaimed data scan/claim flow with `/api/user/claim`, `ClaimDataModal`, and server-side scan/claim helpers
- refactor rate limiting and account deletion to use Drizzle tables (`user_tts_chars`, `user`)
- run migrations via `scripts/migrate-if-auth.mjs` (auto on `pnpm start` + `pnpm migrate`), remove Docker entrypoint and old better-auth migration file
- update README and lockfile for the new migration workflow and dependencies
2026-01-26 17:01:36 -07:00
Richard R
9561d5a816 fix(ui): adjust modal positioning for responsive design
Update modal components to start from top on mobile devices and center on larger screens, improving usability across screen sizes. Also refine transition handling in privacy popup for cleaner unmounting.
2026-01-25 15:13:02 -07:00
Richard R
fb8fc41994 fix(ui): adjust dropdown positioning and scrolling in voices control and improve the ui label for speed controls 2026-01-25 14:55:45 -07:00
Richard R
c345628200 fix: remove unused imports and fix privacy popup text
- Remove unused SettingsIcon and UserMenu imports from epub and pdf pages
- Remove unused UserMenu import from home page
- Remove unused isDev const from home page
- Update SettingsModal to only import clearSignedOut
- Fix apostrophe encoding in privacy popup description
2026-01-25 14:41:49 -07:00
Richard R
50c3829f98 update(auth): enhance rate limiting with device/IP backstops and improve UI
- Add device ID and IP-based rate limiting to prevent abuse
- Refactor UI components for better header menus and settings
- Update privacy popup with detailed data usage info
- Improve PDF handling and caching to prevent react-pdf warnings
- Update README with new Docker instructions and environment variables
2026-01-25 14:38:31 -07:00
Richard R
4689f3676f refactor(auth): consolidate authentication contexts and enhance rate limit UI integration
Streamline auth client factory and remove redundant session manager
Introduce unified auto rate limit context for seamless status updates
Implement problem details in TTS responses for improved error clarity
Add rate limit pause controls and banners across document viewers
Refine PDF scaling logic with height-aware resize handling
Update privacy popup triggers and first-visit modal behavior

BREAKING CHANGE: Daily character limits adjusted (anonymous: 250K → 50K, authenticated: 1M → 500K)
2026-01-25 11:12:19 -07:00
Richard R
9ba20c8a9e feat(auth): add user authentication and rate limiting
- Implement user sign-in, sign-up, and account management using better-auth
- Add rate limiting for TTS API with daily character limits for authenticated and anonymous users
- Integrate SQLite and PostgreSQL database support for user sessions and data persistence
- Update UI components to include authentication flows, user menu, and privacy popup
- Modify Dockerfile and package.json for new dependencies and entrypoint script
- Bump version to v1.3.0
2026-01-24 17:36:11 -07:00
Richard R
64825b626a chore: Update various dependencies and introduce pnpm overrides for specific packages. 2026-01-24 12:33:48 -07:00
Richard R
56004877c0
Merge pull request #75 from richardr1126/v1.2.1
Some checks failed
Create and publish Docker images / prepare (push) Has been cancelled
Create and publish Docker images / build (amd64, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Create and publish Docker images / build (arm64, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Create and publish Docker images / merge (push) Has been cancelled
* **Bug Fixes**
  * More reliable PDF sentence and word highlighting during layout changes and rapid updates, preventing stale or overlapping highlights.

* **Enhancements**
  * Revamped text normalization and TTS block splitting for better handling of long/complex content, EPUB texts, dialogue/quotes, and PDF artifacts.

* **Tests**
  * Expanded unit tests covering block-splitting, EPUB behavior, oversized inputs, and boundary cases.

* **Chores**
  * Package version bumped; test runner config updated to ignore unit tests for select browser projects.
2026-01-21 14:45:15 -07:00
Richard R
ddf8912092 chore(config): ignore unit tests in firefox and webkit browsers 2026-01-21 14:42:13 -07:00