Commit graph

104 commits

Author SHA1 Message Date
Richard Roberson
7046666b3f fix(tts): ensure robust caching for audio and alignments
- New buildCacheKey function creates unique identifiers for TTS cache entries.
- Cache keys now include sentence, voice, speed, provider, and model parameters.
- Prevents serving cached audio or alignment data that mismatches current TTS parameters.
- Removes redundant audioCache.clear() calls when voice or speed change, as entries are now distinct.
2025-11-22 01:57:27 -07:00
Richard Roberson
b576910523 refactor(client): centralize client-side API calls and refine types
Abstracted direct fetch calls across components and contexts into new functions within `src/lib/client.ts`. This provides a consistent and centralized interface for interacting with backend APIs.

- Introduced `src/lib/client.ts` to encapsulate API request logic.
- Standardized audio buffer types (`TTSAudioBuffer`, `TTSAudioBytes`) in `src/types/tts.ts`.
- Moved client-specific request types (`TTSRequestPayload`, `TTSRequestHeaders`, `TTSRetryOptions`) to `src/types/client.ts`.
- Updated API routes and consumer components/contexts to leverage the new client library functions and type definitions.
- Removed `src/utils/audio.ts` as its utility functions are now part of `src/lib/client.ts`.
2025-11-21 23:33:41 -07:00
Richard Roberson
7a29f73d07 refactor(api): move audiobook endpoints to new path
- Relocated all audiobook-related API routes from `/api/audio/convert/*` to `/api/audiobook/*`.
- This change affects endpoints for chapter conversion, retrieval, deletion, and overall audiobook status.
- Updated client-side calls in `AudiobookExportModal.tsx`, `EPUBContext.tsx`, and `PDFContext.tsx` to reflect the new API paths.
- Modified API tests (`api.spec.ts`, `export.spec.ts`) to target the restructured endpoints.
- The new API structure provides better organization and a clearer, more consistent interface for audiobook functionality.
2025-11-21 19:29:49 -07:00
Richard Roberson
733b4180eb feat(tts): add word-by-word highlighting with whisper.cpp
- Introduce `/api/whisper` endpoint which uses `whisper.cpp` (via a `WHISPER_CPP_BIN` executable) and `ffmpeg` to generate word-level audio alignments from provided audio and text.
- Integrate word-level alignments into `TTSContext`, tracking the currently spoken word based on audio seek position and provided timestamps. Alignments are cached in-memory and fetched asynchronously.
- Add new configuration options (`pdfWordHighlightEnabled`, `epubWordHighlightEnabled`) to `ConfigContext` and `Dexie` for enabling/disabling the feature.
- Implement visual word highlighting in both `PDFViewer` and `EPUBViewer` by mapping TTS-aligned words to rendered text elements.
- Enhance `EPUBContext` and `PDFContext` with new `highlightWordIndex` and `clearWordHighlights` functions, utilizing fuzzy string matching (`cmpstr`) to robustly align spoken words with displayed text for accurate highlighting.
- Update `DocumentSettings` to include user-facing toggles for the new highlighting modes.
2025-11-21 19:22:54 -07:00
Richard Roberson
50538bfaba feat(epub): Add EPUB text highlighting for the current TTS sentence with a new configuration option 2025-11-18 12:31:07 -07:00
Richard Roberson
ff150eec01 refactor(audiobook): standardize chapter regeneration APIs and UX
- Removed `onProgress` callback from `regenerateChapter` and related functions across `TTS`, `EPUB`, and `PDF` contexts to simplify the chapter regeneration API.
- Updated `AudiobookExportModal` to align with the refined regeneration API, including removing granular chapter progress display and adding a hint about TTS caching behavior.
- Introduced `TTSAudiobookChapter` interface and renamed `ContinuationMergeResult` to `TTSSmartMergeResult` and `PageTurnEstimate` to `TTSPageTurnEstimate` for better type consistency and clarity.
- Applied minor styling adjustments to buttons and listbox components in modals for visual consistency.
- Added `'use client'` directive to several client-side components for Next.js 13+ compatibility.
- Updated Dockerfile build command from `pnpm run build` to `pnpm build`.
- Added Playwright tests to verify backend chapter state after regeneration.
2025-11-16 21:10:32 -07:00
Richard Roberson
04def62ff5 refactor(core): standardize module structure and enhance TTS types
- Reorganized utility modules from `src/utils` to `src/lib` for clearer separation of concerns.
- Introduced new, dedicated type definitions in `src/types` for improved type safety in configuration and TTS API interactions.
  - Replaced `src/types/appConfig.ts` with `src/types/config.ts`.
  - Added `src/types/tts.ts` for TTS request payloads, error structures, and retry options.
- Updated module imports across several contexts (`Config`, `EPUB`, `HTML`, `PDF`, `TTS`) and components to reflect the new `lib` and `types` locations.
- Enhanced TTS API request and error handling in `src/app/api/tts/route.ts` and TTS-consuming contexts with explicit types.
- Simplified `ProgressCard`, `ProgressPopup`, and `AudiobookExportModal` components by removing the `isProcessing` prop, centralizing processing state management.
- Streamlined `HTMLContext` by removing `createFullAudioBook` and `isAudioCombining` properties, focusing its scope.
2025-11-16 15:12:35 -07:00
Richard Roberson
7271afa2f5 refactor(config): centralize app config in Dexie
Migrates scattered configuration items and last document locations
from the legacy `config` table to new, dedicated Dexie stores.

- Introduces a singleton `app-config` table (`AppConfigRow`) to standardize
  and structure all application settings in a single object.
- Creates a `last-locations` table for efficient storage of document read positions.
- Updates `ConfigContext` to utilize `dexie-react-hooks`' `useLiveQuery`
  for reactive and simplified state management.
- Implements a database upgrade path (from DB_VERSION 4 to 5) to migrate
  all existing user settings seamlessly.
- Simplifies config access and updates with new `getAppConfig` and `updateAppConfig`
  utility functions.
- Removes the deprecated `config` table after successful migration.
- Updates consumers like `SettingsModal` and test helpers to align with
  the new config structure.
2025-11-15 17:29:56 -07:00
Richard Roberson
071d967f8f feat(pdf): modernize text highlighting and parsing for fluidity
- Offload computationally intensive text matching for real-time highlighting to a dedicated Web Worker, ensuring the main thread remains responsive during playback.
- Implement a new overlay-based highlighting system that renders independently of the PDF's text layer, providing smoother and more reliable visual feedback without interfering with document rendering.
- Introduce a new setting allowing users to enable or disable real-time text highlighting in PDFs, offering personalized control over the reading interface.
- Upgrade the underlying text comparison algorithm to Dice similarity for more accurate and context-aware matching of spoken words to on-screen text, improving synchronization precision.
- Improve sentence boundary detection, especially for quoted dialogue and complex structures, by enhancing the NLP processing logic, leading to a more natural audio-text flow.
2025-11-15 14:06:44 -07:00
Richard Roberson
5316596a1b refactor(db): migrate to Dexie with reactive queries and simplify data layer
Replaces custom IndexedDB implementation with Dexie ORM, eliminating 850+ lines of
boilerplate code and introducing reactive live queries across all document types.
Transforms document management from imperative refresh patterns to automatic
reactive updates using dexie-react-hooks.

Simplifies TTS backend by removing concurrency semaphore while maintaining
request de-duplication through in-flight tracking. Streamlines document hooks
by removing manual state management and refresh methods.

Updates package dependencies and type definitions to support new database
architecture while maintaining full backward compatibility for existing
documents and settings.

BREAKING CHANGE: Document hooks no longer expose refresh() methods as updates
are now reactive through live queries.
2025-11-14 16:14:00 -07:00
Richard Roberson
e3799e40aa feat(tts): add smart sentence continuation
introduce configurable smart sentence splitting with persisted state,
UI toggle, and EPUB/PDF contexts that send continuation metadata.
enhance TTS pipeline to merge cross-page sentences, manage carryover,
and trigger visual page changes during playback for smoother narration.

update README with kokoro quick-start guidance, remove the legacy
issues mapping doc, and add deterministic TTS mocks plus sample audio
for Playwright tests.
2025-11-14 12:00:03 -07:00
Richard Roberson
1dcffc82d8 feat(tts,ui,api): add Kokoro multi-voice selection and SDK support
- Introduce voice utils (model detection, voice parsing/weights, limits)
- Enable Kokoro multi-voice strings across OpenAI/Deepinfra/custom providers
- Normalize non-Kokoro voices to single token for SDK calls
- Expose full Kokoro voice list for custom-openai Kokoro models
- Update TTS API to return audio as ArrayBuffer and improve logging
- Add multi-select UI for Kokoro voices with provider-based clamping
- Preserve Kokoro voice strings in TTSContext and coalesce restarts
- Merge multi-sentence quoted dialogue in NLP sentence splitter
- Update tests for single and multi-voice selection flows
2025-11-12 20:46:45 -07:00
Richard Roberson
42665884d7 feat(audiobook): add chapter-based export with UI and API
Introduce end-to-end chapterized audiobook generation with persistent
storage, resumable workflows, and MP3/M4B support.

API:
- add /api/audio/convert/chapter (GET/DELETE) for per-chapter ops
- add /api/audio/convert/chapters (GET/DELETE) for listing/reset
- enhance /api/audio/convert:
  - accept mp3|m4b, stream combined file, cache complete output
  - robust chapter indexing, docstore persistence, list concat
  - AbortSignal-aware ffmpeg/ffprobe, 499 on cancel

UI/UX:
- add AudiobookExportModal with progress, resume, regenerate, download
- add ProgressCard, enhance ProgressPopup (click-to-focus, richer info)
- add Header, ZoomControl; move TTSPlayer to sticky bottom bar
- redesign EPUB/HTML/PDF pages for full-height layout and controls
- add HomeContent; compact uploader variant; document list polish

TTS/Contexts:
- EPUB/PDF contexts now generate per-chapter to disk, return bookId
- support chapter regeneration; progress/cancel propagation
- pass provider/model/instructions; standardize MP3 TTS output
- HTML context updates for model/instructions

Styling:
- globals: overlay-dim, scrollbar styles, prism gradient utilities
- theme vars: secondary-accent, prism-gradient; tailwind color addition

Misc:
- fix PDF scale calc to use container height
- EPUB theme reader area fills height
- time estimation update cadence stab
- audio util passes format to combine endpoint
2025-11-11 13:32:30 -07:00
Richard Roberson
a54d83eda5 fix(settings): set default TTS provider to Deepinfra in production 2025-11-09 17:05:23 -07:00
Richard Roberson
1614bf46bc fix(tts): add missing ttsProvider dependency to useCallback hooks
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
2025-11-09 16:43:13 -07:00
Richard Roberson
b21bdb3a21 feat(tts): add multi-provider TTS support with Deepinfra and custom OpenAI-compatible endpoints
Add comprehensive multi-provider TTS support enabling users to choose between OpenAI, Deepinfra, and custom OpenAI-compatible endpoints. Implement provider-specific voice management with automatic voice restoration per provider-model combination, and migrate package manager to pnpm for improved dependency handling.

Key changes:
- Add TTS provider selection (OpenAI, Deepinfra, custom-openai) in settings UI
- Implement provider-specific model and voice lists with dynamic fetching
- Add voice persistence per provider-model combination in savedVoices
- Support Deepinfra models: Kokoro-82M, Orpheus-3B, Sesame-1B with their voice libraries
- Migrate to pnpm with frozen lockfile for reproducible builds
- Update Docker configuration to use pnpm and Deepinfra API defaults
- Add migration logic for existing users to infer provider from stored baseUrl
- Update test helpers and Playwright configuration for Deepinfra API
- Add example docker-compose.yml with Kokoro-FastAPI integration

BREAKING CHANGE: Voice selection is now provider-model specific. Previously saved voices will be migrated to the new savedVoices structure, but users may need to reselect voices if switching providers.
2025-11-09 15:47:17 -07:00
Richard Roberson
08480f07ed
Merge pull request #52 from thepycoder/feature/better-speed-control 2025-06-29 15:09:05 -06:00
Victor Sonck
89833d596b Allow for player speed changes as well as native voice speed changes. Kokoro for example swallows parts of words when asked to go faster than maybe 1.3x. This fixes that, by simply speeding up the player instead. Since we're already using the html5 version of howler, by default it will pitch-correct the audio and it sounds quite good! 2025-06-23 14:09:51 +02:00
Victor Sonck
ece8bcd35a Added saving and restoring last location in PDFs 2025-06-23 13:57:27 +02:00
Victor Sonck
f594ca836b Refactored NLP into a single, shared function for splitting to sentences.
now both handleClick and global sentences are the exact same, so the sentenceIndex will always be correct.
2025-06-20 08:25:29 +02:00
Richard Roberson
82bb91a850 Add TXT and MD support with and HTML viewer with react markdown + many refactors to support it 2025-04-15 02:28:51 -06:00
Richard Roberson
fc780746aa Bug fixes and readme update 2025-03-30 21:17:16 -06:00
Richard Roberson
7ba807fd95 Pause on page changes + Fix ePub bug when reloading 2025-03-23 17:18:21 -06:00
Richard Roberson
4ed6b17e98 Support gpt-4o-mini-tts with instructions
Some checks failed
Build and Publish Docker Image / build (push) Has been cancelled
2025-03-21 11:32:42 -06:00
Richard Roberson
f464b059e4 Update API settings 2025-03-13 21:00:34 -06:00
Richard Roberson
ce9df9a5ab implement retry logic for TTS API calls and clean up code 2025-03-03 23:03:52 -07:00
Richard Roberson
54dea5da36 Add onloaderror retries 2025-03-02 22:13:08 -07:00
Richard Roberson
4e07e737f0 Stop using audio blobs 2025-03-02 21:16:24 -07:00
Richard Roberson
136fdb9652 Fix Apple WebKit PDF Upload 2025-03-01 23:36:25 -07:00
Richard Roberson
b89d1700e9 Fix for longer audiobooks 2025-03-01 18:21:51 -07:00
Richard Roberson
e3370f7526 Further audiobook extraction optimizations 2025-02-26 04:03:05 -07:00
Richard Roberson
b67aea6b27 Refactor audio conversion API to improve memory management and add streaming response 2025-02-25 23:38:29 -07:00
Richard Roberson
4a2d75e155 Export to m4b from WAV efficiently 2025-02-25 20:01:15 -07:00
Richard Roberson
79d2e7f613 Working m4b 2025-02-25 19:47:30 -07:00
Richard Roberson
9357dddf3a Implement PDF audiobook generation with configurable margins for text extraction 2025-02-25 16:11:08 -07:00
Richard Roberson
92220de5ea Refactor PDF text extraction to support configurable margins for header, footer, left, and right 2025-02-25 15:53:45 -07:00
Richard Roberson
f948601e70 Add configurable text extraction margin for PDF processing 2025-02-25 03:26:37 -07:00
Richard Roberson
b87b83310b Rely on TTS API for splitting 2025-02-25 00:58:32 -07:00
Richard Roberson
1db906f8bc [ePub only] First working export audiobook 2025-02-24 22:41:38 -07:00
Richard Roberson
96376e2324 Environment changes, set in Docker run 2025-02-23 13:20:42 -07:00
Richard Roberson
9ad1f5f117 Pause in background 2025-02-23 04:46:20 -07:00
Richard Roberson
0be34a09fe Re-add AbortController API, fixes duplicate playback 2025-02-23 01:29:10 -07:00
Richard Roberson
0251aae04b Huge iOS playback fix 2025-02-22 13:36:45 -07:00
Richard Roberson
99868f29b2 EPUB resize debounce working perfectly 2025-02-21 12:01:52 -07:00
Richard Roberson
9d6632d2d0 Use server instead of client for open ai request + large refactors and bug fixes 2025-02-21 11:24:07 -07:00
Richard Roberson
86be935737 Fully unified navigation system 2025-02-19 15:19:37 -07:00
Richard Roberson
d447132859 Refactors to fix blank sections 2025-02-19 15:10:11 -07:00
Richard Roberson
8acf3f6ba6 Fix initial system theme 2025-02-19 12:32:16 -07:00
Richard Roberson
0ce14f3aec Refactors + epub location fixes 2025-02-18 02:31:30 -07:00
Richard Roberson
9ae6dd9e82 Fix epub use theme and saved location 2025-02-17 23:03:28 -07:00