Commit graph

54 commits

Author SHA1 Message Date
Richard Roberson
733b4180eb feat(tts): add word-by-word highlighting with whisper.cpp
- Introduce `/api/whisper` endpoint which uses `whisper.cpp` (via a `WHISPER_CPP_BIN` executable) and `ffmpeg` to generate word-level audio alignments from provided audio and text.
- Integrate word-level alignments into `TTSContext`, tracking the currently spoken word based on audio seek position and provided timestamps. Alignments are cached in-memory and fetched asynchronously.
- Add new configuration options (`pdfWordHighlightEnabled`, `epubWordHighlightEnabled`) to `ConfigContext` and `Dexie` for enabling/disabling the feature.
- Implement visual word highlighting in both `PDFViewer` and `EPUBViewer` by mapping TTS-aligned words to rendered text elements.
- Enhance `EPUBContext` and `PDFContext` with new `highlightWordIndex` and `clearWordHighlights` functions, utilizing fuzzy string matching (`cmpstr`) to robustly align spoken words with displayed text for accurate highlighting.
- Update `DocumentSettings` to include user-facing toggles for the new highlighting modes.
2025-11-21 19:22:54 -07:00
Richard Roberson
04def62ff5 refactor(core): standardize module structure and enhance TTS types
- Reorganized utility modules from `src/utils` to `src/lib` for clearer separation of concerns.
- Introduced new, dedicated type definitions in `src/types` for improved type safety in configuration and TTS API interactions.
  - Replaced `src/types/appConfig.ts` with `src/types/config.ts`.
  - Added `src/types/tts.ts` for TTS request payloads, error structures, and retry options.
- Updated module imports across several contexts (`Config`, `EPUB`, `HTML`, `PDF`, `TTS`) and components to reflect the new `lib` and `types` locations.
- Enhanced TTS API request and error handling in `src/app/api/tts/route.ts` and TTS-consuming contexts with explicit types.
- Simplified `ProgressCard`, `ProgressPopup`, and `AudiobookExportModal` components by removing the `isProcessing` prop, centralizing processing state management.
- Streamlined `HTMLContext` by removing `createFullAudioBook` and `isAudioCombining` properties, focusing its scope.
2025-11-16 15:12:35 -07:00
Richard Roberson
728eb185b7 Rename /context --> /contexts 2025-01-19 14:45:32 -07:00
Richard Roberson
3e82def76c Add theme switching + fixes 2025-01-19 03:44:04 -07:00