Introduce configurable segment preloading depth, sentence lookahead, and TTS segment max block length for both PDF and EPUB readers. Add new settings to user preferences, config context, and document settings UI. Refactor TTS segment splitting logic to support per-user max block length and propagate these options through PDF/EPUB adapters and TTS segment generation. Update API, manifest pagination, and segment deduplication for improved performance and correctness. Add tests for new NLP options and manifest logic.
BREAKING CHANGE: TTS segment splitting and manifest APIs now require max block length and preloading parameters; user config schema updated.
- Add logic to split oversized sentences in `splitTextToTtsBlocksEPUB` using `splitOversizedText` to ensure blocks stay within `MAX_BLOCK_LENGTH`
- Remove `extractRawSentences` and `processTextWithMapping` functions as they are no longer needed
- Update tests to reflect the changes, including a new test for oversized sentence splitting and removal of related test suites
Introduce `splitTextToTtsBlocksEPUB` function to handle EPUB text splitting,
treating single newlines as paragraph boundaries for precise highlighting.
Update TTS context to conditionally use EPUB splitting based on document type.
Enhance PDFViewer to clear highlights when current sentence is null.
Add comprehensive tests for the new functionality and refactor existing ones.
- Rename and enhance text processing functions in nlp.ts for better handling of oversized texts, sentence boundaries, and PDF artifacts
- Update PDFViewer to add layout-aware highlighting with retry logic for sentence and word highlights
- Adjust PDFContext and TTSContext to use new normalized text functions
- Expand unit tests for new splitting behaviors, including long texts and punctuation preferences
