Commit graph

24 commits

Author SHA1 Message Date
Richard R
3da13c2970 chore(logging): remove redundant errorCode fields from server logs and error contracts
Eliminate the errorCode property from server logger calls, error contract types, and API error bodies throughout the codebase. This reduces duplication and streamlines error reporting, relying on structured event names and error classes for identification. All affected API routes, server logic, and logging utilities have been updated for consistency. Documentation directory added for future reference.
2026-05-27 06:33:14 -06:00
Richard R
5de9069c20 refactor(logging): finalize phase 6 error contract and pretty output 2026-05-27 06:17:25 -06:00
Richard R
034c1f03a4 logging: complete phase 5 contract and enforce via eslint rules 2026-05-27 04:54:52 -06:00
Richard R
59d0101c0c feat(server): implement unified structured logging with pino and enforce usage
Introduce `serverLogger` utility based on pino for consistent, structured logging across all server and API modules. Replace direct console logging with `serverLogger` and add request-scoped logging helpers. Update environment variable handling, documentation, and deployment guides to reflect new logging configuration (`LOG_FORMAT`, `LOG_LEVEL`, `COMPUTE_LOG_LEVEL`). Enforce no-console in server code via ESLint and add pino/pino-pretty dependencies.

This change standardizes log output, improves observability, and prepares the codebase for ingestion by log platforms.
2026-05-27 04:20:17 -06:00
Richard R
766c04d08d refactor(pdf): implement ONNX-based Docling layout parsing and block-level TTS for PDFs
Migrate PDF parsing to use ONNX Docling layout model for structured block extraction, enabling improved TTS segmentation and chaptering. Add compute backend abstraction for heavy tasks (alignment, layout parsing) with configuration via `OPENREADER_COMPUTE_MODE`. Integrate block-level locators and document settings for per-document PDF parsing options. Update S3 storage for parsed PDF JSON, add migration and schema changes, and extend client and server APIs for parsed data and settings. Remove legacy word highlight flag in favor of compute capability detection.

- Add ONNX model download, local/none compute modes, and `onnxruntime-node` dependency
- Update PDF viewer and TTS pipeline to use parsed blocks and block-level locators
- Add document settings storage and APIs for per-document PDF options
- Update S3 storage layout for parsed PDF JSON
- Add admin/config/docs updates for new compute and parsing features
- Remove obsolete word highlight runtime flag and UI

BREAKING CHANGE: PDF parsing and word highlighting now require `OPENREADER_COMPUTE_MODE=local` and ONNX model; document settings and S3 layout updated; legacy word highlight flag removed.
2026-05-17 21:18:51 -06:00
Richard R
df841210a3 Remove dead segment audio presign route and legacy helpers 2026-05-14 08:01:53 -06:00
Richard R
513128b802 Stream fallback segment audio and support byte-range requests 2026-05-14 07:55:48 -06:00
Richard R
da629b3ecd Enable private caching headers for segment audio routes 2026-05-13 23:44:13 -06:00
Richard R
16bf52f889 Use direct signed segment audio URLs in ensure/manifest 2026-05-13 23:33:00 -06:00
Richard R
b4f4d43d6a feat(tts,config,types): migrate to providerRef/providerType model and add centralized TTS provider policy
Transition all TTS-related logic, types, and UI to use the new providerRef/providerType model in place of legacy ttsProvider fields. Introduce a centralized tts-provider-policy module to encapsulate provider/model capability checks, default value resolution, and compatibility logic. Update all API routes, contexts, hooks, components, and tests to use providerRef and providerType, ensuring consistent handling of built-in and shared TTS providers. Remove legacy defaultTtsModel config in favor of per-provider defaults and shared provider admin control. Add the showAllProviderModels runtime flag to restrict users to provider default models when desired.

BREAKING CHANGE: ttsProvider fields are replaced by providerRef/providerType throughout the codebase; defaultTtsModel config is removed in favor of per-provider defaults.
2026-05-13 10:11:05 -06:00
Richard R
8488ad37e1 refactor(tts,db,manifest): split tts_segments into normalized entry and variant tables
Redesign TTS segment storage by decomposing the `tts_segments` table into two normalized tables: `tts_segment_entries` for unique segment identity and locator projection, and `tts_segment_variants` for per-settings audio variants. Update schema, queries, and API routes to use the new structure, including manifest pagination and segment resolution logic. Refactor segment locator handling to use stable projections and manifest cursors. Migrate tests and data cleanup utilities to support the new model.

BREAKING CHANGE: Database schema for TTS segments is now split into entry and variant tables; all code and consumers must use the new structure. Persisted data and APIs relying on the old `tts_segments` table are incompatible.
2026-05-12 17:44:57 -06:00
Richard R
f9adba791d feat(tts,epub,db): enforce stable EPUB segment locators and add segmentKey identity
Implement strict validation for EPUB TTS segment locators, requiring stable spine coordinates (`spineHref`, `spineIndex`, `charOffset`) and rejecting legacy CFI-only locators. Introduce `segmentKey` as a normalized identity derived from segment text, used for robust merging and deduplication of synthesized and persisted segments. Update database schemas, API routes, and manifest logic to support the new locator format and segmentKey. Add helpers for locator resolution and segmentKey construction, with comprehensive unit tests for identity, normalization, and manifest grouping.

BREAKING CHANGE: Persisted EPUB TTS segment locators must now include stable spine coordinates; legacy CFI-only locators are no longer accepted. Consumers must handle the new `segmentKey` field for segment identity and merging.
2026-05-12 16:18:36 -06:00
Richard R
e8d4c11434 refactor(tts,epub,locator): introduce canonical segment keys and locator comparison utilities
Refactor TTS segment identification to support canonical segment keys, decoupling segment IDs from locator and index for more robust deduplication and handoff. Add `segmentKey` to segment types, manifest, and ensure route. Implement natural sorting for EPUB CFI locations and segment locators using new comparison utilities. Update sidebar and manifest helpers to use locator-aware sorting. Add foundational modules for EPUB word highlighting, TTS segment planning, and EPUB handoff logic. Expand types and tests to cover new canonical segment and locator behaviors.

BREAKING CHANGE: TTS segment IDs now support canonical segment keys; APIs and manifest consumers must handle `segmentKey` and updated locator comparison logic.
2026-05-11 05:17:41 -06:00
Richard R
265e1cb56a feat(tts,config,ui): add advanced segment preloading and block length controls
Introduce configurable segment preloading depth, sentence lookahead, and TTS segment max block length for both PDF and EPUB readers. Add new settings to user preferences, config context, and document settings UI. Refactor TTS segment splitting logic to support per-user max block length and propagate these options through PDF/EPUB adapters and TTS segment generation. Update API, manifest pagination, and segment deduplication for improved performance and correctness. Add tests for new NLP options and manifest logic.

BREAKING CHANGE: TTS segment splitting and manifest APIs now require max block length and preloading parameters; user config schema updated.
2026-05-07 12:26:08 -06:00
Richard R
71e1472650 feat(reader,db): add segments sidebar and manifest API; update TTS segment schema
Introduce SegmentsSidebar component and supporting reader UI for segment-level
navigation and inspection. Add TTS segments manifest and clear API endpoints to
enable efficient retrieval and management of segment variants. Update TTS
segment schema to include settings_hash for improved lookup and indexing,
and adjust indexes to use settings_hash instead of settings_json. Update
contexts and types to support segment variant display and interaction.

This enhances document navigation and TTS segment management for end users.
2026-05-04 15:02:05 -06:00
Richard R
7d7e933323 refactor(tts): replace manifest and audio endpoints with presigned and fallback URLs
Remove legacy TTS segment manifest and audio endpoints in favor of new presigned and fallback audio URL approach. Update segment manifest structure to include `audioPresignUrl` and `audioFallbackUrl` fields, deprecating the previous `audioUrl`. Refactor segment ensure logic and TTS context to utilize the new playback source model. Add server utilities for generating presigned S3 URLs and introduce new API routes for presign and fallback audio retrieval. Clean up obsolete client API functions and types to reflect the new manifest format.
2026-05-04 12:04:24 -06:00
Richard R
9999cafec1 feat(tts): add TTS segments API, schema, storage, and manifest endpoints
Introduce TTS segment support with new database tables and migrations for both Postgres and SQLite. Implement segment ensure, audio, and manifest API endpoints, segment storage in S3, and segment authorization logic. Update TTS context and client API to use segment-based synthesis and retrieval. Add server-side helpers for segment ID, settings hash, locator normalization, and audio probing. Extend user data cleanup and teardown logic to remove TTS segment objects. Add unit tests for segment helpers.

BREAKING CHANGE: TTS synthesis now uses segment-based storage and APIs; database migrations required.
2026-05-04 10:18:56 -06:00
Richard R
0db53fbd4e refactor(tts): extract voice resolution logic to server module and update imports
Move resolveVoices and resolveReplicateVoiceInputKey to a new server-side
voice-resolution module to improve separation of concerns and reduce shared
bundle size. Update all imports and related tests to use the new module.
Remove unused LRUMap and related caches from shared catalog. Update Replicate
cooldown logic to use per-scope LRU cache for improved concurrency handling.
2026-04-16 15:46:55 -06:00
Richard R
3db3fd19ce refactor(tts): centralize upstream response helpers and improve settings validation
Move getUpstreamStatus and getUpstreamRetryAfterSeconds to a shared utility
module for consistent upstream error handling across TTS endpoints. Strengthen
audiobook chapter API by introducing runtime validation for incoming settings
payloads, ensuring type safety and error reporting for malformed requests.
Replace in-memory Map caches with LRUMap for Replicate voice and schema
lookups, improving memory management and eviction logic.
2026-04-16 13:07:31 -06:00
Richard R
893f74f038 refactor(tts): update Replicate model defaults and voice resolution
Align default Replicate model to the versioned Kokoro model across UI, docs,
and server logic. Refactor TTS settings merging for audiobooks to ensure
consistent normalization. Improve Retry-After header handling for upstream
rate limits. Expand Replicate voice resolution to use model schemas when
available, with test coverage for custom and built-in models.

Update documentation to reflect new Replicate defaults, model selection,
and configuration guidance.
2026-04-16 12:52:08 -06:00
Richard R
9da9232d39 feat: add support for Replicate TTS provider and models
- Updated environment variables documentation to include Replicate as a TTS provider option.
- Added Replicate to the sidebar for TTS provider guides.
- Included Replicate as a dependency in package.json and pnpm-lock.yaml.
- Enhanced audiobook chapter generation to normalize native speed settings based on the TTS provider.
- Improved error handling in TTS API routes to provide retry information for rate-limited responses.
- Updated AudiobookExportModal to reflect native speed support for Replicate models.
- Modified SettingsModal to set default model for Replicate.
- Enhanced SpeedControl component to conditionally render native speed controls based on provider support.
- Updated TTSContext to utilize effective native speed for TTS requests.
- Implemented Replicate request handling in the TTS generation logic.
- Added new documentation for configuring Replicate as a TTS provider.
2026-04-16 11:20:38 -06:00
Richard R
f9237f9270 feat(tts): add TTS_UPSTREAM_TIMEOUT_MS for configurable request timeout 2026-04-06 17:48:56 -06:00
Richard R
d6ae2baa6f refactor(tts,config,audiobook): centralize TTS provider catalog and extract audiobook pipeline
- Introduce a shared tts-provider-catalog to centralize provider/model/voice defaults
  and feature flags (supportsTtsInstructions, providerSupportsCustomModel, resolveProviderModels,
  getDefaultVoices, etc.). Replace ad-hoc provider logic across server and client with catalog calls.
- Replace inline voice/model UI logic in SettingsModal, AudiobookExportModal, TTSContext,
  generate server code, and voice hook to use catalog helpers (resolveTtsSettingsViewModel,
  supportsTtsInstructions, getDefaultVoices).
- Extract audiobook generation responsibilities into a small client-side pipeline:
  - Add createEpubAudiobookSourceAdapter and createPdfAudiobookSourceAdapter adapter modules
    to prepare chapters from EPUB/PDF sources.
  - Add runAudiobookGeneration and regenerateAudiobookChapter pipeline functions to handle
    chapter preparation, progress tracking, request header construction, and retry/abort behavior.
  - Wire adapters/pipeline into EPUBContext and PDFContext to simplify and centralize audiobook flow.
- Move config preference helpers out of ConfigContext into dedicated client modules:
  - buildSyncedPreferencePatch (lib/client/config/preferences.ts)
  - applyConfigUpdate and getVoicePreferenceKey (lib/client/config/updates.ts)
  - Use these helpers in ConfigContext to keep update logic small and testable.
- Add TTS settings view model resolver (lib/client/settings/tts-settings.ts) and tests
  for the tts-provider catalog and config helpers (tests/unit/tts-provider-catalog.spec.ts).
- Minor usages updated: AudiobookExportModal, SettingsModal, useVoiceManagement, TTSContext,
  server tts generation, and various context files to consume the new modules.

Why: Reduce duplication, improve testability, and separate concerns for provider metadata,
UI view-model logic, and audiobook generation flow. Breaking changes: none.
2026-04-04 17:53:57 -06:00
Richard R
e41a5ba481 refactor(audiobooks): Refactor TTS generation and audiobook chapter creation
- Removed unused imports and types from TTS route.
- Consolidated TTS buffer generation logic into a new module.
- Implemented caching for TTS audio buffers using LRU cache.
- Updated TTS request handling to support instructions for specific models.
- Refactored audiobook chapter creation to use new TTS generation logic.
- Simplified error handling and response management in TTS API.
- Enhanced client-side logic to manage TTS requests and retries.
- Updated types to reflect changes in TTS request payload structure.
2026-03-19 11:54:33 -06:00