Commit graph

4 commits

Author SHA1 Message Date
Richard R
49d9735237 refactor(worker): unify port environment variable usage and update deployment docs
Align compute worker to use PORT environment variable consistently across
Dockerfile, docker-compose, and server code, replacing COMPUTE_WORKER_PORT.
Update .env.example and deployment documentation to clarify platform-specific
behavior, especially for Railway and similar environments. This simplifies
configuration and reduces platform-specific issues.
2026-05-20 17:05:35 -06:00
Richard R
1bd13c02fe refactor(compute): modularize core contracts and runtime, enable flexible backend resolution
Restructure compute core by extracting job contracts and runtime logic into separate
modules (`contracts.ts`, `local-runtime.ts`) for improved modularity and clearer
API boundaries. Update exports and imports across worker, server, and local
compute backends to use the new modules. Enhance Next.js config to support
dynamic backend selection via build-time constants and aliasing, enabling
seamless switching between local and worker compute modes. Adjust TypeScript
paths and Dockerfile entrypoint to align with the new structure.
2026-05-20 15:07:22 -06:00
Richard R
3a1dd41d91 refactor(build): update Dockerfiles for improved dependency isolation and workspace handling
Remove legacy .npmrc config. Revise root and worker Dockerfiles to enhance
layer caching, explicitly copy only required build artifacts, and streamline
runtime dependencies. Switch to pnpm workspace-aware install for compute worker
to ensure correct dependency resolution. Set Next.js output to standalone mode
for optimized server builds. Update docker-compose sync targets for consistency.
2026-05-20 13:21:12 -06:00
Richard R
f1aa1c3e3b feat(compute): add external compute worker backend and integration
Introduce support for external compute worker mode (`COMPUTE_MODE=worker`) using a new `WorkerComputeBackend`. This enables offloading heavy ONNX Whisper alignment and PDF layout parsing to a standalone worker service (Redis + BullMQ), improving scalability and compatibility with serverless/limited environments.

- Add `@openreader/compute-core` as a shared package for ONNX inference and PDF parsing logic.
- Implement `WorkerComputeBackend` and worker contract/types for remote job execution.
- Update compute backend selection logic and remove previous worker mode guards.
- Extend `WhisperAlignInput` and `PdfLayoutInput` types to support object keys for remote data access.
- Refactor local compute backend to use `@openreader/compute-core` and support both buffer and object key inputs.
- Update job runner, TTS segment alignment, and PDF layout parsing flows to use new compute backend APIs.
- Add scripts, Docker workflow, and documentation for deploying and running the compute worker.
- Update environment variable docs and examples for worker mode, including storage requirements and configuration.
- Document published images and stack changes to reflect the new compute worker architecture.

BREAKING CHANGE: `COMPUTE_MODE=worker` now requires an external compute worker service and S3-compatible object storage. Embedded SeaweedFS (`weed mini`) is not supported in worker mode. See the new documentation for deployment and configuration details.
2026-05-19 15:21:25 -06:00