f1aa1c3e3b
Introduce support for external compute worker mode (`COMPUTE_MODE=worker`) using a new `WorkerComputeBackend`. This enables offloading heavy ONNX Whisper alignment and PDF layout parsing to a standalone worker service (Redis + BullMQ), improving scalability and compatibility with serverless/limited environments. - Add `@openreader/compute-core` as a shared package for ONNX inference and PDF parsing logic. - Implement `WorkerComputeBackend` and worker contract/types for remote job execution. - Update compute backend selection logic and remove previous worker mode guards. - Extend `WhisperAlignInput` and `PdfLayoutInput` types to support object keys for remote data access. - Refactor local compute backend to use `@openreader/compute-core` and support both buffer and object key inputs. - Update job runner, TTS segment alignment, and PDF layout parsing flows to use new compute backend APIs. - Add scripts, Docker workflow, and documentation for deploying and running the compute worker. - Update environment variable docs and examples for worker mode, including storage requirements and configuration. - Document published images and stack changes to reflect the new compute worker architecture. BREAKING CHANGE: `COMPUTE_MODE=worker` now requires an external compute worker service and S3-compatible object storage. Embedded SeaweedFS (`weed mini`) is not supported in worker mode. See the new documentation for deployment and configuration details.
22 lines
374 B
YAML
22 lines
374 B
YAML
packages:
|
|
- .
|
|
- compute/*
|
|
|
|
allowBuilds:
|
|
'@napi-rs/canvas': true
|
|
better-sqlite3: true
|
|
canvas: false
|
|
core-js: false
|
|
es5-ext: false
|
|
esbuild: false
|
|
ffmpeg-static: true
|
|
onnxruntime-node: true
|
|
sharp: false
|
|
unrs-resolver: false
|
|
|
|
overrides:
|
|
'@types/localforage': npm:empty-module@0.0.2
|
|
'@xmldom/xmldom': ^0.9.8
|
|
lodash: ^4.17.23
|
|
|
|
strictDepBuilds: false
|