openreader/docs-site/docs/configure/object-blob-storage.md at 9f25e05cce4d3beb0f4bd0f95b7a9b2a124a9b91

Richard R 9f25e05cce feat(previews): implement document preview generation and caching

- Add document preview caching logic with in-memory and persisted storage.
- Implement S3 blobstore functions for managing document previews.
- Create rendering functions for PDF and EPUB cover images to JPEG format.
- Introduce database schema and functions for managing document preview metadata.
- Add unit tests for rendering PDF and EPUB previews.

2026-02-12 16:05:03 -07:00

3.7 KiB

Raw Blame History

title
Object / Blob Storage

import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem';

This page documents storage backends, blob upload routing, and core Docker mount behavior.

Storage backends

Embedded (default): SQLite metadata + embedded SeaweedFS (weed mini) blobs.
External: Postgres + external S3-compatible object storage.

Storage variables are documented in Environment Variables.

Ports

3003: OpenReader app and API routes
8333: Embedded SeaweedFS S3 endpoint for direct browser blob access

:::info 8333 is only needed for direct browser presigned access to embedded SeaweedFS. :::

Upload behavior

Primary path: browser uploads to presigned URL from /api/documents/blob/upload/presign.
Fallback path: /api/documents/blob/upload/fallback when direct upload fails/unreachable.
Read/download path: blob/content serving route /api/documents/blob (not the upload fallback route).
Preview path: /api/documents/blob/preview (returns 202 while a preview is generating; serves/redirects when ready).

Document previews

PDF/EPUB previews are generated server-side and stored in object storage under document_previews_v1.
Preview generation is triggered on upload registration and also backfills on first preview request for older docs.
Preview artifacts are temporary-cache friendly and can be regenerated from the source document blob.

FS / Volume Mounts

App data mount

Target: /app/docstore
Recommended: yes, for persistence
Purpose: persists SeaweedFS blob data, SQLite metadata DB, migrations, and local runtime temp state
Mount string: -v openreader_docstore:/app/docstore

Library source mount (optional)

Target: /app/docstore/library
Recommended: optional, use read-only (:ro)
Purpose: exposes host files as a source for server library import
Mount string: -v /path/to/your/library:/app/docstore/library:ro
Details: Server Library Import

Private blob endpoint mode

If 8333 is not published externally:

Document uploads still work through upload fallback proxy
Reads/snippets continue through app API routes
Direct presigned browser upload/download to embedded endpoint is unavailable

:::warning Without 8333, expect higher app-server traffic because uploads/downloads go through API routes instead of direct object endpoint access. :::

Audiobook Storage Debug Commands

Audiobook assets are stored in object storage under the audiobooks_v1 keyspace. Use these commands to inspect and download objects for debugging.

# List all audiobook objects
aws s3 ls "s3://$S3_BUCKET/$S3_PREFIX/audiobooks_v1/" --recursive

# Filter to one book id (replace <book-id>)
aws s3 ls "s3://$S3_BUCKET/$S3_PREFIX/audiobooks_v1/" --recursive | grep "<book-id>-audiobook/"

# Download one object by full key
aws s3 cp "s3://$S3_BUCKET/$S3_PREFIX/audiobooks_v1/<path>/<file>.m4b" "./audiobook.m4b"

# List all audiobook objects
aws s3 ls "s3://$S3_BUCKET/$S3_PREFIX/audiobooks_v1/" --recursive --endpoint-url "$S3_ENDPOINT"

# Filter to one book id (replace <book-id>)
aws s3 ls "s3://$S3_BUCKET/$S3_PREFIX/audiobooks_v1/" --recursive --endpoint-url "$S3_ENDPOINT" | grep "<book-id>-audiobook/"

# Download one object by full key
aws s3 cp "s3://$S3_BUCKET/$S3_PREFIX/audiobooks_v1/<path>/<file>.m4b" "./audiobook.m4b" --endpoint-url "$S3_ENDPOINT"

Embedded default example: S3_ENDPOINT=http://127.0.0.1:8333 (or your mapped host/port).

3.7 KiB Raw Blame History