chore: update readme with mandatory env variables
This commit is contained in:
Nicolas Meienberger
parent
8ced05c5a9
commit
1118f45788
2 changed files with 12 additions and 194 deletions
184
AGENTS.md
184
AGENTS.md
|
|
@ -9,26 +9,6 @@
|
||||||
|
|
||||||
Zerobyte is a backup automation tool built on top of Restic that provides a web interface for scheduling, managing, and monitoring encrypted backups. It supports multiple volume backends (NFS, SMB, WebDAV, SFTP, local directories) and repository backends (S3, Azure, GCS, local, and rclone-based storage).
|
Zerobyte is a backup automation tool built on top of Restic that provides a web interface for scheduling, managing, and monitoring encrypted backups. It supports multiple volume backends (NFS, SMB, WebDAV, SFTP, local directories) and repository backends (S3, Azure, GCS, local, and rclone-based storage).
|
||||||
|
|
||||||
## Technology Stack
|
|
||||||
|
|
||||||
- **Runtime**: Bun 1.3.1
|
|
||||||
- **Server**: Hono (web framework) with Bun runtime
|
|
||||||
- **Client**: React Router v7 (SSR) with React 19
|
|
||||||
- **Database**: SQLite with Drizzle ORM
|
|
||||||
- **Validation**: ArkType for runtime schema validation
|
|
||||||
- **Styling**: Tailwind CSS v4 + Radix UI components
|
|
||||||
- **Architecture**: Unified application structure (not a monorepo)
|
|
||||||
- **Code Quality**: Oxfmt for formatting, Oxlint for linting
|
|
||||||
|
|
||||||
## Repository Structure
|
|
||||||
|
|
||||||
This is a unified application with the following structure:
|
|
||||||
|
|
||||||
- `app/server` - Bun-based API server with Hono
|
|
||||||
- `app/client` - React Router SSR frontend components and modules
|
|
||||||
- `app/schemas` - Shared ArkType schemas for validation
|
|
||||||
- `app/drizzle` - Database migrations
|
|
||||||
|
|
||||||
### Type Checking
|
### Type Checking
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|
@ -67,8 +47,6 @@ bunx drizzle-kit generate --custom --name=fix-timestamps-to-ms
|
||||||
### API Client Generation
|
### API Client Generation
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Generate TypeScript API client from OpenAPI spec
|
|
||||||
# Note: Server is always running don't need to start it separately
|
|
||||||
bun run gen:api-client
|
bun run gen:api-client
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
@ -81,165 +59,3 @@ bunx oxfmt format --write <path>
|
||||||
# Lint
|
# Lint
|
||||||
bun run lint
|
bun run lint
|
||||||
```
|
```
|
||||||
|
|
||||||
## Architecture
|
|
||||||
|
|
||||||
### Server Architecture
|
|
||||||
|
|
||||||
The server follows a modular service-oriented architecture:
|
|
||||||
|
|
||||||
**Entry Point**: `app/server/index.ts`
|
|
||||||
|
|
||||||
- Initializes main API server on port 4096 (REST API + serves static frontend)
|
|
||||||
|
|
||||||
**Modules** (`app/server/modules/`):
|
|
||||||
Each module follows a controller <20> service <20> database pattern:
|
|
||||||
|
|
||||||
- `auth/` - User authentication and session management
|
|
||||||
- `volumes/` - Volume mounting/unmounting (NFS, SMB, WebDAV, SFTP, directories)
|
|
||||||
- `repositories/` - Restic repository management (S3, Azure, GCS, local, rclone)
|
|
||||||
- `backups/` - Backup schedule management and execution
|
|
||||||
- `notifications/` - Notification system with multiple providers (Discord, email, Gotify, Ntfy, Slack, Pushover)
|
|
||||||
- `driver/` - Docker volume plugin implementation
|
|
||||||
- `events/` - Server-Sent Events for real-time updates
|
|
||||||
- `system/` - System information and capabilities
|
|
||||||
- `lifecycle/` - Application startup/shutdown hooks
|
|
||||||
|
|
||||||
**Backends** (`app/server/modules/backends/`):
|
|
||||||
Each volume backend (NFS, SMB, WebDAV, SFTP, directory) implements mounting logic using system tools (mount.nfs, mount.cifs, davfs2, sshfs).
|
|
||||||
|
|
||||||
**Jobs** (`app/server/jobs/`):
|
|
||||||
Cron-based background jobs managed by the Scheduler:
|
|
||||||
|
|
||||||
- `backup-execution.ts` - Runs scheduled backups (every minute)
|
|
||||||
- `cleanup-dangling.ts` - Removes stale mounts (hourly)
|
|
||||||
- `healthchecks.ts` - Checks volume health (every 5 minutes)
|
|
||||||
- `repository-healthchecks.ts` - Validates repositories (every 10 minutes)
|
|
||||||
- `cleanup-sessions.ts` - Expires old sessions (daily)
|
|
||||||
|
|
||||||
**Core** (`app/server/core/`):
|
|
||||||
|
|
||||||
- `scheduler.ts` - Job scheduling system using node-cron
|
|
||||||
- `capabilities.ts` - Detects available system features
|
|
||||||
- `constants.ts` - Application-wide constants
|
|
||||||
|
|
||||||
**Utils** (`app/server/utils/`):
|
|
||||||
|
|
||||||
- `restic.ts` - Restic CLI wrapper with type-safe output parsing
|
|
||||||
- `spawn.ts` - Safe subprocess execution helpers
|
|
||||||
- `logger.ts` - Winston-based logging
|
|
||||||
- `crypto.ts` - Encryption utilities
|
|
||||||
- `errors.ts` - Error handling middleware
|
|
||||||
|
|
||||||
**Database** (`app/server/db/`):
|
|
||||||
|
|
||||||
- Uses Drizzle ORM with SQLite
|
|
||||||
- Schema in `schema.ts` defines: volumes, repositories, backup schedules, notifications, users, sessions
|
|
||||||
- Migrations: `app/drizzle/`
|
|
||||||
|
|
||||||
### Client Architecture
|
|
||||||
|
|
||||||
**Framework**: React Router v7 with SSR
|
|
||||||
**Entry Point**: `app/root.tsx`
|
|
||||||
|
|
||||||
The client uses:
|
|
||||||
|
|
||||||
- TanStack Query for server state management
|
|
||||||
- Auto-generated API client from OpenAPI spec (in `app/client/api-client/`)
|
|
||||||
- Radix UI primitives with custom Tailwind styling
|
|
||||||
- Server-Sent Events hook (`use-server-events.ts`) for real-time updates
|
|
||||||
|
|
||||||
Routes are organized in feature modules at `app/client/modules/*/routes/`.
|
|
||||||
|
|
||||||
### Shared Schemas
|
|
||||||
|
|
||||||
`app/schemas/` contains ArkType schemas used by both client and server:
|
|
||||||
|
|
||||||
- Volume configurations (NFS, SMB, WebDAV, SFTP, directory)
|
|
||||||
- Repository configurations (S3, Azure, GCS, local, rclone)
|
|
||||||
- Restic command output parsing types
|
|
||||||
- Backend status types
|
|
||||||
|
|
||||||
These schemas provide runtime validation and TypeScript types.
|
|
||||||
|
|
||||||
## Restic Integration
|
|
||||||
|
|
||||||
Zerobyte is a wrapper around Restic for backup operations. Key integration points:
|
|
||||||
|
|
||||||
**Repository Management**:
|
|
||||||
|
|
||||||
- Creates/initializes Restic repositories via `restic init`
|
|
||||||
- Supports multiple backends: local, S3, Azure Blob Storage, Google Cloud Storage, or any rclone-supported backend
|
|
||||||
- Stores single encryption password in `/var/lib/zerobyte/restic/password` (auto-generated on first run)
|
|
||||||
|
|
||||||
**Backup Operations**:
|
|
||||||
|
|
||||||
- Executes `restic backup` with user-defined schedules (cron expressions)
|
|
||||||
- Supports include/exclude patterns for selective backups
|
|
||||||
- Parses JSON output for progress tracking and statistics
|
|
||||||
- Implements retention policies via `restic forget --prune`
|
|
||||||
|
|
||||||
**Repository Utilities** (`utils/restic.ts`):
|
|
||||||
|
|
||||||
- `buildRepoUrl()` - Constructs repository URLs for different backends
|
|
||||||
- `buildEnv()` - Sets environment variables (credentials, cache dir)
|
|
||||||
- Type-safe parsing of Restic JSON output using ArkType schemas
|
|
||||||
|
|
||||||
**Rclone Integration** (`app/server/modules/repositories/`):
|
|
||||||
|
|
||||||
- Allows using any rclone backend as a Restic repository
|
|
||||||
- Dynamically generates rclone config and passes via environment variables
|
|
||||||
- Supports backends like Dropbox, Google Drive, OneDrive, Backblaze B2, etc.
|
|
||||||
|
|
||||||
## Environment & Configuration
|
|
||||||
|
|
||||||
**Runtime Environment Variables**:
|
|
||||||
|
|
||||||
- Database path: `./data/zerobyte.db` (configurable via `drizzle.config.ts`)
|
|
||||||
- Restic cache: `/var/lib/zerobyte/restic/cache`
|
|
||||||
- Restic password: `/var/lib/zerobyte/restic/password`
|
|
||||||
- Volume mounts: `/var/lib/zerobyte/mounts/<volume-name>`
|
|
||||||
- Local repositories: `/var/lib/zerobyte/repositories/<repo-name>`
|
|
||||||
|
|
||||||
**Capabilities Detection**:
|
|
||||||
On startup, the server detects available capabilities (see `core/capabilities.ts`):
|
|
||||||
|
|
||||||
- **rclone**: Requires `/root/.config/rclone` directory access
|
|
||||||
- System will gracefully degrade if capabilities are unavailable
|
|
||||||
|
|
||||||
## Common Workflows
|
|
||||||
|
|
||||||
### Adding a New Volume Backend
|
|
||||||
|
|
||||||
1. Create backend implementation in `app/server/modules/backends/<backend>/`
|
|
||||||
2. Implement `mount()` and `unmount()` methods
|
|
||||||
3. Add schema to `app/schemas/volumes.ts`
|
|
||||||
4. Update `volumeConfigSchema` discriminated union
|
|
||||||
5. Update backend factory in `app/server/modules/backends/backend.ts`
|
|
||||||
|
|
||||||
### Adding a New Repository Backend
|
|
||||||
|
|
||||||
1. Add backend type to `app/schemas/restic.ts`
|
|
||||||
2. Update `buildRepoUrl()` in `app/server/utils/restic.ts`
|
|
||||||
3. Update `buildEnv()` to handle credentials/configuration
|
|
||||||
4. Add DTO schemas in `app/server/modules/repositories/repositories.dto.ts`
|
|
||||||
5. Update repository service to handle new backend
|
|
||||||
|
|
||||||
### Adding a New Scheduled Job
|
|
||||||
|
|
||||||
1. Create job class in `app/server/jobs/<job-name>.ts` extending `Job`
|
|
||||||
2. Implement `run()` method
|
|
||||||
3. Register in `app/server/modules/lifecycle/startup.ts` with cron expression:
|
|
||||||
```typescript
|
|
||||||
Scheduler.build(YourJob).schedule("* * * * *");
|
|
||||||
```
|
|
||||||
|
|
||||||
## Important Notes
|
|
||||||
|
|
||||||
- **TypeScript**: Uses `"type": "module"` - all imports must include extensions when targeting Node/Bun
|
|
||||||
- **Validation**: Prefer ArkType over Zod - it's used throughout the codebase
|
|
||||||
- **Visibility**: Prefer using the `cn` helper with `{ hidden: condition }` instead of conditional rendering with ternaries or `&&` for toggling element visibility in the DOM.
|
|
||||||
- **Database**: Timestamps are stored as Unix epoch integers, not ISO strings
|
|
||||||
- **Security**: Restic password file has 0600 permissions - never expose it
|
|
||||||
- **Mounting**: Requires privileged container or CAP_SYS_ADMIN for FUSE mounts
|
|
||||||
- **API Documentation**: OpenAPI spec auto-generated at `/api/v1/openapi.json`, docs at `/api/v1/docs`
|
|
||||||
|
|
|
||||||
22
README.md
22
README.md
|
|
@ -66,11 +66,13 @@ services:
|
||||||
|
|
||||||
> [!NOTE]
|
> [!NOTE]
|
||||||
> **TrueNAS Users:** The host path `/var/lib` is ephemeral on TrueNAS and will be reset during system upgrades. Instead of using `/var/lib/zerobyte:/var/lib/zerobyte`, create a dedicated ZFS dataset (e.g., `tank/docker/zerobyte`) and mount it instead:
|
> **TrueNAS Users:** The host path `/var/lib` is ephemeral on TrueNAS and will be reset during system upgrades. Instead of using `/var/lib/zerobyte:/var/lib/zerobyte`, create a dedicated ZFS dataset (e.g., `tank/docker/zerobyte`) and mount it instead:
|
||||||
|
>
|
||||||
> ```yaml
|
> ```yaml
|
||||||
> volumes:
|
> volumes:
|
||||||
> - /etc/localtime:/etc/localtime:ro
|
> - /etc/localtime:/etc/localtime:ro
|
||||||
> - /mnt/tank/docker/zerobyte:/var/lib/zerobyte
|
> - /mnt/tank/docker/zerobyte:/var/lib/zerobyte
|
||||||
> ```
|
> ```
|
||||||
|
>
|
||||||
> This ensures your configuration, encryption keys, and database persist across TrueNAS upgrades.
|
> This ensures your configuration, encryption keys, and database persist across TrueNAS upgrades.
|
||||||
|
|
||||||
Then, run the following command to start Zerobyte:
|
Then, run the following command to start Zerobyte:
|
||||||
|
|
@ -87,16 +89,16 @@ Zerobyte can be customized using environment variables. Below are the available
|
||||||
|
|
||||||
### Environment Variables
|
### Environment Variables
|
||||||
|
|
||||||
| Variable | Description | Default |
|
| Variable | Description | Default |
|
||||||
| :-------------------- | :---------------------------------------------------------------------------------------------------------------------------------- | :--------- |
|
| :-------------------- | :---------------------------------------------------------------------------------------------------------------------------------------- | :--------- |
|
||||||
| `BASE_URL` | The base URL of your Zerobyte instance (e.g., `https://zerobyte.example.com`). See [Authentication](#authentication) below. | (none) |
|
| `BASE_URL` | **Required.** The base URL of your Zerobyte instance (e.g., `https://zerobyte.example.com`). See [Authentication](#authentication) below. | (none) |
|
||||||
| `APP_SECRET` | **Required.** A random secret key (32+ chars) used to encrypt sensitive data in the database. Generate with `openssl rand -hex 32`. | (none) |
|
| `APP_SECRET` | **Required.** A random secret key (32+ chars) used to encrypt sensitive data in the database. Generate with `openssl rand -hex 32`. | (none) |
|
||||||
| `PORT` | The port the web interface and API will listen on. | `4096` |
|
| `PORT` | The port the web interface and API will listen on. | `4096` |
|
||||||
| `RESTIC_HOSTNAME` | The hostname used by Restic when creating snapshots. Automatically detected if a custom hostname is set in Docker. | `zerobyte` |
|
| `RESTIC_HOSTNAME` | The hostname used by Restic when creating snapshots. Automatically detected if a custom hostname is set in Docker. | `zerobyte` |
|
||||||
| `TZ` | Timezone for the container (e.g., `Europe/Paris`). **Crucial for accurate backup scheduling.** | `UTC` |
|
| `TZ` | Timezone for the container (e.g., `Europe/Paris`). **Crucial for accurate backup scheduling.** | `UTC` |
|
||||||
| `TRUSTED_ORIGINS` | Comma-separated list of extra trusted origins for CORS (e.g., `http://localhost:3000,http://example.com`). | (none) |
|
| `TRUSTED_ORIGINS` | Comma-separated list of extra trusted origins for CORS (e.g., `http://localhost:3000,http://example.com`). | (none) |
|
||||||
| `LOG_LEVEL` | Logging verbosity. Options: `debug`, `info`, `warn`, `error`. | `info` |
|
| `LOG_LEVEL` | Logging verbosity. Options: `debug`, `info`, `warn`, `error`. | `info` |
|
||||||
| `SERVER_IDLE_TIMEOUT` | Idle timeout for the server in seconds. | `60` |
|
| `SERVER_IDLE_TIMEOUT` | Idle timeout for the server in seconds. | `60` |
|
||||||
|
|
||||||
### Secret References
|
### Secret References
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue