ProxmoxBackupClientPBSClientTool/CHANGELOG.md

# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Added
- **Docker-based cross-platform solution (v1.2.0 - NEW!)**
  - Full Docker implementation for Windows, macOS, and Linux
  - Dockerfile with PBS client in Debian container
  - Platform-specific docker-compose files (linux/windows/macos)
  - Daemon mode with internal cron scheduler
  - One-shot backup mode for manual runs
  - REST API server for remote management (optional)
  - Health monitoring and status endpoints
  - Automatic encryption key generation and management
  - Build and deployment scripts (`build.sh`, `deploy.sh`)
  - Complete Docker documentation (README-DOCKER.md, QUICKSTART-DOCKER.md)
  - Organized in `docker/` subdirectory
  - Cross-platform backup support matrix in README
  - Platform-specific exclusion patterns
  - Metadata change detection for fast incrementals
- **Multi-target backup support (v1.1.0 - COMPLETE)**
  - Support for multiple backup destinations (different PBS servers for redundancy)
  - Named backup targets (e.g., "offsite", "local", "backup1")
  - Target management functions (COMPLETE):
    - `list_targets()` - List all configured target names
    - `show_targets_list()` - Display all targets with server/datastore/status
    - `show_target_detail()` - Show comprehensive target configuration
    - `add_target()` - Interactive target creation wizard
    - `edit_target()` - Edit connection, settings, or full reconfig
    - `delete_target()` - Remove target with confirmation and cleanup
    - `validate_target_name()` - Enforce naming rules (alphanumeric, dash, underscore)
  - Per-target wrapper functions (COMPLETE):
    - `interactive_config_for_target()` - Configure specific target
    - `reconfigure_connection_for_target()` - Update connection only
    - `reconfigure_backup_settings_for_target()` - Update backup settings only
    - `run_backup_for_target()` - Execute backup with live progress monitoring
  - Systemd service creation (COMPLETE):
    - `create_systemd_service_for_target()` - Generate all services for named target
    - Creates pbs-backup-TARGET.service (scheduled)
    - Creates pbs-backup-TARGET-manual.service (manual/full)
    - Creates pbs-backup-TARGET.timer (scheduler)
    - Generates target-specific backup scripts with embedded target name
  - Automatic migration (COMPLETE):
    - `migrate_legacy_config()` - Auto-migrate single-target to "default" target
    - Renames services: pbs-backup → pbs-backup-default
    - Preserves existing schedules and backups
    - Zero downtime migration
  - Configuration storage (COMPLETE):
    - /etc/proxmox-backup-client/targets/TARGET.conf
    - /etc/proxmox-backup-client/backup-TARGET.sh
    - /etc/systemd/system/pbs-backup-TARGET.{service,timer}
  - Main menu integration (COMPLETE):
    - Multi-target management menu with 8 options
    - First-time setup wizard for new installations
    - Automatic target name prompt for first target
    - Clean menu structure: List, Add, Edit, Delete, Run, View, Reinstall, Exit
  - Ready for testing:
    - All code paths implemented
    - Syntax validated
    - Needs functional testing
    - Needs migration testing from legacy configs
- **Flexible block device scheduling in hybrid mode**
  - Separate schedule configuration for block device backups vs file backups
  - Block device schedule options:
    - Weekly (specific day of week, default Sunday)
    - Biweekly (every other week on specific day)
    - Monthly (specific day of month)
    - Custom day of week (1-7, Monday-Sunday)
    - Custom day of month (1-31)
  - File backups run on main schedule, block device on independent schedule
  - Configuration stored in BLOCK_DEVICE_FREQUENCY and BLOCK_DEVICE_DAY variables
- **Manual backup selection for hybrid mode**
  - Interactive menu when running backup in hybrid mode
  - Three options:
    - Files only (fast ~2-3 minutes) - daily file archives
    - Block device only (slow ~20-30 minutes) - full disk image for VM conversion
    - Both files and block device (complete backup)
  - Allows selective backup execution without changing configuration
  - Backup script supports three FORCE_FULL values: "files", "block", "yes"
- Intelligent reconfiguration options when PBS client is already installed
  - Quick connection-only reconfiguration (server/credentials only)
  - Full reconfiguration of all settings
  - Reinstall option with reconfiguration
  - Exit without changes option
- 3-step connection verification process:
  - Step 1: Server reachability test (5s timeout with curl)
  - Step 2: Authentication test with automatic SSL fingerprint acceptance
  - Step 3: Datastore access verification
- Display available block devices when invalid device is entered
- Step-by-step progress indicators during connection testing
- `test-connection.sh` - Diagnostic script for testing PBS connections manually
  - Parameterized for security (no hardcoded credentials)
  - Tests server reachability, authentication, and datastore access
  - Handles SSL fingerprint acceptance interactively
- Comprehensive documentation in README:
  - Step-by-step walkthrough of complete installation (14 steps)
  - PBS server setup guide (API token creation and permissions)
  - Common permission error troubleshooting
  - Explanation of backup types, schedules, and retention policies
- **Live backup progress monitoring**
  - Real-time log following when running backups
  - Automatic completion detection
  - Shows PBS client's built-in progress bars and statistics
  - Can exit with Ctrl+C (backup continues in background)
  - Applied to both "Run backup now" menu option and post-install backup

### Changed
- Installation instructions now use `git clone` instead of `wget`
- Connection test provides better user feedback during testing
- Script now detects existing configurations and offers appropriate options
- Block device detection now strips btrfs subvolume notation (e.g., `[/@]`)
- Connection test succeeds if authentication works, even if no backups exist yet
- More specific error messages based on which step of connection test fails
- SSL certificate fingerprints are now automatically accepted during setup
- Reduced authentication timeout from 30s to 15s
- README now includes comprehensive step-by-step walkthrough
- Prerequisites section expanded with detailed PBS server setup instructions
- README updated with live backup progress examples in Step 13
- Reconfiguration section now documents "Run backup now" option (option 4)
- Manual Backup section updated to recommend easy method via installer
- **Default realm changed from "pbs" to "pam"** (more common for root authentication)
- **Default encryption setting changed from "yes" to "no"** (user can opt-in if needed)
- Main menu now includes "Run backup now" option for immediate backup testing
- Main menu now includes "Modify backup schedule/type" option (option 5)
- Repository renamed from PBSClientInstaller to PBSClientTool
- **Manual backups now force full backup (files + block device) regardless of day**
  - Created separate pbs-backup-manual.service for manual runs
  - Scheduled backups still follow daily/weekly pattern
  - Manual backups always include block device even on non-Sunday
- Script version bumped to 1.1.0 for multi-target support

### Added
- **System-wide installation support**
  - New `--install` flag to install PBSClientTool to /usr/local/bin
  - Run from anywhere with `sudo PBSClientTool` command
  - No need to navigate to script directory
  - Includes `--uninstall` to remove from system
  - Added `--help` and `--version` flags for command-line usage
  - Installation preserves original script (safe to update via git pull)
  - install_script() function handles installation with overwrite confirmation
  - uninstall_script() function removes command but keeps configurations
  - Command-line argument parsing before main() execution
  - **Menu option 7: "Install as system command"** - Available in main menu
  - **Post-installation prompt** - Automatically asks if you want to install as system command after initial setup
  - Detects if already installed (skips prompt if /usr/local/bin/PBSClientTool exists)
- **Automatic connection testing at startup**
  - All configured targets are automatically tested when script starts
  - Quick authentication check (5 second timeout per target)
  - Shows connection status: ✓ Connected or ✗ Failed
  - Helps identify configuration issues before running backups
  - Warns user if any targets have connection problems
  - Runs in subshell to avoid environment pollution

### Changed
- **Smart menu adaptation based on installation status**
  - Script detects if already installed as system command at startup
  - Hides "Install as system command" menu option if already installed
  - Menu adjusts from 8 options to 7 options when installed
  - Exit option renumbers from 8 to 7 automatically
  - Prompt range adjusts: [1-8] when not installed, [1-7] when installed
  - Installation status tracked in SCRIPT_INSTALLED variable
  - After installing from menu, option disappears immediately (updates status)
  - Cleaner UX - no redundant install option when already installed
- **Documentation reorganization**
  - README.md simplified from 750 lines to 243 lines
  - Removed redundant step-by-step walkthrough sections
  - Moved all troubleshooting content to TROUBLESHOOTING.md
  - Focused README on quick start and essential usage
  - New TROUBLESHOOTING.md covers:
    - Connection issues and SSL certificate problems
    - Permission errors with detailed PBS setup instructions
    - Installation issues (Ubuntu libssl1.1, Arch libfuse3)
    - Backup problems and configuration issues
    - Common misconfigurations with examples
  - Easier to find relevant information
  - Better navigation and organization
- **Clearer installation flow when PBS client not installed**
  - Shows what the script will do before proceeding
  - Lists 3 steps: Install PBS client, Configure target, Set up automation
  - Asks for confirmation before installation
  - Allows user to cancel if they change their mind
  - No menu options shown (install → configure → exit)
  - Clear progress messages throughout installation
- **Improved menu flow for viewing target details**
  - "View target details" moved from main menu option 6 to sub-menu under "List all backup targets"
  - After listing targets, users can choose:
    - View target details (select target to see comprehensive info)
    - Back to main menu
  - Main menu reduced from 8 options to 7 (cleaner, more focused)
  - More intuitive workflow: list → view details → back to menu
- **Target selection now accepts both numbers and names**
  - All target selection prompts now accept either:
    - Target number from the displayed list (e.g., "1")
    - Full target name (e.g., "default")
  - Applies to: Edit, Delete, Run backup, View details
  - Improves user experience - no need to remember target names
  - More intuitive for users familiar with numbered menus

### Fixed
- **CRITICAL: All input fields now trimmed to prevent whitespace issues**
  - Added `| xargs` to all prompted variables (server, port, datastore, username, realm, token name, etc.)
  - Prevents trailing/leading whitespace from breaking PBS_REPOSITORY string
  - Fixes authentication failures caused by malformed repository strings
  - Example: `"root@pam "` (with space) now becomes `"root@pam"` automatically
  - Applied to both interactive_config_for_target() and reconfigure_connection_for_target()
- **Main menu now loops properly after actions**
  - Removed `exit 0` calls from all menu options except Exit
  - Menu now continues after listing targets, running backups, or other actions
  - Users can perform multiple operations without restarting script
- **Target list display improved for incomplete configurations**
  - Shows warning message when PBS_SERVER or PBS_DATASTORE are empty
  - Suggests using "Edit target" option to fix incomplete configs
  - Uses process substitution to avoid variable scoping issues
  - Clears variables between iterations to prevent data carryover
- **CRITICAL**: SSL fingerprint prompt no longer causes authentication timeout
  - Script now automatically accepts SSL fingerprints by piping 'y' to login
  - This was the root cause of "authentication hanging" issues
- **CRITICAL**: Password/token capture no longer includes newline character
  - `prompt_password()` function now outputs formatting to stderr
  - Added defensive newline stripping when writing config file
  - Filters `\n` and `\r` characters from passwords using `tr -d '\n\r'`
  - Fixes "authentication failed - invalid credentials" in backup service
  - Config file now has properly formatted single-line passwords
  - Applied to both `reconfigure_connection()` and `create_systemd_service()`
- Script no longer hangs indefinitely when PBS server is unreachable
- Block device auto-detection now correctly handles btrfs subvolumes
- Invalid device paths like `/dev/mapper/root[/@]` are now properly cleaned
- Connection test now differentiates between network issues and authentication failures
- Shows actual PBS client error messages when authentication fails
- Authentication test now uses correct `login` command instead of non-existent `status` command
- Connection test no longer times out due to using wrong PBS client commands
- **Backup progress logs now display properly during manual backup runs**
  - Fixed journalctl following wrong service (was following pbs-backup.service instead of pbs-backup-manual.service)
  - Removed stderr suppression (2>/dev/null) that was hiding output
  - Added --since flag to capture logs from service start, not just new entries
  - Now shows verbose progress: "processed 65.6 GiB in 18m, uploaded 64.4 GiB"
- Backup logs now close automatically when backup completes (no more blank screen)

## [1.0.0] - 2025-11-01

### Added
- Initial release
- Auto-detection of Linux distribution (Ubuntu, Debian, Arch)
- Interactive configuration via console prompts
- Automatic encryption key generation with paper backup
- Systemd service and timer for automated backups
- Configurable retention policies
- Connection testing before finalization
- Immediate backup option after installation
- Support for file-level (.pxar) backups
- Support for block device (.img) backups
- Hybrid backup mode (daily files + weekly block device)
- Comprehensive error handling and logging
- Uninstaller script