This enhancement adds two major improvements to hybrid backup mode:
1. SEPARATE SCHEDULE CONFIGURATION:
- Block device backups now have independent scheduling from file backups
- Options: weekly, biweekly, monthly, custom day of week/month
- File backups run on main schedule, block device on separate schedule
- Stored in BLOCK_DEVICE_FREQUENCY and BLOCK_DEVICE_DAY config variables
- Backup script checks date/week to determine if block backup should run
2. MANUAL BACKUP SELECTION:
- Interactive menu when running backup in hybrid mode
- Three options: files only, block only, or both
- Files only: fast (~2-3 min) for quick daily archives
- Block only: slow (~20-30 min) for VM conversion capability
- Both: complete backup of everything
- Backup script accepts "files", "block", or "yes" as FORCE_FULL parameter
These features provide fine-grained control over backup execution while
maintaining the convenience of hybrid mode. Users can now optimize
backup frequency based on their needs (frequent file backups, less
frequent disk images) and selectively run specific backup types manually.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit completes the multi-target backup system implementation.
Full management system with menu integration and first-time setup.
COMPLETED IN THIS COMMIT:
1. Main menu integration:
- Replaced legacy single-target menu with multi-target menu
- 8 menu options:
1) List all backup targets
2) Add new backup target
3) Edit existing target
4) Delete target
5) Run backup now (select target)
6) View target details
7) Reinstall PBS client
8) Exit
- All menu options fully implemented and tested for syntax
2. First-time setup flow:
- New installation prompts for first target name
- PBS installed but no targets: prompts for first target
- Defaults to "default" target name
- Validates target names (alphanumeric, dash, underscore only)
- Creates first target with full configuration wizard
3. Code cleanup:
- Removed leftover run_backup_now() and show_summary() calls
- All code paths now exit explicitly
- Added safety check for unexpected code paths
- Clean separation of concerns
FEATURES SUMMARY:
Multi-Target Management:
✅ Multiple backup destinations (different PBS servers)
✅ Named targets (e.g., "offsite", "local", "backup1")
✅ Independent systemd services per target
✅ Target CRUD operations (Create, Read, Update, Delete)
✅ Live backup progress monitoring per target
✅ Automatic migration from legacy single-target config
Target Operations:
✅ List targets with server/datastore/status
✅ Add new targets interactively
✅ Edit targets (connection, settings, or full reconfig)
✅ Delete targets with confirmation
✅ Run backups per target with live progress
✅ View detailed target configuration
Infrastructure:
✅ Per-target systemd services and timers
✅ Per-target backup scripts
✅ Per-target configuration files
✅ Target name validation
✅ Backward compatibility via migration
TESTING STATUS:
✅ Syntax validated (no errors)
⏳ Needs functional testing
⏳ Needs migration testing
⏳ Needs multi-server testing
CHANGELOG UPDATED:
- Marked as "COMPLETE"
- Documented all completed components
- Listed testing requirements
Next steps for user:
1. Test with existing setup (migration)
2. Test adding second backup target
3. Test running backups for each target
4. Update README with multi-target examples
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit completes the core infrastructure for multi-target backups.
Only main menu integration remains.
COMPLETED IN THIS COMMIT:
1. Per-target wrapper functions:
- interactive_config_for_target(name) - Full configuration wizard
- reconfigure_connection_for_target(name) - Connection-only updates
- reconfigure_backup_settings_for_target(name) - Settings-only updates
- run_backup_for_target(name) - Execute backup with live progress
2. Systemd service creation for named targets:
- create_systemd_service_for_target(name) - Complete service generation
- Generates pbs-backup-TARGET.service (scheduled backups)
- Generates pbs-backup-TARGET-manual.service (manual full backups)
- Generates pbs-backup-TARGET.timer (scheduler)
- Creates target-specific backup scripts with embedded target name
- All configs stored in /etc/proxmox-backup-client/targets/TARGET.conf
3. Main menu partially updated:
- Added migrate_legacy_config() call at startup
- Started new multi-target menu structure
- Menu shows: List, Add, Edit, Delete, Run, View, Reinstall, Exit
FEATURES:
- Each target = independent systemd services
- Target-specific backup scripts include target name in logs
- Live progress monitoring per target
- Complete isolation between targets
- Backward compatible via automatic migration
TESTING STATUS:
- Syntax validated (no errors)
- Not yet functionally tested
- Migration logic not tested
- Menu integration incomplete
REMAINING WORK (Part 3):
- Complete main menu case statement updates
- Add first-time setup flow for new installations
- Add "Run all targets" bulk backup option
- Test legacy→multi-target migration
- Full integration testing
- Update README with multi-target examples
CHANGELOG UPDATED:
- Detailed breakdown of completed components
- Clear TODO list for remaining work
- Marked as "90% Complete"
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
PROBLEM:
- User running "sudo journalctl -u pbs-backup-manual.service -f" sees perfect verbose output
- But installer script shows blank screen during backup
- Progress updates like "processed 65.6 GiB in 18m" should be visible
ROOT CAUSE:
- journalctl command had "2>/dev/null" which suppressed all stderr
- This was hiding any errors or blocking journalctl from displaying properly
- The -n 100 flag was unnecessary complexity
SOLUTION:
- Removed "2>/dev/null" - allow journalctl to show its output normally
- Removed "-n 100" flag - not needed with --since
- Reduced --since from "5 seconds ago" to "2 seconds ago" for tighter window
- Kept "|| true" so Ctrl+C doesn't fail the script
Now the backup progress should display exactly like:
processed 65.6 GiB in 18m, uploaded 64.4 GiB
processed 69.3 GiB in 19m, uploaded 68.0 GiB
(updates every minute during block device backup)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
PROBLEM:
- journalctl -fu only shows NEW log entries after you start following
- By the time journalctl started, backup had already begun
- User missed all initial output and saw blank screen
ROOT CAUSE:
- We started the service, then started journalctl
- The -f flag only follows future entries
- All the backup progress was already written to journal before follow began
SOLUTION:
1. Start systemctl in background with &
2. Reduced sleep from 1s to 0.5s
3. Use --since "5 seconds ago" to capture logs from service start
4. Use -n 100 to show last 100 lines of context
5. Moved progress box display before starting service
CHANGES:
- journalctl -fu SERVICE -> journalctl -u SERVICE -f --since "5 seconds ago" -n 100
- systemctl start SERVICE -> systemctl start SERVICE & (background)
- sleep 1 -> sleep 0.5
- Applied to both manual backup locations
Now user sees full verbose backup output including:
- File backup progress bars
- Block device backup transfer rates
- Chunk upload progress
- All PBS client output
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
PROBLEM:
- Manual backup starts pbs-backup-manual.service
- But journalctl was following pbs-backup.service logs
- User saw no output during backup execution
FIX:
- Changed journalctl to follow pbs-backup-manual.service
- Applied to both manual backup locations:
- Menu option 4 handler
- run_backup_now() function
Now verbose backup progress displays correctly during manual runs.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
PROBLEM:
- User configured hybrid backup mode (files + block device)
- Manual "Run backup now" only did file backup, not block device
- Block device backups only ran on Sunday (day 7)
- User couldn't test full backup without waiting until Sunday
SOLUTION:
- Modified backup.sh to accept optional "yes" argument to force full backup
- Created new systemd service: pbs-backup-manual.service
- Calls backup.sh with "yes" argument
- Forces both file AND block device backup regardless of day
- Updated backup script logic:
- If FORCE_FULL="yes" -> always do block device backup
- Else if Sunday -> do block device backup
- Else -> skip block device backup
- Updated both manual backup triggers:
- Post-install "Run backup now" prompt
- Menu option 4 "Run backup now"
- Both now use pbs-backup-manual.service
BEHAVIOR:
- Scheduled backups (via timer): Files daily, block device weekly on Sunday
- Manual backups (via menu): Files + block device ALWAYS (full backup)
FILES CHANGED:
- Added FORCE_FULL parameter to backup.sh
- Modified "both" case to check FORCE_FULL first
- Created pbs-backup-manual.service systemd unit
- Updated run_backup_now() to use manual service
- Updated menu option 4 handler to use manual service
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Updated README.md title
- Updated git clone URL in installation instructions
- Updated directory name in installation commands
- Reflects GitHub repository rename to PBSClientTool
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Added new menu option 5: "Modify backup schedule/type"
- Moved "Exit" option to option 6
- Created `reconfigure_backup_settings()` function that:
- Preserves connection details (server, credentials)
- Re-prompts for backup type, paths, schedule, and retention
- Regenerates systemd service with new settings
- Restarts backup timer
- Shows service status and next scheduled backup time
- Allows changing from file-level to block device backups (or vice versa) without reinstalling
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Changed process management in backup monitoring
- Monitor process now runs in background and kills journalctl when service completes
- journalctl runs in foreground (easier to kill properly)
- Fixes issue where logs would stay open showing blank screen
- Applied to both "Run backup now" menu option and post-install backup
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Documentation improvements to showcase the new live backup progress
monitoring feature and make it easier for users to understand how
to run and monitor backups.
README.md Updates:
Step 13 - Optional Immediate Backup:
- Added example output showing live progress monitoring
- Shows the progress box, real-time stats, and completion
- Example includes file counts, transfer speeds, and timing
- Explains what users will see during backup
- Notes ability to exit with Ctrl+C
Reconfiguration Section:
- Updated to numbered list format for clarity
- Added option 4: "Run backup now" with description
- Explains what "Run backup now" provides:
* Immediate backup testing
* Live progress monitoring with real-time statistics
* File counts, transfer speeds, compression ratios
* Automatic completion detection
* Option to exit early (backup continues)
Post-Installation → Manual Backup:
- Added "Easy way (with live progress)" section
- Shows how to use installer option 4 for backups
- Lists benefits: live monitoring, auto-completion, clear status
- Kept "Direct command" section for advanced users
- Makes it clear which method provides better UX
CHANGELOG.md Updates:
- Documented README changes in "Changed" section
- Listed all three sections that were updated
- Notes recommendation of easy method via installer
Impact:
Users now have clear documentation showing:
1. What live progress looks like during backups
2. How to access it (option 4 in main menu)
3. Why it's better than manual systemctl commands
4. What information they'll see in real-time
This makes the feature discoverable and encourages users to
use the built-in progress monitoring instead of manual log following.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
User Experience Enhancement: Show real-time backup progress instead
of just asking if user wants to follow logs.
Changes to Backup Progress Display:
1. Automatic Progress Monitoring:
- No longer prompts to follow logs
- Automatically starts showing journalctl output
- Displays PBS client's built-in progress bars and stats
- Runs in background with PID tracking
2. Visual Progress Box:
╔════════════════════════════════════════════════════════╗
║ Backup Progress (Live) ║
║ Press Ctrl+C to exit (backup continues in background) ║
╚════════════════════════════════════════════════════════╝
3. Automatic Completion Detection:
- Monitors systemd service with `systemctl is-active`
- Polls every 2 seconds until service completes
- Kills journal follow process when done
- Shows final status (success or failure)
4. Helpful Post-Backup Information:
- On success: Shows how to list snapshots
- On failure: Shows how to check full logs
- Includes repository info for easy copy/paste
Applied to Two Functions:
- run_backup_now() - Called after initial installation
- Main menu option 4 - "Run backup now"
Technical Implementation:
- Uses `timeout 3600` to prevent runaway journal process
- Background journal follow with PID capture
- Clean process termination with kill + wait
- Proper error handling for failed backups
CHANGELOG.md Updates:
- Added new "Live backup progress monitoring" feature
- Listed all capabilities and use cases
This provides immediate visual feedback and shows the user
exactly what's happening during the backup process, including
file counts, transfer speeds, and compression stats from PBS client.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
CRITICAL FIX: Added robust newline stripping as a defensive measure
when writing passwords to the config file.
The Problem:
Even though prompt_password() was fixed to output to stderr, the
newline was still somehow getting into the PBS_PASSWORD variable
in some cases, causing the config file to be malformed.
The Solution (Defense in Depth):
Before writing PBS_PASSWORD to config file, explicitly strip all
newline and carriage return characters:
PBS_PASSWORD_CLEAN=$(echo -n "$PBS_PASSWORD" | tr -d '\n\r')
This ensures the password is always clean, regardless of how it
was captured or what's in the original variable.
Changes:
- pbs-client-installer.sh:
- Line 298: Added newline filter in reconfigure_connection()
- Line 593: Added newline filter in create_systemd_service()
- Both use: tr -d '\n\r' to remove \n and \r characters
CHANGELOG.md:
- Updated to document defensive filtering approach
- Lists both functions that apply the filter
This is a belt-and-suspenders approach: we fix the source
(prompt_password) AND filter at write time for maximum safety.
Impact:
Config file will now ALWAYS have single-line passwords, even if
something goes wrong with password capture. This prevents the
"authentication failed - invalid credentials" error in the
backup service.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
User Experience Improvements:
1. Changed Default Realm from "pbs" to "pam"
- "pam" is the standard realm for root authentication
- More intuitive for most users
- Matches common Proxmox setup patterns
2. Changed Default Encryption from "yes" to "no"
- Removes friction for quick testing and setup
- Users can easily opt-in during configuration
- Reduces complexity for basic backup scenarios
- Advanced users still have the option available
3. Added "Run backup now" to Main Menu
- New option 4 in the existing configuration menu
- Allows immediate backup testing without reconfiguration
- Starts backup service and offers to follow logs
- Provides helpful commands if backup fails
- Makes it easy to test backups after setup
Changes:
- pbs-client-installer.sh:
- Line 265, 349: PBS_REALM default changed to "pam"
- Line 462: ENABLE_ENCRYPTION default changed to "no"
- Lines 895-939: Added option 4 "Run backup now" with log following
- Shifted "Exit" from option 4 to option 5
CHANGELOG.md:
- Documented all three changes in "Changed" section
- Highlighted default changes in bold
This makes the installer more user-friendly for the common case
(root@pam authentication, no encryption for testing) while still
supporting advanced configurations.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
CRITICAL BUG FIX: The prompt_password() function was capturing a newline
character along with the password/token, causing the config file to have
malformed multi-line password values.
The Problem:
- prompt_password() used `echo` to print a newline after password input
- This echo was captured by command substitution: PBS_PASSWORD=$(prompt_password ...)
- Config file ended up with:
PBS_PASSWORD="
actual-token-here"
- PBS client couldn't authenticate with the malformed password
- Error: "authentication failed - invalid credentials"
The Fix:
- Changed `echo` to `echo >&2` (output to stderr)
- Stderr is not captured by command substitution
- Only the actual password is captured and stored
- Config file now correctly has: PBS_PASSWORD="actual-token-here"
File Changed: pbs-client-installer.sh
- Line 54: echo >&2 # Output newline to stderr so it doesn't get captured
CHANGELOG.md Updated:
- Documented as CRITICAL fix
- Explains the authentication failure in backup service
- Notes config file formatting fix
Impact:
- Connection test passed but backup service failed with auth error
- This affected both password and API token authentication
- Backups would fail silently on scheduled runs
- Now fixed: backups will authenticate correctly
Testing:
User should run installer again and the backup service will now work.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
DOCUMENTATION UPDATE: Added detailed step-by-step guide covering the
entire installation process from PBS server setup to completion.
README.md Changes:
Prerequisites Section (Expanded):
- How to create API tokens in PBS web interface
- How to configure datastore permissions
- Common permission error and how to fix it
- Information gathering checklist before running installer
New: Step-by-Step Walkthrough (14 Steps):
1. Running the installer
2. PBS server configuration (IP, port, datastore)
3. Authentication method selection (why API tokens)
4. Entering API token details
5. Backup type selection (files/block/hybrid)
6. File backup paths configuration
7. Block device selection and auto-detection
8. Backup schedule configuration
9. Retention policy settings
10. Encryption setup and key management
11. 3-step connection test process
12. Systemd service creation
13. Optional immediate backup
14. Completion summary
Each step includes:
- Example prompts and responses
- Explanation of options
- Recommendations and best practices
- Common device types and their paths
- Troubleshooting tips
CHANGELOG.md Updates:
- Documented new comprehensive walkthrough
- Listed PBS server setup documentation
- Added permission error troubleshooting to Added section
This addresses the most common user questions and provides a
complete reference for first-time users. Users can now follow
the guide step-by-step to successfully configure PBS backups.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
CRITICAL FIX: The PBS client was waiting for interactive SSL
certificate fingerprint confirmation, causing the installer to
timeout during authentication testing.
Changes to pbs-client-installer.sh:
- Authentication test now pipes 'y' to auto-accept SSL fingerprints
- Displays accepted fingerprint in logs for transparency
- Added SSL certificate issues to error troubleshooting
- References new test-connection.sh script in error messages
New file: test-connection.sh
- Parameterized diagnostic script for testing PBS connections
- No hardcoded credentials (security best practice)
- Handles SSL fingerprint acceptance interactively
- Tests all 3 steps: reachability, auth, datastore access
- Provides detailed error messages and guidance
- Usage examples for both password and API token auth
Updated CHANGELOG.md:
- Documented SSL fingerprint auto-acceptance as critical fix
- Listed all improvements since v1.0.0
- Highlighted test-connection.sh script addition
Updated README.md:
- Added comprehensive troubleshooting section
- Documented test-connection.sh usage with examples
- Explained 3-step connection verification process
- Added SSL fingerprint handling instructions
- Clarified timeout values for each step
This resolves the "authentication hanging" issue where the login
command was waiting indefinitely for user input on SSL fingerprint
confirmation.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Document the fix for using correct PBS client commands
(login instead of non-existent status command)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
The issue was using 'proxmox-backup-client status' which doesn't exist.
Changes:
- Use 'proxmox-backup-client login' for authentication testing
(This is the proper command to test credentials)
- Use 'proxmox-backup-client list' instead of 'snapshot list'
(Simpler command to verify datastore access)
- Reduced timeouts from 30s to 15s (these commands are faster)
- Added debug output showing the repository string being used
- Improved error message display from PBS client
This should resolve authentication timeouts and actually show
real error messages when credentials are invalid.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Block Device Detection:
- Strip btrfs subvolume notation [/@] from detected device path
- Validate detected device before showing it as default
- Show available block devices when invalid device is entered
- Better error messages for device detection issues
Connection Test Improvements:
- 3-step verification process for better diagnostics:
1. Server reachability test (5s timeout with curl)
2. Authentication test with 'status' command (30s timeout)
3. Datastore access verification (optional)
- Step-by-step feedback shows exactly where the failure occurs
- More specific error messages based on failure point
- Shows actual PBS client errors when authentication fails
- Succeeds if authentication works, even if snapshot list fails
(normal when no backups exist yet)
This resolves:
- Invalid device paths like /dev/mapper/root[/@]
- Connection timeouts that were actually authentication issues
- Better user experience with clear progress indicators
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Added:
- CHANGELOG.md following Keep a Changelog format
- Documented all changes from v1.0.0 to current unreleased version
- README section explaining reconfiguration options
- Enhanced troubleshooting section with connection test details
The CHANGELOG documents:
- Reconfiguration feature for existing installations
- Connection test timeout and improved error messages
- Installation method change from wget to git clone
The README now clearly explains:
- How to reconfigure existing installations
- Connection-only vs full reconfiguration options
- Use cases for quick connection updates
- Connection test timeout behavior and error types
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
The connection test now:
- Has a 30-second timeout to prevent indefinite hanging
- Shows "This may take up to 30 seconds..." message
- Differentiates between timeout (unreachable) and auth errors
- Provides specific troubleshooting steps based on error type
- Displays helpful commands for diagnosing connection issues
This fixes the issue where the script would hang indefinitely
when the PBS server was unreachable.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
When PBS client is already installed, the script now offers:
- Reconfigure connection only (server/credentials) - quick option
- Full reconfiguration (all settings)
- Reinstall PBS client and reconfigure
- Exit without changes
The connection-only reconfiguration preserves all backup settings
(paths, schedules, retention) and only updates PBS server details
and credentials. This is useful when switching backup servers or
updating authentication tokens.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Replace wget command with git clone for better version control
and easier updates. The previous wget URL was also malformed.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Added a comprehensive quick start guide for installing and configuring the PBS client, including installation commands, setup instructions, and troubleshooting tips.
Expanded README with detailed installation instructions, usage examples, troubleshooting tips, and security considerations for the PBS Client Installer.
