Commit graph

271 commits

Author SHA1 Message Date
rcourtman
53dec6010c fix(sensor-proxy): eliminate allowed_nodes config corruption
Phase 1 hotfix to address recurring config file corruption that causes
99% of temperature monitoring failures. The root cause was the installer
oscillating between inline and file-based allowlist modes, creating
duplicate `allowed_nodes:` keys in config.yaml.

Changes:
- Force file-based allowlist mode exclusively (refuse versions < v4.31.1)
- Add automatic migration from inline to file-based config
- Remove inline mode code path from update_allowed_nodes()
- Migration runs on every install/self-heal to clean up existing corruption

The self-heal timer runs every 5 minutes and was the primary source of
corruption when version detection failed or encountered edge cases.

This eliminates the dual code paths and ensures config.yaml is never
edited for allowlist changes - only /etc/pulse-sensor-proxy/allowed_nodes.yaml
is modified.

Phase 2 (next release) will implement proper Go-based config management
with atomic writes, locking, and systemd validation to prevent corruption
by design.

Related to recurring temperature monitoring outages
2025-11-19 09:21:54 +00:00
rcourtman
6b5ae18dfe fix: sanitize sensor proxy config during self-heal
Related to #714.
2025-11-18 22:51:40 +00:00
rcourtman
d93a2c1053 Improve host agent binary handling and docker installer purge (Related to #693) 2025-11-18 22:11:44 +00:00
rcourtman
4a8f743d02 Deduplicate inline proxy allow list 2025-11-18 14:58:50 +00:00
rcourtman
54c47fe2f5 Improve temperature proxy detection 2025-11-18 14:25:09 +00:00
rcourtman
91dbe31ea5 Include control-plane allow list in proxy config 2025-11-18 10:42:13 +00:00
rcourtman
7fa39467c0 Fix setup-script tokens and proxy registration timing 2025-11-18 10:22:54 +00:00
rcourtman
f7261f3080 Harden turnkey install and proxy auto-registration 2025-11-18 00:24:50 +00:00
rcourtman
ce595b69fa Avoid blocking self-heal start during install 2025-11-17 23:14:51 +00:00
rcourtman
3ca2bf308e Break self-heal recursion when proxy unregistered 2025-11-17 23:01:57 +00:00
rcourtman
94a0c82439 Fix inline allowed_nodes cleanup 2025-11-17 22:50:25 +00:00
rcourtman
09ee968422 Improve temp proxy install UX 2025-11-17 22:30:32 +00:00
rcourtman
011c7094b2 Restore pending control-plane helpers 2025-11-17 22:04:30 +00:00
rcourtman
16bd34513e Fix pending control-plane helpers 2025-11-17 22:01:11 +00:00
rcourtman
d08322e5f8 Speed up proxy self-heal reconciliation 2025-11-17 21:56:21 +00:00
rcourtman
75d231bf53 Add automatic control-plane reconciliation 2025-11-17 21:55:47 +00:00
rcourtman
2a3ee72b7f Improve sensor proxy installer compatibility 2025-11-17 21:38:28 +00:00
rcourtman
f0f44231a1 Fix proxy install summary and allowed_nodes cleanup 2025-11-17 14:38:01 +00:00
rcourtman
430c98c5ef Move allowed_nodes to managed file 2025-11-16 10:06:58 +00:00
rcourtman
5951a364f7 Auto-deploy proxy for standalone temp monitoring 2025-11-16 09:47:07 +00:00
rcourtman
e375a21d4b Fix backup UX and proxy config dedupe 2025-11-15 23:26:44 +00:00
rcourtman
3223a3a432 Fix allowed_nodes sanitizer indentation handling 2025-11-15 22:42:08 +00:00
rcourtman
2452674c3c Rewrite proxy allowed_nodes sanitizer 2025-11-15 22:34:29 +00:00
rcourtman
daa70280f5 Gracefully handle missing PVE instance during proxy registration 2025-11-15 22:25:50 +00:00
rcourtman
871c2988ff Fix proxy installer dedupe 2025-11-15 22:04:36 +00:00
rcourtman
de5b314842 Improve temperature proxy control-plane flow 2025-11-15 21:49:51 +00:00
rcourtman
1b37de22f6 Ensure sensor proxy installer configures Pulse env 2025-11-15 18:28:42 +00:00
rcourtman
fa4d5de7bd Ensure sensor proxy installer configures Pulse env 2025-11-15 18:23:40 +00:00
rcourtman
671703a0f4 Deduplicate allowed_nodes when installing sensor proxy 2025-11-15 18:14:38 +00:00
rcourtman
dd9089b26c Fix installer adding invalid hostname entries to allowed_nodes
The installer was adding node hostnames (and accidentally the header "Name")
to allowed_nodes in addition to IPs. This caused:
1. Invalid entries like "Name", "minipc", "delly" in config
2. These are not valid for SSH temperature collection

Only IPs should be in allowed_nodes since that's what the proxy uses for SSH.
Removed the loop that added CLUSTER_NODE_NAMES to the array.

Also fixed: Removed extraction of CLUSTER_NODE_NAMES since it's no longer used.
2025-11-15 10:07:22 +00:00
rcourtman
ec2bd8daf6 Fix cleanup systemd-run deadlock
Problem:
Cleanup script uses systemd-run with both --wait and
--property="After=pulse-sensor-cleanup.service", creating a circular
dependency:
- cleanup.service runs and waits for uninstaller to complete
- uninstaller has After=cleanup.service, so it waits for cleanup to finish
- Result: Both services stuck waiting for each other

Fix:
Remove the --property="After=pulse-sensor-cleanup.service" line. The
Conflicts=pulse-sensor-proxy.service is sufficient to ensure the proxy
stops before uninstallation. The cleanup script doesn't need to finish
before the uninstaller starts.

Testing:
Cleanup now completes successfully, removing all artifacts:
- Systemd units removed
- Binaries deleted from /opt/pulse/sensor-proxy/
- Data directory /var/lib/pulse-sensor-proxy/ removed
- SSH keys cleaned from authorized_keys
- pulse-monitor user and API tokens deleted
- LXC bind mounts removed from container configs

Related to #605 (temperature monitoring cleanup)
2025-11-15 09:03:17 +00:00
rcourtman
fbfd086724 Fix missed /usr/local path migration and add backward compatibility
**Missed Migration**:
- Line 2204 still used /usr/local/bin/pulse-sensor-wrapper.sh in fallback path
- Updated to use /opt/pulse/sensor-proxy/bin/pulse-sensor-wrapper.sh

**Backward Compatibility**:
- When pushing SSH keys to cluster nodes, installer now checks if remote node
  has old installation (/usr/local/bin wrapper exists but /opt path doesn't)
- Automatically creates symlink on remote nodes to maintain compatibility
- Prevents temperature collection failures when cluster has mixed old/new installs

**Root Cause**:
When installer runs on upgraded node (delly), it pushes SSH keys with new forced
command path to all cluster nodes. If remote node (minipc) has old installation,
the forced command fails because wrapper doesn't exist at new path.

This fix ensures "it works straight out the box" by bridging old and new paths
automatically during SSH key deployment.
2025-11-15 08:37:44 +00:00
rcourtman
5b7d1726d7 Fix update_allowed_nodes to be properly idempotent
Rewrote AWK state machine to correctly handle:
- Multiple allowed_nodes sections (removes all of them)
- Comment lines immediately preceding allowed_nodes (discards them)
- Empty lines within allowed_nodes section
- Indented list items and comments

The function now:
1. Buffers comment lines that might precede allowed_nodes
2. When allowed_nodes: is detected, discards buffered comments
3. Skips all content until hitting a non-indented, non-comment line
4. Flushes buffered comments when hitting non-comment content

This ensures running the installer multiple times won't create duplicate
allowed_nodes sections in config.yaml.

Tested with script that verifies duplicate sections are removed correctly.
2025-11-15 08:03:28 +00:00
rcourtman
fe53d64730 Address remaining Codex review findings
**LXC Bind Mount Removal**:
- Changed from sed to `pct set -delete` for safer mount removal
- Validates syntax and prevents breaking container configs
- Finds mount points by grepping for pulse-sensor-proxy, extracts mp number

**API Token Parsing** (three-tier fallback):
1. Try `pveum --output-format json` with python3 JSON parsing
2. Fall back to `pvesh get /access/users/pulse-monitor@pam/token` JSON API
3. Last resort: parse table output with improved filtering (handles more Unicode chars)

**Retry Logic**:
- Rename cleanup-request.json to .processing instead of deleting immediately
- Allows retry on failure (processing file persists if script crashes)
- Remove .processing file only on successful completion
- Prevents loops while enabling failure recovery

These complete all 8 issues identified by Codex review.
2025-11-15 00:35:22 +00:00
rcourtman
bcd8d4e0fa Fix critical cleanup implementation issues found by Codex review
**Host Detection**:
- Now detects localhost by hostname and FQDN, not just IP
- Fixes issue where nodes configured as https://hostname:8006 would skip
  localhost cleanup (API tokens, bind mounts, service removal)

**Systemd Sandbox**:
- Added /etc/pve and /etc/systemd/system to ReadWritePaths
- Allows cleanup script to modify Proxmox configs and systemd units

**Uninstaller Improvements**:
- Use UUID for transient unit names (prevents same-second collisions)
- Added --purge flag for complete removal
- Added --wait and --collect flags to capture exit code
- Now fails cleanup if uninstaller exits non-zero

**Path Migration**:
- Fixed all /usr/local references to use /opt/pulse/sensor-proxy
- Updated forced command in SSH authorized_keys
- Updated self-heal script installer path
- Updated Go backend removal helpers (supports both new and legacy paths)

These fixes address Codex findings: hostname detection, sandbox permissions,
transient unit collisions, incomplete purging, and incomplete path migration.

Related to cleanup implementation testing.
2025-11-15 00:33:41 +00:00
rcourtman
17d2e6876e Fix unbound SHARE_DIR variable in installer
After relocating binaries to INSTALL_ROOT, the SHARE_DIR variable was removed
but one reference remained in cache_installer_for_self_heal() causing
'unbound variable' error.

Changed to use INSTALL_ROOT directly since that's where the cached installer
is stored (STORED_INSTALLER=${INSTALL_ROOT}/install-sensor-proxy.sh).
2025-11-15 00:09:34 +00:00
rcourtman
ed48d7555a Fix directory creation order in sensor-proxy installer
The installer was trying to write binaries to /opt/pulse/sensor-proxy/bin/
before creating the directory structure, causing 'No such file or directory'
errors on fresh installs.

Moved directory creation for INSTALL_ROOT and bin/ to before binary installation
section (before line 657), ensuring directories exist before use.

Related to cleanup implementation testing.
2025-11-15 00:08:25 +00:00
rcourtman
6692228e0f Implement full cleanup when nodes are removed from Pulse
Extends cleanup script to completely remove Pulse footprint from hosts
when nodes are removed, not just SSH keys. Now removes: SSH keys, proxy
service, binaries, API tokens, pulse-monitor user, and LXC bind mounts.

Key improvements:

1. **flock Serialization**: Prevents concurrent cleanup runs
   - Acquires exclusive lock on cleanup.lock file
   - Prevents race conditions and cleanup loops

2. **Immediate Request File Deletion**: Delete cleanup-request.json
   before any long-running operations to prevent re-triggering

3. **API Token Cleanup**: Removes all pulse-monitor@pam API tokens
   - Tries JSON output first (Proxmox 7.0+)
   - Falls back to table parsing with proper filtering (no decoration chars)
   - Deletes pulse-monitor@pam user after removing all tokens

4. **LXC Bind Mount Removal**: Scans all container configs and removes
   pulse-sensor-proxy bind mount entries

5. **Process Isolation for Uninstaller**: Uses systemd-run to spawn
   isolated transient unit that won't be killed when proxy service stops
   - Unit name: pulse-uninstall-{timestamp}
   - Properties: Type=oneshot, Conflicts=pulse-sensor-proxy.service
   - Runs non-blocking so cleanup service can exit cleanly
   - Falls back to direct call if systemd-run unavailable

6. **Complete Service/Binary Removal**: Calls installer's --uninstall
   - Stops and disables pulse-sensor-proxy.service
   - Removes all systemd units
   - Deletes all binaries from /opt/pulse/sensor-proxy/
   - Removes configuration files
   - Cleans up directories

Changes to cleanup script logic:
- Added LOCKFILE and INSTALLER_PATH configuration
- Acquire flock before processing (prevents concurrent runs)
- Delete request file immediately after reading
- Full localhost cleanup: SSH keys → API tokens → bind mounts → uninstall
- Remote cleanup still SSH-key-only (can't orchestrate uninstall remotely)
- Better error handling with appropriate log levels

Updated cleanup service unit:
- ExecStart now uses ${CLEANUP_SCRIPT_PATH} variable (new /opt location)
- Changed heredoc from 'SERVICE_EOF' to SERVICE_EOF for variable expansion

Addresses all issues documented in CLEANUP_TODO.md:
-  Read-only filesystem (binaries now in /opt, removable)
-  Process isolation (systemd-run transient unit)
-  Cleanup loops (flock + immediate file deletion)
-  API token parsing (JSON first, filtered table fallback)

The UI message is now accurate: "Removing this proxmox ve node also
scrubs the Pulse footprint on the host — the proxy service, SSH key,
API token, and bind mount are all cleaned up automatically."

Part of: CLEANUP_TODO.md Phase 2-4
Supersedes: ed65fda74 (original cleanup attempt with process issues)
Depends on: b192c60e9 (binary relocation to /opt)
2025-11-15 00:03:09 +00:00
rcourtman
b192c60e90 Relocate sensor-proxy binaries to /opt for guaranteed cleanup
Moves all sensor-proxy binaries and scripts from /usr/local/bin to
/opt/pulse/sensor-proxy/bin to ensure they can be removed during cleanup
even on systems with read-only /usr (hardened Proxmox setups).

Changes:
- INSTALL_ROOT=/opt/pulse/sensor-proxy (new writable location)
- Binary path: /opt/pulse/sensor-proxy/bin/pulse-sensor-proxy
- Wrapper script: /opt/pulse/sensor-proxy/bin/pulse-sensor-wrapper.sh
- Cleanup script: /opt/pulse/sensor-proxy/bin/pulse-sensor-cleanup.sh
- Selfheal script: /opt/pulse/sensor-proxy/bin/pulse-sensor-proxy-selfheal.sh
- Installer storage: /opt/pulse/sensor-proxy/install-sensor-proxy.sh

Updated:
- Directory creation to include ${INSTALL_ROOT}/bin
- Systemd service ExecStart paths to use ${BINARY_PATH}
- Self-heal service ExecStart to use ${SELFHEAL_SCRIPT}
- Changed heredoc delimiters from 'EOF' to EOF for variable expansion

Rationale:
Proxmox VE can mount /usr as read-only in hardened configurations.
The previous /usr/local/bin location prevented complete uninstallation
on these systems, violating Pulse's correctness principle. The /opt
location is guaranteed writable and appropriate for third-party software.

This is Phase 1 of implementing full cleanup functionality per
CLEANUP_TODO.md. Subsequent commits will add process isolation,
API token deletion, and bind mount removal.

Part of: #CLEANUP_TODO.md Phase 1
Related: ed65fda74 (original cleanup attempt)
2025-11-14 23:59:25 +00:00
rcourtman
8204b54051 Fix permission denied error when updating allowed_nodes
The update_allowed_nodes function was changing ownership of the temp file
before all writes were complete, causing 'Permission denied' errors when
appending the allowed_nodes section.

Root cause:
- mktemp creates file owned by script runner (root)
- chown changed ownership to pulse-sensor-proxy:pulse-sensor-proxy
- Subsequent append (>>) failed because root can't write to the file

Fix: Defer chown until after all writes complete and file is moved to
final location. Ownership is still correctly set on the final config file.
2025-11-14 23:21:46 +00:00
rcourtman
ed65fda74d Extend node cleanup to fully remove Pulse footprint
When a Proxmox node is removed from Pulse, the cleanup now performs full uninstallation:

- SSH keys removal (existing functionality)
- Uninstalls pulse-sensor-proxy service
- Removes LXC bind mounts from container configs
- Deletes Proxmox API tokens
- Removes pulse-monitor@pam user

This aligns with security best practices and user expectations - "remove node"
should completely sever trust with that machine, not leave credentials and
privileged services behind.

The cleanup script now calls the uninstaller (--uninstall) and uses pveum
to remove API tokens. This prevents leftover artifacts if the host is
repurposed or compromised.

Related: config_handlers.go triggerPVEHostCleanup() at node deletion
2025-11-14 22:58:50 +00:00
rcourtman
c5a5eb2534 Use unique temp file prefix to avoid permission conflicts
- Change mktemp to use /tmp/pulse-config.XXXXXXXXXX template
- Prevents conflicts with stale temp files from previous runs
- Fixes 'Permission denied' errors when script re-runs
2025-11-14 22:44:33 +00:00
rcourtman
3a381e764a Add cleanup trap and better error handling to update_allowed_nodes
- Add trap to remove temp file on function return (success or failure)
- Add error check for mv command with descriptive message
- Ensure config file has proper permissions after update

This prevents orphaned temp files when errors occur and provides
better diagnostics when file operations fail.
2025-11-14 22:13:35 +00:00
rcourtman
d0080e4adf Remove invalid 'local' keyword from main script scope
The all_nodes arrays were declared with 'local' keyword outside of
functions, causing bash syntax error:
  'local: can only be used in a function'

Fixed by removing 'local' keyword - arrays in main script scope don't
need it and it's actually invalid syntax.
2025-11-14 22:06:44 +00:00
rcourtman
ddf0733353 Fix update_allowed_nodes to remove comment headers
The awk logic was removing allowed_nodes sections but leaving their
comment headers behind. When multiple sections existed, comments would
accumulate.

New approach:
- Buffer all comment lines encountered outside sections
- When a non-comment line is found, flush buffered comments
- When allowed_nodes is found, discard buffered comments (they belonged
  to the section we're removing)
- This cleanly removes section headers like:
  '# Cluster nodes (auto-discovered during installation)'
  '# These nodes are allowed to request...'

Tested with config containing duplicate allowed_nodes sections - now
correctly produces clean output with all duplicates and headers removed.
2025-11-14 22:02:10 +00:00
rcourtman
245d6f5b47 Ensure base config exists before updating allowed_nodes
The installer was only creating base config.yaml in standalone mode,
but update_allowed_nodes() is also called in LXC mode. When the config
didn't exist, update_allowed_nodes() would create an empty file and only
add the allowed_nodes section, missing required fields like
allowed_peer_uids, metrics_address, rate_limit, etc.

This caused the proxy to fail when it tried to parse the incomplete config.

Now creates a proper base config with all required fields if the file
doesn't exist, before any mode-specific configuration is added.
2025-11-14 21:57:16 +00:00
rcourtman
f7936cb540 Fix duplicate allowed_nodes in sensor proxy installer
The install-sensor-proxy.sh script was blindly appending allowed_nodes
sections to the config file without checking if they already existed.
When the script was re-run or if the initial config already had an
allowed_nodes section, this created duplicate YAML keys that caused
the proxy service to fail with parse errors.

Changes:
- Add update_allowed_nodes() helper function that safely updates the
  allowed_nodes section by removing any existing ones first
- Replace all three cat >> config.yaml heredocs with calls to the
  helper function (cluster nodes, standalone mode, pvecm fallback)
- Uses awk to properly parse and remove multi-line YAML sections

This makes the installer idempotent and prevents config corruption on
re-runs.

Fixes issue where proxy service crashed with:
  'mapping key "allowed_nodes" already defined at line X'
2025-11-14 21:43:31 +00:00
rcourtman
e708727725 Improve sensor proxy socket verification 2025-11-14 20:14:25 +00:00
rcourtman
1d0d23d237 Related to #710: harden Windows installer arch detection 2025-11-14 10:50:56 +00:00
rcourtman
1ed69d35b6 docs: highlight runbooks in index and script verification checklist 2025-11-14 10:39:10 +00:00
rcourtman
4d021f8a50 Related to #701: improve sensor proxy installer caching 2025-11-14 00:51:54 +00:00
rcourtman
4f8b807f0c Refine Windows host installer logging (related to #709) 2025-11-13 23:09:22 +00:00
rcourtman
70673c1fdc Improve temperature proxy diagnostics and tests 2025-11-13 22:31:53 +00:00
rcourtman
9cb151d836 Add localhost to allowed_source_subnets for self-monitoring
The HTTP mode installer now includes 127.0.0.1/32 in allowed_source_subnets
to permit self-monitoring queries from localhost. This fixes 403 Forbidden
errors when nodes query their own sensor-proxy instance.

Related to HTTP mode implementation for external PVE hosts.
2025-11-13 19:34:11 +00:00
rcourtman
e2bd514899 Fix HTTP mode for pulse-sensor-proxy and improve installer safety
## HTTP Server Fixes
- Add source IP middleware to enforce allowed_source_subnets
- Fix missing source subnet validation for external HTTP requests
- HTTP health endpoint now respects subnet restrictions

## Installer Improvements
- Auto-configure allowed_source_subnets with Pulse server IP
- Add cluster node hostnames to allowed_nodes (not just IPs)
- Fix node validation to accept both hostnames and IPs
- Add Pulse server reachability check before installation
- Add port availability check for HTTP mode
- Add automatic rollback on service startup failure
- Add HTTP endpoint health check after installation
- Fix config backup and deduplication (prevent duplicate keys)
- Fix IPv4 validation with loopback rejection
- Improve registration retry logic with detailed errors
- Add automatic LXC bind mount cleanup on uninstall

## Temperature Collection Fixes
- Add local temperature collection for self-monitoring nodes
- Fix node identifier matching (use hostname not SSH host)
- Fix JSON double-encoding in HTTP client response

Related to #XXX (temperature monitoring fixes)
2025-11-13 18:22:36 +00:00
rcourtman
71217dfae4 Add HTTP mode support to sensor-proxy installer
Implements complete HTTP mode installation workflow for external PVE hosts.

New installer features:
- `--http-mode` flag: Enable HTTP server mode for remote temperature monitoring
- `--http-addr <addr>` flag: Configure listen address (default :8443)
- Auto-generates self-signed TLS certificates (4096-bit RSA, 10-year validity)
- Registers with Pulse API and receives authentication token
- Configures systemd service with proper security hardening

Installation workflow (HTTP mode):
1. Validate --pulse-server parameter is provided
2. Generate TLS certificate with SAN (hostname + IPs)
3. Call Pulse API POST /api/temperature-proxy/register
4. Receive and store auth token securely (mode 600)
5. Append HTTP config to config.yaml
6. Update systemd service with TLS paths
7. Start service

TLS certificate generation:
- Uses openssl req with RSA 4096-bit keys
- 10-year validity period
- SubjectAltName includes hostname + all IPs
- Files stored in /etc/pulse-sensor-proxy/tls/
- Permissions: 640 root:pulse-sensor-proxy
- Logs SHA256 fingerprint for audit

API registration:
- Calls POST /api/temperature-proxy/register
- Payload: {"hostname": "...", "proxy_url": "https://..."}
- Response: {"token": "...", "pve_instance": "..."}
- Aborts installation on registration failure (fail-fast)
- Token stored in config.yaml

Systemd service updates:
- Adds ReadOnlyPaths=/etc/pulse-sensor-proxy/tls for HTTP mode
- RestrictAddressFamilies already includes AF_INET/AF_INET6
- Maintains all existing security hardening

Error handling:
- Validates required parameters before starting
- Aborts on TLS generation failure
- Aborts on API registration failure
- Provides actionable troubleshooting guidance
- Logs clear error messages

Security:
- Tokens stored with mode 600, owned by service user
- TLS keys protected with mode 640
- Service runs as unprivileged pulse-sensor-proxy user
- Full systemd hardening maintained

Usage example:
  curl -fsSL https://pulse-server/download/install-sensor-proxy.sh | \
    bash -s -- --http-mode --pulse-server https://pulse.example.com:7655

Related to #571
2025-11-13 16:33:12 +00:00
rcourtman
cd9d63d464 Fix #571: Installer now auto-configures allowed_nodes for temperature monitoring
When cluster IPC validation fails (due to systemd hardening), the proxy
falls back to allowlist-based validation. The installer now automatically
populates allowed_nodes with:

- Cluster mode: All discovered cluster member IPs
- Standalone mode: localhost IP addresses (including 127.0.0.1/localhost)
- Fallback mode: localhost IPs when pvecm unavailable

This ensures out-of-the-box temperature monitoring works on fresh installs
without manual configuration.
2025-11-13 15:37:30 +00:00
rcourtman
5ef6ca16fe Fix temperature monitoring for clustered and LXC Proxmox environments (addresses #571)
Root cause: pulse-sensor-proxy runs with strict systemd hardening that prevents
access to Proxmox corosync IPC (abstract UNIX sockets). When pvecm fails with
IPC errors, the code incorrectly treated it as "standalone mode" and only
discovered localhost addresses, rejecting legitimate cluster members and external
nodes.

Changes:

1. **Distinguish IPC failures from true standalone mode**
   - Detect ipcc_send_rec and access control list errors specifically
   - These indicate a cluster exists but isn't accessible (LXC, systemd restrictions)
   - Return error to disable cluster validation instead of misusing standalone logic

2. **Graceful degradation when cluster validation fails**
   - When cluster IPC is unavailable, fall through to permissive mode
   - Log debug message suggesting allowed_nodes configuration
   - Allows requests to proceed rather than blocking all temperature monitoring

3. **Improve local address discovery for true standalone nodes**
   - Use Go's native net.Interfaces() instead of shelling out to 'ip addr'
   - More reliable and works with AF_NETLINK restrictions
   - Add helpful logging when only hostnames are discovered

4. **Systemd hardening adjustments**
   - Add AF_NETLINK to RestrictAddressFamilies (for net.Interfaces())
   - Remove RemoveIPC=true (attempted fix for corosync, insufficient)
   - Add ReadWritePaths=-/run/corosync (optional path, corosync uses abstract sockets anyway)

Result: Temperature monitoring now works in:
- Clustered Proxmox hosts (falls back to permissive when IPC blocked)
- LXC containers (correctly detects IPC failure, allows requests)
- Standalone nodes (proper local address discovery with IPs)

Workaround for maximum security: Configure allowed_nodes in /etc/pulse-sensor-proxy/config.yaml
when cluster validation cannot be used.
2025-11-13 13:25:27 +00:00
rcourtman
ca8cc0844a Fix temperature monitoring on standalone Proxmox nodes (addresses #571)
Root cause: The systemd service hardening blocked AF_NETLINK sockets,
preventing IP address discovery on standalone nodes. The proxy could
only discover hostnames, causing node_not_cluster_member rejections
when users configured Pulse with IP addresses.

Changes:
1. Add AF_NETLINK to RestrictAddressFamilies in all systemd services
   - pulse-sensor-proxy.service
   - install-sensor-proxy.sh (both modes)
   - pulse-sensor-cleanup.service

2. Replace shell-based 'ip addr' with Go native net.Interfaces() API
   - More reliable and doesn't require external commands
   - Works even with strict systemd restrictions
   - Properly filters loopback, link-local, and down interfaces

3. Improve error logging and user guidance
   - Warn when no IP addresses can be discovered
   - Provide clear instructions about allowed_nodes workaround
   - Include address counts in logs for debugging

This fix ensures standalone Proxmox nodes can properly validate
temperature requests by IP address without requiring manual
allowed_nodes configuration.
2025-11-13 13:02:15 +00:00
rcourtman
e822ab7ae1 Fix remote sync check in release trigger script
- Replace unreliable git fetch --dry-run check
- Use git rev-parse to compare local and remote commits
- Prevents false warnings about diverged branches
2025-11-13 11:43:36 +00:00
rcourtman
d400befd35 Add pre-flight validation script for releases
- Check VERSION file matches before triggering workflow
- Validate working directory is clean
- Confirm on main branch and up to date
- Load release notes from /tmp/release_notes_X.Y.Z.md
- Prevents wasting CI time on misconfigured releases
2025-11-13 11:36:53 +00:00
rcourtman
1dd0e74133 Dramatically improve temperature proxy installation robustness
Users were abandoning Pulse due to catastrophic temperature monitoring setup failures. This commit addresses the root causes:

**Problem 1: Silent Failures**
- Installations reported "SUCCESS" even when proxy never started
- UI showed green checkmarks with no temperature data
- Zero feedback when things went wrong

**Problem 2: Missing Diagnostics**
- Service failures logged only in journald
- Users saw "Something going on with the proxy" with no actionable guidance
- No way to troubleshoot from error messages

**Problem 3: Standalone Node Issues**
- Proxy daemon logged continuous pvecm errors as warnings
- "ipcc_send_rec" and "Unknown error -1" messages confused users
- These are expected for non-clustered/LXC setups

**Solutions Implemented:**

1. **Health Gate in install.sh (lines 1588-1629)**
   - Verify service is running after installation
   - Check socket exists on host
   - Confirm socket visible inside container via bind mount
   - Fail loudly with specific diagnostics if any check fails

2. **Actionable Error Messages in install-sensor-proxy.sh (lines 822-877)**
   - When service fails to start: dump full systemctl status + 40 lines of logs
   - When socket missing: show permissions, service status, and remediation command
   - Include common issues checklist (missing user, permission errors, lm-sensors, etc.)
   - Direct link to troubleshooting docs

3. **Better Standalone Node Detection in ssh.go (lines 585-595)**
   - Recognize "Unknown error -1" and "Unable to load access control list" as LXC indicators
   - Log at INFO level (not WARN) since this is expected behavior
   - Clarify message: "using localhost for temperature collection"

**Impact:**
- Eliminates "green checkmark but no temps" scenario
- Users get immediate actionable feedback on failures
- Standalone/LXC installations work silently without error spam
- Reduces support burden from #571 (15+ comments of user frustration)

Related to #571
2025-11-13 10:14:19 +00:00
rcourtman
221c41b2b0 Polish release notes fallback 2025-11-13 09:10:43 +00:00
rcourtman
8330664ed2 Add deterministic release notes fallback 2025-11-13 00:00:25 +00:00
rcourtman
a6a8f0a8ef Improve release notes fallback 2025-11-12 23:40:26 +00:00
rcourtman
92e155ed3f Handle Snap Docker home restrictions (Related to #693) 2025-11-12 19:20:04 +00:00
rcourtman
b829d86d2c Handle docker validation under errexit
Related to #693
2025-11-12 18:34:53 +00:00
rcourtman
b3e8e43736 Related to #698: harden installer release detection 2025-11-12 17:56:16 +00:00
rcourtman
9dd6357328 Improve sensor-proxy release detection (related to #701) 2025-11-12 17:49:20 +00:00
rcourtman
92a5d74ba9 Add Snap Docker support to install-docker-agent.sh
Snap-installed Docker does not automatically create a docker group,
causing permission denied errors when the pulse-docker service user
tries to access /var/run/docker.sock.

Changes:
- Auto-detect Snap Docker installations
- Create docker group if missing when Snap Docker is detected
- Restart Snap Docker after group creation to refresh socket ACLs
- Add socket access validation before starting the service
- Handle symlinked Docker sockets in systemd unit ReadWritePaths
- Document troubleshooting steps in DOCKER_MONITORING.md
2025-11-11 23:07:29 +00:00
rcourtman
ea6cad10ce Release workflow guardrails (related to #695) 2025-11-11 22:34:00 +00:00
rcourtman
135b378820 Fix Windows/macOS host agent downloads for bare metal installs (related to #684)
Bare metal installations couldn't serve Windows host agent downloads because
the Windows and macOS binaries weren't included in the universal tarball. The
download endpoint would return 404 when Windows users tried to install the
host agent from a bare metal Pulse deployment (Proxmox LXC, Debian VM, etc.).

Changes:
- build-release.sh: Copy Windows/macOS host agent binaries into universal tarball
- build-release.sh: Create symlinks for Windows binaries without .exe extension
- validate-release.sh: Add Windows 386 binary and symlink to Docker validation
- validate-release.sh: Add explicit validation that universal tarball contains all Windows/macOS binaries

The universal tarball now matches the Docker image, ensuring both deployment
methods can serve the complete set of downloadable binaries for the /download/
endpoint.
2025-11-11 21:26:33 +00:00
rcourtman
e5dfca6c88 Fix SELinux compatibility in host agent installer
Replace mv with install command to ensure correct SELinux context.
The mv command preserves the user_tmp_t label from /tmp, which
prevents systemd from executing the binary on SELinux systems.

The install command creates a new file with the correct label for
/usr/local/bin. Added automatic restorecon call for SELinux systems
to ensure policy compliance.

Related to #688
2025-11-11 21:13:33 +00:00
rcourtman
34b29610e7 Generate both checksums.txt and .sha256 files for backward compatibility
Following best practices for release format transitions:
- build-release.sh now generates both formats from same sha256sum run
- Workflow uploads both checksums.txt and individual .sha256 files
- Validation ensures both formats exist and match

This provides a safe transition period for users with older install scripts
while maintaining the cleaner checksums.txt format going forward. After 2-3
releases when most users have updated scripts, we can remove .sha256 generation.

Related: Install script already supports both formats (falls back gracefully).
2025-11-11 20:31:15 +00:00
rcourtman
0f78b681c8 Fix validation: Linux host-agent binaries are in main tarballs
Linux host-agent binaries don't have separate archives - they're included in
the main pulse-v*.tar.gz files. Only macOS and Windows have separate archives.
2025-11-11 19:25:14 +00:00
rcourtman
e54f881eea Update validation script to match new asset list
Removed validation checks for standalone binaries that are no longer
uploaded to GitHub releases. These binaries are only needed in Docker
images for the /download/ endpoint.

Updated required assets list to include all versioned tarballs/zips
instead of standalone binaries.
2025-11-11 17:50:02 +00:00
rcourtman
fa8a8f3af3 Reduce release assets by removing duplicates
Removed:
- Individual .sha256 files (checksums.txt already contains all checksums)
- Standalone binaries without version numbers (users should download versioned tarballs/zips)

Standalone binaries are only needed in Docker images for the /download/ endpoint.
GitHub releases should only contain versioned archives for user downloads.

This reduces release assets from ~54 files to ~19 files per release.
2025-11-11 17:26:00 +00:00
rcourtman
13a469362a Exclude development/infrastructure changes from release notes
Users don't care about CI/CD improvements, release workflows, build
processes, or testing infrastructure. Only include user-visible changes.

Related to #671
2025-11-11 17:18:50 +00:00
rcourtman
93d0eb6b8a Remove commit hashes from LLM-generated release notes
Commit hashes clutter the release notes and aren't useful for end users.
Only include issue references when explicitly mentioned in commits.

Related to #671
2025-11-11 17:11:02 +00:00
rcourtman
6e669b46dc Fix commit hash linking in release notes
Remove # symbol from commit hash references so GitHub auto-links them.
Format: (abc123) instead of (#abc123)
Issue references still use #: (#123)

Related to #671
2025-11-11 17:03:39 +00:00
rcourtman
c27f21f33b Update release notes template to match established format
- Use exact template format from v4.28.0 and prior releases
- Include all standard sections: New Features, Bug Fixes, Improvements, Breaking Changes
- Add complete installation instructions (systemd, Docker, Manual Binary, Helm)
- Include Downloads section with all artifact types
- Add Notes section for important highlights and upgrade considerations
- Ensure LLM outputs format exactly matching previous releases

Related to #671 (automated release workflow)
2025-11-11 14:05:15 +00:00
rcourtman
a7828e2d1e Add LLM-powered release notes generation
- Create scripts/generate-release-notes.sh to auto-generate release notes from git commits
- Supports both Anthropic Claude and OpenAI APIs
- Uses Claude Haiku 4.5 (claude-haiku-4-5-20251001) for cost efficiency ($1/$5 per million tokens)
- Falls back to OpenAI gpt-4o-mini if Anthropic key not available
- Integrates into release workflow between validation and release creation
- Compares current version with previous git tag to generate changelog
- Outputs categorized, user-friendly release notes with installation instructions

Workflow now automatically:
1. Finds previous release tag
2. Analyzes all commits since last release
3. Generates structured release notes via LLM
4. Uses generated notes for draft release body

Requires ANTHROPIC_API_KEY or OPENAI_API_KEY in GitHub secrets.

Related to #671 (automated release workflow)
2025-11-11 14:01:34 +00:00
rcourtman
b604a63322 Fix critical release workflow issues identified in review
Addresses 3 critical issues from 4-dev team review:

1. CRITICAL: Fix non-deterministic checksum generation (Dev 2 & 3)
   - Add explicit sorting to checksums.txt generation
   - Prevents #671 checksum mismatches between builds
   - Location: scripts/build-release.sh:348

2. CRITICAL: Fix upload/validation race condition (Dev 1)
   - Change validation trigger from 'release: created' to 'workflow_run'
   - Prevents validation from running while assets still uploading
   - Prevents valid releases from being incorrectly deleted
   - Location: .github/workflows/validate-release-assets.yml:4-8

3. CRITICAL: Fix GitHub token exposure in logs (Dev 1)
   - Replace curl commands with gh CLI
   - Prevents token leakage in workflow logs
   - Location: .github/workflows/validate-release-assets.yml:44, 63

All three issues were blocking issues that could cause release failures.
Remaining high/medium priority issues to be addressed in follow-up PRs.
2025-11-11 11:32:44 +00:00
rcourtman
c73185fcdd Fix validate-release.sh path issues after pushd
The script does pushd into RELEASE_DIR, so tarball paths should not include
the RELEASE_DIR prefix. Also fixed checksum validation glob patterns to
exclude .sha256 files from matching.
2025-11-11 10:54:00 +00:00
rcourtman
e2cd7cf9d9 Fix validation script to check for ./ prefix in tarballs
Tarballs are created with ./bin/pulse paths (relative from inside staging dir)
but validation was looking for bin/pulse paths. Updated all tar -tzf checks
to use correct ./ prefix.
2025-11-11 10:43:26 +00:00
rcourtman
73e95f2262 Fix validate-release.sh to use RELEASE_DIR path prefix
The validation script was looking for tarballs in the current directory
instead of the release/ directory, causing all validations to fail.
Now properly prepends $RELEASE_DIR to all file paths.
2025-11-11 10:32:36 +00:00
Claude
e12980e351
Add automated release workflow with validation
This commit introduces a comprehensive GitHub Actions workflow for
creating releases, ensuring all artifacts are validated before upload.

Changes:
- Add .github/workflows/release.yml: Manual workflow_dispatch trigger
  that builds, validates, and creates draft releases
- Update scripts/validate-release.sh: Add --skip-docker flag to allow
  validation without Docker image checks

Key features:
- Validation runs BEFORE any assets are uploaded
- If validation fails, no release is created
- checksums.txt and artifacts come from the same build
- No manual steps between validation and upload
- Checksums uploaded first, then all other assets
- Creates draft release for manual review before publishing

The workflow ensures that checksums.txt cannot drift from binaries
by running the entire build-validate-upload pipeline atomically.
2025-11-11 09:22:03 +00:00
rcourtman
5bac91a664 Fix pulse-sensor-proxy configuration not applied in LXC containers (related to #600)
This fixes two bugs that prevented temperature monitoring from working
after running install-sensor-proxy.sh on LXC deployments:

1. CRITICAL: Pulse service not restarted after systemd override
   - The installer wrote PULSE_SENSOR_PROXY_SOCKET env var to systemd
     drop-in and ran daemon-reload, but never restarted Pulse service
   - Running Pulse instances continued using old environment variables
   - Temperatures wouldn't work until manual Pulse restart
   - Now: Automatically restart Pulse if running after writing override

2. Added guard to check if Pulse service exists before configuring
   - Installer would write systemd override even if Pulse not installed
   - Left orphaned drop-in files that confused users
   - Now: Check if pulse.service exists, warn and skip if not found

3. MINOR: Fix inconsistent Docker mount instructions
   - docker-compose.yml showed :ro (read-only) mount
   - Installer output showed :rw (read-write) mount
   - Changed installer to match compose file (:ro is correct and secure)

Impact: Users in #600 reported "socketFound=false" even after running
installer successfully. This was because Pulse never picked up the new
socket path without a restart.
2025-11-09 16:44:08 +00:00
rcourtman
23ce2c6d11 Add support for Windows 32-bit (windows-386) architecture (related to #674)
Adds build support for 32-bit Windows (windows-386) for pulse-host-agent.

Changes:
- Add windows-386 build to Dockerfile host-agent build section
- Add windows-386 binary copy and symlink to Dockerfile
- Add windows-386 build to build-release.sh
- Add windows-386 zip package to release artifacts
- Include windows-386 binary in standalone binary copies

This enables pulse-host-agent to run on 32-bit Windows systems, which are still relevant in legacy/industrial monitoring environments through late 2025.
2025-11-09 08:57:30 +00:00
rcourtman
4834dea05b Add support for linux-386 and linux-armv6 architectures (related to #674)
Adds build support for 32-bit x86 (i386/i686) and ARMv6 (older Raspberry Pi models) architectures across all agents and install scripts.

Changes:
- Add linux-386 and linux-armv6 to build-release.sh builds array
- Update Dockerfile to build docker-agent, host-agent, and sensor-proxy for new architectures
- Update all install scripts to detect and handle i386/i686 and armv6l architectures
- Add architecture normalization in router download endpoints
- Update update manager architecture mapping
- Update validate-release.sh to expect 24 binaries (was 18)

This enables Pulse agents to run on older/legacy hardware including 32-bit x86 systems and Raspberry Pi Zero/Zero W devices.
2025-11-09 08:35:24 +00:00
rcourtman
334b8c727f Fix SMART temperature collection on smartctl 7.4+ (related to #672)
Fixes two critical bugs in refresh_smart_cache() that prevented SMART
temperature collection from working:

1. Invalid smartctl parameter: Changed -n standby,after to -n standby
   The 'after' parameter is not valid in smartctl 7.4 and causes:
   "INVALID ARGUMENT TO -n: standby,after"
   Valid syntax is standby[,STATUS[,STATUS2]] where STATUS must be numeric.

2. Broken process detection: Replaced exec -a with lock file approach
   The original exec -a pulse-sensor-wrapper-refresh bash line replaced
   the subshell with a new bash process that had no script to run, causing
   the function to exit immediately without collecting any SMART data.

   New approach uses a lock file ($CACHE_DIR/smart-refresh.lock) with
   trap-based cleanup to prevent concurrent refresh operations.

Credits to @ZaDarkSide for identifying these issues in PR #672.
2025-11-08 23:40:43 +00:00
rcourtman
16c29463f9 Fix Windows host agent installer reliability (related to #654)
The download endpoint had a dangerous fallback that silently served the
wrong binary when the requested platform/arch combination was missing.
If a Docker image shipped without Windows binaries, the installer would
receive a Linux ELF instead of a Windows PE, causing ERROR_BAD_EXE_FORMAT.

Changes:
- Download handler now operates in strict mode when platform+arch are
  specified, returning 404 instead of serving mismatched binaries
- PowerShell installer validates PE header (MZ signature)
- PowerShell installer verifies PE machine type matches requested arch
- PowerShell installer fetches and verifies SHA256 checksums
- PowerShell installer shows diagnostic info: OS arch, download URL,
  file size for better troubleshooting

This prevents silent failures and provides clear error messages when
binaries are missing or corrupted.
2025-11-07 22:55:03 +00:00
rcourtman
679225510e Silence broken pipe error in sensor proxy self-heal script (related to #628)
The self-heal timer runs 'systemctl list-unit-files | grep -q' every hour.
When grep matches and exits early, systemctl logs "Failed to print table:
Broken pipe" to syslog. This is cosmetic but floods Proxmox logs and
confuses operators.

Changes:
- Redirect stderr from systemctl to /dev/null
- Prevents the broken pipe message from reaching syslog
- Self-heal functionality unchanged

This addresses the concern raised in discussion #628.
2025-11-07 17:46:23 +00:00
rcourtman
97f9de6c95 feat(security): Enhance systemd hardening directives
Adds additional systemd security directives for defense in depth:
- MemoryDenyWriteExecute=true (prevents RWX memory)
- RestrictRealtime=true (denies realtime scheduling)
- ProtectHostname=true (hostname protection)
- ProtectKernelLogs=true (kernel log protection)
- SystemCallArchitectures=native (native syscalls only)

These directives provide additional layers to slow/prevent
post-compromise exploitation of the proxy process.

Related to security audit 2025-11-07.
2025-11-07 17:09:47 +00:00
rcourtman
32e0d453c4 Add Windows ARM64 support for host agent (related to #654)
Windows 11 25H2 ships exclusively on ARM64 hardware. When users on ARM64
attempt to install the host agent, the Service Control Manager fails to
load the amd64 binary with ERROR_BAD_EXE_FORMAT, surfaced as "The Pulse
Host Agent is not compatible with this Windows version".

Changes:
- Dockerfile: Build pulse-host-agent-windows-arm64.exe alongside amd64
- Dockerfile: Copy windows-arm64 binary and create symlink for download endpoint
- install-host-agent.ps1: Use RuntimeInformation.OSArchitecture to detect ARM64
- build-release.sh: Build darwin-amd64, darwin-arm64, windows-amd64, windows-arm64
- build-release.sh: Package Windows binaries as .zip archives
- validate-release.sh: Check for windows-arm64 binary and symlink
- validate-release.sh: Add architecture validation for all darwin/windows variants

The installer now correctly detects ARM64 and downloads the appropriate binary.
2025-11-07 12:18:57 +00:00
rcourtman
2a79d57f73 Add SMART temperature collection for physical disks (related to #652)
Extends temperature monitoring to collect SMART temps for SATA/SAS disks,
addressing issue #652 where physical disk temperatures showed as empty.

Architecture:
- Deploys pulse-sensor-wrapper.sh as SSH forced command on Proxmox nodes
- Wrapper collects both CPU/GPU temps (sensors -j) and disk temps (smartctl)
- Implements 30-min cache with background refresh to avoid performance impact
- Uses smartctl -n standby,after to skip sleeping drives without waking them
- Returns unified JSON: {sensors: {...}, smart: [...]}

Backend changes:
- Add DiskTemp model with device, serial, WWN, temperature, lastUpdated
- Extend Temperature model with SMART []DiskTemp field and HasSMART flag
- Add WWN field to PhysicalDisk for reliable disk matching
- Update parseSensorsJSON to handle both legacy and new wrapper formats
- Rewrite mergeNVMeTempsIntoDisks to match SMART temps by WWN → serial → devpath
- Preserve legacy NVMe temperature support for backward compatibility

Performance considerations:
- SMART data cached for 30 minutes per node to avoid excessive smartctl calls
- Background refresh prevents blocking temperature requests
- Respects drive standby state to avoid spinning up idle arrays
- Staggered disk scanning with 0.1s delay to avoid saturating SATA controllers

Install script:
- Deploys wrapper to /usr/local/bin/pulse-sensor-wrapper.sh
- Updates SSH forced command from "sensors -j" to wrapper script
- Backward compatible - falls back to direct sensors output if wrapper missing

Testing note:
- Requires real hardware with smartmontools installed for full functionality
- Empty smart array returned gracefully when smartctl unavailable
- Legacy sensor-only nodes continue working without changes
2025-11-07 11:46:57 +00:00
rcourtman
fa7ca00250 Fix duplicate checksum in build-release.sh
The checksum generation was including pulse-host-agent-v*-darwin-arm64.tar.gz
twice: once from the *.tar.gz pattern and once from the pulse-host-agent-*
pattern. Fixed by using extglob to exclude .tar.gz and .sha256 files from
the agent binary patterns since tarballs are already matched separately.
2025-11-06 22:19:16 +00:00
rcourtman
20099549c6 Add comprehensive release validation to prevent missing artifacts
Adds automated validation script to prevent the pattern of patch
releases caused by missing files/artifacts.

scripts/validate-release.sh validates all 40+ artifacts including:
- Docker image scripts (8 install/uninstall scripts)
- Docker image binaries (17 across all platforms)
- Release tarballs (5 including universal and macOS)
- Standalone binaries (12+)
- Checksums for all distributable assets
- Version embedding in every binary type
- Tarball contents (binaries + scripts + VERSION)
- Binary architectures and file types

The script catches 100% of issues from the last 3 patch releases
(missing scripts, missing install.sh, missing binaries, broken
version embedding).

Updated RELEASE_CHECKLIST.md Phase 3 to require running the
validation script immediately after build-release.sh and before
proceeding to Docker build/publish phases.

Related to #644 and the series of patch releases with missing
artifacts in 4.26.x.
2025-11-06 16:33:49 +00:00
rcourtman
035d872269 Add missing install/uninstall scripts to Docker image and release builds (related to #644)
The Dockerfile and build-release.sh were missing several installer and uninstaller
scripts that the router expects to serve via HTTP endpoints:
- install-container-agent.sh
- install-host-agent.ps1
- uninstall-host-agent.sh
- uninstall-host-agent.ps1

This caused 404 errors when users attempted to add Docker/Podman hosts or use the
PowerShell installer, as reported in #644.

Changes:
- Dockerfile: Added missing scripts to /opt/pulse/scripts/ with proper permissions
- build-release.sh: Added missing scripts to both per-platform and universal tarballs
  to ensure bare-metal deployments serve the same endpoints as Docker deployments
2025-11-06 16:01:40 +00:00
rcourtman
615cb129df Fix checksum verification failure in install.sh (related to #642)
The .sha256 files generated during release builds contained only the hash,
but sha256sum -c expects the format "hash  filename". This caused all
install.sh updates to fail with "Checksum verification failed" even when
the checksum was correct.

Root cause: build-release.sh line 289 was using awk to extract only field 1
(the hash), discarding the filename that sha256sum -c needs.

Fix: Remove the awk filter to preserve the full sha256sum output format.

This affected the demo server update workflow and user installations.
2025-11-06 15:28:05 +00:00
rcourtman
fd3a72606f Add standalone host-agent binaries to releases
Issue: HOST_AGENT.md documented downloading pulse-host-agent binaries
from GitHub releases, but those assets didn't exist. Only tarballs were
available, making manual installation unnecessarily complex.

Changes:
- Copy standalone host-agent binaries (all architectures) to release/
  directory alongside sensor-proxy binaries
- Include host-agent binaries in checksum generation
- Update HOST_AGENT.md to clarify available architectures
- Retroactively uploaded missing binaries to v4.26.1

This enables air-gapped and manual installations without requiring an
already-running Pulse server to download from.
2025-11-06 14:20:59 +00:00
rcourtman
e4378602c1 Fix install.sh missing from GitHub releases (addresses #642)
Root cause: install.sh was not being copied to the release directory
during build-release.sh execution, so it was never uploaded as a
release asset. This caused the download URL to return "Not Found",
which bash attempted to execute as a command.

Changes:
- Copy install.sh to release/ directory in build-release.sh
- Include install.sh in checksums generation

Note: RELEASE_CHECKLIST.md also updated locally to verify install.sh
presence in Phase 3 and Phase 5, but that file is gitignored.
2025-11-06 14:10:46 +00:00
rcourtman
fdcec85931 Fix critical version embedding issues for 4.26 release
Addresses the root cause of issue #631 (infinite Docker agent restart loop)
and prevents similar issues with host-agent and sensor-proxy.

Changes:
- Set dockeragent.Version default to "dev" instead of hardcoded version
- Add version embedding to server build in Dockerfile
- Add version embedding to host-agent builds (all platforms)
- Add version embedding to sensor-proxy builds (all platforms)

This ensures:
1. Server's /api/agent/version endpoint returns correct v4.26.0
2. Downloaded agent binaries have matching embedded versions
3. Dev builds skip auto-update (Version="dev")
4. No version mismatch triggers infinite restart loops

Related to #631
2025-11-06 11:42:52 +00:00
rcourtman
c638a8c28c Fix checksum verification failure during installation
Related to #639

Users reported "Failed to download checksum for Pulse release" errors
during installation. The root cause was a mismatch between what the
build system generates and what the installer expects:

- install.sh downloads individual .sha256 files (e.g., pulse-v4.25.0-linux-amd64.tar.gz.sha256)
- build-release.sh only created a single checksums.txt file

This commit updates build-release.sh to generate both:
1. Individual .sha256 files for each asset (required by install.sh)
2. Combined checksums.txt for manual verification and signing

This maintains backwards compatibility with the installer while keeping
the aggregated checksums.txt for power users and GPG signing.
2025-11-06 11:21:49 +00:00
rcourtman
8a052baa2a Fix temperature monitoring for standalone Proxmox nodes and add multi-arch sensor proxy builds
Related to #571

This addresses multiple temperature monitoring issues:

1. Fix single-node Proxmox installation failure: Add '|| true' to pvecm status
   calls to prevent script exit on standalone (non-clustered) nodes with
   'set -euo pipefail'. The script now properly falls through to standalone
   node configuration when cluster detection fails.

2. Build pulse-sensor-proxy for all Linux architectures (amd64, arm64, armv7)
   in Dockerfile to ensure binaries are available for download on all supported
   platforms. This resolves the missing binary issue from v4.23.0.

Note: AMD Tctl sensor support was already implemented in a previous commit.
2025-11-05 19:41:09 +00:00
rcourtman
3194b10398 Improve Alpine Linux support and agent startup validation
Related to #612

This commit addresses the Alpine Linux installation issues reported where:
1. The OpenRC init system was not properly detected
2. Manual startup instructions were unclear and used placeholder values
3. The agent didn't validate configuration properly at startup

Changes:

Install Script (install-docker-agent.sh):
- Improved OpenRC detection to check for rc-service and rc-update commands
  instead of looking for openrc-run binary in specific paths
- Added specific Alpine Linux detection via /etc/alpine-release and /etc/os-release
- Enhanced manual startup instructions to show actual values instead of placeholders
- Added clearer warnings and guidance when no init system is detected
- Included comprehensive startup command with all required parameters

Agent Startup Validation (pulse-docker-agent):
- Added validation to detect unexpected command-line arguments
- Added helpful note about double-dash flag requirements (--token vs -token)
- Improved error messages to include example usage patterns
- Added warning when defaulting to localhost without explicit URL configuration
- Provide both command-line and environment variable examples in error messages

These improvements ensure that:
- Alpine Linux installations will properly detect and configure OpenRC services
- Users who must start the agent manually get clear, copy-pasteable commands
- Configuration errors are caught early with actionable error messages
- Common mistakes (like missing --url) are clearly explained
2025-11-05 19:01:09 +00:00
rcourtman
26144ae558 Improve temperature proxy setup guidance for Docker deployments
This addresses GitHub Discussion #605 where users were unclear about
configuring the temperature proxy when running Pulse in Docker.

Changes:

**install-sensor-proxy.sh:**
- Add Docker-specific post-install instructions when --standalone flag is used
- Show required docker-compose.yml bind mount configuration
- Provide verification commands for Docker deployments
- Link to full documentation for troubleshooting

**TEMPERATURE_MONITORING.md:**
- Add prominent "Quick Start for Docker Deployments" section at the top
- Move Docker instructions earlier in the document for better visibility
- Provide complete 4-step setup process with verification commands

These changes ensure Docker users immediately see:
1. How to install the proxy on the Proxmox host
2. What bind mount to add to docker-compose.yml
3. How to restart and verify the setup
4. Where to find detailed troubleshooting

The installer now provides actionable next steps instead of just
confirming installation, reducing confusion for containerized deployments.
2025-11-05 18:18:04 +00:00
rcourtman
adda6eea38 Update docker CPU metrics and add OpenRC installer support (Refs #255) 2025-11-04 22:16:50 +00:00
rcourtman
6eb1a10d9b Refactor: Code cleanup and localStorage consolidation
This commit includes comprehensive codebase cleanup and refactoring:

## Code Cleanup
- Remove dead TypeScript code (types/monitoring.ts - 194 lines duplicate)
- Remove unused Go functions (GetClusterNodes, MigratePassword, GetClusterHealthInfo)
- Clean up commented-out code blocks across multiple files
- Remove unused TypeScript exports (helpTextClass, private tag color helpers)
- Delete obsolete test files and components

## localStorage Consolidation
- Centralize all storage keys into STORAGE_KEYS constant
- Update 5 files to use centralized keys:
  * utils/apiClient.ts (AUTH, LEGACY_TOKEN)
  * components/Dashboard/Dashboard.tsx (GUEST_METADATA)
  * components/Docker/DockerHosts.tsx (DOCKER_METADATA)
  * App.tsx (PLATFORMS_SEEN)
  * stores/updates.ts (UPDATES)
- Benefits: Single source of truth, prevents typos, better maintainability

## Previous Work Committed
- Docker monitoring improvements and disk metrics
- Security enhancements and setup fixes
- API refactoring and cleanup
- Documentation updates
- Build system improvements

## Testing
- All frontend tests pass (29 tests)
- All Go tests pass (15 packages)
- Production build successful
- Zero breaking changes

Total: 186 files changed, 5825 insertions(+), 11602 deletions(-)
2025-11-04 21:50:46 +00:00
rcourtman
5c4be1921c chore: snapshot current changes 2025-11-02 22:47:55 +00:00
rcourtman
f913eb9783 Fix hot-dev reload to detect new .go file creation
The file watcher was only triggering on .go file modifications but missing new file creation. This happened because inotifywait sometimes reports the directory path first when a file is created.

Changes:
- Include event type in inotifywait output format
- Trigger rebuild on CREATE/DELETE/MOVED events in addition to .go modifications
- Add exclusions for temp files (.swp, .tmp, ~)

Now creating new .go files will trigger an auto-rebuild.
2025-10-28 23:07:27 +00:00
rcourtman
f2acdd59af Normalize docker agent version handling 2025-10-28 08:42:58 +00:00
rcourtman
e07336dd9f refactor: remove legacy DISABLE_AUTH flag and enhance authentication UX
Major authentication system improvements:

- Remove deprecated DISABLE_AUTH environment variable support
- Update all documentation to remove DISABLE_AUTH references
- Add auth recovery instructions to docs (create .auth_recovery file)
- Improve first-run setup and Quick Security wizard flows
- Enhance login page with better error messaging and validation
- Refactor Docker hosts view with new unified table and tree components
- Add useDebouncedValue hook for better search performance
- Improve Settings page with better security configuration UX
- Update mock mode and development scripts for consistency
- Add ScrollableTable persistence and improved responsive design

Backend changes:
- Remove DISABLE_AUTH flag detection and handling
- Improve auth configuration validation and error messages
- Enhance security status endpoint responses
- Update router integration tests

Frontend changes:
- New Docker components: DockerUnifiedTable, DockerTree, DockerSummaryStats
- Better connection status indicator positioning
- Improved authentication state management
- Enhanced CSRF and session handling
- Better loading states and error recovery

This completes the migration away from the insecure DISABLE_AUTH pattern
toward proper authentication with recovery mechanisms.
2025-10-27 19:46:51 +00:00
rcourtman
68ce8e7520 feat: finalize swarm service monitoring (#598) 2025-10-26 09:35:49 +00:00
rcourtman
138d8facd2 Improve host agent onboarding flow 2025-10-25 09:37:29 +00:00
rcourtman
a6bf2c852b feat: add token revocation tracking and install script improvements
This commit adds comprehensive token revocation tracking across the UI and enhances the agent installation script for better platform support.

Key changes:
- Added token revocation warnings in Docker hosts and host agents UI with amber-colored indicators
- Implemented automatic token revocation detection when tokens are deleted
- Enhanced install scripts with Unraid detection and manual start instructions for non-systemd platforms
- Improved service management with restart instead of start for systemd
- Added visual indicators for revoked tokens with contextual warnings
- Updated table column widths in hosts overview for better layout
2025-10-24 22:30:10 +00:00
rcourtman
655fec2225 refactor: streamline host and Docker agent setup UI
Simplifies the onboarding flow by removing verbose instructions and toggles, consolidating navigation elements, and cleaning up the settings interface. Improves the macOS host agent installer with better Keychain access control and launchd service management.
2025-10-24 14:59:50 +00:00
rcourtman
8fb9ef2e8f fix: correct checksum URL construction in install script
The checksum URL was incorrectly constructed by appending .sha256
to the entire download URL including query parameters, resulting in:
  /download/pulse-host-agent?platform=linux&arch=amd64.sha256

This caused .sha256 to be part of the arch parameter, which prevented
the checksum endpoint from being reached correctly.

Fixed to construct checksum URL with .sha256 as part of the path:
  /download/pulse-host-agent.sha256?platform=linux&arch=amd64

Tested on Proxmox VE host (delly):
- Installation: ✓ Binary downloaded and installed successfully
- Service: ✓ systemd service created, enabled, and started
- Validation: ✓ Service running and attempting to report
- Logs: ✓ JSON logs writing to /var/log/pulse/host-agent.log
- Uninstallation: ✓ Complete cleanup (binary, service, logs)
- Colors: ✓ ANSI colored output working properly

Note: Checksum validation gracefully handled when endpoint
unavailable (server doesn't provide checksums yet)
2025-10-23 22:34:07 +00:00
rcourtman
b4247fc095 feat: add server-side support for agent installation improvements
API Enhancements:
- Add SHA256 checksum endpoint for binary downloads
  - Computes checksum on-the-fly when .sha256 suffix is requested
  - Example: /download/pulse-host-agent?platform=linux&arch=amd64.sha256
  - Enables installer scripts to verify binary integrity
- Add /uninstall-host-agent.sh endpoint for Linux/macOS uninstall script
- Add endpoint to public paths (no auth required)

Checksum Implementation:
- New serveChecksum() function computes SHA256 hash using crypto/sha256
- Returns plain text checksum in hex format
- Supports all binary download endpoints
- Zero performance impact (only computed when requested)

Install Script Updates:
- Add --force/-f flag to skip all interactive prompts
  - URL/token prompts skipped with --force
  - Reinstall confirmation skipped with --force
  - Checksum mismatch still aborts (security first)
- Force mode auto-accepts updates and reinstalls
- Usage: ./install-host-agent.sh --url $URL --token $TOKEN --force

Security Notes:
- Checksum verification protects against:
  - Corrupted downloads due to network issues
  - Man-in-the-middle binary tampering
  - Storage corruption on server
- Force mode maintains security by aborting on checksum mismatch
- No bypass for security-critical validations

These improvements enable:
- Automated deployments (--force flag)
- Binary integrity verification (checksums)
- Better security posture (tamper detection)
- Standardized uninstall process (endpoint)

The /api/version endpoint already exists and returns version info
for update checks (no changes needed).
2025-10-23 22:27:02 +00:00
rcourtman
df8e12df33 feat: enhance macOS/Linux agent installation to match Windows quality
Major improvements to the host agent installation experience:

Installation Enhancements:
- Add interactive mode with prompts for URL/token if not provided
- Add colored output with ANSI codes (✓ ℹ ⚠ ✗) matching Windows style
- Add professional header/footer with bordered sections
- Display masked configuration before installation
- Add version detection and update notifications
- Add reinstall confirmation for existing installations

Security Improvements:
- Store tokens in macOS Keychain instead of plaintext plist
- Create wrapper script to read token from Keychain at runtime
- Add SHA256 checksum verification for downloaded binaries
- Set restrictive permissions (chmod 600) on service configs
- Graceful fallback if Keychain access denied

Validation & Verification:
- Wait 10 seconds and verify service is running
- Query /api/hosts endpoint to confirm agent registered
- Verify hostname appears in Pulse server
- Provide detailed troubleshooting if validation fails

Error Handling:
- Comprehensive error messages with actionable guidance
- Platform-specific install instructions for missing dependencies
- 4-step troubleshooting guide for download failures
- Build-from-source fallback instructions
- Service failure diagnostics with log viewing commands

Logging Improvements:
- Use persistent log directories (no more /tmp)
- macOS: ~/Library/Logs/Pulse/host-agent.log
- Linux: /var/log/pulse/host-agent.log
- Automatically create log directories

Post-Install:
- Display service management commands (start/stop/restart/logs)
- Show installed file locations
- Provide uninstall instructions
- Link to Pulse dashboard

Uninstall Script:
- New dedicated uninstall script with colored output
- Comprehensive cleanup (service, binary, logs, Keychain)
- Platform detection and appropriate cleanup steps
- Remove macOS Keychain entries and wrapper scripts
- Remove temporary logs from old /tmp location
- Retry logic for file locking issues

Progress Indicators:
- Better download progress with curl --progress-bar
- Clear status messages for each installation step
- Visual feedback throughout installation

These changes bring the macOS/Linux installation experience to parity
with Windows and add several features that exceed Windows quality:
- API endpoint verification (Windows doesn't have)
- Checksum verification (Windows doesn't have)
- macOS Keychain integration (more secure than Windows config)
- Update detection (Windows doesn't have)
2025-10-23 22:23:23 +00:00
rcourtman
6333a445e9 feat: add native Windows service support and expandable host details
Windows Host Agent Enhancements:
- Implement native Windows service support using golang.org/x/sys/windows/svc
- Add Windows Event Log integration for troubleshooting
- Create professional PowerShell installation/uninstallation scripts
- Add process termination and retry logic to handle Windows file locking
- Register uninstall endpoint at /uninstall-host-agent.ps1

Host Agent UI Improvements:
- Add expandable drawer to Hosts page (click row to view details)
- Display system info, network interfaces, disks, and temperatures in cards
- Replace status badges with subtle colored indicators
- Remove redundant master-detail sidebar layout
- Add search filtering for hosts

Technical Details:
- service_windows.go: Windows service lifecycle management with graceful shutdown
- service_stub.go: Cross-platform compatibility for non-Windows builds
- install-host-agent.ps1: Full Windows installation with validation
- uninstall-host-agent.ps1: Clean removal with process termination and retries
- HostsOverview.tsx: Expandable row pattern matching Docker/Proxmox pages

Files Added:
- cmd/pulse-host-agent/service_windows.go
- cmd/pulse-host-agent/service_stub.go
- scripts/install-host-agent.ps1
- scripts/uninstall-host-agent.ps1
- frontend-modern/src/components/Hosts/HostsOverview.tsx
- frontend-modern/src/components/Hosts/HostsFilter.tsx

The Windows service now starts reliably with automatic restart on failure,
and the uninstall script handles file locking gracefully without requiring reboots.
2025-10-23 22:11:56 +00:00
rcourtman
a57fb10532 Remove unused toggle-mock-pure.sh script
The toggle-mock-pure.sh script was unused and redundant:
- Not referenced in any documentation or code
- Its PULSE_DISABLE_REAL_NODES variable is not used anywhere
- toggle-mock.sh already provides all necessary mock mode functionality

Consolidating to a single mock toggle script reduces confusion.
2025-10-23 12:09:21 +00:00
rcourtman
5c54685f04 Add API token scopes and standalone host agent
Introduces granular permission scopes for API tokens (docker:report, docker:manage, host-agent:report, monitoring:read/write, settings:read/write) allowing tokens to be restricted to minimum required access. Legacy tokens default to full access until scopes are explicitly configured.

Adds standalone host agent for monitoring Linux, macOS, and Windows servers outside Proxmox/Docker estates. New Servers workspace in UI displays uptime, OS metadata, and capacity metrics from enrolled agents.

Includes comprehensive token management UI overhaul with scope presets, inline editing, and visual scope indicators.
2025-10-23 11:40:31 +00:00
rcourtman
e1fe8354e9 Ensure Docker agent builds stay static (#597) 2025-10-22 21:48:57 +00:00
rcourtman
bc479643e4 release: prepare v4.25.0 2025-10-22 10:46:18 +00:00
rcourtman
ff4dc49ae4 Update Pulse install flow and related components 2025-10-21 19:58:53 +00:00
rcourtman
f9cb96ceb8 feat: add --uninstall support to Docker agent and sensor proxy scripts
Users can now cleanly uninstall components with optional data removal.

Docker Agent (install-docker-agent.sh):
- --uninstall: Remove service, binary, systemd unit, Unraid startup hook
- --purge: Also remove log files (optional, must be used with --uninstall)
- Stops/disables service even if unit file is missing (resilient cleanup)
- Validates --purge requires --uninstall

Sensor Proxy (install-sensor-proxy.sh):
- --uninstall: Remove service, binary, cleanup scripts, socket directory
- Calls existing cleanup helper to remove SSH keys from cluster nodes
- Manual fallback if cleanup helper is missing
- --purge: Also remove state/logs and service account
- Validates --purge requires --uninstall

Usage:
  # Uninstall Docker agent (keep logs)
  curl ... | bash -s -- --uninstall

  # Uninstall Docker agent (remove everything)
  curl ... | bash -s -- --uninstall --purge

  # Uninstall sensor proxy (keep state/logs)
  curl ... | bash -s -- --uninstall

  # Uninstall sensor proxy (remove everything)
  curl ... | bash -s -- --uninstall --purge

Changes:
- scripts/install-docker-agent.sh: Add --purge flag, improve uninstall flow
- scripts/install-sensor-proxy.sh: Add perform_uninstall() function
- Both: Non-interactive, idempotent, resilient cleanup

Next: Update UI to show uninstall commands when removing hosts/nodes

Co-authored-by: Codex AI
2025-10-21 10:21:48 +00:00
rcourtman
7e871780f6 feat: improve LXC installer robustness and temperature monitoring UX
Major improvements to the install script based on comprehensive review:

## 1. Temperature Monitoring - No Restart Required 
- Ask about temperature monitoring BEFORE container creation (not after)
- Add bind mount during `pct create` instead of requiring restart later
- Quick mode defaults to "yes", Advanced mode asks user
- Host path: /run/pulse-sensor-proxy → /mnt/pulse-proxy in container
- Support --skip-restart flag in install-sensor-proxy.sh
- Eliminates disruptive container restart on fresh installs

## 2. Shell Injection Prevention 🔒
- Replace `eval pct create` with array-based command building
- Prevents quoting bugs with special characters in hostnames/nameservers
- Safer handling of user input in container creation

## 3. Non-Interactive Install Support 🤖
- Replace bare `read` with `safe_read_with_default` in prompts
- Prevents hangs when running `curl | bash` non-interactively
- Proper fallback to sensible defaults

## 4. Cleanup on Interrupt 🧹
- Track container ID globally during creation
- Properly cleanup orphaned containers on Ctrl+C/SIGTERM
- New handle_install_interrupt() function
- Prevents leftover containers after cancelled installs

## 5. Air-Gapped Network Support 🌐
- Replace 8.8.8.8 ping check with `hostname -I` IP detection
- Supports restricted/firewalled networks where external ping fails
- More reliable for DHCP-only environments

Changes:
- install.sh: Refactor temperature prompt timing and mount setup
- install.sh: Convert pct create to array-based args (lines 1018-1055)
- install.sh: Add handle_install_interrupt trap (lines 38-48)
- install.sh: Replace ping check with IP detection (line 1082)
- scripts/install-sensor-proxy.sh: Add --skip-restart flag support
- scripts/install-sensor-proxy.sh: Improve mount detection and updates

Impact:
- Fresh installs now complete without any container restarts
- Temperature monitoring works immediately after first boot
- Safer and more robust for automation/CI scenarios
- Better experience on restricted networks

Co-authored-by: Codex AI
2025-10-21 09:22:43 +00:00
rcourtman
b929fdcc6e feat: improve source build installation experience
- Remove confusing --main flag, use --source for clarity
- Fix timeout issues when building from source in LXC containers
  - Increase timeout from 5min to 20min for source builds
  - Add PULSE_CONTAINER_TIMEOUT env var for custom timeouts
  - Support PULSE_CONTAINER_TIMEOUT=0 to disable timeout
- Fix misleading "Latest version: vX.X.X" message during source builds
- Update documentation to use --source instead of --main
- Simplify auto-update script logic for source builds

Changes:
- install.sh: Check BUILD_FROM_SOURCE early to skip version detection
- install.sh: Adaptive timeout (300s binary, 1200s source builds)
- install.sh: Better timeout error messages with recovery instructions
- README.md: Replace --main with --source in examples
- docs/INSTALL.md: Replace --main with --source in examples
- scripts/pulse-auto-update.sh: Remove --main special case
2025-10-21 08:57:29 +00:00
rcourtman
95c85f6e01 fix: use correct service name (pulse.service) for proxy environment override
The installer was configuring pulse-backend.service.d but the actual
service is pulse.service, so the PULSE_SENSOR_PROXY_SOCKET environment
variable wasn't being set.

Changed: pulse-backend.service → pulse.service

This ensures Pulse actually uses the proxy socket for temperature
monitoring instead of attempting SSH connections.
2025-10-20 22:28:33 +00:00
rcourtman
8faa9040fb fix: show curl errors in installer download failures
Changed curl flags from -fsSL to -fSL to enable error output.
The -s flag was silencing all curl errors including SSL/TLS issues,
making it impossible to diagnose download failures.

With -S (show errors), stderr now captures meaningful error messages
like certificate problems, connection failures, etc.
2025-10-20 21:31:54 +00:00
rcourtman
90d51a2b1b feat: add rollback mechanism for container config changes
- Back up container config before making mount modifications
- Restore original config if socket verification fails
- Clean up backup file on success or when verification is skipped
- Leave host-level resources (user, binary, service) in place for idempotency

This ensures failed installations don't leave containers in an
inconsistent state while keeping successfully installed host services
for faster re-runs.
2025-10-20 21:16:06 +00:00
rcourtman
d421f101ba feat: harden temperature proxy installation with better validation and error handling
Setup script improvements (config_handlers.go):
- Remove redundant mount configuration and container restart logic
- Let installer handle all mount/restart operations (single source of truth)
- Eliminate hard-coded mp0 assumption

Installer improvements (install-sensor-proxy.sh):
- Add mount configuration persistence validation via pct config check
- Surface pct set errors instead of silencing with 2>/dev/null
- Capture and display curl download errors with temp files
- Check systemd daemon-reload/enable/restart exit codes
- Show journalctl output when service fails to start
- Make socket verification fatal (was warning)
- Provide clear manual steps when hot-plug fails on running container

This makes the installation fail fast with actionable error messages
instead of silently proceeding with broken configuration.
2025-10-20 21:14:00 +00:00
rcourtman
001d7f5f1c fix: comprehensive temperature proxy setup improvements
Addresses multiple issues that prevented successful temperature monitoring setup:

1. **Missing log directory (install-sensor-proxy.sh)**
   - Added LogsDirectory=pulse/sensor-proxy to both systemd service templates
   - Fixes crash: "open /var/log/pulse/sensor-proxy/audit.log: read-only file system"
   - Uses systemd's LogsDirectory directive for proper permissions

2. **Invalid pct restart command (install-sensor-proxy.sh:822)**
   - Changed from `pct restart` (doesn't exist) to `pct stop && sleep 2 && pct start`
   - Fixes container restart failures during proxy setup

3. **Version compatibility check (config_handlers.go)**
   - Added const minProxyReadyVersion = "4.24.0"
   - Setup script now queries /api/version endpoint
   - Blocks proxy setup on Pulse < v4.24.0 with clear upgrade message
   - Prevents users from attempting proxy setup on incompatible versions

4. **Proxy service health validation (config_handlers.go)**
   - Verifies pulse-sensor-proxy service is actually running
   - Checks socket exists at /run/pulse-sensor-proxy/pulse-sensor-proxy.sock
   - Shows journalctl command for troubleshooting on failure
   - Sets TEMP_MONITORING_AVAILABLE=false to skip remaining steps

5. **Interactive LXC restart prompt (config_handlers.go)**
   - Replaced passive "please restart" message with interactive prompt
   - Default action is "yes" for easy acceptance
   - Actually executes pct stop/start on confirmation
   - Handles non-interactive environments gracefully

6. **Post-restart socket verification (config_handlers.go)**
   - Validates socket is accessible inside container after restart
   - Provides clear error if mount didn't work
   - Prevents claiming success when setup is incomplete

All changes tested with fresh LXC installation. Temperature monitoring now
works end-to-end with proper error handling and user guidance.

Fixes temperature proxy setup flow for v4.24.0+
2025-10-20 18:00:21 +00:00
rcourtman
c91b7874ac docs: comprehensive v4.24.0 documentation audit and updates
Complete documentation overhaul for Pulse v4.24.0 release covering all new
features and operational procedures.

Documentation Updates (19 files):

P0 Release-Critical:
- Operations: Rewrote ADAPTIVE_POLLING_ROLLOUT.md as GA operations runbook
- Operations: Updated ADAPTIVE_POLLING_MANAGEMENT_ENDPOINTS.md with DEFERRED status
- Operations: Enhanced audit-log-rotation.md with scheduler health checks
- Security: Updated proxy hardening docs with rate limit defaults
- Docker: Added runtime logging and rollback procedures

P1 Deployment & Integration:
- KUBERNETES.md: Runtime logging config, adaptive polling, post-upgrade verification
- PORT_CONFIGURATION.md: Service naming, change tracking via update history
- REVERSE_PROXY.md: Rate limit headers, error pass-through, v4.24.0 verification
- PROXY_AUTH.md, OIDC.md, WEBHOOKS.md: Runtime logging integration
- TROUBLESHOOTING.md, VM_DISK_MONITORING.md, zfs-monitoring.md: Updated workflows

Features Documented:
- X-RateLimit-* headers for all API responses
- Updates rollback workflow (UI & CLI)
- Scheduler health API with rich metadata
- Runtime logging configuration (no restart required)
- Adaptive polling (GA, enabled by default)
- Enhanced audit logging
- Circuit breakers and dead-letter queue

Supporting Changes:
- Discovery service enhancements
- Config handlers updates
- Sensor proxy installer improvements

Total Changes: 1,626 insertions(+), 622 deletions(-)
Files Modified: 24 (19 docs, 5 code)

All documentation is production-ready for v4.24.0 release.
2025-10-20 17:20:13 +00:00
rcourtman
0fcfad3dc5 feat: add shared script library system and refactor docker-agent installer
Implements a comprehensive script improvement infrastructure to reduce code
duplication, improve maintainability, and enable easier testing of installer
scripts.

## New Infrastructure

### Shared Library System (scripts/lib/)
- common.sh: Core utilities (logging, sudo, dry-run, cleanup management)
- systemd.sh: Service management helpers with container-safe systemctl
- http.sh: HTTP/download helpers with curl/wget fallback and retry logic
- README.md: Complete API documentation for all library functions

### Bundler System
- scripts/bundle.sh: Concatenates library modules into single-file installers
- scripts/bundle.manifest: Defines bundling configuration for distributables
- Enables both modular development and curl|bash distribution

### Test Infrastructure
- scripts/tests/run.sh: Test harness for running all smoke tests
- scripts/tests/test-common-lib.sh: Common library validation (5 tests)
- scripts/tests/test-docker-agent-v2.sh: Installer smoke tests (4 tests)
- scripts/tests/integration/: Container-based integration tests (5 scenarios)
- All tests passing ✓

## Refactored Installer

### install-docker-agent-v2.sh
- Reduced from 1098 to 563 lines (48% code reduction)
- Uses shared libraries for all common operations
- NEW: --dry-run flag support
- Maintains 100% backward compatibility with original
- Fully tested with smoke and integration tests

### Key Improvements
- Sudo escalation: 100+ lines → 1 function call
- Download logic: 51 lines → 1 function call
- Service creation: 33 lines → 2 function calls
- Logging: Standardized across all operations
- Error handling: Improved with common library

## Documentation

### Rollout Strategy (docs/installer-v2-rollout.md)
- 3-phase rollout plan (Alpha → Beta → GA)
- Feature flag mechanism for gradual deployment
- Testing checklist and success metrics
- Rollback procedures and communication plan

### Developer Guides
- docs/script-library-guide.md: Complete library usage guide
- docs/CONTRIBUTING-SCRIPTS.md: Contribution workflow
- docs/installer-v2-quickref.md: Quick reference for operators

## Metrics

- Code reduction: 48% (1098 → 563 lines)
- Reusable functions: 0 → 30+
- Test coverage: 0 → 8 test scenarios
- Documentation: 0 → 5 comprehensive guides

## Testing

All tests passing:
- Smoke tests: 2/2 passed (8 test cases)
- Integration tests: 5/5 scenarios passed
- Bundled output: Syntax validated, dry-run tested

## Next Steps

This lays the foundation for migrating other installers (install.sh,
install-sensor-proxy.sh) to use the same pattern, reducing overall
maintenance burden and improving code quality across the project.
2025-10-20 15:13:38 +00:00
rcourtman
aa5c08ad4a feat: implement priority queue-based task execution (Phase 2 Task 6)
Replaces immediate polling with queue-based scheduling:
- TaskQueue with min-heap (container/heap) for NextRun-ordered execution
- Worker goroutines that block on WaitNext() until tasks are due
- Tasks only execute when NextRun <= now, respecting adaptive intervals
- Automatic rescheduling after execution via scheduler.BuildPlan
- Queue depth tracking for backpressure-aware interval adjustments
- Upsert semantics for updating scheduled tasks without duplicates

Task 6 of 10 complete (60%). Ready for error/backoff policies.
2025-10-20 15:13:37 +00:00
rcourtman
c554380cb5 feat: verify adaptive interval logic implementation (Phase 2 Task 5)
Confirms adaptive scheduling logic is fully operational:
- EMA smoothing (alpha=0.6) to prevent interval oscillations
- Staleness-based interpolation between min/max intervals
- Error penalty (0.6x per error) for faster recovery detection
- Queue depth stretch (0.1x per task) for backpressure handling
- ±5% jitter to prevent thundering herd effects
- Per-instance state tracking for smooth transitions

Task 5 of 10 complete. Scheduler foundation ready for queue-based execution.
2025-10-20 15:13:37 +00:00
rcourtman
524f42cc28 security: complete Phase 1 sensor proxy hardening
Implements comprehensive security hardening for pulse-sensor-proxy:
- Privilege drop from root to unprivileged user (UID 995)
- Hash-chained tamper-evident audit logging with remote forwarding
- Per-UID rate limiting (0.2 QPS, burst 2) with concurrency caps
- Enhanced command validation with 10+ attack pattern tests
- Fuzz testing (7M+ executions, 0 crashes)
- SSH hardening, AppArmor/seccomp profiles, operational runbooks

All 27 Phase 1 tasks complete. Ready for production deployment.
2025-10-20 15:13:37 +00:00
rcourtman
67862e6f11 feat: add user-friendly explanation for socket bind mount
Added clear messaging to explain why the socket bind mount is configured,
focusing on the security benefits rather than technical implementation.

Changes:
- Add explanatory header "Secure Container Communication Setup"
- Explain the three key benefits:
  • Container communicates via Unix socket (not SSH)
  • No SSH keys exposed inside container (enhanced security)
  • Proxy on host manages all temperature collection
- Update technical messages to be more user-friendly:
  • "Configuring socket bind mount" instead of "Ensuring..."
  • "Restarting container to activate secure communication"
  • "Verifying secure communication channel"
  • "✓ Secure socket communication ready"
  • "Configuring Pulse to use proxy"

This helps users understand WHY the bind mount exists (security) rather
than just seeing technical implementation details.
2025-10-19 16:22:03 +00:00
rcourtman
171723a7d3 fix: automatically restart container when proxy mount is configured
Instead of warning the user to restart the container manually, the script
now automatically restarts it when the socket mount configuration is
updated. This ensures the mount is immediately active and temperature
monitoring works right away without user intervention.

Uses 'pct restart' if running, 'pct start' if stopped.
2025-10-19 15:56:31 +00:00
rcourtman
f81d77bb98 fix: fall back to Pulse server when GitHub download fails for pulse-sensor-proxy
The install-sensor-proxy.sh script now tries GitHub releases first, then falls
back to downloading from the Pulse server if GitHub fails or doesn't have the
binary (common when building from main).

The LXC installer sets PULSE_SENSOR_PROXY_FALLBACK_URL to point to the Pulse
server running inside the newly created LXC, ensuring the proxy binary can be
downloaded from /api/install/pulse-sensor-proxy.

This fixes the issue where installing with --main would fail to install
pulse-sensor-proxy on the host because GitHub releases don't include it yet.
2025-10-19 15:17:59 +00:00
rcourtman
049f79987f feat: add turnkey Docker installer with automatic proxy setup
Adds a one-command Docker deployment flow that:
- Detects if running in LXC and installs Docker if needed
- Automatically installs pulse-sensor-proxy on the Proxmox host
- Configures bind mount for proxy socket into LXC
- Generates optimized docker-compose.yml with proxy socket
- Enables temperature monitoring via host-side proxy

The install-docker.sh script handles the complete setup including:
- Docker installation (if needed)
- ACL configuration for container UIDs
- Bind mount setup
- Automatic apparmor=unconfined for socket access

Accessible via: curl -sSL http://pulse:7655/api/install/install-docker.sh | bash
2025-10-19 15:03:24 +00:00
Pulse Automation Bot
d15ad1d0b4 Add Helm chart tooling, CI, and release packaging 2025-10-18 11:50:57 +00:00
Richard Courtman
02701ca22b fix: gracefully handle standalone node cleanup limitation
- Cleanup script now detects forced command restriction on standalone nodes
- Logs helpful message explaining limitation (security by design)
- Does not fail when standalone nodes cannot be cleaned up
- Documents that standalone node cleanup is limited by forced command security
- Automatic cleanup works fully for cluster nodes
- Manual cleanup command provided for standalone nodes if needed
2025-10-18 07:34:18 +00:00
Richard Courtman
c9bbb5e6fb fix: use proxy SSH key for cleanup of standalone nodes
- Cleanup script now tries proxy's SSH key first for standalone nodes
- Falls back to default SSH if proxy key not available
- Fixes cleanup failure when Proxmox host doesn't have direct SSH to standalone nodes
2025-10-18 07:27:15 +00:00
Richard Courtman
7a7158d9bd feat: add automatic SSH key cleanup when nodes are removed
- Create cleanup script that removes Pulse SSH keys from nodes
- Add systemd path unit to watch for cleanup requests
- Add systemd service to execute cleanup script
- Update install-sensor-proxy.sh to install cleanup system
- Handles both cluster nodes (pulse-managed-key) and standalone nodes (pulse-proxy-key)
- Cleanup is triggered automatically when nodes are deleted from Pulse
- All cleanup actions are logged via syslog for auditability
2025-10-18 07:03:05 +00:00
Richard Courtman
669d7dc05c feat: add turnkey temperature monitoring for standalone nodes
Implements automatic temperature monitoring setup for standalone
Proxmox/Pimox nodes without manual SSH key configuration.

Changes:
- Add /api/system/proxy-public-key endpoint to expose proxy's SSH public key
- Setup script now detects standalone nodes (non-cluster)
- Auto-fetches and installs proxy SSH key with forced commands
- Add Raspberry Pi temperature support via cpu_thermal and /sys/class/thermal
- Enhance setup script with better error handling for lm-sensors installation
- Add RPi detection to skip lm-sensors and use native thermal interface

Security:
- Public key endpoint is safe (public keys are meant to be public)
- All installed keys use forced command="sensors -j" with full restrictions
- No shell access, port forwarding, or other SSH features enabled
2025-10-17 22:15:50 +00:00
rcourtman
5886b920ba fix: improve sensor proxy install script reliability
Fixes two issues with the sensor proxy installation:
1. Local node IP detection now uses exact matching instead of substring matching to avoid false negatives
2. Removes duplicate output filtering in the setup script wrapper

These changes ensure that the proxy SSH key is correctly configured on the local node during cluster installations.
2025-10-17 19:09:54 +00:00
rcourtman
123e0f04ca feat: add comprehensive node cleanup system
Implements automated cleanup workflow when nodes are deleted from Pulse, removing all monitoring footprint from the host. Changes include a new RPC handler in the sensor proxy for cleanup requests, enhanced node deletion modal with detailed cleanup explanations, and improved SSH key management with proper tagging for atomic updates.
2025-10-17 18:53:45 +00:00