Implements bullet-proof configuration management to completely eliminate
allowed_nodes corruption by design. This builds on Phase 1 (file-only mode)
by replacing all shell/Python config manipulation with proper Go tooling.
**New Features:**
- `pulse-sensor-proxy config validate` - parse and validate config files
- `pulse-sensor-proxy config set-allowed-nodes` - atomic node list updates
- File locking via flock prevents concurrent write races
- Atomic writes (temp file + rename) ensure consistency
- systemd ExecStartPre validation prevents startup with bad config
**Architectural Changes:**
1. Installer now calls config CLI instead of embedded Python/shell scripts
2. All config mutations go through single authoritative writer
3. Deduplication and normalization handled in Go (reuses existing logic)
4. Sanitizer kept as noisy failsafe (warns if corruption still occurs)
**Implementation Details:**
- New cmd/pulse-sensor-proxy/config_cmd.go with cobra commands
- withLockedFile() wrapper ensures exclusive access
- atomicWriteFile() uses temp + rename pattern
- Installer update_allowed_nodes() simplified to CLI calls
- Both systemd service modes include ExecStartPre validation
**Why This Works:**
- Single code path for all writes (no shell/Python divergence)
- File locking serializes self-heal timer + manual installer runs
- Validation gate prevents proxy from starting with corrupt config
- CLI uses same YAML parser as the daemon (guaranteed compatibility)
**Phase 2 Benefits:**
- Corruption impossible by design (not just detected and fixed)
- No more Python dependency for config management
- Atomic operations prevent partial writes
- Clear error messages on validation failures
The defensive sanitizer remains active but now logs loudly if triggered,
allowing us to confirm Phase 2 eliminates corruption in production before
removing the safety net entirely.
This completes the fix for the recurring temperature monitoring outages.
Related to Phase 1 commit
|
||
|---|---|---|
| .. | ||
| audit.go | ||
| audit_test.go | ||
| auth.go | ||
| auth_test.go | ||
| capabilities.go | ||
| cleanup.go | ||
| config.example.yaml | ||
| config.go | ||
| config_cmd.go | ||
| config_test.go | ||
| http_server.go | ||
| main.go | ||
| main_test.go | ||
| metrics.go | ||
| README.md | ||
| ssh.go | ||
| ssh_test.go | ||
| throttle.go | ||
| throttle_test.go | ||
| validation.go | ||
| validation_fuzz_test.go | ||
| validation_test.go | ||
pulse-sensor-proxy
The sensor proxy keeps SSH identities and temperature polling logic on the Proxmox host while presenting a small RPC surface (Unix socket or HTTPS) to the Pulse server. It protects SSH keys from container breakouts, enforces per-UID capabilities, and produces append-only audit logs.
Installation Options
| Scenario | Command |
|---|---|
| Recommended (automated) | curl -fsSL https://raw.githubusercontent.com/rcourtman/Pulse/main/scripts/install-sensor-proxy.sh | bash |
| Manual build | go build ./cmd/pulse-sensor-proxy and sudo install -m 0755 pulse-sensor-proxy /usr/local/bin/ |
| Prebuilt artifact | Copy the binary from /opt/pulse/bin/pulse-sensor-proxy-* inside the Pulse Docker image or download via /download/pulse-sensor-proxy?platform=linux&arch=amd64. |
The installer script provisions:
- User & group:
pulse-sensor-proxy - Binary:
/usr/local/bin/pulse-sensor-proxy - Config:
/etc/pulse-sensor-proxy/config.yaml - SSH material:
/var/lib/pulse-sensor-proxy/ssh - Socket:
/run/pulse-sensor-proxy/pulse-sensor-proxy.sock - Logs:
/var/log/pulse/sensor-proxy/{proxy.log,audit.log}(append-only) - Systemd units:
pulse-sensor-proxy.service, cleanup + self-heal timers
Start the service and verify status:
systemctl enable --now pulse-sensor-proxy
systemctl status pulse-sensor-proxy --no-pager
journalctl -u pulse-sensor-proxy -n 50
Configuration
The proxy reads /etc/pulse-sensor-proxy/config.yaml (see
config.example.yaml). Key fields:
| Key | Purpose | Notes |
|---|---|---|
allowed_source_subnets |
Restrict peers by CIDR | Empty list = auto-detect host networks |
allowed_peers[].uid/gid |
Capability-scoped authorisation | Prefer over legacy allowed_peer_uids |
allowed_peers[].capabilities |
read, write, admin |
read covers get_temperature; admin required for ensure_cluster_keys |
metrics_address |
Prometheus listener | Default 127.0.0.1:9127; set disabled to turn off |
require_proxmox_hostkeys |
Enforce known-host matches | Protects against SSH MITM |
max_ssh_output_bytes |
Cap command output | Prevents memory exhaustion (default 1 MiB) |
rate_limit.per_peer_interval_ms / per_peer_burst |
Token bucket guardrails | Keep interval ≥100 ms in production |
http_* keys |
HTTPS bridge mode | Needs TLS files plus bearer token |
Environment Overrides
PULSE_SENSOR_PROXY_SOCKET,PULSE_SENSOR_PROXY_SSH_DIR,PULSE_SENSOR_PROXY_CONFIG– relocate runtime pathsPULSE_SENSOR_PROXY_USER– run under a different service account (defaults topulse-sensor-proxy)PULSE_SENSOR_PROXY_ALLOWED_SUBNETS– comma-separated list appended at bootPULSE_SENSOR_PROXY_ALLOWED_PEER_UIDS/GIDS– extend authorisation without editing YAMLPULSE_SENSOR_PROXY_ALLOW_IDMAPPED_ROOT– explicitly allow/deny ID-mapped rootPULSE_SENSOR_PROXY_READ_TIMEOUT/_WRITE_TIMEOUT– Go duration stringsPULSE_SENSOR_PROXY_AUDIT_LOG– custom log path (still append-only)
HTTP Mode
Set http_enabled: true when the backend cannot mount the Unix socket (for
example, Kubernetes). Requirements:
- Populate
http_listen_addr(e.g.0.0.0.0:9443). - Provide
http_tls_cert/http_tls_key. The installer can place certs under/etc/pulse-sensor-proxy/tls. - Set a long
http_auth_token. Pulse sends it as a bearer token. - Restrict
allowed_source_subnetsto the Pulse control-plane addresses.
The HTTP server exports /temps and /health, enforces Bearer tokens, and logs
all HTTP access attempts to the audit log.
Audit Logging & Rotation
- Location:
/var/log/pulse/sensor-proxy/audit.log - Format: JSON with hash chaining (
prev_hash,event_hash,seq) - Access: Owned by
pulse-sensor-proxy,0640,chattr +a
Follow docs/operations/audit-log-rotation.md for rotation (remove +a,
truncate, restart service, reapply +a). Also consider forwarding with
scripts/setup-log-forwarding.sh; see
docs/operations/sensor-proxy-log-forwarding.md for RELP/TLS forwarding
instructions and verification steps.
Metrics & Monitoring
| Signal | Command |
|---|---|
| Prometheus metrics | curl -s http://127.0.0.1:9127/metrics | head |
| Scheduler health (Pulse) | curl -s http://localhost:7655/api/monitoring/scheduler/health | jq '.instances[] | select(.key | contains(\"temperature\")) | {key, breaker: .breaker.state, deadLetter: .deadLetter.present}' |
| Journal logs | journalctl -u pulse-sensor-proxy -f |
| Rate-limit hits | journalctl -u pulse-sensor-proxy | grep "rate limit" |
Set alerts on:
pulse_proxy_rate_limit_hits_totalspikes (potential abuse)pulse_proxy_hostkey_changes_totalincrements (SSH MITM)- Temperature instances showing
breaker.state != "closed"for >10 minutes
Troubleshooting
| Symptom | Guidance |
|---|---|
Cannot open audit log file |
Check permissions on /var/log/pulse/sensor-proxy. Remove chattr +a only during rotation. |
connection denied in audit log |
UID/GID not listed in allowed_peers. Verify Pulse container UID mapping. |
HTTP request from unauthorized source IP |
Update allowed_source_subnets or run through a reverse proxy that advertises the client IP via ProxyProtocol (not supported yet). |
rate limit exceeded |
Increase rate_limit.per_peer_burst or fix noisy hosts before relaxing limits. |
temperature pollers stuck |
Hit /api/monitoring/scheduler/health, ensure breakers are closed, restart Pulse + proxy if necessary. |
For additional hardening steps, read docs/PULSE_SENSOR_PROXY_HARDENING.md and
docs/TEMPERATURE_MONITORING_SECURITY.md.