danvics/Pulse
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Fix Proxmox 9.x RRD parameter incompatibility causing cluster health issues

Browse source 
Proxmox VE 9.x removed support for the 'ds' parameter in RRD endpoints
(/nodes/{node}/rrddata and /nodes/{node}/lxc/{vmid}/rrddata). When Pulse
sent RRD requests with ds=memused,memavailable,etc., Proxmox responded with:

  API error 400: {"errors":{"ds":"property is not defined in schema..."}}

This caused cluster nodes to be repeatedly marked unhealthy, which cascaded
into storage polling failures showing 'All cluster endpoints are unhealthy'
even though the nodes were actually healthy and reachable.

Changes:
- Added check in cluster_client.go executeWithFailover to recognize the ds
  parameter error as a capability issue rather than node health failure
- Nodes with this error no longer get marked unhealthy
- Storage polling and other operations now succeed even when RRD calls fail
- The RRD data will be unavailable but core monitoring continues

This fix maintains backward compatibility with older Proxmox versions while
gracefully handling the API change in Proxmox 9.x.
This commit is contained in:
rcourtman 2025-11-08 12:06:08 +00:00
parent cb38a886ea
commit a406fe42d8
1 changed files with 2 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
pkg/proxmox/cluster_client.go
View file

@ -579,6 +579,7 @@ func (cc *ClusterClient) executeWithFailover(ctx context.Context, fn func(*Clien
// Error 500 with "No QEMU guest agent configured" means VM-specific issue, not node failure
// Error 500 with "QEMU guest agent is not running" means VM-specific issue, not node failure
// Error 500 with any "guest agent" message means VM-specific issue, not node failure
// Error 400 with "ds" parameter error means Proxmox 9.x doesn't support RRD data source filtering
// JSON unmarshal errors are data format issues, not connectivity problems
if strings.Contains(errStr, "595") ||
(strings.Contains(errStr, "500") && strings.Contains(errStr, "hostname lookup")) ||
@ -589,6 +590,7 @@ func (cc *ClusterClient) executeWithFailover(ctx context.Context, fn func(*Clien
strings.Contains(errStr, "guest agent") ||
(strings.Contains(errStr, "403") && (strings.Contains(errStr, "storage") || strings.Contains(errStr, "datastore"))) ||
strings.Contains(errStr, "permission denied") ||
(strings.Contains(errStr, "400") && strings.Contains(errStr, "\"ds\"") && strings.Contains(errStr, "property is not defined in schema")) ||
strings.Contains(errStr, "json: cannot unmarshal") ||
strings.Contains(errStr, "unexpected response format") {
// This is likely a node-specific failure, not an endpoint failure