Commit graph

31 commits

Author SHA1 Message Date
rcourtman
e7cf40c2c6 test: Add empty ID test for HandleDockerHostRemoved function
- empty host ID is noop (preserves existing alerts/tracking)
- HandleDockerHostRemoved 75%→100%
2025-12-01 15:56:11 +00:00
rcourtman
7580d0b3d5 test: Add comprehensive tests for cleanupHostDiskAlerts function
- clears alerts not in seen set (stale disk alerts)
- empty host ID is noop
- skips nil alerts without panic
- skips non-matching prefix (non-disk alerts)
- cleanupHostDiskAlerts 76.9%→100%
2025-12-01 15:54:33 +00:00
rcourtman
39a856aaf8 test: Add comprehensive tests for clearHostDiskAlerts function
- clears all disk alerts for specified host
- empty hostID is noop
- skips nil alerts without panic
- noop when no matching alerts
- clearHostDiskAlerts 72.7%→100%
2025-12-01 15:52:39 +00:00
rcourtman
32b346b8ce test: Add comprehensive tests for clearHostMetricAlerts function
- clears specified metrics from host
- defaults to cpu and memory when no metrics specified
- empty hostID is noop
- clearHostMetricAlerts 85.7%→100%
2025-12-01 15:50:59 +00:00
rcourtman
a3aef773ec test: Add comprehensive tests for clearStorageOfflineAlert function
- clears existing offline alert and confirmation
- triggers resolved callback
- noop when no alert exists
- clears offline confirmation even without alert
- clearStorageOfflineAlert 50%→100%
2025-12-01 15:48:32 +00:00
rcourtman
2fbf9cbe7b test: Add comprehensive tests for dockerServiceResourceID function
- with/without host ID (prefix selection)
- derives ID from service name with sanitization
- special chars replaced with dashes
- preserves alphanumeric, underscore, hyphen
- trims leading/trailing dashes/underscores
- truncates long IDs to 32 chars
- fallback to 'service' when empty
- dockerServiceResourceID 23.8%→100%
2025-12-01 15:46:34 +00:00
rcourtman
884d04ae61 test: Add comprehensive tests for dockerServiceDisplayName function
- returns name when present
- returns trimmed name
- returns truncated ID when name empty (first 12 chars)
- returns full short ID when < 12 chars
- returns 'service' when both empty/whitespace
- prefers name over ID
- dockerServiceDisplayName 33.3%→100%
2025-12-01 15:44:40 +00:00
rcourtman
a97a4a022c test: Add comprehensive tests for shouldNotifyAfterCooldown function
- cooldown disabled (0) allows notification
- negative cooldown allows notification
- first notification allowed when never notified
- notification blocked during cooldown period
- notification allowed after cooldown expires
- boundary condition (exact cooldown time)
- shouldNotifyAfterCooldown 42.9%→100%
2025-12-01 15:42:35 +00:00
rcourtman
46f0632c53 test: Add comprehensive tests for applyRelaxedGuestThresholds function
- nil thresholds get defaults (CPU 95, Memory 92, Disk 95)
- low thresholds raised to minimum relaxed values
- high thresholds unchanged
- clear adjusted when too close to trigger
- original config unchanged (no mutation)
- alerts package coverage improvement: applyRelaxedGuestThresholds 75%→93.8%
2025-12-01 15:40:19 +00:00
rcourtman
10498a0d54 test: Add comprehensive tests for checkRateLimit function
Cover all branches: disabled rate limit (zero/negative MaxAlertsHour),
under limit, at limit blocking, separate limits per alertID, old entry
cleanup, and mixed old/recent entries. Coverage improved from 71.4% to 100%.
2025-12-01 15:35:19 +00:00
rcourtman
c488956001 test: Add comprehensive tests for getMetricTimeThreshold function
Cover all branches: empty config, empty/whitespace metricType, direct
match, canonical key matching (vm→guest, container→guest), fallback
chain (default, _default, wildcard), precedence, and no-match scenarios.
Coverage improved from 77.8% to 100%.
2025-12-01 15:32:33 +00:00
rcourtman
844b9e5637 test: Add comprehensive tests for getBaseTimeThreshold function
Cover all branches: nil/empty TimeThresholds, direct match, canonical key
matching (vm→guest, container→guest), "all" fallback, specific match
precedence, and global threshold fallback. Coverage improved from 85.7% to 100%.
2025-12-01 15:25:18 +00:00
rcourtman
fecab94dc8 test: Add comprehensive tests for getGlobalMetricTimeThreshold function
Cover all branches: empty config, missing 'all' key, empty perType map,
empty/whitespace metricType, direct match, case insensitivity, default
fallback, _default fallback, wildcard fallback, and no-match scenarios.
Coverage improved from 58.8% to 100%.
2025-12-01 15:17:12 +00:00
rcourtman
606764f931 test: Add comprehensive tests for checkFlapping function
Cover all branches of flapping detection: disabled state, below/at/above
threshold, already-flapping state (no suppression update), and window
expiry pruning. Coverage improved from 61.9% to 95.2%.
2025-12-01 15:11:11 +00:00
rcourtman
c7d832c8d9 test: Add comprehensive tests for BuildGuestKey function
- Test different instance and node (full format)
- Test same instance and node (short format)
- Test empty instance defaults to node
- Test whitespace trimming
- Improves BuildGuestKey coverage from 71.4% to 100%
2025-12-01 15:04:37 +00:00
rcourtman
6ff54beb38 Handle ignored docker containers during alert reevaluation (related to #561) 2025-11-21 22:19:19 +00:00
rcourtman
4e68675e0b Add recovery notifications and grouping controls 2025-11-21 22:07:00 +00:00
rcourtman
8f05fc0a57 Improve backup-age alerts to show VM/CT names in multi-cluster setups (related to #668)
This change fixes backup-age alert notifications to display VM/CT names
instead of just "VMID XXX" in multi-cluster environments where backups
are stored on PBS.

Changes:
- Store all guests per VMID (not just first match) to handle VMID collisions across clusters
- Persist last-known guest names/types in metadata store for deleted VMs
- Enrich backup correlation with persisted metadata when live inventory is empty
- Update CheckBackups to handle multiple VMID matches intelligently

The fix addresses two scenarios:
1. Multiple PVE clusters with same VMID backing up to one PBS
2. VMs deleted from Proxmox but backups still exist on PBS

Backup-age alerts will now show proper VM/CT names when:
- A unique guest exists with that VMID (live or persisted)
- Multiple guests share a VMID (uses first match, consistent with current behavior)

When truly ambiguous (multiple live VMs, same VMID, no way to determine origin),
the alert gracefully falls back to showing "VMID XXX".
2025-11-08 18:24:04 +00:00
rcourtman
4891f06e76 Fix webhook alerts persisting when DisableAll* flags are enabled
The original fix in c6c0ac63e only handled per-resource overrides when
thresholds were disabled (trigger <= 0 or Disabled=true). It did not
handle global DisableAll* flags (DisableAllStorage, DisableAllNodes,
DisableAllGuests, etc.).

When a user toggled a DisableAll* flag from false to true:
- Check* functions returned early without processing
- Existing active alerts remained in m.activeAlerts map
- Those alerts continued generating webhook notifications
- reevaluateActiveAlertsLocked didn't check DisableAll* flags

This commit fixes the issue by:

1. Updating reevaluateActiveAlertsLocked to check all DisableAll* flags
   and resolve alerts for those resource types during config updates

2. Adding alert cleanup to Check* functions before early returns:
   - CheckStorage: clears usage and offline alerts
   - CheckNode: clears cpu/memory/disk/temperature and offline alerts
   - CheckPMG: clears queue/message alerts and offline alerts
   - CheckPBS: clears cpu/memory and offline alerts
   - CheckHost: calls existing cleanup helpers

3. Adding comprehensive test coverage for DisableAllStorage scenario

Related to #561
2025-11-06 21:17:56 +00:00
rcourtman
b44084af3c Skip false health alerts for Samsung 980/990 SSDs and improve Docker CPU calculation
Related to #547 and #622

## Samsung SSD Fix (#547)
Samsung 980 and 990 series SSDs have known firmware bugs that cause them to
report incorrect health status (typically FAILED or critical warnings) even
when the drives are actually healthy. This is commonly due to incorrect
temperature threshold reporting in the firmware.

This change adds special handling to detect these drives and skip health
status alerts while still monitoring wearout metrics, which remain reliable.
The fix also clears any existing false alerts for these drives.

Users experiencing these false alerts should update their Samsung SSD firmware
to the latest version from Samsung, which typically resolves the issue.

## Docker Agent CPU Fix (#622)
Addresses issue where Docker container CPU usage shows 0%. The Docker
agent uses ContainerStatsOneShot which typically doesn't populate
PreCPUStats, requiring manual delta tracking between collection cycles.

Changes:
- Fix logic bug where prevContainerCPU was updated before checking if
  previous sample existed, causing incorrect delta calculations
- Add comprehensive debug logging showing which calculation method
  succeeded (PreCPUStats, system delta, or time-based fallback)
- Add warning after 10 PreCPUStats failures to inform about manual
  tracking mode (normal for one-shot stats)
- Add detailed failure logging when CPU calculation cannot complete

Expected behavior: First collection cycle returns 0% (no previous
sample), subsequent cycles show accurate CPU metrics.
2025-11-05 19:33:16 +00:00
rcourtman
fb22469eb0 Add disk usage threshold support for Docker containers
Extends the Docker monitoring and alerting system to track writable layer
usage as a percentage of the container's root filesystem. This helps
identify containers with bloated copy-on-write layers before they
consume excessive disk space.

- Add disk threshold to DockerThresholdConfig (default: 85% trigger, 80% clear)
- Evaluate disk alerts for running containers when RootFilesystemBytes > 0
- Include disk metadata (writable layer, total filesystem, block I/O stats)
- Update frontend to display and configure disk thresholds
- Add test coverage for disk usage alert hysteresis
- Document disk monitoring in DOCKER_MONITORING.md

Per-container and per-host overrides apply to disk thresholds the same
way they do for CPU and memory.
2025-10-29 14:52:25 +00:00
rcourtman
b3285c05c8 Consolidate pending changes
- Add Docker metadata test comment
- Update alerts configuration and thresholds
- Enhance config file watcher
- Update documentation
- Refine settings UI
2025-10-28 23:20:44 +00:00
rcourtman
68ce8e7520 feat: finalize swarm service monitoring (#598) 2025-10-26 09:35:49 +00:00
rcourtman
77282bd3a6 Implement Pulse tag overrides and alert clear persistence 2025-10-25 14:28:32 +00:00
rcourtman
d643dcf0bc perf: reduce polling allocations and guest metadata load 2025-10-25 13:12:47 +00:00
rcourtman
be26f957c0 Add snapshot size alert thresholds (#585) 2025-10-22 13:30:40 +00:00
Pulse Automation Bot
cfdfe896be Adjust backup and snapshot alert handling 2025-10-18 20:11:01 +00:00
Pulse Automation Bot
80b9d0602a Add Apprise notification integration (#570) 2025-10-18 16:39:39 +00:00
rcourtman
219fcc6de5 Stop disabled metrics from sending webhooks
Refs #561
2025-10-16 08:57:12 +00:00
rcourtman
4838793677 feat: enhance alerts system with tests and improved thresholds
- Add comprehensive test coverage for alerts package with 285+ new tests
- Implement ThresholdsTable component with metric thresholds display
- Enhance Alerts page UI with improved layout and metric filtering
- Add frontend component tests for Alerts page and ThresholdsTable
- Set up Vitest testing infrastructure for SolidJS components
- Improve config persistence with better validation
- Expand discovery tests with 333+ test cases
- Update API, configuration, and Docker monitoring documentation
2025-10-15 22:25:04 +00:00
rcourtman
f46ff1792b Fix settings security tab navigation 2025-10-11 23:29:47 +00:00