danvics/Pulse
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
Pulse/internal
History
rcourtman 99e5a38534 Fix critical monitoring system issues and add robustness improvements 
This commit addresses 9 critical issues identified during the monitoring system audit:

**Race Conditions Fixed:**
- PBS backup pollers: Moved lock earlier to eliminate check-then-act race (lines 7316-7378)
- PVE backup poll timing: Fixed double write to lastPVEBackupPoll with proper synchronization (lines 5927-5977)
- Docker hosts cleanup: Refactored to avoid holding both m.mu and s.mu locks simultaneously (lines 1911-1937)

**Context Propagation Fixed:**
- Replaced all context.Background() calls with parent context for proper cancellation chain:
  - PBS backup poller (line 7367)
  - PVE backup poller (line 5955)
  - PBS fallback check (line 7154)

**Memory Leak Prevention:**
- Added cleanup for guest metadata cache (10 minute TTL, lines 1942-1957)
- Added cleanup for diagnostic snapshots (1 hour TTL, lines 1959-1987)
- Added cleanup for RRD cache (1 minute TTL, lines 1989-2007)
- All cleanup methods called on 10-second ticker (lines 3791-3793)

**Panic Recovery:**
- Added recoverFromPanic helper to log panics with stack traces (lines 1910-1920)
- Protected all critical goroutines:
  - poll (line 4020)
  - taskWorker (line 4200)
  - retryFailedConnections (line 3851)
  - checkMockAlerts (line 8896)
  - pollPVEInstance (line 4886)
  - pollPBSInstance (line 7164)
  - pollPMGInstance (line 7498)

**Import Fixes:**
- Added missing sync import to email_enhanced.go
- Added missing os import to queue.go

All fixes maintain proper lock ordering and release locks before calling methods that acquire other locks to prevent deadlocks.
2025-11-07 08:52:37 +00:00
..
alerts Fix webhook alerts persisting when DisableAll* flags are enabled 2025-11-06 21:17:56 +00:00
api Add encryption status to notification health endpoint (P2) 2025-11-07 08:36:55 +00:00
auth Refactor: Code cleanup and localStorage consolidation 2025-11-04 21:50:46 +00:00
config Add encryption status to notification health endpoint (P2) 2025-11-07 08:36:55 +00:00
crypto Harden setup token flow and enforce encrypted persistence 2025-10-25 16:00:37 +00:00
discovery Refactor: Code cleanup and localStorage consolidation 2025-11-04 21:50:46 +00:00
dockeragent Fix critical version embedding issues for 4.26 release 2025-11-06 11:42:52 +00:00
errors Fix settings security tab navigation 2025-10-11 23:29:47 +00:00
hostagent Normalize docker agent version handling 2025-10-28 08:42:58 +00:00
hostmetrics Normalize docker agent version handling 2025-10-28 08:42:58 +00:00
logging feat: comprehensive diagnostics and observability improvements 2025-10-21 12:37:39 +00:00
metrics Add comprehensive alert system reliability improvements 2025-11-06 16:46:30 +00:00
mock Refactor: Code cleanup and localStorage consolidation 2025-11-04 21:50:46 +00:00
models Add AMD GPU temperature monitoring support 2025-11-06 00:19:04 +00:00
monitoring Fix critical monitoring system issues and add robustness improvements 2025-11-07 08:52:37 +00:00
notifications Fix critical monitoring system issues and add robustness improvements 2025-11-07 08:52:37 +00:00
ssh/knownhosts Add configurable SSH port for temperature monitoring 2025-11-05 20:03:29 +00:00
system Enhance container detection for temperature SSH safeguards (refs #601) 2025-11-04 22:30:35 +00:00
tempproxy Fix temperature data intermittency caused by proxy rate limit retries 2025-11-05 10:20:15 +00:00
types Fix settings security tab navigation 2025-10-11 23:29:47 +00:00
updates Fix critical rollback download URL bug and doc inconsistencies 2025-11-06 14:25:32 +00:00
utils Refactor: Code cleanup and localStorage consolidation 2025-11-04 21:50:46 +00:00
websocket Add comprehensive alert system reliability improvements 2025-11-06 16:46:30 +00:00