Commit graph

578 commits

Author SHA1 Message Date
courtmanr@gmail.com
0a12200e58 fix(tests): resolve flaky ID generation and symlink resolution on macOS 2025-11-27 10:50:13 +00:00
rcourtman
e9ac1429c0 refactor: remove unnecessary type conversions
Remove redundant type conversions identified by unconvert linter:
- Remove int() conversions for already-int VMID fields
- Remove int64() conversions for already-int64 arithmetic results
- Remove uint64() conversions for already-uint64 Disk/MaxDisk fields
- Remove int() on syscall.Stdin (already int constant)
2025-11-27 10:33:35 +00:00
rcourtman
67b60adb65 style: fix revive linter warnings
- Mark unused stub parameters with underscore
- Rename 'copy' variable to avoid shadowing builtin
- Remove unnecessary else blocks after return statements
2025-11-27 10:26:26 +00:00
rcourtman
1fb4a2c2c6 chore: remove deprecated build tags and use strings.ReplaceAll
- Remove redundant // +build directives (go:build is sufficient in Go 1.17+)
- Replace strings.Replace(..., -1) with strings.ReplaceAll
2025-11-27 10:16:08 +00:00
rcourtman
31927f71e9 fix: mark unused parameters to satisfy unparam linter
Mark intentionally unused parameters with underscore to:
- Silence unparam warnings for legitimate unused parameters
- Keep function signatures intact for API compatibility
- Remove unused req from serveChecksum helper
2025-11-27 10:12:48 +00:00
rcourtman
252d060a58 refactor: use builtin max() and fix unused parameter
- Replace custom maxInt64 helper with Go 1.21+ builtin max()
- Mark unused cfg parameter in newAdaptiveIntervalSelector
- Remove test for deleted helper function
2025-11-27 10:08:37 +00:00
rcourtman
16603e8b4a fix: remove ineffectual assignments
- Fix loop variable reassignment in config_handlers.go
- Remove redundant boolean assignments in swarm.go
2025-11-27 09:48:29 +00:00
rcourtman
60cd019d33 chore: update to non-deprecated Docker SDK types
- Use container.Summary instead of types.Container
- Use swarmtypes.ServiceListOptions instead of types.ServiceListOptions
- Use swarmtypes.TaskListOptions instead of types.TaskListOptions

These types were deprecated in favor of package-specific types.
2025-11-27 09:36:05 +00:00
rcourtman
d7d1631e1c style: use direct type conversions in model converters
Replace struct literal copies with direct type conversions where
the source and destination types have identical field layouts.
2025-11-27 09:24:39 +00:00
rcourtman
95a258ac32 refactor: replace deprecated strings.Title with custom titleCase
Add simple titleCase functions that handle ASCII strings without
the Unicode punctuation issues of strings.Title.
2025-11-27 09:22:52 +00:00
rcourtman
7dad9c7a17 style: fix additional staticcheck warnings
- Lowercase error messages (ST1005)
- Use context.Background() instead of nil (SA1012)
- Fix rand.Intn(1) which always returns 0 (SA4030)
- Remove unnecessary nil check before len() (S1009)
2025-11-27 09:21:11 +00:00
rcourtman
8a1ee3b05e style: fix staticcheck style warnings
- Merge variable declaration with assignment (S1021)
- Use unconditional strings.TrimPrefix (S1017)
- Remove unnecessary nil checks around range (S1031)
- Remove unnecessary fmt.Sprintf (S1039)
- Use copy() instead of manual loop (S1001)
- Use time.Until instead of t.Sub(time.Now()) (S1024)
- Use buf.String() instead of string(buf.Bytes()) (S1030)
2025-11-27 09:19:33 +00:00
rcourtman
e25e1af8cb chore: fix staticcheck SA warnings
- Fix SA4006 unused value issues in ssh.go, validation.go, generator.go
- Replace deprecated ioutil with io/os in config.go
- Replace deprecated tar.TypeRegA with tar.TypeReg
- Remove deprecated rand.Seed calls (auto-seeded in Go 1.20+)
- Fix always-true nil check in main.go
- Fix impossible nil comparison in tempproxy/client.go
- Add nil check for config in monitor.New()
2025-11-27 09:16:53 +00:00
rcourtman
4fd3bdbc04 chore: fix staticcheck U1000 unused code warnings
- Remove unused ipv6Regex from validation.go
- Suppress unused recordAlertFired/recordAlertResolved hooks (kept for future use)
- Remove unused apiLimiter rate limiter
- Remove unused stopOnce fields from csrf_store.go and session_store.go
- Remove unused lastBroadcast field from hub.go
- Remove unused lastUsedIndex field from cluster_client.go
2025-11-27 09:12:17 +00:00
rcourtman
4f25aa9ca1 chore: cleanup proxmox IsAuthError and remove stray comment
- Make IsAuthError unexported (isAuthError) since it's only used internally
- Remove stray '// test comment' from docker_metadata.go
2025-11-27 08:59:01 +00:00
rcourtman
3b23d16c23 chore: remove unused ClusterHealth types
Remove internal/models/cluster.go containing ClusterHealth and
ClusterNodeStatus types that were never used in the codebase.
2025-11-27 08:45:50 +00:00
rcourtman
93723d95c6 chore: remove unused API response types
Remove ChartData, Dataset, ConfigImportResponse, ConfigExportResponse,
InstallScriptResponse, ErrorResponse, and SuccessResponse types that
were defined but never used in the codebase.
2025-11-27 08:37:22 +00:00
rcourtman
4f6c24802c style: fix whitespace in middleware.go 2025-11-27 08:34:31 +00:00
rcourtman
c8e76a52b7 chore: remove unused store methods
Remove 121 lines of unused store methods:
- CSRFTokenStore: Stop, ExtendCSRFToken
- SessionStore: Stop, ExtendSession, GetSession
- RecoveryTokenStore: Stop, save, GetActiveTokenCount, ValidateRecoveryToken

These methods were part of a standard store pattern but never wired up
to the application lifecycle. The constant-time validation variant is
used instead of the timing-vulnerable ValidateRecoveryToken.
2025-11-27 08:31:50 +00:00
rcourtman
72b2444a6c chore: remove unused DockerMetadataHandler.Reload method 2025-11-27 05:14:56 +00:00
rcourtman
b0ce0d932f chore: remove additional dead code
Remove 241 lines of unreachable code across internal and pkg:
- internal/crypto/crypto.go: unused NewCryptoManager wrapper
- internal/monitoring/scheduler.go: unused fixedIntervalSelector type
- internal/ssh/knownhosts/manager.go: unused hostKeyExists function
- internal/updates/manager.go: unused getLatestRelease wrapper
- internal/updates/updater.go: unused GetAll method
- pkg/discovery/discovery.go: unused scanWorker and runPhase (legacy compat)
- pkg/proxmox/client.go: unused post, getTaskStatus, waitForTaskCompletion, getTaskLog
- pkg/proxmox/cluster_client.go: unused markUnhealthy wrapper
2025-11-27 05:13:26 +00:00
rcourtman
25542ae51d chore: remove more dead code
Remove 330 lines of unreachable code:
- internal/monitoring/temperature_service.go: unused temperature service abstraction
- internal/monitoring/temperature.go: unused NewTemperatureCollector wrapper
- internal/mock/generator.go: unused GenerateAlerts function
- internal/mock/integration.go: unused ToggleMockMode wrapper
- internal/notifications/notifications.go: unused sendEmailWithContent,
  generatePayloadFromTemplate, isPrivateRange172, groupAlerts
- internal/notifications/email_providers.go: unused GetProviderDefaults
2025-11-27 00:10:55 +00:00
rcourtman
28aaecd74d chore: remove dead code and unused files
Remove 604 lines of unreachable code identified by deadcode analysis:
- internal/config/credentials.go: unused credential resolver
- internal/config/registration.go: unused registration config
- internal/monitoring/poller.go: unused channel-based polling (keep types)
- internal/api/middleware.go: unused TimeoutHandler, JSONHandler, NewAPIError, ValidationError
- internal/api/security.go: unused IsLockedOut, SecurityHeaders
- internal/api/auth.go: unused min helper
- internal/config/config.go: unused SaveConfig
- internal/config/client_helpers.go: unused CreatePBSConfigFromFields
- internal/logging/logging.go: unused NewRequestID
2025-11-27 00:05:04 +00:00
rcourtman
506484b072 chore: remove unused API types
Remove 261 lines of unused type definitions from types.go:
- NodeRequest, SettingsRequest (unused, actual impl in config_handlers.go)
- ConfigResponse, NodeConfig, SettingsConfig (unused)
- BackupResponse, BackupInfo, MetricsResponse, MetricData (unused)
- StorageResponse, StorageInfo, StorageTotals (unused)
- DiagnosticsResponse and related types (unused)
- SecurityStatusResponse, ExportRequest, ImportRequest (unused)
- NotificationTestRequest, UpdateCheckResponse (unused)
- WebSocketMessage, LoginRequest, LoginResponse (unused)
- TestConnectionResponse, NodeConnectionResponse (unused)
- DiscoveryResponse, DiscoveredServer (pkg/discovery has own types)
- AutoRegisterResponse (unused)
2025-11-26 23:51:41 +00:00
rcourtman
2fe7bb6141 style: fix gofmt formatting inconsistencies
Run gofmt -w to fix tab/space inconsistencies across 33 files.
2025-11-26 23:44:36 +00:00
rcourtman
b40a33fd2b fix: prevent context leak in temperature collection
- Use defer for tempCancel() to ensure context is always cancelled
- Remove redundant shouldCollect variable that was always true
- Fix indentation after removing the unnecessary conditional block
2025-11-26 23:43:18 +00:00
rcourtman
91c72a274c feat: add backup status indicator to guest table
Shows shield icon next to guest name when backup is stale or missing:
- Yellow shield: backup 24-72 hours old
- Red shield: backup older than 72 hours
- Gray shield: no backup found

Fresh backups (<24h) show no indicator to keep the UI clean.
Templates are excluded from backup status display.
2025-11-26 21:30:41 +00:00
rcourtman
d2a303ed0c fix: resolve symlinks for agent self-update to avoid cross-device rename
When the agent binary is symlinked (e.g., from /opt/pulse/bin to
/usr/local/bin), the self-update would fail with "invalid cross-device
link" because os.Rename() doesn't work across filesystems.

Now resolves symlinks before update and creates the temp file in the
same directory as the real binary.

Related to #737
2025-11-26 20:16:09 +00:00
courtmanr@gmail.com
c021dc6ca5 Enhance table responsiveness across multiple components 2025-11-26 17:57:09 +00:00
rcourtman
7fd49fb54a test: add additional tests for internal/agentupdate
Add 6 tests covering:
- Config struct zero value defaults
- NewUpdater with InsecureSkipVerify option
- NewUpdater with Disabled option
- determineArch format validation
- verifyBinaryMagic on directories
- Package constants validation
2025-11-26 14:37:01 +00:00
rcourtman
36544d7487 test: add unit tests for internal/models converters
Add 22 tests covering:
- zeroIfNegative helper function
- copyStringFloatMap deep copy helper
- Node.ToFrontend with display name fallback
- VM.ToFrontend including tags, backup time, negative values
- Container.ToFrontend
- DockerContainer.ToFrontend with ports, networks, mounts, block IO
- DockerService.ToFrontend with endpoint ports
- DockerSwarmInfo.ToFrontend
- Storage.ToFrontend
- hostSensorSummaryToFrontend nil handling
- RemovedDockerHost.ToFrontend
- DockerServicePort.ToFrontend
- DockerTask.ToFrontend with timestamps
2025-11-26 14:33:57 +00:00
rcourtman
3c499322b9 test: add unit tests for internal/metrics
Add 14 tests covering:
- RecordAlertFired, RecordAlertResolved, RecordAlertAcknowledged
- RecordNotificationSent with success/failure
- RecordNotificationRetry, RecordNotificationDLQ
- RecordGroupedNotification with various counts
- RecordAlertEscalation with various levels
- RecordAlertSuppressed with various reasons
- UpdateQueueDepth, UpdateHistorySize
- RecordHistorySaveError
- Metric vector initialization verification
2025-11-26 14:27:54 +00:00
rcourtman
cf45a62c12 test: add unit tests for internal/tempproxy
Add 24 tests covering:
- ErrorType enum constants
- ProxyError Error() and Unwrap() methods
- calculateBackoff exponential backoff with jitter
- contains case-insensitive substring matching
- classifyError error classification for node, auth, SSH, sensor, rate limit, timeout, transport
- RPCRequest and RPCResponse struct fields
- Client struct fields
2025-11-26 14:26:59 +00:00
rcourtman
65b0644027 test: add unit tests for internal/agentbinaries 2025-11-26 14:17:58 +00:00
rcourtman
35412b63d8 test: add unit tests for system package 2025-11-26 14:11:25 +00:00
rcourtman
61b2bfe624 test: add unit tests for types package 2025-11-26 14:10:21 +00:00
rcourtman
0396053b73 test: add unit tests for hostagent package 2025-11-26 14:09:55 +00:00
rcourtman
159a1cb3a3 test: add unit tests for errors package 2025-11-26 14:09:14 +00:00
rcourtman
ae876b4a10 test: add unit tests for sensors parser 2025-11-26 14:06:56 +00:00
rcourtman
4bdf1664bb test: add unit tests for utils package
- Test ID generation (uniqueness, format)
- Test JSON response writing (various types, headers)
- Test boolean parsing (truthy/falsy values)
- Test environment variable trimming
- Test data directory resolution
- Test large payload handling
2025-11-26 13:56:30 +00:00
rcourtman
d37247d220 test: add unit tests for crypto package
- Test encrypt/decrypt round-trip (various data types)
- Test string encryption (base64 output)
- Test key persistence across manager instances
- Test key file permissions (0600)
- Test decryption of invalid/corrupted data
- Test encryption uniqueness (random nonce)
- Test orphaned data protection
- Test large data encryption (1MB)
2025-11-26 13:52:05 +00:00
rcourtman
16d70123bf test: add unit tests for auth package
- Test API token generation (uniqueness, format)
- Test API token hashing (SHA3-256, deterministic)
- Test constant-time token comparison
- Test token hash detection
- Test password hashing (bcrypt, salted)
- Test password verification
- Test password complexity validation
- Verify bcrypt cost and minimum password length constants
2025-11-26 13:51:13 +00:00
rcourtman
1ccbdfe046 test: add unit tests for agentupdate security features
- Test binary magic verification (ELF, PE, Mach-O)
- Test architecture detection
- Test Unraid persistent path generation
- Test updater configuration defaults
- Test edge cases: empty files, too short, non-existent
2025-11-26 13:50:19 +00:00
rcourtman
09ec0c3f01 security: harden agent installers and auto-update mechanism
Install script (scripts/install.sh):
- Add multi-platform support: Unraid, OpenRC/Alpine, Synology DSM 6/7
- Add input validation for URL, token format, and interval
- Add binary magic verification (ELF/Mach-O/PE)
- Add cleanup trap for temp files
- Wrap script in main() for partial download protection
- Fix shellcheck compliance issues
- Add curl timeouts

Agent auto-update (agentupdate, dockeragent):
- Enforce TLS 1.2 minimum version
- Make SHA256 checksum verification mandatory
- Add 100MB binary size limit
- Add binary magic verification before replacement
- Add Unraid persistent binary update after self-update
- Add 5-minute download timeout

Frontend:
- Update Linux install description to note auto-detection of init systems
2025-11-26 13:14:58 +00:00
rcourtman
7f449d30e0 fix: cache daemon ID at init to prevent Podman token binding conflicts
Podman can return unstable or empty daemon IDs across API calls. When
the agent fetched info.ID on every report cycle, this could cause the
agent identity to change mid-session, triggering "token already in use"
errors on the server.

Cache the daemon ID at initialization and use it consistently for all
reports.

Related to #740
2025-11-26 10:23:22 +00:00
rcourtman
d07ad8fd3e fix: use correct script paths in unified agent handlers
The unified agent handlers were using r.config.AppRoot which pointed
to /app, but scripts are in /opt/pulse/scripts. Updated to match the
pattern used by other script handlers - check /opt/pulse/scripts first,
then fall back to project root for dev environment.

Also added no-cache headers to prevent stale scripts being served.
2025-11-26 10:05:43 +00:00
rcourtman
21c033c144 fix: remove references to deleted install-host-agent.sh script
The unified agent system replaced install-host-agent.sh with install.sh.
This commit updates all references:
- Dockerfile: removed COPY for deleted script
- router.go: serve install.sh at /install-host-agent.sh endpoint (backwards compatible)
- build-release.sh: removed copy of deleted script
- validate-release.sh: removed validation of deleted script
- install.sh: updated script list for bare-metal installs
2025-11-26 09:57:06 +00:00
rcourtman
e467523a61 feat: serve install scripts from GitHub releases instead of main branch
Scripts like install.sh and install-sensor-proxy.sh are now attached
as release assets and downloaded from releases/latest/download/ URLs.
This ensures users always get scripts compatible with their installed
version, even while development continues on main.

Changes:
- build-release.sh: copy install scripts to release directory
- create-release.yml: upload scripts as release assets
- Updated all documentation and code references to use release URLs
- Scripts reference each other via release URLs for consistency
2025-11-26 08:59:59 +00:00
rcourtman
0f0832d30f fix: propagate unified agent version and improve legacy cleanup
Issues found during scenario testing:

1. Version propagation: The hostagent and dockeragent packages were
   reporting their own Version (0.1.0-dev) instead of the unified
   agent's version. Added AgentVersion config field to pass the
   parent's version down.

2. macOS legacy cleanup: The install.sh script was missing cleanup
   for pulse-docker-agent on macOS.

3. Windows legacy cleanup: The install.ps1 script was missing cleanup
   for legacy PulseHostAgent and PulseDockerAgent services.

These fixes ensure:
- Unified agent reports consistent version across host/docker metrics
- Legacy agents are properly removed on all platforms during upgrade
- Users migrating from legacy agents get a clean transition
2025-11-25 23:39:10 +00:00
rcourtman
920f271b41 feat: improve legacy agent detection and migration UX
Add seamless migration path from legacy agents to unified agent:

- Add AgentType field to report payloads (unified vs legacy detection)
- Update server to detect legacy agents by type instead of version
- Add UI banner showing upgrade command when legacy agents are detected
- Add deprecation notice to install-host-agent.ps1
- Create install-docker-agent.sh stub that redirects to unified installer

Legacy agents (pulse-host-agent, pulse-docker-agent) now show a "Legacy"
badge in the UI with a one-click copy command to upgrade to the unified
agent.
2025-11-25 23:26:22 +00:00
rcourtman
ee35d9e5a5 feat: add auto-update support for unified agent
Implement self-update capability for the unified pulse-agent binary:

- Add internal/agentupdate package with cross-platform update logic
- Hourly version checks against /api/agent/version endpoint
- SHA256 checksum verification for downloaded binaries
- Atomic binary replacement with backup/rollback on failure
- Support for Linux, macOS, and Windows (10 platform/arch combinations)

Build and release changes:
- Dockerfile builds unified agent for all platforms
- build-release.sh includes unified agent in release artifacts
- validate-release.sh validates unified agent binaries
- Install scripts (install.sh, install.ps1) use correct URL format

Related to #727, #737
2025-11-25 23:15:03 +00:00
courtmanr@gmail.com
ebcafa1dac feat: adaptive node table layout, guest row fixes, and legacy agent detection
- Implemented adaptive layout for NodeSummaryTable with responsive columns and sticky name column.
- Fixed GuestRow background display issues.
- Added IsLegacy field to Host and DockerHost models to flag legacy agents (version < 1.0.0).
- Updated monitor to populate IsLegacy based on agent version.
2025-11-25 17:19:36 +00:00
courtmanr@gmail.com
26ebd476da WIP: Save all pending changes including frontend updates and unified agent scaffolding 2025-11-25 11:27:07 +00:00
courtmanr@gmail.com
af2caa4b81 Register unified installer routes
Exposes /api/install/install.sh and /api/install/install.ps1 for the unified agent installer.
2025-11-25 11:25:10 +00:00
courtmanr@gmail.com
3599c6592e Improve installer UX with pauses and popups on failure
Fixes #755. Adds interactive pauses and graphical popups (where available) to installer scripts when critical errors occur, ensuring troubleshooting guides are readable. Also clarifies 'build from source' instructions.
2025-11-25 11:17:37 +00:00
courtmanr@gmail.com
809676cb4f Further reduce setup script verbosity: silence token checks and consolidate permission logs 2025-11-25 10:20:17 +00:00
courtmanr@gmail.com
005a878681 Suppress 'User already exists' message in setup script 2025-11-25 10:16:08 +00:00
courtmanr@gmail.com
1f57a8c77a Improve setup script clarity: reduce verbosity and fix confusing messages 2025-11-25 10:13:20 +00:00
courtmanr@gmail.com
6887773b45 Improve setup script output by hiding irrelevant Docker/proxy info 2025-11-25 10:01:41 +00:00
courtmanr@gmail.com
75a9929afc Refactor: Parallelize PVE node polling 2025-11-25 08:38:03 +00:00
courtmanr@gmail.com
3acd29c3f3 Implement UI toggle for Hide Local Login (related to issue #750) 2025-11-25 08:14:19 +00:00
courtmanr@gmail.com
8b7ff2ad48 Relax container SSH check for temperature monitoring (ref #727) 2025-11-25 08:00:08 +00:00
courtmanr@gmail.com
ea8ae63d70 Fix: Prevent unnecessary config reloads by checking file content hash 2025-11-24 22:42:24 +00:00
courtmanr@gmail.com
5031ba481b Fix: Allow double slashes in install script URLs 2025-11-24 17:58:00 +00:00
courtmanr@gmail.com
6b84b9a2bf Add PULSE_AUTH_HIDE_LOCAL_LOGIN option to hide password form
Implements #750 - allows hiding the username/password login form when
using OIDC SSO to avoid user confusion, while maintaining security.

- Added HideLocalLogin config option (env: PULSE_AUTH_HIDE_LOCAL_LOGIN)
- Exposed hideLocalLogin in /api/security/status endpoint
- Updated Login.tsx to conditionally hide local login form
- Added escape hatch via ?show_local=true URL parameter

This approach avoids the security and upgrade issues that led to
DISABLE_AUTH being removed (see #707, #678), while solving the UX
problem of users being confused by multiple login options.
2025-11-24 17:40:43 +00:00
courtmanr@gmail.com
9caba86389 Fix #735: Allow HEAD requests for download endpoints and fix routing
- Allow HEAD requests in addition to GET for all download handlers
  (install scripts, binaries, checksums) to prevent 405 errors
- Add /uninstall-host-agent.sh to special routes in ServeHTTP
- Add test coverage for HEAD request handling
- Resolves 'method not allowed' errors during agent installation
2025-11-24 15:16:14 +00:00
courtmanr@gmail.com
07d8381858 Fix host agent registration verification issues (#746)
- Change default server listen addresses to empty string (listen on all interfaces including IPv6)
- Add short hostname matching fallback in host lookup API to handle FQDN vs short name mismatches
- Implement retry loop (30s) in both Windows and Linux/macOS installers for registration verification
- Fix lint errors: remove unnecessary fmt.Sprintf and nil checks before len()

This resolves the 'Installer could not yet confirm host registration with Pulse' warning
by addressing timing issues, hostname matching, and network connectivity.
2025-11-24 14:28:09 +00:00
courtmanr@gmail.com
08a83c9adb Add host agent checksum route and bump version to 4.32.6
- Add /download/pulse-host-agent.sha256 route to serve checksums
- Fixes 'Checksum not available' warning during host agent installation
- Bump version to 4.32.6

Related to #746
2025-11-24 07:57:17 +00:00
courtmanr@gmail.com
60721cbc22 Add mutex protection for config watcher reloads (re #748)
Introduced sync.RWMutex to protect concurrent access to configuration
fields (AuthUser, AuthPass, APITokens) that are modified by the
ConfigWatcher at runtime.

- Added global config.Mu RWMutex in internal/config/config.go
- Protected config updates in ConfigWatcher.reloadConfig() and reloadAPITokens()
- Protected config reads in CheckAuth and all API token handlers
- Protected Router.SetConfig() during full config reloads

This prevents race conditions when .env file changes trigger config
reloads while authentication handlers are reading the same fields.
2025-11-24 07:45:21 +00:00
courtmanr@gmail.com
68ffa1bc0b Feat: Add support for Raspberry Pi RP1 ADC temperature sensor (Fixes #745)
- Added 'rp1_adc' to the list of recognized CPU temperature chips
2025-11-23 22:33:16 +00:00
courtmanr@gmail.com
af7eabb56d Fix: Correct context cancellation in loop (Fixes #727)
- Replaced defer in loop with explicit cancellation to avoid resource leak
- Properly tagged issue #727
2025-11-23 22:28:28 +00:00
courtmanr@gmail.com
b34e18e839 Fix: Prevent single node auth failure from disabling global SSH temperature collection
- Removed global legacySSHDisabled flag that was triggered by any single node auth failure
- Changed disableLegacySSHOnAuthFailure to only log warnings
- Fixed potential context leak in monitor.go
- Updated tests to reflect removal of global disable logic
2025-11-23 22:24:15 +00:00
courtmanr@gmail.com
35392c1f96 Improve agent update logging and installer warnings (related to #737) 2025-11-23 22:07:37 +00:00
courtmanr@gmail.com
d0a725fde5 fix: Add dev environment fallback paths for Docker agent downloads
- Add fallback to project root scripts/ directory for install-docker-agent.sh
- Add fallback to project root bin/ directory for pulse-docker-agent binary
- Fixes 404 errors when downloading agent installer and binary in dev mode
- Production paths remain unchanged (/opt/pulse/...)
2025-11-23 16:01:40 +00:00
courtmanr@gmail.com
6e404b24eb Fix install-host-agent.sh function order, remove duplicate, and improve dev serving 2025-11-23 12:27:11 +00:00
courtmanr@gmail.com
4bf9c42dbc Fix temperature collection regression for cluster nodes. Related to #727 2025-11-23 12:13:57 +00:00
courtmanr@gmail.com
1945795d36 Fix ZFS storage reporting on TrueNAS SCALE (#718)
- Refactor collector to support mocking
- Fix ZFS detection to support 'fuse.zfs' and case-insensitivity
- Add regression tests for ZFS dataset deduplication
2025-11-22 23:53:39 +00:00
courtmanr@gmail.com
d5fdf2f471 fix: ensure proxmox nodes are displayed even if cluster endpoints are missing
Fixes #727. Previously, if temperature monitoring was enabled and a node wasn't found in ClusterEndpoints, the entire node processing was skipped. This change ensures we only skip temperature collection.
2025-11-22 23:31:30 +00:00
rcourtman
93d8c624b4 Persist PVE fallback host after portless retry 2025-11-22 17:06:15 +00:00
rcourtman
8606bb47cf Handle PVE portless fallback when default port fails 2025-11-22 17:01:16 +00:00
rcourtman
5dbaf7c596 Add OIDC CA bundle support 2025-11-22 09:44:03 +00:00
rcourtman
cd31bdece1 Add log level control to host agent
Related to #742
2025-11-22 07:48:34 +00:00
rcourtman
19a2cac355 Add log level control for docker agent
Related to #742
2025-11-22 07:43:48 +00:00
rcourtman
ee3f1e0cdb Handle standby SMART temps and capture disk identity 2025-11-22 07:35:13 +00:00
rcourtman
6ff54beb38 Handle ignored docker containers during alert reevaluation (related to #561) 2025-11-21 22:19:19 +00:00
rcourtman
4e68675e0b Add recovery notifications and grouping controls 2025-11-21 22:07:00 +00:00
rcourtman
19408fdb2b Related to #738: make pulse proxy mount migration-safe 2025-11-21 21:29:14 +00:00
rcourtman
1f2993f4a7 Guard WebSocket broadcast buffers when clients stall (Related to #734) 2025-11-21 10:14:10 +00:00
rcourtman
5a6c6091bf Honor configured PVE polling interval in scheduler 2025-11-20 22:00:56 +00:00
rcourtman
ac90859e42 Related to #727: normalize persisted Proxmox hosts 2025-11-20 19:58:05 +00:00
rcourtman
d6cbfc23ec Harden public URL detection and setup token handling 2025-11-20 19:27:14 +00:00
rcourtman
d779328878 Add Apprise test support for notifications
Related to #584
2025-11-20 17:54:20 +00:00
rcourtman
044dadcdf1 Related to #727: restore default Proxmox ports 2025-11-20 16:35:08 +00:00
rcourtman
823508dc48 Related to #712: auto-restore host agent binaries for download 2025-11-20 15:45:21 +00:00
courtmanr@gmail.com
838993cf40 Implement sensor proxy installation and configuration updates 2025-11-20 13:23:21 +00:00
courtmanr@gmail.com
6ca543beb6 refactor: simplify sensor proxy installer argument detection by validating CTID and defaulting to standalone mode. 2025-11-20 12:37:08 +00:00
courtmanr@gmail.com
8bfdf3769b Update config persistence, crypto, and dev script 2025-11-20 11:46:20 +00:00
rcourtman
f4e2cbca48 Improve token collision handling and installer subnet support 2025-11-20 09:45:36 +00:00
rcourtman
22f9750044 Respect user-provided node host URLs (Related to #724) 2025-11-20 09:40:38 +00:00
rcourtman
844c895521 WIP: Fix temperature proxy registration persistence (incomplete)
This commit contains multiple fixes for temperature proxy registration,
but the core issue remains unresolved.

## What's Fixed:
1. Added config pointer and reloadFunc to TemperatureProxyHandlers
2. Added SetConfig method to keep handler in sync with router config changes
3. Added config reload after registration to prevent monitor from overwriting
4. Fixed installer port conflict detection and duplicate YAML key issues
5. Added comprehensive debug logging throughout registration flow

## What's Still Broken:
The TemperatureProxyURL, TemperatureProxyToken, and TemperatureProxyControlToken
fields are NOT persisting to nodes.enc after SaveNodesConfig is called.

Debug logs confirm:
- HandleRegister correctly updates nodesConfig.PVEInstances[matchedIndex]
- The correct data is passed to SaveNodesConfig (verified in logs)
- SaveNodesConfig completes without errors
- Config reload executes successfully
- BUT after Pulse restart, the fields are empty when loaded from disk

The bug is in SaveNodesConfig serialization or file writing logic itself.

Related files:
- internal/api/temperature_proxy.go: Registration handler
- internal/config/persistence.go: SaveNodesConfig implementation
- internal/config/config.go: PVEInstance struct definition
2025-11-19 20:12:19 +00:00
rcourtman
c9c4a07508 test(setup): add fmt.Sprintf argument alignment validation test
Added TestPVESetupScriptArgumentAlignment to prevent future fmt.Sprintf
argument mismatch bugs in the PVE quick setup script template.

The test uses sentinel values (SENTINEL_URL, SENTINEL_HOST, deadbeef...)
to verify that critical placeholders receive the correct argument types:

✓ Repair block INSTALLER_URL uses pulseURL (not authToken)
✓ Repair --pulse-server flags use pulseURL (not authToken)
✓ Authorization headers use runtime $AUTH_TOKEN variable (not hardcoded)
✓ Token ID uses tokenName (pulse-*) (not pulseURL or authToken)

This test would have caught the bugs fixed in commits 2bb73d3c7 and
2053bc5e2, where:
- authToken appeared in --pulse-server URLs (argument shift)
- Authorization headers were hardcoded instead of using runtime variable

Recommended by Codex as a safeguard against this class of regression.
2025-11-19 14:53:44 +00:00
rcourtman
708675803f fix(setup): use runtime AUTH_TOKEN variable for Authorization headers
Changed Authorization headers in ssh-config and verify-temperature-ssh API
calls to use the runtime $AUTH_TOKEN variable instead of compile-time
hardcoded authToken.

This fixes a bug where users who override the auth token via:
- PULSE_SETUP_TOKEN environment variable
- Interactive prompt (when auth_token URL param omitted)

...would still send an empty Bearer token in the Authorization headers,
causing API calls to fail with 401 Unauthorized.

Changes:
- Line 4748: -H "Authorization: Bearer %s" → -H "Authorization: Bearer $AUTH_TOKEN"
- Line 4937: -H "Authorization: Bearer %s" → -H "Authorization: Bearer $AUTH_TOKEN"
- Removed 2 authToken arguments from fmt.Sprintf (lines 5059)

Now the script respects runtime token overrides in all code paths.

Identified by Codex during fmt.Sprintf argument alignment review.
2025-11-19 14:53:44 +00:00
rcourtman
82def8c679 fix(setup): correct fmt.Sprintf argument alignment for PVE quick setup
Fixed critical argument mismatch bug where fmt.Sprintf arguments didn't align
with template placeholders. This caused:
- authToken being passed where pulseURL expected (curl errors)
- pulseURL being passed where authToken expected (empty Authorization headers)
- tokenName misalignment (Token ID placeholder broken)

Root cause: Template has 51 %s placeholders (54 total - 3 escaped %%s), but
argument list had wrong count and ordering.

Solution: Rebuilt argument list (lines 5049-5059) with correct mapping:
- 27 pulseURL (all installer URLs, --pulse-server flags, API endpoints)
- 11 tokenName (token creation, checks, final Token ID)
- 3 authToken (AUTH_TOKEN variable + 2 Authorization headers)
- 3 serverHost (error message rerun hints)
- 1 each: serverName, time, pulseIP, storagePerms, SSH keys, minProxyReadyVersion

Verified with go vet (passes). Mapping confirmed by walking each placeholder
in template and matching to correct argument type.

Related to #TBD (user will test)
2025-11-19 14:53:44 +00:00
rcourtman
69133b45fc fix(setup): make manual repair instructions actionable
Issue: When deployment type cannot be determined, error message referenced
$PROXY_INSTALLER but deleted it immediately, making instructions unusable.

Fix: Provide complete curl commands that users can copy-paste directly:
  curl -fsSL $PULSE_URL/api/install/install-sensor-proxy.sh | bash -s -- ...

This ensures users have a working repair path even when auto-detection fails.

Identified by Codex final review.
2025-11-19 13:33:28 +00:00
rcourtman
7e505775ee fix(setup): production-ready sensor-proxy repair logic
Addresses all remaining issues from Codex final review:

Issue 1: SUMMARY_PROXY_INSTALLED unreliable (only set by install.sh)
Fix: Use PROXY_SOCKET_EXISTED_AT_START flag set at script start - works
     for all installation methods (manual, older installers, etc.)

Issue 2: CTID detection fails when container offline/renamed
Fix: Read SUMMARY_CTID from install_summary.json as fallback. Priority:
     1) Live PULSE_CTID detection
     2) SUMMARY_CTID from json file
     3) Standalone node detection

Issue 3: Failed repair disables working proxy (TEMPERATURE_ENABLED=false)
Fix: Keep TEMPERATURE_ENABLED=true in all failure paths. Comments explain:
     proxy was working before, keep it enabled even if repair fails.

This ensures turnkey repair works reliably across all deployment scenarios
without breaking existing working proxies.
2025-11-19 13:24:08 +00:00
rcourtman
6367c7265f fix(setup): comprehensive repair logic for existing sensor-proxy installations
Addresses all issues found in Codex review:

1. Prevent double-install: Check SUMMARY_PROXY_INSTALLED to distinguish
   between fresh installs (skip repair) vs existing installs (run repair)

2. Fix clustered node failures: Explicitly detect deployment type and bail
   out with clear error message if neither --ctid nor --standalone can be
   determined

3. Add health validation: Mirror main install path - verify service active,
   socket exists, and fetch SSH public key after repair

4. Capture installer output: Show full diagnostics on failure (tail -20)

5. Better error messages: Provide specific manual repair commands when
   deployment type cannot be auto-detected

This ensures the turnkey repair experience works reliably without regressing
fresh install UX.
2025-11-19 13:16:15 +00:00
rcourtman
217ed67ee6 fix(setup): properly reinstall sensor-proxy when socket exists (the real fix)
The previous attempt (ed04926) was ineffective - it only set TEMPERATURE_ENABLED=true
which was redundant (already set at line 4051) and didn't trigger the auto-install block
because that block is gated by SKIP_TEMPERATURE_PROMPT != true.

This fix actually downloads and runs install-sensor-proxy.sh when an existing
socket is detected, which:
- Refreshes control plane tokens (fixes 401 errors)
- Updates control plane URL to correct Pulse instance
- Rewrites config atomically (Phase 2 installer is idempotent)
- Maintains turnkey UX - rerunning setup script now actually works

Detected by Codex final review.
2025-11-19 13:08:54 +00:00
rcourtman
ed049263aa fix(setup): always reinstall sensor-proxy to refresh tokens and config
When sensor-proxy socket is detected, the setup script was skipping
temperature monitoring setup with 'already configured' message. This
left stale control plane URLs/tokens, breaking temperature monitoring.

Now follows Codex recommendation: treat existing installations as
upgrade/repair opportunities. The installer is idempotent (Phase 2),
so rerunning it safely refreshes tokens, updates URLs, and ensures
turnkey operation even on hosts with existing installations.

Changes:
- Remove early return when sensor-proxy socket detected
- Set TEMPERATURE_ENABLED=true to proceed with reinstall
- Update message to clarify repair/upgrade behavior
- Maintains turnkey promise: rerun setup and it just works
2025-11-19 12:52:08 +00:00
rcourtman
f4fd4924d6 fix(hostmetrics): dedupe ZFS pools for usable storage
Related to #718
2025-11-18 23:38:11 +00:00
rcourtman
bf63515ff4 feat: avoid redundant PBS snapshot polling (Related to #717) 2025-11-18 23:10:43 +00:00
rcourtman
999e4271a2 feat: surface LXC mountpoints in UI (related to #715) 2025-11-18 22:57:20 +00:00
rcourtman
3f46d35a81 feat: make PVE polling interval configurable (related to #467) 2025-11-18 21:30:04 +00:00
rcourtman
274c45799f Add direct node fallback for storage polling 2025-11-18 19:58:38 +00:00
rcourtman
2e7f693c8b Improve installer temperature proxy and backup polling 2025-11-18 18:42:33 +00:00
rcourtman
5438a25682 Don't disable storages when cluster metadata omits flags 2025-11-18 17:18:23 +00:00
rcourtman
d9c0ddaae2 Allow PBS backup poll to finish after poller returns 2025-11-18 16:55:46 +00:00
rcourtman
54c47fe2f5 Improve temperature proxy detection 2025-11-18 14:25:09 +00:00
rcourtman
b807fe8773 Allow temperature proxy to authorize standalone nodes 2025-11-18 10:30:41 +00:00
rcourtman
7fa39467c0 Fix setup-script tokens and proxy registration timing 2025-11-18 10:22:54 +00:00
rcourtman
527b7ceacc Skip inactive storages during content scans 2025-11-18 09:46:48 +00:00
rcourtman
5a570c89a5 Fix auto-registration token parsing and hostname 2025-11-18 09:10:03 +00:00
rcourtman
f7261f3080 Harden turnkey install and proxy auto-registration 2025-11-18 00:24:50 +00:00
rcourtman
a479040651 Improve temperature proxy workflow 2025-11-17 14:25:46 +00:00
rcourtman
5951a364f7 Auto-deploy proxy for standalone temp monitoring 2025-11-16 09:47:07 +00:00
rcourtman
3ffd79361d Allow socket proxy registration without URL 2025-11-15 22:14:32 +00:00
rcourtman
de5b314842 Improve temperature proxy control-plane flow 2025-11-15 21:49:51 +00:00
rcourtman
ec822575dd Improve update procedure tracking 2025-11-15 16:43:42 +00:00
rcourtman
4fbb118072 Add PULSE_LXC_CTID env override for LXC CTID detection
Modern Proxmox LXC containers (cgroup v2 + systemd) don't expose the CTID
inside the guest namespace. The auto-detection in DetectLXCCTID() works
for older LXC setups and when hostname is numeric, but fails for most
production containers where users set custom hostnames.

Changes:
- Added PULSE_LXC_CTID environment variable override in router.go:490-495
- Graceful fallback: auto-detect first, then check env var, then show placeholder
- UI already handles missing CTID by showing "pct exec <ctid>" placeholder

This provides a robust solution for thousands of users:
- Stock Proxmox LXC: Shows `pct exec <ctid>` placeholder (user substitutes manually)
- Custom hostname containers: Can set PULSE_LXC_CTID=171 in compose/systemd
- Numeric hostname containers: Auto-detected (backwards compatible)

Related: FirstRunSetup.tsx already has graceful fallback (line 336-339)
2025-11-15 13:25:07 +00:00
rcourtman
639635a84b Add Docker container name auto-detection to bootstrap UI
- Added DetectDockerContainerName() to detect container name from hostname
- Extended /api/security/status to expose dockerContainerName field
- Updated FirstRunSetup to show actual container name when detected:
  * Before: 'docker exec <container-name> cat /data/.bootstrap_token'
  * After: 'docker exec pulse cat /data/.bootstrap_token'

This reduces friction for users - no need to look up the container name.
Works when Docker container is named (--name flag), falls back to
placeholder for auto-generated container IDs.
2025-11-15 10:45:00 +00:00
rcourtman
ae0481c982 Improve bootstrap token UX with smart environment detection
- Added DetectLXCCTID() to internal/system/container.go to detect Proxmox container ID
- Extended /api/security/status to expose inContainer and lxcCtid fields
- Updated FirstRunSetup to show most relevant command based on detected environment:
  * LXC with CTID: Shows 'pct exec 171 -- cat /etc/pulse/.bootstrap_token'
  * Docker: Shows 'docker exec <container-name> cat /data/.bootstrap_token'
  * Bare metal: Shows 'cat /etc/pulse/.bootstrap_token'
- Collapsed alternative methods behind 'Show other retrieval methods' button

This addresses user feedback that showing all options was overwhelming.
Now users see the command most likely to work for their setup first,
with alternatives hidden but still accessible.
2025-11-15 10:18:59 +00:00
rcourtman
bcd8d4e0fa Fix critical cleanup implementation issues found by Codex review
**Host Detection**:
- Now detects localhost by hostname and FQDN, not just IP
- Fixes issue where nodes configured as https://hostname:8006 would skip
  localhost cleanup (API tokens, bind mounts, service removal)

**Systemd Sandbox**:
- Added /etc/pve and /etc/systemd/system to ReadWritePaths
- Allows cleanup script to modify Proxmox configs and systemd units

**Uninstaller Improvements**:
- Use UUID for transient unit names (prevents same-second collisions)
- Added --purge flag for complete removal
- Added --wait and --collect flags to capture exit code
- Now fails cleanup if uninstaller exits non-zero

**Path Migration**:
- Fixed all /usr/local references to use /opt/pulse/sensor-proxy
- Updated forced command in SSH authorized_keys
- Updated self-heal script installer path
- Updated Go backend removal helpers (supports both new and legacy paths)

These fixes address Codex findings: hostname detection, sandbox permissions,
transient unit collisions, incomplete purging, and incomplete path migration.

Related to cleanup implementation testing.
2025-11-15 00:33:41 +00:00
rcourtman
c9b4f7e88b Fix incorrect temperature data during cluster initialization
During cluster startup, nodes were temporarily using the primary cluster
endpoint for temperature collection before cluster metadata validation
completed. This caused all nodes to show the same (incorrect) temperature
values for ~4 minutes until validation finished and per-node endpoints
were established.

Example: minipc would show delly's temperature (90°C) instead of its own
(50°C) from startup until cluster validation completed.

Root cause:
- Temperature collection started immediately at startup
- Cluster endpoint validation happened asynchronously
- Code fell back to primary endpoint when ClusterEndpoints was empty
- All nodes used same endpoint, got same temperature data

Fix: Skip temperature collection for cluster nodes until:
1. ClusterEndpoints array is populated (validation complete)
2. Node's specific endpoint is found in the cluster metadata

This ensures correct temperature data from the very first collection,
maintaining data integrity during startup. When persisted config exists,
endpoints are available immediately so no delay occurs. For new clusters,
temperature collection begins once validation completes (~30s).

Preserves Pulse's correctness guarantee: users can trust metrics
immediately after restart without waiting for "warm-up" period.
2025-11-14 23:38:44 +00:00
rcourtman
387a68ef18 Allow dev builds to skip proxy version gate 2025-11-14 21:34:55 +00:00
rcourtman
7cfe998fca Make download tests use temp bin dir 2025-11-14 13:59:50 +00:00
rcourtman
20194d9bb7 Add CI build workflow and tighten proxy diagnostics 2025-11-14 13:32:29 +00:00
rcourtman
349f5627e5 monitoring: add poll watchdog to prevent worker leaks (refs #696) 2025-11-14 11:24:59 +00:00
rcourtman
6e9a62f663 Ensure Windows download finds .exe (related to #684) 2025-11-14 10:59:45 +00:00
rcourtman
70673c1fdc Improve temperature proxy diagnostics and tests 2025-11-13 22:31:53 +00:00
rcourtman
3c707a7368 Fix HTTP mode reliability: add context timeouts to SSH collection
Critical fix for intermittent HTTP endpoint hangs identified by Codex analysis.

## Root Cause
SSH collection via getTemperatureViaSSH() had no timeout, causing HTTP
handlers to block indefinitely when sensors command hung. This held node-level
mutexes and rate limit slots, creating cascading failures where subsequent
requests queued indefinitely.

## Solution
- Thread request context through to SSH execution
- Add exec.CommandContext with 15s timeout (vs 30s HTTP client timeout)
- Create execCommandWithLimitsContext() to wrap SSH commands
- Ensures handlers always release locks and respond within deadline

## Impact
- HTTP temps endpoint now responds in ~70ms consistently
- Temperature data successfully collected and displayed in Pulse
- Eliminates 'context deadline exceeded' errors
- Prevents node gate deadlocks from slow/stuck SSH sessions

Related to Codex session 019a7e99-00fc-7903-afa3-01100baf47c6
2025-11-13 19:09:50 +00:00
rcourtman
e2bd514899 Fix HTTP mode for pulse-sensor-proxy and improve installer safety
## HTTP Server Fixes
- Add source IP middleware to enforce allowed_source_subnets
- Fix missing source subnet validation for external HTTP requests
- HTTP health endpoint now respects subnet restrictions

## Installer Improvements
- Auto-configure allowed_source_subnets with Pulse server IP
- Add cluster node hostnames to allowed_nodes (not just IPs)
- Fix node validation to accept both hostnames and IPs
- Add Pulse server reachability check before installation
- Add port availability check for HTTP mode
- Add automatic rollback on service startup failure
- Add HTTP endpoint health check after installation
- Fix config backup and deduplication (prevent duplicate keys)
- Fix IPv4 validation with loopback rejection
- Improve registration retry logic with detailed errors
- Add automatic LXC bind mount cleanup on uninstall

## Temperature Collection Fixes
- Add local temperature collection for self-monitoring nodes
- Fix node identifier matching (use hostname not SSH host)
- Fix JSON double-encoding in HTTP client response

Related to #XXX (temperature monitoring fixes)
2025-11-13 18:22:36 +00:00
rcourtman
a2b8940873 Add Pulse API endpoints for temperature proxy registration
Implements REST API endpoints to enable automatic registration of
temperature proxies during sensor-proxy installation.

API endpoints:
- POST /api/temperature-proxy/register
  - Accepts: hostname, proxy_url
  - Returns: authentication token
  - Finds matching PVE instance and configures proxy URL/token
  - No authentication required (called during installation)

- DELETE /api/temperature-proxy/unregister?hostname=X
  - Removes proxy configuration from PVE instance
  - Requires admin authentication

Implementation:
- Uses config.ConfigPersistence for loading/saving nodes.enc
- Matches PVE instances by hostname in Host field or ClusterEndpoints
- Generates cryptographically secure random tokens (32 bytes, base64)
- Atomic config updates (load → modify → save)

Next step: Update install-sensor-proxy.sh to call registration API

Related to #571
2025-11-13 16:20:47 +00:00
rcourtman
22f092f941 Add HTTP mode to pulse-sensor-proxy for multi-instance temperature monitoring
This implements HTTP/HTTPS support for pulse-sensor-proxy to enable
temperature monitoring across multiple separate Proxmox instances.

Architecture changes:
- Dual-mode operation: Unix socket (local) + HTTPS (remote)
- Unix socket remains default for security/performance (no breaking change)
- HTTP mode enables temps from external PVE hosts

Backend implementation:
- Add HTTPS server with TLS + Bearer token authentication to sensor-proxy
- Add TemperatureProxyURL and TemperatureProxyToken fields to PVEInstance
- Add HTTP client (internal/tempproxy/http_client.go) for remote proxy calls
- Update temperature collector to prefer HTTP proxy when configured
- Fallback logic: HTTP proxy → Unix socket → direct SSH (if not containerized)

Configuration:
- pulse-sensor-proxy config: http_enabled, http_listen_addr, http_tls_cert/key, http_auth_token
- PVEInstance config: temperature_proxy_url, temperature_proxy_token
- Environment variables: PULSE_SENSOR_PROXY_HTTP_* for all HTTP settings

Security:
- TLS 1.2+ with modern cipher suites
- Constant-time token comparison (timing attack prevention)
- Rate limiting applied to HTTP requests (shared with socket mode)
- Audit logging for all HTTP requests

Next steps:
- Update installer script to support HTTP mode + auto-registration
- Add Pulse API endpoint for proxy registration
- Generate TLS certificates during installation
- Test multi-instance temperature collection

Related to #571 (multi-instance architecture)
2025-11-13 16:13:53 +00:00
rcourtman
56a7579c99 Add Remember Me feature with sliding session expiration (Related to #707)
Implements a "Remember Me" option that allows users to stay logged in
for 30 days instead of the default 24 hours. This addresses the pain
point of frequent re-authentication in LAN-only environments while
maintaining authentication security.

Backend changes:
- Add rememberMe field to login request handling
- Support variable session durations (24h default, 30d with Remember Me)
- Implement sliding session expiration that extends sessions on each
  authenticated request using the original duration
- Store OriginalDuration in session data for proper sliding window
- Update session cookie MaxAge to match session duration

Frontend changes:
- Add "Remember Me for 30 days" checkbox to login form
- Pass rememberMe flag in login request
- Improve UI with clear duration indication

Key features:
- Sessions extend automatically on each request (sliding window)
- Original duration preserved across session extension
- Backward compatible with existing sessions (legacy sessions work)
- Sessions persist across server restarts

This provides a better user experience for LAN deployments without
compromising security by completely disabling authentication.
2025-11-13 10:37:08 +00:00
rcourtman
1166576b21 Ensure agent ID collisions respect token boundaries (Related to #658) 2025-11-12 22:46:56 +00:00
rcourtman
bb55144637 Improve update integration diagnostics 2025-11-12 22:27:05 +00:00
rcourtman
98adb5e08e Preserve storage backups after partial failures (Related to #704) 2025-11-12 21:10:18 +00:00
rcourtman
3b079eeddb Add release dry run workflow and API update integration test 2025-11-12 21:02:52 +00:00
rcourtman
923293fbf7 Related to #692: Skip unsupported guest OS info calls 2025-11-12 19:17:09 +00:00
rcourtman
ff5de0147b Fix missing regexp import for path traversal validation 2025-11-12 16:34:16 +00:00
rcourtman
946e2e455f Security: Fix path traversal vulnerability in host-agent download endpoint
CRITICAL SECURITY FIX: The /download/pulse-host-agent endpoint was directly
concatenating user-supplied platform and arch query parameters into file paths
without validation, allowing path traversal attacks.

An attacker could request:
  /download/pulse-host-agent?platform=../../etc/passwd
to read arbitrary files from the container filesystem.

Fix: Add input validation to only allow alphanumeric characters and hyphens
in platform/arch parameters before using them in file paths.

Related: Codex security audit identified this during pre-release review
2025-11-12 16:27:11 +00:00
rcourtman
72642a2e52 Fix: Skip draft releases in update checker
Bug: Pulse was showing update notifications for draft releases because
the update checker didn't filter them out.

The GitHub API returns draft releases in the releases endpoint, and
Pulse was treating them as available updates even though they're not
published yet.

Fix:
- Added Draft field to ReleaseInfo struct
- Added draft filtering in both RC and stable channel logic
- Draft releases are now skipped with debug logging

This prevents users from seeing "Update available" notifications
when maintainers create draft releases during the release workflow.
2025-11-12 12:31:58 +00:00
rcourtman
48f8473200 Fix router to allow frontend pages without authentication
When a request for /login (or any other frontend route) comes in without
proper Accept headers (like from curl or some browsers), the server was
returning 'Authentication required' text instead of serving the frontend HTML.

This is because the router was checking authentication before serving ANY
non-API route, including frontend pages like /login, /dashboard, etc.

The fix: Frontend routes should always be served without backend auth checks.
The authentication logic runs in the frontend JavaScript after the page loads.

Backend auth should only block:
- API endpoints (/api/*)
- WebSocket connections (/ws*, /socket.io/*)
- Download endpoints (/download/*)
- Special scripts (/install-*.sh, etc.)

All other routes are frontend pages that need to be served to everyone so
the login page can load and handle auth in the browser.

This fixes the integration tests where Playwright couldn't see the login
form because the server was rejecting the /login request before serving HTML.

Related to #695 (release workflow integration tests)
2025-11-12 11:30:22 +00:00
rcourtman
db35ad7c88 Filter read-only filesystems from host agent disk metrics (related to #690)
Squashfs snap mounts on Ubuntu (and similar read-only filesystems like
erofs on Home Assistant OS) always report near-full usage and trigger
false disk alerts. The filter logic existed in Proxmox monitoring but
wasn't applied to host agents.

Changes:
- Extract read-only filesystem filter to shared pkg/fsfilters package
- Apply filter in hostmetrics.collectDisks() for host/docker agents
- Apply filter in monitor.ApplyHostReport() for backward compatibility
- Convert internal/monitoring/fs_filters.go to wrapper functions

This prevents squashfs, erofs, iso9660, cdfs, udf, cramfs, romfs, and
saturated overlay filesystems from generating alerts. Filtering happens
at both collection time (agents) and ingestion time (server) to ensure
older agents don't cause false alerts until they're updated.
2025-11-12 09:47:02 +00:00
rcourtman
d9e0505394 Fix notification manager deadlock in Stop()
Critical deadlock fix:
- Stop() was holding n.mu lock while calling queue.Stop()
- queue.Stop() waits for worker goroutines to finish
- Worker goroutines call ProcessQueuedNotification() which needs n.mu lock
- This created a classic lock-order deadlock

Fix:
- Unlock n.mu before calling queue.Stop()
- Relock after queue shutdown completes
- Workers can now finish and acquire lock as needed

This resolves 30-second test timeouts in notifications package.

Tests now complete in <1s instead of timing out at 30s.
2025-11-11 23:58:18 +00:00
rcourtman
fbeaa91fc0 Fix NVMe temperature merge test expectations
Update test expectations to match new SMART-preferred behavior:
- mergeNVMeTempsIntoDisks now prioritizes SMART temps over NVMe temps
- NVMe temps only applied to disks with Temperature == 0
- Tests were failing because disks started with non-zero temperatures
- Changed test disks to start with Temperature: 0 to simulate fresh disks

This change was introduced in commit 2a79d57f7 (Add SMART temperature
collection for physical disks) but tests weren't updated.

Fixes TestMergeNVMeTempsIntoDisks and TestMergeNVMeTempsIntoDisksClearsMissingOrInvalid.
2025-11-11 23:54:45 +00:00
rcourtman
02273e7fcb Fix monitoring test panic and goroutine leaks
Two critical fixes to prevent test timeouts:

1. Nil map panic in TestPollPVEInstanceUsesRRDMemUsedFallback:
   - Test monitor was missing nodeLastOnline map initialization
   - Panic occurred when pollPVEInstance tried to update nodeLastOnline[nodeID]
   - Caused deadlock when panic recovery tried to acquire already-held mutex
   - Added nodeLastOnline: make(map[string]time.Time) to test monitor

2. Alert manager goroutine leak in Docker tests:
   - newTestMonitor() created alert manager but never stopped it
   - Background goroutines (escalationChecker, periodicSaveAlerts) kept running
   - Added t.Cleanup(func() { m.alertManager.Stop() }) to test helper

These fixes resolve the 10+ minute test timeouts in CI workflows.

Related to workflow run 19281508603.
2025-11-11 23:52:24 +00:00
rcourtman
bbe11d1e7f Fix discovery test Prometheus metric collision
Remove t.Parallel() from tests that verify global Prometheus gauge values.
When tests run in parallel, they update the same global gauges
(discoveryScanServers, discoveryScanErrors) causing race conditions and
incorrect metric values.

Fixes test failure in workflow run 19281332332:
- TestPerformScanRecordsHistoryAndMetrics expected 2 servers, got 1

Related to release workflow preflight tests.
2025-11-11 23:34:49 +00:00
rcourtman
b41f8a2ac4 Fix backend test failures blocking release workflow
Three categories of fixes:

1. Goroutine leak causing 10-minute timeout:
   - Add defer mon.notificationMgr.Stop() in monitor_memory_test.go
   - Background goroutines from notification manager weren't being stopped

2. Database NULL column scanning errors:
   - Change LastError from string to *string in queue.go
   - Change PayloadBytes from int to *int in queue.go
   - SQL NULL values require pointer types in Go

3. SSRF protection blocking test servers:
   - Check allowlist for localhost before rejecting in notifications.go
   - Set PULSE_DATA_DIR to temp directory in tests
   - Add defer nm.Stop() calls to prevent goroutine leaks

Fixes for preflight test failures in workflow run 19280879903.
2025-11-11 23:27:03 +00:00
rcourtman
515987cc8b Fix failing backend tests in preflight checks
Fixes three test failures that were blocking release workflow:

1. TestApplyDockerReportGeneratesUniqueIDsForCollidingHosts:
   - Initialize dockerTokenBindings and dockerMetadataStore in test helper
   - These maps were nil causing panic on first access

2. TestSendGroupedAppriseHTTP & TestSendTestNotificationAppriseHTTP:
   - Configure allowlist to permit localhost (127.0.0.1) for test servers
   - SSRF protection was blocking httptest.NewServer() URLs
   - Tests need to allowlist the test server IP to bypass security checks

Related to workflow fix in 5fa78c3e3.
2025-11-11 23:02:45 +00:00
rcourtman
3e90737448 Fix guest agent OS info calls causing OpenBSD VM crashes (related to #692)
Add defensive mitigation to prevent repeated guest-get-osinfo calls that
trigger buggy behavior in QEMU guest agent 9.0.2 on OpenBSD 7.6.

The issue: OpenBSD doesn't have /etc/os-release (Linux convention), and
qemu-ga 9.0.2 appears to spawn excessive helper processes trying to read
this file whenever guest-get-osinfo is called. These helpers don't clean
up properly, eventually exhausting the process table and crashing the VM.

The fix: Track consecutive OS info failures per VM. After 3 failures,
automatically skip future guest-get-osinfo calls for that VM while
continuing to fetch other guest agent data (network interfaces, version).
This prevents triggering the buggy code path while maintaining most guest
agent functionality.

The counter resets on success, so if the guest agent is upgraded or the
issue is resolved, Pulse will automatically resume OS info collection.

Related to #692
2025-11-11 22:27:22 +00:00
rcourtman
b2d441852d Merge update service refactor with SSE and job queue
- Add job queue system to ensure only one update runs at a time
- Add Server-Sent Events (SSE) for real-time push updates
- Increase rate limit from 20/min to 60/min for update endpoints
- Add unit tests for queue and SSE functionality
- Frontend: Update modal now uses SSE with polling fallback

Eliminates: 429 rate limit errors, duplicate modals, race conditions
Related to #671
2025-11-11 10:06:16 +00:00
Claude
cc46e3c6f0
Refactor update service to eliminate polling and race conditions
This commit implements a comprehensive refactoring of the update system
to address race conditions, redundant polling, and rate limiting issues.

Backend changes:
- Add job queue system to ensure only ONE update runs at a time
- Implement Server-Sent Events (SSE) for real-time update progress
- Add rate limiting to /api/updates/status (5-second minimum per client)
- Create SSE broadcaster for push-based status updates
- Integrate job queue with update manager for atomic operations
- Add comprehensive unit tests for queue and SSE components

Frontend changes:
- Update UpdateProgressModal to use SSE as primary mechanism
- Implement automatic fallback to polling when SSE unavailable
- Maintain backward compatibility with existing update flow
- Clean up SSE connections on component unmount

API changes:
- Add new endpoint: GET /api/updates/stream (SSE)
- Enhance /api/updates/status with client-based rate limiting
- Return cached status with appropriate headers when rate limited

Benefits:
- Eliminates 429 rate limit errors during updates
- Only one update job can run at a time (prevents race conditions)
- Real-time updates via SSE reduce unnecessary polling
- Graceful degradation to polling when SSE unavailable
- Better resource utilization and reduced server load

Testing:
- All existing tests pass
- New unit tests for queue and SSE functionality
- Integration tests verify complete update flow
2025-11-11 09:33:05 +00:00
rcourtman
e894bc7b1d Fix recurring update issues (related to #671)
This commit addresses three recurring issues with the update system:

1. **Checksum mismatches (v4.27.0, v4.28.0):**
   - Root cause: Release process uploads checksums.txt first, but if artifacts
     are rebuilt after that upload, checksums become stale
   - Fix: Update RELEASE_CHECKLIST.md to REQUIRE running validate-release.sh
     before publishing (step 9, non-negotiable)
   - The validation script exists and catches these errors, but wasn't being
     enforced in the release process

2. **Duplicate error modals:**
   - Root cause: UpdateProgressModal rendered in both App.tsx
     (GlobalUpdateProgressWatcher) and UpdateBanner.tsx
   - Fix: Remove UpdateProgressModal from UpdateBanner.tsx
   - GlobalUpdateProgressWatcher automatically shows the modal when updates
     start, so the banner's modal is redundant

3. **Rate limiting too strict:**
   - Root cause: UpdateProgressModal polls /api/updates/status every 2 seconds
     (30 req/min), but rate limit was 20/min
   - Fix: Increase UpdateEndpoints rate limit from 20/min to 60/min
   - Allows modal to poll without hitting rate limits during updates

These were all manual process errors and configuration issues, not code bugs.
The validation script enforcement prevents future checksum mismatches.
2025-11-11 09:09:30 +00:00
rcourtman
accecdb50b Make api_tokens.json authoritative source for API tokens (fixes #685)
This is the proper architectural fix for #685. The previous commit was a
bandaid that prevented unnecessary .env writes. This commit addresses the
root cause: dual-source-of-truth for API tokens (.env vs api_tokens.json).

Changes:

1. Startup Migration (config.go:896-951):
   - When loading config, if API_TOKEN/API_TOKENS exist in .env but not in
     api_tokens.json, automatically migrate them
   - Migrated tokens are named "Migrated from .env (prefix)" for clarity
   - Logs a deprecation warning: API_TOKEN/API_TOKENS in .env are deprecated
   - Leaves .env untouched (safe for existing deployments)

2. Config Watcher Changes (watcher.go:338-424):
   - Only load tokens from .env if api_tokens.json is EMPTY
   - Once api_tokens.json has records, it becomes the authoritative source
   - .env changes no longer trigger token overwrites when api_tokens.json exists
   - Logs debug message when ignoring env tokens

Result:
- Existing deployments: env tokens automatically migrated to api_tokens.json
- UI-created tokens: never overwritten by .env changes
- Dark mode toggle: no longer triggers token reload from .env
- Backward compatible: fresh installs with API_TOKEN in .env still work
- Migration path: users can safely keep API_TOKEN in .env, it will be ignored

Future improvement: Add UI warning when API_TOKEN/API_TOKENS still present
in .env, prompting users to rotate tokens via the UI.
2025-11-11 00:17:40 +00:00
rcourtman
5d99fc2f2d Fix dark mode toggle wiping API tokens (related to #685)
Root cause: SaveSystemSettings calls updateEnvFile which rewrites .env on
any setting change, triggering the config watcher. The watcher sees API_TOKEN
in .env and replaces all UI-created tokens with "Environment token" records,
wiping out host-agent scoped tokens.

Fix: updateEnvFile now compares the new content with existing content and
skips the write if nothing changed. Since dark mode (and other UI settings)
are stored in system.json, not .env, toggling theme no longer triggers
unnecessary .env rewrites.

This prevents the config watcher from being triggered unnecessarily and
preserves UI-created API tokens when changing cosmetic settings.

Future improvement: Deprecate API_TOKEN/API_TOKENS from .env entirely and
make api_tokens.json the single source of truth (requires migration logic).
2025-11-11 00:11:41 +00:00
rcourtman
df185985eb Fix bootstrap token path display for Docker deployments (related to #680)
The first-run setup UI was displaying incorrect bootstrap token paths for
Docker deployments. It showed `/etc/pulse/.bootstrap_token` regardless of
deployment type, but Docker containers use `/data/.bootstrap_token` by
default (via PULSE_DATA_DIR env var).

Changes:
- Extended `/api/security/status` endpoint to include `bootstrapTokenPath`
  and `isDocker` fields when a bootstrap token is active
- Updated FirstRunSetup component to fetch and display the correct path
  dynamically based on actual deployment configuration
- For Docker deployments, UI now shows both `docker exec` command and
  in-container command
- Falls back to showing both standard and Docker paths if API data
  unavailable (backward compatibility)

This fix ensures users always see the correct command for their specific
deployment, including custom PULSE_DATA_DIR configurations.
2025-11-09 23:41:55 +00:00
rcourtman
a82a345cd6 Improve table column widths and sparkline visibility 2025-11-09 23:36:52 +00:00
rcourtman
425ea00ba2 Fix upgrade path when DISABLE_AUTH detected but no credentials exist (fixes #678)
Users upgrading from v4.25 (where DISABLE_AUTH actually disabled auth) to
v4.27.1 (where DISABLE_AUTH is ignored but triggers a deprecation warning)
were stuck in a catch-22:

- They had no credentials (old version had auth disabled)
- DISABLE_AUTH detection incorrectly required authentication
- Setup wizard returned 401, preventing first credential creation
- Could not complete setup to create credentials and remove flag

Root cause: When DISABLE_AUTH was detected, the code set forceRequested=true
which triggered the authentication requirement even when authConfigured=false.

Fix: Only require authentication when credentials actually exist. When no
auth is configured, allow the bootstrap token flow regardless of whether
DISABLE_AUTH is detected.

This lets users upgrade from legacy DISABLE_AUTH deployments by using the
bootstrap token to create their first credentials, then removing the flag.
2025-11-09 20:33:58 +00:00
rcourtman
62a9f40cc7 Fix diagnostics incorrectly warning about /run mount in Docker (related to #600)
The diagnostic code was warning ALL deployments using /run/pulse-sensor-proxy
socket path to "remove and re-add" their configuration to use /mnt/pulse-proxy
instead. This was incorrect for Docker deployments where /run is the correct
and documented mount path (see docker-compose.yml line 15).

The warning was only meant for LXC containers where the managed mount at
/mnt/pulse-proxy is preferred over a legacy hand-crafted /run mount.

Fix: Only show the warning in non-Docker environments (check PULSE_DOCKER env).
Docker deployments correctly use /run/pulse-sensor-proxy per compose file.

Impact: Docker users were seeing confusing diagnostic warnings telling them
to reconfigure a correct setup.
2025-11-09 16:49:49 +00:00
rcourtman
bb7ca93c18 feat: Add mdadm RAID monitoring support for host agents
Implements comprehensive mdadm RAID array monitoring for Linux hosts
via pulse-host-agent. Arrays are automatically detected and monitored
with real-time status updates, rebuild progress tracking, and automatic
alerting for degraded or failed arrays.

Key changes:

**Backend:**
- Add mdadm package for parsing mdadm --detail output
- Extend host agent report structure with RAID array data
- Integrate mdadm collection into host agent (Linux-only, best-effort)
- Add RAID array processing in monitoring system
- Implement automatic alerting:
  - Critical alerts for degraded arrays or arrays with failed devices
  - Warning alerts for rebuilding/resyncing arrays with progress tracking
  - Auto-clear alerts when arrays return to healthy state

**Frontend:**
- Add TypeScript types for RAID arrays and devices
- Display RAID arrays in host details drawer with:
  - Array status (clean/degraded/recovering) with color-coded indicators
  - Device counts (active/total/failed/spare)
  - Rebuild progress percentage and speed when applicable
  - Green for healthy, amber for rebuilding, red for degraded

**Documentation:**
- Document mdadm monitoring feature in HOST_AGENT.md
- Explain requirements (Linux, mdadm installed, root access)
- Clarify scope (software RAID only, hardware RAID not supported)

**Testing:**
- Add comprehensive tests for mdadm output parsing
- Test parsing of healthy, degraded, and rebuilding arrays
- Verify proper extraction of device states and rebuild progress

All builds pass successfully. RAID monitoring is automatic and best-effort
- if mdadm is not installed or no arrays exist, host agent continues
reporting other metrics normally.

Related to #676
2025-11-09 16:36:33 +00:00
rcourtman
4834dea05b Add support for linux-386 and linux-armv6 architectures (related to #674)
Adds build support for 32-bit x86 (i386/i686) and ARMv6 (older Raspberry Pi models) architectures across all agents and install scripts.

Changes:
- Add linux-386 and linux-armv6 to build-release.sh builds array
- Update Dockerfile to build docker-agent, host-agent, and sensor-proxy for new architectures
- Update all install scripts to detect and handle i386/i686 and armv6l architectures
- Add architecture normalization in router download endpoints
- Update update manager architecture mapping
- Update validate-release.sh to expect 24 binaries (was 18)

This enables Pulse agents to run on older/legacy hardware including 32-bit x86 systems and Raspberry Pi Zero/Zero W devices.
2025-11-09 08:35:24 +00:00
rcourtman
1b221cca71 feat: Add configurable allowlist for webhook private IP targets (addresses #673)
Allow homelab users to send webhooks to internal services while maintaining security defaults.

Changes:
- Add webhookAllowedPrivateCIDRs field to SystemSettings (persistent config)
- Implement CIDR parsing and validation in NotificationManager
- Convert ValidateWebhookURL to instance method to access allowlist
- Add UI controls in System Settings for configuring trusted CIDR ranges
- Maintain strict security by default (block all private IPs)
- Keep localhost, link-local, and cloud metadata services blocked regardless of allowlist
- Re-validate on both config save and webhook delivery (DNS rebinding protection)
- Add comprehensive tests for CIDR parsing and IP matching

Backend:
- UpdateAllowedPrivateCIDRs() parses comma-separated CIDRs with validation
- Support for bare IPs (auto-converts to /32 or /128)
- Thread-safe allowlist updates with RWMutex
- Logging when allowlist is updated or used
- Validation errors prevent invalid CIDRs from being saved

Frontend:
- New "Webhook Security" section in System Settings
- Input field with examples and helpful placeholder text
- Real-time unsaved changes tracking
- Loads and saves allowlist via system settings API

Security:
- Default behavior unchanged (all private IPs blocked)
- Explicit opt-in required via configuration
- Localhost (127/8) always blocked
- Link-local (169.254/16) always blocked
- Cloud metadata services always blocked
- DNS resolution checked at both save and send time

Testing:
- Tests for CIDR parsing (valid/invalid inputs)
- Tests for IP allowlist matching
- Tests for bare IP address handling
- Tests for security boundaries (localhost, link-local remain blocked)

Related to #673

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-09 08:31:12 +00:00
rcourtman
6bb53eaadb Surface update errors to UI for better user feedback (related to #671)
User ZaDarkSide reported that when updates fail, the UI shows a loading
spinner indefinitely with no feedback about what went wrong. Users had to
check backend logs to understand failures like "checksum verification failed".

The infrastructure was already in place:
- UpdateStatus struct had an Error field
- Frontend already renders error details when present
- But updateStatus() never populated the Error field

Changes:
- Modified updateStatus() to accept optional error parameter
- Added sanitizeError() to cap error message length (500 chars max)
- Updated all error cases in ApplyUpdate() to pass error details:
  - Temp directory creation failures
  - Download failures
  - Checksum verification failures (most common user complaint)
  - Extraction failures
  - Backup creation failures
  - Apply update failures
- Also updated CheckForUpdates() error cases

Now when updates fail, users immediately see the error message in the UI's
red error panel instead of being stuck on a loading spinner.

Security: Errors are only shown to authenticated admin users with update
permissions. Error messages are capped at 500 chars to prevent extremely
long output. Current error messages don't contain sensitive data (mainly
HTTP status codes, file paths, checksum mismatches).
2025-11-09 08:23:04 +00:00
rcourtman
6bf32f98d6 Fix storage/disk/backup disappearing for clusters with VerifySSL enabled
Related to #670, #657

The fix in v4.26.5 (commit 59a97f2e3) attempted to resolve storage disappearing
by preferring hostnames over IPs when TLS hostname verification is required
(VerifySSL=true and no fingerprint). However, that fix was ineffective because
the cluster discovery code was populating BOTH the Host and IP fields with the
IP address.

**Root Cause:**
In internal/api/config_handlers.go, the detectPVECluster function was setting:
- endpoint.Host = schemePrefix + clusterNode.IP (when IP was available)
- endpoint.IP = clusterNode.IP

This meant both fields contained the same IP address. When the monitoring code
tried to prefer endpoint.Host for TLS validation (internal/monitoring/monitor.go:
361-368), it was still getting an IP, causing certificate validation to fail
with "certificate is valid for pve01.example.com, not 10.0.0.44".

**Solution:**
Separate the Host and IP fields properly during cluster discovery:
- endpoint.Host = hostname (e.g., "https://pve01:8006") for TLS validation
- endpoint.IP = IP address (e.g., "10.0.0.44") for DNS-free connections

The existing logic in clusterEndpointEffectiveURL() can now correctly choose
between them based on TLS requirements.

**Impact:**
Users with VerifySSL=true who upgraded to v4.26.1-v4.26.5 and lost storage
visibility should now see storage, VM disks, and backups again after this fix.
2025-11-08 23:07:49 +00:00
rcourtman
8f05fc0a57 Improve backup-age alerts to show VM/CT names in multi-cluster setups (related to #668)
This change fixes backup-age alert notifications to display VM/CT names
instead of just "VMID XXX" in multi-cluster environments where backups
are stored on PBS.

Changes:
- Store all guests per VMID (not just first match) to handle VMID collisions across clusters
- Persist last-known guest names/types in metadata store for deleted VMs
- Enrich backup correlation with persisted metadata when live inventory is empty
- Update CheckBackups to handle multiple VMID matches intelligently

The fix addresses two scenarios:
1. Multiple PVE clusters with same VMID backing up to one PBS
2. VMs deleted from Proxmox but backups still exist on PBS

Backup-age alerts will now show proper VM/CT names when:
- A unique guest exists with that VMID (live or persisted)
- Multiple guests share a VMID (uses first match, consistent with current behavior)

When truly ambiguous (multiple live VMs, same VMID, no way to determine origin),
the alert gracefully falls back to showing "VMID XXX".
2025-11-08 18:24:04 +00:00
rcourtman
5ec2947d86 Fix Pushover webhook custom field overrides (related to #665)
The Pushover webhook template now honors user-defined custom fields
for sound, priority, and device. Previously, these fields were
hardcoded based on alert level, ignoring any custom values set by
users in the UI.

Changes:
- sound: Uses CustomFields.sound if provided, otherwise falls back to
  level-based default (critical=siren, warning=tugboat, else=pushover)
- priority: Uses CustomFields.priority if provided, otherwise falls back
  to level-based default (critical=1, warning=0, else=-1)
- device: Uses CustomFields.device if provided, otherwise falls back to
  ResourceName

Updated setup instructions to document optional custom fields for sound,
priority, and device configuration.

This allows users to customize Pushover notification behavior without
editing webhook templates, consistent with Pulse's maintainability goals.
2025-11-08 10:32:27 +00:00
rcourtman
8cea433443 Fix Docker host custom display name not persisting in UI (related to #662)
The custom display name feature added in cd627f33c had a critical bug where
the backend successfully stored custom names but the frontend never received
them, making the feature appear non-functional.

Root cause:
- DockerHost.CustomDisplayName was stored in backend state (models.go:201)
- SetDockerHostCustomDisplayName() correctly updated the field
- BUT DockerHostFrontend struct was missing customDisplayName field
- AND ToFrontend() converter didn't copy CustomDisplayName
- Result: WebSocket state broadcasts stripped out the custom name

When users edited a Docker host display name:
- API returned 200 OK ✓
- Success notification appeared ✓
- Edit state cleared ✓
- But subsequent state broadcasts lacked customDisplayName ✗
- UI continued showing original name ✗

Fix:
- Add CustomDisplayName field to DockerHostFrontend (models_frontend.go:105)
- Copy d.CustomDisplayName in ToFrontend() converter (converters.go:204)
- Now custom display names properly propagate to frontend via WebSocket

The feature now works as originally intended - custom names persist across
agent reconnections and display correctly in the UI.
2025-11-08 10:28:20 +00:00
rcourtman
1a3abf7f3f Fix pulse-host-agent temperature collection on all Linux distros (related to #661)
The temperature collection in pulse-host-agent was broken on all Linux
distributions due to an incorrect platform check.

Root cause:
- collectTemperatures() checked `if a.platform != "linux"` at agent.go:316
- normalisePlatform() returns the raw distro name from gopsutil (debian, ubuntu, pve)
- This caused temperature collection to be skipped on ALL Linux hosts

Fix:
- Changed check to `if runtime.GOOS != "linux"` which correctly identifies Linux
- runtime.GOOS returns "linux" regardless of distribution

Also fixed documentation typo:
- Changed "Servers tab" to "Hosts tab" in HOST_AGENT.md and TEMPERATURE_MONITORING.md
- Reported by user in issue #661 comments

Testing:
- Verified build succeeds
- Confirmed runtime.GOOS returns "linux" on Linux systems

Related to #661
2025-11-08 10:25:01 +00:00
rcourtman
270840801a Fix setup script fmt.Sprintf argument misalignment (related to #663)
The setup script template had 44 %s placeholders, but the fmt.Sprintf call
arguments were out of order starting at position 15. This caused the Pulse
URL to be inserted where the token name should be, resulting in errors like:

  Token ID: pulse-monitor@pam!http://192.168.0.44:7655

Instead of the correct format:

  Token ID: pulse-monitor@pam!pulse-192-168-0-44-1762545916

Changes:
- Escaped %s in printf helper (line 3949) so it doesn't consume arguments
- Reordered fmt.Sprintf arguments (lines 4727-4732) to match template order
- Removed 2 extra pulseURL arguments that were causing the shift

This fix ensures all 44 placeholders receive the correct values in order.
2025-11-08 07:52:19 +00:00
rcourtman
3ad35976b2 Clarify Docker agent cycling troubleshooting for cloned VMs/LXCs (related to #648)
Enhanced the "Docker hosts cycling" troubleshooting entry to explicitly
call out VM/LXC cloning as a cause of identical agent IDs. Added specific
remediation steps for regenerating machine IDs on cloned systems.

This addresses the resolution path discovered in discussion #648 where a
user cloned a Proxmox LXC and encountered cycling behavior even with
separate API tokens because the agent IDs were duplicated.
2025-11-07 22:59:19 +00:00
rcourtman
16c29463f9 Fix Windows host agent installer reliability (related to #654)
The download endpoint had a dangerous fallback that silently served the
wrong binary when the requested platform/arch combination was missing.
If a Docker image shipped without Windows binaries, the installer would
receive a Linux ELF instead of a Windows PE, causing ERROR_BAD_EXE_FORMAT.

Changes:
- Download handler now operates in strict mode when platform+arch are
  specified, returning 404 instead of serving mismatched binaries
- PowerShell installer validates PE header (MZ signature)
- PowerShell installer verifies PE machine type matches requested arch
- PowerShell installer fetches and verifies SHA256 checksums
- PowerShell installer shows diagnostic info: OS arch, download URL,
  file size for better troubleshooting

This prevents silent failures and provides clear error messages when
binaries are missing or corrupted.
2025-11-07 22:55:03 +00:00
rcourtman
2b7492ac59 feat: Add temperature collection to pulse-host-agent (related to #661)
Implements temperature monitoring in pulse-host-agent to support Docker-in-VM
deployments where the sensor proxy socket cannot cross VM boundaries.

Changes:
- Create internal/sensors package with local collection and parsing
- Add temperature collection to host agent (Linux only, best-effort)
- Support CPU package/core, NVMe, and GPU temperature sensors
- Update TEMPERATURE_MONITORING.md with Docker-in-VM setup instructions
- Update HOST_AGENT.md to document temperature feature

The host agent now automatically collects temperature data on Linux systems
with lm-sensors installed. This provides an alternative path for temperature
monitoring when running Pulse in a VM, avoiding the unix socket limitation.

Temperature collection is best-effort and fails gracefully if lm-sensors is
not available, ensuring other metrics continue to be reported.

Related to #661
2025-11-07 22:54:40 +00:00
rcourtman
cb9d8d1ab1 Fix config backup/restore by enforcing 12-char minimum password (related to #646)
Users with 8-11 character passwords could not export/restore config backups
because the export encryption requires 12+ character passphrases for security,
but the password creation UI only enforced an 8-character minimum.

This created a confusing UX where users with short passwords saw validation
errors when trying to export backups, with the only solution being to use a
custom passphrase or change their password.

Root cause:
- FirstRunSetup and ChangePasswordModal allowed 8+ char passwords
- Config export/import requires 12+ char passphrases (backend validation)
- The v4.26.4 fix added frontend validation that showed the mismatch
- Users hit client-side validation before request was sent (no backend logs)

This fix raises the minimum password length to 12 characters everywhere:
- internal/auth/password.go: MinPasswordLength 8 → 12
- FirstRunSetup.tsx: validation and placeholder updated
- ChangePasswordModal.tsx: validation, minLength, and help text updated
- QuickSecuritySetup.tsx: validation and label updated

Impact:
- New users must create 12+ character passwords
- Existing users with <12 char passwords are unaffected (can't detect from hash)
- Those users will see the existing helpful error directing them to use custom
  passphrase for backups
- "Use your login password" option now works for all future passwords

This aligns password requirements across the system and eliminates the
confusing mismatch between login credentials and backup encryption requirements.

Related to #646 where user confirmed backups still failed in v4.26.5
2025-11-07 22:51:55 +00:00
rcourtman
b2e65f7b3e feat(security): Add SSH output limits and improve host key management
Addresses two security vulnerabilities:

1. SSH Output Size Limits:
   - Prevents memory exhaustion from malicious remote nodes
   - Configurable max_ssh_output_bytes (default 1MB)
   - Stream with io.LimitReader to cap output size
   - New metric: pulse_proxy_ssh_output_oversized_total{node}
   - WARN logging for oversized outputs

2. Improved Host Key Management:
   - Seed host keys from Proxmox cluster store (/etc/pve/priv/known_hosts)
   - Falls back to ssh-keyscan only if Proxmox unavailable (with WARN)
   - Fingerprint change detection with ERROR logging
   - require_proxmox_hostkeys option for strict mode
   - New metric: pulse_proxy_hostkey_changes_total{node}
   - Reduces MITM attack surface significantly

Known hosts manager now normalizes entries, reuses existing fingerprints,
and raises typed HostKeyChangeError when fingerprints differ.

Related to security audit 2025-11-07.

Co-authored-by: Codex <codex@openai.com>
2025-11-07 17:09:02 +00:00
rcourtman
59a97f2e3e Fix storage disappearing after upgrade by preserving TLS validation
Fixes #657

Between v4.25.0 and v4.26.4, commit 72865ff62 changed cluster endpoint
resolution to prefer IP addresses over hostnames to reduce DNS lookups
(refs #620). However, this caused TLS certificate validation to fail for
installations with VerifySSL=true, because Proxmox certificates typically
contain hostnames (e.g., pve01.example.com), not IP addresses.

When all cluster endpoints failed TLS validation during the initial health
check, the ClusterClient marked all nodes as unhealthy. Subsequent calls
to GetAllStorage() would fail with "no healthy nodes available in cluster",
causing storage data to disappear from the UI despite the cluster being
fully operational.

**Root Cause:**
The IP-first approach breaks TLS hostname verification when:
- VerifySSL is enabled (common for production environments)
- Certificates are issued with hostnames, not IPs (standard practice)
- Result: x509 certificate validation fails (e.g., "certificate is valid
  for pve01.example.com, not 10.0.0.44")

**Solution:**
Conditionally prefer hostnames vs IPs based on TLS validation requirements:

1. When TLS hostname verification is required (VerifySSL=true AND no
   fingerprint override), prefer hostname to ensure certificate CN/SAN
   validation succeeds.

2. When TLS verification is bypassed (VerifySSL=false OR fingerprint
   provided), prefer IP to reduce DNS lookups.

This approach:
- Fixes the regression for users with VerifySSL enabled
- Preserves the DNS optimization for self-signed/fingerprint configs
- Maintains backwards compatibility with v4.25.0 behavior
- Does not compromise TLS security

**Testing:**
Users reported that rolling back to v4.25.0 fixed their storage visibility.
This fix should restore storage for v4.26.4+ while maintaining the DNS
optimization for appropriate scenarios.
2025-11-07 15:36:52 +00:00
rcourtman
19091d47c9 Enforce Docker agent API token uniqueness (related to #658)
Problem: Multiple Docker agents can share the same API token, which causes
serious operational and security issues:

1. Host identity collision - agents overwrite each other in state (the bug
   fixed in aa0aa7d4f only addressed the symptom, not the root cause)
2. Security/audit gap - can't attribute actions to specific agents
3. User confusion - easy mistake that causes subtle, hard-to-debug issues
4. State corruption - race conditions on startup and racey metric updates

Root cause: The system treats API tokens as the agent's identity credential,
but never enforced uniqueness. This allowed users to accidentally (or
intentionally) reuse tokens across multiple agents, breaking the 1:1
token-to-agent relationship that the architecture assumes.

Solution: Enforce token uniqueness at the agent report ingestion point.

Implementation:
- Add dockerTokenBindings map[tokenID]agentID to Monitor state
- In ApplyDockerReport, check if token is already bound to a different agent
- On first report from a token, bind it to that agent's ID
- On subsequent reports, verify the binding matches
- Reject mismatches with clear error naming the conflicting host
- Unbind tokens when hosts are removed (allows token reuse after cleanup)

Error message example:
  "API token (pk_abc…xyz) is already in use by agent 'agent-123'
  (host: docker-host-1). Each Docker agent must use a unique API token.
  Generate a new token for this agent"

Why fail-fast instead of phased rollout:
- Shared tokens are architecturally wrong and cannot work correctly
- The system cannot safely multiplex state for duplicate identities
- A clear, immediate error is better UX than silent corruption
- Users would need to generate per-agent tokens eventually anyway

Why in-memory instead of persisted:
- Aligns with Pulse's existing state model (JSON config + in-memory state)
- Bindings naturally rebuild as agents report in after restart
- No schema migration or additional persistence complexity needed
- Sufficient for correctness since overwrite can't happen until both
  agents report, at which point the binding exists and rejects duplicates

Migration path for existing users with shared tokens:
- Generate new unique token for each agent
- Update agent configuration with new token
- Restart agents one at a time

This enforces the token-as-identity invariant and prevents users from
creating unsupportable configurations.
2025-11-07 15:19:42 +00:00
rcourtman
48fabdd827 Improve Docker temperature monitoring documentation for clarity (related to #600)
Updated the Quick Start for Docker section in TEMPERATURE_MONITORING.md to be
more user-friendly and address common setup issues:

- Added clear explanation of why the proxy is needed (containers can't access hardware)
- Provided concrete IP example instead of placeholder
- Showed full docker-compose.yml context with proper YAML structure
- Added sudo to commands where needed
- Updated docker-compose commands to v2 syntax with note about v1
- Expanded verification steps with clearer success indicators
- Added reminder to check container name in verification commands

These improvements should help users who encounter blank temperature displays
due to missing proxy installation or bind mount configuration.
2025-11-07 15:09:42 +00:00
rcourtman
7ee252bd84 Fix Docker host display bug when multiple agents share API tokens (related to #658)
Root cause: findMatchingDockerHost() was matching hosts by token ID alone,
causing multiple Docker agents using the same API token to overwrite each
other in state. This resulted in only N visible hosts (where N = number of
unique tokens) instead of all M agents, with hosts "rotating" as each agent
reported every 10 seconds.

Example: 4 agents using 2 tokens would show only 2 hosts, rotating between
agents 1↔2 (token A) and agents 3↔4 (token B).

Fix: Remove token-only matching from findMatchingDockerHost(). Hosts should
only match by:
1. Agent ID (unique per agent)
2. Machine ID + hostname combination (with optional token validation)
3. Machine ID or hostname alone (only for tokenless agents)

This allows multiple agents to share the same API token without colliding.

Additional fix: UpsertDockerHost() now preserves Hidden, PendingUninstall,
and Command fields from existing hosts, preventing these flags from being
reset to defaults on every agent report.
2025-11-07 13:46:35 +00:00
rcourtman
2a79d57f73 Add SMART temperature collection for physical disks (related to #652)
Extends temperature monitoring to collect SMART temps for SATA/SAS disks,
addressing issue #652 where physical disk temperatures showed as empty.

Architecture:
- Deploys pulse-sensor-wrapper.sh as SSH forced command on Proxmox nodes
- Wrapper collects both CPU/GPU temps (sensors -j) and disk temps (smartctl)
- Implements 30-min cache with background refresh to avoid performance impact
- Uses smartctl -n standby,after to skip sleeping drives without waking them
- Returns unified JSON: {sensors: {...}, smart: [...]}

Backend changes:
- Add DiskTemp model with device, serial, WWN, temperature, lastUpdated
- Extend Temperature model with SMART []DiskTemp field and HasSMART flag
- Add WWN field to PhysicalDisk for reliable disk matching
- Update parseSensorsJSON to handle both legacy and new wrapper formats
- Rewrite mergeNVMeTempsIntoDisks to match SMART temps by WWN → serial → devpath
- Preserve legacy NVMe temperature support for backward compatibility

Performance considerations:
- SMART data cached for 30 minutes per node to avoid excessive smartctl calls
- Background refresh prevents blocking temperature requests
- Respects drive standby state to avoid spinning up idle arrays
- Staggered disk scanning with 0.1s delay to avoid saturating SATA controllers

Install script:
- Deploys wrapper to /usr/local/bin/pulse-sensor-wrapper.sh
- Updates SSH forced command from "sensors -j" to wrapper script
- Backward compatible - falls back to direct sensors output if wrapper missing

Testing note:
- Requires real hardware with smartmontools installed for full functionality
- Empty smart array returned gracefully when smartctl unavailable
- Legacy sensor-only nodes continue working without changes
2025-11-07 11:46:57 +00:00
rcourtman
94b07a892e Fix test failures from API signature changes
Fixed two test failures identified by go vet:

1. SSH knownhosts manager tests
   - Updated keyscanFunc signatures from (ctx, host, timeout) to (ctx, host, port, timeout)
   - Affected 4 test functions in manager_test.go
   - Matches recent API change adding port parameter for flexibility

2. Monitor temperature toggle test
   - Removed obsolete test file monitor_temperature_toggle_test.go
   - Test was checking internal implementation details that have changed
   - Enable/DisableTemperatureMonitoring() now only log (interface compatibility)
   - Temperature collection is managed differently in current architecture

Impact:
- All tests now compile successfully
- Removes obsolete test that no longer reflects current behavior
- Updates remaining tests to match current API signatures
2025-11-07 10:43:06 +00:00
rcourtman
d30d76bb92 Fix P1: Add shutdown mechanism to WebSocket Hub
Fixed goroutine leaks in WebSocket hub from missing shutdown mechanism:

Problem:
1. Hub.Run() has infinite loop with no exit condition
2. runBroadcastSequencer() reads from channel forever
3. No way to cleanly shutdown hub during restarts or tests

Solution:
- Added stopChan chan struct{} field to Hub
- Initialize stopChan in NewHub()
- Added Stop() method that closes stopChan
- Modified Run() main loop to select on stopChan
  - On shutdown: close all client connections and return
- Modified runBroadcastSequencer() from 'for range' to select
  - Changed from: for msg := range h.broadcastSeq
  - Changed to: for { select { case msg := <-h.broadcastSeq: ... case <-h.stopChan: ... }}
  - On shutdown: stop coalesce timer and return

Shutdown sequence:
1. Call hub.Stop() to close stopChan
2. Both Run() and runBroadcastSequencer() exit their loops
3. All client send channels are closed
4. Clients map is cleared
5. Pending coalesce timer is stopped

Impact:
- Enables graceful shutdown during service restarts
- Prevents goroutine leaks in tests
- Allows proper cleanup of WebSocket connections
- No more orphaned broadcast sequencer goroutines
2025-11-07 10:20:26 +00:00
rcourtman
e30757720a Fix P1: Resource leaks in Recovery Tokens, Rate Limiter, and OIDC Service
Fixed three P1 goroutine/memory leaks that prevent proper resource cleanup:

1. Recovery Tokens goroutine leak
   - Cleanup routine runs forever without stop mechanism
   - Added stopCleanup channel and Stop() method
   - Cleanup loop now uses select with stopCleanup case

2. Rate Limiter goroutine leak
   - Cleanup routine runs forever without stop mechanism
   - Added stopCleanup channel and Stop() method
   - Changed from 'for range ticker.C' to select with stopCleanup case

3. OIDC Service memory leak (DoS vector)
   - Abandoned OIDC flows never cleaned up
   - State entries accumulate unboundedly
   - Added cleanup routine with 5-minute ticker
   - Periodically removes expired state entries (10min TTL)
   - Added Stop() method for proper shutdown

All three follow consistent pattern:
- Add stopCleanup chan struct{} field
- Initialize in constructor
- Use select with ticker and stopCleanup cases
- Close channel in Stop() method to signal goroutine exit

Impact:
- Prevents goroutine leaks during service restarts/reloads
- Prevents memory exhaustion from abandoned OIDC login attempts
- Enables proper cleanup in tests and graceful shutdown
2025-11-07 10:18:44 +00:00
rcourtman
1bf9cfea88 Fix critical P0 security and crash issues in API/WebSocket layer
This commit addresses 5 critical P0 bugs that cause security vulnerabilities, crashes, and data corruption:

**P0-1: Recovery Tokens Replay Attack Vulnerability** (recovery_tokens.go:153-159)
- **SECURITY CRITICAL**: Single-use recovery tokens could be replayed
- **Problem**: Lock upgrade race - two concurrent requests both pass initial Used check
  1. Both acquire RLock, see token.Used = false
  2. Both release RLock
  3. Both acquire Lock and mark token.Used = true
  4. Both return true - TOKEN REUSED
- **Impact**: Attacker with intercepted token can use it multiple times
- **Fix**: Re-check token.Used after acquiring write lock (TOCTOU prevention)

**P0-2: WebSocket Hub Concurrent Map Panic** (hub.go:345-347, 376-378)
- **Problem**: Initial state goroutine reads h.clients map without lock
  - Line 345: `if _, ok := h.clients[client]` (NO LOCK)
  - Main loop writes to h.clients with lock (line 326, 394)
- **Impact**: "fatal error: concurrent map read and write" crashes hub
- **Fix**: Acquire RLock before all client map reads in goroutine

**P0-3: WebSocket Send on Closed Channel Panic** (hub.go:348, 380)
- **Problem**: Check client exists, then send - channel can close between
- **Impact**: "send on closed channel" panic crashes hub
- **Fix**: Hold RLock during both check and send (defensive select already present)

**P0-4: CSRF Store Shutdown Data Corruption** (csrf_store.go:189-196)
- **Problem**: Stop() calls save() after signaling worker. Both hold only RLock
  - Worker's final save writes to csrf_tokens.json.tmp
  - Stop()'s save writes to same file concurrently
- **Impact**: Corrupted/truncated csrf_tokens.json on shutdown
- **Fix**: Added saveMu mutex to serialize all disk writes

**P0-5: CSRF Store Deadlock on Double-Stop** (csrf_store.go:103-108)
- **Problem**: stopChan unbuffered, no sync.Once guard, uses send not close
- **Impact**: Second Stop() call blocks forever waiting for receiver
- **Fix**:
  - Added sync.Once field stopOnce
  - Changed to close(stopChan) within stopOnce.Do()
  - Prevents double-close panic and deadlock

All fixes maintain backwards compatibility. The recovery token fix is particularly critical as it closes a security vulnerability allowing replay attacks on password reset flows.
2025-11-07 10:13:15 +00:00
rcourtman
431769024f Fix P1: Config Persistence transaction field synchronization
**Problem**: writeConfigFileLocked() accessed c.tx field without synchronization
- Function reads c.tx to check if transaction is active (line 109)
- c.tx modified by begin/endTransaction under lock, but read without lock
- Race condition: c.tx could change between check and use

**Impact**:
- Inconsistent transaction handling
- File could be written directly when it should be staged
- Or staged when it should be written directly
- Data corruption risk during config imports

**Fix** (lines 108-128):
- Added documentation that caller MUST hold c.mu lock
- Read c.tx into local variable tx while lock is held
- Use local copy for transaction check
- Safe because all callers hold c.mu when calling writeConfigFileLocked
- Transaction field only modified while holding c.mu in begin/endTransaction

This maintains the existing contract (callers hold lock) while making the transaction read safe and explicit.
2025-11-07 10:00:31 +00:00
rcourtman
6ca4d9b750 Fix P1/P2 infrastructure issues: panic recovery and optimizations
This commit addresses 4 P1 important issues and 1 P2 optimization in infrastructure components:

**P1-1: Missing Panic Recovery in Discovery Service** (service.go:172-195, 499-542)
- **Problem**: No panic recovery in Start(), ForceRefresh(), SetSubnet() goroutines
- **Impact**: Silent service death if scan panics, broken discovery with no monitoring
- **Fix**:
  - Wrapped initial scan goroutine with defer/recover (lines 172-182)
  - Wrapped scanLoop goroutine with defer/recover (lines 185-195)
  - Wrapped ForceRefresh scan with defer/recover (lines 499-509)
  - Wrapped SetSubnet scan with defer/recover (lines 532-542)
  - All log panics with stack traces for debugging

**P1-2: Missing Panic Recovery in Config Watcher Callback** (watcher.go:546-556)
- **Problem**: User-provided onMockReload callback could panic and crash watcher
- **Impact**: Panicking callback kills watcher goroutine, no config updates
- **Fix**: Wrapped callback invocation with defer/recover and stack trace logging

**P1-3: Session Store Stop() Using Send Instead of Close** (session_store.go:16-84)
- **Problem**: Stop() used channel send which blocks if nobody reads
- **Impact**: Stop() hangs if backgroundWorker already exited
- **Fix**:
  - Added sync.Once field stopOnce (line 22)
  - Changed Stop() to use close() within stopOnce.Do() (lines 80-84)
  - Prevents double-close panic and ensures all readers are signaled

**P2-1: Backup Cleanup Inefficient O(n²) Sort** (persistence.go:1424-1427)
- **Problem**: Bubble sort used to sort backups by modification time
- **Impact**: Inefficient for large backup counts (>100 files)
- **Fix**:
  - Replaced bubble sort with sort.Slice() using O(n log n) algorithm
  - Added "sort" import (line 9)
  - Maintains same oldest-first ordering for deletion logic

All fixes add defensive programming without changing external behavior. Panic recovery ensures services continue operating even with bugs, while optimization reduces cleanup time for backup-heavy environments.
2025-11-07 09:55:22 +00:00
rcourtman
ba6d934204 Fix critical P0 infrastructure concurrency issues
This commit addresses 3 critical P0 race conditions and resource leaks in core infrastructure:

**P0-1: Discovery Service Goroutine Leak** (service.go:468, 488)
- **Problem**: ForceRefresh() and SetSubnet() spawned unbounded goroutines without checking if scan already in progress
- **Impact**: Rapid API calls create goroutine explosion, resource exhaustion
- **Fix**:
  - ForceRefresh: Check isScanning before spawning goroutine (lines 470-476)
  - SetSubnet: Check isScanning, defer scan if already running (lines 491-504)
  - Both now log when skipping to aid debugging

**P0-2: Config Persistence Unlock/Relock Race** (persistence.go:1177-1206)
- **Problem**: LoadNodesConfig() unlocked RLock, called SaveNodesConfig (acquires Lock), then relocked
- **Impact**: Another goroutine could modify config between unlock/relock, causing migrated data loss
- **Fix**:
  - Copy instance slices while holding RLock to ensure consistency (lines 1189-1194)
  - Release lock, save copies, then return without relocking (lines 1196-1205)
  - Prevents TOCTOU vulnerability where migrations could be overwritten

**P0-3: Config Watcher Channel Close Race** (watcher.go:19-178)
- **Problem**: Stop() used select-check-close pattern vulnerable to concurrent calls
- **Impact**: Multiple Stop() calls panic on double-close
- **Fix**:
  - Added sync.Once field stopOnce to ConfigWatcher struct (line 26)
  - Changed Stop() to use stopOnce.Do() ensuring single execution (lines 175-178)
  - Removed racy select-based guard

All fixes maintain backwards compatibility and add defensive logging for operational visibility.
2025-11-07 09:49:55 +00:00
rcourtman
1183b87fa1 Fix critical alert system concurrency and memory leak issues
This commit addresses 7 critical issues identified during the alert system audit:

**P0 Critical - Race Conditions Fixed:**

1. **dispatchAlert race in NotifyExistingAlert** (lines 5486-5497)
   - Changed from RLock to Lock to hold mutex during dispatchAlert call
   - dispatchAlert calls checkFlapping which writes to maps (flappingHistory, flappingActive, suppressedUntil)
   - Previous code: grabbed RLock, got alert pointer, released lock, then called dispatchAlert (RACE)
   - Fixed: hold Lock through dispatchAlert call

2. **dispatchAlert race in LoadActiveAlerts startup** (lines 8216-8235)
   - Startup goroutines called dispatchAlert without holding lock
   - Added m.mu.Lock/Unlock around dispatchAlert call in goroutine
   - Also added cancellation via escalationStop channel to prevent goroutine leaks on shutdown

3. **checkFlapping documentation** (line 738)
   - Added clear comment that checkFlapping requires caller to hold m.mu
   - Prevents future race conditions from improper usage

**P1 Important - Data Loss Prevention:**

4. **History save race condition** (lines 177-180 in history.go)
   - Added saveMu mutex to serialize disk writes
   - Previous: concurrent saves could interleave, causing newer data to be overwritten by older snapshots
   - Fixed: saveMu.Lock at start of saveHistoryWithRetry ensures atomic disk writes
   - Newer snapshots now always win over older ones

**P2 Memory Leak Prevention:**

5. **PMG anomaly tracker cleanup** (lines 7318-7331)
   - Added cleanup for pmgAnomalyTrackers map (24 hour TTL based on LastSampleTime)
   - Prevents unbounded growth from decommissioned/transient PMG instances
   - Each tracker: ~1-2KB (48 samples + baselines)

6. **PMG quarantine history cleanup** (lines 7333-7354)
   - Added cleanup for pmgQuarantineHistory map (7 day TTL based on last snapshot)
   - Prevents memory leak for deleted PMG instances
   - Removes both empty histories and very old histories

**P2 Goroutine Leak Prevention:**

7. **Startup notification goroutine cancellation** (lines 8218-8234)
   - Added select with escalationStop channel to cancel startup notifications
   - Prevents goroutines from continuing after Stop() is called
   - Scales with number of restored critical alerts

All fixes maintain proper lock ordering and prevent deadlocks by ensuring locks are held when accessing shared maps.
2025-11-07 09:12:28 +00:00
rcourtman
99e5a38534 Fix critical monitoring system issues and add robustness improvements
This commit addresses 9 critical issues identified during the monitoring system audit:

**Race Conditions Fixed:**
- PBS backup pollers: Moved lock earlier to eliminate check-then-act race (lines 7316-7378)
- PVE backup poll timing: Fixed double write to lastPVEBackupPoll with proper synchronization (lines 5927-5977)
- Docker hosts cleanup: Refactored to avoid holding both m.mu and s.mu locks simultaneously (lines 1911-1937)

**Context Propagation Fixed:**
- Replaced all context.Background() calls with parent context for proper cancellation chain:
  - PBS backup poller (line 7367)
  - PVE backup poller (line 5955)
  - PBS fallback check (line 7154)

**Memory Leak Prevention:**
- Added cleanup for guest metadata cache (10 minute TTL, lines 1942-1957)
- Added cleanup for diagnostic snapshots (1 hour TTL, lines 1959-1987)
- Added cleanup for RRD cache (1 minute TTL, lines 1989-2007)
- All cleanup methods called on 10-second ticker (lines 3791-3793)

**Panic Recovery:**
- Added recoverFromPanic helper to log panics with stack traces (lines 1910-1920)
- Protected all critical goroutines:
  - poll (line 4020)
  - taskWorker (line 4200)
  - retryFailedConnections (line 3851)
  - checkMockAlerts (line 8896)
  - pollPVEInstance (line 4886)
  - pollPBSInstance (line 7164)
  - pollPMGInstance (line 7498)

**Import Fixes:**
- Added missing sync import to email_enhanced.go
- Added missing os import to queue.go

All fixes maintain proper lock ordering and release locks before calling methods that acquire other locks to prevent deadlocks.
2025-11-07 08:52:37 +00:00
rcourtman
9257071ca1 Add encryption status to notification health endpoint (P2)
Backend:
- Add IsEncryptionEnabled() method to ConfigPersistence
- Include encryption status in /api/notifications/health response
- Allows frontend to warn when credentials are stored in plaintext

Frontend:
- Update NotificationHealth type to include encryption.enabled field
- Frontend can now display warnings when encryption is disabled

This addresses the P2 requirement for encryption visibility, allowing
operators to know when notification credentials are not encrypted at rest.
2025-11-07 08:36:55 +00:00
rcourtman
b70dc3d00d Document layered retry semantics (P2 documentation)
Add documentation to explain how transport-level and queue-level retries interact:
- Email: MaxRetries (transport) * MaxAttempts (queue) = total SMTP attempts
- Webhooks: RetryCount (transport) * MaxAttempts (queue) = total HTTP attempts
- Example: 3 * 3 = 9 total delivery attempts for a single notification

This clarifies the multiplicative retry behavior and helps operators understand
the actual retry counts when using the persistent queue.
2025-11-07 08:35:00 +00:00