Commit graph

122 commits

Author SHA1 Message Date
rcourtman
d2330cf405 refactor(ai): Remove over-engineered URL discovery service
Keep only the simple AI-powered approach:
- set_resource_url tool lets AI save discovered URLs
- Users ask AI directly: 'Find URLs for my containers'
- AI uses its intelligence to discover and set URLs

Removed:
- URLDiscoveryService (rigid port scanning)
- Bulk discovery API endpoints
- Frontend discovery button

The AI itself is smart enough to iterate through resources
and discover URLs when asked.
2025-12-10 08:35:24 +00:00
rcourtman
67351bf58c feat: AI integration, Docker metrics, RAID display, and infrastructure improvements
- Add Claude OAuth authentication support with hybrid API key/OAuth flow
- Implement Docker container historical metrics in backend and charts API
- Add CEPH cluster data collection and new Ceph page
- Enhance RAID status display with detailed tooltips and visual indicators
- Fix host deduplication logic with Docker bridge IP filtering
- Fix NVMe temperature collection in host agent
- Add comprehensive test coverage for new features
- Improve frontend sparklines and metrics history handling
- Fix navigation issues and frontend reload loops
2025-12-09 09:29:27 +00:00
rcourtman
cc4b32ad4a fix: complete unified resources WebSocket integration
Backend:
- Call SetMonitor after router creation to inject resource store
- Add debug logging for resource population and broadcast

Frontend:
- Add resources array to WebSocket store initial state
- Handle resources in WebSocket message processing
- Use reconcile for efficient state updates

The unified resources are now properly:
1. Populated from StateSnapshot on each broadcast cycle
2. Converted to frontend format (ResourceFrontend)
3. Included in WebSocket state messages
4. Received and stored in frontend state
5. Consumed by migrated route components

Console now shows '[DashboardView] Using unified resources: VMs: X'
confirming the migration is working end-to-end.
2025-12-07 23:52:00 +00:00
rcourtman
90c45968e7 AI Problem Solver implementation and various fixes
- Implement 'Show Problems Only' toggle combining degraded status, high CPU/memory alerts, and needs backup filters
- Add 'Investigate with AI' button to filter bar for problematic guests
- Fix dashboard column sizing inconsistencies between bars and sparklines view modes
- Fix PBS backups display and polling
- Refine AI prompt for general-purpose usage
- Fix frontend flickering and reload loops during initial load
- Integrate persistent SQLite metrics store with Monitor
- Fortify AI command routing with improved validation and logging
- Fix CSRF token handling for note deletion
- Debug and fix AI command execution issues
- Various AI reliability improvements and command safety enhancements
2025-12-06 23:46:08 +00:00
rcourtman
cc3c0187a0 feat: AI features, agent improvements, and host monitoring enhancements
AI Chat Integration:
- Multi-provider support (Anthropic, OpenAI, Ollama)
- Streaming responses with markdown rendering
- Agent command execution for remote troubleshooting
- Context-aware conversations with host/container metadata

Agent Updates:
- Add --enable-proxmox flag for automatic PVE/PBS token setup
- Improve auto-update with semver comparison (prevents downgrades)
- Add updatedFrom tracking to report previous version after update
- Reduce initial update check delay from 30s to 5s
- Add agent version column to Hosts page table

Host Metrics:
- Add DiskIO stats collection (read/write bytes, ops, time)
- Improve disk filtering to exclude Docker overlay mounts
- Add RAID array monitoring via mdadm
- Enhanced temperature sensor parsing

Frontend:
- New Agent Version column on Hosts overview table
- Improved node modal with agent-first installation flow
- Add DiskIO display in host drawer
- Better responsive handling for metric bars
2025-12-05 10:37:02 +00:00
rcourtman
3f08bfe8d0 wip: AI chat integration with multi-provider support
- Add AI service with Anthropic, OpenAI, and Ollama providers
- Add AI chat UI component with streaming responses
- Add AI settings page for configuration
- Add agent exec framework for command execution
- Add API endpoints for AI chat and configuration
2025-12-04 20:16:53 +00:00
rcourtman
71e3ab4c58 fix: Force tcp4 network for IPv4 addresses in sensor-proxy HTTP mode
On dual-stack systems with net.ipv6.bindv6only=1 (like some Proxmox 8
configurations), Go's net.Listen("tcp", "0.0.0.0:8443") may still bind
to IPv6-only. This caused IPv4 localhost connections to hang while
IPv6 worked.

Fix by detecting IPv4 addresses and explicitly using "tcp4" network
type when creating the listener. Related to #805
2025-12-04 20:09:37 +00:00
rcourtman
95ca0c0e67 fix: Default sensor-proxy HTTP to 0.0.0.0:8443 for IPv4 binding
On systems with net.ipv6.bindv6only=1 (including some Proxmox 8
configurations), using ":8443" results in IPv6-only binding. Users
reported curl to 127.0.0.1:8443 hanging while [::1]:8443 worked.

Changed default from ":8443" to "0.0.0.0:8443" to explicitly bind IPv4.

Related to #805
2025-12-03 20:25:08 +00:00
rcourtman
b6221dcc27 Add health/metrics server and proper cleanup to unified agent
- Add /healthz (liveness) and /readyz (readiness) endpoints
- Add /metrics endpoint with Prometheus metrics (pulse_agent_info, pulse_agent_up)
- Properly call dockerAgent.Close() on shutdown
- New config: -health-addr flag and PULSE_HEALTH_ADDR env (default :9191)
- Set to empty string to disable health server
2025-12-02 22:42:05 +00:00
rcourtman
72af7bed5e style: Apply gofmt to 37 files
Standardize code formatting across test files and monitor.go.
No functional changes.
2025-12-02 17:21:48 +00:00
rcourtman
29f5202b72 Add unit tests for validation utility functions (pulse-sensor-proxy)
Add 42 test cases for security-critical validation utility functions:

- TestStripNodeDelimiters (10 cases): IPv6 bracket handling, edge cases
- TestParseNodeIP (10 cases): IPv4/IPv6 parsing with bracket support
- TestNormalizeAllowlistEntry (11 cases): case normalization, whitespace
  handling, IPv6 full form compression
- TestIPAllowed (11 cases): CIDR matching, hosts map lookup, nil handling

These functions are used for node allowlist validation to prevent SSRF
attacks in the sensor proxy.
2025-11-30 17:04:43 +00:00
rcourtman
43be83b60a Add unit tests for dedupeUint32 and dedupeStrings (pulse-sensor-proxy) 2025-11-30 15:26:42 +00:00
rcourtman
af80f303a5 Add unit tests for pulse-sensor-proxy utility functions
- hashIPToUID: 11 test cases covering IP hashing for rate limiting
  (determinism, range bounds, collision detection, boundary values)
- extractNodesFromYAML: 17 test cases covering YAML node list parsing
  (map format, list format, mixed types, edge cases)

First test files for config_cmd.go and http_server.go utilities.
2025-11-30 14:19:43 +00:00
rcourtman
6ae7ed4ae0 Add pagination to backup list for large backup counts
- Add pagination (100 items per page) to prevent UI lockup with 2500+ backups
- Show year in date labels for non-current year backups
- Reset to page 1 when filters change
- Add First/Previous/Next/Last navigation controls

Fixes #541
2025-11-30 09:55:01 +00:00
rcourtman
59067411a9 Add unit tests for pulse-agent utility functions (unified agent)
53 test cases covering 4 functions:
- gatherTags: environment/flag tag merging (18 cases)
- parseLogLevel: log level parsing (30 cases)
- defaultLogLevel: default value resolution (10 cases)
- multiValue: flag.Value interface for repeatable flags (6 cases)

Key difference from pulse-host-agent: parseLogLevel in unified agent delegates
directly to zerolog.ParseLevel without range validation, so trace/fatal/panic
levels are accepted (unlike pulse-host-agent which restricts to debug-error).

First test file for cmd/pulse-agent package.
2025-11-30 08:34:09 +00:00
rcourtman
1e0418b0e2 Add unit tests for pulse-host-agent utility functions
60 test cases for gatherTags, parseLogLevel, defaultLogLevel, multiValue.
First test file for cmd/pulse-host-agent package.
2025-11-30 08:17:18 +00:00
rcourtman
e8bf978544 Add unit tests for parseLogLevel and splitStringList (dockeragent) 2025-11-30 06:50:09 +00:00
rcourtman
ccc248f497 Add unit tests for sanitizeNodeLabel (pulse-sensor-proxy/metrics.go) 2025-11-30 04:50:51 +00:00
rcourtman
6889d09f8f Add unit tests for pulse-sensor-proxy capability functions
Test Capability.Has, parseCapabilityList, capabilityNames, and
constant values. 54 test cases covering bitmask operations, parsing,
case insensitivity, whitespace handling, unknown values, and round-trip
consistency.
2025-11-30 03:32:53 +00:00
rcourtman
e29fda5e5a Rebuild agent token bindings on API token config reload
When api_tokens.json is modified on disk, the ConfigWatcher reloads
the tokens into memory. However, the Monitor's dockerTokenBindings and
hostTokenBindings maps were not synchronized with the new token set,
causing orphaned bindings when agents reconnect after reinstall.

Add SetAPITokenReloadCallback to ConfigWatcher that triggers Monitor's
new RebuildTokenBindings method after token reload. This method
reconstructs the binding maps from current Docker host and host agent
state, keeping only bindings for tokens that still exist in config.

Related to #773
2025-11-29 14:09:30 +00:00
rcourtman
c104ceb19e feat: add self-update capability to standalone pulse-host-agent
The standalone pulse-host-agent was missing self-update functionality
that existed in pulse-docker-agent and the unified pulse-agent.

Changes:
- Add agentupdate integration to pulse-host-agent
- Add --no-auto-update flag and PULSE_NO_AUTO_UPDATE env var
- Update Windows service to use errgroup pattern with auto-updater
- Move version from internal/hostagent to main package for ldflags

Related to #737
2025-11-27 20:21:05 +00:00
rcourtman
cf62ebae27 Add Windows service support to unified agent
Port Windows SCM integration from pulse-host-agent to pulse-agent,
enabling the unified agent to run as a Windows service with proper
start/stop handling and event logging.

Related to #766
2025-11-27 17:00:03 +00:00
rcourtman
f7ffb36c41 chore: remove dead code and unused exports
Remove ~900 lines of unused code identified by static analysis:

Go:
- internal/logging: Remove 10 unused functions (InitFromConfig, New,
  FromContext, WithLogger, etc.) that were built but never integrated
- cmd/pulse-sensor-proxy: Remove 7 dead validation functions for a
  removed command execution feature
- internal/metrics: Remove 8 unused notification metric functions and
  10 Prometheus metrics that were never wired up

Frontend:
- Delete ActivationBanner.tsx stub component
- Remove unused exports: stopMetricsSampler, getSamplerStatus,
  formatSpeedCompact, parseMetricKey, getResourceAlerts
2025-11-27 13:17:39 +00:00
rcourtman
e9ac1429c0 refactor: remove unnecessary type conversions
Remove redundant type conversions identified by unconvert linter:
- Remove int() conversions for already-int VMID fields
- Remove int64() conversions for already-int64 arithmetic results
- Remove uint64() conversions for already-uint64 Disk/MaxDisk fields
- Remove int() on syscall.Stdin (already int constant)
2025-11-27 10:33:35 +00:00
rcourtman
67b60adb65 style: fix revive linter warnings
- Mark unused stub parameters with underscore
- Rename 'copy' variable to avoid shadowing builtin
- Remove unnecessary else blocks after return statements
2025-11-27 10:26:26 +00:00
rcourtman
1fb4a2c2c6 chore: remove deprecated build tags and use strings.ReplaceAll
- Remove redundant // +build directives (go:build is sufficient in Go 1.17+)
- Replace strings.Replace(..., -1) with strings.ReplaceAll
2025-11-27 10:16:08 +00:00
rcourtman
e25e1af8cb chore: fix staticcheck SA warnings
- Fix SA4006 unused value issues in ssh.go, validation.go, generator.go
- Replace deprecated ioutil with io/os in config.go
- Replace deprecated tar.TypeRegA with tar.TypeReg
- Remove deprecated rand.Seed calls (auto-seeded in Go 1.20+)
- Fix always-true nil check in main.go
- Fix impossible nil comparison in tempproxy/client.go
- Add nil check for config in monitor.New()
2025-11-27 09:16:53 +00:00
rcourtman
4fd3bdbc04 chore: fix staticcheck U1000 unused code warnings
- Remove unused ipv6Regex from validation.go
- Suppress unused recordAlertFired/recordAlertResolved hooks (kept for future use)
- Remove unused apiLimiter rate limiter
- Remove unused stopOnce fields from csrf_store.go and session_store.go
- Remove unused lastBroadcast field from hub.go
- Remove unused lastUsedIndex field from cluster_client.go
2025-11-27 09:12:17 +00:00
rcourtman
c0fff33aed chore: remove legacy proxy handlers and unused functions
Remove legacy V1 handlers replaced by V2 versions:
- sendError (replaced by sendErrorV2)
- handleGetStatus (replaced by handleGetStatusV2)
- handleEnsureClusterKeys (replaced by handleEnsureClusterKeysV2)
- handleRegisterNodes (replaced by handleRegisterNodesV2)
- handleGetTemperature (replaced by handleGetTemperatureV2)

Also remove related unused functions:
- getPublicKey wrapper (only getPublicKeyFrom is used)
- pushSSHKey wrapper (only pushSSHKeyFrom is used)
- nodeValidator.ipAllowed method (standalone ipAllowed is used)
- validateConfigFile (never called)
- runServiceDebug (Windows debug mode, never called)
2025-11-27 08:41:28 +00:00
rcourtman
2fe7bb6141 style: fix gofmt formatting inconsistencies
Run gofmt -w to fix tab/space inconsistencies across 33 files.
2025-11-26 23:44:36 +00:00
rcourtman
e467523a61 feat: serve install scripts from GitHub releases instead of main branch
Scripts like install.sh and install-sensor-proxy.sh are now attached
as release assets and downloaded from releases/latest/download/ URLs.
This ensures users always get scripts compatible with their installed
version, even while development continues on main.

Changes:
- build-release.sh: copy install scripts to release directory
- create-release.yml: upload scripts as release assets
- Updated all documentation and code references to use release URLs
- Scripts reference each other via release URLs for consistency
2025-11-26 08:59:59 +00:00
rcourtman
0f0832d30f fix: propagate unified agent version and improve legacy cleanup
Issues found during scenario testing:

1. Version propagation: The hostagent and dockeragent packages were
   reporting their own Version (0.1.0-dev) instead of the unified
   agent's version. Added AgentVersion config field to pass the
   parent's version down.

2. macOS legacy cleanup: The install.sh script was missing cleanup
   for pulse-docker-agent on macOS.

3. Windows legacy cleanup: The install.ps1 script was missing cleanup
   for legacy PulseHostAgent and PulseDockerAgent services.

These fixes ensure:
- Unified agent reports consistent version across host/docker metrics
- Legacy agents are properly removed on all platforms during upgrade
- Users migrating from legacy agents get a clean transition
2025-11-25 23:39:10 +00:00
rcourtman
920f271b41 feat: improve legacy agent detection and migration UX
Add seamless migration path from legacy agents to unified agent:

- Add AgentType field to report payloads (unified vs legacy detection)
- Update server to detect legacy agents by type instead of version
- Add UI banner showing upgrade command when legacy agents are detected
- Add deprecation notice to install-host-agent.ps1
- Create install-docker-agent.sh stub that redirects to unified installer

Legacy agents (pulse-host-agent, pulse-docker-agent) now show a "Legacy"
badge in the UI with a one-click copy command to upgrade to the unified
agent.
2025-11-25 23:26:22 +00:00
rcourtman
ee35d9e5a5 feat: add auto-update support for unified agent
Implement self-update capability for the unified pulse-agent binary:

- Add internal/agentupdate package with cross-platform update logic
- Hourly version checks against /api/agent/version endpoint
- SHA256 checksum verification for downloaded binaries
- Atomic binary replacement with backup/rollback on failure
- Support for Linux, macOS, and Windows (10 platform/arch combinations)

Build and release changes:
- Dockerfile builds unified agent for all platforms
- build-release.sh includes unified agent in release artifacts
- validate-release.sh validates unified agent binaries
- Install scripts (install.sh, install.ps1) use correct URL format

Related to #727, #737
2025-11-25 23:15:03 +00:00
courtmanr@gmail.com
26ebd476da WIP: Save all pending changes including frontend updates and unified agent scaffolding 2025-11-25 11:27:07 +00:00
courtmanr@gmail.com
9fdea3c9cd fix: filter out qdevice from cluster node discovery 2025-11-24 22:54:58 +00:00
courtmanr@gmail.com
53836ea9b0 fix(sensor-proxy): relax pvecm status parsing to support decimal node IDs
Fixes an issue where pvecm status output using decimal node IDs (e.g. '1' instead of '0x1') caused node discovery to fail. Added test case for this format.
2025-11-23 08:21:58 +00:00
courtmanr@gmail.com
b2c4a583f7 Fix pvecm status parsing for QDevice flags (#738) 2025-11-22 23:44:01 +00:00
rcourtman
cd31bdece1 Add log level control to host agent
Related to #742
2025-11-22 07:48:34 +00:00
rcourtman
19a2cac355 Add log level control for docker agent
Related to #742
2025-11-22 07:43:48 +00:00
rcourtman
335795d354 Ensure sensor proxy wrapper delivers SMART temps locally 2025-11-21 10:07:42 +00:00
courtmanr@gmail.com
17d55b8cf4 feat: implement atomic config management in sensor proxy 2025-11-20 19:01:24 +00:00
rcourtman
823508dc48 Related to #712: auto-restore host agent binaries for download 2025-11-20 15:45:21 +00:00
courtmanr@gmail.com
838993cf40 Implement sensor proxy installation and configuration updates 2025-11-20 13:23:21 +00:00
rcourtman
3f10c97c4e docs: align sensor proxy config with current defaults 2025-11-20 12:40:01 +00:00
courtmanr@gmail.com
3f974c06ca Fix macOS build for sensor-proxy and improve hot-dev script 2025-11-20 12:28:01 +00:00
rcourtman
9d20acd35e fix(sensor-proxy): eliminate all uncoordinated config writers
Remove all code paths that manipulate config files without Phase 2 locking:

1. Installer: Remove ensure_allowed_nodes_file_reference() call (line 1674)
   - Migration now handled exclusively by config migrate-to-file

2. Installer: Make migration failures fatal in update_allowed_nodes()
   - Prevents fallback to unsafe Python manipulation

3. Daemon sanitizer: Remove os.WriteFile() call
   - Now only sanitizes in-memory copy, doesn't write back to disk
   - Logs warning instructing admin to run `config migrate-to-file`

4. Self-heal script: Replace 132 lines of Python with CLI call
   - sanitize_allowed_nodes() now calls `config migrate-to-file`
   - Eliminates uncoordinated Python-based config rewriting

All config mutations now flow exclusively through Phase 2 CLI with
atomic operations and file locking. No code paths remain that can
create duplicate allowed_nodes blocks.

Addresses Codex review feedback on Phase 2 gaps.
2025-11-19 10:55:01 +00:00
rcourtman
0e2559a6d4 fix(sensor-proxy): sanitize duplicate blocks before migration
The migrate-to-file command now calls sanitizeDuplicateAllowedNodesBlocks
before parsing the config, allowing it to handle corrupted configs with
duplicate allowed_nodes blocks.

This ensures migration works even on hosts that were affected by the
original corruption issue.
2025-11-19 10:38:04 +00:00
rcourtman
c1185ebc9d feat(sensor-proxy): complete Phase 2 with CLI-based config migration
Add `config migrate-to-file` command and update installer to eliminate
all shell/Python config manipulation, ensuring atomic operations throughout.

Changes:
- Add `config migrate-to-file` command to atomically migrate inline
  allowed_nodes blocks to file-based configuration
- Update installer's update_allowed_nodes() to call CLI exclusively
- Simplify migrate_inline_allowed_nodes_to_file() to use CLI
- Remove dependency on Python/sed for config manipulation
- Implement dual-file locking (config.yaml + allowed_nodes.yaml) to
  prevent race conditions during migration

All config mutations now flow through the Phase 2 CLI with:
- File locking (flock)
- Atomic writes (temp + rename + fsync)
- Proper YAML parsing/generation

This completes Phase 2 architecture and eliminates the root cause of
config corruption issues.

Related to prior commits: 53dec6010, 3dc073a28, 804a638ea, 131666bc1
2025-11-19 10:35:49 +00:00
rcourtman
a4cd3a275e docs(sensor-proxy): comprehensive config management documentation
Adds complete documentation for the new sensor-proxy config management CLI
implemented in Phase 2. Addresses user-facing aspects of the corruption fix.

**New Documentation:**
- docs/operations/sensor-proxy-config-management.md (469 lines)
  - Complete operations runbook for config management
  - Full CLI reference with examples
  - Migration guide from inline config
  - Architecture explanation
  - Common operational tasks
  - Troubleshooting guide
  - Best practices and automation

**Updated Documentation:**
- cmd/pulse-sensor-proxy/README.md
  - Configuration Management CLI section
  - Allowed Nodes File format
  - Enhanced troubleshooting
  - Config corruption recovery

- docs/TEMPERATURE_MONITORING.md
  - Config validation failure troubleshooting
  - Configuration Management quick reference
  - Cross-links to detailed docs

- docs/TROUBLESHOOTING.md
  - Sensor proxy config validation errors
  - Comprehensive diagnosis steps
  - Automatic and manual recovery

- README.md & docs/README.md
  - Added new runbook to operations index
  - Positioned for discoverability

**Coverage:**
- Both CLI commands fully documented
- Phase 1 & Phase 2 architecture explained
- Migration path from pre-v4.31.1
- Config corruption recovery procedures
- Safe config editing practices
- Automation examples
- Troubleshooting all failure modes

**Documentation Quality:**
- Cross-linked from 5 different documents
- Clear examples for common use cases
- Target audience: system administrators
- Follows project documentation style
- Production-ready

This completes the sensor-proxy config corruption fix by providing users
with comprehensive guidance for the new config management system.

Related to Phase 2 commits 3dc073a28, 804a638ea, 131666bc1
2025-11-19 10:01:33 +00:00