Commit graph

131 commits

Author SHA1 Message Date
rcourtman
67351bf58c feat: AI integration, Docker metrics, RAID display, and infrastructure improvements
- Add Claude OAuth authentication support with hybrid API key/OAuth flow
- Implement Docker container historical metrics in backend and charts API
- Add CEPH cluster data collection and new Ceph page
- Enhance RAID status display with detailed tooltips and visual indicators
- Fix host deduplication logic with Docker bridge IP filtering
- Fix NVMe temperature collection in host agent
- Add comprehensive test coverage for new features
- Improve frontend sparklines and metrics history handling
- Fix navigation issues and frontend reload loops
2025-12-09 09:29:27 +00:00
rcourtman
cc4b32ad4a fix: complete unified resources WebSocket integration
Backend:
- Call SetMonitor after router creation to inject resource store
- Add debug logging for resource population and broadcast

Frontend:
- Add resources array to WebSocket store initial state
- Handle resources in WebSocket message processing
- Use reconcile for efficient state updates

The unified resources are now properly:
1. Populated from StateSnapshot on each broadcast cycle
2. Converted to frontend format (ResourceFrontend)
3. Included in WebSocket state messages
4. Received and stored in frontend state
5. Consumed by migrated route components

Console now shows '[DashboardView] Using unified resources: VMs: X'
confirming the migration is working end-to-end.
2025-12-07 23:52:00 +00:00
rcourtman
881f00a5d9 ensure resource store is populated before WebSocket broadcasts
- Added PopulateFromSnapshot method to resources.Store
- Extended ResourceStoreInterface to include PopulateFromSnapshot
- Monitor now calls updateResourceStore before broadcasts
- This ensures resources are fresh on every WebSocket broadcast

Without this, the store would only be populated when /api/resources or
/api/state endpoints are hit, leaving WebSocket broadcasts empty.
2025-12-07 23:15:02 +00:00
rcourtman
cdd2249e00 phase 1: add unified resources to WebSocket state broadcasts
- Extended StateFrontend with Resources field containing unified resource data
- Added ResourceFrontend and related types for frontend-compatible resource data
- Extended ResourceStoreInterface to include GetAll() method
- Monitor now injects resources into WebSocket broadcasts
- Added helper method getResourcesForBroadcast() to convert resources to frontend format
- All existing tests pass

This enables the frontend to access unified resources via WebSocket state.
2025-12-07 23:02:56 +00:00
rcourtman
5f000b7974 feat: Complete Unified Resource Architecture (Phases 1-3)
This commit implements the Unified Resource Architecture for AI-first
infrastructure management. Key features:

Phase 1 - Backend Unification:
- New unified Resource type with 9 resource types, 7 platforms, 7 statuses
- Resource store with identity-based deduplication (hostname, machineID, IP)
- 8 converter functions (FromNode, FromVM, FromContainer, etc.)
- REST API endpoints: /api/resources, /api/resources/stats, /api/resources/{id}
- 28 comprehensive unit tests

Phase 2 - AI Context Enhancement:
- Unified context builder for AI system prompts
- Cross-platform query methods: GetTopByCPU, GetTopByMemory, GetTopByDisk
- Resource correlation: GetRelated (parent, children, siblings, cluster)
- Infrastructure summary: GetResourceSummary with health status counts
- AI context now includes top consumers and infrastructure overview

Phase 3 - Agent Preference & Hybrid Mode:
- Polling optimization methods in resource store
- ResourceStoreInterface added to Monitor
- SetResourceStore() and shouldSkipNodeMetrics() helper methods
- Store automatically wired into Monitor via Router.SetMonitor()
- Foundation ready for reduced API polling when agents are active

Files added:
- internal/resources/resource.go - Core Resource type
- internal/resources/store.go - Store with deduplication
- internal/resources/converters.go - Type converters
- internal/resources/platform_data.go - Platform-specific data
- internal/resources/store_test.go - 28 tests
- internal/resources/converters_test.go - Converter tests
- internal/api/resource_handlers.go - REST API handlers
- internal/ai/resource_context.go - AI context builder
- .gemini/docs/unified-resource-architecture.md - Architecture docs

All tests pass.
2025-12-07 13:49:00 +00:00
rcourtman
90c45968e7 AI Problem Solver implementation and various fixes
- Implement 'Show Problems Only' toggle combining degraded status, high CPU/memory alerts, and needs backup filters
- Add 'Investigate with AI' button to filter bar for problematic guests
- Fix dashboard column sizing inconsistencies between bars and sparklines view modes
- Fix PBS backups display and polling
- Refine AI prompt for general-purpose usage
- Fix frontend flickering and reload loops during initial load
- Integrate persistent SQLite metrics store with Monitor
- Fortify AI command routing with improved validation and logging
- Fix CSRF token handling for note deletion
- Debug and fix AI command execution issues
- Various AI reliability improvements and command safety enhancements
2025-12-06 23:46:08 +00:00
rcourtman
c9915ead8b feat: Add OS type display for LXC containers
- Extract ostype from LXC container config (debian, ubuntu, alpine, etc.)
- Map ostype values to human-readable names (e.g., "debian" -> "Debian")
- Add OSName field to Container model and ContainerFrontend
- Add icons for NixOS, openSUSE, and Gentoo in frontend
- LXC containers now show OS icons alongside VMs in the dashboard

Supported LXC OS types: alpine, archlinux, centos, debian, devuan,
fedora, gentoo, nixos, opensuse, ubuntu, unmanaged
2025-12-05 12:43:32 +00:00
rcourtman
cc3c0187a0 feat: AI features, agent improvements, and host monitoring enhancements
AI Chat Integration:
- Multi-provider support (Anthropic, OpenAI, Ollama)
- Streaming responses with markdown rendering
- Agent command execution for remote troubleshooting
- Context-aware conversations with host/container metadata

Agent Updates:
- Add --enable-proxmox flag for automatic PVE/PBS token setup
- Improve auto-update with semver comparison (prevents downgrades)
- Add updatedFrom tracking to report previous version after update
- Reduce initial update check delay from 30s to 5s
- Add agent version column to Hosts page table

Host Metrics:
- Add DiskIO stats collection (read/write bytes, ops, time)
- Improve disk filtering to exclude Docker overlay mounts
- Add RAID array monitoring via mdadm
- Enhanced temperature sensor parsing

Frontend:
- New Agent Version column on Hosts overview table
- Improved node modal with agent-first installation flow
- Add DiskIO display in host drawer
- Better responsive handling for metric bars
2025-12-05 10:37:02 +00:00
rcourtman
72af7bed5e style: Apply gofmt to 37 files
Standardize code formatting across test files and monitor.go.
No functional changes.
2025-12-02 17:21:48 +00:00
rcourtman
6a29e20b5e perf: Cache err.Error() in storage timeout error handling
Cache err.Error() result in two locations:
- monitor.go: storage query retry logic (2x calls to 1)
- monitor_polling.go: storage timeout handling (2x calls to 1)
2025-12-02 15:39:37 +00:00
rcourtman
5d2eca3660 perf: Cache err.Error() in disk monitoring error handling
Compute err.Error() once and reuse errStr instead of calling Error()
four times when checking disk monitoring error types.
2025-12-02 15:37:48 +00:00
rcourtman
b47a04aa97 perf: Use strconv.Itoa instead of fmt.Sprintf for int conversion
strconv.Itoa is faster than fmt.Sprintf("%d", ...) because it doesn't
need to parse a format string. Changed 4 occurrences in monitoring
package where integers are converted to strings.
2025-12-02 15:21:41 +00:00
rcourtman
2453198f62 refactor: Remove unreachable dead code branches
- firstForwardedValue: strings.Split always returns at least one element
- shouldRunBackupPoll: remaining is always >= 1 by math
- convertContainerDiskInfo: lowerLabel is never empty for non-rootfs

All three functions now at 100% coverage.
2025-12-02 14:41:53 +00:00
rcourtman
e07bd834b6 Filter virtual/system filesystems from host disk display
Host disk bars were showing virtual filesystems like tmpfs, /dev, /run,
/sys, and Docker overlay mounts. These clutter the UI and don't represent
meaningful disk usage.

Changed from `shouldIgnoreReadOnlyFilesystem` (read-only only) to the
full `fsfilters.ShouldSkipFilesystem` which also excludes:
- Virtual FS types: tmpfs, devtmpfs, sysfs, proc, cgroup, etc.
- Special mountpoints: /dev, /proc, /sys, /run, /var/lib/docker, /snap
- Network filesystems: fuse, nfs, cifs, etc.

Related to #790
2025-12-02 00:16:39 +00:00
rcourtman
e7bc06033a test: Add tests for convertDockerSwarmInfo, namespacePathsForDatastore, preserveFailedStorageBackups
- convertDockerSwarmInfo: 66.7%→100% (4 cases for nil, empty, populated structs)
- namespacePathsForDatastore: 92.3%→100% (removed unreachable dead code)
- preserveFailedStorageBackups: 91.3%→100% (6 cases for filtering, deduplication)
2025-12-01 19:04:23 +00:00
rcourtman
46a72a7600 Fix outdated error message path for removed Docker hosts
The error message referenced "Settings -> Docker -> Removed hosts" but
that UI path no longer exists. The correct path is now
"Settings -> Agents -> Removed Docker Hosts".

Related to #778
2025-12-01 12:03:05 +00:00
rcourtman
56726f15e0 refactor: Extract guest metadata functions to separate file
Move 17 guest-metadata-related functions from monitor.go to new
guest_metadata.go file. This includes:
- Guest agent API call retry logic
- Guest metadata caching (fetch, reserve, schedule, clear)
- Network interface processing and cloning
- OS info extraction
- Type conversion utilities (stringValue, anyToInt64)

Reduces monitor.go by 541 lines (8932 → 8391).
2025-12-01 10:53:41 +00:00
rcourtman
0b2485b26b refactor: Extract container parsing functions to separate file
Move 13 related container parsing functions from monitor.go (9359 lines)
to container_parsing.go (451 lines). Reduces monitor.go by 427 lines.

Functions extracted:
- ensureContainerRootDiskEntry
- convertContainerDiskInfo
- sanitizeRootFSDevice
- parseContainerRawIPs
- collectIPsFromInterface
- sanitizeGuestAddressStrings
- dedupeStringsPreserveOrder
- containerNetworkDetails struct
- containerMountMetadata struct
- parseContainerConfigNetworks
- parseContainerMountMetadata
- mergeContainerNetworkInterface
- extractContainerRootDeviceFromConfig
2025-12-01 10:40:48 +00:00
rcourtman
5a4bc3b053 refactor: Extract Docker host identifier functions to separate file
Move 9 related pure functions from monitor.go (9616 lines) to
docker_host_identity.go (281 lines). Reduces monitor.go by 257 lines.

Functions extracted:
- tokenHintFromRecord
- resolveDockerHostIdentifier
- findMatchingDockerHost
- dockerHostIDExists
- generateDockerHostIdentifier
- dockerHostSuffixCandidates
- sanitizeDockerHostSuffix
- fallbackDockerHostID
- uniqueNonEmptyStrings
2025-12-01 10:03:47 +00:00
rcourtman
54fab46bd6 Fix backup status indicator not showing for guests
The backup status indicator feature was incomplete - it added the UI
component but never populated VM/Container LastBackup from actual
backup data. Now SyncGuestBackupTimes() is called after storage
backups and PBS backups are polled, matching each guest's VMID to
its most recent backup timestamp.

Fixes #786
2025-11-30 22:13:46 +00:00
rcourtman
61bb124692 Extract filesystem filtering logic into pkg/fsfilters
Move the inline filesystem skip logic from pollVMsAndContainersEfficient
into a reusable ShouldSkipFilesystem function. This consolidates filtering
for virtual filesystems (tmpfs, cgroup, etc.), network mounts (nfs, cifs,
fuse), and special mountpoints (/dev, /proc, /snap, etc.) into one tested
location.

Reduces cyclomatic complexity of pollVMsAndContainersEfficient and adds
28 test cases covering virtual fs types, network mounts, special mounts,
Windows paths, and edge cases.
2025-11-29 16:38:08 +00:00
rcourtman
e29fda5e5a Rebuild agent token bindings on API token config reload
When api_tokens.json is modified on disk, the ConfigWatcher reloads
the tokens into memory. However, the Monitor's dockerTokenBindings and
hostTokenBindings maps were not synchronized with the new token set,
causing orphaned bindings when agents reconnect after reinstall.

Add SetAPITokenReloadCallback to ConfigWatcher that triggers Monitor's
new RebuildTokenBindings method after token reload. This method
reconstructs the binding maps from current Docker host and host agent
state, keeping only bindings for tokens that still exist in config.

Related to #773
2025-11-29 14:09:30 +00:00
rcourtman
e9ac1429c0 refactor: remove unnecessary type conversions
Remove redundant type conversions identified by unconvert linter:
- Remove int() conversions for already-int VMID fields
- Remove int64() conversions for already-int64 arithmetic results
- Remove uint64() conversions for already-uint64 Disk/MaxDisk fields
- Remove int() on syscall.Stdin (already int constant)
2025-11-27 10:33:35 +00:00
rcourtman
31927f71e9 fix: mark unused parameters to satisfy unparam linter
Mark intentionally unused parameters with underscore to:
- Silence unparam warnings for legitimate unused parameters
- Keep function signatures intact for API compatibility
- Remove unused req from serveChecksum helper
2025-11-27 10:12:48 +00:00
rcourtman
252d060a58 refactor: use builtin max() and fix unused parameter
- Replace custom maxInt64 helper with Go 1.21+ builtin max()
- Mark unused cfg parameter in newAdaptiveIntervalSelector
- Remove test for deleted helper function
2025-11-27 10:08:37 +00:00
rcourtman
e25e1af8cb chore: fix staticcheck SA warnings
- Fix SA4006 unused value issues in ssh.go, validation.go, generator.go
- Replace deprecated ioutil with io/os in config.go
- Replace deprecated tar.TypeRegA with tar.TypeReg
- Remove deprecated rand.Seed calls (auto-seeded in Go 1.20+)
- Fix always-true nil check in main.go
- Fix impossible nil comparison in tempproxy/client.go
- Add nil check for config in monitor.New()
2025-11-27 09:16:53 +00:00
rcourtman
920f271b41 feat: improve legacy agent detection and migration UX
Add seamless migration path from legacy agents to unified agent:

- Add AgentType field to report payloads (unified vs legacy detection)
- Update server to detect legacy agents by type instead of version
- Add UI banner showing upgrade command when legacy agents are detected
- Add deprecation notice to install-host-agent.ps1
- Create install-docker-agent.sh stub that redirects to unified installer

Legacy agents (pulse-host-agent, pulse-docker-agent) now show a "Legacy"
badge in the UI with a one-click copy command to upgrade to the unified
agent.
2025-11-25 23:26:22 +00:00
courtmanr@gmail.com
ebcafa1dac feat: adaptive node table layout, guest row fixes, and legacy agent detection
- Implemented adaptive layout for NodeSummaryTable with responsive columns and sticky name column.
- Fixed GuestRow background display issues.
- Added IsLegacy field to Host and DockerHost models to flag legacy agents (version < 1.0.0).
- Updated monitor to populate IsLegacy based on agent version.
2025-11-25 17:19:36 +00:00
courtmanr@gmail.com
75a9929afc Refactor: Parallelize PVE node polling 2025-11-25 08:38:03 +00:00
courtmanr@gmail.com
af7eabb56d Fix: Correct context cancellation in loop (Fixes #727)
- Replaced defer in loop with explicit cancellation to avoid resource leak
- Properly tagged issue #727
2025-11-23 22:28:28 +00:00
courtmanr@gmail.com
b34e18e839 Fix: Prevent single node auth failure from disabling global SSH temperature collection
- Removed global legacySSHDisabled flag that was triggered by any single node auth failure
- Changed disableLegacySSHOnAuthFailure to only log warnings
- Fixed potential context leak in monitor.go
- Updated tests to reflect removal of global disable logic
2025-11-23 22:24:15 +00:00
courtmanr@gmail.com
4bf9c42dbc Fix temperature collection regression for cluster nodes. Related to #727 2025-11-23 12:13:57 +00:00
courtmanr@gmail.com
d5fdf2f471 fix: ensure proxmox nodes are displayed even if cluster endpoints are missing
Fixes #727. Previously, if temperature monitoring was enabled and a node wasn't found in ClusterEndpoints, the entire node processing was skipped. This change ensures we only skip temperature collection.
2025-11-22 23:31:30 +00:00
rcourtman
93d8c624b4 Persist PVE fallback host after portless retry 2025-11-22 17:06:15 +00:00
rcourtman
8606bb47cf Handle PVE portless fallback when default port fails 2025-11-22 17:01:16 +00:00
rcourtman
4e68675e0b Add recovery notifications and grouping controls 2025-11-21 22:07:00 +00:00
rcourtman
5a6c6091bf Honor configured PVE polling interval in scheduler 2025-11-20 22:00:56 +00:00
rcourtman
f4e2cbca48 Improve token collision handling and installer subnet support 2025-11-20 09:45:36 +00:00
rcourtman
bf63515ff4 feat: avoid redundant PBS snapshot polling (Related to #717) 2025-11-18 23:10:43 +00:00
rcourtman
999e4271a2 feat: surface LXC mountpoints in UI (related to #715) 2025-11-18 22:57:20 +00:00
rcourtman
3f46d35a81 feat: make PVE polling interval configurable (related to #467) 2025-11-18 21:30:04 +00:00
rcourtman
274c45799f Add direct node fallback for storage polling 2025-11-18 19:58:38 +00:00
rcourtman
2e7f693c8b Improve installer temperature proxy and backup polling 2025-11-18 18:42:33 +00:00
rcourtman
d9c0ddaae2 Allow PBS backup poll to finish after poller returns 2025-11-18 16:55:46 +00:00
rcourtman
54c47fe2f5 Improve temperature proxy detection 2025-11-18 14:25:09 +00:00
rcourtman
527b7ceacc Skip inactive storages during content scans 2025-11-18 09:46:48 +00:00
rcourtman
f7261f3080 Harden turnkey install and proxy auto-registration 2025-11-18 00:24:50 +00:00
rcourtman
a479040651 Improve temperature proxy workflow 2025-11-17 14:25:46 +00:00
rcourtman
de5b314842 Improve temperature proxy control-plane flow 2025-11-15 21:49:51 +00:00
rcourtman
c9b4f7e88b Fix incorrect temperature data during cluster initialization
During cluster startup, nodes were temporarily using the primary cluster
endpoint for temperature collection before cluster metadata validation
completed. This caused all nodes to show the same (incorrect) temperature
values for ~4 minutes until validation finished and per-node endpoints
were established.

Example: minipc would show delly's temperature (90°C) instead of its own
(50°C) from startup until cluster validation completed.

Root cause:
- Temperature collection started immediately at startup
- Cluster endpoint validation happened asynchronously
- Code fell back to primary endpoint when ClusterEndpoints was empty
- All nodes used same endpoint, got same temperature data

Fix: Skip temperature collection for cluster nodes until:
1. ClusterEndpoints array is populated (validation complete)
2. Node's specific endpoint is found in the cluster metadata

This ensures correct temperature data from the very first collection,
maintaining data integrity during startup. When persisted config exists,
endpoints are available immediately so no delay occurs. For new clusters,
temperature collection begins once validation completes (~30s).

Preserves Pulse's correctness guarantee: users can trust metrics
immediately after restart without waiting for "warm-up" period.
2025-11-14 23:38:44 +00:00