Commit graph

244 commits

Author SHA1 Message Date
rcourtman
67351bf58c feat: AI integration, Docker metrics, RAID display, and infrastructure improvements
- Add Claude OAuth authentication support with hybrid API key/OAuth flow
- Implement Docker container historical metrics in backend and charts API
- Add CEPH cluster data collection and new Ceph page
- Enhance RAID status display with detailed tooltips and visual indicators
- Fix host deduplication logic with Docker bridge IP filtering
- Fix NVMe temperature collection in host agent
- Add comprehensive test coverage for new features
- Improve frontend sparklines and metrics history handling
- Fix navigation issues and frontend reload loops
2025-12-09 09:29:27 +00:00
rcourtman
cc4b32ad4a fix: complete unified resources WebSocket integration
Backend:
- Call SetMonitor after router creation to inject resource store
- Add debug logging for resource population and broadcast

Frontend:
- Add resources array to WebSocket store initial state
- Handle resources in WebSocket message processing
- Use reconcile for efficient state updates

The unified resources are now properly:
1. Populated from StateSnapshot on each broadcast cycle
2. Converted to frontend format (ResourceFrontend)
3. Included in WebSocket state messages
4. Received and stored in frontend state
5. Consumed by migrated route components

Console now shows '[DashboardView] Using unified resources: VMs: X'
confirming the migration is working end-to-end.
2025-12-07 23:52:00 +00:00
rcourtman
881f00a5d9 ensure resource store is populated before WebSocket broadcasts
- Added PopulateFromSnapshot method to resources.Store
- Extended ResourceStoreInterface to include PopulateFromSnapshot
- Monitor now calls updateResourceStore before broadcasts
- This ensures resources are fresh on every WebSocket broadcast

Without this, the store would only be populated when /api/resources or
/api/state endpoints are hit, leaving WebSocket broadcasts empty.
2025-12-07 23:15:02 +00:00
rcourtman
cdd2249e00 phase 1: add unified resources to WebSocket state broadcasts
- Extended StateFrontend with Resources field containing unified resource data
- Added ResourceFrontend and related types for frontend-compatible resource data
- Extended ResourceStoreInterface to include GetAll() method
- Monitor now injects resources into WebSocket broadcasts
- Added helper method getResourcesForBroadcast() to convert resources to frontend format
- All existing tests pass

This enables the frontend to access unified resources via WebSocket state.
2025-12-07 23:02:56 +00:00
rcourtman
ef35120afb checkpoint: remove unified resources UI, keep backend model
- Removed /resources page and associated frontend components
- Removed ResourcesOverview.tsx, UnifiedResourceRow.tsx, columns.ts
- Removed frontend types/resource.ts
- Updated unified-resource-architecture.md to mark Phase 4 as ABANDONED
- Removed unified-view-migration-plan.md
- Backend unified resource model remains for AI context

This is a checkpoint before attempting full frontend migration to unified model.
2025-12-07 22:54:27 +00:00
rcourtman
5f000b7974 feat: Complete Unified Resource Architecture (Phases 1-3)
This commit implements the Unified Resource Architecture for AI-first
infrastructure management. Key features:

Phase 1 - Backend Unification:
- New unified Resource type with 9 resource types, 7 platforms, 7 statuses
- Resource store with identity-based deduplication (hostname, machineID, IP)
- 8 converter functions (FromNode, FromVM, FromContainer, etc.)
- REST API endpoints: /api/resources, /api/resources/stats, /api/resources/{id}
- 28 comprehensive unit tests

Phase 2 - AI Context Enhancement:
- Unified context builder for AI system prompts
- Cross-platform query methods: GetTopByCPU, GetTopByMemory, GetTopByDisk
- Resource correlation: GetRelated (parent, children, siblings, cluster)
- Infrastructure summary: GetResourceSummary with health status counts
- AI context now includes top consumers and infrastructure overview

Phase 3 - Agent Preference & Hybrid Mode:
- Polling optimization methods in resource store
- ResourceStoreInterface added to Monitor
- SetResourceStore() and shouldSkipNodeMetrics() helper methods
- Store automatically wired into Monitor via Router.SetMonitor()
- Foundation ready for reduced API polling when agents are active

Files added:
- internal/resources/resource.go - Core Resource type
- internal/resources/store.go - Store with deduplication
- internal/resources/converters.go - Type converters
- internal/resources/platform_data.go - Platform-specific data
- internal/resources/store_test.go - 28 tests
- internal/resources/converters_test.go - Converter tests
- internal/api/resource_handlers.go - REST API handlers
- internal/ai/resource_context.go - AI context builder
- .gemini/docs/unified-resource-architecture.md - Architecture docs

All tests pass.
2025-12-07 13:49:00 +00:00
rcourtman
90c45968e7 AI Problem Solver implementation and various fixes
- Implement 'Show Problems Only' toggle combining degraded status, high CPU/memory alerts, and needs backup filters
- Add 'Investigate with AI' button to filter bar for problematic guests
- Fix dashboard column sizing inconsistencies between bars and sparklines view modes
- Fix PBS backups display and polling
- Refine AI prompt for general-purpose usage
- Fix frontend flickering and reload loops during initial load
- Integrate persistent SQLite metrics store with Monitor
- Fortify AI command routing with improved validation and logging
- Fix CSRF token handling for note deletion
- Debug and fix AI command execution issues
- Various AI reliability improvements and command safety enhancements
2025-12-06 23:46:08 +00:00
rcourtman
c9915ead8b feat: Add OS type display for LXC containers
- Extract ostype from LXC container config (debian, ubuntu, alpine, etc.)
- Map ostype values to human-readable names (e.g., "debian" -> "Debian")
- Add OSName field to Container model and ContainerFrontend
- Add icons for NixOS, openSUSE, and Gentoo in frontend
- LXC containers now show OS icons alongside VMs in the dashboard

Supported LXC OS types: alpine, archlinux, centos, debian, devuan,
fedora, gentoo, nixos, opensuse, ubuntu, unmanaged
2025-12-05 12:43:32 +00:00
rcourtman
cc3c0187a0 feat: AI features, agent improvements, and host monitoring enhancements
AI Chat Integration:
- Multi-provider support (Anthropic, OpenAI, Ollama)
- Streaming responses with markdown rendering
- Agent command execution for remote troubleshooting
- Context-aware conversations with host/container metadata

Agent Updates:
- Add --enable-proxmox flag for automatic PVE/PBS token setup
- Improve auto-update with semver comparison (prevents downgrades)
- Add updatedFrom tracking to report previous version after update
- Reduce initial update check delay from 30s to 5s
- Add agent version column to Hosts page table

Host Metrics:
- Add DiskIO stats collection (read/write bytes, ops, time)
- Improve disk filtering to exclude Docker overlay mounts
- Add RAID array monitoring via mdadm
- Enhanced temperature sensor parsing

Frontend:
- New Agent Version column on Hosts overview table
- Improved node modal with agent-first installation flow
- Add DiskIO display in host drawer
- Better responsive handling for metric bars
2025-12-05 10:37:02 +00:00
rcourtman
72af7bed5e style: Apply gofmt to 37 files
Standardize code formatting across test files and monitor.go.
No functional changes.
2025-12-02 17:21:48 +00:00
rcourtman
0bcaa41499 test: Fix unreachable code warning in panic recovery test
Refactor the test to avoid unreachable code after panic by
checking a flag set before the panic instead of after. This
resolves the go vet warning while maintaining test coverage.
2025-12-02 16:30:31 +00:00
rcourtman
6a29e20b5e perf: Cache err.Error() in storage timeout error handling
Cache err.Error() result in two locations:
- monitor.go: storage query retry logic (2x calls to 1)
- monitor_polling.go: storage timeout handling (2x calls to 1)
2025-12-02 15:39:37 +00:00
rcourtman
5d2eca3660 perf: Cache err.Error() in disk monitoring error handling
Compute err.Error() once and reuse errStr instead of calling Error()
four times when checking disk monitoring error types.
2025-12-02 15:37:48 +00:00
rcourtman
8f5b258424 perf: Cache lowercase error string in guest agent error handling
Compute strings.ToLower(errStr) once and reuse errStrLower instead of
calling ToLower three times on the same string in permission check.
2025-12-02 15:26:40 +00:00
rcourtman
b47a04aa97 perf: Use strconv.Itoa instead of fmt.Sprintf for int conversion
strconv.Itoa is faster than fmt.Sprintf("%d", ...) because it doesn't
need to parse a format string. Changed 4 occurrences in monitoring
package where integers are converted to strings.
2025-12-02 15:21:41 +00:00
rcourtman
ac4e48fa4a fix: Properly show disabled storage status from Proxmox
Storage that is disabled in Proxmox (Datacenter > Storage > enabled=no)
was incorrectly showing as "available" in Pulse. The issue was that
Enabled/Active fields defaulted to true and were never set to false
from the per-node API response.

Now the model correctly initializes Enabled/Active from the Proxmox
per-node storage API response, and the status determination prioritizes
checking the disabled state first.

Related to #796
2025-12-02 15:03:40 +00:00
rcourtman
c1d98e6aa8 refactor: Remove unreachable dead code branches
- errors.isRetryable: Convert final case to default (all ErrorType values covered)
- scheduler.SelectInterval: Remove bounds checks after mathematical computation
  that guarantees target ∈ [min, max]

Both functions now at 100% coverage.
2025-12-02 14:48:57 +00:00
rcourtman
2453198f62 refactor: Remove unreachable dead code branches
- firstForwardedValue: strings.Split always returns at least one element
- shouldRunBackupPoll: remaining is always >= 1 by math
- convertContainerDiskInfo: lowerLabel is never empty for non-rootfs

All three functions now at 100% coverage.
2025-12-02 14:41:53 +00:00
rcourtman
3f8455dfa9 test: Add DispatchDue enqueue error test for monitoring package 2025-12-02 14:21:25 +00:00
rcourtman
80739cce94 test: Add rescheduleTask tests for monitoring package
Add comprehensive tests for the rescheduleTask function covering:
- Nil taskQueue handling (early return)
- Successful task outcome (regular rescheduling)
- Transient failure with backoff
- Non-transient failure routing to dead letter queue
- Exceeded retry attempts routing to dead letter queue
- No outcome uses default interval
- PBS and PMG instance type intervals
- Adaptive polling max interval capping
- Existing interval preservation

Coverage: rescheduleTask 32.1% → 58.9%
Coverage: monitoring package 52.8% → 53.5%
2025-12-02 12:38:50 +00:00
rcourtman
fc96ba7b69 test: Add WaitNext and key() tests for TaskQueue
Tests cover context cancellation, immediately due tasks, waiting for
future tasks, empty queue timeout, and multiple task priority ordering.
WaitNext coverage 0%→92.3%, key() 0%→100%. Monitoring package 52.3%→52.9%.
2025-12-02 12:02:13 +00:00
rcourtman
e63c297ca3 test: Add BuildPlan, FilterDue, DispatchDue, LastScheduled tests
Comprehensive tests for AdaptiveScheduler methods covering empty inventory,
single/multiple instances, priority ordering, interval clamping, task caching,
nil receiver handling, and task filtering. Monitoring package 51.4%→52.3%.
2025-12-02 11:59:37 +00:00
rcourtman
ea120b78d7 test: Add RecordNodeResult, RecordQueueWait, SetQueueDepth tests
Improves metrics.go coverage to 100%. Added comprehensive tests for
node-level poll metrics, queue wait time recording, and queue depth
setting with edge cases (nil receiver, negative values, label normalization).
2025-12-02 11:57:05 +00:00
rcourtman
42e7359506 test: Add describeInstancesForScheduler tests
Add 6 tests for instance descriptor generation:
- No clients returns nil
- PVE only with sorted order
- PBS only
- PMG only
- All types combined
- Nil scheduler/stalenessTracker safety

Coverage: 60.7% → 62.5%
2025-12-02 02:11:14 +00:00
rcourtman
2d09c28129 test: Add recordTaskResult tests
Add 6 tests for polling task result recording:
- Nil monitor safety check
- Success case resets counters
- Failure case increments counters
- Consecutive failures tracking
- Success resets failure state
- Nil internal maps don't panic

Coverage: 53.7% → 97.6%
2025-12-02 02:06:08 +00:00
rcourtman
67d322c456 test: Add ApplyHostReport error path tests
Add 4 tests for error and edge cases:
- Missing hostname returns error
- Whitespace-only hostname returns error
- Nil hostTokenBindings map is initialized
- Fallback identifier generation

Coverage: 63.7% → 70.8%
2025-12-02 02:03:03 +00:00
rcourtman
7c94f3001e test: Add ApplyDockerReport error path tests
Add 4 tests for error conditions:
- Missing identifier (no agent ID or hostname)
- Removed host rejection
- Token bound to different agent
- Missing hostname

Coverage: 63.0% → 69.5%
2025-12-02 02:00:06 +00:00
rcourtman
781f72c953 test: Add RemoveHostAgent error path tests
Add 4 tests for uncovered branches:
- Empty/whitespace hostID returns error
- Host not found returns synthetic host
- No token binding (token exists but not bound)
- Nil alertManager doesn't panic

Coverage: 64.3% → 78.6%
2025-12-02 01:56:50 +00:00
rcourtman
e07bd834b6 Filter virtual/system filesystems from host disk display
Host disk bars were showing virtual filesystems like tmpfs, /dev, /run,
/sys, and Docker overlay mounts. These clutter the UI and don't represent
meaningful disk usage.

Changed from `shouldIgnoreReadOnlyFilesystem` (read-only only) to the
full `fsfilters.ShouldSkipFilesystem` which also excludes:
- Virtual FS types: tmpfs, devtmpfs, sysfs, proc, cgroup, etc.
- Special mountpoints: /dev, /proc, /sys, /run, /var/lib/docker, /snap
- Network filesystems: fuse, nfs, cifs, etc.

Related to #790
2025-12-02 00:16:39 +00:00
rcourtman
7c27a74c42 test: Add comprehensive tests for PBS snapshot conversion
Add 13 test cases for convertPBSSnapshots covering:
- Empty/nil input handling
- Basic snapshot field mapping
- Files as string array
- Files as object array with filename field
- Invalid file objects ignored gracefully
- Verification status as string ("ok" vs other)
- Verification status as object with state field
- Verification object without state field
- Multiple snapshots with unique IDs
- Empty namespace handling in ID generation
- Container (ct) backup type

Coverage: convertPBSSnapshots 0% -> 100%
2025-12-01 22:23:47 +00:00
rcourtman
b403cdd7b7 test: Add comprehensive tests for convertDockerTasks function
- Test nil input returns nil
- Test empty slice returns nil
- Test single task with all fields populated
- Test task with error state (failed tasks)
- Test multiple tasks conversion
- Test nil time pointers preserved as nil
- Test zero time values converted to nil (defensive handling)
- Test all optional time fields (UpdatedAt, StartedAt, CompletedAt)

Improves monitoring package coverage for Docker Swarm task handling.
2025-12-01 22:13:43 +00:00
rcourtman
cb683597c6 test: Add tests for GPU temperature parsing functions
- parseGPUTemps: AMD GPU edge/junction/mem temperatures, sensor mapping
- parseNouveauGPUTemps: NVIDIA nouveau GPU core temperature parsing
- Case insensitive sensor name matching
- Invalid/zero/negative temperature handling
- Append behavior for multiple GPUs
2025-12-01 21:56:33 +00:00
rcourtman
334cffb1c7 test: Add tests for monitor helper and metrics functions
- clearGuestMetadataCache: nil safety, cache clearing, key isolation
- shouldRunBackupPoll: interval-based, cycle-based, config validation
- storeNodeLastSuccess/lastNodeSuccessFor: store/retrieve, overwrite, missing keys

Improves coverage for backup polling logic and node metrics tracking.
2025-12-01 21:18:33 +00:00
rcourtman
8198e1d839 test: Add comprehensive tests for guest metadata functions
Tests for rate limiting, retry, and slot management:
- retryGuestAgentCall: timeout retry, non-timeout break, context cancellation
- acquireGuestMetadataSlot: nil safety, slot acquisition, context cancellation
- releaseGuestMetadataSlot: nil safety, non-blocking release
- tryReserveGuestMetadataFetch: reservation logic, default hold duration
- scheduleNextGuestMetadataFetch: interval, jitter, RNG handling
- deferGuestMetadataRetry: backoff logic, defaults

Covers error handling and edge cases in guest agent metadata fetching.
2025-12-01 21:10:28 +00:00
rcourtman
a3eb81778f test: Add table-driven tests for parseSensorsJSON
Comprehensive coverage for temperature sensor JSON parsing:
- Empty/whitespace input handling
- Invalid JSON error handling
- Legacy and wrapper format support
- Multiple chip types: coretemp, k10temp, acpitz, nvme, amdgpu, nouveau, zenpower, nct6795, it87, rp1_adc
- Edge cases: non-map chip data, CPUMax calculation from cores
2025-12-01 21:00:46 +00:00
rcourtman
1af38e3f99 test: Add tests for parseContainerMountMetadata, convertContainerDiskInfo, StalenessScore
- parseContainerMountMetadata: parts without equals, mountpoint key variant,
  whitespace handling, single source value (96.4% -> 100%)
- convertContainerDiskInfo: nil metadata, device from metadata, negative free
  clamping, whitespace label handling (95.1% -> 97.6%, remaining is dead code)
- StalenessScore: negative maxStale default, metrics lookup failure, score
  clamping edge cases (95.7% - remaining is defensive dead code)
2025-12-01 20:44:00 +00:00
rcourtman
ed0644035f test: Add tests for sanitizeGuestAddressStrings, parseContainerConfigNetworks, mergeNVMeTempsIntoDisks
- sanitizeGuestAddressStrings: IPv6 loopback with CIDR (::1/128) edge case
  (96.2% -> 100%)
- parseContainerConfigNetworks: parts without equals, mac/ips/ip6addr/ip6prefix
  key variants, whitespace-only values, all-empty results (97.2% -> 100%)
- mergeNVMeTempsIntoDisks: NVMe disk with no legacy temps (continue branch),
  more disks than temps (break branch) (97% -> 100%)
2025-12-01 20:34:48 +00:00
rcourtman
4e33b396e0 test: Add tests for normalizeEndpointHost, SelectInterval, generateDockerHostIdentifier
- normalizeEndpointHost: port-only URL host (parsed.Host fallback),
  edge cases for URL parsing (94.4% -> 100%)
- SelectInterval: negative penalty, instance key derivation, clamping
  edge cases (96% - remaining is unreachable dead code guards)
- generateDockerHostIdentifier: empty base fallbacks, suffix candidates,
  hash suffix, numeric suffix increments (91.7% -> 100%)
2025-12-01 20:26:21 +00:00
rcourtman
4c91dce2f8 test: Add tests for allow, ensureContainerRootDiskEntry; remove dead code
- allow (circuitBreaker): all states including unknown/default branch,
  open→half-open transition, half-open window timing (93.8% -> 100%)
- ensureContainerRootDiskEntry: nil container, existing disks, used>total
  clamping, negative free clamping, usage calculation (92.3% -> 100%)
- convertPoolInfoToModel: removed unreachable nil check (dead code since
  ConvertToModelZFSPool only returns nil for nil receiver) (88.9% -> 100%)
2025-12-01 20:15:32 +00:00
rcourtman
0a9177f161 test: Add tests for lookupClusterEndpointLabel, GetNodeMetrics, stateDetails
- lookupClusterEndpointLabel: nil instance, no match, Host vs IP fallback,
  case-insensitive NodeName, whitespace handling (84.6% -> 100%)
- GetNodeMetrics: unknown metric types, empty nodeID, zero duration,
  empty metrics data, disk type (94.1% -> 100%)
- stateDetails: all breaker states including unknown/invalid state,
  retryAt calculations, timestamps (90% -> 100%)
2025-12-01 20:07:22 +00:00
rcourtman
2fdc051c11 fix: Make parseNVMeTemps deterministic by checking Composite before Sensor 1
Go map iteration order is non-deterministic, causing flaky test failures.
This fix ensures "Composite" sensor is always preferred over "Sensor 1"
by checking them in separate loops rather than relying on iteration order.
2025-12-01 20:03:05 +00:00
rcourtman
f79d20296b test: Add tests for GetGuestMetrics, mergeSnapshot, getDockerCommandPayload
- GetGuestMetrics: all metric types, duration filtering, guest not found,
  unknown metric type, empty data (81% -> 100%)
- mergeSnapshot: timestamp merging for LastSuccess/LastError/LastMutated,
  ChangeHash preservation (86.7% -> 100%)
- getDockerCommandPayload: empty hostID, whitespace normalization, host
  not found, expired command cleanup, queued->dispatched marking,
  already-dispatched preservation (91.3% -> 100%)
2025-12-01 19:58:43 +00:00
rcourtman
196389853e test: Add tests for StalenessScore, parseCPUTemps, queueDockerStopCommand
- StalenessScore: 78.3%→95.7% (4 cases for metrics, future timestamp, defaults)
- parseCPUTemps: 98.1%→100% (core temp exceeding package case)
- queueDockerStopCommand: 76.9%→100% (12 cases for validation, status checks)
2025-12-01 19:50:42 +00:00
rcourtman
2594a6c124 test: Add tests for handleProxyHostFailure, recordNodeSnapshot, evaluateHostAgents
- handleProxyHostFailure: 76.5%→100% (9 cases for per-host failure tracking)
- recordNodeSnapshot: 75%→100% (6 cases for diagnostic snapshot storage)
- evaluateHostAgents: 87%→100% (10 cases for host health evaluation)
2025-12-01 19:40:32 +00:00
rcourtman
c6a34c4412 test: Add tests for handleProxyFailure, parseRPiTemperature, RecordResult
- handleProxyFailure: 86.7%→100% (6 cases for proxy disable threshold logic)
- parseRPiTemperature: 77.8%→100% (7 cases for RPi temp parsing edge cases)
- RecordResult: 95.8%→100% (negative staleness clamping case)
2025-12-01 19:33:06 +00:00
rcourtman
1f380eaaf3 test: Add tests for isProxyEnabled, taskHeap.Pop, NewAdaptiveScheduler
- isProxyEnabled: 94.4%→100% (7 cases for proxy cooldown/restore logic)
- taskHeap.Pop: 87.5%→100% (3 cases for empty/single/multi-element heap)
- NewAdaptiveScheduler: 84.6%→100% (14 cases for default value handling)
2025-12-01 19:26:37 +00:00
rcourtman
0ed84f3179 test: Add tests for disableLegacySSHOnAuthFailure, parseNVMeTemps, updateDeadLetterMetrics
- disableLegacySSHOnAuthFailure: 87.5%→100% (13 cases for auth error detection)
- parseNVMeTemps: 88.9%→100% (16 cases for NVMe temp parsing)
- updateDeadLetterMetrics: 75%→100% (6 cases for nil handling, queue states)
2025-12-01 19:20:09 +00:00
rcourtman
7bab2f86af test: Add tests for AllowDockerHostReenroll, SelectInterval, ensureBreaker
- AllowDockerHostReenroll: 95%→100% (7 cases for empty/blocked/cleanup paths)
- ensureBreaker: 94.4%→100% (6 cases for nil map, existing, custom config)
- SelectInterval: 94%→96% (20+ cases for edge cases, remaining guards unreachable)
2025-12-01 19:13:40 +00:00
rcourtman
e7bc06033a test: Add tests for convertDockerSwarmInfo, namespacePathsForDatastore, preserveFailedStorageBackups
- convertDockerSwarmInfo: 66.7%→100% (4 cases for nil, empty, populated structs)
- namespacePathsForDatastore: 92.3%→100% (removed unreachable dead code)
- preserveFailedStorageBackups: 91.3%→100% (6 cases for filtering, deduplication)
2025-12-01 19:04:23 +00:00
rcourtman
0bfc9ed447 test: Add tests for storageNamesForNode, GetDockerHost, and SchedulerHealth edge cases
- storageNamesForNode: 83.3%→100% (8 cases for filtering logic)
- GetDockerHost: 75%→100% (5 cases for lookup and whitespace handling)
- SchedulerHealth: 85.3%→89.1% (13 cases for nil handling, breaker key parsing)
2025-12-01 18:57:09 +00:00