Commit graph

1616 commits

Author SHA1 Message Date
rcourtman
2def8dee09 chore: remove internal development documentation
These were internal planning/architecture docs not meant for end users:
- .gemini/docs/unified-resource-architecture.md (design doc)
- .gemini/tasks/persistent-metrics-storage.md (implementation plan)
- frontend-modern/PLAN-column-visibility.md (implementation plan)
2025-12-08 09:27:21 +00:00
rcourtman
f7eed7b052 cleanup: remove legacy AI context fallback (buildInfrastructureContext)
The AI service now uses only buildUnifiedResourceContext() for
infrastructure context, since the resourceProvider is always set
during router initialization.

Removed:
- buildInfrastructureContext() function (~288 lines of dead code)
- Legacy fallback path in buildSystemPrompt()

The unified resource context provides a cleaner, deduplicated view
of infrastructure that includes:
- All resources grouped by platform and type
- Top CPU/Memory/Disk consumers
- Active alerts on resources
- Infrastructure summary statistics

This completes the AI service migration to unified resources.
2025-12-08 09:21:03 +00:00
rcourtman
7e8611e0ae cleanup: consolidate unified resources adapters into useResourcesAsLegacy hook
This cleanup addresses transition debt from the unified resources migration:

Frontend cleanup:
- Move all Resource→Legacy type conversions to useResourcesAsLegacy() hook
- Add asNodes() and asDockerHosts() adapter functions to the hook
- Simplify DockerRoute, HostsRoute, DashboardView to use the centralized hook
- Remove ~300 lines of duplicated adapter code from App.tsx
- Remove debug console.log statements from Dashboard.tsx
- Fix CPU value conversion (divide by 100) for Dashboard compatibility

Backend fixes (from previous session):
- Fix parentID format in converters (VM, Container, Storage) to match Node.ID
- Format changed from 'instance/node/nodename' to 'instance-nodename'
- Update tests to match new parentID format

This consolidates all legacy type conversion logic in one place,
making future cleanup easier when components are migrated to use
unified resources directly.
2025-12-08 09:16:26 +00:00
rcourtman
f003ad7239 fix: use platformData.node/instance for correct guest grouping
The Dashboard grouping was broken because:
- node was set to r.parentId (full resource ID)
- instance was set to r.platformId

Fixed to read from platformData which contains the correct values:
- node = platformData.node (e.g., 'minipc')
- instance = platformData.instance (e.g., 'https://pve:8006')

This matches the legacy data format and fixes the grouped/list toggle.
2025-12-07 23:59:02 +00:00
rcourtman
ef135126b9 remove legacy fallbacks: routes now exclusively use unified resources
BREAKING: Route components no longer fall back to legacy state arrays.
All data now flows through the unified resource model:

- DockerRoute: uses state.resources filtered for docker-host/docker-container
- HostsRoute: uses state.resources filtered for host
- DashboardView: uses state.resources filtered for node/vm/container

The legacy arrays (state.nodes, state.vms, etc.) are still broadcast
by the backend for API compatibility, but the main UI routes no longer
use them.

If resources array is empty, pages will show no data rather than
falling back to legacy data. This ensures a clean data model with
no hidden fallback behavior.
2025-12-07 23:56:35 +00:00
rcourtman
435e0af522 cleanup: remove debug console.log statements from route components
Phase 6 partial cleanup:
- Removed [DockerRoute], [HostsRoute], [DashboardView] console logs
- Updated architecture doc to reflect completed cleanup

Remaining Phase 6 tasks (future):
- Remove legacy arrays from State (keep fallback for safety)
- Remove legacy AI context fallback
- Simplify route components to use resources directly
2025-12-07 23:54:21 +00:00
rcourtman
cc4b32ad4a fix: complete unified resources WebSocket integration
Backend:
- Call SetMonitor after router creation to inject resource store
- Add debug logging for resource population and broadcast

Frontend:
- Add resources array to WebSocket store initial state
- Handle resources in WebSocket message processing
- Use reconcile for efficient state updates

The unified resources are now properly:
1. Populated from StateSnapshot on each broadcast cycle
2. Converted to frontend format (ResourceFrontend)
3. Included in WebSocket state messages
4. Received and stored in frontend state
5. Consumed by migrated route components

Console now shows '[DashboardView] Using unified resources: VMs: X'
confirming the migration is working end-to-end.
2025-12-07 23:52:00 +00:00
rcourtman
a10c13dfe9 test: add PopulateFromSnapshot test to verify resource conversion
Test confirms that StateSnapshot data is correctly converted to unified
resources. Validates conversion for nodes, VMs, containers, and hosts.
2025-12-07 23:38:15 +00:00
rcourtman
9a1458700a debug: add console logging to verify unified resources usage
Temporary logging to verify which code path is being used:
- DockerRoute: logs docker-host and docker-container counts
- HostsRoute: logs host count
- DashboardView: logs VM count

Check browser console to confirm unified resources are being received.
These logs can be removed once migration is verified.
2025-12-07 23:34:21 +00:00
rcourtman
6cfcc47c71 docs: update unified-resource-architecture with migration status
- Documented WebSocket state migration as completed
- Listed all pages migrated to unified resources
- Outlined future Phase 6 cleanup tasks
- Clarified the strategic shift from dedicated /resources view to
  migrating existing pages
2025-12-07 23:22:30 +00:00
rcourtman
f81c04915f phase 4-5: migrate Docker and Dashboard pages to unified resources
Docker page:
- DockerRoute now uses unified resources with fallback to legacy data
- Reconstructs container hierarchy from flat resource list
- Maps docker-host and docker-container resources to DockerHost type

Dashboard page:
- DashboardView now uses unified resources with fallback
- Converts vm, container, and node resources to legacy types
- Maintains full backward compatibility with existing components

Both pages use resource type filtering and platform data extraction
to adapt the unified model to existing component interfaces.
2025-12-07 23:21:40 +00:00
rcourtman
9f93fb1f88 phase 3: migrate Hosts page to use unified resources
- Updated HostsRoute to consume unified resources with fallback to legacy data
- Added asHosts adapter to useResourcesAsLegacy hook
- Adapter converts Resource type to Host type for existing component

The Hosts page now uses resources from state.resources when available,
falling back to state.hosts for backward compatibility. This approach
allows gradual migration without breaking the existing HostsOverview
component.
2025-12-07 23:17:21 +00:00
rcourtman
881f00a5d9 ensure resource store is populated before WebSocket broadcasts
- Added PopulateFromSnapshot method to resources.Store
- Extended ResourceStoreInterface to include PopulateFromSnapshot
- Monitor now calls updateResourceStore before broadcasts
- This ensures resources are fresh on every WebSocket broadcast

Without this, the store would only be populated when /api/resources or
/api/state endpoints are hit, leaving WebSocket broadcasts empty.
2025-12-07 23:15:02 +00:00
rcourtman
55f10ce36c phase 2: frontend types and useResources hook
- Created resource.ts with TypeScript types for unified Resource model
  - ResourceType, PlatformType, SourceType, ResourceStatus enums
  - Resource interface matching backend ResourceFrontend
  - Helper functions: isInfrastructure, isWorkload, getDisplayName, etc.
  - ResourceFilter interface for complex filtering

- Updated api.ts State interface to include optional resources array

- Created useResources hook for accessing unified resources
  - Reactive access via getGlobalWebSocketStore
  - Pre-computed memos for infra, workloads, statusCounts
  - Filtering methods: byType, byPlatform, filtered
  - Query helpers: get, children, topByCpu, topByMemory

- Created useResourcesAsLegacy helper for migration
  - Converts resources to legacy VM/Container formats
  - Enables gradual component migration

This provides the foundation for migrating frontend pages to use
the unified resource model.
2025-12-07 23:07:45 +00:00
rcourtman
cdd2249e00 phase 1: add unified resources to WebSocket state broadcasts
- Extended StateFrontend with Resources field containing unified resource data
- Added ResourceFrontend and related types for frontend-compatible resource data
- Extended ResourceStoreInterface to include GetAll() method
- Monitor now injects resources into WebSocket broadcasts
- Added helper method getResourcesForBroadcast() to convert resources to frontend format
- All existing tests pass

This enables the frontend to access unified resources via WebSocket state.
2025-12-07 23:02:56 +00:00
rcourtman
ef35120afb checkpoint: remove unified resources UI, keep backend model
- Removed /resources page and associated frontend components
- Removed ResourcesOverview.tsx, UnifiedResourceRow.tsx, columns.ts
- Removed frontend types/resource.ts
- Updated unified-resource-architecture.md to mark Phase 4 as ABANDONED
- Removed unified-view-migration-plan.md
- Backend unified resource model remains for AI context

This is a checkpoint before attempting full frontend migration to unified model.
2025-12-07 22:54:27 +00:00
rcourtman
cac109b497 fix: Populate resources on-demand when /api/resources is called
The Resources page was showing 0 resources because the store was only
populated when /api/state was called (from the dashboard). Now the
resources are populated on-demand when /api/resources is accessed.

Changes:
- Added StateProvider interface to ResourceHandlers
- SetStateProvider() method for injecting the monitor
- HandleGetResources now calls PopulateFromSnapshot before querying
- Router injects monitor as state provider during SetMonitor()

This ensures the /resources page works even when accessed directly
without visiting the main dashboard first.
2025-12-07 14:47:29 +00:00
rcourtman
b99ba78c6e feat(ui): Add unified Resources view (Phase 4)
This implements Phase 4 of the Unified Resource Architecture - the frontend
unified resources view.

New Features:
- Unified resources page at /resources route
- Fetches from /api/resources REST endpoint
- Auto-refreshes every 10 seconds
- Filtering by search, type, platform, status
- Grouping by type, platform, or parent
- Status indicators with alert badges
- CPU/Memory/Disk progress bars

Files Added:
- frontend-modern/src/types/resource.ts - TypeScript types matching Go backend
- frontend-modern/src/components/Resources/ResourcesOverview.tsx - Main component

Files Modified:
- frontend-modern/src/App.tsx - Added lazy import and route for ResourcesOverview
- .gemini/docs/unified-resource-architecture.md - Updated Phase 4 status

Access the unified view by navigating to /resources directly.
The route is not yet in the main navigation (power user feature).
2025-12-07 13:55:35 +00:00
rcourtman
5f000b7974 feat: Complete Unified Resource Architecture (Phases 1-3)
This commit implements the Unified Resource Architecture for AI-first
infrastructure management. Key features:

Phase 1 - Backend Unification:
- New unified Resource type with 9 resource types, 7 platforms, 7 statuses
- Resource store with identity-based deduplication (hostname, machineID, IP)
- 8 converter functions (FromNode, FromVM, FromContainer, etc.)
- REST API endpoints: /api/resources, /api/resources/stats, /api/resources/{id}
- 28 comprehensive unit tests

Phase 2 - AI Context Enhancement:
- Unified context builder for AI system prompts
- Cross-platform query methods: GetTopByCPU, GetTopByMemory, GetTopByDisk
- Resource correlation: GetRelated (parent, children, siblings, cluster)
- Infrastructure summary: GetResourceSummary with health status counts
- AI context now includes top consumers and infrastructure overview

Phase 3 - Agent Preference & Hybrid Mode:
- Polling optimization methods in resource store
- ResourceStoreInterface added to Monitor
- SetResourceStore() and shouldSkipNodeMetrics() helper methods
- Store automatically wired into Monitor via Router.SetMonitor()
- Foundation ready for reduced API polling when agents are active

Files added:
- internal/resources/resource.go - Core Resource type
- internal/resources/store.go - Store with deduplication
- internal/resources/converters.go - Type converters
- internal/resources/platform_data.go - Platform-specific data
- internal/resources/store_test.go - 28 tests
- internal/resources/converters_test.go - Converter tests
- internal/api/resource_handlers.go - REST API handlers
- internal/ai/resource_context.go - AI context builder
- .gemini/docs/unified-resource-architecture.md - Architecture docs

All tests pass.
2025-12-07 13:49:00 +00:00
rcourtman
190e02fbbc Add host alert deduplication with tests
- Modified alerts.go for improved host alert handling
- Added host_dedup_test.go for deduplication test coverage
2025-12-07 12:38:38 +00:00
rcourtman
721f973271 AI features checkpoint: Host selection, memory sparklines, UI refinements
- Extended AI context selection to host rows in HostsOverview
- Added resourceId prop to StackedMemoryBar for sparkline support
- Relocated guest URL editing from GuestRow name click
- Added GuestNotes component with URL field in AI sidebar
- Refined host routing in AI service backend
- Minor animation and styling improvements
2025-12-07 12:25:26 +00:00
rcourtman
f2e6927436 Additional updates 2025-12-07 10:22:42 +00:00
rcourtman
6988eb746a Frontend updates 2025-12-07 00:49:32 +00:00
rcourtman
c0da4768bb Metric bar and CPU bar improvements 2025-12-07 00:27:47 +00:00
rcourtman
fc8d88a7c7 Dashboard and sparkline improvements 2025-12-07 00:08:52 +00:00
rcourtman
90c45968e7 AI Problem Solver implementation and various fixes
- Implement 'Show Problems Only' toggle combining degraded status, high CPU/memory alerts, and needs backup filters
- Add 'Investigate with AI' button to filter bar for problematic guests
- Fix dashboard column sizing inconsistencies between bars and sparklines view modes
- Fix PBS backups display and polling
- Refine AI prompt for general-purpose usage
- Fix frontend flickering and reload loops during initial load
- Integrate persistent SQLite metrics store with Monitor
- Fortify AI command routing with improved validation and logging
- Fix CSRF token handling for note deletion
- Debug and fix AI command execution issues
- Various AI reliability improvements and command safety enhancements
2025-12-06 23:46:08 +00:00
rcourtman
469307d607 refactor: Use text labels for OS column instead of icons
Simplified OS display to plain "Windows" and "Linux" text labels.
Previous icon attempts were rejected as too complex or unclear.
Text labels are cleaner and more universally recognizable.
2025-12-05 12:58:18 +00:00
rcourtman
c9915ead8b feat: Add OS type display for LXC containers
- Extract ostype from LXC container config (debian, ubuntu, alpine, etc.)
- Map ostype values to human-readable names (e.g., "debian" -> "Debian")
- Add OSName field to Container model and ContainerFrontend
- Add icons for NixOS, openSUSE, and Gentoo in frontend
- LXC containers now show OS icons alongside VMs in the dashboard

Supported LXC OS types: alpine, archlinux, centos, debian, devuan,
fedora, gentoo, nixos, opensuse, ubuntu, unmanaged
2025-12-05 12:43:32 +00:00
rcourtman
41acb4f2ce feat: Replace OS column text with icons and rich tooltips
- Add OSInfoCell component with OS-specific icons (Windows, Ubuntu,
  Debian, Alpine, CentOS/RHEL, Fedora, Arch, FreeBSD, generic Linux)
- Each OS type has a distinct color for quick visual identification
- Portal tooltip shows full OS name, version, and guest agent version
- Much more compact than text strings like "Microsoft Windows Server 2022"
2025-12-05 12:34:52 +00:00
rcourtman
c169ead4d2 fix: Correct column render order (Tags before OS) 2025-12-05 12:23:24 +00:00
rcourtman
871a714a3e fix: Move OS column to detailed tier (xl+ screens)
OS info requires guest agent to be installed and configured, so most
guests won't have this data. Move to detailed tier so it only shows
on extra-wide screens or when explicitly enabled by user.
2025-12-05 12:21:43 +00:00
rcourtman
3b38578bce feat: Add icons to backup status column with rich tooltip
- Add checkmark icon for fresh backups
- Add warning triangle for stale backups
- Add X icon for critical/never backups
- Use consistent Portal-based tooltip matching other columns
- Show formatted date, time, and relative age in tooltip
2025-12-05 12:16:43 +00:00
rcourtman
fcb258ac2d wip: AI provider improvements and chat store enhancements
- Improve Anthropic provider error handling
- Add AI service enhancements
- Update AI chat store with additional state management
2025-12-05 12:14:00 +00:00
rcourtman
a15dc6b899 feat: Toggleable table columns with rich tooltips for Proxmox dashboard
Replace drawer-based info display with inline columns that can be toggled:
- Add IP, Uptime, Node, Backup, OS, Tags columns (user-toggleable)
- Add ColumnPicker dropdown to show/hide columns with localStorage persistence
- Columns auto-show based on screen width using priority system
- Remove GuestDrawer - all info now visible inline or via tooltips

Rich hover tooltips:
- Disk bar: Shows all mount points with usage %, color-coded by severity
- Memory bar: Shows used/free/balloon/swap breakdown
- IP column: Shows network icon + count, hover for interfaces, MACs, IPs, traffic

Also:
- Create useColumnVisibility hook for responsive column management
- Create ColumnPicker component for column toggle UI
- Update drawer layouts in Hosts/Docker tabs for consistency
2025-12-05 12:12:56 +00:00
rcourtman
309302a793 fix: StatusDot not updating reactively when VM status changes
The StatusDot component was computing variant, size, and className once
at mount time, not reactively. When a VM transitioned from stopped to
running, the tooltip updated (it accessed props.title directly) but the
dot color stayed red because className was stale.

Fix: Convert plain variable assignments to getter functions that access
props reactively, and call them in the JSX template.
2025-12-05 10:40:03 +00:00
rcourtman
cc3c0187a0 feat: AI features, agent improvements, and host monitoring enhancements
AI Chat Integration:
- Multi-provider support (Anthropic, OpenAI, Ollama)
- Streaming responses with markdown rendering
- Agent command execution for remote troubleshooting
- Context-aware conversations with host/container metadata

Agent Updates:
- Add --enable-proxmox flag for automatic PVE/PBS token setup
- Improve auto-update with semver comparison (prevents downgrades)
- Add updatedFrom tracking to report previous version after update
- Reduce initial update check delay from 30s to 5s
- Add agent version column to Hosts page table

Host Metrics:
- Add DiskIO stats collection (read/write bytes, ops, time)
- Improve disk filtering to exclude Docker overlay mounts
- Add RAID array monitoring via mdadm
- Enhanced temperature sensor parsing

Frontend:
- New Agent Version column on Hosts overview table
- Improved node modal with agent-first installation flow
- Add DiskIO display in host drawer
- Better responsive handling for metric bars
2025-12-05 10:37:02 +00:00
rcourtman
3f08bfe8d0 wip: AI chat integration with multi-provider support
- Add AI service with Anthropic, OpenAI, and Ollama providers
- Add AI chat UI component with streaming responses
- Add AI settings page for configuration
- Add agent exec framework for command execution
- Add API endpoints for AI chat and configuration
2025-12-04 20:16:53 +00:00
rcourtman
71e3ab4c58 fix: Force tcp4 network for IPv4 addresses in sensor-proxy HTTP mode
On dual-stack systems with net.ipv6.bindv6only=1 (like some Proxmox 8
configurations), Go's net.Listen("tcp", "0.0.0.0:8443") may still bind
to IPv6-only. This caused IPv4 localhost connections to hang while
IPv6 worked.

Fix by detecting IPv4 addresses and explicitly using "tcp4" network
type when creating the listener. Related to #805
2025-12-04 20:09:37 +00:00
rcourtman
71a43c002d docs: Update Home Assistant addon link to new repo
The maintainer consolidated both agent and server addons into a single
repository at homeassistant-addons.
2025-12-04 03:04:52 +00:00
rcourtman
29e4f54243 fix: Exclude TrueNAS Docker overlay mounts from disk stats
Host agent was including Docker overlay2 mounts from TrueNAS SCALE's
.ix-apps directory in disk totals. These mounts inherit the ZFS pool's
AVAIL space, causing massively inflated storage numbers (e.g., 173 TB
per container overlay instead of actual usage).

Changes:
- Add /mnt/.ix-apps/docker/ to container overlay path exclusions
- Use ShouldSkipFilesystem() in host agent disk collection (was only
  using ShouldIgnoreReadOnlyFilesystem() which missed container paths)
- Add test cases for TrueNAS overlay paths

Related to #718
2025-12-04 03:03:04 +00:00
rcourtman
284afbbda1 chore: bump version to 4.36.2 2025-12-03 22:26:52 +00:00
rcourtman
db9955e187 fix: TrueNAS SCALE 24.04+ has read-only /usr/local/bin
On TrueNAS SCALE 24.04+, the root filesystem including /usr/local/bin
is read-only. The installer now tries multiple locations for the
runtime binary:

1. Execute directly from /data (if no noexec mount)
2. /usr/local/bin (older TrueNAS versions)
3. /root/bin (TrueNAS SCALE 24.04+)
4. /var/tmp (last resort)

The bootstrap script is also updated to use the determined runtime
location rather than hardcoding /usr/local/bin.

Related to #801
2025-12-03 21:02:55 +00:00
rcourtman
95ca0c0e67 fix: Default sensor-proxy HTTP to 0.0.0.0:8443 for IPv4 binding
On systems with net.ipv6.bindv6only=1 (including some Proxmox 8
configurations), using ":8443" results in IPv6-only binding. Users
reported curl to 127.0.0.1:8443 hanging while [::1]:8443 worked.

Changed default from ":8443" to "0.0.0.0:8443" to explicitly bind IPv4.

Related to #805
2025-12-03 20:25:08 +00:00
rcourtman
bfdc2cfcfe docs: Add Community Integrations section with Home Assistant addon
Added link to Kosztyk's community-maintained Home Assistant addon for
running the Pulse Docker Agent.

Related to discussion #807
2025-12-03 20:18:49 +00:00
rcourtman
7535f1f56d fix: Handle --http-addr with bind address in sensor-proxy installer
When using --http-addr 0.0.0.0:8443 (to bind to IPv4 only), the URL
construction was broken, producing URLs like https://192.168.31.110.0.0.0:8443

Now correctly extracts the port number from both ":8443" and "0.0.0.0:8443"
formats using ${HTTP_ADDR##*:} instead of ${HTTP_ADDR#:}

Related to #805
2025-12-03 20:16:30 +00:00
rcourtman
86d303b94a fix: TrueNAS agent binary now runs from /usr/local/bin to avoid noexec
TrueNAS SCALE's /data partition may have exec=off, preventing binaries
from executing. The installer now:
- Stores the binary in /data/pulse-agent/ for persistence
- Copies it to /usr/local/bin (tmpfs, allows exec) for runtime
- Updates the bootstrap script to copy on each boot

Related to #801
2025-12-03 20:14:48 +00:00
rcourtman
bc501df4e8 fix: Improve TrueNAS detection for immutable filesystem installs
Added fallback detection for TrueNAS systems that may not have
/etc/truenas-version or other standard markers:

1. Check if hostname contains "truenas" (common default hostname)
2. Test if /usr/local/bin is actually writable - if not and /data
   exists, use TrueNAS installation paths

This fixes installations on TrueNAS systems where the standard
detection files are missing but the filesystem is still immutable.

Related to #801
2025-12-03 18:04:10 +00:00
rcourtman
d195c8357e fix: Use specific checkbox selector in UnifiedAgents test
The test was using getByRole('checkbox') which now matches multiple
elements after adding the "Skip certificate verification" checkbox.
Use name matcher to select the specific Docker monitoring checkbox.
2025-12-03 14:32:05 +00:00
rcourtman
4585db8a9d feat: Add "Skip certificate verification" option for agent install commands
Adds a checkbox in Settings → Host Agents that enables insecure mode for
users running Pulse behind self-signed HTTPS certificates.

When enabled:
- Adds -k flag to curl commands for downloading the install script
- Adds --insecure flag to the agent for connecting back to Pulse

Related to #806
2025-12-03 14:15:17 +00:00
rcourtman
d3f22f06bf fix: Truncate very long container names in Docker table
Add max-width constraint to the resource column in the Docker table to
ensure very long container names (like Kubernetes UUID-based names) are
properly truncated instead of extending the table width.

Related to #789
2025-12-03 14:09:16 +00:00