Commit graph

91 commits

Author SHA1 Message Date
rcourtman
d2330cf405 refactor(ai): Remove over-engineered URL discovery service
Keep only the simple AI-powered approach:
- set_resource_url tool lets AI save discovered URLs
- Users ask AI directly: 'Find URLs for my containers'
- AI uses its intelligence to discover and set URLs

Removed:
- URLDiscoveryService (rigid port scanning)
- Bulk discovery API endpoints
- Frontend discovery button

The AI itself is smart enough to iterate through resources
and discover URLs when asked.
2025-12-10 08:35:24 +00:00
rcourtman
0ae4daca8d feat(ai): Add bulk URL discovery service
- Add URLDiscoveryService for scanning all resources at once
- Scans common web ports (80, 443, 8080, 8096, 3000, etc.)
- Automatically saves discovered URLs to resource metadata
- Add API endpoints for start/status/cancel discovery
- Progress tracking with results reporting

Endpoints:
- POST /api/ai/discover-urls/start - Start bulk discovery
- GET /api/ai/discover-urls/status - Check progress
- POST /api/ai/discover-urls/cancel - Cancel discovery
2025-12-10 08:30:56 +00:00
rcourtman
fac2bf91d9 feat(ai): Add URL discovery tool - AI can find and set resource URLs
- Add MetadataProvider interface for AI to update resource URLs
- Add set_resource_url tool to AI service
- Wire up metadata stores to AI service via router
- Add URL discovery guidance to AI system prompt
- AI can now inspect guests/containers/hosts for web services
  and automatically save discovered URLs to Pulse metadata

Usage: Ask the AI 'Find the web URL for this container' and it will:
1. Check for listening ports and web servers
2. Get the IP address
3. Verify the URL works
4. Save it to Pulse for quick dashboard access
2025-12-10 00:29:07 +00:00
rcourtman
a6656381c8 Add AI monitoring enhancements and host metadata features
- Add host metadata API for custom URL editing on hosts page
- Enhance AI routing with unified resource provider lookup
- Add encryption key watcher script for debugging key issues
- Improve AI service with better command timeout handling
- Update dev environment workflow with key monitoring docs
- Fix resource store deduplication logic
2025-12-09 16:27:46 +00:00
rcourtman
67351bf58c feat: AI integration, Docker metrics, RAID display, and infrastructure improvements
- Add Claude OAuth authentication support with hybrid API key/OAuth flow
- Implement Docker container historical metrics in backend and charts API
- Add CEPH cluster data collection and new Ceph page
- Enhance RAID status display with detailed tooltips and visual indicators
- Fix host deduplication logic with Docker bridge IP filtering
- Fix NVMe temperature collection in host agent
- Add comprehensive test coverage for new features
- Improve frontend sparklines and metrics history handling
- Fix navigation issues and frontend reload loops
2025-12-09 09:29:27 +00:00
rcourtman
cc4b32ad4a fix: complete unified resources WebSocket integration
Backend:
- Call SetMonitor after router creation to inject resource store
- Add debug logging for resource population and broadcast

Frontend:
- Add resources array to WebSocket store initial state
- Handle resources in WebSocket message processing
- Use reconcile for efficient state updates

The unified resources are now properly:
1. Populated from StateSnapshot on each broadcast cycle
2. Converted to frontend format (ResourceFrontend)
3. Included in WebSocket state messages
4. Received and stored in frontend state
5. Consumed by migrated route components

Console now shows '[DashboardView] Using unified resources: VMs: X'
confirming the migration is working end-to-end.
2025-12-07 23:52:00 +00:00
rcourtman
cac109b497 fix: Populate resources on-demand when /api/resources is called
The Resources page was showing 0 resources because the store was only
populated when /api/state was called (from the dashboard). Now the
resources are populated on-demand when /api/resources is accessed.

Changes:
- Added StateProvider interface to ResourceHandlers
- SetStateProvider() method for injecting the monitor
- HandleGetResources now calls PopulateFromSnapshot before querying
- Router injects monitor as state provider during SetMonitor()

This ensures the /resources page works even when accessed directly
without visiting the main dashboard first.
2025-12-07 14:47:29 +00:00
rcourtman
5f000b7974 feat: Complete Unified Resource Architecture (Phases 1-3)
This commit implements the Unified Resource Architecture for AI-first
infrastructure management. Key features:

Phase 1 - Backend Unification:
- New unified Resource type with 9 resource types, 7 platforms, 7 statuses
- Resource store with identity-based deduplication (hostname, machineID, IP)
- 8 converter functions (FromNode, FromVM, FromContainer, etc.)
- REST API endpoints: /api/resources, /api/resources/stats, /api/resources/{id}
- 28 comprehensive unit tests

Phase 2 - AI Context Enhancement:
- Unified context builder for AI system prompts
- Cross-platform query methods: GetTopByCPU, GetTopByMemory, GetTopByDisk
- Resource correlation: GetRelated (parent, children, siblings, cluster)
- Infrastructure summary: GetResourceSummary with health status counts
- AI context now includes top consumers and infrastructure overview

Phase 3 - Agent Preference & Hybrid Mode:
- Polling optimization methods in resource store
- ResourceStoreInterface added to Monitor
- SetResourceStore() and shouldSkipNodeMetrics() helper methods
- Store automatically wired into Monitor via Router.SetMonitor()
- Foundation ready for reduced API polling when agents are active

Files added:
- internal/resources/resource.go - Core Resource type
- internal/resources/store.go - Store with deduplication
- internal/resources/converters.go - Type converters
- internal/resources/platform_data.go - Platform-specific data
- internal/resources/store_test.go - 28 tests
- internal/resources/converters_test.go - Converter tests
- internal/api/resource_handlers.go - REST API handlers
- internal/ai/resource_context.go - AI context builder
- .gemini/docs/unified-resource-architecture.md - Architecture docs

All tests pass.
2025-12-07 13:49:00 +00:00
rcourtman
f2e6927436 Additional updates 2025-12-07 10:22:42 +00:00
rcourtman
90c45968e7 AI Problem Solver implementation and various fixes
- Implement 'Show Problems Only' toggle combining degraded status, high CPU/memory alerts, and needs backup filters
- Add 'Investigate with AI' button to filter bar for problematic guests
- Fix dashboard column sizing inconsistencies between bars and sparklines view modes
- Fix PBS backups display and polling
- Refine AI prompt for general-purpose usage
- Fix frontend flickering and reload loops during initial load
- Integrate persistent SQLite metrics store with Monitor
- Fortify AI command routing with improved validation and logging
- Fix CSRF token handling for note deletion
- Debug and fix AI command execution issues
- Various AI reliability improvements and command safety enhancements
2025-12-06 23:46:08 +00:00
rcourtman
cc3c0187a0 feat: AI features, agent improvements, and host monitoring enhancements
AI Chat Integration:
- Multi-provider support (Anthropic, OpenAI, Ollama)
- Streaming responses with markdown rendering
- Agent command execution for remote troubleshooting
- Context-aware conversations with host/container metadata

Agent Updates:
- Add --enable-proxmox flag for automatic PVE/PBS token setup
- Improve auto-update with semver comparison (prevents downgrades)
- Add updatedFrom tracking to report previous version after update
- Reduce initial update check delay from 30s to 5s
- Add agent version column to Hosts page table

Host Metrics:
- Add DiskIO stats collection (read/write bytes, ops, time)
- Improve disk filtering to exclude Docker overlay mounts
- Add RAID array monitoring via mdadm
- Enhanced temperature sensor parsing

Frontend:
- New Agent Version column on Hosts overview table
- Improved node modal with agent-first installation flow
- Add DiskIO display in host drawer
- Better responsive handling for metric bars
2025-12-05 10:37:02 +00:00
rcourtman
3f08bfe8d0 wip: AI chat integration with multi-provider support
- Add AI service with Anthropic, OpenAI, and Ollama providers
- Add AI chat UI component with streaming responses
- Add AI settings page for configuration
- Add agent exec framework for command execution
- Add API endpoints for AI chat and configuration
2025-12-04 20:16:53 +00:00
rcourtman
93223d8283 Add refresh-cluster button to detect new Proxmox cluster members
When new nodes are added to a Proxmox cluster after Pulse was
initially configured, they weren't showing up in Settings. The
existing "Refresh" button only triggered network discovery, not
cluster membership re-detection.

Changes:
- Add POST /api/config/nodes/{id}/refresh-cluster endpoint
- Add "Refresh" button in cluster node panel in Settings
- Re-detect cluster membership and update stored endpoints

Related to #799
2025-12-02 22:01:00 +00:00
rcourtman
2453198f62 refactor: Remove unreachable dead code branches
- firstForwardedValue: strings.Split always returns at least one element
- shouldRunBackupPoll: remaining is always >= 1 by math
- convertContainerDiskInfo: lowerLabel is never empty for non-rootfs

All three functions now at 100% coverage.
2025-12-02 14:41:53 +00:00
rcourtman
073d5ffa55 fix: Use --ctid instead of --standalone --http-mode in quick-setup command
The quick-setup command for temperature monitoring was generating
--standalone --http-mode which is meant for Docker deployments. This
confused users trying to set up multi-server Proxmox monitoring.

Now uses --ctid which works for both local and remote Proxmox hosts.
The installer detects when the container doesn't exist locally and
installs in "host monitoring only" mode automatically.

If we can determine the actual CTID from the host proxy summary,
we use it; otherwise we show <PULSE_CTID> for the user to replace.

Related to #785
2025-12-02 11:38:47 +00:00
rcourtman
31927f71e9 fix: mark unused parameters to satisfy unparam linter
Mark intentionally unused parameters with underscore to:
- Silence unparam warnings for legitimate unused parameters
- Keep function signatures intact for API compatibility
- Remove unused req from serveChecksum helper
2025-11-27 10:12:48 +00:00
courtmanr@gmail.com
c021dc6ca5 Enhance table responsiveness across multiple components 2025-11-26 17:57:09 +00:00
rcourtman
21c033c144 fix: remove references to deleted install-host-agent.sh script
The unified agent system replaced install-host-agent.sh with install.sh.
This commit updates all references:
- Dockerfile: removed COPY for deleted script
- router.go: serve install.sh at /install-host-agent.sh endpoint (backwards compatible)
- build-release.sh: removed copy of deleted script
- validate-release.sh: removed validation of deleted script
- install.sh: updated script list for bare-metal installs
2025-11-26 09:57:06 +00:00
rcourtman
e467523a61 feat: serve install scripts from GitHub releases instead of main branch
Scripts like install.sh and install-sensor-proxy.sh are now attached
as release assets and downloaded from releases/latest/download/ URLs.
This ensures users always get scripts compatible with their installed
version, even while development continues on main.

Changes:
- build-release.sh: copy install scripts to release directory
- create-release.yml: upload scripts as release assets
- Updated all documentation and code references to use release URLs
- Scripts reference each other via release URLs for consistency
2025-11-26 08:59:59 +00:00
rcourtman
ee35d9e5a5 feat: add auto-update support for unified agent
Implement self-update capability for the unified pulse-agent binary:

- Add internal/agentupdate package with cross-platform update logic
- Hourly version checks against /api/agent/version endpoint
- SHA256 checksum verification for downloaded binaries
- Atomic binary replacement with backup/rollback on failure
- Support for Linux, macOS, and Windows (10 platform/arch combinations)

Build and release changes:
- Dockerfile builds unified agent for all platforms
- build-release.sh includes unified agent in release artifacts
- validate-release.sh validates unified agent binaries
- Install scripts (install.sh, install.ps1) use correct URL format

Related to #727, #737
2025-11-25 23:15:03 +00:00
courtmanr@gmail.com
af2caa4b81 Register unified installer routes
Exposes /api/install/install.sh and /api/install/install.ps1 for the unified agent installer.
2025-11-25 11:25:10 +00:00
courtmanr@gmail.com
3599c6592e Improve installer UX with pauses and popups on failure
Fixes #755. Adds interactive pauses and graphical popups (where available) to installer scripts when critical errors occur, ensuring troubleshooting guides are readable. Also clarifies 'build from source' instructions.
2025-11-25 11:17:37 +00:00
courtmanr@gmail.com
5031ba481b Fix: Allow double slashes in install script URLs 2025-11-24 17:58:00 +00:00
courtmanr@gmail.com
6b84b9a2bf Add PULSE_AUTH_HIDE_LOCAL_LOGIN option to hide password form
Implements #750 - allows hiding the username/password login form when
using OIDC SSO to avoid user confusion, while maintaining security.

- Added HideLocalLogin config option (env: PULSE_AUTH_HIDE_LOCAL_LOGIN)
- Exposed hideLocalLogin in /api/security/status endpoint
- Updated Login.tsx to conditionally hide local login form
- Added escape hatch via ?show_local=true URL parameter

This approach avoids the security and upgrade issues that led to
DISABLE_AUTH being removed (see #707, #678), while solving the UX
problem of users being confused by multiple login options.
2025-11-24 17:40:43 +00:00
courtmanr@gmail.com
9caba86389 Fix #735: Allow HEAD requests for download endpoints and fix routing
- Allow HEAD requests in addition to GET for all download handlers
  (install scripts, binaries, checksums) to prevent 405 errors
- Add /uninstall-host-agent.sh to special routes in ServeHTTP
- Add test coverage for HEAD request handling
- Resolves 'method not allowed' errors during agent installation
2025-11-24 15:16:14 +00:00
courtmanr@gmail.com
08a83c9adb Add host agent checksum route and bump version to 4.32.6
- Add /download/pulse-host-agent.sha256 route to serve checksums
- Fixes 'Checksum not available' warning during host agent installation
- Bump version to 4.32.6

Related to #746
2025-11-24 07:57:17 +00:00
courtmanr@gmail.com
60721cbc22 Add mutex protection for config watcher reloads (re #748)
Introduced sync.RWMutex to protect concurrent access to configuration
fields (AuthUser, AuthPass, APITokens) that are modified by the
ConfigWatcher at runtime.

- Added global config.Mu RWMutex in internal/config/config.go
- Protected config updates in ConfigWatcher.reloadConfig() and reloadAPITokens()
- Protected config reads in CheckAuth and all API token handlers
- Protected Router.SetConfig() during full config reloads

This prevents race conditions when .env file changes trigger config
reloads while authentication handlers are reading the same fields.
2025-11-24 07:45:21 +00:00
courtmanr@gmail.com
d0a725fde5 fix: Add dev environment fallback paths for Docker agent downloads
- Add fallback to project root scripts/ directory for install-docker-agent.sh
- Add fallback to project root bin/ directory for pulse-docker-agent binary
- Fixes 404 errors when downloading agent installer and binary in dev mode
- Production paths remain unchanged (/opt/pulse/...)
2025-11-23 16:01:40 +00:00
courtmanr@gmail.com
6e404b24eb Fix install-host-agent.sh function order, remove duplicate, and improve dev serving 2025-11-23 12:27:11 +00:00
rcourtman
d6cbfc23ec Harden public URL detection and setup token handling 2025-11-20 19:27:14 +00:00
rcourtman
823508dc48 Related to #712: auto-restore host agent binaries for download 2025-11-20 15:45:21 +00:00
courtmanr@gmail.com
838993cf40 Implement sensor proxy installation and configuration updates 2025-11-20 13:23:21 +00:00
rcourtman
844c895521 WIP: Fix temperature proxy registration persistence (incomplete)
This commit contains multiple fixes for temperature proxy registration,
but the core issue remains unresolved.

## What's Fixed:
1. Added config pointer and reloadFunc to TemperatureProxyHandlers
2. Added SetConfig method to keep handler in sync with router config changes
3. Added config reload after registration to prevent monitor from overwriting
4. Fixed installer port conflict detection and duplicate YAML key issues
5. Added comprehensive debug logging throughout registration flow

## What's Still Broken:
The TemperatureProxyURL, TemperatureProxyToken, and TemperatureProxyControlToken
fields are NOT persisting to nodes.enc after SaveNodesConfig is called.

Debug logs confirm:
- HandleRegister correctly updates nodesConfig.PVEInstances[matchedIndex]
- The correct data is passed to SaveNodesConfig (verified in logs)
- SaveNodesConfig completes without errors
- Config reload executes successfully
- BUT after Pulse restart, the fields are empty when loaded from disk

The bug is in SaveNodesConfig serialization or file writing logic itself.

Related files:
- internal/api/temperature_proxy.go: Registration handler
- internal/config/persistence.go: SaveNodesConfig implementation
- internal/config/config.go: PVEInstance struct definition
2025-11-19 20:12:19 +00:00
rcourtman
3f46d35a81 feat: make PVE polling interval configurable (related to #467) 2025-11-18 21:30:04 +00:00
rcourtman
a479040651 Improve temperature proxy workflow 2025-11-17 14:25:46 +00:00
rcourtman
5951a364f7 Auto-deploy proxy for standalone temp monitoring 2025-11-16 09:47:07 +00:00
rcourtman
de5b314842 Improve temperature proxy control-plane flow 2025-11-15 21:49:51 +00:00
rcourtman
ec822575dd Improve update procedure tracking 2025-11-15 16:43:42 +00:00
rcourtman
4fbb118072 Add PULSE_LXC_CTID env override for LXC CTID detection
Modern Proxmox LXC containers (cgroup v2 + systemd) don't expose the CTID
inside the guest namespace. The auto-detection in DetectLXCCTID() works
for older LXC setups and when hostname is numeric, but fails for most
production containers where users set custom hostnames.

Changes:
- Added PULSE_LXC_CTID environment variable override in router.go:490-495
- Graceful fallback: auto-detect first, then check env var, then show placeholder
- UI already handles missing CTID by showing "pct exec <ctid>" placeholder

This provides a robust solution for thousands of users:
- Stock Proxmox LXC: Shows `pct exec <ctid>` placeholder (user substitutes manually)
- Custom hostname containers: Can set PULSE_LXC_CTID=171 in compose/systemd
- Numeric hostname containers: Auto-detected (backwards compatible)

Related: FirstRunSetup.tsx already has graceful fallback (line 336-339)
2025-11-15 13:25:07 +00:00
rcourtman
639635a84b Add Docker container name auto-detection to bootstrap UI
- Added DetectDockerContainerName() to detect container name from hostname
- Extended /api/security/status to expose dockerContainerName field
- Updated FirstRunSetup to show actual container name when detected:
  * Before: 'docker exec <container-name> cat /data/.bootstrap_token'
  * After: 'docker exec pulse cat /data/.bootstrap_token'

This reduces friction for users - no need to look up the container name.
Works when Docker container is named (--name flag), falls back to
placeholder for auto-generated container IDs.
2025-11-15 10:45:00 +00:00
rcourtman
ae0481c982 Improve bootstrap token UX with smart environment detection
- Added DetectLXCCTID() to internal/system/container.go to detect Proxmox container ID
- Extended /api/security/status to expose inContainer and lxcCtid fields
- Updated FirstRunSetup to show most relevant command based on detected environment:
  * LXC with CTID: Shows 'pct exec 171 -- cat /etc/pulse/.bootstrap_token'
  * Docker: Shows 'docker exec <container-name> cat /data/.bootstrap_token'
  * Bare metal: Shows 'cat /etc/pulse/.bootstrap_token'
- Collapsed alternative methods behind 'Show other retrieval methods' button

This addresses user feedback that showing all options was overwhelming.
Now users see the command most likely to work for their setup first,
with alternatives hidden but still accessible.
2025-11-15 10:18:59 +00:00
rcourtman
7cfe998fca Make download tests use temp bin dir 2025-11-14 13:59:50 +00:00
rcourtman
20194d9bb7 Add CI build workflow and tighten proxy diagnostics 2025-11-14 13:32:29 +00:00
rcourtman
6e9a62f663 Ensure Windows download finds .exe (related to #684) 2025-11-14 10:59:45 +00:00
rcourtman
e2bd514899 Fix HTTP mode for pulse-sensor-proxy and improve installer safety
## HTTP Server Fixes
- Add source IP middleware to enforce allowed_source_subnets
- Fix missing source subnet validation for external HTTP requests
- HTTP health endpoint now respects subnet restrictions

## Installer Improvements
- Auto-configure allowed_source_subnets with Pulse server IP
- Add cluster node hostnames to allowed_nodes (not just IPs)
- Fix node validation to accept both hostnames and IPs
- Add Pulse server reachability check before installation
- Add port availability check for HTTP mode
- Add automatic rollback on service startup failure
- Add HTTP endpoint health check after installation
- Fix config backup and deduplication (prevent duplicate keys)
- Fix IPv4 validation with loopback rejection
- Improve registration retry logic with detailed errors
- Add automatic LXC bind mount cleanup on uninstall

## Temperature Collection Fixes
- Add local temperature collection for self-monitoring nodes
- Fix node identifier matching (use hostname not SSH host)
- Fix JSON double-encoding in HTTP client response

Related to #XXX (temperature monitoring fixes)
2025-11-13 18:22:36 +00:00
rcourtman
a2b8940873 Add Pulse API endpoints for temperature proxy registration
Implements REST API endpoints to enable automatic registration of
temperature proxies during sensor-proxy installation.

API endpoints:
- POST /api/temperature-proxy/register
  - Accepts: hostname, proxy_url
  - Returns: authentication token
  - Finds matching PVE instance and configures proxy URL/token
  - No authentication required (called during installation)

- DELETE /api/temperature-proxy/unregister?hostname=X
  - Removes proxy configuration from PVE instance
  - Requires admin authentication

Implementation:
- Uses config.ConfigPersistence for loading/saving nodes.enc
- Matches PVE instances by hostname in Host field or ClusterEndpoints
- Generates cryptographically secure random tokens (32 bytes, base64)
- Atomic config updates (load → modify → save)

Next step: Update install-sensor-proxy.sh to call registration API

Related to #571
2025-11-13 16:20:47 +00:00
rcourtman
56a7579c99 Add Remember Me feature with sliding session expiration (Related to #707)
Implements a "Remember Me" option that allows users to stay logged in
for 30 days instead of the default 24 hours. This addresses the pain
point of frequent re-authentication in LAN-only environments while
maintaining authentication security.

Backend changes:
- Add rememberMe field to login request handling
- Support variable session durations (24h default, 30d with Remember Me)
- Implement sliding session expiration that extends sessions on each
  authenticated request using the original duration
- Store OriginalDuration in session data for proper sliding window
- Update session cookie MaxAge to match session duration

Frontend changes:
- Add "Remember Me for 30 days" checkbox to login form
- Pass rememberMe flag in login request
- Improve UI with clear duration indication

Key features:
- Sessions extend automatically on each request (sliding window)
- Original duration preserved across session extension
- Backward compatible with existing sessions (legacy sessions work)
- Sessions persist across server restarts

This provides a better user experience for LAN deployments without
compromising security by completely disabling authentication.
2025-11-13 10:37:08 +00:00
rcourtman
ff5de0147b Fix missing regexp import for path traversal validation 2025-11-12 16:34:16 +00:00
rcourtman
946e2e455f Security: Fix path traversal vulnerability in host-agent download endpoint
CRITICAL SECURITY FIX: The /download/pulse-host-agent endpoint was directly
concatenating user-supplied platform and arch query parameters into file paths
without validation, allowing path traversal attacks.

An attacker could request:
  /download/pulse-host-agent?platform=../../etc/passwd
to read arbitrary files from the container filesystem.

Fix: Add input validation to only allow alphanumeric characters and hyphens
in platform/arch parameters before using them in file paths.

Related: Codex security audit identified this during pre-release review
2025-11-12 16:27:11 +00:00
rcourtman
48f8473200 Fix router to allow frontend pages without authentication
When a request for /login (or any other frontend route) comes in without
proper Accept headers (like from curl or some browsers), the server was
returning 'Authentication required' text instead of serving the frontend HTML.

This is because the router was checking authentication before serving ANY
non-API route, including frontend pages like /login, /dashboard, etc.

The fix: Frontend routes should always be served without backend auth checks.
The authentication logic runs in the frontend JavaScript after the page loads.

Backend auth should only block:
- API endpoints (/api/*)
- WebSocket connections (/ws*, /socket.io/*)
- Download endpoints (/download/*)
- Special scripts (/install-*.sh, etc.)

All other routes are frontend pages that need to be served to everyone so
the login page can load and handle auth in the browser.

This fixes the integration tests where Playwright couldn't see the login
form because the server was rejecting the /login request before serving HTML.

Related to #695 (release workflow integration tests)
2025-11-12 11:30:22 +00:00