Commit graph

352 commits

Author SHA1 Message Date
rcourtman
d2330cf405 refactor(ai): Remove over-engineered URL discovery service
Keep only the simple AI-powered approach:
- set_resource_url tool lets AI save discovered URLs
- Users ask AI directly: 'Find URLs for my containers'
- AI uses its intelligence to discover and set URLs

Removed:
- URLDiscoveryService (rigid port scanning)
- Bulk discovery API endpoints
- Frontend discovery button

The AI itself is smart enough to iterate through resources
and discover URLs when asked.
2025-12-10 08:35:24 +00:00
rcourtman
0ae4daca8d feat(ai): Add bulk URL discovery service
- Add URLDiscoveryService for scanning all resources at once
- Scans common web ports (80, 443, 8080, 8096, 3000, etc.)
- Automatically saves discovered URLs to resource metadata
- Add API endpoints for start/status/cancel discovery
- Progress tracking with results reporting

Endpoints:
- POST /api/ai/discover-urls/start - Start bulk discovery
- GET /api/ai/discover-urls/status - Check progress
- POST /api/ai/discover-urls/cancel - Cancel discovery
2025-12-10 08:30:56 +00:00
rcourtman
fac2bf91d9 feat(ai): Add URL discovery tool - AI can find and set resource URLs
- Add MetadataProvider interface for AI to update resource URLs
- Add set_resource_url tool to AI service
- Wire up metadata stores to AI service via router
- Add URL discovery guidance to AI system prompt
- AI can now inspect guests/containers/hosts for web services
  and automatically save discovered URLs to Pulse metadata

Usage: Ask the AI 'Find the web URL for this container' and it will:
1. Check for listening ports and web servers
2. Get the IP address
3. Verify the URL works
4. Save it to Pulse for quick dashboard access
2025-12-10 00:29:07 +00:00
rcourtman
a6656381c8 Add AI monitoring enhancements and host metadata features
- Add host metadata API for custom URL editing on hosts page
- Enhance AI routing with unified resource provider lookup
- Add encryption key watcher script for debugging key issues
- Improve AI service with better command timeout handling
- Update dev environment workflow with key monitoring docs
- Fix resource store deduplication logic
2025-12-09 16:27:46 +00:00
rcourtman
67351bf58c feat: AI integration, Docker metrics, RAID display, and infrastructure improvements
- Add Claude OAuth authentication support with hybrid API key/OAuth flow
- Implement Docker container historical metrics in backend and charts API
- Add CEPH cluster data collection and new Ceph page
- Enhance RAID status display with detailed tooltips and visual indicators
- Fix host deduplication logic with Docker bridge IP filtering
- Fix NVMe temperature collection in host agent
- Add comprehensive test coverage for new features
- Improve frontend sparklines and metrics history handling
- Fix navigation issues and frontend reload loops
2025-12-09 09:29:27 +00:00
rcourtman
cc4b32ad4a fix: complete unified resources WebSocket integration
Backend:
- Call SetMonitor after router creation to inject resource store
- Add debug logging for resource population and broadcast

Frontend:
- Add resources array to WebSocket store initial state
- Handle resources in WebSocket message processing
- Use reconcile for efficient state updates

The unified resources are now properly:
1. Populated from StateSnapshot on each broadcast cycle
2. Converted to frontend format (ResourceFrontend)
3. Included in WebSocket state messages
4. Received and stored in frontend state
5. Consumed by migrated route components

Console now shows '[DashboardView] Using unified resources: VMs: X'
confirming the migration is working end-to-end.
2025-12-07 23:52:00 +00:00
rcourtman
cac109b497 fix: Populate resources on-demand when /api/resources is called
The Resources page was showing 0 resources because the store was only
populated when /api/state was called (from the dashboard). Now the
resources are populated on-demand when /api/resources is accessed.

Changes:
- Added StateProvider interface to ResourceHandlers
- SetStateProvider() method for injecting the monitor
- HandleGetResources now calls PopulateFromSnapshot before querying
- Router injects monitor as state provider during SetMonitor()

This ensures the /resources page works even when accessed directly
without visiting the main dashboard first.
2025-12-07 14:47:29 +00:00
rcourtman
5f000b7974 feat: Complete Unified Resource Architecture (Phases 1-3)
This commit implements the Unified Resource Architecture for AI-first
infrastructure management. Key features:

Phase 1 - Backend Unification:
- New unified Resource type with 9 resource types, 7 platforms, 7 statuses
- Resource store with identity-based deduplication (hostname, machineID, IP)
- 8 converter functions (FromNode, FromVM, FromContainer, etc.)
- REST API endpoints: /api/resources, /api/resources/stats, /api/resources/{id}
- 28 comprehensive unit tests

Phase 2 - AI Context Enhancement:
- Unified context builder for AI system prompts
- Cross-platform query methods: GetTopByCPU, GetTopByMemory, GetTopByDisk
- Resource correlation: GetRelated (parent, children, siblings, cluster)
- Infrastructure summary: GetResourceSummary with health status counts
- AI context now includes top consumers and infrastructure overview

Phase 3 - Agent Preference & Hybrid Mode:
- Polling optimization methods in resource store
- ResourceStoreInterface added to Monitor
- SetResourceStore() and shouldSkipNodeMetrics() helper methods
- Store automatically wired into Monitor via Router.SetMonitor()
- Foundation ready for reduced API polling when agents are active

Files added:
- internal/resources/resource.go - Core Resource type
- internal/resources/store.go - Store with deduplication
- internal/resources/converters.go - Type converters
- internal/resources/platform_data.go - Platform-specific data
- internal/resources/store_test.go - 28 tests
- internal/resources/converters_test.go - Converter tests
- internal/api/resource_handlers.go - REST API handlers
- internal/ai/resource_context.go - AI context builder
- .gemini/docs/unified-resource-architecture.md - Architecture docs

All tests pass.
2025-12-07 13:49:00 +00:00
rcourtman
f2e6927436 Additional updates 2025-12-07 10:22:42 +00:00
rcourtman
90c45968e7 AI Problem Solver implementation and various fixes
- Implement 'Show Problems Only' toggle combining degraded status, high CPU/memory alerts, and needs backup filters
- Add 'Investigate with AI' button to filter bar for problematic guests
- Fix dashboard column sizing inconsistencies between bars and sparklines view modes
- Fix PBS backups display and polling
- Refine AI prompt for general-purpose usage
- Fix frontend flickering and reload loops during initial load
- Integrate persistent SQLite metrics store with Monitor
- Fortify AI command routing with improved validation and logging
- Fix CSRF token handling for note deletion
- Debug and fix AI command execution issues
- Various AI reliability improvements and command safety enhancements
2025-12-06 23:46:08 +00:00
rcourtman
cc3c0187a0 feat: AI features, agent improvements, and host monitoring enhancements
AI Chat Integration:
- Multi-provider support (Anthropic, OpenAI, Ollama)
- Streaming responses with markdown rendering
- Agent command execution for remote troubleshooting
- Context-aware conversations with host/container metadata

Agent Updates:
- Add --enable-proxmox flag for automatic PVE/PBS token setup
- Improve auto-update with semver comparison (prevents downgrades)
- Add updatedFrom tracking to report previous version after update
- Reduce initial update check delay from 30s to 5s
- Add agent version column to Hosts page table

Host Metrics:
- Add DiskIO stats collection (read/write bytes, ops, time)
- Improve disk filtering to exclude Docker overlay mounts
- Add RAID array monitoring via mdadm
- Enhanced temperature sensor parsing

Frontend:
- New Agent Version column on Hosts overview table
- Improved node modal with agent-first installation flow
- Add DiskIO display in host drawer
- Better responsive handling for metric bars
2025-12-05 10:37:02 +00:00
rcourtman
3f08bfe8d0 wip: AI chat integration with multi-provider support
- Add AI service with Anthropic, OpenAI, and Ollama providers
- Add AI chat UI component with streaming responses
- Add AI settings page for configuration
- Add agent exec framework for command execution
- Add API endpoints for AI chat and configuration
2025-12-04 20:16:53 +00:00
rcourtman
8fca8c6eb4 Refactor alert system: fix race conditions, memory leaks, and improve code quality
- Rename checkFlapping to checkFlappingLocked to clarify lock contract
- Replace goto statements with structured control flow
- Wire up unused recordAlertFired/recordAlertResolved metric hooks
- Add trackingMapCleanup goroutine to prevent memory leaks from stale entries
- Tighten alert ID validation to alphanumeric + safe punctuation
- Fix history save error handling to properly manage backup lifecycle
- Add auto-migration for deprecated GroupingWindow field
- Refactor 300+ line UpdateConfig into focused helper functions
- Unify duplicate evaluateVMCondition/evaluateContainerCondition
- Add constants for magic numbers (thresholds, timing, flapping)
- Update tests to match new backup behavior
2025-12-02 23:31:36 +00:00
rcourtman
93223d8283 Add refresh-cluster button to detect new Proxmox cluster members
When new nodes are added to a Proxmox cluster after Pulse was
initially configured, they weren't showing up in Settings. The
existing "Refresh" button only triggered network discovery, not
cluster membership re-detection.

Changes:
- Add POST /api/config/nodes/{id}/refresh-cluster endpoint
- Add "Refresh" button in cluster node panel in Settings
- Re-detect cluster membership and update stored endpoints

Related to #799
2025-12-02 22:01:00 +00:00
rcourtman
72af7bed5e style: Apply gofmt to 37 files
Standardize code formatting across test files and monitor.go.
No functional changes.
2025-12-02 17:21:48 +00:00
rcourtman
c108cba8bb security: Add request body size limits to remaining API handlers
Add http.MaxBytesReader to 8 additional handlers to complete API
hardening against memory exhaustion attacks:

- guest_metadata.go: HandleUpdateMetadata (16KB)
- notification_queue.go: RetryDLQItem, DeleteDLQItem (8KB each)
- temperature_proxy.go: HandleRegister (8KB)
- host_agents.go: HandleReport (256KB)
- updates.go: HandleApplyUpdate (8KB)
- docker_metadata.go: HandleUpdateMetadata (16KB)
- system_settings.go: UpdateSystemSettings (64KB)

All API handlers that decode JSON request bodies now have size limits.
2025-12-02 16:47:13 +00:00
rcourtman
7b55564b62 security: Add request body size limits to API handlers
Add http.MaxBytesReader to 16 additional handlers to prevent memory
exhaustion attacks via oversized request bodies:

- docker_agents.go: HandleReport (512KB), HandleCommandAck (8KB),
  HandleSetCustomDisplayName (8KB)
- alerts.go: UpdateAlertConfig (64KB), BulkAcknowledgeAlerts (32KB),
  BulkClearAlerts (32KB)
- config_handlers.go: HandleAddNode, HandleTestConnection,
  HandleUpdateNode, HandleTestNodeConfig (32KB each),
  HandleVerifyTemperatureSSH, HandleExportConfig, HandleDiscoverServers,
  HandleSetupScriptURL (8KB each), HandleImportConfig (1MB),
  HandleUpdateMockMode (16KB)
2025-12-02 16:43:13 +00:00
rcourtman
ccc26624e8 security: Add request body size limits to notification handlers
Add http.MaxBytesReader limits to prevent memory exhaustion attacks:
- UpdateEmailConfig: 32KB limit
- UpdateAppriseConfig: 64KB limit
- CreateWebhook: 64KB limit
- UpdateWebhook: 64KB limit

This follows the pattern already used in system_settings.go for
SSH config validation.
2025-12-02 16:37:30 +00:00
rcourtman
c019e4cd0e docs: Add godoc comments to exported functions
Add missing godoc comments to:
- NewRateLimiter and Allow in ratelimit.go
- SnapshotSyncStatus in temperature_proxy.go
- NewClient and GetVersion in pkg/pmg/client.go
2025-12-02 15:58:59 +00:00
rcourtman
25294da5d5 perf: Cache hostname lowercase in temperature proxy lookups
Pre-compute strings.ToLower(hostname) before loops that search for
matching PVE instances. Avoids repeated lowercasing in two functions.
2025-12-02 15:43:41 +00:00
rcourtman
6931fa34cd perf: Cache err.Error() in cluster node validation
Cache err.Error() result in two locations in config_handlers.go:
- TLS mismatch detection (3x calls to 1)
- Standalone node detection (2x calls to 1)
2025-12-02 15:41:18 +00:00
rcourtman
2453198f62 refactor: Remove unreachable dead code branches
- firstForwardedValue: strings.Split always returns at least one element
- shouldRunBackupPoll: remaining is always >= 1 by math
- convertContainerDiskInfo: lowerLabel is never empty for non-rootfs

All three functions now at 100% coverage.
2025-12-02 14:41:53 +00:00
rcourtman
eb302e6d4e test: Add SessionStore.load legacy format tests for API package 2025-12-02 14:12:32 +00:00
rcourtman
e7fa996e03 test: Add CSRFTokenStore.load format tests for API package
Cover legacy JSON format migration and current format with nil/expired
entries. Improves load function coverage from 67.9% to 100%.
2025-12-02 14:07:00 +00:00
rcourtman
daafb51493 test: Add CheckAuth tests for API package
Add tests for sliding expiration session validation and no-auth
configured scenarios. These test explicit paths for better coverage
documentation even though they were already exercised indirectly.
2025-12-02 13:57:23 +00:00
rcourtman
2f203bf2d9 test: Add CheckCSRF valid token test for 100% coverage
Test the success path where a valid CSRF token is provided with a
matching session. This covers the final branch in CheckCSRF.
2025-12-02 13:51:27 +00:00
rcourtman
50769e12ea test: Add HandleSetupScript PBS path tests for API package
Cover the PBS script generation branch that was previously untested.
Verifies PBS-specific content, auth token handling, and placeholder host.
2025-12-02 13:36:23 +00:00
rcourtman
28533975f6 test: Add CheckAuth tests for API package
Cover proxy auth headers, OIDC session validation, and session cookie
paths that were previously untested.
2025-12-02 13:32:21 +00:00
rcourtman
d058b2ab96 test: Add CheckProxyAuth tests for API package
Add comprehensive direct tests for the CheckProxyAuth function covering:
- Not configured (returns false)
- Invalid secret (returns false)
- Missing secret header (returns false)
- Valid secret without user header configured (returns true, admin)
- Missing user header when configured (returns false)
- Valid auth with username (returns true with username)
- Role checking with empty roles header (defaults to admin)
- Role checking with admin role present (returns admin=true)
- Role checking without admin role (returns admin=false)
- Custom role separator (comma instead of pipe)
- Role with whitespace (trimmed correctly)

Coverage: CheckProxyAuth 89.3% → 100%
2025-12-02 13:28:02 +00:00
rcourtman
d219d9965d test: Add ValidateSession tests for API package
Add comprehensive tests for the ValidateSession wrapper function covering:
- Non-existent token (returns false)
- Empty token (returns false)
- Valid token (returns true)
- Expired token (returns false)

The ValidateSession function is a simple wrapper around the SessionStore's
ValidateSession method, but having direct tests ensures the wrapper is
exercised and documents its expected behavior.

Coverage: ValidateSession 0% → 100%
2025-12-02 13:22:45 +00:00
rcourtman
ef6a370144 test: Add ensureScope tests for API package
Add comprehensive tests for the ensureScope function covering:
- Empty scope parameter (always allows access)
- No token in context (session-based request, allows access)
- Token with matching scope (allows access)
- Token with multiple scopes including required one (allows access)
- Token missing required scope (rejects with 403)
- Token with empty scopes (defaults to wildcard, allows access)
- Rejection returns proper JSON response format

Coverage: ensureScope 0% → 100%
Coverage: API package 32.1% → 32.2%
2025-12-02 13:19:11 +00:00
rcourtman
d970055532 test: Add RequireAuth tests for API package
Add comprehensive tests for the RequireAuth middleware covering:
- No auth configured (allows access by design)
- API-only mode (rejects requests without token)
- API-only mode (accepts valid X-API-Token)
- Basic auth with invalid credentials
- Basic auth JSON vs plain text error responses
- Valid basic auth (allowed)
- Proxy auth (allowed)
- Proxy auth with invalid secret (rejected)
- Bearer token with basic auth configured (allowed)
- Invalid Bearer token (rejected)

Coverage: RequireAuth 7.1% → 78.6%
Coverage: CheckAuth 66.9% → 69.1%
Coverage: API package 31.9% → 32.1%
2025-12-02 13:09:48 +00:00
rcourtman
1073c511b3 test: Add RequireAdmin tests for API package
Add comprehensive tests for the RequireAdmin middleware covering:
- No auth configured (allows access by design)
- API-only mode (rejects requests without token)
- Basic auth with invalid credentials
- Proxy auth with admin role (allowed)
- Proxy auth with non-admin role (forbidden)
- Proxy auth with invalid secret (unauthorized)
- Proxy auth without role header (defaults to admin)
- Proxy auth with custom role separator
- Proxy auth with spaces in roles (trimmed)
- Basic auth authenticated users (allowed as admin)
- JSON vs plain text error responses based on path/Accept header

Also improves CheckProxyAuth coverage as a side effect.

Coverage: RequireAdmin 20.8% → 87.5%
Coverage: CheckProxyAuth 0.0% → 89.3%
Coverage: API package 30.9% → 31.9%
2025-12-02 13:06:06 +00:00
rcourtman
71e15886ed test: Add isRequestAuthenticated tests for API package
Add comprehensive tests for the isRequestAuthenticated function covering:
- Nil inputs (config, request, both)
- Basic auth (valid, invalid password, invalid username, malformed base64, missing colon)
- API token via X-API-Token header
- API token via Bearer authorization header (case insensitive)
- Invalid/empty/whitespace API tokens
- No auth configured scenarios
- Empty session cookie handling

Coverage: isRequestAuthenticated 26.1% → 82.6%
Coverage: API package 30.7% → 30.9%
2025-12-02 12:59:18 +00:00
rcourtman
cf6149e884 test: Add CheckCSRF tests for API package
Add comprehensive tests for the CheckCSRF function covering:
- Safe methods (GET, HEAD, OPTIONS) bypass
- API token authentication bypass
- Basic auth bypass
- No session cookie handling
- Missing CSRF token rejection with new token issuance
- Invalid CSRF token rejection with new token issuance
- CSRF token from FormValue
- Unsafe methods (POST, PUT, DELETE, PATCH) enforcement

Coverage: CheckCSRF 32.0% → 96.0%
Coverage: API package 30.5% → 30.7%
2025-12-02 12:53:32 +00:00
rcourtman
cd33c577f8 test: Add resolvePublicURL tests for API package
Add comprehensive tests for the resolvePublicURL function covering:
- Configured PublicURL (simple, trailing slashes, ports, whitespace)
- Request-derived URL (HTTP, HTTPS via TLS, X-Forwarded-Proto)
- No host fallback (with/without frontend port)
- Nil request handling

Coverage: resolvePublicURL 12.5% → 100%
Coverage: API package 30.3% → 30.5%
2025-12-02 12:45:04 +00:00
rcourtman
073d5ffa55 fix: Use --ctid instead of --standalone --http-mode in quick-setup command
The quick-setup command for temperature monitoring was generating
--standalone --http-mode which is meant for Docker deployments. This
confused users trying to set up multi-server Proxmox monitoring.

Now uses --ctid which works for both local and remote Proxmox hosts.
The installer detects when the container doesn't exist locally and
installs in "host monitoring only" mode automatically.

If we can determine the actual CTID from the host proxy summary,
we use it; otherwise we show <PULSE_CTID> for the user to replace.

Related to #785
2025-12-02 11:38:47 +00:00
rcourtman
350d21c80d test: Add diagnostics function tests for error handling
Add comprehensive tests for untested diagnostics functions:

- fingerprintPublicKey: 14 test cases covering empty/whitespace input,
  invalid key formats, truncated/malformed keys, and valid ED25519 keys

- countLegacySSHKeys: 8 test cases covering non-existent directories,
  empty directories, files without id_ prefix, multiple key types,
  and directory filtering (subdirectories not counted)

- resolveUserName: 4 test cases covering UID 0 (root), current user,
  non-existent UID fallback, and max uint32 boundary

- resolveGroupName: 4 test cases covering GID 0 (root/wheel), current
  group, non-existent GID fallback, and max uint32 boundary

Coverage: fingerprintPublicKey 0% -> 100%, countLegacySSHKeys 0% -> 100%,
resolveUserName 0% -> 100%, resolveGroupName 0% -> 100%
2025-12-02 03:39:52 +00:00
rcourtman
b053a2848b fix: Update TestPublicURLDetectionUsesForwardedHeaders for proxy hardening
The test was failing after commit d6cbfc23 added security hardening
that requires authentication and trusted proxy configuration for
X-Forwarded-* headers to be read during public URL detection.

- Add API token authentication to the test request
- Configure 127.0.0.1 as trusted proxy for the test
- Add export_test.go with ResetTrustedProxyConfigForTests() to allow
  external tests to reset the trusted proxy configuration
2025-12-02 03:16:52 +00:00
rcourtman
de90519b21 test: Add handleHealth method tests
Add 2 tests for health endpoint:
- POST/PUT/DELETE/PATCH return 405 Method Not Allowed
- HEAD is allowed (same as GET)

Coverage: 50% → 83.3%
2025-12-02 02:09:04 +00:00
rcourtman
5ff69a7e35 test: Add adminBypassEnabled tests
Add 5 tests to cover all branches:
- Not requested (ALLOW_ADMIN_BYPASS != "1")
- Enabled with PULSE_DEV=true
- Enabled with NODE_ENV=development
- Case-insensitive NODE_ENV check
- Declined when outside dev mode

Coverage: 40% → 100%
2025-12-02 01:53:55 +00:00
rcourtman
7ed8ef2074 test: Add loadTrustedProxyCIDRs tests
Cover invalid CIDR, invalid IP, IPv6, and empty entry handling (48% to 100%)
2025-12-02 01:48:41 +00:00
rcourtman
24fbce1281 test: Add LogAuditEvent tests
Cover success and failure logging branches (66.7% to 100%)
2025-12-02 01:46:27 +00:00
rcourtman
8f42156781 test: Add SecurityHeadersWithConfig tests
Cover all CSP/X-Frame-Options embedding configurations (57.1% to 100%)
2025-12-02 01:44:22 +00:00
rcourtman
07997b3516 test: Add filesystem error tests for SessionStore
Cover saveUnsafe (57.9% to 89.5%) and load (55.6% to 74.1%) error paths
2025-12-02 01:40:54 +00:00
rcourtman
21b69607f0 test: Add filesystem error tests for CSRFTokenStore
Cover saveUnsafe (57.9% to 89.5%) and load (50.0% to 67.9%) error paths
2025-12-02 01:38:49 +00:00
rcourtman
999f19beed test: Add filesystem error tests for RecoveryTokenStore.saveUnsafe
Cover MkdirAll, WriteFile, and Rename error paths (46.7% to 86.7%)
2025-12-02 01:36:21 +00:00
rcourtman
b3f88d7a8c test: Add edge case tests for HandleSetupScript
Tests method not allowed, missing type parameter, and invalid host
parameter error paths (71.4% to 79.2% coverage).
2025-12-02 01:11:57 +00:00
rcourtman
7d3de1bbcc test: Add invalid PEM data test for newOIDCHTTPClient
Tests the error path when a CA bundle file contains non-PEM data
(81.8% to 86.4% coverage).
2025-12-02 01:00:06 +00:00
rcourtman
1a595b3b2e test: Add edge cases for UniversalRateLimitMiddleware
Cover nil config initialization and static asset bypass paths.
Coverage: UniversalRateLimitMiddleware 87.5% -> 100%
2025-12-02 00:54:37 +00:00