When there's only 1 day of AI usage data, the sparkline was invisible
because a single point draws at x=0 with no width. Now draws a
horizontal line across the full width so users can see the value.
This happens when AI has just been enabled and there's only one
day of recorded usage so far.
Complete Phase 2 baseline integration:
- Add baseline_exports.go for clean type aliasing
- Wire baseline store initialization into StartPatrol
- Implement startBaselineLearning background loop
- Runs initial learning after 5 min delay
- Updates baselines every hour from metrics history
- Learns from 7 days of data for nodes, VMs, containers
- Add SetBaselineStore methods throughout the chain
(Router -> AIHandler -> Service -> PatrolService)
- Persists baselines to data directory as JSON
The baseline learning loop:
1. Starts automatically when AI patrol starts
2. Queries metrics history for all resources
3. Computes mean, stddev, percentiles for cpu/memory/disk
4. Saves baselines to disk for durability
5. Anomaly detection uses these baselines in context builder
All tests passing.
Phase 2 of Pulse AI differentiation:
- Create internal/ai/baseline package for learned baselines
- Implement statistical baseline learning with mean, stddev, percentiles
- Add z-score based anomaly detection with severity classification
(low, medium, high, critical based on standard deviations)
- Integrate baseline provider into context builder
- Wire baseline store into patrol service with adapters
- Add anomaly enrichment to resource contexts
Key features:
- Learn computes baseline from historical metric data points
- IsAnomaly and CheckAnomaly detect deviations from normal
- Persists baselines to disk as JSON for durability
- Formatted anomaly descriptions for AI consumption
Example: 'Memory is high above normal (85.2% vs typical 42.1% ± 8.3%)'
The baseline store needs to be initialized and triggered to learn
from metrics history. Next step is adding the learning loop.
All tests passing.
Phase 1 of Pulse AI differentiation:
- Create internal/ai/context package with types, trends, builder, formatter
- Implement linear regression for trend computation (growing/declining/stable/volatile)
- Add storage capacity predictions (predicts days until 90% and 100%)
- Wire MetricsHistory from monitor to patrol service
- Update patrol to use buildEnrichedContext instead of basic summary
- Update patrol prompt to reference trend indicators and predictions
This gives the AI awareness of historical patterns, enabling it to:
- Identify resources with concerning growth rates
- Predict capacity exhaustion before it happens
- Distinguish between stable high usage vs growing problems
- Provide more actionable, time-aware insights
All tests passing. Falls back to basic summary if metrics history unavailable.
- Add exponential backoff retry for Docker agent startup (main.go)
- Fix Docker resource/image column widths with proper truncation
- Unify IP tooltip styling across hosts and guests with detailed network info
- Improve column visibility defaults and sticky column handling
- Various component refinements for Dashboard, Storage, and Backups views
- Changed patrol schedule from preset dropdown to freeform number input
- Users can now set any interval (min 10 minutes, max 7 days, or 0 to disable)
- Added patrol_interval_minutes to API request/response (preset is now deprecated)
- Backend validates: min 10 minutes when enabled, max 10080 (7 days)
- Frontend shows human-readable duration next to input (e.g., '6h', '2h 30m')
Also improved Auto-Fix Mode safety:
- Removed '(recommended)' from preset options (was subjective)
- Added 'I understand the risks' acknowledgement checkbox
- Toggle is disabled until user explicitly acknowledges the risks
- Shows prominent warning when Auto-Fix is enabled
- Acknowledgement is session-based (must re-acknowledge on page reload)
- Add 'content' type to StreamDisplayEvent for tracking text chunks
- Track content events in streamEvents array for chronological display
- Update render to use Switch/Match for cleaner conditional rendering
- Interleave thinking, tool calls, and content as they stream in
- Add fallback for old messages without streamEvents for backwards compat
Previously, tool/command outputs stayed at top while AI text responses
accumulated at the bottom. Now all events appear in order like a
normal chatbot.
- Remove model references from provider labels ('OpenAI' not 'OpenAI (GPT-4)')
- Remove DEFAULT_MODELS usage in form initialization
- Use generic placeholders instead of specific model names
- Models are now fetched dynamically from each provider's API
- UI won't become outdated when new models are released
- Show number of configured providers and available models
- Display friendly model name (without provider prefix)
- Better status message: 'Ready • 1 10 models'provider
- Adds 'Refresh Models' button next to Default Model label
- Spinning icon animation during loading
- Allows manual refresh after configuring new providers
- Add clear_anthropic_key, clear_openai_key, clear_deepseek_key, clear_ollama_url flags to API
- Backend handles clearing with confirmation prompt
- Each provider accordion shows Test and Clear buttons when configured
- Clear button requires confirmation before removing credentials
- Frontend automatically refreshes settings after clearing
- Add /api/ai/test/{provider} endpoint for testing individual providers
- Add 'Test' button to each provider accordion (visible when configured)
- Shows test result inline (success/error message)
- Update help links with direct URLs to API key pages:
- Anthropic: console.anthropic.com/settings/keys
- OpenAI: platform.openai.com/api-keys
- DeepSeek: platform.deepseek.com/api_keys
- Ollama: ollama.ai
Backend:
- Add per-provider API key fields to AIConfig (AnthropicAPIKey, OpenAIAPIKey, DeepSeekAPIKey, OllamaBaseURL, OpenAIBaseURL)
- Add NewForProvider() and NewForModel() factory functions for multi-provider instantiation
- Update ListModels() to aggregate models from all configured providers with provider:model format
- Update Execute/ExecuteStream to dynamically create provider based on selected model
- Update TestConnection to use multi-provider aware provider creation
- Add helper functions: HasProvider(), GetConfiguredProviders(), GetAPIKeyForProvider(), GetBaseURLForProvider(), ParseModelString(), FormatModelString()
Frontend:
- Remove legacy single-provider UI (provider grid, single API key input, single base URL)
- Add accordion-style UI for configuring all providers independently
- Add model grouping by provider in selectors using optgroup
- Update AIChat model dropdown with grouped provider sections
- Add helper functions for parsing provider from model ID and grouping models
API:
- Add multi-provider fields to AISettingsResponse and AISettingsUpdateRequest
- Add /api/ai/models endpoint for dynamic model listing
- Update settings handlers for per-provider credential management
Users can now:
1. View all active suppression rules in a collapsible section
2. Add new rules manually with resource ID, category, and description
3. Delete rules to re-enable alerts
4. See whether rules came from dismissed findings or were manually created
The UI shows:
- 🔇 Suppression Rules (N active) header with expand/collapse
- + Add Rule button to open the form
- Each rule shows resource, category, origin (Manual/From Finding), and description
- Delete button to remove rules
Users can now:
1. View all suppression rules (both from dismissed findings and manually created)
2. Create manual rules like 'ignore performance issues on debian-go'
3. Delete rules when they want alerts to come back
Backend:
- Added SuppressionRule type for user-defined rules
- Added suppressionRules storage to FindingsStore
- Added AddSuppressionRule/GetSuppressionRules/DeleteSuppressionRule methods
- Added isSuppressedInternal check for manual rules
- Added API handlers and routes for /api/ai/patrol/suppressions
Frontend:
- Added SuppressionRule interface
- Added getSuppressionRules/addSuppressionRule/deleteSuppressionRule API functions
- Added getDismissedFindings for viewing dismissed findings
Example usage:
POST /api/ai/patrol/suppressions
{
'resource_id': 'debian-go',
'category': 'performance',
'description': 'Dev container runs hot - expected'
}
The main issue was that finding IDs included the title, which the LLM
generates differently each time. 'High CPU on minipc' vs 'Node minipc
experiencing high CPU load' got different IDs, making dismissals useless.
Changes:
1. LLM findings now get IDs based on resource+category only, not title
2. Add() now checks if finding is suppressed before adding as new
3. Add() now checks dismissed findings and only reactivates on severity escalation
4. IsSuppressed() now matches by resource+category only, not title
5. Added isSuppressedInternal() for use when lock is already held
Now when you dismiss 'performance issues on minipc', any future patrol finding
about performance on minipc will be recognized as the same issue and stay dismissed.
The ForcePatrol() function was using the HTTP request context, which gets
cancelled immediately when the API response is sent. This caused LLM analysis
to fail with 'context canceled' before it could complete.
Now uses context.Background() so the goroutine runs independently of the
HTTP request lifecycle.
Also fixed dropdown hover gap issue in the dismiss menu.
Knowledge store notes are now included in the patrol LLM prompt. When users
save notes about resources (e.g., 'This VM intentionally runs hot'), the patrol
AI will see these notes and avoid flagging documented behavior as issues.
Changes:
- Added knowledge store reference to PatrolService
- Added SetKnowledgeStore() method to configure the store
- Enhanced buildPatrolPrompt() to include knowledge context
- Connected knowledge store to patrol in service.go SetStateProvider()
This complements the dismissed findings context to give the LLM a complete
picture of what the user considers normal/expected behavior.
Implements a comprehensive feedback system that allows the LLM to 'remember'
user decisions about findings, preventing repetitive/annoying alerts.
Backend changes:
- Extended Finding struct with dismissed_reason, user_note, times_raised, suppressed
- Added Dismiss(), Suppress(), SetUserNote(), IsSuppressed() methods to FindingsStore
- Added GetDismissedForContext() to format dismissed findings for LLM context
- Enhanced buildPatrolPrompt() to inject user feedback context
- Added POST /api/ai/patrol/dismiss and /api/ai/patrol/suppress endpoints
- Updated IsActive() to exclude suppressed findings
Frontend changes:
- Added Dismiss dropdown with options: Not an Issue, Expected Behavior, Will Fix Later
- Added Never Alert Again option for permanent suppression
- Expected Behavior prompts for optional note to help LLM understand context
- Added visual badges: recurrence count (×N), dismissed status, suppressed indicator
- Display user notes in expanded finding view
Also fixes:
- Fixed 403 error on Run Patrol (compilation errors from partial refactoring)
- Removed non-LLM patrol checks - patrol now uses LLM analysis only
- Fixed function signature mismatches in alert_triggered.go
The LLM now receives context about previously dismissed findings and is
instructed not to re-raise them unless severity has significantly worsened.
- Add alert-triggered AI analysis for real-time incident response
- Implement patrol history persistence across restarts
- Add patrol schedule configuration UI in AI Settings
- Enhance AIChat with patrol status and manual trigger controls
- Add resource store improvements for AI context building
- Expand Alerts page with AI-powered analysis integration
- Add Vite proxy config for AI API endpoints
- Support both Anthropic and OpenAI providers with streaming
The acquire() function blocked indefinitely without respecting context
cancellation. When clients disconnect while waiting for the per-node
lock, goroutines would remain blocked forever, connections accumulate
in CLOSE_WAIT state, and rate limiter semaphores are never released.
Added acquireContext() that respects context cancellation and updated
both HTTP and RPC handlers to use it. This prevents:
- Goroutine leaks from cancelled requests
- CLOSE_WAIT connection accumulation
- Cascading failures from filled semaphores
Related to #832
Keep only the simple AI-powered approach:
- set_resource_url tool lets AI save discovered URLs
- Users ask AI directly: 'Find URLs for my containers'
- AI uses its intelligence to discover and set URLs
Removed:
- URLDiscoveryService (rigid port scanning)
- Bulk discovery API endpoints
- Frontend discovery button
The AI itself is smart enough to iterate through resources
and discover URLs when asked.
- Add URLDiscoveryService for scanning all resources at once
- Scans common web ports (80, 443, 8080, 8096, 3000, etc.)
- Automatically saves discovered URLs to resource metadata
- Add API endpoints for start/status/cancel discovery
- Progress tracking with results reporting
Endpoints:
- POST /api/ai/discover-urls/start - Start bulk discovery
- GET /api/ai/discover-urls/status - Check progress
- POST /api/ai/discover-urls/cancel - Cancel discovery
- Add MetadataProvider interface for AI to update resource URLs
- Add set_resource_url tool to AI service
- Wire up metadata stores to AI service via router
- Add URL discovery guidance to AI system prompt
- AI can now inspect guests/containers/hosts for web services
and automatically save discovered URLs to Pulse metadata
Usage: Ask the AI 'Find the web URL for this container' and it will:
1. Check for listening ports and web servers
2. Get the IP address
3. Verify the URL works
4. Save it to Pulse for quick dashboard access
- Add host metadata API for custom URL editing on hosts page
- Enhance AI routing with unified resource provider lookup
- Add encryption key watcher script for debugging key issues
- Improve AI service with better command timeout handling
- Update dev environment workflow with key monitoring docs
- Fix resource store deduplication logic
- Add Claude OAuth authentication support with hybrid API key/OAuth flow
- Implement Docker container historical metrics in backend and charts API
- Add CEPH cluster data collection and new Ceph page
- Enhance RAID status display with detailed tooltips and visual indicators
- Fix host deduplication logic with Docker bridge IP filtering
- Fix NVMe temperature collection in host agent
- Add comprehensive test coverage for new features
- Improve frontend sparklines and metrics history handling
- Fix navigation issues and frontend reload loops
Replaced 18+ console.log statements in AI-related files with
logger.debug/warn/error calls. This ensures debug output only
appears in development mode, keeping production logs clean.
Files updated:
- frontend-modern/src/api/ai.ts (15 statements)
- frontend-modern/src/components/AI/AIChat.tsx (4 statements)
These were internal planning/architecture docs not meant for end users:
- .gemini/docs/unified-resource-architecture.md (design doc)
- .gemini/tasks/persistent-metrics-storage.md (implementation plan)
- frontend-modern/PLAN-column-visibility.md (implementation plan)
The AI service now uses only buildUnifiedResourceContext() for
infrastructure context, since the resourceProvider is always set
during router initialization.
Removed:
- buildInfrastructureContext() function (~288 lines of dead code)
- Legacy fallback path in buildSystemPrompt()
The unified resource context provides a cleaner, deduplicated view
of infrastructure that includes:
- All resources grouped by platform and type
- Top CPU/Memory/Disk consumers
- Active alerts on resources
- Infrastructure summary statistics
This completes the AI service migration to unified resources.
This cleanup addresses transition debt from the unified resources migration:
Frontend cleanup:
- Move all Resource→Legacy type conversions to useResourcesAsLegacy() hook
- Add asNodes() and asDockerHosts() adapter functions to the hook
- Simplify DockerRoute, HostsRoute, DashboardView to use the centralized hook
- Remove ~300 lines of duplicated adapter code from App.tsx
- Remove debug console.log statements from Dashboard.tsx
- Fix CPU value conversion (divide by 100) for Dashboard compatibility
Backend fixes (from previous session):
- Fix parentID format in converters (VM, Container, Storage) to match Node.ID
- Format changed from 'instance/node/nodename' to 'instance-nodename'
- Update tests to match new parentID format
This consolidates all legacy type conversion logic in one place,
making future cleanup easier when components are migrated to use
unified resources directly.
The Dashboard grouping was broken because:
- node was set to r.parentId (full resource ID)
- instance was set to r.platformId
Fixed to read from platformData which contains the correct values:
- node = platformData.node (e.g., 'minipc')
- instance = platformData.instance (e.g., 'https://pve:8006')
This matches the legacy data format and fixes the grouped/list toggle.
BREAKING: Route components no longer fall back to legacy state arrays.
All data now flows through the unified resource model:
- DockerRoute: uses state.resources filtered for docker-host/docker-container
- HostsRoute: uses state.resources filtered for host
- DashboardView: uses state.resources filtered for node/vm/container
The legacy arrays (state.nodes, state.vms, etc.) are still broadcast
by the backend for API compatibility, but the main UI routes no longer
use them.
If resources array is empty, pages will show no data rather than
falling back to legacy data. This ensures a clean data model with
no hidden fallback behavior.
Backend:
- Call SetMonitor after router creation to inject resource store
- Add debug logging for resource population and broadcast
Frontend:
- Add resources array to WebSocket store initial state
- Handle resources in WebSocket message processing
- Use reconcile for efficient state updates
The unified resources are now properly:
1. Populated from StateSnapshot on each broadcast cycle
2. Converted to frontend format (ResourceFrontend)
3. Included in WebSocket state messages
4. Received and stored in frontend state
5. Consumed by migrated route components
Console now shows '[DashboardView] Using unified resources: VMs: X'
confirming the migration is working end-to-end.
Temporary logging to verify which code path is being used:
- DockerRoute: logs docker-host and docker-container counts
- HostsRoute: logs host count
- DashboardView: logs VM count
Check browser console to confirm unified resources are being received.
These logs can be removed once migration is verified.
