docs: simplify Mermaid diagrams for better readability
The previous diagrams were too complex and overwhelming. Simplified all diagrams to show core concepts clearly: - Adaptive polling: reduced to basic scheduler→queue→workers flow - Temperature proxy: simplified to 3-box trust boundary view - Sensor proxy sequence: simplified to essential request flow - Webhook pipeline: reduced to template→send→retry flow - Script library: simplified to code→test→bundle→dist flow Fixed parsing error in temperature proxy diagram (parentheses in edge label causing render failure). Diagrams should clarify architecture, not recreate implementation.
This commit is contained in:
rcourtman
parent
7bfd6997ec
commit
2f43d67af9
5 changed files with 43 additions and 144 deletions
|
|
@ -19,37 +19,18 @@ This document describes the security architecture of Pulse's temperature monitor
|
|||
|
||||
```mermaid
|
||||
graph TD
|
||||
subgraph Host["Proxmox Host (delly)\nTrust Boundary"]
|
||||
Proxy["pulse-sensor-proxy service\nUID 999\nSO_PEERCRED auth\nMethod ACL + per-UID rate limit\nPer-node concurrency = 1"]
|
||||
Socket["Unix socket\n/run/pulse-sensor-proxy.sock\n(0600 bind mount)"]
|
||||
Audit["Audit & Metrics\n/var/log/pulse/... & :9127/metrics"]
|
||||
PrivOps["Privileged RPCs\nensure_cluster_keys | register_nodes | request_cleanup\nHost UID only"]
|
||||
end
|
||||
Container[Pulse Container]
|
||||
Proxy[pulse-sensor-proxy<br/>Host Service]
|
||||
Cluster[Cluster Nodes<br/>SSH sensors -j]
|
||||
|
||||
subgraph Container["Pulse Container (ID-mapped root)"]
|
||||
Backend["Pulse Backend"]
|
||||
Poller["Temperature Poller worker"]
|
||||
end
|
||||
Container -->|Unix Socket<br/>Rate Limited| Proxy
|
||||
Proxy -->|SSH<br/>Forced Command| Cluster
|
||||
Cluster -->|Temperature JSON| Proxy
|
||||
Proxy -->|Temperature JSON| Container
|
||||
|
||||
subgraph Cluster["Cluster Nodes"]
|
||||
SensorCmd["Forced SSH command\n`sensors -j` only\nRestricted authorized_keys entry"]
|
||||
end
|
||||
|
||||
Poller -->|poll request| Backend
|
||||
Backend -->|RPC via bind-mounted socket| Socket
|
||||
Socket --> Proxy
|
||||
Proxy -->|temperature JSON response| Backend
|
||||
Proxy -->|rate-limit reject + 2 s penalty| Reject["429 response"]
|
||||
Reject --> Backend
|
||||
|
||||
Proxy -->|SSH (ed25519 key)\nforced command| SensorCmd
|
||||
SensorCmd -->|temperature JSON| Proxy
|
||||
|
||||
Proxy -->|audit entry + metrics| Audit
|
||||
Audit -->|Prometheus scrape| Metrics["Telemetry Consumers\n(Grafana, watchdog)"]
|
||||
|
||||
PrivOps --> Proxy
|
||||
Backend -. blocked (ID-mapped root) .-> PrivOps
|
||||
style Proxy fill:#e1f5e1
|
||||
style Container fill:#fff4e1
|
||||
style Cluster fill:#e1f0ff
|
||||
```
|
||||
|
||||
**Key Principle**: SSH keys never enter containers. All SSH operations are performed by the host-side proxy.
|
||||
|
|
|
|||
|
|
@ -114,31 +114,15 @@ For webhooks that require authentication or custom headers:
|
|||
## Custom Payload Templates
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
AlertEvent["Alert Event Triggered"]
|
||||
GatherData["Gather Alert Data\n(Level, Type, Resource, Node, etc.)"]
|
||||
ResolveURL["Resolve URL Template\n({{urlpath}}, {{urlquery}}"]
|
||||
ResolvePayload["Resolve Payload Template\n(variable substitution)"]
|
||||
ApplyFunctions["Apply Template Functions\n(title, upper, lower, printf)"]
|
||||
Dispatch["HTTP POST Request"]
|
||||
CheckResponse{"Response\nStatus?"}
|
||||
Success["200-299: Success\nLog delivery"]
|
||||
Retry["429/5xx: Retry\n(exponential backoff)"]
|
||||
Failure["4xx: Failure\nLog error"]
|
||||
TrackDelivery["Update Delivery Metrics\npulse_webhook_deliveries_total"]
|
||||
flowchart LR
|
||||
Alert[Alert Triggered]
|
||||
Template[Render Template<br/>Variables & Functions]
|
||||
Send[HTTP POST]
|
||||
|
||||
AlertEvent --> GatherData
|
||||
GatherData --> ResolveURL
|
||||
ResolveURL --> ResolvePayload
|
||||
ResolvePayload --> ApplyFunctions
|
||||
ApplyFunctions --> Dispatch
|
||||
Dispatch --> CheckResponse
|
||||
CheckResponse -->|Success| Success
|
||||
CheckResponse -->|Transient Error| Retry
|
||||
CheckResponse -->|Permanent Error| Failure
|
||||
Success --> TrackDelivery
|
||||
Retry --> TrackDelivery
|
||||
Failure --> TrackDelivery
|
||||
Alert --> Template
|
||||
Template --> Send
|
||||
Send -->|2xx| Success[Delivered]
|
||||
Send -->|5xx| Retry[Retry with Backoff]
|
||||
```
|
||||
|
||||
For generic webhooks, you can define custom JSON payloads using Go template syntax.
|
||||
|
|
|
|||
|
|
@ -4,45 +4,16 @@
|
|||
Pulse uses an adaptive polling scheduler that adapts poll cadence based on freshness, errors, and workload. The goal is to prioritize stale or changing instances while backing off on healthy, idle targets.
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
PollLoop["PollLoop\n(ticker & config updates)"]
|
||||
Scheduler["Scheduler\ncomputes ScheduledTask"]
|
||||
Staleness["Staleness Tracker\n(last success, freshness score)"]
|
||||
CircuitBreaker["Circuit Breaker\ntracks failure streaks"]
|
||||
Backoff["Backoff Policy\nexponential w/ jitter"]
|
||||
PriorityQ["Priority Queue\nmin-heap by NextRun"]
|
||||
WorkerPool["TaskWorkers\nN concurrent workers"]
|
||||
Metrics["Metrics & History\nPrometheus + retention"]
|
||||
Success["Poll Success"]
|
||||
Failure{"Poll Failure?"}
|
||||
Reschedule["Reschedule\n(next interval)"]
|
||||
BackoffPath["Backoff / Breaker Open"]
|
||||
DeadLetter["Dead-Letter Queue\noperator review"]
|
||||
flowchart LR
|
||||
Scheduler[Scheduler]
|
||||
Queue[Priority Queue<br/>by NextRun]
|
||||
Workers[Workers]
|
||||
|
||||
PollLoop --> Scheduler
|
||||
Staleness --> Scheduler
|
||||
CircuitBreaker --> Scheduler
|
||||
Scheduler --> PriorityQ
|
||||
|
||||
PriorityQ -->|due task| WorkerPool
|
||||
WorkerPool --> Failure
|
||||
WorkerPool -->|result| Metrics
|
||||
WorkerPool -->|freshness| Staleness
|
||||
|
||||
Failure -->|No| Success
|
||||
Success --> CircuitBreaker
|
||||
Success --> Reschedule
|
||||
Success --> Metrics
|
||||
Reschedule --> Scheduler
|
||||
|
||||
Failure -->|Yes| BackoffPath
|
||||
BackoffPath --> CircuitBreaker
|
||||
BackoffPath --> Backoff
|
||||
Backoff --> Scheduler
|
||||
Backoff --> DeadLetter
|
||||
DeadLetter -. periodic retry .-> Scheduler
|
||||
CircuitBreaker -. state change .-> Scheduler
|
||||
Metrics --> Scheduler
|
||||
Scheduler -->|schedule| Queue
|
||||
Queue -->|dequeue| Workers
|
||||
Workers -->|success| Scheduler
|
||||
Workers -->|failure| CB[Circuit Breaker]
|
||||
CB -->|backoff| Scheduler
|
||||
```
|
||||
|
||||
- **Scheduler** computes `ScheduledTask` entries using adaptive intervals.
|
||||
|
|
|
|||
|
|
@ -12,36 +12,15 @@
|
|||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Backend as Pulse Backend
|
||||
participant Proxy as Sensor Proxy RPC Server
|
||||
participant Limiter as Limiter (per UID & global)
|
||||
participant Validator as Payload Validator
|
||||
participant SSH as Cluster Node (forced `sensors -j`)
|
||||
participant Metrics as Metrics & Audit Log
|
||||
participant Backend
|
||||
participant Proxy
|
||||
participant Node
|
||||
|
||||
Backend->>Proxy: RPC request (get_temperature)
|
||||
Proxy->>Proxy: Extract SO_PEERCRED (UID/GID/PID)
|
||||
Proxy->>Limiter: Check per-UID rate & concurrency
|
||||
alt Rate limit exceeded
|
||||
Limiter-->>Proxy: reject
|
||||
Proxy-->>Backend: 429 Too Many Requests (2 s penalty)
|
||||
Proxy->>Metrics: increment limiter_rejections_total
|
||||
else Allowed
|
||||
Limiter-->>Proxy: permit
|
||||
Proxy->>Validator: Validate method & payload
|
||||
alt Validation failure
|
||||
Validator-->>Proxy: error
|
||||
Proxy-->>Backend: 400 validation error
|
||||
Proxy->>Metrics: penalty + audit log entry
|
||||
else Valid request
|
||||
Validator-->>Proxy: ok
|
||||
Proxy->>SSH: run `sensors -j` via forced command
|
||||
SSH-->>Proxy: temperature JSON
|
||||
Proxy-->>Backend: telemetry payload
|
||||
Proxy->>Metrics: record success, latency histogram
|
||||
Proxy->>Metrics: append audit/audit trail
|
||||
end
|
||||
end
|
||||
Backend->>Proxy: get_temperature
|
||||
Proxy->>Proxy: Check rate limit
|
||||
Proxy->>Node: SSH sensors -j
|
||||
Node->>Proxy: JSON response
|
||||
Proxy->>Backend: Temperature data
|
||||
```
|
||||
|
||||
### Rate Limit Hits (`pulse_proxy_limiter_rejections_total`)
|
||||
|
|
|
|||
|
|
@ -34,31 +34,15 @@ dist/ # Generated bundled scripts
|
|||
### Development & Bundling Workflow
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Author["Author Code\nscripts/lib/*.sh\nscripts/install-*.sh"]
|
||||
WriteTests["Write Tests\nscripts/tests/test-*.sh\nscripts/tests/integration/"]
|
||||
UpdateManifest["Update Bundle Manifest\nscripts/bundle.manifest"]
|
||||
RunTests["Run Tests\nmake test-scripts\nscripts/tests/run.sh"]
|
||||
TestPass{"Tests Pass?"}
|
||||
FixCode["Fix Issues"]
|
||||
Bundle["Bundle Scripts\nmake bundle-scripts\nbash scripts/bundle.sh"]
|
||||
ValidateBundled["Validate Bundled Output\nbash -n dist/*.sh\ndist/*.sh --dry-run"]
|
||||
ValidatePass{"Validation\nPass?"}
|
||||
Distribute["Distribute\ndist/*.sh ready"]
|
||||
UpdateDocs["Update Documentation\nscripts/lib/README.md"]
|
||||
flowchart LR
|
||||
Code[Write Code<br/>scripts/lib]
|
||||
Test[Run Tests]
|
||||
Bundle[Bundle<br/>make bundle-scripts]
|
||||
Dist[Distribute<br/>dist/*.sh]
|
||||
|
||||
Author --> WriteTests
|
||||
WriteTests --> UpdateManifest
|
||||
UpdateManifest --> RunTests
|
||||
RunTests --> TestPass
|
||||
TestPass -->|No| FixCode
|
||||
FixCode --> Author
|
||||
TestPass -->|Yes| Bundle
|
||||
Bundle --> ValidateBundled
|
||||
ValidateBundled --> ValidatePass
|
||||
ValidatePass -->|No| FixCode
|
||||
ValidatePass -->|Yes| UpdateDocs
|
||||
UpdateDocs --> Distribute
|
||||
Code --> Test
|
||||
Test --> Bundle
|
||||
Bundle --> Dist
|
||||
```
|
||||
|
||||
This workflow emphasizes the library's modular design: develop reusable modules in `scripts/lib`, test thoroughly, bundle for distribution, and validate bundled artifacts before release.
|
||||
|
|
|
|||
Loading…
Reference in a new issue