chore: remove unnecessary development files and docs
- Remove CLEANUP_TODO.md and MIGRATION_SCAFFOLDING.md (internal notes) - Remove temporary scripts: copy_and_run.sh, work.sh - Remove AI assistant utility scripts: backup-claude-md.sh, codex-router.sh These files were used during development but don't belong in the repository.
This commit is contained in:
courtmanr@gmail.com
parent
9fdea3c9cd
commit
a17cd53111
6 changed files with 0 additions and 367 deletions
209
CLEANUP_TODO.md
209
CLEANUP_TODO.md
|
|
@ -1,209 +0,0 @@
|
|||
# Node Cleanup Implementation - CODEX REVIEW COMPLETE
|
||||
|
||||
**Status**: ✅ Implementation complete, Codex review passed, ready for deployment testing.
|
||||
|
||||
**Latest Update**: Addressed all 8 critical issues found by Codex review (conv-1763166192078-1076).
|
||||
|
||||
## Implementation History
|
||||
|
||||
**Commit b192c60e9**: Relocate binaries to /opt/pulse/sensor-proxy/
|
||||
- ✅ Binary path moved from /usr/local/bin to /opt/pulse/sensor-proxy/bin
|
||||
- ✅ All systemd ExecStart paths updated
|
||||
- ✅ Guarantees cleanup works on read-only /usr systems
|
||||
|
||||
**Commit 6692228e0**: Full cleanup script implementation
|
||||
- ✅ SSH key removal (working)
|
||||
- ✅ Service uninstallation (via isolated systemd-run transient unit)
|
||||
- ✅ API token deletion (JSON first, filtered table fallback)
|
||||
- ✅ Bind mount removal (scans all LXC configs)
|
||||
- ✅ pulse-monitor user deletion
|
||||
- ✅ flock serialization (prevents concurrent runs)
|
||||
- ✅ Immediate request file deletion (prevents loops)
|
||||
|
||||
**Commits ed48d7555, 17d2e6876**: Bug fixes during testing
|
||||
- ✅ Fixed directory creation order (create before binary download)
|
||||
- ✅ Fixed SHARE_DIR unbound variable
|
||||
|
||||
**Commit bcd8d4e0f**: Critical Codex review fixes #1-4
|
||||
- ✅ Host detection now includes hostname/FQDN (not just IP)
|
||||
- ✅ Systemd sandbox relaxed (/etc/pve and /etc/systemd/system writable)
|
||||
- ✅ Uninstaller called with --purge flag for complete removal
|
||||
- ✅ All /usr/local references migrated to /opt paths
|
||||
- ✅ UUID used for transient unit names (prevents collisions)
|
||||
- ✅ --wait and --collect flags capture uninstaller exit code
|
||||
|
||||
**Commit fe53d6473**: Remaining Codex review fixes #5-8
|
||||
- ✅ LXC bind mounts removed via `pct set -delete` (not sed)
|
||||
- ✅ API token parsing: three-tier fallback (pveum JSON → pvesh JSON → table)
|
||||
- ✅ Retry logic: rename to .processing, delete only on success
|
||||
|
||||
## Issues Discovered and Resolved
|
||||
|
||||
### 1. Host Detection Failure (CRITICAL) - ✅ FIXED
|
||||
**Problem**: Cleanup script only compared against IPs from `hostname -I`, missing nodes configured as `https://hostname:8006`. This caused localhost cleanup (API tokens, bind mounts, service removal) to be skipped entirely.
|
||||
|
||||
**Fix**: Check against `hostname`, `hostname -f`, and all IPs. Now catches all localhost variations.
|
||||
|
||||
**Impact**: Without this fix, cleanup would only remove remote SSH keys, leaving services/binaries/tokens intact.
|
||||
|
||||
### 2. Systemd Sandbox Blocked Critical Operations - ✅ FIXED
|
||||
**Problem**: Cleanup service ran with `ProtectSystem=strict` and `ReadWritePaths=/var/lib/pulse-sensor-proxy /root/.ssh`, blocking writes to `/etc/pve` (Proxmox configs) and `/etc/systemd/system`.
|
||||
|
||||
**Fix**: Added `/etc/pve` and `/etc/systemd/system` to `ReadWritePaths`.
|
||||
|
||||
**Impact**: `pveum` token deletion and `pct set -delete` would fail with "read-only file system".
|
||||
|
||||
### 3. Incomplete Purging - ✅ FIXED
|
||||
**Problem**: Uninstaller called without `--purge`, leaving `/var/lib/pulse-sensor-proxy`, service user, and SSH private keys on disk. Request file deleted before work completed, preventing retry on failure.
|
||||
|
||||
**Fix**: Added `--purge` flag, added `--wait --collect` to capture exit code, fail cleanup if uninstaller fails.
|
||||
|
||||
**Impact**: Claimed "cleanup completed successfully" even when artifacts remained.
|
||||
|
||||
### 4. Incomplete Path Migration - ✅ FIXED
|
||||
**Problem**: After relocating binaries to `/opt`, three references still pointed to `/usr/local`:
|
||||
- Forced command in SSH authorized_keys: `/usr/local/bin/pulse-sensor-wrapper.sh`
|
||||
- Self-heal script: `/usr/local/share/pulse/install-sensor-proxy.sh`
|
||||
- Backend removal helpers: `/usr/local/bin/pulse-sensor-cleanup.sh`
|
||||
|
||||
**Fix**: Updated all three locations. Go backend now checks both paths (new + legacy).
|
||||
|
||||
**Impact**: New installs would lose telemetry immediately (SSH command not found). UI-triggered cleanup wouldn't find helpers.
|
||||
|
||||
### 5. Transient Unit Name Collisions - ✅ FIXED
|
||||
**Problem**: Used `date +%s` for unit names. If two cleanup requests fired within same second, second would fail (unit already exists). Error suppressed, logged as "started" anyway.
|
||||
|
||||
**Fix**: Use `/proc/sys/kernel/random/uuid` for unique names.
|
||||
|
||||
**Impact**: Multiple concurrent cleanups could race, with silent failures.
|
||||
|
||||
### 6. Bind Mount Removal Too Broad - ✅ FIXED
|
||||
**Problem**: Used `sed -i '/pulse-sensor-proxy/d'` which would delete ANY line mentioning the substring (including unrelated comments/hooks). Also couldn't run inside systemd sandbox.
|
||||
|
||||
**Fix**: Use `pct set <ctid> -delete mp<N>` which validates syntax and is sandbox-compatible.
|
||||
|
||||
**Impact**: Could break container configs. Sed approach would fail anyway due to sandbox.
|
||||
|
||||
### 7. API Token Parsing Fragile - ✅ FIXED
|
||||
**Problem**: Table parser filtered only `│┌└╞`, failing on other Unicode borders or locales. "User not found" vs "feature unsupported" both treated as "no tokens".
|
||||
|
||||
**Fix**: Three-tier fallback:
|
||||
1. `pveum --output-format json` with python3 parsing
|
||||
2. `pvesh get /access/users/pulse-monitor@pam/token` (always JSON)
|
||||
3. Improved table parser with better filtering
|
||||
|
||||
**Impact**: Non-English locales or Proxmox versions with different table formatting would silently skip token cleanup.
|
||||
|
||||
### 8. No Retry on Failure - ✅ FIXED
|
||||
**Problem**: Request file deleted immediately. Any crash left no way to retry.
|
||||
|
||||
**Fix**: Rename to `.processing`, delete only on success. Failures leave `.processing` file for manual investigation/retry.
|
||||
|
||||
**Impact**: Transient failures (network issues, systemd hiccups) couldn't be retried automatically.
|
||||
|
||||
## Read-Only Filesystem
|
||||
|
||||
**Problem**: Proxmox VE can mount `/usr` as read-only (hardened setups, boot-from-snapshots, appliance builds).
|
||||
|
||||
**Solution**: All binaries relocated to `/opt/pulse/sensor-proxy/` where we control permissions.
|
||||
|
||||
## Process Isolation During Uninstall
|
||||
|
||||
**Problem**: Cleanup script runs as systemd service. When it calls the uninstaller which stops the proxy service, systemd kills the cleanup service with SIGTERM.
|
||||
|
||||
**Solution**: Use transient systemd unit with:
|
||||
- UUID-based unique name
|
||||
- `Conflicts=pulse-sensor-proxy.service`
|
||||
- `--wait --collect` to capture exit code
|
||||
- `--purge` flag for complete removal
|
||||
|
||||
## Implementation Status
|
||||
|
||||
### Phase 1: Relocate Binaries ✅ COMPLETE
|
||||
- ✅ Update installer to use `/opt/pulse/sensor-proxy/bin/`
|
||||
- ✅ Update systemd unit ExecStart path
|
||||
- ✅ Update cleanup script to use new paths
|
||||
- ✅ Update forced command in SSH authorized_keys
|
||||
- ✅ Update self-heal script paths
|
||||
- ✅ Update Go backend removal helpers (supports both new and legacy paths)
|
||||
|
||||
### Phase 2: Fix Uninstall Orchestration ✅ COMPLETE
|
||||
- ✅ Cleanup script spawns transient systemd unit via systemd-run
|
||||
- ✅ Added `Conflicts=pulse-sensor-proxy.service` to transient unit
|
||||
- ✅ UUID-based unit names prevent collisions
|
||||
- ✅ Added `--wait --collect` to capture exit code
|
||||
- ✅ Uninstaller runs with `--purge --quiet` flags
|
||||
- ✅ Cleanup fails if uninstaller exits non-zero
|
||||
|
||||
### Phase 3: Prevent Cleanup Loops ✅ COMPLETE
|
||||
- ✅ Added `flock` serialization via exec 200>lockfile
|
||||
- ✅ Rename request file to `.processing` (allows retry on failure)
|
||||
- ✅ Delete `.processing` only on successful completion
|
||||
- ✅ `.path` unit uses PathChanged/PathModified (correct semantics)
|
||||
- ✅ Comprehensive logging at info/warn/error levels
|
||||
|
||||
### Phase 4: Improve API Token Handling ✅ COMPLETE
|
||||
- ✅ Three-tier fallback: pveum JSON → pvesh JSON → table parsing
|
||||
- ✅ Python3 JSON parsing for robustness
|
||||
- ✅ Better table filtering (handles more Unicode characters)
|
||||
- ✅ Error handling for token deletion failures (logs warnings, continues)
|
||||
- ✅ Logs each token removal attempt
|
||||
|
||||
### Phase 5: LXC Bind Mount Removal ✅ COMPLETE
|
||||
- ✅ Use `pct set -delete` instead of sed (validates syntax)
|
||||
- ✅ Sandbox-compatible (works with ProtectSystem=strict + /etc/pve writable)
|
||||
- ✅ Only removes mounts containing "pulse-sensor-proxy"
|
||||
|
||||
### Phase 6: Host Detection ✅ COMPLETE
|
||||
- ✅ Check against hostname, FQDN, and all local IPs
|
||||
- ✅ Localhost detection works for `https://hostname:8006` URLs
|
||||
- ✅ Ensures full cleanup runs for all localhost variations
|
||||
|
||||
### Phase 7: Testing & Validation ⏸️ PENDING
|
||||
- ⏸️ Deploy updated installer to test host
|
||||
- ⏸️ Test node removal via Pulse UI
|
||||
- ⏸️ Verify complete cleanup:
|
||||
- ⏸️ No systemd units
|
||||
- ⏸️ No binaries in /opt or /usr/local
|
||||
- ⏸️ No bind mounts in LXC configs
|
||||
- ⏸️ No API tokens or pulse-monitor user
|
||||
- ⏸️ No SSH keys in authorized_keys
|
||||
- ⏸️ No /var/lib/pulse-sensor-proxy directory
|
||||
- ⏸️ Test retry logic (simulate failure, verify .processing file persists)
|
||||
- ⏸️ Test on fresh Proxmox VE install
|
||||
- ⏸️ Test on hardened PVE with read-only `/usr`
|
||||
- ⏸️ Test cluster vs standalone scenarios
|
||||
|
||||
## Codex Review Summary
|
||||
|
||||
**Review Session**: conv-1763166192078-1076
|
||||
|
||||
**Findings**: 8 critical issues identified:
|
||||
1. ❌ Host detection only checked IPs, skipped cleanup for hostname-based URLs
|
||||
2. ❌ Systemd sandbox blocked /etc/pve and /etc/systemd writes
|
||||
3. ❌ Uninstaller missing --purge, request file deleted too early
|
||||
4. ❌ Incomplete /usr/local → /opt migration (SSH forced command, self-heal, backend)
|
||||
5. ❌ Timestamp-based unit names caused collisions
|
||||
6. ❌ Brittle sed-based bind mount removal, Unicode table parsing
|
||||
7. ❌ API token parsing failed on locales, couldn't distinguish error types
|
||||
8. ❌ No retry mechanism for transient failures
|
||||
|
||||
**Resolution**: All 8 issues fixed in commits bcd8d4e0f and fe53d6473.
|
||||
|
||||
## References
|
||||
|
||||
- Initial commit: ed65fda74 "Extend node cleanup to fully remove Pulse footprint"
|
||||
- Binary relocation: b192c60e9 "Relocate binaries to /opt/pulse/sensor-proxy/"
|
||||
- Full implementation: 6692228e0 "Full cleanup script implementation"
|
||||
- Bug fixes: ed48d7555, 17d2e6876
|
||||
- Codex review fixes #1-4: bcd8d4e0f
|
||||
- Codex review fixes #5-8: fe53d6473
|
||||
- Codex review: conv-1763166192078-1076
|
||||
- Related files:
|
||||
- `scripts/install-sensor-proxy.sh` (installer + cleanup script generation)
|
||||
- `internal/api/config_handlers.go` (triggerPVEHostCleanup + manual removal)
|
||||
- `cmd/pulse-sensor-proxy/cleanup.go` (handleRequestCleanup)
|
||||
|
||||
## Priority
|
||||
|
||||
**High**: Implementation complete and Codex-reviewed. Ready for deployment testing. All critical security and correctness issues addressed.
|
||||
|
|
@ -1,58 +0,0 @@
|
|||
# Migration & Rollout Scaffolding
|
||||
|
||||
This document tracks temporary code paths, feature flags, and kill switches that
|
||||
let us roll out risky changes safely. Keep it current so on-call engineers know
|
||||
how to disable new behavior without spelunking through the codebase.
|
||||
|
||||
## How to Use This Document
|
||||
|
||||
1. **Add an entry** whenever you introduce code that needs a manual cleanup
|
||||
window (feature flags, compatibility layers, installers with rollback paths).
|
||||
2. **State the kill switch** (environment variable, UI toggle, config file, or
|
||||
systemd override) and who owns the rollout.
|
||||
3. **Define removal criteria**. Once criteria are met, delete the scaffolding
|
||||
and update this file.
|
||||
|
||||
### Entry Template
|
||||
|
||||
```
|
||||
## <Feature / Migration Name>
|
||||
- Introduced: vX.Y.Z (date/commit, if known)
|
||||
- Owner: <team/person>
|
||||
- Kill switch: how to disable + default state
|
||||
- Monitoring: metrics/logs to watch
|
||||
- Cleanup criteria: what must be true before removing scaffolding
|
||||
- Notes: cross-links to docs or runbooks
|
||||
```
|
||||
|
||||
## Active Scaffolding
|
||||
|
||||
### Adaptive Polling Scheduler
|
||||
- **Introduced**: v4.24.0 (documented in `docs/monitoring/ADAPTIVE_POLLING.md`)
|
||||
- **Owner**: Monitoring subsystem (rcourtman)
|
||||
- **Kill switch**: Toggle **Settings → System → Monitoring → Adaptive
|
||||
Polling**, or set `ADAPTIVE_POLLING_ENABLED=false` before starting Pulse
|
||||
(Docker/Kubernetes). Helm users can override `adaptivePollingEnabled: false`.
|
||||
- **Monitoring**: `pulse_monitor_poll_queue_depth`,
|
||||
`pulse_monitor_poll_staleness_seconds`, and
|
||||
`/api/monitoring/scheduler/health` for breaker/dead-letter status.
|
||||
- **Cleanup criteria**: Remove flag and legacy scheduler code after three stable
|
||||
releases with `queue.depth < 50` during peak hours and no dead-letter growth.
|
||||
- **Notes**: Operational guidance lives in
|
||||
`docs/operations/ADAPTIVE_POLLING_ROLLOUT.md`.
|
||||
|
||||
### Automatic Updates Service
|
||||
- **Introduced**: commit `f46ff1792` (2025-10-11) added `scripts/pulse-auto-update.sh`
|
||||
- **Owner**: Installer/Update subsystem (rcourtman)
|
||||
- **Kill switch**: Disable **Settings → System → Updates → Automatic Updates** or
|
||||
set `AUTO_UPDATE_ENABLED=false` in `/var/lib/pulse/system.json`. Installers
|
||||
accept `--no-auto-update` to force the flag off.
|
||||
- **Monitoring**: `journalctl -u pulse-update.service`, UI update history
|
||||
(Settings → System → Updates), and webhook notifications tagged with
|
||||
`event_id=system.update`.
|
||||
- **Cleanup criteria**: Remove legacy opt-out plumbing once the opt-in rate is
|
||||
≥90 % across supported installers and we have at least two release cycles with
|
||||
zero rollback events triggered by auto-update.
|
||||
- **Notes**: Keep release-specific caveats in `docs/RELEASE_NOTES.md`. The
|
||||
update scheduler runs only on systemd installs; container builds ignore the
|
||||
flag.
|
||||
|
|
@ -1,8 +0,0 @@
|
|||
cd /
|
||||
if [ -d /tmp/pulse-copy ]; then
|
||||
rm -rf /tmp/pulse-copy
|
||||
fi
|
||||
mkdir -p /tmp/pulse-copy
|
||||
cp -R /opt/pulse /tmp/pulse-copy/repo
|
||||
cd /tmp/pulse-copy/repo
|
||||
GOCACHE=/tmp/go-cache HOME=/tmp/pulse-copy/repo go test ./...
|
||||
|
|
@ -1,39 +0,0 @@
|
|||
#!/bin/bash
|
||||
# Daily LOCAL backup script for CLAUDE.md
|
||||
# Keeps last 30 days of backups in a LOCAL directory (never committed)
|
||||
|
||||
set -e
|
||||
|
||||
CLAUDE_MD="/opt/pulse/CLAUDE.md"
|
||||
BACKUP_DIR="$HOME/.claude-md-backups" # User's home directory, not in repo
|
||||
DATE=$(date +%Y-%m-%d)
|
||||
BACKUP_FILE="${BACKUP_DIR}/CLAUDE.md.${DATE}"
|
||||
|
||||
# Create backup directory if it doesn't exist
|
||||
mkdir -p "$BACKUP_DIR"
|
||||
|
||||
# Only create backup if CLAUDE.md exists
|
||||
if [ ! -f "$CLAUDE_MD" ]; then
|
||||
echo "Error: $CLAUDE_MD not found"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Only create backup if one doesn't exist for today
|
||||
if [ -f "$BACKUP_FILE" ]; then
|
||||
echo "Backup already exists for today: $BACKUP_FILE"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Create backup
|
||||
cp "$CLAUDE_MD" "$BACKUP_FILE"
|
||||
echo "Created backup: $BACKUP_FILE"
|
||||
|
||||
# Remove backups older than 30 days
|
||||
find "$BACKUP_DIR" -name "CLAUDE.md.*" -type f -mtime +30 -delete
|
||||
echo "Cleaned up backups older than 30 days"
|
||||
|
||||
# Show current backups
|
||||
echo ""
|
||||
echo "Current backups in $BACKUP_DIR:"
|
||||
ls -lh "$BACKUP_DIR" | tail -n +2 | wc -l
|
||||
echo "backups found"
|
||||
|
|
@ -1,50 +0,0 @@
|
|||
#!/bin/bash
|
||||
# Heuristic router for selecting a Codex reasoning tier.
|
||||
# Usage: codex-router.sh "Fix the typo" or echo "..." | codex-router.sh
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
WORKDIR="/opt/pulse"
|
||||
|
||||
# Collect a prompt from stdin and/or arguments.
|
||||
PROMPT_INPUT=""
|
||||
if [[ ! -t 0 ]]; then
|
||||
PROMPT_INPUT="$(cat)"
|
||||
fi
|
||||
|
||||
if [[ $# -gt 0 ]]; then
|
||||
if [[ -n "${PROMPT_INPUT}" ]]; then
|
||||
PROMPT="${PROMPT_INPUT}"$'\n'"$*"
|
||||
else
|
||||
PROMPT="$*"
|
||||
fi
|
||||
else
|
||||
PROMPT="${PROMPT_INPUT}"
|
||||
fi
|
||||
|
||||
PROMPT="$(printf '%s' "${PROMPT}" | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//')"
|
||||
|
||||
if [[ -z "${PROMPT}" ]]; then
|
||||
echo "Usage: $0 <prompt>" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
reasoning_profile="low"
|
||||
|
||||
# Escalate when the task hints at analysis, design, or larger edits.
|
||||
if [[ ${#PROMPT} -gt 400 ]] \
|
||||
|| grep -qiE '(analysis|investigate|explain|design|architecture|strategy|refactor|spec|diagnos|trade[- ]?off|postmortem)' <<<"${PROMPT}"; then
|
||||
reasoning_profile="medium"
|
||||
fi
|
||||
|
||||
case "${reasoning_profile}" in
|
||||
medium)
|
||||
profile_arg=(--profile medium)
|
||||
;;
|
||||
*)
|
||||
profile_arg=(--profile low)
|
||||
;;
|
||||
esac
|
||||
|
||||
echo "Routing prompt to Codex profile '${reasoning_profile}'" >&2
|
||||
exec codex exec "${profile_arg[@]}" -C "${WORKDIR}" "${PROMPT}"
|
||||
3
work.sh
3
work.sh
|
|
@ -1,3 +0,0 @@
|
|||
#!/bin/bash
|
||||
cd /opt/pulse
|
||||
git checkout cleanup-auto-uninstall
|
||||
Loading…
Reference in a new issue