e3890c2925
Related to systematic release workflow failures. The workflow has never successfully completed from start to finish since validation was added. Root causes identified and fixed: 1. **GraphQL node_id vs numeric release ID**: The create-release job was using `gh release view --json id` which returns a GraphQL node_id (RE_kwDON5nJtM4PmlTt) instead of the numeric database ID (261772525) needed by the REST API. The validation workflow then failed with 404 when trying to download assets. Fixed by using `gh api` to get the numeric ID from the releases list endpoint. 2. **Missing binaries in Docker image**: The validation script expects 26 binaries + 3 Windows symlinks in /opt/pulse/bin/, but the Dockerfile was only copying a subset. Missing binaries included the main pulse server binary, armv6/386 builds for all agents, and caused immediate validation failure. Fixed by copying all built binaries from backend-builder stage. 3. **Assets-only validation fallback broken**: When Docker image pull times out, the workflow falls back to assets-only validation but was still calling the validation script without --skip-docker flag, causing it to fail on the first docker command. Fixed by passing --skip-docker flag in the fallback path. 4. **Asset download pagination**: The asset download was not using --paginate, which would cause silent failures once we exceed 30 assets (currently at 27). Fixed by adding --paginate to gh api call. All fixes verified locally and address the complete failure chain.
410 lines
16 KiB
YAML
410 lines
16 KiB
YAML
name: Validate Release Assets
|
|
|
|
on:
|
|
workflow_call:
|
|
inputs:
|
|
tag:
|
|
description: 'Release tag (e.g., v4.29.0)'
|
|
required: true
|
|
type: string
|
|
version:
|
|
description: 'Version number without v prefix (e.g., 4.29.0)'
|
|
required: true
|
|
type: string
|
|
release_id:
|
|
description: 'GitHub release ID'
|
|
required: true
|
|
type: string
|
|
draft:
|
|
description: 'Whether the release is still a draft'
|
|
required: true
|
|
type: boolean
|
|
target_commitish:
|
|
description: 'Commit SHA associated with the release'
|
|
required: true
|
|
type: string
|
|
release:
|
|
types: [edited]
|
|
workflow_dispatch:
|
|
inputs:
|
|
tag:
|
|
description: 'Release tag (e.g., v4.29.0)'
|
|
required: true
|
|
type: string
|
|
version:
|
|
description: 'Version number without v prefix (e.g., 4.29.0)'
|
|
required: true
|
|
type: string
|
|
release_id:
|
|
description: 'GitHub release ID'
|
|
required: true
|
|
type: string
|
|
draft:
|
|
description: 'Set to true to run against a draft release'
|
|
required: true
|
|
type: boolean
|
|
target_commitish:
|
|
description: 'Commit SHA associated with the release'
|
|
required: true
|
|
type: string
|
|
|
|
jobs:
|
|
validate:
|
|
runs-on: ubuntu-latest
|
|
permissions:
|
|
contents: write
|
|
issues: write
|
|
|
|
steps:
|
|
- name: Checkout repository
|
|
uses: actions/checkout@v4
|
|
|
|
- name: Determine release context
|
|
id: context
|
|
env:
|
|
EVENT_NAME: ${{ github.event_name }}
|
|
INPUT_TAG: ${{ inputs.tag }}
|
|
INPUT_VERSION: ${{ inputs.version }}
|
|
INPUT_RELEASE_ID: ${{ inputs.release_id }}
|
|
INPUT_DRAFT: ${{ inputs.draft }}
|
|
INPUT_COMMIT: ${{ inputs.target_commitish }}
|
|
run: |
|
|
python3 <<'EOF' > context.env
|
|
import json, os, sys
|
|
|
|
event_name = os.environ.get("EVENT_NAME", "")
|
|
result = {}
|
|
|
|
if event_name == "release":
|
|
with open(os.environ["GITHUB_EVENT_PATH"], "r", encoding="utf-8") as handle:
|
|
data = json.load(handle)
|
|
release = data.get("release") or {}
|
|
result["tag"] = release.get("tag_name", "")
|
|
tag = result["tag"]
|
|
result["version"] = tag[1:] if tag.startswith("v") else tag
|
|
result["release_id"] = str(release.get("id", ""))
|
|
result["target_commitish"] = release.get("target_commitish", "")
|
|
result["draft"] = str(release.get("draft", False)).lower()
|
|
else:
|
|
result["tag"] = os.environ.get("INPUT_TAG", "")
|
|
result["version"] = os.environ.get("INPUT_VERSION", "")
|
|
result["release_id"] = os.environ.get("INPUT_RELEASE_ID", "")
|
|
result["target_commitish"] = os.environ.get("INPUT_COMMIT", "")
|
|
draft_value = os.environ.get("INPUT_DRAFT", "false")
|
|
result["draft"] = str(draft_value).lower()
|
|
|
|
if not result["tag"] or not result["release_id"]:
|
|
sys.stderr.write("::error::Release metadata is missing. Provide tag, version, release_id, and target_commitish.\n")
|
|
sys.exit(1)
|
|
|
|
should_run = "true"
|
|
if event_name == "release" and result["draft"] != "true":
|
|
should_run = "false"
|
|
result["should_run"] = should_run
|
|
|
|
for key, value in result.items():
|
|
print(f"{key}={value}")
|
|
EOF
|
|
cat context.env >> "$GITHUB_OUTPUT"
|
|
cat context.env
|
|
|
|
- name: Skip validation for published releases
|
|
if: steps.context.outputs.should_run != 'true'
|
|
run: echo "Release is already published; skipping validation checks."
|
|
|
|
- name: Download all release assets
|
|
if: steps.context.outputs.should_run == 'true'
|
|
id: download
|
|
env:
|
|
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
run: |
|
|
echo "Downloading all assets from release ${{ steps.context.outputs.tag }}..."
|
|
|
|
mkdir -p release
|
|
cd release
|
|
|
|
# Get asset info (id and name) - API works with GITHUB_TOKEN for draft releases
|
|
# Use --paginate to handle releases with >30 assets
|
|
ASSETS_JSON=$(gh api --paginate "repos/${{ github.repository }}/releases/${{ steps.context.outputs.release_id }}/assets")
|
|
|
|
if [ "$(echo "$ASSETS_JSON" | jq '. | length')" -eq 0 ]; then
|
|
echo "::error::No assets found in release"
|
|
echo "has_assets=false" >> $GITHUB_OUTPUT
|
|
exit 1
|
|
fi
|
|
|
|
echo "has_assets=true" >> $GITHUB_OUTPUT
|
|
echo "Found $(echo "$ASSETS_JSON" | jq '. | length') assets"
|
|
|
|
# Download each asset using the API (works with draft releases)
|
|
echo "$ASSETS_JSON" | jq -r '.[] | "\(.id) \(.name)"' | while read -r asset_id filename; do
|
|
echo "Downloading $filename (ID: $asset_id)..."
|
|
|
|
# Use GitHub API to download with authentication
|
|
curl -L -H "Authorization: token $GH_TOKEN" \
|
|
-H "Accept: application/octet-stream" \
|
|
"https://api.github.com/repos/${{ github.repository }}/releases/assets/$asset_id" \
|
|
-o "$filename"
|
|
|
|
if [ $? -eq 0 ] && [ -f "$filename" ]; then
|
|
echo "✓ Downloaded $filename ($(du -h "$filename" | cut -f1))"
|
|
else
|
|
echo "::error::Failed to download $filename"
|
|
exit 1
|
|
fi
|
|
done
|
|
|
|
echo ""
|
|
echo "All assets downloaded:"
|
|
ls -lh
|
|
|
|
- name: Install Docker
|
|
if: steps.context.outputs.should_run == 'true'
|
|
run: docker --version
|
|
|
|
- name: Pull Docker image (with retry logic)
|
|
if: steps.context.outputs.should_run == 'true'
|
|
id: docker
|
|
continue-on-error: true
|
|
run: |
|
|
IMAGE="rcourtman/pulse:${{ steps.context.outputs.tag }}"
|
|
echo "Attempting to pull Docker image: $IMAGE"
|
|
echo "Docker Hub CDN propagation can take 2-5 minutes after push..."
|
|
echo ""
|
|
|
|
# Retry logic: 10 attempts with exponential backoff
|
|
# Total time: ~10 minutes max
|
|
MAX_ATTEMPTS=10
|
|
ATTEMPT=1
|
|
WAIT_TIME=30
|
|
|
|
while [ $ATTEMPT -le $MAX_ATTEMPTS ]; do
|
|
echo "Attempt $ATTEMPT/$MAX_ATTEMPTS: Pulling image..."
|
|
|
|
if docker pull "$IMAGE" 2>/dev/null; then
|
|
echo "✓ Docker image available: $IMAGE"
|
|
echo "image_available=true" >> $GITHUB_OUTPUT
|
|
echo "image=$IMAGE" >> $GITHUB_OUTPUT
|
|
exit 0
|
|
fi
|
|
|
|
if [ $ATTEMPT -lt $MAX_ATTEMPTS ]; then
|
|
echo "⚠️ Image not yet available, waiting ${WAIT_TIME}s before retry..."
|
|
sleep $WAIT_TIME
|
|
WAIT_TIME=$((WAIT_TIME * 2)) # Exponential backoff
|
|
if [ $WAIT_TIME -gt 120 ]; then
|
|
WAIT_TIME=120 # Cap at 2 minutes
|
|
fi
|
|
fi
|
|
|
|
ATTEMPT=$((ATTEMPT + 1))
|
|
done
|
|
|
|
echo "⚠️ Docker image not available after $MAX_ATTEMPTS attempts"
|
|
echo "⚠️ Will skip Docker image validation"
|
|
echo "image_available=false" >> $GITHUB_OUTPUT
|
|
|
|
- name: Run validation script
|
|
if: steps.context.outputs.should_run == 'true'
|
|
id: validate
|
|
run: |
|
|
set +e
|
|
echo "Running validation script..."
|
|
chmod +x scripts/validate-release.sh
|
|
OUTPUT_FILE=$(mktemp)
|
|
|
|
if [ "${{ steps.docker.outputs.image_available }}" = "true" ]; then
|
|
echo "Running full validation (Docker + assets)..."
|
|
scripts/validate-release.sh \
|
|
"${{ steps.context.outputs.version }}" \
|
|
"${{ steps.docker.outputs.image }}" \
|
|
"release" 2>&1 | tee "$OUTPUT_FILE"
|
|
else
|
|
echo "Running assets-only validation (Docker image not available)..."
|
|
scripts/validate-release.sh \
|
|
"${{ steps.context.outputs.version }}" \
|
|
--skip-docker \
|
|
"release" 2>&1 | tee "$OUTPUT_FILE"
|
|
fi
|
|
|
|
VALIDATION_EXIT_CODE=$?
|
|
|
|
echo "VALIDATION_OUTPUT<<EOF" >> $GITHUB_OUTPUT
|
|
cat "$OUTPUT_FILE" >> $GITHUB_OUTPUT
|
|
echo "EOF" >> $GITHUB_OUTPUT
|
|
|
|
if [ $VALIDATION_EXIT_CODE -eq 0 ]; then
|
|
echo "validation_passed=true" >> $GITHUB_OUTPUT
|
|
echo "✅ Validation PASSED"
|
|
else
|
|
echo "validation_passed=false" >> $GITHUB_OUTPUT
|
|
echo "❌ Validation FAILED"
|
|
fi
|
|
|
|
exit $VALIDATION_EXIT_CODE
|
|
|
|
- name: Set commit status - Success
|
|
if: steps.context.outputs.should_run == 'true' && steps.validate.outputs.validation_passed == 'true'
|
|
run: |
|
|
curl -X POST \
|
|
-H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" \
|
|
-H "Accept: application/vnd.github+json" \
|
|
"https://api.github.com/repos/${{ github.repository }}/statuses/${{ steps.context.outputs.target_commitish }}" \
|
|
-d '{
|
|
"state": "success",
|
|
"target_url": "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}",
|
|
"description": "All release assets validated successfully",
|
|
"context": "Release Asset Validation"
|
|
}'
|
|
|
|
- name: Update release body - Success
|
|
if: steps.context.outputs.should_run == 'true' && steps.validate.outputs.validation_passed == 'true'
|
|
run: |
|
|
echo "✅ Validation passed - updating release description"
|
|
|
|
CURRENT_BODY=$(curl -s -H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" \
|
|
"https://api.github.com/repos/${{ github.repository }}/releases/${{ steps.context.outputs.release_id }}" \
|
|
| jq -r '.body // ""')
|
|
|
|
CURRENT_BODY=$(echo "$CURRENT_BODY" | sed '/<!-- VALIDATION_STATUS_START -->/,/<!-- VALIDATION_STATUS_END -->/d')
|
|
|
|
read -r -d '' VALIDATION_BLOCK <<'EOF' || true
|
|
<!-- VALIDATION_STATUS_START -->
|
|
## ✅ Release Asset Validation: PASSED
|
|
|
|
All release assets have been validated successfully!
|
|
|
|
**Status**: Ready for publication ✅
|
|
**Validated**: TIMESTAMP_PLACEHOLDER
|
|
**Workflow**: WORKFLOW_LINK_PLACEHOLDER
|
|
|
|
### Validation Summary
|
|
- All required assets present ✓
|
|
- Checksums verified ✓
|
|
- Version strings correct ✓
|
|
- Binary architectures validated ✓
|
|
|
|
<!-- VALIDATION_STATUS_END -->
|
|
EOF
|
|
|
|
VALIDATION_BLOCK="${VALIDATION_BLOCK//TIMESTAMP_PLACEHOLDER/$(date -u +"%Y-%m-%d %H:%M:%S UTC")}"
|
|
VALIDATION_BLOCK="${VALIDATION_BLOCK//WORKFLOW_LINK_PLACEHOLDER/[${{ github.workflow }} #${{ github.run_number }}](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }})}"
|
|
|
|
NEW_BODY="$VALIDATION_BLOCK
|
|
|
|
$CURRENT_BODY"
|
|
|
|
curl -X PATCH \
|
|
-H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" \
|
|
-H "Accept: application/vnd.github+json" \
|
|
"https://api.github.com/repos/${{ github.repository }}/releases/${{ steps.context.outputs.release_id }}" \
|
|
-d "$(jq -n --arg body "$NEW_BODY" '{body: $body}')"
|
|
|
|
- name: Set commit status - Failure
|
|
if: steps.context.outputs.should_run == 'true' && (failure() || steps.validate.outputs.validation_passed == 'false')
|
|
run: |
|
|
curl -X POST \
|
|
-H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" \
|
|
-H "Accept: application/vnd.github+json" \
|
|
"https://api.github.com/repos/${{ github.repository }}/statuses/${{ steps.context.outputs.target_commitish }}" \
|
|
-d '{
|
|
"state": "failure",
|
|
"target_url": "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}",
|
|
"description": "Release asset validation failed - assets deleted",
|
|
"context": "Release Asset Validation"
|
|
}'
|
|
|
|
- name: Delete all release assets on failure
|
|
if: steps.context.outputs.should_run == 'true' && (failure() || steps.validate.outputs.validation_passed == 'false')
|
|
run: |
|
|
echo "❌ Validation failed - deleting all release assets"
|
|
|
|
ASSET_IDS=$(curl -s -H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" \
|
|
"https://api.github.com/repos/${{ github.repository }}/releases/${{ steps.context.outputs.release_id }}/assets" \
|
|
| jq -r '.[].id')
|
|
|
|
if [ -n "$ASSET_IDS" ]; then
|
|
echo "$ASSET_IDS" | while read -r asset_id; do
|
|
if [ -n "$asset_id" ] && [ "$asset_id" != "null" ]; then
|
|
echo "Deleting asset ID: $asset_id"
|
|
curl -X DELETE \
|
|
-H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" \
|
|
"https://api.github.com/repos/${{ github.repository }}/releases/assets/$asset_id"
|
|
fi
|
|
done
|
|
echo "✓ Asset deletion process completed"
|
|
else
|
|
echo "No assets to delete"
|
|
fi
|
|
|
|
- name: Update release body - Failure
|
|
if: steps.context.outputs.should_run == 'true' && (failure() || steps.validate.outputs.validation_passed == 'false')
|
|
run: |
|
|
echo "❌ Validation failed - updating release description"
|
|
|
|
CURRENT_BODY=$(curl -s -H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" \
|
|
"https://api.github.com/repos/${{ github.repository }}/releases/${{ steps.context.outputs.release_id }}" \
|
|
| jq -r '.body // ""')
|
|
|
|
CURRENT_BODY=$(echo "$CURRENT_BODY" | sed '/<!-- VALIDATION_STATUS_START -->/,/<!-- VALIDATION_STATUS_END -->/d')
|
|
|
|
VALIDATION_OUTPUT="${{ steps.validate.outputs.VALIDATION_OUTPUT }}"
|
|
if [ -n "$VALIDATION_OUTPUT" ]; then
|
|
ERRORS=$(echo "$VALIDATION_OUTPUT" | grep -i '\[ERROR\]' | head -20 || echo "See workflow logs for details")
|
|
else
|
|
ERRORS="Validation script failed to run. Check workflow logs."
|
|
fi
|
|
|
|
read -r -d '' VALIDATION_BLOCK <<'EOF' || true
|
|
<!-- VALIDATION_STATUS_START -->
|
|
## ❌ Release Asset Validation: FAILED
|
|
|
|
**Critical**: Release assets failed validation checks. All assets have been deleted to prevent publishing an invalid release.
|
|
|
|
**Status**: ⛔ DO NOT PUBLISH until validation passes
|
|
**Failed**: TIMESTAMP_PLACEHOLDER
|
|
**Workflow**: WORKFLOW_LINK_PLACEHOLDER
|
|
|
|
### What Happened
|
|
The automated validation process detected issues with the uploaded release assets. This could be due to:
|
|
- Missing required files
|
|
- Checksum mismatches
|
|
- Incorrect version strings in binaries
|
|
- Corrupted or incomplete uploads
|
|
|
|
### Action Required
|
|
1. Review the validation errors in the workflow logs
|
|
2. Fix the underlying issues in the release build process
|
|
3. Re-upload the corrected assets
|
|
4. Validation will run automatically when assets are edited
|
|
|
|
### Validation Errors (first 20 lines)
|
|
```
|
|
ERRORS_PLACEHOLDER
|
|
```
|
|
|
|
[View full workflow logs for complete details](WORKFLOW_URL_PLACEHOLDER)
|
|
|
|
<!-- VALIDATION_STATUS_END -->
|
|
EOF
|
|
|
|
VALIDATION_BLOCK="${VALIDATION_BLOCK//TIMESTAMP_PLACEHOLDER/$(date -u +"%Y-%m-%d %H:%M:%S UTC")}"
|
|
VALIDATION_BLOCK="${VALIDATION_BLOCK//WORKFLOW_LINK_PLACEHOLDER/[${{ github.workflow }} #${{ github.run_number }}](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }})}"
|
|
VALIDATION_BLOCK="${VALIDATION_BLOCK//WORKFLOW_URL_PLACEHOLDER/${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}}"
|
|
VALIDATION_BLOCK="${VALIDATION_BLOCK//ERRORS_PLACEHOLDER/$ERRORS}"
|
|
|
|
NEW_BODY="$VALIDATION_BLOCK
|
|
|
|
$CURRENT_BODY"
|
|
|
|
curl -X PATCH \
|
|
-H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" \
|
|
-H "Accept: application/vnd.github+json" \
|
|
"https://api.github.com/repos/${{ github.repository }}/releases/${{ steps.context.outputs.release_id }}" \
|
|
-d "$(jq -n --arg body "$NEW_BODY" '{body: $body}')"
|
|
|
|
- name: Fail the workflow
|
|
if: steps.context.outputs.should_run == 'true' && (failure() || steps.validate.outputs.validation_passed == 'false')
|
|
run: |
|
|
echo "::error::Release asset validation failed. All assets have been deleted."
|
|
exit 1
|