Handling Rollbacks

This workflow covers revoking problematic releases and restoring previous versions. Rollbacks are necessary when releases contain critical bugs, security vulnerabilities, or compatibility issues.

Prerequisites

• API credentials with release management permissions
• A release that needs to be revoked
• A previous stable version available (or ability to create a hotfix)
• Understanding of the issue requiring rollback

Understanding Rollbacks

Rollbacks involve:

1. Revoking the problematic release — Prevents clients from receiving it
2. Restoring previous version — Makes previous stable version available again
3. Creating hotfix — Optionally create a patched version

Revoked releases remain in the catalog for audit purposes but are not distributed to clients.

Step 1: Identify the Issue

Before rolling back, identify:

• What version has the issue
• What the issue is (bug, security vulnerability, compatibility)
• Severity and impact
• Whether a hotfix is needed

Step 2: Revoke the Release

Revoke the problematic release:

curl -X POST "${BP_BASE_URL}/v1/releases/rel_abc123/revoke" \
  -H "Authorization: Bearer $BP_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "reason": "Critical security vulnerability discovered in version 1.2.3"
  }'

Response:

{
  "release_id": "rel_abc123",
  "status": "revoked",
  "revoked_at": "2024-01-15T10:30:00Z",
  "revoked_by": "admin@example.com",
  "reason": "Critical security vulnerability discovered in version 1.2.3"
}

Revoked releases are immediately excluded from update decisions.

Step 3: Verify Revocation

Verify the release is no longer returned to clients:

curl -X POST "${BP_BASE_URL}/v1/decision" \
  -H "Content-Type: application/json" \
  -d '{
    "products": [
      {
        "product_id": "prod_xyz789",
        "current_version": "1.1.0",
        "channel": "stable",
        "tenant_id": "tenant_abc123"
      }
    ]
  }'

The revoked version should not appear in the response.

Step 4: Restore Previous Version

If a previous stable version exists, it automatically becomes available again. Clients querying for updates will receive the previous version instead of the revoked one.

Check available versions:

curl "${BP_BASE_URL}/v1/products/prod_xyz789/releases?channel=stable" \
  -H "Authorization: Bearer $BP_API_TOKEN"

The previous version should be the latest non-revoked release.

Step 5: Create Hotfix (Optional)

If a hotfix is needed, create a patched version:

# Register hotfix artifact
curl -X POST "${BP_BASE_URL}/v1/artifacts" \
  -H "Authorization: Bearer $BP_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "source_type": "EXTERNAL",
    "sha256": "hotfix-hash-here",
    "size_bytes": 15728640,
    "external_url": "https://artifacts.example.com/releases/v1.2.4/installer.msi"
  }'

# Create hotfix release
curl -X POST "${BP_BASE_URL}/v1/releases" \
  -H "Authorization: Bearer $BP_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "product_id": "prod_xyz789",
    "version": "1.2.4",
    "channel": "stable",
    "artifacts": [
      {
        "artifact_id": "art_hotfix",
        "platform": "windows",
        "arch": "x86_64",
        "installer_type": "msi",
        "requires_admin": true
      }
    ],
    "metadata": {
      "release_notes": "Hotfix for critical security vulnerability in 1.2.3"
    }
  }'

Rollback Scenarios

Scenario 1: Critical Security Vulnerability

1. Revoke the vulnerable release immediately
2. Create a hotfix release with the security patch
3. Notify affected tenants
4. Monitor adoption of the hotfix

Scenario 2: Critical Bug

1. Revoke the buggy release
2. Restore previous stable version
3. Create hotfix if needed
4. Test hotfix thoroughly before publishing

Scenario 3: Compatibility Issue

1. Revoke the incompatible release
2. Restore previous compatible version
3. Fix compatibility issues
4. Publish fixed version

Version Blocklisting

For additional protection, blocklist the problematic version:

curl -X POST "${BP_BASE_URL}/v1/tenants/tenant_abc123/blocklist" \
  -H "Authorization: Bearer $BP_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "product_id": "prod_xyz789",
    "blocked_version": "1.2.3",
    "reason": "Critical security vulnerability"
  }'

Blocklisted versions are excluded from update decisions even if not revoked.

Monitoring Rollback Impact

Monitor the impact of rollbacks:

Check client versions:

curl "${BP_BASE_URL}/v1/products/prod_xyz789/telemetry/versions" \
  -H "Authorization: Bearer $BP_API_TOKEN"

Monitor update decisions:

• Track which versions clients are receiving
• Monitor adoption of hotfix releases
• Alert on clients still using revoked versions

Best Practices

Act quickly: Revoke problematic releases as soon as issues are identified.

Document reasons: Always include a reason when revoking releases for audit purposes.

Test hotfixes: Thoroughly test hotfix releases before publishing.

Notify stakeholders: Notify affected tenants and users about rollbacks and hotfixes.

Monitor adoption: Track adoption of hotfix releases to ensure clients are protected.

Maintain audit trail: Revoked releases remain in the catalog for audit purposes.

Next Steps

• Manage version blocklists — see Managing Release Versions
• Monitor release health — see Tracking License Usage
• Configure automated rollbacks — see Managing Update Policies