From 512fbb9044b972628d32c7944f87bf95771a11dd Mon Sep 17 00:00:00 2001 From: Yaseen Hamdulay Date: Wed, 10 Jun 2026 14:33:07 +0200 Subject: [PATCH 1/6] ci: consolidate release into one dispatch-driven workflow Replace the three event-chained release workflows (tag-and-release -> deploy-release -> pypi-publish) with a single `Release` orchestrator that runs the whole pipeline in one workflow_dispatch run, ordered by `needs:`: validate -> test -> tag -> goreleaser -> verify-assets -> wrapper-smoke -> pypi -> cleanup (on failure) Why: the old flow chained workflows via tag-push / release-published events. A tag pushed with GITHUB_TOKEN does not trigger downstream workflows, so the build silently never ran and the PyPI wrapper shipped pointing at binaries that did not exist -> `pip install` then 404'd for every user. Event chaining also hid where a release stalled. This design removes the chaining entirely: - One run, visible end to end. No silent hand-offs. - Tag is created as a pipeline step (no PAT-trigger hack needed). - Every job operates on the resolved release commit/tag, so GoReleaser's git-state check can't be tripped by a dispatch running off a branch. - verify-assets + wrapper-smoke gate PyPI on the binaries actually existing and the wrapper actually running against the release; PyPI (which can't be un-published) is the last step. - cleanup deletes the tag + release on failure for clean re-runs. Reusable sub-workflows (workflow_call): test.yml (shared with PR CI), release-goreleaser.yml, release-pypi.yml. release-pypi.yml also fixes the old "Test installation" step whose unconditional `exit 0` swallowed the binary-download failure. Docs (RELEASING.md, PYPI_RELEASE_STRATEGY.md) and the version-bump comments updated to the dispatch flow. Co-Authored-By: Claude Opus 4.8 (1M context) --- .github/workflows/deploy-release.yml | 232 ----------------------- .github/workflows/pypi-publish.yml | 104 ---------- .github/workflows/release-goreleaser.yml | 57 ++++++ .github/workflows/release-pypi.yml | 158 +++++++++++++++ .github/workflows/release.yml | 211 +++++++++++++++++++++ .github/workflows/tag-and-release.yml | 67 ------- .github/workflows/test.yml | 10 + RELEASING.md | 103 +++++----- python-wrapper/PYPI_RELEASE_STRATEGY.md | 8 +- python-wrapper/cerebrium_cli.py | 2 +- python-wrapper/pyproject.toml | 2 +- 11 files changed, 498 insertions(+), 456 deletions(-) delete mode 100644 .github/workflows/deploy-release.yml delete mode 100644 .github/workflows/pypi-publish.yml create mode 100644 .github/workflows/release-goreleaser.yml create mode 100644 .github/workflows/release-pypi.yml create mode 100644 .github/workflows/release.yml delete mode 100644 .github/workflows/tag-and-release.yml diff --git a/.github/workflows/deploy-release.yml b/.github/workflows/deploy-release.yml deleted file mode 100644 index 22d6c99..0000000 --- a/.github/workflows/deploy-release.yml +++ /dev/null @@ -1,232 +0,0 @@ -name: Deploy Release - -on: - push: - tags: - - "v*" - workflow_dispatch: - inputs: - skip-tests: - description: "Skip tests before release" - required: false - default: false - type: boolean - version: - description: "Version to release (optional, for manual releases)" - required: false - type: string - -permissions: - contents: write - packages: write - id-token: write - -jobs: - test: - name: Test before release - # Run tests for non-prerelease versions (v1.0.0) or when manually triggered without skip - if: | - (github.event_name == 'push' && !contains(github.ref_name, '-RC') && !contains(github.ref_name, '-rc') && !contains(github.ref_name, '-beta') && !contains(github.ref_name, '-alpha')) || - (github.event_name == 'workflow_dispatch' && github.event.inputs.skip-tests == 'false') - runs-on: ubuntu-latest - steps: - - name: Checkout code - uses: actions/checkout@v5 - - - name: Set up Go - uses: actions/setup-go@v6 - with: - go-version: "1.25" - cache: true - - - name: Download dependencies - run: go mod download - - - name: Run tests - run: go test -race -shuffle=on ./... - - smoke-test: - name: Smoke test before release - # Run smoke tests for non-prerelease versions or when manually triggered without skip - if: | - (github.event_name == 'push' && !contains(github.ref_name, '-RC') && !contains(github.ref_name, '-rc') && !contains(github.ref_name, '-beta') && !contains(github.ref_name, '-alpha')) || - (github.event_name == 'workflow_dispatch' && github.event.inputs.skip-tests == 'false') - runs-on: ubuntu-latest - steps: - - name: Checkout code - uses: actions/checkout@v5 - - - name: Set up Go - uses: actions/setup-go@v6 - with: - go-version: "1.25" - cache: true - - - name: Build CLI - run: | - # Strip 'v' prefix from tag to get version (e.g., v1.2.3 -> 1.2.3) - VERSION="${GITHUB_REF_NAME#v}" - COMMIT=$(git rev-parse --short HEAD) - BUILD_DATE=$(date -u +%Y-%m-%dT%H:%M:%SZ) - - go build -ldflags "\ - -X github.com/cerebriumai/cerebrium/internal/version.Version=${VERSION} \ - -X github.com/cerebriumai/cerebrium/internal/version.Commit=${COMMIT} \ - -X github.com/cerebriumai/cerebrium/internal/version.BuildDate=${BUILD_DATE}" \ - -o bin/cerebrium ./cmd/cerebrium - - # Save build metadata for verification - echo "VERSION=${VERSION}" >> $GITHUB_ENV - echo "COMMIT=${COMMIT}" >> $GITHUB_ENV - echo "BUILD_DATE=${BUILD_DATE}" >> $GITHUB_ENV - - - name: Add bin to PATH - run: echo "$GITHUB_WORKSPACE/bin" >> $GITHUB_PATH - - - name: Create Cerebrium config - run: | - mkdir -p ~/.cerebrium - cat > ~/.cerebrium/config.yaml << EOF - project: p-12345678 - EOF - - - name: Run smoke tests - run: | - EXPECTED_OUTPUT="cerebrium ${VERSION} (commit: ${COMMIT}, built: ${BUILD_DATE})" - - # Verify version output matches exactly - VERSION_OUTPUT=$(cerebrium version) - echo "Version output: $VERSION_OUTPUT" - echo "Expected: $EXPECTED_OUTPUT" - if [[ "$VERSION_OUTPUT" != "$EXPECTED_OUTPUT" ]]; then - echo "ERROR: Version output doesn't match expected" - exit 1 - fi - - # Verify projects current shows our configured project - PROJECT_OUTPUT=$(cerebrium projects current) - echo "Project output: $PROJECT_OUTPUT" - if [[ ! "$PROJECT_OUTPUT" =~ "p-12345678" ]]; then - echo "ERROR: Project output doesn't contain expected project ID" - exit 1 - fi - - pip-smoke-test: - name: Pip smoke test before release - # Run on all releases (including prereleases) since PyPI supports them - if: github.event_name == 'push' || (github.event_name == 'workflow_dispatch' && github.event.inputs.skip-tests == 'false') - runs-on: ubuntu-latest - steps: - - name: Checkout code - uses: actions/checkout@v5 - - - name: Set up Go - uses: actions/setup-go@v6 - with: - go-version: "1.25" - cache: true - - - name: Set up Python - uses: actions/setup-python@v6 - with: - python-version: "3.12" - - - name: Build Go binary - run: | - # Strip 'v' prefix from tag to get version (e.g., v1.2.3 -> 1.2.3) - VERSION="${GITHUB_REF_NAME#v}" - COMMIT=$(git rev-parse --short HEAD) - BUILD_DATE=$(date -u +%Y-%m-%dT%H:%M:%SZ) - - go build -ldflags "\ - -X github.com/cerebriumai/cerebrium/internal/version.Version=${VERSION} \ - -X github.com/cerebriumai/cerebrium/internal/version.Commit=${COMMIT} \ - -X github.com/cerebriumai/cerebrium/internal/version.BuildDate=${BUILD_DATE}" \ - -o bin/cerebrium ./cmd/cerebrium - - # Save build metadata for verification - echo "VERSION=${VERSION}" >> $GITHUB_ENV - echo "COMMIT=${COMMIT}" >> $GITHUB_ENV - echo "BUILD_DATE=${BUILD_DATE}" >> $GITHUB_ENV - - - name: Pre-install Go binary for pip wrapper - run: | - # Extract VERSION from cerebrium_cli.py (GitHub/semver format) - PIP_VERSION=$(grep "^VERSION = " python-wrapper/cerebrium_cli.py | sed 's/VERSION = "\(.*\)"/\1/') - echo "Pip wrapper version: $PIP_VERSION" - - # Install binary where pip wrapper expects it - mkdir -p ~/.cerebrium/bin - cp bin/cerebrium ~/.cerebrium/bin/cerebrium - chmod +x ~/.cerebrium/bin/cerebrium - - # Write version file so pip wrapper doesn't try to download - echo "$PIP_VERSION" > ~/.cerebrium/bin/.version - - - name: Build pip wheel - working-directory: python-wrapper - run: | - pip install build - python -m build - - - name: Install pip wheel - working-directory: python-wrapper - run: pip install dist/cerebrium-*.whl - - - name: Run pip smoke tests - run: | - EXPECTED_OUTPUT="cerebrium ${VERSION} (commit: ${COMMIT}, built: ${BUILD_DATE})" - - # Verify cerebrium runs through pip wrapper with correct version - VERSION_OUTPUT=$(cerebrium version) - echo "Version output: $VERSION_OUTPUT" - echo "Expected: $EXPECTED_OUTPUT" - if [[ "$VERSION_OUTPUT" != "$EXPECTED_OUTPUT" ]]; then - echo "ERROR: Version output doesn't match expected" - exit 1 - fi - - goreleaser: - name: Release - needs: [test, smoke-test, pip-smoke-test] - # Always run if tests were skipped, otherwise wait for tests to pass - if: always() && (needs.test.result == 'success' || needs.test.result == 'skipped') && (needs.smoke-test.result == 'success' || needs.smoke-test.result == 'skipped') && (needs.pip-smoke-test.result == 'success' || needs.pip-smoke-test.result == 'skipped') - runs-on: ubuntu-latest - steps: - - name: Checkout code - uses: actions/checkout@v5 - with: - fetch-depth: 0 - - - name: Set up Go - uses: actions/setup-go@v6 - with: - go-version: "1.25" - cache: true - - # Setup certificates and keys for signing (only for tag pushes, not manual dispatch) - - name: Setup macOS signing certificates - if: github.event_name == 'push' - run: | - # Note: For GoReleaser's cross-platform notarization, certificates are passed - # as base64-encoded environment variables directly to GoReleaser. - # No need to decode them to files when using anchore/quill backend. - echo "✓ macOS signing credentials configured" - - - name: Run GoReleaser - uses: goreleaser/goreleaser-action@v6 - with: - version: "~> v2" - args: release --clean - env: - GITHUB_TOKEN: ${{ secrets.GH_PAT }} - GH_PAT: ${{ secrets.GH_PAT }} - # macOS code signing and notarization using anchore/quill - # All credentials are passed as base64-encoded strings - MACOS_CERTIFICATE_P12: ${{ secrets.MACOS_CERTIFICATE_P12 }} - MACOS_CERTIFICATE_PASSWORD: ${{ secrets.MACOS_CERTIFICATE_PASSWORD }} - MACOS_NOTARIZATION_ISSUER_ID: ${{ secrets.MACOS_NOTARIZATION_ISSUER_ID }} - MACOS_NOTARIZATION_KEY_ID: ${{ secrets.MACOS_NOTARIZATION_KEY_ID }} - MACOS_NOTARIZATION_KEY: ${{ secrets.MACOS_NOTARIZATION_KEY }} - # Bugsnag error tracking configuration - BUGSNAG_API_KEY: ${{ secrets.BUGSNAG_API_KEY }} diff --git a/.github/workflows/pypi-publish.yml b/.github/workflows/pypi-publish.yml deleted file mode 100644 index 8739760..0000000 --- a/.github/workflows/pypi-publish.yml +++ /dev/null @@ -1,104 +0,0 @@ -name: Publish to PyPI - -on: - release: - types: [published] - -permissions: - id-token: write # Required for OIDC trusted publishing - contents: read - -jobs: - build-and-publish: - runs-on: ubuntu-latest - steps: - - name: Checkout repository - uses: actions/checkout@v5 - - - name: Set up Python - uses: actions/setup-python@v6 - with: - python-version: "3.12" - - - name: Install build tools - run: | - python -m pip install --upgrade pip - pip install build twine - - - name: Update version in Python package - run: | - # Get version from tag (e.g., "v2.1.0" or "v2.1.0-beta.1") - GITHUB_VERSION="${{ github.event.release.tag_name }}" - GITHUB_VERSION="${GITHUB_VERSION#v}" # Remove 'v' prefix - - # Update VERSION in cerebrium_cli.py (stores GitHub/semver format) - sed -i "s/^VERSION = \".*\"/VERSION = \"$GITHUB_VERSION\"/" python-wrapper/cerebrium_cli.py - - echo "Updated VERSION in cerebrium_cli.py to: $GITHUB_VERSION" - - # Convert to PEP 440 format for pyproject.toml - cd python-wrapper - PYPI_VERSION=$(python -c "from cerebrium_cli import github_to_pypi_version; print(github_to_pypi_version('$GITHUB_VERSION'))") - cd .. - - # Update version in pyproject.toml (stores PEP 440 format) - sed -i "s/^version = \".*\"/version = \"$PYPI_VERSION\"/" python-wrapper/pyproject.toml - - echo "Updated version in pyproject.toml to: $PYPI_VERSION" - - - name: Build Python package - working-directory: python-wrapper - run: | - python -m build - ls -la dist/ - - - name: Check package - working-directory: python-wrapper - run: | - twine check dist/* - - - name: Publish to PyPI - uses: pypa/gh-action-pypi-publish@release/v1 - with: - packages-dir: python-wrapper/dist/ - skip-existing: true - - - name: Test installation from PyPI - run: | - # Get the PEP 440 version for pip install - cd python-wrapper - PYPI_VERSION=$(python -c "from cerebrium_cli import __version__; print(__version__)") - cd .. - - # Create a clean virtual environment - python -m venv test_env - source test_env/bin/activate - - # Retry installation with exponential backoff - MAX_ATTEMPTS=5 - ATTEMPT=1 - DELAY=30 - - while [ $ATTEMPT -le $MAX_ATTEMPTS ]; do - echo "Attempt $ATTEMPT of $MAX_ATTEMPTS..." - - if pip install cerebrium==$PYPI_VERSION; then - echo "Successfully installed cerebrium $PYPI_VERSION from PyPI" - - # Test that the wrapper runs (will download binary on first run) - echo "Testing cerebrium command..." - cerebrium version && echo "Binary download and execution successful!" - exit 0 - else - if [ $ATTEMPT -lt $MAX_ATTEMPTS ]; then - echo "Package not yet available on PyPI. Waiting ${DELAY}s before retry..." - sleep $DELAY - DELAY=$((DELAY * 2)) # Exponential backoff - fi - fi - - ATTEMPT=$((ATTEMPT + 1)) - done - - echo "ERROR: Failed to install cerebrium $PYPI_VERSION from PyPI after $MAX_ATTEMPTS attempts" - exit 1 diff --git a/.github/workflows/release-goreleaser.yml b/.github/workflows/release-goreleaser.yml new file mode 100644 index 0000000..ea89c34 --- /dev/null +++ b/.github/workflows/release-goreleaser.yml @@ -0,0 +1,57 @@ +name: Build & Publish Binaries (reusable) + +# Reusable workflow: builds the cross-platform binaries with GoReleaser and +# uploads them (+ checksums.txt) to the GitHub release for the given tag. +# Called by the Release orchestrator after the tag has been created. Not meant +# to be run on its own. + +on: + workflow_call: + inputs: + ref: + description: "Tag ref to build (e.g. refs/tags/v2.5.2). Must already exist and point at HEAD." + required: true + type: string + +permissions: + contents: write + packages: write + id-token: write + +jobs: + goreleaser: + name: GoReleaser + runs-on: ubuntu-latest + steps: + - name: Checkout code + uses: actions/checkout@v5 + with: + # Pin to the tag so GoReleaser's git-state check ("the current tag + # must point at HEAD") always holds. fetch-depth: 0 is required for + # the changelog and tag discovery. + ref: ${{ inputs.ref }} + fetch-depth: 0 + + - name: Set up Go + uses: actions/setup-go@v6 + with: + go-version: "1.25" + cache: true + + - name: Run GoReleaser + uses: goreleaser/goreleaser-action@v6 + with: + version: "~> v2" + args: release --clean + env: + GITHUB_TOKEN: ${{ secrets.GH_PAT }} + GH_PAT: ${{ secrets.GH_PAT }} + # macOS code signing and notarization using anchore/quill + # All credentials are passed as base64-encoded strings + MACOS_CERTIFICATE_P12: ${{ secrets.MACOS_CERTIFICATE_P12 }} + MACOS_CERTIFICATE_PASSWORD: ${{ secrets.MACOS_CERTIFICATE_PASSWORD }} + MACOS_NOTARIZATION_ISSUER_ID: ${{ secrets.MACOS_NOTARIZATION_ISSUER_ID }} + MACOS_NOTARIZATION_KEY_ID: ${{ secrets.MACOS_NOTARIZATION_KEY_ID }} + MACOS_NOTARIZATION_KEY: ${{ secrets.MACOS_NOTARIZATION_KEY }} + # Bugsnag error tracking configuration + BUGSNAG_API_KEY: ${{ secrets.BUGSNAG_API_KEY }} diff --git a/.github/workflows/release-pypi.yml b/.github/workflows/release-pypi.yml new file mode 100644 index 0000000..bf24713 --- /dev/null +++ b/.github/workflows/release-pypi.yml @@ -0,0 +1,158 @@ +name: Publish to PyPI (reusable) + +# Reusable workflow: builds the thin Python wrapper and publishes it to PyPI. +# Called by the Release orchestrator AFTER the binaries are on the GitHub +# release (the wrapper downloads those binaries at runtime). Not meant to be +# run on its own. + +on: + workflow_call: + inputs: + tag: + description: "Release tag (e.g. v2.5.2 or 2.5.2)." + required: true + type: string + +permissions: + id-token: write # Required for OIDC trusted publishing + contents: read + +jobs: + build-and-publish: + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@v5 + + # Defence in depth: refuse to publish the wrapper unless the binaries it + # downloads at runtime exist on the release. The orchestrator already + # orders this after the build, so this should pass immediately — but it + # keeps this workflow safe to call in any order. + - name: Verify release binaries exist + env: + GH_TOKEN: ${{ github.token }} + TAG: ${{ inputs.tag }} + run: | + set -euo pipefail + TAG="v${TAG#v}" + REQUIRED=( + checksums.txt + cerebrium_cli_darwin_amd64.tar.gz + cerebrium_cli_darwin_arm64.tar.gz + cerebrium_cli_linux_amd64.tar.gz + cerebrium_cli_linux_arm64.tar.gz + cerebrium_cli_windows_amd64.zip + cerebrium_cli_windows_arm64.zip + ) + DEADLINE=$(( SECONDS + 300 )) + while :; do + ASSETS="$(gh release view "$TAG" --json assets --jq '.assets[].name' 2>/dev/null || true)" + MISSING=() + for a in "${REQUIRED[@]}"; do + grep -qx "$a" <<<"$ASSETS" || MISSING+=("$a") + done + if [ ${#MISSING[@]} -eq 0 ]; then + echo "All release binaries present for $TAG." + break + fi + if [ "$SECONDS" -ge "$DEADLINE" ]; then + echo "ERROR: release $TAG is missing binaries: ${MISSING[*]}" >&2 + echo "Refusing to publish the PyPI wrapper for a release without binaries." >&2 + exit 1 + fi + echo "Waiting for binaries (missing: ${MISSING[*]})..." + sleep 15 + done + + - name: Set up Python + uses: actions/setup-python@v6 + with: + python-version: "3.12" + + - name: Install build tools + run: | + python -m pip install --upgrade pip + pip install build twine + + - name: Update version in Python package + env: + TAG: ${{ inputs.tag }} + run: | + set -euo pipefail + # Get version from tag (e.g., "v2.1.0" or "v2.1.0-beta.1") + GITHUB_VERSION="${TAG#v}" # Remove 'v' prefix + + # Update VERSION in cerebrium_cli.py (stores GitHub/semver format) + sed -i "s/^VERSION = \".*\"/VERSION = \"$GITHUB_VERSION\"/" python-wrapper/cerebrium_cli.py + + echo "Updated VERSION in cerebrium_cli.py to: $GITHUB_VERSION" + + # Convert to PEP 440 format for pyproject.toml + cd python-wrapper + PYPI_VERSION=$(python -c "from cerebrium_cli import github_to_pypi_version; print(github_to_pypi_version('$GITHUB_VERSION'))") + cd .. + + # Update version in pyproject.toml (stores PEP 440 format) + sed -i "s/^version = \".*\"/version = \"$PYPI_VERSION\"/" python-wrapper/pyproject.toml + + echo "Updated version in pyproject.toml to: $PYPI_VERSION" + + - name: Build Python package + working-directory: python-wrapper + run: | + python -m build + ls -la dist/ + + - name: Check package + working-directory: python-wrapper + run: | + twine check dist/* + + - name: Publish to PyPI + uses: pypa/gh-action-pypi-publish@release/v1 + with: + packages-dir: python-wrapper/dist/ + skip-existing: true + + - name: Test installation from PyPI + run: | + set -euo pipefail + + # Get the PEP 440 version for pip install + cd python-wrapper + PYPI_VERSION=$(python -c "from cerebrium_cli import __version__; print(__version__)") + cd .. + + # Create a clean virtual environment + python -m venv test_env + source test_env/bin/activate + + # Retry installation with exponential backoff (PyPI propagation lag) + MAX_ATTEMPTS=5 + DELAY=30 + installed=false + for ATTEMPT in $(seq 1 "$MAX_ATTEMPTS"); do + echo "Attempt $ATTEMPT of $MAX_ATTEMPTS..." + if pip install "cerebrium==$PYPI_VERSION"; then + installed=true + break + fi + if [ "$ATTEMPT" -lt "$MAX_ATTEMPTS" ]; then + echo "Package not yet available on PyPI. Waiting ${DELAY}s before retry..." + sleep "$DELAY" + DELAY=$((DELAY * 2)) # Exponential backoff + fi + done + + if [ "$installed" != true ]; then + echo "ERROR: Failed to install cerebrium $PYPI_VERSION from PyPI after $MAX_ATTEMPTS attempts" >&2 + exit 1 + fi + echo "Successfully installed cerebrium $PYPI_VERSION from PyPI" + + # Run the wrapper end-to-end: downloads the real binary + checksums + # from the GitHub release. A 404 / checksum failure MUST fail the job + # (set -e ensures it does — no swallowed exit code). + echo "Testing cerebrium command (downloads binary from release)..." + cerebrium version + echo "Binary download and execution successful!" diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml new file mode 100644 index 0000000..4d23d73 --- /dev/null +++ b/.github/workflows/release.yml @@ -0,0 +1,211 @@ +name: Release + +# Single entry point for cutting a release. One dispatch runs the whole +# pipeline in order, in one place you can watch: +# +# validate → test → tag → goreleaser → verify-assets → wrapper-smoke → pypi +# └ cleanup (on failure) +# +# Steps are ordered with `needs:` within ONE run — they do NOT trigger each +# other via tag-push / release-published events. Event chaining is what made +# the old flow fail silently (a GITHUB_TOKEN-pushed tag never triggered the +# build, so the wrapper shipped to PyPI with no binaries behind it). + +on: + workflow_dispatch: + inputs: + version: + description: "Version to release (e.g. v2.5.2 or 2.5.2 — leading v added if missing)" + required: true + type: string + commit: + description: "Commit SHA to release (defaults to main HEAD)" + required: false + type: string + skip-tests: + description: "Skip the test matrix (use only for re-runs of already-tested commits)" + required: false + default: false + type: boolean + +permissions: + contents: write + packages: write + id-token: write + +jobs: + validate: + name: Validate + runs-on: ubuntu-latest + outputs: + tag: ${{ steps.v.outputs.tag }} + version: ${{ steps.v.outputs.version }} + ref: ${{ steps.v.outputs.ref }} + sha: ${{ steps.v.outputs.sha }} + steps: + - name: Checkout + uses: actions/checkout@v5 + with: + ref: ${{ github.event.inputs.commit || 'main' }} + fetch-depth: 0 + fetch-tags: true + + - name: Normalize version, resolve commit, reject existing tag + id: v + run: | + set -euo pipefail + RAW="${{ github.event.inputs.version }}" + TAG="v${RAW#v}" + if [[ ! "$TAG" =~ ^v[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?$ ]]; then + echo "ERROR: '$TAG' is not a valid semver tag (expected vMAJOR.MINOR.PATCH[-prerelease])" >&2 + exit 1 + fi + if git rev-parse -q --verify "refs/tags/$TAG" >/dev/null; then + echo "ERROR: tag $TAG already exists" >&2 + exit 1 + fi + SHA="$(git rev-parse HEAD)" + { + echo "tag=$TAG" + echo "version=${TAG#v}" + echo "ref=refs/tags/$TAG" + echo "sha=$SHA" + } >> "$GITHUB_OUTPUT" + echo "Will release $TAG at $SHA" + + test: + name: Test + needs: [validate] + if: ${{ github.event.inputs.skip-tests != 'true' }} + uses: ./.github/workflows/test.yml + with: + ref: ${{ needs.validate.outputs.sha }} + + tag: + name: Create tag + needs: [validate, test] + # Run once tests pass (or were skipped). The tag must exist before + # GoReleaser, which derives the version from it. + if: ${{ always() && needs.validate.result == 'success' && (needs.test.result == 'success' || needs.test.result == 'skipped') }} + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v5 + with: + ref: ${{ needs.validate.outputs.sha }} + fetch-depth: 0 + + - name: Create and push annotated tag + env: + TAG: ${{ needs.validate.outputs.tag }} + SHA: ${{ needs.validate.outputs.sha }} + run: | + set -euo pipefail + git config user.name "github-actions[bot]" + git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + git tag -a "$TAG" -m "$TAG" "$SHA" + git push origin "$TAG" + echo "Tagged $SHA as $TAG" + + goreleaser: + name: Build & publish binaries + needs: [validate, tag] + uses: ./.github/workflows/release-goreleaser.yml + with: + ref: ${{ needs.validate.outputs.ref }} + secrets: inherit + + verify-assets: + name: Verify release assets + needs: [validate, goreleaser] + runs-on: ubuntu-latest + steps: + - name: Assert all binaries + checksums are present + env: + GH_TOKEN: ${{ github.token }} + GH_REPO: ${{ github.repository }} + TAG: ${{ needs.validate.outputs.tag }} + run: | + set -euo pipefail + REQUIRED=( + checksums.txt + cerebrium_cli_darwin_amd64.tar.gz + cerebrium_cli_darwin_arm64.tar.gz + cerebrium_cli_linux_amd64.tar.gz + cerebrium_cli_linux_arm64.tar.gz + cerebrium_cli_windows_amd64.zip + cerebrium_cli_windows_arm64.zip + ) + ASSETS="$(gh release view "$TAG" --json assets --jq '.assets[].name')" + MISSING=() + for a in "${REQUIRED[@]}"; do + grep -qx "$a" <<<"$ASSETS" || MISSING+=("$a") + done + if [ ${#MISSING[@]} -ne 0 ]; then + echo "ERROR: release $TAG is missing assets: ${MISSING[*]}" >&2 + exit 1 + fi + echo "All release assets present for $TAG." + + wrapper-smoke: + name: Wrapper smoke test + needs: [validate, verify-assets] + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v5 + with: + ref: ${{ needs.validate.outputs.ref }} + + - name: Set up Python + uses: actions/setup-python@v6 + with: + python-version: "3.12" + + # Build and run the wrapper BEFORE publishing to PyPI. This exercises the + # real "download binary + checksums from the GitHub release" path against + # the freshly published release. If it fails, we never publish to PyPI + # (which can't be un-published), and cleanup tears down the release. + - name: Build, install and run the wrapper against the release + env: + VERSION: ${{ needs.validate.outputs.version }} + run: | + set -euo pipefail + # Point the wrapper at the version we just released. + sed -i "s/^VERSION = \".*\"/VERSION = \"$VERSION\"/" python-wrapper/cerebrium_cli.py + python -m pip install --upgrade pip build + ( cd python-wrapper && python -m build ) + python -m venv smoke_env + source smoke_env/bin/activate + pip install python-wrapper/dist/cerebrium-*.whl + echo "Running cerebrium version (downloads binary from release $VERSION)..." + cerebrium version + echo "Wrapper works against release $VERSION." + + pypi: + name: Publish to PyPI + needs: [validate, wrapper-smoke] + uses: ./.github/workflows/release-pypi.yml + with: + tag: ${{ needs.validate.outputs.tag }} + secrets: inherit + + cleanup: + name: Cleanup failed release + needs: [validate, tag, goreleaser, verify-assets, wrapper-smoke, pypi] + # Only on failure, and only if we got far enough to create a tag. + if: ${{ failure() && needs.validate.result == 'success' }} + runs-on: ubuntu-latest + steps: + - name: Delete the tag and (draft/partial) release + env: + GH_TOKEN: ${{ github.token }} + GH_REPO: ${{ github.repository }} + TAG: ${{ needs.validate.outputs.tag }} + run: | + set -uo pipefail + echo "Release failed — cleaning up $TAG so it can be re-run." + # PyPI cannot be un-published; this only tears down the GitHub side. + gh release delete "$TAG" --yes --cleanup-tag 2>/dev/null || true + gh api -X DELETE "repos/${GH_REPO}/git/refs/tags/${TAG}" 2>/dev/null || true + echo "Cleanup done." diff --git a/.github/workflows/tag-and-release.yml b/.github/workflows/tag-and-release.yml deleted file mode 100644 index b7d61a7..0000000 --- a/.github/workflows/tag-and-release.yml +++ /dev/null @@ -1,67 +0,0 @@ -name: Release - -on: - workflow_dispatch: - inputs: - version: - description: "Version to tag (e.g. v2.4.1 or 2.4.1 — leading v added if missing)" - required: true - type: string - commit: - description: "Commit SHA to tag (defaults to main HEAD)" - required: false - type: string - -permissions: - contents: write - -jobs: - tag: - runs-on: ubuntu-latest - steps: - - name: Checkout - uses: actions/checkout@v5 - with: - fetch-depth: 0 - ref: ${{ inputs.commit || 'main' }} - - - name: Normalize and validate version - id: ver - run: | - RAW="${{ inputs.version }}" - TAG="${RAW#v}" - TAG="v${TAG}" - if [[ ! "$TAG" =~ ^v[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?$ ]]; then - echo "ERROR: '$TAG' is not a valid semver tag (expected vMAJOR.MINOR.PATCH[-prerelease])" >&2 - exit 1 - fi - echo "tag=$TAG" >> "$GITHUB_OUTPUT" - - - name: Reject if tag already exists - run: | - TAG="${{ steps.ver.outputs.tag }}" - if git rev-parse -q --verify "refs/tags/$TAG" >/dev/null; then - echo "ERROR: tag $TAG already exists" >&2 - exit 1 - fi - - - name: Create GitHub release and tag - # Creating the release with a PAT (not the default GITHUB_TOKEN) makes - # gh create the tag, which produces a tag-push event that triggers the - # Deploy Release workflow. Tags created with GITHUB_TOKEN do not trigger - # other workflows. - env: - GH_TOKEN: ${{ secrets.GH_PAT }} - run: | - TAG="${{ steps.ver.outputs.tag }}" - SHA="$(git rev-parse HEAD)" - PRERELEASE_FLAG="" - if [[ "$TAG" =~ -([Rr][Cc]|beta|alpha) ]]; then - PRERELEASE_FLAG="--prerelease" - fi - gh release create "$TAG" \ - --title "$TAG" \ - --target "$SHA" \ - --generate-notes \ - $PRERELEASE_FLAG - echo "Created release $TAG at $SHA. Deploy Release workflow will run on the tag push." >> "$GITHUB_STEP_SUMMARY" diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index c047f12..dfb569a 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -4,6 +4,14 @@ on: pull_request: branches: [main, development] workflow_dispatch: + # Callable from the Release orchestrator so the exact release commit is + # exercised by the same matrix that gates PRs. + workflow_call: + inputs: + ref: + description: "Git ref (commit/tag) to test. Defaults to the triggering ref." + required: false + type: string permissions: contents: read @@ -20,6 +28,8 @@ jobs: steps: - name: Checkout code uses: actions/checkout@v5 + with: + ref: ${{ inputs.ref || github.ref }} - name: Configure Git line endings # Ensure .gitattributes rules are respected on all platforms diff --git a/RELEASING.md b/RELEASING.md index 4e00163..e637dcf 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -16,32 +16,40 @@ For the Go CLI migration: ## How to Create a Release -### Step 1: Tag the Release +Releases are cut from a **single workflow** — `Release` (`.github/workflows/release.yml`). +Do **not** push a tag by hand: a manually pushed tag triggers nothing, and a tag +pushed by CI's `GITHUB_TOKEN` cannot trigger downstream workflows. The orchestrator +creates the tag itself as one ordered step in the pipeline. + +### Run the Release workflow + +From the GitHub UI: **Actions → Release → Run workflow**, then enter the version +(e.g. `v2.1.0` or `2.1.0` — the `v` is added if missing). Or via the CLI: ```bash -# Create and push a tag -git tag -a v2.1.0 -m "Release v2.1.0" -git push origin v2.1.0 +gh workflow run release.yml -f version=v2.1.0 +# optionally pin a commit (defaults to main HEAD) or skip tests on a re-run: +# -f commit= -f skip-tests=true ``` -**Tag format:** Always use `v` prefix followed by semantic version (e.g., `v2.0.0`) +### What the workflow does (one run, ordered by `needs:`) -### Step 2: Automated Workflows - -Once you push the tag, the following happens automatically: +``` +validate → test → tag → goreleaser → verify-assets → wrapper-smoke → pypi + └ cleanup (on failure) +``` -1. **release.yml**: GoReleaser builds and publishes: - - Binaries for all platforms (macOS, Linux, Windows) - - Archives (tar.gz, zip) - - Checksums - - Updates Homebrew tap - - Creates Debian/RPM packages - - Creates GitHub release with changelog +1. **validate** — normalize the version, reject it if the tag already exists, resolve the commit. +2. **test** — run the full test matrix against that commit (skippable with `skip-tests=true`). +3. **tag** — create and push the annotated tag (no event-chaining; the build is the next job). +4. **goreleaser** — build binaries for all platforms, checksums, Homebrew tap, deb/rpm, and the GitHub release. +5. **verify-assets** — assert every archive + `checksums.txt` is actually on the release. +6. **wrapper-smoke** — build the Python wrapper and run it against the freshly published release (exercises the real binary download) *before* touching PyPI. +7. **pypi** — publish the wrapper to PyPI (`pip install cerebrium`). +8. **cleanup** — on any failure, delete the tag and release so the run can be retried cleanly. (PyPI cannot be un-published — only yanked — which is why PyPI is the very last step, after every other check has passed.) -2. **pypi-publish.yml**: Python wrapper publishing: - - Builds the Python package - - Publishes to PyPI (for `pip install cerebrium`) - - Handles beta/RC versions appropriately +Because everything runs in one workflow, the Actions run page shows exactly where a +release stopped — there is no silent hand-off between workflows. ## What Gets Released @@ -85,22 +93,17 @@ cerebrium version ## Pre-release Versions -For beta or release candidate versions: +For beta or release candidate versions, run the same workflow with a prerelease version: ```bash -# Beta release -git tag -a v2.1.0-beta.1 -m "Release v2.1.0-beta.1" -git push origin v2.1.0-beta.1 - -# Release candidate -git tag -a v2.1.0-rc.1 -m "Release v2.1.0-rc.1" -git push origin v2.1.0-rc.1 +gh workflow run release.yml -f version=v2.1.0-beta.1 +gh workflow run release.yml -f version=v2.1.0-rc.1 ``` -These will: -- Create a GitHub pre-release -- Not update the Homebrew formula (stable releases only) -- Be available on PyPI with appropriate version specifier +GoReleaser detects the prerelease from the tag (`prerelease: auto`) and: +- Creates a GitHub pre-release +- Does **not** update the Homebrew formula (stable releases only) +- Publishes to PyPI with the appropriate version specifier (e.g. `2.1.0b1`) ## Local Testing @@ -115,12 +118,17 @@ make build VERSION=2.1.0 ./bin/cerebrium version ``` +To exercise the **full pipeline** end-to-end without affecting stable users, run the +workflow against a throwaway prerelease tag (e.g. `gh workflow run release.yml -f version=v0.0.1-rc.1`) +and let `cleanup` (or a manual `gh release delete v0.0.1-rc.1 --cleanup-tag`) tear it down. + ## Required Secrets The following secrets must be configured in GitHub repository settings: -- **GH_PAT**: GitHub Personal Access Token with repo scope (for releases and Homebrew tap updates) -- **PYPI_API_TOKEN**: PyPI API token for publishing Python packages +- **GH_PAT**: GitHub Personal Access Token with `repo` + `workflow` scope (used by GoReleaser for the release and Homebrew tap updates). +- PyPI publishing uses **OIDC trusted publishing** (`id-token: write`), so no PyPI API token is required — the project must be configured as a trusted publisher on PyPI for this repo's `release-pypi.yml` workflow. +- macOS signing/notarization secrets (`MACOS_CERTIFICATE_P12`, `MACOS_CERTIFICATE_PASSWORD`, `MACOS_NOTARIZATION_ISSUER_ID`, `MACOS_NOTARIZATION_KEY_ID`, `MACOS_NOTARIZATION_KEY`) and `BUGSNAG_API_KEY`. ## 🔔 Update Notifications @@ -147,21 +155,20 @@ Update with: # Check current version cerebrium version -# Create a patch release (bug fixes) -git tag -a v2.0.1 -m "Release v2.0.1: Fix authentication bug" -git push origin v2.0.1 +# Patch release (bug fixes) +gh workflow run release.yml -f version=v2.0.1 + +# Minor release (new features) +gh workflow run release.yml -f version=v2.1.0 -# Create a minor release (new features) -git tag -a v2.1.0 -m "Release v2.1.0: Add support for custom regions" -git push origin v2.1.0 +# Major release (breaking changes) +gh workflow run release.yml -f version=v3.0.0 -# Create a major release (breaking changes) -git tag -a v3.0.0 -m "Release v3.0.0: New configuration format" -git push origin v3.0.0 +# Watch the run +gh run watch "$(gh run list --workflow=release.yml --limit 1 --json databaseId -q '.[0].databaseId')" -# Delete a tag if needed -git tag -d v2.0.1 -git push origin --delete v2.0.1 +# Roll back a botched release (tag + GitHub release; PyPI can only be yanked) +gh release delete v2.0.1 --cleanup-tag ``` ## Troubleshooting @@ -172,9 +179,9 @@ git push origin --delete v2.0.1 - Verify `.goreleaser.yaml` configuration is valid ### PyPI publish fails -- Ensure version doesn't already exist on PyPI -- Check PyPI API token is valid -- Verify `python-wrapper/setup.py` is correctly formatted +- Ensure the version doesn't already exist on PyPI (publishes use `skip-existing`, but a partial prior upload can still conflict) +- Confirm the repo is registered as a PyPI trusted publisher for `release-pypi.yml` (OIDC) +- Verify `python-wrapper/pyproject.toml` is correctly formatted ### Homebrew formula not updating - Only stable releases update Homebrew (not pre-releases) diff --git a/python-wrapper/PYPI_RELEASE_STRATEGY.md b/python-wrapper/PYPI_RELEASE_STRATEGY.md index 1f7112b..565ffe2 100644 --- a/python-wrapper/PYPI_RELEASE_STRATEGY.md +++ b/python-wrapper/PYPI_RELEASE_STRATEGY.md @@ -2,14 +2,16 @@ ## How It Works -1. Create a GitHub release (e.g., `v2.1.0` or `v2.1.0-beta.1`) -2. `.github/workflows/pypi-publish.yml` automatically: +1. Run the `Release` workflow (`.github/workflows/release.yml`) with a version (e.g. `v2.1.0` or `v2.1.0-beta.1`). See [RELEASING.md](../RELEASING.md). +2. After the binaries are built and verified on the GitHub release, the orchestrator calls `.github/workflows/release-pypi.yml`, which: - Updates `VERSION` in `cerebrium_cli.py` (GitHub format: `2.1.0-beta.1`) - Updates `version` in `pyproject.toml` (PEP 440 format: `2.1.0b1`) - Builds and publishes to PyPI - Tests installation -The pip package is a thin wrapper that downloads the Go binary on first run. +The PyPI publish runs **only after** the binaries exist on the release, so the wrapper +never ships pointing at missing binaries. The pip package is a thin wrapper that +downloads the Go binary on first run. ## Version Formats diff --git a/python-wrapper/cerebrium_cli.py b/python-wrapper/cerebrium_cli.py index 2eec384..37f902d 100644 --- a/python-wrapper/cerebrium_cli.py +++ b/python-wrapper/cerebrium_cli.py @@ -19,7 +19,7 @@ from urllib.request import urlopen # DO NOT EDIT: This version is automatically updated by the GitHub Action -# (.github/workflows/pypi-publish.yml) during release. It uses GitHub/semver +# (.github/workflows/release-pypi.yml) during release. It uses GitHub/semver # format (e.g., "2.1.0-beta.1" for beta, "2.1.0" for stable). VERSION = "2.1.0-beta.1" diff --git a/python-wrapper/pyproject.toml b/python-wrapper/pyproject.toml index ab03711..6b5573f 100644 --- a/python-wrapper/pyproject.toml +++ b/python-wrapper/pyproject.toml @@ -5,7 +5,7 @@ build-backend = "setuptools.build_meta" [project] name = "cerebrium" # DO NOT EDIT: This version is automatically updated by the GitHub Action -# (.github/workflows/pypi-publish.yml) during release. Uses PEP 440 format. +# (.github/workflows/release-pypi.yml) during release. Uses PEP 440 format. version = "2.1.0b1" description = "CLI for deploying and managing Cerebrium apps" readme = "README.md" From f8d409dbb0864c3049ac291d83501a5ea1236de2 Mon Sep 17 00:00:00 2001 From: Yaseen Hamdulay Date: Wed, 10 Jun 2026 14:39:00 +0200 Subject: [PATCH 2/6] ci: keep PyPI workflow filename as pypi-publish.yml Reuse the existing pypi-publish.yml filename (swap its trigger to workflow_call + apply the fixes) rather than renaming to release-pypi.yml. PyPI trusted publishing binds to the workflow filename, so keeping the name avoids reconfiguring the trusted publisher on PyPI. Co-Authored-By: Claude Opus 4.8 (1M context) --- .github/workflows/{release-pypi.yml => pypi-publish.yml} | 0 .github/workflows/release.yml | 2 +- RELEASING.md | 4 ++-- python-wrapper/PYPI_RELEASE_STRATEGY.md | 2 +- python-wrapper/cerebrium_cli.py | 2 +- python-wrapper/pyproject.toml | 2 +- 6 files changed, 6 insertions(+), 6 deletions(-) rename .github/workflows/{release-pypi.yml => pypi-publish.yml} (100%) diff --git a/.github/workflows/release-pypi.yml b/.github/workflows/pypi-publish.yml similarity index 100% rename from .github/workflows/release-pypi.yml rename to .github/workflows/pypi-publish.yml diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 4d23d73..bd695ca 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -185,7 +185,7 @@ jobs: pypi: name: Publish to PyPI needs: [validate, wrapper-smoke] - uses: ./.github/workflows/release-pypi.yml + uses: ./.github/workflows/pypi-publish.yml with: tag: ${{ needs.validate.outputs.tag }} secrets: inherit diff --git a/RELEASING.md b/RELEASING.md index e637dcf..85a6937 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -127,7 +127,7 @@ and let `cleanup` (or a manual `gh release delete v0.0.1-rc.1 --cleanup-tag`) te The following secrets must be configured in GitHub repository settings: - **GH_PAT**: GitHub Personal Access Token with `repo` + `workflow` scope (used by GoReleaser for the release and Homebrew tap updates). -- PyPI publishing uses **OIDC trusted publishing** (`id-token: write`), so no PyPI API token is required — the project must be configured as a trusted publisher on PyPI for this repo's `release-pypi.yml` workflow. +- PyPI publishing uses **OIDC trusted publishing** (`id-token: write`), so no PyPI API token is required — the project must be configured as a trusted publisher on PyPI for this repo's `pypi-publish.yml` workflow. - macOS signing/notarization secrets (`MACOS_CERTIFICATE_P12`, `MACOS_CERTIFICATE_PASSWORD`, `MACOS_NOTARIZATION_ISSUER_ID`, `MACOS_NOTARIZATION_KEY_ID`, `MACOS_NOTARIZATION_KEY`) and `BUGSNAG_API_KEY`. ## 🔔 Update Notifications @@ -180,7 +180,7 @@ gh release delete v2.0.1 --cleanup-tag ### PyPI publish fails - Ensure the version doesn't already exist on PyPI (publishes use `skip-existing`, but a partial prior upload can still conflict) -- Confirm the repo is registered as a PyPI trusted publisher for `release-pypi.yml` (OIDC) +- Confirm the repo is registered as a PyPI trusted publisher for `pypi-publish.yml` (OIDC) - Verify `python-wrapper/pyproject.toml` is correctly formatted ### Homebrew formula not updating diff --git a/python-wrapper/PYPI_RELEASE_STRATEGY.md b/python-wrapper/PYPI_RELEASE_STRATEGY.md index 565ffe2..e317e8f 100644 --- a/python-wrapper/PYPI_RELEASE_STRATEGY.md +++ b/python-wrapper/PYPI_RELEASE_STRATEGY.md @@ -3,7 +3,7 @@ ## How It Works 1. Run the `Release` workflow (`.github/workflows/release.yml`) with a version (e.g. `v2.1.0` or `v2.1.0-beta.1`). See [RELEASING.md](../RELEASING.md). -2. After the binaries are built and verified on the GitHub release, the orchestrator calls `.github/workflows/release-pypi.yml`, which: +2. After the binaries are built and verified on the GitHub release, the orchestrator calls `.github/workflows/pypi-publish.yml`, which: - Updates `VERSION` in `cerebrium_cli.py` (GitHub format: `2.1.0-beta.1`) - Updates `version` in `pyproject.toml` (PEP 440 format: `2.1.0b1`) - Builds and publishes to PyPI diff --git a/python-wrapper/cerebrium_cli.py b/python-wrapper/cerebrium_cli.py index 37f902d..2eec384 100644 --- a/python-wrapper/cerebrium_cli.py +++ b/python-wrapper/cerebrium_cli.py @@ -19,7 +19,7 @@ from urllib.request import urlopen # DO NOT EDIT: This version is automatically updated by the GitHub Action -# (.github/workflows/release-pypi.yml) during release. It uses GitHub/semver +# (.github/workflows/pypi-publish.yml) during release. It uses GitHub/semver # format (e.g., "2.1.0-beta.1" for beta, "2.1.0" for stable). VERSION = "2.1.0-beta.1" diff --git a/python-wrapper/pyproject.toml b/python-wrapper/pyproject.toml index 6b5573f..ab03711 100644 --- a/python-wrapper/pyproject.toml +++ b/python-wrapper/pyproject.toml @@ -5,7 +5,7 @@ build-backend = "setuptools.build_meta" [project] name = "cerebrium" # DO NOT EDIT: This version is automatically updated by the GitHub Action -# (.github/workflows/release-pypi.yml) during release. Uses PEP 440 format. +# (.github/workflows/pypi-publish.yml) during release. Uses PEP 440 format. version = "2.1.0b1" description = "CLI for deploying and managing Cerebrium apps" readme = "README.md" From 2725cfca0de7bb4b54ef100d79f83f1d4aa95cf2 Mon Sep 17 00:00:00 2001 From: Yaseen Hamdulay Date: Wed, 10 Jun 2026 14:40:22 +0200 Subject: [PATCH 3/6] ci: keep goreleaser reusable as deploy-release.yml MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Avoid the needless rename to release-goreleaser.yml — reuse the existing deploy-release.yml filename for the (now workflow_call-only) GoReleaser build. The only genuinely new file is the orchestrator (release.yml); no existing workflow is renamed. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../workflows/{release-goreleaser.yml => deploy-release.yml} | 0 .github/workflows/release.yml | 2 +- 2 files changed, 1 insertion(+), 1 deletion(-) rename .github/workflows/{release-goreleaser.yml => deploy-release.yml} (100%) diff --git a/.github/workflows/release-goreleaser.yml b/.github/workflows/deploy-release.yml similarity index 100% rename from .github/workflows/release-goreleaser.yml rename to .github/workflows/deploy-release.yml diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index bd695ca..1fc75d4 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -110,7 +110,7 @@ jobs: goreleaser: name: Build & publish binaries needs: [validate, tag] - uses: ./.github/workflows/release-goreleaser.yml + uses: ./.github/workflows/deploy-release.yml with: ref: ${{ needs.validate.outputs.ref }} secrets: inherit From 4d33bb0d166107595f180459821156375a0c68b1 Mon Sep 17 00:00:00 2001 From: Yaseen Hamdulay Date: Wed, 10 Jun 2026 14:46:16 +0200 Subject: [PATCH 4/6] ci: auto-bump patch version when release version is omitted Make the Release workflow's `version` input optional. When left blank, validate resolves the latest stable vX.Y.Z tag and bumps the patch (e.g. v2.5.2 -> v2.5.3). Minor/major bumps still require an explicit version. Errors if no version is given and no stable tag exists. Co-Authored-By: Claude Opus 4.8 (1M context) --- .github/workflows/release.yml | 20 +++++++++++++++++--- RELEASING.md | 13 +++++++++++-- 2 files changed, 28 insertions(+), 5 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 1fc75d4..e3d7cce 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -15,8 +15,8 @@ on: workflow_dispatch: inputs: version: - description: "Version to release (e.g. v2.5.2 or 2.5.2 — leading v added if missing)" - required: true + description: "Version to release (e.g. v2.5.2 or 2.5.2). Leave blank to auto-bump the patch of the latest stable release." + required: false type: string commit: description: "Commit SHA to release (defaults to main HEAD)" @@ -55,7 +55,21 @@ jobs: run: | set -euo pipefail RAW="${{ github.event.inputs.version }}" - TAG="v${RAW#v}" + if [ -z "$RAW" ]; then + # No version given: bump the patch of the latest stable (vX.Y.Z) tag. + LATEST="$(git tag -l 'v[0-9]*' | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -n1 || true)" + if [ -z "$LATEST" ]; then + echo "ERROR: no version provided and no existing vX.Y.Z release tag to bump from" >&2 + exit 1 + fi + base="${LATEST#v}" + major="${base%%.*}"; rest="${base#*.}" + minor="${rest%%.*}"; patch="${rest#*.}" + TAG="v${major}.${minor}.$((patch + 1))" + echo "No version provided; bumping $LATEST -> $TAG" + else + TAG="v${RAW#v}" + fi if [[ ! "$TAG" =~ ^v[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?$ ]]; then echo "ERROR: '$TAG' is not a valid semver tag (expected vMAJOR.MINOR.PATCH[-prerelease])" >&2 exit 1 diff --git a/RELEASING.md b/RELEASING.md index 85a6937..35fc66f 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -23,15 +23,24 @@ creates the tag itself as one ordered step in the pipeline. ### Run the Release workflow -From the GitHub UI: **Actions → Release → Run workflow**, then enter the version -(e.g. `v2.1.0` or `2.1.0` — the `v` is added if missing). Or via the CLI: +From the GitHub UI: **Actions → Release → Run workflow**. Enter a version +(e.g. `v2.1.0` or `2.1.0` — the `v` is added if missing), or **leave it blank to +auto-bump the patch** of the latest stable release (e.g. `v2.5.2` → `v2.5.3`). Or via the CLI: ```bash +# Patch bump of the latest stable release (no version needed): +gh workflow run release.yml + +# Or pin an explicit version: gh workflow run release.yml -f version=v2.1.0 + # optionally pin a commit (defaults to main HEAD) or skip tests on a re-run: # -f commit= -f skip-tests=true ``` +A minor or major bump must be given explicitly (`-f version=v2.6.0`); the blank-version +default only ever increments the patch. + ### What the workflow does (one run, ordered by `needs:`) ``` From 91b8392cf489b970bbcb7d147360888f8e6c817c Mon Sep 17 00:00:00 2001 From: Yaseen Hamdulay Date: Wed, 10 Jun 2026 14:50:53 +0200 Subject: [PATCH 5/6] ci: trim explanatory comments from release workflows Co-Authored-By: Claude Opus 4.8 (1M context) --- .github/workflows/deploy-release.yml | 8 -------- .github/workflows/pypi-publish.yml | 5 ----- .github/workflows/release.yml | 5 ----- 3 files changed, 18 deletions(-) diff --git a/.github/workflows/deploy-release.yml b/.github/workflows/deploy-release.yml index ea89c34..0c362ea 100644 --- a/.github/workflows/deploy-release.yml +++ b/.github/workflows/deploy-release.yml @@ -1,10 +1,5 @@ name: Build & Publish Binaries (reusable) -# Reusable workflow: builds the cross-platform binaries with GoReleaser and -# uploads them (+ checksums.txt) to the GitHub release for the given tag. -# Called by the Release orchestrator after the tag has been created. Not meant -# to be run on its own. - on: workflow_call: inputs: @@ -26,9 +21,6 @@ jobs: - name: Checkout code uses: actions/checkout@v5 with: - # Pin to the tag so GoReleaser's git-state check ("the current tag - # must point at HEAD") always holds. fetch-depth: 0 is required for - # the changelog and tag discovery. ref: ${{ inputs.ref }} fetch-depth: 0 diff --git a/.github/workflows/pypi-publish.yml b/.github/workflows/pypi-publish.yml index bf24713..4500eb6 100644 --- a/.github/workflows/pypi-publish.yml +++ b/.github/workflows/pypi-publish.yml @@ -1,10 +1,5 @@ name: Publish to PyPI (reusable) -# Reusable workflow: builds the thin Python wrapper and publishes it to PyPI. -# Called by the Release orchestrator AFTER the binaries are on the GitHub -# release (the wrapper downloads those binaries at runtime). Not meant to be -# run on its own. - on: workflow_call: inputs: diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index e3d7cce..3c1599a 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -5,11 +5,6 @@ name: Release # # validate → test → tag → goreleaser → verify-assets → wrapper-smoke → pypi # └ cleanup (on failure) -# -# Steps are ordered with `needs:` within ONE run — they do NOT trigger each -# other via tag-push / release-published events. Event chaining is what made -# the old flow fail silently (a GITHUB_TOKEN-pushed tag never triggered the -# build, so the wrapper shipped to PyPI with no binaries behind it). on: workflow_dispatch: From 2b1a474570b2b07cb4efb37549e98947ceb5642b Mon Sep 17 00:00:00 2001 From: Yaseen Hamdulay Date: Wed, 10 Jun 2026 15:28:02 +0200 Subject: [PATCH 6/6] ci: create releases as prerelease, promote after checks, add dry-run - GitHub release is now created as a prerelease (.goreleaser.yaml prerelease: true). A new `promote` job flips a stable release to "Latest" only after verify-assets + wrapper-smoke + pypi all pass. Actual prerelease tags (-rc/-beta) are never promoted. - Add a `dry-run` input to the Release workflow: still pushes the tag and creates a real GitHub prerelease with binaries, and runs the verification jobs, but never promotes, never publishes to PyPI, and skips the Homebrew tap push + macOS notarization. For testing the pipeline end to end without affecting stable users. - Homebrew skip_upload is now keyed explicitly off the tag's semver prerelease component (not release.prerelease, which is forced true), so stable releases still update the tap. - cleanup now only tears down when the build itself fails; once goreleaser succeeds the prerelease is valid and is left in place on a later failure (avoids dangling a pushed Homebrew cask). Co-Authored-By: Claude Opus 4.8 (1M context) --- .github/workflows/deploy-release.yml | 9 ++++- .github/workflows/pypi-publish.yml | 13 +++++++ .github/workflows/release.yml | 57 +++++++++++++++++++++++----- .goreleaser.yaml | 11 ++++-- RELEASING.md | 21 ++++++++-- 5 files changed, 94 insertions(+), 17 deletions(-) diff --git a/.github/workflows/deploy-release.yml b/.github/workflows/deploy-release.yml index 0c362ea..b593dc8 100644 --- a/.github/workflows/deploy-release.yml +++ b/.github/workflows/deploy-release.yml @@ -7,6 +7,11 @@ on: description: "Tag ref to build (e.g. refs/tags/v2.5.2). Must already exist and point at HEAD." required: true type: string + dry-run: + description: "Dry run: create the (pre)release + binaries, but skip the Homebrew tap push and macOS notarization." + required: false + default: false + type: boolean permissions: contents: write @@ -34,7 +39,9 @@ jobs: uses: goreleaser/goreleaser-action@v6 with: version: "~> v2" - args: release --clean + # Dry run still creates the (pre)release + binaries, but skips the + # public Homebrew tap push and the slow macOS notarization. + args: ${{ inputs.dry-run && 'release --clean --skip=homebrew,notarize' || 'release --clean' }} env: GITHUB_TOKEN: ${{ secrets.GH_PAT }} GH_PAT: ${{ secrets.GH_PAT }} diff --git a/.github/workflows/pypi-publish.yml b/.github/workflows/pypi-publish.yml index 4500eb6..df0952b 100644 --- a/.github/workflows/pypi-publish.yml +++ b/.github/workflows/pypi-publish.yml @@ -7,6 +7,11 @@ on: description: "Release tag (e.g. v2.5.2 or 2.5.2)." required: true type: string + dry-run: + description: "Dry run: build the wheel and run twine check, but do not publish to PyPI." + required: false + default: false + type: boolean permissions: id-token: write # Required for OIDC trusted publishing @@ -24,6 +29,7 @@ jobs: # orders this after the build, so this should pass immediately — but it # keeps this workflow safe to call in any order. - name: Verify release binaries exist + if: ${{ !inputs.dry-run }} env: GH_TOKEN: ${{ github.token }} TAG: ${{ inputs.tag }} @@ -104,12 +110,19 @@ jobs: twine check dist/* - name: Publish to PyPI + if: ${{ !inputs.dry-run }} uses: pypa/gh-action-pypi-publish@release/v1 with: packages-dir: python-wrapper/dist/ skip-existing: true + - name: Dry run — skipping PyPI publish + if: ${{ inputs.dry-run }} + run: | + echo "dry-run: built wheel and ran twine check; not publishing to PyPI." + - name: Test installation from PyPI + if: ${{ !inputs.dry-run }} run: | set -euo pipefail diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 3c1599a..d0fd302 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -3,8 +3,13 @@ name: Release # Single entry point for cutting a release. One dispatch runs the whole # pipeline in order, in one place you can watch: # -# validate → test → tag → goreleaser → verify-assets → wrapper-smoke → pypi -# └ cleanup (on failure) +# validate → test → tag → goreleaser → verify-assets → wrapper-smoke → pypi → promote +# └ cleanup (on failure) +# +# The GitHub release is always created as a PRERELEASE; `promote` flips a stable +# release to "Latest" only after every check passes. `dry-run` exercises the +# whole pipeline for real (tag + prerelease release + binaries) but never +# promotes and never publishes to PyPI/Homebrew. on: workflow_dispatch: @@ -22,6 +27,11 @@ on: required: false default: false type: boolean + dry-run: + description: "Dry run: build everything but publish nothing (no tag push, no GitHub release, no PyPI, no Homebrew). For testing the pipeline." + required: false + default: false + type: boolean permissions: contents: write @@ -37,6 +47,7 @@ jobs: version: ${{ steps.v.outputs.version }} ref: ${{ steps.v.outputs.ref }} sha: ${{ steps.v.outputs.sha }} + is_prerelease: ${{ steps.v.outputs.is_prerelease }} steps: - name: Checkout uses: actions/checkout@v5 @@ -74,13 +85,18 @@ jobs: exit 1 fi SHA="$(git rev-parse HEAD)" + case "$TAG" in + *-*) PRERELEASE=true ;; # has a semver prerelease suffix (-rc/-beta/...) + *) PRERELEASE=false ;; + esac { echo "tag=$TAG" echo "version=${TAG#v}" echo "ref=refs/tags/$TAG" echo "sha=$SHA" + echo "is_prerelease=$PRERELEASE" } >> "$GITHUB_OUTPUT" - echo "Will release $TAG at $SHA" + echo "Will release $TAG at $SHA (prerelease=$PRERELEASE)" test: name: Test @@ -122,6 +138,7 @@ jobs: uses: ./.github/workflows/deploy-release.yml with: ref: ${{ needs.validate.outputs.ref }} + dry-run: ${{ inputs.dry-run }} secrets: inherit verify-assets: @@ -197,24 +214,46 @@ jobs: uses: ./.github/workflows/pypi-publish.yml with: tag: ${{ needs.validate.outputs.tag }} + dry-run: ${{ inputs.dry-run }} secrets: inherit + promote: + name: Promote to latest + needs: [validate, pypi] + # Flip the prerelease to "Latest" only for a real, stable release. Actual + # prerelease tags (-rc/-beta) stay prereleases; dry runs are never promoted, + # so the release is left as a prerelease for inspection. + if: ${{ !inputs.dry-run && needs.validate.outputs.is_prerelease == 'false' }} + runs-on: ubuntu-latest + steps: + - name: Mark the GitHub release as latest + env: + GH_TOKEN: ${{ github.token }} + GH_REPO: ${{ github.repository }} + TAG: ${{ needs.validate.outputs.tag }} + run: | + set -euo pipefail + gh release edit "$TAG" --prerelease=false --latest + echo "Promoted $TAG to the latest stable release." + cleanup: name: Cleanup failed release - needs: [validate, tag, goreleaser, verify-assets, wrapper-smoke, pypi] - # Only on failure, and only if we got far enough to create a tag. - if: ${{ failure() && needs.validate.result == 'success' }} + needs: [validate, tag, goreleaser] + # Only tear down when the BUILD itself failed — i.e. there's a half-created + # tag/release. Once goreleaser succeeds the prerelease (and any binaries the + # Homebrew cask points at) is valid, so a later failure (verify/smoke/pypi) + # leaves it in place as an un-promoted prerelease to inspect or retry. + if: ${{ failure() && needs.validate.result == 'success' && needs.goreleaser.result != 'success' }} runs-on: ubuntu-latest steps: - - name: Delete the tag and (draft/partial) release + - name: Delete the tag and partial release env: GH_TOKEN: ${{ github.token }} GH_REPO: ${{ github.repository }} TAG: ${{ needs.validate.outputs.tag }} run: | set -uo pipefail - echo "Release failed — cleaning up $TAG so it can be re-run." - # PyPI cannot be un-published; this only tears down the GitHub side. + echo "Build failed — cleaning up $TAG so it can be re-run." gh release delete "$TAG" --yes --cleanup-tag 2>/dev/null || true gh api -X DELETE "repos/${GH_REPO}/git/refs/tags/${TAG}" 2>/dev/null || true echo "Cleanup done." diff --git a/.goreleaser.yaml b/.goreleaser.yaml index fcca94b..8829406 100644 --- a/.goreleaser.yaml +++ b/.goreleaser.yaml @@ -123,8 +123,10 @@ homebrew_casks: homepage: https://www.cerebrium.ai description: CLI for deploying and managing Cerebrium apps license: MIT - # Skip upload for prereleases - skip_upload: auto + # Skip the tap update for prerelease tags (-rc/-beta). Keyed off the tag's + # semver prerelease component, NOT release.prerelease (which we force to + # true), so stable releases still update Homebrew. + skip_upload: '{{ if .Prerelease }}true{{ else }}false{{ end }}' # Linux packages nfpms: @@ -152,7 +154,10 @@ release: owner: CerebriumAI name: cerebrium draft: false - prerelease: auto + # Always create the release as a prerelease. The Release workflow's `promote` + # job flips a stable release to "Latest" only after PyPI + the wrapper smoke + # test pass. Actual prerelease tags (-rc/-beta) are never promoted. + prerelease: true name_template: "v{{.Version}}" # Docker images (optional) # dockers: diff --git a/RELEASING.md b/RELEASING.md index 35fc66f..08ab03c 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -44,22 +44,35 @@ default only ever increments the patch. ### What the workflow does (one run, ordered by `needs:`) ``` -validate → test → tag → goreleaser → verify-assets → wrapper-smoke → pypi - └ cleanup (on failure) +validate → test → tag → goreleaser → verify-assets → wrapper-smoke → pypi → promote + └ cleanup (on failure) ``` 1. **validate** — normalize the version, reject it if the tag already exists, resolve the commit. 2. **test** — run the full test matrix against that commit (skippable with `skip-tests=true`). 3. **tag** — create and push the annotated tag (no event-chaining; the build is the next job). -4. **goreleaser** — build binaries for all platforms, checksums, Homebrew tap, deb/rpm, and the GitHub release. +4. **goreleaser** — build binaries for all platforms, checksums, Homebrew tap, deb/rpm, and the GitHub release. **The release is created as a *prerelease*.** 5. **verify-assets** — assert every archive + `checksums.txt` is actually on the release. 6. **wrapper-smoke** — build the Python wrapper and run it against the freshly published release (exercises the real binary download) *before* touching PyPI. 7. **pypi** — publish the wrapper to PyPI (`pip install cerebrium`). -8. **cleanup** — on any failure, delete the tag and release so the run can be retried cleanly. (PyPI cannot be un-published — only yanked — which is why PyPI is the very last step, after every other check has passed.) +8. **promote** — flip a stable release from prerelease to **"Latest"**. Runs only after everything above passes; actual prerelease tags (`-rc`/`-beta`) are left as prereleases. +9. **cleanup** — if the *build* fails (half-created tag/release), delete the tag + release so the run can be retried. Once goreleaser succeeds the prerelease is valid, so a later failure leaves it in place as an un-promoted prerelease. (PyPI cannot be un-published — only yanked — which is why PyPI runs late and `promote` is last.) Because everything runs in one workflow, the Actions run page shows exactly where a release stopped — there is no silent hand-off between workflows. +### Testing the pipeline (dry run) + +```bash +gh workflow run release.yml -f version=v0.0.1-rc.1 -f dry-run=true +``` + +A dry run exercises the whole pipeline for real — it **pushes the tag** and creates a +real GitHub **prerelease** with binaries, and runs `verify-assets` + `wrapper-smoke` — +but it never **promotes** the release to "Latest", never **publishes to PyPI**, and skips +the **Homebrew** tap push and macOS notarization. The prerelease is left in place for you +to inspect; tear it down with `gh release delete --cleanup-tag`. + ## What Gets Released ### GoReleaser Produces: