Skip to content

ci: add CI health gate to release tagging and skip CI for release PRs#4198

Draft
zackverham wants to merge 1 commit into
mainfrom
release-ci-gate
Draft

ci: add CI health gate to release tagging and skip CI for release PRs#4198
zackverham wants to merge 1 commit into
mainfrom
release-ci-gate

Conversation

@zackverham

Copy link
Copy Markdown
Collaborator

Summary

  • Adds a CI health check to tag-on-release-merge.yaml that verifies the last main.yaml run succeeded before creating a release tag. This replaces the manual "wait for main.yaml to complete" step in the release process.
  • Extracts the detect-changes logic from pull-request.yaml into a testable Python script (scripts/detect-code-changes.py). On release branches, package.json changes are validated to contain only version bumps — if they do, full CI is skipped.
  • Updates CONTRIBUTING.md to reflect the automated gate.

Test plan

  • Verify just test-scripts passes (includes 23 new unit tests for the detection logic)
  • Confirm PR CI still runs full suite for non-release branches
  • On a future release, verify the tag workflow checks main CI status before tagging

Closes #4188

🤖 Generated with Claude Code

The tag-on-release-merge workflow now verifies that the last main.yaml
run succeeded before creating a release tag. This replaces the manual
"wait for main.yaml to complete" step in the release process.

Release PRs (which only bump versions and update changelogs) now skip
full CI via a testable Python script that validates package.json diffs
contain only version field changes.

Closes #4188

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@zackverham zackverham requested a review from a team as a code owner May 20, 2026 13:13
@posit-connect-projects

posit-connect-projects Bot commented May 20, 2026

Copy link
Copy Markdown
Contributor

Claude finished @zackverham's task in 1m 23s —— View job


Code Review Complete

  • Read and analyze changed files
  • Review workflow changes
  • Review new Python script and tests
  • Provide detailed feedback

Summary

This PR makes two valuable improvements to the CI/release process:

  1. CI Health Gate: Prevents tagging releases when main.yaml is failing
  2. Release CI Skip: Automatically skips full CI for release PRs that only contain version bumps

Overall, this is a well-structured PR with good test coverage. The extraction of shell logic into a testable Python script is a solid engineering choice.


Review Findings

Strengths

  • Excellent test coverage: 23 unit tests thoroughly cover the detection logic, including edge cases like empty diffs and release branch handling
  • Clean refactor: The 60-line bash script → Python extraction improves maintainability and testability
  • Good separation of concerns: The classify_changes function accepts a get_diff_fn parameter, making it easy to test without Git operations

Suggestions

1. Empty diff edge case in is_version_only_change (Minor)

scripts/detect-code-changes.py:68 returns True for an empty diff:

def is_version_only_change(diff_output: str) -> bool:
    # ...
    return True  # Returns True for empty string

The test at scripts/test_detect_code_changes.py:74 verifies this is intentional. However, consider adding a brief comment explaining why an empty diff should be treated as version-only (e.g., "Empty diff means no content changes, treat as version-only").

Add clarifying comment →

2. Consider logging which files triggered CI skip (Minor)

The script prints "Only documentation/release-metadata files changed" but doesn't detail which files were classified as release-metadata vs docs. For debugging release CI skip behavior, it might help to log:

print(f"Skipped as release metadata: {skipped_files}")

3. CI health check timing consideration (Documentation)

In tag-on-release-merge.yaml:26-43, the check queries the most recent completed main.yaml run. After a release PR merges, there's a window where the previous main run is checked (before the new merge-triggered run completes). This is actually correct behavior since you want to verify main was green before the merge. Consider adding a comment to clarify this is intentional, as future maintainers might wonder.


Verification

The test plan items look correct:

  • just test-scripts includes the new tests via scripts/test_detect_code_changes.py
  • The workflow conditions correctly gate CI based on branch name and file changes
  • The tag workflow will now fail fast if main CI is red

Verdict: Looks good to merge after the minor documentation suggestions are considered. The core logic is solid and well-tested.

@zackverham zackverham marked this pull request as draft May 20, 2026 13:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Investigate whether CI wait is necessary during release

1 participant