Skip to content

ci(ai-validation): skip prepare on Mergify-authored PRs (sagitta hotfix)#2046

Closed
andamasov wants to merge 1 commit into
sagittafrom
yuriy/ai-validation-skip-prepare-on-bot-author-sagitta
Closed

ci(ai-validation): skip prepare on Mergify-authored PRs (sagitta hotfix)#2046
andamasov wants to merge 1 commit into
sagittafrom
yuriy/ai-validation-skip-prepare-on-bot-author-sagitta

Conversation

@andamasov
Copy link
Copy Markdown
Member

Summary

Test plan

  • Merge this PR.
  • Re-run AI Validation on vyos-documentation#2042 — `prepare` + `validate` should both report skipped.

🤖 Generated by robots

Lifts the existing Mergify-author short-circuit (today inside validate's
`secrets-check` step) to a job-level `if:` on `prepare`, so the whole
pipeline skips for backport/queue PRs.

Why now: every Mergify backport whose merge ref shares no shallow
ancestor with the (advanced) base branch fails the prepare step at

  git diff "$BASE...HEAD" --name-only ...
  fatal: FETCH_HEAD...HEAD: no merge base

(because base is `git fetch --no-tags --depth=1` and the merge ref is
`fetch-depth: 2`). Proximate symptom: run 25842928620 on PR #2042
(sagitta backport of #2023). AI Validation isn't a required check so
the queue isn't blocked, but every Mergify backport is left with a red
"prepare" check that adds noise to PR review.

The validate-level skip in commit 0e8a295 was correct for the
"claude-code-action rejects bot-initiated runs" failure mode but
fires too late — prepare has already run and crashed before validate's
`if: needs.prepare.outputs.has_md_changes == 'true'` even evaluates.

Implementation: single job-level `if:` on prepare. validate's
`needs: [prepare]` cascades the skip naturally (skipped needs make
the dependent's expression-based `if:` evaluate against empty outputs).
The in-step author check in validate stays as defense-in-depth.

🤖 Generated by [robots](https://vyos.io)
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 14, 2026

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Central YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: a78725d3-129b-4154-98b4-4543a154e55b

📥 Commits

Reviewing files that changed from the base of the PR and between 32c0775 and e18192c.

📒 Files selected for processing (1)
  • .github/workflows/ai-validation.yml
📜 Recent review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Summary
🧰 Additional context used
🧠 Learnings (2)
📚 Learning: 2026-05-10T22:41:30.936Z
Learnt from: andamasov
Repo: vyos/vyos-documentation PR: 1969
File: .github/workflows/ai-validation.yml:371-371
Timestamp: 2026-05-10T22:41:30.936Z
Learning: When verifying that a GitHub Actions step using a pinned SHA for an action reference is correct for a given tag (especially when tags may be annotated), fetch the tag’s *commit* SHA rather than the tag-object SHA. Do NOT use the tag-object ref path `/git/refs/tags/<tag>` because it returns the SHA of the annotated tag object, not the commit SHA. Instead, query `/repos/<org>/<repo>/tags` and select the tag by name (e.g., `.[] | select(.name == "<tag>") | .commit.sha`), which dereferences annotated tags to the underlying commit SHA required for `uses: <org>/<repo>@<SHA>`.

Applied to files:

  • .github/workflows/ai-validation.yml
📚 Learning: 2026-05-13T21:34:07.309Z
Learnt from: andamasov
Repo: vyos/vyos-documentation PR: 2020
File: .github/workflows/lint-doc.yml:14-27
Timestamp: 2026-05-13T21:34:07.309Z
Learning: In this repo’s GitHub Actions workflows (/.github/workflows/*.{yml,yaml}), do not treat `uses: <action>@<mutable-tag>` patterns (e.g., `actions/checkoutv6`) as a new PR-specific security issue if the PR is only inlining/replicating the already-preexisting workflow pattern used across the fleet from the `rolling` branch. The intended fix is to pin all affected actions to commit SHAs fleet-wide on `rolling` first, then rely on Mergify to backport those pinned changes to `sagitta` and other branches; individual PRs that simply copy the existing `rolling` pattern should not be flagged for that alone.

Applied to files:

  • .github/workflows/ai-validation.yml
🔍 Remote MCP

Based on the search results, I've gathered relevant context for reviewing this PR. Let me compile the findings:

Additional Context for PR #2046 Review

GitHub Actions Job-Level Conditionals and Skip Cascade Behavior

If a job fails or is skipped, all jobs that need it are skipped unless the jobs use a conditional expression that causes the job to continue. This is the core behavior the PR relies on. However, there are documented complexities: when a preceding job is skipped, dependent jobs with needs dependencies require special handling—the skipped job can affect how the conditional on dependent jobs is evaluated, causing them to evaluate unexpectedly even when the condition should evaluate to true.

The PR's approach of adding a job-level if condition to the prepare job aligns with the intended behavior—using always() in conjunction with other conditions can force evaluation and ensure jobs run even if previous jobs were skipped, provided the conditions are met.

Mergify Bot and Backport Pull Requests

The Mergify backport action enables automatic creation of backport pull requests—when specified conditions are met, Mergify will create a new pull request to merge the changes into a specified base branch. If you want to create a backport impersonating the original pull request author, you can configure the backport action parameter to use a bot_account with the original pull request author account. This explains why the PR is checking for mergify[bot] as the author—backports may be authored by Mergify rather than the original contributor.

Shallow Clone and "No Merge Base" Error Root Cause

The PR's motivation references shallow fetch depth causing "no merge base" errors. Shallow clones skip the commit history, which makes commands such as git log or git merge-base unavailable. More specifically, when cloning a repository shallowly (like with the checkout GitHub Action), you may not have all commits required for git diff operations using the ... syntax, and this can result in "fatal: no merge base" errors. If there isn't enough history to identify the branch point, comparison can fail—this is most common if the git clone is shallow by default or if depth has been set to a small value.

Defense-in-Depth Strategy

The PR's approach of keeping the existing author check in the validate job's secrets-check step while adding an earlier job-level guard is consistent with defense-in-depth principles—having checks at multiple levels ensures robustness even if one layer fails or changes.

[::web_search::

🔇 Additional comments (1)
.github/workflows/ai-validation.yml (1)

49-61: LGTM!


📝 Walkthrough

Summary by CodeRabbit

  • Chores
    • Optimized CI/CD pipeline to skip validation processing for automated pull requests, improving pipeline efficiency.

Walkthrough

The PR adds an early job-level guard to .github/workflows/ai-validation.yml that skips the entire prepare job when a pull request is authored by mergify[bot], preventing unnecessary validation pipeline execution for automated bot-generated PRs.

Changes

CI skip guard for automated pulls

Layer / File(s) Summary
Prepare job skip condition for mergify[bot]
.github/workflows/ai-validation.yml
Job-level if: condition (lines 49–61) gates the prepare job to skip execution when github.event.pull_request.user.login is mergify[bot], preventing the pipeline from running on mergify's automated pull requests.

Possibly related PRs

  • vyos/vyos-documentation#2027: Extends the validate job's secrets-check step with author-based short-circuiting, while this PR adds an earlier job-level guard in prepare.
  • vyos/vyos-documentation#1960: Introduces the overall AI Validation workflow and its prepare/validate job structure that this PR builds upon.
  • vyos/vyos-documentation#1956: Modifies .github/workflows/ai-validation.yml to add skip-related gating behavior for workflow comment logic.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main change: adding a job-level guard to skip the prepare job for Mergify-authored PRs on the sagitta branch.
Description check ✅ Passed The description is directly related to the changeset, explaining the motivation (clearing failing prepare checks), implementation details (job-level if: condition), and test plan.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch yuriy/ai-validation-skip-prepare-on-bot-author-sagitta

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@andamasov
Copy link
Copy Markdown
Member Author

@copilot review

Copy link
Copy Markdown
Contributor

Copilot AI commented May 14, 2026

@copilot review

Reviewed. No further code changes are needed from my side: the hotfix commit e18192c is in place, and AI Validation on this branch completed successfully with the intended skip behavior.

@andamasov andamasov marked this pull request as ready for review May 14, 2026 07:08
@andamasov
Copy link
Copy Markdown
Member Author

Superseded by vyos-documentation#2045 (Mergify backport of vyos-documentation#2043 to sagitta) — landed byte-identical commit 003b219d ahead of this PR.

@andamasov andamasov closed this May 14, 2026
@andamasov andamasov deleted the yuriy/ai-validation-skip-prepare-on-bot-author-sagitta branch May 14, 2026 07:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants