Skip to content

fix(hooks): treat neutral check conclusion as non-failing in require-ci-green-before-stop#410

Merged
NiveditJain merged 5 commits into
mainfrom
fix/ci-policy-allow-neutral-conclusion
Jun 7, 2026
Merged

fix(hooks): treat neutral check conclusion as non-failing in require-ci-green-before-stop#410
NiveditJain merged 5 commits into
mainfrom
fix/ci-policy-allow-neutral-conclusion

Conversation

@NiveditJain

@NiveditJain NiveditJain commented Jun 7, 2026

Copy link
Copy Markdown
Member

Summary

  • require-ci-green-before-stop previously treated any GitHub check-run conclusion other than success / skipped / cancelled as failing. GitHub's neutral conclusion means "neither passing nor failing" — Socket Security: Pull Request Alerts (and similar apps) emit it when they can't process a PR (e.g. the head branch is from an outside contributor and the app lacks access).
  • The result was a false-positive Stop block: the policy demanded the agent "fix" a check that nobody can fix from the agent side. Encountered live while a user was just trying to gh pr checkout an outside-contributor PR with a neutral Socket check.
  • This PR adds neutral to the non-failing set in src/hooks/builtin-policies.ts, mirrors that into the policy reference in docs/built-in-policies.mdx, adds a unit test next to the existing skipped / cancelled cases, and adds a CHANGELOG Fixes entry under the current 0.0.11-beta.3 section.

Test plan

  • bun run test:run -- __tests__/hooks/builtin-policies.test.ts (419 passed, incl. the new treats neutral conclusions as non-failing case)
  • bun run lint (clean — 2 pre-existing warnings in unrelated files)
  • bunx tsc --noEmit (clean)
  • CI green on this PR

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Bug Fixes

    • Treat CI checks with a neutral conclusion as non-failing to avoid false-positive stop blocks.
    • Fix cross-repo post-merge push authentication to ensure reliable workflow re-pushes.
  • Tests

    • Added a test verifying neutral check conclusions are treated as non-failing.
  • Documentation

    • Updated policy docs and release notes; hardened docs pipeline with spellcheck adjustments and a vocabulary allow-list.

NiveditJain and others added 2 commits June 6, 2026 17:09
…ci-green-before-stop

Socket Security: Pull Request Alerts (and other apps) report `neutral`
when they can't process a PR — typically because the head branch is
from an outside contributor and the app lacks access. The CI-green
policy previously filtered failing checks as "not success / skipped /
cancelled", which lumped `neutral` in with real failures and produced
false-positive Stop blocks demanding the agent "fix" a check it cannot
affect.

Add `neutral` to the non-failing set, matching GitHub's documented
semantics (`neutral` = "neither passing nor failing"). Add a unit test
mirroring the existing `cancelled` / `skipped` cases.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Mirror the policy code change from the prior commit into the
built-in-policies reference, so the documented behavior for
require-ci-green-before-stop matches what the policy actually does.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@mintlify

mintlify Bot commented Jun 7, 2026

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
failproofai 🟢 Ready View Preview Jun 7, 2026, 12:11 AM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@coderabbitai

coderabbitai Bot commented Jun 7, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 0294b500-c09f-4ccd-85dc-e4ff9b5b37d2

📥 Commits

Reviewing files that changed from the base of the PR and between 6f10fe4 and d025c6c.

📒 Files selected for processing (2)
  • docs/.vale.ini
  • docs/built-in-policies.mdx
💤 Files with no reviewable changes (1)
  • docs/.vale.ini
🚧 Files skipped from review as they are similar to previous changes (1)
  • docs/built-in-policies.mdx

📝 Walkthrough

Walkthrough

The require-ci-green-before-stop policy now treats GitHub Check Run conclusions neutral as non-failing. Implementation, a test, documentation, and changelog entries were updated. Vale config and a Mintlify vocabulary accept-list were added to the docs pipeline and recorded in the changelog.

Changes

CI Policy Update

Layer / File(s) Summary
Treat neutral as non-failing in require-ci-green-before-stop
src/hooks/builtin-policies.ts, __tests__/hooks/builtin-policies.test.ts, docs/built-in-policies.mdx, CHANGELOG.md
The requireCiGreenBeforeStop policy excludes neutral check conclusions from the failing set (alongside cancelled/skipped). A test validates behavior when neutral and success checks coexist. Docs and changelog updated.

Docs Pipeline and Vocabulary

Layer / File(s) Summary
Vale config and Mintlify accept-list
docs/.vale.ini, docs/styles/config/vocabularies/Mintlify/accept.txt, CHANGELOG.md
Adds docs/.vale.ini enabling/disabling Vale spelling per path and a Mintlify accept-list of allowed terms to reduce noisy spelling failures in translated directories; changelog documents the change.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐰 I hopped through docs and CI gates,
A neutral check now calmly waits,
Vale learns words I gently bring,
Changelog sings the tiny thing,
Quiet builds and happier crates.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The pull request title clearly summarizes the main change: treating neutral check conclusions as non-failing in the require-ci-green-before-stop policy.
Description check ✅ Passed The pull request description is comprehensive, covering the problem statement, solution, affected files, and includes a detailed test plan with results.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

docs/built-in-policies.mdx

ESLint skipped: missing config or dependency (missing-dependency). The ESLint configuration references a package that is not available in the sandbox.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/built-in-policies.mdx`:
- Around line 814-815: In the sentence that reads "Treats `skipped`,
`cancelled`, and `neutral` conclusions as non-failing (the latter covers e.g.
Socket Security alerts on outside-contributor PRs, where the app intentionally
reports neutral rather than success/failure)", replace the hyphenated term
"outside-contributor PRs" with a non-hyphenated form such as "outside
contributor PRs" (or "PRs from outside contributors") to satisfy the Vale
spellcheck; update the surrounding text for grammar if needed to keep the
sentence clear and consistent with the existing phrase "Treats `skipped`,
`cancelled`, and `neutral` conclusions..." so CI passes.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 66ce3088-0868-4e31-b0ce-e13254d8ab0c

📥 Commits

Reviewing files that changed from the base of the PR and between 490c04f and abb08eb.

📒 Files selected for processing (4)
  • CHANGELOG.md
  • __tests__/hooks/builtin-policies.test.ts
  • docs/built-in-policies.mdx
  • src/hooks/builtin-policies.ts

Comment thread docs/built-in-policies.mdx Outdated
The Mintlify Validation (exosphere) - vale-spellcheck CI check was failing
on this PR with 8152 suggested changes across 15 files — almost all of it
noise from running the default English dictionary over the 14 translated
copies of the modified docs file plus dozens of legitimate brand/tooling
terms in the English source.

Add docs/.vale.ini (alongside docs.json, per Mintlify's CI contract) that
keeps Vale.Spelling enabled on canonical English *.{md,mdx} files but
disables it on the {ar,de,es,fr,he,hi,it,ja,ko,pt-br,ru,tr,vi,zh,i18n}/**
subdirs where running an English dictionary produces zero signal. Add a
project Vocab at docs/styles/config/vocabularies/Mintlify/accept.txt (the
Vale 3.0 path Mintlify requires, not the legacy styles/Vocab/) covering
the brand names, CLI tooling (npx, bunx, gcloud, systemctl, …), and
Claude Code event names (PreToolUse, SessionStart, …) the dictionary
doesn't know.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ellcheck startup error

First attempt failed with "There was an error running Vale: please check
your .vale.ini file and styles/ directory." The culprit was the
BasedOnStyles = Vale + Vale.Spelling = YES enable section: vale's built-in
Vale style requires a styles/Vale/ scaffold present on disk to enable via
BasedOnStyles, even though the rules themselves are bundled in the binary.

Switch to a disable-only config: let Mintlify's default spellcheck still
run (which is what was producing the original flags), but point it at the
project Vocab to whitelist legitimate terms, and disable Vale.Spelling on
the 14 translated subdirs + i18n/.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
CodeRabbit nit on PR #410: switch to the non-hyphenated form so the phrase
reads cleanly without compound-modifier punctuation. Vale-spellcheck no
longer flags either form (the project Vocab + disable globs make it pass),
but this is the form CR will keep re-flagging on every review until it's
gone.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@NiveditJain NiveditJain merged commit 72119f7 into main Jun 7, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant