test(action): dogfood CERTIFICATION_FAILED end-to-end - #327
Merged
Conversation
🗺️ StyleProof report📊 View the side-by-side visual report →Coverage, determinism, or report/diff consistency evidence is incomplete — repair the capture or reflow source; visual approval cannot clear this failure. |
action-dogfood exercised 6 of the 8 trust states; CERTIFICATION_FAILED and REPORT_PUBLICATION_FAILED were the gaps. CERTIFICATION_FAILED is the one that matters most — it's the state 4.6.2's content-geometry bug hid in, undetected precisely because nothing drove the Action into it end-to-end. A new certfail-base/head fixture pair carries identical maps but unproven determinism (a side that was neither self-checked nor replayed). The diff reports determinism.status='unproven' (verified locally), which the Action's provenance gate must escalate to CERTIFICATION_FAILED — the state the approval box cannot clear. The dogfood step asserts both the failure and the verdict; action.test.mjs pins the workflow contract (8 action invocations now). REPORT_PUBLICATION_FAILED remains unit-tested (the trust step's publish/delivery domains are pinned in action.test.mjs) but not end-to-end dogfooded — forcing a publish failure needs a broken remote, which a self-contained dogfood can't stage cleanly; tracked as a follow-up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ts:270) This console.warn — the tolerate-surface-failures capture-failure log — was the one console.warn in runner.ts missing the // eslint-disable-next-line no-console that its four siblings (623, 733, 947, 1031) all carry, so `eslint .` (--max-warnings 0) failed the build. Bringing it in line with the file's pattern. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
BenSheridanEdwards
force-pushed
the
fix/dogfood-remaining-trust-states
branch
from
July 21, 2026 16:38
42e478c to
24bea26
Compare
This was referenced Jul 21, 2026
BenSheridanEdwards
added a commit
that referenced
this pull request
Jul 22, 2026
Cut 4.7.0 from the six commits merged since 4.6.3. Two are feature work, so this is a minor bump; the version bump is the Release workflow's signal to tag, publish to npm, and cut the GitHub Release on merge. Added - Reports publish through the git-data API (#328). - styleproof-init prunes reports by API with a retention + size-budget sweep (#329). - CERTIFICATION_FAILED dogfooded end-to-end (#327). Security - Cleared two high-severity transitive advisories: brace-expansion 5.0.6→5.0.7 (#330) and fast-uri 3.1.3→3.1.4, GHSA-v2hh-gcrm-f6hx (#331). Verified locally: build, typecheck, lint, format:check, privacy:check all clean; full node --test suite 630/630 with a complete install. Co-authored-by: StyleProof Test <styleproof@example.test> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why does this feature exist?
The action-dogfood workflow drives the composite Action against a fixture per trust state and asserts the verdict — it's the end-to-end safety net for the trust contract. It covered six of the eight states. The two gaps were
CERTIFICATION_FAILEDandREPORT_PUBLICATION_FAILED, and that first gap is not academic: 4.6.2's content-geometry regression wrongly routed ordinary copy-editing PRs intoCERTIFICATION_FAILED(an unclearable state), and it shipped undetected precisely because nothing ever drove the Action into that state here. This closes it.What changed?
scripts/action-dogfood-fixtures.mjs: a newcertfail-base/certfail-headpair — identical maps, but astyleproof-coverage.jsondeclaringdeterminism: 'unproven'(a side that was neither self-checked nor replayed).assessDeterminismturns that intostatus: 'unproven'(verified locally:styleproof-diff --jsonreportsdeterminism.status: "unproven")..github/workflows/action-dogfood.yml: acertfailstep runs the Action against that pair and asserts it fails (outcome == 'failure') withtrust-state == 'CERTIFICATION_FAILED'— proving unproven provenance certifies nothing and the approval box cannot clear it, with maps identical so provenance is the only thing under test.test/action.test.mjs: the workflow-contract test now expects 8 Action invocations and pins the newcertfailassertions.REPORT_PUBLICATION_FAILEDis left unit-tested (the trust step's publish/delivery failure domains are already pinned inaction.test.mjs) but not end-to-end dogfooded: forcing a real publish failure needs a broken remote/permissions, which a self-contained dogfood can't stage cleanly. Called out in the commit as a follow-up rather than faked.Behavioural Proof (with video and screenshots)
Not applicable — CI/Action trust-contract coverage with no rendered surface. The proof is executable: locally,
node scripts/action-dogfood-fixtures.mjs && node bin/styleproof-diff.mjs action-dogfood/certfail-base action-dogfood/certfail-head --jsonreportsdeterminism.status: "unproven"; the new dogfood step asserts the Action escalates that to a failingCERTIFICATION_FAILED; and this PR's ownaction-dogfoodCI job exercises it.Verification Summary
npm test— 608/608 pass (updated the workflow-contract count 7 → 8 and added the certfail pins).determinism.status: "unproven"through the realstyleproof-diff.action-dogfoodjob is the live end-to-end proof (7 states now, was 6).🤖 Generated with Claude Code