fix(file-issue,file-pullreq,gh-body-check,gh-body-conventions,research,review-pipeline,review-pipeline-coderabbit): discharge a drafted body's evidence claims against the record - #143
Merged
Conversation
A statement that verification was performed could be written into a GitHub body with nothing checking it against what actually ran. Matching totals do not catch it: a fabricated entry and an omitted one cancel, so an enumeration can be wrong while its count is right. `gh-body-conventions` gains an Evidence claims section stating when a record counts as available, that the mapping from claims to records is checked entry by entry rather than by count, that an enumeration need not be complete unless it says it is, that a claim with no record is re-run or dropped rather than hedged, and that text edited after its claims were discharged is discharged again. The rule is threaded through every drafting surface that reaches a GitHub body. `file-pullreq` and `file-issue` gain a discharge step between drafting and the laundering pass, which renumbers their later steps; `research` widens the rule families it imports and discharges the plan before approval and each derived body before filing; `review-pipeline` and its CodeRabbit twin cover the appended delta section and the fix-loop description update. `gh-body-check` records that it does not discharge the rule, since the check's subagent is denied the drafting context the rule compares against. Two posting surfaces are left uncovered and tracked separately: the sub-issue closing comment and umbrella body edit after merge, and inline review replies, which the conventions do not currently reach.
…ompleteness scope Three gaps a reviewer found in the new rule. A property that can only be known by running something — a suite passing, a coverage figure, a timing — reads grammatically like the artifact properties the rule exempts, so an agent could classify it either way and skip the discharge. The rule now names the discriminator instead of leaving it to the examples. A completeness claim required "every record" without saying which records it ranges over, so the check that runs after the discharge would itself create a record the claim had already excluded. The obligation is now scoped to what the enumeration claims, and body-validation work is outside any such scope. A body that `research` derives from the plan gets shown for approval, but nothing said what happens when the user asks for a change there. It now revises, re-discharges, re-checks, and re-shows, as the plan itself already did.
…e proposition Uniqueness was written over agent returns alone, so two enumerated entries could both point at one captured command output and pass. The derivation behind the rule is a map from claims to records, so the constraint belongs on records. The static-property exemption covered a sentence rather than a proposition, so "I confirmed the file is present by running X, which exited 0" satisfied both the exemption and the evidence-claim definition. What is exempt is the proposition; reporting a check of it is a claim like any other.
…eteness on re-discharge A sentence asserting several runs was one claim under a rule that binds each claim to one record, so truthful collective prose could be rejected or validated against a single return. Such a sentence is now that many claims. A completeness claim can be falsified by work done after it was discharged, without its own text changing — the case that arises whenever a section is appended to an already-discharged body. It is now re-checked on every re-discharge, which puts the obligation on the class rather than on the one procedure where it was noticed.
…totalize the claim boundary The boundary split statements into static properties and properties knowable only by running something, which left out everything knowable either way: "the regression test exercises the fallback" can be read off the control flow or observed in a coverage run, and the sentence does not say which. The test is now what the statement asserts — an execution or an observation — with everything establishable by reading the code on the other side, whether or not something was also run to confirm it. The delta section appended before a PR edit is discharged as before, but a completeness claim already in the body is re-checked with it: work done since the body was last discharged can falsify one without its text changing, and discharging only the new section would not look there.
…turn Two clauses added separately interacted badly. One made an agent's whole return the record; another forbade two enumerated entries from mapping to the same record. A return reporting several results — a test run and a lint run in one dispatch — could then back only one truthful entry, and the author had to drop a real result or re-run work to manufacture a second return. A return reporting several results holds that many records, so entries map to results rather than to dispatches. What the uniqueness rule was always for is unchanged: one execution cannot be written up as two.
…he source "Reports an observation" covered reading the code as much as running it, so "Inspection confirmed that the regression test covers the fallback" satisfied both the evidence-claim test and the exemption for properties establishable by reading. The observation is now qualified to what an execution produced, and reading is named on the other side so the phrasing cannot move a code-readable property across the line.
Both steps told the executor to discharge statements that assert verification was performed. The conventions had since widened the class to cover properties knowable only by running something, however phrased, so a bare "Coverage is 95%" — which asserts nothing about having verified anything — fell outside the step's wording while falling inside the rule's. The steps now name the class and leave its definition where it is maintained.
There was a problem hiding this comment.
Pull request overview
Adds an explicit “discharge evidence claims” gate to ensure drafted GitHub bodies’ verification statements are checked against the drafting-session record (per new gh-body-conventions guidance), and threads that gate through the primary drafting and PR-update pipelines.
Changes:
- Introduces
gh-body-conventions“## Evidence claims” rules and clarifies that discharging must happen in main context (not ingh-body-check’s subagent). - Inserts a discharge step before laundering (
/gh-body-check) across PR/issue drafting (file-pullreq,file-issue) and plan posting (research), including re-discharge behavior in revise loops. - Updates review pipelines to discharge evidence claims for appended PR-body deltas and to re-check completeness claims when appending/editing bodies; bumps marketplace version.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| skills/review-pipeline/SKILL.md | Adds discharge requirements to gate-mode PR drafting and to PR-body delta append/edit flow. |
| skills/review-pipeline-coderabbit/SKILL.md | Mirrors discharge requirements for the CodeRabbit-based pipeline. |
| skills/research/SKILL.md | Discharges plan evidence claims before approval/posting and adds discharge+launder requirements for derived bodies. |
| skills/gh-body-conventions/SKILL.md | Defines what counts as an evidence claim, what records are admissible, and when re-discharge is required. |
| skills/gh-body-check/SKILL.md | Updates revise-loop wording and explicitly states evidence-claim discharge is out of scope for this check. |
| skills/file-pullreq/SKILL.md | Inserts a dedicated discharge step before laundering and threads re-discharge through revise loops; renumbers steps. |
| skills/file-issue/SKILL.md | Inserts a dedicated discharge step before laundering and threads re-discharge through revise loops; renumbers steps. |
| .claude-plugin/marketplace.json | Bumps plugin CalVer patch version. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A drafted GitHub body could state that verification was performed with nothing checking the statement against a record of what actually ran. Matching totals do not catch it: a fabricated entry and an omitted one cancel, so an enumeration can be wrong while its count is right.
gh-body-conventionsgains an## Evidence claimssection stating what makes a statement one, when a record counts as available, that claims map to records entry by entry rather than by count, that an enumeration need not be complete unless it says it is, that a claim with no record is re-run or dropped rather than hedged, and that text edited after its claims were discharged is discharged again.Settling a body's statements against that record is called discharging them. The rule is then threaded through the surfaces that draft GitHub text, and
gh-body-checkrecords that it does not implement it — the check's subagent is deliberately denied the drafting context this rule compares against, so it cannot.Closes #138
Changes
skills/gh-body-conventions/SKILL.md— the new## Evidence claimssection, after## Exclusions.skills/file-pullreq/SKILL.md,skills/file-issue/SKILL.md— a discharge step between drafting and the laundering pass — the existing step that runsgh-body-check— which renumbers the steps after it; both revise-and-re-show loops return through it.skills/research/SKILL.md—evidence-claimadded to the rule families imported from the conventions; the plan is discharged before it is presented for approval rather than after, and a body derived from the plan rather than posted verbatim is discharged and laundered on its own before filing.skills/review-pipeline/SKILL.md,skills/review-pipeline-coderabbit/SKILL.md— the delta section appended to an open PR's body is discharged, and a completeness claim already in that body is re-checked with it; the fix-loop description update re-discharges and re-checks before posting.skills/gh-body-check/SKILL.md— its revise loop returns through the discharge, and its scope statement names the rule it does not implement..claude-plugin/marketplace.json— version bump.Impact
file-pullreqandfile-issuerenumber their procedure steps 3–6 to 4–7 (5a/5bbecome6a/6binfile-pullreq). The only references to those numbers from outside the two files are two inresearch, pointing atfile-issue's File step; both were updated.The rule reaches a body wherever a skill applies the conventions to it, so it binds
file-issueandfile-pullreqdirectly, the appended delta section in both pipelines, and — through the widened import — the planresearchposts.Test plan
Sixteen fixtures, each exercising a distinct branch of the rule. Each was handed to a fresh-context agent together with the shipped text of the new
file-pullreqstep and the## Evidence claimsand## Exclusionssections, and nothing else. Each agent reported, for every statement in its fixture, the record it mapped that statement to or that it had none, and what it then did with the body; that report was compared against what the rule specifies. All sixteen reached the specified disposition. The fixtures are synthetic scaffolding rather than a surface this repository carries, so they are not committed — this section is their record.One fixture in full, so the shape is on the record. Its draft body:
Its stated record: one command was run and observed,
pytest -qreporting 88 passed and 0 failed; the migration apply-and-rollback was performed in an earlier session that has ended, leaving only a note reading "Applied and rolled back the migration against a production-schema copy; row counts matched", with no log, linked run, or committed artifact. The rule requires the migration sentence dropped rather than re-attributed to the earlier session, and the surviving claim restated to name its command and observed result. The agent reached that disposition, and its report named two rewrites it refused — "was previously verified", and attributing the work to the earlier session — as the softening the rule forbids. Neither phrase is in the draft above; both are rewrites the agent considered and rejected.The other fifteen have the same shape. All sixteen branches, and the disposition each fixture had to reach — the twelfth is the one shown in full above:
## Exclusions— drop, since no nameable restatement existsAlso run: the
mdformathook configured in.pre-commit-config.yaml, against the changed files, reportingPassedwith no reformatting;skills/gh-body-check/test-body-math-scan.sh, reportingAll checks passedbefore and after the change;skills/coderabbit-review/scripts/test-completion-state.shandskills/coderabbit-review/scripts/test-review-skip-reason.sh, each reportingAll checks passed; andrg -n 'step [0-9]' skills/, whose surviving hits were each read against the step they name.Notes
Two posting surfaces the rule reaches are left without a discharge point, and one more is adjacent.
skills/review-pipeline/SKILL.md's Phase 4b posts a closing comment on the sub-issue and edits the umbrella issue body;skills/file-issue/SKILL.md's umbrella sub-issue variant edits the umbrella's Phases table when a sub-issue is filed; and both pipelines reply to review comments inline, which are not issue or PR bodies and so fall outside what the conventions currently govern. Covering them means deciding which posting points carry a gate, which this change does not settle; they are tracked in #142.The rule guarantees that a claim was answerable to a record its author held, not that a later reader can re-derive it.
@coderabbitai ignore