Skip to content

fix(file-issue,file-pullreq,gh-body-check,gh-body-conventions,research,review-pipeline,review-pipeline-coderabbit): discharge a drafted body's evidence claims against the record - #143

Merged
ultimatile merged 8 commits into
mainfrom
feat/138-evidence-claim-discharge
Jul 27, 2026
Merged

fix(file-issue,file-pullreq,gh-body-check,gh-body-conventions,research,review-pipeline,review-pipeline-coderabbit): discharge a drafted body's evidence claims against the record#143
ultimatile merged 8 commits into
mainfrom
feat/138-evidence-claim-discharge

Conversation

@ultimatile

Copy link
Copy Markdown
Owner

Summary

A drafted GitHub body could state that verification was performed with nothing checking the statement against a record of what actually ran. Matching totals do not catch it: a fabricated entry and an omitted one cancel, so an enumeration can be wrong while its count is right.

gh-body-conventions gains an ## Evidence claims section stating what makes a statement one, when a record counts as available, that claims map to records entry by entry rather than by count, that an enumeration need not be complete unless it says it is, that a claim with no record is re-run or dropped rather than hedged, and that text edited after its claims were discharged is discharged again.

Settling a body's statements against that record is called discharging them. The rule is then threaded through the surfaces that draft GitHub text, and gh-body-check records that it does not implement it — the check's subagent is deliberately denied the drafting context this rule compares against, so it cannot.

Closes #138

Changes

  • skills/gh-body-conventions/SKILL.md — the new ## Evidence claims section, after ## Exclusions.
  • skills/file-pullreq/SKILL.md, skills/file-issue/SKILL.md — a discharge step between drafting and the laundering pass — the existing step that runs gh-body-check — which renumbers the steps after it; both revise-and-re-show loops return through it.
  • skills/research/SKILL.mdevidence-claim added to the rule families imported from the conventions; the plan is discharged before it is presented for approval rather than after, and a body derived from the plan rather than posted verbatim is discharged and laundered on its own before filing.
  • skills/review-pipeline/SKILL.md, skills/review-pipeline-coderabbit/SKILL.md — the delta section appended to an open PR's body is discharged, and a completeness claim already in that body is re-checked with it; the fix-loop description update re-discharges and re-checks before posting.
  • skills/gh-body-check/SKILL.md — its revise loop returns through the discharge, and its scope statement names the rule it does not implement.
  • .claude-plugin/marketplace.json — version bump.

Impact

file-pullreq and file-issue renumber their procedure steps 3–6 to 4–7 (5a / 5b become 6a / 6b in file-pullreq). The only references to those numbers from outside the two files are two in research, pointing at file-issue's File step; both were updated.

The rule reaches a body wherever a skill applies the conventions to it, so it binds file-issue and file-pullreq directly, the appended delta section in both pipelines, and — through the widened import — the plan research posts.

Test plan

Sixteen fixtures, each exercising a distinct branch of the rule. Each was handed to a fresh-context agent together with the shipped text of the new file-pullreq step and the ## Evidence claims and ## Exclusions sections, and nothing else. Each agent reported, for every statement in its fixture, the record it mapped that statement to or that it had none, and what it then did with the body; that report was compared against what the rule specifies. All sixteen reached the specified disposition. The fixtures are synthetic scaffolding rather than a surface this repository carries, so they are not committed — this section is their record.

One fixture in full, so the shape is on the record. Its draft body:

Summary

Adds a migration for the new index.

Test plan

pytest -q passes. The migration was also applied and rolled back against a copy of the production schema, with row counts identical before and after.

Its stated record: one command was run and observed, pytest -q reporting 88 passed and 0 failed; the migration apply-and-rollback was performed in an earlier session that has ended, leaving only a note reading "Applied and rolled back the migration against a production-schema copy; row counts matched", with no log, linked run, or committed artifact. The rule requires the migration sentence dropped rather than re-attributed to the earlier session, and the surviving claim restated to name its command and observed result. The agent reached that disposition, and its report named two rewrites it refused — "was previously verified", and attributing the work to the earlier session — as the softening the rule forbids. Neither phrase is in the draft above; both are rewrites the agent considered and rejected.

The other fifteen have the same shape. All sixteen branches, and the disposition each fixture had to reach — the twelfth is the one shown in full above:

  • a fabricated entry and an omitted record leaving the totals equal — flag the unbacked entry rather than pass on the count
  • an enumeration that omits a record and claims no completeness — pass unedited
  • three entries framed as three runs when two runs produced them, one return having reported two results — keep all three entries, correct the count
  • a record lost to a summarized stretch with nothing durable holding it — drop, and do not hedge
  • an artifact's existence and static properties, plus a report of having run a check on them — drop the report, keep the properties
  • a completeness claim falsified by work done after the body was discharged — complete the enumeration
  • a body edited after discharge to clear that check, the edit introducing a claim — discharge again and drop it
  • a command-backed claim naming neither command nor result, beside a claim for a command never run — restate the first, drop the second
  • a record that outlived the drafting context in a committed report and a linked run — keep the claim and cite them
  • one run described a second time outside the enumeration — not double-counting
  • a claim whose only backing cannot be named under ## Exclusions — drop, since no nameable restatement exists
  • work performed in an earlier session that ended, with only a handoff note surviving — drop
  • two entries each backed by a different result inside a single return — keep both, the count of runs being right already
  • a property knowable by reading or by running, phrased as a coverage run's output — restate it against the code
  • inspection reported as having confirmed a code-readable property — not an evidence claim
  • bare runtime properties asserting no verification at all — a coverage figure, a suite's pass state, a timing — each caught

Also run: the mdformat hook configured in .pre-commit-config.yaml, against the changed files, reporting Passed with no reformatting; skills/gh-body-check/test-body-math-scan.sh, reporting All checks passed before and after the change; skills/coderabbit-review/scripts/test-completion-state.sh and skills/coderabbit-review/scripts/test-review-skip-reason.sh, each reporting All checks passed; and rg -n 'step [0-9]' skills/, whose surviving hits were each read against the step they name.

Notes

Two posting surfaces the rule reaches are left without a discharge point, and one more is adjacent. skills/review-pipeline/SKILL.md's Phase 4b posts a closing comment on the sub-issue and edits the umbrella issue body; skills/file-issue/SKILL.md's umbrella sub-issue variant edits the umbrella's Phases table when a sub-issue is filed; and both pipelines reply to review comments inline, which are not issue or PR bodies and so fall outside what the conventions currently govern. Covering them means deciding which posting points carry a gate, which this change does not settle; they are tracked in #142.

The rule guarantees that a claim was answerable to a record its author held, not that a later reader can re-derive it.

@coderabbitai ignore

A statement that verification was performed could be written into a
GitHub body with nothing checking it against what actually ran. Matching
totals do not catch it: a fabricated entry and an omitted one cancel, so
an enumeration can be wrong while its count is right.

`gh-body-conventions` gains an Evidence claims section stating when a
record counts as available, that the mapping from claims to records is
checked entry by entry rather than by count, that an enumeration need
not be complete unless it says it is, that a claim with no record is
re-run or dropped rather than hedged, and that text edited after its
claims were discharged is discharged again.

The rule is threaded through every drafting surface that reaches a
GitHub body. `file-pullreq` and `file-issue` gain a discharge step
between drafting and the laundering pass, which renumbers their later
steps; `research` widens the rule families it imports and discharges the
plan before approval and each derived body before filing;
`review-pipeline` and its CodeRabbit twin cover the appended delta
section and the fix-loop description update. `gh-body-check` records
that it does not discharge the rule, since the check's subagent is
denied the drafting context the rule compares against.

Two posting surfaces are left uncovered and tracked separately: the
sub-issue closing comment and umbrella body edit after merge, and
inline review replies, which the conventions do not currently reach.
…ompleteness scope

Three gaps a reviewer found in the new rule.

A property that can only be known by running something — a suite
passing, a coverage figure, a timing — reads grammatically like the
artifact properties the rule exempts, so an agent could classify it
either way and skip the discharge. The rule now names the discriminator
instead of leaving it to the examples.

A completeness claim required "every record" without saying which
records it ranges over, so the check that runs after the discharge would
itself create a record the claim had already excluded. The obligation is
now scoped to what the enumeration claims, and body-validation work is
outside any such scope.

A body that `research` derives from the plan gets shown for approval,
but nothing said what happens when the user asks for a change there.
It now revises, re-discharges, re-checks, and re-shows, as the plan
itself already did.
…e proposition

Uniqueness was written over agent returns alone, so two enumerated
entries could both point at one captured command output and pass. The
derivation behind the rule is a map from claims to records, so the
constraint belongs on records.

The static-property exemption covered a sentence rather than a
proposition, so "I confirmed the file is present by running X, which
exited 0" satisfied both the exemption and the evidence-claim
definition. What is exempt is the proposition; reporting a check of it
is a claim like any other.
…eteness on re-discharge

A sentence asserting several runs was one claim under a rule that binds
each claim to one record, so truthful collective prose could be rejected
or validated against a single return. Such a sentence is now that many
claims.

A completeness claim can be falsified by work done after it was
discharged, without its own text changing — the case that arises
whenever a section is appended to an already-discharged body. It is now
re-checked on every re-discharge, which puts the obligation on the class
rather than on the one procedure where it was noticed.
…totalize the claim boundary

The boundary split statements into static properties and properties
knowable only by running something, which left out everything knowable
either way: "the regression test exercises the fallback" can be read off
the control flow or observed in a coverage run, and the sentence does
not say which. The test is now what the statement asserts — an execution
or an observation — with everything establishable by reading the code on
the other side, whether or not something was also run to confirm it.

The delta section appended before a PR edit is discharged as before, but
a completeness claim already in the body is re-checked with it: work
done since the body was last discharged can falsify one without its text
changing, and discharging only the new section would not look there.
…turn

Two clauses added separately interacted badly. One made an agent's whole
return the record; another forbade two enumerated entries from mapping
to the same record. A return reporting several results — a test run and
a lint run in one dispatch — could then back only one truthful entry,
and the author had to drop a real result or re-run work to manufacture a
second return.

A return reporting several results holds that many records, so entries
map to results rather than to dispatches. What the uniqueness rule was
always for is unchanged: one execution cannot be written up as two.
…he source

"Reports an observation" covered reading the code as much as running it,
so "Inspection confirmed that the regression test covers the fallback"
satisfied both the evidence-claim test and the exemption for properties
establishable by reading. The observation is now qualified to what an
execution produced, and reading is named on the other side so the
phrasing cannot move a code-readable property across the line.
Both steps told the executor to discharge statements that assert
verification was performed. The conventions had since widened the class
to cover properties knowable only by running something, however phrased,
so a bare "Coverage is 95%" — which asserts nothing about having
verified anything — fell outside the step's wording while falling inside
the rule's. The steps now name the class and leave its definition where
it is maintained.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an explicit “discharge evidence claims” gate to ensure drafted GitHub bodies’ verification statements are checked against the drafting-session record (per new gh-body-conventions guidance), and threads that gate through the primary drafting and PR-update pipelines.

Changes:

  • Introduces gh-body-conventions “## Evidence claims” rules and clarifies that discharging must happen in main context (not in gh-body-check’s subagent).
  • Inserts a discharge step before laundering (/gh-body-check) across PR/issue drafting (file-pullreq, file-issue) and plan posting (research), including re-discharge behavior in revise loops.
  • Updates review pipelines to discharge evidence claims for appended PR-body deltas and to re-check completeness claims when appending/editing bodies; bumps marketplace version.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.

Show a summary per file
File Description
skills/review-pipeline/SKILL.md Adds discharge requirements to gate-mode PR drafting and to PR-body delta append/edit flow.
skills/review-pipeline-coderabbit/SKILL.md Mirrors discharge requirements for the CodeRabbit-based pipeline.
skills/research/SKILL.md Discharges plan evidence claims before approval/posting and adds discharge+launder requirements for derived bodies.
skills/gh-body-conventions/SKILL.md Defines what counts as an evidence claim, what records are admissible, and when re-discharge is required.
skills/gh-body-check/SKILL.md Updates revise-loop wording and explicitly states evidence-claim discharge is out of scope for this check.
skills/file-pullreq/SKILL.md Inserts a dedicated discharge step before laundering and threads re-discharge through revise loops; renumbers steps.
skills/file-issue/SKILL.md Inserts a dedicated discharge step before laundering and threads re-discharge through revise loops; renumbers steps.
.claude-plugin/marketplace.json Bumps plugin CalVer patch version.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@ultimatile
ultimatile merged commit 956cc39 into main Jul 27, 2026
2 checks passed
@ultimatile
ultimatile deleted the feat/138-evidence-claim-discharge branch July 27, 2026 20:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

file-pullreq: a PR body's evidence claims go unverified

2 participants