fix(authoritative-text-rules,done-check,quality-list,todo-check,implement,finding-to-audit,review-pipeline,review-pipeline-coderabbit): audit text the agent executes as instructions - #146
Conversation
…ment,finding-to-audit,review-pipeline,review-pipeline-coderabbit): audit text the agent executes as instructions Add `authoritative-text-rules`, a rule-set SSOT for text an agent executes as instructions and that describes no separate artifact — skill bodies, rule-definition files, agent-instruction files. Five items: case-space-totality, single-reading, clause-composition, executor-fitness, consumer-closure. Each is written as a sweep procedure rather than a quality predicate, because in code a compiler, type checker, or test suite catches these shapes and in prose nothing does; the enumeration is the substitute oracle. Wire it into `done-check`: a second, conditionally dispatched fresh-context auditor alongside the `quality-list` one, firing when any diff path could be text an agent executes as instructions and when unsure. The firing rule over-approximates because the membership predicate is a property of file content while a dispatch decision has only paths; a missed dispatch is a silently skipped audit, a needless one costs a subagent returning not-applicable rows. Auditor 2's returned rows get a coverage check against the index-predicted slug set. Both auditor prompts now take the rule-set root and the target root as separate paths, so a marketplace install auditing another checkout resolves correctly. The verdict vocabulary has one definition, in the merge step, which both prompts point at. Callers whose unconditional statements the change qualifies: `todo-check` and `implement` now say the preflight covers one rule set of two; `quality-list` scopes its index and lane tags to its own items; `finding-to-audit` routes a promoted rule to a rule-set SSOT rather than to the runner, and drops the item format that no longer exists; both review pipelines name the actual host.
…th directions An item body names no runner, so a rule set built only from outbound references made every item-only diff a single unit and returned not-applicable — hiding the runner-versus-item conflict the item exists to catch.
There was a problem hiding this comment.
Pull request overview
This PR introduces a new rule-set SSOT (authoritative-text-rules) to audit “authoritative text” (instructional prose that is itself the executed artifact), and wires it into done-check as a conditionally-dispatched second fresh-context auditor alongside the existing quality-list audit. It also updates supporting workflow skills and manifests to reflect that done-check can now apply multiple rule sets.
Changes:
- Add
authoritative-text-rulesSSOT with an index + five new item files for auditing authoritative text. - Update
done-checkto dispatch a second auditor when the diff paths suggest authoritative text may be present, and to merge those rows. - Update workflow docs/skills (
todo-check, pipelines,finding-to-audit, README) and marketplace manifest to reference the new rule set.
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| skills/todo-check/SKILL.md | Clarifies todo-check preflights quality-list only (not the new authoritative-text audit). |
| skills/review-pipeline/SKILL.md | Updates /finding-to-audit guidance to target SSOT rule sets or domain audit skills. |
| skills/review-pipeline-coderabbit/SKILL.md | Same /finding-to-audit guidance update as review-pipeline. |
| skills/quality-list/SKILL.md | Clarifies “this rule set only” semantics and lane tagging when runners apply multiple rule sets. |
| skills/implement/SKILL.md | Notes todo-check doesn’t preview non-quality-list rule sets that done-check may apply later. |
| skills/finding-to-audit/SKILL.md | Updates host table and rule-drafting guidance for SSOT-style rule sets (incl. authoritative-text-rules). |
| skills/done-check/SKILL.md | Adds conditional authoritative-text auditor, row merge rules, and coverage checking behavior. |
| skills/authoritative-text-rules/SKILL.md | New SSOT defining scope/ownership boundary and indexing five authoritative-text audit items. |
| skills/authoritative-text-rules/items/case-space-totality.md | New item: totality of case→outcome mappings in authoritative text. |
| skills/authoritative-text-rules/items/single-reading.md | New item: ambiguity / multi-reading sweep for authoritative text changes. |
| skills/authoritative-text-rules/items/clause-composition.md | New item: contradiction/shadowing/drift across authoritative-text units. |
| skills/authoritative-text-rules/items/executor-fitness.md | New item: checks executor inputs suffice for demanded judgments. |
| skills/authoritative-text-rules/items/consumer-closure.md | New item: ensures produced values/obligations are consumed/terminated. |
| README.md | Adds authoritative-text-rules to the skill list and updates done-check description. |
| .claude-plugin/marketplace.json | Bumps version and registers the new authoritative-text-rules skill path. |
Comments suppressed due to low confidence (1)
skills/done-check/SKILL.md:163
- Step 4’s coverage check reads
authoritative-text-rules/SKILL.mdwithout anchoring it to<RULES_ROOT>. When<RULES_ROOT>and<TARGET_ROOT>differ (marketplace / symlink install), this path will be resolved against the target project and the coverage check can’t run.
**Coverage check on auditor 2's rows.** Only when it was dispatched. Read `authoritative-text-rules/SKILL.md`'s Items index for the predicted slug set — the index, never an item body, and never a slug list written into this file. A return that is exactly one row per predicted slug and no others passes. Any other return — a slug missing, duplicated, or outside the set — gets one re-dispatch of auditor 2 with its prompt unchanged; if the second return still does not match, carry the mismatch as a cross-cutting concern naming the offending slugs. It closes through step 5 like any other concern, so an incomplete rule set blocks completion without introducing an exit of its own.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…-file read to it Step 2 resolved the two roots but only its own prompts used them: Step 3's contextual reads and Step 4's coverage check still named rule files by bare relative path, which resolves against the audited project. Under a marketplace or symlinked install those are a different tree, so both reads fail. Resolution moves to Step 0 and every rule-file read is anchored. Auditor 1's prompt also now labels each returned row with the item's index slug, as auditor 2's already did, so the merge does not rely on row ordering.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (2)
skills/done-check/SKILL.md:16
- Step 0 introduces
<RULES_ROOT>/<TARGET_ROOT>separation but never defines how to discover<RULES_ROOT>. In a marketplace / symlink install where cwd is<TARGET_ROOT>, the rest of the procedure can’t be executed as written because none of the rule-set file paths can be resolved.
0. **Resolve the active rule sets.** Resolve two absolute paths first, and use them everywhere below: `<RULES_ROOT>`, the repo / package root holding the `skills/quality-list/` and `skills/authoritative-text-rules/` directories, and `<TARGET_ROOT>`, the project under audit (cwd). They coincide only when the rule sets are vendored inside the target; in a marketplace or symlinked install they differ, and every read of a rule file — main context's as much as a subagent's — resolves against `<RULES_ROOT>`.
`quality-list` always applies: its base items live in `<RULES_ROOT>/skills/quality-list/SKILL.md`, and language-specific addenda at `<RULES_ROOT>/skills/quality-list/lang-<language>.md` realize them concretely. `authoritative-text-rules` applies conditionally, by the firing rule in Step 2.
Detect language from the project's `CLAUDE.md` `Language:` declaration; otherwise auto-detect from diff file extensions (`.rs` → rust, `.cpp`/`.cc`/`.cxx`/`.h`/`.hpp` → cpp, `.py` → python, `.ts`/`.tsx` → typescript, `.go` → go, etc.). Multi-language projects detect every present language; each matching addendum applies. Missing addendum → base rules only for that language (not a concern). Step 0 only **detects** the language(s); it routes nothing. Each consumer that applies `quality-list` — the Step 2 mechanical subagent and the Step 3 contextual pass — loads every matching addendum file itself.
skills/done-check/SKILL.md:65
- Auditor 2 concerns are required to quote the specific item clause because main context can’t open
authoritative-text-rulesitem bodies; the same problem applies to auditor 1 for mechanicalquality-listitems (Step 2 prohibits main from loading those bodies). Without a required quote, main context can’t reliably triage or fix a ⚠ without violating the Step 2 prohibition.
Return one verdict per item, from the three defined in Step 4 of
<RULES_ROOT>/skills/done-check/SKILL.md; read that step's verdict
sentence, which states what backs each of the three.
…ment,review-pipeline,review-pipeline-coderabbit): close the findings #146 left unfixed (#152) * fix(done-check,todo-check,authoritative-text-rules,quality-list,implement,review-pipeline,review-pipeline-coderabbit): close the findings #146 left unfixed (#147) Retire `<RULES_ROOT>` / `<QUALITY_LIST_ROOT>` for a single `<SKILLS_DIR>`, resolved once in Step 0 of each runner and carried to every consumer, so a marketplace / symlinked install resolves rule files from one place. Extend the coverage check to both done-check auditors and make an unmatched return halt without a table; require auditor rows to quote the item clause they apply, since a purely-mechanical body stays closed to main context. Define delta mode as a report narrowing over a caller-supplied baseline, with the step 5 gate still binding the whole audit, and point the three callers at that gate instead of restating its dispositions. Mark the unbounded trigger enumerations in `consumer-closure` and `executor-fitness` as illustrative, add their tie-breaks against `single-reading` and `clause-composition`, and replace the stale `item 11` reference with `paired-artifact-drift`. * fix(done-check,review-pipeline,review-pipeline-coderabbit): state each halt and gate-scope rule once Step 6 claimed a halted run "reports at all", contradicting the halt sites that report the file looked for and the directory looked in. Narrow the claim to the audit table, which is what the halts actually suppress. The Output format section restated the halt-to-no-table rule instead of referring to step 6, and both pipelines restated done-check's gate scope instead of referring to its Delta mode section. Each condition now has one statement and the other sites point at it. * fix(done-check): retry an auditor row that omits its required clause A concern row whose body main context may not open is judgeable only through the clause the auditor quotes. Step 4 sent a row missing that clause straight to the step 5 gate, where none of the three closures fits: main cannot derive what to fix against, a deferral has nothing to track, and only a user waiver remains. A missing slug in the same return already gets one re-dispatch. Give the unquoted row the same allowance, with the replacing return standing in for the first, and keep the existing disposition when the clause is still absent. No new halt: the branch falls through to the gate it already had.
Closes #145.
What this adds
authoritative-text-rules, a rule-set SSOT for text an agent executes as instructions and that describes no artifact separate from itself — skill bodies, rule-definition files, and repository agent-instruction files. It holds five items, each written as a sweep procedure rather than a quality predicate:case-space-totality,single-reading,clause-composition,executor-fitness,consumer-closure.The sweep form is the load-bearing choice. A bottom-up pass over the 106 findings #145 identifies placed 14 under referential and parse indeterminacy and 9 under executor mismatch; the remaining 83 are shapes a compiler, a type checker, or a test suite catches in code and nothing catches in prose. For those 83, what is missing is an oracle rather than a rule concept, so the item states the enumeration the auditor performs and that enumeration stands in for the missing oracle. The other two items take the same form for a different reason: a fresh-context auditor is not primed for an ambiguity or a mismatched executor either, so the enumeration is what makes those items executable rather than aspirational.
done-checkgains a second, conditionally dispatched fresh-context auditor beside thequality-listone. It fires when any path in the diff could be text an agent executes as instructions, and when unsure. The rule approximates because it has to: membership is a property of a file's content, while the dispatch decision is made from the diff's path list before any file is opened, so the only choice is which way it errs — a missed dispatch is an audit that silently did not happen, a needless one costs one subagent returning not-applicable rows. Auditor 2's returned rows are checked against the slug set its SSOT's index predicts.Both auditor prompts now take the rule-set root and the target root as two paths rather than one. The single path silently assumed the rules were vendored inside the audited project; the two-path shape lets a marketplace install pass a different rule-set root, though finding out what that root is remains unaddressed — see the open findings. The three verdicts — pass, concern, and not-applicable — have one definition, in the merge step, which both prompts point at.
Plan-vs-actual delta
The plan in #145 said the SSOT would be one file with no
items/split. It is split, and four further changes go beyond it.SKILL.mdplusitems/<slug>.md.quality-list's no-hardcoded-slug-list rule requires the coverage check to derive its expected set by reading an index, and main context can read an index but not an item body. Reading a single file loads both, so the split is what makes the coverage check possible at all.todo-check,quality-list,implement,finding-to-audit, and both review pipelines are edited. The plan listed only the manifest and the README as collateral. Makingdone-check's rule set conditional stranded unconditional statements of it across those files.finding-to-audit's item-format template and host list were rewritten, not just its routing row. Removingdone-checkfrom the host table left the template's "Fordone-check:" heading and the step-5 host list without a referent.done-check, though not in the callers — see the open findings.Verification
The pre-commit hook (
mdformatwith the gfm and frontmatter plugins) reports a pass on both commits.done-checkran against this diff six times, dispatching both auditors each time — the change is itself authoritative text, so the new rule set audited the diff that defines it. Every one of those six iterations returned concerns; all were resolved, and triage classified none of them a false positive. Several were defects the fixes themselves introduced — rule text whose repair opens a new seam with the rule text around it, which is the failure #145 attributes to this repository's diffs. The Claude code-review gate ran three times, returning six findings, then five, then six; every round's findings were resolved except four of the third round's, which are recorded below. Codex reviewed the branch twice and returned no findings on the second pass.Known open findings
Four of these came out of the code-review gate's third round and the last from the
done-checkloop; all five are landing unresolved. #147 tracks them together with the deferrals below. The loops stopped converging: the gate's three rounds returned six, five, and six findings, most of each round's traceable to the previous round's fixes, and in thedone-checkloop two rows that had passed twice came back as concerns against text neither round had changed. Fixing was regenerating the class about as fast as it closed it, so it was stopped rather than continued.<RULES_ROOT>must be resolved before dispatch, and the text says it differs from the working directory in a marketplace install, but gives no way to find it.todo-checkhas the same gap for its own root, so this is not new.done-check's own step 5 but is explicitly not a row, and both review pipelines gate on rows. A caller can exit clean with one unresolved.single-readingandclause-compositionstate none againstexecutor-fitnessorconsumer-closure, so one ambiguous dispatch step can yield two concerns for a single defect.executor-fitness's trigger list is not marked illustrative, unlike its siblings'.Deferred
todo-check. The longest run in the local telemetry log, at 44code-reviewrounds, was the change that mirroreddone-check's lane split into it.done-check'squality-listrows still have no coverage check whiletodo-check's do. Closing it changes an existing gate's behaviour.done-check"in delta mode", whichdone-checkdefines nowhere.quality-list/items/ported-code-attribution.mdrefers to "item 11", a numbering scheme the index no longer uses.#145's success measure — this repository's
code-reviewiterations per run falling toward the sub-1.8 band, and the findings carryingcode-reviewas their owninjected_at_gateshrinking — cannot be measured here. It needs future runs.