Skip to content

fix(research,authoritative-text-rules,done-check): rule-set baseline - #188

Merged
ultimatile merged 2 commits into
mainfrom
fix/185-rule-set-baseline
Aug 3, 2026
Merged

fix(research,authoritative-text-rules,done-check): rule-set baseline#188
ultimatile merged 2 commits into
mainfrom
fix/185-rule-set-baseline

Conversation

@ultimatile

@ultimatile ultimatile commented Aug 3, 2026

Copy link
Copy Markdown
Owner

Summary

research formed hypotheses without reading the rules its output would later be audited against, so a plan could commit to wording that done-check rejects only once the work is finished. It now reads authoritative-text-rules as a third baseline, before hypothesis formation. Two sentences elsewhere described that rule set as reaching an audit only through a fresh-context subagent, and authoritative-text-rules and done-check now say what is true of it once a caller reads it in main context.

Refs #185

Changes

  • research establishes a rule-set baseline beside its test and memory recall baselines. It invokes authoritative-text-rules, reads that skill's Scope section and Items index in main context, and reads every item body the index lists. A hypothesis contradicting a loaded item is rejected at formation, on the terms the memory recall baseline already states.
  • The item-body read is skipped only where the issue and the layout skim positively establish that no surface the change could touch qualifies under that Scope section. A surface the skim left unenumerated establishes nothing, and having no stated reason to touch a qualifying surface is not an exclusion.
  • A skipped read is repairable rather than final. Whenever a surface the change could touch is found to qualify, the baseline re-runs and the run returns to hypothesis formation. Movement the other way — a qualifying surface dropping out — calls for nothing.
  • The step presenting hypotheses for approval admits a widened scope, not only a narrowed one.
  • authoritative-text-rules says that either a fresh-context subagent or main context may take any item, in place of directing every item to the subagent.
  • done-check's prohibition binds the audit's own reads rather than the state of main context, so it holds whether or not an earlier phase of the same session loaded a body. Its Step 4 scopes quote-based adjudication by what Step 2 forbids that audit to load into main context, and the two clauses asserting that main context cannot open those bodies are gone.
  • README.md no longer names done-check as the rule set's referrer.

Impact

  • research gains a dependency on authoritative-text-rules being resolvable as a skill wherever the run executes. A run whose invocation of it does not resolve halts.
  • research and done-check compose within one session: main context may hold the item bodies when done-check runs, and done-check's own prohibition is unaffected by that.
  • todo-check is unchanged and still applies quality-list alone.

Verification

Nothing here executes, and skills/gh-body-check/test-body-math-scan.sh — the repository's only test script — exercises that skill's math scanner, which this change does not touch.

Properties checkable by re-running these against a checkout of this branch:

  • rg -n '^## (Scope|Items)$' skills/authoritative-text-rules/SKILL.md returns both headings the new research text names. That file's index lists five items, and ls skills/authoritative-text-rules/items/ returns exactly those five files, so every link the baseline is told to follow has a target.
  • rg -n 'sends all of them to the subagent' --glob '*.md' . returns nothing: no surface still asserts the routing this change replaced.
  • rg -n 'referenced by' README.md returns one line, which belongs to the gh-body-conventions row.
  • jq -r '.metadata.version' .claude-plugin/marketplace.json prints 2026.8.4, one patch above 2026.8.3, the newest tag git tag --sort=-v:refname reports.
  • diff <(git show f42f4e5:.claude-plugin/marketplace.json | jq -S '.plugins[0].skills') <(jq -S '.plugins[0].skills' .claude-plugin/marketplace.json) prints nothing, so no skill was added or removed and the README table's membership is unaffected.
  • pre-commit run --all-files reports mdformat Passed; it is the only hook .pre-commit-config.yaml declares.

The changed units are instructions, so the remaining property — that an executing agent reaches one intended action — was checked by running them. Each row below is a single run of one fresh-context agent, handed the changed steps alone with the expected outcome withheld. The prompts are not committed and no two rows share a run, so a row is evidence that the instruction can produce the outcome, not that it always does.

Situation the agent was given What it did
Scope is a tree of skill files Read the index and all five item bodies
Scope is a C++ tree the skim enumerated whole, holding no file the Scope section qualifies Skipped the item-body read, reading only Scope and the index
Hypothesis approval widens scope to a new CLAUDE.md, after a skipped read Re-ran the baseline, then returned to hypothesis formation
A plan revised at the review gate without user involvement adds a file under .claude/rules/, after a skipped read Re-ran the baseline, then returned to hypothesis formation
Hypothesis formation itself finds a qualifying file the skim had enumerated, after a skipped read Re-ran the baseline, then returned to hypothesis formation
Approval narrows scope while the remaining surface still qualifies, after the bodies were read Took no baseline action
The skill invocation fails Halted, named what it looked for, and declined the skip arm for want of a Scope section to establish exclusion
A real run against this repository, tools enabled Invoked the skill and read all five item bodies

That last row rests on the runner handing back the invoked skill's own directory, which the index's relative links resolve against. That is behaviour of the harness rather than of anything in this repository, and this PR assumes it.

Five done-check rounds, three /code-review iterations and one Codex review of the branch also ran. The last of each returned no actionable finding, and the final round's authoritative-text-rules auditor returned no concern at all. All of that was local; this repository declares no CI workflow that would witness it, and no output is linked.

Notes

  • todo-check does not apply authoritative-text-rules, so a clean preflight still does not predict a clean audit. todo-check states that asymmetry itself.
  • done-check does not say what an audit may do when main context legitimately already holds the item body a subagent row would be judged against — a state this PR makes reachable. done-check: the missing-clause re-dispatch allowance has no stated relation to the coverage check's #186 covers the neighbouring question of how many re-dispatches Step 4 allows, and places this one out of scope.

`research` reads `authoritative-text-rules` before it forms hypotheses,
so the rules constrain the hypothesis set rather than only the finished
diff. The read is skipped only when the issue and the layout skim
establish that no surface the change could touch qualifies, and it
re-runs whenever a qualifying surface turns up after such a skip.

`authoritative-text-rules` no longer routes its items to a fresh-context
subagent. Its premise -- that every judgment is available from literal
text -- supports only that either receiver can take any item.

`done-check` scopes its prohibition to the audit's own reads, so it
composes with a main context an earlier phase already loaded, and drops
two rationale clauses that such a load makes false.

`README.md` drops the claim that `done-check` is the rule set's only
referrer.

Refs #185

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the workflow skill instructions so research reads authoritative-text-rules early (before hypothesis formation) and aligns done-check / authoritative-text-rules wording with the fact that items can be read in main context when appropriate, while preserving done-check’s fresh-context audit model.

Changes:

  • Added a research “Rule-set baseline” that reads authoritative-text-rules Scope + Items index (and item bodies when applicable) before hypothesis formation.
  • Updated done-check text to reflect that its prohibition applies to the audit’s own reads (not “what main context happens to already contain”), and tightened wording around quote-based adjudication.
  • Updated metadata/docs: bumped marketplace version and removed “referenced by done-check” from the README description of authoritative-text-rules.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
skills/research/SKILL.md Adds an authoritative-text rules baseline before hypothesis formation; adjusts hypothesis approval step to allow scope widening.
skills/done-check/SKILL.md Updates audit prohibitions and clarifies how clause-quoting is used for adjudication without opening item bodies in main context.
skills/authoritative-text-rules/SKILL.md Updates Items section wording to allow items to be read either in main context or a fresh-context subagent.
README.md Removes the claim that authoritative-text-rules is “referenced by done-check”.
.claude-plugin/marketplace.json Bumps plugin version from 2026.8.3 to 2026.8.4.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread skills/done-check/SKILL.md Outdated
Step 4 selected the rows a quoted clause adjudicates by what "step 2
forbids this audit to load". Step 2 forbids main-context loading and
in the same breath directs each subagent to read the bodies it needs,
so read at whole-audit granularity the clause selected nothing and the
quote requirement, with the re-dispatch that backs it, went vacuous.

Naming main context restores the scope without restoring the claim
this branch removed: the clause says what the audit may load, not what
main context happens to hold.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Suppressed comments (1)

skills/done-check/SKILL.md:38

  • The Step 2 prohibition is intended to bind what this audit reads, not the pre-existing state of main context (e.g., if research already loaded an authoritative-text item body earlier in the same session). Using “load … into main context” can be read as a state constraint rather than “do not read during this audit,” which reintroduces the ambiguity this PR is trying to remove. Reword to make it explicitly about reads performed as part of this audit.
   **This audit MUST NOT load a purely-mechanical `quality-list` item body into main context, and MUST NOT load any `authoritative-text-rules` item body there** — whether or not an earlier phase of the same session already did. Each subagent reads its own SSOT's index and the bodies it needs in its own fresh context, deriving its item set from that index; main only composes the prompts (diff + resolved paths) and dispatches. Main reads the contextual-lane `quality-list` bodies it audits in Step 3, and may read either SSOT's index — that is how Step 3 selects contextual items and how Step 4 predicts each auditor's row set.

@ultimatile
ultimatile merged commit 909096c into main Aug 3, 2026
1 check passed
@ultimatile
ultimatile deleted the fix/185-rule-set-baseline branch August 3, 2026 10:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants