diff --git a/.constitution/prose-baseline.json b/.constitution/prose-baseline.json new file mode 100644 index 0000000..f22cbdf --- /dev/null +++ b/.constitution/prose-baseline.json @@ -0,0 +1,40 @@ +{ + "createdAt": "2026-07-06T22:30:28.735Z", + "keys": [ + "PROSE-SENTENCE-LEN::CONSTITUTION.md:54", + "PROSE-SENTENCE-LEN::CONSTITUTION.md:114", + "PROSE-SENTENCE-LEN::CONSTITUTION.md:135", + "PROSE-SENTENCE-LEN::cli/AGENTS.md:24", + "PROSE-SENTENCE-LEN::cli/AGENTS.md:37", + "PROSE-SENTENCE-LEN::cli/AGENTS.md:53", + "PROSE-SENTENCE-LEN::cli/AGENTS.md:67", + "PROSE-SENTENCE-LEN::cli/AGENTS.md:86", + "PROSE-SENTENCE-LEN::cli/AGENTS.md:98", + "PROSE-SENTENCE-LEN::cli/AGENTS.md:106", + "PROSE-SENTENCE-LEN::decisions/0001-package-managed-distribution.md", + "PROSE-SENTENCE-LEN::decisions/0002-version-axis-separation.md", + "PROSE-STACKED-QUALIFIER::decisions/0002-version-axis-separation.md", + "PROSE-SENTENCE-LEN::decisions/0003-tighten-version-axis-clause.md", + "PROSE-STACKED-QUALIFIER::decisions/0003-tighten-version-axis-clause.md", + "LEDGER-LENGTH::CONSTITUTION.md:204", + "LEDGER-LENGTH::CONSTITUTION.md:233", + "LEDGER-LENGTH::CONSTITUTION.md:254", + "LEDGER-LENGTH::CONSTITUTION.md:272", + "LEDGER-LENGTH::CONSTITUTION.md:301", + "LEDGER-LENGTH::CONSTITUTION.md:341", + "LEDGER-LENGTH::CONSTITUTION.md:378", + "LEDGER-LENGTH::CONSTITUTION.md:430", + "LEDGER-LENGTH::CONSTITUTION.md:458", + "LEDGER-LENGTH::CONSTITUTION.md:489", + "LEDGER-LENGTH::CONSTITUTION.md:511", + "LEDGER-LENGTH::CONSTITUTION.md:532", + "LEDGER-LENGTH::CONSTITUTION.md:561", + "LEDGER-LENGTH::CONSTITUTION.md:584", + "LEDGER-LENGTH::CONSTITUTION.md:605", + "LEDGER-LENGTH::CONSTITUTION.md:632", + "LEDGER-LENGTH::CONSTITUTION.md:675", + "LEDGER-LENGTH::CONSTITUTION.md:695", + "LEDGER-LENGTH::CONSTITUTION.md:708", + "LEDGER-LENGTH::CONSTITUTION.md:727" + ] +} diff --git a/.gitignore b/.gitignore index a36df67..a418c5a 100644 --- a/.gitignore +++ b/.gitignore @@ -7,4 +7,11 @@ build/ cli/test-run/ # Framework namespace -.constitution/ +# Excluding contents (not the directory itself, `.constitution/*`) so the +# negation below can still reach inside it — git can't un-ignore a file whose +# parent directory is itself excluded. +.constitution/* +# ...except the EXP-0001 baseline snapshot: this must be committed so the +# WARN-ONLY window's false-positive-rate evidence is consistent across +# commits and CI, not re-seeded from scratch on every fresh checkout. +!.constitution/prose-baseline.json diff --git a/AGENTS.md b/AGENTS.md index b7e4eef..eaad1fa 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -11,13 +11,17 @@ - `skills/`, `process/`, `templates/`, `decisions/`, `cli/` at the repo root are the **source of truth** for this repo's own tooling — edit here, then release (version bump + tag). +- `experiments/` — this repo's own pre-registered candidate rules under measurement + (`process/experiment-lifecycle.md` governs the shape and states; no separate `AGENTS.md` + yet — an experiment file follows `templates/experiment.md` directly, not a folder-local + statute). ## Source vs. installed artifacts This repo dogfoods its own CLI: running `constitution install` here (as anywhere) writes compiled, agent-specific copies into `.claude/skills/`, `.agents/skills/`, and `.cursor/rules/`. -- **Edit**: `skills/`, `process/`, `templates/`, `decisions/`, `cli/` — these are git-tracked and versioned. +- **Edit**: `skills/`, `process/`, `templates/`, `decisions/`, `experiments/`, `cli/` — these are git-tracked and versioned. - **Never edit**: `.claude/`, `.agents/`, `.cursor/` — gitignored, generated, and overwritten on every install/upgrade. Treat them like a `dist/` folder. If content there looks stale or wrong, fix the source in `skills/` and re-run the CLI — don't hand-patch the copy. diff --git a/CONSTITUTION.md b/CONSTITUTION.md index 6cec562..956a35f 100644 --- a/CONSTITUTION.md +++ b/CONSTITUTION.md @@ -1,7 +1,7 @@ # The constitution framework — Constitution ``` -framework: constitution@0.17.5 (self-hosted) +framework: constitution@0.17.6 (self-hosted) ratifier: Chinmay ``` @@ -174,6 +174,18 @@ on the same Article is the signal that the Article itself needs amending. Superseded clauses are never deleted — they are kept here with a forward link and the ADR that justified the change. +### [0.17.6] — 2026-07-06 — Governance prose clarity: WARN-ONLY checks (EXP-0001) +- **What changed:** `constitution audit` gained three WARN-ONLY findings + (`PROSE-SENTENCE-LEN`, `PROSE-STACKED-QUALIFIER`, `LEDGER-LENGTH`) checking + Article/Statute/ADR prose and Ledger entry length. Never blocks; a pre-registered + F-III experiment (`experiments/EXP-0001-governance-prose-clarity.md`) governs + promotion. +- **Why:** F-II's own text got denser with every amendment (ADR-0002, ADR-0003); this + framework's law should catch exactly this kind of drift on itself, not just on the + products it governs. +- **ADR:** none — below the firewall throughout (new engine checks, a new experiment + file, template guidance; no Article text or status touched). + ### [0.17.5] — 2026-07-05 — Re-audit Conformance - **Audit:** Ran `audit-conformance` against all seven Articles, same session as ADR-0003. - **Findings:** diff --git a/README.md b/README.md index 95f1936..c21a09a 100644 --- a/README.md +++ b/README.md @@ -85,8 +85,10 @@ anything. - `CONSTITUTION.md` — the framework's own L0–L1 + amendments ledger - `process/` — the spec: layers, amendment + experiment lifecycles, conflict resolution, the compiler -- `templates/` — copy-me templates (Article, Statute, ADR, experiment, compiled prompt) +- `templates/` — copy-me templates (Article, Statute, ADR, ledger entry, experiment, constitution, governance map, compiled prompt) - `decisions/` — the framework's own L3 case law +- `experiments/` — this repo's own pre-registered, WARN-ONLY candidate rules under measurement + (see `process/experiment-lifecycle.md`); dogfooding the lifecycle on the framework itself - `skills/` — the LLM-judgment skills (`define-preamble`, `harvest-articles`, `compile-prompt`, …) - `cli/` — the engine + installer (`constitution-cli`; see `cli/README.md`) - `docs/` — architecture, quickstart, firewall, tone, ops diff --git a/TODOS.md b/TODOS.md new file mode 100644 index 0000000..a76869d --- /dev/null +++ b/TODOS.md @@ -0,0 +1,55 @@ +# TODOS + +## Governance Engine + +### Delta-based ratification gate (Approach C) + +**What:** `propose-amendment` computes word-count/complexity of a proposed Article +rewrite against the text it replaces, and refuses to mark a "minor/wording" amendment +ratification-ready if density increased on both axes, without an explicit ratifier +override. + +**Why:** the WARN-ONLY prose checks (PROSE-SENTENCE-LEN, PROSE-STACKED-QUALIFIER, +LEDGER-LENGTH) catch dense text once it exists. This would catch the *act* of making an +amendment denser, at the moment it happens — the actual observed failure pattern: every +one of F-II's 3 revisions added density, none removed any. + +**Context:** proposed during the EXP-0001 (governance-prose-clarity) design review as a +fast-follow, deliberately deferred until the WARN-ONLY window produces real evidence +about what "denser" should mean numerically. No off-the-shelf tool exists for +text-delta complexity scoring; readability-proxy metrics (Flesch-Kincaid-style) are +blunt for technical/legal prose. Reuses `cli/src/engine/prose.ts`'s word-count +primitives as the delta-comparison input. + +**Effort:** L +**Priority:** P2 +**Depends on:** EXP-0001 shipping first (this PR) — do not start before that lands and +produces the false-positive-rate evidence this gate's thresholds should be based on. + +### Fix parseBoldBullets truncating multi-paragraph bullets at blank lines + +**What:** `parseBoldBullets` (`cli/src/engine/parse.ts`) stops capturing a bold-bullet's +continuation text at the first blank line, silently dropping any further paragraphs. + +**Why:** discovered while calibrating `PROSE-STACKED-QUALIFIER` against Article F-II — +its actual `Principle` text has a second paragraph ("These three are never compared as +one axis, with one exception...") separated from the preceding numbered list by a blank +line. `Article.principle` silently ends before that paragraph, so the "with one +exception" clause is invisible to every check that reads `.principle`, not just the new +prose checks — `ART-*` checks and anything else consuming that field see a truncated +version of the ratified text. + +**Context:** not fixed as part of the governance-prose-brevity work — pre-existing +parser behavior, out of scope for that PR. If a future check needs full multi-paragraph +bullet text, `parseBoldBullets` needs to keep consuming past single blank lines, +stopping only at the next bold-bullet marker or a real section boundary. Touches a +shared, load-bearing parser function every existing check depends on — needs its own +careful test coverage (multi-paragraph bullets, where a blank line legitimately means +"the bullet ended" vs. "just a paragraph break") before changing behavior other checks +silently rely on today. + +**Effort:** M +**Priority:** P2 +**Depends on:** None + +## Completed diff --git a/cli/package-lock.json b/cli/package-lock.json index 358c0f5..2197a92 100644 --- a/cli/package-lock.json +++ b/cli/package-lock.json @@ -1,12 +1,12 @@ { "name": "constitution-cli", - "version": "0.17.5", + "version": "0.17.6", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "constitution-cli", - "version": "0.17.5", + "version": "0.17.6", "license": "MIT", "dependencies": { "prompts": "^2.4.2" diff --git a/cli/package.json b/cli/package.json index de42bf2..5d1b414 100644 --- a/cli/package.json +++ b/cli/package.json @@ -1,6 +1,6 @@ { "name": "constitution-cli", - "version": "0.17.5", + "version": "0.17.6", "description": "The constitution governance engine: scaffold, audit, firewall-gate, compile, render, and track AI-native product development", "license": "MIT", "author": "Chinmay", diff --git a/cli/src/engine/audit.ts b/cli/src/engine/audit.ts index 4c8de97..ce3b499 100644 --- a/cli/src/engine/audit.ts +++ b/cli/src/engine/audit.ts @@ -8,6 +8,8 @@ import * as fs from 'fs'; import * as path from 'path'; import { Instance } from './model'; import { diffLock, readLock } from './lock'; +import { opsDir } from './events'; +import { countWords, hasStackedQualifier, sentenceLengths } from './prose'; export interface Finding { code: string; @@ -15,6 +17,11 @@ export interface Finding { firewall: 'above' | 'below'; where: string; // file[:line] or unit id message: string; + // Set only on PROSE-*/LEDGER-LENGTH findings (EXP-0001, WARN-ONLY candidate + // rule): true if this finding's (code, where) pair was already present in + // the baseline snapshot taken when the experiment started, false if it's + // new since then. Undefined for every other check. + baseline?: boolean; } const ARTICLE_STATUS = ['PROPOSED', 'RATIFIED', 'SUPERSEDED']; @@ -23,6 +30,21 @@ const ENFORCEMENT = ['UNGUARDED', 'AUDITED', 'GATED', 'STRUCTURAL']; const ADR_STATUS = ['proposed', 'accepted', 'superseded']; const PLACEHOLDER_RE = /<[^>]+>|your name|todo|tbd|xxx/i; +// EXP-0001 draft thresholds — tunable during the WARN-ONLY window, not final. +// experiments/EXP-0001-governance-prose-clarity.md's Hypothesis section states +// these same two numbers in prose; if you retune one here, update that file's +// wording too (or the experiment's own record of what it tested goes stale). +const SENTENCE_WORD_CEILING = 30; +const LEDGER_WORD_CEILING = 150; +const PROSE_BASELINE_FILE = 'prose-baseline.json'; + +// NOT side-effect-free: on its first call in a given instance, this writes +// .constitution/prose-baseline.json (see writeProseBaseline below) to seed +// the EXP-0001 false-positive baseline. Every caller — the pre-commit hook, +// `constitution audit --json` in CI, and every test that calls audit() +// directly — triggers this on a fresh instance. A future dry-run mode or a +// read-only caller needs to know this before assuming audit() never touches +// disk. export function audit(instance: Instance): Finding[] { const f: Finding[] = []; const doc = instance.constitution; @@ -154,6 +176,89 @@ export function audit(instance: Instance): Finding[] { } } + // -- governance prose clarity (EXP-0001, WARN-ONLY candidate rule) ----------- + // Never blocks anything (severity is always 'warn') — this is evidence + // gathering for a pre-registered experiment (F-III), not yet a ratified + // rule. See experiments/0001-governance-prose-clarity.md. + const proseFindings: Finding[] = []; + const checkProse = (where: string, label: string, text: string) => { + if (!text) return; + const longSentences = sentenceLengths(text).filter((len) => len > SENTENCE_WORD_CEILING); + if (longSentences.length > 0) { + proseFindings.push({ + code: 'PROSE-SENTENCE-LEN', + severity: 'warn', + firewall: 'below', + where, + message: `${label}: ${longSentences.length} sentence(s) over ${SENTENCE_WORD_CEILING} words (worst: ${Math.max(...longSentences)}) — candidate rule EXP-0001, not yet ratified`, + }); + } + if (hasStackedQualifier(text)) { + proseFindings.push({ + code: 'PROSE-STACKED-QUALIFIER', + severity: 'warn', + firewall: 'below', + where, + message: `${label}: stacks 2+ distinct qualifier patterns (em-dash aside / except-unless-scoped-to / nested parenthetical) — candidate rule EXP-0001, not yet ratified`, + }); + } + }; + for (const a of doc.articles) { + const where = `${rel}:${a.line}`; + checkProse(where, `Article ${a.id} Principle`, a.principle); + checkProse(where, `Article ${a.id} Fitness`, a.fitness); + checkProse(where, `Article ${a.id} Why`, a.why); + } + for (const s of instance.statutes) { + const where = `${s.home}:${s.line}`; + checkProse(where, `statute "${truncate(s.rule)}"`, s.rule); + checkProse(where, `statute "${truncate(s.rule)}" Why`, s.why); + } + for (const adr of instance.adrs) { + checkProse(adr.file, `ADR ${adr.id || adr.file}`, adr.body); + } + for (const entry of doc.ledger) { + const where = `${rel}:${entry.line}`; + const words = countWords(entry.body); + if (words > LEDGER_WORD_CEILING) { + proseFindings.push({ + code: 'LEDGER-LENGTH', + severity: 'warn', + firewall: 'below', + where, + message: `ledger entry [${entry.version}] runs ${words} words (cap ${LEDGER_WORD_CEILING}) — narrative belongs in BUILDLOG.md, not the ledger (candidate rule EXP-0001)`, + }); + } + } + // Baseline-snapshot: isolate pre-existing (already-known) findings from new + // ones, so the WARN-ONLY window's false-positive-rate metric isn't + // contaminated by the same known-dense text re-firing every commit — these + // checks run over whole documents, not a diff. Self-initializes on first + // run: if no baseline exists yet, today's findings ARE the baseline + // (nothing is "new" on day one). A missing/corrupt baseline file degrades + // to "nothing known" rather than crashing or silently re-seeding over data + // that might still be recoverable. + // Keyed by (code, where) — location, not field. Two dense fields on the same + // Article/Statute line (e.g. both Principle and Fitness) share one key, so a + // fixed Principle can still read as "known" via a still-bad Fitness at the + // same line. Acceptable for a WARN-ONLY evidence-gathering signal; tighten + // to a per-field key only if the WARN-ONLY window's data shows this + // coarseness is actually masking real false-positive-rate signal. + const keyOf = (fnd: Finding) => `${fnd.code}::${fnd.where}`; + const baseline = readProseBaseline(instance.root); + if (baseline === null) { + // Dedupe before writing — multiple fields (Principle + Fitness) on the + // same Article/Statute line produce the same (code, where) key, and an + // undeduped array would grow duplicate entries every time this branch + // ran (adversarial review finding: the committed baseline had 39 keys, + // only 35 unique). A Set is the correct on-disk shape for a key set. + writeProseBaseline(instance.root, [...new Set(proseFindings.map(keyOf))]); + for (const pf of proseFindings) pf.baseline = true; + } else { + for (const pf of proseFindings) pf.baseline = baseline.has(keyOf(pf)); + } + f.push(...proseFindings); + // -- the firewall lock (F-IV) ------------------------------------------------- const lock = readLock(instance.root); if (!lock) { @@ -175,6 +280,30 @@ function truncate(s: string, n = 60): string { return s.length > n ? s.slice(0, n - 1) + '…' : s; } +// null = no baseline file exists yet (caller should seed one from today's +// findings). A Set (possibly empty) = a baseline exists; empty specifically +// covers a corrupt/malformed file — degrade to "nothing known" rather than +// crash or silently overwrite whatever's there. +function readProseBaseline(root: string): Set | null { + const p = path.join(opsDir(root), PROSE_BASELINE_FILE); + if (!fs.existsSync(p)) return null; + try { + const parsed = JSON.parse(fs.readFileSync(p, 'utf8')); + return new Set(Array.isArray(parsed?.keys) ? parsed.keys : []); + } catch { + return new Set(); + } +} + +function writeProseBaseline(root: string, keys: string[]): void { + const dir = opsDir(root); + fs.mkdirSync(dir, { recursive: true }); + fs.writeFileSync( + path.join(dir, PROSE_BASELINE_FILE), + JSON.stringify({ createdAt: new Date().toISOString(), keys }, null, 2) + '\n' + ); +} + export function formatFindings(findings: Finding[]): string { if (findings.length === 0) return 'audit clean — 0 findings.'; const lines: string[] = []; @@ -182,7 +311,11 @@ export function formatFindings(findings: Finding[]): string { const warns = findings.filter((x) => x.severity === 'warn'); for (const x of findings) { const fw = x.firewall === 'above' ? 'ABOVE-FIREWALL' : 'below'; - lines.push(`${x.severity.toUpperCase().padEnd(5)} ${x.code.padEnd(24)} [${fw}] ${x.where} — ${x.message}`); + // baseline === false means new since the EXP-0001 snapshot was taken — + // surfaced here too, not just in --json, so a human running `constitution + // audit` can tell "known since day one" apart from "this commit's doing." + const tag = x.baseline === false ? ' [NEW]' : ''; + lines.push(`${x.severity.toUpperCase().padEnd(5)} ${x.code.padEnd(24)} [${fw}] ${x.where} — ${x.message}${tag}`); } lines.push(''); lines.push(`${errors.length} error(s), ${warns.length} warning(s). Above-firewall findings need the ratifier; the rest are fixable below (see \`constitution doctor\`).`); diff --git a/cli/src/engine/model.ts b/cli/src/engine/model.ts index 557ba8e..da71c87 100644 --- a/cli/src/engine/model.ts +++ b/cli/src/engine/model.ts @@ -35,6 +35,7 @@ export interface LedgerEntry { version: string; date: string; title: string; + body: string; // prose after the heading, to the next entry or end of file line: number; } @@ -65,6 +66,7 @@ export interface Adr { title: string; status: string; date: string; + body: string; // prose after the YAML frontmatter fence, to end of file supersedes: string[]; supersededBy: string[]; serves: string[]; diff --git a/cli/src/engine/parse.ts b/cli/src/engine/parse.ts index 5754c85..40e7189 100644 --- a/cli/src/engine/parse.ts +++ b/cli/src/engine/parse.ts @@ -201,10 +201,20 @@ function parseBoldBullets(lines: string[]): Record { } function parseLedger(lines: string[]): LedgerEntry[] { - const out: LedgerEntry[] = []; + const headings: { version: string; date: string; title: string; line: number; index: number }[] = []; for (let i = 0; i < lines.length; i++) { const m = lines[i].match(/^### \[([\w.\-]+)\]\s+—\s+(\S+)\s+—\s+(.+)$/); - if (m) out.push({ version: m[1], date: m[2], title: m[3].trim(), line: i + 1 }); + if (m) headings.push({ version: m[1], date: m[2], title: m[3].trim(), line: i + 1, index: i }); + } + // Body runs from just after one heading to the next heading — or, for the + // LAST entry, to end of file. The Amendments Ledger is typically the last + // section in CONSTITUTION.md, so there is no trailing heading to stop at. + const out: LedgerEntry[] = []; + for (let k = 0; k < headings.length; k++) { + const start = headings[k].index + 1; + const end = k + 1 < headings.length ? headings[k + 1].index : lines.length; + const body = normalize(lines.slice(start, end).join('\n')); + out.push({ version: headings[k].version, date: headings[k].date, title: headings[k].title, body, line: headings[k].line }); } return out; } @@ -320,11 +330,13 @@ export function parseAdr(root: string, file: string): Adr { const notes: string[] = []; const fm: Record = {}; const fmMatch = raw.match(/^---\n([\s\S]*?)\n---/); + let body = raw; if (fmMatch) { for (const line of fmMatch[1].split('\n')) { const m = line.match(/^([\w_]+):\s*(.*)$/); if (m) fm[m[1]] = m[2].replace(/#.*$/, '').trim(); } + body = raw.slice(fmMatch[0].length); } else { notes.push('no YAML frontmatter'); } @@ -340,6 +352,7 @@ export function parseAdr(root: string, file: string): Adr { title: fm['title'] ?? '', status: fm['status'] ?? '', date: fm['date'] ?? '', + body: normalize(body), supersedes: list(fm['supersedes']), supersededBy: list(fm['superseded_by']), serves: list(fm['serves']), @@ -374,8 +387,14 @@ export function parseExperiment(root: string, file: string): Experiment { } const section = (name: string): string => { - // (?=\n## |$) — up to the next section heading or end of file; JS has no \Z. - const m = raw.match(new RegExp(`^## ${name}[^\\n]*\\n([\\s\\S]*?)(?=\\n## |$)`, 'm')); + // Up to the next section heading or true end of file. Deliberately NOT + // using the 'm' flag: with 'm', `$` matches end-of-LINE, not end-of-string, + // which silently truncated any multi-line section body to its first line + // (every existing fixture/test used single-line sections, so this went + // unnoticed until EXP-0001's real multi-paragraph Hypothesis/Metric/ + // Decision rule sections exposed it). `(?:^|\n)` replaces the `^` anchor + // so a heading is still found either at file start or after a newline. + const m = raw.match(new RegExp(`(?:^|\\n)## ${name}[^\\n]*\\n([\\s\\S]*?)(?=\\n## |$)`)); const text = normalize(m?.[1] ?? ''); return PLACEHOLDER_SECTION.test(text) ? '' : text; }; diff --git a/cli/src/engine/prose.ts b/cli/src/engine/prose.ts new file mode 100644 index 0000000..ad70d82 --- /dev/null +++ b/cli/src/engine/prose.ts @@ -0,0 +1,74 @@ +// Text-analysis primitives for the governance-prose-clarity checks +// (EXP-0001, F-III pre-registered — see experiments/0001-governance-prose-clarity.md). +// Pure functions only: no I/O, no knowledge of Finding/audit. Consumed by +// audit.ts against Article.principle/.fitness/.why, Statute.rule/.why, +// Adr.body, and LedgerEntry.body. +// +// raw field text +// │ +// ▼ +// stripInlineMarkup() — drop `code spans` and [link](url) syntax (keeps +// link text) so a long URL or an inline `status: X` +// snippet never inflates a word/sentence count +// │ +// ├──► countWords() ── LEDGER-LENGTH (a size cap, not a style check) +// ├──► sentenceLengths() ── PROSE-SENTENCE-LEN (per-sentence word count) +// └──► hasStackedQualifier() ── PROSE-STACKED-QUALIFIER (field-level: 2+ +// DISTINCT qualifier patterns anywhere in the +// field, not required within one sentence — +// the real F-II example stacks an em-dash +// aside in one clause with an "exception" +// clause in another, within the same bullet) + +// Strips backtick code spans and markdown link syntax (kept: the link text, +// dropped: the URL) before any word/sentence analysis. Without this, a long +// URL counts as one giant "word" and an inline `field: VALUE` snippet can +// trip the stacked-qualifier check for reasons that have nothing to do with +// prose style — a known false-positive class for regex-based prose checks +// on raw markdown. +export function stripInlineMarkup(text: string): string { + return text.replace(/`[^`]*`/g, '').replace(/\[([^\]]*)\]\([^)]*\)/g, '$1'); +} + +export function countWords(text: string): number { + const stripped = stripInlineMarkup(text).trim(); + if (!stripped) return 0; + return stripped.split(/\s+/).filter(Boolean).length; +} + +// Splits on `.`/`!`/`?` followed by whitespace and an UPPERCASE letter (or +// end of string) — deliberately simple, not full sentence-boundary +// detection. Requiring an uppercase letter after the break is what keeps a +// numbered list ("... exactly one home: 1. the instance's own ... 2. the +// framework spec ...") from being sliced into meaningless fragments — it +// reads as one long sentence instead, which is the point: a numbered list is +// not itself convoluted, but a 50+ word run-on hiding inside one is exactly +// what this check exists to catch. Known, accepted limitation: an +// abbreviation followed by a capitalized word ("e.g. Foo does X") can still +// false-split; not fixed here — a genuine sentence-boundary detector is out +// of scope for this narrow check (see NOT in scope, eng review). +export function sentenceLengths(text: string): number[] { + const stripped = stripInlineMarkup(text).trim(); + if (!stripped) return []; + const parts = stripped.split(/(?<=[.!?])\s+(?=[A-Z])/); + return parts.map((s) => countWords(s)); +} + +const QUALIFIER_PATTERNS: RegExp[] = [ + /\s—\s/, // em-dash used as a clause-setting aside + /\b(except|unless|scoped to|exception)\b/i, // scope-narrowing clause + /\([^()]*\([^()]*\)[^()]*\)/, // a parenthetical nested inside another +]; + +// True only when 2+ DISTINCT pattern types appear somewhere in the field — +// not "the same pattern twice." A field enumerating N related facts (e.g. a +// numbered list, or a single em-dash aside on its own) is not itself +// convoluted; qualifier stacked on qualifier is. Checked at the FIELD level +// (the whole Principle/Fitness/Why/rule/body string), not per-sentence: the +// real calibration example (F-II's current Principle) stacks an em-dash +// aside in one clause with an "exception" clause in a different sentence of +// the same bullet — no single sentence contains both. +export function hasStackedQualifier(text: string): boolean { + const stripped = stripInlineMarkup(text); + return QUALIFIER_PATTERNS.filter((re) => re.test(stripped)).length >= 2; +} diff --git a/cli/test/engine.test.ts b/cli/test/engine.test.ts index 59e4973..afcbb36 100644 --- a/cli/test/engine.test.ts +++ b/cli/test/engine.test.ts @@ -1,9 +1,10 @@ import * as fs from 'fs'; import * as path from 'path'; import { describe, it, expect } from 'vitest'; -import { makeInstanceDir, MINI_CONSTITUTION } from './fixture'; +import { makeInstanceDir, MINI_CONSTITUTION, MINI_ADR } from './fixture'; import { loadInstance, canonicalHash, parseConstitution } from '../src/engine/parse'; import { audit } from '../src/engine/audit'; +import { stripInlineMarkup, countWords, sentenceLengths, hasStackedQualifier } from '../src/engine/prose'; import { computeLock, diffLock, writeLock } from '../src/engine/lock'; import { appendEvent, readEvents } from '../src/engine/events'; import { foldBoard, renderBoardHtml } from '../src/engine/board'; @@ -247,6 +248,274 @@ describe('audit', () => { }); }); +describe('prose.ts: text-analysis primitives (EXP-0001)', () => { + it('stripInlineMarkup drops code spans and keeps link text, not the URL', () => { + expect(stripInlineMarkup('')).toBe(''); + expect(stripInlineMarkup('see `status: RATIFIED` here')).toBe('see here'); + expect(stripInlineMarkup('see [registry.md](registry.md) here')).toBe('see registry.md here'); + expect(stripInlineMarkup('a `code span` and a [link](https://example.com/very/long/path) together')).toBe( + 'a and a link together' + ); + }); + + it('countWords handles empty/whitespace and strips markup before counting', () => { + expect(countWords('')).toBe(0); + expect(countWords(' \n\t ')).toBe(0); + expect(countWords('one two three')).toBe(3); + // a long URL must not inflate the count — it's stripped before counting + expect(countWords('see [it](https://example.com/a/b/c/d/e/f/g/h) now')).toBe(3); + }); + + it('sentenceLengths treats text with no terminal punctuation as one sentence', () => { + expect(sentenceLengths('')).toEqual([]); + expect(sentenceLengths('no ending punctuation here')).toEqual([4]); + }); + + it('sentenceLengths does not fragment a numbered list into meaningless pieces', () => { + const text = + 'A version number is a governed fact, and three exist: 1. the ledger version, ' + + '2. the framework spec version, 3. the tooling version. Axis three is different.'; + const lens = sentenceLengths(text); + // one long sentence covering the whole numbered list, then a short one + expect(lens.length).toBe(2); + expect(lens[0]).toBeGreaterThan(20); + expect(lens[1]).toBeLessThan(6); + }); + + it('hasStackedQualifier requires 2+ DISTINCT pattern types, not repeats of the same type', () => { + expect(hasStackedQualifier('a plain sentence with nothing unusual')).toBe(false); + expect(hasStackedQualifier('a clause set off — like this — on its own')).toBe(false); // 1 type only + expect(hasStackedQualifier('two separate — asides — used here — again')).toBe(false); // same type twice + expect(hasStackedQualifier('this holds — with one exception noted here')).toBe(true); // em-dash + "exception" + expect(hasStackedQualifier('unless scoped to a narrower case (see below)')).toBe(false); // 1 type only + expect(hasStackedQualifier('a rule (with a nested (inner) clause) — and an exception')).toBe(true); // nested paren + em-dash + exception + }); +}); + +describe('audit: governance prose clarity (EXP-0001, WARN-ONLY)', () => { + it('never blocks — findings are always severity warn, firewall below', () => { + const dir = makeInstanceDir({ + constitution: (s) => + s.replace( + '- **Principle** — Every widget passes verification before it ships.', + '- **Principle** — Every widget passes verification before it ships, except — as a carved-out exception — legacy widgets grandfathered in before the rule existed, which is a separate case entirely and always will be.' + ), + }); + const findings = audit(loadInstance(dir)); + const prose = findings.filter((f) => f.code.startsWith('PROSE-')); + expect(prose.length).toBeGreaterThan(0); + for (const p of prose) { + expect(p.severity).toBe('warn'); + expect(p.firewall).toBe('below'); + } + // warn findings never flip the exit-determining error filter + expect(findings.filter((f) => f.severity === 'error')).toEqual([]); + }); + + it('checkProse fires independently on every field it is wired to, not just Article Principle', () => { + // Covers the remaining checkProse() call sites (Article Fitness/Why, Statute + // rule/Why, ADR body) — regression guard against a wrong-field reference + // (e.g. checking .enforcedBy instead of .why) that unit tests on prose.ts + // alone wouldn't catch, since that would still be a syntactically valid call. + const dense = + 'This holds — with one exception noted here — because the rule was written before ' + + 'the edge case existed and nobody has revisited it since the original decision, which ' + + 'is itself now a separate problem entirely.'; + + const fitnessDir = makeInstanceDir({ + constitution: (s) => s.replace('- **Fitness** — CI runs the verify suite on every widget build.', `- **Fitness** — ${dense}`), + }); + const fitnessFindings = audit(loadInstance(fitnessDir)).filter((f) => f.code.startsWith('PROSE-')); + expect(fitnessFindings.some((f) => f.message.includes('A1 Fitness'))).toBe(true); + + const whyDir = makeInstanceDir({ + constitution: (s) => s.replace('- **Why** — unverified widgets break user trust.', `- **Why** — ${dense}`), + }); + const whyFindings = audit(loadInstance(whyDir)).filter((f) => f.code.startsWith('PROSE-')); + expect(whyFindings.some((f) => f.message.includes('A1 Why'))).toBe(true); + + const statuteRuleDir = makeInstanceDir({ + map: (s) => s.replace('All widget checks run through the single verify entrypoint.', dense), + }); + const statuteRuleFindings = audit(loadInstance(statuteRuleDir)).filter((f) => f.code.startsWith('PROSE-')); + expect(statuteRuleFindings.some((f) => f.message.startsWith('statute "'))).toBe(true); + + const statuteWhyDir = makeInstanceDir({ + map: (s) => s.replace('two entrypoints drift apart silently.', dense), + }); + const statuteWhyFindings = audit(loadInstance(statuteWhyDir)).filter((f) => f.code.startsWith('PROSE-')); + expect(statuteWhyFindings.some((f) => f.message.includes('Why'))).toBe(true); + + const adrDir = makeInstanceDir(); + fs.writeFileSync( + path.join(adrDir, 'decisions', '0001-verify-pre-merge.md'), + MINI_ADR.replace('Pre-merge, always.', `Pre-merge, always. ${dense}`) + ); + const adrFindings = audit(loadInstance(adrDir)).filter((f) => f.code.startsWith('PROSE-')); + expect(adrFindings.some((f) => f.message.includes('ADR 0001'))).toBe(true); + }); + + it('checkProse skips empty/undefined fields without producing findings', () => { + const dir = makeInstanceDir({ + constitution: (s) => s.replace('- **Why** — unverified widgets break user trust.', '- **Why** — '), + }); + const findings = audit(loadInstance(dir)).filter((f) => f.code.startsWith('PROSE-')); + expect(findings.some((f) => f.message.includes('A1 Why'))).toBe(false); + }); + + it('a non-last ledger entry stops its body at the next heading, not before or after', () => { + // The fixture's only prior ledger test covers the LAST-entry/EOF boundary + // (headings.length - 1). This covers the other branch of parseLedger's + // ternary: end = headings[k+1].index — an off-by-one here would silently + // bleed the next entry's heading (or its own last line) across the boundary. + const dir = makeInstanceDir({ + constitution: (s) => + s.replace( + '### [1.2.3] — 2026-07-01 — founding ratification\n- Founding entry. Ratifier: Ada Lovelace.', + '### [1.2.3] — 2026-07-01 — founding ratification\n- Founding entry line one.\n- Founding entry line two.\n\n' + + '### [1.3.0] — 2026-07-02 — second entry\n- Second entry text. Ratifier: Grace Hopper.' + ), + }); + const inst = loadInstance(dir); + expect(inst.constitution.ledger.length).toBe(2); + expect(inst.constitution.ledger[0].body).toContain('line two'); + expect(inst.constitution.ledger[0].body).not.toContain('Second entry text'); + expect(inst.constitution.ledger[1].body).toContain('Grace Hopper'); + }); + + it('parseAdr body falls back to the whole raw file when there is no YAML frontmatter', () => { + const dir = makeInstanceDir(); + fs.writeFileSync( + path.join(dir, 'decisions', '0002-no-frontmatter.md'), + '## Question of law\nWhat happens with no frontmatter?\n\n## Ruling\nIt still parses.\n' + ); + const inst = loadInstance(dir); + const adr = inst.adrs.find((a) => a.file.endsWith('0002-no-frontmatter.md'))!; + expect(adr.body).toContain('It still parses'); + expect(adr.parseNotes).toContain('no YAML frontmatter'); + }); + + it('a baseline file with valid JSON but a non-array keys field degrades to nothing-known', () => { + // Distinct from the "corrupt JSON" test above — this is syntactically + // valid JSON with the wrong shape, exercising the other branch of + // readProseBaseline's Array.isArray(parsed?.keys) check. + const dir = makeInstanceDir(); + const opsDir = path.join(dir, '.constitution'); + fs.mkdirSync(opsDir, { recursive: true }); + fs.writeFileSync(path.join(opsDir, 'prose-baseline.json'), JSON.stringify({ keys: 'oops' })); + const findings = audit(loadInstance(dir)).filter((f) => f.code.startsWith('PROSE-') || f.code === 'LEDGER-LENGTH'); + expect(findings.every((f) => f.baseline === false)).toBe(true); + }); + + it('flags a ledger entry over the word cap and leaves a short one alone', () => { + const longEntry = 'Session narrative. '.repeat(40); // ~120 words, pad to exceed 150 + const dir = makeInstanceDir({ + constitution: (s) => + s.replace( + '### [1.2.3] — 2026-07-01 — founding ratification\n- Founding entry. Ratifier: Ada Lovelace.', + `### [1.2.3] — 2026-07-01 — founding ratification\n- ${longEntry}${longEntry}` + ), + }); + const findings = audit(loadInstance(dir)); + const ledgerFinding = findings.find((f) => f.code === 'LEDGER-LENGTH'); + expect(ledgerFinding).toBeDefined(); + expect(ledgerFinding!.message).toContain('[1.2.3]'); + + const shortDir = makeInstanceDir(); // unmodified fixture, short ledger entry + expect(audit(loadInstance(shortDir)).find((f) => f.code === 'LEDGER-LENGTH')).toBeUndefined(); + }); + + it('the last ledger entry (no trailing heading) is still checked, not silently dropped', () => { + // MINI_CONSTITUTION's single ledger entry IS the last section in the file — + // this is the real EOF-boundary case (verified against CONSTITUTION.md itself). + const dir = makeInstanceDir(); + const inst = loadInstance(dir); + expect(inst.constitution.ledger[0].body.length).toBeGreaterThan(0); + expect(inst.constitution.ledger[0].body).toContain('Ada Lovelace'); + }); + + it('baseline-snapshot: self-initializes on first run, then isolates new findings from known ones', () => { + const dir = makeInstanceDir({ + constitution: (s) => + s.replace( + '- **Principle** — Every widget passes verification before it ships.', + '- **Principle** — Every widget passes verification before it ships, except — as a carved-out exception — legacy widgets grandfathered in before the rule existed, which is a separate case entirely and always will be.' + ), + }); + const first = audit(loadInstance(dir)); + const firstProse = first.filter((f) => f.code.startsWith('PROSE-')); + expect(firstProse.length).toBeGreaterThan(0); + expect(firstProse.every((f) => f.baseline === true)).toBe(true); // day-one: nothing is "new" + expect(fs.existsSync(path.join(dir, '.constitution', 'prose-baseline.json'))).toBe(true); + + // Second run, nothing changed — still baseline (matches the snapshot). + const second = audit(loadInstance(dir)).filter((f) => f.code.startsWith('PROSE-')); + expect(second.every((f) => f.baseline === true)).toBe(true); + + // Introduce a NEW dense field (A2's Principle) — only the new one is baseline:false. + fs.writeFileSync( + path.join(dir, 'CONSTITUTION.md'), + fs + .readFileSync(path.join(dir, 'CONSTITUTION.md'), 'utf8') + .replace( + '- **Principle** — Failures are always surfaced to the user.', + '- **Principle** — Failures are always surfaced to the user, except — in one narrow exception — during a graceful shutdown, which is a separate case entirely and always will be.' + ) + ); + const third = audit(loadInstance(dir)).filter((f) => f.code.startsWith('PROSE-')); + const newOnes = third.filter((f) => f.baseline === false); + const oldOnes = third.filter((f) => f.baseline === true); + expect(newOnes.length).toBeGreaterThan(0); + expect(oldOnes.length).toBeGreaterThan(0); + }); + + it('the written baseline has no duplicate keys, even when 2+ fields share a where', () => { + // Adversarial review finding: Principle and Fitness on the same Article + // line both key to the same `where`, and an undeduped write grew + // duplicate array entries every time the self-init branch ran. + const dir = makeInstanceDir({ + constitution: (s) => + s + .replace( + '- **Principle** — Every widget passes verification before it ships.', + '- **Principle** — Every widget passes verification before it ships, except — as a carved-out exception — legacy widgets grandfathered in before the rule existed, which is a separate case entirely and always will be.' + ) + .replace( + '- **Fitness** — CI runs the verify suite on every widget build.', + '- **Fitness** — This holds — with one exception noted here — because the rule predates the edge case and nobody has revisited it since, which is itself now a separate problem entirely.' + ), + }); + audit(loadInstance(dir)); + const raw = JSON.parse(fs.readFileSync(path.join(dir, '.constitution', 'prose-baseline.json'), 'utf8')); + expect(raw.keys.length).toBe(new Set(raw.keys).size); + }); + + it('a missing or corrupt baseline file degrades gracefully instead of crashing', () => { + const dir = makeInstanceDir(); + const opsDir = path.join(dir, '.constitution'); + fs.mkdirSync(opsDir, { recursive: true }); + fs.writeFileSync(path.join(opsDir, 'prose-baseline.json'), '{ not valid json'); + expect(() => audit(loadInstance(dir))).not.toThrow(); + const findings = audit(loadInstance(dir)).filter((f) => f.code.startsWith('PROSE-') || f.code === 'LEDGER-LENGTH'); + // corrupt baseline degrades to "nothing known" — everything reads as not-yet-baselined + expect(findings.every((f) => f.baseline === false)).toBe(true); + }); + + it('the real F-II example: sentence-length fires on the actual known-bad text', () => { + // Regression guard against the exact case the WARN-ONLY experiment is + // calibrated on (see design doc + eng review). Uses the real repo, not the + // fixture, since this is the specific known-bad text the thresholds were + // chosen against. + const repoRoot = path.resolve(__dirname, '..', '..'); + const inst = loadInstance(repoRoot); + const findings = audit(inst); + const f2 = findings.find((f) => f.code === 'PROSE-SENTENCE-LEN' && f.message.includes('F-II')); + expect(f2).toBeDefined(); + const ledgerHit = findings.find((f) => f.code === 'LEDGER-LENGTH' && f.message.includes('[0.17.0]')); + expect(ledgerHit).toBeDefined(); + }); +}); + describe('ops plane: events + board', () => { it('folds the event log into kanban columns with blocked flags', () => { const dir = makeInstanceDir(); diff --git a/cli/test/experiments.test.ts b/cli/test/experiments.test.ts index 4eaefa6..2c3ef39 100644 --- a/cli/test/experiments.test.ts +++ b/cli/test/experiments.test.ts @@ -102,4 +102,22 @@ describe('experiment section parsing edge case', () => { expect(inst.experiments[0].decisionRule).toContain('RATIFY if'); expect(audit(inst).filter((f) => f.code.startsWith('EXP-'))).toEqual([]); }); + + it('captures a section spanning multiple physical lines, not just the first line', () => { + // Regression: the section regex used to run with the 'm' flag, which + // makes `$` match end-of-LINE rather than end-of-string — silently + // truncating any section body wrapped across multiple lines to just its + // first line. Every prior fixture happened to use single-line sections, + // so this went unnoticed until a real multi-paragraph experiment file + // (EXP-0001-governance-prose-clarity.md) exposed it. + const multiLine = GOOD_EXP.replace( + '## Hypothesis\nThe verify suite catches 90% of seeded widget defects.', + '## Hypothesis\nThe verify suite catches 90% of seeded widget defects,\nmeasured across a full release cycle,\nnot just a single sprint.' + ); + const dir = withExperiment(multiLine); + const inst = loadInstance(dir); + expect(inst.experiments[0].hypothesis).toContain('90%'); + expect(inst.experiments[0].hypothesis).toContain('full release cycle'); + expect(inst.experiments[0].hypothesis).toContain('single sprint'); + }); }); diff --git a/experiments/EXP-0001-governance-prose-clarity.md b/experiments/EXP-0001-governance-prose-clarity.md new file mode 100644 index 0000000..78a1835 --- /dev/null +++ b/experiments/EXP-0001-governance-prose-clarity.md @@ -0,0 +1,55 @@ +# EXP-0001 · Governance prose clarity + +``` +candidate → Statute or Article, TBD (see Constitutional impact below) +status RUNNING +pre-registered 2026-07-06 +ratifier Chinmay +``` + +## Hypothesis +A sentence-length ceiling (30 words) plus a stacked-qualifier pattern check (2+ +distinct qualifier types — em-dash aside, except/unless/scoped-to clause, nested +parenthetical — anywhere in the same field) catches real convolution-drift in +Articles, Statutes, and ADRs with an acceptably low false-positive rate. A +150-word cap on Amendments Ledger entries catches narrative that belongs in +BUILDLOG.md instead. + +## Enforcement during the experiment +WARN-ONLY. `constitution audit` reports `PROSE-SENTENCE-LEN`, `PROSE-STACKED-QUALIFIER`, +and `LEDGER-LENGTH` as `severity: warn` findings — never `error`, never blocking the +pre-commit hook or CI. A one-time baseline snapshot (`.constitution/prose-baseline.json`, +written automatically on the first `constitution audit` run after this file's +pre-registered date) separates pre-existing known-dense text from genuinely new +findings, so already-known bloat (the current F-II text, the existing Ledger +entries) doesn't contaminate the false-positive-rate signal below. + +## Metric +- **False-positive rate**: of all NEW findings (`baseline: false`) surfaced across + the next 15 commits touching `CONSTITUTION.md` or `decisions/`, what share are + judged (by the ratifier, at MEASURED) to be flagging text that is NOT actually + convoluted — i.e. legitimate enumeration, necessary precision, or a false split + from the sentence-splitter's known abbreviation/numbered-list limitations. +- **Guardrail (catch rate)**: the checks correctly flag the two known-bad + calibration cases — Article F-II's current `Principle`/`Fitness` text + (`PROSE-SENTENCE-LEN`) and Ledger entry `[0.17.0]` (`LEDGER-LENGTH`) — verified + in `cli/test/engine.test.ts` as a standing regression guard, independent of the + 15-commit window. + +## Decision rule (frozen — do not edit after PRE-REGISTERED) +- RATIFY if false-positive rate is under 20% over the 15-commit window AND both + guardrail cases still fire. +- REJECT if false-positive rate is 20% or higher after one retuning pass of the + thresholds (sentence-word ceiling, qualifier patterns, ledger word cap). +- ITERATE if the window closes with fewer than 5 NEW findings to judge (too little + signal to call either way) — extend the window by 15 more commits before + re-measuring. + +At RATIFY: the ratifier decides Statute vs. Article placement. The L0-trace/ +tech-swap analysis done during the eng review (a Vale-enforced or any +tool-enforced rule fails "survives a tech swap" per `process/layers.md`) +provisionally points to Statute, but is not binding — the ratifier may override +with the false-positive/catch-rate evidence in hand. + +## Result (fill at MEASURED) +Not yet measured. diff --git a/process/amendment-lifecycle.md b/process/amendment-lifecycle.md index 5703d3c..314d2da 100644 --- a/process/amendment-lifecycle.md +++ b/process/amendment-lifecycle.md @@ -34,6 +34,10 @@ A superseded rule is preserved in **three** places: So "what did this rule used to say, when did it change, and why" is always answerable: the ledger gives the what/when, the ADR gives the why. +Ledger entry shape: [../templates/ledger-entry.md](../templates/ledger-entry.md) — a +change-log record, not a narrative. Session narrative belongs in the product's own +build log, if it keeps one, never in the ledger itself. + ## Velocity by layer L0 ossifies on purpose; L4 churns freely. The tiered velocity is the mitigation for both diff --git a/templates/adr.md b/templates/adr.md index d079ce2..ae365b2 100644 --- a/templates/adr.md +++ b/templates/adr.md @@ -32,3 +32,12 @@ interpreting one Article is the evidence that the Article itself should be amend ## Alternatives considered + + diff --git a/templates/article.md b/templates/article.md index c6cbdd9..14ba410 100644 --- a/templates/article.md +++ b/templates/article.md @@ -18,4 +18,10 @@ Before adding this, run the inclusion test (process/layers.md). It is an Article only if it passes ALL four: general · traces to L0 · falsifiable · survives a tech swap. Fails "survives a tech swap" → it's a Statute (L2). Fails "falsifiable" → it's L0 prose. + +Keep every bullet short and plain: one idea per sentence, no stacking a second qualifier +onto a first (an em-dash aside plus an "except" clause in the same bullet is exactly the +shape that got flagged — see F-II's own re-amendment history). A numbered list of related +facts is fine; a run-on sentence hiding inside one is not. `constitution audit` runs a +WARN-ONLY check for this (EXP-0001) — it never blocks, but a warning is a real signal. --> diff --git a/templates/ledger-entry.md b/templates/ledger-entry.md new file mode 100644 index 0000000..ba3e86e --- /dev/null +++ b/templates/ledger-entry.md @@ -0,0 +1,20 @@ + + +### [] — — +- **What changed:** . +- **Why:** . +- **ADR:** [-](../decisions/-.md) (omit if wording-only, no ruling needed). + +