The governance engine: parse/audit/lock/tone/ops planes in the CLI (v0.17.0)

.github/workflows/governance.yml

@@ -0,0 +1,45 @@
+name: governance
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+jobs:
+  engine:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 22
+      - name: Install
+        working-directory: cli
+        run: npm ci
+      - name: Build (strict tsc)
+        working-directory: cli
+        run: npm run build
+      - name: Engine tests
+        working-directory: cli
+        run: npm test
+
+  law:
+    runs-on: ubuntu-latest
+    needs: engine
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 22
+      - name: Build engine
+        working-directory: cli
+        run: npm ci && npm run build
+      - name: Structural audit (L0–L4 graph)
+        run: node cli/dist/index.js audit
+      - name: Firewall gate (F-IV — ratified text vs. accepted lock)
+        run: |
+          if [ -f constitution.lock.json ]; then
+            node cli/dist/index.js firewall
+          else
+            echo "no constitution.lock.json yet — firewall gate skipped (audit already warns)."
+          fi

BUILDLOG.md

@@ -0,0 +1,131 @@
+# BUILDLOG — the overhaul, session by session
+
+Running log for the multi-session overhaul (goal set 2026-07-04): turn `constitution`
+into an installable AI-governance product. Later sessions: read this before doing
+anything. Every entry records what was tried, what broke, and what was verified by
+actually running it. "Untested" means untested.
+
+Design: [docs/architecture.md](docs/architecture.md). Non-negotiables from the goal:
+law plane stays concise; tone is a read-time view (one canonical text, ever); ops
+visibility lives in `.constitution/`, not the law; F-IV firewall — no agent writes
+`status: RATIFIED` or edits ratified L0/L1; above-firewall changes are queued as
+proposals, never applied.
+
+---
+
+## Session 1 — 2026-07-04 (worktree elastic-chebyshev-ec0c49, base f85c793 v0.16.12)
+
+### Scope chosen
+Phase 1 of the overhaul: build the **governance engine** into the existing CLI package
+(`cli/`, `@chinmaygit/constitution-cli`) rather than a new package — it already has the
+bin, the vendoring pipeline, and a GitHub Packages release path. Workstreams:
+
+1. Engine core: typed model + parser for the law plane (CONSTITUTION.md, governance
+   map, statute homes, ADRs) + canonical hashing.
+2. Deterministic structural audit (machine version of audit-structure's deterministic
+   subset) with findings classified above/below the firewall.
+3. `constitution.lock.json` + `constitution firewall` — the firewall as a CI gate
+   (F-IV enforcement AUDITED → GATED). `lock accept` is TTY-only + typed confirmation.
+4. Ops plane: `.constitution/events.jsonl`, `constitution feature <verb>`,
+   `constitution board` (terminal + static HTML dashboard).
+5. Tone rendering with hash-keyed cache + drift check (`render`, `tones check`).
+6. Compile pack (`constitution compile`), proposals queue + human-only `ratify`,
+   `doctor` (autofix below firewall, queue drafts above).
+7. Tests (vitest) incl. dogfooding parse/audit against this repo itself.
+
+### Decisions (and why)
+- **Engine lives in the existing `cli/` package.** A second package would split the
+  version axis the ledger's [0.16.10] statute just unified.
+- **Parser targets the existing document shapes** (header fence, `**P1.**` lines,
+  `### Article <id> — <name>` + backtick field line, statute bullets with `· serves:`
+  / `· enforced-by:`, ADR YAML frontmatter). No format migration — the law documents
+  are already machine-regular; changing the format would churn ratified text (above
+  the firewall) for no gain.
+- **Lockfile hashes normalized text** (collapse whitespace) so reflow ≠ amendment.
+- **No new runtime deps** beyond existing `prompts`. Tone generation shells out to
+  `claude -p` when present; degrades to explicit "generation unavailable" rather than
+  silently serving stale renders.
+- **Ledger entry for this work is drafted but the version bump is real** — below the
+  firewall (tooling + docs; no Article text touched, no status changed). Entry marked
+  as agent-authored pending operator review.
+
+### What happened
+- Built all seven workstreams into `cli/src/engine/` (`model`, `parse`, `audit`, `lock`,
+  `events`, `board`, `tone`, `compile`, `proposals`, `doctor`) + rewrote `cli/src/index.ts`
+  as the full subcommand dispatcher. `npm run build` (strict tsc) clean.
+- Docs: `docs/{architecture,quickstart,firewall,tone,ops}.md`, README rewritten as a
+  product, `cli/README.md` intro updated. CI: `.github/workflows/governance.yml`
+  (build + test + audit + conditional firewall gate).
+- Version: `0.17.0` in CONSTITUTION.md header + ledger entry; `cli/package.json` synced
+  **by `constitution doctor` itself** via the new committed `constitution.config.json`
+  (`versionSync`) — the self-healing path validated on its first real use.
+- Two new `cli/AGENTS.md` statutes: engine determinism (only LLM call is tone's
+  injectable generator); engine changes ship with failing-first tests.
+
+### What broke (and the fixes)
+1. **Map parser over-matched**: existence-checking every backtick token in AGENTS.md
+   produced 6 false MAP-BROKEN-LINK errors (prose mentions like `SKILL.md`, gitignored
+   dirs). Fix: only real markdown links are existence-checked; backticks only classify
+   the constitution/decisions declarations.
+2. **Statute parser dropped 13 of 16 statutes**: bullets whose bold rule closes on line 1
+   but continue with indented commentary before the `· serves:` lines bailed out with no
+   annotations. Fix: pre-annotation indented prose folds into the rule text. 3 → 16
+   statutes parsed on this repo.
+3. **`normalize` kept newlines**, so re-wrapping a paragraph changed the canonical hash —
+   contradicting the reflow-safe lock design. Fix: collapse ALL whitespace.
+4. **Tone cache read returned the embedded "derived artifact" HTML comment** as part of
+   the render body. Fix: strip it on read.
+5. **Doctor test fixture** orphaned P2, producing a second (correct) above-firewall
+   finding my test didn't expect — the engine was right, the test mutation was wrong.
+
+### Verified by running (all on this machine, this session)
+- `cd cli && npm run build` — strict tsc, no errors.
+- `cd cli && npm test` — **14/14 vitest cases pass**, incl. the dogfood test: parses this
+  repo (P1; F-I…F-VII; 16 statutes; ADR-0001) and audits it with zero errors.
+- `node cli/dist/index.js audit` on this repo → exit 0, two honest warnings:
+  F-III `HOLDS+UNGUARDED` mechanization debt (a real, pre-existing condition) and
+  `LOCK-MISSING`.
+- `lock status` lists the 8 ratified units with hashes; `lock accept < /dev/null`
+  **refuses** (non-TTY, F-IV message) — the agent-can't-cross-it property, demonstrated.
+- Lock unit tests prove: editing ratified text → `changed`; agent flipping
+  PROPOSED→RATIFIED → `added`; repeal → `removed`; conformance/enforcement edits do NOT
+  trip the gate; reflow does NOT trip the gate.
+- `feature declare/start` + `compile --out` + `board` + `board --html` → real events in
+  `.constitution/events.jsonl`, correct terminal Kanban, `board.html` written.
+- `doctor` → pruned nothing (nothing stale), queued nothing (no above-firewall findings),
+  synced `cli/package.json` 0.16.12 → 0.17.0. Proposal queue/dedupe/ruling covered by
+  tests (doctor queues above-firewall drafts exactly once; never edits the law file —
+  byte-compared in the test).
+- Tone: stub-generator tests prove generate → cache-hit → stale-on-amend → refuse/prune.
+
+### Addendum — tarball install verified (same session, commit 2)
+- Added non-interactive `constitution init --name N --ratifier R --agents a,b` (CI/
+  scripted installs; interactive path unchanged).
+- Ran the [0.16.11] pre-publish standard: `npm pack` → installed the real
+  `chinmaygit-constitution-cli-0.17.0.tgz` into a scratch consumer via npm →
+  `constitution init` (non-interactive) wrote CONSTITUTION.md (placeholders correctly
+  substituted: name, ratifier, `constitution@0.17.0` pin), AGENTS.md map, vendored
+  templates/process, compiled `.claude/` skills, ops scaffold with its `.gitignore`.
+- In that fresh consumer, from the installed binary: `audit` → 0 errors, 1 honest
+  warning (LOCK-MISSING); `feature declare` + `compile --out` + `board` + `doctor` all
+  worked. The product loop is real for a brand-new team, end to end.
+
+### Known-untested / deferred (next sessions pick up here)
+- **Tone generation with a real LLM**: `claude -p` exists here but nested invocation gets
+  401 inside this session — engine degrades honestly (verified); real render quality
+  unverified.
+- **`constitution lock accept` on this repo is the operator's act** (F-IV): Chinmay runs
+  it in a terminal, commits `constitution.lock.json`; CI's firewall step then goes live.
+- Publishing `0.17.0` to GitHub Packages (operator npm auth).
+- `ratify` interactive flow untested end-to-end (needs a TTY); logic unit-tested via
+  `recordRuling`.
+- Skills not yet rewired to consume engine output (`compile-prompt` should ingest
+  `constitution compile` packs; `audit-structure` should start from `audit --json`).
+- DSAMind adoption; `init` against oddly-shaped repos (pre-existing known gap); a served
+  (live) dashboard beyond static HTML; multi-instance registry telemetry.
+
+### Known-untested / deferred
+- Publishing 0.17.0 to GitHub Packages (needs operator's npm auth).
+- Tone generation quality (needs `claude` CLI at runtime; engine tested with a stub).
+- DSAMind adoption of the new engine (separate repo, separate session).
+- Multi-instance/registry telemetry, web dashboard beyond static HTML — later phases.

CONSTITUTION.md

@@ -1,7 +1,7 @@
 # The constitution framework — Constitution
 
 ```
-framework: constitution@0.16.12   (self-hosted)
+framework: constitution@0.17.0   (self-hosted)
 ratifier:  Chinmay
 ```
 
@@ -157,6 +157,46 @@ on the same Article is the signal that the Article itself needs amending.
 Superseded clauses are never deleted — they are kept here with a forward link and the
 ADR that justified the change.
 
+### [0.17.0] — 2026-07-04 — The governance engine: the CLI becomes the product's deterministic core
+- **Overhaul session 1** (see `BUILDLOG.md` + `docs/architecture.md` for the full record and
+  design). The CLI grows from installer to engine — everything below is deterministic code in
+  `cli/src/engine/`, tested (`cli/test/`, 14 vitest cases incl. a dogfood test that parses and
+  audits this very repo) and wired into CI (`.github/workflows/governance.yml`).
+- **Three-plane architecture made physical**: law plane (this file, statute homes, `decisions/`
+  — unchanged format, now machine-parsed), engine (the CLI), ops plane (`.constitution/` —
+  events, tone cache, proposal queue, compiles, board; volume lives there, never here).
+- **The firewall becomes a gate** (F-IV enforcement path AUDITED → GATED once adopted):
+  `constitution.lock.json` records canonical hashes of ratified L0/L1 units, written only by
+  `constitution lock accept` (interactive TTY + typed confirmation — refuses agents/pipes,
+  verified); `constitution firewall` fails CI on changed/added/removed ratified units.
+  **The lock is not yet accepted** — that is the ratifier's own act, pending.
+- **`constitution audit`** — deterministic structural audit (refs resolve, layers trace up,
+  fields legal, ledger/version sync, lock drift), findings classified by what the FIX touches
+  (above/below firewall). Ran clean on this repo: 0 errors, 2 honest warnings (F-III
+  mechanization debt; lock missing).
+- **`constitution doctor`** — self-healing below the firewall (prunes stale tone renders,
+  version-syncs `constitution.config.json` targets, repairs ops scaffold); above-firewall
+  findings are DRAFTED into `.constitution/proposals/` and wait for `constitution ratify`
+  (human-only, interactive) — never applied.
+- **Tone as a view** (`constitution render <unit> --tone plain|casual|formal`): one canonical
+  text ever; renders are derived artifacts cache-keyed by canonical hash + transform version,
+  stale by construction on amendment; `tones check` detects, doctor prunes. LLM generation via
+  `claude -p` (untested end-to-end in-session — no nested auth; stub-tested).
+- **Ops visibility** (`constitution feature <verb>`, `constitution board [--html]`): Kanban
+  over `.constitution/events.jsonl` (Declared → Compiled → Building → Validating → Shipped)
+  plus a governance-health strip; reads the law by id, stores nothing in it.
+- **`constitution compile "<task>" [--out]`** — emits the deterministic L4 compile pack (all
+  ratified canonical units + statute/ADR indexes + the briefing contract); judgment stays in
+  the `compile-prompt` skill, which now compiles over guaranteed-complete, current law.
+- Docs rewritten as a product (`README.md`, `docs/`); two new `cli/AGENTS.md` statutes
+  (engine determinism; failing-first engine tests). Statute parser fix along the way: bullets
+  whose bold rule closes before indented commentary were silently dropped (3 → 16 statutes
+  parsed here).
+- **Below the firewall throughout** — no Article text, status, or L0 line touched; the parser
+  targets the existing document shapes. Authored autonomously per the standing overhaul goal;
+  entry pending the operator's review. `cli/package.json` → `0.17.0` (sync statute holding).
+  Not yet published to GitHub Packages (operator's npm auth required).
+
 ### [0.16.12] — 2026-07-01 — `AGENT.md` → `AGENTS.md` (amends F-VII); scaffold reads real templates
 - **`AGENT.md` renamed to `AGENTS.md` everywhere** — singular was wrong. `AGENTS.md` (plural)
   is the actual cross-tool convention; DSAMind itself already has a real one (its own