Skip to content

Clotho Task 3: thread-ledger.mjs — signed append-only thread ledger (held for TELOS review)#115

Merged
dsmcewan merged 14 commits into
mainfrom
clotho-task3-ledger
Jul 16, 2026
Merged

Clotho Task 3: thread-ledger.mjs — signed append-only thread ledger (held for TELOS review)#115
dsmcewan merged 14 commits into
mainfrom
clotho-task3-ledger

Conversation

@dsmcewan

@dsmcewan dsmcewan commented Jul 16, 2026

Copy link
Copy Markdown
Owner

HELD — implementation PR (Task 3). Awaits deterministic gate + required-seat review + The Eye's acceptance (per-task cadence, #109).

Traceability

What this builds (v12 Task 3)

clotho/thread-ledger.mjs — signed, append-only, hash-chained canonical-JSONL ledger (zero-dependency; node:crypto Ed25519 — merkle-dag/crypto.mjs implements a different envelope + no chaining, so per v12 signing is done directly; canonicalJson/validators reused from registry.mjs):

  • createLedger(path, {signKey?, wovenAt?, repoHead?, repositoryRef?}) → {header, appendEdge, appendStatus, close, abort}
  • Header (SPKI-b64 pub_key, one woven_at/repo_head/repository_ref, weave_version:1); edge/status records = payload + prev_hash (sha256 of prior line) + record_hash (sha256 of canonicalJson(payload+prev_hash)) + Ed25519 signature over the raw 32-byte digest; LF-terminated; chain over the complete signed line.
  • close(coverage) — D19 generic validation against injected fixture coverage (5 weavers, executed|skipped, D24 inspected_source_counts schema, file: content-address refs); rejects failed and a skipped weaver that produced edges; no committed-inventory dependency.
  • abort() idempotent; every append/close failure poisons + closes the descriptor (D22); exclusive wx creation.
  • verifyLedger — canonical encoding, single header, chain + signature, human-only status adjudication, endpoint re-validation, D24 schema, one final trailer, CRLF/missing-LF errors. readEdges yields edges structurally.

Acceptance criteria (v12 Task 3 exit — verified)

  • ledger tests green against injected fixture coverage
  • every failure path closes its descriptor (abort/poison)
  • references no committed inventory (D19)
  • npm test green; diff confined to clotho/; zero deps; no spine change
  • deterministic gate meets (11/11)

Note: atomic no-replace publication (D20/D28) is the driver's job (Task 5 weave.mjs), not this ledger primitive.

🤖 Generated with Claude Code

dsmcewan and others added 14 commits July 16, 2026 14:44
@dsmcewan @claude
Clotho Task 3: thread-ledger.mjs — signed append-only thread ledger
d57c630 
Implements v12 Task 3: createLedger/verifyLedger/readEdges with the signed,
hash-chained canonical-JSONL record format. Zero-dependency, node:crypto for
Ed25519 (merkle-dag/crypto.mjs implements a different envelope/no chaining, so
per v12 Task 3 signing is done directly), canonicalJson + validators reused from
clotho/registry.mjs.

- header (pub_key SPKI b64, woven_at, repo_head, repository_ref, weave_version:1)
- edge/status records: payload + prev_hash (sha256 of prior line) + record_hash
  (sha256 of canonicalJson(payload+prev_hash)) + Ed25519 signature over the raw
  32-byte digest; LF-terminated; chain over the complete signed line.
- close(coverage): D19 generic validation of the trailer against INJECTED
  fixture coverage (5 weavers, executed|skipped, D24 inspected_source_counts
  schema, file: content-address refs); rejects 'failed' state and a skipped
  weaver that produced edges; no committed-inventory dependency.
- abort() idempotent; every append/close failure poisons + closes the descriptor;
  exclusive wx creation; appends rejected after close.
- verifyLedger: canonical encoding, single header, chain + signature, status
  adjudication (human-only), endpoint re-validation, D24 schema, one final
  trailer, CRLF/missing-LF errors. readEdges yields edges structurally.

test-ledger.mjs added to the runner. npm test green; confined to clotho/; zero deps.

Traceability: v12 sha256:bdc93901, authz-005, Eye impl-authorization #109.
Decisions D5/D19/D20/D22/D24/D28/D29.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X8pTab2QMpfM2KsZmwkeYg
@dsmcewan @claude
Clotho Task 3 (revision): verifier trust boundary, streaming readEdge…
160cbf4 
…s, exact weaver order

Addresses the required-seat review of PR #115:
- verifyLedger: `records` contains ONLY trusted edge+status records (never
  header/trailer); on the first failing line it stops conferring trust (no
  record on/after that line); cross-checks weaver-assertor vs manifest state
  (a skipped weaver that asserted an edge fails). Empty/CRLF/missing-LF/no-final-
  trailer all fail closed.
- readEdges: streams via fs.createReadStream + incremental line splitter, yields
  before EOF (proven by a gated injected stream), skips only the header, and
  yields status records and the trailer too (not just edges).
- createLedger: exclusive `wx` is the sole atomic existence gate (no existsSync
  TOCTOU); injectable file handle enables descriptor-closure proofs (D22).
- close(): coverage carries exactly the five weaver ids in stable declared order.
Tests expanded toward the frozen surface: injected key, human accept/reject/
supersede + model/weaver rejection, D24 rejections, exact-order rejection,
descriptor-closure via injected handle (append/close/abort), incremental gated-
stream proof, and independently-signed adversarial fixtures (failed state,
skipped-nonzero-count, skipped-weaver-with-edge, valid human supersedes).

npm test green; confined to clotho/; zero deps.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X8pTab2QMpfM2KsZmwkeYg
@dsmcewan @claude
Clotho Task 3 (revision 2): frozen per-weaver inventory-id table + wo…
ec77e2d 
…ven_at equality

Addresses the round-2 required-seat blockers (claude, codex) on PR #115:
- Task-3-local frozen per-weaver inventory-id table (v12 D24/D26/D31): every
  weaver's inspected_source_counts must carry EXACTLY its required sorted ids
  (git: package-files/package-symbols; code: package-modules; test:
  package-manifests/test-files; doc: doc-files; ledger: contract-files/
  ledger-sources/run-sources) at both close and verify. Count cardinalities and
  committed-inventory equality remain Task 5 (D19). Fixed the fixtures, which
  previously used ad-hoc ids invalid for the ledger weaver.
- verifyLedger enforces every record's woven_at exactly equals the header's
  single canonical timestamp (edges, status, trailer); a mismatch fails closed.
Tests added: wrong-inventory-id close rejection (missing + extra), and
independently-signed adversarial fixtures for wrong woven_at and wrong ids.

npm test green; confined to clotho/; zero deps.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X8pTab2QMpfM2KsZmwkeYg
@dsmcewan @claude
Clotho Task 3 (revision 3): verifier trust-freeze + exact header/repo…
f7eb391 
…sitory_ref shape

Addresses the round-3 required-seat blockers (claude, codex, grok) on PR #115:
- verifyLedger freezes ALL trust-conferring state at the first failing line:
  edgeHashes, weaverEdgeIds, and manifest acceptance are guarded by !trustBroken;
  a manifest from a non-final trailer (a record after the trailer) is not
  trusted (nulled when any error occurs).
- verifyLedger enforces the exact header envelope and fields (no extra top-level
  or nested fields), validates repository_ref as git-root:<40-hex> (even in
  edge-free ledgers), and requires pub_key to parse as an Ed25519 SPKI key. The
  trailer top-level envelope is shape-checked exactly.
- createLedger validates an injected repositoryRef shape.
Tests: injected bad repositoryRef, tampered-header (extra field, bad
repository_ref) verification failures, and non-final-trailer manifest-not-trusted.

npm test green; confined to clotho/; zero deps.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X8pTab2QMpfM2KsZmwkeYg
@dsmcewan @claude
Clotho Task 3 (revision 4): streaming verifyLedger + full adversarial…
0c50d5d 
… test surface

Addresses the round-4 required-seat blockers (claude, codex, grok):
- verifyLedger now consumes the ledger via fs.createReadStream (or an injected
  stream) with an incremental line splitter — no whole-file readFileSync. It
  never throws on malformed input (returns {ok:false}): null/primitive/array
  records, non-canonicalizable payloads, and an unterminated final line are all
  reported, never trusted. All prior trust-boundary/shape checks preserved.
- Tests completed to the frozen checklist: duplicate header, duplicate trailer,
  middle-line removal, null record, altered signature, altered record_hash,
  signed unknown edge_kind, and an incremental injected-stream verify proof.

npm test green; confined to clotho/; zero deps.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X8pTab2QMpfM2KsZmwkeYg
@dsmcewan @claude
Clotho Task 3 (revision 5): close() idempotency + appendEdge id-misma…
31682c1 
…tch proof

Round-5 required-seat blockers (claude, codex):
- close() is now idempotent ONLY after a successful close: a second/subsequent
  close returns the stored trailer record and writes nothing further; close after
  an aborted/failed (poisoned) ledger still throws.
- appendEdge's re-derivation of endpoint ids (via validateEdgeInput/deriveNodeId,
  rejecting mismatches per Ruling A) is made explicit and proven through a unit:
  a caller from_node mismatching the locator-derived id is rejected.

npm test green; confined to clotho/; zero deps.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X8pTab2QMpfM2KsZmwkeYg
@dsmcewan @claude
Clotho Task 3 (revision 6): canonical pub_key + strict file-ref path …
3acd6ce 
…validation

Round-6 required-seat blockers (claude, codex):
- verifyLedger rejects a header pub_key that parses but is not canonical SPKI
  base64 (re-export must equal the input), closing base64 malleability.
- coverage content-reference validation now delegates to
  registry.validateSourceRef: a file: content address must carry a canonical
  POSIX repository-relative path (no absolute/traversal/backslash/NUL) and a
  lowercase 40-hex blob sha. Applied uniformly at close and verification.
Tests: non-canonical pub_key rejection; absolute-path and traversal file-ref
rejections.

npm test green; confined to clotho/; zero deps.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X8pTab2QMpfM2KsZmwkeYg
@dsmcewan @claude
Clotho Task 3 (revision 7): canonical signatures + status back-ref sc…
c8a8d6d 
…ope fixtures

Round-7 required-seat blockers (claude, codex):
- verifyLedger validates every record signature as canonical standard base64 of
  exactly 64 bytes (Ed25519) before verification, rejecting non-canonical
  encodings (embedded whitespace) that Buffer.from would otherwise accept.
- Signed fixtures prove the status back-reference scope is closed to edges only:
  a status record referencing another status record's hash is rejected at verify.

npm test green; confined to clotho/; zero deps.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X8pTab2QMpfM2KsZmwkeYg
@dsmcewan @claude
Clotho Task 3 (revision 8): close-order/D22, all-or-error writes, byt…
42e09f3 
…e-exact verify

Round-8 required-seat blockers (claude, codex):
- close() now writes the trailer, flushes, and closes the descriptor BEFORE
  committing state.closed/closeResult — a descriptor-close failure leaves the
  ledger permanently poisoned (later close/append throw; never a cached success).
- default file writes are all-or-error: the complete LF-terminated line is
  written byte-by-byte (short write is fatal) then fsync'd.
- verifyLedger and readEdges consume raw bytes, split on the LF byte, strict-
  decode UTF-8 (invalid UTF-8 rejected, no replacement), and hash the exact prior
  line bytes for the chain — no whole-file/string re-encoding.
Tests added: descriptor-close-failure poisoning, close-after-abort, the full
depends-on endpoint matrix + a wrong-endpoint rejection, more D24 rejections
(non-integer/unsafe count, extra field, duplicate, empty impl_refs, malformed
inventories_consumed), misplaced header, tail-record+trailer removal, invalid
UTF-8 line, and a timeout-race on the gated incremental-read proof.

npm test green; confined to clotho/; zero deps.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X8pTab2QMpfM2KsZmwkeYg
@dsmcewan @claude
Clotho Task 3 (revision 9): type-strict header validation
93c652f 
Round-9 sole remaining required-seat blocker (codex): header validation applied
HEX40/REPO_REF regexes without first requiring primitive strings, so a non-string
repo_head/repository_ref (e.g. an array) could be coerced by RegExp.test and slip
through. createLedger and verifyLedger now require typeof === "string" for
pub_key, repo_head, repository_ref, and woven_at before any regex/parse.
Tests: non-string repoHead/repositoryRef rejected at createLedger; non-string
header fields rejected at verify.

npm test green; confined to clotho/; zero deps.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X8pTab2QMpfM2KsZmwkeYg
@dsmcewan @claude
Clotho Task 3 (revision 10): line-level CR, wovenAt validation, froze…
082347e 
…n test matrix

Round-10 blockers (claude, codex):
- verifyLedger detects CR at the LINE level (attributed to the exact failing
  line, truncating trust there) and routes all stream-level defects (missing
  final LF, empty, missing trailer) through the same fail() channel; documented
  records semantics for tail defects.
- createLedger validates wovenAt (invalid dates rejected with a stable error
  before any descriptor is opened).
- Completed the frozen unit/exit matrix in test-ledger.mjs: independently-signed
  D24 rejection fixtures for every rule (missing/extra id, extra field, unsorted,
  duplicate, negative/non-integer/unsafe count, skipped-nonzero, failed state,
  empty impl/orchestrator refs); locator repository_ref disagreement; assertion-
  status/producer coupling; status_of non-edge rejection; no-coverage close;
  no-header ledger; non-canonical line; record-precedes-header (records empty);
  records-content on tail defects; distinct generated keys; parent-dir creation;
  appendStatus-after-close; to_node mismatch; and descriptor closure across the
  header/edge/trailer write stages.

npm test green; confined to clotho/; zero deps.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X8pTab2QMpfM2KsZmwkeYg
@dsmcewan @claude
Clotho Task 3 (revision 11): strict own-enumerable schema validation
408cb8d 
Round-11 sole remaining required-seat blocker (codex): the local closed-schema
validators used `in`/key-count checks. Added requireExactOwnKeys (rejects own
symbols, own non-enumerable required fields, extra fields, and prototype-
pollution inherited-enumerable fields — matching registry.requireExactKeys) and
applied it uniformly to the coverage object, each weaver entry, each
inspected_source_counts entry, each inventories_consumed entry, the status input,
and the trailer envelope. Tests: prototype-pollution, non-enumerable-required,
and symbol-key rejections at close/appendStatus.

npm test green; confined to clotho/; zero deps.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X8pTab2QMpfM2KsZmwkeYg
@dsmcewan @claude
Clotho Task 3 (revision 12): version is any integer label (not safe-i…
59478b8 
…nteger)

Round-12 sole remaining required-seat blocker (codex): validateCoverage narrowed
the frozen "integer version label" to Number.isSafeInteger, rejecting integers
beyond 2**53. Changed to Number.isInteger (the count's safe-integer restriction
stays with counts, D24). Boundary tests: version 2**53 accepted; 1.5 rejected.

npm test green; confined to clotho/; zero deps.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X8pTab2QMpfM2KsZmwkeYg
@dsmcewan @claude
Clotho Task 3 (revision 13): reject non-private signKey; frozen prope…
8eda602 
…rty-absent matrix

Folds BOTH of codex's round-13 findings plus an exhaustive follow-up pass (which
found nothing further) into one revision:

1. createLedger now rejects a non-private Ed25519 key. asymmetricKeyType names
   the algorithm, not the role: a PUBLIC Ed25519 KeyObject also reports
   "ed25519". Require key.type === "private" as well, BEFORE deriving the public
   key, making directories, opening the path, or writing the header — so a
   non-private key fails closed at creation instead of at the first sign().

2. Frozen unit matrix: add property-ABSENT implementation_refs and
   orchestrator_refs cases at BOTH close-time and independently-signed
   verify-time (previously only malformed/empty arrays were covered). Add a
   regression proving createLedger rejects a public KeyObject immediately
   without touching the injected openFile or creating the file, with a private
   KeyObject positive control.

npm test green; confined to clotho/; zero deps.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X8pTab2QMpfM2KsZmwkeYg
@dsmcewan
dsmcewan marked this pull request as ready for review July 16, 2026 21:54
@dsmcewan
dsmcewan merged commit f619217 into main Jul 16, 2026
17 checks passed
@dsmcewan dsmcewan mentioned this pull request Jul 16, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@dsmcewan