Skip to content

feat(rfc49): catalog-sampling rewire + ciphertext strip (WS-A..E)#1196

Closed
branarakic wants to merge 1 commit into
feat/rfc49-public-projectionfrom
feat/rfc49-catalog-sampling-strip
Closed

feat(rfc49): catalog-sampling rewire + ciphertext strip (WS-A..E)#1196
branarakic wants to merge 1 commit into
feat/rfc49-public-projectionfrom
feat/rfc49-catalog-sampling-strip

Conversation

@branarakic

Copy link
Copy Markdown
Contributor

OT-RFC-49 — catalog-sampling rewire + ciphertext strip

"Hosting follows access": for curated/private CGs, cores stop proving the private ciphertext chunk and instead prove only the public _catalog leaves. Cores hold zero private bytes; private data stays member-held. Stacked on feat/rfc49-public-projection (the SWM-half / catalog-model work is in its own PRs; this PR is only the contract+strip half).

Testnet; backcompat waived (clean break — legacy curated KCs re-publish).

Contract change (one atomic redeploy — traps 1+2 must ship together)

  • RandomSamplingLib.Challenge snapshots (challengeLeafCount, challengeRoot) at issuance; submitProof verifies the pinned pair and deletes the 4 live reads → kills the proof-race (an update mid-period no longer fails an honest prover).
  • DKGKnowledgeAssets gains catalogRoots/catalogLeafCounts + getters/setter/event; the ciphertext maps are deprecated in place (slot-preserving — avoids the "legacy ciphertext root masquerades as catalog root" footgun and never shifts _authorKaNumberHighWater).
  • KnowledgeAssetsLifecycle PublishParams/UpdateParams ciphertext→catalog (same byte widths); errors renamed; ACK_DIGEST_VERSION prefixed onto the publish/update ACK preimage (Trap 3 — raw abi.encodePacked, not the EIP-712 domain).
  • RandomSampling curated draw + proof target swapped ciphertext→catalog.
  • RandomSamplingStorage clearOutstandingChallenges migration fn + version bump.

Off-chain (must byte-agree with the contract verify)

  • computeCatalogRoot (core/crypto) — single source of truth, a catalog-leaves-only tree (Trap 2 interleave); ack.ts version + catalog members.
  • Prover proves the catalog leaves read from the served _catalog; ciphertext-chunk extractor removed.
  • Publisher (WS-D) computes the catalog commitment, ships plaintext catalog, inverts the storage-ACK "must be encrypted" gate → "must carry+verify catalogRoot"; byteSize = catalog footprint. A shared catalogCommittedLeaves filter (excludes the post-publish committedRoot stamp) is used by both the producer and the prover so they cannot drift.
  • Agent (WS-A) swmHostMode.stripCiphertext flag (default ON): cores decline all private-ciphertext custody for curated CGs, retire the serve responders, gossip-off private fan-out; CLI plumbing.
  • Agent publishFromSharedMemory auto-injects the _catalog for curated CGs so the raw /api/shared-memory from-SWM publish shortcut works too (idempotent, before the author seal).

The load-bearing invariant

The catalog leaf-hash + tree shape are byte-identical at publisher / contract / prover, and catalogLeafCount is the post-dedupe V10MerkleTree.leafCount.

Validation

  • 721 evm tests pass (2 pre-existing, unrelated EpochStorage/ShardingTable version asserts).
  • ACK contract-recovers-signer e2e green (the authoritative ACK-digest gate).
  • Parity e2e: real curated publish → rebuilt catalog root == on-chain getCatalogRoot, committedRoot correctly excluded.
  • 105/105 swm tests, 44/44 core crypto, abi-pinning re-pinned.
  • Devnet (4 cores + 2 edges) via scripts/devnet-test-rfc49-catalog-sampling.sh: stripped cores hold zero ciphertext and prove the _catalog; a strip-OFF baseline core holds the ciphertext (non-vacuousness discriminator); finalize AND from-SWM publish paths both green; member edges hold the private data.

Not in this PR (out-of-band / operational)

  • Manual atomic redeploy of the 4 contracts + RandomSamplingStorage at an epoch boundary, running clearOutstandingChallenges as part of the cut.
  • Internal authorization review (this changes what is proven, not who attests — not an external audit).
  • Cross-rotation chaos soak + DMaaST/HOLOS partner re-publish before flipping the strip in production.

🤖 Generated with Claude Code

OT-RFC-49 "hosting follows access": curated/private CGs stop having cores
prove the private ciphertext chunk and instead prove ONLY the public `_catalog`
leaves. Cores hold zero private bytes; private data stays member-held.

Contract (atomic redeploy — traps 1+2 ship together):
- RandomSamplingLib.Challenge: snapshot (challengeLeafCount, challengeRoot) at
  issuance; submitProof verifies the PINNED pair and deletes the 4 live reads
  (kills the proof-race).
- DKGKnowledgeAssets: catalogRoots/catalogLeafCounts + getters/setter/event;
  ciphertext maps deprecated-in-place (slot-preserving, avoids the masquerade
  footgun).
- KnowledgeAssetsLifecycle: PublishParams/UpdateParams ciphertext->catalog;
  errors renamed; ACK_DIGEST_VERSION prefixed onto the publish/update ACK
  preimage (Trap 3, raw abi.encodePacked not EIP-712).
- RandomSampling: curated draw + proof target ciphertext->catalog; stale
  exclusion comments corrected.
- RandomSamplingStorage: clearOutstandingChallenges migration fn + version bump.

Off-chain (must byte-agree with the contract verify):
- core/crypto: computeCatalogRoot (single source of truth) over catalog leaves
  only (Trap 2 interleave); ack.ts ACK_DIGEST_VERSION + catalog members.
- random-sampling: prover proves the catalog leaves read from the served
  `_catalog`; catalog-extractor; ciphertext-chunk extractor removed.
- publisher (WS-D): computes the catalog commitment, ships PLAINTEXT catalog,
  inverts the storage-ACK "must be encrypted" gate to "must carry+verify
  catalogRoot"; byteSize = catalog footprint; shared catalogCommittedLeaves
  filter (excludes the post-publish committedRoot stamp) used by BOTH producer
  and prover so they cannot drift.
- agent (WS-A): swmHostMode.stripCiphertext flag (default ON) — cores decline
  ALL private-ciphertext custody for curated CGs, retire the serve responders,
  gossip-off private fan-out; CLI plumbing.
- agent: publishFromSharedMemory auto-injects the `_catalog` for curated CGs so
  the raw from-SWM publish shortcut works too (idempotent, before the seal).

Validated: 721 evm tests; ACK contract-recovers-signer e2e; parity e2e
(rebuilt root == on-chain catalogRoot, committedRoot excluded); 105/105 swm +
44/44 core crypto. Devnet (4 cores + 2 edges): stripped cores hold ZERO
ciphertext + prove the catalog; finalize AND from-SWM paths both green; a
strip-OFF baseline core holds the ciphertext (discriminator).

Testnet, backcompat waived (clean break). Remaining is out-of-band: manual
atomic redeploy + clear-challenges at an epoch boundary, internal authorization
review, cross-rotation soak, DMaaST/HOLOS partner re-publish.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex review skipped: filtered diff is 8366 lines (cap: 5,000). Please consider splitting this into smaller PRs for reviewability.

@branarakic

Copy link
Copy Markdown
Contributor Author

Closing as redundant — the RFC-49 catalog-sampling strip (WS-A..E) landed on main via #1203 (integration/rfc49-full). Verified: the strip's contract symbols (catalogRoot/catalogLeafCount/challengeRoot) are present in main's contracts, and main is now ahead of this branch (KnowledgeAssetsLifecycle +81 / RandomSampling +35 lines — main also carries the v10.2.0 PoS content-binding fix #1213 that this branch predates). No unique unmerged code; the remaining strip work (atomic redeploy + clear-challenges, internal auth review, soak, partner re-publish) is out-of-band deploy-time, not PR code. Reopen if I've missed something.

@branarakic branarakic closed this Jun 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant