feat(core,publisher): public catalog model + publish partition by branarakic · Pull Request #1171 · OriginTrail/dkg

branarakic · 2026-06-14T23:27:18Z

Part 2/4 · base: feat/storage-graph-set-index.

The public catalog data model + publish-time partition for private context graphs.

core — the DCAT dataset-record model for a private CG's public catalog entry (contextGraphCatalogUri, genesis, constants). Only allowed catalog classes are publicized — an arbitrary rdf:type on the catalog subject stays private.
publisher — partitions the public catalog out of the private ciphertext; writes it on the confirmed publish path (a failed publish exposes nothing) and replaces (not appends) on re-publish so stale metadata can't accumulate. Ownership-key derivation is registration-based (no ambiguous last-slash split).

publisher tests green. Please do not merge before review / out of order.

🤖 Generated with Claude Code

Depends on #1170 — please merge that first.

github-actions · 2026-06-14T23:29:32Z

      } UNION {
        GRAPH <${ontologyGraph}> { <${cgData}> <${DKG_ONTOLOGY.DKG_ACCESS_POLICY}> ?policy }
+      } UNION {
+        GRAPH <${agentsGraph}> { <${cgData}> <${DKG_ONTOLOGY.DKG_ALLOWED_AGENT}> ?agent }


🔴 Bug: This makes outbound sender-key resolution honor AGENTS-graph ACLs, but the receive path still ignores that source in production. SharedMemoryHandler only consults _meta unless a contextGraphMetaOracle is supplied, and DKGAgent.getOrCreateSharedMemoryHandler() does not pass one. An AGENTS-only private CG will therefore encrypt outbound writes while inbound gossip is still authorized as if the graph were open. Wire the same metadata source into SharedMemoryHandler before reading AGENTS here, or keep both paths on _meta until they move together.

github-actions · 2026-06-14T23:29:32Z

   */
  private async getContextGraphAllowedPeers(contextGraphId: string): Promise<string[] | null> {
+    const projected = await this.contextGraphMetaOracle?.(contextGraphId);
+    if (projected) {


🔴 Bug: projected is treated as a complete snapshot here, but ContextGraphMetaOracleRecord is intentionally partial. If a caller only populates accessPolicy or agent fields, this returns null and verifyHostModeEnvelopeAuthority() skips the peer allowlist entirely; the same pattern below can also fall back to stale store allowlists when only revokedAgents is projected. Only short-circuit when the specific field is present, otherwise fall back to the store for that field.

github-actions · 2026-06-14T23:29:32Z

+      // the context-graph DID. B4: CLEAR/REPLACE — purge any prior catalog
+      // entry for these subjects in this graph before inserting the refreshed
+      // one, so repeated publishes don't accumulate stale catalog triples.
+      const catalogGraph = contextGraphCatalogUri(contextGraphId);


🔴 Bug: This introduces _catalog as a new durable graph, but the existing context-graph lifecycle cleanup still only knows about _meta, _private, and _shared_memory*. Dropping a context graph will leave its public catalog graph behind, exposing stale discovery metadata after deletion. Please update the graph-management cleanup paths in the same change that writes this graph.

github-actions · 2026-06-15T01:05:11Z

+export function partitionCatalogQuads<Q extends CatalogQuadLike>(quads: readonly Q[]): CatalogPartition<Q> {
+  const catalogSubjects = new Set<string>();
+  for (const q of quads) {
+    if (q.predicate === DKG_ONTOLOGY.RDF_TYPE && q.object === DKG_ONTOLOGY.DKG_PRIVATE_CONTEXT_GRAPH) {


🔴 Bug: This trusts a user-supplied rdf:type dkg:PrivateContextGraph to identify catalog subjects. validatePublishRequest() does not reserve that subject or predicate, so an ordinary entity can be authored with this type plus dct:* quads and those triples will be routed into plaintext _catalog and removed from the encrypted payload. The split needs to be anchored to the canonical CG subject (or to system-generated catalog quads), not inferred from publish data alone.

github-actions · 2026-06-15T01:05:11Z

      } UNION {
        GRAPH <${ontologyGraph}> { <${cgData}> <${DKG_ONTOLOGY.DKG_ACCESS_POLICY}> ?policy }
+      } UNION {
+        GRAPH <${agentsGraph}> { <${cgData}> <${DKG_ONTOLOGY.DKG_ALLOWED_AGENT}> ?agent }


🔴 Bug: Reading access-policy / agent-gate declarations from the AGENTS graph here makes the sender switch to sender-key encryption before the receiver can observe the same state. SharedMemoryHandler still derives accessPolicy and agent gates from _meta (or the optional oracle), so a peer that only has the AGENTS declaration will reject the incoming encrypted envelope as "not private or agent-gated". Keep the sender and receiver metadata sources aligned, or extend the receive path to read AGENTS as well.

github-actions · 2026-06-15T01:05:11Z

+    const projected = await this.contextGraphMetaOracle?.(contextGraphId);
+
+    const allowlistProjected =
+      projected?.allowedAgents !== undefined || projected?.participantAgents !== undefined;


🔴 Bug: allowlistProjected flips to true when either projected field is present, but line 1555 then builds the gate only from the projected arrays. If the oracle projected allowedAgents but omitted participantAgents (the partial-record shape this PR explicitly expects), store fallback for participants is skipped and legitimate participant agents are dropped from the writer gate. Resolve allowedAgents and participantAgents independently, the same way revokedAgents is handled.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

github-actions · 2026-06-15T08:05:35Z

+    // DID (round-3 SECURITY). A forged `rdf:type dkg:PrivateContextGraph` on a
+    // user entity no longer routes that entity into the plaintext `_catalog`.
+    const { catalogQuads, otherQuads } = partitionCatalogQuads(allSkolemizedQuads, contextGraphDataUri(contextGraphId));
+    const encryptableNquadsStr = catalogQuads.length === 0


🔴 Bug: otherQuads can be empty here when a curated publish carries only catalog triples. That makes encryptableNquadsStr empty, and the chunked curated path blows up because sliceIntoCiphertextChunks rejects zero-length plaintext. Handle the zero-private-payload case explicitly (skip encryption / keep (ciphertextChunksRoot, ciphertextChunkCount) at zero) instead of calling the encryptor with an empty buffer.

github-actions · 2026-06-15T08:05:35Z

      await this.store.insert(tentativeMeta);
+      // B3: only now that the local publish has persisted do we refresh the
+      // public catalog entry (CLEAR/REPLACE — see persistCatalogEntry).
+      await persistCatalogEntry();


🔴 Bug: this only materializes /_catalog on the publishing node. Replicas finalize the same publish through PublishHandler.confirmPublish(), which never calls persistCatalogEntry, so the dedicated catalog graph will be missing on every non-publisher host. If /_catalog is the public serving surface, move this derivation into the shared confirm/promotion path or mirror it in PublishHandler.

github-actions · 2026-06-15T08:05:35Z

+  private async isContextGraphRegistered(contextGraphId: string): Promise<boolean> {
+    const cgUri = contextGraphDataUri(contextGraphId);
+    const registered = await this.store.query(
+      `ASK { GRAPH ?g {


🔴 Bug: GRAPH ?g lets any triple anywhere prove root registration. User data is allowed to mention <did:dkg:context-graph:...> a dkg:ContextGraph, so a stray data quad can make a sub-graph path look like a registered root and send reconstructSharedMemoryOwnership() back to the wrong ownership key after restart. Restrict this check to the authoritative declaration graphs that actually store CG registration metadata.

…ng (Codex #1171) A curated publish whose every quad is a public catalog entry on the context-graph DID (reachable — that DID namespace is not reserved) partitions to empty otherQuads, leaving encryptableNquadsStr empty, which threw an opaque 'rejects empty plaintext' deep in the chunked encryptor. Guard early, curated-only (public CGs never enter the encrypt branch), with an actionable error: the catalog entry alone cannot satisfy the required ciphertext commitment. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

github-actions

Codex review produced 2 comment(s) but all targeted lines outside the diff and were dropped. Check the workflow logs for details.

…1171, security) isContextGraphRegistered used a cross-graph ASK { GRAPH ?g { <cgUri> a dkg:ContextGraph } }, so a publisher could author that type triple in their own content graph (the rdf:type object isn't a reserved IRI) and SPOOF registration — making reconstructSharedMemoryOwnership treat a sub-graph as a registered root and derive the wrong ownership key after restart. The triple is only ever written into the system ONTOLOGY data graph (public CGs + every publish) or the CG's own _meta graph (curated; createContextGraph defGraph = isCurated ? _meta : ontology). Scope the ASK to exactly those two graphs. Adds a focused security test (legit in either authoritative graph -> true; spoof in any user content graph -> false). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

github-actions · 2026-06-15T20:59:22Z

      } UNION {
        GRAPH <${ontologyGraph}> { <${cgData}> <${DKG_ONTOLOGY.DKG_ACCESS_POLICY}> ?policy }
+      } UNION {
+        GRAPH <${agentsGraph}> { <${cgData}> <${DKG_ONTOLOGY.DKG_ALLOWED_AGENT}> ?agent }


🔴 Bug: this teaches the sender-side recipient resolver to trust AGENTS-graph declarations, but the receiver path still enforces accessPolicy and agent gates from _meta/ontology only (SharedMemoryHandler#getContextGraphAgentGateAddresses / contextGraphHasPrivateAccessPolicy), and DKGAgent does not wire contextGraphMetaOracle by default. A CG declared private only in did:dkg:context-graph:agents will now encrypt outbound SWM while inbound raw gossip is still accepted as non-private. Please update the receiver-side lookup (or wire the oracle everywhere) in the same PR.

github-actions · 2026-06-15T20:59:22Z

+      const catalogGraph = contextGraphCatalogUri(contextGraphId);
+      const catalogSubjects = new Set(catalogQuads.map((q) => q.subject));
+      for (const subject of catalogSubjects) {
+        await this.store.deleteByPattern({ graph: catalogGraph, subject });


🔴 Bug: this destructive clear happens before the replacement insert, with no rollback/transaction. If the insert fails, the previous _catalog entry is lost; on the confirmed branch that exception also bubbles out after the on-chain publish has already succeeded, so callers see a failed publish even though the KA was minted. Make the catalog refresh atomic, or treat catalog persistence as a best-effort side effect that cannot invalidate a successful publish.

branarakic · 2026-06-17T09:16:54Z

Closing as redundant — superseded by #1203 (integration/rfc49-full) + #1201, which landed the RFC-49 SWM/agent work on main.

Verified against current main: the stack tip #1193 has 0 residual (every file it touches is byte-identical in main), and this branch adds nothing not already in main. Any per-file delta is a superseded intermediate — e.g. a relocated contextGraphCatalogUri, or a pre-curator-ack dkg-publisher that main has since advanced past (main carries the confirmBeforeCommit curator-ack gate this branch lacks).

No unique unmerged work. Reopen if I've missed something.

…ng (Codex OriginTrail#1171) A curated publish whose every quad is a public catalog entry on the context-graph DID (reachable — that DID namespace is not reserved) partitions to empty otherQuads, leaving encryptableNquadsStr empty, which threw an opaque 'rejects empty plaintext' deep in the chunked encryptor. Guard early, curated-only (public CGs never enter the encrypt branch), with an actionable error: the catalog entry alone cannot satisfy the required ciphertext commitment. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> (cherry picked from commit 73eb710)

…riginTrail#1171, security) isContextGraphRegistered used a cross-graph ASK { GRAPH ?g { <cgUri> a dkg:ContextGraph } }, so a publisher could author that type triple in their own content graph (the rdf:type object isn't a reserved IRI) and SPOOF registration — making reconstructSharedMemoryOwnership treat a sub-graph as a registered root and derive the wrong ownership key after restart. The triple is only ever written into the system ONTOLOGY data graph (public CGs + every publish) or the CG's own _meta graph (curated; createContextGraph defGraph = isCurated ? _meta : ontology). Scope the ASK to exactly those two graphs. Adds a focused security test (legit in either authoritative graph -> true; spoof in any user content graph -> false). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> (cherry picked from commit 9c350d2)

branarakic mentioned this pull request Jun 14, 2026

feat(agent): public/meta context-graph projection + catalog open-serve #1172

Closed

github-actions Bot reviewed Jun 14, 2026

View reviewed changes

branarakic force-pushed the feat/storage-graph-set-index branch from dfcce84 to 2cd2c1a Compare June 15, 2026 01:02

branarakic force-pushed the feat/public-catalog-model branch from 8a8681a to 1b40501 Compare June 15, 2026 01:02

github-actions Bot reviewed Jun 15, 2026

View reviewed changes

feat(core,publisher): public catalog model + publish-time partition

394a029

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

branarakic force-pushed the feat/storage-graph-set-index branch from 2cd2c1a to c0afc5f Compare June 15, 2026 08:02

branarakic force-pushed the feat/public-catalog-model branch from 1b40501 to 394a029 Compare June 15, 2026 08:02

github-actions Bot reviewed Jun 15, 2026

View reviewed changes

branarakic mentioned this pull request Jun 16, 2026

RFC-49 agent half: catalog model + SWM recovery + curator-leader convergence + strict curator-ack gate #1201

Merged

branarakic closed this Jun 17, 2026

Uh oh!

Conversation

branarakic commented Jun 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot Jun 14, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Jun 14, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Jun 14, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Jun 15, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Jun 15, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Jun 15, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Jun 15, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Jun 15, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Jun 15, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Jun 15, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Jun 15, 2026

Choose a reason for hiding this comment

Uh oh!

branarakic commented Jun 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

branarakic commented Jun 14, 2026 •

edited

Loading