Skip to content

UCP agentic-commerce policy examples (apps/ucp) - #15

Open
oschaaf wants to merge 9 commits into
mainfrom
experiment/ucp-agentic-commerce-policies
Open

UCP agentic-commerce policy examples (apps/ucp)#15
oschaaf wants to merge 9 commits into
mainfrom
experiment/ucp-agentic-commerce-policies

Conversation

@oschaaf

@oschaaf oschaaf commented Jun 27, 2026

Copy link
Copy Markdown
Member

Experimental — for discussion. An exploratory set of example policies for a seam we think matters; review has already reshaped it substantially (thanks @denisknj) and it's better for it.

What this adds

Eight buyer-side governance policies for AI agents that transact over the Universal Commerce Protocol (UCP) through UCP checkout MCP servers — plus the vendor-neutral ucp app, an agentic-commerce bundle, and a commerce industry landing page.

Policy Enforcement State Purpose
checkout-spend-cap ingress (observes at egress) stateful Deny complete_checkout when the observed grand total exceeds the org cap
merchant-allowlist ingress stateless Deny completion to a non-approved merchant
cumulative-spend-ceiling ingress (observes at egress) stateful Running session spend vs. org budget
approved-cart-integrity ingress (observes at egress) stateful Bind completion to the approved cart, single-use and checkout-scoped (anti prompt-injection)
attribution-disclosure ingress (observes at egress) stateful Routing-disclosure nudge + no-overcharge vs. the advertised price
order-pii-egress-redaction egress stateless Redact buyer PII from get_order/get_checkout responses
high-value-handoff-gate egress stateless Hand off high-value checkouts to a human via continue_url
restricted-category-block ingress stateless Block restricted categories/SKUs where cart content enters (create/update)

Four policies are stateful because the spec forces it: a UCP complete_checkout request carries no cart content (inputs are id, signals?, attribution?, payment?) — totals and line items exist only in the merchant's responses. So each of these policies observes create/update/get responses into policy-accessible session state (keyed by checkout id) and enforces at the completion call, failing closed when the needed observation is absent. All four follow one shared reference pattern: marker discovery over input.context.session.policies (no hardcoded writer ids), one object-valued key per concern merged with prior state, set-semantics reads, and the nested decision := {allow, session_writes} envelope.

The idea (for discussion)

UCP standardizes how a trusted AI platform transacts with merchants, and merchants enforce fraud / payment / PII / scope server-side. What neither covers — and UCP by design leaves to "the platform" — is an enterprise's control over the agents it runs: which merchants/tools they may call, org spend and data-egress limits beyond a single AP2 consent mandate, and per-call Rego on tool args/results. These policies explore that buyer-side egress seam. Complementary to UCP and its implementors, not a competing trust referee.

Verification

  • All eight opa check --strict-clean; every policy ships a tests.yaml run by pnpm test (CI-green, 85 cases repo-wide) with exact-reason assertions captured from real opa evaluation. Session-write expectations ride along as expectedSessionWrites (runner-ignored, asserted end-to-end).
  • Verified end-to-end behind a live gateway against a mock UCP merchant MCP server whose tool signatures mirror the confirmed spec arg shapes (top-level id; completion without cart; cancel_checkout present; status/totals strictly server-side). All eight policies; the composition layer consumes each stateful policy's own decision envelope, so the documented contract is the thing under test. Latest results in the round-3 comment below.
  • PARC-only: no policy authorizes on input.payload.name / input.kind / input.user or stripped claims.
  • manifest.json regenerated; pnpm manifest:check passes.

Honest caveats

  • MCP-path only. When checkout escalates to a browser via continue_url, that session is invisible to the gateway. These are agent-autonomy guardrails on the MCP path, not end-to-end guarantees.
  • The stateful four require a session-state-capable gateway with the policy's writable keys declared in its configuration (each policy's doc states its keys, TTLs, and on_drop), and both hooks bound (observe at tool_post_invoke, enforce at tool_pre_invoke).
  • order-pii-egress-redaction rewrites the structured result; the JSON copy serialized in content[].text is not rewritten — a stated limitation for text-first clients.
  • input.context.mandate.* (caps/allowlists/budget/advertised price) is gateway-supplied policy input, not a UCP field — UCP carries the buyer's constraints only as an AP2 mandate that is unstructured to a checkout policy.
  • attribution-disclosure's disclosure check is a compliance nudge, not enforced provenance — the value is agent-supplied.
  • The approval that approved-cart-integrity binds to is the merchant's ready_for_complete response; whether that transition encodes a human approval is a property of the deployment's flow.

Resolved during review

  • Naming (maintainer's leans): vendor-neutral apps/ucp, industry commerce, bundle agentic-commerce, high-value-approval-gatehigh-value-handoff-gate.
  • tests.yaml is the canonical test format (fixtures converted, JSON removed).
  • Rego indentation standardized on 4 spaces (repo-level opa fmt convention tracked separately).
  • Round-1: fail-open slugified tool-name matching → shape-set convention. Round-2: approval single-use + checkout-scoped; price check → no-overcharge. Round-3: the spec-shape redesign above.

@denisknj

Copy link
Copy Markdown
Contributor

Do you have a UCP test rig setup somewhere and tested these policies in an end-to-end case to make sure they work as advertised?

@oschaaf

oschaaf commented Jun 28, 2026

Copy link
Copy Markdown
Member Author

I tested the policies and verified the specs

Do you have a UCP test rig setup somewhere and tested these policies in an end-to-end case to make sure they work as advertised?

No. These are unit/contract-tested (verified against the real spec), but not end-to-end tested against the real thing. But we can try and go all the way (I think we should) if we want to do this;

If we want to truthfully say "works as advertised end-to-end," a UCP mock (or Shopify's hosted UCP MCP server) behind a real DTwo gateway, with an agent driving create→update→complete_checkout, would allow us to pull that off. I didn't scope it yet, but I think it is very doable.

Eight buyer-side egress governance policies for AI agents that transact
over the Universal Commerce Protocol (UCP) through Shopify's hosted
Cart/Checkout/Order MCP servers. Experimental — shared for discussion,
not a hardened or endorsed set.

Adds the `shopify` app, the `agentic-commerce` bundle, and the `retail`
industry landing page; manifest regenerated (manifest:check passes).
All eight verified green under opa 1.17.1.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Otto van der Schaaf <otto@dtwo.ai>
Findings from driving all eight policies end-to-end behind a live
gateway against a mock UCP MCP server:

- Gateways slugify federated tool names ("ucp-shop-complete-checkout"),
  so suffix-matching the underscored name alone never fires and deny
  policies fail open. All eight now match hyphenated/underscored/
  collapsed shapes anchored at a separator, plus the bare un-prefixed
  name (the apps/slack matching convention). New slugified-name deny
  fixtures lock this in per policy.
- restricted-category-block read line_items/messages from top-level
  args, unlike its siblings; it now reads the canonical args.checkout.*
  shape first, keeping the old shape as a fallback.
- READMEs: order-pii-egress-redaction gains a known limitation
  (structured-result redaction does not rewrite the serialized
  content[].text copy); the two stateful policies replace hedging with
  verified deployment requirements (declared writable session keys,
  matching write attribution); apps/shopify README notes the set is
  exercised end-to-end.

opa check --strict clean on all eight; 26 fixtures green; manifest
regenerated (manifest:check passes).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Otto van der Schaaf <otto@dtwo.ai>
@oschaaf

oschaaf commented Jul 3, 2026

Copy link
Copy Markdown
Member Author

Do you have a UCP test rig setup somewhere and tested these policies in an end-to-end case to make sure they work as advertised?

Followed through on this — there is now a rig, and it earned its keep immediately.

The rig

A mock UCP MCP server (spec-faithful checkout shapes: totals[] with the type == "total" entry, the status state machine, continue_url) registered behind a live DTwo gateway, with an agent-shaped client driving create/update/complete_checkout and get_order/get_checkout through the gateway's authenticated JSON-RPC endpoint, and OPA deciding every call. All eight policies, 21 scenarios — allow, deny (with proof the upstream never executed), redaction, and both stateful flows — three consecutive fully green runs. It lives in our gateway's test tree with a runbook; happy to walk anyone through a run.

What end-to-end testing caught (fixes pushed in 0a6b7ee)

  1. Fail-open on federated tool names. The gateway prefixes and slugifies tool names (complete_checkout arrives as e.g. ucp-shop-complete-checkout), so the underscored suffix match in the original policies never fired on a live gateway — every deny policy failed open. Exactly the class of bug contract tests can't see. All eight now use the same shape-set matching convention as apps/slack/deny-channel-creation (hyphenated/underscored/collapsed, separator-anchored, plus the bare un-prefixed name), and each policy gained a slugified-name deny fixture so this can't regress.
  2. restricted-category-block read a different arg shape (top-level args.line_items) than its seven siblings (args.checkout.*). It now reads the canonical checkout-object shape first and keeps the old one as a fallback.
  3. Redaction scope limitation (documented, not fixed here): order-pii-egress-redaction rewrites the structured tool result, but MCP responses typically also carry the same JSON serialized in content[].text — that copy is not rewritten, so a text-first client can still see raw values. Verified live and now a stated known limitation: deployments must handle the text channel at the gateway, or treat the policy as structured-result-only.

The stateful pair — answer to my own open question

Both stateful policies were proven end-to-end through the live gateway: cumulative-spend-ceiling allowed a first checkout, committed the running total, denied the second when the projected total crossed the budget, and resumed allowing under it; approved-cart-integrity failed closed with no approval, recorded the approved cart, allowed the identical completion, and denied a post-approval gift-card injection. Two deployment requirements made the difference and are now stated in their READMEs (declared writable session keys; consistent write attribution). So my recommendation on the open question: ship both stateful policies in this PR — the earlier "propagation still being wired" hedge is resolved and has been replaced with the verified requirements.

Also verified but worth stating: the org-constraint inputs (input.context.mandate.*, restricted_skus, merchant, advertised_total) are gateway-supplied configuration — a stock gateway doesn't populate them by itself. The READMEs now spell out that contract.

Remaining maintainer calls (would like your take, @denisknj)

  1. Slug: apps/shopify/, or a vendor-neutral home for UCP servers generally?
  2. Industry page: retail vs commerce; bundle name agentic-commerce OK?
  3. Test runner: pre-existing gap — this set ships tests/*.json fixtures but pnpm test wants a tests.yaml per policy (fails identically before and after this PR's changes). Convert the fixtures to tests.yaml in this PR, or track separately?

@oschaaf
oschaaf marked this pull request as ready for review July 3, 2026 11:46
@oschaaf
oschaaf requested a review from a team as a code owner July 3, 2026 11:46
@oschaaf oschaaf changed the title [Experimental, draft] UCP agentic-commerce policy examples (shopify) UCP agentic-commerce policy examples (shopify) Jul 3, 2026
oschaaf and others added 2 commits July 3, 2026 13:52
Converted from the tests/*.json fixtures (same PARC inputs): allow/deny
plus exact-reason and transform assertions in the runner's schema. The
JSON fixtures stay: they carry the session-write expectations the
runner schema cannot express, and the end-to-end run uses them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Otto van der Schaaf <otto@dtwo.ai>
Per review: tests.yaml (run by pnpm test) is now the single test
suite; the parallel tests/*.json fixtures are removed so the two
cannot drift. Case names survive as comments; the session-state
expectations ride along as expectedSessionWrites (ignored by the
runner, asserted by the end-to-end gateway run). READMEs updated to
reference cases instead of fixture files.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Otto van der Schaaf <otto@dtwo.ai>
@oschaaf

oschaaf commented Jul 3, 2026

Copy link
Copy Markdown
Member Author

Maintainer call #3 resolved as convert (per otto): tests.yaml is now the canonical test format for the set — generated from the fixtures with exact-reason assertions, run by pnpm test, CI-green — and the parallel tests/*.json files are removed in 4639dff so the two suites can't drift. Case names survive as # case: comments; the stateful pair's session-write expectations ride along as expectedSessionWrites (ignored by the runner, asserted by the end-to-end gateway run). READMEs now reference cases instead of fixture files.

Still open for a maintainer: the slug (apps/shopify vs vendor-neutral) and retail vs commerce / bundle naming.

@denisknj

denisknj commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

approved-cart-integrity: the approval marker is never cleared (success-flow only)

The observe→enforce binding works on the happy path, but the approval_recorded marker is a write-once latch — nothing ever clears it, on complete_checkout or on abort. The only rules that emit session_writes are gated on is_approval_record (an update_checkoutready_for_complete):

session_writes["approved_line_items"] := baseline if { is_approval_record ... }
session_writes["approval_recorded"]  := true      if { is_approval_record }

On complete_checkout the enforce path only reads — it emits no writes — and there's no cancel_checkout handling (that tool isn't in the matcher at all). So once set, the latch lives until the TTL (~3600s) expires. Three edge cases fall out, none currently tested (suite is allow / deny / deny-slugified / no-approval fail-closed):

  1. Replay / double-completion. After a successful complete_checkout, the baseline is still on file. A second identical complete_checkout in the same session re-matches and is allowed → the approved cart can be charged twice. Approval is never consumed.
  2. Stale approval survives an abort. Buyer approves cart A, checkout is abandoned/cancelled → approval_recorded + approved_line_items remain; nothing resets them.
  3. Content-scoped, not event-scoped. The baseline is a set of {id, item_id, quantity} tuples — not bound to a checkout id, nonce, or approval timestamp. Combined with (2), a later completion in the same session whose line items happen to match A passes without a fresh human approval — the exact "consent and charge separated in time" gap this policy claims to close. The latch only protects the first completion of a session; after any approval, every subsequent completion sees "approval on file."

The intended mitigation appears to be that a new ready_for_complete overwrites approved_line_items (last-write-wins), so the happy path re-approves per checkout. But nothing enforces that a completion was preceded by a fresh approval of that checkout.

Suggested hardening (make approval single-use and checkout-scoped):

  • Consume on complete: on a matched complete_checkout, emit a session_writes that clears approval_recorded/baseline so a replay fails closed.
  • Bind to the checkout: record checkout.id alongside the line items and require the completion's checkout.id to match, so a stale approval can't authorize a different checkout.
  • Add tests: replay-after-complete → deny; complete-after-cancel with matching contents → deny.

@denisknj

denisknj commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

attribution-disclosure: price-equivalence — deny any divergence, or only an overcharge?

The price check is currently strict equality, so a submitted total that's lower than advertised is denied just as hard as an overcharge:

price_matches if {
	is_number(advertised_total)
	grand_total_amount == advertised_total
}

Two readings, and they genuinely diverge — @oschaaf, this is your call:

A) "Buyer must not be overcharged" → deny only submitted > advertised.
The harm is paying more than you were shown. A lower submitted total is usually benign — a coupon/promo applied after display, or a legitimate merchant reprice-down — and strict == produces false denials on all of those, blocking a valid purchase. This would be grand_total_amount > advertised_total denies, <= allows.

B) "Checkout must not silently drift from what was displayed" → keep ==.
A tamper-evidence / anti-injection stance: any deviation means the cart mutated between display and submit, and a downward change is still a signal. Deny all drift on principle. If we go this way, I'd suggest reframing from "price equivalence" toward "price/tamper integrity" and documenting that any legitimate discount must be reflected in the gateway-supplied advertised_total or it'll be denied.

For what it's worth (and it's your call), the overcharge (A) use case is the more interesting one to me: this policy already sits next to approved-cart-integrity (binds cart content, so an item swap is caught there regardless of price direction) and checkout-spend-cap (bounds the upper amount). Given those companions, the non-overlapping guarantee this one could own is "you weren't charged more than you were shown" — and strict equality mostly just adds false denials on legitimate discounts.

Either way the comparison is exact, since both are integer minor units.

@denisknj

denisknj commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

cumulative-spend-ceiling: no way to reset the running total — it's a per-session sliding ceiling, not a per-day budget

Digging into how the running total ages out (against the d2 session-state design ADR, docs/adr/plan-policy-accessible-session-state.md), the reset semantics aren't what the name "budget" suggests:

  • Per-key TTLs are sliding — refreshed on every write (ADR lines 1563–1573), not absolute. This policy writes running_total on every allowed complete_checkout, so each purchase slides the TTL forward.
  • Result: for an actively-buying agent the running total never resets on its own. It only clears on (a) inactivity longer than the TTL, (b) real re-authentication (claim-fingerprint drift), or (c) refresh-token expiry (days–weeks on Auth0) — none of which is a calendar boundary.

So this cannot express a per-day (or any fixed-window) limit, and TTL tuning can't fix it. Concretely, a "$10k/day" intent with an agent spending $6k/day fails the wrong way: day 1 writes 6000 (TTL slides +24h), day 2 adds 600012000 > 10000blocked on day 2, when you wanted a fresh $10k.

Recommendation — keep the policy, but scope it honestly and plan the follow-up:

  1. Keep it. It's a good example and a genuinely useful control as a per-session cumulative spend ceiling — it just isn't a time-windowed budget. I'd only adjust the docs so a reader doesn't mistake session_budget for a daily/periodic budget (call out the sliding-TTL reset semantics explicitly).
  2. Add the missing capability. A per-day / per-timeframe spend limit would be a very useful policy, and the clean way to build it is a trusted, gateway-supplied UTC clock at input.context.now, so a policy can key state by time bucket (running_total_<YYYY-MM-DD>) — a new day reads a fresh 0 and yesterday's bucket ages out. Filed as dtwoai/d2#1309. Once that lands, a daily-spend-ceiling variant becomes straightforward (and the bucketing helper would live in the shared MCP Rego lib, dtwoai/d2#1308).

No change required to merge this PR — flagging it as a scope clarification plus a tracked follow-up.

@denisknj

denisknj commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Naming: high-value-approval-gate → consider handoff (or escalation)

The name tripped me up while reading it, and I think it's worth a rename — @oschaaf, your call. "approval-gate" implies the policy verifies or records a human approval, but (as discussed elsewhere in this PR) it doesn't: it detects a high-value checkout and hands the workflow off from the agent to a human via the continue_url. Good policy — the name just overclaims what it does.

Two options, both strict improvements over "approval":

  • high-value-handoff-gate — describes what actually happens (agent → human handoff). My preference: routing high-value items into a human workflow is normal, day-to-day business, and "handoff" carries that neutral, business-as-usual tone.
  • high-value-escalation-gate — aligns with the UCP vocabulary the policy's own reason strings already cite (requires_escalation / requires_buyer_review). Also fine — I lean away from it only because "escalation" reads as something-went-wrong, whereas this is really just standard human-in-the-loop routing.

Genuinely happy with either; flagging the preference, not blocking on it.

If you rename, worth doing it across all four surfaces so they don't drift:

  • directory slug (high-value-approval-gatehigh-value-<x>-gate)
  • Rego package (shopify.egress.high_value_approval_gate…high_value_<x>_gate) — changes the body, so policyChecksum regenerates via pnpm manifest
  • frontmatter name: (currently "Require Human Approval Above a Purchase Threshold" → e.g. "Hand Off High-Value Checkouts to a Human")
  • the apps/shopify/README.md row plus the agentic-commerce bundle and retail industry landing-page links

Cheap to do now while it's a draft.

…overcharge

Per review:

- approved-cart-integrity: the approval was a write-once, content-scoped
  latch — replayable, and a stale approval could authorize a later
  same-content completion. The approval now records the checkout id and
  denies completion of any other checkout, and an allowed completion
  consumes the approval (session write clearing the marker), so a
  replay fails closed. No cancel handling: no cancellation tool exists
  in the confirmed UCP surface; id binding plus TTL covers abandonment.
  New cases: deny-replay, deny-checkout-mismatch; allow now asserts the
  consume writes.

- attribution-disclosure: strict price equality relaxed to no-overcharge
  (submitted <= advertised allows) — lower totals are coupons and
  reprice-downs, content tampering is approved-cart-integrity's job,
  and the ceiling is checkout-spend-cap's. New case: allow-discount.
  Docs reframed; strict any-drift noted as a one-comparison variant.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Otto van der Schaaf <otto@dtwo.ai>
@oschaaf

oschaaf commented Jul 3, 2026

Copy link
Copy Markdown
Member Author

approved-cart-integrity: the approval marker is never cleared (success-flow only)

Conceded on all three edges — the latch was write-once and content-scoped, and your edge 3 is the policy's own threat narrative ("consent and charge separated in time") turned against it. Fixed in 6197cc3 with both hardenings:

  • Bound to the checkout: approval now records approved_checkout_id (new session key, added to the writable-keys contract); completion of any other checkout id denies with a reason naming both ids. This also covers the abandonment case without cancel_checkout handling — that tool isn't in the confirmed UCP surface, and a stale approval can no longer authorize a different checkout (a re-created checkout gets a new id); TTL clears the rest. Documented under "Abandoned checkouts".
  • Consumed on complete: an allowed completion emits session writes clearing the marker, baseline, and id, so a replayed complete_checkout fails closed. The consume fires on exactly the allow conditions; observe and consume writes are mutually exclusive.

Both of your asked-for tests are in: deny-replay (consumed approval + matching items → deny) and deny-checkout-mismatch (approval for id X, completion for id Y, identical items → deny), and the allow case now asserts the consume via expectedSessionWrites. One deliberate edge: a flow whose checkouts carry no id at all records and matches "", degrading gracefully to the previous content-scoped behavior rather than breaking id-less servers; any recorded id vs a different or missing submitted id denies.

@oschaaf

oschaaf commented Jul 3, 2026

Copy link
Copy Markdown
Member Author

attribution-disclosure: price-equivalence — deny any divergence, or only an overcharge?

Taking A (no-overcharge) — your companion-policy argument settles it: content tampering is approved-cart-integrity's job (stronger now, after your other finding), the upper bound is checkout-spend-cap's, so the non-overlapping guarantee this policy owns is "the buyer is not charged more than they were shown." Strict equality was mostly a false-denial generator on coupons and reprice-downs.

In 6197cc3: price_matches is now not_overcharged (submitted <= advertised allows, integer minor units, exact), the deny reason states the overcharge with both amounts, docs are reframed accordingly, and a new allow-discount case (4499 submitted vs 4999 advertised) locks the direction in. For deployments that want the tamper-evidence stance, the README notes strict any-drift is a one-comparison change (<===).

Next: refreshing the end-to-end rig with the current policy heads and adding live replay + discount scenarios; results to follow.

@denisknj

denisknj commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

cumulative-spend-ceiling: session_writes emitted at the wrong level — the running total likely never persists

The gateway's session-state contract is a nested envelope: it reads decision.session_writes, and a top-level session_writes is rejected (per the d2 session-state ADRs — docs/adr/policy-writer-attribution.md "the gateway reads decision.session_writes per policy", and docs/adr/plan-policy-accessible-session-state.md "nested envelope only. Top-level session_writes rejected with a clear error").

approved-cart-integrity follows this:

decision := {"allow": allow, "session_writes": session_writes}

but cumulative-spend-ceiling emits only a top-level rule and defines no decision object:

session_writes["running_total"] := projected_total if { is_complete_checkout; allow; grand_total != null }

Under the documented contract that write is dropped → running_total never persists → every checkout is evaluated against prior_total = 0 → the cumulative ceiling silently degrades to a per-single-checkout check (i.e. it fails open across calls; it only ever denies a single checkout that alone exceeds the budget).

This also contradicts the "verified end-to-end" note in the results comment, so it's worth confirming against the exact gateway build you tested: if the running total genuinely accumulated there, then either that build accepts top-level session_writes or the ADR contract has drifted — either way the two stateful policies disagree on the envelope and can't both be right.

Fix: wrap the write in a decision object like approved-cart-integrity, and add an e2e assertion that the second checkout sees the accumulated prior total (not merely that an over-budget call denies — a stateless policy would pass that check too).

@denisknj

denisknj commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

checkout-spend-cap: fails open when a completion carries no grand-total entry

With a cap configured and the currency matching, if checkout.totals has no type == "total" entry, total_amount is undefined → over_cap is false → the call is allowed:

allow if { is_complete_checkout; not over_cap; not currency_mismatch }
over_cap if { is_complete_checkout; is_number(cap); is_number(total_amount); total_amount > cap }
over_cap if { is_complete_checkout; not is_number(cap) }

totals lives in the agent-influenced args, so an agent that omits (or renames) the grand-total entry bypasses the cap entirely. The sibling attribution-disclosure fails closed on exactly this ("cannot-verify → deny"); this policy should match that posture.

Related robustness bug in the same extractor: total_amount := amount if { … type == "total" … } is a single-value rule, so two type == "total" entries with different amounts raise an eval_conflict_error and crash the decision. attribution-disclosure guards this by requiring exactly one.

Fix: deny complete_checkout when a single, well-formed grand total can't be resolved (mirror attribution-disclosure's exactly-one-or-deny handling), and add a test for "completion with no type==total → deny". (This is also the grand-total-extraction divergence that would be solved by a shared helper — dtwoai/d2#1308.)

@denisknj

denisknj commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

attribution-disclosure: the disclosure check is satisfiable by the very agent it governs

attribution_disclosed if { count(attribution) > 0 }

attribution comes from input.payload.args.checkout.attribution — agent-supplied. Since this policy governs your own agents, a misbehaving or compromised agent can put any non-empty value there to pass. So the check enforces "this field is non-empty," not "routing is truthfully disclosed."

That's fine as a compliance nudge, but it's worth stating explicitly in the README so no one reads it as enforced provenance. If real assurance is wanted, the disclosed value would need to be validated against a gateway-supplied trusted source under input.context.*, not taken from the request payload the agent controls.

@denisknj

denisknj commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

The two stateful policies should converge on one pattern

approved-cart-integrity and cumulative-spend-ceiling diverge on two session-state conventions, and future stateful policies will copy whichever they see first — so it's worth picking one reference now:

  1. writer_id resolution. approved-cart-integrity discovers its own namespace by scanning input.context.session.policies for its own marker, so it's correct regardless of the uid the gateway assigns. cumulative-spend-ceiling hardcodes the literal shopify.ingress.cumulative_spend_ceiling and — as its own README admits — silently fails to accumulate if the gateway's attributed writer id differs. The marker-discovery approach is the safer one; recommend cumulative adopt it.

  2. session_writes envelope. Nested decision.session_writes vs top-level — see UCP agentic-commerce policy examples (apps/ucp) #15 (comment). Per the ADR contract only the nested form is honored.

These can't both be idiomatic. Suggest standardizing both on the approved-cart-integrity shape (marker discovery + nested decision) so there's a single stateful-policy reference to copy.

@denisknj

denisknj commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Use consistent indentation across the Rego blocks

order-pii-egress-redaction and restricted-category-block use 4-space indentation, while the other six policies use tabs. Let's standardize on spaces across the set for consistency.

One caveat to settle at the repo level: opa fmt normalizes to tabs by default, so if we standardize on spaces the repo should either not run opa fmt as a gate or configure/skip formatting accordingly — otherwise this will ping-pong. Worth a quick decision on the repo's formatting convention so the whole catalog stays consistent, not just this PR.

@oschaaf

oschaaf commented Jul 3, 2026

Copy link
Copy Markdown
Member Author

Live end-to-end results for the 6197cc3 hardening, as promised — the rig re-vendored the policies at this PR's head and drove the new behavior through a real gateway (mock UCP merchant, real per-call decisions):

Scenario Result
Approve → complete (id-bound) → consume writes committed (approval_recorded=false, baseline and id cleared, confirmed in session storage) → identical replay DENIED fail-closed
Approval recorded for checkout chk_cart_x, completion of chk_cart_y with byte-identical line items DENIED, reason names both ids
Attribution: submitted 3999 < advertised 4999 (discount), disclosed ALLOWED, order minted at the merchant
Attribution: submitted 5999 > advertised 4999 DENIED with the new overcharge reason

Full suite: 23 passed, 1 xfail (the pre-existing text-channel redaction finding, unchanged), three consecutive green runs. Two nice side effects of this round: the policies' native shape-set matching let us delete the tool-name shim from the rig's composition layer (one less gateway-reality workaround), and the deny-reason strings in the live assertions were all taken from evaluation, not transcription.

That closes out both findings end-to-end. Remaining open items are the naming calls (slug, retail vs commerce, bundle) and review itself.

@denisknj

denisknj commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

approved-cart-integrity: confirm the live complete_checkout MCP arg shape — the enforce step may read fields a real call doesn't send

The single-use + checkout-scoping change in 6197cc36 is the right fix, and checkout.id is a real, required UCP field — so no concern there. But the way the enforce step sources the id and line items looks like it may not match a real complete_checkout call:

submitted_checkout_id := object.get(object.get(input.payload.args, "checkout", {}), "id", "")
submitted_line_items  := object.get(object.get(input.payload.args, "checkout", {}), "line_items", [])

This assumes complete_checkout carries a full nested checkout object (with id and line_items) in its arguments. The UCP spec suggests it doesn't:

  • REST binding: complete_checkout's request body is only payment + signals — not the checkout object; the checkout id comes from the URL path (/checkout-sessions/{id}/complete).
  • On update_checkout, the body id is marked deprecated ("id is provided in URL path").

(Refs: https://ucp.dev/2026-04-08/specification/checkout-rest/ , https://github.com/Universal-Commerce-Protocol/ucp-schema)

MCP bindings can differ from REST, so this isn't conclusive — but it's a strong signal that a live complete_checkout MCP tool likely takes something like {checkout_id, payment, signals} rather than a nested checkout with line_items. If so, on a real gateway submitted_checkout_id"" and submitted_line_items[], so:

  • approved_baseline_set == normalized_submitted is <non-empty> == {} → false, and
  • approved_checkout_ids == {submitted_checkout_id}{"chk_…"} == {""} → false,

→ the enforce step denies every completion. Fail-closed, so safe — but the policy would be silently non-functional in production.

This isn't limited to the new id binding: the entire enforce comparison (including the original line-item baseline) rests on complete_checkout delivering the cart in its args.

Why the tests can't catch it: the tests.yaml fixtures fabricate a full checkout object (with line_items, now id) on the complete_checkout call, so CI stays green regardless of the real arg shape — the same "contract tests can't see it, only a live gateway can" class as the slugified-name fail-open fixed in 0a6b7ee. The top-level args.line_items fallback already added to restricted-category-block hints the real shape is genuinely uncertain.

Ask: confirm against the real Shopify UCP complete_checkout / update_checkout MCP tools (not just the mock) whether the checkout id and line items arrive as a nested checkout object or as flat args. If flat, the enforce reads — and the observe baseline capture — need to source checkout_id/line items from wherever the live tool actually puts them. Worth double-checking the mock in the e2e rig mirrors the real tool's arg schema rather than the shape these policies assume.

@denisknj

denisknj commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Is apps/shopify/ the right home, or do we want a vendor-neutral slug for UCP servers generally? - I would have gone with vendor neutral slug since it's an open standard, but I'm fine with shopify if that is your preference.

retail vs commerce for the industry; agentic-commerce bundle name. - I'm fine with either. I like commerce because it's the U in UCP and retail because it seems to currently be more retail focused as opposed to all commerce. Your call - whatever you prefer

Vendor-neutralize the agentic-commerce example set: UCP is an open
standard, so the app slug becomes ucp and packages move from
shopify.ingress.*/shopify.egress.* to ucp.ingress.*/ucp.egress.*. The
retail industry landing page becomes commerce. Rename
high-value-approval-gate to high-value-handoff-gate.

Mechanical and spec fixes on the non-completion-gating policies:

- restricted-category-block: gate content where it enters the cart
  (create_checkout / update_checkout / update_cart) and drop the dead
  complete_checkout leg (completion args carry no line_items on the real
  arg shape).
- high-value-handoff-gate: adopt the shared exactly-one-total idiom so an
  ambiguous totals array can never eval-conflict.
- All rego reindented to 4 spaces.

Remove the per-policy README.md files: the repo convention (see
CONTRIBUTING) is that a policy's human docs live in its policy.md
frontmatter description, and no other app ships per-policy READMEs. Their
content is folded into the policy.md descriptions.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Otto van der Schaaf <otto@dtwo.ai>
oschaaf and others added 2 commits July 3, 2026 18:22
On the confirmed UCP MCP binding, complete_checkout carries only the
checkout id (top level) plus finalization data — no totals, no cart. Any
policy gating completion on cart facts must observe those facts from an
earlier checkout RESPONSE into policy-accessible session state, then
enforce at the completion request keyed on args.id.

Convert the four completion-gating policies to one shared stateful
pattern:

- Observe leg (tool_post_invoke, mode "output"): record what is needed —
  an observed total, currency, or approval baseline — into one
  object-valued key keyed by checkout id, merged with prior state via
  object.union, discovering its own namespace by a unique marker key
  (never a hard-coded writer id). Grand total is resolved only when there
  is exactly one numeric type=="total" entry, else an unresolvable marker
  is stored so completion fails closed.
- Enforce leg (tool_pre_invoke, mode "input"): read the observation for
  args.id and decide, failing closed when nothing was observed.
- Nested decision envelope: decision := {allow, session_writes}, with the
  package-level allow/reason/session_writes rules retained.

Per policy:

- checkout-spend-cap: now stateful; observed total vs mandate cap and
  currency; fail closed on no/ambiguous observation or no cap.
- cumulative-spend-ceiling: marker discovery replaces the hard-coded
  writer id; the running-total advance now rides the decision envelope
  (it was previously emitted top-level only and was non-functional
  standalone); an allowed completion consumes the observed id so a replay
  cannot double-count.
- approved-cart-integrity: approval is sourced from a server
  ready_for_complete RESPONSE (the request-status trigger can never fire
  on the real arg shape); a non-ready response or cancel_checkout
  invalidates; completion allows on a live keyed approval and consumes it;
  a divergent echoed cart is denied as defense in depth. cancel_checkout
  exists on the confirmed surface and is now handled.
- attribution-disclosure: the no-overcharge leg is now stateful (observed
  total vs advertised); disclosure reads the completion's attribution and
  is documented as an agent-supplied compliance nudge, not enforced
  provenance.

Tests rewritten to the real spec arg shapes with observe cases and the
reviewer-requested deny paths (replay-after-complete, cancel-then-complete,
no-observation, duplicate type=="total", second-checkout-accumulated).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Otto van der Schaaf <otto@dtwo.ai>
Reflects the renamed app slug (ucp), industry (commerce), policy
packages, and the high-value-handoff-gate rename, with recomputed
policy checksums.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Otto van der Schaaf <otto@dtwo.ai>
@oschaaf oschaaf changed the title UCP agentic-commerce policy examples (shopify) UCP agentic-commerce policy examples (apps/ucp) Jul 3, 2026
@oschaaf

oschaaf commented Jul 3, 2026

Copy link
Copy Markdown
Member Author

cumulative-spend-ceiling: session_writes emitted at the wrong level — the running total likely never persists

Conceded — and you also correctly called the contradiction with the "verified end-to-end" note. On inspection, the e2e composition layer was reading the package-level session_writes document and building the decision envelope itself, so the live run proved accumulation but never exercised this policy's own envelope. Standalone, under the documented contract, the write was dropped exactly as you said.

Fixed in fabec5b, both sides:

  • cumulative-spend-ceiling now defines the nested decision := {allow, session_writes} envelope (package-level documents retained for composition layers that read those).
  • The e2e composition layer now consumes each stateful policy's own decision.session_writes (and decision.allow) instead of reconstructing the envelope — a policy missing its decision object now fails the suite rather than being masked.

Your asked-for assertion is in the live suite and green: the second checkout's deny reason proves the accumulated prior total was read back — session spend ceiling exceeded: prior=90000 + this=90000 = 180000 > budget=150000, where prior=90000 is the committed running total from the first allowed checkout, additionally asserted as the stored value in session storage (then 120000 after a third, smaller checkout). A stateless policy cannot produce that reason.

While in there the policy also adopted marker discovery (dropping the hardcoded writer-id literal — your convergence point) and gained replay protection: an allowed completion consumes its observation, so completing the same checkout twice cannot double-count.

@oschaaf

oschaaf commented Jul 3, 2026

Copy link
Copy Markdown
Member Author

checkout-spend-cap: fails open when a completion carries no grand-total entry

Both confirmed — and chasing them surfaced something bigger. Checking the extractor against the spec's complete_checkout definition: the completion request carries no cart data at all (inputs are id, signals?, attribution?, payment?; the MCP binding puts id top-level with a checkout argument holding only finalization data). So "completion with no type=="total" entry" isn't an edge case an agent has to construct — it's every spec-shaped completion. A fail-closed deny at completion would have denied everything, while the old code allowed everything.

The fix in fabec5b restructures the policy around where totals actually exist: it now observes the merchant's create/update/get responses — server-computed totals — into session state keyed by checkout id, and enforces at complete_checkout by looking up args.id. No observation on file → deny (your fail-closed posture, now functional, since any real flow reads or updates the checkout through the gateway before completing). The exactly-one-type=="total" guard is applied with set semantics at observe time: a response with duplicate total entries records an explicit unresolvable observation and the completion denies with a reason saying so — no eval_conflict_error possible.

Live coverage: over-cap deny citing the observed amount ($1,390.00 — provably from the earlier response, since the completion call carried no totals), no-observation deny, duplicate-totals deny, currency-mismatch deny, under-cap allow.

@oschaaf

oschaaf commented Jul 3, 2026

Copy link
Copy Markdown
Member Author

attribution-disclosure: the disclosure check is satisfiable by the very agent it governs

Agreed on all points, and the doc now says so in nearly your words (fabec5b): the disclosure value is agent-supplied, so the check enforces "this field is non-empty" — a compliance nudge, not enforced provenance; real assurance would need the disclosed value validated against a gateway-supplied trusted source under input.context.*.

Two related shape fixes in the same commit: disclosure is read from where a spec-shaped completion actually carries it (checkout.attribution, falling back to top-level attribution — it's one of the few things complete_checkout does carry), and the no-overcharge leg became stateful, because the submitted total doesn't exist at completion time (see the spend-cap thread): the comparison is now the observed total for this checkout id vs advertised_total, failing closed when unobserved. The discount/overcharge pair is asserted live.

@oschaaf

oschaaf commented Jul 3, 2026

Copy link
Copy Markdown
Member Author

The two stateful policies should converge on one pattern

Done in fabec5b — and the convergence went further than the original two. After the arg-shape findings (your other thread), four policies are stateful (spend-cap, ceiling, attribution's price leg, cart-integrity), and all four now follow the single reference shape you asked for:

  • marker discovery over input.context.session.policies via a policy-unique key name — no hardcoded writer ids anywhere (unique key names are what keep each policy reading only its own namespace);
  • one object-valued key per concern, keyed by checkout id, merged with prior state at observe time;
  • set-semantics reads, so a degenerate multi-namespace session can only deny, never eval_conflict;
  • the nested decision envelope defined by every stateful policy (and consumed as such by the e2e composition layer — see the envelope thread);
  • fail-closed at completion when the needed observation is absent.

There's now exactly one shape for the next stateful policy to copy.

@oschaaf

oschaaf commented Jul 3, 2026

Copy link
Copy Markdown
Member Author

Use consistent indentation across the Rego blocks

Standardized on 4-space across all eight in fabec5b; opa fmt deliberately not run. Agreed the fmt-vs-spaces tension deserves a repo-level decision so the catalog doesn't ping-pong — either document the spaces convention in CONTRIBUTING and keep opa fmt out of the gates, or adopt tabs repo-wide with an fmt gate. Happy to open a separate issue for that so this PR doesn't decide it implicitly.

@oschaaf

oschaaf commented Jul 3, 2026

Copy link
Copy Markdown
Member Author

approved-cart-integrity: confirm the live complete_checkout MCP arg shape — the enforce step may read fields a real call doesn't send

You called it. Confirmed against the spec (checkout core + MCP binding, tag 2026-04-08), and it's more consequential than the id sourcing:

  • complete_checkout inputs are id, signals?, attribution?, payment? — the MCP tool takes id top-level, with a checkout argument carrying only payment/finalization data. No line_items, no totals, no cart content, ever.
  • update_checkout is top-level id + cart fields — and no status, no totals (server-owned, response-only). So the old observe trigger (args.checkout.status == "ready_for_complete") could never fire on a spec-shaped call either: fail-closed-but-dead in production, exactly the failure class you predicted.
  • One correction, on me: cancel_checkout exists in the MCP binding (meta + id). My earlier "not in the confirmed tool surface" claim was wrong.

The rebuild in fabec5b sources everything from where it actually lives:

  • Approval is observed from responses, not requests: a ready_for_complete response (server-asserted — strictly stronger than trusting an agent-submitted status) records the approved cart and id; any later response for that id showing different items, or a status that left ready_for_complete, invalidates the approval; cancel_checkout clears it (your abort case, now handled properly).
  • Enforcement at completion reads top-level args.id and requires a live approval for exactly that id — single-use consume as before. The line-item comparison is gone from the allow path (it can't exist on the real surface) and survives only as a defense-in-depth deny if a call echoes divergent items. One deliberate simplification to flag: the id-mismatch deny now surfaces the generic "no live approval on file for this checkout" reason (the lookup is id-keyed), not the previous both-ids-naming reason — easy to restore a richer reason if you think deny observability warrants it.
  • The mock was re-shaped to the spec signatures (top-level id, completion without cart, cancel_checkout added, status and totals strictly server-side), so the rig can no longer green-light a policy that reads fields real calls don't send — the same class-closure as the slugified-name fixtures in 0a6b7ee.

Full live re-run against the redesigned set: allow, deny, redaction, and all stateful flows — approve → complete → consume (asserted in session storage) → replay denied; tamper-via-update invalidates → deny; cancel → deny; approval for checkout X does not authorize Y; completion with no prior observation fails closed. Three consecutive fully-green runs (plus an independent fourth), one xfail unchanged (the documented text-channel redaction limitation).

Honest scope note that fell out of this: with approval keyed to the server's ready_for_complete, the control binds completion to "the cart state the merchant last confirmed completable — unchanged and uncanceled since, on the checkout it was confirmed for, once." Whether that transition encodes a human approval is a property of the deployment's flow, and the policy doc now says exactly that instead of implying the policy verifies a human.

@oschaaf

oschaaf commented Jul 3, 2026

Copy link
Copy Markdown
Member Author

I would have gone with vendor neutral slug since it's an open standard, but I'm fine with shopify if that is your preference.

Went with your leans on both (fabec5b):

  • Vendor-neutral it is: the set now lives at apps/ucp/ (packages ucp.ingress.* / ucp.egress.*). You're right that an open standard shouldn't sit under one vendor's name; Shopify stays only where it's genuinely contextual.
  • commerce for the industry — the U in UCP won the argument. Bundle stays agentic-commerce.
  • Also took your high-value-handoff-gate preference from the other thread — renamed across all four surfaces (slug, package, frontmatter name "Hand Off High-Value Checkouts to a Human", landing pages), for your reason: handoff is business-as-usual routing, not something-went-wrong.

And on the per-day-budget scoping comment: docs updated as recommended — the policy is now explicitly a per-session cumulative ceiling with sliding-TTL reset semantics (each write slides the TTL; the total never resets on a calendar boundary), pointing at a gateway-supplied trusted clock as the prerequisite for a daily-spend-ceiling variant. Glad it's tracked; your time-bucket sketch is exactly what the doc points to.

@oschaaf

oschaaf commented Jul 3, 2026

Copy link
Copy Markdown
Member Author

Round 3 is in as fabec5b (three commits: renames + mechanical fixes; the stateful observe/enforce redesign; manifest regen). CI fully green — 85 tests across the catalog, every expected reason captured from real opa evaluation.

The one-paragraph version of the redesign: the spec check you prompted showed a complete_checkout request carries no cart content at all, so every completion-gating policy now observes the merchant's responses (where totals, line items, and status actually live, server-asserted) into session state keyed by checkout id, and enforces at the completion call by args.id, failing closed when the observation is absent. Four stateful policies, one shared pattern; the e2e harness consumes each policy's own decision envelope and its mock now mirrors the spec tool signatures exactly, so both of this round's masking classes (envelope reconstruction, fabricated arg shapes) are structurally closed.

With the naming settled per your calls (apps/ucp, commerce, high-value-handoff-gate), everything raised in this PR is addressed — ready for review from my side, and happy to iterate if anything in the redesign reads off.

@oschaaf
oschaaf requested a review from denisknj July 3, 2026 16:57
…-end run

The redesigned observe/enforce set has been re-verified behind a live
gateway since this note was written; align the status text with the
results reported on the PR.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Otto van der Schaaf <otto@dtwo.ai>
@oschaaf
oschaaf force-pushed the experiment/ucp-agentic-commerce-policies branch 2 times, most recently from 23671e0 to 36f6155 Compare July 16, 2026 06:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants