You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Agents currently execute tool calls and actions directly — there is no centralized gatekeeper that evaluates whether an action should proceed, routes high-risk operations for human review, or guarantees that what the human approved is exactly what runs. The agent's reasoning loop has unchecked authority over the action plane.
Context: The WYSIWYE Safety Model
This is the output/action side of the WYSIWYE (What You See Is What You Execute) safety architecture:
Broker (this issue): handles the output/action side. The agent proposes actions → the Broker evaluates them against rules → auto-approves low-risk actions or routes high-risk ones to a human → executes exactly what was approved. The agent never executes directly.
Together, the Airlock and Broker form a complete mediation layer: nothing tainted gets in without sanitization, and nothing consequential goes out without approval.
The key invariant is WYSIWYE: the action representation shown to the human for approval is the exact representation that executes. No hidden transforms, no post-approval rewrites, no string interpolation between the approval UI and the execution path. The serialized action is frozen at approval time and replayed byte-for-byte at execution time.
Proposed Solution
A Broker service that is the sole execution gateway for all agent actions:
Core flow
Agent reasoning loop
→ proposes ActionPlan (one or more tool calls with arguments)
→ Broker receives the ActionPlan
→ Broker evaluates each action against the rule engine
→ Low-risk actions: auto-approved, executed immediately
→ High-risk actions: routed to human via HITL channel
→ Human sees the exact action representation
→ Human approves or denies
→ Broker executes the approved ActionPlan verbatim (WYSIWYE guarantee)
→ Results flow back to the agent
What it enforces
Sole executor: the agent process has no direct path to execute actions. All tool dispatch goes through the Broker. This is a structural guarantee, not a convention.
WYSIWYE integrity: the ActionPlan is serialized and hashed at proposal time. The hash is shown alongside the human-readable representation. At execution time, the Broker verifies the hash before dispatching. Any mutation between approval and execution is a hard failure.
Rule-based auto-approval: a configurable rule engine determines which actions are low-risk enough to auto-approve without human review.
Audit trail: every action — auto-approved or human-reviewed — is logged with the full ActionPlan, the rule that matched, the decision, and the execution result.
ActionPlan structure
@dataclassclassAction:
tool: strarguments: dict[str, Any]
rationale: str# agent's explanation of why this action is needed@dataclassclassActionPlan:
actions: list[Action]
plan_hash: str# SHA-256 of canonical serializationproposed_at: datetimecontext: str# summary of the agent's reasoning state
Rule engine for auto-approval
The rule engine determines the disposition of each action. Rules are evaluated in order; first match wins.
The per-tool permission system (#164) operates at the individual tool-call level within the agent process — it gates whether a tool call is allowed, denied, or requires an ask. The Broker operates at a higher level:
Broker (this issue): "Should this proposed action actually execute, and does a human need to see it first?" — enforced outside the agent process as the execution gateway.
They compose: a tool call passes permission checks first, then the resulting ActionPlan passes through the Broker. A tool can be allowed by permissions but still require_human at the Broker level (e.g., a read-only Kubernetes tool is allowed but the Broker still wants human sign-off on production namespace queries).
Integration points
UI (ui-template#21): the HITL approval UI renders the ActionPlan for human review. The UI must display the action representation and hash, and transmit the approval decision back to the Broker.
Gateway (gateway-template#16): the HITL routing in the gateway carries approval requests from the Broker to the UI and approval decisions back. The gateway does not interpret or transform the ActionPlan — it is an opaque envelope.
Where does the Broker live? Options: sidecar container, embedded in BaseAgent (but architecturally separated from the reasoning loop), standalone service, or gateway extension. The WYSIWYE guarantee is strongest when the Broker is a separate process — the agent literally cannot bypass it.
Rule language: The YAML rule format above is a starting point. Should we adopt a real policy language (OPA/Rego, CEL, Casbin) for more expressive rules, or is a simple allowlist/denylist sufficient for v1? OPA gives us policy-as-code with testing; CEL is lighter weight and embeddable. Simple YAML is easiest to start with but may not scale.
ActionPlan granularity: Should the Broker gate individual tool calls, or batches of related calls (an "action plan")? Batch approval is better UX (one approval for a coherent set of actions) but harder to reason about (what if the human approves 3 of 5 actions?).
WYSIWYE hash verification: SHA-256 of canonical JSON is straightforward, but what is the canonical form? Sorted keys, no whitespace, UTF-8 normalized? Need a spec.
Timeout and failure semantics: If a human doesn't respond to an approval request within N minutes, does the action fail, retry, or escalate? What if the Broker process crashes between approval and execution?
Multi-agent delegation: When an agent delegates to a subagent, does the subagent's Broker inherit the parent's rules, or does it have its own rule set? This interacts with permission scopes from feat: per-tool permission policy (allow | deny | ask) #164.
Problem
Agents currently execute tool calls and actions directly — there is no centralized gatekeeper that evaluates whether an action should proceed, routes high-risk operations for human review, or guarantees that what the human approved is exactly what runs. The agent's reasoning loop has unchecked authority over the action plane.
Context: The WYSIWYE Safety Model
This is the output/action side of the WYSIWYE (What You See Is What You Execute) safety architecture:
Together, the Airlock and Broker form a complete mediation layer: nothing tainted gets in without sanitization, and nothing consequential goes out without approval.
The key invariant is WYSIWYE: the action representation shown to the human for approval is the exact representation that executes. No hidden transforms, no post-approval rewrites, no string interpolation between the approval UI and the execution path. The serialized action is frozen at approval time and replayed byte-for-byte at execution time.
Proposed Solution
A Broker service that is the sole execution gateway for all agent actions:
Core flow
What it enforces
ActionPlan structure
Rule engine for auto-approval
The rule engine determines the disposition of each action. Rules are evaluated in order; first match wins.
Relationship with per-tool permissions (#164)
The per-tool permission system (#164) operates at the individual tool-call level within the agent process — it gates whether a tool call is allowed, denied, or requires an ask. The Broker operates at a higher level:
They compose: a tool call passes permission checks first, then the resulting ActionPlan passes through the Broker. A tool can be
allowed by permissions but stillrequire_humanat the Broker level (e.g., a read-only Kubernetes tool is allowed but the Broker still wants human sign-off on production namespace queries).Integration points
Design Questions (need design session)
Non-Goals
Acceptance Criteria