Skip to content

Commit 2fcdeee

Browse files
harcheharche
authored
Merge pull request #53 from harche/multi-sandbox-support
feat: multi-sandbox support — per-session isolated Sandbox CRDs
2 parents 793866c + ef2c10a commit 2fcdeee

15 files changed

Lines changed: 1248 additions & 487 deletions

‎README.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,12 @@ Demo use-cases (optional):
1515

1616
[![Watch the demo](https://img.youtube.com/vi/W-DyrsGRJeQ/maxresdefault.jpg)](https://www.youtube.com/watch?v=W-DyrsGRJeQ)
1717

18+
## Architecture
19+
20+
![Architecture](docs/arch.png)
21+
22+
Each MCP client session gets its own isolated Sandbox CRD (Kata VM), with per-session gRPC connections, idle timeout cleanup, and max session limits. See [docs/grpc-sandbox-architecture.md](docs/grpc-sandbox-architecture.md) for details.
23+
1824
---
1925

2026
## Table of Contents

‎docs/arch.png‎

214 KB
Loading

‎docs/grpc-sandbox-architecture.md‎

Lines changed: 2 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -55,36 +55,9 @@ This document describes the gRPC-based sandbox execution architecture used in Pr
5555

5656
## Overview
5757

58-
The sandbox system follows a client-server model inspired by Kubernetes' kubelet/containerd architecture:
58+
The sandbox system follows a client-server model with per-session isolation. Each MCP client session gets its own Sandbox CRD, which provisions a dedicated pod and service:
5959

60-
```
61-
+---------------------------------------------------------------------+
62-
| MCP Server |
63-
| +-------------------+ +--------------------------------------+ |
64-
| | searchTools | | runSandbox Tool | |
65-
| | (API discovery) | | (thin gRPC client wrapper) | |
66-
| +-------------------+ +------------------+-------------------+ |
67-
| | |
68-
+----------------------------------------------+-----------------------+
69-
| gRPC over Unix Socket
70-
| unix:///tmp/prodisco-sandbox.sock
71-
v
72-
+---------------------------------------------------------------------+
73-
| Sandbox gRPC Server |
74-
| +---------------------------------------------------------------+ |
75-
| | SandboxService | |
76-
| | (Execute, ExecuteStream, ExecuteAsync, Cancel, List...) | |
77-
| +---------------------------------------------------------------+ |
78-
| | |
79-
| +----------------+ +-------+--------+ +----------------------+ |
80-
| | Executor | | Execution | | CacheManager | |
81-
| | (VM + esbuild | | Registry | | (dedup, persist) | |
82-
| | transform) | | (state, output)| | | |
83-
| +----------------+ +----------------+ +----------------------+ |
84-
| |
85-
| Pre-configured: k8s client, KubeConfig, prometheus-query |
86-
+---------------------------------------------------------------------+
87-
```
60+
![Architecture](arch.png)
8861

8962
---
9063

‎k8s/agent-sandbox-deployment.yaml‎

Lines changed: 39 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -73,7 +73,9 @@ subjects:
7373
name: prodisco-sandbox
7474
namespace: prodisco
7575
---
76-
# Sandbox resource for sandbox-server (using agent-sandbox CRD)
76+
# Static Sandbox resource for single-sandbox mode (SANDBOX_MODE=single).
77+
# In multi-sandbox mode (SANDBOX_MODE=multi), the MCP server creates
78+
# per-session Sandbox CRDs dynamically — remove this resource in that case.
7779
apiVersion: agents.x-k8s.io/v1alpha1
7880
kind: Sandbox
7981
metadata:
@@ -145,7 +147,8 @@ spec:
145147
- name: scripts-cache
146148
emptyDir: {}
147149
---
148-
# Service for sandbox-server (points to pods managed by the Sandbox)
150+
# Static Service for single-sandbox mode.
151+
# In multi-sandbox mode, the Sandbox controller creates a Service per CRD automatically.
149152
apiVersion: v1
150153
kind: Service
151154
metadata:
@@ -183,6 +186,34 @@ subjects:
183186
name: mcp-server
184187
namespace: prodisco
185188
---
189+
# RBAC for MCP server to manage Sandbox CRDs (multi-sandbox mode)
190+
apiVersion: rbac.authorization.k8s.io/v1
191+
kind: ClusterRole
192+
metadata:
193+
name: mcp-server-sandbox-manager
194+
rules:
195+
- apiGroups: ["agents.x-k8s.io"]
196+
resources:
197+
- sandboxes
198+
verbs: ["get", "list", "create", "delete", "watch"]
199+
- apiGroups: ["agents.x-k8s.io"]
200+
resources:
201+
- sandboxes/status
202+
verbs: ["get"]
203+
---
204+
apiVersion: rbac.authorization.k8s.io/v1
205+
kind: ClusterRoleBinding
206+
metadata:
207+
name: mcp-server-sandbox-manager
208+
roleRef:
209+
apiGroup: rbac.authorization.k8s.io
210+
kind: ClusterRole
211+
name: mcp-server-sandbox-manager
212+
subjects:
213+
- kind: ServiceAccount
214+
name: mcp-server
215+
namespace: prodisco
216+
---
186217
# MCP Server as regular Deployment
187218
apiVersion: apps/v1
188219
kind: Deployment
@@ -219,11 +250,12 @@ spec:
219250
value: "3000"
220251
- name: SCRIPTS_CACHE_DIR
221252
value: "/tmp/prodisco-scripts"
222-
# Connect to sandbox-server via TCP
223-
- name: SANDBOX_USE_TCP
224-
value: "true"
225-
- name: SANDBOX_TCP_HOST
226-
value: "sandbox-server.prodisco.svc.cluster.local"
253+
# Multi-sandbox mode: each MCP session gets its own Sandbox CRD
254+
# Set to "single" to use a shared sandbox (requires static Sandbox CRD above)
255+
- name: SANDBOX_MODE
256+
value: "multi"
257+
- name: SANDBOX_NAMESPACE
258+
value: "prodisco"
227259
- name: SANDBOX_TCP_PORT
228260
value: "50051"
229261
resources:

‎k8s/mcp-server.yaml‎

Lines changed: 86 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -4,12 +4,50 @@ kind: Namespace
44
metadata:
55
name: prodisco
66
---
7+
# ConfigMap with ProDisco configuration
8+
apiVersion: v1
9+
kind: ConfigMap
10+
metadata:
11+
name: prodisco-config
12+
namespace: prodisco
13+
data:
14+
# Endpoint configuration (edit these for your cluster)
15+
PROMETHEUS_ENDPOINT: "http://prometheus-server.monitoring.svc.cluster.local:80"
16+
LOKI_ENDPOINT: "http://loki.monitoring.svc.cluster.local:3100"
17+
18+
.prodisco-config.yaml: |
19+
libraries:
20+
- name: "@kubernetes/client-node"
21+
description: "Kubernetes API client"
22+
- name: "@prodisco/prometheus-client"
23+
description: |
24+
Prometheus queries & metric discovery. PROMETHEUS_ENDPOINT env var is pre-configured in the sandbox.
25+
Quick start: `const client = new PrometheusClient({ endpoint: process.env.PROMETHEUS_ENDPOINT });`
26+
Range query: `client.executeRange('node_cpu_seconds_total', { start: new Date(Date.now() - 3600000), end: new Date(), step: '1m' })`
27+
IMPORTANT: TimeRange.start/end must be Date objects or Unix timestamps in SECONDS (not milliseconds). Use `new Date()` or `Math.floor(Date.now()/1000)`.
28+
Workflow: (1) create client with endpoint (2) client.execute('metric_name') for instant, client.executeRange() for time series.
29+
- name: "@prodisco/loki-client"
30+
description: |
31+
Loki LogQL querying. LOKI_ENDPOINT env var is pre-configured. Auto-authenticates in Kubernetes using service account token.
32+
Quick start: `const client = new LokiClient({ baseUrl: process.env.LOKI_ENDPOINT });`
33+
Workflow: (1) create client (2) client.labels() and client.labelValues('namespace') to discover labels (3) client.queryRange('{namespace="default"}') to fetch logs.
34+
- name: "simple-statistics"
35+
description: "Statistical functions including mean, median, standard deviation, linear regression, and more."
36+
---
737
apiVersion: v1
838
kind: ServiceAccount
939
metadata:
1040
name: mcp-server
1141
namespace: prodisco
1242
---
43+
# ServiceAccount for dynamically created sandbox pods
44+
apiVersion: v1
45+
kind: ServiceAccount
46+
metadata:
47+
name: sandbox-server
48+
namespace: prodisco
49+
---
50+
# Read-only cluster access for MCP server and sandbox pods
1351
apiVersion: rbac.authorization.k8s.io/v1
1452
kind: ClusterRole
1553
metadata:
@@ -36,6 +74,37 @@ roleRef:
3674
apiGroup: rbac.authorization.k8s.io
3775
kind: ClusterRole
3876
name: mcp-server
77+
subjects:
78+
- kind: ServiceAccount
79+
name: mcp-server
80+
namespace: prodisco
81+
- kind: ServiceAccount
82+
name: sandbox-server
83+
namespace: prodisco
84+
---
85+
# RBAC for MCP server to manage Sandbox CRDs (multi-sandbox mode)
86+
apiVersion: rbac.authorization.k8s.io/v1
87+
kind: ClusterRole
88+
metadata:
89+
name: mcp-server-sandbox-manager
90+
rules:
91+
- apiGroups: ["agents.x-k8s.io"]
92+
resources:
93+
- sandboxes
94+
verbs: ["get", "list", "create", "delete", "watch"]
95+
- apiGroups: ["agents.x-k8s.io"]
96+
resources:
97+
- sandboxes/status
98+
verbs: ["get"]
99+
---
100+
apiVersion: rbac.authorization.k8s.io/v1
101+
kind: ClusterRoleBinding
102+
metadata:
103+
name: mcp-server-sandbox-manager
104+
roleRef:
105+
apiGroup: rbac.authorization.k8s.io
106+
kind: ClusterRole
107+
name: mcp-server-sandbox-manager
39108
subjects:
40109
- kind: ServiceAccount
41110
name: mcp-server
@@ -78,11 +147,17 @@ spec:
78147
value: "/tmp/prodisco-scripts"
79148
- name: PRODISCO_ENABLE_APPS
80149
value: "true"
81-
# In-cluster service URLs for monitoring clients
82-
- name: PROMETHEUS_ENDPOINT
83-
value: "http://prometheus-server.monitoring.svc.cluster.local:80"
84-
- name: LOKI_ENDPOINT
85-
value: "http://loki.monitoring.svc.cluster.local:3100"
150+
# Multi-sandbox mode: each MCP session gets its own Sandbox CRD
151+
- name: SANDBOX_MODE
152+
value: "multi"
153+
- name: SANDBOX_NAMESPACE
154+
value: "prodisco"
155+
- name: SANDBOX_TCP_PORT
156+
value: "50051"
157+
- name: SANDBOX_IMAGE
158+
value: "prodisco/sandbox-server:test"
159+
- name: PRODISCO_CONFIG_PATH
160+
value: "/config/.prodisco-config.yaml"
86161
resources:
87162
requests:
88163
memory: "256Mi"
@@ -105,9 +180,15 @@ spec:
105180
volumeMounts:
106181
- name: scripts-cache
107182
mountPath: /tmp/prodisco-scripts
183+
- name: prodisco-config
184+
mountPath: /config
185+
readOnly: true
108186
volumes:
109187
- name: scripts-cache
110188
emptyDir: {}
189+
- name: prodisco-config
190+
configMap:
191+
name: prodisco-config
111192
---
112193
apiVersion: v1
113194
kind: Service

‎k8s/openshift-deployment.yaml‎

Lines changed: 40 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -151,6 +151,34 @@ subjects:
151151
name: mcp-server
152152
namespace: prodisco
153153
---
154+
# RBAC for MCP server to manage Sandbox CRDs (multi-sandbox mode)
155+
apiVersion: rbac.authorization.k8s.io/v1
156+
kind: ClusterRole
157+
metadata:
158+
name: prodisco-sandbox-manager
159+
rules:
160+
- apiGroups: ["agents.x-k8s.io"]
161+
resources:
162+
- sandboxes
163+
verbs: ["get", "list", "create", "delete", "watch"]
164+
- apiGroups: ["agents.x-k8s.io"]
165+
resources:
166+
- sandboxes/status
167+
verbs: ["get"]
168+
---
169+
apiVersion: rbac.authorization.k8s.io/v1
170+
kind: ClusterRoleBinding
171+
metadata:
172+
name: prodisco-sandbox-manager
173+
roleRef:
174+
apiGroup: rbac.authorization.k8s.io
175+
kind: ClusterRole
176+
name: prodisco-sandbox-manager
177+
subjects:
178+
- kind: ServiceAccount
179+
name: mcp-server
180+
namespace: prodisco
181+
---
154182
# ClusterRoleBinding for Prometheus/Thanos Querier access (sandbox-server)
155183
apiVersion: rbac.authorization.k8s.io/v1
156184
kind: ClusterRoleBinding
@@ -203,7 +231,10 @@ subjects:
203231
name: agent-sandbox-controller
204232
namespace: agent-sandbox-system
205233
---
206-
# Sandbox Server using agent-sandbox CRD for isolation
234+
# Static Sandbox resource for single-sandbox mode (SANDBOX_MODE=single).
235+
# In multi-sandbox mode (SANDBOX_MODE=multi), the MCP server creates
236+
# per-session Sandbox CRDs dynamically — remove this resource and
237+
# the static sandbox-server Service below in that case.
207238
apiVersion: agents.x-k8s.io/v1alpha1
208239
kind: Sandbox
209240
metadata:
@@ -290,7 +321,8 @@ spec:
290321
configMap:
291322
name: prodisco-config
292323
---
293-
# Sandbox Server Service
324+
# Static Service for single-sandbox mode.
325+
# In multi-sandbox mode, the Sandbox controller creates a Service per CRD automatically.
294326
apiVersion: v1
295327
kind: Service
296328
metadata:
@@ -349,11 +381,12 @@ spec:
349381
value: "/tmp/cache"
350382
- name: HOME
351383
value: "/tmp"
352-
# Connect to sandbox-server via TCP
353-
- name: SANDBOX_USE_TCP
354-
value: "true"
355-
- name: SANDBOX_TCP_HOST
356-
value: "sandbox-server.prodisco.svc.cluster.local"
384+
# Multi-sandbox mode: each MCP session gets its own Sandbox CRD
385+
# Set to "single" to use the shared static sandbox (requires static Sandbox CRD above)
386+
- name: SANDBOX_MODE
387+
value: "multi"
388+
- name: SANDBOX_NAMESPACE
389+
value: "prodisco"
357390
- name: SANDBOX_TCP_PORT
358391
value: "50051"
359392
- name: PRODISCO_CONFIG_PATH

0 commit comments

Comments
 (0)