Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 21 additions & 1 deletion docs/agents/deploy-agents.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,15 @@ secrets. Pass the image with `--image`, or set

**2. A configured deployments executor.** The platform operator defines named
executors in the platform configuration and points the agents plugin at them.
A minimal Docker + Kubernetes configuration:
Each executor names a `backend`:

| Backend | Runs the agent as |
|---------|-------------------|
| `docker` | A Docker container |
| `k8s` | A Kubernetes Deployment and Service |
| `openshell` | An OpenShell sandbox under a generated SandboxPolicy (Landlock filesystem isolation, `run_as_user: sandbox`, and default-deny network egress). Opt-in per deployment rather than a `--mode`. See [Sandboxed Deployments](/documentation/agents/sandboxed-deployments). |

A minimal configuration for the Docker, Kubernetes, and OpenShell backends:

```yaml
agents:
Expand All @@ -187,6 +195,16 @@ deployments:
config:
# Omit kubeconfig_path to use in-cluster ServiceAccount auth
default_namespace: default
- name: openshell-local
backend: openshell
config:
# OpenShell gateway (docker-driver default port; not 8080, the platform's)
gateway_endpoint: http://127.0.0.1:17670
# Sandbox-writable working directory for the agent's `nat serve` process
serve_workdir: /home/sandbox
# null grants the sandbox no direct network egress; model traffic is
# brokered through the gateway-managed inference.local route
platform_egress: null
```

<Note>
Expand All @@ -198,6 +216,8 @@ run in the core controller, whose Role already grants those permissions.

</Note>

The `openshell` executor is not reachable through `nemo agents deploy --mode`; a sandboxed deployment is created directly through the deployments API with `executor: openshell-local`. It also needs a running OpenShell gateway and a one-time `inference.local` route wired on that gateway. See [Sandboxed Deployments](/documentation/agents/sandboxed-deployments) for the full prerequisites and deploy flow.

### Deploy on Docker

<Tabs>
Expand Down
180 changes: 180 additions & 0 deletions docs/agents/sandboxed-deployments.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
---
title: "Sandboxed Deployments"
description: "Deploy a NAT agent as an OpenShell sandbox under a generated SandboxPolicy: Landlock filesystem isolation, a non-root user, and default-deny network egress."
---
<a id="sandboxed-deployments"></a>

Deploy an ordinary NAT agent through the standard deployments API, but with the executor set to `openshell-local` so the agent runs inside an [OpenShell](https://github.com/NVIDIA/openshell) sandbox under an automatically generated SandboxPolicy. The agent image must be packaged with the OpenShell runtime profile (`nemo agents package --sandbox-runtime openshell`), which adds the non-root `sandbox` user the policy runs as; packaged that way, the agent gets Landlock filesystem isolation, that non-root user, and a default-deny network policy for free. Its model calls are brokered by the OpenShell gateway through the `inference.local` route, so the sandbox itself is granted no direct network egress at all: a call from the sandbox to anywhere else is blocked at the boundary.

<Note>

Sandboxed deployments are experimental and verified end-to-end on Linux (glibc >= 2.39) with OpenShell's docker driver. The exposed endpoint host and the sandbox's hop back to the platform are docker-driver specific, and the `inference.local` hop is unverified on macOS.

</Note>

## What the Sandbox Enforces

When you deploy with `executor: openshell-local`, the OpenShell backend generates a default-deny SandboxPolicy from the executor config and the sandbox filesystem defaults, then applies it to the created sandbox:

- **Landlock filesystem policy.** Read-only access to the interpreter and base system paths (`/opt`, `/usr`, `/bin`, `/lib`, `/etc`), and read-write access only to the paths the runtime needs (`/home/sandbox`, `/tmp`, `/dev/shm` for the Dask cluster `nat serve` spins up). The executor's `landlock_compatibility` defaults to `best_effort`, which enforces this policy where the kernel supports Landlock but fails open (no filesystem confinement) on kernels without it. Set `landlock_compatibility: hard_requirement` on the executor to make a missing-Landlock kernel fail the sandbox instead of running it unconfined.
- **`run_as_user: sandbox`.** The workload runs as the image's non-root `sandbox` user, not root.
- **Default-deny network egress.** With `platform_egress: null` (the shipped default) the policy has no egress rules at all. The agent reaches models through the gateway-managed `inference.local` route, so the sandbox needs no direct egress. A direct connection to any other host is blocked.

With the shipped `platform_egress: null`, the generated policy is pure default-deny with no egress rules at all. The reference at `plugins/nemo-deployments/examples/openshell/local-sandbox-policy.yaml` documents the policy shape and the alternate direct-egress variant: its `network_policies.igw` rule allows the sandbox to reach the Inference Gateway directly at `host.docker.internal:8080`, which the zero-egress default does not grant. Pin a hand-written policy by pointing the executor's `default_policy_path` at your own YAML.

## Model Access Through `inference.local`

The agent reaches models through OpenShell's gateway-managed `inference.local` route. The gateway routes `https://inference.local/v1` to a registered provider (the platform Inference Gateway) and injects the real credential, so the sandbox makes no direct model call itself. This one-time operator wiring is per gateway and is not auto-registered by the deployment backend yet.

Pick a model the Inference Gateway serves, register it as an OpenShell provider, and bind `inference.local` to it:

```bash
# Pick a model the gateway serves (the list paginates)
nemo models list --all-pages
MODEL='default/openai-openai-gpt-4o-mini' # example; use an id the list above shows

# Register the platform Inference Gateway as an openshell provider. The
# credential is a placeholder; the gateway injects the real one. host.openshell.internal
# is the gateway's alias for its own host, where the platform listens.
openshell provider create --name nemo-igw --type openai \
--credential OPENAI_API_KEY=empty \
--config OPENAI_BASE_URL=http://host.openshell.internal:8080/apis/inference-gateway/v2/workspaces/default/openai/-/v1

# Bind inference.local to that provider and model
openshell inference set --provider nemo-igw --model "$MODEL"
openshell inference get # verifies the route gateway-side
```

<Note>

`model_name` must be a model the Inference Gateway can resolve. A model it cannot resolve makes `nat serve` exit inside the sandbox, and the deployment lands in `FAILED` rather than `READY`. Prefer a gpt-4o-mini-class model for clean tool-calling; verbose reasoning models can emit empty content that breaks the ReAct loop.

</Note>

## Prerequisites

- **An OpenShell gateway reachable on `:17670`.** This is OpenShell's docker-driver default. It must not be `:8080`, which the platform owns. Docker-driver sandboxes need the gateway to mint sandbox JWTs, so its signing keys must be generated once before the gateway starts. The compose stack and one-time key generation live in `plugins/nemo-deployments/examples/openshell/`.
- **The platform started with the executor config and bound to all interfaces.** The `openshell-local` executor lives in `packages/nmp_platform/config/local.yaml`, which is not the config `nemo services run` loads by default, so pass it with `--config`. `--host 0.0.0.0` matters: the `inference.local` route is dialed from the sandbox at the sandbox network's gateway address, so the platform has to be listening there. On `--host 127.0.0.1` every model call fails with a 503.

```bash
export NMP_BASE_URL=http://localhost:8080
nemo services run --host 0.0.0.0 --port 8080 \
--config packages/nmp_platform/config/local.yaml
curl -sf http://localhost:8080/health/ready # {"status":"ready"}
```

<Note>

`--host 0.0.0.0` exposes an unauthenticated dev platform on every interface, LAN included. Do not run this on an untrusted network.

</Note>

- **Packaging and backend dependencies.** Packaging needs the agents plugin `container` extra (`python-on-whales`), and the OpenShell backend needs the deployments plugin `openshell` extra (the `openshell` SDK). Base bootstrap installs neither, so add them additively (do not use `uv sync --package`, which narrows the workspace venv and uninstalls the platform): `uv pip install "openshell>=0.0.92" "grpcio>=1.78.0" "protobuf>=6.31.1"` and `uv pip install -e 'plugins/nemo-agents[container]'`.

## Package the Agent for the Sandbox

A sandbox-ready agent config is a normal NAT config with three properties. The shipped demo config at `plugins/nemo-deployments/examples/openshell/agent/config.yaml` already satisfies them:

- **The LLM points at `inference.local`.** Set `base_url: https://inference.local/v1`, and keep `api_key` non-empty (for example `not-used`). It must be non-empty because the gateway swaps in the real credential, but an empty string makes NAT reject the config.
- **No `general.telemetry` block.** The stock `react-agent.yml` enables the `nemo_files` tracer, which is not installed in the packaged sandbox image, so `nat serve` would reject the config. Remove the whole block before packaging.
- **The serve workdir is sandbox-writable.** A packaged image chowns `/workspace` to its own `agent` user, which the `sandbox` user cannot write, so `serve_workdir` defaults to `/home/sandbox`.

Build the image with the OpenShell runtime profile, which adds the `sandbox` user and the packages the supervisor needs:

```bash
nemo agents package \
--agent plugins/nemo-deployments/examples/openshell/agent/config.yaml \
--nat-version 1.8.0 \
--sandbox-runtime openshell \
--tag nemo-agent-igw:test
```

The config is baked into the image at `/workspace/config.yaml`. The model id is injected at deploy time with `nat serve --override llms.llm.model_name <id>`, so an unset `${NEMO_DEFAULT_MODEL}` in the baked config is fine.

## Deploy on the `openshell-local` Executor

A sandboxed deployment is created directly through the deployments API, naming the executor explicitly. The default executor stays `local-docker`, so ordinary deployments keep their normal docker or k8s path.

```bash
BASE=http://localhost:8080/apis/deployments/v2/workspaces/default

# Describe the container: image, the serve command (with the model injected via
# --override), and the port. MODEL must match the id you passed to inference set.
curl -sf -X POST "$BASE/deployment-configs" -H 'content-type: application/json' -d "{
\"name\": \"igw-agent-cfg\",
\"containers\": [{
\"name\": \"agent\",
\"image\": \"nemo-agent-igw:test\",
\"command\": [\"/workspace/.venv/bin/nat\",\"serve\",\"--config_file\",\"/workspace/config.yaml\",\"--override\",\"llms.llm.model_name\",\"$MODEL\",\"--host\",\"0.0.0.0\",\"--port\",\"9000\"],
\"ports\": [{\"containerPort\": 9000, \"name\": \"http\"}]
}]
}" | jq .

# Deploy with the sandbox executor
curl -sf -X POST "$BASE/deployments" -H 'content-type: application/json' -d '{
"name": "igw-agent",
"deployment_config": "igw-agent-cfg",
"executor": "openshell-local"
}' | jq .
```

Behind the scenes the backend generates the default-deny SandboxPolicy, creates the sandbox from the image and applies the policy, runs the serve command in the sandbox-writable workdir, and exposes the service through the gateway.

Wait for `READY`, then read the endpoint and invoke the agent:

```bash
status=""
for i in $(seq 1 30); do
status=$(curl -sf "$BASE/deployments/igw-agent" | jq -r '.status')
echo "$status"
[ "$status" = "READY" ] && break
[ "$status" = "FAILED" ] && break
sleep 3
done

# Only invoke once the deployment is actually READY. On FAILED or timeout, stop and
# read the status message / logs instead of curling a URL that will not answer.
if [ "$status" != "READY" ]; then
echo "deployment did not reach READY (last status: ${status:-unknown})"
else
URL=$(curl -sf "$BASE/deployments/igw-agent" | jq -r '.endpoints[0].url')

# The agent's LLM call goes to inference.local, which the gateway routes to the
# platform Inference Gateway. The sandbox makes no direct egress.
curl -sf -X POST "${URL}generate" -H 'content-type: application/json' \
-d '{"input_message":"What is the current date and time? Use your current_datetime tool"}' | jq .
fi
```

A non-empty `value` in the response confirms the full path: sandbox to gateway to Inference Gateway and back.

## Prove Zero Direct Egress

The security punchline: the agent's model call worked, but the sandbox can reach nothing on the network directly.

```bash
# Derive the sandbox name (nmp-<hash>) from the endpoint, or read `openshell sandbox list`
SBX=$(curl -sf "$BASE/deployments/igw-agent" | jq -r '.endpoints[0].url' | sed -E 's#^https?://##; s#--.*##')

# Blocked by the default-deny policy: a nonzero exit from the inner curl is the pass
openshell sandbox exec --name "$SBX" -- curl -sS -m 5 https://example.com
```

## Clean Up

```bash
curl -sf -X DELETE "$BASE/deployments/igw-agent" # tears down the sandbox
curl -sf -X DELETE "$BASE/deployment-configs/igw-agent-cfg"
```

## Current Limitations

- The NAT config is baked into the image; deploy-time config injection is a follow-up.
- The `inference.local` route requires the one-time operator wiring (provider create plus `inference set`) per gateway; the deployment backend does not auto-register it yet.
- The exposed endpoint host and the sandbox's hop back to the platform are docker-driver specific.
- The verified end-to-end flow depends on the `openshell-local` executor being loaded via `--config` and the `inference.local` route being wired on the gateway.

## Related

- [Deploy Agents](/documentation/agents/deploy-agents): subprocess and container deployment modes.
- [Models and Inference](/documentation/models-and-inference): register providers and virtual models the agent's model name resolves to.
2 changes: 2 additions & 0 deletions docs/fern/versions/latest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -261,6 +261,8 @@ navigation:
contents:
- page: Deploy Agents
path: ../../agents/deploy-agents.mdx
- page: Sandboxed Deployments
path: ../../agents/sandboxed-deployments.mdx
- page: Observe Agents
path: ../../agents/observability.mdx
- page: Optimize Agents
Expand Down
Loading