diff --git a/docs/agents/deploy-agents.mdx b/docs/agents/deploy-agents.mdx index 3c74746d27..e496c3aa2c 100644 --- a/docs/agents/deploy-agents.mdx +++ b/docs/agents/deploy-agents.mdx @@ -161,7 +161,15 @@ secrets. Pass the image with `--image`, or set **2. A configured deployments executor.** The platform operator defines named executors in the platform configuration and points the agents plugin at them. -A minimal Docker + Kubernetes configuration: +Each executor names a `backend`: + +| Backend | Runs the agent as | +|---------|-------------------| +| `docker` | A Docker container | +| `k8s` | A Kubernetes Deployment and Service | +| `openshell` | An OpenShell sandbox under a generated SandboxPolicy (Landlock filesystem isolation, `run_as_user: sandbox`, and default-deny network egress). Opt-in per deployment rather than a `--mode`. See [Sandboxed Deployments](/documentation/agents/sandboxed-deployments). | + +A minimal configuration for the Docker, Kubernetes, and OpenShell backends: ```yaml agents: @@ -187,6 +195,16 @@ deployments: config: # Omit kubeconfig_path to use in-cluster ServiceAccount auth default_namespace: default + - name: openshell-local + backend: openshell + config: + # OpenShell gateway (docker-driver default port; not 8080, the platform's) + gateway_endpoint: http://127.0.0.1:17670 + # Sandbox-writable working directory for the agent's `nat serve` process + serve_workdir: /home/sandbox + # null grants the sandbox no direct network egress; model traffic is + # brokered through the gateway-managed inference.local route + platform_egress: null ``` @@ -198,6 +216,8 @@ run in the core controller, whose Role already grants those permissions. +The `openshell` executor is not reachable through `nemo agents deploy --mode`; a sandboxed deployment is created directly through the deployments API with `executor: openshell-local`. It also needs a running OpenShell gateway and a one-time `inference.local` route wired on that gateway. See [Sandboxed Deployments](/documentation/agents/sandboxed-deployments) for the full prerequisites and deploy flow. + ### Deploy on Docker diff --git a/docs/agents/sandboxed-deployments.mdx b/docs/agents/sandboxed-deployments.mdx new file mode 100644 index 0000000000..5daf40d417 --- /dev/null +++ b/docs/agents/sandboxed-deployments.mdx @@ -0,0 +1,180 @@ +--- +title: "Sandboxed Deployments" +description: "Deploy a NAT agent as an OpenShell sandbox under a generated SandboxPolicy: Landlock filesystem isolation, a non-root user, and default-deny network egress." +--- + + +Deploy an ordinary NAT agent through the standard deployments API, but with the executor set to `openshell-local` so the agent runs inside an [OpenShell](https://github.com/NVIDIA/openshell) sandbox under an automatically generated SandboxPolicy. The agent image must be packaged with the OpenShell runtime profile (`nemo agents package --sandbox-runtime openshell`), which adds the non-root `sandbox` user the policy runs as; packaged that way, the agent gets Landlock filesystem isolation, that non-root user, and a default-deny network policy for free. Its model calls are brokered by the OpenShell gateway through the `inference.local` route, so the sandbox itself is granted no direct network egress at all: a call from the sandbox to anywhere else is blocked at the boundary. + + + +Sandboxed deployments are experimental and verified end-to-end on Linux (glibc >= 2.39) with OpenShell's docker driver. The exposed endpoint host and the sandbox's hop back to the platform are docker-driver specific, and the `inference.local` hop is unverified on macOS. + + + +## What the Sandbox Enforces + +When you deploy with `executor: openshell-local`, the OpenShell backend generates a default-deny SandboxPolicy from the executor config and the sandbox filesystem defaults, then applies it to the created sandbox: + +- **Landlock filesystem policy.** Read-only access to the interpreter and base system paths (`/opt`, `/usr`, `/bin`, `/lib`, `/etc`), and read-write access only to the paths the runtime needs (`/home/sandbox`, `/tmp`, `/dev/shm` for the Dask cluster `nat serve` spins up). The executor's `landlock_compatibility` defaults to `best_effort`, which enforces this policy where the kernel supports Landlock but fails open (no filesystem confinement) on kernels without it. Set `landlock_compatibility: hard_requirement` on the executor to make a missing-Landlock kernel fail the sandbox instead of running it unconfined. +- **`run_as_user: sandbox`.** The workload runs as the image's non-root `sandbox` user, not root. +- **Default-deny network egress.** With `platform_egress: null` (the shipped default) the policy has no egress rules at all. The agent reaches models through the gateway-managed `inference.local` route, so the sandbox needs no direct egress. A direct connection to any other host is blocked. + +With the shipped `platform_egress: null`, the generated policy is pure default-deny with no egress rules at all. The reference at `plugins/nemo-deployments/examples/openshell/local-sandbox-policy.yaml` documents the policy shape and the alternate direct-egress variant: its `network_policies.igw` rule allows the sandbox to reach the Inference Gateway directly at `host.docker.internal:8080`, which the zero-egress default does not grant. Pin a hand-written policy by pointing the executor's `default_policy_path` at your own YAML. + +## Model Access Through `inference.local` + +The agent reaches models through OpenShell's gateway-managed `inference.local` route. The gateway routes `https://inference.local/v1` to a registered provider (the platform Inference Gateway) and injects the real credential, so the sandbox makes no direct model call itself. This one-time operator wiring is per gateway and is not auto-registered by the deployment backend yet. + +Pick a model the Inference Gateway serves, register it as an OpenShell provider, and bind `inference.local` to it: + +```bash +# Pick a model the gateway serves (the list paginates) +nemo models list --all-pages +MODEL='default/openai-openai-gpt-4o-mini' # example; use an id the list above shows + +# Register the platform Inference Gateway as an openshell provider. The +# credential is a placeholder; the gateway injects the real one. host.openshell.internal +# is the gateway's alias for its own host, where the platform listens. +openshell provider create --name nemo-igw --type openai \ + --credential OPENAI_API_KEY=empty \ + --config OPENAI_BASE_URL=http://host.openshell.internal:8080/apis/inference-gateway/v2/workspaces/default/openai/-/v1 + +# Bind inference.local to that provider and model +openshell inference set --provider nemo-igw --model "$MODEL" +openshell inference get # verifies the route gateway-side +``` + + + +`model_name` must be a model the Inference Gateway can resolve. A model it cannot resolve makes `nat serve` exit inside the sandbox, and the deployment lands in `FAILED` rather than `READY`. Prefer a gpt-4o-mini-class model for clean tool-calling; verbose reasoning models can emit empty content that breaks the ReAct loop. + + + +## Prerequisites + +- **An OpenShell gateway reachable on `:17670`.** This is OpenShell's docker-driver default. It must not be `:8080`, which the platform owns. Docker-driver sandboxes need the gateway to mint sandbox JWTs, so its signing keys must be generated once before the gateway starts. The compose stack and one-time key generation live in `plugins/nemo-deployments/examples/openshell/`. +- **The platform started with the executor config and bound to all interfaces.** The `openshell-local` executor lives in `packages/nmp_platform/config/local.yaml`, which is not the config `nemo services run` loads by default, so pass it with `--config`. `--host 0.0.0.0` matters: the `inference.local` route is dialed from the sandbox at the sandbox network's gateway address, so the platform has to be listening there. On `--host 127.0.0.1` every model call fails with a 503. + + ```bash + export NMP_BASE_URL=http://localhost:8080 + nemo services run --host 0.0.0.0 --port 8080 \ + --config packages/nmp_platform/config/local.yaml + curl -sf http://localhost:8080/health/ready # {"status":"ready"} + ``` + + + + `--host 0.0.0.0` exposes an unauthenticated dev platform on every interface, LAN included. Do not run this on an untrusted network. + + + +- **Packaging and backend dependencies.** Packaging needs the agents plugin `container` extra (`python-on-whales`), and the OpenShell backend needs the deployments plugin `openshell` extra (the `openshell` SDK). Base bootstrap installs neither, so add them additively (do not use `uv sync --package`, which narrows the workspace venv and uninstalls the platform): `uv pip install "openshell>=0.0.92" "grpcio>=1.78.0" "protobuf>=6.31.1"` and `uv pip install -e 'plugins/nemo-agents[container]'`. + +## Package the Agent for the Sandbox + +A sandbox-ready agent config is a normal NAT config with three properties. The shipped demo config at `plugins/nemo-deployments/examples/openshell/agent/config.yaml` already satisfies them: + +- **The LLM points at `inference.local`.** Set `base_url: https://inference.local/v1`, and keep `api_key` non-empty (for example `not-used`). It must be non-empty because the gateway swaps in the real credential, but an empty string makes NAT reject the config. +- **No `general.telemetry` block.** The stock `react-agent.yml` enables the `nemo_files` tracer, which is not installed in the packaged sandbox image, so `nat serve` would reject the config. Remove the whole block before packaging. +- **The serve workdir is sandbox-writable.** A packaged image chowns `/workspace` to its own `agent` user, which the `sandbox` user cannot write, so `serve_workdir` defaults to `/home/sandbox`. + +Build the image with the OpenShell runtime profile, which adds the `sandbox` user and the packages the supervisor needs: + +```bash +nemo agents package \ + --agent plugins/nemo-deployments/examples/openshell/agent/config.yaml \ + --nat-version 1.8.0 \ + --sandbox-runtime openshell \ + --tag nemo-agent-igw:test +``` + +The config is baked into the image at `/workspace/config.yaml`. The model id is injected at deploy time with `nat serve --override llms.llm.model_name `, so an unset `${NEMO_DEFAULT_MODEL}` in the baked config is fine. + +## Deploy on the `openshell-local` Executor + +A sandboxed deployment is created directly through the deployments API, naming the executor explicitly. The default executor stays `local-docker`, so ordinary deployments keep their normal docker or k8s path. + +```bash +BASE=http://localhost:8080/apis/deployments/v2/workspaces/default + +# Describe the container: image, the serve command (with the model injected via +# --override), and the port. MODEL must match the id you passed to inference set. +curl -sf -X POST "$BASE/deployment-configs" -H 'content-type: application/json' -d "{ + \"name\": \"igw-agent-cfg\", + \"containers\": [{ + \"name\": \"agent\", + \"image\": \"nemo-agent-igw:test\", + \"command\": [\"/workspace/.venv/bin/nat\",\"serve\",\"--config_file\",\"/workspace/config.yaml\",\"--override\",\"llms.llm.model_name\",\"$MODEL\",\"--host\",\"0.0.0.0\",\"--port\",\"9000\"], + \"ports\": [{\"containerPort\": 9000, \"name\": \"http\"}] + }] +}" | jq . + +# Deploy with the sandbox executor +curl -sf -X POST "$BASE/deployments" -H 'content-type: application/json' -d '{ + "name": "igw-agent", + "deployment_config": "igw-agent-cfg", + "executor": "openshell-local" +}' | jq . +``` + +Behind the scenes the backend generates the default-deny SandboxPolicy, creates the sandbox from the image and applies the policy, runs the serve command in the sandbox-writable workdir, and exposes the service through the gateway. + +Wait for `READY`, then read the endpoint and invoke the agent: + +```bash +status="" +for i in $(seq 1 30); do + status=$(curl -sf "$BASE/deployments/igw-agent" | jq -r '.status') + echo "$status" + [ "$status" = "READY" ] && break + [ "$status" = "FAILED" ] && break + sleep 3 +done + +# Only invoke once the deployment is actually READY. On FAILED or timeout, stop and +# read the status message / logs instead of curling a URL that will not answer. +if [ "$status" != "READY" ]; then + echo "deployment did not reach READY (last status: ${status:-unknown})" +else + URL=$(curl -sf "$BASE/deployments/igw-agent" | jq -r '.endpoints[0].url') + + # The agent's LLM call goes to inference.local, which the gateway routes to the + # platform Inference Gateway. The sandbox makes no direct egress. + curl -sf -X POST "${URL}generate" -H 'content-type: application/json' \ + -d '{"input_message":"What is the current date and time? Use your current_datetime tool"}' | jq . +fi +``` + +A non-empty `value` in the response confirms the full path: sandbox to gateway to Inference Gateway and back. + +## Prove Zero Direct Egress + +The security punchline: the agent's model call worked, but the sandbox can reach nothing on the network directly. + +```bash +# Derive the sandbox name (nmp-) from the endpoint, or read `openshell sandbox list` +SBX=$(curl -sf "$BASE/deployments/igw-agent" | jq -r '.endpoints[0].url' | sed -E 's#^https?://##; s#--.*##') + +# Blocked by the default-deny policy: a nonzero exit from the inner curl is the pass +openshell sandbox exec --name "$SBX" -- curl -sS -m 5 https://example.com +``` + +## Clean Up + +```bash +curl -sf -X DELETE "$BASE/deployments/igw-agent" # tears down the sandbox +curl -sf -X DELETE "$BASE/deployment-configs/igw-agent-cfg" +``` + +## Current Limitations + +- The NAT config is baked into the image; deploy-time config injection is a follow-up. +- The `inference.local` route requires the one-time operator wiring (provider create plus `inference set`) per gateway; the deployment backend does not auto-register it yet. +- The exposed endpoint host and the sandbox's hop back to the platform are docker-driver specific. +- The verified end-to-end flow depends on the `openshell-local` executor being loaded via `--config` and the `inference.local` route being wired on the gateway. + +## Related + +- [Deploy Agents](/documentation/agents/deploy-agents): subprocess and container deployment modes. +- [Models and Inference](/documentation/models-and-inference): register providers and virtual models the agent's model name resolves to. diff --git a/docs/fern/versions/latest.yml b/docs/fern/versions/latest.yml index a89a379789..a2dd8d8377 100644 --- a/docs/fern/versions/latest.yml +++ b/docs/fern/versions/latest.yml @@ -261,6 +261,8 @@ navigation: contents: - page: Deploy Agents path: ../../agents/deploy-agents.mdx + - page: Sandboxed Deployments + path: ../../agents/sandboxed-deployments.mdx - page: Observe Agents path: ../../agents/observability.mdx - page: Optimize Agents