From ae8af29bd2dc73ef149430719d0241c08641c66e Mon Sep 17 00:00:00 2001 From: Musilah Date: Tue, 20 Jan 2026 10:41:03 +0300 Subject: [PATCH 1/2] fix readme Signed-off-by: Musilah --- README.md | 110 ++++++++++++++++++++++++------------------- agent/README.md | 83 ++++++++++++++++++++++++++++++++ api/README.md | 9 +++- guardrails/README.md | 71 ++++++++++++++++++++++++++++ 4 files changed, 224 insertions(+), 49 deletions(-) create mode 100644 agent/README.md create mode 100644 guardrails/README.md diff --git a/README.md b/README.md index c1e652eb..a5430e89 100644 --- a/README.md +++ b/README.md @@ -1,11 +1,32 @@ +
+ # Cube AI -Cube AI is a framework for building GPT-based AI applications using confidential computing. It protects user data and the AI model by using a trusted execution environment (TEE). TEE is a secure area of a processor that ensures that code and data loaded inside it are protected with respect to confidentiality and integrity. Data confidentiality prevents unauthorized access of data from outside the TEE, while code integrity ensures that code inside the TEE remains unchanged and unaltered from unauthorized access. +Confidential computing framework for GPT-based applications. + +OpenAI-compatible API, multiple LLM backends, and TEE-backed isolation for data and model privacy. + +[![CI](https://github.com/ultravioletrs/cube/actions/workflows/main.yaml/badge.svg)](https://github.com/ultravioletrs/cube/actions/workflows/main.yaml) +[![UI CI](https://github.com/ultravioletrs/cube/actions/workflows/ui-ci.yaml/badge.svg)](https://github.com/ultravioletrs/cube/actions/workflows/ui-ci.yaml) +[![Check License](https://github.com/ultravioletrs/cube/actions/workflows/check-license.yaml/badge.svg)](https://github.com/ultravioletrs/cube/actions/workflows/check-license.yaml) +[![Go Report Card](https://goreportcard.com/badge/github.com/ultravioletrs/cube)](https://goreportcard.com/report/github.com/ultravioletrs/cube) +[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE) + +[Guide](https://github.com/ultravioletrs/cube-docs) | [Docs](https://github.com/ultravioletrs/cube-docs) | [License](LICENSE) +
+ +## Introduction + +Cube AI is a framework for building GPT-based applications using confidential computing. It protects user data and AI models with a trusted execution environment (TEE), which is a secure area of the processor that ensures code and data loaded inside it remain confidential and intact. This provides strong data confidentiality and code integrity even when the host environment is not fully trusted.

+## Why Cube AI + +Traditional GPT-based applications often rely on public cloud services where operators or hardware providers can access prompts and model responses. Cube AI addresses these privacy concerns by executing inference inside TEEs, ensuring that user data and AI models remain protected from unauthorized access outside the enclave. + ## Key Features - **Secure Computing**: Cube AI uses secure enclaves to protect user data and AI models from unauthorized access. @@ -27,19 +48,15 @@ Cube AI now supports vLLM, a high-throughput and memory-efficient inference engi ### Ollama Integration -Cube AI also integrates with Ollama for local model deployment, providing: +Cube AI integrates with Ollama for local model deployment, providing: -- Easy model management and deployment -- Local inference capabilities +- Model management and deployment +- Local inference - Support for various open-source models -## Why Cube AI? - -Traditional GPT-based AI applications often rely on public cloud services, which can be vulnerable to security breaches and unauthorized access. The tenant for example openai, and the hardware provider for example Azure, are not always transparent about their security practices and can be easily compromised. They can also access your prompts and model responses. Cube AI addresses these privacy concerns by using TEEs. Using TEEs, Cube AI ensures that user data and AI models are protected from unauthorized access outside the TEE. This helps to maintain user privacy and ensures that AI models are used in a controlled and secure manner. - -## How does Cube AI work? +## How Cube AI Works -Cube AI uses TEEs to protect user data and AI models from unauthorized access. TEE offers an execution space that provides a higher level of security for trusted applications running on the device. In Cube AI, the TEE ensures that AI models are executed securely and in a controlled environment. +Cube AI uses TEEs to protect user data and AI models from unauthorized access. The TEE provides a secure execution space for trusted applications. In Cube AI, inference runs inside the TEE so prompts, responses, and model data are protected even if the host OS is compromised. ## Getting Started @@ -52,19 +69,22 @@ Cube AI uses TEEs to protect user data and AI models from unauthorized access. T ### Quick Start 1. **Clone the repository** + ```bash git clone https://github.com/ultravioletrs/cube.git cd cube ``` 2. **Start Cube AI services** + ```bash make up ``` 3. **Get your authentication token** - All API requests require authentication using JWT tokens. Once the services are running, obtain a JWT token: + All API requests require JWT authentication. Once services are running, obtain a token: + ```bash curl -ksSiX POST https://localhost/users/tokens/issue \ -H "Content-Type: application/json" \ @@ -73,8 +93,9 @@ Cube AI uses TEEs to protect user data and AI models from unauthorized access. T "password": "m2N2Lfno" }' ``` - - The response will contain your JWT token: + + Response: + ```json { "access_token": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9...", @@ -84,7 +105,7 @@ Cube AI uses TEEs to protect user data and AI models from unauthorized access. T 4. **Create a domain** - All API requests require a domain ID in the URL path. You can either get the domain ID from the UI or create a new domain via the API: + All API requests require a domain ID in the URL path. You can fetch a domain ID from the UI or create one via the API: ```bash curl -ksSiX POST https://localhost/domains \ @@ -99,8 +120,9 @@ Cube AI uses TEEs to protect user data and AI models from unauthorized access. T } }' ``` - - The response will contain your domain information including the `id`: + + Response (includes `id`): + ```json { "id": "d7f9b3b8-4f7e-4f44-8d47-1a6e5e6f7a2b", @@ -116,25 +138,24 @@ Cube AI uses TEEs to protect user data and AI models from unauthorized access. T "updated_at": "2025-10-29T14:12:01Z" } ``` - - **Notes:** - - `name` and `route` are required fields - - `route` must be unique and cannot be changed after creation - - `metadata` must be a valid JSON object - - The `id` is automatically generated if not provided - - Save the `id` value as you'll need it for all subsequent API requests + + Notes: + - `name` and `route` are required fields. + - `route` must be unique and cannot be changed after creation. + - `metadata` must be a valid JSON object. + - Save the `id` value for subsequent API requests. 5. **Verify the installation** - + List available models (replace `YOUR_DOMAIN_ID` with the domain ID from step 4): + ```bash curl -k https://localhost/proxy/YOUR_DOMAIN_ID/v1/models \ -H "Authorization: Bearer YOUR_ACCESS_TOKEN" ``` 6. **Make your first AI request** - - Replace `YOUR_DOMAIN_ID` with your actual domain ID: + ```bash curl -k https://localhost/proxy/YOUR_DOMAIN_ID/v1/chat/completions \ -H "Content-Type: application/json" \ @@ -150,19 +171,16 @@ Cube AI uses TEEs to protect user data and AI models from unauthorized access. T }' ``` -### API Endpoints - -Cube AI exposes all services through Traefik reverse proxy. +## API Endpoints -All protected endpoints require the `Authorization: Bearer ` header with a valid JWT token. +Cube AI exposes all services through a Traefik reverse proxy. All protected endpoints require the `Authorization: Bearer ` header with a valid JWT token. -#### Proxy Endpoints (OpenAI-Compatible) +### Proxy Endpoints (OpenAI-Compatible) **Base URL:** `https://localhost/proxy/` -Replace `{domainID}` with your actual domain ID obtained from step 4 in the Getting Started guide. +Replace `{domainID}` with your domain ID from the Getting Started section. -### OpenAI-Compatible Endpoints | Method | Path | Description | |--------|-----------------------------------|-------------------------| | GET | `/{domainID}/v1/models` | List available models | @@ -172,24 +190,24 @@ Replace `{domainID}` with your actual domain ID obtained from step 4 in the Gett | POST | `/{domainID}/api/generate` | Generate completions | | POST | `/{domainID}/api/chat` | Chat completions | +Example: -**Example:** ```bash -# Using OpenAI-compatible endpoint +# OpenAI-compatible endpoint curl -k https://localhost/proxy/YOUR_DOMAIN_ID/v1/chat/completions \ -H "Authorization: Bearer YOUR_ACCESS_TOKEN" \ -H "Content-Type: application/json" \ -d '{"model":"tinyllama:1.1b","messages":[{"role":"user","content":"Hello"}]}' -# Using Ollama API endpoint +# Ollama API endpoint curl -k https://localhost/proxy/YOUR_DOMAIN_ID/api/tags \ -H "Authorization: Bearer YOUR_ACCESS_TOKEN" ``` -#### Auth Endpoints +### Auth Endpoints + **Base URL:** `https://localhost/users` -### User Registration & Authentication | Method | Path | Description | |--------|-------------------------------|----------------------------------------| | POST | `/users` | Register new user account | @@ -198,8 +216,8 @@ curl -k https://localhost/proxy/YOUR_DOMAIN_ID/api/tags \ | POST | `/password/reset-request` | Request password reset | | PUT | `/password/reset` | Reset password with token | +Example: -**Example** ```bash curl -ksSiX POST https://localhost/users/tokens/issue \ -H "Content-Type: application/json" \ @@ -209,12 +227,10 @@ curl -ksSiX POST https://localhost/users/tokens/issue \ }' ``` ---- +### Domains Endpoints -#### Domains Endpoints **Base URL:** `https://localhost/domains` -### Domain Management | Method | Path | Description | |--------|-------------------------------|----------------------------------------| | POST | `/domains` | Create new domain | @@ -224,9 +240,9 @@ curl -ksSiX POST https://localhost/users/tokens/issue \ | POST | `/domains/{domainID}/enable` | Enable a domain | | POST | `/domains/{domainID}/disable` | Disable a domain | | POST | `/domains/{domainID}/freeze` | Freeze a domain | - -**Example** +Example: + ```bash curl -ksSiX POST https://localhost/domains \ -H "Content-Type: application/json" \ @@ -241,13 +257,11 @@ curl -ksSiX POST https://localhost/domains \ }' ``` ---- - ## Configuration ### vLLM Backend -Configure vLLM settings through environment: +Configure vLLM settings through the environment: ```bash make up-vllm @@ -267,4 +281,4 @@ Project documentation is hosted at [Cube AI docs repository](https://github.com/ ## License -Cube AI is published under permissive open-source [Apache-2.0](LICENSE) license. +Cube AI is published under the permissive [Apache-2.0](LICENSE) license. diff --git a/agent/README.md b/agent/README.md new file mode 100644 index 00000000..89e2b58e --- /dev/null +++ b/agent/README.md @@ -0,0 +1,83 @@ +# Cube Agent + +Cube Agent runs inside the CVM and exposes a small HTTP API for CVM attestation plus a reverse proxy to the configured LLM backend (Ollama, vLLM, or custom). Cube Proxy uses the agent for attestation and backend access. + +## Configuration + +The service is configured via environment variables (unset variables use defaults): + +| Variable | Description | Default | +| --- | --- | --- | +| `UV_CUBE_AGENT_LOG_LEVEL` | Log level (debug, info, warn, error) | `info` | +| `UV_CUBE_AGENT_INSTANCE_ID` | Instance ID returned by `/health` (auto-generated if empty) | `""` | +| `UV_CUBE_AGENT_HOST` | HTTP bind host | `0.0.0.0` | +| `UV_CUBE_AGENT_PORT` | HTTP bind port | `8901` | +| `UV_CUBE_AGENT_SERVER_CERT` | Path to PEM-encoded server certificate (TLS) | `""` | +| `UV_CUBE_AGENT_SERVER_KEY` | Path to PEM-encoded server key (TLS) | `""` | +| `UV_CUBE_AGENT_TARGET_URL` | LLM backend base URL for reverse proxy | `http://localhost:11434` | +| `UV_CUBE_AGENT_CA_URL` | CA URL for aTLS certificate provisioning | `""` | +| `UV_CUBE_AGENT_CERTS_TOKEN` | CA auth token used for aTLS certificate provisioning | `""` | +| `UV_CUBE_AGENT_CVM_ID` | CVM identifier used for certificate provisioning | `""` | +| `AGENT_MAA_URL` | Azure Attestation service URL | `https://sharedeus2.eus2.attest.azure.net` | +| `AGENT_OS_BUILD` | Attestation OS build identifier | `UVC` | +| `AGENT_OS_DISTRO` | Attestation OS distro identifier | `UVC` | +| `AGENT_OS_TYPE` | Attestation OS type identifier | `UVC` | +| `AGENT_VMPL` | AMD SEV-SNP VMPL value used for vTPM provider | `2` | + +## Features + +- **Attestation API**: Generates attestation reports for SEV-SNP and TDX-backed CVMs. +- **Reverse Proxy**: Forwards requests to the configured LLM backend. +- **Health and Metrics**: `/health` and `/metrics` endpoints for operational status. +- **Header Hygiene**: Drops `Authorization` when proxying and enforces JSON content type. + +## Endpoints + +| Method | Path | Description | +| --- | --- | --- | +| GET | `/health` | Health check with version and instance ID | +| GET | `/metrics` | Prometheus metrics | +| POST | `/attestation` | Generate attestation report | +| * | `/*` | Reverse proxy to `UV_CUBE_AGENT_TARGET_URL` | + +### Attestation Request + +`report_data` must be base64-encoded 64 bytes and `nonce` must be base64-encoded 32 bytes. Supported `attestation_type` values are `snp`, `tdx`, `vtpm`, and `snpvtpm`. + +```bash +curl -X POST http://localhost:8901/attestation \ + -H "Content-Type: application/json" \ + -d '{ + "report_data": "", + "nonce": "", + "attestation_type": "snp", + "to_json": true + }' +``` + +If `to_json` is `true`, the response is JSON. Otherwise it returns raw binary with `Content-Type: application/octet-stream`. + +## Architecture + +- **Go**: Core service implementation. +- **HTTP (chi + go-kit)**: Routing and transport. +- **Cocos attestation providers**: SNP, SNPvTPM, and TDX attestation sources. +- **Reverse proxy**: Standard library `httputil` with header sanitization. + +## Deployment + +### Docker Compose + +The `cube-agent` service is defined in `docker/cube-compose.yaml`. Update values in `docker/.env` to configure the agent. + +### Local Run + +```bash +UV_CUBE_AGENT_TARGET_URL=http://localhost:11434 \ +UV_CUBE_AGENT_PORT=8901 \ +go run ./cmd/agent +``` + +## Attestation Policy Workflow + +For policy generation and aTLS setup, see `agent/workflows/attestation-policy.md`. diff --git a/api/README.md b/api/README.md index 6585e7bd..3c7a5c73 100644 --- a/api/README.md +++ b/api/README.md @@ -7,9 +7,11 @@ This directory contains OpenAPI 3.0 specifications for all HTTP endpoints expose ### Proxy Service #### `proxy-routes.yaml` + REST API for managing dynamic routing rules. This enables runtime creation, retrieval, updating, and deletion of route rules that determine how requests are forwarded to backend services. **Key Endpoints:** + - `POST /api/routes` - Create a new routing rule - `GET /api/routes` - List all routing rules - `GET /api/routes/{name}` - Get a specific routing rule @@ -17,16 +19,20 @@ REST API for managing dynamic routing rules. This enables runtime creation, retr - `DELETE /api/routes/{name}` - Delete a routing rule #### `proxy-attestation.yaml` + API for managing attestation policies at the proxy level. Policies define trust and verification requirements for requests. **Key Endpoints:** + - `POST /attestation/policy` - Update global attestation policy - `GET /{domainID}/attestation/policy` - Get attestation policy for a domain #### `proxy-request-forwarding.yaml` + API for forwarding requests to backend services through the proxy with dynamic routing. Supports all HTTP methods (GET, POST, PUT, DELETE, PATCH). **Key Endpoints:** + - `GET /{domainID}/{path}` - Forward GET request - `POST /{domainID}/{path}` - Forward POST request - `PUT /{domainID}/{path}` - Forward PUT request @@ -87,7 +93,8 @@ These OpenAPI specifications can be used to: ### Example: Swagger UI To view these specs in Swagger UI, serve the files and navigate to: -``` + +```bash https://editor.swagger.io/?url= ``` diff --git a/guardrails/README.md b/guardrails/README.md new file mode 100644 index 00000000..7b8aa548 --- /dev/null +++ b/guardrails/README.md @@ -0,0 +1,71 @@ +# Cube Guardrails + +Cube Guardrails provides safety controls for LLM interactions using Nemo Guardrails, backed by a versioned configuration store. It runs as a dedicated service that Cube Proxy can call for guarded chat completions and configuration management. + +## Configuration + +The service is configured via environment variables (unset variables use defaults): + +| Variable | Description | Default | +| --- | --- | --- | +| `UV_GUARDRAILS_DB_HOST` | PostgreSQL host | `guardrails-db` | +| `UV_GUARDRAILS_DB_PORT` | PostgreSQL port | `5432` | +| `UV_GUARDRAILS_DB_USER` | PostgreSQL user | `guardrails` | +| `UV_GUARDRAILS_DB_PASS` | PostgreSQL password | `guardrails` | +| `UV_GUARDRAILS_DB_NAME` | PostgreSQL database name | `guardrails` | + +## Features + +- **Guarded chat**: OpenAI-compatible chat-style responses via `/guardrails/messages`. +- **Config management**: CRUD for guardrail configs with versioning. +- **Runtime reload**: Swap active configurations without restarting the service. +- **Migrations**: Database schema migrations run on startup. +- **Bootstrap config**: Loads `./rails` as a default config when no active config exists. + +## Endpoints + +Base path: `/guardrails` + +| Method | Path | Description | +| --- | --- | --- | +| POST | `/guardrails/messages` | Guarded chat completion | +| GET | `/guardrails/health` | Health check | +| GET | `/guardrails` | Service info and endpoint list | +| POST | `/guardrails/configs` | Create config | +| GET | `/guardrails/configs` | List configs | +| GET | `/guardrails/configs/{config_id}` | Get config | +| PUT | `/guardrails/configs/{config_id}` | Update config | +| DELETE | `/guardrails/configs/{config_id}` | Delete config | +| POST | `/guardrails/configs/{config_id}/versions` | Create config version | +| GET | `/guardrails/configs/{config_id}/versions` | List config versions | +| POST | `/guardrails/versions/{version_id}/activate` | Activate version | +| POST | `/guardrails/reload` | Reload active configuration | + +## Architecture + +- **FastAPI**: REST API surface and OpenAPI spec. +- **PostgreSQL**: Stores guardrail configs and versions. +- **Nemo Guardrails**: Runtime policy engine for safe responses. +- **asyncpg**: Async database access layer. + +## Deployment + +### Docker Compose + +The `cube-guardrails` service is defined in `docker/cube-compose.yaml`. Configure environment values in `docker/.env`. + +### Local Run + +```bash +cd guardrails +UV_GUARDRAILS_DB_HOST=localhost \ +UV_GUARDRAILS_DB_PORT=5432 \ +UV_GUARDRAILS_DB_USER=guardrails \ +UV_GUARDRAILS_DB_PASS=guardrails \ +UV_GUARDRAILS_DB_NAME=guardrails \ +uvicorn main:app --host 0.0.0.0 --port 8001 --reload +``` + +## API Spec + +See `api/guardrails.yaml` for the full OpenAPI definition. From 3eaa45e0978b1a0b12e957c94d6cd6ade0af079a Mon Sep 17 00:00:00 2001 From: Musilah Date: Tue, 20 Jan 2026 16:06:59 +0300 Subject: [PATCH 2/2] remove messges endpoint Signed-off-by: Musilah --- guardrails/README.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/guardrails/README.md b/guardrails/README.md index 7b8aa548..4afcd385 100644 --- a/guardrails/README.md +++ b/guardrails/README.md @@ -16,7 +16,6 @@ The service is configured via environment variables (unset variables use default ## Features -- **Guarded chat**: OpenAI-compatible chat-style responses via `/guardrails/messages`. - **Config management**: CRUD for guardrail configs with versioning. - **Runtime reload**: Swap active configurations without restarting the service. - **Migrations**: Database schema migrations run on startup. @@ -28,7 +27,6 @@ Base path: `/guardrails` | Method | Path | Description | | --- | --- | --- | -| POST | `/guardrails/messages` | Guarded chat completion | | GET | `/guardrails/health` | Health check | | GET | `/guardrails` | Service info and endpoint list | | POST | `/guardrails/configs` | Create config |