diff --git a/app/_gateway_entities/vault.md b/app/_gateway_entities/vault.md index 1f1143ff32..b56195b7ac 100644 --- a/app/_gateway_entities/vault.md +++ b/app/_gateway_entities/vault.md @@ -11,6 +11,8 @@ description: | related_resources: - text: Secrets Management url: /gateway/secrets-management/ + - text: Create a custom vault backend + url: /how-to/create-custom-vault/ - text: Workspaces url: /gateway/entities/workspace/ - text: RBAC @@ -260,6 +262,11 @@ features: oss: false enterprise: true supports_konnect: true + - title: Custom vault backend + url: /how-to/create-custom-vault/ + oss: true + enterprise: true + supports_konnect: false {% endfeature_table %} ## How do I reference secrets stored in a Vault? diff --git a/app/_how-tos/gateway/create-custom-vault.md b/app/_how-tos/gateway/create-custom-vault.md new file mode 100644 index 0000000000..2e38de138b --- /dev/null +++ b/app/_how-tos/gateway/create-custom-vault.md @@ -0,0 +1,351 @@ +--- +title: Create a custom vault backend +permalink: /how-to/create-custom-vault/ +content_type: how_to + +description: Learn how to write a custom vault backend for {{site.base_gateway}} and use it to inject secrets into plugin configuration at request time. + +products: + - gateway + +works_on: + - on-prem + +min_version: + gateway: '3.4' + +entities: + - vault + - service + - route + - plugin + +plugins: + - request-transformer-advanced + +tags: + - secrets-management + - security + +tldr: + q: How do I create a custom vault backend in {{site.base_gateway}}? + a: | + Write a Lua module in `kong/vaults/.lua` that exports `name`, `VERSION`, `init`, and `get`. Add a schema at `kong/vaults//schema.lua`. Start {{site.base_gateway}} with `KONG_VAULTS: bundled,` and `KONG_LUA_PACKAGE_PATH` pointing to your code. Create a Vault entity, then reference secrets with `{vault:///}` in any referenceable field. + +tools: + - deck + +prereqs: + skip_product: true + inline: + - title: Docker and Docker Compose + content: | + Install [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/). + icon_url: /assets/icons/gateway.svg + - title: Export your Kong Gateway license + content: | + Export your {{site.ee_product_name}} license as an environment variable: + ```bash + export KONG_LICENSE_DATA='your-license-contents' + ``` + icon_url: /assets/icons/gateway.svg + +cleanup: + inline: + - title: Destroy the Docker Compose environment + content: | + Stop and remove all containers and volumes: + ```bash + docker compose down -v + ``` + icon_url: /assets/icons/gateway.svg + +related_resources: + - text: Secrets management + url: /gateway/secrets-management/ + - text: Vault entity + url: /gateway/entities/vault/ + - text: What can be stored as a secret? + url: /gateway/entities/vault/#what-can-be-stored-as-a-secret + +next_steps: + - text: Vault entity reference + url: /gateway/entities/vault/ + - text: Secrets management overview + url: /gateway/secrets-management/ + +faqs: + - q: When is `get` called? + a: | + {{site.base_gateway}} calls `get` whenever it needs to resolve a `{vault://...}` reference. With the default TTL, the resolved value is cached and `get` is not called again until the TTL expires. Set `KONG_VAULT__TTL=0` to disable caching and call `get` on every request. + - q: Can my vault return a value other than a string? + a: | + No. `get` must return a string or `nil`. {{site.base_gateway}} stores the resolved value as a string and injects it into the plugin config field as-is. + - q: | + {% include /gateway/vaults-format-faq.md type='question' %} + a: | + {% include /gateway/vaults-format-faq.md type='answer' %} + +--- + +## Create the vault module + +A custom vault backend is a Lua module with two required functions: + + +{% table %} +columns: + - title: Function + key: function + - title: Description + key: description +rows: + - function: "`init(conf)`" + description: Called once at startup. Use it to validate config or open persistent connections. + - function: "`get(conf, resource, version)`" + description: "Called to resolve a secret reference. Returns the secret value as a string, or `nil` if not found. `resource` is the key portion of the `{vault://prefix/key}` reference." +{% endtable %} + + +Create the directory structure: + +```bash +mkdir -p kong/vaults/http +``` + +Create `kong/vaults/http.lua`: + +```lua +echo ' +local http = require("resty.http") +local cjson = require("cjson") + +local function init(conf) +end + +local function get(conf, resource, version) + local base_url = conf.base_url or "http://localhost:9876/sekretz" + + local httpc = http.new() + local res, err = httpc:request_uri(base_url .. "/" .. resource, { + method = "GET", + headers = { + ["Accept"] = "application/json", + }, + }) + + if not res then + ngx.log(ngx.WARN, "http vault: request failed for ", resource, ": ", err) + return nil + end + + if res.status ~= 200 then + ngx.log(ngx.WARN, "http vault: status ", res.status, " for ", resource) + return nil + end + + local ok, data = pcall(cjson.decode, res.body) + if not ok or data == nil then + ngx.log(ngx.WARN, "http vault: failed to decode JSON for ", resource) + return nil + end + + return data.value +end + +return { + name = "http", + VERSION = "1.0.0", + init = init, + get = get, +}' > kong/vaults/http.lua +``` + +The module fetches `/` and returns the `value` field from the JSON response body. + +## Create the vault schema + +The schema declares the configuration fields exposed on the Vault entity. Create `kong/vaults/http/schema.lua`: + +```lua +echo 'return { + name = "http", + fields = { + { + config = { + type = "record", + fields = { + { + base_url = { + type = "string", + default = "http://localhost:9876/sekretz", + description = "Base URL of the HTTP secret store. Secrets are fetched from /.", + }, + }, + }, + }, + }, + }, +}' > kong/vaults/http/schema.lua +``` + +## Start Kong with the custom vault + +Kong must be started with the custom vault registered. This requires setting `KONG_VAULTS` and mounting the Lua files into the container before {{site.base_gateway}} starts. + +1. Create a secret that the vault will serve: + + ```bash + mkdir -p secrets/sekretz + echo '{"value":"X-From-Vault:top-secret-value"}' > secrets/sekretz/x-from-vault + ``` + +2. Create `docker-compose.yml`: + + ```yaml + echo ' + services: + secret-store: + image: busybox + command: httpd -f -p 9876 -h /www + volumes: + - ./secrets:/www + healthcheck: + test: ["CMD", "wget", "-q", "-O", "/dev/null", "http://localhost:9876/sekretz/x-from-vault"] + interval: 3s + timeout: 5s + retries: 10 + start_period: 5s + + postgres: + image: postgres:16 + environment: + POSTGRES_USER: kong + POSTGRES_PASSWORD: kong + POSTGRES_DB: kong + healthcheck: + test: ["CMD", "pg_isready", "-U", "kong"] + interval: 5s + timeout: 5s + retries: 10 + restart: unless-stopped + + kong-migrations: + image: kong/kong-gateway:latest + command: kong migrations bootstrap + environment: + KONG_DATABASE: postgres + KONG_PG_HOST: postgres + KONG_PG_USER: kong + KONG_PG_PASSWORD: kong + KONG_PG_DATABASE: kong + KONG_LICENSE_DATA: ${KONG_LICENSE_DATA} + depends_on: + postgres: + condition: service_healthy + + kong: + image: kong/kong-gateway:latest + environment: + KONG_DATABASE: postgres + KONG_PG_HOST: postgres + KONG_PG_USER: kong + KONG_PG_PASSWORD: kong + KONG_PG_DATABASE: kong + KONG_LICENSE_DATA: ${KONG_LICENSE_DATA} + KONG_PROXY_LISTEN: "0.0.0.0:8000" + KONG_ADMIN_LISTEN: "0.0.0.0:8001" + KONG_VAULTS: bundled,http + KONG_LUA_PACKAGE_PATH: /custom-code/?.lua;; + ports: + - "8000:8000" + - "8001:8001" + volumes: + - ./kong:/custom-code/kong + depends_on: + postgres: + condition: service_healthy + kong-migrations: + condition: service_completed_successfully + secret-store: + condition: service_healthy + healthcheck: + test: ["CMD", "kong", "health"] + interval: 10s + timeout: 5s + retries: 10 + restart: unless-stopped' > docker-compose.yml + ``` + + The two key environment variables for custom vaults are: + - `KONG_VAULTS: bundled,http` — registers the built-in vaults plus the custom `http` vault + - `KONG_LUA_PACKAGE_PATH: /custom-code/?.lua;;` — tells Kong where to find the Lua modules + +3. Start all services: + + ```bash + docker compose up -d + ``` + +## Create the Vault entity + +Create a Vault entity that configures the `http` vault backend. The `prefix` value (`http-vault`) is used in secret references: + +{% entity_examples %} +entities: + vaults: + - name: http + prefix: http-vault + description: Custom HTTP vault backend + config: + base_url: http://secret-store:9876/sekretz +{% endentity_examples %} + +The `base_url` uses the Docker Compose service name (`secret-store`) as the hostname, since Kong and the secret store run in the same Docker network. + +## Configure the plugin with a vault reference + +Create a service and route, then enable the [Request Transformer Advanced](/plugins/request-transformer-advanced/) plugin. The `add.headers` list accepts items in `header-name:value` format. Here the vault reference is used as the entire item — Kong resolves `{vault://http-vault/x-from-vault}` to `X-From-Vault:top-secret-value`, which the plugin then parses into header name `X-From-Vault` and value `top-secret-value`. + +{% entity_examples %} +entities: + services: + - name: httpbin + url: http://httpbin.konghq.com/anything + routes: + - name: httpbin-route + paths: + - / + protocols: + - http + plugins: + - name: request-transformer-advanced + config: + add: + headers: + - "{vault://http-vault/x-from-vault}" +{% endentity_examples %} + +## Validate + +Send a request through the proxy: + + +{% validation request-check %} +url: / +status_code: 200 +{% endvalidation %} + + +The [httpbin](https://httpbin.konghq.com) upstream echoes the full incoming request as a JSON response body. Check the `headers` object for `X-From-Vault`: + +```json +{ + "headers": { + "X-From-Vault": "top-secret-value" + } +} +``` +{:.no-copy-code} + +If the header is present with the value from `secrets/sekretz/x-from-vault`, the custom vault is working correctly. diff --git a/app/_landing_pages/gateway/secrets-management.yaml b/app/_landing_pages/gateway/secrets-management.yaml index 40beb36dea..eef73de9ef 100644 --- a/app/_landing_pages/gateway/secrets-management.yaml +++ b/app/_landing_pages/gateway/secrets-management.yaml @@ -147,13 +147,23 @@ rows: title: "CyberArk Secrets Manager (Conjur) {% new_in 3.11 %}" description: | Connect CyberArk Secrets Manager to {{site.base_gateway}} to reference secrets. - + icon: /assets/icons/cyberark.svg ctas: - text: Configuration url: "/gateway/entities/vault/?tab=conjur#vault-provider-specific-configuration-parameters" - text: With {{ site.base_gateway }} url: "/how-to/configure-cyberark-as-a-vault-backend/" + - blocks: + - type: card + config: + title: Custom vault backend + description: | + Build your own vault backend to fetch secrets from any HTTP endpoint or custom secret store. + icon: /assets/icons/code.svg + ctas: + - text: Create a custom vault + url: "/how-to/create-custom-vault/" - header: text: Secrets rotation type: h2