Skip to content

sbx: Support custom OpenAI-compatible API providers in credential proxy #904

@williamzujkowski

Description

@williamzujkowski

Problem

The sbx credential proxy currently only injects stored secrets for a fixed set of known provider hostnames (api.openai.com, api.anthropic.com, generativelanguage.googleapis.com, etc.). There is no mechanism to register a custom provider with its own base URL and have the proxy inject credentials for that hostname.

This means anyone using an OpenAI-compatible API at a custom endpoint — vLLM, Ollama, LiteLLM, private API gateways, on-premise LLM servers, or any self-hosted inference service — cannot benefit from sbx's credential proxy security model.

Current Workaround

# Must export API key as plaintext env var on the host
export OPENAI_API_KEY="sk-my-key"
export OPENAI_BASE_URL="https://llm.internal.example.com/v1"

# Must bypass the proxy entirely for the custom host
sbx run opencode /workspace \
  --bypass-host llm.internal.example.com \
  --allow-host llm.internal.example.com

This loses the key benefits of the credential proxy:

  • Secrets are exposed as host environment variables (not kept in the proxy's secret store)
  • --bypass-host disables MITM inspection for that hostname, losing visibility
  • No credential injection — the agent must have the API key directly in its config or env

Proposed Solution

Allow users to register custom providers with a base URL and stored credential:

# Register a custom provider
sbx provider add \
  --name my-llm \
  --base-url https://llm.internal.example.com/v1

# Store the API key for the custom provider
sbx secret set -g my-llm

# The credential proxy now injects for this hostname automatically
sbx run opencode /workspace
# → Requests to llm.internal.example.com get Authorization header injected

Or as a more minimal version — extend sbx secret set to accept a custom hostname:

sbx secret set --host llm.internal.example.com --header "Authorization: Bearer"

Use Cases

  1. On-premise LLM serving — Teams running vLLM, TGI, or Ollama behind a reverse proxy
  2. API gateways — Organizations using LiteLLM, OpenRouter, or custom API gateways that expose OpenAI-compatible endpoints
  3. Private cloud deployments — LLM inference services running on internal infrastructure with custom hostnames
  4. Enterprise environments — Locked-down networks where all API traffic must go through internal endpoints, not public cloud APIs

Expected Behavior

Scenario Current Proposed
Request to api.openai.com Proxy injects stored OpenAI key Same (no change)
Request to llm.internal.example.com No injection, must use --bypass-host Proxy injects stored custom provider key
sbx secret set -g my-llm Not supported Stores key for custom provider

Additional Context

The OpenAI-compatible API format (/v1/chat/completions, /v1/models, Authorization: Bearer <key>) has become a de facto standard. Most self-hosted inference servers (vLLM, Ollama, LiteLLM, LocalAI, llama.cpp server) implement this same interface. Supporting custom providers would make sbx's security model available to the entire ecosystem, not just the handful of hardcoded cloud providers.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions