From b0b28232853afc4ff4e853b71d671dfaebee6015 Mon Sep 17 00:00:00 2001 From: OnlineChef Date: Sun, 2 Aug 2026 12:08:27 +0200 Subject: [PATCH 01/18] docs(ratelimit): define runtime activation contract --- structure/rate-limit-runtime-wiring.md | 135 +++++++++++++++++++++++++ 1 file changed, 135 insertions(+) create mode 100644 structure/rate-limit-runtime-wiring.md diff --git a/structure/rate-limit-runtime-wiring.md b/structure/rate-limit-runtime-wiring.md new file mode 100644 index 000000000..db0c33d11 --- /dev/null +++ b/structure/rate-limit-runtime-wiring.md @@ -0,0 +1,135 @@ +# Rate-limit runtime wiring contract + +Status: implementation contract for the runtime activation PR. Parent dependency: token-bucket policy-transition fix. + +## Non-negotiable boundaries + +- Feature is optional and disabled by default. With no `rateLimit` config, all existing responses, auth behavior, streaming, upgrades, logging, and metrics remain unchanged. +- Authentication runs before limiting. Invalid credentials receive the existing 401 and must not allocate buckets. +- `Origin` is never a bypass or identity signal. +- Raw credentials, session tokens, remote addresses, fingerprints, provider/model/account/request/conversation identities, prompts, and errors never leave auth/admission internals. +- Token consumption and WebSocket reservation remain synchronous with no `await` inside their atomic state transitions. +- Buckets and fingerprint secrets remain process-local and are never persisted. +- Do not alter provider routing, provider account selection, upstream credentials, or request bodies. + +## Canonical configuration + +Add one optional top-level `rateLimit` object through `OcxConfig`, the canonical Zod schema, config validation/save/load reconciliation, safe DTO handling, and docs. + +Suggested shape: + +```ts +interface OcxRateLimitConfig { + enabled?: boolean; // default false + bypassLoopback?: boolean; // default false; explicit only + maxBuckets?: number; // positive bounded integer + staleAfterMs?: number; // positive bounded integer + policies?: Partial>; + webSocket?: { + perPrincipal: number; + global: number; + maxTrackedPrincipals?: number; + }; +} +``` + +Rules: + +- Unknown policy keys are rejected. +- Rates are positive finite numbers; burst/caps are positive integers. +- Enabled configuration must not silently leave protected routes unlimited. Either require a complete policy set or define and document fixed bounded defaults for every surface. +- `bypassLoopback` has no effect unless `enabled` is true. +- WebSocket concurrency settings are required when Responses WebSocket admission is enabled, or use explicit documented bounded defaults. +- All defaults and caps are stable and documented. + +## Typed authentication results + +Add additive internal helpers that return a successful authenticated identity without changing existing public response helpers. + +Data-plane success: + +- exact accepted OpenCodex admission secret -> `rateLimitFingerprinter.admissionKey(secret)`; +- loopback/no-auth success -> trustworthy remote address only when supplied directly by Bun server APIs, otherwise shared anonymous principal. + +Management success: + +- admin token and valid GUI session -> `rateLimitFingerprinter.management(acceptedCredential)`; +- never fingerprint CSRF tokens, Origin, Host, provider keys, or upstream bearer credentials. + +Raw accepted credentials may exist only inside auth helpers long enough to fingerprint. Limiter/runtime code receives only `RateLimitPrincipal`. + +## Runtime coordinator + +Add one small server-side module owning one `TokenBucketLimiter` and one `WebSocketConcurrencyLimiter`. + +Responsibilities: + +- stable route/method -> `RateLimitSurface` mapping; +- no-op when disabled or explicit loopback bypass applies; +- token consume once per admitted request after auth and before expensive parsing/upstream work; +- API-specific 429 responses with integer `Retry-After` and bounded `X-RateLimit-Limit`, `X-RateLimit-Remaining`, `X-RateLimit-Reset` headers where compatible; +- aggregate-only test snapshots and reset hooks; +- no identity values in responses, logs, snapshots, or metrics. + +Surface mapping: + +- management: `/api/*`, `/metrics`; +- model-discovery: `GET /v1/models`; +- responses-http: `POST /v1/responses`, `POST /v1/responses/compact`; +- responses-websocket: WebSocket upgrade on `/v1/responses`; +- chat-completions: `POST /v1/chat/completions`; +- claude-messages: `POST /v1/messages`, `POST /v1/messages/count_tokens`; +- images: `POST /v1/images/generations`, `POST /v1/images/edits`; +- search: `POST /v1/alpha/search`; +- live: `POST /v1/live`, `POST /v1/realtime/calls`, and live/realtime sideband WebSocket upgrades. + +Artifact downloads and `/healthz` are not added implicitly; any protection there requires an explicit decision and tests. + +## Error envelopes + +- OpenAI/Responses/Chat/images/search/live/model discovery: preserve the existing `formatErrorResponse` vocabulary with a rate-limit error/code. +- Claude endpoints: preserve `anthropicErrorResponse(..., "rate_limit_error")`. +- Management: preserve management JSON/CORS conventions. +- WebSocket handshake denial returns HTTP 429 and does not upgrade. +- Retry metadata is integer, non-negative, and derived only from limiter decisions. + +## WebSocket lifecycle + +For Responses WebSocket and long-lived live sideband sockets: + +1. authenticate; +2. apply request token charge; +3. reserve per-principal/global concurrency; +4. attempt upgrade; +5. release immediately when upgrade fails; +6. store the idempotent release handle in typed `WsData`; +7. release on close and every explicit abort/error/timeout path. + +Repeated close/error paths must not underflow. A stale release after limiter reset must not affect new reservations. + +## Metrics + +Project aggregate bounded statistics into the existing metrics registry and both exports: + +- allow/deny totals by bounded surface/source/result; +- concurrency accepted/denied totals by bounded reason; +- current global WebSocket count and tracked-principal count as gauges. + +No principal or dynamic labels. Scraping must not mutate limiter state or create buckets. + +## Required tests + +- feature absent/disabled is byte-compatible; +- config validation and round-trip; +- authentication precedes limiter allocation; +- spoofed Origin cannot bypass or split buckets; +- two valid admission keys get distinct buckets without exposing identity; +- loopback/no-auth uses only remote-address or shared anonymous fallback; +- exact route mapping and one charge per request; +- correct OpenAI, Anthropic, and management 429 envelopes/headers; +- management admin token and GUI session principal behavior; +- principal-cap overflow is bounded/fail-closed; +- Responses WebSocket token charge and per-principal/global reservation; +- failed upgrade rollback, normal close, abnormal close/error, duplicate release; +- metrics cardinality/privacy and no secret/fingerprint leakage; +- full existing streaming and auth/CORS suites remain green. From 0a77ee0973c4160718d0c0621dcfdbb37a73a8a7 Mon Sep 17 00:00:00 2001 From: OnlineChef Date: Sun, 2 Aug 2026 12:31:57 +0200 Subject: [PATCH 02/18] feat(ratelimit): define complete runtime config defaults --- src/ratelimit/config.ts | 109 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 109 insertions(+) create mode 100644 src/ratelimit/config.ts diff --git a/src/ratelimit/config.ts b/src/ratelimit/config.ts new file mode 100644 index 000000000..69670369c --- /dev/null +++ b/src/ratelimit/config.ts @@ -0,0 +1,109 @@ +import type { ConcurrencyLimits } from "./concurrency"; +import type { RateLimitPolicy, RateLimitSurface } from "./token-bucket"; + +/** Canonical bounded surface order shared by config, runtime admission, metrics, and docs. */ +export const RATE_LIMIT_SURFACES = Object.freeze([ + "management", + "responses-http", + "responses-websocket", + "chat-completions", + "claude-messages", + "images", + "search", + "live", + "model-discovery", +] as const satisfies readonly RateLimitSurface[]); + +export interface RateLimitWebSocketConfig extends ConcurrencyLimits { + maxTrackedPrincipals: number; +} + +/** + * Persisted/user-facing shape after canonical schema validation. + * + * This module deliberately does not parse unknown input. `src/config.ts` remains the single + * Zod validation boundary; the resolver below only completes an already-validated value with + * fixed defaults for runtime use. + */ +export interface RateLimitConfigInput { + enabled?: boolean; + bypassLoopback?: boolean; + maxBuckets?: number; + staleAfterMs?: number; + policies?: Partial>; + webSocket?: Partial; +} + +export interface ResolvedRateLimitConfig { + enabled: true; + bypassLoopback: boolean; + maxBuckets: number; + staleAfterMs: number; + policies: Readonly>>; + webSocket: Readonly; +} + +export const DEFAULT_RATE_LIMIT_MAX_BUCKETS = 10_000; +export const DEFAULT_RATE_LIMIT_STALE_AFTER_MS = 10 * 60_000; + +/** + * Complete process-local defaults. Rate limiting remains disabled unless `enabled: true` is + * present, but once enabled no supported surface silently stays unlimited. + */ +export const DEFAULT_RATE_LIMIT_POLICIES: Readonly>> = Object.freeze({ + management: Object.freeze({ requestsPerMinute: 240, burst: 60 }), + "responses-http": Object.freeze({ requestsPerMinute: 120, burst: 20 }), + "responses-websocket": Object.freeze({ requestsPerMinute: 60, burst: 10 }), + "chat-completions": Object.freeze({ requestsPerMinute: 120, burst: 20 }), + "claude-messages": Object.freeze({ requestsPerMinute: 120, burst: 20 }), + images: Object.freeze({ requestsPerMinute: 20, burst: 4 }), + search: Object.freeze({ requestsPerMinute: 60, burst: 10 }), + live: Object.freeze({ requestsPerMinute: 60, burst: 10 }), + "model-discovery": Object.freeze({ requestsPerMinute: 240, burst: 60 }), +}); + +export const DEFAULT_RATE_LIMIT_WEBSOCKET: Readonly = Object.freeze({ + perPrincipal: 4, + global: 64, + maxTrackedPrincipals: 10_000, +}); + +function resolvedPolicies( + overrides: RateLimitConfigInput["policies"], +): Readonly>> { + return Object.freeze({ + management: Object.freeze({ ...DEFAULT_RATE_LIMIT_POLICIES.management, ...overrides?.management }), + "responses-http": Object.freeze({ ...DEFAULT_RATE_LIMIT_POLICIES["responses-http"], ...overrides?.["responses-http"] }), + "responses-websocket": Object.freeze({ ...DEFAULT_RATE_LIMIT_POLICIES["responses-websocket"], ...overrides?.["responses-websocket"] }), + "chat-completions": Object.freeze({ ...DEFAULT_RATE_LIMIT_POLICIES["chat-completions"], ...overrides?.["chat-completions"] }), + "claude-messages": Object.freeze({ ...DEFAULT_RATE_LIMIT_POLICIES["claude-messages"], ...overrides?.["claude-messages"] }), + images: Object.freeze({ ...DEFAULT_RATE_LIMIT_POLICIES.images, ...overrides?.images }), + search: Object.freeze({ ...DEFAULT_RATE_LIMIT_POLICIES.search, ...overrides?.search }), + live: Object.freeze({ ...DEFAULT_RATE_LIMIT_POLICIES.live, ...overrides?.live }), + "model-discovery": Object.freeze({ ...DEFAULT_RATE_LIMIT_POLICIES["model-discovery"], ...overrides?.["model-discovery"] }), + }); +} + +/** + * Complete an already-schema-validated config for one process lifetime. + * + * Absent/disabled input returns null so existing runtime behavior remains byte-compatible. The + * returned value is detached and deeply frozen: callers cannot mutate defaults, persisted config, + * or another server instance through shared nested objects. + */ +export function resolveValidatedRateLimitConfig( + config: Readonly | undefined, +): Readonly | null { + if (config?.enabled !== true) return null; + return Object.freeze({ + enabled: true as const, + bypassLoopback: config.bypassLoopback === true, + maxBuckets: config.maxBuckets ?? DEFAULT_RATE_LIMIT_MAX_BUCKETS, + staleAfterMs: config.staleAfterMs ?? DEFAULT_RATE_LIMIT_STALE_AFTER_MS, + policies: resolvedPolicies(config.policies), + webSocket: Object.freeze({ + ...DEFAULT_RATE_LIMIT_WEBSOCKET, + ...config.webSocket, + }), + }); +} From fcbc9ba46870b0c311291082867d7e4336170b83 Mon Sep 17 00:00:00 2001 From: OnlineChef Date: Sun, 2 Aug 2026 12:32:08 +0200 Subject: [PATCH 03/18] feat(ratelimit): export runtime config contract --- src/ratelimit/index.ts | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/src/ratelimit/index.ts b/src/ratelimit/index.ts index 8aff3fd11..88dd9212a 100644 --- a/src/ratelimit/index.ts +++ b/src/ratelimit/index.ts @@ -21,3 +21,14 @@ export { type ConcurrencyReservation, type ConcurrencyStats, } from "./concurrency"; +export { + DEFAULT_RATE_LIMIT_MAX_BUCKETS, + DEFAULT_RATE_LIMIT_POLICIES, + DEFAULT_RATE_LIMIT_STALE_AFTER_MS, + DEFAULT_RATE_LIMIT_WEBSOCKET, + RATE_LIMIT_SURFACES, + resolveValidatedRateLimitConfig, + type RateLimitConfigInput, + type RateLimitWebSocketConfig, + type ResolvedRateLimitConfig, +} from "./config"; From 605592eab1ec2d07ac47eef7fbabb2fa335c0702 Mon Sep 17 00:00:00 2001 From: OnlineChef Date: Sun, 2 Aug 2026 12:32:26 +0200 Subject: [PATCH 04/18] test(ratelimit): cover complete runtime config defaults --- tests/ratelimit-config-defaults.test.ts | 92 +++++++++++++++++++++++++ 1 file changed, 92 insertions(+) create mode 100644 tests/ratelimit-config-defaults.test.ts diff --git a/tests/ratelimit-config-defaults.test.ts b/tests/ratelimit-config-defaults.test.ts new file mode 100644 index 000000000..fd56fa65f --- /dev/null +++ b/tests/ratelimit-config-defaults.test.ts @@ -0,0 +1,92 @@ +import { describe, expect, test } from "bun:test"; +import { + DEFAULT_RATE_LIMIT_MAX_BUCKETS, + DEFAULT_RATE_LIMIT_POLICIES, + DEFAULT_RATE_LIMIT_STALE_AFTER_MS, + DEFAULT_RATE_LIMIT_WEBSOCKET, + RATE_LIMIT_SURFACES, + resolveValidatedRateLimitConfig, +} from "../src/ratelimit"; + +describe("rate-limit runtime config defaults", () => { + test("absent and disabled config preserve the existing unlimited runtime", () => { + expect(resolveValidatedRateLimitConfig(undefined)).toBeNull(); + expect(resolveValidatedRateLimitConfig({})).toBeNull(); + expect(resolveValidatedRateLimitConfig({ enabled: false })).toBeNull(); + }); + + test("enabled config resolves every bounded surface and WebSocket limit", () => { + const resolved = resolveValidatedRateLimitConfig({ enabled: true }); + expect(resolved).not.toBeNull(); + expect(Object.keys(resolved!.policies)).toEqual([...RATE_LIMIT_SURFACES]); + expect(resolved).toMatchObject({ + enabled: true, + bypassLoopback: false, + maxBuckets: DEFAULT_RATE_LIMIT_MAX_BUCKETS, + staleAfterMs: DEFAULT_RATE_LIMIT_STALE_AFTER_MS, + webSocket: DEFAULT_RATE_LIMIT_WEBSOCKET, + }); + + for (const surface of RATE_LIMIT_SURFACES) { + expect(resolved!.policies[surface]).toEqual(DEFAULT_RATE_LIMIT_POLICIES[surface]); + expect(Number.isFinite(resolved!.policies[surface].requestsPerMinute)).toBe(true); + expect(resolved!.policies[surface].requestsPerMinute).toBeGreaterThan(0); + expect(Number.isInteger(resolved!.policies[surface].burst)).toBe(true); + expect(resolved!.policies[surface].burst).toBeGreaterThan(0); + } + }); + + test("partial validated overrides never leave another surface unprotected", () => { + const resolved = resolveValidatedRateLimitConfig({ + enabled: true, + bypassLoopback: true, + maxBuckets: 123, + staleAfterMs: 45_000, + policies: { + images: { requestsPerMinute: 7, burst: 2 }, + }, + webSocket: { + perPrincipal: 2, + }, + }); + + expect(resolved).toMatchObject({ + bypassLoopback: true, + maxBuckets: 123, + staleAfterMs: 45_000, + webSocket: { + perPrincipal: 2, + global: DEFAULT_RATE_LIMIT_WEBSOCKET.global, + maxTrackedPrincipals: DEFAULT_RATE_LIMIT_WEBSOCKET.maxTrackedPrincipals, + }, + }); + expect(resolved!.policies.images).toEqual({ requestsPerMinute: 7, burst: 2 }); + expect(resolved!.policies.management).toEqual(DEFAULT_RATE_LIMIT_POLICIES.management); + expect(Object.keys(resolved!.policies)).toHaveLength(RATE_LIMIT_SURFACES.length); + }); + + test("resolved objects are detached and deeply frozen", () => { + const input = { + enabled: true as const, + policies: { + search: { requestsPerMinute: 9, burst: 3 }, + }, + webSocket: { + global: 8, + }, + }; + const resolved = resolveValidatedRateLimitConfig(input)!; + + expect(Object.isFrozen(resolved)).toBe(true); + expect(Object.isFrozen(resolved.policies)).toBe(true); + expect(Object.isFrozen(resolved.policies.search)).toBe(true); + expect(Object.isFrozen(resolved.webSocket)).toBe(true); + + input.policies.search.requestsPerMinute = 99; + input.webSocket.global = 99; + expect(resolved.policies.search.requestsPerMinute).toBe(9); + expect(resolved.webSocket.global).toBe(8); + expect(DEFAULT_RATE_LIMIT_POLICIES.search.requestsPerMinute).toBe(60); + expect(DEFAULT_RATE_LIMIT_WEBSOCKET.global).toBe(64); + }); +}); From be44cb7f00440f25ba67411b14d89ff9768353f2 Mon Sep 17 00:00:00 2001 From: OnlineChef Date: Sun, 2 Aug 2026 12:34:30 +0200 Subject: [PATCH 05/18] feat(ratelimit): add inert runtime admission coordinator --- src/server/rate-limit-admission.ts | 140 +++++++++++++++++++++++++++++ 1 file changed, 140 insertions(+) create mode 100644 src/server/rate-limit-admission.ts diff --git a/src/server/rate-limit-admission.ts b/src/server/rate-limit-admission.ts new file mode 100644 index 000000000..b92c9c782 --- /dev/null +++ b/src/server/rate-limit-admission.ts @@ -0,0 +1,140 @@ +import { + TokenBucketLimiter, + WebSocketConcurrencyLimiter, + type ConcurrencyReservation, + type ConcurrencyStats, + type RateLimitDecision, + type RateLimitPrincipal, + type RateLimitStatsRow, + type RateLimitSurface, + type ResolvedRateLimitConfig, +} from "../ratelimit"; + +export interface RateLimitRouteInput { + method: string; + pathname: string; + webSocket?: boolean; +} + +export type RuntimeAdmissionDecision = + | { outcome: "disabled" | "bypassed"; surface: RateLimitSurface } + | { outcome: "allowed" | "denied"; surface: RateLimitSurface; decision: RateLimitDecision }; + +export type RuntimeWebSocketReservation = + | { outcome: "disabled" | "bypassed"; allowed: true; release: () => void } + | ({ outcome: "allowed" | "denied" } & ConcurrencyReservation); + +export interface RuntimeAdmissionSnapshot { + rateLimits: readonly RateLimitStatsRow[]; + buckets: Readonly<{ principals: number; overflowSurfaces: number }>; + webSocket: Readonly; +} + +const NOOP_RELEASE = (): void => {}; + +/** Exact stable route-to-surface mapping. Unknown routes are intentionally not charged. */ +export function rateLimitSurfaceForRequest(input: Readonly): RateLimitSurface | null { + const method = input.method.toUpperCase(); + const path = input.pathname; + + if (path === "/metrics" || path === "/api" || path.startsWith("/api/")) return "management"; + if (method === "GET" && path === "/v1/models") return "model-discovery"; + + if (input.webSocket === true) { + if (path === "/v1/responses") return "responses-websocket"; + if (path === "/v1/live" || path === "/v1/realtime" || path === "/v1/realtime/calls") return "live"; + return null; + } + + if (method !== "POST") return null; + if (path === "/v1/responses" || path === "/v1/responses/compact") return "responses-http"; + if (path === "/v1/chat/completions") return "chat-completions"; + if (path === "/v1/messages" || path === "/v1/messages/count_tokens") return "claude-messages"; + if (path === "/v1/images/generations" || path === "/v1/images/edits") return "images"; + if (path === "/v1/alpha/search") return "search"; + if (path === "/v1/live" || path === "/v1/realtime/calls") return "live"; + return null; +} + +export function rateLimitHeaders(decision: Readonly): Readonly> { + const headers: Record = { + "X-RateLimit-Limit": String(Math.max(0, Math.trunc(decision.limit))), + "X-RateLimit-Remaining": String(Math.max(0, Math.trunc(decision.remaining))), + "X-RateLimit-Reset": String(Math.max(0, Math.trunc(decision.resetAfterSeconds))), + }; + if (!decision.allowed) { + headers["Retry-After"] = String(Math.max(1, Math.trunc(decision.retryAfterSeconds))); + } + return Object.freeze(headers); +} + +/** + * One process-local admission boundary. It owns all mutable limiter state but receives only an + * opaque principal and fixed route surface; raw credentials, addresses, origins, and request + * metadata are outside this contract. + */ +export class RuntimeRateLimitAdmission { + private readonly tokenBuckets: TokenBucketLimiter | null; + private readonly webSockets: WebSocketConcurrencyLimiter | null; + + constructor(private readonly config: Readonly | null) { + this.tokenBuckets = config + ? new TokenBucketLimiter({ maxBuckets: config.maxBuckets, staleAfterMs: config.staleAfterMs }) + : null; + this.webSockets = config + ? new WebSocketConcurrencyLimiter({ maxTrackedPrincipals: config.webSocket.maxTrackedPrincipals }) + : null; + } + + admit( + surface: RateLimitSurface, + principal: RateLimitPrincipal, + options: Readonly<{ isLoopback: boolean }> = { isLoopback: false }, + ): RuntimeAdmissionDecision { + if (!this.config || !this.tokenBuckets) return { outcome: "disabled", surface }; + if (this.config.bypassLoopback && options.isLoopback) return { outcome: "bypassed", surface }; + const decision = this.tokenBuckets.consume(surface, principal, this.config.policies[surface]); + return { + outcome: decision.allowed ? "allowed" : "denied", + surface, + decision, + }; + } + + reserveWebSocket( + principal: RateLimitPrincipal, + options: Readonly<{ isLoopback: boolean }> = { isLoopback: false }, + ): RuntimeWebSocketReservation { + if (!this.config || !this.webSockets) { + return { outcome: "disabled", allowed: true, release: NOOP_RELEASE }; + } + if (this.config.bypassLoopback && options.isLoopback) { + return { outcome: "bypassed", allowed: true, release: NOOP_RELEASE }; + } + const reservation = this.webSockets.reserve(principal, this.config.webSocket); + return { + outcome: reservation.allowed ? "allowed" : "denied", + ...reservation, + }; + } + + snapshotForTests(): Readonly { + return Object.freeze({ + rateLimits: this.tokenBuckets?.statsSnapshot() ?? Object.freeze([]), + buckets: this.tokenBuckets?.bucketCounts() ?? Object.freeze({ principals: 0, overflowSurfaces: 0 }), + webSocket: this.webSockets?.statsSnapshot() ?? Object.freeze({ + currentGlobal: 0, + currentPrincipals: 0, + accepted: 0, + deniedGlobal: 0, + deniedPrincipal: 0, + deniedPrincipalCapacity: 0, + }), + }); + } + + resetForTests(): void { + this.tokenBuckets?.reset(); + this.webSockets?.reset(); + } +} From 37f0deac7ec03aabfcc0506d553b8ba3ab6c8d9f Mon Sep 17 00:00:00 2001 From: OnlineChef Date: Sun, 2 Aug 2026 12:35:17 +0200 Subject: [PATCH 06/18] fix(ratelimit): align admission coordinator with concurrency contract --- src/server/rate-limit-admission.ts | 48 ++++++++++++++++++++++-------- 1 file changed, 36 insertions(+), 12 deletions(-) diff --git a/src/server/rate-limit-admission.ts b/src/server/rate-limit-admission.ts index b92c9c782..ab535e022 100644 --- a/src/server/rate-limit-admission.ts +++ b/src/server/rate-limit-admission.ts @@ -21,13 +21,23 @@ export type RuntimeAdmissionDecision = | { outcome: "allowed" | "denied"; surface: RateLimitSurface; decision: RateLimitDecision }; export type RuntimeWebSocketReservation = - | { outcome: "disabled" | "bypassed"; allowed: true; release: () => void } + | { + outcome: "disabled" | "bypassed"; + accepted: true; + principalCount: 0; + globalCount: 0; + release: () => void; + } | ({ outcome: "allowed" | "denied" } & ConcurrencyReservation); export interface RuntimeAdmissionSnapshot { rateLimits: readonly RateLimitStatsRow[]; buckets: Readonly<{ principals: number; overflowSurfaces: number }>; - webSocket: Readonly; + webSocket: Readonly<{ + globalCount: number; + trackedPrincipals: number; + stats: Readonly; + }>; } const NOOP_RELEASE = (): void => {}; @@ -106,14 +116,26 @@ export class RuntimeRateLimitAdmission { options: Readonly<{ isLoopback: boolean }> = { isLoopback: false }, ): RuntimeWebSocketReservation { if (!this.config || !this.webSockets) { - return { outcome: "disabled", allowed: true, release: NOOP_RELEASE }; + return { + outcome: "disabled", + accepted: true, + principalCount: 0, + globalCount: 0, + release: NOOP_RELEASE, + }; } if (this.config.bypassLoopback && options.isLoopback) { - return { outcome: "bypassed", allowed: true, release: NOOP_RELEASE }; + return { + outcome: "bypassed", + accepted: true, + principalCount: 0, + globalCount: 0, + release: NOOP_RELEASE, + }; } const reservation = this.webSockets.reserve(principal, this.config.webSocket); return { - outcome: reservation.allowed ? "allowed" : "denied", + outcome: reservation.accepted ? "allowed" : "denied", ...reservation, }; } @@ -122,13 +144,15 @@ export class RuntimeRateLimitAdmission { return Object.freeze({ rateLimits: this.tokenBuckets?.statsSnapshot() ?? Object.freeze([]), buckets: this.tokenBuckets?.bucketCounts() ?? Object.freeze({ principals: 0, overflowSurfaces: 0 }), - webSocket: this.webSockets?.statsSnapshot() ?? Object.freeze({ - currentGlobal: 0, - currentPrincipals: 0, - accepted: 0, - deniedGlobal: 0, - deniedPrincipal: 0, - deniedPrincipalCapacity: 0, + webSocket: this.webSockets?.snapshot() ?? Object.freeze({ + globalCount: 0, + trackedPrincipals: 0, + stats: Object.freeze({ + accepted: 0, + deniedGlobal: 0, + deniedPrincipal: 0, + deniedPrincipalCapacity: 0, + }), }), }); } From 1ea5eb711c9e2d832133f6f5181a61405a6375cb Mon Sep 17 00:00:00 2001 From: OnlineChef Date: Sun, 2 Aug 2026 12:35:52 +0200 Subject: [PATCH 07/18] test(ratelimit): cover inert admission coordinator --- tests/rate-limit-admission.test.ts | 152 +++++++++++++++++++++++++++++ 1 file changed, 152 insertions(+) create mode 100644 tests/rate-limit-admission.test.ts diff --git a/tests/rate-limit-admission.test.ts b/tests/rate-limit-admission.test.ts new file mode 100644 index 000000000..1dd625306 --- /dev/null +++ b/tests/rate-limit-admission.test.ts @@ -0,0 +1,152 @@ +import { describe, expect, test } from "bun:test"; +import { PrincipalFingerprinter, resolveValidatedRateLimitConfig } from "../src/ratelimit"; +import { + RuntimeRateLimitAdmission, + rateLimitHeaders, + rateLimitSurfaceForRequest, +} from "../src/server/rate-limit-admission"; + +const fingerprinter = new PrincipalFingerprinter(new Uint8Array(32).fill(0x47)); +const principalA = fingerprinter.admissionKey("admission-a"); +const principalB = fingerprinter.admissionKey("admission-b"); + +function enabledAdmission(options: { bypassLoopback?: boolean } = {}) { + const config = resolveValidatedRateLimitConfig({ + enabled: true, + bypassLoopback: options.bypassLoopback, + policies: { + "responses-http": { requestsPerMinute: 1, burst: 1 }, + }, + webSocket: { + perPrincipal: 1, + global: 2, + maxTrackedPrincipals: 2, + }, + }); + return new RuntimeRateLimitAdmission(config); +} + +describe("rate-limit route mapping", () => { + test("maps every protected HTTP surface exactly", () => { + const cases = [ + ["GET", "/api/settings", false, "management"], + ["GET", "/metrics", false, "management"], + ["GET", "/v1/models", false, "model-discovery"], + ["POST", "/v1/responses", false, "responses-http"], + ["POST", "/v1/responses/compact", false, "responses-http"], + ["POST", "/v1/chat/completions", false, "chat-completions"], + ["POST", "/v1/messages", false, "claude-messages"], + ["POST", "/v1/messages/count_tokens", false, "claude-messages"], + ["POST", "/v1/images/generations", false, "images"], + ["POST", "/v1/images/edits", false, "images"], + ["POST", "/v1/alpha/search", false, "search"], + ["POST", "/v1/live", false, "live"], + ["POST", "/v1/realtime/calls", false, "live"], + ["GET", "/v1/responses", true, "responses-websocket"], + ["GET", "/v1/realtime", true, "live"], + ] as const; + + for (const [method, pathname, webSocket, expected] of cases) { + expect(rateLimitSurfaceForRequest({ method, pathname, webSocket })).toBe(expected); + } + }); + + test("does not broaden protection through prefixes or unsupported methods", () => { + expect(rateLimitSurfaceForRequest({ method: "GET", pathname: "/healthz" })).toBeNull(); + expect(rateLimitSurfaceForRequest({ method: "GET", pathname: "/v1/responses" })).toBeNull(); + expect(rateLimitSurfaceForRequest({ method: "POST", pathname: "/v1/models" })).toBeNull(); + expect(rateLimitSurfaceForRequest({ method: "POST", pathname: "/v1/responses-extra" })).toBeNull(); + expect(rateLimitSurfaceForRequest({ method: "GET", pathname: "/v1/unknown", webSocket: true })).toBeNull(); + }); +}); + +describe("runtime rate-limit admission", () => { + test("disabled admission is a true no-op with no allocated state", () => { + const admission = new RuntimeRateLimitAdmission(null); + expect(admission.admit("responses-http", principalA, { isLoopback: false })).toEqual({ + outcome: "disabled", + surface: "responses-http", + }); + const reservation = admission.reserveWebSocket(principalA, { isLoopback: false }); + expect(reservation).toMatchObject({ outcome: "disabled", accepted: true }); + reservation.release(); + expect(admission.snapshotForTests()).toEqual({ + rateLimits: [], + buckets: { principals: 0, overflowSurfaces: 0 }, + webSocket: { + globalCount: 0, + trackedPrincipals: 0, + stats: { + accepted: 0, + deniedGlobal: 0, + deniedPrincipal: 0, + deniedPrincipalCapacity: 0, + }, + }, + }); + }); + + test("explicit loopback bypass consumes no token or concurrency state", () => { + const admission = enabledAdmission({ bypassLoopback: true }); + expect(admission.admit("responses-http", principalA, { isLoopback: true })).toEqual({ + outcome: "bypassed", + surface: "responses-http", + }); + expect(admission.reserveWebSocket(principalA, { isLoopback: true })).toMatchObject({ + outcome: "bypassed", + accepted: true, + }); + expect(admission.snapshotForTests().buckets.principals).toBe(0); + expect(admission.snapshotForTests().webSocket.globalCount).toBe(0); + }); + + test("tokens are isolated by opaque principal and expose bounded headers only", () => { + const admission = enabledAdmission(); + expect(admission.admit("responses-http", principalA, { isLoopback: false }).outcome).toBe("allowed"); + const denied = admission.admit("responses-http", principalA, { isLoopback: false }); + expect(denied.outcome).toBe("denied"); + if (denied.outcome !== "denied") throw new Error("expected denial"); + expect(rateLimitHeaders(denied.decision)).toEqual({ + "Retry-After": "60", + "X-RateLimit-Limit": "1", + "X-RateLimit-Remaining": "0", + "X-RateLimit-Reset": "60", + }); + + expect(admission.admit("responses-http", principalB, { isLoopback: false }).outcome).toBe("allowed"); + expect(JSON.stringify(admission.snapshotForTests())).not.toContain(principalA.fingerprint); + expect(JSON.stringify(admission.snapshotForTests())).not.toContain(principalB.fingerprint); + }); + + test("WebSocket reservation enforces principal/global caps and releases idempotently", () => { + const admission = enabledAdmission(); + const first = admission.reserveWebSocket(principalA, { isLoopback: false }); + expect(first).toMatchObject({ outcome: "allowed", accepted: true, principalCount: 1, globalCount: 1 }); + + const samePrincipal = admission.reserveWebSocket(principalA, { isLoopback: false }); + expect(samePrincipal).toMatchObject({ outcome: "denied", accepted: false, reason: "principal_limit" }); + + const second = admission.reserveWebSocket(principalB, { isLoopback: false }); + expect(second).toMatchObject({ outcome: "allowed", accepted: true, globalCount: 2 }); + + first.release(); + first.release(); + expect(admission.snapshotForTests().webSocket.globalCount).toBe(1); + second.release(); + expect(admission.snapshotForTests().webSocket.globalCount).toBe(0); + }); + + test("reset invalidates stale release handles", () => { + const admission = enabledAdmission(); + const beforeReset = admission.reserveWebSocket(principalA, { isLoopback: false }); + expect(beforeReset.accepted).toBe(true); + admission.resetForTests(); + + const afterReset = admission.reserveWebSocket(principalA, { isLoopback: false }); + expect(afterReset).toMatchObject({ accepted: true, globalCount: 1 }); + beforeReset.release(); + expect(admission.snapshotForTests().webSocket.globalCount).toBe(1); + afterReset.release(); + expect(admission.snapshotForTests().webSocket.globalCount).toBe(0); + }); +}); From 0311f4b1e8eb5d09ac8411397045bb7d56e09b36 Mon Sep 17 00:00:00 2001 From: OnlineChef Date: Sun, 2 Aug 2026 12:36:38 +0200 Subject: [PATCH 08/18] fix(ratelimit): narrow WebSocket reservation unions in tests --- tests/rate-limit-admission.test.ts | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/tests/rate-limit-admission.test.ts b/tests/rate-limit-admission.test.ts index 1dd625306..636da7b28 100644 --- a/tests/rate-limit-admission.test.ts +++ b/tests/rate-limit-admission.test.ts @@ -9,6 +9,7 @@ import { const fingerprinter = new PrincipalFingerprinter(new Uint8Array(32).fill(0x47)); const principalA = fingerprinter.admissionKey("admission-a"); const principalB = fingerprinter.admissionKey("admission-b"); +const principalC = fingerprinter.admissionKey("admission-c"); function enabledAdmission(options: { bypassLoopback?: boolean } = {}) { const config = resolveValidatedRateLimitConfig({ @@ -20,7 +21,7 @@ function enabledAdmission(options: { bypassLoopback?: boolean } = {}) { webSocket: { perPrincipal: 1, global: 2, - maxTrackedPrincipals: 2, + maxTrackedPrincipals: 3, }, }); return new RuntimeRateLimitAdmission(config); @@ -69,6 +70,7 @@ describe("runtime rate-limit admission", () => { }); const reservation = admission.reserveWebSocket(principalA, { isLoopback: false }); expect(reservation).toMatchObject({ outcome: "disabled", accepted: true }); + if (!reservation.accepted) throw new Error("disabled reservation must be accepted"); reservation.release(); expect(admission.snapshotForTests()).toEqual({ rateLimits: [], @@ -122,12 +124,17 @@ describe("runtime rate-limit admission", () => { const admission = enabledAdmission(); const first = admission.reserveWebSocket(principalA, { isLoopback: false }); expect(first).toMatchObject({ outcome: "allowed", accepted: true, principalCount: 1, globalCount: 1 }); + if (!first.accepted) throw new Error("first reservation must be accepted"); const samePrincipal = admission.reserveWebSocket(principalA, { isLoopback: false }); expect(samePrincipal).toMatchObject({ outcome: "denied", accepted: false, reason: "principal_limit" }); const second = admission.reserveWebSocket(principalB, { isLoopback: false }); expect(second).toMatchObject({ outcome: "allowed", accepted: true, globalCount: 2 }); + if (!second.accepted) throw new Error("second reservation must be accepted"); + + const globalDenied = admission.reserveWebSocket(principalC, { isLoopback: false }); + expect(globalDenied).toMatchObject({ outcome: "denied", accepted: false, reason: "global_limit" }); first.release(); first.release(); @@ -140,10 +147,12 @@ describe("runtime rate-limit admission", () => { const admission = enabledAdmission(); const beforeReset = admission.reserveWebSocket(principalA, { isLoopback: false }); expect(beforeReset.accepted).toBe(true); + if (!beforeReset.accepted) throw new Error("pre-reset reservation must be accepted"); admission.resetForTests(); const afterReset = admission.reserveWebSocket(principalA, { isLoopback: false }); expect(afterReset).toMatchObject({ accepted: true, globalCount: 1 }); + if (!afterReset.accepted) throw new Error("post-reset reservation must be accepted"); beforeReset.release(); expect(admission.snapshotForTests().webSocket.globalCount).toBe(1); afterReset.release(); From e7c3d03143432ab391a07b1db92e7ef476802ced Mon Sep 17 00:00:00 2001 From: OnlineChef Date: Sun, 2 Aug 2026 12:39:51 +0200 Subject: [PATCH 09/18] feat(ratelimit): add inert authenticated-principal boundary --- src/server/rate-limit-auth-principal.ts | 45 +++++++++++++++++++++++++ 1 file changed, 45 insertions(+) create mode 100644 src/server/rate-limit-auth-principal.ts diff --git a/src/server/rate-limit-auth-principal.ts b/src/server/rate-limit-auth-principal.ts new file mode 100644 index 000000000..2f45c7d95 --- /dev/null +++ b/src/server/rate-limit-auth-principal.ts @@ -0,0 +1,45 @@ +import { + rateLimitFingerprinter, + type PrincipalFingerprinter, + type RateLimitPrincipal, +} from "../ratelimit"; + +/** + * Convert a credential that the existing data-plane auth boundary already accepted. + * + * This function never inspects Request headers. Callers must first prove the value is an + * OpenCodex admission secret (for example through isDataPlaneAdmissionSecret). Upstream bearer + * credentials, provider API keys, arbitrary x-api-key values, Origin, and Host never enter here. + */ +export function principalForAcceptedAdmissionSecret( + acceptedSecret: string, + fingerprinter: PrincipalFingerprinter = rateLimitFingerprinter, +): RateLimitPrincipal { + return fingerprinter.admissionKey(acceptedSecret); +} + +/** + * Convert an already-accepted management credential (admin token or valid GUI session token). + * CSRF tokens and Origin are not authentication identities and must never be passed here. + */ +export function principalForAcceptedManagementCredential( + acceptedCredential: string, + fingerprinter: PrincipalFingerprinter = rateLimitFingerprinter, +): RateLimitPrincipal { + return fingerprinter.management(acceptedCredential); +} + +/** + * Principal for a successfully admitted request that has no stable OpenCodex credential. + * + * `remoteAddress` must come directly from Bun's trusted server socket API, never from + * Forwarded/X-Forwarded-For or another request header. A missing/blank address intentionally + * collapses to one bounded shared anonymous principal rather than allocating per-header state. + */ +export function principalForAcceptedUnauthenticatedPeer( + remoteAddress: string | null | undefined, + fingerprinter: PrincipalFingerprinter = rateLimitFingerprinter, +): RateLimitPrincipal { + const normalized = remoteAddress?.trim(); + return normalized ? fingerprinter.remoteAddress(normalized) : fingerprinter.anonymous(); +} From 775507c95c75d609a33c10fe00b8ab4f6aedb4d8 Mon Sep 17 00:00:00 2001 From: OnlineChef Date: Sun, 2 Aug 2026 12:40:12 +0200 Subject: [PATCH 10/18] test(ratelimit): cover accepted auth principal boundary --- tests/rate-limit-auth-principal.test.ts | 64 +++++++++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100644 tests/rate-limit-auth-principal.test.ts diff --git a/tests/rate-limit-auth-principal.test.ts b/tests/rate-limit-auth-principal.test.ts new file mode 100644 index 000000000..3203fd791 --- /dev/null +++ b/tests/rate-limit-auth-principal.test.ts @@ -0,0 +1,64 @@ +import { describe, expect, test } from "bun:test"; +import { PrincipalFingerprinter } from "../src/ratelimit"; +import { + principalForAcceptedAdmissionSecret, + principalForAcceptedManagementCredential, + principalForAcceptedUnauthenticatedPeer, +} from "../src/server/rate-limit-auth-principal"; + +const fingerprinter = new PrincipalFingerprinter(new Uint8Array(32).fill(0x61)); + +describe("rate-limit authenticated-principal boundary", () => { + test("accepted admission secrets are opaque, stable, and domain separated", () => { + const secret = "opencodex-admission-secret"; + const first = principalForAcceptedAdmissionSecret(secret, fingerprinter); + const second = principalForAcceptedAdmissionSecret(secret, fingerprinter); + const management = principalForAcceptedManagementCredential(secret, fingerprinter); + + expect(first).toEqual(second); + expect(first.kind).toBe("admission-key"); + expect(management.kind).toBe("management"); + expect(first.fingerprint).not.toBe(management.fingerprint); + expect(first.fingerprint).not.toContain(secret); + expect(management.fingerprint).not.toContain(secret); + }); + + test("admin tokens and GUI sessions share only the management domain, not identity", () => { + const admin = principalForAcceptedManagementCredential("admin-token", fingerprinter); + const gui = principalForAcceptedManagementCredential("gui-session-token", fingerprinter); + + expect(admin.kind).toBe("management"); + expect(gui.kind).toBe("management"); + expect(admin.fingerprint).not.toBe(gui.fingerprint); + }); + + test("trusted socket addresses are distinct from credentials", () => { + const remote = principalForAcceptedUnauthenticatedPeer(" 127.0.0.1 ", fingerprinter); + const sameRemote = principalForAcceptedUnauthenticatedPeer("127.0.0.1", fingerprinter); + const admission = principalForAcceptedAdmissionSecret("127.0.0.1", fingerprinter); + + expect(remote).toEqual(sameRemote); + expect(remote.kind).toBe("remote-address"); + expect(remote.fingerprint).not.toBe(admission.fingerprint); + expect(remote.fingerprint).not.toContain("127.0.0.1"); + }); + + test("missing and blank peer addresses collapse to one shared anonymous principal", () => { + const missing = principalForAcceptedUnauthenticatedPeer(undefined, fingerprinter); + const absent = principalForAcceptedUnauthenticatedPeer(null, fingerprinter); + const blank = principalForAcceptedUnauthenticatedPeer(" ", fingerprinter); + + expect(missing.kind).toBe("anonymous"); + expect(missing).toEqual(absent); + expect(missing).toEqual(blank); + }); + + test("the module exposes no Request/header parser", async () => { + const module = await import("../src/server/rate-limit-auth-principal"); + expect(Object.keys(module).sort()).toEqual([ + "principalForAcceptedAdmissionSecret", + "principalForAcceptedManagementCredential", + "principalForAcceptedUnauthenticatedPeer", + ]); + }); +}); From 2a79f447c1a402f3b9ef9c91ffe3ee730a73ddf7 Mon Sep 17 00:00:00 2001 From: OnlineChef Date: Sun, 2 Aug 2026 12:42:48 +0200 Subject: [PATCH 11/18] feat(ratelimit): add inert API-specific 429 renderers --- src/server/rate-limit-response.ts | 54 +++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) create mode 100644 src/server/rate-limit-response.ts diff --git a/src/server/rate-limit-response.ts b/src/server/rate-limit-response.ts new file mode 100644 index 000000000..26d2fbb2d --- /dev/null +++ b/src/server/rate-limit-response.ts @@ -0,0 +1,54 @@ +import { formatErrorResponse } from "../bridge"; +import { anthropicErrorResponse } from "../claude/outbound"; +import type { RateLimitDecision } from "../ratelimit"; +import { rateLimitHeaders } from "./rate-limit-admission"; + +export type RateLimitResponseKind = "openai" | "anthropic" | "management"; + +const RATE_LIMIT_MESSAGE = "Rate limit exceeded"; + +function assertDenied(decision: Readonly): void { + if (decision.allowed) throw new Error("rate-limit response requires a denied decision"); +} + +function attachRateLimitHeaders(response: Response, decision: Readonly): Response { + for (const [name, value] of Object.entries(rateLimitHeaders(decision))) { + response.headers.set(name, value); + } + response.headers.set("Cache-Control", "no-store"); + return response; +} + +/** + * Render one denied admission decision in the existing API vocabulary. + * + * The decision contains only bounded limiter metadata. Principal identity and request-derived + * values are not accepted by this boundary. Management CORS wrapping remains the caller's job, + * exactly like the existing management auth/error responses. + */ +export function rateLimitResponse( + kind: RateLimitResponseKind, + decision: Readonly, +): Response { + assertDenied(decision); + let response: Response; + switch (kind) { + case "openai": + response = formatErrorResponse(429, "rate_limit_error", RATE_LIMIT_MESSAGE, { + code: "rate_limit_exceeded", + retryAfter: String(Math.max(1, Math.trunc(decision.retryAfterSeconds))), + }); + break; + case "anthropic": + response = anthropicErrorResponse(429, RATE_LIMIT_MESSAGE, "rate_limit_error"); + break; + case "management": + response = Response.json({ error: "rate limit exceeded" }, { status: 429 }); + break; + default: { + const exhaustive: never = kind; + throw new Error(`unsupported rate-limit response kind: ${exhaustive}`); + } + } + return attachRateLimitHeaders(response, decision); +} From 1c39da84c8e3012d3e21cce82eef879a1d748989 Mon Sep 17 00:00:00 2001 From: OnlineChef Date: Sun, 2 Aug 2026 12:43:25 +0200 Subject: [PATCH 12/18] fix(ratelimit): rely on canonical 429 error classification --- src/server/rate-limit-response.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/server/rate-limit-response.ts b/src/server/rate-limit-response.ts index 26d2fbb2d..52eadfdb8 100644 --- a/src/server/rate-limit-response.ts +++ b/src/server/rate-limit-response.ts @@ -34,8 +34,8 @@ export function rateLimitResponse( let response: Response; switch (kind) { case "openai": + // Canonical classifyError(429, ...) supplies rate_limit_error/rate_limit_exceeded. response = formatErrorResponse(429, "rate_limit_error", RATE_LIMIT_MESSAGE, { - code: "rate_limit_exceeded", retryAfter: String(Math.max(1, Math.trunc(decision.retryAfterSeconds))), }); break; From 00f5552d6c3d9240d25bea2a1d1b77fdcb6c1f77 Mon Sep 17 00:00:00 2001 From: OnlineChef Date: Sun, 2 Aug 2026 12:43:44 +0200 Subject: [PATCH 13/18] test(ratelimit): cover API-specific 429 envelopes --- tests/rate-limit-response.test.ts | 74 +++++++++++++++++++++++++++++++ 1 file changed, 74 insertions(+) create mode 100644 tests/rate-limit-response.test.ts diff --git a/tests/rate-limit-response.test.ts b/tests/rate-limit-response.test.ts new file mode 100644 index 000000000..e2b9b01fc --- /dev/null +++ b/tests/rate-limit-response.test.ts @@ -0,0 +1,74 @@ +import { describe, expect, test } from "bun:test"; +import type { RateLimitDecision } from "../src/ratelimit"; +import { rateLimitResponse } from "../src/server/rate-limit-response"; + +const denied: RateLimitDecision = { + allowed: false, + surface: "responses-http", + source: "principal", + limit: 20, + remaining: 0, + retryAfterSeconds: 7, + resetAfterSeconds: 19, + reason: "rate_limited", +}; + +async function body(response: Response): Promise> { + return await response.json() as Record; +} + +function expectBoundedHeaders(response: Response): void { + expect(response.status).toBe(429); + expect(response.headers.get("Retry-After")).toBe("7"); + expect(response.headers.get("X-RateLimit-Limit")).toBe("20"); + expect(response.headers.get("X-RateLimit-Remaining")).toBe("0"); + expect(response.headers.get("X-RateLimit-Reset")).toBe("19"); + expect(response.headers.get("Cache-Control")).toBe("no-store"); +} + +describe("rate-limit response envelopes", () => { + test("OpenAI surfaces preserve the canonical error vocabulary", async () => { + const response = rateLimitResponse("openai", denied); + expectBoundedHeaders(response); + expect(response.headers.get("Content-Type")).toContain("application/json"); + expect(await body(response)).toEqual({ + error: { + message: "Rate limit exceeded", + type: "rate_limit_error", + code: "rate_limit_exceeded", + }, + }); + }); + + test("Anthropic surfaces use rate_limit_error", async () => { + const response = rateLimitResponse("anthropic", denied); + expectBoundedHeaders(response); + expect(await body(response)).toEqual({ + type: "error", + error: { + type: "rate_limit_error", + message: "Rate limit exceeded", + }, + }); + }); + + test("management surfaces keep the existing simple error JSON convention", async () => { + const response = rateLimitResponse("management", denied); + expectBoundedHeaders(response); + expect(await body(response)).toEqual({ error: "rate limit exceeded" }); + }); + + test("an allowed decision cannot be rendered as a 429", () => { + expect(() => rateLimitResponse("openai", { ...denied, allowed: true, reason: "allowed" })).toThrow( + "rate-limit response requires a denied decision", + ); + }); + + test("rendered responses contain no identity or dynamic request dimensions", async () => { + for (const kind of ["openai", "anthropic", "management"] as const) { + const response = rateLimitResponse(kind, denied); + const serialized = `${JSON.stringify(await body(response))}\n${[...response.headers].join("\n")}`; + expect(serialized).not.toMatch(/principal|fingerprint|provider|model|account|conversation|origin/i); + } + }); +}); From b3287be1f3ce9351a602f240609bae24b98af61a Mon Sep 17 00:00:00 2001 From: OnlineChef Date: Sun, 2 Aug 2026 12:45:23 +0200 Subject: [PATCH 14/18] fix(ratelimit): share live sideband route recognition --- src/server/rate-limit-admission.ts | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/src/server/rate-limit-admission.ts b/src/server/rate-limit-admission.ts index ab535e022..bdb22d9bd 100644 --- a/src/server/rate-limit-admission.ts +++ b/src/server/rate-limit-admission.ts @@ -9,10 +9,12 @@ import { type RateLimitSurface, type ResolvedRateLimitConfig, } from "../ratelimit"; +import { parseLiveSidebandTarget } from "./live"; export interface RateLimitRouteInput { method: string; pathname: string; + searchParams?: URLSearchParams; webSocket?: boolean; } @@ -42,6 +44,20 @@ export interface RuntimeAdmissionSnapshot { const NOOP_RELEASE = (): void => {}; +function isSupportedLiveWebSocketRoute(input: Readonly): boolean { + if (input.method.toUpperCase() !== "GET") return false; + try { + return parseLiveSidebandTarget( + input.pathname, + input.searchParams ?? new URLSearchParams(), + ) !== null; + } catch { + // Malformed percent-encoding or another invalid route remains unclassified. The live + // dispatcher applies the same parser later; admission must never broaden support. + return false; + } +} + /** Exact stable route-to-surface mapping. Unknown routes are intentionally not charged. */ export function rateLimitSurfaceForRequest(input: Readonly): RateLimitSurface | null { const method = input.method.toUpperCase(); @@ -51,8 +67,8 @@ export function rateLimitSurfaceForRequest(input: Readonly) if (method === "GET" && path === "/v1/models") return "model-discovery"; if (input.webSocket === true) { - if (path === "/v1/responses") return "responses-websocket"; - if (path === "/v1/live" || path === "/v1/realtime" || path === "/v1/realtime/calls") return "live"; + if (method === "GET" && path === "/v1/responses") return "responses-websocket"; + if (isSupportedLiveWebSocketRoute(input)) return "live"; return null; } From 96e084cffe5de983a6f8b41523bedbcf307dace7 Mon Sep 17 00:00:00 2001 From: OnlineChef Date: Sun, 2 Aug 2026 12:46:14 +0200 Subject: [PATCH 15/18] test(ratelimit): cover canonical live sideband routes --- tests/rate-limit-admission.test.ts | 58 ++++++++++++++++++++---------- 1 file changed, 39 insertions(+), 19 deletions(-) diff --git a/tests/rate-limit-admission.test.ts b/tests/rate-limit-admission.test.ts index 636da7b28..858815dd5 100644 --- a/tests/rate-limit-admission.test.ts +++ b/tests/rate-limit-admission.test.ts @@ -28,36 +28,56 @@ function enabledAdmission(options: { bypassLoopback?: boolean } = {}) { } describe("rate-limit route mapping", () => { - test("maps every protected HTTP surface exactly", () => { + test("maps every protected HTTP and WebSocket surface exactly", () => { const cases = [ - ["GET", "/api/settings", false, "management"], - ["GET", "/metrics", false, "management"], - ["GET", "/v1/models", false, "model-discovery"], - ["POST", "/v1/responses", false, "responses-http"], - ["POST", "/v1/responses/compact", false, "responses-http"], - ["POST", "/v1/chat/completions", false, "chat-completions"], - ["POST", "/v1/messages", false, "claude-messages"], - ["POST", "/v1/messages/count_tokens", false, "claude-messages"], - ["POST", "/v1/images/generations", false, "images"], - ["POST", "/v1/images/edits", false, "images"], - ["POST", "/v1/alpha/search", false, "search"], - ["POST", "/v1/live", false, "live"], - ["POST", "/v1/realtime/calls", false, "live"], - ["GET", "/v1/responses", true, "responses-websocket"], - ["GET", "/v1/realtime", true, "live"], + { method: "GET", pathname: "/api/settings", expected: "management" }, + { method: "GET", pathname: "/metrics", expected: "management" }, + { method: "GET", pathname: "/v1/models", expected: "model-discovery" }, + { method: "POST", pathname: "/v1/responses", expected: "responses-http" }, + { method: "POST", pathname: "/v1/responses/compact", expected: "responses-http" }, + { method: "POST", pathname: "/v1/chat/completions", expected: "chat-completions" }, + { method: "POST", pathname: "/v1/messages", expected: "claude-messages" }, + { method: "POST", pathname: "/v1/messages/count_tokens", expected: "claude-messages" }, + { method: "POST", pathname: "/v1/images/generations", expected: "images" }, + { method: "POST", pathname: "/v1/images/edits", expected: "images" }, + { method: "POST", pathname: "/v1/alpha/search", expected: "search" }, + { method: "POST", pathname: "/v1/live", expected: "live" }, + { method: "POST", pathname: "/v1/realtime/calls", expected: "live" }, + { method: "GET", pathname: "/v1/responses", webSocket: true, expected: "responses-websocket" }, + { method: "GET", pathname: "/v1/live/call_123", webSocket: true, expected: "live" }, + { method: "GET", pathname: "/v1/live/call_123/", webSocket: true, expected: "live" }, + { method: "GET", pathname: "/v1/realtime/calls/call_123", webSocket: true, expected: "live" }, + { method: "GET", pathname: "/v1/realtime", search: "call_id=call_123", webSocket: true, expected: "live" }, + { method: "GET", pathname: "/v1/realtime/", search: "call_id=call_123", webSocket: true, expected: "live" }, ] as const; - for (const [method, pathname, webSocket, expected] of cases) { - expect(rateLimitSurfaceForRequest({ method, pathname, webSocket })).toBe(expected); + for (const { method, pathname, search, webSocket, expected } of cases) { + expect(rateLimitSurfaceForRequest({ + method, + pathname, + ...(search ? { searchParams: new URLSearchParams(search) } : {}), + ...(webSocket ? { webSocket: true } : {}), + })).toBe(expected); } }); - test("does not broaden protection through prefixes or unsupported methods", () => { + test("does not broaden protection through prefixes, invalid call ids, or unsupported methods", () => { expect(rateLimitSurfaceForRequest({ method: "GET", pathname: "/healthz" })).toBeNull(); expect(rateLimitSurfaceForRequest({ method: "GET", pathname: "/v1/responses" })).toBeNull(); expect(rateLimitSurfaceForRequest({ method: "POST", pathname: "/v1/models" })).toBeNull(); expect(rateLimitSurfaceForRequest({ method: "POST", pathname: "/v1/responses-extra" })).toBeNull(); expect(rateLimitSurfaceForRequest({ method: "GET", pathname: "/v1/unknown", webSocket: true })).toBeNull(); + expect(rateLimitSurfaceForRequest({ method: "GET", pathname: "/v1/live", webSocket: true })).toBeNull(); + expect(rateLimitSurfaceForRequest({ method: "GET", pathname: "/v1/realtime", webSocket: true })).toBeNull(); + expect(rateLimitSurfaceForRequest({ + method: "GET", + pathname: "/v1/realtime", + searchParams: new URLSearchParams("call_id=bad/value"), + webSocket: true, + })).toBeNull(); + expect(rateLimitSurfaceForRequest({ method: "GET", pathname: "/v1/live/bad%2Fvalue", webSocket: true })).toBeNull(); + expect(rateLimitSurfaceForRequest({ method: "POST", pathname: "/v1/live/call_123", webSocket: true })).toBeNull(); + expect(rateLimitSurfaceForRequest({ method: "GET", pathname: "/v1/live/call_123/extra", webSocket: true })).toBeNull(); }); }); From 3e07591730d0527d3f78008657deca50c40899ce Mon Sep 17 00:00:00 2001 From: OnlineChef Date: Sun, 2 Aug 2026 12:49:31 +0200 Subject: [PATCH 16/18] feat(ratelimit): add strict canonical config subschema --- src/ratelimit/schema.ts | 61 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+) create mode 100644 src/ratelimit/schema.ts diff --git a/src/ratelimit/schema.ts b/src/ratelimit/schema.ts new file mode 100644 index 000000000..05b30580c --- /dev/null +++ b/src/ratelimit/schema.ts @@ -0,0 +1,61 @@ +import * as z from "zod/v4"; +import type { RateLimitConfigInput } from "./config"; + +export const MAX_RATE_LIMIT_REQUESTS_PER_MINUTE = 1_000_000; +export const MAX_RATE_LIMIT_BURST = 100_000; +export const MAX_RATE_LIMIT_BUCKETS = 1_000_000; +export const MIN_RATE_LIMIT_STALE_AFTER_MS = 1_000; +export const MAX_RATE_LIMIT_STALE_AFTER_MS = 24 * 60 * 60_000; +export const MAX_RATE_LIMIT_WEBSOCKET_CONCURRENCY = 100_000; +export const MAX_RATE_LIMIT_TRACKED_PRINCIPALS = 1_000_000; + +export const rateLimitPolicySchema = z.object({ + requestsPerMinute: z.number().finite().positive().max(MAX_RATE_LIMIT_REQUESTS_PER_MINUTE), + burst: z.number().int().positive().max(MAX_RATE_LIMIT_BURST), +}).strict(); + +const rateLimitPoliciesSchema = z.object({ + management: rateLimitPolicySchema.optional(), + "responses-http": rateLimitPolicySchema.optional(), + "responses-websocket": rateLimitPolicySchema.optional(), + "chat-completions": rateLimitPolicySchema.optional(), + "claude-messages": rateLimitPolicySchema.optional(), + images: rateLimitPolicySchema.optional(), + search: rateLimitPolicySchema.optional(), + live: rateLimitPolicySchema.optional(), + "model-discovery": rateLimitPolicySchema.optional(), +}).strict(); + +const rateLimitWebSocketSchema = z.object({ + perPrincipal: z.number().int().positive().max(MAX_RATE_LIMIT_WEBSOCKET_CONCURRENCY).optional(), + global: z.number().int().positive().max(MAX_RATE_LIMIT_WEBSOCKET_CONCURRENCY).optional(), + maxTrackedPrincipals: z.number().int().positive().max(MAX_RATE_LIMIT_TRACKED_PRINCIPALS).optional(), +}).strict().superRefine((value, ctx) => { + if (value.perPrincipal !== undefined && value.global !== undefined && value.perPrincipal > value.global) { + ctx.addIssue({ + code: "custom", + path: ["perPrincipal"], + message: "perPrincipal must not exceed global", + }); + } +}); + +/** + * Canonical persisted `rateLimit` subschema. + * + * The root config intentionally remains passthrough for historic compatibility, so this nested + * object must be strict: unknown surfaces or runtime-only fields are rejected instead of silently + * surviving load/save. Complete fixed defaults are applied later by resolveValidatedRateLimitConfig. + */ +export const rateLimitConfigSchema: z.ZodType = z.object({ + enabled: z.boolean().optional(), + bypassLoopback: z.boolean().optional(), + maxBuckets: z.number().int().positive().max(MAX_RATE_LIMIT_BUCKETS).optional(), + staleAfterMs: z.number() + .int() + .min(MIN_RATE_LIMIT_STALE_AFTER_MS) + .max(MAX_RATE_LIMIT_STALE_AFTER_MS) + .optional(), + policies: rateLimitPoliciesSchema.optional(), + webSocket: rateLimitWebSocketSchema.optional(), +}).strict(); From 0e24ede65850cc297cafb9c9ac8fdca3d88eba50 Mon Sep 17 00:00:00 2001 From: OnlineChef Date: Sun, 2 Aug 2026 12:49:57 +0200 Subject: [PATCH 17/18] feat(ratelimit): export canonical config schema --- src/ratelimit/index.ts | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/src/ratelimit/index.ts b/src/ratelimit/index.ts index 88dd9212a..758d98d17 100644 --- a/src/ratelimit/index.ts +++ b/src/ratelimit/index.ts @@ -32,3 +32,14 @@ export { type RateLimitWebSocketConfig, type ResolvedRateLimitConfig, } from "./config"; +export { + MAX_RATE_LIMIT_BUCKETS, + MAX_RATE_LIMIT_BURST, + MAX_RATE_LIMIT_REQUESTS_PER_MINUTE, + MAX_RATE_LIMIT_STALE_AFTER_MS, + MAX_RATE_LIMIT_TRACKED_PRINCIPALS, + MAX_RATE_LIMIT_WEBSOCKET_CONCURRENCY, + MIN_RATE_LIMIT_STALE_AFTER_MS, + rateLimitConfigSchema, + rateLimitPolicySchema, +} from "./schema"; From c4c286defaa759038084cae161d171c9e9324d2b Mon Sep 17 00:00:00 2001 From: OnlineChef Date: Sun, 2 Aug 2026 12:50:22 +0200 Subject: [PATCH 18/18] test(ratelimit): cover strict canonical config schema --- tests/ratelimit-config-schema.test.ts | 92 +++++++++++++++++++++++++++ 1 file changed, 92 insertions(+) create mode 100644 tests/ratelimit-config-schema.test.ts diff --git a/tests/ratelimit-config-schema.test.ts b/tests/ratelimit-config-schema.test.ts new file mode 100644 index 000000000..5198a71cb --- /dev/null +++ b/tests/ratelimit-config-schema.test.ts @@ -0,0 +1,92 @@ +import { describe, expect, test } from "bun:test"; +import { + MAX_RATE_LIMIT_BUCKETS, + MAX_RATE_LIMIT_BURST, + MAX_RATE_LIMIT_REQUESTS_PER_MINUTE, + MAX_RATE_LIMIT_STALE_AFTER_MS, + MAX_RATE_LIMIT_TRACKED_PRINCIPALS, + MAX_RATE_LIMIT_WEBSOCKET_CONCURRENCY, + MIN_RATE_LIMIT_STALE_AFTER_MS, + rateLimitConfigSchema, +} from "../src/ratelimit"; + +function expectRejected(value: unknown): void { + expect(rateLimitConfigSchema.safeParse(value).success).toBe(false); +} + +describe("canonical rate-limit config subschema", () => { + test("accepts absent tuning, complete defaults, and partial known overrides", () => { + expect(rateLimitConfigSchema.parse({})).toEqual({}); + expect(rateLimitConfigSchema.parse({ enabled: false })).toEqual({ enabled: false }); + expect(rateLimitConfigSchema.parse({ + enabled: true, + bypassLoopback: true, + maxBuckets: 200, + staleAfterMs: 30_000, + policies: { + management: { requestsPerMinute: 10, burst: 2 }, + "responses-http": { requestsPerMinute: 20.5, burst: 4 }, + }, + webSocket: { perPrincipal: 2, global: 8, maxTrackedPrincipals: 100 }, + })).toEqual({ + enabled: true, + bypassLoopback: true, + maxBuckets: 200, + staleAfterMs: 30_000, + policies: { + management: { requestsPerMinute: 10, burst: 2 }, + "responses-http": { requestsPerMinute: 20.5, burst: 4 }, + }, + webSocket: { perPrincipal: 2, global: 8, maxTrackedPrincipals: 100 }, + }); + }); + + test("rejects unknown root, policy, policy-field, and WebSocket fields", () => { + expectRejected({ enabled: true, secret: "nope" }); + expectRejected({ enabled: true, policies: { unknown: { requestsPerMinute: 1, burst: 1 } } }); + expectRejected({ + enabled: true, + policies: { management: { requestsPerMinute: 1, burst: 1, identity: "nope" } }, + }); + expectRejected({ enabled: true, webSocket: { perPrincipal: 1, global: 2, principal: "nope" } }); + }); + + test("rejects non-finite, zero, fractional integer fields, and hard-cap overflow", () => { + for (const requestsPerMinute of [0, -1, Number.NaN, Number.POSITIVE_INFINITY, MAX_RATE_LIMIT_REQUESTS_PER_MINUTE + 1]) { + expectRejected({ policies: { management: { requestsPerMinute, burst: 1 } } }); + } + for (const burst of [0, -1, 1.5, MAX_RATE_LIMIT_BURST + 1]) { + expectRejected({ policies: { management: { requestsPerMinute: 1, burst } } }); + } + for (const maxBuckets of [0, 1.5, MAX_RATE_LIMIT_BUCKETS + 1]) expectRejected({ maxBuckets }); + for (const staleAfterMs of [0, MIN_RATE_LIMIT_STALE_AFTER_MS - 1, 1.5, MAX_RATE_LIMIT_STALE_AFTER_MS + 1]) { + expectRejected({ staleAfterMs }); + } + for (const value of [0, 1.5, MAX_RATE_LIMIT_WEBSOCKET_CONCURRENCY + 1]) { + expectRejected({ webSocket: { perPrincipal: value } }); + expectRejected({ webSocket: { global: value } }); + } + expectRejected({ webSocket: { maxTrackedPrincipals: MAX_RATE_LIMIT_TRACKED_PRINCIPALS + 1 } }); + }); + + test("rejects contradictory WebSocket limits", () => { + const result = rateLimitConfigSchema.safeParse({ webSocket: { perPrincipal: 5, global: 4 } }); + expect(result.success).toBe(false); + if (!result.success) { + expect(result.error.issues).toContainEqual(expect.objectContaining({ + path: ["webSocket", "perPrincipal"], + message: "perPrincipal must not exceed global", + })); + } + }); + + test("parsed values contain no runtime state or identity fields", () => { + const parsed = rateLimitConfigSchema.parse({ + enabled: true, + policies: { live: { requestsPerMinute: 5, burst: 2 } }, + webSocket: { perPrincipal: 1, global: 2 }, + }); + const serialized = JSON.stringify(parsed); + expect(serialized).not.toMatch(/fingerprint|principalCount|globalCount|accepted|denied|token|secret|address|origin/i); + }); +});