[codex] Bound local HTTP API concurrency

[zergzorg]

Summary

• cap the local HTTP API to 16 active connection workers
• return 503 {"error":"server_busy"} without spawning another worker when the cap is reached
• add write timeouts alongside the existing read timeout
• add focused tests for permit release and the busy response

Why

This is a small follow-up slice of #487. The local API currently accepts each TCP connection and immediately spawns an OS thread for it. Under accidental or intentional local stress traffic, that can grow worker threads without a local app-side bound.

The new connection limiter keeps normal local API behavior unchanged for regular clients, while making overload explicit and bounded. This is intentionally independent from the probe-resource PR stack (#498/#499/#500) and avoids touching host_api.rs, which already has active PRs.

Notes

The local API is read-only and localhost-bound, so returning 503 server_busy during overload is preferable to creating unbounded OS threads. The cap is deliberately conservative; normal use should be one or a few concurrent requests.

Validation

• cargo test local_http_api::server::tests
• cargo test --lib -- --skip ccusage_timeout_kills_descendant_and_closes_pipes
• bun run build
• node ./node_modules/vitest/vitest.mjs run

I skipped ccusage_timeout_kills_descendant_and_closes_pipes in the broad Rust run because that unrelated flaky test is fixed separately in #501 but is not in main yet.

Part of #487.

---

Summary by cubic

Bound the local HTTP API to 16 concurrent connections and return 503 {"error":"server_busy"} without spawning a worker when at capacity. Added 5s read/write timeouts, tests for permit release and 503 JSON/headers, and updated docs to include server_busy with backoff guidance.

^{Written for commit 0708889. Summary will update on new commits. Review in cubic}

[cubic-dev-ai]

No issues found across 1 file

_{Re-trigger cubic}

[validatedev]

@codex review

[Copilot]

Pull request overview

This PR bounds the local loopback HTTP API’s per-connection worker concurrency to prevent unbounded OS thread growth under local stress traffic, while keeping normal behavior unchanged for typical clients.

Changes:

• Add a connection limiter capped at 16 concurrent active connections; reject excess connections with 503 {"error":"server_busy"}.
• Add a 5s write timeout alongside the existing read timeout for connection handling.
• Add unit tests to verify permit release behavior and the 503 busy response shape/headers.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Delightful!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[robinebers]

Another great contribution. Thank you so much.

[robinebers]

Hey! 👋 This is Rob's AI reviewer. Thanks for the careful follow-up to #487!

Really clean change overall — atomic CAS loop is correct, Drop-based permit release means no leaks, the 503 path preserves CORS/Connection: close, and the new tests cover both the limiter invariant and the response shape. Two small things before merge:

1. Document the new 503 in docs/local-http-api.md — the "Error Responses" section currently lists provider_not_found, not_found, method_not_allowed as the possible error codes. Worth adding server_busy and a short note that the server may return 503 Service Unavailable when the concurrent-connection cap is reached, so API consumers know to back off and retry rather than treat it as a hard failure.

2. Drop the unused #[derive(Clone)] on ConnectionLimiter — the limiter is constructed once in start_server and never cloned (permits hold their own Arc<AtomicUsize>), so the derive is dead. Tiny nit.

Everything else (timeouts, conservative cap, scope staying out of host_api.rs, test coverage) looks great.

[zergzorg]

Addressed both requested changes in the latest push:

• documented server_busy in docs/local-http-api.md as a 503 Service Unavailable response that callers should back off and retry;
• removed the unused Clone derive from ConnectionLimiter.

Verified locally with:

• rustfmt --check src/local_http_api/server.rs
• cargo test local_http_api::server::tests
• bun run build
• git diff --check

Note: bun run build succeeds, but Vite prints the existing Node 22.0.0 version warning.

[zergzorg]

Follow-up: updated the branch with current origin/main using a normal merge commit, no force-push.

Re-verified after the merge:

• cargo test local_http_api::server::tests
• bun run build

The Vite Node 22.0.0 warning is still present, but the build exits successfully.