Add bench-proxy: Rust supervisor + reverse proxy with /meta snapshot

[wokalski]

Summary

• Replaces bin/smoke-* with a single Rust binary bench-proxy --listen <addr> --config <yaml>. The proxy spawns the engine on a kernel-assigned random loopback port that only it knows, so by construction every client request to /v1/* and /meta describes the same process — no "different config than what we benchmarked" drift.
• /meta is a frozen JSON snapshot collected post-warmup. It captures: command argv/env/cwd read from /proc/<pid>/{cmdline,environ,cwd} (observed, not synthesized); the full descendant process tree with per-PID env (catches per-rank CUDA_VISIBLE_DEVICES divergence on TP workers); host/CPU/memory/THP/governor/NUMA tunables; nvidia-smi summary + raw -q -x XML; lshw; engine version; an exhaustive sha256 walk of every file under <venv>/{bin,lib,lib64} with a single merkle_sha256 rollup; and a deduped loaded_libs union of /proc/*/maps across the process tree, sha256'd, with versions parsed from filenames.
• Child env is fully sanitized — env_clear() + only what the YAML's env: lists, no implicit inheritance. \$VAR/\${VAR} interpolate from the proxy's runtime env (and \$REPO_ROOT is auto-injected) so configs are portable across nix store hashes. Secrets tagged as { secret: <value> } appear in /meta as { redacted: true, sha256: ... } so two runs can be diffed without exposing the value.
• SSE passthrough is hyper-direct (no collect, no buffering) for low-jitter streaming.

Schema (excerpt)

description: ...                # required
engine: vllm | sglang | trt-llm
cmd: vllm
args: [serve, openai/gpt-oss-120b, --tensor-parallel-size, \"2\", ...]
cwd: vllm                       # optional, repo-root-relative
venv: vllm/.venv                # optional; drives the venv snapshot
warmup: 3                       # optional; default 3, 0 to skip
env:
  PATH: \$REPO_ROOT/vllm/.venv/bin:\$PATH
  CUDA_HOME: \$CUDA_HOME
  HF_TOKEN: { secret: \$HF_TOKEN }

args may not contain --host or --port; the proxy injects them.

Test plan

• cargo build --release --manifest-path proxy/Cargo.toml clean, no warnings
• Validation rejects --port in args, empty description, unset \$VAR
• Smoke against a Python mock OpenAI server (single + multi-PID): /healthz, /meta 503-then-200, command.env exactly matches YAML (no inherited LANG/USER/TERM), SSE chunks visible progressively
• Secret redaction: HF_TOKEN: { secret: ... } surfaces as { redacted, sha256 }; plaintext absent from logs
• Real vllm 0.19.1 + Qwen2.5-0.5B on H100 NVL (configs/vllm-smoke.yaml):
  • 3-PID process_tree — vllm (APIServer), python (mp helper), VLLM::EngineCore — comm-renaming captured
  • loaded_libs = 164 entries including libcudnn.so.9, libnccl.so.2, libcublas.so.12, libtorch_cuda.so from venv site-packages, plus libcuda.so.570.172.08 and libnvidia-ml.so.570.172.08 from /usr/lib/x86_64-linux-gnu (vast-mounted host driver) — versions parsed from filenames
  • venv_snapshot = 84,131 files, 21.0 GB, single merkle hash
  • warmup.elapsed_ms = [350, 193, 192] — clear cold→warm shape, exactly the lazy-load capture we want pre-snapshot
  • 5 concurrent chat completions served correctly
  • SIGINT → child exits 0, no orphans
• Real run against vllm-gpt-oss-120b.yaml / sglang-gpt-oss-120b.yaml / trt-llm-gpt-oss-120b.yaml once weights are cached

Known follow-ups (not in this PR)

• If the proxy's listen port is already bound (e.g. by a stale instance), proxy::serve logs the EADDRINUSE and the spawned task ends, but main keeps running and "publishes" the snapshot to no one. Should propagate the bind failure and exit non-zero.
• TTFT-against-real-engine is too noisy as a proxy-overhead check (engine variance dominates: 3 keep-alive runs of 500 samples gave +4647 / +34 / −264 µs deltas). For a real overhead measurement, hit a no-compute upstream or use 10k+ samples.

🤖 Generated with Claude Code