Problem
The heap-crash investigation found a separate pressure amplifier from #369: packages/codex-headless/src/proxy/responsesProxy.ts mirrors every proxy event synchronously, and response-chunk events serialize raw Buffers through JSON/base64 before normal listeners run. This creates large transient strings, blocks the event loop, and writes unbounded active proxy-events.jsonl files.
Local evidence from the investigation found proxy storage near 965 MiB, with individual proxy-events.jsonl files over 100-280 MiB. Feed-debug and performance logging have similar per-file cap gaps.
Findings
ResponsesProxy.emit mirrors before super.emit, so slow/string-heavy mirror writes delay the stream adapter.response-chunk mirroring can hold raw bytes, base64 text, JSON text, and append buffers at once.- Request capture buffers and parses full bodies for shape extraction and may base64 encode request bodies.
- Retention eventually prunes disk, but active writers have no per-run byte cap or drop mode.
- Feed-debug renderer memory is count-capped, but disk files are not per-file byte-capped.
- Retention sweeps are cleanup, not pressure control, and can themselves be expensive during pressure.
Proposed Fix
- Stop raw/base64 mirroring of every
response-chunk by default. Store metadata plus bounded samples/tails; gate full raw capture behind an explicit forensic flag.
- Add per-run/per-file byte caps and dropped counters for proxy mirrors, feed-debug, and performance logs.
- Move mirror writes off the hot synchronous path into a bounded async queue/drop policy.
- Add lightweight counters: chunk bytes, serialized bytes, mirror write ms, cap drops, queue depth, per-run bytes.
- Make retention cheaper and visible: one scan per prune, protected bytes, active-grace skips, removals, and scan duration.
Acceptance
- Large Responses streams no longer synchronously base64/JSON mirror every chunk by default.
- Active proxy/feed/perf log files have hard caps and record dropped counters.
- A stream stress test demonstrates bounded proxy mirror memory/disk behavior.
- Debug/retention metrics make it clear when diagnostics are being dropped because of caps.
Related: #327, #365, #369, #370
Problem
The heap-crash investigation found a separate pressure amplifier from #369:
packages/codex-headless/src/proxy/responsesProxy.tsmirrors every proxy event synchronously, andresponse-chunkevents serialize raw Buffers through JSON/base64 before normal listeners run. This creates large transient strings, blocks the event loop, and writes unbounded activeproxy-events.jsonlfiles.Local evidence from the investigation found proxy storage near 965 MiB, with individual
proxy-events.jsonlfiles over 100-280 MiB. Feed-debug and performance logging have similar per-file cap gaps.Findings
ResponsesProxy.emitmirrors beforesuper.emit, so slow/string-heavy mirror writes delay the stream adapter.response-chunkmirroring can hold raw bytes, base64 text, JSON text, and append buffers at once.Proposed Fix
response-chunkby default. Store metadata plus bounded samples/tails; gate full raw capture behind an explicit forensic flag.Acceptance
Related: #327, #365, #369, #370