vector-to-vector throughput capped by default HTTP/2 flow-control windows

[Evador]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

A single vector-to-vector (native protocol) connection plateaus at a few MB/s
of event bytes regardless of batch.* and request.concurrency settings.
On our fleet (0.44.0, 2 ms LAN, TLS, end-to-end acknowledgements enabled) each
connection tops out around 7k events/s (~5 MB/s of event bytes) with CPU
nearly idle on both peers.

Evidence that it's flow control rather than processing:

• doubling batch.max_bytes exactly doubles per-request latency, throughput unchanged
• request.concurrency 1 vs 8 vs 32 makes no difference over the network
• N parallel connections scale linearly (~20 connections reach ~45k events/s aggregate)
• on localhost the same binaries and configs sustain 65-74k events/s per connection

The sink builds its hyper client with only .http2_only(true) and the shared
gRPC server builder doesn't set window options either, so both peers run with
the RFC 7540 default 65,535-byte stream/connection windows. With
acknowledgements enabled the server only completes a request (and the client
only gets WINDOW_UPDATE credit back) after events are accepted downstream, so
effective throughput ≈ 64 KiB / (network RTT + ack latency).

Configuration

Version

0.56.0

Debug Output

Example Data

No response

Additional Context

Proposed fix

Enable adaptive (BDP-sized) HTTP/2 windows on the vector sink client and the
shared gRPC server builder — #25782.

References

No response