Skip to content

feat(codec): Add ServerCompressionInterceptor for gRPC compression#2649

Open
sauravzg wants to merge 1 commit into
sauravz/compression-apifrom
sauravz/server-compression
Open

feat(codec): Add ServerCompressionInterceptor for gRPC compression#2649
sauravzg wants to merge 1 commit into
sauravz/compression-apifrom
sauravz/server-compression

Conversation

@sauravzg

Copy link
Copy Markdown
Contributor

⚠️ TRADEOFFS & LIMITATIONS ⚠️ Due to the current limitations of the RecvStream API (which lacks out-of-band compression signals), this implementation introduces a high degree of coupling between the HTTP transport, compression, and serialization layers. Consequently, this design necessitates type erasure(IncomingRawMessage) and incurs a forced Box allocation penalty to function within the existing stream boundaries.

This change introduces the ServerCompressionInterceptor, which integrates compression and decompression capabilities into the server-side gRPC request/response lifecycle.

Key additions:

  • ServerCompressionInterceptor: Implements the Intercept trait to wrap incoming requests. It parses the grpc-encoding and grpc-accept-encoding headers to determine the appropriate Compressor and Decompressor to use.
  • Decompression Bomb Mitigation: Protects against decompression bomb (zip bomb) attacks by strictly limiting writes during the decompression process.
  • CompressionResolver Integration: Uses a CompressionResolver registry to look up algorithms by name.
  • Stream Wrapping: Wraps the underlying RecvStream (to decompress incoming messages) and SendStream (to compress outgoing messages and inject the correct grpc-encoding header).
  • Error Handling: Returns standard gRPC status errors if an encoding is unsupported or if compression/decompression fails during stream processing.
  • Unit Tests: Includes a comprehensive suite of Tokio-based tests covering successful compression/decompression, unsupported encoding errors, registry lookup failures, and fallback mechanisms.

@sauravzg sauravzg force-pushed the sauravz/server-compression branch 2 times, most recently from beb2595 to 5acb682 Compare May 19, 2026 14:26
@sauravzg sauravzg requested a review from dfawley May 19, 2026 14:27
@sauravzg sauravzg force-pushed the sauravz/server-compression branch from 5acb682 to b4af9e3 Compare May 22, 2026 11:13
@sauravzg sauravzg force-pushed the sauravz/compression-api branch from db718f9 to 2e65c81 Compare May 22, 2026 11:13
⚠️ TRADEOFFS & LIMITATIONS ⚠️ Due to the current limitations of the
`RecvStream` API (which lacks out-of-band compression signals), this
implementation introduces a high degree of coupling between the HTTP
transport, compression, and serialization layers. Consequently, this
design necessitates type erasure(IncomingRawMessage) and incurs a forced
`Box` allocation penalty to function within the existing stream
boundaries.

This change introduces the `ServerCompressionInterceptor`, which
integrates compression and decompression capabilities into the
server-side gRPC request/response lifecycle.

Key additions:

- **`ServerCompressionInterceptor`**: Implements the `Intercept` trait
  to wrap incoming requests. It parses the `grpc-encoding` and
  `grpc-accept-encoding` headers to determine the appropriate
  `Compressor` and `Decompressor` to use.
- **Decompression Bomb Mitigation**: Protects against decompression bomb
  (zip bomb) attacks by strictly limiting writes during the
  decompression process.
- **`CompressionResolver` Integration**: Uses a `CompressionResolver`
  registry to look up algorithms by name.
- **Stream Wrapping**: Wraps the underlying `RecvStream` (to decompress
  incoming messages) and `SendStream` (to compress outgoing messages and
  inject the correct `grpc-encoding` header).
- **Error Handling**: Returns standard gRPC status errors if an encoding
  is unsupported or if compression/decompression fails during stream
  processing.
- **Unit Tests**: Includes a comprehensive suite of Tokio-based tests
  covering successful compression/decompression, unsupported encoding
  errors, registry lookup failures, and fallback mechanisms.
@sauravzg sauravzg force-pushed the sauravz/server-compression branch from b4af9e3 to bcfd493 Compare May 22, 2026 11:39
@sauravzg sauravzg force-pushed the sauravz/compression-api branch from 2e65c81 to c4d2db5 Compare May 22, 2026 11:39
@dfawley

dfawley commented Jun 29, 2026

Copy link
Copy Markdown
Member

@saurav - this PR is not targetting master. Is that intentional?

@sauravzg

Copy link
Copy Markdown
Contributor Author

@saurav - this PR is not targetting master. Is that intentional?

Yes. I am trying out a stacked PR approach, every PR is targetting its parent PR.

@dfawley

dfawley commented Jun 30, 2026

Copy link
Copy Markdown
Member

Yes. I am trying out a stacked PR approach, every PR is targetting its parent PR.

So then.... we end up with a branch that has all the changes on it that we want, and then we create a final PR that merges that branch to master once it's all done?

@sauravzg

sauravzg commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

I plan to rely on https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-branches#working-with-branches to retarget branch to head when the parent PR gets merged.

So, I am supposed to merge the parent PR first i.e. #2648 . After which , github will auto target this PR to master.

So, the the features missing from https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-branches#working-with-branches , are

  1. A fancy UI for the stack.
  2. A check that prevents merging child before merging parent.

I don't care about 1 right now and plan to manually handle 2 myself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants