Add server-side WebSocket compression by mkurz · Pull Request #1114 · apache/pekko-http

mkurz · 2026-06-30T22:31:10Z

What changed

Adds server-side support for RFC 7692 permessage-deflate WebSocket compression.

Compression is negotiated during the WebSocket handshake when the client offers permessage-deflate in Sec-WebSocket-Extensions. It is enabled by default and can be disabled globally through configuration.

Also adds an API option to decline negotiated compression for a single accepted WebSocket upgrade.

Fixes Please add support for WebSocket compression #61.

Details

This PR adds:

permessage-deflate negotiation for server-side WebSockets
inbound compressed message inflation
outbound message deflation
fragmented compressed message handling
context takeover and no-context-takeover negotiation
configurable compression level
configurable max decompressed allocation
configurable preferred client_max_window_bits
per-upgrade compression disabling via new WebSocketUpgrade overloads
documentation, reference config, release notes, and MiMa filters

Netty comparison

The implementation follows Netty’s permessage-deflate behavior where it makes sense for Pekko HTTP:

compressed inbound messages are inflated before they reach the application
outbound text/binary messages are compressed when compression was negotiated
fragmented compressed messages keep RSV1 only on the first frame
control frames can be interleaved with fragmented compressed messages
context takeover is retained by default
server_no_context_takeover and client_no_context_takeover can be negotiated
client_max_window_bits can be negotiated
decompressed output is bounded through max-allocation

One deliberate difference is zlib implementation scope. Netty also uses the JDK Deflater and Inflater APIs for the default zlib settings, and only switches to JZlib when non-default zlib windowBits or memLevel support is needed. Pekko HTTP currently uses only the JDK Deflater and Inflater APIs, like Tomcat and Jetty. The JDK API does not expose zlib windowBits or memLevel, so Pekko HTTP does not accept server_max_window_bits values below 15 and does not provide server window-size or memory-level settings.

Clients may still request client_max_window_bits; when they do, Pekko HTTP can include the configured preferred-client-window-size in the handshake response to ask the client to use that window size for client-to-server messages.

Security notes

WebSocket compression can increase CPU and memory usage.

The default context takeover settings match common server defaults and favor compression efficiency. Applications that need a more conservative setup can configure no-context-takeover behavior or disable compression globally/per accepted WebSocket.

max-allocation limits decompressed message size. If the limit is exceeded while inflating a compressed message, Pekko HTTP closes the connection with a WebSocket protocol error.

Testing

Ran:

sbt "http-core / Test / testOnly org.apache.pekko.http.impl.engine.ws.WebSocketServerSpec"
sbt +mimaReportBinaryIssues

Also verified Play Framework integration against the locally published Pekko HTTP artifact:
(This is a WIP pull request for Play which I will submit tomorrow - I will you ping you then)

sbt -Dpekko.http.version=<local-version> \
  "Play-Integration-Test / Test / testOnly play.it.http.websocket.PekkoHttpWebSocketCompressionSpec"

sbt -Dpekko.http.version=<local-version> \
  "Play-Integration-Test / Test / testOnly play.it.http.websocket.PekkoHttpWebSocketSpec"

References

pjfanning · 2026-06-30T23:24:44Z

 * Dependency versions have been updated
 * Jackson3 is supported in a new pekko-http-jackson3 lib
 * Changed Java DSL methods that return Scala Durations to return Java Durations ([PR788](https://github.com/apache/pekko-http/pull/788))
+* Server-side WebSocket connections can now negotiate RFC 7692 `permessage-deflate` compression when requested by the client.


2.0.0-M1 is already released. Just revert this and we will form 2.0.0-M2 release notes when that release is ready, hopefully in a month or 2.

He-Pin · 2026-07-01T04:37:33Z

I see netty have many compression support, should we do that too?

He-Pin · 2026-07-01T04:39:10Z

+        # Whether the server should support WebSocket compression using the RFC 7692
+        # permessage-deflate extension. Compression is negotiated during the
+        # WebSocket handshake and is only used when the client requests it.
+        enabled = true


Should we do adaptive compression?

Could you clarify what kind of adaptive compression you mean?

With this PR, Pekko HTTP already supports adaptation in two ways:

Compression is only negotiated when the client offers permessage-deflate.

Compression can be disabled globally, or per accepted WebSocket via the new compressionEnabled overloads.

If you mean adapting per outbound message, for example compressing a large message but sending a small message uncompressed, I think that could make sense as a follow-up PR. permessage-deflate allows this: once the extension is negotiated, a sender can still choose to leave RSV1 unset for individual messages. This would need careful handling around fragmented messages and context takeover, but a minimum-size threshold or application-controlled filter could be useful.
In Play we do basically the same for the gzip filter: https://github.com/playframework/playframework/blob/d87116f880789e0dbc40b7f971e08638e5ab7557/web/play-filters-helpers/src/main/resources/reference.conf#L297-L300 - which avoids gziping content that is smaller then a certain threshold.
But, as said, I would do that in a follow up PR if desired.

Similarly, skipping compression for certain message types or already-compressed payloads could make sense later, but Pekko HTTP currently does not know application-level message semantics at this layer, so that would also need separate design.

If you mean changing the compression level dynamically, that is more complicated and probably less useful as a general automatic feature. A simpler and more realistic variant would be allowing applications to choose a compression level per WebSocket connection, similar to how this PR allows enabling/disabling compression per accepted WebSocket. That should be much easier to reason about than changing the level message-by-message.
I would also do that in a follow up PR if desired.

If you mean disabling compression automatically under CPU or memory pressure, I think that is much larger in scope and probably not a good fit for this PR. It would need metrics, policy, and predictable behavior, and I am not sure the added complexity would be worth it.

Did some research if netty has similiar feature(s):

What Netty does have is lower-level extension filters:

WebSocketExtensionFilter.mustSkip(WebSocketFrame frame)

Those filters can decide per frame whether an extension encoder/decoder should be skipped. Netty’s own tests show a threshold-style example:

return (frame instanceof TextWebSocketFrame || frame instanceof BinaryWebSocketFrame) && frame.content().readableBytes() < 100;

So Netty supports the mechanism for "don’t compress small messages", but it does not ship a named adaptive compression feature/config. Its default is WebSocketExtensionFilterProvider.DEFAULT, which means NEVER_SKIP, so negotiated messages are compressed.

Important detail: for permessage-deflate, Netty forbids changing the decision in the middle of a fragmented message. If compression/decompression is already in progress and the filter suddenly says “skip” for a continuation frame, Netty throws.

So yeah, maybe in a follow up PR it might make sense to think about some mechanism to dynamicaly decide if an outbound message should be compressed or not

yes, but that can come up later, we are using something like this later.

mkurz · 2026-07-01T08:27:47Z

I see netty have many compression support, should we do that too?

For this PR, no. This PR should stay focused on RFC 7692 permessage-deflate, which is the relevant standardized WebSocket compression extension. Netty also has legacy deflate-frame / x-webkit-deflate-frame support. I intentionally did not add those because they are historical, non-standard extension names from before RFC 7692 standardized permessage-deflate. For our new implementation in pekko-http, I think it is better to support the standardized extension first and only add legacy extensions later if users report a concrete compatibility need (which I highly doubt). This avoids increased code, tests, and configuration surface for very little practical benefit.

mkurz · 2026-07-01T09:03:38Z

All fixed.

For full disclosure: I was working on this for like 1,5 days intensively with heavy use of codex (gpt-5.5), doing lots of rubberducking, analysing how netty or projects like tomcat implements things and going over the RFC. Did many dozens iterations, back and forth (trying different approaches). Zero code copied, some manual adjustments. Since the netty implementation is different - by having different architecture in general - and Akka HTTP does not even have support for compressed websocket, this code here is unique ("my own work").

Also, made sure that this will play nicely together with Play - where I locally build support for websocket compression as well based on this implementation. Will submit the Play PR soon so you see what I mean.

pjfanning · 2026-07-01T09:03:55Z


+  /**
+   * Java API
+   */


can you add @since 2.0.0 here?

pjfanning

lgtm - thanks

Hopefully, we can make some of the suggested improvements over the next few weeks but I think this is already a very useful addition as is.

* Negotiate RFC 7692 permessage-deflate for server-side WebSockets. * Add compression settings, documentation, release notes, and MiMa filters. * Cover negotiation, compression, fragmentation, context takeover, max allocation, and low-level frame handling.

* Add WebSocketUpgrade overloads that accept a compressionEnabled flag. * Wire accepted WebSockets to decline negotiated compression per request. * Document and test per-upgrade compression disabling.

mkurz · 2026-07-01T10:05:52Z

(only rebased on latest main with last force push)

He-Pin

Nice work on this PR — server-side WebSocket compression is a long-requested feature and the implementation looks solid overall. The negotiation logic is thorough, the test coverage is impressive (fragmentation, context takeover, max-allocation, low-level frames), and the per-upgrade compression control is a nice touch.

A few things I noticed while reading through the code:

`PerMessageDeflate.scala`

Buffer allocation in inflate/deflate: The 1024-byte buffer is allocated on every call to inflate()/deflate(). Since these are called per-frame within a single connection, it might be worth making the buffer a class-level field in InflaterFlow/DeflaterFlow to reduce allocation pressure. Also, 1024 bytes is quite small — 8192 is a more typical buffer size for streaming decompression/compression and would reduce loop iterations for larger messages.
CompressedFrame.append and UncompressedFrame.append: Using data ++ next on ByteString for each incoming fragment creates a new ByteString each time, which can become O(n²) for heavily fragmented messages. Consider collecting fragments in a Vector[ByteString] and concatenating once in finishFrame(), or using a ByteStringBuilder.
asInstanceOf[WebSocketSettingsImpl] in Handshake.scala: The unchecked cast settings.asInstanceOf[WebSocketSettingsImpl].compression will throw a ClassCastException if a custom WebSocketSettings implementation is provided. Consider using a pattern match with a fallback (e.g. skip compression negotiation for unknown settings types) or documenting the requirement.
validWindowBits edge case: The function doesn't guard against an empty string explicitly. While the empty-string case for client_max_window_bits is handled before validWindowBits is called, the function itself would throw NumberFormatException on "".toInt if ever called with an empty value in the future. A simple value.nonEmpty && guard would make it more defensive.
LifecycleMapConcatStage — incomplete message on upstream finish: If upstream finishes while a fragmented compressed message is still in progress (compressedMessageInProgress = true or compressedFrame.isDefined), the pending data is silently dropped. Consider whether this should emit a protocol error or at least log a warning.

Configuration

Default max-allocation = 64k: 64 KB is quite conservative for a default decompressed message size limit. Many real-world WebSocket applications (e.g. real-time collaboration, gaming, large JSON payloads) routinely exceed this. While it's a security default and can be overridden, a value like 1 MB or 256 KB might be more practical and still provide protection against decompression bombs. Tomcat defaults to no limit, and Jetty defaults to 128 KB for the text message size limit.

Minor / Style

Header list construction in buildResponse: The List(...) ::: perMessageDeflate.map(...).toList ::: List(...) pattern is a bit awkward. A Seq.newBuilder or a conditional ++ might read more cleanly.
WebSocketCompressionSettingsImpl.Disabled sentinel: The Disabled value has maxAllocation = 0, which the reference.conf documents as "no limit". While this value is never used when compression is disabled, the semantic mismatch is slightly confusing.

The test coverage is excellent — especially the fragmented compressed messages with interleaved control frames, the context takeover reset tests, and the max-allocation boundary tests. Overall this is a well-structured PR.

He-Pin

Left a few inline suggestions on the core implementation files.

He-Pin · 2026-07-01T10:37:15Z

+        val input = if (appendTail) data ++ EmptyStoredBlock else data
+        inflater.setInput(input.toArray)
+        val output = new ByteArrayOutputStream()
+        val buffer = new Array[Byte](1024)


The 1024-byte buffer is allocated fresh on every inflate() call. Since this method runs per frame within a single connection, it might be worth making this a class-level field to reduce GC pressure. Also, 1024 is pretty small — something like 8192 would cut down loop iterations for larger messages without meaningfully increasing memory usage.

The 1024-byte buffer is allocated fresh on every inflate() call. Since this method runs per frame within a single connection, it might be worth making this a class-level field to reduce GC pressure.

codex was suggesting the same, but we came to the conclusion its not worth it. anyway, let me see again.

in the Artery Transport, there is a bytebufPoll. I think that would help.

He-Pin · 2026-07-01T10:37:15Z

+    private def deflate(data: ByteString, removeTail: Boolean): ByteString = {
+      deflater.setInput(data.toArray)
+      val output = new ByteArrayOutputStream()
+      val buffer = new Array[Byte](1024)


Same as above — this buffer could be a field of DeflaterFlow and sized to 8192 or so. The deflater loop will spin many more times than necessary on larger payloads with a 1KB buffer.

He-Pin · 2026-07-01T10:37:15Z

+
+  private final case class CompressedFrame(header: FrameHeader, data: ByteString, appendTail: Boolean) {
+    def append(next: ByteString): CompressedFrame = copy(data = data ++ next)
+  }


'data ++ next creates a new ByteString copy on every fragment. For heavily fragmented messages this becomes O(n²) in the total payload size. Consider accumulating fragments in a Vector[ByteString] (or a ByteStringBuilder) and concatenating once in finishFrame(). Same applies to UncompressedFrame.append below.'

He-Pin · 2026-07-01T10:37:15Z

+  }
+
+  private def validWindowBits(value: String): Boolean =
+    value.length <= 2 && value.forall(_.isDigit) && {


Nit: if this ever gets called with an empty string (it won't today because the empty case is handled before reaching here), value.toInt would throw NumberFormatException. A quick value.nonEmpty && guard at the start would make this more defensive for future callers.

He-Pin · 2026-07-01T10:37:15Z

+        override def onPull(): Unit =
+          pushOrPull()
+
+        override def onUpstreamFinish(): Unit = {


If upstream finishes while a fragmented compressed message is still in progress (compressedMessageInProgress == true or compressedFrame.isDefined), the pending data is silently dropped here. Not sure if that's the right behavior — maybe emit a protocol error or at least log a warning so it's not a silent data loss?

He-Pin · 2026-07-01T10:37:15Z

+          val perMessageDeflate =
+            PerMessageDeflate.negotiate(
+              clientRequestedExtensions,
+              settings.asInstanceOf[WebSocketSettingsImpl].compression)


This unchecked cast will throw a ClassCastException if someone provides a custom WebSocketSettings implementation that isn't WebSocketSettingsImpl. Might be worth a pattern match with a graceful fallback (e.g. skip compression negotiation for unknown settings types), or at least documenting that WebSocketSettings must be a WebSocketSettingsImpl.

He-Pin · 2026-07-01T10:37:15Z

+        # exceeded while inflating a compressed message, the connection is closed
+        # with a WebSocket protocol error.
+        # Set to 0 to disable this limit.
+        max-allocation = 64k


64 KB feels quite conservative for a default. Real-world WebSocket apps (collaborative editing, gaming, large JSON payloads) routinely exceed this. Tomcat has no limit by default, Jetty uses 128 KB for text message size. Something like 256 KB or 1 MB might be a more practical default while still protecting against decompression bombs. Users who need tighter limits can always lower it.

I would be ok with 256 KB - 1 MB seems a bit too much.
@pjfanning what do you think?

I have no strong preference. 256Kb can be the starting point. If we get requests to increase the default later on, we can do that. The value is a configurable but a default setting that only some people need to worry about is ideal.

mkurz · 2026-07-01T10:49:31Z

@He-Pin Give me a minute, so my agent can answer your agent ;)
btw, which model are you using?

He-Pin

overall lgtm , would it possible to coodinate with apache/pekko#2409 ?

He-Pin · 2026-07-01T10:53:44Z

          }
+          val clientRequestedExtensions = headers.collect {
+            case extensions: `Sec-WebSocket-Extensions` => extensions.extensions
+          }.flatten


Would it be possible to avoid the flatten and do it in one call?

He-Pin · 2026-07-01T10:57:02Z

+      BidiFlow.fromFlows(inflaterFlow, deflaterFlow)
+
+    def frameEventBidiFlow(
+        maskRandom: () => Random): BidiFlow[FrameEvent, FrameEvent, FrameEvent, FrameEvent, NotUsed] =


Use RandomGenerator in Java 17?

He-Pin · 2026-07-01T10:59:56Z

+
+  private trait LifecycleMapConcat[-In, +Out] extends (In => immutable.Iterable[Out]) {
+    def close(): Unit
+  }


I think we should avoid this MapConcat name because there is one in the pekko stream too.

And there is a statefulMap which can be used

He-Pin · 2026-07-01T11:00:58Z

+   *
+   * @since 2.0.0
+   */
+  def handleMessagesWith(


pjfanning · 2026-07-01T11:37:32Z

overall lgtm , would it possible to coodinate with apache/pekko#2409 ?

2409 is not making progress and we can always add zstd later - I don't see why we can't just use the compression here initially

He-Pin · 2026-07-01T18:45:56Z

I'm fine with this, Which I'm asking is will @mkurz introduce some common interface into pekko and used it here.

mkurz · 2026-07-01T19:06:16Z

I will address the comments later or tomorrow

He-Pin · 2026-07-01T19:18:32Z

+          output.write(buffer, 0, count)
+          count = inflater.inflate(buffer)
+        }
+        ByteString.fromArray(output.toByteArray)


There is an fromUnsafeArray iirc

He-Pin · 2026-07-01T19:20:07Z

+    }
+
+    private def finishFrame(): immutable.Iterable[FrameEventOrError] = {
+      val frame = compressedFrame.get


Netty (DeflateDecoder.java:48-49, 122-123) handles empty EMPTY_DEFLATE_BLOCK

He-Pin · 2026-07-01T19:21:34Z

+        val input = if (appendTail) data ++ EmptyStoredBlock else data
+        inflater.setInput(input.toArray)
+        val output = new ByteArrayOutputStream()
+        val buffer = new Array[Byte](1024)


is it possible we reuse this buffer?

He-Pin · 2026-07-01T19:24:32Z

I think we can merge this, and improvement can come up later, wdyt @pjfanning I checked the netty's implemenation, the current is great and correct. but yes, netty can have better performance.

pjfanning · 2026-07-01T23:47:18Z

merged the existing PR - we can mark this as experimental in the next release if there are still issues that need addressing at that point

mkurz · 2026-07-08T10:17:37Z

Support Pekko HTTP WebSocket compression playframework/playframework#14075

This was referenced Jun 30, 2026

Compression Extensions for WebSocket playframework/playframework#4186

Closed

Please add support for WebSocket compression #61

Closed

pjfanning reviewed Jun 30, 2026

View reviewed changes

Comment thread http-core/src/main/scala/org/apache/pekko/http/impl/engine/ws/PerMessageDeflate.scala

pjfanning reviewed Jun 30, 2026

View reviewed changes

Comment thread http-core/src/main/scala/org/apache/pekko/http/javadsl/model/ws/WebSocketUpgrade.scala

pjfanning reviewed Jun 30, 2026

View reviewed changes

pjfanning added this to the 2.0.0-M2 milestone Jun 30, 2026

He-Pin reviewed Jul 1, 2026

View reviewed changes

mkurz force-pushed the websocket-compression branch from 5af04ba to 50c0cde Compare July 1, 2026 08:51

pjfanning reviewed Jul 1, 2026

View reviewed changes

mkurz force-pushed the websocket-compression branch from 50c0cde to c6e2be2 Compare July 1, 2026 09:16

pjfanning approved these changes Jul 1, 2026

View reviewed changes

mkurz added 2 commits July 1, 2026 12:04

Allow disabling WebSocket compression per upgrade apache#61

d626ab3

* Add WebSocketUpgrade overloads that accept a compressionEnabled flag. * Wire accepted WebSockets to decline negotiated compression per request. * Document and test per-upgrade compression disabling.

mkurz force-pushed the websocket-compression branch from c6e2be2 to d626ab3 Compare July 1, 2026 10:04

He-Pin reviewed Jul 1, 2026

View reviewed changes

He-Pin approved these changes Jul 1, 2026

View reviewed changes

He-Pin reviewed Jul 1, 2026

View reviewed changes

Comment thread http-core/src/main/scala/org/apache/pekko/http/impl/engine/ws/PerMessageDeflate.scala

He-Pin reviewed Jul 1, 2026

View reviewed changes

He-Pin requested a review from jrudolph July 1, 2026 18:48

He-Pin added the enhancement New feature or request label Jul 1, 2026

He-Pin reviewed Jul 1, 2026

View reviewed changes

pjfanning merged commit 07bd955 into apache:main Jul 1, 2026
5 checks passed

mkurz deleted the websocket-compression branch July 2, 2026 04:46

mkurz mentioned this pull request Jul 8, 2026

Support Pekko HTTP WebSocket compression playframework/playframework#14075

Merged

Uh oh!

Conversation

mkurz commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changed

Details

Netty comparison

Security notes

Testing

References

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

He-Pin commented Jul 1, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mkurz commented Jul 1, 2026

Uh oh!

mkurz commented Jul 1, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pjfanning left a comment

Choose a reason for hiding this comment

Uh oh!

mkurz commented Jul 1, 2026

Uh oh!

He-Pin left a comment

Choose a reason for hiding this comment

PerMessageDeflate.scala

Configuration

Minor / Style

Uh oh!

He-Pin left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mkurz Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mkurz commented Jul 1, 2026

Uh oh!

He-Pin left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mkurz commented Jun 30, 2026 •

edited

Loading

`PerMessageDeflate.scala`

mkurz Jul 1, 2026 •

edited

Loading