IGNITE-28853 Harden compressed payload deserialization against corrupted streams#3
Open
anton-vinogradov wants to merge 1 commit into
Open
Conversation
…ted streams Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Open
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the fixes for the issues raised in the review of apache#13329 (see apache#13329 (comment)). Targets this PR's branch, so merging brings them straight into the PR.
What's fixed
tmpReader. The payload buffer handed totmpReaderis always complete, solastFinished == falseafter the inner deserialization is stream corruption, not a lack of data. Previously it propagated aslastRead = false, making the outer parser wait for socket bytes that never come (protocol desync), while the reusedtmpReaderkept half-read stream state (msgTypeDone, partialmsg,readSize) thatreset()doesn't clear and poisoned every subsequent compressed field on the connection. Now it throwsIgniteException, andtmpReaderis dropped in afinallyon any failure — including exceptions thrown from inside the inner deserialization.tmpReaderno longer pins the last uncompressed payload. The reader lives in session meta for the connection lifetime, and every stream that participated in the nested read kept a reference to the payload array (megabytes for exchange messages). NewreleasePayload()swaps all created streams to a shared empty buffer viaDirectMessageState.forEachItem().uncompress()validates the stream against the size header.chunks == nullwithdataSize > 0throws a diagnosableIgniteExceptioninstead of returningnullintoByteBuffer.wrap(null)(NPE); a 1032:1 sanity bound (max raw-deflate expansion) rejects a corrupted size header before thenew byte[dataSize]allocation; a one-byte probe inflate after the main loop restores the second direction of master'slength == dataSizeassert — a stream that inflates to more thandataSizenow fails instead of being silently truncated.readFrom()rejects negativedataSize(previouslyIllegalArgumentException/NegativeArraySizeExceptionfrom the guts of the serializer) and clamps the chunk-list pre-size, so a lying header can't inflate theArrayListcapacity allocation.assert finishedin@Setupreplaced with an explicit check — asserts are disabled in the JMH fork, a message outgrowing the buffer would silently benchmark a truncated stub.Tests
Five new cases in
CompressedMessageTest: negativedataSize,finalChunkwith no chunks, understateddataSize(single- and multi-chunk), and a fullreadMessage(true)path over an envelope whose payload is a truncated message serialization.Verification:
CompressedMessageTest7/7,DirectMarshallingMessagesTestgreen,-Pcheckstyleclean, benchmarks module compiles with-Pbenchmarks.Known limitations (intentional)
new byte[dataSize]allocation is bounded only by ~1032× of the actually received compressed bytes. Full protection would need incremental allocation (reintroduces the copying this PR removes) or a transport-level message size limit; note that master had no bound on the inflated size at all.IgniteExceptionstill lands in theGridNioServerworker catch block (pre-existing route, shared with this PR's own null-chunk guard).🤖 Generated with Claude Code