Skip to content

KAFKA-16517 Do not decode metadata records in the internal kraft partition listener#22805

Open
0xffff-zhiyan wants to merge 2 commits into
apache:trunkfrom
0xffff-zhiyan:dont-decode-metadata-records
Open

KAFKA-16517 Do not decode metadata records in the internal kraft partition listener#22805
0xffff-zhiyan wants to merge 2 commits into
apache:trunkfrom
0xffff-zhiyan:dont-decode-metadata-records

Conversation

@0xffff-zhiyan

Copy link
Copy Markdown
Contributor

Summary

The internal kraft partition listener KRaftControlRecordStateMachine only tracks control records. When it reads the log and the snapshot it also decodes every data record through RecordsIterator, allocating a buffer and running the serde for each record even though the decoded records are never used. This change lets the listener skip decoding data records on both its log and snapshot read paths.

Changes

Introduce DecodingStrategy to decide which records RecordsIterator decodes. It has three implementations:

  1. ControlAndDataDecodingStrategy decodes both control and data records and preserves the existing behavior.
  2. ControlOnlyDecodingStrategy decodes only the control records and skips the data records, and needs no serde.
  3. DataOnlyDecodingStrategy decodes only the data records and skips the control records, added for completeness with no caller yet.
  • RecordsIterator now takes a DecodingStrategy instead of a RecordSerde, and the shared record and batch decoding logic is moved into static helpers on DecodingStrategy.
  • RecordsSnapshotReader.of also takes a DecodingStrategy instead of a RecordSerde so each snapshot reader can choose. Callers that consume data records pass ControlAndDataDecodingStrategy.
  • Add Batch.notDecoded which builds a batch that carries only the offset information for records that were skipped.
  • KRaftControlRecordStateMachine uses ControlOnlyDecodingStrategy for both its log and snapshot reads, so its serde field and constructor parameter are removed.

Testing

Existing RecordsIteratorTest, KRaftControlRecordStateMachineTest, snapshot reader tests, KafkaRaftClientReconfigTest, RaftEventSimulationTest, RaftClusterSnapshotTest, DynamicBrokerConfigTest and QuorumControllerTest pass with the change.

@github-actions github-actions Bot added triage PRs from the community core Kafka Broker kraft labels Jul 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Kafka Broker kraft triage PRs from the community

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant