Skip to content

KAFKA-20771: Enforce max msg bytes in share group DLQ produce path.#22802

Open
smjn wants to merge 2 commits into
apache:trunkfrom
smjn:KAFKA-20771
Open

KAFKA-20771: Enforce max msg bytes in share group DLQ produce path.#22802
smjn wants to merge 2 commits into
apache:trunkfrom
smjn:KAFKA-20771

Conversation

@smjn

@smjn smjn commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator
  • Add dlqTopicMaxMessageBytes(topic) to
    ShareGroupDLQMetadataCacheHelper,
    honoring a topic-level max.message.bytes override or else falling back
    to the broker's dynamically-reconfigurable message.max.bytes (via
    IntSupplier, not a captured value).
  • ShareGroupDLQStateManager now chunks
    an oversized DLQ offset range across multiple produce requests with a
    resumable per-handler integer, and defers (rather than fails) coalesced
    handlers that would push a merged batch over the limit, re-enqueuing
    them for a later round instead of sending oversized requests.
  • Also, switches the per-record size check to an incremental calculation
    (DefaultRecord.sizeInBytes) instead of re-serializing the batch-so-far
    on every offset.
  • Pins message.timestamp.type=CreateTime on
    auto-created DLQ topics so timestamps aren't silently overwritten by
    the cluster's broker-level default.
  • Add new tests and update existing wherever applicable.
  • Add integ test to verify the code path.

@github-actions github-actions Bot added core Kafka Broker KIP-932 Queues for Kafka triage PRs from the community labels Jul 10, 2026
@smjn smjn added ci-approved and removed triage PRs from the community labels Jul 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-approved clients core Kafka Broker KIP-932 Queues for Kafka

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant