Skip to content

Deprecated ContainerProperties#setTransactionManager breaks proper implementation of Idempotent Receiver pattern #4211

@RuslanHryn

Description

@RuslanHryn

The ContainerProperties#setTransactionManager method was deprecated in favor of KafkaAwareTransactionManager.
However, the transactionManager plays an important role when implementing Idempotent Receiver patterns that rely on storing processed message IDs in a SQL database to ensure consistency and prevent duplicate processing.

See Spring Kafka documentation: Filtering Messages

Implementation
A common approach to building an idempotent receiver involves coupling message consumption and database operations within the same local transaction:

  1. Configure a JDBC TransactionManager using ContainerProperties#setTransactionManager.
    → The container opens a database transaction for each processed record.
  2. Implement a RecordFilterStrategy that checks if the incoming message ID already exists in the database; if yes, the record is filtered.
  3. Since both @KafkaListener and RecordFilterStrategy execute in the same transaction, it guarantees atomic writes of business data and idempotent consumer records.

Problem

Removing support for transactionManager eliminates the ability to perform this pattern correctly using database transactions.

Recommendation
Retain support for transactionManager for database transaction usage while aligning Kafka transaction handling with KafkaAwareTransactionManager.
A potential implementation approach could look like this:

if (kafkaAwareTransactionManager != null) {
    invokeRecordListenerInKafkaTx(records); // use AfterRollbackProcessor
} else if (jdbcTransactionManager != null) {
    invokeRecordListenerInJdbcTx(records); // use CommonErrorHandler
} else {
    doInvokeWithRecords(records); // use CommonErrorHandler
}

This allows KafkaAwareTransactionManager for Kafka and DB transactions and transactionManager for local database transactions, preserving the ability to implement Idempotent Receivers safely and consistently.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions