[FLINK-39426] Add flink-connector-migration bridge module for Sink V2…#28084
Open
negiparas-bridge4 wants to merge 2 commits intoapache:masterfrom
Open
[FLINK-39426] Add flink-connector-migration bridge module for Sink V2…#28084negiparas-bridge4 wants to merge 2 commits intoapache:masterfrom
negiparas-bridge4 wants to merge 2 commits intoapache:masterfrom
Conversation
Collaborator
…ttingSink as backward compatibility bridges Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fde62e7 to
b1b89f3
Compare
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
|
Thanks for this PR. Why can't we migrate the connectors to use the new interfaces? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is the purpose of the change
In Flink 2.0, the deprecated convenience interfaces
StatefulSinkandTwoPhaseCommittingSinkwere removed as part of FLINK-36245. Many existing connectors (e.g., JDBC connector 3.3.0-1.20, Kafka connector 3.x) were compiled against Flink 1.x and still reference these types in their bytecode. When users add such connectors to a Flink 2.x project, they get:cannot access org.apache.flink.api.connector.sink2.StatefulSink class file for org.apache.flink.api.connector.sink2.StatefulSink not foundThese interfaces were originally deprecated in Flink 1.19 via FLIP-372 and replaced with the mixin pattern (
Sink+SupportsWriterState,Sink+SupportsCommitter). However, the removal in 2.0 happened after only ~6 months of deprecation, while the migration umbrella (FLINK-28045) still has open subtasks and major connectors had not yet migrated.This PR restores both interfaces directly in
flink-coreas@Deprecatedbridges — matching FLIP-372's original backward-compatible design. This requires zero action from users: connectors compiled against Flink 1.x resolve the class names automatically, and the runtime'sinstanceof SupportsWriterState/instanceof SupportsCommitterchecks pass transparently.Brief change log
StatefulSinkinflink-coreas a deprecated bridge extendingSink+SupportsWriterState, including inner type aliases (StatefulSinkWriter,WithCompatibleState)TwoPhaseCommittingSinkinflink-coreas a deprecated bridge extendingSink+SupportsCommitter, including inner type alias (PrecommittingSinkWriter) and thecreateCommitter()no-arg delegation chain for old connectorsinstanceofchecks) that the Flink runtime relies on, including a test for the no-argcreateCommitter()delegation pathVerifying this change
This change is already covered by new tests:
StatefulSinkBridgeTest— verifiesStatefulSinkis recognized asSinkandSupportsWriterState, and that inner type aliases extend the correct core interfacesTwoPhaseCommittingSinkBridgeTest— verifiesTwoPhaseCommittingSinkis recognized asSinkandSupportsCommitter, thatPrecommittingSinkWriterextendsCommittingSinkWriter, and that the no-argcreateCommitter()delegation works for legacy connectorsDoes this pull request potentially affect one of the following parts
@Public(Evolving): yes — restores@PublicEvolving@Deprecatedbridge interfaces inflink-coreDocumentation
Was generative AI tooling used to co-author this PR?