Skip to content

[dependency] Update flink version and constraint.#684

Merged
wenjin272 merged 2 commits into
apache:mainfrom
wenjin272:update-flink-version
May 16, 2026
Merged

[dependency] Update flink version and constraint.#684
wenjin272 merged 2 commits into
apache:mainfrom
wenjin272:update-flink-version

Conversation

@wenjin272
Copy link
Copy Markdown
Collaborator

@wenjin272 wenjin272 commented May 16, 2026

Linked issue: #683

Purpose of change

  1. Update flink version to: 1.20.4, 2.0.2, 2.1.2 & 2.2.1
  2. Update async execution for 1.20.4 & 2.1.2 don't contain pemja fix

Tests

it

API

No

Documentation

  • doc-needed
  • doc-not-needed
  • doc-included

@github-actions github-actions Bot added doc-not-needed Your PR changes do not impact docs fixVersion/0.3.0 The feature or bug should be implemented/fixed in the 0.3.0 version. priority/major Default priority of the PR or issue. and removed doc-not-needed Your PR changes do not impact docs labels May 16, 2026
wenjin272 and others added 2 commits May 16, 2026 20:51
When a JavaCollectionManageableVectorStore wraps a Java vector store
backed by a Python embedding model, the previous flow delegated query/
add/update to the Java side, which then called the embedding model from
Java. For a PYTHON-backed embedding model that callback is a Java→Python
re-entry on the Flink operator thread, and pemja's sub-interpreter state
on that thread ends up corrupted; the next pemja conversion in the same
thread crashes inside JcpPyDecimal_Check → PyImport_ImportModule with a
NULL sys.modules dereference.

Drop the add/query/update overrides in JavaVectorStoreImpl and inherit
BaseVectorStore's canonical flow: embed in Python first, then make a
single-direction Python→Java call to addEmbedding/queryEmbedding/
updateEmbedding. open() now also chains super().open() so the embedding
model resolves from string to instance before the Python embed step
runs. fromPythonVectorStoreQuery (its only caller is gone) and the
matching python_java_utils helpers (get_mode_value,
from_java_vector_store_query_result) are removed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@wenjin272 wenjin272 force-pushed the update-flink-version branch from be28e8d to 12dbe85 Compare May 16, 2026 12:51
@wenjin272 wenjin272 merged commit cfb7127 into apache:main May 16, 2026
44 of 46 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc-not-needed Your PR changes do not impact docs fixVersion/0.3.0 The feature or bug should be implemented/fixed in the 0.3.0 version. priority/major Default priority of the PR or issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant