[hotfix] Move chat-model os.environ mutations out of module scope in e2e tests#689
Merged
Merged
Conversation
…e2e tests Same collection-time pollution pattern that PR apache#686 fixed for OLLAMA_EMBEDDING_MODEL exists for the chat-model side env vars (OLLAMA_CHAT_MODEL, TONGYI_CHAT_MODEL, OPENAI_CHAT_MODEL, AZURE_OPENAI_CHAT_MODEL, AZURE_OPENAI_API_VERSION) in four e2e test files: e2e_tests_resource_cross_language/chat_model_cross_language_test.py e2e_tests_resource_cross_language/yaml_cross_language_test.py e2e_tests_integration/chat_model_integration_test.py e2e_tests_integration/react_agent_test.py tools/ut.sh runs `pytest flink_agents -k "not e2e_tests"`, and -k only filters which tests execute (not which files are collected). pytest imports every matching file during the collection phase, so any module-scope os.environ mutation in an e2e file leaks into the rest of the test process. Removed the module-scope assignments and moved them into the test functions via monkeypatch.setenv (one of three approaches @wenjin272 listed in PR apache#686 review), which preserves runtime propagation to the Flink subprocess while auto-restoring after each test. In chat_model_integration_test.py also converted the already-inside-function `os.environ["MODEL_PROVIDER"] = model_provider` to monkeypatch.setenv for consistency, since the parametrize iterations would otherwise leak MODEL_PROVIDER between runs. Reported by @wenjin272 in apache#686 (comment) ("we can fix the issues with the Open and Ollama scripts in this PR alone, and submit a separate PR to address all environment variable leakage issues"). PR apache#686 addressed the OLLAMA_EMBEDDING_MODEL side; this PR covers the chat-model side.
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose of change
os.environ[X] = Ymutations at module scope in e2e test files leak into the rest of the pytest process at collection time. The mechanism:tools/ut.shrunspytest flink_agents -k "not e2e_tests".-konly filters which tests execute, not which files are collected. Pytest imports every matching test file during the collection phase.os.environ[X] = Yat module scope executes the assignment as a side effect, pollutingXfor every later-collected test (e.g. theintegrations/tests).PR #686 fixed this for
OLLAMA_EMBEDDING_MODEL. This PR covers the same pattern for the chat-model-side env vars.Files fixed
e2e_tests_resource_cross_language/chat_model_cross_language_test.py:44OLLAMA_CHAT_MODELe2e_tests_resource_cross_language/yaml_cross_language_test.py:72OLLAMA_CHAT_MODELe2e_tests_integration/chat_model_integration_test.py:32,34,36,38,42TONGYI_CHAT_MODEL,OLLAMA_CHAT_MODEL,OPENAI_CHAT_MODEL,AZURE_OPENAI_CHAT_MODEL,AZURE_OPENAI_API_VERSIONe2e_tests_integration/react_agent_test.py:54OLLAMA_CHAT_MODELApproach
Used
monkeypatch.setenv(...)inside the test function bodies. This:In
chat_model_integration_test.pyalso converted the already-inside-functionos.environ["MODEL_PROVIDER"] = model_providertomonkeypatch.setenvfor consistency — otherwise theparametrizeiterations (Ollama / Tongyi / OpenAI / AzureOpenAI) would leakMODEL_PROVIDERbetween runs.Tests
./tools/lint.sh -ccleanAPI
No API changes — test files only.
Documentation
doc-neededdoc-not-neededdoc-included