Skip to content

[hotfix] Move chat-model os.environ mutations out of module scope in e2e tests#689

Merged
wenjin272 merged 1 commit into
apache:mainfrom
weiqingy:fix-e2e-env-var-leakage
May 19, 2026
Merged

[hotfix] Move chat-model os.environ mutations out of module scope in e2e tests#689
wenjin272 merged 1 commit into
apache:mainfrom
weiqingy:fix-e2e-env-var-leakage

Conversation

@weiqingy
Copy link
Copy Markdown
Collaborator

@weiqingy weiqingy commented May 19, 2026

Purpose of change

os.environ[X] = Y mutations at module scope in e2e test files leak into the rest of the pytest process at collection time. The mechanism:

  • tools/ut.sh runs pytest flink_agents -k "not e2e_tests".
  • -k only filters which tests execute, not which files are collected. Pytest imports every matching test file during the collection phase.
  • Importing an e2e module that does os.environ[X] = Y at module scope executes the assignment as a side effect, polluting X for every later-collected test (e.g. the integrations/ tests).

PR #686 fixed this for OLLAMA_EMBEDDING_MODEL. This PR covers the same pattern for the chat-model-side env vars.

Files fixed

File Env var(s) moved out of module scope
e2e_tests_resource_cross_language/chat_model_cross_language_test.py:44 OLLAMA_CHAT_MODEL
e2e_tests_resource_cross_language/yaml_cross_language_test.py:72 OLLAMA_CHAT_MODEL
e2e_tests_integration/chat_model_integration_test.py:32,34,36,38,42 TONGYI_CHAT_MODEL, OLLAMA_CHAT_MODEL, OPENAI_CHAT_MODEL, AZURE_OPENAI_CHAT_MODEL, AZURE_OPENAI_API_VERSION
e2e_tests_integration/react_agent_test.py:54 OLLAMA_CHAT_MODEL

Approach

Used monkeypatch.setenv(...) inside the test function bodies. This:

  • Sets the env var only when the test actually runs (no collection-time pollution)
  • Auto-restores the original value after the test (no inter-test leakage either)
  • Preserves runtime propagation to the Flink subprocess

In chat_model_integration_test.py also converted the already-inside-function os.environ["MODEL_PROVIDER"] = model_provider to monkeypatch.setenv for consistency — otherwise the parametrize iterations (Ollama / Tongyi / OpenAI / AzureOpenAI) would leak MODEL_PROVIDER between runs.

Tests

  • ./tools/lint.sh -c clean
  • AST parse of all 4 files clean
  • The actual proof comes when CI runs and the chat-model tests no longer pollute the integrations test process. (No new test added because the underlying behavior — running an e2e test — is unchanged; the fix is purely about test-process hygiene.)

API

No API changes — test files only.

Documentation

  • doc-needed
  • doc-not-needed
  • doc-included

…e2e tests

Same collection-time pollution pattern that PR apache#686 fixed for
OLLAMA_EMBEDDING_MODEL exists for the chat-model side env vars
(OLLAMA_CHAT_MODEL, TONGYI_CHAT_MODEL, OPENAI_CHAT_MODEL,
AZURE_OPENAI_CHAT_MODEL, AZURE_OPENAI_API_VERSION) in four e2e test files:

  e2e_tests_resource_cross_language/chat_model_cross_language_test.py
  e2e_tests_resource_cross_language/yaml_cross_language_test.py
  e2e_tests_integration/chat_model_integration_test.py
  e2e_tests_integration/react_agent_test.py

tools/ut.sh runs `pytest flink_agents -k "not e2e_tests"`, and -k only
filters which tests execute (not which files are collected). pytest
imports every matching file during the collection phase, so any
module-scope os.environ mutation in an e2e file leaks into the rest of
the test process.

Removed the module-scope assignments and moved them into the test
functions via monkeypatch.setenv (one of three approaches @wenjin272
listed in PR apache#686 review), which preserves runtime propagation to the
Flink subprocess while auto-restoring after each test. In
chat_model_integration_test.py also converted the
already-inside-function `os.environ["MODEL_PROVIDER"] = model_provider`
to monkeypatch.setenv for consistency, since the parametrize
iterations would otherwise leak MODEL_PROVIDER between runs.

Reported by @wenjin272 in
apache#686 (comment)
("we can fix the issues with the Open and Ollama scripts in this PR
alone, and submit a separate PR to address all environment variable
leakage issues"). PR apache#686 addressed the OLLAMA_EMBEDDING_MODEL side;
this PR covers the chat-model side.
@github-actions github-actions Bot added doc-not-needed Your PR changes do not impact docs fixVersion/0.3.0 The feature or bug should be implemented/fixed in the 0.3.0 version. priority/major Default priority of the PR or issue. and removed doc-not-needed Your PR changes do not impact docs labels May 19, 2026
Copy link
Copy Markdown
Collaborator

@wenjin272 wenjin272 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wenjin272 wenjin272 merged commit bb29d0e into apache:main May 19, 2026
26 checks passed
@weiqingy weiqingy deleted the fix-e2e-env-var-leakage branch May 19, 2026 23:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc-not-needed Your PR changes do not impact docs fixVersion/0.3.0 The feature or bug should be implemented/fixed in the 0.3.0 version. priority/major Default priority of the PR or issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants