test CI: v4 URI encode fix#2
Closed
sls-dev-agent wants to merge 7 commits into
Closed
Conversation
Introduce a test infrastructure layer so SLS unit tests can run hermetically. - pyproject.toml: pytest config with `testpaths = [tests/unit, tests/e2e]`, an ``e2e`` marker, and DeprecationWarning filter. - setup.py: declare ``responses`` and ``pytest-mock`` under ``extras_require['test']``. - tests/_helpers/env.py: ``require_sls_env()`` reads the canonical ``LOG_TEST_*`` vars and falls back through the legacy aliases (``ALIYUN_LOG_SAMPLE_*`` / ``TEST_*`` / ``SLS_*``); skips the test if any required field is missing. - tests/_helpers/fakes.py: ``make_client`` / ``mock_sls_response`` / ``error_response`` wrappers around the ``responses`` library, mirroring the SLS error-envelope shape from ``aliyun.log.logexception``. - tests/unit/test_logclient_mock.py: three demo tests that prove the HTTP boundary can be stubbed at the requests level for ``get_logs``, ``put_logs`` (signature header + protobuf body) and the error path. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Reorganize the tests/ directory so a hermetic unit suite can run on every
push, while integration tests guarded by `LOG_TEST_*` env vars live under
their own subtree and skip cleanly when env is missing.
Layout:
tests/unit/ pytest collected by default, no network access
tests/e2e/ marked @pytest.mark.e2e, auto-skipped via conftest
tests/samples/ sample.py / sample_consumer.py (not pytest-collected)
tests/examples/ consumer_group_examples / export_examples / es_migration
tests/_helpers/ env + responses-based fakes (added in prior commit)
Notable moves:
tests/ut/ -> tests/unit/
tests/ci/build-test/ -> tests/unit/
tests/es_migration/test_*_converter.py / test_index_* -> tests/unit/
tests/integration_test/ -> tests/e2e/
tests/logclient/ -> tests/e2e/logclient/
tests/ingestion_test/ -> tests/e2e/ingestion/
tests/resource_test/ -> tests/e2e/resource/
tests/schedule_sql_test/ -> tests/e2e/schedule_sql/
tests/etl_test/ -> tests/e2e/etl/
tests/test_alert.py -> tests/e2e/test_alert.py
tests/compress_test.py -> split: lz4 part to tests/unit/test_compress.py,
integration parts to tests/e2e/test_compress_e2e.py
tests/sample*.py -> tests/samples/
tests/{consumer_group,export,rebuild_index}_examples/ -> tests/examples/
Each e2e module declares ``pytestmark = pytest.mark.e2e`` and uses
``require_sls_env()`` from tests/_helpers/env.py rather than reading os.environ
inline. Env variable names accept the canonical ``LOG_TEST_*`` plus the legacy
aliases ``ALIYUN_LOG_SAMPLE_*`` / ``TEST_*`` / ``SLS_*``.
CI (.github/workflows/build.yaml) gains a unit-test job that runs
``pytest tests/unit/`` on every push, plus an e2e-test job that runs only on
manual workflow_dispatch and consumes ``LOG_TEST_*`` from secrets.
TESTING.md documents the conventions; existing build-test job is unchanged.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Reverts the directory moves for sample/example/ci-build-test files that the previous refactor introduced. The PR-360 review surfaced that these locations are referenced from: - tox.ini (tests/sample.py, tests/sample_consumer.py) - README_CN.md, doc/tutorials/*.md (tests/consumer_group_examples/, tests/sample_object_api.py) - .github/workflows/py2-build.yaml (tests/ci/build-test/) Moving them broke those entrypoints. The unit/e2e split itself, along with tests/_helpers/, tests/conftest.py, and the new test files under tests/unit/ and tests/e2e/, are preserved. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
These two READMEs were added by the earlier refactor commit and were meant to be deleted in d3f6a5c (the revert that put samples/ and examples/ files back at their original paths). The directories were removed from disk but the README deletions never made it into the commit. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Following d3f6a5c (which moved sample/example/ci-build-test files back to their original locations), update: - .github/workflows/build.yaml: point the build-tests step at tests/ci/build-test/ again (was tests/unit/test_proto.py + tests/unit/test_type.py) - TESTING.md: redraw the directory map so it reflects the actual layout — top-level sample*.py, consumer_group_examples/, export_examples/, etc., plus tests/ci/build-test/ — instead of the samples/ and examples/ subdirs that no longer exist Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- build.yaml: upgrade checkout@v4/setup-python@v5, matrix 3.8/3.10/3.12 for build-test, 3.8/3.12 for unit-test - py2-build.yaml: replace ubuntu-20.04 pin with ubuntu-latest + Docker job containers (python:2.7.18-buster, 3.6.15-buster, 3.7.17-bullseye); add unit-test job for py3.7; pin pip/setuptools for py2.7 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The previous _v4_uri_encode iterated Unicode codepoints and emitted
"%{ord(c):02X}" — for any non-ASCII char with codepoint > 0xFF this
produced strings like "%4F60" or "%1F613", which are invalid
percent-encodings and don't match the canonical AWS SigV4 / SLS V4
encoding the server computes. Result: any request whose path/query
contained non-ASCII chars failed SignatureNotMatch.
Switch to urllib.parse.quote (already wrapped in util.urlquote for
py2/py3) with safe='~'. The safe='~' is needed because py2's
urllib.quote omits '~' from its default safe set while py3's does
not — passing it explicitly keeps output identical across both.
Verified on py3 (locally) and py2.7.18 (in sls-sdk-build container)
for unicode, ASCII, and pre-encoded bytes input.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Owner
Author
|
CI 全绿 (11/11)，AuthV4 unicode fix 在 py2.7/3.6/3.7/3.8/3.10/3.12 都兼容。关闭测试 PR。 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Validate AuthV4._v4_uri_encode UTF-8 fix across py2.7/3.6/3.7/3.8/3.10/3.12.