Skip to content

test(py): declare numpy so the ZPayloadView tests cannot silently skip - #274

Open
YuanYuYuan wants to merge 4 commits into
mainfrom
test/numpy-zero-copy
Open

test(py): declare numpy so the ZPayloadView tests cannot silently skip#274
YuanYuYuan wants to merge 4 commits into
mainfrom
test/numpy-zero-copy

Conversation

@YuanYuYuan

@YuanYuYuan YuanYuYuan commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

Summary

TestZPayloadViewNumpy guards on pytest.importorskip("numpy"), and numpy is declared nowhere in this repo — not in pyproject.toml dependencies, not as an extra, not in scripts/test-python.nu, not in hiroz-msgs-py. It resolves in CI today through some path the repo does not declare.

This PR declares it and makes its absence fail instead of skip.

Relates to #266.

Key Changes

  • Declare a test extra (numpy>=1.21) in pyproject.toml, and install via pip install -e ".[test]". The dependency is declared once, and the tests' own failure message points at that command.
  • Replace importorskip with an import that calls pytest.fail. numpy is now declared, so its absence is a misconfiguration rather than a reason to quietly skip.

What fails without this

Nothing fails today, and an earlier revision of this description was wrong to claim otherwise. It said both tests "have always skipped" and that every run ends 86 passed, 2 skipped. Neither is true. On main right now:

tests/test_payload_view.py::TestZPayloadViewNumpy::test_numpy_frombuffer PASSED
tests/test_payload_view.py::TestZPayloadViewNumpy::test_numpy_zero_copy_verification PASSED
======================= 48 passed, 7 warnings in 17.03s ========================

48 passed, 0 skipped. numpy is importable in CI's venv, so importorskip passes through and the assertions run.

So this is a hardening change with no failing baseline, and the justification is that the current green is contingent rather than guaranteed:

Declaring the dependency and failing loudly converts "green because numpy happened to be present" into "green because numpy is required and present". That is worth doing on its own terms, but it is a smaller and different claim than restoring lost coverage, because no coverage is currently lost.

Scope of what the assertions prove, unchanged and still worth stating: OWNDATA is False proves numpy borrowed the buffer ZPayloadView exported rather than copying it. It does not prove the transport was zero-copy — ZPayloadView::new falls back to PayloadBytes::Owned for a fragmented payload, and numpy borrows that owned copy just as happily. The transport-level check is payload.is_zero_copy_py, asserted by test_payload_view_is_zero_copy.

Breaking Changes

None. Test-environment change only, plus a new optional extra.

`TestZPayloadViewNumpy` guarded on `pytest.importorskip("numpy")`, and
numpy was installed by nothing -- not pyproject dependencies, not an
extra, not the test script. The venv is built with plain `python -m venv`
so it inherits nothing from the devShell. Both tests had therefore always
skipped, and every run ended "86 passed, 2 skipped".

What never ran is the only assertion in the suite that proves the
zero-copy claim:

    assert arr.flags["OWNDATA"] is False

If ZPayloadView regressed to copying, the whole suite would still pass.

Install numpy as a test dependency, and replace the importorskip with a
real import so a missing numpy fails instead of skipping. A permanently
skipped test is indistinguishable from a passing one in the summary line.

Closes #266
Adversarial review caught that the new failure message instructs the user
to "install it with the test extra", and no such extra existed --
pyproject.toml has no [project.optional-dependencies] at all. numpy was
installed by exactly one imperative line in the test script.

A contributor doing the natural manual setup (venv, pip install -e .,
maturin develop, pytest) previously got 2 skips and now gets 2 hard errors
whose stated remedy fails with "no extra 'test'".

Declare test = ["numpy>=1.21"] and have the script install via it, so the
dependency is declared in one place and the message is true.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Ensures NumPy-based ZPayloadView buffer-sharing tests execute instead of skipping.

Changes:

  • Adds NumPy as a test extra.
  • Installs test dependencies in the Python venv.
  • Fails tests when NumPy is unavailable.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
scripts/test-python.nu Installs the test extra during setup.
crates/hiroz-py/tests/test_payload_view.py Replaces conditional skipping with failure.
crates/hiroz-py/pyproject.toml Declares the NumPy test dependency.
Comments suppressed due to low confidence (1)

scripts/test-python.nu:50

  • pip install -e '.[test]' invokes the maturin PEP 660 backend and builds the Rust extension, after which maturin develop builds it again. Because the second build changes RUSTFLAGS, Cargo cannot reuse the first compilation, doubling this expensive setup work for every distro. Pass RUSTFLAGS to the editable install and use it as the single extension build.
    run-cmd "cd crates/hiroz-py; source .venv/bin/activate && pip install -e '.[test]'" --shell bash --distro (get-distro)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread crates/hiroz-py/tests/test_payload_view.py Outdated
Comment thread crates/hiroz-py/pyproject.toml Outdated
Comment thread scripts/test-python.nu Outdated
Copilot was right and the claim was overstated in three places. OWNDATA
== False proves numpy borrowed the buffer ZPayloadView exported rather
than copying it. It does not prove the transport was zero-copy:
ZPayloadView::new falls back to PayloadBytes::Owned for a fragmented
payload, and numpy borrows that owned copy just as happily.

The transport-level check is `payload.is_zero_copy_py`, asserted by
test_payload_view_is_zero_copy, which was already running.

The reason to fix #266 is unchanged -- two tests had never executed and a
permanent skip reads as a pass -- but the value claim now matches what
the assertion can support.
@YuanYuYuan YuanYuYuan changed the title test(py): actually run the ZPayloadView zero-copy assertions test(py): declare numpy so the ZPayloadView tests cannot silently skip Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants