Skip to content

CI: gate PRs on the measurement/blufi/mcp/root unit suites (matrix)#129

Open
danielrmerskine wants to merge 1 commit into
mainfrom
de/unit-ci-matrix
Open

CI: gate PRs on the measurement/blufi/mcp/root unit suites (matrix)#129
danielrmerskine wants to merge 1 commit into
mainfrom
de/unit-ci-matrix

Conversation

@danielrmerskine

Copy link
Copy Markdown
Collaborator

Summary

Wire the remaining non-hardware unit suites into CI. unit-tests.yml ran only test/unit/cli/, so the measurement, blufi, MCP, and root unit suites executed in no CI at all. This PR runs each non-box suite as its own ubuntu-latest matrix job so they gate pull requests.

Supersedes #103, which fell 52 commits behind main — this is a fresh cut off current main. Unlike #103, the box suite is not wired here: on GitHub-hosted runners import lager resolves as a namespace package ("unknown location") and several box test files fail collection, a failure not reproducible locally or in a plain python:3.11 container. Box CI is tracked as a follow-up (and #103's three box test fixes move there with it).

Changes

  • .github/workflows/unit-tests.yml: run each suite as its own matrix job (cli, measurement, blufi, mcp, root). The suites can't share a pytest process — test/unit/measurement/conftest.py stubs the lager package in sys.modules (to avoid heavy box deps) — so each runs separately with --import-mode=importlib -c /dev/null --timeout=60. The job sets PYTHONPATH = repo root + box/ so box-side imports and test.* subprocess imports resolve consistently, in-process and in spawned subprocesses.
  • test/requirements-unit.txt (new): the pure-Python deps needed to import the code under test headless (numpy, simplejson, pyvisa, pyvisa-py, flask, pyserial, pyusb, cryptography, fastmcp, pexpect). Hardware SDKs stay mocked by the conftests, so nothing here pulls a device driver. pymupdf is intentionally excluded (AGPL-3.0, incompatible with this project's Apache-2.0 license); test/unit/test_pdf_pages.py importorskips fitz and skips without it.

Coverage

suite tests
cli (already gated) 545
measurement 102
blufi 72 (+4 skipped)
mcp 157
root 90 (+1 skipped)

~421 tests newly gate PRs; ~966 total in the matrix.

Testing

Notes / follow-ups

  • Box suite (~825 tests): runner-specific namespace-package collection failure (details above). The follow-up will add a diagnostic step for the runner import resolution, import hygiene (a single test/unit/box/conftest.py instead of per-file sys.path surgery), the box-only deps pygdbmi/flask-socketio, and CI: run box/mcp/measurement/blufi unit suites (matrix); fix 3 CI-fragile box tests #103's three box test fixes.
  • test/mcp/integration/ still needs live hardware, so it stays on the self-hosted bench, not this ubuntu job.

Checklist

  • Code follows the project's style guidelines
  • Copyright headers are present on new files
  • CHANGELOG.md updated — n/a (no user-facing change; CI only)
  • Documentation updated (if applicable) — the workflow header documents the per-suite rationale and the box exclusion

unit-tests.yml previously ran only test/unit/cli/. Run each non-box,
non-hardware suite as its own ubuntu-latest matrix job (cli, measurement,
blufi, mcp, root) so they gate pull requests. The suites cannot share a
pytest process (test/unit/measurement/conftest.py stubs `lager` in
sys.modules), so each job runs pytest separately with
--import-mode=importlib -c /dev/null --timeout=60, and with PYTHONPATH set
to repo root + box/ so box-side imports and `test.*` subprocess imports
resolve consistently.

Add test/requirements-unit.txt: the pure-Python deps needed to import the
code under test headless. Hardware SDKs stay mocked by the conftests.
pymupdf is intentionally excluded (AGPL-3.0, incompatible with this
project's Apache-2.0 license); test/unit/test_pdf_pages.py importorskips
fitz and skips without it.

The box suite is excluded for now: on GitHub-hosted runners `import lager`
resolves as a namespace package and several files fail collection, a
failure not reproducible locally or in a plain python:3.11 container.
Wiring it up is tracked as a follow-up.

Supersedes #103 (fresh cut off current main).

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new test/requirements-unit.txt file containing runtime dependencies for unit tests. The reviewer recommends pinning these dependencies to specific versions or compatible ranges to ensure reproducible CI runs and prevent unexpected breakages.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +13 to +22
numpy
simplejson
pyvisa
pyvisa-py
flask
pyserial
pyusb
cryptography
fastmcp
pexpect

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The dependencies listed here are completely unpinned. To ensure reproducible CI runs and prevent unexpected breakages when new versions of these packages are released, it is highly recommended to pin these dependencies to specific versions (e.g., numpy==1.26.4) or at least define compatible version ranges (e.g., numpy>=1.24.0,<2.0.0).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant