Support numpy 1.24.x, make tests pass with latest dependencies#2767
Open
rly wants to merge 22 commits intoAllenInstitute:masterfrom
Open
Support numpy 1.24.x, make tests pass with latest dependencies#2767rly wants to merge 22 commits intoAllenInstitute:masterfrom
rly wants to merge 22 commits intoAllenInstitute:masterfrom
Conversation
- Update numpy<1.24 to numpy<1.25 in requirements.txt and doc_requirements.txt - Fix numpy 1.24+ compatibility issue in test conftest.py where rng.choice() no longer accepts sequences with inhomogeneous shapes (lists of different lengths). Use rng.integers() to generate indices instead. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Update numpy version constraint to allow 1.24.x
- Update pytest_ignore_collect hooks to use collection_path parameter instead of deprecated path parameter (py.path.local -> pathlib.Path) - Remove pytest.mark decorators from fixtures (marks on fixtures have no effect in pytest 9+) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Fix pytest 9 deprecation warnings
- Update moto from 3.0.7 to >=5.0.0 in test_requirements.txt - Replace deprecated `from moto import mock_s3` with `from moto import mock_aws` - Replace all `@mock_s3` decorators with `@mock_aws` This fixes JSONDecodeError issues caused by moto 3.x incompatibility with newer botocore versions returning chunked transfer encoding data. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Update moto to v5+ and fix mock_s3 to mock_aws imports
- Replace deprecated nwbfile.modules with nwbfile.processing - Fix IndexSeries unit from 'None' to 'N/A' (required in newer pynwb) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
- Replace deprecated nwbfile.modules with nwbfile.processing - Fix IndexSeries unit from 'None' to 'N/A' (required in newer pynwb) - Deprecate _add_image_index_to_nwb method (11/2025): IndexSeries indexed_timeseries field is deprecated in pynwb 2.x - Add warning when to_nwb() would have previously added IndexSeries 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Fix pynwb usage
Replace pytest.warns(None) with warnings.catch_warnings(record=True) to capture warnings in newer pytest versions where pytest.warns(None) raises TypeError. Also rename local 'warnings' variables from pytest.warns() contexts to avoid shadowing the warnings module. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
- Change macos-latest to macos-13 for x86_64 architecture since macos-latest now refers to ARM64 (Apple Silicon) runners which have conda package compatibility issues - Update setup-miniconda from v2 to v3 for better compatibility - Update actions/checkout from v2/v3 to v4 across all workflows 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
In aiohttp 3.9+, ClientSession must be created within a running event loop. Changed AsyncHttpEngine to lazily create the session on first access rather than in __init__, fixing "RuntimeError: no running event loop" errors when instantiating AsyncHttpEngine outside async context. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Fix misc test errors
In pupil_position_on_monitor_in_degrees(), np.linalg.norm([x, mag]) created an inhomogeneous array where x is an array and mag is a scalar. Use np.vstack with np.full_like to create a proper 2D homogeneous array. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
The test was not mocking res.x, causing MagicMock objects to be mixed with numpy scalars when creating arrays, which fails in numpy 1.24+ due to inhomogeneous array restrictions. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Feature/fix inhomogeneous array errors
- Update python_requires to >=3.10 in setup.py - Update classifiers to list Python 3.10, 3.11, 3.12 - Update CI workflows to test only Python 3.10, 3.11, 3.12 - Update on-prem docker images from py38 to py310 - Handle missing jpeg_twok/glymur by deferring ImportError to use time - Remove Python 2 compatibility code (six.iteritems, __future__ division) Python 3.8 reached EOL in October 2024 and Python 3.9 is in its final security-only maintenance phase. Dropping these versions allows us to use modern glymur which requires Python 3.10+ syntax (match-case). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
numpy 1.24.x doesn't support Python 3.12+ due to removed pkgutil.ImpImporter. Restrict to Python 3.10-3.11 until numpy version constraints are relaxed. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Drop Python 3.8/3.9 support and handle missing jpeg_twok/glymur gracefully
|
|
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
scipy.optimize.curve_fit can converge to different solutions on ARM64 vs x86_64 due to floating point differences. Added rtol/atol tolerance to affected tests. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
12 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview:
My main goal was to update the numpy dependency from
<1.24to<1.25to address compatibility issues with other packages in my environment -- primarily pynwb and hdmf for which the latest versions require numpy>=1.24. However, running the test suite with the latest dependencies, regardless of the numpy change, presented several errors and deprecations. Those are addressed in this PR as well. I am happy to break up this PR into smaller ones if that is preferred, but most of the tests will not pass unless all of these changes (except the numpy change) are merged. I believe the tests that are failing are not due to changes in this PR. The linting issues derive from other parts of the code base -- I think the linter has become more strict.Addresses:
Type of Fix:
functionality to not work as expected)
Solution:
Changes:
This was done with the help of Claude Code.
Validation:
Screenshots:
Unit Tests:
Script to reproduce error and fix:
Configuration details:
Checklist
Allen Institute Contribution Guidelines
Numpy Standards
appropriate
rc/x.y.z as its merge target
Notes: