Feature/asv workflow

[prasad-sawantdesai]

GitHub Actions workflow (asv.yml) added to run ASV performance benchmarks on public test data

run_benchmarks.sh (Bamboo) refactored: adds a proper upstream remote pointing to iterorganization/IBEX, fetches develop/main from there, and returns to the triggering branch before running benchmarks.

configure_env.sh updated for modules

run_pytest.sh adds pytest-xdist and fixes -n=auto → -n auto for parallel test execution.

build_docs.sh simplifies by replacing explicit package installs with pip install -r ../docs/requirements.txt.

pyproject.toml fixes license metadata for setuptools <77 compatibility,
pins sphinx>=7,<9 and sphinx-autosummary-accessors==2025.3.1 for reproducible doc builds

Both, ITER CI plan on bamboo and Github workflow are up and running..

[maarten-ic]

Hi @prasad-sawantdesai,

Please note that github runners are Virtual Machines and the underlying hardware can change widely between different runs. Benchmark results of different runs are therefore incomparable: it may have run on faster or slower hardware, or on higher/lower loaded physical machines but you cannot know that. Github Actions is therefore not really suitable for tracking the software performance over time.

Doing performance comparisons against main or develop with PRs could be done, but then there should also be a process for following up on the results (e.g. fail the CI when performance has degraded "too much" -- and you'll need to think what threshold you'd find acceptable). I doubt that anyone would notice a performance regression if the benchmark workflow is always green 😉

Hope this helps!
Maarten

[prasad-sawantdesai]

Please note that github runners are Virtual Machines and the underlying hardware can change widely between different runs. Benchmark results of different runs are therefore incomparable: it may have run on faster or slower hardware, or on higher/lower loaded physical machines but you cannot know that. Github Actions is therefore not really suitable for tracking the software performance over time.

Doing performance comparisons against main or develop with PRs could be done, but then there should also be a process for following up on the results (e.g. fail the CI when performance has degraded "too much" -- and you'll need to think what threshold you'd find acceptable). I doubt that anyone would notice a performance regression if the benchmark workflow is always green 😉

Hope this helps! Maarten

Thanks Maarten. You are right about GitHub runners, we will also check from IT if we can setup few self hosted runners for these specific cases. Now instead of cross-run comparisons we are using asv continuous which runs both commits back-to-back on the same runner, so hardware is identical within a single comparison. Here added factor 2.0 makes asv continuous fails if any benchmark is >2x slower.

Lets see how it goes.. Meanwhile we will also keep running ITER CI to run test cases in MDSPLUS and HDF5 backend.

[deepakmaroo]

Looks good to me