Re-implement benchmarks in pytest

[pipliggins]

Description

Moves PyBaMM's speed and peak memory tests away from asv and into the pytest ecosystem, using pytest-benchmark and pytest-memray.

Adds a new workflow to check that the speed benchmarks in a PR do not regress compared to main.

To discuss: What to do with pybamm-bench. This stores the asv benchmarks over time and provides a gh-pages website to display them. The workflow to populate the repo hasn't run since April 2024 so the site is fairly out of date.
One option is to use the https://github.com/benchmark-action/github-action-benchmark, which should be able to provide a similar workflow and display plots in a similar manner to asv, should we want to keep some record.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.09%. Comparing base (8013d49) to head (54d25d1).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5630      +/-   ##
==========================================
- Coverage   98.19%   98.09%   -0.10%     
==========================================
  Files         339      339              
  Lines       32069    32069              
==========================================
- Hits        31489    31459      -30     
- Misses        580      610      +30     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[martinjrobins]

thanks @pipliggins, this looks good, a few questions below. https://github.com/benchmark-action/github-action-benchmark looks really good, can you please add these plots? (good find, I think I might add this to diffsol too!)

[agriyakhetarpal]

Thanks! I only did a quick pass just yet; it would be useful for us to set up these new benchmarks in such a way that we ensure they are as reliable as possible. In conda, we used CodSpeed (https://codspeed.io) until very recently – it is a good solution to eliminate statistical outliers from GitHub Actions runners and has a free tier. There is also https://bencher.dev, which is also free, which we've started using. I would recommend setting up either of those to go with this PR, for a start (I am not opposed to https://github.com/benchmark-action/github-action-benchmark at all, just that it might make sense to adopt these solutions in addition for their low variability).

We already have a 301 redirect set up for the GitHub Pages website at https://pybamm.org/benchmarks, so I can add that in the footer somewhere or change the link wherever our new location will be.

[pipliggins]

Thanks! I only did a quick pass just yet; it would be useful for us to set up these new benchmarks in such a way that we ensure they are as reliable as possible. In conda, we used CodSpeed (https://codspeed.io) until very recently – it is a good solution to eliminate statistical outliers from GitHub Actions runners and has a free tier. There is also https://bencher.dev, which is also free, which we've started using. I would recommend setting up either of those to go with this PR, for a start (I am not opposed to https://github.com/benchmark-action/github-action-benchmark at all, just that it might make sense to adopt these solutions in addition for their low variability).

We already have a 301 redirect set up for the GitHub Pages website at https://pybamm.org/benchmarks, so I can add that in the footer somewhere or change the link wherever our new location will be.

I saw Bencher come up a lot as I was googling around this, it looks good. We'd need an account associated with pybamm to 'claim' the project and view results - @brosaplanella is that something you could do? A potential issue with the free tier is it limits you to a 5m job timeout. On my mac it takes 4.30 - 5 mins to run the full suite, so we're likely to be close to/over that limit.