[tmva][pymva] Improve evaluations in pymva and add support for Keras3 (TF >= 2.16) #15790

lmoneta · 2024-06-07T14:16:46Z

This Pull request starts adding support for new TF version (2.16) coming with Keras3 API.

In addition, it improves the evaluation of models for regression and multi-class by using all events instead of the single event as before. With TF 2.16 s not possible anymore to disable eager execution and have a decent speed i evaluating the model

github-actions · 2024-06-07T16:24:23Z

Test Results

23 files 23 suites 3d 19h 12m 38s ⏱️
3 835 tests 3 833 ✅ 0 💤 2 ❌
80 507 runs 80 505 ✅ 0 💤 2 ❌

For more details on these failures, see this check.

Results for commit 1414858.

♻️ This comment has been updated with latest results.

dpiparo · 2024-06-10T18:05:39Z

Thanks for the PR. Is this code tested anywhere?

dpiparo

lgtm

guitargeek · 2025-05-04T18:06:57Z

Hi @lmoneta, how should we proceed here? There is now TensorFlow 2.19 already, and the last TensorFlow we support is still the 2.15 release. I think the real question is: it it actually worth it to support inference with TensorFlow/Keras given the declining popularity of TensorFlow?

The list can go on.

guitargeek · 2025-05-04T18:09:43Z

Also, maybe it makes sense to split the TensorFlow compatibility and the other commit with the general improvements into two separate PRs? Then we can already merge the other improvements, because they are covered by the CI. For the TensorFlow compatibility, we have once again the chicken-and-egg problem that it will only be covered once we allow TF>=2.16 in our requirements.txt for the CI images to pick up, but then builds will be red until the compatibility fix is merged...

Right now we don't support the newest TensorFlow releases after the 2.15 series, initially released in November 2023 [1]. Support for TensorFlow>=2.16 is worked on [2], but it's difficult to make progress if we don't have TensorFlow>=2.16 in our CI images. And we veto it from our CI image via the `requirements.txt` mechanism. To continue, we need to temporarily disable PyMVA in the CI, such that we can include the newest TensorFlow version in our images, and then re-enable it in the PR that fixes support for the newest TensorFlow. This is what is done in this commit. Supporting the newest TensorFlow became a bit more important now that we also have another feature besides PyMVA that uses TensorFlow: the RBatchGenerator. It would not good to hold off testing the actively-developed RBatchGenerator on newer TensorFlow versions, just because PyMVA prevents us from installing these on our CI. [1] https://github.com/tensorflow/tensorflow/releases [2] root-project#15790

Right now we don't support the newest TensorFlow releases after the 2.15 series, initially released in November 2023 [1]. Support for TensorFlow>=2.16 is worked on [2], but it's difficult to make progress if we don't have TensorFlow>=2.16 in our CI images. And we veto it from our CI image via the `requirements.txt` mechanism. To continue, we need to temporarily disable PyMVA in the CI, such that we can include the newest TensorFlow version in our images, and then re-enable it in the PR that fixes support for the newest TensorFlow. This is what is done in this commit. Supporting the newest TensorFlow became a bit more important now that we also have another feature besides PyMVA that uses TensorFlow: the RBatchGenerator. It would not good to hold off testing the actively-developed RBatchGenerator on newer TensorFlow versions, just because PyMVA prevents us from installing these on our CI. [1] https://github.com/tensorflow/tensorflow/releases [2] #15790

Right now we don't support the newest TensorFlow releases after the 2.15 series, initially released in November 2023 [1]. Support for TensorFlow>=2.16 is worked on [2], but it's difficult to make progress if we don't have TensorFlow>=2.16 in our CI images. And we veto it from our CI image via the `requirements.txt` mechanism. To continue, we need to temporarily disable PyMVA in the CI, such that we can include the newest TensorFlow version in our images, and then re-enable it in the PR that fixes support for the newest TensorFlow. This is what is done in this commit. Supporting the newest TensorFlow became a bit more important now that we also have another feature besides PyMVA that uses TensorFlow: the RBatchGenerator. It would not good to hold off testing the actively-developed RBatchGenerator on newer TensorFlow versions, just because PyMVA prevents us from installing these on our CI. [1] https://github.com/tensorflow/tensorflow/releases [2] root-project#15790

Right now we don't support the newest TensorFlow releases after the 2.15 series, initially released in November 2023 [1]. Support for TensorFlow>=2.16 is worked on [2], but it's difficult to make progress if we don't have TensorFlow>=2.16 in our CI images. And we veto it from our CI image via the `requirements.txt` mechanism. To continue, we need to temporarily disable PyMVA in the CI, such that we can include the newest TensorFlow version in our images, and then re-enable it in the PR that fixes support for the newest TensorFlow. This is what is done in this commit. Supporting the newest TensorFlow became a bit more important now that we also have another feature besides PyMVA that uses TensorFlow: the RBatchGenerator. It would not good to hold off testing the actively-developed RBatchGenerator on newer TensorFlow versions, just because PyMVA prevents us from installing these on our CI. [1] https://github.com/tensorflow/tensorflow/releases [2] #15790

guitargeek · 2025-06-25T07:44:14Z

For the TensorFlow compatibility, we have once again the chicken-and-egg problem that it will only be covered once we allow TF>=2.16 in our requirements.txt for the CI images to pick up, but then builds will be red until the compatibility fix is merged...

This problem was resolved now by disabling tmva-pymva in the CI, so that the newest TensorFlow could be installed in the CI images. @lmoneta, if you want, you can now resolve the merge conflicts in this PR to continue working on compatibility with the newest TensorFlow, and our CI will cover it! You just have to turn tmva-pymva back on as well:
https://github.com/root-project/root/blob/master/.github/workflows/root-ci-config/buildconfig/global.txt#L94

Right now we don't support the newest TensorFlow releases after the 2.15 series, initially released in November 2023 [1]. Support for TensorFlow>=2.16 is worked on [2], but it's difficult to make progress if we don't have TensorFlow>=2.16 in our CI images. And we veto it from our CI image via the `requirements.txt` mechanism. To continue, we need to temporarily disable PyMVA in the CI, such that we can include the newest TensorFlow version in our images, and then re-enable it in the PR that fixes support for the newest TensorFlow. This is what is done in this commit. Supporting the newest TensorFlow became a bit more important now that we also have another feature besides PyMVA that uses TensorFlow: the RBatchGenerator. It would not good to hold off testing the actively-developed RBatchGenerator on newer TensorFlow versions, just because PyMVA prevents us from installing these on our CI. [1] https://github.com/tensorflow/tensorflow/releases [2] root-project#15790

This follows up on 3923ba5, where I forgot to change this. If a feature is not tested in the CI and included in the binaries, we should also not enable it by default when people build ROOT from source. Also, clarify the TensorFlow support status in the option description (see also root-project#15790).

This follows up on 3923ba5, where I forgot to change this. If a feature is not tested in the CI and included in the binaries, we should also not enable it by default when people build ROOT from source. Also, clarify the TensorFlow support status in the option description (see also #15790).

Right now we don't support the newest TensorFlow releases after the 2.15 series, initially released in November 2023 [1]. Support for TensorFlow>=2.16 is worked on [2], but it's difficult to make progress if we don't have TensorFlow>=2.16 in our CI images. And we veto it from our CI image via the `requirements.txt` mechanism. To continue, we need to temporarily disable PyMVA in the CI, such that we can include the newest TensorFlow version in our images, and then re-enable it in the PR that fixes support for the newest TensorFlow. This is what is done in this commit. Supporting the newest TensorFlow became a bit more important now that we also have another feature besides PyMVA that uses TensorFlow: the RBatchGenerator. It would not good to hold off testing the actively-developed RBatchGenerator on newer TensorFlow versions, just because PyMVA prevents us from installing these on our CI. [1] https://github.com/tensorflow/tensorflow/releases [2] root-project#15790

The new Keras 3 API broke the existing TMVA `PyKeras` method. Re-implementing the method for Keras 3 turned out to be difficult, as it is not possible anymore to disable eager execution and have a decent speed in evaluating the model for single events. So large-scale refactoring would be necessary to implement `PyKeras` again with good performance (see also root-project#15790). Nowadays, we also have the RBatchGenerator to train Keras models directly with batches that are provided by ROOT. Therefore, the TMVA `PyKeras` method is not the only way anymore to train a Keras model with ROOT data without 3rd party libraries for the IO. That means it's not an essential feature anymore, and deprecating it will even make the situation clearer for the user, as there are not two different ways anymore to train Keras models on ROOT data.

Add support for the new Keras API when using tensorflow 2.16 Note that with the new Keras one needs in pymva to use the ".keras" format instead of the ".h5" format

…ticlass on all dataset events Implement the new MethodBase functions GetAllRegressionValues and GetAllMultiClassValues for MethodPyKeras and MethodPyTorch. This speeds up the model testng and evaluation also for regression and multi-class problems. Fix also some memory issues with the Python objects used to keep the input/output values in the model evaluation. Remove timer logging in the pymva GetMultiClassValues functions. The logging is done anyway at the level of MethodBase Increase statistics and re-add the test on regression error in testPyKerasRegression

Do not use anymore .h5 files but .keras since .h5 is not anymore supported in keras3

guitargeek · 2026-01-13T10:15:29Z

Hi, thanks for picking this up again! Could you put tmva-pymva=ON in some Linux CI configs of your choice so we get test coverage of these developments?

guitargeek

Nice, thank you very much! The test failures on alma10 are related to other PyMVA features and SOFIE, so for PyKeras it's all good 👍

2026-01-13T16:45:47.3921839Z TEST FAILURES:
2026-01-13T16:45:47.3922008Z 497:PyMVA-AdaBoost-Classification
2026-01-13T16:45:47.3922208Z 498:PyMVA-AdaBoost-Multiclass
2026-01-13T16:45:47.3922424Z 1442:tutorial-machine_learning-TMVA_SOFIE_RDataFrame-py

Once the temporary test commit changing alma10.txt is removed, the PR is good to be merged!

the algorithm option is not anymore supported from scikit versions 1.6 Remove it in the MethodAdaBoost The SOFIE tutorials which require pymva need also a valid Keras parser, so exclude them if Keras is an unsupported version

guitargeek · 2026-01-13T21:14:49Z

Looks like some of the PyMVA features don't work on ARM, which is shown on our alma10 ARM build.

Maybe it's fine for you to enable the PyMVA features on another platform, e.g. alma9 instead? For this platform, we don't have an ARM build. Like this, we can already get coverage for tmva-pymva in this PR, and maybe worry about ARM support later (if you don't know an easy fix).

lmoneta requested a review from bellenot as a code owner June 7, 2024 14:16

lmoneta force-pushed the pymva_support_keras3 branch from 584c49f to 6e3a9d9 Compare June 10, 2024 08:20

dpiparo assigned lmoneta Jun 10, 2024

dpiparo self-requested a review June 26, 2024 08:27

dpiparo approved these changes Jun 26, 2024

View reviewed changes

guitargeek added the in:TMVA label Aug 21, 2024

guitargeek mentioned this pull request Mar 30, 2025

[TMVA] Python module compatibility problems #15309

Closed

guitargeek mentioned this pull request Jun 22, 2025

[CI] Disable PyMVA so we can support the newest TensorFlow versions #19124

Merged

guitargeek mentioned this pull request Jun 24, 2025

[CI] Enable PyMVA #19163

Closed

guitargeek mentioned this pull request Aug 2, 2025

[CMake] Disable tmva-pymva by default #19512

Merged

guitargeek mentioned this pull request Nov 30, 2025

[tmva][pymva] Exercise removal of deprecated TMVA PyKeras method #20577

Closed

guitargeek mentioned this pull request Dec 1, 2025

[tmva][pymva] Deprecate the PyKeras method #20588

Open

lmoneta added 2 commits January 12, 2026 10:45

[tmva][pymva] Add support for new Keras (3) API when using TF 2.16

8580318

Add support for the new Keras API when using tensorflow 2.16 Note that with the new Keras one needs in pymva to use the ".keras" format instead of the ".h5" format

lmoneta force-pushed the pymva_support_keras3 branch from 6e3a9d9 to 81ce66b Compare January 12, 2026 10:39

lmoneta requested a review from couet as a code owner January 12, 2026 11:19

[tmva][pymva] Apply fixes in tutorials and test for Keras3

13b40ca

Do not use anymore .h5 files but .keras since .h5 is not anymore supported in keras3

lmoneta force-pushed the pymva_support_keras3 branch from 1c1763f to 13b40ca Compare January 12, 2026 15:55

lmoneta requested a review from guitargeek January 13, 2026 14:48

guitargeek approved these changes Jan 13, 2026

View reviewed changes

guitargeek mentioned this pull request Jan 13, 2026

[tmva][sofie] New Keras Parser from SOFIE #20871

Open

[tmva][pymva] Fix AdaBoost for new version of scikit-learn

9f0d6cb

the algorithm option is not anymore supported from scikit versions 1.6 Remove it in the MethodAdaBoost The SOFIE tutorials which require pymva need also a valid Keras parser, so exclude them if Keras is an unsupported version

lmoneta force-pushed the pymva_support_keras3 branch from 3b0fe11 to c3956bd Compare January 13, 2026 18:10

Enable tmva-pymva on alma9 alma10 , ubunu 22 and 24 for testing

1414858

lmoneta force-pushed the pymva_support_keras3 branch from c3956bd to 1414858 Compare January 13, 2026 22:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[tmva][pymva] Improve evaluations in pymva and add support for Keras3 (TF >= 2.16) #15790

[tmva][pymva] Improve evaluations in pymva and add support for Keras3 (TF >= 2.16) #15790

lmoneta commented Jun 7, 2024

Uh oh!

github-actions bot commented Jun 7, 2024 •

edited

Loading

Uh oh!

dpiparo commented Jun 10, 2024

Uh oh!

dpiparo left a comment

Uh oh!

guitargeek commented May 4, 2025 •

edited

Loading

Uh oh!

guitargeek commented May 4, 2025

Uh oh!

guitargeek commented Jun 25, 2025 •

edited

Loading

Uh oh!

guitargeek commented Jan 13, 2026

Uh oh!

guitargeek left a comment

Uh oh!

guitargeek commented Jan 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

[tmva][pymva] Improve evaluations in pymva and add support for Keras3 (TF >= 2.16) #15790

Are you sure you want to change the base?

[tmva][pymva] Improve evaluations in pymva and add support for Keras3 (TF >= 2.16) #15790

Conversation

lmoneta commented Jun 7, 2024

Uh oh!

github-actions bot commented Jun 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results

Uh oh!

dpiparo commented Jun 10, 2024

Uh oh!

dpiparo left a comment

Choose a reason for hiding this comment

Uh oh!

guitargeek commented May 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

guitargeek commented May 4, 2025

Uh oh!

guitargeek commented Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

guitargeek commented Jan 13, 2026

Uh oh!

guitargeek left a comment

Choose a reason for hiding this comment

Uh oh!

guitargeek commented Jan 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions bot commented Jun 7, 2024 •

edited

Loading

guitargeek commented May 4, 2025 •

edited

Loading

guitargeek commented Jun 25, 2025 •

edited

Loading