Add external python predictor

[danielwolff1]

Description and Context

With this PR, I'm proposing a new predictor class which allows to defer the actually implementation of the predictor to python. We need this feature as we want to study the behavior of Neural Network-based structural predictors.

Implementation details

I implemented this feature similarly to what has been done in the context of a python surrogate model for rough surface contact laws as merged in #1735. The general idea of the implementation are:

• Only static cases are supported so far, i.e., load stepping is fine, but no dynamics.
• 4C execution might happen in parallel, yet the computations in python only happen in a single interpreter, i.e., in a single process.
• Process with id 0 communicates with python.
• The logic for communicating with python via pybind11 functionality is completely encapsulated in an Implementation class so that it does not interfere with the actual predictor.
• As for my personal application case I need to evaluate the neural network on the computational mesh, I need a lightweight representation of the discretization in python. Since this is a special requirement of my own research, the feature is implemented in a way that the communication of the mesh happens in an optional setup step, which will simply be skipped, if the python file does not provide a setup() method. That way the user can easily control whether to have this once-per-simulation overhead of communicating the mesh or whether they only need the "regular" information that also the C++ predictors see.

Testing

I have implemented two integration tests where I test the two major different control flows, i.e., once the python script has a setup method and the mesh needs to be communicated and once not. In both cases, the python script implements the simple logic of a ConstDis predictor and the corresponding values in the RESULTS DESCRIPTION section have been generated by running the same input scripts however with PREDICT = ConstDis.

I'm still planning to look into unit tests for this feature but the general implementation already works and won't change anymore, so I already wanted to put it out for review.

I appreciate any comments and insights for potential improvements 😊

[mayrmt]

@danielwolff1 I left some comments and raised some questions, that I could not answer for myself right away. For me, it looks like this is going into the right direction.

[danielwolff1]

Thanks for the fast review, I appreciate the input! I still have to look into some of the points you mentioned and will do so next week probably.

[isteinbrecher]

Thanks for this feature @danielwolff1 ! Nice to see how easy it is to incorporate python within 4C.

[mayrmt]

@danielwolff1 You could squash the commits.

[amgebauer]

@danielwolff1 could you have a look at the failing nightly pipeline? https://github.com/4C-multiphysics/4C/actions/runs/24694150094/job/72223568409

[danielwolff1]

After a brief look, the tests fail because numpy is not known to the python interpreter used to run the tests, which essentially means that the runner uses a different python environment than the one that is generally available in 4C (because this features numpy as a dependency). I encountered this error already earlier which is why I adapted the workflow file here in the attempt to make the installed python dependencies known to the executing python interpreter.

I'll have to investigate this more closely and will do so as quickly as possible, but unfortunately I'm at a conference this week, so it might take a couple of days.

[amgebauer]

I encountered this error already earlier which is why I adapted the workflow file here in the attempt to make the installed python dependencies known to the executing python interpreter.

That explains why the nightly pipeline if failing. These tests are defined in nightly_tests.yml and not in buildtest.yml. However, I'm not sure whether the added modification of the python path is what we want actually.

Edit: I think it makes more sense to just activate the corresponding python environment so that the correct environment is in the path during testing. Then, 4C should pick the correct one and should find all dependencies.

[danielwolff1]

okay perfect, I didn't know that we already have a python environment (or how to activate it). Do you know if I can simply run . venv/bin/activate instead of modifying the paths as I do at the moment? I will push a quick fix then.