[tmva][sofie] Add ONNX Gelu operator support

[AdityaDRathore]

This Pull request:

This PR implements the GELU operator (Opset 20) for TMVA/SOFIE.

Changes or fixes:

This PR implements the Gelu operator for the SOFIE inference engine, targeting ONNX Opset 20 specification. This addition extends SOFIE's support to modern transformer architectures which rely on the Gaussian Error Linear Unit.

Technical Approach

• Exact Formula: The implementation uses the exact formula $y = 0.5 \cdot x \cdot (1 + \text{erf}(x / \sqrt{2}))$.
• Precision: To guarantee bit-exact reproducibility across compilers, the code uses compile-time hexfloat constants for the coefficients (0x1.6a09e667f3bcdp-1 and 0.5).
• Parser Logic: The updated RModelParser_ONNX registers the Gelu node and explicitly validates attributes to reject unsupported approximations (like tanh), ensuring no silent numerical mismatches.

Key Features:

1. Bit-Exact Reproducibility: - Uses hexfloat constants (0x1.6a09e667f3bcdp-1 for $1/\sqrt{2}$) to guarantee identical floating-point behavior across all IEEE-754 platforms (x86/ARM/PPC).

   • avoids runtime std::sqrt(2) division which is susceptible to compiler-specific fast-math optimizations.

2. Safety: - Parser explicitly rejects approximate='tanh' (unsupported) to prevent silent model degradation.

3. Validation (New):

   • Added TestSofieGELU.cxx (GTest).
   • Validates generated code structure.
   • Verifies numeric correctness against SciPy/WolframAlpha reference values (Golden Data).
   • Enforces the presence of hexfloat constants in the generated output to prevent regression.

Validation

Validation was performed using a custom Opset-20 ONNX graph constructed via onnx.helper to prevent the operator decomposition often produced by torch.onnx.export.

• Ground Truth: Computed using SciPy's erf implementation.
• Result: Bit-exact match (Max error 0.0) against the ground truth on the test interval between [-3.0, 3.0].

Checklist:

• tested changes locally
• updated the docs (if necessary)

This PR fixes #

cc: @lmoneta @sanjibansg @omazapa

[AdityaDRathore]

@lmoneta @sanjibansg I saw there's an older PR #20788 for GELU, so I wanted to point out a few differences in my approach for your review:

1. Precision: I used compile-time hexfloats for 1/sqrt(2) instead of runtime division (x / sqrt(2)). This guarantees bit-exact results on all platforms and saves a division op.
2. Safety: My parser checks for approximate='tanh'. Since SOFIE doesn't support the approximation yet, I throw an error to prevent silent math mismatches.
3. Validation: I verified this against a forced Opset-20 graph (built with onnx.helper) to ensure I wasn't accidentally validating a decomposed PyTorch subgraph.

[Shlok-Saxena]

Hi @AdityaDRathore — thanks for the detailed comparison.

I’ve pushed updates to my PR (#20788) incorporating your suggestions:

• added explicit parsing of the ONNX GELU approximate attribute
• validated that approximate is of type STRING
• throw a clear runtime error for approximate="tanh" (not supported yet in SOFIE)
• throw an error for any other unsupported approximate value to avoid silent mismatches

Thanks again — this made the implementation more robust.

[AdityaDRathore]

Hi @Shlok-Saxena , glad the parser suggestions were helpful for robustness.

However, my primary concern regarding numeric stability remains addressed only in this PR (#20876). For HEP physics validation, we typically require strict bit-exactness across platforms.

Relying on x / std::sqrt(2) introduces runtime division and platform-dependent rounding differences depending on the compiler's FPU flags.

This PR uses compile-time hexfloats to lock in the precision.

I have added a GTest suite (TestSofieGELU.cxx) that not only checks the numeric output against SciPy Golden Data but also inspects the generated C++ code to enforce the presence of these hexfloat constants.