Improve Arm64 for .NET11

[a74nh]

We will continue to improve the Arm64 port for .NET11. This will include improving code quality, enhancing SVE support and other general performance improvements.

Improvements to SVE APIs

• SVE2 APIs - Finishing the remaining SVE2 APIs that were started in .NET10 and fix up any inconsistencies
• SVE Extensions: AES, SHA3, SM4
• AdvSimd versions of the above SVE Extensions: SHA3, SM4
• Testing cleanups. This will simplify testing, expand the testing scope and possibly find bugs
• Add benchmarks to dotnet/performance

Agnostic SVE

• Accelerating Vector with SVE on ARM64

SVE Performance

• General performance improvements
• Fix movpfrx usage

Codegen Improvements

• Continue to add more compact encodings

Improvements in GC

• Add an Arm64 version of the vxsort algorithm in the GC. vxsort: Add Arm64 Neon implementation and tests #110692

PAC-RET

• Enable PAC-RET protection for the stack

[a74nh]

@dotnet/arm64-contrib @dotnet/jit-contrib @JulieLeeMSFT

[tannergooding]

Add benchmartks to dotnet/performance

I left a comment related to this here: dotnet/performance#5046, I will repost for simplicity

I have a general concern over all the SVE specific microbenchmarks being added.

Benchmarks are fairly expensive in terms of runtime and even a small number of them can have significant cost to CI and our tracking. We don't really have any platform specific intrinsic benchmarks correspondingly (i.e. you don't see explicit benchmarks covering AdvSimd or Avx512).

Rather instead we typically have our normal benchmarks like for Span, Linq, Tensors and other areas which internally accelerate using SIMD and we measure those across a range of hardware. We have some machines that support newer ISAs and can then correlate and see the more real world improvements based on that.

I would expect here that we aren't directly testing SVE either. But rather would be testing with SVE enabled and comparing that against a run with it disabled. This will require a bit more work in the JIT to enable first, but significantly reduces cost and gives better metrics as to the benefit customers will see.

CC. @DrewScoggins

[a74nh]

Add benchmartks to dotnet/performance

I left a comment related to this here: dotnet/performance#5046, I will repost for simplicity

I have a general concern over all the SVE specific microbenchmarks being added.
Benchmarks are fairly expensive in terms of runtime and even a small number of them can have significant cost to CI and our tracking. We don't really have any platform specific intrinsic benchmarks correspondingly (i.e. you don't see explicit benchmarks covering AdvSimd or Avx512).
Rather instead we typically have our normal benchmarks like for Span, Linq, Tensors and other areas which internally accelerate using SIMD and we measure those across a range of hardware. We have some machines that support newer ISAs and can then correlate and see the more real world improvements based on that.
I would expect here that we aren't directly testing SVE either. But rather would be testing with SVE enabled and comparing that against a run with it disabled. This will require a bit more work in the JIT to enable first, but significantly reduces cost and gives better metrics as to the benefit customers will see.
CC. @DrewScoggins

Let's take a step back then. What are my motivations for the benchmarks?

• We know that codegen for SVE is harder to get right compared to AdvSimd. Mostly due to mask handling, but probably other reasons too.
• Right now we don't know how much impact the non-optimal codegen is having on performance
• Even if codegen is fixed, it's fairly easy for the user to accidentally write non-optimal SVE code.
• Having some well coded C# routines for SVE, alongside their scalar and AdvSimd equivalents will allow us to keep track of the performance limitations.
• In addition, they will serve as examples for users writing their own code - either directly or via AI scraping.
• We don't require these benchmarks to be run constantly. Once a week, or even less, feels reasonable given we don't expect figures to change unless explicit intrinsic changes are made.
• We do want something a developer can easily run to check their own changes. We need to know if the changes we are doing are worthwhile.
• It doesn't have to be in dotnet/performance, but that felt like the most public place
• Do we have different tiers for the performance tests?

[tannergooding]

Writing intrinsics is an advanced scenario that is primarily getting done by low level framework authors, it isn't something we want apps and other common library code to deal with or think about. Rather we want most apps/libraries to just use things like Span<T>, LINQ, TensorPrimitives, and other key APIs we provide that are already accelerated for them, allowing them to work at a higher level.

For the low level framework authors, we then don't want them to end up writing platform specific code paths for most scenarios. We rather want and encourage them to use the xplat API surface because it trivializes the support considerations and gives the JIT the flexibility to optimized based on the underlying hardware.

For example, consider that we have the following potential SIMD platforms and ISA targeting considerations:

• Arm64
  • AdvSimd
  • Sve
• x86/x64
  • Sse42
  • Avx2
  • Avx512
  • Avx10v2
• WASM
  • PackedSimd
• RISC-V
• Loongarch
• etc

Writing a platform-specific path for each platform is untenable and largely unnecessary for most domains; so, we would much prefer they write "one path" using Vector<T> which then allows it to implicitly work and be optimal across all of these. There will of course be exceptions, particularly where there is some opportunistic lightup and the platform-specific path can be mixed into the xplat logic to enable a key significant optimization. But that should still end up being with most logic using the xplat APIs (i.e. vector + vector, not Sve.Add(vector, vector)) and will effectively look like:

// xplat logic

if (Sve.IsSupported)
{
    // sve specific logic
}
else
{
    // xplat fallback
}

// xplat logic

We ultimately want to provide an API surface that is easy, natural, and largely intuitive for users to get correct. This is critical to their success and the success of the APIs we expose as well. So, the lower level the API, the more "responsibility" is on the user to write correct code and understand the potential implications of what they write.

---

Given all of that, I feel many of the points raised on why SVE perf tests are being added end up being related to internal details to the runtime, which are not relevant to the user. The things that are important to measure, particularly as they apply to the user, are the higher level APIs which should have Vector<T> paths or have those paths added, and which should be measured with SVE enabled vs disabled so that we can get more accurate "real world" impact measurements of the SVE changes. Those same users wanting to test their code are ideally just running their existing end to end apps, getting their existing profiling data, etc

For AI, this feels rather something we should create a base prompt or explicit documentation for instead. We shouldn't be relying on AI taking the "average scrapings of the internet" for the user-experience. I expect doing that will end up suboptimal, particularly given this is an advanced and niche scenario that is hand rolling your own vectorization logic, so the ratio of "correct information" vs "subtly incorrect or partial information" is likely not high enough to get reliably accurate results. Providing explicit context/prompts for developers working in this space will help ensure they get the best suggestions without relying solely on the "average scrapings"

Do we have different tiers for the performance tests?

I don't believe so, but @DrewScoggins would likely be the one to answer