Reduce false sharing in MemoryCache

[EgorBo]

Follow-up to #131470, which fixed the same class of problem in System.Runtime.Caching.Counters.

CoherentState._cacheSize is Interlocked-updated on every Set/Remove when SizeLimit is configured, and sat immediately after _stringEntries/_nonStringEntries, which every TryGetValue loads. Moved onto its own cache line.

Results

Benchmark: #131696 (comment)

@EgorBot, main vs PR, mixed 7:1 read/write against a SizeLimit-configured cache. Ratio > 1 means the PR is faster:

Threads	linux-x64 (Turin)	linux-arm64 (Cobalt 100)	windows-arm64 (Cobalt 100)
1	1.00	1.00	1.00
4	1.53	1.06	1.26
8	2.27	1.64	1.58
16	2.43	1.52	1.28
32	2.20	1.09	1.57

Neutral single-threaded and improving under contention on all three targets. The Unsized control arm - same workload with no SizeLimit, so nothing writes _cacheSize - stays within 0.99-1.02 everywhere, confirming the gain is the padding rather than an unrelated effect.

Note

This section was generated with GitHub Copilot.

[azure-pipelines]

Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-extensions-caching
See info in area-owners.md if you want to be subscribed.

[Copilot]

Pull request overview

This PR aims to reduce false sharing in two hot, contended library paths by adjusting field layout/padding so contended atomics don’t share cache lines with frequently-read data.

Changes:

• CounterAggregator: move PaddedDouble.Value to the end of the 64-byte struct to avoid colliding with array metadata cache-line traffic.
• MemoryCache: move the size-limit accounting counter off the same cache line as the hot dictionary references by introducing padded storage and accessing it via a ref return.
• Microsoft.Extensions.Caching.Memory.csproj: include the shared Internal/Padding.cs so PaddingHelpers.CACHE_LINE_SIZE is available.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
src/libraries/System.Diagnostics.DiagnosticSource/src/System/Diagnostics/Metrics/CounterAggregator.cs	Adjusts per-slot padding layout so Value is placed at the end of the cache-line-sized struct.
src/libraries/Microsoft.Extensions.Caching.Memory/src/Microsoft.Extensions.Caching.Memory.csproj	Links in shared Internal/Padding.cs to provide Internal.PaddingHelpers.CACHE_LINE_SIZE.
src/libraries/Microsoft.Extensions.Caching.Memory/src/MemoryCache.cs	Introduces a padded wrapper for the cache-size counter and updates all atomic reads/writes to target it.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (1)

src/libraries/System.Diagnostics.DiagnosticSource/src/System/Diagnostics/Metrics/CounterAggregator.cs:79

• The comment hard-codes the array Length field offset as 8 bytes. That’s true on 64-bit, but it’s pointer-size dependent (e.g., 4 bytes on 32-bit) and may mislead future readers. Consider removing the specific offset and referring to the array header/Length field more generally.

        // 64 bytes is the size of a cache line on many systems; larger ones may see a little more
        // false sharing. Value sits at the end rather than at offset 0 so element 0 doesn't share a
        // line with the array's Length field at offset 8: the bounds check in Update loads Length on
        // every call from every thread, and a plain load of a line that is also a contended atomic's
        // target defeats far-atomic handling on Arm64.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (1)

src/libraries/Microsoft.Extensions.Caching.Memory/src/MemoryCache.cs:806

• The comment explains 64-byte cache lines, but the padding struct is sized to 128 bytes; it’s not obvious to a future reader why 128 is chosen or what guarantee it provides. Consider explicitly stating that the struct is 128 bytes so the Value at +64 is at least one 64B cache line away from adjacent fields on both sides (even if the runtime reorders class fields).

            // _stringEntries/_nonStringEntries are read on every TryGetValue; the padding keeps the
            // write-hot atomic off the line holding those read-mostly references. It has to live in
            // a struct -- layout attributes on a class only affect marshaling, and the runtime
            // reorders class fields freely. 64 is the size of a cache line on many systems; larger
            // ones may see a little more false sharing.

[EgorBo]

@EgorBot -windows_arm -linux_arm -linux_x64

using BenchmarkDotNet.Attributes;
using Microsoft.Extensions.Caching.Memory;
using System.Diagnostics.Metrics;
using System.Diagnostics.Tracing;

[MemoryDiagnoser(false)]
public class MemoryCacheContention
{
    private const int TotalOps = 1 << 20;
    private const int Keys = 1024;

    private MemoryCache _sized;
    private MemoryCache _unsized;
    private string[] _keys;

    [Params(1, 4, 8, 16, 32)]
    public int Threads;

    [GlobalSetup]
    public void Setup()
    {
        _keys = new string[Keys];
        for (int i = 0; i < Keys; i++)
            _keys[i] = "key" + i;
        _sized = new MemoryCache(new MemoryCacheOptions { SizeLimit = 1 << 20 });
        _unsized = new MemoryCache(new MemoryCacheOptions());
        for (int i = 0; i < Keys; i++)
        {
            _sized.Set(_keys[i], i, new MemoryCacheEntryOptions { Size = 1 });
            _unsized.Set(_keys[i], i);
        }
    }

    [GlobalCleanup]
    public void Cleanup()
    {
        _sized.Dispose();
        _unsized.Dispose();
    }

    // SizeLimit set => every Set does Interlocked.Add on CoherentState._cacheSize, which before
    // this change shared a cache line with the dictionary references every TryGetValue loads.
    [Benchmark(OperationsPerInvoke = TotalOps)]
    public void Sized() => Mixed(_sized, sized: true);

    // Control: no SizeLimit, so _cacheSize is never written. Should be flat across PR and main.
    [Benchmark(OperationsPerInvoke = TotalOps)]
    public void Unsized() => Mixed(_unsized, sized: false);

    // 7 reads per write: readers load _stringEntries while a writer bumps _cacheSize.
    private void Mixed(MemoryCache cache, bool sized)
    {
        int perThread = TotalOps / Threads;
        var barrier = new Barrier(Threads);
        var workers = new Thread[Threads];
        for (int t = 0; t < Threads; t++)
        {
            int id = t;
            workers[t] = new Thread(() =>
            {
                barrier.SignalAndWait();
                int k = id * 31;
                for (int i = 0; i < perThread; i++)
                {
                    k = (k + 1) & (Keys - 1);
                    if ((i & 7) == 0)
                        if (sized)
                            cache.Set(_keys[k], i, new MemoryCacheEntryOptions { Size = 1 });
                        else
                            cache.Set(_keys[k], i);
                    else
                        cache.TryGetValue(_keys[k], out _);
                }
            });
            workers[t].Start();
        }
        for (int t = 0; t < Threads; t++)
            workers[t].Join();
    }
}

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

Suppressed comments (1)

src/libraries/Microsoft.Extensions.Caching.Memory/src/MemoryCache.cs:809

• The cache-line padding uses hard-coded 64/128-byte constants. In this repo, other false-sharing mitigations typically use Internal.PaddingHelpers.CACHE_LINE_SIZE (e.g., System.Runtime.Caching/Counters.cs) to cover larger cache lines; with the current values, the separation is only guaranteed for 64-byte cache lines and may be ineffective on 128-byte systems.

Consider linking $(CommonPath)Internal\Padding.cs into this project (as other libs do) and basing CacheSizePadded on PaddingHelpers.CACHE_LINE_SIZE (e.g., Size = 2 * CacheLineSize and [FieldOffset(CacheLineSize)]).

            // reorders class fields freely. 64 is the size of a cache line on many systems; larger
            // ones may see a little more false sharing.
            private CacheSizePadded _cacheSizePadded;

            [StructLayout(LayoutKind.Explicit, Size = 128)]