std: store key-based TLS variables in a hashmap

[joboet]

TLS keys are quite a limited resource (typical systems only have a few hundred of them), leading to issues like #155758 for complex programs like rustc that utilise too many. Also, the order of destruction cannot be easily modified, leading to behavioural inconsistencies between linker- and key-based TLS; with linker-based TLS: With linker-based TLS, variables without destructors are always available during destructor execution and variables with destructors cannot be initialised again once destroyed. However, with key-based TLS, the destruction order is arbitrary and there is no mechanism to prevent variable reinitialisation.¹

To fix these issues, this PR replaces the existing variable-per-key storage with a single hashmap for all variables, with the key into the hashmap being the address of a (non-thread-local) static. Variable destructors are stored in a list alongside the hashmap, and executed before the storage for destructorless variables is freed, matching the behaviour of linker-based TLS.

On GNU/Windows this will also simplify the destructor handling in the future by avoiding the need to keep track of destructors in the TLS key abstraction layer just to match the behaviour of the UNIX API. I've left that for a future PR, which means this PR will introduce some duplication; but I figured this PR is involved enough as-is.

On other systems like Fortanix SGX and Xous, this will improve things even further since they can now get rid of their manual TLS key implementation in favour of just storing a single pointer to the hashmap/destructor list state. But that's also not included here.

Fixes #155758

Footnotes

1. Aside from preventing reinitialisation during the execution of the variable's own destructor. ↩

[rustbot]

r? @tgross35

rustbot has assigned @tgross35.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

• Owners of files modified in this PR: @ChrisDenton, libs
• @ChrisDenton, libs expanded to 8 candidates
• Random selection from Mark-Simulacrum, tgross35

[joboet]

@bors try job=x86_64-mingw-1,x86_64-mingw-2

[rust-bors]

☀️ Try build successful (CI)
Build commit: a25b4be (a25b4be0732fd6bdb13c4d2d10ce00ecdbc3a1bb, parent: ca9203f29c609b344566d4eed9b5d9a3bf10290c)

[joboet]

@rustbot label +I-libs-nominated

[mati865]

FYI, it might be possible to get windows-gnu to use proper native TLS at some point this year: rust-lang/compiler-team#993

[rust-bors]

☔ The latest upstream changes (presumably #157616) made this pull request unmergeable. Please resolve the merge conflicts.

[the8472]

We discussed this during today's T-libs meeting.
The only raised concern was potential performance impact (do you have numbers?) and if there's an impact then what is the blast radius, i.e. which platforms are affected.

[tgross35]

There is some more detail at the meeting notes https://hackmd.io/@rust-libs/BJFCwR8Zfe#Nominated-rusttf156743-std-store-key-based-TLS-variables-in-a-hashmap, but it sounds like the team would like to see perf numbers.

If this is slower then maybe we could do a hybrid on systems where the platform TLS is fast? Like using the system implementation by default then have the hashmap fallback if we run out of keys.

[tgross35]

@rustbot author