Guarantee needs_drop thread-local variables are destroyed at a well-defined point

[orlp]

First, some terminology. When a thread-local variable gets destroyed there are two possible relevant destructors:

1. The Drop implementation of the type stored in the thread-local variable. I will call these "user-space" destructors, as they are defined by Rust users.
2. The cleanup needed by the thread-local implementation to free any related memory. I will call these "internal" destructors, as they are defined by the standard library internals.

I've noticed that on some platforms we leave the moment when TLS variables are destroyed entirely up to the operating system/pthread implementation, for both user-space destructors as well as the internal destructors, whereas on other platforms we have an explicit destructor registry system where user-space destructors are registered to be called during thread clean-up.

I propose we instead always use a registry system for user-space destructors on all platforms, and call them at a well-defined point during thread clean-up. This has several advantages:

1. It guarantees that if needs_drop::<T>() is false for some thread-local variable, then LocalKey::with can never fail (assuming the constructor doesn't panic), even when user-space TLS destructors are being ran. This is of potential interest to global allocators, which are regularly called in other user-space destructors.
2. It can simplify the std::thread::current implementation which currently has to handle the case where its thread-local storage has been cleaned up already, meaning it has to synthesize a new Thread handle.
3. Because of (2) we can simplify the std::thread::Thread handle implementation to have a single shared canonical handle per thread. This allows us to implement extra functionality which was previously impossible/impractical, such as Thread::is_finished to see if a thread is truly finished just from its handle (which can be shared and passed around, unlike JoinHandle).
4. We can strengthen the guarantee of std::thread::JoinHandle::is_finished such that it only returns true if truly all user-defined Rust code on that thread has finished running - currently it only claims main is done.

@joboet Since this is your area of expertise I would be interested to hear your thoughts.

[RalfJung]

FWIW the issue title is hard to read since needs_drop::<T>() doesn't look like an adjective... I had to read it three times before I understood what you meant.

[RalfJung]

I've noticed that on some platforms we leave the moment when TLS variables are destroyed entirely up to the operating system/pthread implementation, for both user-space destructors as well as the internal destructors, whereas on other platforms we have an explicit destructor registry system where user-space destructors are registered to be called during thread clean-up.

I think the history is that we used the OS mechanism when it was possible, but sometimes no suitable mechanisms exists or it was deemed buggy, and then we bring our own implementation.

Would changing this to always use our own logic do anything about #28129 ?

[orlp]

@RalfJung I've edited the title. And yes, I think that would resolve #28129 if properly implemented.

[orlp]

One big open question mark here is how to handle the case when foreign (non-Rust) code also installs thread-local destructors, and those thread-local destructors call back into Rust code.