RFC: Add a thread local storage module, std::tls #461

alexcrichton · 2014-11-11T19:06:00Z

Introduce a new thread local storage module to the standard library, std::tls,
providing:

Scoped TLS, a non-owning variant of TLS for any value.
Owning TLS, an owning, dynamically initialized, dynamically destructed
variant, similar to std::local_data today.

Introduce a new thread local storage module to the standard library, `std::tls`, providing: * Scoped TLS, a non-owning variant of TLS for any value. * Owning TLS, an owning, dynamically initialized, dynamically destructed variant, similar to `std::local_data` today.

bill-myers · 2014-11-12T12:29:45Z

text/0000-tls-overhaul.md

+lost when writing Unix-only code.
+
+Destructor support for Windows will be provided through a custom implementation
+of tracking known destructors for TLS keys.


On Windows, you can set a callback for when a thread exits, and you can then iterate over all TLS keys and destroy them if they are set.

See https://github.com/ChromiumWebApps/chromium/blob/master/base/threading/thread_local_storage_win.cc#L42 for how to do it without using DllMain.

Iterating over all possible TLS keys is quadratic instead of linear if they are sparsely used, but it should be possible to write code that directly accesses the Windows TEB to avoid this if desired.

Indeed! I ran across that same trick when whipping up the sample implementation. I definitely agree that the implementation can be improved as well!

aturon · 2014-11-13T20:44:46Z

cc @thestinger, would be good to get your feedback on this proposal.

thestinger · 2014-11-13T20:53:13Z

text/0000-tls-overhaul.md

+
+### Destructor support
+
+The major difference between Unix and Windows TLS support is that Unix supports


It's also possible to use only #[thread_local] with no secondary pthread_key_create / synchronization by using C++11 destructor support. On Linux, glibc defines a fn __cxa_thread_atexit_impl(dtor: unsafe extern "C" fn(ptr: *mut c_void), ptr: *mut c_void, dso_symbol: *mut i8) where dso_symbol can just be retrived by defining static mut __dso_handle: i8. On OS X, there's a fn _tlv_atexit(dtor: unsafe extern "C" fn(ptr: *mut c_void), ptr: *mut c_void); function. Rust could use the weak symbol trick to call these when available and fall back to a crappier implementation on top of dynamic TLS.

Indeed! If you take a look at the sample implementation you'll see that it does precisely that!

brson · 2014-11-18T19:45:38Z

ISTM like scoped TLS may be the 'preferred' way to use TLS as it is more structured, harder to create error-prone access patterns, but this design gives traditional tls the better module name.

brson · 2014-11-18T19:56:12Z

This RFC doesn't seem to explain how it achieves acceptable performance compared to local_data and #[thread_local], which is one of the main concerns.

alexcrichton · 2014-11-18T20:54:42Z

@brson that's a good point about naming, we could take no preference at all and perhaps move everything in to std::tls as well with DynamicKey and ScopedKey with dynamic_tls! and scoped_tls! macros. I think I'm somewhat biased towards "owned by default" (and so is the repo), but that's also probably because it's all we have today.

The sample implementation's benchmarking numbers are currently all on par with #[thread_local] (numbers are in some of the above comments), but I'll try to call it out specifically how this is trying to mirror the system primitives as much as possible

Today I also discovered an unsoundness with this API as proposed, specifically with the get method on owning TLS keys. The return Ref<T> satisfies the 'static bound, allowing you to store references to other TLS variables in TLS itself, perhaps causing undefined behavior during destructors. Which is to say that TLS variables can store stale references to one another:

#![feature(phase)]

#[phase(plugin, link)]
extern crate tls;

use std::cell::RefCell;

dynamic_tls!(static FOO: Box<int> = box 1)
dynamic_tls!(static BAR: RefCell<Option<tls::dynamic::Ref<Box<int>>>> = RefCell::new(None))

fn main() {
    *BAR.get().unwrap().borrow_mut() = Some(FOO.get().unwrap());
}

What this really wants to do is to store a &'thread T inside of Ref<T> to ensure it doesn't satisfy 'static and can't be stored in another global itself. I've been talking with @nikomatsakis on IRC about this and it may be possible, but it would prevent this from landing in the interim. For that reason I'm going to modify the get() api to with() so it has the same semantics as scoped tls.

jnicklas · 2014-11-20T21:32:55Z

When I hear TLS I think "Transport Layer Security", not "Thread Local Storage". Wouldn't it make more sense to call this std::thread_local or something?

reem · 2014-11-20T23:01:22Z

@alexcrichton Could &'thread T be emulated with:

struct ThreadRef<T> {
   marker: NoSend,
   data: &'static T
}

and the appropriate Deref impls?

alexcrichton · 2014-11-20T23:04:54Z

@reem I thought so! (and I was really hoping so). The catch is that the 'static > 'thread (note >, not >=). It should be the case that &'thread T does not satisfy 'static, but the struct you've shown sadly does. The details are in this comment above.

@alexcrichton

This PR completes the removal of the runtime system and green-threaded abstractions as part of implementing [RFC 230](rust-lang/rfcs#230). Specifically: * It removes the `Runtime` trait, welding the scheduling infrastructure directly to native threads. * It removes `libgreen` and `libnative` entirely. * It rewrites `sync::mutex` as a trivial layer on top of native mutexes. Eventually, the two modules will be merged. * It hides the vast majority of `std::rt`. This completes the basic task of removing the runtime system (I/O and scheduling) and components that depend on it. After this lands, a follow-up PR will pull the `rustrt` crate back into `std`, turn `std::task` into `std::thread` (with API changes to go along with it), and completely cut out the remaining startup/teardown sequence. Other changes, including new [TLS](rust-lang/rfcs#461) and synchronization are in the RFC or pre-RFC phase. Closes #17325 Closes #18687 [breaking-change] r? @alexcrichton

aturon · 2014-11-21T06:37:36Z

RFC accepted.
Discussion
Tracking

reem · 2014-11-21T13:19:33Z

@alexcrichton it looks like the simplest solution would just be to add a NoStatic marker - that seems like it may be generally useful to unsafe code.

This commit removes the `std::local_data` module in favor of a new `std::thread_local` module providing thread local storage. The module provides two variants of TLS: one which owns its contents and one which is based on scoped references. Each implementation has pros and cons listed in the documentation. Both flavors have accessors through a function called `with` which yield a reference to a closure provided. Both flavors also panic if a reference cannot be yielded and provide a function to test whether an access would panic or not. This is an implementation of [RFC 461][rfc] and full details can be found in that RFC. This is a breaking change due to the removal of the `std::local_data` module. All users can migrate to the new thread local system like so: thread_local!(static FOO: Rc<RefCell<Option<T>>> = Rc::new(RefCell::new(None))) The old `local_data` module inherently contained the `Rc<RefCell<Option<T>>>` as an implementation detail which must now be explicitly stated by users. [rfc]: rust-lang/rfcs#461 [breaking-change]

This commit removes the `std::local_data` module in favor of a new `std::thread_local` module providing thread local storage. The module provides two variants of TLS: one which owns its contents and one which is based on scoped references. Each implementation has pros and cons listed in the documentation. Both flavors have accessors through a function called `with` which yield a reference to a closure provided. Both flavors also panic if a reference cannot be yielded and provide a function to test whether an access would panic or not. This is an implementation of [RFC 461][rfc] and full details can be found in that RFC. This is a breaking change due to the removal of the `std::local_data` module. All users can migrate to the new tls system like so: thread_local!(static FOO: Rc<RefCell<Option<T>>> = Rc::new(RefCell::new(None))) The old `local_data` module inherently contained the `Rc<RefCell<Option<T>>>` as an implementation detail which must now be explicitly stated by users. [rfc]: rust-lang/rfcs#461 [breaking-change]

This commit removes the `std::local_data` module in favor of a new `std::thread_local` module providing thread local storage. The module provides two variants of TLS: one which owns its contents and one which is based on scoped references. Each implementation has pros and cons listed in the documentation. Both flavors have accessors through a function called `with` which yield a reference to a closure provided. Both flavors also panic if a reference cannot be yielded and provide a function to test whether an access would panic or not. This is an implementation of [RFC 461][rfc] and full details can be found in that RFC. This is a breaking change due to the removal of the `std::local_data` module. All users can migrate to the new thread local system like so: thread_local!(static FOO: Rc<RefCell<Option<T>>> = Rc::new(RefCell::new(None))) The old `local_data` module inherently contained the `Rc<RefCell<Option<T>>>` as an implementation detail which must now be explicitly stated by users. [rfc]: rust-lang/rfcs#461 [breaking-change]

This commit removes the `std::local_data` module in favor of a new `std::thread_local` module providing thread local storage. The module provides two variants of TLS: one which owns its contents and one which is based on scoped references. Each implementation has pros and cons listed in the documentation. Both flavors have accessors through a function called `with` which yield a reference to a closure provided. Both flavors also panic if a reference cannot be yielded and provide a function to test whether an access would panic or not. This is an implementation of [RFC 461][rfc] and full details can be found in that RFC. This is a breaking change due to the removal of the `std::local_data` module. All users can migrate to the new tls system like so: thread_local!(static FOO: Rc<RefCell<Option<T>>> = Rc::new(RefCell::new(None))) The old `local_data` module inherently contained the `Rc<RefCell<Option<T>>>` as an implementation detail which must now be explicitly stated by users. [rfc]: rust-lang/rfcs#461 [breaking-change]

This commit removes the `std::local_data` module in favor of a new `std::thread_local` module providing thread local storage. The module provides two variants of TLS: one which owns its contents and one which is based on scoped references. Each implementation has pros and cons listed in the documentation. Both flavors have accessors through a function called `with` which yield a reference to a closure provided. Both flavors also panic if a reference cannot be yielded and provide a function to test whether an access would panic or not. This is an implementation of [RFC 461][rfc] and full details can be found in that RFC. This is a breaking change due to the removal of the `std::local_data` module. All users can migrate to the new thread local system like so: thread_local!(static FOO: Rc<RefCell<Option<T>>> = Rc::new(RefCell::new(None))) The old `local_data` module inherently contained the `Rc<RefCell<Option<T>>>` as an implementation detail which must now be explicitly stated by users. [rfc]: rust-lang/rfcs#461 [breaking-change]

This commit removes the `std::local_data` module in favor of a new `std::thread_local` module providing thread local storage. The module provides two variants of TLS: one which owns its contents and one which is based on scoped references. Each implementation has pros and cons listed in the documentation. Both flavors have accessors through a function called `with` which yield a reference to a closure provided. Both flavors also panic if a reference cannot be yielded and provide a function to test whether an access would panic or not. This is an implementation of [RFC 461][rfc] and full details can be found in that RFC. This is a breaking change due to the removal of the `std::local_data` module. All users can migrate to the new tls system like so: thread_local!(static FOO: Rc<RefCell<Option<T>>> = Rc::new(RefCell::new(None))) The old `local_data` module inherently contained the `Rc<RefCell<Option<T>>>` as an implementation detail which must now be explicitly stated by users. [rfc]: rust-lang/rfcs#461 [breaking-change]

This commit removes the `std::local_data` module in favor of a new `std::thread_local` module providing thread local storage. The module provides two variants of TLS: one which owns its contents and one which is based on scoped references. Each implementation has pros and cons listed in the documentation. Both flavors have accessors through a function called `with` which yield a reference to a closure provided. Both flavors also panic if a reference cannot be yielded and provide a function to test whether an access would panic or not. This is an implementation of [RFC 461][rfc] and full details can be found in that RFC. This is a breaking change due to the removal of the `std::local_data` module. All users can migrate to the new thread local system like so: thread_local!(static FOO: Rc<RefCell<Option<T>>> = Rc::new(RefCell::new(None))) The old `local_data` module inherently contained the `Rc<RefCell<Option<T>>>` as an implementation detail which must now be explicitly stated by users. [rfc]: rust-lang/rfcs#461 [breaking-change]

bill-myers reviewed Nov 12, 2014
View reviewed changes

thestinger reviewed Nov 13, 2014
View reviewed changes

pnkfelix assigned alexcrichton Nov 13, 2014

aturon mentioned this pull request Nov 15, 2014

Finish runtime removal rust-lang/rust#18967

Merged

Remove an outdated drawback

f6ad78d

Tweak the API of owning TLS

4fc3b6d

alexcrichton mentioned this pull request Nov 18, 2014

std: Add a new top-level thread_local module rust-lang/rust#19094

Closed

aturon mentioned this pull request Nov 21, 2014

Overhaul thread local storage rust-lang/rust#19175

Closed

aturon merged commit 4fc3b6d into rust-lang:master Nov 21, 2014

This was referenced Nov 24, 2014

Release channels and feature staging #475

Closed

Stabilization metabug: 1.0-alpha rust-lang/rust#19260

Closed

Centril added the A-thread-local Proposals relating to thread local storage (TLS). label Nov 23, 2018

yoshuawuyts mentioned this pull request Jul 23, 2021

Scoped threads in the standard library, take 2 #3151

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

RFC: Add a thread local storage module, std::tls #461

RFC: Add a thread local storage module, std::tls #461

alexcrichton commented Nov 11, 2014

bill-myers Nov 12, 2014

alexcrichton Nov 12, 2014

aturon commented Nov 13, 2014

thestinger Nov 13, 2014

alexcrichton Nov 14, 2014

brson commented Nov 18, 2014

brson commented Nov 18, 2014

alexcrichton commented Nov 18, 2014

jnicklas commented Nov 20, 2014

reem commented Nov 20, 2014

alexcrichton commented Nov 20, 2014

aturon commented Nov 21, 2014

reem commented Nov 21, 2014


		### Destructor support

		The major difference between Unix and Windows TLS support is that Unix supports

RFC: Add a thread local storage module, std::tls #461

RFC: Add a thread local storage module, std::tls #461

Conversation

alexcrichton commented Nov 11, 2014

bill-myers Nov 12, 2014

Choose a reason for hiding this comment

alexcrichton Nov 12, 2014

Choose a reason for hiding this comment

aturon commented Nov 13, 2014

thestinger Nov 13, 2014

Choose a reason for hiding this comment

alexcrichton Nov 14, 2014

Choose a reason for hiding this comment

brson commented Nov 18, 2014

brson commented Nov 18, 2014

alexcrichton commented Nov 18, 2014

jnicklas commented Nov 20, 2014

reem commented Nov 20, 2014

alexcrichton commented Nov 20, 2014

aturon commented Nov 21, 2014

reem commented Nov 21, 2014