-
Notifications
You must be signed in to change notification settings - Fork 12.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Replace pthread RwLock
with custom implementation
#110211
Conversation
Hey! It looks like you've submitted a new PR for the library teams! If this PR contains changes to any Examples of
|
☔ The latest upstream changes (presumably #110324) made this pull request unmergeable. Please resolve the merge conflicts. |
I've found the bottleneck! Because the The implementation now uses a separate |
Few functions here with boolean args looks like have compile time known arguments values (i.e. true/false), so probably optimizer can see it and inline it's values. Maybe try to make this arguments const generics instead and see if this will work better? |
At least locally, the performance doesn't change at all (that makes sense, the condition in |
☔ The latest upstream changes (presumably #118996) made this pull request unmergeable. Please resolve the merge conflicts. |
☔ The latest upstream changes (presumably #117285) made this pull request unmergeable. Please resolve the merge conflicts. |
☔ The latest upstream changes (presumably #120496) made this pull request unmergeable. Please resolve the merge conflicts. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I finally had the time to review this. It looks great! Simpler and a lot more readable than I was expecting. ^^
Just a few small comments:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great to me!
Since this is a bunch of new unsafe+atomics code, I'd feel slightly more comfortable if another reviewer took a look at it too: @Amanieu, would you have time to also take a look at the soundness of this lock implementation?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
r=me with the small nit addressed
@bors r+ |
I recall for mutex we stuck to pthreads on some platforms because that gives the scheduler more information and it can apply priority boosting when a high-priority thread wants to take a lock from a low-priority thread (specifically, on macOS). Do similar concerns not apply here? |
I was just thinking about the same thing, but I think PI is not an issue here. For higher-priority readers, it can't practically be supported, as you'd have to memorize all current readers. For higher-priority writers, while it's possible to support PI in principle, it appears that no platform actually does so (macOS doesn't (I think), NetBSD doesn't, QNX doesn't). I can't find any at least, and the POSIX standard does not mandate or facilitate PI for rwlocks, on the contrary, it warns against priority inversions. |
☀️ Test successful - checks-actions |
Replace pthread `RwLock` with custom implementation This is one of the last items in #93740. I'm doing `RwLock` first because it is more self-contained and has less tradeoffs to make. The motivation is explained in the documentation, but in short: the pthread rwlock is slow and buggy and `std` can do much better. I considered implementing a parking lot, as was discussed in the tracking issue, but settled for the queue-based version because writing self-balancing binary trees is not fun in Rust... This is a rather complex change, so I have added quite a bit of documentation to help explain it. Please point out any part that could be explained better. ~~The read performance is really good, I'm getting 4x the throughput of the pthread version and about the same performance as usync/parking_lot on an Apple M1 Max in the usync benchmark suite, but the write performance still falls way behind what usync and parking_lot achieve. I tried using a separate queue lock like what usync uses, but that didn't help. I'll try to investigate further in the future, but I wanted to get some eyes on this first.~~ [Resolved](rust-lang/rust#110211 (comment)) r? `@m-ou-se` CC `@kprotty`
Finished benchmarking commit (b17491c): comparison URL. Overall result: ✅ improvements - no action needed@rustbot label: -perf-regression Instruction countThis is a highly reliable metric that was used to determine the overall result at the top of this comment.
Max RSS (memory usage)This benchmark run did not return any relevant results for this metric. CyclesThis benchmark run did not return any relevant results for this metric. Binary sizeThis benchmark run did not return any relevant results for this metric. Bootstrap: 664.937s -> 663.718s (-0.18%) |
So now that we have this, is it even worth still also having the Futex-based rwlock for Linux? |
Probably yes? I don't have a Linux machine to run benchmarks on, but my guess would be that there is little difference in performance between the implementations. The Linux version is a lot simpler, so I don't think there is any harm in keeping it. The true benefit of this version is that we can finally get rid of the unsound lock code on SGX and provide a fast and well-tested (heh!) lock for all the weird platforms out there (e.g. xous just uses spinning right now). |
Replace pthread `RwLock` with custom implementation This is one of the last items in #93740. I'm doing `RwLock` first because it is more self-contained and has less tradeoffs to make. The motivation is explained in the documentation, but in short: the pthread rwlock is slow and buggy and `std` can do much better. I considered implementing a parking lot, as was discussed in the tracking issue, but settled for the queue-based version because writing self-balancing binary trees is not fun in Rust... This is a rather complex change, so I have added quite a bit of documentation to help explain it. Please point out any part that could be explained better. ~~The read performance is really good, I'm getting 4x the throughput of the pthread version and about the same performance as usync/parking_lot on an Apple M1 Max in the usync benchmark suite, but the write performance still falls way behind what usync and parking_lot achieve. I tried using a separate queue lock like what usync uses, but that didn't help. I'll try to investigate further in the future, but I wanted to get some eyes on this first.~~ [Resolved](rust-lang/rust#110211 (comment)) r? `@m-ou-se` CC `@kprotty`
Use queue-based `RwLock` on more platforms This switches over Windows 7, SGX and Xous to the queue-based `RwLock` implementation added in rust-lang#110211, thereby fixing rust-lang#121949 for Windows 7 and partially resolving rust-lang#114581 on SGX. TEEOS can't currently be switched because it doesn't have a good thread parking implementation. CC `@roblabla` `@raoulstrackx` `@xobs` Could you help me test this, please? r? `@ChrisDenton` the Windows stuff should be familiar to you
Rollup merge of rust-lang#123811 - joboet:queue_em_up, r=ChrisDenton Use queue-based `RwLock` on more platforms This switches over Windows 7, SGX and Xous to the queue-based `RwLock` implementation added in rust-lang#110211, thereby fixing rust-lang#121949 for Windows 7 and partially resolving rust-lang#114581 on SGX. TEEOS can't currently be switched because it doesn't have a good thread parking implementation. CC `@roblabla` `@raoulstrackx` `@xobs` Could you help me test this, please? r? `@ChrisDenton` the Windows stuff should be familiar to you
Replace pthread `RwLock` with custom implementation This is one of the last items in #93740. I'm doing `RwLock` first because it is more self-contained and has less tradeoffs to make. The motivation is explained in the documentation, but in short: the pthread rwlock is slow and buggy and `std` can do much better. I considered implementing a parking lot, as was discussed in the tracking issue, but settled for the queue-based version because writing self-balancing binary trees is not fun in Rust... This is a rather complex change, so I have added quite a bit of documentation to help explain it. Please point out any part that could be explained better. ~~The read performance is really good, I'm getting 4x the throughput of the pthread version and about the same performance as usync/parking_lot on an Apple M1 Max in the usync benchmark suite, but the write performance still falls way behind what usync and parking_lot achieve. I tried using a separate queue lock like what usync uses, but that didn't help. I'll try to investigate further in the future, but I wanted to get some eyes on this first.~~ [Resolved](rust-lang/rust#110211 (comment)) r? `@m-ou-se` CC `@kprotty`
Pkgsrc changes: * Adapt checksums and patches, some have beene intregrated upstream. Upstream chnages: Version 1.78.0 (2024-05-02) =========================== Language -------- - [Stabilize `#[cfg(target_abi = ...)]`] (rust-lang/rust#119590) - [Stabilize the `#[diagnostic]` namespace and `#[diagnostic::on_unimplemented]` attribute] (rust-lang/rust#119888) - [Make async-fn-in-trait implementable with concrete signatures] (rust-lang/rust#120103) - [Make matching on NaN a hard error, and remove the rest of `illegal_floating_point_literal_pattern`] (rust-lang/rust#116284) - [static mut: allow mutable reference to arbitrary types, not just slices and arrays] (rust-lang/rust#117614) - [Extend `invalid_reference_casting` to include references casting to bigger memory layout] (rust-lang/rust#118983) - [Add `non_contiguous_range_endpoints` lint for singleton gaps after exclusive ranges] (rust-lang/rust#118879) - [Add `wasm_c_abi` lint for use of older wasm-bindgen versions] (rust-lang/rust#117918) This lint currently only works when using Cargo. - [Update `indirect_structural_match` and `pointer_structural_match` lints to match RFC] (rust-lang/rust#120423) - [Make non-`PartialEq`-typed consts as patterns a hard error] (rust-lang/rust#120805) - [Split `refining_impl_trait` lint into `_reachable`, `_internal` variants] (rust-lang/rust#121720) - [Remove unnecessary type inference when using associated types inside of higher ranked `where`-bounds] (rust-lang/rust#119849) - [Weaken eager detection of cyclic types during type inference] (rust-lang/rust#119989) - [`trait Trait: Auto {}`: allow upcasting from `dyn Trait` to `dyn Auto`] (rust-lang/rust#119338) Compiler -------- - [Made `INVALID_DOC_ATTRIBUTES` lint deny by default] (rust-lang/rust#111505) - [Increase accuracy of redundant `use` checking] (rust-lang/rust#117772) - [Suggest moving definition if non-found macro_rules! is defined later] (rust-lang/rust#121130) - [Lower transmutes from int to pointer type as gep on null] (rust-lang/rust#121282) Target changes: - [Windows tier 1 targets now require at least Windows 10] (rust-lang/rust#115141) - [Enable CMPXCHG16B, SSE3, SAHF/LAHF and 128-bit Atomics in tier 1 Windows] (rust-lang/rust#120820) - [Add `wasm32-wasip1` tier 2 (without host tools) target] (rust-lang/rust#120468) - [Add `wasm32-wasip2` tier 3 target] (rust-lang/rust#119616) - [Rename `wasm32-wasi-preview1-threads` to `wasm32-wasip1-threads`] (rust-lang/rust#122170) - [Add `arm64ec-pc-windows-msvc` tier 3 target] (rust-lang/rust#119199) - [Add `armv8r-none-eabihf` tier 3 target for the Cortex-R52] (rust-lang/rust#110482) - [Add `loongarch64-unknown-linux-musl` tier 3 target] (rust-lang/rust#121832) Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support. Libraries --------- - [Bump Unicode to version 15.1.0, regenerate tables] (rust-lang/rust#120777) - [Make align_offset, align_to well-behaved in all cases] (rust-lang/rust#121201) - [PartialEq, PartialOrd: document expectations for transitive chains] (rust-lang/rust#115386) - [Optimize away poison guards when std is built with panic=abort] (rust-lang/rust#100603) - [Replace pthread `RwLock` with custom implementation] (rust-lang/rust#110211) - [Implement unwind safety for Condvar on all platforms] (rust-lang/rust#121768) - [Add ASCII fast-path for `char::is_grapheme_extended`] (rust-lang/rust#121138) Stabilized APIs --------------- - [`impl Read for &Stdin`] (https://doc.rust-lang.org/stable/std/io/struct.Stdin.html#impl-Read-for-%26Stdin) - [Accept non `'static` lifetimes for several `std::error::Error` related implementations] (rust-lang/rust#113833) - [Make `impl<Fd: AsFd>` impl take `?Sized`] (rust-lang/rust#114655) - [`impl From<TryReserveError> for io::Error`] (https://doc.rust-lang.org/stable/std/io/struct.Error.html#impl-From%3CTryReserveError%3E-for-Error) These APIs are now stable in const contexts: - [`Barrier::new()`] (https://doc.rust-lang.org/stable/std/sync/struct.Barrier.html#method.new) Cargo ----- - [Stabilize lockfile v4](rust-lang/cargo#12852) - [Respect `rust-version` when generating lockfile] (rust-lang/cargo#12861) - [Control `--charset` via auto-detecting config value] (rust-lang/cargo#13337) - [Support `target.<triple>.rustdocflags` officially] (rust-lang/cargo#13197) - [Stabilize global cache data tracking] (rust-lang/cargo#13492) Misc ---- - [rustdoc: add `--test-builder-wrapper` arg to support wrappers such as RUSTC_WRAPPER when building doctests] (rust-lang/rust#114651) Compatibility Notes ------------------- - [Many unsafe precondition checks now run for user code with debug assertions enabled] (rust-lang/rust#120594) This change helps users catch undefined behavior in their code, though the details of how much is checked are generally not stable. - [riscv only supports split_debuginfo=off for now] (rust-lang/rust#120518) - [Consistently check bounds on hidden types of `impl Trait`] (rust-lang/rust#121679) - [Change equality of higher ranked types to not rely on subtyping] (rust-lang/rust#118247) - [When called, additionally check bounds on normalized function return type] (rust-lang/rust#118882) - [Expand coverage for `arithmetic_overflow` lint] (rust-lang/rust#119432) Internal Changes ---------------- These changes do not affect any public interfaces of Rust, but they represent significant improvements to the performance or internals of rustc and related tools. - [Update to LLVM 18](rust-lang/rust#120055) - [Build `rustc` with 1CGU on `x86_64-pc-windows-msvc`] (rust-lang/rust#112267) - [Build `rustc` with 1CGU on `x86_64-apple-darwin`] (rust-lang/rust#112268) - [Introduce `run-make` V2 infrastructure, a `run_make_support` library and port over 2 tests as example] (rust-lang/rust#113026) - [Windows: Implement condvar, mutex and rwlock using futex] (rust-lang/rust#121956)
This is based on the pkgsrc-wip rust180 package, retaining the main pkgsrc changes as best as I could. Pkgsrc changes: * Adapt checksums and patches. * Make this work again on big-endian aarch64 (at least on NetBSD). * Make the choice of GCC = 12 work for sparc64 by testing options after options.mk is included (which is required...). Makes this work on NetBSD/sparc64 10.0 again. Upstream chnages: Version 1.80.1 (2024-08-08) =========================== - [Fix miscompilation in the jump threading MIR optimization when comparing floats] (rust-lang/rust#128271) - [Revert changes to the `dead_code` lint from 1.80.0] (rust-lang/rust#128618) Version 1.80.0 (2024-07-25) ========================== Language -------- - [Document maximum allocation size] (rust-lang/rust#116675) - [Allow zero-byte offsets and ZST read/writes on arbitrary pointers] (rust-lang/rust#117329) - [Support C23's variadics without a named parameter] (rust-lang/rust#124048) - [Stabilize `exclusive_range_pattern` feature] (rust-lang/rust#124459) - [Guarantee layout and ABI of `Result` in some scenarios] (rust-lang/rust#124870) Compiler -------- - [Update cc crate to v1.0.97 allowing additional spectre mitigations on MSVC targets] (rust-lang/rust#124892) - [Allow field reordering on types marked `repr(packed(1))`] (rust-lang/rust#125360) - [Add a lint against never type fallback affecting unsafe code] (rust-lang/rust#123939) - [Disallow cast with trailing braced macro in let-else] (rust-lang/rust#125049) - [Expand `for_loops_over_fallibles` lint to lint on fallibles behind references.] (rust-lang/rust#125156) - [self-contained linker: retry linking without `-fuse-ld=lld` on CCs that don't support it] (rust-lang/rust#125417) - [Do not parse CVarArgs (`...`) as a type in trait bounds] (rust-lang/rust#125863) - Improvements to LLDB formatting [#124458] (rust-lang/rust#124458) [#124500] (rust-lang/rust#124500) - [For the wasm32-wasip2 target default to PIC and do not use `-fuse-ld=lld`] (rust-lang/rust#124858) - [Add x86_64-unknown-linux-none as a tier 3 target] (rust-lang/rust#125023) - [Lint on `foo.into_iter()` resolving to `&Box<[T]>: IntoIterator`] (rust-lang/rust#124097) Libraries --------- - [Add `size_of` and `size_of_val` and `align_of` and `align_of_val` to the prelude] (rust-lang/rust#123168) - [Abort a process when FD ownership is violated] (rust-lang/rust#124210) - [io::Write::write_fmt: panic if the formatter fails when the stream does not fail] (rust-lang/rust#125012) - [Panic if `PathBuf::set_extension` would add a path separator] (rust-lang/rust#125070) - [Add assert_unsafe_precondition to unchecked_{add,sub,neg,mul,shl,shr} methods] (rust-lang/rust#121571) - [Update `c_char` on AIX to use the correct type] (rust-lang/rust#122986) - [`offset_of!` no longer returns a temporary] (rust-lang/rust#124484) - [Handle sigma in `str.to_lowercase` correctly] (rust-lang/rust#124773) - [Raise `DEFAULT_MIN_STACK_SIZE` to at least 64KiB] (rust-lang/rust#126059) Stabilized APIs --------------- - [`impl Default for Rc<CStr>`] (https://doc.rust-lang.org/beta/alloc/rc/struct.Rc.html#impl-Default-for-Rc%3CCStr%3E) - [`impl Default for Rc<str>`] (https://doc.rust-lang.org/beta/alloc/rc/struct.Rc.html#impl-Default-for-Rc%3Cstr%3E) - [`impl Default for Rc<[T]>`] (https://doc.rust-lang.org/beta/alloc/rc/struct.Rc.html#impl-Default-for-Rc%3C%5BT%5D%3E) - [`impl Default for Arc<str>`] (https://doc.rust-lang.org/beta/alloc/sync/struct.Arc.html#impl-Default-for-Arc%3Cstr%3E) - [`impl Default for Arc<CStr>`] (https://doc.rust-lang.org/beta/alloc/sync/struct.Arc.html#impl-Default-for-Arc%3CCStr%3E) - [`impl Default for Arc<[T]>`] (https://doc.rust-lang.org/beta/alloc/sync/struct.Arc.html#impl-Default-for-Arc%3C%5BT%5D%3E) - [`impl IntoIterator for Box<[T]>`] (https://doc.rust-lang.org/beta/alloc/boxed/struct.Box.html#impl-IntoIterator-for-Box%3C%5BI%5D,+A%3E) - [`impl FromIterator<String> for Box<str>`] (https://doc.rust-lang.org/beta/alloc/boxed/struct.Box.html#impl-FromIterator%3CString%3E-for-Box%3Cstr%3E) - [`impl FromIterator<char> for Box<str>`] (https://doc.rust-lang.org/beta/alloc/boxed/struct.Box.html#impl-FromIterator%3Cchar%3E-for-Box%3Cstr%3E) - [`LazyCell`] (https://doc.rust-lang.org/beta/core/cell/struct.LazyCell.html) - [`LazyLock`] (https://doc.rust-lang.org/beta/std/sync/struct.LazyLock.html) - [`Duration::div_duration_f32`] (https://doc.rust-lang.org/beta/std/time/struct.Duration.html#method.div_duration_f32) - [`Duration::div_duration_f64`] (https://doc.rust-lang.org/beta/std/time/struct.Duration.html#method.div_duration_f64) - [`Option::take_if`] (https://doc.rust-lang.org/beta/std/option/enum.Option.html#method.take_if) - [`Seek::seek_relative`] (https://doc.rust-lang.org/beta/std/io/trait.Seek.html#method.seek_relative) - [`BinaryHeap::as_slice`] (https://doc.rust-lang.org/beta/std/collections/struct.BinaryHeap.html#method.as_slice) - [`NonNull::offset`] (https://doc.rust-lang.org/beta/std/ptr/struct.NonNull.html#method.offset) - [`NonNull::byte_offset`] (https://doc.rust-lang.org/beta/std/ptr/struct.NonNull.html#method.byte_offset) - [`NonNull::add`] (https://doc.rust-lang.org/beta/std/ptr/struct.NonNull.html#method.add) - [`NonNull::byte_add`] (https://doc.rust-lang.org/beta/std/ptr/struct.NonNull.html#method.byte_add) - [`NonNull::sub`] (https://doc.rust-lang.org/beta/std/ptr/struct.NonNull.html#method.sub) - [`NonNull::byte_sub`] (https://doc.rust-lang.org/beta/std/ptr/struct.NonNull.html#method.byte_sub) - [`NonNull::offset_from`] (https://doc.rust-lang.org/beta/std/ptr/struct.NonNull.html#method.offset_from) - [`NonNull::byte_offset_from`] (https://doc.rust-lang.org/beta/std/ptr/struct.NonNull.html#method.byte_offset_from) - [`NonNull::read`] (https://doc.rust-lang.org/beta/std/ptr/struct.NonNull.html#method.read) - [`NonNull::read_volatile`] (https://doc.rust-lang.org/beta/std/ptr/struct.NonNull.html#method.read_volatile) - [`NonNull::read_unaligned`] (https://doc.rust-lang.org/beta/std/ptr/struct.NonNull.html#method.read_unaligned) - [`NonNull::write`] (https://doc.rust-lang.org/beta/std/ptr/struct.NonNull.html#method.write) - [`NonNull::write_volatile`] (https://doc.rust-lang.org/beta/std/ptr/struct.NonNull.html#method.write_volatile) - [`NonNull::write_unaligned`] (https://doc.rust-lang.org/beta/std/ptr/struct.NonNull.html#method.write_unaligned) - [`NonNull::write_bytes`] (https://doc.rust-lang.org/beta/std/ptr/struct.NonNull.html#method.write_bytes) - [`NonNull::copy_to`] (https://doc.rust-lang.org/beta/std/ptr/struct.NonNull.html#method.copy_to) - [`NonNull::copy_to_nonoverlapping`] (https://doc.rust-lang.org/beta/std/ptr/struct.NonNull.html#method.copy_to_nonoverlapping) - [`NonNull::copy_from`] (https://doc.rust-lang.org/beta/std/ptr/struct.NonNull.html#method.copy_from) - [`NonNull::copy_from_nonoverlapping`] (https://doc.rust-lang.org/beta/std/ptr/struct.NonNull.html#method.copy_from_nonoverlapping) - [`NonNull::replace`] (https://doc.rust-lang.org/beta/std/ptr/struct.NonNull.html#method.replace) - [`NonNull::swap`] (https://doc.rust-lang.org/beta/std/ptr/struct.NonNull.html#method.swap) - [`NonNull::drop_in_place`] (https://doc.rust-lang.org/beta/std/ptr/struct.NonNull.html#method.drop_in_place) - [`NonNull::align_offset`] (https://doc.rust-lang.org/beta/std/ptr/struct.NonNull.html#method.align_offset) - [`<[T]>::split_at_checked`] (https://doc.rust-lang.org/beta/std/primitive.slice.html#method.split_at_checked) - [`<[T]>::split_at_mut_checked`] (https://doc.rust-lang.org/beta/std/primitive.slice.html#method.split_at_mut_checked) - [`str::split_at_checked`] (https://doc.rust-lang.org/beta/std/primitive.str.html#method.split_at_checked) - [`str::split_at_mut_checked`] (https://doc.rust-lang.org/beta/std/primitive.str.html#method.split_at_mut_checked) - [`str::trim_ascii`] (https://doc.rust-lang.org/beta/std/primitive.str.html#method.trim_ascii) - [`str::trim_ascii_start`] (https://doc.rust-lang.org/beta/std/primitive.str.html#method.trim_ascii_start) - [`str::trim_ascii_end`] (https://doc.rust-lang.org/beta/std/primitive.str.html#method.trim_ascii_end) - [`<[u8]>::trim_ascii`] (https://doc.rust-lang.org/beta/core/primitive.slice.html#method.trim_ascii) - [`<[u8]>::trim_ascii_start`] (https://doc.rust-lang.org/beta/core/primitive.slice.html#method.trim_ascii_start) - [`<[u8]>::trim_ascii_end`] (https://doc.rust-lang.org/beta/core/primitive.slice.html#method.trim_ascii_end) - [`Ipv4Addr::BITS`] (https://doc.rust-lang.org/beta/core/net/struct.Ipv4Addr.html#associatedconstant.BITS) - [`Ipv4Addr::to_bits`] (https://doc.rust-lang.org/beta/core/net/struct.Ipv4Addr.html#method.to_bits) - [`Ipv4Addr::from_bits`] (https://doc.rust-lang.org/beta/core/net/struct.Ipv4Addr.html#method.from_bits) - [`Ipv6Addr::BITS`] (https://doc.rust-lang.org/beta/core/net/struct.Ipv6Addr.html#associatedconstant.BITS) - [`Ipv6Addr::to_bits`] (https://doc.rust-lang.org/beta/core/net/struct.Ipv6Addr.html#method.to_bits) - [`Ipv6Addr::from_bits`] (https://doc.rust-lang.org/beta/core/net/struct.Ipv6Addr.html#method.from_bits) - [`Vec::<[T; N]>::into_flattened`] (https://doc.rust-lang.org/beta/alloc/vec/struct.Vec.html#method.into_flattened) - [`<[[T; N]]>::as_flattened`] (https://doc.rust-lang.org/beta/core/primitive.slice.html#method.as_flattened) - [`<[[T; N]]>::as_flattened_mut`] (https://doc.rust-lang.org/beta/core/primitive.slice.html#method.as_flattened_mut) These APIs are now stable in const contexts: - [`<[T]>::last_chunk`] (https://doc.rust-lang.org/beta/core/primitive.slice.html#method.last_chunk) - [`BinaryHeap::new`] (https://doc.rust-lang.org/beta/std/collections/struct.BinaryHeap.html#method.new) Cargo ----- - [Stabilize `-Zcheck-cfg` as always enabled] (rust-lang/cargo#13571) - [Warn, rather than fail publish, if a target is excluded] (rust-lang/cargo#13713) - [Add special `check-cfg` lint config for the `unexpected_cfgs` lint] (rust-lang/cargo#13913) - [Stabilize `cargo update --precise <yanked>`] (rust-lang/cargo#13974) - [Don't change file permissions on `Cargo.toml` when using `cargo add`] (rust-lang/cargo#13898) - [Support using `cargo fix` on IPv6-only networks] (rust-lang/cargo#13907) Rustdoc ----- - [Allow searching for references] (rust-lang/rust#124148) - [Stabilize `custom_code_classes_in_docs` feature] (rust-lang/rust#124577) - [fix: In cross-crate scenarios show enum variants on type aliases of enums] (rust-lang/rust#125300) Compatibility Notes ------------------- - [rustfmt estimates line lengths differently when using non-ascii characters] (rust-lang/rustfmt#6203) - [Type aliases are now handled correctly in orphan check] (rust-lang/rust#117164) - [Allow instructing rustdoc to read from stdin via `-`] (rust-lang/rust#124611) - [`std::env::{set_var, remove_var}` can no longer be converted to safe function pointers and no longer implement the `Fn` family of traits] (rust-lang/rust#124636) - [Warn (or error) when `Self` constructor from outer item is referenced in inner nested item] (rust-lang/rust#124187) - [Turn `indirect_structural_match` and `pointer_structural_match` lints into hard errors] (rust-lang/rust#124661) - [Make `where_clause_object_safety` lint a regular object safety violation] (rust-lang/rust#125380) - [Turn `proc_macro_back_compat` lint into a hard error.] (rust-lang/rust#125596) - [Detect unused structs even when implementing private traits] (rust-lang/rust#122382) - [`std::sync::ReentrantLockGuard<T>` is no longer `Sync` if `T: !Sync`] (rust-lang/rust#125527) which means [`std::io::StdoutLock` and `std::io::StderrLock` are no longer Sync] (rust-lang/rust#127340) Internal Changes ---------------- These changes do not affect any public interfaces of Rust, but they represent significant improvements to the performance or internals of rustc and related tools. - Misc improvements to size of generated html by rustdoc e.g. [#124738] (rust-lang/rust#124738) and [#123734] (rust-lang/rust#123734) - [MSVC targets no longer depend on libc] (rust-lang/rust#124050) Version 1.79.0 (2024-06-13) ========================== Language -------- - [Stabilize inline `const {}` expressions.] (rust-lang/rust#104087) - [Prevent opaque types being instantiated twice with different regions within the same function.] (rust-lang/rust#116935) - [Stabilize WebAssembly target features that are in phase 4 and 5.] (rust-lang/rust#117457) - [Add the `redundant_lifetimes` lint to detect lifetimes which are semantically redundant.] (rust-lang/rust#118391) - [Stabilize the `unnameable_types` lint for public types that can't be named.] (rust-lang/rust#120144) - [Enable debuginfo in macros, and stabilize `-C collapse-macro-debuginfo` and `#[collapse_debuginfo]`.] (rust-lang/rust#120845) - [Propagate temporary lifetime extension into `if` and `match` expressions.] (rust-lang/rust#121346) - [Restrict promotion of `const fn` calls.] (rust-lang/rust#121557) - [Warn against refining impls of crate-private traits with `refining_impl_trait` lint.] (rust-lang/rust#121720) - [Stabilize associated type bounds (RFC 2289).] (rust-lang/rust#122055) - [Stabilize importing `main` from other modules or crates.] (rust-lang/rust#122060) - [Check return types of function types for well-formedness] (rust-lang/rust#115538) - [Rework `impl Trait` lifetime inference] (rust-lang/rust#116891) - [Change inductive trait solver cycles to be ambiguous] (rust-lang/rust#122791) Compiler -------- - [Define `-C strip` to only affect binaries, not artifacts like `.pdb`.] (rust-lang/rust#115120) - [Stabilize `-Crelro-level` for controlling runtime link hardening.] (rust-lang/rust#121694) - [Stabilize checking of `cfg` names and values at compile-time with `--check-cfg`.] (rust-lang/rust#123501) *Note that this only stabilizes the compiler part, the Cargo part is still unstable in this release.* - [Add `aarch64-apple-visionos` and `aarch64-apple-visionos-sim` tier 3 targets.] (rust-lang/rust#121419) - [Add `riscv32ima-unknown-none-elf` tier 3 target.] (rust-lang/rust#122696) - [Promote several Windows targets to tier 2] (rust-lang/rust#121712): `aarch64-pc-windows-gnullvm`, `i686-pc-windows-gnullvm`, and `x86_64-pc-windows-gnullvm`. Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support. Libraries --------- - [Implement `FromIterator` for `(impl Default + Extend, impl Default + Extend)`.] (rust-lang/rust#107462) - [Implement `{Div,Rem}Assign<NonZero<X>>` on `X`.] (rust-lang/rust#121952) - [Document overrides of `clone_from()` in core/std.] (rust-lang/rust#122201) - [Link MSVC default lib in core.] (rust-lang/rust#122268) - [Caution against using `transmute` between pointers and integers.] (rust-lang/rust#122379) - [Enable frame pointers for the standard library.] (rust-lang/rust#122646) Stabilized APIs --------------- - [`{integer}::unchecked_add`] (https://doc.rust-lang.org/stable/core/primitive.i32.html#method.unchecked_add) - [`{integer}::unchecked_mul`] (https://doc.rust-lang.org/stable/core/primitive.i32.html#method.unchecked_mul) - [`{integer}::unchecked_sub`] (https://doc.rust-lang.org/stable/core/primitive.i32.html#method.unchecked_sub) - [`<[T]>::split_at_unchecked`] (https://doc.rust-lang.org/stable/core/primitive.slice.html#method.split_at_unchecked) - [`<[T]>::split_at_mut_unchecked`] (https://doc.rust-lang.org/stable/core/primitive.slice.html#method.split_at_mut_unchecked) - [`<[u8]>::utf8_chunks`] (https://doc.rust-lang.org/stable/core/primitive.slice.html#method.utf8_chunks) - [`str::Utf8Chunks`] (https://doc.rust-lang.org/stable/core/str/struct.Utf8Chunks.html) - [`str::Utf8Chunk`] (https://doc.rust-lang.org/stable/core/str/struct.Utf8Chunk.html) - [`<*const T>::is_aligned`] (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.is_aligned) - [`<*mut T>::is_aligned`] (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.is_aligned-1) - [`NonNull::is_aligned`] (https://doc.rust-lang.org/stable/core/ptr/struct.NonNull.html#method.is_aligned) - [`<*const [T]>::len`] (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.len) - [`<*mut [T]>::len`] (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.len-1) - [`<*const [T]>::is_empty`] (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.is_empty) - [`<*mut [T]>::is_empty`] (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.is_empty-1) - [`NonNull::<[T]>::is_empty`] (https://doc.rust-lang.org/stable/core/ptr/struct.NonNull.html#method.is_empty) - [`CStr::count_bytes`] (https://doc.rust-lang.org/stable/core/ffi/c_str/struct.CStr.html#method.count_bytes) - [`io::Error::downcast`] (https://doc.rust-lang.org/stable/std/io/struct.Error.html#method.downcast) - [`num::NonZero<T>`] (https://doc.rust-lang.org/stable/core/num/struct.NonZero.html) - [`path::absolute`] (https://doc.rust-lang.org/stable/std/path/fn.absolute.html) - [`proc_macro::Literal::byte_character`] (https://doc.rust-lang.org/stable/proc_macro/struct.Literal.html#method.byte_character) - [`proc_macro::Literal::c_string`] (https://doc.rust-lang.org/stable/proc_macro/struct.Literal.html#method.c_string) These APIs are now stable in const contexts: - [`Atomic*::into_inner`] (https://doc.rust-lang.org/stable/core/sync/atomic/struct.AtomicUsize.html#method.into_inner) - [`io::Cursor::new`] (https://doc.rust-lang.org/stable/std/io/struct.Cursor.html#method.new) - [`io::Cursor::get_ref`] (https://doc.rust-lang.org/stable/std/io/struct.Cursor.html#method.get_ref) - [`io::Cursor::position`] (https://doc.rust-lang.org/stable/std/io/struct.Cursor.html#method.position) - [`io::empty`] (https://doc.rust-lang.org/stable/std/io/fn.empty.html) - [`io::repeat`] (https://doc.rust-lang.org/stable/std/io/fn.repeat.html) - [`io::sink`] (https://doc.rust-lang.org/stable/std/io/fn.sink.html) - [`panic::Location::caller`] (https://doc.rust-lang.org/stable/std/panic/struct.Location.html#method.caller) - [`panic::Location::file`] (https://doc.rust-lang.org/stable/std/panic/struct.Location.html#method.file) - [`panic::Location::line`] (https://doc.rust-lang.org/stable/std/panic/struct.Location.html#method.line) - [`panic::Location::column`] (https://doc.rust-lang.org/stable/std/panic/struct.Location.html#method.column) Cargo ----- - [Prevent dashes in `lib.name`, always normalizing to `_`.] (rust-lang/cargo#12783) - [Stabilize MSRV-aware version requirement selection in `cargo add`.] (rust-lang/cargo#13608) - [Switch to using `gitoxide` by default for listing files.] (rust-lang/cargo#13696) Rustdoc ----- - [Always display stability version even if it's the same as the containing item.] (rust-lang/rust#118441) - [Show a single search result for items with multiple paths.] (rust-lang/rust#119912) - [Support typing `/` in docs to begin a search.] (rust-lang/rust#123355) Misc ---- Compatibility Notes ------------------- - [Update the minimum external LLVM to 17.] (rust-lang/rust#122649) - [`RustcEncodable` and `RustcDecodable` are soft-destabilized, to be removed from the prelude in next edition.] (rust-lang/rust#116016) - [The `wasm_c_abi` future-incompatibility lint will warn about use of the non-spec-compliant C ABI.] (rust-lang/rust#117918) Use `wasm-bindgen v0.2.88` to generate forward-compatible bindings. - [Check return types of function types for well-formedness] (rust-lang/rust#115538) Version 1.78.0 (2024-05-02) =========================== Language -------- - [Stabilize `#[cfg(target_abi = ...)]`] (rust-lang/rust#119590) - [Stabilize the `#[diagnostic]` namespace and `#[diagnostic::on_unimplemented]` attribute] (rust-lang/rust#119888) - [Make async-fn-in-trait implementable with concrete signatures] (rust-lang/rust#120103) - [Make matching on NaN a hard error, and remove the rest of `illegal_floating_point_literal_pattern`] (rust-lang/rust#116284) - [static mut: allow mutable reference to arbitrary types, not just slices and arrays] (rust-lang/rust#117614) - [Extend `invalid_reference_casting` to include references casting to bigger memory layout] (rust-lang/rust#118983) - [Add `non_contiguous_range_endpoints` lint for singleton gaps after exclusive ranges] (rust-lang/rust#118879) - [Add `wasm_c_abi` lint for use of older wasm-bindgen versions] (rust-lang/rust#117918) This lint currently only works when using Cargo. - [Update `indirect_structural_match` and `pointer_structural_match` lints to match RFC] (rust-lang/rust#120423) - [Make non-`PartialEq`-typed consts as patterns a hard error] (rust-lang/rust#120805) - [Split `refining_impl_trait` lint into `_reachable`, `_internal` variants] (rust-lang/rust#121720) - [Remove unnecessary type inference when using associated types inside of higher ranked `where`-bounds] (rust-lang/rust#119849) - [Weaken eager detection of cyclic types during type inference] (rust-lang/rust#119989) - [`trait Trait: Auto {}`: allow upcasting from `dyn Trait` to `dyn Auto`] (rust-lang/rust#119338) Compiler -------- - [Made `INVALID_DOC_ATTRIBUTES` lint deny by default] (rust-lang/rust#111505) - [Increase accuracy of redundant `use` checking] (rust-lang/rust#117772) - [Suggest moving definition if non-found macro_rules! is defined later] (rust-lang/rust#121130) - [Lower transmutes from int to pointer type as gep on null] (rust-lang/rust#121282) Target changes: - [Windows tier 1 targets now require at least Windows 10] (rust-lang/rust#115141) - [Enable CMPXCHG16B, SSE3, SAHF/LAHF and 128-bit Atomics in tier 1 Windows] (rust-lang/rust#120820) - [Add `wasm32-wasip1` tier 2 (without host tools) target] (rust-lang/rust#120468) - [Add `wasm32-wasip2` tier 3 target] (rust-lang/rust#119616) - [Rename `wasm32-wasi-preview1-threads` to `wasm32-wasip1-threads`] (rust-lang/rust#122170) - [Add `arm64ec-pc-windows-msvc` tier 3 target] (rust-lang/rust#119199) - [Add `armv8r-none-eabihf` tier 3 target for the Cortex-R52] (rust-lang/rust#110482) - [Add `loongarch64-unknown-linux-musl` tier 3 target] (rust-lang/rust#121832) Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support. Libraries --------- - [Bump Unicode to version 15.1.0, regenerate tables] (rust-lang/rust#120777) - [Make align_offset, align_to well-behaved in all cases] (rust-lang/rust#121201) - [PartialEq, PartialOrd: document expectations for transitive chains] (rust-lang/rust#115386) - [Optimize away poison guards when std is built with panic=abort] (rust-lang/rust#100603) - [Replace pthread `RwLock` with custom implementation] (rust-lang/rust#110211) - [Implement unwind safety for Condvar on all platforms] (rust-lang/rust#121768) - [Add ASCII fast-path for `char::is_grapheme_extended`] (rust-lang/rust#121138) Stabilized APIs --------------- - [`impl Read for &Stdin`] (https://doc.rust-lang.org/stable/std/io/struct.Stdin.html#impl-Read-for-%26Stdin) - [Accept non `'static` lifetimes for several `std::error::Error` related implementations] (rust-lang/rust#113833) - [Make `impl<Fd: AsFd>` impl take `?Sized`] (rust-lang/rust#114655) - [`impl From<TryReserveError> for io::Error`] (https://doc.rust-lang.org/stable/std/io/struct.Error.html#impl-From%3CTryReserveError%3E-for-Error) These APIs are now stable in const contexts: - [`Barrier::new()`] (https://doc.rust-lang.org/stable/std/sync/struct.Barrier.html#method.new) Cargo ----- - [Stabilize lockfile v4](rust-lang/cargo#12852) - [Respect `rust-version` when generating lockfile] (rust-lang/cargo#12861) - [Control `--charset` via auto-detecting config value] (rust-lang/cargo#13337) - [Support `target.<triple>.rustdocflags` officially] (rust-lang/cargo#13197) - [Stabilize global cache data tracking] (rust-lang/cargo#13492) Misc ---- - [rustdoc: add `--test-builder-wrapper` arg to support wrappers such as RUSTC_WRAPPER when building doctests] (rust-lang/rust#114651) Compatibility Notes ------------------- - [Many unsafe precondition checks now run for user code with debug assertions enabled] (rust-lang/rust#120594) This change helps users catch undefined behavior in their code, though the details of how much is checked are generally not stable. - [riscv only supports split_debuginfo=off for now] (rust-lang/rust#120518) - [Consistently check bounds on hidden types of `impl Trait`] (rust-lang/rust#121679) - [Change equality of higher ranked types to not rely on subtyping] (rust-lang/rust#118247) - [When called, additionally check bounds on normalized function return type] (rust-lang/rust#118882) - [Expand coverage for `arithmetic_overflow` lint] (rust-lang/rust#119432) Internal Changes ---------------- These changes do not affect any public interfaces of Rust, but they represent significant improvements to the performance or internals of rustc and related tools. - [Update to LLVM 18](rust-lang/rust#120055) - [Build `rustc` with 1CGU on `x86_64-pc-windows-msvc`] (rust-lang/rust#112267) - [Build `rustc` with 1CGU on `x86_64-apple-darwin`] (rust-lang/rust#112268) - [Introduce `run-make` V2 infrastructure, a `run_make_support` library and port over 2 tests as example] (rust-lang/rust#113026) - [Windows: Implement condvar, mutex and rwlock using futex] (rust-lang/rust#121956) Version 1.77.0 (2024-03-21) ========================== - [Reveal opaque types within the defining body for exhaustiveness checking.] (rust-lang/rust#116821) - [Stabilize C-string literals.] (rust-lang/rust#117472) - [Stabilize THIR unsafeck.] (rust-lang/rust#117673) - [Add lint `static_mut_refs` to warn on references to mutable statics.] (rust-lang/rust#117556) - [Support async recursive calls (as long as they have indirection).] (rust-lang/rust#117703) - [Undeprecate lint `unstable_features` and make use of it in the compiler.] (rust-lang/rust#118639) - [Make inductive cycles in coherence ambiguous always.] (rust-lang/rust#118649) - [Get rid of type-driven traversal in const-eval interning] (rust-lang/rust#119044), only as a [future compatiblity lint] (rust-lang/rust#122204) for now. - [Deny braced macro invocations in let-else.] (rust-lang/rust#119062) Compiler -------- - [Include lint `soft_unstable` in future breakage reports.] (rust-lang/rust#116274) - [Make `i128` and `u128` 16-byte aligned on x86-based targets.] (rust-lang/rust#116672) - [Use `--verbose` in diagnostic output.] (rust-lang/rust#119129) - [Improve spacing between printed tokens.] (rust-lang/rust#120227) - [Merge the `unused_tuple_struct_fields` lint into `dead_code`.] (rust-lang/rust#118297) - [Error on incorrect implied bounds in well-formedness check] (rust-lang/rust#118553), with a temporary exception for Bevy. - [Fix coverage instrumentation/reports for non-ASCII source code.] (rust-lang/rust#119033) - [Fix `fn`/`const` items implied bounds and well-formedness check.] (rust-lang/rust#120019) - [Promote `riscv32{im|imafc}-unknown-none-elf` targets to tier 2.] (rust-lang/rust#118704) - Add several new tier 3 targets: - [`aarch64-unknown-illumos`] (rust-lang/rust#112936) - [`hexagon-unknown-none-elf`] (rust-lang/rust#117601) - [`riscv32imafc-esp-espidf`] (rust-lang/rust#119738) - [`riscv32im-risc0-zkvm-elf`] (rust-lang/rust#117958) Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support. Libraries --------- - [Implement `From<&[T; N]>` for `Cow<[T]>`.] (rust-lang/rust#113489) - [Remove special-case handling of `vec.split_off (0)`.](rust-lang/rust#119917) Stabilized APIs --------------- - [`array::each_ref`] (https://doc.rust-lang.org/stable/std/primitive.array.html#method.each_ref) - [`array::each_mut`] (https://doc.rust-lang.org/stable/std/primitive.array.html#method.each_mut) - [`core::net`] (https://doc.rust-lang.org/stable/core/net/index.html) - [`f32::round_ties_even`] (https://doc.rust-lang.org/stable/std/primitive.f32.html#method.round_ties_even) - [`f64::round_ties_even`] (https://doc.rust-lang.org/stable/std/primitive.f64.html#method.round_ties_even) - [`mem::offset_of!`] (https://doc.rust-lang.org/stable/std/mem/macro.offset_of.html) - [`slice::first_chunk`] (https://doc.rust-lang.org/stable/std/primitive.slice.html#method.first_chunk) - [`slice::first_chunk_mut`] (https://doc.rust-lang.org/stable/std/primitive.slice.html#method.first_chunk_mut) - [`slice::split_first_chunk`] (https://doc.rust-lang.org/stable/std/primitive.slice.html#method.split_first_chunk) - [`slice::split_first_chunk_mut`] (https://doc.rust-lang.org/stable/std/primitive.slice.html#method.split_first_chunk_mut) - [`slice::last_chunk`] (https://doc.rust-lang.org/stable/std/primitive.slice.html#method.last_chunk) - [`slice::last_chunk_mut`] (https://doc.rust-lang.org/stable/std/primitive.slice.html#method.last_chunk_mut) - [`slice::split_last_chunk`] (https://doc.rust-lang.org/stable/std/primitive.slice.html#method.split_last_chunk) - [`slice::split_last_chunk_mut`] (https://doc.rust-lang.org/stable/std/primitive.slice.html#method.split_last_chunk_mut) - [`slice::chunk_by`] (https://doc.rust-lang.org/stable/std/primitive.slice.html#method.chunk_by) - [`slice::chunk_by_mut`] (https://doc.rust-lang.org/stable/std/primitive.slice.html#method.chunk_by_mut) - [`Bound::map`] (https://doc.rust-lang.org/stable/std/ops/enum.Bound.html#method.map) - [`File::create_new`] (https://doc.rust-lang.org/stable/std/fs/struct.File.html#method.create_new) - [`Mutex::clear_poison`] (https://doc.rust-lang.org/stable/std/sync/struct.Mutex.html#method.clear_poison) - [`RwLock::clear_poison`] (https://doc.rust-lang.org/stable/std/sync/struct.RwLock.html#method.clear_poison) Cargo ----- - [Extend the build directive syntax with `cargo::`.] (rust-lang/cargo#12201) - [Stabilize metadata `id` format as `PackageIDSpec`.] (rust-lang/cargo#12914) - [Pull out as `cargo-util-schemas` as a crate.] (rust-lang/cargo#13178) - [Strip all debuginfo when debuginfo is not requested.] (rust-lang/cargo#13257) - [Inherit jobserver from env for all kinds of runners.] (rust-lang/cargo#12776) - [Deprecate rustc plugin support in cargo.] (rust-lang/cargo#13248) Rustdoc ----- - [Allows links in markdown headings.] (rust-lang/rust#117662) - [Search for tuples and unit by type with `()`.] (rust-lang/rust#118194) - [Clean up the source sidebar's hide button.] (rust-lang/rust#119066) - [Prevent JS injection from `localStorage`.] (rust-lang/rust#120250) Misc ---- - [Recommend version-sorting for all sorting in style guide.] (rust-lang/rust#115046) Internal Changes ---------------- These changes do not affect any public interfaces of Rust, but they represent significant improvements to the performance or internals of rustc and related tools. - [Add more weirdness to `weird-exprs.rs`.] (rust-lang/rust#119028)
This is one of the last items in #93740. I'm doing
RwLock
first because it is more self-contained and has less tradeoffs to make. The motivation is explained in the documentation, but in short: the pthread rwlock is slow and buggy andstd
can do much better. I considered implementing a parking lot, as was discussed in the tracking issue, but settled for the queue-based version because writing self-balancing binary trees is not fun in Rust...This is a rather complex change, so I have added quite a bit of documentation to help explain it. Please point out any part that could be explained better.
The read performance is really good, I'm getting 4x the throughput of the pthread version and about the same performance as usync/parking_lot on an Apple M1 Max in the usync benchmark suite, but the write performance still falls way behind what usync and parking_lot achieve. I tried using a separate queue lock like what usync uses, but that didn't help. I'll try to investigate further in the future, but I wanted to get some eyes on this first.Resolvedr? @m-ou-se
CC @kprotty