Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace RwLock by a futex based one on Linux #95801

Merged
merged 7 commits into from
Apr 12, 2022

Conversation

m-ou-se
Copy link
Member

@m-ou-se m-ou-se commented Apr 8, 2022

This replaces the pthread-based RwLock on Linux by a futex based one.

This implementation is similar to the algorithm suggested by @kprotty, but modified to prefer writers and spin before sleeping. It uses two futexes: One for the readers to wait on, and one for the writers to wait on. The readers futex contains the state of the RwLock: The number of readers, a bit indicating whether writers are waiting, and a bit indicating whether readers are waiting. The writers futex is used as a simple condition variable and its contents are meaningless; it just needs to be changed on every notification.

Using two futexes rather than one has the obvious advantage of allowing a separate queue for readers and writers, but it also means we avoid the problem a single-futex RwLock would have of making it hard for a writer to go to sleep while the number of readers is rapidly changing up and down, as the writers futex is only changed when we actually want to wake up a writer.

It always prefers writers, as we decided here.

To be able to prefer writers, it relies on futex_wake to return the number of awoken threads to be able to handle write-unlocking while both the readers-waiting and writers-waiting bits are set. Instead of waking both and letting them race, it first wakes writers and only continues to wake the readers too if futex_wake reported there were no writers to wake up.

r? @Amanieu

@m-ou-se m-ou-se added O-linux Operating system: Linux A-concurrency Area: Concurrency T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Apr 8, 2022
@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Apr 8, 2022
@m-ou-se
Copy link
Member Author

m-ou-se commented Apr 8, 2022

I'm running a stress test for this new RwLock with 200 threads randomly read locking (99.5% chance) or write locking (0.5% chance), while tracking that every thread keeps making progress. Another computer is running the same stress test with a much lower number of threads that also randomly sleep sometimes. Both have been running for 40 minutes now without any problems.

@m-ou-se
Copy link
Member Author

m-ou-se commented Apr 8, 2022

stress test

Interestingly, the current (pthread) RwLock fails the same test, due to writer starvation.

library/std/src/sys/unix/locks/futex_rwlock.rs Outdated Show resolved Hide resolved
library/std/src/sys/unix/locks/futex_rwlock.rs Outdated Show resolved Hide resolved
const WRITE_LOCKED: i32 = MASK;
const MAX_READERS: i32 = MASK - 1;
const READERS_WAITING: i32 = 1 << 30;
const WRITERS_WAITING: i32 = 1 << 31;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any reason why these are placed in the high bits rather than the low bits? In parking_lot I put the flags in the low bits so that counter overflows can be caught with checked_add which is slightly more efficient assembly code.

Copy link
Member Author

@m-ou-se m-ou-se Apr 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Putting them in the high bits makes it possible for the read condition to compile down to a single comparison:

readers(state) < MAX_READERS && !readers_waiting(state) && !writers_waiting(state)

is then equal to

(state as u32) < 0x3ffffffe.

(Not sure if llvm sees that though.)

Copy link
Contributor

@erikdesjardins erikdesjardins Apr 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LLVM doesn't currently see this, although it sees a related pattern where the 0x3ffffffe mask has only one set bit.

Reported upstream: llvm/llvm-project#54856

(Note that I looked at readers(state) < MAX_READERS & !readers_waiting(state) & !writers_waiting(state) with bitwise and--LLVM has more trouble with logical and.)

library/std/src/sys/unix/locks/futex_rwlock.rs Outdated Show resolved Hide resolved
library/std/src/sys/unix/locks/futex_rwlock.rs Outdated Show resolved Hide resolved

const READ_LOCKED: i32 = 1;
const MASK: i32 = (1 << 30) - 1;
const WRITE_LOCKED: i32 = MASK;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems a bit strange to represent WRITE_LOCKED with a special reader count. I think using a separate bit for WRITE_LOCKED would make the code more efficient (e.g. using native overflow flags to detect reader count overflow, using fetch_or for locking instead of a CAS loop, etc).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll try that out and see if it simplifies things. It probably does. Thanks. :)

Copy link
Member Author

@m-ou-se m-ou-se Apr 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried it, but it does not seem to simplify things.

fetch_or doesn't seem very useful, because it should only set the bit if the lock isn't read-locked yet. If we unconditionally set the bit when attempting to lock, it will also be set when it's still read-locked and read-unlocking will need special handling for that situation.

Simplifying the overflow check doesn't help, because at the points where this is checked, we also need to check the waiting bits too.

I also prefer to combine the fields, to make invalid states unrepresentable. (Effectively enum { Unlocked, ReadLocked(NonZeroNonMaxU30), WriteLocked } instead of struct { read_locked: u29, write_locked: bool }.)

library/std/src/sys/unix/locks/futex_rwlock.rs Outdated Show resolved Hide resolved
@Amanieu
Copy link
Member

Amanieu commented Apr 8, 2022

I think the design is sound. At the very least I couldn't find a way to break it.

Co-authored-by: Amanieu d'Antras <amanieu@gmail.com>
@m-ou-se
Copy link
Member Author

m-ou-se commented Apr 8, 2022

Thanks for the quick review! I'll add some comments to the code to clarify the things you pointed out.

@m-ou-se m-ou-se added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 8, 2022
library/std/src/sys/unix/locks/futex_rwlock.rs Outdated Show resolved Hide resolved
library/std/src/sys/unix/locks/futex_rwlock.rs Outdated Show resolved Hide resolved
library/std/src/sys/unix/locks/futex_rwlock.rs Outdated Show resolved Hide resolved
library/std/src/sys/unix/locks/futex_rwlock.rs Outdated Show resolved Hide resolved
@m-ou-se m-ou-se added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Apr 11, 2022
@m-ou-se
Copy link
Member Author

m-ou-se commented Apr 11, 2022

@Amanieu I've addressed all your comments. :)

@Amanieu
Copy link
Member

Amanieu commented Apr 11, 2022

@bors r+

@bors
Copy link
Contributor

bors commented Apr 11, 2022

📌 Commit 8339381 has been approved by Amanieu

@bors bors removed the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Apr 11, 2022
@bors bors added the S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. label Apr 11, 2022
Dylan-DPC added a commit to Dylan-DPC/rust that referenced this pull request Apr 11, 2022
Replace RwLock by a futex based one on Linux

This replaces the pthread-based RwLock on Linux by a futex based one.

This implementation is similar to [the algorithm](https://gist.github.com/kprotty/3042436aa55620d8ebcddf2bf25668bc) suggested by `@kprotty,` but modified to prefer writers and spin before sleeping. It uses two futexes: One for the readers to wait on, and one for the writers to wait on. The readers futex contains the state of the RwLock: The number of readers, a bit indicating whether writers are waiting, and a bit indicating whether readers are waiting. The writers futex is used as a simple condition variable and its contents are meaningless; it just needs to be changed on every notification.

Using two futexes rather than one has the obvious advantage of allowing a separate queue for readers and writers, but it also means we avoid the problem a single-futex RwLock would have of making it hard for a writer to go to sleep while the number of readers is rapidly changing up and down, as the writers futex is only changed when we actually want to wake up a writer.

It always prefers writers, as we decided [here](rust-lang#93740 (comment)).

To be able to prefer writers, it relies on futex_wake to return the number of awoken threads to be able to handle write-unlocking while both the readers-waiting and writers-waiting bits are set. Instead of waking both and letting them race, it first wakes writers and only continues to wake the readers too if futex_wake reported there were no writers to wake up.

r? `@Amanieu`
@Soveu
Copy link
Contributor

Soveu commented Apr 11, 2022

Great job!

@m-ou-se
Copy link
Member Author

m-ou-se commented Apr 11, 2022

@bors rollup=never

bors added a commit to rust-lang-ci/rust that referenced this pull request Apr 11, 2022
Rollup of 7 pull requests

Successful merges:

 - rust-lang#95008 ([`let_chains`] Forbid `let` inside parentheses)
 - rust-lang#95801 (Replace RwLock by a futex based one on Linux)
 - rust-lang#95864 (Fix miscompilation of inline assembly with outputs in cases where we emit an invoke instead of call instruction.)
 - rust-lang#95894 (Fix formatting error in pin.rs docs)
 - rust-lang#95895 (Clarify str::from_utf8_unchecked's invariants)
 - rust-lang#95901 (Remove duplicate aliases for `check codegen_{cranelift,gcc}` and fix `build codegen_gcc`)
 - rust-lang#95927 (CI: do not compile libcore twice when performing LLVM PGO)

Failed merges:

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit a15ac30 into rust-lang:master Apr 12, 2022
@rustbot rustbot added this to the 1.62.0 milestone Apr 12, 2022
@m-ou-se m-ou-se deleted the futex-rwlock branch April 12, 2022 04:32
wip-sync pushed a commit to NetBSD/pkgsrc-wip that referenced this pull request Jul 12, 2022
Pkgsrc changes:
 * adapt patches (libc crate version bump)
 * no longer pass -I/usr/pkg/include through via gcc-wrap script.
   Attempt at fixing version skew with curl package vs. internal
   version of curl.
 * new checksums

Upstream changes:

Version 1.62.0 (2022-06-30)
==========================

Language
--------

- [Stabilize `#[derive(Default)]` on enums with a `#[default]` variant][94457]
- [Stop validating some checks in dead code after functions with
  uninhabited return types][93313]
- [Fix constants not getting dropped if part of a diverging expression][94775]
- [Support unit struct/enum variant in destructuring assignment][95380]
- [Remove mutable_borrow_reservation_conflict lint and allow the
  code pattern][96268]

Compiler
--------

- [linker: Stop using whole-archive on dependencies of dylibs][96436]
- [Make `unaligned_references` lint deny-by-default][95372]
  This lint is also a future compatibility lint, and is expected to eventually
  become a hard error.
- [Only add codegen backend to dep info if -Zbinary-dep-depinfo is used][93969]
- [Reject `#[thread_local]` attribute on non-static items][95006]
- [Add tier 3 `aarch64-pc-windows-gnullvm` and `x86_64-pc-windows-gnullvm`
  targets\*][94872]
- [Implement a lint to warn about unused macro rules][96150]
- [Promote `x86_64-unknown-none` target to Tier 2\*][95705]

\* Refer to Rust's [platform support page][platform-support-doc] for more
   information on Rust's tiered platform support.

Libraries
---------

- [Move `CStr` to libcore, and `CString` to liballoc][94079]
- [Windows: Use a pipe relay for chaining pipes][95841]
- [Replace Linux Mutex and Condvar with futex based ones.][95035]
- [Replace RwLock by a futex based one on Linux][95801]
- [std: directly use pthread in UNIX parker implementation][96393]

Stabilized APIs
---------------

- [`bool::then_some`]
- [`f32::total_cmp`]
- [`f64::total_cmp`]
- [`Stdin::lines`]
- [`windows::CommandExt::raw_arg`]
- [`impl<T: Default> Default for AssertUnwindSafe<T>`]
- [`From<Rc<str>> for Rc<[u8]>`][rc-u8-from-str]
- [`From<Arc<str>> for Arc<[u8]>`][arc-u8-from-str]
- [`FusedIterator for EncodeWide`]
- [RDM intrinsics on aarch64][stdarch/1285]

Clippy
------

- [Create clippy lint against unexpectedly late drop for temporaries
  in match scrutinee expressions][94206]

Cargo
-----

- Added the `cargo add` command for adding dependencies to `Cargo.toml` from
  the command-line.
  [docs](https://doc.rust-lang.org/nightly/cargo/commands/cargo-add.html)
- Package ID specs now support `name@version` syntax in addition to the
  previous `name:version` to align with the behavior in `cargo add` and other
  tools. `cargo install` and `cargo yank` also now support this syntax so the
  version does not need to passed as a separate flag.
- The `git` and `registry` directories in Cargo's home directory (usually
  `~/.cargo`) are now marked as cache directories so that they are not
  included in backups or content indexing (on Windows).
- Added automatic `@` argfile support, which will use "response files" if the
  command-line to `rustc` exceeds the operating system's limit.

Compatibility Notes
-------------------

- `cargo test` now passes `--target` to `rustdoc` if the specified target is
  the same as the host target.
  [#10594](rust-lang/cargo#10594)
- [rustdoc: Remove .woff font files][96279]
- [Enforce Copy bounds for repeat elements while considering lifetimes][95819]

Internal Changes
----------------

- [Unify ReentrantMutex implementations across all platforms][96042]

These changes provide no direct user facing benefits, but represent
significant improvements to the internals and overall performance
of rustc and related tools.

[93313]: rust-lang/rust#93313
[93969]: rust-lang/rust#93969
[94079]: rust-lang/rust#94079
[94206]: rust-lang/rust#94206
[94457]: rust-lang/rust#94457
[94775]: rust-lang/rust#94775
[94872]: rust-lang/rust#94872
[95006]: rust-lang/rust#95006
[95035]: rust-lang/rust#95035
[95372]: rust-lang/rust#95372
[95380]: rust-lang/rust#95380
[95431]: rust-lang/rust#95431
[95705]: rust-lang/rust#95705
[95801]: rust-lang/rust#95801
[95819]: rust-lang/rust#95819
[95841]: rust-lang/rust#95841
[96042]: rust-lang/rust#96042
[96150]: rust-lang/rust#96150
[96268]: rust-lang/rust#96268
[96279]: rust-lang/rust#96279
[96393]: rust-lang/rust#96393
[96436]: rust-lang/rust#96436
[96557]: rust-lang/rust#96557

[`bool::then_some`]: https://doc.rust-lang.org/stable/std/primitive.bool.html#method.then_some
[`f32::total_cmp`]: https://doc.rust-lang.org/stable/std/primitive.f32.html#method.total_cmp
[`f64::total_cmp`]: https://doc.rust-lang.org/stable/std/primitive.f64.html#method.total_cmp
[`Stdin::lines`]: https://doc.rust-lang.org/stable/std/io/struct.Stdin.html#method.lines
[`impl<T: Default> Default for AssertUnwindSafe<T>`]: https://doc.rust-lang.org/stable/std/panic/struct.AssertUnwindSafe.html#impl-Default
[rc-u8-from-str]: https://doc.rust-lang.org/stable/std/rc/struct.Rc.html#impl-From%3CRc%3Cstr%3E%3E
[arc-u8-from-str]: https://doc.rust-lang.org/stable/std/sync/struct.Arc.html#impl-From%3CArc%3Cstr%3E%3E
[stdarch/1285]: rust-lang/stdarch#1285
[`windows::CommandExt::raw_arg`]: https://doc.rust-lang.org/stable/std/os/windows/process/trait.CommandExt.html#tymethod.raw_arg
[`FusedIterator for EncodeWide`]: https://doc.rust-lang.org/stable/std/os/windows/ffi/struct.EncodeWide.html#impl-FusedIterator
@vireshk
Copy link

vireshk commented Aug 30, 2022

This replaces the pthread-based RwLock on Linux by a futex based one.

Any chance following issue is related to this ? Started getting this since 1.62.0.

#101194

@m-ou-se
Copy link
Member Author

m-ou-se commented Aug 30, 2022

This replaces the pthread-based RwLock on Linux by a futex based one.

Any chance following issue is related to this ? Started getting this since 1.62.0.

#101194

Yes, but that snippet never worked correctly on other operating systems, even before this change. We never guaranteed that read-locking a read-locked RwLock always succeeds. That's just a property of pthread's rwlock that your snippet accidentally relied on. E.g. the RwLock on Windows didn't have this property, and would always block reads as soon as there's a writer waiting.

netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Aug 31, 2022
Pkgsrc changes:

 * Bump required GCC to 7 (same as LLVM) to avoid ABI issues
   Fixes native i386 and powerpc 8.x build w/pkgsrc LLVM 14
 * Bump available bootstraps to 1.61.0.
 * Also unlimit stacksize
 * Sync patches over from wip/rust
 * Adjust line number in patches which had non-zero offsets.
 * no longer pass -I/usr/pkg/include through via gcc-wrap script
   when building natively.  Attempt at fixing version skew with curl
   package vs. internal version of curl (may not work...)
 * The NetBSD bootstraps now use .xz compression.
 * Use mk/atomic64.mk.  Still have conditional for libatomic-links.
 * Default to using the internal LLVM when cross-building.


Upstream changes:

Version 1.62.1 (2022-07-19)
==========================

Rust 1.62.1 addresses a few recent regressions in the compiler and standard
library, and also mitigates a CPU vulnerability on Intel SGX.

* [The compiler fixed unsound function coercions involving `impl
  Trait` return types.][98608]
* [The compiler fixed an incremental compilation bug with `async
  fn` lifetimes.][98890]
* [Windows added a fallback for overlapped I/O in synchronous reads
  and writes.][98950]
* [The `x86_64-fortanix-unknown-sgx` target added a mitigation for the
  MMIO stale data vulnerability][98126], advisory [INTEL-SA-00615].

[98608]: rust-lang/rust#98608
[98890]: rust-lang/rust#98890
[98950]: rust-lang/rust#98950
[98126]: rust-lang/rust#98126
[INTEL-SA-00615]: https://www.intel.com/content/www/us/en/security-center/advisory/intel-sa-00615.html


Version 1.62.0 (2022-06-30)
==========================

Language
--------

- [Stabilize `#[derive(Default)]` on enums with a `#[default]` variant][94457]
- [Stop validating some checks in dead code after functions with
  uninhabited return types][93313]
- [Fix constants not getting dropped if part of a diverging expression][94775]
- [Support unit struct/enum variant in destructuring assignment][95380]
- [Remove mutable_borrow_reservation_conflict lint and allow the
  code pattern][96268]

Compiler
--------

- [linker: Stop using whole-archive on dependencies of dylibs][96436]
- [Make `unaligned_references` lint deny-by-default][95372]
  This lint is also a future compatibility lint, and is expected to eventually
  become a hard error.
- [Only add codegen backend to dep info if -Zbinary-dep-depinfo is used][93969]
- [Reject `#[thread_local]` attribute on non-static items][95006]
- [Add tier 3 `aarch64-pc-windows-gnullvm` and `x86_64-pc-windows-gnullvm`
  targets\*][94872]
- [Implement a lint to warn about unused macro rules][96150]
- [Promote `x86_64-unknown-none` target to Tier 2\*][95705]

\* Refer to Rust's [platform support page][platform-support-doc] for more
   information on Rust's tiered platform support.

Libraries
---------

- [Move `CStr` to libcore, and `CString` to liballoc][94079]
- [Windows: Use a pipe relay for chaining pipes][95841]
- [Replace Linux Mutex and Condvar with futex based ones.][95035]
- [Replace RwLock by a futex based one on Linux][95801]
- [std: directly use pthread in UNIX parker implementation][96393]

Stabilized APIs
---------------

- [`bool::then_some`]
- [`f32::total_cmp`]
- [`f64::total_cmp`]
- [`Stdin::lines`]
- [`windows::CommandExt::raw_arg`]
- [`impl<T: Default> Default for AssertUnwindSafe<T>`]
- [`From<Rc<str>> for Rc<[u8]>`][rc-u8-from-str]
- [`From<Arc<str>> for Arc<[u8]>`][arc-u8-from-str]
- [`FusedIterator for EncodeWide`]
- [RDM intrinsics on aarch64][stdarch/1285]

Clippy
------

- [Create clippy lint against unexpectedly late drop for temporaries
  in match scrutinee expressions][94206]

Cargo
-----

- Added the `cargo add` command for adding dependencies to `Cargo.toml` from
  the command-line.
  [docs](https://doc.rust-lang.org/nightly/cargo/commands/cargo-add.html)
- Package ID specs now support `name@version` syntax in addition to the
  previous `name:version` to align with the behavior in `cargo add` and other
  tools. `cargo install` and `cargo yank` also now support this syntax so the
  version does not need to passed as a separate flag.
- The `git` and `registry` directories in Cargo's home directory (usually
  `~/.cargo`) are now marked as cache directories so that they are not
  included in backups or content indexing (on Windows).
- Added automatic `@` argfile support, which will use "response files" if the
  command-line to `rustc` exceeds the operating system's limit.

Compatibility Notes
-------------------

- `cargo test` now passes `--target` to `rustdoc` if the specified target is
  the same as the host target.
  [#10594](rust-lang/cargo#10594)
- [rustdoc: Remove .woff font files][96279]
- [Enforce Copy bounds for repeat elements while considering lifetimes][95819]

Internal Changes
----------------

- [Unify ReentrantMutex implementations across all platforms][96042]

These changes provide no direct user facing benefits, but represent
significant improvements to the internals and overall performance
of rustc and related tools.

[93313]: rust-lang/rust#93313
[93969]: rust-lang/rust#93969
[94079]: rust-lang/rust#94079
[94206]: rust-lang/rust#94206
[94457]: rust-lang/rust#94457
[94775]: rust-lang/rust#94775
[94872]: rust-lang/rust#94872
[95006]: rust-lang/rust#95006
[95035]: rust-lang/rust#95035
[95372]: rust-lang/rust#95372
[95380]: rust-lang/rust#95380
[95431]: rust-lang/rust#95431
[95705]: rust-lang/rust#95705
[95801]: rust-lang/rust#95801
[95819]: rust-lang/rust#95819
[95841]: rust-lang/rust#95841
[96042]: rust-lang/rust#96042
[96150]: rust-lang/rust#96150
[96268]: rust-lang/rust#96268
[96279]: rust-lang/rust#96279
[96393]: rust-lang/rust#96393
[96436]: rust-lang/rust#96436
[96557]: rust-lang/rust#96557

[`bool::then_some`]: https://doc.rust-lang.org/stable/std/primitive.bool.html#method.then_some
[`f32::total_cmp`]: https://doc.rust-lang.org/stable/std/primitive.f32.html#method.total_cmp
[`f64::total_cmp`]: https://doc.rust-lang.org/stable/std/primitive.f64.html#method.total_cmp
[`Stdin::lines`]: https://doc.rust-lang.org/stable/std/io/struct.Stdin.html#method.lines
[`impl<T: Default> Default for AssertUnwindSafe<T>`]: https://doc.rust-lang.org/stable/std/panic/struct.AssertUnwindSafe.html#impl-Default
[rc-u8-from-str]: https://doc.rust-lang.org/stable/std/rc/struct.Rc.html#impl-From%3CRc%3Cstr%3E%3E
[arc-u8-from-str]: https://doc.rust-lang.org/stable/std/sync/struct.Arc.html#impl-From%3CArc%3Cstr%3E%3E
[stdarch/1285]: rust-lang/stdarch#1285
[`windows::CommandExt::raw_arg`]: https://doc.rust-lang.org/stable/std/os/windows/process/trait.CommandExt.html#tymethod.raw_arg
[`FusedIterator for EncodeWide`]: https://doc.rust-lang.org/stable/std/os/windows/ffi/struct.EncodeWide.html#impl-FusedIterator


Version 1.61.0 (2022-05-19)
==========================

Language
--------

- [`const fn` signatures can now include generic trait bounds][93827]
- [`const fn` signatures can now use `impl Trait` in argument and return
  position][93827]
- [Function pointers can now be created, cast, and passed around in a
  `const fn`][93827]
- [Recursive calls can now set the value of a function's opaque
  `impl Trait` return type][94081]

Compiler
--------

- [Linking modifier syntax in `#[link]` attributes and on the command
  line, as well as the `whole-archive` modifier specifically, are now
  supported][93901]
- [The `char` type is now described as UTF-32 in debuginfo][89887]
- The [`#[target_feature]`][target_feature] attribute
  [can now be used with aarch64 features][90621]
- X86 [`#[target_feature = "adx"]` is now stable][93745]

Libraries
---------

- [`ManuallyDrop<T>` is now documented to have the same layout as `T`][88375]
- [`#[ignore = "#"]` messages are printed when running tests][92714]
- [Consistently show absent stdio handles on Windows as NULL handles][93263]
- [Make `std::io::stdio::lock()` return `'static` handles.][93965]
  Previously, the creation of locked handles to stdin/stdout/stderr would
  borrow the handles being locked, which prevented writing
  `let out = std::io::stdout().lock();` because `out` would outlive
  the return value of `stdout()`.
  Such code now works, eliminating a common pitfall that affected many
  Rust users.
- [`Vec::from_raw_parts` is now less restrictive about its inputs][95016]
- [`std::thread::available_parallelism` now takes cgroup quotas into
  account.][92697] Since `available_parallelism` is often used to create a
  thread pool for parallel computation, which may be CPU-bound for
  performance, `available_parallelism` will return a value consistent with
  the ability to use that many threads continuously, if possible.
  For instance, in a container with 8 virtual CPUs but quotas only allowing
  for 50% usage, `available_parallelism` will return 4.

Stabilized APIs
---------------

- [`Pin::static_mut`]
- [`Pin::static_ref`]
- [`Vec::retain_mut`]
- [`VecDeque::retain_mut`]
- [`Write` for `Cursor<[u8; N]>`][cursor-write-array]
- [`std::os::unix::net::SocketAddr::from_pathname`]
- [`std::process::ExitCode`] and [`std::process::Termination`].
  The stabilization of these two API s now makes it possible for
  programs to return errors from `main` with custom exit codes.
- [`std::thread::JoinHandle::is_finished`]

These APIs are now usable in const contexts:

- [`<*const T>::offset` and `<*mut T>::offset`][ptr-offset]
- [`<*const T>::wrapping_offset` and `<*mut T>::wrapping_offset`]
  [ptr-wrapping_offset]
- [`<*const T>::add` and `<*mut T>::add`][ptr-add]
- [`<*const T>::sub` and `<*mut T>::sub`][ptr-sub]
- [`<*const T>::wrapping_add` and `<*mut T>::wrapping_add`][ptr-wrapping_add]
- [`<*const T>::wrapping_sub` and `<*mut T>::wrapping_sub`][ptr-wrapping_sub]
- [`<[T]>::as_mut_ptr`][slice-as_mut_ptr]
- [`<[T]>::as_ptr_range`][slice-as_ptr_range]
- [`<[T]>::as_mut_ptr_range`][slice-as_mut_ptr_range]

Cargo
-----

No feature changes, but see compatibility notes.

Compatibility Notes
-------------------

- Previously native static libraries were linked as `whole-archive` in
  some cases, but now rustc tries not to use `whole-archive` unless
  explicitly requested. This [change][93901] may result in linking errors
  in some cases. To fix such errors, native libraries linked from the
  command line, build scripts, or [`#[link]` attributes][link-attr] need to
  - (more common) either be reordered to respect dependencies between them
    (if `a` depends on `b` then `a` should go first and `b` second)
  - (less common) or be updated to use the [`+whole-archive`] modifier.
- [Catching a second unwind from FFI code while cleaning up from a Rust
  panic now causes the process to abort][92911]
- [Proc macros no longer see `ident` matchers wrapped in groups][92472]
- [The number of `#` in `r#` raw string literals is now required to be
  less than 256][95251]
- [When checking that a dyn type satisfies a trait bound, supertrait
  bounds are now enforced][92285]
- [`cargo vendor` now only accepts one value for each `--sync` flag]
  [cargo/10448]
- [`cfg` predicates in `all()` and `any()` are always evaluated to detect
  errors, instead of short-circuiting.][94295] The compatibility
  considerations here arise in nightly-only code that used the
  short-circuiting behavior of `all` to write something like
  `cfg(all(feature = "nightly", syntax-requiring-nightly))`, which
  will now fail to compile. Instead, use either `cfg_attr(feature
  = "nightly", ...)` or nested uses of `cfg`.
- [bootstrap: static-libstdcpp is now enabled by default, and can
  now be disabled when llvm-tools is enabled][94832]

Internal Changes
----------------

These changes provide no direct user facing benefits, but represent
significant improvements to the internals and overall performance
of rustc and related tools.

- [debuginfo: Refactor debuginfo generation for types][94261]
- [Remove the everybody loops pass][93913]

[88375]: rust-lang/rust#88375
[89887]: rust-lang/rust#89887
[90621]: rust-lang/rust#90621
[92285]: rust-lang/rust#92285
[92472]: rust-lang/rust#92472
[92697]: rust-lang/rust#92697
[92714]: rust-lang/rust#92714
[92911]: rust-lang/rust#92911
[93263]: rust-lang/rust#93263
[93745]: rust-lang/rust#93745
[93827]: rust-lang/rust#93827
[93901]: rust-lang/rust#93901
[93913]: rust-lang/rust#93913
[93965]: rust-lang/rust#93965
[94081]: rust-lang/rust#94081
[94261]: rust-lang/rust#94261
[94295]: rust-lang/rust#94295
[94832]: rust-lang/rust#94832
[95016]: rust-lang/rust#95016
[95251]: rust-lang/rust#95251
[`+whole-archive`]: https://doc.rust-lang.org/stable/rustc/command-line-arguments.html#linking-modifiers-whole-archive
[`Pin::static_mut`]: https://doc.rust-lang.org/stable/std/pin/struct.Pin.html#method.static_mut
[`Pin::static_ref`]: https://doc.rust-lang.org/stable/std/pin/struct.Pin.html#method.static_ref
[`Vec::retain_mut`]: https://doc.rust-lang.org/stable/std/vec/struct.Vec.html#method.retain_mut
[`VecDeque::retain_mut`]: https://doc.rust-lang.org/stable/std/collections/struct.VecDeque.html#method.retain_mut
[`std::os::unix::net::SocketAddr::from_pathname`]: https://doc.rust-lang.org/stable/std/os/unix/net/struct.SocketAddr.html#method.from_pathname
[`std::process::ExitCode`]: https://doc.rust-lang.org/stable/std/process/struct.ExitCode.html
[`std::process::Termination`]: https://doc.rust-lang.org/stable/std/process/trait.Termination.html
[`std::thread::JoinHandle::is_finished`]: https://doc.rust-lang.org/stable/std/thread/struct.JoinHandle.html#method.is_finished
[cargo/10448]: rust-lang/cargo#10448
[cursor-write-array]: https://doc.rust-lang.org/stable/std/io/struct.Cursor.html#impl-Write-4
[link-attr]: https://doc.rust-lang.org/stable/reference/items/external-blocks.html#the-link-attribute
[ptr-add]: https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.add
[ptr-offset]: https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.offset
[ptr-sub]: https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.sub
[ptr-wrapping_add]: https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.wrapping_add
[ptr-wrapping_offset]: https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.wrapping_offset
[ptr-wrapping_sub]: https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.wrapping_sub
[slice-as_mut_ptr]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.as_mut_ptr
[slice-as_mut_ptr_range]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.as_mut_ptr_range
[slice-as_ptr_range]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.as_ptr_range
[target_feature]: https://doc.rust-lang.org/reference/attributes/codegen.html#the-target_feature-attribute
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-concurrency Area: Concurrency O-linux Operating system: Linux S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-libs Relevant to the library team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants