Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add assertion that transmute_copy's U is not larger than T #98839

Merged
merged 1 commit into from
Jul 18, 2022

Conversation

5225225
Copy link
Contributor

@5225225 5225225 commented Jul 3, 2022

This is called out as a safety requirement in the docs, but because knowing this can be done at compile time and constant folded (just like the align_of branch is removed), we can just panic here.

I've looked at the asm (using cargo-asm) of a function that both is correct and incorrect, and the panic is completely removed, or is unconditional, without needing build-std.

I don't expect this to cause much breakage in the wild. I scanned through https://miri.saethlin.dev/ub for issues that would look like this (error: Undefined Behavior: memory access failed: alloc1768 has size 1, so pointer to 8 bytes starting at offset 0 is out-of-bounds), but couldn't find any.

That doesn't rule out it happening in crates tested that fail earlier for some other reason, though, but it indicates that doing this is rare, if it happens at all. A crater run for this would need to be build and test, since this is a runtime thing.

Also added a few more transmute_copy tests.

@rustbot rustbot added the T-libs Relevant to the library team, which will review and decide on the PR/issue. label Jul 3, 2022
@rustbot
Copy link
Collaborator

rustbot commented Jul 3, 2022

Hey! It looks like you've submitted a new PR for the library teams!

If this PR contains changes to any rust-lang/rust public library APIs then please comment with @rustbot label +T-libs-api -T-libs to tag it appropriately. If this PR contains changes to any unstable APIs please edit the PR description to add a link to the relevant API Change Proposal or create one if you haven't already. If you're unsure where your change falls no worries, just leave it as is and the reviewer will take a look and make a decision to forward on if necessary.

Examples of T-libs-api changes:

  • Stabilizing library features
  • Introducing insta-stable changes such as new implementations of existing stable traits on existing stable types
  • Introducing new or changing existing unstable library APIs (excluding permanently unstable features / features without a tracking issue)
  • Changing public documentation in ways that create new stability guarantees
  • Changing observable runtime behavior of library APIs

@rust-highfive
Copy link
Collaborator

r? @thomcc

(rust-highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jul 3, 2022
@thomcc
Copy link
Member

thomcc commented Jul 3, 2022

https://doc.rust-lang.org/nightly/std/mem/fn.transmute_copy.html says

This function triggers undefined behavior if U is larger than T.

So we're allowed to make this change. It also should either always vanish, or turn into an unconditional panic (in which case it was UB anyway), so there's no performance reason to make it a debug assertion (the way there was for #92686, but I wonder if it's worth doing a crater run to avoid potential nasty surprises (I wouldn't be surprised if this happens in FFI code, which wouldn't be testable under miri, and often plays it very fast and loose with references to ZSTs, treating them basically like raw pointers).

I'm not sure if that sounds likely enough to warrant it though (anybody feel strongly?), but a try run is step one either way.

@bors try

@bors
Copy link
Contributor

bors commented Jul 3, 2022

⌛ Trying commit 5f5ca88 with merge 61cd42370816fd135c0f3b9debf2f5dfcf194594...

@bors
Copy link
Contributor

bors commented Jul 3, 2022

☀️ Try build successful - checks-actions
Build commit: 61cd42370816fd135c0f3b9debf2f5dfcf194594 (61cd42370816fd135c0f3b9debf2f5dfcf194594)

@thomcc
Copy link
Member

thomcc commented Jul 4, 2022

Any code broken by this violates the requirement in the documentation, but it has Worked In Practice (TM) for a very long time, and this would change it to become an unconditional panic, even in release builds.

Notably, whether or not it's always UB to read past the end of a &T is still up in the air rust-lang/unsafe-code-guidelines#256, and in some situations this operation may be allowed in certain situations (so long as the read is in bounds for the allocation, isn't a data race, doesn't overlap some other &mut T, doesn't contain bytes that violate a validity invariant for U, etc...).

That said, it's pretty clearly currently against the rules to use transmute_copy like that, and has been explicitly documented as such since before Rust 1.0.


For what it's worth, my hestitation is that I've personally had code broken by this kind of change, and as a result, think that it should be made cautiously.

Concretely, I can no longer run the examples of https://github.com/thomcc/bad3d (one of my first Rust projects), because the ancient version of glium it uses ends up unconditionally aborting in modern versions of Rust, as it contains a broken offset_of! implementation which misuses mem::zeroed.

That's not to say that we should allow this: If we say something is UB, then we're obviously within our rights to change it to an unconditional abort. And in particular, I do think the case of mem::zeroed was justified, even if it broke my code in a way that cannot easily be fixed, short of a rewrite (this is mostly due to winit changes, as updating glium would updating it winit too). Rather, I'm just saying that these sorts of things deserve should be made cautiously.


Anyway, all that is my justification for doing a crater run rather than just going ahead with it. Concretely, IMO if this shows much breakage, I think we may want to make this into a debug_assert! instead.

@craterbot run mode=build-and-test

P.S. Sorry if I'm not following proper crater process -- https://std-dev-guide.rust-lang.org/tools-and-bots/crater.html is a just stub and doesn't indicate how much justification is needed, hence why I included a rationale for why I'm running it. Note that we need to run tests too, as this won't be caught at compile time.

@craterbot
Copy link
Collaborator

👌 Experiment pr-98839 created and queued.
🤖 Automatically detected try build 61cd42370816fd135c0f3b9debf2f5dfcf194594
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot craterbot added S-waiting-on-crater Status: Waiting on a crater run to be completed. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 4, 2022
@craterbot
Copy link
Collaborator

🚧 Experiment pr-98839 is now running

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot
Copy link
Collaborator

🚨 Report generation of pr-98839 failed: Failed to upload to "logs-archives/all.tar.gz": DispatchFailure(ConnectorError { err: hyper::Error(BodyWrite, Os { code: 32, kind: BrokenPipe, message: "Broken pipe" }), kind: Io })
🛠️ If the error is fixed use the retry-report command.

🆘 Can someone from the infra team check in on this? @rust-lang/infra
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@jyn514
Copy link
Member

jyn514 commented Jul 8, 2022

"broken pipe" means the remote (or a proxy?) closed the connection halfway through the upload; seems spurious. I'm surprised there isn't a built-in retry?

@craterbot retry-report

@craterbot
Copy link
Collaborator

🛠️ Generation of the report for pr-98839 queued again.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot
Copy link
Collaborator

🚨 Report generation of pr-98839 failed: Failed to upload to "logs-archives/all.tar.gz": DispatchFailure(ConnectorError { err: hyper::Error(BodyWrite, Os { code: 32, kind: BrokenPipe, message: "Broken pipe" }), kind: Io })
🛠️ If the error is fixed use the retry-report command.

🆘 Can someone from the infra team check in on this? @rust-lang/infra
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot
Copy link
Collaborator

🎉 Experiment pr-98839 is completed!
📊 142 regressed and 72 fixed (238905 total)
📰 Open the full report.

⚠️ If you notice any spurious failure please add them to the blacklist!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot craterbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-crater Status: Waiting on a crater run to be completed. labels Jul 8, 2022
@5225225
Copy link
Contributor Author

5225225 commented Jul 8, 2022

Okay, notified/opened PRs on all the regressed crates. Alacritty already merged theirs, but the other two haven't seen activity in 2 years.

I'll write up a rustsec advisory for pkcs11 since it both has a lot of users (66k recent downloads), and this doesn't seem to be the only issue there.

message-passing-framework is github only, so I doubt anyone is using it as a dependency. So the main issue is pkcs11.

@5225225
Copy link
Contributor Author

5225225 commented Jul 8, 2022

I created rustsec/advisory-db#1280 as a note, there's a fair few issues with that crate it seems, and I can't work out what the issues actually are, so I don't really feel able to write an advisory for it.

@thomcc
Copy link
Member

thomcc commented Jul 16, 2022

Hm, alright. Hopefully this doesn't end up causing issues.

@bors r+

@bors
Copy link
Contributor

bors commented Jul 16, 2022

📌 Commit 5f5ca88 has been approved by thomcc

It is now in the queue for this repository.

@thomcc
Copy link
Member

thomcc commented Jul 18, 2022

Hm, well, I don't love the thought of breaking a bunch of code without a migration path (e.g. if it's in a dependency they use), especially given the fact that this is mostly artificial (rather than genuine UB, like in the uninitialized::<HasANiche>() case), but I don't think we want to allow this, so...

@bors r+

@bors
Copy link
Contributor

bors commented Jul 18, 2022

📌 Commit 5f5ca88 has been approved by thomcc

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jul 18, 2022
@5225225
Copy link
Contributor Author

5225225 commented Jul 18, 2022

Well, it's reading a reference out of bounds of its provenance, though I suppose we don't tell LLVM anything about that so it's "only" language UB.

Still, might as well discourage the use of transmute_copy like this.

And also it catches issues like the alacritty bug which did look to be straight up UB, since it was both reading out of bounds of an object, and passing uninit data (possibly? depends how the OOB read went) to a library. Granted, that was in a test so it's Fine but there's no reason people can't write that outside of a test.

@thomcc
Copy link
Member

thomcc commented Jul 18, 2022

Well, it's reading a reference out of bounds of its provenance, though I suppose we don't tell LLVM anything about that so it's "only" language UB.

I mean, it's equivalent to rust-lang/unsafe-code-guidelines#256, which is currently UB, but may not be UB in the future. I believe that there have been several indications in that thread and elsewhere that we would like for this not to be UB in the future.

I agree that the alacritty issue was definitely UB.

Dylan-DPC added a commit to Dylan-DPC/rust that referenced this pull request Jul 18, 2022
… r=thomcc

Add assertion that `transmute_copy`'s U is not larger than T

This is called out as a safety requirement in the docs, but because knowing this can be done at compile time and constant folded (just like the `align_of` branch is removed), we can just panic here.

I've looked at the asm (using `cargo-asm`) of a function that both is correct and incorrect, and the panic is completely removed, or is unconditional, without needing build-std.

I don't expect this to cause much breakage in the wild. I scanned through https://miri.saethlin.dev/ub for issues that would look like this (error: Undefined Behavior: memory access failed: alloc1768 has size 1, so pointer to 8 bytes starting at offset 0 is out-of-bounds), but couldn't find any.

That doesn't rule out it happening in crates tested that fail earlier for some other reason, though, but it indicates that doing this is rare, if it happens at all. A crater run for this would need to be build and test, since this is a runtime thing.

Also added a few more transmute_copy tests.
Dylan-DPC added a commit to Dylan-DPC/rust that referenced this pull request Jul 18, 2022
… r=thomcc

Add assertion that `transmute_copy`'s U is not larger than T

This is called out as a safety requirement in the docs, but because knowing this can be done at compile time and constant folded (just like the `align_of` branch is removed), we can just panic here.

I've looked at the asm (using `cargo-asm`) of a function that both is correct and incorrect, and the panic is completely removed, or is unconditional, without needing build-std.

I don't expect this to cause much breakage in the wild. I scanned through https://miri.saethlin.dev/ub for issues that would look like this (error: Undefined Behavior: memory access failed: alloc1768 has size 1, so pointer to 8 bytes starting at offset 0 is out-of-bounds), but couldn't find any.

That doesn't rule out it happening in crates tested that fail earlier for some other reason, though, but it indicates that doing this is rare, if it happens at all. A crater run for this would need to be build and test, since this is a runtime thing.

Also added a few more transmute_copy tests.
Dylan-DPC added a commit to Dylan-DPC/rust that referenced this pull request Jul 18, 2022
… r=thomcc

Add assertion that `transmute_copy`'s U is not larger than T

This is called out as a safety requirement in the docs, but because knowing this can be done at compile time and constant folded (just like the `align_of` branch is removed), we can just panic here.

I've looked at the asm (using `cargo-asm`) of a function that both is correct and incorrect, and the panic is completely removed, or is unconditional, without needing build-std.

I don't expect this to cause much breakage in the wild. I scanned through https://miri.saethlin.dev/ub for issues that would look like this (error: Undefined Behavior: memory access failed: alloc1768 has size 1, so pointer to 8 bytes starting at offset 0 is out-of-bounds), but couldn't find any.

That doesn't rule out it happening in crates tested that fail earlier for some other reason, though, but it indicates that doing this is rare, if it happens at all. A crater run for this would need to be build and test, since this is a runtime thing.

Also added a few more transmute_copy tests.
Dylan-DPC added a commit to Dylan-DPC/rust that referenced this pull request Jul 18, 2022
… r=thomcc

Add assertion that `transmute_copy`'s U is not larger than T

This is called out as a safety requirement in the docs, but because knowing this can be done at compile time and constant folded (just like the `align_of` branch is removed), we can just panic here.

I've looked at the asm (using `cargo-asm`) of a function that both is correct and incorrect, and the panic is completely removed, or is unconditional, without needing build-std.

I don't expect this to cause much breakage in the wild. I scanned through https://miri.saethlin.dev/ub for issues that would look like this (error: Undefined Behavior: memory access failed: alloc1768 has size 1, so pointer to 8 bytes starting at offset 0 is out-of-bounds), but couldn't find any.

That doesn't rule out it happening in crates tested that fail earlier for some other reason, though, but it indicates that doing this is rare, if it happens at all. A crater run for this would need to be build and test, since this is a runtime thing.

Also added a few more transmute_copy tests.
bors added a commit to rust-lang-ci/rust that referenced this pull request Jul 18, 2022
Rollup of 7 pull requests

Successful merges:

 - rust-lang#98839 (Add assertion that `transmute_copy`'s U is not larger than T)
 - rust-lang#98998 (Remove branch target prologues from `#[naked] fn`)
 - rust-lang#99198 (add missing null ptr check in alloc example)
 - rust-lang#99344 (rustdoc: avoid inlining items with duplicate `(type, name)`)
 - rust-lang#99351 (Use `typeck_results` to get accurate qpath res for arg mismatch error)
 - rust-lang#99378 (interpret/visitor: add missing early return)
 - rust-lang#99394 (Add regression test for rust-lang#95230)

Failed merges:

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit 5ccdf1f into rust-lang:master Jul 18, 2022
@rustbot rustbot added this to the 1.64.0 milestone Jul 18, 2022
@scottmcm
Copy link
Member

My 2¢ from the peanut gallery:

I think it's notable that transmute_copy takes &T, and thus I think this change is absolutely justified.

If it took *const T, then I might be more hesitant, since people definitely expect to do more weird things with pointers, and we allow more things through them.

If people want p.cast().read(), they can write that. But something in mem that takes a reference should not permit these shenanigans.

@thomcc
Copy link
Member

thomcc commented Jul 20, 2022

Right, that's part of why I accepted it. I don't think we want to support this on references.

@5225225 5225225 deleted the assert_transmute_copy_size branch July 23, 2022 17:51
@Mark-Simulacrum Mark-Simulacrum added the relnotes Marks issues that should be documented in the release notes of the next release. label Aug 14, 2022
wip-sync pushed a commit to NetBSD/pkgsrc-wip that referenced this pull request Oct 11, 2022
Pkgsrc changes:
 * Add patch to fix vendor/kqueue issue (on 32-bit hosts)
 * Adjust other patches & line numbers
 * Version bumps & checksum changes.

Upstream changes:

Version 1.64.0 (2022-09-22)
===========================

Language
--------
- [Unions with mutable references or tuples of allowed types are
  now allowed](rust-lang/rust#97995)

- It is now considered valid to deallocate memory pointed to by a
  shared reference `&T` [if every byte in `T` is inside an
  `UnsafeCell`](rust-lang/rust#98017)

- Unused tuple struct fields are now warned against in an
  allow-by-default lint, [`unused_tuple_struct_fields`]
  (rust-lang/rust#95977), similar to the
  existing warning for unused struct fields. This lint will become
  warn-by-default in the future.

Compiler
--------
- [Add Nintendo Switch as tier 3 target]
  (rust-lang/rust#88991)
  - Refer to Rust's [platform support page][platform-support-doc] for more
    information on Rust's tiered platform support.
- [Only compile `#[used]` as llvm.compiler.used for ELF targets]
  (rust-lang/rust#93718)
- [Add the `--diagnostic-width` compiler flag to define the terminal width.]
  (rust-lang/rust#95635)
- [Add support for link-flavor `rust-lld` for iOS, tvOS and watchOS]
  (rust-lang/rust#98771)

Libraries
---------
- [Remove restrictions on compare-exchange memory ordering.]
  (rust-lang/rust#98383)
- You can now `write!` or `writeln!` into an `OsString`: [Implement
  `fmt::Write` for `OsString`](rust-lang/rust#97915)
- [Make RwLockReadGuard covariant]
  (rust-lang/rust#96820)
- [Implement `FusedIterator` for `std::net::[Into]Incoming`]
  (rust-lang/rust#97300)
- [`impl<T: AsRawFd> AsRawFd for {Arc,Box}<T>`]
  (rust-lang/rust#97437)
- [`ptr::copy` and `ptr::swap` are doing untyped copies]
  (rust-lang/rust#97712)
- [Add cgroupv1 support to `available_parallelism`]
  (rust-lang/rust#97925)
- [Mitigate many incorrect uses of `mem::uninitialized`]
  (rust-lang/rust#99182)

Stabilized APIs
---------------
- [`future::IntoFuture`]
  (https://doc.rust-lang.org/stable/std/future/trait.IntoFuture.html)
- [`future::poll_fn`]
  (https://doc.rust-lang.org/stable/std/future/fn.poll_fn.html)
- [`task::ready!`]
  (https://doc.rust-lang.org/stable/std/task/macro.ready.html)
- [`num::NonZero*::checked_mul`]
  (https://doc.rust-lang.org/stable/std/num/struct.NonZeroUsize.html#method.checked_mul)
- [`num::NonZero*::checked_pow`]
  (https://doc.rust-lang.org/stable/std/num/struct.NonZeroUsize.html#method.checked_pow)
- [`num::NonZero*::saturating_mul`]
  (https://doc.rust-lang.org/stable/std/num/struct.NonZeroUsize.html#method.saturating_mul)
- [`num::NonZero*::saturating_pow`]
  (https://doc.rust-lang.org/stable/std/num/struct.NonZeroUsize.html#method.saturating_pow)
- [`num::NonZeroI*::abs`]
  (https://doc.rust-lang.org/stable/std/num/struct.NonZeroIsize.html#method.abs)
- [`num::NonZeroI*::checked_abs`]
  (https://doc.rust-lang.org/stable/std/num/struct.NonZeroIsize.html#method.checked_abs)
- [`num::NonZeroI*::overflowing_abs`]
  (https://doc.rust-lang.org/stable/std/num/struct.NonZeroIsize.html#method.overflowing_abs)
- [`num::NonZeroI*::saturating_abs`]
  (https://doc.rust-lang.org/stable/std/num/struct.NonZeroIsize.html#method.saturating_abs)
- [`num::NonZeroI*::unsigned_abs`]
  (https://doc.rust-lang.org/stable/std/num/struct.NonZeroIsize.html#method.unsigned_abs)
- [`num::NonZeroI*::wrapping_abs`]
  (https://doc.rust-lang.org/stable/std/num/struct.NonZeroIsize.html#method.wrapping_abs)
- [`num::NonZeroU*::checked_add`]
  (https://doc.rust-lang.org/stable/std/num/struct.NonZeroUsize.html#method.checked_add)
- [`num::NonZeroU*::checked_next_power_of_two`]
  (https://doc.rust-lang.org/stable/std/num/struct.NonZeroUsize.html#method.checked_next_power_of_two)
- [`num::NonZeroU*::saturating_add`]
  (https://doc.rust-lang.org/stable/std/num/struct.NonZeroUsize.html#method.saturating_add)
- [`os::unix::process::CommandExt::process_group`]
  (https://doc.rust-lang.org/stable/std/os/unix/process/trait.CommandExt.html#tymethod.process_group)
- [`os::windows::fs::FileTypeExt::is_symlink_dir`]
  (https://doc.rust-lang.org/stable/std/os/windows/fs/trait.FileTypeExt.html#tymethod.is_symlink_dir)
- [`os::windows::fs::FileTypeExt::is_symlink_file`]
  (https://doc.rust-lang.org/stable/std/os/windows/fs/trait.FileTypeExt.html#tymethod.is_symlink_file)

These types were previously stable in `std::ffi`, but are now also
available in `core` and `alloc`:

- [`core::ffi::CStr`]
  (https://doc.rust-lang.org/stable/core/ffi/struct.CStr.html)
- [`core::ffi::FromBytesWithNulError`]
  (https://doc.rust-lang.org/stable/core/ffi/struct.FromBytesWithNulError.html)
- [`alloc::ffi::CString`]
  (https://doc.rust-lang.org/stable/alloc/ffi/struct.CString.html)
- [`alloc::ffi::FromVecWithNulError`]
  (https://doc.rust-lang.org/stable/alloc/ffi/struct.FromVecWithNulError.html)
- [`alloc::ffi::IntoStringError`]
  (https://doc.rust-lang.org/stable/alloc/ffi/struct.IntoStringError.html)
- [`alloc::ffi::NulError`]
  (https://doc.rust-lang.org/stable/alloc/ffi/struct.NulError.html)

These types were previously stable in `std::os::raw`, but are now also available in `core::ffi` and `std::ffi`:

- [`ffi::c_char`]
  (https://doc.rust-lang.org/stable/std/ffi/type.c_char.html)
- [`ffi::c_double`]
  (https://doc.rust-lang.org/stable/std/ffi/type.c_double.html)
- [`ffi::c_float`]
  (https://doc.rust-lang.org/stable/std/ffi/type.c_float.html)
- [`ffi::c_int`]
  (https://doc.rust-lang.org/stable/std/ffi/type.c_int.html)
- [`ffi::c_long`]
  (https://doc.rust-lang.org/stable/std/ffi/type.c_long.html)
- [`ffi::c_longlong`]
  (https://doc.rust-lang.org/stable/std/ffi/type.c_longlong.html)
- [`ffi::c_schar`]
  (https://doc.rust-lang.org/stable/std/ffi/type.c_schar.html)
- [`ffi::c_short`]
  (https://doc.rust-lang.org/stable/std/ffi/type.c_short.html)
- [`ffi::c_uchar`]
  (https://doc.rust-lang.org/stable/std/ffi/type.c_uchar.html)
- [`ffi::c_uint`]
  (https://doc.rust-lang.org/stable/std/ffi/type.c_uint.html)
- [`ffi::c_ulong`]
  (https://doc.rust-lang.org/stable/std/ffi/type.c_ulong.html)
- [`ffi::c_ulonglong`]
  (https://doc.rust-lang.org/stable/std/ffi/type.c_ulonglong.html)
- [`ffi::c_ushort`]
  (https://doc.rust-lang.org/stable/std/ffi/type.c_ushort.html)

These APIs are now usable in const contexts:

- [`slice::from_raw_parts`]
  (https://doc.rust-lang.org/stable/core/slice/fn.from_raw_parts.html)

Cargo
-----
- [Packages can now inherit settings from the workspace so that
  the settings can be centralized in one place.]
  (rust-lang/cargo#10859) See
  [`workspace.package`](https://doc.rust-lang.org/nightly/cargo/reference/workspaces.html#the-workspacepackage-table)
  and
  [`workspace.dependencies`](https://doc.rust-lang.org/nightly/cargo/reference/workspaces.html#the-workspacedependencies-table)
  for more details on how to define these common settings.
- [Cargo commands can now accept multiple `--target` flags to build
  for multiple targets at once]
  (rust-lang/cargo#10766), and the
  [`build.target`](https://doc.rust-lang.org/nightly/cargo/reference/config.html#buildtarget)
  config option may now take an array of multiple targets.
- [The `--jobs` argument can now take a negative number to count
  backwards from the max CPUs.]
  (rust-lang/cargo#10844)
- [`cargo add` will now update `Cargo.lock`.]
  (rust-lang/cargo#10902)
- [Added](rust-lang/cargo#10838) the
  [`--crate-type`](https://doc.rust-lang.org/nightly/cargo/commands/cargo-rustc.html#option-cargo-rustc---crate-type)
  flag to `cargo rustc` to override the crate type.
- [Significantly improved the performance fetching git dependencies from GitHub
  when using a hash in the `rev` field.]
  (rust-lang/cargo#10079)

Misc
----
- [The `rust-analyzer` rustup component is now available on the stable channel.]
  (rust-lang/rust#98640)

Compatibility Notes
-------------------
- The minimum required versions for all `-linux-gnu` targets are
  now at least kernel 3.2 and glibc 2.17, for targets that previously
  supported older versions: [Increase the minimum linux-gnu
  versions](rust-lang/rust#95026)
- [Network primitives are now implemented with the ideal Rust
  layout, not the C system layout]
  (rust-lang/rust#78802). This can
  cause problems when transmuting the types.
- [Add assertion that `transmute_copy`'s `U` is not larger than `T`]
  (rust-lang/rust#98839)
- [A soundness bug in `BTreeMap` was fixed]
  (rust-lang/rust#99413) that allowed data
  it was borrowing to be dropped before the container.
- [The Drop behavior of C-like enums cast to ints has changed]
  (rust-lang/rust#96862). These are already
  discouraged by a compiler warning.
- [Relate late-bound closure lifetimes to parent fn in NLL]
  (rust-lang/rust#98835)
- [Errors at const-eval time are now in future incompatibility reports]
  (rust-lang/rust#97743)
- On the `thumbv6m-none-eabi` target, some incorrect `asm!` statements
  were erroneously accepted if they used the high registers (r8 to
  r14) as an input/output operand. [This is no longer accepted]
  (rust-lang/rust#99155).
- [`impl Trait` was accidentally accepted as the associated type
  value of return-position `impl Trait`]
  (rust-lang/rust#97346), without
  fulfilling all the trait bounds of that associated type, as long
  as the hidden type satisfies said bounds. This has been fixed.

Internal Changes
----------------

These changes do not affect any public interfaces of Rust, but they
represent significant improvements to the performance or internals
of rustc and related tools.

- Windows builds now use profile-guided optimization, providing
  10-20% improvements to compiler performance: [Utilize PGO for
  windows x64 rustc dist builds]
  (rust-lang/rust#96978)
- [Stop keeping metadata in memory before writing it to disk]
  (rust-lang/rust#96544)
- [compiletest: strip debuginfo by default for mode=ui]
  (rust-lang/rust#98140)
- Many improvements to generated code for derives, including
  performance improvements:
  - [Don't use match-destructuring for derived ops on structs.]
    (rust-lang/rust#98446)
  - [Many small deriving cleanups]
    (rust-lang/rust#98741)
  - [More derive output improvements]
    (rust-lang/rust#98758)
  - [Clarify deriving code](rust-lang/rust#98915)
  - [Final derive output improvements]
    (rust-lang/rust#99046)
  - [Stop injecting `#[allow(unused_qualifications)]` in generated
    `derive` implementations](rust-lang/rust#99485)
  - [Improve `derive(Debug)`](rust-lang/rust#98190)
- [Bump to clap 3](rust-lang/rust#98213)
- [fully move dropck to mir](rust-lang/rust#98641)
- [Optimize `Vec::insert` for the case where `index == len`.]
  (rust-lang/rust#98755)
- [Convert rust-analyzer to an in-tree tool]
  (rust-lang/rust#99603)
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Nov 16, 2022
Pkgsrc changes:
 * This package now contains rust-analyzer, so implicitly
   conflicts with that pkgsrc package.  The same goes for
   the rust-src package.
 * Add NetBSD/arm6 port
 * Add unfinished NetBSD/mipsel port
 * Revert the use of the internal LLVM,
   should now build with the new pkgsrc LLVM (15).
 * Add depndence on compat80 for sparc64 to fix the build
 * Adapt patches
 * Add CHECK_INTERPRETER_SKIP for a few (mostly unused) files.
   (A proper fix may come later.)

Upstream changes:

Version 1.64.0 (2022-09-22)
===========================

Language
--------
- [Unions with mutable references or tuples of allowed types are
  now allowed](rust-lang/rust#97995)

- It is now considered valid to deallocate memory pointed to by a
  shared reference `&T` [if every byte in `T` is inside an
  `UnsafeCell`](rust-lang/rust#98017)

- Unused tuple struct fields are now warned against in an
  allow-by-default lint, [`unused_tuple_struct_fields`]
  (rust-lang/rust#95977), similar to the
  existing warning for unused struct fields. This lint will become
  warn-by-default in the future.

Compiler
--------
- [Add Nintendo Switch as tier 3 target]
  (rust-lang/rust#88991)
  - Refer to Rust's [platform support page][platform-support-doc] for more
    information on Rust's tiered platform support.
- [Only compile `#[used]` as llvm.compiler.used for ELF targets]
  (rust-lang/rust#93718)
- [Add the `--diagnostic-width` compiler flag to define the terminal width.]
  (rust-lang/rust#95635)
- [Add support for link-flavor `rust-lld` for iOS, tvOS and watchOS]
  (rust-lang/rust#98771)

Libraries
---------
- [Remove restrictions on compare-exchange memory ordering.]
  (rust-lang/rust#98383)
- You can now `write!` or `writeln!` into an `OsString`: [Implement
  `fmt::Write` for `OsString`](rust-lang/rust#97915)
- [Make RwLockReadGuard covariant]
  (rust-lang/rust#96820)
- [Implement `FusedIterator` for `std::net::[Into]Incoming`]
  (rust-lang/rust#97300)
- [`impl<T: AsRawFd> AsRawFd for {Arc,Box}<T>`]
  (rust-lang/rust#97437)
- [`ptr::copy` and `ptr::swap` are doing untyped copies]
  (rust-lang/rust#97712)
- [Add cgroupv1 support to `available_parallelism`]
  (rust-lang/rust#97925)
- [Mitigate many incorrect uses of `mem::uninitialized`]
  (rust-lang/rust#99182)

Stabilized APIs
---------------
- [`future::IntoFuture`]
  (https://doc.rust-lang.org/stable/std/future/trait.IntoFuture.html)
- [`future::poll_fn`]
  (https://doc.rust-lang.org/stable/std/future/fn.poll_fn.html)
- [`task::ready!`]
  (https://doc.rust-lang.org/stable/std/task/macro.ready.html)
- [`num::NonZero*::checked_mul`]
  (https://doc.rust-lang.org/stable/std/num/struct.NonZeroUsize.html#method.checked_mul)
- [`num::NonZero*::checked_pow`]
  (https://doc.rust-lang.org/stable/std/num/struct.NonZeroUsize.html#method.checked_pow)
- [`num::NonZero*::saturating_mul`]
  (https://doc.rust-lang.org/stable/std/num/struct.NonZeroUsize.html#method.saturating_mul)
- [`num::NonZero*::saturating_pow`]
  (https://doc.rust-lang.org/stable/std/num/struct.NonZeroUsize.html#method.saturating_pow)
- [`num::NonZeroI*::abs`]
  (https://doc.rust-lang.org/stable/std/num/struct.NonZeroIsize.html#method.abs)
- [`num::NonZeroI*::checked_abs`]
  (https://doc.rust-lang.org/stable/std/num/struct.NonZeroIsize.html#method.checked_abs)
- [`num::NonZeroI*::overflowing_abs`]
  (https://doc.rust-lang.org/stable/std/num/struct.NonZeroIsize.html#method.overflowing_abs)
- [`num::NonZeroI*::saturating_abs`]
  (https://doc.rust-lang.org/stable/std/num/struct.NonZeroIsize.html#method.saturating_abs)
- [`num::NonZeroI*::unsigned_abs`]
  (https://doc.rust-lang.org/stable/std/num/struct.NonZeroIsize.html#method.unsigned_abs)
- [`num::NonZeroI*::wrapping_abs`]
  (https://doc.rust-lang.org/stable/std/num/struct.NonZeroIsize.html#method.wrapping_abs)
- [`num::NonZeroU*::checked_add`]
  (https://doc.rust-lang.org/stable/std/num/struct.NonZeroUsize.html#method.checked_add)
- [`num::NonZeroU*::checked_next_power_of_two`]
  (https://doc.rust-lang.org/stable/std/num/struct.NonZeroUsize.html#method.checked_next_power_of_two)
- [`num::NonZeroU*::saturating_add`]
  (https://doc.rust-lang.org/stable/std/num/struct.NonZeroUsize.html#method.saturating_add)
- [`os::unix::process::CommandExt::process_group`]
  (https://doc.rust-lang.org/stable/std/os/unix/process/trait.CommandExt.html#tymethod.process_group)
- [`os::windows::fs::FileTypeExt::is_symlink_dir`]
  (https://doc.rust-lang.org/stable/std/os/windows/fs/trait.FileTypeExt.html#tymethod.is_symlink_dir)
- [`os::windows::fs::FileTypeExt::is_symlink_file`]
  (https://doc.rust-lang.org/stable/std/os/windows/fs/trait.FileTypeExt.html#tymethod.is_symlink_file)

These types were previously stable in `std::ffi`, but are now also
available in `core` and `alloc`:

- [`core::ffi::CStr`]
  (https://doc.rust-lang.org/stable/core/ffi/struct.CStr.html)
- [`core::ffi::FromBytesWithNulError`]
  (https://doc.rust-lang.org/stable/core/ffi/struct.FromBytesWithNulError.html)
- [`alloc::ffi::CString`]
  (https://doc.rust-lang.org/stable/alloc/ffi/struct.CString.html)
- [`alloc::ffi::FromVecWithNulError`]
  (https://doc.rust-lang.org/stable/alloc/ffi/struct.FromVecWithNulError.html)
- [`alloc::ffi::IntoStringError`]
  (https://doc.rust-lang.org/stable/alloc/ffi/struct.IntoStringError.html)
- [`alloc::ffi::NulError`]
  (https://doc.rust-lang.org/stable/alloc/ffi/struct.NulError.html)

These types were previously stable in `std::os::raw`, but are now
also available in `core::ffi` and `std::ffi`:

- [`ffi::c_char`]
  (https://doc.rust-lang.org/stable/std/ffi/type.c_char.html)
- [`ffi::c_double`]
  (https://doc.rust-lang.org/stable/std/ffi/type.c_double.html)
- [`ffi::c_float`]
  (https://doc.rust-lang.org/stable/std/ffi/type.c_float.html)
- [`ffi::c_int`]
  (https://doc.rust-lang.org/stable/std/ffi/type.c_int.html)
- [`ffi::c_long`]
  (https://doc.rust-lang.org/stable/std/ffi/type.c_long.html)
- [`ffi::c_longlong`]
  (https://doc.rust-lang.org/stable/std/ffi/type.c_longlong.html)
- [`ffi::c_schar`]
  (https://doc.rust-lang.org/stable/std/ffi/type.c_schar.html)
- [`ffi::c_short`]
  (https://doc.rust-lang.org/stable/std/ffi/type.c_short.html)
- [`ffi::c_uchar`]
  (https://doc.rust-lang.org/stable/std/ffi/type.c_uchar.html)
- [`ffi::c_uint`]
  (https://doc.rust-lang.org/stable/std/ffi/type.c_uint.html)
- [`ffi::c_ulong`]
  (https://doc.rust-lang.org/stable/std/ffi/type.c_ulong.html)
- [`ffi::c_ulonglong`]
  (https://doc.rust-lang.org/stable/std/ffi/type.c_ulonglong.html)
- [`ffi::c_ushort`]
  (https://doc.rust-lang.org/stable/std/ffi/type.c_ushort.html)

These APIs are now usable in const contexts:

- [`slice::from_raw_parts`]
  (https://doc.rust-lang.org/stable/core/slice/fn.from_raw_parts.html)

Cargo
-----
- [Packages can now inherit settings from the workspace so that
  the settings can be centralized in one place.]
  (rust-lang/cargo#10859) See
  [`workspace.package`](https://doc.rust-lang.org/nightly/cargo/reference/workspaces.html#the-workspacepackage-table)
  and
  [`workspace.dependencies`](https://doc.rust-lang.org/nightly/cargo/reference/workspaces.html#the-workspacedependencies-table)
  for more details on how to define these common settings.
- [Cargo commands can now accept multiple `--target` flags to build
  for multiple targets at once]
  (rust-lang/cargo#10766), and the
  [`build.target`](https://doc.rust-lang.org/nightly/cargo/reference/config.html#buildtarget)
  config option may now take an array of multiple targets.
- [The `--jobs` argument can now take a negative number to count
  backwards from the max CPUs.]
  (rust-lang/cargo#10844)
- [`cargo add` will now update `Cargo.lock`.]
  (rust-lang/cargo#10902)
- [Added](rust-lang/cargo#10838) the
  [`--crate-type`](https://doc.rust-lang.org/nightly/cargo/commands/cargo-rustc.html#option-cargo-rustc---crate-type)
  flag to `cargo rustc` to override the crate type.
- [Significantly improved the performance fetching git dependencies from GitHub
  when using a hash in the `rev` field.]
  (rust-lang/cargo#10079)

Misc
----
- [The `rust-analyzer` rustup component is now available on the stable channel.]
  (rust-lang/rust#98640)

Compatibility Notes
-------------------
- The minimum required versions for all `-linux-gnu` targets are
  now at least kernel 3.2 and glibc 2.17, for targets that previously
  supported older versions: [Increase the minimum linux-gnu
  versions](rust-lang/rust#95026)
- [Network primitives are now implemented with the ideal Rust
  layout, not the C system layout]
  (rust-lang/rust#78802). This can
  cause problems when transmuting the types.
- [Add assertion that `transmute_copy`'s `U` is not larger than `T`]
  (rust-lang/rust#98839)
- [A soundness bug in `BTreeMap` was fixed]
  (rust-lang/rust#99413) that allowed data
  it was borrowing to be dropped before the container.
- [The Drop behavior of C-like enums cast to ints has changed]
  (rust-lang/rust#96862). These are already
  discouraged by a compiler warning.
- [Relate late-bound closure lifetimes to parent fn in NLL]
  (rust-lang/rust#98835)
- [Errors at const-eval time are now in future incompatibility reports]
  (rust-lang/rust#97743)
- On the `thumbv6m-none-eabi` target, some incorrect `asm!` statements
  were erroneously accepted if they used the high registers (r8 to
  r14) as an input/output operand. [This is no longer accepted]
  (rust-lang/rust#99155).
- [`impl Trait` was accidentally accepted as the associated type
  value of return-position `impl Trait`]
  (rust-lang/rust#97346), without
  fulfilling all the trait bounds of that associated type, as long
  as the hidden type satisfies said bounds. This has been fixed.

Internal Changes
----------------

These changes do not affect any public interfaces of Rust, but they
represent significant improvements to the performance or internals
of rustc and related tools.

- Windows builds now use profile-guided optimization, providing
  10-20% improvements to compiler performance: [Utilize PGO for
  windows x64 rustc dist builds]
  (rust-lang/rust#96978)
- [Stop keeping metadata in memory before writing it to disk]
  (rust-lang/rust#96544)
- [compiletest: strip debuginfo by default for mode=ui]
  (rust-lang/rust#98140)
- Many improvements to generated code for derives, including
  performance improvements:
  - [Don't use match-destructuring for derived ops on structs.]
    (rust-lang/rust#98446)
  - [Many small deriving cleanups]
    (rust-lang/rust#98741)
  - [More derive output improvements]
    (rust-lang/rust#98758)
  - [Clarify deriving code](rust-lang/rust#98915)
  - [Final derive output improvements]
    (rust-lang/rust#99046)
  - [Stop injecting `#[allow(unused_qualifications)]` in generated
    `derive` implementations](rust-lang/rust#99485)
  - [Improve `derive(Debug)`](rust-lang/rust#98190)
- [Bump to clap 3](rust-lang/rust#98213)
- [fully move dropck to mir](rust-lang/rust#98641)
- [Optimize `Vec::insert` for the case where `index == len`.]
  (rust-lang/rust#98755)
- [Convert rust-analyzer to an in-tree tool]
  (rust-lang/rust#99603)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
relnotes Marks issues that should be documented in the release notes of the next release. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-libs Relevant to the library team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants