Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rustup panics with WSLv1 + glibc 2.31 #2245

Closed
bsekura opened this issue Feb 29, 2020 · 19 comments
Closed

rustup panics with WSLv1 + glibc 2.31 #2245

bsekura opened this issue Feb 29, 2020 · 19 comments
Labels
forwarded This issue as been forwarded "upstream"

Comments

@bsekura
Copy link

bsekura commented Feb 29, 2020

Panics like:

info: installing component 'rust-std' for 'wasm32-wasi'
thread 'main' panicked at 'assertion failed: `(left == right)`
  left: `22`,
 right: `4`', src/libstd/sys/unix/thread.rs:166:21
stack backtrace:
   0:     0x7fad0b4dbc9c - backtrace::backtrace::libunwind::trace::h65597d255cb1398b
                               at /cargo/registry/src/github.aaakk.us.kg-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/libunwind.rs:88
   1:     0x7fad0b4dbc9c - backtrace::backtrace::trace_unsynchronized::hd4f479d7150ec4a0
                               at /cargo/registry/src/github.aaakk.us.kg-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/mod.rs:66
   2:     0x7fad0b4dbc9c - std::sys_common::backtrace::_print_fmt::h015072984a2b172c

Are caused by WSLv1 + glibc 2.31 which changed the implementation of nanosleep to depend on a different kernel codepath.

Verification:

  1. WSLv1 can be confirmed using the wsl command:
wsl --list --verbose

If --verbose is not supported, then WSL v2 is not installed on the machine.
If verbose is supported, the version of WSL being used for each WSL instance will be reported against the name of the instance.

Workarounds:

  1. use older linux versions
  2. some users report that exporting RUSTUP_IO_THREADS=1 mitigates the problem
  3. use WSLv2

=====

Performing rustup update on WSL1 results in panic:

info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu'
info: latest update on 2020-02-27, rust version 1.41.1 (f3e1a954d 2020-02-24)

image

Windows 10 Pro, 1909, OS Build 18363.657

Previous updates using rustup worked without problems

@bsekura bsekura added the bug label Feb 29, 2020
@rbtcollins
Copy link
Contributor

Just did a rustup self update + rustup update in a WSL 1 instance, no issues.

robertc@lifelesstr2:~$ rustup self update
info: checking for self-updates
info: downloading self-update
info: rustup updated successfully to 1.21.1
robertc@lifelesstr2:~$ rustup update
info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu'
info: latest update on 2020-02-27, rust version 1.41.1 (f3e1a954d 2020-02-24)
info: downloading component 'rustc'
 58.0 MiB /  58.0 MiB (100 %)  11.7 MiB/s in  5s ETA:  0s
info: downloading component 'rust-std'
info: downloading component 'cargo'
info: downloading component 'rust-docs'
info: downloading component 'rls'
info: downloading component 'rust-src'
info: downloading component 'rust-analysis'
info: downloading component 'clippy'
info: downloading component 'rustfmt'
info: removing previous version of component 'rustc'
info: removing previous version of component 'rust-std'
info: removing previous version of component 'cargo'
info: removing previous version of component 'rust-docs'
info: removing previous version of component 'rls'
info: removing previous version of component 'rust-src'
info: removing previous version of component 'rust-analysis'
info: removing previous version of component 'clippy'
info: removing previous version of component 'rustfmt'
info: installing component 'rustc'
 58.0 MiB /  58.0 MiB (100 %)  10.3 MiB/s in 10s ETA:  0s
info: installing component 'rust-std'
 17.7 MiB /  17.7 MiB (100 %)  12.5 MiB/s in  1s ETA:  0s
info: installing component 'cargo'
info: installing component 'rust-docs'
 12.0 MiB /  12.0 MiB (100 %) 459.2 KiB/s in 13s ETA:  0s
info: installing component 'rls'
info: installing component 'rust-src'
info: installing component 'rust-analysis'
info: installing component 'clippy'
info: installing component 'rustfmt'
info: checking for self-updates

  stable-x86_64-unknown-linux-gnu updated - rustc 1.41.1 (f3e1a954d 2020-02-24) (from rustc 1.34.1 (fc50f328b 2019-04-24))

info: cleaning up downloads & tmp directories

So I don't think this is WSL1 per se causing the problem. Is it reproducible?

@bsekura
Copy link
Author

bsekura commented Feb 29, 2020

Yes, this happens on two different PCs (running different versions of Windows 10).
I'll keep digging, thanks.

@hideaki-t
Copy link

I have the same issue on Arch Linux on WSL1.

$ uname -rv
4.4.0-18362-Microsoft #476-Microsoft Fri Nov 01 16:53:00 PST 2019
$ ~/.cargo/bin/rustup --version
rustup 1.21.1 (7832b2ebe 2019-12-20)
$ RUST_BACKTRACE=full ~/.cargo/bin/rustup update stable
info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu'
info: latest update on 2020-02-27, rust version 1.41.1 (f3e1a954d 2020-02-24)
info: downloading component 'rust-std' for 'wasm32-wasi'
info: downloading component 'rustc'
info: downloading component 'rust-std'
info: downloading component 'cargo'
info: downloading component 'rust-docs'
info: downloading component 'rust-std' for 'aarch64-unknown-linux-musl'
info: downloading component 'rust-std' for 'armv7-unknown-linux-musleabihf'
info: downloading component 'rust-std' for 'thumbv7m-none-eabi'
info: downloading component 'rust-std' for 'armv7-unknown-linux-gnueabihf'
info: downloading component 'rust-std' for 'aarch64-unknown-linux-gnu'
info: downloading component 'rust-std' for 'wasm32-unknown-unknown'
info: downloading component 'rust-std' for 'x86_64-unknown-linux-musl'
info: downloading component 'rust-analysis'
info: downloading component 'rust-src'
info: downloading component 'rls'
info: downloading component 'llvm-tools-preview'
info: downloading component 'clippy'
info: downloading component 'rustfmt'
info: removing previous version of component 'rust-std' for 'wasm32-wasi'
info: removing previous version of component 'rustc'
info: removing previous version of component 'rust-std'
info: removing previous version of component 'cargo'
info: removing previous version of component 'rust-docs'
info: removing previous version of component 'rust-std' for 'aarch64-unknown-linux-musl'
info: removing previous version of component 'rust-std' for 'armv7-unknown-linux-musleabihf'
info: removing previous version of component 'rust-std' for 'thumbv7m-none-eabi'
info: removing previous version of component 'rust-std' for 'armv7-unknown-linux-gnueabihf'
info: removing previous version of component 'rust-std' for 'aarch64-unknown-linux-gnu'
info: removing previous version of component 'rust-std' for 'wasm32-unknown-unknown'
info: removing previous version of component 'rust-std' for 'x86_64-unknown-linux-musl'
info: removing previous version of component 'rust-analysis'
info: removing previous version of component 'rust-src'
info: removing previous version of component 'rls'
info: removing previous version of component 'llvm-tools-preview'
info: removing previous version of component 'clippy'
info: removing previous version of component 'rustfmt'
info: installing component 'rust-std' for 'wasm32-wasi'
thread 'main' panicked at 'assertion failed: `(left == right)`
  left: `22`,
 right: `4`', src/libstd/sys/unix/thread.rs:166:21
stack backtrace:
   0:     0x7fad0b4dbc9c - backtrace::backtrace::libunwind::trace::h65597d255cb1398b
                               at /cargo/registry/src/github.aaakk.us.kg-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/libunwind.rs:88
   1:     0x7fad0b4dbc9c - backtrace::backtrace::trace_unsynchronized::hd4f479d7150ec4a0
                               at /cargo/registry/src/github.aaakk.us.kg-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/mod.rs:66
   2:     0x7fad0b4dbc9c - std::sys_common::backtrace::_print_fmt::h015072984a2b172c
                               at src/libstd/sys_common/backtrace.rs:77
   3:     0x7fad0b4dbc9c - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h6df05d3335f32194
                               at src/libstd/sys_common/backtrace.rs:61
   4:     0x7fad0b1ba81c - core::fmt::write::h1f444f4312eb6c27
                               at src/libcore/fmt/mod.rs:1028
   5:     0x7fad0b4db526 - std::io::Write::write_fmt::h8d147888220078ef
                               at src/libstd/io/mod.rs:1412
   6:     0x7fad0b4db19e - std::sys_common::backtrace::_print::h8a6df0fa81d6af62
                               at src/libstd/sys_common/backtrace.rs:65
   7:     0x7fad0b4db19e - std::sys_common::backtrace::print::h6f05b4733407e509
                               at src/libstd/sys_common/backtrace.rs:50
   8:     0x7fad0b4db19e - std::panicking::default_hook::{{closure}}::h0d0a23bd02315dd8
                               at src/libstd/panicking.rs:188
   9:     0x7fad0b4da943 - std::panicking::default_hook::h8d15a9aecb4efac6
                               at src/libstd/panicking.rs:205
  10:     0x7fad0b4da943 - std::panicking::rust_panic_with_hook::hbe174577402a475d
                               at src/libstd/panicking.rs:464
  11:     0x7fad0b4da4be - std::panicking::continue_panic_fmt::h4d855dad868accf3
                               at src/libstd/panicking.rs:373
  12:     0x7fad0b4da450 - std::panicking::begin_panic_fmt::ha0f013e3301a9528
                               at src/libstd/panicking.rs:328
  13:     0x7fad0b4aae86 - <rustup::diskio::threaded::Threaded as rustup::diskio::Executor>::join::hf33124263a81d2a4
  14:     0x7fad0b49cd40 - rustup::dist::component::package::unpack_without_first_dir::h352b57d236248e9a
  15:     0x7fad0b4762f6 - rustup::dist::manifestation::Manifestation::update::h8c800deec8167b5b
  16:     0x7fad0b464b8a - rustup::dist::dist::try_update_from_dist_::h113375517e7a85ca
  17:     0x7fad0b44964c - rustup::toolchain::Toolchain::install::h048b51ffab245b48
  18:     0x7fad0b446c8c - rustup::toolchain::Toolchain::install_from_dist::h00b9aafeb93470e5
  19:     0x7fad0b105f1f - rustup_init::rustup_mode::update::hbe56e74e72e7ae31
  20:     0x7fad0b0da55b - rustup_init::rustup_mode::main::h2c97a39c05d9bf7c
  21:     0x7fad0b123b1c - rustup_init::run_rustup_inner::ha545371fd2dc19a6
  22:     0x7fad0b122d64 - rustup_init::main::hba9a23e308c96901
  23:     0x7fad0b0b7a03 - std::rt::lang_start::{{closure}}::h1778d9ce6385bef5
  24:     0x7fad0b12c458 - main
  25:     0x7fad0ac65023 - __libc_start_main
  26:     0x7fad0b0b4029 - <unknown>
info: rolling back changes
$

@bsekura
Copy link
Author

bsekura commented Mar 1, 2020

This affects Arch Linux installation on WSL. The solution is to downgrade glibc:

pacman -U /var/cache/pacman/pkg/glibc-2.30-3-x86_64.pkg.tar.xz

@rbtcollins
Copy link
Contributor

We also have some commentary from discord - its in sleep, with EINVAL from the syscall. "I'd guess that the EINVAL comes from secs going negative
if libc::nanosleep went bonkers perhaps"

@rbtcollins
Copy link
Contributor

microsoft/WSL#4898 seems to be the underlying cause; glibc has now implemented nanosleep in terms of clock_nanosleep, which depends on an unsupported WSLv1 codepath https://sourceware.org/ml/libc-alpha/2019-11/msg00155.html

@rbtcollins rbtcollins changed the title rustup panic on WSL1 rustup panic with WSLv1 + glibc 2.31 Mar 1, 2020
@rbtcollins rbtcollins added forwarded This issue as been forwarded "upstream" and removed bug labels Mar 1, 2020
@rbtcollins
Copy link
Contributor

Having slept on this I'm going to close it - we're panicing rather deep in a support library, not even in rustup code itself; the library code isn't faulty in any way, rather the runtime that we're on isn't POSIX and has fix in progress. Should they decide that they are unable to roll that fix out to older versions of windows, then it will be up to distros to make nanosleep work for the long term in WSL, as many other applications are also affected.

@steelx
Copy link

steelx commented Jun 11, 2020

Since I had existing Ubuntu installed with WSL 1. And after updating WSL 1 to WSL2 I had to do below fix. And it works now curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

wsl --set-default-version 2
wsl --list --verbose
wsl --set-version Ubuntu-20.04 2

@ustchcl
Copy link

ustchcl commented Jul 6, 2020

Since I had existing Ubuntu installed with WSL 1. And after updating WSL 1 to WSL2 I had to do below fix. And it works now curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

wsl --set-default-version 2
wsl --list --verbose
wsl --set-version Ubuntu-20.04 2

Works For Me~

@leth
Copy link

leth commented Jul 10, 2020

I accidentally discovered that RUSTUP_IO_THREADS=1 fixes things for me in WSL 1!
(I thought I was ssh-ed into raspberry pi 🤦)

I'd move to WSL2, but for some reason my desktop isn't eligible for the update yet 🤷‍♂️

@m42martin
Copy link

@leth

I accidentally discovered that RUSTUP_IO_THREADS=1 fixes things for me in WSL 1!

Confirmed. Great tip!
Win 10 1909 OS 18363.836 WSL 1.

export RUSTUP_IO_THREADS=1
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

@JosiahRooney
Copy link

@leth

I accidentally discovered that RUSTUP_IO_THREADS=1 fixes things for me in WSL 1!

Confirmed. Great tip!
Win 10 1909 OS 18363.836 WSL 1.

export RUSTUP_IO_THREADS=1
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

@m42martin worked for me! Thanks

@karlramberg
Copy link

@m42martin worked for me as well! Thank you so much

@icorbrey
Copy link

@m42martin Seems to work perfectly, thank you so much!

@mark-i-m
Copy link
Member

Btw, I was running into this with the rustup installer on WSL1, and the threads trick worked. But I just now did rustup install nightly on WSL2 and it worked fine.

@broander
Copy link

Worked for me too, great!

@tommyZZM
Copy link

tommyZZM commented Aug 9, 2020

paritytech/subport#27 (comment)

For WSL1 add this to the .bash_profile or .bashrc, works for me

export RUSTUP_IO_THREADS=1

@PLQin
Copy link

PLQin commented Aug 20, 2020

@m42martin Solved my problem. Cool!

@rbtcollins
Copy link
Contributor

This bug is long closed; I'm locking it so we don't spam everyone that has ever commented with further notifications.

If:

  • You have a WSL1 installation that you cannot get working after trying everything in this bug and in the linked bugs, please do file a bug. But only after trying downgrading libc, or running with threading disabled.
  • You have gotten WSL1 to work, great, carry on, no need to tell anyone.
  • You have an issue with WSL2, please file a new bug. Unless it looks like this, in which case, follow the advice here for WSL1.
  • You have an a different issue, please file a new bug.

@rust-lang rust-lang locked as resolved and limited conversation to collaborators Aug 21, 2020
@rbtcollins rbtcollins changed the title rustup panic with WSLv1 + glibc 2.31 rustup panics with WSLv1 + glibc 2.31 Aug 21, 2020
@kinnison kinnison unpinned this issue Nov 1, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
forwarded This issue as been forwarded "upstream"
Projects
None yet
Development

No branches or pull requests