CI: Update riscv64gc-linux job to Ubuntu 22.04, rename to riscv64gc-gnu

[Hoverbear]

Together with [email protected], we've been starting to explore improving the state of the riscv64gc-unknown-linux-gnu target. Additionally, I'm looking to add support for this platform in Ferrocene (Related PR).

There currently exists a src/ci/docker/host-x86_64/disabled/riscv64gc-linux job for the CI, however it is currently experiencing errors.

Errors

$ DEPLOY=1 ./src/ci/docker/run.sh riscv64gc-linux
# ...
[RUSTC-TIMING] addr2line test:false 0.371
[RUSTC-TIMING] gimli test:false 3.159
[RUSTC-TIMING] object test:false 4.249
error: linking with `riscv64-linux-gnu-gcc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/bin:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/bin:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" VSLANG="1033" "riscv64-linux-gnu-gcc" "-Wl,--version-script=/tmp/rustcQaIpWi/list" "-Wl,--no-undefined-version" "/tmp/rustcQaIpWi/symbols.o" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/std-ff89a9732cd5d858.std.1b5d59225ff40bd2-cgu.0.rcgu.o" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/std-ff89a9732cd5d858.dalhl7sfna1ffn4nhy6pyfa7f.rcgu.rmeta" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/std-ff89a9732cd5d858.ef0znsdf1ihn2bjkmclodhclp.rcgu.o" "-Wl,--as-needed" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/release/deps" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/build/compiler_builtins-9e9a40064e2f2bd3/out" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/riscv64gc-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libpanic_unwind-d968371aba64a26c.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libobject-da5b6473912e89d6.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libmemchr-9cfa08d2baa3643e.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libaddr2line-06e0d2153cecb6ce.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libgimli-6fdf5551cec83840.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/librustc_demangle-8ada6466f763fa2e.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libstd_detect-edc0d12d029c4c86.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libhashbrown-9c782935934c8c14.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/librustc_std_workspace_alloc-b6984e43b381efa4.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libminiz_oxide-37ee29bf49ccaa96.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libadler-591133f6804fa0f4.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libunwind-94d98075f42175f3.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libcfg_if-e267a7b9dd7af3a7.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/liblibc-503571a038f8d9fd.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/liballoc-e36c72a5cf0ee45f.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/librustc_std_workspace_core-076c2b8501e25f03.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libcore-c446fff80486d0bb.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libcompiler_builtins-26dc6b5e31e1fdb9.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/riscv64gc-unknown-linux-gnu/lib" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libstd-ff89a9732cd5d858.so" "-shared" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-nodefaultlibs" "-Wl,-z,origin" "-Wl,-rpath,$ORIGIN/../lib"
# ...
          /usr/lib/gcc-cross/riscv64-linux-gnu/9/../../../../riscv64-linux-gnu/bin/ld: failed to merge target specific data of file /checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libcompiler_builtins-26dc6b5e31e1fdb9.rlib(compiler_builtins-26dc6b5e31e1fdb9.compiler_builtins.74504a151a6bdbbf-cgu.124.rcgu.o)
          /usr/lib/gcc-cross/riscv64-linux-gnu/9/../../../../riscv64-linux-gnu/bin/ld: -march=rv64i2p1_m2p0_a2p1_f2p2_d2p2_c2p0_zicsr2p0: unsupported ISA subset `z'
          /usr/lib/gcc-cross/riscv64-linux-gnu/9/../../../../riscv64-linux-gnu/bin/ld: failed to merge target specific data of file /checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libcompiler_builtins-26dc6b5e31e1fdb9.rlib(compiler_builtins-26dc6b5e31e1fdb9.compiler_builtins.74504a151a6bdbbf-cgu.004.rcgu.o)
          collect2: error: ld returned 1 exit status
          

[RUSTC-TIMING] std test:false 15.138
error: could not compile `std` (lib) due to 1 previous error
Building bootstrap
Build completed unsuccessfully in 0:04:41
  local time: Tue May 28 16:25:09 UTC 2024
  network time: Tue, 28 May 2024 16:25:17 GMT

This PR fixes the breakage enough to get the tests running. It does so through bringing the riscv64gc-unknown-linux-gnu related test job in line with other related jobs, adopting many of the recent changes present in src/ci/docker/host-x86_64/armhf-gnu such as:

• Using Ubuntu 22.04
• Installing a more narrowly scoped package set
• Using curl instead of debootstrap to set up the root
• No longer patching busybox
• Removing the cmake.sh script related steps

Justifying Renaming riscv64gc-linux to riscv64gc-gnu

The src/ci/docker/host-x86_64/disabled/riscv64gc-linux job runs the tests for risv64gc-unknown-linux-gnu, it is based off src/ci/docker/host-x86_64/armhf-gnu.

There are other jobs that follow a $arch-gnu naming scheme:

• src/ci/docker/host-x86_64/armhf-gnu
• src/ci/docker/host-x86_64/x86_64-gnu
• src/ci/docker/host-aarch64/aarch64-gnu

It follows that the name src/ci/docker/host-x86_64/disabled/riscv64gc-linux should be src/ci/docker/host-x86_64/disabled/riscv64gc-gnu, like the others.

Testing

Note

riscv64gc-unknown-linux-gnu is a Tier 2 with Host Tools platform, all tests may not necessarily pass! There is work in #125220 which helps fix several related tests.

You can test out the renamed job:

DEPLOY=1 ./src/ci/docker/run.sh riscv64gc-gnu

DEPLOY=1 helps reproduce the CI's environment and also avoids the chance of a llvm-c/BitReader.h error (detailed in #85424 and #56650).

Sample of output (expected test failure)

$ DEPLOY=1 ./src/ci/docker/run.sh riscv64gc-gnu
# ...
test [ui] tests/ui/where-clauses/where-clause-method-substituion-rpass.rs ... ok

failures:

---- [ui] tests/ui/debuginfo/debuginfo-emit-llvm-ir-and-split-debuginfo.rs stdout ----

error: test compilation failed although it shouldn't!
status: exit status: 1
command: env -u RUSTC_LOG_COLOR RUSTC_ICE="0" RUST_BACKTRACE="short" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/tests/ui/debuginfo/debuginfo-emit-llvm-ir-and-split-debuginfo.rs" "-Zthreads=1" "-Zsimulate-remapped-rust-src-base=/rustc/FAKE_PREFIX" "-Ztranslate-remapped-path-to-local-path=no" "-Z" "ignore-directory-in-diagnostics-source-blocks=/cargo" "-Z" "ignore-directory-in-diagnostics-source-blocks=/checkout/vendor" "--sysroot" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2" "--target=riscv64gc-unknown-linux-gnu" "--check-cfg" "cfg(FALSE)" "--error-format" "json" "--json" "future-incompat" "-Ccodegen-units=1" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "-Zwrite-long-types-to-disk=no" "-Cstrip=debuginfo" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/debuginfo/debuginfo-emit-llvm-ir-and-split-debuginfo" "-A" "unused" "-A" "internal_features" "-Crpath" "-Lnative=/checkout/obj/build/riscv64gc-unknown-linux-gnu/native/rust-test-helpers" "-Clinker=riscv64-linux-gnu-gcc" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/debuginfo/debuginfo-emit-llvm-ir-and-split-debuginfo/auxiliary" "-g" "--emit=llvm-ir" "-Csplit-debuginfo=unpacked"
stdout: none
--- stderr -------------------------------
error: `-Csplit-debuginfo=unpacked` is unstable on this platform

error: aborting due to 1 previous error
------------------------------------------



failures:
    [ui] tests/ui/debuginfo/debuginfo-emit-llvm-ir-and-split-debuginfo.rs

test result: FAILED. 5 passed; 1 failed; 16897 ignored; 0 measured; 0 filtered out; finished in 410.99ms

Some tests failed in compiletest suite=ui mode=ui host=x86_64-unknown-linux-gnu target=riscv64gc-unknown-linux-gnu
  local time: Tue May 28 16:28:22 UTC 2024
  network time: Tue, 28 May 2024 16:28:30 GMT

try-job: riscv64gc-gnu

[rustbot]

r? @Mark-Simulacrum

rustbot has assigned @Mark-Simulacrum.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[Kobzol]

Hi, could you please move the corresponding files out of the disabled directory, and add them to jobs.yml? Then we can easily test the new workflow on CI (even before merging the PR). The jobs.yml file is described here.

[Kobzol]

I see you also added it to the PR job. For good measure, let's also do a try build.

@bors try

[bors]

⌛ Trying commit 3c8821f with merge edb58cc...

[bors]

💔 Test failed - checks-actions

[Hoverbear]

Absolutely!

I've not seen this particular error before.

fatal: detected dubious ownership in repository at '/checkout'
To add an exception for this directory, call:
 git config --global --add safe.directory /checkout

Normally this is an issue related to git submodules, however I don't think this PR touches submodules?

[Kobzol]

What the heck. @onur-ozkan Did you perhaps sometime encounter a bootstrap error like this?

fatal: detected dubious ownership in repository at '/checkout'
To add an exception for this directory, call:

	git config --global --add safe.directory /checkout
thread 'main' panicked at src/core/config/config.rs:2530:25:
command did not execute successfully: cd "/checkout" && "git" "rev-parse" "--show-toplevel"
expected success, got: exit status: 128
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Build completed unsuccessfully in 0:00:36
make: *** [Makefile:93: prepare] Error 1

[onur-ozkan]

What the heck. @onur-ozkan Did you perhaps sometime encounter a bootstrap error like this?

fatal: detected dubious ownership in repository at '/checkout'
To add an exception for this directory, call:

	git config --global --add safe.directory /checkout
thread 'main' panicked at src/core/config/config.rs:2530:25:
command did not execute successfully: cd "/checkout" && "git" "rev-parse" "--show-toplevel"
expected success, got: exit status: 128
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Build completed unsuccessfully in 0:00:36
make: *** [Makefile:93: prepare] Error 1

No, and as far as I see that's more related with the git context in the CI environment.

[onur-ozkan]

fwiw I added this

rust/src/ci/docker/run.sh

Lines 287 to 292 in f2e1a3a

	if [ "$dev" = "1" ]
	then
	# Interactive + TTY
	args="$args -it"
	if [ $IS_GIT_SOURCE -eq 1 ]; then
	command=(/bin/bash -c 'git config --global --add safe.directory /checkout;bash')

for development purposes because I was unable to make change in the container for. iirc there are various other places too which runs this command in order to make changes on git tracked files.

[Hoverbear]

I rebased this, just in case it was some transient issue from last checkout I had.

[Hoverbear]

Well, adding @onur-ozkan 's suggestion into the Dockerfile seems to have helped (Thanks!!!!), but I don't like it!

The failure now is the one I expected to see (It's solved in #125220).

Should I move the job back to disabled?

[Kobzol]

That depends on whether you actually want this job to run on CI at this moment (which would mean that all the tests have to pass), or not :)

[Hoverbear]

Think I'll leave enabling it for when #125220 lands. I'll move this back to disabled and get this ready for (I hope!) merge. :)

[Hoverbear]

Okay, there are still more failures. 😓 I will try to re-enable this at a later date. I'd still like to get these improvements in, though!

[Kobzol]

Ok, since it's still disabled, there's no harm in merging this.

@bors r+ rollup

[bors]

📌 Commit e1b489d has been approved by Kobzol

It is now in the queue for this repository.