Auto merge of rust-lang#110865 - cuviper:ct-ng-1.25, r=pietroalbini

ci: upgrade and refactor crosstool-ng builders

The first commit upgrades our builders from crosstool-ng 1.24.0 to 1.25.0. There are otherwise no changes intended to the toolchains we're using, but there are some minor version upgrades as a result, especially GCC 8.3.0 to 8.5.0. The newer crosstool-ng will position us well to make toolchain upgrades in the future though, as we were maxed out before and it now goes up to GCC 11.

The second commit refactors our config management to only commit a "mini-defconfig" for each target, produced by `ct-ng savedefconfig`. This makes it much clearer which settings we're actually changing, and also makes it easier to ensure consistency for things like mirror management.

src/ci/docker/README.md

@@ -83,12 +83,12 @@ To run the image,
 A number of these images take quite a long time to compile as they're building
 whole gcc toolchains to do cross builds with. Much of this is relatively
 self-explanatory but some images use [crosstool-ng] which isn't quite as self
-explanatory. Below is a description of where these `*.config` files come form,
+explanatory. Below is a description of where these `*.defconfig` files come form,
 how to generate them, and how the existing ones were generated.
 
 [crosstool-ng]: https://github.com/crosstool-ng/crosstool-ng
 
-### Generating a `.config` file
+### Generating a `.defconfig` file
 
 **NOTE:** Existing Dockerfiles can also be a good guide for the process and order
 of script execution.
@@ -100,14 +100,14 @@ next two steps.
   these steps are outside the container:
 
 ```
-# Note: We use ubuntu:16.04 because that's the "base" of linux-cross Docker
+# Note: We use ubuntu:22.04 because that's the "base" of linux-cross Docker
 # image, or simply run ./src/ci/docker/run.sh once, which will download the correct
 # one and you can check it out with `docker images`
-$ docker run -it ubuntu:16.04 bash
+$ docker run -it ubuntu:22.04 bash
 # in another terminal:
 $ docker ps
 CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
-cfbec05ed730        ubuntu:16.04        "bash"              16 seconds ago      Up 15 seconds                           drunk_murdock
+cfbec05ed730        ubuntu:22.04        "bash"              16 seconds ago      Up 15 seconds                           drunk_murdock
 $ docker cp src/ci/docker/scripts drunk_murdock:/tmp/
 ```
 
@@ -127,7 +127,7 @@ $ bash ./crosstool-ng.sh
   present. Otherwise one can use the TUI to load any config-file.
 
 ```
-$ docker cp arm-linux-gnueabi.config drunk_murdock:/tmp/.config
+$ docker cp arm-linux-gnueabi.defconfig drunk_murdock:/tmp/.config
 ```
 
 - Now, inside the container run the following command to configure the
@@ -136,66 +136,73 @@ $ docker cp arm-linux-gnueabi.config drunk_murdock:/tmp/.config
 
 ```
 $ cd /tmp/
+$ ct-ng olddefconfig
 $ ct-ng menuconfig
+$ ct-ng savedefconfig
 ```
 
-- Finally, we retrieve the `.config` file from the container and give it a
+- Finally, we retrieve the `defconfig` file from the container and give it a
   meaningful name. This is done outside the container.
 
 ```
-$ docker cp drunk_murdock:/tmp/.config arm-linux-gnueabi.config
+$ docker cp drunk_murdock:/tmp/defconfig arm-linux-gnueabi.defconfig
 ```
 
 - Now you can shutdown the container or repeat the two last steps to generate a
-  new `.config` file.
+  new `.defconfig` file.
 
 ### Toolchain configuration
 
 Changes on top of the default toolchain configuration used to generate the
-`.config` files in this directory. The changes are formatted as follows:
+`.defconfig` files in this directory. The changes are formatted as follows:
 
 ```
 $category > $option = $value -- $comment
 ```
 
-### `arm-linux-gnueabi.config`
+### `arm-linux-gnueabi.defconfig`
 
 For targets: `arm-unknown-linux-gnueabi`
 
 - Path and misc options > Prefix directory = /x-tools/${CT\_TARGET}
-- Path and misc options > Patches origin = Bundled only
+- Path and misc options > Use a mirror = ENABLE
+- Path and misc options > Base URL = https://ci-mirrors.rust-lang.org/rustc
 - Target options > Target Architecture = arm
 - Target options > Architecture level = armv6 -- (+)
 - Target options > Floating point = software (no FPU) -- (\*)
 - Operating System > Target OS = linux
 - Operating System > Linux kernel version = 3.2.101
+- Binary utilities > Version of binutils = 2.32
 - C-library > glibc version = 2.17.0
-- C compiler > gcc version = 8.3.0
+- C compiler > gcc version = 8.5.0
 - C compiler > C++ = ENABLE -- to cross compile LLVM
 
-### `arm-linux-gnueabihf.config`
+### `arm-linux-gnueabihf.defconfig`
 
 For targets: `arm-unknown-linux-gnueabihf`
 
 - Path and misc options > Prefix directory = /x-tools/${CT\_TARGET}
-- Path and misc options > Patches origin = Bundled only
+- Path and misc options > Use a mirror = ENABLE
+- Path and misc options > Base URL = https://ci-mirrors.rust-lang.org/rustc
 - Target options > Target Architecture = arm
 - Target options > Architecture level = armv6 -- (+)
 - Target options > Use specific FPU = vfp -- (+)
 - Target options > Floating point = hardware (FPU) -- (\*)
 - Target options > Default instruction set mode = arm -- (+)
 - Operating System > Target OS = linux
 - Operating System > Linux kernel version = 3.2.101
+- Binary utilities > Version of binutils = 2.32
 - C-library > glibc version = 2.17.0
-- C compiler > gcc version = 8.3.0
+- C compiler > gcc version = 8.5.0
 - C compiler > C++ = ENABLE -- to cross compile LLVM
 
-### `armv7-linux-gnueabihf.config`
+### `armv7-linux-gnueabihf.defconfig`
 
 For targets: `armv7-unknown-linux-gnueabihf`
 
 - Path and misc options > Prefix directory = /x-tools/${CT\_TARGET}
-- Path and misc options > Patches origin = Bundled only
+- Path and misc options > Use a mirror = ENABLE
+- Path and misc options > Base URL = https://ci-mirrors.rust-lang.org/rustc
 - Target options > Target Architecture = arm
 - Target options > Suffix to the arch-part = v7
 - Target options > Architecture level = armv7-a -- (+)
@@ -204,8 +211,9 @@ For targets: `armv7-unknown-linux-gnueabihf`
 - Target options > Default instruction set mode = thumb -- (\*)
 - Operating System > Target OS = linux
 - Operating System > Linux kernel version = 3.2.101
+- Binary utilities > Version of binutils = 2.32
 - C-library > glibc version = 2.17.0
-- C compiler > gcc version = 8.3.0
+- C compiler > gcc version = 8.5.0
 - C compiler > C++ = ENABLE -- to cross compile LLVM
 
 (\*) These options have been selected to match the configuration of the arm
@@ -214,7 +222,7 @@ For targets: `armv7-unknown-linux-gnueabihf`
     libraries like jemalloc. See the mk/cfg/arm(v7)-unknown-linux-gnueabi{,hf}.mk
     file in Rust's source code.
 
-### `aarch64-linux-gnu.config`
+### `aarch64-linux-gnu.defconfig`
 
 For targets: `aarch64-unknown-linux-gnu`
 
@@ -225,17 +233,16 @@ For targets: `aarch64-unknown-linux-gnu`
 - Target options > Bitness = 64-bit
 - Operating System > Target OS = linux
 - Operating System > Linux kernel version = 4.1.49
-- Binary utilities > Version of binutils = 2.32
+- Binary utilities > Version of binutils = 2.29.1
 - C-library > glibc version = 2.17 -- aarch64 support was introduced in this version
-- C compiler > gcc version = 8.3.0
+- C compiler > gcc version = 8.5.0
 - C compiler > C++ = ENABLE -- to cross compile LLVM
 
-### `i586-linux-gnu.config`
+### `i586-linux-gnu.defconfig`
 
 For targets: `i586-unknown-linux-gnu`
 
 - Path and misc options > Prefix directory = /x-tools/${CT\_TARGET}
-- Path and misc options > Patches origin = Bundled only
 - Target options > Target Architecture = x86
 - Target options > Architecture level = i586
 - Target options > Target CFLAGS = -Wa,-mrelax-relocations=no
@@ -244,13 +251,13 @@ For targets: `i586-unknown-linux-gnu`
 - Binary utilities > Version of binutils = 2.32
 - Binary utilities > binutils extra config = --enable-compressed-debug-sections=none -- (\*)
 - C-library > glibc version = 2.17.0
-- C compiler > gcc version = 8.3.0
+- C compiler > gcc version = 8.5.0
 - C compiler > C++ = ENABLE
 
 (\*) Compressed debug is enabled by default for gas (assembly) on Linux/x86 targets,
      but that makes our `compiler_builtins` incompatible with binutils < 2.32.
 
-### `mips-linux-gnu.config`
+### `mips-linux-gnu.defconfig`
 
 For targets: `mips-unknown-linux-gnu`
 
@@ -265,14 +272,14 @@ For targets: `mips-unknown-linux-gnu`
 - Target options > Bitness = 32-bit
 - Target options > Architecture level = mips32r2
 - Operating System > Target OS = linux
-- Operating System > Linux kernel version = 4.4.174
+- Operating System > Linux kernel version = 4.4.302
 - Binary utilities > Version of binutils = 2.32
 - C-library > glibc version = 2.23
-- C compiler > gcc version = 8.3.0
+- C compiler > gcc version = 8.5.0
 - C compiler > gcc extra config = --with-fp-32=xx --with-odd-spreg-32=no
 - C compiler > C++ = ENABLE -- to cross compile LLVM
 
-### `mipsel-linux-gnu.config`
+### `mipsel-linux-gnu.defconfig`
 
 For targets: `mipsel-unknown-linux-gnu`
 
@@ -287,14 +294,14 @@ For targets: `mipsel-unknown-linux-gnu`
 - Target options > Bitness = 32-bit
 - Target options > Architecture level = mips32r2
 - Operating System > Target OS = linux
-- Operating System > Linux kernel version = 4.4.174
+- Operating System > Linux kernel version = 4.4.302
 - Binary utilities > Version of binutils = 2.32
 - C-library > glibc version = 2.23
-- C compiler > gcc version = 8.3.0
+- C compiler > gcc version = 8.5.0
 - C compiler > gcc extra config = --with-fp-32=xx --with-odd-spreg-32=no
 - C compiler > C++ = ENABLE -- to cross compile LLVM
 
-### `mips64-linux-gnu.config`
+### `mips64-linux-gnu.defconfig`
 
 For targets: `mips64-unknown-linux-gnuabi64`
 
@@ -309,13 +316,13 @@ For targets: `mips64-unknown-linux-gnuabi64`
 - Target options > Bitness = 64-bit
 - Target options > Architecture level = mips64r2
 - Operating System > Target OS = linux
-- Operating System > Linux kernel version = 4.4.174
+- Operating System > Linux kernel version = 4.4.302
 - Binary utilities > Version of binutils = 2.32
 - C-library > glibc version = 2.23
-- C compiler > gcc version = 8.3.0
+- C compiler > gcc version = 8.5.0
 - C compiler > C++ = ENABLE -- to cross compile LLVM
 
-### `mips64el-linux-gnu.config`
+### `mips64el-linux-gnu.defconfig`
 
 For targets: `mips64el-unknown-linux-gnuabi64`
 
@@ -330,13 +337,13 @@ For targets: `mips64el-unknown-linux-gnuabi64`
 - Target options > Bitness = 64-bit
 - Target options > Architecture level = mips64r2
 - Operating System > Target OS = linux
-- Operating System > Linux kernel version = 4.4.174
+- Operating System > Linux kernel version = 4.4.302
 - Binary utilities > Version of binutils = 2.32
 - C-library > glibc version = 2.23
-- C compiler > gcc version = 8.3.0
+- C compiler > gcc version = 8.5.0
 - C compiler > C++ = ENABLE -- to cross compile LLVM
 
-### `powerpc-linux-gnu.config`
+### `powerpc-linux-gnu.defconfig`
 
 For targets: `powerpc-unknown-linux-gnu`
 
@@ -349,10 +356,10 @@ For targets: `powerpc-unknown-linux-gnu`
 - Operating System > Linux kernel version = 3.2.101
 - Binary utilities > Version of binutils = 2.30
 - C-library > glibc version = 2.17 -- ~RHEL7 glibc
-- C compiler > gcc version = 8.3.0
+- C compiler > gcc version = 8.5.0
 - C compiler > C++ = ENABLE -- to cross compile LLVM
 
-### `powerpc64-linux-gnu.config`
+### `powerpc64-linux-gnu.defconfig`
 
 For targets: `powerpc64-unknown-linux-gnu`
 
@@ -367,12 +374,28 @@ For targets: `powerpc64-unknown-linux-gnu`
 - Operating System > Linux kernel version = 3.2.101
 - Binary utilities > Version of binutils = 2.32
 - C-library > glibc version = 2.17 -- ~RHEL7 glibc
-- C compiler > gcc version = 8.3.0
+- C compiler > gcc version = 8.5.0
 - C compiler > C++ = ENABLE -- to cross compile LLVM
 
 (+) These CPU options match the configuration of the toolchains in RHEL6.
 
-### `s390x-linux-gnu.config`
+### `riscv64-unknown-linux-gnu.defconfig`
+
+For targets: `riscv64-unknown-linux-gnu`
+
+- Path and misc options > Prefix directory = /x-tools/${CT\_TARGET}
+- Path and misc options > Use a mirror = ENABLE
+- Path and misc options > Base URL = https://ci-mirrors.rust-lang.org/rustc
+- Target options > Target Architecture = riscv
+- Target options > Bitness = 64-bit
+- Operating System > Target OS = linux
+- Operating System > Linux kernel version = 4.20.17
+- Binary utilities > Version of binutils = 2.32
+- C-library > glibc version = 2.29
+- C compiler > gcc version = 8.5.0
+- C compiler > C++ = ENABLE -- to cross compile LLVM
+
+### `s390x-linux-gnu.defconfig`
 
 For targets: `s390x-unknown-linux-gnu`
 
@@ -385,6 +408,6 @@ For targets: `s390x-unknown-linux-gnu`
 - Operating System > Linux kernel version = 3.2.101
 - Binary utilities > Version of binutils = 2.32
 - C-library > glibc version = 2.17 -- ~RHEL7 glibc
-- C compiler > gcc version = 8.3.0
+- C compiler > gcc version = 8.5.0
 - C compiler > gcc extra config = --with-arch=z10 -- LLVM's minimum support
 - C compiler > C++ = ENABLE -- to cross compile LLVM

src/ci/docker/host-x86_64/dist-aarch64-linux/Dockerfile

@@ -1,20 +1,18 @@
-FROM ubuntu:20.04
+FROM ubuntu:22.04
 
 COPY scripts/cross-apt-packages.sh /scripts/
 RUN sh /scripts/cross-apt-packages.sh
 
-COPY scripts/crosstool-ng-1.24.sh /scripts/
-RUN sh /scripts/crosstool-ng-1.24.sh
+COPY scripts/crosstool-ng.sh /scripts/
+RUN sh /scripts/crosstool-ng.sh
 
 COPY scripts/rustbuild-setup.sh /scripts/
 RUN sh /scripts/rustbuild-setup.sh
-USER rustbuild
 WORKDIR /tmp
 
-COPY host-x86_64/dist-aarch64-linux/aarch64-linux-gnu.config host-x86_64/dist-aarch64-linux/build-toolchains.sh /tmp/
-RUN ./build-toolchains.sh
-
-USER root
+COPY scripts/crosstool-ng-build.sh /scripts/
+COPY host-x86_64/dist-aarch64-linux/aarch64-linux-gnu.defconfig /tmp/crosstool.defconfig
+RUN /scripts/crosstool-ng-build.sh
 
 COPY scripts/sccache.sh /scripts/
 RUN sh /scripts/sccache.sh