Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check tier 3 targets in CI #109099

Open
jyn514 opened this issue Mar 13, 2023 · 7 comments
Open

Check tier 3 targets in CI #109099

jyn514 opened this issue Mar 13, 2023 · 7 comments
Labels
E-hard Call for participation: Hard difficulty. Experience needed to fix: A lot. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. T-libs Relevant to the library team, which will review and decide on the PR/issue.

Comments

@jyn514
Copy link
Member

jyn514 commented Mar 13, 2023

NOTE: this issue does not affect the documented target tier policy of the Rust project. We reserve the right to break tier 3 targets at any time, according to the guidelines at https://doc.rust-lang.org/nightly/rustc/target-tier-policy.html#tier-3-target-policy.

Currently, maintaining std for tier 3 targets is very painful for t-libs: std::sys is highly platform dependent, and we have no tier 2 targets that use sys::unsupported so it's hard to tell if it even compiles. To reduce the maintenance burden, I propose we start running x check --target foo for each tier 3 target, to make sure it compiles.

Note that this does not involve distributing any release artifacts; users wanting to target tier 3 platforms will still need to build rustc from source or use -Zbuild-std, but it's more likely the standard library will at least compile, even if we make no guarantees about whether it will run correctly.

We have a few options about how to test this:

  1. Run this on PR CI only, which acts as a janky "allow-fail" action.
  2. Run this in bors merges, which requires it to pass for the PR to merge. To allow the target to fail to compile, we'd manually disable it in the src/ci test runner file.
  3. Add an allow-fail action which runs on bors merges, which doesn't block the merge but still lets us know when the target fails to build. I'm a little hesitant about this because I think it's rare people will actually look at the CI for the master branch, but maybe it's good enough.

I suggest that we wait to decide until the PR fixing these targets, since that will let us know the extent of the current breakage, and gives us a rough estimate of how much effort it will be to keep these compiling (option 2).

I'm very curious to hear @rust-lang/libs's opinion on this, particularly on the testing strategy and whether they think this will increase or decrease the maintenance burden.

cc https://rust-lang.zulipchat.com/#narrow/stream/242791-t-infra/topic/.60x.20check.60.20for.20tier.203.20targets


Mentoring instructions:

  1. Add a new CI job which runs x check --stage 1 for each tier 3 platform.
    target | std | host | notes
    -------|:---:|:----:|-------
    `aarch64-apple-ios-macabi` | ? | | Apple Catalyst on ARM64
    `aarch64-apple-tvos` | * | | ARM64 tvOS
    [`aarch64-apple-watchos-sim`](platform-support/apple-watchos.md) | ✓ | | ARM64 Apple WatchOS Simulator
    [`aarch64-kmc-solid_asp3`](platform-support/kmc-solid.md) | ✓ | | ARM64 SOLID with TOPPERS/ASP3
    [`aarch64-nintendo-switch-freestanding`](platform-support/aarch64-nintendo-switch-freestanding.md) | * | | ARM64 Nintendo Switch, Horizon
    [`aarch64-pc-windows-gnullvm`](platform-support/pc-windows-gnullvm.md) | ✓ | ✓ |
    [`aarch64-unknown-nto-qnx710`](platform-support/nto-qnx.md) | ? | | ARM64 QNX Neutrino 7.1 RTOS |
    `aarch64-unknown-freebsd` | ✓ | ✓ | ARM64 FreeBSD
    `aarch64-unknown-hermit` | ✓ | | ARM64 HermitCore
    `aarch64-unknown-linux-gnu_ilp32` | ✓ | ✓ | ARM64 Linux (ILP32 ABI)
    `aarch64-unknown-netbsd` | ✓ | ✓ |
    [`aarch64-unknown-openbsd`](platform-support/openbsd.md) | ✓ | ✓ | ARM64 OpenBSD
    `aarch64-unknown-redox` | ? | | ARM64 Redox OS
    `aarch64-uwp-windows-msvc` | ? | |
    `aarch64-wrs-vxworks` | ? | |
    `aarch64_be-unknown-linux-gnu_ilp32` | ✓ | ✓ | ARM64 Linux (big-endian, ILP32 ABI)
    `aarch64_be-unknown-linux-gnu` | ✓ | ✓ | ARM64 Linux (big-endian)
    [`arm64_32-apple-watchos`](platform-support/apple-watchos.md) | ✓ | | ARM Apple WatchOS 64-bit with 32-bit pointers
    [`armeb-unknown-linux-gnueabi`](platform-support/armeb-unknown-linux-gnueabi.md) | ✓ | ? | ARM BE8 the default ARM big-endian architecture since [ARMv6](https://developer.arm.com/documentation/101754/0616/armlink-Reference/armlink-Command-line-Options/--be8?lang=en).
    `armv4t-none-eabi` | * | | ARMv4T A32
    `armv4t-unknown-linux-gnueabi` | ? | |
    [`armv5te-none-eabi`](platform-support/armv5te-none-eabi.md) | * | | ARMv5TE A32
    `armv5te-unknown-linux-uclibceabi` | ? | | ARMv5TE Linux with uClibc
    `armv6-unknown-freebsd` | ✓ | ✓ | ARMv6 FreeBSD
    `armv6-unknown-netbsd-eabihf` | ? | |
    [`armv6k-nintendo-3ds`](platform-support/armv6k-nintendo-3ds.md) | ? | | ARMv6K Nintendo 3DS, Horizon (Requires devkitARM toolchain)
    `armv7-apple-ios` | ✓ | | ARMv7 iOS, Cortex-a8
    [`armv7-sony-vita-newlibeabihf`](platform-support/armv7-sony-vita-newlibeabihf.md) | ? | | ARM Cortex-A9 Sony PlayStation Vita (requires VITASDK toolchain)
    [`armv7-unknown-linux-uclibceabi`](platform-support/armv7-unknown-linux-uclibceabi.md) | ✓ | ✓ | ARMv7 Linux with uClibc, softfloat
    [`armv7-unknown-linux-uclibceabihf`](platform-support/armv7-unknown-linux-uclibceabihf.md) | ✓ | ? | ARMv7 Linux with uClibc, hardfloat
    `armv7-unknown-freebsd` | ✓ | ✓ | ARMv7 FreeBSD
    `armv7-unknown-netbsd-eabihf` | ✓ | ✓ |
    `armv7-wrs-vxworks-eabihf` | ? | |
    [`armv7a-kmc-solid_asp3-eabi`](platform-support/kmc-solid.md) | ✓ | | ARM SOLID with TOPPERS/ASP3
    [`armv7a-kmc-solid_asp3-eabihf`](platform-support/kmc-solid.md) | ✓ | | ARM SOLID with TOPPERS/ASP3, hardfloat
    `armv7a-none-eabihf` | * | | ARM Cortex-A, hardfloat
    [`armv7k-apple-watchos`](platform-support/apple-watchos.md) | ✓ | | ARM Apple WatchOS
    `armv7s-apple-ios` | ✓ | |
    `avr-unknown-gnu-atmega328` | * | | AVR. Requires `-Z build-std=core`
    `bpfeb-unknown-none` | * | | BPF (big endian)
    `bpfel-unknown-none` | * | | BPF (little endian)
    `hexagon-unknown-linux-musl` | ? | |
    `i386-apple-ios` | ✓ | | 32-bit x86 iOS
    `i686-apple-darwin` | ✓ | ✓ | 32-bit macOS (10.7+, Lion+)
    `i686-pc-windows-msvc` | * | | 32-bit Windows XP support
    `i686-unknown-haiku` | ✓ | ✓ | 32-bit Haiku
    `i686-unknown-netbsd` | ✓ | ✓ | NetBSD/i386 with SSE2
    [`i686-unknown-openbsd`](platform-support/openbsd.md) | ✓ | ✓ | 32-bit OpenBSD
    `i686-uwp-windows-gnu` | ? | |
    `i686-uwp-windows-msvc` | ? | |
    `i686-wrs-vxworks` | ? | |
    [`m68k-unknown-linux-gnu`](platform-support/m68k-unknown-linux-gnu.md) | ? | | Motorola 680x0 Linux
    `mips-unknown-linux-uclibc` | ✓ | | MIPS Linux with uClibc
    [`mips64-openwrt-linux-musl`](platform-support/mips64-openwrt-linux-musl.md) | ? | | MIPS64 for OpenWrt Linux MUSL
    `mipsel-sony-psp` | * | | MIPS (LE) Sony PlayStation Portable (PSP)
    [`mipsel-sony-psx`](platform-support/mipsel-sony-psx.md) | * | | MIPS (LE) Sony PlayStation 1 (PSX)
    `mipsel-unknown-linux-uclibc` | ✓ | | MIPS (LE) Linux with uClibc
    `mipsel-unknown-none` | * | | Bare MIPS (LE) softfloat
    `mipsisa32r6-unknown-linux-gnu` | ? | |
    `mipsisa32r6el-unknown-linux-gnu` | ? | |
    `mipsisa64r6-unknown-linux-gnuabi64` | ? | |
    `mipsisa64r6el-unknown-linux-gnuabi64` | ? | |
    `msp430-none-elf` | * | | 16-bit MSP430 microcontrollers
    `powerpc-unknown-linux-gnuspe` | ✓ | | PowerPC SPE Linux
    `powerpc-unknown-linux-musl` | ? | |
    `powerpc-unknown-netbsd` | ✓ | ✓ |
    `powerpc-unknown-openbsd` | ? | |
    `powerpc-wrs-vxworks-spe` | ? | |
    `powerpc-wrs-vxworks` | ? | |
    `powerpc64-unknown-freebsd` | ✓ | ✓ | PPC64 FreeBSD (ELFv1 and ELFv2)
    `powerpc64le-unknown-freebsd` | | | PPC64LE FreeBSD
    `powerpc-unknown-freebsd` | | | PowerPC FreeBSD
    `powerpc64-unknown-linux-musl` | ? | |
    `powerpc64-wrs-vxworks` | ? | |
    `powerpc64le-unknown-linux-musl` | ? | |
    [`powerpc64-unknown-openbsd`](platform-support/openbsd.md) | ✓ | ✓ | OpenBSD/powerpc64
    `powerpc64-ibm-aix` | ? | | 64-bit AIX (7.2 and newer)
    `riscv32gc-unknown-linux-gnu` | | | RISC-V Linux (kernel 5.4, glibc 2.33)
    `riscv32gc-unknown-linux-musl` | | | RISC-V Linux (kernel 5.4, musl + RISCV32 support patches)
    `riscv32im-unknown-none-elf` | * | | Bare RISC-V (RV32IM ISA)
    [`riscv32imac-unknown-xous-elf`](platform-support/riscv32imac-unknown-xous-elf.md) | ? | | RISC-V Xous (RV32IMAC ISA)
    `riscv32imc-esp-espidf` | ✓ | | RISC-V ESP-IDF
    `riscv64gc-unknown-freebsd` | | | RISC-V FreeBSD
    `riscv64gc-unknown-linux-musl` | | | RISC-V Linux (kernel 4.20, musl 1.2.0)
    [`riscv64gc-unknown-openbsd`](platform-support/openbsd.md) | ✓ | ✓ | OpenBSD/riscv64
    `s390x-unknown-linux-musl` | | | S390x Linux (kernel 3.2, MUSL)
    `sparc-unknown-linux-gnu` | ✓ | | 32-bit SPARC Linux
    `sparc64-unknown-netbsd` | ✓ | ✓ | NetBSD/sparc64
    [`sparc64-unknown-openbsd`](platform-support/openbsd.md) | ✓ | ✓ | OpenBSD/sparc64
    `thumbv4t-none-eabi` | * | | ARMv4T T32
    [`thumbv5te-none-eabi`](platform-support/armv5te-none-eabi.md) | * | | ARMv5TE T32
    `thumbv7a-pc-windows-msvc` | ? | |
    `thumbv7a-uwp-windows-msvc` | ✓ | |
    `thumbv7neon-unknown-linux-musleabihf` | ? | | Thumb2-mode ARMv7a Linux with NEON, MUSL
    [`wasm64-unknown-unknown`](platform-support/wasm64-unknown-unknown.md) | ? | | WebAssembly
    `x86_64-apple-ios-macabi` | ✓ | | Apple Catalyst on x86_64
    `x86_64-apple-tvos` | * | | x86 64-bit tvOS
    [`x86_64-apple-watchos-sim`](platform-support/apple-watchos.md) | ✓ | | x86 64-bit Apple WatchOS simulator
    [`x86_64-pc-nto-qnx710`](platform-support/nto-qnx.md) | ? | | x86 64-bit QNX Neutrino 7.1 RTOS |
    [`x86_64-pc-windows-gnullvm`](platform-support/pc-windows-gnullvm.md) | ✓ | ✓ |
    `x86_64-pc-windows-msvc` | * | | 64-bit Windows XP support
    `x86_64-sun-solaris` | ? | | Deprecated target for 64-bit Solaris 10/11, illumos
    `x86_64-unknown-dragonfly` | ✓ | ✓ | 64-bit DragonFlyBSD
    `x86_64-unknown-haiku` | ✓ | ✓ | 64-bit Haiku
    `x86_64-unknown-hermit` | ✓ | | HermitCore
    `x86_64-unknown-l4re-uclibc` | ? | |
    [`x86_64-unknown-openbsd`](platform-support/openbsd.md) | ✓ | ✓ | 64-bit OpenBSD
    `x86_64-uwp-windows-gnu` | ✓ | |
    `x86_64-uwp-windows-msvc` | ✓ | |
    `x86_64-wrs-vxworks` | ? | |
    should be up to date; feel free to post here or on zulip if there are targets that are supported by the compiler but not in that document.
    1a. maybe we should add a tool to keep those in sync?
  2. Put up a PR so the CI job runs. Fix compile errors as appropriate. Some targets may not compile on all hosts (e.g. iOS will likely need a macOS host, any MSVC target will need an MSVC host); either find a way to add support or add more runners so all targets are covered. Try to avoid spending too much time on any single target; if it takes too long it's ok to skip it for now. If it ends up being an ongoing issue to support a specific target we may want to consider asking one of the target maintainers (and if they're unresponsive, or there are no maintainers, making a t-compiler MCP to remove it).

cc @joboet, I think you said you were working on making it easier to cross-check targets from any host.

@jyn514 jyn514 added E-hard Call for participation: Hard difficulty. Experience needed to fix: A lot. E-mentor Call for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. T-libs Relevant to the library team, which will review and decide on the PR/issue. I-libs-nominated Nominated for discussion during a libs team meeting. labels Mar 13, 2023
@the8472
Copy link
Member

the8472 commented Mar 13, 2023

Add an allow-fail action which runs on bors merges, which doesn't block the merge but still lets us know when the target fails to build. I'm a little hesitant about this because I think it's rare people will actually look at the CI for the master branch, but maybe it's good enough.

Maybe that could be added to the bors comment that happens when tests finish?

@cuviper
Copy link
Member

cuviper commented Mar 13, 2023

Non-blocking feedback comments will need to propagate back to PRs that were rolled up too. That will need some state tracking, so it doesn't keep nagging if a tier-3 target stays broken for a while.

@jyn514
Copy link
Member Author

jyn514 commented Mar 13, 2023

If we're going to do state tracking then IMO we should go all the way to the existing toolstate infrastructure, so we're not reinventing it from scratch.

@m-ou-se
Copy link
Member

m-ou-se commented Mar 29, 2023

This was discussed two weeks ago in the library team meeting. There was a 3:2 split between

a. "just add the tier 3 checks as regular CI checks, and document that manually disabling tier 3 checks is fine", and
b. "use something like the existing toolstate tracking (that doesn't block CI)".

(See Zulip.)

I voted for the second option, but maybe we should just try the first option to see how that works out. Maybe the kind of breakage that happens for those targets will be simple and small enough to not be an issue. (And if it does become an issue, we can change it in the future.)

The first option does probably become quite annoying if we don't have a good way to run those checks manually though. So I think we need local ./x.py check would "Just Work™" when cross compiling for all tier 3 targets, and/or there needs to be a way to ask bors to run those checks, or those checks should always run on PRs directly (or at least those that touch the standard library).

@m-ou-se m-ou-se removed the I-libs-nominated Nominated for discussion during a libs team meeting. label Mar 29, 2023
@thomcc
Copy link
Member

thomcc commented Mar 29, 2023

The first option does probably become quite annoying if we don't have a good way to run those checks manually though.

How many (if any) targets currently supriously fail to check the parts of the stdlib they support1 if you deinit the llvm submodule? I think doing so fixes all of the failures I knew of (because we no longer try to build the compiler-rt code in compiler-builtins), but there might be some weird ones I don't know about.

Obviously even if the answer is "all of them", that's not a good long term solution (it's annoying and non-obvious to have to deinit it).

Footnotes

  1. E.g. obviously no_std-only targets should only check core (or core+alloc if they support alloc).

@jyn514
Copy link
Member Author

jyn514 commented Jul 17, 2023

The first option does probably become quite annoying if we don't have a good way to run those checks manually though. So I think we need local ./x.py check would "Just Work™" when cross compiling for all tier 3 targets, and/or there needs to be a way to ask bors to run those checks, or those checks should always run on PRs directly (or at least those that touch the standard library).

this should be the case once #102579 is merged

@jieyouxu jieyouxu removed the E-mentor Call for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion. label Nov 25, 2024
@Noratrieb
Copy link
Member

Noratrieb commented Nov 25, 2024

I think this issue needs data on when breakage actually happens, and I happen to have all the data regarding tier 3 breakage within the last two years. See the second section on https://does-it-build.noratrieb.dev (yes it takes long to load it's very bad). It also includes detailed build logs if you click on a build.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
E-hard Call for participation: Hard difficulty. Experience needed to fix: A lot. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. T-libs Relevant to the library team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

7 participants