GitHub Actions eval doesn’t fail on evaluation warnings

[emilazy]

As reported in #370661 (comment), ofborg-eval would fail when evaluation of the Nixpkgs jobset produced warnings, but the GitHub Actions eval doesn’t. This can lead to the channel being unexpectedly blocked by the release-checks job.

cc @infinisil @Mic92 @JohnRTitor @wolfgangwalther (not sure exactly who to ping for this)

[wolfgangwalther]

(not sure exactly who to ping for this)

All the eval stuff is still mostly a mystery to me, so I can't help much here :)

[Mic92]

There is a abort-on-warn flag we can enable in nix.

[emilazy]

That’s not quite enough; pkgs/top-level/nixpkgs-basic-release-checks.nix also aborts on non‐warning traces. I’d recommend following the logic there, as there are some other subtle channel‐blocking checks that aren’t related to warnings that I’m guessing the current eval CI doesn’t catch.

[infinisil]

PR with the fix: #381266