improve handling of unsandboxed checks

[sellout]

Some checks (e.g., Vale’s¹) currently need to disable the chroot. The way it's currently handled, they are just regular checks, so you need to either manually exclude them from your flake or set sandbox = false if you want to nix flake check. Setting sandbox = false in the flake has already bit me on multiple occasions, so I want to get rid of that.

There are a few different thoughts I've had for managing this:

• provide a way to add only sandboxed checks to the flake
• add an option to include the unsandboxed checks as devShells instead
• make use of the lax-checks devShell that sort of exists to bundle all of the unsandboxed checks similarly to nix flake check

All three is probably the way to go. Additionally, it'd be great to automatically generate GH or other CI jobs for unsandboxed derivations, since garnix can't run them.

Footnotes

1. In the case of Vale, I want to add a valeWithPackages to Nixpkgs so that Vale packages are just derivations and don't break the sandbox. But this issue still holds more broadly. ↩

[sellout]

This is much improved by #55, but not quite finished. I think the changes in that PR are generally better than what’s suggested above. The first bullet point is the same, but the second is unnecessary, as they can still be checks and the last one is there but not quite in a satisfactory state.

I hadn’t known that sandbox = "relaxed" was a thing, so that helps a lot. But the thing that’s missing is being able to somehow run the unsandboxed checks when sandbox = true. It seems that CLI options like --option sandbox false don’t take precedence, which is surprising.