Enable x86_64-linux-gnu packager

[staticfloat]

No description provided.

[DilumAluthge]

Rootfs hash mismatch?

[DilumAluthge]

What if we try doing export JULIA_NUM_THREADS=2 for the rr job?

[DilumAluthge]

What commit of Julia are you using? Perhaps the fork is behind. --force-net is a recent addition to Julia.

[DilumAluthge]

🤦 I both added and removed --force-net, so you'd think I would have remembered.

[DilumAluthge]

Personally, I think that "packaging" is too coarse of a group.

I would suggest that we have five groups:

1. Windows
2. macOS
3. Linux
4. FreeBSD
5. Buildkite

And here are the jobs that would live inside each group:

1. Windows
   • build_windows_i686
   • build_windows_x86_64
   • test_windows_i686
   • test_windows_x86_64
2. macOS
   • build_macos_aarch64
   • build_macos_x86_64
   • test_macos_aarch64
   • test_macos_x86_64
3. Linux
   • build_linux_aarch64
   • build_linux_armv7l
   • build_linux_i686
   • build_linux_powerpc64le
   • build_linux_x86_64
   • doctest_linux_x86_64
   • embedding
   • llvmpasses
   • sanitizers
   • whitespace
   • test_linux_aarch64
   • test_linux_armv7l
   • test_linux_i686
   • test_linux_powerpc64le
   • test_linux_x86_64
   • test_linux_x86_64_assert
   • test_linux_x86_64_rr
4. FreeBSD
   • build_freebsd_x86_64
   • test_freebsd_x86_64
5. Buildkite
   • Launch signed pipelines
   • Launch unsigned jobs
   • Unlock secrets, launch pipelines

I think this finds a good balance between being too coarse and being too granular.

Also I think we should rename "packaging" to "build".

[DilumAluthge]

The other option would be to have nine groups:

1. Build Windows
2. Test Windows
3. Build macOS
4. Test macOS
5. Build Linux
6. Test Linux
7. Build FreeBSD
8. Test FreeBSD
9. Buildkite

I would be fine with either the five groups (as listed above), or the nine groups. I have a slight preference for the five groups.

[DilumAluthge]

One of the reasons that I have for wanting to group by OS is that it makes it easier to see what's going on with the agents.

For example, if every PR is green on all OSs except for Windows, and the Windows groups are all yellow on all PRs, then we know that the Windows queue is really backed up.

[DilumAluthge]

I think that the namespace for keys is shared among all entities. So if you have a group with key mykey, you can't also have a step that has key mykey. But you can have a group with key mygroupkey and a step (within that group) that has key mystepkey.

[DilumAluthge]

Instead of these terrible whitespace-delimited tables that I created, should we just bite the bullet and use TOML/YAML/JSON?

[DilumAluthge]

We have Julia available, so we can just use the TOML stdlib.

Most entries won't be very verbose, because for most entries we'll just omit most keys, and the default values will get used.

[staticfloat]

I like the terrible tables; it allows me to implement scary features like default values and whatnot. 😈

If we jump ship, I think we can use the new build matrix functionality that buildkite is cooking up.

[DilumAluthge]

Can you take the remaining three ungrounded checks and either put them into an existing group, or make a Buildkite group and put them in there?

[DilumAluthge]

I would just rename Repo Checks to something like Infrastructure, and then it can contain the two existing "repo checks" as well as the three currently ungrouped jobs.

[staticfloat]

Alright, we're going to call this good for now. There are some issues still:

• Cryptic makes it difficult to use groups, but I'll work on that separately.
• We're not signing the tarballs with our GPG key yet (We'll wait for gpg to be included in our aws_uploader rootfs image: We need gpg when uploading rootfs-images#169)

I think this is sufficient to deploy to the main repository. It's better organized, and importantly, it should reduce our CI burden significantly by reducing the number of times we run the full test suite by a factor of 6. 😳