Add ARM support for Linux

[j8r]

Description

The ARM architecture, aarch64 and eventually armhf, are gaining traction since the raise of Raspberry PIs and recently cheap ARM offerings in various clouds.
Several projects now fully supports ARM, thus needing a CI.

Context

The Crystal Language aims to fully support Linux aarch64, and eventually armhf. That's not easy since the current CI doesn't support this architecture.

Anything Else

Thanks to have suggested us to file a feature request in our issue tracker, I don't know how you have found our thread 😅

[RDIL]

For FreeBSD?

[j8r]

That's mainly to support Linux aarch64, and eventually armhf. FreeBSD aarch64 is a plus, but since Crystal doens't support it at tier 2 unlike Linux aarch64 that's not a high priority.

[fkorotkov]

@j8r I'm spending too much time on GitHub and Twitter. Sorry, it it creeped you out. 😅

[emaste]

I'm very much interested in seeing aarch64 (arm64) support for FreeBSD

[ped7g]

For my current use case I would need in ideal case 32bit ARMv6 target (Raspberry Pi0), but even any 32b ARM (v7+) would already help a lot to make sure that v6 variant is highly likely buildable too.

And finally even 64b ARM would be nice just to have another target platform, in case it would uncover some weird bugs in project. (I'm not sure if the 64b ARM is already available? I think it is not, but I didn't research the current situation thoroughly, so just ignore this, if it's already available, I will eventually find out... :) )

(edit: of course I had linux at mind .. anything else would be nice bonus, but I need the base OS first)

[fkorotkov]

Work on #397 might help to run arm containers via emulation. 🤔

[ped7g]

Actually meanwhile it occurred to me, that qemu can emulate ARM CPUs (with quite some wide range of choice), i.e. I guess it may be possible to run qemu in ARM configuration on community instance of x86_64 linux box.

But I'm in the stage of "idea" at this moment, would have to experiment first locally if I can configure qemu to emulate at least partially also quite specific environment (Pi0 running custom minimal linux distribution, with some custom drivers) and run binaries inside that and extract results of such runs.

And I would ideally build the binaries first, with x86_64 gcc cross-compiling for arm, but I'm not sure if the official gcc docker images contain also cross-variants...

I.e. many technical details to research, verify and reconsider, and right now I'm busy with different things.

Adding KVM support may help to widen the choice of possible emulators, so nonetheless thank you for your effort, it certainly helps to move this one step closer.

[fkorotkov]

@ped7g yes! But my understanding is that you'll need nested virtualization enabled which comes with #397 🤷‍♂

[j8r]

Why not directly using ARM hardware?

[fkorotkov]

Google Cloud where Linux, FreeBSD and Windows community tasks are executed is not supporting ARM yet. BTW #263 will allow to have persistent worker for any architecture.

[Minoru]

@dankegel I see a number of Intel CPUs and one AMD, but no ARM. You sure you aren't mixing up the names?

[dankegel]

Aw, foo, wishful thinking.

[maflcko]

I think travis is using https://www.packet.com/ to spin up their arm machines, maybe cirrus ci could use them as well?

[fkorotkov]

I'm also pretty bored waiting for GCP so probably we'll go with AWS Graviton. 😅Will give GCP the last chance on July 14th when they have Google Cloud Next conference.

[maflcko]

It is possible to use self-hosted arm via persistent workers: https://cirrus-ci.org/guide/persistent-workers/

[fkorotkov]

For anyone interested please take a look at #905 and give arm_container a try.

[emaste]

probably we'll go with AWS Graviton

Is this likely to be Linux containers only for the foreseeable future, or is there a chance we can run (FreeBSD) VM images on Graviton?

[fkorotkov]

Just linux containers for now. Please create a separate feature request for FreeBSD. Seems there AMIs for FreeBSD 12 and 13 for ARM so it's feasible to be able to add the support.

[emaste]

Thanks, added #906

[asomers]

Thanks for adding aarch64 support! Now Nix is tested on real aarch64 hardware, at least for Linux.
nix-rust/nix#1506

[fkorotkov]

@asomers, awesome! Did you notice any performance gains?

[asomers]

@fkorotkov previously we tested Linux/aarch64 using user mode emulation with QEMU in a 1-CPU compute_engine_instance. Now we're using the arm_container. The build+test time decreased from 1m33s to 1m12s, a modest improvement. But the setup time decreased all the way from 2m13s to 1s, since we no longer have to install QEMU and the cross-toolchain. Not bad!