Apple M1 Support

[maxmcd]

Is this planned? Is this something that would be helpful for me to add and PR?

[alexcrichton]

Thanks for the report! Support first needs to be added upstream to Wasmtime itself, and I don't think that aarch64 on macOS works there just yet. I suspect it won't be the hardest thing in the world to add but it unfortunately also won't be trivial.

[tschneidereit]

It seems like Wasmtime works on M1 now? That said, we don't have any tests running for it in CI, so for the time being couldn't treat aarch64-darwin as a supported target.

[evaporei]

Any updates on this? We use Rust version of wasmtime and having issues on Macbook M1: graphprotocol/graph-node#2325

@maxmcd @alexcrichton @tschneidereit

[cfallin]

@bnjbvr I believe had this working recently (at least, there were two PRs we recently merged related to signal/exception handling and the calling convention; I was under the impression that no other issues had to be solved). Ben, could you comment on this / the error in the linked issue?

(I wonder if it has to do with release versions actually -- the two PRs above were merged only recently...)

[evaporei]

Just to add up, we're using version 0.25.0, idk if those two were merged before or after the version was created.

[cfallin]

Ah, yes, this could be it: Ben's bytecodealliance/wasmtime#2742 added the calling convention support for M1 and that came after the 0.25 release.

We should probably do a release soon anyway -- I'll raise this on our Zulip and see what folks think! (We're planning to merge another big PR on Monday, switching the x86 backend, so maybe we could release right after that.)

[cfallin]

We just released 0.26.0 -- please do let us know if something still doesn't work! (I don't have an M1 so I can't test myself, but if someone else confirms it does work, we can close the issue I think.)

[bnjbvr]

Hi! So sufficient Cranelift support for wasm has been added, and I can confirm this works nicely with Wasmtime. Known caveats:

• unwinding wasm frames is pretty much broken, when using the system's libunwind (= the default). It is closed source, and no way reproducing the issue with a userland build of libunwind. Linking with a userland build of libunwind makes it work 🤷.
• when running the CI's tests, some WASI tests don't pass. I'll make a PR to mark tests as ignored, so they become known failures and can be investigated independently. investigated since then, it seems they're now passing! \o/

[srenatus]

I must be missing something, the latest release, https://github.com/bytecodealliance/wasmtime/releases/tag/v0.27.0, doesn't have anything for macos that isn't x86_64. We'd need that for starters, don't we?

[cfallin]

Hi @srenatus -- yes, ideally we'll eventually be able to add a build/test CI job for macOS/aarch64, but unfortunately GitHub Actions doesn't currently support the platform, so we're blocked on that. As far as I'm aware, there isn't a way to emulate macOS/aarch64 on macOS/x86_64 (a sort of reverse Rosetta 2), so the qemu-based approach we take on the Linux runners isn't applicable. For now, we have @bnjbvr and the Embark Studios folks monitoring M1 compatibility with their own CI, so there is at least some coverage.

[srenatus]

Ah, got it. Thanks for clarifying. 😃

[anistark]

Any idea when it'll be fixed?

I get

# github.com/bytecodealliance/wasmtime-go
ld: library not found for -lwasmtime

when trying to go install.

[cfallin]

Hi @anistark -- unfortunately the situation with GitHub CI support for macOS on aarch64 (M1) is still the same, i.e. there is none. The issue actions/runner-images#2187 seems to be tracking this. Without CI on M1, we can't build release binaries, unfortunately.

[anistark]

@cfallin Understood. Thanks for the explanation.

[danielledeleo]

Basic support was added in #102 if you build wasmtime from source. It can also run in a native aarch64 Linux Docker container on M1 (since #103) using the existing wasmtime tarballs, which is great!

[howjmay]

Hi @anistark -- unfortunately the situation with GitHub CI support for macOS on aarch64 (M1) is still the same, i.e. there is none. The issue actions/virtual-environments#2187 seems to be tracking this. Without CI on M1, we can't build release binaries, unfortunately.

@cfallin how about using qemu to simulate the aarch64? Maybe it will help.

[cfallin]

@howjmay we do use qemu for Linux/aarch64, and it works great for that! That's why we're able to have Linux/aarch64 binaries even though GitHub CI doesn't natively support aarch64. Unfortunately though, qemu doesn't support running macOS binaries in its usermode configuration, which is why we don't have any way to test or release binaries for "macOS on M1" yet. (If anyone is aware of any emulators that do support this though, that would definitely be an option in the future.)

[howjmay]

Hi @cfallin Whether cross-compile a possible solution for this?
rust-lang/rust#73908

It seems we can use aarch64-apple-darwin to cross-compile rust program to M1 version

[srenatus]

From my understanding, compiling for that target is not a problem, but testing in CI is: #53 (comment)

[kyleconroy]

The GitHub Actions runner now has support for M1 Macs (actions/runner#805 (comment))

I'd be happy to host an M1 runner for this project until GitHub offers them by default. Would that be useful?

[cfallin]

From a technical standpoint that would work but in the past we've avoided self-hosting runners due to the associated security concerns.

It looks like in the issue related to GitHub-hosted M1 runners, actions/runner-images#2187, someone linked a CI-focused VM system Tart that might be useful, though on a skim it looks like it might be some-assembly-required, so to speak. Would you be willing to look into ephemeral-VM-based isolation for CI runs on a selfhosted M1 runner? If we could work something like that out, and convince ourselves the security properties were comparable to GitHub-hosted infra (i.e., no persisted state between runs and fully hermetic wrt the host system), we could consider it IMHO.

(Also IMHO, the lack of communication on timelines here is kind of baffling; if GitHub were to give some indication that M1 runners were coming in a month or two, vs indefinite-and-no-one's-working-on-it, that would be very useful. But we're far from the only ones wanting this, judging from the linked issues.)

[cfallin]

One further thought on security: I think we could potentially accept a selfhosted runner (with proper isolation) giving us test results on macos-aarch64, but we should absolutely remain on GitHub-hosted actions for the release build that becomes an artifact, IMHO, using our current cross-compilation from macos-x86_64 setup. In this way we get strictly more info than we have today (one "pass/fail" bit) but a compromised or untrusted runner can't propagate tainted artifacts into our releases.

[kyleconroy]

Would you be willing to look into ephemeral-VM-based isolation for CI runs on a selfhosted M1 runner?

Yes

but we should absolutely remain on GitHub-hosted actions for the release build that becomes an artifact

If this is the case, candidly I don't think it's worth the time setting up the self-hosted runner. I'm interested in setting this up so that users on M1 Macs can install go-wasmtime without having to install wasmtime first.

[cfallin]

I don't think there's any conflict, given cross-compilation?

Ah, so actually, taking a step back, I'm not sure if latest context with wasmtime proper has been incorporated here: we do now have macOS/aarch64 binaries in the wasmtime release artifacts, and the last release (0.37) was the first to occur since we added this to CI. It works by cross-compiling from macOS/x86_64, and is a compromise in that we don't actually test except for manually. (I have M1 hardware and a few other contributors do too.) I had been tacitly assuming your offer above was to give us true CI coverage of the platform until we can move away from cross-compilation.

But I'm actually curious if enough already exists for wasmtime-go to work today? In other words, to your point

I'm interested in setting this up so that users on M1 Macs can install go-wasmtime without having to install wasmtime first.

are we already there, given the aarch64-macos binaries here?

(I'm on PTO starting tomorrow but cc @alexcrichton to follow up on this maybe?)

[kyleconroy]

I didn't realize that v0.37.0 included wasmtime-v0.37.0-aarch64-macos-c-api.tar.xz. That solves the first part of the issue.

Is the only thing blocking this a change to the build / tagging process? Once the version is bumped to v0.37.0 , @alexcrichton usually creates a commit with the artifacts. If that can include build/macos-aarch64/libwasmtime.a we should be set.

I had been tacitly assuming your offer above was to give us true CI coverage of the platform until we can move away from cross-compilation.

And that makes sense! I didn't have the full context. Since we now have the necessary pre-built libwasmtime.a, I don't think a self-hosted runner is needed.

EDIT: I've confirmed that a small change to ci/download-wasmtime.py works.

[alexcrichton]

Personally I dont think the Go bindings can aspire further than the Wasmtime project itself at this point which is to say that I don't think running tests in CI is in the cards but producing releases should be fine (e.g. committing the precompiled artifacts as is done for other platforms). #131 is a good step towards that and I can do the 0.37.0 release (I completely forgot about it originally) when that's sorted.

[kyleconroy]

I can confirm that v0.37.0 builds on my M1 Mac! Here's what I'm using to test it out (https://gist.github.com/kyleconroy/a50054984159918d78aa7c0448e56db5).

[danielledeleo]

Thank you so much! We've been eagerly waiting for this at Suborbital. ^^