Experimental aarch64 support #6200

Ekleog · 2022-01-26T16:26:36Z

feat: aarch64 (m1) support

Wasmer singlepass does not support aarch64, so disable it there and use
wasmtime instead

feat: make everything compile on m1 (aarch64)

Condition all wasmer compilation on not being on aarch64. This could
also be done by using features, but then the reverse-dependencies would
have to know about this limitation.

Currently the code is a bit of a mess due to the amount of #![cfg()]
elements, but hopefully this can be improved on later on by having one
module with its own submodules per backend.

The support is still very experimental, but at least near-vm-runner's test will work now.

Related to #3803

The first commit is pretty much required, the second commit is this way in order for reverse-dependencies of near-vm-runner to not have to care about which features get enabled. An alternative could have been to make wasmer build on aarch64 (even if panicking on the first call), and the other alternative would have been to have all reverse-dependencies expose the wasm* features and ask that people on aarch64 manually toggle the proper features for things to work.

matklad · 2022-01-27T15:15:19Z

runtime/near-vm-runner/Cargo.toml

@@ -52,6 +38,22 @@ threadpool = "1.8.1"
 pwasm-utils_12 = { package = "pwasm-utils", version = "0.12" }
 parity-wasm_41 = { package = "parity-wasm", version = "0.41" }

+[target.'cfg(not(target = "aarch64"))'.dependencies]


@nagisa curious, what you think is the right approach here?

as is, the caller just has to avoid depending on wasmer on non-x86_64 platforms

make wasmer compiler for any platform, but error out when trying to compile/run stuff?

Given that wasmer2 just landed aarch64 support for singlepass, I suspect that we might want to undo this eventually.

But for now I think this makes sense – why bother compiling in guaranteed-to-be-dead code?

matklad

Yeah, I think "make it just work on aarch64" is the approach. Ideally, we should also add a check that aarch64 builds are not allowed on the mainnet (see #5890), but I wouldn't worry too much about it.

I think we should land this roughly in this shape. One significant question/suggestions I have is that maybe it makes more sense to gate wasmer on x64, rather that on not aarch64?

matklad · 2022-01-27T15:20:57Z

runtime/near-vm-runner/src/vm_kind.rs

+        compile_error!("Wasmer does not support aarch64, but a force_wasmer* feature was passed to near-vm-runner");
+
+        VMKind::Wasmtime
+    }


I generally prefer to have only one pub fn for_protocol_version(_protocol_version: ProtocolVersion) -> VMKind { and pull conditional compilation inside function's body. This is much easier for an IDE to work with.

matklad · 2022-01-27T15:21:50Z

runtime/near-vm-runner/src/vm_kind.rs

@@ -19,6 +19,7 @@ pub enum VMKind {
 }

 impl VMKind {
+    #[cfg(not(target_arch = "aarch64"))]


I wonder whether the logic should be if x86_64 rather than if !aarch64?

jakmeier · 2022-01-27T17:57:46Z

I'm excited to see progress here!

This gives me some warnings when compiling. If we want full aarch64 support, we should also have CI checks done for that platform. Something to keep in mind for the future I guess.

I don't know how to make suggestions for code lines you didn't touch. So I just put the diff here in the comment. Probably there is also a nicer way than my changes, but this at least shows how I got rid of all the warnings.

diff --git a/runtime/near-vm-runner/src/cache.rs b/runtime/near-vm-runner/src/cache.rs
index 6502a0bf4..1a49a9d0f 100644
--- a/runtime/near-vm-runner/src/cache.rs
+++ b/runtime/near-vm-runner/src/cache.rs
@@ -1,16 +1,20 @@
 use crate::errors::ContractPrecompilatonResult;
-use crate::prepare;
 use crate::vm_kind::VMKind;
 use borsh::{BorshDeserialize, BorshSerialize};
 use near_primitives::contract::ContractCode;
 use near_primitives::hash::CryptoHash;
 use near_primitives::types::CompiledContractCache;
-use near_vm_errors::{CacheError, CompilationError, FunctionCallError, VMError};
+use near_vm_errors::{CacheError, CompilationError};
 use near_vm_logic::{ProtocolVersion, VMConfig};
 use std::collections::HashMap;
 use std::fmt;
 use std::sync::{Arc, Mutex};
 
+#[cfg(not(target_arch = "aarch64"))]
+use crate::prepare;
+#[cfg(not(target_arch = "aarch64"))]
+use near_vm_errors::{FunctionCallError, VMError};
+
 #[derive(Debug, Clone, BorshSerialize)]
 enum ContractCacheKey {
     _Version1,
@@ -62,6 +66,7 @@ pub fn get_contract_cache_key(
     near_primitives::hash::hash(&key.try_to_vec().unwrap())
 }
 
+#[cfg(not(target_arch = "aarch64"))]
 fn cache_error(
     error: &CompilationError,
     key: &CryptoHash,
@@ -73,6 +78,7 @@ fn cache_error(
     Ok(())
 }
 
+#[cfg(not(target_arch = "aarch64"))]
 pub fn into_vm_result<T>(
     res: Result<Result<T, CompilationError>, CacheError>,
 ) -> Result<T, VMError> {
@@ -114,7 +120,7 @@ impl fmt::Debug for MockCompiledContractCache {
     }
 }
 
-#[cfg(not(feature = "no_cache"))]
+#[cfg(all(not(feature = "no_cache"), not(target_arch = "aarch64")))]
 const CACHE_SIZE: usize = 128;
 
 #[cfg(all(feature = "wasmer0_vm", not(feature = "no_cache"), not(target_arch = "aarch64")))]
diff --git a/runtime/near-vm-runner/src/vm_kind.rs b/runtime/near-vm-runner/src/vm_kind.rs
index 7099e4489..e588a7ef8 100644
--- a/runtime/near-vm-runner/src/vm_kind.rs
+++ b/runtime/near-vm-runner/src/vm_kind.rs
@@ -1,8 +1,10 @@
 use borsh::BorshSerialize;
-use near_primitives::checked_feature;
 use near_vm_logic::ProtocolVersion;
 use std::hash::Hash;
 
+#[cfg(not(target_arch = "aarch64"))]
+use near_primitives::checked_feature;
+
 #[derive(Clone, Copy, Debug, Hash, BorshSerialize)]
 // Note, that VMKind is part of serialization protocol, so we cannor remove entri

nagisa · 2022-01-27T17:58:13Z

runtime/near-vm-runner/src/tests/cache.rs

@@ -1,4 +1,6 @@
-//! Tests that `CompiledContractCache` is working correctly.
+//! Tests that `CompiledContractCache` is working correctly. Currently testing only wasmer code, so disabled on aarch64
+#![cfg(not(target_arch = "aarch64"))]


#[cfg_attr(..., ignore)] the tests instead.

Wasmer singlepass does not support aarch64, so disable it there and use wasmtime instead

Condition all wasmer compilation on not being on aarch64. This could also be done by using features, but then the reverse-dependencies would have to know about this limitation. Currently the code is a bit of a mess due to the amount of #![cfg()] elements, but hopefully this can be improved on later on by having one module with its own submodules per backend.

Ekleog · 2022-01-31T13:42:07Z

I've just handled all the review comments, hopefully this is ready to land now :)

runtime/near-vm-runner/src/vm_kind.rs

matklad

LGTM (but obviously we need make sure this at least compiles)

Is there some trick we can use on Linux to check compilation for macs? Would something like cargo check -p near-vm-runner --target aarch64-apple-darwin or cargo check -p aarch64-unknown-linux-gnu added to CI help?

I think, long term, we should add mac-based CI, but that seems like a chore, so I am wondering if there's some "start small" we can do here? Better to do that as a follow up probably.

matklad · 2022-02-01T10:52:01Z

runtime/near-vm-runner/src/tests/contract_preload.rs

@@ -1,3 +1,6 @@
+// Currently only testing wasmer code, so disabled outside of x86_64
+#![cfg_attr(not(target_arch = "x86_64"), ignore)]


Hm, ignore isn't supported for modules, no?

Yes, this doesn't seem to work at all.

Oh right. @nagisa I'm cancelling the changes relating to #6200 (comment) ; as the file doesn't compile without wasmer at all (because the things it tests don't compile without wasmer)

jakmeier · 2022-02-01T11:14:19Z

Is there some trick we can use on Linux to check compilation for macs? Would something like cargo check -p near-vm-runner --target aarch64-apple-darwin or cargo check -p aarch64-unknown-linux-gnu added to CI help?

That would indeed be nice. But we would still need an aarch64 machine, or otherwise make sure cross-compilation works. At least on my desktop, I'm missing dependencies for successful cross-compilation of wasmtime-runtime from x86 to aarch64.

Ekleog · 2022-02-01T13:22:47Z

This time to be as sure as I could that this builds properly, I ran something akin to s/x86_64/aarch64/ and checked things build correctly, so hopefully (once @jakmeier confirms on actual hardware) it should be good to go this time :)

As for the CI, I'd think the best would be to get an aarch64 machine to run CI on, but I'm not sure how easy that'd be? (in particular, buildkite seems to be using distro=amazonlinux so I guess CI runs on aws in which case it should not be too hard as aws has aarch64 VMs, but I may be misunderstanding the setup… if that's too hard we can always use qemu to run the whole thing I guess)

jakmeier · 2022-02-01T13:43:22Z

Yeah must be frustrating to code blindly... But I can confirm it now compiles and runs tests successfully. :)

Unfortunately, there is a new aarch64 specific error in the nearcore build which has nothing to do with your change.

error[E0080]: evaluation of constant value failed
   --> chain/network/src/network_protocol.rs:219:15
    |
219 | const _: () = assert!(std::mem::size_of::<PeerMessage>() <= 1144, "PeerMessage > 1144 bytes");
    |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the evaluated program panicked at 'PeerMessage > 1144 bytes', chain/network/src/network_protocol.rs:219:15

This also fails before your changes and was introduced with a change from only a few days ago. It means I cannot test if there are any errors hidden behind this failure. But at this point, I don't think that should stop this from merging. You have my approval (as a non code owener).

Ekleog · 2022-02-01T13:50:25Z

Hmm can you try cargo test -p near-vm-runner? Hopefully that'd pass, despite nearcore as-a-whole failing to build with a seemingly on-purpose failure

jakmeier · 2022-02-01T14:05:15Z

Hmm can you try cargo test -p near-vm-runner? Hopefully that'd pass, despite nearcore as-a-whole failing to build with a seemingly on-purpose failure

Yes that runs through without any errors or warnings. That's what I meant with "compiles and runs tests successfully".

I also just created #6222 to track the other issue.

Ekleog · 2022-02-01T15:36:29Z

Awesome thanks!

ChzenChzen · 2022-03-05T01:28:30Z

https://wasmer.io/posts/wasmer-2.2
The Wasmer 2.2 release features significant advancements shaping up to impact our Web3 and blockchain community in a big way. Wasmer is reintroducing Aarch64 compatibility for our Singlepass compiler. With the newly overhauled Singlepass, Web3 and blockchain developers can efficiently run Wasmer Runtime with Singepass on Windows, Linux, and macOS. The new release also fully supports the much anticipated Apple M1 processor.

Ekleog requested a review from jakmeier January 26, 2022 16:26

Ekleog requested review from matklad, nagisa and olonho as code owners January 26, 2022 16:26

matklad reviewed Jan 27, 2022

View reviewed changes

nagisa reviewed Jan 27, 2022

View reviewed changes

Ekleog added 2 commits January 31, 2022 14:41

feat: aarch64 (m1) support

e8d1241

Wasmer singlepass does not support aarch64, so disable it there and use wasmtime instead

Ekleog force-pushed the aarch64-support branch from 3275b32 to 4dffb65 Compare January 31, 2022 13:41

Ekleog added the S-automerge label Jan 31, 2022

Ekleog force-pushed the aarch64-support branch from 4dffb65 to 7c49024 Compare January 31, 2022 13:50

jakmeier reviewed Feb 1, 2022

View reviewed changes

runtime/near-vm-runner/src/vm_kind.rs Outdated Show resolved Hide resolved

Ekleog force-pushed the aarch64-support branch from 7c49024 to 0525ab9 Compare February 1, 2022 10:51

matklad approved these changes Feb 1, 2022

View reviewed changes

debt: handle review comments

8d43764

Ekleog force-pushed the aarch64-support branch from 0525ab9 to 8d43764 Compare February 1, 2022 13:12

Merge branch 'master' into aarch64-support

a13f475

near-bulldozer bot merged commit d1d4559 into near:master Feb 1, 2022

matklad mentioned this pull request Feb 2, 2022

Compile error in macos(M1) #4995

Closed

This was referenced Feb 15, 2022

Account deserialization fails on ARM64 #6293

Closed

Incompatible with Macs based on the Apple M1 chip #3803

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Experimental aarch64 support #6200

Experimental aarch64 support #6200

Ekleog commented Jan 26, 2022

matklad Jan 27, 2022

nagisa Jan 27, 2022

matklad left a comment

matklad Jan 27, 2022

matklad Jan 27, 2022

jakmeier commented Jan 27, 2022

nagisa Jan 27, 2022

Ekleog commented Jan 31, 2022

matklad left a comment

matklad Feb 1, 2022

jakmeier Feb 1, 2022

Ekleog Feb 1, 2022

jakmeier commented Feb 1, 2022

Ekleog commented Feb 1, 2022

jakmeier commented Feb 1, 2022

Ekleog commented Feb 1, 2022

jakmeier commented Feb 1, 2022

Ekleog commented Feb 1, 2022

ChzenChzen commented Mar 5, 2022

		@@ -1,3 +1,6 @@
		// Currently only testing wasmer code, so disabled outside of x86_64
		#![cfg_attr(not(target_arch = "x86_64"), ignore)]

Experimental aarch64 support #6200

Experimental aarch64 support #6200

Conversation

Ekleog commented Jan 26, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

matklad left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jakmeier commented Jan 27, 2022

Choose a reason for hiding this comment

Ekleog commented Jan 31, 2022

matklad left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jakmeier commented Feb 1, 2022

Ekleog commented Feb 1, 2022

jakmeier commented Feb 1, 2022

Ekleog commented Feb 1, 2022

jakmeier commented Feb 1, 2022

Ekleog commented Feb 1, 2022

ChzenChzen commented Mar 5, 2022