Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set --define wasm=wasmtime for macOS by default #23235

Closed
dio opened this issue Sep 24, 2022 · 4 comments
Closed

Set --define wasm=wasmtime for macOS by default #23235

dio opened this issue Sep 24, 2022 · 4 comments
Labels
triage Issue requires triage

Comments

@dio
Copy link
Member

dio commented Sep 24, 2022

Currently, since #20889 is merged, macOS binaries v1.23.x (e.g. fetched via Homebrew) by default doesn't support Wasm. Should we use build:macos --define wasm=wasmtime instead? I think at least people getting envoy binary from Homebrew can play with Wasm filter too.

cc. @keith @cpakulski

@dio dio added the triage Issue requires triage label Sep 24, 2022
@dio
Copy link
Member Author

dio commented Sep 25, 2022

Seems like @keith's patch (mentioned in #20889) https://chromium.googlesource.com/v8/v8.git/+log/refs/tags/10.4.132.18/bazel/config is included in main (v8 version 10.4.132.18: https://chromium.googlesource.com/v8/v8.git/+log/refs/tags/10.4.132.18/bazel/config). However, when we build it on macOS M1 and try to load a .wasm file (full log: https://gist.githubusercontent.com/dio/7ef745c09327aa6da0be3b32ef28016a/raw/d220e891afe52208d346ed0dae0d72220275fe73/v8-log.txt):

Caught on: https://chromium.googlesource.com/v8/v8.git/+/refs/tags/10.4.132.18/src/base/platform/platform-posix.cc#478 (which indicates a bug in the caller (mprotect() behavior changes (only on macOS > 11.2 on Apple Silicon)? Probably the following is on the same topic: https://gist.github.com/hikalium/75ae822466ee4da13cbbe486498a191f?permalink_comment_id=3623067#gistcomment-3623067, https://bugs.chromium.org/p/v8/issues/detail?id=11389#c14)). The patch mentioned here: nodejs/node#37276 is also included in v8 10.4.132.18.

We expect ENOMEM (12), but got EACCES (13).

#
# Fatal error in external/v8/src/base/platform/platform-posix.cc, line 478
# Check failed: 12 == (*__error()) (12 vs. 13).
#
#
#
#FailureMessage Object: 0x16b63df60
==== C stack trace ===============================

    0   envoy-static                        0x000000010a935be4 v8::base::debug::StackTrace::StackTrace() + 32
    1   envoy-static                        0x000000010a935c20 v8::base::debug::StackTrace::StackTrace() + 28
    2   envoy-static                        0x0000000109e23308 v8::platform::(anonymous namespace)::PrintStackTrace() + 28
    3   envoy-static                        0x000000010a8ff940 V8_Fatal(char const*, int, char const*, ...) + 208
    4   envoy-static                        0x000000010a92fdd0 v8::base::OS::SetPermissions(void*, unsigned long, v8::base::OS::MemoryPermission) + 424
    5   envoy-static                        0x000000010a90fb20 v8::base::PageAllocator::SetPermissions(void*, unsigned long, v8::PageAllocator::Permission) + 44
    6   envoy-static                        0x000000010a8f13f8 v8::base::BoundedPageAllocator::SetPermissions(void*, unsigned long, v8::PageAllocator::Permission) + 252
    7   envoy-static                        0x0000000108a5c2b0 v8::internal::VirtualMemory::SetPermissions(unsigned long, unsigned long, v8::PageAllocator::Permission) + 140
    8   envoy-static                        0x0000000108044b08 v8::internal::MemoryAllocator::PartialFreeMemory(v8::internal::BasicMemoryChunk*, unsigned long, unsigned long, unsigned long) + 520
    9   envoy-static                        0x000000010810aa78 v8::internal::Page::ShrinkToHighWaterMark() + 1136
    10  envoy-static                        0x00000001080961a4 v8::internal::PagedSpaceBase::ShrinkPageToHighWaterMark(v8::internal::Page*) + 36
    11  envoy-static                        0x0000000108096524 v8::internal::PagedSpaceBase::ShrinkImmortalImmovablePages() + 256
    12  envoy-static                        0x0000000107ee83b0 v8::internal::Heap::NotifyDeserializationComplete() + 92
    13  envoy-static                        0x0000000107d21a94 v8::internal::Isolate::Init(v8::internal::SnapshotData*, v8::internal::SnapshotData*, v8::internal::SnapshotData*, bool) + 5192
    14  envoy-static                        0x0000000107d22230 v8::internal::Isolate::InitWithSnapshot(v8::internal::SnapshotData*, v8::internal::SnapshotData*, v8::internal::SnapshotData*, bool) + 216
    15  envoy-static                        0x0000000108a132a0 v8::internal::Snapshot::Initialize(v8::internal::Isolate*) + 792
    16  envoy-static                        0x0000000107862a54 v8::Isolate::Initialize(v8::Isolate*, v8::Isolate::CreateParams const&) + 980
    17  envoy-static                        0x00000001078631e4 v8::Isolate::New(v8::Isolate::CreateParams const&) + 36
    18  envoy-static                        0x00000001072bf48c wasm::Store::make(wasm::Engine*) + 308
    19  envoy-static                        0x00000001072779d4 proxy_wasm::v8::V8::load(std::__1::basic_string_view<char, std::__1::char_traits<char> >, std::__1::basic_string_view<char, std::__1::char_traits<char> >, std::__1::unordered_map<unsigned int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::hash<unsigned int>, std::__1::equal_to<unsigned int>, std::__1::allocator<std::__1::pair<unsigned int const, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > > const&) + 64
    20  envoy-static                        0x0000000107247b04 proxy_wasm::WasmBase::load(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, bool) + 1536
    21  envoy-static                        0x000000010724a014 proxy_wasm::createWasm(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<proxy_wasm::PluginBase> const&, std::__1::function<std::__1::shared_ptr<proxy_wasm::WasmHandleBase> (std::__1::basic_string_view<char, std::__1::char_traits<char> >)> const&, std::__1::function<std::__1::shared_ptr<proxy_wasm::WasmHandleBase> (std::__1::shared_ptr<proxy_wasm::WasmHandleBase>)> const&, bool) + 576
    22  envoy-static                        0x0000000106723bd4 _ZZN5Envoy10Extensions6Common4Wasm10createWasmERKNSt3__110shared_ptrINS2_6PluginEEERKNS4_INS_5Stats5ScopeEEERNS_8Upstream14ClusterManagerERNS_4Init7ManagerERNS_5Event10DispatcherERNS_3Api3ApiERNS_6Server23ServerLifecycleNotifierERNS3_10unique_ptrINS_6Config10DataSource23RemoteAsyncDataProviderENS3_14default_deleteISW_EEEEONS3_8functionIFvNS4_INS2_10WasmHandleEEEEEENS11_IFPN10proxy_wasm11ContextBaseEPNS2_4WasmES8_EEEENK3$_5clENS3_12basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEE + 380
    23  envoy-static                        0x00000001067229b4 Envoy::Extensions::Common::Wasm::createWasm(std::__1::shared_ptr<Envoy::Extensions::Common::Wasm::Plugin> const&, std::__1::shared_ptr<Envoy::Stats::Scope> const&, Envoy::Upstream::ClusterManager&, Envoy::Init::Manager&, Envoy::Event::Dispatcher&, Envoy::Api::Api&, Envoy::Server::ServerLifecycleNotifier&, std::__1::unique_ptr<Envoy::Config::DataSource::RemoteAsyncDataProvider, std::__1::default_delete<Envoy::Config::DataSource::RemoteAsyncDataProvider> >&, std::__1::function<void (std::__1::shared_ptr<Envoy::Extensions::Common::Wasm::WasmHandle>)>&&, std::__1::function<proxy_wasm::ContextBase* (Envoy::Extensions::Common::Wasm::Wasm*, std::__1::shared_ptr<Envoy::Extensions::Common::Wasm::Plugin> const&)>) + 6228
    24  envoy-static                        0x0000000105bd6fd4 Envoy::Extensions::HttpFilters::Wasm::FilterConfig::FilterConfig(envoy::extensions::filters::http::wasm::v3::Wasm const&, Envoy::Server::Configuration::FactoryContext&) + 720
    25  envoy-static                        0x0000000105bd75ec Envoy::Extensions::HttpFilters::Wasm::FilterConfig::FilterConfig(envoy::extensions::filters::http::wasm::v3::Wasm const&, Envoy::Server::Configuration::FactoryContext&) + 44
    26  envoy-static                        0x0000000105bcb27c std::__1::__shared_ptr_emplace<Envoy::Extensions::HttpFilters::Wasm::FilterConfig, std::__1::allocator<Envoy::Extensions::HttpFilters::Wasm::FilterConfig> >::__shared_ptr_emplace<envoy::extensions::filters::http::wasm::v3::Wasm const&, Envoy::Server::Configuration::FactoryContext&>(std::__1::allocator<Envoy::Extensions::HttpFilters::Wasm::FilterConfig>, envoy::extensions::filters::http::wasm::v3::Wasm const&, Envoy::Server::Configuration::FactoryContext&) + 148
    27  envoy-static                        0x0000000105bcaf0c std::__1::__shared_ptr_emplace<Envoy::Extensions::HttpFilters::Wasm::FilterConfig, std::__1::allocator<Envoy::Extensions::HttpFilters::Wasm::FilterConfig> >::__shared_ptr_emplace<envoy::extensions::filters::http::wasm::v3::Wasm const&, Envoy::Server::Configuration::FactoryContext&>(std::__1::allocator<Envoy::Extensions::HttpFilters::Wasm::FilterConfig>, envoy::extensions::filters::http::wasm::v3::Wasm const&, Envoy::Server::Configuration::FactoryContext&) + 44
    28  envoy-static                        0x0000000105bcadc0 std::__1::shared_ptr<Envoy::Extensions::HttpFilters::Wasm::FilterConfig> std::__1::allocate_shared<Envoy::Extensions::HttpFilters::Wasm::FilterConfig, std::__1::allocator<Envoy::Extensions::HttpFilters::Wasm::FilterConfig>, envoy::extensions::filters::http::wasm::v3::Wasm const&, Envoy::Server::Configuration::FactoryContext&, void>(std::__1::allocator<Envoy::Extensions::HttpFilters::Wasm::FilterConfig> const&, envoy::extensions::filters::http::wasm::v3::Wasm const&, Envoy::Server::Configuration::FactoryContext&) + 100
    29  envoy-static                        0x0000000105bc6170 std::__1::shared_ptr<Envoy::Extensions::HttpFilters::Wasm::FilterConfig> std::__1::make_shared<Envoy::Extensions::HttpFilters::Wasm::FilterConfig, envoy::extensions::filters::http::wasm::v3::Wasm const&, Envoy::Server::Configuration::FactoryContext&, void>(envoy::extensions::filters::http::wasm::v3::Wasm const&, Envoy::Server::Configuration::FactoryContext&) + 80
    30  envoy-static                        0x0000000105bc60bc Envoy::Extensions::HttpFilters::Wasm::WasmFilterConfig::createFilterFactoryFromProtoTyped(envoy::extensions::filters::http::wasm::v3::Wasm const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Envoy::Server::Configuration::FactoryContext&) + 132
    31  envoy-static                        0x0000000105bc6530 Envoy::Extensions::HttpFilters::Common::FactoryBase<envoy::extensions::filters::http::wasm::v3::Wasm, envoy::extensions::filters::http::wasm::v3::Wasm>::createFilterFactoryFromProto(google::protobuf::Message const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Envoy::Server::Configuration::FactoryContext&) + 124
    32  envoy-static                        0x000000010c9674d0 Envoy::Http::FilterChainHelper<Envoy::Server::Configuration::FactoryContext, Envoy::Server::Configuration::NamedHttpFilterConfigFactory>::processFilter(envoy::extensions::filters::network::http_connection_manager::v3::HttpFilter const&, int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, bool, std::__1::list<std::__1::unique_ptr<Envoy::Config::ExtensionConfigProvider<Envoy::Filter::NamedHttpFilterFactoryCb>, std::__1::default_delete<Envoy::Config::ExtensionConfigProvider<Envoy::Filter::NamedHttpFilterFactoryCb> > >, std::__1::allocator<std::__1::unique_ptr<Envoy::Config::ExtensionConfigProvider<Envoy::Filter::NamedHttpFilterFactoryCb>, std::__1::default_delete<Envoy::Config::ExtensionConfigProvider<Envoy::Filter::NamedHttpFilterFactoryCb> > > > >&, Envoy::Http::DependencyManager&) + 1572

The x64_86 version one is fine.

@dio
Copy link
Member Author

dio commented Sep 25, 2022

Checking on https://github.com/nodejs/node/blob/dc4398c9cfb86754f69631a9e69da4314ac0cbea/deps/v8/src/base/platform/platform-posix.cc#L468 and compare it with the current linked v8 version (10.4.132.18): https://chromium.googlesource.com/v8/v8.git/+/refs/tags/10.4.132.18/src/base/platform/platform-posix.cc#468, the later has the following code (https://chromium.googlesource.com/v8/v8.git/+/refs/tags/10.4.132.18/src/base/platform/platform-posix.cc#478, due to: v8/v8@18159ba) but the node's one doesn't have it:

if (ret != 0) CHECK_EQ(ENOMEM, errno);

So I tested locally to remove that line, by appending the following diff block to bazel/v8.patch:

diff --git a/src/base/platform/platform-posix.cc b/src/base/platform/platform-posix.cc
index 131ff9614e..a43d247c8d 100644
--- a/src/base/platform/platform-posix.cc
+++ b/src/base/platform/platform-posix.cc
@@ -472,10 +472,12 @@ bool OS::SetPermissions(void* address, size_t size, MemoryPermission access) {
   int prot = GetProtectionFromMemoryPermission(access);
   int ret = mprotect(address, size, prot);

+#if not defined(V8_OS_DARWIN)
   // Any failure that's not OOM likely indicates a bug in the caller (e.g.
   // using an invalid mapping) so attempt to catch that here to facilitate
   // debugging of these failures.
   if (ret != 0) CHECK_EQ(ENOMEM, errno);
+#endif

   // MacOS 11.2 on Apple Silicon refuses to switch permissions from
   // rwx to none. Just use madvise instead.

I can load the .wasm on macOS M1 (by running the compiled envoy arm64 binary with configuration from examples/wasm-cc/envoy.yaml).

   wasm-cc (mac-wasm): uname -a
Darwin Dhis-MacBook-Air.local 21.6.0 Darwin Kernel Version 21.6.0: Mon Aug 22 20:20:05 PDT 2022; root:xnu-8020.140.49~2/RELEASE_ARM64_T8101 arm64
   wasm-cc (mac-wasm): file ~/src/envoy/bazel-bin/source/exe/envoy-static
/Users/dio/src/envoy/bazel-bin/source/exe/envoy-static: Mach-O 64-bit executable arm64
   wasm-cc (mac-wasm): pwd
/Users/dio/src/envoy/examples/wasm-cc
   wasm-cc (mac-wasm): ~/src/envoy/bazel-bin/source/exe/envoy-static -c envoy.yaml 
...
[2022-09-25 20:43:18.254][2501953][info][main] [source/server/server.cc:904] starting main dispatch loop

Do you think updating bazel/v8.patch (on main) makes sense?

@dio
Copy link
Member Author

dio commented Sep 26, 2022

Ah probably the right patch is to switch the place of checking (worth submitting to upstream?):

diff --git a/src/base/platform/platform-posix.cc b/src/base/platform/platform-posix.cc
index 131ff9614e..8699699861 100644
--- a/src/base/platform/platform-posix.cc
+++ b/src/base/platform/platform-posix.cc
@@ -472,11 +472,6 @@ bool OS::SetPermissions(void* address, size_t size, MemoryPermission access) {
   int prot = GetProtectionFromMemoryPermission(access);
   int ret = mprotect(address, size, prot);

-  // Any failure that's not OOM likely indicates a bug in the caller (e.g.
-  // using an invalid mapping) so attempt to catch that here to facilitate
-  // debugging of these failures.
-  if (ret != 0) CHECK_EQ(ENOMEM, errno);
-
   // MacOS 11.2 on Apple Silicon refuses to switch permissions from
   // rwx to none. Just use madvise instead.
 #if defined(V8_OS_DARWIN)
@@ -486,6 +481,11 @@ bool OS::SetPermissions(void* address, size_t size, MemoryPermission access) {
   }
 #endif

+  // Any failure that's not OOM likely indicates a bug in the caller (e.g.
+  // using an invalid mapping) so attempt to catch that here to facilitate
+  // debugging of these failures.
+  if (ret != 0) CHECK_EQ(ENOMEM, errno);
+
   if (ret == 0 && access == OS::MemoryPermission::kNoAccess) {
     // This is advisory; ignore errors and continue execution.
     USE(DiscardSystemPages(address, size));

@dio
Copy link
Member Author

dio commented Sep 26, 2022

Closed in favor of: #23243

@dio dio closed this as completed Sep 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
triage Issue requires triage
Projects
None yet
Development

No branches or pull requests

1 participant