Update Ruby gem with arm64 binaries (Apple M1 support) #8199

mloughran · 2021-01-12T09:29:09Z

The Ruby gem is shipped with compiled extensions (via https://github.com/rake-compiler/rake-compiler), but it is missing arm64 binaries.

This results in errors like the following when installing the gem and requiring:

$ gem install google-protobuf

$ ruby -e 'require "google/protobuf"'
Traceback (most recent call last):
	2: from -e:1:in `<main>'
	1: from <snip>/rubygems/core_ext/kernel_require.rb:92:in `require'
<snip>/rubygems/core_ext/kernel_require.rb:92:in `require': cannot load such file -- google/protobuf (LoadError)
	6: from -e:1:in `<main>'
	5: from <snip>/rubygems/core_ext/kernel_require.rb:156:in `require'
	4: from <snip>/rubygems/core_ext/kernel_require.rb:168:in `rescue in require'
	3: from <snip>/rubygems/core_ext/kernel_require.rb:168:in `require'
	2: from <snip>/gems/google-protobuf-3.14.0-universal-darwin/lib/google/protobuf.rb:49:in `<top (required)>'
	1: from <snip>/rubygems/core_ext/kernel_require.rb:92:in `require'
<snip>/rubygems/core_ext/kernel_require.rb:92:in `require': dlopen(<snip>/gems/google-protobuf-3.14.0-universal-darwin/lib/google/2.7/protobuf_c.bundle, 9): no suitable image found.  Did find: (LoadError)
	<snip>/gems/google-protobuf-3.14.0-universal-darwin/lib/google/2.7/protobuf_c.bundle: mach-o, but wrong architecture
	<snip>/gems/google-protobuf-3.14.0-universal-darwin/lib/google/2.7/protobuf_c.bundle: mach-o, but wrong architecture - <snip>/gems/google-protobuf-3.14.0-universal-darwin/lib/google/2.7/protobuf_c.bundle
	7: from -e:1:in `<main>'
	6: from <snip>/rubygems/core_ext/kernel_require.rb:156:in `require'
	5: from <snip>/rubygems/core_ext/kernel_require.rb:168:in `rescue in require'
	4: from <snip>/rubygems/core_ext/kernel_require.rb:168:in `require'
	3: from <snip>/gems/google-protobuf-3.14.0-universal-darwin/lib/google/protobuf.rb:48:in `<top (required)>'
	2: from <snip>/gems/google-protobuf-3.14.0-universal-darwin/lib/google/protobuf.rb:51:in `rescue in <top (required)>'
	1: from <snip>/rubygems/core_ext/kernel_require.rb:92:in `require'
<snip>/rubygems/core_ext/kernel_require.rb:92:in `require': cannot load such file -- google/protobuf_c (LoadError)

A workaround is to force compilation of native extensions on install which works perfectly:

$ gem install google-protobuf --platform=ruby

or globally via this bundler setting:

$ bundle config set force_ruby_platform true

Related to #8062

The text was updated successfully, but these errors were encountered:

LucaProvencal · 2021-02-04T18:51:49Z

Confirmed that

$ bundle config set force_ruby_platform true

in combination with grpc v1.35.0 upgrade solved this issue for me.

NielsKSchjoedt · 2021-03-04T08:30:14Z

This still does not work for me. I have v. 3.15.3 installed and it throws this:

Traceback (most recent call last):
       16: from bootsnap (1.7.0) lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require'
       15: from google-protobuf-3.15.3-universal (darwin) lib/google/protobuf/any_pb.rb:4:in `<main>'
       14: from zeitwerk (2.4.2) lib/zeitwerk/kernel.rb:34:in `require'
       13: from bootsnap (1.7.0) lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:31:in `require'
       12: from bootsnap (1.7.0) lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require_with_bootsnap_lfi'
       11: from bootsnap (1.7.0) lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register'
       10: from bootsnap (1.7.0) lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `block in require_with_bootsnap_lfi'
        9: from bootsnap (1.7.0) lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require'
        8: from google-protobuf-3.15.3-universal (darwin) lib/google/protobuf.rb:48:in `<main>'
        7: from google-protobuf-3.15.3-universal (darwin) lib/google/protobuf.rb:51:in `rescue in <main>'
        6: from zeitwerk (2.4.2) lib/zeitwerk/kernel.rb:34:in `require'
        5: from bootsnap (1.7.0) lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:44:in `require'
        4: from bootsnap (1.7.0) lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require_with_bootsnap_lfi'
        3: from bootsnap (1.7.0) lib/bootsnap/load_path_cache/loaded_features_index.rb:89:in `register'
        2: from bootsnap (1.7.0) lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `block in require_with_bootsnap_lfi'
        1: from bootsnap (1.7.0) lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require'
LoadError (cannot load such file -- google/protobuf_c)

If I try to load the gem locally and alter lines inside lib/google/protobuf.rb like so:

  # begin
    require "google/#{RUBY_VERSION.sub(/\.\d+$/, '')}/protobuf_c"
  # rescue LoadError
  #   require 'google/protobuf_c'
  # end

I get:

LoadError (dlopen(/Users/niels-kristian/.rvm/gems/ruby-2.7.2@rails6/gems/google-protobuf-3.15.3-universal-darwin/lib/google/2.7/protobuf_c.bundle, 9): no suitable image found.  Did find:)
	/Users/niels-kristian/.rvm/gems/ruby-2.7.2@rails6/gems/google-protobuf-3.15.3-universal-darwin/lib/google/2.7/protobuf_c.bundle: mach-o, but wrong architecture
	/Users/niels-kristian/.rvm/gems/ruby-2.7.2@rails6/gems/google-protobuf-3.15.3-universal-darwin/lib/google/2.7/protobuf_c.bundle: stat() failed with errno=25 - /Users/niels-kristian/.rvm/gems/ruby-2.7.2@rails6/gems/google-protobuf-3.15.3-universal-darwin/lib/google/2.7/protobuf_c.bundle

Any clue why this happens?

NielsKSchjoedt · 2021-03-04T08:39:38Z

Okay I tried updating to v 3.15.4 then it worked. 1 hour later it suddenly stopped working again 🤯 Something is really fishy...

LoadError (dlopen(/Users/niels-kristian/.rvm/gems/ruby-2.7.2@rails6/gems/google-protobuf-3.15.4-universal-darwin/lib/google/2.7/protobuf_c.bundle, 9): no suitable image found.  Did find:)
	/Users/niels-kristian/.rvm/gems/ruby-2.7.2@rails6/gems/google-protobuf-3.15.4-universal-darwin/lib/google/2.7/protobuf_c.bundle: mach-o, but wrong architecture
	/Users/niels-kristian/.rvm/gems/ruby-2.7.2@rails6/gems/google-protobuf-3.15.4-universal-darwin/lib/google/2.7/protobuf_c.bundle: mach-o, but wrong architecture - /Users/niels-kristian/.rvm/gems/ruby-2.7.2@rails6/gems/google-protobuf-3.15.4-universal-darwin/lib/google/2.7/protobuf_c.bundle

I just tried to checkout the ruby gem and see if it could build, it cannot: #8372

teddy1004 · 2021-05-17T04:52:26Z

solved on my new M1 Mac, try the following steps:

gem uninstall google-protobuf, and remove all google-protobuf
reinstall with gem install google-protobuf -v 3.15.8 --platform=ruby or any version you want

seems bundle config force ruby platform not working, we have to reinstall google-protobuf platform: ruby from scratch

ericboehs · 2021-08-30T18:32:31Z

None of these solutions worked for me on 3.17.3 on a 13" M1 MBP. I'm running the latest Monterey public beta (beta 5 I believe) so maybe that's the issue?

Installation went fine but when I go to require it I get a "No such file" error when looking for the arm64e bundle. Here's the full error:

.bundle/bundle/ruby/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/dependencies.rb:332:in `require': dlopen(.bundle/bundle/ruby/2.7.0/gems/google-protobuf-3.17.3-universal-darwin/lib/google/2.7/protobuf_c.bundle, 0x0009): tried: '.bundle/bundle/ruby/2.7.0/gems/google-protobuf-3.17.3-universal-darwin/lib/google/2.7/protobuf_c.bundle' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/local/lib/protobuf_c.bundle' (no such file), '/usr/lib/protobuf_c.bundle' (no such file) - .bundle/bundle/ruby/2.7.0/gems/google-protobuf-3.17.3-universal-darwin/lib/google/2.7/protobuf_c.bundle (LoadError)

I tried on Ruby 2.6.8p205 and 2.7.4p191. I tried recompiling Ruby several times using Xcode Command Line tools and the full version of Xcode (Mac App Store).

Overload119 · 2021-09-08T21:20:53Z

Is there anyway to force bundler to use the Rubygems version installed with --platform ruby ?

Whenever I run bundle I see it's using the universal-darwin version.
I've tried doing platform: :ruby to the line in the Gemfile but that doesn't work.

bouk · 2021-10-13T11:07:44Z

If you want to prevent bundler from installing these gems, add the following to you Gemfile:

# HACK(bouk): Overwrite Bundler's platform matcher to ignore universal CPU
# The protobuf and gRPC 'universal' macOS gems break on M1
module Bundler::MatchPlatform
  def match_platform(p)
    return false if ::Gem::Platform === platform && platform.cpu == "universal"
    Bundler::MatchPlatform.platforms_match?(platform, p)
  end
end

This will overwrite bundler's platform matcher to ignore any universal gems

mloughran · 2021-10-25T14:13:59Z

This was closed by #8232 (and released in 3.19.0).

ngan · 2021-10-25T14:58:42Z

I'm not sure if this is actually fixed. I'm still getting the same problem (protobuf_c.bundle: mach-o, but wrong architecture) after upgrading to 3.19.0.

I also don't see an arm64-darwin build for 3.19.0:

acozzette · 2021-10-28T18:39:54Z

I think some more work is required on the protobuf team's part before we can start publishing Mac arm64 binaries. The build is happening on an x86 machine and we probably need to tweak some things to get cross-compilation working.

jnarowski · 2021-12-15T15:20:19Z

+1 for this

dangerrg · 2022-01-07T10:47:25Z

I've tried all of the suggestions above, but still doesn't work for me:

I get this error:

/Users/dangerrg/.rvm/gems/ruby-3.0.3/gems/bootsnap-1.9.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `
require': dlopen(/Users/dangerrg/.rvm/gems/ruby-3.0.3/gems/digest-3.1.0/lib/digest.bundle, 0x0009): tried: '/Users/dange
rrg/.rvm/gems/ruby-3.0.3/gems/digest-3.1.0/lib/digest.bundle' (mach-o file, but is an incompatible architecture (have 'x
86_64', need 'arm64e')), '/usr/local/lib/digest.bundle' (no such file), '/usr/lib/digest.bundle' (no such file) - /Users
/dangerrg/.rvm/gems/ruby-3.0.3/gems/digest-3.1.0/lib/digest.bundle (LoadError)

I'm running on Mac M1:

About your application's environment
Rails version             7.0.0
Ruby version              ruby 3.0.3p157 (2021-11-24 revision 3fb7d2cadc) [arm64-darwin21]
RubyGems version          3.2.32
Rack version              2.2.3
Middleware                ActionDispatch::HostAuthorization, Rack::Sendfile, ActionDispatch::Static, ActionDispatch::Exe
cutor, ActionDispatch::ServerTiming, ActiveSupport::Cache::Strategy::LocalCache::Middleware, Rack::Runtime, Rack::Method
Override, ActionDispatch::RequestId, ActionDispatch::RemoteIp, Sprockets::Rails::QuietAssets, Rails::Rack::Logger, Actio
nDispatch::ShowExceptions, WebConsole::Middleware, ActionDispatch::DebugExceptions, ActionDispatch::ActionableExceptions
, ActionDispatch::Reloader, ActionDispatch::Callbacks, ActiveRecord::Migration::CheckPending, ActionDispatch::Cookies, A
ctionDispatch::Session::CookieStore, ActionDispatch::Flash, ActionDispatch::ContentSecurityPolicy::Middleware, ActionDis
patch::PermissionsPolicy::Middleware, Rack::Head, Rack::ConditionalGet, Rack::ETag, Rack::TempfileReaper
Application root          /Users/dangerrg/Documents/rails_7_projects/rails_7_blog
Environment               development
Database adapter          sqlite3
Database schema version   20220105220427

nickmealey · 2022-06-21T21:41:21Z

Updates on this issue? It appears to be ongoing and I'm only able to bundle with bundle config set force_ruby_platform true

JasonLunn · 2022-09-01T20:10:14Z

Similar to #8682, closing this because in version 3.22 protobuf is planning to experiment with releasing only a source gem.

If this experiment is unsuccessful and we revert to building binary gems for some platforms, we'll evaluate whether it makes sense to add arm64 darwin at that time.

CC: @deannagarcia @haberman

njmulsqb · 2023-01-03T07:40:22Z

I am running this on M2, facing the same issue

``require': dlopen(/Library/Ruby/Gems/2.6.0/gems/google-protobuf-3.21.12-x86_64-darwin/lib/google/2.6/protobuf_c.bundle, 0x0009): tried: '/Library/Ruby/Gems/2.6.0/gems/google-protobuf-3.21.12-x86_64-darwin/lib/google/2.6/protobuf_c.bundle' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Library/Ruby/Gems/2.6.0/gems/google-protobuf-3.21.12-x86_64-darwin/lib/google/2.6/protobuf_c.bundle' (no such file), '/Library/Ruby/Gems/2.6.0/gems/google-protobuf-3.21.12-x86_64-darwin/lib/google/2.6/protobuf_c.bundle' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')) - /Library/Ruby/Gems/2.6.0/gems/google-protobuf-3.21.12-x86_64-darwin/lib/google/2.6/protobuf_c.bundle (LoadError)`

dont know what to do?

njmulsqb · 2023-01-16T02:38:52Z

I am running this on M2, facing the same issue

``require': dlopen(/Library/Ruby/Gems/2.6.0/gems/google-protobuf-3.21.12-x86_64-darwin/lib/google/2.6/protobuf_c.bundle, 0x0009): tried: '/Library/Ruby/Gems/2.6.0/gems/google-protobuf-3.21.12-x86_64-darwin/lib/google/2.6/protobuf_c.bundle' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Library/Ruby/Gems/2.6.0/gems/google-protobuf-3.21.12-x86_64-darwin/lib/google/2.6/protobuf_c.bundle' (no such file), '/Library/Ruby/Gems/2.6.0/gems/google-protobuf-3.21.12-x86_64-darwin/lib/google/2.6/protobuf_c.bundle' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')) - /Library/Ruby/Gems/2.6.0/gems/google-protobuf-3.21.12-x86_64-darwin/lib/google/2.6/protobuf_c.bundle (LoadError)`

dont know what to do?

I resolved the issue by ditching homebrew and following the ruby installation docs provided on Jekyll site

Hazel233 · 2023-08-25T05:52:49Z

solved on my new M1 Mac, try the following steps:

gem uninstall google-protobuf, and remove all google-protobuf

reinstall with gem install google-protobuf -v 3.15.8 --platform=ruby or any version you want

seems bundle config force ruby platform not working, we have to reinstall google-protobuf platform: ruby from scratch

This really helps after searching for a lot methods.

haberman · 2023-08-28T22:33:53Z

Protobuf is releasing with arm64-darwin binary gems. For example, here is the latest one: https://rubygems.org/gems/google-protobuf/versions/3.24.2-arm64-darwin

However, the current gems require Ruby >=2.7, and I believe macOS is still shipping with Ruby 2.6.

So to use the current version of the protobuf package, you would need to install a separate Ruby from Homebrew (or from source).

quartzmo mentioned this issue Jan 29, 2021

Support for Apple M1 architecture googleapis/google-cloud-ruby#8959

Closed

perezd added the ruby label Feb 2, 2021

cabello mentioned this issue Feb 3, 2021

grpc_c.bundle fails to load on Apple M1 (wrong architecture) grpc/grpc#24846

Closed

ghost mentioned this issue Apr 1, 2021

google-protobuf dependency doesn't work on Apple Silicon pganalyze/pg_query#204

Closed

haberman mentioned this issue Apr 3, 2021

Ruby gem cannot build on ruby 2.7.2 on Mac M1 #8372

Closed

jtattermusch changed the title ~~Update Ruby gem with arm64 binaries (M1 support)~~ Update Ruby gem with arm64 binaries (Apple M1 support) Jun 1, 2021

elharo added mac packaging & distribution platform related Any issue releated to specific platform or OS labels Aug 30, 2021

haynesgt mentioned this issue Oct 14, 2021

Ruby: Gem version 3.11.2 doesn't load on Ruby 2.7 #7070

Closed

mloughran closed this as completed Oct 25, 2021

acozzette reopened this Oct 28, 2021

This was referenced Mar 14, 2022

I can't use bcrypt on Apple M1 Chip bcrypt-ruby/bcrypt-ruby#239

Open

Issue with devise and bcrypt on a macbook m1 rails install bcrypt-ruby/bcrypt-ruby#250

Open

JasonLunn closed this as completed Sep 1, 2022

Panajev mentioned this issue Sep 24, 2022

Issue running danger-junit on M1 CI machines orta/danger-junit#33

Closed

Panajev mentioned this issue Sep 24, 2022

Issue with gems on M1 that depends on ox ohler55/ox#297

Closed

hojongs mentioned this issue Jan 15, 2023

protobuf incompatible with arm64 #11439

Closed

Aikufurr mentioned this issue Apr 5, 2024

Docker container fails to start up with LoadError /usr/local/bundle/gems/bundler-2.3.3/exe/bundle Freika/dawarich#3

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update Ruby gem with arm64 binaries (Apple M1 support) #8199

Update Ruby gem with arm64 binaries (Apple M1 support) #8199

mloughran commented Jan 12, 2021

LucaProvencal commented Feb 4, 2021 •

edited

Loading

NielsKSchjoedt commented Mar 4, 2021 •

edited

Loading

NielsKSchjoedt commented Mar 4, 2021 •

edited

Loading

teddy1004 commented May 17, 2021 •

edited

Loading

ericboehs commented Aug 30, 2021 •

edited

Loading

Overload119 commented Sep 8, 2021

bouk commented Oct 13, 2021

mloughran commented Oct 25, 2021 •

edited

Loading

ngan commented Oct 25, 2021

acozzette commented Oct 28, 2021

jnarowski commented Dec 15, 2021

dangerrg commented Jan 7, 2022

nickmealey commented Jun 21, 2022

JasonLunn commented Sep 1, 2022

njmulsqb commented Jan 3, 2023

njmulsqb commented Jan 16, 2023

Hazel233 commented Aug 25, 2023

haberman commented Aug 28, 2023

Update Ruby gem with arm64 binaries (Apple M1 support) #8199

Update Ruby gem with arm64 binaries (Apple M1 support) #8199

Comments

mloughran commented Jan 12, 2021

LucaProvencal commented Feb 4, 2021 • edited Loading

NielsKSchjoedt commented Mar 4, 2021 • edited Loading

NielsKSchjoedt commented Mar 4, 2021 • edited Loading

teddy1004 commented May 17, 2021 • edited Loading

ericboehs commented Aug 30, 2021 • edited Loading

Overload119 commented Sep 8, 2021

bouk commented Oct 13, 2021

mloughran commented Oct 25, 2021 • edited Loading

ngan commented Oct 25, 2021

acozzette commented Oct 28, 2021

jnarowski commented Dec 15, 2021

dangerrg commented Jan 7, 2022

nickmealey commented Jun 21, 2022

JasonLunn commented Sep 1, 2022

njmulsqb commented Jan 3, 2023

njmulsqb commented Jan 16, 2023

Hazel233 commented Aug 25, 2023

haberman commented Aug 28, 2023

LucaProvencal commented Feb 4, 2021 •

edited

Loading

NielsKSchjoedt commented Mar 4, 2021 •

edited

Loading

NielsKSchjoedt commented Mar 4, 2021 •

edited

Loading

teddy1004 commented May 17, 2021 •

edited

Loading

ericboehs commented Aug 30, 2021 •

edited

Loading

mloughran commented Oct 25, 2021 •

edited

Loading