ruby: fix building from source with brewed glibc #114313
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This change only affects Linux with |
XuehaiPan
force-pushed
the
ruby
branch
2 times, most recently
from
53b572e to
741b1af
Compare
XuehaiPan
force-pushed
the
ruby
branch
6 times, most recently
from
730f327 to
78b1a64
Compare
XuehaiPan
force-pushed
the
ruby
branch
4 times, most recently
from
2e110cc to
182f55f
Compare
Closed
|
Some facts:
|
XuehaiPan
commented
Dec 5, 2022
2 tasks
|
I'm not entirely opposed to this idea, but I'd still like to get a better idea of why these gems are failing to build. I strongly suspect there is some shim bypass occurring, and it would be much better if we could fix that issue rather trying to hack around building the gems. Importantly, now that you've identified the two gems that are responsible for the failures, we know the scope of the problem and can try to track down what is really happening. We may end up being able to solve this with a much simpler |
@danielnachun I agree. I tried my best to locate where the bug is but the bug does not make sense to me. It fails when called in a subprocess by The failed Code def self.make(dest_path, results, make_dir = Dir.pwd)
...
['clean', '', 'install'].each do |target|
# Pass DESTDIR via command line to override what's in MAKEFLAGS
cmd = [
*make_program,
*env,
target,
].reject(&:empty?)
begin
run(cmd, results, "make #{target}".rstrip, make_dir)
rescue Gem::InstallError
raise unless target == 'clean' # ignore clean failure
end
end
end
def self.run(command, results, command_name = nil, dir = Dir.pwd)
verbose = Gem.configuration.really_verbose
begin
rubygems_gemdeps, ENV['RUBYGEMS_GEMDEPS'] = ENV['RUBYGEMS_GEMDEPS'], nil
if verbose
puts("current directory: #{dir}")
p(command)
end
results << "current directory: #{dir}"
require "shellwords"
results << command.shelljoin
require "open3"
# Set $SOURCE_DATE_EPOCH for the subprocess.
build_env = {'SOURCE_DATE_EPOCH' => Gem.source_date_epoch_string}.merge(env)
output, status = begin
Open3.capture2e(build_env, *command, :chdir => dir)
rescue => error
raise Gem::InstallError, "#{command_name || class_name} failed#{error.message}"
end
if verbose
puts output
else
results << output
end
ensure
ENV['RUBYGEMS_GEMDEPS'] = rubygems_gemdeps
end
unless status.success?
results << "Building has failed. See above output for more information on the failure." if verbose
end
yield(status, results) if block_given?
unless status.success?
exit_reason =
if status.exited?
", exit code #{status.exitstatus}"
elsif status.signaled?
", uncaught signal #{status.termsig}"
end
raise Gem::InstallError, "#{command_name || class_name} failed#{exit_reason}"
end
endThe LogsReplacing However, call $ ./ruby --disable-gems -e 'require "open3"; puts Open3.capture2e(*["/bin/bash", "-c", "echo Message"])'
Message
pid 335201 exit 0The call stack:
|
@danielnachun I found a better fix by using the full path of Glibc libraries, and it works. Replace: $ patchelf --print-needed libruby.so.3.1.3
libz.so.1
libcrypt.so.2
libm.so.6
libc.so.6
ld-linux-x86-64.so.2with $ patchelf --print-needed libruby.so.3.1.3
libz.so.1
libcrypt.so.2
<glibc.opt_lib>/libm.so.6
<glibc.opt_lib>/libc.so.6
ld-linux-x86-64.so.2Although the bug is fixed, I still have no idea why it would happen. |
|
Thank you for working on this issue, @XuehaiPan! An anecdotal data point in case it's useful for troubleshooting, I tested this PR on CentOS 7: $ curl -sLO https://raw.githubusercontent.com/Homebrew/homebrew-core/bda5b1c4c9d9eaca8874e25a9779320917e31f08/Formula/ruby.rb
$ brew install ./ruby.rb
…
$ grep -m1 -A6 '/usr/include/math.h' /mnt/home/shaun.jackman/.cache/Homebrew/Logs/ruby/02.make
In file included from /usr/include/math.h:223,
from /mnt/home/shaun.jackman/.linuxbrew/Cellar/gcc/12.2.0/include/c++/12/cmath:45,
from ../../.././include/ruby/missing.h:19,
from ../../.././include/ruby/defines.h:79,
from ../../.././include/ruby/ruby.h:25,
from cxxanyargs.cpp:1:
/mnt/home/shaun.jackman/.linuxbrew/opt/glibc/include/bits/mathcalls-helper-functions.h:20:24: error: '__fpclassify' has not been declaredIt fails at this point because it's using the host's
HOMEBREW_VERSION: 3.6.14 ORIGIN: https://github.com/Homebrew/brew HEAD: 001bacee18c512657e1535dff757c8cfdaefc116 Last commit: 5 days ago Core tap ORIGIN: https://github.com/Homebrew/homebrew-core Core tap HEAD: 3f8a401283628ca6bdd1f3a12bc063912f5cd74d Core tap last commit: 3 hours ago Core tap branch: master HOMEBREW_PREFIX: /mnt/home/shaun.jackman/.linuxbrew HOMEBREW_REPOSITORY: /mnt/home/shaun.jackman/.linuxbrew/Homebrew HOMEBREW_CELLAR: /mnt/home/shaun.jackman/.linuxbrew/Cellar HOMEBREW_CASK_OPTS: [] HOMEBREW_EDITOR: vim HOMEBREW_MAKE_JOBS: 96 HOMEBREW_NO_AUTO_UPDATE: set HOMEBREW_NO_INSTALL_CLEANUP: set Homebrew Ruby: 2.6.8 => /mnt/home/shaun.jackman/.linuxbrew/Homebrew/Library/Homebrew/vendor/portable-ruby/2.6.8_1/bin/ruby CPU: 96-core 64-bit skylake Clang: 15.0.0 Git: 2.37.3 => /mnt/home/shaun.jackman/.linuxbrew/bin/git Curl: 7.79.1 => /bin/curl Kernel: Linux 5.10.135-122.509.amzn2.x86_64 x86_64 GNU/Linux OS: Amazon Linux release 2 (Karoo) (Karoo) Host glibc: 2.26 /usr/bin/gcc: 7.3.1 /usr/bin/ruby: 2.0.0 glibc: 2.35_1 gcc@11: N/A gcc: 12.2.0 xorg: N/A |
@sjackman I cannot find As I commented:
You can run $ docker run -it --rm -h ubuntu --pull always ghcr.io/homebrew/ubuntu20.04:latest
linuxbrew@ubuntu:~$ curl -sLO https://raw.githubusercontent.com/Homebrew/homebrew-core/bda5b1c4c9d9eaca8874e25a9779320917e31f08/Formula/ruby.rb
linuxbrew@ubuntu:~$ brew install --build-from-source ./ruby.rb
linuxbrew@ubuntu:~$ grep -r '/usr/' ~/.cache/Homebrew/Logs/ruby
/home/linuxbrew/.cache/Homebrew/Logs/ruby/05.make:./miniruby -I./lib -I. -I.ext/common ./tool/runruby.rb --extout=.ext -- --disable-gems -r./x86_64-linux-fake ./tool/rbinstall.rb --make="/usr/bin/make" --dest-dir="" --extout=".ext" --ext-build-dir="./ext" --mflags="-j12 --jobserver-auth=4,5" --make-flags=" -j12 --jobserver-auth=4,5" --data-mode=0644 --prog-mode=0755 --installed-list .installed.list --mantype="man" --gnumake --install=all --rdoc-output=".ext/rdoc" --html-output=".ext/html"
/home/linuxbrew/.cache/Homebrew/Logs/ruby/01.configure:MJIT_CC=/usr/bin/gcc
/home/linuxbrew/.cache/Homebrew/Logs/ruby/01.configure:checking for /usr/bin/gcc... /usr/bin/gcc
/home/linuxbrew/.cache/Homebrew/Logs/ruby/01.configure:checking for a BSD-compatible install... /usr/bin/install -c
/home/linuxbrew/.cache/Homebrew/Logs/ruby/01.configure:checking for a race-free mkdir -p... /usr/bin/mkdir -p
/home/linuxbrew/.cache/Homebrew/Logs/ruby/01.configure:checking for grep that handles long lines and -e... /usr/bin/grep
/home/linuxbrew/.cache/Homebrew/Logs/ruby/01.configure:checking for egrep... /usr/bin/grep -E
/home/linuxbrew/.cache/Homebrew/Logs/ruby/config.log: $ ./configure --prefix=/home/linuxbrew/.linuxbrew/Cellar/ruby/3.1.3 --enable-shared --disable-silent-rules --with-sitedir=/home/linuxbrew/.linuxbrew/lib/ruby/site_ruby --with-vendordir=/home/linuxbrew/.linuxbrew/lib/ruby/vendor_ruby --with-opt-dir=/home/linuxbrew/.linuxbrew/opt/libyaml:/home/linuxbrew/.linuxbrew/opt/[email protected]:/home/linuxbrew/.linuxbrew/opt/readline --without-gmp MJIT_CC=/usr/bin/gcc
/home/linuxbrew/.cache/Homebrew/Logs/ruby/config.log:/usr/bin/uname -p = x86_64
/home/linuxbrew/.cache/Homebrew/Logs/ruby/config.log:/usr/bin/arch -k = unknown
/home/linuxbrew/.cache/Homebrew/Logs/ruby/config.log:/usr/convex/getsysinfo = unknown
/home/linuxbrew/.cache/Homebrew/Logs/ruby/config.log:/usr/bin/hostinfo = unknown
/home/linuxbrew/.cache/Homebrew/Logs/ruby/config.log:/usr/bin/oslevel = unknown
/home/linuxbrew/.cache/Homebrew/Logs/ruby/config.log:PATH: /usr/bin/
/home/linuxbrew/.cache/Homebrew/Logs/ruby/config.log:PATH: /usr/sbin/
/home/linuxbrew/.cache/Homebrew/Logs/ruby/config.log:configure:9142: checking for /usr/bin/gcc
/home/linuxbrew/.cache/Homebrew/Logs/ruby/config.log:configure:9177: result: /usr/bin/gcc
/home/linuxbrew/.cache/Homebrew/Logs/ruby/config.log:configure:9873: result: /usr/bin/install -c
/home/linuxbrew/.cache/Homebrew/Logs/ruby/config.log:configure:9929: result: /usr/bin/mkdir -p
/home/linuxbrew/.cache/Homebrew/Logs/ruby/config.log:configure:25260: result: /usr/bin/grep
/home/linuxbrew/.cache/Homebrew/Logs/ruby/config.log:configure:25333: result: /usr/bin/grep -E
/home/linuxbrew/.cache/Homebrew/Logs/ruby/config.log:ac_cv_path_EGREP='/usr/bin/grep -E'
/home/linuxbrew/.cache/Homebrew/Logs/ruby/config.log:ac_cv_path_GREP=/usr/bin/grep
/home/linuxbrew/.cache/Homebrew/Logs/ruby/config.log:ac_cv_path_MJIT_CC=/usr/bin/gcc
/home/linuxbrew/.cache/Homebrew/Logs/ruby/config.log:ac_cv_path_install='/usr/bin/install -c'
/home/linuxbrew/.cache/Homebrew/Logs/ruby/config.log:ac_cv_path_mkdir=/usr/bin/mkdir
/home/linuxbrew/.cache/Homebrew/Logs/ruby/config.log:EGREP='/usr/bin/grep -E'
/home/linuxbrew/.cache/Homebrew/Logs/ruby/config.log:GREP='/usr/bin/grep'
/home/linuxbrew/.cache/Homebrew/Logs/ruby/config.log:MAKEDIRS='/usr/bin/mkdir -p'
/home/linuxbrew/.cache/Homebrew/Logs/ruby/config.log:MJIT_CC='/usr/bin/gcc'
/home/linuxbrew/.cache/Homebrew/Logs/ruby/config.log:MKDIR_P='/usr/bin/mkdir -p'
/home/linuxbrew/.cache/Homebrew/Logs/ruby/config.log:configure_args=' '\''--prefix=/home/linuxbrew/.linuxbrew/Cellar/ruby/3.1.3'\'' '\''--enable-shared'\'' '\''--disable-silent-rules'\'' '\''--with-sitedir=/home/linuxbrew/.linuxbrew/lib/ruby/site_ruby'\'' '\''--with-vendordir=/home/linuxbrew/.linuxbrew/lib/ruby/vendor_ruby'\'' '\''--with-opt-dir=/home/linuxbrew/.linuxbrew/opt/libyaml:/home/linuxbrew/.linuxbrew/opt/[email protected]:/home/linuxbrew/.linuxbrew/opt/readline'\'' '\''--without-gmp'\'' '\''MJIT_CC=/usr/bin/gcc'\'' '\''CC=gcc-12'\'' '\''CXX=g++-12'\'''
/home/linuxbrew/.cache/Homebrew/Logs/ruby/config.log:oldincludedir='/usr/include' |
|
@XuehaiPan, interesting, I would use this |
Bo98
reviewed
Dec 19, 2022
Comment on lines
+114
to
+113
| if glibc.any_version_installed? | ||
| # Change to full path to Homebrew Glibc | ||
| %w[libc.so.6 libm.so.6].each do |lib| | ||
| system "patchelf", "--replace-needed", lib, "#{glibc.opt_lib}/#{lib}", | ||
| "libruby.so.#{version}" | ||
| end | ||
| end |
There was a problem hiding this comment.
Is it not using our ld.so symlink? That sounds like the core issue.
|
Fixed by #118926. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
brew install --build-from-source <formula>, where<formula>is the name of the formula you're submitting?brew test <formula>, where<formula>is the name of the formula you're submitting?brew audit --strict <formula>(after doingbrew install --build-from-source <formula>)? If this is a new formula, does it passbrew audit --new <formula>?Fixes #110558