Install fails on openSUSE with site-config and lib64
#1562
Comments
|
Fails with the same symptoms for the newly released 1.7.0 version (builds libraries in lib64, looks for them in lib). |
mbixby17451
changed the title
|
HUGE +1 ON THIS Have been working through this for days trying to figure out the problem with building nokogiri 1.7.0 on OpenSUSE Leap 42.1. This issue has existed for more than a year and is noted in #1334 by @abbjohn; @cabo helped but ultimately didn't reach any conclusion other than "It built a lib64 dir and nokogiri wants a lib dir." William Pursell provided a StackOverflow answer that helps: http://stackoverflow.com/questions/9247769/lib-directory-on-64-bit-opensuse-distribution may have some clues. It leads me to /usr/share/site/x86_64-unknown-linux-gnu, which is what sets libdir if it's not already set, and seems to be guessing lib64 vs. lib. As I'm building via bundle install, and doing that via a cap deploy with an rbenv installation, there's a fair number of layers to get through to influence Nokogiri's build. Adding my info for good Google Fu as it took me a while to get here. gem_make.out: mkmf.log: /usr/share/site/x86_64-unknown-linux-gnu: |
|
Hi, Thanks for raising this issue. We'd certainly be willing to accept a patch to support openSUSE. Unfortunately, I've never used SUSE or openSUSE and am not prepared to try to find a working configuration and diagnose this issue. I'd welcome some instructions on how to prepare a docker image to reproduce what you're seeing, but generally my expectation is that people with an interest in Nokogiri support for a distro will help provide that support (e.g. #1543 for OpenBSD support). Make sense? |
|
First, thanks for taking the time, and for Nokogiri. I use the heck out of it, so I really appreciate what you and Team Nokogiri have made here. I wish I had the time or expertise to make a larger contribution here; sadly, I've spent time I don't have figuring this out, and I don't have more than a well-specified, easy to reproduce bug report, so I'll likely have to move on and That said, I don't think this is strongly related to OpenSUSE since I can reproduce it on a Mac. This seems strongly related to how Nokogiri uses configure and builds its own libxml/libxslt. I know far less about how this works than you and Team Nokogiri, and I've already proven I'm not very effective at figuring it out. I've already given all of the time I can without blowing a deadline; I wish I could do more, but I've got to move on now. How to Reproduce on MacOS (or any other 64-bit *nix)If you take I'm hoping that's enough to make it straightforward for someone with more expertise in Nokogiri's build system(s) to hunt down and squash this bug. |
|
Also for posterity: Installing nokogiri manually on OpenSUSE Leap 42.1:I can't say I understand why I had to specify those dirs as that's the system default, but it works and I'm in a hurry. Also documented in a gist that I may update someday. |
|
I've tracked it down to hardcoded 'lib' in ext/nokogiri/extconf.rb: if static_p
$libs = $libs.shellsplit.map do |arg|
case arg
when '-lxml2'
File.join(libxml2_recipe.path, 'lib', lib_a(arg))
when '-lxslt', '-lexslt'
File.join(libxslt_recipe.path, 'lib', lib_a(arg))
else
arg
end
end.shelljoin
endSUSE like MacOS uses a CONFIG_SITE script to tell autoconf to use lib64, which causes trouble for the hardcoded 'lib' references above. Hacking at my failed gem install, modifying the code to look for 'lib64' library existence and then falling back to 'lib', allows extconf.rb to complete and generate a Makefile which can then build nokogiri. This is the solution I chose (with a similar permutation for the xslt case: (lib64 = File.join(libxml2_recipe.path, 'lib64', lib_a(arg))) && File.exist?(lib64) ? lib64 : File.join(libxml2_recipe.path, 'lib', lib_a(arg)) I chose that strategy because the other options seemed dubious and brittle. The xml2 and xslt libraries have their path information stored in libtool *.la files, pkgconfig *.pc files, and *-config scripts. I couldn't find documentation on libtool *.la or any libtool script way of extracting the info. I didn't want to introduce any new dependency on pkgconfig. And parsing or executing *-config shell scripts would just be too ugly. I need to get all of the nokogiri development dependency cruft together today to try to formalize this fix into a pull. |
|
Pull request #1574 created. I'm new to pull requests, so please be gentle. It passes all tests except for 4 skips which I can't seem to get it to divulge details on (adding --verbose doesn't change anything). Fabulous run in 6.871694s, 201.4059 runs/s, 578.1689 assertions/s. 1384 runs, 3973 assertions, 0 failures, 0 errors, 4 skips You have skipped tests. Run with --verbose for details. |
|
+1 |
|
Ok, all, I need more help understanding what's going on here. I went through the exercise of trying to reproduce what you're seeing in an opensuse docker image. Running this compiles and installs successfully: So, I still am not able to reproduce what y'all are seeing. Help me do that, so I can fix this properly? Finally, when I try to use that installation of nokogiri, I get a new error y'all haven't mentioned, and would like some help diagnosing it: I imagine this is cause by some related path problem; but I really don't want to start evaluating PRs for this unless I can reproduce what you're seeing, and I haven't managed to do that yet. Can someone help me? |
|
Also - please wrap your logs with the triple-backtick so they format well. I've gone through all the previous comments on this thread and done it for you. |
|
I need to get to the next milestone of what my day job pays me for ;-) before I can play around with this in a docker container. Hopefully sometime next week. |
|
I was able to reproduce my original problem using the docker opensuse 42.1 container. My day job web application is deployed by building ruby (and a lot of other stuff) from source into a non-system location on the target system, followed by installing Rails and various other dependent gems including nokogiri. On opensuse, the site-config package deploys a bit of autoconf script that sets up the lib64 naming convention. This script is specified by the env var CONFIG_SITE. When installing opensuse from an ISO, you get site-config by default and the default shell profiles set CONFIG_SITE. The docker container is too stripped down and does not include site-config and the other glue which sets up the env var via shell profiles. Once you make the docker image more real-life, the nokogiri failure is reproducible. Invoke the docker image as above: docker run -it opensuse:42.1 bash Then from within the docker bash shell, do the following to reproduce my original problem: zypper install -y aaa_base gcc libopenssl-devel make site-config tar wget zlib-devel The nokogiri gem install will fail the same way as in my original posting above, i.e. looking for its bundled libs in a "lib" subdirectory rather than the "lib64" subdirectory where autoconf decided they would be built. |
|
@mbixby17451 It looks like you're building your own Ruby; I'd like to eliminate the possibility that the problem you're seeing is due to how Ruby is built, versus how Nokogiri behaves on OpenSuse. Can you reproduce this issue using the |
|
@flavorjones I provided instructions above to reproduce on a MacBook Pro. They work using any version of Ruby. I don't think this is a OpenSUSE issue; this is the thing @mbixby17451 pointed out: there's a hard-coded 'lib' where Nokogiri should be using the appropriate variable. |
|
@purp Thanks for the additional comments. However, I've not been able to reproduce this (I'm not a Mac user); and unless and until I can reproduce it, I can't fix it (nor am I comfortable accepting a blind fix from a non-core contributor). Can you help me reproduce this issue? |
|
@flavorjones Totally understand the (proper) caution about taking code from randos like me. I can certainly try to help reproduce it. Did you try the steps I gave in my earlier comment? I think if you put that content into a file, set that environment variable, and try to install nokogiri, it should happen in any OS on any architecture. If that fails to reproduce the issue, tell me what OS/arch you're using and I'll see if I can manage to reproduce there. |
|
OpenSuse Leap 42.2 user here. Just want to add that I have to use this workaround: However, |
|
@purp I read your steps for reproducing earlier, but I have to admit I don't quite understand them. As I asked in my earlier comment, the ideal is for someone to provide instructions on how to reproduce this in a Docker image. And I showed my attempt to reproduce this in Docker in another earlier comment, so really I just want guidance on how to see what you're seeing. For what it's worth, my hesitation has nothing to do with you being a "rando" ;). It has to do with a) understanding the underlying issue so that I can b) determine if the fix is appropriate or whether there's a better fix, and c) maintain and support that fix in the future. I'll re-show my work. Using the If I do this: it totally works: If I do this: it totally works: Note that it finds the What am I missing? |
|
My Docker recipe above for reproducing the failure in my usage case (non-system Ruby) builds that Ruby with all default options except for directory prefix (necessary for non-system build) and suppression of doc generation (why spend the time?). I can't make it any more clear or stripped down than that and still reflect my usage case where nokogiri is failing. Nokogiri itself supports the same usage case of bundling essential dependencies and building in non-system locations rather than relying possibly incompatible system packages. So what I'm doing with a non-system Ruby should not be particularly surprising or unreasonable. |
|
@mbixby17451 I asked in my comment above if you could reproduce this issue with the system Ruby because I'm trying to bisect the problem and remove variables until it's clear where the problem lies. if you can't reproduce this issue using the system ruby, then can you help me understand why it's assumed the problem is with Nokogiri and not with how the non-system Ruby is being built? |
|
I'm still digging ... I think I've reproduced this root issue (albeit with different symptoms) using the system ruby. Give me a bit. |
|
I believe the issue has to do with how the shared object file On Ubuntu and other Linux systems, the makefile created by the key portions being the The same line in OpenSuse's makefile: Anybody know why OpenSuse is being difficult about this? I'll keep poking. |
|
OK, so with the Ruby on my Ubuntu development machine: but on OpenSuse: (this is with the system ruby, haven't tried with my own compiled Ruby yet) ... this is part of the problem. |
|
I am unable to get the OpenSUSE system ruby to fail. RPATHFLAG is as you note on system vs. non-system ruby. However, I can turn my docker nokogiri failure duplication recipe into successful nokogiri installation if I unset the CONFIG_SITE env var (that comes from the OpenSUSE site-config and aaa_base packages) prior to running the non-system ruby configure script. CONFIG_SITE points to a bit of autoconf script that imposes the "lib64" naming convention. Thus, when I unset that prior to building a non-system ruby, the usual "lib" naming convention is used, matching the expectation of the hardcoded "lib" in nokogiri. RPATHFLAG is the same for me when building non-system ruby in both the success and failure cases. Thus, it alone seems unlikely to be the cause of the failure. |
|
This seems clearly tied to the CONFIG_SITE/autoconf settings, as that's the way I can reproduce it on any 64-bit x86 system, Ubuntu included.
Contents of |
|
OK, some more progress: When I build my own ruby, using @mbixby17451's recipe from this comment, And, what's more, nokogiri installs properly: Not sure why I can't reproduce @mbixby17451's results here ... going to try again from scratch to see if any additional zypper packages I may have installed made a difference. |
|
OK, I've confimed that when |
flavorjones
changed the title
lib64
flavorjones
changed the title
lib64lib64
|
Hi, @flavorjones ... I don't understand why you've closed this. The @mbixby17451 noted the source of the issue, which is in Can you say more on why you chose to close this without further comment beyond understanding how it was broken? I'm hoping to stop using this workaround every time I install nokogiri on a 64-bit OpenSUSE system. And, as always, thanks to you and Team Nokogiri. If I didn't use it as much, this wouldn't be as painful to endure. |
|
Ahhhhh. Sorry, I'm clearly dumbing today in a "No Dumbing" area. Completely overlooked that it was closed with a commit. Thanks for the fix, and sorry for the noise! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
What is happening is that libxml2.a and libxslt.a get built beneath "lib64" subdirectories per the openSUSE standard, but then when the nokogiri code attempts to link to those libraries, it looks for them in a "lib" subdirectory instead of "lib64".
So it seems that there may be a hardcoded "lib" subdirectory path somewhere or else the various config/build tools aren't properly agreeing on the correct path.
This version of the nokogiri gem installs correctly on CentOS (where "lib" is the standard).
I've seen similar issues posted here, but often the OP falls back to "--use-system-libraries" or else fails to respond with additional requested details and then the issue gets closed. This is a priority problem for me and I don't like the system libraries workaround.
Let me know if you require further diagnostic information. I rely on "It Just Works" and so I don't have familiarity with pkgconfig and friends, so if you need me to make tweaks there, please give detailed instructions. Thanks.
The failure logs below come from a cleanly installed openSUSE Leap 42.2 virtual machine.
What is the output of
gem install?What are the contents of the
mkmf.logfile?What operating system are you using?
openSUSE Leap 42.2 x86_64 (as well as openSUSE 12.3 x86_64) with ruby 2.3.1
The text was updated successfully, but these errors were encountered: