nokogiri 1.6.0 fails to load embedded version of libxml2 if absolute path changes #923
Conversation
|
This might be related so I'm going to stick this here for now but if it's not then I'll move it to it's own ticket. We have a similar problem but in a way a bit opposite of his result. We package our production scripts and send them off and they too get untared into a different directory. Instead of doing what @ConradIrwin states, for us, it claims that it was built against 2.8.0 of libxml2 and loaded 2.9.0 through dynamic linking, none of our dev or production servers contain libxml 2.8.0 and it was in-fact built against 2.9.0 but this was triggered by the paths changing and it being unpackaged into /srv instead of /home/user/development/user/project. The message: |
|
This problem also makes it difficult to build packages for nokogiri, as I used to do here. |
|
I'm seeing this problem, and I agree it's an issue. Have we any expectation that this will be fixed ? I've just mailed the mailing list. I would expect that libxml2_path would refer to the relative ./vendor/bundle path at the time of execution not the ./vendor/bundle path at the time it was built. https://gist.github.com/sage-andrew-taylor/cbdd3c6852e2490e5e00 Thanks andrew |
|
We're running into this issue on Heroku as well. Our build environment is different than our runtime, which is spewing this error. |
|
I guess we should link the bundled libraries statically. Since they are only used by nokogiri.so, there is no benefit we can get from dynamic linking. Static linking allows us to remove the stuff "ports" have installed after successfully building the extension as a bonus. |
|
+1 on static linking. |
|
Hello. This is a message for people following the URL in the message shown by the new version of the Heroku Ruby buildpack, as of heroku/heroku-buildpack-ruby#124. Since that landed, I've been unable to deploy because my slug build times went over 15 minutes. Turns out this is because the new version of the buildpack clears the Bundler cache to get rid of any old builds of Nokogiri. If you have enough dependencies, that slows things down enough to time out a slug build. Fortunately I've solved the problem. If you've run into this problem then your cache has already been cleared, so the buildpack no longer needs to clear it for you. I adjusted the buildpack on a branch to skip clearing the cache. First, point to the tweaked buildpack: $ heroku config:set [-a app-name] BUILDPACK_URL=https://github.com/nerdfighteria/heroku-buildpack-ruby.git#nokogiri-busted-cache-fixThen push to deploy as normal. It's possible you'll need to try to deploy more than once: each time you'll get through re-caching more gems. (That would mean you have a whole lot of dependencies, though.) Once you've deployed, you can switch back to the main-line buildpack. In the future, it will see that you've built with a newer version of the buildpack and that you therefore don't need your cache cleared. $ heroku config:unset [-a app-name] BUILDPACK_URLHope that's helpful to someone! |
ghost
assigned
|
I'm currently working on static linking. |
|
I've just pushed my commits to the static_clean branch which makes nokogiri statically link the packaged libraries by default. Would you guys give it a try? Run @flavorjones Could you review my changes? The branch also includes a patch that cleans files generated by ports that are unnecessary to run nokogiri, which will hopefully eliminate recent complaints on how nokogiri installation is bloated. |
…ions.
Now that we use packaged libraries by default, Nokogiri should build
for most usrs without bothering system libraries.
We have the options --with-{xml2,xslt,exslt}-{dir,include,lib}=dir
supported and also consult pkg-config(1) for libxml2, libxslt and
libexslt. What else should you want?
A new option --disable-static allows user to link packaged libraries dynaminally. Fix priority problems with include directories and library directories by manipulating $CPPFLAGS, $LIBPATH and $libs correctly.
|
@flavorjones What do you think about the changes? |
|
@flavorjones bump from gogaruco :) |
|
@knu: I did some testing of your static_clean branch on Ubuntu-13.10. After When installed with When installed with default params it results in the following shared object: I wonder about the missing reference to liblzma although Both variants are usable to run Beside that I'm not sure if extconf.rb lines 244-258 are really necessary, but I didn't do any tests to that. Line 272 is probably not very portable across compilers other than gcc. By all means the clean step is a clever mechanism. |
|
@larskanis Thanks for testing. Here's a couple of comments as a reponse.
|
|
I'll merge this after making it an option to entirely remove the tmp directory, unless any objection is raised. |
|
I made it just remove the ports build directory by default. |
|
Where is this at? When will nokogiri go back to building with system libs? |
As of nokogiri 1.6.0, they default to using the vendored libxml2 libs. This is problematic for two reasons. First, with the way the heroku build environment works, sparklemotion/nokogiri#923. This means it won't link nokogiri.so properly since it's dependent on hardcoded paths that don't exist during runtime. Second, compiling libxml2 and friends is unnecessary since we have them already setup. We should skip this to speed up deploys.
As of nokogiri 1.6.0, they default to using the vendored libxml2 libs. This is problematic for two reasons. First, with the way the heroku build environment works, sparklemotion/nokogiri#923. This means it won't link nokogiri.so properly since it's dependent on hardcoded paths that don't exist during runtime. Second, compiling libxml2 and friends is unnecessary since we have them already setup. We should skip this to speed up deploys.
Our build system does a
bundle install --path ./bundleron one machine; then tars up the result and sends it to the rest of production.Unfortunately the tar-ball gets extracted into a different directory on the production servers from the temporary build server directory, which means that the absolute path hard-coded into
nokogiri.soat compile time is wrong. This means that nokogiri picks up the system version of libxml2 and not the embedded version and you get a warning on start-up.I think this is fixable on Linux by using Dylib.dlopen to open the libxml2.so file eagerly; not certain the correct way to fix on a Mac.
To replicate see the instructions at https://github.com/ConradIrwin/nokogiri-test.