[openssl][0.9.24] Fail to build on hosts with openssl 1.1.1

[otavio]

Today I updated my host development machine and found that it was not building my source code anymore.

Got the following failure:

error: failed to run custom build command for `openssl v0.9.24`
process didn't exit successfully: `.../target/debug/build/openssl-4dcd4d0c1d4575a6/build-script-build` (exit code: 101)
--- stderr
thread 'main' panicked at 'Unable to detect OpenSSL version', .../.cargo/registry/src/github.aaakk.us.kg-1ecc6299db9ec823/openssl-0.9.24/build.rs:16:14
note: Run with `RUST_BACKTRACE=1` for a backtrace.

It needs to be upgraded to openssl-sys last releases. This is important as multiple vendors will start upgrading OpenSSL and more people will face it. For now, I downgraded my host to openssl 1.1.0

[andrzejressel]

I'm having the same error, but I think it's dependency problem and not library's. Current library version is 0.10.12 and I think we should send PR to dependencies to use the latest version.

[sfackler]

I would prefer that people update to the current release of this crate that already supports 1.1.1.

[est31]

Note that updating to the current release is often non-trivial and takes longer than it should.

E.g. I filed seanmonstar/reqwest#308 in June. It took until August until it was fixed on master. That's 8 months after the openssl 0.10 release. And there still isn't a released version of reqwest yet that ships with the newer openssl.

I do agree that it's prefferable if people updated to the current release of reqwest, but I don't think that letting compilation fail is a good way to convince them to do this update. If you want the ecosystem to switch to your newer versions more eagerly, you should bring it to them. Ask them to update it. But don't force them, please. The current way of treatment is just annoying everyone in the ecosystem.

[est31]

E.g. the iron framework still uses hyper 0.10 and depends on hyper-native-tls. And hyper-native-tls only supports native-tls 0.1 which only supports openssl 0.9. So what to do now? Sure, optimally iron updates its hyper dependency from 0.10 to 0.12, but this will neccessitate major changes and most likely get to users as well through a change in the iron API.

[andrzejressel]

Frankly, this example for me shows "fault" of iron, not openssl. As a library developer I should ensure that I'm using the latest dependencies. openssl-rust developer shouldn't care which version native-tls, hyper or iron uses - his job is to ensure compatibility with latest openssl.

[est31]

native-tls, hyper-native-tls are both maintained by the same person as the openssl crate is. Yet both required breaking changes to update from openssl 0.9 to 0.10.

[sfackler]

If you want the ecosystem to switch to your newer versions more eagerly, you should bring it to them.

There are 161 crates on crates.io which directly depend on rust-openssl, and 105 that directly depend on native-tls. It is not feasible for me to push out upgrades to all of those whenever I cut a new release.

But don't force them, please. The current way of treatment is just annoying everyone in the ecosystem.

What are your expectations around support? How many crates continue maintaining old major versions in perpetuity?

[mqus]

just a question for my usecase: I have openssl 1.0.2p installed alongside 1.1.1, how do I tell cargo to use only openssl1.0.2p?

On topic: I would really appreciate it if you push a 0.9 update but I completely understand your concerns and won't try to convince you otherwise.

[sfackler]

Set the OPENSSL_DIR environment variable to point to the location 1.0.2 is installed

…

On Sun, Sep 16, 2018 at 1:08 PM mqus ***@***.***> wrote: just a question for my usecase: I have openssl 1.0.2p installed alongside 1.1.1, how do I tell cargo to use only openssl1.0.2p? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#994 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABY2UdGF_QH79i_GQrDdD6go5v_CXX7Aks5ubq_HgaJpZM4WqWbx> .

[est31]

There are 161 crates on crates.io which directly depend on rust-openssl, and 105 that directly depend on native-tls. It is not feasible for me to push out upgrades to all of those whenever I cut a new release.

I think it's important to get the populous projects that are using your crate to use the newer version before you end support for the older one. And with enough time in between the two events so that users are given time to switch. The number of popolous projects is small, I'd say. Even simple pings would help: "hello, I'd like you to update to openssl 0.10, as openssl 0.9 will go out of maintenance on ABC"

What are your expectations around support? How many crates continue maintaining old major versions in perpetuity?

My expectation is that if there is a significant amount of users of a specific version of a crate, that version should still see some little level of support. Given that high-profile projects like reqwest or iron are still using openssl 0.9, this seems to be fulfilled.

[otavio]

I understand that ultimately, 0.9.x need to be abandoned and all energy and effort to be focused on 0.10.x or any other future release, however, we are facing some major changes in multiple projects and upgrades have been slowed down a bit.

The openssl upgrade from 1.1.0 to 1.1.1 is a minor release, from user point of view. I understand it ended having more features added along the way but adding support for existing features would be enough and provide an upgrade path for users.

[sfackler]

And with enough time in between the two events so that users are given time to switch.

0.10 was released 10 months ago. The 0.7.x series was last released over 2 years go, and is still downloaded 75-100 times a day. I have very little interest in year long maintenance periods for old releases.

Given that high-profile projects like reqwest or iron are still using openssl 0.9, this seems to be fulfilled.

Iron hasn't had a release in 10 months - is it abandoned? It's not clear to me why the onus is on me to maintain old releases rather than those high profile projects to update.

Supporting 1.1.1 in 0.9 is not necessarily a trivial one line change. The addition of TLS 1.3 is pretty significant, and 0.9 doesn't provide any of the APIs to interact or control that.

[est31]

Iron hasn't had a release in 10 months - is it abandoned?

Apparently iron has new maintainers, but things are going slowly: iron/iron#578

It's not clear to me why the onus is on me to maintain old releases rather than those high profile projects to update.

Ideally, it's not an either or, but a both: those projects make their update, while you provide minimal support for those older releases until a large chunk of your users has migrated.

Right now the situation is quite bad: the openssl update from 0.9 to 0.10 is in the (breaking) native-tls update from 0.1 to 0.2, and this update is only performed in the (breaking) reqwest update from 0.8 to 0.9, where 0.9 isn't even released yet so you must use it from git.

Supporting 1.1.1 in 0.9 is not necessarily a trivial one line change. The addition of TLS 1.3 is pretty significant, and 0.9 doesn't provide any of the APIs to interact or control that.

Thanks for the info! This info is in fact useful for a solution to the problem. Iif fixing it in openssl is not viable, what about trying to remove the semver breaking change one item higher in the chain?

For the native-tls crate, a new 0.1 branch release could be made from this commit... it doesn't seem to change any of the native-tls API and it's directly after the 1.5 release so it should be possible to do that?

For the hyper-native-tls crate that's creating this situation for iron, you could consider merging something like this pr and making a new release.

[sfackler]

native-tls 0.1 unfortunately can't take major version bumps to openssl because it's exposed as a public dependency (there are extension traits that allow you to downcast from the native-tls types to the openssl types). That's one of the things that 0.2 fixed.

[est31]

@sfackler oh I see. Thanks for the help.

I'll try to have a look whether the claims made in seanmonstar/reqwest#344 are true, that reqwest needs a breaking change as well, but if yes, then I guess we'll have to live with the current situation :/. Thanks again.

[dns2utf8]

@sfackler Thank you for putting so much time into it!
I am successfully building my applications with docker for now. Will look at it on the weekend and try to help release the fixed deps on the weekend.

[jebrosen]

native-tls 0.1 unfortunately can't take major version bumps to openssl because it's exposed as a public dependency

I believe the situation is actually worse than this: no crate should bump its openssl(-sys) dependency unless it also does a major version bump itself, regardless of whether it's publicly exposed or not. Otherwise, a minor update in a dependency changing its openssl dependency to 0.10 (for example) would conflict with another dependency still using 0.9 because of links.

[sfackler]

@jebrosen that's not a problem in this case - openssl 0.10 still uses openssl-sys 0.9.

[rustonaut]

For some setups this might work as a workaround:

# derived from the answers in this thread: https://bbs.archlinux.org/viewtopic.php?id=225658
# tested with a rust (non-public) backend server build having `native-tls`, `tls-api-openssl`
# `reqwest`, `tokio-tls` and `hyper-tls` in it's dependency tree
export PKG_CONFIG_PATH=/usr/lib/openssl-1.0/pkgconfig
export CFLAGS+=" -I/usr/include/openssl-1.0"
export LDFLAGS+=" -L/usr/lib/openssl-1.0 -lssl"

But this is potentially brittle, and I have to little experience with the C/C++ build system
to judge how brittle it is and which other thinks this might break.
(It would be nice if you could inject this environment variables just to the "openssl" crates, somehow)

EDIT:
The workaround proposed by @mqus, seems to be much more reliable

[mqus]

In my case (archlinux, bitwarden_rs) it was enough to rely on the already installed openssl-1.0 package and compile via

$> OPENSSL_LIB_DIR="/usr/lib/openssl-1.0" OPENSSL_INCLUDE_DIR="/usr/include/openssl-1.0" makepkg

(for archlinux packages) or

OPENSSL_LIB_DIR="/usr/lib/openssl-1.0" OPENSSL_INCLUDE_DIR="/usr/include/openssl-1.0" cargo build 

like suggested above.

[ishitatsuyuki]

If you have none of the workaround working, you can also try to replace with the branch of #1001. I believe it has no compatibility issues, but it's good to test it out.

[jens1o]

Hey @sfackler, how have this been fixed exactly?

[jonhoo]

@jens1o I think this is being closed as wontfix given this comment.

[sfackler]

That's correct.

[jens1o]

Alright, thanks. :)

[bnewbold]

Apologies for necro-posting on this already very long thread!

A work around for me on Ubuntu 18.04 bionic was fairly straight forward and I didn't see a working solution in the thread above:

# note that libssl1.0-dev conflicts with libssl-dev, but openssl1.0 does not conflict with openssl, and that the libssl1.0.0 package version is 1.0.2
sudo apt install openssl1.0 libssl1.0.0 libssl1.0-dev
cargo clean
OPENSSL_LIB_DIR="/usr/lib/x86_64-linux-gnu" OPENSSL_INCLUDE_DIR="/usr/include/openssl" cargo build

A gotcha for me was a host setup script that kept installing libssl-dev, replacing libssl1.0-dev.

Thank you to everybody involved for their patience with both gnarly API dependencies and the long-tail of folks with old environments!