Update OpenSSL to 1.1.1c #3822

yan12125 · 2019-03-08T19:16:45Z

Description

This is a WIP. The migration plan can be found at https://trac.macports.org/ticket/52101#comment:30.

Type(s)

bugfix
enhancement
security fix

Tested on

Not tested

Verification

Have you

followed our Commit Message Guidelines?
squashed and minimized your commits?
checked that there aren't other open pull requests for the same change?
referenced existing tickets on Trac with full URL?
checked your Portfile with port lint?
tried existing tests with sudo port test?
tried a full install with sudo port -vst install?
tested basic functionality of all binary files?

TODOs:

Check whether openssl patches are really unneeded for 1.1.1 or not

install-headers-HFS+.patch is no longer needed after OpenSSL 0.9.8k [1]; also the problematic files crypto/{bf,des}/INSTALL are removed in OpenSSL 1.1 branch [2][3].
parallel-building.patch: use_parallel_build no is back in [4], so this patch is not useful anyway. In the new build system introduced in OpenSSL 1.1, dependencies info is recorded in various build.info files. Hopefully parallel building will work fine. Enabling parallel building can be postponed to the future (i.e., after this pull request is merged).
remove-duplicate-bn_print-doc.patch: one of conflicting files doc/crypto/bn_internal.pod is moved out of doc/ since OpenSSL 1.1 [5], and only files under doc/ are processed [6].
[1] openssl/openssl@7d368fc
[2] openssl/openssl@b8a9af6#diff-1bd050ccaca35f3bb9c7971131671ed1
[3] openssl/openssl@24956ca#diff-3f5df9c20c5f557ccd10ae5a5de91519
[4] https://trac.macports.org/ticket/46719
[5] openssl/openssl@5393482#diff-7cd9a43108bb52726514b6da2243e1d8
[6] https://github.com/openssl/openssl/blob/OpenSSL_1_1_1c/util/process_docs.pl#L49

macportsbot · 2019-03-08T19:17:36Z

Notifying maintainers:
@larryv for port openssl.
@jcvernaleo for port alpine.
@cssdev for port apg.
@ci42 for port fossil, git, nodejs4, nodejs5, nodejs6, nodejs7, nodejs8, nodejs9, erlang, ejabberd.
@kencu for port openrct2, stellarium-qt4.
@jerryyhom for port alpine.
@danielluke for port bind9, nmap, sysmon, clamav, ntp.
@aphor for port haproxy.
@Gminfly for port libssh2, medusa.
@markemer for port Io.
@kimuraw for port ruby, ruby186, ruby19, ruby20, ruby21, ruby22, ruby23, ruby24, ruby25, ruby26.
@cjones051073 for port davix, root5, root6, xrootd.
@Jakker for port softhsm.
@scantor for port xml-security-c.
@NicosPavlov for port kdelibs4, smokekde.
@ghosthound for port wireshark2, wireshark3.
@emaros for port lscsoft-deps.
@michaelld for port qt4-mac.
@l2dy for port getdns, irssi, pdns-recursor, libu2f-server.
@Veence for port qgis.
@Schamschula for port lftp, tcpdump, certbot-nginx, certbot, stunnel, apache2, goaccess, nghttp2, nginx.
@roederja for port mico.
@davidnich for port qore-asn1-module, qore-devel, qore-json-module, qore-xml-module, qore.
@jdberry for port dovecot.
@fhgwright for port libaes_siv, ntpsec.
@mklein-de for port cadaver.
@skymoo for port lscsoft-deps.
@bwalle for port crackpkcs12.
@ryandesign for port dar, freetds, mysql4, eet, libhsplasma, PlasmaClient, bahamut, php4, docsis, kerberos5, net-snmp, openvas-client, apache2, apache22, lighttpd.
@pmetzger for port libetpan.
@alfredh for port libre.
@mojca for port iAIDA, root6.
@lpsinger for port lscsoft-deps.
@tobypeterson for port epic4, epic5, mailx.
@drkp for port mit-scheme, ipmitool.
@slewsys for port mosquitto.
@scarface-one for port mtxclient.
@nerdling for port virtuoso-6, virtuoso-7, unbound, xml-security-c.
@xeron for port ngircd.
@jyrkiwahlstedt for port postgresql10, postgresql11, postgresql93, postgresql94, postgresql95, postgresql96.
@g5pw for port pev, radare2, esniper, lnav, socat.
@eborisch for port dcmtk.
@mohd-akram for port crystal, cpprestsdk.
@amake for port yubikey-manager.
@stromnov for port phantomjs.
@dbevans for port opusfile, telepathy-idle, podofo, deluge.
@easye for port bitcoin.
@kurthindenburg for port ssmtp, dokuwiki.
@Ionic for port libssh.
@blair for port thrift-devel, thrift, zeroc-ice33, zeroc-ice34, zeroc-ice35, serf0, serf1.
@mkhon for port tnftp.
@bgilbert for port tarsnap.
@raimue for port gmpc, hexchat.
@nareshov for port git-crypt.
@mattbishop for port tomcat-native.
@lbschenkel for port openpace, opensc.
@lpefferkorn for port borgbackup.
@schiebel for port mico.
@neverpanic for port openssl, nginx.
@jmroot for port python27, python34, python35, python36, python37, squid2, squid3, squid4.
@ecronin for port blackbag, boxbackup.

science/lscsoft-deps/Portfile

xeron

+1 for ngircd if it builds.

danielluke · 2019-03-08T19:56:04Z

revbumps for my ports are all OK

Schamschula · 2019-03-08T20:32:06Z

As I don't yet have openssl 1.1.1 installed, I haven't tested all my ports. However, all ports other than certbot, have upstream documentation that current released are compatible with openssl 1.1.x.

certbot (and certbot-apache & certbot-nginx) don't directly use openssl. However, it is brought in via Python and py-openssl (which lists openssl 1.1.0 as supported).

Schamschula · 2019-03-08T20:33:52Z

openssh will need to be updated to version 7.9p1, as prior versions don't support openssl 1.1.x.

scantor · 2019-03-08T20:36:38Z

xml-security-c is compatible with 1.1.1

jcvernaleo · 2019-03-08T21:08:30Z

Pretty sure alpine uses that openssl on other platforms so as long as it builds, +1 from me.

security/openpace/Portfile

fhgwright · 2019-03-09T00:11:10Z

+1 for ntpsec and libaes_siv.

I verified that the current MacPorts source for those ports tests and builds with the new OpenSSL, albeit directly rather than via MacPorts. To do that, I temporarily installed the new OpenSSL port and rebuilt the curl and python27 ports to unbreak them (ignoring a zillion other temporarily broken ports).

Schamschula · 2019-03-09T01:02:12Z

Looks like all my ports build correctly.

However, revision bumps on certbot and certbot-nginx are unneeded, while a revision bump for py-cryptography will be needed for certbot et al. to work.

Schamschula · 2019-03-09T03:19:42Z

So far I have found:

broken
-ettercap @0.8.2+gtk+plugins
-gstreamer1-gst-plugins-bad @1.14.4+x11
-liboauth @1.0.3
-qt4-mac @4.8.7
-rtmpdump @2.4-20151223
-ruby @1.8.7-p374

missing from PR but building:
+gdal @2.3.2
+cargo @0.31.1
+dvisvgm @2.6.2
+libewf @20171104
+libevent @2.1.8
+py27-cryptography @2.5
+sane-backends @1.0.27+avahi
+w3m @0.5.3
+xar @1.6.1

nomaintainer in PR and building:
+afflib @3.7.18
+bro @2.6.1
+cyrus-sasl2 @2.1.27+kerberos
+httping @2.5
+mod_qos @11.62
+nagios-plugins @2.2.1
+neon @0.30.2
+openldap @2.4.47
+p5-net-ssleay @1.850.0
+pkcs11-helper @1.25.1
+proftpd @1.3.6+sqlite3
+snort @2.9.12+mysql57

yan12125 · 2019-09-02T02:47:25Z

Archlinux recently dropped QT4 without applying patches for OpenSSL 1.1.x ArchLinux stopped porting QT4 to OpenSSL 1.1.x

Wrong statements. Arch Linux patched Qt4 for OpenSSL 1.1 compatiblity in 2017 [1], and the drop occurred in 2019 [2]. Qt4 + OpenSSL 1.1 had been worked fine on Arch Linux for almost 2 years.

@yan12125 Could I ask you to rebase/merge your branch again?

Will do after the fix for qt4-mac (#5179) is merged. [1] https://git.archlinux.org/svntogit/packages.git/commit/?id=d22be862cbcc117a02bd71369ae362d9e1a420e4 [2] https://git.archlinux.org/svntogit/packages.git/commit/?id=979cd87bda942d70c604a01dc90153f7369a1b38

RJVB · 2019-09-02T07:34:50Z

Wrong statements. Arch Linux patched Qt4 for OpenSSL 1.1 compatiblity in 2017 [1], and the drop occurred in 2019 [2]. Qt4 + OpenSSL 1.1 had been worked fine on Arch Linux for almost 2 years.

And they probably shouldn't have dropped Qt4 altogether (if that's what they did). There will always remain software not ported to Qt5 (a good deal of which will continue to work just fine without notable security implications), and users who need that software for reasons they alone can judge/justify. Removing the Qt4 package means those users will have to figure out how to do the build themselves, or run whatever binary packages they can still find, and there's a good chance those will not have all the latest patches.

pmetzger · 2019-09-02T11:58:32Z

@mojca

With macOS it's quite a bit different. If you have an old PPC box lying around, or a 32-bit piece of Apple hardware, or ... well, simply some decently working 64-bit hardware that Apple decided to abandon, you can either throw it to trash bin, or install MacPorts on it, and still run the latest version of almost any opensource software you may need, and keep the box fully functional.

If we keep old PPC stuff functional at the expense of the bulk of the Mac user community (most of whom are on the latest operating system), we probably make a lot of people unhappy to make a small number of people happy. I think it's okay to keep the old stuff running to the extent that it doesn't interfere with users of current operating systems.

kencu · 2019-09-02T12:48:22Z

let's please not have more older systems debate here in this PR, os it is not the right forum. Feel free to start a thread on the mailing list if you care to.

eirnym · 2019-09-02T19:24:26Z

@yan12125 thanks, for the information, I don't use it, so I didn't know it.

michaelld · 2019-09-06T15:05:05Z

qt4-mac done in facea54 .

yan12125 · 2019-09-06T17:00:13Z

This pull request is now ready for merge. Any last-minute comments?

eirnym · 2019-09-06T17:14:36Z

I think it's better to merge this way and then proceed with updating packages when it's required.

yan12125 · 2019-09-06T17:37:48Z

Oops I forgot about squid2 and squid3. Anyway, as those versions are no longer supported by upstream and there is only 1 user per statistics on ports.macports.org [1][2], I don't consider it a blocker.

[1] https://ports.macports.org/port/squid2/stats?days=30&days_ago=0
[2] https://ports.macports.org/port/squid3/stats?days=30&days_ago=0

eirnym · 2019-09-06T17:44:56Z

as we saw with some ports above, even 180 days with no downloads was blocked to drop the support.

jmroot · 2019-09-06T17:56:25Z

It's fine, go ahead and merge and I'll fix the old squids shortly.

jmroot · 2019-09-06T18:17:13Z

as we saw with some ports above, even 180 days with no downloads was blocked to drop the support.

Technically there is currently no way of determining that there were no downloads. We would have to parse the CDN logs, which isn't a bad idea to supplement the opt-in stats, it's just nobody has done it yet. (And even if there were no downloads, it doesn't necessarily follow that nobody is using the port.)

Anyway, the point is, don't over-extrapolate from the opt-in statistics, which only represent a sample of around 100 installations (compared to 300k downloads for some of our releases, not counting those who just selfupdated or downloaded from somewhere other than GitHub). There's a considerable "long tail" of less popular ports.

eirnym · 2019-09-06T18:25:33Z

It's fine, go ahead and merge and I'll fix the old squids shortly.

I'm fine with that and with the overall result.

neverpanic

Three minor issues with the revision bumps, other than that this lg2m.

multimedia/mythtv.27/Portfile

multimedia/mythtv.28/Portfile

neverpanic · 2019-09-06T18:57:20Z

net/iperf3/Portfile

@@ -40,6 +40,7 @@ if {${subport} eq ${name}} {
 subport ${name}-devel {


Chances are the non-devel subport will also need a revbump for this port.

yan12125 · 2019-09-07T08:40:20Z

@neverpanic Thank you very much! I added fixes for mythtv* in "openssl: Fix special cases manually" (3739a77657cb6de74a8ab47d1e62f4b3f5ecb7aa after the latest rebase). iperf3 and other ports with similar issues are fixed in "openssl: add missing revbumps for ports with some subports bumped but not all" (6dee32e597aca3762354c8312663157a4677cae9).

eirnym · 2019-09-07T11:59:05Z

and one more conflicting port

neverpanic

Thanks! Now if people stop pushing new things, we can get this merged! ;-)

Closes: https://trac.macports.org/ticket/52101 Ref: https://trac.macports.org/ticket/58218 (old Qt5 versions) Ref: https://trac.macports.org/ticket/58607 (old MySQL versions) Ref: https://trac.macports.org/ticket/58630 (old PostgreSQL versions)

Generated with the following command: rg -l -g Portfile ':openssl\b' ../macports-ports | grep -v '/lang/php/' | grep -v '/aqua/qt5' | xargs ./revbump-ports.sh revbump-ports.sh is from https://github.com/yan12125/revbump-ports/tree/4a47ad53d5bc1553b86f48dfbf946bc9a466306c Skipping lang/php and aqua/qt5 as only php*-{ftp,openssl} and qt*-qtbase subports need revbumping, respectively.

1. False-positive version patterns $ git diff HEAD^ | nvim -R - and search in nvim using: ^\s*version\s\+\S\+\s 2. False-positive (commented) openssl dependency declaration Search using: $ rg ':openssl\b' | rg '#' | sed 's#:.*##' Note that squid4 still depends on openssl. 3. Non-numeric revision strings Search using: rg 'revision\s+\$\{.+\}' $(rg -l -g Portfile ':openssl\b' | grep -v 'lang/php/' | grep -v 'aqua/qt5') Only mythtv* is affected. mysql-like ports are already handled by https://github.com/yan12125/revbump-ports/blob/master/increase-revision-mysql.sed

… not all Thanks neverpanic for identifying such issues! Approach: 1. git diff HEAD~4..HEAD~3 > diffs 2. Search pattern +\s\+revision from `diffs` with neovim and check relevant ports manually.

yan12125 · 2019-09-07T14:35:02Z

I think there are already enough reviews and I cannot imagine any further issue other than build failures for a few uncommon ports, so let's get it done! Thanks everyone for testing, reviewing and sharing awesome ideas!

macportsbot assigned larryv, ci42, kencu, danielluke, markemer, kimuraw, cjones051073, scantor, NicosPavlov and ghosthound Mar 8, 2019

macportsbot added by: member Created by a member with commit rights maintainer: requires approval labels Mar 8, 2019

skymoo reviewed Mar 8, 2019

View reviewed changes

science/lscsoft-deps/Portfile Outdated Show resolved Hide resolved

xeron approved these changes Mar 8, 2019

View reviewed changes

l2dy requested review from neverpanic, jmroot, raimue, lbschenkel, bgilbert and ecronin March 8, 2019 20:36

lbschenkel reviewed Mar 8, 2019

View reviewed changes

security/openpace/Portfile Outdated Show resolved Hide resolved

yan12125 changed the title ~~[WIP] Update OpenSSL to 1.1.1c~~ Update OpenSSL to 1.1.1c Sep 6, 2019

yan12125 removed the wip Work in progress label Sep 6, 2019

neverpanic reviewed Sep 6, 2019

View reviewed changes

neverpanic approved these changes Sep 7, 2019

View reviewed changes

Chih-Hsuan Yen and others added 6 commits September 7, 2019 22:23

openssl: update to 1.1.1c

ef2d58e

Closes: https://trac.macports.org/ticket/52101 Ref: https://trac.macports.org/ticket/58218 (old Qt5 versions) Ref: https://trac.macports.org/ticket/58607 (old MySQL versions) Ref: https://trac.macports.org/ticket/58630 (old PostgreSQL versions)

php: revbump for openssl 1.1

46caa75

qt5,qt511: revbump for openssl 1.1

6fc7bb9

openssl: add missing revbumps for ports with some subports bumped but…

e6612a4

… not all Thanks neverpanic for identifying such issues! Approach: 1. git diff HEAD~4..HEAD~3 > diffs 2. Search pattern +\s\+revision from `diffs` with neovim and check relevant ports manually.

yan12125 merged commit 022f7da into macports:master Sep 7, 2019

yan12125 deleted the openssl11 branch September 7, 2019 17:16

eborisch mentioned this pull request Sep 17, 2019

ipmitool: Update to post-1.8.18 #5317

Merged

11 tasks

yan12125 mentioned this pull request Sep 21, 2019

Add missing openssl dependencies #5349

Merged

11 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update OpenSSL to 1.1.1c #3822

Update OpenSSL to 1.1.1c #3822

yan12125 commented Mar 8, 2019 •

edited

Loading

macportsbot commented Mar 8, 2019

xeron left a comment

danielluke commented Mar 8, 2019

Schamschula commented Mar 8, 2019

Schamschula commented Mar 8, 2019

scantor commented Mar 8, 2019

jcvernaleo commented Mar 8, 2019

fhgwright commented Mar 9, 2019

Schamschula commented Mar 9, 2019

Schamschula commented Mar 9, 2019

yan12125 commented Sep 2, 2019 via email

RJVB commented Sep 2, 2019 via email

pmetzger commented Sep 2, 2019

kencu commented Sep 2, 2019

eirnym commented Sep 2, 2019

michaelld commented Sep 6, 2019

yan12125 commented Sep 6, 2019

eirnym commented Sep 6, 2019

yan12125 commented Sep 6, 2019

eirnym commented Sep 6, 2019

jmroot commented Sep 6, 2019

jmroot commented Sep 6, 2019

eirnym commented Sep 6, 2019

neverpanic left a comment

neverpanic Sep 6, 2019

yan12125 commented Sep 7, 2019

eirnym commented Sep 7, 2019

neverpanic left a comment

yan12125 commented Sep 7, 2019

		@@ -40,6 +40,7 @@ if {${subport} eq ${name}} {
		subport ${name}-devel {

Update OpenSSL to 1.1.1c #3822

Update OpenSSL to 1.1.1c #3822

Conversation

yan12125 commented Mar 8, 2019 • edited Loading

Description

Type(s)

Tested on

Verification

macportsbot commented Mar 8, 2019

xeron left a comment

Choose a reason for hiding this comment

danielluke commented Mar 8, 2019

Schamschula commented Mar 8, 2019

Schamschula commented Mar 8, 2019

scantor commented Mar 8, 2019

jcvernaleo commented Mar 8, 2019

fhgwright commented Mar 9, 2019

Schamschula commented Mar 9, 2019

Schamschula commented Mar 9, 2019

yan12125 commented Sep 2, 2019 via email

RJVB commented Sep 2, 2019 via email

pmetzger commented Sep 2, 2019

kencu commented Sep 2, 2019

eirnym commented Sep 2, 2019

michaelld commented Sep 6, 2019

yan12125 commented Sep 6, 2019

eirnym commented Sep 6, 2019

yan12125 commented Sep 6, 2019

eirnym commented Sep 6, 2019

jmroot commented Sep 6, 2019

jmroot commented Sep 6, 2019

eirnym commented Sep 6, 2019

neverpanic left a comment

Choose a reason for hiding this comment

neverpanic Sep 6, 2019

Choose a reason for hiding this comment

yan12125 commented Sep 7, 2019

eirnym commented Sep 7, 2019

neverpanic left a comment

Choose a reason for hiding this comment

yan12125 commented Sep 7, 2019

yan12125 commented Mar 8, 2019 •

edited

Loading