Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

certs: add certificates update cmd #241

Closed
wants to merge 1 commit into from
Closed

certs: add certificates update cmd #241

wants to merge 1 commit into from

Conversation

DomT4
Copy link
Member

@DomT4 DomT4 commented May 13, 2016

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same change?
  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your changes? Here's an example.
  • Have you successfully run brew tests with your changes locally?

Adds a relatively simple command to update the certificates Homebrew uses for OpenSSL/LibreSSL/GnuTLS installations.

Currently we just update the certs on upgrade/reinstall of those formulae but Apple themselves sometimes push certificate changes on OS X update or doesn't update certs at all for certain OS X releases. This provides a way for people to update out-of-band, and might save Shaun & Misty some work on Linuxbrew/Tigerbrew respectively.

For the three most recent OS X releases that can still claim to be kind of to completely supported by Apple it leans on the existing post_install logic in OpenSSL/LibreSSL/GnuTLS but could just as easily be flipped around so the core logic behind that resides in this command and those formulae call this command in postinstall, if that's preferred.

For the older OS X releases and Linux it uses Homebrew's curl to retrieve Mozilla certs and wraps them into a nice PEM, which is similar to what Misty and Shaun already do.

Reviving a three year old idea. Ref: Homebrew/legacy-homebrew#21065
More discussion over in Homebrew/homebrew-core#971.

@DomT4
certs: add certificates update cmd
a6e6e51 
Adds a relatively simple command to update the certificates Homebrew uses for
OpenSSL/LibreSSL/GnuTLS installations.

Currently we just update the certs on upgrade/reinstall of those formulae but
Apple themselves sometimes pushes certificate changes on OS X update, doesn't
update certs at all for certain OS X releases. This provides a way for people to
update out-of-band, and might save Shaun & Misty some work on Linuxbrew/Tigerbrew
respectively.

For the three most recent OS X releases that can still claim to be kind of to completely
supported by Apple it leans on the existing post_install logic in OpenSSL/LibreSSL/GnuTLS
but could just as easily be flipped around so the core logic behind that resides in this
command and those formulae call this command in postinstall, if that's preferred.

For the older OS X releases and Linux it uses Homebrew's curl to retrieve Mozilla
certs and wraps them into a nice PEM, which is similar to what Misty and Shaun already do.

Reviving a three year old idea. Ref: Homebrew/legacy-homebrew#21065
More discussion over in Homebrew/homebrew-core#971.
@DomT4 DomT4 added features New features discussion Input solicited from others labels May 13, 2016
@DomT4
Copy link
Member Author

DomT4 commented May 13, 2016

Small notations:

  • This isn't the finished product in terms of documentation, Linux/ancient OS X testing or necessarily design/goals.
  • Opening the PR at this point allows for some discussion on the idea.
  • Opening the PR at this point also allows you to roast my Ruby skills, which people tend to enjoy 😉.

@DomT4 DomT4 mentioned this pull request May 13, 2016
Closed
4 tasks
geoff-nixon
geoff-nixon reviewed May 13, 2016
View reviewed changes
Library/Homebrew/cmd/certs.rb
BREWED_OPENSSL = Pathname.new(HOMEBREW_PREFIX/"opt/openssl")
BREWED_LIBRESSL = Pathname.new(HOMEBREW_PREFIX/"opt/libressl")
BREWED_GNUTLS = Pathname.new(HOMEBREW_PREFIX/"opt/gnutls")

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aptly named branch Mr. Tiller...

For the sake of thoroughness:

> cd $(brew --prefix) 
> find . -name \*.pem -o -name \*.crt -type f | # Try to exclude as much as possible.
                                 grep -v gems          | # Not our problem.
                                 grep -v node_modules  | # Not our problem.
                                 grep -v site-packages | # Not our problem.
                                 grep -v python        | # 'site-packages' doesn't seem to cut it.
                                 grep -v test          | # Not actually used. Hopefully.
                                 grep -v example       | # Not actually used. Hopefully.
                                 grep -v sample        | # Not actually used. Hopefully.
                                 sort | uniq

leaves

./Cellar/android-sdk/24.4.1_1/tools/lib/ca-bundle.pem
./Cellar/app-engine-go-32/1.9.34/share/app-engine-go-32/lib/requests/requests/cacert.pem
./Cellar/app-engine-go-64/1.9.37/share/app-engine-go-64/lib/requests/requests/cacert.pem
./Cellar/arcanist/5/libphutil/resources/ssl/default.pem
./Cellar/couchpotatoserver/3.0.1/libexec/libs/certifi/cacert.pem
./Cellar/couchpotatoserver/3.0.1/libexec/libs/requests/cacert.pem
./Cellar/crash/1.3.1/share/doc/crash/src/crash/hostkey.pem
./Cellar/drush/8.0.5/libexec/vendor/phpunit/phpunit/build/ca.pem
./Cellar/ec2-ami-tools/1.5.7/libexec/etc/ec2/amitools/cert-ec2-cn-north-1.pem
./Cellar/ec2-ami-tools/1.5.7/libexec/etc/ec2/amitools/cert-ec2-gov.pem
./Cellar/ec2-ami-tools/1.5.7/libexec/etc/ec2/amitools/cert-ec2.pem
./Cellar/ettercap/0.8.2/share/ettercap/etter.ssl.crt
./Cellar/freeradius-server/3.0.11/etc/raddb/certs/01.pem
./Cellar/freeradius-server/3.0.11/etc/raddb/certs/02.pem
./Cellar/freeradius-server/3.0.11/etc/raddb/certs/ca.pem
./Cellar/freeradius-server/3.0.11/etc/raddb/certs/client.crt
./Cellar/freeradius-server/3.0.11/etc/raddb/certs/client.pem
./Cellar/freeradius-server/3.0.11/etc/raddb/certs/server.crt
./Cellar/freeradius-server/3.0.11/etc/raddb/certs/server.pem
./Cellar/gnupg2/2.0.30/share/gnupg/com-certs.pem
./Cellar/gtmess/0.97_1/share/gtmess/root.pem
./Cellar/gwenhywfar/4.15.3/share/gwenhywfar/ca-bundle.crt
./Cellar/headphones/0.5.9/libexec/lib/requests/cacert.pem
./Cellar/heroku/3.43.2/libexec/data/cacert.pem
./Cellar/i2p/0.9.21/libexec/certificates/news/ampernand_at_gmail.com.crt
./Cellar/i2p/0.9.21/libexec/certificates/news/echelon_at_mail.i2p.crt
./Cellar/i2p/0.9.21/libexec/certificates/news/killyourtv_at_mail.i2p.crt
./Cellar/i2p/0.9.21/libexec/certificates/news/str4d_at_mail.i2p.crt
./Cellar/i2p/0.9.21/libexec/certificates/news/zzz_at_mail.i2p.crt
./Cellar/i2p/0.9.21/libexec/certificates/plugin/cacapo_at_mail.i2p.crt
./Cellar/i2p/0.9.21/libexec/certificates/plugin/str4d_at_mail.i2p.crt
./Cellar/i2p/0.9.21/libexec/certificates/plugin/zzz-plugin_at_mail.i2p.crt
./Cellar/i2p/0.9.21/libexec/certificates/reseed/backup_at_mail.i2p.crt
./Cellar/i2p/0.9.21/libexec/certificates/reseed/bugme_at_mail.i2p.crt
./Cellar/i2p/0.9.21/libexec/certificates/reseed/cheezybudz_at_mail.i2p.crt
./Cellar/i2p/0.9.21/libexec/certificates/reseed/echelon_at_mail.i2p.crt
./Cellar/i2p/0.9.21/libexec/certificates/reseed/matt_at_drollette.com.crt
./Cellar/i2p/0.9.21/libexec/certificates/reseed/meeh_at_mail.i2p.crt
./Cellar/i2p/0.9.21/libexec/certificates/reseed/parg_at_mail.i2p.crt
./Cellar/i2p/0.9.21/libexec/certificates/router/echelon_at_mail.i2p.crt
./Cellar/i2p/0.9.21/libexec/certificates/router/killyourtv_at_mail.i2p.crt
./Cellar/i2p/0.9.21/libexec/certificates/router/str4d_at_mail.i2p.crt
./Cellar/i2p/0.9.21/libexec/certificates/router/zzz_at_mail.i2p.crt
./Cellar/i2p/0.9.21/libexec/certificates/ssl/193.150.121.66.crt
./Cellar/i2p/0.9.21/libexec/certificates/ssl/i2p.mooo.com.crt
./Cellar/i2p/0.9.21/libexec/certificates/ssl/i2pseed.zarrenspry.info.crt
./Cellar/i2p/0.9.21/libexec/certificates/ssl/ieb9oopo.mooo.com.crt
./Cellar/i2p/0.9.21/libexec/certificates/ssl/link.mx24.eu.crt
./Cellar/i2p/0.9.21/libexec/certificates/ssl/netdb.i2p2.no.crt
./Cellar/i2p/0.9.21/libexec/certificates/ssl/netdb.rows.io.crt
./Cellar/i2p/0.9.21/libexec/certificates/ssl/reseed.i2p-projekt.de.crt
./Cellar/i2p/0.9.21/libexec/certificates/ssl/reseed.i2p.vzaws.com.crt
./Cellar/i2p/0.9.21/libexec/certificates/ssl/uk.reseed.i2p2.no.crt
./Cellar/i2p/0.9.21/libexec/certificates/ssl/us.reseed.i2p2.no.crt
./Cellar/jruby/9.1.0.0/libexec/lib/ruby/truffle/pr-zlib/certs/djberg96_pub.pem
./Cellar/macvim/7.4-103/MacVim.app/Contents/Resources/dsa_pub.pem
./Cellar/mysql-cluster/7.4.9/share/mysql/mcc/cfg.pem
./Cellar/nmap/7.12/share/ncat/ca-bundle.crt
./Cellar/openlitespeed/1.3.10/admin/conf/webadmin.crt
./Cellar/passenger/5.0.26/libexec/resources/union_station_gateway.crt
./Cellar/phan/0.4/vendor/phpunit/phpunit/build/ca.pem
./Cellar/phpmyadmin3/3.5.8.2/share/phpmyadmin3/libraries/auth/swekey/musbe-ca.crt
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/AOL_Member_CA.pem
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/AddTrust_External_Root.pem
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/America_Online_Root_Certification_Authority_1.pem
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/Baltimore_CyberTrust_Root.pem
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/CAcert_Class3.pem
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/CAcert_Root.pem
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/Certum_Root_CA.pem
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/Certum_Trusted_Network_CA.pem
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/Deutsche_Telekom_Root_CA_2.pem
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/DigiCertHighAssuranceCA-3.pem
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/DigiCertHighAssuranceEVRootCA.pem
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/Entrust.net_2048.pem
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/Entrust.net_Secure_Server_CA.pem
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/Equifax_Secure_CA.pem
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/Equifax_Secure_Global_eBusiness_CA-1.pem
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/GTE_CyberTrust_Global_Root.pem
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/Go_Daddy_Class_2_CA.pem
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/Microsoft_Internet_Authority_2010.pem
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/Microsoft_Secure_Server_Authority_2010.pem
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/StartCom_Certification_Authority.pem
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/Thawte_Premium_Server_CA.pem
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/Thawte_Primary_Root_CA.pem
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/ValiCert_Class_2_VA.pem
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/VeriSign_Class3_Extended_Validation_CA.pem
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/VeriSign_Class_3_Primary_CA-G2.pem
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/VeriSign_Class_3_Primary_CA-G5-2.pem
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/VeriSign_Class_3_Primary_CA-G5.pem
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/VeriSign_International_Server_Class_3_CA.pem
./Cellar/pidgin/2.10.12_1/share/purple/ca-certs/Verisign_Class3_Primary_CA.pem
./Cellar/poretools/0.5.1_3/libexec/vendor/R/library/openssl/cacert.pem
./Cellar/qca/2.1.0/certs/rootcerts.pem
./Cellar/racket/6.5/share/racket/collects/openssl/dh4096.pem
./Cellar/sickrage/4.0.76.2/libexec/autoProcessTV/lib/requests/cacert.pem
./Cellar/sickrage/4.0.76.2/libexec/lib/certifi/cacert.pem
./Cellar/sickrage/4.0.76.2/libexec/lib/certifi/mkcert.pem
./Cellar/sickrage/4.0.76.2/libexec/lib/certifi/old_root.pem
./Cellar/sickrage/4.0.76.2/libexec/lib/requests/cacert.pem
./Cellar/sslmate/1.5.1/share/sslmate/dhparams/dh2048-group14.pem
./Cellar/sslmate/1.5.1/share/sslmate/dhparams/dh3072-group15.pem
./Cellar/sslmate/1.5.1/share/sslmate/dhparams/dh4096-group16.pem
./Cellar/sslmate/1.5.1/share/sslmate/dhparams/dh6144-group17.pem
./Cellar/sslmate/1.5.1/share/sslmate/dhparams/dh8192-group18.pem
./Cellar/sslyze/0.12.0/libexec/plugins/data/trust_stores/apple.pem
./Cellar/sslyze/0.12.0/libexec/plugins/data/trust_stores/google.pem
./Cellar/sslyze/0.12.0/libexec/plugins/data/trust_stores/java.pem
./Cellar/sslyze/0.12.0/libexec/plugins/data/trust_stores/microsoft.pem
./Cellar/sslyze/0.12.0/libexec/plugins/data/trust_stores/mozilla.pem
./Cellar/ucspi-tools/1.2_10/libexec/vendor/libressl/etc/cert.pem
./Cellar/wp-cli/0.23.0/vendor/rmccue/requests/library/Requests/Transport/cacert.pem
./Cellar/yaws/2.0/etc/yaws/yaws-cert.pem
./Cellar/yaws/2.0/etc/yaws/yaws-key.pem
./Cellar/znapzend/0.15.5/lib/Mojo/IOLoop/resources/server.crt
./etc/libressl/cert.pem
./etc/openssl/cert.pem
./etc/raddb/certs/01.pem
./etc/raddb/certs/02.pem
./etc/raddb/certs/ca.pem
./etc/raddb/certs/client.pem
./etc/raddb/certs/server.pem
./etc/shibboleth/sp-cert.pem
./etc/shibboleth/sp-key.pem
./etc/stunnel/stunnel.pem
./etc/yaws/yaws-cert.pem
./etc/yaws/yaws-key.pem
./share/gnupg/com-certs.pem
./var/mysql/ca-key.pem
./var/mysql/ca.pem
./var/mysql/client-cert.pem
./var/mysql/client-key.pem
./var/mysql/private_key.pem
./var/mysql/public_key.pem
./var/mysql/server-cert.pem
./var/mysql/server-key.pem

Not all of those are root CA lists, obviously.
And I'd argue a case could be made that any of these that:

  • doesn't provide libraries, and
  • don't have any dependent formula

could be excluded.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I don't have much of a desire to create a command that updates every cert, only really the OpenSSL/LibreSSL/GnuTLS ones because we create them.

I think it's reasonable to presume that if we create them we'll provide a easy method of keeping them refreshed, but beyond that we don't police packages.

Copy link
Contributor

@geoff-nixon geoff-nixon May 13, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's fair. But since you do "police" formula versions, it might be nice to have a list for formula that install certs, and make sure their update PRs don't languish. Or something. Just a suggestion.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

People file PRs, we'll accept them as fast as we can, more or less. We don't intentionally ice updates unless there's a hard blocking reason that we can't workaround sanely.

@geoff-nixon
Copy link
Contributor

geoff-nixon commented May 13, 2016 •
edited
Loading

Other thoughts (which maybe you're already working on, of course):

  • Keep a SHA256 of the latest bundle somewhere
  • Use it to check the bundle(s) during brew audit
  • Possibly consider one central file, and hardlink or symlink to it, rather that copy each time?

MikeMcQuaid
MikeMcQuaid reviewed May 14, 2016
View reviewed changes
Library/Homebrew/cmd/certs.rb
def certs
# Needs further investigation, just a rough guess currently that Apple
# isn't regularly updating certificates for Mountain Lion or less now.
if OS.mac? && MacOS.version <= :mountain_lion || OS.linux? || ARGV.include?("--force-curl")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just assume OS.mac? for now and/or pull that part to extend/os

@MikeMcQuaid
Copy link
Member

👍 on the idea. I wonder if there's a way to check if this needs to be run and do it automatically, though.

xu-cheng
xu-cheng reviewed May 15, 2016
View reviewed changes
Library/Homebrew/cmd/certs.rb
BREWED_CURL = Pathname.new(HOMEBREW_PREFIX/"opt/curl")
BREWED_OPENSSL = Pathname.new(HOMEBREW_PREFIX/"opt/openssl")
BREWED_LIBRESSL = Pathname.new(HOMEBREW_PREFIX/"opt/libressl")
BREWED_GNUTLS = Pathname.new(HOMEBREW_PREFIX/"opt/gnutls")
Copy link
Member

@xu-cheng xu-cheng May 15, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would avoid all these paths and #exist? method in the blow. Instead, I think using formula is better. So it would be like safe_system "brew", "postinstall", "openssl" if Formula["openssl"].installed? or Formula["openssl"].opt_libexec

@xu-cheng
Copy link
Member

Can we have a more specific name like brew update-certs or something?

@xu-cheng
Copy link
Member

Also, here is a crazy thought. Would it make more sense to make this a cmd in core tap instead of brew. Because this seems to be very specific to certain formula.

@DomT4 DomT4 mentioned this pull request Jun 27, 2016
Closed
@DomT4
Copy link
Member Author

DomT4 commented Sep 1, 2016

Someone else is welcome to pick this up as desired, but I have no immediate plans to complete it.

@DomT4 DomT4 closed this Sep 1, 2016
@DomT4 DomT4 deleted the hell_is_ssl branch September 1, 2016 16:25
@Homebrew Homebrew locked and limited conversation to collaborators May 3, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers
No reviews
Assignees
No one assigned
Labels
discussion Input solicited from others features New features
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@DomT4 @geoff-nixon @MikeMcQuaid @xu-cheng