Embedded httplib2 does not appear request the correct certificate, triggers DOWN state

[miketheman]

It appears that sites hosted on a service that uses a subjectAltName to enable many sites to use SSL on the same IP address, like Fastly or WPEngine, will send along the certificate when the Host header matches a site they provide.

When setting disable_ssl_validation flag to false, we see this error:

Server presented certificate that does not match host www.datadoghq.com: {'notAfter': 'May 19 22:01:42 2018 GMT', 'subjectAltName': (('DNS', '*.wpengine.com'), ('DNS', 'wpengine.com')), 'subject': ((('organizationalUnitName', u'GT41552380'),), (('organizationalUnitName', u'See www.rapidssl.com/resources/cps (c)14'),), (('organizationalUnitName', u'Domain Control Validated - RapidSSL(R)'),), (('commonName', u'*.wpengine.com'),))}. Connection failed after 128 ms

This is due to the remote server responding with a certificate that is not the certificate requested.

Here's a command to retrieve the remote certificate (you can compare commonName and subjectAltName).

echo | openssl s_client -connect www.datadoghq.com:443 -servername www.datadoghq.com 2>&1 |sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' | openssl x509 -text -noout

The -servername flag is the equivalent of setting the HTTP Host header flag.

I tried setting the headers hash to contain the host header I was requesting, but get the same response.

>>> from httplib2 import Http, HttpLib2Error
>>> h = Http()
>>> resp, content = h.request("https://www.datadoghq.com/", "GET", headers={"host": "www.datadoghq.com", 'user-agent': 'anything'})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/datadog-agent/embedded/lib/python2.7/site-packages/httplib2/__init__.py", line 1593, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/opt/datadog-agent/embedded/lib/python2.7/site-packages/httplib2/__init__.py", line 1335, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/opt/datadog-agent/embedded/lib/python2.7/site-packages/httplib2/__init__.py", line 1300, in _conn_request
    conn.connect()
  File "/opt/datadog-agent/embedded/lib/python2.7/site-packages/httplib2/__init__.py", line 1032, in connect
    'host %s: %s' % (hostname, cert), hostname, cert)
httplib2.CertificateHostnameMismatch: Server presented certificate that does not match host www.datadoghq.com: {'notAfter': 'May 19 22:01:42 2018 GMT', 'subjectAltName': (('DNS', '*.wpengine.com'), ('DNS', 'wpengine.com')), 'subject': ((('organizationalUnitName', u'GT41552380'),), (('organizationalUnitName', u'See www.rapidssl.com/resources/cps (c)14'),), (('organizationalUnitName', u'Domain Control Validated - RapidSSL(R)'),), (('commonName', u'*.wpengine.com'),))}

[remh]

Seems like a bug in httplib2 to me.

We will investigate for 5.2.0