setting a federation_certificate_verification_whitelist breaks .well-known lookups

[richvdh]

2019-08-30 12:12:33,479 - synapse.http.federation.well_known_resolver - 153 - INFO - GET-17- Error fetching https://matrix.org/.well-known/matrix/server: cannot use a string pattern on a bytes-like object

Adding some more logging, we see the exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/synapse/http/federation/well_known_resolver.py", line 114, in _do_get_well_known
    self._well_known_agent.request(b"GET", uri)
  File "/usr/local/lib/python3.7/site-packages/twisted/web/client.py", line 2126, in request
    deferred = self._agent.request(method, uri, headers, bodyProducer)
  File "/usr/local/lib/python3.7/site-packages/twisted/web/client.py", line 1732, in request
    endpoint = self._getEndpoint(parsedURI)
  File "/usr/local/lib/python3.7/site-packages/twisted/web/client.py", line 1715, in _getEndpoint
    return self._endpointFactory.endpointForURI(uri)
  File "/usr/local/lib/python3.7/site-packages/twisted/web/client.py", line 1590, in endpointForURI
    uri.port)
  File "/usr/local/lib/python3.7/site-packages/synapse/crypto/context_factory.py", line 155, in creatorForNetloc
    return self.get_options(hostname)
  File "/usr/local/lib/python3.7/site-packages/synapse/crypto/context_factory.py", line 124, in get_options
    if regex.match(host):
TypeError: cannot use a string pattern on a bytes-like object

Introduced by #5794. The problem is ClientTLSOptionsFactory.creatorForNetloc is called with a bytes, but get_options expects a string.

[richvdh]

given that the other place get_options is called from looks like this:

            tls_options = self._tls_client_options_factory.get_options(
                res.tls_server_name.decode("ascii")
            )

... we probably need to move that decode down into get_options so that it takes a bytes instead.

[richvdh]

(a regression test would be nice, of course)

[richvdh]

I think this got fixed by #5997.