Fallback on deprecated SecTrustEvaluate function on macOS 10.13 and earlier

[versionbayjc]

I am trying to use truststore on Mac OS X 10.11 (El Capitan). Both Python 3.10 and truststore would seem to work with this combination, but it is failing to load a symbol, which seems to have been introduced in 10.14 when verifying a certificate in a HTTPS request.

Are older macOS versions really tested with the current truststore?
Please let me know what additional debug info I can provide, but here is output from a Terminal with some useful steps to check if truststore is working properly:

Last login: Fri Oct 13 14:03:12 on ttys004
Mac-mini-ElCapitan-VM:~ jorik$ uname -a
Darwin Mac-mini-ElCapitan-VM.local 15.6.0 Darwin Kernel Version 15.6.0: Thu Jun 21 20:07:40 PDT 2018; root:xnu-3248.73.11~1/RELEASE_X86_64 x86_64
Mac-mini-ElCapitan-VM:~ jorik$ source ~/src/trust_test/venv/bin/activate
(venv) Mac-mini-ElCapitan-VM:~ jorik$ pip list
Package            Version
------------------ ---------
certifi            2023.7.22
charset-normalizer 3.3.0
idna               3.4
pip                23.2.1
requests           2.31.0
setuptools         68.1.2
truststore         0.8.0
urllib3            2.0.6
(venv) Mac-mini-ElCapitan-VM:~ jorik$ python
Python 3.10.13 (main, Oct 12 2023, 14:16:20) [Clang 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> R = requests.get('https://www.python.org/')
>>> 
(venv) Mac-mini-ElCapitan-VM:~ jorik$ python
Python 3.10.13 (main, Oct 12 2023, 14:16:20) [Clang 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import truststore
>>> truststore.inject_into_ssl()
>>> import requests
>>> R = requests.get('https://www.python.org/')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
  File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 790, in urlopen
    response = self._make_request(
  File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 467, in _make_request
    self._validate_conn(conn)
  File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1092, in _validate_conn
    conn.connect()
  File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/urllib3/connection.py", line 642, in connect
    sock_and_verified = _ssl_wrap_socket_and_match_hostname(
  File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/urllib3/connection.py", line 783, in _ssl_wrap_socket_and_match_hostname
    ssl_sock = ssl_wrap_socket(
  File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 469, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls, server_hostname)
  File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 513, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/truststore/_api.py", line 104, in wrap_socket
    _verify_peercerts(ssl_sock, server_hostname=server_hostname)
  File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/truststore/_api.py", line 300, in _verify_peercerts
    _verify_peercerts_impl(
  File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/truststore/_macos.py", line 437, in _verify_peercerts_impl
    sec_trust_eval_result = Security.SecTrustEvaluateWithError(
  File "/usr/local/Cellar/[email protected]/3.10.13/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ctypes/__init__.py", line 387, in __getattr__
    func = self.__getitem__(name)
  File "/usr/local/Cellar/[email protected]/3.10.13/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ctypes/__init__.py", line 392, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: dlsym(0x7fc3d9462db0, SecTrustEvaluateWithError): symbol not found
>>> 

From this page, it seems SecTrustEvaluateWithError is available since 10.14? Or maybe I'm misreading the documentation?

[sethmlarson]

@versionbayjc Thanks for opening this issue. We haven't tested with older macOS and I see what happened. To support that version of macOS we'd need to use the deprecated SecTrustEvaluate function. I'm going to edit this issue's title to reflect that.

[illume]

Hellos.

I started a PR here: #156 to use the deprecated SecTrustEvaluate function.

Would love any help/pointers/testing :)

ps. AFAIK it seems this issue triggered pip dropping 10.9-10.13 support, which triggered python itself in a patch release 3.12.6 dropping support, which then caused cibuildwheel to drop support, which caused other packages to not be built with 10.9-10.13 support.

[ThomasWaldmann]

Thanks for working on this! I left a comment in the PR.

[versionbayjc]

I can also confirm that after using pip install git+https://github.com/sethmlarson/truststore.git instead of installing truststore 0.9.2 from PyPI, the steps that caused the symbol not found earlier are now working correctly on my 10.11 El Capitan VM.
Thanks @sethmlarson, @ThomasWaldmann, @illume !!