[24.2] installs fail on macOS 10.12 and older due to bundled `truststore` issue

[ned-deily]

Description

When attempting to use pip 24.2 to install packages on macOS 10.12 or older, pip's vendored version of truststore fails with:

[...]
 File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/truststore/_macos.py", line 435, in _verify_peercerts_impl
    sec_trust_eval_result = Security.SecTrustEvaluateWithError(
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/ctypes/__init__.py", line 392, in __getattr__
    func = self.__getitem__(name)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/ctypes/__init__.py", line 397, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: dlsym(0x7f8bab7b33d0, SecTrustEvaluateWithError): symbol not found

This is due to a known open issue in truststore.

Pip users can work around the issue by adding --use-deprecated=legacy-certs to the install command, as in:

python3 -m pip install --use-deprecated=legacy-certs <package>

The problem was noted during the release process for Python 3.12.5 for which the ensurepip bundled pip version is now 24.2; unfortunately, the python.org Python for macOS installers produced for Python 3.12.x still support back to macOS 10.9. To workaround the issue for 3.12.5, the Python release team has agreed to provide a workaround in the Install Certificates.command included with the installer which is used to install the latest certifi bundle of certificates. For 3.12.5, when running Install Certificates on macOS 10.9.x through 10.12.x, the command will attempt to revert to pip 24.1.2. (Moving forward, we expect to update the minimum supported version of future 3.12.x installers to at least macOS 10.13, as we are already doing for pre-releases of Python 3.13.)

Expected behavior

No response

pip version

24.2

Python version

all

OS

macOS

How to Reproduce

$ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.9.5
BuildVersion:	13F1911
$ python3 -m pip install --upgrade pip
Requirement already satisfied: pip in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (24.1.2)
Collecting pip
  Using cached pip-24.2-py3-none-any.whl.metadata (3.6 kB)
Using cached pip-24.2-py3-none-any.whl (1.8 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 24.1.2
    Uninstalling pip-24.1.2:
      Successfully uninstalled pip-24.1.2
Successfully installed pip-24.2
$ python3 -m pip install certifi
ERROR: Exception:
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_internal/cli/base_command.py", line 105, in _run_wrapper
    status = _inner_run()
             ^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_internal/cli/base_command.py", line 96, in _inner_run
    return self.run(options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_internal/cli/req_command.py", line 67, in wrapper
    return func(self, options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_internal/commands/install.py", line 379, in run
    requirement_set = resolver.resolve(
                      ^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 95, in resolve
    result = self._result = resolver.resolve(
                            ^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 397, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _add_to_criteria
    if not criterion.candidates:
           ^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/resolvelib/structs.py", line 156, in __bool__
    return bool(self._sequence)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 174, in __bool__
    return any(self)
           ^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 162, in <genexpr>
    return (c for c in iterator if id(c) not in self._incompatible_ids)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 49, in _iter_built
    for version, func in infos:
                         ^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 301, in iter_index_candidate_infos
    result = self._finder.find_best_candidate(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_internal/index/package_finder.py", line 883, in find_best_candidate
    candidates = self.find_all_candidates(project_name)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_internal/index/package_finder.py", line 824, in find_all_candidates
    page_candidates = list(page_candidates_it)
                      ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_internal/index/sources.py", line 194, in page_candidates
    yield from self._candidates_from_page(self._link)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_internal/index/package_finder.py", line 784, in process_project_url
    index_response = self._link_collector.fetch_response(project_url)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_internal/index/collector.py", line 448, in fetch_response
    return _get_index_content(location, session=self.session)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_internal/index/collector.py", line 352, in _get_index_content
    resp = _get_simple_response(url, session=session)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_internal/index/collector.py", line 131, in _get_simple_response
    resp = session.get(
           ^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/requests/sessions.py", line 602, in get
    return self.request("GET", url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_internal/network/session.py", line 522, in request
    return super().request(method, url, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/cachecontrol/adapter.py", line 76, in send
    resp = super().send(request, stream, timeout, verify, cert, proxies)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/requests/adapters.py", line 667, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/urllib3/connectionpool.py", line 715, in urlopen
    httplib_response = self._make_request(
                       ^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/urllib3/connectionpool.py", line 404, in _make_request
    self._validate_conn(conn)
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/urllib3/connectionpool.py", line 1058, in _validate_conn
    conn.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/urllib3/connection.py", line 419, in connect
    self.sock = ssl_wrap_socket(
                ^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
               ^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/truststore/_api.py", line 105, in wrap_socket
    _verify_peercerts(ssl_sock, server_hostname=server_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/truststore/_api.py", line 311, in _verify_peercerts
    _verify_peercerts_impl(
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/truststore/_macos.py", line 435, in _verify_peercerts_impl
    sec_trust_eval_result = Security.SecTrustEvaluateWithError(
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/ctypes/__init__.py", line 392, in __getattr__
    func = self.__getitem__(name)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/ctypes/__init__.py", line 397, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: dlsym(0x7fc5a06e1520, SecTrustEvaluateWithError): symbol not found
$ python3 -m pip install --use-deprecated=legacy-certs pip==24.1.2
Collecting pip==24.1.2
  Using cached pip-24.1.2-py3-none-any.whl.metadata (3.6 kB)
Using cached pip-24.1.2-py3-none-any.whl (1.8 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 24.2
    Uninstalling pip-24.2:
      Successfully uninstalled pip-24.2
Successfully installed pip-24.1.2

[notice] A new release of pip is available: 24.1.2 -> 24.2
[notice] To update, run: pip install --upgrade pip
$ python3 -m pip install certifi
Collecting certifi
  Using cached certifi-2024.7.4-py3-none-any.whl.metadata (2.2 kB)
Using cached certifi-2024.7.4-py3-none-any.whl (162 kB)
Installing collected packages: certifi
Successfully installed certifi-2024.7.4

[notice] A new release of pip is available: 24.1.2 -> 24.2
[notice] To update, run: pip install --upgrade pip

Output

No response

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[hugovk]

To get a rough idea of numbers, here's the pip installs on macOS of pip itself from PyPI, for the last 28 days:

system_name	distro_version	download_count	percent	summed_percent
Darwin	1.0	71	0.00%	99.99%
Darwin	10.6	33	0.00%	99.99%
Darwin	10.9	54	0.00%	99.99%
Darwin	10.10	453	0.01%	99.99%
Darwin	10.11	532	0.01%	99.98%
Darwin	10.12	590	0.01%	99.97%
Darwin	10.13	13,197	0.32%	99.96%
Darwin	10.14	16,013	0.39%	99.64%
Darwin	10.15	33,907	0.82%	99.25%
Darwin	11	373,213	9.04%	98.43%
Darwin	12	782,115	18.94%	89.39%
Darwin	13	463,612	11.23%	70.45%
Darwin	14	2,373,138	57.47%	59.22%
Darwin	15	39,512	0.96%	1.75%
Darwin	16	575	0.01%	0.79%
Darwin	17	10,773	0.26%	0.78%
Darwin	18	21,555	0.52%	0.52%
Darwin	19	62	0.00%	0.00%
Darwin	None	101	0.00%	0.00%

(via pypinfo --days 28 --percent --limit 1000 --json pip system distro-version > macos.json using https://github.com/ofek/pypinfo and summarised with macos-versions.py)

This shows 99.96% is macOS 10.13 or newer.

[uranusjr]

Would it be a good idea to simply fall back to certifi on macOS 12? cc @sethmlarson

[ned-deily]

That's macOS 10.12, not macOS 12. We already depend on certifi for users of the python.org macOS installers. We think that the workaround of installing the older version of pip on these older systems will not prove too burdensome for the small number of users we expect that might be affected. And that allows the vast majority of users who are on newer versions of macOS to have the benefits of the latest version of pip.

[uranusjr]

Oh, the ancient 10.12. Got it. What should pip do to improve this then?

[ned-deily]

Talking with @sethmlarson, I think the best thing to do is to fix the issue in truststore and eventually release it in a version of pip. I've offered assistance in testing. There probably won't be a lot of users affected by this. The python.org installer for 3.12.5 will reference this issue and provides the downgrade workaround when running on the older macOS versions. Presumably, other users and downstream distributors will find this issue here.

[sethmlarson]

Agreed with @ned-deily, thanks for the offer to help test Ned! We have an issue already created on the Truststore project if you'd like to follow along there: sethmlarson/truststore#119