test_ssl often fails against OpenSSL 3.4.0 #127026
Description
Bug report
Bug description:
test_ssl often fails on the main, 3.13 and 3.12 branches when running make test. (This is on an M1 MacBook Pro running MacOS 15.1.) Sometimes it fails the first time, succeeds when rerun, fails both times, or doesn't fail at all. Though the tracebacks seem to be substantially different, the fundamental problem on 3.13 and main seems to be a bad TLS certificate. I see a different error on 3.12.
Here's the latest (long) failure traceback on main:
Traceback (most recent call last):
File "/Users/skip/src/python/cpython/Lib/test/test_ssl.py", line 2262, in wrap_conn
self.sslconn = self.server.context.wrap_socket(
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
self.sock, server_side=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/skip/src/python/cpython/Lib/ssl.py", line 455, in wrap_socket
return self.sslsocket_class._create(
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
sock=sock,
^^^^^^^^^^
...<5 lines>...
session=session
^^^^^^^^^^^^^^^
)
^
File "/Users/skip/src/python/cpython/Lib/ssl.py", line 1076, in _create
self.do_handshake()
~~~~~~~~~~~~~~~~~^^
File "/Users/skip/src/python/cpython/Lib/ssl.py", line 1372, in do_handshake
self._sslobj.do_handshake()
~~~~~~~~~~~~~~~~~~~~~~~~~^^
ssl.SSLError: [SSL: SSLV3_ALERT_BAD_CERTIFICATE] ssl/tls alert bad certificate (_ssl.c:1024)
server: new connection from ('127.0.0.1', 62987)
server: new connection from ('127.0.0.1', 62988)
server: bad connection attempt from ('127.0.0.1', 62984):
Traceback (most recent call last):
File "/Users/skip/src/python/cpython/Lib/test/test_ssl.py", line 2262, in wrap_conn
self.sslconn = self.server.context.wrap_socket(
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
self.sock, server_side=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/skip/src/python/cpython/Lib/ssl.py", line 455, in wrap_socket
return self.sslsocket_class._create(
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
sock=sock,
^^^^^^^^^^
...<5 lines>...
session=session
^^^^^^^^^^^^^^^
)
^
File "/Users/skip/src/python/cpython/Lib/ssl.py", line 1076, in _create
self.do_handshake()
~~~~~~~~~~~~~~~~~^^
File "/Users/skip/src/python/cpython/Lib/ssl.py", line 1372, in do_handshake
self._sslobj.do_handshake()
~~~~~~~~~~~~~~~~~~~~~~~~~^^
ssl.SSLError: [SSL: SSLV3_ALERT_BAD_CERTIFICATE] ssl/tls alert bad certificate (_ssl.c:1024)
server: bad connection attempt from ('127.0.0.1', 62987):
Traceback (most recent call last):
File "/Users/skip/src/python/cpython/Lib/test/test_ssl.py", line 2262, in wrap_conn
self.sslconn = self.server.context.wrap_socket(
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
self.sock, server_side=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/skip/src/python/cpython/Lib/ssl.py", line 455, in wrap_socket
return self.sslsocket_class._create(
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
sock=sock,
^^^^^^^^^^
...<5 lines>...
session=session
^^^^^^^^^^^^^^^
)
^
File "/Users/skip/src/python/cpython/Lib/ssl.py", line 1076, in _create
self.do_handshake()
~~~~~~~~~~~~~~~~~^^
File "/Users/skip/src/python/cpython/Lib/ssl.py", line 1372, in do_handshake
self._sslobj.do_handshake()
~~~~~~~~~~~~~~~~~~~~~~~~~^^
ssl.SSLError: [SSL: SSLV3_ALERT_BAD_CERTIFICATE] ssl/tls alert bad certificate (_ssl.c:1024)
server: connection cipher is now ('TLS_AES_256_GCM_SHA384', 'TLSv1.3', 256)
Connection reset by peer: ('127.0.0.1', 62988)
server: new connection from ('127.0.0.1', 62993)
Here's the latest traceback from the 3.13 branch`:
Re-running test_ssl in verbose mode (matching: test_wrong_cert_tls13)
test_ssl: testing with 'OpenSSL 3.4.0 22 Oct 2024' (3, 4, 0, 0, 0)
under Mac ('15.1', ('', '', ''), 'arm64')
HAS_SNI = True
OP_ALL = 0x80000050
OP_NO_TLSv1_1 = 0x10000000
test_wrong_cert_tls13 (test.test_ssl.ThreadedTests.test_wrong_cert_tls13) ... server: new connection from ('127.0.0.1', 53960)
server: bad connection attempt from ('127.0.0.1', 53960):
Traceback (most recent call last):
File "/Users/skip/src/python/py3.13/Lib/test/test_ssl.py", line 2275, in wrap_conn
self.sslconn = self.server.context.wrap_socket(
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
self.sock, server_side=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/skip/src/python/py3.13/Lib/ssl.py", line 455, in wrap_socket
return self.sslsocket_class._create(
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
sock=sock,
^^^^^^^^^^
...<5 lines>...
session=session
^^^^^^^^^^^^^^^
)
^
File "/Users/skip/src/python/py3.13/Lib/ssl.py", line 1076, in _create
self.do_handshake()
~~~~~~~~~~~~~~~~~^^
File "/Users/skip/src/python/py3.13/Lib/ssl.py", line 1372, in do_handshake
self._sslobj.do_handshake()
~~~~~~~~~~~~~~~~~~~~~~~~~^^
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1020)
FAIL
3.12:
% ./python.exe -m test test_ssl
Using random seed: 2271976119
0:00:00 load avg: 1.25 Run 1 test sequentially
0:00:00 load avg: 1.25 [1/1] test_ssl
test test_ssl failed -- Traceback (most recent call last):
File "/Users/skip/src/python/py3.12/Lib/test/test_ssl.py", line 4882, in test_preauth_data_to_tls_server
self.assertIn("before TLS handshake with data", wrap_error.args[1])
AssertionError: 'before TLS handshake with data' not found in '[SSL] record layer failure (_ssl.c:1000)'
Just after the above failure, I ran the same command and it succeeded...
In all three cases, _ssl...so was linked against OpenSSL 3.4.0, for example:
% otool -L Modules/_ssl.cpython-313t-darwin.so
Modules/_ssl.cpython-313t-darwin.so:
/opt/homebrew/opt/openssl@3/lib/libssl.3.dylib (compatibility version 3.0.0, current version 3.0.0)
/opt/homebrew/opt/openssl@3/lib/libcrypto.3.dylib (compatibility version 3.0.0, current version 3.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1351.0.0)
(py313t) ~/src/python/py3.13% ls -l /opt/homebrew/opt/openssl@3
lrwxr-xr-x 1 skip admin 25 Oct 31 14:02 /opt/homebrew/opt/openssl@3 -> ../Cellar/openssl@3/3.4.0
I don't recall if I saw failures against earlier OpenSSL versions. Is 3.4.0 maybe different enough that the _ssl.c code has to be tweaked a bit?
CPython versions tested on:
3.12, 3.13, CPython main branch
Operating systems tested on:
macOS