Skip to content

Segfault on Python 3.13.0rc3 (free-threaded) with requests #124984

Closed
@AA-Turner

Description

@AA-Turner

Crash report

What happened?

Reproducer:

import queue
import threading

import requests


class HyperlinkAvailabilityCheckWorker(threading.Thread):
    def __init__(self, rqueue, wqueue) -> None:
        self.rqueue = rqueue
        self.wqueue = wqueue
        self._session = requests.Session()
        super().__init__(daemon=True)

    def run(self) -> None:
        while True:
            uri = self.wqueue.get()
            if not uri:
                self._session.close()
                break

            self._session.request(
                'HEAD',
                url=uri,
                timeout=30,
                verify=True,
            )
            self.rqueue.put(uri)
            self.wqueue.task_done()


def test_crash():
    for i in range(1_000):
        print(f'loop: {i}')

        # setup
        rqueue = queue.Queue()
        wqueue = queue.Queue()
        workers: list[HyperlinkAvailabilityCheckWorker] = []

        # invoke threads
        num_workers = 2
        for _ in range(num_workers):
            thread = HyperlinkAvailabilityCheckWorker(rqueue, wqueue)
            thread.start()
            workers.append(thread)

        # check
        total_links = 0
        for hyperlink in (
            'https://python.org/dev/',
            'https://peps.python.org/pep-0008/',
        ):
            wqueue.put(hyperlink, False)
            total_links += 1

        done = 0
        while done < total_links:
            result = rqueue.get()
            print(result)
            done += 1

        # shutdown_threads
        wqueue.join()
        for _worker in workers:
            wqueue.put('', False)


if __name__ == '__main__':
    import sys

    print(f'GIL enabled?: {sys._is_gil_enabled()}')
    print()
    test_crash()

Sample output:

GIL enabled?: False

loop: 0
https://peps.python.org/pep-0008/
https://python.org/dev/
loop: 1
https://peps.python.org/pep-0008/
https://python.org/dev/
loop: 2
https://peps.python.org/pep-0008/
https://python.org/dev/
loop: 3
Fatal Python error: Segmentation fault

Thread 0x00007f3dc5e00640 (most recent call first):
  File "/usr/lib/python3.13/ssl.py", line 513 in set_alpn_protocols
  File "/home/runner/venv-3.13/lib/python3.13t/site-packages/urllib3/util/ssl_.py", line 467 in ssl_wrap_socket
  File "/home/runner/venv-3.13/lib/python3.13t/site-packages/urllib3/connection.py", line 909 in _ssl_wrap_socket_and_match_hostname
  File "/home/runner/venv-3.13/lib/python3.13t/site-packages/urllib3/connection.py", line 730 in connect
  File "/home/runner/venv-3.13/lib/python3.13t/site-packages/urllib3/connectionpool.py", line 1095 in _validate_conn
  File "/home/runner/venv-3.13/lib/python3.13t/site-packages/urllib3/connectionpool.py", line 466 in _make_request
  File "/home/runner/venv-3.13/lib/python3.13t/site-packages/urllib3/connectionpool.py", line 789 in urlopen
  File "/home/runner/venv-3.13/lib/python3.13t/site-packages/requests/adapters.py", line 667 in send
  File "/home/runner/venv-3.13/lib/python3.13t/site-packages/requests/sessions.py", line 703 in send
  File "/home/runner/venv-3.13/lib/python3.13t/site-packages/requests/sessions.py", line 589 in request
  File "/home/runner/work/sphinx/sphinx/./tests/test_build.py", line 21 in run
  File "/usr/lib/python3.13/threading.py", line 1041 in _bootstrap_inner
  File "/usr/lib/python3.13/threading.py", line 1012 in _bootstrap

Current thread 0x00007f3dbfe00640 (most recent call first):
  File "/usr/lib/python3.13/ssl.py", line 1372 in do_handshake
  File "/usr/lib/python3.13/ssl.py", line 1076 in _create
  File "/usr/lib/python3.13/ssl.py", line 455 in wrap_socket
  File "/home/runner/venv-3.13/lib/python3.13t/site-packages/urllib3/util/ssl_.py", line 513 in _ssl_wrap_socket_impl
  File "/home/runner/venv-3.13/lib/python3.13t/site-packages/urllib3/util/ssl_.py", line 469 in ssl_wrap_socket
  File "/home/runner/venv-3.13/lib/python3.13t/site-packages/urllib3/connection.py", line 909 in _ssl_wrap_socket_and_match_hostname
  File "/home/runner/venv-3.13/lib/python3.13t/site-packages/urllib3/connection.py", line 730 in connect
  File "/home/runner/venv-3.13/lib/python3.13t/site-packages/urllib3/connectionpool.py", line 1095 in _validate_conn
  File "/home/runner/venv-3.13/lib/python3.13t/site-packages/urllib3/connectionpool.py", line 466 in _make_request
  File "/home/runner/venv-3.13/lib/python3.13t/site-packages/urllib3/connectionpool.py", line 789 in urlopen
  File "/home/runner/venv-3.13/lib/python3.13t/site-packages/requests/adapters.py", line 667 in send
  File "/home/runner/venv-3.13/lib/python3.13t/site-packages/requests/sessions.py", line 703 in send
  File "/home/runner/venv-3.13/lib/python3.13t/site-packages/requests/sessions.py", line 589 in request
  File "/home/runner/work/sphinx/sphinx/./tests/test_build.py", line 21 in run
  File "/usr/lib/python3.13/threading.py", line 1041 in _bootstrap_inner
  File "/usr/lib/python3.13/threading.py", line 1012 in _bootstrap

Thread 0x00007f3dc9140000 (most recent call first):
  File "/usr/lib/python3.13/threading.py", line 359 in wait
  File "/usr/lib/python3.13/queue.py", line 202 in get
  File "/home/runner/work/sphinx/sphinx/./tests/test_build.py", line 58 in test_crash
  File "/home/runner/work/sphinx/sphinx/./tests/test_build.py", line 73 in <module>
/home/runner/work/_temp/f534164e-5195-4157-abd9-f2f4e6fed6dd.sh: line 5:  2783 Segmentation fault      (core dumped) python -u ./tests/test_build.py

@JelleZijlstra ran this on macOS with the following lldb traceback:

(lldb) bt
* thread #6, stop reason = signal SIGABRT
  * frame #0: 0x0000000194852a60 libsystem_kernel.dylib`__pthread_kill + 8
    frame #1: 0x000000019488ac20 libsystem_pthread.dylib`pthread_kill + 288
    frame #2: 0x0000000194797a30 libsystem_c.dylib`abort + 180
    frame #3: 0x00000001946a7dc4 libsystem_malloc.dylib`malloc_vreport + 896
    frame #4: 0x00000001946ab430 libsystem_malloc.dylib`malloc_report + 64
    frame #5: 0x00000001946c5494 libsystem_malloc.dylib`find_zone_and_free + 528
    frame #6: 0x0000000100871448 libssl.3.dylib`SSL_CTX_set_alpn_protos + 180
    frame #7: 0x000000010075e5d0 _ssl.cpython-313t-darwin.so`_ssl__SSLContext__set_alpn_protocols [inlined] _ssl__SSLContext__set_alpn_protocols_impl(self=0x0000028ca0798f30, protos=0x0000000172eaa3e0) at _ssl.c:3381:9 [opt]
    frame #8: 0x000000010075e59c _ssl.cpython-313t-darwin.so`_ssl__SSLContext__set_alpn_protocols(self=0x0000028ca0798f30, arg=<unavailable>) at _ssl.c.h:528:20 [opt]
    frame #9: 0x0000000100066354 python`method_vectorcall_O(func=0x0000028ca09bec90, args=0x00000001005ecc20, nargsf=<unavailable>, kwnames=<unavailable>) at descrobject.c:475:24 [opt]
    frame #10: 0x00000001000585d0 python`PyObject_Vectorcall [inlined] _PyObject_VectorcallTstate(tstate=0x000000010380a600, callable=0x0000028ca09bec90, args=<unavailable>, nargsf=<unavailable>, kwnames=<unavailable>) at pycore_call.h:168:11 [opt]
    frame #11: 0x00000001000585a8 python`PyObject_Vectorcall(callable=0x0000028ca09bec90, args=<unavailable>, nargsf=<unavailable>, kwnames=<unavailable>) at call.c:327:12 [opt]
    frame #12: 0x0000000100194098 python`_PyEval_EvalFrameDefault(tstate=<unavailable>, frame=<unavailable>, throwflag=<unavailable>) at generated_cases.c.h:813:23 [opt]
python(84085,0x172eab000) malloc: *** error for object 0x600000840000: pointer being freed was not allocated
python(84085,0x172eab000) malloc: *** set a breakpoint in malloc_error_break to debug

CPython versions tested on:

3.13

Operating systems tested on:

Linux, macOS

Output from running 'python -VV' on the command line:

Python 3.13.0rc3+ experimental free-threading build (main, Oct 4 2024, 08:50:03) [GCC 11.4.0]

Linked PRs

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.13bugs and security fixes3.14bugs and security fixesextension-modulesC modules in the Modules dirtopic-SSLtopic-free-threadingtype-crashA hard crash of the interpreter, possibly with a core dump

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions