Description
Feature or enhancement
This issue is to track the remaining bugs that prevent the Python test suite from passing with the GIL disabled, as well as the PRs under review that fix known issues.
Note that this is not intended to cover all free-threading related issues: some thread-safety issues don't result in tests crashing or failing and are not tracked here. This is intended to cover the minimum set of changes needed to ensure that the test suite passes reliably with the GIL disabled.
PRs
- gh-112075: _Py_dict_lookup needs to lock shared keys #117528
- gh-112075: Lock when inserting into shared keys #117824
- gh-117783: Immortalize objects that use deferred reference counting #118112
Integration branch
Branches to merge
git merge disable-gil-default &&
git merge gh-112075-lock-split-keys &&
git merge gh-117783-immortalize
This branch contains the merge of the above PRs and is used to find bugs that are not covered by outstanding PRs. Note that setting PYTHON_GIL=0 isn't sufficient because some tests do not propagate the environment variable to tests in subprocesses. Instead, the branch changes the default mode to disable the GIL for testing purposes.
Issues without PRs that fix them
- N/A
Bugs without issues
Windows
- test_no_stale_references times out - https://github.com/python/cpython/actions/runs/8514458340/job/23320231100?pr=117437
Linux
test.test_concurrent_futures.test_deadlockdeadlocks ? - https://buildbot.python.org/all/#/builders/1281/builds/148/steps/5/logs/stdio
macOS
test_interrupt_main_subthread-RuntimeError: release unlocked lockhttps://buildbot.python.org/all/#/builders/1367/builds/117/steps/5/logs/stdiotest.test_multiprocessing_spawn.test_processes.WithProcessesTestPool.test_release_task_refs-10 != 0
https://buildbot.python.org/all/#/builders/1367/builds/119/steps/5/logs/stdio
test_daemon_threads_shutdown_stderr_deadlock-Assertion failed: (tstate == NULL || tstate->interp == interp), function _PyEval_ReleaseLock, file ceval_gil.c,test_queuetest_shutdown_all_methods_in_many_threads- AttributeError: 'Queue' object has no attribute '_lock'
test_shutdown_all_methods_in_many_threads (test.test_queue.CQueueTest.test_shutdown_all_methods_in_many_threads) ... Warning -- Uncaught thread exception: AttributeError
Exception in thread Thread-2081 (_read_msg_thread):
Traceback (most recent call last):
File "/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/threading.py", line 1039, in _bootstrap_inner
self.run()
~~~~~~~~^^
File "/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/threading.py", line 990, in run
self._target(*self._args, **self._kwargs)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/test/test_queue.py", line 352, in _read_msg_thread
q.get(False)
~~~~~^^^^^^^
File "/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/queue.py", line 216, in get
item = self._get()
~~~~~~~~~^^
File "/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/threading.py", line 304, in __enter__
return self._lock.__enter__()
^^^^^^^^^^
AttributeError: 'Queue' object has no attribute '_lock'
Old bugs?
test.test_smtpnet.SmtpSSLTest.test_connect_default_portconnection closed (possibly related)test.test_multiprocessing_forkserver.test_processeshangs ?