Attempt to run test suite on clean build of main branch hangs on test.test_concurrent_futures.test_deadlock (OSX)

[Phil305]

Bug report

Bug description:

When running python.exe -m test on a clean build from the main branch (commit d07483292b), it hangs for hours and won't continue (in my case, it hang for 3 hours 45 minutes before I killed the process).

Python built fine, it's just running the tests that fails... is this expected on OSX? By the way, I'm not running on the M1 chip.

My environment:
OS: macOS Sonoma (14.1.1)
Processor: 2.4 GHz 8-Core Intel Core i9
Memory: 32 GB 2667 MHz DDR4

Result of uname -a:
Darwin pjulien-mbp 23.1.0 Darwin Kernel Version 23.1.0: Mon Oct 9 21:27:27 PDT 2023; root:xnu-10002.41.9~6/RELEASE_X86_64 x86_64

Output from me running python.exe -m test:

➜  cpython git:(main) time ./python.exe -m test
== CPython 3.13.0a2+ (heads/main:20b7e0842d, Dec 14 2023, 18:51:12) [Clang 15.0.0 (clang-1500.0.40.1)]
== macOS-14.1.1-x86_64-i386-64bit little-endian
== Python build: debug
== cwd: /Users/pjulien/repos/cpython/build/test_python_worker_95138æ
== CPU count: 16
== encodings: locale=UTF-8 FS=utf-8
== resources: all test resources are disabled, use -u option to unskip tests

Using random seed: 4025894355
0:00:00 load avg: 5.20 Run 469 tests sequentially
0:00:00 load avg: 5.20 [  1/469] test.test_asyncio.test_base_events
0:00:01 load avg: 5.20 [  2/469] test.test_asyncio.test_buffered_proto
0:00:01 load avg: 5.20 [  3/469] test.test_asyncio.test_context
0:00:02 load avg: 5.20 [  4/469] test.test_asyncio.test_eager_task_factory
0:00:03 load avg: 5.20 [  5/469] test.test_asyncio.test_events
0:00:13 load avg: 5.45 [  6/469] test.test_asyncio.test_futures
0:00:16 load avg: 5.17 [  7/469] test.test_asyncio.test_futures2
0:00:17 load avg: 5.17 [  8/469] test.test_asyncio.test_locks
0:00:18 load avg: 5.17 [  9/469] test.test_asyncio.test_pep492
0:00:18 load avg: 5.88 [ 10/469] test.test_asyncio.test_proactor_events
0:00:19 load avg: 5.88 [ 11/469] test.test_asyncio.test_protocols
0:00:19 load avg: 5.88 [ 12/469] test.test_asyncio.test_queues
0:00:20 load avg: 5.88 [ 13/469] test.test_asyncio.test_runners
0:00:20 load avg: 5.88 [ 14/469] test.test_asyncio.test_selector_events
0:00:20 load avg: 5.88 [ 15/469] test.test_asyncio.test_sendfile
0:00:23 load avg: 5.88 [ 16/469] test.test_asyncio.test_server
/Users/pjulien/repos/cpython/Lib/asyncio/events.py:84: ResourceWarning: unclosed <socket.socket fd=7, family=1, type=1, proto=0, laddr=./test_python_pv_abkp5.sock>
  self._context.run(self._callback, *self._args)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
0:00:23 load avg: 5.88 [ 17/469] test.test_asyncio.test_sock_lowlevel
0:00:26 load avg: 5.97 [ 18/469] test.test_asyncio.test_ssl
0:00:33 load avg: 5.97 [ 19/469] test.test_asyncio.test_sslproto
0:00:37 load avg: 5.73 [ 20/469] test.test_asyncio.test_streams
0:00:38 load avg: 5.51 [ 21/469] test.test_asyncio.test_subprocess
0:00:49 load avg: 4.98 [ 22/469] test.test_asyncio.test_taskgroups
0:00:55 load avg: 4.82 [ 23/469] test.test_asyncio.test_tasks
0:01:16 load avg: 4.55 [ 24/469] test.test_asyncio.test_threads
0:01:17 load avg: 4.55 [ 25/469] test.test_asyncio.test_timeouts
0:01:17 load avg: 4.55 [ 26/469] test.test_asyncio.test_transports
0:01:18 load avg: 4.55 [ 27/469] test.test_asyncio.test_unix_events
0:01:19 load avg: 4.91 [ 28/469] test.test_asyncio.test_waitfor
0:01:20 load avg: 4.91 [ 29/469] test.test_asyncio.test_windows_events
test.test_asyncio.test_windows_events skipped -- Windows only
0:01:20 load avg: 4.91 [ 30/469] test.test_asyncio.test_windows_utils -- test.test_asyncio.test_windows_events skipped
test.test_asyncio.test_windows_utils skipped -- Windows only
0:01:20 load avg: 4.91 [ 31/469] test.test_concurrent_futures.test_as_completed -- test.test_asyncio.test_windows_utils skipped
0:01:35 load avg: 4.18 [ 32/469] test.test_concurrent_futures.test_deadlock
^CWarning -- multiprocessing.process._dangling was modified by test.test_concurrent_futures.test_deadlock
Warning --   Before: set()
Warning --   After:  {<weakref at 0x10edf5710; to 'ForkProcess' at 0x10ed75000>, <weakref at 0x10edf7540; to 'ForkProcess' at 0x10ec8b580>} 
Warning -- threading._dangling was modified by test.test_concurrent_futures.test_deadlock
Warning --   Before: {<weakref at 0x10ed827b0; to '_MainThread' at 0x10cd52770>}
Warning --   After:  {<weakref at 0x10edf59b0; to '_ExecutorManagerThread' at 0x10ec8a090>, <weakref at 0x10edf70e0; to 'Thread' at 0x10edf91e0>, <weakref at 0x10edf5c50; to '_MainThread' at 0x10cd52770>} 


^CWarning -- Unraisable exception
Exception ignored in: <function _ExecutorManagerThread.__init__.<locals>.weakref_cb at 0x10ddd7dd0>
Traceback (most recent call last):
  File "/Users/pjulien/repos/cpython/Lib/concurrent/futures/process.py", line 312, in weakref_cb
    with shutdown_lock:
KeyboardInterrupt: 

== Tests result: INTERRUPTED ==

438 tests omitted:
    test.test_concurrent_futures.test_deadlock
    test.test_concurrent_futures.test_future
    test.test_concurrent_futures.test_init
    test.test_concurrent_futures.test_process_pool
    test.test_concurrent_futures.test_shutdown
    test.test_concurrent_futures.test_thread_pool
    test.test_concurrent_futures.test_wait
    test.test_future_stmt.test_future
    test.test_future_stmt.test_future_flags
    test.test_future_stmt.test_future_multiple_features
    test.test_future_stmt.test_future_multiple_imports
    test.test_future_stmt.test_future_single_import
    test.test_gdb.test_backtrace test.test_gdb.test_cfunction
    test.test_gdb.test_cfunction_full test.test_gdb.test_misc
    test.test_gdb.test_pretty_print test.test_inspect.test_inspect
    test.test_multiprocessing_fork.test_manager
    test.test_multiprocessing_fork.test_misc
    test.test_multiprocessing_fork.test_processes
    test.test_multiprocessing_fork.test_threads
    test.test_multiprocessing_forkserver.test_manager
    test.test_multiprocessing_forkserver.test_misc
    test.test_multiprocessing_forkserver.test_processes
    test.test_multiprocessing_forkserver.test_threads
    test.test_multiprocessing_spawn.test_manager
    test.test_multiprocessing_spawn.test_misc
    test.test_multiprocessing_spawn.test_processes
    test.test_multiprocessing_spawn.test_threads test___all__
    test__locale test__opcode test__osx_support test__xxinterpchannels
    test__xxsubinterpreters test_abc test_abstract_numbers
    test_argparse test_array test_asdl_parser test_ast test_asyncgen
    test_atexit test_audit test_augassign test_base64
    test_baseexception test_bdb test_bigaddrspace test_bigmem
    test_binascii test_binop test_bisect test_bool test_buffer
    test_bufio test_builtin test_bytes test_bz2 test_c_locale_coercion
    test_calendar test_call test_capi test_charmapcodec test_class
    test_clinic test_cmath test_cmd test_cmd_line test_cmd_line_script
    test_code test_code_module test_codeccallbacks
    test_codecencodings_cn test_codecencodings_hk
    test_codecencodings_iso2022 test_codecencodings_jp
    test_codecencodings_kr test_codecencodings_tw test_codecmaps_cn
    test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr
    test_codecmaps_tw test_codecs test_codeop test_collections
    test_colorsys test_compare test_compile test_compileall
    test_compiler_assemble test_compiler_codegen test_complex
    test_configparser test_contains test_context test_contextlib
    test_contextlib_async test_copy test_copyreg test_coroutines
    test_cppext test_cprofile test_csv test_ctypes test_curses
    test_dataclasses test_datetime test_dbm test_dbm_dumb test_dbm_gnu
    test_dbm_ndbm test_decimal test_decorators test_defaultdict
    test_deque test_descr test_descrtut test_devpoll test_dict
    test_dict_version test_dictcomps test_dictviews test_difflib
    test_dis test_doctest test_doctest2 test_docxmlrpc test_dtrace
    test_dynamic test_dynamicclassattribute test_eintr test_email
    test_embed test_ensurepip test_enum test_enumerate test_eof
    test_epoll test_errno test_except_star test_exception_group
    test_exception_hierarchy test_exception_variations test_exceptions
    test_extcall test_faulthandler test_fcntl test_file
    test_file_eintr test_filecmp test_fileinput test_fileio
    test_fileutils test_finalization test_float test_flufl
    test_fnmatch test_fork1 test_format test_fractions test_frame
    test_frozen test_fstring test_ftplib test_funcattrs test_functools
    test_gc test_generated_cases test_generator_stop test_generators
    test_genericalias test_genericclass test_genericpath test_genexps
    test_getopt test_getpass test_getpath test_gettext test_glob
    test_global test_grammar test_graphlib test_grp test_gzip
    test_hash test_hashlib test_heapq test_hmac test_html
    test_htmlparser test_http_cookiejar test_http_cookies test_httplib
    test_httpservers test_idle test_imaplib test_import test_importlib
    test_index test_int test_int_literal test_interpreters test_io
    test_ioctl test_ipaddress test_isinstance test_iter test_iterlen
    test_itertools test_json test_keyword test_keywordonlyarg
    test_kqueue test_largefile test_launcher test_linecache test_list
    test_listcomps test_lltrace test_locale test_logging test_long
    test_longexp test_lzma test_mailbox test_marshal test_math
    test_math_property test_memoryio test_memoryview test_metaclass
    test_mimetypes test_minidom test_mmap test_module
    test_modulefinder test_monitoring test_msvcrt test_multibytecodec
    test_multiprocessing_main_handling test_named_expressions
    test_netrc test_ntpath test_numeric_tower test_opcache
    test_opcodes test_openpty test_operator test_optparse
    test_ordered_dict test_os test_osx_env test_pathlib test_patma
    test_pdb test_peepholer test_peg_generator test_pep646_syntax
    test_perf_profiler test_perfmaps test_pickle test_picklebuffer
    test_pickletools test_pkg test_pkgutil test_platform test_plistlib
    test_poll test_popen test_poplib test_positional_only_arg
    test_posix test_posixpath test_pow test_pprint test_print
    test_profile test_property test_pstats test_pty test_pulldom
    test_pwd test_py_compile test_pyclbr test_pydoc test_pyexpat
    test_queue test_quopri test_raise test_random test_range test_re
    test_readline test_regrtest test_repl test_reprlib test_resource
    test_richcmp test_rlcompleter test_robotparser test_runpy test_sax
    test_sched test_scope test_script_helper test_secrets test_select
    test_selectors test_set test_setcomps test_shelve test_shlex
    test_shutil test_signal test_site test_slice test_smtplib
    test_smtpnet test_socket test_socketserver test_sort
    test_source_encoding test_sqlite3 test_ssl test_stable_abi_ctypes
    test_startfile test_stat test_statistics test_str test_strftime
    test_string test_string_literals test_stringprep test_strptime
    test_strtod test_struct test_structseq test_subclassinit
    test_subprocess test_sundry test_super test_support test_symtable
    test_syntax test_sys test_sys_setprofile test_sys_settrace
    test_sysconfig test_syslog test_tabnanny test_tarfile test_tcl
    test_tempfile test_termios test_textwrap test_thread
    test_threadedtempfile test_threading test_threading_local
    test_threadsignals test_time test_timeit test_timeout test_tkinter
    test_tokenize test_tomllib test_tools test_trace test_traceback
    test_tracemalloc test_ttk test_ttk_textonly test_tty test_tuple
    test_turtle test_type_aliases test_type_annotations
    test_type_cache test_type_comments test_type_params
    test_typechecks test_types test_typing test_ucn test_unary
    test_unicode_file test_unicode_file_functions
    test_unicode_identifiers test_unicodedata test_unittest
    test_univnewlines test_unpack test_unpack_ex test_unparse
    test_urllib test_urllib2 test_urllib2_localnet test_urllib2net
    test_urllib_response test_urllibnet test_urlparse test_userdict
    test_userlist test_userstring test_utf8_mode test_utf8source
    test_uuid test_venv test_wait3 test_wait4 test_warnings test_wave
    test_weakref test_weakset test_webbrowser test_winconsoleio
    test_winreg test_winsound test_with test_wmi test_wsgiref
    test_xml_dom_minicompat test_xml_etree test_xml_etree_c
    test_xmlrpc test_xxlimited test_xxtestfuzz test_yield_from
    test_zipapp test_zipfile test_zipfile64 test_zipimport
    test_zipimport_support test_zlib test_zoneinfo

2 tests skipped:
    test.test_asyncio.test_windows_events
    test.test_asyncio.test_windows_utils

29 tests OK.

Test suite interrupted by signal SIGINT.

Total duration: 3 hour 45 min
Total tests: run=2,441 skipped=56
Total test files: run=31/469 skipped=2
Result: INTERRUPTED
^C

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

[ronaldoussoren]

A hang in this test is not expected and is not something I've seem on my machine.

Does this (fairly) consistently happen for you? If so, could you test if the PR #113567 improves things? This is a PR that fixes another issue with multiprocessing and I wonder if this is related.

[Phil305]

A hang in this test is not expected and is not something I've seem on my machine.

Does this (fairly) consistently happen for you? If so, could you test if the PR #113567 improves things? This is a PR that fixes another issue with multiprocessing and I wonder if this is related.

Thanks!

I checked out that commit, and it got me past the test_concurrent_futures.test_deadlock test, but now the tests freeze on test.test_gdb.test_backtrace.

I'm going to give up on trying to build/run Python on my Mac directly and simply use the included dev container; seems like a better solution than me keeping this task open and taking up space in the issue tracker for an issue that might only be affecting me.