Tests crash on macos Sonoma

[sobolevn]

Crash report

I've experienced a crash on main branch.
I just used ./python.exe -m test

This is the all output I have for now.
Cannot reproduce with smaller set of tests.

0:13:53 load avg: 2.12 [371/463/4] test_super
0:13:53 load avg: 2.12 [372/463/4] test_support
[1]    12160 killed     ./python.exe -m test

Linked PRs

• gh-109981: Fix support.fd_count() on macOS 14 #112797
• [3.12] gh-109981: Fix support.fd_count() on macOS 14 (GH-112797) #112824
• [3.11] gh-109981: Fix support.fd_count() on macOS 14 (GH-112797) #112825
• gh-109981: Resolve situation on iOS regarding fd_count. #132823
• [3.13] gh-109981: Resolve situation on iOS regarding fd_count. (GH-132823) #132824

[sobolevn]

With more info:

test_detect_api_mismatch (test.test_support.TestSupport.test_detect_api_mismatch) ... ok
test_detect_api_mismatch__ignore (test.test_support.TestSupport.test_detect_api_mismatch__ignore) ... ok
test_fd_count (test.test_support.TestSupport.test_fd_count) ... [1]    16728 killed     PYTHONFAULTHANDLER=1 ./python.exe -m test -v

However, this is dependent on some other test, for now I don't know which one.

[vstinner]

I don't know which one.

Try to run:

./python.exe -m test.bisect_cmd -o bisect test_support

Then look into bisect file result.

[sobolevn]

I wrote this simple script (because bisect's output didn't reproduce this problem):

set -e

tests=$(cat << EOF
test.test_asyncio.test_base_events
test.test_asyncio.test_buffered_proto
test.test_asyncio.test_context
test.test_asyncio.test_eager_task_factory
test.test_asyncio.test_events
test.test_asyncio.test_futures
test.test_asyncio.test_futures2
test.test_asyncio.test_locks
test.test_asyncio.test_pep492
test.test_asyncio.test_proactor_events
test.test_asyncio.test_protocols
test.test_asyncio.test_queues
test.test_asyncio.test_runners
test.test_asyncio.test_selector_events
test.test_asyncio.test_sendfile
test.test_asyncio.test_server
test.test_asyncio.test_sock_lowlevel
test.test_asyncio.test_ssl
test.test_asyncio.test_sslproto
test.test_asyncio.test_streams
test.test_asyncio.test_subprocess
test.test_asyncio.test_taskgroups
test.test_asyncio.test_tasks
test.test_asyncio.test_threads
test.test_asyncio.test_timeouts
test.test_asyncio.test_transports
test.test_asyncio.test_unix_events
test.test_asyncio.test_waitfor
test.test_asyncio.test_windows_events
test.test_asyncio.test_windows_utils
test.test_concurrent_futures.test_as_completed
test.test_concurrent_futures.test_deadlock
test.test_concurrent_futures.test_future
test.test_concurrent_futures.test_init
test.test_concurrent_futures.test_process_pool
test.test_concurrent_futures.test_shutdown
test.test_concurrent_futures.test_thread_pool
test.test_concurrent_futures.test_wait
test.test_future_stmt.test_future
test.test_future_stmt.test_future_flags
test.test_future_stmt.test_future_multiple_features
test.test_future_stmt.test_future_multiple_imports
test.test_future_stmt.test_future_single_import
test.test_multiprocessing_fork.test_manager
test.test_multiprocessing_fork.test_misc
test.test_multiprocessing_fork.test_processes
test.test_multiprocessing_fork.test_threads
test.test_multiprocessing_forkserver.test_manager
test.test_multiprocessing_forkserver.test_misc
test.test_multiprocessing_forkserver.test_processes
test.test_multiprocessing_forkserver.test_threads
test.test_multiprocessing_spawn.test_manager
test.test_multiprocessing_spawn.test_misc
test.test_multiprocessing_spawn.test_processes
test.test_multiprocessing_spawn.test_threads
test___all__
test__locale
test__opcode
test__osx_support
test__xxinterpchannels
test__xxsubinterpreters
test_abc
test_abstract_numbers
test_argparse
test_array
test_asdl_parser
test_ast
test_asyncgen
test_atexit
test_audit
test_augassign
test_base64
test_baseexception
test_bdb
test_bigaddrspace
test_bigmem
test_binascii
test_binop
test_bisect
test_bool
test_buffer
test_bufio
test_builtin
test_bytes
test_bz2
test_calendar
test_call
test_capi
test_charmapcodec
test_class
test_clinic
test_cmath
test_cmd
test_cmd_line
test_cmd_line_script
test_code
test_code_module
test_codeccallbacks
test_codecencodings_cn
test_codecencodings_hk
test_codecencodings_iso2022
test_codecencodings_jp
test_codecencodings_kr
test_codecencodings_tw
test_codecmaps_cn
test_codecmaps_hk
test_codecmaps_jp
test_codecmaps_kr
test_codecmaps_tw
test_codecs
test_codeop
test_collections
test_colorsys
test_compare
test_compile
test_compileall
test_compiler_assemble
test_compiler_codegen
test_complex
test_configparser
test_contains
test_context
test_contextlib
test_contextlib_async
test_copy
test_copyreg
test_coroutines
test_cppext
test_cprofile
test_csv
test_ctypes
test_curses
test_dataclasses
test_datetime
test_dbm
test_dbm_dumb
test_dbm_gnu
test_dbm_ndbm
test_decimal
test_decorators
test_defaultdict
test_deque
test_descr
test_descrtut
test_devpoll
test_dict
test_dict_version
test_dictcomps
test_dictviews
test_difflib
test_dis
test_doctest
test_doctest2
test_docxmlrpc
test_dtrace
test_dynamic
test_dynamicclassattribute
test_eintr
test_email
test_embed
test_ensurepip
test_enum
test_enumerate
test_eof
test_epoll
test_errno
test_except_star
test_exception_group
test_exception_hierarchy
test_exception_variations
test_exceptions
test_extcall
test_faulthandler
test_fcntl
test_file
test_file_eintr
test_filecmp
test_fileinput
test_fileio
test_fileutils
test_finalization
test_float
test_flufl
test_fnmatch
test_fork1
test_format
test_fractions
test_frame
test_frozen
test_fstring
test_ftplib
test_funcattrs
test_functools
test_gc
test_gdb
test_generated_cases
test_generator_stop
test_generators
test_genericalias
test_genericclass
test_genericpath
test_genexps
test_getopt
test_getpass
test_getpath
test_gettext
test_glob
test_global
test_grammar
test_graphlib
test_grp
test_gzip
test_hash
test_hashlib
test_heapq
test_hmac
test_html
test_htmlparser
test_http_cookiejar
test_http_cookies
test_httplib
test_httpservers
test_idle
test_imaplib
test_import
test_importlib
test_index
test_inspect
test_int
test_int_literal
test_interpreters
test_io
test_ioctl
test_ipaddress
test_isinstance
test_iter
test_iterlen
test_itertools
test_json
test_keyword
test_keywordonlyarg
test_kqueue
test_largefile
test_launcher
test_linecache
test_list
test_listcomps
test_lltrace
test_locale
test_logging
test_long
test_longexp
test_lzma
test_mailbox
test_marshal
test_math
test_math_property
test_memoryio
test_memoryview
test_metaclass
test_mimetypes
test_minidom
test_mmap
test_module
test_modulefinder
test_monitoring
test_msvcrt
test_multibytecodec
test_multiprocessing_main_handling
test_named_expressions
test_netrc
test_ntpath
test_numeric_tower
test_opcache
test_opcodes
test_openpty
test_operator
test_optparse
test_ordered_dict
test_os
test_osx_env
test_pathlib
test_patma
test_pdb
test_peepholer
test_peg_generator
test_pep646_syntax
test_perf_profiler
test_perfmaps
test_pickle
test_picklebuffer
test_pickletools
test_pkg
test_pkgutil
test_platform
test_plistlib
test_poll
test_popen
test_poplib
test_positional_only_arg
test_posix
test_posixpath
test_pow
test_pprint
test_print
test_profile
test_property
test_pstats
test_pty
test_pulldom
test_pwd
test_py_compile
test_pyclbr
test_pydoc
test_pyexpat
test_queue
test_quopri
test_raise
test_random
test_range
test_re
test_readline
test_regrtest
test_repl
test_reprlib
test_resource
test_richcmp
test_rlcompleter
test_robotparser
test_runpy
test_sax
test_sched
test_scope
test_script_helper
test_secrets
test_select
test_selectors
test_set
test_setcomps
test_shelve
test_shlex
test_signal
test_site
test_slice
test_smtplib
test_smtpnet
test_socket
test_socketserver
test_sort
test_source_encoding
test_sqlite3
test_ssl
test_stable_abi_ctypes
test_startfile
test_stat
test_statistics
test_str
test_strftime
test_string
test_string_literals
test_stringprep
test_strptime
test_strtod
test_struct
test_structseq
test_subclassinit
test_subprocess
test_sundry
test_super
EOF
)

stringarray=($tests)

for i in "${stringarray[@]}"; do
./python.exe -m test $i test_support
done

And tests still pass. So, I think that we need several tests, not just two.

[sobolevn]

@shamsherfairfest this is unrelated, please open a new issue with the minimal reproduction. We would be happy to help you in that case :)

[shamsherfairfest]

@shamsherfairfest this is unrelated, please open a new issue with the minimal reproduction. We would be happy to help you in that case :)

ok sure
thanks

[vstinner]

I wrote this simple script (because bisect's output didn't reproduce this problem)

Did you run test.bisect_cmd on test_support, or on the whole Python test suite? What do you mean that it doesn't reproduce this problem? Which behavior do you get when you run ./python.exe -m test test_support --matchfile=bisect (or ./python.exe -m test --matchfile=bisect if you ran it in the whole test suite)?

[sobolevn]

I've reproduced it with only three test files: ./python.exe -m test test_sax test_socket test_support

Now, I need to find exact tests that are related.
It might be related to #110013 since it is also about test_socket

[vstinner]

I've reproduced it with only three test files: ./python.exe -m test test_sax test_socket test_support

Wow, that's very helpful, thanks!

Now, I need to find exact tests that are related.

You may give a new try to bisect_cmd:

./python.exe -m test.bisect_cmd -o bisect test_sax test_socket test_support

And then:

./python.exe -m test --matchfile=bisect test_sax test_socket test_support

It should help to reduce the quantity of code that you have to review, by skipping many tests, and help you to find the minimum set of tests which trigger the bug.

[vstinner]

By the way, it would be interesting to have a "files" mode in bisect_cmd to filter test files, rather than test cases.

[sobolevn]

I found all three offending tests 🎉

test.test_sax.ExpatReaderTest.test_expat_external_dtd_enabled
test.test_socket.GeneralModuleTests.testHostnameRes
test.test_support.TestSupport.test_fd_count