Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[subinterpreters] crash in _elementtree when parsing in parallel #92123

Closed
po1vo opened this issue May 1, 2022 · 11 comments
Closed

[subinterpreters] crash in _elementtree when parsing in parallel #92123

po1vo opened this issue May 1, 2022 · 11 comments
Labels
topic-subinterpreters type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@po1vo
Copy link

po1vo commented May 1, 2022 •
edited by bedevere-bot
Loading

Crash report

I have Kodi crashing when certain add-ons are enabled. I traced the error to _elementtree and repurposed a script from #90228 to reproduce the bug. See bug.py.txt

Error messages

/opt/python-dbg/bin/python3 bug.py 
Fatal Python error: Segmentation fault

Current thread 0x00007fd85f7fe640 (most recent call first):
  File "/home/plv/tmp/bug.py", line 18 in doIt
  File "/opt/python-dbg/lib/python3.10/threading.py", line 946 in run
  File "/opt/python-dbg/lib/python3.10/threading.py", line 1009 in _bootstrap_inner
  File "/opt/python-dbg/lib/python3.10/threading.py", line 966 in _bootstrap

Thread 0x00007fd8654d2640 (most recent call first):
  File "/home/plv/tmp/bug.py", line 18 in doIt
  File "/opt/python-dbg/lib/python3.10/threading.py", line 946 in run
  File "/opt/python-dbg/lib/python3.10/threading.py", line 1009 in _bootstrap_inner
  File "/opt/python-dbg/lib/python3.10/threading.py", line 966 in _bootstrap

Thread 0x00007fd865f20740 (most recent call first):
  File "/opt/python-dbg/lib/python3.10/threading.py", line 1109 in _wait_for_tstate_lock
  File "/opt/python-dbg/lib/python3.10/threading.py", line 1089 in join
  File "/home/plv/tmp/bug.py", line 25 in func
  File "/home/plv/tmp/bug.py", line 27 in <module>

Extension modules: _testcapi (total: 1)
Segmentation fault (core dumped)


Reading symbols from /opt/python-dbg/bin/python3.10...
[New LWP 3737917]
[New LWP 3737913]
[New LWP 3737914]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
Core was generated by `/opt/python-dbg/bin/python3 bug.py'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007fd865da234c in __pthread_kill_implementation () from /usr/lib/libc.so.6
[Current thread is 1 (Thread 0x7fd85f7fe640 (LWP 3737917))]
(gdb) bt
#0  0x00007fd865da234c in __pthread_kill_implementation () from /usr/lib/libc.so.6
#1  0x00007fd865d554b8 in raise () from /usr/lib/libc.so.6
#2  <signal handler called>
#3  0x00007fd8640c4c8a in expat_parse (self=0x7fd85efb45f0, data=0x7fd84c0c39e0 "<data />", data_len=8, final=0)
    at /home/plv/Documents/projects/aur/python-dbg/src/Python-3.10.2/Modules/_elementtree.c:3835
#4  0x00007fd8640c5690 in _elementtree_XMLParser__parse_whole (self=0x7fd85efb45f0, file=<optimized out>)
    at /home/plv/Documents/projects/aur/python-dbg/src/Python-3.10.2/Modules/_elementtree.c:3994
#5  0x00007fd8660bfbdf in method_vectorcall_O (func=func@entry=0x7fd85eef49b0, args=args@entry=0x7fd8642061e0, nargsf=<optimized out>, kwnames=0x0) at Objects/descrobject.c:460
#6  0x00007fd8660b544b in _PyObject_VectorcallTstate (kwnames=<optimized out>, nargsf=<optimized out>, args=<optimized out>, callable=<optimized out>, tstate=<optimized out>)
    at ./Include/cpython/abstract.h:114
#7  PyObject_Vectorcall (kwnames=<optimized out>, nargsf=<optimized out>, args=<optimized out>, callable=<optimized out>) at ./Include/cpython/abstract.h:123
#8  call_function (tstate=0x7fd84c01cc00, trace_info=<optimized out>, pp_stack=0x7fd85f7fc940, oparg=<optimized out>, kwnames=<optimized out>) at Python/ceval.c:5867
#9  0x00007fd8660ab172 in _PyEval_EvalFrameDefault (tstate=<optimized out>, f=0x7fd864206050, throwflag=<optimized out>) at Python/ceval.c:4198
#10 0x00007fd8660a980b in _PyEval_EvalFrame (throwflag=0, f=0x7fd864206050, tstate=0x7fd84c01cc00) at ./Include/internal/pycore_ceval.h:46
#11 _PyEval_Vector (tstate=<optimized out>, con=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=3, kwnames=<optimized out>) at Python/ceval.c:5065
#12 0x00007fd8660b544b in _PyObject_VectorcallTstate (kwnames=<optimized out>, nargsf=<optimized out>, args=<optimized out>, callable=<optimized out>, tstate=<optimized out>)
    at ./Include/cpython/abstract.h:114
#13 PyObject_Vectorcall (kwnames=<optimized out>, nargsf=<optimized out>, args=<optimized out>, callable=<optimized out>) at ./Include/cpython/abstract.h:123
#14 call_function (tstate=0x7fd84c01cc00, trace_info=<optimized out>, pp_stack=0x7fd85f7fcc10, oparg=<optimized out>, kwnames=<optimized out>) at Python/ceval.c:5867
#15 0x00007fd8660ab172 in _PyEval_EvalFrameDefault (tstate=<optimized out>, f=0x7fd864124240, throwflag=<optimized out>) at Python/ceval.c:4198
#16 0x00007fd8660a980b in _PyEval_EvalFrame (throwflag=0, f=0x7fd864124240, tstate=0x7fd84c01cc00) at ./Include/internal/pycore_ceval.h:46
#17 _PyEval_Vector (tstate=<optimized out>, con=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=1, kwnames=<optimized out>) at Python/ceval.c:5065
#18 0x00007fd8660b544b in _PyObject_VectorcallTstate (kwnames=<optimized out>, nargsf=<optimized out>, args=<optimized out>, callable=<optimized out>, tstate=<optimized out>)
    at ./Include/cpython/abstract.h:114
#19 PyObject_Vectorcall (kwnames=<optimized out>, nargsf=<optimized out>, args=<optimized out>, callable=<optimized out>) at ./Include/cpython/abstract.h:123
#20 call_function (tstate=0x7fd84c01cc00, trace_info=<optimized out>, pp_stack=0x7fd85f7fcee0, oparg=<optimized out>, kwnames=<optimized out>) at Python/ceval.c:5867
#21 0x00007fd8660b12a6 in _PyEval_EvalFrameDefault (tstate=<optimized out>, f=0x7fd864356af0, throwflag=<optimized out>) at Python/ceval.c:4181
#22 0x00007fd8660a980b in _PyEval_EvalFrame (throwflag=0, f=0x7fd864356af0, tstate=0x7fd84c01cc00) at ./Include/internal/pycore_ceval.h:46
#23 _PyEval_Vector (tstate=<optimized out>, con=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=0, kwnames=<optimized out>) at Python/ceval.c:5065
#24 0x00007fd8661269b4 in PyEval_EvalCode (co=0x7fd864240520, globals=0x7fd86422f7d0, locals=<optimized out>) at Python/ceval.c:1134
#25 0x00007fd866140884 in run_eval_code_obj (tstate=0x7fd84c01cc00, co=0x7fd864240520, globals=0x7fd86422f7d0, locals=0x7fd86422f7d0) at Python/pythonrun.c:1291
#26 0x00007fd8661382d6 in run_mod (mod=<optimized out>, filename=<optimized out>, globals=0x7fd86422f7d0, locals=0x7fd86422f7d0, flags=<optimized out>, arena=<optimized out>)
    at Python/pythonrun.c:1312
#27 0x00007fd86612c561 in PyRun_StringFlags (str=<optimized out>, start=257, globals=0x7fd86422f7d0, locals=0x7fd86422f7d0, flags=0x7fd85f7fd220) at Python/pythonrun.c:1183
#28 0x00007fd86612c480 in PyRun_SimpleStringFlags (command=0x7fd8656802b0 "\nimport xml.etree.ElementTree as ETree\nETree.parse(\"data.xml\")\n", flags=flags@entry=0x7fd85f7fd220)
    at Python/pythonrun.c:503
#29 0x00007fd8655e0120 in run_in_subinterp (self=<optimized out>, args=<optimized out>) at /home/plv/Documents/projects/aur/python-dbg/src/Python-3.10.2/Modules/_testcapimodule.c:3639
#30 0x00007fd8660b9438 in cfunction_call (func=0x7fd8656e9c10, args=<optimized out>, kwargs=<optimized out>) at Objects/methodobject.c:552
#31 0x00007fd8660b6468 in _PyObject_MakeTpCall (tstate=0x556284058d10, callable=0x7fd8656e9c10, args=<optimized out>, nargs=<optimized out>, keywords=<optimized out>) at Objects/call.c:215
#32 0x00007fd8660b55d9 in _PyObject_VectorcallTstate (kwnames=<optimized out>, nargsf=<optimized out>, args=<optimized out>, callable=<optimized out>, tstate=<optimized out>)
    at ./Include/cpython/abstract.h:112
#33 PyObject_Vectorcall (kwnames=<optimized out>, nargsf=<optimized out>, args=<optimized out>, callable=<optimized out>) at ./Include/cpython/abstract.h:123
#34 call_function (tstate=0x556284058d10, trace_info=<optimized out>, pp_stack=0x7fd85f7fd3c0, oparg=<optimized out>, kwnames=<optimized out>) at Python/ceval.c:5867
#35 0x00007fd8660b12a6 in _PyEval_EvalFrameDefault (tstate=<optimized out>, f=0x7fd865563b60, throwflag=<optimized out>) at Python/ceval.c:4181
#36 0x00007fd8660a980b in _PyEval_EvalFrame (throwflag=0, f=0x7fd865563b60, tstate=0x556284058d10) at ./Include/internal/pycore_ceval.h:46
#37 _PyEval_Vector (tstate=<optimized out>, con=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=0, kwnames=<optimized out>) at Python/ceval.c:5065
#38 0x00007fd8660ae50a in PyObject_Call (kwargs=0x7fd864c33650, args=0x7fd865700250, callable=0x7fd8657d4940) at Objects/call.c:317
--Type <RET> for more, q to quit, c to continue without paging--c
#39 do_call_core (kwdict=0x7fd864c33650, callargs=0x7fd865700250, func=0x7fd8657d4940, trace_info=0x7fd85f7fd660, tstate=<optimized out>) at Python/ceval.c:5919
#40 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=0x7fd86552abd0, throwflag=<optimized out>) at Python/ceval.c:4277
#41 0x00007fd8660a980b in _PyEval_EvalFrame (throwflag=0, f=0x7fd86552abd0, tstate=0x556284058d10) at ./Include/internal/pycore_ceval.h:46
#42 _PyEval_Vector (tstate=<optimized out>, con=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=1, kwnames=<optimized out>) at Python/ceval.c:5065
#43 0x00007fd8660b544b in _PyObject_VectorcallTstate (kwnames=<optimized out>, nargsf=<optimized out>, args=<optimized out>, callable=<optimized out>, tstate=<optimized out>) at ./Include/cpython/abstract.h:114
#44 PyObject_Vectorcall (kwnames=<optimized out>, nargsf=<optimized out>, args=<optimized out>, callable=<optimized out>) at ./Include/cpython/abstract.h:123
#45 call_function (tstate=0x556284058d10, trace_info=<optimized out>, pp_stack=0x7fd85f7fd900, oparg=<optimized out>, kwnames=<optimized out>) at Python/ceval.c:5867
#46 0x00007fd8660ab172 in _PyEval_EvalFrameDefault (tstate=<optimized out>, f=0x7fd84c000ba0, throwflag=<optimized out>) at Python/ceval.c:4198
#47 0x00007fd8660a980b in _PyEval_EvalFrame (throwflag=0, f=0x7fd84c000ba0, tstate=0x556284058d10) at ./Include/internal/pycore_ceval.h:46
#48 _PyEval_Vector (tstate=<optimized out>, con=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=1, kwnames=<optimized out>) at Python/ceval.c:5065
#49 0x00007fd8660b544b in _PyObject_VectorcallTstate (kwnames=<optimized out>, nargsf=<optimized out>, args=<optimized out>, callable=<optimized out>, tstate=<optimized out>) at ./Include/cpython/abstract.h:114
#50 PyObject_Vectorcall (kwnames=<optimized out>, nargsf=<optimized out>, args=<optimized out>, callable=<optimized out>) at ./Include/cpython/abstract.h:123
#51 call_function (tstate=0x556284058d10, trace_info=<optimized out>, pp_stack=0x7fd85f7fdbd0, oparg=<optimized out>, kwnames=<optimized out>) at Python/ceval.c:5867
#52 0x00007fd8660ab172 in _PyEval_EvalFrameDefault (tstate=<optimized out>, f=0x7fd86552aa00, throwflag=<optimized out>) at Python/ceval.c:4198
#53 0x00007fd8660a980b in _PyEval_EvalFrame (throwflag=0, f=0x7fd86552aa00, tstate=0x556284058d10) at ./Include/internal/pycore_ceval.h:46
#54 _PyEval_Vector (tstate=<optimized out>, con=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=1, kwnames=<optimized out>) at Python/ceval.c:5065
#55 0x00007fd8660c0e2c in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=1, args=0x7fd85f7fddb8, callable=0x7fd86555f540, tstate=0x556284058d10) at ./Include/cpython/abstract.h:114
#56 method_vectorcall (method=<optimized out>, args=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at Objects/classobject.c:61
#57 0x00007fd86618ac9a in thread_run (boot_raw=0x7fd864bf5ee0) at ./Modules/_threadmodule.c:1090
#58 0x00007fd866165f18 in pythread_wrapper (arg=<optimized out>) at Python/thread_pthread.h:248
#59 0x00007fd865da05c2 in start_thread () from /usr/lib/libc.so.6
#60 0x00007fd865e25584 in clone () from /usr/lib/libc.so.6

Your environment

  • CPython versions tested on: python --with-pydebug 3.10.2
  • Operating system and architecture: Linux 5.17.1-zen1-1-zen #1 ZEN SMP PREEMPT Mon, 28 Mar 2022 21:56:46 +0000 x86_64 GNU/Linux

Linked PRs
@po1vo po1vo added the type-crash A hard crash of the interpreter, possibly with a core dump label May 1, 2022
@JelleZijlstra
Copy link
Member

#90619 is another Kodi crash, but the stack trace looks different.

@erlend-aasland
Copy link
Contributor

FTR, I am unable to reproduce on macOS (3.9 through 3.11), but I can reproduce on Debian 3.10 (though not on 3.11).

@po1vo
Copy link
Author

po1vo commented May 1, 2022

Also interrupting the script with Ctrl+C sometimes produces a crash:

Traceback (most recent call last):
  File "/home/plv/tmp/bug.py", line 27, in <module>
    func()
  File "/home/plv/tmp/bug.py", line 25, in func
    t.join()
  File "/opt/python-dbg/lib/python3.10/threading.py", line 1089, in join
    self._wait_for_tstate_lock()
  File "/opt/python-dbg/lib/python3.10/threading.py", line 1109, in _wait_for_tstate_lock
    if lock.acquire(block, timeout):
KeyboardInterrupt
Fatal Python error: PyInterpreterState_Delete: remaining subinterpreters
Python runtime state: finalizing (tstate=0x0000557789f4f680)

Aborted (core dumped)

(gdb) bt
#0  0x00007fd89ddd234c in __pthread_kill_implementation () from /usr/lib/libc.so.6
#1  0x00007fd89dd854b8 in raise () from /usr/lib/libc.so.6
#2  0x00007fd89dd6f534 in abort () from /usr/lib/libc.so.6
#3  0x00007fd89e08a5b0 in fatal_error_exit (status=-1) at Python/pylifecycle.c:2553
#4  fatal_error (fd=<optimized out>, header=header@entry=1, prefix=prefix@entry=0x7fd89e28bb70 <__func__.35.lto_priv.1> "PyInterpreterState_Delete", 
    msg=msg@entry=0x7fd89e20d12f "remaining subinterpreters", status=status@entry=-1) at Python/pylifecycle.c:2664
#5  0x00007fd89e08a963 in _Py_FatalErrorFunc (func=0x7fd89e28bb70 <__func__.35.lto_priv.1> "PyInterpreterState_Delete", msg=0x7fd89e20d12f "remaining subinterpreters")
    at Python/pylifecycle.c:2750
#6  0x00007fd89e02cb3a in PyInterpreterState_Delete (interp=<optimized out>) at Python/pystate.c:407
#7  0x00007fd89e195e97 in Py_FinalizeEx () at Python/pylifecycle.c:1844
#8  0x00007fd89e180d2a in Py_RunMain () at Modules/main.c:668
#9  0x00007fd89e14477d in Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at Modules/main.c:720
#10 0x00007fd89dd70310 in __libc_start_call_main () from /usr/lib/libc.so.6
#11 0x00007fd89dd703c1 in __libc_start_main_impl () from /usr/lib/libc.so.6
#12 0x0000557789e4e045 in _start ()

@fuzzard fuzzard mentioned this issue Jan 2, 2023
Closed
7 tasks
@fuzzard
Copy link

fuzzard commented Jan 2, 2023 •
edited
Loading

FTR, I am unable to reproduce on macOS (3.9 through 3.11), but I can reproduce on Debian 3.10 (though not on 3.11).

Just regarding the macOS reproducibility, it appears to be flakey, but if i increase the NTHREAD to 20, it segfaults reasonably regularly (~1 in 5). I was also able to repro with NTHREAD=8, but was less consistent (~1 in 10), whilst i was unable to repro using NTHREAD=4

My repro is using Python 3.11.1 on arm64 macOS. Built from source, no patches

brent@Brents-MacBook-Pro-2 ~ % sw_vers
ProductName:	macOS
ProductVersion:	12.6.2
BuildVersion:	21G320
brent@Brents-MacBook-Pro-2 ~ % /Users/Shared/xbmc-depends/arm-darwin21.6.0-native/bin/python3 --version
Python 3.11.1
brent@Brents-MacBook-Pro-2 ~ % /Users/Shared/xbmc-depends/arm-darwin21.6.0-native/bin/python3 ~/Downloads/bug.py
Fatal Python error: Segmentation fault

Thread 0x0000000282013000 (most recent call first):
  File "/Users/brent/Downloads/bug.py", line 18 in doIt
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 975 in run
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x0000000281007000 (most recent call first):
  File "/Users/brent/Downloads/bug.py", line 18 in doIt
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 975 in run
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x000000017f49f000 (most recent call first):
  File "/Users/brent/Downloads/bug.py", line 18 in doIt
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 975 in run
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x000000017e493000 (most recent call first):
  File "/Users/brent/Downloads/bug.py", line 18 in doIt
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 975 in run
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x000000017d487000 (most recent call first):
  File "/Users/brent/Downloads/bug.py", line 18 in doIt
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 975 in run
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x000000017c47b000 (most recent call first):
  File "/Users/brent/Downloads/bug.py", line 18 in doIt
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 975 in run
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x000000017b46f000 (most recent call first):
  File "/Users/brent/Downloads/bug.py", line 18 in doIt
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 975 in run
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x000000017a463000 (most recent call first):
  File "/Users/brent/Downloads/bug.py", line 18 in doIt
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 975 in run
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x0000000179457000 (most recent call first):
  File "/Users/brent/Downloads/bug.py", line 18 in doIt
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 975 in run
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x000000017844b000 (most recent call first):
  File "/Users/brent/Downloads/bug.py", line 18 in doIt
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 975 in run
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x000000017743f000 (most recent call first):
  File "/Users/brent/Downloads/bug.py", line 18 in doIt
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 975 in run
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x0000000176433000 (most recent call first):
  File "/Users/brent/Downloads/bug.py", line 18 in doIt
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 975 in run
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x0000000175427000 (most recent call first):
  File "/Users/brent/Downloads/bug.py", line 18 in doIt
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 975 in run
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 995 in _bootstrap

Current thread 0x000000017441b000 (most recent call first):
  File "/Users/brent/Downloads/bug.py", line 18 in doIt
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 975 in run
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x000000017340f000 (most recent call first):
  File "/Users/brent/Downloads/bug.py", line 18 in doIt
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 975 in run
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x0000000172403000 (most recent call first):
  File "/Users/brent/Downloads/bug.py", line 18 in doIt
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 975 in run
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x00000001713f7000 (most recent call first):
  File "/Users/brent/Downloads/bug.py", line 18 in doIt
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 975 in run
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x00000001703eb000 (most recent call first):
  File "/Users/brent/Downloads/bug.py", line 18 in doIt
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 975 in run
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x000000016f3df000 (most recent call first):
  File "/Users/brent/Downloads/bug.py", line 18 in doIt
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 975 in run
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x0000000103194580 (most recent call first):
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 1132 in _wait_for_tstate_lock
  File "/Users/Shared/xbmc-depends/arm-darwin21.6.0-native/lib/python3.11/threading.py", line 1112 in join
  File "/Users/brent/Downloads/bug.py", line 25 in func
  File "/Users/brent/Downloads/bug.py", line 27 in <module>

Extension modules: _testcapi (total: 1)
zsh: segmentation fault  /Users/Shared/xbmc-depends/arm-darwin21.6.0-native/bin/python3

Ive attached 2 x logs from crashes. Not sure if they are of any use. If there is something else i can provide, let me know and i'll try and wrangle it up.

python3.11-2023-01-02-152005.ips.txt
python3.11-2023-01-02-152001.ips.txt

@fuzzard
Copy link

fuzzard commented Jan 2, 2023

Heres a more complete crash generated from macos universal 3.11.1 installer.
Note, NTHREAD=20 was used

py3.11.1_92123_crash.txt

@jurialmunkey jurialmunkey mentioned this issue Jan 2, 2023
Closed
4 tasks
@erlend-aasland erlend-aasland mentioned this issue Jan 18, 2023
Merged
erlend-aasland added a commit that referenced this issue Jan 20, 2023
@erlend-aasland
gh-92123: Convert _elementtree types to heap types (#99221)
3847a6c
This was referenced Jan 20, 2023
Merged
Merged
Merged
erlend-aasland added a commit that referenced this issue Jan 21, 2023
@erlend-aasland
GH-92123: Move _elementtree heap types to module state (#101187)
13566a3
erlend-aasland added a commit that referenced this issue Jan 23, 2023
@erlend-aasland
GH-92123: Store _elementtree state in type contexts (#101190)
984387f 
- add state pointer to TreeBuilderObject
- add state pointer to XMLParserObject
erlend-aasland added a commit that referenced this issue Jan 24, 2023
@erlend-aasland
GH-92123: Pass _elementtree state as parameter (#101189)
b2ac396
@bedevere-bot bedevere-bot mentioned this issue Jan 24, 2023
Merged
erlend-aasland added a commit to erlend-aasland/cpython that referenced this issue Jan 24, 2023
@erlend-aasland
pythongh-92123: use PyModule_GetState in _elementtree
18109aa
erlend-aasland added a commit that referenced this issue Jan 24, 2023
@erlend-aasland
gh-92123: Adapt _elementtree to multi-phase init (PEP 489) (#101285)
fee7a99
@erlend-aasland
Copy link
Contributor

@fuzzard, @po1vo: are you able to reproduce the crashes on latest main? (commit fee7a99)

@fuzzard
Copy link

fuzzard commented Jan 25, 2023

I'll give it a run this weekend

@po1vo
Copy link
Author

po1vo commented Jan 25, 2023

@fuzzard, @po1vo: are you able to reproduce the crashes on latest main? (commit fee7a99)

I couldn't reproduce.
Thank you!

@fuzzard
Copy link

fuzzard commented Jan 28, 2023

Looks to be good from my testing. Cant get it to crash with the repro anymore using commit fee7a99

@erlend-aasland
Copy link
Contributor

Thanks, both of you!

@kumaraditya303 IMO, we can close this now, or do you want to fix the pyexpat C API first?

@kumaraditya303
Copy link
Contributor

IMO, we can close this now, or do you want to fix the pyexpat C API first?

Yeah, lets close this now, will handle rest in specific issue.

@erlend-aasland erlend-aasland mentioned this issue Mar 28, 2023
Open
25 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
topic-subinterpreters type-crash A hard crash of the interpreter, possibly with a core dump
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@JelleZijlstra @fuzzard @po1vo @erlend-aasland @kumaraditya303 @AlexWaygood