bpo-45506: Re-enable test_embed.

[ericsnowcurrently]

In gh-29063 I ended up disabling test_embed on non-Windows by accident. This gets it running again.

https://bugs.python.org/issue45506

[ericsnowcurrently]

Hmm, GH-29274 broke ASAN for PyLongObject but this wasn't caught because test_embed was skipped. Guess I'll be fixing that too. 😞

asan report

$ Programs/_testembed test_run_main_loop
Py_RunMain(): sys.argv=['-c', 'arg2']
=================================================================
==25816==ERROR: AddressSanitizer: heap-use-after-free on address 0x6340000017e0 at pc 0x5655423764d5 bp 0x7ffda59989b0 sp 0x7ffda59989a0
READ of size 8 at 0x6340000017e0 thread T0
    #0 0x5655423764d4 in PyLong_AsSsize_t Objects/longobject.c:553
    #1 0x5655423e56d5 in get_type_attr_as_size Objects/structseq.c:42
    #2 0x5655423e56d5 in PyStructSequence_New Objects/structseq.c:58
    #3 0x565542879fa6 in PyFloat_GetInfo Objects/floatobject.c:97
    #4 0x5655425f6555 in _PySys_InitCore Python/sysmodule.c:2782
    #5 0x5655425f6555 in _PySys_Create Python/sysmodule.c:3074
    #6 0x565542599271 in pycore_interp_init Python/pylifecycle.c:823
    #7 0x56554259ff5d in pyinit_config Python/pylifecycle.c:865
    #8 0x56554259ff5d in pyinit_core Python/pylifecycle.c:1028
    #9 0x5655425a111e in Py_InitializeFromConfig Python/pylifecycle.c:1218
    #10 0x565542316f69 in init_from_config_clear Programs/_testembed.c:398
    #11 0x565542316f69 in test_run_main Programs/_testembed.c:1672
    #12 0x565542317122 in test_run_main_loop Programs/_testembed.c:1683
    #13 0x7f8cd539eb96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)
    #14 0x565542312579 in _start (/home/esnow/projects/work/cpython-perf/cpython/Programs/_testembed+0x19a579)

0x6340000017e0 is located 4064 bytes inside of 119848-byte region [0x634000000800,0x63400001dc28)
freed by thread T0 here:
    #0 0x7f8cd62107a8 in __interceptor_free (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xde7a8)
    #1 0x56554259ba6b in finalize_interp_delete Python/pylifecycle.c:1699
    #2 0x56554259ba6b in Py_FinalizeEx Python/pylifecycle.c:1859
    #3 0x56554262f356 in Py_RunMain Modules/main.c:669
    #4 0x565542316fa3 in test_run_main Programs/_testembed.c:1674
    #5 0x565542317122 in test_run_main_loop Programs/_testembed.c:1683
    #6 0x7f8cd539eb96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)

previously allocated by thread T0 here:
    #0 0x7f8cd6210d28 in __interceptor_calloc (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xded28)
    #1 0x5655425a6b62 in PyInterpreterState_New Python/pystate.c:214
    #2 0x56554259fcf4 in pycore_create_interpreter Python/pylifecycle.c:630
    #3 0x56554259fcf4 in pyinit_config Python/pylifecycle.c:859
    #4 0x56554259fcf4 in pyinit_core Python/pylifecycle.c:1028
    #5 0x5655425a111e in Py_InitializeFromConfig Python/pylifecycle.c:1218
    #6 0x565542316f69 in init_from_config_clear Programs/_testembed.c:398
    #7 0x565542316f69 in test_run_main Programs/_testembed.c:1672
    #8 0x565542317122 in test_run_main_loop Programs/_testembed.c:1683
    #9 0x7f8cd539eb96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)

SUMMARY: AddressSanitizer: heap-use-after-free Objects/longobject.c:553 in PyLong_AsSsize_t
Shadow bytes around the buggy address:
  0x0c687fff82a0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c687fff82b0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c687fff82c0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c687fff82d0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c687fff82e0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
=>0x0c687fff82f0: fd fd fd fd fd fd fd fd fd fd fd fd[fd]fd fd fd
  0x0c687fff8300: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c687fff8310: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c687fff8320: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c687fff8330: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c687fff8340: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==25816==ABORTING

to duplicate:

./configure --with-address-sanitizer --without-pymalloc
make -j8
Programs/_testembed test_run_main_loop

@markshannon

[ericsnowcurrently]

FYI, "ASAN_OPTIONS=abort_on_error=1" has been useful to break in GDB.

[ericsnowcurrently]

The problem is that _PyStructSequence_InitType() sets a number of small ints on the type's dict. If it's a static type (and never cleared) then it will still point to the small int objects from the original interpreter no matter how many times you finalize and re-init the runtime. Hence the ASan failures.

[markshannon]

See https://bugs.python.org/issue45691