Found Heap-use-after-free errors and SEGV in Python

[JohenanLi]

Your environment

• CPython versions tested on: 3.12.0 alpha 7
• Operating system and architecture: ubuntu20.04.1,x86_64
• Compiler flags: clang with ASAN and UBSAN instrument

Bug description

The AddressSanitizer (ASAN) tool has detected multiple heap-use-after-free errors and a segmentation fault (SEGV) in the Python interpreter. The heap-use-after-free errors occurred in the ascii_decode and unicode_decode_utf8 functions in the unicodeobject.c file, and the SEGV occurred in the tok_backup function in the tokenizer.c file. Additionally, a memory leak was detected in the pystate.c file.

Steps to reproduce

1. Compile Python with ASAN enabled: ./configure && make
2. Run Python with ASAN enabled: ./python < poc_file
3. The heap-use-after-free errors and SEGV should be detected and logged by ASAN.

Expected behavior

No heap-use-after-free errors or SEGV should occur.

Actual behavior

ASAN detected multiple heap-use-after-free errors and a SEGV, as well as a memory leak.

Relevant logs and/or screenshots

The ASAN summary output is as follows:

AddressSanitizer: heap-use-after-free /src/cpython/Objects/unicodeobject.c:4474:28 in ascii_decode
AddressSanitizer: heap-use-after-free /src/cpython/Objects/unicodeobject.c:4506:28 in ascii_decode
AddressSanitizer: heap-use-after-free /src/cpython/Objects/unicodeobject.c:4483:32 in ascii_decode
AddressSanitizer: SEGV /src/cpython/Parser/tokenizer.c:1234:33 in tok_backup
AddressSanitizer: heap-use-after-free /src/cpython/Objects/unicodeobject.c:4526:37 in unicode_decode_utf8
AddressSanitizer: 3824 byte(s) leaked in 4 allocation(s).
AddressSanitizer: heap-use-after-free /src/cpython/Python/pystate.c:229:23 in bind_tstate
The full ASAN log can be found in the asan.log file.

asan.log
python_bug_poc.zip

Linked PRs

• gh-103824: fix use-after-free error in Parser/tokenizer.c #103993

[sunmy2019]

CC: @lysnikolaou

[sobolevn]

Sorry, I might be missing something, but your archive does not have poc_file.

There are several bugN files. Do you mean to actually run them?

[JohenanLi]

Sorry, I might be missing something, but your archive does not have poc_file.

There are several bugN files. Do you mean to actually run them?

yes，run them. python < bugN.

[chgnrdv]

I can reproduce use-after-free errors detected in unicodeobject.c with bug_2, bug_6, bug_7 and bug_9 files. I don't get a segfault in tok_backup func with bug_7 on my machine, bug I get the same use-after-free.
I'll submit a PR with possible fix for these errors, if nobody minds.

Unfortunately, I don't get use-after-free error with bug_4 on my machine. I can't reproduce use-after-free in bind_tstate with bug_15 as well.

[JohenanLi]

I can reproduce use-after-free errors detected in unicodeobject.c with bug_2, bug_6, bug_7 and bug_9 files. I don't get a segfault in tok_backup func with bug_7 on my machine, bug I get the same use-after-free. I'll submit a PR with possible fix for these errors, if nobody minds.

Unfortunately, I don't get use-after-free error with bug_4 on my machine. I can't reproduce use-after-free in bind_tstate with bug_15 as well.

Thanks a lot.

[lysnikolaou]

Resolved in #103993.