Repository navigation
Found Heap-use-after-free errors and SEGV in Python #103824
Description
Your environment
- CPython versions tested on: 3.12.0 alpha 7
- Operating system and architecture: ubuntu20.04.1,x86_64
- Compiler flags: clang with ASAN and UBSAN instrument
Bug description
The AddressSanitizer (ASAN) tool has detected multiple heap-use-after-free errors and a segmentation fault (SEGV) in the Python interpreter. The heap-use-after-free errors occurred in the ascii_decode and unicode_decode_utf8 functions in the unicodeobject.c file, and the SEGV occurred in the tok_backup function in the tokenizer.c file. Additionally, a memory leak was detected in the pystate.c file.
Steps to reproduce
- Compile Python with ASAN enabled:
./configure && make - Run Python with ASAN enabled:
./python < poc_file - The heap-use-after-free errors and SEGV should be detected and logged by ASAN.
Expected behavior
No heap-use-after-free errors or SEGV should occur.
Actual behavior
ASAN detected multiple heap-use-after-free errors and a SEGV, as well as a memory leak.
Relevant logs and/or screenshots
The ASAN summary output is as follows:
AddressSanitizer: heap-use-after-free /src/cpython/Objects/unicodeobject.c:4474:28 in ascii_decode
AddressSanitizer: heap-use-after-free /src/cpython/Objects/unicodeobject.c:4506:28 in ascii_decode
AddressSanitizer: heap-use-after-free /src/cpython/Objects/unicodeobject.c:4483:32 in ascii_decode
AddressSanitizer: SEGV /src/cpython/Parser/tokenizer.c:1234:33 in tok_backup
AddressSanitizer: heap-use-after-free /src/cpython/Objects/unicodeobject.c:4526:37 in unicode_decode_utf8
AddressSanitizer: 3824 byte(s) leaked in 4 allocation(s).
AddressSanitizer: heap-use-after-free /src/cpython/Python/pystate.c:229:23 in bind_tstate
The full ASAN log can be found in the asan.log file.
Activity
sunmy2019 commentedon Apr 25, 2023
CC: @lysnikolaou
sobolevn commentedon Apr 25, 2023
Sorry, I might be missing something, but your archive does not have
poc_file.There are several
bugNfiles. Do you mean to actually run them?JohenanLi commentedon Apr 25, 2023
yes，run them. python < bugN.
chgnrdv commentedon Apr 29, 2023
I can reproduce use-after-free errors detected in
unicodeobject.cwithbug_2,bug_6,bug_7andbug_9files. I don't get a segfault intok_backupfunc withbug_7on my machine, bug I get the same use-after-free.I'll submit a PR with possible fix for these errors, if nobody minds.
Unfortunately, I don't get use-after-free error with
bug_4on my machine. I can't reproduce use-after-free inbind_tstatewithbug_15as well.JohenanLi commentedon May 1, 2023
Thanks a lot.
gh-103824: fix use-after-free error in Parser/tokenizer.c (#103993)
Merge branch 'main' into inlinecomp2
lysnikolaou commentedon May 3, 2023
Resolved in #103993.
ajakk commentedon Jun 9, 2023
Did this ever affect other release lines?
schribl commentedon Jul 10, 2023
We would also be interested in the answer to this question, if possible. Thanks a lot!
lysnikolaou commentedon Jul 13, 2023
I just did another check on 3.11 and everything appears to be okay. As far as I can see, this only ever affected 3.12, since it was introduced in the implementation of PEP 701.