-
-
Notifications
You must be signed in to change notification settings - Fork 30.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Using internal tokenize module's TokenizerIter in multiple threads crashes #120317
Labels
3.13
bugs and security fixes
3.14
new features, bugs and security fixes
topic-free-threading
type-crash
A hard crash of the interpreter, possibly with a core dump
Comments
lysnikolaou
added
type-crash
A hard crash of the interpreter, possibly with a core dump
3.13
bugs and security fixes
topic-free-threading
3.14
new features, bugs and security fixes
labels
Jun 10, 2024
lysnikolaou
added a commit
to lysnikolaou/cpython
that referenced
this issue
Jun 10, 2024
lysnikolaou
added a commit
to lysnikolaou/cpython
that referenced
this issue
Jun 10, 2024
lysnikolaou
added a commit
to lysnikolaou/cpython
that referenced
this issue
Jun 10, 2024
lysnikolaou
added a commit
that referenced
this issue
Jul 16, 2024
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
miss-islington
pushed a commit
to miss-islington/cpython
that referenced
this issue
Jul 16, 2024
…honGH-120318) (cherry picked from commit 8549559) Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com> Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
lysnikolaou
added a commit
that referenced
this issue
Jul 16, 2024
estyxx
pushed a commit
to estyxx/cpython
that referenced
this issue
Jul 17, 2024
…hon#120318) Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
3.13
bugs and security fixes
3.14
new features, bugs and security fixes
topic-free-threading
type-crash
A hard crash of the interpreter, possibly with a core dump
Crash report
What happened?
Because the tokenizer is not thread-safe, using the same
TokenizerIter
in multiple threads under the free-threaded build leads to all kinds of unpredicted behavior. It sometimes succeeds, sometimes throws aSyntaxError
when there's none and sometimes crashes with the following.Example error backtrace
A minimal reproducer is the following:
CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS
Output from running 'python -VV' on the command line:
Python 3.14.0a0 experimental free-threading build (heads/main:c3b6dbff2c8, Jun 10 2024, 14:33:07) [Clang 15.0.0 (clang-1500.3.9.4)]
Linked PRs
The text was updated successfully, but these errors were encountered: