Summary
YAPF 0.43.0 can hang while importing on Windows when its grammar cache directory exists and appears writable to os.access(), but creation of the temporary cache file is denied (for example, by a restricted-token sandbox).
Even yapf --version hangs before argument processing. The process consumes CPU in Python's temporary-file name retry loop.
This is distinct from #1263: closing/flushing the temporary file before os.rename() fixes the open-file rename/cache-persistence problem, but not a denial during the initial NamedTemporaryFile creation.
Environment
- Windows
- YAPF 0.43.0
- Python 3.11.8 from Chromium depot_tools
- Restricted-token filesystem sandbox
YAPF officially supports Python versions where Windows tempfile.TMP_MAX can be 2,147,483,647. CPython 3.13 bounds this retry to 20 attempts, but Python 3.11 and 3.12 retain the effectively unbounded behavior.
Reproduction
Run YAPF in a Windows restricted-token sandbox where the platformdirs cache path under %LOCALAPPDATA%:
- Exists.
- Is reported writable by
os.access(path, os.W_OK).
- Rejects file creation with
PermissionError for the sandboxed process.
Then run:
The same behavior occurs when running YAPF as a directory. No source file is required.
A faulthandler dump from the stuck process ends in:
File ...\Lib\tempfile.py, line 263, in _mkstemp_inner
File ...\Lib\tempfile.py, line 580, in NamedTemporaryFile
File ...\pgen2\grammar.py, line 107, in dump
File ...\pgen2\driver.py, line 242, in load_grammar
File ...\_ylib2to3\pygram.py, line 29, in <module>
Expected behavior
Grammar caching is optional, so a cache write failure should be caught by load_grammar() and YAPF should continue uncached.
Actual behavior
grammar.Grammar.dump() calls tempfile.NamedTemporaryFile(). On Python 3.11/3.12 for Windows, tempfile._mkstemp_inner() treats PermissionError as a possible name collision when the directory exists and os.access() reports it writable. It retries up to Windows' TMP_MAX (2,147,483,647), so control effectively never returns to YAPF's except OSError fallback.
There is a second contributing issue in 0.43.0: Grammar.dump() calls os.rename() while the NamedTemporaryFile is still open. On Windows that rename fails, the final cache is never created, and randomly named pickle files are left behind. Consequently YAPF attempts cache generation again on every new process. This appears to be the issue partially addressed by #1263.
Verified workaround
Pointing platformdirs' Windows override at an existing regular file makes cache-directory creation fail immediately, allowing YAPF's existing OSError fallback to continue uncached:
$env:WIN_PD_OVERRIDE_LOCAL_APPDATA = (Resolve-Path .\LICENSE).Path
python -m yapf --version
Suggested fix
Please make denied grammar-cache writes reliably fail once and continue uncached, for example by one or more of:
Related cache changes/issues: #1243, #1263.
Summary
YAPF 0.43.0 can hang while importing on Windows when its grammar cache directory exists and appears writable to
os.access(), but creation of the temporary cache file is denied (for example, by a restricted-token sandbox).Even
yapf --versionhangs before argument processing. The process consumes CPU in Python's temporary-file name retry loop.This is distinct from #1263: closing/flushing the temporary file before
os.rename()fixes the open-file rename/cache-persistence problem, but not a denial during the initialNamedTemporaryFilecreation.Environment
YAPF officially supports Python versions where Windows
tempfile.TMP_MAXcan be 2,147,483,647. CPython 3.13 bounds this retry to 20 attempts, but Python 3.11 and 3.12 retain the effectively unbounded behavior.Reproduction
Run YAPF in a Windows restricted-token sandbox where the platformdirs cache path under
%LOCALAPPDATA%:os.access(path, os.W_OK).PermissionErrorfor the sandboxed process.Then run:
The same behavior occurs when running YAPF as a directory. No source file is required.
A faulthandler dump from the stuck process ends in:
Expected behavior
Grammar caching is optional, so a cache write failure should be caught by
load_grammar()and YAPF should continue uncached.Actual behavior
grammar.Grammar.dump()callstempfile.NamedTemporaryFile(). On Python 3.11/3.12 for Windows,tempfile._mkstemp_inner()treatsPermissionErroras a possible name collision when the directory exists andos.access()reports it writable. It retries up to Windows'TMP_MAX(2,147,483,647), so control effectively never returns to YAPF'sexcept OSErrorfallback.There is a second contributing issue in 0.43.0:
Grammar.dump()callsos.rename()while theNamedTemporaryFileis still open. On Windows that rename fails, the final cache is never created, and randomly named pickle files are left behind. Consequently YAPF attempts cache generation again on every new process. This appears to be the issue partially addressed by #1263.Verified workaround
Pointing platformdirs' Windows override at an existing regular file makes cache-directory creation fail immediately, allowing YAPF's existing
OSErrorfallback to continue uncached:Suggested fix
Please make denied grammar-cache writes reliably fail once and continue uncached, for example by one or more of:
NamedTemporaryFile's internal Windows retry loop for this optional cache write;flushandclosebeforeos.renameto fixEOFError: Ran out of input#1263 so the temporary file is closed before replacement.Related cache changes/issues: #1243, #1263.