Skip to content

gh-115999: Implement thread-local bytecode and enable specialization for BINARY_OP #123926

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 84 commits into from
Nov 4, 2024
Merged
Changes from 1 commit
Commits
Show all changes
84 commits
Select commit Hold shift + click to select a range
776a1e1
Assign threads indices into bytecode copies
mpage Aug 15, 2024
2b40870
Replace most usage of PyCode_CODE
mpage Aug 27, 2024
344d7ad
Get bytecode copying working
mpage Aug 20, 2024
f203d00
Refactor remove_tools
mpage Aug 30, 2024
82b456a
Refactor remove_line_tools
mpage Aug 30, 2024
b021704
Instrument thread-local bytecode
mpage Sep 1, 2024
aea69c5
Use locks for instrumentation
mpage Sep 3, 2024
552277d
Add ifdef guards for each specialization family
mpage Sep 3, 2024
50a6089
Specialize BINARY_OP
mpage Sep 4, 2024
3f1d941
Limit the amount of memory consumed by bytecode copies
mpage Sep 6, 2024
7d2eb27
Make thread-local bytecode limits user configurable
mpage Sep 7, 2024
d5476b9
Fix a few data races when (de)instrumenting opcodes
mpage Sep 8, 2024
e3b367a
Make branch taken recording thread-safe
mpage Sep 8, 2024
b2375bf
Lock thread-local bytecode when specializing
mpage Sep 9, 2024
2707f8e
Load bytecode on RESUME_CHECK
mpage Sep 9, 2024
3fdcb28
Load tlbc on generator.throw()
mpage Sep 9, 2024
4a55ce5
Use tlbc instead of thread_local_bytecode
mpage Sep 9, 2024
8b3ff60
Use tlbc everywhere
mpage Sep 9, 2024
862afa1
Explicitly manage tlbc state
mpage Sep 9, 2024
0b4d952
Refactor API for fetching tlbc
mpage Sep 9, 2024
7795e99
Add unit tests
mpage Sep 10, 2024
693a4cc
Fix initconfig in default build
mpage Sep 10, 2024
b43531e
Fix instrumentation in default build
mpage Sep 10, 2024
9025f43
Synchronize bytecode modifications between specialization and instrum…
mpage Sep 10, 2024
c44c7d9
Add a high-level comment
mpage Sep 10, 2024
e2a6656
Fix unused variable warning in default build
mpage Sep 10, 2024
e6513d1
Fix test_config in free-threaded builds
mpage Sep 10, 2024
a18396f
Fix formatting
mpage Sep 10, 2024
81fe1a2
Remove comment
mpage Sep 10, 2024
837645e
Fix data race in _PyInstruction_GetLength
mpage Sep 10, 2024
f13e132
Fix tier2 optimizer
mpage Sep 11, 2024
942f628
Use __VA_ARGS__ for macros
mpage Sep 11, 2024
66cb24d
Update vcxproj files to include newly added files
mpage Sep 11, 2024
ad12bd4
Mark unused params
mpage Sep 11, 2024
1bbbbbc
Keep tier2 and the JIT disabled in free-threaded builds
mpage Sep 12, 2024
e63e403
Only allow enabling/disabling tlbc
mpage Sep 13, 2024
8b97771
Update libpython for gdb
mpage Sep 13, 2024
d34adeb
Merge branch 'main' into gh-115999-thread-local-bytecode
mpage Sep 13, 2024
6d4fe73
Handle out of memory errors
mpage Sep 13, 2024
c2d8693
Merge branch 'main' into gh-115999-thread-local-bytecode
mpage Sep 17, 2024
b104782
Fix warnings on windows
mpage Sep 17, 2024
deb5216
Fix another warning
mpage Sep 18, 2024
2f11cc7
Ugh actually fix it
mpage Sep 18, 2024
04f1ac3
Add high-level comment about index pools
mpage Sep 25, 2024
aa330b1
Merge branch 'main' into gh-115999-thread-local-bytecode
mpage Sep 25, 2024
7dfd1ca
Merge branch 'main' into gh-115999-thread-local-bytecode
mpage Sep 26, 2024
7c9da24
Exclude tlbc from refleak counts
mpage Sep 27, 2024
dd144d0
Merge branch 'main' into gh-115999-thread-local-bytecode
mpage Sep 28, 2024
ad180d1
Regen files
mpage Sep 28, 2024
95d2264
Move `get_tlbc_blocks` into the sys module
mpage Sep 30, 2024
b6380de
Merge branch 'main' into gh-115999-thread-local-bytecode
mpage Sep 30, 2024
adb59ef
Merge branch 'main' into gh-115999-thread-local-bytecode
mpage Oct 5, 2024
39c947d
Merge branch 'main' into gh-115999-thread-local-bytecode
mpage Oct 10, 2024
2cc5830
Work around `this_instr` now being const
mpage Oct 11, 2024
96ec126
Make RESUME_CHECK cheaper
mpage Oct 11, 2024
5ecebd9
Pass tstate to _PyCode_GetTLBCFast
mpage Oct 11, 2024
815b2fe
Rename test_tlbc.py to test_thread_local_bytecode.py
mpage Oct 11, 2024
fb90d23
Remove per-family defines for specialization
mpage Oct 11, 2024
4e42414
Replace bytecode pointer with tlbc_index
mpage Oct 13, 2024
814e4ca
Add a test verifying that we clean up tlbc when the code object is de…
mpage Oct 14, 2024
ba3930a
Merge branch 'main' into gh-115999-thread-local-bytecode
mpage Oct 14, 2024
cb8a774
Fix indentation
mpage Oct 14, 2024
0f8a55b
Clarify comment
mpage Oct 14, 2024
70ce0fe
Fix TSAN
mpage Oct 14, 2024
f512353
Add test for cleaning up tlbc in correct place, not old emacs buffer
mpage Oct 14, 2024
4be2b1f
Remove test_tlbc.py
mpage Oct 14, 2024
61c7aa9
Merge branch 'main' into gh-115999-thread-local-bytecode
mpage Oct 17, 2024
ab6222c
Use int32_t instead of Py_ssize_t for tlbc indices
mpage Oct 17, 2024
6bbb220
Use _PyCode_CODE instead of PyFrame_GetBytecode in super_init_without…
mpage Oct 17, 2024
4580e3c
Update comment
mpage Oct 17, 2024
b992f44
Consolidate _PyCode_{Quicken,DisableSpecialization} into _PyCode_Init…
mpage Oct 17, 2024
4c040d3
Merge branch 'main' into gh-115999-thread-local-bytecode
mpage Oct 18, 2024
5b7658c
Fix incorrect types
mpage Oct 18, 2024
bec5bce
Add command-line tests for enabling TLBC
mpage Oct 18, 2024
c9054b7
Update libpython.py for tlbc_index
mpage Oct 18, 2024
1a48ab2
Avoid special casing in _PyEval_GetExecutableCode
mpage Oct 19, 2024
b16ae5f
Merge branch 'main' into gh-115999-thread-local-bytecode
mpage Oct 19, 2024
176b24e
Merge branch 'main' into gh-115999-thread-local-bytecode
mpage Oct 23, 2024
c107495
Clear TLBC when other caches are cleared
mpage Oct 23, 2024
07f9140
Remove _get_tlbc_blocks
mpage Oct 24, 2024
4cbe237
Merge branch 'main' into gh-115999-thread-local-bytecode
mpage Oct 30, 2024
38ff315
Rename _PyCode_InitCounters back to _PyCode_Quicken
mpage Oct 30, 2024
338f7e5
Merge branch 'main' into gh-115999-thread-local-bytecode
mpage Nov 4, 2024
bcd1bb2
Merge branch 'main' into gh-115999-thread-local-bytecode
mpage Nov 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix test_config in free-threaded builds
  • Loading branch information
mpage committed Sep 10, 2024
commit e6513d1a945f63826ace6f21c75e11bdec1a1774
1 change: 1 addition & 0 deletions Lib/test/test_capi/test_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,7 @@ def test_config_get(self):
options.append(("run_presite", str | None, None))
if sysconfig.get_config_var('Py_GIL_DISABLED'):
options.append(("enable_gil", int, None))
options.append(("tlbc_limit", int, None))
if support.MS_WINDOWS:
options.extend((
("legacy_windows_stdio", bool, None),
Expand Down