Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-101659: Isolate "obmalloc" State to Each Interpreter #101660

Merged
merged 58 commits into from
Apr 24, 2023
Merged
Changes from 1 commit
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
07a09d4
Pass PyInterpreterState to pymalloc_*().
ericsnowcurrently Oct 6, 2022
ca75048
Move the object arenas to the interpreter state.
ericsnowcurrently Oct 7, 2022
4ee199b
Drop an errant #define.
ericsnowcurrently Feb 7, 2023
2768fa4
Leave dump_debug_stats in the global state.
ericsnowcurrently Feb 7, 2023
bf9425f
Dynamically initialize obmalloc for subinterpreters.
ericsnowcurrently Feb 9, 2023
d5da34b
Merge branch 'main' into per-interpreter-alloc
ericsnowcurrently Mar 9, 2023
6c3111c
Pass around struct _obmalloc_state* instead of PyInterpeterState*.
ericsnowcurrently Mar 8, 2023
4dc087d
Add _PyInterpreterConfig.use_main_obmalloc.
ericsnowcurrently Mar 9, 2023
1ae33a0
Add a comment about why per-interpreter obmalloc requires multi-phase…
ericsnowcurrently Mar 9, 2023
5b54d63
Add a TODO comment.
ericsnowcurrently Mar 9, 2023
9f4f8f3
Optionally use the main interpreter's obmalloc state.
ericsnowcurrently Mar 9, 2023
aa10204
Pass use_main_obmalloc to run_in_subinterp() in test_import.
ericsnowcurrently Mar 9, 2023
69d9a2d
_Py_GetAllocatedBlocks() -> _Py_GetGlobalAllocatedBlocks().
ericsnowcurrently Mar 10, 2023
25378f8
Errors from _Py_NewInterpreterFromConfig() are no longer fatal.
ericsnowcurrently Mar 10, 2023
1c5b109
Chain the exceptions.
ericsnowcurrently Mar 13, 2023
f36426b
Swap out the failed tstate.
ericsnowcurrently Mar 10, 2023
54b9f09
Remaining static builtin types must be fixed.
ericsnowcurrently Mar 13, 2023
2358a42
Add PyInterpreterState.sysdict_copy.
ericsnowcurrently Mar 13, 2023
b6502e1
Set m_copy to None for sys and builtins.
ericsnowcurrently Mar 13, 2023
678e67b
Add _PyIO_InitTypes().
ericsnowcurrently Mar 13, 2023
69a5829
Fix test_capi.
ericsnowcurrently Mar 13, 2023
3feb408
Avoid allocation for shared exceptions.
ericsnowcurrently Mar 13, 2023
05806fc
Fix the ChannelID tp_name.
ericsnowcurrently Mar 13, 2023
b1cd7bb
Merge branch 'main' into per-interpreter-alloc
ericsnowcurrently Mar 29, 2023
4feb2b7
Do not include the total from interpreters sharing with main.
ericsnowcurrently Mar 29, 2023
136ad2f
Add _PyRuntime.obmalloc.interpreter_leaks.
ericsnowcurrently Mar 29, 2023
e19bb37
Track leaked blocks across init/fini cycles.
ericsnowcurrently Mar 29, 2023
6c51997
Clean up assumptions around runtime fini.
ericsnowcurrently Mar 29, 2023
f0fcaf6
Merge branch 'main' into per-interpreter-alloc
ericsnowcurrently Mar 29, 2023
0ff65ff
Add stubs for when WITH_PYMALLOC isn't defined.
ericsnowcurrently Mar 30, 2023
7db8d4a
Decref the key in the right interpreter in _extensions_cache_set().
ericsnowcurrently Mar 31, 2023
38bee89
Don't test against sys (for now).
ericsnowcurrently Mar 31, 2023
375a8f2
Clean up SubinterpImportTests.
ericsnowcurrently Mar 31, 2023
b0a9e11
Ensure we are testing against the right type of extension.
ericsnowcurrently Mar 31, 2023
5e5d5d5
Add a test that uses an isolated interpreter.
ericsnowcurrently Mar 31, 2023
25809ce
Fix is_core_module().
ericsnowcurrently Apr 4, 2023
616d3dd
Merge branch 'main' into per-interpreter-alloc
ericsnowcurrently Apr 4, 2023
43a836b
Ignore last_final_leaks.
ericsnowcurrently Apr 4, 2023
1841b55
Fix a typo.
ericsnowcurrently Apr 4, 2023
299527e
Merge branch 'main' into per-interpreter-alloc
ericsnowcurrently Apr 4, 2023
0091e48
Add a note about global state owned by the module.
ericsnowcurrently Apr 5, 2023
9f74f7b
Factor out GLOBAL_MALLOC() and GLOBAL_FREE().
ericsnowcurrently Apr 5, 2023
10c3589
Switch to the raw allocator.
ericsnowcurrently Apr 5, 2023
ff727ec
Merge branch 'channels-raw-allocator' into per-interpreter-alloc
ericsnowcurrently Apr 5, 2023
593430b
Use the raw allocator for _PyCrossInterpreterData_InitWithSize().
ericsnowcurrently Apr 5, 2023
f5ae710
atexit_callback -> atexit_py_callback.
ericsnowcurrently Apr 5, 2023
e6d4776
Add pycore_atexit.h.
ericsnowcurrently Apr 5, 2023
c719f02
Add _Py_AtExit().
ericsnowcurrently Apr 5, 2023
47c302d
Add a TODO comment.
ericsnowcurrently Apr 5, 2023
aaeaaa6
Move _Py_AtExit() to the public API.
ericsnowcurrently Apr 5, 2023
b5396e4
Test a constraint.
ericsnowcurrently Apr 5, 2023
448b48a
Add an atexit callback for _xxinterpchannels.
ericsnowcurrently Apr 5, 2023
c86f738
Implement the callback.
ericsnowcurrently Apr 5, 2023
1827feb
Drop the _PyCrossInterpreterData_Clear() call in _xxinterpchannels.
ericsnowcurrently Apr 5, 2023
82b395c
Drop the _PyCrossInterpreterData_Clear() call in _xxsubinterpreters.
ericsnowcurrently Apr 5, 2023
df77a64
Merge branch 'atexit-c-callback' into per-interpreter-alloc
ericsnowcurrently Apr 6, 2023
22758a3
Merge branch 'main' into per-interpreter-alloc
ericsnowcurrently Apr 6, 2023
0fd74a9
Merge branch 'main' into per-interpreter-alloc
ericsnowcurrently Apr 24, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix is_core_module().
  • Loading branch information
ericsnowcurrently committed Apr 4, 2023
commit 25809ce4d4b538686320f82a4ebe73ff99104630
14 changes: 13 additions & 1 deletion Python/import.c
Original file line number Diff line number Diff line change
Expand Up @@ -1110,7 +1110,17 @@ get_core_module_dict(PyInterpreterState *interp,
static inline int
is_core_module(PyInterpreterState *interp, PyObject *name, PyObject *filename)
{
return get_core_module_dict(interp, name, filename) != NULL;
/* This might be called before the core dict copies are in place,
so we can't rely on get_core_module_dict() here. */
if (filename == name) {
if (PyUnicode_CompareWithASCIIString(name, "sys") == 0) {
return 1;
}
if (PyUnicode_CompareWithASCIIString(name, "builtins") == 0) {
return 1;
}
}
return 0;
}

static int
Expand All @@ -1136,6 +1146,8 @@ fix_up_extension(PyObject *mod, PyObject *name, PyObject *filename)
// when the extension module doesn't support sub-interpreters.
if (def->m_size == -1) {
if (!is_core_module(tstate->interp, name, filename)) {
assert(PyUnicode_CompareWithASCIIString(name, "sys") != 0);
assert(PyUnicode_CompareWithASCIIString(name, "builtins") != 0);
if (def->m_base.m_copy) {
/* Somebody already imported the module,
likely under a different name.
Expand Down