Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve Python 3.11 support #3694

Merged
merged 21 commits into from
Mar 2, 2022
Merged

Improve Python 3.11 support #3694

Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
a0eb05b
Test out Python 3.11 migration
Skylion007 Feb 7, 2022
6a0a4d0
Clean up a bit
Skylion007 Feb 7, 2022
e3f3644
Remove todo
Skylion007 Feb 7, 2022
c0a1306
Test workaround
Skylion007 Feb 7, 2022
a731769
Fix potential bug uncovered in 3.11
Skylion007 Feb 7, 2022
be95611
Try to fix it more
Skylion007 Feb 7, 2022
5864a2a
last ditch fix
Skylion007 Feb 7, 2022
4428748
Merge branch 'master' of https://github.com/pybind/pybind11 into test…
Skylion007 Feb 12, 2022
b292dd8
Revert. Tp-traverse isn't the problem
Skylion007 Feb 12, 2022
729bb4f
Test workaround
Skylion007 Feb 12, 2022
8bd9214
Try this hack
Skylion007 Feb 12, 2022
1457185
Merge branch 'master' of https://github.com/skylion007/pybind11 into …
Skylion007 Feb 27, 2022
b375591
Revert MRO changes
Skylion007 Feb 27, 2022
6fa3fc5
Use f_back properly
Skylion007 Feb 28, 2022
38f9096
Qualify auto
Skylion007 Feb 28, 2022
9b9df3d
Update include/pybind11/pybind11.h
henryiii Mar 1, 2022
6868334
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 1, 2022
1eccd5d
Simplify code slightly
Skylion007 Mar 1, 2022
b87c337
Merge branch 'master' of https://github.com/pybind/pybind11 into test…
Skylion007 Mar 1, 2022
eadcb87
Ensure co_varnames decref if dict_getitem throws
Skylion007 Mar 2, 2022
4add4c2
Eager decref f_code
Skylion007 Mar 2, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 13 additions & 2 deletions include/pybind11/detail/class.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -653,6 +653,16 @@ inline PyObject *make_new_python_type(const type_record &rec) {
auto bases = tuple(rec.bases);
auto *base = (bases.empty()) ? internals.instance_base : bases[0].ptr();

bool has_dynamic_base = false;
for (auto b : bases) {
auto *obj = (PyTypeObject *) b.ptr();
// hack to see if enable_dynamic_attrs if any of the bases do.
if (obj->tp_traverse == pybind11_traverse) {
has_dynamic_base = true;
break;
}
}

/* Danger zone: from now (and until PyType_Ready), make sure to
issue no Python C API calls which could potentially invoke the
garbage collector (the GC will call type_traverse(), which will in
Expand Down Expand Up @@ -694,7 +704,7 @@ inline PyObject *make_new_python_type(const type_record &rec) {
type->tp_flags |= Py_TPFLAGS_BASETYPE;
}

if (rec.dynamic_attr) {
if (rec.dynamic_attr || has_dynamic_base) {
enable_dynamic_attributes(heap_type);
}

Expand All @@ -710,7 +720,8 @@ inline PyObject *make_new_python_type(const type_record &rec) {
pybind11_fail(std::string(rec.name) + ": PyType_Ready failed (" + error_string() + ")!");
}

assert(!rec.dynamic_attr || PyType_HasFeature(type, Py_TPFLAGS_HAVE_GC));
assert(type->tp_traverse != nullptr || !PyType_HasFeature(type, Py_TPFLAGS_HAVE_GC));
// assert(!rec.dynamic_attr || PyType_HasFeature(type, Py_TPFLAGS_HAVE_GC));

/* Register type with the parent scope */
if (rec.scope) {
Expand Down
11 changes: 5 additions & 6 deletions include/pybind11/pybind11.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2636,19 +2636,18 @@ get_type_override(const void *this_ptr, const type_info *this_type, const char *

/* Don't call dispatch code if invoked from overridden function.
Unfortunately this doesn't work on PyPy. */
#if !defined(PYPY_VERSION) && PY_VERSION_HEX < 0x030B0000
// TODO: Remove PyPy workaround for Python 3.11.
// Current API fails on 3.11 since co_varnames can be null.
#if !defined(PYPY_VERSION)
# if PY_VERSION_HEX >= 0x03090000
PyFrameObject *frame = PyThreadState_GetFrame(PyThreadState_Get());
if (frame != nullptr) {
PyCodeObject *f_code = PyFrame_GetCode(frame);
// f_code is guaranteed to not be NULL
if ((std::string) str(f_code->co_name) == name && f_code->co_argcount > 0) {
PyObject *locals = PyEval_GetLocals();
if (locals != nullptr && f_code->co_varnames != nullptr) {
PyObject *self_caller
= dict_getitem(locals, PyTuple_GET_ITEM(f_code->co_varnames, 0));
if (locals != nullptr) {
PyObject *self_arg = PyTuple_GET_ITEM(
PyObject_GetAttrString((PyObject *) f_code, "co_varnames"), 0);
PyObject *self_caller = dict_getitem(locals, self_arg);
henryiii marked this conversation as resolved.
Show resolved Hide resolved
if (self_caller == self.ptr()) {
Py_DECREF(f_code);
Py_DECREF(frame);
Expand Down
4 changes: 3 additions & 1 deletion tests/test_multiple_inheritance.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,9 @@ TEST_SUBMODULE(multiple_inheritance, m) {
m.def("i801e_b2", []() -> I801B2 * { return new I801E(); });

// test_mi_static_properties
py::class_<Vanilla>(m, "Vanilla").def(py::init<>()).def("vanilla", &Vanilla::vanilla);
py::class_<Vanilla>(m, "Vanilla", py::dynamic_attr())
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making all bases in the hierarchy either static or dynamic_attrs works, but is not a tenable solution.

.def(py::init<>())
.def("vanilla", &Vanilla::vanilla);

py::class_<WithStatic1>(m, "WithStatic1")
.def(py::init<>())
Expand Down