Skip to content

bpo-33930: Fix segfault with deep recursion when cleaning method objects #27678

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Aug 10, 2021
Merged

bpo-33930: Fix segfault with deep recursion when cleaning method objects #27678

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
a444686
bpo-33930: Fix segfault with deep recursion when cleaning method objects
pablogsal Aug 9, 2021
6bcd0a0
fixup! bpo-33930: Fix segfault with deep recursion when cleaning meth…
pablogsal Aug 9, 2021
80ac4cf
fixup! fixup! bpo-33930: Fix segfault with deep recursion when cleani…
pablogsal Aug 10, 2021
738fecc
Update Misc/NEWS.d/next/Core and Builtins/2021-08-09-14-29-52.bpo-339…
pablogsal Aug 10, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions Lib/test/test_exceptions.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1027,6 +1027,21 @@ def g():
self.assertIsInstance(v, RecursionError, type(v))
self.assertIn("maximum recursion depth exceeded", str(v))


@cpython_only
def test_crashcan_recursion(self):
# See bpo-33930

def foo():
o = object()
for x in range(1_000_000):
# Create a big chain of method objects that will trigger
# a deep chain of calls when they need to be destructed.
o = o.__dir__

foo()
support.gc_collect()

@cpython_only
def test_recursion_normalizing_exception(self):
# Issue #22898.
Expand Down
2 changes: 2 additions & 0 deletions Misc/NEWS.d/next/Core and Builtins/2021-08-09-14-29-52.bpo-33930.--5LQ-.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Fix segmentation fault with deep recursion when cleaning method objects.
Patch by Augusto Goulart and Pablo Galindo.
6 changes: 5 additions & 1 deletion Objects/methodobject.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,10 @@ PyCMethod_GetClass(PyObject *op)
static void
meth_dealloc(PyCFunctionObject *m)
{
_PyObject_GC_UNTRACK(m);
// The Py_TRASHCAN mechanism requires that we be able to
// call PyObject_GC_UnTrack twice on an object.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand that, but thanks for the comment!

Copy link
Contributor

@ambv ambv Aug 10, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tim-one added this comment in 803526b but the original change is by @nascheme, committed by @gvanrossum in ff413af with the following description:

    This is Neil's fix for SF bug 535905 (Evil Trashcan and GC interaction).

    The fix makes it possible to call PyObject_GC_UnTrack() more than once
    on the same object, and then move the PyObject_GC_UnTrack() call to
    *before* the trashcan code is invoked.

The "SF bug" is also available on BPO: https://bugs.python.org/issue535905

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the comment is unclear, maybe we should try to improve it. I believe the main point is that, since the object uses the trashcan, the function version of untrack is used rather than the macro version. The macro is not safe to call twice because it assumes the object is a part of the GC double linked list. At least, that's my memory of it.

Why the trashcan calls untrack more than once is obscure and I don't recall why it does so. I would need to study the code. My guess would be because it (ab)uses the GC head pointers to store the object in a special trashcan linked list.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it would be more clear if you would say that the code before Py_TRASHCAN_BEGIN can be executed more than once. Calling the macro version of untrack twice is not safe while the function version is safe.

PyObject_GC_UnTrack(m);
Py_TRASHCAN_BEGIN(m, meth_dealloc);
if (m->m_weakreflist != NULL) {
PyObject_ClearWeakRefs((PyObject*) m);
}
Expand All @@ -170,6 +173,7 @@ meth_dealloc(PyCFunctionObject *m)
Py_XDECREF(m->m_self);
Py_XDECREF(m->m_module);
PyObject_GC_Del(m);
Py_TRASHCAN_END;
}

static PyObject *
Expand Down