Generator .close does not release resources

[gsbrodal]

Bug report

Bug description:

Sending a generator .close() in Python 3.12 does not release resources used by local variables as it used to do in earlier versions of Python. Wrapping the generator content in a try: ... except GeneratorExit: pass releases the resources as expected.

The following shows the difference in space usage between wrapping the code with try-except and not.

def generator_using_a_lot_of_space():
    a_big_set = set(range(10_000_000))
    yield 42

g = generator_using_a_lot_of_space()
input('A <press enter>')  # 13.4 MB
print(next(g))
input('B <press enter>')  # 576.6 MB
g.close()
input('C <press enter>')  # 576.6 MB  <== space usage used to drop to 13-14 MB

def generator_using_a_lot_of_space():
    a_big_set = set(range(10_000_000))
    try: 
        yield 42
    except GeneratorExit:
        pass
    
g = generator_using_a_lot_of_space()
input('D <press enter>')  # 14.6 MB
print(next(g))
input('E <press enter>')  # 575.6 MB
g.close()
input('F <press enter>')  # 14.6 MB  <== drop in space usage as expected

The above space measurements were done with
Python 3.12.3 (tags/v3.12.3:f6650f9, Apr 9 2024, 14:05:25) [MSC v.1938 64 bit (AMD64)] on win32
and looking at the space usage reported by the Windows 11 Task Manager.

CPython versions tested on:

3.12

Operating systems tested on:

Windows

Linked PRs

• gh-118272: Clear generator frame's locals when the generator is closed #118277
• [3.12] gh-118272: Clear generator frame's locals when the generator is closed #118451
• gh-118272: set stacktop to 0 before freeing contents, to avoid access to invalid objects during GC #118478

[graingert]

bisected to eb4774d using:

import gc
import weakref

class Foo:
    pass

f = Foo()
f_wr = weakref.ref(f)

def genfn():
    a = f
    yield


g = genfn()
next(g)
del f
g.close()
gc.collect()
print(f_wr)

outputs:

<weakref at 0x7fd58a1917b0; to 'Foo' at 0x7fd58ab12cf0>

should output:

<weakref at 0x7ad6faf19d50; dead>

passes on 3.11, fails on 3.12 and 3.13a6

[iritkatriel]

Thank you, @gsbrodal for the report, and @graingert for the investigation.