Numerous refleaks on main

[JelleZijlstra]

https://buildbot.python.org/all/#/release_status shows that the refleak buildbots are failing on current main. Example output on https://buildbot.python.org/all/#/builders/75/builds/745/steps/5/logs/warnings__111_.

Failing tests:

test_functools leaked [1, 1, 2] references, sum=4
test_warnings leaked [4, 4, 4] references, sum=12
test_ipaddress leaked [6, 6, 6] references, sum=18
test_zoneinfo leaked [7, 6, 11] references, sum=24
FAIL: test_flock (__main__.FNTLEINTRTest.test_flock)
test_random leaked [1, 2, 4] references, sum=7
test_statistics leaked [175, 174, 175] references, sum=524
test_dataclasses leaked [3, 3, 2] references, sum=8
test_argparse leaked [69, 70, 76] references, sum=215
/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z.refleak/build/Lib/test/support/__init__.py:743: ResourceWarning: unclosed <socket.socket [closed] fd=3, family=2, type=1, proto=6>
ResourceWarning: Enable tracemalloc to get the object allocation traceback
./home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z.refleak/build/Lib/test/support/__init__.py:743: ResourceWarning: unclosed <socket.socket [closed] fd=3, family=2, type=1, proto=6>
ResourceWarning: Enable tracemalloc to get the object allocation traceback
./home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z.refleak/build/Lib/test/support/__init__.py:743: ResourceWarning: unclosed <socket.socket [closed] fd=3, family=2, type=1, proto=6>
ResourceWarning: Enable tracemalloc to get the object allocation traceback
./home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z.refleak/build/Lib/test/support/__init__.py:743: ResourceWarning: unclosed <socket.socket [closed] fd=3, family=2, type=1, proto=6>
ResourceWarning: Enable tracemalloc to get the object allocation traceback
./home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z.refleak/build/Lib/test/support/__init__.py:743: ResourceWarning: unclosed <socket.socket [closed] fd=3, family=2, type=1, proto=6>
ResourceWarning: Enable tracemalloc to get the object allocation traceback
./home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z.refleak/build/Lib/test/support/__init__.py:743: ResourceWarning: unclosed <socket.socket [closed] fd=3, family=2, type=1, proto=6>
ResourceWarning: Enable tracemalloc to get the object allocation traceback
test_tarfile leaked [17, 17, 16] references, sum=50
test_typing leaked [95, 121, 95] references, sum=311
test_typing leaked [1, 14, 1] memory blocks, sum=16
test_lib2to3 leaked [5, 8, 6] references, sum=19
test_shutil leaked [2, 2, 2] references, sum=6
test_unittest leaked [8, 12, 15] references, sum=35
test_logging leaked [3, 2, 1] references, sum=6
test__xxsubinterpreters leaked [46, 45, 45] references, sum=136
test_mailbox leaked [2, 3, 2] references, sum=7
test_socket leaked [24, 23, 23] references, sum=70
test_htmlparser leaked [2, 2, 2] references, sum=6
test_regrtest leaked [9, 12, 8] references, sum=29
test_datetime leaked [11, 11, 11] references, sum=33
test_nntplib leaked [1, 3, 2] references, sum=6
test_ftplib leaked [10, 13, 9] references, sum=32
test_interpreters leaked [54, 54, 54] references, sum=162
test_zipfile leaked [3, 4, 4] references, sum=11
test_sys_settrace leaked [6, 6, 6] references, sum=18
test_descr leaked [3, 65, 67] references, sum=135
test_compile leaked [7, 7, 7] references, sum=21
test_bytes leaked [2, 2, 2] references, sum=6
test_ast leaked [9, 9, 9] references, sum=27
test_poplib leaked [3, 3, 2] references, sum=8
test_types leaked [8, 50, 43] references, sum=101
test_concurrent_futures leaked [11, 12, 11] references, sum=34
test_genericalias leaked [2, 2, 3] references, sum=7
test_math leaked [32, 31, 33] references, sum=96
test_numeric_tower leaked [2, 1, 2] references, sum=5
test_capi leaked [99, 99, 99] references, sum=297
test_import leaked [170, 171, 168] references, sum=509
test_import leaked [71, 71, 70] memory blocks, sum=212
.test test_monitoring failed -- multiple errors occurred; run in verbose mode for details
test_selectors leaked [5, 6, 4] references, sum=15
test_email leaked [27, 26, 25] references, sum=78
test_subprocess leaked [1, 1, 1] references, sum=3
test_struct leaked [1, 1, 1] references, sum=3
test_weakref leaked [1, 1, 1] references, sum=3
test_imaplib leaked [2, 2, 1] references, sum=5
0:45:54 load avg: 0.13 Re-running failed tests in verbose mode
0:45:54 load avg: 0.13 Re-running test_functools in verbose mode
test_functools leaked [1, 1, 2] references, sum=4
0:45:55 load avg: 0.13 Re-running test_warnings in verbose mode
test_warnings leaked [4, 4, 4] references, sum=12
0:45:59 load avg: 0.20 Re-running test_ipaddress in verbose mode
test_ipaddress leaked [6, 6, 6] references, sum=18
0:46:00 load avg: 0.20 Re-running test_zoneinfo in verbose mode
test_zoneinfo leaked [7, 6, 11] references, sum=24

I saw similar leaks triggering the refleak buildbots on #103866 and #103764, before I realized the issue was probably on main.

Based on @sunmy2019's work in #103764 (comment), we're likely leaking references to the object.__setattr__ wrapper descriptor.

Linked PRs

• gh-103879: Fix refleak in super specialization #103882

[JelleZijlstra]

The failure on test.test_htmlparser bisects to

ef25febcf2ede92a03c5ea00a13e167e0b5cb274 is the first bad commit
commit ef25febcf2ede92a03c5ea00a13e167e0b5cb274
Author: Carl Meyer <carl@oddbird.net>
Date:   Tue Apr 25 11:45:51 2023 -0600

    gh-87729: specialize LOAD_SUPER_ATTR_METHOD (#103809)

@carljm

[carljm]

Given that refleak buildbots passed on #103882, it seems this is fixed now.

[carljm]

Thanks @JelleZijlstra for fixing!

[JelleZijlstra]

No problem!

https://buildbot.python.org/all/#/release_status is still red but I assume the buildbots are running now. Given that the refleak buildbots were green on my PR, we probably did fix the issue.