Repository navigation
Performance of attribute lookup for type objects #92216
Description
Bug report
The performance of attribute lookup for type objects is worse than for other objects. A benchmark
import pyperf
runner=pyperf.Runner()
setup="""
class Class:
def all(self):
pass
x=Class()
"""
runner.timeit('hasattr x.all', "hasattr(x, 'all')", setup=setup)
runner.timeit('hasattr x.__array_ufunc__', "hasattr(x, '__array_ufunc__')", setup=setup)
runner.timeit('hasattr Class.all', "hasattr(Class, 'all')", setup=setup)
runner.timeit('hasattr Class.__array_ufunc__', "hasattr(Class, '__array_ufunc__')", setup=setup) # worse performance
Results:
hasattr x.all: Mean +- std dev: 68.1 ns +- 1.1 ns
hasattr x.__array_ufunc__: Mean +- std dev: 40.4 ns +- 0.3 ns
hasattr Class.all: Mean +- std dev: 38.1 ns +- 0.6 ns
hasattr Class.__array_ufunc__: Mean +- std dev: 255 ns +- 2 ns
The reason seems to be that the type_getattro always executes PyErr_Format, wheras for the "normal" attribute lookup this is avoided (see here and here)
Notes:
- The benchmark is from the python side, but we are working with the C-API
- The performance is important for numpy, see PERF: Improve performance of special attribute lookups numpy/numpy#21423
- Another location where this is a bottleneck: https://github.com/python/cpython/blob/v3.12.0a2/Lib/dataclasses.py#L1301
Your environment
- CPython versions tested on: Python 3.11.0a7+
- Operating system and architecture: Linux Ubuntu
Activity
seberg commentedon May 3, 2022
Related to this issue, I am wondering if we can consider
_PyObject_LookupAttreither as stable (removing the_) or as "semi stable"?Of course that is only faster currently for the default
tp_getattro, but I suspect that is a huge amount of calls.EDIT: Ah, I half thought it was not fully internal API right now, but "semi" exposed.
eendebakpt commentedon May 3, 2022
Also see: #76752 (PRs where
_PyObject_LookupAttris introduced)serhiy-storchaka commentedon May 4, 2022
_PyObject_LookupAttris an internal API.I'll look if it is possible to speed up lookup of the absent attribute in a class object.
PERF: Skip probing `__array_ufunc__` for NumPy builtin scalars (#21470)
gh-92216: improve performance of `hasattr` for type objects (GH-99979)
Fidget-Spinner commentedon Dec 23, 2022
@eendebakpt thanks for the improvement. Can I close this issue now or do you plan to submit more PRs?
eendebakpt commentedon Dec 23, 2022
The issue is solved, i will close
Merged main into regmachine branch (#50)