[Linux] Performance optimization for Process.memory_full_info()

[itamarst]

Summary

• OS: Linux
• Type: performance

Description

Current Process().memory_full_info() reads /proc/<pid>/smaps, and then sums across maps. This can be slow if there are many maps.

Linux provides a /proc/<pid>/smaps_rollup that is the information in smaps summarized across the whole process: exactly what this function is calculating. For processes with large number of mmaps, using this file directly would reduce the cost of memory_full_info() significantly.

This was apparently added to Linux sometime in 2017 or 2018.

It's the same format, so you could the exact same code, even, just open that file instead if it exists.

[giampaolo]

Well, this is awesome.

[giampaolo]

I've just implemented this in #2108.

Without patch:

~/svn/psutil {linux-smaps-rollup}$ python3 -m timeit -s "import psutil; p = psutil.Process()" "p.memory_full_info()"
500 loops, best of 5: 518 usec per loop

With patch (5 times faster):

~/svn/psutil {linux-smaps-rollup}$ python3 -m timeit -s "import psutil; p = psutil.Process()" "p.memory_full_info()"
2000 loops, best of 5: 111 usec per loop

---

make test-memleaks suite, who heavily rely on Process.memory_full_info(), also received a nice speedup:

Before patch:

$ make test-memleaks
----------------------------------------------------------------------
Ran 99 tests in 1.646s

OK (skipped=9)
SUCCESS

After patch:

$ make test-memleaks
----------------------------------------------------------------------
Ran 99 tests in 1.195s

OK (skipped=9)
SUCCESS