Making Pylint faster 3

[nickdrozd]

This PR makes just one change, namely caching the result of the function lookup. The impact of this change seems to be proportional to the size of the project. Here are the results of timing before and after against zulip/zerver (a large directory belonging to the Zulip chat app) and against pycodestyle.py.

zulip/zerver

Before

real	3m27.529s
user	3m2.148s
sys	0m10.265s

real	3m29.610s
user	3m3.250s
sys	0m10.677s

real	3m30.208s
user	3m3.869s
sys	0m10.154s

After

real	2m54.406s
user	2m29.059s
sys	0m9.832s

real	2m55.215s
user	2m30.475s
sys	0m10.268s

real	2m53.733s
user	2m28.744s
sys	0m9.856s

pycodestyle.py

Before

real	0m5.523s
user	0m5.380s
sys	0m0.109s

real	0m5.430s
user	0m5.311s
sys	0m0.095s

real	0m5.365s
user	0m5.223s
sys	0m0.092s

After

real	0m4.834s
user	0m4.660s
sys	0m0.119s

real	0m4.736s
user	0m4.579s
sys	0m0.104s

real	0m4.736s
user	0m4.557s
sys	0m0.110s

(By the way, you might look at the numbers here and notice that they are substantially less than the numbers presented in my previous PRs. This is because those times were recorded with a profiler enabled. It seems obvious in hindsight that running a program with a profiler enabled would cause it to run slower, even substantially slower, but live and learn I guess. In any case, these are real times.)

[PCManticore]

This should be a lot simpler with decorators.cached_property

[nickdrozd]

@PCManticore How would that work? I thought that decorator only worked with functions of no arguments.

[PCManticore]

Sorry, that was supposed to be decorators.cached.

[nickdrozd]

Maybe I'm missing something, but it looks to me like cached also applies to functions with no arguments. Is there another way of using it?

[nickdrozd]

By the way, I rebased on master (for both pylint and astroid) and ran the timer again, and it seems pylint's running time has increased more than a little in the past week or so. I suspect pylint-dev/pylint@98c171a might be a culprit, since it introduces two new uses of nodes_of_class, which is known to be inefficient.

[PCManticore]

Thanks for finding it! I'll check to see if it is possible to use something else instead of nodes_of_class for that check. Regarding this cached argument, you're right, it's not even used anymore, so I'm removing it. Since we're on Python 3, we can also use functools.lru_cache with a bounded cache, this should work with multiple parameters, instead of rolling our own caching in lookup function.