Improving inference caching (unnecessary context clones?)

[brycepg]

From investigating the code I'm getting the impression that context cache_generator isn't working very well for caching, and may have some low hanging fruit for improving performance.

Possible improvements:

• Allow partial caching. Currently an inference value is only cached if the inference is completely exhausted. Since there are many instances of only retrieving the first value of the inference, caching is ignored because there is no partial caching.
• Have context.push attempt to find inference results in the cache. This could potentially reduce the amount of context.clones required to stop inference from failing due to already traveled values in the context path.
• Centralize caching. Currently caching is tied to the context. Since pylint does many disparate inferences visiting the ast, this may have a significant speedup. One possible downside would be a larger memory-speed tradeoff.

[PCManticore]

These all sound good to me, but I would move the third point into its own issue, as I don't see it a trivial low hanging fruit. We'd also have to monitor that with these changes we don't get a too much increase in the used memory, but with your dataset this should be relatively easy to figure out.