Description
openedon Nov 20, 2017
Currently, Cassette's mechanism for intercepting method calls triggers certain inference heuristics that wouldn't normally be triggered on the underlying calls themselves. This can cause inference to give up on inferring a bit too early in some cases, hindering the performance of the resulting code.
For example: Cassette wraps every downstream function call in its own callable type, which causes inference's recursion limiting heuristic to think "hmm, there's a lot of the same function calling itself here, so I should stop inferring now before this recursive madness gets out of hand."
Here's a two-part solution to this problem that @vtjnash, @Keno and I cooked up:
-
Add an extra
method_for_heuristic_purposes
field toCodeInfo
. When inference runs certain heuristics, it will check if this field is inflated. If so, it can use the method in this field rather than the "actual" method for certain parts of the heuristic calculation. This way,CodeInfo
-emitting@generated
functions can spoof inference's work-limiting heuristics by inflating the field with the appropriate method. -
As it stands,
@generated
functions can't actually leverage the above solution, because the limits that need to be spoofed are computed before generator expansion time. To overcome this hurdle, we can add anexpand_generator_early
flag toMethod
that inference can query before it starts applying certain heuristics. If this flag is set for a@generated
method call that inference is working on, inference will expand the generator earlier than normal (e.g. pre-abstract_call_method
), thus making themethod_for_heuristic_purposes
field available to downstream heuristics.
This approach should fix some performances problems for Cassette-like packages by allowing them to tell inference "Hey, don't use my weird interception functor to calculate heuristic work limits; use this underlying method instead."
cc @maleadt