Skip to content

Allow CodeInfo-emitting generated functions to somewhat spoof inference's work-limiting heuristics #24676

Closed

Description

Currently, Cassette's mechanism for intercepting method calls triggers certain inference heuristics that wouldn't normally be triggered on the underlying calls themselves. This can cause inference to give up on inferring a bit too early in some cases, hindering the performance of the resulting code.

For example: Cassette wraps every downstream function call in its own callable type, which causes inference's recursion limiting heuristic to think "hmm, there's a lot of the same function calling itself here, so I should stop inferring now before this recursive madness gets out of hand."

Here's a two-part solution to this problem that @vtjnash, @Keno and I cooked up:

  1. Add an extra method_for_heuristic_purposes field to CodeInfo. When inference runs certain heuristics, it will check if this field is inflated. If so, it can use the method in this field rather than the "actual" method for certain parts of the heuristic calculation. This way, CodeInfo-emitting @generated functions can spoof inference's work-limiting heuristics by inflating the field with the appropriate method.

  2. As it stands, @generated functions can't actually leverage the above solution, because the limits that need to be spoofed are computed before generator expansion time. To overcome this hurdle, we can add an expand_generator_early flag to Method that inference can query before it starts applying certain heuristics. If this flag is set for a @generated method call that inference is working on, inference will expand the generator earlier than normal (e.g. pre-abstract_call_method), thus making the method_for_heuristic_purposes field available to downstream heuristics.

This approach should fix some performances problems for Cassette-like packages by allowing them to tell inference "Hey, don't use my weird interception functor to calculate heuristic work limits; use this underlying method instead."

cc @maleadt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions