PEP 695: Lazy evaluation, concrete scoping semantics, other changes

[JelleZijlstra]

This adds some clarifications and changes that I ran into while implementing the PEP.

The most important change is that TypeVar bounds and type alias values are lazily evaluated using PEP 649-like semantics. This will make the language more consistent once type annotations are evaluated using PEP 649.

Prior discussion:

• PEP 649-like semantics: Lazy evaluation and PEP 649 erictraut/cpython#11
• __type_params__ attribute replacing __type_variables__: Semantics of __type_variables__ erictraut/cpython#10, Clarification in the choice of __type_variables__ vs __type_param(eter)?s__ erictraut/cpython#8
• Disallowing yield etc.: Prohibit yield, yield from, await, walrus in aliases and bounds? erictraut/cpython#13

I am marking this as draft for now in case I run into more things that need to be changed. We'll have to ask the SC for approval, but hopefully these changes are not controversial.

cc @erictraut

---

📚 Documentation preview 📚: https://pep-previews--3122.org.readthedocs.build/

[erictraut]

LGTM

[gvanrossum]

Some preliminary comments while I have this freshly in my head.

[JelleZijlstra]

@erictraut @gvanrossum I pushed a few more changes; please take a look.

I am also considering adding more information about the implementation technique I'm currently using (a dummy immediately evaluated function, which I just learned is "lambda lifting"), but I'm not sure we need to specify that in the PEP.

[JelleZijlstra]

Submitted python/steering-council#186 to the SC to approve these changes. Marking as "DO-NOT-MERGE" until the SC approves.

[gvanrossum]

Thanks for taking care to explain all these details clearly!

[cdce8p]

Could you include the small AST change for TypeAlias.name here as well?

Couldn't comment inline unfortunately. It should be here:

peps/pep-0695.rst

Line 802 in 63a594f

TypeAlias(identifier name, typeparam* typeparams, expr value)

- TypeAlias(identifier name, typeparam* typeparams, expr value)
+ TypeAlias(expr name, typeparam* typeparams, expr value)

[JelleZijlstra]

Pushed another change related to name mangling (I realized that the wording in the PEP would require severe complications in the symtable).

[gvanrossum]

+1 on the mangling change.

[erictraut]

Does the mangling change mean that inner scopes cannot refer to that type parameter without generating a runtime exception?

class Foo[__T]:
    def method(self, val: __T): # Does this work?
        x: __T = val # Does this work?

[JelleZijlstra]

@erictraut yes, that will work. (Though note that the annotation within the function will always work without a runtime exception, because annotations on locals are never evaluated.) The name is mangled at both the definition site and all usage sites within the class. This was quite easy to implement: python/cpython@bd49622.

The PEP previously proposed not mangling the TypeVar names. That would require deep changes to the symtable to track whether a name refers to a TypeVar. The new proposed approach is more consistent with the rest of the language: names that are syntactically within a class are always mangled.

I think there are very few ways the mangling is even visible to user code. The only way I can think of is this:

>>> class A: pass
... 
>>> class Foo[__T](print(locals()) or A): pass
... 
{'_Foo__T': __T, '.generic_base': typing.Generic[__T], '.type_params': (__T,)}

[markshannon]

Definitely an improvement.

It still isn't clear to me which of the def695 "functions" exist to provide scopes for type variables, and which are to support lazy evaluation. Are the semantics of both identical?

[JelleZijlstra]

It still isn't clear to me which of the def695 "functions" exist to provide scopes for type variables, and which are to support lazy evaluation. Are the semantics of both identical?

Yes, they have identical semantics. Let me push something to make that explicit.

[JelleZijlstra]

The SC approved the proposed changes.

[hugovk]

For reference: python/steering-council#186 (comment)