Type parameters: Incorrect text in SyntaxError for disallowed expression

[JelleZijlstra]

Bug report

Bug description:

We disallow certain expressions (e.g., yield) in type parameters bounds, constraints, and defaults. But the error message always says it's a bound:

>>> def f[T=(yield)](): pass
  File "<python-input-0>", line 1
SyntaxError: yield expression cannot be used within a TypeVar bound

>>> def f[T: (int, (yield))](): pass
  File "<python-input-2>", line 1
SyntaxError: yield expression cannot be used within a TypeVar bound

We could either add some machinery in the symbol table so it knows whether we're in a bound, constraints, or default, or just change the error message to something like "within a TypeVar bound, constraints, or default".

CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

Linked PRs

• gh-119933 : Improve SyntaxError message for invalid type parameters expressions #119976
• [3.13] gh-119933: Improve SyntaxError message for invalid type parameters expressions (GH-119976) #120641

[picnixz]

I know that you self-assigned that task, but can I perhaps help for this one if you are not in a hurry for the latter? I'd be interested in knowing how to make it work.

[JelleZijlstra]

Sure, this issue would be a good way to get some familiarity with the symtable code. I partly self-assigned this because the solution is likely to create some conflicts with #119361, and I want to get that landed first :). But the conflict is likely manageable, so feel free to get started now.

The solution is going to look a little different on 3.12 (which has only TypeVar bounds and constraints), 3.13 (which adds TypeVar/ParamSpec/TypeVarTuple defaults), and 3.14 (which will implement PEP 649, using annotation scopes in more contexts).

I think there's two approaches we can take to solving the issue:

• Give a more generic error, maybe just "yield expression cannot be used here". That's easier to implement since we don't have to keep track of exactly what flavor of annotation scope we're in, but the error becomes more generic.
• Make the errors more specific. When I implemented PEP 695, I attempted to do this by adding to the _Py_block_ty enum. However, now that we have type parameter defaults too, we'd have to add another three entries to that enum, which feels bad for maintenance. Maybe alternatively, we could add a new string field to the PySTEntryObject struct to hold a string representing the kind of scope we're in. Then we can just pass that string when entering the block and unify a couple of the entries in the _Py_block_ty enum.

[picnixz]

I actually just implemented something along the lines for 3.14:

• Add an extra integer field to _symtable_entry objects of the following type:

/* Additional flags that are set temporarily on a symtable object.
 *
 * Those flags are only used to add some information on the current context.
 */
typedef enum _extra_flags {
    // Mutually exclusive flags indicating which component of
    // a type parameters block (PEP 695) is being processed.
    InTypeVarBound = 1,
    InTypeVarConstraint = 2,
    InTypeVarDefault = 4,
} _Py_extra_fls;

Here, it's only used for adding some extra information for the "component" being processed in a type parameter block, but it can be used for other kind of blocks in the future. I also tried a first implementation where you add more block types but it became a bit messy...

By the way, since I didn't see anything for distinguishing between bounds and constraints, I assumed that a tuple expression is always considered a constraints but if this is not the case (namely, you could have cases where the expression's kind is Tuple_kind but we are actually dealing with a bound, then please let me know how I can distinguish and test those cases).

I'm sending the PR in a few moments (I'll just skip the news and changes for now, since this is something I can do later).

[JelleZijlstra]

InTypeVarDefault

This isn't enough, because ParamSpec and TypeVarTuple can also have defaults.

I assumed that a tuple expression is always considered a constraints

That is correct; it's constraints if the top level is a Tuple and a bound otherwise.

[picnixz]

This isn't enough, because ParamSpec and TypeVarTuple can also have defaults.

Oh, yes, I should maybe change the naming then (actually, ParamSpec and TypeVarTuple are correctly handled but I just badly named the fields).

[picnixz]

Actually, what kind of error should I use for a TypeVarTuple and ParamSpec? should I also change the "in a TypeVar default" into "in a TypeVarTuple/ParamSpec default"? (same question for class definitions and so on).

[JelleZijlstra]

Yes, I think it should say e.g. "yield expression cannot be used in a ParamSpec default".