Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support PEP 696 – Type defaults for TypeVarLikes #14851

Open
intgr opened this issue Mar 7, 2023 · 9 comments
Open

Support PEP 696 – Type defaults for TypeVarLikes #14851

intgr opened this issue Mar 7, 2023 · 9 comments
Labels
feature topic-pep-696 TypeVar defaults

Comments

@intgr
Copy link
Contributor

intgr commented Mar 7, 2023

Feature

See PEP 696 – Type defaults for TypeVarLikes for details. The PEP has not yet been accepted though. Example from the PEP:

T = TypeVar("T", default=int)  # This means that if no type is specified T = int

@dataclass
class Box(Generic[T]):
    value: T | None = None

reveal_type(Box())                      # type is Box[int]
reveal_type(Box(value="Hello World!"))  # type is Box[str]

Since I did not find an existing tracking issue for this feature, I decided to open one.

The default= argument is already supported in typing_extensions version 4.4.0+ and typeshed (python/typeshed#8821)


Edit from maintainer: mypy supports basic PEP 696 use cases, but is missing some support for less common use cases. Give it a try!

@intgr intgr added the feature label Mar 7, 2023
JelleZijlstra pushed a commit that referenced this issue May 29, 2023
Start implementing [PEP 696](https://peps.python.org/pep-0696/) TypeVar
defaults. This PR
* Adds a `default` parameter to `TypeVarLikeExpr` and `TypeVarLikeType`.
* Updates most visitors to account for the new `default` parameter.
* Update existing calls to add value for `default` =>
`AnyType(TypeOfAny.from_omitted_generics)`.

A followup PR will update the semantic analyzer and add basic tests for
`TypeVar`, `ParamSpec`, and `TypeVarTuple` calls with a `default`
argument. -> #14873

Ref #14851
@JelleZijlstra JelleZijlstra added the topic-pep-696 TypeVar defaults label May 29, 2023
JelleZijlstra pushed a commit that referenced this issue Jun 6, 2023
This PR updates the semantic analyzer to support most forms of TypeVars
with defaults while also providing basic argument validation.

Ref: #14851
JelleZijlstra pushed a commit that referenced this issue Jun 15, 2023
Check that `default` type is a subtype of `bound` or one of the
constraint types.

Add `default` to nodes `__match_args__`. Missed that initially.

Ref: #14851
hauntsaninja pushed a commit that referenced this issue Aug 12, 2023
Use TypeVar defaults to resolve fallback return type of functions.

**Note**: Defaults for TypeVarTuples don't yet work, probably a result
of the limited support for `Unpack` / `TypeVarTuple`.

Ref: #14851
JelleZijlstra pushed a commit that referenced this issue Jan 24, 2024
…96) (#16812)

Start using TypeVar defaults when fixing instance types, instead of
filling those with `Any`.
This PR preserves the way an invalid amount of args is handled. I.e.
filling all with `Any` / defaults, instead of cutting off additional
args. Thus preserving full backwards compatibility. This can be easily
changed later if necessary.

`TypeVarTuple` defaults aren't handled correctly yet. Those will require
additional logic which would have complicated the change here and made
it more difficult to review.

Ref: #14851
JelleZijlstra pushed a commit that referenced this issue Jan 28, 2024
…696) (#16825)

This PR applies the TypeVar defaults to `TypeAlias` types instead of
using `Any` exclusively, similar to
#16812.
Again `TypeVarTuple` defaults aren't handled correctly yet.

Ref: #14851
JelleZijlstra pushed a commit that referenced this issue Jan 31, 2024
Implement type application for callables with TypeVar defaults.
Similar to previous PRs, support for TypeVarTuples is still TODO.

Ref: #14851
JelleZijlstra pushed a commit that referenced this issue Jan 31, 2024
Similar to TypeAlias types `Missing type parameters for generic type`
should not be emitted if too many arguments are given. There is a
separate error message for that.

Ref: #14851
JelleZijlstra pushed a commit that referenced this issue Feb 3, 2024
Small change to fix an error when expanding a TypeVarTuple default.

```
RuntimeError: Invalid type replacement to expand: Unpack[tuple[builtins.int, builtins.str]]
```

Ref: #14851
JelleZijlstra pushed a commit that referenced this issue Feb 20, 2024
This PR adds some additional error handling for recursive TypeVar
defaults.
Open issue for future PRs:
- Expanding nested recursive defaults, e.g. `T2 = list[T1 = str]`
- Scope binding, especially for TypeAliasTypes

Ref: #14851
hamdanal pushed a commit to hamdanal/mypy that referenced this issue Feb 20, 2024
This PR adds some additional error handling for recursive TypeVar
defaults.
Open issue for future PRs:
- Expanding nested recursive defaults, e.g. `T2 = list[T1 = str]`
- Scope binding, especially for TypeAliasTypes

Ref: python#14851
@JelleZijlstra
Copy link
Member

The original example passes now with mypy 1.9.0, thanks to @cdce8p's work (https://mypy-play.net/?mypy=latest&python=3.10&gist=ca609b35e9cbf97a600343b249c6b0a9). What still needs to be done here?

@intgr
Copy link
Contributor Author

intgr commented Mar 11, 2024

One missing feature is TypeVar(default=) referencing another TypeVar. django-stubs has been patiently waiting for this feature, it would allow us to clean up some pretty ugly hacks.

T = TypeVar("T")
U = TypeVar("U", default=T)  # If just T is specified, then U = T

class Box(Generic[T, U]):  # Box[T] would be expanded to Box[T, T]
    ...

Right now there's no warning when using this, mypy silently does the wrong thing. Playground: https://mypy-play.net/?mypy=latest&python=3.12&gist=22121ee5ef0b55d52d651a1d8636df73

@JelleZijlstra
Copy link
Member

Ah yes, that doesn't quite work yet:

from typing import Generic, TypeVar

T = TypeVar("T")
U = TypeVar("U", default=T)  # If just T is specified, then U = T

class Box(Generic[T, U]):  # Box[T] would be expanded to Box[T, T]
    ...

x = Box[int]()
reveal_type(x)  # E: Revealed type is "__main__.Box[builtins.int, T?]"

@intgr
Copy link
Contributor Author

intgr commented Mar 24, 2024

@cdce8p Are you interested in implementing this part of PEP 696 too -- allowing TypeVar to default to another TypeVar? Thanks for your work so far!

@cdce8p
Copy link
Collaborator

cdce8p commented Mar 24, 2024

@cdce8p Are you interested in implementing this part of PEP 696 too -- allowing TypeVar to default to another TypeVar?

I've already started working on it. See #16878. That one didn't make in time for the 1.9.0 release though. I do plan to continue the work, just not sure when I'll get to it. It might take some time.

@EwoutH
Copy link

EwoutH commented Aug 17, 2024

If I'm correct, this issue is required for Python 3.13 support right? What's needed to move it forward?

@JelleZijlstra
Copy link
Member

It's required to support some new syntax in Python 3.13, but mypy will type check most code in 3.13 just fine without this, so it depends on what you count as "Python 3.13 support".

As an example (probably relevant to why you're here :) ), to make Black support 3.13 mypyc compilation, this issue isn't relevant because Black doesn't use TypeVars with defaults at the moment.

However, @cdce8p implemented much of what's needed for this feature in any case (thanks!). I don't know how much is left, but most basic use cases should already work.

@cdce8p
Copy link
Collaborator

cdce8p commented Aug 17, 2024

As an example (probably relevant to why you're here :) ), to make Black support 3.13 mypyc compilation, this issue isn't relevant because Black doesn't use TypeVars with defaults at the moment.

The issue you're looking for is probably mypyc/mypyc#1056.

However, @cdce8p implemented much of what's needed for this feature in any case (thanks!). I don't know how much is left, but most basic use cases should already work.

The next step would be to add support for the new TypeVar defaults syntax in 3.13. I've started working on it already, just need to find some time soon to finish it and open a PR.

Besides that, the support for ParamSpec / TypeVarTuple defaults and recursive TypeVar defaults is still quite limited unfortunately. Although it would be great to fully support these, PEP 696 is already useable in the most common cases.

JelleZijlstra pushed a commit that referenced this issue Oct 19, 2024
Add initial support for TypeVar defaults using the new syntax. Similar
to the old syntax, it doesn't fully work yet for ParamSpec, TypeVarTuple
and recursive TypeVar defaults.

Refs: #14851
@hauntsaninja
Copy link
Collaborator

Examples like the following still fail:

from typing import Generic, assert_type
from typing_extensions import TypeVar

DefaultStrT = TypeVar("DefaultStrT", default=str)
DefaultIntT = TypeVar("DefaultIntT", default=int)

class NoNonDefaults(Generic[DefaultStrT, DefaultIntT]): ...

assert_type(NoNonDefaults, type[NoNonDefaults[str, int]])
assert_type(NoNonDefaults[str], type[NoNonDefaults[str, int]])
assert_type(NoNonDefaults[str, int], type[NoNonDefaults[str, int]])

It looks like this is due to the thing I find slightly confusing where mypy represents type objects as CallableType. One possible way to fix is to replace type variables with their defaults when comparing CallableType to TypeType and vice versa. Maybe there's a better way too

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature topic-pep-696 TypeVar defaults
Projects
None yet
Development

No branches or pull requests

5 participants