Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix TypeVar upper bounds sometimes not being displayed in pretty callables #17802

Merged

Conversation

brianschubert
Copy link
Collaborator

@brianschubert brianschubert commented Sep 21, 2024

Fixes #17792. Related to #17791.

Currently, pretty_callable only renders TypeVar upper bounds if they are of type Instance:

mypy/mypy/messages.py

Lines 2943 to 2949 in 5dfc7d9

if isinstance(tvar, TypeVarType):
upper_bound = get_proper_type(tvar.upper_bound)
if (
isinstance(upper_bound, Instance)
and upper_bound.type.fullname != "builtins.object"
):
tvars.append(f"{tvar.name}: {format_type_bare(upper_bound, options)}")

However, there are some types that can appear as TypeVar upper bounds which are not represented by Instance, such as UnionType and CallableType.

This PR allows such non-Instance upper bounds to be rendered as well.

Effect

Consider the below code.
Playground link: https://mypy-play.net/?mypy=1.11.2&python=3.12&enable-incomplete-feature=NewGenericSyntax&gist=ba30c820cc3668e0919dadf2f391ff4b

from collections.abc import Callable
from typing import Any, overload

### No matching overloads

@overload
def f1[T: int](x: T) -> T: ...
@overload
def f1[T: Callable[..., None]](x: T) -> T: ...
@overload
def f1[T: tuple[int]](x: T) -> T: ...
@overload
def f1[T: None](x: T) -> T: ...
@overload
def f1[T: type[int]](x: T) -> T: ...
@overload
def f1[T: bytes | bytearray](x: T) -> T: ...
def f1(x): return x

f1(1.23)

### Mismatching conditional definitions

if input():
    def f2[T](x: T) -> T:
        return x
else:
    def f2[T: Callable[..., None]](x: T) -> T:
        return x

Before

  • In the first error on line 20, all overloads aside from the first one are displayed as def [T] f1(x: T) -> T (upper bound missing). Duplicate entries are suppressed.
  • In the second error on line 28, the second definition is displayed as def [T] f2(x: T) -> T (upper bound missing), and is removed as an apparent duplicate of the first.
main.py:20: error: No overload variant of "f1" matches argument type "float"  [call-overload]
main.py:20: note: Possible overload variants:
main.py:20: note:     def [T: int] f1(x: T) -> T
main.py:20: note:     def [T] f1(x: T) -> T
main.py:28: error: All conditional function variants must have identical signatures  [misc]
main.py:28: note: Original:
main.py:28: note:     def [T] f2(x: T) -> T
main.py:28: note: Redefinition:
Found 2 errors in 1 file (checked 1 source file)

After

  • All type var upper bounds are rendered.
main.py:20: error: No overload variant of "f1" matches argument type "float"  [call-overload]
main.py:20: note: Possible overload variants:
main.py:20: note:     def [T: int] f1(x: T) -> T
main.py:20: note:     def [T: Callable[..., None]] f1(x: T) -> T
main.py:20: note:     def [T: tuple[int]] f1(x: T) -> T
main.py:20: note:     def [T: None] f1(x: T) -> T
main.py:20: note:     def [T: type[int]] f1(x: T) -> T
main.py:20: note:     def [T: bytes | bytearray] f1(x: T) -> T
main.py:28: error: All conditional function variants must have identical signatures  [misc]
main.py:28: note: Original:
main.py:28: note:     def [T] f2(x: T) -> T
main.py:28: note: Redefinition:
main.py:28: note:     def [T: Callable[..., None]] f2(x: T) -> T
Found 2 errors in 1 file (checked 1 source file)

Copy link
Contributor

Diff from mypy_primer, showing the effect of this PR on open source code:

hydra-zen (https://github.com/mit-ll-responsible-ai/hydra-zen)
- src/hydra_zen/structured_configs/_implementations.py:1137: note:     def [Importable] builds(cls, type[Builds[Importable]] | type[BuildsWithSig[Importable, Any]], /, *pos_args: T, zen_partial: Literal[False] | None = ..., populate_full_signature: bool = ..., zen_wrappers: Builds[Callable[[Callable[..., Any]], Callable[..., Any]]] | ZenPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]] | HydraPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]] | Just[Callable[[Callable[..., Any]], Callable[..., Any]]] | type[Builds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[ZenPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[HydraPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[Just[Callable[[Callable[..., Any]], Callable[..., Any]]]] | Callable[[Callable[..., Any]], Callable[..., Any]] | str | None | Sequence[Builds[Callable[[Callable[..., Any]], Callable[..., Any]]] | ZenPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]] | HydraPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]] | Just[Callable[[Callable[..., Any]], Callable[..., Any]]] | type[Builds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[ZenPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[HydraPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[Just[Callable[[Callable[..., Any]], Callable[..., Any]]]] | Callable[[Callable[..., Any]], Callable[..., Any]] | str | None] = ..., zen_meta: Mapping[str, SupportedPrimitive] | None = ..., hydra_recursive: bool | None = ..., hydra_convert: Literal['none', 'partial', 'all', 'object'] | None = ..., hydra_defaults: list[str | DataClass_ | type[DataClass_] | Mapping[str, str | Sequence[str] | None]] | None = ..., dataclass_name: str | None = ..., builds_bases: tuple[type[DataClass_], ...] = ..., zen_dataclass: DataclassOptions | None = ..., frozen: bool = ..., zen_convert: ZenConvert | None = ..., **kwargs_for_target: T) -> type[Builds[Importable]]
+ src/hydra_zen/structured_configs/_implementations.py:1137: note:     def [Importable: Callable[..., Any]] builds(cls, type[Builds[Importable]] | type[BuildsWithSig[Importable, Any]], /, *pos_args: T, zen_partial: Literal[False] | None = ..., populate_full_signature: bool = ..., zen_wrappers: Builds[Callable[[Callable[..., Any]], Callable[..., Any]]] | ZenPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]] | HydraPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]] | Just[Callable[[Callable[..., Any]], Callable[..., Any]]] | type[Builds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[ZenPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[HydraPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[Just[Callable[[Callable[..., Any]], Callable[..., Any]]]] | Callable[[Callable[..., Any]], Callable[..., Any]] | str | None | Sequence[Builds[Callable[[Callable[..., Any]], Callable[..., Any]]] | ZenPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]] | HydraPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]] | Just[Callable[[Callable[..., Any]], Callable[..., Any]]] | type[Builds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[ZenPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[HydraPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[Just[Callable[[Callable[..., Any]], Callable[..., Any]]]] | Callable[[Callable[..., Any]], Callable[..., Any]] | str | None] = ..., zen_meta: Mapping[str, SupportedPrimitive] | None = ..., hydra_recursive: bool | None = ..., hydra_convert: Literal['none', 'partial', 'all', 'object'] | None = ..., hydra_defaults: list[str | DataClass_ | type[DataClass_] | Mapping[str, str | Sequence[str] | None]] | None = ..., dataclass_name: str | None = ..., builds_bases: tuple[type[DataClass_], ...] = ..., zen_dataclass: DataclassOptions | None = ..., frozen: bool = ..., zen_convert: ZenConvert | None = ..., **kwargs_for_target: T) -> type[Builds[Importable]]
- src/hydra_zen/structured_configs/_implementations.py:1137: note:     def [Importable] builds(cls, Importable, /, *pos_args: T, zen_partial: Literal[False] | None = ..., populate_full_signature: bool = ..., zen_wrappers: Builds[Callable[[Callable[..., Any]], Callable[..., Any]]] | ZenPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]] | HydraPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]] | Just[Callable[[Callable[..., Any]], Callable[..., Any]]] | type[Builds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[ZenPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[HydraPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[Just[Callable[[Callable[..., Any]], Callable[..., Any]]]] | Callable[[Callable[..., Any]], Callable[..., Any]] | str | None | Sequence[Builds[Callable[[Callable[..., Any]], Callable[..., Any]]] | ZenPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]] | HydraPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]] | Just[Callable[[Callable[..., Any]], Callable[..., Any]]] | type[Builds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[ZenPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[HydraPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[Just[Callable[[Callable[..., Any]], Callable[..., Any]]]] | Callable[[Callable[..., Any]], Callable[..., Any]] | str | None] = ..., zen_meta: Mapping[str, SupportedPrimitive] | None = ..., hydra_recursive: bool | None = ..., hydra_convert: Literal['none', 'partial', 'all', 'object'] | None = ..., hydra_defaults: list[str | DataClass_ | type[DataClass_] | Mapping[str, str | Sequence[str] | None]] | None = ..., dataclass_name: str | None = ..., builds_bases: tuple[type[DataClass_], ...] = ..., zen_dataclass: DataclassOptions | None = ..., frozen: bool = ..., zen_convert: ZenConvert | None = ..., **kwargs_for_target: T) -> type[Builds[Importable]]
+ src/hydra_zen/structured_configs/_implementations.py:1137: note:     def [Importable: Callable[..., Any]] builds(cls, Importable, /, *pos_args: T, zen_partial: Literal[False] | None = ..., populate_full_signature: bool = ..., zen_wrappers: Builds[Callable[[Callable[..., Any]], Callable[..., Any]]] | ZenPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]] | HydraPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]] | Just[Callable[[Callable[..., Any]], Callable[..., Any]]] | type[Builds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[ZenPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[HydraPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[Just[Callable[[Callable[..., Any]], Callable[..., Any]]]] | Callable[[Callable[..., Any]], Callable[..., Any]] | str | None | Sequence[Builds[Callable[[Callable[..., Any]], Callable[..., Any]]] | ZenPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]] | HydraPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]] | Just[Callable[[Callable[..., Any]], Callable[..., Any]]] | type[Builds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[ZenPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[HydraPartialBuilds[Callable[[Callable[..., Any]], Callable[..., Any]]]] | type[Just[Callable[[Callable[..., Any]], Callable[..., Any]]]] | Callable[[Callable[..., Any]], Callable[..., Any]] | str | None] = ..., zen_meta: Mapping[str, SupportedPrimitive] | None = ..., hydra_recursive: bool | None = ..., hydra_convert: Literal['none', 'partial', 'all', 'object'] | None = ..., hydra_defaults: list[str | DataClass_ | type[DataClass_] | Mapping[str, str | Sequence[str] | None]] | None = ..., dataclass_name: str | None = ..., builds_bases: tuple[type[DataClass_], ...] = ..., zen_dataclass: DataclassOptions | None = ..., frozen: bool = ..., zen_convert: ZenConvert | None = ..., **kwargs_for_target: T) -> type[Builds[Importable]]

... (truncated 12 lines) ...

discord.py (https://github.com/Rapptz/discord.py)
- discord/ext/commands/help.py:1237: note:          def [BotT] prepare_help_command(self, Context[BotT], str | None = ..., /) -> Coroutine[Any, Any, None]
+ discord/ext/commands/help.py:1237: note:          def [BotT: Bot | AutoShardedBot] prepare_help_command(self, Context[BotT], str | None = ..., /) -> Coroutine[Any, Any, None]
- discord/ext/commands/help.py:1237: note:          def [BotT] prepare_help_command(self, Context[BotT], str | None, /) -> Coroutine[Any, Any, None]
+ discord/ext/commands/help.py:1237: note:          def [BotT: Bot | AutoShardedBot] prepare_help_command(self, Context[BotT], str | None, /) -> Coroutine[Any, Any, None]
- discord/ext/commands/help.py:1500: note:          def [BotT] prepare_help_command(self, Context[BotT], str | None = ..., /) -> Coroutine[Any, Any, None]
+ discord/ext/commands/help.py:1500: note:          def [BotT: Bot | AutoShardedBot] prepare_help_command(self, Context[BotT], str | None = ..., /) -> Coroutine[Any, Any, None]
- discord/ext/commands/help.py:1500: note:          def [BotT] prepare_help_command(self, Context[BotT], str | None, /) -> Coroutine[Any, Any, None]
+ discord/ext/commands/help.py:1500: note:          def [BotT: Bot | AutoShardedBot] prepare_help_command(self, Context[BotT], str | None, /) -> Coroutine[Any, Any, None]

steam.py (https://github.com/Gobot1234/steam.py)
- steam/gateway.py:825: note:     def [ProtoMsgsT] wait_for(self, *, emsg: EMsg | None, check: Callable[[ProtoMsgsT], bool] = ...) -> Future[ProtoMsgsT]
+ steam/gateway.py:825: note:     def [ProtoMsgsT: ProtobufMessage | Message] wait_for(self, *, emsg: EMsg | None, check: Callable[[ProtoMsgsT], bool] = ...) -> Future[ProtoMsgsT]

streamlit (https://github.com/streamlit/streamlit)
- lib/tests/streamlit/elements/layouts_test.py:581:14: note:     def [F] dialog_decorator(title: F, *, width: Literal['small', 'large'] = ...) -> F
+ lib/tests/streamlit/elements/layouts_test.py:581:14: note:     def [F: Callable[..., None]] dialog_decorator(title: F, *, width: Literal['small', 'large'] = ...) -> F
- lib/tests/streamlit/elements/layouts_test.py:595:14: note:     def [F] dialog_decorator(title: F, *, width: Literal['small', 'large'] = ...) -> F
+ lib/tests/streamlit/elements/layouts_test.py:595:14: note:     def [F: Callable[..., None]] dialog_decorator(title: F, *, width: Literal['small', 'large'] = ...) -> F
- lib/tests/streamlit/runtime/caching/cache_data_api_test.py:356:14: note:     def [F] __call__(self, func: F) -> F
+ lib/tests/streamlit/runtime/caching/cache_data_api_test.py:356:14: note:     def [F: Callable[..., Any]] __call__(self, func: F) -> F

@JelleZijlstra JelleZijlstra merged commit 94109aa into python:master Sep 21, 2024
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Upper bound is missing from generic function signature in error report
2 participants