[red-knot] Add __init__ arguments check when doing try_call on a class literal (astral-sh#16512)

## Summary

* Addresses #16511 for simple cases where only `__init__` method is
bound on class or doesn't exist at all.
* fixes a bug with argument counting in bound method diagnostics

Caveats:
* No handling of `__new__` or modified `__call__` on metaclass.
* This leads to a couple of false positive errors in tests

## Test Plan

- A couple new cases in mdtests
- cargo nextest run -p red_knot_python_semantic --no-fail-fast

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
Co-authored-by: David Peter <sharkdp@users.noreply.github.com>

Author: 3 people

crates/red_knot_python_semantic/resources/mdtest/annotations/new_types.md

@@ -8,7 +8,11 @@ Currently, red-knot doesn't support `typing.NewType` in type annotations.
 from typing_extensions import NewType
 from types import GenericAlias
 
+X = GenericAlias(type, ())
 A = NewType("A", int)
+# TODO: typeshed for `typing.GenericAlias` uses `type` for the first argument. `NewType` should be special-cased
+# to be compatible with `type`
+# error: [invalid-argument-type] "Object of type `NewType` cannot be assigned to parameter 2 (`origin`) of function `__new__`; expected type `type`"
 B = GenericAlias(A, ())
 
 def _(

crates/red_knot_python_semantic/resources/mdtest/call/constructor.md

@@ -1,7 +1,325 @@
 # Constructor
 
+When classes are instantiated, Python calls the meta-class `__call__` method, which can either be
+customized by the user or `type.__call__` is used.
+
+The latter calls the `__new__` method of the class, which is responsible for creating the instance
+and then calls the `__init__` method on the resulting instance to initialize it with the same
+arguments.
+
+Both `__new__` and `__init__` are looked up using full descriptor protocol, but `__new__` is then
+called as an implicit static, rather than bound method with `cls` passed as the first argument.
+`__init__` has no special handling, it is fetched as bound method and is called just like any other
+dunder method.
+
+`type.__call__` does other things too, but this is not yet handled by us.
+
+Since every class has `object` in it's MRO, the default implementations are `object.__new__` and
+`object.__init__`. They have some special behavior, namely:
+
+- If neither `__new__` nor `__init__` are defined anywhere in the MRO of class (except for `object`)
+    \- no arguments are accepted and `TypeError` is raised if any are passed.
+- If `__new__` is defined, but `__init__` is not - `object.__init__` will allow arbitrary arguments!
+
+As of today there are a number of behaviors that we do not support:
+
+- `__new__` is assumed to return an instance of the class on which it is called
+- User defined `__call__` on metaclass is ignored
+
+## Creating an instance of the `object` class itself
+
+Test the behavior of the `object` class itself. As implementation has to ignore `object` own methods
+as defined in typeshed due to behavior not expressible in typeshed (see above how `__init__` behaves
+differently depending on whether `__new__` is defined or not), we have to test the behavior of
+`object` itself.
+
+```py
+reveal_type(object())  # revealed: object
+
+# error: [too-many-positional-arguments] "Too many positional arguments to class `object`: expected 0, got 1"
+reveal_type(object(1))  # revealed: object
+```
+
+## No init or new
+
 ```py
 class Foo: ...
 
 reveal_type(Foo())  # revealed: Foo
+
+# error: [too-many-positional-arguments] "Too many positional arguments to bound method `__init__`: expected 0, got 1"
+reveal_type(Foo(1))  # revealed: Foo
+```
+
+## `__new__` present on the class itself
+
+```py
+class Foo:
+    def __new__(cls, x: int) -> "Foo":
+        return object.__new__(cls)
+
+reveal_type(Foo(1))  # revealed: Foo
+
+# error: [missing-argument] "No argument provided for required parameter `x` of function `__new__`"
+reveal_type(Foo())  # revealed: Foo
+# error: [too-many-positional-arguments] "Too many positional arguments to function `__new__`: expected 1, got 2"
+reveal_type(Foo(1, 2))  # revealed: Foo
+```
+
+## `__new__` present on a superclass
+
+If the `__new__` method is defined on a superclass, we can still infer the signature of the
+constructor from it.
+
+```py
+from typing_extensions import Self
+
+class Base:
+    def __new__(cls, x: int) -> Self: ...
+
+class Foo(Base): ...
+
+reveal_type(Foo(1))  # revealed: Foo
+
+# error: [missing-argument] "No argument provided for required parameter `x` of function `__new__`"
+reveal_type(Foo())  # revealed: Foo
+# error: [too-many-positional-arguments] "Too many positional arguments to function `__new__`: expected 1, got 2"
+reveal_type(Foo(1, 2))  # revealed: Foo
+```
+
+## Conditional `__new__`
+
+```py
+def _(flag: bool) -> None:
+    class Foo:
+        if flag:
+            def __new__(cls, x: int): ...
+        else:
+            def __new__(cls, x: int, y: int = 1): ...
+
+    reveal_type(Foo(1))  # revealed: Foo
+    # error: [invalid-argument-type] "Object of type `Literal["1"]` cannot be assigned to parameter 2 (`x`) of function `__new__`; expected type `int`"
+    reveal_type(Foo("1"))  # revealed: Foo
+    # error: [missing-argument] "No argument provided for required parameter `x` of function `__new__`"
+    reveal_type(Foo())  # revealed: Foo
+    # error: [too-many-positional-arguments] "Too many positional arguments to function `__new__`: expected 1, got 2"
+    reveal_type(Foo(1, 2))  # revealed: Foo
+```
+
+## A descriptor in place of `__new__`
+
+```py
+class SomeCallable:
+    def __call__(self, cls, x: int) -> "Foo":
+        obj = object.__new__(cls)
+        obj.x = x
+        return obj
+
+class Descriptor:
+    def __get__(self, instance, owner) -> SomeCallable:
+        return SomeCallable()
+
+class Foo:
+    __new__: Descriptor = Descriptor()
+
+reveal_type(Foo(1))  # revealed: Foo
+# error: [missing-argument] "No argument provided for required parameter `x` of bound method `__call__`"
+reveal_type(Foo())  # revealed: Foo
+```
+
+## A callable instance in place of `__new__`
+
+### Bound
+
+```py
+class Callable:
+    def __call__(self, cls, x: int) -> "Foo":
+        return object.__new__(cls)
+
+class Foo:
+    __new__ = Callable()
+
+reveal_type(Foo(1))  # revealed: Foo
+# error: [missing-argument] "No argument provided for required parameter `x` of bound method `__call__`"
+reveal_type(Foo())  # revealed: Foo
+```
+
+### Possibly Unbound
+
+```py
+def _(flag: bool) -> None:
+    class Callable:
+        if flag:
+            def __call__(self, cls, x: int) -> "Foo":
+                return object.__new__(cls)
+
+    class Foo:
+        __new__ = Callable()
+
+    # error: [call-non-callable] "Object of type `Callable` is not callable (possibly unbound `__call__` method)"
+    reveal_type(Foo(1))  # revealed: Foo
+    # TODO should be - error: [missing-argument] "No argument provided for required parameter `x` of bound method `__call__`"
+    # but we currently infer the signature of `__call__` as unknown, so it accepts any arguments
+    # error: [call-non-callable] "Object of type `Callable` is not callable (possibly unbound `__call__` method)"
+    reveal_type(Foo())  # revealed: Foo
+```
+
+## `__init__` present on the class itself
+
+If the class has an `__init__` method, we can infer the signature of the constructor from it.
+
+```py
+class Foo:
+    def __init__(self, x: int): ...
+
+reveal_type(Foo(1))  # revealed: Foo
+
+# error: [missing-argument] "No argument provided for required parameter `x` of bound method `__init__`"
+reveal_type(Foo())  # revealed: Foo
+# error: [too-many-positional-arguments] "Too many positional arguments to bound method `__init__`: expected 1, got 2"
+reveal_type(Foo(1, 2))  # revealed: Foo
+```
+
+## `__init__` present on a superclass
+
+If the `__init__` method is defined on a superclass, we can still infer the signature of the
+constructor from it.
+
+```py
+class Base:
+    def __init__(self, x: int): ...
+
+class Foo(Base): ...
+
+reveal_type(Foo(1))  # revealed: Foo
+
+# error: [missing-argument] "No argument provided for required parameter `x` of bound method `__init__`"
+reveal_type(Foo())  # revealed: Foo
+# error: [too-many-positional-arguments] "Too many positional arguments to bound method `__init__`: expected 1, got 2"
+reveal_type(Foo(1, 2))  # revealed: Foo
+```
+
+## Conditional `__init__`
+
+```py
+def _(flag: bool) -> None:
+    class Foo:
+        if flag:
+            def __init__(self, x: int): ...
+        else:
+            def __init__(self, x: int, y: int = 1): ...
+
+    reveal_type(Foo(1))  # revealed: Foo
+    # error: [invalid-argument-type] "Object of type `Literal["1"]` cannot be assigned to parameter 2 (`x`) of bound method `__init__`; expected type `int`"
+    reveal_type(Foo("1"))  # revealed: Foo
+    # error: [missing-argument] "No argument provided for required parameter `x` of bound method `__init__`"
+    reveal_type(Foo())  # revealed: Foo
+    # error: [too-many-positional-arguments] "Too many positional arguments to bound method `__init__`: expected 1, got 2"
+    reveal_type(Foo(1, 2))  # revealed: Foo
+```
+
+## A descriptor in place of `__init__`
+
+```py
+class SomeCallable:
+    # TODO: at runtime `__init__` is checked to return `None` and
+    # a `TypeError` is raised if it doesn't. However, apparently
+    # this is not true when the descriptor is used as `__init__`.
+    # However, we may still want to check this.
+    def __call__(self, x: int) -> str:
+        return "a"
+
+class Descriptor:
+    def __get__(self, instance, owner) -> SomeCallable:
+        return SomeCallable()
+
+class Foo:
+    __init__: Descriptor = Descriptor()
+
+reveal_type(Foo(1))  # revealed: Foo
+# error: [missing-argument] "No argument provided for required parameter `x` of bound method `__call__`"
+reveal_type(Foo())  # revealed: Foo
+```
+
+## A callable instance in place of `__init__`
+
+### Bound
+
+```py
+class Callable:
+    def __call__(self, x: int) -> None:
+        pass
+
+class Foo:
+    __init__ = Callable()
+
+reveal_type(Foo(1))  # revealed: Foo
+# error: [missing-argument] "No argument provided for required parameter `x` of bound method `__call__`"
+reveal_type(Foo())  # revealed: Foo
+```
+
+### Possibly Unbound
+
+```py
+def _(flag: bool) -> None:
+    class Callable:
+        if flag:
+            def __call__(self, x: int) -> None:
+                pass
+
+    class Foo:
+        __init__ = Callable()
+
+    # error: [call-non-callable] "Object of type `Callable` is not callable (possibly unbound `__call__` method)"
+    reveal_type(Foo(1))  # revealed: Foo
+    # TODO should be - error: [missing-argument] "No argument provided for required parameter `x` of bound method `__call__`"
+    # but we currently infer the signature of `__call__` as unknown, so it accepts any arguments
+    # error: [call-non-callable] "Object of type `Callable` is not callable (possibly unbound `__call__` method)"
+    reveal_type(Foo())  # revealed: Foo
+```
+
+## `__new__` and `__init__` both present
+
+### Identical signatures
+
+A common case is to have `__new__` and `__init__` with identical signatures (except for the first
+argument). We report errors for both `__new__` and `__init__` if the arguments are incorrect.
+
+At runtime `__new__` is called first and will fail without executing `__init__` if the arguments are
+incorrect. However, we decided that it is better to report errors for both methods, since after
+fixing the `__new__` method, the user may forget to fix the `__init__` method.
+
+```py
+class Foo:
+    def __new__(cls, x: int) -> "Foo":
+        return object.__new__(cls)
+
+    def __init__(self, x: int): ...
+
+# error: [missing-argument] "No argument provided for required parameter `x` of function `__new__`"
+# error: [missing-argument] "No argument provided for required parameter `x` of bound method `__init__`"
+reveal_type(Foo())  # revealed: Foo
+
+reveal_type(Foo(1))  # revealed: Foo
+```
+
+### Compatible signatures
+
+But they can also be compatible, but not identical. We should correctly report errors only for the
+mthod that would fail.
+
+```py
+class Foo:
+    def __new__(cls, *args, **kwargs):
+        return object.__new__(cls)
+
+    def __init__(self, x: int) -> None:
+        self.x = x
+
+# error: [missing-argument] "No argument provided for required parameter `x` of bound method `__init__`"
+reveal_type(Foo())  # revealed: Foo
+reveal_type(Foo(1))  # revealed: Foo
+
+# error: [too-many-positional-arguments] "Too many positional arguments to bound method `__init__`: expected 1, got 2"
+reveal_type(Foo(1, 2))  # revealed: Foo
 ```

crates/red_knot_python_semantic/resources/mdtest/call/subclass_of.md

@@ -20,9 +20,11 @@ class C:
 def _(subclass_of_c: type[C]):
     reveal_type(subclass_of_c(1))  # revealed: C
 
-    # TODO: Those should all be errors
+    # error: [invalid-argument-type] "Object of type `Literal["a"]` cannot be assigned to parameter 2 (`x`) of bound method `__init__`; expected type `int`"
     reveal_type(subclass_of_c("a"))  # revealed: C
+    # error: [missing-argument] "No argument provided for required parameter `x` of bound method `__init__`"
     reveal_type(subclass_of_c())  # revealed: C
+    # error: [too-many-positional-arguments] "Too many positional arguments to bound method `__init__`: expected 1, got 2"
     reveal_type(subclass_of_c(1, 2))  # revealed: C
 ```
 

crates/red_knot_python_semantic/resources/mdtest/generics/classes.md

@@ -111,14 +111,17 @@ class E[T]:
     def __init__(self, x: T) -> None: ...
 
 # TODO: revealed: E[int] or E[Literal[1]]
+# TODO should not emit an error
+# error: [invalid-argument-type] "Object of type `Literal[1]` cannot be assigned to parameter 2 (`x`) of bound method `__init__`; expected type `T`"
 reveal_type(E(1))  # revealed: E
 ```
 
 The types inferred from a type context and from a constructor parameter must be consistent with each
 other:
 
 ```py
-# TODO: error
+# TODO: the error should not leak the `T` typevar and should mention `E[int]`
+# error: [invalid-argument-type] "Object of type `Literal["five"]` cannot be assigned to parameter 2 (`x`) of bound method `__init__`; expected type `T`"
 wrong_innards: E[int] = E("five")
 ```
 

crates/red_knot_python_semantic/resources/mdtest/unary/invert_add_usub.md

@@ -18,7 +18,7 @@ class Number:
     def __invert__(self) -> Literal[True]:
         return True
 
-a = Number()
+a = Number(0)
 
 reveal_type(+a)  # revealed: int
 reveal_type(-a)  # revealed: int