[ty] Splat variadic arguments into parameter list (#18996)

This PR updates our call binding logic to handle splatted arguments.

Complicating matters is that we have separated call bind analysis into
two phases: parameter matching and type checking. Parameter matching
looks at the arity of the function signature and call site, and assigns
arguments to parameters. Importantly, we don't yet know the type of each
argument! This is needed so that we can decide whether to infer the type
of each argument as a type form or value form, depending on the
requirements of the parameter that the argument was matched to.

This is an issue when splatting an argument, since we need to know how
many elements the splatted argument contains to know how many positional
parameters to match it against. And to know how many elements the
splatted argument has, we need to know its type.

To get around this, we now make the assumption that splatted arguments
can only be used with value-form parameters. (If you end up splatting an
argument into a type-form parameter, we will silently pass in its
value-form type instead.) That allows us to preemptively infer the
(value-form) type of any splatted argument, so that we have its arity
available during parameter matching. We defer inference of non-splatted
arguments until after parameter matching has finished, as before.

We reuse a lot of the new tuple machinery to make this happen — in
particular resizing the tuple spec representing the number of arguments
passed in with the tuple length representing the number of parameters
the splat was matched with.

This work also shows that we might need to change how we are performing
argument expansion during overload resolution. At the moment, when we
expand parameters, we assume that each argument will still be matched to
the same parameters as before, and only retry the type-checking phase.
With splatted arguments, this is no longer the case, since the inferred
arity of each union element might be different than the arity of the
union as a whole, which can affect how many parameters the splatted
argument is matched to. See the regression test case in
`mdtest/call/function.md` for more details.

Author: dcreager

crates/ty_ide/src/signature_help.rs

@@ -149,7 +149,7 @@ fn create_signature_details_from_call_signature_details(
             details
                 .argument_to_parameter_mapping
                 .get(current_arg_index)
-                .and_then(|&param_index| param_index)
+                .and_then(|mapping| mapping.parameters.first().copied())
                 .or({
                     // If we can't find a mapping for this argument, but we have a current
                     // argument index, use that as the active parameter if it's within bounds.
@@ -242,11 +242,11 @@ fn find_active_signature_from_details(signature_details: &[CallSignatureDetails]
 
     // First, try to find a signature where all arguments have valid parameter mappings.
     let perfect_match = signature_details.iter().position(|details| {
-        // Check if all arguments have valid parameter mappings (i.e., are not None).
+        // Check if all arguments have valid parameter mappings.
         details
             .argument_to_parameter_mapping
             .iter()
-            .all(Option::is_some)
+            .all(|mapping| mapping.matched)
     });
 
     if let Some(index) = perfect_match {
@@ -261,7 +261,7 @@ fn find_active_signature_from_details(signature_details: &[CallSignatureDetails]
             details
                 .argument_to_parameter_mapping
                 .iter()
-                .filter(|mapping| mapping.is_some())
+                .filter(|mapping| mapping.matched)
                 .count()
         })?;
 

crates/ty_python_semantic/resources/mdtest/call/function.md

@@ -69,6 +69,246 @@ def _(flag: bool):
     reveal_type(foo())  # revealed: int
 ```
 
+## Splatted arguments
+
+### Unknown argument length
+
+```py
+def takes_zero() -> None: ...
+def takes_one(x: int) -> None: ...
+def takes_two(x: int, y: int) -> None: ...
+def takes_two_positional_only(x: int, y: int, /) -> None: ...
+def takes_two_different(x: int, y: str) -> None: ...
+def takes_two_different_positional_only(x: int, y: str, /) -> None: ...
+def takes_at_least_zero(*args) -> None: ...
+def takes_at_least_one(x: int, *args) -> None: ...
+def takes_at_least_two(x: int, y: int, *args) -> None: ...
+def takes_at_least_two_positional_only(x: int, y: int, /, *args) -> None: ...
+
+# Test all of the above with a number of different splatted argument types
+
+def _(args: list[int]) -> None:
+    takes_zero(*args)
+    takes_one(*args)
+    takes_two(*args)
+    takes_two_positional_only(*args)
+    takes_two_different(*args)  # error: [invalid-argument-type]
+    takes_two_different_positional_only(*args)  # error: [invalid-argument-type]
+    takes_at_least_zero(*args)
+    takes_at_least_one(*args)
+    takes_at_least_two(*args)
+    takes_at_least_two_positional_only(*args)
+
+def _(args: tuple[int, ...]) -> None:
+    takes_zero(*args)
+    takes_one(*args)
+    takes_two(*args)
+    takes_two_positional_only(*args)
+    takes_two_different(*args)  # error: [invalid-argument-type]
+    takes_two_different_positional_only(*args)  # error: [invalid-argument-type]
+    takes_at_least_zero(*args)
+    takes_at_least_one(*args)
+    takes_at_least_two(*args)
+    takes_at_least_two_positional_only(*args)
+```
+
+### Fixed-length tuple argument
+
+```py
+def takes_zero() -> None: ...
+def takes_one(x: int) -> None: ...
+def takes_two(x: int, y: int) -> None: ...
+def takes_two_positional_only(x: int, y: int, /) -> None: ...
+def takes_two_different(x: int, y: str) -> None: ...
+def takes_two_different_positional_only(x: int, y: str, /) -> None: ...
+def takes_at_least_zero(*args) -> None: ...
+def takes_at_least_one(x: int, *args) -> None: ...
+def takes_at_least_two(x: int, y: int, *args) -> None: ...
+def takes_at_least_two_positional_only(x: int, y: int, /, *args) -> None: ...
+
+# Test all of the above with a number of different splatted argument types
+
+def _(args: tuple[int]) -> None:
+    takes_zero(*args)  # error: [too-many-positional-arguments]
+    takes_one(*args)
+    takes_two(*args)  # error: [missing-argument]
+    takes_two_positional_only(*args)  # error: [missing-argument]
+    takes_two_different(*args)  # error: [missing-argument]
+    takes_two_different_positional_only(*args)  # error: [missing-argument]
+    takes_at_least_zero(*args)
+    takes_at_least_one(*args)
+    takes_at_least_two(*args)  # error: [missing-argument]
+    takes_at_least_two_positional_only(*args)  # error: [missing-argument]
+
+def _(args: tuple[int, int]) -> None:
+    takes_zero(*args)  # error: [too-many-positional-arguments]
+    takes_one(*args)  # error: [too-many-positional-arguments]
+    takes_two(*args)
+    takes_two_positional_only(*args)
+    takes_two_different(*args)  # error: [invalid-argument-type]
+    takes_two_different_positional_only(*args)  # error: [invalid-argument-type]
+    takes_at_least_zero(*args)
+    takes_at_least_one(*args)
+    takes_at_least_two(*args)
+    takes_at_least_two_positional_only(*args)
+
+def _(args: tuple[int, str]) -> None:
+    takes_zero(*args)  # error: [too-many-positional-arguments]
+    takes_one(*args)  # error: [too-many-positional-arguments]
+    takes_two(*args)  # error: [invalid-argument-type]
+    takes_two_positional_only(*args)  # error: [invalid-argument-type]
+    takes_two_different(*args)
+    takes_two_different_positional_only(*args)
+    takes_at_least_zero(*args)
+    takes_at_least_one(*args)
+    takes_at_least_two(*args)  # error: [invalid-argument-type]
+    takes_at_least_two_positional_only(*args)  # error: [invalid-argument-type]
+```
+
+### Mixed tuple argument
+
+```toml
+[environment]
+python-version = "3.11"
+```
+
+```py
+def takes_zero() -> None: ...
+def takes_one(x: int) -> None: ...
+def takes_two(x: int, y: int) -> None: ...
+def takes_two_positional_only(x: int, y: int, /) -> None: ...
+def takes_two_different(x: int, y: str) -> None: ...
+def takes_two_different_positional_only(x: int, y: str, /) -> None: ...
+def takes_at_least_zero(*args) -> None: ...
+def takes_at_least_one(x: int, *args) -> None: ...
+def takes_at_least_two(x: int, y: int, *args) -> None: ...
+def takes_at_least_two_positional_only(x: int, y: int, /, *args) -> None: ...
+
+# Test all of the above with a number of different splatted argument types
+
+def _(args: tuple[int, *tuple[int, ...]]) -> None:
+    takes_zero(*args)  # error: [too-many-positional-arguments]
+    takes_one(*args)
+    takes_two(*args)
+    takes_two_positional_only(*args)
+    takes_two_different(*args)  # error: [invalid-argument-type]
+    takes_two_different_positional_only(*args)  # error: [invalid-argument-type]
+    takes_at_least_zero(*args)
+    takes_at_least_one(*args)
+    takes_at_least_two(*args)
+    takes_at_least_two_positional_only(*args)
+
+def _(args: tuple[int, *tuple[str, ...]]) -> None:
+    takes_zero(*args)  # error: [too-many-positional-arguments]
+    takes_one(*args)
+    takes_two(*args)  # error: [invalid-argument-type]
+    takes_two_positional_only(*args)  # error: [invalid-argument-type]
+    takes_two_different(*args)
+    takes_two_different_positional_only(*args)
+    takes_at_least_zero(*args)
+    takes_at_least_one(*args)
+    takes_at_least_two(*args)  # error: [invalid-argument-type]
+    takes_at_least_two_positional_only(*args)  # error: [invalid-argument-type]
+
+def _(args: tuple[int, int, *tuple[int, ...]]) -> None:
+    takes_zero(*args)  # error: [too-many-positional-arguments]
+    takes_one(*args)  # error: [too-many-positional-arguments]
+    takes_two(*args)
+    takes_two_positional_only(*args)
+    takes_two_different(*args)  # error: [invalid-argument-type]
+    takes_two_different_positional_only(*args)  # error: [invalid-argument-type]
+    takes_at_least_zero(*args)
+    takes_at_least_one(*args)
+    takes_at_least_two(*args)
+    takes_at_least_two_positional_only(*args)
+
+def _(args: tuple[int, int, *tuple[str, ...]]) -> None:
+    takes_zero(*args)  # error: [too-many-positional-arguments]
+    takes_one(*args)  # error: [too-many-positional-arguments]
+    takes_two(*args)
+    takes_two_positional_only(*args)
+    takes_two_different(*args)  # error: [invalid-argument-type]
+    takes_two_different_positional_only(*args)  # error: [invalid-argument-type]
+    takes_at_least_zero(*args)
+    takes_at_least_one(*args)
+    takes_at_least_two(*args)
+    takes_at_least_two_positional_only(*args)
+
+def _(args: tuple[int, *tuple[int, ...], int]) -> None:
+    takes_zero(*args)  # error: [too-many-positional-arguments]
+    takes_one(*args)  # error: [too-many-positional-arguments]
+    takes_two(*args)
+    takes_two_positional_only(*args)
+    takes_two_different(*args)  # error: [invalid-argument-type]
+    takes_two_different_positional_only(*args)  # error: [invalid-argument-type]
+    takes_at_least_zero(*args)
+    takes_at_least_one(*args)
+    takes_at_least_two(*args)
+    takes_at_least_two_positional_only(*args)
+
+def _(args: tuple[int, *tuple[str, ...], int]) -> None:
+    takes_zero(*args)  # error: [too-many-positional-arguments]
+    takes_one(*args)  # error: [too-many-positional-arguments]
+    takes_two(*args)  # error: [invalid-argument-type]
+    takes_two_positional_only(*args)  # error: [invalid-argument-type]
+    takes_two_different(*args)
+    takes_two_different_positional_only(*args)
+    takes_at_least_zero(*args)
+    takes_at_least_one(*args)
+    takes_at_least_two(*args)  # error: [invalid-argument-type]
+    takes_at_least_two_positional_only(*args)  # error: [invalid-argument-type]
+```
+
+### Argument expansion regression
+
+This is a regression that was highlighted by the ecosystem check, which shows that we might need to
+rethink how we perform argument expansion during overload resolution. In particular, we might need
+to retry both `match_parameters` _and_ `check_types` for each expansion. Currently we only retry
+`check_types`.
+
+The issue is that argument expansion might produce a splatted value with a different arity than what
+we originally inferred for the unexpanded value, and that in turn can affect which parameters the
+splatted value is matched with.
+
+The first example correctly produces an error. The `tuple[int, str]` union element has a precise
+arity of two, and so parameter matching chooses the first overload. The second element of the tuple
+does not match the second parameter type, which yielding an `invalid-argument-type` error.
+
+The third example should produce the same error. However, because we have a union, we do not see the
+precise arity of each union element during parameter matching. Instead, we infer an arity of "zero
+or more" for the union as a whole, and use that less precise arity when matching parameters. We
+therefore consider the second overload to still be a potential candidate for the `tuple[int, str]`
+union element. During type checking, we have to force the arity of each union element to match the
+inferred arity of the union as a whole (turning `tuple[int, str]` into `tuple[int | str, ...]`).
+That less precise tuple type-checks successfully against the second overload, making us incorrectly
+think that `tuple[int, str]` is a valid splatted call.
+
+If we update argument expansion to retry parameter matching with the precise arity of each union
+element, we will correctly rule out the second overload for `tuple[int, str]`, just like we do when
+splatting that tuple directly (instead of as part of a union).
+
+```py
+from typing import overload
+
+@overload
+def f(x: int, y: int) -> None: ...
+@overload
+def f(x: int, y: str, z: int) -> None: ...
+def f(*args): ...
+
+# Test all of the above with a number of different splatted argument types
+
+def _(t: tuple[int, str]) -> None:
+    f(*t)  # error: [invalid-argument-type]
+
+def _(t: tuple[int, str, int]) -> None:
+    f(*t)
+
+def _(t: tuple[int, str] | tuple[int, str, int]) -> None:
+    # TODO: error: [invalid-argument-type]
+    f(*t)
+```
+
 ## Wrong argument type
 
 ### Positional argument, positional-or-keyword parameter