Proposal: Variadic Kinds -- Give specific types to variadic functions #5453
Description
Variadic Kinds
Give Specific Types to Variadic Functions
This proposal lets Typescript give types to higher-order functions that take a variable number of parameters.
Functions like this include concat, apply, curry, compose and almost any decorator that wraps a function.
In Javascript, these higher-order functions are expected to accept variadic functionsas arguments.
With the ES2015 and ES2017 standards, this use will become even more common as programmers start using spread arguments and rest parameters for both arrays and objects.
This proposal addresses these use cases with a single, very general typing strategy based on higher-order kinds.
This proposal would completely or partially address several issues, including:
- Tuples as types for rest ...arguments #5331 -- Tuples as types for rest ...arguments
- Compiler incorrectly reports parameter/call target signature mismatch when using the spread operator #4130 -- Compiler incorrectly reports parameter/call target signature mismatch when using the spread operator
- Tuples should be clonable with Array.prototype.slice() #4988 -- Tuples should be clonable with Array.prototype.slice()
- Thoughts about variadic generics? #1773 -- Variadic generics?
- Rest type parameter in generics for use with intersection types #3870 -- Rest types in generics for intersection types.
- bind(), call(), and apply() are untyped #212 -- bind, call and apply are untyped (requires Supporting 'this' type #3694's this-function types).
- Typed ...rest parameters with generics #1024 -- Typed ...rest parameters with generics
I'll be updating this proposal on my fork of the Typescript-Handbook: sandersn/TypeScript-Handbook@76f5a75
I have an in-progress implementation at sandersn/TypeScript@f3c327a which currently has the simple parts of the proposal implemented.
It supercedes part 2 of my previous proposal, #5296.
Edit: Added a section on assignability. I'm no longer sure that it strictly supercedes #5296.
Preview example with curry
curry for functions with two arguments is simple to write in Javascript and Typescript:
function curry(f, a) {
return b => f(a, b);
}and in Typescript with type annotations:
function curry<T, U, V>(f: (t: T, u: U) => V, a:T): (b:U) => V {
return b => f(a, b);
}However, a variadic version is easy to write in Javascript but cannot be given a type in TypeScript:
function curry(f, ...a) {
return ...b => f(...a, ...b);
}Here's an example of using variadic kinds from this proposal to type curry:
function curry<...T,...U,V>(f: (...ts: [...T, ...U]) => V, ...as:...T): (...bs:...U) => V {
return ...b => f(...a, ...b);
}The syntax for variadic tuple types that I use here matches the spread and rest syntax used for values in Javascript.
This is easier to learn but might make it harder to distinguish type annotations from value expressions.
Similarly, the syntax for concatenating looks like tuple construction, even though it's really concatenation of two tuple types.
Now let's look at an example call to curry:
function f(n: number, m: number, s: string, c: string): [number, number, string, string] {
return [n,m,s,c];
}
let [n,m,s,c] = curry(f, 1, 2)('foo', 'x');
let [n,m,s,c] = curry(f, 1, 2, 'foo', 'x')();In the first call,
V = [number, number, string, string]
...T = [number, number]
...U = [string, string]
In the second call,
V = [number, number, string, string]
...T = [number, number, string, string]
...U = []
Syntax
The syntax of a variadic kind variable is ...T where T is an identifier that is by convention a single upper-case letter, or T followed by a PascalCase identifier.
Variadic kind variables can be used in a number of syntactic contexts:
Variadic kinds can be bound in the usual location for type parameter binding, including functions and classes:
function f<...T,...U>() {}
}
class C<...T> {
}And they can be referenced in any type annotation location:
function makeTuple<...T>(ts:...T): ...T {
return ts;
}
function f<...T,...U>(ts:...T): [...T,...U] {
// note that U is constrained to [string,string] in this function
let us: ...U = makeTuple('hello', 'world');
return [...ts, ...us];
}Variadic kind variables, like type variables, are quite opaque.
They do have one operation, unlike type variables.
They can be concatenated with other kinds or with actual tuples.
The syntax used for this is identical to the tuple-spreading syntax, but in type annotation location:
let t1: [...T,...U] = [...ts,...uProducer<...U>()];
let t2: [...T,string,string,...U,number] = [...ts,'foo','bar',...uProducer<...U>(),12];Tuple types are instances of variadic kinds, so they continue to appear wherever type annotations were previously allowed:
function f<...T>(ts:...T): [...T,string,string] {
// note the type of `us` could have been inferred here
let us: [string,string] = makeTuple('hello', 'world');
return [...ts, ...us];
}
let tuple: [number, string] = [1,'foo'];
f<[number,string]>(tuple);Semantics
A variadic kind variable represents a tuple type of any length.
Since it represents a set of types, we use the term 'kind' to refer to it, following its use in type theory.
Because the set of types it represents is tuples of any length, we qualify 'kind' with 'variadic'.
Therefore, declaring a variable of variadic tuple kind allows it to take on any single tuple type.
Like type variables, kind variables can only be declared as parameters to functions, classes, etc, which then allows them to be used inside the body:
function f<...T>(): ...T {
let a: ...T;
}Calling a function with arguments typed as a variadic kind will assign a specific tuple type to the kind:
f([1,2,"foo"]);Assigns the tuple type ...T=[number,number,string]...T. So in this application off,let a:...Tis instantiated aslet a:[number,number,string]. However, because the type ofais not known when the function is written, the elements of the tuple cannot be referenced in the body of the function. Only creating a new tuple froma` is allowed.
For example, new elements can be added to the tuple:
function cons<H,...Tail>(head: H, tail: ...Tail): [H,...Tail] {
return [head, ...tail];
}
let l: [number, string, string, boolean];
l = cons(1, cons("foo", ["baz", false]));Like type variables, variadic kind variables can usually be inferred.
The calls to cons could have been annotated:
l = cons<number,[string,string,boolean]>(1, cons<string,[string,boolean]>("foo", ["baz", false]));For example, cons must infer two variables, a type H and a kind ...Tail.
In the innermost call, cons("foo", ["baz", false]), H=string and ...Tail=[string,boolean].
In the outermost call, H=number and ...Tail=[string, string, boolean].
The types assigned to ...Tail are obtained by typing list literals as tuples -- variables of a tuple type can also be used:
let tail: [number, boolean] = ["baz", false];
let l = cons(1, cons("foo", tail));Additionally, variadic kind variables can be inferred when concatenated with types:
function car<H,...Tail>(l: [H, ...Tail]): H {
let [head, ...tail] = l;
return head;
}
car([1, "foo", false]);Here, the type of l is inferred as [number, string, boolean].
Then H=number and ...Tail=[string, boolean].
Limits on type inference
Concatenated kinds cannot be inferred because the checker cannot guess where the boundary between two kinds should be:
function twoKinds<...T,...U>(total: [...T,string,...U]) {
}
twoKinds("an", "ambiguous", "call", "to", "twoKinds")The checker cannot decide whether to assign
...T = [string,string,string], ...U = [string]...T = [string,string], ...U = [string,string]...T = [string], ...U = [string,string,string]
Some unambiguous calls are a casualty of this restriction:
twoKinds(1, "unambiguous", 12); // but still needs an annotation!The solution is to add type annotations:
twoKinds<[string,string],[string,string]>("an", "ambiguous", "call", "to", "twoKinds");
twoKinds<[number],[number]>(1, "unambiguous", 12);Uncheckable dependencies between type arguments and the function body can arise, as in rotate:
function rotate(l:[...T, ...U], n: number): [...U, ...T] {
let first: ...T = l.slice(0, n);
let rest: ...U = l.slice(n);
return [...rest, ...first];
}
rotate<[boolean, boolean, string], [string, number]>([true, true, 'none', 12', 'some'], 3);This function can be typed, but there is a dependency between n and the kind variables: n === ...T.length must be true for the type to be correct.
I'm not sure whether this is code that should actually be allowed.
Semantics on classes and interfaces
The semantics are the same on classes and interfaces.
TODO: There are probably some class-specific wrinkles in the semantics.
Assignability between tuples and parameter lists
Tuple kinds can be used to give a type to rest arguments of functions inside their scope:
function apply<...T,U>(ap: (...args:...T) => U, args: ...T): U {
return ap(...args);
}
function f(a: number, b: string) => string {
return b + a;
}
apply(f, [1, 'foo']);In this example, the parameter list of f: (a: number, b:string) => string must be assignable to the tuple type instantiated for the kind ...T.
The tuple type that is inferred is [number, string], which means that (a: number, b: string) => string must be assignable to (...args: [number, string]) => string.
As a side effect, function calls will be able to take advantage of this assignability by spreading tuples into rest parameters, even if the function doesn't have a tuple kind:
function g(a: number, ...b: [number, string]) {
return a + b[0];
}
g(a, ...[12, 'foo']);Tuple types generated for optional and rest parameters
Since tuples can't represent optional parameters directly, when a function is assigned to a function parameter that is typed by a tuple kind, the generated tuple type is a union of tuple types.
Look at the type of h after it has been curried:
function curry<...T,...U,V>(cur: (...args:[...T,...U]) => V, ...ts:...T): (...us:...U) => V {
return ...us => cur(...ts, ...us);
}
function h(a: number, b?:string): number {
}
let curried = curry(h, 12);
curried('foo'); // ok
curried(); // okHere ...T=([number] | [number, string]), so curried: ...([number] | [number, string]) => number which can be called as you would expect. Unfortunately, this strategy does not work for rest parameters. These just get turned into arrays:
function i(a: number, b?: string, ...c: boolean[]): number {
}
let curried = curry(i, 12);
curried('foo', [true, false]);
curried([true, false]);Here, curried: ...([string, boolean[]] | [boolean[]]) => number.
I think this could be supported if there were a special case for functions with a tuple rest parameter, where the last element of the tuple is an array.
In that case the function call would allow extra arguments of the correct type to match the array.
However, that seems too complex to be worthwhile.
Extensions to the other parts of typescript
- Typescript does not allow users to write an empty tuple type.
However, this proposal requires variadic kinds to be bindable to a empty tuple.
So Typescript will need to support empty tuples, even if only internally.
Examples
Most of these examples are possible as fixed-argument functions in current Typescript, but with this proposal they can be written as variadic.
Some, like cons and concat, can be written for homogeneous arrays in current Typescript but can now be written for heteregoneous tuples using tuple kinds.
This follows typical Javascript practise more closely.
Return a concatenated type
function cons<H,...T>(head: H, tail:...T): [H, ...T] {
return [head, ...tail];
}
function concat<...T,...U>(first: ...T, ...second: ...U): [...T, ...U] {
return [...first, ...second];
}
cons(1, ["foo", false]); // === [1, "foo", false]
concat(['a', true], 1, 'b'); // === ['a', true, 1, 'b']
concat(['a', true]); // === ['a', true, 1, 'b']
let start: [number,number] = [1,2]; // type annotation required here
cons(3, start); // == [3,1,2]Concatenated type as parameter
function car<H,...T>(l: [H,...T]): H {
let [head, ...tail] = l;
return head;
}
function cdr<H,...T>(l: [H,...T]): ...T {
let [head, ...tail] = l;
return ...tail;
}
cdr(["foo", 1, 2]); // => [1,2]
car(["foo", 1, 2]); // => "foo"Variadic functions as arguments
function apply<...T,U>(f: (...args:...T) => U, args: ...T): U {
return f(...args);
}
function f(x: number, y: string) {
}
function g(x: number, y: string, z: string) {
}
apply(f, [1, 'foo']); // ok
apply(f, [1, 'foo', 'bar']); // too many arguments
apply(g, [1, 'foo', 'bar']); // okfunction curry<...T,...U,V>(f: (...args:[...T,...U]) => V, ...ts:...T): (...us: ...U) => V {
return us => f(...ts, ...us);
}
let h: (...us: [string, string]) = curry(f, 1);
let i: (s: string, t: string) = curry(f, 2);
h('hello', 'world');function compose<...T,U,V>(f: (u:U) => U, g: (ts:...T) => V): (args: ...T) => V {
return ...args => f(g(...args));
}
function first(x: number, y: number): string {
}
function second(s: string) {
}
let j: (x: number, y: number) => void = compose(second, first);
j(1, 2);TODO: Could f return ...U instead of U?
Decorators
function logged<...T,U>(target, name, descriptor: { value: (...T) => U }) {
let method = descriptor.value;
descriptor.value = function (...args: ...T): U {
console.log(args);
method.apply(this, args);
}
}Open questions
- Does the tuple-to-parameter-list assignability story hold up? It's especially shaky around optional and rest parameters.
- Will the inferred type be a union of tuples as with the optional-parameter case? Because
bind,callandapplyare methods defined on Function, their type arguments need to be bound at function-creation time rather than thebindcall site (for example). But this means that functions with overloads can't take or return types specific to their arguments -- they have to be a union of the overload types. Additionally, Function doesn't have a constructor that specifies type arguments directly, so there's really no way provide the correct types tobindet al. TODO: Add an example here. Note that this problem isn't necessarily unique to variadic functions. - Should rest parameters be special-cased to retain their nice calling syntax, even when they are generated from a tuple type? (In this proposal, functions typed by a tuple kind have to pass arrays to their rest parameters, they can't have extra parameters.)
Activity
ivogabe commentedon Oct 31, 2015
+1, this is really useful for functional programming in TypeScript! How would this work with optional or rest arguments? More concrete, can the
composefunction be used on functions with rest arguments or optional arguments?sandersn commentedon Nov 2, 2015
Good point. I think you could assign the smallest allowed tuple type to an optional-param function since tuple are just objects, which allow additional members. But that's not ideal. I'll see if I can figure out the
composeexample and then I'll update the proposal.sandersn commentedon Nov 2, 2015
Actually union types would probably work better. Something like
This still breaks rest parameters, though.
sandersn commentedon Nov 4, 2015
Anders Hejlsberg (@ahejlsberg), you had some ideas how tuple kinds would work, I think.
kitsonk commentedon Nov 5, 2015
So 👍 on this. For information this is related to (and would fulfill) #3870. We have tried to implement a compose type API in TypeScript but are having to work around some of the limitations noted in this proposal. This would certainly solve some of those problems!
It seems though that sometimes you may want to "merge" such tuple types instead of persisting them, especially with something like compose. For example:
Also, in a lot of your examples, you have been using primitives. How would you see something more complex working, especially if there are conflicts?
sandersn commentedon Nov 5, 2015
Unfortunately this proposal as-is does not address #3870 or the type composition, since the only composition operator for tuple kinds is
[T,...U]. You could also write this asT + ...U(which is more indicative of what happens to the types), but #3870 and your type composition library needT & ...U. I think that might be possible, but I need to understand Jason Freeman (@JsonFreeman)'s and Jon (@jbondc)'s ideas from #3870 first. I'll expand the proposal if I can figure out how it should work.Note: I decided to go with the syntax
[...T, ...U]because it looks like the equivalent value spreading syntax, butT + ...Uis more indicative of what's happening with the types. If we end up with both, then+and&might be the operators to use.sccolbert commentedon Nov 5, 2015
Big 👍 on this!
Igorbek commentedon Nov 6, 2015
+1 awesome! It would allow to express such things much more expressive and lightweight.
JsonFreeman commentedon Nov 6, 2015
My point in #3870 seems to be an issue here. Specifically, I worry about inferring type arguments for variadic type parameters.
Type argument inference is a rather complicated process, and it has changed in subtle ways over time. When arguments are matched against parameters in order to infer type type arguments, there are no guarantees about the order in which candidates are inferred, nor how many candidates are inferred (for a given type parameter). This has generally not been a problem because the result surfaced to the user does not (in most cases) expose these details. But if you make a tuple type out of the inference results, it certainly does expose both the order and the count of the inferences. These details were not intended to be observable.
How serious is this? I think it depends on how exactly the inference works. What is the result of the following:
sandersn commentedon Nov 6, 2015
Jon (@jbondc),
-seems like a good idea. I'll keep it in mind but not explore it here, because I think we should introduce new type operators one at a time. Both&and+create new types, but&creates an intersection type whereas+creates a new tuple type (which is why I prefer the syntax[T,...U]instead ofT + ...U, because[T,U]already does this for types).Jason Freeman (@JsonFreeman) l think it's OK to do one of two things with repeated kind parameters:
f(['hello', 1], [1, false]): [string | number, number | boolean]I think real libraries (like the reactive extensions Igor Oleinikov (@Igorbek) linked to) will usually only have one tuple kind parameter so even though neither (1) nor (2) are particularly usable, it shouldn't impact real-world code much.
In the examples above,
curryis the hardest to infer -- you have to skipf: (...args:[...T,...U]) => V, infer...ts:...T, then go back and set...Uto what's left after consuming...Tfromf's parameters.I've started prototyping this (sandersn/TypeScript@1d5725d), but haven't got that far yet. Any idea if that will work?
JsonFreeman commentedon Nov 6, 2015
I would err on the side of disallowing anything where the semantics is not clear (like repeated inferences to the same spreaded type parameter). That allays my concern above as well.
I can't think of a good mechanism for typing curry. As you point out, you have to skip the parameter list of the first function to consume the
...Targument and then see what's left over. There would have to be some policy to postpone inferences to a spreaded type parameter if it's not final in its list. It could get messy.That said, I think this is worth a try. There is high demand for the feature.
sandersn commentedon Nov 6, 2015
I think you would have to skip multiple tuple kinds that occur in the same context (eg top-level like
(...T,string,...U) => Vor concatenated like[...T,...U,...T]). Then you can make multiple passes on the skipped kinds, eliminating already-inferred kinds and re-skipping kinds that are still ambiguous. If at any point no single kind is available for inference, stop and return an error.So, yeah. Complicated.
JsonFreeman commentedon Nov 7, 2015
You may be able to draw inspiration from a similar problem. It is actually somewhat similar to the problem of inferring to a union or intersection. When inferring to a union type that includes a type parameter that is a member of the inference context, as in
function f<T>(x: T | string[]), you don't know whether to infer to T. The intended manifestation of the union type may have beenstring[]. So typescript first infers to all other constituents, and then if no inferences were made, infers to T.In the case of intersection, it's even harder because you may have to split the type of the argument across the different intersection constituents. Typescript doesn't make inferences to intersection types at all.
What if you only allowed spreading tuple if it is the last type in its sequence? So
[string, ...T]would be allowed, but[...T, string]would not be?246 remaining items
AlexAegis commentedon Jun 19, 2020
I don't believe this PR solves this particular issue tbh.
With the PR this works
But this issue would like to see an implementation for this as far as I see:
There is a lot angle brackets there, looks a little redundant.
ahejlsberg commentedon Jun 19, 2020
Sandor (@AlexAegis) I'm not sure I see a lot of value in "rest type parameters" like that. You can already do this:
Don't think we really want a whole new concept (i.e. rest type parameters) just so the square brackets can be avoided in the rare cases where inference can't figure it out.
AlexAegis commentedon Jun 19, 2020
Anders Hejlsberg (@ahejlsberg) I see. I was asking because some libraries (RxJS as mentioned) used workarounds to provide this functionality. But it's finite.
So now they either stick with that, or have the users type the brackets, which is a breaking change, and not that intuitive.
The reason why I used this example is because here it's straightforward that I defined the type of that tuple. One square bracket here, one there
Here it's not so obvious that the tuple refers to that rest parameter if you look at it from the outside
But yes as you said if we let the inference figure it out then these trivial cases are not requiring any modification from the user. But we don't know if they did set them explicitly or not, it's up to them, so it still counts as a breaking change. But that's the lib's concern and not this change's.
That said I just find it odd that if there are rest parameters, defined from the outside one by one, that are a single array on the inside differentiated by
..., cannot be made generic the same way: one by one on the outside, single array on the inside, differentiated by....reverofevil commentedon Jun 19, 2020
clawoflight commentedon Jun 19, 2020
I am of course nowhere close to his caliber, but I respectfully disagree with Anders Hejlsberg (@ahejlsberg) .
In my experience, much of the complexity of typescript comes from the fact that a lot of (interesting and useful to be sure) features are special-cased in as their own concepts.
This complexity is not inherently a function of the number of features though!
Instead, the language could be designed around larger, more overarching concepts from which these special cases could then be trivially deduced, or implemented in the std (type) library.
The most general such concept would of course be to fully implement dependent types, from which everything else could then be derived, but going that far is not necessary:
As C++ and, to a lesser extent, Rust have shown, a few large scale, consistent concepts give you a ton of features for free.
This is similar to what OCaml and Haskell (and I assume F#?) have done on the value level, just on the type level.
Type level programming is nothing to be scared of as long as it is designed into the language instead of tacked on to provide specific features.
The facilities in C++ 14/17 are very intuitive except for their syntax, which is purely due to historical baggage.
reverofevil commentedon Jun 19, 2020
clawoflight commentedon Jun 19, 2020
@polkovnikov-ph I'm glad we agree on the issue at hand :)
As for the solution, I think it would still be worth considering progressively moving towards a more carefully designed type system. Major versions are a thing after all, and the alternative is to end up in the cluster**** that is C++ 20 - an admirable attempt at adding even more nicely designed features on top of 2 layers of previous attempts that cannot be removed, in a syntax that is already not deterministically parseable.
reverofevil commentedon Jun 19, 2020
All of this is off-topic to this thread and is being discussed here. So I'll try to be frank:
It took decades for academia to figure out correct approach to subtyping: mlsub type system was created only 6 years ago, well after TypeScript was first released. It could be that foundation for classes, interfaces, union and intersection types with overarching concepts.
But also remember there are conditional types. I'm not aware of any papers giving them formal semantics, or describing a minimal type system with conditional types with progress/preservation proofs. I believe that might have something to do with scientists still being shy to print their failed attempts. If your proposal assumes those major incompatible versions will be made in 2040's, when academia gets comfortable with conditional types, I can agree.
Otherwise "carefully designed type system" would have to remove conditional types from the language, and I don't think anyone is up to the task of converting 60% of DefinitelyTyped to use whatever alternative is chosen to replace them. (And then do it several more times, because it's not the only issue.)
I'm afraid the only viable solution is to create a separate programming language that would somehow resemble TS, and somehow (not only by being more pleasurable to write code in) lure developers to use it. Ryan was quite vocal recommending this approach for TS improvement previously.
k4b7 commentedon Dec 16, 2020
Could this be used to improve the type definitions for
Array.prototype.map? Right now it returnsT[]even if tuple like[T, T]was passed in.karol-majewski commentedon Dec 16, 2020
@kevinbarabash You can already do that with overloads:
Playground
A similar overload can be added to
ReadonlyArrayso that it works with tuples created withas const.k4b7 commentedon Dec 18, 2020
Karol Majewski (@karol-majewski) thank you. I'm surprised that that snippet handles tuples of different lengths. I need to do some reading on
this: T[] | [T]because I'd expect it only to handle 1-tuples.ExE-Boss commentedon Dec 19, 2020
See also: #36554
TeoTN commentedon Feb 14, 2022
Interestingly, this doesn't work either, I've added a simple example in the playground that fails.
I've got to something similar independently and was surprised that it didn't work :(
Does anyone know what's the way to type
pipein TSv4.5?