Description
A generator is a syntactic way to declare a function that can yield. Yielding will give a value to the caller of the next() method of the generator, and will suspend execution at the yield point. A generator also supports yield *
which means that it will delegate to another generator and yield the results that the inner generator yields. yield
and yield *
are also bi-directional. A value can flow in as well as out.
Like an iterator, the thing returned by the next method has a done property and a value property. Yielding sets done to false, and returning sets done to true.
A generator is also iterable. You can iterate over the yielded values of the generator, using for-of, spread or array destructuring. However, only yielded values come out when you use a generator in this way. Returned values are never exposed. As a result, this proposal only considers the value type of next() when the done property is false, since those are the ones that will normally be observed.
Basic support for generators
Type annotation on a generator
A generator function can have a return type annotation, just like a function. The annotation represents the type of the generator returned by the function. Here is an example:
function *g(): Iterable<string> {
for (var i = 0; i < 100; i++) {
yield ""; // string is assignable to string
}
yield * otherStringGenerator(); // otherStringGenerator must be iterable and element type assignable to string
}
Here are the rules:
The type annotation must be assignable to.Iterable<any>
- This has been revised:
IterableIterator<any>
must be assignable to the type annotation instead.
- This has been revised:
- The operand of every yield expression (if present) must be assignable to the element type of the generator (string in this case)
- The operand of every
yield *
expression must be assignable toIterable<any>
- The element type of the operand of every
yield *
expression must be assignable to the element type of the generator. (string is assignable to string) - The operand of a
yield
(if present) expression is contextually typed by the element type of the generator (string) - The operand of a
yield *
expression is contextually typed by the type of the generator (Iterable<string>
) - A
yield
expression has type any. - A
yield *
expression has type any. The generator is allowed to have return expressions as well, but they are ignored for the purposes of type checking the generator type.The generator cannot have return expressions- Open question: Do we want to give an error for a return expression that is not assignable to the element type? If so, we would also contextually type it by the element type.
- Answer: we will give an error on all return expressions in a generator. Consider relaxing this later.
- Open question: Should we allow void generators?
- Answer: no
Inferring the type of a generator
A generator function with no type annotation can have the type annotation inferred. So in the following case, the type will be inferred from the yield statements:
function *g() {
for (var i = 0; i < 100; i++) {
yield ""; // infer string
}
yield * otherStringGenerator(); // infer element type of otherStringGenerator
}
- Rather than inferring Iterable, we will infer IterableIterator, with some element type. The reason is that someone can call next directly on the generator without first getting its iterator. A generator is in fact an iterator as well as an iterable.
- The element type is the common supertype of all the yield operands and the element types of all the
yield *
operands. - It is an error if there is no common supertype.
- As before, the operand of every
yield *
expression must be assignable toIterable<any>
yield
andyield *
expressions again have type any- If the generator is contextually typed, the operands of
yield
expressions are contextually typed by the element type of the contextual type - If the generator is contextually typed, the operands of
yield *
expressions are contextually typed by the contextual type. Again, return expressions are allowed, but not used for inferring the element type.Return expressions are not allowed. Consider relaxing this later, particularly if there is no type annotation.- Open question: Should we give an error for return expressions not assignable to element type (same as the question above)
- Answer: no return expressions.
- If there are no yield operands and no
yield *
expressions, what should the element type be?- Answer: implicit any
The *
type constructor
Since the Iterable type will be used a lot, it is a good opportunity to add a syntactic form for iterable types. We will use T*
to mean Iterable<T>
, much the same as T[]
is Array<T>
. It does not do anything special, it's just a shorthand. It will have the same grammatical precedence as []
.
Question: Should it be an error to use *
type if you are compiling below ES6.
The good things about this design is that it is super easy to create an iterable by declaring a generator function. And it is super easy to consume it like you would any other type of iterable.
function *g(limit) {
for (var i = 0; i < limit; i++) {
yield i;
}
}
for (let i of g(100)) {
console.log(i);
}
var array = [...g(50)];
var [first, second, ...rest] = g(100);
Drawbacks of this basic design
- The type returned by a call to next is not always correct if the generator has a return expression.
function *g() {
yield 0;
return "";
}
var instance = g();
var x = instance.next().value; // x is number, correct
var x2 = instance.next().value; // x2 is given type number, but it's actually a string!
This implies that maybe we should give an error when return expressions are not assignable to the element type. Though if we do, there is no way out.
2. The types of yield
and yield *
expressions are just any. Many users will not care about these, but the type of the yield
expression is useful if for example, you are implementing await on top of yield.
3. If you type your generator with the *
type, it does not allow someone to call next directly on the generator. Instead they must cast the generator or get the iterator from the generator.
function *g(): number* {
yield 0;
}
var gen = g();
gen.next(); // Error, but allowed in ES6 (preferred in fact)
(<IterableIterator<number>>gen).next(); // works, but really ugly
gen[Symbol.iterator]().next(); // works, but pretty ugly as well
To clarify, issue 3 is not an issue for for-of, spread, and destructuring. It is only an issue for direct calls to next. The good thing is that you can get around this by either leaving off the type annotation from the generator, or by typing it as an IterableIterator.
Advanced additions to proposal
To help alleviate issue 2, we can introduce a nominal Generator type (already in es6.d.ts today). It is an interface, but the compiler would have a special understanding of its type arguments. It would look something like this:
interface Generator<TYield, TReturn, TNext> extends IterableIterator<TYield /*| TReturn*/> {
next(n: TNext): IteratorResult<TYield /*|TReturn*/>;
// throw and return methods elided
}
Notice that TReturn is not used in the type, but it will have special meaning if you are using something that is nominally a Generator. Use of the Generator type annotation is purely optional. The reason that we need to omit TReturn in the next method is so that Generator can be assignable to IterableIterator<TYield>
. Note that this means issue 1 still remains.
- The type of a
yield
expression will be the type of TNext
function *g(): Generator<number, any, string> {
var x = yield 0; // x has type string
}
- If the user does not specify the Generator type annotation, then consuming a yield expression as an expression will be an implicit any. Yield expression statements will be unaffected.
- For a return expression not assignable to the yield type of the generator, we can give an error (require a type annotation) or we can infer
Generator<TYield, TReturn, any>
?
function *g() {
yield 0;
return ""; // Error or infer TReturn as string
}
Once we have TReturn in place, the following rules are added:
- If the operand of
yield *
is a Generator, then theyield *
expression has the type TReturn (the second type argument of that generator) - If the operand of a
yield *
is a Generator, and theyield *
expression is inside a Generator, TNext of the outer generator must be assignable to TNext of the inner one.
function *g1(): Generator<any, any, string> {
var t = yield * g2(); // Error that string is not assignable to number
}
function *g2(): Generator<any, any, number> {
var s = yield 0;
}
- If the operand of
yield *
is not a Generator, and theyield *
is used as an expression, it will be an implicit any.
Ok, now for issue 1, the incorrectness of next. There is no great way to do this. But one idea, courtesy of @CyrusNajmabadi, is to use TReturn in the body of the Generator interface, so that it looks like this:
interface Generator<TYield, TReturn, TNext> extends IterableIterator<TYield> {
next(n: TNext): IteratorResult<TYield | TReturn>;
// throw and return methods elided
}
As it is, Generator will not be assignable to IterableIterator<TYield>
. To make it assignable, we would change assignability so that every time we assign Generator<TYield, TReturn, TNext>
to something, assignability changes this to Generator<TYield, any, TNext>
for the purposes of the assignment. This is very easy to do in the compiler.
When we do this, we get the following result:
function *g() {
yield 0;
return "";
}
var g1 = g();
var x1 = g1.next().value; // number | string (was number with old typing)
var x2 = g1.next().value; // number | string (was number with old typing, and should be string)
var g2: Iterator<number> = g(); // Assignment is allowed by special rule!
var x3 = g2.next(); // number, correct
var x4 = g2.next(); // number, should be string
So you lose the correctness of next when you subsume the generator into an iterable/iterator. But you at least get general correctness when you are using it raw, as a generator.
Additionally, operators like for-of, spread, and destructuring would just get TYield, and would be unaffected by this addition, including if they are done on a Generator.
Thank you to everyone who helped come up with these ideas.