Repository navigation
Async iterator promises are not resolved after stream end #387
Description
Given the following code, any promises for iterator results after the stream has ended will never resolve.
const a = new stream.PassThrough()
const b = a[Symbol.asyncIterator]()
const c = b.next()
const d = b.next()
a.end()
await c // { done: true, value: null }
await d // never resolvesif you repeat the same process for a generator function
const a = (async function* (){})()
const b = a[Symbol.asyncIterator]()
const c = b.next()
const d = b.next()
a.end()
await c // { value: undefined, done: true }
await d // { value: undefined, done: true }Any promises from calls to next from an exhausted async generator function's iterator will immediately resolve as done: true.
I noticed this difference when trying to perform parallel async transforms on the values from a stream that resolved in a non deterministic order. I rely on all calls to next after the source has been exhausted to resolve, and since they don't for streams it hung forever.
I'm unsure of if this is defined in the async iterator spec, but it does seem like it might want to match behaviors of async generator functions.
Activity
[-]Async iterator promises don't resolve after stream end[/-][+]Async iterator promises doesn't resolve after stream end[/+][-]Async iterator promises doesn't resolve after stream end[/-][+]Async iterator promises are not resolved after stream end[/+]leobalter commentedon Nov 26, 2018
from the ecmascript specs, next iterations should always resolve with the completion information. The behavior of your second example, using an ordinary async generator, is correct and works as expected.
mcollina commentedon Nov 27, 2018
Thanks! keep those coming, so we can move these out of experimental!
mcollina commentedon Nov 27, 2018
Here is the fix: nodejs/node#24668!
reconbot commentedon Nov 27, 2018
Any thoughts on the final value being null? It's not used in for/await/of because it only comes with the end, but I don't know the reasoning behind it.
mcollina commentedon Nov 27, 2018
What do you mean, I'm kind of lost.
Can you please expand on this?
reconbot commentedon Nov 27, 2018
The final iterator value of
{value: null, done: true}doesn't match the async generator function's final value of{value: undefined, done: true}and I was just wondering if that was intentional. It doesn't effect using it in a for/await/of loop. Looks like you figured it out.I have a small library that has a
transformfunction that is a parallel map over an async iterable's values. It's able to work a bit faster than aparallelMapfunction because it doesn't care about the order of the events. It queues up a bunch of promises fornext(), I used a concurrency higher than the number of objects in a stream, which in turn asked fornext()after the stream had ended, which is how I found this bug.stream: make async iterator .next() always resolve
stream: make async iterator .next() always resolve
stream: make async iterator .next() always resolve
stream: make async iterator .next() always resolve
stream: make async iterator .next() always resolve
stream: make async iterator .next() always resolve