streams: implement Readable.from async iterator utility #27660

guybedford · 2019-05-12T17:05:00Z

This implements a Readable.from utility method for easily converting async iterators into readable streams.

The implementation and approach are taken directly from the toReadable implementation in https://github.com/mcollina/stream-iterators-utils.

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

nodejs-github-bot · 2019-05-12T17:05:03Z

Lite-CI: https://ci.nodejs.org/job/node-test-pull-request-lite-pipeline/3492

devsnek · 2019-05-12T17:22:31Z

fun fact: web streams are also async iterators :P

lib/_stream_readable.js

test/parallel/test-readable-from.js

benjamingr

Added some comments, generally LGTM and 👍

benjamingr · 2019-05-12T17:35:43Z

fun fact: web streams are also async iterators :P

@devsnek oh cool that means you ended up following through and making that change?

doc/api/stream.md

lib/_stream_readable.js

doc/api/stream.md

mcollina

LGTM, awesome work!

benjamingr

This is great, thanks for this and for the changes 🙇

I think this, its counterpart of readable streams being async iterable, once promisify, its counterpart callbackify and the promise warnings mean we finally have a good async story for Node.js

I'm excited :D

doc/api/stream.md

nodejs-github-bot · 2019-05-16T23:12:18Z

CI: https://ci.nodejs.org/job/node-test-pull-request/23151/

nodejs-github-bot · 2019-05-16T23:52:30Z

CI: https://ci.nodejs.org/job/node-test-pull-request/23155/

PR-URL: #27660 Reviewed-By: Gus Caplan <me@gus.host> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net>

Notable changes: * process: * Log errors using `util.inspect` in case of fatal exceptions (Ruben Bridgewater) #27243 * repl: * Add `process.on('uncaughtException')` support (Ruben Bridgewater) #27151 * stream: * Implemented `Readable.from` async iterator utility (Guy Bedford) #27660 * tls: * Expose built-in root certificates (Ben Noordhuis) #26415 * Support `net.Server` options (Luigi Pinca) #27665 * Expose `keylog` event on TLSSocket (Alba Mendez) #27654 * worker: * Added the ability to unshift messages from the `MessagePort` (Anna Henningsen) #27294

Notable changes: * esm: * Added the `--experimental-wasm-modules` flag to support WebAssembly modules (Myles Borins & Guy Bedford) #27659 * process: * Log errors using `util.inspect` in case of fatal exceptions (Ruben Bridgewater) #27243 * repl: * Add `process.on('uncaughtException')` support (Ruben Bridgewater) #27151 * stream: * Implemented `Readable.from` async iterator utility (Guy Bedford) #27660 * tls: * Expose built-in root certificates (Ben Noordhuis) #26415 * Support `net.Server` options (Luigi Pinca) #27665 * Expose `keylog` event on TLSSocket (Alba Mendez) #27654 * worker: * Added the ability to unshift messages from the `MessagePort` (Anna Henningsen) #27294 PR-URL: #27799

felixfbecker · 2019-05-21T19:12:56Z

doc/api/stream.md

+(async function() {
+  for await (const chunk of iterator) {
+    // Handle backpressure on write
+    if (!writeable.write(value))


where does value come from?

Should be chunk, feel free to open a PR.

Added in v12.3.0 from PR nodejs/node#27660

PR-URL: #27660 Reviewed-By: Gus Caplan <me@gus.host> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net>

Notable changes: * crypto: * add support for chacha20-poly1305 for AEAD (chux0519) #24081 * increase maxmem range from 32 to 53 bits (Tobias Nießen) #28799 * deps: * update npm to 6.11.3 (claudiahdz) #29430 * upgrade openssl sources to 1.1.1d (Sam Roberts) #29921 * dns: * remove dns.promises experimental warning (cjihrig) #26592 * fs: * remove experimental warning for fs.promises (Anna Henningsen) #26581 * http: * makes response.writeHead return the response (Mark S. Everitt) #25974 * http2: * makes response.writeHead return the response (Mark S. Everitt) #25974 * n-api: * make func argument of napi\_create\_threadsafe\_function optional (legendecas) #27791 * mark version 5 N-APIs as stable (Gabriel Schulhof) #29401 * implement date object (Jarrod Connolly) #25917 * process: * add --unhandled-rejections flag (Ruben Bridgewater) #26599 * stream: * implement Readable.from async iterator utility (Guy Bedford) #27660 * make Symbol.asyncIterator support stable (Matteo Collina) #26989 PR-URL: #29875

nodejs-github-bot added the stream Issues and PRs related to the stream subsystem. label May 12, 2019

guybedford mentioned this pull request May 12, 2019

Handle next call debouncing mcollina/stream-iterators-utils#2

Merged

guybedford requested a review from mcollina May 12, 2019 17:05

guybedford force-pushed the readable-from branch from e951386 to 41a88b1 Compare May 12, 2019 17:11

devsnek approved these changes May 12, 2019

View reviewed changes