Split streams into more features

[MattiasBuelens]

Prompted by the discussion in web-platform-tests/wpt#49829, I had a look at the current list of features in the streams group. Currently, there's only the "base" streams feature and a single async-iterable-streams extension for async iteration. However, there have been several additions over the years since we shipped BYOB support:

• Transferable streams, added to the standard in August 2020 and shipped in Chrome 87 and Firefox 103
• ReadableStream.from(asyncIterable), added to the standard in June 2023 and shipped in Firefox 117
• Transformer.cancel(), added to the standard in September 2023 but not yet shipping in any browser
• ReadableStreamBYOBReader.read(view, {min}), added to the standard in November 2023 and shipped in Firefox 134

I suggest we add the following new features, and move them out of the base streams feature:

• transferable-streams
• readablestream-from
• transformstream-transformer-cancel
  • I've added a new BCD feature for this: Add transformer.cancel method to TransformStream constructor mdn/browser-compat-data#25519
• readablestreambyobreader-read-min

[MattiasBuelens]

@jcscottiii FYI

[captainbrosset]

Given the differences in browser support and ship dates, I agree that it makes sense to split Streams into multiple sub-features. These seem separate enough to me.

That said, I don't have enough experience with streams to know whether that's also how developers think of them.
The key aspect we use to define features on this repo is developer perception.
Some of the questions we try to answer to split things up might be:

• Do web developers think of these features independently from each other?
• Can they use streams already without these later additions?
• Are these later additions improvements over the base feature?
• Is there evidence of these additions being talked about on the web (StackOverflow, blogs, etc.)?

Let us know your thoughts. I'm not saying this isn't the right way to split, merely just leaving thought to provoke conversation.

Beyond that, I made a few tiny description proposals. Also, this should probably be merged again on the latest main branch and dist files should be re-generated.

[MattiasBuelens]

Can they use streams already without these later additions?

Yes, streams have been usable for quite a while, even before these recent additions.

• 2016 - the year of web streams (Jake Archibald) shows that back in 2016, streams were already very usable.
• Streams - The definitive guide on web.dev talks about all the APIs available in the base feature, how to get a stream from fetch(), which other Web APIs provide streams (such as Encoding Streams and Compression Streams)

Are these later additions improvements over the base feature?

Yes, these are all additive features. Generally, the objective has been to make streams easier to use, and better integrated with various parts of the Web Platform:

• Transferable streams enables streams to be passed around between contexts (e.g. main thread and workers), similar to how ArrayBuffers are transferable. You could have done this yourself before (by manually postMessage()ing each chunk), but this makes it much easier to correctly handle backpressure, errors and cancellation.
• ReadableStream.from() makes it easier to create a stream from other data sources, e.g. from an array. You could have done this yourself before (by manually constructing a stream and calling controller.enqueue()), but this makes it a one-liner in most cases.
• Transformer.cancel() adds a way to respond to cancellation for a user-defined TransformStream. You could have done this before with a custom ReadableStream and WritableStream, but the new transformer method is much better at handling all possible edge cases.
• ReadableStreamBYOBReader.read(view, {min}) extends the existing read(view) method. You could have done this before by manually calling read(view) in a loop until the view is sufficiently filled (like in this example), but the new min option makes this easier and enables potential optimizations in the future.

Do web developers think of these features independently from each other?
Is there evidence of these additions being talked about on the web (StackOverflow, blogs, etc.)?

• Transferable streams
  • This is an enabler for WebRTC Insertable Streams, allowing developers to run their processing in a worker thread.
    • In-browser media compositing via WebRTC Insertable Streams API (Yakup Çilesiz)
    • WebRTC samples Breakout Box crop
• ReadableStream.from()
  • Node.js added Readable.toWeb() to convert a Node.js Readable to a Web ReadableStream. With ReadableStream.from(), there's now a more standardized way to do this conversion.
  • Developers on StackOverflow are interested in polyfilling this method.
  • It makes for much more concise example code for streams, see Deno docs for @std/streams.
• Transformer.cancel
  • This arose from a Deno developer working on their TextDecoderStream implementation needing a reliable way to clean up their native decoder when the stream is cancelled or aborted.
  • Developers on StackOverflow are working around the lack of Transform.cancel using an extra wrapper WritableStream. However, as explained in that answer, this workaround doesn't handle all edge cases properly.
• ReadableStreamBYOBReader.read(view, {min})
  • This arose from a Cloudflare developer wanting to provide a more optimized method to read from a byte stream, when the user knows that they'll need at least N amount of bytes from the stream before they can continue useful work.

[captainbrosset]

This looks good to me. Thank you for providing the detailed rationale about these sub-features.

I don't think this needs to block this PR, but I do have one follow-up question: should the parent streams feature be Baseline Widely Available then?
Looking at that feature, it uses compute_from with the api.ReadableStreamBYOBReader BCD key, which makes it not available on Safari. But, reading the Streams guide you provided, it feels like the streams feature should be baseline if the base use cases have been available for a while already.

[MattiasBuelens]

I don't think this needs to block this PR, but I do have one follow-up question: should the parent streams feature be Baseline Widely Available then?
Looking at that feature, it uses compute_from with the api.ReadableStreamBYOBReader BCD key, which makes it not available on Safari. But, reading the Streams guide you provided, it feels like the streams feature should be baseline if the base use cases have been available for a while already.

Personally, I would like to see BYOB become a separate feature. It's a bit weird how e.g. Compression Streams (which builds upon Streams) is Baseline Newly Available, while Streams is only Baseline Limited Available.

There was a discussion regarding BYOB as part of #2358. The decision there was to keep BYOB as part of the base feature, in order to match the status quo on https://caniuse.com/streams. We could consider splitting up the feature in both Baseline and caniuse, but I don't know what the impact of such a split would be on developers using this feature data today. 🤷‍♂️

[ddbeck]

I have a call with Alexis from caniuse scheduled for today. I'm putting this PR on our agenda, to learn more about developer impact of feature splitting on caniuse. I'll report back when I learn something.

[ddbeck]

After some discussion with Alexis, this is what I learned:

• Historically, caniuse has not split features very often, but not never. As he put it, features on caniuse aren't always the last word on how we should perceive those features, so there's nothing in principle that would stop changing an existing feature on caniuse.
• Alexis said he'd be receptive to a PR that splits BYOB into its own feature on caniuse, modifying the existing streams feature to cover the API without BYOB.
• We also discussed the possibility of a transitional phase, where we create a BYOB feature and set the caniuse: streams value on it, until caniuse has been updated to reflect both general Streams and BYOB.

For the purposes of this PR, I'd be open to seeing a proposed BYOB feature that takes the streams mapping.

I'd also appreciate some assistance on contributing the splitting work to caniuse. I think the hardest part would be sifting through caniuse's notes, to determine which notes would belong with which feature (though I would, of course, welcome a complete PR to Fyrd/caniuse to make the split).

[MattiasBuelens]

Thanks @ddbeck! I've implemented your proposal with BYOB as a separate readable-byte-streams feature that maps to caniuse's streams.

• The main streams feature is now computed from ReadableStream + WritableStream + TransformStream, representing the three main classes for streams. This moves the feature to Baseline Widely Available.
• The readable-byte-streams feature has the same status as the original streams feature, remaining at Baseline Limited Availability (due to missing support in Safari).

I'll see if I can find my way around the Caniuse codebase and make a similar split there. 🙂

[ddbeck]

I think this is getting very close to landing. I have some suggestions around names and descriptions in line comments. The major thing I'm wondering about here is the min option and whether that justifies its own feature.

Thanks for your ongoing effort into this!

[ddbeck]

For most features, we try to mention the most likely "entry point" to that feature in the description and then frame the feature in that context—in other words, what would a developer literally do with this feature? That said, I don't know a lot about this feature, other than what I've skimmed from the MDN and web.dev articles—so I'm really leaning on your input here. What do you think of something along these lines?

Also, given our discussion so far, I figure we should mention BYOB here. We have an "also known as" convention for feature names that I've suggested here.

Suggested change

	description: Readable byte streams allow you to efficiently read bytes from a stream into a buffer without incurring extra copies.
	description: A `ReadableStream` constructed with `{ type: "bytes" }` reads bytes from a stream without making extra copies, improving efficiency for streams of large chunks. Also known as BYOB or bring your own buffer.

[ddbeck]

"wraps … as" reads a little funny to me (though I see that MDN uses the same language). What do you think of this instead, which borrows a construction from the iterator-methods feature description?

Suggested change

	description: The `ReadableStream.from()` static method wraps iterable and async iterable objects, such as arrays or async generator functions, as a readable stream.
	description: The `ReadableStream.from()` static method converts an iterable or async iterable object, such as an array or async generator function, into a readable stream.

[ddbeck]

"Between" implies two things and I think making it explicit is nice. Does this wording work for you?

Suggested change

	description: Streams are transferable objects, which can be moved between contexts such as workers.
	description: Streams are transferable objects, which can be moved between contexts such as windows and workers.

[ddbeck]

To align with our naming conventions here:

Suggested change

	name: "TransformStream: transformer.cancel method"
	name: TransformStream transformer cancel() method

[ddbeck]

Style nits:

Suggested change

	description: The `cancel` method of a `TransformStream` transformer cleans up resources when the readable side is cancelled or the writable side is aborted.
	description: The `cancel()` method of a `TransformStream` transformer cleans up resources when the readable side cancels or the writable side aborts.

[ddbeck]

I'm still not sure whether developers think of this as a distinct feature or an embellishment on the BYOB feature above. In other features, we've taken narrow options like this to be "later additions" to an existing feature. Searching Stack Overflow doesn't turn up much for me (though I'm not sure I'm using the right terminology). On the other hand, this is very new, so maybe it's an unsettled question in general.

Basically, how sad would you be if we stuffed this compat_features key back into readable-byte-streams?

(We're working on some ways to improve this in the data model—to make it easier to split and merge things—so this isn't to say it must always be this way, but this the ongoing negotiation we've had to do in some features.)