consider allowing a stream to be removed from a pipeline

[wanderview]

Talking with @trevnorris, it sounds like it would be useful to allow a stream to be removed from a pipeline as an optimization.

For example, if a.pipeThrough(b).pipeTo(c), remove b at some point so a goes straight to c.

This optimization is particularly useful if a and c are native C++ implementations and removing b means its not necessary to execute javascript any more.

Potential APIs for this would be b.collapse() or perhaps b could return a special value from its write or read methods signalling it should be removed from the pipe chain.

[tyoshino]

Can you elaborate the use case? Initially we need b but at some point b becomes unnecessary?

What's going on in piping is basically not observable from the streams API. Which of the objects tells the timing to collapse b?

[wanderview]

The example I was provided was a processor which decrypts cookies in a network stream. Once the cookies have passed they want only the native C++ stream processors operating for the rest of the http response body. Bouncing through js for a function that will never do anything forces a de-opt.

[tyoshino]

If the cookie decrypter is designed to become "closed" when it sees the end of headers, we can use pipeTo with the preventCancel parameter set to true to get notified when the decrypter finishes consuming headers, but without cancel()-ing the source. The decrypter needs to have a method to returns unconsumed data written by pipeTo so that the user can write the data to the body receiver first before resuming pipeTo from the source to the body receiver.

This is handled as non-purposeful closure (https://github.com/whatwg/streams/blob/master/reference-implementation/lib/readable-stream.js#L79), so, we might need to change this line, ... or, making the decrypter "closed" may be a wrong approach.

---

Another approach:

The decrypter starts exerting backpressure not to receive data from the network stream anymore. cookieDecrypter.ready becomes fulfilled, and then the user cancels the pipeTo. Then, re-pipeTo the body receiver after giving the unused data retrieved from cookieDecrypter.

[wanderview]

@trevnorris, does the above address your use case?