Support reading bytes into buffers allocated by user code on platforms where only async read is available

[domenic]

Had a thought-provoking conversation with @piscisaureus today. He works on libuv and was giving me their perspective on how they do I/O and how it interfaces with Node.js's streams.

He pointed out that resources like files and pipes shared with other processes (seekable resources, he called them) don't have the epoll + read interface that ReadableByteStream's current design is based on. In Node.js, they do blocking I/O in a thread pool with pre-allocated buffers.

This works for a ready + read() JS interface, since then the implementation can pre-allocate (say) a 1 MiB buffer, read into it off-thread using blocking I/O, then fulfill the ready promise. When JS calls read(), it just returns the pre-allocated buffer.

It doesn't work as well with ReadableByteStream's ready + readInto(ab, offset, sizeDesired) interface, since we don't know what size buffer to allocate until too late. If we pre-allocate something too small, readInto will always be returning a number below the desired size, which is a bit user-hostile. If we pre-allocate too large, we are at the least wasting memory, and at the worst getting ourselves into a bunch of trouble as we need to figure out how to merge the "leftovers" with other chunks we read in the future.

(I am assuming here it is easy in V8 to map the [offset, offset + max(sizeRead, sizeDesired)] portion of ab onto a given C++ backing buffer.)

A readIntoAsync(ab, offset, desiredSize) model would work a bit better, since then we'd know to pre-allocate a buffer of size desiredSize. But, I was curious if you guys had any other thoughts? Did you think of this issue when designing the API, @tyoshino? I imagine @willchan could be helpful here too.

[tyoshino]

If the blocking I/O:

• accepts a buffer
• based on the size of the given buffer, controls how much data it pulls from the source

Then, we want to allow the user code to allocate a buffer with size determined based on their status. To realize this, yes, we need some API like readIntoAsync(). But between ready + read() and ready + readInto(ab, offset, sizeDesired), I don't think there's so much difference. ready + read() doesn't know what size is appropriate. 1MiB may be too big or too small. The only disadvantage of ready + readInto() for this kind of I/O is an extra copy (internal buffer to user allocated buffer).

In what form can we provide readIntoAsync() together with read()? To allow sync read() we have on the ReadableStream, we need to pull data from the source at least when we're in the "waiting" state. Otherwise, we never enter "readable" state. But this spoils the readIntoAsync()'s benefit. We'll have some data already pulled (to serve for read() user) when readIntoAsync() is called.

So, we need to add some interface to suppress automatic pull when we want to use readIntoAsync() style reading?

---

If the blocking I/O:

• tells us how much data is readable on unblocking
• and then, accepts a buffer

Then, ready + read()'s

• pros: can avoid using too small buffer (includes cost of leftover merging as you say, etc.). can avoid allocating too big buffer.
• cons: the user cannot limit size to read. all available data will be passed to the user code.

(I assumed that the unblocking event leads to fulfillment of ready, and read() allocates a buffer with suitable size and populates it)

For this case, there's no big difference between readIntoAsync() and readInto(), I think.

[domenic]

Sorry for the delay on this. I had a hard time paging it back in.

Unfortunately I think it is the first case:

• accepts a buffer
• based on the size of the given buffer, controls how much data it pulls from the source

This can be seen by looking at e.g. Win32 ReadFile or Unix fread and pread.

In what form can we provide readIntoAsync() together with read()? To allow sync read() we have on the ReadableStream, we need to pull data from the source at least when we're in the "waiting" state. Otherwise, we never enter "readable" state. But this spoils the readIntoAsync()'s benefit. We'll have some data already pulled (to serve for read() user) when readIntoAsync() is called.

Yes, I agree, this is a problem. The only thing I can think of immediately is to remove sync read() and the whole waiting/readable state transition :(. Assuming that we would carry that over to ReadableStream that would be a lot of churn this late in the game which is frustrating. Maybe we can think of a better solution. E.g. maybe SeekableReadableByteStream only matches ReadableStream in its pipeTo API? :-/

[tyoshino]

Shall we add a new mode to ReadableByteStream, say, manual feed mode, named after manual paper feed of printers.

ReadableByteStream would have a method named .feedArrayBuffer(arrayBuffer, offset, size). Via this method, the user feeds a tuple of an ArrayBuffer, offset and size. The tuples are queued and used by the ReadableByteStream to create the result ArrayBuffers to return from the .read() method. How to interpret the size argument is up to the API that returns the ReadableByteStream. The state and read() are kept the same as ReadableStream but ArrayBuffer feeding happens asynchronous to them.

[tyoshino]

PR #282 is quick prototype of the manual feedable stream.

[wanderview]

Warning: naming bikeshed ahead... feel free to ignore.

I think an interface for manual allocation might be useful, but not sure the "feedable" metaphor resonates. I've seen this kind of thing called an "allocation strategy" other places, etc.

It seems this would definitely be a power user feature, though. So lets make sure its optional?

[domenic]

@tyoshino this is sounding promising. I think I need a bit more detail to understand how it's supposed to work though. Especially how it interacts with readInto. I think I like the general approach though, of a more-specialized variant of ReadableByteStream that helps with this case.

Would love if you could flesh out the PR with some examples of consumer code, using read/readInto/feedArrayBuffer.

As for naming, I don't have any strong preferences. "Strategy" is kind of already taken, but something about allocation might be nice.

[tyoshino]

wanderview: I agree that this kind of stuff is called allocation something or allocator. I used feeding since it's not passive (asked to allocate) but the user tells it to pull by putting a new ArrayBuffer via the feedArrayBuffer() method. I don't care so much about the naming but that's my feeling. Feedable might be misleading. ManualArrayBufferFeeding... is the most descriptive name I come up with but too long.

[wanderview]

Could we use the revealing constructor pattern here again? Something like:

var rbs = new ReadableByteStream({
  allocate: function() { return new ArrayBuffer(myPreferredSize); }
});

Then we don't need a new stream name. :-)

If we add a free hook, then you could build a block pool allocator strategy:

// use ping pong buffers
var blocks = [];
blocks[0] = new ArrayBuffer(4096);
blocks[1] = new ArrayBuffer(4096);

var rbs = new ReadableByteStream({
  allocate: function() { return blocks.pop(); },
  free: function(buf) { return blocks.push(buf); }
});

If we don't like confusing these attributes with the constructor "source" concept, then it could be set in a different method. Something like:

var rbs = new ReadableByteStream(/*...*/);

// use ping pong buffers
var blocks = [];
blocks[0] = new ArrayBuffer(4096);
blocks[1] = new ArrayBuffer(4096);

rbs.setAllocator({
  allocate: function() { return blocks.pop(); },
  free: function(buf) { return blocks.push(buf); }
});

Would this address the problem?

[domenic]

I don't think so, unless I'm misunderstanding. The scenario I think we're trying to solve is that the consumer of the stream wants to be able to do "fill up count bytes of ab starting at offset" and then the stream will use this to do an async (i.e., blocking in another thread) operation like fread.

I guess setAllocator would do that, since the consumer gets access to it. The revealing constructor would not though, because it assumes the consumer is the same as the creator of the stream, which is rarely the case.

[domenic]

What would be most helpful for me to understand for all these proposed ideas is, how would a consumer---given a (Feedable?)ReadableByteStream from the UA---use the stream.

For example, if we took a drastic approach and overhauled everything so that readInto and read became async, it would be pretty clear:

// Passes ab, 0, CHUNK_SIZE to fread, which uses it
rbs.readInto(ab, 0, CHUNK_SIZE).then(...)

// Uses the default chunk size, currently specified by the stream constructor (although we could allow it to be specified by consumers, hmm).
rbs.read().then(...)

What would it look like for the feedBuffer and setAllocator situations?