Description
In scope of #208, @Totktonada stated
I think we can write a potentially large request chunk-by-chunk to a socket. To achieve that we need to learn our msgpack encoder to work in a lazy manner: receive a stream as an input parameter and return a stream as a result.
My main concern is that we can avoid materializing streams in memory.
JDBC streams support was provided for PreparedStaement in #208. Due to the lack of streams support in IPPROTO as well as MsgPackLite implementation, the driver materializes a stream in memory before it will be encoded and sent to the socket.
Rationale: it's possible to send/receive quite long binary data. (for instance, MP_BIN32 stores a byte array whose length is up to (2^32)-1 bytes == 4Gb). As a result, it will be convenient to push/fetch bytes partially chunk-by-chunk for such enormous data. The InputStream, especially BufferredStream, looks a suitable abstraction over the bytes "on-demand".
There are a few open questions:
Do we support both byte arrays and streams to represent binary data?
Maybe it's worth providing such support only for mp_bin16/32 and for mp_bin8 use byte[].
The current implementation requires a packet must be fully encoded to be sent. And vice versa response, before it will be returned, should be completely read and decoded. How to support laziness here is a big question.
It seems the last point is the most important to be clarified.