leverage lazy bytestring for 'Serialise' CBOR-in-CBOR.

  The main use case for this type lies within the StateQuery protocol, where it can be particularly useful to obtain a plain CBOR response. This is particularly useful for large query results such as DebugNewEpochState (multiple GB on mainnet). The network library makes a great effort at trying to serialise and deserialise bytes lazily throughout; and these efforts are unfortunately destroyed by this implementation that would here evaluate the entire ByteString when decoding and encoding. So on a machine that would have both a server and client using this library, we would pay twice the cost of fully evaluating in memory the entire response, instead of leverage lazy IO as, I believe, is originally intended.

  Note that 'decode' and 'encode' here rely on the default implementation for lazy ByteString in Codec.Serialise, which do the right thing: encode lazy bytestrings as indefinite sequences of byte chunks; effectively preserving the laziness in both directions.

Author: KtorZ

ouroboros-network-api/src/Ouroboros/Network/Block.hs

@@ -473,16 +473,13 @@ fromSerialised dec (Serialised payload) =
 --
 -- TODO: replace with encodeEmbeddedCBOR from cborg-0.2.4 once
 -- it is available, since that will be faster.
---
--- TODO: Avoid converting to a strict ByteString, as that requires copying O(n)
--- in case the lazy ByteString consists of more than one chunks.
 instance Serialise (Serialised a) where
   encode (Serialised bs) = mconcat [
         Enc.encodeTag 24
-      , Enc.encodeBytes (Lazy.toStrict bs)
+	encode bs
       ]
 
   decode = do
       tag <- Dec.decodeTag
       when (tag /= 24) $ fail "expected tag 24 (CBOR-in-CBOR)"
-      Serialised . Lazy.fromStrict <$> Dec.decodeBytes
+      Serialised <$> decode