Replication without decoding #2317

roman-khimov · 2023-04-24T21:12:58Z

Is your feature request related to a problem? Please describe.

I'm always frustrated when... Ah, it's the same frustration as with #2316.

Describe the solution you'd like

But there is one more thing we can do -- try skipping object decoding entirely. Again, maybe not feasible at the moment with APIs we have, but theoretically we may work with the raw encoded object because it's to be serialized in the same way for the PUT operation. This means skipping the decoding stage with all of the associated processing/allocations.

Additional context

#2300, #2178.

It's more lightweight and supports binary copying without additional decode-encode round. Closes #2317. Signed-off-by: Leonard Lyubich <leonard@morphbits.io>

Previously, replication service of storage nodes encoded single object into protocol message N times, where N is a number of nodes to accept new object replica. Based on the fact that if the object is fixed, the request remains unchanged, it makes sense to prepare the message once and send it to all nodes. Refs #2317. Signed-off-by: Leonard Lyubich <leonard@morphbits.io>

Previously, replication service of storage nodes encoded single object into protocol message N times, where N is a number of nodes to accept new object replica. Based on the fact that if the object is fixed, the request remains unchanged, it makes sense to prepare the message once and send it to all nodes. Closes #2316. Refs #2317. Signed-off-by: Leonard Lyubich <leonard@morphbits.io>

It's more lightweight and supports binary copying without additional decode-encode round. Closes #2317. Signed-off-by: Leonard Lyubich <leonard@morphbits.io>

Previously, replication service of storage nodes encoded single object into protocol message N times, where N is a number of nodes to accept new object replica. Based on the fact that if the object is fixed, the request remains unchanged, it makes sense to prepare the message once and send it to all nodes. Closes #2316. Refs #2317. Signed-off-by: Leonard Lyubich <leonard@morphbits.io>

It's more lightweight and supports binary copying without additional decode-encode round. Closes #2317. Signed-off-by: Leonard Lyubich <leonard@morphbits.io>

Previously, replication service of storage nodes encoded single object into protocol message N times, where N is a number of nodes to accept new object replica. Based on the fact that if the object is fixed, the request remains unchanged, it makes sense to prepare the message once and send it to all nodes. Closes #2316. Refs #2317. Signed-off-by: Leonard Lyubich <leonard@morphbits.io>

Previously, storage nodes always sent "ready" objects after client data slicing via NeoFS API `ObjectService.Put` RPC. Recently, NeoFS protocol was extended with `ObjectService.Replicate` RPC allowing to replicate object from one container node to another. New RPC is more efficient, and now used by Policer+Replicator tandem to replicate objects on shortage. Actually, when `ObjectService.Put` server makes initial object save, it does completely the same. In total, when `Put` server represents storage node from the container, it can use `Replicate` RPC for better performance. An additional advantage is the one-time encoding of the protocol message, which is reused for sending to different nodes. This reduces the memory cost of processing each data stream. Refs #2317. Signed-off-by: Leonard Lyubich <leonard@morphbits.io>

NeoFS protocol has been recently extended with new object replication RPC `ObjectService.Replicate` separated from the general-purpose `ObjectService.Put` one. According to API of the new RPC, all physically stored objects are transmitted in one message. Also, replication request and response formats are much simpler than for other operations. Serve new RPC by the storage node app. Requests are served similar to `ObjectService.Put` ones with TTL=1 (local only). Refs #2317. Signed-off-by: Leonard Lyubich <leonard@morphbits.io>

It's more lightweight and supports binary copying without additional decode-encode round. Closes #2317. Signed-off-by: Leonard Lyubich <leonard@morphbits.io>

Previously, replication service of storage nodes encoded single object into protocol message N times, where N is a number of nodes to accept new object replica. Based on the fact that if the object is fixed, the request remains unchanged, it makes sense to prepare the message once and send it to all nodes. Closes #2316. Refs #2317. Signed-off-by: Leonard Lyubich <leonard@morphbits.io>

Previously, storage nodes always sent "ready" objects after client data slicing via NeoFS API `ObjectService.Put` RPC. Recently, NeoFS protocol was extended with `ObjectService.Replicate` RPC allowing to replicate object from one container node to another. New RPC is more efficient, and now used by Policer+Replicator tandem to replicate objects on shortage. Actually, when `ObjectService.Put` server makes initial object save, it does completely the same. In total, when `Put` server represents storage node from the container, it can use `Replicate` RPC for better performance. An additional advantage is the one-time encoding of the protocol message, which is reused for sending to different nodes. This reduces the memory cost of processing each data stream. Refs #2317. Signed-off-by: Leonard Lyubich <leonard@morphbits.io>

Previously, the local object storage engine always decoded the requested objects. In cases where it was necessary to obtain a binary object, an excessive decoding-encoding round was performed. Now you can specify an optional `bytes.Buffer` into which the object will be read. Refs #2316. Refs #2317. Signed-off-by: Leonard Lyubich <leonard@morphbits.io>

Previously, the local object storage engine always decoded the requested objects. In cases where it was necessary to obtain a binary object, an excessive decoding-encoding round was performed. Now `GetBytes` method to access object binary is provided. The method allows to optimize allocations via optional parameter. Refs #2316. Refs #2317. Signed-off-by: Leonard Lyubich <leonard@morphbits.io>

NeoFS protocol has been recently extended with new object replication RPC `ObjectService.Replicate` separated from the general-purpose `ObjectService.Put` one. According to API of the new RPC, all physically stored objects are transmitted in one message. Also, replication request and response formats are much simpler than for other operations. Serve new RPC by the storage node app. Requests are served similar to `ObjectService.Put` ones with TTL=1 (local only). Refs #2317. Signed-off-by: Leonard Lyubich <leonard@morphbits.io>

It's more lightweight and supports binary copying without additional decode-encode round. Closes #2317. Signed-off-by: Leonard Lyubich <leonard@morphbits.io>

Previously, storage nodes always sent "ready" objects after client data slicing via NeoFS API `ObjectService.Put` RPC. Recently, NeoFS protocol was extended with `ObjectService.Replicate` RPC allowing to replicate object from one container node to another. New RPC is more efficient, and now used by Policer+Replicator tandem to replicate objects on shortage. Actually, when `ObjectService.Put` server makes initial object save, it does completely the same. In total, when `Put` server represents storage node from the container, it can use `Replicate` RPC for better performance. An additional advantage is the one-time encoding of the protocol message, which is reused for sending to different nodes. This reduces the memory cost of processing each data stream. Closes #2317. Signed-off-by: Leonard Lyubich <leonard@morphbits.io>