Optimization opportunity: encoded_len gets computed twice when encoding a message

[ivoanjo]

Hey 👋! I work at Datadog and we use prost in our libdatadog rust component.

While looking at some performance profiling of our protobuf encoding, I noticed that there's quite a bit of repeated work between encode and encode_raw since both need encoded_len, and it gets computed twice:

Specifically, Message::encode computes the encoded_len to check if the buffer has enough space, and then throws this info away, and then encode_raw gets the number again as it needs to record the field size in the pprof.

Computing encoded_len is around 5% of the time of this benchmark, so there would be a nice gain in encoding speed if it was computed only once.

We're happy users of prost so I decided to report this in the spirit of "see something, say something".

Thanks again for your amazing work so far 🙏

[caspermeijn]

Thanks, that is great that you found this. Do you have a suggestion on how to fix this? Are you willing to create a PR for that?

[ivoanjo]

Thanks, that is great that you found this. Do you have a suggestion on how to fix this?

I would probably go for either introducing a encode_raw_with_len that receives the len as an argument or modifying the existing encode_raw to always then the len as an argument (I'm not sure where else it gets used).

Are you willing to create a PR for that?

The million dollar question 😅. I'm working on a few other bigger optimizations on our codebase first, and I'm not sure when I'll have enough time to circle back to this one.

So I'll keep it on my radar, but it may take a while -- this is why I decided to report, so it wasn't forgotten/and in case someone wanted to pick it up faster than I can :)