Determining space needed for serialization

[stevehickman]

As far as I can tell, there is no way to know how much space a particular serialization will take without actually doing it. In embedded or other space constrained situations this might lead to buffer overflows, out of memory errors, and other unpleasantness.

Is it possible to determine how much space a serialization will take without actually doing it? If not, could that be added?

[nabijaczleweli]

There's a function that determines the upper bound of the compressed size, but I take that's not what you're looking for.

[stevehickman]

That's good enough for starters, although knowing the exact size would be better. Where's the doc for the upper bound?

[nabijaczleweli]

Whoops, apologies, I have mistaken this for another repository this issue also fits perfectly, please ignore my earlier comment :/

[AzothAmmo]

Not currently possible but I think this could be easily accomplished. A quick solution would be to have a version of the binary/portable binary archive that, instead of actually performing serialization, just increments a counter to track serialized storage requirements. This unfortunately can't be done at compile time because some serializations are dependent on runtime events (e.g. we don't serialize pointers twice).

I'll give this some more thought and see if a more elegant solution could be rolled into cereal, but the above would definitely work for now with very minimal runtime overhead.

[erichkeane]

Perhaps an easier way would be to make a ostream with a 'null' backing field (or custom base to deal with this), then just going through a 'write' cycle with no buffer. We PERHAPS would have to template the output archives to take an arbitrary class rather than std::ostream, but I suspect it would be possible without much modification.

[erichkeane]

Actually, this guy got a good solution to a similar problem that just replaced the stream-buffer: #316 . Perhaps a custom stream type for this instead?

[dtmoodie]

Any chance anyone has solved and tested something for this? I'd really like to be able to determine the size that I will need for serialization, allocated it as one ZeroMQ message, then serialize directly into that message.

[eclazi]

http://stackoverflow.com/a/41539998/871108 implements a streambuf that only counts as @AzothAmmo AzothAmmo suggested.