EPIC: Extend PBJ to support forward compatibility with unknown fields and round-trip serialization #203
Description
Problem
The BN uses PBJ which currently does not completely support forward compatibility.
While non-strict parsing will allow unknown fields, that data is lost if the parsed object is later serialized.
Suggested Solution
Enhance PBJ to optionally retain unknown fields (or perhaps just the original binary data, since PBJ objects are immutable) as this will allow for the BN to process and reify data it doesn't yet fully understand.
Alternatives
We could use PBJ as-is if we are careful to never parse and re-serialize any block stream data. This is somewhat error-prone, however, and requires very careful (and sometimes very difficult) design. Additionally, it becomes much more difficult to filter a block stream, and we may still encounter issues as the state changes data must be parsed in order to update live states, and that can result in data loss without a secondary byte-level manipulation of the input to extract the data to be stored without a parse-and-serialize round-trip.