Skip to content

EPIC: Extend PBJ to support forward compatibility with unknown fields and round-trip serialization #203

Open
@Nana-EC

Description

Problem

The BN uses PBJ which currently does not completely support forward compatibility.
While non-strict parsing will allow unknown fields, that data is lost if the parsed object is later serialized.

Suggested Solution

Enhance PBJ to optionally retain unknown fields (or perhaps just the original binary data, since PBJ objects are immutable) as this will allow for the BN to process and reify data it doesn't yet fully understand.

Alternatives

We could use PBJ as-is if we are careful to never parse and re-serialize any block stream data. This is somewhat error-prone, however, and requires very careful (and sometimes very difficult) design. Additionally, it becomes much more difficult to filter a block stream, and we may still encounter issues as the state changes data must be parsed in order to update live states, and that can result in data loss without a secondary byte-level manipulation of the input to extract the data to be stored without a parse-and-serialize round-trip.

Metadata

Assignees

No one assigned

    Labels

    EpicPhase 2Issue/PR related to Phase 2

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions