Skip to content

Switch messages encoding to binary hex - Phase 1 #2536

Open
@jpraynaud

Description

@jpraynaud

Why

The JSON Hex encoding we use is not very efficient in terms of compression. We want to progressively replace it with a binary encoding in the messages.

What

Replace the JSON Hex encoding to binary encoding in the messages without sudden breaking changes:

  • Certificate chain sent to client and aggregator
  • Epoch settings sent to signer
  • Signer registrations sent to aggregator via REST API
  • Signature registrations sent to aggregator:
    • Via REST API
    • Via DMQ

How

  • Phase 1: prepare nodes to support binary decoding:
    • Implement IntoBytes/TryFromBytes in the protocol objects
    • Add from_bytes_hex and to_bytes_hex in the ProtocolKey
    • Implement a fallback decoder to binary encoding if JSON Hex encoding fails in the ProtocolKey (ProtocolKeyCodec) on the binary encoding
    • Send signer registrations and signature registrations in binary encoding in signer
    • Store new signer registrations in binary encoding in the signer database (previous will be pruned automatically, no need for migration)
    • Run backward compatibility test
    • Add golden tests for bytes encoding
    • Assess compression gain with new encoding on StmSig and StmAggrSig

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions