Change how we handle Sorted JSON encoding

[ValarDragon]

Currently what we do is inefficient. We marshal a JSON string, and then run MustSortJSON on it. MustSortJSON JSON decodes the input, and then re-JSON encodes, but using the stdlib instead of amino.

Either we should make amino provide a method to JSON sort its input, or we should make a method to just sort the lines of a JSON encoded blob. In a 30 block simulation I ran, it was responsible for about 10% of the time. (A little bit skewed though, since the simulation didn't really hit anything that actually makes the chain slow down. This simulation did however have 525 signature verifications)

---

For Admin Use

• Not duplicate issue

[alexanderbez]

Either we should make amino provide a method to JSON sort its input...

What is the input here? A byte string or an interface?

[ValarDragon]

The sort JSON method takes a JSON encoded byte string.

Amino takes in a concrete struct and returns a JSON encoded byte string. The order currently getting called is effectively:

amino_bz           := amino_json_encode(msg_struct)
decoded_bytes      := stdlib_json_decode(amino_bz)
return_val         := stdlib_json_encode(decoded_bytes)

example: https://github.com/cosmos/cosmos-sdk/blob/develop/x/gov/msgs.go#L132

[tac0turtle]

@alexanderbez is this still applicable with the client encoding changes that will be done in the coming weeks

[alexanderbez]

I imagine this will still be useful for clients that still which to use amino. But we should fix the SDK's (Must)SortJSON to not decode.

[tac0turtle]

reopen if still applicable

[ValarDragon]

This is still relevant.

Most messages sign bytes are implemented of the form

func (msg MsgSetWithdrawAddress) GetSignBytes() []byte {
	bz := ModuleCdc.MustMarshalJSON(&msg)
	return sdk.MustSortJSON(bz)
}

The implementation now does

unsorted_json_bz           := module_cdc_encode(msg_struct)
decoded_bytes      := stdlib_json_decode(unsorted_json_bz)
return_val         := stdlib_json_encode(decoded_bytes)

See

cosmos-sdk/types/utils.go

Lines 29 to 35 in ab24d0e

	func SortJSON(toSortJSON []byte) ([]byte, error) {
	var c interface{}
	err := json.Unmarshal(toSortJSON, &c)
	if err != nil {
	return nil, err
	}
	js, err := json.Marshal(c)

I propose we add a ModuleCDC.MustMarshalSortedJSON method, and switch to using this everywhere applicable. Then for the moduleCDC, we can just make sorted json directly via the stdlib.