Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Protobuf: x/auth & x/supply #5533

Merged
merged 70 commits into from
Feb 18, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
adfae21
Migrate types to use proto
alexanderbez Jan 7, 2020
72f70c0
Add newline
alexanderbez Jan 7, 2020
80ab9d6
Fix rand_util
alexanderbez Jan 7, 2020
bf49ef2
Add nolint
alexanderbez Jan 7, 2020
d9c08cf
Implement amino marshal interface for int, uint, and dec
alexanderbez Jan 7, 2020
c5e5c6b
Update encoding doc
alexanderbez Jan 8, 2020
c1b17a5
Update CircleCI config
alexanderbez Jan 8, 2020
28b2da6
Remove tools target from proto job
alexanderbez Jan 8, 2020
f039b99
Rename to CustomProtobufType
alexanderbez Jan 8, 2020
49aa60d
Merge branch 'master' into bez/5444-types-proto-enc
alexanderbez Jan 8, 2020
cd092c0
Merge branch 'master' into bez/5444-types-proto-enc
alexanderbez Jan 8, 2020
b3463f7
changed proto docker image
Jan 8, 2020
f05929c
changed proto job to skip the binary installs
Jan 8, 2020
52e0a49
removed extra files and update proto CI
Jan 8, 2020
161f23b
Use third_party over vendor
alexanderbez Jan 9, 2020
5098419
Merge branch 'master' into bez/5444-types-proto-enc
alexanderbez Jan 9, 2020
7501af3
Merge PR #5505: Convert Int and Dec to string's on Coin and DecCoin p…
aaronc Jan 13, 2020
5c6c08c
Update go.sum
alexanderbez Jan 13, 2020
2e80b77
Fix protoc-gen-gocosmos target by allowing versioning
alexanderbez Jan 13, 2020
9e6a906
Lint
alexanderbez Jan 13, 2020
33400ef
Update go.{mod, sum} and install
alexanderbez Jan 13, 2020
241eafb
wildcard import
alexanderbez Jan 13, 2020
489d5e0
Merge branch 'master' into bez/5444-types-proto-enc
alexanderbez Jan 13, 2020
01f7766
go mod tidy
alexanderbez Jan 13, 2020
bb4ae4e
Bump cosmos-proto to v0.1.0
alexanderbez Jan 13, 2020
d6182fb
Update go.{mod, sum}
alexanderbez Jan 13, 2020
ba12a87
Update third party proto deps
alexanderbez Jan 15, 2020
153dadd
Define Marshaler and ProtoMarshaler interfaces along with BaseCodec type
alexanderbez Jan 16, 2020
94ca99c
Initial commit/demo
alexanderbez Jan 16, 2020
4113099
Fix MarshalAccount
alexanderbez Jan 17, 2020
7fd26dd
Prep merge from master
alexanderbez Feb 11, 2020
b268466
Merge branch 'master' into bez/5444-auth-proto-enc
alexanderbez Feb 11, 2020
fb6fc91
Update codec pkg
alexanderbez Feb 11, 2020
2d7e8a8
Fix GetAccountWithHeight
alexanderbez Feb 11, 2020
f8c4632
Use codec json over json pkg
alexanderbez Feb 11, 2020
aaeaf2e
Update auth types
alexanderbez Feb 11, 2020
305db41
Update account marshal tests
alexanderbez Feb 11, 2020
31af710
Update vesting types
alexanderbez Feb 11, 2020
c21d5fe
Update module APIs
alexanderbez Feb 11, 2020
769e57f
Update client logic to use pkg-level codec
alexanderbez Feb 11, 2020
c242f1c
Proto
alexanderbez Feb 11, 2020
41773be
Update vesting (remove codec)
alexanderbez Feb 13, 2020
b218b1e
Update auth types
alexanderbez Feb 13, 2020
56f824b
Update proto
alexanderbez Feb 13, 2020
38338fd
Undo account interface naming
alexanderbez Feb 13, 2020
df62b47
Update APIs
alexanderbez Feb 13, 2020
d9dbdd6
Update alias
alexanderbez Feb 13, 2020
6849bcb
Update simapp codec
alexanderbez Feb 13, 2020
e2fccc0
Add missing UnmarshalAccountJSON
alexanderbez Feb 13, 2020
66c8beb
Update simapp
alexanderbez Feb 13, 2020
96e36b4
Migrate x/supply to proto and update Supply interface
alexanderbez Feb 13, 2020
4d49c14
cleanup module codecs
alexanderbez Feb 13, 2020
4b1b6bb
Define SupplyCodec
alexanderbez Feb 13, 2020
f0aab9f
Restructure x/supply
alexanderbez Feb 13, 2020
fd1758f
Supply proto updates
alexanderbez Feb 14, 2020
98de856
Merge branch 'master' into bez/5444-auth-proto-enc
alexanderbez Feb 14, 2020
eebcba3
Refactor Module Manager to use codec
alexanderbez Feb 14, 2020
348a7e1
Update genutil
alexanderbez Feb 14, 2020
c52731d
Fix cal to DefaultGenesis
alexanderbez Feb 14, 2020
0dfd3ef
Lint
alexanderbez Feb 14, 2020
e9d161c
Fix supply tests
alexanderbez Feb 14, 2020
098e3a9
Update changelog
alexanderbez Feb 14, 2020
2c721af
Update changelog
alexanderbez Feb 14, 2020
2482e86
Add error checking
alexanderbez Feb 15, 2020
08502d6
Remove x/auth Account test type and codec
alexanderbez Feb 15, 2020
fe2c0a8
Lint
alexanderbez Feb 18, 2020
85ebbda
Use a single reference to the auth account codec
alexanderbez Feb 18, 2020
9480f1e
Merge branch 'master' into bez/5444-auth-proto-enc
alexanderbez Feb 18, 2020
cb22da9
Rename vesting proto message
alexanderbez Feb 18, 2020
af00a40
Remove vesting codec
alexanderbez Feb 18, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update auth types
  • Loading branch information
alexanderbez committed Feb 11, 2020
commit aaeaf2eb19052d53ced507ac1f303d80c5d68a9c
13 changes: 11 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,15 +75,15 @@ is the latest height, we'll use the store's `lastCommitInfo`. Otherwise, we fetc
* Callers to `NewBaseVestingAccount` are responsible for verifying account balance in relation to
the original vesting amount.
* The `SendKeeper` and `ViewKeeper` interfaces in `x/bank` have been modified to account for changes.
* (staking) [\#5600](https://github.com/cosmos/cosmos-sdk/pull/5600) Migrate the `x/staking` module to use Protocol Buffer for state
* (x/staking) [\#5600](https://github.com/cosmos/cosmos-sdk/pull/5600) Migrate the `x/staking` module to use Protocol Buffer for state
serialization instead of Amino. The exact codec used is `codec.HybridCodec` which utilizes Protobuf for binary encoding and Amino
for JSON encoding.
* `BondStatus` is now of type `int32` instead of `byte`.
* Types of `int16` in the `Params` type are now of type `int32`.
* Every reference of `crypto.Pubkey` in context of a `Validator` is now of type string. `GetPubKeyFromBech32` must be used to get the `crypto.Pubkey`.
* The `Keeper` constructor now takes a `codec.Marshaler` instead of a concrete Amino codec. This exact type
provided is specified by `ModuleCdc`.
* (distr) [\#5610](https://github.com/cosmos/cosmos-sdk/pull/5610) Migrate the `x/distribution` module to use Protocol Buffer for state
* (x/distribution) [\#5610](https://github.com/cosmos/cosmos-sdk/pull/5610) Migrate the `x/distribution` module to use Protocol Buffer for state
serialization instead of Amino. The exact codec used is `codec.HybridCodec` which utilizes Protobuf for binary encoding and Amino
for JSON encoding.
* `ValidatorHistoricalRewards.ReferenceCount` is now of types `uint32` instead of `uint16`.
Expand All @@ -92,6 +92,15 @@ for JSON encoding.
* `ValidatorAccumulatedCommission` is now a struct with `commission`.
* The `Keeper` constructor now takes a `codec.Marshaler` instead of a concrete Amino codec. This exact type
provided is specified by `ModuleCdc`.
* (x/auth) [\#5533](https://github.com/cosmos/cosmos-sdk/pull/5533) Migrate the `x/auth` module to use Protocol Buffer for state
serialization instead of Amino.
* The `BaseAccount.PubKey` field is now represented as a Bech32 string instead of a `crypto.Pubkey`.
* `NewBaseAccountWithAddress` now returns a reference to a `BaseAccount`.
* Rename `Account` interface to `AccountI`.
* The `x/auth` module now accepts an `AuthCodec` which extends the `codec.Marshaler` interface by
requiring a concrete codec to know to serialize accounts.
* The `AccountRetriever` type now accepts an `AuthCodec` in its constructor in order to know how to
serialize accounts.

### Improvements

Expand Down
140 changes: 51 additions & 89 deletions x/auth/types/account.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@ package types

import (
"bytes"
"encoding/json"
"errors"
"fmt"

"github.com/tendermint/tendermint/crypto"
yaml "gopkg.in/yaml.v2"
Expand All @@ -15,38 +15,32 @@ import (
//-----------------------------------------------------------------------------
// BaseAccount

var _ exported.Account = (*BaseAccount)(nil)
var _ exported.AccountI = (*BaseAccount)(nil)
var _ exported.GenesisAccount = (*BaseAccount)(nil)

// BaseAccount - a base account structure.
// This can be extended by embedding within in your AppAccount.
// However one doesn't have to use BaseAccount as long as your struct
// implements Account.
type BaseAccount struct {
Address sdk.AccAddress `json:"address" yaml:"address"`
PubKey crypto.PubKey `json:"public_key" yaml:"public_key"`
AccountNumber uint64 `json:"account_number" yaml:"account_number"`
Sequence uint64 `json:"sequence" yaml:"sequence"`
}

// NewBaseAccount creates a new BaseAccount object
func NewBaseAccount(address sdk.AccAddress, pubKey crypto.PubKey, accountNumber, sequence uint64) *BaseAccount {
var pkStr string
if pubKey != nil {
pkStr = sdk.MustBech32ifyPubKey(sdk.Bech32PubKeyTypeAccPub, pubKey)
}

return &BaseAccount{
Address: address,
PubKey: pubKey,
PubKey: pkStr,
AccountNumber: accountNumber,
Sequence: sequence,
}
}

// ProtoBaseAccount - a prototype function for BaseAccount
func ProtoBaseAccount() exported.Account {
func ProtoBaseAccount() exported.AccountI {
return &BaseAccount{}
}

// NewBaseAccountWithAddress - returns a new base account with a given address
func NewBaseAccountWithAddress(addr sdk.AccAddress) BaseAccount {
return BaseAccount{
func NewBaseAccountWithAddress(addr sdk.AccAddress) *BaseAccount {
return &BaseAccount{
Address: addr,
}
}
Expand All @@ -61,23 +55,40 @@ func (acc *BaseAccount) SetAddress(addr sdk.AccAddress) error {
if len(acc.Address) != 0 {
return errors.New("cannot override BaseAccount address")
}

acc.Address = addr
return nil
}

// GetPubKey - Implements sdk.Account.
func (acc BaseAccount) GetPubKey() crypto.PubKey {
return acc.PubKey
if acc.PubKey == "" {
return nil
}

return sdk.MustGetPubKeyFromBech32(sdk.Bech32PubKeyTypeAccPub, acc.PubKey)
}

// SetPubKey - Implements sdk.Account.
func (acc *BaseAccount) SetPubKey(pubKey crypto.PubKey) error {
acc.PubKey = pubKey
var (
pkStr string
err error
)

if pubKey != nil {
pkStr, err = sdk.Bech32ifyPubKey(sdk.Bech32PubKeyTypeAccPub, pubKey)
if err != nil {
return err
}
}

acc.PubKey = pkStr
return nil
}

// GetAccountNumber - Implements Account
func (acc *BaseAccount) GetAccountNumber() uint64 {
func (acc BaseAccount) GetAccountNumber() uint64 {
return acc.AccountNumber
}

Expand All @@ -88,7 +99,7 @@ func (acc *BaseAccount) SetAccountNumber(accNumber uint64) error {
}

// GetSequence - Implements sdk.Account.
func (acc *BaseAccount) GetSequence() uint64 {
func (acc BaseAccount) GetSequence() uint64 {
return acc.Sequence
}

Expand All @@ -100,90 +111,41 @@ func (acc *BaseAccount) SetSequence(seq uint64) error {

// Validate checks for errors on the account fields
func (acc BaseAccount) Validate() error {
if acc.PubKey != nil && acc.Address != nil &&
!bytes.Equal(acc.PubKey.Address().Bytes(), acc.Address.Bytes()) {
if acc.PubKey != "" && acc.Address != nil &&
!bytes.Equal(acc.GetPubKey().Address().Bytes(), acc.Address.Bytes()) {
return errors.New("pubkey and address pair is invalid")
}

return nil
}

type baseAccountPretty struct {
Address sdk.AccAddress `json:"address" yaml:"address"`
PubKey string `json:"public_key" yaml:"public_key"`
AccountNumber uint64 `json:"account_number" yaml:"account_number"`
Sequence uint64 `json:"sequence" yaml:"sequence"`
}

func (acc BaseAccount) String() string {
out, _ := acc.MarshalYAML()
return out.(string)
out, _ := yaml.Marshal(acc)
return string(out)
}

// MarshalYAML returns the YAML representation of an account.
func (acc BaseAccount) MarshalYAML() (interface{}, error) {
alias := baseAccountPretty{
Address: acc.Address,
AccountNumber: acc.AccountNumber,
Sequence: acc.Sequence,
// SetAccountI sets the Account's oneof sum type to the provided AccountI type.
// The provided AccountI type must be a reference to a BaseAccount.
func (acc *Account) SetAccountI(value exported.AccountI) error {
if value == nil {
acc.Sum = nil
return nil
}

if acc.PubKey != nil {
pks, err := sdk.Bech32ifyPubKey(sdk.Bech32PubKeyTypeAccPub, acc.PubKey)
if err != nil {
return nil, err
}

alias.PubKey = pks
baseAcc, ok := value.(*BaseAccount)
if ok {
acc.Sum = &Account_BaseAccount{baseAcc}
return nil
}

bz, err := yaml.Marshal(alias)
if err != nil {
return nil, err
}

return string(bz), err
}

// MarshalJSON returns the JSON representation of a BaseAccount.
func (acc BaseAccount) MarshalJSON() ([]byte, error) {
alias := baseAccountPretty{
Address: acc.Address,
AccountNumber: acc.AccountNumber,
Sequence: acc.Sequence,
}

if acc.PubKey != nil {
pks, err := sdk.Bech32ifyPubKey(sdk.Bech32PubKeyTypeAccPub, acc.PubKey)
if err != nil {
return nil, err
}

alias.PubKey = pks
}

return json.Marshal(alias)
return fmt.Errorf("failed to encode value of type %T as message AccountI", value)
}

// UnmarshalJSON unmarshals raw JSON bytes into a BaseAccount.
func (acc *BaseAccount) UnmarshalJSON(bz []byte) error {
var alias baseAccountPretty
if err := json.Unmarshal(bz, &alias); err != nil {
return err
// GetAccountI returns an AccountI based on the internal oneof sum type.
func (acc *Account) GetAccountI() exported.AccountI {
if x := acc.GetBaseAccount(); x != nil {
return x
}

if alias.PubKey != "" {
pk, err := sdk.GetPubKeyFromBech32(sdk.Bech32PubKeyTypeAccPub, alias.PubKey)
if err != nil {
return err
}

acc.PubKey = pk
}

acc.Address = alias.Address
acc.AccountNumber = alias.AccountNumber
acc.Sequence = alias.Sequence

return nil
}
5 changes: 2 additions & 3 deletions x/auth/types/account_retriever_test.go
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
package types

import (
"encoding/json"
"errors"
"testing"

Expand All @@ -18,9 +17,9 @@ func TestAccountRetriever(t *testing.T) {
defer mockCtrl.Finish()

mockNodeQuerier := mocks.NewMockNodeQuerier(mockCtrl)
accRetr := NewAccountRetriever(NewCodec(nil), mockNodeQuerier)
accRetr := NewAccountRetriever(ModuleCdc, mockNodeQuerier)
addr := []byte("test")
bs, err := json.Marshal(NewQueryAccountParams(addr))
bs, err := ModuleCdc.MarshalJSON(NewQueryAccountParams(addr))
require.NoError(t, err)

mockNodeQuerier.EXPECT().QueryWithData(gomock.Eq("custom/acc/account"),
Expand Down
25 changes: 0 additions & 25 deletions x/auth/types/account_test.go
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
package types

import (
"encoding/json"
"errors"
"testing"

Expand Down Expand Up @@ -115,27 +114,3 @@ func TestGenesisAccountValidate(t *testing.T) {
})
}
}

func TestBaseAccountJSON(t *testing.T) {
pubkey := secp256k1.GenPrivKey().PubKey()
addr := sdk.AccAddress(pubkey.Address())
baseAcc := NewBaseAccount(addr, pubkey, 10, 50)

bz, err := json.Marshal(baseAcc)
require.NoError(t, err)

bz1, err := baseAcc.MarshalJSON()
require.NoError(t, err)
require.Equal(t, string(bz1), string(bz))

var a BaseAccount
require.NoError(t, json.Unmarshal(bz, &a))
require.Equal(t, baseAcc.String(), a.String())

bz, err = ModuleCdc.MarshalJSON(baseAcc)
require.NoError(t, err)

var b BaseAccount
require.NoError(t, ModuleCdc.UnmarshalJSON(bz, &b))
require.Equal(t, baseAcc.String(), b.String())
}
10 changes: 5 additions & 5 deletions x/auth/types/codec.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ type Codec struct {
}

func NewCodec(amino *codec.Codec) *Codec {
return &Codec{Marshaler: codec.NewBaseCodec(amino), amino: amino}
return &Codec{Marshaler: codec.NewHybridCodec(amino), amino: amino}
}

// MarshalAccount marshals an AccountI interface. If the given type implements
Expand Down Expand Up @@ -72,7 +72,7 @@ func (c *Codec) UnmarshalAccountJSON(bz []byte) (exported.AccountI, error) {
//
// NOTE: This codec is deprecated, where a codec via NewCodec without an Amino
// codec should be used.
var ModuleCDC = NewCodec(codec.New())
var ModuleCdc = NewCodec(codec.New())

// RegisterCodec registers concrete types on the codec
func RegisterCodec(cdc *codec.Codec) {
Expand All @@ -85,10 +85,10 @@ func RegisterCodec(cdc *codec.Codec) {
// RegisterAccountTypeCodec registers an external account type defined in
// another module for the internal ModuleCdc.
func RegisterAccountTypeCodec(o interface{}, name string) {
ModuleCDC.amino.RegisterConcrete(o, name, nil)
ModuleCdc.amino.RegisterConcrete(o, name, nil)
}

func init() {
RegisterCodec(ModuleCDC.amino)
codec.RegisterCrypto(ModuleCDC.amino)
RegisterCodec(ModuleCdc.amino)
codec.RegisterCrypto(ModuleCdc.amino)
}
5 changes: 2 additions & 3 deletions x/auth/types/genesis.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ import (
"fmt"
"sort"

"github.com/cosmos/cosmos-sdk/codec"
"github.com/cosmos/cosmos-sdk/x/auth/exported"
)

Expand All @@ -30,7 +29,7 @@ func DefaultGenesisState() GenesisState {

// GetGenesisStateFromAppState returns x/auth GenesisState given raw application
// genesis state.
func GetGenesisStateFromAppState(cdc *codec.Codec, appState map[string]json.RawMessage) GenesisState {
func GetGenesisStateFromAppState(cdc AuthCodec, appState map[string]json.RawMessage) GenesisState {
var genesisState GenesisState
if appState[ModuleName] != nil {
cdc.MustUnmarshalJSON(appState[ModuleName], &genesisState)
Expand Down Expand Up @@ -86,7 +85,7 @@ type GenesisAccountIterator struct{}
// appGenesis and invokes a callback on each genesis account. If any call
// returns true, iteration stops.
func (GenesisAccountIterator) IterateGenesisAccounts(
cdc *codec.Codec, appGenesis map[string]json.RawMessage, cb func(exported.Account) (stop bool),
cdc AuthCodec, appGenesis map[string]json.RawMessage, cb func(exported.AccountI) (stop bool),
) {

for _, genAcc := range GetGenesisStateFromAppState(cdc, appGenesis).Accounts {
Expand Down
Loading