Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate x/staking to Protobuf #5600

Merged
merged 26 commits into from
Feb 6, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
dec703b
Migrate staking types to proto
alexanderbez Feb 1, 2020
31311cb
Use int32 for validator status
alexanderbez Feb 3, 2020
0d2938b
Update staking types
alexanderbez Feb 3, 2020
60dc367
Add benchmarks for bech32 pubkey funcs
alexanderbez Feb 3, 2020
f0f6f50
Update BondStatus type alias to int32
alexanderbez Feb 3, 2020
c268a9e
Remove PubKey proto type
alexanderbez Feb 3, 2020
95edc79
Fix unit test
alexanderbez Feb 3, 2020
46be4b4
Update staking keeper
alexanderbez Feb 3, 2020
7503396
Fix staking keeper tests
alexanderbez Feb 3, 2020
97ac0b5
Update query client
alexanderbez Feb 3, 2020
8e84801
Update staking genesis and app_test
alexanderbez Feb 3, 2020
c35939a
Update staking handler
alexanderbez Feb 4, 2020
febaae8
Use Int64Value instead of IntProto
alexanderbez Feb 4, 2020
5b83d9b
Updates tests and APIs
alexanderbez Feb 4, 2020
0a904f1
Fix iteration
alexanderbez Feb 4, 2020
2450409
Linting
alexanderbez Feb 4, 2020
97148e2
Fix linting
alexanderbez Feb 4, 2020
da9d51f
Merge branch 'master' into bez/5444-staking-proto-enc
alexanderbez Feb 4, 2020
11a0d8b
Add changelog entry
alexanderbez Feb 4, 2020
db45cfc
Update changelog entry
alexanderbez Feb 4, 2020
c5a7baa
Merge branch 'master' into bez/5444-staking-proto-enc
alexanderbez Feb 4, 2020
3e1b445
Merge branch 'master' into bez/5444-staking-proto-enc
alexanderbez Feb 5, 2020
d31577c
Start boilerplate for app-level codec
alexanderbez Feb 5, 2020
3dcfea6
Merge branch 'master' into bez/5444-staking-proto-enc
alexanderbez Feb 6, 2020
05ceef7
Merge branch 'master' into bez/5444-staking-proto-enc
alexanderbez Feb 6, 2020
c84d960
Lint proto and update codecov settings to ignore proto codegen
alexanderbez Feb 6, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ require (
github.com/cosmos/ledger-cosmos-go v0.11.1
github.com/gogo/protobuf v1.3.1
github.com/golang/mock v1.3.1-0.20190508161146-9fa652df1129
github.com/golang/protobuf v1.3.2
Copy link
Member

@aaronc aaronc Feb 5, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is golang/protobuf required? This likely means there's a directive needed to make a golang/protobuf type point to a gogo/protobuf type.

Copy link
Contributor Author

@alexanderbez alexanderbez Feb 5, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because of our dependency on

import "google/protobuf/timestamp.proto";
import "google/protobuf/duration.proto";

e.g.

message Commission {
  option (gogoproto.equal) = true;
  option (gogoproto.goproto_stringer) = false;

  // ...
  google.protobuf.Timestamp update_time = 2 [
    (gogoproto.nullable) = false,
    (gogoproto.stdtime) = true,
    (gogoproto.moretags) = "yaml:\"update_time\""
  ];
}

In addition, gogo proto itself depends on google/protobuf. Does gogo proto itself provide these? If so, what advantage is there with one over the other?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need this in protocgen.sh:

protoc -I=. -I=$GOPATH/src -I=$GOPATH/src/github.com/gogo/protobuf/protobuf --{binary}_out=\
Mgoogle/protobuf/any.proto=github.com/gogo/protobuf/types,\
Mgoogle/protobuf/duration.proto=github.com/gogo/protobuf/types,\
Mgoogle/protobuf/struct.proto=github.com/gogo/protobuf/types,\
Mgoogle/protobuf/timestamp.proto=github.com/gogo/protobuf/types,\
Mgoogle/protobuf/wrappers.proto=github.com/gogo/protobuf/types:. \
myproto.proto

See https://github.com/gogo/protobuf#more-speed-and-more-generated-code

github.com/gorilla/mux v1.7.3
github.com/hashicorp/golang-lru v0.5.4
github.com/mattn/go-isatty v0.0.12
Expand Down
350 changes: 350 additions & 0 deletions third_party/proto/tendermint/abci/types/types.proto
Original file line number Diff line number Diff line change
@@ -0,0 +1,350 @@
syntax = "proto3";
package tendermint.abci.types;
option go_package = "github.com/tendermint/tendermint/abci/types";

// For more information on gogo.proto, see:
// https://github.com/gogo/protobuf/blob/master/extensions.md
import "third_party/proto/gogoproto/gogo.proto";
import "third_party/proto/tendermint/crypto/merkle/merkle.proto";
import "third_party/proto/tendermint/libs/kv/types.proto";
import "google/protobuf/timestamp.proto";
import "google/protobuf/duration.proto";

// This file is copied from http://github.com/tendermint/abci
// NOTE: When using custom types, mind the warnings.
// https://github.com/gogo/protobuf/blob/master/custom_types.md#warnings-and-issues

option (gogoproto.marshaler_all) = true;
option (gogoproto.unmarshaler_all) = true;
option (gogoproto.sizer_all) = true;
option (gogoproto.goproto_registration) = true;

// Generate tests
option (gogoproto.populate_all) = true;
option (gogoproto.equal_all) = true;
option (gogoproto.testgen_all) = true;

//----------------------------------------
// Request types

message Request {
oneof value {
RequestEcho echo = 2;
RequestFlush flush = 3;
RequestInfo info = 4;
RequestSetOption set_option = 5;
RequestInitChain init_chain = 6;
RequestQuery query = 7;
RequestBeginBlock begin_block = 8;
RequestCheckTx check_tx = 9;
RequestDeliverTx deliver_tx = 19;
RequestEndBlock end_block = 11;
RequestCommit commit = 12;
}
}

message RequestEcho { string message = 1; }

message RequestFlush {}

message RequestInfo {
string version = 1;
uint64 block_version = 2;
uint64 p2p_version = 3;
}

// nondeterministic
message RequestSetOption {
string key = 1;
string value = 2;
}

message RequestInitChain {
google.protobuf.Timestamp time = 1
[ (gogoproto.nullable) = false, (gogoproto.stdtime) = true ];
string chain_id = 2;
ConsensusParams consensus_params = 3;
repeated ValidatorUpdate validators = 4 [ (gogoproto.nullable) = false ];
bytes app_state_bytes = 5;
}

message RequestQuery {
bytes data = 1;
string path = 2;
int64 height = 3;
bool prove = 4;
}

message RequestBeginBlock {
bytes hash = 1;
Header header = 2 [ (gogoproto.nullable) = false ];
LastCommitInfo last_commit_info = 3 [ (gogoproto.nullable) = false ];
repeated Evidence byzantine_validators = 4 [ (gogoproto.nullable) = false ];
}

enum CheckTxType {
New = 0;
Recheck = 1;
}

message RequestCheckTx {
bytes tx = 1;
CheckTxType type = 2;
}

message RequestDeliverTx { bytes tx = 1; }

message RequestEndBlock { int64 height = 1; }

message RequestCommit {}

//----------------------------------------
// Response types

message Response {
oneof value {
ResponseException exception = 1;
ResponseEcho echo = 2;
ResponseFlush flush = 3;
ResponseInfo info = 4;
ResponseSetOption set_option = 5;
ResponseInitChain init_chain = 6;
ResponseQuery query = 7;
ResponseBeginBlock begin_block = 8;
ResponseCheckTx check_tx = 9;
ResponseDeliverTx deliver_tx = 10;
ResponseEndBlock end_block = 11;
ResponseCommit commit = 12;
}
}

// nondeterministic
message ResponseException { string error = 1; }

message ResponseEcho { string message = 1; }

message ResponseFlush {}

message ResponseInfo {
string data = 1;

string version = 2;
uint64 app_version = 3;

int64 last_block_height = 4;
bytes last_block_app_hash = 5;
}

// nondeterministic
message ResponseSetOption {
uint32 code = 1;
// bytes data = 2;
string log = 3;
string info = 4;
}

message ResponseInitChain {
ConsensusParams consensus_params = 1;
repeated ValidatorUpdate validators = 2 [ (gogoproto.nullable) = false ];
}

message ResponseQuery {
uint32 code = 1;
// bytes data = 2; // use "value" instead.
string log = 3; // nondeterministic
string info = 4; // nondeterministic
int64 index = 5;
bytes key = 6;
bytes value = 7;
tendermint.crypto.merkle.Proof proof = 8;
int64 height = 9;
string codespace = 10;
}

message ResponseBeginBlock {
repeated Event events = 1 [
(gogoproto.nullable) = false,
(gogoproto.jsontag) = "events,omitempty"
];
}

message ResponseCheckTx {
uint32 code = 1;
bytes data = 2;
string log = 3; // nondeterministic
string info = 4; // nondeterministic
int64 gas_wanted = 5;
int64 gas_used = 6;
repeated Event events = 7 [
(gogoproto.nullable) = false,
(gogoproto.jsontag) = "events,omitempty"
];
string codespace = 8;
}

message ResponseDeliverTx {
uint32 code = 1;
bytes data = 2;
string log = 3; // nondeterministic
string info = 4; // nondeterministic
int64 gas_wanted = 5;
int64 gas_used = 6;
repeated Event events = 7 [
(gogoproto.nullable) = false,
(gogoproto.jsontag) = "events,omitempty"
];
string codespace = 8;
}

message ResponseEndBlock {
repeated ValidatorUpdate validator_updates = 1
[ (gogoproto.nullable) = false ];
ConsensusParams consensus_param_updates = 2;
repeated Event events = 3 [
(gogoproto.nullable) = false,
(gogoproto.jsontag) = "events,omitempty"
];
}

message ResponseCommit {
// reserve 1
bytes data = 2;
}

//----------------------------------------
// Misc.

// ConsensusParams contains all consensus-relevant parameters
// that can be adjusted by the abci app
message ConsensusParams {
BlockParams block = 1;
EvidenceParams evidence = 2;
ValidatorParams validator = 3;
}

// BlockParams contains limits on the block size.
message BlockParams {
// Note: must be greater than 0
int64 max_bytes = 1;
// Note: must be greater or equal to -1
int64 max_gas = 2;
}

message EvidenceParams {
// Note: must be greater than 0
int64 max_age_num_blocks = 1;
google.protobuf.Duration max_age_duration = 2
[ (gogoproto.nullable) = false, (gogoproto.stdduration) = true ];
}

// ValidatorParams contains limits on validators.
message ValidatorParams { repeated string pub_key_types = 1; }

message LastCommitInfo {
int32 round = 1;
repeated VoteInfo votes = 2 [ (gogoproto.nullable) = false ];
}

message Event {
string type = 1;
repeated tendermint.libs.kv.Pair attributes = 2 [
(gogoproto.nullable) = false,
(gogoproto.jsontag) = "attributes,omitempty"
];
}

//----------------------------------------
// Blockchain Types

message Header {
// basic block info
Version version = 1 [ (gogoproto.nullable) = false ];
string chain_id = 2 [ (gogoproto.customname) = "ChainID" ];
int64 height = 3;
google.protobuf.Timestamp time = 4
[ (gogoproto.nullable) = false, (gogoproto.stdtime) = true ];

// prev block info
BlockID last_block_id = 5 [ (gogoproto.nullable) = false ];

// hashes of block data
bytes last_commit_hash = 6; // commit from validators from the last block
bytes data_hash = 7; // transactions

// hashes from the app output from the prev block
bytes validators_hash = 8; // validators for the current block
bytes next_validators_hash = 9; // validators for the next block
bytes consensus_hash = 10; // consensus params for current block
bytes app_hash = 11; // state after txs from the previous block
bytes last_results_hash =
12; // root hash of all results from the txs from the previous block

// consensus info
bytes evidence_hash = 13; // evidence included in the block
bytes proposer_address = 14; // original proposer of the block
}

message Version {
uint64 Block = 1;
uint64 App = 2;
}

message BlockID {
bytes hash = 1;
PartSetHeader parts_header = 2 [ (gogoproto.nullable) = false ];
}

message PartSetHeader {
int32 total = 1;
bytes hash = 2;
}

// Validator
message Validator {
bytes address = 1;
// PubKey pub_key = 2 [(gogoproto.nullable)=false];
int64 power = 3;
}

// ValidatorUpdate
message ValidatorUpdate {
PubKey pub_key = 1 [ (gogoproto.nullable) = false ];
int64 power = 2;
}

// VoteInfo
message VoteInfo {
Validator validator = 1 [ (gogoproto.nullable) = false ];
bool signed_last_block = 2;
}

message PubKey {
string type = 1;
bytes data = 2;
}

message Evidence {
string type = 1;
Validator validator = 2 [ (gogoproto.nullable) = false ];
int64 height = 3;
google.protobuf.Timestamp time = 4
[ (gogoproto.nullable) = false, (gogoproto.stdtime) = true ];
int64 total_voting_power = 5;
}

//----------------------------------------
// Service Definition

service ABCIApplication {
rpc Echo(RequestEcho) returns (ResponseEcho);
rpc Flush(RequestFlush) returns (ResponseFlush);
rpc Info(RequestInfo) returns (ResponseInfo);
rpc SetOption(RequestSetOption) returns (ResponseSetOption);
rpc DeliverTx(RequestDeliverTx) returns (ResponseDeliverTx);
rpc CheckTx(RequestCheckTx) returns (ResponseCheckTx);
rpc Query(RequestQuery) returns (ResponseQuery);
rpc Commit(RequestCommit) returns (ResponseCommit);
rpc InitChain(RequestInitChain) returns (ResponseInitChain);
rpc BeginBlock(RequestBeginBlock) returns (ResponseBeginBlock);
rpc EndBlock(RequestEndBlock) returns (ResponseEndBlock);
}
31 changes: 31 additions & 0 deletions third_party/proto/tendermint/crypto/merkle/merkle.proto
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
syntax = "proto3";
package tendermint.crypto.merkle;
option go_package = "github.com/tendermint/tendermint/crypto/merkle";

// For more information on gogo.proto, see:
// https://github.com/gogo/protobuf/blob/master/extensions.md
import "third_party/proto/gogoproto/gogo.proto";

option (gogoproto.marshaler_all) = true;
option (gogoproto.unmarshaler_all) = true;
option (gogoproto.sizer_all) = true;

option (gogoproto.populate_all) = true;
option (gogoproto.equal_all) = true;

//----------------------------------------
// Message types

// ProofOp defines an operation used for calculating Merkle root
// The data could be arbitrary format, providing nessecary data
// for example neighbouring node hash
message ProofOp {
string type = 1;
bytes key = 2;
bytes data = 3;
}

// Proof is Merkle proof defined by the list of ProofOps
message Proof {
repeated ProofOp ops = 1 [(gogoproto.nullable)=false];
}
Loading