Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Simulation is not deterministic due to GenSignedMockTx #12374

Merged
merged 8 commits into from
Jun 30, 2022

Conversation

adu-web3
Copy link
Contributor

@adu-web3 adu-web3 commented Jun 28, 2022

Description

Closes: #12373
we should change GenSignedMockTx function signature to enable passing simState.Rand as an argument.


Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

  • included the correct type prefix in the PR title
  • added ! to the type prefix if API or client breaking change
  • targeted the correct branch (see PR Targeting)
  • provided a link to the relevant issue or specification
  • followed the guidelines for building modules
  • included the necessary unit and integration tests
  • added a changelog entry to CHANGELOG.md
  • included comments for documenting Go code
  • updated the relevant documentation or specification
  • reviewed "Files changed" and left comments if necessary
  • confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

  • confirmed the correct type prefix in the PR title
  • confirmed ! in the type prefix if API or client breaking change
  • confirmed all author checklist items have been addressed
  • reviewed state machine logic
  • reviewed API design and naming
  • reviewed documentation is accurate
  • reviewed tests and test coverage
  • manually tested (if applicable)

@faddat
Copy link
Contributor

faddat commented Jun 28, 2022

Hey just out of curiosity, do you mean to say that you have solve the intermittent failure in CI?

Copy link
Contributor

@alexanderbez alexanderbez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks reasonable to me, but there's no need to assign all the r := rand.New(rand.NewSource(time.Now().UnixNano())) calls to a variable -- just pass it directly to the function call.

@@ -257,6 +259,7 @@ func SignCheckDeliver(
chainID string, accNums, accSeqs []uint64, expSimPass, expPass bool, priv ...cryptotypes.PrivKey,
) (sdk.GasInfo, *sdk.Result, error) {
tx, err := simtestutil.GenSignedMockTx(
rand.New(rand.NewSource(time.Now().UnixNano())),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pass rand.New(rand.NewSource(time.Now().UnixNano())) directly to the function

@@ -307,6 +310,7 @@ func GenSequenceOfTxs(txGen client.TxConfig, msgs []sdk.Msg, accNums []uint64, i
var err error
for i := 0; i < numToGenerate; i++ {
txs[i], err = simtestutil.GenSignedMockTx(
rand.New(rand.NewSource(time.Now().UnixNano())),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pass rand.New(rand.NewSource(time.Now().UnixNano())) directly to the function

@adu-web3
Copy link
Contributor Author

Looks reasonable to me, but there's no need to assign all the r := rand.New(rand.NewSource(time.Now().UnixNano())) calls to a variable -- just pass it directly to the function call.

thanks for the advice.

@adu-web3
Copy link
Contributor Author

adu-web3 commented Jun 29, 2022

Hey just out of curiosity, do you mean to say that you have solve the intermittent failure in CI?

@faddat I think this could fix TestAppStateDeterminism failure.
I would run locally to see if this can help fix two other tests in simulation as well.

update: TestAppStateDeterminism could fail occasionally with log:

simulate.go:302: error on block  90/100, operation (442/670) from x/authz:
  : invalid coins [/Users/adudu/cosmos-sdk/x/bank/types/msgs.go:44]
   Comment: : invalid coins

--- FAIL: TestAppStateDeterminism (57.65s)

but this never happens to release/v0.45.x branch.

update: If we use empty coins slice to create banktype.MsgSend, the ValidateBasic() method would return an invalid coins error. By adding coins slice length checks, we can avoid this error.
Now after the fix, test-sim-import-export test-sim-import-export test-sim-after-import and test-sim-multi-seed-short could run smoothly in local environment.

@adu-web3
Copy link
Contributor Author

@alexanderbez I think it's ready to be merged?

Copy link
Contributor

@alexanderbez alexanderbez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hell yeah! Nice work 👏

@codecov
Copy link

codecov bot commented Jun 30, 2022

Codecov Report

Merging #12374 (5fc1435) into main (1642dcd) will increase coverage by 0.02%.
The diff coverage is 70.73%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main   #12374      +/-   ##
==========================================
+ Coverage   65.66%   65.68%   +0.02%     
==========================================
  Files         676      676              
  Lines       71420    71456      +36     
==========================================
+ Hits        46900    46939      +39     
+ Misses      21886    21879       -7     
- Partials     2634     2638       +4     
Impacted Files Coverage Δ
simapp/test_helpers.go 16.30% <0.00%> (-0.15%) ⬇️
x/simulation/util.go 0.00% <0.00%> (ø)
x/bank/simulation/operations.go 80.32% <40.00%> (-1.53%) ⬇️
x/authz/simulation/operations.go 66.51% <57.14%> (-0.62%) ⬇️
x/distribution/simulation/operations.go 90.42% <100.00%> (+9.67%) ⬆️
x/gov/simulation/operations.go 85.35% <100.00%> (+0.08%) ⬆️
x/group/simulation/operations.go 68.03% <100.00%> (-0.06%) ⬇️
x/nft/simulation/operations.go 68.36% <100.00%> (+0.32%) ⬆️
x/slashing/simulation/operations.go 58.88% <100.00%> (+0.46%) ⬆️
x/staking/simulation/operations.go 75.75% <100.00%> (+0.80%) ⬆️
... and 6 more

@alexanderbez alexanderbez merged commit 17dc431 into cosmos:main Jun 30, 2022
mergify bot pushed a commit that referenced this pull request Jul 1, 2022
(cherry picked from commit 17dc431)

# Conflicts:
#	CHANGELOG.md
#	simapp/helpers/test_helpers.go
#	simapp/test_helpers.go
#	x/authz/simulation/operations.go
#	x/bank/simulation/operations.go
#	x/genutil/gentx_test.go
#	x/gov/simulation/operations.go
#	x/group/simulation/operations.go
#	x/nft/simulation/operations.go
#	x/simulation/util.go
#	x/slashing/simulation/operations.go
likhita-809 pushed a commit that referenced this pull request Jul 14, 2022
(cherry picked from commit 17dc431)

# Conflicts:
#	CHANGELOG.md
#	simapp/helpers/test_helpers.go
#	simapp/test_helpers.go
#	x/authz/simulation/operations.go
#	x/bank/simulation/operations.go
#	x/genutil/gentx_test.go
#	x/gov/simulation/operations.go
#	x/group/simulation/operations.go
#	x/nft/simulation/operations.go
#	x/simulation/util.go
#	x/slashing/simulation/operations.go
randy75828 pushed a commit to Switcheo/cosmos-sdk that referenced this pull request Aug 10, 2022
johnletey added a commit to kyve-org/cosmos-sdk-old that referenced this pull request Aug 25, 2022
* Update tag.yml

* fix: update index of crisis invariant check logs (backport cosmos#12208) (cosmos#12210)

* fix: update index of crisis invariant check logs (cosmos#12208)

## Description

the info log messages describing invariant checks use the index to state
progress (eg. `asserting crisis invariants inv=0/15`). this simple change
makes them 1-indexed (eg. `asserting crisis invariants inv=1/15`).

example before:

<img width="374" alt="Screen Shot 2022-06-09 at 12 06 58 PM" src="https://user-images.githubusercontent.com/14897503/172925006-8810706c-0948-4e36-85b8-22813ccc9311.png">

Closes: #XXXX

---

### Author Checklist

*All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.*

I have...

- [x] included the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [x] added `!` to the type prefix if API or client breaking change - N/A
- [x] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/main/CONTRIBUTING.md#pr-targeting))
- [x] provided a link to the relevant issue or specification - N/A
- [x] followed the guidelines for [building modules](https://github.com/cosmos/cosmos-sdk/blob/main/docs/building-modules) - N/A
- [x] included the necessary unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/main/CONTRIBUTING.md#testing) - N/A
- [x] added a changelog entry to `CHANGELOG.md`
- [x] included comments for [documenting Go code](https://blog.golang.org/godoc) - N/A
- [x] updated the relevant documentation or specification - N/A
- [x] reviewed "Files changed" and left comments if necessary - N/A
- [x] confirmed all CI checks have passed

### Reviewers Checklist

*All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.*

I have...

- [ ] confirmed the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [ ] confirmed `!` in the type prefix if API or client breaking change
- [ ] confirmed all author checklist items have been addressed
- [ ] reviewed state machine logic
- [ ] reviewed API design and naming
- [ ] reviewed documentation is accurate
- [ ] reviewed tests and test coverage
- [ ] manually tested (if applicable)

(cherry picked from commit 907df32)

# Conflicts:
#	CHANGELOG.md

* fix conflict

Co-authored-by: Robert Pirtle <Astropirtle@gmail.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* fix: Refactor GetLastCompletedUpgrade [v0.45.x] (cosmos#12264)

* fix: defaultGenTxGas to 10 times (cosmos#12314)

* fix: edit validator bug from cli (cosmos#12317)

* chore: update release notes (cosmos#12377)

* feat: add query.GenericFilteredPaginated (backport cosmos#12253) (cosmos#12371)

* fix: update x/mint parameter validation (backport cosmos#12384) (cosmos#12396)

* chore: optimize get last completed upgrade (cosmos#12268)

* feat: improve GetLastCompletedUpgrade

* rename

* use var block

* fix: Simulation is not deterministic due to GenTx (backport cosmos#12374) (cosmos#12437)

* fix: use go install instead of go get in makefile (cosmos#12435)

* chore: fumpt sdk v45 series cosmos#12442

* feat: Move AppModule.BeginBlock and AppModule.EndBlock to extension interfaces (backport cosmos#12603) (cosmos#12638)

* feat: Move AppModule.BeginBlock and AppModule.EndBlock to extension interfaces (cosmos#12603)

## Description
Most modules right now have a no-op for AppModule.BeginBlock and AppModule.EndBlock. We should move these methods off the AppModule interface so we have less deadcode, and instead move them to extension interfaces.

1. I added `BeginBlockAppModule` and `EndBlockAppModule` interface.
2. Remove the dead-code from modules that do no implement them
3. Add type casting in the the module code to use the new interface

Closes: cosmos#12462

---

### Author Checklist

*All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.*

I have...

- [ ] included the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [ ] added `!` to the type prefix if API or client breaking change
- [ ] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/main/CONTRIBUTING.md#pr-targeting))
- [ ] provided a link to the relevant issue or specification
- [ ] followed the guidelines for [building modules](https://github.com/cosmos/cosmos-sdk/blob/main/docs/building-modules)
- [ ] included the necessary unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/main/CONTRIBUTING.md#testing)
- [ ] added a changelog entry to `CHANGELOG.md`
- [ ] included comments for [documenting Go code](https://blog.golang.org/godoc)
- [ ] updated the relevant documentation or specification
- [ ] reviewed "Files changed" and left comments if necessary
- [ ] confirmed all CI checks have passed

### Reviewers Checklist

*All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.*

I have...

- [ ] confirmed the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [ ] confirmed `!` in the type prefix if API or client breaking change
- [ ] confirmed all author checklist items have been addressed
- [ ] reviewed state machine logic
- [ ] reviewed API design and naming
- [ ] reviewed documentation is accurate
- [ ] reviewed tests and test coverage
- [ ] manually tested (if applicable)

(cherry picked from commit b65f3fe)

# Conflicts:
#	CHANGELOG.md
#	x/authz/module/module.go
#	x/group/module/module.go
#	x/nft/module/module.go
#	x/params/module.go
#	x/slashing/module.go
#	x/upgrade/module.go

* remove conflicts

* remove conflicts

* remove unused modules

Co-authored-by: Sishir Giri <sishirg27@gmail.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>

* feat: add message index event attribute to authz message execution (backport cosmos#12668) (cosmos#12673)

* chore(store): upgrade iavl to v0.19.0 (backport cosmos#12626) (cosmos#12697)

* feat: Add `GetParamSetIfExists` to prevent panic on breaking param changes (cosmos#12724)

* feat: Add convenience method for constructing key to access account's balance for a given denom (backport cosmos#12674) (cosmos#12745)

* feat: Add convenience method for constructing key to access account's balance for a given denom (cosmos#12674)

This PR adds a convenience method for constructing the key necessary to query for the account's balance of a given denom.

I ran into this issue since we are using ABCI query now to perform balance requests because we are also requesting merkle proofs for the returned balance [here](https://github.com/celestiaorg/celestia-node/pull/911/files#diff-0ee31f5a7bd88e9f758e6bebdf3ee36365519e55a451098d9638c39afe5eac42R144).

It would be nice to have a definitive convenience method for constructing the key.

[Ref.](github.com/celestiaorg/celestia-node/pull/911)

(cherry picked from commit a1777a8)

# Conflicts:
#	CHANGELOG.md
#	x/bank/legacy/v043/store.go
#	x/bank/types/keys.go

* Update CHANGELOG.md

* fix conflict

Co-authored-by: rene <41963722+renaynay@users.noreply.github.com>
Co-authored-by: Marko <marbar3778@yahoo.com>

* chore: bump tm in 0.45.x (cosmos#12784)

* bump tendermint version

* add changelog entry

* replace on jhump

* updates

* updates

* updates

Co-authored-by: Aleksandr Bezobchuk <aleks.bezobchuk@gmail.com>

* chore: 0.45.7 changelog prep (cosmos#12821)

* prepare for release

* modify release notes

* chore: migrate from registry to delegation keeper

Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Robert Pirtle <Astropirtle@gmail.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>
Co-authored-by: HaeSung <hea9549.github@gmail.com>
Co-authored-by: likhita-809 <78951027+likhita-809@users.noreply.github.com>
Co-authored-by: Robert Zaremba <robert@zaremba.ch>
Co-authored-by: Adu <foriteration@gmail.com>
Co-authored-by: Jacob Gadikian <jacobgadikian@gmail.com>
Co-authored-by: Sishir Giri <sishirg27@gmail.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
Co-authored-by: Federico Kunze Küllmer <31522760+fedekunze@users.noreply.github.com>
Co-authored-by: rene <41963722+renaynay@users.noreply.github.com>
Co-authored-by: Aleksandr Bezobchuk <aleks.bezobchuk@gmail.com>
p0mvn pushed a commit to osmosis-labs/cosmos-sdk that referenced this pull request Oct 13, 2022
mattverse pushed a commit to osmosis-labs/cosmos-sdk that referenced this pull request Oct 13, 2022
mergify bot pushed a commit to osmosis-labs/cosmos-sdk that referenced this pull request Oct 13, 2022
cosmos#12374) (cosmos#12437) (#356)

Co-authored-by: Adu <foriteration@gmail.com>
(cherry picked from commit d738efc)
p0mvn added a commit to osmosis-labs/cosmos-sdk that referenced this pull request Oct 14, 2022
#356) (#362)

* chore(backport): Simulation is not deterministic due to GenTx (backport cosmos#12374) (cosmos#12437) (#356)

Co-authored-by: Adu <foriteration@gmail.com>
(cherry picked from commit d738efc)

* try fix

Co-authored-by: Roman <roman@osmosis.team>
tomtau pushed a commit to crypto-org-chain/cosmos-sdk that referenced this pull request Oct 19, 2022
@GNaD13
Copy link
Contributor

GNaD13 commented Oct 27, 2022

Hey just out of curiosity, do you mean to say that you have solve the intermittent failure in CI?

@faddat I think this could fix TestAppStateDeterminism failure. I would run locally to see if this can help fix two other tests in simulation as well.

update: TestAppStateDeterminism could fail occasionally with log:

simulate.go:302: error on block  90/100, operation (442/670) from x/authz:
  : invalid coins [/Users/adudu/cosmos-sdk/x/bank/types/msgs.go:44]
   Comment: : invalid coins

--- FAIL: TestAppStateDeterminism (57.65s)

but this never happens to release/v0.45.x branch.

update: If we use empty coins slice to create banktype.MsgSend, the ValidateBasic() method would return an invalid coins error. By adding coins slice length checks, we can avoid this error. Now after the fix, test-sim-import-export test-sim-import-export test-sim-after-import and test-sim-multi-seed-short could run smoothly in local environment.

Hi @adu-web3 , Can this be backported to release/v0.45.x? I got this problem when trying to create new sims test in ver 0.45.9
in this PR

@julienrbrt
Copy link
Member

Hey just out of curiosity, do you mean to say that you have solve the intermittent failure in CI?

@faddat I think this could fix TestAppStateDeterminism failure. I would run locally to see if this can help fix two other tests in simulation as well.

update: TestAppStateDeterminism could fail occasionally with log:

simulate.go:302: error on block 90/100, operation (442/670) from x/authz:

: invalid coins [/Users/adudu/cosmos-sdk/x/bank/types/msgs.go:44]

Comment: : invalid coins

--- FAIL: TestAppStateDeterminism (57.65s)

but this never happens to release/v0.45.x branch.

update: If we use empty coins slice to create banktype.MsgSend, the ValidateBasic() method would return an invalid coins error. By adding coins slice length checks, we can avoid this error. Now after the fix, test-sim-import-export test-sim-import-export test-sim-after-import and test-sim-multi-seed-short could run smoothly in local environment.

Hi @adu-web3 , Can this be backported to release/v0.45.x? I got this problem when trying to create new sims test in ver 0.45.9

in this PR

Hey, this is already backported in 0.45.10

@GNaD13
Copy link
Contributor

GNaD13 commented Oct 27, 2022

Hey, this is already backported in 0.45.10

Yeahhh, it was so good. Tysm <3 @julienrbrt

larry0x pushed a commit to larry0x/cosmos-sdk that referenced this pull request May 22, 2023
p0mvn added a commit to osmosis-labs/cosmos-sdk that referenced this pull request Aug 23, 2023
p0mvn added a commit to osmosis-labs/cosmos-sdk that referenced this pull request Aug 29, 2023
* Revert feat!: ABCI 1.0 Integration 48fe175

* Revert "chore(backport): Simulation is not deterministic due to GenTx (backport cosmos#12374) (cosmos#12437) (#356)"

This reverts commit d738efc.

* updates
p0mvn added a commit to osmosis-labs/cosmos-sdk that referenced this pull request Sep 7, 2023
p0mvn added a commit to osmosis-labs/cosmos-sdk that referenced this pull request Sep 7, 2023
… (backport cosmos#12374) (cosmos#12437) (#356)" (#486)

This reverts commit d738efc.

Co-authored-by: Roman <roman@osmosis.team>
p0mvn added a commit to osmosis-labs/cosmos-sdk that referenced this pull request Sep 12, 2023
JeancarloBarrios pushed a commit to agoric-labs/cosmos-sdk that referenced this pull request Sep 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bug: Simulation is not deterministic due to GenSignedMockTx
5 participants