Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove t.Skip in testcases #307

Merged
merged 10 commits into from
Sep 1, 2021
Merged

Remove t.Skip in testcases #307

merged 10 commits into from
Sep 1, 2021

Conversation

tnasu
Copy link
Member

@tnasu tnasu commented Aug 20, 2021

Description

I fixed t.Skip and t.Skipf test cases.

.//consensus/state_test.go:     t.Skip("NON-deterministic test: no such LastProofHash making index validator to be proposer")
.//consensus/replay_test.go:    t.Skip("With decision to skip TestSimulateValidatorsChange, we also skip this test case" +
.//consensus/replay_test.go:    t.Skip("With decision to skip TestSimulateValidatorsChange, we also skip this test case" +
.//consensus/replay_test.go:    t.Skip("With decision to skip TestSimulateValidatorsChange, we also skip this test case" +
.//consensus/replay_test.go:    t.Skip("With decision to skip TestSimulateValidatorsChange, we also skip this test case" +
.//consensus/replay_test.go:    t.Skip("With decision to skip TestSimulateValidatorsChange, we also skip this test case" +
.//types/voter_set_test.go:     t.Skip("this test case need a new reward rule")
.//types/validator_set_test.go: t.Skip("Temporarily excluded because this a case that doesn't end due to Proposer selection changes.")
.//types/validator_set_test.go: t.Skip("The ProposerPriority validated in this test case is excluded because it is not used in VRF.")
.//test/e2e/tests/e2e_test.go:          t.Skip("E2E_MANIFEST not set, not an end-to-end test run")
.//test/e2e/tests/validator_test.go:    t.Skip("NON-deterministic test: SelectProposer is not well since valSchedule.Set is changed")
.//state/state_test.go: t.Skip("This test is for priority based proposer. Vrf selection based proposer skips this test.")
.//state/state_test.go: t.Skip("Ostracon doesn't select a Proposer based on ProposerPriority")
.//p2p/transport_test.go:       t.Skip("NON-deterministic test: write tcp 127.0.0.1:40973->127.0.0.1:54732: i/o timeout")
.//mempool/reactor_test.go:             t.Skip("skipping test in short mode.")
.//mempool/reactor_test.go:             t.Skip("skipping test in short mode.")
.//light/detector_test.go:      t.Skip("Voter selection in Ostracon only supports sequential verification mode, " +
.//light/detector_test.go:      t.Skip("Voter selection in Ostracon only supports sequential verification mode, " +
.//light/client_test.go:        t.Skip("Skipping verification disabled under selection of voters")
.//consensus/replay_test.go:    t.Skip("We will skip this test case and prepare another one later" +
.//consensus/state_test.go:     t.Skipf("NON-deterministic test: race detected during execution of test")
.//libs/clist/clist_test.go:            t.Skipf("Skipping on non-amd64 machine")
.//libs/clist/clist_test.go:            t.Skipf("Skipping on non-amd64 machine")

@codecov
Copy link

codecov bot commented Aug 24, 2021

Codecov Report

Merging #307 (fd45f8e) into main (f53ee28) will increase coverage by 1.73%.
The diff coverage is 68.37%.

@@            Coverage Diff             @@
##             main     #307      +/-   ##
==========================================
+ Coverage   61.18%   62.91%   +1.73%     
==========================================
  Files         272      272              
  Lines       29988    30166     +178     
==========================================
+ Hits        18347    18978     +631     
+ Misses       9963     9481     -482     
- Partials     1678     1707      +29     
Impacted Files Coverage Δ
abci/types/application.go 0.00% <0.00%> (ø)
abci/types/messages.go 16.05% <0.00%> (-1.55%) ⬇️
blockchain/v2/processor_context.go 63.15% <0.00%> (ø)
blockchain/v2/reactor.go 34.91% <ø> (ø)
cmd/ostracon/commands/light.go 15.82% <0.00%> (ø)
cmd/ostracon/commands/root.go 46.87% <0.00%> (ø)
config/toml.go 68.00% <ø> (ø)
consensus/replay_file.go 0.00% <0.00%> (ø)
consensus/replay_stubs.go 60.00% <0.00%> (+42.05%) ⬆️
crypto/bls/bls.go 48.41% <ø> (ø)
... and 93 more

- TestHandshakeReplayAll
- TestHandshakeReplaySome
- TestHandshakeReplayOne
- TestHandshakeReplayNone
- TestMockProxyApp
@tnasu
Copy link
Member Author

tnasu commented Aug 31, 2021

Addressed the below:

.//consensus/state_test.go:     t.Skip("NON-deterministic test: no such LastProofHash making index validator to be proposer")
TestStateLockPOLRelock

.//consensus/replay_test.go:    t.Skip("With decision to skip TestSimulateValidatorsChange, we also skip this test case" +
TestHandshakeReplayAll

.//consensus/replay_test.go:    t.Skip("With decision to skip TestSimulateValidatorsChange, we also skip this test case" +
TestHandshakeReplaySome

.//consensus/replay_test.go:    t.Skip("With decision to skip TestSimulateValidatorsChange, we also skip this test case" +
TestHandshakeReplayOne

.//consensus/replay_test.go:    t.Skip("With decision to skip TestSimulateValidatorsChange, we also skip this test case" +
TestHandshakeReplayNone

.//consensus/replay_test.go:    t.Skip("With decision to skip TestSimulateValidatorsChange, we also skip this test case" +
TestMockProxyApp

.//types/voter_set_test.go:     t.Skip("this test case need a new reward rule")
TestElectVotersNonDupEquity -> BenchmarkElectVotersNonDupEquity

.//types/validator_set_test.go: t.Skip("Temporarily excluded because this a case that doesn't end due to Proposer selection changes.")
TestValSetUpdatesOrderIndependenceTestsExecute

.//types/validator_set_test.go: t.Skip("The ProposerPriority validated in this test case is excluded because it is not used in VRF.")
TestAveragingInIncrementProposerPriority

.//state/state_test.go: t.Skip("This test is for priority based proposer. Vrf selection based proposer skips this test.")
TestProposerFrequency

.//p2p/transport_test.go:       t.Skip("NON-deterministic test: write tcp 127.0.0.1:40973->127.0.0.1:54732: i/o timeout")
TestTransportHandshake

.//consensus/replay_test.go:    t.Skip("We will skip this test case and prepare another one later" +
TestSimulateValidatorsChange

.//consensus/state_test.go:     t.Skipf("NON-deterministic test: race detected during execution of test")
TestResetTimeoutPrecommitUponNewHeight

Keep t.Skip and t.Skipf

//
// Skip
//
.//test/e2e/tests/e2e_test.go:          t.Skip("E2E_MANIFEST not set, not an end-to-end test run")
.//mempool/reactor_test.go:             t.Skip("skipping test in short mode.")
.//mempool/reactor_test.go:             t.Skip("skipping test in short mode.")

//
// Skipf
//
.//libs/clist/clist_test.go:            t.Skipf("Skipping on non-amd64 machine")
.//libs/clist/clist_test.go:            t.Skipf("Skipping on non-amd64 machine")

//
// These tests depend on the deprecated `Skipping verification` mode of Ostracon
//
.//light/detector_test.go:      t.Skip("Voter selection in Ostracon only supports sequential verification mode, " +
.//light/detector_test.go:      t.Skip("Voter selection in Ostracon only supports sequential verification mode, " +
.//light/client_test.go:        t.Skip("Skipping verification disabled under selection of voters")

//
// These tests depend on selecting a proposer with ProposerPriority
//
.//test/e2e/tests/validator_test.go:    t.Skip("NON-deterministic test: SelectProposer is not well since valSchedule.Set is changed")
.//state/state_test.go: t.Skip("Ostracon doesn't select a Proposer based on ProposerPriority")

@tnasu tnasu marked this pull request as ready for review August 31, 2021 10:51
@tnasu tnasu requested review from torao and Kynea0b August 31, 2021 10:51
@tnasu tnasu self-assigned this Aug 31, 2021
if !pubKey1.Equals(pubKey2) {
// block creator must be the cs.privValidator
cs1.privValidator = vs.PrivValidator
cs1.privValidatorPubKey = pubKey2
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These technics are happened deadlock by cs1.createProposalBlock(round) with test --race. And I made createProposalBlockSlim.

*/

// Chi-squared test for `SelectProposer`
chiSquareds := make([]ChiSquared, N)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I replaced frequencies test(bound: margin of error) with Chi-squared test

Copy link
Contributor

@torao torao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@Kynea0b Kynea0b left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tnasu tnasu merged commit e2e0dd3 into Finschia:main Sep 1, 2021
@tnasu tnasu deleted the main-fix-testcases branch September 3, 2021 06:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants