suite: Multiple test runs #1245
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Suite now accepts flag testify.c of type uint that defines how many times each test will run. Test creation has been refactored into a separate function for readability purposes. In Run each test is now appended to tests testCount times.
Fixed tests where the hardcoded expected value should scale with the amount of times the test is run.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Added ability to run tests multiple times with a single SetupSuite and TearDownSuite
Changes
Running go test now accepts flag testify.c which defines how many times we want each test to run
Modified tests that expect the test to run only once
Motivation
Previously the method to run multiple tests was using flag -test.count or -count.
The problem with that is that SetupSuite is called multiple times, and if it takes a long time to complete then the tests will also take a long time. Since these tests are all in the same test suite, they should be able to reuse the same SetupSuite.
As an example, running go test -count=10 completes in about 15.5s and running go test -testify.c=10 completes in about 8.9s, halving the time to complete.
Related issues
#1154