Merkledb Attempt to reduce test runtime #1990
Conversation
dboehm-avalabs
requested review from
danlaine,
darioush and
StephenButtolph
as code owners
x/merkledb/db_test.go
Outdated
| r := rand.New(rand.NewSource(now)) // #nosec G404 | ||
| runRandDBTest(require, r, generateInitialValues(require, r, initialValues, updates, checkHashProbability)) | ||
| func Fuzz_MerkleDB_RandomCases(f *testing.F) { | ||
| f.Add(int64(0), uint16(1000)) |
There was a problem hiding this comment.
I'm not sure what the CI setup is, but Fuzz tests don't run by default when you just do go test unless they have some base corpus added. This, and any others we may want to add, ensure that this test runs.
There was a problem hiding this comment.
Oh, looking at the CI, we do separately run the fuzz tests anyway, so it isn't strictly needed.
x/merkledb/helpers_test.go
Outdated
| "github.com/ava-labs/avalanchego/utils/maybe" | ||
| ) | ||
|
|
||
| func protoProofsEqual(proof1, proof2 *pb.Proof) bool { |
There was a problem hiding this comment.
I feel like we should just use require.Equal. I'd rather the UT take a bit longer than have to maintain this additional code, and I think it's more readable to just use require.Equal in the tests.
There was a problem hiding this comment.
It makes the tests take 4x as long though and we are hitting the timeout, so we either need to speed up existing tests or start removing tests.
There was a problem hiding this comment.
Run With Normal Equals vs
Run With Custom Equals
This is just one run and I'm not sure what the variance is but it saves:
~7 seconds on ARM
~3 seconds on Ubuntu
~8 seconds on Mac
~13 seconds on Windows
Average savings is about ~30% of the total run time.
There was a problem hiding this comment.
If you feel that strongly about it, I'll make them fuzz tests instead and still use require.equal
danlaine
added
testing
Co-authored-by: Dan Laine <daniel.laine@avalabs.org> Signed-off-by: David Boehm <91908103+dboehm-avalabs@users.noreply.github.com>
This reverts commit ff7c4cd.
Co-authored-by: Dan Laine <daniel.laine@avalabs.org> Signed-off-by: David Boehm <91908103+dboehm-avalabs@users.noreply.github.com>
Co-authored-by: Dan Laine <daniel.laine@avalabs.org> Signed-off-by: David Boehm <91908103+dboehm-avalabs@users.noreply.github.com>
MerkleDB package often times out during tests, so make some adjustments to reduce total runtime. Improvements come from turning several long running randomized tests into fuzz tests