Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[randomness part 1] New utils/rand package for non-deterministic true randomness #4062

Merged
merged 9 commits into from
Mar 24, 2023

Conversation

tarakby
Copy link
Contributor

@tarakby tarakby commented Mar 17, 2023

( result of splitting #4052 into several PRs)

  • add new random package flow-go/utils/rand that uses non-deterministic crypto/rand to export some tools needed in the code base (Unitn, Shuffle, Sample..). This package provides the same useful APIs as math/rand and it should be used instead of math/rand in production code.
  • add tests for new tools.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 17, 2023

FVM Benchstat comparison

This branch with compared with the base branch onflow:master commit 2f1a1d0

The command (for i in {1..7}; do go test ./fvm ./engine/execution/computation --bench . --tags relic -shuffle=on --benchmem --run ^$; done) was used.

Collapsed results for better readability

old.txtnew.txt
time/opdelta
ComputeBlock/16/cols/128/txes-24.28s ± 0%4.34s ± 0%~(p=1.000 n=1+1)
 
us/txdelta
ComputeBlock/16/cols/128/txes-22.09k ± 0%2.12k ± 0%~(p=1.000 n=1+1)
 
alloc/opdelta
ComputeBlock/16/cols/128/txes-21.21GB ± 0%1.21GB ± 0%~(p=1.000 n=1+1)
 
allocs/opdelta
ComputeBlock/16/cols/128/txes-217.4M ± 0%17.4M ± 0%~(p=1.000 n=1+1)
 

package math

// MinUint returns the minimum of a list of uints.
func MinUint(uints ...uint) uint {
Copy link
Contributor Author

@tarakby tarakby Mar 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not used anywhere

@@ -50,6 +50,15 @@ const (
DefaultAddress = "localhost:0"
)

// returns a deterministic math/rand PRG that can be used for deterministic randomness in tests only.
// The PRG seed is logged in case the test iteration needs to be reproduced.
func GetPRG(t *testing.T) *rand.Rand {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will be used in subsequent PRs

utils/rand/rand.go Show resolved Hide resolved
utils/rand/rand.go Outdated Show resolved Hide resolved
utils/rand/rand.go Show resolved Hide resolved
// and use big number modular reduction by `n`.
random := n
for random > max {
if _, err := rand.Read(buffer[:size]); err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you need to either check for numRead (aka _ in your code), or use io.ReadFull (same elsewhere)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the case of crypto/rand, checking one output is enough, as per the doc:

// Read is a helper function that calls Reader.Read using io.ReadFull.
// On return, n == len(b) if and only if err == nil.

I'll add a comment to clarify.

utils/rand/rand.go Outdated Show resolved Hide resolved
utils/rand/rand.go Show resolved Hide resolved
utils/rand/rand.go Show resolved Hide resolved
utils/rand/rand.go Outdated Show resolved Hide resolved
utils/rand/rand.go Outdated Show resolved Hide resolved
}
stdev := stat.StdDev(distribution, nil)
mean := stat.Mean(distribution, nil)
assert.Greater(t, tolerance*mean, stdev, fmt.Sprintf("basic randomness test failed. stdev %v, mean %v", stdev, mean))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

similar to my comment in the other pr. stddev is meaningless in uniform distribution. may just check for expectedMean * (1 - tolerance) < mean < expectedMean * (1 + tolerance).

Also, maybe refactor the test cases to use the same skeleton (the only change that change is how bucket width/index is computed)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I answered your comment here: #4067 (comment)

Good idea about the refactoring the test common logic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

refactor and clarifying comment about the standard dev test: 7cf858a and 003882a

@tarakby
Copy link
Contributor Author

tarakby commented Mar 24, 2023

bors merge

@codecov-commenter
Copy link

codecov-commenter commented Mar 24, 2023

Codecov Report

Merging #4062 (77d4e2b) into master (2f1a1d0) will decrease coverage by 1.36%.
The diff coverage is 75.67%.

@@            Coverage Diff             @@
##           master    #4062      +/-   ##
==========================================
- Coverage   52.56%   51.20%   -1.36%     
==========================================
  Files         772      672     -100     
  Lines       72985    60187   -12798     
==========================================
- Hits        38365    30821    -7544     
+ Misses      31503    26898    -4605     
+ Partials     3117     2468     -649     
Flag Coverage Δ
unittests 51.20% <75.67%> (-1.36%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
utils/unittest/fixtures.go 0.00% <0.00%> (ø)
utils/rand/rand.go 82.35% <82.35%> (ø)

... and 225 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@bors bors bot merged commit aa38b85 into master Mar 24, 2023
@bors bors bot deleted the tarak/randomness-part1-utils branch March 24, 2023 22:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants