-
Notifications
You must be signed in to change notification settings - Fork 3.7k
[TEST][FLAKY] fix random fail #6312
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Nerver mind, I noticed that this only used the random engine inside TVM. |
We should avoid the usage of seed to resolve the test error wrt to randomness. Since setting seed can hide problems in real usecases. Instead, we should make sure the testcase pass for most randomness generated, perhaps by increasing the error change range in this case |
@tqchen I undenstand your concern. However, I have done some experiments and the generated data is random, we can not make sure completely like |
I do think increase the size and error range should help, and as long as the error happens in low probability(we can calculate the confidence) we should be fine. |
Have increased size and error. When to run 100 times, previous code will occur 6 errors, 1000 times will occur 44 errors. New code doesn't have check error. Wish this could let CI be more robust. @tqchen |
* [TEST][FLAKY] fix random fail * increase size and error check range
* [TEST][FLAKY] fix random fail * increase size and error check range
* [TEST][FLAKY] fix random fail * increase size and error check range
* [TEST][FLAKY] fix random fail * increase size and error check range
* [TEST][FLAKY] fix random fail * increase size and error check range
* [TEST][FLAKY] fix random fail * increase size and error check range
fix #6309