Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Seed RNG to a fixed value at start of the test suite #1454

Merged
merged 1 commit into from
Sep 29, 2023

Conversation

fingolfin
Copy link
Member

Resolves #1437 (maybe?)

CC @benlorenz

@codecov
Copy link

codecov bot commented Sep 29, 2023

Codecov Report

Merging #1454 (bd92d69) into master (d82760d) will decrease coverage by 0.14%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #1454      +/-   ##
==========================================
- Coverage   84.73%   84.59%   -0.14%     
==========================================
  Files         110      110              
  Lines       29361    29361              
==========================================
- Hits        24879    24838      -41     
- Misses       4482     4523      +41     

see 8 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@benlorenz
Copy link
Collaborator

I am in favor of this, the number of tests for various testsets do look like that the rng has the desired effect. But there are still some huge differences in runtime between 1.10 and nightly:
1.9

Generic.UnivPoly.adhoc_binary_operations | 5313   5313  0.3s

1.10

Generic.UnivPoly.adhoc_binary_operations | 5313   5313  0.5s

nightly

Generic.UnivPoly.adhoc_binary_operations | 5313   5313  1m32.8s

For older CI runs of this testset the number of tests varies from run to run but here it is constant across julia versions except for 1.6. Maybe running this with the output from #1432 would be good.

@fingolfin
Copy link
Member Author

We merged #1432. Shall we merge this one, too, @thofma ?

@thofma
Copy link
Member

thofma commented Sep 29, 2023

I am not too big of fan of fixed seeds, but I can see that this might be a good idea in view of our recent CI problems.

@fingolfin
Copy link
Member Author

Yeah we can always un-fix it in a future version (but then we should at least print the initial seed for reproducibility)

@fingolfin fingolfin merged commit aa9d18b into Nemocas:master Sep 29, 2023
13 of 15 checks passed
@fingolfin fingolfin deleted the mh/rng-seed branch September 29, 2023 16:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Please add more seeding to the testsuite
3 participants