Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add nightly test run to GitHub Actions #769

Closed
Askaholic opened this issue Apr 10, 2021 · 0 comments · Fixed by #781
Closed

Add nightly test run to GitHub Actions #769

Askaholic opened this issue Apr 10, 2021 · 0 comments · Fixed by #781
Labels
good first issue

Comments

@Askaholic
Copy link
Collaborator

According to the hypothesis docs, hypothesis works best when you can run tests with a large number of examples (thousands or tens of thousands). Of course this takes a long time so it isn't really viable for a local test run or for CI on pull requests.

We should set up a github action that runs on the develop branch once every 24 hours with a hypothesis profile that runs a large number of examples. Using profiles is described here: https://hypothesis.readthedocs.io/en/latest/settings.html#hypothesis.settings.load_profile. I would say we set max_examples to be at least 1000.
@Askaholic Askaholic mentioned this issue Apr 23, 2021
Merged
Askaholic added a commit that referenced this issue Apr 29, 2021
@Askaholic
Issue/#769 nightly test run (#781)
4d81156 
* Add nightly hypothesis settings profile

* Set no deadline for async hypothesis tests

* Add nightly test run

The nightly run should not report coverage since it runs only a subset 
of tests.

* Cache hypothesis example database

* Fix flaky test

Ensures we don't use the player_info message from earlier
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
good first issue
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Issue/#769 nightly test run Askaholic/server
1 participant
@Askaholic