-
Notifications
You must be signed in to change notification settings - Fork 54
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MNT Remove redundant test in CI #124
MNT Remove redundant test in CI #124
Conversation
As discussed, the "Tests" and "Doc Tests" tasks are redundant. Since CI is already quite slow as is, we really don't want to run all the tests twice. Therefore, the "Doc Tests" task is removed. Ideally, we would have one task for only doctests and one for unit tests but AFAICT, there is no option to _only_ run doctests.
@skops-dev/maintainers Ready for review (disregard false codecov complaint) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice catch 🙂
.github/workflows/build-test.yml
Outdated
SUPER_SECRET: ${{ secrets.HF_HUB_TOKEN }} | ||
run: | | ||
python -m pytest -s --cov=skops --doctest-modules skops/ | ||
python -m pytest -s --cov=skops --cov-report=xml --doctest-modules -v skops/ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Aren't the options redundant with here?
Lines 9 to 17 in ca853d4
[tool.pytest.ini_options] | |
filterwarnings = [ | |
"error::DeprecationWarning", | |
"error::FutureWarning", | |
] | |
markers = [ | |
"network: marks tests as requiring internet (deselect with '-m \"not network\"')", | |
] | |
addopts = "--cov=skops --cov-report=term-missing --doctest-modules" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes they are. I thought about removing the options there but then was unsure for the following reason: Is the main purpose of pyproject.toml to be for local development or for CI? If it's for the former, and we make a change that should only affect local development, we don't want to accidentally affect CI too. Therefore, I chose to have the options explicitly in CI (I assume that explicitly setting arguments takes precedence over the config file). If you think this distinction is not relevant, I can remove all redundant options.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interesting point of view. To me the point was to have it easier for people to have a local env more similar to the CI, so that the same tests are run with less effort. I personally added them there because I wanted to have them in the CI (definitely true for --doctest-modules
).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is definitely a decision we can make -- running locally should be as close as possible to CI and the pyproject.toml should be the source of truth for that. It's just good to make that assumption explicit, so that we can be aware of that for the future.
As discussed, the "Tests" and "Doc Tests" tasks are redundant. Since CI
is already quite slow as is, we really don't want to run all the tests
twice. Therefore, the "Doc Tests" task is removed.
Ideally, we would have one task for only doctests and one for unit tests
but AFAICT, there is no option to only run doctests.