Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ci] fix masked tests #2567

Merged
merged 4 commits into from
May 15, 2020
Merged

[ci] fix masked tests #2567

merged 4 commits into from
May 15, 2020

Conversation

jameslamb
Copy link

Thanks for contributing to Prefect!

Please describe your work and make sure your PR:

  • adds new tests (if appropriate)
  • updates CHANGELOG.md (if appropriate)
  • updates docstrings for any new functions or function arguments, including docs/outline.toml for API reference docs (if appropriate)

What does this PR change?

Thanks for this awesome project! I've been getting familiar with prefect this week, and wanted to contribute back where I could. I noticed you aren't running flake8 in this project, so I ran that over the codebase to see if there were any issues that could be fixed.

I saw one pattern that I think it would be helpful to address...duplicated tests.

If you run this command from the root of the repo

flake8 | grep -E "F811 .*unused .*test_"

you'll see the following:

./tests/engine/test_flow_runner.py:1187:1: F811 redefinition of unused 'test_parameters_can_be_set_in_context_if_none_passed' from line 1160
./tests/engine/test_flow_runner.py:1194:1: F811 redefinition of unused 'test_parameters_overwrite_context' from line 1167
./tests/engine/test_flow_runner.py:1203:1: F811 redefinition of unused 'test_parameters_overwrite_context_only_if_key_matches' from line 1176
./tests/cli/test_auth.py:133:1: F811 redefinition of unused 'test_switch_tenants' from line 123
./tests/cli/test_execute.py:57:1: F811 redefinition of unused 'test_execute_cloud_flow_fails' from line 24
./tests/agent/test_fargate_agent.py:569:1: F811 redefinition of unused 'test_deploy_flow_raises' from line 545
./tests/tasks/dropbox/test_dropbox.py:39:5: F811 redefinition of unused 'test_creds_are_pulled_from_secret_at_runtime' from line 28
./tests/tasks/azureml/test_datastore.py:191:5: F811 redefinition of unused 'test_missing_datastore_raises_error' from line 184
./tests/client/test_client_auth.py:121:5: F811 redefinition of unused 'test_load_local_api_token_is_called_when_the_client_is_initialized_without_token' from line 110
./tests/core/test_flow.py:1906:5: F811 redefinition of unused 'test_flow_dot_run_handles_cached_states_across_runs' from line 1815
./server/tests/database/test_hasura.py:837:5: F811 redefinition of unused 'test_execute_respects_as_box' from line 830

In this PR, I tried to address those warnings by fixing those test files. I saw three flavors of problem:

  1. Identical test copied in two places --> removed one of them
  2. Partial test and more-complete version of it --> removed the partial one
  3. Two different tests with the same name --> renamed one or both to fix the conflict

I know that I don't have full context for what these tests are doing, so please let me know if I made any mistakes.

Why is this PR important?

This PR makes sure that all of the project's tests are run in CI. If pytest finds duplicated tests methods in a file, it will only run one of them. You can test by creating a file called test_stuff.py with this content:

import unittest

def test_something():
    assert False

def test_something():
    assert True

Run pytest test_stuff.py and you'll see this:

image

Change the first test_something() to test_something_else(), re-run pytest test_stuff.py, and you'll see this:

image

Thanks for your time and consideration!

@codecov
Copy link

codecov bot commented May 14, 2020

Codecov Report

Merging #2567 into master will increase coverage by 0.01%.
The diff coverage is n/a.

@jameslamb
Copy link
Author

It seems that one of these masked tests has been failing! (build link)

image

@joshmeek
Copy link

joshmeek commented May 15, 2020

@jameslamb It seems like the test was not recognized before due to the double naming and looks like it never should pass like that anyway! Nice catch 😄 I think the assert isinstance(result.data.y[0].a, Box) should be changed to assert isinstance(result.data.y[0], Box) to one level up from the .a (which is definitely not intended to be of type Box and is 1)

@jameslamb
Copy link
Author

@jameslamb It seems like the test was not recognized before due to the double naming and looks like it never should pass like that anyway! Nice catch I think the assert isinstance(result.data.y[0].a, Box) should be changed to assert isinstance(result.data.y[0], Box) to one level up from the .a (which is definitely not intended to be of type Box and is 1)

wow! Makes me glad I did this 😀

I just pushed 859b6b2, let's see if it helps. Seems like it should, based on the test below it.

Copy link

@joshmeek joshmeek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good @jameslamb!

@joshmeek joshmeek merged commit 9c57eba into PrefectHQ:master May 15, 2020
@jameslamb jameslamb mentioned this pull request May 21, 2020
3 tasks
@jameslamb jameslamb deleted the fix/duplicated-tests branch September 15, 2020 20:15
abrookins pushed a commit that referenced this pull request Jul 27, 2022
Update favicon manifest for orion UI
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants