Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create utility function for getting Kubernetes client #2845

Merged
merged 8 commits into from
Jun 24, 2020

Conversation

pcieslinski
Copy link

@pcieslinski pcieslinski commented Jun 23, 2020

Thanks for contributing to Prefect!

Please describe your work and make sure your PR:

  • adds new tests (if appropriate)
  • add a changelog entry in the changes/ directory (if appropriate)
  • updates docstrings for any new functions or function arguments, including docs/outline.toml for API reference docs (if appropriate)

Note that your PR will not be reviewed unless all three boxes are checked.

What does this PR change?

While working on resolving this issue: #2281, I discovered a lot of duplicate code and tests. I managed to extract into a separate utility function the logic responsible for creating and authorizing the client for Kubernetes, which is used in the task library.

get_kubernetes_client was created in a similar way to get_boto_client. Thanks to this, I wanted to be consistent with the rest of the codebase.

Example of use:

from prefect.utilities.kubernetes import get_kubernetes_client

api_client = get_kubernetes_client(resource="pod", kubernetes_api_key_secret="KUBERNETES_API_KEY")

When the logic of creating and authorizing a client for Kubernetes was extracted into a separate function, some of the tests from the tasks were removed. Now all testing is done at the level of function get_kubernetes_client.

Why is this PR important?

Generally, this PR strives to stick to good software development practices and DRY principle. Thanks to this refactor we will have gathered in one place the logic to create and authorize the client for Kubernetes. We can use this part of the code throughout the codebase.

By removing duplicate tests, the project code will be easier to maintain.

The `get_kubernetes_client` function takes two parameters and
returns the initialized and authorized object with appropriate
kubernetes client. It can be used across entire application.
After extracting the logic of initializing and authorizing the
kubernetes client, the tests responsible for checking this
functionality were moved to `utilities/test_kubernetes.py`.
@pcieslinski
Copy link
Author

pcieslinski commented Jun 24, 2020

I missed that this api_client, which is used in Kubernetes tasks, is mocked in other tests for these tasks. I will fix these tests and everything should be fine. The logic of operation is the same, only the form of the code changed. ;)

@joshmeek
Copy link

Thanks for the big PR @pcieslinski! Yeah the client is mocked heavily for these tests and with this new utility could definitely be cleaned up / possibly separated into a test fixture.

@codecov
Copy link

codecov bot commented Jun 24, 2020

Codecov Report

Merging #2845 into master will decrease coverage by 0.12%.
The diff coverage is 100.00%.

@pcieslinski
Copy link
Author

@joshmeek I know that this PR has unfortunately grown quite a lot and can be difficult to check, but it is mainly because the same code was used in many places. I have improved the tests for all Kubernetes tasks and put the mocked Kubernetes client in the fixture. :)

Copy link

@jcrist jcrist left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @pcieslinski, this looks good to me! Love to see that many deleted lines of code :).

@jcrist jcrist merged commit 467bc5b into PrefectHQ:master Jun 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants