Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce unit tests for GH actions scripts #10533

Merged
merged 37 commits into from
Jan 17, 2022
Merged

Conversation

markus-hinsche
Copy link
Contributor

@markus-hinsche markus-hinsche commented Dec 15, 2021

Proposed changes:

  • Setup target test-gh-actions in Makefile, specify category
  • Add poetry dev-dependencies and update poetry.lock
  • Prepare modules under test: Drop strict dependence of environment variables so the module is testable without
  • Add two test modules

Potential caveats (Feedback from reviewers encouraged):

  • I currently use sys.path.append because .github/scripts/ is not yet in the PYTHONPATH. Any ideas?
  • The test coverage (reported after running unit tests in CI) only for rasa/, but not .github/scripts

Status (please check what you already did):

  • added some tests for the functionality
  • updated the documentation
  • updated the changelog (please check changelog for instructions)
  • reformat files using black (please check Readme for instructions)

Base automatically changed from regr-test-datadog-f1 to main December 16, 2021 08:55
@markus-hinsche markus-hinsche marked this pull request as ready for review December 16, 2021 10:31
@markus-hinsche markus-hinsche requested a review from a team as a code owner December 16, 2021 10:31
@markus-hinsche markus-hinsche requested review from virtualroot and RASADSA and removed request for a team December 16, 2021 10:31
@virtualroot virtualroot requested review from joejuzl, wochinge and a team December 17, 2021 09:22
def _get_is_external_and_dataset_repository_branch():
is_external = os.environ["IS_EXTERNAL"]
dataset_repository_branch = os.environ["DATASET_REPOSITORY_BRANCH"]
if str(is_external).lower() in ("yes", "true", "t", "1"):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

environment variables are always strings, aren't they?

Suggested change
if str(is_external).lower() in ("yes", "true", "t", "1"):
if is_external.lower() in ("yes", "true", "t", "1"):

@@ -254,10 +251,10 @@ def read_results(file: str) -> Dict[str, Any]:
return result


def push_results(file_name: str, file: str):
def _push_results(file_name: str, file: str):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def _push_results(file_name: str, file: str):
def _push_results(file_name: str, file: str) -> None:

runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we even need the matrix? There will be always just one set to test right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed the matrix

with:
poetry-version: ${{ env.POETRY_VERSION }}

- name: Load Poetry Cached Libraries ⬇
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't we need an extra step to also install the dependencies? Fully relying on the cache seems risky

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙈, good catch

Copy link
Contributor

@wochinge wochinge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Do we want these tests to run on each PR or only when the scripts where changed?

@markus-hinsche
Copy link
Contributor Author

I thought about running it on every PR in the beginning. If it turns out to be annoying for others, we can restrict it of course

@github-actions
Copy link
Contributor

🚀 A preview of the docs have been deployed at the following URL: https://10533--rasahq-docs-rasa-v2.netlify.app/docs/rasa

@markus-hinsche markus-hinsche merged commit 6b28f00 into main Jan 17, 2022
@markus-hinsche markus-hinsche deleted the gha-scripts-unit-tests branch January 17, 2022 20:14
@wochinge
Copy link
Contributor

@markus-hinsche Sounds good - probably worth making it a required check then, right? Optional failing CI steps tend to be confusing

@markus-hinsche
Copy link
Contributor Author

@wochinge Sure, making it required would be great. I am not sure if I have enough permissions to configure this in GitHub though. Could you set the check as required?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants