Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: move test-race to its own job #10895

Merged
merged 6 commits into from
Jan 11, 2022
Merged

ci: move test-race to its own job #10895

merged 6 commits into from
Jan 11, 2022

Conversation

tac0turtle
Copy link
Member

Description

moving test-race to its own job allows us to run it or normal tests on their own. this reduces running jobs. The down side is if they both fail we then have to rerun both of them.


Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

  • included the correct type prefix in the PR title
  • added ! to the type prefix if API or client breaking change
  • targeted the correct branch (see PR Targeting)
  • provided a link to the relevant issue or specification
  • followed the guidelines for building modules
  • included the necessary unit and integration tests
  • added a changelog entry to CHANGELOG.md
  • included comments for documenting Go code
  • updated the relevant documentation or specification
  • reviewed "Files changed" and left comments if necessary
  • confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

  • confirmed the correct type prefix in the PR title
  • confirmed ! in the type prefix if API or client breaking change
  • confirmed all author checklist items have been addressed
  • reviewed state machine logic
  • reviewed API design and naming
  • reviewed documentation is accurate
  • reviewed tests and test coverage
  • manually tested (if applicable)

@github-actions github-actions bot added the T: CI label Jan 6, 2022
Copy link
Collaborator

@robert-zaremba robert-zaremba left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we want to optimize it then the better strategy is to avoid duplicated runs:

  • run all race tests
  • run other tests which were not run with race tests (to not duplicate).

@tac0turtle
Copy link
Member Author

If we want to optimize it then the better strategy is to avoid duplicated runs:

  • run all race tests
  • run other tests which were not run with race tests (to not duplicate).

I don't get which part of this is not being done. We are running the tests that can't be run in the race detector in tests and the ones that can here. This isn't meant as a speed up. We need to redo the test suite to fix this imo.

@robert-zaremba
Copy link
Collaborator

I don't get which part of this is not being done.

The second bullet point. We are duplicate test runs. We will need some smarter script to do it.

@robert-zaremba
Copy link
Collaborator

So if this PR won't speed up anything, then what's the improvement?

@tac0turtle
Copy link
Member Author

tac0turtle commented Jan 7, 2022

So if this PR won't speed up anything, then what's the improvement?

moving test-race to its own job allows us to run it or normal tests on their own. this reduces running jobs. The down side is if they both fail we then have to rerun both of them.

as its stated in the description, if normal tests fail there is no need to rerun 10 odd jobs, but only need to rerun 4-5. This reduces the amount of jobs running when rerunning things which unblocks runners for other jobs.

Copy link
Collaborator

@robert-zaremba robert-zaremba left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, is there a way to re run only one job? I have never seen that option.

I'm not against this PR, so approving, but let's clarify how to use it / take benefit of it.

@robert-zaremba
Copy link
Collaborator

I created a task for the optimization I mentioned: #10913

@tac0turtle tac0turtle added the A:automerge Automatically merge PR once all prerequisites pass. label Jan 11, 2022
@mergify mergify bot merged commit ddf5639 into master Jan 11, 2022
@mergify mergify bot deleted the test-race branch January 11, 2022 11:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A:automerge Automatically merge PR once all prerequisites pass. T: CI
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants