Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Actions hygiene #1821

Merged
merged 1 commit into from
Mar 10, 2021
Merged

Improve Actions hygiene #1821

merged 1 commit into from
Mar 10, 2021

Conversation

sethvargo
Copy link
Member

👋 hello there! I'm a fellow Googler who works on projects that leverage GitHub Actions for CI/CD. Recently I noticed a large increase in our queue time, and I've tracked it down to the limit of 180 concurrent jobs for an organization. To help be better citizens, I'm proposing changes across a few repositories that will reduce GitHub Actions hours and consumption. I hope these changes are reasonable and I'm happy to talk through them in more detail.

  • Only run GitHub Actions for pushes and PRs against the main branch of the repository. If your team uses a forking model, this change will not affect you. If your team pushes branches to the repository directly, this changes actions to only run against the primary branches or if you open a Pull Request against a primary branch.

  • For long-running jobs (especially tests), I added the "Cancel previous" workflow. This is very helpful to prevent a large queue backlog when you are doing rapid development and pushing multiple commits. Without this, GitHub Actions' default behavior is to run all actions on all commits.

There are other changes you could make, depending on your project (but I'm not an expert):

  • If you have tests that should only run when a subset of code changes, consider gating your workflow to particular file paths. For example, we have some jobs that do Terraform linting, but they only run when Terraform files are changed.

Hopefully these changes are not too controversial and also hopefully you can see how this would reduce actions consumption to be good citizens to fellow Googlers. If you have any questions, feel free to respond here or ping me on chat. Thank you!

👋 hello there! I'm a fellow Googler who works on projects that leverage GitHub Actions for CI/CD. Recently I noticed a large increase in our queue time, and I've tracked it down to the [limit of 180 concurrent jobs](https://docs.github.com/en/actions/reference/usage-limits-billing-and-administration) for an organization. To help be better citizens, I'm proposing changes across a few repositories that will reduce GitHub Actions hours and consumption. I hope these changes are reasonable and I'm happy to talk through them in more detail.

- Only run GitHub Actions for pushes and PRs against the main branch of the repository. If your team uses a forking model, this change will not affect you. If your team pushes branches to the repository directly, this changes actions to only run against the primary branches or if you open a Pull Request against a primary branch.

- For long-running jobs (especially tests), I added the "Cancel previous" workflow. This is very helpful to prevent a large queue backlog when you are doing rapid development and pushing multiple commits. Without this, GitHub Actions' default behavior is to run all actions on all commits.

There are other changes you could make, depending on your project (but I'm not an expert):

- If you have tests that should only run when a subset of code changes, consider gating your workflow to particular file paths. For example, we have some jobs that do Terraform linting, but [they only run when Terraform files are changed](https://github.com/google/exposure-notifications-verification-server/blob/c4f59fee71042cf668747e599e7c769fca736554/.github/workflows/terraform.yml#L3-L11).

Hopefully these changes are not too controversial and also hopefully you can see how this would reduce actions consumption to be good citizens to fellow Googlers. If you have any questions, feel free to respond here or ping me on chat. Thank you!
@google-cla google-cla bot added the cla: yes Indication that the PR author has signed a Google Contributor License Agreement. label Mar 10, 2021
@codecov
Copy link

codecov bot commented Mar 10, 2021

Codecov Report

Merging #1821 (2aafcd3) into master (5a468ce) will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #1821   +/-   ##
=======================================
  Coverage   97.55%   97.55%           
=======================================
  Files         102      102           
  Lines        6578     6578           
=======================================
  Hits         6417     6417           
  Misses         87       87           
  Partials       74       74           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5a468ce...2aafcd3. Read the comment docs.

@willnorris
Copy link
Collaborator

These changes seem totally reasonable (especially since I don't think they will impact this repo much at all), and we could probably simplify things a little further like reducing the size of the build matrix. If you're not already, you should probably also coordinate with @taquitos and @gauntface on broader cleanup efforts (I'm not at Google myself anymore, so don't know if that's already happening).

@willnorris willnorris merged commit d2c75a4 into google:master Mar 10, 2021
@sethvargo
Copy link
Member Author

@taquitos and I are in active conversations 😄

@sethvargo sethvargo deleted the patch-1 branch March 10, 2021 22:21
Copy link

@taquitos taquitos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎈🐐

copybara-service bot pushed a commit to google/error-prone that referenced this pull request Mar 11, 2021
context: google/go-github#1821
PiperOrigin-RevId: 362351653
copybara-service bot pushed a commit to google/error-prone that referenced this pull request Mar 11, 2021
context: google/go-github#1821
PiperOrigin-RevId: 362351653
copybara-service bot pushed a commit to google/error-prone that referenced this pull request Mar 11, 2021
context: google/go-github#1821
PiperOrigin-RevId: 362380769
copybara-service bot pushed a commit to google/dagger that referenced this pull request Mar 11, 2021
See google/go-github#1821

This CL makes a few changes to reduce the load of GitHub Actions from Dagger:
  * Restricts GitHub Actions to only the master or gh-pages branches
  * Cancels previous GitHub Action runs for the same PR branch.
  * Removes excessive API levels for emulator tests.

RELNOTES=N/A
PiperOrigin-RevId: 362394043
copybara-service bot pushed a commit to google/dagger that referenced this pull request Mar 11, 2021
See google/go-github#1821

This CL makes a few changes to reduce the load of GitHub Actions from Dagger:
  * Restricts GitHub Actions to only the master or gh-pages branches
  * Cancels previous GitHub Action runs for the same PR branch.
  * Removes excessive API levels for emulator tests.

RELNOTES=N/A
PiperOrigin-RevId: 362394043
copybara-service bot pushed a commit to google/dagger that referenced this pull request Mar 12, 2021
See google/go-github#1821

This CL makes a few changes to reduce the load of GitHub Actions from Dagger:
  * Restricts GitHub Actions to only the master or gh-pages branches
  * Cancels previous GitHub Action runs for the same PR branch.
  * Removes excessive API levels for emulator tests.

RELNOTES=N/A
PiperOrigin-RevId: 362394043
copybara-service bot pushed a commit to google/dagger that referenced this pull request Mar 12, 2021
See google/go-github#1821

This CL makes a few changes to reduce the load of GitHub Actions from Dagger:
  * Restricts GitHub Actions to only the master or gh-pages branches
  * Cancels previous GitHub Action runs for the same PR branch.
  * Removes excessive API levels for emulator tests.

RELNOTES=N/A
PiperOrigin-RevId: 362394043
copybara-service bot pushed a commit to google/turbine that referenced this pull request Mar 12, 2021
context: unknown commit, google/go-github#1821
PiperOrigin-RevId: 362381574
copybara-service bot pushed a commit to google/google-java-format that referenced this pull request Mar 12, 2021
context: unknown commit, google/go-github#1821
PiperOrigin-RevId: 362381574
copybara-service bot pushed a commit to google/turbine that referenced this pull request Mar 12, 2021
context: unknown commit, google/go-github#1821
PiperOrigin-RevId: 362600035
copybara-service bot pushed a commit to google/google-java-format that referenced this pull request Mar 12, 2021
context: unknown commit, google/go-github#1821
PiperOrigin-RevId: 362600035
copybara-service bot pushed a commit to google/dagger that referenced this pull request Mar 15, 2021
See google/go-github#1821

This CL makes a few changes to reduce the load of GitHub Actions from Dagger:
  * Restricts GitHub Actions to only the master or gh-pages branches
  * Cancels previous GitHub Action runs for the same PR branch.
  * Removes excessive API levels for emulator tests.

RELNOTES=N/A
PiperOrigin-RevId: 362394043
copybara-service bot pushed a commit to google/dagger that referenced this pull request Mar 15, 2021
See google/go-github#1821

This CL makes a few changes to reduce the load of GitHub Actions from Dagger:
  * Restricts GitHub Actions to only the master or gh-pages branches
  * Cancels previous GitHub Action runs for the same PR branch.
  * Removes excessive API levels for emulator tests.

RELNOTES=N/A
PiperOrigin-RevId: 362394043
copybara-service bot pushed a commit to google/dagger that referenced this pull request Mar 15, 2021
See google/go-github#1821

This CL makes a few changes to reduce the load of GitHub Actions from Dagger:
  * Restricts GitHub Actions to only the master or gh-pages branches
  * Cancels previous GitHub Action runs for the same PR branch.
  * Removes excessive API levels for emulator tests.

RELNOTES=N/A
PiperOrigin-RevId: 362394043
copybara-service bot pushed a commit to google/dagger that referenced this pull request Mar 15, 2021
See google/go-github#1821

This CL makes a few changes to reduce the load of GitHub Actions from Dagger:
  * Restricts GitHub Actions to only the master or gh-pages branches
  * Cancels previous GitHub Action runs for the same PR branch.
  * Removes excessive API levels for emulator tests.

RELNOTES=N/A
PiperOrigin-RevId: 362394043
copybara-service bot pushed a commit to google/dagger that referenced this pull request Mar 15, 2021
See google/go-github#1821

This CL makes a few changes to reduce the load of GitHub Actions from Dagger:
  * Restricts GitHub Actions to only the master or gh-pages branches
  * Cancels previous GitHub Action runs for the same PR branch.
  * Removes excessive API levels for emulator tests.

RELNOTES=N/A
PiperOrigin-RevId: 362394043
copybara-service bot pushed a commit to google/dagger that referenced this pull request Mar 15, 2021
See google/go-github#1821

This CL makes a few changes to reduce the load of GitHub Actions from Dagger:
  * Restricts GitHub Actions to only the master or gh-pages branches
  * Cancels previous GitHub Action runs for the same PR branch.
  * Removes excessive API levels for emulator tests.

RELNOTES=N/A
PiperOrigin-RevId: 362394043
copybara-service bot pushed a commit to google/dagger that referenced this pull request Mar 15, 2021
See google/go-github#1821

This CL makes a few changes to reduce the load of GitHub Actions from Dagger:
  * Restricts GitHub Actions to only the master or gh-pages branches (Note: For
    PR, this filter applies to the branch to merge into, not the branch itself).
  * Cancels previous GitHub Action runs for the same PR branch.
  * Removes excessive API levels for emulator tests.

RELNOTES=N/A
PiperOrigin-RevId: 362394043
copybara-service bot pushed a commit to google/dagger that referenced this pull request Mar 15, 2021
See google/go-github#1821

This CL makes a few changes to reduce the load of GitHub Actions from Dagger:
  * Restricts GitHub Actions to only the master or gh-pages branches (Note: For
    PR, this filter applies to the branch to merge into, not the branch itself).
  * Cancels previous GitHub Action runs for the same PR branch.
  * Removes excessive API levels for emulator tests.

RELNOTES=N/A
PiperOrigin-RevId: 362394043
copybara-service bot pushed a commit to google/dagger that referenced this pull request Mar 15, 2021
See google/go-github#1821

This CL makes a few changes to reduce the load of GitHub Actions from Dagger:
  * Restricts GitHub Actions to only the master or gh-pages branches (Note: For
    PR, this filter applies to the branch to merge into, not the branch itself).
  * Cancels previous GitHub Action runs for the same PR branch.
  * Removes excessive API levels for emulator tests.

RELNOTES=N/A
PiperOrigin-RevId: 362394043
copybara-service bot pushed a commit to google/dagger that referenced this pull request Mar 15, 2021
See google/go-github#1821

This CL makes a few changes to reduce the load of GitHub Actions from Dagger:
  * Restricts GitHub Actions to only the master or gh-pages branches (Note: For
    PR, this filter applies to the branch to merge into, not the branch itself).
  * Cancels previous GitHub Action runs for the same PR branch.
  * Removes excessive API levels for emulator tests.

RELNOTES=N/A
PiperOrigin-RevId: 362394043
copybara-service bot pushed a commit to google/dagger that referenced this pull request Mar 15, 2021
See google/go-github#1821

This CL makes a few changes to reduce the load of GitHub Actions from Dagger:
  * Restricts GitHub Actions to only the master or gh-pages branches (Note: For
    PR, this filter applies to the branch to merge into, not the branch itself).
  * Cancels previous GitHub Action runs for the same PR branch.
  * Removes excessive API levels for emulator tests.

RELNOTES=N/A
PiperOrigin-RevId: 362394043
copybara-service bot pushed a commit to google/dagger that referenced this pull request Mar 15, 2021
See google/go-github#1821

This CL makes a few changes to reduce the load of GitHub Actions from Dagger:
  * Restricts GitHub Actions to only the master or gh-pages branches (Note: For
    PR, this filter applies to the branch to merge into, not the branch itself).
  * Cancels previous GitHub Action runs for the same PR branch.
  * Removes excessive API levels for emulator tests.

RELNOTES=N/A
PiperOrigin-RevId: 362394043
copybara-service bot pushed a commit to google/dagger that referenced this pull request Mar 15, 2021
See google/go-github#1821

This CL makes a few changes to reduce the load of GitHub Actions from Dagger:
  * Restricts GitHub Actions to only the master or gh-pages branches (Note: For
    PR, this filter applies to the branch to merge into, not the branch itself).
  * Cancels previous GitHub Action runs for the same PR branch.
  * Removes excessive API levels for emulator tests.

RELNOTES=N/A
PiperOrigin-RevId: 363054339
copybara-service bot pushed a commit to google/dagger that referenced this pull request Mar 15, 2021
See google/go-github#1821

This CL makes a few changes to reduce the load of GitHub Actions from Dagger:
  * Restricts GitHub Actions to only the master or gh-pages branches (Note: For
    PR, this filter applies to the branch to merge into, not the branch itself).
  * Cancels previous GitHub Action runs for the same PR branch.
  * Removes excessive API levels for emulator tests.

RELNOTES=N/A
PiperOrigin-RevId: 363054339
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes Indication that the PR author has signed a Google Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants