Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Update Github Actions for Node 20. #6670

Conversation

philipaconrad
Copy link
Contributor

@philipaconrad philipaconrad commented Apr 4, 2024

This PR updates the upload-artifact and download-artifact Github actions to the latest version (v4), which should eliminate the deprecation warning spam we're seeing in CI.

Screenshot of an older CI run's warnings (non-exhaustive)

image

Note to reviewers: The release (post-tag) and post-merge workflows have been updated as well, with the same general strategies used for the more complex pull-request workflow.

@philipaconrad philipaconrad added the github_actions Pull requests that update GitHub Actions code label Apr 4, 2024
@philipaconrad philipaconrad self-assigned this Apr 4, 2024
@philipaconrad
Copy link
Contributor Author

philipaconrad commented Apr 4, 2024

Ah, too late, didn't read: apparently, updating the upload-artifacts action to v4 introduces breaking changes to the behavior of uploading multiple times to the same artifact target. 🤦

The fix will be reworking the actions jobs to target unique artifact names/paths, but I or someone else will need to do some digging to make sure we don't break anything in silly ways from changing artifacts names around.

@srenatus
Copy link
Contributor

srenatus commented Apr 4, 2024

See also #6650 😉

@ashutosh-narkar
Copy link
Member

@philipaconrad we can do this in stages I think. We can fix the pull request workflow as that should not be affected by this. We can then look into the others which may need some reworking.

@philipaconrad
Copy link
Contributor Author

So! The good news is that splitting the artifacts targets up by their OS/arch target in the build matrix should be just fine. It will mean that we will have a number of binaries targets equal to the number of arches. 🤔

Copy link

netlify bot commented Apr 5, 2024

Deploy Preview for openpolicyagent ready!

Name Link
🔨 Latest commit 4db599a
🔍 Latest deploy log https://app.netlify.com/sites/openpolicyagent/deploys/661d4002445b030008001239
😎 Deploy Preview https://deploy-preview-6670--openpolicyagent.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@philipaconrad philipaconrad force-pushed the update-versions-gha-actions branch 9 times, most recently from 12e8053 to 373039a Compare April 5, 2024 21:30
@philipaconrad
Copy link
Contributor Author

philipaconrad commented Apr 5, 2024

Github Actions is annoying because you basically always have to test-by-running. 😅

At any rate, I've whacked the pull-request.yaml workflow into shape for the v4 update. We'll likely need to update the release actions as well, unless those do their own Go builds for the release artifacts.

Major changes:

  • Unique artifact targets for each OS + Arch combination (ex: binaries-darwin-arm64, binaries-linux-amd64)
  • GO_TAGS variants have a different name prefix: binaries-variant-{...}
    • To get all normal build targets, download-artifacts@v4 jobs can use the property: pattern: binaries-*
    • To get just the variant build targets, pattern: binaries-variant-*

I'm also adding a few comments here and there to document weirdness in the build system as it's come up (such as only smoke-testing amd64 targets, for instance).

To properly test the post-*.yaml actions, I'll need to get careful review of those files + we'll need to cut 1+ patch releases to exercise the jobs, once they're ready. 😅

@@ -240,7 +251,8 @@ jobs:
- name: Download release binaries
uses: actions/download-artifact@v4
with:
name: binaries
pattern: binaries-*
merge-multiple: true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh cool

johanfylling
johanfylling previously approved these changes Apr 8, 2024
Copy link
Contributor

@johanfylling johanfylling left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! 👍

- os: darwin
run: macos-latest
targets: ci-build-darwin ci-build-darwin-arm64-static
arch: amd64
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this also contains arm64, it would have been nice if we could use all or something. But I suppose that would mess up the GOARCH env var ..

Perhaps we could do a third Upload Binaries step to handle the case where arch is empty 🤔.

This is very much a nit, so feel free to disregard 😄.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the interest of getting this merged, it may be worth adding a comment for that target about the situation, leaving the arch: amd64 field alone for now. 🤔

@srenatus
Copy link
Contributor

srenatus commented Apr 8, 2024

This PR will never be green because the names of two required steps have changed:
image
↕️
image

So let's merge it and adjust the branch settings afterwards.

@johanfylling
Copy link
Contributor

johanfylling commented Apr 8, 2024

Yes, I was thinking the same @srenatus 👍 .
All open PRs would then need to be synced with main.

@@ -49,13 +49,17 @@ jobs:
run: ubuntu-22.04
targets: ci-go-ci-build-linux-static
go_tags: GO_TAGS="-tags=opa_no_oci"
variant_name: opa_no_ci
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be named opa_no_oci like it is above? or are they just confusingly similar? :D

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, the go_tags item in YAML is actually the whole string GO_TAGS="-tags=opa_no_oci", which is horrendous for downstream steps that expect just [a-zA-Z0-9_-]* as the expected character set. 😅 I took the lazy route, and added a new YAML item with just the important bits.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, but I meant that there’s a typo, or no? “ci” -> “oci”

This commit updates the `upload-artifact` and `download-artifact`
Github actions to the latest version (v4), which should eliminate
the deprecation warning spam we're seeing in CI.

Note: We had to break up the merged "binaries" target into multiple,
smaller targets, which are group by OS for the `post-tag` and
`post-merge` workflows, and by OS + arch for the `pull-request`
workflow.

Signed-off-by: Philip Conrad <philipaconrad@gmail.com>
@philipaconrad
Copy link
Contributor Author

philipaconrad commented Apr 15, 2024

Thank you all for the review comments! 😄

I've updated the post-tag and post-merge workflows to use the same general naming strategy for uploads (binaries-<something...>) and the same download strategy for grouping published artifacts.

Those two workflows are noticeably less complex in how they partition the build jobs up, so they were much easier to refactor. Note that I cannot really test them, because they fire after major actions, like a PR merge or release tag, occur.

After this PR merges, I'll keep an eye on the GH actions to make sure I didn't b0rk the merge job. If it fails, then the post-tag action will fail almost identically.

@philipaconrad
Copy link
Contributor Author

Also, for the 2x lingering status checks, see @srenatus's comment.

@philipaconrad philipaconrad merged commit 3954ba0 into open-policy-agent:main Apr 15, 2024
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
github_actions Pull requests that update GitHub Actions code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants