build(oci): add config to allow building images for other archs #1352

andrewazores · 2023-02-01T23:02:08Z

Welcome to Cryostat! 👋

Before contributing, make sure you have:

Read the contributing guidelines
Linked a relevant issue which this PR resolves
Linked any other relevant issues, PR's, or documentation, if any
Resolved all conflicts, if any
Rebased your branch PR on top of the latest upstream main branch
Attached at least one of the following labels to the PR: [chore, ci, docs, feat, fix, test]
Signed the last commit: git commit --amend --signoff

Related to #1329

Description of the change:

This adds some configuration to the pom.xml so that the build architecture (and OS, though this only makes sense to use linux now) can be changed by setting -Dbuild.arch=arm64. This will cause the image build to pull the specified architecture for the UBI layer, and to build the application image for that specified architecture as well. qemu-user-static (on Fedora, or equivalent package elsewhere) must be installed for this step to work, since part of building the application base image involves pulling the cross-arch UBI and running commands within it (ex. microdnf), so this part must be emulated to prepare the image.

Motivation for the change:

Once we can verify that these images work as expected on real ARM hardware, we can continue to publish these on push via CI alongside the existing linux/amd64 images so that developers who may be running on ARM hardware can also run Cryostat. Or, Cryostat upstream releases could be used in production ARM environments.

I expect/hope the CI setup will basically consist of adding one of these actions as a step:

https://github.com/docker/setup-qemu-action (equivalent to installing qemu-user-static on your workstation)
https://github.com/uraimo/run-on-arch-action
and then invoking mvn -Dbuild.arch=arm64 clean package ... as usual.

~~TODO figure if there's a nice way with the existing base image build and jib plugin to do an image manifest, rather than creating separate tagged images for different archs.~~

github-actions · 2023-02-01T23:22:31Z

Test image available:

$ CRYOSTAT_IMAGE=ghcr.io/cryostatio/cryostat:pr-1352-36dce7912ebd9729cb64c71d94c2c10f239e975e sh smoketest.sh

github-actions · 2023-02-02T16:05:18Z

Test image available:

$ CRYOSTAT_IMAGE=ghcr.io/cryostatio/cryostat:pr-1352-8f25cd201cc8065edf10d3c9407b3dde53395d7f sh smoketest.sh

ebaron

Looks good. If Jib can't create manifest lists, we could add a podman step as part of the build or CI to create it using the image produced by Jib:
podman manifest create quay.io/cryostat/cryostat:2.3.0-snapshot containers-storage:quay.io/cryostat/cryostat:2.3.0-snapshot-linux-arm64

We would just need to make sure the tags for the manifest lists aren't already in use.

andrewazores · 2023-02-07T15:54:26Z

That makes sense. On each developer workstation there probably isn't much need to build multiarch and create a manifest image on a regular basis, so just doing it in CI before publishing the upstream builds should work well.

I tried a setup with this:

https://github.com/GoogleContainerTools/jib/blob/master/docs/faq.md#how-do-i-specify-a-platform-in-the-manifest-list-or-oci-index-of-a-base-image

but ran into some hurdles with the localhost/cryostat-base intermediate image that also gets built.

github-actions · 2023-02-13T16:48:57Z

Test image available:

$ CRYOSTAT_IMAGE=ghcr.io/cryostatio/cryostat:pr-1352-85a26d60e40ab73aee96f30d05546ad7c9fb9163 sh smoketest.sh

github-actions · 2023-02-22T17:29:09Z

Test image available:

$ CRYOSTAT_IMAGE=ghcr.io/cryostatio/cryostat:pr-1352-929b44bc334edd1806a7c84bac9a0791e5f23696 sh smoketest.sh

github-actions · 2023-03-08T22:52:10Z

Test image available:

$ CRYOSTAT_IMAGE=ghcr.io/cryostatio/cryostat:pr-1352-51525c146b75f26cb2c1d6f4f74531ce741886b6 sh smoketest.sh

andrewazores · 2023-04-06T21:11:14Z

The ARM64 image I built manually has been verified as working in the comments of https://github.com/cryostatio/cryostat/issues/1329 , so I'm picking this back up.

@ebaron how do you think we should tag the published images? I'm thinking of reusing the old naming scheme for manifests, and then having the arch-specific images with the os/arch appended. So 2.3.0-snapshot would be a manifest containing entries for 2.3.0-snapshot-linux-arm64 and 2.3.0-snapshot-linux-amd64.

ebaron · 2023-04-06T21:21:05Z

The ARM64 image I built manually has been verified as working in the comments of #1329 , so I'm picking this back up.

@ebaron how do you think we should tag the published images? I'm thinking of reusing the old naming scheme for manifests, and then having the arch-specific images with the os/arch appended. So 2.3.0-snapshot would be a manifest containing entries for 2.3.0-snapshot-linux-arm64 and 2.3.0-snapshot-linux-amd64.

That should be fine, but do we need to publish the arch-specific images at their own tags? Would just the manifest be enough?

andrewazores · 2023-04-06T21:23:31Z

The manifest looks like it's really just a manifest with no real image contents:

REPOSITORY                                          TAG                           IMAGE ID      CREATED            SIZE
quay.io/cryostat/cryostat                           2.3.0-snapshot                6a43618fc67c  14 minutes ago     1.04 kB
quay.io/cryostat/cryostat                           latest                        9a178511b0f3  15 minutes ago     524 MB
quay.io/cryostat/cryostat                           2.3.0-snapshot-linux-arm64    9a178511b0f3  15 minutes ago     524 MB
localhost/cryostat-base                             latest-linux-arm64            ec66379b936c  15 minutes ago     382 MB
quay.io/cryostat/cryostat                           2.3.0-snapshot-linux-amd64    7117f129abe4  17 minutes ago     510 MB

so I think the arch-specific images still would need to be published in some form or another, otherwise the manifest alone would be pointing at layer hashes that can't be found and pulled.

andrewazores · 2023-04-06T22:11:22Z

Ah nevermind, podman manifest push knows how to sort it out:

$ podman manifest push quay.io/andrewazores/cryostat:2.3.0-snapshot quay.io/andrewazores/cryostat:2.3.0-snapshot 
Getting image list signatures
Copying 2 of 2 images in list
Copying image sha256:84875109a4ce24d31be30b2eecc68a8edda1c3cc09b120c6c29ff4b726e7fc8e (1/2)
Getting image source signatures
Copying blob 26a6ef8302d7 done  
Copying blob 713205907b3d done  
Copying blob 4a5c0f835ad6 done  
Copying blob 751997c37749 done  
Copying blob 0e713ec45614 done  
Copying blob 982b35ff53d0 done  
Copying blob ee59c67c1a2e done  
Copying blob 7e8f73a57865 done  
Copying blob 3c08dd77cd66 skipped: already exists  
Copying blob 63e1f2e00257 skipped: already exists  
Copying blob 64f8d6fd9d56 skipped: already exists  
Copying blob 531f60635336 skipped: already exists  
Copying blob 412593307974 skipped: already exists  
Copying blob 0c043eb9db83 skipped: already exists  
Copying config b24421d31d done  
Writing manifest to image destination
Storing signatures
Copying image sha256:661e88a131e5a45026296771f4722888d27fb893a0545c2741e0404933d2cfb4 (2/2)
Getting image source signatures
Copying blob bd80078cd8d3 skipped: already exists  
Copying blob a5c4a523d913 skipped: already exists  
Copying blob 4e2e2a4aef30 skipped: already exists  
Copying blob b451555644cd skipped: already exists  
Copying blob c86a62483f4c skipped: already exists  
Copying blob f38ae51e36f6 skipped: already exists  
Copying blob cc71b362f2cb skipped: already exists  
Copying blob 3c08dd77cd66 skipped: already exists  
Copying blob 14ca54efaf16 skipped: already exists  
Copying blob 63e1f2e00257 skipped: already exists  
Copying blob 64f8d6fd9d56 skipped: already exists  
Copying blob 531f60635336 skipped: already exists  
Copying blob 0c043eb9db83 skipped: already exists  
Copying blob 412593307974 skipped: already exists  
Copying config 28301ac9cc done  
Writing manifest to image destination
Storing signatures
Writing manifest list to image destination
Storing list signatures

https://quay.io/repository/andrewazores/cryostat?tab=tags

github-actions · 2023-04-06T23:01:15Z

Test image available:

$ CRYOSTAT_IMAGE=ghcr.io/cryostatio/cryostat:pr-1352-2d259f27a38b71353472d1f0f4fe64f73a4a0832 sh smoketest.sh

github-actions · 2023-04-06T23:48:21Z

Test image available:

$ CRYOSTAT_IMAGE=ghcr.io/cryostatio/cryostat:pr-1352-3eb5690f78ec4847ccbce59f6f3381f733972c91 sh smoketest.sh

github-actions · 2023-06-02T14:26:23Z

Test image available:

$ CRYOSTAT_IMAGE=ghcr.io/cryostatio/cryostat:pr-1352-4ea7876defc2e486bbb823b5b553df4a6493bc73 sh smoketest.sh

github-actions · 2023-06-13T16:16:54Z

Test image available:

$ CRYOSTAT_IMAGE=ghcr.io/cryostatio/cryostat:pr-1352-1817c22b7ab019cee6cca0e7b4b29c403a87d02c sh smoketest.sh

github-actions bot added the needs-triage Needs thorough attention from code reviewers label Feb 1, 2023

mergify bot added the safe-to-test label Feb 1, 2023

andrewazores added build chore Refactor, rename, cleanup, etc. and removed needs-triage Needs thorough attention from code reviewers labels Feb 1, 2023

andrewazores marked this pull request as ready for review February 1, 2023 23:14

andrewazores requested a review from ebaron as a code owner February 1, 2023 23:14

andrewazores force-pushed the multiarch-images branch from 36dce79 to 8f25cd2 Compare February 2, 2023 01:57

ebaron previously approved these changes Feb 7, 2023

View reviewed changes

andrewazores mentioned this pull request Feb 10, 2023

[Request] Provide ARM images #1329

Closed

andrewazores dismissed ebaron’s stale review via 85a26d6 February 13, 2023 16:31

andrewazores force-pushed the multiarch-images branch from 8f25cd2 to 85a26d6 Compare February 13, 2023 16:31

ebaron previously approved these changes Feb 13, 2023

View reviewed changes

andrewazores force-pushed the multiarch-images branch from 85a26d6 to 929b44b Compare February 22, 2023 17:14

andrewazores dismissed ebaron’s stale review via 929b44b February 22, 2023 20:01

ebaron previously approved these changes Feb 23, 2023

View reviewed changes

andrewazores dismissed ebaron’s stale review via 51525c1 March 8, 2023 22:36

andrewazores force-pushed the multiarch-images branch from 929b44b to 51525c1 Compare March 8, 2023 22:36

ebaron previously approved these changes Mar 9, 2023

View reviewed changes

andrewazores dismissed ebaron’s stale review via 2d259f2 April 6, 2023 22:44

andrewazores force-pushed the multiarch-images branch from 51525c1 to 2d259f2 Compare April 6, 2023 22:44

andrewazores force-pushed the multiarch-images branch from 2d259f2 to 3eb5690 Compare April 6, 2023 23:01

andrewazores mentioned this pull request Apr 12, 2023

build(operator-sdk): upgrade operator sdk version to 1.28.0 cryostatio/cryostat-operator#543

Merged

18 tasks

andrewazores force-pushed the multiarch-images branch from 3eb5690 to 4ea7876 Compare June 2, 2023 14:01

andrewazores force-pushed the multiarch-images branch from 4ea7876 to 45a21d3 Compare June 13, 2023 15:49

ebaron previously approved these changes Jun 13, 2023

View reviewed changes

build(oci): add config to allow building images for other archs

1817c22

andrewazores dismissed ebaron’s stale review via 1817c22 June 13, 2023 16:00

andrewazores force-pushed the multiarch-images branch from 45a21d3 to 1817c22 Compare June 13, 2023 16:00

ebaron approved these changes Jun 13, 2023

View reviewed changes

andrewazores merged commit 20ea619 into cryostatio:main Jun 13, 2023

andrewazores deleted the multiarch-images branch June 13, 2023 16:17

andrewazores restored the multiarch-images branch June 13, 2023 18:04

andrewazores mentioned this pull request Jun 13, 2023

ci(multiarch): build multiarch manifest image #1540

Merged

7 tasks

andrewazores deleted the multiarch-images branch June 13, 2023 18:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

build(oci): add config to allow building images for other archs #1352

build(oci): add config to allow building images for other archs #1352

andrewazores commented Feb 1, 2023 •

edited

Loading

github-actions bot commented Feb 1, 2023

github-actions bot commented Feb 2, 2023

ebaron left a comment

andrewazores commented Feb 7, 2023

github-actions bot commented Feb 13, 2023

github-actions bot commented Feb 22, 2023

github-actions bot commented Mar 8, 2023

andrewazores commented Apr 6, 2023

ebaron commented Apr 6, 2023

andrewazores commented Apr 6, 2023

andrewazores commented Apr 6, 2023

github-actions bot commented Apr 6, 2023

github-actions bot commented Apr 6, 2023

github-actions bot commented Jun 2, 2023

github-actions bot commented Jun 13, 2023

build(oci): add config to allow building images for other archs #1352

build(oci): add config to allow building images for other archs #1352

Conversation

andrewazores commented Feb 1, 2023 • edited Loading

Welcome to Cryostat! 👋

Before contributing, make sure you have:

Description of the change:

Motivation for the change:

github-actions bot commented Feb 1, 2023

github-actions bot commented Feb 2, 2023

ebaron left a comment

Choose a reason for hiding this comment

andrewazores commented Feb 7, 2023

github-actions bot commented Feb 13, 2023

github-actions bot commented Feb 22, 2023

github-actions bot commented Mar 8, 2023

andrewazores commented Apr 6, 2023

ebaron commented Apr 6, 2023

andrewazores commented Apr 6, 2023

andrewazores commented Apr 6, 2023

github-actions bot commented Apr 6, 2023

github-actions bot commented Apr 6, 2023

github-actions bot commented Jun 2, 2023

github-actions bot commented Jun 13, 2023

andrewazores commented Feb 1, 2023 •

edited

Loading