Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve build and release system #5991

Merged
merged 3 commits into from
Dec 6, 2021
Merged

Conversation

scholzj
Copy link
Member

@scholzj scholzj commented Dec 5, 2021

Type of change

  • Refactoring

Description

This PR improves the build and release process of the Strimzi operators. It follows the ideas discussed in the past during the community calls and similar pipeline setups for Drain Cleaner and Kafka Bridge. The main targets are:

  • Add new release pipelines to facilitate better release process for release candidates
    • Release artifacts (docs, release YAMLs, Helm Chart ZIP) should be produced by the pipeline
    • The same build / containers should be used for RCs as for GAs
  • Make it easier to do CVE re-spins
    • Take the released binaries and just rebuild the containers
  • Add support for multi-arch builds

To provide this, our build had to be refactored to make it easier to separate the binaries used to build the container images. They are now collected in one place (./docker-images/artifacts) and can be easily passed between pipelines. It also separates the make targets for Java and container builds to be able to build these separately with the artifacts being the shared point between them. This change should have only a minimal impact on the users building Strimzi locally.

The Azure pipelines were refactored. Apart form adding the new pipelines for release and CVE re-spins, I also refactored the different templates used by build and system test pipelines. The build pipeline has also been redesigned and consists from multiple jobs with different tasks:

Screenshot 2021-12-05 at 20 59 48

This PR also adds support for ARM64 / AArch64 images (-> it is part of this PR instead of separate PR since it allows to properly test the multi-arch build capability).

Signed-off-by: Jakub Scholz <www@scholzj.com>
@scholzj scholzj added this to the 0.27.0 milestone Dec 5, 2021
@scholzj
Copy link
Member Author

scholzj commented Dec 5, 2021

/azp run acceptance

@azure-pipelines
Copy link

Azure Pipelines failed to run 1 pipeline(s).

@scholzj
Copy link
Member Author

scholzj commented Dec 5, 2021

/azp run acceptance

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@scholzj scholzj force-pushed the improve-build-and-release-system branch from 664c539 to 9ba2795 Compare December 5, 2021 15:21
Signed-off-by: Jakub Scholz <www@scholzj.com>
@scholzj scholzj force-pushed the improve-build-and-release-system branch from 677220a to 5a84beb Compare December 5, 2021 22:43
@scholzj
Copy link
Member Author

scholzj commented Dec 5, 2021

/azp run regression

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@scholzj scholzj requested review from Frawless and ppatierno December 6, 2021 00:35
@scholzj scholzj marked this pull request as ready for review December 6, 2021 00:37
.azure/cve-pipeline.yaml Show resolved Hide resolved
systemtest/pom.xml Show resolved Hide resolved
Copy link
Member

@ppatierno ppatierno left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

artifactRunId: ''
architectures: ['amd64', 'arm64']
- stage: manual_validation
displayName: Validate container before pushing container as ${{ parameters.releaseVersion }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't remember we have this manual validation in Drain Cleaner and canary for example. Why do we need it here? Is that anything that we should think of the other repo as well?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have it in Drain Cleaner and Bridge. IIRC you do not have CVE re-spin pipeline in Canary. So you probably don't have it. This is needed for testing. You use the pipeline to:

  • Build the images with suffix - e.g. quay.io/strimzi/operator:0.27.0-2
  • You test that the rebuild works -> for example with manual smoke test
  • You approve the validation and the pipeline continues and pushes the images as quay.io/strimzi/operator:0.27.0 -> that way everyone will use them

Signed-off-by: Jakub Scholz <www@scholzj.com>
@scholzj scholzj merged commit c3bd3a8 into main Dec 6, 2021
@scholzj scholzj deleted the improve-build-and-release-system branch December 6, 2021 14:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants