OADP-5973, OADP-3340, OADP-6212: AWS, GCP, Azure Standardized Flow Implementation #1712
Conversation
|
Skipping CI for Draft Pull Request. |
openshift-ci
bot
added
the
do-not-merge/work-in-progress
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: kaovilai The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci
bot
added
the
approved
|
/test ci/prow/unit-test |
|
@kaovilai: The specified target(s) for The following commands are available to trigger optional jobs: Use In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
/test unit-test |
|
let's keep pushing on this please |
openshift-ci-robot
added
the
jira/valid-reference
|
@kaovilai: This pull request references OADP-5973 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.19.0" version, but no target version was set. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@kaovilai: This pull request references OADP-5973 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.19.0" version, but no target version was set. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@kaovilai: This pull request references OADP-5973 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.19.0" version, but no target version was set. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@kaovilai: This pull request references OADP-5973 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.19.0" version, but no target version was set. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
kaovilai
changed the title
kaovilai
force-pushed
the
cco-flow-gcp
branch
7 times, most recently
from
1f8aa7e to
381a1dc
Compare
|
/hold for further testing Now aligns with openshift/enhancements#1800 and not using credentialsRequest CR |
openshift-ci
bot
added
the
do-not-merge/hold
|
Got a valid backup and available bsl.. /unhold TODO: validate azure and aws also works |
openshift-ci
bot
removed
the
do-not-merge/hold
|
@kaovilai: This pull request references OADP-5973 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
kaovilai
changed the title
|
@kaovilai: This pull request references OADP-5973 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
1 similar comment
|
@kaovilai: This pull request references OADP-5973 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@kaovilai: This pull request references OADP-5973 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@kaovilai: This pull request references OADP-5973 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@kaovilai: This pull request references OADP-5973 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@kaovilai: This pull request references OADP-5973 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@kaovilai: This pull request references OADP-5973 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
1 similar comment
|
@kaovilai: This pull request references OADP-5973 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
kaovilai
force-pushed
the
cco-flow-gcp
branch
5 times, most recently
from
a10818c to
3cf7182
Compare
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com> Add make targets sts-flow testing Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
|
/hold While the work on GCP seem successful, this PR will be further enhanced with
|
openshift-ci
bot
added
the
do-not-merge/hold
kaovilai
changed the title
|
@kaovilai: This pull request references OADP-5973 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set. This pull request references OADP-3340 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set. This pull request references OADP-6212 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
- Label STS-created secrets with "oadp.openshift.io/secret-type": "sts-credentials" - Implement automatic region patching for AWS STS secrets from BSL configuration - Implement automatic resource group patching for Azure STS secrets from BSL configuration - Ensure only STS-created secrets are patched by checking for specific keys: - AWS: "credentials" key with role_arn and web_identity_token_file content - Azure: "azurekey" key with AZURE_CLIENT_ID but no AZURE_CLIENT_SECRET - Add comprehensive test coverage for all patching scenarios - Update documentation to reflect dynamic configuration capabilities This enhancement allows the first BSL to automatically configure region (AWS) or resource group (Azure) in STS secrets, eliminating manual configuration needs.
| deploy-olm-stsflow-aws: deploy-olm-stsflow ## Deploy via OLM with AWS STS standardized flow and create subscription with AWS env vars | ||
| @if [ -n "$(AWS_ROLE_ARN)" ]; then \ | ||
| echo "Creating subscription with AWS STS environment variables..."; \ | ||
| $(call create-sts-subscription,/tmp/oadp-aws-subscription.yaml); \ | ||
| echo " - name: ROLEARN" >> /tmp/oadp-aws-subscription.yaml; \ | ||
| echo " value: \"$(AWS_ROLE_ARN)\"" >> /tmp/oadp-aws-subscription.yaml; \ | ||
| $(call apply-sts-subscription,/tmp/oadp-aws-subscription.yaml,AWS STS); \ | ||
| else \ | ||
| echo ""; \ | ||
| echo "AWS STS environment variable not set. Please set:"; \ | ||
| echo " AWS_ROLE_ARN"; \ | ||
| echo ""; \ | ||
| echo "Example:"; \ | ||
| echo " make deploy-olm-stsflow-aws AWS_ROLE_ARN=arn:aws:iam::123456789012:role/my-oadp-role"; \ | ||
| fi |
There was a problem hiding this comment.
AWS secret creation works
Creating subscription with AWS STS environment variables...
subscription.operators.coreos.com/oadp-operator created
Subscription created with AWS STS environment variables.
Waiting for operator to be ready...
Waiting for InstallPlan to be created...
....InstallPlan install-kmnzr found
Waiting for CSV to exist...
CSV oadp-operator.v99.0.0 found
Waiting for CSV to be ready...
clusterserviceversion.operators.coreos.com/oadp-operator.v99.0.0 condition met
Operator is ready!
NAME PACKAGE SOURCE CHANNEL
oadp-operator oadp-operator oadp-operator-catalog operator-sdk-run-bundle
NAME DISPLAY VERSION REPLACES PHASE
oadp-operator.v99.0.0 OADP Operator 99.0.0 Succeeded
~/oadp-operator-cco-flow-gcp cco-flow-gcp 2m 2s
❯ oc get secrets -n openshift-adp
NAME TYPE DATA AGE
builder-dockercfg-4fhws kubernetes.io/dockercfg 1 9m12s
cloud-credentials Opaque 1 30s
default-dockercfg-46hxm kubernetes.io/dockercfg 1 9m12s
deployer-dockercfg-pqs7b kubernetes.io/dockercfg 1 9m12s
non-admin-controller-dockercfg-z8sdp kubernetes.io/dockercfg 1 32s
openshift-adp-controller-manager-dockercfg-2tv74 kubernetes.io/dockercfg 1 33s
velero-dockercfg-kz5vt kubernetes.io/dockercfg 1 32sThere was a problem hiding this comment.
dev verified that this PR secret allows for CloudStorage API for aws sts to create bucket and bsl is available.
There was a problem hiding this comment.
enhancing with secret region patching to close https://issues.redhat.com/browse/CCO-625
and remove https://issues.redhat.com/browse/OADP-5354 workaround.
There was a problem hiding this comment.
secret region from bsl is now working
The BSL controller patches AWS secrets with region information by modifying the Data field directly, but the STS flow was completely replacing StringData which caused region patches to be overridden. This change preserves existing Data when updating STS secrets by only updating specific StringData fields rather than clearing all existing data.
|
@kaovilai: all tests passed! Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
| fi | ||
|
|
||
| .PHONY: deploy-olm-stsflow-azure | ||
| deploy-olm-stsflow-azure: deploy-olm-stsflow ## Deploy via OLM with Azure Workload Identity standardized flow and create subscription with Azure env vars |
There was a problem hiding this comment.
Currently blocked from verifying azure.
Signed-off-by: Tiger Kaovilai tkaovila@redhat.com
Why the changes were made
openshift/enhancements#1800 is allowing us to pursue direct secret creation using fields from Standardized Flow Operator Install.
This PR implements direct secret creation from Standardized Flow user inputs for GCP, Azure, and refactored AWS to do the same for convenience.
The dev testing in this PR however covers only GCP for OADP-5973.
AWS will have further enhancements around adding region to secret from BSL in the future.
Azure STS (OADP-3340) will be tested separately and may not need another code PR if all goes well.
Further, this PR is not intended to cover CloudStorage API enablement for GCP and Azure which will also by proxy test the functionality here.
How to test the changes made
Create GCP WIF cluster
Prepare following information from cluster
GCP Project Number
workload-identity-pool
workload-identity-provider
Create a service account (SA) with following roles
Note SA email for next steps.
Grant velero k8s SA
system:serviceaccount:openshift-adp:veleroroles/iam.serviceAccountTokenCreatorrole to the GCP SA email above like so.Tip
The url contains &tokenizedAuth=GCP and so you can actually test the secret creation functionality on a non GCP WIF cluster.
You can input dummy data and see secret created.
Enter requested info
create dpa
bsl is available
run backup, successful.