Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enabel external oidc in capi private rosa hcp cluster #51689

Closed
wants to merge 36 commits into from

Conversation

heliubj18
Copy link
Contributor

enable external oidc in capi rosa hcp

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 6, 2024
Copy link
Contributor

openshift-ci bot commented May 6, 2024

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

- chain: openshift-e2e-test-hypershift-qe
- ref: cucushift-hypershift-extended-health-check
- ref: openshift-extended-test
- ref: openshift-extended-web-tests
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ref: openshift-extended-web-tests is currently unsupported for this type of configuration.

@fxierh
Copy link
Contributor

fxierh commented May 6, 2024

You might find the following debug job useful:

- as: hypershift-ext-oidc-debug
  cron: '@yearly'
  steps:
    cluster_profile: aws-qe
    env:
      TEST_FILTERS: ~ChkUpgrade&;~DisconnectedOnly&;~NonPreRelease&;~HyperShiftMGMT&;~MicroShiftOnly&;~NonHyperShiftHOST&;~Serial&;~Disruptive&
    post:
    - as: dummy-post
      commands: sleep 3600
      from: tools
      resources:
        limits:
          memory: 1Gi
        requests:
          cpu: "1"
          memory: 1Gi
      timeout: 2h0m0s
    pre:
    - as: dummy-pre
      commands: sleep 1
      from: tools
      resources:
        limits:
          memory: 1Gi
        requests:
          cpu: "1"
          memory: 1Gi
      timeout: 10m0s
    test:
    - as: download-kubeconfig
      commands: |
        set -euxo pipefail

        export AWS_SHARED_CREDENTIALS_FILE="${CLUSTER_PROFILE_DIR}/.awscred"
        aws s3 cp s3://fxie-test/hcp.kubeconfig ${SHARED_DIR}/kubeconfig
        oc get co --kubeconfig ${SHARED_DIR}/kubeconfig

        echo "UjMTK-DWpjr-XFKaD-DjMIh" > ${SHARED_DIR}/kubeadmin-password
      from_image:
        name: "4.16"
        namespace: ocp
        tag: upi-installer
      resources:
        limits:
          memory: 1Gi
        requests:
          cpu: "1"
          memory: 1Gi
      timeout: 10m0s
    - ref: openshift-extended-web-tests-ext-oidc-cli-login
    - ref: cucushift-hypershift-extended-external-oidc-grant-user-role
    - ref: openshift-extended-test
    - as: post-extended-test
      commands: |
        set -euxo pipefail

        echo "Checking the existence of external OIDC cache files"
        if [[ ! -r "$SHARED_DIR/oc-oidc-token" ]]; then
          echo "SHARED_DIR/oc-oidc-token is either not found or not readable, exiting"
          exit 1
        fi
        cat "$SHARED_DIR/oc-oidc-token"
        if [[ ! -r "$SHARED_DIR/oc-oidc-token-filename" ]]; then
          echo "SHARED_DIR/oc-oidc-token-filename is either not found or not readable, exiting"
          exit 1
        fi
        cat "$SHARED_DIR/oc-oidc-token-filename"

        echo "Restoring external OIDC cache dir"
        mkdir -p ~/.kube/cache/oc
        cat "$SHARED_DIR/oc-oidc-token" > ~/.kube/cache/oc/"$(cat "$SHARED_DIR/oc-oidc-token-filename")"

        oc whoami
        oc get co
      from: tools
      resources:
        limits:
          memory: 1Gi
        requests:
          cpu: "1"
          memory: 1Gi
      timeout: 10m0s

Comment on lines 84 to 142
export EXTERNAL_AUTH_PROVIDERS=" externalAuthProviders:
- name: entra-id
issuer:
issuerURL: ${ISSUER_URL}
audiences:
- ${CONSOLE_CLIENT_ID}
- ${CLI_CLIENT_ID}
oidcClients:
- componentName: console
componentNamespace: openshift-console
clientID: ${CONSOLE_CLIENT_ID}
clientSecret:
name: ${CONSOLE_CLIENT_SECRET_NAME}
claimMappings:
username:
claim: email
prefixPolicy:
prefix: \"oidc-user-test:\"
groups:
claim: groups
prefix: \"oidc-groups-test:\""
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not 100% sure about this but we may need to add the cli OIDC client as well:

Suggested change
export EXTERNAL_AUTH_PROVIDERS=" externalAuthProviders:
- name: entra-id
issuer:
issuerURL: ${ISSUER_URL}
audiences:
- ${CONSOLE_CLIENT_ID}
- ${CLI_CLIENT_ID}
oidcClients:
- componentName: console
componentNamespace: openshift-console
clientID: ${CONSOLE_CLIENT_ID}
clientSecret:
name: ${CONSOLE_CLIENT_SECRET_NAME}
claimMappings:
username:
claim: email
prefixPolicy:
prefix: \"oidc-user-test:\"
groups:
claim: groups
prefix: \"oidc-groups-test:\""
}
export EXTERNAL_AUTH_PROVIDERS=" externalAuthProviders:
- name: entra-id
issuer:
issuerURL: ${ISSUER_URL}
audiences:
- ${CONSOLE_CLIENT_ID}
- ${CLI_CLIENT_ID}
oidcClients:
- clientID: ...
componentName: cli
componentNamespace: openshift-console
- componentName: console
componentNamespace: openshift-console
clientID: ${CONSOLE_CLIENT_ID}
clientSecret:
name: ${CONSOLE_CLIENT_SECRET_NAME}
claimMappings:
username:
claim: email
prefixPolicy:
prefix: \"oidc-user-test:\"
groups:
claim: groups
prefix: \"oidc-groups-test:\""
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/cc @yanpzhan

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done, I have not tested this scenario. Let's debug it in this pr.thx

@openshift-ci openshift-ci bot requested a review from yanpzhan May 6, 2024 08:00
@heliubj18
Copy link
Contributor Author

/pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.15-amd64-nightly-aws-rosa-hcp-capi-private-stage-critical-f7

@openshift-ci-robot
Copy link
Contributor

@heliubj18: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@heliubj18
Copy link
Contributor Author

/pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.15-amd64-nightly-aws-rosa-hcp-capi-private-stage-critical-f7

@openshift-ci-robot
Copy link
Contributor

@heliubj18: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@heliubj18
Copy link
Contributor Author

/pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.15-amd64-nightly-aws-rosa-hcp-capi-private-stage-critical-f7

@openshift-ci-robot
Copy link
Contributor

@heliubj18: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@heliubj18
Copy link
Contributor Author

/pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.15-amd64-nightly-aws-rosa-hcp-capi-private-stage-critical-f7

@openshift-ci-robot
Copy link
Contributor

@heliubj18: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@fxierh
Copy link
Contributor

fxierh commented May 7, 2024

Are the jobs expected to be included in version 4.15? If so, I will need to cherry-pick my changes from openshift-tests-private back into the corresponding branch.

@heliubj18
Copy link
Contributor Author

/pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.15-amd64-nightly-aws-rosa-hcp-capi-private-stage-critical-f7

@openshift-ci-robot
Copy link
Contributor

@heliubj18: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@heliubj18
Copy link
Contributor Author

/pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.15-amd64-nightly-aws-rosa-hcp-capi-private-stage-critical-f7

@openshift-ci-robot
Copy link
Contributor

@heliubj18: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@heliubj18
Copy link
Contributor Author

/pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.15-amd64-nightly-aws-rosa-hcp-capi-private-stage-critical-f7

@openshift-ci-robot
Copy link
Contributor

@heliubj18: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-ci-robot
Copy link
Contributor

@heliubj18, pj-rehearse: unable to set up jobs ERROR:

failed resolve ReleaseBuildConfiguration: Failed resolve MultiStageTestConfiguration: test/aws-rosa-hcp-capi-private-stage-critical-f7: workflow/rosa-aws-sts-hcp-capi-private: parameter "E2E_RUN_TAGS" is overridden in [test/aws-rosa-hcp-capi-private-stage-critical-f7] but not declared in any step

If the problem persists, please contact Test Platform.

@heliubj18
Copy link
Contributor Author

/pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.15-amd64-nightly-aws-rosa-hcp-capi-private-stage-critical-f7

@openshift-ci-robot
Copy link
Contributor

@heliubj18: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@heliubj18
Copy link
Contributor Author

/pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.15-amd64-nightly-aws-rosa-hcp-capi-private-stage-critical-f7

@openshift-ci-robot
Copy link
Contributor

@heliubj18: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@heliubj18
Copy link
Contributor Author

/pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.15-amd64-nightly-aws-rosa-hcp-capi-private-stage-critical-f7

Copy link
Contributor

openshift-ci bot commented May 27, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: heliubj18
Once this PR has been reviewed and has the lgtm label, please assign jianzhangbjz for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@heliubj18
Copy link
Contributor Author

/pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.15-amd64-nightly-aws-rosa-hcp-capi-private-stage-critical-f7

@openshift-ci-robot
Copy link
Contributor

@heliubj18: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-ci-robot
Copy link
Contributor

[REHEARSALNOTIFIER]
@heliubj18: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name Repo Type Reason
pull-ci-rh-ecosystem-edge-recert-main-e2e-aws-ovn-single-node-recert-openshift-e2e-test-qe rh-ecosystem-edge/recert presubmit Registry content changed
pull-ci-rh-ecosystem-edge-recert-release-4.16-e2e-aws-ovn-single-node-recert-openshift-e2e-test-qe rh-ecosystem-edge/recert presubmit Registry content changed
pull-ci-rh-ecosystem-edge-recert-release-4.15-e2e-aws-ovn-single-node-recert-openshift-e2e-test-qe rh-ecosystem-edge/recert presubmit Registry content changed
pull-ci-rh-ecosystem-edge-recert-release-4.14-e2e-aws-ovn-single-node-recert-openshift-e2e-test-qe rh-ecosystem-edge/recert presubmit Registry content changed
pull-ci-rh-ecosystem-edge-recert-v0-e2e-aws-ovn-single-node-recert-openshift-e2e-test-qe rh-ecosystem-edge/recert presubmit Registry content changed
pull-ci-openshift-aws-ebs-csi-driver-master-e2e-aws-csi-extended openshift/aws-ebs-csi-driver presubmit Registry content changed
pull-ci-openshift-aws-ebs-csi-driver-release-4.18-e2e-aws-csi-extended openshift/aws-ebs-csi-driver presubmit Registry content changed
pull-ci-openshift-aws-ebs-csi-driver-release-4.17-e2e-aws-csi-extended openshift/aws-ebs-csi-driver presubmit Registry content changed
pull-ci-openshift-aws-ebs-csi-driver-release-4.16-e2e-aws-csi-extended openshift/aws-ebs-csi-driver presubmit Registry content changed
pull-ci-openshift-aws-ebs-csi-driver-release-4.15-e2e-aws-csi-extended openshift/aws-ebs-csi-driver presubmit Registry content changed
pull-ci-openshift-aws-ebs-csi-driver-release-4.14-e2e-aws-csi-extended openshift/aws-ebs-csi-driver presubmit Registry content changed
pull-ci-openshift-ibm-vpc-block-csi-driver-operator-master-e2e-ibmcloud-csi-extended openshift/ibm-vpc-block-csi-driver-operator presubmit Registry content changed
pull-ci-openshift-ibm-vpc-block-csi-driver-operator-release-4.18-e2e-ibmcloud-csi-extended openshift/ibm-vpc-block-csi-driver-operator presubmit Registry content changed
pull-ci-openshift-ibm-vpc-block-csi-driver-operator-release-4.17-e2e-ibmcloud-csi-extended openshift/ibm-vpc-block-csi-driver-operator presubmit Registry content changed
pull-ci-openshift-ibm-vpc-block-csi-driver-operator-release-4.16-e2e-ibmcloud-csi-extended openshift/ibm-vpc-block-csi-driver-operator presubmit Registry content changed
pull-ci-openshift-ibm-vpc-block-csi-driver-operator-release-4.15-e2e-ibmcloud-csi-extended openshift/ibm-vpc-block-csi-driver-operator presubmit Registry content changed
pull-ci-openshift-ibm-vpc-block-csi-driver-operator-release-4.14-e2e-ibmcloud-csi-extended openshift/ibm-vpc-block-csi-driver-operator presubmit Registry content changed
pull-ci-openshift-ibm-vpc-block-csi-driver-master-e2e-ibmcloud-csi-extended openshift/ibm-vpc-block-csi-driver presubmit Registry content changed
pull-ci-openshift-ibm-vpc-block-csi-driver-release-4.18-e2e-ibmcloud-csi-extended openshift/ibm-vpc-block-csi-driver presubmit Registry content changed
pull-ci-openshift-ibm-vpc-block-csi-driver-release-4.17-e2e-ibmcloud-csi-extended openshift/ibm-vpc-block-csi-driver presubmit Registry content changed
pull-ci-openshift-ibm-vpc-block-csi-driver-release-4.16-e2e-ibmcloud-csi-extended openshift/ibm-vpc-block-csi-driver presubmit Registry content changed
pull-ci-openshift-ibm-vpc-block-csi-driver-release-4.15-e2e-ibmcloud-csi-extended openshift/ibm-vpc-block-csi-driver presubmit Registry content changed
pull-ci-openshift-ibm-vpc-block-csi-driver-release-4.14-e2e-ibmcloud-csi-extended openshift/ibm-vpc-block-csi-driver presubmit Registry content changed
pull-ci-openshift-azure-file-csi-driver-master-e2e-azure-file-csi-extended openshift/azure-file-csi-driver presubmit Registry content changed
pull-ci-openshift-azure-file-csi-driver-release-4.18-e2e-azure-file-csi-extended openshift/azure-file-csi-driver presubmit Registry content changed

A total of 1260 jobs have been affected by this change. The above listing is non-exhaustive and limited to 25 jobs.

A full list of affected jobs can be found here

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse abort to abort all active rehearsals

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

Copy link
Contributor

openshift-ci bot commented May 27, 2024

@heliubj18: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/rehearse/periodic-ci-openshift-openshift-tests-private-release-4.15-amd64-nightly-aws-rosa-hcp-capi-stage-critical-f2 4d014bb link unknown /pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.15-amd64-nightly-aws-rosa-hcp-capi-stage-critical-f2
ci/rehearse/periodic-ci-openshift-openshift-tests-private-release-4.15-amd64-nightly-aws-rosa-hcp-capi-private-stage-critical-f7 2ce9e71 link unknown /pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.15-amd64-nightly-aws-rosa-hcp-capi-private-stage-critical-f7

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@heliubj18
Copy link
Contributor Author

replaced by #52499

@heliubj18
Copy link
Contributor Author

/close

@openshift-merge-robot
Copy link
Contributor

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 28, 2024
@openshift-ci openshift-ci bot closed this May 28, 2024
Copy link
Contributor

openshift-ci bot commented May 28, 2024

@heliubj18: Closed this PR.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants