CNTRLPLANE-2247:Add kms-plugin ci job by gangwgr · Pull Request #73750 · openshift/release

gangwgr · 2026-01-20T17:10:13Z

openshift-ci · 2026-01-20T17:10:20Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

gangwgr · 2026-01-20T17:10:40Z

/retest

openshift-ci-robot · 2026-01-20T17:34:11Z

@gangwgr: This pull request references CNTRLPLANE-2247 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

This PR introduces reusable step-registry components for deploying and testing KMS v1/v2 encryption in OpenShift CI jobs. The implementation uses a DaemonSet-based approach which is platform-agnostic, fast, and doesn't require node reboots.

Currently, testing KMS encryption in apiserver operators (cluster-kube-apiserver-operator, cluster-openshift-apiserver-operator, cluster-authentication-operator) requires manual plugin deployment or complex MachineConfig setups. This PR provides a simple, reusable solution that works across all platforms (AWS, GCP, Azure, bare metal).

Related Issues

Enhancement Proposal: CNTRLPLANE-2120: Add KMS foundations in encryption controllers in library-go enhancements#1900

JIRA Ticket: https://issues.redhat.com/browse/CNTRLPLANE-2247

Co-authored By - Claude

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

gangwgr · 2026-01-21T04:35:56Z

/pj-rehearse

openshift-ci-robot · 2026-01-21T04:35:58Z

@gangwgr: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

ardaguclu

This is much nicer than mine. I dropped some comments.

ardaguclu · 2026-01-21T06:55:20Z

ci-operator/step-registry/kms/mock-plugin/README.md

+### Steps
+
+- **kms-mock-plugin-deploy**: Deploys the mock KMS plugin as a DaemonSet on control plane nodes
+  - Supports KMSv1 and KMSv2 via `KMS_VERSION` environment variable (default: v2)


We definitely don't need anything about KMS v1. We don't need KMS_VERSION env var.

ardaguclu · 2026-01-21T06:56:40Z

ci-operator/step-registry/kms/mock-plugin/README.md

+
+1. **Init Container**: Builds the mock KMS plugin from kubernetes/kubernetes repo
+2. **Main Container**: Runs the plugin listening on `/var/run/kmsplugin/socket.sock`
+3. **HostPath Mount**: Exposes the socket to the host so kube-apiserver can access it


This will be handled by openshift/enhancements#1900 EP.

ardaguclu · 2026-01-21T06:56:50Z

ci-operator/step-registry/kms/mock-plugin/README.md

+### How It Works
+
+1. **Init Container**: Builds the mock KMS plugin from kubernetes/kubernetes repo
+2. **Main Container**: Runs the plugin listening on `/var/run/kmsplugin/socket.sock`


Suggested change

2. **Main Container**: Runs the plugin listening on `/var/run/kmsplugin/socket.sock`

2. **Main Container**: Runs the plugin listening on `/var/run/kmsplugin/kms.sock`

ardaguclu · 2026-01-21T07:01:53Z

ci-operator/step-registry/kms/mock-plugin/configure/kms-mock-plugin-configure-commands.sh

+oc patch apiserver cluster --type=merge -p '{
+  "spec": {
+    "encryption": {
+      "type": "aescbc"


Suggested change

"type": "aescbc"

"type": "KMS"

ardaguclu · 2026-01-21T07:02:09Z

ci-operator/step-registry/kms/mock-plugin/configure/kms-mock-plugin-configure-commands.sh

+  name: cluster
+spec:
+  encryption:
+    type: aescbc


Suggested change

type: aescbc

type: KMS

ardaguclu · 2026-01-21T07:02:24Z

ci-operator/step-registry/kms/mock-plugin/configure/kms-mock-plugin-configure-commands.sh

+  encryption:
+    type: aescbc
+---
+apiVersion: v1


This resource is not needed

ardaguclu · 2026-01-21T07:46:03Z

ci-operator/step-registry/kms/mock-plugin/verify/kms-mock-plugin-verify-commands.sh

+    echo "  ✓ Socket verified"
+  else
+    echo "  ERROR: Socket not found at ${KMS_SOCKET}"
+    oc exec -n "${KMS_NAMESPACE}" "${POD}" -- ls -la /var/run/kmsplugin/ || true


This is redundant?

ardaguclu · 2026-01-21T07:48:54Z

ci-operator/step-registry/kms/mock-plugin/deploy/kms-mock-plugin-deploy-commands.sh

+
+# Validate and set KMS_VERSION
+KMS_VERSION="${KMS_VERSION:-v2}"
+if [[ "${KMS_VERSION}" != "v1" && "${KMS_VERSION}" != "v2" ]]; then


ardaguclu · 2026-01-21T07:49:02Z

ci-operator/step-registry/kms/mock-plugin/deploy/kms-mock-plugin-deploy-commands.sh

+echo "Deploying mock KMS ${KMS_VERSION} plugin DaemonSet"
+echo "========================================="
+
+SOCKET_PATH="/var/run/kmsplugin/socket.sock"


Suggested change

SOCKET_PATH="/var/run/kmsplugin/socket.sock"

SOCKET_PATH="/var/run/kmsplugin/kms.sock"

ardaguclu · 2026-01-21T07:49:56Z

ci-operator/step-registry/kms/mock-plugin/deploy/kms-mock-plugin-deploy-commands.sh

+
+# Create namespace for KMS plugin
+echo "Creating namespace ${KMS_NAMESPACE}..."
+oc create namespace "${KMS_NAMESPACE}" || echo "Namespace already exists"


Namespace should be privileged to access host file

ardaguclu · 2026-01-21T07:53:51Z

...penshift/cluster-kube-apiserver-operator/openshift-cluster-kube-apiserver-operator-main.yaml

+  steps:
+    cluster_profile: aws
+    env:
+      TEST_SUITE: openshift/cluster-kube-apiserver-operator/encryption/kms


Can we update here to not test anything in order to ensure that chains successfully deploy kms-plugin. Deploying kms-plugin require many steps and it would be awesome, if we ensure that this flow simply works.

gangwgr · 2026-01-21T08:23:40Z

/pj-rehearse

openshift-ci-robot · 2026-01-21T08:23:43Z

@gangwgr: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

ardaguclu · 2026-01-21T09:07:59Z

ci-operator/step-registry/kms/mock-plugin/deploy/kms-mock-plugin-deploy-commands.sh

+        - |
+          set -euxo pipefail
+          echo "Starting mock KMS v2 plugin..."
+          mkdir -p /var/run/kmsplugin


In L:130 we create this directory already, we don't need this.

ardaguclu · 2026-01-21T09:09:15Z

ci-operator/step-registry/kms/mock-plugin/deploy/kms-mock-plugin-deploy-commands.sh

+
+# Wait for pods to be created first
+echo "Waiting for pods to be created..."
+for i in {1..60}; do


Wait command in L:156 already waits, we may not need this for loop?

ardaguclu · 2026-01-21T09:10:54Z

ci-operator/step-registry/kms/mock-plugin/configure/kms-mock-plugin-configure-commands.sh

+# Create KMS configuration secret
+echo "Creating KMS v2 configuration..."
+cat <<EOF | oc apply -f -
+apiVersion: v1


We don't need this secret. According to this EP openshift/enhancements#1900, once apiserver config is configured for KMS, controller will automatically generate encryptionconfiguration.

ardaguclu · 2026-01-21T09:11:46Z

ci-operator/step-registry/kms/mock-plugin/configure/kms-mock-plugin-configure-ref.yaml

+
+    The step will:
+      1. Read the KMS socket path from SHARED_DIR
+      2. Generate EncryptionConfiguration for KMS v2


ardaguclu · 2026-01-21T09:12:36Z

ci-operator/step-registry/kms/mock-plugin/verify/kms-mock-plugin-verify-commands.sh

+
+KMS_SOCKET=$(cat "${SHARED_DIR}/kms-plugin-socket-path")
+KMS_NAMESPACE=$(cat "${SHARED_DIR}/kms-plugin-namespace")
+KMS_VERSION=$(cat "${SHARED_DIR}/kms-plugin-version")


ardaguclu · 2026-01-21T09:13:25Z

...penshift/cluster-kube-apiserver-operator/openshift-cluster-kube-apiserver-operator-main.yaml

          cpu: 100m
      timeout: 4h0m0s
    workflow: ipi-gcp
+- always_run: false


If this works, that is great

it does, we use it for the other tests too.

the downside is that the tests are executed only when the files specified below are changed. we will change this with OTE so that we can catch potential regression.

gangwgr · 2026-01-21T09:34:27Z

/pj-rehearse

openshift-ci-robot · 2026-01-21T09:34:31Z

@gangwgr: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

p0lyn0mial · 2026-01-23T14:31:40Z

...penshift/cluster-kube-apiserver-operator/openshift-cluster-kube-apiserver-operator-main.yaml

      timeout: 4h0m0s
    workflow: ipi-gcp
+- always_run: false
+  as: e2e-aws-operator-encryption-kms


ah, this is now gcp

p0lyn0mial · 2026-01-23T14:32:22Z

/lgtm cancel

for #73750 (comment)

openshift-ci-robot · 2026-01-23T15:09:30Z

[REHEARSALNOTIFIER]
@gangwgr: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name	Repo	Type	Reason
pull-ci-openshift-cluster-kube-apiserver-operator-main-e2e-gcp-operator-encryption-kms	openshift/cluster-kube-apiserver-operator	presubmit	Presubmit changed

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

p0lyn0mial · 2026-01-26T10:16:34Z

/retitle CNTRLPLANE-2247:Add kms-plugin ci job

p0lyn0mial · 2026-01-26T10:19:57Z

/pj-rehearse

openshift-ci-robot · 2026-01-26T10:20:00Z

@p0lyn0mial: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

ardaguclu · 2026-01-26T10:38:56Z

/pj-rehearse pull-ci-openshift-cluster-kube-apiserver-operator-main-e2e-gcp-operator-encryption-kms

openshift-ci-robot · 2026-01-26T10:38:59Z

@ardaguclu: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

p0lyn0mial · 2026-01-26T10:39:00Z

/pj-rehearse

openshift-ci-robot · 2026-01-26T10:39:03Z

@p0lyn0mial: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

p0lyn0mial · 2026-01-26T12:47:23Z

/lgtm

openshift-ci · 2026-01-26T12:48:09Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: gangwgr, p0lyn0mial

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~ci-operator/config/openshift/cluster-kube-apiserver-operator/OWNERS~~ [p0lyn0mial]
~~ci-operator/jobs/openshift/cluster-kube-apiserver-operator/OWNERS~~ [p0lyn0mial]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

p0lyn0mial · 2026-01-26T12:53:52Z

/pj-rehearse ack

openshift-ci-robot · 2026-01-26T12:53:55Z

@p0lyn0mial: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 20, 2026

gangwgr marked this pull request as ready for review January 20, 2026 17:10

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 20, 2026

gangwgr changed the title ~~Add kms-plugin installation step to be reused by apiserver~~ [WIP]Add kms-plugin installation step to be reused by apiserver Jan 20, 2026

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 20, 2026

openshift-ci bot requested review from benluddy and sanchezl January 20, 2026 17:12

gangwgr force-pushed the kms-ci branch from ba58d68 to 0f98630 Compare January 20, 2026 17:28

gangwgr changed the title ~~[WIP]Add kms-plugin installation step to be reused by apiserver~~ [WIP]CNTRLPLANE-2247:Add kms-plugin installation step to be reused by apiserver Jan 20, 2026

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jan 20, 2026

gangwgr force-pushed the kms-ci branch from 0f98630 to 0ee75c5 Compare January 21, 2026 04:24

gangwgr force-pushed the kms-ci branch from 0ee75c5 to 710e64e Compare January 21, 2026 06:58

ardaguclu reviewed Jan 21, 2026

View reviewed changes

gangwgr force-pushed the kms-ci branch 3 times, most recently from 49a585d to 8aae987 Compare January 21, 2026 08:16

ardaguclu reviewed Jan 21, 2026

View reviewed changes

gangwgr force-pushed the kms-ci branch from 8aae987 to 79591a8 Compare January 21, 2026 09:20

gangwgr force-pushed the kms-ci branch 2 times, most recently from 073f54a to c11c6f2 Compare January 21, 2026 13:53

openshift-ci bot added lgtm Indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Jan 23, 2026

p0lyn0mial reviewed Jan 23, 2026

View reviewed changes

openshift-ci bot removed lgtm Indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Jan 23, 2026

gangwgr added 2 commits January 23, 2026 20:35

Adjust KMS encryption test config

f525343

Update generated presubmit config

7222fdb

gangwgr force-pushed the kms-ci branch from 3b7ff6a to 7222fdb Compare January 23, 2026 15:06

openshift-ci bot changed the title ~~[WIP]CNTRLPLANE-2247:Add kms-plugin ci job~~ CNTRLPLANE-2247:Add kms-plugin ci job Jan 26, 2026

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 26, 2026

ardaguclu mentioned this pull request Jan 26, 2026

OCPBUGS-68343: Introduce KMSEncryption feature gate openshift/api#2669

Merged

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jan 26, 2026

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 26, 2026

openshift-ci-robot added the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Jan 26, 2026

openshift-merge-bot bot merged commit cbebae6 into openshift:master Jan 26, 2026
16 checks passed

This was referenced Jan 27, 2026

CNTRLPLANE-2247: Add KMS CI job for openshift-apiserver-operator #74006

Merged

CNTRLPLANE-2247: Add KMS CI job for cluster-authentication-operator #74013

Merged

	2. Main Container: Runs the plugin listening on `/var/run/kmsplugin/socket.sock`
	2. Main Container: Runs the plugin listening on `/var/run/kmsplugin/kms.sock`

	SOCKET_PATH="/var/run/kmsplugin/socket.sock"
	SOCKET_PATH="/var/run/kmsplugin/kms.sock"

Conversation

gangwgr commented Jan 20, 2026 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci bot commented Jan 20, 2026

Uh oh!

gangwgr commented Jan 20, 2026

Uh oh!

openshift-ci-robot commented Jan 20, 2026 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gangwgr commented Jan 21, 2026

Uh oh!

openshift-ci-robot commented Jan 21, 2026

Uh oh!

ardaguclu left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gangwgr commented Jan 21, 2026

Uh oh!

openshift-ci-robot commented Jan 21, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gangwgr commented Jan 21, 2026

Uh oh!

openshift-ci-robot commented Jan 21, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

p0lyn0mial commented Jan 23, 2026

Uh oh!

openshift-ci-robot commented Jan 23, 2026

Uh oh!

p0lyn0mial commented Jan 26, 2026 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

p0lyn0mial commented Jan 26, 2026

Uh oh!

openshift-ci-robot commented Jan 26, 2026

Uh oh!

ardaguclu commented Jan 26, 2026

Uh oh!

openshift-ci-robot commented Jan 26, 2026

gangwgr commented Jan 20, 2026 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Jan 20, 2026 •

edited by openshift-ci bot

Loading

p0lyn0mial commented Jan 26, 2026 •

edited by openshift-ci bot

Loading