Skip to content

bug: hot-loop: cluster_authentication_trust_controller keeps trying to update its configmap #2325

Closed
@ncdc

Description

Describe the bug

When running in a sharded setup, cluster_authentication_trust_controller gets into a hotloop

Steps To Reproduce

  1. make
  2. bin/sharded-test-server
  3. Observe .kcp-0/kcp.log:
I1107 22:31:37.569013     761 cluster_authentication_trust_controller.go:165] writing updated authentication info to  kube-system configmaps/extension-apiserver-authentication
I1107 22:31:37.569822     761 httplog.go:131] "HTTP" verb="GET" URI="/clusters/root/api/v1/namespaces/kube-system" latency="659.302µs" userAgent="kcp/v1.24.3+kcp (linux/amd64) kubernetes/c5130b2" audit-ID="aa2bc8de-f15b-4433-b413-e3aac0ec93c8" srcIP="[::1]:54568" resp=200
I1107 22:31:37.571202     761 httplog.go:131] "HTTP" verb="PUT" URI="/clusters/root/api/v1/namespaces/kube-system/configmaps/extension-apiserver-authentication" latency="1.039978ms" userAgent="kcp/v1.24.3+kcp (linux/amd64) kubernetes/c5130b2" audit-ID="5817d2ca-4aed-4e19-beef-b88e2d6bfa83" srcIP="[::1]:54568" resp=200
I1107 22:31:37.571277     761 resource_controller.go:221] "queueing resource" reconciler="kcp-workload-resource-scheduler" key="configmaps.v1.::root|kube-system/extension-apiserver-authentication"
I1107 22:31:37.571303     761 permissionclaimlabel_resource_controller.go:105] "queuing resource" reconciler="kcp-permissionclaimlabel" key="configmaps.v1.::root|kube-system/extension-apiserver-authentication"
I1107 22:31:37.571321     761 permissionclaimlabel_resource_controller.go:141] "processing key" reconciler="kcp-resource-permissionclaimlabel" key="configmaps.v1.::root|kube-system/extension-apiserver-authentication"
I1107 22:31:37.571323     761 resource_controller.go:299] "processing key" component="kcp" reconciler="kcp-workload-resource-scheduler" key="configmaps.v1.::root|kube-system/extension-apiserver-authentication"
I1107 22:31:37.571348     761 resource_reconcile.go:48] "reconciling resource" reconciler="kcp-workload-resource-scheduler" partialobjectmetadata.workspace="root" partialobjectmetadata.namespace="kube-system" partialobjectmetadata.name="extension-apiserver-authentication" partialobjectmetadata.apiVersion="meta.k8s.io/v1" groupVersionResource="/v1, Resource=configmaps" logicalCluster="root"
I1107 22:31:37.571364     761 resource_reconcile.go:68] "skipping syncing namespace because it is in the block list" reconciler="kcp-workload-resource-scheduler" partialobjectmetadata.workspace="root" partialobjectmetadata.namespace="kube-system" partialobjectmetadata.name="extension-apiserver-authentication" partialobjectmetadata.apiVersion="meta.k8s.io/v1" groupVersionResource="/v1, Resource=configmaps" logicalCluster="root" namespace="kube-system"
I1107 22:31:37.571541     761 cluster_authentication_trust_controller.go:165] writing updated authentication info to  kube-system configmaps/extension-apiserver-authentication
I1107 22:31:37.572434     761 httplog.go:131] "HTTP" verb="GET" URI="/clusters/root/api/v1/namespaces/kube-system" latency="690.932µs" userAgent="kcp/v1.24.3+kcp (linux/amd64) kubernetes/c5130b2" audit-ID="d157448f-566e-4418-a99e-4440c94d30a0" srcIP="[::1]:54568" resp=200
I1107 22:31:37.573972     761 httplog.go:131] "HTTP" verb="PUT" URI="/clusters/root/api/v1/namespaces/kube-system/configmaps/extension-apiserver-authentication" latency="1.059628ms" userAgent="kcp/v1.24.3+kcp (linux/amd64) kubernetes/c5130b2" audit-ID="ba8ed41d-9f61-47b3-8926-8fd7dc41b7ab" srcIP="[::1]:54568" resp=200
I1107 22:31:37.574082     761 resource_controller.go:221] "queueing resource" reconciler="kcp-workload-resource-scheduler" key="configmaps.v1.::root|kube-system/extension-apiserver-authentication"
I1107 22:31:37.574108     761 permissionclaimlabel_resource_controller.go:105] "queuing resource" reconciler="kcp-permissionclaimlabel" key="configmaps.v1.::root|kube-system/extension-apiserver-authentication"
I1107 22:31:37.574130     761 permissionclaimlabel_resource_controller.go:141] "processing key" reconciler="kcp-resource-permissionclaimlabel" key="configmaps.v1.::root|kube-system/extension-apiserver-authentication"
I1107 22:31:37.574165     761 resource_controller.go:299] "processing key" component="kcp" reconciler="kcp-workload-resource-scheduler" key="configmaps.v1.::root|kube-system/extension-apiserver-authentication"
I1107 22:31:37.574198     761 resource_reconcile.go:48] "reconciling resource" reconciler="kcp-workload-resource-scheduler" partialobjectmetadata.workspace="root" partialobjectmetadata.namespace="kube-system" partialobjectmetadata.name="extension-apiserver-authentication" partialobjectmetadata.apiVersion="meta.k8s.io/v1" groupVersionResource="/v1, Resource=configmaps" logicalCluster="root"
I1107 22:31:37.574221     761 resource_reconcile.go:68] "skipping syncing namespace because it is in the block list" reconciler="kcp-workload-resource-scheduler" partialobjectmetadata.workspace="root" partialobjectmetadata.namespace="kube-system" partialobjectmetadata.name="extension-apiserver-authentication" partialobjectmetadata.apiVersion="meta.k8s.io/v1" groupVersionResource="/v1, Resource=configmaps" logicalCluster="root" namespace="kube-system"
I1107 22:31:37.574380     761 cluster_authentication_trust_controller.go:165] writing updated authentication info to  kube-system configmaps/extension-apiserver-authentication

Expected Behaviour

No hotloop

Additional Context

For some reason, this keeps thinking the old and new data are different:

if equality.Semantic.DeepEqual(authConfigMap, originalAuthConfigMap) {
		klog.V(5).Info("no changes to configmap")
		return nil
	}
	klog.V(2).Infof("writing updated authentication info to  %s configmaps/%s", configMapNamespace, configMapName)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

Labels

kind/bugCategorizes issue or PR as related to a bug.

Type

No type

Projects

  • Status

    Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions