Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix PodAntiAffinity issues in case of multiple affinityTerms #68173

Merged

Conversation

Huang-Wei
Copy link
Member

@Huang-Wei Huang-Wei commented Sep 2, 2018

  • rename podMatchesAffinityTermProperties to podMatchesAllAffinityTermProperties
  • add podMatchesAnyAffinityTermProperties which should be used in some PodAntiAffinity cases
  • add unit tests

What this PR does / why we need it:

There are 2 phases to verify symmetry of PodAntiAffinity:

  • verify labels of incoming pod not violates antiaffinity of existing pods
  • verify antiaffinity of incoming pod not violates labels of existing pods

This PR is to fix issue in second step.

Currently logic of second step tries to invoke podMatchesAffinityTermProperties when checking both affinity and antiaffinity.

It's not that accurate, as podMatchesAffinityTermProperties is a "match all" semantics, say for each term, ensure it matches target selector/labels, and return true if all term passes check. This is good for affinity, but not antiaffinity.

Regarding antiaffinity, it's a "match any" semantics - i.e. if found any term match target selector/labels, return true.

So without this fix, unit tests (which is added in this PR) failed on following cases:

  • a) incoming pod's anti-affinity has more than one term - existing pod violates all terms
  • b) incoming pod's anti-affinity has more than one term - existing pod violates partial term - case 1
  • c) incoming pod's anti-affinity has more than one term - existing pod violates partial term - case 2

Case a) is guarded in before-mentioned step 1 of symmetric logic. So we don't see it in a running k8s cluster. Case b) and c) are real problems and specifically case c) is what issue #68101 reported.

Which issue(s) this PR fixes:
Fixes #68101

Special notes for your reviewer:
After reviewing the correctness of this PR, bsalamat and k82cn can decide if it needs to go into 1.12 release cycle.

Release note:

Fix PodAntiAffinity issues in case of multiple affinityTerms.

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. sig/scheduling Categorizes an issue or PR as relevant to SIG Scheduling. labels Sep 2, 2018
@Huang-Wei
Copy link
Member Author

/sig scheduling

@Huang-Wei
Copy link
Member Author

@Huang-Wei
Copy link
Member Author

/retest

},
},
},
wantAffinityPods: make(map[string][]*v1.Pod),
Copy link
Contributor

@resouer resouer Sep 3, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You used nil above instead of make(), should be consistent :D

btw, I think nil is fine.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, lol...

getPodsMatchingAffinity simply return nil if it doesn't have affinity/antiaffinity:

func getPodsMatchingAffinity(pod *v1.Pod, nodeInfoMap map[string]*schedulercache.NodeInfo) (affinityPods map[string][]*v1.Pod, antiAffinityPods map[string][]*v1.Pod, err error) {
allNodeNames := make([]string, 0, len(nodeInfoMap))
affinity := pod.Spec.Affinity
if affinity == nil || (affinity.PodAffinity == nil && affinity.PodAntiAffinity == nil) {
return nil, nil, nil
}

But Mr. reflect.DeepEqual is unhappy to treat "nil" as equality of a "nil map" - so in some cases, I put nil, while some I put nil map there to make it happy...

BTW: apiequality.Semantic.DeepEqual is considerate to say yes to "nil equals to nil map". But maybe not worth introducing an extra package dependency.

Copy link
Contributor

@resouer resouer Sep 3, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see, seems like using apiequality.Semantic.DeepEqual is a approach. I noticed that function in other test files.

Copy link
Member Author

@Huang-Wei Huang-Wei Sep 4, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm... Have to rebase code as PR #67788 refactored the data structure.

Now it's all consistent. PTAL.


// TestGetPodsMatchingAffinity_AntiAffinity tests againt method getPodsMatchingAffinity
// on Anti Affinity cases
func TestGetPodsMatchingAffinity_AntiAffinity(t *testing.T) {
Copy link
Contributor

@resouer resouer Sep 3, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a very detailed test, thanks!

Maybe I am verbose, but I am thinking if you can add a test case to verify this bug is fixed in predicates.go as well?

Because we still have many logics there after we got the antiAffinityPods from metadata, so it would be great if we can ensure this change does work and will not break anything in predicates.go.

Sounds reasonable?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@resouer Thanks for the detailed review!

Sure, it's totally reasonable. Do you think is it good enough to add additional UTs for predicates.go#satisfiesPodsAffinityAntiAffinity()? (step 2 of symmetry check this PR tries to fix)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, should be.

Or, I think you can just add one more test case to verify this in TestInterPodAffinity? which seems quicker?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added one testcase - which only passed with this PR. PTAL.

Copy link
Contributor

@resouer resouer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea is valid to me, while some nits need to be fixed first :-)

Also, I would suggest a test case for predicates.go to verify this fix solves the reported issue.

affPods = append(affPods, existingPod)
}
// Check anti-affinity properties.
if podMatchesAffinityTermProperties(existingPod, antiAffinityProperties) {
if podMatchesAnyAffinityTermProperties(existingPod, antiAffinityProperties) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did we check InterPodAffinityMatches ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, implicitly - InterPodAffinityMatches calls satisfiesPodsAffinityAntiAffinity, satisfiesPodsAffinityAntiAffinity needs "meta", and "meta" is generated in GetMetadata where calls getPodsMatchingAffinity.

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 3, 2018
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 4, 2018
@Huang-Wei
Copy link
Member Author

@resouer @k82cn Code is rebased and added testcase in TestInterPodAffinity().

Please take another look. Thanks!

@rphillips
Copy link
Member

/xref kubernetes-retired/bootkube#1001

The changes in this PR are causing affinity sections to appear in checkpoint files causing the Kubelet to not restore pods. The affinity block within the checkpoint manifest errors with the following on kubelet restart:

predicate.go:133] Predicate failed on Pod: pod-checkpointer-l9cgg-172.17.4.101_kube-system(231cd4bc9c8f63b3131c1ec25716fe91), for reason: Predicate MatchNodeSelector failed

Checkpoint manifest json includes:

"affinity": {
      "nodeAffinity": {
        "requiredDuringSchedulingIgnoredDuringExecution": {
          "nodeSelectorTerms": [
            {
              "matchExpressions": null
            }
          ]
        }
      }
    },

Removing the affinity section within the checkpoint manifest fixes the restore issue. I'll be investigating the issue further, but am commenting to see if the upstream authors have any thoughts. This issue is a breaking change on upgrades and new installs for 1.12.x.

@rphillips
Copy link
Member

I suspect setting the affinity to nil when matchExpressions == null/nil would preserve the prior behavior.

@Huang-Wei
Copy link
Member Author

Huang-Wei commented Oct 11, 2018

@rphillips thanks for bringing this up.

If I understand correctly, you're suspicious that after applying this PR, scheduler rejects a pod which has null matchExpression in nodeAffinity?

@Huang-Wei
Copy link
Member Author

Considering this PR doesn't go to v1.11.3, I did a test on v1.11.3:

kind: Pod
apiVersion: v1
metadata:
  name: pause
spec:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions: null
  containers:
  - name: pause
    image: k8s.gcr.io/pause:3.1

The above pod is also rejected. In other words, this PR seems not to break behavior of null matchExpressions.

$ k get po
NAME      READY     STATUS    RESTARTS   AGE
pause     0/1       Pending   0          2m
$ k describe po pause | tail -3
  Type     Reason            Age               From               Message
  ----     ------            ----              ----               -------
  Warning  FailedScheduling  1m (x25 over 3m)  default-scheduler  0/3 nodes are available: 3 node(s) didn't match node selector.

rphillips added a commit to rphillips/bootkube that referenced this pull request Oct 11, 2018
Kubernetes 1.12.x introduced new logic for Affinity [1]. In addition to
new logic, the Pod contains a default affinity. The new default affinity
gets serialized into the checkpoint file, and the 1.12.x kubelet does
not restore the pod due to the affinity.

This PR removes the affinity from the spec and documents that affinity's
are not supported.

```
"affinity": {
      "nodeAffinity": {
        "requiredDuringSchedulingIgnoredDuringExecution": {
          "nodeSelectorTerms": [
            {
              "matchExpressions": null
            }
          ]
        }
      }
    },

```

[1] kubernetes/kubernetes#68173
[2] https://github.com/kubernetes/kubernetes/blob/e39b510726113581c6f6a9c2db1753d794aa9cce/pkg/controller/daemon/util/daemonset_util.go#L183-L196
@rphillips
Copy link
Member

@Huang-Wei Thank you for the prompt reply. The concern is with a static pod the kubelet will restore on restart. Bootkube, Tectonic, and other distributions may use a checkpointer to save the state in a manifest.

core@c1 /etc/kubernetes/manifests $ jq < kube-system-pod-checkpointer-q8zww.json 
{
  "kind": "Pod",
  "apiVersion": "v1",
  "metadata": {
    "name": "pod-checkpointer-q8zww",
    "namespace": "kube-system",
    "creationTimestamp": null,
    "labels": {
      "controller-revision-hash": "6775765d66",
      "k8s-app": "pod-checkpointer",
      "pod-template-generation": "1",
      "tier": "control-plane"
    },
    "annotations": {
      "checkpointer.alpha.coreos.com/checkpoint-of": "pod-checkpointer-q8zww"
    },
    "ownerReferences": [
      {
        "apiVersion": "v1",
        "kind": "Pod",
        "name": "pod-checkpointer-q8zww",
        "uid": "55c42d8e-cd79-11e8-9476-080027385a9b",
        "controller": true
      }
    ]
  },
  "spec": {
    "volumes": [
      {
        "name": "kubeconfig",
        "hostPath": {
          "path": "/etc/kubernetes/checkpoint-configmaps/kube-system/pod-checkpointer-q8zww/kubeconfig-in-cluster"
        }
      },
      {
        "name": "etc-kubernetes",
        "hostPath": {
          "path": "/etc/kubernetes",
          "type": ""
        }
      },
      {
        "name": "var-run",
        "hostPath": {
          "path": "/var/run",
          "type": ""
        }
      },
      {
        "name": "pod-checkpointer-token-ljvdc",
        "hostPath": {
          "path": "/etc/kubernetes/checkpoint-secrets/kube-system/pod-checkpointer-q8zww/pod-checkpointer-token-ljvdc"
        }
      }
    ],
    "containers": [
      {
        "name": "pod-checkpointer",
        "image": "quay.io/coreos/pod-checkpointer:9dc83e1ab3bc36ca25c9f7c18ddef1b91d4a0558",
        "command": [
          "/checkpoint",
          "--lock-file=/var/run/lock/pod-checkpointer.lock",
          "--kubeconfig=/etc/checkpointer/kubeconfig",
          "--checkpoint-grace-period=5m"
        ],
        "env": [
          {
            "name": "NODE_NAME",
            "valueFrom": {
              "fieldRef": {
                "apiVersion": "v1",
                "fieldPath": "spec.nodeName"
              }
            }
          },
          {
            "name": "POD_NAME",
            "valueFrom": {
              "fieldRef": {
                "apiVersion": "v1",
                "fieldPath": "metadata.name"
              }
            }
          },
          {
            "name": "POD_NAMESPACE",
            "valueFrom": {
              "fieldRef": {
                "apiVersion": "v1",
                "fieldPath": "metadata.namespace"
              }
            }
          }
        ],
        "resources": {},
        "volumeMounts": [
          {
            "name": "kubeconfig",
            "mountPath": "/etc/checkpointer"
          },
          {
            "name": "etc-kubernetes",
            "mountPath": "/etc/kubernetes"
          },
          {
            "name": "var-run",
            "mountPath": "/var/run"
          },
          {
            "name": "pod-checkpointer-token-ljvdc",
            "readOnly": true,
            "mountPath": "/var/run/secrets/kubernetes.io/serviceaccount"
          }
        ],
        "terminationMessagePath": "/dev/termination-log",
        "terminationMessagePolicy": "File",
        "imagePullPolicy": "Always"
      }
    ],
    "restartPolicy": "Always",
    "terminationGracePeriodSeconds": 30,
    "dnsPolicy": "ClusterFirst",
    "nodeSelector": {
      "node-role.kubernetes.io/master": ""
    },
    "nodeName": "172.17.4.101",
    "hostNetwork": true,
    "securityContext": {},
    "affinity": {
      "nodeAffinity": {
        "requiredDuringSchedulingIgnoredDuringExecution": {
          "nodeSelectorTerms": [
            {
              "matchExpressions": null
            }
          ]
        }
      }
    },
    "schedulerName": "default-scheduler",
    "tolerations": [
      {
        "key": "node-role.kubernetes.io/master",
        "operator": "Exists",
        "effect": "NoSchedule"
      },
      {
        "key": "node.kubernetes.io/not-ready",
        "operator": "Exists",
        "effect": "NoExecute"
      },
      {
        "key": "node.kubernetes.io/unreachable",
        "operator": "Exists",
        "effect": "NoExecute"
      },
      {
        "key": "node.kubernetes.io/disk-pressure",
        "operator": "Exists",
        "effect": "NoSchedule"
      },
      {
        "key": "node.kubernetes.io/memory-pressure",
        "operator": "Exists",
        "effect": "NoSchedule"
      },
      {
        "key": "node.kubernetes.io/unschedulable",
        "operator": "Exists",
        "effect": "NoSchedule"
      },
      {
        "key": "node.kubernetes.io/network-unavailable",
        "operator": "Exists",
        "effect": "NoSchedule"
      }
    ],
    "priority": 0
  },
  "status": {}
}

@rphillips
Copy link
Member

Note: We do not create the checkpointer pod with an affinity, so I'm not sure where it is being added in.

@Huang-Wei
Copy link
Member Author

We do not create the checkpointer pod with an affinity, so I'm not sure where it is being added in.

Agree, this is the key point.

rphillips added a commit to rphillips/bootkube that referenced this pull request Oct 12, 2018
Kubernetes 1.12.x introduced new logic for Affinity [1]. In addition to
new logic, the Pod contains a default affinity. The new default affinity
gets serialized into the checkpoint file, and the 1.12.x kubelet does
not restore the pod due to the affinity.

This PR removes the affinity from the spec and documents that affinity's
are not supported.

```
"affinity": {
      "nodeAffinity": {
        "requiredDuringSchedulingIgnoredDuringExecution": {
          "nodeSelectorTerms": [
            {
              "matchExpressions": null
            }
          ]
        }
      }
    },

```

[1] kubernetes/kubernetes#68173
[2] https://github.com/kubernetes/kubernetes/blob/e39b510726113581c6f6a9c2db1753d794aa9cce/pkg/controller/daemon/util/daemonset_util.go#L183-L196
dghubble added a commit to poseidon/terraform-render-bootstrap that referenced this pull request Oct 14, 2018
* Mount an empty dir for the controller-manager to work around
kubernetes/kubernetes#68973
* Use a patched pod-checkpointer that strips affinity from
checkpointed pod manifests. Kubernetes v1.12.0-rc.1 introduced
a default affinity that appears on checkpointed manifests; but
it prevented scheduling and checkpointed pods should not have an
affinity, they're run directly by the Kubelet on the local node
* kubernetes-retired/bootkube#1001
* kubernetes/kubernetes#68173
dghubble added a commit to poseidon/typhoon that referenced this pull request Oct 14, 2018
* Mount an empty dir for the controller-manager to work around
kubernetes/kubernetes#68973
* Use a patched pod-checkpointer that strips affinity from
checkpointed pod manifests. Kubernetes v1.12.0-rc.1 introduced
a default affinity that appears on checkpointed manifests; but
it prevented scheduling and checkpointed pods should not have an
affinity, they're run directly by the Kubelet on the local node
* kubernetes-retired/bootkube#1001
* kubernetes/kubernetes#68173
dghubble added a commit to poseidon/typhoon that referenced this pull request Oct 14, 2018
* Mount an empty dir for the controller-manager to work around
kubernetes/kubernetes#68973
* Use a patched pod-checkpointer that strips affinity from
checkpointed pod manifests. Kubernetes v1.12.0-rc.1 introduced
a default affinity that appears on checkpointed manifests; but
it prevented scheduling and checkpointed pods should not have an
affinity, they're run directly by the Kubelet on the local node
* kubernetes-retired/bootkube#1001
* kubernetes/kubernetes#68173
rphillips added a commit to kubernetes-retired/bootkube that referenced this pull request Oct 16, 2018
* checkpointer: ignore Affinity within podspec

Kubernetes 1.12.x introduced new logic for Affinity [1]. In addition to
new logic, the Pod contains a default affinity. The new default affinity
gets serialized into the checkpoint file, and the 1.12.x kubelet does
not restore the pod due to the affinity.

This PR removes the affinity from the spec and documents that affinity's
are not supported.

```
"affinity": {
      "nodeAffinity": {
        "requiredDuringSchedulingIgnoredDuringExecution": {
          "nodeSelectorTerms": [
            {
              "matchExpressions": null
            }
          ]
        }
      }
    },

```

[1] kubernetes/kubernetes#68173
[2] https://github.com/kubernetes/kubernetes/blob/e39b510726113581c6f6a9c2db1753d794aa9cce/pkg/controller/daemon/util/daemonset_util.go#L183-L196

* gofmt

* golang: bump to 1.11.1

* fixes checkpointer run

* checkpointer test will have a modified asset file
dghubble added a commit to poseidon/terraform-render-bootstrap that referenced this pull request Oct 17, 2018
* Mount an empty dir for the controller-manager to work around
kubernetes/kubernetes#68973
* Update coreos/pod-checkpointer to strips affinity from
checkpointed pod manifests. Kubernetes v1.12.0-rc.1 introduced
a default affinity that appears on checkpointed manifests; but
it prevented scheduling and checkpointed pods should not have an
affinity, they're run directly by the Kubelet on the local node
* kubernetes-retired/bootkube#1001
* kubernetes/kubernetes#68173
dghubble added a commit to poseidon/terraform-render-bootstrap that referenced this pull request Oct 17, 2018
* Mount an empty dir for the controller-manager to work around
kubernetes/kubernetes#68973
* Update coreos/pod-checkpointer to strip affinity from
checkpointed pod manifests. Kubernetes v1.12.0-rc.1 introduced
a default affinity that appears on checkpointed manifests; but
it prevented scheduling and checkpointed pods should not have an
affinity, they're run directly by the Kubelet on the local node
* kubernetes-retired/bootkube#1001
* kubernetes/kubernetes#68173
dghubble added a commit to poseidon/typhoon that referenced this pull request Oct 17, 2018
* Mount an empty dir for the controller-manager to work around
kubernetes/kubernetes#68973
* Update coreos/pod-checkpointer to strip affinity from
checkpointed pod manifests. Kubernetes v1.12.0-rc.1 introduced
a default affinity that appears on checkpointed manifests; but
it prevented scheduling and checkpointed pods should not have an
affinity, they're run directly by the Kubelet on the local node
* kubernetes-retired/bootkube#1001
* kubernetes/kubernetes#68173
dghubble-robot pushed a commit to poseidon/terraform-onprem-kubernetes that referenced this pull request Oct 17, 2018
* Mount an empty dir for the controller-manager to work around
kubernetes/kubernetes#68973
* Update coreos/pod-checkpointer to strip affinity from
checkpointed pod manifests. Kubernetes v1.12.0-rc.1 introduced
a default affinity that appears on checkpointed manifests; but
it prevented scheduling and checkpointed pods should not have an
affinity, they're run directly by the Kubelet on the local node
* kubernetes-retired/bootkube#1001
* kubernetes/kubernetes#68173
dghubble-robot pushed a commit to poseidon/terraform-digitalocean-kubernetes that referenced this pull request Oct 17, 2018
* Mount an empty dir for the controller-manager to work around
kubernetes/kubernetes#68973
* Update coreos/pod-checkpointer to strip affinity from
checkpointed pod manifests. Kubernetes v1.12.0-rc.1 introduced
a default affinity that appears on checkpointed manifests; but
it prevented scheduling and checkpointed pods should not have an
affinity, they're run directly by the Kubelet on the local node
* kubernetes-retired/bootkube#1001
* kubernetes/kubernetes#68173
dghubble-robot pushed a commit to poseidon/terraform-aws-kubernetes that referenced this pull request Oct 17, 2018
* Mount an empty dir for the controller-manager to work around
kubernetes/kubernetes#68973
* Update coreos/pod-checkpointer to strip affinity from
checkpointed pod manifests. Kubernetes v1.12.0-rc.1 introduced
a default affinity that appears on checkpointed manifests; but
it prevented scheduling and checkpointed pods should not have an
affinity, they're run directly by the Kubelet on the local node
* kubernetes-retired/bootkube#1001
* kubernetes/kubernetes#68173
dghubble-robot pushed a commit to poseidon/terraform-google-kubernetes that referenced this pull request Oct 17, 2018
* Mount an empty dir for the controller-manager to work around
kubernetes/kubernetes#68973
* Update coreos/pod-checkpointer to strip affinity from
checkpointed pod manifests. Kubernetes v1.12.0-rc.1 introduced
a default affinity that appears on checkpointed manifests; but
it prevented scheduling and checkpointed pods should not have an
affinity, they're run directly by the Kubelet on the local node
* kubernetes-retired/bootkube#1001
* kubernetes/kubernetes#68173
dghubble-robot pushed a commit to poseidon/terraform-azure-kubernetes that referenced this pull request May 25, 2020
* Mount an empty dir for the controller-manager to work around
kubernetes/kubernetes#68973
* Update coreos/pod-checkpointer to strip affinity from
checkpointed pod manifests. Kubernetes v1.12.0-rc.1 introduced
a default affinity that appears on checkpointed manifests; but
it prevented scheduling and checkpointed pods should not have an
affinity, they're run directly by the Kubelet on the local node
* kubernetes-retired/bootkube#1001
* kubernetes/kubernetes#68173
Snaipe pushed a commit to aristanetworks/monsoon that referenced this pull request Apr 13, 2023
* Mount an empty dir for the controller-manager to work around
kubernetes/kubernetes#68973
* Update coreos/pod-checkpointer to strip affinity from
checkpointed pod manifests. Kubernetes v1.12.0-rc.1 introduced
a default affinity that appears on checkpointed manifests; but
it prevented scheduling and checkpointed pods should not have an
affinity, they're run directly by the Kubelet on the local node
* kubernetes-retired/bootkube#1001
* kubernetes/kubernetes#68173
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/scheduling Categorizes an issue or PR as relevant to SIG Scheduling. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

StatefulSet pod anti-affinity update not working
8 participants