controlPlane InstanceGroup must have maxSize set to 1, add more InstanceGroups instead

[arpanadhikari]

I'm trying to create a kOps cluster on single AZ in AWS (to avoid inter AZ data transfer costs). Single AZ is enough to meet our availability requirements.

However, I still need more than one master node so I scaled up my instanceGroup to have minSize:3 and maxSize:3 on the existing master instanceGroup spec. However, kOps appears to have a built in limitation on using only one instance per instanceGroup for master nodes.

kops/pkg/apis/kops/validation/instancegroup.go

Lines 52 to 56 in 61e7ac2

	if fi.ValueOf(g.Spec.MinSize) > 1 {
	allErrs = append(allErrs, field.Invalid(field.NewPath("spec", "minSize"), fi.ValueOf(g.Spec.MinSize), "controlPlane InstanceGroup must have minSize set to 1"))
	}
	if fi.ValueOf(g.Spec.MaxSize) > 1 {
	allErrs = append(allErrs, field.Invalid(field.NewPath("spec", "maxSize"), fi.ValueOf(g.Spec.MaxSize), "controlPlane InstanceGroup must have maxSize set to 1, add more InstanceGroups instead"))

I could create multiple instanceGroups on the same AZ but that would mean duplicating the same config and also it just doesn't make sense.

Any advise/workarounds welcome. Happy to make a pull request to fix it if required.

My configuration:

# master instance group snippet
apiVersion: kops.k8s.io/v1alpha2
kind: InstanceGroup
metadata:
  labels:
    kops.k8s.io/cluster: {{.kops.clusterName}}.{{.kops.dnsZone}}
  name: masters
spec:
  image: {{ ChannelRecommendedImage .kops.cloudProvider .kops.kubernetesVersion .kops.architecture }}
  kubernetesVersion: {{.kops.kubernetesVersion}}
  machineType: {{ .kops.masters.machineType }}
  {{ if .kops.masters.spot }}
  maxPrice: {{ .kops.masters.maxPrice | quote }}
  {{ end }}
  maxSize: 3
  minSize: 3
  role: Master
  rootVolumeSize: 80
  subnets:
  - {{.kops.awsRegion}}a
---

# etcd config snippet
  etcdClusters:
  - cpuRequest: 200m
    etcdMembers:
    - encryptedVolume: true
      instanceGroup: masters
      name: a
    - encryptedVolume: true
      instanceGroup: masters
      name: b
    - encryptedVolume: true
      instanceGroup: masters
      name: c

Also, Is this a bug?

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale