Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion api/v1alpha1/aws_node_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,13 @@ type AWSControlPlaneNodeSpec struct {
}

type AWSWorkerNodeSpec struct {
// The IAM instance profile to use for the cluster Machines.
// The failureDomain the machine deployment will use.
// +kubebuilder:validation:Optional
// +kubebuilder:validation:MinLength=1
// +kubebuilder:validation:MaxLength=128
FailureDomain string `json:"failureDomain,omitempty"`

// The IAM instance profile to use for the cluster Machines.
// +kubebuilder:validation:Optional
// +kubebuilder:default=nodes.cluster-api-provider-aws.sigs.k8s.io
// +kubebuilder:validation:MinLength=1
Expand Down
5 changes: 5 additions & 0 deletions api/v1alpha1/crds/caren.nutanix.com_awsworkernodeconfigs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,11 @@ spec:
type: string
type: object
type: object
failureDomain:
description: The failureDomain the machine deployment will use.
maxLength: 128
minLength: 1
type: string
iamInstanceProfile:
default: nodes.cluster-api-provider-aws.sigs.k8s.io
description: The IAM instance profile to use for the cluster Machines.
Expand Down
5 changes: 5 additions & 0 deletions api/v1alpha1/crds/caren.nutanix.com_eksworkernodeconfigs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,11 @@ spec:
type: string
type: object
type: object
failureDomain:
description: The failureDomain the machine deployment will use.
maxLength: 128
minLength: 1
type: string
iamInstanceProfile:
default: nodes.cluster-api-provider-aws.sigs.k8s.io
description: The IAM instance profile to use for the cluster Machines.
Expand Down
30 changes: 30 additions & 0 deletions common/pkg/testutils/capitest/request/items.go
Original file line number Diff line number Diff line change
Expand Up @@ -245,3 +245,33 @@ func NewWorkerDockerMachineTemplateRequestItem(
uid,
)
}

func NewWorkerMachineDeploymentRequestItem(
uid types.UID,
) runtimehooksv1.GeneratePatchesRequestItem {
return NewRequestItem(
&clusterv1.MachineDeployment{
TypeMeta: metav1.TypeMeta{
APIVersion: clusterv1.GroupVersion.String(),
Kind: "MachineDeployment",
},
ObjectMeta: metav1.ObjectMeta{
Name: "test-machinedeployment",
Namespace: Namespace,
},
Spec: clusterv1.MachineDeploymentSpec{
Template: clusterv1.MachineTemplateSpec{
Spec: clusterv1.MachineSpec{
ClusterName: ClusterName,
},
},
},
},
&runtimehooksv1.HolderReference{
APIVersion: clusterv1.GroupVersion.String(),
Kind: "MachineDeployment",
FieldPath: "",
},
uid,
)
}
73 changes: 73 additions & 0 deletions docs/content/customization/aws/failure-domain.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
+++
title = "AWS Failure Domain"
+++

The AWS failure domain customization allows the user to specify the AWS availability zone (failure domain) for worker node deployments.
This customization can be applied to individual MachineDeployments to distribute worker nodes across different availability zones for high availability.
This customization will be available when the
[provider-specific cluster configuration patch]({{< ref "..">}}) is included in the `ClusterClass`.

## Example

To specify a failure domain for worker nodes, use the following configuration:

```yaml
apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
name: <NAME>
spec:
topology:
variables:
- name: workerConfig
value:
aws:
failureDomain: us-west-2a
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Were you able to verify this manually for EKS? From the docs it uses failureDomain: "1"
https://cluster-api-aws.sigs.k8s.io/topics/failure-domains/worker-nodes#failure-domains-in-worker-nodes

But in the CAPA code it does look like it should be the AZ like you have it documented

```

You can customize individual MachineDeployments by using the overrides field to deploy workers across multiple availability zones:

```yaml
spec:
topology:
# ...
workers:
machineDeployments:
- class: default-worker
name: md-0
variables:
overrides:
- name: workerConfig
value:
aws:
failureDomain: us-west-2a
- class: default-worker
name: md-1
variables:
overrides:
- name: workerConfig
value:
aws:
failureDomain: us-west-2b
- class: default-worker
name: md-2
variables:
overrides:
- name: workerConfig
value:
aws:
failureDomain: us-west-2c
```

## Resulting CAPA Configuration

Applying this configuration will result in the following value being set:

- worker `MachineDeployment`:

- ```yaml
spec:
template:
spec:
failureDomain: us-west-2a
```
73 changes: 73 additions & 0 deletions docs/content/customization/eks/failure-domain.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
+++
title = "AWS Failure Domain"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
title = "AWS Failure Domain"
title = "EKS Failure Domain"

+++

The AWS failure domain customization allows the user to specify the AWS availability zone (failure domain) for worker node deployments.
This customization can be applied to individual MachineDeployments to distribute worker nodes across different availability zones for high availability.
This customization will be available when the
[provider-specific cluster configuration patch]({{< ref "..">}}) is included in the `ClusterClass`.

## Example

To specify a failure domain for worker nodes, use the following configuration:

```yaml
apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
name: <NAME>
spec:
topology:
variables:
- name: workerConfig
value:
aws:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
aws:
eks:

failureDomain: us-west-2a
```

You can customize individual MachineDeployments by using the overrides field to deploy workers across multiple availability zones:

```yaml
spec:
topology:
# ...
workers:
machineDeployments:
- class: default-worker
name: md-0
variables:
overrides:
- name: workerConfig
value:
aws:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
aws:
eks:

failureDomain: us-west-2a
- class: default-worker
name: md-1
variables:
overrides:
- name: workerConfig
value:
aws:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
aws:
eks:

failureDomain: us-west-2b
- class: default-worker
name: md-2
variables:
overrides:
- name: workerConfig
value:
aws:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
aws:
eks:

failureDomain: us-west-2c
```

## Resulting CAPA Configuration

Applying this configuration will result in the following value being set:

- worker `MachineDeployment`:

- ```yaml
spec:
template:
spec:
failureDomain: us-west-2a
```
16 changes: 16 additions & 0 deletions pkg/handlers/aws/mutation/failuredomain/inject_suite_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
// Copyright 2023 Nutanix. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

package failuredomain

import (
"testing"

. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)

func TestFailureDomainPatch(t *testing.T) {
RegisterFailHandler(Fail)
RunSpecs(t, "AWS failure domain mutator suite")
}
103 changes: 103 additions & 0 deletions pkg/handlers/aws/mutation/failuredomain/inject_worker.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
// Copyright 2023 Nutanix. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

package failuredomain

import (
"context"

apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
clusterv1 "sigs.k8s.io/cluster-api/api/v1beta1"
runtimehooksv1 "sigs.k8s.io/cluster-api/exp/runtime/hooks/api/v1alpha1"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client"

"github.com/nutanix-cloud-native/cluster-api-runtime-extensions-nutanix/api/v1alpha1"
"github.com/nutanix-cloud-native/cluster-api-runtime-extensions-nutanix/common/pkg/capi/clustertopology/handlers/mutation"
"github.com/nutanix-cloud-native/cluster-api-runtime-extensions-nutanix/common/pkg/capi/clustertopology/variables"
)

const (
// VariableName is the external patch variable name.
VariableName = "failureDomain"
)

type awsFailureDomainWorkerPatchHandler struct {
variableName string
variableFieldPath []string
}

func NewWorkerPatch() *awsFailureDomainWorkerPatchHandler {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add failure domain patch for AWS control plane too.

return NewAWSFailureDomainWorkerPatchHandler(
v1alpha1.WorkerConfigVariableName,
v1alpha1.AWSVariableName,
VariableName,
)
}

func NewAWSFailureDomainWorkerPatchHandler(
variableName string,
variableFieldPath ...string,
) *awsFailureDomainWorkerPatchHandler {
return &awsFailureDomainWorkerPatchHandler{
variableName: variableName,
variableFieldPath: variableFieldPath,
}
}

func (h *awsFailureDomainWorkerPatchHandler) Mutate(
ctx context.Context,
obj *unstructured.Unstructured,
vars map[string]apiextensionsv1.JSON,
holderRef runtimehooksv1.HolderReference,
_ client.ObjectKey,
_ mutation.ClusterGetter,
) error {
log := ctrl.LoggerFrom(ctx).WithValues(
"holderRef", holderRef,
)

failureDomainVar, err := variables.Get[string](
vars,
h.variableName,
h.variableFieldPath...,
)
if err != nil {
if variables.IsNotFoundError(err) {
log.V(5).Info("AWS failure domain variable for worker not defined")
return nil
}
return err
}

log = log.WithValues(
"variableName",
h.variableName,
"variableFieldPath",
h.variableFieldPath,
"variableValue",
failureDomainVar,
)

// Check if this is a MachineDeployment
if obj.GetKind() != "MachineDeployment" || obj.GetAPIVersion() != clusterv1.GroupVersion.String() {
log.V(5).Info("not a MachineDeployment, skipping")
return nil
}

log.WithValues(
"patchedObjectKind", obj.GetKind(),
"patchedObjectName", client.ObjectKeyFromObject(obj),
).Info("setting failure domain in worker MachineDeployment spec")

if err := unstructured.SetNestedField(
obj.Object,
failureDomainVar,
"spec", "template", "spec", "failureDomain",
); err != nil {
return err
Comment on lines +84 to +99
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should be using patches.MutateIfApplicable instead for consistency between all the handlers and use concrete types instead of unstructured

}

return nil
}
Loading
Loading