From e0a98670a3c1687075072c048fdc18a14bbd523d Mon Sep 17 00:00:00 2001 From: LandoVMW <45016198+LandoVMW@users.noreply.github.com> Date: Fri, 20 Dec 2024 07:38:05 -0800 Subject: [PATCH] Update tpsm-ag-on-vsphere-ref.md last edits Signed-off-by: LandoVMW --- .../tpsm-ag-on-vsphere-ref.md | 22 +++++++++---------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/src/reference-designs/tpsm-ag-on-vsphere-ref.md b/src/reference-designs/tpsm-ag-on-vsphere-ref.md index 77a71aef..44bd308a 100644 --- a/src/reference-designs/tpsm-ag-on-vsphere-ref.md +++ b/src/reference-designs/tpsm-ag-on-vsphere-ref.md @@ -221,7 +221,7 @@ As per the proposed architecture, all the Tanzu Components will be deployed on a | :---- | :---- | :---- | :---- | | NSX ALB Management Network | /27 | The NSX Advanced Load Balancer (NSX ALB) controller nodes at both sites will connect to this network, which is part of the management domain. Based on the VCF Management Domain topology outlined in this documentation, it is assumed that the Management Domain/Cluster is stretched across both sites, along with the associated networks. **Note**: If the management domain is not stretched and a dedicated management domain is used per site, two separate networks must be configured, one at each site. This document does not cover scenarios with a single standard (non-stretched) management domain in a multi-site environment, as this configuration is not recommended. | Stretched | | NSX ALB SE Management Network | /24 | The management interface of the Service Engines (SEs) will connect to this network, and its sizing must align with the Service Engine group configuration at each site. **Considerations: Number of Networks:** One network per availability zone is recommended. **Stretched Network Option:** This network can optionally be stretched across both zones, allowing the same network to be used for SE management at both sites. In such cases, the subnet must be sized to accommodate the total number of Service Engines across both sites. The available IP addresses should then be divided into two distinct blocks, with each block assigned to the NSX ALB instance in the respective availability zone. | Optional | -| TKG Control Plane/Application VIP Network | /24 | This network is used for hosting L4 Virtual Services for Control plane HA of all Kubernetes clusters (Supervisor and Workload) and to host Virtual services for the applications deployed on the Tanzu Kubernetes Clusters (TKC) Reserve sufficient IPs depending on the number of TKG clusters and applications planned to be deployed in the environment. **Considerations:** **IP Allocation:** Reserve an adequate pool of IP addresses based on the number of planned TKG clusters and applications at both sites. **Number of Networks:** One. **Network Configuration:** This network must be stretched across both sites. The available IP addresses should be divided into three distinct blocks: **Block 1:** Added to the DHCP range in the NSX ALB instance at the primary site. **Block 2:** Added to the DHCP range in the NSX ALB instance at the secondary site. **Block 3:** This block, containing a single IP address, is added to the DHCP range of the NSX ALB instance at the primary site where TP-SM is deployed. The IP address is used to expose the TP-SM instance. In the event of a TP-SM recovery at the secondary site, this IP block must be reassigned to the NSX ALB instance at the secondary site to maintain service continuity. | Stretched | +| TKG Control Plane/Application VIP Network | /24 | This network is used for hosting L4 Virtual Services for Control plane HA of all Kubernetes clusters (Supervisor and Workload) and to host Virtual services for the applications deployed on the Tanzu Kubernetes Clusters (TKC).
Reserve sufficient IPs depending on the number of TKG clusters and applications planned to be deployed in the environment.
**Considerations:**
**IP Allocation:** Reserve an adequate pool of IP addresses based on the number of planned TKG clusters and applications at both sites.
**Number of Networks:** One.
**Network Configuration:** This network must be stretched across both sites.
The available IP addresses should be divided into three distinct blocks:
**Block 1:** Added to the DHCP range in the NSX ALB instance at the primary site.
**Block 2:** Added to the DHCP range in the NSX ALB instance at the secondary site.
**Block 3:** This block, containing a single IP address, is added to the DHCP range of the NSX ALB instance at the primary site where TP-SM is deployed. The IP address is used to expose the TP-SM instance.
In the event of a TP-SM recovery at the secondary site, this IP block must be reassigned to the NSX ALB instance at the secondary site to maintain service continuity. | Stretched | | IaaS Control Plane Mgmt Network | /27 | Supervisor Cluster nodes will utilize this network, with each cluster requiring a reserved block of five consecutive IP addresses. Initially, two Supervisor Clusters (one at each site) will be deployed, necessitating two such blocks. **Considerations: Number of Networks:** One network per availability zone is recommended. **Stretched Network Option:** If the network is stretched, the same network can be used by both the supervisor cluster, in such case reserve two blocks of five consecutive IP addresses from the same network to accommodate both sites. | Optional | | Primary Workload Network | /24 | The second interface of the Supervisor nodes will be connected to this network, requiring 3 IP addresses per Supervisor cluster. Supervisor services such as Harbor, Contour, and Velero vSphere services will be deployed as vSphere Pods and will consume IPs from this range. **Considerations:** **Number of Networks**: One network per availability zone is recommended. **Stretched Network Option**: This network can optionally be stretched across both sites. In such cases, the same network can be used as the primary workload network at both sites. The network should be divided into two blocks, with each block allocated to the respective site. | Optional | | Secondary Workload Network | ​​/24 | The control plane and worker nodes of TKG workload clusters will connect to this network. As depicted in the diagram, this network is associated with a dedicated Supervisor namespace (Admin Namespace) utilized by the platform administrator to deploy TP-SM components on one of the TKCs. Optionally, this network can also be used to host TP Build and/or Run clusters. **Considerations:** **Number of Networks**: One network per availability zone is recommended. **Stretched Network Option**: This network can optionally be stretched across both sites. In such cases, the same network can be used as the secondary workload network at both sites. The network should be divided into two blocks, with each block allocated to the respective site. | Optional | @@ -378,15 +378,15 @@ The following are the minimum design recommendations for configuring NSX ALB Ser | ID | Design Recommendations | Justification | Implication | | :---- | :---- | :---- | :---- | -| | Utilize the default Service Engine (SE) group. | As of the writing of this document, the IaaS control plane (version 8.0.3) only supports the use of the default Service Engine (SE) group. | All applications and the HA endpoints of the TKC clusters will share the same SE group for their data path. | -| | NSX ALB Service Engine High Availability set to Active/Active | Isolate NSX ALB Controller traffic from other Tanzu Components | Some features like ‘Preserve Client IP’ require legacy Active-Standby mode for bidirectional traffic. | -| | Enable ALB Service Engine Self Elections | Enable SEs to elect a primary amongst themselves in the absence of connectivity to the NSX ALB controller | None | -| | Enable 'Dedicated dispatcher CPU' on Service Engine Groups that contain the Service Engine VMs of 4 or more vCPUs. Note: This setting should be enabled on SE Groups that are servicing applications that have high network requirements. | This will enable a dedicated core for packet processing enabling high packet pipeline on the Service Engine VMs | None | -| | Set 'Placement across the Service Engines' setting to ‘Distributed’. | This allows for maximum fault tolerance and even utilization of capacity. | Might require more SE VMs as compared to 'compact' placement mode. | -| | Set the SE size to a minimum 2vCPU and 4GB of Memory | This configuration should meet the most generic use case | For services that require higher throughput, these configuration needs to be looked into and modified accordingly For more details refer [Service Engine Sizing documentation](https://techdocs.broadcom.com/us/en/vmware-security-load-balancing/avi-load-balancer/avi-load-balancer/22-1/vmware-avi-load-balancer-configuration-guide/se-advanced-networking/sizing-service-engines.html) | -| | Reserve Memory and CPU for Service Engines | The Service Engines are a critical infrastructure component providing load-balancing services to mission-critical applications. Guarantees the CPU and Memory allocation for SE VM and avoids performance degradation in case of resource contention | Resources are locked even when the SEs are underutilised | -| | The minimum number of active Service Engines for the Virtual Service is set to 2 | Ensures that any Virtual Service is active on at least 2 Service Engines, which improves SLA in case of individual Service Engine failure | None | -| | NSX ALB Service engines placed in respective VI Workload domain/clusters | NSX ALB Service engines provide Load Balancing services for tenant workloads and applications. Placing NSX ALB SEs in the same workload domains as tenant applications ensures optimal performance by reducing latency and improving data locality | NSX ALB Service Engine (SE) components must be accounted for when sizing the workload clusters | +| NSXALB-SE-01 | Utilize the default Service Engine (SE) group. | As of the writing of this document, the IaaS control plane (version 8.0.3) only supports the use of the default Service Engine (SE) group. | All applications and the HA endpoints of the TKC clusters will share the same SE group for their data path. | +| NSXALB-SE-02 | NSX ALB Service Engine High Availability set to Active/Active | Isolate NSX ALB Controller traffic from other Tanzu Components | Some features like ‘Preserve Client IP’ require legacy Active-Standby mode for bidirectional traffic. | +| NSXALB-SE-03 | Enable ALB Service Engine Self Elections | Enable SEs to elect a primary amongst themselves in the absence of connectivity to the NSX ALB controller | None | +| NSXALB-SE-04 | Enable 'Dedicated dispatcher CPU' on Service Engine Groups that contain the Service Engine VMs of 4 or more vCPUs. Note: This setting should be enabled on SE Groups that are servicing applications that have high network requirements. | This will enable a dedicated core for packet processing enabling high packet pipeline on the Service Engine VMs | None | +| NSXALB-SE-05 | Set 'Placement across the Service Engines' setting to ‘Distributed’. | This allows for maximum fault tolerance and even utilization of capacity. | Might require more SE VMs as compared to 'compact' placement mode. | +| NSXALB-SE-06 | Set the SE size to a minimum 2vCPU and 4GB of Memory | This configuration should meet the most generic use case | For services that require higher throughput, these configuration needs to be looked into and modified accordingly For more details refer [Service Engine Sizing documentation](https://techdocs.broadcom.com/us/en/vmware-security-load-balancing/avi-load-balancer/avi-load-balancer/22-1/vmware-avi-load-balancer-configuration-guide/se-advanced-networking/sizing-service-engines.html) | +| NSXALB-SE-07 | Reserve Memory and CPU for Service Engines | The Service Engines are a critical infrastructure component providing load-balancing services to mission-critical applications. Guarantees the CPU and Memory allocation for SE VM and avoids performance degradation in case of resource contention | Resources are locked even when the SEs are underutilised | +| NSXALB-SE-08 | The minimum number of active Service Engines for the Virtual Service is set to 2 | Ensures that any Virtual Service is active on at least 2 Service Engines, which improves SLA in case of individual Service Engine failure | None | +| NSXALB-SE-09 | NSX ALB Service engines placed in respective VI Workload domain/clusters | NSX ALB Service engines provide Load Balancing services for tenant workloads and applications. Placing NSX ALB SEs in the same workload domains as tenant applications ensures optimal performance by reducing latency and improving data locality | NSX ALB Service Engine (SE) components must be accounted for when sizing the workload clusters | @@ -447,7 +447,7 @@ Below is a highly simplified component architecture of Tanzu Platform ![](./img/tpsm-ag-on-vsphere/tp-comp-arch.png) -* Not officially supported +   \* Not officially supported An optional but significant component is the Global Server Load Balancer (GSLB). The platform supports integration with NSX ALB GSLB. Alternatively, a custom DNS provider can be used, requiring manual configuration of GSLB and related DNS entries.