New post - AKS: Things you wish you knew! #4437
Open
+71
−0
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
adding pavneet
chzbrgr71
reviewed
Jul 25, 2024
| @@ -0,0 +1,35 @@ | |||
| --- | |||
| title: "2024-07-24-AKS:Things you wish you knew!" | |||
There was a problem hiding this comment.
You can remove the date from this title. This field represents how the post will be titled, do it should be something like "Things you wish you knew!"
The file name needs to have the date and follow an exact format. Such as: 2024-07-24-aks-things-you-wish-you-knew.md
|
@Azure/aks-pm issue needs labels |
1 similar comment
|
@Azure/aks-pm issue needs labels |
pauldotyu
reviewed
Aug 7, 2024
| @@ -0,0 +1,59 @@ | |||
| --- | |||
| title: "AKS:Things you wish you knew!" | |||
| description: "List of features and configuraitons that make your life easier in running your workloads on AKS in produciton environments" | |||
There was a problem hiding this comment.
Suggested change
| description: "List of features and configuraitons that make your life easier in running your workloads on AKS in produciton environments" | |
| description: "List of features and configurations that make your life easier in running your workloads on AKS in production environments" |
| ## Background | ||
|
|
||
| Networking, Observability and Upgrades are among the most common topics of discussions that we face with customers on AKS. Users have a number of options for these on AKS and they often wonder what are the best features to adopt for the same and what are AKS's recommended options. | ||
| Furthermore, we have seen a common patterns in majority of the support issues that customers create and the conversations that we have with customers, where they are either not using best practice configuraiton or features, in-turn making life harder for themselves. *So, we wanted to share some of those common mistakes that customers make and what are the right settings and features for customers to use to make running AKS in production at scale easier.* |
There was a problem hiding this comment.
Suggested change
| Furthermore, we have seen a common patterns in majority of the support issues that customers create and the conversations that we have with customers, where they are either not using best practice configuraiton or features, in-turn making life harder for themselves. *So, we wanted to share some of those common mistakes that customers make and what are the right settings and features for customers to use to make running AKS in production at scale easier.* | |
| Furthermore, we have seen a common patterns in majority of the support issues that customers create and the conversations that we have with customers, where they are either not using best practice configuration or features, in-turn making life harder for themselves. **So, we wanted to share some of those common mistakes that customers make and what are the right settings and features for customers to use to make running AKS in production at scale easier.** |
|
|
||
| Networking, Observability and Upgrades are among the most common topics of discussions that we face with customers on AKS. Users have a number of options for these on AKS and they often wonder what are the best features to adopt for the same and what are AKS's recommended options. | ||
| Furthermore, we have seen a common patterns in majority of the support issues that customers create and the conversations that we have with customers, where they are either not using best practice configuraiton or features, in-turn making life harder for themselves. *So, we wanted to share some of those common mistakes that customers make and what are the right settings and features for customers to use to make running AKS in production at scale easier.* | ||
| If you are new to AKS, you should skip this and first check out our new offering [AKS Automatic] - that lets you create and manage production-ready clusters with minimal effort and added confidence. |
There was a problem hiding this comment.
Linking to other posts within Jekyll can be a bit tricky, but this should work.
Suggested change
| If you are new to AKS, you should skip this and first check out our new offering [AKS Automatic] - that lets you create and manage production-ready clusters with minimal effort and added confidence. | |
| If you are new to AKS, you should skip this and first check out our new offering [AKS Automatic](/AKS{% post_url 2024-05-22-aks-automatic %}) - that lets you create and manage production-ready clusters with minimal effort and added confidence. |
| If you are new to AKS, you should skip this and first check out our new offering [AKS Automatic] - that lets you create and manage production-ready clusters with minimal effort and added confidence. | ||
|
|
||
| ## AKS Auto Upgrades | ||
| Kubernetes upgrades like any change event can be hard to manage, especially when you are running multiple clusters across your fleet or you are falling behind on the n-3 supported versions. It is important to maintain a secure Kuberntes environment by applying the latest security patches, while ensuring your applicaitons and workloads do not see any disruption that can have an impact on the business. |
There was a problem hiding this comment.
Suggested change
| Kubernetes upgrades like any change event can be hard to manage, especially when you are running multiple clusters across your fleet or you are falling behind on the n-3 supported versions. It is important to maintain a secure Kuberntes environment by applying the latest security patches, while ensuring your applicaitons and workloads do not see any disruption that can have an impact on the business. | |
| Kubernetes upgrades like any change event can be hard to manage, especially when you are running multiple clusters across your fleet or you are falling behind on the n-3 supported versions. It is important to maintain a secure Kubernetes environment by applying the latest security patches, while ensuring your applications and workloads do not see any disruption that can have an impact on the business. |
Comment on lines
+19
to
+28
| AKS Cluster auto-upgrade provides a "set once and forget" mechanism that yields tangible time and operational cost benefits. Azure Kubernetes Service (AKS) auto-upgrade feature addresses the critical challenge of maintaining up-to-date Kubernetes clusters without manual intervention. By automating the upgrade process, it ensures that clusters receive the latest security patches, bug fixes, and new features, significantly reducing the risk of vulnerabilities and operational issues associated with outdated software. | ||
| It provides you with the flexibility to schedule your upgrades on a specific, repeatable cadnece to ensure critical business is not effected, and allows you to pick from 4 upgrade channels "none". "patch", "stable" and "rapid" so that you can controls the timing of the upgrade. To learn more visit [here](https://learn.microsoft.com/en-us/azure/aks/auto-upgrade-cluster?tabs=azure-cli) | ||
|
|
||
| Enable Auto-upgrade on your new or existing clusters today: | ||
|
|
||
|  | ||
|
|
||
| ## System pool VM SKU and Managed Disk Performance Tiers | ||
| AKS clusters have two types of nodepool. System pool, which is meant to run the system processes, addons and critical compoenets such as coredns, metrics-server, Konnectivity agents etc and the User pool, which are meant to run the user's worklaod applications. | ||
| One of the most common mistakes that we see users make , is that since they are not running thier user workload on the system pool, they will try to save cost by chosing the smallest possible VM SKUs for system pools, often even 2 core VMS. While that might work at very small scale, that is not the recommended guidance and users are essentially are settting themselves up for failure and pain later, impacting business and ultimately revenue. |
There was a problem hiding this comment.
Suggested change
| AKS Cluster auto-upgrade provides a "set once and forget" mechanism that yields tangible time and operational cost benefits. Azure Kubernetes Service (AKS) auto-upgrade feature addresses the critical challenge of maintaining up-to-date Kubernetes clusters without manual intervention. By automating the upgrade process, it ensures that clusters receive the latest security patches, bug fixes, and new features, significantly reducing the risk of vulnerabilities and operational issues associated with outdated software. | |
| It provides you with the flexibility to schedule your upgrades on a specific, repeatable cadnece to ensure critical business is not effected, and allows you to pick from 4 upgrade channels "none". "patch", "stable" and "rapid" so that you can controls the timing of the upgrade. To learn more visit [here](https://learn.microsoft.com/en-us/azure/aks/auto-upgrade-cluster?tabs=azure-cli) | |
| Enable Auto-upgrade on your new or existing clusters today: | |
|  | |
| ## System pool VM SKU and Managed Disk Performance Tiers | |
| AKS clusters have two types of nodepool. System pool, which is meant to run the system processes, addons and critical compoenets such as coredns, metrics-server, Konnectivity agents etc and the User pool, which are meant to run the user's worklaod applications. | |
| One of the most common mistakes that we see users make , is that since they are not running thier user workload on the system pool, they will try to save cost by chosing the smallest possible VM SKUs for system pools, often even 2 core VMS. While that might work at very small scale, that is not the recommended guidance and users are essentially are settting themselves up for failure and pain later, impacting business and ultimately revenue. | |
| AKS Cluster auto-upgrade provides a "set once and forget" mechanism that yields tangible time and operational cost benefits. Azure Kubernetes Service (AKS) auto-upgrade feature addresses the critical challenge of maintaining up-to-date Kubernetes clusters without manual intervention. By automating the upgrade process, it ensures that clusters receive the latest security patches, bug fixes, and new features, significantly reducing the risk of vulnerabilities and operational issues associated with outdated software. | |
| It provides you with the flexibility to schedule your upgrades on a specific, repeatable cadence to ensure critical business is not affected, and allows you to pick from 4 upgrade channels "none". "patch", "stable" and "rapid" so that you can controls the timing of the upgrade. To learn more visit [here](https://learn.microsoft.com/azure/aks/auto-upgrade-cluster?tabs=azure-cli) | |
| Enable Auto-upgrade on your new or existing clusters today: | |
|  | |
| ## System pool VM SKU and Managed Disk Performance Tiers | |
| AKS clusters have two types of nodepool. System pool, which is meant to run the system processes, addons and critical components such as coredns, metrics-server, Konnectivity agents etc. and the User pool, which are meant to run the user's workload applications. | |
| One of the most common mistakes that we see users make, is that since they are not running their user workload on the system pool, they will try to save cost by choosing the smallest possible VM SKUs for system pools, often even 2 core VMS. While that might work at very small scale, that is not the recommended guidance and users are essentially are setting themselves up for failure and pain later, impacting business and ultimately revenue. |
| ## System pool VM SKU and Managed Disk Performance Tiers | ||
| AKS clusters have two types of nodepool. System pool, which is meant to run the system processes, addons and critical compoenets such as coredns, metrics-server, Konnectivity agents etc and the User pool, which are meant to run the user's worklaod applications. | ||
| One of the most common mistakes that we see users make , is that since they are not running thier user workload on the system pool, they will try to save cost by chosing the smallest possible VM SKUs for system pools, often even 2 core VMS. While that might work at very small scale, that is not the recommended guidance and users are essentially are settting themselves up for failure and pain later, impacting business and ultimately revenue. | ||
| By not providing sufficeint resoruces for your system component pods, users can run into issues such as dns resulition issues(Coredns), loss of metrics and alerting(metrics-server), control plane response latency or failures (Konnectivity) due to resource starvation for these critical components or even node failure, forcing them to spend time diagnosing and mitigating easily avoidable issues. |
There was a problem hiding this comment.
Suggested change
| By not providing sufficeint resoruces for your system component pods, users can run into issues such as dns resulition issues(Coredns), loss of metrics and alerting(metrics-server), control plane response latency or failures (Konnectivity) due to resource starvation for these critical components or even node failure, forcing them to spend time diagnosing and mitigating easily avoidable issues. | |
| By not providing sufficient resources for your system component pods, users can run into issues such as DNS resolution issues (coredns), loss of metrics and alerting (metrics-server), control plane response latency or failures (Konnectivity) due to resource starvation for these critical components or even node failure, forcing them to spend time diagnosing and mitigating easily avoidable issues. |
Comment on lines
+30
to
+42
| For these reasons, we recommend using a minimum of 4 core VMs in general with 3 nodes for system pool, and at larger scale (think +1,000 nodes and +10,0000) use 16 core VMs. | ||
|
|
||
| ###Ideal recommendaiton for system pool: | ||
| VM SKU: Standard_D8ds_v5 | ||
| OS disk type: Ephemeral | ||
| No. of VMs : >3 | ||
|
|
||
| When using VM SKUs that do not support ephemeral OS disks, be sure to set a high performance tier for your managed disks to ensure sufficient I/O throughput for your critical addons and processes. | ||
|
|
||
| ## AKS Diagnostic Settings and recommended alerts | ||
| Troubleshooting issues in Kubernetes can be hard, Users often struggle with pinpointing performance issues, security threats, and operational glitches within their clusters. Part of the problem is ensuring that the right set of metrics, logs , events are captured and stored in a easy to query mannger. | ||
| AKS Diagnostics setting centralizes the enablement and data collection experience into a single view, leveraging built in data collection, storage and visualization tools in Azure Monitor and Log Analytics. | ||
| With a few clicks, you can select which logs are collected such as k8s audit logs, control plane logs and audit admin logs; you can select the destination in which to store that data - from log analytics workspace, cold storage to third-part montiroing solutions and you can also configure the granularity of the logs to resoruce level logs. |
There was a problem hiding this comment.
Suggested change
| For these reasons, we recommend using a minimum of 4 core VMs in general with 3 nodes for system pool, and at larger scale (think +1,000 nodes and +10,0000) use 16 core VMs. | |
| ###Ideal recommendaiton for system pool: | |
| VM SKU: Standard_D8ds_v5 | |
| OS disk type: Ephemeral | |
| No. of VMs : >3 | |
| When using VM SKUs that do not support ephemeral OS disks, be sure to set a high performance tier for your managed disks to ensure sufficient I/O throughput for your critical addons and processes. | |
| ## AKS Diagnostic Settings and recommended alerts | |
| Troubleshooting issues in Kubernetes can be hard, Users often struggle with pinpointing performance issues, security threats, and operational glitches within their clusters. Part of the problem is ensuring that the right set of metrics, logs , events are captured and stored in a easy to query mannger. | |
| AKS Diagnostics setting centralizes the enablement and data collection experience into a single view, leveraging built in data collection, storage and visualization tools in Azure Monitor and Log Analytics. | |
| With a few clicks, you can select which logs are collected such as k8s audit logs, control plane logs and audit admin logs; you can select the destination in which to store that data - from log analytics workspace, cold storage to third-part montiroing solutions and you can also configure the granularity of the logs to resoruce level logs. | |
| For these reasons, we recommend using a minimum of 4 core VMs in general with 3 nodes for system pool, and at larger scale (think +1,000 nodes and +10,0000) use 16 core VMs. | |
| ### Ideal recommendation for system pool: | |
| VM SKU: `Standard_D8ds_v5` | |
| OS disk type: `Ephemeral` | |
| No. of VMs : `>3` | |
| When using VM SKUs that do not support ephemeral OS disks, be sure to set a high-performance tier for your managed disks to ensure sufficient I/O throughput for your critical addons and processes. | |
| ## AKS Diagnostic Settings and recommended alerts | |
| Troubleshooting issues in Kubernetes can be hard, Users often struggle with pinpointing performance issues, security threats, and operational glitches within their clusters. Part of the problem is ensuring that the right set of metrics, logs, events are captured and stored in an "easy to query" manner. | |
| AKS Diagnostics setting centralizes the enablement and data collection experience into a single view, leveraging built in data collection, storage and visualization tools in Azure Monitor and Log Analytics. | |
| With a few clicks, you can select which logs are collected such as k8s audit logs, control plane logs and audit admin logs; you can select the destination in which to store that data - from log analytics workspace, cold storage to third-part monitoring solutions and you can also configure the granularity of the logs to resource level logs. |
| ## Summary | ||
|
|
||
|
|
||
| [AKS Automatic](blog/_posts/2024-05-22-aks-automatic.md) |
There was a problem hiding this comment.
Move link to be inline instead.
Suggested change
| [AKS Automatic](blog/_posts/2024-05-22-aks-automatic.md) |
|
@Azure/aks-pm issue needs labels |
7 similar comments
|
@Azure/aks-pm issue needs labels |
|
@Azure/aks-pm issue needs labels |
|
@Azure/aks-pm issue needs labels |
|
@Azure/aks-pm issue needs labels |
|
@Azure/aks-pm issue needs labels |
|
@Azure/aks-pm issue needs labels |
|
@Azure/aks-pm issue needs labels |
|
@Azure/aks-pm issue needs labels |
5 similar comments
|
@Azure/aks-pm issue needs labels |
|
@Azure/aks-pm issue needs labels |
|
@Azure/aks-pm issue needs labels |
|
@Azure/aks-pm issue needs labels |
|
@Azure/aks-pm issue needs labels |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Creating a new blog post with 5 features that customers should use but dont use them often enough and how they help